Compare commits
@@ -14,3 +14,16 @@
|
||||
# 3. Restart ComfyUI
|
||||
|
||||
UNSLOTH_API_KEY=
|
||||
|
||||
# ComfyUI-Workflow-Studio - Gallery Semantic Search (Gemini Embedding 2)
|
||||
# Get a key at https://aistudio.google.com (free tier available).
|
||||
# Same steps as above: copy to ".env", paste the key, restart ComfyUI.
|
||||
GEMINI_API_KEY=
|
||||
|
||||
# ComfyUI-Workflow-Studio - Partner API nodes in GenerateUI (Nano Banana, Kling, Seedream, ...)
|
||||
# A Comfy API key issued at https://platform.comfy.org (separate from the Gemini key above;
|
||||
# usage is billed as Comfy credits). With it set, GenerateUI queues prompts through the plugin
|
||||
# server, which adds the key to the request only when the workflow contains a partner API node.
|
||||
# Same steps as above: copy to ".env", paste the key, restart ComfyUI.
|
||||
|
||||
COMFY_API_KEY=
|
||||
|
||||
@@ -2,6 +2,33 @@
|
||||
|
||||
Condensed version-by-version index of DEVLOG.md (each line: version + one-line summary of what changed and why). Full rationale, code details, and "How to apply" lessons live in the excluded DEVLOG.md itself; this index exists so graphify can extract a queryable semantic node per release without feeding the full log to the local Ollama model. Regenerate with `python tools/generate_doc_index.py devlog` whenever DEVLOG.md changes.
|
||||
|
||||
- **v0.9.9（2026-11-15）** — プロンプト翻訳（自国語 → 英語、PLaMo 3 Translate）
|
||||
- **v0.9.8（2026-11-14）** — セキュリティ: クライアントから受け取るパスを解決前に検証
|
||||
- **v0.9.6（2026-11-12）** — マニュアル v0.9.6版（ja/en）
|
||||
- **v0.9.5（2026-11-11）** — AI TOOL Chat: スキル v2（metadata・同梱スキルの版管理・プロンプトボタン）
|
||||
- **v0.9.4（2026-11-07）** — Gallery: 3Dモデル対応（GLB / glTF / VRM / PLY / SPZ）
|
||||
- **v0.9.3（2026-11-05）** — Image Edit: Drawツールのカラーパレット・ショートカット・スポイト（Comic Creator v1.67.0 から移植）
|
||||
- **v0.9.2（2026-11-03）** — Selectorイベントループ有効時に画像が壊れて届く不具合を修正
|
||||
- **v0.9.1（2026-11-03）** — TagSpace: 選んだタグだけを追加 / タグのカッコをエスケープして追加
|
||||
- **v0.9.0（2026-11-03）** — マニュアル（画像なし配布版）を v0.9.0 に改訂
|
||||
- **v0.8.14（2026-11-03）** — Fibo 2.0 Edit / FastH3 OmniRef / Pika-H3-Flex 対応（GenerateUI・Metadata・Videoタブ）
|
||||
- **v0.8.13（2026-10-27）** — Edit in Mask Editor One: 送信画像をノードへ永続化（Mask Editor One v0.1.12 対応）
|
||||
- **v0.8.12（2026-10-27）** — Comic Creator向け複数画像I2Iブリッジ + GenerateUI Inputタブ: Prompt/Image統合・Raw JSON折りたたみ
|
||||
- **v0.8.11（2026-10-26）** — Gallery: Send CC をレイアウト／Imageタブの2ボタンに分割
|
||||
- **v0.8.10（2026-10-23）** — Tagger: PixAI Tagger v1.0 対応
|
||||
- **v0.8.9（2026-10-20）** — Video: アセット登録ボタン / Editの音声変換 / ソースパネルのレイアウト
|
||||
- **v0.8.8（2026-10-19）** — MOSS-Image 対応（GenerateUI / Gallery / Metadata / Iタブ）
|
||||
- **v0.8.7（2026-10-16）** — Qwen Image 2.1 Turbo（ComfyUI公式テンプレート T2I / I2I）対応
|
||||
- **v0.8.6（2026-10-14）** — GenerateUI: パートナーAPIノード（Nano Banana 等）の実行に対応
|
||||
- **v0.8.5（2026-10-14）** — Metadata / I タブ: Gemini 系ノード（Nano Banana）の画像を読めるように
|
||||
- **v0.8.4（2026-10-14）** — ComfyUI v0.40.0 対応: ネイティブ LoRA スタックノード（`LoadLoraModel` / `LoadLoraTextEncoder`）
|
||||
- **v0.8.3（2026-10-13）** — GenerateUI: シードモード「Workflow」を追加（改善候補メモ 2-A）
|
||||
- **v0.8.2（2026-10-12）** — Gallery: セマンティック検索（Gemini Embedding 2 / EmbeddingGemma 2）
|
||||
- **v0.8.1（2026-10-11）** — バグ修正
|
||||
- **v0.8.0（2026-10-11）** — Gallery: 音声ファイル対応（wav / mp3 / ogg / m4a / flac / opus）
|
||||
- **v0.7.14（2026-10-09）** — Comic Creator連携: nanobananaタブの生成履歴をGallery/Metadataタブに表示
|
||||
- **v0.7.13（2026-10-06）** — VRAM調整（Ollama）— Comic Creatorから移植
|
||||
- **v0.7.12（2026-10-04）** — セキュリティ: バックエンド接続先をブラウザから任意指定できた問題（SSRF）を修正
|
||||
- **v0.7.11（2026-10-02）** — セキュリティ: CivitAI情報の説明文のHTML無害化（保存型XSS対策）
|
||||
- **v0.7.10（2026-10-02）** — Windows: ComfyUIが固まる問題の原因特定と対策（Selectorイベントループ）
|
||||
- **v0.7.9（2026-10-02）** — Models: CivitAIベースモデルからの自動バッジ作成
|
||||
|
||||
@@ -2,6 +2,715 @@
|
||||
|
||||
---
|
||||
|
||||
## v0.9.9（2026-11-15）
|
||||
|
||||
### プロンプト翻訳（自国語 → 英語、PLaMo 3 Translate）
|
||||
|
||||
- 設定タブに「プロンプト翻訳」セクションを新設（既定オフ）。A1111 の prompt translator 拡張と同じく、英数字以外の文字を含む部分だけを翻訳し、カンマ・`(語句:1.2)`・`[ ]`・`{a|b}`・`<lora:...>`・`__wildcard__`・`embedding:`・`BREAK`/`AND` は保護する（`static/js/prompt-translate.js` の `splitPrompt`）。
|
||||
- 🌐ボタン（＋「原文」ボタン、Ctrl+Z も可）を GenerateUI の Positive/Negative、Lab プロンプトモーダル、Video Plan のブロックプロンプトに追加。
|
||||
- モデルは Ollama 経由。PLaMo は `/api/generate` に `raw:true` で PFML3 プロンプトを直接送る（モデルカード推奨、stop `<|plamo:tag|>`/`<|plamo:bos|>`）。PLaMo 以外の Ollama モデルは `/api/chat` ＋翻訳指示。PLaMo の4サイズはプリセットとして表示し、未取得なら「ダウンロード」（`/api/wfm/translate/pull`、プリセットのみ許可）。
|
||||
- 実行場所: GPU（1語句約0.1秒、翻訳後アンロード可）／CPU常駐（`num_gpu:0`＋`keep_alive:-1`、1語句約0.3秒、VRAM 0、GenerateUI表示時に `/warmup` で裏読み込み、デバイス・モデル切替や無効化時は `/unload`）。実測でコールド約4秒の大半は Ollama ランナー起動（CPUでも4.4秒）だったため、常駐でほぼ解消。
|
||||
- 「実行時に自動で翻訳する」: GenerateUI Generate（欄の文を置き換え）／Lab Run（各回のAPIワークフローのコピーを翻訳）／Video Plan Run（送信するプロンプトだけ翻訳）。Lab/Video はプランを原文のまま残す。失敗時は生成せず中断し、理由を進行状況欄に残す（終了トーストが上書きするため）。
|
||||
- サーバー側に翻訳結果の上限付きキャッシュ（1024件）。設定は `settings.json` の `prompt_translate_*`。
|
||||
- 実機確認: Z-Image で Generate 時自動翻訳と Lab Run（2枚）を実生成、Video Plan Run は `/prompt` 送信をブラウザで横取りして英訳の送信を確認。
|
||||
|
||||
### 修正: PLaMo を選んだときの各翻訳ボタン
|
||||
|
||||
- Prompt タブ AI アシスタント・AI TOOL タブ・サイドパネル A タブの翻訳は、指示文＋本文（または指示の system メッセージ）で送っていたため、PLaMo では指示文まで翻訳される／日本語⇔英語の自動切替になり ZH→EN・EN→ZH・自由入力が誤った言語で出ていた。PLaMo 系モデルのときは先頭 system に `[plamo-translate]` と `target_lang`/`source_lang` を書き本文だけを送る（モデルカードの Ollama/LM Studio 向け方式）。言語名（French・フランス語・français 等）を PLaMo の言語コードに変換する `static/js/plamo-chat.js` を追加（サイドパネルは import できないためインラインコピー）。
|
||||
- Prompt タブ AI アシスタントに応答待ちの表示を追加（初回はモデル読み込みで十数秒かかることがある）。
|
||||
|
||||
### 修正: Video Plan のプロンプト等がテンプレートの値のまま送られていた
|
||||
|
||||
- 新しいフロントエンドで保存したワークフローは、サブグラフのインスタンスノードに名前付きの値のコピー `widgets_values_named` を持ち、`convertUiToApi()` はそちらを優先する。`applyBlockToWorkflow` は `widgets_values` だけを書いていたため、Plan のプロンプト・Duration・Turbo 等が無視されていた（手元の動画テンプレート8本中7本）。書き込み後に `widgets_values_named` へ反映する `_syncNamedWidgetValues` を追加。修正後は12本中11本で反映（残る `minimaax_h3_fast.json` はプロンプトを LiveChatStream から配線で受けるため対象外）。
|
||||
|
||||
### ヘルプ
|
||||
|
||||
- 設定タブ（helpSettings26）と GenerateUI（helpGen33）にプロンプト翻訳を追加（en/ja/zh）。
|
||||
|
||||
### マニュアル v0.9.9版（ja/en）
|
||||
|
||||
- 新節 5-10「日本語のままプロンプトを書く（プロンプト翻訳）」（準備・モデルの大きさ・GPU/CPU常駐・🌐と原文・実行時の自動翻訳・Lab/Video Plan）。7-5・10-2 に PLaMo の注記、11-4 に Video Plan の🌐と v0.9.9 修正の注意、11-5 の表・12-3 のトラブル表・付録の用語集と参考リンクを追記。対象バージョンを v0.9.9 に更新し、全章MD・ZIPを `manual/rebuild_repo_edition.py` で再生成（v0.9.6版は削除）。
|
||||
- EPUB（画像付き、75枚＝新規3枚: 設定のプロンプト翻訳・生成UIで翻訳後・Labモーダル）を Google ドライブで公開。
|
||||
|
||||
## v0.9.8（2026-11-14）
|
||||
|
||||
### セキュリティ: クライアントから受け取るパスを解決前に検証
|
||||
|
||||
- `py/config.py` に `is_safe_rel` / `resolve_untrusted` を追加。文字列だけで正規化・配下チェックし、UNC・ドライブ相対・NUL 混入を最初の `resolve()`/`exists()` より前に拒否（Windows では UNC を解決した時点でアカウント資格情報付きで相手ホストへ接続するため）。UNC は許可ルート自体が UNC の場合のみ許可。G'MIC 結果・Gallery のパス受け取りに適用（ea44cf4）。
|
||||
- 追加の強化: 受け付ける形をドライブレター絶対パス（Windows）または許可された UNC に限定し、NT デバイス形式（`\??\...`、`\.\...`）等を拒否。Eagle の SVG パスで配下チェック前に `os.path.isfile` していた箇所を修正（f39227a）。
|
||||
- ※v0.9.8 はリリース時に README バッジと DEVLOG の更新が漏れていたため、v0.9.9 で追記。
|
||||
|
||||
## v0.9.6（2026-11-12）
|
||||
|
||||
### マニュアル v0.9.6版（ja/en）
|
||||
|
||||
- 5-4 Settingsタブを3列（スイッチ列の説明を追加）、5-9 対応表に MiniMax H3 Image と Compare Images 修正の注記、12-3 の Apply 注意にスイッチ列の例外を追記。対象バージョンを v0.9.6 に更新し、全章MD・ZIPを `manual/rebuild_repo_edition.py` で再生成（v0.9.5版は削除）。
|
||||
|
||||
### GenerateUI Settingsタブ: スイッチ列
|
||||
|
||||
- KSampler・Latent Imageの右に3列目「スイッチ」を追加。APIワークフロー内のリテラルな true/false 入力（ComfySwitchNode の switch、PrimitiveBoolean の value、TextGenerateV2 の thinking など）をノード名・入力名付きで一覧し、チェックボックスで即時切り替え（Apply不要、Raw JSONへ同期）。リンクされた入力は配列なので対象外で、配線元のPrimitiveBooleanが出る。
|
||||
- 選択肢がちょうど enable/disable・enabled/disabled・on/off・true/false・yes/no の2択のCOMBO（KSamplerAdvanced add_noise、DWPreprocessor detect_* など）も対象。object_info を型別取得してから追加描画し、値はそのノード自身の綴りの文字列で書き戻す。DynamicCombo（TextGenerate の sampling_mode）は子入力が変わるため対象外。
|
||||
- 保存済みワークフロー347本（App形式5本は読込対象外）をブラウザで順に読み込み、128本・426項目でオン→オフ→元に戻すの往復と Raw JSON 同期を自動検証、失敗・コンソールエラー0。
|
||||
- MiniMax H3 Image のT2I（サブグラフ内のプロンプト強化スイッチ）とEdit（PrimitiveBoolean「Enable PE?」）でブラウザ実操作を確認。
|
||||
|
||||
### MiniMax-H3-Image対応
|
||||
|
||||
- `MiniMaxH3ImageConditioning`（Comfy-Org の MiniMax H3 Image T2I/Edit テンプレート）をGenerateUI/Metadata/Iタブに追加。入力は `prompt`/`width`/`height`/`images.image_N` で `PH3ImageConditioning` と同形のため同じ扱い（negativeなし、promptはPreviewAny→プロンプト強化スイッチ経由で配線元のユーザープロンプトをpositiveに）。width/height=0は image_1 のサイズ、画像なしなら2048。
|
||||
- 修正前はプロンプト欄が空で、Metadataタブもプロンプトを読めなかった。T2I（プロンプト強化オフ、2048x2048）とEdit（プロンプト強化オン、2720x1536）で実生成確認。
|
||||
|
||||
### 修正: ImageCompareノードを含むワークフローで検証エラー
|
||||
|
||||
- `ImageCompare` の `compare_view`（表示専用の socketless ウィジェット、`widgets_values` に保存されない）がAPI変換で欠落し、`required_input_missing` になっていた。フロントエンドと同じ `{"__value__": ["", ""]}` を補う（`_DISPLAY_WIDGET_DEFAULTS`）。MiniMax H3 Image Edit テンプレートで発覚。
|
||||
|
||||
## v0.9.5（2026-11-11）
|
||||
|
||||
### AI TOOL Chat: スキル v2（metadata・同梱スキルの版管理・プロンプトボタン）
|
||||
|
||||
- スキルの frontmatter に `metadata:` を追加（`output` prompt/svg の自動チェック＋1回だけ再依頼、`tools`、`temperature`、`think`、`context`: workflow / current-prompt / workflow-detail / logs）。`name`/`description` は Agent Skills（SKILL.md）と互換のまま。
|
||||
- 同梱スキルを `py/services/builtin_skills/*.md` に分離し `wfs-version` で版管理。`ai_skills/.builtin_state.json` に版とハッシュを記録し、未編集なら自動更新、編集済みなら残して ↻ ボタンを表示。v0.3.79 版の未編集コピーはハッシュで判別して更新。
|
||||
- 同梱スキル6本: T2I Prompt Enhancer / Image to Prompt / Storyboard to Lab / Workflow Doctor（新規）、SVG Icon Generator v2、Skill Creator v3。
|
||||
- ```prompt ブロックに「GenerateUIに反映」「このプロンプトで生成」「コピー」。応答は依存なしの Markdown 表示（先に全体をエスケープ）。
|
||||
- ワークフロー要約から「今のプロンプト」を既定で外した。「参考のみ・写すな」と書いても gemma4 e4b は 6/6 回で古いネガティブ（pink wear）を新しいプロンプトへ混ぜたため。発言が今のプロンプトに触れたときと `context: [current-prompt]` のときだけ渡す。
|
||||
|
||||
### AI TOOL Chat: 生成画像のチェックと修正ループ（意思決定モデル）
|
||||
|
||||
- 新規 `static/js/chat-image-check.js`。チャットモデルが依頼を3〜8個の yes/no 項目にし（JSON Schema）、Ollama の画像対応意思決定モデル（既定 d3:4b）が確率で判定。✔≥0.85 / ? 0.3〜0.85 / ✘<0.3。
|
||||
- 実測: 生成VLM（gemma4 uncensored e4b）は「剣を持っている」を誤って合格（7/8）。d3:4b・clef-flash・clef は引っかけ含む18/18正解。属性の取り違え（帽子が女の子なのに「男の子が帽子」0.70〜0.79）があるため合格ラインを0.85にし、`expect: no` の逆向き確認項目を加えた。
|
||||
- 「足りない点を直して再生成」で未達項目だけを伝えて書き直し・再生成。自動修正0〜3回。SVG も描画して部位ごとにチェック。
|
||||
|
||||
### AI TOOL Chat: Workflow Doctor / Storyboard to Lab / フォルダ形式スキル
|
||||
|
||||
- 新規 `static/js/chat-context.js`: 未インストールのノード、ComfyUI の一覧に無いファイル・選択肢（近い名前の候補）、切れたリンク、未接続のコア型必須入力、`/history` の直近エラー、ComfyUI ログの最新エラー部分を文脈に入れる。手元77ワークフローで誤検出0（ウィジェット系の必須入力チェックは1/3で誤検出したため除外）。
|
||||
- ```labplan → Lab プラン（1シーン=1イテレーション、T2I、seed 固定）。`lab-tab.js` に `openLabPlanData()`。
|
||||
- スキルの zip 取り込み（SKILL.md 必須、テキストのみ、スクリプトは展開しない、zip-slip とサイズの検査）。参照ファイルは `read_skill_file` ツールで必要時に読む。
|
||||
- Lab: KSampler 列が seed / sampler / scheduler だけ指定のとき「ワークフローの設定のまま」と表示していた不具合を修正。
|
||||
|
||||
### AI TOOL: OCR・ペインの折りたたみ
|
||||
|
||||
- 新規 `static/js/ocr-tools.js`（Comic Creator の OCR タブから汎用部分を移植）。実測（日英混在・縦書きあり）: LightOnOCR-3 4b 1.1秒（位置付き7.9秒）100%、PaddleOCR-VL-manga 3.9秒100%、unlimited-ocr 14秒100%、PaddleOCR-VL 1.6 / GLM-OCR は縦書きを読み落とし94%。
|
||||
- 位置の表示（LightOnOCR / DeepSeek-OCR）、枠クリックで行選択、「翻訳へ送る」「Chatへ送る」。Translation / Settings ペインを細い帯にたためる。
|
||||
|
||||
### 修正: 非推奨の import（GitHub issue #3）
|
||||
|
||||
- `top_menu_extension.js` の `/scripts/ui/components/button.js`・`buttonGroup.js` の import は、a6f45ef で actionBarButtons に移行して以来呼ばれていない旧メニュー用コードだった。`window.comfyAPI` への置き換えではなく削除。`web/comfyui/*.bak` も削除。
|
||||
|
||||
### ドキュメント
|
||||
|
||||
- ヘルプ（AI TOOL を12カードに再編、Lab に Storyboard）、マニュアル v0.9.5（v0.9.2〜v0.9.4 の未反映分も追記）、README の TagSpace スクリーンショット、Registry アイコン用に `docs/thumb.png` を 800x400 に縮小。
|
||||
|
||||
**検証**: 各機能を ComfyUI_5（:8189）で実機確認（Chrome / Playwright）。単体テスト: スキルサービス35件、JS（解析・チェック・Markdown・OCR・文脈）約50件。
|
||||
|
||||
**未確認**: サイドパネル（A タブ）はスキル本文のみ対応（metadata・ボタン・チェックは SPA のみ）。判定の閾値は画像約5枚から決めた値。OpenAI 互換バックエンドでの `read_skill_file` 往復。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.4（2026-11-07）
|
||||
|
||||
### Gallery: 3Dモデル対応（GLB / glTF / VRM / PLY / SPZ）
|
||||
|
||||
- 新規 `static/js/gallery-3d-viewer.js`: three.jsのビューア。詳細パネル・ライトボックス・比較で開き、枠がDOMから外れると自分で破棄する（WebGLコンテキストを解放）。
|
||||
- 新規 `static/lib/three-viewer/`: three 0.187 / @pixiv/three-vrm 3.5.5 / @sparkjsdev/spark 2.4.0 のバンドル。Spark(2.8MB)は別チャンクで、スプラットを開くときだけ読む。再ビルド手順は同フォルダのREADME.txt。
|
||||
- サムネイル: サーバーに3Dレンダラーが無いため、`/thumb` の404を合図にブラウザが描画し `/wfm/gallery/image/thumb3d` へ保存する。1件ずつ処理し、128MB超は開いたときに保存する。
|
||||
- `.gltf` の外部ファイルは `/wfm/gallery/model3d/asset` で配信する（.bin/画像のみ、許可ルート内のみ）。
|
||||
- PLYは `f_dc_0`/`scale_0` の有無でスプラットを判定する。`double` 属性はFloat64ArrayになりWebGLが拒否するのでFloat32へ変換する。
|
||||
- `asset.extras` からSaveGLBのprompt/workflowを読む（GLBはJSONチャンクのみ読む）。
|
||||
- 3DファイルはImage Edit / GenerateUI / Send to LI node / Tagger / Feederへ送れないようにした。
|
||||
|
||||
### P-H3 Image対応
|
||||
|
||||
- `PH3ImageConditioning` をGenerateUI/Metadata/Iタブに追加（`TextEncodeFibo2` と同じ扱い、negativeなし）。`_unifyPromptEnhancerText` は `TextGenerateV2` も対象にした。
|
||||
|
||||
### 修正: TextGenerate系の値ずれ
|
||||
|
||||
- `_normalizeLinkedWidgetValues` が、リンク済みpromptの空枠を持つ形式を「枠なし形式」と誤判定していた。値が1つずれても、無効なDynamicComboキーがoptions[0]へフォールバックして枠数が偶然一致するため。Krea-2 / Ernie / LTX-2.5 などでも `sampling_mode:2048` のようにずれていた。選択キーの有効性も判定に加えた。
|
||||
|
||||
**検証**: 3Dは実サンプルで描画・自動サムネイル・詳細・拡大・破棄・送信ガード（Image Editボタン）を確認（SPZとスプラットPLYは自作サンプル）。パストラバーサルは全て404。変換は手元の640ワークフローを修正前後で比較し、差分21件は全てTextGenerate系。うち5種類は `widgets_values_named` 等で正しい値と確認した。P-H3は編集（1024x1024、約60秒）とT2I（3360x2496、約40秒）をGenerateUIから実生成（RTX 4070 12GB、モデルロード済み）。
|
||||
|
||||
**未確認**: 比較表示、テーブル表示のサムネイル、128MB超のファイル、実データのスプラット、サイドパネルIタブのP-H3、Draco/Meshopt圧縮のモデル。
|
||||
|
||||
**補足**: KTX2圧縮テクスチャは未対応。`.gltf` を移動・削除しても外部ファイルは追従しない。GenerateUIにプロンプト強化のオン/オフ切り替えは無い。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.3（2026-11-05）
|
||||
|
||||
### Image Edit: Drawツールのカラーパレット・ショートカット・スポイト（Comic Creator v1.67.0 から移植）
|
||||
|
||||
- 新規 `static/js/image-edit/DrawPalette.js`: Drawツールの2段目（`ie-tool-options`）に色1/色2、基本色20色、カスタム色20枠を表示する。右クリックで現在の色の登録・色の編集・削除（カスタム色）・初期の色に戻す（変更済みの基本色）。保存先は localStorage `ie_draw_palette_v1`。
|
||||
- `image-edit-tab.js`: 色の変更を `_setDrawColor` に集約し、モードは `_setDrawMode`、サイズは `_setDrawBrushSize` / `_stepDrawBrushSize` に分離。
|
||||
- ショートカット（Draw中のみ）: `D` Draw / `E` Erase / `X` 色1・色2切替 / `[` `]` サイズ増減（1〜10pxは1、10〜50pxは5、50〜200pxは10刻み）/ `B` ブラシ選択の開閉。ブラシ選択の表示中は D/E/X/[/] を無効にし、Escで閉じる。他のツールからのBはこれまでどおりDrawへの切り替え。
|
||||
- スポイト: WFSのDrawツールには無かったためCC版から追加。右パネルのColorの隣のボタンからキャンバスの色を拾う。Esc・再クリック・ツール切替・キャンバス外クリック・Closeで解除。
|
||||
- ブラシの初期色を赤から黒に変更（色1/色2 = 黒/白）。
|
||||
|
||||
**CC版との違い**
|
||||
|
||||
- 色の引き継ぎは、Drawツールの表示時ではなく `_initCanvases` で `DrawTool` を作り直す時点で行う。
|
||||
- スポイトは `ie-canvas-draw` から色を読むため、アクティブレイヤーのみの表示なら合成し直してから読む。
|
||||
- 右クリックメニューを開いたままツールを切り替えても残らないよう、`_renderToolOptions` で閉じる。
|
||||
|
||||
**検証（Playwright、ComfyUI :8189）**: パレットからの描画色、登録と再読み込み後の保持、重複登録の通知、右クリックメニュー、基本色の復元、Color入力との連動、各ショートカット、入力欄やCtrl/Alt同時押しで反応しないこと、サイズの刻み、ブラシ選択の開閉、ブラシ未登録時の通知、Newでの色の引き継ぎ、スポイトの取得と解除、Undo履歴が増えないこと、別レイヤーの色の取得を確認。コンソールエラーなし。
|
||||
|
||||
**未確認**: 「色を編集…」で開くブラウザ標準のカラーピッカーの表示と、日本語キーボードでの `[` `]` の実打鍵。
|
||||
|
||||
**補足**: 透明部分をスポイトすると黒を拾う（CCと同じ）。描いた線から拾うと値が1ずれることがある。READMEのバージョンバッジが0.8.11のままだったので0.9.3に更新した。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.2（2026-11-03）
|
||||
|
||||
### Selectorイベントループ有効時に画像が壊れて届く不具合を修正
|
||||
|
||||
- 症状: 設定「Selectorイベントループ」（`windows_selector_event_loop`）が有効だと、ギャラリーのプレビューと Download ボタンで取得した画像が壊れる（元ファイルと同サイズのまま、32KB 分のブロックが直後のブロックの内容に置き換わる）。出力フォルダのファイル自体は正常。
|
||||
- 原因: Selector 方式では aiohttp の `FileResponse` が asyncio の `_sendfile_fallback` を通る。これは 16KB のバッファを使い回しながら、送り切れなかった分をトランスポートにコピーせず積むため（Python 3.12）、送信が詰まると次の読み込みで上書きされる。ギャラリーは多数の画像を同時に読むので詰まりやすい。
|
||||
- 修正: `prestartup_script.py` で Selector に切り替えるとき `AIOHTTP_NOSENDFILE=1` も設定し、aiohttp がチャンクごとに新しい bytes を読んで送る経路を使うようにした（aiohttp 読み込み済みの場合はモジュール定数も上書き）。プロセス全体に効くので ComfyUI 本体の `/view` なども対象。
|
||||
- 検証: 受信を遅くしたクライアントで、修正前は 5/5 回破損、修正後は 5/5 回ハッシュ一致。再起動後の実サーバー（ギャラリー配信 URL）でも 5/5 回一致。
|
||||
- 壊れた応答は `Cache-Control: max-age=3600` でブラウザに残ることがあるので、更新後は再読み込み（Ctrl+F5）が必要。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.1（2026-11-03）
|
||||
|
||||
### TagSpace: 選んだタグだけを追加 / タグのカッコをエスケープして追加
|
||||
|
||||
- 選択中パネルでタグを1つ以上選んでいると、「追加」ボタンが「選択したタグを追加（N）」になり、選んだタグだけ（プロンプト中の順）を1つのチップ（名前は「rapunzel（2タグ）」）として追加する。選んでいなければ従来どおり全体（カードはプロンプト全文、Gallery点は全タグ）。3D画面のダブルクリックと一覧の＋は常に全体。
|
||||
- 不具合修正: タグは分解時にエスケープを外している（`rapunzel \(disney\)` → `rapunzel (disney)`）ため、タグ・Gallery点のチップをそのままプロンプトに入れると `(disney)` が強調指定になっていた。プロンプトに入れるときにカッコを `\(` `\)` に戻す（表示はそのまま）。カード全文の `\(...\)` と同じ形になるので、重複除去でも1つにまとまる。
|
||||
- 検証（Playwright）: rapunzel カードで未選択の「追加」→全文、2つ選んで「選択したタグを追加（2）」→ `rapunzel \(disney\), braided ponytail`、一覧の＋で同じタグを追加しても重複せず。
|
||||
- ヘルプ（tagspace-10 英/日/中）、README、マニュアル第11章 11-7（日英、全章まとめ・ZIP 再作成）。
|
||||
|
||||
### TagSpace: Gallery点から「Galleryで開く」
|
||||
|
||||
- 選択中パネル（Gallery点）に「Galleryで開く」ボタン。`gallery-tab.js` に `revealInGallery(path)` を新設（export）: ギャラリータブ／Output サブタブへ切り替え（初回はここで出力フォルダとツリーを読み込むので `state.folderTree` を待つ）→ 絞り込み（検索・お気に入り・タグ・グループ・類似表示）を「Clear filters」と同じ要領で解除 → ツリーを根から目的のフォルダまで矢印クリックで展開（子は展開時に描画される遅延描画のため）→ フォルダを選んで `await loadImages()` → 一覧で `[data-path]` の要素をクリック（通常クリックと同じ選択・詳細表示）して中央へスクロール。
|
||||
- サムネイル表示は50件ずつ描くので、目的の画像まで `_appendNextPage` で描き足し、無限スクロールの目印を末尾へ戻す（全件描いたら外す）。パスは `\` / 大文字小文字の違いを無視して比較。出力フォルダ外・一覧に無い画像は false を返し、TagSpace 側でトーストを出す。
|
||||
- 検証（Playwright）: 検索語を入れた状態から TagSpace の Gallery 点で押す → 0.3秒でギャラリーに切り替わり、ツリー「2026-06-07」選択・一覧で目的の画像が選択されて画面内・検索欄クリア・詳細表示。456枚のフォルダの453枚目でも 0.6秒で表示（全件描画・目印も解除）。出力フォルダ外／存在しないパスは false。コンソールエラーなし。
|
||||
- ヘルプ（tagspace-10 英/日/中）、README、マニュアル第11章 11-7（日英、全章まとめ・ZIP 再作成）。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.0（2026-11-03）
|
||||
|
||||
### マニュアル（画像なし配布版）を v0.9.0 に改訂
|
||||
|
||||
- 日本語・英語の `docs/manual/` を v0.8.9 → v0.9.0 に更新（全章まとめ・ZIP・README を作り直し、v0.8.9 のファイルは削除）。
|
||||
- 反映した変更: TagSpace タブ（第11章に 11-7 を新設、第3章のタブ一覧・付録の用語集・第12章のトラブル表）、Gallery セマンティック検索の Ollama 化（第8章 8-6・第2章・第12章。ONNX の GPU 設定の記述を Tagger 側へ移動）、Tagger の PixAI 対応と GPU 確認（第11章 11-2・第12章）、GenerateUI の Input タブ横並び・Raw JSON 折りたたみ（第5章）、Fibo 2.0 Edit／FastH3 OmniRef／Pika-H3-Flex（第5章 5-9）、Video Plan の Reference Image（第11章 11-4）、Comic Creator からの複数参照画像の実行（第11章 11-6）。
|
||||
- 前回の全章まとめは v0.8.11 の章ファイル修正（Send CC）を取り込んでいなかった。今回は章ファイルから全章まとめを作り直したので解消。
|
||||
- 英語版の Raw JSON 折りたたみボタンは、実際の表示（»）ではなく ASCII の `>>` と書いた。
|
||||
|
||||
### TagSpace タブ新設（latenttag-3d の WFS 統合）
|
||||
|
||||
- **目的**: latenttag-3d（単独アプリは別途継続）を WFS に組み込み、WFS のデータだけでタグを視覚的に選んでプロンプトを作る。
|
||||
- **材料**: カード＝ImagePrompt（`ws_image_prompt`、`.txt` または ponyxlWildcardsVault の YAML から `_read_sidecar_prompt` でプロンプト解決、プロンプトのある951枚）／タグ＝ImagePrompt・ワイルドカード（`.txt` 行・YAML の `- ` 行）・Tagger DB・`prompts.json` の文を `, \n | { } BREAK` で分解し正規化（重み・括弧・Dynamic Prompts の `n::`・LoRA・`__wildcard__`・`rating:` を除去、`_`→空白、2〜40文字・5語以内）。出典の種類数→出現回数の順で上位 N（既定2000）／背景＝Gallery 索引から seed 固定で間引いた画像（既定4000）。
|
||||
- **ベクトル**: カードは Gallery セマンティック索引の画像ベクトルを流用（未索引分は構築時に `SemanticService.start_index(..., "missing")` で追加）。タグは Ollama でテキスト埋め込み（`task: search result | query:` 付き）し `tagspace_tags.npz` にキャッシュ。索引が Gemini のときはエラーにする。
|
||||
- **modality gap**: 同じ空間でも画像とテキストは別の塊になる。実測で、そのまま PCA すると第1・第2軸が画像/テキストの分離に使われた（|Δ平均|/標準偏差 2.99・4.43）。画像とテキストを別々の平均で中心化すると 0 になり、画像→近いタグも改善（例 `alley`: 中心化前 anime, twintails… → 後 street, neon lights, city lights, night）。
|
||||
- 画像側の平均を「カード＋背景」で取ると、数の多い背景に引っ張られてカードがタグの塊から離れた（中心間距離 5.06）。カードの平均に変えて 0.00 にした。
|
||||
- 近傍・検索も同じ中心化コサイン。同種どうしのスコアが高く出るため、カードとタグを別々に順位付けして交互に並べる。
|
||||
- **3D**: three.js r187（MIT）から使う部品＋OrbitControls だけを esbuild で束ねた `static/lib/three/three-tagspace.min.js`（559KB）を同梱し、タブを初めて開いたときに動的 import。カードは Sprite（サムネイルを6並列で順次読み込み）、タグ・背景は Points、選択/近傍/検索は重ね描き、ラベルは HTML オーバーレイ。タブが非表示の間は描画ループを止める。
|
||||
- **API**: `/wfm/tagspace/{status,space,build,cancel,search,neighbors}`（`tagspace_routes.py`。Gallery と同じ GalleryService/SemanticService を共有）。生成物は `tagspace_space.json`（表示用）・`tagspace_space.npz`（近傍/検索用）・`tagspace_tags.npz`（タグのベクトル）。
|
||||
- **検証**（ComfyUI_5 / 8189）: 初回構築 約2.5分（カード画像1,054枚の索引追加＋タグ2,000件）、キャッシュありの再構築 7秒。英語・日本語の検索、キャンバスのホバー/クリック（タグ・カード）、チップのトグル、重複除去、GenerateUI の Positive への追記（`wfs_sdxl_gallery.json`）、ワークフロー未読込時のトーストを確認。コンソールエラー0。
|
||||
- 注意: Git Bash の curl で日本語の JSON を送ると文字化けして空の検索になる（API 側の問題ではない）。ヘルプ（TagSpace ページ 英/日/中）と README を追加。マニュアルは未更新。
|
||||
- **追加: Gallery 画像のプロンプト由来タグ**（Tagger による補完は次の段階で検討）
|
||||
- 表示自体にタグは不要（背景点は画像ベクトルで配置）。目的は、自分の生成画像の語をタグの材料にすることと、Gallery 点をクリックしてその画像のタグを入れること。
|
||||
- 抽出: 埋め込みの API 形式グラフで、サンプラー/ガイダーの `positive`（無ければ Guider 系の `conditioning`）から上流をたどり、`text` / `prompt` / `clip_l` 等の文字列入力を集める。`negative` 系の入力はたどらない（ControlNet 等の両方を持つノード経由で negative が混ざるのを防ぐ）。UI 形式しか無い画像は対象外。
|
||||
- 実測: 2.9万枚中 64%（18,723枚）で取得、初回 約2〜3分・キャッシュ（`tagspace_prompts.json`、size/mtime 一致）で 0.7秒。典型的なネガティブ語が3つ以上混ざるのは5件で、いずれも利用者がプロンプト欄に画面テキストを貼っていたもの。区切りはカンマ等のみ（スペース区切りは「blue eyes」が分かれるため不採用）。
|
||||
- **出典の偏り**: タグを「出典の種類数→出現数」で選ぶと、Gallery が上位2000件中1982件を占めた。出典ごとに「そのタグを含む文の数」で並べ、出典を順番に1件ずつ取る方式に変更（Gallery 682・ImagePrompt 639・ワイルドカード 476・Tagger DB 199・Prompt ライブラリ 4）。
|
||||
- 背景点に `tags` を持たせ、クリックでまとめて1チップ（`kind: image`）。プロンプトが無い画像（背景4000中 約41%）は追加不可で、詳細に「タグなし」と表示。
|
||||
- バグ修正: 背景の点（`userData.items`）が `kind` を付ける前の生データだったため、ホバー/クリックで種類と名前が undefined になっていた。同じオブジェクトに `kind` を付けるよう修正。
|
||||
- バグ修正: セマンティック索引に削除済み画像が残っており、背景のサムネイルが 404 になった。背景は多めに取って実在するファイルだけに絞り、プロンプト読み取りでも削除済みは除外。
|
||||
- 既知: 画像編集モデルの指示文（例 `do not use the colors`）も5語以内ならタグとして残る。
|
||||
- 検証（ComfyUI_5 再起動後）: 構築148秒（プロンプト読み取り111秒）→キャッシュ有りで15〜16秒。背景4000点で存在しないファイル0件・サムネイル404は40件抽出で0件。Gallery 点のクリック（タグありは1チップで追加、タグなしは追加不可）と、ホバー331回でコンソールエラー0を確認。
|
||||
- **追加: Tagger 補完**（プロンプトを取り出せない Gallery 画像、2.9万枚中 10,652枚）
|
||||
- ツールバー「Tagger補完: [OFF/モデル] [Gallery点のみ/プロンプトの無い全画像]」。モデルは Tagger タブの ONNX モデル（`.onnx` の無い `wd-vit-tagger-v3` は出ない）。`TaggerService.tag_image()` を追加し、しきい値は既定値（WD 0.35/0.85、PixAI は推奨値 0.17/0.27）、評価ラベル（general/sensitive/questionable/explicit）は除外。
|
||||
- 背景点の選択をタグ集めより前に移し、「Gallery点のみ」はその中のプロンプト由来タグが無い画像だけを処理。結果は `tagspace_tagger.json`（path→size/mtime/model/tags）にキャッシュし、100件ごとと終了時に保存（中止しても残る）。補完のために読み込んだモデルは終了後に Tagger のキャッシュから外す（Tagger タブで読み込み済みなら残す）。
|
||||
- 新出典 `tagger_auto`（「Tagger補完」、黄緑）。背景点は `tag_src: prompt|tagger` を持ち、Tagger 由来はツールチップ/詳細に「（Tagger補完）」。進み具合に実行先（CUDA/CPU）を表示。
|
||||
- **環境の問題（再発）**: ComfyUI_5 の venv で `onnxruntime` 1.31.0（CPU）と `onnxruntime-gpu` 1.30.0 が共存し CUDA が無効。CPU では wd-v1-4-convnext 465ms/枚・wd-v1-4-vit 641ms/枚（全画像なら約1.5時間）。修復は ComfyUI 停止→`fix_onnxruntime_gpu.bat`（外側リポジトリ、v0.8.10 と同じ）。今回は未実施。
|
||||
- 検証（CPU）: Gallery点600・Gallery点のみ・convnext で 288枚を142秒（0.49秒/枚）、全600点にタグ（プロンプト312・Tagger 288・なし0）。`tagger_auto` を含むタグ571件（うち補完でしか出ないもの180件: 1boy, multiple girls, scenery 等）。キャッシュで再構築17秒。UI（選択欄の有効/無効、凡例、「（Tagger補完）」表示、クリックで1チップ）とコンソールエラー0を確認。確認後は Gallery点4000・補完OFFで作り直し済み。
|
||||
- GPU 修復後（利用者が `fix_onnxruntime_gpu.bat` を実行、`onnxruntime-gpu` 1.30.0 のみ・CUDA/TensorRT 有効）: Gallery点4000・Gallery点のみ・convnext で 1,918枚を約136秒（約0.07秒/枚、画像読み込み込み。CPU の約7倍）、全4000点にタグ（プロンプト2,082・Tagger 1,918）。全画像（10,652枚）なら約12〜13分の見込み。
|
||||
- **追加: Gallery 点の近傍**
|
||||
- 以前は Gallery 点を選んでも `select()` が近傍を問い合わせずに抜け、直前の一覧が残っていた（ファイル名とは無関係）。近傍用 `tagspace_space.npz` にも背景のベクトルが無かった。
|
||||
- 背景のベクトル（`bg_ids` / `bg_vecs`、4000×768 で約12MB）を保存し、カード/タグ/画像の中心化行列を読み込み時にキャッシュ。Gallery 点を基準にしたときだけ「画像→タグ→カード」を交互に並べる。カード・タグ・検索では画像を混ぜない（利用者の指定）。一覧の Gallery 画像は丸いサムネイル、タグの無い画像は「＋」なし。
|
||||
- 検証: `ComfyUI_00510_.png` → 似た画像（0.69〜0.71）・pink body suits 等のタグ・カードが交互、57ms。タグ/カード/検索の一覧は card・tag のみ。画面でも Tagger 補完の Gallery 点で確認。
|
||||
- 未解明: 再起動直後の最初の構築が146秒かかり `tagspace_tagger.json` が書き換えられた（2回目以降は13秒・書き換えなし、ログに警告なし）。再現したら段階ごとの時間を記録して調べる。
|
||||
- **追加: 表示と操作の改善**
|
||||
- 発光: 放射グラデーションのテクスチャ（明るい芯＋にじみ）を加算合成（`blending: 2`。同梱サブセットに `AdditiveBlending` が無いため数値で指定）、`depthWrite: false`、点サイズ×2.4。背景は夜空色 `#05070d`。Gallery点2万点だと重なりで中心が白く飛んだので、発光時は Gallery点の不透明度を 0.55→0.3。
|
||||
- またたき: `PointsMaterial.onBeforeCompile` で頂点シェーダーに `twinkle`（位相・速さ。添字から決まる疑似乱数で再構築しても同じ）と時刻 uniform を差し込み、`vColor` を 0.15〜1.25 倍（`pow(0.5+0.5 sin, 2)` で暗い時間を長く・ピークを鋭く）。タグ・カード・Gallery点のみで、選択・近傍の強調は瞬かない。検証: 0.9秒で変化した画素 約2.6万（オフ時0）。
|
||||
- 拡大表示: 「選択中」の画像と「近く」の一覧の画像に `data-ts-zoom`。マウスを乗せるとサイド列の左に 360px で表示（読み込み済みの小画像を先に出し 720px に差し替え、`pointer-events: none`）。
|
||||
- 追加/削除ボタンを「選択中」の名前の行（`position: sticky`）へ移動。本文をスクロールしても押せる。
|
||||
- クリックは選択のみ（`e.detail > 1` は無視）、ダブルクリックで追加/削除。「追加」ボタンと一覧の＋は従来どおり。ヒント・空欄の案内・ヘルプ（英/日/中）・README を更新し、ヘルプに表示の項目（tagspace-8）を追加。
|
||||
|
||||
- **追加: ワイルドカード YAML をパーサーで読む**
|
||||
- 従来は「`- ` で始まる行」を1項目として読んでいたため、`- >-` 等のブロックスカラー（本文が次行以降）は `>-` だけになり本文を取りこぼしていた（billions_of_all_in_one.yaml、corn-flakes-doll.yaml）。二重引用符内の `’` もエスケープのまま残っていた。
|
||||
- PyYAML（ComfyUI 本体の依存）の `CSafeLoader`（無ければ `SafeLoader`）で読み、文字列の葉だけを取る（キー＝カテゴリ名、yes/no・数値に変換された葉は除く）。YAML として読めないファイルは従来の行読みにフォールバック。4.2万行で 0.19 秒（純 Python の safe_load は 1.86 秒）。
|
||||
- 実測（ワイルドカード7ファイル）: 異なるタグ 28,861 → 30,130（+1,269。female / wings / dress / lace 等）。他5ファイルは不変。
|
||||
- 実機（ComfyUI_5 再起動後、2000タグ・背景20000で再構築）: ワイルドカード由来 748 → 762、入れ替わり22件（male / benevolent / pointed ears 等が入り、出典を順番に取る方式なので他出典の下位が抜ける）。female の出現数 164 → 208 等。
|
||||
- **追加: 全タグ・最低出現回数・ワイルドカードのオン/オフ**
|
||||
- 背景: 「全部の点を表示したい」。タグの種類は全出典で約4.8万、うち約3.7万がワイルドカードにしか無い語（1回だけの語が大半）。ワイルドカードを除くと約1.1万。
|
||||
- ツールバーに「全タグ」（`max_tags=0`、上限 `ALL_TAGS_LIMIT`=20,000）、「最低出現回数」（1/2/3/5。全出典合計の出現数で候補から外す）、「ワイルドカード」（材料に含めるか）。meta に `max_tags` / `min_count` / `wildcard` / `tags_available` を記録し、一部だけ使ったときは状態表示に「（候補 N）」。既定値（ワイルドカードあり・最低1回）は従来と同じ結果（2000タグの集合・出現数・出典が完全一致を確認）。
|
||||
- **正規化の追加**: 全タグにすると、MOSS-Image のレイアウト指定（`{"bbox":[...],"label":"text"}`）や文字入れ指定の JSON 断片が Gallery プロンプトから大量に入り、`bbox":-8` 等 234件が空間の外に孤立した塊を作った。`"key":` を含む断片とモデルのファイル名（`.safetensors` 等）を除外し、片側だけ残った `"` は取り除く。括弧を外した後の `embedding:` も除外。ワイルドカードなし全タグ 11,406 → 10,435（2回以上 7,383 → 6,843）。
|
||||
- **Ollama の一時ポート枯渇**: 9,056件を連続で埋め込んだところ 6,880件目で `HTTP 400 ... dial tcp 127.0.0.1:NNNNN: connectex: Only one usage of each socket address` で失敗。Ollama はテキストごとにランナーへ新しい接続を張るため、ループバックの TIME_WAIT が 14,228 / 16,384 に達していた。このエラーを一時的扱いにし、30秒待って再試行（最大5回、Windows の TIME_WAIT は約2分で解放）。単体テスト（2回失敗→3回目で回復、失敗が続けば6回で諦める）で確認。実構築では以後枯渇せず（新規5,203件で TIME_WAIT 最大 11,968）、本番経路での再試行は未発生。
|
||||
- 実測（ComfyUI_5、Gallery点20,000・Tagger補完あり）: ワイルドカードなし全タグ 79秒（新規埋め込み 2,656件）、最低2回 22秒、ワイルドカードあり全タグ・最低2回（14,188件）127秒（新規 5,203件）。空間データ 9.4MB（2000タグ）→ 10.8MB（11,406タグ）。ブラウザで約3.2万点を表示し、毎フレームのホバー判定込みで 60fps・JSヒープ 35MB。PCA 寄与率は 10.5% → 7.9%（タグの種類が増えて軸が分散）。
|
||||
- **追加: Gallery点のタグを選んで空間検索**
|
||||
- 選択中パネルで Gallery 画像のプロンプト（またはTagger補完）のタグを、文ではなくタグごとのボタンで表示。クリックで選択/解除（複数可、`state.picked` に画像IDと Set で保持し、別の要素を選ぶとリセット）。「選択したタグで空間検索（N）」で、選んだタグをプロンプト中の順に `, ` でつないだ文を既存の `runSearch()` に渡す（検索欄にも入れる）。
|
||||
- タグが多いとパネル（最大260px）からボタンがはみ出したため、ボタン行を `position: sticky; bottom: 0` で下端に固定。
|
||||
- 検証（Playwright）: 33タグの画像で2つ選択→ボタン有効化→「brown hair, east asian」で検索し、east asian 0.67 / brown hair 0.67 / dark brown hair 0.62 等が並ぶ。検索後も選択を保持。15タグの画像でパネルがスクロールする状態でもボタンが見えることを確認。
|
||||
- **追加: カードのタグも選んで空間検索**
|
||||
- カードはプロンプトを文で持つだけだったので、構築時に `split_tags`（Gallery点と同じ正規化）で `tags` を付ける。詳細パネルの表示処理を `tagPicker()` に共通化し、カードはタグのボタン＋「プロンプト全文」（`<details>`）＋カテゴリ。タグが無いカード・作り直し前の空間（`tags` 無し）は従来どおり全文表示。
|
||||
- 検証（ComfyUI_5 再起動・再構築後）: 951枚中948枚にタグ（平均3.4個）。無いのは `zero two \(darling in the franxx\)` のように1語が5語を超えるもの等3枚。「resentful」カードで angry / evil smile を選んで検索 → evil smile 0.89、evil grin 0.84、arrogant smile 0.64。
|
||||
- **修正: ズーム時に広い範囲で同じ点が反応する（ホバー・クリック）**
|
||||
- 原因: `Raycaster.params.Points.threshold` が3D空間の固定値 0.12 で、しかも結果はカメラに近い順に並ぶのに先頭を採用していた。ズームして点群に入ると、手前の1点が画面の広い範囲で「当たり幅の内側」になり続ける。
|
||||
- 修正（`pick()`）: 候補は広めに取り（注視点の3倍の奥行きで 8px になる幅と、描かれた点の3D上の半径の大きい方）、各点を画面へ投影してカーソルとの距離（px）が最小のものを選ぶ。点はカーソルから 8px 以内、拡大で大きく描かれた点は見た目の半径（発光時は芯）まで。カードのサムネイル（Sprite）は従来どおり当たれば候補。
|
||||
- 検証（Playwright、ホイールで大きくズーム、10×10 の格子でホバー）: 修正前 100か所すべてで反応し2種類（boke 55回）→ 修正後（適度なズーム）73か所で反応し72種類、点の無い場所は無反応。4か所のクリックもそれぞれ別の要素を選択。カメラがカードの中に入るほどズームするとサムネイルが画面全体を覆うので、そのカードが反応するのは正しい。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.14（2026-11-03）
|
||||
|
||||
### Fibo 2.0 Edit / FastH3 OmniRef / Pika-H3-Flex 対応（GenerateUI・Metadata・Videoタブ）
|
||||
|
||||
- **GenerateUI解析**（`comfyui-workflow.js`）: `TextEncodeFibo2` を `TextEncodeQwenImage21` と同じ固定ロール扱いに（prompt は PreviewAny → プロンプト強化スイッチ経由でリンクされるので配線元へ positive を伝播、negative_prompt は VGL JSON のままプロンプト欄に出す、width/height/batch_size を設定タブへ。0 は自動サイズ）。動画オールインワンノードの判定を `VIDEO_ALL_IN_ONE_RE`（`ImageToVideo|TextToVideo|ReferenceToVideo`）に集約し、`MiniMaxH3ReferenceToVideo` の prompt が別ノード（PrimitiveStringMultiline）から配線されていれば配線元を positive にする。
|
||||
- **Metadata / サイドパネルIタブ**: 動画ノードの prompt がリンクなら配線元テキストを解決。Fibo2 は MageFlow/Boogu と同じく1ノードで positive/negative を分けて取り出す。
|
||||
- **Videoタブ（Plan）**: 既存の MiniMax H3 / LTX-2.5 / Wan2.2 はサブグラフの公開入力1ノードに書く方式だが、FastH3 系はサブグラフ無しのフラット構成。新ファミリー `fasth3` として、各入力の**配線元ノード**を書き込み先にした（prompt → PrimitiveStringMultiline、duration → PrimitiveFloat（length が未配線なら自ノードの length へ 24fps・17k+5 グリッドのフレーム数で書く）、解像度 → ResolutionSelector）。First Image / Last Image 欄を **Reference Image / Reference Image 2**（`ref_images.ref_image_0` / `ref_image_1`）として流用し、Asset タブのボタン表記・トーストも切り替える。
|
||||
- **未指定の参照画像は接続を外す**: FastH3 系のみ、None / Clear の参照画像はテンプレートの LoadImage が繋がっていても切断し、使われなくなった LoadImage はノードごと除く（外さないと UI 上は未指定なのにテンプレートの画像で生成される）。他ファミリーは従来通り。
|
||||
- **chain_previous**: Reference Image 2 は「前ブロックの最終フレーム」として扱わず、常に生成された最終フレームを抽出して次ブロックの1枚目の参照画像にする。
|
||||
- **罠**: FastH3 テンプレートの PrimitiveString/Float・ResolutionSelector・LoadImage は `inputs[]` に widget 定義を持たない形式で保存されており、`widgetIndex` が -1 を返して何も書けなかった。ノード型ごとの既定の並び順にフォールバックするようにした。
|
||||
- **ノード側の仕様**: `MiniMaxH3ReferenceToVideo` は `ref_images.values()` を順に使うだけなので、1枚目が欠けて2枚目だけでも動く（その画像が `<Picture 1>` になる）。
|
||||
- **検証**: Node 上で実コード＋ ComfyUI_5（8189）の object_info を使い、3種のワークフローの変換・解析・抽出と、参照画像の差し替え／注入／切断を確認（既存 minimax / ltx25 テンプレートは結果不変）。Playwright で Video タブを実操作（ラベル・既定値・Clear・＋Block・ドロップ・Asset 表記）。Pika-H3-Flex を Plan から実生成（2ブロック×2秒・0.2MP → 352×608・56フレーム・音声付き、約160秒）し、chain_previous でブロック1の最終フレームがブロック2の参照画像になって自然に繋がることを確認。
|
||||
- **未対応**: 参照画像の3枚目以降・参照動画・参照音声はワークフロー側の設定のまま。Pika は ManualSigmas のため GenerateUI の Steps は効かない（linked 表示）。
|
||||
|
||||
|
||||
### Gallery セマンティック検索: EmbeddingGemma 2 を ONNX から Ollama へ移行
|
||||
|
||||
- **背景**: latenttag-3d（Ollama の `embeddinggemma-2` でタグを埋め込む）と Gallery 索引を共通の空間で使いたい。ONNX 版（`semantic_local.py`、q4）は onnxruntime-gpu の入れ替え問題も抱えていた。
|
||||
- **互換性の実測**: 同じ入力で ONNX 版と Ollama 版（Q8_0）を比較。テキスト cos 0.985〜0.993、画像（12枚）平均 0.967（最小 0.923）。既存索引に Ollama の検索文を当てた上位10件の一致は 8〜9/10。→ npz の `model` 名は `embeddinggemma-2` のまま、既存索引（2.9万枚）をそのまま使う。
|
||||
- **740m / 440m**: 740m=テキスト+画像+音声、440m=テキスト+画像。画像・テキストとも cos 1.0（同じベクトル）なので、切り替えても再作成不要。`:latest` は 740m と同じ（parent_model 740m-q8_0）で、740m 未取得時の代わりに使う。270m / 570m は画像非対応のため選択肢に入れない。
|
||||
- **Ollama の画像埋め込みの罠**: `/api/embed` に `images` フィールドを付けても**エラーにならず無視され、テキストだけの埋め込みが返る**。正しくは `input: [{"image": "<base64>"}]`。`[{"images": ...}]` は 400、壊れた画像は 500 "Failed to load image or audio file"、未取得モデルは 404。`input: []` + `keep_alive: 0` でアンロードできる。
|
||||
- **実装**: `semantic_ollama.py` を新設し `semantic_local.py` を削除。画像は4枚ずつ送信（1枚ずつより約1.8倍速い）し、バッチが失敗したら1枚ずつ送り直して原因の画像だけを失敗扱いにする。Ollama 内部の接続切断（400 "forcibly closed" 等）と 5xx は最大5回再試行。接続不可・モデル未取得はジョブ開始前に検出し、作成中に起きたら中止。作成後は VRAM から外す。URL は `url_guard` で検査。設定は `semantic_settings.json`（`url` / `tag`）、保存用に `/wfm/gallery/semantic/options` を追加。
|
||||
- **UI**: モデル設定タブのデバイス/CPUスレッド/画像精度を廃止し、Ollama URL・モデルサイズ（未取得表示つき）に置き換え。ヘルプ（helpGallery33/35/36・helpTrouble10/11 英/日/中）、README、requirements.txt を更新。マニュアル（v0.8.9版）は未更新。
|
||||
- **実機確認**（ComfyUI_5 / 8189、再起動 13:17 > 反映 13:13 をプロセス起動時刻で確認）: 既存索引 29,356件をそのまま読み込み。テキスト検索 0.9秒、類似画像 約0.6秒（4件）。URL ガードで `169.254.169.254` を拒否。モデル設定の 740m⇔440m 切替が保存され状態欄に反映。`ponyxl/scenes/city`（未索引9枚）を「新規・変更のみ」で作成→約1.7秒・失敗0（4+4+1枚のバッチ）、作成後 `/api/ps` が空（アンロード済み）、追加した `alley.jpeg` の類似上位は同フォルダの画像（0.88〜0.93）。コンソールエラーなし。
|
||||
- 確認中に Python の urllib から `/similar` が1回120秒タイムアウト、`/folders` が1回 WinError 10054 になった。いずれも再試行や curl では毎回0.6秒前後で正常で、変更していない `/folders` でも起きたため今回の変更とは別の現象と判断（このPCで以前から出ている接続切断と同種の可能性）。
|
||||
- **残作業**: 旧 ONNX モデル（`user/default/Workflow-Studio/models/embeddinggemma-2`、302MB）は削除可能。`onnxruntime` は Tagger で使うため依存に残す。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.13（2026-10-27）
|
||||
|
||||
### Edit in Mask Editor One: 送信画像をノードへ永続化（Mask Editor One v0.1.12 対応）
|
||||
|
||||
- **背景**: Mask Editor One v0.1.12（PR #4）が、背景画像を `input/mask_editor/` へアップロードして `node.properties.bg_image` に保存し、ワークフロー切り替え／リロード時の `onConfigure` で復元するようになった。一方 `window.wfmOpenMaskEditorForNode`（`web/comfyui/node_sets_menu.js`）は `node._bgDataUrl` / `node._bgImg` を直接差し替えるだけで `properties` を更新しないため、そのままだとリロード時に**以前の画像が復元され、`/mask_editor/store_image` のサーバーキャッシュを上書きする**（WFSが送った画像が消える）。
|
||||
- **修正**: 画像送信後に `node._persistBgDataUrl(imageDataUrl, "wfs_image.png")` を呼んで `properties.bg_image` を同期（Mask Editor One 側が公開する関数。旧バージョンでは `typeof` チェックで無視）。
|
||||
- **layer_data**: 既存マスクを引き継ぐ際の `layer_data` に、送信画像の `width` / `height` を付与。ノードのマスクプレビュー復元はこのサイズでレイヤーを作る（無い場合は復元済みの背景画像サイズ、それも無ければスキップ）。付与順は「`_bgImg` ロード後」に変更。
|
||||
- **How to apply**: 外部から `MaskEditorOne` ノードの画像を差し替えるコードは、`node._bgDataUrl` を書き換えたら必ず `node._persistBgDataUrl()` で参照も同期すること。`applyImageAndMaskToMaskEditorOneNode`（API形式ワークフローへ書き込む経路）はグラフ上のノードを触らないので影響なし。
|
||||
- **検証**: ComfyUI_5（8189）で、画像A保存→WFS相当でB送信→シリアライズ／復元で、背景がB（300×600）・マスクプレビューも300×600になること、`width/height` なしの `layer_data` でも背景サイズで復元されることを確認。
|
||||
- ヘルプ（`helpImageEditMask14` 英/日/中）とREADMEを更新。マニュアル（v0.8.9版）は未更新。
|
||||
|
||||
|
||||
---
|
||||
|
||||
## v0.8.12（2026-10-27）
|
||||
|
||||
### Comic Creator向け複数画像I2Iブリッジ + GenerateUI Inputタブ: Prompt/Image統合・Raw JSON折りたたみ
|
||||
|
||||
- **背景**: Comic Creator v1.65.0 の Nanobanana+ タブ（ローカルモード）が、Flux.3 Image Klein / Qwen Image 2.1 等の Image Edit 系ワークフローを複数の参照画像つきで実行したい、という要望。既存の `_wfmReceiveI2IRunRequest` は画像1枚（slot 0固定）だった。
|
||||
- **複数画像ブリッジ**: `gallery-tab.js` に `window._wfmReceiveMultiImageI2IRunRequest(imageBlobs, params, workflowData, workflowFilename)` を追加（ワークフロー読込→`image-edit-tab.js` の `runI2IExternalMulti` へ委譲）。`InpaintI2IActions.js` の `_runI2IWithImages` が `imageBlobs[i]` を i 番目のLoadImageへ `applyImageToSlot(file, i, ...)` で割り当て（ノード数超過分は無視）。戻り値は `{ ok, url, slotCount, usedImages }`。
|
||||
- **生成パラメータ**: width/height/steps/cfg/denoise/sampler_name/scheduler は既存の `comfyEditor.setGenerationParams` を再利用（指定時のみ書き込み。未指定=ワークフローの値のまま）。シードは `generate(workflow, { seedValue })` で固定。
|
||||
- **How to apply（限界）**: `setGenerationParams` はKSampler系・解析済みlatentノードにしか書けない。出力サイズを入力画像から決める構成（例: Flux.3 Image Klein の `ReferenceMultiLatent`）では `EmptyFlux2LatentImage` が解析対象外になり、width/height は反映されない（Steps/CFG/シード/プロンプトは反映）。
|
||||
- **Inputタブ（GenerateUI）**: Prompt / Image の内部タブを廃止し、同一画面に左右（Prompt:Image = 3:2）で並べた。親カラム（`.wfm-gen-params-col--input`）をCSSコンテナにして、幅が720px以下になるとImageがPromptの下へ折り返す（`@container`）。JSのタブ切替ハンドラは削除（`gallery-tab.js` の `[data-input-tab="image"]` クリックは `?.` のため無害）。
|
||||
- **Raw JSONの折りたたみ（Inputのみ）**: 右カラムの上端に »/« ボタン。折りたたむと幅30pxの縦書き「Raw JSON」の帯だけが右端に残り、Prompt/Imageが広がる。状態は `localStorage`（`wfm_gen_rawjson_collapsed`）。Model/Settings の Raw JSON は従来どおり。
|
||||
- 落とし穴: `.wfm-gen-rawjson-col`（幅540px）より前に書いた `--collapsed { width:30px }` は詳細度が同じで負ける。`.wfm-gen-rawjson-col.wfm-gen-rawjson-col--collapsed` と2クラスで指定する。
|
||||
- **検証**: Comic Creator内のiframe（ComfyUI_5、8189）で、横並び（757px : 514px）・折りたたみ（540→30px）・狭幅での折り返しと、Comic Creator経由の実生成（Qwen2.1 1枚/2枚入力、Flux.3 Klein）を確認。
|
||||
- ヘルプ（helpGen4 英/日/中・`index.html`）とREADMEを更新。マニュアル（v0.8.9版）は未更新。
|
||||
|
||||
|
||||
---
|
||||
|
||||
## v0.8.11（2026-10-26）
|
||||
|
||||
### Gallery: Send CC をレイアウト／Imageタブの2ボタンに分割
|
||||
|
||||
- **背景**: これまで「Send CC」はレイアウトタブ（選択コマ／オーバーレイ）専用で、Imageタブへ送れるのはComic Creatorの「I2Iへ送る」を押した直後（`window.parent._ccI2ITargetMode === 'image'`）の戻り経路だけだった。送り先を自分で選べるようにしたい、という要望。
|
||||
- **UI**（`templates/index.html`）: `wfm-gallery-send-cc-btn` を「Send CC Layout」に改名し、隣に `wfm-gallery-send-cc-image-btn`「Send CC Image」を追加。どちらもComic Creator内にiframe埋め込みされたときだけ表示（従来どおり）。
|
||||
- **Send CC Layout**（`gallery-tab.js`）: 従来の経路（画像は `insertImageFromUrl`、MP4は `insertVideoFromUrl`）のまま。`_ccI2ITargetMode` による分岐は撤去（Imageタブへは新ボタンで送る）。
|
||||
- **Send CC Image**: Comic Creator側の新設ブリッジ `window.parent.sendImageToImageTab(url, name)` を呼ぶ。ブリッジは Imageタブへ `switchTab` してから `_ccImageTab.loadFromUrl` で読み込む（作業中のレイヤーがあれば新規レイヤー、無ければ新規キャンバス）。動画はImageタブ未対応のため従来どおりトーストで弾く。ブリッジが無い（古いComic Creator）／失敗時は「Comic Creator内から開いてください」のトースト。
|
||||
- **連携の前提**: Comic Creator側の `window.sendImageToImageTab`（`14-integrations.js`、feature/ocr-text-insert ブランチ）が必要。
|
||||
- **検証**: Comic Creator内のGalleryで両ボタンの表示と、画像選択→Send CC Image→Imageタブへの読み込み（トースト表示）を実機（ComfyUI_5）で確認。Send CC Layoutの実クリックは未検証（処理は従来のまま）。
|
||||
- ヘルプ（helpGallery37、日英中）、README、マニュアル第8章・第11章（日英）を更新。
|
||||
- 教訓: ComfyUI_5への同期は変更したファイルだけでなく、事前に実行側とHEADが同一であることを `diff --strip-trailing-cr` で確認してからコピーする。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.10（2026-10-23）
|
||||
|
||||
### Tagger: PixAI Tagger v1.0 対応
|
||||
|
||||
- **対象**: コミュニティ変換の `Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8`（FP32/FP16/INT8、sigmoid込みの確率出力、`selected_tags.csv`にcategory列）。モデルフォルダに`preprocess.json`と`selected_tags.csv`があり、行数が出力数(30,877)と一致すればPixAI形式とみなす（`TaggerService._load_pixai_categories`）。`list_models`は該当モデルに`pixai: true`を付ける。
|
||||
- **前処理**（`_predict_pixai`）: 透過は白合成 → 長辺1008にBICUBICで縮小 → 黒で1008角にセンタリング → RGBのまま`(x/255-0.5)/0.5` → NCHW。WD系(白パディング/BGR/0-255)とは別経路。
|
||||
- **タグ抽出**: general(0)/style(1)/copyright(3)/character(4)のみ採用（meta/ratingは除外）、確率の高い順。推奨しきい値は general 0.17 / style 0.15 / copyright 0.24 / character 0.27。UIのThreshold(general,style)とChar Thr.(copyright,character)は推奨値を基準にした相対指定（スライダー=0.17/0.27で推奨値そのもの）。
|
||||
- **UI**（`tagger-tab.js`）: PixAIモデル選択でスライダーが0.17/0.27へ、WD系へ戻すと0.35/0.85へ切替（Single/Batch両方）。手動で変えた値は上書きしない。
|
||||
- **性能**: CUDA(onnxruntime-gpu)で約0.4秒/枚（FP16・1008px）、CPUでは約21秒/枚。
|
||||
- 教訓: `onnxruntime`と`onnxruntime-gpu`が同じvenvに共存すると、後から入ったCPU版が同じ`onnxruntime/`フォルダを上書きしてCUDAが無言で無効になる（`get_available_providers()`にCUDAが出ない）。insightface/rtmlib/qwen-tts/onnxruntime-genaiが`onnxruntime`を要求するため再発しやすい。直し方は両方uninstallして`onnxruntime-gpu`のみ入れ直し。
|
||||
- ヘルプ（helpTagger1/2/4、日英中）を更新。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.9（2026-10-20）
|
||||
|
||||
### Video: アセット登録ボタン / Editの音声変換 / ソースパネルのレイアウト
|
||||
|
||||
- **アセット登録**: 動画ソースパネルに「アセット登録」ボタンを追加（`video-tab.js`）。ローカルファイルは`comfyUI.uploadImage`でinputへ上げ、バックエンド`POST /api/wfm/video/asset/register`（`VideoService.register_asset`）でoutput/videoへコピーしてから`__Video Assets__`グループへ追加する。Galleryはoutputフォルダしか一覧しないためコピーが必要。すでにoutput配下のソース（Asset選択・生成結果）はコピーせずタグ付けのみ。同名はアンダースコア+連番で回避。登録後はAssetタブを更新。
|
||||
- **Editの音声変換**: クロップ行のボタンをクロップ関連（編集/キャンセル/リセット）左寄せ→`|`区切り→「音声変換...」の並びに変更。Galleryの変換モーダルを`openAudioConvertDialog()`として共通化・export（`convertOne`コールバック方式。`allowDelete`/`allowCopyMeta`で項目の出し分け）。Gallery側の挙動は不変。Editのクリップはinputフォルダにあるため`POST /api/wfm/video/edit/convert-audio`（`VideoService.convert_audio_to_output`、PyAVを`asyncio.to_thread`で実行）がoutput/videoへ保存する。音声付き動画クリップのみ対象、読み込み中のクリップはトーストで案内。
|
||||
- **ソースパネルのレイアウト**: パネル幅を220→320pxに拡大（左右プレビューはflex:1で自動的に縮む）、ファイル名を専用の1行（省略表示）にし、ボタン3つを折り返さず横一列に。
|
||||
- ヘルプ（helpVideo5/helpVideo11、日英中）を更新。
|
||||
- 教訓: i18n.jsの英語文字列内に`"`を入れると全体が構文エラーになる（`node --check`は.mjsコピーで確認）。
|
||||
|
||||
## v0.8.8（2026-10-19）
|
||||
|
||||
### MOSS-Image 対応（GenerateUI / Gallery / Metadata / Iタブ）
|
||||
|
||||
ユーザー提供の5本（公式テンプレートのBase/SFT × T2I/バウンディングボックス4本と、公式ではない自作のmanga 8step LoRA構成1本）で、`convertUiToApi`→`analyzeWorkflow`を実測して直した。
|
||||
|
||||
- **サブグラフの外側ウィジェット値のズレ**: 新フロントエンドは外側ノードに`widgets_values_named`（名前キーの値）も保存し、`inputs[]`にはseed/steps/cfg/sampler/schedulerが載らない一方`widgets_values`は全ポートを持つ。従来の「`inputs[]`内の位置」参照だとunet_nameに25、vae_nameに`er_sde`が入っていた。`_flattenSubgraphs()`で`widgets_values_named`を優先（無い場合は従来どおり）。Iタブ用の複製（`node_sets_menu.js`の`_wfmFlattenSubgraphs`）にも同じ修正。
|
||||
- **`CreateBoundingBoxes`の変換**: `bboxes`の型`BOUNDING_BOX,ARRAY,STRING`が共用体の途中にSTRINGを含むため入力欄と誤判定され、本来の入力欄`editor_state`/`last_incoming`（型`BOUNDING_BOXES`）が読まれていなかった。共用体型は先頭セグメントだけで判定し、`BOUNDING_BOXES`を入力欄型に追加。配列値はAPIでリンク扱いされるため`{"__value__": [...]}`で送る（バックエンドが展開）。Flux.3 Klein 4種にも同ノードがあり、変換前は検証エラーになる出力だったものが直った。
|
||||
- **プロンプト検出**: `MOSSImageLayoutPrompt`の`prompt`をpositiveとして検出（`textKey: prompt`）。リンクされた`PrimitiveStringMultiline`/`PrimitiveString`はロールを上流へ伝播し、リンクのままの値は編集対象にしない（従来は配列がtextに入っていた）。
|
||||
- **Gallery Promptタブ**: APIの`prompt`チャンクが無くサブグラフ入りのUI形式のみの画像は、変換してから抽出（Metadataタブと同じ扱い）。通常の生成画像は従来どおりAPI形式を使う。
|
||||
- 実機(8189)で公式bboxes版と自作manga版の生成成功を確認。ワークフロー357件で変換結果を新旧比較し、差分はFlux.3 Klein 4件のCreateBoundingBoxesのみ。
|
||||
|
||||
### 変換チェック（GenerateUI Check）の初回待ち時間を短縮
|
||||
|
||||
初回に`/object_info`全件（この環境で約26MB・約43秒、毎回）を取得していたのが原因。ワークフロー詳細を開いた時点で全件の先読みも走らせており、サーバーを長時間塞いでいた。
|
||||
|
||||
- `_loadObjectInfoFor(types)`: 使用ノード型（サブグラフ内も含む）だけを`/object_info/<型>`で取得（1型約20ms、16並列、型ごとにキャッシュ）。転送失敗時や、既知の型が1つも無いときは全件にフォールバック。全件キャッシュがあればそれを優先。`invalidateObjectInfoCache()`は型別キャッシュも消す。
|
||||
- 変換（`_convertUiToApiImpl`）と互換性チェックで使用。GenerateUIの初回変換も速くなる。先読みは削除。
|
||||
- MOSSで初回変換 約43秒→約250ms、2回目以降は約2ms。362ワークフローで全件版との変換結果が完全一致。
|
||||
|
||||
## v0.8.7（2026-10-16）
|
||||
|
||||
### Qwen Image 2.1 Turbo（ComfyUI公式テンプレート T2I / I2I）対応
|
||||
|
||||
テンプレート（`SamplerCustomAdvanced`+`ManualSigmas`+`CFGGuider`、プロンプトエンハンサー=`TextGenerate`+`ComfySwitchNode`構成）をGenerateUIで使えるようにした。
|
||||
|
||||
- **Positiveプロンプトが空になる問題**: プロンプトが`PreviewAny → ComfySwitchNode(on_false=文字列リテラル)`経由で`TextEncodeQwenImage21.prompt`に入るため検出できなかった。`comfyui-workflow.js`のPass 1で`TextEncodeQwenImage21`のリンクされた`prompt`/`negative_prompt`へロールを伝播し、Pass 2で文字列リテラルを持つ`ComfySwitchNode`を`textKey: "on_false"`のプロンプトノードとして登録。
|
||||
- **サブグラフ展開のファンアウト**: `_flattenSubgraphs()`は外部リンクをポートの最初の接続先にしか張らなかったため、`images.image_1`がTextEncodeとプロンプトエンハンサー用`BatchImagesNode`の2か所へ分岐するI2Iで、後者が画像未接続になっていた。2番目以降の接続先にも同じ外部リンクを複製するよう修正。
|
||||
- 実機(8189)でT2I/I2Iとも生成成功を確認。
|
||||
- プロンプトエンハンサー対応: `TextGenerate.prompt`とスイッチ`on_false`が展開後に別々のリテラルになる問題を、`_unifyPromptEnhancerText()`で共有の`PrimitiveStringMultiline`（`<switchId>:wfs_prompt`）へ集約して解消。GenerateUIの入力欄の編集がエンハンサー入力にも反映される。I2Iでエンハンサーをオンにして実機確認（入力プロンプトと参照画像の両方がエンハンサーに届く）。
|
||||
|
||||
### Flux.3 Image Klein（ComfyUI公式テンプレート 4種）対応
|
||||
|
||||
Klein（cfg 3.5/25step）とKlein SD（蒸留、cfg 1/16step）それぞれのT2I（プロンプトエンハンサー付きサブグラフ）とImage Edit（`ReferenceMultiLatent`）。`CLIPLoader`のtype=`flux3`は既存の変換でそのまま通る。
|
||||
|
||||
- **`ReferenceMultiLatent`のロール伝播**: positive/negative入力が別々の出力に出るノードなので、`InstructPixToPixConditioning`と同様にロールを各入力へ個別に伝播。従来はEdit版のPositive/Negativeが両方とも"Positive Prompt"というタイトルだったため、Negative側もpositiveと誤判定されていた。
|
||||
- **未使用の空Latent**: `ReferenceMultiLatent`は参照画像サイズの`empty_latent`を自前で出力するため、テンプレートに残っている未接続の`EmptyFlux2LatentImage`はSettingsのLatent Imageに出さない（`ReferenceMultiLatent`を含むワークフローに限定）。
|
||||
- T2I側はQwen Image 2.1 Turboと同様、エンハンサーの`TextGenerate.prompt`が入力用`PrimitiveStringMultiline`に直結しているため、そのまま編集が両方に反映される。SD版T2I(SD_4)はテンプレート既定でエンハンサーがオン。
|
||||
- **COMBO値の誤置換**: テンプレートは`unet_name`をフォルダ接頭辞なし（`flux3_image_klein_int8_convrot.safetensors`）で保存しているが、実機の選択肢は`diffusion_models\flux3_...`。従来の「選択肢に無い値はchoices[0]へ無警告で差し替え」により、無関係なモデルに差し替わって実行されていた。選択肢に無い場合はまずファイル名（区切り文字`\`/`/`の後ろ）が一致する選択肢が1つだけあればそれを採用してから、従来のフォールバックに回すよう変更（`_convertUiToApiImpl`のCOMBO検証）。
|
||||
- 実機で4種とも生成成功（エンハンサーはオフ）。Edit版は参照画像の編集が効くことを画像で確認。
|
||||
|
||||
## v0.8.6（2026-10-14）
|
||||
|
||||
### GenerateUI: パートナーAPIノード（Nano Banana 等）の実行に対応
|
||||
|
||||
v0.8.5 で見送った課題。パートナーAPIノードは `/prompt` の `extra_data` に Comfy の認証情報（`auth_token_comfy_org` = ブラウザのログイン、または `api_key_comfy_org` = Comfy APIキー）が必要。ComfyUI 本体の画面は自動で付けるが、WFS の `queuePrompt`（`fetch('/prompt')` 直叩き）は付けないため、GenerateUI からは `Unauthorized: Please login first to use this node.` で失敗していた。
|
||||
|
||||
- **`py/routes/partner_routes.py`（新規）**: `GET /api/wfm/partner/status`（キー設定の有無のみ）、`POST /api/wfm/partner/prompt`（`/prompt` の中継）。`.env` の `COMFY_API_KEY`（prestartup_script.py が python-dotenv で読み込み）が設定されているとき、プロンプトに API ノード（`/object_info` の `api_node: true`、クラスごとにキャッシュ）が含まれる場合だけ `extra_data.api_key_comfy_org` を付与して自 ComfyUI の `/prompt` へ中継し、応答はそのまま返す。クライアントが既に認証情報を付けている場合は上書きしない。キーはブラウザへ出さない。`.env.example` に追記。
|
||||
- **`comfyui-client.js`**: `queuePrompt` は、キー設定あり（`isPartnerKeyConfigured()`、60秒キャッシュ）かつ接続先が同一オリジンのときだけ中継を使う（別サーバーへキーを送らない）。`execution_error` が `Unauthorized` / `login first` のときは、設定方法を案内する文言（`partnerAuthHint`、3言語）を付ける。
|
||||
- **`comfyui-workflow.js`**: `Gemini(Image|NanoBanana)*` ノードの `prompt` 入力を `prompt_nodes`（positive、`textKey: "prompt"`）に追加し、GenerateUI の Prompt タブで編集できるようにした。参考画像は従来どおり Image タブの `LoadImage` 経由。
|
||||
- ヘルプ（`helpGen32`、`index.html` / `i18n.js` 3言語 / `app.js` の `helpIdMap`）、README、サンプルワークフロー `workflows/ws_nano_banana_2_1_t2i.json` を追加。
|
||||
|
||||
### UI形式→API形式の変換: DynamicCombo ノードの `widgets_values` ずれを修正
|
||||
|
||||
Nano Banana 2.1 の UI 形式ワークフローを変換すると値がずれていた（`seed` が `"IMAGE"`、システムプロンプトが `response_modalities` に入る等）。原因は2つで、どちらも修正した。
|
||||
|
||||
1. `_getDynamicComboSubNames` が、選択肢の子入力のうちソケット（`images` = Autogrow、`files` = `GEMINI_INPUT_FILES`）まで `widgets_values` の1枠として数えていた。ウィジェット型の子入力だけを数えるようにした（`_resolveWidgetType` を再利用）。
|
||||
2. `seed` 等の `control_after_generate` 付きウィジェットの直後に入るフロントエンド専用の `"randomize"` 等の1枠を、次のウィジェットが数値型のときしか読み飛ばせなかった（Nano Banana 2.1 は `seed` の次が COMBO の `response_modalities`）。object_info の `control_after_generate` フラグで判定する `_hasControlAfterGenerate` を追加。
|
||||
|
||||
- 検証: ComfyUI 本体のフロントエンドで作ったワークフローの `graphToPrompt()` 出力と、変換結果が完全に一致。実機の保存済み UI 形式ワークフロー301件を修正前後で比較し、300件は変換結果が同一、変化は今回の Nano Banana サンプルのみ。
|
||||
|
||||
### Gallery: `cc_nanobanana` フォルダの画像でもメタデータがあれば Prompt タブを表示
|
||||
|
||||
Comic Creator v1.55.0 で、半自動マンガがネイティブノード経由で生成した画像が `cc_nanobanana` に保存され、PNG に `prompt` / `workflow` が埋め込まれるようになるため。`gallery-tab.js`: 埋め込みワークフローがある画像は `isNanobananaFolderImage` による Prompt タブの非表示を解除する（埋め込みの無い Gemini API 直叩きの画像だけ従来どおり非表示）。
|
||||
|
||||
**実機確認（ComfyUI_5 / ポート8189、v0.40.0）**
|
||||
- `.env` に `COMFY_API_KEY` を設定し、GenerateUI でサンプル（Nano Banana 2.1）を読み込み、Prompt タブで編集して Apply → Generate を実クリック: リクエストは `/api/wfm/partner/prompt` 経由、16.4秒で成功、編集どおりの絵（青いティーポット）が `output/nano_banana_00001_.png` に出力された。
|
||||
- キー未設定時の案内文（`partnerAuthHint`）が表示用文言として解決されること、ヘルプ（`helpGen32`）が表示されることを確認。
|
||||
|
||||
### 今後の作業候補（2026-10-14 時点・未着手。v0.8.4〜v0.8.6 の続き）
|
||||
|
||||
**ネイティブ LoRA スタックノード（`LoadLoraModel` / `LoadLoraTextEncoder`、v0.8.4）**
|
||||
- **UI形式への対応**: ComfyUI フロントエンド 1.55.17 は DynamicGroup を実装していない（行を追加できず `widgets_values` も空）。フロントエンドが対応したら、実物の UI 形式 JSON で `widgets_values` の並びを確認し、`analyzeWorkflow` / Metadata / I タブの UI 形式分岐を追加する。
|
||||
- **画面操作の確認**: GenerateUI の Stack タブ（Stack グループを複数行として書き込む処理）は、関数の直接呼び出しでのみ確認済み。Models タブで Stack グループを登録した状態での画面確認が未実施。
|
||||
|
||||
**Gemini / Nano Banana 系ノード（v0.8.5 / v0.8.6）**
|
||||
- **Metadata / I タブの表示拡充**: プロンプトは Positive / Negative に分けず全文を1件で表示している（送信全文に `NEGATIVE_PROMPT:` が含まれる）。`NEGATIVE_PROMPT:` で分割して Positive / Negative に振り分ける、`aspect_ratio` / `resolution` / `thinking_level` を設定欄に出す、UI 形式の `widgets_values`（並びは `[prompt, model, aspect_ratio, resolution, thinking_level, seed, control_after_generate, response_modalities, system_prompt, temperature, top_p]`）に対応する、が候補。
|
||||
- **他のパートナーAPIノードの Prompt タブ対応**: 現状は `Gemini(Image|NanoBanana)*` のみを `prompt_nodes` に登録している。`/object_info` の `api_node: true` かつ `prompt` 入力を持つノード（Seedream / Kling 等）へ一般化できないか検討する。
|
||||
- **費用ガード**: Batch / Lab / カタログ作成などの繰り返し実行でパートナーAPIノードを使うと、回数ぶんのクレジットが連続して消費される。`/object_info` の `price_badge` から概算を出して、実行前に表示・確認する案。
|
||||
- **Settings タブに `COMFY_API_KEY` の設定状態を表示**: 現状は `.env` 編集のみ。`GET /api/wfm/partner/status`（設定の有無のみ返す）を使って、設定済み／未設定と設定手順を Settings タブに出す。キー自体は表示・入力させない方針を維持する。
|
||||
- **サンプルの追加**: 参考画像付き（I2I、`LoadImage` → `model.images.image_1`）の Nano Banana ワークフロー、複数参考画像のもの。Nano Banana Pro（`GeminiImage2Node`）は Comic Creator 側のネイティブ対応と合わせて。
|
||||
|
||||
**運用・開発環境**
|
||||
- **同期スクリプトの整備**: 開発リポジトリから実機の `custom_nodes` への反映は、今回も手作業（追跡ファイルの `cmp` 比較 + `cp`、`workflows/` / `data/` / `.env` は除外）だった。CLAUDE.md の「同期スクリプトでパッケージ全体をデプロイ」に沿って、`tools/` に同期スクリプト（除外リスト付き、ハッシュ比較・`__pycache__` 削除まで）を用意する。部分同期が残ると古い版が混在する（今回、実機は v0.8.2 開発途中の版のまま止まっていた）。
|
||||
- **comfy-cli / comfy-mcp の既定ワークスペース**が ComfyUI_6（プラグイン 0.3.78）で、実際に使っている ComfyUI_5（8189）と一致しない。`server_info` の `running` / `freshness` は ComfyUI_6 の情報になるため、MCP で実機を扱うときは注意するか、既定ワークスペースを揃える。
|
||||
- **英語・中国語UIでの表示確認**: 今回追加したヘルプ（`helpGen32` ほか）と `partnerAuthHint` は日本語UIでのみ画面確認した。
|
||||
|
||||
## v0.8.5（2026-10-14）
|
||||
|
||||
### Metadata / I タブ: Gemini 系ノード（Nano Banana）の画像を読めるように
|
||||
|
||||
Comic Creator v1.54.0 で、Nano Banana を ComfyUI ネイティブノード（`GeminiNanoBanana2V2`）経由で生成できるようにした（併用）。この経路の画像は SaveImage により `prompt` / `workflow` が PNG に埋め込まれるため、Workflow Studio 側で通常の ComfyUI 画像として読めるようにした。
|
||||
|
||||
- `metadata-tab.js` / `web/comfyui/node_sets_menu.js`: 自ノードの `prompt` 入力を持つ `Gemini(Image|NanoBanana)*` ノードを、動画ノード（`*ImageToVideo` / `*TextToVideo`）と同じ扱いでプロンプト候補（`texts`）へ追加。`NEGATIVE_PROMPT:` を含む送信全文がそのまま1件として出る（Positive / Negative の分割はしない）。
|
||||
- `extractDiffusionModels`（API形式）: Gemini 系ノードの `model` 入力（例: `Gemini Nano Banana 2.1`）をモデル名として表示。
|
||||
- `extractSamplerSettings`: Gemini 系ノードの `seed` を補助ノードとして拾い、設定欄に表示。
|
||||
- 対象は API 形式のみ（DynamicCombo のため UI 形式の `widgets_values` の並びは未確認）。`GenerateUI` のプロンプト編集・実行は対象外。
|
||||
- `cc_nanobanana` フォルダ用の特別扱い（`history.jsonl` 参照・Promptタブ非表示）は、従来の Gemini API 直叩きの画像のため変更なし。ネイティブ経路の出力先は別フォルダ（`cc_nanobanana_native`）なので、この特別扱いに該当せず、埋め込みメタデータが使われる。
|
||||
|
||||
**実機確認（ComfyUI_5 / ポート8189、v0.40.0）**
|
||||
- Comic Creator のネイティブ経路で実際に生成した画像（`cc_nanobanana_native/NB_00001_.png`、848×1264）を `extractAllMetadata` に通し、モデル名（`Gemini Nano Banana 2.1`）・シード・プロンプト（`NEGATIVE_PROMPT:` 付き全文）が取れることを確認。Gallery の一覧・画像メタ取得でも通常の画像として扱われる（`nanobanana` エントリは null）。
|
||||
|
||||
**ネイティブ API ノードの認証について（今回の調査で判明）**
|
||||
- API ノードは `/prompt` の `extra_data` に `auth_token_comfy_org`（ブラウザのログイン）または `api_key_comfy_org`（Comfy APIキー）が必要。ComfyUI 本体の画面は自動で付けるが、Workflow Studio の `queuePrompt`（`fetch('/prompt')` 直叩き）は付けないため、GenerateUI からパートナーAPIノードを実行すると認証エラーになる。今回は対応せず（Comic Creator はサーバー側で `COMFY_API_KEY` を渡す方式で対応）。必要になれば Workflow Studio 側にも同様のサーバー側プロキシが必要。
|
||||
|
||||
## v0.8.4（2026-10-14）
|
||||
|
||||
### ComfyUI v0.40.0 対応: ネイティブ LoRA スタックノード（`LoadLoraModel` / `LoadLoraTextEncoder`）
|
||||
|
||||
ComfyUI v0.40.0 の変更点を確認し、影響のあった LoRA 検出・編集を対応させた（コミット 5078dd7）。
|
||||
|
||||
**背景・調査結果**
|
||||
- 新ノードは `comfy_extras/nodes_lora_stack.py`。1ノードに DynamicGroup `loras`（1〜20行）を持ち、API形式では `loras.<行>.lora_name|strength|enabled` のドット結合キーで渡す。Model用とText Encoder用で別ノード。
|
||||
- Workflow Studio の LoRA 検出は固定リスト（`LoraLoader` / `LoraLoaderModelOnly` / Power Lora / LoraManager / ImageMetadataLoRALoader）だったため新ノードは未検出。また既存の編集コードが `inputs.lora_name` を書き込むため、そのまま使うと DynamicGroup の検証エラーになる状態だった。
|
||||
- **ComfyUI フロントエンド 1.55.17 には DynamicGroup の実装が無い**（型名ラベルのみ）。GUI のグラフエディタではこのノードに行を追加できず、UI形式の `widgets_values` も空になる。そのため対応は **API形式のみ**。UI形式の対応はフロントエンドの対応後に実物のJSONで再検証が必要。
|
||||
|
||||
**実装**
|
||||
- `comfyui-workflow.js`: `analyzeWorkflow` が2ノードを `is_lora_stack` として `lora_nodes` に追加（1ノード1エントリ、代表は有効な最初の行。無効行は飛ばし、行番号順に解釈）。
|
||||
- `comfyui-editor.js`: 共通の書き込み処理 `applyLoraStack`（`comfyEditor` から公開）を追加。ワークフロー内の全スタックノードを同期して書き込む（Modelノード=Str M、Text Encoderノード=Str C）。Single / Stack Apply（Stackは全モデルを行として書く、最大20行）、バイパスボタン（全行のstrengthを0、解除で行ごと復元）、LoRAペインの現在値表示に対応。
|
||||
- `lab-tab.js` / `models-tab.js` / `generate-tab.js`: Lab、Models タブの Apply、LoRA バッチも同じ処理経由に。Models Apply は既存の強度を保持する。
|
||||
- `metadata-tab.js` / `web/comfyui/node_sets_menu.js`: Metadata タブ・サイドパネル I タブの LoRA 抽出（API形式）に対応。Model/Text Encoder 両ノードの同名LoRAは1件にまとめ、両方の強度を表示する。
|
||||
- ヘルプ（helpGen20 / helpGen21 / helpGen29 / helpSidepanel15、en/ja/zh）と index.html を更新。README に項目を追加。
|
||||
|
||||
**実機確認（ComfyUI_5 / ポート8189、v0.40.0）**
|
||||
- テスト用 API 形式ワークフロー（Checkpoint → Load LoRA 2ノード×2行 → SDXL）を `/prompt` へ投入: HTTP 200・`node_errors` 空・1枚生成成功（18秒）。
|
||||
- 生成時ログの `lora key not loaded: ...`（約5千行）は、Model/Text Encoder で別ノードに分かれる設計上、各ノードが対象外キーを報告する無害なノイズ（従来 `LoraLoader` 単独では0件、件数も2LoRA×キー数と一致）。
|
||||
- GenerateUI 画面で、ID:2/ID:3 の選択、バイパスボタンの往復（0→元の値）、Single Apply（RAW JSON・LORA SYNTAX・トリガーワード更新）を実クリックで確認。
|
||||
- 書き込み処理は実ファイルから切り出して Node で単体試験（強度維持・行入替・両ノード同期・バイパス復元・ワークフロー引数指定）。
|
||||
|
||||
**影響なしと確認した項目**
|
||||
- WebSocket のメタデータ付与: メタデータは `workflow_metadata` の送信または機能フラグ `supports_preview_metadata` の宣言時のみ付く。本プラグインのクライアントはどちらも行わず、メッセージも既知フィールドだけを `try/catch` 付きで読むため影響なし。
|
||||
- 3D 出力・カメラ系ノード: 結果取得は `images` と `saved_svg` のみを見るため無視されるだけ。
|
||||
- Comic Creator（v1.53.2）: LoRA / WebSocket / `/prompt` を使わず、`/upload/image` の応答形式も互換。`/ccc` と `/wfm` の iframe 連携も正常。
|
||||
|
||||
**その他（運用上の発見）**
|
||||
- ランタイム（ComfyUI_5 の `custom_nodes`）が v0.8.2 開発途中の版のまま古いファイルが残っていた（部分同期）。リポジトリの追跡ファイルを同期し直した（`workflows/`・`data/`・`.env` は上書きしない）。
|
||||
- comfy-cli の既定ワークスペースは ComfyUI_6（プラグイン 0.3.78）。comfy-mcp の `server_info` の running / freshness は 8189 の ComfyUI_5 とは別物。
|
||||
|
||||
## v0.8.3（2026-10-13）
|
||||
|
||||
### GenerateUI: シードモード「Workflow」を追加（改善候補メモ 2-A）
|
||||
|
||||
生成パネルのSeed欄が常に優先され、ギャラリーから読み込んだ画像を元のシードで再現しにくかった問題への対応。
|
||||
|
||||
- `templates/index.html`: `#wfm-gen-seed-mode` に `workflow` を追加（Random / Fixed / Increment / Decrement / Workflow）。
|
||||
- `comfyui-client.js`: `getWorkflowSeed()`（API形式ワークフローの最初のリテラルな `seed` / `noise_seed` を返す。リンク入力・負値は無視）を追加し、`generate()` が `seedMode === "workflow"` のときシードを上書きせずワークフロー値を使うようにした。数値のシード入力が無いワークフローではパネル値（-1ならランダム）にフォールバック。生成後はSeed欄に実際に使った値が入る。
|
||||
- 副次的に見つけた不具合の修正: 呼び出し側が渡す `{ ...workflow }` は浅いコピーでノードオブジェクトを `currentWorkflow` と共有していたため、Random生成のシード上書きが読み込み済みワークフローに漏れ、その後のWorkflowモードが元のシードではなく直前のランダム値を使っていた。`generate()` 冒頭で `structuredClone(workflow)` するようにした。
|
||||
- 実機確認（ComfyUI_5 / chat_t2i_sdxl）: Random→Workflowの順でも元のシード（1097716508243681）で生成され、本体のシードは変化しないことを確認。同じシード・同じ設定の連続生成はComfyUIのキャッシュで新規画像が増えない（仕様）。
|
||||
- ヘルプ（helpGen8、en/ja/zh）とindex.htmlのヘルプ行にWorkflowモードを追記。
|
||||
|
||||
### サンプルワークフロー: Gallery Feeder用I2I（改善候補メモ 7）
|
||||
|
||||
- 追加: `workflows/ws_sdxl_gallery_feeder_i2i.json`（+ `.png`）。既存の `ws_sdxl_image_feeder_i2i`（SDXL I2I）の ImageFeeder を `WFS_GalleryFeeder`（group_name=`__Feeder__` / sort_mode=filename_asc）に差し替えたもの。リンク・ノードIDは元サンプルと同じ構成で整合。サムネイルPNGは既存サンプルのものを流用（要差し替え）。
|
||||
|
||||
### ドキュメント: 初心者向けマニュアル（日本語・英語）
|
||||
|
||||
- `docs/manual/`（日本語）と `docs/manual/en/`（英語）に、v0.8.3対応・画像なしのテキスト版マニュアルを追加（章ごとのMarkdown、全章まとめMarkdown、配布用ZIP）。Kindle版（画像入りEPUB）は作者のリポジトリ外原稿から別途作成。
|
||||
- 内容はv0.7.13版から更新: Workflowシードモード、Gallery Feederサンプル、セマンティック検索（8-6）、音声ファイルの扱いと変換（8-7）、メタデータタブのレイアウト変更、WS Library NタブのAlt+クリックなど。Kindleでの文字化け対策として、U+2015等の記号を代替表記に置換。
|
||||
- 配布版の再生成スクリプト（`make_repo_version.py` / `make_repo_version_en.py`）はリポジトリ外のマニュアル作業フォルダにある。
|
||||
|
||||
## v0.8.2（2026-10-12）
|
||||
|
||||
### Gallery: セマンティック検索（Gemini Embedding 2 / EmbeddingGemma 2）
|
||||
|
||||
画像を文章の意味で探す検索と、選択画像に類似した画像の一覧表示を追加した。
|
||||
|
||||
**UI**
|
||||
- 検索ボックス横に「S」ボタン。ONで検索ボックスが意味検索になる（Enterで実行、スコア順、サムネイルにスコアバッジ）。OFFは従来のファイル名/タグ/プロンプト検索。
|
||||
- Sを入れるとツールバーに2行目が現れ、`Similar`（トグル）・`↻`（更新）・`⚙`（設定）・検索結果件数を左詰めで表示する。Similarは、類似一覧の中で別の画像を選んでも一覧はそのままで、↻でその画像の類似を再表示、Similar再押下で通常表示に戻る。フォルダ切替・フィルタクリア・S OFFでも解除。Similar/↻は当初詳細パネルに置いていたが2行目へ移設（Taggerの紫と区別するためティール）。
|
||||
- ⚙のモーダルを3タブ化: インデックス（対象フォルダ・サブフォルダ・新規のみ/全件再作成・進捗・停止・全削除）/ モデル設定（モデルのラジオボタン・次元・ローカルのデバイス/CPUスレッド/画像精度・接続テスト）/ 検索設定（類似画像と文章検索の最小スコア、最大表示件数）。「?」でヘルプの該当項目へジャンプ。
|
||||
- 最小スコア（例: 0.6以上）はモデルごとに保存（スコアの水準がモデルで違うため。少数画像の実測では、類似画像はEmbeddingGemma 2で約0.6〜0.8、Geminiで約0.6、文章検索はそれぞれ約0.6〜0.7・約0.3）。
|
||||
- フォルダツリーの枚数バッジの右に緑のドット = そのフォルダ直下の画像が全件インデックス済み（`GET /wfm/gallery/semantic/folders`）。インデックスは全フォルダ共通の1つで、フォルダ単位で少しずつ作れる。
|
||||
- ヘルプページの幅を760→1076px（上部タブバーのAI TOOLタブの右端に揃える）。
|
||||
|
||||
**バックエンド**
|
||||
- `py/services/semantic_service.py`: ベクトルを `semantic_index.npz` に保存（パス→正規化ベクトル、サイズ/mtimeで鮮度判定、モデル/次元/画像精度をヘッダに保持）。インデックス作成はバックグラウンドスレッド（進捗ポーリング・停止可）、検索はコサイン類似度（`mat @ q`）、未インデックスの基準画像はその場で1枚だけ埋め込む。動画/PSDはサムネイルから、音声は対象外。
|
||||
- Gemini: `gemini-embedding-2`（`x-goog-api-key`ヘッダ、接続先固定、429/5xxは指数バックオフで再試行）。APIキーは `.env` の `GEMINI_API_KEY`（UNSLOTHと同じく settings.json・フロントには出さない）。
|
||||
- EmbeddingGemma 2: `py/services/semantic_local.py`。onnx-community/embeddinggemma-2-ONNX を `onnxruntime` + `tokenizers` + `huggingface_hub` だけで実行（`transformers` の対応を待たない）。Transformers.jsの `Gemma4ImageProcessor` / `EmbeddingGemma2Processor` を移植（アスペクト比維持リサイズ → 16pxパッチ化 → vision_encoder → `<|image|>` 位置へ `image_features` を差し込み → `sentence_embedding`）。クエリは `task: search result | query: ` プレフィックス。MRL（128/256/512/768）で切り詰め＋再正規化。
|
||||
- 実行デバイス: 自動（CUDAがあればGPU）/ CPU / GPU(CUDA)。CPUスレッド数の既定はコアの半分。設定は `semantic_settings.json` に保存。画像精度はソフトトークン数 280/140/70（高精度/標準/軽量、CPUで約2.5倍/約5倍速）でインデックスに記録し、変えたら全件再作成が必要。インデックス作成後はセッションを解放しVRAM/RAMを画像生成に譲る。
|
||||
|
||||
**実測（RTX 4070 / 32コア）**
|
||||
- 768×576の画像1枚あたり: CUDA 0.13秒（標準 0.07秒）、CPU 4スレッド 5.8秒（標準 2.3秒、軽量 1.1秒）。CPUとCUDAのベクトルは一致（cos 1.0）。実機で122枚を約20秒で作成。
|
||||
- DirectMLはvision_encoderのReshapeノードで実行時エラーとなり非対応（テキストモデルのみ動作）。WindowsMLは未検証。
|
||||
- 環境メモ: ComfyUI_5のvenvには `onnxruntime` と `onnxruntime-gpu`（ともに1.30.0）が同居し、CPU版が勝っていたため、両方を削除して `onnxruntime-gpu==1.30.0` のみを入れ直した（`pip check` は `onnxruntime` 名での依存を警告するが動作に影響なし）。
|
||||
|
||||
**安全性（セキュリティレビュー指摘への対応）**
|
||||
- 数値パラメータ（dim / soft / threads / limit / min_score）を共通関数で範囲内に丸め、数値でない値は500ではなく400（`bad_param`）で返す。スレッド数はコア数の2倍まで。
|
||||
- ローカルモデルへの入力を制限: テキスト1024トークン、検索文2000文字、画像は長辺2048pxを超えたら先に縮小。
|
||||
|
||||
**その他**
|
||||
- `.env.example` に `GEMINI_API_KEY`、`requirements.txt` にGPU用の手順コメントを追加。ヘルプ（3言語）・READMEを更新。
|
||||
- Gemini Embedding 2には無料枠があるが、無料枠で送った内容はGoogleの製品改善に使われる場合がある（2026年10月時点の公式料金: 画像1枚あたり約$0.00012）。
|
||||
|
||||
## v0.8.1（2026-10-11）
|
||||
|
||||
v0.8.0 のコードレビュー指摘（#1〜#10）への対応。
|
||||
|
||||
### バグ修正
|
||||
- Nodes: 検索結果が0件のとき `renderToken` を進めておらず、描画途中のカードが「No nodes found」の下に追加され続けていた。
|
||||
- Metadata: `convertUiToApi()` に `{ readOnly: true }` を追加。表示用の変換ではCOMBO値の無警告差し替え（サンプラー名の誤表示）を行わず、Workflowタブの互換性チェックが参照する `_last*` 記録も更新しない。
|
||||
- Metadata: 画像を連続で開いたとき、先に始めた処理の結果が後から描画されて混ざる問題を、リクエストID（`_loadSeq`）で防止。
|
||||
- Metadata: サンプラー設定の抽出を改善。Flux/SD3系（RandomNoise / BasicScheduler / KSamplerSelect / CFGGuider）の設定を統合し、複数KSamplerでは設定が最も多いノード（同数なら後ろ）を採用。
|
||||
- Gallery: `history.jsonl` に dict でない行・`filenames` が list でない行があっても、フォルダ内の全画像でメタ取得が失敗しないようにした。
|
||||
- Gallery: 同じファイルを同時に変換すると出力名が衝突して上書きされる問題を、出力名の決定を `_convert_lock` 内に移して解消。
|
||||
- Gallery: PNGチャンクの読み取りを先勝ちに統一（`png_extractor` と同じ）。ワークフロー読み込みとの食い違いを防止。
|
||||
|
||||
### パフォーマンス・保守性
|
||||
- Gallery: 一括選択の変換対象判定を O(選択数×画像数) から O(選択数) に（パス→画像Mapのキャッシュ + `some` で打ち切り）。12,780枚の全選択で約33ms。
|
||||
- Gallery: `history.jsonl` を `(mtime_ns, size)` キーの索引としてキャッシュ（`OrderedDict`、上限16件）。
|
||||
- 音声・動画の拡張子判定を `isAudioFile()` / `isVideoFile()` に統一し、波形サムネイルの版数を `WAVE_THUMB_VERSION` 定数に集約（JS/Python）。
|
||||
|
||||
## v0.8.0（2026-10-11）
|
||||
|
||||
### Gallery: 音声ファイル対応（wav / mp3 / ogg / m4a / flac / opus）
|
||||
|
||||
ComfyUIの音楽生成の増加に合わせ、Galleryで音声ファイルを画像・動画と同じ一覧で扱えるようにした。
|
||||
|
||||
- `gallery_service.py`: `AUDIO_EXTENSIONS` を追加し `IMAGE_EXTENSIONS` に合流。再生時間は`_read_media_dimensions()`（PyAV）、埋め込みタグは`_read_audio_metadata()`で取得（ストリーム側タグも統合し、`prompt`/`workflow`は小文字キーでも引ける）。`_extract_embedded_workflow()`も音声に対応し、Prompt/JSONタブが動く。
|
||||
- サムネイルは`_render_audio_waveform()`でPyAVデコード→8kHzモノラル→列ごとの最大振幅を描画してJPEGキャッシュ。拡張子ごとに波形色を変える（`AUDIO_WAVE_COLORS`）。キャッシュキーに版数（`wave2`）、サムネイルURLにも`&v=wave2`を付けブラウザの24hキャッシュを回避。
|
||||
- `gallery_routes.py`: Windowsで音声MIMEが返らない/不統一なため`mimetypes.add_type`で明示登録。`FileResponse`のRange対応でシークも可能。
|
||||
- `gallery-tab.js`: `isAudioFile()`、♪バッジ、詳細パネル/ライトボックス/Compareで「波形＋`<audio controls>`」を表示（音量は動画と共通設定）。詳細パネルは音声のとき固定高さをやめ、再生メニューが見切れないようにした。PSD書き出し・Edit as Layersでは音声を除外。
|
||||
|
||||
### Video: Assetサブタブで音声を素材として利用
|
||||
|
||||
- 先に音声を一覧から除外していたのを改め、種別フィルタに「Audio Only」を追加。音声はプレイヤー付きで、「BGMに設定」「サウンドとして追加（Audioトラックの選択クリップ先頭に配置）」ボタンが使える。`video-edit-tab.js`に`addSoundFromFile()`を追加。Editに送る/オーバーレイ/First・Last Imageと、Frame/GIFツールへのSourceプレビュー投入は音声では出さない。
|
||||
|
||||
### Gallery: ファイル変換（音声変換 / 動画からの音声抽出）
|
||||
|
||||
- 新規`py/services/convert_service.py`（PyAVのみ、システムffmpeg不要）。出力はFLAC / MP3 / Opus / M4A / WAV（Vorbisエンコーダが無いため`.ogg`は入力のみ）。ビットレート・サンプルレート・チャンネル指定可。
|
||||
- `GalleryService.convert_file()` + `POST /wfm/gallery/convert`（1ファイルずつ呼ぶ。`run_in_executor`＋同時1本のロック）。同じフォルダに`name.ext`（既存なら`name_converted.ext`、以降連番）で保存し上書きしない。
|
||||
- メタデータ: `prompt`/`workflow`と曲情報タグを、タグを持てる形式（FLAC/MP3/Opus/M4A）へ引き継ぐ。WAVは対象外（通知）。お気に入り/タグ/メモ/グループの引き継ぎ、変換成功後の元ファイル削除（確認ダイアログ付き）を選択可。
|
||||
- UI: 一括バーに「変換...」ボタン（音声/動画を含む選択時のみ表示）とダイアログ。i18n（en/ja/zh）対応。
|
||||
|
||||
### Gallery: メタデータ読み取りの修正（メタデータ抽出 比較レポートA1〜A4）
|
||||
|
||||
- A1: PNGの読み取りがIENDで止まっていたため、ComfyUI-Custom-ScriptsがIENDの後ろに追記する`workflow`チャンクを読めなかった（`_read_png_metadata()` / `extract_png_workflow()`）。
|
||||
- A2: WebPのEXIFから`workflow:`/`prompt:`を取り出す`_read_webp_metadata()`を追加。
|
||||
- A3: ワークフローが無い画像（A1111/Forge/Fooocus）は`parameters`からPromptタブを表示（`extractPromptsFromParameters()`）。
|
||||
- A4: API形式の`ImageMetadataLoRALoader`（`lora_N`/`strength_model_N`/`strength_clip_N`）をMetadataタブとサイドパネルIタブで検出。
|
||||
|
||||
### Library/Nodes: 表示速度の改善とAlt+クリックお気に入り
|
||||
|
||||
- LibraryのNタブ: Alt+クリックでお気に入りをON/OFF。`/object_info`（数MB）の取得を待たずに一覧表示し、Category/Packageは`LiteGraph.registered_node_types[..].nodeData`から取得（無い環境のみ裏で取得）。
|
||||
- SPAのNodesタブ: 数千件を一括でDOM生成していたのを分割描画に変更し、検索をデバウンス＋検索文字列の事前計算で軽量化。さらに`/api/wfm/nodes/light`（`INPUT_TYPES`を呼ばず約6ms）で一覧を先に表示し、入力定義を含む`/object_info`（実測で約14秒）は裏で取得して補完する方式にした。
|
||||
|
||||
### Gallery: Move Toダイアログにフォルダ絞り込み
|
||||
|
||||
- フォルダ名/パスの部分一致（空白区切りAND）で絞り込み、Enterで先頭候補へ移動。音声/動画も移動できるため、文言を「N件移動しました」に変更（`movedNImages`、`moveToFolderTitle`）。
|
||||
|
||||
### その他
|
||||
|
||||
- ヘルプ（en/ja/zh）とREADMEを更新。READMEのバージョンバッジが0.7.13のままだったのを修正。
|
||||
|
||||
---
|
||||
|
||||
## v0.7.14（2026-10-09）
|
||||
|
||||
### Comic Creator連携: nanobananaタブの生成履歴をGallery/Metadataタブに表示
|
||||
|
||||
Comic Creator（`eagle_comic_creator_spa`）のnanobananaタブが`cc_nanobanana`フォルダに書く生成履歴`history.jsonl`（1行=1生成リクエスト、`filenames`配列で画像ファイルと対応、prompt/negative_prompt/engine/model/image_size/seed/timestampを記録）を、WFS側のGallery/Metadataタブから閲覧できるようにした。
|
||||
|
||||
- `gallery_service.py`に`_read_nanobanana_entry()`を追加。画像の親フォルダ名が`cc_nanobanana`の場合のみ同フォルダの`history.jsonl`を新しい行から探索し、自分のファイル名を含むエントリを返す（既存の`get_image_metadata()`レスポンスに`nanobanana`フィールドとして追加、新規エンドポイントは不要）。
|
||||
- Gallery詳細パネルに「Nanobanana」タブを新設。`cc_nanobanana`フォルダの画像のみタブボタンを表示し、engine/model/size/seed/timestamp/prompt/negative promptを表示。
|
||||
- Metadataタブ右ペインにも同内容の「Nanobanana」セクションを追加。Galleryの「Metadata」ボタンから開いた場合は画像パスも渡されるようになり（`loadFileIntoMetadataTab(file, path)`）、サーバーの`/wfm/gallery/image/meta`から画像サイズとnanobanana履歴を補完する。nanobanana画像はComfyUIワークフローを埋め込んでいないため、従来は「メタデータなし」エラーになっていたが、nanobanana履歴があればエラーにせずNanobananaセクションのみ表示するようにした。
|
||||
- `cc_nanobanana`フォルダの画像はComfyUIワークフローを埋め込んでおらずPromptタブに表示する情報が無いため、Gallery詳細パネルのPromptタブ自体を非表示化（フォルダ名で判定、history.jsonlに対応エントリが無い古い画像でも同様に隠れる）。
|
||||
|
||||
### Metadataタブ: レイアウト再編（LoRA/Promptを中央へ、右ペインをSettings+Nanobananaに）
|
||||
|
||||
- 中央ペイン（Col2）: Checkpoint / VAE / Diffusion Model / Text Encoder に加えて LoRA / Prompt もここへ移動。
|
||||
- 右ペイン（Col3）: 「Settings」（画像サイズ、Seed、Steps、CFG、Sampler、Scheduler、Denoise）と、上記「Nanobanana」セクション。
|
||||
- KSampler系ノードの設定値を抽出する`extractSamplerSettings()`を新規実装。API形式ワークフローの名前付き`inputs`（`seed`/`noise_seed`/`steps`/`cfg`/`sampler_name`/`scheduler`/`denoise`）から読む。UI形式ワークフローは`convertUiToApi()`を経由して同じ抽出を通す（サブグラフの有無に関わらず常に変換を試みるよう変更。既存のCheckpoint/LoRA/Prompt抽出ロジックには影響なし、変換結果はSettings抽出にのみ使う）。
|
||||
- 画像サイズは、Galleryから開いた場合はサーバー側の実測値（`/wfm/gallery/image/meta`のwidth/height）、ドラッグ&ドロップされた単体ファイルの場合はプレビュー画像の`naturalWidth`/`naturalHeight`を使う。
|
||||
- i18n（en/ja/zh）・ヘルプカードを更新。
|
||||
|
||||
### バグ修正: LiveChatStreamなど複数出力テキストノードがAPI形式プロンプト抽出で解決できない不具合
|
||||
|
||||
ComfyUI-LiveChatStreamの`response`（LLM応答全文）出力をCLIPTextEncodeに直結した場合にGallery/Metadataタブでプロンプトが表示されない、という報告を調査。
|
||||
|
||||
- 原因: PNGの`prompt`（API形式）埋め込みからのプロンプト抽出は`prompt`チャンクを優先して使うが（Metadataタブ・Gallery Promptタブとも）、API形式の`resolveLinkedText()`は接続元ノードの値を決め打ちのキー名リスト（`text_positive`/`text`/`prompt`/`value`等）でしか探しておらず、LiveChatStreamの実際のウィジェット名（`prompt_text`/`negative_text`/`response_text`/`chat_p_text`/`chat_n_text`/`thinking_text`、`nodes.py`の`INPUT_TYPES`で確認）がどれとも一致しないため、**responseだけでなくpositive/negativeも含め、API形式経由では本来解決できていなかった**（UI形式の`workflow`埋め込みは`widgets_values`を出力スロット順でそのまま読むため、たまたま正しく解決できていた＝どちらの埋め込みがPNGにあるかで体感が変わっていたと推測）。
|
||||
- `metadata-tab.js`の`resolveLinkedText()`、およびサイドパネルIタブ（`node_sets_menu.js`）の複製ロジック`_resolveLinkedText()`の両方に、`class_type === "LiveChatStream"`の個別対応を追加（`LIVE_CHAT_STREAM_OUTPUT_KEYS`で出力スロット→ウィジェット名を対応付け）。
|
||||
- 実機（稼働中のComfyUI_5）の`/wfm_static/js/metadata-tab.js`に対し、約19,000〜40,000文字の長文応答を模したAPI形式ワークフローで検証し、全文正しく（0.4ms程度で）抽出されることを確認。文字数そのものは無関係だった。
|
||||
|
||||
### Gallery: Send to LV node ボタン（ネイティブLoad Videoへの送信）
|
||||
|
||||
既存の「Send to LI node」（選択画像をComfyUIキャンバス上の選択中ノードの`image`ウィジェットへ送る）の動画版。
|
||||
|
||||
- `node_sets_menu.js`に`_wfmFindVideoWidget()`/`window.wfmSendVideoToSelectedNode()`を追加。ネイティブの`LoadVideo`ノード（`comfy_extras/nodes_video.py`、`file`ウィジェット）に絞って探す（`image`と違い`file`という名前は他ノードとも衝突しうるため`node.type === "LoadVideo"`で限定）。
|
||||
- Galleryツールバーに「Send to LV node」ボタンを追加。動画以外が選択されている場合はエラートースト。アップロードは既存の`/upload/image`エンドポイント（拡張子を問わず任意バイナリをinputフォルダへ保存できる汎用エンドポイント）をそのまま流用。
|
||||
- 実機でLoadVideoノードを作成→選択→送信し、`file`ウィジェットの値が正しく書き換わることを確認（`ComfyUI_00013_.mp4` → `test_video_12345.webm`）。
|
||||
|
||||
### Gallery: WebM対応
|
||||
|
||||
- `gallery_service.py`の`.mp4`限定だった5箇所（埋め込みメタデータ読み込み・寸法/再生時間取得・サムネイル抽出・PSDレイヤー除外判定・一覧フィルタ）を`VIDEO_EXTENSIONS = {".mp4", ".webm"}`ベースに統一。いずれもPyAV（`av`）経由でコンテナ形式を問わず同じコードパスのため、webm固有の実装追加は不要だった。
|
||||
- `gallery-tab.js`の`isVideoFile()`をwebm対応に拡張。一覧表示・プレビュー・ライトボックス・Comic Creator連携・一括レイヤー編集の動画除外フィルタなど、これを参照する全箇所が自動的にwebm対応。
|
||||
- ヘルプカード（Gallery/3言語）を更新。
|
||||
|
||||
### 意思決定モデルクライアントにvision対応を追加（移植、消費する機能はまだ無い）
|
||||
|
||||
Comic Creatorの`decision-client.js`が2026-10に追加したvision対応（Ollamaの`clef`等、画像入力可能な意思決定モデル向け）を、WFS本家の`static/js/decision-client.js`へ移植した。
|
||||
|
||||
- `decide(state, questions, opts, images)`に第4引数`images`（base64配列、`data:`プレフィックス無し）を追加。
|
||||
- `imageToBase64(src)`（Blob/data URL/画像URL→base64）、`supportsDecisionVision(opts)`（設定中のモデルが画像入力に対応しているかをOllamaの`/api/show`で判定。`capabilities`に`vision`が無いビルドでは`projector_info`の有無でも判定するフォールバック込み）を新規追加。
|
||||
- `DECISION_BACKENDS.ollama.models`に`"clef"`を追加（27B・vision対応、Ollama 0.35.1以降が必要。`clef-flash`は2026-10時点でOllama側の不安定挙動があるため含めていない）。
|
||||
- 実機（稼働中のOllama、`clef`モデル）で`supportsDecisionVision`→`true`、`imageToBase64`→`decide(images=[...])`の一連の流れを実画像（canvas生成の赤い正方形）で検証し、`red: 0.99` / `blue: 0.009`と正しく判定されることを確認。
|
||||
- 既存の呼び出し箇所（`settings-tab.js`、`vram-prepare.js`）は`images`引数を使わないため後方互換。消費する機能（Gallery自動分類等）はまだWFS側に無く、クライアントライブラリの同期のみ。
|
||||
|
||||
---
|
||||
|
||||
## v0.7.13（2026-10-06）
|
||||
|
||||
### VRAM調整（Ollama）— Comic Creatorから移植
|
||||
|
||||
画像生成の直前にOllamaでロード中のモデルをアンロードし、ComfyUIの空きVRAMを確保する機能をComfyUI Comic Creator（`/api/ccc/vram/prepare`）から移植。
|
||||
|
||||
- 設定タブに「VRAM調整（Ollama）」セクションを新設。生成前の動作は「何もしない」（既定）／「空きが目標未満のときだけ必要な分をアンロード」／「毎回すべてアンロード」、目標の空き（GB）、「今すぐすべてアンロード」ボタン。設定はブラウザの `wfm_vram_settings`。
|
||||
- フックは `comfyUI.queuePrompt()` の直前（`static/js/vram-prepare.js` の `prepareVramForGeneration()`）。GenerateUI・Lab・バッチ・Video・AI TOOLのChat生成など、Workflow Studio経由の生成すべてに効く。失敗しても生成は止めず、アンロード時・目標未達・失敗時のみトーストで通知（連続生成で同じ目標未達の警告は繰り返さない）。ComfyUIのキャンバスから直接実行する生成は対象外。
|
||||
- サーバー: 新規 `py/routes/vram_routes.py`（`POST /api/wfm/vram/prepare`、Content-Typeは `application/json` のみ＝他サイトからの単純リクエストでモデルを外されないため）。空きVRAMは **nvidia-smiの実測＋ComfyUI自身のtorch未使用キャッシュ**（ComfyUIの `get_free_memory` はOllamaのロード/アンロードに反応しないことがあるため）。取れなければComfyUIの値。autoは `size_vram` が実際に空く量より小さいため、大きいモデルから1台ずつアンロードしてそのつど実測で止める。アンロードは `keep_alive:0` の `/api/generate`、受け付けない意思決定モデルは `/api/chat` で再試行し、最後に `/api/ps` で確認。同時実行はロックで409。
|
||||
- 対象のOllama: 既定の `127.0.0.1:11434`（常に追加）＋OllamaのときのAI TOOL・Prompt・TaggerのAI設定と意思決定モデル。接続先は v0.7.12 の `url_guard`（loopback＋`WFS_ALLOWED_BACKEND_HOSTS`）を通し、許可外のURLはリクエスト全体を拒否せずそのURLだけ飛ばす。
|
||||
- 「モデルをアンロード」ボタン（AI TOOLタブ、Prompt/Taggerの⚙設定モーダル）は、Ollamaのとき意思決定モデルを含む全モデルをアンロードするよう変更（同じサーバー処理）。サイドパネルAタブの Settings にも「VRAM (Ollama) → Unload all now」を追加（サイドパネルからはSPAの `vram-prepare.js` を import できないため、同じlocalStorageキーから接続先を集める小関数を `node_sets_menu.js` に内蔵）。
|
||||
- 実機（ComfyUI_5、ポート8189、RTX 12GB）で確認: qwen3.5:9b＋tev1:0.8bをロードし空き1.5GBの状態から、auto（目標4GB）はqwen3.5:9bのみアンロードして空き1.7→9.1GB・tev1は残る、allはtev1も外す（各約1.1秒）。415/400の入力検証、許可外URLのスキップ、設定タブUI、`queuePrompt()` 経由のフック（`/prompt` 送信のみ差し止めて確認）、各アンロードボタンを確認。
|
||||
|
||||
### Workflowタブ: GenerateUIチェック（変換・操作性バッジ）
|
||||
|
||||
ワークフローをGenerateUIに読み込む前に、動くかどうかを確認できるようにした。
|
||||
|
||||
- 詳細モーダル（ダブルクリック）のタイトル下に、形式バッジ（UI/API/App形式）・「GenerateUIチェック」ボタン・✅⚠❌ の結果バッジ2つ。サイドの詳細ペインにも名前の下に形式バッジと結果バッジ（選択時に自動チェック、結果のみ）。バッジにマウスを乗せると理由をCSSチップで表示（詳細ペインはペイン幅いっぱいに表示）。
|
||||
- **変換**: GenerateUIと同じ `convertUiToApi()` で変換。未対応形式・変換エラー・このComfyUIに無いノード型は ❌、この環境に無いCOMBO値（モデル・LoRA・画像など）は ⚠。UI形式は変換時に先頭の選択肢へ無警告で差し替わる（全COMBOの差し替えを記録する `getLastComboSubstitutions()` を追加。GenerateUIの挙動は不変）。API形式は変換しないため値を直接照合し、「そのまま送るとComfyUIの検証エラー（Value not in list）」と案内。
|
||||
- **操作性**: 設定を持つノードのうち、`analyzeWorkflow()` でGenerateUIに表示されるノードの割合（✅80%以上／⚠50%以上／❌それ未満）。チップにプロンプト欄・サンプラー数と、ノードグラフで変更が必要なノードを列挙。
|
||||
- 当初は意思決定モデル（Unsloth laya）のscoreで操作性を評価する設計だったが、実測で区別できなかった（全ノード非表示の状態2.54 > 全表示2.35、状態を単純化しても2.24対1.87、yes確率0.41対0.44、choiceはどちらも同じ選択肢）ためルール判定に変更。
|
||||
- 結果は保存せずセッション中のメモリのみ。チェックは約20ms。`/object_info` の初回取得（約10秒）を避けるため、モーダルを開いた時点で先読み。
|
||||
|
||||
### ヘルプ
|
||||
|
||||
設定タブ（VRAM調整）、Workflowタブ（サイドパネル・GenerateUIチェック）、Prompt/Taggerタブ（アンロード）、AI TOOLタブ、サイドパネルAタブの記述を3言語で更新。
|
||||
|
||||
---
|
||||
|
||||
## v0.7.12（2026-10-04）
|
||||
|
||||
### セキュリティ: バックエンド接続先をブラウザから任意指定できた問題（SSRF）を修正
|
||||
|
||||
ComfyUI-Manager登録PR（LiveChatStream、Comfy-Org/ComfyUI-Manager#3340）のレビューで指摘された「リクエストから接続先URLを受け取り、サーバーが接続する」問題が、WFSにも同じ構造で存在していたため修正。
|
||||
|
||||
- 対象: `POST /api/wfm/ollama/chat`（bodyの `url`。従来は**検証なし**で、任意ホスト・クラウドメタデータ・`file://` に接続できた）、Tagger VLM（`/wfm/tagger/vlm/models`・`vlm/predict`・`batch/start` の `api_url`。Unslothのみ制限されていた）、Eagle（`/api/wfm/eagle/add`・`/test` の `eagleUrl`）、Unslothプロキシ（`baseUrl`）。
|
||||
- 新規 `py/services/url_guard.py` に共通化。許可は **loopback（localhost / 127.0.0.1 / ::1、ポート自由）** と、運用者が環境変数 `WFS_ALLOWED_BACKEND_HOSTS`（`host` または `host:port` のカンマ区切り）で指定したホストのみ。`http(s)` 以外（`file://` 等）、URL内の認証情報、`localhost.evil.com` のような偽装ホストは拒否。**リダイレクトは追従しない**（302は接続せずエラー）。
|
||||
- **挙動変更**: LAN上の別マシンのOllama/Eagle等を使っている場合は `WFS_ALLOWED_BACKEND_HOSTS` の設定が必要（拒否時のエラーメッセージに案内を表示）。
|
||||
- 実機（ComfyUI_5、ポート8189）で、メタデータIP・`file://`・公開IPの拒否、loopbackのOllama chat/Tagger一覧/Eagle test/Unsloth Decision APIの正常動作を確認。
|
||||
- 注意: ComfyUIを `--listen` で公開すると、到達できる相手は引き続きこれらのルートを使える（接続先を変えられないだけ）。信頼できるネットワークでのみ使うこと。
|
||||
|
||||
---
|
||||
|
||||
## v0.7.11（2026-10-02）
|
||||
|
||||
### セキュリティ: CivitAI情報の説明文のHTML無害化（保存型XSS対策）
|
||||
|
||||
+36
-13
@@ -11,6 +11,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Workflow, model, image, and prompt management
|
||||
- AI-powered prompt writing assistance, translation, and tag generation
|
||||
- File drop & Gallery-linked metadata viewing — inspect and reuse generation settings instantly
|
||||
- Semantic image search in the Gallery — find images by describing them, or list images similar to a selected one
|
||||
|
||||
**⚡ GenerateUI** — a productivity-focused generation tab connected to every other tab
|
||||
|
||||
@@ -26,33 +27,33 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Built-in AI tools (translation and more)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Features (tab -> feature names)
|
||||
|
||||
### Workflow Tab
|
||||
Thumbnail / Table views, Thumbnail side panel, Badge filtering, Search, Side panel tabs, Badge management, AI summary, Import / Export, Side panel toolbar buttons, Default view setting, Search clear (✕), Clear all filters (✕ Clear), Sort
|
||||
Thumbnail / Table views, Thumbnail side panel, Badge filtering, Search, Side panel tabs, Badge management, AI summary, Import / Export, Side panel toolbar buttons, Default view setting, Search clear (✕), Clear all filters (✕ Clear), Sort, GenerateUI Check
|
||||
|
||||
### Canvas Snapshot (v0.1.2)
|
||||
One-click capture, Auto-save as thumbnail, Embedded workflow metadata, Auto-import
|
||||
|
||||
### GenerateUI Tab (v0.3.5)
|
||||
6-tab layout, Save button, Input tab, Model tab, Settings tab, Always-visible Raw JSON, Bypass/Mute node handling, One-click generation, Seed control, Style selector, Create Catalog / Catalog buttons, Batch type selector, Batch tab, UI-to-API conversion, Video workflow support, Eagle integration, Qwen Image 2.1 support, Ming Image support
|
||||
6-tab layout, Save button, Input tab, Multi-image I2I for Comic Creator, Model tab, Settings tab, Always-visible Raw JSON, Bypass/Mute node handling, One-click generation, Seed control, Style selector, Create Catalog / Catalog buttons, Batch type selector, Batch tab, UI-to-API conversion, Video workflow support, Eagle integration, Qwen Image 2.1 support, Ming Image support, P-H3 Image support, MiniMax-H3-Image support
|
||||
|
||||
### Feeder subtab (v0.3.5 / v0.3.42)
|
||||
ImageFeeder node control, Image library, Selection management, Presets, Continuous Run loop, Index sync, Seed, WFS_GalleryFeeder node, Node & group selector, Image grid, After Gen modes, Run / Stop controls
|
||||
|
||||
### Lab subtab (v0.3.87 – v0.5.2)
|
||||
Setting / Results / Plan JSON sub-panels, Per-column keyframes, Model column — Checkpoint / LoRA Single / VAE, Empty state when no matching node exists, Wildcards, Prompt cell extras, Keyframe #1 live-reflect, Bypass checkbox, LoRA node bypass & prompt injection, T2I workflow support, Use generated image for next, Workflow recall, Plan files, Plan Load, Plan JSON tab, Save index image to Output on Run, Index images are self-contained plan files, Eagle integration, Select vs. edit, and keyframe reordering, PG — apply a Prompt tab preset group, Video workflow support, Use generated image for next, now video-aware
|
||||
### Lab subtab (v0.3.87 – v0.9.5)
|
||||
Setting / Results / Plan JSON sub-panels, Per-column keyframes, Model column — Checkpoint / LoRA Single / VAE, Empty state when no matching node exists, Wildcards, Prompt cell extras, Keyframe #1 live-reflect, Bypass checkbox, LoRA node bypass & prompt injection, T2I workflow support, Use generated image for next, Workflow recall, Plan files, Plan Load, Plan JSON tab, Save index image to Output on Run, Index images are self-contained plan files, Eagle integration, Select vs. edit, and keyframe reordering, PG — apply a Prompt tab preset group, Video workflow support, Use generated image for next, now video-aware, Plans from a story
|
||||
|
||||
### Prompt Tab
|
||||
3-column layout, AI chat assistant, Image attachment, Translation, Prompt presets, Preset Manager, Group management, Clipboard copy, GenUI Set, Wildcard/Style tab bar, Wildcard input toolbar, Wildcard file manager, Style manager, Form / Table toggle, Table view — Presets / Presets Group / Wildcards / Style
|
||||
|
||||
### Settings Tab
|
||||
2-column layout, Collapsible sections, Theme selection, Theme customization, Workflows directory, Gallery output directory, Eagle connection, CivitAI Host, CivitAI API Key, Default workflow, Default Checkpoint, Video Playback Volume, Data Management, Text Size, RAW JSON Colors, GenerateUI Model Tab Highlight, Wildcard Integration, G'MIC-Qt Integration, Language
|
||||
2-column layout, Collapsible sections, Theme selection, Theme customization, Workflows directory, Gallery output directory, Eagle connection, CivitAI Host, CivitAI API Key, Default workflow, Default Checkpoint, VRAM Management (Ollama), Video Playback Volume, Data Management, Text Size, RAW JSON Colors, GenerateUI Model Tab Highlight, Wildcard Integration, G'MIC-Qt Integration, Prompt Translation (native language → English), Language
|
||||
|
||||
### Gallery Tab (v0.3.44)
|
||||
Output / ImagePrompt / Style_Catalog / Metadata sub-tabs, Image browser, Folder tree root label, Thumbnail / Table views, Folder management, File operations, Download, MP4 video support, MP4 embedded metadata, Info tab: dimensions & duration, Multi-select, Image Compare, Prompt search, Server-side filtering, Group management, Thumbnail F button, Favorites, Detail panel, Prompt tab, Workflow viewer, GenUI button, Image Edit button, Send GenUI Image button, Send to LI node button, Send CC button, Search clear (✕), Clear all filters (✕ Clear), Workflow auto-save, Output folder configurable, SVG file support, PSD file support, Performance
|
||||
Output / ImagePrompt / Style_Catalog / Metadata sub-tabs, Image browser, Folder tree root label, Thumbnail / Table views, Folder management, File operations, Download, MP4 video support, MP4 embedded metadata, WebM support, Embedded metadata reading fixes, Audio support, File conversion, Move To folder filter, Semantic Search, Audio waveform colors, Info tab: dimensions & duration, Multi-select, Image Compare, Prompt search, Server-side filtering, Group management, Thumbnail F button, Favorites, Detail panel, Prompt tab, Nanobanana tab, Gemini / Nano Banana nodes, MOSS-Image in GenerateUI, Faster GenerateUI Check / conversion, Partner API nodes in GenerateUI, Workflow viewer, GenUI button, Image Edit button, Send GenUI Image button, Send to LI node button, Send to LV node button, Send CC Layout / Send CC Image buttons, Search clear (✕), Clear all filters (✕ Clear), Workflow auto-save, Output folder configurable, SVG file support, PSD file support, 3D model support, Performance
|
||||
|
||||
### ImagePrompt Gallery subtab (v0.3.94, renamed from Style/Prompt in v0.3.95)
|
||||
Visual prompt library, 3-column layout, Prompt builder (right pane), Plain-text prompt storage, ponyxlWildcardsVault format support, Seed data importer
|
||||
@@ -61,7 +62,7 @@ Visual prompt library, 3-column layout, Prompt builder (right pane), Plain-text
|
||||
Visual Style picker, 3-column layout, Positive/Negative panel, Select as Style button, Load in GenerateUI / Open in Metadata Tab buttons, Double-click to enlarge
|
||||
|
||||
### Metadata Gallery subtab (v0.3.8, moved from a top-level tab in v0.4.6)
|
||||
Now inside Gallery, 3-column layout, File drop, MP4 embedded metadata, Model extraction, LoRA extraction, Prompt extraction, Prompt actions, Format support, Format note
|
||||
Now inside Gallery, 3-column layout, File drop, MP4 embedded metadata, Settings section, Nanobanana section, Model extraction, LoRA extraction, Prompt extraction, Prompt actions, Format support, Format note
|
||||
|
||||
### Nodes Tab (v0.1.7)
|
||||
Node Browser, Search & Filter, Package badges, Node detail panel, Node Sets, Right-click context menu, Multi-select, Search clear (✕), Clear all filters (✕ Clear)
|
||||
@@ -72,14 +73,17 @@ Model Browser, Thumbnail / Table views, Table column sort, Search & Filter, User
|
||||
### Tagger Tab (v0.3.38)
|
||||
3 sub-tabs, Model support, Threshold sliders, VLM tagging, Single tab, Single output options, Batch tab, DB tab, Dependencies
|
||||
|
||||
### TagSpace Tab (v0.9.0)
|
||||
Choose tags visually in a 3D latent space, Build / Update, Choosing tags, Explore, Search with a card's or Gallery image's tags, Make a prompt, Tagger fill, Glow / Twinkle, Hover preview, Requirements
|
||||
|
||||
### Image Edit Tab (v0.3.65)
|
||||
Layer-based image editor, Loading images, Open PSD button, Close button, New button, Tools, Layer panel, Duplicate layer (⧉), Layer lock, Text quality, Export, Canvas navigation, Undo / Redo
|
||||
|
||||
### Video Tab (v0.5.0 – v0.7.4)
|
||||
Dedicated video generation UI, independent of GenerateUI, Plan subtab, Base settings row, Edit subtab, Edit: Multi-track, Edit: Audio / BGM, Edit: Text overlays, Edit: Crop, Edit: Undo / Redo, Edit project persistence, Asset subtab, Project subtab, Two center previews, First Frame / Last Frame are both optional, Semantic node lookup, not hardcoded IDs, Field highlight color, Load in Video, Frame tab, GIF tab, Playback volume
|
||||
Dedicated video generation UI, independent of GenerateUI, Plan subtab, Base settings row, Edit subtab, Edit: Multi-track, Edit: Audio / BGM, Edit: Text overlays, Edit: Crop, Edit: Convert audio, Edit: Undo / Redo, Edit project persistence, Asset subtab, Project subtab, Two center previews, Register as Asset, First Frame / Last Frame are both optional, Semantic node lookup, not hardcoded IDs, Field highlight color, Load in Video, Frame tab, GIF tab, Playback volume
|
||||
|
||||
### AI TOOL Tab (v0.3.14)
|
||||
4-pane layout, Translation pane, Chat pane, Chat pane — image generation via Tool Calling, Chat pane — generate_image extra parameters, TOOLS pane (VLM), TOOLS pane — shared Chat attachment, TOOLS pane (Wildcards), Chat pane — image-to-image (I2I), Chat pane — SVG generation, no ComfyUI workflow involved, Chat pane — Skills, Settings pane, Unsloth backend, Decision API client, Settings pane — Chat Image Generation, Settings pane — Chat I2I Generation, Settings pane — Generation, Model unload, Settings shared, Backend support, URL security, Lemonade's other endpoints (not integrated)
|
||||
### AI TOOL Tab (v0.3.14 – v0.9.5)
|
||||
4-pane layout, Translation pane, Chat pane, Chat pane — image generation via Tool Calling, Chat pane — generate_image extra parameters, TOOLS pane (VLM), TOOLS pane — shared Chat attachment, TOOLS pane (Wildcards), Chat pane — image-to-image (I2I), Chat pane — SVG generation, no ComfyUI workflow involved, Chat pane — Skills, Chat pane — Skills v2, Built-in skills, Chat pane — Image check & refine loop, Chat pane — Workflow Doctor, Chat pane — Storyboard to Lab, Chat pane — Agent Skills folders, TOOLS pane — OCR, Collapsible Translation / Settings panes, Settings pane, Unsloth backend, Decision API client, Settings pane — Chat Image Generation, Settings pane — Chat I2I Generation, Settings pane — Generation, Model unload, Settings shared, Backend support, URL security, Lemonade's other endpoints (not integrated)
|
||||
|
||||
### Workflow Studio Library (ComfyUI Side Panel) (v0.3.9)
|
||||
Tab layout (W / N / P / M / I / A), W — Workflows tab, N — Nodes tab, M — Models tab, P — Prompts tab, I — Information tab, A — AI TOOL tab, Drag & drop workflows, Drag & drop nodes, Drag & drop prompts, Send to Canvas, Copy prompts, Double-click, Search, ⚙ Theme settings
|
||||
@@ -108,6 +112,8 @@ Restart ComfyUI after installation.
|
||||
|
||||
Sample workflows are included in the `workflows/` folder. You can open them directly in ComfyUI via drag & drop, or load them from the Workflow tab.
|
||||
|
||||
Samples for the Feeder tab: `ws_sdxl_image_feeder_i2i` / `ws_dwpose_image_feeder` (Image Loop mode, require the `comfyui-image-feeder` custom node) and `ws_sdxl_gallery_feeder_i2i` (v0.8.3, SDXL I2I with the bundled **WFS_GalleryFeeder** node — no extra custom node needed; put images in the `__Feeder__` Gallery group and use the Feeder tab's Gallery mode).
|
||||
|
||||
> **Note:** Some sample workflows require additional custom nodes.
|
||||
> If a node is shown as missing (red/unknown) after loading, install the required custom nodes via **ComfyUI Manager** or by cloning the repository from GitHub into your `ComfyUI/custom_nodes/` directory.
|
||||
>
|
||||
@@ -120,6 +126,17 @@ Sample workflows are included in the `workflows/` folder. You can open them dire
|
||||
|
||||
---
|
||||
|
||||
## Manual
|
||||
|
||||
A beginner-oriented manual (covers v0.9.9, text-only edition without screenshots) is available in [`docs/manual/`](docs/manual/):
|
||||
|
||||
- English: [`docs/manual/en/`](docs/manual/en/) — ZIP: `Workflow_Studio_Manual_v0.9.9_en.zip`
|
||||
- 日本語: [`docs/manual/`](docs/manual/) — ZIP: `Workflow_Studio_Manual_v0.9.9.zip`
|
||||
|
||||
An EPUB edition with screenshots (Japanese and English, v0.9.9) is available on [Google Drive](https://drive.google.com/drive/folders/14mOHcnLYrFgNrOo6fy7ELyFjjWrAoU-_?usp=sharing).
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### Launch
|
||||
@@ -154,14 +171,19 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
|
||||
### Optional
|
||||
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features; also runs PLaMo 3 Translate for Settings → Prompt Translation (`ollama pull hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M`, or the Download button there)
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** in [Ollama](https://ollama.com) (no key; `ollama pull embeddinggemma-2:740m` or `embeddinggemma-2:440m`)
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
|
||||
#### Backend connection security (v0.7.12)
|
||||
|
||||
Server-side proxies for Ollama, LM Studio, Lemonade, Unsloth, the Tagger VLM and Eagle only connect to **localhost / 127.0.0.1 / ::1** (any port); `http(s)` only, no credentials in the URL, and redirects are never followed. To use a backend on another machine (e.g. an Ollama server on your LAN), list it in the `WFS_ALLOWED_BACKEND_HOSTS` environment variable (comma-separated `host` or `host:port`, e.g. `192.168.1.20:11434,gpu-box`) and restart ComfyUI. If you start ComfyUI with `--listen`, anyone who can reach it can use these routes, so only do that on a trusted network.
|
||||
|
||||
---
|
||||
|
||||
## Supported Languages
|
||||
@@ -193,7 +215,8 @@ ComfyUI-Workflow-Studio/
|
||||
│ │ ├── settings_routes.py # Settings API
|
||||
│ │ ├── ollama_routes.py # Ollama proxy API
|
||||
│ │ ├── eagle_routes.py # Eagle integration API
|
||||
│ │ └── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ ├── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ └── tagspace_routes.py # TagSpace tab API (build / space / search / neighbors)
|
||||
│ └── services/
|
||||
│ ├── workflow_service.py # Workflow file operations
|
||||
│ ├── nodes_service.py # Node metadata & node sets
|
||||
|
||||
@@ -7,6 +7,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Workflow, model, image, and prompt management
|
||||
- AI-powered prompt writing assistance, translation, and tag generation
|
||||
- File drop & Gallery-linked metadata viewing — inspect and reuse generation settings instantly
|
||||
- Semantic image search in the Gallery — find images by describing them, or list images similar to a selected one
|
||||
|
||||
**⚡ GenerateUI** — a productivity-focused generation tab connected to every other tab
|
||||
|
||||
@@ -22,7 +23,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Built-in AI tools (translation and more)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Screenshots
|
||||
|
||||
@@ -54,6 +55,10 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
| :--------------------------------: | :------------------------------------------: |
|
||||
|  |  |
|
||||
|
||||
| Gallery Tab - Semantic Search ("sunset city street") | TagSpace Tab - Search ("sunset city street") |
|
||||
| :-----------------------------------------------------------------: | :-----------------------------------------: |
|
||||
|  |  |
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
@@ -74,6 +79,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- **Search clear (✕)** — inline ✕ button appears inside the search box whenever text is entered; click to clear immediately
|
||||
- **Clear all filters (✕ Clear)** — rightmost toolbar button; resets search text, group filter, and badge filter (ALL) in one click
|
||||
- **Sort** (v0.7.2) — Date (Newest/Oldest) / Name (A-Z/Z-A) dropdown next to the group filter, persisted via localStorage — same pattern as the Gallery tab's sort control
|
||||
- **GenerateUI Check** (v0.7.13) — tells whether a workflow will run in GenerateUI before you load it. The side panel shows the format badge (UI / API / App) and two ✅ / ⚠ / ❌ result badges below the workflow name, checked automatically on selection; the detail modal shows the format badge, a **GenerateUI Check** button and the same badges below the title. Hover a badge for the reasons. **Conversion** runs the same UI→API conversion as GenerateUI: ❌ for unsupported formats, conversion errors or node types not installed in this ComfyUI (the prompt would be rejected), ⚠ for values not available here (models, LoRAs, images…) — UI-format workflows get them silently replaced with the first choice, API-format workflows are sent as-is and rejected by ComfyUI's validation. **Operability** is the share of nodes with settings that GenerateUI's fields expose (✅ 80%+ / ⚠ 50%+ / ❌ below), listing the nodes you'd still have to change in the node graph. Results aren't saved; a check takes about 20 ms once ComfyUI's node list (`/object_info`) is cached (the first one after opening the page waits about 10 s for it; the detail modal prefetches it). The Decision Model was evaluated for the operability rating but couldn't tell good and bad workflows apart, so it is rule-based
|
||||
|
||||
</details>
|
||||
|
||||
@@ -93,7 +99,8 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- **6-tab layout** — Input / Model / Settings / Feeder / Batch / **Lab** tabs; Input, Model, and Settings each include a Raw JSON column on the right for instant preview and direct editing
|
||||
- **Save button** — located at the right end of the subtab row; opens a filename dialog (default: current workflow name) and saves the current workflow as a `.json` file to the Workflow tab via the import API; hidden while the Batch or Lab subtab is active (v0.4.6), since those work on workflow copies / their own Plan files rather than the currently loaded workflow
|
||||
- **Default-workflow snapshot fix** (v0.4.8) — overwriting the loaded workflow under a filename that's currently set as the auto-load **Default workflow** (Settings tab, see below) now also refreshes that cached snapshot; previously only the file on disk was updated, so the old (pre-edit) version — including any checkpoint no longer installed — kept reappearing after every page reload until the default was re-set by hand
|
||||
- **Input tab** — Prompt and Image inner tabs (drag-and-drop upload); Prompt tab shows Positive Prompt and Negative Prompt textareas, plus an **Embeddings selector** at the bottom (Filter + Select + Weight input + Paste button); Paste inserts `(embedding:Name:weight)` at the cursor position of the last focused textarea (defaults to Positive when neither is focused); Raw JSON (540px) in the right column
|
||||
- **Input tab** — Prompt (left) and Image (right, about 3:2) side by side on one screen (drag-and-drop upload; Image wraps under Prompt when the area is narrow, about 720px or less); Prompt area shows Positive Prompt and Negative Prompt textareas, plus an **Embeddings selector** at the bottom (Filter + Select + Weight input + Paste button); Paste inserts `(embedding:Name:weight)` at the cursor position of the last focused textarea (defaults to Positive when neither is focused); Raw JSON (540px) in the right column, collapsible to a slim strip with the » button (Input tab only; remembered)
|
||||
- **Multi-image I2I for Comic Creator** (v0.8.12) — new iframe bridge `window._wfmReceiveMultiImageI2IRunRequest(imageBlobs, params, workflowData?, workflowFilename?)` used by Comic Creator's Nanobanana+ tab (Local mode): image *i* goes to the workflow's *i*-th LoadImage node (extra images are ignored), and optional width / height / steps / cfg / denoise / sampler / scheduler / seed are written through `setGenerationParams`; returns `{ ok, url, slotCount, usedImages }`
|
||||
- **Revert button** (v0.3.70) — a ↺ button next to each Apply button reverts the textarea to the node's currently applied value (the reverse of Apply)
|
||||
- **Emphasis weight editing** (v0.3.70) — select text (or just place the cursor on a word/parenthesis block) and press Ctrl+↑/↓ to adjust `(text:weight)` by ±0.05, A1111/native-ComfyUI style; parentheses are removed automatically when the weight returns to 1.0
|
||||
- **Mask slot for inpainting** (v0.3.71) — LoadImage slots whose MASK output feeds an inpaint node (e.g. `VAEEncodeForInpaint`) show an additional Mask drop zone; when a mask is set, image + mask are composited into a single RGBA upload (mask baked into the alpha channel) matching ComfyUI's native alpha→MASK extraction — no extra nodes required
|
||||
@@ -101,12 +108,14 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- **Model tab** — Checkpoint, VAE, Diffusion Model (UNETLoader / UnetLoaderGGUF / LoaderGGUF), and Text Encoder (CLIPLoader / DualCLIPLoader / ClipLoaderGGUF / DualClipLoaderGGUF — single clip: type + device; dual clip: two clip selectors + type + device) are always shown; **ControlNet** and **Hypernetwork** (with Strength field) are collapsed by default (v0.3.98) since they're used less often — click their header to expand; **LoRA** (Single/Stack tabs) has its own column; all selectors have a filter; Raw JSON on the right
|
||||
- **Highlight color** (v0.3.98) — the label of each model type above, plus the LoRA column header, turns a distinct color (default bright green, customizable in the Settings tab) when the loaded workflow actually contains that type of node; stays visible even while ControlNet/Hypernetwork are collapsed, so relevance is visible without expanding
|
||||
- **LoRA Stack Group toggle** (v0.3.92) — for `Lora Loader (LoraManager)` nodes, a **Use Stack Group** checkbox next to the Single/Stack tabs switches what the Stack tab shows and edits: OFF (the default whenever a workflow is loaded) displays and edits the LoRAs actually configured in the workflow's own Lora Manager node — correctly showing every LoRA it holds, not just the first one; ON switches to the LoRA group registered in the Models tab instead; toggling either direction clears the node's LoRA list first, then loads it from the newly selected source, and switching back to OFF restores exactly the list that was present when the workflow was loaded
|
||||
- **Native Load LoRA nodes** (v0.8.4) — ComfyUI v0.40's `Load LoRA (Model)` / `Load LoRA (Text Encoder)` pair (each node holds up to 20 rows of LoRA / strength / enabled) is detected and edited like the other LoRA nodes: Single / Stack Apply, the Models tab "Apply", the LoRA batch, Lab and the Bypass button write both nodes together (the Model node takes Str M, the Text Encoder node takes Str C); Stack Apply writes the whole Stack group as rows, and the Metadata tab / side-panel I tab list their LoRAs. API-format workflows only for now — ComfyUI frontend 1.55.17 cannot add rows to these nodes in its own graph editor, so UI-format workflows carry no `widgets_values` for them yet
|
||||
- **LoRA Bypass button** (v0.4.4) — a ⛔ toggle next to the Refresh Stack group button in the LoRA column header; forces the target LoRA node's strength to 0 (`strength_model`/`strength_clip` for a standard LoraLoader, or every entry's `strength`/`clipStrength` for a Lora Manager node — its LoRA list stays intact, not replaced) without touching the actual ComfyUI node's mode; click again to restore the exact values that were there before toggling; targets whichever node Single/Stack Apply default to (the workflow's Lora Manager node if it has one, else the first LoRA node); clicking a Single or Stack Apply button clears the bypass state so it can't silently reassert itself later
|
||||
- **Settings tab** — KSampler and Latent Image side by side at 50% width each; Raw JSON on the right
|
||||
- **Always-visible Raw JSON** — edit the API-format JSON directly from any tab with syntax highlighting; Apply button reloads the workflow; built-in **search bar** (always shown) finds all matches as you type with count display (`3/12`); navigate with ↑/↓ buttons or Enter / Shift+Enter; Escape or ✕ clears; current match highlighted in orange, other matches in yellow
|
||||
- **Bypass/Mute node handling** (v0.3.75) — loading a UI-format workflow with Bypass-mode nodes now reroutes their wires to the upstream source (matching same-type input/output slots, recursively through chained bypasses) instead of leaving a dangling reference that failed validation; Mute-mode nodes are excluded without rerouting, same as ComfyUI itself; when either is present, a note listing the affected nodes appears above the Raw JSON search bar in the Input/Model/Settings tabs
|
||||
- **One-click generation** — queue prompts to ComfyUI without leaving the studio
|
||||
- **Seed control** — randomize, lock, or manually set seeds; seed input and mode selector stacked vertically for readability
|
||||
- **Workflow seed mode** (v0.8.3) — the mode selector's new **Workflow** option generates with the loaded workflow's own seed as-is (no override), e.g. to reproduce a Gallery image with its original seed; falls back to the Seed field only if the workflow has no literal seed input. Seed overrides no longer leak back into the loaded workflow (generation now works on a deep copy), so Workflow mode still sees the original seed after Random runs
|
||||
- **Style selector** — checkbox and dropdown next to the Reset Workflow button; enable to apply a Fooocus-style JSON to positive and negative prompts at generation time; style files (`*.json`) are loaded from `user/default/Workflow-Studio/style/`; the style's `{prompt}` placeholder is replaced with the original prompt text, or the style is appended if no placeholder exists; `negative_prompt` is appended to the existing negative prompt; styles are applied to a per-generation copy and do not modify the loaded workflow
|
||||
- **Wildcards inside a Style now expand** (v0.4.7) — an A1111-style `__wildcard__` reference written into a Style's Positive/Negative prompt is expanded at generation time just like one typed directly into GenerateUI; the Style text is merged into the prompt *before* wildcard expansion runs (previously the order was reversed, so a Style's own wildcard references were sent to ComfyUI unexpanded)
|
||||
- **Create Catalog / Catalog buttons** (v0.3.95) — next to the Style dropdown. **Create Catalog** generates one preview image per Style checked in the Batch tab's Style sub-tab (using the currently loaded workflow, typically T2I), then saves each result into a chosen `ws_style_catalog` subfolder named after the Style (existing files with the same name are overwritten); a modal lets you create a new destination folder or pick an existing one — separate folders are useful for grouping catalogs by checkpoint or by purpose. **Catalog** jumps straight to the Gallery tab's Style_Catalog sub-tab to browse the results (see below)
|
||||
@@ -124,6 +133,8 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- **Eagle integration** — auto-save generated images to [Eagle](https://eagle.cool/) with metadata
|
||||
- **Qwen Image 2.1 support** (v0.7.3) — `TextEncodeQwenImage21` (the unified T2I/Edit encoder used by Comfy-Org's Qwen-Image-2.1 workflow templates) is now recognized: its `prompt`/`negative_prompt` widgets populate the Input tab's Positive/Negative Prompt textareas and are editable like any other prompt node; its `resolution` (a pixel-budget int, not width/height) is exposed for Raw JSON editing. No extra work was needed for the model's up to 10 reference images (`image_1`…`image_10`, used for image editing) — each one arrives as an ordinary LoadImage node, so it already gets its own card in the Input tab's Image sub-panel via the existing generic LoadImage detection
|
||||
- **Ming Image support** (v0.7.5) — `TextEncodeMingImageEdit` (single `prompt` widget, up to 8 optional `images.image_N` reference inputs; role taken from the sampler/guider wiring, so the empty-prompt negative instance in layer-decomposition workflows shows as Negative) and `EmptyQwenImageLayeredLatentImage` (Latent panel; `layers` kept as-is) are recognized in GenerateUI, and Gallery/Metadata/sidepanel I tab now show their prompts as Positive. The Sampler field is disabled (N/A) when the SAMPLER input comes from a node without `sampler_name` (e.g. `SamplerLCM`). **Fix:** top-level nodes whose widgets are converted to wired inputs (e.g. `EmptyLatentImage` width/height fed by `GetImageSize`) keep those values in `widgets_values`; they were not stripped outside subgraphs, shifting later values (`batch_size` became `1024`, KSampler `steps`/`cfg` misread). Also: `BasicGuider`'s `conditioning` is treated as positive, and a linked text input wins over the stale widget text in UI-format metadata
|
||||
- **P-H3 Image support** (v0.9.4) — the `PH3ImageConditioning` templates (Image Edit and T2I) load and run from GenerateUI, with the prompt shown on the Input tab and read back by the Metadata tab. Width and height follow the workflow's wiring, and the prompt upsampler runs as set in the workflow
|
||||
- **MiniMax-H3-Image support** (v0.9.6) — the `MiniMaxH3ImageConditioning` templates (Image Edit and T2I) load and run from GenerateUI the same way, with the prompt shown on the Input tab and read back by the Metadata tab and the sidepanel I tab
|
||||
|
||||
</details>
|
||||
|
||||
@@ -156,7 +167,7 @@ Two independent modes selectable via **[Image Loop] / [Gallery]** toggle buttons
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>Lab subtab (v0.3.87 – v0.5.2)</h3></summary>
|
||||
<summary><h3>Lab subtab (v0.3.87 – v0.9.5)</h3></summary>
|
||||
|
||||
An experimental batch generator: runs the workflow currently loaded in GenerateUI N times, letting Model (Checkpoint/LoRA Single/VAE) / Prompt / KSampler each change independently starting at a chosen iteration — unlike the Batch tab's single-axis queue, every column keeps its own list of change points. Never mutates the loaded workflow; every iteration runs on a fresh clone. Works with both I2I and T2I workflows (v0.3.90).
|
||||
|
||||
@@ -182,6 +193,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **PG — apply a Prompt tab preset group** (v0.4.8) — a **PG** button on the Prompt column header opens a picker for the Prompt tab's Table-view preset groups (the same groups the Batch tab's checklist uses); choosing one replaces the entire Prompt column with one keyframe per preset in that group's order, starting at #1 (#1 is overwritten, not appended after). If the group has more presets than the current Batch count, Batch count is raised automatically to fit (capped at 50, the Batch input's own limit); anything still beyond that shows the existing "keyframes beyond Batch count" warning instead of being dropped
|
||||
- **Video workflow support** (v0.5.2) — loading a MiniMax H3 (or other all-in-one video-generation) workflow into GenerateUI and running Lab now works end to end: the Prompt column writes to its `prompt` widget, the KSampler column auto-hides (same "no matching node" treatment as Model sub-columns) since the sampler wired internally is a fixed part of the model's own preset rather than something meant to be tuned per keyframe, the Results grid plays mp4 results inline, and the contact-sheet index image draws each video result's last frame instead of failing silently onto a black square
|
||||
- **Use generated image for next, now video-aware** (v0.5.2) — when a Run's output is a video, the "Use generated image for next" chain extracts its last frame client-side (canvas) and uploads it as the next iteration's source image, instead of trying to feed the raw mp4 into a LoadImage node
|
||||
- **Plans from a story** (v0.9.5) — AI TOOL Chat's **Storyboard to Lab** skill loads a ready plan here via **Open in Lab**; KSampler cells now show every value that is set (Step / CFG / Denoise / sampler / scheduler / Seed) instead of "(workflow default)" when only the seed or sampler is set
|
||||
|
||||
</details>
|
||||
|
||||
@@ -236,6 +248,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Default workflow** — set a workflow to auto-load on startup
|
||||
- **Default Checkpoint** (v0.3.73) — optional checkbox to force every checkpoint loader node's model to a chosen default whenever a workflow is loaded into GenerateUI; a safeguard against generation errors or garbled images from a stale/mismatched checkpoint saved in the workflow file
|
||||
- ~~AI Assistant Settings~~ (removed in v0.5.4) — the central Ollama backend/URL/model panel that used to live here was replaced by a ⚙ Settings button + modal directly in the Prompt tab and the Tagger tab, each with its own independent configuration (backend, URL, model, Thinking mode, Max tokens, and a model Unload button) — see those tabs below; the AI TOOL tab's own Settings pane is unaffected
|
||||
- **VRAM Management (Ollama)** (v0.7.13) — right before every image generation from Workflow Studio (GenerateUI, Lab, batches, Video, AI TOOL chat generation — anything that goes through its queue), unload the models Ollama has loaded so ComfyUI has free VRAM. **Before generating**: Do nothing (default) / Unload only as much as needed when free VRAM is below a target (biggest model first, one at a time, stopping once the target is reached) / Unload everything every time; **Target free VRAM (GB)** for the second mode; **Unload all now** button. Free VRAM is measured with `nvidia-smi` (plus ComfyUI's own unused torch cache), because ComfyUI's free-VRAM figure may not change when Ollama loads a model; non-NVIDIA GPUs fall back to ComfyUI's value. Targets Ollama on localhost — the default `127.0.0.1:11434` plus the AI TOOL / Prompt / Tagger AI settings and the Decision Model when they use Ollama (other hosts only via `WFS_ALLOWED_BACKEND_HOSTS`); LM Studio etc. and ComfyUI's own models are not touched. A toast reports what was unloaded, an unreachable target or a failure; generation continues either way. Running a workflow from the ComfyUI canvas itself is not covered. Server route: `POST /api/wfm/vram/prepare` (JSON only). Ported from ComfyUI Comic Creator
|
||||
- **Video Playback Volume** (v0.5.4) — the video preview's volume is remembered here and shared across every video player in the plugin (Video tab, Gallery tab), instead of resetting to maximum whenever a different video is opened
|
||||
- **Data Management** — export core settings/metadata/prompts/CivitAI cache to a single JSON file; import to restore data (useful when migrating or reinstalling); API keys are excluded from exports for security
|
||||
- **Full Backup (ZIP)** (v0.5.2) — additionally covers what the JSON bundle can't represent: folder-based data (AI Skills, Lab Plans, Video Plans, Video Edit projects, Style presets) and the Tagger SQLite database; two opt-in checkboxes can also include ComfyUI's default workflows folder (`user/default/workflows`) and the Impact Pack's wildcard folder in the export — both of those are export-only and stored under an `_external/` prefix in the ZIP that Restore always skips, since auto-overwriting files a different plugin/ComfyUI core manages would be unsafe; copy them back by hand if needed. A third opt-in checkbox (v0.7.6) bundles the clips/BGM/sounds/overlay clips that saved Video Edit projects reference in ComfyUI's input folder (the project JSON only stores references) under `_video_edit_media/`; unlike `_external/`, these are restored into the input folder, never overwriting a file that already exists there. Restoring overwrites existing files; restart ComfyUI afterward for either restore path
|
||||
@@ -244,6 +257,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **GenerateUI Model Tab Highlight** (v0.3.98) — pick the color used to mark Checkpoint/VAE/Diffusion Model/Text Encoder/ControlNet/Hypernetwork labels and the LoRA column header in GenerateUI's Model tab when the loaded workflow actually contains that type of node; default bright green; changes apply immediately, Reset Defaults restores it; saved to `localStorage` under `wfm_settings.modelTabActiveColor`; the same color and mechanism also drives the Video tab's field highlighting (v0.5.4, see above)
|
||||
- **Wildcard Integration** — link the WFS wildcard directory to ComfyUI-Impact-Pack's `wildcards/` directory (directory junction on Windows, symlink on other OS); existing WFS wildcard files are automatically migrated; requires [ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack)
|
||||
- **G'MIC-Qt Integration** — set the path to `gmic_qt.exe` for the Image Edit Filter tool; download G'MIC-Qt Standalone from `gmic.eu/download.html` (Windows: `gmic_qt-win64.zip`), extract anywhere, then enter the full path to `gmic_qt.exe` and click Save
|
||||
- **Prompt Translation (native language → English)** (v0.9.9) — off by default. Adds a 🌐 button to the GenerateUI Positive/Negative prompts, the Lab prompt modal and the Video Plan block prompt that translates only the parts written in your own language into English with a local Ollama model, keeping prompt syntax as it is (commas, `(word:1.2)`, `[ ]`, `{a|b}`, `<lora:...>`, `__wildcards__`, `embedding:name`, `BREAK`); "Original" restores the text. Model: [PLaMo 3 Translate](https://huggingface.co/pfnet/plamo-3-translate-2b-GGUF) 0.8B / 2B / 8B / 31B (Download button pulls it into Ollama; free for personal and non-commercial use under the PLaMo Community License) or any other Ollama model. Source language auto-detect or one of 51 languages; run on GPU (about 0.1 s per phrase, optionally unloaded after each translation) or **CPU kept loaded** (about 0.3 s per phrase, no VRAM, preloaded when GenerateUI opens). **Translate automatically when running**: GenerateUI Generate (replaces the prompt text), Lab Run and Video Plan Run (translate only what is sent; the plan keeps your text); if translation fails nothing is generated. Server routes: `POST /api/wfm/translate`, `GET /api/wfm/translate/config|models`, `POST /api/wfm/translate/pull|warmup|unload`
|
||||
- **Language** — English / Japanese / Chinese
|
||||
|
||||
</details>
|
||||
@@ -260,6 +274,17 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Download** (v0.5.0) — a dedicated orange **Download** button in the detail panel's action-button row (next to Tagger); replaces the old preview-image hover overlay, which blocked video playback controls
|
||||
- **MP4 video support** (v0.5.0) — `.mp4` files are browsable, previewable (detail panel with native `<video controls>`, lightbox, and Compare), and included in favorites/tags/groups/move/delete/export alongside images; server-side thumbnails are generated from the first video frame via PyAV (falls back to a placeholder if PyAV/ffmpeg decoding isn't available); a small ▶ badge overlays video thumbnails in the grid
|
||||
- **MP4 embedded metadata** (v0.5.1) — the Prompt and JSON tabs now also work for `.mp4` files: reads the `workflow`/`prompt` container tags ComfyUI's `SaveVideo` writes into the mp4 (via PyAV server-side), including all-in-one video-generation nodes that carry their prompt directly on a `prompt` input rather than a separate `CLIPTextEncode`-style node (e.g. `MiniMaxH3ImageToVideo`)
|
||||
- **WebM support** (v0.7.14) — `.webm` files get the same treatment as `.mp4` everywhere above (browsing, thumbnails, dimensions/duration, embedded workflow/prompt metadata, favorites/tags/groups/move/delete/export, Compare, Comic Creator video send): all the video-specific code paths go through PyAV, which handles both containers identically, so no format-specific logic was needed beyond recognizing the extension
|
||||
|
||||
- **Embedded metadata reading fixes** (v0.8.0) — PNG chunks appended after `IEND` (ComfyUI-Custom-Scripts writes `workflow` there) are now read; `.webp` files expose their EXIF `workflow`/`prompt` to the Prompt and JSON tabs; images without a ComfyUI workflow (A1111 / Forge / Fooocus) show their `parameters` prompts in the Prompt tab; API-format `ImageMetadataLoRALoader` LoRAs are listed in the Metadata tab and the side-panel I tab
|
||||
- **Audio support** (v0.8.0) — `.wav` / `.mp3` / `.ogg` / `.m4a` / `.flac` / `.opus` files are browsable alongside images and videos (for ComfyUI music generation output): thumbnails are waveform images generated server-side (PyAV, cached), the detail panel / lightbox / Compare show the waveform with a built-in audio player (shared volume setting with videos), the Info tab shows the duration, and the Prompt / JSON tabs read the `prompt` / `workflow` tags that SaveAudio-type nodes embed (FLAC / MP3 / Opus); favorites, tags, groups, move, delete, and export work as for other files, while image-only actions (PSD export, Edit Layers) skip audio; in the Video tab's Asset sub-tab, audio files added to the Video Assets group appear with an "Audio Only" filter, an inline player, and **Set as BGM** / **Add as sound** buttons that feed the Edit sub-tab
|
||||
- **File conversion** (v0.8.0) — a **Convert...** button in the Gallery bulk bar (shown when the selection contains audio or video) converts audio files to FLAC / MP3 / Opus / M4A / WAV, or extracts the audio track of videos, using PyAV (no system ffmpeg needed); bitrate, sample rate (44.1 / 48 kHz) and channels (mono / stereo) are selectable, output goes next to the source as a new file (`name.ext`, or `name_converted.ext` if taken). ComfyUI `prompt` / `workflow` tags (and title / artist / album tags) are carried over to formats that can hold them — WAV cannot, so they are skipped there; favorite / tags / memo / groups can be carried over, and the original can optionally be deleted after a successful conversion (with a confirmation). `.ogg` is readable but not offered as an output (the bundled PyAV has no Vorbis encoder)
|
||||
- **Move To folder filter** (v0.8.0) — the Move To dialog (detail panel and bulk bar) has a filter box that narrows the folder list by name or path (space = AND), Enter moves to the first match; the move toast now reads "Moved N file(s)" since audio/video files are moved too
|
||||
- **Semantic Search** (v0.8.2) — find images by meaning instead of keywords. The **S** button next to the search box turns it on: the search box then takes a free-text description (e.g. "sunset city street"), Enter searches, and results are ordered by similarity with a score badge on each thumbnail. Turning **S** on also shows a second toolbar row (left-aligned): **Similar** (lists images similar to the selected one; selecting another image keeps the list, **↻** re-runs it for the newly selected image, pressing Similar again returns to the normal list), the **⚙** settings button, and the result count. Searches cover the selected folder and its subfolders; sort and tag/group filters do not apply to semantic results
|
||||
- **Index window (⚙)** — three tabs. **Index**: target folder (optionally with subfolders), "new/changed only" or "rebuild all", progress bar, stop, delete all. **Model**: choose with radio buttons between **Gemini Embedding 2** (API; set `GEMINI_API_KEY` in the plugin's `.env` and restart ComfyUI — the free tier exists but content sent on it may be used by Google to improve its products, beyond it the API is paid) and **EmbeddingGemma 2** (runs locally in Ollama, free, no key), plus vector size (Gemini 768 / 1536 / 3072, EmbeddingGemma 2 128 / 256 / 512 / 768), and for EmbeddingGemma 2 the Ollama URL and the model size (740m or 440m). **Search**: minimum score for similar-image results (e.g. `0.6` shows only images scoring 0.6 or higher), minimum score for text search, and maximum results — stored per model because score levels differ between models
|
||||
- **Index state in the folder tree** — a green dot next to a folder's image count means every image directly in that folder is indexed; the index is one shared store, so folders can be indexed one at a time. Images, plus videos / PSD via their thumbnails (audio is skipped), are embedded; changing the model, vector size or image precision requires "Rebuild all"
|
||||
- **EmbeddingGemma 2 in Ollama** — download the model with `ollama pull embeddinggemma-2:740m` (text + image + audio) or `ollama pull embeddinggemma-2:440m` (text + image, smaller). Both return identical vectors, so you can switch sizes without rebuilding (`:latest` is the same as 740m and is used when 740m is missing; 270m / 570m cannot embed images). Images are sent 4 per request (≈0.1–0.2 s per image on an NVIDIA GPU) and retried automatically when Ollama drops a connection. The model is unloaded from VRAM when indexing finishes. Indexes built with the earlier built-in ONNX runtime keep working (cos ≈0.97 with the Ollama vectors); use "rebuild all" to regenerate them exactly. Other Ollama hosts must be listed in `WFS_ALLOWED_BACKEND_HOSTS`
|
||||
- **Audio waveform colors** (v0.8.0) — waveform thumbnails are colored per format (MP3 blue, WAV green, FLAC amber, OGG/Opus purple, M4A rose)
|
||||
- **Info tab: dimensions & duration** (v0.5.1) — the detail panel's Info tab now also shows image dimensions (width×height) or video duration, in addition to filename, size, date, tags, and memo
|
||||
- **Multi-select** — Ctrl+click to select multiple images; Bulk Bar appears for batch operations: select group → "Add to Group" / "Remove from Group"; Favorite All / Unfavorite All; Compare (2–4 images); Move To...; **Export** (downloads all selected images as a ZIP file); **Create PSD** (v0.6.0, combines the selected images into one layered PSD file, positioned top-left on a canvas sized to the largest image); **Edit as Layers** (v0.6.0, opens all selected images in the Image Edit tab at once, each as its own layer); Delete File
|
||||
- **Image Compare** — select 2–4 images with Ctrl+click, then click "Compare" in the Bulk Bar to open a side-by-side lightbox; grid adapts to the number of selected images
|
||||
@@ -268,20 +293,27 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Group management** — create, rename, delete groups and assign/remove images using the same 4-section panel as Models tab; **`__Feeder__`** is a reserved group (🔒 prefix) that cannot be renamed or deleted; used by the Feeder Gallery mode to define the generation queue; **FC** button in the toolbar clears all members without deleting the group
|
||||
- **Thumbnail F button** — cyan overlay button (top-left of each image card) toggles the image's membership in the `__Feeder__` group; cyan and always-visible when the image is in the group; visible on hover only when inactive
|
||||
- **Favorites** — star images inline without reopening the detail panel
|
||||
- **Detail panel** — view filename, path, tags, groups, and metadata in a slide-out panel; four tabs — **Info** / **JSON** / **Groups** / **Prompt**
|
||||
- **Prompt tab** (v0.4.7) — selected by default (the most frequently checked piece of info shouldn't need an extra click); lists every positive/negative prompt candidate found in the embedded workflow with POS/NEG badges, reusing the exact same list→click→full-text UI as the Metadata tab; extracts from the API-format embedding when available rather than the UI-format one, since a workflow with independent top-level and subgraph sampling chains (e.g. Krea-2) can otherwise have some of its prompts missed
|
||||
- **Detail panel** — view filename, path, tags, groups, and metadata in a slide-out panel; four or five tabs — **Info** / **JSON** / **Groups** / **Prompt** / **Nanobanana** (only for `cc_nanobanana`-folder images, see below)
|
||||
- **Prompt tab** (v0.4.7) — selected by default (the most frequently checked piece of info shouldn't need an extra click); lists every positive/negative prompt candidate found in the embedded workflow with POS/NEG badges, reusing the exact same list→click→full-text UI as the Metadata tab; extracts from the API-format embedding when available rather than the UI-format one, since a workflow with independent top-level and subgraph sampling chains (e.g. Krea-2) can otherwise have some of its prompts missed; hidden entirely for images in the `cc_nanobanana` folder (see Nanobanana tab below), since they carry no embedded ComfyUI workflow to extract a prompt from
|
||||
- **Nanobanana tab** (v0.7.14) — shown only for images in a `cc_nanobanana` folder (written by ComfyUI Comic Creator's Nanobanana tab) that have a matching entry in that folder's `history.jsonl` generation log; shows engine, model, image size, seed, timestamp, and the prompt/negative prompt actually used — the same info is also surfaced in the Metadata tab (see below) when opened from here via the **Metadata** button
|
||||
- **Gemini / Nano Banana nodes** (v0.8.5) — images generated by ComfyUI's native Nano Banana nodes (`GeminiNanoBanana2V2` etc., e.g. via ComfyUI Comic Creator's "ComfyUI native node" engine) are readable like any other ComfyUI image: the Metadata tab and the side-panel I tab show the prompt (in the prompt list, including the appended `NEGATIVE_PROMPT:` text), the model name (under Diffusion Model, e.g. `Gemini Nano Banana 2.1`) and the seed; these images need no history file and are not special-cased like the `cc_nanobanana` folder (API-format workflows only)
|
||||
- **MOSS-Image in GenerateUI** (v0.8.8) — the official MOSS-Image templates (Base / SFT, T2I and the bounding-box layout versions) load and run from GenerateUI; a custom (non-official) manga workflow using MOSS-Image Base with the 8-step distillation LoRA was also tested and works. The `MOSSImageLayoutPrompt` text is edited as the positive prompt, `CreateBoundingBoxes` converts correctly (the boxes themselves are still drawn on ComfyUI's canvas), and the Gallery Prompt tab, Metadata tab and side-panel I tab read the prompt, models and sampler settings of MOSS images. Newer frontends save subgraph widget values by name (`widgets_values_named`); these are now read correctly, so model names and sampler values no longer shift
|
||||
- **Faster GenerateUI Check / conversion** (v0.8.8) — the first conversion or check used to download the full `/object_info` (tens of MB, 40+ s on installs with many custom nodes); it now fetches only the node types the workflow uses (about 0.25 s) and caches them per type
|
||||
- **Partner API nodes in GenerateUI** (v0.8.6) — workflows containing ComfyUI's partner API nodes (Nano Banana / Gemini image nodes, etc.) can be run from GenerateUI. These nodes need Comfy credentials in the `/prompt` request's `extra_data`, which ComfyUI's own screen adds automatically but a separate page cannot: add `COMFY_API_KEY=...` (a Comfy API key issued at platform.comfy.org) to the plugin's `.env` and restart ComfyUI. With it set, GenerateUI queues prompts through the plugin server (`/api/wfm/partner/prompt`), which adds the key only when the workflow contains an API node (`api_node` in `/object_info`) and only for this ComfyUI instance; the key never reaches the browser, and usage is billed as Comfy credits. Without a key, an authentication error now shows how to set it up. A Gemini image node's prompt shows in the Prompt tab (reference images go through `LoadImage` in the Image tab); sample: `ws_nano_banana_2_1_t2i`. Also fixed: UI-format → API-format conversion misaligned `widgets_values` for nodes whose DynamicCombo option has socket sub-inputs (Nano Banana 2.1's `images` / `files`) or a widget right after a `control_after_generate` seed; Gallery now shows the Prompt tab for `cc_nanobanana`-folder images that carry embedded metadata (images made through Comic Creator's ComfyUI native node engine)
|
||||
- **Workflow viewer** — Metadata tab displays workflow JSON from PNG embedded data (`prompt` / `workflow` keys) or from workflow saved by the Generate UI tab; **Copy & Send Canvas** button copies the JSON to the clipboard and sends the workflow directly to the ComfyUI canvas (UI and API formats both supported)
|
||||
- **GenUI button** (renamed from "Load GenUI" in v0.5.0 for readability) — loads the embedded ComfyUI workflow from the selected image directly into the GenerateUI tab; shows a warning toast if no workflow is embedded or the format is unsupported; Metadata button is styled green, GenUI button uses the primary accent color
|
||||
- **Image Edit button** — toolbar button; sends the selected image directly to the Image Edit tab as the base layer; PSD files (v0.6.0) are fully decomposed into their original layers instead of being flattened into one image
|
||||
- **Send GenUI Image button** — toolbar button (next to Image Edit); uploads the selected image to ComfyUI and sets it as the input for the first LoadImage node in the GenerateUI tab; automatically switches to GenerateUI → Input → Image; requires a workflow with at least one LoadImage node to be loaded
|
||||
- **Send to LI node button** (v0.5.2) — toolbar button (next to Send GenUI Image); uploads the selected image and writes its filename into the `image` widget of the node currently selected on the ComfyUI canvas (falling back to the first LoadImage-type node found) — same destination and cross-window bridge as the Image Edit tab's **Send to LI node** button (renamed from "Send to Workflow" in this same release), so both entry points now share one consistent name; requires Workflow Studio to have been opened from ComfyUI's top menu button
|
||||
- **Send CC button** (v0.3.69) — toolbar button, shown only when this Gallery tab is embedded as an iframe inside ComfyUI Comic Creator (a separate custom node); sends the selected image directly into Comic Creator's currently selected panel/overlay; when the selected file is an MP4 video (v0.6.3), it is routed to Comic Creator's Layout-tab Video tool instead — inserted as a new video object in the selected panel (playable, capturable to a still frame) rather than as a flattened image; sending a video while the Image tab is the intended destination is not yet supported and shows an info toast instead
|
||||
- **Send to LV node button** (v0.7.14) — toolbar button (next to Send to LI node), the video equivalent: uploads the selected video and writes its filename into the `file` widget of the node currently selected on the ComfyUI canvas, restricted to ComfyUI's native **Load Video** node (`LoadVideo`) since `file` as a widget name isn't unique enough to search for generically the way `image` is; only enabled when the selected item is a video (mp4/webm); same cross-window bridge and launch requirement as Send to LI node
|
||||
- **Send CC Layout / Send CC Image buttons** (v0.3.69, split in v0.8.11) — toolbar buttons, shown only when this Gallery tab is embedded as an iframe inside ComfyUI Comic Creator (a separate custom node). **Send CC Layout** sends the selected image directly into Comic Creator's Layout tab (the currently selected panel/overlay); when the selected file is an MP4 video (v0.6.3), it is routed to the Layout-tab Video tool instead — inserted as a new video object in the selected panel (playable, capturable to a still frame) rather than as a flattened image. **Send CC Image** (v0.8.11) switches Comic Creator to its Image tab and loads the selected image there — as a new layer if a document is already open, otherwise as a new canvas; videos are not supported there and show an info toast. Requires a Comic Creator version that provides `window.sendImageToImageTab` (the one released together with v0.8.11).
|
||||
- **Search clear (✕)** — inline ✕ button appears inside the search box whenever text is entered; click to clear immediately
|
||||
- **Clear all filters (✕ Clear)** — toolbar button next to the view-mode (Thumbnail/Table) switch (v0.5.2, moved from the toolbar's rightmost position so it sits with the other display-affecting controls); resets search text, tag filter, group filter, and favorites filter in one click (sort order is preserved)
|
||||
- **Workflow auto-save** — images generated from the Generate UI tab have their workflow automatically saved to gallery metadata
|
||||
- **Output folder configurable** — set the scanned output folder from Settings tab
|
||||
- **SVG file support** (v0.3.77) — `.svg` files are browsable, previewable (detail panel and lightbox), and included in favorites/tags/groups/move/delete/export alongside raster formats; served as-is with no rasterized thumbnail, since `<img>` scales vector graphics natively at any size
|
||||
- **PSD file support** (v0.6.0) — `.psd` files are browsable alongside other formats; thumbnails and full-size previews show every layer merged together (hidden layers excluded), rendered server-side via `psd-tools` and disk-cached; the underlying PSD file itself is never modified — only the Gallery preview is a flattened rendering
|
||||
- **3D model support** (v0.9.4) — `.glb` / `.gltf` / `.vrm` / `.ply` (mesh, point cloud or Gaussian splat) / `.spz` files are browsable alongside images with a **3D** badge and open in a built-in viewer in the detail panel, lightbox and compare view (drag to rotate, right-drag to pan, wheel to zoom). Thumbnails are rendered by your browser the first time a file appears and then kept on the server (files over 128 MB get one when opened). Workflows saved by ComfyUI's SaveGLB are read like an image's. A `.gltf` using separate `.bin` or texture files needs them to stay in place: Move and Delete only affect the `.gltf` itself. KTX2-compressed textures are not supported
|
||||
- **Performance** — server-side 256px JPEG thumbnail generation with disk cache (`data/thumb_cache/`); infinite-scroll paging (50 images per page, IntersectionObserver); folder-level mtime cache (60s TTL); bulk operations use single-request API endpoints
|
||||
|
||||
</details>
|
||||
@@ -314,15 +346,18 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
<summary><h3>Metadata Gallery subtab (v0.3.8, moved from a top-level tab in v0.4.6)</h3></summary>
|
||||
|
||||
- **Now inside Gallery** (v0.4.6) — this used to be its own top-level tab; it's now the fourth Gallery sub-tab, to the right of Style_Catalog — everything below works exactly the same as before
|
||||
- **3-column layout** — Drop zone (left) | Model info (center) | LoRA + Prompt (right)
|
||||
- **3-column layout** — Drop zone (left) | Model info + LoRA + Prompt (center) | Settings + Nanobanana (right, v0.7.14 — LoRA and Prompt moved here from the right column, swapping places with these two new sections)
|
||||
- **File drop** — drop a ComfyUI-generated PNG / WebP / MP4 or workflow JSON onto the drop zone (or click to open a file picker); PNG/WebP images are shown as a preview
|
||||
- **MP4 embedded metadata** (v0.5.1) — reads the `workflow`/`prompt` container tags ComfyUI's `SaveVideo` writes into the mp4's `moov/udta/meta` box (parsed client-side, no server round-trip); also recognizes all-in-one video-generation nodes that carry their prompt directly on a `prompt` input instead of a separate `CLIPTextEncode`-style node (e.g. `MiniMaxH3ImageToVideo`)
|
||||
- **Model extraction** — automatically extracts Checkpoint, VAE, Diffusion Model, and Text Encoder names from the workflow; supports both standard and subgraph-based workflows (Flux.2 Dev/Klein, Qwen-Image-Edit/2511/Layered, Z-Image Base/Turbo, Ernie Image, WAN2.2, Mage-Flow, LongCat, Boogu, HiDream E1, FireRed, Krea-2 Turbo); node types covered: UNETLoader, UnetLoaderGGUF, UNETLoaderGGUF (e.g. HiDream GGUF), CLIPLoader, DualCLIPLoader, TripleCLIPLoader, QuadrupleCLIPLoader (e.g. HiDream 4-CLIP)
|
||||
- **MP4 embedded metadata** (v0.5.1) — reads the `workflow`/`prompt` container tags ComfyUI's `SaveVideo` writes into the mp4's `moov/udta/meta` box (parsed client-side, no server round-trip); also recognizes all-in-one video-generation nodes that carry their prompt directly on a `prompt` input instead of a separate `CLIPTextEncode`-style node (e.g. `MiniMaxH3ImageToVideo`); WebM isn't supported here (this parser only understands MP4's ISO-BMFF container) — the Gallery tab's own video metadata reading (PyAV-based, see Gallery Tab above) covers WebM instead
|
||||
- **Settings section** (v0.7.14) — image size (width×height) and, when the embedded workflow has a KSampler-type node, its Seed / Steps / CFG / Sampler / Scheduler / Denoise values, read from the API-format embedding's named inputs (UI-format workflows are converted through the same UI→API path GenerateUI uses first)
|
||||
- **Nanobanana section** (v0.7.14) — shown only for images from ComfyUI Comic Creator's Nanobanana tab (a `cc_nanobanana`-folder image opened via the Gallery tab's **Metadata** button, with a matching entry in that folder's `history.jsonl` log); shows engine, model, image size, seed, timestamp, and the prompt/negative prompt actually used — since these images carry no embedded ComfyUI workflow, this section's data replaces what would otherwise be a "no metadata found" error, with the rest of the layout (Checkpoint/LoRA/Prompt/Settings) simply staying empty
|
||||
- **Model extraction** — automatically extracts Checkpoint, VAE, Diffusion Model, and Text Encoder names from the workflow; supports both standard and subgraph-based workflows (Flux.2 Dev/Klein, Qwen-Image-Edit/2511/Layered, Z-Image Base/Turbo, Ernie Image, WAN2.2, Mage-Flow, LongCat, Boogu, HiDream E1, FireRed, Krea-2 Turbo, MOSS-Image); node types covered: UNETLoader, UnetLoaderGGUF, UNETLoaderGGUF (e.g. HiDream GGUF), CLIPLoader, DualCLIPLoader, TripleCLIPLoader, QuadrupleCLIPLoader (e.g. HiDream 4-CLIP)
|
||||
- **LoRA extraction** — lists all LoRA models with `strength_model / strength_clip` values
|
||||
- **Prompt extraction** — lists prompts with POS / NEG badges when positive/negative can be determined; when distinction is not possible (e.g. `SamplerCustomAdvanced`, intermediate nodes, cross-level connections), prompts are shown without a badge as plain **Text**; click any entry to view the full text below
|
||||
- **CLIP Text Encode edit+ support** (v0.3.92) — `CLIPTextEncodeEditPlus` nodes are resolved using the same RAW / EDIT / front / back combination rule as the node itself, correctly merging the linked `text1` (e.g. a Lora Manager's trigger words) with the editable `text_edit` field instead of showing only one side
|
||||
- **Custom multi-output text node fix** (v0.7.14) — nodes whose text output comes from a differently-named widget (e.g. ComfyUI-LiveChatStream's `response`/`positive`/`negative` outputs, backed by `response_text`/`prompt_text`/`negative_text` widgets) previously resolved to nothing when the API-format embedding was used, since the generic key lookup didn't recognize those widget names — LiveChatStream is now resolved by name explicitly; confirmed working with response text up to 40,000 characters
|
||||
- **Prompt actions** — Copy to clipboard, **GenUI:P/N** (set GenerateUI positive/negative prompt), **Prompt:P/N** (set Prompt tab preset positive/negative)
|
||||
- **Format support** — ComfyUI PNG/WebP/MP4/JSON (standard + Flux.2 / Qwen-Image / Z-Image / Ernie Image / WAN2.2 / Mage-Flow / LongCat / Boogu / HiDream E1 / FireRed / Krea-2 Turbo subgraph workflows), SD WebUI, SD Forge, Fooocus
|
||||
- **Format support** — ComfyUI PNG/WebP/MP4/JSON (standard + Flux.2 / Qwen-Image / Z-Image / Ernie Image / WAN2.2 / Mage-Flow / LongCat / Boogu / HiDream E1 / FireRed / Krea-2 Turbo / MOSS-Image subgraph workflows), SD WebUI, SD Forge, Fooocus
|
||||
- **Format note** — supported formats and covered model types are always shown in the left column
|
||||
|
||||
</details>
|
||||
@@ -374,7 +409,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
<summary><h3>Tagger Tab (v0.3.38)</h3></summary>
|
||||
|
||||
- **3 sub-tabs** — Single / Batch / DB for single-image tagging, folder batch processing, and tag database management
|
||||
- **Model support** — WD Tagger (ONNX, NCHW/NHWC auto-detect), SwinV2 (ONNX), DeepDanbooru (.h5, requires TensorFlow); place each model in its own subfolder under `ComfyUI/models/tagger/<model-name>/` containing the `.onnx` + `selected_tags.csv` (or `.h5` + label file)
|
||||
- **Model support** — WD Tagger (ONNX, NCHW/NHWC auto-detect), SwinV2 (ONNX), DeepDanbooru (.h5, requires TensorFlow); place each model in its own subfolder under `ComfyUI/models/tagger/<model-name>/` containing the `.onnx` + `selected_tags.csv` (or `.h5` + label file). **PixAI Tagger v1.0** (v0.8.10) is supported via the community ONNX conversion [Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8](https://huggingface.co/Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8): put one of `model.onnx` / `model_fp16.onnx` / `model_int8.onnx` (only one `.onnx` per folder) together with `selected_tags.csv` and `preprocess.json` in a subfolder. It is detected by `preprocess.json` and uses its own preprocessing (1008px, black padding, RGB, mean/std 0.5; the model already outputs probabilities); tags come from the general / style / copyright / character categories (meta and rating are left out). Selecting it switches the sliders to its recommended defaults (Threshold 0.17 / Char Thr. 0.27); the sliders then offset the recommended per-category values (Threshold moves general and style, Char Thr. moves character and copyright). FP16 needs a GPU to be fast: with both `onnxruntime` and `onnxruntime-gpu` installed the CPU package overwrites the GPU one and CUDA is silently unavailable, so keep only `onnxruntime-gpu`
|
||||
- **Threshold sliders** — General threshold (default 0.35) and Character threshold for `character:` prefix tags (default 0.85); sliders update the display value in real time
|
||||
- **VLM tagging** (v0.3.38, multi-backend: v0.5.4) — optional vision model alongside WD Tagger, running on Ollama, LM Studio, Lemonade, or Unsloth through the same unified backend used by the Prompt tab; results from both models are merged into one comma-separated tag string
|
||||
- **⚙ Settings modal** (v0.5.4) — a Settings button (Single and Batch tabs) opens the same modal component as the Prompt tab's AI Assistant (Backend / Connection / Model / Generation / Unload), but saved under its own `localStorage` key so Tagger's VLM backend/model choice is independent of the Prompt tab's; **Thinking mode** and **Max tokens** apply to VLM requests too, and the **Unload** button frees the selected model (Ollama/LM Studio/Lemonade; Unsloth unsupported) — replaces the old Tagger-only direct Ollama URL/model fields
|
||||
@@ -386,6 +421,22 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>TagSpace Tab (v0.9.0)</h3></summary>
|
||||
|
||||
- **Choose tags visually in a 3D latent space** — WFS's own data is placed in the EmbeddingGemma 2 vector space and shown in 3D (three.js, bundled): **Cards** (ImagePrompt images with their prompt, drawn as thumbnails), **Tags** (words split out of ImagePrompt prompts, the positive prompts embedded in Gallery images, wildcards, the Tagger DB and the Prompt library, colored by source) and **Gallery points** (a sample of indexed Gallery images as a faint background, each carrying the tags from its own prompt). Nearby items have similar meanings
|
||||
- **Build / Update** — collects the cards, adds card images that are not yet indexed to the Gallery semantic index, reads the positive prompt embedded in each indexed Gallery image (followed upstream from the sampler's `positive` input, never the negative; cached in `tagspace_prompts.json`, deleted images are skipped), embeds new tags with Ollama (cached in `tagspace_tags.npz`) and projects everything to 3D with PCA. Prompts are split only at commas (and `|` `{}` `BREAK`), not at spaces; phrases over 5 words are dropped. **Tags** sets how many tags are kept — each source is ranked by how many prompts use a tag and the sources take turns, so the many Gallery prompts do not crowd out the others; **Gallery points** sets how many Gallery images are shown. Images and text form two separate clusters in the same space (modality gap), so each is centered on its own mean (images: the cards) before PCA, which makes cards and their matching tags overlap. First build: about 2.5 min for ~1,000 cards plus about 2 min to read the prompts of 29,000 Gallery images (64% had a readable positive prompt); rebuilds with caches: about 15 s
|
||||
- **Choosing tags** — **All tags** uses every candidate tag instead of the Tags number (up to 20,000); **Min uses** drops tags used in fewer prompts than that (all sources combined); **Wildcards** turns wildcard files on or off as a tag source. Words found only in wildcards are most of the distinct tags (about 37,000 of 47,600 in testing), so All tags without wildcards gives about 10,400 tags (about 6,800 with Min uses 2); ~32,000 points still render at 60 fps. Wildcard YAML is read with a YAML parser, so multi-line (`- >-`) items are included; JSON fragments embedded in prompts (e.g. MOSS-Image bbox layouts) and model file names are not turned into tags
|
||||
- **Explore** — drag to rotate, right-drag to pan, wheel to zoom; hover shows name, thumbnail and prompt/sources; layer checkboxes for Cards / Thumbnails / Tags / Gallery, and source checkboxes (ImagePrompt / Gallery prompts / Tagger DB / Prompt library / Wildcards) that show or hide tags by origin — a tag stays visible while any of its sources is on, and hidden tags are left out of the nearby/search lists. The search box (Enter, any language) places a marker for the text and lists the nearest items
|
||||
- **Search with a card's or Gallery image's tags** — a selected card or Gallery point lists its prompt tags as buttons; pick some and search the space with them (joined in prompt order); while tags are picked, the Add button becomes "Add selected (N)" and adds only those tags. A selected Gallery point also has **Open in Gallery**, which opens its folder in the Gallery tab with the image selected (filters are cleared)
|
||||
- **Make a prompt** — click a card or tag to select it and list nearby items (clicking does not change the prompt); double-click it, press **Add** next to its name under Selection, or press **+** in the list to add it (again to remove); cards and tags are ranked separately and interleaved, so a card also shows its closest tags. Adding a Gallery point adds all tags from that image's prompt as one chip (images without readable tags cannot be added); clicking one lists similar Gallery images together with nearby tags and cards (cards, tags and searches never list Gallery images). The prompt joins the chips with duplicates removed; **Add to GenerateUI (Positive)** appends it to the loaded workflow's positive prompt, **Copy** puts it on the clipboard
|
||||
- **Tagger fill** — Gallery images without a readable prompt (about a third) can be tagged with a Tagger model (ONNX models from the Tagger tab) during Build / Update — either only the sampled Gallery points or every indexed image without prompt tags. Default thresholds, rating labels dropped, results cached per image and model in `tagspace_tagger.json` (stopping keeps finished images). The tags form the **Tagger fill** source and become the tags of those Gallery points. The status shows CUDA/CPU: on the CPU it is about 0.5 s per image, so make sure `onnxruntime-gpu` is active before tagging everything
|
||||
- **Glow / Twinkle** — points are drawn as glowing stars on a night-sky background (Gallery points dimmed so dense areas don't wash out); **Twinkle** makes tags, cards and Gallery points twinkle slowly with individual rhythms (selection/nearby highlights stay steady). Both are toolbar toggles; turning Glow off restores the plain dots
|
||||
- **Hover preview** — hovering the image under Selection or a thumbnail in the Nearby list shows an enlarged preview without changing the selection; the Add / Remove button sits on Selection's title row so it stays reachable while the text scrolls
|
||||
- **Requirements** — Ollama with EmbeddingGemma 2 (Gallery > Semantic settings > Model); a Gallery semantic index built with Gemini must be rebuilt with EmbeddingGemma 2 first
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>Image Edit Tab (v0.3.65)</h3></summary>
|
||||
|
||||
@@ -396,11 +447,11 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **New button** — create a blank canvas with custom dimensions (WxH prompt); clears all existing layers and automatically adds an empty Layer 1 (v0.3.70)
|
||||
- **Tools** — Select (V), Draw (B), Text (T), Shape (S), Fill (G), Mask (🎭), Blur (≈), BG Remove (⬚), Filter (★), Inpaint (🩹); tool options bar updates per active tool
|
||||
- **Select** — click to select; drag to move; drag corner handles to resize; drag circle handle to rotate; Flip H / Flip V / Rotate angle in the options bar; double-click a text layer to re-edit its content
|
||||
- **Draw** — freehand brush; options: color, brush size, blend mode; paints directly onto the active draw layer while all other layers remain visible; brush cursor shown as a size-accurate circle; strokes can start from outside the canvas margin and continue past the edge; **pen tablet support** (v0.6.2) — brush size and opacity respond to pen pressure (mouse behaves exactly as before); the canvas listens for Pointer Events so pen, touch, and mouse all work through the same code path
|
||||
- **Draw** — freehand brush; options: color, brush size, blend mode; paints directly onto the active draw layer while all other layers remain visible; brush cursor shown as a size-accurate circle; strokes can start from outside the canvas margin and continue past the edge; **pen tablet support** (v0.6.2) — brush size and opacity respond to pen pressure (mouse behaves exactly as before); the canvas listens for Pointer Events so pen, touch, and mouse all work through the same code path; **color palette** on the second row — Color 1/Color 2, 20 basic colors, and 20 custom slots (registered and edited colors are saved in the browser); **eyedropper** button next to Color picks a color from the canvas (`Esc` cancels); shortcuts while Draw is active: `D` Draw, `E` Erase, `X` switch Color 1/2, `[` `]` brush size, `B` open or close the brush picker (`Esc` also closes it)
|
||||
- **Text** — click to place; configure font, size, bold/italic, align, and color; placed as an exact-size text layer sized to the measured bounding box; double-click to re-edit
|
||||
- **Shape** — drag to draw geometric shapes (Rect / Ellipse / Line / FreeLine); options: shape type, Rounded toggle (Rect/Ellipse), Fill color, Stroke color + width, Opacity; each committed shape becomes an independent draw layer; Stroke None hidden for Line/FreeLine (stroke always active)
|
||||
- **Fill** (v0.6.2) — bucket fill; click a contiguous same-color region on the active layer to flood-fill it (scanline-based, ported from the Comic Creator plugin's Image tab); **Properties pane**: Mode (Solid / Gradient) — Solid: Color picker; Gradient: Shape (Linear / Radial), a draggable color-stop Ramp (click a stop to select it, drag to reposition, ＋/－ to add/remove stops, color swatch to edit the selected stop's color), and a Direction dial (drag to set angle/radius); both Tolerance (0–255, color-match threshold) and Opacity apply to either mode; Ramp and Direction dial are pointer-event based with pointer capture, so pen tablets work the same as mouse
|
||||
- **Mask** — 7 sub-tools for painting masks onto a dedicated mask layer; click **Mask** on the second row of the Layers header (renamed from "M" and moved to its own row in v0.6.0) to add a mask layer (Mask tool activates automatically); clicking a mask layer in the list also switches to the Mask tool; **Tool Options bar**: sub-tool buttons (Paint / Color / Alpha / Text / Vector / Shape / SAM3), Invert checkbox, Overlay color picker (default red), Blur slider (0–50 px); the overlay is only shown for the currently active mask layer, other mask layers stay hidden until selected (v0.3.70); SAM3 requires Mask Editor One (disabled when not installed); Color / Alpha / Text / Vector / Shape work without Mask Editor One; **Properties pane — Paint**: Mode (Add / Erase), Size (1–200 px), Hardness (0–100%); brush size responds to pen pressure on a tablet (v0.6.2, same as the Draw tool); **MASK EDITOR ONE section** (Paint sub-tool, visible when installed): Select opens the ABR stamp-brush library; ✕ clears back to Circle; when an image brush is active, Hardness is disabled and Spacing (5–100%), Angle (0–359°), Sz Jitter (0–100%), Rot. Jitter (on/off) appear; **Edit in Mask Editor One →** (v0.4.0) — sends the current canvas composite (plus the active mask layer's content, if one is selected) to a MaskEditorOne node on the ComfyUI canvas — the currently selected node if it is one, otherwise the first MaskEditorOne node found in the graph (same selection logic as **Send to LI node**) — and opens its full interactive editor (paint, SAM3, BiRefNet, shapes, etc.) in the ComfyUI browser tab; clicking Apply there imports the result back here as a new mask layer; requires Workflow Studio to have been launched from ComfyUI's top-menu button and at least one MaskEditorOne node to exist on the ComfyUI canvas; **Properties pane — Color**: click canvas to flood-select pixels by color similarity; Mode (Add / Subtract), Tolerance (0–255), Feather % (0–100); **Properties pane — Alpha**: extract the active layer's transparency as a mask; Threshold, Invert checkbox, Extract Alpha button; **Properties pane — Text**: click canvas to open a stamp overlay — enter text, set Font / Size / Bold / Italic / Align / Mode (Add / Erase), Stamp or Ctrl+Enter to apply; multi-line supported; **Properties pane — Vector**: click to add Catmull-Rom spline points; Enter (or click first point) closes and fills the polygon; Backspace removes the last point; Esc resets; Mode (Add / Erase); **Properties pane — Shape**: drag to draw Rect or Ellipse; hold Shift for square/circle; blue dashed outline preview while dragging; Mode (Add / Erase); **Properties pane — SAM3**: enter a text prompt, set Max candidates (3/6/9/12), click Segment; select candidate thumbnails (multi-select), Apply Selected (N) applies with current Mode; **Mask layer A/S operation**: A/S toggle — Add (green) composites normally, Subtract (red) cuts the mask from below; **✂ clipping mask button** clips the layer below to the painted area; brush cursor shown as size-accurate circle; strokes can start from outside the canvas margin and continue past the edge
|
||||
- **Mask** — 7 sub-tools for painting masks onto a dedicated mask layer; click **Mask** on the second row of the Layers header (renamed from "M" and moved to its own row in v0.6.0) to add a mask layer (Mask tool activates automatically); clicking a mask layer in the list also switches to the Mask tool; **Tool Options bar**: sub-tool buttons (Paint / Color / Alpha / Text / Vector / Shape / SAM3), Invert checkbox, Overlay color picker (default red), Blur slider (0–50 px); the overlay is only shown for the currently active mask layer, other mask layers stay hidden until selected (v0.3.70); SAM3 requires Mask Editor One (disabled when not installed); Color / Alpha / Text / Vector / Shape work without Mask Editor One; **Properties pane — Paint**: Mode (Add / Erase), Size (1–200 px), Hardness (0–100%); brush size responds to pen pressure on a tablet (v0.6.2, same as the Draw tool); **MASK EDITOR ONE section** (Paint sub-tool, visible when installed): Select opens the ABR stamp-brush library; ✕ clears back to Circle; when an image brush is active, Hardness is disabled and Spacing (5–100%), Angle (0–359°), Sz Jitter (0–100%), Rot. Jitter (on/off) appear; **Edit in Mask Editor One →** (v0.4.0) — sends the current canvas composite (plus the active mask layer's content, if one is selected) to a MaskEditorOne node on the ComfyUI canvas — the currently selected node if it is one, otherwise the first MaskEditorOne node found in the graph (same selection logic as **Send to LI node**) — and opens its full interactive editor (paint, SAM3, BiRefNet, shapes, etc.) in the ComfyUI browser tab; clicking Apply there imports the result back here as a new mask layer; requires Workflow Studio to have been launched from ComfyUI's top-menu button and at least one MaskEditorOne node to exist on the ComfyUI canvas; **Properties pane — Color**: click canvas to flood-select pixels by color similarity; Mode (Add / Subtract), Tolerance (0–255), Feather % (0–100); **Properties pane — Alpha**: extract the active layer's transparency as a mask; Threshold, Invert checkbox, Extract Alpha button; **Properties pane — Text**: click canvas to open a stamp overlay — enter text, set Font / Size / Bold / Italic / Align / Mode (Add / Erase), Stamp or Ctrl+Enter to apply; multi-line supported; **Properties pane — Vector**: click to add Catmull-Rom spline points; Enter (or click first point) closes and fills the polygon; Backspace removes the last point; Esc resets; Mode (Add / Erase); **Properties pane — Shape**: drag to draw Rect or Ellipse; hold Shift for square/circle; blue dashed outline preview while dragging; Mode (Add / Erase); **Properties pane — SAM3**: enter a text prompt, set Max candidates (3/6/9/12), click Segment; select candidate thumbnails (multi-select), Apply Selected (N) applies with current Mode; **Mask layer A/S operation**: A/S toggle — Add (green) composites normally, Subtract (red) cuts the mask from below; **✂ clipping mask button** clips the layer below to the painted area; brush cursor shown as size-accurate circle; strokes can start from outside the canvas margin and continue past the edge Since v0.8.13 the sent image is also saved to the node's `node.properties.bg_image` (via `node._persistBgDataUrl`, Mask Editor One v0.1.12 or later; ignored on older versions) and the mask layer data carries the image `width`/`height`, so the image and the mask preview survive a ComfyUI page reload or workflow tab switch instead of an older image being restored.
|
||||
- **Blur** — **Whole Blur**: Gaussian blur to the entire active layer with intensity slider (1–50 px); **Whole Mosaic**: pixelation mosaic with block-size slider (5–100 px); **Rect Blur / Rect Mosaic**: enable the toggle then drag a rectangle on the canvas to apply blur or mosaic to that region only (blue preview for blur, orange for mosaic); Rect Blur and Rect Mosaic are mutually exclusive; all operations support Undo
|
||||
- **BG Remove** — remove the background from the active layer; **Lightweight (@imgly)**: runs entirely in the browser via CDN (no server required; ~40 MB model downloaded on first use); **BiRefNet**: high-quality removal via Mask Editor One's Python backend (requires Mask Editor One installed and `birefnet.safetensors` in `ComfyUI/models/background_removal/`); **New Layer** option (default on) adds the result as a new layer above the original; when off, replaces the active layer in-place
|
||||
- **Filter (G'MIC)** — apply G'MIC-Qt filter effects to the active image layer; click **Edit with G'MIC** to launch G'MIC-Qt GUI; select a filter and click OK — a "G'MIC-Qt filter output" window appears; close it to save the result back as the active layer; requires G'MIC-Qt Standalone installed and path configured in Settings → G'MIC-Qt Executable Path
|
||||
@@ -418,7 +469,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
<details>
|
||||
<summary><h3>Video Tab (v0.5.0 – v0.7.4)</h3></summary>
|
||||
|
||||
- **Dedicated video generation UI, independent of GenerateUI** — a purpose-built tab for MiniMax H3 (Image to Video), LTX-2.5 (Text / Image / First & Last Frame to Video), and Wan2.2 (Text to Video) workflows (LTX-2.5/Wan2.2: v0.5.4); does not share state with the GenerateUI tab, and model selection (UNet/CLIP/VAE/Audio VAE) is fixed to the loaded workflow's own defaults rather than editable here
|
||||
- **Dedicated video generation UI, independent of GenerateUI** — a purpose-built tab for MiniMax H3 (Image to Video), LTX-2.5 (Text / Image / First & Last Frame to Video), Wan2.2 (Text to Video), and FastH3 OmniRef (MiniMax H3 Reference to Video) workflows (LTX-2.5/Wan2.2: v0.5.4); does not share state with the GenerateUI tab, and model selection (UNet/CLIP/VAE/Audio VAE) is fixed to the loaded workflow's own defaults rather than editable here
|
||||
- **Plan subtab** (center panel, batch redesign in v0.5.5) — a **plan** is a sequence of timeline blocks, each a fully self-contained generation job with its own Prompt, Duration, and First/Last Image; split the timeline with **＋ Split** / **＋ Block** / **Delete**, then **Run** to generate every block in order (Pause/Cancel work mid-run) — a plan with a single block behaves exactly like the old one-shot form. First Image can be None, an explicit image, or "use previous block's last frame" (chaining — an explicit Last Image on the previous block always wins over extracting one from its generated output). Save / Save As / Clear a plan (`ws_videoplan_*.json`, with a PNG contact-sheet thumbnail that embeds the full plan data so dropping the image back on works too)
|
||||
- **Base settings row** (topbar, v0.5.7) — Aspect Ratio / Megapixels / Multiple / Duration (seconds) / Noise Seed on one row — Duration always reflects the currently selected block (each block keeps its own value), the rest are plan-wide — plus a second row of model-specific fields: Width / Height (become directly editable for workflows with no Resolution Selector node, e.g. Wan2.2 and LTX-2.5's First & Last Frame to Video), Turbo Mode / Strength / Steps (MiniMax H3's turbo LoRA — Turbo Mode doubles as Wan2.2's own turbo toggle, sharing one field), and Prompt Enhance (LTX-2.5 only). Every field in both rows is enabled only when the loaded workflow actually has it — see Field highlight color below (one exception: Wan2.2's Text-to-Video variant doesn't expose Noise Seed at all, so it alone leaves that field disabled)
|
||||
- **Edit subtab** (v0.7.0) — a lightweight clip timeline for trimming and concatenating video/image clips outside of generation: add clips via the Video Source panel's **Add to Edit** button or the Asset subtab's **Send to Edit** button, drag blocks to reorder (or use the ◀/▶ toolbar buttons), Duplicate/Delete the selected clip, Clear to empty the timeline. The trim panel for a selected video clip shows a Clipchamp-style scrubber — a time ruler, a draggable playhead synced to the Source preview, and draggable in/out handles with time badges (image clips get a simple hold-duration field instead); the playhead's own time badge sits a row above the in/out handle badges (v0.7.2) so it stays legible even when the playhead lands right on a trim point, and the scrubber has extra headroom/height and spacing from the clip name above it (v0.7.2). **▶ Preview** plays every ready clip back-to-back (trimmed to its in/out points) in the result preview, and **Export** builds and runs the actual ComfyUI graph (trim → crop → concatenate if more than one clip → save), tagging the result into Video Temp automatically. Multi-clip exports keep each clip's own audio (v0.7.4 — previously they were always silent). **Clear** empties the clips and also resets the Audio section to its defaults (v0.7.4)
|
||||
@@ -426,15 +477,17 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Edit: Audio / BGM** (v0.7.4) — the export panel's **Audio** section sets the soundtrack: **Keep original audio** (with its own volume in dB) keeps every clip's audio aligned to its segment even across multiple clips (image clips contribute silence), and one **BGM** file is laid over the whole video with its own volume and start offset (a longer BGM is cut to the video length). Pick the BGM with **BGM...**, by dropping an audio/video file onto the Audio section, or with **Set as BGM** on a video in the Asset subtab (that video's audio track is used; videos without audio are rejected). Built entirely inside the export graph with ComfyUI core audio nodes (EmptyAudio / AudioMerge / AudioConcat / LoadAudio / TrimAudioDuration / AudioAdjustVolume) passed to ConcatenateVideo's `complete_audio` — no extra dependencies. ▶ Preview plays the BGM alongside the clips
|
||||
- **Edit: Text overlays** (v0.7.4) — each clip can carry any number of text overlays (**+ Add text** under the trim panel): text, start/end seconds within the clip, size as % of frame height, color, 9-position anchor, outline, and an optional background box. They follow the clip when it's reordered, are previewed live over the Source/Result panes, and are burned in on Export as a final pass (PyAV decode → Pillow → H.264 encode, frame by frame so memory stays flat; the soundtrack is remuxed untouched). Japanese/CJK text renders with a system CJK font (Meiryo / Yu Gothic / Noto Sans JP on Windows)
|
||||
- **Edit: Crop** (v0.7.4) — **Edit crop** puts a draggable/corner-resizable rectangle over the Source preview (optionally locked to 16:9 / 9:16 / 1:1 / 4:3 / 3:4); **Done** keeps it, **Cancel** restores the crop from before editing, and switching to another subtab/tab ends editing like Done. Export applies it with ComfyUI core `VideoCrop`; the output size is the first video clip's cropped size, and other clips of a different size are center-fit to it. Text overlay previews are drawn relative to the cropped frame
|
||||
- **Edit: Convert audio** (v0.8.9) — a **Convert audio...** button sits next to the crop buttons (separated by `|`); it converts the selected video clip's audio track to FLAC / MP3 / Opus / M4A / WAV with the same dialog as Gallery's **Convert...** (bitrate, sample rate, channels), and saves the result into the output/video folder so it shows up in Gallery. Delete-original and carry-over-gallery-meta options are not offered here. Server route: `POST /api/wfm/video/edit/convert-audio`
|
||||
- **Edit: Undo / Redo** (v0.7.4) — **↶ / ↷** in the timeline toolbar (or Ctrl+Z / Ctrl+Y / Ctrl+Shift+Z outside text fields) step through edits to clips, trims, crops, text and audio settings; a deleted clip comes back without re-uploading. History resets when a project is loaded
|
||||
- **Edit project persistence** (v0.7.2) — **Save** / **Save As** / **Load** buttons in the Edit subtab's export panel persist the timeline itself (clip order, trim points, and which uploaded server-side file each clip refers to — plus crops, text overlays and audio/BGM settings since v0.7.4) as a `ws_videoeditproj_*.json` project file; Load re-fetches and re-probes each clip from its saved server reference, so the original files need to still exist
|
||||
- **Asset subtab** (center panel, next to Plan/Edit — moved here from the sidebar in v0.7.0) — browses generated videos/images through three groups: **Video Assets** (a manually-curated keeper group you organize yourself, on your own schedule), **Video Temp** (every plan run's output lands here automatically regardless of whether the plan itself was saved — a **Clear Video Temp** button in the Gallery tab's detail panel empties it in one click), and **All Video Assets** (the union of both); a filename search box, a video/image type filter dropdown (All Types / Video Only / Image Only, v0.7.2), and a thumbnail-grid/table (▦/☰) view toggle sit above the list (v0.7.0). Selecting an item shows it in the center preview and lets you tag/memo it, **Send to Edit** (queues it as a clip in the Edit subtab), **Set as BGM** for videos (v0.7.4 — uses its audio track as the Edit BGM), jump straight to the full Gallery tab, or — for images only — **Set as First Image** / **Set as Last Image** (v0.7.0, uploads it straight into the currently-selected Plan block and switches to the Plan subtab)
|
||||
- **Project subtab** (sidebar) — lists saved plan files while the center Plan subtab is active, or saved Edit projects while the center Edit subtab is active (v0.7.2 — thumbnail/block-or-clip-count/last-updated) — click one to load it into the matching editor, or delete it with the × button; the panel collapses back down when its own tab is clicked again, and switching Plan↔Edit while it's open refreshes the list in place
|
||||
- **Two center previews** (v0.5.5) — the left preview shows either the video currently selected in the Asset subtab or whatever's dropped/picked in the **Video Source** panel to its left (**Clear** resets it to empty); the right preview always shows the most recently generated clip; both preview frames are a fixed height (v0.7.0) so switching sources never shifts the Plan/Edit/Asset panel below them. The Frame/GIF tools act on whichever preview was touched most recently
|
||||
- **Register as Asset** (v0.8.9) — a **Register as Asset** button in the Video Source panel (next to **Add to Edit** and **Clear**) puts the loaded video into the **Video Assets** group so it appears in the Asset subtab: local files are uploaded and copied into output/video (Gallery only lists the output folder), sources already in output are just tagged. The Video Source panel was also widened so the file name stays on one line and the three buttons fit in a row. Server route: `POST /api/wfm/video/asset/register`
|
||||
- **First Frame / Last Frame are both optional** — leaving First Image empty runs the workflow text-only (the node is simply left unconnected — a "bypass"); workflows that don't even include a LoadImage node for First Frame are supported out of the box (Wan2.2's Text-to-Video graph has neither slot at all), and a LoadImage node is only injected into the graph on the fly if an image is actually supplied
|
||||
- **Semantic node lookup, not hardcoded IDs** (generalized in v0.5.4) — detects which of the three families is loaded from its subgraph's internal node types (or, for Wan2.2, the subgraph's own name — it has no distinctive internal node type), then locates its Resolution Selector and LoadImage node(s) by the subgraph's *public input label* (`prompt`, `duration`, `width`, `first_frame`, `last_frame`) rather than a fixed slot index — so it keeps working even though the three families expose those inputs under different internal names; a workflow no longer needs to match any bundled reference file's exact node IDs, and no fallback test workflow is auto-loaded on tab open
|
||||
- **Field highlight color** — Prompt / Duration / First Image / Last Image / Aspect Ratio / Megapixels / Multiple / Noise Seed / Width / Height / Turbo Mode / Strength / Steps / Prompt Enhance labels turn the same highlight color as the GenerateUI Model tab (shared setting, customizable in the Settings tab — see below) when the loaded workflow actually exposes that field; e.g. a Wan2.2 workflow (no Resolution Selector node) leaves Aspect Ratio / Megapixels / Multiple unhighlighted while Width / Height light up instead
|
||||
- **Load in Video** (Workflow tab) — load any workflow containing a MiniMax H3 Image-to-Video node, an LTX-2.5 Text/Image/First&Last-Frame-to-Video node, or a Wan2.2 Text-to-Video node straight into this tab from the Workflow tab's side-panel toolbar (next to Load in GenerateUI) — starts a fresh single-block plan seeded from the workflow's own current values; unsupported workflows show an error toast instead of loading
|
||||
- **Load in Video** (Workflow tab) — load any workflow containing a MiniMax H3 Image-to-Video node, an LTX-2.5 Text/Image/First&Last-Frame-to-Video node, a Wan2.2 Text-to-Video node, or a FastH3 OmniRef (MiniMax H3 Reference-to-Video) node — for FastH3 / Pika-H3-Flex the First Image and Last Image fields become Reference Image (ref_image_0) and Reference Image 2 (ref_image_1); leaving either one empty (None / Clear) disconnects that reference input even if the workflow had an image wired to it; Reference Image 2 is never used as the previous block's last frame for "Use previous block's last frame" (the actually generated last frame is used instead) straight into this tab from the Workflow tab's side-panel toolbar (next to Load in GenerateUI) — starts a fresh single-block plan seeded from the workflow's own current values; unsupported workflows show an error toast instead of loading
|
||||
- **Frame tab** (right panel) — captures the active preview's current playback position as a still frame: play/seek to the position you want, click **Capture Frame** (client-side canvas, no server round-trip), then **Save to Output** to write it as a PNG into ComfyUI's own Output folder (auto-numbered, same pattern as other generated images); an optional **Add to Video Assets group** checkbox (v0.7.0) also tags the saved frame into Gallery's Video Assets group
|
||||
- **GIF tab** (right panel) — converts the active preview to an animated GIF; set Start/End (seconds), FPS, and an optional Max Width, then **Convert to GIF** — the server decodes the video (PyAV) and encodes an animated GIF (Pillow), both dependencies ComfyUI itself already requires, so no ffmpeg binary or extra install is needed; the result is saved into ComfyUI's Output folder and previewed inline
|
||||
- **Playback volume** — the preview player's volume is saved to the Settings tab and shared across every video player in the plugin (Video tab, Gallery tab), so it no longer resets to maximum each time a different video is opened
|
||||
@@ -442,7 +495,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>AI TOOL Tab (v0.3.14)</h3></summary>
|
||||
<summary><h3>AI TOOL Tab (v0.3.14 – v0.9.5)</h3></summary>
|
||||
|
||||
- **4-pane layout** — Translation | Chat | TOOLS | Settings; all panes always visible simultaneously; no sub-tab switching required
|
||||
- **Translation pane** — translate text between Japanese, English, Chinese, or a custom Free language using Ollama, LM Studio, Lemonade, or Unsloth; language selectors with ⇄ swap button (swaps both language selectors and text content); selections saved automatically
|
||||
@@ -456,13 +509,21 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Chat pane — image-to-image (I2I)** (v0.3.74) — when an image is attached and `generate_image` is invoked, the attachment is uploaded and swapped into the target workflow's LoadImage node instead of running text-to-image; the target workflow is the one currently loaded in GenerateUI, or a dedicated I2I workflow configured in the Settings pane
|
||||
- **Chat pane — SVG generation, no ComfyUI workflow involved** (v0.3.77) — the LLM can also produce SVG graphics directly as text; if a reply contains SVG code (wrapped in a markdown code fence or written directly as a bare `<svg>...</svg>`), it's rendered inline as a preview and a "Save to Gallery" button writes it as a `.svg` file to the Gallery tab's output folder; unlike the Tool Calling image generation flow above, this is plain text pattern matching, so it works with any model regardless of tool-calling support
|
||||
- **Chat pane — Skills** (v0.3.79) — select a `.md` skill file from a dropdown above the message list to prepend its instructions to the conversation as a system prompt (not shown in the visible chat log); a pencil button opens an in-pane manager (list + inline editor, same pattern as the Prompt tab's wildcard file manager) to create, edit, or delete skill files stored in `user/default/Workflow-Studio/ai_skills/`; a file's `---`-delimited frontmatter (`name`/`description`) is shown in the dropdown; two starter skills are bundled — **SVG Icon Generator** (explicitly decomposes the subject into parts before drawing, which markedly improves recognizability from small local models) and **Skill Creator** (interviews you conversationally and drafts a new skill as a ```skill code block; a "Save as new skill" button appears under such replies and opens the manager pre-filled with the draft)
|
||||
- **Chat pane — Skills v2** (v0.9.5) — skills keep the portable `name`/`description` frontmatter (Agent Skills / SKILL.md style) and add optional `metadata:` keys: `output` (`prompt` or `svg` — the reply is checked and, if the model tried to output one but it is broken, it is asked once to fix it), `tools` (`[]` = never start image generation), `temperature`, `think`, and `context` (`workflow` summary, `current-prompt`, `workflow-detail`, `logs`). Six built-in skills ship in `py/services/builtin_skills/` with a version number: untouched copies are upgraded automatically, edited ones keep your text and show a ↻ button. A ```` ```prompt ```` block in a reply gets **Apply to GenerateUI** / **Generate with this prompt** / **Copy** buttons, and replies are rendered as Markdown (HTML-escaped first)
|
||||
- **Built-in skills** (v0.9.5) — **T2I Prompt Enhancer** (writes natural sentences for Qwen-Image / FLUX / Z-Image, Danbooru tags for Illustrious / Pony, phrases for realistic SDXL, based on the workflow loaded in GenerateUI), **Image to Prompt** (recreates an image attached in TOOLS), **Storyboard to Lab**, **Workflow Doctor**, **SVG Icon Generator** v2 (line/flat/duotone/mono styles) and **Skill Creator** v3. The prompt text currently in GenerateUI is only passed to the model when your message refers to it, because small models copied an old negative prompt into new ones even when told not to
|
||||
- **Chat pane — Image check & refine loop** (v0.9.5) — with **Check images** on, the chat model turns the request into 3–8 yes/no checks and an Ollama decision model with image input (`d3:4b` by default; `clef-flash` / `clef` also work) answers them with probabilities: ✔ ≥ 85%, ? 30–85%, ✘ < 30%. Swap checks ("the hat is not on the girl") catch the attribute mix-ups the judge otherwise passes. **Fix missing items and regenerate** sends only the unmet items back for a rewritten prompt; 0–3 automatic rounds are configurable. SVG replies are rendered and checked part by part the same way
|
||||
- **Chat pane — Workflow Doctor** (v0.9.5) — diagnoses the workflow loaded in GenerateUI from facts checked on every message: node types that are not installed, model files or options missing from ComfyUI's lists (with the closest existing names), broken links, unconnected core-type required inputs, the last execution error from `/history`, and the latest error block of the ComfyUI log (no false positives across 77 local workflows)
|
||||
- **Chat pane — Storyboard to Lab** (v0.9.5) — a ```` ```labplan ```` reply becomes a Lab plan (one scene per iteration, T2I mode, identical character descriptions, fixed seed); **Open in Lab** loads it into GenerateUI > Lab
|
||||
- **Chat pane — Agent Skills folders** (v0.9.5) — **Import** in the skill manager adds a `.zip` containing `SKILL.md` (or a single `.md`); only text files are kept, scripts are never extracted or run, and zip-slip / size limits apply. Reference files are listed to the model and read on demand through a `read_skill_file` tool (progressive disclosure; needs a tool-capable model)
|
||||
- **TOOLS pane — OCR** (v0.9.5) — **OCR (extract text)** reads text with an Ollama OCR model (LightOnOCR-3 by default; PaddleOCR-VL, GLM-OCR and DeepSeek-OCR are listed too). Ported from Comic Creator's OCR tab: 1600 px white-background JPEG, temperature 0, grayscale retry on empty / looping output. **Show text positions** (LightOnOCR / DeepSeek-OCR) overlays numbered boxes; clicking one selects its line. **To Translation** / **To Chat** pass any TOOLS result on
|
||||
- **Collapsible Translation / Settings panes** (v0.9.5) — the arrow in each header folds the pane into a thin strip (click the strip to reopen); the state is remembered
|
||||
- **Settings pane** — choose backend (Ollama / LM Studio / Lemonade / Unsloth), set the API URL, test connection, select a model (with refresh button), and configure Free language names for translation source and destination
|
||||
- **Unsloth backend** (v0.3.99) — Unsloth Desktop's OpenAI-compatible API requires an API key even for local access unless its **Keyless API access → "Chat and inference"** setting is on; unlike the other backends it's proxied server-side (`py/routes/unsloth_routes.py`, `POST /api/wfm/unsloth/proxy`) so the key never reaches the browser: set `UNSLOTH_API_KEY` in a `.env` file in the plugin folder (copy `.env.example`, loaded via `python-dotenv` in `prestartup_script.py`) and restart ComfyUI — or, since v0.7.7, leave `UNSLOTH_API_KEY` unset and turn on Keyless API access → "Chat and inference" in Unsloth Desktop (requests are then relayed without an `Authorization` header, still only to localhost; that Unsloth setting also opens the API to your private LAN, and "Allow tools" should stay off). A 401 with no key set returns a message explaining both options; also requires **"Auto model switching (OpenAI API)"** to be turned on in the Unsloth Desktop app's own Settings screen — with it off, requests fail with HTTP 404 unless a model was already manually loaded in the Unsloth Studio UI
|
||||
- **Decision API client** (v0.7.5, foundation) — `static/js/decision-client.js` calls Unsloth Desktop's Decision API (`/v1/systemone`, TypeSafe-compatible Laya decision models) through the same key-attaching server proxy: `decide(state, questions)` / `decideMany`, question builders `noul`/`choice`/`score`, and `isYes`/`pickChoice`/`pickScore` helpers that threshold on `probabilities` (as Unsloth recommends) and flag low-probability answers as not confident. Requires **Decision API → Serve requests** enabled in Unsloth Desktop. Text only — Laya has no image input. **Ollama 0.35+ backend** (v0.7.7) — Ollama's own `/v1/systemone` (decision models `tev1` 4B / `tev1:0.8b` 812 MB / `nimble` 9B, pulled with `ollama pull`; no key, called directly from the browser); the model list shows only installed models whose `capabilities` include `decision`, with a ↻ refresh button and a pull hint when none are installed. Connection settings live in the Settings tab's own **Decision Model** section (v0.7.6): backend (Unsloth / Ollama), URL, model and default probability threshold, stored under `wfm_decision_settings` independently of the AI TOOL / Tagger backends, with a Test button (one yes/no question). No feature uses it yet
|
||||
- **Decision API client** (v0.7.5, foundation) — `static/js/decision-client.js` calls Unsloth Desktop's Decision API (`/v1/systemone`, TypeSafe-compatible Laya decision models) through the same key-attaching server proxy: `decide(state, questions)` / `decideMany`, question builders `noul`/`choice`/`score`, and `isYes`/`pickChoice`/`pickScore` helpers that threshold on `probabilities` (as Unsloth recommends) and flag low-probability answers as not confident. Requires **Decision API → Serve requests** enabled in Unsloth Desktop. **Ollama 0.35+ backend** (v0.7.7) — Ollama's own `/v1/systemone` (decision models `tev1` 4B / `tev1:0.8b` 812 MB / `nimble` 9B / `clef` 27B vision-capable, pulled with `ollama pull`; no key, called directly from the browser); the model list shows only installed models whose `capabilities` include `decision`, with a ↻ refresh button and a pull hint when none are installed. **Image input** (v0.7.14, ported from ComfyUI Comic Creator) — most decision models are text-only, but Ollama's `clef` accepts images directly: `decide()` takes an optional 4th `images` argument (base64, no `data:` prefix), `imageToBase64()` converts a Blob/data URL/image URL, and `supportsDecisionVision()` checks whether the currently-configured model accepts them (via `/api/show`'s `capabilities`/`projector_info`, since not every Ollama build reports `vision` for decision models) — call it before passing images, since a non-vision model errors out on them; confirmed working end to end against `clef`. Connection settings live in the Settings tab's own **Decision Model** section (v0.7.6): backend (Unsloth / Ollama), URL, model and default probability threshold, stored under `wfm_decision_settings` independently of the AI TOOL / Tagger backends, with a Test button (one yes/no question). No feature uses it yet
|
||||
- **Settings pane — Chat Image Generation** (v0.3.73) — "Use dedicated workflow" checkbox lets Chat-triggered image generation use a saved workflow of your choice instead of the one currently loaded in GenerateUI, without disturbing what's shown there (same pattern as Image Edit's Inpaint dedicated workflow)
|
||||
- **Settings pane — Chat I2I Generation** (v0.3.74) — same "Use dedicated workflow" pattern as above, but for image-to-image generation when an image is attached; independent of the text-to-image dedicated workflow setting
|
||||
- **Settings pane — Generation** (v0.3.98) — **Thinking mode** toggle (off by default; sends Ollama's `think` flag, and regardless of backend, any `<think>...</think>` block is stripped from the output when off so reasoning models don't clutter Translation/Chat/TOOLS results) and **Max tokens** (Ollama: `options.num_predict`; LM Studio/Lemonade/Unsloth: `max_tokens`; `0` = unlimited)
|
||||
- **Model unload** (v0.5.4) — an Unload button next to the model selector frees VRAM/memory by unloading the currently selected model from Ollama (`keep_alive:0` via `/api/generate`, the officially documented method — Ollama has no dedicated unload endpoint), LM Studio (`/api/v1/models/unload`), or Lemonade (`/api/v1/unload`); Unsloth exposes no equivalent API and the button is disabled for that backend
|
||||
- **Model unload** (v0.5.4) — an Unload button next to the model selector frees VRAM/memory by unloading the currently selected model from Ollama (`keep_alive:0` via `/api/generate`, the officially documented method — Ollama has no dedicated unload endpoint), LM Studio (`/api/v1/models/unload`), or Lemonade (`/api/v1/unload`); Unsloth exposes no equivalent API and the button is disabled for that backend. **Since v0.7.13**, with the Ollama backend the Unload button here and in the Prompt / Tagger ⚙ Settings modals unloads **every** model loaded in Ollama on localhost, decision models included — the same server step as Settings → VRAM Management's **Unload all now** (a toast shows the models unloaded and the free VRAM before → after)
|
||||
- **Settings shared** — settings saved to `localStorage` under `wfm_ai_settings`; shared with the Library panel's AI TOOL tab so configuration is consistent across both interfaces
|
||||
- **Backend support** — Ollama (`/api/generate` for text, `/api/chat` for conversations, `/api/tags` for model list); LM Studio and Lemonade via the OpenAI-compatible API directly from the browser (`/v1/chat/completions`, `/v1/models`); Unsloth via the same OpenAI-compatible API but relayed through a server-side proxy that attaches the `Authorization: Bearer` header; VLM images sent as base64 (`images:[]` for Ollama, `image_url` content block for LM Studio/Lemonade/Unsloth)
|
||||
- **URL security** — backend URL validated via `new URL()` to enforce `http://` or `https://` scheme
|
||||
@@ -475,14 +536,15 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
|
||||
- **Tab layout (W / N / P / M / I / A)** — compact single-letter tabs with full name shown on hover
|
||||
- **W — Workflows tab** — browse favorite workflows (All / ★ Favorites / Groups / By Badge sub-tabs), ★ star shown for favorites in All view
|
||||
- **N — Nodes tab** — browse favorite nodes (All / ★ Favorites / Groups / Sets / 📂 Category / 🧩 Package sub-tabs), ★ star shown for favorites in All view
|
||||
- **N — Nodes tab** — browse favorite nodes (All / ★ Favorites / Groups / Sets / 📂 Category / 🧩 Package sub-tabs), ★ star shown for favorites in All view; **Alt+click** a node to add it to (or remove it from) Favorites; the tab now opens without waiting for `/object_info` (v0.8.0)
|
||||
- **Category sub-tab** — dropdown to filter nodes by top-level category
|
||||
- **Package sub-tab** — dropdown to filter nodes by custom node package name
|
||||
- **M — Models tab** — browse installed models (All / ★ Favorites / Groups / By Type sub-tabs); LoRA groups show an **All N LoRAs** item — drag to canvas to place a `Lora Loader (LoraManager)` node with all LoRAs pre-loaded
|
||||
- **P — Prompts tab** — browse prompt presets with All / ★ Favorites / Categories sub-tabs; **Groups sub-tab** (row 2) — view presets by group (shared with the Batch tab's `wfm_prompt_preset_groups`)
|
||||
- **I — Information tab** — drop a ComfyUI-generated PNG/WebP/MP4 or workflow JSON in the side panel to view its metadata; detects LoRAs from `LoraLoader`, `LoraLoaderModelOnly`, and `Lora Loader (LoraManager)` nodes (API format supported); supports `UnetLoaderGGUF` and `QuadrupleCLIPLoader` node types; preview area fixed at 110px
|
||||
- **I — Information tab** — drop a ComfyUI-generated PNG/WebP/MP4 or workflow JSON in the side panel to view its metadata; detects LoRAs from `LoraLoader`, `LoraLoaderModelOnly`, `Lora Loader (LoraManager)`, and `ImageMetadataLoRALoader` nodes (API format supported); supports `UnetLoaderGGUF` and `QuadrupleCLIPLoader` node types; preview area fixed at 110px
|
||||
- **MP4 embedded metadata** (v0.5.1) — reads the `workflow`/`prompt` container tags ComfyUI's `SaveVideo` writes into the mp4's `moov/udta/meta` box, parsed entirely client-side (same box-walking logic as the Gallery tab's Metadata sub-tab); also recognizes all-in-one video-generation nodes that carry their prompt directly on a `prompt` input rather than a separate `CLIPTextEncode`-style node (e.g. `MiniMaxH3ImageToVideo`)
|
||||
- **A — AI TOOL tab** — Translation, Chat, TOOLS, and Settings sub-tabs powered by Ollama, LM Studio, Lemonade, or Unsloth; Chat supports multi-turn conversations (full history sent each turn); TOOLS includes VLM image analysis and wildcard generation; settings (backend, URL, model) shared with the SPA AI TOOL tab via `localStorage`
|
||||
- **VRAM (Ollama) → Unload all now** (v0.7.13) — in the A tab's Settings sub-tab; unloads every model loaded in Ollama on localhost, decision models included (same as Workflow Studio's Settings → VRAM Management)
|
||||
- **model sub-tab** — Checkpoint, VAE, Diffusion Model, and Text Encoder; drag items to canvas to place the corresponding loader node (Checkpoint → `CheckpointLoaderSimple`, VAE → `VAELoader`, Diffusion Model → `UNETLoader`, Text Encoder → `CLIPLoader`); double-click also places at canvas center
|
||||
- **lora sub-tab** — detects LoRAs from `LoraLoader`, `LoraLoaderModelOnly`, and `Lora Loader (LoraManager)` nodes (API format `inputs.loras.__value__` supported); shows `strength_model / strength_clip` values; drag individual LoRA to place `LoraLoader`; **Multiple LORA** section (appears for 1+ LoRAs) drags all LoRAs into a single `Lora Loader (LoraManager)` node with LoRA syntax pre-filled
|
||||
- **Prompts sub-tab** — POS / NEG badge list; drag a prompt to place `CLIPTextEncode` with text pre-filled; click any entry to view full text + Copy button
|
||||
@@ -530,6 +592,8 @@ Restart ComfyUI after installation.
|
||||
|
||||
Sample workflows are included in the `workflows/` folder. You can open them directly in ComfyUI via drag & drop, or load them from the Workflow tab.
|
||||
|
||||
Samples for the Feeder tab: `ws_sdxl_image_feeder_i2i` / `ws_dwpose_image_feeder` (Image Loop mode, require the `comfyui-image-feeder` custom node) and `ws_sdxl_gallery_feeder_i2i` (v0.8.3, SDXL I2I with the bundled **WFS_GalleryFeeder** node — no extra custom node needed; put images in the `__Feeder__` Gallery group and use the Feeder tab's Gallery mode).
|
||||
|
||||
> **Note:** Some sample workflows require additional custom nodes.
|
||||
> If a node is shown as missing (red/unknown) after loading, install the required custom nodes via **ComfyUI Manager** or by cloning the repository from GitHub into your `ComfyUI/custom_nodes/` directory.
|
||||
>
|
||||
@@ -542,6 +606,17 @@ Sample workflows are included in the `workflows/` folder. You can open them dire
|
||||
|
||||
---
|
||||
|
||||
## Manual
|
||||
|
||||
A beginner-oriented manual (covers v0.9.9, text-only edition without screenshots) is available in [`docs/manual/`](docs/manual/):
|
||||
|
||||
- English: [`docs/manual/en/`](docs/manual/en/) — ZIP: `Workflow_Studio_Manual_v0.9.9_en.zip`
|
||||
- 日本語: [`docs/manual/`](docs/manual/) — ZIP: `Workflow_Studio_Manual_v0.9.9.zip`
|
||||
|
||||
An EPUB edition with screenshots (Japanese and English, v0.9.9) is available on [Google Drive](https://drive.google.com/drive/folders/14mOHcnLYrFgNrOo6fy7ELyFjjWrAoU-_?usp=sharing).
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### Launch
|
||||
@@ -576,14 +651,19 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
|
||||
### Optional
|
||||
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features; also runs PLaMo 3 Translate for Settings → Prompt Translation (`ollama pull hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M`, or the Download button there)
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** in [Ollama](https://ollama.com) (no key; `ollama pull embeddinggemma-2:740m` or `embeddinggemma-2:440m`)
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
|
||||
#### Backend connection security (v0.7.12)
|
||||
|
||||
Server-side proxies for Ollama, LM Studio, Lemonade, Unsloth, the Tagger VLM and Eagle only connect to **localhost / 127.0.0.1 / ::1** (any port); `http(s)` only, no credentials in the URL, and redirects are never followed. To use a backend on another machine (e.g. an Ollama server on your LAN), list it in the `WFS_ALLOWED_BACKEND_HOSTS` environment variable (comma-separated `host` or `host:port`, e.g. `192.168.1.20:11434,gpu-box`) and restart ComfyUI. If you start ComfyUI with `--listen`, anyone who can reach it can use these routes, so only do that on a trusted network.
|
||||
|
||||
---
|
||||
|
||||
## Supported Languages
|
||||
@@ -615,7 +695,8 @@ ComfyUI-Workflow-Studio/
|
||||
│ │ ├── settings_routes.py # Settings API
|
||||
│ │ ├── ollama_routes.py # Ollama proxy API
|
||||
│ │ ├── eagle_routes.py # Eagle integration API
|
||||
│ │ └── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ ├── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ └── tagspace_routes.py # TagSpace tab API (build / space / search / neighbors)
|
||||
│ └── services/
|
||||
│ ├── workflow_service.py # Workflow file operations
|
||||
│ ├── nodes_service.py # Node metadata & node sets
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 942 KiB |
@@ -0,0 +1,32 @@
|
||||
# はじめてのComfyUI Workflow Studio
|
||||
|
||||
ノードが苦手でも大丈夫 ― 画像生成をもっと手軽に、もっと整理して
|
||||
|
||||
- 対象バージョン：Workflow Studio v0.9.9
|
||||
- 著者：ketle-man
|
||||
- © 2026 ketle-man
|
||||
|
||||
このマニュアルは、Kindle版をもとにした **画像なしのテキスト版** です。
|
||||
章ごとのファイルは `chapters/` フォルダ、全章をまとめたファイルは `Workflow_Studio_Manual_v0.9.9.md` です。
|
||||
画面の例は省略しているため、実際の画面はWorkflow Studioのヘルプタブとあわせてご覧ください。
|
||||
|
||||
## 目次
|
||||
|
||||
- [00_はじめに](chapters/00_はじめに.md)
|
||||
- [01_第1章_基礎知識](chapters/01_第1章_基礎知識.md)
|
||||
- [02_第2章_インストールと起動](chapters/02_第2章_インストールと起動.md)
|
||||
- [03_第3章_画面の見方](chapters/03_第3章_画面の見方.md)
|
||||
- [04_第4章_ワークフロータブ](chapters/04_第4章_ワークフロータブ.md)
|
||||
- [05_第5章_生成UI_前半](chapters/05_第5章_生成UI_前半.md)
|
||||
- [05_第5章_生成UI_後半](chapters/05_第5章_生成UI_後半.md)
|
||||
- [06_第6章_モデルタブ](chapters/06_第6章_モデルタブ.md)
|
||||
- [07_第7章_プロンプトタブ](chapters/07_第7章_プロンプトタブ.md)
|
||||
- [08_第8章_ギャラリータブ](chapters/08_第8章_ギャラリータブ.md)
|
||||
- [09_第9章_WS_Library](chapters/09_第9章_WS_Library.md)
|
||||
- [10_第10章_AI_TOOL](chapters/10_第10章_AI_TOOL.md)
|
||||
- [11_第11章_さらに広がる機能](chapters/11_第11章_さらに広がる機能.md)
|
||||
- [12_第12章_困ったときは](chapters/12_第12章_困ったときは.md)
|
||||
- [13_付録](chapters/13_付録.md)
|
||||
- [14_おわりに](chapters/14_おわりに.md)
|
||||
|
||||
全章まとめ：[Workflow_Studio_Manual_v0.9.9.md](Workflow_Studio_Manual_v0.9.9.md)
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -0,0 +1,172 @@
|
||||
# はじめに
|
||||
|
||||
「ComfyUIを入れてみたけれど、ノード画面を見て手が止まってしまった」
|
||||
「ネットで拾ったワークフローが増えすぎて、どれが何だか分からない」
|
||||
「前にうまくいった画像の設定を、もう一度使いたいのに見つからない」
|
||||
|
||||
本書は、そんな方のための **ComfyUI-Workflow-Studio**（以下、Workflow Studio）の入門書です。
|
||||
|
||||
---
|
||||
|
||||
## ComfyUIは、なぜ難しいのか
|
||||
|
||||
ComfyUIは、いま最も自由度の高い画像生成AIソフトの1つです。
|
||||
新しいモデルや技術が発表されると、真っ先に使えるようになるのは、たいていComfyUIです。
|
||||
それなのに、多くの人が最初の一歩でつまずいてしまいます。その理由は、ComfyUIが **普通のアプリケーションとはまったく違う考え方** で作られているからです。
|
||||
|
||||
### 普通のアプリケーションとの違い
|
||||
| | 普通のアプリケーション | ComfyUI |
|
||||
| --- | --- | --- |
|
||||
| 画面 | 開発者が設計した、決まった画面 | ノード（箱）を線でつないで、自分で組み立てる |
|
||||
| 使い方 | ボタンやメニューを押せば動く | どのノードを、どうつなぐかを知らないと動かない |
|
||||
| 機能の追加 | アップデートを待つ | 誰かが作ったカスタムノードを、自分で追加する |
|
||||
| 設定の場所 | 設定画面にまとまっている | ワークフローの中の、あちこちのノードに散らばっている |
|
||||
|
||||
普通のアプリケーションは、言わば「完成品の家電」です。電源を入れれば、決められた使い方ですぐに使えます。
|
||||
一方のComfyUIは、「部品から組み立てる工作キット」です。
|
||||
組み立て方さえ分かれば、どんなものでも作れますが、組み立て方を知らなければ何も作れません。
|
||||
|
||||
### 自由さと、その副作用
|
||||
ComfyUIの最大の魅力は、この **カスタマイズの自由さ** です。
|
||||
世界中の開発者が作ったカスタムノードを組み合わせて、自分だけの処理の流れ（ワークフロー）を作れます。
|
||||
ネットには、上級者が作った高度なワークフローがたくさん公開されていて、ダウンロードすればすぐに試せます。
|
||||
|
||||
しかし、自由さには副作用もあります。
|
||||
|
||||
- **ワークフローが増えすぎる** － 試したいものを次々ダウンロードするうちに、どれが何だか分からなくなる
|
||||
- **どこを触ればいいか分からない** － 他人が作った大きなワークフローは、プロンプトを変えたいだけでも、該当するノードを探すのに苦労する
|
||||
- **素材がバラバラになる** － モデル・LoRA・プロンプト・生成画像が、それぞれ別の場所に散らばり、管理する道具がない
|
||||
- **試行錯誤に手間がかかる** － 条件を少しずつ変えて比べたいだけでも、1回ごとにノードの値を書き換える必要がある
|
||||
|
||||
ComfyUIは「組み立てる」ための道具としては非常に優秀ですが、
|
||||
「整理する」「繰り返し使う」「手軽に試す」ための道具は、ほとんど用意されていません。
|
||||
この空白を埋めるために作られたのが、Workflow Studioです。
|
||||
|
||||
---
|
||||
|
||||
## Workflow Studioの特色
|
||||
|
||||
### 1. なぜ専用画面（SPA）なのか － 「組み立てる」と「使う」を分ける
|
||||
Workflow Studioは、ComfyUIの上部メニューの **W** ボタンから開く、**独立した専用画面**（SPA：シングルページアプリケーション）です。
|
||||
|
||||
ワークフローを組み立てる作業は、ComfyUIのノード画面で試行錯誤するのがいちばんです。
|
||||
Workflow Studioは、そうして作り上げたワークフローを **素早く活用する** ための画面として生まれました。
|
||||
|
||||
そのため、Workflow Studioはワークフローを **入力・処理・出力** の3つに分けて考え、次のように設計されています。
|
||||
|
||||
| 段階 | 設計の考え方 |
|
||||
| --- | --- |
|
||||
| 入力 | プロンプトや画像を入れやすくする |
|
||||
| 処理 | 値の変更は、最小限の操作で済むようにする |
|
||||
| 出力 | 生成されたデータを管理しやすくする |
|
||||
|
||||
設計の根底には、製造現場などで使われる **5S（整理・整頓・清掃・清潔・しつけ）** の考え方があります。
|
||||
必要なものを、必要なときに、すぐ取り出して使える状態を保つことを目指しています。
|
||||
|
||||
そして、ComfyUIのノード画面とWorkflow Studioの2つの画面をつなぐのが、ノード画面の横に開く **WS Library** です。
|
||||
補助機能とあわせて、2つの画面を行き来する作業をスムーズにします。
|
||||
|
||||
### 2. 管理 － 散らばった素材を1か所に
|
||||
| 管理できるもの | できること |
|
||||
| --- | --- |
|
||||
| ワークフロー | サムネイル付きの一覧、バッジ・グループで分類、検索、生成UIで動くかの事前チェック |
|
||||
| モデル・LoRA | 種類ごとの一覧、Civitaiからの情報（トリガーワード・サンプル画像）の自動取得 |
|
||||
| プロンプト | プリセットとして保存・分類、ワイルドカード、スタイル |
|
||||
| 生成画像・音声 | フォルダごとの閲覧、プロンプトや「意味」での検索、埋め込まれた設定の読み取りと再現（音声ファイルにも対応） |
|
||||
|
||||
### 3. 効率化 － ノードを触らずに、すばやく試す
|
||||
- **生成UIタブ** － ワークフローの中から、プロンプト・モデル・サイズなど、よく変える項目だけを自動で取り出し、入力フォームにします。他人が作った大きなワークフローでも、ノードを探さずに使えます
|
||||
- **バッチ生成** － 「手持ちのモデルを全部試す」「プロンプトを順番に試す」「サンプラーを比べる」といった試行錯誤を、まとめて自動で実行します
|
||||
- **Feeder・Lab** － たくさんの画像を順番に処理したり、複数の条件を段階的に切り替えたりする、さらに進んだ自動化もできます
|
||||
|
||||
### 4. 補助機能 － 生成の前後を手助けする
|
||||
- **画像編集（Image Edit）** － レイヤー付きの簡易編集、マスクを使った部分的な描き直し（インペイント）
|
||||
- **ローカルAI連携** － 日本語からのプロンプト翻訳、画像からのプロンプト・タグ作成
|
||||
- **Tagger・Video** － 画像からのタグ自動生成、動画の生成と簡単な編集
|
||||
- **TagSpace** － 手持ちのプロンプト・タグ・生成画像を「意味の近さ」で3Dの空間に並べ、眺めながらタグを選ぶ
|
||||
|
||||
---
|
||||
|
||||
## インストールして損のないカスタムノード
|
||||
|
||||
Workflow Studioは、次の理由から、ComfyUIを使うすべての人に **「とりあえず入れておいて損はない」** カスタムノードだと言えます。
|
||||
|
||||
- **無料で、ComfyUI Managerからすぐに入れられる**
|
||||
- **基本機能に、追加の準備がいらない** － 必要なものはComfyUIに最初から含まれています。AIや画像編集などの追加機能は、使いたくなったときに準備すれば十分です
|
||||
- **今の使い方を変えなくていい** － ワークフローはComfyUIの保存フォルダをそのまま使います。ノード画面での作業も、これまでどおりできます
|
||||
- **使った分だけ便利になる** － ワークフローを一覧で眺めるだけ、生成画像の設定を確かめるだけ、といった使い方から始めて、慣れてきたら生成UIやバッチ生成へと、少しずつ活用の幅を広げられます
|
||||
|
||||
ComfyUIの「自由さ」はそのままに、「分かりやすさ」と「手軽さ」を足してくれる － それがWorkflow Studioです。
|
||||
|
||||
## この本で分かること
|
||||
|
||||
- Workflow Studioのインストールと、基本的な画面の見方
|
||||
- 手持ちのワークフローを、サムネイル付きで整理する方法
|
||||
- ノード画面を触らずに、はじめての1枚を生成する方法
|
||||
- モデル・LoRA・プロンプトを管理して、うまくいった設定を再利用する方法
|
||||
- 生成した画像から、使った設定を読み取って再現する方法
|
||||
- ローカルAIを使った、プロンプトの翻訳や画像の読み取り
|
||||
- よくあるトラブルとその対処法
|
||||
|
||||
## この本で扱わないこと
|
||||
|
||||
- ComfyUI本体のインストール方法
|
||||
（ComfyUIをすでに入れて、起動できる状態から説明を始めます）
|
||||
- ノードを一から組み立てて、ワークフローを自作する方法
|
||||
- 画像生成AIのしくみや、モデルの学習（LoRAの作り方など）の詳しい解説
|
||||
|
||||
---
|
||||
|
||||
## 対象読者
|
||||
|
||||
- ComfyUIを入れたばかりの方、これから本格的に使いたい方
|
||||
- ノード画面が苦手で、もっと手軽に画像を生成したい方
|
||||
- ワークフローやモデル、プロンプトが増えて、整理に困っている方
|
||||
|
||||
パソコンの基本的な操作（ファイルのコピー、フォルダを開く、ブラウザを使う）ができれば、読み進められるように書いています。
|
||||
|
||||
---
|
||||
|
||||
## 本書の対象バージョンと環境
|
||||
|
||||
- **Workflow Studio**：v0.9.9
|
||||
- **ComfyUI**：本書執筆時点の最新版
|
||||
- **OS**：Windows（画面の例はWindowsのものです）
|
||||
|
||||
Workflow Studioは活発に開発が続けられているため、
|
||||
バージョンによっては、画面の見た目やボタンの位置が本書と異なる場合があります。
|
||||
大きく変わった機能は、Workflow Studioの **ヘルプ** タブや、GitHubのリリースノートも確認してください。
|
||||
|
||||
---
|
||||
|
||||
## 本書の表記ルール
|
||||
|
||||
| 表記 | 意味 |
|
||||
| --- | --- |
|
||||
| **太字** | 画面に表示されるボタン名・項目名 |
|
||||
| 「生成UI（GenerateUI）」 | 日本語表示の名前（英語表示の名前） |
|
||||
| `ComfyUI/models/` | フォルダやファイルの場所。環境による読み替えは第2章 2-6 を参照 |
|
||||
| `Ctrl＋Z` | キーボードの操作。「＋」は同時に押すことを表します |
|
||||
|
||||
> **ポイント**
|
||||
> 知っておくと便利な情報です。
|
||||
|
||||
> **注意**
|
||||
> 間違えるとデータが消えたり、うまく動かなかったりする、気をつけてほしい情報です。
|
||||
|
||||
### 画面の言語について
|
||||
Workflow Studioは英語・日本語・中国語に対応しています。本書では、**日本語表示** の画面で説明します。
|
||||
ただし、生成UIなど一部の画面は、日本語表示にしても英語のまま表示されるボタンや項目があります。
|
||||
その場合は、英語の表示名をそのまま使って説明しています。
|
||||
|
||||
---
|
||||
|
||||
## 本書の読み進め方
|
||||
|
||||
はじめての方は、**第1章から第5章まで** を順番に読み、実際に手を動かしてみてください。
|
||||
第5章で「はじめての1枚」が生成できれば、Workflow Studioの基本は身についています。
|
||||
|
||||
第6章以降は、必要になったところから読んでもかまいません。
|
||||
困ったときは、第12章のトラブルシューティングと、付録の用語集を活用してください。
|
||||
|
||||
それでは、Workflow Studioで、もっと手軽に、もっと楽しく画像生成を始めましょう。
|
||||
@@ -0,0 +1,148 @@
|
||||
# 第1章 ComfyUIとWorkflow Studioの基礎知識
|
||||
|
||||
この章では、操作を始める前に知っておきたい考え方と言葉を整理します。
|
||||
パソコンを触りながら読む必要はありません。通勤中やすきま時間に、気軽に読み進めてください。
|
||||
|
||||
---
|
||||
|
||||
## 1-1 ComfyUIってなに？
|
||||
|
||||
ComfyUI（コンフィーユーアイ）は、画像生成AIを自分のパソコンで動かすための無料ソフトです。
|
||||
Stable Diffusion や FLUX など、さまざまな画像生成AIのモデルに対応しています。
|
||||
|
||||
ComfyUIのいちばんの特徴は、**処理の流れを「箱」と「線」でつないで組み立てる**ことです。
|
||||
|
||||
「モデルを読み込む箱」→「プロンプトを入れる箱」→「画像を作る箱」→「保存する箱」……
|
||||
というように、箱どうしを線でつなぐと、その順番で処理が行われます。
|
||||
|
||||
この方式には、大きなメリットがあります。
|
||||
|
||||
- 処理の中身がすべて見えるので、細かく調整できる
|
||||
- 新しい技術が出ても、箱を追加するだけで試せる
|
||||
- 組み立てた流れを保存して、何度でも使い回せる
|
||||
|
||||
一方で、はじめて触る人にとっては「どこを触ればいいのか分からない」画面でもあります。
|
||||
この本で紹介する **Workflow Studio** は、まさにその「分かりにくさ」をやわらげるための道具です。
|
||||
|
||||
---
|
||||
|
||||
## 1-2 覚えておきたい5つの言葉
|
||||
|
||||
ComfyUIを使ううえで、最低限知っておきたい言葉は次の5つです。
|
||||
|
||||
### ① ノード（Node）
|
||||
画面上に並ぶ「箱」のことです。
|
||||
1つのノードは1つの仕事を受け持ちます。たとえば「モデルを読み込む」「画像を保存する」などです。
|
||||
ノードには入口（入力）と出口（出力）があり、線でつないでデータを受け渡します。
|
||||
|
||||
### ② ワークフロー（Workflow）
|
||||
ノードを線でつないだ「処理の流れ全体」のことです。
|
||||
料理にたとえると、ノードが「切る」「炒める」などの作業で、ワークフローは「レシピ」にあたります。
|
||||
ワークフローはJSON（ジェイソン）という形式のファイルで保存できます。
|
||||
|
||||
> **ポイント**
|
||||
> ComfyUIで作った画像には、多くの場合ワークフローの情報が埋め込まれています。
|
||||
> その画像をComfyUIの画面にドラッグすると、同じワークフローを呼び出せます。
|
||||
|
||||
### ③ モデル（Model）
|
||||
画像を作る「AIの頭脳」にあたるファイルです。数GB〜数十GBと大きいのが一般的です。
|
||||
代表的なものに次の種類があります（詳しくは第6章で説明します）。
|
||||
|
||||
| 種類 | ひとことで言うと |
|
||||
| --- | --- |
|
||||
| Checkpoint（チェックポイント） | 画像生成の本体となるモデル |
|
||||
| LoRA（ローラ） | 絵柄やキャラクターを追加で覚えさせる小さな追加モデル |
|
||||
| VAE | 画像の色合いや細部の仕上げを担当するモデル |
|
||||
|
||||
### ④ プロンプト（Prompt）
|
||||
AIに「どんな画像を作ってほしいか」を伝える文章です。いわゆる「呪文」です。
|
||||
描いてほしいものを書く **ポジティブプロンプト** と、
|
||||
描いてほしくないものを書く **ネガティブプロンプト** の2種類があります。
|
||||
|
||||
### ⑤ キュー（Queue）
|
||||
「生成の予約リスト」のことです。
|
||||
ComfyUIでは、生成ボタンを押すとまずキューに予約が入り、順番に処理されます。
|
||||
何枚分も予約しておけば、席を外している間にまとめて生成してくれます。
|
||||
|
||||
---
|
||||
|
||||
## 1-3 ComfyUIで初心者がつまずくところ
|
||||
|
||||
ComfyUIを入れてみたものの、こんなところで手が止まっていませんか。
|
||||
|
||||
1. **ワークフローが増えて、どれが何だか分からない**
|
||||
ネットで拾ったワークフローを保存していくうちに、ファイル名だけでは見分けがつかなくなります。
|
||||
|
||||
2. **毎回ノード画面で値を探すのが大変**
|
||||
プロンプトやシード値を変えたいだけなのに、大きなワークフローの中から該当するノードを探す必要があります。
|
||||
|
||||
3. **モデルやプロンプトの管理がバラバラ**
|
||||
どのモデルをどこに置いたか、前にうまくいったプロンプトは何だったか……が分からなくなりがちです。
|
||||
|
||||
4. **前に作った画像の設定を再現できない**
|
||||
「あの画像、どの設定で作ったっけ？」となっても、調べ方が分からないことがあります。
|
||||
|
||||
どれも、ComfyUIの機能が足りないというより、**「整理」と「入口」が足りない**ことが原因です。
|
||||
|
||||
---
|
||||
|
||||
## 1-4 Workflow Studioでできること
|
||||
|
||||
Workflow Studioは、ComfyUIに追加して使う **拡張機能（カスタムノード）** です。
|
||||
ComfyUIの中に「管理と生成のための専用画面」を追加してくれます。
|
||||
|
||||
先ほどの4つのつまずきに対して、Workflow Studioは次のように応えます。
|
||||
|
||||
| つまずき | Workflow Studioの解決策 | 本書の章 |
|
||||
| --- | --- | --- |
|
||||
| ワークフローが見分けられない | サムネイル付きの一覧、バッジ・グループ分け、検索 | 第4章 |
|
||||
| ノード画面で値を探すのが大変 | 「生成UI」タブで、必要な項目だけをフォームとして表示 | 第5章 |
|
||||
| モデル・プロンプトがバラバラ | モデルタブ・プロンプトタブでまとめて管理 | 第6・7章 |
|
||||
| 設定を再現できない | ギャラリーで画像のメタデータを確認し、キャンバスへ送る | 第8章 |
|
||||
|
||||
さらに、ComfyUIのノード画面の横に **WS Library** というサイドパネルが追加され、
|
||||
モデルやプロンプトをドラッグ＆ドロップでノード画面に置けるようになります（第9章）。
|
||||
|
||||
ローカルで動くAI（OllamaやLM Studioなど）を用意すれば、
|
||||
プロンプトの翻訳や、画像からのタグ生成といったAI補助も使えます（第10章）。
|
||||
|
||||
> **ポイント**
|
||||
> Workflow Studioは、ComfyUIの代わりになるソフトではありません。
|
||||
> 画像を作るのはあくまでComfyUIで、Workflow Studioはその「操作パネル」と「整理棚」の役割です。
|
||||
> そのため、ComfyUIが起動していないと使えません。
|
||||
|
||||
### 画面は日本語で使えます
|
||||
Workflow Studioは英語・日本語・中国語に対応しています。
|
||||
本書では日本語表示の画面で説明し、必要に応じて英語表示の名前を（ ）で添えます。
|
||||
|
||||
例：「生成UI（GenerateUI）」タブ、「ワークフロー（Workflow）」タブ
|
||||
|
||||
---
|
||||
|
||||
## 1-5 本書の進め方
|
||||
|
||||
本書は、次の順番で読み進めることを想定しています。
|
||||
|
||||
1. **準備する**（第2〜3章）
|
||||
インストールして、画面の見方を覚えます。
|
||||
|
||||
2. **整理する**（第4章）
|
||||
手持ちのワークフローを一覧で見られるようにします。
|
||||
|
||||
3. **作る**（第5章）
|
||||
ノード画面を触らずに、はじめての1枚を生成します。
|
||||
ここまで読めば、Workflow Studioの基本は使えるようになります。
|
||||
|
||||
4. **貯める・見返す**（第6〜8章）
|
||||
モデル・プロンプト・生成画像を管理し、うまくいった設定を再利用します。
|
||||
|
||||
5. **広げる**（第9〜11章）
|
||||
サイドパネルやAI補助など、便利な機能を必要に応じて取り入れます。
|
||||
|
||||
困ったときは、第12章のトラブルシューティングと付録の用語集を見てください。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ComfyUIは、ノード（箱）をつないだワークフローで画像を作るソフト
|
||||
> - 覚える言葉は「ノード」「ワークフロー」「モデル」「プロンプト」「キュー」の5つ
|
||||
> - Workflow Studioは、ComfyUIに「整理棚」と「かんたん操作パネル」を追加する拡張機能
|
||||
> - まずは第5章の「はじめての1枚」を目標にしよう
|
||||
@@ -0,0 +1,254 @@
|
||||
# 第2章 インストールと起動
|
||||
|
||||
この章では、Workflow StudioをComfyUIに追加して、はじめて画面を開くところまで進めます。
|
||||
ComfyUI本体はすでにインストール済みで、起動できる状態であることを前提にしています。
|
||||
|
||||
---
|
||||
|
||||
## 2-1 動作に必要なもの
|
||||
|
||||
### 必須
|
||||
| 必要なもの | 説明 |
|
||||
| --- | --- |
|
||||
| ComfyUI | 最近のバージョンであればOK。上部メニューにボタンを並べる機能を使うため、新しめのバージョン（フロントエンド v1.33.9 以降）がおすすめです |
|
||||
| Python 3.10 以上 | ComfyUIに同梱されているものでかまいません |
|
||||
| Webブラウザ | Chrome や Edge など、ComfyUIの画面を開いているブラウザ |
|
||||
|
||||
Workflow Studioの基本機能は、ComfyUIが動いていれば追加の準備なしで使えます。
|
||||
|
||||
### あとから必要に応じて
|
||||
次のものは、特定の機能を使うときだけ必要です。最初は気にしなくてかまいません。
|
||||
|
||||
- **Ollama / LM Studio など** － プロンプトの翻訳やAIチャットなどのAI補助機能（第10章）。
|
||||
ギャラリーのセマンティック検索をローカルで行うとき（第8章）と、TagSpaceタブ（第11章）には **Ollama** が必要です
|
||||
- **Eagle** － 生成画像を画像管理ソフトEagleに自動保存する機能（第8章）
|
||||
- **追加のPythonパッケージ** － Tagger（画像からのタグ生成）やPSD書き出しなど（第11章）
|
||||
|
||||
> **ポイント**
|
||||
> 「全部そろえないと使えない」わけではありません。
|
||||
> まずは本体だけ入れて、使いたい機能が出てきたときに追加しましょう。
|
||||
|
||||
---
|
||||
|
||||
## 2-2 ComfyUI Managerでインストールする（おすすめ）
|
||||
|
||||
**ComfyUI Manager** は、カスタムノード（拡張機能）を画面上から検索・インストールできる管理ツールです。
|
||||
最近のComfyUIには最初から入っていることが多く、いちばん手軽な方法です。
|
||||
|
||||
1. ComfyUIを起動し、ブラウザで画面を開きます。
|
||||
2. 上部メニューの **「Manager」** ボタンをクリックします。
|
||||
|
||||
3. **「Custom Nodes Manager」** をクリックします。
|
||||
4. 検索欄に **`Workflow Studio`** と入力します。
|
||||
5. 一覧に表示された **ComfyUI-Workflow-Studio** の **「Install」** をクリックします。
|
||||
|
||||
6. インストールが終わったら、画面の案内に従って **ComfyUIを再起動** します。
|
||||
|
||||
> **注意**
|
||||
> ComfyUIは、起動したときに一度だけ拡張機能を読み込みます。
|
||||
> インストール後は必ずComfyUIを再起動してください。ブラウザの再読み込みだけでは反映されません。
|
||||
|
||||
---
|
||||
|
||||
## 2-3 手動でインストールする（git clone）
|
||||
|
||||
ComfyUI Managerが使えない場合や、Gitに慣れている方は、手動でもインストールできます。
|
||||
コマンドを入力する画面（Windowsなら「コマンドプロンプト」や「ターミナル」）を使います。
|
||||
|
||||
1. ComfyUIのフォルダの中にある **`custom_nodes`** フォルダに移動します。
|
||||
|
||||
```
|
||||
cd ComfyUI/custom_nodes
|
||||
```
|
||||
|
||||
2. 次のコマンドを入力して、Workflow Studioをダウンロードします。
|
||||
|
||||
```
|
||||
git clone https://github.com/ketle-man/ComfyUI-Workflow-Studio.git
|
||||
```
|
||||
|
||||
3. `custom_nodes` の中に **`ComfyUI-Workflow-Studio`** フォルダができていればOKです。
|
||||
4. ComfyUIを再起動します。
|
||||
|
||||
> **ポイント**
|
||||
> `git` コマンドが使えない場合は、Git（Git for Windowsなど）を先にインストールする必要があります。
|
||||
> 不安な方は、2-2のComfyUI Managerを使う方法がおすすめです。
|
||||
|
||||
---
|
||||
|
||||
## 2-4 起動する
|
||||
|
||||
### Wボタンから開く
|
||||
ComfyUIを再起動すると、画面上部のメニューに **「W」** のボタンが追加されています。
|
||||
これがWorkflow Studioを開くボタンです。
|
||||
|
||||
- **クリック** － 新しいタブでWorkflow Studioが開きます。
|
||||
- **Shiftキーを押しながらクリック** － 新しいウィンドウで開きます。ComfyUIの画面と並べて使いたいときに便利です。
|
||||
|
||||
Wボタンの近くには、**カメラのボタン** も追加されています。
|
||||
これは、ワークフローの画面をサムネイルとして保存するためのボタンです（第4章で使います）。
|
||||
|
||||
### アドレスとポート番号のしくみ
|
||||
Wボタンを押すと、ブラウザのアドレス欄にWorkflow Studioのアドレスが表示されます。
|
||||
このアドレスの見方を知っておくと、ブックマークから直接開いたり、トラブルのときに原因を探したりできます。
|
||||
|
||||
アドレスは、次の3つの部分でできています。
|
||||
|
||||
| 部分 | 例 | 意味 |
|
||||
| --- | --- | --- |
|
||||
| ホスト | `127.0.0.1` | どのパソコンにつなぐか。`127.0.0.1`（または `localhost`）は「自分のパソコン」を表す特別な番号です |
|
||||
| ポート番号 | `8188` | そのパソコンの中の、どのソフトにつなぐか |
|
||||
| パス | `/wfm` | そのソフトの中の、どのページを開くか |
|
||||
|
||||
この3つを「`ホスト:ポート番号/パス`」の形でつなげたものが、Workflow Studioのアドレスになります。
|
||||
|
||||
#### ポート番号とは
|
||||
1台のパソコンの中では、たくさんのソフトが同時に通信しています。
|
||||
そこで、どのソフトあての通信なのかを区別するために使われるのが **ポート番号** です。
|
||||
|
||||
マンションにたとえると、ホスト（`127.0.0.1`）が「建物の住所」、ポート番号が「部屋番号」にあたります。
|
||||
同じ建物でも、部屋番号を間違えると目的の相手には届きません。
|
||||
|
||||
ComfyUIは、起動したときに決められたポート番号で待ち受けています。
|
||||
よく使われる番号は次のとおりです。
|
||||
|
||||
| ComfyUIの種類・起動方法 | よく使われるポート番号 |
|
||||
| --- | --- |
|
||||
| ポータブル版・手動インストール版（標準設定） | 8188 |
|
||||
| ComfyUI Desktop（デスクトップアプリ版） | 8000 |
|
||||
| 起動オプション `--port` を指定した場合 | 指定した番号 |
|
||||
|
||||
Stability Matrixなどのランチャーを使っている場合も、ランチャー側の設定で番号が変わることがあります。
|
||||
|
||||
#### 自分のポート番号を確かめる
|
||||
いちばん確実なのは、**いまComfyUIを開いているブラウザのアドレス欄を見る** ことです。
|
||||
`127.0.0.1:` や `localhost:` のあとに続く数字が、ポート番号です。
|
||||
|
||||
ComfyUIを起動したときの黒い画面（コンソール）にも、
|
||||
`To see the GUI go to: http://127.0.0.1:〇〇〇〇` のように表示されます。
|
||||
|
||||
#### アドレスを直接入力して開く
|
||||
ポート番号が分かれば、Wボタンを使わずにWorkflow Studioを開けます。
|
||||
ComfyUIのアドレスの末尾に **`/wfm`** を付けるだけです。
|
||||
|
||||
- ComfyUIが `127.0.0.1:8188` で開いている → Workflow Studioは `127.0.0.1:8188/wfm`
|
||||
- ComfyUIが `127.0.0.1:8000` で開いている → Workflow Studioは `127.0.0.1:8000/wfm`
|
||||
|
||||
よく使う方は、このアドレスをブックマークしておくと便利です。
|
||||
|
||||
> **注意**
|
||||
> ブックマークから開いた場合、ワークフローをComfyUIのキャンバスへ直接送る「Send to Canvas」が使えません。
|
||||
> 代わりに、ComfyUI側のサイドパネル（WS Library）にそのワークフローが表示されるので、自分でキャンバスへドラッグします（第4章・第9章）。
|
||||
> ComfyUIの画面と行き来しながら使うときは、Wボタンから開くのがおすすめです。
|
||||
|
||||
> **ポイント**
|
||||
> Workflow StudioはComfyUIの一部として動いているため、ComfyUIと同じポート番号を使います。
|
||||
> Workflow Studio専用のポート番号はありません。
|
||||
|
||||
---
|
||||
|
||||
## 2-5 表示言語を日本語にする
|
||||
|
||||
Workflow Studioは、はじめは **英語** で表示されます。次の手順で日本語に切り替えましょう。
|
||||
|
||||
1. 上部のタブから **「Settings」** を開きます。
|
||||
2. **「UI Language」** の欄で **「日本語」** を選びます。
|
||||
|
||||
3. 「言語を変更しました。再読み込みします...」という表示のあと、ページが自動で再読み込みされ、日本語表示に切り替わります。
|
||||
|
||||
日本語に切り替えると、タブの名前は次のように変わります（一部は英語のままです）。
|
||||
|
||||
| 英語表示 | 日本語表示 |
|
||||
| --- | --- |
|
||||
| Workflow | ワークフロー |
|
||||
| GenerateUI | 生成UI |
|
||||
| Gallery | ギャラリー |
|
||||
| Prompt | プロンプト |
|
||||
| Models | モデル |
|
||||
| Nodes | ノード |
|
||||
| Settings | 設定 |
|
||||
| Help | ヘルプ |
|
||||
| AI TOOL / Tagger / Image Edit / Video | （英語のまま） |
|
||||
|
||||
> **ポイント**
|
||||
> 言語の設定は、使っている **ブラウザごと** に保存されます。
|
||||
> 別のブラウザや別のパソコンから開いた場合は、もう一度設定してください。
|
||||
|
||||
同じ場所にある **「要約言語」** は、AIでワークフローの説明文を作るときの言語です（第10章）。今は変更しなくてかまいません。
|
||||
|
||||
---
|
||||
|
||||
## 2-6 環境ごとのフォルダの場所
|
||||
|
||||
ComfyUIには、いくつかの入れ方があります。入れ方によって、フォルダの場所が少しずつ違います。
|
||||
本書では、フォルダの場所を **`ComfyUI/` から始まる形** で書いています。
|
||||
お使いの環境に合わせて、下の表の「`ComfyUI/` にあたる場所」に読み替えてください。
|
||||
|
||||
### 本書の書き方と、各環境での場所
|
||||
| 本書の表記 | 中身 | ポータブル版 | ComfyUI Desktop | Stability Matrix |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `ComfyUI/` | ComfyUIの本体フォルダ | `ComfyUI_windows_portable/ComfyUI/` | インストール時に選んだフォルダ（標準では「ドキュメント」内の `ComfyUI`） | `（データフォルダ）/Packages/ComfyUI/` |
|
||||
| `ComfyUI/custom_nodes/` | カスタムノード（Workflow Studioもここ） | `ComfyUI/custom_nodes/` | `ComfyUI/custom_nodes/` | `ComfyUI/custom_nodes/` |
|
||||
| `ComfyUI/models/` | モデル | `ComfyUI/models/` | `ComfyUI/models/` | `（データフォルダ）/Models/`（全パッケージで共有） |
|
||||
| `ComfyUI/output/` | 生成した画像 | `ComfyUI/output/` | `ComfyUI/output/` | `（データフォルダ）/Images/` |
|
||||
| `ComfyUI/user/default/workflows/` | 保存したワークフロー | `ComfyUI/user/default/workflows/` | `ComfyUI/user/default/workflows/` | `ComfyUI/user/default/workflows/` |
|
||||
| ComfyUIのPython | 追加パッケージのインストール先 | `ComfyUI_windows_portable/python_embeded/python.exe` | `ComfyUI/.venv/Scripts/python.exe` | `ComfyUI/venv/Scripts/python.exe` |
|
||||
| 標準のポート番号 | 2-4参照 | 8188 | 8000 | 8188 |
|
||||
|
||||
> **ポイント**
|
||||
> 表の中の `ComfyUI/` は、それぞれの環境の「`ComfyUI/` にあたる場所」を指します。
|
||||
> パスの区切りは、Windowsでは `/` の代わりに `\` で表示されます。
|
||||
|
||||
### ポータブル版
|
||||
ダウンロードしたZIPファイルを展開してできた `ComfyUI_windows_portable` フォルダの中に、すべてが入っています。
|
||||
起動は、フォルダ内の `run_nvidia_gpu.bat` などで行います。
|
||||
**Python** は、ComfyUI専用の `python_embeded` フォルダの中のものを使います（第11章 11-2）。
|
||||
|
||||
### ComfyUI Desktop
|
||||
インストーラーで入れるアプリ版です。最初の起動時に、データを置くフォルダを選びます。
|
||||
フォルダの場所を忘れたときは、アプリのメニューからフォルダを開く項目（「Open Folder」など）を使うと確認できます。
|
||||
標準のポート番号が **8000** なので、アドレスを直接入力するときは注意してください（2-4）。
|
||||
|
||||
### Stability Matrix
|
||||
複数の画像生成ソフトをまとめて管理できるランチャーです。
|
||||
最初に決めた **データフォルダ**（例：`StabilityMatrix/Data/`）の中に、次のように分かれて入っています。
|
||||
|
||||
- **ComfyUI本体** － `Data/Packages/ComfyUI/`
|
||||
- **モデル** － `Data/Models/` の中に、種類ごとのフォルダ（Checkpointは `StableDiffusion`、LoRAは `Lora` など）。ほかのソフトと共有されます
|
||||
- **生成した画像** － `Data/Images/` の中
|
||||
|
||||
> **注意**
|
||||
> Stability Matrixでは、モデルや画像のフォルダが **ComfyUIの本体フォルダの外** にあります。
|
||||
> 本書で「`ComfyUI/models/checkpoints` に置く」と書いてある場合は、Stability Matrixの共有モデルフォルダ（`Data/Models/StableDiffusion` など）に置いてください。
|
||||
> ギャラリーに画像が出てこないときは、設定タブの「ギャラリー Outputフォルダ」が `Data/Images/` の中を指しているか確認しましょう。
|
||||
|
||||
> **ポイント**
|
||||
> フォルダの名前や構成は、各ソフトのバージョンアップで変わることがあります。
|
||||
> 見つからないときは、各ソフトの公式の説明も確認してください。
|
||||
|
||||
---
|
||||
|
||||
## 2-7 うまく起動しないときは
|
||||
|
||||
### Wボタンが表示されない
|
||||
- **ComfyUIを再起動しましたか？** ブラウザの再読み込みだけでは反映されません。
|
||||
- **インストール先は正しいですか？** `ComfyUI/custom_nodes/ComfyUI-Workflow-Studio` にフォルダがあるか確認してください。
|
||||
フォルダの中にさらに同じ名前のフォルダがある（二重になっている）と読み込まれません。
|
||||
- **ComfyUIのバージョンが古くありませんか？** ComfyUI Managerなどから本体を更新してみてください。
|
||||
|
||||
### ページが開かない・「接続できません」と表示される
|
||||
- **ComfyUIは起動していますか？** Workflow StudioはComfyUIの中で動くため、ComfyUIを終了すると開けません。
|
||||
- **ポート番号は合っていますか？** 2-4の方法で、ComfyUIのポート番号を確認してください。
|
||||
たとえばComfyUI Desktopを使っているのに `8188` を入力していると、つながりません。
|
||||
|
||||
### 起動時のコンソールにエラーが出る
|
||||
ComfyUIを起動したときの黒い画面に、`ComfyUI-Workflow-Studio` に関するエラーが表示されていないか確認します。
|
||||
エラーの内容をメモしておくと、第12章のトラブルシューティングや、GitHubで質問するときに役立ちます。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - インストールはComfyUI Managerで「Workflow Studio」を検索するのがかんたん
|
||||
> - インストール後は必ずComfyUIを再起動する
|
||||
> - 上部メニューの「W」ボタンで開く（Shift＋クリックで別ウィンドウ）
|
||||
> - アドレスは「ホスト:ポート番号/wfm」。ポート番号はComfyUIを開いているアドレス欄で確認できる
|
||||
> - 設定タブの「UI Language」で日本語に切り替える
|
||||
> - フォルダの場所は環境（ポータブル版・Desktop・Stability Matrix）によって違う。本書の `ComfyUI/` は読み替える
|
||||
@@ -0,0 +1,146 @@
|
||||
# 第3章 画面の見方
|
||||
|
||||
Workflow Studioにはたくさんのタブがありますが、最初から全部を覚える必要はありません。
|
||||
この章では、それぞれのタブが「何をする場所か」をざっくりつかみ、
|
||||
ComfyUI側に追加されるボタンとサイドパネルの役割を確認します。
|
||||
|
||||
---
|
||||
|
||||
## 3-1 タブの一覧と役割
|
||||
|
||||
Workflow Studioを開くと、画面の上部にタブが横一列に並んでいます。
|
||||
|
||||
左から順に、次のタブがあります。
|
||||
「よく使う」に★が付いたタブから覚えていけば十分です。
|
||||
|
||||
| タブ（日本語表示） | 英語表示 | 何をする場所か | よく使う | 本書の章 |
|
||||
| --- | --- | --- | :---: | --- |
|
||||
| ワークフロー | Workflow | 保存したワークフローの一覧・整理 | ★ | 第4章 |
|
||||
| ノード | Nodes | ノードの一覧・お気に入り登録 | | 第11章 |
|
||||
| モデル | Models | Checkpoint・LoRAなどモデルの一覧・整理 | ★ | 第6章 |
|
||||
| 生成UI | GenerateUI | ノード画面を触らずに画像を生成 | ★ | 第5章 |
|
||||
| プロンプト | Prompt | プロンプトの保存・分類・AI補助 | ★ | 第7章 |
|
||||
| ギャラリー | Gallery | 生成した画像の閲覧・設定の確認 | ★ | 第8章 |
|
||||
| Image Edit | Image Edit | 画像の簡易編集・マスク作成 | | 第11章 |
|
||||
| Video | Video | 動画生成・動画編集 | | 第11章 |
|
||||
| Tagger | Tagger | 画像からタグ（プロンプト）を自動生成 | | 第11章 |
|
||||
| TagSpace | TagSpace | タグ・プロンプト・生成画像を意味の近さで3D表示し、タグを選ぶ | | 第11章 |
|
||||
| 設定 | Settings | 言語・テーマ・フォルダなどの設定 | | 本章・第11章 |
|
||||
| ヘルプ | Help | 機能一覧・ショートカット・トラブル対処 | | 本章 |
|
||||
| AI TOOL | AI TOOL | ローカルAIによる翻訳・チャット・画像解析・文字起こし（OCR） | | 第10章 |
|
||||
|
||||
### タブは「整理する」「作る」「補助する」の3グループで考える
|
||||
タブが多くて迷ったときは、次の3つのグループに分けて考えると分かりやすくなります。
|
||||
|
||||
- **整理する** － ワークフロー／ノード／モデル／プロンプト／ギャラリー
|
||||
手持ちの素材を一覧にして、探しやすくするタブです。
|
||||
- **作る** － 生成UI／Image Edit／Video
|
||||
実際に画像や動画を作ったり、手を加えたりするタブです。
|
||||
- **補助する** － AI TOOL／Tagger／TagSpace／設定／ヘルプ
|
||||
他のタブでの作業を助けたり、全体の設定をしたりするタブです。
|
||||
|
||||
> **ポイント**
|
||||
> 各タブは互いにつながっています。
|
||||
> たとえば、ワークフロータブで選んだワークフローを生成UIに読み込んだり、
|
||||
> プロンプトタブで保存したプロンプトを生成UIで呼び出したりできます。
|
||||
|
||||
### ヘルプタブ
|
||||
困ったときに最初に開きたいのが **ヘルプ** タブです。
|
||||
左側に話題の一覧が並び、クリックすると右側に説明が表示されます。
|
||||
|
||||
- 機能一覧（タブごとの機能の概要）
|
||||
- キーボードショートカット
|
||||
- トラブルシューティング
|
||||
- サポート（GitHubなどへのリンク）
|
||||
|
||||
本書を読みながら「このボタンは何だっけ？」と思ったときにも役立ちます。
|
||||
|
||||
---
|
||||
|
||||
## 3-2 ComfyUI側に追加されるボタン
|
||||
|
||||
Workflow Studioをインストールすると、ComfyUIの画面上部のメニューに **3つのボタン** が追加されます。
|
||||
|
||||
| ボタン | 名前 | 役割 |
|
||||
| --- | --- | --- |
|
||||
| ① W | Workflow Studioを開く | Workflow Studioを新しいタブで開きます。Shift＋クリックで新しいウィンドウ（第2章） |
|
||||
| ② カメラ | キャンバススナップショット | いまのノード画面を画像として保存し、ワークフローのサムネイルにします（第4章） |
|
||||
| ③ 四角が並んだアイコン | WS Library | ノード画面の横にサイドパネルを開閉します（3-3） |
|
||||
|
||||
ボタンにマウスを重ねると、英語で説明が表示されます。
|
||||
|
||||
> **ポイント**
|
||||
> Workflow Studio本体（タブが並ぶ画面）と、ComfyUIのノード画面は **別のブラウザタブ** で開きます。
|
||||
> 「整理や生成はWorkflow Studio」「細かい組み立てはComfyUIのノード画面」と使い分け、
|
||||
> 2つの画面を行き来しながら使うのが基本のスタイルです。
|
||||
|
||||
---
|
||||
|
||||
## 3-3 サイドパネル「WS Library」
|
||||
|
||||
③のボタンを押すと、ComfyUIのノード画面の横に **WS Library（Workflow Studio Library）** というパネルが開きます。
|
||||
|
||||
WS Libraryは、**Workflow Studioで整理した素材を、ノード画面ですぐ使うための棚** です。
|
||||
パネルの上部には、1文字のタブが並んでいます（マウスを重ねると正式な名前が表示されます）。
|
||||
|
||||
| タブ | 名前 | できること |
|
||||
| --- | --- | --- |
|
||||
| W | Workflows | ワークフローをドラッグしてキャンバスに読み込む |
|
||||
| N | Nodes | お気に入りのノードやノードセットをキャンバスに置く |
|
||||
| P | Prompts | 保存したプロンプトをキャンバスに置く・コピーする |
|
||||
| M | Models | モデルをドラッグして、読み込み用のノードごとキャンバスに置く |
|
||||
| I | Information | 画像やワークフローのファイルを落として、中身（使ったモデルやプロンプト）を調べる |
|
||||
| A | AI TOOL | 翻訳やチャットなどのAI機能を、ノード画面のまま使う |
|
||||
|
||||
たとえば M タブからモデルをキャンバスへドラッグすると、そのモデルを読み込むノードが自動で配置されます。
|
||||
ノードを一覧から探して置いて、ファイル名を選んで……という手間が省けます。
|
||||
|
||||
詳しい使い方は第9章で説明します。今は「ノード画面の横に、素材の棚を出せる」とだけ覚えておけば大丈夫です。
|
||||
|
||||
---
|
||||
|
||||
## 3-4 テーマと見た目の設定
|
||||
|
||||
Workflow Studioの見た目は、**設定** タブで好みに合わせて変えられます。
|
||||
長時間使う道具なので、目が疲れにくい配色や読みやすい文字サイズにしておくのがおすすめです。
|
||||
|
||||
### テーマ
|
||||
**「テーマ」** の欄から、画面全体の配色を選べます。標準は暗い配色の「Deep Ocean Dark」です。
|
||||
|
||||
| テーマの例 | 雰囲気 |
|
||||
| --- | --- |
|
||||
| Deep Ocean Dark（デフォルト） | 暗めの青。画像が見やすい |
|
||||
| Light Minimalist（クリーン） | 白基調で明るくシンプル |
|
||||
| Pastel Dream（パステル） | やわらかいパステル調 |
|
||||
| Corporate Trust（企業向け） | 落ち着いた白と青 |
|
||||
|
||||
このほか、ポップ調・サイバーパンク・レトロなど、全部で13種類のテーマがあります。
|
||||
|
||||
> **ポイント**
|
||||
> 生成した画像の色を確認するときは、暗めのテーマのほうが色を見分けやすくなります。
|
||||
> 迷ったら、まずはデフォルトのまま使ってみましょう。
|
||||
|
||||
### カスタマイズ
|
||||
**「カスタマイズ」** を開くと、選んだテーマをさらに細かく調整できます。
|
||||
|
||||
- **カラー調整** － 背景・テキスト・アクセントなどの色を個別に変更
|
||||
- **背景パターン** － ストライプやドットなどの模様を背景に追加
|
||||
- **フォント** － 文字の書体を変更（Zen丸ゴシック、BIZ UDPゴシックなど日本語フォントも選べます）
|
||||
|
||||
調整が終わったら **「適用して保存」** を押します。
|
||||
元に戻したいときは **「カスタマイズをリセット」** を押します。
|
||||
|
||||
### 文字サイズ
|
||||
**「文字サイズ」** の欄で、画面全体の文字の大きさを変えられます。
|
||||
文字が小さくて読みにくいと感じたら、ここで大きくしましょう。
|
||||
|
||||
### WS Libraryの見た目
|
||||
WS Library（3-3）の配色は、パネル内の **歯車ボタン**から別に設定できます。
|
||||
ComfyUIの画面の色に合わせて、背景や文字の色を調整できます。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - タブは「整理する」「作る」「補助する」の3グループで考えると分かりやすい
|
||||
> - まずはワークフロー・モデル・生成UI・プロンプト・ギャラリーの5つを覚えれば十分
|
||||
> - ComfyUIの上部には「W」「カメラ」「WS Library」の3つのボタンが追加される
|
||||
> - WS Libraryは、ノード画面の横に出せる素材の棚
|
||||
> - 見た目は設定タブのテーマ・カスタマイズ・文字サイズで調整できる
|
||||
@@ -0,0 +1,305 @@
|
||||
# 第4章 ワークフロータブ － 作ったワークフローを整理する
|
||||
|
||||
ワークフロータブは、Workflow Studioを開いたときに最初に表示される、いわば「玄関」です。
|
||||
この章では、手持ちのワークフローを一覧で見られるようにし、
|
||||
あとから探しやすいように整理する方法を学びます。
|
||||
|
||||
---
|
||||
|
||||
## 4-1 ワークフローが自動で一覧表示されるしくみ
|
||||
|
||||
ワークフロータブを開くと、すでにいくつかのワークフローが並んでいるかもしれません。
|
||||
これは、Workflow StudioがComfyUIのワークフロー保存フォルダを自動で読み込んでいるためです。
|
||||
|
||||
ComfyUIのノード画面で「保存（Save）」したワークフローは、通常、次のフォルダに保存されます。
|
||||
|
||||
```
|
||||
ComfyUI/user/default/workflows/
|
||||
```
|
||||
|
||||
Workflow Studioは、このフォルダの中のワークフローを一覧にして表示します。
|
||||
つまり、**ComfyUIで保存したワークフローは、何もしなくてもワークフロータブに並ぶ** というわけです。
|
||||
|
||||
> **ポイント**
|
||||
> ComfyUIで新しく保存したワークフローが一覧に出てこないときは、
|
||||
> ツールバーの **「更新」** ボタンを押してください。最新の状態に読み込み直します。
|
||||
|
||||
> **ポイント**
|
||||
> 読み込むフォルダは、設定タブの **「ワークフローデータフォルダ」** で変更できます。
|
||||
> 特に理由がなければ、そのままで問題ありません。
|
||||
|
||||
---
|
||||
|
||||
## 4-2 画面の見方と表示の切り替え
|
||||
|
||||
ワークフロータブは、大きく3つの部分に分かれています。
|
||||
|
||||
| 番号 | 場所 | 役割 |
|
||||
| --- | --- | --- |
|
||||
| ① | ツールバー（上部） | 検索・絞り込み・並べ替え・各種ボタン |
|
||||
| ② | 一覧（中央） | ワークフローがカードまたは表で並ぶ |
|
||||
| ③ | サイドパネル（右側） | 選んだワークフローのサムネイルや中身を表示 |
|
||||
|
||||
### サムネイル表示とテーブル表示
|
||||
ツールバーの切り替えボタンで、一覧の表示方法を変えられます。
|
||||
|
||||
- **サムネイル** － 画像付きのカードで並べます。見た目で探したいときに便利です。
|
||||
- **テーブル** － 表形式で並べます。たくさんのワークフローを名前で探したいときに便利です。
|
||||
|
||||
### クリックとダブルクリック
|
||||
一覧のワークフローは、クリックの仕方で動きが変わります。
|
||||
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| クリック | 右側のサイドパネルに内容を表示（ワークフローを「選択」した状態になる） |
|
||||
| ダブルクリック | 詳細画面を開く（バッジ・タグ・メモの編集など。4-4で説明） |
|
||||
|
||||
### サイドパネル
|
||||
ワークフローをクリックすると、右側のサイドパネルに次の3つのタブが表示されます。
|
||||
|
||||
- **サムネイル** － ワークフローの画像を大きく表示
|
||||
- **JSON** － ワークフローの中身（JSONデータ）を色分けして表示。「コピー」ボタンで丸ごとコピーできます
|
||||
- **グループ** － ワークフローをグループに分類（4-4で説明）
|
||||
|
||||
ワークフロー名の下には、**ワークフローの形式**（UI形式／API形式など）と、
|
||||
生成UIで使えるかどうかの **チェック結果** が表示されます（4-7で説明）。
|
||||
|
||||
---
|
||||
|
||||
## 4-3 カメラボタンでサムネイルを撮る
|
||||
|
||||
ComfyUIで保存しただけのワークフローは、サムネイル画像がなく、一覧では見分けにくいことがあります。
|
||||
そこで使うのが、ComfyUIの上部メニューに追加された **カメラボタン**（キャンバススナップショット）です。
|
||||
|
||||
### サムネイルの撮り方
|
||||
1. ComfyUIのノード画面で、サムネイルにしたいワークフローを開きます。
|
||||
2. ワークフロー全体が見やすくなるように、画面の位置や拡大率を調整します。
|
||||
3. 上部メニューの **カメラボタン** をクリックします。
|
||||
|
||||
4. 保存する名前を確認する画面が表示されます。必要に応じて名前を変えて保存します。
|
||||
（最初は、いま開いているワークフローの名前が入っています）
|
||||
5. 「Workflow saved」と表示されれば完了です。
|
||||
|
||||
Workflow Studioのワークフロータブを開く（または「更新」を押す）と、
|
||||
撮影した画像がサムネイルとして付いたワークフローが一覧に表示されます。
|
||||
|
||||
> **ポイント**
|
||||
> カメラボタンで保存した画像（PNG）には、ワークフローの情報も埋め込まれています。
|
||||
> この画像をComfyUIのノード画面にドラッグ＆ドロップすると、そのワークフローを読み込めます。
|
||||
> 画像ファイル1つで「見た目」と「中身」の両方を持ち運べるので便利です。
|
||||
|
||||
### あとからサムネイルを変える
|
||||
すでにあるワークフローのサムネイルを差し替えたいときは、
|
||||
ワークフローをダブルクリックして詳細画面を開き、**「サムネイル変更」** を押して画像を選びます。
|
||||
|
||||
---
|
||||
|
||||
## 4-4 お気に入り・バッジ・グループで分類する
|
||||
|
||||
ワークフローが増えてきたら、分類しておくと目的のものをすぐに見つけられます。
|
||||
Workflow Studioには、分類のしかたが3種類あります。
|
||||
|
||||
| 方法 | どんなときに使う？ | 例 |
|
||||
| --- | --- | --- |
|
||||
| お気に入り（★） | よく使うものに印を付けたい | 毎日使う基本のワークフロー |
|
||||
| バッジ | 色付きのラベルで、ひと目で種類を分けたい | 「SDXL」「FLUX」「動画」「実験中」 |
|
||||
| グループ | 用途やプロジェクトごとにまとめたい | 「キャラクター制作」「背景素材」 |
|
||||
|
||||
### お気に入り（★）
|
||||
カードの **★** をクリックすると、お気に入りに登録されます。
|
||||
ツールバー上部の **★** ボタンを押すと、お気に入りだけを表示できます（**ALL** で全件表示に戻ります）。
|
||||
|
||||
お気に入りに登録したワークフローは、ComfyUI側のサイドパネル（WS Library）の
|
||||
「★ Favorites」にも表示されます（第9章）。
|
||||
|
||||
### バッジ
|
||||
バッジは、ワークフローに付ける **色付きのラベル** です。
|
||||
1つのワークフローに複数のバッジを付けることもできます。
|
||||
|
||||
**① バッジを作る**
|
||||
1. ツールバーの **「Badge」** ボタンを押します。
|
||||
2. **「バッジ名...」** の欄に名前を入力し、**「追加」** を押します。
|
||||
3. 色の見本をクリックすると、バッジの色を変えられます。
|
||||
|
||||
**② ワークフローにバッジを付ける**
|
||||
1. ワークフローをダブルクリックして、詳細画面を開きます。
|
||||
2. **「バッジ」** の欄で、付けたいバッジにチェックを入れます。
|
||||
3. **「保存」** を押します。
|
||||
|
||||
**③ バッジで絞り込む**
|
||||
バッジを作ると、ツールバー上部の ★・ALL ボタンの横にバッジが並びます。
|
||||
バッジをクリックすると、そのバッジが付いたワークフローだけが表示されます。
|
||||
|
||||
> **ポイント**
|
||||
> バッジの種類と色は、モデルタブ（第6章）と共通です。
|
||||
> 「SDXL」というバッジを作れば、ワークフローにもモデルにも同じ色で付けられます。
|
||||
|
||||
### グループ
|
||||
グループは、ワークフローを **用途ごとにまとめる箱** です。
|
||||
|
||||
1. ワークフローをクリックして選択します。
|
||||
2. 右側のサイドパネルで **「グループ」** タブを開きます。
|
||||
3. **「新規グループ作成」** でグループを作るか、既存のグループを選んで **「グループに追加」** します。
|
||||
|
||||
ツールバーの **「全グループ」** と書かれたプルダウンからグループを選ぶと、そのグループのワークフローだけが表示されます。
|
||||
|
||||
### タグとメモ
|
||||
詳細画面では、ほかにも次の情報を書き込めます。
|
||||
|
||||
- **タグ** － 検索用のキーワード。カンマ（,）で区切って複数入力できます。
|
||||
- **メモ** － 使い方の注意点や、うまくいった設定など、自由なメモ。
|
||||
|
||||
ネットから入手したワークフローに「配布元」や「必要なカスタムノード」をメモしておくと、あとで困りません。
|
||||
|
||||
### 名前の変更と削除
|
||||
- **名前を変える** － 詳細画面の上部にあるワークフロー名をクリックすると、名前を編集できます。
|
||||
- **削除する** － 詳細画面の **「削除」** ボタンを押します。
|
||||
|
||||
> **注意**
|
||||
> 削除すると、ComfyUIのワークフロー保存フォルダにあるファイルそのもの（サムネイル画像も含む）が削除されます。
|
||||
> 大切なワークフローは、削除する前に別の場所へバックアップしておきましょう。
|
||||
|
||||
---
|
||||
|
||||
## 4-5 検索・並べ替え・フィルタ解除
|
||||
|
||||
### 検索
|
||||
ツールバーの **「検索...」** 欄にキーワードを入力すると、名前やタグなどに一致するワークフローだけが表示されます。
|
||||
入力欄の右端の **×** を押すと、検索文字がすぐに消えます。
|
||||
|
||||
### 並べ替え
|
||||
検索欄の横のプルダウンで、並び順を変えられます。
|
||||
|
||||
| 選択肢 | 並び順 |
|
||||
| --- | --- |
|
||||
| Date (Newest) | 新しい順 |
|
||||
| Date (Oldest) | 古い順 |
|
||||
| Name A-Z | 名前の昇順 |
|
||||
| Name Z-A | 名前の降順 |
|
||||
|
||||
選んだ並び順は記憶され、次に開いたときも同じ順番で表示されます。
|
||||
|
||||
### フィルタをまとめて解除
|
||||
検索・グループ・バッジで絞り込んでいるうちに、「なぜか一覧に出てこない」と迷うことがあります。
|
||||
そんなときは、ツールバー右端の **「× クリア」** を押してください。
|
||||
検索文字・グループ・バッジの絞り込みが一度に解除され、すべてのワークフローが表示されます。
|
||||
|
||||
---
|
||||
|
||||
## 4-6 インポートとキャンバスへ送る
|
||||
|
||||
### ワークフローを取り込む（インポート）
|
||||
ネットから入手したワークフローのファイル（`.json` または ワークフロー入りの `.png`）は、
|
||||
次のどちらかの方法でワークフロータブに取り込めます。
|
||||
|
||||
- ツールバーの **「インポート」** ボタンを押して、ファイルを選ぶ（複数選択も可能）
|
||||
- ファイルを、ワークフロータブの一覧部分に **ドラッグ＆ドロップ** する
|
||||
|
||||
取り込んだワークフローは、ComfyUIのワークフロー保存フォルダに保存され、一覧に追加されます。
|
||||
|
||||
### 選んだワークフローを使う
|
||||
ワークフローをクリックして選択すると、ツールバーの次の3つのボタンが押せるようになります。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| 生成UIに読み込み | 生成UIタブに読み込み、ノード画面を触らずに生成できる状態にする（第5章） |
|
||||
| Videoに読み込み | 動画生成用のワークフローをVideoタブに読み込む（第11章） |
|
||||
| キャンバスへ送る | ComfyUIのノード画面にワークフローを読み込む |
|
||||
|
||||
> **ポイント**
|
||||
> ボタンが押せない（灰色の）ときは、まだワークフローを選択していません。
|
||||
> 一覧のワークフローを一度クリックしてから押してください。
|
||||
|
||||
**「キャンバスへ送る」** を押すと、Workflow Studioを開いたComfyUIのノード画面に、そのワークフローが読み込まれます。
|
||||
Workflow Studioで探して、ComfyUIで細かく編集する、という使い方ができます。
|
||||
|
||||
> **注意**
|
||||
> 「キャンバスへ送る」は、ComfyUIの **Wボタンから開いた** Workflow Studioで使えます。
|
||||
> ブックマークなどから直接開いた場合は、ComfyUI側のWS Libraryにそのワークフローが表示されるので、
|
||||
> そこからキャンバスへドラッグしてください（第2章・第9章）。
|
||||
|
||||
---
|
||||
|
||||
## 4-7 生成UIで動くか事前にチェックする（GenerateUIチェック）
|
||||
|
||||
すべてのワークフローが、生成UI（第5章）でそのまま使えるわけではありません。
|
||||
特殊なカスタムノードを使っていたり、手元にないモデルを指定していたりすると、うまく動かないことがあります。
|
||||
|
||||
そこで、生成UIに読み込む前に、使えそうかどうかを自動でチェックする機能があります。
|
||||
|
||||
### チェック結果の見方
|
||||
ワークフローをクリックすると、サイドパネルのワークフロー名の下に、次の情報が表示されます。
|
||||
|
||||
- **形式バッジ** － ワークフローの保存形式（UI形式／API形式／App形式）
|
||||
- **変換** － 生成UIで使える形に変換できるか
|
||||
- **操作性** － 生成UIの画面だけで、どのくらい設定を変えられるか
|
||||
|
||||
それぞれ、次の3段階で表示されます。
|
||||
|
||||
| 表示 | 意味 |
|
||||
| --- | --- |
|
||||
| ○（チェックマーク） | 問題なし |
|
||||
| △（警告マーク） | 使えるが注意が必要 |
|
||||
| ×（バツ印） | 生成UIでは使えない、または使いにくい |
|
||||
|
||||
バッジにマウスを重ねると、判定の理由が表示されます。
|
||||
|
||||
### よくある判定と対処
|
||||
| 判定 | よくある理由 | 対処 |
|
||||
| --- | --- | --- |
|
||||
| 変換 × | ワークフローに必要なカスタムノードがインストールされていない | ComfyUI Managerで不足しているノードをインストールする（第12章） |
|
||||
| 変換 △ | 指定されたモデルやLoRAが手元にない | 生成UIで手元のモデルを選び直す。UI形式の場合は、自動で一覧の先頭のモデルに差し替えられるので注意 |
|
||||
| 操作性 △／× | 生成UIに表示されない設定が多い | 表示されないノードの設定は、ComfyUIのノード画面で変更する |
|
||||
|
||||
> **ポイント**
|
||||
> 操作性は、「設定を持つノードのうち、生成UIに表示されるものの割合」です。
|
||||
> 80%以上で ○、50%以上で △、それ未満で × になります。
|
||||
> △ や × でも生成自体はできることが多いので、「生成UIだけで完結しにくい」という目安として見てください。
|
||||
|
||||
> **ポイント**
|
||||
> チェックは、そのワークフローで使っているノードの情報だけをComfyUIから取得して行います（v0.8.8〜）。
|
||||
> 以前は最初の1回に数十秒かかることがありましたが、今は一瞬で結果が出ます。
|
||||
|
||||
---
|
||||
|
||||
## 4-8 付属のサンプルワークフローを使ってみる
|
||||
|
||||
Workflow Studioには、すぐに試せるサンプルワークフローが付属しています。
|
||||
サンプルは、Workflow Studioをインストールしたフォルダの中の **`workflows`** フォルダにあります。
|
||||
|
||||
```
|
||||
ComfyUI/custom_nodes/ComfyUI-Workflow-Studio/workflows/
|
||||
```
|
||||
|
||||
各サンプルには、同じ名前の `.json`（ワークフロー本体）と `.png`（サムネイル画像）が入っています。
|
||||
|
||||
### はじめての方におすすめのサンプル
|
||||
| ファイル名 | 内容 |
|
||||
| --- | --- |
|
||||
| `ws_sdxl_t2i_basic` | SDXLモデルで、文章から画像を作る基本のワークフロー |
|
||||
| `ws_sd15_t2i_basic` | SD1.5モデルで、文章から画像を作る基本のワークフロー |
|
||||
|
||||
お手元にあるモデルの種類（SDXLかSD1.5か）に合わせて選んでください。
|
||||
|
||||
### サンプルを取り込む
|
||||
1. ワークフロータブの **「インポート」** を押します。
|
||||
2. 上記の `workflows` フォルダを開き、使いたいサンプルの `.png` ファイルを選びます。
|
||||
（`.png` を選ぶと、サムネイル付きで取り込まれます）
|
||||
3. 一覧に追加されたことを確認します。
|
||||
|
||||
ほかにも、ポーズ指定やインペイント（画像の一部の描き直し）など、応用的なサンプルも入っています。
|
||||
|
||||
> **注意**
|
||||
> 一部のサンプルは、追加のカスタムノードが必要です。
|
||||
> ノード画面で赤く表示されるノードがある場合は、ComfyUI Managerで不足しているノードをインストールしてください（第12章）。
|
||||
> 4-7のGenerateUIチェックで「変換 ×」と表示されるのも、多くはこのケースです。
|
||||
|
||||
次の章では、取り込んだ基本のサンプルを生成UIに読み込んで、いよいよはじめての1枚を生成します。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ComfyUIで保存したワークフローは、自動でワークフロータブに並ぶ
|
||||
> - クリックでサイドパネルに表示、ダブルクリックで詳細画面
|
||||
> - カメラボタンで、ワークフローに見分けやすいサムネイルを付けられる
|
||||
> - お気に入り・バッジ・グループで分類し、検索・並べ替えで探す。迷ったら「× クリア」
|
||||
> - 「生成UIに読み込み」「キャンバスへ送る」で、選んだワークフローをすぐ使える
|
||||
> - GenerateUIチェックで、生成UIで動くかを事前に確認できる
|
||||
@@ -0,0 +1,255 @@
|
||||
# 第5章 生成UIタブ － ノードを触らずに画像を作る
|
||||
|
||||
いよいよ、Workflow Studioで画像を生成します。
|
||||
**生成UI（GenerateUI）** タブは、ワークフローの中から「プロンプト」「モデル」「サイズ」など
|
||||
よく変える項目だけを取り出して、入力フォームとして表示してくれる画面です。
|
||||
ノードを探して線をたどる必要はありません。
|
||||
|
||||
この章の前半（5-1〜5-4）では、第4章で取り込んだ基本のサンプルを使って、はじめての1枚を生成します。
|
||||
|
||||
> **事前に準備しておくもの**
|
||||
> - ComfyUIが起動していること
|
||||
> - Checkpointモデルが1つ以上、ComfyUIの `models/checkpoints` フォルダに入っていること
|
||||
> - 第4章で、サンプルワークフロー（`ws_sdxl_t2i_basic` など）を取り込んでいること
|
||||
|
||||
---
|
||||
|
||||
## 5-1 ワークフローを生成UIに読み込む
|
||||
|
||||
1. **ワークフロー** タブを開きます。
|
||||
2. 使いたいワークフロー（例：`ws_sdxl_t2i_basic`）を **クリック** して選択します。
|
||||
3. サイドパネルの **GenerateUIチェック**（第4章 4-7）で、「変換」が ○ になっていることを確認します。
|
||||
4. ツールバーの **「生成UIに読み込み」** を押します。
|
||||
|
||||
自動で **生成UI** タブに切り替わり、ワークフローの内容が読み込まれます。
|
||||
画面上部に、読み込んだワークフローの名前が表示されていれば成功です。
|
||||
|
||||
> **ポイント**
|
||||
> 生成UIの画面は、ボタンや項目名の多くが英語のまま表示されます。
|
||||
> 本書では、英語の表示名をそのまま使い、必要に応じて意味を添えて説明します。
|
||||
|
||||
---
|
||||
|
||||
## 5-2 生成UIの画面の見方
|
||||
|
||||
生成UIタブは、大きく次の部分に分かれています。
|
||||
|
||||
| 番号 | 場所 | 役割 |
|
||||
| --- | --- | --- |
|
||||
| ① | 上部のバー | ComfyUIとの接続状態、ワークフロー名、各種ボタン |
|
||||
| ② | サブタブ | Input／Model／Settings／Feeder／Batch／Lab の切り替え |
|
||||
| ③ | 編集エリア（中央） | 選んだサブタブの設定項目 |
|
||||
| ④ | Raw JSON（右側） | ワークフローの中身をJSONで表示（上級者向け） |
|
||||
| ⑤ | 生成パネル | Generateボタン、進み具合、シード設定、生成結果 |
|
||||
|
||||
### ① 上部のバー
|
||||
| 表示・ボタン | 意味 |
|
||||
| --- | --- |
|
||||
| Connected／Disconnected | ComfyUIとつながっているか。緑の **Connected** なら生成できます |
|
||||
| Connect | ComfyUIに接続し直す |
|
||||
| Refresh Models | モデルの一覧を読み込み直す（ComfyUIにモデルを追加したあとに押す） |
|
||||
| Reset Workflow | 編集した内容を捨てて、ワークフローを読み込んだ直後の状態に戻す |
|
||||
| ワークフロー名 | いま読み込んでいるワークフロー |
|
||||
|
||||
> **ポイント**
|
||||
> 生成UIタブを開くと、ComfyUIへの接続は自動で行われます。
|
||||
> **Disconnected** と表示されている場合は、ComfyUIが起動しているか確認して **Connect** を押してください。
|
||||
|
||||
### ② サブタブ
|
||||
| サブタブ | 何を設定するか | 本書 |
|
||||
| --- | --- | --- |
|
||||
| Input | プロンプト、入力画像 | 5-3 |
|
||||
| Model | Checkpoint・VAE・LoRAなどのモデル | 5-3 |
|
||||
| Settings | シード・ステップ数・サンプラー・画像サイズなど | 5-4 |
|
||||
| Feeder | フォルダの画像を順番に処理する | 5-6 |
|
||||
| Batch | 条件を変えながらまとめて生成する | 5-5 |
|
||||
| Lab | 実験的なI2Iバッチ生成（上級） | 5-7 |
|
||||
|
||||
まずは **Input・Model・Settings** の3つだけ覚えれば十分です。
|
||||
|
||||
### ④ Raw JSON
|
||||
Input・Model・Settingsの右側には、ワークフローの中身がJSON形式で表示されます。
|
||||
フォームで値を変えると、ここにも反映されます。
|
||||
慣れないうちは、ここは「見るだけ」にしておきましょう。
|
||||
|
||||
Inputサブタブでは、Raw JSONの上端にある **»** ボタンで、Raw JSONを右端の細い帯に折りたためます（v0.8.12〜）。
|
||||
プロンプトや画像の欄が広くなります。もう一度押すと元に戻り、折りたたんだ状態は次回も記憶されます。
|
||||
|
||||
---
|
||||
|
||||
## 5-3 はじめての1枚を生成する
|
||||
|
||||
いよいよ生成です。最低限、次の3つを確認・設定します。
|
||||
|
||||
1. どのモデルで描くか（Model）
|
||||
2. 何を描くか（Input）
|
||||
3. 生成する（Generate）
|
||||
|
||||
### 手順1：モデルを選ぶ（Model）
|
||||
1. サブタブの **Model** を開きます。
|
||||
2. **Checkpoint** の欄で、手元にあるモデルを選びます。
|
||||
一覧が長い場合は、欄の上の入力欄に文字を入れると絞り込めます。
|
||||
3. 欄の下にある **Apply** ボタンを押します。
|
||||
|
||||
> **ポイント：モデルが見つからないとき**
|
||||
> ワークフローに保存されているモデルが、あなたのパソコンに入っていないことがあります。
|
||||
> 配布されているワークフローを使うときは、ほぼ必ず起こります。次のどちらかの形で知らせてくれます。
|
||||
> - 欄に「△ （モデル名）（見つかりません）」と表示される
|
||||
> - 読み込んだときに「このワークフローのCheckpoint「…」が見つからず、無警告で別のモデルに差し替えられました」と表示される
|
||||
> （この場合、手元のモデルの一覧の先頭のものに自動で差し替わっています）
|
||||
>
|
||||
> なお v0.8.7〜は、フォルダ名が違うだけで同じファイル名のモデルが手元に1つだけあれば、先頭のものではなく、そのモデルに自動で差し替わります。
|
||||
> （公式テンプレートはフォルダ名なしでモデル名を保存していることが多く、以前は無関係なモデルに差し替わることがありました）
|
||||
>
|
||||
> どちらの場合も、一覧から使いたいモデルを選び直して、Apply を押してください。
|
||||
> 毎回選び直すのが面倒なときは、設定タブの **デフォルトCheckpoint**（第11章）が便利です。
|
||||
|
||||
> **注意：Applyを押し忘れない**
|
||||
> 生成UIでは、モデルやSettings（5-4）の値を変えたあと **Apply** ボタンを押して、はじめてワークフローに反映されます。
|
||||
> Applyを押さずにGenerateすると、**変更前の値のまま** 生成されてしまいます。
|
||||
> 「設定を変えたのに結果が変わらない」ときは、まずApplyの押し忘れを疑いましょう。
|
||||
> （プロンプトの入力欄だけは、Generateを押したときに自動で反映されます）
|
||||
|
||||
> **ポイント**
|
||||
> 項目名が明るい緑色で表示されているのは、読み込んだワークフローにそのモデルのノードが含まれている項目です。
|
||||
> 緑色でない項目は、変更してもこのワークフローには影響しません。
|
||||
|
||||
### 手順2：プロンプトを書く（Input）
|
||||
1. サブタブの **Input** を開きます。
|
||||
左側にプロンプト（Prompt）の欄、右側に入力画像（Image）の欄が並んでいます（v0.8.12〜。それまではPrompt／Imageを切り替える形でした）。
|
||||
画面の幅が狭いときは、Imageの欄はプロンプトの下に回ります。
|
||||
2. **Positive Prompt** の欄に、描いてほしい内容を英語で入力します。
|
||||
|
||||
```
|
||||
a cute cat sitting on a windowsill, sunlight, soft colors, high quality
|
||||
```
|
||||
|
||||
3. **Positive Prompt** の **Apply** を押します。
|
||||
4. 必要に応じて **Negative Prompt** の欄に、描いてほしくない内容を入力し、こちらも **Apply** を押します。
|
||||
|
||||
> プロンプトは、Applyを押し忘れてもGenerateを押したときに自動で反映されます。
|
||||
> ただし、Raw JSON（右側）で反映結果を確かめられるので、Applyを押す習慣をつけておくと安心です。
|
||||
|
||||
```
|
||||
low quality, blurry, bad anatomy
|
||||
```
|
||||
|
||||
> **ポイント**
|
||||
> Apply の横にある **↺（戻す）** ボタンを押すと、入力欄の内容を、いまワークフローに反映されている値に戻せます。
|
||||
> 書き換えに失敗したときに便利です。
|
||||
|
||||
> **ポイント**
|
||||
> 英語が苦手な方は、第7章・第10章で紹介するAI翻訳機能を使うと、日本語で考えたプロンプトを英語にできます。
|
||||
|
||||
### 手順3：生成する（Generate）
|
||||
1. 画面の生成パネルにある **Generate** ボタンを押します。
|
||||
2. 進み具合を表すバーが伸びていき、パーセントが表示されます。
|
||||
3. 「Done」と表示されると、生成された画像が生成パネルに表示されます。
|
||||
|
||||
おめでとうございます。これで、Workflow Studioではじめての1枚が完成しました。
|
||||
|
||||
生成した画像は、ComfyUIで生成した場合と同じく、ComfyUIの `output` フォルダに保存されます。
|
||||
あとから見返したいときは、**ギャラリー** タブ（第8章）で一覧できます。
|
||||
|
||||
> **ポイント：途中で止めたいとき**
|
||||
> 生成中は、Generateボタンの横に **Stop** ボタンが表示されます。押すと生成を中断します。
|
||||
|
||||
> **ポイント：ApplyとGenerateを一度に**
|
||||
> **Alt キーを押しながら Apply** をクリックすると、値の反映と生成を一度に行えます。
|
||||
> 慣れてきたら、プロンプトを書き換えて Alt＋Apply、という流れで手早く試せます。
|
||||
|
||||
### 生成したワークフローを保存する
|
||||
生成UIで変えた設定を、あとでまた使いたいときは、サブタブの右端にある **Save** ボタンを押します。
|
||||
名前を付けて保存すると、ワークフロータブに新しいワークフローとして追加されます。
|
||||
|
||||
---
|
||||
|
||||
## 5-4 シード・サンプラー・ステップ数の意味
|
||||
|
||||
1枚目が生成できたら、次は **Settings** サブタブで、画像の仕上がりを左右する設定を見てみましょう。
|
||||
|
||||
Settingsタブは、3つの列に分かれています。
|
||||
|
||||
- **KSampler（左）** － 画像をどのように描くかの設定
|
||||
- **Latent Image（中央）** － 画像の大きさと枚数の設定
|
||||
- **スイッチ（右、v0.9.6〜）** － ワークフローの中のオン・オフの切り替え
|
||||
|
||||
ここでも、KSamplerとLatent Imageは、値を変えたら各ブロックの **Apply** を押すのを忘れないようにしましょう（スイッチの列はApply不要です）。
|
||||
|
||||
### KSampler の項目
|
||||
| 項目 | 意味 | 初心者向けの目安 |
|
||||
| --- | --- | --- |
|
||||
| Seed（シード） | 画像のもとになる「乱数の種」。同じ設定・同じシードなら、ほぼ同じ画像になる | 生成時は生成パネルのシード設定（下記）が優先されるので、ここは「読み込んだワークフローのシードの確認用」と考える。ワークフローのシードをそのまま使いたいときは、生成パネルのモードを **Workflow** にする |
|
||||
| Steps（ステップ数） | 描き込みを何回くり返すか。多いほど丁寧だが時間がかかる | 20〜30 |
|
||||
| CFG | プロンプトにどれだけ忠実に従うか。高すぎると色や形が崩れやすい | SDXLなら5〜8程度 |
|
||||
| Sampler（サンプラー） | 描き方のアルゴリズム | `euler`、`dpmpp_2m` など |
|
||||
| Scheduler（スケジューラー） | ステップごとの描き込み量の配分 | `normal`、`karras` など |
|
||||
| Denoise | 元の画像をどれだけ描き変えるか。文章から作る場合は 1.0 | 1.0（画像から作る場合は 0.4〜0.7 など） |
|
||||
|
||||
> **ポイント**
|
||||
> 目安の値はモデルによって異なります。モデルの配布ページ（Civitaiなど）に推奨値が書かれていることが多いので、確認してみましょう。
|
||||
> 迷ったら、サンプルワークフローに最初から入っている値のまま使えば大丈夫です。
|
||||
|
||||
### Latent Image の項目
|
||||
| 項目 | 意味 | 目安 |
|
||||
| --- | --- | --- |
|
||||
| Width（幅） | 画像の横の大きさ（ピクセル） | SDXLなら1024前後、SD1.5なら512前後 |
|
||||
| Height（高さ） | 画像の縦の大きさ（ピクセル） | 同上 |
|
||||
| Batch Size | 1回の生成で何枚作るか | 1〜4（多いほどメモリを使う） |
|
||||
|
||||
> **注意**
|
||||
> モデルが学習したサイズから大きく外れると、人物が二重になったり、構図が崩れたりしやすくなります。
|
||||
> 縦長・横長にしたいときも、合計のピクセル数がおおよそ同じになるように調整しましょう（例：SDXLなら 832×1216 など）。
|
||||
|
||||
### スイッチの項目（v0.9.6〜）
|
||||
ワークフローの中にある「オン・オフ」の設定が、ノード名と項目名つきで一覧になります。
|
||||
チェックボックスをクリックすると、その場で切り替わります。**Apply を押す必要はありません**。切り替えた値は右側の Raw JSON にもすぐ反映されます。
|
||||
|
||||
一覧に出るのは、次の2種類です。
|
||||
|
||||
- 値が true／false の項目 － プロンプト強化（PE）のオン・オフを切り替えるスイッチや「Boolean」ノード、TextGenerate の thinking など
|
||||
- 選択肢が2つだけのオン・オフの項目 － enable／disable、on／off、yes／no など。KSamplerAdvanced の add_noise、DWPreprocessor の detect_hand（手を検出するか）など
|
||||
|
||||
各行の右端には、いまの値（true／false、enable／disable など）が表示されます。
|
||||
行にマウスを乗せると、ノードの番号と種類を確認できます。
|
||||
|
||||
> **ポイント**
|
||||
> 別のノードから線でつながっている項目は、一覧に出ません。代わりに、つながっている元のノード（たとえば「Enable PE?」という名前の Boolean ノード）が出るので、そちらを切り替えてください。
|
||||
> ノードに名前を付けていないワークフローでは、ノードの種類名（ComfySwitchNode など）で表示されます。
|
||||
> オン・オフの項目が1つも無いワークフローでは「このワークフローに true/false の項目はありません」と表示されます。
|
||||
|
||||
### シードの管理（生成パネル）
|
||||
シードは、生成パネルの **Seed** 欄と、その下のプルダウンで管理します。
|
||||
|
||||
| モード | 動き | 使いどころ |
|
||||
| --- | --- | --- |
|
||||
| Random | 毎回ランダムなシードで生成 | いろいろな絵を試したいとき |
|
||||
| Fixed | Seed欄の値で固定 | 気に入った絵をもとに、プロンプトだけ変えて比べたいとき |
|
||||
| Increment | 生成するたびに1ずつ増やす | 少しずつ違う絵を順番に作りたいとき |
|
||||
| Decrement | 生成するたびに1ずつ減らす | Incrementの逆 |
|
||||
| Workflow | ワークフローに入っているシードを、上書きせずそのまま使う | ギャラリーから読み込んだ画像や、配布されたワークフローを、元のシードで再現したいとき |
|
||||
|
||||
生成が終わると、Seed欄には **実際に使われたシード値** が表示されます。
|
||||
|
||||
> **ポイント：Workflowモード（v0.8.3〜）**
|
||||
> モードを **Workflow** にすると、Seed欄の値やランダムではなく、読み込んだワークフロー自身のシードで生成します。
|
||||
> Randomで何枚か試したあとでも、ワークフロー本来のシード（読み込んだ時点の値）に戻って生成されます。
|
||||
> ワークフローにシードの入力が見つからない場合は、Seed欄の値（−1ならランダム）が使われます。
|
||||
> 同じシードと同じ設定で続けて2回押すと、ComfyUIは計算済みの結果を再利用するため、新しい画像は増えません（故障ではありません）。
|
||||
|
||||
### 気に入った絵を「育てる」流れ
|
||||
シードを使うと、気に入った1枚をもとに少しずつ改良できます。
|
||||
|
||||
1. **Random** で何枚か生成し、気に入った構図の絵を探す
|
||||
2. 気に入った絵が出たら、モードを **Fixed** に切り替える（Seed欄にはその絵のシードが入っています）
|
||||
3. プロンプトに言葉を足したり、Stepsを変えたりして、違いを比べる
|
||||
|
||||
シードを固定すると、「何を変えたら、どう変わったか」が分かりやすくなります。
|
||||
これは、画像生成に慣れるためのいちばんの近道です。
|
||||
|
||||
> **この章（前半）のまとめ**
|
||||
> - ワークフロータブで選んで「生成UIに読み込み」を押すと、生成UIで編集できる
|
||||
> - 基本は Input（プロンプト）・Model（モデル）・Settings（細かい設定）の3つのサブタブ
|
||||
> - モデルやSettingsの値を変えたら必ず **Apply**。押し忘れると変更前の値で生成される（プロンプトとSettingsのスイッチ列は自動で反映）
|
||||
> - Generate で生成。Alt＋Apply なら反映と生成を一度に行える
|
||||
> - シードを Fixed にすると、気に入った絵をもとに条件を変えて比べられる。ワークフローのシードをそのまま使うなら Workflow
|
||||
@@ -0,0 +1,358 @@
|
||||
## 5-5 バッチ生成 － 条件を変えながらまとめて試す
|
||||
|
||||
画像生成に慣れてくると、こんなことを試したくなります。
|
||||
|
||||
- 同じプロンプトで、手持ちのモデルを全部試して比べたい
|
||||
- いくつかのプロンプトを、順番に生成したい
|
||||
- どのサンプラーがこのモデルに合うのか、まとめて確かめたい
|
||||
|
||||
1枚ずつ設定を変えてGenerateを押すのは大変です。
|
||||
**Batch** サブタブを使うと、「何を変えながら生成するか」をまとめて指定し、自動で順番に生成できます。
|
||||
|
||||
### バッチで変えられるもの
|
||||
バッチでは、次の7種類のうち **1種類** を選んで、順番に変えながら生成します。
|
||||
|
||||
| 種類 | 何が変わるか | 使いどころ |
|
||||
| --- | --- | --- |
|
||||
| Checkpoint | モデル | モデルの描き味を比べる |
|
||||
| Lora | LoRA | LoRAの効果を比べる |
|
||||
| Prompt | プロンプト | 保存したプロンプトを順番に試す |
|
||||
| Workflow | ワークフロー | 複数のワークフローを順番に実行する |
|
||||
| Sampler | サンプラー | サンプラーの違いを比べる |
|
||||
| Scheduler | スケジューラー | スケジューラーの違いを比べる |
|
||||
| Style | スタイル | スタイルの違いを比べる |
|
||||
|
||||
> **ポイント**
|
||||
> 一度に変えられるのは1種類だけです。
|
||||
> 「モデルもプロンプトも同時に変えたい」という場合は、5-7の Lab を使います。
|
||||
|
||||
### Batchタブの画面
|
||||
Batchタブは、左・中央・右の3つの部分に分かれています。
|
||||
|
||||
| 場所 | 役割 |
|
||||
| --- | --- |
|
||||
| 左（リスト） | Checkpoints／Sampler／Scheduler／Style を一覧から選ぶ |
|
||||
| 中央（グループ） | Checkpoint／Lora／Prompt／Workflow を「グループ」単位で選ぶ |
|
||||
| 右（Batch Queue） | 選んだ項目の一覧。実行する種類をここで決める |
|
||||
|
||||
中央のグループは、それぞれのタブで作ったグループと連動しています。
|
||||
|
||||
- Checkpoint・Lora のグループ → モデルタブ（第6章）
|
||||
- Prompt のグループ → プロンプトタブ（第7章）
|
||||
- Workflow のグループ → ワークフロータブ（第4章）
|
||||
|
||||
### 例：手持ちのモデルを比べる
|
||||
同じプロンプト・同じシードで、3つのモデルの描き味を比べてみましょう。
|
||||
|
||||
1. 5-3の手順で、比べたいプロンプトを入れたワークフローを読み込んでおきます。
|
||||
2. 生成パネルのシードのモードを **Fixed** にします。
|
||||
（シードを固定すると、モデル以外の条件がそろい、違いが分かりやすくなります）
|
||||
3. **Batch** サブタブを開きます。
|
||||
4. 左側の **Checkpoints** タブで、比べたいモデルにチェックを入れます。
|
||||
5. 右側の **Batch Queue** で、**Checkpoint** の列の見出しにあるチェックボックスをオンにします。
|
||||
列には、選んだモデルの一覧と件数が表示されます。
|
||||
|
||||
6. **Generate** を押します。
|
||||
|
||||
選んだモデルで順番に生成が行われます。
|
||||
生成パネルの **Batch** 欄に、いま何番目を生成しているか、全体の進み具合が表示されます。
|
||||
|
||||
- **Pause** － 一時停止します。もう一度押すと再開します。
|
||||
- **Stop** － バッチ全体を中止します。
|
||||
|
||||
> **注意**
|
||||
> Batch Queueの列のチェックがオンのままだと、次にGenerateを押したときもバッチ生成になります。
|
||||
> 1枚だけ生成したいときは、列のチェックをオフにしてください。
|
||||
|
||||
> **ポイント：LoRAのバッチ**
|
||||
> Loraのバッチでは、選んだLoRAの書式（`<lora:名前:強さ>`）と、Civitaiから取得したトリガーワードが、
|
||||
> 自動でPositive Promptに追加されます。バッチが終わると、元のプロンプトに戻ります。
|
||||
|
||||
---
|
||||
|
||||
## 5-6 Feeder － フォルダの画像を順番に処理する
|
||||
|
||||
**Feeder** は、たくさんの画像を1枚ずつワークフローに送り込んで、順番に処理する機能です。
|
||||
たとえば、次のような使い方ができます。
|
||||
|
||||
- 撮りためた写真を、まとめてイラスト風に変換する（I2I：画像から画像を作る）
|
||||
- 複数の画像からポーズを読み取って、同じポーズのキャラクターを描く
|
||||
|
||||
Feederには2つのモードがあり、Feederタブ左上の **[Image Loop]／[Gallery]** ボタンで切り替えます。
|
||||
|
||||
| モード | 画像の取り出し元 | 必要なもの |
|
||||
| --- | --- | --- |
|
||||
| Image Loop | パソコン内のフォルダ | カスタムノード **comfyui-image-feeder**（別途インストール） |
|
||||
| Gallery | ギャラリータブのグループ | 追加インストール不要（Workflow Studioに付属の **Gallery Feeder (WFS)** ノード） |
|
||||
|
||||
> **ポイント**
|
||||
> どちらのモードも、**ワークフローの中に専用のノード** が入っている必要があります。
|
||||
> 普通のワークフローを読み込んだだけでは使えません。
|
||||
|
||||
### Galleryモードの使い方（追加インストール不要）
|
||||
1. ComfyUIのノード画面で、ワークフローの「画像を読み込む」部分に **Gallery Feeder (WFS)** ノードを置き、
|
||||
画像の出力をつなぎます。保存して、生成UIに読み込みます。
|
||||
2. **ギャラリー** タブ（第8章）で、処理したい画像をグループにまとめておきます。
|
||||
（Feederタブを初めて開くと、`__Feeder__` というグループが自動で作られます。これを使ってもかまいません）
|
||||
3. 生成UIの **Feeder** サブタブを開き、**[Gallery]** を選びます。
|
||||
4. 対象の **Gallery Feeder Node** と、画像を取り出す **グループ** を選び、**Apply** を押します。
|
||||
5. 中央にグループの画像が並びます。画像をクリックすると、その画像から処理を始められます。
|
||||
6. **After gen**（生成後の動き）を選び、**▶ Run** を押します。
|
||||
|
||||
### After gen（生成後の動き）
|
||||
| モード | 動き |
|
||||
| --- | --- |
|
||||
| Loop | 次の画像へ進み、最後まで行ったら最初に戻って続ける（■ Stopを押すまで止まらない） |
|
||||
| Increment | 次の画像へ進み、最後の画像を処理したら自動で止まる |
|
||||
| Fixed | 同じ画像を使い続ける（シードだけ変えて何枚も作りたいとき） |
|
||||
|
||||
はじめは **Increment** を選ぶと、全部の画像を1回ずつ処理して自動で止まるので安心です。
|
||||
|
||||
### Image Loopモード
|
||||
Image Loopモードでは、パソコン内のフォルダ（`ComfyUI/user/default/image-loop-data/`）の画像を使います。
|
||||
フォルダの画像を選んで、プリセットとして保存しておくこともできます。
|
||||
別途 **comfyui-image-feeder** カスタムノードが必要です。
|
||||
|
||||
付属のサンプル `ws_sdxl_image_feeder_i2i`（I2I）と `ws_dwpose_image_feeder`（ポーズ読み取り）は、このImage Loopモード用です。
|
||||
|
||||
### Galleryモードのサンプル（v0.8.3〜）
|
||||
Galleryモードには、付属のサンプル `ws_sdxl_gallery_feeder_i2i`（SDXLのI2I）が使えます。
|
||||
ノードを自分で置く必要がなく、**Gallery Feeder (WFS)** ノードがあらかじめ入っています（グループ名は `__Feeder__`、並び順は filename_asc）。
|
||||
|
||||
1. ワークフロータブでサンプルを取り込み、**生成UIに読み込み** を押します。
|
||||
2. ギャラリーで、処理したい画像のサムネイル左上の **F** ボタンを押して `__Feeder__` グループに入れます。
|
||||
3. **Feeder** サブタブで **[Gallery]** を選び、Gallery Feeder Node とグループを選んで **Apply**。
|
||||
4. Model サブタブで、手元のCheckpointとVAEを選び直して **Apply**（サンプルのモデル名は作者の環境のものです）。
|
||||
5. **After gen** を **Increment** にして **▶ Run** を押します。
|
||||
|
||||
---
|
||||
|
||||
## 5-7 Lab － 実験的なI2Iバッチ生成（上級）
|
||||
|
||||
**Lab** は、5-5のバッチ生成をさらに発展させた、上級者向けの実験的な機能です。
|
||||
はじめのうちは読み飛ばしてもかまいません。
|
||||
|
||||
### Batchとの違い
|
||||
| | Batch（5-5） | Lab |
|
||||
| --- | --- | --- |
|
||||
| 一度に変えられるもの | 1種類だけ | モデル・プロンプト・KSamplerを **それぞれ別々に** |
|
||||
| 変えるタイミング | 毎回 | 「何回目から変えるか」を自由に指定 |
|
||||
| 生成した画像を次の入力に | できない | できる（連鎖させられる） |
|
||||
|
||||
### キーフレームという考え方
|
||||
Labでは、「何回目の生成から、どの設定に切り替えるか」を **キーフレーム** として指定します。
|
||||
|
||||
例：10回生成する場合
|
||||
|
||||
| 回 | モデル | プロンプト | KSampler |
|
||||
| --- | --- | --- | --- |
|
||||
| 1〜10 | モデルA | － | － |
|
||||
| 1〜4 | － | 「朝の街」 | － |
|
||||
| 5〜10 | － | 「夜の街」 | － |
|
||||
| 1〜7 | － | － | Steps 20 |
|
||||
| 8〜10 | － | － | Steps 40 |
|
||||
|
||||
このように、列ごとに切り替えのタイミングを別々に決められます。
|
||||
次のキーフレームが来るまでは、前の設定が引き継がれます。
|
||||
|
||||
### 基本の流れ
|
||||
1. 生成UIにワークフローを読み込みます。
|
||||
2. **Lab** サブタブを開きます。
|
||||
3. I2I（画像から画像）の場合は、元になる画像をドロップします。
|
||||
文章から作る（T2I）場合は、**T2I workflow (no source image)** にチェックを入れます。
|
||||
4. 生成回数（Batch）を決めます。
|
||||
5. Model・Prompt・KSampler の各列で **＋** を押してキーフレームを追加し、
|
||||
キーフレームをダブルクリックして、「何回目から」「どの設定に」するかを入力します。
|
||||
KSampler列のセルには、指定した項目（Step・CFG・Denoise・サンプラー・スケジューラー・Seed）が表示されます。
|
||||
何も指定していないセルは「（ワークフローの設定のまま）」と表示されます。
|
||||
6. **Run** を押します。
|
||||
|
||||
結果は **Results** パネルに一覧表示されます。
|
||||
|
||||
### 便利な機能
|
||||
- **Use generated image for next** － 生成した画像を、次の回の入力画像として使います。
|
||||
少しずつ変化していく連続画像を作れます。
|
||||
- **Plan Save** － 設定（プラン）を保存できます。保存すると、結果をまとめた一覧画像も作られ、
|
||||
その画像をドロップすれば同じプランを読み込み直せます。
|
||||
- **物語からプランを作る**（v0.9.5〜） － AI TOOLのChatで **Storyboard to Lab** スキルを使うと、
|
||||
物語をシーンごとのプロンプトに分けたプランを作り、**Labで開く** ボタンでここに読み込めます（第10章 10-5）。
|
||||
|
||||
---
|
||||
|
||||
## 5-8 パートナーAPI（Nano Banana）を使う
|
||||
|
||||
ComfyUIには、Googleの画像生成モデル **Nano Banana**（Gemini画像ノード）のように、
|
||||
クラウドのサービスに生成を任せる **パートナーAPIノード** があります。
|
||||
v0.8.6〜は、こうしたノードを含むワークフローも生成UIで実行できます。
|
||||
|
||||
> **注意**
|
||||
> パートナーAPIノードは、自分のパソコンのGPUではなくクラウドで動き、**Comfyクレジット（有料）を消費** します。
|
||||
> BatchやLabなどで繰り返し実行すると、回数ぶんのクレジットが続けて消費されます。
|
||||
|
||||
### 準備：Comfy APIキーを設定する
|
||||
1. platform.comfy.org でComfy APIキーを発行します。
|
||||
2. プラグインのフォルダにある `.env` に、`COMFY_API_KEY=発行したキー` という行を追加します。
|
||||
（`.env` がなければ `.env.example` をコピーして作ります）
|
||||
3. ComfyUIを再起動します。`.env` は起動時にしか読み込まれません。
|
||||
|
||||
キーはサーバー側でだけ使われ、ブラウザには送られません。
|
||||
キーを設定していない状態で実行すると、「パートナーAPIノードの認証に失敗しました」という案内が出ます。
|
||||
|
||||
### 使い方
|
||||
1. ワークフロータブで、Nano Bananaのサンプル（`ws_nano_banana_2_1_t2i`）を生成UIに読み込みます。
|
||||
2. **Input** サブタブの左側（Prompt）でプロンプトを編集し、**Apply** を押します。
|
||||
3. 参考画像を使うときは、**Input** サブタブの右側（Image）の画像カードで指定します。
|
||||
4. **Generate** を押します。
|
||||
|
||||
> **ポイント**
|
||||
> ComfyUI本体の画面でログインして実行する方法もあります。その場合は `.env` の設定は要りません。
|
||||
|
||||
---
|
||||
|
||||
## 5-9 対応ワークフローが増えました（v0.8.4〜）
|
||||
|
||||
生成UIで使えるワークフローが、次のように増えています。使い方は、ほかのワークフローと同じです。
|
||||
|
||||
| 対応したもの | 内容 |
|
||||
| --- | --- |
|
||||
| Qwen Image 2.1 Turbo | ComfyUI公式テンプレートの文章から画像（T2I）・画像から画像（I2I）。プロンプトエンハンサー（プロンプトを自動で補う機能）の入力欄にも、編集が反映される |
|
||||
| Flux.3 Klein | 公式テンプレート4種（KleinとKlein SDのT2IとImage Edit） |
|
||||
| MOSS-Image | Base／SFTのT2Iと、枠で位置を指定するバウンディングボックス版 |
|
||||
| Fibo 2.0 Edit（v0.8.14〜） | 画像編集のワークフロー。プロンプト強化（PE）を使う構成でも、プロンプト欄の編集がPEのオン・オフどちらにも反映される。ネガティブの欄にはVGL形式（JSON）の説明文がそのまま入る。幅・高さ（Settings）は0で自動 |
|
||||
| FastH3 OmniRef／Pika-H3-Flex（v0.8.14〜） | MiniMax H3の「参照画像から動画」（Reference to Video）。動画のワークフローとして扱われ、Videoタブ（第11章）でも使える |
|
||||
| ネイティブLoRAスタック | ComfyUI v0.40のLoRAをまとめて読み込むノード（LoadLoraModel／LoadLoraTextEncoder）。Apply・バイパス・Metadata表示に対応 |
|
||||
| P-H3 Image（v0.9.4〜） | 画像編集とT2Iのテンプレート。プロンプトはInputタブに表示され、Metadataタブでも読める。ネガティブの欄は無い。幅・高さはワークフローのつなぎ方に従い、プロンプトを補う機能（プロンプトアップサンプラー）もワークフローの設定どおりに動く |
|
||||
| MiniMax H3 Image（v0.9.6〜） | ComfyUI公式の画像編集とT2Iのテンプレート。P-H3 Imageと同じく、プロンプトはInputタブに表示され、MetadataタブとサイドパネルのIタブでも読める。ネガティブの欄は無い。プロンプト強化（PE）のオン・オフは、Settingsタブのスイッチの列（5-4）で切り替えられる |
|
||||
|
||||
> **注意**
|
||||
> ネイティブLoRAスタックのノードは、保存形式が **API形式のワークフローのみ** 対応です。
|
||||
> ComfyUI本体の画面がこのノードの行追加にまだ対応していないためで、UI形式のワークフローは対象外です。
|
||||
|
||||
> **ポイント**
|
||||
> 2枚の画像をスライダーで見比べる **Compare Images**（ImageCompare）ノードを含むワークフローは、以前は生成UIから実行すると、このノードだけエラーになっていました（v0.9.6で修正）。
|
||||
|
||||
> **ポイント**
|
||||
> 画像のメタデータを見るMetadataタブとサイドパネルの **I** タブでも、Nano Bananaで作った画像を読めます（v0.8.5〜）。
|
||||
> プロンプトは、Positive／Negativeに分けずに全文が1件で表示されます。
|
||||
|
||||
## 5-10 日本語のままプロンプトを書く（プロンプト翻訳、v0.9.9〜）
|
||||
|
||||
多くの画像生成モデルは、英語のプロンプトで学習されています。
|
||||
**プロンプト翻訳** をオンにすると、プロンプトを日本語で書き、ボタン1つで英語に直せます。
|
||||
Stable Diffusion web UI（A1111）の翻訳拡張機能と同じように、プロンプトの中の **日本語の部分だけ** を翻訳します。
|
||||
|
||||
翻訳には、パソコンで動くAI（**Ollama**）と翻訳専用のモデル **PLaMo 3 Translate** を使います。
|
||||
インターネット上の翻訳サービスにプロンプトを送ることはありません。
|
||||
|
||||
> **事前に必要なもの**
|
||||
> **Ollama** をインストールして起動しておきます（第10章 10-1）。翻訳モデルのダウンロードは、設定画面からできます（下記）。
|
||||
|
||||
### 準備：設定でオンにする
|
||||
1. **設定** タブの **プロンプト翻訳（自国語 → 英語）** を開きます。
|
||||
2. **生成UIでプロンプト翻訳を使う** にチェックを入れます。
|
||||
3. **翻訳モデル** で、使うモデルを選びます。まだOllamaに入っていないモデルには **（未取得）** と付いています。選んで **ダウンロード** を押すと、Ollamaに取り込まれます。
|
||||
4. **テスト** を押して、見本のプロンプトが英語になれば準備完了です。
|
||||
|
||||
PLaMo 3 Translate には4つの大きさがあります。
|
||||
|
||||
| モデル | 大きさ | 特徴 |
|
||||
| --- | --- | --- |
|
||||
| 0.8B | 約0.6GB | 最速。短いタグの翻訳なら十分 |
|
||||
| 2B | 約1.8GB | おすすめ。文章の翻訳も正確 |
|
||||
| 8B | 約5.3GB | より高品質 |
|
||||
| 31B | 約19.5GB | 最高品質。メモリに余裕がある人向け |
|
||||
|
||||
> **ポイント**
|
||||
> 0.8Bは、文章の翻訳で意味がずれることがあります（例：「カメラは後ろから追いかける」が「her camera following behind」＝「彼女のカメラが後ろについてくる」になった）。
|
||||
> 動画のプロンプトのように文章で書くことが多いなら、2B以上を選びましょう。
|
||||
|
||||
設定のほかの項目は、次のとおりです。
|
||||
|
||||
| 項目 | 内容 |
|
||||
| --- | --- |
|
||||
| 入力する言語 | ふつうは **自動判定** のままでよい。日本語だけで書くなら「日本語」を選ぶと確実 |
|
||||
| Ollama URL | Ollamaのアドレス。空欄なら `http://localhost:11434` |
|
||||
| 翻訳モデルの実行場所 | **GPU** か **CPUで常駐** か（下記） |
|
||||
| 実行時に自動で翻訳する | Generateなどを押したときに自動で翻訳する（下記） |
|
||||
|
||||
PLaMo以外のOllamaのモデル（ふだんチャットに使っているモデルなど）も、**その他のOllamaモデル** から選べます。その場合は「英語に翻訳して」という指示を付けて送ります。
|
||||
|
||||
### 翻訳モデルをどこで動かすか
|
||||
| 実行場所 | 1語句あたりの時間 | 向いている人 |
|
||||
| --- | --- | --- |
|
||||
| GPU（1回の翻訳が最速・VRAMを使う） | 約0.1秒 | GPUのメモリに余裕がある |
|
||||
| CPUで常駐（VRAMを使わず、再読み込みの待ちなし） | 約0.2〜0.3秒 | 画像生成にGPUのメモリをすべて使いたい |
|
||||
|
||||
GPUのときは、**翻訳後に翻訳モデルをVRAMから解放する** をオンにしておくと、画像生成の前にGPUのメモリを空けられます（初期値はオン）。
|
||||
ただし、翻訳のたびにモデルの読み込み直しで **約4秒** 待つことになります。
|
||||
|
||||
**CPUで常駐** にすると、モデルをメインメモリに置いたままにします（2Bで約2.7GB）。
|
||||
GPUのメモリを使わず、生成UIにワークフローを読み込んだときに裏で読み込んでおくので、最初の翻訳から待たされません。
|
||||
|
||||
> **ポイント**
|
||||
> Ollamaは、ほかのモデル（AI TOOLのチャット用など）を読み込むときに、常駐させた翻訳モデルを外すことがあります。
|
||||
> その場合は、次の翻訳のときに自動で読み込み直します（その1回だけ数秒かかります）。
|
||||
|
||||
### 使い方：🌐ボタンで翻訳する
|
||||
プロンプト翻訳をオンにすると、InputタブのPositive Prompt・Negative Promptの **↺** ボタンの右に **🌐** ボタンが表示されます。
|
||||
|
||||
1. プロンプト欄に、日本語で書きます。英語のタグと混ぜてもかまいません。
|
||||
|
||||
```
|
||||
masterpiece, 夕焼けの海辺に立つ少女、(長い黒髪:1.2), <lora:test_lora:0.8>, {猫耳|狐耳}, 水彩画風
|
||||
```
|
||||
|
||||
2. **🌐** を押します。
|
||||
3. 日本語の部分だけが英語になります。
|
||||
|
||||
```
|
||||
masterpiece, A girl standing by the seaside at sunset, (long black hair:1.2), <lora:test_lora:0.8>, {cat ears|fox ears}, watercolor style
|
||||
```
|
||||
|
||||
次のようなプロンプトの書き方（記法）は、翻訳せずにそのまま残します。
|
||||
|
||||
- カンマ・改行
|
||||
- 重み付け `(語句:1.2)`、`[ ]`
|
||||
- ランダムに選ぶ `{A|B}`、ワイルドカード `__名前__`
|
||||
- LoRAの指定 `<lora:名前:0.8>`、`embedding:名前`
|
||||
- `BREAK`、`AND`
|
||||
|
||||
翻訳したあとに **原文** ボタンが表示されます。押すと、翻訳する前の文に戻ります（Ctrl＋Zでも戻せます）。
|
||||
|
||||
> **注意**
|
||||
> 翻訳されるのは、英数字以外の文字を含む部分だけです。英語だけの部分は、そのまま残ります。
|
||||
> 反対に、é などのアクセント付きの文字を含む英単語は、翻訳の対象になります。
|
||||
|
||||
### 実行時に自動で翻訳する
|
||||
設定の **実行時に自動で翻訳する** で、どの実行のときに自動で翻訳するかを、チェックボックスで選べます。
|
||||
|
||||
| チェックボックス | 動き |
|
||||
| --- | --- |
|
||||
| 生成UI － Generate | Generateを押すと、Positive／Negativeの欄の日本語を英語に置き換えてから生成する |
|
||||
| 生成UI Lab － Run | Labの実行で、送る内容だけを翻訳する。Labのプランには日本語が残る |
|
||||
| Video Plan － Run | Videoタブ Planの実行で、送るプロンプトだけを翻訳する。プランには日本語が残る |
|
||||
|
||||
LabとVideo Planでは、プランを日本語のまま残すので、あとから日本語で書き直せます。
|
||||
生成した画像や動画の情報（メタデータ）には、実際に送った英語のプロンプトが記録されます。
|
||||
|
||||
> **注意**
|
||||
> 自動翻訳がオンのときに翻訳に失敗すると（Ollamaが起動していないなど）、日本語のまま生成しないよう、生成を止めます。
|
||||
> LabとVideo Planでは、止まった理由が進行状況の欄に表示されます。
|
||||
|
||||
### Labのモーダルと、Video Planでも使える
|
||||
🌐ボタンは、次の欄にもあります。使い方は同じです。
|
||||
|
||||
- Lab（5-7）のプロンプト列のセルをダブルクリックして開く画面の、Positive・Negativeの欄
|
||||
- Videoタブ Planサブタブの、ブロックの **Prompt** の欄（第11章 11-4）
|
||||
|
||||
> **ポイント**
|
||||
> PLaMo 3 Translate は、個人の利用や趣味、研究などの非商用の利用なら無料で使えます（PLaMoコミュニティライセンス）。
|
||||
> 商用で使う場合は、ライセンスの条件を確認してください。
|
||||
|
||||
> **この章（後半）のまとめ**
|
||||
> - Batchサブタブで、モデル・プロンプト・サンプラーなどを1種類ずつ変えながらまとめて生成できる
|
||||
> - Batch Queueの列のチェックで種類を選び、Generateで開始。終わったらチェックを外す
|
||||
> - Feederは、たくさんの画像を1枚ずつワークフローに送り込む機能。専用ノードが必要
|
||||
> - Labは、複数の設定を別々のタイミングで切り替えられる上級者向けの機能
|
||||
> - パートナーAPIノード（Nano Bananaなど）は、`.env` の `COMFY_API_KEY` を設定すれば実行できる。クレジットが消費される
|
||||
> - プロンプト翻訳をオンにすると、日本語で書いたプロンプトを🌐ボタンで英語にできる。記法はそのまま残り、Generate・Lab・Video Planの実行時に自動で翻訳することもできる
|
||||
@@ -0,0 +1,184 @@
|
||||
# 第6章 モデルタブ － モデルとLoRAを管理する
|
||||
|
||||
画像生成を続けていると、モデルやLoRAのファイルはどんどん増えていきます。
|
||||
「このファイル、何のモデルだっけ？」「あのLoRAのトリガーワードは何だった？」
|
||||
－ そんな悩みを解決するのが **モデル（Models）** タブです。
|
||||
|
||||
---
|
||||
|
||||
## 6-1 モデルの種類
|
||||
|
||||
モデルタブの上部には、モデルの種類ごとのタブが並んでいます。
|
||||
それぞれ、ComfyUIの `models` フォルダの中の対応するフォルダに入っているファイルが表示されます。
|
||||
|
||||
| 種類 | ひとことで言うと | 主な置き場所 |
|
||||
| --- | --- | --- |
|
||||
| Checkpoint | 画像生成の本体。これがないと始まらない | `models/checkpoints` |
|
||||
| LoRA | 絵柄・キャラクター・服装などを追加で覚えさせる小さなモデル | `models/loras` |
|
||||
| VAE | 画像の色合いや細部を仕上げる。Checkpointに内蔵されていることも多い | `models/vae` |
|
||||
| ControlNet | ポーズや線画など、構図を指定するためのモデル | `models/controlnet` |
|
||||
| UNET | FLUXなど新しい形式のモデルの本体（Diffusion Model） | `models/diffusion_models` など |
|
||||
| TextEncoder | プロンプトの文章をAIが理解できる形に変換するモデル | `models/text_encoders` など |
|
||||
| Hypernetwork | 絵柄を調整する古い形式の追加モデル | `models/hypernetworks` |
|
||||
| Embedding | 特定の単語に意味を覚えさせた小さなファイル（ネガティブ用が多い） | `models/embeddings` |
|
||||
|
||||
> **ポイント**
|
||||
> はじめは **Checkpoint** と **LoRA** の2つを使いこなせれば十分です。
|
||||
> FLUXなどの新しいモデルを使うようになると、UNETやTextEncoderの出番が増えてきます。
|
||||
|
||||
---
|
||||
|
||||
## 6-2 一覧の表示と探し方
|
||||
|
||||
### 最初の読み込み
|
||||
モデルタブを初めて開いたときは、一覧が空で「更新ボタンでモデルを読み込み...」と表示されます。
|
||||
ツールバーの **「更新」** を押すと、モデルの一覧が表示されます。
|
||||
新しくモデルを追加したときも、「更新」を押してください。
|
||||
|
||||
### サムネイル表示とテーブル表示
|
||||
ワークフロータブと同じく、**サムネイル** と **テーブル** を切り替えられます。
|
||||
1ページに24件ずつ表示されます。
|
||||
|
||||
テーブル表示では、列の見出しをクリックすると、その列で並べ替えができます
|
||||
（1回目：昇順 ▲、2回目：降順 ▼、3回目：並べ替え解除）。
|
||||
|
||||
> **ポイント**
|
||||
> サムネイルは、モデルファイルと同じ場所にある `（モデル名）.preview.png` が自動で使われます。
|
||||
> 画像がないモデルも、6-3の方法でCivitaiから取得したり、自分で設定したりできます。
|
||||
|
||||
### 検索と絞り込み
|
||||
ツールバーには、絞り込みのための項目が並んでいます。
|
||||
|
||||
| 項目 | 絞り込み内容 |
|
||||
| --- | --- |
|
||||
| モデルを検索... | 名前などのキーワード |
|
||||
| ★ | お気に入りだけ |
|
||||
| すべてのタグ | タグ |
|
||||
| All Folders | サブフォルダ |
|
||||
| All Groups | グループ |
|
||||
| All／Enabled only／Disabled only | 有効・無効の状態（6-5） |
|
||||
|
||||
絞り込みすぎて分からなくなったら、ツールバー右端の **「× クリア」** ですべて解除できます。
|
||||
|
||||
### お気に入り・バッジ・グループ
|
||||
分類のしかたは、ワークフロータブ（第4章 4-4）とほぼ同じです。
|
||||
|
||||
- **お気に入り（★）** － よく使うモデルに印を付ける
|
||||
- **バッジ** － **「Badge」** で作った色付きラベルを付ける。バッジの種類と色はワークフロータブと共通です
|
||||
- **グループ** － モデルをクリックし、サイドパネルの **「グループ」** タブで追加する
|
||||
|
||||
> **ポイント**
|
||||
> グループは **モデルの種類ごと** に分かれています。
|
||||
> Checkpointのグループは、Checkpointタブにだけ表示されます。
|
||||
> ここで作ったグループは、生成UIのバッチ生成（第5章 5-5）でも使えます。
|
||||
|
||||
---
|
||||
|
||||
## 6-3 Civitaiの情報を取り込む
|
||||
|
||||
**Civitai（シビタイ）** は、画像生成モデルが多数公開されている共有サイトです。
|
||||
ネットから入手したモデルの多くは、Civitaiで配布されています。
|
||||
|
||||
Workflow Studioは、手元のモデルファイルから **ハッシュ値**（ファイルの指紋のような値）を計算し、
|
||||
Civitaiに登録されている情報を自動で探し出せます。
|
||||
|
||||
### 取得できる情報
|
||||
- モデルの名前・種類（Checkpoint／LoRAなど）
|
||||
- **Base Model**（SDXL、SD1.5、FLUX など、どの系統のモデルか）
|
||||
- **トリガーワード**（LoRAの効果を出すために、プロンプトに入れる言葉）
|
||||
- タグ・説明文
|
||||
- サンプル画像（サムネイルにも使われます）
|
||||
|
||||
### 1つずつ取得する
|
||||
1. モデルをクリックして選択します。
|
||||
2. 右側のサイドパネルで **「CivitAI」** タブを開きます（モデルを選ぶと最初にこのタブが開きます）。
|
||||
3. **「CivitAIから取得」** を押します。
|
||||
|
||||
取得が終わると、モデルの情報が表示されます。
|
||||
**Sample** タブでは、Civitaiに投稿されたサンプル画像を見られます。
|
||||
|
||||
### まとめて取得する
|
||||
ツールバーの **「CivitAI」** ボタンを押すと、表示中の種類のモデルについて、まとめて情報を取得します。
|
||||
すでに情報を取得済みのモデルは飛ばされるので、モデルを追加したあとに何度押しても大丈夫です。
|
||||
サムネイル画像がないモデルには、プレビュー画像も自動で保存されます。
|
||||
|
||||
> **ポイント**
|
||||
> ハッシュ値の計算には、モデルの大きさに応じて時間がかかります。
|
||||
> 数GBのモデルがたくさんある場合は、時間に余裕があるときに実行しましょう。
|
||||
|
||||
> **ポイント**
|
||||
> Civitaiで見つからなかったモデルは「確認済みですが、このモデルはCivitAIに見つかりませんでした。」と表示されます。
|
||||
> 自分で作ったモデルや、Civitai以外で配布されているモデルは、見つからないのが普通です。
|
||||
> **「未登録はスキップ」** にチェックを入れておくと、まとめて取得するときに、見つからなかったモデルを再確認せずに飛ばします。
|
||||
|
||||
> **ポイント：APIキー**
|
||||
> 設定タブの **「CivitAI APIキー」** は、入力しなくても使えます（省略可）。
|
||||
> 一部の情報の取得に認証が必要な場合に、Civitaiのアカウント設定で発行したキーを入力します。
|
||||
|
||||
### LoRAを使うときに特に便利
|
||||
LoRAは、**トリガーワード** をプロンプトに入れないと効果が出ないものが多くあります。
|
||||
Civitaiの情報を取得しておけば、サイドパネルでトリガーワードをすぐに確認できます。
|
||||
また、生成UIのバッチ生成でLoRAを使うと、トリガーワードが自動でプロンプトに追加されます（第5章 5-5）。
|
||||
|
||||
---
|
||||
|
||||
## 6-4 モデルを生成やキャンバスで使う
|
||||
|
||||
### 生成UIにセットする（GenUI Model）
|
||||
モデルタブで選んだモデルを、そのまま生成UIのワークフローにセットできます。
|
||||
|
||||
1. 生成UIに、ワークフローを読み込んでおきます（第5章 5-1）。
|
||||
2. モデルタブで、使いたいモデルを選択します。
|
||||
3. **「GenUI Model」** ボタンを押します。
|
||||
|
||||
これで、生成UIのワークフローのモデルが、選んだモデルに差し替わります。
|
||||
Civitaiのサンプル画像を見比べながら、気になったモデルをすぐ試せます。
|
||||
|
||||
> **ポイント**
|
||||
> Embeddingの場合は、**GenUI PP**（ポジティブプロンプトに追加）と **GenUI NP**（ネガティブプロンプトに追加）のボタンが表示されます。
|
||||
> ネガティブ用のEmbeddingは、GenUI NP で追加しましょう。
|
||||
|
||||
> **注意**
|
||||
> 生成UIのワークフローに、そのモデルを読み込むノードがない場合は使えません。
|
||||
> たとえば、LoRAのノードがないワークフローに、LoRAはセットできません。
|
||||
|
||||
### ComfyUIのノード画面で使う
|
||||
ComfyUIのノード画面の横にある **WS Library** の **M（Models）** タブからモデルをキャンバスへドラッグすると、
|
||||
そのモデルを読み込むノードが自動で置かれます（第9章）。
|
||||
|
||||
---
|
||||
|
||||
## 6-5 使わないモデルを片付ける
|
||||
|
||||
### 有効・無効の切り替え
|
||||
しばらく使わないモデルは、削除せずに **無効（Disable）** にしておけます。
|
||||
無効にしたモデルは、ComfyUIのモデル選択の一覧に表示されなくなるので、一覧がすっきりします。
|
||||
|
||||
- カードの **一時停止（⏸）** ボタンで、1つずつ有効・無効を切り替えられます。
|
||||
- グループ単位で、まとめて有効・無効にすることもできます。
|
||||
|
||||
> **ポイント**
|
||||
> 無効にすると、ファイルの名前の末尾に `.disabled` が付きます。ファイル自体は消えません。
|
||||
> 有効に戻すと、元の名前に戻ります。
|
||||
|
||||
### まとめて操作する（Select）
|
||||
緑色の **Select** ボタンを押すと、複数のモデルにチェックを入れて、まとめて操作できます。
|
||||
|
||||
- グループへの追加・削除
|
||||
- バッジの追加・削除
|
||||
- 別のサブフォルダへの移動（プレビュー画像などの関連ファイルも一緒に移動）
|
||||
- 削除
|
||||
|
||||
### 削除
|
||||
詳細画面やSelectから、モデルを削除できます。
|
||||
|
||||
> **注意**
|
||||
> 削除すると、モデルファイルと、関連するファイル（プレビュー画像、Civitai情報など）が **完全に削除** されます。
|
||||
> モデルは再ダウンロードに時間がかかるものが多いので、迷ったら削除ではなく「無効」にしておきましょう。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - モデルタブでは、ComfyUIのモデルを種類ごとに一覧できる。最初に「更新」を押す
|
||||
> - お気に入り・バッジ・グループで分類できる。グループはバッチ生成でも使える
|
||||
> - 「CivitAIから取得」で、Base Model・トリガーワード・サンプル画像などを自動で取り込める
|
||||
> - 「GenUI Model」で、選んだモデルをすぐ生成UIで試せる
|
||||
> - 使わないモデルは削除せず「無効」にしておくと安心
|
||||
@@ -0,0 +1,258 @@
|
||||
# 第7章 プロンプトタブ － 呪文を貯めて、育てる
|
||||
|
||||
うまくいったプロンプトは、あなただけの財産です。
|
||||
でも、メモ帳に貼り付けておくだけでは、どれが何の呪文だったか分からなくなりがちです。
|
||||
|
||||
**プロンプト（Prompt）** タブは、プロンプトを **保存・分類・再利用** するための場所です。
|
||||
さらに、ローカルAIを使った翻訳や、プロンプトを毎回少しずつ変える「ワイルドカード」も使えます。
|
||||
|
||||
---
|
||||
|
||||
## 7-1 プロンプトの基本
|
||||
|
||||
プロンプトタブの使い方に入る前に、プロンプトの書き方の基本を押さえておきましょう。
|
||||
|
||||
### ポジティブとネガティブ
|
||||
| 種類 | 書く内容 | 例 |
|
||||
| --- | --- | --- |
|
||||
| Positive Prompt（ポジティブ） | 描いてほしいもの | `1girl, smiling, school uniform, cherry blossoms, spring` |
|
||||
| Negative Prompt（ネガティブ） | 描いてほしくないもの | `low quality, blurry, extra fingers, watermark` |
|
||||
|
||||
### 書き方のコツ
|
||||
- **英語で書く** － 多くのモデルは英語で学習されています。
|
||||
- **カンマ（,）で区切る** － 単語や短いフレーズを、カンマで区切って並べるのが一般的です。
|
||||
- **大事なものを前に** － 前に書いた言葉ほど、強く反映されやすい傾向があります。
|
||||
- **強調したい言葉は (言葉:数値)** － `(red hair:1.3)` のように書くと、その言葉を強調できます（1.0が標準）。
|
||||
|
||||
> **ポイント：強調の数値をキーボードで変える**
|
||||
> 生成UIのプロンプト入力欄で、強調したい言葉を選んで **Ctrl＋↑／↓** を押すと、
|
||||
> `(言葉:数値)` の数値を0.05ずつ増減できます。数値が1.0に戻ると、カッコは自動で外れます。
|
||||
|
||||
> **ポイント**
|
||||
> SDXLのアニメ系モデルは「タグを並べる」書き方、FLUXなどの新しいモデルは「普通の英文で説明する」書き方が向いています。
|
||||
> モデルの配布ページに、おすすめの書き方が載っていることも多いので確認してみましょう。
|
||||
|
||||
---
|
||||
|
||||
## 7-2 プロンプトタブの画面
|
||||
|
||||
プロンプトタブは、左・中央・右の3つの列に分かれています。
|
||||
|
||||
| 列 | 名前 | 役割 |
|
||||
| --- | --- | --- |
|
||||
| 左 | AI Assistant | ローカルAIとのチャット・翻訳（7-5） |
|
||||
| 中央 | Presets／Preset Manager | プロンプトの保存・一覧・分類（7-3） |
|
||||
| 右 | Wildcard／Style | ワイルドカードとスタイルの管理（7-4） |
|
||||
|
||||
画面上部の **Form／Table** で、表示方法を切り替えられます。
|
||||
|
||||
- **Form** － 1件ずつ編集する画面（この章で説明する画面です）
|
||||
- **Table** － プリセットやワイルドカードを表形式でまとめて編集する画面（件数が増えてきたら便利）
|
||||
|
||||
---
|
||||
|
||||
## 7-3 プリセットとして保存・分類する
|
||||
|
||||
**プリセット** は、ポジティブ・ネガティブのプロンプトをひと組にして、名前を付けて保存したものです。
|
||||
|
||||
### プリセットを保存する
|
||||
1. 中央の列で **Presets** タブを開きます。
|
||||
2. 一番上のプルダウンが **New Preset (none selected)** になっていることを確認します。
|
||||
3. 次の項目を入力します。
|
||||
|
||||
| 項目 | 内容 |
|
||||
| --- | --- |
|
||||
| Preset Name | プリセットの名前（例：`桜と制服`） |
|
||||
| Category | 分類（例：`Portrait`、`Landscape`）。省略してもかまいません |
|
||||
| Positive Prompt | 描いてほしいもの |
|
||||
| Negative Prompt | 描いてほしくないもの |
|
||||
|
||||
4. **Save** を押します。「プリセットを保存しました」と表示されれば完了です。
|
||||
|
||||
> **ポイント**
|
||||
> 生成UIで試して「これは良い！」と思ったプロンプトを、その場でプリセットに保存する習慣をつけましょう。
|
||||
> あとから同じ雰囲気の画像を作りたいときに、すぐに呼び出せます。
|
||||
|
||||
### プリセットを呼び出して使う
|
||||
1. 一番上のプルダウンから、使いたいプリセットを選びます。
|
||||
2. 使い方に応じてボタンを押します。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| GenUI Set | 生成UIのプロンプト欄に、ポジティブとネガティブをまとめてセットする |
|
||||
| PP Copy | ポジティブプロンプトをコピーする |
|
||||
| NP Copy | ネガティブプロンプトをコピーする |
|
||||
|
||||
> **注意**
|
||||
> **GenUI Set** は、生成UIにワークフローが読み込まれているときに使えます。
|
||||
> 先に第5章の手順でワークフローを読み込んでおきましょう。
|
||||
|
||||
### グループで分類する
|
||||
プリセットも、グループで分類できます。
|
||||
|
||||
1. プリセットを選んだ状態で、**+ Group** を押して新しいグループを作ります。
|
||||
2. **Select group...** からグループを選び、**Add to Group** を押します。
|
||||
|
||||
グループは削除することもできます（**Del Group**）。
|
||||
|
||||
ここで作ったグループは、生成UIのバッチ生成（第5章 5-5）で「Prompt」のグループとして使えます。
|
||||
たとえば「季節シリーズ」というグループに春・夏・秋・冬のプリセットを入れておけば、
|
||||
4つの季節の絵をまとめて生成できます。
|
||||
|
||||
### Preset Manager で一覧する
|
||||
中央の列の **Preset Manager** タブでは、保存したプリセットを一覧で見られます。
|
||||
|
||||
- **All** － すべてのプリセット
|
||||
- **★** － お気に入りに登録したプリセット
|
||||
- **Groups** － グループごとの表示
|
||||
|
||||
上部の **Search presets...** 欄で、プリセットを検索できます。
|
||||
|
||||
保存したプリセットは、ComfyUIのノード画面の横にある **WS Library** の **P（Prompts）** タブにも表示され、
|
||||
キャンバスへドラッグして使えます（第9章）。
|
||||
|
||||
---
|
||||
|
||||
## 7-4 ワイルドカードとスタイル
|
||||
|
||||
右の列では、プロンプトを自動で変化させる **ワイルドカード** と、
|
||||
プロンプトに雰囲気を足す **スタイル** を管理します。
|
||||
列の上部の **Wildcard／Style** で切り替えます。
|
||||
|
||||
### ワイルドカードとは
|
||||
ワイルドカードは、**生成するたびに、プロンプトの一部をランダムに入れ替える** しくみです。
|
||||
同じプロンプトのまま、バリエーションのある画像をたくさん作れます。
|
||||
|
||||
Workflow Studioでいちばん手軽なのは、**ワイルドカードファイル** を使う方法です。
|
||||
候補を書いたファイルを作っておき、プロンプトに `__ファイル名__` と書くと、
|
||||
生成するたびにファイルの中から1行がランダムに選ばれます。
|
||||
|
||||
### ワイルドカードファイルを作る
|
||||
例として、色の候補を入れた `colors` というファイルを作ってみましょう。
|
||||
|
||||
1. 右の列で **Wildcard** を選び、**Wildcard Files** の **+ New** を押します。
|
||||
2. ファイル名（例：`colors`）を入力し、形式は **txt** を選びます。
|
||||
3. 候補を1行に1つずつ入力します。
|
||||
|
||||
```
|
||||
red
|
||||
blue
|
||||
green
|
||||
golden
|
||||
```
|
||||
|
||||
4. **Save** を押します。
|
||||
|
||||
### ワイルドカードを使う
|
||||
プロンプトに、ファイル名を `__`（アンダースコア2つ）で囲んで書きます。
|
||||
|
||||
```
|
||||
1girl, __colors__ dress, standing in a garden
|
||||
```
|
||||
|
||||
生成UIで生成するたびに、`__colors__` の部分が `red`・`blue`・`green`・`golden` のどれかに置き換わります。
|
||||
|
||||
> **ポイント**
|
||||
> `__colors__` を手で入力する代わりに、右の列の **File** ボタンでファイルを選んで挿入することもできます。
|
||||
|
||||
### もっと高度な書き方（Impact Pack 用）
|
||||
右の列の上部には、`{ }`、`$$`、`[ ]` などの記号を挿入するボタンもあります。
|
||||
これらは、次のような書き方のためのものです。
|
||||
|
||||
| 書き方 | 意味 |
|
||||
| --- | --- |
|
||||
| `{cat｜dog｜rabbit}` | 3つの中から1つを選ぶ |
|
||||
| `{2$$cat｜dog｜bird}` | 3つの中から2つを選ぶ |
|
||||
|
||||
※ 本書では見やすさのため `｜` と全角で表記しています。実際には半角の `|`（縦棒）を使います。
|
||||
|
||||
> **注意**
|
||||
> 生成UIが自動で展開するのは、`__ファイル名__` の書き方だけです。
|
||||
> `{A|B}` などの書き方を使うには、カスタムノード **ComfyUI-Impact-Pack** のワイルドカード用ノード
|
||||
> （ImpactWildcardProcessor など）をワークフローに入れておく必要があります。
|
||||
> 慣れないうちは、`__ファイル名__` の書き方だけ使えば十分です。
|
||||
|
||||
### 作ったプロンプトを送る
|
||||
右の列で組み立てたプロンプトは、下のボタンで送り出せます。
|
||||
|
||||
- **→ Pos** － プリセットのPositive Promptへ送る
|
||||
- **→ Neg** － プリセットのNegative Promptへ送る
|
||||
- **Copy** － コピーする
|
||||
|
||||
`__ファイル名__` の部分は、生成UIで生成するときに自動で展開されます。
|
||||
|
||||
### スタイル
|
||||
**スタイル** は、プロンプトに付け足す「雰囲気のテンプレート」です。
|
||||
たとえば「映画風」というスタイルには、`cinematic still, {prompt}, dramatic lighting` のような内容が登録されています。
|
||||
`{prompt}` の部分に、あなたのプロンプトが入ります。
|
||||
|
||||
スタイルを使うには、生成UIの上部のバーで、**Style** のチェックボックスをオンにして、プルダウンからスタイルを選びます。
|
||||
生成するときに、選んだスタイルがプロンプトに自動で合成されます（ワークフローそのものは書き換わりません）。
|
||||
|
||||
Workflow Studioにはたくさんのスタイルが付属しています。
|
||||
プロンプトタブの **Style** では、スタイルの一覧を見たり、**+ New** で自分のスタイルを作ったりできます。
|
||||
|
||||
> **ポイント**
|
||||
> どのスタイルがどんな雰囲気になるかは、生成UIの **Create Catalog** で、スタイルごとの見本画像をまとめて作ると分かりやすくなります。
|
||||
> 作った見本は、ギャラリータブの Style_Catalog で一覧できます（第8章）。
|
||||
|
||||
---
|
||||
|
||||
## 7-5 AIで翻訳・プロンプト作りを手伝ってもらう
|
||||
|
||||
左の列の **AI Assistant** では、パソコンで動くAI（ローカルAI）に、プロンプト作りを手伝ってもらえます。
|
||||
|
||||
> **事前に必要なもの**
|
||||
> この機能を使うには、**Ollama** や **LM Studio** などのローカルAIソフトと、AIモデルの準備が必要です。
|
||||
> 準備の方法は第10章で説明します。準備ができていない場合、AI Assistantの上部に **Disconnected** と表示されます。
|
||||
|
||||
### 日本語を英語に翻訳する
|
||||
英語のプロンプトが苦手でも大丈夫です。
|
||||
|
||||
1. 左の列の下にある入力欄に、日本語で描きたい内容を書きます。
|
||||
|
||||
```
|
||||
桜の木の下で微笑む制服姿の少女、春の柔らかい光
|
||||
```
|
||||
|
||||
2. **Translate:** の **JA→EN** ボタンを押します。
|
||||
3. チャット欄に、英語に翻訳された結果が表示されます。
|
||||
|
||||
翻訳結果の下に表示されるボタンで、結果を使えます。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| 生成UIに適用 | 生成UIのポジティブプロンプト欄に入れる |
|
||||
| コピー | コピーする（プリセットに貼り付けるときなど） |
|
||||
|
||||
英語のプロンプトの意味を知りたいときは、反対に **EN→JA** で日本語に訳せます。
|
||||
|
||||
> **ポイント**
|
||||
> 生成UIのプロンプト欄で直接翻訳したいときは、第5章 5-10 の **プロンプト翻訳**（🌐ボタン）が便利です。重み付けやLoRAの指定を残したまま、日本語の部分だけを英語にできます。
|
||||
> AI Assistantの翻訳にも、翻訳専用のモデル **PLaMo 3 Translate** を選べます（v0.9.9〜。以前は指示文まで翻訳されたり、ZH→ENで日本語が出たりしていました）。
|
||||
|
||||
### AIと相談しながらプロンプトを作る
|
||||
入力欄に質問を書いて **Send**（または Enter キー）を押すと、AIとチャットできます。
|
||||
|
||||
- 「夕暮れの海辺の風景のプロンプトを、英語のタグ形式で作って」
|
||||
- 「このプロンプトに、もっと幻想的な雰囲気を足して」
|
||||
|
||||
のように頼むと、プロンプトの案を出してくれます。
|
||||
画像を読み取れるAIモデルを使っている場合は、**Attach Image** で画像を添付して、
|
||||
「この画像の雰囲気をプロンプトにして」と頼むこともできます。
|
||||
|
||||
- **Shift＋Enter** － 改行
|
||||
- **Clear Chat** － 会話をリセット
|
||||
- **歯車ボタン** － 使うAIソフトやモデルの設定（第10章）
|
||||
|
||||
> **注意**
|
||||
> ローカルAIの答えは、いつも正しいとは限りません。
|
||||
> 出てきたプロンプトは、実際に生成して確かめながら調整しましょう。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - プロンプトは英語・カンマ区切りが基本。強調は `(言葉:数値)`
|
||||
> - うまくいったプロンプトは、プリセットとして名前を付けて保存する
|
||||
> - 「GenUI Set」で、プリセットをすぐ生成UIにセットできる
|
||||
> - グループに分けると、バッチ生成でまとめて試せる
|
||||
> - ワイルドカードファイルを作って `__ファイル名__` と書くと、生成のたびにプロンプトを変化させられる
|
||||
> - ローカルAIを用意すれば、日本語からの翻訳やプロンプトの相談ができる
|
||||
@@ -0,0 +1,332 @@
|
||||
# 第8章 ギャラリータブ － 生成した画像を見返す
|
||||
|
||||
画像生成を続けていると、ComfyUIの `output` フォルダには何百、何千枚もの画像がたまっていきます。
|
||||
**ギャラリー（Gallery）** タブは、それらの画像を快適に見返し、
|
||||
「この画像、どうやって作ったんだっけ？」を解決するための場所です。
|
||||
|
||||
ComfyUIで生成した画像には、生成に使ったワークフローやプロンプトの情報（**メタデータ**）が埋め込まれています。
|
||||
ギャラリータブでは、この情報を読み取って、同じ設定をすぐに再現できます。
|
||||
また、動画（MP4・WebM）や音声ファイル（WAV・MP3・OGG・M4A・FLAC・Opus）、3Dモデル（GLB・glTF・VRM・PLY・SPZ）も、画像と同じ一覧で扱えます。
|
||||
|
||||
---
|
||||
|
||||
## 8-1 画像を一覧する
|
||||
|
||||
ギャラリータブの上部には、4つのサブタブがあります。
|
||||
|
||||
| サブタブ | 役割 | 本書 |
|
||||
| --- | --- | --- |
|
||||
| Output | 生成した画像の一覧（ふだん使うのはここ） | 8-1〜8-3 |
|
||||
| ImagePrompt | 見本画像を見ながらプロンプトを組み立てる | 8-4 |
|
||||
| Style_Catalog | スタイルの見本画像から選ぶ | 8-4 |
|
||||
| メタデータ | 画像ファイルをドロップして、中の情報を調べる | 8-2 |
|
||||
|
||||
### Outputの画面
|
||||
**Output** サブタブは、左・中央・右の3つの部分に分かれています。
|
||||
|
||||
| 場所 | 役割 |
|
||||
| --- | --- |
|
||||
| 左（フォルダ） | `output` フォルダの中のフォルダ一覧。いちばん上の **[root]** が `output` フォルダ全体 |
|
||||
| 中央（一覧） | 選んだフォルダの画像。下へスクロールすると続きが読み込まれる |
|
||||
| 右（詳細パネル） | 選んだ画像の情報（8-2） |
|
||||
|
||||
まず、左のフォルダ一覧からフォルダを選ぶと、中央に画像が表示されます。
|
||||
|
||||
> **ポイント**
|
||||
> 生成UIやComfyUIで新しく生成した画像が出てこないときは、ツールバーの **Refresh** を押してください。
|
||||
|
||||
> **ポイント**
|
||||
> 読み込む `output` フォルダの場所は、設定タブの **「ギャラリー Outputフォルダ」** で変更できます。
|
||||
|
||||
### 表示の切り替え・検索・並べ替え
|
||||
ほかのタブと同じく、次の操作ができます。
|
||||
|
||||
- **サムネイル／テーブル** － サムネイル表示とテーブル表示の切り替え
|
||||
- **検索欄** － ファイル名・タグ・メモ・**プロンプトの内容** で検索。横の **S** ボタンをオンにすると、文章の意味で探せます（8-6）
|
||||
- **並べ替え** － Date (Newest)／Date (Oldest)／Name A-Z／Name Z-A
|
||||
- **★** － お気に入りだけ表示
|
||||
- **× クリア** － 絞り込みをまとめて解除
|
||||
|
||||
> **ポイント**
|
||||
> 検索欄では、プロンプトの言葉でも探せます。
|
||||
> たとえば「cat」で検索すると、プロンプトに cat を含む画像が見つかります。
|
||||
> （プロンプトの検索は、一度詳細パネルで開いた画像が対象になります）
|
||||
|
||||
### 大きく見る・比べる
|
||||
- 画像をクリックすると、右の詳細パネルに大きく表示されます。
|
||||
- **Ctrl キーを押しながら** 画像をクリックすると、複数の画像を選べます。
|
||||
2〜4枚選んで **比較** を押すと、並べて見比べられます。
|
||||
|
||||
シードやモデルだけを変えて作った画像を並べると、違いがよく分かります。
|
||||
|
||||
### 整理する
|
||||
- **お気に入り（★）・タグ・メモ・グループ** で分類できます（詳細パネルで設定）。
|
||||
- 複数選択して、まとめてグループに追加したり、別のフォルダへ移動したりできます。
|
||||
移動先を選ぶ画面（**移動先...**）には絞り込み欄があり、フォルダ名の一部を入れると候補が絞られ、Enterで先頭の候補へ移動できます。
|
||||
- **エクスポート** を押すと、選んだ画像をZIPファイルにまとめてダウンロードできます。
|
||||
|
||||
> **注意**
|
||||
> 詳細パネルの **Delete** や、複数選択の **ファイル削除** を使うと、画像ファイルそのものが削除されます。
|
||||
|
||||
---
|
||||
|
||||
## 8-2 メタデータから生成設定を読み取る
|
||||
|
||||
画像をクリックすると、右の詳細パネルに4つのタブが表示されます。
|
||||
|
||||
| タブ | 内容 |
|
||||
| --- | --- |
|
||||
| プロンプト | 画像の生成に使われたプロンプト（最初に表示されるタブ） |
|
||||
| 基本情報 | ファイル名・大きさ・日時・タグ・メモ |
|
||||
| メタデータ | 画像に埋め込まれたワークフロー（JSON） |
|
||||
| グループ | グループの設定 |
|
||||
|
||||
### プロンプトを確認する
|
||||
**プロンプト** タブには、画像に埋め込まれていたプロンプトが一覧表示されます。
|
||||
ポジティブには **POS**、ネガティブには **NEG** の印が付きます。
|
||||
項目をクリックすると、全文が表示されます。
|
||||
|
||||
### もっと詳しく調べる（メタデータサブタブ）
|
||||
使ったモデルやLoRAまで詳しく知りたいときは、詳細パネルの **Metadata** ボタンを押します
|
||||
（または、Alt キーを押しながら画像をクリック）。
|
||||
ギャラリーの **メタデータ** サブタブが開き、次の情報が整理されて表示されます。
|
||||
|
||||
- **モデル** － Checkpoint、VAE、Diffusion Model、Text Encoder（中央の列）
|
||||
- **LoRA** － 使われたLoRAと、その強さ（中央の列）
|
||||
- **プロンプト** － ポジティブ・ネガティブ（中央の列）
|
||||
- **Settings** － 画像サイズ（幅×高さ）と、KSamplerの Seed・Steps・CFG・Sampler・Scheduler・Denoise（右の列。v0.7.14〜）
|
||||
- **Nanobanana** － Comic Creatorのnanobananaタブで作った画像の、エンジン・モデル・プロンプトなどの生成履歴（右の列。該当する画像のみ）
|
||||
|
||||
プロンプトの横のボタンで、内容をほかのタブに送れます。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| Copy | コピー |
|
||||
| GenUI:P／GenUI:N | 生成UIのポジティブ／ネガティブに入れる |
|
||||
| Prompt:P／Prompt:N | プロンプトタブのプリセットのポジティブ／ネガティブに入れる |
|
||||
|
||||
> **ポイント：ネットで見つけた画像も調べられる**
|
||||
> メタデータサブタブの左側に、画像ファイルを **ドロップ** すると、その画像の情報を調べられます。
|
||||
> ComfyUIの画像だけでなく、SD WebUI（AUTOMATIC1111）、SD Forge、Fooocus で作られたPNG画像にも対応しています。
|
||||
> 配布サイトで見つけた画像の作り方を知りたいときに便利です
|
||||
> （ただし、投稿時にメタデータが消されている画像からは読み取れません）。
|
||||
|
||||
---
|
||||
|
||||
## 8-3 同じ設定を再現する
|
||||
|
||||
ギャラリーのいちばん便利な使い方は、**気に入った画像と同じ設定で、もう一度生成する** ことです。
|
||||
方法は2つあります。
|
||||
|
||||
### 方法1：生成UIで再現する（GenUI）
|
||||
1. ギャラリーで、再現したい画像をクリックします。
|
||||
2. 詳細パネルの **GenUI** ボタンを押します。
|
||||
3. 画像に埋め込まれていたワークフローが、生成UIに読み込まれます。
|
||||
|
||||
あとは第5章と同じように、プロンプトを少し変えたり、シードを変えたりして生成できます。
|
||||
|
||||
> **ポイント**
|
||||
> **同じ画像をもう一度作りたいとき（v0.8.3〜）**
|
||||
> 画像に埋め込まれているワークフローには、そのときのシード値も入っています。
|
||||
> 生成パネルのシードのモードを **Workflow** にして Generate を押すと、そのシードのまま生成されます。
|
||||
>
|
||||
> モードが Random のままだと、別のシードで生成されます。
|
||||
> また、Fixed を使う場合は、**Settings** サブタブの **Seed** 欄の値を生成パネルの **Seed** 欄に貼り付けてください（生成パネルの値が優先されるため）。
|
||||
|
||||
### 方法2：ComfyUIのノード画面で再現する（コピー＆キャンバスへ送る）
|
||||
1. ギャラリーで、再現したい画像をクリックします。
|
||||
2. 詳細パネルの **JSON** タブを開きます。
|
||||
3. **Copy & Send Canvas**（コピー＆キャンバスへ送る）を押します。
|
||||
（このボタンは英語表示のままです。ワークフローが埋め込まれた画像でだけ押せます）
|
||||
|
||||
ComfyUIのノード画面に、その画像を作ったワークフローが読み込まれます。
|
||||
ノード画面で細かく手を加えたいときは、こちらを使います。
|
||||
|
||||
> **注意**
|
||||
> 「この画像にはComfyUIワークフローが埋め込まれていません」と表示される場合は、再現できません。
|
||||
> ワークフローの情報が保存されない設定で作った画像や、別のソフトで編集・圧縮した画像などが当てはまります。
|
||||
|
||||
### 画像を次の生成の材料にする
|
||||
ギャラリーの画像を、生成の「入力画像」として使うこともできます。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| Send GenUI Image | 選んだ画像を、生成UIのワークフローの入力画像（LoadImageノード）にセットする |
|
||||
| Send to LI node | 選んだ画像を、ComfyUIのノード画面で選んでいる画像読み込みノードにセットする |
|
||||
| Image Edit | 選んだ画像を Image Edit タブで開く（第11章） |
|
||||
| Send CC Layout | 選んだ画像を Comic Creator のレイアウトタブ（選択中のコマ／オーバーレイ）に送る（Comic Creatorの中でギャラリーを開いたときだけ表示。第11章 11-6） |
|
||||
| Send CC Image | 選んだ画像を Comic Creator の Image タブに送る（作業中のレイヤーがあれば新規レイヤー、なければ新規キャンバス。動画は未対応。表示条件は Send CC Layout と同じ） |
|
||||
|
||||
I2I（画像から画像を作る）ワークフローで、生成した画像をさらに描き直していくときに便利です。
|
||||
|
||||
---
|
||||
|
||||
## 8-4 ImagePrompt と Style_Catalog
|
||||
|
||||
### ImagePrompt － 見本画像からプロンプトを組み立てる
|
||||
**ImagePrompt** は、「見本画像」と「プロンプト」をセットで並べて、
|
||||
見た目で選びながらプロンプトを組み立てるためのサブタブです。
|
||||
|
||||
1. 見本画像をクリックすると、右側にその画像のプロンプトが表示されます。
|
||||
2. **+ Add** を押すと、**Selected Prompts** に追加されます。
|
||||
3. いくつか追加すると、**Final Prompt** に、カンマ区切りでまとめたプロンプトができあがります。
|
||||
4. **Copy** でコピーして、生成UIやプリセットに貼り付けます。
|
||||
|
||||
見本画像は、`output` フォルダの中に自動で作られる `ws_image_prompt` フォルダに置きます。
|
||||
画像と同じ名前の `.txt` ファイル（中身がプロンプト）を一緒に置くと、見本として使えるようになります。
|
||||
|
||||
> **例**
|
||||
> `ws_image_prompt/髪型/ponytail.png` と `ws_image_prompt/髪型/ponytail.txt`（中身：`ponytail, long hair`）
|
||||
|
||||
### Style_Catalog － スタイルを見本画像で選ぶ
|
||||
第7章で紹介したスタイルは、名前だけではどんな雰囲気になるか分かりにくいものです。
|
||||
**Style_Catalog** では、スタイルごとの見本画像を並べて、見た目で選べます。
|
||||
|
||||
見本画像は、生成UIの上部のバーにある **Create Catalog** で作ります。
|
||||
|
||||
1. 生成UIに、文章から画像を作るワークフローを読み込みます。
|
||||
2. **Batch** サブタブの **Style** で、見本を作りたいスタイルにチェックを入れます。
|
||||
3. 上部のバーの **Create Catalog** を押し、保存先のフォルダを決めて実行します。
|
||||
4. 上部のバーの **Catalog** を押すと、ギャラリーの Style_Catalog に移動します。
|
||||
|
||||
Style_Catalog で見本画像を選び、**Select as Style** を押すと、
|
||||
生成UIのスタイルが、そのスタイルに切り替わります。
|
||||
|
||||
> **ポイント**
|
||||
> モデルによってスタイルの効き方は変わります。
|
||||
> 保存先のフォルダをモデルごとに分けておくと、比べやすくなります。
|
||||
|
||||
---
|
||||
|
||||
## 8-5 Eagle連携・PSD書き出し（任意）
|
||||
|
||||
ここからは、必要な方だけが使う機能です。
|
||||
|
||||
### Eagle連携
|
||||
**Eagle** は、画像を整理するための有料ソフトです。
|
||||
設定タブの **Eagle連携** を有効にすると、生成UIで生成した画像を、プロンプトなどの情報付きで自動的にEagleに保存できます。
|
||||
Eagleで画像を管理している方は、試してみてください。
|
||||
|
||||
### PSD書き出し
|
||||
ギャラリーで複数の画像を選び、**PSD作成** を押すと、
|
||||
それぞれの画像をレイヤーとして重ねたPSD（Photoshop形式）のファイルを作れます。
|
||||
差分画像をまとめて、画像編集ソフトで仕上げたいときに便利です。
|
||||
|
||||
> **注意**
|
||||
> PSD関連の機能を使うには、追加のPythonパッケージ（psd-tools）が必要です。
|
||||
> 入っていない場合は、インストール方法がエラーメッセージで表示されます（第11章）。
|
||||
|
||||
---
|
||||
|
||||
## 8-6 画像を「意味」で探す（セマンティック検索）
|
||||
|
||||
通常の検索は、ファイル名・タグ・プロンプトの文字が一致した画像を探します。
|
||||
**セマンティック検索**（v0.8.2〜）は、画像の「意味」で探します。
|
||||
たとえば「夕暮れの街並み」と入れると、プロンプトにその言葉が無くても、雰囲気の合う画像が見つかります。
|
||||
また、選んだ画像に **似た画像** の一覧を出すこともできます。
|
||||
|
||||
### 使い方
|
||||
1. 検索欄の横の **S** ボタンをオンにします。検索欄が意味検索に切り替わり、ツールバーに2行目が現れます。
|
||||
2. 2行目の **歯車ボタン**（設定）を押し、**インデックス**（検索用の索引）を作ります（下記）。
|
||||
3. 検索欄に文章を入れて **Enter** を押します。結果はスコア順に並び、サムネイルにスコアが表示されます。
|
||||
4. 画像を選んで2行目の **Similar** を押すと、その画像に似た画像が並びます。別の画像を選んで **↻（更新）** を押すと、その画像の類似を出し直します。もう一度 **Similar** を押すと、通常の表示に戻ります。
|
||||
|
||||
> **ポイント**
|
||||
> 意味検索の結果には、並べ替えやタグ・グループの絞り込みは効きません。
|
||||
> 検索の範囲は、選んでいるフォルダの中（サブフォルダを含む）です。
|
||||
> フォルダツリーの枚数の横に緑の点があるフォルダは、直下の画像がすべてインデックス済みです。
|
||||
|
||||
### モデルを選ぶ（歯車ボタンの「モデル設定」タブ）
|
||||
| モデル | 特徴 | 準備 |
|
||||
| --- | --- | --- |
|
||||
| Gemini Embedding 2 | Googleのサービスで計算する。速い。画像をGoogleに送る | APIキーを取得し、プラグインの `.env` に `GEMINI_API_KEY=キー` と書いて、ComfyUIを再起動する |
|
||||
| EmbeddingGemma 2 | パソコンの中で、**Ollama** を使って計算する（ローカル・無料）。外部に送らない | Ollamaをインストールして起動し、`ollama pull embeddinggemma-2:740m`（または軽量版の `embeddinggemma-2:440m`）を実行する。モデル設定タブで **Ollama URL**（通常は変更不要）と **モデルサイズ** を選ぶ |
|
||||
|
||||
> **注意：Geminiを使うとき**
|
||||
> 画像がGoogleのサービスに送られます。無料枠で送った内容は、Googleの製品改善に使われる場合があります。
|
||||
> 公開したくない画像が多い場合は、ローカルの EmbeddingGemma 2 を選んでください。
|
||||
> APIキーは `.env` にだけ書き、他人に見せないようにしましょう。
|
||||
|
||||
### インデックスを作る（歯車ボタンの「インデックス」タブ）
|
||||
1. 対象のフォルダを選びます（サブフォルダを含めるかも選べます）。
|
||||
2. 「新規のみ」か「全件再作成」を選んで開始します。進み具合が表示され、途中で停止もできます。
|
||||
3. インデックスは全フォルダで共通の1つです。フォルダごとに少しずつ作っても構いません。
|
||||
|
||||
モデルやベクトル次元を変えたときは、**全件再作成** が必要です。
|
||||
|
||||
### 速さの目安と設定
|
||||
- ローカルのEmbeddingGemma 2は、v0.8.14からOllamaで動くようになりました（それまでの、ComfyUIのPythonで動かす方式は廃止）。
|
||||
NVIDIAのGPUなら1枚あたり約0.1〜0.2秒です。インデックスを作り終えると、モデルはGPUメモリから自動で降ろされます。
|
||||
- インデックスを作るときと、検索するときは、Ollamaが起動している必要があります。
|
||||
- モデルサイズの740m（テキスト＋画像＋音声）と440m（テキスト＋画像・軽量）は同じ結果を返すので、切り替えても全件再作成は不要です。
|
||||
v0.8.13以前に作ったインデックスも、そのまま使えます。
|
||||
- **検索設定** タブで、類似画像・文章検索それぞれの「最小スコア」と最大表示件数を決められます。
|
||||
スコアの水準はモデルによって違うため、モデルごとに保存されます。
|
||||
|
||||
---
|
||||
|
||||
## 8-7 音声ファイルの扱いと変換
|
||||
|
||||
ComfyUIで音楽や効果音を作る機会が増えたため、ギャラリーは音声ファイルも扱えます（v0.8.0〜）。
|
||||
対応しているのは、WAV・MP3・OGG・M4A・FLAC・Opus です。
|
||||
|
||||
- 一覧には、音の形（波形）のサムネイルと、♪ のバッジが表示されます。波形の色は形式ごとに違います（MP3は青、WAVは緑、FLACは琥珀色、OGG／Opusは紫、M4Aはピンク）。
|
||||
- 選ぶと、右の詳細パネルに波形とプレイヤーが表示され、その場で再生できます。
|
||||
- 保存時にワークフロー（`prompt`／`workflow`）が埋め込まれている音声は、プロンプトやJSONのタブも使えます。
|
||||
- 動画（MP4・WebM）も同様に一覧できます。**Send to LV node** ボタンで、選んだ動画を、ComfyUIの画面で選択中の **Load Video** ノードに送れます（画像版の Send to LI node の動画版です）。
|
||||
|
||||
### ファイルを変換する
|
||||
音声または動画を含めて複数選択すると、一括操作バーに **変換...** が現れます。
|
||||
|
||||
1. 変換したいファイルを選び、**変換...** を押します。
|
||||
2. 出力形式（FLAC／MP3／Opus／M4A／WAV）と、必要ならビットレート・サンプルレート・チャンネル数を決めます。
|
||||
3. 実行すると、同じフォルダに別のファイルとして保存されます（同名があるときは `名前_converted` のように連番が付き、上書きはされません）。
|
||||
|
||||
動画を選んだときは、音声トラックだけを取り出して保存します。
|
||||
|
||||
| 引き継ぎ | 説明 |
|
||||
| --- | --- |
|
||||
| ワークフロー情報 | MP3・FLAC・Opus・M4A には引き継がれる。WAVには引き継げない（通知が出る） |
|
||||
| お気に入り・タグ・メモ・グループ | 引き継ぐかを選べる |
|
||||
| 元ファイルの削除 | 変換に成功したあとで削除するかを選べる（確認画面あり） |
|
||||
|
||||
> **ポイント**
|
||||
> OGG は再生はできますが、変換の出力形式には選べません。
|
||||
> ほかの動画編集ソフトを入れなくても、Workflow Studioだけで変換できます。
|
||||
|
||||
> **ポイント：Videoタブでの活用**
|
||||
> Video の Asset サブタブでは、音声ファイルを「BGMに設定」したり、「サウンドとして追加」したりできます（第11章 11-4）。
|
||||
|
||||
---
|
||||
|
||||
## 8-8 3Dモデルを見る（v0.9.4〜）
|
||||
|
||||
ComfyUIで3Dモデルを作るワークフロー（画像から3Dなど）の結果も、ギャラリーで確かめられます。
|
||||
対応しているのは、GLB・glTF・VRM・PLY（メッシュ／点群／ガウシアンスプラット）・SPZ です。
|
||||
|
||||
- 一覧には「**3D**」のバッジ付きで並びます。サムネイルは、最初に一覧に出たときにブラウザが描いてサーバーに保存します
|
||||
（128MBを超えるファイルは、開いたときに保存します）。
|
||||
- 選ぶと、右の詳細パネルの内蔵ビューアで立体として表示されます。ダブルクリックで大きく表示できます。
|
||||
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| ドラッグ | 回転する |
|
||||
| 右ドラッグ | 位置を動かす |
|
||||
| マウスホイール | 拡大・縮小 |
|
||||
| ⟲（リセット）ボタン | 最初の向きに戻す |
|
||||
|
||||
- ComfyUIの **SaveGLB** で保存したモデルは、画像と同じようにワークフローを読み取れます（8-2・8-3）。
|
||||
|
||||
> **注意**
|
||||
> - 3Dファイルは、Image Edit・生成UI・Tagger・Feederには送れません。
|
||||
> - `.gltf` の中には、別の `.bin` や画像ファイルを参照しているものがあります。ギャラリーで移動・削除しても動くのは `.gltf` 本体だけなので、参照しているファイルの場所は変えないでください。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ギャラリーのOutputで、生成した画像をフォルダごとに見返せる。プロンプトの言葉でも検索できる
|
||||
> - 詳細パネルのプロンプトタブで、画像の生成に使ったプロンプトがすぐ分かる
|
||||
> - 「GenUI」で生成UIに、「コピー＆キャンバスへ送る」でノード画面に、同じワークフローを再現できる
|
||||
> - 再現するときは、シードのモードを Workflow（または Fixed）にする
|
||||
> - メタデータサブタブに画像をドロップすると、モデル・LoRA・プロンプトを詳しく調べられる
|
||||
> - Sボタンで、文章の意味や「似た画像」から探せる（セマンティック検索）
|
||||
> - 音声ファイルも一覧・再生・変換できる
|
||||
> - 3Dモデル（GLB・glTF・VRM・PLY・SPZ）も、ギャラリーで回して確かめられる（8-8）
|
||||
@@ -0,0 +1,171 @@
|
||||
# 第9章 WS Library － ComfyUI画面の中で使う
|
||||
|
||||
ここまでの章では、Workflow Studio本体（タブが並ぶ画面）を中心に使ってきました。
|
||||
この章では、ComfyUIのノード画面の横に開く **WS Library（Workflow Studio Library）** を使います。
|
||||
|
||||
WS Libraryを使うと、Workflow Studioで整理したワークフロー・ノード・モデル・プロンプトを、
|
||||
**ノード画面にドラッグ＆ドロップするだけ** で使えます。
|
||||
ノード画面で作業する時間が長い方ほど、便利さを実感できる機能です。
|
||||
|
||||
---
|
||||
|
||||
## 9-1 WS Libraryを開く
|
||||
|
||||
1. ComfyUIのノード画面を開きます。
|
||||
2. 上部メニューに追加された **WS Library** ボタン（四角が並んだアイコン）を押します。
|
||||
|
||||
画面の端にパネルが開きます。もう一度ボタンを押すと閉じます。
|
||||
|
||||
### 6つのタブ
|
||||
パネルの上部には、1文字のタブが並んでいます。マウスを重ねると、正式な名前が表示されます。
|
||||
|
||||
| タブ | 名前 | できること | 本書 |
|
||||
| --- | --- | --- | --- |
|
||||
| W | Workflows | ワークフローを読み込む | 9-2 |
|
||||
| N | Nodes | ノードやノードセットを置く | 9-2 |
|
||||
| M | Models | モデルを読み込むノードを置く | 9-2 |
|
||||
| P | Prompts | 保存したプロンプトを置く・コピーする | 9-2 |
|
||||
| I | Information | 画像やワークフローの中身を調べて、使われた素材を置く | 9-3 |
|
||||
| A | AI TOOL | 翻訳やチャットなどのAI機能を使う | 第10章 |
|
||||
|
||||
各タブの中には、さらに小さなタブ（サブタブ）があります。
|
||||
多くのタブで、次のサブタブが共通して使えます。
|
||||
|
||||
- **All** － すべて
|
||||
- **★ Favorites** － Workflow Studio本体でお気に入りに登録したもの
|
||||
- **Groups** － Workflow Studio本体で作ったグループごと
|
||||
|
||||
また、各サブタブの検索欄で、名前を入力して絞り込めます。
|
||||
|
||||
> **ポイント**
|
||||
> WS Libraryに表示される内容は、Workflow Studio本体での整理（お気に入り・グループ・バッジ）と連動しています。
|
||||
> 本体でしっかり整理しておくほど、WS Libraryも使いやすくなります。
|
||||
|
||||
---
|
||||
|
||||
## 9-2 ドラッグ＆ドロップでキャンバスに置く
|
||||
|
||||
WS Libraryの基本操作は、とてもシンプルです。
|
||||
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| キャンバスへドラッグ＆ドロップ | ドロップした位置に置く・読み込む |
|
||||
| ダブルクリック | 画面の中央付近に置く・読み込む |
|
||||
|
||||
### W － ワークフローを読み込む
|
||||
ワークフローをキャンバスへドラッグ（またはダブルクリック）すると、そのワークフローが読み込まれます。
|
||||
エクスプローラーからワークフローのファイルをドロップしたときと同じ動きです。
|
||||
|
||||
サブタブ：All／★ Favorites／Groups／By Badge（バッジごと）
|
||||
|
||||
> **ポイント**
|
||||
> Workflow Studio本体をブックマークから開いていて「キャンバスへ送る」が使えない場合は、
|
||||
> このWタブに対象のワークフローが表示されるので、ここからドラッグしてください（第4章 4-6）。
|
||||
|
||||
### M － モデルを置く
|
||||
モデルをキャンバスへドラッグすると、**そのモデルが選ばれた状態の読み込みノード** が置かれます。
|
||||
|
||||
| モデルの種類 | 置かれるノード |
|
||||
| --- | --- |
|
||||
| Checkpoint | Load Checkpoint（CheckpointLoaderSimple） |
|
||||
| LoRA | Load LoRA（LoraLoader） |
|
||||
| VAE | Load VAE（VAELoader） |
|
||||
| ControlNet | Load ControlNet Model（ControlNetLoader） |
|
||||
| UNET | Load Diffusion Model（UNETLoader） |
|
||||
| TextEncoder | Load CLIP（CLIPLoader） |
|
||||
| Hypernetwork | HypernetworkLoader |
|
||||
|
||||
ノードを一覧から探して、置いて、ファイル名をプルダウンから選んで……という手間がなくなります。
|
||||
置いたあとは、ほかのノードと線でつないで使います。
|
||||
|
||||
サブタブ：All／★ Favorites／Groups／By Type（種類ごと）
|
||||
|
||||
> **ポイント**
|
||||
> Embeddingには読み込み用のノードがないため、キャンバスに置くことはできません。
|
||||
|
||||
> **ポイント：LoRAをまとめて置く**
|
||||
> LoRAのグループには **All N LoRAs**（Nはグループ内のLoRAの数）という項目が表示されます。
|
||||
> これをドラッグすると、グループ内のLoRAがすべて入った状態の `Lora Loader (LoraManager)` ノードが置かれます。
|
||||
> （カスタムノード **ComfyUI-Lora-Manager** が必要です）
|
||||
|
||||
### P － プロンプトを置く
|
||||
保存したプリセット（第7章）をキャンバスへドラッグすると、
|
||||
ポジティブ・ネガティブのプロンプトが入った **Prompt Text (WFS)** ノードが置かれます。
|
||||
これはWorkflow Studioに付属しているノードで、プロンプトの文章を出力します。
|
||||
|
||||
プロンプトを貼り付けたいだけのときは、項目の **P**（ポジティブ）／**N**（ネガティブ）ボタンでコピーできます。
|
||||
|
||||
サブタブ：All／★ Favorites／Categories（カテゴリーごと）／Groups
|
||||
|
||||
### N － ノードを置く
|
||||
ノードをキャンバスへドラッグすると、そのノードが置かれます。
|
||||
よく使うノードをWorkflow Studio本体の **ノード** タブでお気に入りに登録しておくと、すぐに取り出せます。
|
||||
Nタブの一覧で、ノードを **Alt＋クリック** すると、その場でお気に入りのオン／オフを切り替えられます（v0.8.0〜）。
|
||||
Nタブは、ノードの詳細情報を待たずに一覧を先に表示するため、以前より速く開きます。
|
||||
|
||||
サブタブ：All／★ Favorites／Groups／Sets／Category（カテゴリーごと）／Package（カスタムノードのパッケージごと）
|
||||
|
||||
### ノードセット － よく使う組み合わせを丸ごと保存する
|
||||
**ノードセット** は、複数のノードとそのつながりを、ひとまとまりで保存したものです。
|
||||
「いつも同じ組み合わせで使うノード群」を保存しておけば、ドラッグ1回で丸ごと置けます。
|
||||
|
||||
**ノードセットを作る**
|
||||
1. ComfyUIのノード画面で、保存したいノードを選択します（Ctrlキーを押しながらクリック、またはドラッグで範囲選択）。
|
||||
2. 選んだノードの上で **右クリック** し、**Save as Node Set** を選びます。
|
||||
3. 名前を付けて保存します。
|
||||
|
||||
**ノードセットを使う**
|
||||
WS Libraryの **N** タブの **Sets** サブタブから、キャンバスへドラッグします。
|
||||
|
||||
> **例**
|
||||
> 「顔を自動で補正するノード群」や「高解像度化（アップスケール）のノード群」など、
|
||||
> どのワークフローにも後から付け足したい部品をセットにしておくと便利です。
|
||||
|
||||
---
|
||||
|
||||
## 9-3 Informationタブで画像の中身を調べる
|
||||
|
||||
**I（Information）** タブは、画像やワークフローのファイルを調べて、
|
||||
**使われていたモデルやプロンプトを、そのままキャンバスに置ける** 機能です。
|
||||
ギャラリーのメタデータ（第8章 8-2）を、ノード画面の中で使えるようにしたものと考えてください。
|
||||
|
||||
### 使い方
|
||||
1. WS Libraryの **I** タブを開きます。
|
||||
2. 調べたいファイル（ComfyUIで作ったPNG・WebP・MP4・WebMの画像や動画、またはワークフローのJSON）を、パネルにドロップします。
|
||||
3. ファイルの中から見つかった素材が、サブタブに分かれて表示されます。
|
||||
|
||||
| サブタブ | 内容 | キャンバスへドラッグすると |
|
||||
| --- | --- | --- |
|
||||
| model | Checkpoint・VAE・Diffusion Model・Text Encoder | 対応する読み込みノードが置かれる |
|
||||
| lora | 使われていたLoRAと、その強さ | LoRAの読み込みノードが置かれる |
|
||||
| Prompts | ポジティブ（POS）・ネガティブ（NEG）のプロンプト | プロンプトの入ったテキストエンコードノードが置かれる |
|
||||
|
||||
プロンプトの項目をクリックすると全文が表示され、**Copy** でコピーもできます。
|
||||
|
||||
### こんなときに便利
|
||||
- 「前に作ったあの画像と、同じモデルとLoRAで新しいワークフローを組みたい」
|
||||
- 「ネットで見つけたワークフローのプロンプトだけ借りたい」
|
||||
- 「この画像、どのLoRAをどれくらいの強さで使っていたっけ？」
|
||||
|
||||
ワークフローを丸ごと読み込むのではなく、**必要な部品だけを取り出して使える** のがポイントです。
|
||||
|
||||
---
|
||||
|
||||
## 9-4 WS Libraryの見た目を変える
|
||||
|
||||
WS Libraryのパネル内にある **歯車ボタン**から、パネルの配色を設定できます。
|
||||
|
||||
- パネルの背景色
|
||||
- サブヘッダーの背景色
|
||||
- 文字の色・補助的な文字の色
|
||||
- 枠線の色
|
||||
|
||||
ComfyUIのテーマに合わせて調整すると、違和感なく使えます。
|
||||
設定はブラウザに保存され、次に開いたときも反映されます。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - WS Libraryは、ComfyUIのノード画面の横に開く「素材の棚」
|
||||
> - W・N・M・P のタブから、ワークフロー・ノード・モデル・プロンプトをドラッグ（またはダブルクリック）で置ける
|
||||
> - Mタブからモデルをドラッグすると、モデルが選ばれた状態の読み込みノードが置かれる
|
||||
> - 右クリックの「Save as Node Set」で、よく使うノードの組み合わせを保存できる
|
||||
> - Iタブに画像をドロップすると、使われていたモデル・LoRA・プロンプトを取り出して使える
|
||||
@@ -0,0 +1,443 @@
|
||||
# 第10章 AI TOOL － ローカルAIを助手にする
|
||||
|
||||
Workflow Studioは、パソコンの中で動く **ローカルAI（大規模言語モデル）** と連携できます。
|
||||
ローカルAIを用意すると、次のようなことができるようになります。
|
||||
|
||||
- 日本語で考えた内容を、英語のプロンプトに翻訳する
|
||||
- AIとチャットしながら、プロンプトのアイデアを出してもらう
|
||||
- 画像を見せて、その画像の説明やプロンプト、タグを作ってもらう
|
||||
- ワイルドカード用の候補リストを作ってもらう
|
||||
- **スキル** を選んで、プロンプト作り・ストーリーボード・不具合の相談など、得意分野を切り替える（v0.9.5〜）
|
||||
- 生成した画像が依頼どおりかをAIにチェックさせて、足りない点だけ直す（v0.9.5〜）
|
||||
- 画像に写っている文字を読み取る（OCR、v0.9.5〜）
|
||||
|
||||
インターネット上のサービスではなく、自分のパソコンの中で動くので、
|
||||
**入力した内容や画像が外部に送られない** のが大きな特長です。
|
||||
|
||||
> **ポイント**
|
||||
> AI機能は「あると便利」な機能です。使わなくても、Workflow Studioの基本機能はすべて使えます。
|
||||
> パソコンの性能（特にGPUのメモリ）に余裕がある方は、ぜひ試してみてください。
|
||||
|
||||
---
|
||||
|
||||
## 10-1 ローカルAIソフトを準備する
|
||||
|
||||
### 対応しているソフト
|
||||
Workflow Studioは、次の4つのローカルAIソフトに対応しています。
|
||||
|
||||
| ソフト | 特徴 | 標準のアドレス |
|
||||
| --- | --- | --- |
|
||||
| **Ollama**（オラマ） | 定番。モデルのダウンロードから実行までかんたん | `http://localhost:11434` |
|
||||
| **LM Studio** | 画面で操作できる。モデル探しもアプリ内でできる | `http://localhost:1234` |
|
||||
| Lemonade Server | AMD製のGPU・NPUでも動かしやすい | `http://localhost:13305` |
|
||||
| Unsloth | APIキーなどの追加設定が必要（上級者向け） | `http://localhost:8888` |
|
||||
|
||||
はじめての方には **Ollama** をおすすめします。本章では、Ollamaを例に説明します。
|
||||
|
||||
> **ポイント**
|
||||
> アドレスの `localhost` は「自分のパソコン」、その後ろの数字は **ポート番号** です（第2章 2-4）。
|
||||
> ComfyUIと同じように、AIソフトもそれぞれ決まったポート番号で待ち受けています。
|
||||
|
||||
### Ollamaの準備
|
||||
1. Ollamaの公式サイトから、お使いのOS用のインストーラーをダウンロードして、インストールします。
|
||||
2. Ollamaを起動します（Windowsでは、インストール後に自動で起動し、タスクバーの通知領域に常駐します）。
|
||||
3. 使うAIモデルをダウンロードします。コマンドプロンプトなどで、次のように入力します。
|
||||
|
||||
```
|
||||
ollama pull （モデル名）
|
||||
```
|
||||
|
||||
モデル名は、Ollamaの公式サイトのモデル一覧で確認できます。
|
||||
|
||||
> **モデル選びの目安**
|
||||
> - **画像を読み取る機能（VLM）も使いたい** → 画像入力（vision）に対応したモデルを選ぶ
|
||||
> （作者の検証では、Gemma 4 の `e4b` などで画像生成・画像読み取りがうまく動いています）
|
||||
> - **GPUのメモリが少ない** → パラメータ数の小さいモデル（数B＝数十億程度）を選ぶ
|
||||
> - 翻訳の精度を重視するなら、日本語が得意なモデルを選ぶ
|
||||
>
|
||||
> AIモデルの世界は変化が速いので、新しいモデルも試してみてください。
|
||||
|
||||
> **注意：GPUのメモリの取り合い**
|
||||
> ローカルAIも、画像生成と同じく **GPUのメモリ（VRAM）** を使います。
|
||||
> 両方を同時に動かすと、メモリが足りなくなって画像生成が遅くなったり、エラーになったりします。
|
||||
> 対策は 10-3 で説明します。
|
||||
|
||||
---
|
||||
|
||||
## 10-2 AI TOOLタブを使う
|
||||
|
||||
Workflow Studio本体の **AI TOOL** タブを開きます。
|
||||
画面は4つの部分に分かれていて、すべて同時に表示されます。
|
||||
|
||||
| 部分 | できること |
|
||||
| --- | --- |
|
||||
| 翻訳 | 日本語・英語・中国語などの翻訳 |
|
||||
| Chat | AIとの会話。スキルで得意分野を切り替えられる。条件がそろえば、会話から画像生成も |
|
||||
| TOOLS | 画像の読み取り（VLM）、文字起こし（OCR）、ワイルドカード作成 |
|
||||
| 設定 | 使うAIソフト・モデルの設定 |
|
||||
|
||||
> **ポイント：翻訳と設定はたためる（v0.9.5〜）**
|
||||
> 翻訳と設定の見出しの右端にある矢印（◀／▶）を押すと、その部分が細い帯にたたまれ、空いた幅をChatとTOOLSが使います。
|
||||
> 帯のどこかをクリックすると元に戻ります。たたんだ状態は、次に開いたときも保たれます。
|
||||
|
||||
### 最初に：設定でAIソフトとつなぐ
|
||||
1. **設定** の **バックエンド** で、使うソフト（例：Ollama）を選びます。
|
||||
2. **接続設定** のアドレスを確認します。Ollamaを標準のまま使っている場合は、変更不要です。
|
||||
3. **接続テスト** を押します。「接続成功」と表示されればOKです。
|
||||
4. **モデル選択** の **更新** を押し、一覧から使うモデルを選びます。
|
||||
5. **設定を保存** を押します。
|
||||
|
||||
> **ポイント**
|
||||
> ここでの設定は、ComfyUI側のWS Libraryの **A（AI TOOL）** タブと共通です。
|
||||
> 一方、プロンプトタブ（第7章）とTaggerタブには、それぞれ専用の **歯車ボタン** 設定があります。
|
||||
> 「AI TOOLタブでは使えるのに、プロンプトタブでは Disconnected」というときは、プロンプトタブの歯車ボタンでも同じように設定してください。
|
||||
|
||||
> **うまくつながらないとき**
|
||||
> - Ollamaなどのソフトが起動しているか確認します。
|
||||
> - LM Studioは、アプリ内で **サーバーを起動** しておく必要があります。
|
||||
> - アドレスのポート番号が、ソフト側の設定と合っているか確認します。
|
||||
> - モデルの一覧が空のときは、モデルがダウンロードされているか確認します。
|
||||
|
||||
### 翻訳
|
||||
1. 翻訳元と翻訳先の言語を選びます（**⇄（入れ替え）** ボタンで入れ替え）。
|
||||
2. 入力欄に文章を書き、**翻訳** を押します。
|
||||
3. 結果を **コピー** して、生成UIやプリセットに貼り付けます。
|
||||
|
||||
> **ポイント**
|
||||
> TOOLSで読み取った文字（10-8のOCRなど）は、TOOLSの **翻訳へ送る** で、この入力欄にそのまま入れられます。
|
||||
|
||||
> **ポイント**
|
||||
> 小さなモデルは、翻訳せずに元の文章をそのまま返してしまうことがあります。
|
||||
> その場合は「モデルが原文をそのまま返しました」と表示されるので、別のモデルを試してください。
|
||||
|
||||
> **ポイント**
|
||||
> 翻訳専用のモデル **PLaMo 3 Translate**（第5章 5-10）を、設定のモデルに選ぶこともできます（v0.9.9〜）。
|
||||
> 日本語・英語・中国語のほか、Free（自由入力）では French・フランス語・français のような言語名で、PLaMoが対応する言語を指定できます。PLaMoが対応していない言語（ポルトガル語など）を翻訳先にすると、エラーになります。
|
||||
> WS LibraryのAタブの翻訳も同じです。
|
||||
|
||||
### Chat
|
||||
入力欄に話しかけて、**Enter** で送信します（**Shift＋Enter** で改行）。
|
||||
会話の流れを覚えているので、「もっと明るい雰囲気にして」のような続けての依頼もできます。
|
||||
**Clear** で会話をリセットします。
|
||||
|
||||
> **ポイント：チャットから画像を生成する（上級）**
|
||||
> 「ツール呼び出し（Tool Calling）」に対応したモデルを使うと、
|
||||
> チャットで「猫の絵を描いて」と頼むだけで、生成UIに読み込んだワークフローで画像が生成され、チャット内に表示されます。
|
||||
> この機能は、入力欄の横の **画像生成を許可する** のチェックでオン・オフできます。
|
||||
|
||||
> **ポイント**
|
||||
> Chatの上にあるドロップダウンで **スキル** を選ぶと、AIの得意分野を切り替えられます（10-5）。
|
||||
> AIの返事は、見出し・箇条書き・コードの枠を整えて表示されます（v0.9.5〜）。
|
||||
|
||||
### TOOLS － 画像を読み取る（VLM）
|
||||
画像入力に対応したモデル（VLM）を使うと、画像の内容をAIに読み取らせることができます。
|
||||
|
||||
1. **TOOLS** の枠に、画像（PNG／JPG／WebP）をドロップします。
|
||||
2. 作業を選びます。
|
||||
|
||||
| 作業 | 内容 |
|
||||
| --- | --- |
|
||||
| 画像の解説 | 画像に何が写っているかを文章で説明する |
|
||||
| プロンプト作成 | 画像を再現するためのプロンプトを作る |
|
||||
| Create tags（英語表示のまま） | 画像の要素をタグ形式で書き出す |
|
||||
| Create wildcards（英語表示のまま） | カテゴリー名と件数を指定して、ワイルドカードの候補を作る（画像は不要） |
|
||||
| 文字起こし（OCR） | 画像に写っている文字を読み取る（10-8） |
|
||||
|
||||
3. **実行** を押します。
|
||||
4. 結果の下のボタンで、結果を使います（v0.9.5〜）。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| 翻訳へ送る | 翻訳の入力欄に入れる（翻訳がたたまれていれば開く） |
|
||||
| Chatへ送る | Chatの入力欄に追加する |
|
||||
| コピー | クリップボードにコピーする |
|
||||
|
||||
> **例**
|
||||
> 気に入ったイラストの雰囲気をまねしたいとき、その画像から「プロンプト作成」をして、
|
||||
> できたプロンプトを生成UIで使ってみる － という使い方ができます。
|
||||
|
||||
### ComfyUIの画面の中でも使える
|
||||
同じAI機能は、ComfyUI側のWS Libraryの **A** タブでも使えます。
|
||||
ノード画面で作業しながら、ちょっと翻訳したいときに便利です。
|
||||
|
||||
---
|
||||
|
||||
## 10-3 VRAMの解放とメモリ管理
|
||||
|
||||
10-1で触れたとおり、ローカルAIと画像生成は、GPUのメモリ（VRAM）を取り合います。
|
||||
Ollamaは、一度使ったモデルをしばらくメモリに置いたままにするため、
|
||||
AIを使ったあとに画像生成をすると、メモリが足りなくなることがあります。
|
||||
|
||||
### 手動で解放する
|
||||
AI TOOLの設定にある **モデルをアンロード** を押すと、AIモデルがメモリから降ろされ、VRAMが空きます。
|
||||
（Ollamaの場合は、読み込まれているすべてのモデルが降ろされます）
|
||||
|
||||
### 生成の前に自動で解放する
|
||||
設定タブの **VRAM調整（Ollama）** で、画像生成の直前に自動でOllamaのモデルを降ろすように設定できます。
|
||||
|
||||
| 生成前の動作 | 動き |
|
||||
| --- | --- |
|
||||
| 何もしない | 自動では降ろさない |
|
||||
| 空きが目標未満のときだけ必要な分をアンロード | 空きVRAMが **目標の空き（GB）** より少ないときだけ降ろす |
|
||||
| 毎回すべてアンロード | 生成のたびに、必ずすべて降ろす |
|
||||
|
||||
> **ポイント**
|
||||
> どれを選べばよいか迷ったら、「空きが目標未満のときだけ必要な分をアンロード」がおすすめです。
|
||||
> 目標の空きには、画像生成に必要なメモリの量（使うモデルによって異なります）を目安に入力します。
|
||||
> **今すぐすべてアンロード** ボタンで、その場ですぐに解放することもできます。
|
||||
|
||||
> **注意**
|
||||
> この自動解放は、Ollamaだけが対象です。LM Studioなどを使っている場合は、手動で解放してください。
|
||||
|
||||
---
|
||||
|
||||
## 10-4 セキュリティ上の注意
|
||||
|
||||
### 標準では「自分のパソコン」のAIにだけつながる
|
||||
安全のため、Workflow StudioがつなぐAIソフト（Ollama、LM Studioなど）は、標準では
|
||||
**同じパソコンの中（localhost／127.0.0.1）で動いているもの** に限られています。
|
||||
|
||||
### 別のパソコンのAIを使いたいとき
|
||||
家庭内LANにある別のパソコン（GPUの強いパソコンなど）でOllamaを動かして使いたい場合は、
|
||||
環境変数 **`WFS_ALLOWED_BACKEND_HOSTS`** に、つなぎたいパソコンのアドレスを登録して、ComfyUIを再起動します。
|
||||
|
||||
```
|
||||
WFS_ALLOWED_BACKEND_HOSTS=192.168.1.20:11434
|
||||
```
|
||||
|
||||
複数ある場合は、カンマ（,）で区切って並べます。
|
||||
環境変数の設定方法は、ComfyUIの起動方法によって異なります。
|
||||
|
||||
### ComfyUIを `--listen` 付きで起動している場合
|
||||
ComfyUIを `--listen` オプション付きで起動すると、同じネットワークにいるほかのパソコンやスマホからも
|
||||
ComfyUI（とWorkflow Studio）にアクセスできるようになります。
|
||||
その場合、アクセスできる人なら誰でも、あなたのAIソフトや画像を使えてしまいます。
|
||||
|
||||
> **注意**
|
||||
> `--listen` は、家庭内LANなど **信頼できるネットワークでだけ** 使ってください。
|
||||
> カフェや公共のWi-Fiでは使わないようにしましょう。
|
||||
|
||||
---
|
||||
|
||||
## 10-5 スキル － Chatの得意分野を切り替える（v0.9.5〜）
|
||||
|
||||
**スキル** は、Chatに「どんな役割で、どんな形で答えるか」を教える指示書です。
|
||||
Chatの上のドロップダウン（最初は「-- スキルなし --」）で選ぶと、そのスキルの指示が毎回の会話に付けて送られます
|
||||
（指示の文章はチャットの画面には表示されません）。
|
||||
|
||||
### 最初から入っているスキル
|
||||
Workflow Studioには、次の6つのスキルが入っています（名前は英語表示のままです）。
|
||||
|
||||
| スキル | 何をするか | 結果の使い方 |
|
||||
| --- | --- | --- |
|
||||
| T2I Prompt Enhancer | 短いアイデアを、生成UIのワークフローのモデルに合った書き方のプロンプトにする | プロンプトの下のボタン |
|
||||
| Image to Prompt | TOOLSに入れた画像を再現するプロンプトを書く（画像を読み取れるモデルが必要） | プロンプトの下のボタン |
|
||||
| Storyboard to Lab | 物語をシーンごとのプロンプトに分け、Labで連続生成できる形にする | **Labで開く** |
|
||||
| Workflow Doctor | ワークフローの説明や、エラーの原因の相談にのる（10-7） | 文章で答える |
|
||||
| SVG Icon Generator | 簡単なアイコンやイラストをSVG（ベクター画像）で描く | プレビューと **Galleryに保存** |
|
||||
| Skill Creator | 質問に答えていくと、新しいスキルを作ってくれる（10-9） | **スキルとして保存** |
|
||||
|
||||
### プロンプトを作って、そのまま生成する（T2I Prompt Enhancer）
|
||||
1. 生成UIに、使うワークフローを読み込みます。スキルはこのワークフローのモデルを見て、プロンプトの書き方を決めます。
|
||||
2. AI TOOLのChatで、ドロップダウンから **T2I Prompt Enhancer** を選びます。
|
||||
3. 「雨の夜の東京、ネオン、傘を持つ猫」のように、日本語で作りたい絵を書いて送信します。
|
||||
4. 返事に、選んだ書き方（`Style:` の行）と、英語のプロンプトの枠が表示されます。その下のボタンで使います。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| GenerateUIに反映 | 生成UIのポジティブ／ネガティブの欄に書き込む |
|
||||
| このプロンプトで生成 | その場で生成し、チャットに画像を表示する |
|
||||
| コピー | ポジティブプロンプトをコピーする |
|
||||
|
||||
モデルに合わせた書き方は、次のとおりです。
|
||||
|
||||
| 生成UIのモデル | プロンプトの書き方 |
|
||||
| --- | --- |
|
||||
| Qwen-Image、FLUX、Z-Image など | 英語の文章（ネガティブは、CFGが1前後なら空にする） |
|
||||
| Illustrious、NoobAI、Pony など（SDXLのアニメ系） | Danbooruのタグ（品質タグから始める） |
|
||||
| 写実系のSDXL、SD1.5 | 短い言葉をカンマで並べる |
|
||||
|
||||
> **ポイント**
|
||||
> 「もっと夜らしく」「猫を2匹に」のように続けて頼むと、直したプロンプト全体を出し直します。
|
||||
|
||||
> **ポイント：今入っているプロンプトは渡さない**
|
||||
> 生成UIに今入っているプロンプトは、ふだんはAIに渡しません。渡すと、小さなモデルが古いネガティブプロンプトなどを新しいプロンプトに混ぜてしまうためです。
|
||||
> 「**今のプロンプトを改善して**」のように、今のプロンプトに触れて頼んだときだけ渡します。
|
||||
|
||||
### ストーリーボードから連続生成する（Storyboard to Lab）
|
||||
1. 生成UIに、文章から画像を作る（T2I）ワークフローを読み込みます。
|
||||
2. Chatで **Storyboard to Lab** を選び、「茶トラの子猫が朝目覚めて、庭で蝶を追いかけ、夕方に縁側で昼寝する。3シーンで」のように頼みます。
|
||||
3. 返事に、シーンの一覧と **Labで開く（3シーン）** ボタンが表示されます。ボタンを押します。
|
||||
4. 生成UIの **Lab**（第5章 5-7）が開き、まだ保存していないプランとして読み込まれます。
|
||||
T2Iモードになり、Batchがシーンの数に、Prompt列がシーンごとのキーフレームになります。KSampler列には、全シーン共通のシード（`Seed …`）が入ります。
|
||||
5. 内容を確かめて **Run** を押します。残したいときは **Plan Save** で保存します。
|
||||
|
||||
> **ポイント**
|
||||
> 登場人物の見た目（髪・服の色など）は、全シーンで同じ文章が入るように書かれ、シードも固定されます。
|
||||
> そのため、シーンが変わっても同じキャラクターに見えやすくなります。
|
||||
|
||||
### 返事の自動チェック
|
||||
プロンプトやSVGを出すスキルでは、Workflow Studioが返事の形を自動で確かめます。
|
||||
プロンプトの枠が崩れている、SVGが閉じていない、といった問題があると、
|
||||
点線の枠で「**自動チェック：…**」と表示し、AIに1回だけ直させます。
|
||||
説明だけを頼んだときなど、プロンプトやSVGを出していない返事はそのままにします。
|
||||
|
||||
### スキルの管理（鉛筆ボタン）
|
||||
ドロップダウンの右の鉛筆ボタンで、スキルの一覧が開きます。
|
||||
|
||||
- **＋ 新規** で新しいスキルを書く、各行の鉛筆ボタンで編集・削除する、**取り込み** で他のスキルを追加する（10-9）
|
||||
- 最初から入っているスキルには **同梱** と表示されます。
|
||||
編集していない同梱スキルは、Workflow Studioを更新すると自動で新しい版になります。
|
||||
編集したスキルは自動では置き換わらず、新しい版があると **↻（更新）** ボタンが表示されます。
|
||||
- スキルのファイルは `ComfyUI/user/default/Workflow-Studio/ai_skills/` に保存されます。
|
||||
|
||||
> **注意**
|
||||
> ↻（更新）で新しい版に置き換えると、そのスキルに加えた編集は消えます。
|
||||
|
||||
---
|
||||
|
||||
## 10-6 生成した画像をチェックして直す（v0.9.5〜）
|
||||
|
||||
チャットから生成した画像が、頼んだとおりになっているかをAIに確かめさせる機能です。
|
||||
足りないところがあれば、その点だけを直したプロンプトで描き直せます。
|
||||
|
||||
### 準備：判定用のモデル
|
||||
チェックには、ふつうのチャット用モデルではなく、Ollamaの **意思決定モデル**（はい／いいえを確率で答えるモデル）を使います。
|
||||
画像を読み取れるもの（作者の環境では `d3:4b`、`clef-flash`、`clef` で動作を確認）を、`ollama pull` で用意します。
|
||||
|
||||
> **ポイント：なぜ専用のモデルを使うのか**
|
||||
> 作者の検証では、チャット用のモデルに画像を見せて「剣を持っているか」と聞くと、持っていないのに「はい」と答えることがありました。
|
||||
> 意思決定モデルは「はい」の確率を数字で返すので、あいまいな答えを見分けられます。
|
||||
|
||||
1. AI TOOLの **設定** にある **CHAT 画像チェック** で、**判定モデル（Ollamaの意思決定モデル）** を選びます（一覧が空なら **更新**）。
|
||||
2. **自動修正の回数** を選びます。最初は **0（ボタンで修正）** がおすすめです。
|
||||
3. **設定を保存** を押します。
|
||||
|
||||
### 使い方
|
||||
1. Chatの入力欄の下にある **画像をチェック** にチェックを入れます。
|
||||
2. チャットから画像を生成します（AIの画像生成、または **このプロンプトで生成**）。
|
||||
3. 画像の下に結果のカードが表示されます。AIが頼んだ内容を3〜8個の確認項目に分け、1つずつ確かめた結果です。
|
||||
|
||||
| 表示 | 意味 |
|
||||
| --- | --- |
|
||||
| ✔（満たしている） | 「はい」の確率が85％以上 |
|
||||
| ?（はっきりしない） | 30〜85％ |
|
||||
| ✘（満たしていない） | 30％未満 |
|
||||
|
||||
4. ✘や?があるときは **足りない点を直して再生成** を押します。満たしていない項目だけをAIに伝え、プロンプトを書き直して描き直します。
|
||||
|
||||
自動修正の回数を1〜3にすると、ボタンを押さなくても、その回数まで自動でくり返します。
|
||||
|
||||
> **ポイント：SVGもチェックできる**
|
||||
> SVG Icon Generatorで描いたSVGも、画像にしてから「頼んだものに見えるか」「部品（くちばし・翼など）が描けているか」をチェックします。
|
||||
> **分かりにくい部分を描き直させる** で直させられます。
|
||||
|
||||
> **注意**
|
||||
> - チェックは完ぺきではありません。とくに「誰が何を身につけているか」の取り違えに弱いため、
|
||||
> 人物が複数いるときは「帽子は女の子ではない」のような逆向きの確認項目を自動で加えて補っています。
|
||||
> - 描き直しの上手さは、チャット用モデルの力しだいです。直らないときは、頼み方を変えるか、モデルを変えてください。
|
||||
> - 判定モデルもGPUのメモリを使います（`d3:4b` で約6GB）。読み込んだあとは、1回のチェックが数秒です。
|
||||
|
||||
---
|
||||
|
||||
## 10-7 ワークフローの不具合を相談する（Workflow Doctor、v0.9.5〜）
|
||||
|
||||
**Workflow Doctor** は、生成UIに読み込んだワークフローの説明や、エラーの原因の相談にのるスキルです。
|
||||
AIの知識だけに頼らず、送信するたびにWorkflow Studioが実際に確かめた次の内容をAIに渡します。
|
||||
|
||||
| 確かめる内容 | 例 |
|
||||
| --- | --- |
|
||||
| インストールされていないノード | ComfyUIが知らないノードの名前 |
|
||||
| ComfyUIの一覧に無いファイル・設定値 | 見つからないモデルやLoRA（似た名前の候補も） |
|
||||
| 線のつながり | つながっていない入力、存在しないノードへの線 |
|
||||
| 直近の実行エラー | ComfyUIの履歴に残ったエラーの内容 |
|
||||
| ComfyUIのログ | ログに出た最新のエラーの部分 |
|
||||
|
||||
### 使い方
|
||||
1. 相談したいワークフローを生成UIに読み込みます（エラーの相談なら、一度実行してエラーを出しておきます）。
|
||||
2. Chatで **Workflow Doctor** を選びます。
|
||||
3. 「実行する前に問題がないか確認して」「さっきのエラーの原因は？」のように聞きます。
|
||||
|
||||
> **例**
|
||||
> LoRAのファイルを別のフォルダに移したワークフローで聞くと、
|
||||
> 「`temp\` のフォルダに指定されたファイルが見つからない。`Illustrious\` に同じ名前のファイルがある」と、直し方まで答えました。
|
||||
|
||||
> **ポイント**
|
||||
> 確かめる内容は、送信のたびに作り直されます。直したあとは、もう一度聞いてみてください。
|
||||
|
||||
---
|
||||
|
||||
## 10-8 画像の文字を読み取る（OCR、v0.9.5〜）
|
||||
|
||||
TOOLSで、画像に写っている文字を読み取れます。読み取った文字は、そのまま翻訳やChatに送れます。
|
||||
|
||||
### 準備：OCR用のモデル
|
||||
OCRには、チャット用とは別の、文字を読むのが得意なモデルを使います。Ollamaで用意してください。
|
||||
作者の環境で、横書きと縦書きの日本語・英語がまじった画像を読ませた結果は次のとおりです。
|
||||
|
||||
| モデル | 時間 | 読み取り |
|
||||
| --- | --- | --- |
|
||||
| LightOnOCR-3（`maternion/LightOnOCR-3:4b`） | 約1秒（位置つきは約6〜8秒） | すべて読めた |
|
||||
| PaddleOCR-VL（マンガ向けの版） | 約4秒 | すべて読めた |
|
||||
| DeepSeek-OCR系 | 約14秒 | すべて読めた（いつも位置つき） |
|
||||
| PaddleOCR-VL 1.6、GLM-OCR | 約11〜15秒 | 縦書きを読み落とした |
|
||||
|
||||
### 使い方
|
||||
1. TOOLSに画像をドロップします。
|
||||
2. 作業で **文字起こし（OCR）** を選びます。
|
||||
3. **OCRモデル** を選びます。名前に「ocr」を含むOllamaのモデルが、向いている順に並びます（一覧が空なら ↻ を押します）。
|
||||
4. **実行** を押します。読み取った文字が、結果の欄に1行ずつ表示されます。
|
||||
|
||||
> **ポイント：文字の位置を表示**
|
||||
> **文字の位置を表示** にチェックを入れると、画像の上に番号つきの枠が重なり、どこの文字かが分かります。
|
||||
> 枠をクリックすると、結果の欄の同じ行が選ばれます。
|
||||
> 位置を返せるのは LightOnOCR と DeepSeek-OCR系だけです。ほかのモデルでは、このチェックは選べません。
|
||||
|
||||
> **ポイント**
|
||||
> 結果が空だったり、同じ行がくり返されたりしたときは、自動で白黒（グレースケール）にして1回だけ読み直します。
|
||||
|
||||
---
|
||||
|
||||
## 10-9 スキルを自作・取り込む（上級、v0.9.5〜）
|
||||
|
||||
### Skill Creatorで作る
|
||||
Chatで **Skill Creator** を選び、「〇〇が得意なスキルを作りたい」と話しかけます。
|
||||
AIが1つずつ質問してくるので答えていくと、スキルの下書きができます。
|
||||
返事の下の **スキルとして保存** を押すと、スキルの一覧に下書きが入った状態で開くので、確かめて保存します。
|
||||
「このスキルの、ここがうまくいかない」と相談して、今あるスキルを直してもらうこともできます。
|
||||
|
||||
### スキルのファイルの中身
|
||||
スキルは、テキストファイル（`.md`）です。先頭の `---` で囲んだ部分に名前と説明を書き、その下にAIへの指示を書きます。
|
||||
先頭の部分の `metadata:` には、次の設定も書けます（書かなくてもかまいません）。
|
||||
|
||||
| 設定 | 意味 |
|
||||
| --- | --- |
|
||||
| `output` | `prompt` か `svg`。返事の形を自動でチェックし、ボタンを付ける |
|
||||
| `tools` | `[]` にすると、AIが自分で画像生成を始めない |
|
||||
| `temperature` | 答えのばらつき（0.3なら安定、0.9なら自由） |
|
||||
| `think` | 考える（Thinking）モードを、このスキルだけオン／オフにする |
|
||||
| `context` | AIに渡す情報。`workflow`（ワークフローの要約）、`current-prompt`（今のプロンプト）、`workflow-detail`／`logs`（Workflow Doctorと同じ確認内容） |
|
||||
|
||||
### ほかのスキルを取り込む
|
||||
スキルの一覧の **取り込み** で、次のファイルを追加できます。
|
||||
|
||||
- `.md` のスキル1つ
|
||||
- `SKILL.md` を含むフォルダをZIPにしたもの（**Agent Skills** という、多くのAIツールで共通の形式）
|
||||
|
||||
フォルダの中のテキストファイル（参考資料など）も一緒に取り込まれ、AIが必要になったときだけ読みに行きます
|
||||
（ツール呼び出しに対応したモデルが必要です）。
|
||||
|
||||
> **注意**
|
||||
> - 取り込むのはテキストファイルだけです。スクリプトなどのプログラムは取り込まず、実行もしません。
|
||||
> - スキルの指示は、そのままAIに渡されます。他の人が作ったスキルは、中身を読んでから使ってください。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ローカルAI（Ollamaなど）を用意すると、翻訳・チャット・画像の読み取りができる。入力内容は外部に送られない
|
||||
> - AI TOOLタブの設定で、バックエンド・アドレス・モデルを選び、接続テストをする
|
||||
> - プロンプトタブとTaggerタブには、別に歯車ボタンの設定がある
|
||||
> - AIと画像生成はVRAMを取り合う。「モデルをアンロード」や「VRAM調整（Ollama）」で解放する
|
||||
> - 標準では自分のパソコンのAIにだけつながる。`--listen` は信頼できるネットワークだけで
|
||||
> - **スキル** で得意分野を切り替える。T2I Prompt Enhancer でプロンプトを作り、ボタンで生成UIに反映・生成できる
|
||||
> - **画像をチェック** をオンにすると、意思決定モデルが生成画像を確かめ、足りない点だけ直して描き直せる
|
||||
> - Workflow Doctor は、実際に確かめた事実をもとにワークフローの不具合を相談にのる
|
||||
> - TOOLSの **文字起こし（OCR）** で画像の文字を読み取り、翻訳やChatへ送れる
|
||||
@@ -0,0 +1,362 @@
|
||||
# 第11章 さらに広がる機能（紹介編）
|
||||
|
||||
ここまでで、Workflow Studioの基本的な使い方はひととおり身につきました。
|
||||
この章では、まだ紹介していないタブや機能を、**どんなことができるか** を中心に紹介します。
|
||||
すべてを今すぐ使う必要はありません。「こんなこともできるんだ」と頭の片隅に置いておき、
|
||||
必要になったときに試してみてください。
|
||||
|
||||
---
|
||||
|
||||
## 11-1 ノードタブ － インストール済みのノードを一覧する
|
||||
|
||||
**ノード（Nodes）** タブでは、ComfyUIにインストールされているすべてのノードを一覧できます。
|
||||
|
||||
- **検索・絞り込み** － 名前、カテゴリー、パッケージ（どのカスタムノードに含まれるか）で絞り込めます。数千個あっても、一覧が先に表示されて検索も軽く動きます（v0.8.0〜）。
|
||||
- **詳細パネル** － ノードを選ぶと、入力と出力の種類（I/O仕様）を確認できます。
|
||||
- **お気に入り・グループ・タグ** － よく使うノードを整理できます。
|
||||
|
||||
ここでお気に入りやグループに登録したノードは、WS Libraryの **N** タブに表示され、
|
||||
ノード画面へドラッグで置けるようになります（第9章）。
|
||||
|
||||
> **こんなときに**
|
||||
> 「カスタムノードを入れたけど、どんなノードが増えたのか分からない」
|
||||
> → **パッケージ** で絞り込むと、そのカスタムノードに含まれるノードの一覧が見られます。
|
||||
|
||||
---
|
||||
|
||||
## 11-2 Tagger － 画像からタグを自動で作る
|
||||
|
||||
**Tagger** タブは、画像を読み取って、その画像を表すタグ（`1girl, long hair, smile` など）を自動で作る機能です。
|
||||
アニメ系のモデルのプロンプト作りや、学習用画像のタグ付けに使えます。
|
||||
|
||||
### 3つのサブタブ
|
||||
| サブタブ | できること |
|
||||
| --- | --- |
|
||||
| Single | 1枚の画像からタグを作る |
|
||||
| Batch | フォルダ内の画像に、まとめてタグを付ける（画像と同じ名前の `.txt` ファイルを書き出すことも可能） |
|
||||
| DB | タグを付けた画像の記録を検索・編集する |
|
||||
|
||||
### 使い方（Single）
|
||||
1. 画像を、プレビュー部分にドロップします（ギャラリーの詳細パネルの **Tagger** ボタンからも開けます）。
|
||||
2. 使うモデルを選び、実行ボタンを押してタグを作ります。
|
||||
3. 結果を使います。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| GenUI:P | 生成UIのポジティブプロンプトに追加する |
|
||||
| Send to Prompt | プロンプトタブのポジティブプロンプトに追加する |
|
||||
|
||||
**しきい値** のスライダーで、どのくらい確かなタグまで出すかを調整できます。
|
||||
数値を上げるとタグは少なく確かなものだけに、下げると多くなります。
|
||||
|
||||
### 準備が必要
|
||||
Taggerを使うには、次の準備が必要です。
|
||||
|
||||
1. **タグ付け用のモデル**（WD Tagger など）を、ComfyUIの `models/tagger/（モデル名）/` フォルダに置きます。
|
||||
フォルダには、`.onnx` ファイルと `selected_tags.csv` を入れます。
|
||||
**PixAI Tagger v1.0**（v0.8.10〜）も使えます。Hugging Faceのコミュニティ変換版（`Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8`）から、
|
||||
`.onnx` を1つ（GPUなら `model_fp16.onnx` がおすすめ）と、`selected_tags.csv`・`preprocess.json` を同じフォルダに入れると、自動で判別されます。
|
||||
PixAIのモデルを選ぶと、しきい値のスライダーが、PixAIの推奨値（0.17／0.27）に切り替わります。
|
||||
2. **追加のPythonパッケージ** をインストールします。
|
||||
Workflow Studioのフォルダにある `requirements.txt` を使います。
|
||||
|
||||
```
|
||||
（ComfyUIのPython） -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
「ComfyUIのPython」の場所は環境によって違います。第2章 2-6 の表で確認してください
|
||||
（例：Windowsのポータブル版では `python_embeded` フォルダの中の `python.exe`）。
|
||||
|
||||
> **注意**
|
||||
> パソコンにもともと入っているPythonにインストールしても、ComfyUIからは使えません。
|
||||
> 必ず、**ComfyUIが使っているPython** にインストールしてください。
|
||||
> 分からない場合は、ComfyUI Managerの機能でパッケージをインストールする方法もあります。
|
||||
|
||||
> **注意：GPUで動いているか確認する**
|
||||
> Taggerは、NVIDIAのGPUで動かすと1枚あたり1秒かからないのに対し、CPUで動くとPixAIでは1枚あたり20秒ほどかかります。
|
||||
> とても遅いときは、GPU用のパッケージが正しく入っているか確認してください（第12章 12-5）。
|
||||
|
||||
> **ポイント**
|
||||
> ローカルAI（第10章）の画像読み取り（VLM）と組み合わせて、両方の結果をまとめたタグを作ることもできます。
|
||||
|
||||
---
|
||||
|
||||
## 11-3 Image Edit － 生成した画像にちょっと手を加える
|
||||
|
||||
**Image Edit** タブは、レイヤー機能を持った簡易画像編集ツールです。
|
||||
生成した画像に文字を入れたり、一部を描き直すためのマスクを作ったりできます。
|
||||
|
||||
### 主な機能
|
||||
| ツール | できること |
|
||||
| --- | --- |
|
||||
| Select | レイヤーの移動・拡大縮小・回転 |
|
||||
| Draw | ペンで描く（ペンタブレットの筆圧にも対応） |
|
||||
| Text | 文字を入れる |
|
||||
| Shape | 四角・丸・線を描く |
|
||||
| Fill | 塗りつぶし・グラデーション |
|
||||
| Mask | 描き直したい部分を指定する（マスク作成） |
|
||||
| Blur | ぼかし・モザイク |
|
||||
| BG Remove | 背景を消す |
|
||||
| Inpaint | マスクした部分を、生成UIのワークフローで描き直す |
|
||||
|
||||
- 画像は、ドラッグ＆ドロップ、**File Open**、またはギャラリーの **Image Edit** ボタンで開きます。
|
||||
- 編集した画像は、**Save PNG**（ダウンロード）、**Save to Gallery**（ギャラリーに保存）などで保存します。
|
||||
- **Ctrl＋Z** で元に戻す、**Ctrl＋Y** でやり直しができます。
|
||||
|
||||
### Drawの色とショートカット（v0.9.3〜）
|
||||
**Draw** を選ぶと、ツールの2段目に **カラーパレット** が表示されます。
|
||||
|
||||
- **色1／色2**、基本色20色、カスタム色20枠が並びます。色をクリックすると、その色で描けます。
|
||||
- **＋** か空いている枠をクリックすると、今の色をカスタム色に登録します。色の上で右クリックすると、登録・編集・削除ができます（基本色も変えられ、最初の色に戻せます）。
|
||||
登録した色は、ブラウザごとに保存されます。
|
||||
- 右側の **Color** の隣にある **スポイト** ボタンで、キャンバスの色を拾えます（Escで取り消し）。
|
||||
- 描き始めの色は黒（色1）と白（色2）です。
|
||||
|
||||
| キー（Draw中） | 動き |
|
||||
| --- | --- |
|
||||
| D／E | 描く（Draw）／消す（Erase） |
|
||||
| X | 色1と色2を入れ替える |
|
||||
| ［ ／ ］ | ブラシを小さく／大きくする（実際は半角の `[` `]`） |
|
||||
| B | ブラシの選択を開く・閉じる（Escでも閉じる） |
|
||||
|
||||
### インペイント（部分の描き直し）
|
||||
Image Editのいちばん便利な使い方が、**インペイント** です。
|
||||
「顔だけ描き直したい」「手が崩れたところだけ直したい」というときに使います。
|
||||
|
||||
1. 生成UIに、インペイント用のワークフロー（付属サンプルの `ws_inpaint_basic` など）を読み込みます。
|
||||
2. Image Editで画像を開き、**Mask** で描き直したい部分を塗ります。
|
||||
3. **Inpaint** ツールで実行すると、塗った部分だけが描き直されます。
|
||||
|
||||
> **ポイント**
|
||||
> 背景を消す機能の高品質版や、より高度なマスク機能を使うには、
|
||||
> 作者の別のカスタムノード **comfyui-mask-editor-one** が必要です。
|
||||
> PSD形式での保存・読み込みには、追加のPythonパッケージ（psd-tools）が必要です。
|
||||
|
||||
---
|
||||
|
||||
## 11-4 Video － 動画を作る・つなぐ
|
||||
|
||||
**Video** タブは、動画生成モデルを使って、画像や文章から短い動画を作るための専用画面です。
|
||||
|
||||
### 対応している動画生成モデル
|
||||
- **MiniMax H3**（画像から動画）
|
||||
- **LTX-2.5**（文章から動画、画像から動画、最初と最後の画像を指定して動画）
|
||||
- **Wan2.2**（文章から動画）
|
||||
- **FastH3 OmniRef／Pika-H3-Flex**（MiniMax H3の参照画像から動画。v0.8.14〜）
|
||||
|
||||
対応するワークフローを、ワークフロータブの **Videoに読み込み** で読み込んで使います（第4章 4-6）。
|
||||
|
||||
### 主な機能
|
||||
| サブタブ | できること |
|
||||
| --- | --- |
|
||||
| Plan | 動画を「ブロック」に分け、ブロックごとにプロンプト・長さ・最初／最後の画像を決めて、順番に生成する |
|
||||
| Edit | 生成した動画や画像を並べて、切り取り・つなぎ合わせ・文字入れ・BGM付けをして書き出す |
|
||||
| Asset | 生成した動画や画像、音声を一覧する。音声は「BGMに設定」や「サウンドとして追加」ができる（種類フィルタの Audio Only で音声だけ表示） |
|
||||
|
||||
そのほか、動画の好きな場面を静止画として保存する **Frame** や、動画をGIFアニメに変換する **GIF** の機能もあります。
|
||||
|
||||
> **ポイント**
|
||||
> Planサブタブのブロックの **Prompt** の欄にも、プロンプト翻訳の **🌐** ボタンがあります（第5章 5-10）。
|
||||
> 設定の **Video Plan － Run** をオンにすると、日本語のプロンプトのまま実行して、送る内容だけを英語にできます。
|
||||
|
||||
> **注意**
|
||||
> v0.9.8までは、新しいComfyUIで保存したワークフローを使うと、Planで入力したプロンプト・長さ（Duration）・Turboの設定が反映されず、ワークフローに保存されている値のまま生成されていました（v0.9.9で修正）。
|
||||
> 以前と同じPlanを実行しても、Planの値が反映されるようになったため、結果が変わることがあります。
|
||||
|
||||
### アセット登録と音声変換（v0.8.9〜）
|
||||
- **アセット登録** － 左側の動画ソースパネルのボタンです。パソコン内の動画を、outputフォルダ（`output/video`）へコピーして「Video Assets」グループに登録します。
|
||||
登録すると、Assetサブタブの一覧に出て、Editなどで使えるようになります。
|
||||
すでにoutputフォルダにある動画（Assetで選んだものや生成結果）は、コピーせずにグループへの登録だけを行います。同じ名前があるときは、名前の後ろに連番が付きます。
|
||||
- **音声変換...** － Editサブタブの、クロップのボタンの横にあるボタンです。選んだ動画クリップの音声だけを、FLAC・MP3・Opus・M4A・WAVに変換して `output/video` に保存します。
|
||||
ギャラリーの変換と同じ画面です。音声が入っている動画クリップだけが対象です。
|
||||
- 音声変換のボタンを押すと、次の画面が開きます。**出力形式**（FLACなど）、**サンプルレート**、**チャンネル** を選び、**変換** を押します。
|
||||
「埋め込みのprompt/workflowタグを引き継ぐ」にチェックを入れると、タグを持つ形式（FLACなど）で、元の動画の情報を引き継げます。
|
||||
|
||||
- 動画ソースパネルは少し広くなり、ファイル名が1行で表示され、ボタンも横一列に並ぶようになりました。
|
||||
|
||||
### 参照画像から動画（FastH3 OmniRef／Pika-H3-Flex）（v0.8.14〜）
|
||||
このタイプのワークフローは、最初と最後の画像ではなく、**参照画像** をもとに動画を作ります。
|
||||
そのため、Planサブタブの画像の欄は、次のように名前が変わります。
|
||||
|
||||
| 欄 | このタイプでの意味 |
|
||||
| --- | --- |
|
||||
| Reference Image | 1枚目の参照画像（ふだんの First Image の欄） |
|
||||
| Reference Image 2 (optional) | 2枚目の参照画像（ふだんの Last Image の欄） |
|
||||
|
||||
- プロンプト・長さ（秒）・縦横比などは、ほかの動画ワークフローと同じように使えます。
|
||||
- 画像の欄を **None** や **Clear** で空にしたブロックは、ワークフローに参照画像が設定されていても、その画像を使わずに生成します。
|
||||
「＋ Block」で追加したブロックは画像が空の状態で始まるので、参照画像が必要なブロックでは、それぞれ画像を指定してください。
|
||||
- 画像の選び方で **Use previous block's last frame**（前のブロックの最後のフレームを使う）を選ぶと、前のブロックで実際に生成された動画の最後のフレームが、そのブロックの1枚目の参照画像になります。
|
||||
- Assetサブタブのボタンも「Reference Imageに設定」「Reference Image 2に設定」に変わります。
|
||||
|
||||
> **注意**
|
||||
> 動画生成は、画像生成よりもはるかに多くのGPUメモリと時間を使います。
|
||||
> 使うモデルの必要スペックを確認してから試しましょう。
|
||||
|
||||
---
|
||||
|
||||
## 11-5 設定タブの主な項目
|
||||
|
||||
**設定** タブでは、Workflow Studio全体の設定をします。
|
||||
項目ごとに折りたたまれているので、見出しをクリックして開きます。
|
||||
ここでは、これまでの章で紹介していない項目を中心に紹介します。
|
||||
|
||||
| 項目 | 内容 | 関連する章 |
|
||||
| --- | --- | --- |
|
||||
| テーマ | 画面の配色・フォント | 第3章 |
|
||||
| UI言語／要約言語 | 表示言語 | 第2章 |
|
||||
| 文字サイズ | プロンプト入力欄などの文字の大きさ | 第3章 |
|
||||
| ワークフローデータフォルダ | ワークフローを読み込むフォルダ | 第4章 |
|
||||
| ギャラリー Outputフォルダ | ギャラリーで表示するフォルダ | 第8章 |
|
||||
| デフォルトワークフロー | Workflow Studioを開いたとき、生成UIに自動で読み込むワークフロー | 下記 |
|
||||
| デフォルトCheckpoint | 生成UIにワークフローを読み込むとき、モデルを自動で差し替える | 下記 |
|
||||
| CivitAI APIキー | Civitaiの情報取得用（省略可） | 第6章 |
|
||||
| Eagle 連携 (オプション) | 生成画像をEagleに自動保存 | 第8章 |
|
||||
| VRAM調整（Ollama） | 生成前にAIモデルをメモリから降ろす | 第10章 |
|
||||
| プロンプト翻訳（自国語 → 英語） | 日本語のプロンプトを🌐ボタンや実行時の自動翻訳で英語にする | 第5章 5-10 |
|
||||
| データ管理 | 設定やデータのバックアップ・復元 | 下記 |
|
||||
|
||||
### デフォルトワークフロー
|
||||
いつも使う基本のワークフローを設定しておくと、Workflow Studioを開いたときに生成UIへ自動で読み込まれます。
|
||||
ワークフロータブの詳細画面の **デフォルトに設定** ボタンからも設定できます。
|
||||
|
||||
### デフォルトCheckpoint
|
||||
ネットから入手したワークフローには、手元にないモデルが指定されていることがよくあります（第5章 5-3）。
|
||||
**デフォルトCheckpoint** を設定してオンにしておくと、生成UIにワークフローを読み込むたびに、
|
||||
Checkpointが自動で指定のモデルに差し替わります。モデルの選び直しの手間が省けます。
|
||||
|
||||
### データ管理（バックアップ）
|
||||
**データ管理** では、Workflow Studioで整理した情報（お気に入り・バッジ・グループ・プリセットなど）を
|
||||
ファイルに書き出したり（エクスポート）、読み込んで元に戻したり（インポート）できます。
|
||||
|
||||
**フルバックアップ (ZIP)** を使うと、スタイルやLabのプランなどのフォルダ単位のデータも含めて、まとめてバックアップできます。
|
||||
|
||||
> **ポイント**
|
||||
> パソコンの買い替えやComfyUIの入れ直しの前には、必ずバックアップを取っておきましょう。
|
||||
> 時間をかけて整理した情報を、そのまま新しい環境に引き継げます。
|
||||
> （安全のため、APIキーはバックアップに含まれません）
|
||||
|
||||
## 11-6 連携ソフト：ComfyUI Comic Creator － 生成した画像でマンガを作る
|
||||
|
||||
**ComfyUI Comic Creator**（以下、Comic Creator）は、Workflow Studioと同じ作者が開発している、
|
||||
ComfyUI上で動く **マンガ制作用** のカスタムノードです。
|
||||
Workflow Studioとは別にインストールするソフトですが、両方を入れると連携して使えます。
|
||||
|
||||
### Comic Creatorでできること
|
||||
| 機能 | 内容 |
|
||||
| --- | --- |
|
||||
| コマ割り | テンプレートを選んで、ページにコマを作る（自作テンプレートも登録可能） |
|
||||
| 画像の配置 | コマに画像をドラッグ＆ドロップで配置する |
|
||||
| フキダシ・テキスト | 縦書き・横書きのセリフ、さまざまな形のフキダシ、効果文字を入れる |
|
||||
| マンガ効果 | 集中線・スクリーントーン・網点（ハーフトーン）などを加える |
|
||||
| 画像の編集 | レイヤー付きの画像編集（Image タブ） |
|
||||
| AIでの自動作成 | お題から、ストーリー・脚本・コマ割りをAIで下書きする（Auto タブ） |
|
||||
| 書き出し | JPEG／PNG／WebP／SVG／PDF／**EPUB** 形式で出力する |
|
||||
|
||||
### Workflow Studioとの連携
|
||||
両方をインストールすると、次のような連携ができます。
|
||||
|
||||
- **Comic Creatorの中でWorkflow Studioのギャラリーを使う**
|
||||
Comic Creatorの中にWorkflow Studioのギャラリーが表示され、生成した画像を選んで **Send CC Layout**（レイアウトタブのコマ／オーバーレイへ）または **Send CC Image**（Imageタブへ）ボタンで送れます。
|
||||
（Send CCボタンは、Comic Creatorの中で開いたギャラリーにだけ表示されます）
|
||||
- **Comic Creatorから画像を生成する**
|
||||
Comic Creatorの画面から、Workflow Studio（生成UI）を通して、コマに入れる画像を生成したり（T2I）、
|
||||
描いたラフや配置した画像をもとに描き直したり（I2I・インペイント）できます。
|
||||
Comic CreatorのNanobanana+タブ（ローカルモード）からは、複数の参照画像を使う画像編集のワークフロー（Flux.3 Klein、Qwen Image 2.1 など）も実行できます（v0.8.12〜）。
|
||||
|
||||
「Workflow Studioで画像を作る → Comic Creatorでマンガにまとめる」という流れで、
|
||||
画像生成からマンガの完成までを、ComfyUIの中だけで行えます。
|
||||
|
||||
### インストールと起動
|
||||
1. ComfyUI Managerの **Install via Git URL** に、次のURLを入力してインストールします。
|
||||
|
||||
```
|
||||
https://github.com/ketle-man/comfyui-comic-creator
|
||||
```
|
||||
|
||||
（手動の場合は、Workflow Studioと同じく `ComfyUI/custom_nodes/` に `git clone` します）
|
||||
|
||||
2. ComfyUIを再起動します。
|
||||
3. ComfyUIの上部メニューに追加された **CC** ボタンを押すと、Comic Creatorが新しいタブで開きます。
|
||||
|
||||
> **ポイント**
|
||||
> Comic Creatorの詳しい使い方は、アプリ内の **ヘルプ** タブ（日本語対応・検索機能あり）を参照してください。
|
||||
|
||||
---
|
||||
|
||||
## 11-7 TagSpace － タグとプロンプトを「意味の地図」から選ぶ（v0.9.0〜）
|
||||
|
||||
**TagSpace** タブは、Workflow Studioの中にあるプロンプトやタグ、生成した画像を、
|
||||
**意味の近さ** にしたがって3Dの空間に並べて表示する機能です。
|
||||
近くにあるものほど意味が近いので、「この雰囲気に近いタグは何だろう」と眺めながら、タグを選んでプロンプトを作れます。
|
||||
|
||||
### 空間に並ぶもの
|
||||
| 種類 | 中身 | 表示 |
|
||||
| --- | --- | --- |
|
||||
| カード | ギャラリーの **ImagePrompt**（第8章 8-4）の見本画像と、そのプロンプト | サムネイル |
|
||||
| タグ | ImagePromptのプロンプト・生成画像に埋め込まれたプロンプト・ワイルドカード・Tagger DB・Promptライブラリの文を、カンマで区切った語 | 出典ごとに色分けした点 |
|
||||
| Gallery点 | ギャラリーの生成画像の一部（それぞれ、自分のプロンプトのタグを持つ） | 薄い点 |
|
||||
|
||||
### 準備
|
||||
ギャラリーのセマンティック検索（第8章 8-6）と同じく、**Ollama** と **EmbeddingGemma 2** が必要です。
|
||||
ギャラリーのセマンティック検索の索引を、Gemini Embedding 2で作っている場合は、先にEmbeddingGemma 2で作り直してください。
|
||||
|
||||
### 空間を作る
|
||||
ツールバーで設定を選び、**作成 / 更新** を押します。
|
||||
|
||||
| 項目 | 内容 |
|
||||
| --- | --- |
|
||||
| タグ数 | 空間に入れるタグの数。出典ごとに、よく使われている順に交互に選ぶ |
|
||||
| 全タグ | オンにすると、タグ数の代わりに候補のタグをすべて使う（上限2万件） |
|
||||
| 最低出現回数 | この回数未満しか使われていないタグを除く。2にすると、1回だけの表記ゆれなどが消える |
|
||||
| ワイルドカード | ワイルドカードをタグの材料に含めるかどうか |
|
||||
| Gallery点数 | 表示する生成画像の数（0で表示なし） |
|
||||
| Tagger補完 | プロンプトを取り出せない生成画像に、Taggerのモデルでタグを付ける（OFF／モデル名。「Gallery点のみ」か「プロンプトの無い全画像」を選ぶ） |
|
||||
|
||||
- 1回目は、画像のプロンプトの読み取りやタグの準備に数分かかります。2回目からは、前回の結果を使うので数秒〜数十秒で終わります。
|
||||
- 状態表示に、カード・タグ・Gallery点の数が出ます。候補の一部だけを使ったときは「（候補 N）」と出ます。
|
||||
|
||||
> **ポイント**
|
||||
> タグの種類の大半は、ワイルドカードにしか出てこない語です。
|
||||
> 「ワイルドカード」をオフにして「全タグ」をオンにすると、自分が実際に使ってきたタグだけを全部並べられます（検証環境では約1万種）。
|
||||
|
||||
> **注意**
|
||||
> Tagger補完は、CPUで動くと1枚あたり0.5秒ほどかかります。「プロンプトの無い全画像」を選ぶ前に、TaggerがGPUで動いているか確認してください（第12章 12-5）。
|
||||
|
||||
### 見る・探す
|
||||
- **ドラッグ** で回転、**右ドラッグ** で移動、**ホイール** でズームします。
|
||||
- 点にカーソルを合わせると、名前・サムネイル・プロンプトや出典が表示されます。
|
||||
- ツールバーのチェックボックスで、カード・サムネイル・タグ・Galleryの表示や、**発光**（夜空に光る星のような表示）・**またたき** を切り替えます。
|
||||
出典ごとのチェックボックスで、タグを出典ごとに表示・非表示にできます。
|
||||
- 右上の検索欄に言葉や文章を入れて **Enter** を押すと（日本語も可）、その意味の位置に白い印が付き、近いカードやタグが一覧に出ます。
|
||||
|
||||
### タグを選んでプロンプトを作る
|
||||
1. カードやタグをクリックすると選択され、右の **近くの要素** に、意味の近いカード・タグが並びます。
|
||||
（Gallery点をクリックすると、似ている生成画像も並びます）
|
||||
2. プロンプトに入れたいものを **ダブルクリック** するか、**選択中** の **追加**、または一覧の **＋** を押します。
|
||||
カードはプロンプト全体、タグはタグそのもの、Gallery点はその画像のタグが、右端の **プロンプト** 欄に入ります。
|
||||
3. **GenerateUIへ追記（Positive）** で、生成UIに読み込んだワークフローのポジティブプロンプトに追記します（先に生成UIでワークフローを読み込んでおきます）。
|
||||
**コピー** でクリップボードにコピーすることもできます。
|
||||
|
||||
### カードや生成画像のタグで探す
|
||||
カードやGallery点を選ぶと、**選択中** に、そのプロンプトのタグがボタンで並びます。
|
||||
タグをクリックして選び（もう一度で解除）、**選択したタグで空間検索** を押すと、選んだタグで空間を検索します。
|
||||
「この画像の、髪型と服装だけに近いものを探したい」といったときに便利です。
|
||||
タグを選んでいる間は、**選択中** の **追加** ボタンが **選択したタグを追加（N）** に変わり、選んだタグだけをプロンプト欄に追加できます。
|
||||
タグを選んでいなければ、これまでどおり全体（カードはプロンプト全文、Gallery点はすべてのタグ）が追加されます。
|
||||
|
||||
Gallery点を選んでいるときは、**選択中** に **Galleryで開く** ボタンも表示されます。
|
||||
押すとギャラリータブに切り替わり、その画像のフォルダが選ばれ、画像一覧でその画像が選択された状態になります（右の詳細も開きます）。
|
||||
画像が一覧に必ず出るよう、ギャラリーの検索・お気に入り・タグ・グループの絞り込みは解除されます。
|
||||
カードのプロンプト全文は、**プロンプト全文** を開くと読めます。
|
||||
|
||||
---
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ノードタブでインストール済みのノードを一覧・整理でき、WS Libraryで使える
|
||||
> - Taggerは画像からタグを自動生成する。モデルと追加パッケージの準備が必要
|
||||
> - Image Editはレイヤー付きの簡易編集ツール。インペイントで部分的な描き直しもできる
|
||||
> - Videoタブで、対応する動画生成モデルを使った動画作りと簡単な編集ができる
|
||||
> - 設定タブの「デフォルトCheckpoint」と「データ管理（バックアップ）」は特に便利
|
||||
> - 同じ作者の Comic Creator と組み合わせると、生成した画像でマンガを作れる
|
||||
> - TagSpaceでは、手持ちのタグ・プロンプト・生成画像を意味の近さで3Dに並べ、眺めながらタグを選んでプロンプトを作れる
|
||||
@@ -0,0 +1,239 @@
|
||||
# 第12章 困ったときは（トラブルシューティング／FAQ）
|
||||
|
||||
この章では、Workflow Studioを使っていてよく起こるトラブルと、その対処方法をまとめます。
|
||||
困ったときは、まず **「症状」** の見出しから、自分の状況に近いものを探してください。
|
||||
|
||||
> **まず試してほしい3つのこと**
|
||||
> どんなトラブルでも、次の3つで解決することがよくあります。
|
||||
> 1. ブラウザのページを再読み込みする（F5キー）
|
||||
> 2. Workflow Studioの各タブの **更新**（Refresh）ボタンを押す
|
||||
> 3. ComfyUIを完全に終了して、起動し直す
|
||||
|
||||
---
|
||||
|
||||
## 12-1 起動・接続のトラブル
|
||||
|
||||
### Wボタンが表示されない
|
||||
| 原因 | 対処 |
|
||||
| --- | --- |
|
||||
| インストール後にComfyUIを再起動していない | ComfyUIを完全に終了して、起動し直す（ブラウザの再読み込みだけでは反映されません） |
|
||||
| フォルダの場所が違う | `ComfyUI/custom_nodes/ComfyUI-Workflow-Studio` にフォルダがあるか確認する。フォルダが二重になっていないかも確認 |
|
||||
| ComfyUIが古い | ComfyUI本体を更新する |
|
||||
|
||||
### Workflow Studioのページが開かない
|
||||
- ComfyUIは起動していますか？ Workflow StudioはComfyUIの中で動いているため、ComfyUIを終了すると開けません。
|
||||
- アドレスの **ポート番号** は合っていますか？ ComfyUIを開いているブラウザのアドレス欄で確認できます（第2章 2-4）。
|
||||
|
||||
### 生成UIが「Disconnected」になる
|
||||
1. ComfyUIが起動していて、ComfyUIの画面で画像を生成できるか確認します。
|
||||
2. 生成UIの上部の **Connect** を押します。
|
||||
3. それでもつながらない場合は、ブラウザのページを再読み込みします。
|
||||
|
||||
### 更新したのに動きが変わらない
|
||||
Workflow Studioを更新（アップデート）したあとは、必ずComfyUIを再起動してください。
|
||||
それでも古い動きのままのときは、ブラウザのキャッシュが残っている可能性があります。
|
||||
**Ctrl＋F5**（強制再読み込み）を試してください。
|
||||
|
||||
---
|
||||
|
||||
## 12-2 ワークフローのトラブル
|
||||
|
||||
### ワークフローが一覧に出てこない
|
||||
| 原因 | 対処 |
|
||||
| --- | --- |
|
||||
| 一覧が古いまま | ワークフロータブの **更新** を押す |
|
||||
| 絞り込みがかかっている | ツールバー右端の **× クリア** を押す |
|
||||
| 別のフォルダに保存している | 設定タブの「ワークフローデータフォルダ」を確認する |
|
||||
|
||||
### ノードが赤く表示される（missing）
|
||||
ワークフローを読み込んだとき、ノードが赤く表示されたり、「見つからないノードがあります」と表示されたりすることがあります。
|
||||
これは、そのワークフローに必要な **カスタムノードがインストールされていない** という意味です。
|
||||
|
||||
**対処**
|
||||
1. ComfyUIの **Manager** を開きます。
|
||||
2. **Install Missing Custom Nodes**（不足しているカスタムノードをインストール）を選びます。
|
||||
3. 表示されたカスタムノードをインストールし、ComfyUIを再起動します。
|
||||
|
||||
> **ポイント**
|
||||
> ワークフロータブの **GenerateUIチェック**（第4章 4-7）で「変換 ×」になるのも、多くはこれが原因です。
|
||||
> バッジにマウスを重ねると、足りないノードの名前が表示されます。
|
||||
|
||||
### 生成UIで「使えない」と判定された
|
||||
| 判定 | 意味と対処 |
|
||||
| --- | --- |
|
||||
| 変換 × | 必要なカスタムノードがない → 上の「ノードが赤く表示される」を参照 |
|
||||
| 変換 △ | 手元にないモデルなどが指定されている → 生成UIで選び直す（12-3） |
|
||||
| 操作性 △／× | 生成UIに表示されない設定が多い → 生成はできることが多い。細かい設定はComfyUIのノード画面で変更する |
|
||||
|
||||
---
|
||||
|
||||
## 12-3 生成のトラブル
|
||||
|
||||
### 設定を変えたのに、結果が変わらない
|
||||
いちばん多い原因は、**Apply の押し忘れ** です。
|
||||
生成UIでは、モデルやSettings（ステップ数・サイズなど）の値を変えたあと、**Apply** を押さないとワークフローに反映されません（第5章 5-3）。
|
||||
ただし、Settingsタブの右端のスイッチの列は、クリックした時点で反映されます（第5章 5-4）。
|
||||
右側の **Raw JSON** を見ると、実際に反映されている値を確認できます。
|
||||
|
||||
### 読み込んだワークフローとは違う内容で生成される
|
||||
**Batch** サブタブで、バッチの種類のチェックがオンのまま残っていると、
|
||||
Generateを押したときにバッチ生成が実行されます（第5章 5-5）。
|
||||
Batchサブタブを開き、Batch Queueの列のチェックをすべてオフにしてください。
|
||||
|
||||
### 思っていたのと違うモデルで生成される
|
||||
ワークフローに保存されているモデルが手元にないと、別のモデルに自動で差し替えられることがあります。
|
||||
その場合は「Checkpoint「…」が見つからず、無警告で別のモデルに差し替えられました」と表示されます。
|
||||
（v0.8.7〜は、フォルダ名だけが違う同名のモデルが1つだけ手元にあれば、それに自動で差し替わります）
|
||||
生成UIの **Model** サブタブで、使いたいモデルを選び直して **Apply** を押してください。
|
||||
|
||||
いつも同じモデルを使いたい場合は、設定タブの **デフォルトCheckpoint** を使うと便利です（第11章 11-5）。
|
||||
|
||||
### プロンプト翻訳の🌐ボタンが出ない・翻訳できない
|
||||
| 症状 | 原因 | 対処 |
|
||||
| --- | --- | --- |
|
||||
| 🌐ボタンが表示されない | プロンプト翻訳がオフになっている | 設定タブの **プロンプト翻訳（自国語 → 英語）** で **生成UIでプロンプト翻訳を使う** にチェックを入れる（第5章 5-10） |
|
||||
| 「翻訳エラー」と出る | Ollamaが起動していない、または翻訳モデルがOllamaに無い | Ollamaを起動する。設定の **翻訳モデル** に **（未取得）** と出ていれば **ダウンロード** を押す |
|
||||
| 押しても何も変わらない（「翻訳する部分がありません」） | 英語（英数字）だけのプロンプトになっている | 翻訳されるのは英数字以外の文字を含む部分だけ。英語の部分はそのまま残る |
|
||||
| 自動翻訳がオンのとき、生成が始まらない | 翻訳に失敗したので、日本語のまま生成しないよう止めた | 上の「翻訳エラー」と同じ対処をする。LabとVideo Planでは、理由が進行状況の欄に出る |
|
||||
| 訳がおかしい | 小さいモデル（0.8B）は、文章で意味がずれることがある | 設定の **翻訳モデル** で2B以上を選ぶ |
|
||||
|
||||
### 同じ設定なのに、同じ画像にならない
|
||||
生成パネルのシードのモードが **Random** になっていませんか？
|
||||
同じ画像を作るには、シードを **Workflow**（ワークフローのシードをそのまま使う）か **Fixed**（Seed欄の値で固定）にする必要があります。
|
||||
同じシード・同じ設定で続けて生成して新しい画像が増えないのは、ComfyUIが計算済みの結果を再利用しているためで、故障ではありません。
|
||||
ギャラリーの画像を再現する手順は、第8章 8-3を参照してください。
|
||||
|
||||
### 生成が遅い・エラーで止まる（メモリ不足）
|
||||
| 原因 | 対処 |
|
||||
| --- | --- |
|
||||
| ローカルAIがGPUメモリを使っている | AI TOOLの **モデルをアンロード** を押す。設定タブの **VRAM調整（Ollama）** を使う（第10章 10-3） |
|
||||
| 画像サイズやBatch Sizeが大きすぎる | Settingsサブタブで Width・Height・Batch Size を小さくする |
|
||||
| モデルがGPUに対して大きすぎる | より小さいモデル（GGUF版など）を使う |
|
||||
|
||||
### モデルを移動・削除したあとにエラーが出る
|
||||
ComfyUIが、モデルの古い場所を覚えていることがあります。
|
||||
ComfyUIのメニューの **Hard Reset**（キューのメニュー内）を実行するか、ComfyUIを再起動してください。
|
||||
|
||||
### 新しく入れたモデルが一覧に出てこない
|
||||
生成UIの **Refresh Models**、またはモデルタブの **更新** を押してください。
|
||||
それでも出てこない場合は、モデルを置いたフォルダが正しいか（第6章 6-1）を確認します。
|
||||
|
||||
---
|
||||
|
||||
## 12-4 AI機能のトラブル
|
||||
|
||||
### AI機能が「Disconnected」になる・つながらない
|
||||
1. Ollamaなどのソフトが起動しているか確認します。
|
||||
LM StudioやLemonadeの場合は、アプリ内で **サーバーを起動** し、モデルを読み込んでおく必要があります。
|
||||
2. 設定のアドレス（ポート番号）が、ソフト側と合っているか確認します。
|
||||
3. **接続テスト** を押して、結果を確認します。
|
||||
|
||||
### AI TOOLタブでは使えるのに、プロンプトタブでは使えない
|
||||
AIの設定は、場所によって別々に保存されています（第10章 10-2）。
|
||||
|
||||
| 場所 | 設定する場所 |
|
||||
| --- | --- |
|
||||
| AI TOOLタブ／WS LibraryのAタブ | AI TOOLタブの設定（共通） |
|
||||
| プロンプトタブ | プロンプトタブのAI Assistantの **歯車ボタン** |
|
||||
| Taggerタブ | Taggerタブの **歯車ボタン** |
|
||||
|
||||
### モデルの一覧が空になる
|
||||
AIモデルがダウンロードされていない可能性があります。
|
||||
Ollamaの場合は `ollama pull （モデル名）` でモデルをダウンロードしてから、モデル選択の **更新** を押してください。
|
||||
|
||||
### 別のパソコンのOllamaにつながらない
|
||||
安全のため、標準では同じパソコンの中のAIソフトにしかつながりません。
|
||||
環境変数 `WFS_ALLOWED_BACKEND_HOSTS` の設定が必要です（第10章 10-4）。
|
||||
|
||||
### 画像チェックの判定モデルや、OCRモデルが一覧に出てこない
|
||||
- 判定モデルの一覧には、Ollamaの **意思決定モデル** のうち、画像を読み取れるものだけが出ます。
|
||||
ふつうのチャット用モデルは出ません。Ollamaを新しい版に更新し、意思決定モデルを `ollama pull` してから **更新** を押します（第10章 10-6）。
|
||||
- OCRモデルの一覧には、名前に「ocr」を含むOllamaのモデルだけが出ます。取得後に ↻ を押します（第10章 10-8）。
|
||||
- どちらも、AI TOOLのバックエンドがOllamaならそのアドレス、そうでなければ `http://localhost:11434` のOllamaを見に行きます。
|
||||
|
||||
### 画像チェックで、何度直しても満たされない
|
||||
- 描き直しの上手さは、チャット用モデルの力しだいです。頼み方を短く具体的にするか、別のモデルを試してください。
|
||||
- チェック自体もまちがえることがあります。画像を自分の目でも確かめてください。
|
||||
- 自動修正の回数を上げすぎると、生成の時間とGPUのメモリを使い続けます。最初は「0（ボタンで修正）」がおすすめです。
|
||||
|
||||
### AIがスキルの資料を読まない・チャットから画像を生成しない
|
||||
取り込んだスキルの参考資料を読む機能と、チャットからの画像生成は、
|
||||
**ツール呼び出し（Tool Calling）に対応したモデル** でないと動きません。対応しているモデルに切り替えてください。
|
||||
|
||||
---
|
||||
|
||||
## 12-5 そのほかの機能のトラブル
|
||||
|
||||
### Taggerにモデルが表示されない
|
||||
- `ComfyUI/models/tagger/（モデル名）/` フォルダに、`.onnx` ファイル（と `selected_tags.csv`）が入っているか確認します。
|
||||
- 追加のPythonパッケージ（onnxruntime）が、**ComfyUIが使っているPython** にインストールされているか確認します（第11章 11-2）。
|
||||
|
||||
### Taggerがとても遅い（GPUが使われない）
|
||||
ComfyUIのPythonに、CPU版の `onnxruntime` とGPU版の `onnxruntime-gpu` が両方入っていると、GPUが使われなくなります。
|
||||
（ほかのカスタムノードのインストールで、CPU版が入ることがあります）
|
||||
ComfyUIを停止し、ComfyUIのPythonで `pip uninstall -y onnxruntime onnxruntime-gpu` のあと `pip install onnxruntime-gpu` を実行して、ComfyUIを再起動してください（DirectMLは非対応）。
|
||||
|
||||
### PSD関連の機能がエラーになる
|
||||
追加のPythonパッケージ（psd-tools）が必要です。
|
||||
エラーメッセージに表示されるインストール用のコマンドを、ComfyUIが使っているPythonで実行してから、ComfyUIを再起動してください。
|
||||
|
||||
### セマンティック検索（Gallery）がうまく動かない
|
||||
| 症状 | 原因 | 対処 |
|
||||
| --- | --- | --- |
|
||||
| 「Ollamaに接続できません」と出る | Ollamaが起動していない、またはOllama URLが違う | Ollamaを起動する（インデックス作成中と検索時に必要）。歯車ボタンのモデル設定タブでOllama URLを確認する |
|
||||
| 「Ollamaのモデルが未取得です」と出る | EmbeddingGemma 2をダウンロードしていない | `ollama pull embeddinggemma-2:740m`（または `embeddinggemma-2:440m`）を実行する |
|
||||
| インデックス作成が「connection was forcibly closed」などでときどき失敗する | セキュリティソフト（AdGuardなど）がOllamaの通信に割り込んでいる | Ollamaをフィルタの対象から外す。失敗した画像は、次回の「新規・変更された画像のみ」で作り直される |
|
||||
| Geminiで「GEMINI_API_KEYが未設定」と出る・キーが拒否される | `.env` にキーが無い、または行頭に `#` が付いている | プラグインの `.env`（`.env.example` をコピー）に `GEMINI_API_KEY=キー` と書き、ComfyUIを再起動する。`.env` は起動時にしか読み込まれない |
|
||||
| 検索やSimilarの結果が出ない | 選んだフォルダにインデックス済みの画像が無い | 歯車ボタンでインデックスを作る。緑の点があるフォルダを選ぶ。モデルやベクトル次元を変えたときは「全件再作成」 |
|
||||
|
||||
### TagSpaceの空間が作れない
|
||||
| 症状 | 原因 | 対処 |
|
||||
| --- | --- | --- |
|
||||
| 「OllamaのEmbeddingGemma 2が必要です」と出る | Ollamaが起動していない、またはモデルが無い | セマンティック検索と同じ準備をする（上の表） |
|
||||
| 「Galleryのセマンティック索引は … で作られています」と出る | ギャラリーの索引をGeminiで作っている | ギャラリーのセマンティック設定でEmbeddingGemma 2を選び、「全件再作成」する |
|
||||
| 作成にとても時間がかかる | 1回目は、画像のプロンプトの読み取りと、新しいタグの準備に時間がかかる | 2回目からは速くなる。Tagger補完を使うときは、TaggerがGPUで動いているか確認する（上の「Taggerがとても遅い」） |
|
||||
|
||||
### パートナーAPI（Nano Banana）が「認証に失敗しました」と出る
|
||||
`.env` にComfy APIキーが無いのが原因です。プラグインの `.env` に `COMFY_API_KEY=キー`（platform.comfy.orgで発行）を書き、ComfyUIを再起動してください（第5章 5-8）。
|
||||
`.env` は起動時にしか読み込まれません。
|
||||
|
||||
### ComfyUIが固まる（どのタブも反応しなくなる）
|
||||
Windowsで、プレビュー画像の多いモデルタブの一覧を切り替えたあとなどに、ComfyUIが反応しなくなる（生成もできなくなる）場合があります。
|
||||
設定タブの **Windowsの安定性（イベントループ）** で、**WindowsでSelectorイベントループを使用する** をオンにして、ComfyUIを再起動してください。
|
||||
|
||||
### ギャラリーの画像の一部が、別の画像のように壊れて見える
|
||||
上の **Selectorイベントループ** をオンにしているとき、v0.9.1までは、ギャラリーのプレビューや Download で受け取った画像の一部が壊れることがありました
|
||||
（出力フォルダのファイル自体は正常です）。v0.9.2で直っています。
|
||||
Workflow Studioを更新したあと、壊れた画像がブラウザに残っていることがあるので、**Ctrl＋F5** で再読み込みしてください。
|
||||
|
||||
### ギャラリーに画像が出てこない
|
||||
- ギャラリーの **Refresh** を押します。
|
||||
- 設定タブの「ギャラリー Outputフォルダ」が、ComfyUIの画像保存先と合っているか確認します。
|
||||
|
||||
---
|
||||
|
||||
## 12-6 それでも解決しないときは
|
||||
|
||||
### Workflow Studioのヘルプを見る
|
||||
Workflow Studioの **ヘルプ** タブにも、トラブルシューティングの情報があります。
|
||||
本書の発行後に追加された情報が載っていることもあります。
|
||||
|
||||
### GitHubで質問・報告する
|
||||
解決しない問題は、Workflow StudioのGitHubリポジトリの **Issues** で報告できます。
|
||||
報告するときは、次の情報を添えると、原因が見つかりやすくなります。
|
||||
|
||||
- Workflow Studioのバージョン
|
||||
- ComfyUIのバージョンと、インストール方法（ポータブル版、Desktop版、Stability Matrixなど）
|
||||
- どの画面で、何をしたら、どうなったか
|
||||
- ComfyUIを起動した黒い画面（コンソール）に表示されたエラーメッセージ
|
||||
- ブラウザの開発者ツール（F12キー）の「Console」に表示された赤いエラーメッセージ
|
||||
|
||||
> **注意**
|
||||
> エラーメッセージを貼り付けるときは、パソコンのユーザー名などの個人情報が含まれていないか確認しましょう。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - 困ったら「再読み込み」「更新ボタン」「ComfyUIの再起動」をまず試す
|
||||
> - 赤いノードは、カスタムノードの不足。ComfyUI Managerの Install Missing Custom Nodes で解決
|
||||
> - 設定が反映されないときは、Applyの押し忘れとBatchのチェックの残りを確認
|
||||
> - メモリ不足は、ローカルAIのアンロードや画像サイズの見直しで対処
|
||||
> - 解決しないときは、ヘルプタブやGitHubのIssuesを活用する
|
||||
@@ -0,0 +1,146 @@
|
||||
# 付録
|
||||
|
||||
## 付録A 用語集
|
||||
|
||||
本書に登場する主な言葉を、五十音順・アルファベット順にまとめました。
|
||||
|
||||
### 日本語
|
||||
| 用語 | 意味 | 本書 |
|
||||
| --- | --- | --- |
|
||||
| 意思決定モデル | 質問に「はい」の確率などを数字で答えるAIモデル。画像チェックの判定に使う | 第10章 |
|
||||
| インペイント | 画像の一部（マスクした部分）だけを描き直すこと | 第11章 |
|
||||
| カスタムノード | ComfyUIに後から追加するノード・拡張機能。Workflow Studioもその1つ | 第1章 |
|
||||
| キャンバス | ComfyUIのノード画面で、ノードを並べる作業スペース | 第3章 |
|
||||
| キュー | 生成の予約リスト。順番に処理される | 第1章 |
|
||||
| キーフレーム | Labで「何回目から設定を切り替えるか」を指定したもの | 第5章 |
|
||||
| サンプラー | 画像を描くときのアルゴリズム（euler など） | 第5章 |
|
||||
| シード | 画像のもとになる乱数の種。同じなら、ほぼ同じ画像になる | 第5章 |
|
||||
| セマンティック検索 | 文章や画像の「意味」で画像を探す検索。ギャラリーのSボタンで使う | 第8章 |
|
||||
| スキル | AI TOOLのChatに、役割や答え方を教える指示書。選ぶと得意分野が切り替わる | 第10章 |
|
||||
| スケジューラー | ステップごとの描き込み量の配分のしかた | 第5章 |
|
||||
| スタイル | プロンプトに付け足す雰囲気のテンプレート | 第7章 |
|
||||
| ステップ数 | 描き込みをくり返す回数 | 第5章 |
|
||||
| トリガーワード | LoRAの効果を出すために、プロンプトに入れる言葉 | 第6章 |
|
||||
| ネガティブプロンプト | 描いてほしくないものを書くプロンプト | 第7章 |
|
||||
| ノード | ComfyUIの画面に並ぶ「箱」。1つの処理を受け持つ | 第1章 |
|
||||
| ノードセット | 複数のノードとつながりを、ひとまとまりで保存したもの | 第9章 |
|
||||
| バッジ | ワークフローやモデルに付ける、色付きのラベル | 第4章 |
|
||||
| バッチ生成 | 条件を変えながら、まとめて自動で生成すること | 第5章 |
|
||||
| ハッシュ値 | ファイルの中身から計算される「指紋」のような値。Civitaiでモデルを探すのに使う | 第6章 |
|
||||
| プリセット | ポジティブ・ネガティブのプロンプトをひと組にして保存したもの | 第7章 |
|
||||
| プロンプト | AIに「どんな画像を作ってほしいか」を伝える文章 | 第1章 |
|
||||
| ポジティブプロンプト | 描いてほしいものを書くプロンプト | 第7章 |
|
||||
| ポート番号 | 1台のパソコンの中で、通信先のソフトを区別するための番号 | 第2章 |
|
||||
| マスク | 画像の中で、処理の対象にする部分を指定したもの | 第11章 |
|
||||
| メタデータ | 画像に埋め込まれた、生成時の設定などの情報 | 第8章 |
|
||||
| モデル | 画像を作る「AIの頭脳」にあたるファイル | 第1章 |
|
||||
| ローカルAI | 自分のパソコンの中で動くAI（Ollamaなど） | 第10章 |
|
||||
| ワークフロー | ノードを線でつないだ、処理の流れ全体 | 第1章 |
|
||||
| ワイルドカード | 生成のたびに、プロンプトの一部をランダムに入れ替えるしくみ | 第7章 |
|
||||
|
||||
### アルファベット
|
||||
| 用語 | 意味 | 本書 |
|
||||
| --- | --- | --- |
|
||||
| API形式／UI形式 | ワークフローの保存形式。UI形式はノードの位置なども含み、API形式は処理の内容だけを含む | 第4章 |
|
||||
| Apply | 生成UIで、変更した値をワークフローに反映するボタン | 第5章 |
|
||||
| CFG | プロンプトにどれだけ忠実に従うかの強さ | 第5章 |
|
||||
| Checkpoint | 画像生成の本体となるモデル | 第6章 |
|
||||
| Civitai | 画像生成モデルの共有サイト | 第6章 |
|
||||
| ComfyUI Manager | カスタムノードを検索・インストールするための管理ツール | 第2章 |
|
||||
| Denoise | 元の画像をどれだけ描き変えるかの強さ | 第5章 |
|
||||
| Embedding | 特定の単語に意味を覚えさせた小さなファイル | 第6章 |
|
||||
| I2I（Image to Image） | 画像をもとに、新しい画像を作ること | 第5章 |
|
||||
| JSON | ワークフローの保存に使われるデータ形式 | 第1章 |
|
||||
| LoRA | 絵柄やキャラクターなどを追加で覚えさせる小さなモデル | 第6章 |
|
||||
| OCR | 画像に写っている文字を読み取ること | 第10章 |
|
||||
| Ollama | ローカルAIを手軽に動かせるソフト | 第10章 |
|
||||
| PLaMo 3 Translate | Preferred Networks（PFN）の翻訳専用AIモデル。プロンプト翻訳で使う | 第5章 |
|
||||
| T2I（Text to Image） | 文章（プロンプト）から画像を作ること | 第5章 |
|
||||
| TagSpace | タグ・プロンプト・生成画像を意味の近さで3Dの空間に並べて表示するタブ | 第11章 |
|
||||
| VAE | 画像の色合いや細部を仕上げるモデル | 第6章 |
|
||||
| VLM | 画像を読み取れるAIモデル | 第10章 |
|
||||
| VRAM | GPU（グラフィックボード）のメモリ | 第10章 |
|
||||
| WS Library | ComfyUIのノード画面の横に開く、Workflow Studioのサイドパネル | 第9章 |
|
||||
|
||||
---
|
||||
|
||||
## 付録B キーボード・マウス操作
|
||||
|
||||
### 全体
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| Esc | 開いている画面（モーダル・パネル）を閉じる |
|
||||
| Enter | 名前の変更を確定する／チャットを送信する |
|
||||
| Shift＋Enter | チャットで改行する |
|
||||
| ダブルクリック | 詳細画面を開く（ワークフロー・モデルなど） |
|
||||
| Shift＋クリック（Wボタン） | Workflow Studioを新しいウィンドウで開く |
|
||||
|
||||
### 生成UI
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| Alt＋クリック（Apply） | 値の反映と生成を一度に行う |
|
||||
| Ctrl＋↑／↓（プロンプト入力欄） | 選んだ言葉の強調 `(言葉:数値)` を0.05ずつ増減する |
|
||||
| Enter／Shift＋Enter（Raw JSONの検索欄） | 次の一致／前の一致へ移動する |
|
||||
|
||||
### ギャラリー
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| Ctrl＋クリック | 複数の画像を選ぶ |
|
||||
| Alt＋クリック | 画像をメタデータサブタブで開く |
|
||||
| Enter（意味検索がオンのとき） | 検索欄の文章で意味検索を実行する |
|
||||
|
||||
### WS Library
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| ドラッグ＆ドロップ | キャンバスのドロップした位置に置く・読み込む |
|
||||
| ダブルクリック | キャンバスの中央付近に置く・読み込む |
|
||||
| Alt＋クリック（Nタブ） | ノードのお気に入りをオン／オフする |
|
||||
|
||||
### Image Edit
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| V／B／T／S／G | Select／Draw／Text／Shape／Fill ツールに切り替え |
|
||||
| Ctrl＋Z／Ctrl＋Y | 元に戻す／やり直す |
|
||||
| マウスホイール | 拡大・縮小 |
|
||||
| Space＋ドラッグ | 表示位置を移動する |
|
||||
| Delete | 選んでいるレイヤーを削除する（レイヤーが2つ以上あるとき） |
|
||||
| D／E（Draw中） | 描く／消す |
|
||||
| X（Draw中） | 色1と色2を入れ替える |
|
||||
| ［／］（Draw中） | ブラシを小さく／大きくする（実際は半角の `[` `]`） |
|
||||
| B（Draw中） | ブラシの選択を開く・閉じる |
|
||||
|
||||
---
|
||||
|
||||
## 付録C 参考リンク
|
||||
|
||||
### Workflow Studio
|
||||
- GitHub（最新版・リリースノート・不具合報告）
|
||||
https://github.com/ketle-man/ComfyUI-Workflow-Studio
|
||||
|
||||
### ComfyUI本体と管理ツール
|
||||
- ComfyUI
|
||||
https://github.com/comfyanonymous/ComfyUI
|
||||
- ComfyUI Manager
|
||||
https://github.com/Comfy-Org/ComfyUI-Manager
|
||||
|
||||
### 本書で紹介した関連ソフト・サービス
|
||||
| 名前 | 用途 | 本書 |
|
||||
| --- | --- | --- |
|
||||
| Civitai（https://civitai.com/） | モデルの入手・情報取得 | 第6章 |
|
||||
| Ollama（https://ollama.com/） | ローカルAI | 第10章 |
|
||||
| PLaMo 3 Translate（https://huggingface.co/pfnet/plamo-3-translate-2b-GGUF） | プロンプト翻訳の翻訳モデル | 第5章 |
|
||||
| LM Studio（https://lmstudio.ai/） | ローカルAI | 第10章 |
|
||||
| Lemonade Server（https://lemonade-server.ai/） | ローカルAI | 第10章 |
|
||||
| Eagle（https://eagle.cool/） | 画像管理ソフト | 第8章 |
|
||||
|
||||
### 一緒に使うと便利なカスタムノード
|
||||
| 名前 | 用途 | 本書 |
|
||||
| --- | --- | --- |
|
||||
| ComfyUI Comic Creator（https://github.com/ketle-man/comfyui-comic-creator） | マンガ制作。ギャラリー・画像生成と連携 | 第11章 |
|
||||
| comfyui-image-feeder（https://github.com/ketle-man/comfyui-image-feeder） | FeederのImage Loopモード | 第5章 |
|
||||
| comfyui-mask-editor-one（https://github.com/ketle-man/comfyui-mask-editor-one） | Image Editの高度なマスク・背景除去 | 第11章 |
|
||||
| ComfyUI-Impact-Pack（https://github.com/ltdrdata/ComfyUI-Impact-Pack） | `{A｜B}` 形式のワイルドカード | 第7章 |
|
||||
| ComfyUI-Lora-Manager | LoRAの管理・WS LibraryでのLoRAのまとめ置き | 第9章 |
|
||||
|
||||
> **注意**
|
||||
> リンク先の内容やURLは、変更される場合があります。
|
||||
@@ -0,0 +1,27 @@
|
||||
# おわりに
|
||||
|
||||
最後まで読んでいただき、ありがとうございました。
|
||||
|
||||
ComfyUIは、とても自由で奥の深いソフトです。
|
||||
その自由さゆえに、はじめての方には「どこから手をつければいいのか分からない」ソフトでもあります。
|
||||
Workflow Studioは、その入口を少しでも広げ、
|
||||
「作ること」と「整理すること」をもっと気軽にしたいという思いから生まれました。
|
||||
|
||||
本書で紹介した使い方を、あらためて振り返ってみましょう。
|
||||
|
||||
1. **整理する** － ワークフロー・モデル・プロンプトを、サムネイルやバッジで見分けやすくする
|
||||
2. **作る** － 生成UIで、ノード画面を触らずに画像を生成する
|
||||
3. **貯める** － うまくいったプロンプトや設定を、プリセットやギャラリーに残す
|
||||
4. **再現する** － 画像のメタデータから、同じ設定をすぐに呼び出す
|
||||
5. **広げる** － バッチ生成やローカルAIで、試行錯誤のスピードを上げる
|
||||
|
||||
画像生成が上達するいちばんの近道は、**たくさん試して、うまくいったものを残していく** ことです。
|
||||
Workflow Studioが、その積み重ねを支える道具になれば幸いです。
|
||||
|
||||
Workflow Studioは、これからも機能の追加や改善を続けていきます。
|
||||
新しい機能や変更点は、GitHubのリリースノートでお知らせしています。
|
||||
使ってみての感想や、「こんな機能がほしい」というご要望も、GitHubでお待ちしています。
|
||||
|
||||
あなたの画像生成ライフが、もっと楽しく、もっと豊かなものになりますように。
|
||||
|
||||
ketle-man
|
||||
@@ -0,0 +1,32 @@
|
||||
# Getting Started with ComfyUI Workflow Studio
|
||||
|
||||
Don't Be Afraid of Nodes - Generate Images More Easily and Keep Them Organized
|
||||
|
||||
- Covers: Workflow Studio v0.9.9
|
||||
- Author: ketle-man
|
||||
- (c) 2026 ketle-man
|
||||
|
||||
This is the **text-only edition** (without screenshots) of the Kindle book.
|
||||
Chapter files are in the `chapters/` folder; the whole book in one file is `Workflow_Studio_Manual_v0.9.9_en.md`.
|
||||
Screen examples are omitted, so please use the book together with Workflow Studio's Help tab.
|
||||
|
||||
## Contents
|
||||
|
||||
- [00_Introduction](chapters/00_Introduction.md)
|
||||
- [01_Chapter1_Basics](chapters/01_Chapter1_Basics.md)
|
||||
- [02_Chapter2_Install_and_Launch](chapters/02_Chapter2_Install_and_Launch.md)
|
||||
- [03_Chapter3_Screen_Tour](chapters/03_Chapter3_Screen_Tour.md)
|
||||
- [04_Chapter4_Workflow_Tab](chapters/04_Chapter4_Workflow_Tab.md)
|
||||
- [05_Chapter5_GenerateUI_Part1](chapters/05_Chapter5_GenerateUI_Part1.md)
|
||||
- [05_Chapter5_GenerateUI_Part2](chapters/05_Chapter5_GenerateUI_Part2.md)
|
||||
- [06_Chapter6_Models_Tab](chapters/06_Chapter6_Models_Tab.md)
|
||||
- [07_Chapter7_Prompt_Tab](chapters/07_Chapter7_Prompt_Tab.md)
|
||||
- [08_Chapter8_Gallery_Tab](chapters/08_Chapter8_Gallery_Tab.md)
|
||||
- [09_Chapter9_WS_Library](chapters/09_Chapter9_WS_Library.md)
|
||||
- [10_Chapter10_AI_TOOL](chapters/10_Chapter10_AI_TOOL.md)
|
||||
- [11_Chapter11_More_Features](chapters/11_Chapter11_More_Features.md)
|
||||
- [12_Chapter12_Troubleshooting](chapters/12_Chapter12_Troubleshooting.md)
|
||||
- [13_Appendix](chapters/13_Appendix.md)
|
||||
- [14_Afterword](chapters/14_Afterword.md)
|
||||
|
||||
Whole book: [Workflow_Studio_Manual_v0.9.9_en.md](Workflow_Studio_Manual_v0.9.9_en.md)
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -0,0 +1,171 @@
|
||||
# Introduction
|
||||
|
||||
"I installed ComfyUI, but the node screen made me freeze."
|
||||
"I downloaded so many workflows from the web that I can't tell which is which."
|
||||
"I want to reuse the settings of an image that turned out well, but I can't find them."
|
||||
|
||||
This book is for people like that. It is an introduction to **ComfyUI-Workflow-Studio** (Workflow Studio from here on).
|
||||
|
||||
---
|
||||
|
||||
## Why is ComfyUI hard?
|
||||
|
||||
ComfyUI is one of the most flexible image-generation AI tools available today.
|
||||
When a new model or technique is announced, ComfyUI is usually the first place you can use it.
|
||||
Even so, many people stumble at the very first step. The reason is that ComfyUI is built on **a completely different way of thinking from ordinary applications**.
|
||||
|
||||
### How it differs from ordinary applications
|
||||
| | Ordinary application | ComfyUI |
|
||||
| --- | --- | --- |
|
||||
| Screen | A fixed screen designed by the developer | You assemble it yourself by connecting nodes (boxes) with lines |
|
||||
| Usage | Press buttons and menus and it works | You must know which nodes to connect, and how |
|
||||
| Adding features | Wait for an update | You add custom nodes made by other people yourself |
|
||||
| Where settings live | Collected in a settings screen | Scattered across many nodes inside the workflow |
|
||||
|
||||
An ordinary application is like a "finished appliance". Switch it on and you can use it right away, the way it was designed.
|
||||
ComfyUI, on the other hand, is "a kit you build from parts".
|
||||
If you know how to build, you can make anything. If you don't, you can make nothing.
|
||||
|
||||
### Freedom, and its side effects
|
||||
The greatest appeal of ComfyUI is this **freedom to customize**.
|
||||
You can combine custom nodes made by developers around the world to build your own flow of processing (a workflow).
|
||||
Advanced workflows made by experienced users are published all over the web, and you can try them right after downloading.
|
||||
|
||||
But freedom has side effects too.
|
||||
|
||||
- **Workflows pile up** - as you keep downloading things to try, you lose track of which is which
|
||||
- **You don't know where to touch** - in a large workflow made by someone else, even changing just the prompt means hunting for the right node
|
||||
- **Your materials get scattered** - models, LoRAs, prompts and generated images end up in separate places, with no tool to manage them
|
||||
- **Trial and error takes effort** - even to compare a few conditions, you must rewrite node values by hand every time
|
||||
|
||||
ComfyUI is excellent as a tool for "building", but
|
||||
it offers almost nothing for "organizing", "reusing" and "trying things quickly".
|
||||
Workflow Studio was made to fill that gap.
|
||||
|
||||
---
|
||||
|
||||
## What makes Workflow Studio different
|
||||
|
||||
### 1. Why a dedicated screen (SPA)? - separating "building" from "using"
|
||||
Workflow Studio is an **independent, dedicated screen** (an SPA: single-page application) that you open from the **W** button in ComfyUI's top menu.
|
||||
|
||||
Building a workflow is best done by trial and error on ComfyUI's node screen.
|
||||
Workflow Studio was created as a screen for **putting the workflows you have built to quick use**.
|
||||
|
||||
For that reason, Workflow Studio thinks of a workflow as three stages, **input, processing and output**, and is designed as follows.
|
||||
|
||||
| Stage | Design idea |
|
||||
| --- | --- |
|
||||
| Input | Make it easy to enter prompts and images |
|
||||
| Processing | Keep changing values down to the fewest possible operations |
|
||||
| Output | Make generated data easy to manage |
|
||||
|
||||
Underlying the design is the **5S** way of thinking (sort, set in order, shine, standardize, sustain) used in manufacturing.
|
||||
The aim is to keep what you need in a state where you can take it out and use it, when you need it, right away.
|
||||
|
||||
The thing that connects the two screens, ComfyUI's node screen and Workflow Studio, is the **WS Library**, which opens beside the node screen.
|
||||
Together with the supporting features, it makes moving between the two screens smooth.
|
||||
|
||||
### 2. Management - everything scattered, in one place
|
||||
| What you can manage | What you can do |
|
||||
| --- | --- |
|
||||
| Workflows | Thumbnail list, sorting by badges and groups, search, a pre-check of whether a workflow will work in GenerateUI |
|
||||
| Models and LoRAs | Lists by type, automatic retrieval of Civitai information (trigger words, sample images) |
|
||||
| Prompts | Save and sort as presets, wildcards, styles |
|
||||
| Generated images and audio | Browsing folder by folder, search by prompt or by "meaning", reading and reproducing embedded settings (audio files are supported too) |
|
||||
|
||||
### 3. Efficiency - try things quickly without touching nodes
|
||||
- **GenerateUI tab** - automatically picks out, from inside a workflow, the items you change often (prompt, model, size and so on) and turns them into an input form. Even large workflows made by others can be used without hunting for nodes
|
||||
- **Batch generation** - automatically runs your trial and error in bulk: "try all my models", "try prompts one after another", "compare samplers"
|
||||
- **Feeder and Lab** - more advanced automation, such as processing many images in order, or switching several conditions step by step
|
||||
|
||||
### 4. Support features - help before and after generation
|
||||
- **Image Edit** - simple layer-based editing, and redrawing parts with a mask (inpainting)
|
||||
- **Local AI integration** - translating prompts from Japanese, creating prompts and tags from images
|
||||
- **Tagger and Video** - automatic tag generation from images, and video generation with simple editing
|
||||
- **TagSpace** - lays out your own prompts, tags and generated images in a 3D space by closeness of meaning, so you can browse it and pick tags
|
||||
|
||||
---
|
||||
|
||||
## A custom node that is worth installing
|
||||
|
||||
For the following reasons, Workflow Studio is a custom node that is **"worth installing just in case"** for everyone who uses ComfyUI.
|
||||
|
||||
- **It is free, and you can install it right away from ComfyUI Manager**
|
||||
- **The basic features need no extra preparation** - everything required is already included with ComfyUI. Extras such as AI and image editing can be prepared when you want to use them
|
||||
- **You don't have to change how you work now** - workflows use ComfyUI's own save folder as it is. You can keep working on the node screen as before
|
||||
- **It gets more useful the more you use it** - start with just browsing a list of workflows or checking the settings of generated images, then widen your use to GenerateUI and batch generation as you get used to it
|
||||
|
||||
It keeps ComfyUI's "freedom" and adds "clarity" and "ease". That is Workflow Studio.
|
||||
|
||||
## What you will learn in this book
|
||||
|
||||
- Installing Workflow Studio and how to read its screens
|
||||
- Organizing the workflows you already have, with thumbnails
|
||||
- Generating your first image without touching the node screen
|
||||
- Managing models, LoRAs and prompts so you can reuse settings that worked
|
||||
- Reading the settings used from a generated image and reproducing them
|
||||
- Translating prompts and reading images with local AI
|
||||
- Common problems and how to fix them
|
||||
|
||||
## What this book does not cover
|
||||
|
||||
- How to install ComfyUI itself
|
||||
(the explanation starts from the point where ComfyUI is already installed and can be launched)
|
||||
- How to build a workflow from scratch by assembling nodes
|
||||
- Detailed explanations of how image-generation AI works, or of model training (such as how to make a LoRA)
|
||||
|
||||
---
|
||||
|
||||
## Who this book is for
|
||||
|
||||
- People who have just installed ComfyUI, or who want to start using it seriously
|
||||
- People who find the node screen hard and want to generate images more easily
|
||||
- People who are struggling to organize growing numbers of workflows, models and prompts
|
||||
|
||||
It is written so that you can follow it if you know basic computer operations (copying files, opening folders, using a browser).
|
||||
|
||||
---
|
||||
|
||||
## Version and environment covered
|
||||
|
||||
- **Workflow Studio**: v0.9.9
|
||||
- **ComfyUI**: the latest version at the time of writing
|
||||
- **OS**: Windows (screen examples are from Windows)
|
||||
|
||||
Workflow Studio is under active development, so
|
||||
depending on the version, the look of the screens and the positions of buttons may differ from this book.
|
||||
For features that have changed significantly, also check Workflow Studio's **Help** tab and the release notes on GitHub.
|
||||
|
||||
---
|
||||
|
||||
## Notation used in this book
|
||||
|
||||
| Notation | Meaning |
|
||||
| --- | --- |
|
||||
| **Bold** | Button names and item names shown on screen |
|
||||
| "GenerateUI" | The name as shown on screen |
|
||||
| `ComfyUI/models/` | The location of a folder or file. For how to read this for your environment, see Chapter 2, section 2-6 |
|
||||
| `Ctrl + Z` | A keyboard operation. "+" means pressing the keys at the same time |
|
||||
|
||||
> **Tip**
|
||||
> Information that is handy to know.
|
||||
|
||||
> **Caution**
|
||||
> Information to be careful about, because getting it wrong can lose data or stop things from working.
|
||||
|
||||
### About the display language
|
||||
Workflow Studio supports English, Japanese and Chinese. This English edition explains things using the **English display**.
|
||||
You can change the display language in the Settings tab (Chapter 2, section 2-5).
|
||||
|
||||
---
|
||||
|
||||
## How to read this book
|
||||
|
||||
If you are new, read **Chapters 1 to 5** in order and try things by hand.
|
||||
Once you have generated your first image in Chapter 5, you have the basics of Workflow Studio.
|
||||
|
||||
From Chapter 6 onward, feel free to read from whichever place you need.
|
||||
When in trouble, use the troubleshooting in Chapter 12 and the glossary in the appendix.
|
||||
|
||||
Now, let's start generating images with Workflow Studio, more easily and more enjoyably.
|
||||
@@ -0,0 +1,146 @@
|
||||
# Chapter 1 Basics of ComfyUI and Workflow Studio
|
||||
|
||||
This chapter sorts out the ideas and terms worth knowing before you start operating anything.
|
||||
You don't need to read it at the computer. Read it casually on your commute or in spare moments.
|
||||
|
||||
---
|
||||
|
||||
## 1-1 What is ComfyUI?
|
||||
|
||||
ComfyUI is free software for running image-generation AI on your own computer.
|
||||
It supports many image-generation models, such as Stable Diffusion and FLUX.
|
||||
|
||||
The biggest feature of ComfyUI is that **you build the flow of processing by connecting "boxes" with "lines"**.
|
||||
|
||||
"A box that loads the model" -> "a box that takes the prompt" -> "a box that makes the image" -> "a box that saves it"...
|
||||
When you connect boxes with lines like this, processing happens in that order.
|
||||
|
||||
This approach has big advantages.
|
||||
|
||||
- Everything about the processing is visible, so you can fine-tune it
|
||||
- Even when a new technique appears, you can try it just by adding a box
|
||||
- You can save a flow you have built and reuse it as many times as you like
|
||||
|
||||
On the other hand, for someone touching it for the first time, it is also a screen where "you don't know where to touch".
|
||||
**Workflow Studio**, introduced in this book, is exactly a tool for softening that "hard-to-understand" quality.
|
||||
|
||||
---
|
||||
|
||||
## 1-2 Five terms to remember
|
||||
|
||||
The minimum set of terms worth knowing for using ComfyUI is these five.
|
||||
|
||||
### 1. Node
|
||||
The "boxes" arranged on the screen.
|
||||
One node handles one job, for example "load a model" or "save an image".
|
||||
A node has entrances (inputs) and exits (outputs), and passes data along through connecting lines.
|
||||
|
||||
### 2. Workflow
|
||||
The "whole flow of processing" made by connecting nodes with lines.
|
||||
To use a cooking analogy, nodes are tasks such as "chop" and "stir-fry", and the workflow is the "recipe".
|
||||
A workflow can be saved as a file in a format called JSON.
|
||||
|
||||
> **Tip**
|
||||
> Images made in ComfyUI usually have the workflow information embedded in them.
|
||||
> If you drag such an image onto the ComfyUI screen, you can recall the same workflow.
|
||||
|
||||
### 3. Model
|
||||
The file that acts as the "AI brain" that makes images. They are typically large, from a few GB to tens of GB.
|
||||
The main kinds are as follows (Chapter 6 explains them in detail).
|
||||
|
||||
| Kind | In a phrase |
|
||||
| --- | --- |
|
||||
| Checkpoint | The model that forms the core of image generation |
|
||||
| LoRA | A small add-on model that teaches an art style or a character |
|
||||
| VAE | The model in charge of finishing the colors and fine details of an image |
|
||||
|
||||
### 4. Prompt
|
||||
The text that tells the AI "what kind of image you want". It is what people call a "spell".
|
||||
There are two kinds: the **positive prompt**, where you write what you want drawn, and
|
||||
the **negative prompt**, where you write what you do not want drawn.
|
||||
|
||||
### 5. Queue
|
||||
The "reservation list for generation".
|
||||
In ComfyUI, pressing the generate button first puts a reservation in the queue, and the reservations are processed in order.
|
||||
If you reserve many images, they are generated in bulk while you are away from your seat.
|
||||
|
||||
---
|
||||
|
||||
## 1-3 Where beginners get stuck in ComfyUI
|
||||
|
||||
You installed ComfyUI, but have you stopped at any of these points?
|
||||
|
||||
1. **Workflows pile up and you can't tell which is which**
|
||||
As you keep saving workflows you found online, file names alone stop being enough to tell them apart.
|
||||
|
||||
2. **Hunting for values on the node screen every time is tiring**
|
||||
You only want to change the prompt or seed value, yet you have to find the right node inside a large workflow.
|
||||
|
||||
3. **Models and prompts are managed in a scattered way**
|
||||
It is easy to lose track of where you put which model, or what prompt worked last time.
|
||||
|
||||
4. **You can't reproduce the settings of an image you made before**
|
||||
Even when you wonder "which settings made that image?", you may not know how to look it up.
|
||||
|
||||
None of these is really a lack of ComfyUI features. The cause is **a lack of "organization" and of an "entrance"**.
|
||||
|
||||
---
|
||||
|
||||
## 1-4 What Workflow Studio can do
|
||||
|
||||
Workflow Studio is an **extension (a custom node)** that you add to ComfyUI.
|
||||
It adds "a dedicated screen for management and generation" inside ComfyUI.
|
||||
|
||||
Workflow Studio answers the four sticking points above as follows.
|
||||
|
||||
| Sticking point | Workflow Studio's solution | Chapter |
|
||||
| --- | --- | --- |
|
||||
| Can't tell workflows apart | A thumbnail list, badges and groups, search | Chapter 4 |
|
||||
| Hunting for values on the node screen is tiring | The GenerateUI tab shows only the items you need as a form | Chapter 5 |
|
||||
| Models and prompts are scattered | Managed together in the Models tab and the Prompt tab | Chapters 6 and 7 |
|
||||
| Can't reproduce settings | Check an image's metadata in the Gallery and send it to the canvas | Chapter 8 |
|
||||
|
||||
In addition, a side panel called the **WS Library** is added beside ComfyUI's node screen,
|
||||
letting you place models and prompts onto the node screen by drag and drop (Chapter 9).
|
||||
|
||||
If you prepare AI that runs locally (such as Ollama or LM Studio),
|
||||
you can also use AI support such as prompt translation and tag generation from images (Chapter 10).
|
||||
|
||||
> **Tip**
|
||||
> Workflow Studio is not software that replaces ComfyUI.
|
||||
> It is ComfyUI that makes the images; Workflow Studio plays the role of its "control panel" and "storage shelf".
|
||||
> For that reason, it cannot be used unless ComfyUI is running.
|
||||
|
||||
### The screens can be used in your language
|
||||
Workflow Studio supports English, Japanese and Chinese.
|
||||
This edition explains things using the English display.
|
||||
|
||||
---
|
||||
|
||||
## 1-5 How this book proceeds
|
||||
|
||||
This book is meant to be read in the following order.
|
||||
|
||||
1. **Prepare** (Chapters 2 and 3)
|
||||
Install it and learn how to read the screens.
|
||||
|
||||
2. **Organize** (Chapter 4)
|
||||
Make the workflows you already have viewable as a list.
|
||||
|
||||
3. **Create** (Chapter 5)
|
||||
Generate your first image without touching the node screen.
|
||||
If you read this far, you can use the basics of Workflow Studio.
|
||||
|
||||
4. **Collect and review** (Chapters 6 to 8)
|
||||
Manage models, prompts and generated images, and reuse settings that worked.
|
||||
|
||||
5. **Expand** (Chapters 9 to 11)
|
||||
Bring in handy features such as the side panel and AI support, as you need them.
|
||||
|
||||
When in trouble, see the troubleshooting in Chapter 12 and the glossary in the appendix.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - ComfyUI is software that makes images with a workflow of connected nodes (boxes)
|
||||
> - The five terms to remember are "node", "workflow", "model", "prompt" and "queue"
|
||||
> - Workflow Studio is an extension that adds a "storage shelf" and an "easy control panel" to ComfyUI
|
||||
> - First, aim for the "first image" in Chapter 5
|
||||
@@ -0,0 +1,254 @@
|
||||
# Chapter 2 Installation and Launch
|
||||
|
||||
In this chapter we add Workflow Studio to ComfyUI and go as far as opening its screen for the first time.
|
||||
It assumes that ComfyUI itself is already installed and can be launched.
|
||||
|
||||
---
|
||||
|
||||
## 2-1 What you need
|
||||
|
||||
### Required
|
||||
| What you need | Description |
|
||||
| --- | --- |
|
||||
| ComfyUI | A recent version is fine. Because it uses a feature that places buttons in the top menu, a fairly new version (frontend v1.33.9 or later) is recommended |
|
||||
| Python 3.10 or later | The one bundled with ComfyUI is fine |
|
||||
| A web browser | Such as Chrome or Edge: the browser in which you open the ComfyUI screen |
|
||||
|
||||
The basic features of Workflow Studio work with no extra preparation as long as ComfyUI is running.
|
||||
|
||||
### Later, as needed
|
||||
The following are needed only when you use particular features. You can ignore them at first.
|
||||
|
||||
- **Ollama, LM Studio and similar** - AI support features such as prompt translation and AI chat (Chapter 10).
|
||||
**Ollama** is also needed for local semantic search in the Gallery (Chapter 8) and for the TagSpace tab (Chapter 11)
|
||||
- **Eagle** - a feature that automatically saves generated images to the image-management software Eagle (Chapter 8)
|
||||
- **Extra Python packages** - Tagger (tag generation from images), PSD export and the like (Chapter 11)
|
||||
|
||||
> **Tip**
|
||||
> It is not the case that "you can't use it unless you have everything".
|
||||
> Start by installing only the main part, and add things when features you want to use come up.
|
||||
|
||||
---
|
||||
|
||||
## 2-2 Install with ComfyUI Manager (recommended)
|
||||
|
||||
**ComfyUI Manager** is a management tool that lets you search for and install custom nodes (extensions) from the screen.
|
||||
Recent versions of ComfyUI often include it from the start, and it is the easiest method.
|
||||
|
||||
1. Start ComfyUI and open its screen in the browser.
|
||||
2. Click the **"Manager"** button in the top menu.
|
||||
|
||||
3. Click **"Custom Nodes Manager"**.
|
||||
4. Type **`Workflow Studio`** in the search box.
|
||||
5. Click **"Install"** on **ComfyUI-Workflow-Studio** in the list.
|
||||
|
||||
6. When the installation finishes, **restart ComfyUI** following the on-screen guidance.
|
||||
|
||||
> **Caution**
|
||||
> ComfyUI loads extensions only once, when it starts.
|
||||
> Always restart ComfyUI after installing. Reloading the browser alone will not take effect.
|
||||
|
||||
---
|
||||
|
||||
## 2-3 Install manually (git clone)
|
||||
|
||||
If ComfyUI Manager is not available, or if you are comfortable with Git, you can also install manually.
|
||||
You use a screen where you type commands (on Windows, "Command Prompt" or "Terminal").
|
||||
|
||||
1. Move to the **`custom_nodes`** folder inside the ComfyUI folder.
|
||||
|
||||
```
|
||||
cd ComfyUI/custom_nodes
|
||||
```
|
||||
|
||||
2. Type the following command to download Workflow Studio.
|
||||
|
||||
```
|
||||
git clone https://github.com/ketle-man/ComfyUI-Workflow-Studio.git
|
||||
```
|
||||
|
||||
3. If a **`ComfyUI-Workflow-Studio`** folder now exists inside `custom_nodes`, it worked.
|
||||
4. Restart ComfyUI.
|
||||
|
||||
> **Tip**
|
||||
> If the `git` command is not available, you first need to install Git (for example Git for Windows).
|
||||
> If you are unsure, the ComfyUI Manager method in 2-2 is recommended.
|
||||
|
||||
---
|
||||
|
||||
## 2-4 Launching
|
||||
|
||||
### Open it from the W button
|
||||
When you restart ComfyUI, a **"W"** button has been added to the menu at the top of the screen.
|
||||
This is the button that opens Workflow Studio.
|
||||
|
||||
- **Click** - Workflow Studio opens in a new tab.
|
||||
- **Shift + click** - It opens in a new window. Handy when you want to use it side by side with the ComfyUI screen.
|
||||
|
||||
A **camera button** has also been added near the W button.
|
||||
It is the button for saving a workflow screen as a thumbnail (you use it in Chapter 4).
|
||||
|
||||
### How addresses and port numbers work
|
||||
When you press the W button, the address of Workflow Studio appears in the browser's address bar.
|
||||
Knowing how to read this address lets you open it directly from a bookmark and track down causes when something goes wrong.
|
||||
|
||||
The address is made of the following three parts.
|
||||
|
||||
| Part | Example | Meaning |
|
||||
| --- | --- | --- |
|
||||
| Host | `127.0.0.1` | Which computer to connect to. `127.0.0.1` (or `localhost`) is a special address meaning "this computer" |
|
||||
| Port number | `8188` | Which piece of software on that computer to connect to |
|
||||
| Path | `/wfm` | Which page inside that software to open |
|
||||
|
||||
Joining these three in the form "`host:port-number/path`" gives the address of Workflow Studio.
|
||||
|
||||
#### What is a port number?
|
||||
Many pieces of software on one computer communicate at the same time.
|
||||
The **port number** is what is used to distinguish which software a communication is meant for.
|
||||
|
||||
To use an apartment building as an analogy, the host (`127.0.0.1`) is the "address of the building" and the port number is the "room number".
|
||||
Even in the same building, if you get the room number wrong, you will not reach the person you want.
|
||||
|
||||
ComfyUI listens on a port number decided when it starts.
|
||||
Commonly used numbers are as follows.
|
||||
|
||||
| Kind of ComfyUI / how it was started | Commonly used port number |
|
||||
| --- | --- |
|
||||
| Portable version or manual install (default settings) | 8188 |
|
||||
| ComfyUI Desktop (the desktop app version) | 8000 |
|
||||
| When the `--port` startup option is given | The number you specified |
|
||||
|
||||
If you use a launcher such as Stability Matrix, the number may also change depending on the launcher's settings.
|
||||
|
||||
#### Checking your own port number
|
||||
The most reliable way is to **look at the address bar of the browser in which you have ComfyUI open right now**.
|
||||
The number after `127.0.0.1:` or `localhost:` is the port number.
|
||||
|
||||
The black screen (console) shown when you start ComfyUI also displays a line such as
|
||||
`To see the GUI go to: http://127.0.0.1:XXXX`.
|
||||
|
||||
#### Opening it by typing the address directly
|
||||
If you know the port number, you can open Workflow Studio without the W button.
|
||||
Just add **`/wfm`** to the end of ComfyUI's address.
|
||||
|
||||
- ComfyUI is open at `127.0.0.1:8188` -> Workflow Studio is at `127.0.0.1:8188/wfm`
|
||||
- ComfyUI is open at `127.0.0.1:8000` -> Workflow Studio is at `127.0.0.1:8000/wfm`
|
||||
|
||||
If you use it often, bookmarking this address is handy.
|
||||
|
||||
> **Caution**
|
||||
> When you open it from a bookmark, "Send to Canvas", which sends a workflow directly to ComfyUI's canvas, cannot be used.
|
||||
> Instead, that workflow appears in ComfyUI's side panel (the WS Library), and you drag it onto the canvas yourself (Chapters 4 and 9).
|
||||
> If you use it while going back and forth with the ComfyUI screen, opening it from the W button is recommended.
|
||||
|
||||
> **Tip**
|
||||
> Workflow Studio runs as a part of ComfyUI, so it uses the same port number as ComfyUI.
|
||||
> There is no port number dedicated to Workflow Studio.
|
||||
|
||||
---
|
||||
|
||||
## 2-5 Changing the display language
|
||||
|
||||
Workflow Studio is displayed in **English** at first. You can switch it to Japanese or Chinese as follows.
|
||||
|
||||
1. Open **"Settings"** from the tabs at the top.
|
||||
2. In the **"UI Language"** field, choose the language you want.
|
||||
|
||||
3. After a notice that the language was changed, the page reloads automatically and switches to the new display language.
|
||||
|
||||
If you switch to Japanese, the tab names change as follows (some stay in English).
|
||||
|
||||
| English display | Japanese display |
|
||||
| --- | --- |
|
||||
| Workflow | ワークフロー |
|
||||
| GenerateUI | 生成UI |
|
||||
| Gallery | ギャラリー |
|
||||
| Prompt | プロンプト |
|
||||
| Models | モデル |
|
||||
| Nodes | ノード |
|
||||
| Settings | 設定 |
|
||||
| Help | ヘルプ |
|
||||
| AI TOOL / Tagger / Image Edit / Video | (stay in English) |
|
||||
|
||||
> **Tip**
|
||||
> The language setting is saved **per browser** you use.
|
||||
> If you open it from another browser or another computer, set it again.
|
||||
|
||||
The **"Summary Language"** field in the same place is the language used when AI creates descriptions of workflows (Chapter 10). You do not need to change it for now.
|
||||
|
||||
---
|
||||
|
||||
## 2-6 Folder locations in each environment
|
||||
|
||||
There are several ways to install ComfyUI. Depending on how it was installed, the folder locations differ slightly.
|
||||
In this book, folder locations are written in a form **starting from `ComfyUI/`**.
|
||||
Read it as the "place that corresponds to `ComfyUI/`" in the table below, to match your environment.
|
||||
|
||||
### How this book writes it, and where it is in each environment
|
||||
| Notation in this book | Contents | Portable version | ComfyUI Desktop | Stability Matrix |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `ComfyUI/` | ComfyUI's main folder | `ComfyUI_windows_portable/ComfyUI/` | The folder you chose at install time (by default `ComfyUI` inside "Documents") | `(data folder)/Packages/ComfyUI/` |
|
||||
| `ComfyUI/custom_nodes/` | Custom nodes (Workflow Studio too) | `ComfyUI/custom_nodes/` | `ComfyUI/custom_nodes/` | `ComfyUI/custom_nodes/` |
|
||||
| `ComfyUI/models/` | Models | `ComfyUI/models/` | `ComfyUI/models/` | `(data folder)/Models/` (shared by all packages) |
|
||||
| `ComfyUI/output/` | Generated images | `ComfyUI/output/` | `ComfyUI/output/` | `(data folder)/Images/` |
|
||||
| `ComfyUI/user/default/workflows/` | Saved workflows | `ComfyUI/user/default/workflows/` | `ComfyUI/user/default/workflows/` | `ComfyUI/user/default/workflows/` |
|
||||
| ComfyUI's Python | Where extra packages are installed | `ComfyUI_windows_portable/python_embeded/python.exe` | `ComfyUI/.venv/Scripts/python.exe` | `ComfyUI/venv/Scripts/python.exe` |
|
||||
| Default port number | See 2-4 | 8188 | 8000 | 8188 |
|
||||
|
||||
> **Tip**
|
||||
> `ComfyUI/` in the table refers to the "place that corresponds to `ComfyUI/`" in each environment.
|
||||
> On Windows, path separators are shown as `\` instead of `/`.
|
||||
|
||||
### Portable version
|
||||
Everything is inside the `ComfyUI_windows_portable` folder created by extracting the downloaded ZIP file.
|
||||
You launch it with `run_nvidia_gpu.bat` or similar inside the folder.
|
||||
For **Python**, you use the one inside the `python_embeded` folder dedicated to ComfyUI (Chapter 11, section 11-2).
|
||||
|
||||
### ComfyUI Desktop
|
||||
This is the app version installed with an installer. At first launch, you choose the folder in which to put data.
|
||||
If you forget where the folder is, you can check it with the app's menu item that opens the folder ("Open Folder" or similar).
|
||||
The default port number is **8000**, so be careful when typing the address directly (2-4).
|
||||
|
||||
### Stability Matrix
|
||||
This is a launcher that manages several image-generation programs together.
|
||||
Inside the **data folder** you chose first (for example `StabilityMatrix/Data/`), things are separated as follows.
|
||||
|
||||
- **ComfyUI itself** - `Data/Packages/ComfyUI/`
|
||||
- **Models** - folders by type inside `Data/Models/` (`StableDiffusion` for Checkpoints, `Lora` for LoRAs, and so on). They are shared with other programs
|
||||
- **Generated images** - inside `Data/Images/`
|
||||
|
||||
> **Caution**
|
||||
> In Stability Matrix, the folders for models and images are **outside ComfyUI's main folder**.
|
||||
> Where this book says "put it in `ComfyUI/models/checkpoints`", put it in Stability Matrix's shared model folder (such as `Data/Models/StableDiffusion`).
|
||||
> If images do not appear in the Gallery, check that the "Gallery Output Folder" in the Settings tab points inside `Data/Images/`.
|
||||
|
||||
> **Tip**
|
||||
> Folder names and structure can change with each program's version updates.
|
||||
> If you cannot find something, also check each program's official documentation.
|
||||
|
||||
---
|
||||
|
||||
## 2-7 If it does not start properly
|
||||
|
||||
### The W button does not appear
|
||||
- **Did you restart ComfyUI?** Reloading the browser alone does not take effect.
|
||||
- **Is the install location correct?** Check that the folder exists at `ComfyUI/custom_nodes/ComfyUI-Workflow-Studio`.
|
||||
If there is another folder with the same name inside it (a doubled folder), it will not be loaded.
|
||||
- **Is your ComfyUI version old?** Try updating the main program from ComfyUI Manager or similar.
|
||||
|
||||
### The page does not open, or "cannot connect" is shown
|
||||
- **Is ComfyUI running?** Workflow Studio runs inside ComfyUI, so you cannot open it if you quit ComfyUI.
|
||||
- **Is the port number right?** Check ComfyUI's port number using the method in 2-4.
|
||||
For example, if you use ComfyUI Desktop but type `8188`, it will not connect.
|
||||
|
||||
### An error appears in the console at startup
|
||||
Check whether the black screen shown when you start ComfyUI displays any error about `ComfyUI-Workflow-Studio`.
|
||||
Noting the content of the error will help with the troubleshooting in Chapter 12, and when asking questions on GitHub.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - The easy way to install is to search for "Workflow Studio" in ComfyUI Manager
|
||||
> - Always restart ComfyUI after installing
|
||||
> - Open it with the "W" button in the top menu (Shift + click for a separate window)
|
||||
> - The address is "host:port-number/wfm". You can check the port number in the address bar where ComfyUI is open
|
||||
> - Switch the language in the Settings tab with "UI Language"
|
||||
> - Folder locations differ by environment (portable version, Desktop, Stability Matrix). Read `ComfyUI/` in this book accordingly
|
||||
@@ -0,0 +1,146 @@
|
||||
# Chapter 3 Reading the Screens
|
||||
|
||||
Workflow Studio has many tabs, but you do not need to memorize all of them from the start.
|
||||
In this chapter you will get a rough idea of "what each tab is for",
|
||||
and check the roles of the buttons and the side panel added to the ComfyUI side.
|
||||
|
||||
---
|
||||
|
||||
## 3-1 The tabs and their roles
|
||||
|
||||
When you open Workflow Studio, tabs are lined up in a row across the top of the screen.
|
||||
|
||||
From the left, there are the following tabs.
|
||||
It is enough to learn the tabs marked with a star under "Often used" first.
|
||||
|
||||
| Tab | What it is for | Often used | Chapter in this book |
|
||||
| --- | --- | :---: | --- |
|
||||
| Workflow | Listing and organizing saved workflows | * | Chapter 4 |
|
||||
| Nodes | Listing nodes and registering favorites | | Chapter 11 |
|
||||
| Models | Listing and organizing models such as Checkpoints and LoRAs | * | Chapter 6 |
|
||||
| GenerateUI | Generating images without touching the node screen | * | Chapter 5 |
|
||||
| Prompt | Saving and sorting prompts, AI assistance | * | Chapter 7 |
|
||||
| Gallery | Browsing generated images, checking their settings | * | Chapter 8 |
|
||||
| Image Edit | Simple image editing, making masks | | Chapter 11 |
|
||||
| Video | Video generation and video editing | | Chapter 11 |
|
||||
| Tagger | Automatically generating tags (prompts) from images | | Chapter 11 |
|
||||
| TagSpace | Showing tags, prompts and generated images in 3D by closeness of meaning, and picking tags | | Chapter 11 |
|
||||
| Settings | Language, theme, folders and other settings | | This chapter, Chapter 11 |
|
||||
| Help | Feature list, shortcuts, troubleshooting | | This chapter |
|
||||
| AI TOOL | Translation, chat, image analysis and text reading (OCR) with local AI | | Chapter 10 |
|
||||
|
||||
### Think of the tabs in three groups: "organize", "create" and "support"
|
||||
If the number of tabs makes you lose your way, it helps to think of them in these three groups.
|
||||
|
||||
- **Organize** - Workflow / Nodes / Models / Prompt / Gallery
|
||||
Tabs that list your materials so they are easy to find.
|
||||
- **Create** - GenerateUI / Image Edit / Video
|
||||
Tabs for actually making images and videos, or touching them up.
|
||||
- **Support** - AI TOOL / Tagger / TagSpace / Settings / Help
|
||||
Tabs that help with work in other tabs, or hold overall settings.
|
||||
|
||||
> **Tip**
|
||||
> The tabs are connected to each other.
|
||||
> For example, you can load a workflow picked in the Workflow tab into GenerateUI,
|
||||
> or recall a prompt saved in the Prompt tab from GenerateUI.
|
||||
|
||||
### The Help tab
|
||||
The first thing to open when in trouble is the **Help** tab.
|
||||
A list of topics is lined up on the left, and clicking one shows the explanation on the right.
|
||||
|
||||
- The feature list (an overview of each tab's features)
|
||||
- Keyboard shortcuts
|
||||
- Troubleshooting
|
||||
- Support (links to GitHub and so on)
|
||||
|
||||
It is also useful when, reading this book, you wonder "what was this button?".
|
||||
|
||||
---
|
||||
|
||||
## 3-2 Buttons added to the ComfyUI side
|
||||
|
||||
When you install Workflow Studio, **three buttons** are added to the menu at the top of ComfyUI's screen.
|
||||
|
||||
| Button | Name | Role |
|
||||
| --- | --- | --- |
|
||||
| 1. W | Open Workflow Studio | Opens Workflow Studio in a new tab. Shift + click opens a new window (Chapter 2) |
|
||||
| 2. Camera | Canvas snapshot | Saves the current node screen as an image and makes it the workflow's thumbnail (Chapter 4) |
|
||||
| 3. An icon of lined-up squares | WS Library | Opens and closes a side panel beside the node screen (3-3) |
|
||||
|
||||
When you hover the mouse over a button, a description is shown.
|
||||
|
||||
> **Tip**
|
||||
> The main part of Workflow Studio (the screen with the row of tabs) and ComfyUI's node screen open in **separate browser tabs**.
|
||||
> The basic style is to use the two screens back and forth: "organizing and generating in Workflow Studio",
|
||||
> "detailed building on ComfyUI's node screen".
|
||||
|
||||
---
|
||||
|
||||
## 3-3 The side panel "WS Library"
|
||||
|
||||
When you press button 3, a panel called the **WS Library (Workflow Studio Library)** opens beside ComfyUI's node screen.
|
||||
|
||||
The WS Library is **a shelf for using the materials you organized in Workflow Studio right away on the node screen**.
|
||||
Single-letter tabs are lined up at the top of the panel (hover the mouse over one to see its proper name).
|
||||
|
||||
| Tab | Name | What you can do |
|
||||
| --- | --- | --- |
|
||||
| W | Workflows | Drag a workflow to load it onto the canvas |
|
||||
| N | Nodes | Place favorite nodes and node sets on the canvas |
|
||||
| P | Prompts | Place or copy saved prompts onto the canvas |
|
||||
| M | Models | Drag a model to place it onto the canvas together with a loader node |
|
||||
| I | Information | Drop an image or workflow file to inspect its contents (the models and prompts used) |
|
||||
| A | AI TOOL | Use AI features such as translation and chat while staying on the node screen |
|
||||
|
||||
For example, when you drag a model from the M tab onto the canvas, a node that loads that model is placed automatically.
|
||||
It saves the trouble of finding a node in a list, placing it, choosing the file name, and so on.
|
||||
|
||||
Chapter 9 explains how to use it in detail. For now, it is enough to remember only that "you can bring out a shelf of materials beside the node screen".
|
||||
|
||||
---
|
||||
|
||||
## 3-4 Theme and appearance settings
|
||||
|
||||
The look of Workflow Studio can be changed to your liking in the **Settings** tab.
|
||||
Because it is a tool you use for long periods, a color scheme that is easy on the eyes and a readable text size are recommended.
|
||||
|
||||
### Theme
|
||||
From the **"Theme"** field you can choose the color scheme of the whole screen. The default is the dark scheme "Deep Ocean Dark".
|
||||
|
||||
| Example theme | Feel |
|
||||
| --- | --- |
|
||||
| Deep Ocean Dark (default) | Dark blue. Images are easy to see |
|
||||
| Light Minimalist (clean) | White-based, bright and simple |
|
||||
| Pastel Dream (pastel) | Soft pastel tones |
|
||||
| Corporate Trust (for business) | Calm white and blue |
|
||||
|
||||
Besides these, there are 13 themes in all, including pop, cyberpunk and retro.
|
||||
|
||||
> **Tip**
|
||||
> When checking the colors of generated images, a darker theme makes colors easier to tell apart.
|
||||
> If you can't decide, try using the default for now.
|
||||
|
||||
### Customize
|
||||
Opening **"Customize"** lets you fine-tune the chosen theme further.
|
||||
|
||||
- **Color Override** - change colors such as background, text and accent individually
|
||||
- **Background Pattern** - add patterns such as stripes or dots to the background
|
||||
- **Font** - change the typeface (Japanese fonts such as Zen Maru Gothic and BIZ UDPGothic can also be chosen)
|
||||
|
||||
When you finish adjusting, press **"Apply & Save"**.
|
||||
To go back, press **"Reset Customization"**.
|
||||
|
||||
### Text Size
|
||||
With the **"Text Size"** field you can change the size of the text across the whole screen.
|
||||
If the text feels too small to read, make it bigger here.
|
||||
|
||||
### The look of the WS Library
|
||||
The colors of the WS Library (3-3) can be set separately from the **gear button** inside the panel.
|
||||
You can adjust the background and text colors to match the colors of the ComfyUI screen.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - Thinking of the tabs in three groups, "organize", "create" and "support", makes them easy to follow
|
||||
> - For now, it is enough to learn five: Workflow, Models, GenerateUI, Prompt and Gallery
|
||||
> - Three buttons, "W", "camera" and "WS Library", are added to the top of ComfyUI
|
||||
> - The WS Library is a shelf of materials you can bring out beside the node screen
|
||||
> - The look can be adjusted with the Settings tab's Theme, Customize and Text Size
|
||||
@@ -0,0 +1,306 @@
|
||||
# Chapter 4 The Workflow Tab - Organizing the Workflows You Made
|
||||
|
||||
The Workflow tab is the first thing shown when you open Workflow Studio, a kind of "front door".
|
||||
In this chapter you will learn how to make the workflows you already have viewable as a list,
|
||||
and how to organize them so they are easy to find later.
|
||||
|
||||
---
|
||||
|
||||
## 4-1 How workflows are listed automatically
|
||||
|
||||
When you open the Workflow tab, some workflows may already be lined up.
|
||||
This is because Workflow Studio automatically reads ComfyUI's workflow save folder.
|
||||
|
||||
A workflow you "Save" on ComfyUI's node screen is normally saved in the following folder.
|
||||
|
||||
```
|
||||
ComfyUI/user/default/workflows/
|
||||
```
|
||||
|
||||
Workflow Studio lists the workflows inside this folder.
|
||||
In other words, **workflows saved in ComfyUI line up in the Workflow tab without your doing anything**.
|
||||
|
||||
> **Tip**
|
||||
> If a workflow you just saved in ComfyUI does not appear in the list,
|
||||
> press the **"Refresh"** button on the toolbar. It reloads to the latest state.
|
||||
|
||||
> **Tip**
|
||||
> The folder that is read can be changed with the **"Workflow Data Folder"** setting in the Settings tab.
|
||||
> Unless you have a reason, leaving it as it is causes no problem.
|
||||
|
||||
---
|
||||
|
||||
## 4-2 Reading the screen and switching the display
|
||||
|
||||
The Workflow tab is divided into three main parts.
|
||||
|
||||
| No. | Place | Role |
|
||||
| --- | --- | --- |
|
||||
| 1 | Toolbar (top) | Search, filtering, sorting, various buttons |
|
||||
| 2 | List (center) | Workflows lined up as cards or in a table |
|
||||
| 3 | Side panel (right) | Shows the thumbnail and contents of the selected workflow |
|
||||
|
||||
### Thumbnail view and table view
|
||||
With the switch buttons on the toolbar you can change how the list is displayed.
|
||||
|
||||
- **Thumbnail** - lines them up as cards with images. Handy when you want to find things by how they look.
|
||||
- **Table** - lines them up in a table. Handy when you want to find one among many workflows by name.
|
||||
|
||||
### Click and double-click
|
||||
Workflows in the list behave differently depending on how you click.
|
||||
|
||||
| Operation | What happens |
|
||||
| --- | --- |
|
||||
| Click | Shows the contents in the right-hand side panel (the workflow becomes "selected") |
|
||||
| Double-click | Opens the detail screen (editing badges, tags and memo; explained in 4-4) |
|
||||
|
||||
### The side panel
|
||||
When you click a workflow, the following three tabs appear in the right-hand side panel.
|
||||
|
||||
- **Thumbnail** - shows the workflow's image large
|
||||
- **JSON** - shows the contents (JSON data) of the workflow with color coding. The "Copy" button copies it all
|
||||
- **Groups** - sorts the workflow into groups (explained in 4-4)
|
||||
|
||||
Below the workflow name, the **format of the workflow** (UI format, API format and so on) and
|
||||
the **check result** of whether it can be used in GenerateUI are shown (explained in 4-7).
|
||||
|
||||
---
|
||||
|
||||
## 4-3 Taking a thumbnail with the camera button
|
||||
|
||||
A workflow that you only saved in ComfyUI has no thumbnail image, and can be hard to tell apart in the list.
|
||||
That is where the **camera button** (canvas snapshot) added to ComfyUI's top menu comes in.
|
||||
|
||||
### How to take a thumbnail
|
||||
1. On ComfyUI's node screen, open the workflow you want to make a thumbnail of.
|
||||
2. Adjust the screen position and zoom so that the whole workflow is easy to see.
|
||||
3. Click the **camera button** in the top menu.
|
||||
|
||||
4. A screen for confirming the name to save appears. Change the name if you wish and save.
|
||||
(At first, the name of the workflow currently open is filled in)
|
||||
5. When "Workflow saved" is shown, you are done.
|
||||
|
||||
When you open the Workflow tab of Workflow Studio (or press "Refresh"),
|
||||
the workflow appears in the list with the image you took attached as its thumbnail.
|
||||
|
||||
> **Tip**
|
||||
> The image (PNG) saved with the camera button also has the workflow information embedded in it.
|
||||
> If you drag and drop this image onto ComfyUI's node screen, you can load that workflow.
|
||||
> A single image file carries both the "look" and the "contents", which is handy.
|
||||
|
||||
### Changing a thumbnail later
|
||||
To replace the thumbnail of an existing workflow,
|
||||
double-click the workflow to open the detail screen, press **"Change Thumbnail"** and choose an image.
|
||||
|
||||
---
|
||||
|
||||
## 4-4 Sorting with favorites, badges and groups
|
||||
|
||||
As workflows increase, sorting them lets you find what you want right away.
|
||||
Workflow Studio has three ways to sort.
|
||||
|
||||
| Method | When to use it | Example |
|
||||
| --- | --- | --- |
|
||||
| Favorites (star) | You want to mark the ones you use often | The basic workflow you use every day |
|
||||
| Badges | You want to tell kinds apart at a glance with colored labels | "SDXL", "FLUX", "Video", "Experimenting" |
|
||||
| Groups | You want to gather them by purpose or project | "Character work", "Background materials" |
|
||||
|
||||
### Favorites (star)
|
||||
Click the **star** on a card to register it as a favorite.
|
||||
Press the **star** button at the top of the toolbar to show only favorites (**ALL** returns to showing everything).
|
||||
|
||||
Workflows registered as favorites also appear under
|
||||
"Favorites" in ComfyUI's side panel (the WS Library) (Chapter 9).
|
||||
|
||||
### Badges
|
||||
A badge is a **colored label** you attach to a workflow.
|
||||
You can also attach several badges to one workflow.
|
||||
|
||||
**1. Make a badge**
|
||||
1. Press the **"Badge"** button on the toolbar.
|
||||
2. Type a name in the **"Badge label..."** field and press **"Add"**.
|
||||
3. Click a color swatch to change the badge's color.
|
||||
|
||||
**2. Attach a badge to a workflow**
|
||||
1. Double-click the workflow to open the detail screen.
|
||||
2. In the **"Badges"** field, tick the badges you want to attach.
|
||||
3. Press **"Save"**.
|
||||
|
||||
**3. Filter by badge**
|
||||
Once you have made badges, they line up next to the star and ALL buttons at the top of the toolbar.
|
||||
Click a badge to show only the workflows that have that badge.
|
||||
|
||||
> **Tip**
|
||||
> The kinds and colors of badges are shared with the Models tab (Chapter 6).
|
||||
> If you make a badge called "SDXL", you can attach it in the same color to both workflows and models.
|
||||
|
||||
### Groups
|
||||
A group is **a box for gathering workflows by purpose**.
|
||||
|
||||
1. Click a workflow to select it.
|
||||
2. In the right-hand side panel, open the **"Groups"** tab.
|
||||
3. Make a group with **"Create New Group"**, or pick an existing group and **"Add to Group"**.
|
||||
|
||||
If you choose a group from the pull-down labeled **"All Groups"** on the toolbar, only the workflows in that group are shown.
|
||||
|
||||
### Tags and memo
|
||||
On the detail screen you can also write the following.
|
||||
|
||||
- **Tags** - keywords for searching. You can enter several, separated by commas (,).
|
||||
- **Memo** - free notes, such as cautions on how to use it or settings that worked well.
|
||||
|
||||
If you note the "source" and the "required custom nodes" for a workflow you got online, you will not be in trouble later.
|
||||
|
||||
### Renaming and deleting
|
||||
- **Rename** - click the workflow name at the top of the detail screen to edit it.
|
||||
- **Delete** - press the **"Delete"** button on the detail screen.
|
||||
|
||||
> **Caution**
|
||||
> Deleting removes the file itself in ComfyUI's workflow save folder (including the thumbnail image).
|
||||
> For workflows that matter, back them up somewhere else before deleting.
|
||||
|
||||
---
|
||||
|
||||
## 4-5 Search, sorting and clearing filters
|
||||
|
||||
### Search
|
||||
Type a keyword in the **"Search..."** box on the toolbar, and only workflows whose name, tags and so on match are shown.
|
||||
Pressing the **x** at the right end of the box clears the search text immediately.
|
||||
|
||||
### Sorting
|
||||
With the pull-down next to the search box you can change the order.
|
||||
|
||||
| Choice | Order |
|
||||
| --- | --- |
|
||||
| Date (Newest) | Newest first |
|
||||
| Date (Oldest) | Oldest first |
|
||||
| Name A-Z | Ascending by name |
|
||||
| Name Z-A | Descending by name |
|
||||
|
||||
The order you choose is remembered, and the same order is used the next time you open it.
|
||||
|
||||
### Clearing filters all at once
|
||||
While narrowing down by search, group and badge, you may wonder "why doesn't it show up in the list?".
|
||||
In that case, press **"x Clear"** at the right end of the toolbar.
|
||||
The search text, group and badge filters are all cleared at once, and every workflow is shown.
|
||||
|
||||
---
|
||||
|
||||
## 4-6 Importing, and sending to the canvas
|
||||
|
||||
### Bringing in workflows (import)
|
||||
Workflow files you got online (`.json`, or `.png` containing a workflow)
|
||||
can be brought into the Workflow tab in either of these ways.
|
||||
|
||||
- Press the **"Import"** button on the toolbar and choose files (you can select several)
|
||||
- **Drag and drop** the files onto the list area of the Workflow tab
|
||||
|
||||
Imported workflows are saved in ComfyUI's workflow save folder and added to the list.
|
||||
|
||||
### Using the selected workflow
|
||||
When you click a workflow to select it, the following three buttons on the toolbar become pressable.
|
||||
|
||||
| Button | What happens |
|
||||
| --- | --- |
|
||||
| Load in GenerateUI | Loads it into the GenerateUI tab so you can generate without touching the node screen (Chapter 5) |
|
||||
| Load in Video | Loads a workflow for video generation into the Video tab (Chapter 11) |
|
||||
| Send to Canvas | Loads the workflow onto ComfyUI's node screen |
|
||||
|
||||
> **Tip**
|
||||
> If a button cannot be pressed (it is gray), you have not yet selected a workflow.
|
||||
> Click a workflow in the list once, then press it.
|
||||
|
||||
When you press **"Send to Canvas"**, that workflow is loaded onto the node screen of the ComfyUI from which you opened Workflow Studio.
|
||||
You can use it like this: find in Workflow Studio, edit in detail in ComfyUI.
|
||||
|
||||
> **Caution**
|
||||
> "Send to Canvas" works in Workflow Studio **opened from ComfyUI's W button**.
|
||||
> If you opened it directly, for example from a bookmark, that workflow appears in the WS Library on the ComfyUI side,
|
||||
> so drag it onto the canvas from there (Chapters 2 and 9).
|
||||
|
||||
---
|
||||
|
||||
## 4-7 Checking in advance whether it works in GenerateUI (GenerateUI Check)
|
||||
|
||||
Not every workflow can be used as it is in GenerateUI (Chapter 5).
|
||||
If it uses special custom nodes, or specifies models you do not have, it may not work well.
|
||||
|
||||
So before you load one into GenerateUI, there is a feature that automatically checks whether it looks usable.
|
||||
|
||||
### Reading the check result
|
||||
When you click a workflow, the following information appears below the workflow name in the side panel.
|
||||
|
||||
- **Format badge** - the workflow's save format (UI format / API format / App format)
|
||||
- **Convert** - whether it can be converted into a form GenerateUI can use
|
||||
- **Operability** - how many of the settings can be changed on the GenerateUI screen alone
|
||||
|
||||
Each is shown in one of three levels.
|
||||
|
||||
| Display | Meaning |
|
||||
| --- | --- |
|
||||
| Check mark | No problem |
|
||||
| Warning mark | Usable, but needs care |
|
||||
| Cross mark | Cannot be used in GenerateUI, or is hard to use |
|
||||
|
||||
Hovering the mouse over a badge shows the reason for the judgment.
|
||||
|
||||
### Common judgments and what to do
|
||||
| Judgment | Common reason | What to do |
|
||||
| --- | --- | --- |
|
||||
| Convert: cross | The custom nodes the workflow needs are not installed | Install the missing nodes with ComfyUI Manager (Chapter 12) |
|
||||
| Convert: warning | The specified model or LoRA is not on your computer | Choose a model you have in GenerateUI. For UI-format workflows, note that it is automatically replaced with the first model in the list |
|
||||
| Operability: warning / cross | Many settings are not shown in GenerateUI | Change settings of nodes that are not shown on ComfyUI's node screen |
|
||||
|
||||
> **Tip**
|
||||
> Operability is "the share of nodes that have settings which are shown in GenerateUI".
|
||||
> It is a check mark at 80% or more, a warning at 50% or more, and a cross below that.
|
||||
> Even with a warning or cross, generation itself is often possible, so treat it as a rough indication of "hard to finish using GenerateUI alone".
|
||||
|
||||
> **Tip**
|
||||
> The check fetches from ComfyUI only the information of the nodes that workflow uses (v0.8.8 and later).
|
||||
> It used to take tens of seconds the first time, but now the result appears almost instantly.
|
||||
> From the second time on, results appear immediately.
|
||||
|
||||
---
|
||||
|
||||
## 4-8 Trying the bundled sample workflows
|
||||
|
||||
Workflow Studio comes with sample workflows you can try right away.
|
||||
The samples are in the **`workflows`** folder inside the folder where you installed Workflow Studio.
|
||||
|
||||
```
|
||||
ComfyUI/custom_nodes/ComfyUI-Workflow-Studio/workflows/
|
||||
```
|
||||
|
||||
Each sample has a `.json` (the workflow itself) and a `.png` (the thumbnail image) of the same name.
|
||||
|
||||
### Samples recommended for beginners
|
||||
| File name | Contents |
|
||||
| --- | --- |
|
||||
| `ws_sdxl_t2i_basic` | A basic workflow that makes images from text with an SDXL model |
|
||||
| `ws_sd15_t2i_basic` | A basic workflow that makes images from text with an SD1.5 model |
|
||||
|
||||
Choose according to the kind of model you have (SDXL or SD1.5).
|
||||
|
||||
### Importing a sample
|
||||
1. Press **"Import"** in the Workflow tab.
|
||||
2. Open the `workflows` folder above and choose the `.png` file of the sample you want to use.
|
||||
(If you choose the `.png`, it is imported with its thumbnail)
|
||||
3. Check that it has been added to the list.
|
||||
|
||||
There are also more advanced samples, such as pose specification and inpainting (redrawing part of an image).
|
||||
|
||||
> **Caution**
|
||||
> Some samples require additional custom nodes.
|
||||
> If any nodes are shown in red on the node screen, install the missing nodes with ComfyUI Manager (Chapter 12).
|
||||
> The "Convert: cross" shown in the GenerateUI Check of 4-7 is also mostly this case.
|
||||
|
||||
In the next chapter, you load the basic sample you just imported into GenerateUI and finally generate your first image.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - Workflows saved in ComfyUI line up automatically in the Workflow tab
|
||||
> - Click to show in the side panel; double-click for the detail screen
|
||||
> - With the camera button you can give a workflow a thumbnail that is easy to recognize
|
||||
> - Sort with favorites, badges and groups, and find with search and sorting. If lost, use "x Clear"
|
||||
> - With "Load in GenerateUI" and "Send to Canvas" you can use the selected workflow right away
|
||||
> - The GenerateUI Check lets you confirm in advance whether it works in GenerateUI
|
||||
@@ -0,0 +1,255 @@
|
||||
# Chapter 5 The GenerateUI Tab - Making Images Without Touching Nodes
|
||||
|
||||
At last, you generate an image with Workflow Studio.
|
||||
The **GenerateUI** tab is a screen that picks out, from inside a workflow, the items you change often, such as "prompt", "model" and "size",
|
||||
and shows them as an input form.
|
||||
You no longer need to find nodes and follow lines.
|
||||
|
||||
In the first half of this chapter (5-1 to 5-4), you use the basic sample imported in Chapter 4 to generate your first image.
|
||||
|
||||
> **Before you start**
|
||||
> - ComfyUI is running
|
||||
> - At least one Checkpoint model is in ComfyUI's `models/checkpoints` folder
|
||||
> - You have imported a sample workflow (such as `ws_sdxl_t2i_basic`) in Chapter 4
|
||||
|
||||
---
|
||||
|
||||
## 5-1 Loading a workflow into GenerateUI
|
||||
|
||||
1. Open the **Workflow** tab.
|
||||
2. **Click** the workflow you want to use (for example `ws_sdxl_t2i_basic`) to select it.
|
||||
3. In the side panel's **GenerateUI Check** (Chapter 4, section 4-7), confirm that "Convert" shows a check mark.
|
||||
4. Press **"Load in GenerateUI"** on the toolbar.
|
||||
|
||||
The screen switches automatically to the **GenerateUI** tab, and the contents of the workflow are loaded.
|
||||
If the name of the loaded workflow is shown at the top of the screen, it worked.
|
||||
|
||||
> **Tip**
|
||||
> On the GenerateUI screen, button and item names are shown in English.
|
||||
> This book uses the displayed English names as they are.
|
||||
|
||||
---
|
||||
|
||||
## 5-2 Reading the GenerateUI screen
|
||||
|
||||
The GenerateUI tab is divided into the following main parts.
|
||||
|
||||
| No. | Place | Role |
|
||||
| --- | --- | --- |
|
||||
| 1 | Top bar | Connection state with ComfyUI, workflow name, various buttons |
|
||||
| 2 | Subtabs | Switch between Input / Model / Settings / Feeder / Batch / Lab |
|
||||
| 3 | Editing area (center) | The settings of the chosen subtab |
|
||||
| 4 | Raw JSON (right side) | Shows the contents of the workflow as JSON (for advanced users) |
|
||||
| 5 | Generate panel | The Generate button, progress, seed settings, generated results |
|
||||
|
||||
### 1. The top bar
|
||||
| Display / button | Meaning |
|
||||
| --- | --- |
|
||||
| Connected / Disconnected | Whether it is connected to ComfyUI. If it is a green **Connected**, you can generate |
|
||||
| Connect | Reconnects to ComfyUI |
|
||||
| Refresh Models | Reloads the list of models (press it after adding models to ComfyUI) |
|
||||
| Reset Workflow | Discards edits and returns to the state right after loading the workflow |
|
||||
| Workflow name | The workflow currently loaded |
|
||||
|
||||
> **Tip**
|
||||
> When you open the GenerateUI tab, it connects to ComfyUI automatically.
|
||||
> If it shows **Disconnected**, check that ComfyUI is running and press **Connect**.
|
||||
|
||||
### 2. The subtabs
|
||||
| Subtab | What you set | In this book |
|
||||
| --- | --- | --- |
|
||||
| Input | Prompts, input images | 5-3 |
|
||||
| Model | Models such as Checkpoint, VAE and LoRA | 5-3 |
|
||||
| Settings | Seed, step count, sampler, image size and so on | 5-4 |
|
||||
| Feeder | Process images in a folder in order | 5-6 |
|
||||
| Batch | Generate in bulk while changing conditions | 5-5 |
|
||||
| Lab | Experimental I2I batch generation (advanced) | 5-7 |
|
||||
|
||||
At first, it is enough to learn just the three: **Input, Model and Settings**.
|
||||
|
||||
### 4. Raw JSON
|
||||
To the right of Input, Model and Settings, the contents of the workflow are shown in JSON format.
|
||||
When you change values in the form, they are reflected here too.
|
||||
Until you are used to it, treat this as "view only".
|
||||
|
||||
On the Input subtab, the **>>** button at the top of the Raw JSON folds it into a thin strip at the right edge (v0.8.12 and later),
|
||||
which gives the prompt and image areas more room. Press it again to unfold it; the folded state is remembered.
|
||||
|
||||
---
|
||||
|
||||
## 5-3 Generating your first image
|
||||
|
||||
Now, to generate. At minimum, you check and set these three things.
|
||||
|
||||
1. Which model to draw with (Model)
|
||||
2. What to draw (Input)
|
||||
3. Generate (Generate)
|
||||
|
||||
### Step 1: Choose a model (Model)
|
||||
1. Open the **Model** subtab.
|
||||
2. In the **Checkpoint** field, choose a model you have.
|
||||
If the list is long, typing characters in the box above the field narrows it down.
|
||||
3. Press the **Apply** button below the field.
|
||||
|
||||
> **Tip: when a model cannot be found**
|
||||
> The model saved in a workflow may not be on your computer.
|
||||
> You will almost always run into this when using distributed workflows. It is signaled in one of two ways.
|
||||
> - The field shows "(model name) (not found)" with a warning mark
|
||||
> - When loading, a message says the workflow's Checkpoint "..." was not found and was replaced with another model without warning
|
||||
> (in this case it has been replaced automatically with the first model in the list you have)
|
||||
>
|
||||
> From v0.8.7, if you have exactly one model with the same file name that differs only in its folder, it is replaced with that model instead of the first one.
|
||||
> (Official templates often save model names without a folder, and the old behavior could swap in an unrelated model.)
|
||||
>
|
||||
> In either case, choose the model you want to use from the list and press Apply.
|
||||
> If choosing again each time is a nuisance, the **Default Checkpoint** in the Settings tab (Chapter 11) is handy.
|
||||
|
||||
> **Caution: Don't forget Apply**
|
||||
> In GenerateUI, after changing a model or the Settings values (5-4), you must press **Apply** for them to be reflected in the workflow.
|
||||
> If you Generate without pressing Apply, the image is generated **with the values from before the change**.
|
||||
> When "I changed a setting but the result didn't change", suspect a forgotten Apply first.
|
||||
> (Only the prompt input fields are reflected automatically when you press Generate)
|
||||
|
||||
> **Tip**
|
||||
> Item names shown in bright green are items for which the loaded workflow contains a node for that model.
|
||||
> Items that are not green have no effect on this workflow even if you change them.
|
||||
|
||||
### Step 2: Write a prompt (Input)
|
||||
1. Open the **Input** subtab.
|
||||
The prompt area (Prompt) is on the left and the input image area (Image) is on the right (v0.8.12 and later; before that you switched between Prompt and Image).
|
||||
When the screen is narrow, the Image area moves below the prompts.
|
||||
2. In the **Positive Prompt** field, enter in English what you want drawn.
|
||||
|
||||
```
|
||||
a cute cat sitting on a windowsill, sunlight, soft colors, high quality
|
||||
```
|
||||
|
||||
3. Press **Apply** for the **Positive Prompt**.
|
||||
4. If needed, enter in the **Negative Prompt** field what you do not want drawn, and press **Apply** for that too.
|
||||
|
||||
> Even if you forget Apply, the prompt is reflected automatically when you press Generate.
|
||||
> However, you can check the reflected result in the Raw JSON (right side), so it is reassuring to get into the habit of pressing Apply.
|
||||
|
||||
```
|
||||
low quality, blurry, bad anatomy
|
||||
```
|
||||
|
||||
> **Tip**
|
||||
> Pressing the **reset arrow** button next to Apply returns the contents of the field to the value currently reflected in the workflow.
|
||||
> Handy when an edit went wrong.
|
||||
|
||||
> **Tip**
|
||||
> If you are not good at English, the AI translation features introduced in Chapters 7 and 10 let you turn a prompt you thought up in your own language into English.
|
||||
|
||||
### Step 3: Generate (Generate)
|
||||
1. Press the **Generate** button in the generate panel on the screen.
|
||||
2. A bar showing progress extends, and a percentage is displayed.
|
||||
3. When "Done" is shown, the generated image appears in the generate panel.
|
||||
|
||||
Congratulations. You have completed your first image with Workflow Studio.
|
||||
|
||||
A generated image is saved in ComfyUI's `output` folder, just as when generated in ComfyUI.
|
||||
To look at it again later, you can browse it in the **Gallery** tab (Chapter 8).
|
||||
|
||||
> **Tip: when you want to stop partway**
|
||||
> During generation, a **Stop** button appears next to the Generate button. Pressing it interrupts generation.
|
||||
|
||||
> **Tip: Apply and Generate at once**
|
||||
> **Click Apply while holding the Alt key** to reflect the values and generate in one go.
|
||||
> Once you are used to it, you can try things quickly: rewrite the prompt, then Alt + Apply.
|
||||
|
||||
### Saving the workflow you generated with
|
||||
If you want to use settings changed in GenerateUI again later, press the **Save** button at the right end of the subtabs.
|
||||
If you save it with a name, it is added as a new workflow in the Workflow tab.
|
||||
|
||||
---
|
||||
|
||||
## 5-4 What seed, sampler and step count mean
|
||||
|
||||
Once the first image is out, next look at the **Settings** subtab, where settings that decide how the result turns out are found.
|
||||
|
||||
The Settings tab is divided into three columns.
|
||||
|
||||
- **KSampler (left)** - settings for how the image is drawn
|
||||
- **Latent Image (middle)** - settings for the size and number of images
|
||||
- **Switches (right, v0.9.6+)** - the on/off settings inside the workflow
|
||||
|
||||
Here too, for KSampler and Latent Image, don't forget to press **Apply** on each block after changing values (the Switches column needs no Apply).
|
||||
|
||||
### KSampler items
|
||||
| Item | Meaning | Rule of thumb for beginners |
|
||||
| --- | --- | --- |
|
||||
| Seed | The "seed of the random numbers" the image grows from. With the same settings and the same seed, you get almost the same image | When generating, the seed settings in the generate panel (below) take priority, so think of this as "for checking the seed of the loaded workflow". To use the workflow's seed as it is, set the generate panel's mode to **Workflow** |
|
||||
| Steps | How many times to repeat the drawing. More is more careful but takes longer | 20 to 30 |
|
||||
| CFG | How faithfully to follow the prompt. Too high and colors and shapes tend to break down | About 5 to 8 for SDXL |
|
||||
| Sampler | The drawing algorithm | `euler`, `dpmpp_2m` and so on |
|
||||
| Scheduler | How the drawing amount is distributed across steps | `normal`, `karras` and so on |
|
||||
| Denoise | How much to redraw the original image. 1.0 when making from text | 1.0 (0.4 to 0.7 and so on when making from an image) |
|
||||
|
||||
> **Tip**
|
||||
> The suggested values differ by model. The model's distribution page (Civitai and so on) often lists recommended values, so check it.
|
||||
> If unsure, using the values the sample workflow starts with is fine.
|
||||
|
||||
### Latent Image items
|
||||
| Item | Meaning | Rule of thumb |
|
||||
| --- | --- | --- |
|
||||
| Width | Horizontal size of the image (pixels) | Around 1024 for SDXL, around 512 for SD1.5 |
|
||||
| Height | Vertical size of the image (pixels) | Same as above |
|
||||
| Batch Size | How many images to make in one generation | 1 to 4 (more uses more memory) |
|
||||
|
||||
> **Caution**
|
||||
> If you stray far from the size the model learned, people tend to come out doubled or the composition falls apart.
|
||||
> Even when you want portrait or landscape, adjust so the total pixel count stays about the same (for example 832x1216 for SDXL).
|
||||
|
||||
### Switches items (v0.9.6+)
|
||||
The "on/off" settings inside the workflow are listed with their node name and item name.
|
||||
Clicking a checkbox switches it right away. **You don't need to press Apply**. The new value also shows up in the Raw JSON on the right at once.
|
||||
|
||||
Two kinds of items are listed:
|
||||
|
||||
- Items whose value is true/false - for example the switch that turns the prompt enhancer (PE) on and off, "Boolean" nodes, and TextGenerate's thinking
|
||||
- On/off items with only two choices - enable/disable, on/off, yes/no and so on. For example KSamplerAdvanced's add_noise and DWPreprocessor's detect_hand (whether to detect hands)
|
||||
|
||||
The right end of each row shows the current value (true/false, enable/disable and so on).
|
||||
Hover over a row to see the node's number and type.
|
||||
|
||||
> **Tip**
|
||||
> Items wired from another node are not listed. Instead, the source node they are wired from (for example a Boolean node named "Enable PE?") is listed, so switch that one.
|
||||
> In workflows where nodes have no names, the node type name (such as ComfySwitchNode) is shown.
|
||||
> If the workflow has no on/off items at all, "No true/false settings in this workflow" is shown.
|
||||
|
||||
### Managing the seed (generate panel)
|
||||
The seed is managed with the **Seed** field in the generate panel and the pull-down below it.
|
||||
|
||||
| Mode | What it does | When to use it |
|
||||
| --- | --- | --- |
|
||||
| Random | Generates with a random seed every time | When you want to try various pictures |
|
||||
| Fixed | Fixes it to the value in the Seed field | When you want to keep a picture you like and change only the prompt to compare |
|
||||
| Increment | Increases the seed by 1 each time you generate | When you want to make slightly different pictures in sequence |
|
||||
| Decrement | Decreases the seed by 1 each time you generate | The reverse of Increment |
|
||||
| Workflow | Uses the seed in the workflow as it is, without overriding it | When you want to reproduce, with the original seed, an image loaded from the Gallery or a distributed workflow |
|
||||
|
||||
When generation finishes, the Seed field shows **the seed value actually used**.
|
||||
|
||||
> **Tip: Workflow mode (v0.8.3 and later)**
|
||||
> If you set the mode to **Workflow**, it generates with the loaded workflow's own seed, not the Seed field's value or a random one.
|
||||
> Even after trying several images with Random, it returns to the workflow's original seed (the value at the time of loading) when generating.
|
||||
> If no seed input can be found in the workflow, the value in the Seed field is used (random if it is -1).
|
||||
> If you press it twice in a row with the same seed and the same settings, ComfyUI reuses the already computed result, so no new image is added (this is not a malfunction).
|
||||
|
||||
### The flow of "growing" a picture you like
|
||||
Using the seed, you can improve a picture you like a little at a time.
|
||||
|
||||
1. Generate several with **Random** and look for a composition you like
|
||||
2. When a picture you like appears, switch the mode to **Fixed** (the Seed field already holds that picture's seed)
|
||||
3. Add words to the prompt or change Steps, and compare the differences
|
||||
|
||||
Fixing the seed makes it easy to see "what you changed, and how it changed".
|
||||
This is the shortest path to getting used to image generation.
|
||||
|
||||
> **Summary of this chapter (first half)**
|
||||
> - Select in the Workflow tab and press "Load in GenerateUI" to edit in GenerateUI
|
||||
> - The basics are the three subtabs Input (prompt), Model (model) and Settings (detailed settings)
|
||||
> - After changing the model or Settings values, always press **Apply**. If you forget, it generates with the values from before the change (prompts and the Settings Switches column are reflected automatically)
|
||||
> - Generate to produce. Alt + Apply reflects and generates in one go
|
||||
> - If you set the seed to Fixed, you can keep a picture you like and compare by changing conditions. To use the workflow's seed as it is, use Workflow
|
||||
@@ -0,0 +1,353 @@
|
||||
## 5-5 Batch generation - trying while changing conditions
|
||||
|
||||
As you get used to image generation, you start wanting to try things like these.
|
||||
|
||||
- Try all the models you have with the same prompt and compare them
|
||||
- Generate several prompts one after another
|
||||
- Check in bulk which sampler suits this model
|
||||
|
||||
Changing the settings and pressing Generate for each image is a lot of work.
|
||||
With the **Batch** subtab, you specify "what to change while generating" all at once, and it generates in order automatically.
|
||||
|
||||
### What can be changed in a batch
|
||||
In a batch, you choose **one** of the following seven kinds and generate while changing it in order.
|
||||
|
||||
| Kind | What changes | When to use it |
|
||||
| --- | --- | --- |
|
||||
| Checkpoint | The model | Compare how models draw |
|
||||
| Lora | The LoRA | Compare the effect of LoRAs |
|
||||
| Prompt | The prompt | Try saved prompts in order |
|
||||
| Workflow | The workflow | Run several workflows in order |
|
||||
| Sampler | The sampler | Compare the differences between samplers |
|
||||
| Scheduler | The scheduler | Compare the differences between schedulers |
|
||||
| Style | The style | Compare the differences between styles |
|
||||
|
||||
> **Tip**
|
||||
> Only one kind can be changed at a time.
|
||||
> If you want to change both the model and the prompt at once, use the Lab in 5-7.
|
||||
|
||||
### The Batch tab screen
|
||||
The Batch tab is divided into three parts: left, center and right.
|
||||
|
||||
| Place | Role |
|
||||
| --- | --- |
|
||||
| Left (lists) | Choose Checkpoints / Sampler / Scheduler / Style from a list |
|
||||
| Center (groups) | Choose Checkpoint / Lora / Prompt / Workflow by "group" |
|
||||
| Right (Batch Queue) | The list of chosen items. This is where you decide which kind to run |
|
||||
|
||||
The groups in the center are linked to the groups made in each tab.
|
||||
|
||||
- Checkpoint and Lora groups -> the Models tab (Chapter 6)
|
||||
- Prompt groups -> the Prompt tab (Chapter 7)
|
||||
- Workflow groups -> the Workflow tab (Chapter 4)
|
||||
|
||||
### Example: comparing the models you have
|
||||
Let's compare how three models draw, using the same prompt and the same seed.
|
||||
|
||||
1. Using the steps in 5-3, load beforehand a workflow with the prompt you want to compare.
|
||||
2. Set the seed mode in the generate panel to **Fixed**.
|
||||
(Fixing the seed evens out every condition except the model, so the difference is easy to see)
|
||||
3. Open the **Batch** subtab.
|
||||
4. On the left **Checkpoints** tab, tick the models you want to compare.
|
||||
5. In the right-hand **Batch Queue**, turn on the checkbox in the heading of the **Checkpoint** column.
|
||||
The column shows the list of chosen models and their count.
|
||||
|
||||
6. Press **Generate**.
|
||||
|
||||
It generates in order with the chosen models.
|
||||
The **Batch** area of the generate panel shows which one is being generated now, and the overall progress.
|
||||
|
||||
- **Pause** - pauses. Pressing it again resumes.
|
||||
- **Stop** - stops the whole batch.
|
||||
|
||||
> **Caution**
|
||||
> If a Batch Queue column's checkbox stays on, the next time you press Generate it will run as a batch again.
|
||||
> If you want to generate just one image, turn the column's checkbox off.
|
||||
|
||||
> **Tip: LoRA batches**
|
||||
> In a Lora batch, the format of the chosen LoRA (`<lora:name:strength>`) and the trigger words obtained from Civitai
|
||||
> are added automatically to the Positive Prompt. When the batch ends, it returns to the original prompt.
|
||||
|
||||
---
|
||||
|
||||
## 5-6 Feeder - processing images from a folder in order
|
||||
|
||||
**Feeder** is a feature that feeds many images one at a time into a workflow and processes them in order.
|
||||
For example, you can use it like this.
|
||||
|
||||
- Convert photos you have collected into an illustration style in bulk (I2I: making an image from an image)
|
||||
- Read poses from several images and draw a character in the same pose
|
||||
|
||||
Feeder has two modes, switched with the **[Image Loop] / [Gallery]** buttons at the top left of the Feeder tab.
|
||||
|
||||
| Mode | Where images are taken from | What you need |
|
||||
| --- | --- | --- |
|
||||
| Image Loop | A folder on your computer | The custom node **comfyui-image-feeder** (installed separately) |
|
||||
| Gallery | A group in the Gallery tab | No extra installation (the **Gallery Feeder (WFS)** node bundled with Workflow Studio) |
|
||||
|
||||
> **Tip**
|
||||
> In both modes, **a dedicated node must be in the workflow**.
|
||||
> It cannot be used just by loading an ordinary workflow.
|
||||
|
||||
### How to use Gallery mode (no extra installation)
|
||||
1. On ComfyUI's node screen, place a **Gallery Feeder (WFS)** node where the workflow "loads an image",
|
||||
and connect its image output. Save, and load it into GenerateUI.
|
||||
2. In the **Gallery** tab (Chapter 8), gather the images you want to process into a group.
|
||||
(When you first open the Feeder tab, a group called `__Feeder__` is created automatically. You may use this)
|
||||
3. Open the **Feeder** subtab of GenerateUI and choose **[Gallery]**.
|
||||
4. Choose the target **Gallery Feeder Node** and the **group** to take images from, and press **Apply**.
|
||||
5. The images of the group line up in the center. Click an image to start processing from that image.
|
||||
6. Choose **After gen** (what to do after generation) and press **Run**.
|
||||
|
||||
### After gen (what to do after generation)
|
||||
| Mode | What it does |
|
||||
| --- | --- |
|
||||
| Loop | Moves to the next image and, on reaching the end, returns to the start and continues (does not stop until you press Stop) |
|
||||
| Increment | Moves to the next image, and stops automatically after processing the last one |
|
||||
| Fixed | Keeps using the same image (when you want to make many images changing only the seed) |
|
||||
|
||||
At first, choosing **Increment** is reassuring: it processes each image once and stops automatically.
|
||||
|
||||
### Image Loop mode
|
||||
In Image Loop mode, you use the images in a folder on your computer (`ComfyUI/user/default/image-loop-data/`).
|
||||
You can also choose images from the folder and save them as a preset.
|
||||
The **comfyui-image-feeder** custom node is needed separately.
|
||||
|
||||
The bundled samples `ws_sdxl_image_feeder_i2i` (I2I) and `ws_dwpose_image_feeder` (pose reading) are for this Image Loop mode.
|
||||
|
||||
### A sample for Gallery mode (v0.8.3 and later)
|
||||
For Gallery mode you can use the bundled sample `ws_sdxl_gallery_feeder_i2i` (SDXL I2I).
|
||||
You do not need to place nodes yourself: a **Gallery Feeder (WFS)** node is already in it (group name `__Feeder__`, sort order filename_asc).
|
||||
|
||||
1. Import the sample in the Workflow tab and press **Load in GenerateUI**.
|
||||
2. In the Gallery, press the **F** button at the top left of the thumbnail of each image you want to process, to put it into the `__Feeder__` group.
|
||||
3. In the **Feeder** subtab, choose **[Gallery]**, choose the Gallery Feeder Node and the group, and press **Apply**.
|
||||
4. In the Model subtab, choose again the Checkpoint and VAE you have, and press **Apply** (the model names in the sample are those of the author's environment).
|
||||
5. Set **After gen** to **Increment** and press **Run**.
|
||||
|
||||
---
|
||||
|
||||
## 5-7 Lab - experimental I2I batch generation (advanced)
|
||||
|
||||
**Lab** is an advanced, experimental feature that develops the batch generation of 5-5 further.
|
||||
Feel free to skip it at first.
|
||||
|
||||
### Differences from Batch
|
||||
| | Batch (5-5) | Lab |
|
||||
| --- | --- | --- |
|
||||
| What can be changed at once | Only one kind | Model, prompt and KSampler **separately from each other** |
|
||||
| When changes happen | Every time | You freely specify "from which run to change" |
|
||||
| Using a generated image as the next input | Not possible | Possible (can be chained) |
|
||||
|
||||
### The idea of keyframes
|
||||
In Lab, you specify "from which generation run to switch to which setting" as **keyframes**.
|
||||
|
||||
Example: generating 10 times
|
||||
|
||||
| Runs | Model | Prompt | KSampler |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 to 10 | Model A | - | - |
|
||||
| 1 to 4 | - | "Morning city" | - |
|
||||
| 5 to 10 | - | "Night city" | - |
|
||||
| 1 to 7 | - | - | Steps 20 |
|
||||
| 8 to 10 | - | - | Steps 40 |
|
||||
|
||||
In this way, the timing of switching can be decided separately for each column.
|
||||
Until the next keyframe arrives, the previous setting is carried over.
|
||||
|
||||
### Basic flow
|
||||
1. Load a workflow into GenerateUI.
|
||||
2. Open the **Lab** subtab.
|
||||
3. For I2I (image to image), drop the source image.
|
||||
When making from text (T2I), tick **T2I workflow (no source image)**.
|
||||
4. Decide the number of generations (Batch).
|
||||
5. In each of the Model, Prompt and KSampler columns, press **+** to add a keyframe,
|
||||
then double-click the keyframe and enter "from which run" and "to which setting".
|
||||
Cells in the KSampler column show the items you set (Step, CFG, Denoise, sampler, scheduler, Seed).
|
||||
A cell with nothing set shows "(workflow default)".
|
||||
6. Press **Run**.
|
||||
|
||||
Results are listed in the **Results** panel.
|
||||
|
||||
### Handy features
|
||||
- **Use generated image for next** - uses the generated image as the input image of the next run.
|
||||
You can make a series of images that change little by little.
|
||||
- **Plan Save** - lets you save the settings (the plan). When saved, an overview image summarizing the results is also made,
|
||||
and dropping that image lets you load the same plan again.
|
||||
- **Making a plan from a story** (v0.9.5+) - with the **Storyboard to Lab** skill in the AI TOOL chat,
|
||||
a story is split into one prompt per scene and the **Open in Lab** button loads the plan here (Chapter 10, section 10-5).
|
||||
|
||||
---
|
||||
|
||||
## 5-8 Using partner APIs (Nano Banana)
|
||||
|
||||
ComfyUI has **partner API nodes**, such as Google's **Nano Banana** image model (the Gemini image nodes), that hand the generation over to a cloud service.
|
||||
From v0.8.6, GenerateUI can also run workflows that contain these nodes.
|
||||
|
||||
> **Caution**
|
||||
> Partner API nodes run in the cloud, not on your PC's GPU, and **consume Comfy credits (paid)**.
|
||||
> If you run them repeatedly with Batch, Lab and so on, credits are consumed once per run.
|
||||
|
||||
### Preparation: set your Comfy API key
|
||||
1. Issue a Comfy API key at platform.comfy.org.
|
||||
2. Add the line `COMFY_API_KEY=your key` to the `.env` file in the plugin folder.
|
||||
(If there is no `.env`, copy `.env.example` to make one.)
|
||||
3. Restart ComfyUI. `.env` is read only at startup.
|
||||
|
||||
The key is used only on the server side and is never sent to the browser.
|
||||
If you run without a key, a message says that partner API authentication failed and how to set it up.
|
||||
|
||||
### How to use
|
||||
1. In the Workflow tab, load the Nano Banana sample (`ws_nano_banana_2_1_t2i`) into GenerateUI.
|
||||
2. Edit the prompt on the left side (Prompt) of the **Input** subtab and press **Apply**.
|
||||
3. To use reference images, specify them on the image cards on the right side (Image) of the **Input** subtab.
|
||||
4. Press **Generate**.
|
||||
|
||||
> **Tip**
|
||||
> You can also run it from ComfyUI's own screen while logged in. In that case you do not need the `.env` setting.
|
||||
|
||||
---
|
||||
|
||||
## 5-9 More workflows are supported (v0.8.4 and later)
|
||||
|
||||
The workflows you can use in GenerateUI have increased as follows. They are used the same way as other workflows.
|
||||
|
||||
- **Qwen Image 2.1 Turbo** - ComfyUI's official templates for text to image (T2I) and image to image (I2I). Edits also carry over to the prompt enhancer's input (the feature that expands your prompt automatically).
|
||||
- **Flux.3 Klein** - the four official templates (T2I and Image Edit, for Klein and Klein SD).
|
||||
- **MOSS-Image** - Base/SFT T2I, and the bounding-box versions that specify positions with frames.
|
||||
- **Fibo 2.0 Edit** (v0.8.14 and later) - an image-editing workflow. With the prompt enhancer (PE) setup, edits in the prompt area apply whether PE is on or off. The negative area holds a VGL-format (JSON) description as is. Width and height (Settings) of 0 mean automatic.
|
||||
- **FastH3 OmniRef / Pika-H3-Flex** (v0.8.14 and later) - MiniMax H3 reference to video. Treated as video workflows, and also usable in the Video tab (Chapter 11).
|
||||
- **Native LoRA stack** - ComfyUI v0.40's nodes that load several LoRAs at once (LoadLoraModel / LoadLoraTextEncoder). Apply, bypass and the Metadata display are supported.
|
||||
- **P-H3 Image** (v0.9.4+) - the image-edit and T2I templates. The prompt is shown on the Input tab and can be read by the Metadata tab; there is no negative prompt field. Width and height follow the workflow's wiring, and the prompt upsampler runs as set in the workflow.
|
||||
- **MiniMax H3 Image** (v0.9.6+) - ComfyUI's official image-edit and T2I templates. As with P-H3 Image, the prompt is shown on the Input tab and can be read by the Metadata tab and the sidepanel I tab; there is no negative prompt field. The prompt enhancer (PE) can be turned on and off in the Settings tab's Switches column (5-4).
|
||||
|
||||
> **Caution**
|
||||
> The native LoRA stack nodes are supported **only in API-format workflows**.
|
||||
> ComfyUI's own screen cannot yet add rows to this node, so UI-format workflows are not covered.
|
||||
|
||||
> **Tip**
|
||||
> Workflows containing the **Compare Images** (ImageCompare) node, which compares two images with a slider, used to fail on that node when run from GenerateUI (fixed in v0.9.6).
|
||||
|
||||
> **Tip**
|
||||
> The Metadata tab and the side panel's **I** tab can also read images made with Nano Banana (v0.8.5 and later).
|
||||
> The prompt appears as one whole entry, not split into Positive/Negative.
|
||||
|
||||
## 5-10 Writing prompts in your own language (Prompt Translation, v0.9.9 and later)
|
||||
|
||||
Most image generation models were trained on English prompts.
|
||||
With **Prompt Translation** turned on, you can write a prompt in your own language and turn it into English with one button.
|
||||
Like the translation extensions for Stable Diffusion web UI (A1111), it translates **only the parts written in your own language**.
|
||||
|
||||
Translation uses AI running on your PC (**Ollama**) and the translation model **PLaMo 3 Translate**, which supports 51 languages besides English (Japanese, Chinese, Korean, French, German, Spanish, Russian, Thai, Vietnamese and more).
|
||||
Your prompts are not sent to an online translation service.
|
||||
|
||||
> **What you need first**
|
||||
> Install and start **Ollama** (Chapter 10, section 10-1). The translation model can be downloaded from the Settings screen (see below).
|
||||
|
||||
### Preparation: turn it on in Settings
|
||||
1. In the **Settings** tab, open **Prompt Translation (native language -> English)**.
|
||||
2. Check **Enable prompt translation in GenerateUI**.
|
||||
3. Choose the model in **Translation model**. Models not yet in Ollama are marked **(not downloaded)**; choose one and press **Download** to pull it into Ollama.
|
||||
4. Press **Test**. When the sample prompt comes back in English, you are ready.
|
||||
|
||||
PLaMo 3 Translate comes in four sizes.
|
||||
|
||||
| Model | Size | Notes |
|
||||
| --- | --- | --- |
|
||||
| 0.8B | about 0.6 GB | Fastest. Enough for short tags |
|
||||
| 2B | about 1.8 GB | Recommended. Translates sentences accurately too |
|
||||
| 8B | about 5.3 GB | Higher quality |
|
||||
| 31B | about 19.5 GB | Highest quality, for PCs with plenty of memory |
|
||||
|
||||
> **Tip**
|
||||
> 0.8B sometimes shifts the meaning of a sentence (for example, "the camera follows from behind" in Japanese became "her camera following behind").
|
||||
> If you often write prompts as sentences, such as video prompts, choose 2B or larger.
|
||||
|
||||
The other settings are:
|
||||
|
||||
| Setting | What it does |
|
||||
| --- | --- |
|
||||
| Source language | Usually leave it at **Auto-detect**. If you always write in one language, choosing it makes detection certain |
|
||||
| Ollama URL | Ollama's address. Blank means `http://localhost:11434` |
|
||||
| Where to run the translation model | **GPU** or **CPU, kept loaded** (see below) |
|
||||
| Translate automatically when running | Translates automatically when you press Generate and so on (see below) |
|
||||
|
||||
Other Ollama models (such as the one you use for chat) can also be chosen under **Other Ollama models**. They are sent the text with a "translate into English" instruction.
|
||||
|
||||
### Where to run the translation model
|
||||
| Where | Time per phrase | Good for |
|
||||
| --- | --- | --- |
|
||||
| GPU (fastest per translation, uses VRAM) | about 0.1 s | GPUs with VRAM to spare |
|
||||
| CPU, kept loaded (no VRAM, no reload wait) | about 0.2-0.3 s | Keeping all VRAM for image generation |
|
||||
|
||||
On GPU, turning on **Unload the translation model from VRAM after translating** frees VRAM before image generation (on by default).
|
||||
However, every translation then waits **about 4 seconds** while the model loads again.
|
||||
|
||||
**CPU, kept loaded** keeps the model in system RAM (about 2.7 GB for 2B).
|
||||
It uses no VRAM and is loaded in the background when a workflow is loaded into GenerateUI, so even the first translation does not wait.
|
||||
|
||||
> **Tip**
|
||||
> When Ollama loads another model (for example the AI TOOL chat model), it may drop the resident translation model.
|
||||
> It then loads again automatically at the next translation (that one translation takes a few seconds).
|
||||
|
||||
### Using it: translate with the globe button
|
||||
With Prompt Translation on, a **globe (🌐)** button appears to the right of the **↺** button of Positive Prompt and Negative Prompt on the Input tab.
|
||||
|
||||
1. Write the prompt in your own language. Mixing in English tags is fine.
|
||||
|
||||
```
|
||||
masterpiece, 夕焼けの海辺に立つ少女、(長い黒髪:1.2), <lora:test_lora:0.8>, {猫耳|狐耳}, 水彩画風
|
||||
```
|
||||
|
||||
2. Press the globe button.
|
||||
3. Only the parts in your language become English.
|
||||
|
||||
```
|
||||
masterpiece, A girl standing by the seaside at sunset, (long black hair:1.2), <lora:test_lora:0.8>, {cat ears|fox ears}, watercolor style
|
||||
```
|
||||
|
||||
The following prompt syntax is kept as it is, without translating:
|
||||
|
||||
- Commas and line breaks
|
||||
- Weights `(word:1.2)` and `[ ]`
|
||||
- Random choice `{A|B}` and wildcards `__name__`
|
||||
- LoRA `<lora:name:0.8>` and `embedding:name`
|
||||
- `BREAK` and `AND`
|
||||
|
||||
After translating, an **Original** button appears. Press it to go back to the text before translation (Ctrl+Z also works).
|
||||
|
||||
> **Note**
|
||||
> Only parts that contain letters outside ASCII are translated; English-only parts stay as they are.
|
||||
> Conversely, an English word that contains an accented letter such as é is translated too.
|
||||
|
||||
### Translating automatically when running
|
||||
Under **Translate automatically when running** in Settings, checkboxes choose when to translate automatically.
|
||||
|
||||
- **GenerateUI - Generate**: pressing Generate replaces the text of the Positive/Negative fields with the translation, then generates
|
||||
- **GenerateUI Lab - Run**: a Lab run translates only what is sent. The Lab plan keeps your text
|
||||
- **Video Plan - Run**: a Video tab Plan run translates only the prompts that are sent. The plan keeps your text
|
||||
|
||||
Because Lab and Video Plan keep your text, you can keep editing the plan in your own language.
|
||||
The information saved with the generated images and videos (metadata) records the English prompt that was actually sent.
|
||||
|
||||
> **Note**
|
||||
> If translation fails while automatic translation is on (for example, Ollama is not running), generation stops so that untranslated text is not sent.
|
||||
> For Lab and Video Plan, the reason is shown in the progress line.
|
||||
|
||||
### Also in the Lab modal and Video Plan
|
||||
The globe button is also on these fields, and works the same way:
|
||||
|
||||
- The Positive and Negative fields of the screen that opens when you double-click a cell in Lab's prompt column (5-7)
|
||||
- The **Prompt** field of a block in the Video tab's Plan subtab (Chapter 11, section 11-4)
|
||||
|
||||
> **Tip**
|
||||
> PLaMo 3 Translate is free for personal use, hobbies, research and other non-commercial use (PLaMo Community License).
|
||||
> Check the license terms before using it commercially.
|
||||
|
||||
> **Summary of this chapter (second half)**
|
||||
> - With the Batch subtab, you can generate in bulk while changing the model, prompt, sampler and so on, one kind at a time
|
||||
> - Choose the kind with the Batch Queue column checkbox and start with Generate. Turn the check off when done
|
||||
> - Feeder is a feature that feeds many images one at a time into a workflow. A dedicated node is required
|
||||
> - Lab is an advanced feature that can switch several settings at separate timings
|
||||
> - Partner API nodes (Nano Banana and others) run once `COMFY_API_KEY` is set in `.env`. They consume credits
|
||||
> - With Prompt Translation on, the globe button turns a prompt written in your own language into English. Prompt syntax is kept, and Generate, Lab and Video Plan runs can translate automatically
|
||||
@@ -0,0 +1,184 @@
|
||||
# Chapter 6 The Models Tab - Managing Models and LoRAs
|
||||
|
||||
As you keep generating images, model and LoRA files keep increasing.
|
||||
"What model was this file?" "What was that LoRA's trigger word?"
|
||||
- The **Models** tab is what solves worries like these.
|
||||
|
||||
---
|
||||
|
||||
## 6-1 Kinds of models
|
||||
|
||||
At the top of the Models tab, tabs for each kind of model are lined up.
|
||||
Each shows the files in the corresponding folder inside ComfyUI's `models` folder.
|
||||
|
||||
| Kind | In a phrase | Main location |
|
||||
| --- | --- | --- |
|
||||
| Checkpoint | The core of image generation. Nothing starts without it | `models/checkpoints` |
|
||||
| LoRA | A small model that teaches an art style, character, outfit and so on | `models/loras` |
|
||||
| VAE | Finishes the colors and fine details of an image. Often built into the Checkpoint | `models/vae` |
|
||||
| ControlNet | A model for specifying composition, such as poses and line art | `models/controlnet` |
|
||||
| UNET | The core of newer-format models such as FLUX (Diffusion Model) | `models/diffusion_models` and so on |
|
||||
| TextEncoder | A model that converts the prompt text into a form the AI can understand | `models/text_encoders` and so on |
|
||||
| Hypernetwork | An older-format add-on model for adjusting art style | `models/hypernetworks` |
|
||||
| Embedding | A small file that teaches a particular word a meaning (often used for negatives) | `models/embeddings` |
|
||||
|
||||
> **Tip**
|
||||
> At first, it is enough to make good use of just two: **Checkpoint** and **LoRA**.
|
||||
> When you start using newer models such as FLUX, UNET and TextEncoder come up more often.
|
||||
|
||||
---
|
||||
|
||||
## 6-2 Displaying and finding in the list
|
||||
|
||||
### The first load
|
||||
When you open the Models tab for the first time, the list is empty and a message says to load models with the refresh button.
|
||||
Press **"Refresh"** on the toolbar and the list of models appears.
|
||||
Press "Refresh" also when you have added new models.
|
||||
|
||||
### Thumbnail view and table view
|
||||
As in the Workflow tab, you can switch between **Thumbnail** and **Table**.
|
||||
24 items are shown per page.
|
||||
|
||||
In table view, clicking a column heading sorts by that column
|
||||
(first click: ascending, second click: descending, third click: sorting off).
|
||||
|
||||
> **Tip**
|
||||
> For thumbnails, `(model name).preview.png` located in the same place as the model file is used automatically.
|
||||
> For models without an image, you can get one from Civitai using the method in 6-3, or set one yourself.
|
||||
|
||||
### Searching and filtering
|
||||
The toolbar has items for filtering.
|
||||
|
||||
| Item | What it filters |
|
||||
| --- | --- |
|
||||
| Search models... | Keywords such as the name |
|
||||
| Star | Favorites only |
|
||||
| All Tags | Tags |
|
||||
| All Folders | Subfolders |
|
||||
| All Groups | Groups |
|
||||
| All / Enabled only / Disabled only | Enabled or disabled state (6-5) |
|
||||
|
||||
If you narrow down so far that you lose track, press **"x Clear"** at the right end of the toolbar to clear everything.
|
||||
|
||||
### Favorites, badges and groups
|
||||
The way of sorting is almost the same as in the Workflow tab (Chapter 4, section 4-4).
|
||||
|
||||
- **Favorites (star)** - mark models you use often
|
||||
- **Badges** - attach colored labels made with **"Badge"**. The kinds and colors of badges are shared with the Workflow tab
|
||||
- **Groups** - click a model and add it in the side panel's **"Groups"** tab
|
||||
|
||||
> **Tip**
|
||||
> Groups are divided **by kind of model**.
|
||||
> A Checkpoint group is shown only in the Checkpoint tab.
|
||||
> Groups made here can also be used in the GenerateUI batch generation (Chapter 5, section 5-5).
|
||||
|
||||
---
|
||||
|
||||
## 6-3 Bringing in Civitai information
|
||||
|
||||
**Civitai** is a sharing site where many image-generation models are published.
|
||||
Many of the models you get online are distributed on Civitai.
|
||||
|
||||
Workflow Studio computes a **hash value** (a value like a fingerprint of the file) from your local model file,
|
||||
and can automatically look up the information registered on Civitai.
|
||||
|
||||
### What information you can get
|
||||
- The model's name and kind (Checkpoint, LoRA and so on)
|
||||
- **Base Model** (which family the model belongs to: SDXL, SD1.5, FLUX and so on)
|
||||
- **Trigger words** (words to put in the prompt to bring out a LoRA's effect)
|
||||
- Tags and description
|
||||
- Sample images (also used for thumbnails)
|
||||
|
||||
### Getting one at a time
|
||||
1. Click a model to select it.
|
||||
2. In the right-hand side panel, open the **"CivitAI"** tab (this tab opens first when you select a model).
|
||||
3. Press **"Fetch from CivitAI"**.
|
||||
|
||||
When fetching finishes, the model's information is shown.
|
||||
In the **Sample** tab, you can look at sample images posted to Civitai.
|
||||
|
||||
### Getting them in bulk
|
||||
Press the **"CivitAI"** button on the toolbar to fetch information in bulk for the models of the kind being shown.
|
||||
Models whose information has already been fetched are skipped, so it is fine to press it again and again after adding models.
|
||||
For models without a thumbnail image, a preview image is also saved automatically.
|
||||
|
||||
> **Tip**
|
||||
> Computing the hash value takes time depending on the size of the model.
|
||||
> If you have many models of several GB, run it when you have time to spare.
|
||||
|
||||
> **Tip**
|
||||
> A model that was not found on Civitai shows a message saying it was checked but this model was not found on CivitAI.
|
||||
> For models you made yourself, or models distributed somewhere other than Civitai, not being found is normal.
|
||||
> If you tick **"Skip not found"**, models that were not found are skipped without being rechecked when fetching in bulk.
|
||||
|
||||
> **Tip: API key**
|
||||
> The **"CivitAI API Key"** in the Settings tab can be left empty (optional).
|
||||
> If authentication is needed to get some information, enter a key issued in your Civitai account settings.
|
||||
|
||||
### Especially handy when using LoRAs
|
||||
Many LoRAs have no effect unless you put their **trigger words** in the prompt.
|
||||
If you have fetched the Civitai information, you can check the trigger words in the side panel right away.
|
||||
Also, when you use a LoRA in GenerateUI batch generation, the trigger words are added to the prompt automatically (Chapter 5, section 5-5).
|
||||
|
||||
---
|
||||
|
||||
## 6-4 Using models in generation and on the canvas
|
||||
|
||||
### Set it in GenerateUI (GenUI Model)
|
||||
You can set the model chosen in the Models tab directly into the workflow in GenerateUI.
|
||||
|
||||
1. Load a workflow into GenerateUI beforehand (Chapter 5, section 5-1).
|
||||
2. In the Models tab, select the model you want to use.
|
||||
3. Press the **"GenUI Model"** button.
|
||||
|
||||
The model in the GenerateUI workflow is now replaced with the one you chose.
|
||||
While comparing Civitai sample images, you can try any model that catches your eye right away.
|
||||
|
||||
> **Tip**
|
||||
> For an Embedding, **GenUI PP** (add to the positive prompt) and **GenUI NP** (add to the negative prompt) buttons are shown.
|
||||
> Add negative Embeddings with GenUI NP.
|
||||
|
||||
> **Caution**
|
||||
> It cannot be used if the GenerateUI workflow has no node that loads that model.
|
||||
> For example, you cannot set a LoRA in a workflow that has no LoRA node.
|
||||
|
||||
### Using it on ComfyUI's node screen
|
||||
If you drag a model from the **M (Models)** tab of the **WS Library** beside ComfyUI's node screen onto the canvas,
|
||||
a node that loads that model is placed automatically (Chapter 9).
|
||||
|
||||
---
|
||||
|
||||
## 6-5 Tidying away models you do not use
|
||||
|
||||
### Switching enabled and disabled
|
||||
Models you will not use for a while can be set to **Disable** instead of being deleted.
|
||||
A disabled model no longer appears in ComfyUI's model-selection lists, so the lists become tidier.
|
||||
|
||||
- With the **pause (II)** button on a card, you can switch enabled and disabled one at a time.
|
||||
- You can also enable or disable them in bulk, by group.
|
||||
|
||||
> **Tip**
|
||||
> When disabled, `.disabled` is added to the end of the file name. The file itself is not deleted.
|
||||
> When you enable it again, the original name comes back.
|
||||
|
||||
### Operating in bulk (Select)
|
||||
Press the green **Select** button to tick several models and operate on them together.
|
||||
|
||||
- Add to or remove from a group
|
||||
- Add or remove badges
|
||||
- Move to another subfolder (related files such as preview images move together)
|
||||
- Delete
|
||||
|
||||
### Deleting
|
||||
You can delete a model from the detail screen or from Select.
|
||||
|
||||
> **Caution**
|
||||
> Deleting **permanently removes** the model file and its related files (preview image, Civitai information and so on).
|
||||
> Many models take a long time to download again, so if in doubt, "disable" instead of deleting.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - In the Models tab you can list ComfyUI's models by kind. Press "Refresh" first
|
||||
> - You can sort with favorites, badges and groups. Groups can also be used in batch generation
|
||||
> - With "Fetch from CivitAI", you can automatically bring in Base Model, trigger words, sample images and more
|
||||
> - With "GenUI Model", you can try the chosen model in GenerateUI right away
|
||||
> - For models you do not use, "disable" rather than delete, to be safe
|
||||
@@ -0,0 +1,254 @@
|
||||
# Chapter 7 The Prompt Tab - Collecting and Growing Your Spells
|
||||
|
||||
A prompt that worked well is your own treasure.
|
||||
But if you only paste it into a notepad, you can easily lose track of which was which.
|
||||
|
||||
The **Prompt** tab is a place to **save, sort and reuse** prompts.
|
||||
In addition, you can use translation with local AI, and "wildcards" that change the prompt a little each time.
|
||||
|
||||
---
|
||||
|
||||
## 7-1 Prompt basics
|
||||
|
||||
Before getting into how to use the Prompt tab, let's cover the basics of writing prompts.
|
||||
|
||||
### Positive and negative
|
||||
| Kind | What to write | Example |
|
||||
| --- | --- | --- |
|
||||
| Positive Prompt | What you want drawn | `1girl, smiling, school uniform, cherry blossoms, spring` |
|
||||
| Negative Prompt | What you do not want drawn | `low quality, blurry, extra fingers, watermark` |
|
||||
|
||||
### Tips for writing
|
||||
- **Write in English** - most models were trained on English.
|
||||
- **Separate with commas (,)** - it is common to line up words and short phrases separated by commas.
|
||||
- **Put important things first** - words written earlier tend to be reflected more strongly.
|
||||
- **Emphasize words with (word:number)** - writing `(red hair:1.3)` emphasizes that word (1.0 is the default).
|
||||
|
||||
> **Tip: change the emphasis number from the keyboard**
|
||||
> In the prompt field of GenerateUI, select the word to emphasize and press **Ctrl + Up/Down**
|
||||
> to raise or lower the number in `(word:number)` by 0.05. When the number returns to 1.0, the parentheses are removed automatically.
|
||||
|
||||
> **Tip**
|
||||
> Anime-style SDXL models suit "lining up tags", and newer models such as FLUX suit "describing in ordinary English sentences".
|
||||
> The model's distribution page often lists a recommended way of writing, so check it.
|
||||
|
||||
---
|
||||
|
||||
## 7-2 The Prompt tab screen
|
||||
|
||||
The Prompt tab is divided into three columns: left, center and right.
|
||||
|
||||
| Column | Name | Role |
|
||||
| --- | --- | --- |
|
||||
| Left | AI Assistant | Chat and translation with local AI (7-5) |
|
||||
| Center | Presets / Preset Manager | Saving, listing and sorting prompts (7-3) |
|
||||
| Right | Wildcard / Style | Managing wildcards and styles (7-4) |
|
||||
|
||||
With **Form / Table** at the top of the screen, you can switch how things are displayed.
|
||||
|
||||
- **Form** - a screen for editing one item at a time (the screen this chapter explains)
|
||||
- **Table** - a screen for editing presets and wildcards together in a table (handy when the count grows)
|
||||
|
||||
---
|
||||
|
||||
## 7-3 Saving and sorting as presets
|
||||
|
||||
A **preset** is a pair of positive and negative prompts saved together under a name.
|
||||
|
||||
### Saving a preset
|
||||
1. In the center column, open the **Presets** tab.
|
||||
2. Check that the topmost pull-down shows **New Preset (none selected)**.
|
||||
3. Fill in the following items.
|
||||
|
||||
| Item | Contents |
|
||||
| --- | --- |
|
||||
| Preset Name | The name of the preset (for example `Sakura and uniform`) |
|
||||
| Category | A classification (for example `Portrait`, `Landscape`). May be omitted |
|
||||
| Positive Prompt | What you want drawn |
|
||||
| Negative Prompt | What you do not want drawn |
|
||||
|
||||
4. Press **Save**. When a message says the preset was saved, you are done.
|
||||
|
||||
> **Tip**
|
||||
> Get into the habit of saving prompts you tried in GenerateUI and thought "this is good!" as presets on the spot.
|
||||
> Later, when you want to make an image with the same feel, you can recall it right away.
|
||||
|
||||
### Recalling and using a preset
|
||||
1. From the topmost pull-down, choose the preset you want to use.
|
||||
2. Press a button according to how you want to use it.
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| GenUI Set | Sets both the positive and negative into the prompt fields of GenerateUI at once |
|
||||
| PP Copy | Copies the positive prompt |
|
||||
| NP Copy | Copies the negative prompt |
|
||||
|
||||
> **Caution**
|
||||
> **GenUI Set** can be used while a workflow is loaded in GenerateUI.
|
||||
> Load a workflow first, using the steps in Chapter 5.
|
||||
|
||||
### Sorting with groups
|
||||
Presets can also be sorted into groups.
|
||||
|
||||
1. With a preset selected, press **+ Group** to make a new group.
|
||||
2. Choose a group from **Select group...** and press **Add to Group**.
|
||||
|
||||
A group can also be deleted (**Del Group**).
|
||||
|
||||
Groups made here can be used as "Prompt" groups in GenerateUI batch generation (Chapter 5, section 5-5).
|
||||
For example, if you put spring, summer, autumn and winter presets in a group called "Seasons series",
|
||||
you can generate the pictures of all four seasons in bulk.
|
||||
|
||||
### Listing with Preset Manager
|
||||
In the **Preset Manager** tab of the center column, you can see the saved presets as a list.
|
||||
|
||||
- **All** - all presets
|
||||
- **Star** - presets registered as favorites
|
||||
- **Groups** - shown by group
|
||||
|
||||
With the **Search presets...** box at the top, you can search presets.
|
||||
|
||||
Saved presets are also shown in the **P (Prompts)** tab of the **WS Library** beside ComfyUI's node screen,
|
||||
and can be dragged onto the canvas for use (Chapter 9).
|
||||
|
||||
---
|
||||
|
||||
## 7-4 Wildcards and styles
|
||||
|
||||
In the right column, you manage **wildcards**, which change the prompt automatically,
|
||||
and **styles**, which add a mood to the prompt.
|
||||
Switch with **Wildcard / Style** at the top of the column.
|
||||
|
||||
### What is a wildcard?
|
||||
A wildcard is a mechanism that **randomly swaps out a part of the prompt every time you generate**.
|
||||
With the same prompt, you can make many images with variation.
|
||||
|
||||
The easiest way in Workflow Studio is to use a **wildcard file**.
|
||||
Make a file listing candidates, and when you write `__filename__` in the prompt,
|
||||
one line from the file is chosen at random every time you generate.
|
||||
|
||||
### Making a wildcard file
|
||||
As an example, let's make a file called `colors` containing candidate colors.
|
||||
|
||||
1. In the right column, choose **Wildcard** and press **+ New** under **Wildcard Files**.
|
||||
2. Enter a file name (for example `colors`) and choose **txt** as the format.
|
||||
3. Enter one candidate per line.
|
||||
|
||||
```
|
||||
red
|
||||
blue
|
||||
green
|
||||
golden
|
||||
```
|
||||
|
||||
4. Press **Save**.
|
||||
|
||||
### Using a wildcard
|
||||
In the prompt, write the file name enclosed in `__` (two underscores).
|
||||
|
||||
```
|
||||
1girl, __colors__ dress, standing in a garden
|
||||
```
|
||||
|
||||
Each time you generate in GenerateUI, the `__colors__` part is replaced with one of `red`, `blue`, `green` or `golden`.
|
||||
|
||||
> **Tip**
|
||||
> Instead of typing `__colors__` by hand, you can also choose a file with the **File** button in the right column and insert it.
|
||||
|
||||
### More advanced notation (for Impact Pack)
|
||||
At the top of the right column there are also buttons that insert symbols such as `{ }`, `$$` and `[ ]`.
|
||||
These are for notations like the following.
|
||||
|
||||
- `{cat|dog|rabbit}` - choose one out of the three
|
||||
- `{2$$cat|dog|bird}` - choose two out of the three
|
||||
|
||||
> **Caution**
|
||||
> What GenerateUI expands automatically is only the `__filename__` notation.
|
||||
> To use notations such as `{A|B}`, you need to put the wildcard node of the custom node **ComfyUI-Impact-Pack**
|
||||
> (ImpactWildcardProcessor and so on) in the workflow.
|
||||
> Until you are used to it, using only the `__filename__` notation is enough.
|
||||
|
||||
### Sending the prompt you built
|
||||
The prompt you built in the right column can be sent out with the buttons below.
|
||||
|
||||
- **-> Pos** - send to the preset's Positive Prompt
|
||||
- **-> Neg** - send to the preset's Negative Prompt
|
||||
- **Copy** - copy it
|
||||
|
||||
The `__filename__` parts are expanded automatically when you generate in GenerateUI.
|
||||
|
||||
### Styles
|
||||
A **style** is a "mood template" added to the prompt.
|
||||
For example, the "cinematic" style has contents registered such as `cinematic still, {prompt}, dramatic lighting`.
|
||||
Your own prompt goes into the `{prompt}` part.
|
||||
|
||||
To use a style, in the top bar of GenerateUI, turn on the **Style** checkbox and choose a style from the pull-down.
|
||||
When you generate, the chosen style is combined into the prompt automatically (the workflow itself is not rewritten).
|
||||
|
||||
Many styles come with Workflow Studio.
|
||||
In **Style** of the Prompt tab, you can look at the list of styles and make your own with **+ New**.
|
||||
|
||||
> **Tip**
|
||||
> To see what mood each style gives, it helps to use **Create Catalog** in GenerateUI to make sample images for each style in bulk.
|
||||
> The samples you make can be listed in the Style_Catalog of the Gallery tab (Chapter 8).
|
||||
|
||||
---
|
||||
|
||||
## 7-5 Getting AI to help with translation and prompt making
|
||||
|
||||
In the **AI Assistant** in the left column, you can have AI running on your computer (local AI) help you make prompts.
|
||||
|
||||
> **What you need in advance**
|
||||
> To use this feature, you need local AI software such as **Ollama** or **LM Studio**, and an AI model prepared.
|
||||
> How to prepare is explained in Chapter 10. If it is not ready, **Disconnected** is shown at the top of the AI Assistant.
|
||||
|
||||
### Translating your language into English
|
||||
Even if you are not good at English prompts, it is fine.
|
||||
|
||||
1. In the input box at the bottom of the left column, write in your language what you want drawn.
|
||||
|
||||
```
|
||||
A girl in a school uniform smiling under a cherry tree, soft spring light
|
||||
```
|
||||
|
||||
2. Press the **JA->EN** button under **Translate:** (choose the languages to suit your text).
|
||||
3. The translated result is shown in the chat area.
|
||||
|
||||
With the buttons shown below the translation result, you can use the result.
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| Apply to GenerateUI | Puts it into the positive prompt field of GenerateUI |
|
||||
| Copy | Copies it (for pasting into a preset and so on) |
|
||||
|
||||
When you want to know the meaning of an English prompt, you can conversely translate it with **EN->JA**.
|
||||
|
||||
> **Tip**
|
||||
> To translate directly in GenerateUI's prompt fields, **Prompt Translation** (the globe button) in Chapter 5, section 5-10 is handy. It turns only the parts in your language into English and keeps weights and LoRA tags.
|
||||
> The AI Assistant's translation can also use the translation model **PLaMo 3 Translate** (v0.9.9 and later; before that, the instruction text was translated too, and ZH->EN came out in Japanese).
|
||||
|
||||
### Making prompts while consulting the AI
|
||||
Write a question in the input box and press **Send** (or the Enter key) to chat with the AI.
|
||||
|
||||
- "Make a prompt for a sunset seaside scene, in English tag format"
|
||||
- "Add a more fantastical mood to this prompt"
|
||||
|
||||
If you ask like this, it offers draft prompts.
|
||||
If you are using an AI model that can read images, you can attach an image with **Attach Image**
|
||||
and ask "turn the mood of this image into a prompt".
|
||||
|
||||
- **Shift + Enter** - line break
|
||||
- **Clear Chat** - reset the conversation
|
||||
- **Gear button** - settings for which AI software and model to use (Chapter 10)
|
||||
|
||||
> **Caution**
|
||||
> A local AI's answers are not always correct.
|
||||
> Adjust the prompts that come out while actually generating to check them.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - Prompts are basically English and comma-separated. Emphasis is `(word:number)`
|
||||
> - Save prompts that worked as presets with a name
|
||||
> - With "GenUI Set", you can set a preset into GenerateUI right away
|
||||
> - If you sort them into groups, you can try them in bulk with batch generation
|
||||
> - Make a wildcard file and write `__filename__` to make the prompt vary with each generation
|
||||
> - If you prepare local AI, you can translate and consult about prompts
|
||||
@@ -0,0 +1,331 @@
|
||||
# Chapter 8 The Gallery Tab - Looking Back at Generated Images
|
||||
|
||||
As you keep generating images, hundreds and thousands of images pile up in ComfyUI's `output` folder.
|
||||
The **Gallery** tab is a place to browse them comfortably,
|
||||
and to solve "how did I make this image?".
|
||||
|
||||
Images generated in ComfyUI have information about the workflow and prompt used for generation (**metadata**) embedded in them.
|
||||
The Gallery tab reads this information so that you can reproduce the same settings right away.
|
||||
Videos (MP4 and WebM), audio files (WAV, MP3, OGG, M4A, FLAC and Opus) and 3D models (GLB, glTF, VRM, PLY and SPZ) can also be handled in the same list as images.
|
||||
|
||||
---
|
||||
|
||||
## 8-1 Listing images
|
||||
|
||||
At the top of the Gallery tab there are four subtabs.
|
||||
|
||||
| Subtab | Role | In this book |
|
||||
| --- | --- | --- |
|
||||
| Output | The list of generated images (this is what you normally use) | 8-1 to 8-3 |
|
||||
| ImagePrompt | Build prompts while looking at sample images | 8-4 |
|
||||
| Style_Catalog | Choose from sample images of styles | 8-4 |
|
||||
| Metadata | Drop an image file to inspect the information in it | 8-2 |
|
||||
|
||||
### The Output screen
|
||||
The **Output** subtab is divided into three parts: left, center and right.
|
||||
|
||||
| Place | Role |
|
||||
| --- | --- |
|
||||
| Left (folders) | The list of folders inside the `output` folder. The topmost **[root]** is the whole `output` folder |
|
||||
| Center (list) | The images of the chosen folder. Scroll down and more are loaded |
|
||||
| Right (detail panel) | Information about the chosen image (8-2) |
|
||||
|
||||
First, choose a folder from the folder list on the left, and its images appear in the center.
|
||||
|
||||
> **Tip**
|
||||
> If images you newly generated in GenerateUI or ComfyUI do not appear, press **Refresh** on the toolbar.
|
||||
|
||||
> **Tip**
|
||||
> The location of the `output` folder that is read can be changed with **"Gallery Output Folder"** in the Settings tab.
|
||||
|
||||
### Switching the display, searching and sorting
|
||||
As in other tabs, you can do the following.
|
||||
|
||||
- **Thumbnail / Table** - switch between thumbnail display and table display
|
||||
- **Search box** - search by file name, tags, memo and **the contents of the prompt**. If you turn on the **S** button next to it, you can search by the meaning of a sentence (8-6)
|
||||
- **Sorting** - Date (Newest) / Date (Oldest) / Name A-Z / Name Z-A
|
||||
- **Star** - show favorites only
|
||||
- **x Clear** - clear all filters at once
|
||||
|
||||
> **Tip**
|
||||
> In the search box you can also search by words in the prompt.
|
||||
> For example, searching "cat" finds images whose prompt contains cat.
|
||||
> (Prompt search targets images that you have opened once in the detail panel)
|
||||
|
||||
### Looking large, and comparing
|
||||
- Clicking an image shows it large in the right-hand detail panel.
|
||||
- **Hold the Ctrl key** and click images to select several.
|
||||
Select two to four and press **Compare** to view them side by side.
|
||||
|
||||
Lining up images made by changing only the seed or the model makes the differences easy to see.
|
||||
|
||||
### Organizing
|
||||
- You can sort with **favorites (star), tags, memo and groups** (set in the detail panel).
|
||||
- You can select several and add them to a group together, or move them to another folder.
|
||||
The screen for choosing the destination (**Move To...**) has a filter box: enter part of a folder name to narrow the candidates, and press Enter to move to the first candidate.
|
||||
- Pressing **Export** bundles the chosen images into a ZIP file for download.
|
||||
|
||||
> **Caution**
|
||||
> Using **Delete** in the detail panel, or **Delete File** for a multiple selection, deletes the image files themselves.
|
||||
|
||||
---
|
||||
|
||||
## 8-2 Reading generation settings from metadata
|
||||
|
||||
When you click an image, four tabs appear in the right-hand detail panel.
|
||||
|
||||
| Tab | Contents |
|
||||
| --- | --- |
|
||||
| Prompt | The prompts used to generate the image (the tab shown first) |
|
||||
| Info | File name, size, date, tags, memo |
|
||||
| JSON | The workflow embedded in the image (JSON) |
|
||||
| Groups | Group settings |
|
||||
|
||||
### Checking the prompt
|
||||
The **Prompt** tab lists the prompts that were embedded in the image.
|
||||
Positive ones are marked **POS**, negative ones **NEG**.
|
||||
Clicking an item shows the whole text.
|
||||
|
||||
### Examining more closely (the Metadata subtab)
|
||||
When you want to know in detail even the models and LoRAs used, press the **Metadata** button in the detail panel
|
||||
(or click an image while holding the Alt key).
|
||||
The Gallery's **Metadata** subtab opens and the following information is shown in an organized way.
|
||||
|
||||
- **Models** - Checkpoint, VAE, Diffusion Model, Text Encoder (center column)
|
||||
- **LoRA** - the LoRAs used and their strength (center column)
|
||||
- **Prompt** - positive and negative (center column)
|
||||
- **Settings** - the image size (width x height), and the KSampler's Seed, Steps, CFG, Sampler, Scheduler and Denoise (right column; v0.7.14 and later)
|
||||
- **Nanobanana** - the generation history of images made in Comic Creator's nanobanana tab: engine, model, prompt and so on (right column; only for matching images)
|
||||
|
||||
With the buttons beside the prompt, you can send the contents to other tabs.
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| Copy | Copy |
|
||||
| GenUI:P / GenUI:N | Put it into GenerateUI's positive / negative |
|
||||
| Prompt:P / Prompt:N | Put it into the positive / negative of a preset in the Prompt tab |
|
||||
|
||||
> **Tip: you can also examine images you found online**
|
||||
> If you **drop** an image file onto the left side of the Metadata subtab, you can examine that image's information.
|
||||
> Besides ComfyUI images, PNG images made with SD WebUI (AUTOMATIC1111), SD Forge and Fooocus are also supported.
|
||||
> Handy when you want to know how an image found on a distribution site was made
|
||||
> (however, it cannot be read from images whose metadata was stripped when they were posted).
|
||||
|
||||
---
|
||||
|
||||
## 8-3 Reproducing the same settings
|
||||
|
||||
The handiest use of the Gallery is to **generate again with the same settings as an image you like**.
|
||||
There are two ways.
|
||||
|
||||
### Method 1: Reproduce in GenerateUI (Load GenUI)
|
||||
1. In the Gallery, click the image you want to reproduce.
|
||||
2. Press the **Load GenUI** button in the detail panel.
|
||||
3. The workflow embedded in the image is loaded into GenerateUI.
|
||||
|
||||
After that, as in Chapter 5, you can generate while changing the prompt a little or changing the seed.
|
||||
|
||||
> **Tip**
|
||||
> **When you want to make the same image again (v0.8.3 and later)**
|
||||
> The workflow embedded in the image also contains the seed value from that time.
|
||||
> If you set the generate panel's seed mode to **Workflow** and press Generate, it generates with that seed as it is.
|
||||
>
|
||||
> If the mode stays at Random, it generates with a different seed.
|
||||
> Also, if you use Fixed, paste the value of the **Seed** field in the **Settings** subtab into the generate panel's **Seed** field (because the generate panel's value takes priority).
|
||||
|
||||
### Method 2: Reproduce on ComfyUI's node screen (Copy & Send Canvas)
|
||||
1. In the Gallery, click the image you want to reproduce.
|
||||
2. Open the **JSON** tab in the detail panel.
|
||||
3. Press **Copy & Send Canvas**.
|
||||
|
||||
The workflow that made that image is loaded onto ComfyUI's node screen.
|
||||
Use this when you want to adjust things in detail on the node screen.
|
||||
|
||||
> **Caution**
|
||||
> If a message says "this image has no ComfyUI workflow embedded", it cannot be reproduced.
|
||||
> This applies to images made with settings that do not save workflow information, and images edited or compressed with other software.
|
||||
|
||||
### Using an image as material for the next generation
|
||||
You can also use a Gallery image as an "input image" for generation.
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| Send GenUI Image | Sets the chosen image as the input image (LoadImage node) of the GenerateUI workflow |
|
||||
| Send to LI node | Sets the chosen image on the image-loading node currently selected on ComfyUI's node screen |
|
||||
| Image Edit | Opens the chosen image in the Image Edit tab (Chapter 11) |
|
||||
| Send CC Layout | Sends the chosen image to Comic Creator's Layout tab (the selected panel / overlay) (shown only when the Gallery is opened inside Comic Creator; Chapter 11, section 11-6) |
|
||||
| Send CC Image | Sends the chosen image to Comic Creator's Image tab (as a new layer if a document is open, otherwise as a new canvas; videos are not supported; shown under the same condition as Send CC Layout) |
|
||||
|
||||
Handy when redrawing a generated image further with an I2I (image to image) workflow.
|
||||
|
||||
---
|
||||
|
||||
## 8-4 ImagePrompt and Style_Catalog
|
||||
|
||||
### ImagePrompt - building prompts from sample images
|
||||
**ImagePrompt** is a subtab for building prompts by choosing visually,
|
||||
with "sample images" and "prompts" lined up as sets.
|
||||
|
||||
1. Click a sample image and the prompt of that image appears on the right side.
|
||||
2. Press **+ Add** and it is added to **Selected Prompts**.
|
||||
3. After adding several, a **Final Prompt**, with them joined by commas, is produced.
|
||||
4. Use **Copy** to copy it and paste it into GenerateUI or a preset.
|
||||
|
||||
Put the sample images in the `ws_image_prompt` folder, which is created automatically inside the `output` folder.
|
||||
If you put a `.txt` file with the same name as the image (its contents being the prompt) alongside it, it becomes usable as a sample.
|
||||
|
||||
> **Example**
|
||||
> `ws_image_prompt/hairstyle/ponytail.png` and `ws_image_prompt/hairstyle/ponytail.txt` (contents: `ponytail, long hair`)
|
||||
|
||||
### Style_Catalog - choosing styles by sample images
|
||||
The styles introduced in Chapter 7 are hard to picture from their names alone.
|
||||
In **Style_Catalog**, you can line up sample images for each style and choose by how they look.
|
||||
|
||||
Sample images are made with **Create Catalog** in the top bar of GenerateUI.
|
||||
|
||||
1. Load a workflow that makes images from text into GenerateUI.
|
||||
2. In the **Style** section of the **Batch** subtab, tick the styles you want samples of.
|
||||
3. Press **Create Catalog** in the top bar, decide the destination folder and run.
|
||||
4. Press **Catalog** in the top bar to move to Style_Catalog in the Gallery.
|
||||
|
||||
When you choose a sample image in Style_Catalog and press **Select as Style**,
|
||||
GenerateUI's style switches to that style.
|
||||
|
||||
> **Tip**
|
||||
> How a style takes effect changes with the model.
|
||||
> If you separate the destination folders by model, they are easier to compare.
|
||||
|
||||
---
|
||||
|
||||
## 8-5 Eagle integration and PSD export (optional)
|
||||
|
||||
From here on, these are features only for those who need them.
|
||||
|
||||
### Eagle integration
|
||||
**Eagle** is paid software for organizing images.
|
||||
If you enable **Eagle integration** in the Settings tab, images generated in GenerateUI can be automatically saved to Eagle together with information such as the prompt.
|
||||
If you manage images in Eagle, give it a try.
|
||||
|
||||
### PSD export
|
||||
Select several images in the Gallery and press **Create PSD**,
|
||||
and you can make a PSD (Photoshop format) file with each image stacked as a layer.
|
||||
Handy when you want to gather difference images together and finish them in image-editing software.
|
||||
|
||||
> **Caution**
|
||||
> To use PSD-related features, an extra Python package (psd-tools) is needed.
|
||||
> If it is not installed, the installation method is shown in an error message (Chapter 11).
|
||||
|
||||
---
|
||||
|
||||
## 8-6 Finding images by "meaning" (semantic search)
|
||||
|
||||
Normal search finds images whose file name, tags or prompt characters match.
|
||||
**Semantic search** (v0.8.2 and later) searches by the "meaning" of an image.
|
||||
For example, if you enter "a city street at dusk", it finds images with a matching mood even when the prompt does not contain those words.
|
||||
You can also bring up a list of **similar images** to the one you choose.
|
||||
|
||||
### How to use it
|
||||
1. Turn on the **S** button next to the search box. The search box switches to semantic search, and a second row appears on the toolbar.
|
||||
2. Press the **gear** (settings) button on the second row and build the **index** (an index for searching) (see below).
|
||||
3. Enter a sentence in the search box and press **Enter**. Results line up in score order, with the score shown on each thumbnail.
|
||||
4. Select an image and press **Similar** on the second row, and images similar to it line up. Select another image and press the **refresh** button to redo the similar-image list for that image. Press **Similar** again to return to the normal display.
|
||||
|
||||
> **Tip**
|
||||
> Sorting and tag/group filtering do not apply to semantic search results.
|
||||
> The search range is inside the folder you have selected (including subfolders).
|
||||
> A folder with a green dot next to its image count has all the images directly in it indexed.
|
||||
|
||||
### Choosing a model (the "Model" tab of the gear)
|
||||
| Model | Characteristics | Preparation |
|
||||
| --- | --- | --- |
|
||||
| Gemini Embedding 2 | Computed on Google's service. Fast. Images are sent to Google | Get an API key, write `GEMINI_API_KEY=key` in the plugin's `.env`, and restart ComfyUI |
|
||||
| EmbeddingGemma 2 | Computed inside your computer with **Ollama** (local, free). Nothing is sent outside | Install and start Ollama, then run `ollama pull embeddinggemma-2:740m` (or the lighter `embeddinggemma-2:440m`). On the Model tab, choose the **Ollama URL** (usually unchanged) and the **Model size** |
|
||||
|
||||
> **Caution: when using Gemini**
|
||||
> Images are sent to Google's service. Content sent under the free tier may be used by Google to improve its products.
|
||||
> If you have many images you do not want to expose, choose the local EmbeddingGemma 2.
|
||||
> Write the API key only in `.env`, and do not show it to others.
|
||||
|
||||
### Building the index (the "Index" tab of the gear)
|
||||
1. Choose the target folder (you can also choose whether to include subfolders).
|
||||
2. Choose "new only" or "rebuild all" and start. Progress is shown, and you can stop partway.
|
||||
3. The index is one shared across all folders. You may build it a little at a time, folder by folder.
|
||||
|
||||
When you change the model or the vector dimension, a **rebuild of everything** is needed.
|
||||
|
||||
### Speed guidelines and settings
|
||||
- Since v0.8.14 the local EmbeddingGemma 2 runs in Ollama (the earlier way of running it in ComfyUI's Python was removed).
|
||||
On an NVIDIA GPU it takes about 0.1 to 0.2 second per image. When indexing finishes, the model is unloaded from GPU memory automatically.
|
||||
- Ollama must be running while you build the index and while you search.
|
||||
- The 740m (text + image + audio) and 440m (text + image, lighter) model sizes return the same results, so switching does not need a rebuild.
|
||||
Indexes made with v0.8.13 or earlier keep working as they are.
|
||||
- In the **Search** tab, you can decide the "minimum score" for similar-image and text search, and the maximum number of results.
|
||||
Because the score level differs by model, it is saved per model.
|
||||
|
||||
---
|
||||
|
||||
## 8-7 Handling audio files, and converting
|
||||
|
||||
As opportunities to make music and sound effects in ComfyUI have increased, the Gallery can handle audio files too (v0.8.0 and later).
|
||||
Supported are WAV, MP3, OGG, M4A, FLAC and Opus.
|
||||
|
||||
- In the list, a waveform thumbnail and a note badge are shown. The waveform color differs by format (MP3 blue, WAV green, FLAC amber, OGG/Opus purple, M4A pink).
|
||||
- When you select one, the waveform and a player appear in the right-hand detail panel, and you can play it right there.
|
||||
- For audio saved with a workflow (`prompt` / `workflow`) embedded, the Prompt and JSON tabs also work.
|
||||
- Videos (MP4 and WebM) can be listed in the same way. With the **Send to LV node** button, you can send the chosen video to the **Load Video** node selected on the ComfyUI screen (the video version of Send to LI node for images).
|
||||
|
||||
### Converting files
|
||||
When you select several items including audio or video, **Convert...** appears on the bulk-action bar.
|
||||
|
||||
1. Select the files you want to convert and press **Convert...**.
|
||||
2. Decide the output format (FLAC / MP3 / Opus / M4A / WAV) and, if needed, the bitrate, sample rate and number of channels.
|
||||
3. When run, the result is saved as a separate file in the same folder (if the same name exists, a number is added such as `name_converted`, and nothing is overwritten).
|
||||
|
||||
When you choose a video, only the audio track is extracted and saved.
|
||||
|
||||
| Carried over | Description |
|
||||
| --- | --- |
|
||||
| Workflow information | Carried over to MP3, FLAC, Opus and M4A. It cannot be carried over to WAV (a notice appears) |
|
||||
| Favorites, tags, memo, groups | You can choose whether to carry them over |
|
||||
| Deleting the original file | You can choose whether to delete it after a successful conversion (with a confirmation screen) |
|
||||
|
||||
> **Tip**
|
||||
> OGG can be played, but cannot be chosen as an output format of conversion.
|
||||
> You can convert with Workflow Studio alone, without installing other video-editing software.
|
||||
|
||||
> **Tip: use in the Video tab**
|
||||
> In the Asset subtab of Video, you can set an audio file as "Set as BGM" or "Add as sound" (Chapter 11, section 11-4).
|
||||
|
||||
---
|
||||
|
||||
## 8-8 Viewing 3D models (v0.9.4+)
|
||||
|
||||
The results of ComfyUI workflows that make 3D models (such as image to 3D) can also be checked in the Gallery.
|
||||
Supported formats are GLB, glTF, VRM, PLY (mesh, point cloud or Gaussian splat) and SPZ.
|
||||
|
||||
- They are listed with a "**3D**" badge. The thumbnail is drawn by your browser the first time the file appears in the list and is then kept on the server
|
||||
(files over 128 MB get one when opened).
|
||||
- When selected, the file is shown in 3D in the built-in viewer of the detail panel on the right. Double-click to enlarge it.
|
||||
|
||||
| Action | What it does |
|
||||
| --- | --- |
|
||||
| Drag | Rotate |
|
||||
| Right-drag | Move (pan) |
|
||||
| Mouse wheel | Zoom in / out |
|
||||
| ⟲ (reset) button | Return to the starting view |
|
||||
|
||||
- For a model saved with ComfyUI's **SaveGLB**, the workflow can be read just like an image's (sections 8-2 and 8-3).
|
||||
|
||||
> **Caution**
|
||||
> - 3D files cannot be sent to Image Edit, GenerateUI, Tagger or Feeder.
|
||||
> - Some `.gltf` files refer to separate `.bin` or image files. Moving or deleting in the Gallery only affects the `.gltf` itself, so do not change where those files are.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - In the Gallery's Output, you can look back at generated images folder by folder. You can also search by words in the prompt
|
||||
> - The detail panel's Prompt tab shows right away the prompt used to generate the image
|
||||
> - With "Load GenUI" you can reproduce the same workflow in GenerateUI, and with "Copy & Send Canvas" on the node screen
|
||||
> - To reproduce, set the seed mode to Workflow (or Fixed)
|
||||
> - If you drop an image onto the Metadata subtab, you can examine the models, LoRAs and prompt in detail
|
||||
> - With the S button you can search by the meaning of a sentence or by "similar images" (semantic search)
|
||||
> - Audio files can also be listed, played and converted
|
||||
> - 3D models (GLB, glTF, VRM, PLY, SPZ) can also be turned around and checked in the Gallery (8-8)
|
||||
@@ -0,0 +1,171 @@
|
||||
# Chapter 9 The WS Library - Using It Inside the ComfyUI Screen
|
||||
|
||||
In the chapters so far, we mainly used the main part of Workflow Studio (the screen with the row of tabs).
|
||||
In this chapter, we use the **WS Library (Workflow Studio Library)** that opens beside ComfyUI's node screen.
|
||||
|
||||
With the WS Library, you can use the workflows, nodes, models and prompts you organized in Workflow Studio
|
||||
**just by dragging and dropping onto the node screen**.
|
||||
It is a feature whose convenience you feel more the longer you spend working on the node screen.
|
||||
|
||||
---
|
||||
|
||||
## 9-1 Opening the WS Library
|
||||
|
||||
1. Open ComfyUI's node screen.
|
||||
2. Press the **WS Library** button (the icon of lined-up squares) added to the top menu.
|
||||
|
||||
A panel opens at the edge of the screen. Press the button again to close it.
|
||||
|
||||
### Six tabs
|
||||
At the top of the panel, single-letter tabs are lined up. Hovering the mouse shows the proper name.
|
||||
|
||||
| Tab | Name | What you can do | In this book |
|
||||
| --- | --- | --- | --- |
|
||||
| W | Workflows | Load workflows | 9-2 |
|
||||
| N | Nodes | Place nodes and node sets | 9-2 |
|
||||
| M | Models | Place nodes that load models | 9-2 |
|
||||
| P | Prompts | Place or copy saved prompts | 9-2 |
|
||||
| I | Information | Inspect images and workflows and place the materials used | 9-3 |
|
||||
| A | AI TOOL | Use AI features such as translation and chat | Chapter 10 |
|
||||
|
||||
Inside each tab there are further small tabs (subtabs).
|
||||
In many tabs, the following subtabs are common.
|
||||
|
||||
- **All** - everything
|
||||
- **Favorites** - things registered as favorites in the main Workflow Studio
|
||||
- **Groups** - by the groups made in the main Workflow Studio
|
||||
|
||||
Also, in each subtab's search box, you can narrow down by typing a name.
|
||||
|
||||
> **Tip**
|
||||
> What is shown in the WS Library is linked to the organization done in the main Workflow Studio (favorites, groups, badges).
|
||||
> The better you organize in the main part, the easier the WS Library is to use.
|
||||
|
||||
---
|
||||
|
||||
## 9-2 Placing onto the canvas with drag and drop
|
||||
|
||||
The basic operation of the WS Library is very simple.
|
||||
|
||||
| Operation | What happens |
|
||||
| --- | --- |
|
||||
| Drag and drop onto the canvas | Places or loads it at the dropped position |
|
||||
| Double-click | Places or loads it near the center of the screen |
|
||||
|
||||
### W - Load a workflow
|
||||
When you drag a workflow onto the canvas (or double-click it), that workflow is loaded.
|
||||
It behaves the same as when you drop a workflow file from the file explorer.
|
||||
|
||||
Subtabs: All / Favorites / Groups / By Badge (by badge)
|
||||
|
||||
> **Tip**
|
||||
> If you opened the main Workflow Studio from a bookmark and "Send to Canvas" is unavailable,
|
||||
> the target workflow is shown in this W tab, so drag it from here (Chapter 4, section 4-6).
|
||||
|
||||
### M - Place a model
|
||||
When you drag a model onto the canvas, **a loader node with that model already chosen** is placed.
|
||||
|
||||
| Kind of model | Node placed |
|
||||
| --- | --- |
|
||||
| Checkpoint | Load Checkpoint (CheckpointLoaderSimple) |
|
||||
| LoRA | Load LoRA (LoraLoader) |
|
||||
| VAE | Load VAE (VAELoader) |
|
||||
| ControlNet | Load ControlNet Model (ControlNetLoader) |
|
||||
| UNET | Load Diffusion Model (UNETLoader) |
|
||||
| TextEncoder | Load CLIP (CLIPLoader) |
|
||||
| Hypernetwork | HypernetworkLoader |
|
||||
|
||||
The trouble of finding a node in a list, placing it, choosing the file name from a pull-down and so on goes away.
|
||||
After placing it, you connect it with lines to other nodes and use it.
|
||||
|
||||
Subtabs: All / Favorites / Groups / By Type (by kind)
|
||||
|
||||
> **Tip**
|
||||
> There is no loader node for Embeddings, so they cannot be placed on the canvas.
|
||||
|
||||
> **Tip: placing LoRAs in bulk**
|
||||
> In a LoRA group, an item called **All N LoRAs** (N is the number of LoRAs in the group) is shown.
|
||||
> If you drag it, a `Lora Loader (LoraManager)` node containing all the LoRAs in the group is placed.
|
||||
> (The custom node **ComfyUI-Lora-Manager** is required)
|
||||
|
||||
### P - Place a prompt
|
||||
When you drag a saved preset (Chapter 7) onto the canvas,
|
||||
a **Prompt Text (WFS)** node containing the positive and negative prompts is placed.
|
||||
This is a node bundled with Workflow Studio, and it outputs the prompt text.
|
||||
|
||||
If you only want to paste the prompt, you can copy it with the **P** (positive) / **N** (negative) buttons on the item.
|
||||
|
||||
Subtabs: All / Favorites / Categories (by category) / Groups
|
||||
|
||||
### N - Place a node
|
||||
When you drag a node onto the canvas, that node is placed.
|
||||
If you register frequently used nodes as favorites in the **Nodes** tab of the main Workflow Studio, you can take them out quickly.
|
||||
In the N tab's list, **Alt + click** a node to switch its favorite on or off on the spot (v0.8.0 and later).
|
||||
The N tab opens faster than before, because it shows the list first without waiting for the nodes' detailed information.
|
||||
|
||||
Subtabs: All / Favorites / Groups / Sets / Category (by category) / Package (by custom-node package)
|
||||
|
||||
### Node sets - saving a frequently used combination as a whole
|
||||
A **node set** is a bundle of several nodes and their connections saved together.
|
||||
If you save "a group of nodes you always use in the same combination", you can place the whole thing with one drag.
|
||||
|
||||
**Making a node set**
|
||||
1. On ComfyUI's node screen, select the nodes you want to save (click while holding the Ctrl key, or drag to select a range).
|
||||
2. **Right-click** on the selected nodes and choose **Save as Node Set**.
|
||||
3. Save it with a name.
|
||||
|
||||
**Using a node set**
|
||||
From the **Sets** subtab of the WS Library's **N** tab, drag it onto the canvas.
|
||||
|
||||
> **Example**
|
||||
> It is handy to make sets of parts you want to add afterwards to any workflow,
|
||||
> such as "a group of nodes that automatically corrects faces" or "a group of nodes for upscaling".
|
||||
|
||||
---
|
||||
|
||||
## 9-3 Inspecting the contents of an image in the Information tab
|
||||
|
||||
The **I (Information)** tab is a feature that inspects image and workflow files and lets you
|
||||
**place the models and prompts that were used straight onto the canvas**.
|
||||
Think of it as the Gallery's metadata (Chapter 8, section 8-2) made usable inside the node screen.
|
||||
|
||||
### How to use it
|
||||
1. Open the **I** tab of the WS Library.
|
||||
2. Drop the file you want to inspect (a PNG, WebP, MP4 or WebM image or video made in ComfyUI, or a workflow JSON) onto the panel.
|
||||
3. The materials found in the file are shown, split into subtabs.
|
||||
|
||||
| Subtab | Contents | When dragged onto the canvas |
|
||||
| --- | --- | --- |
|
||||
| model | Checkpoint, VAE, Diffusion Model, Text Encoder | The corresponding loader node is placed |
|
||||
| lora | The LoRAs used and their strength | A LoRA loader node is placed |
|
||||
| Prompts | Positive (POS) and negative (NEG) prompts | A text-encode node containing the prompt is placed |
|
||||
|
||||
Clicking a prompt item shows the whole text, and **Copy** also copies it.
|
||||
|
||||
### When it is handy
|
||||
- "I want to build a new workflow with the same model and LoRAs as that image I made before"
|
||||
- "I just want to borrow the prompt from a workflow I found online"
|
||||
- "Which LoRA did this image use, and at what strength?"
|
||||
|
||||
The point is that rather than loading the whole workflow, **you can take out and use only the parts you need**.
|
||||
|
||||
---
|
||||
|
||||
## 9-4 Changing the look of the WS Library
|
||||
|
||||
From the **gear button** inside the WS Library panel, you can set the panel's colors.
|
||||
|
||||
- The panel's background color
|
||||
- The sub-header's background color
|
||||
- Text color, and the color of auxiliary text
|
||||
- Border color
|
||||
|
||||
If you adjust them to match ComfyUI's theme, you can use it without any sense of mismatch.
|
||||
The settings are saved in the browser and applied the next time you open it.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - The WS Library is a "shelf of materials" that opens beside ComfyUI's node screen
|
||||
> - From the W, N, M and P tabs, you can place workflows, nodes, models and prompts by dragging (or double-clicking)
|
||||
> - If you drag a model from the M tab, a loader node with the model already chosen is placed
|
||||
> - With "Save as Node Set" in the right-click menu, you can save a combination of frequently used nodes
|
||||
> - If you drop an image onto the I tab, you can take out and use the models, LoRAs and prompts that were used
|
||||
@@ -0,0 +1,442 @@
|
||||
# Chapter 10 AI TOOL - Making Local AI Your Assistant
|
||||
|
||||
Workflow Studio can work together with **local AI (large language models)** running inside your computer.
|
||||
If you prepare local AI, you can do things like the following.
|
||||
|
||||
- Translate what you thought up in your own language into an English prompt
|
||||
- Chat with the AI and have it come up with prompt ideas
|
||||
- Show it an image and have it make a description, prompt or tags for that image
|
||||
- Have it make candidate lists for wildcards
|
||||
- Pick a **skill** to switch what it is good at: writing prompts, storyboards, troubleshooting and more (v0.9.5+)
|
||||
- Have it check whether a generated image matches your request and fix only what is missing (v0.9.5+)
|
||||
- Read the text shown in an image (OCR, v0.9.5+)
|
||||
|
||||
Because it runs inside your own computer rather than as a service on the internet,
|
||||
a big feature is that **what you type and the images you use are not sent outside**.
|
||||
|
||||
> **Tip**
|
||||
> AI features are "nice to have" features. Even if you do not use them, all of Workflow Studio's basic features are available.
|
||||
> If your computer has room to spare (especially GPU memory), please give them a try.
|
||||
|
||||
---
|
||||
|
||||
## 10-1 Preparing local AI software
|
||||
|
||||
### Supported software
|
||||
Workflow Studio supports the following four pieces of local AI software.
|
||||
|
||||
| Software | Characteristics | Default address |
|
||||
| --- | --- | --- |
|
||||
| **Ollama** | The standard. Easy from downloading models to running them | `http://localhost:11434` |
|
||||
| **LM Studio** | Operated from a screen. You can also look for models inside the app | `http://localhost:1234` |
|
||||
| Lemonade Server | Easy to run on AMD GPUs and NPUs | `http://localhost:13305` |
|
||||
| Unsloth | Requires extra setup such as an API key (for advanced users) | `http://localhost:8888` |
|
||||
|
||||
For beginners, **Ollama** is recommended. This chapter explains using Ollama as the example.
|
||||
|
||||
> **Tip**
|
||||
> `localhost` in the address means "this computer", and the number after it is the **port number** (Chapter 2, section 2-4).
|
||||
> Just like ComfyUI, AI software also listens on its own fixed port number.
|
||||
|
||||
### Preparing Ollama
|
||||
1. From Ollama's official site, download the installer for your OS and install it.
|
||||
2. Start Ollama (on Windows it starts automatically after installation and stays resident in the taskbar's notification area).
|
||||
3. Download the AI model you will use. In Command Prompt or similar, type the following.
|
||||
|
||||
```
|
||||
ollama pull (model name)
|
||||
```
|
||||
|
||||
Model names can be checked in the model list on Ollama's official site.
|
||||
|
||||
> **Guide to choosing a model**
|
||||
> - **You also want the image-reading feature (VLM)** -> choose a model that supports image input (vision)
|
||||
> (in the author's testing, image generation and image reading work well with Gemma 4 `e4b` and similar)
|
||||
> - **GPU memory is small** -> choose a model with a small parameter count (a few B, that is a few billion)
|
||||
> - If translation accuracy matters, choose a model that is good at your language
|
||||
>
|
||||
> The world of AI models changes quickly, so please try newer models too.
|
||||
|
||||
> **Caution: competing for GPU memory**
|
||||
> Local AI also uses **GPU memory (VRAM)**, just like image generation.
|
||||
> If you run both at the same time, memory runs short and image generation slows down or errors out.
|
||||
> Countermeasures are explained in 10-3.
|
||||
|
||||
---
|
||||
|
||||
## 10-2 Using the AI TOOL tab
|
||||
|
||||
Open the **AI TOOL** tab of the main Workflow Studio.
|
||||
The screen is divided into four parts, all shown at the same time.
|
||||
|
||||
| Part | What you can do |
|
||||
| --- | --- |
|
||||
| Translate | Translation between languages such as Japanese, English and Chinese |
|
||||
| Chat | Conversation with the AI. Skills switch what it is good at. If the conditions are met, generating images from the conversation too |
|
||||
| TOOLS | Reading images (VLM), reading text (OCR), making wildcards |
|
||||
| Settings | Settings for which AI software and model to use |
|
||||
|
||||
> **Tip: Translate and Settings can be collapsed (v0.9.5+)**
|
||||
> Press the arrow (◀ / ▶) at the right end of the Translate or Settings header to collapse that part into a thin strip, giving Chat and TOOLS more room.
|
||||
> Click anywhere on the strip to open it again. The collapsed state is kept the next time you open the tab.
|
||||
|
||||
### First: connect to the AI software in the settings
|
||||
1. In **Backend** under the settings, choose the software to use (for example Ollama).
|
||||
2. Check the address under **Connection**. If you use Ollama with its defaults, no change is needed.
|
||||
3. Press **Test Connection**. If it shows success, it is OK.
|
||||
4. Press **Refresh** next to **Model** and choose the model to use from the list.
|
||||
5. Press **Save Settings**.
|
||||
|
||||
> **Tip**
|
||||
> The settings here are shared with the **A (AI TOOL)** tab of the WS Library on the ComfyUI side.
|
||||
> On the other hand, the Prompt tab (Chapter 7) and the Tagger tab each have their own **gear button** settings.
|
||||
> If "it works in the AI TOOL tab but the Prompt tab says Disconnected", set the same thing with the gear button in the Prompt tab as well.
|
||||
|
||||
> **If it does not connect well**
|
||||
> - Check that software such as Ollama is running.
|
||||
> - For LM Studio, you need to **start the server** inside the app.
|
||||
> - Check that the port number in the address matches the software's settings.
|
||||
> - If the model list is empty, check that the model has been downloaded.
|
||||
|
||||
### Translate
|
||||
1. Choose the source and target languages (swap them with the **swap** button).
|
||||
2. Write the text in the input box and press **Translate**.
|
||||
3. **Copy** the result and paste it into GenerateUI or a preset.
|
||||
|
||||
> **Tip**
|
||||
> Text read in TOOLS (such as OCR in 10-8) can be put straight into this input box with **To Translation** in TOOLS.
|
||||
|
||||
> **Tip**
|
||||
> Small models sometimes return the original text as it is without translating.
|
||||
> In that case a message says the model returned the original text, so try another model.
|
||||
|
||||
> **Tip**
|
||||
> You can also choose the translation model **PLaMo 3 Translate** (Chapter 5, section 5-10) as the model in Settings (v0.9.9 and later).
|
||||
> Besides Japanese, English and Chinese, **Free** accepts language names such as French, フランス語 or français for any language PLaMo supports. A target language PLaMo does not support (such as Portuguese) gives an error.
|
||||
> The same applies to the translation in WS Library's A tab.
|
||||
|
||||
### Chat
|
||||
Talk in the input box and send with **Enter** (**Shift + Enter** for a line break).
|
||||
It remembers the flow of the conversation, so follow-up requests such as "make it a brighter mood" also work.
|
||||
**Clear** resets the conversation.
|
||||
|
||||
> **Tip: generating images from chat (advanced)**
|
||||
> If you use a model that supports "Tool Calling",
|
||||
> just asking in chat "draw a cat" generates an image with the workflow loaded in GenerateUI, and it is shown inside the chat.
|
||||
> This feature can be turned on or off with the **Allow image generation** check next to the input box.
|
||||
|
||||
> **Tip**
|
||||
> Choose a **skill** in the dropdown above the chat to switch what the AI is good at (10-5).
|
||||
> The AI's replies are shown with headings, lists and code boxes formatted (v0.9.5+).
|
||||
|
||||
### TOOLS - reading images (VLM)
|
||||
With a model that supports image input (a VLM), you can have the AI read the contents of an image.
|
||||
|
||||
1. Drop an image (PNG / JPG / WebP) into the **TOOLS** frame.
|
||||
2. Choose a task.
|
||||
|
||||
| Task | Contents |
|
||||
| --- | --- |
|
||||
| Describe | Explains in text what is in the image |
|
||||
| Create prompt | Makes a prompt for reproducing the image |
|
||||
| Create tags | Writes out the elements of the image in tag format |
|
||||
| Create wildcards | Makes candidate wildcard lists by specifying a category name and count (no image needed) |
|
||||
| OCR (extract text) | Reads the text shown in the image (10-8) |
|
||||
|
||||
3. Press **Run**.
|
||||
4. Use the result with the buttons under it (v0.9.5+).
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| To Translation | Puts it into the Translate input box (opens Translate if it is collapsed) |
|
||||
| To Chat | Adds it to the chat input box |
|
||||
| Copy | Copies it to the clipboard |
|
||||
|
||||
> **Example**
|
||||
> When you want to imitate the mood of an illustration you like, run "Create prompt" on that image
|
||||
> and try the resulting prompt in GenerateUI - you can use it like that.
|
||||
|
||||
### It can be used inside the ComfyUI screen too
|
||||
The same AI features can also be used in the **A** tab of the WS Library on the ComfyUI side.
|
||||
Handy when you want to translate something briefly while working on the node screen.
|
||||
|
||||
---
|
||||
|
||||
## 10-3 Freeing VRAM and managing memory
|
||||
|
||||
As touched on in 10-1, local AI and image generation compete for GPU memory (VRAM).
|
||||
Ollama keeps a model that has been used in memory for a while,
|
||||
so if you generate images after using the AI, memory may run short.
|
||||
|
||||
### Freeing manually
|
||||
Pressing **Unload Model** in the AI TOOL settings takes the AI model out of memory and frees VRAM.
|
||||
(With Ollama, all loaded models are unloaded)
|
||||
|
||||
### Freeing automatically before generation
|
||||
In **VRAM Management (Ollama)** in the Settings tab, you can set Ollama's models to be unloaded automatically just before image generation.
|
||||
|
||||
| Behavior before generation | What it does |
|
||||
| --- | --- |
|
||||
| Do nothing | Does not unload automatically |
|
||||
| Unload what is needed only when free memory is below the target | Unloads only when free VRAM is less than the **Target free VRAM (GB)** |
|
||||
| Unload everything every time | Always unloads everything at every generation |
|
||||
|
||||
> **Tip**
|
||||
> If you can't decide which to choose, "Unload what is needed only when free memory is below the target" is recommended.
|
||||
> For the target free VRAM, enter a rough value of the amount of memory image generation needs (it differs by the model you use).
|
||||
> The **Unload all now** button can also free it immediately, on the spot.
|
||||
|
||||
> **Caution**
|
||||
> This automatic unloading applies to Ollama only. If you use LM Studio or similar, free memory manually.
|
||||
|
||||
---
|
||||
|
||||
## 10-4 Security notes
|
||||
|
||||
### By default it connects only to AI on "your own computer"
|
||||
For safety, the AI software Workflow Studio connects to (Ollama, LM Studio and so on) is by default limited to
|
||||
**software running inside the same computer (localhost / 127.0.0.1)**.
|
||||
|
||||
### When you want to use AI on another computer
|
||||
If you want to run Ollama on another computer on your home LAN (such as a computer with a strong GPU) and use it,
|
||||
register the address of the computer you want to connect to in the environment variable **`WFS_ALLOWED_BACKEND_HOSTS`**, and restart ComfyUI.
|
||||
|
||||
```
|
||||
WFS_ALLOWED_BACKEND_HOSTS=192.168.1.20:11434
|
||||
```
|
||||
|
||||
If there are several, list them separated by commas (,).
|
||||
How to set environment variables differs depending on how you start ComfyUI.
|
||||
|
||||
### If you start ComfyUI with `--listen`
|
||||
If you start ComfyUI with the `--listen` option, other computers and smartphones on the same network
|
||||
can also access ComfyUI (and Workflow Studio).
|
||||
In that case, anyone who can access it can use your AI software and images.
|
||||
|
||||
> **Caution**
|
||||
> Use `--listen` **only on a trusted network**, such as a home LAN.
|
||||
> Do not use it in cafes or on public Wi-Fi.
|
||||
|
||||
---
|
||||
|
||||
## 10-5 Skills - switching what the chat is good at (v0.9.5+)
|
||||
|
||||
A **skill** is a set of instructions that tells the chat what role to play and how to answer.
|
||||
When you pick one in the dropdown above the chat (it starts as "-- No skill --"), its instructions are sent along with every message
|
||||
(the instructions themselves are not shown in the chat).
|
||||
|
||||
### Built-in skills
|
||||
Workflow Studio comes with the following six skills.
|
||||
|
||||
| Skill | What it does | How you use the result |
|
||||
| --- | --- | --- |
|
||||
| T2I Prompt Enhancer | Turns a short idea into a prompt written the way the model in your GenerateUI workflow understands best | Buttons under the prompt |
|
||||
| Image to Prompt | Writes a prompt that recreates the image you put in TOOLS (needs a model that can read images) | Buttons under the prompt |
|
||||
| Storyboard to Lab | Splits a story into one prompt per scene, ready for continuous generation in Lab | **Open in Lab** |
|
||||
| Workflow Doctor | Explains the workflow and helps you find the cause of errors (10-7) | Answers in text |
|
||||
| SVG Icon Generator | Draws simple icons and illustrations as SVG (vector images) | Preview and **Save to Gallery** |
|
||||
| Skill Creator | Asks you questions and builds a new skill (10-9) | **Save as new skill** |
|
||||
|
||||
### Making a prompt and generating with it (T2I Prompt Enhancer)
|
||||
1. Load the workflow you want to use into GenerateUI. The skill looks at that workflow's model to decide how to write the prompt.
|
||||
2. In the AI TOOL chat, choose **T2I Prompt Enhancer** in the dropdown.
|
||||
3. Describe the picture in your own words, for example "a cat holding an umbrella in rainy Tokyo at night, neon", and send it.
|
||||
4. The reply shows which writing style was chosen (the `Style:` line) and the English prompt in a box. Use the buttons under it.
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| Apply to GenerateUI | Writes the positive / negative prompt into GenerateUI |
|
||||
| Generate with this prompt | Generates right away and shows the image in the chat |
|
||||
| Copy | Copies the positive prompt |
|
||||
|
||||
The writing style depends on the model:
|
||||
|
||||
| Model in GenerateUI | How the prompt is written |
|
||||
| --- | --- |
|
||||
| Qwen-Image, FLUX, Z-Image, etc. | English sentences (the negative is left empty when CFG is about 1) |
|
||||
| Illustrious, NoobAI, Pony, etc. (SDXL anime models) | Danbooru tags (starting with quality tags) |
|
||||
| Realistic SDXL, SD1.5 | Short phrases separated by commas |
|
||||
|
||||
> **Tip**
|
||||
> Follow-up requests such as "make it more night-like" or "two cats" give you the whole updated prompt again.
|
||||
|
||||
> **Tip: the prompt currently in GenerateUI is not sent**
|
||||
> Normally the prompt already typed into GenerateUI is not passed to the AI, because small models tend to mix it (for example an old negative prompt) into the new prompt.
|
||||
> It is passed only when your message refers to it, such as "**improve the current prompt**".
|
||||
|
||||
### Generating a storyboard in a row (Storyboard to Lab)
|
||||
1. Load a text-to-image (T2I) workflow into GenerateUI.
|
||||
2. In the chat, choose **Storyboard to Lab** and ask, for example, "A tabby kitten wakes up in the morning, chases a butterfly in the garden, and naps on the veranda in the evening. 3 scenes."
|
||||
3. The reply shows the scene list and an **Open in Lab (3 scenes)** button. Press it.
|
||||
4. GenerateUI's **Lab** (Chapter 5, section 5-7) opens with the plan loaded as an unsaved plan.
|
||||
T2I mode is on, Batch is the number of scenes, and the Prompt column has one keyframe per scene. The KSampler column holds a seed shared by all scenes (`Seed ...`).
|
||||
5. Check it and press **Run**. To keep it, use **Plan Save**.
|
||||
|
||||
> **Tip**
|
||||
> The characters' looks (hair, clothing colors and so on) are written with the same words in every scene, and the seed is fixed,
|
||||
> so the same character is easier to keep across scenes.
|
||||
|
||||
### Automatic check of replies
|
||||
For skills that output a prompt or an SVG, Workflow Studio checks the shape of the reply automatically.
|
||||
If, for example, the prompt box is broken or the SVG is not closed, a dashed notice "**Auto-check: ...**" appears and the AI is asked once to fix it.
|
||||
Replies that do not try to output a prompt or SVG (for example when you only asked for an explanation) are left alone.
|
||||
|
||||
### Managing skills (pencil button)
|
||||
The pencil button to the right of the dropdown opens the skill list.
|
||||
|
||||
- **+ New** writes a new skill, the pencil on each row edits or deletes it, and **Import** adds other skills (10-9)
|
||||
- Skills that ship with Workflow Studio are marked **built-in**.
|
||||
Built-in skills you have not edited are updated automatically when Workflow Studio is updated.
|
||||
An edited one is not replaced automatically; when a newer version exists, a **↻ (update)** button appears.
|
||||
- Skill files are saved in `ComfyUI/user/default/Workflow-Studio/ai_skills/`.
|
||||
|
||||
> **Caution**
|
||||
> Replacing a skill with the newer version via ↻ (update) discards your edits to that skill.
|
||||
|
||||
---
|
||||
|
||||
## 10-6 Checking generated images and fixing them (v0.9.5+)
|
||||
|
||||
This feature has the AI check whether an image generated from the chat matches what you asked for.
|
||||
If something is missing, you can redraw with a prompt that fixes just that.
|
||||
|
||||
### Preparation: a judging model
|
||||
The check does not use a normal chat model but an Ollama **decision model** (a model that answers yes/no questions with a probability).
|
||||
Get one that can read images with `ollama pull` (the author confirmed `d3:4b`, `clef-flash` and `clef`).
|
||||
|
||||
> **Tip: why a separate model**
|
||||
> In the author's tests, a chat model shown an image sometimes answered "yes" to "is the person holding a sword?" when there was no sword.
|
||||
> A decision model returns the probability of "yes" as a number, so uncertain answers can be told apart.
|
||||
|
||||
1. In AI TOOL's **Settings**, under **Chat Image Check**, choose the **Judge model (Ollama decision model)** (press **Refresh** if the list is empty).
|
||||
2. Choose **Automatic fix rounds**. **0 (fix by button)** is recommended at first.
|
||||
3. Press **Save Settings**.
|
||||
|
||||
### How to use it
|
||||
1. Tick **Check images** below the chat input.
|
||||
2. Generate an image from the chat (the AI's image generation, or **Generate with this prompt**).
|
||||
3. A result card appears under the image. The AI splits your request into 3 to 8 check items and checks them one by one.
|
||||
|
||||
| Mark | Meaning |
|
||||
| --- | --- |
|
||||
| ✔ (met) | Probability of "yes" is 85% or more |
|
||||
| ? (unclear) | 30-85% |
|
||||
| ✘ (not met) | Less than 30% |
|
||||
|
||||
4. If there is a ✘ or ?, press **Fix missing items and regenerate**. Only the unmet items are sent to the AI, which rewrites the prompt and draws again.
|
||||
|
||||
If you set Automatic fix rounds to 1-3, this repeats automatically up to that number of times.
|
||||
|
||||
> **Tip: SVGs can be checked too**
|
||||
> SVGs drawn with SVG Icon Generator are turned into an image and checked for "does it look like what you asked for" and "are its parts (beak, wings and so on) drawn".
|
||||
> **Ask to redraw unclear parts** asks the AI to fix them.
|
||||
|
||||
> **Caution**
|
||||
> - The check is not perfect. It is especially weak at "who is wearing what",
|
||||
> so when there are several people it automatically adds reverse check items such as "the hat is not on the girl" to compensate.
|
||||
> - How well the redraw works depends on the chat model. If it does not improve, change your request or the model.
|
||||
> - The judging model also uses GPU memory (about 6 GB for `d3:4b`). Once loaded, a check takes a few seconds.
|
||||
|
||||
---
|
||||
|
||||
## 10-7 Asking about workflow problems (Workflow Doctor, v0.9.5+)
|
||||
|
||||
**Workflow Doctor** is a skill that explains the workflow loaded in GenerateUI and helps you find the cause of errors.
|
||||
Instead of relying only on the AI's knowledge, every time you send a message Workflow Studio passes it the following facts it has actually checked.
|
||||
|
||||
| What is checked | Example |
|
||||
| --- | --- |
|
||||
| Nodes that are not installed | Names of nodes ComfyUI does not know |
|
||||
| Files and values not in ComfyUI's lists | Models or LoRAs that cannot be found (with similar names) |
|
||||
| Connections | Inputs that are not connected, links to nodes that do not exist |
|
||||
| The last execution error | The error recorded in ComfyUI's history |
|
||||
| ComfyUI's log | The latest error part of the log |
|
||||
|
||||
### How to use it
|
||||
1. Load the workflow you want to ask about into GenerateUI (for an error, run it once so the error is recorded).
|
||||
2. In the chat, choose **Workflow Doctor**.
|
||||
3. Ask, for example, "check this workflow for problems before I run it" or "what caused the last error?".
|
||||
|
||||
> **Example**
|
||||
> For a workflow whose LoRA file had been moved to another folder, it answered
|
||||
> "the file specified in the `temp\` folder cannot be found; a file with the same name exists in `Illustrious\`", including how to fix it.
|
||||
|
||||
> **Tip**
|
||||
> The checked facts are rebuilt every time you send a message. After fixing something, just ask again.
|
||||
|
||||
---
|
||||
|
||||
## 10-8 Reading text in images (OCR, v0.9.5+)
|
||||
|
||||
In TOOLS you can read the text shown in an image. The text you read can be sent straight to translation or the chat.
|
||||
|
||||
### Preparation: an OCR model
|
||||
OCR uses a model that is good at reading text, separate from the chat model. Prepare one in Ollama.
|
||||
These are the author's results on an image mixing horizontal and vertical Japanese and English text.
|
||||
|
||||
| Model | Time | Result |
|
||||
| --- | --- | --- |
|
||||
| LightOnOCR-3 (`maternion/LightOnOCR-3:4b`) | About 1 s (6-8 s with positions) | Read everything |
|
||||
| PaddleOCR-VL (manga version) | About 4 s | Read everything |
|
||||
| DeepSeek-OCR family | About 14 s | Read everything (always with positions) |
|
||||
| PaddleOCR-VL 1.6, GLM-OCR | About 11-15 s | Missed the vertical text |
|
||||
|
||||
### How to use it
|
||||
1. Drop an image into TOOLS.
|
||||
2. Choose **OCR (extract text)** as the task.
|
||||
3. Choose the **OCR model**. Ollama models whose name contains "ocr" are listed, best first (press ↻ if the list is empty).
|
||||
4. Press **Run**. The text is shown in the result box, one line each.
|
||||
|
||||
> **Tip: Show text positions**
|
||||
> Tick **Show text positions** to draw numbered boxes over the image so you can see where each text is.
|
||||
> Clicking a box selects the same line in the result box.
|
||||
> Only LightOnOCR and the DeepSeek-OCR family can return positions; for other models this check cannot be selected.
|
||||
|
||||
> **Tip**
|
||||
> If the result is empty or the same line repeats, it is automatically read once more in grayscale.
|
||||
|
||||
---
|
||||
|
||||
## 10-9 Making and importing skills (advanced, v0.9.5+)
|
||||
|
||||
### Making one with Skill Creator
|
||||
In the chat, choose **Skill Creator** and say something like "I want a skill that is good at ...".
|
||||
The AI asks questions one at a time; answer them and you get a draft skill.
|
||||
Press **Save as new skill** under the reply; the skill list opens with the draft filled in, so check it and save.
|
||||
You can also ask it to improve an existing skill, for example "this part of this skill does not work well".
|
||||
|
||||
### What a skill file contains
|
||||
A skill is a text file (`.md`). Write the name and description in the part enclosed by `---` at the top, and the instructions for the AI below it.
|
||||
Under `metadata:` in the top part you can also write these settings (all optional).
|
||||
|
||||
| Setting | Meaning |
|
||||
| --- | --- |
|
||||
| `output` | `prompt` or `svg`. Checks the shape of the reply automatically and adds the buttons |
|
||||
| `tools` | `[]` stops the AI from starting image generation on its own |
|
||||
| `temperature` | How varied the answers are (0.3 stable, 0.9 free) |
|
||||
| `think` | Turns Thinking mode on or off just for this skill |
|
||||
| `context` | Information passed to the AI: `workflow` (workflow summary), `current-prompt` (the current prompt), `workflow-detail` / `logs` (the same checks as Workflow Doctor) |
|
||||
|
||||
### Importing other skills
|
||||
**Import** in the skill list adds:
|
||||
|
||||
- a single `.md` skill
|
||||
- a ZIP of a folder containing `SKILL.md` (the **Agent Skills** format shared by many AI tools)
|
||||
|
||||
Text files in the folder (reference material and so on) are imported too, and the AI reads them only when it needs them
|
||||
(this needs a model that supports tool calling).
|
||||
|
||||
> **Caution**
|
||||
> - Only text files are imported. Programs such as scripts are not imported and never run.
|
||||
> - A skill's instructions are passed to the AI as they are. Read a skill made by someone else before using it.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - If you prepare local AI (Ollama and so on), you can translate, chat and read images. What you type is not sent outside
|
||||
> - In the AI TOOL tab's settings, choose the backend, address and model, and test the connection
|
||||
> - The Prompt tab and Tagger tab have separate gear-button settings
|
||||
> - AI and image generation compete for VRAM. Free it with "Unload Model" or "VRAM Management (Ollama)"
|
||||
> - By default it connects only to AI on your own computer. Use `--listen` only on a trusted network
|
||||
> - **Skills** switch what the chat is good at. T2I Prompt Enhancer writes a prompt that buttons apply to GenerateUI or generate with
|
||||
> - With **Check images** on, a decision model checks the generated image and you can redraw fixing only what is missing
|
||||
> - Workflow Doctor helps with workflow problems based on facts it has actually checked
|
||||
> - **OCR (extract text)** in TOOLS reads the text in an image and sends it to translation or the chat
|
||||
@@ -0,0 +1,356 @@
|
||||
# Chapter 11 Even More Features (Overview)
|
||||
|
||||
By now, you have covered the basic usage of Workflow Studio from start to finish.
|
||||
This chapter introduces the tabs and features not yet covered, focusing on **what you can do with them**.
|
||||
You do not need to use everything right now. Keep in the back of your mind that "you can do this too",
|
||||
and try them when you need them.
|
||||
|
||||
---
|
||||
|
||||
## 11-1 The Nodes tab - listing the installed nodes
|
||||
|
||||
In the **Nodes** tab, you can list all the nodes installed in ComfyUI.
|
||||
|
||||
- **Search and filtering** - narrow down by name, category and package (which custom node it belongs to). Even with thousands of nodes, the list appears first and search runs lightly (v0.8.0 and later).
|
||||
- **Detail panel** - when you select a node, you can check the kinds of its inputs and outputs (the I/O specification).
|
||||
- **Favorites, groups and tags** - organize the nodes you use often.
|
||||
|
||||
Nodes you register as favorites or in groups here are shown in the **N** tab of the WS Library,
|
||||
and can be placed by dragging onto the node screen (Chapter 9).
|
||||
|
||||
> **When it helps**
|
||||
> "I installed a custom node but can't tell what nodes were added"
|
||||
> -> narrow down by **Package** and you can see the list of nodes contained in that custom node.
|
||||
|
||||
---
|
||||
|
||||
## 11-2 Tagger - automatically making tags from images
|
||||
|
||||
The **Tagger** tab is a feature that reads an image and automatically makes tags that describe it (such as `1girl, long hair, smile`).
|
||||
It can be used for making prompts for anime-style models, and for tagging images for training.
|
||||
|
||||
### Three subtabs
|
||||
| Subtab | What you can do |
|
||||
| --- | --- |
|
||||
| Single | Make tags from one image |
|
||||
| Batch | Tag the images in a folder in bulk (it can also write out a `.txt` file with the same name as each image) |
|
||||
| DB | Search and edit the records of images you tagged |
|
||||
|
||||
### How to use it (Single)
|
||||
1. Drop an image onto the preview area (you can also open it from the **Tagger** button in the Gallery's detail panel).
|
||||
2. Choose the model to use and press the run button to make tags.
|
||||
3. Use the result.
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| GenUI:P | Adds it to GenerateUI's positive prompt |
|
||||
| Send to Prompt | Adds it to the positive prompt in the Prompt tab |
|
||||
|
||||
With the **threshold** slider, you can adjust how certain a tag must be to be shown.
|
||||
Raising the number leaves only fewer, more certain tags; lowering it gives more.
|
||||
|
||||
### Preparation needed
|
||||
To use Tagger, you need the following preparation.
|
||||
|
||||
1. Put a **model for tagging** (such as WD Tagger) in ComfyUI's `models/tagger/(model name)/` folder.
|
||||
Put a `.onnx` file and `selected_tags.csv` in the folder.
|
||||
**PixAI Tagger v1.0** (v0.8.10 and later) also works. From the community conversion on Hugging Face (`Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8`),
|
||||
put one `.onnx` file (`model_fp16.onnx` is recommended for a GPU) together with `selected_tags.csv` and `preprocess.json` in one folder, and it is detected automatically.
|
||||
When you choose a PixAI model, the threshold sliders switch to PixAI's recommended values (0.17 / 0.27).
|
||||
2. Install the **extra Python packages**.
|
||||
Use the `requirements.txt` in the Workflow Studio folder.
|
||||
|
||||
```
|
||||
(ComfyUI's Python) -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The location of "ComfyUI's Python" differs by environment. Check the table in Chapter 2, section 2-6
|
||||
(for example, for the Windows portable version it is `python.exe` inside the `python_embeded` folder).
|
||||
|
||||
> **Caution**
|
||||
> Installing into the Python that was already on your computer cannot be used from ComfyUI.
|
||||
> Always install into **the Python that ComfyUI uses**.
|
||||
> If you are not sure, there is also a method of installing packages with a ComfyUI Manager feature.
|
||||
|
||||
> **Caution: check that it runs on the GPU**
|
||||
> On an NVIDIA GPU, Tagger takes less than a second per image, but on the CPU, PixAI takes about 20 seconds per image.
|
||||
> If it is very slow, check that the GPU package is installed correctly (Chapter 12, section 12-5).
|
||||
|
||||
> **Tip**
|
||||
> You can also combine it with the local AI image reading (VLM) from Chapter 10 to make tags that combine both results.
|
||||
|
||||
---
|
||||
|
||||
## 11-3 Image Edit - touching up generated images
|
||||
|
||||
The **Image Edit** tab is a simple image-editing tool with layers.
|
||||
You can put text on a generated image, or make a mask for redrawing part of it.
|
||||
|
||||
### Main features
|
||||
| Tool | What you can do |
|
||||
| --- | --- |
|
||||
| Select | Move, scale and rotate layers |
|
||||
| Draw | Draw with a pen (pen-tablet pressure is also supported) |
|
||||
| Text | Put in text |
|
||||
| Shape | Draw rectangles, circles and lines |
|
||||
| Fill | Fill and gradients |
|
||||
| Mask | Specify the part you want redrawn (making a mask) |
|
||||
| Blur | Blur and mosaic |
|
||||
| BG Remove | Remove the background |
|
||||
| Inpaint | Redraw the masked part with the GenerateUI workflow |
|
||||
|
||||
- Open an image by drag and drop, **File Open**, or the **Image Edit** button in the Gallery.
|
||||
- Save the edited image with **Save PNG** (download), **Save to Gallery** and so on.
|
||||
- Undo with **Ctrl + Z**, redo with **Ctrl + Y**.
|
||||
|
||||
### Draw colors and shortcuts (v0.9.3+)
|
||||
When you choose **Draw**, a **color palette** appears in the second row of the tool options.
|
||||
|
||||
- It shows **Color 1 / Color 2**, 20 basic colors and 20 custom slots. Click a color to paint with it.
|
||||
- Click **+** or an empty slot to register the current color as a custom color. Right-click a color to register, edit or delete it (basic colors can be changed too, and returned to their original color).
|
||||
Registered colors are saved per browser.
|
||||
- The **eyedropper** button next to **Color** on the right picks a color from the canvas (Esc cancels).
|
||||
- The starting colors are black (Color 1) and white (Color 2).
|
||||
|
||||
| Key (while Draw is active) | What it does |
|
||||
| --- | --- |
|
||||
| D / E | Draw / Erase |
|
||||
| X | Swap Color 1 and Color 2 |
|
||||
| [ / ] | Make the brush smaller / larger |
|
||||
| B | Open or close the brush picker (Esc also closes it) |
|
||||
|
||||
### Inpainting (redrawing part of an image)
|
||||
The handiest use of Image Edit is **inpainting**.
|
||||
It is for when "I want to redraw only the face" or "I want to fix only the place where the hand broke".
|
||||
|
||||
1. Load an inpainting workflow (such as the bundled sample `ws_inpaint_basic`) into GenerateUI.
|
||||
2. Open the image in Image Edit and paint the part you want redrawn with **Mask**.
|
||||
3. Run it with the **Inpaint** tool, and only the painted part is redrawn.
|
||||
|
||||
> **Tip**
|
||||
> To use a higher-quality version of background removal, or more advanced masking,
|
||||
> the author's separate custom node **comfyui-mask-editor-one** is needed.
|
||||
> Saving and loading in PSD format requires an extra Python package (psd-tools).
|
||||
|
||||
---
|
||||
|
||||
## 11-4 Video - making and joining videos
|
||||
|
||||
The **Video** tab is a dedicated screen for making short videos from images or text using video-generation models.
|
||||
|
||||
### Supported video-generation models
|
||||
- **MiniMax H3** (image to video)
|
||||
- **LTX-2.5** (text to video, image to video, and video from specified first and last images)
|
||||
- **Wan2.2** (text to video)
|
||||
- **FastH3 OmniRef / Pika-H3-Flex** (MiniMax H3 reference to video, v0.8.14 and later)
|
||||
|
||||
You load a matching workflow with **Load in Video** in the Workflow tab and use it (Chapter 4, section 4-6).
|
||||
|
||||
### Main features
|
||||
| Subtab | What you can do |
|
||||
| --- | --- |
|
||||
| Plan | Divide a video into "blocks", decide the prompt, length and first/last image for each block, and generate in order |
|
||||
| Edit | Line up generated videos and images, then cut, join, add text and BGM, and export |
|
||||
| Asset | List generated videos, images and audio. Audio can be set as BGM or added as sound (the Audio Only type filter shows audio only) |
|
||||
|
||||
There are also the **Frame** feature, which saves a favorite scene of a video as a still image, and **GIF**, which converts a video to an animated GIF.
|
||||
|
||||
> **Tip**
|
||||
> The **Prompt** field of a Plan block also has the Prompt Translation globe button (Chapter 5, section 5-10).
|
||||
> With **Video Plan - Run** turned on in Settings, you can run the plan with prompts in your own language and send only English.
|
||||
|
||||
> **Note**
|
||||
> Up to v0.9.8, with workflows saved by newer ComfyUI versions, the prompt, length (Duration) and Turbo settings entered in Plan were not applied, and the values saved in the workflow were used instead (fixed in v0.9.9).
|
||||
> Because Plan values are applied now, running the same plan as before may give different results.
|
||||
|
||||
### Register as Asset and audio conversion (v0.8.9 and later)
|
||||
- **Register as Asset** - a button in the Video Source panel on the left. It copies a video from your PC into the output folder (`output/video`) and registers it in the "Video Assets" group.
|
||||
Once registered, it appears in the Asset subtab list and can be used in Edit and elsewhere.
|
||||
Videos already in the output folder (ones chosen in Asset or generation results) are only tagged into the group, not copied. If the name already exists, a number is added after it.
|
||||
- **Convert audio...** - a button next to the crop buttons in the Edit subtab. It converts only the audio of the selected video clip to FLAC, MP3, Opus, M4A or WAV and saves it into `output/video`.
|
||||
It is the same dialog as Gallery's Convert. Only video clips that contain audio are covered.
|
||||
- Pressing the audio-conversion button opens the dialog below. Choose the **output format** (such as FLAC), the **sample rate** and the **channels**, then press **Convert**.
|
||||
If you check the option to carry over the embedded prompt/workflow tags, the original video's information is kept in formats that can hold tags (such as FLAC).
|
||||
|
||||
- The Video Source panel is a little wider, the file name shows on one line, and the buttons sit in a single row.
|
||||
|
||||
### Reference to video (FastH3 OmniRef / Pika-H3-Flex) (v0.8.14 and later)
|
||||
These workflows make a video from **reference images** instead of a first and last frame,
|
||||
so the image fields in the Plan subtab are renamed:
|
||||
|
||||
- **Reference Image** - the first reference image (the usual First Image field)
|
||||
- **Reference Image 2 (optional)** - the second reference image (the usual Last Image field)
|
||||
|
||||
- Prompt, duration (seconds), aspect ratio and so on work the same as for other video workflows.
|
||||
- A block whose image field is left empty with **None** or **Clear** is generated without that image, even if the workflow has a reference image wired in.
|
||||
Blocks added with "+ Block" start with empty images, so set an image on each block that needs a reference.
|
||||
- Choosing **Use previous block's last frame** makes the last frame of the video actually generated by the previous block the first reference image of that block.
|
||||
- The Asset subtab buttons also change to "Set as Reference Image" and "Set as Reference Image 2".
|
||||
|
||||
> **Caution**
|
||||
> Video generation uses far more GPU memory and time than image generation.
|
||||
> Check the required specs of the model before trying it.
|
||||
|
||||
---
|
||||
|
||||
## 11-5 Main items of the Settings tab
|
||||
|
||||
In the **Settings** tab, you configure Workflow Studio as a whole.
|
||||
Items are collapsed, so click a heading to open it.
|
||||
Here we mainly introduce items not covered in earlier chapters.
|
||||
|
||||
| Item | Contents | Related chapter |
|
||||
| --- | --- | --- |
|
||||
| Theme | Screen color scheme and font | Chapter 3 |
|
||||
| UI Language / Summary Language | Display language | Chapter 2 |
|
||||
| Text Size | Size of text such as in the prompt fields | Chapter 3 |
|
||||
| Workflow Data Folder | The folder workflows are read from | Chapter 4 |
|
||||
| Gallery Output Folder | The folder shown in the Gallery | Chapter 8 |
|
||||
| Default Workflow | The workflow automatically loaded into GenerateUI when you open Workflow Studio | Below |
|
||||
| Default Checkpoint | Automatically replaces the model when loading a workflow into GenerateUI | Below |
|
||||
| CivitAI API Key | For fetching Civitai information (optional) | Chapter 6 |
|
||||
| Eagle integration (optional) | Automatically save generated images to Eagle | Chapter 8 |
|
||||
| VRAM Management (Ollama) | Unload AI models from memory before generation | Chapter 10 |
|
||||
| Prompt Translation (native language -> English) | Turn prompts in your own language into English with the globe button or automatic translation when running | Chapter 5, 5-10 |
|
||||
| Data Management | Backing up and restoring settings and data | Below |
|
||||
|
||||
### Default Workflow
|
||||
If you set the basic workflow you always use, it is loaded into GenerateUI automatically when you open Workflow Studio.
|
||||
You can also set it with the **Set as Default** button on the Workflow tab's detail screen.
|
||||
|
||||
### Default Checkpoint
|
||||
Workflows you get online often specify models you do not have (Chapter 5, section 5-3).
|
||||
If you set a **Default Checkpoint** and turn it on, each time you load a workflow into GenerateUI,
|
||||
the Checkpoint is automatically replaced with the specified model. It saves the trouble of choosing a model again.
|
||||
|
||||
### Data Management (backup)
|
||||
In **Data Management**, you can write out the information you organized in Workflow Studio (favorites, badges, groups, presets and so on)
|
||||
to a file (Export), and read it back to restore it (Import).
|
||||
|
||||
With **Full Backup (ZIP)**, you can back up everything together, including folder-based data such as styles and Lab plans.
|
||||
|
||||
> **Tip**
|
||||
> Always take a backup before buying a new computer or reinstalling ComfyUI.
|
||||
> The information you spent time organizing can be carried over as it is to the new environment.
|
||||
> (For safety, API keys are not included in backups)
|
||||
|
||||
## 11-6 A companion app: ComfyUI Comic Creator - making manga from generated images
|
||||
|
||||
**ComfyUI Comic Creator** (Comic Creator from here on) is a custom node **for making manga** that runs on ComfyUI,
|
||||
developed by the same author as Workflow Studio.
|
||||
It is software you install separately from Workflow Studio, but if you install both, they work together.
|
||||
|
||||
### What you can do with Comic Creator
|
||||
| Feature | Contents |
|
||||
| --- | --- |
|
||||
| Panel layout | Choose a template and make panels on a page (your own templates can also be registered) |
|
||||
| Placing images | Place images into panels by drag and drop |
|
||||
| Speech balloons and text | Vertical and horizontal dialogue, balloons of various shapes, sound-effect text |
|
||||
| Manga effects | Add speed lines, screen tones, halftone dots and so on |
|
||||
| Image editing | Image editing with layers (the Image tab) |
|
||||
| Automatic creation with AI | Draft a story, script and panel layout from a theme with AI (the Auto tab) |
|
||||
| Export | Output as JPEG / PNG / WebP / SVG / PDF / **EPUB** |
|
||||
|
||||
### Working together with Workflow Studio
|
||||
If you install both, the following integrations are possible.
|
||||
|
||||
- **Use Workflow Studio's Gallery inside Comic Creator**
|
||||
Workflow Studio's Gallery appears inside Comic Creator, and you can choose generated images and send them to panels with **Send CC Layout** (Layout tab panel / overlay) or to the Image tab with **Send CC Image**.
|
||||
(The Send CC buttons are shown only in a Gallery opened inside Comic Creator)
|
||||
- **Generate images from Comic Creator**
|
||||
From the Comic Creator screen, through Workflow Studio (GenerateUI), you can generate images to put in panels (T2I),
|
||||
or redraw based on a rough you drew or an image you placed (I2I and inpainting).
|
||||
From Comic Creator's Nanobanana+ tab (local mode), you can also run image-editing workflows that use several reference images (Flux.3 Klein, Qwen Image 2.1 and so on) (v0.8.12 and later).
|
||||
|
||||
With the flow "make images in Workflow Studio -> put them together as manga in Comic Creator",
|
||||
you can go from image generation to finished manga entirely inside ComfyUI.
|
||||
|
||||
### Installing and launching
|
||||
1. In ComfyUI Manager's **Install via Git URL**, enter the following URL and install.
|
||||
|
||||
```
|
||||
https://github.com/ketle-man/comfyui-comic-creator
|
||||
```
|
||||
|
||||
(For a manual install, `git clone` into `ComfyUI/custom_nodes/`, same as Workflow Studio)
|
||||
|
||||
2. Restart ComfyUI.
|
||||
3. Press the **CC** button added to ComfyUI's top menu, and Comic Creator opens in a new tab.
|
||||
|
||||
> **Tip**
|
||||
> For detailed usage of Comic Creator, see the **Help** tab inside the app (with search).
|
||||
|
||||
---
|
||||
|
||||
## 11-7 TagSpace - picking tags and prompts from a "map of meaning" (v0.9.0 and later)
|
||||
|
||||
The **TagSpace** tab lays out the prompts, tags and generated images inside Workflow Studio in a 3D space
|
||||
according to **closeness of meaning**. Things close together mean similar things, so you can browse the space
|
||||
("which tags are close to this mood?") and pick tags to build a prompt.
|
||||
|
||||
### What is in the space
|
||||
- **Cards** - the sample images of the Gallery's **ImagePrompt** (Chapter 8, section 8-4) with their prompts, shown as thumbnails
|
||||
- **Tags** - words taken (split at commas) from ImagePrompt prompts, prompts embedded in generated images, wildcards, the Tagger DB and the Prompt library, shown as dots colored by source
|
||||
- **Gallery points** - some of your generated images, shown as faint dots (each carries the tags of its own prompt)
|
||||
|
||||
### Preparation
|
||||
As with the Gallery's semantic search (Chapter 8, section 8-6), **Ollama** and **EmbeddingGemma 2** are needed.
|
||||
If the Gallery's semantic index was made with Gemini Embedding 2, rebuild it with EmbeddingGemma 2 first.
|
||||
|
||||
### Building the space
|
||||
Choose the settings in the toolbar and press **Build / Update**.
|
||||
|
||||
- **Tags** - how many tags to include. The sources take turns, each in order of how often a tag is used
|
||||
- **All tags** - when on, every candidate tag is used instead of the Tags number (up to 20,000)
|
||||
- **Min uses** - leaves out tags used fewer times than this. 2 removes one-off spelling variants and the like
|
||||
- **Wildcards** - whether wildcards are used as a tag source
|
||||
- **Gallery points** - how many generated images to show (0 = none)
|
||||
- **Tagger fill** - tags generated images whose prompt cannot be read, using a Tagger model (OFF / a model name; choose "Gallery points only" or "All images without a prompt")
|
||||
|
||||
- The first build takes a few minutes to read image prompts and prepare tags. Later builds reuse the previous results and finish in seconds to tens of seconds.
|
||||
- The status line shows the numbers of cards, tags and Gallery points. When only part of the candidates is used, "(of N candidates)" is shown.
|
||||
|
||||
> **Tip**
|
||||
> Most distinct tags are words that only appear in wildcards.
|
||||
> Turning "Wildcards" off and "All tags" on lays out every tag you have actually used (about 10,000 in testing).
|
||||
|
||||
> **Caution**
|
||||
> Tagger fill takes about 0.5 second per image on the CPU. Before choosing "All images without a prompt", check that Tagger runs on the GPU (Chapter 12, section 12-5).
|
||||
|
||||
### Looking and searching
|
||||
- **Drag** to rotate, **right-drag** to pan, and use the **wheel** to zoom.
|
||||
- Hovering over a point shows its name, thumbnail and prompt or sources.
|
||||
- The toolbar checkboxes switch Cards, Thumbnails, Tags and Gallery on or off, as well as **Glow** (stars shining in a night sky) and **Twinkle**.
|
||||
The checkboxes for each source show or hide tags by source.
|
||||
- Type words or a sentence in the search box at the top right and press **Enter** (any language). A white mark is placed at that meaning, and nearby cards and tags are listed.
|
||||
|
||||
### Picking tags to build a prompt
|
||||
1. Click a card or tag to select it. Cards and tags close in meaning are listed under **Nearby** on the right.
|
||||
(Clicking a Gallery point also lists similar generated images)
|
||||
2. **Double-click** what you want in the prompt, or press **Add** in **Selection**, or the **+** in the list.
|
||||
A card adds its whole prompt, a tag adds the tag itself, and a Gallery point adds that image's tags to the **Prompt** area at the far right.
|
||||
3. **Add to GenerateUI (Positive)** appends it to the positive prompt of the workflow loaded in GenerateUI (load a workflow in GenerateUI first).
|
||||
You can also **Copy** it to the clipboard.
|
||||
|
||||
### Searching with the tags of a card or generated image
|
||||
When you select a card or Gallery point, its prompt tags are shown as buttons under **Selection**.
|
||||
Click tags to choose them (click again to deselect) and press **Search with selected** to search the space with them.
|
||||
This helps when you want to find things close to, say, only the hairstyle and clothing of an image.
|
||||
While tags are selected, the **Add** button in **Selection** becomes **Add selected (N)** and adds only those tags to the prompt area.
|
||||
With no tags selected it adds the whole item as before (a card's whole prompt, or all tags of a Gallery point).
|
||||
|
||||
When a Gallery point is selected, **Selection** also shows an **Open in Gallery** button.
|
||||
It switches to the Gallery tab, selects the image's folder, and selects the image in the list (its details open on the right).
|
||||
So that the image is sure to appear, the Gallery's search, favorite, tag and group filters are cleared.
|
||||
A card's whole prompt can be read under **Full prompt**.
|
||||
|
||||
---
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - In the Nodes tab you can list and organize installed nodes, and use them in the WS Library
|
||||
> - Tagger automatically generates tags from images. A model and extra packages must be prepared
|
||||
> - Image Edit is a simple editing tool with layers. Inpainting also lets you redraw parts
|
||||
> - In the Video tab, you can make videos with supported video-generation models and do simple editing
|
||||
> - "Default Checkpoint" and "Data Management (backup)" in the Settings tab are especially handy
|
||||
> - Combined with Comic Creator by the same author, you can make manga from generated images
|
||||
> - TagSpace lays out your tags, prompts and generated images in 3D by closeness of meaning, so you can browse it and pick tags for a prompt
|
||||
@@ -0,0 +1,239 @@
|
||||
# Chapter 12 When You Are in Trouble (Troubleshooting / FAQ)
|
||||
|
||||
This chapter collects the troubles that often occur when using Workflow Studio, and how to deal with them.
|
||||
When in trouble, first look in the **"symptom"** headings for the one closest to your situation.
|
||||
|
||||
> **Three things to try first**
|
||||
> For almost any trouble, these three often solve it.
|
||||
> 1. Reload the browser page (F5 key)
|
||||
> 2. Press the **Refresh** button of each Workflow Studio tab
|
||||
> 3. Quit ComfyUI completely and start it again
|
||||
|
||||
---
|
||||
|
||||
## 12-1 Startup and connection troubles
|
||||
|
||||
### The W button does not appear
|
||||
| Cause | What to do |
|
||||
| --- | --- |
|
||||
| You did not restart ComfyUI after installing | Quit ComfyUI completely and start it again (reloading the browser alone does not take effect) |
|
||||
| The folder is in the wrong place | Check that the folder exists at `ComfyUI/custom_nodes/ComfyUI-Workflow-Studio`. Also check that the folder is not doubled |
|
||||
| ComfyUI is old | Update ComfyUI itself |
|
||||
|
||||
### The Workflow Studio page does not open
|
||||
- Is ComfyUI running? Workflow Studio runs inside ComfyUI, so you cannot open it if you quit ComfyUI.
|
||||
- Is the **port number** in the address right? You can check it in the address bar of the browser where ComfyUI is open (Chapter 2, section 2-4).
|
||||
|
||||
### GenerateUI shows "Disconnected"
|
||||
1. Check that ComfyUI is running and that you can generate an image on ComfyUI's screen.
|
||||
2. Press **Connect** at the top of GenerateUI.
|
||||
3. If it still does not connect, reload the browser page.
|
||||
|
||||
### You updated but the behavior did not change
|
||||
After updating Workflow Studio, always restart ComfyUI.
|
||||
If it still behaves the old way, the browser cache may remain.
|
||||
Try **Ctrl + F5** (forced reload).
|
||||
|
||||
---
|
||||
|
||||
## 12-2 Workflow troubles
|
||||
|
||||
### Workflows do not appear in the list
|
||||
| Cause | What to do |
|
||||
| --- | --- |
|
||||
| The list is still old | Press **Refresh** in the Workflow tab |
|
||||
| A filter is applied | Press **x Clear** at the right end of the toolbar |
|
||||
| You saved to another folder | Check "Workflow Data Folder" in the Settings tab |
|
||||
|
||||
### Nodes are shown in red (missing)
|
||||
When you load a workflow, nodes may be shown in red, or a message may say there are missing nodes.
|
||||
This means **the custom nodes the workflow needs are not installed**.
|
||||
|
||||
**What to do**
|
||||
1. Open ComfyUI's **Manager**.
|
||||
2. Choose **Install Missing Custom Nodes**.
|
||||
3. Install the custom nodes shown, and restart ComfyUI.
|
||||
|
||||
> **Tip**
|
||||
> The "Convert: cross" in the Workflow tab's **GenerateUI Check** (Chapter 4, section 4-7) is also mostly caused by this.
|
||||
> Hovering the mouse over the badge shows the names of the missing nodes.
|
||||
|
||||
### GenerateUI judged it "unusable"
|
||||
| Judgment | Meaning and what to do |
|
||||
| --- | --- |
|
||||
| Convert: cross | A required custom node is missing -> see "Nodes are shown in red" above |
|
||||
| Convert: warning | A model you do not have is specified -> choose again in GenerateUI (12-3) |
|
||||
| Operability: warning / cross | Many settings are not shown in GenerateUI -> generation is often still possible. Change detailed settings on ComfyUI's node screen |
|
||||
|
||||
---
|
||||
|
||||
## 12-3 Generation troubles
|
||||
|
||||
### You changed a setting but the result did not change
|
||||
The most common cause is **forgetting to press Apply**.
|
||||
In GenerateUI, after changing the model or Settings values (steps, size and so on), the change is not reflected in the workflow unless you press **Apply** (Chapter 5, section 5-3).
|
||||
The Switches column at the right of the Settings tab is the exception: it takes effect the moment you click (Chapter 5, section 5-4).
|
||||
Looking at the **Raw JSON** on the right lets you check the values actually reflected.
|
||||
|
||||
### It generates something different from the loaded workflow
|
||||
If a batch kind's checkbox in the **Batch** subtab stays on,
|
||||
a batch generation runs when you press Generate (Chapter 5, section 5-5).
|
||||
Open the Batch subtab and turn off all the Batch Queue column checkboxes.
|
||||
|
||||
### It generates with a different model than you expected
|
||||
If the model saved in the workflow is not on your computer, it may be replaced automatically with another model.
|
||||
In that case a message says the Checkpoint "..." was not found and was replaced with another model without warning.
|
||||
(From v0.8.7, if you have exactly one model with the same name that differs only in its folder, it is replaced with that one.)
|
||||
In the **Model** subtab of GenerateUI, choose the model you want to use again and press **Apply**.
|
||||
|
||||
If you always want to use the same model, the **Default Checkpoint** in the Settings tab is handy (Chapter 11, section 11-5).
|
||||
|
||||
### The Prompt Translation globe button does not appear or translation fails
|
||||
| Symptom | Cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| No globe button | Prompt Translation is off | In the Settings tab, open **Prompt Translation (native language -> English)** and check **Enable prompt translation in GenerateUI** (Chapter 5, section 5-10) |
|
||||
| "Translation error" appears | Ollama is not running, or the translation model is not in Ollama | Start Ollama. If **Translation model** shows **(not downloaded)**, press **Download** |
|
||||
| Nothing changes ("Nothing to translate") | The prompt is English (ASCII) only | Only parts with letters outside ASCII are translated; English parts stay as they are |
|
||||
| Generation does not start with automatic translation on | Translation failed, so generation stopped instead of sending untranslated text | Same fix as "Translation error". In Lab and Video Plan the reason is shown in the progress line |
|
||||
| The translation is off | Small models (0.8B) can shift the meaning of sentences | Choose 2B or larger in **Translation model** |
|
||||
|
||||
### It does not become the same image even with the same settings
|
||||
Is the seed mode in the generate panel set to **Random**?
|
||||
To make the same image, you need to set the seed to **Workflow** (use the workflow's seed as it is) or **Fixed** (fix it to the value in the Seed field).
|
||||
If no new image is added when you generate in a row with the same seed and settings, that is because ComfyUI reuses the already computed result, not a malfunction.
|
||||
For the steps to reproduce a Gallery image, see Chapter 8, section 8-3.
|
||||
|
||||
### Slow generation, or it stops with an error (out of memory)
|
||||
| Cause | What to do |
|
||||
| --- | --- |
|
||||
| Local AI is using GPU memory | Press **Unload Model** in AI TOOL. Use **VRAM Management (Ollama)** in the Settings tab (Chapter 10, section 10-3) |
|
||||
| The image size or Batch Size is too large | Make Width, Height and Batch Size smaller in the Settings subtab |
|
||||
| The model is too large for your GPU | Use a smaller model (such as a GGUF version) |
|
||||
|
||||
### Errors appear after moving or deleting a model
|
||||
ComfyUI may remember the old location of a model.
|
||||
Run **Hard Reset** in ComfyUI's menu (inside the queue menu), or restart ComfyUI.
|
||||
|
||||
### A newly added model does not appear in the list
|
||||
Press **Refresh Models** in GenerateUI, or **Refresh** in the Models tab.
|
||||
If it still does not appear, check that the folder where you put the model is correct (Chapter 6, section 6-1).
|
||||
|
||||
---
|
||||
|
||||
## 12-4 AI feature troubles
|
||||
|
||||
### AI features show "Disconnected" or do not connect
|
||||
1. Check that software such as Ollama is running.
|
||||
For LM Studio or Lemonade, you need to **start the server** inside the app and have a model loaded.
|
||||
2. Check that the address (port number) in the settings matches the software.
|
||||
3. Press **Test Connection** and check the result.
|
||||
|
||||
### It works in the AI TOOL tab but not in the Prompt tab
|
||||
AI settings are saved separately depending on the place (Chapter 10, section 10-2).
|
||||
|
||||
| Place | Where to set it |
|
||||
| --- | --- |
|
||||
| AI TOOL tab / the A tab of the WS Library | The AI TOOL tab's settings (shared) |
|
||||
| Prompt tab | The **gear button** of the AI Assistant in the Prompt tab |
|
||||
| Tagger tab | The **gear button** in the Tagger tab |
|
||||
|
||||
### The model list is empty
|
||||
The AI model may not have been downloaded.
|
||||
For Ollama, download the model with `ollama pull (model name)`, then press **Refresh** in the model selection.
|
||||
|
||||
### It does not connect to Ollama on another computer
|
||||
For safety, by default it connects only to AI software inside the same computer.
|
||||
You need to set the environment variable `WFS_ALLOWED_BACKEND_HOSTS` (Chapter 10, section 10-4).
|
||||
|
||||
### The judge model for image checks, or the OCR model, does not appear in the list
|
||||
- The judge model list shows only Ollama **decision models** that can read images; normal chat models are not shown.
|
||||
Update Ollama, `ollama pull` a decision model, then press **Refresh** (Chapter 10, section 10-6).
|
||||
- The OCR model list shows only Ollama models whose name contains "ocr". Press ↻ after getting one (Chapter 10, section 10-8).
|
||||
- Both look at Ollama at the AI TOOL backend address when the backend is Ollama, otherwise at `http://localhost:11434`.
|
||||
|
||||
### Image checks never pass no matter how often it fixes
|
||||
- How well the redraw works depends on the chat model. Make your request shorter and more concrete, or try another model.
|
||||
- The check itself can be wrong too. Look at the image yourself as well.
|
||||
- Too many automatic fix rounds keep using generation time and GPU memory. "0 (fix by button)" is recommended at first.
|
||||
|
||||
### The AI does not read a skill's reference files, or does not generate images from the chat
|
||||
Reading the reference files of an imported skill and generating images from the chat only work with
|
||||
**a model that supports tool calling**. Switch to such a model.
|
||||
|
||||
---
|
||||
|
||||
## 12-5 Troubles with other features
|
||||
|
||||
### No model appears in Tagger
|
||||
- Check that a `.onnx` file (and `selected_tags.csv`) is in the `ComfyUI/models/tagger/(model name)/` folder.
|
||||
- Check that the extra Python package (onnxruntime) is installed in **the Python that ComfyUI uses** (Chapter 11, section 11-2).
|
||||
|
||||
### Tagger is very slow (the GPU is not used)
|
||||
If ComfyUI's Python has both the CPU package `onnxruntime` and the GPU package `onnxruntime-gpu`, the GPU stops being used.
|
||||
(Installing other custom nodes can bring in the CPU package.)
|
||||
Stop ComfyUI, then in ComfyUI's Python run `pip uninstall -y onnxruntime onnxruntime-gpu` followed by `pip install onnxruntime-gpu`, and restart ComfyUI (DirectML is not supported).
|
||||
|
||||
### PSD-related features give an error
|
||||
An extra Python package (psd-tools) is needed.
|
||||
Run the installation command shown in the error message in the Python that ComfyUI uses, then restart ComfyUI.
|
||||
|
||||
### Semantic search (Gallery) does not work well
|
||||
| Symptom | Cause | What to do |
|
||||
| --- | --- | --- |
|
||||
| "Cannot reach Ollama" is shown | Ollama is not running, or the Ollama URL is wrong | Start Ollama (it is needed while indexing and searching). Check the Ollama URL on the gear button's Model tab |
|
||||
| "The Ollama model is not downloaded" is shown | EmbeddingGemma 2 has not been downloaded | Run `ollama pull embeddinggemma-2:740m` (or `embeddinggemma-2:440m`) |
|
||||
| Indexing sometimes fails with "connection was forcibly closed" or similar | Security software (such as AdGuard) is intercepting Ollama's traffic | Exclude Ollama from filtering. Failed images are processed again by the next "New / changed images only" run |
|
||||
| Gemini says "GEMINI_API_KEY is not set", or the key is rejected | There is no key in `.env`, or the line starts with `#` | Write `GEMINI_API_KEY=key` in the plugin's `.env` (copy `.env.example`) and restart ComfyUI. `.env` is only read at startup |
|
||||
| Search or Similar gives no results | The selected folder has no indexed images | Build the index with the gear button. Choose a folder with a green dot. When you change the model or vector dimension, "rebuild all" |
|
||||
|
||||
### The TagSpace space cannot be built
|
||||
| Symptom | Cause | What to do |
|
||||
| --- | --- | --- |
|
||||
| "EmbeddingGemma 2 in Ollama is required" is shown | Ollama is not running, or the model is missing | Prepare the same way as for semantic search (table above) |
|
||||
| "The Gallery semantic index was made with ..." is shown | The Gallery index was made with Gemini | Choose EmbeddingGemma 2 in the Gallery's semantic settings and rebuild all |
|
||||
| Building takes a very long time | The first build reads image prompts and prepares new tags | Later builds are faster. When using Tagger fill, check that Tagger runs on the GPU ("Tagger is very slow" above) |
|
||||
|
||||
### A partner API (Nano Banana) says authentication failed
|
||||
The cause is a missing Comfy API key in `.env`. Write `COMFY_API_KEY=your key` (issued at platform.comfy.org) in the plugin's `.env` and restart ComfyUI (Chapter 5, section 5-8).
|
||||
`.env` is read only at startup.
|
||||
|
||||
### ComfyUI freezes (no tab responds)
|
||||
On Windows, ComfyUI can stop responding (generation included), for example after switching a Models tab list with many preview images.
|
||||
In the Settings tab, under **Windows Stability (Event Loop)**, turn on **Use the Selector event loop on Windows (restart required)** and restart ComfyUI.
|
||||
|
||||
### Part of a Gallery image looks broken, as if mixed with another image
|
||||
With the **Selector event loop** above turned on, up to v0.9.1 part of an image received by the Gallery preview or Download could be corrupted
|
||||
(the file in the output folder itself is fine). This is fixed in v0.9.2.
|
||||
After updating Workflow Studio, broken images may remain in the browser cache, so reload with **Ctrl + F5**.
|
||||
|
||||
### Images do not appear in the Gallery
|
||||
- Press **Refresh** in the Gallery.
|
||||
- Check that the "Gallery Output Folder" in the Settings tab matches where ComfyUI saves images.
|
||||
|
||||
---
|
||||
|
||||
## 12-6 If it is still not solved
|
||||
|
||||
### Look at Workflow Studio's Help
|
||||
The **Help** tab of Workflow Studio also has troubleshooting information.
|
||||
It may include information added after this book was published.
|
||||
|
||||
### Ask or report on GitHub
|
||||
Problems that are not solved can be reported in **Issues** on Workflow Studio's GitHub repository.
|
||||
When reporting, attaching the following information makes the cause easier to find.
|
||||
|
||||
- The Workflow Studio version
|
||||
- The ComfyUI version and how it was installed (portable version, Desktop version, Stability Matrix and so on)
|
||||
- On which screen, what you did, and what happened
|
||||
- The error message shown on the black screen (console) where ComfyUI was started
|
||||
- The red error message shown in the "Console" of the browser's developer tools (F12 key)
|
||||
|
||||
> **Caution**
|
||||
> When pasting error messages, check that they do not contain personal information such as your computer's user name.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - When in trouble, first try "reload", "Refresh buttons" and "restarting ComfyUI"
|
||||
> - Red nodes mean missing custom nodes. Solve with ComfyUI Manager's Install Missing Custom Nodes
|
||||
> - When settings are not reflected, check for a forgotten Apply and a leftover Batch check
|
||||
> - For out-of-memory, deal with it by unloading local AI and reviewing the image size
|
||||
> - When not solved, make use of the Help tab and GitHub Issues
|
||||
@@ -0,0 +1,141 @@
|
||||
# Appendix
|
||||
|
||||
## Appendix A Glossary
|
||||
|
||||
The main terms that appear in this book, in alphabetical order.
|
||||
|
||||
| Term | Meaning | In this book |
|
||||
| --- | --- | --- |
|
||||
| API format / UI format | The save format of a workflow. UI format includes things such as node positions; API format contains only the processing content | Chapter 4 |
|
||||
| Apply | The button in GenerateUI that reflects changed values into the workflow | Chapter 5 |
|
||||
| Badge | A colored label attached to a workflow or model | Chapter 4 |
|
||||
| Batch generation | Automatically generating in bulk while changing conditions | Chapter 5 |
|
||||
| Canvas | The work space where nodes are laid out on ComfyUI's node screen | Chapter 3 |
|
||||
| CFG | How strongly to follow the prompt | Chapter 5 |
|
||||
| Checkpoint | The model that forms the core of image generation | Chapter 6 |
|
||||
| Civitai | A sharing site for image-generation models | Chapter 6 |
|
||||
| ComfyUI Manager | A management tool for searching for and installing custom nodes | Chapter 2 |
|
||||
| Custom node | A node or extension added to ComfyUI afterwards. Workflow Studio is one of them | Chapter 1 |
|
||||
| Decision model | An AI model that answers questions with numbers such as the probability of "yes". Used to judge image checks | Chapter 10 |
|
||||
| Denoise | How strongly to redraw the original image | Chapter 5 |
|
||||
| Embedding | A small file that teaches a particular word a meaning | Chapter 6 |
|
||||
| Hash value | A "fingerprint"-like value computed from a file's contents. Used to look up models on Civitai | Chapter 6 |
|
||||
| I2I (Image to Image) | Making a new image based on an image | Chapter 5 |
|
||||
| Inpainting | Redrawing only part of an image (the masked part) | Chapter 11 |
|
||||
| JSON | The data format used to save workflows | Chapter 1 |
|
||||
| Keyframe | In Lab, a specification of "from which run to switch settings" | Chapter 5 |
|
||||
| Local AI | AI that runs inside your own computer (such as Ollama) | Chapter 10 |
|
||||
| LoRA | A small model that teaches an art style, character and so on as an add-on | Chapter 6 |
|
||||
| Mask | A specification of the part of an image to be processed | Chapter 11 |
|
||||
| Metadata | Information embedded in an image, such as the settings used at generation | Chapter 8 |
|
||||
| Model | The file that acts as the "AI brain" that makes images | Chapter 1 |
|
||||
| Negative prompt | A prompt where you write what you do not want drawn | Chapter 7 |
|
||||
| Node | A "box" arranged on ComfyUI's screen. It handles one process | Chapter 1 |
|
||||
| Node set | Several nodes and their connections saved together | Chapter 9 |
|
||||
| OCR | Reading the text shown in an image | Chapter 10 |
|
||||
| Ollama | Software for running local AI easily | Chapter 10 |
|
||||
| PLaMo 3 Translate | A translation AI model by Preferred Networks (PFN), used by Prompt Translation | Chapter 5 |
|
||||
| Port number | A number for distinguishing which software to communicate with inside one computer | Chapter 2 |
|
||||
| Positive prompt | A prompt where you write what you want drawn | Chapter 7 |
|
||||
| Preset | A pair of positive and negative prompts saved together | Chapter 7 |
|
||||
| Prompt | The text that tells the AI "what kind of image you want" | Chapter 1 |
|
||||
| Queue | The reservation list for generation. Processed in order | Chapter 1 |
|
||||
| Sampler | The algorithm used when drawing an image (such as euler) | Chapter 5 |
|
||||
| Scheduler | How the drawing amount is distributed across steps | Chapter 5 |
|
||||
| Seed | The seed of random numbers an image grows from. If the same, you get almost the same image | Chapter 5 |
|
||||
| Semantic search | Searching images by the "meaning" of a sentence or image. Used with the Gallery's S button | Chapter 8 |
|
||||
| Skill | Instructions that tell the AI TOOL chat its role and how to answer. Picking one switches what it is good at | Chapter 10 |
|
||||
| Steps | The number of times drawing is repeated | Chapter 5 |
|
||||
| Style | A mood template added to a prompt | Chapter 7 |
|
||||
| T2I (Text to Image) | Making an image from text (a prompt) | Chapter 5 |
|
||||
| TagSpace | A tab that lays out tags, prompts and generated images in 3D by closeness of meaning | Chapter 11 |
|
||||
| Trigger word | A word to put in the prompt to bring out a LoRA's effect | Chapter 6 |
|
||||
| VAE | The model that finishes an image's colors and fine details | Chapter 6 |
|
||||
| VLM | An AI model that can read images | Chapter 10 |
|
||||
| VRAM | The memory of the GPU (graphics card) | Chapter 10 |
|
||||
| Wildcard | A mechanism that randomly swaps a part of the prompt at every generation | Chapter 7 |
|
||||
| Workflow | The whole flow of processing made by connecting nodes with lines | Chapter 1 |
|
||||
| WS Library | Workflow Studio's side panel that opens beside ComfyUI's node screen | Chapter 9 |
|
||||
|
||||
---
|
||||
|
||||
## Appendix B Keyboard and mouse operations
|
||||
|
||||
### General
|
||||
| Operation | What happens |
|
||||
| --- | --- |
|
||||
| Esc | Closes the open screen (modal or panel) |
|
||||
| Enter | Confirms a rename / sends a chat message |
|
||||
| Shift + Enter | Line break in chat |
|
||||
| Double-click | Opens the detail screen (workflows, models and so on) |
|
||||
| Shift + click (W button) | Opens Workflow Studio in a new window |
|
||||
|
||||
### GenerateUI
|
||||
| Operation | What happens |
|
||||
| --- | --- |
|
||||
| Alt + click (Apply) | Reflects the values and generates at once |
|
||||
| Ctrl + Up/Down (prompt field) | Raises or lowers the emphasis `(word:number)` of the selected word by 0.05 |
|
||||
| Enter / Shift + Enter (Raw JSON search box) | Moves to the next / previous match |
|
||||
|
||||
### Gallery
|
||||
| Operation | What happens |
|
||||
| --- | --- |
|
||||
| Ctrl + click | Select several images |
|
||||
| Alt + click | Open the image in the Metadata subtab |
|
||||
| Enter (when semantic search is on) | Run a semantic search with the text in the search box |
|
||||
|
||||
### WS Library
|
||||
| Operation | What happens |
|
||||
| --- | --- |
|
||||
| Drag and drop | Place or load at the dropped position on the canvas |
|
||||
| Double-click | Place or load near the center of the canvas |
|
||||
| Alt + click (N tab) | Switch a node's favorite on or off |
|
||||
|
||||
### Image Edit
|
||||
| Operation | What happens |
|
||||
| --- | --- |
|
||||
| V / B / T / S / G | Switch to the Select / Draw / Text / Shape / Fill tool |
|
||||
| Ctrl + Z / Ctrl + Y | Undo / redo |
|
||||
| Mouse wheel | Zoom in and out |
|
||||
| Space + drag | Move the view position |
|
||||
| Delete | Delete the selected layer (when there are two or more layers) |
|
||||
| D / E (while Draw is active) | Draw / Erase |
|
||||
| X (while Draw is active) | Swap Color 1 and Color 2 |
|
||||
| [ / ] (while Draw is active) | Make the brush smaller / larger |
|
||||
| B (while Draw is active) | Open or close the brush picker |
|
||||
|
||||
---
|
||||
|
||||
## Appendix C Reference links
|
||||
|
||||
### Workflow Studio
|
||||
- GitHub (latest version, release notes, bug reports)
|
||||
https://github.com/ketle-man/ComfyUI-Workflow-Studio
|
||||
|
||||
### ComfyUI itself and management tools
|
||||
- ComfyUI
|
||||
https://github.com/comfyanonymous/ComfyUI
|
||||
- ComfyUI Manager
|
||||
https://github.com/Comfy-Org/ComfyUI-Manager
|
||||
|
||||
### Related software and services introduced in this book
|
||||
| Name | Use | In this book |
|
||||
| --- | --- | --- |
|
||||
| Civitai (https://civitai.com/) | Getting models, fetching information | Chapter 6 |
|
||||
| Ollama (https://ollama.com/) | Local AI | Chapter 10 |
|
||||
| PLaMo 3 Translate (https://huggingface.co/pfnet/plamo-3-translate-2b-GGUF) | Translation model for Prompt Translation | Chapter 5 |
|
||||
| LM Studio (https://lmstudio.ai/) | Local AI | Chapter 10 |
|
||||
| Lemonade Server (https://lemonade-server.ai/) | Local AI | Chapter 10 |
|
||||
| Eagle (https://eagle.cool/) | Image-management software | Chapter 8 |
|
||||
|
||||
### Custom nodes that are handy to use together
|
||||
| Name | Use | In this book |
|
||||
| --- | --- | --- |
|
||||
| ComfyUI Comic Creator (https://github.com/ketle-man/comfyui-comic-creator) | Manga making. Works with the Gallery and image generation | Chapter 11 |
|
||||
| comfyui-image-feeder (https://github.com/ketle-man/comfyui-image-feeder) | Feeder's Image Loop mode | Chapter 5 |
|
||||
| comfyui-mask-editor-one (https://github.com/ketle-man/comfyui-mask-editor-one) | Advanced masking and background removal in Image Edit | Chapter 11 |
|
||||
| ComfyUI-Impact-Pack (https://github.com/ltdrdata/ComfyUI-Impact-Pack) | Curly-brace-style wildcards | Chapter 7 |
|
||||
| ComfyUI-Lora-Manager | Managing LoRAs; placing LoRAs together from the WS Library | Chapter 9 |
|
||||
|
||||
> **Caution**
|
||||
> The contents and URLs of linked pages may change.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Afterword
|
||||
|
||||
Thank you for reading to the end.
|
||||
|
||||
ComfyUI is very free and deep software.
|
||||
Because of that freedom, it is also software where beginners think "I don't know where to start".
|
||||
Workflow Studio was born from the wish to widen that entrance a little,
|
||||
and to make "creating" and "organizing" more casual.
|
||||
|
||||
Let's look back once more at the ways of using it introduced in this book.
|
||||
|
||||
1. **Organize** - make workflows, models and prompts easy to tell apart with thumbnails and badges
|
||||
2. **Create** - generate images in GenerateUI without touching the node screen
|
||||
3. **Collect** - keep prompts and settings that worked in presets and the Gallery
|
||||
4. **Reproduce** - recall the same settings right away from an image's metadata
|
||||
5. **Expand** - speed up trial and error with batch generation and local AI
|
||||
|
||||
The shortest path to improving at image generation is to **try a lot, and keep what worked**.
|
||||
I will be glad if Workflow Studio becomes a tool that supports that accumulation.
|
||||
|
||||
Workflow Studio will continue to gain features and improvements.
|
||||
New features and changes are announced in the release notes on GitHub.
|
||||
Your impressions after using it, and requests such as "I want this feature", are also welcome on GitHub.
|
||||
|
||||
May your life with image generation become more fun and richer.
|
||||
|
||||
ketle-man
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.7 MiB After Width: | Height: | Size: 422 KiB |
+10
-1
@@ -52,6 +52,15 @@ def _maybe_use_selector_event_loop():
|
||||
print("[Workflow Studio] Using WindowsSelectorEventLoopPolicy (windows_selector_event_loop enabled)")
|
||||
except Exception as e:
|
||||
print(f"[WARNING] Workflow Studio: failed to switch to selector event loop: {e}")
|
||||
return
|
||||
# Selector方式ではloop.sendfileがasyncioの_sendfile_fallbackになるが、これは16KBバッファを
|
||||
# 使い回しつつ未送信分をコピーせずトランスポートに積むため、送信が詰まると後続ブロックで
|
||||
# 上書きされ画像等が壊れて届く(Python 3.12で確認)。aiohttpのsendfileを無効化し、
|
||||
# チャンクごとに新しいbytesを読む自前フォールバックを使わせる
|
||||
os.environ["AIOHTTP_NOSENDFILE"] = "1"
|
||||
fileresponse = sys.modules.get("aiohttp.web_fileresponse")
|
||||
if fileresponse is not None:
|
||||
fileresponse.NOSENDFILE = True
|
||||
|
||||
|
||||
_maybe_use_selector_event_loop()
|
||||
@@ -63,7 +72,7 @@ if ENV_FILE.exists():
|
||||
except ImportError:
|
||||
print(
|
||||
"[WARNING] Workflow Studio: .env file found but python-dotenv is not installed "
|
||||
"(pip install -r requirements.txt). UNSLOTH_API_KEY will not be loaded from .env."
|
||||
"(pip install -r requirements.txt). UNSLOTH_API_KEY / GEMINI_API_KEY will not be loaded from .env."
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"[WARNING] Workflow Studio: failed to load .env file: {e}")
|
||||
|
||||
@@ -1,9 +1,66 @@
|
||||
import os
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ─── パス検査（クライアント由来のパスを resolve() する前に文字列のまま検査する）───
|
||||
# Windows では UNC パス(\\host\share)は resolve()・exists() を呼んだ時点で相手ホストへ
|
||||
# Windows アカウントの資格情報付きで接続してしまうため、チェックはパス操作より先に行う。
|
||||
|
||||
_DRIVE_RE = re.compile(r"^[A-Za-z]:")
|
||||
|
||||
|
||||
def is_unc(value) -> bool:
|
||||
"""ネットワーク共有/デバイスパス(\\\\host\\share, //host/share, \\\\?\\ 等)か。"""
|
||||
return str(value or "").replace("/", "\\").startswith("\\\\")
|
||||
|
||||
|
||||
def is_safe_rel(rel) -> bool:
|
||||
"""相対パスとして安全か。UNC・ドライブ指定・絶対パス・「..」・空セグメントを
|
||||
結合・resolve() より前に文字列のまま拒否する。"""
|
||||
s = str(rel or "")
|
||||
if not s or "\x00" in s:
|
||||
return False
|
||||
if s.startswith(("/", "\\")) or _DRIVE_RE.match(s):
|
||||
return False
|
||||
return not any(part in ("", "..") for part in re.split(r"[/\\]", s))
|
||||
|
||||
|
||||
def resolve_untrusted(value, roots=()) -> Path:
|
||||
"""クライアント由来の絶対パス文字列を検査してから resolve() して返す。
|
||||
|
||||
ファイルシステムに触れない文字列処理(normpath)だけで「..」を正規化・拒否し、
|
||||
UNC は roots のうち UNC のルート配下に文字列上一致する場合のみ許可する
|
||||
(ギャラリーのルートをネットワーク共有に置いている構成だけを通すため)。
|
||||
拒否時は ValueError。"""
|
||||
s = str(value or "")
|
||||
if not s or "\x00" in s:
|
||||
raise ValueError("Access denied: invalid path")
|
||||
p = Path(os.path.normpath(s))
|
||||
if ".." in p.parts:
|
||||
raise ValueError("Access denied: path traversal")
|
||||
if is_unc(s):
|
||||
for root in roots:
|
||||
if root is None or not is_unc(str(root)):
|
||||
continue
|
||||
try:
|
||||
p.relative_to(Path(root)) # 文字列上の照合(解決しない)
|
||||
return p.resolve()
|
||||
except (ValueError, OSError):
|
||||
continue
|
||||
raise ValueError("Access denied: network path")
|
||||
# 受け付ける形はドライブレター絶対パス(Windows)または / 始まり(その他OS)に限る。
|
||||
# ルート相対(\foo)・ドライブ相対(C:foo)・NT デバイス形式(\??\, \.\ 等)・相対パスは拒否する
|
||||
if os.name == "nt":
|
||||
if not re.match(r"^[A-Za-z]:[\\/]", s):
|
||||
raise ValueError("Access denied: invalid path")
|
||||
elif not s.startswith("/"):
|
||||
raise ValueError("Access denied: invalid path")
|
||||
return p.resolve()
|
||||
|
||||
|
||||
# Plugin root directory
|
||||
PLUGIN_DIR = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
@@ -9,6 +9,9 @@ import urllib.error
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..config import is_safe_rel
|
||||
from ..services.url_guard import require_backend_url, urlopen as _guarded_urlopen
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@@ -36,6 +39,11 @@ def _resolve_local_path(filename, subfolder, type_name):
|
||||
if not filename:
|
||||
return None
|
||||
|
||||
# subfolder の UNC・ドライブ指定・絶対パス・「..」は、結合より前に文字列のまま拒否する
|
||||
# (別ドライブ/UNC だと下の commonpath が ValueError を投げるため、その前に弾く)
|
||||
if subfolder and not is_safe_rel(subfolder):
|
||||
return None
|
||||
|
||||
full_dir = os.path.abspath(os.path.join(base_dir, subfolder or ""))
|
||||
# Windowsはパス比較で大文字小文字を区別しないため normcase で正規化してから比較する
|
||||
if os.path.commonpath((os.path.normcase(full_dir), os.path.normcase(base_dir))) != os.path.normcase(base_dir):
|
||||
@@ -60,9 +68,9 @@ def _resolve_absolute_svg_path(abs_path):
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
# isfile() も UNC(\\host\share)だと相手ホストへ資格情報付きで接続してしまうため、
|
||||
# 文字列処理(abspath/commonpath)だけで ComfyUI 管理下と確認できるまでファイルシステムに触れない
|
||||
abs_path = os.path.abspath(abs_path)
|
||||
if not os.path.isfile(abs_path):
|
||||
return None
|
||||
|
||||
for type_name in ("output", "temp", "input"):
|
||||
base_dir = folder_paths.get_directory_by_type(type_name)
|
||||
@@ -72,9 +80,9 @@ def _resolve_absolute_svg_path(abs_path):
|
||||
try:
|
||||
# Windowsはパス比較で大文字小文字を区別しないため normcase で正規化してから比較する
|
||||
if os.path.commonpath((os.path.normcase(abs_path), os.path.normcase(base_dir))) == os.path.normcase(base_dir):
|
||||
return abs_path
|
||||
return abs_path if os.path.isfile(abs_path) else None
|
||||
except ValueError:
|
||||
# Windowsで異なるドライブレターの場合 commonpath が例外を投げる
|
||||
# Windowsで異なるドライブレターやUNCの場合 commonpath が例外を投げる
|
||||
continue
|
||||
return None
|
||||
|
||||
@@ -92,7 +100,7 @@ def _eagle_add_from_path(eagle_url, file_path, name, tags):
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
with _guarded_urlopen(req, timeout=10) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
@@ -127,7 +135,7 @@ def _eagle_add(eagle_url, image_url, name, tags):
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
with _guarded_urlopen(req, timeout=10) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
@@ -138,7 +146,7 @@ def _eagle_test(eagle_url):
|
||||
f"{eagle_url}/api/application/info",
|
||||
method="GET",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
with _guarded_urlopen(req, timeout=5) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
@@ -146,7 +154,7 @@ async def handle_add(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/eagle/add - Add image to Eagle."""
|
||||
try:
|
||||
body = await request.json()
|
||||
eagle_url = body.get("eagleUrl", "http://localhost:41595")
|
||||
eagle_url = require_backend_url(body.get("eagleUrl", "http://localhost:41595"))
|
||||
image_url = body.get("url", "")
|
||||
name = body.get("name", "image.png")
|
||||
tags = body.get("tags", [])
|
||||
@@ -193,7 +201,7 @@ async def handle_test(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/eagle/test - Test Eagle connection."""
|
||||
try:
|
||||
body = await request.json()
|
||||
eagle_url = body.get("eagleUrl", "http://localhost:41595")
|
||||
eagle_url = require_backend_url(body.get("eagleUrl", "http://localhost:41595"))
|
||||
result = await asyncio.to_thread(_eagle_test, eagle_url)
|
||||
return web.json_response({
|
||||
"status": "success",
|
||||
|
||||
@@ -10,10 +10,12 @@ from aiohttp import web
|
||||
|
||||
from ..config import DATA_DIR
|
||||
from ..services.gallery_service import GalleryService
|
||||
from ..services.semantic_service import SemanticService, SemanticError, DEFAULT_DIM, MODEL as SEMANTIC_DEFAULT_MODEL
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_service = GalleryService(DATA_DIR)
|
||||
_semantic = SemanticService(DATA_DIR, _service)
|
||||
|
||||
|
||||
def _init_allowed_root() -> None:
|
||||
@@ -55,13 +57,33 @@ def _init_allowed_root() -> None:
|
||||
_init_allowed_root()
|
||||
|
||||
|
||||
# Windowsのレジストリ依存で音声のMIMEが返らない/不統一なため明示登録する（<audio>の再生に必要）
|
||||
for _ext, _mime in ((".wav", "audio/wav"), (".mp3", "audio/mpeg"), (".ogg", "audio/ogg"),
|
||||
(".opus", "audio/ogg"), (".m4a", "audio/mp4"), (".flac", "audio/flac"),
|
||||
(".glb", "model/gltf-binary"), (".vrm", "model/gltf-binary"), (".gltf", "model/gltf+json"),
|
||||
(".ply", "application/octet-stream"), (".spz", "application/octet-stream"),
|
||||
(".bin", "application/octet-stream")):
|
||||
mimetypes.add_type(_mime, _ext)
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_get("/wfm/gallery/folders", list_folders)
|
||||
app.router.add_get("/wfm/gallery/semantic/status", semantic_status)
|
||||
app.router.add_get("/wfm/gallery/semantic/folders", semantic_folders)
|
||||
app.router.add_post("/wfm/gallery/semantic/index", semantic_index)
|
||||
app.router.add_post("/wfm/gallery/semantic/cancel", semantic_cancel)
|
||||
app.router.add_post("/wfm/gallery/semantic/search", semantic_search)
|
||||
app.router.add_post("/wfm/gallery/semantic/similar", semantic_similar)
|
||||
app.router.add_post("/wfm/gallery/semantic/clear", semantic_clear)
|
||||
app.router.add_post("/wfm/gallery/semantic/test", semantic_test)
|
||||
app.router.add_post("/wfm/gallery/semantic/options", semantic_options)
|
||||
app.router.add_get("/wfm/gallery/images", list_images)
|
||||
app.router.add_get("/wfm/gallery/image/meta", get_image_meta)
|
||||
app.router.add_get("/wfm/gallery/image/workflow", get_image_workflow)
|
||||
app.router.add_get("/wfm/gallery/image/serve", serve_image)
|
||||
app.router.add_get("/wfm/gallery/image/thumb", serve_thumb)
|
||||
app.router.add_post("/wfm/gallery/image/thumb3d", save_model3d_thumb)
|
||||
app.router.add_get("/wfm/gallery/model3d/asset", serve_model3d_asset)
|
||||
app.router.add_post("/wfm/gallery/image/meta", save_image_meta)
|
||||
app.router.add_get("/wfm/gallery/image-prompt/root", get_image_prompt_root)
|
||||
app.router.add_post("/wfm/gallery/image/image-prompt", save_image_prompt)
|
||||
@@ -81,6 +103,7 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_get("/wfm/gallery/groups/{name}/images", list_group_images)
|
||||
# バルク操作
|
||||
app.router.add_post("/wfm/gallery/bulk/favorite", bulk_favorite)
|
||||
app.router.add_post("/wfm/gallery/convert", convert_file_route)
|
||||
app.router.add_post("/wfm/gallery/bulk/group", bulk_group)
|
||||
# フォルダ・ファイル操作
|
||||
app.router.add_post("/wfm/gallery/folder", create_folder_route)
|
||||
@@ -94,6 +117,143 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_get("/wfm/gallery/image/psd-layers", get_psd_layers)
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# セマンティック検索 (Gemini Embedding 2 / Ollama EmbeddingGemma 2)
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _semantic_error(e: SemanticError) -> web.Response:
|
||||
# code はフロントのi18n用。message は詳細(APIキー等の秘密は含まない)
|
||||
return web.json_response({"error": str(e), "code": e.code}, status=400)
|
||||
|
||||
|
||||
def _num(body: dict, key: str, default, lo, hi, cast=int):
|
||||
"""リクエストの数値パラメータを検証して範囲内に収める。数値でなければ400相当のSemanticErrorにする"""
|
||||
raw = body.get(key, default)
|
||||
if raw is None or raw == "":
|
||||
raw = default
|
||||
try:
|
||||
v = cast(raw)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
raise SemanticError("bad_param", key)
|
||||
if v != v: # NaN
|
||||
raise SemanticError("bad_param", key)
|
||||
return max(lo, min(v, hi))
|
||||
|
||||
|
||||
async def _json_body(request: web.Request) -> dict:
|
||||
try:
|
||||
body = await request.json()
|
||||
return body if isinstance(body, dict) else {}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
async def semantic_status(request: web.Request) -> web.Response:
|
||||
folder = request.rel_url.query.get("folder", "")
|
||||
recursive = request.rel_url.query.get("recursive", "true") == "true"
|
||||
try:
|
||||
return web.json_response(await asyncio.to_thread(_semantic.status, folder, recursive))
|
||||
except Exception as e:
|
||||
logger.error("semantic_status error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_folders(request: web.Request) -> web.Response:
|
||||
root = request.rel_url.query.get("root", "")
|
||||
try:
|
||||
return web.json_response({"folders": await asyncio.to_thread(_semantic.folder_status, root)})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_folders error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
def _apply_local_options(body: dict) -> None:
|
||||
"""リクエストに Ollama の接続先/モデルサイズが含まれていれば反映・保存する"""
|
||||
if "ollama_url" in body:
|
||||
_semantic.set_local_options(str(body.get("ollama_url", "")), str(body.get("ollama_tag", "")))
|
||||
|
||||
|
||||
async def semantic_index(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
_apply_local_options(body)
|
||||
await asyncio.to_thread(
|
||||
_semantic.start_index,
|
||||
str(body.get("folder", "")),
|
||||
bool(body.get("recursive", True)),
|
||||
"all" if body.get("mode") == "all" else "missing",
|
||||
str(body.get("model", SEMANTIC_DEFAULT_MODEL)),
|
||||
_num(body, "dim", DEFAULT_DIM, 0, 100000),
|
||||
)
|
||||
return web.json_response({"ok": True})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_index error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_cancel(request: web.Request) -> web.Response:
|
||||
_semantic.cancel()
|
||||
return web.json_response({"ok": True})
|
||||
|
||||
|
||||
async def semantic_search(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
limit = _num(body, "limit", 200, 1, 1000)
|
||||
min_score = _num(body, "min_score", 0.0, -1.0, 1.0, float)
|
||||
images = await asyncio.to_thread(
|
||||
_semantic.search_text, str(body.get("query", "")), str(body.get("folder", "")),
|
||||
bool(body.get("recursive", True)), limit, min_score,
|
||||
)
|
||||
return web.json_response({"images": images, "total": len(images)})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_search error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_similar(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
limit = _num(body, "limit", 200, 1, 1000)
|
||||
min_score = _num(body, "min_score", 0.0, -1.0, 1.0, float)
|
||||
images = await asyncio.to_thread(
|
||||
_semantic.search_similar, str(body.get("path", "")), str(body.get("folder", "")),
|
||||
bool(body.get("recursive", True)), limit, min_score,
|
||||
)
|
||||
return web.json_response({"images": images, "total": len(images)})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_similar error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_clear(request: web.Request) -> web.Response:
|
||||
try:
|
||||
await asyncio.to_thread(_semantic.clear)
|
||||
return web.json_response({"ok": True})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
|
||||
|
||||
async def semantic_test(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
_apply_local_options(body)
|
||||
res = await asyncio.to_thread(_semantic.test_connection, str(body.get("model", SEMANTIC_DEFAULT_MODEL)), _num(body, "dim", DEFAULT_DIM, 0, 100000))
|
||||
return web.json_response({"ok": True, **res})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# フォルダツリー
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
@@ -231,10 +391,56 @@ async def serve_thumb(request: web.Request) -> web.Response:
|
||||
})
|
||||
|
||||
|
||||
async def save_model3d_thumb(request: web.Request) -> web.Response:
|
||||
"""POST /wfm/gallery/image/thumb3d — ブラウザが描画した3Dモデルのサムネイルを保存する"""
|
||||
try:
|
||||
body = await request.json()
|
||||
path = body.get("path", "")
|
||||
image = body.get("image", "")
|
||||
if not path or not image:
|
||||
return web.json_response({"ok": False, "error": "path and image required"}, status=400)
|
||||
loop = asyncio.get_event_loop()
|
||||
result = await loop.run_in_executor(None, _service.save_model3d_thumbnail, path, image)
|
||||
return web.json_response(result, status=200 if result.get("ok") else 400)
|
||||
except Exception as e:
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def serve_model3d_asset(request: web.Request) -> web.Response:
|
||||
"""GET /wfm/gallery/model3d/asset — .gltfが参照する外部ファイル(.bin/テクスチャ)を配信する"""
|
||||
path = request.rel_url.query.get("path", "")
|
||||
uri = request.rel_url.query.get("uri", "")
|
||||
if not path or not uri:
|
||||
return web.Response(status=400)
|
||||
asset = _service.serve_model3d_asset(path, uri)
|
||||
if asset is None:
|
||||
return web.Response(status=404)
|
||||
content_type, _ = mimetypes.guess_type(str(asset))
|
||||
return web.FileResponse(asset, headers={
|
||||
"Content-Type": content_type or "application/octet-stream",
|
||||
"Cache-Control": "max-age=3600",
|
||||
})
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# バルク操作
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
async def convert_file_route(request: web.Request) -> web.Response:
|
||||
"""POST /wfm/gallery/convert — 音声/動画の音声を別形式へ変換（1ファイルずつ呼ぶ）"""
|
||||
try:
|
||||
body = await request.json()
|
||||
path = body.get("path", "")
|
||||
if not path:
|
||||
return web.json_response({"ok": False, "error": "path required"}, status=400)
|
||||
# 変換はCPU負荷が高く同期実行するとイベントループ全体が止まるためexecutorで実行する
|
||||
loop = asyncio.get_event_loop()
|
||||
result = await loop.run_in_executor(None, _service.convert_file, path, body)
|
||||
return web.json_response(result, status=200 if result.get("ok") else 400)
|
||||
except Exception as e:
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def bulk_favorite(request: web.Request) -> web.Response:
|
||||
"""POST /wfm/gallery/bulk/favorite — 複数画像のお気に入りを一括設定"""
|
||||
try:
|
||||
@@ -678,3 +884,16 @@ async def get_psd_layers(request: web.Request) -> web.Response:
|
||||
except Exception as e:
|
||||
logger.error("Error reading PSD layers: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_options(request: web.Request) -> web.Response:
|
||||
"""Ollama の接続先/モデルサイズを保存して、新しい状態を返す"""
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
await asyncio.to_thread(_apply_local_options, body)
|
||||
return web.json_response(await asyncio.to_thread(_semantic.status))
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_options error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from aiohttp import web
|
||||
|
||||
from ..services.settings_service import SettingsService
|
||||
from ..config import DATA_DIR
|
||||
from ..config import DATA_DIR, resolve_untrusted
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -153,17 +153,20 @@ async def handle_result(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
result_path = body.get("result_path", "")
|
||||
if not result_path or not os.path.exists(result_path):
|
||||
return web.json_response({"error": f"Result file not found: {result_path}"}, status=404)
|
||||
|
||||
# Path traversal guard: only allow files inside gmic_temp
|
||||
# Path traversal guard: only allow files inside gmic_temp.
|
||||
# UNC(\\host\share)は exists()/resolve() を呼んだ時点で相手ホストへ資格情報付きで
|
||||
# 接続してしまうため、パス操作より先に文字列のまま検査する(resolve_untrusted)。
|
||||
allowed_dir = (Path(DATA_DIR) / "gmic_temp").resolve()
|
||||
try:
|
||||
Path(result_path).resolve().relative_to(allowed_dir)
|
||||
resolved = resolve_untrusted(result_path)
|
||||
resolved.relative_to(allowed_dir)
|
||||
except ValueError:
|
||||
return web.json_response({"error": "Access denied: path outside gmic_temp"}, status=403)
|
||||
if not resolved.is_file():
|
||||
return web.json_response({"error": f"Result file not found: {result_path}"}, status=404)
|
||||
|
||||
with open(result_path, "rb") as f:
|
||||
with open(resolved, "rb") as f:
|
||||
img_bytes = f.read()
|
||||
|
||||
ext = os.path.splitext(result_path)[1].lower()
|
||||
|
||||
@@ -15,6 +15,7 @@ _service = NodesService()
|
||||
def setup_routes(app: web.Application):
|
||||
"""Register all node management API routes."""
|
||||
# Node metadata
|
||||
app.router.add_get("/api/wfm/nodes/light", handle_get_light_nodes)
|
||||
app.router.add_get("/api/wfm/nodes/metadata", handle_get_metadata)
|
||||
app.router.add_post("/api/wfm/nodes/metadata", handle_save_metadata)
|
||||
# Node groups
|
||||
@@ -28,6 +29,50 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_get("/api/wfm/node-sets/export", handle_export_set)
|
||||
|
||||
|
||||
# ── Light node list ────────────────────────────────────────
|
||||
|
||||
|
||||
def _build_light_node_list() -> list[dict]:
|
||||
"""全ノードの軽量情報（INPUT_TYPES を呼ばない）。
|
||||
|
||||
/object_info は各ノードの INPUT_TYPES() を呼ぶため、モデル一覧を走査するノードが
|
||||
多い環境では十数秒かかる。Nodesタブの一覧表示に必要なのはクラス属性だけなので、
|
||||
それらのみを読んで高速に返す（入力定義は呼び出し側が /object_info で後から補う）。
|
||||
"""
|
||||
import nodes as comfy_nodes
|
||||
|
||||
result = []
|
||||
for name, cls in list(comfy_nodes.NODE_CLASS_MAPPINGS.items()):
|
||||
try:
|
||||
return_types = getattr(cls, "RETURN_TYPES", ()) or ()
|
||||
result.append({
|
||||
"name": name,
|
||||
"display_name": comfy_nodes.NODE_DISPLAY_NAME_MAPPINGS.get(name, name),
|
||||
"description": getattr(cls, "DESCRIPTION", "") or "",
|
||||
"category": getattr(cls, "CATEGORY", "sd") or "sd",
|
||||
"python_module": getattr(cls, "RELATIVE_PYTHON_MODULE", "nodes") or "nodes",
|
||||
"output": list(return_types),
|
||||
"output_name": list(getattr(cls, "RETURN_NAMES", None) or return_types),
|
||||
"output_node": getattr(cls, "OUTPUT_NODE", False) is True,
|
||||
"search_aliases": list(getattr(cls, "SEARCH_ALIASES", []) or []),
|
||||
"deprecated": bool(getattr(cls, "DEPRECATED", False)),
|
||||
"experimental": bool(getattr(cls, "EXPERIMENTAL", False)),
|
||||
})
|
||||
except Exception as e:
|
||||
logger.debug("light node info failed for %s: %s", name, e)
|
||||
return result
|
||||
|
||||
|
||||
async def handle_get_light_nodes(request: web.Request) -> web.Response:
|
||||
"""GET /api/wfm/nodes/light"""
|
||||
try:
|
||||
result = await asyncio.to_thread(_build_light_node_list)
|
||||
return web.json_response(result)
|
||||
except Exception as e:
|
||||
logger.error("Error building light node list: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
# ── Node Metadata ──────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ import urllib.request
|
||||
from aiohttp import web
|
||||
|
||||
from ..services.settings_service import SettingsService
|
||||
from ..services.url_guard import require_backend_url, urlopen
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -39,11 +40,12 @@ async def handle_models(request: web.Request) -> web.Response:
|
||||
import asyncio
|
||||
try:
|
||||
cfg = _get_ollama_config()
|
||||
base_url = require_backend_url(cfg["url"])
|
||||
|
||||
def _fetch():
|
||||
url = f"{cfg['url']}/api/tags"
|
||||
url = f"{base_url}/api/tags"
|
||||
req = urllib.request.Request(url)
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
with urlopen(req, timeout=5) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
data = await asyncio.to_thread(_fetch)
|
||||
@@ -61,11 +63,12 @@ async def handle_test(request: web.Request) -> web.Response:
|
||||
import asyncio
|
||||
try:
|
||||
cfg = _get_ollama_config()
|
||||
base_url = require_backend_url(cfg["url"])
|
||||
|
||||
def _test():
|
||||
url = f"{cfg['url']}/api/tags"
|
||||
url = f"{base_url}/api/tags"
|
||||
req = urllib.request.Request(url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
with urlopen(req, timeout=5) as resp:
|
||||
return resp.status == 200
|
||||
|
||||
ok = await asyncio.to_thread(_test)
|
||||
@@ -83,7 +86,7 @@ async def handle_chat(request: web.Request) -> web.Response:
|
||||
body = await request.json()
|
||||
cfg = _get_ollama_config()
|
||||
model = body.get("model") or cfg["model"]
|
||||
base_url = (body.get("url") or cfg["url"]).rstrip("/")
|
||||
base_url = require_backend_url(body.get("url") or cfg["url"])
|
||||
messages = body.get("messages", [])
|
||||
|
||||
def _chat():
|
||||
@@ -100,7 +103,7 @@ async def handle_chat(request: web.Request) -> web.Response:
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
with urlopen(req, timeout=120) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
resp_data = await asyncio.to_thread(_chat)
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
"""Partner (API) node execution support for GenerateUI.
|
||||
|
||||
ComfyUI's partner API nodes (Nano Banana / Gemini, Kling, Seedream, ...) authenticate through the
|
||||
``extra_data`` of the ``/prompt`` request: ``auth_token_comfy_org`` (the browser's Comfy login) or
|
||||
``api_key_comfy_org`` (a Comfy API key issued at platform.comfy.org). ComfyUI's own web UI adds
|
||||
the login token automatically, but Workflow Studio is a separate page that POSTs to ``/prompt`` itself,
|
||||
so such a prompt fails with "Unauthorized: Please login first to use this node."
|
||||
|
||||
When COMFY_API_KEY is set (loaded from the plugin's .env by prestartup_script.py), the frontend
|
||||
queues prompts through this proxy instead. The proxy adds the key to ``extra_data`` — only when the
|
||||
prompt actually contains an API node (``api_node: true`` in /object_info) — and relays the request
|
||||
to the local ComfyUI. The key never reaches the browser. Usage is billed as Comfy credits.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from urllib.parse import quote
|
||||
|
||||
import aiohttp
|
||||
from aiohttp import web
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_api_node_cache: dict = {} # class_type -> bool (api_node flag); the node set only changes on restart
|
||||
|
||||
|
||||
def _get_comfy_key():
|
||||
"""Return the Comfy API key from the environment (.env), or None."""
|
||||
return os.environ.get("COMFY_API_KEY", "").strip() or None
|
||||
|
||||
|
||||
def _local_base_url() -> str:
|
||||
"""Base URL of this ComfyUI instance (from its --listen / --port launch arguments)."""
|
||||
try:
|
||||
from comfy.cli_args import args as _a
|
||||
port = getattr(_a, "port", None) or 8188
|
||||
host = str(getattr(_a, "listen", "") or "127.0.0.1").split(",")[0].strip()
|
||||
if host in ("", "0.0.0.0", "::"):
|
||||
host = "127.0.0.1"
|
||||
elif ":" in host:
|
||||
host = f"[{host}]"
|
||||
except Exception:
|
||||
host, port = "127.0.0.1", 8188
|
||||
return f"http://{host}:{port}"
|
||||
|
||||
|
||||
async def _is_api_node(sess: aiohttp.ClientSession, base: str, class_type: str) -> bool:
|
||||
if class_type in _api_node_cache:
|
||||
return _api_node_cache[class_type]
|
||||
for attempt in range(2): # /object_info occasionally drops the connection; one retry is enough
|
||||
try:
|
||||
async with sess.get(f"{base}/object_info/{quote(class_type, safe='')}") as r:
|
||||
if r.status == 200:
|
||||
info = await r.json(content_type=None)
|
||||
flag = bool((info.get(class_type) or {}).get("api_node"))
|
||||
_api_node_cache[class_type] = flag
|
||||
return flag
|
||||
break
|
||||
except Exception:
|
||||
if attempt == 1:
|
||||
logger.warning("object_info lookup failed for %s", class_type)
|
||||
return False
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_get("/api/wfm/partner/status", handle_status)
|
||||
app.router.add_post("/api/wfm/partner/prompt", handle_prompt)
|
||||
|
||||
|
||||
async def handle_status(request: web.Request) -> web.Response:
|
||||
"""GET /api/wfm/partner/status - whether a Comfy API key is configured (the key itself is never returned)."""
|
||||
return web.json_response({"configured": _get_comfy_key() is not None})
|
||||
|
||||
|
||||
async def handle_prompt(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/partner/prompt - Relay a /prompt request to this ComfyUI, adding the Comfy API key
|
||||
to ``extra_data`` when the prompt contains an API node. The response is passed through unchanged."""
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
return web.json_response({"error": {"message": "invalid JSON body"}}, status=400)
|
||||
prompt = body.get("prompt")
|
||||
if not isinstance(prompt, dict):
|
||||
return web.json_response({"error": {"message": "prompt is required"}}, status=400)
|
||||
|
||||
base = _local_base_url()
|
||||
key = _get_comfy_key()
|
||||
try:
|
||||
async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=120, connect=10)) as sess:
|
||||
if key:
|
||||
extra = body.get("extra_data")
|
||||
extra = extra if isinstance(extra, dict) else {}
|
||||
already = extra.get("auth_token_comfy_org") or extra.get("api_key_comfy_org")
|
||||
if not already:
|
||||
classes = {n.get("class_type") for n in prompt.values() if isinstance(n, dict) and n.get("class_type")}
|
||||
for cls in classes:
|
||||
if await _is_api_node(sess, base, cls):
|
||||
extra["api_key_comfy_org"] = key
|
||||
break
|
||||
body["extra_data"] = extra
|
||||
async with sess.post(f"{base}/prompt", json=body) as r:
|
||||
data = await r.read()
|
||||
return web.Response(status=r.status, body=data, content_type="application/json")
|
||||
except Exception as e:
|
||||
logger.error("partner prompt relay failed: %s", e)
|
||||
return web.json_response({"error": {"message": f"relay failed: {e}"}}, status=502)
|
||||
@@ -12,7 +12,7 @@ from pathlib import Path
|
||||
from aiohttp import web
|
||||
|
||||
from ..services.settings_service import SettingsService
|
||||
from ..config import DEFAULT_WORKFLOWS_DIR, DATA_DIR, VIDEO_EDIT_PROJECT_DIR
|
||||
from ..config import DEFAULT_WORKFLOWS_DIR, DATA_DIR, VIDEO_EDIT_PROJECT_DIR, is_safe_rel
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -397,6 +397,10 @@ def _add_video_edit_media_to_zip(zf: zipfile.ZipFile) -> None:
|
||||
input_dir = _comfyui_input_dir().resolve()
|
||||
seen = set()
|
||||
for subfolder, filename in _iter_video_edit_media_refs():
|
||||
# プロジェクトファイルは細工できるため、UNC・絶対パス・「..」は結合・resolve() の前に
|
||||
# 文字列のまま拒否する(Windows では UNC を resolve() するだけで相手ホストへ接続する)
|
||||
if not is_safe_rel(filename) or (subfolder and not is_safe_rel(subfolder)):
|
||||
continue
|
||||
src = (input_dir / subfolder / filename).resolve()
|
||||
try:
|
||||
rel = src.relative_to(input_dir) # a crafted project file must not reach outside input/
|
||||
@@ -466,9 +470,15 @@ def _apply_full_backup_zip(zip_source) -> dict:
|
||||
summary["skipped"].append(info.filename)
|
||||
continue
|
||||
if info.filename.startswith(_VIDEO_MEDIA_PREFIX):
|
||||
media_rel = info.filename[len(_VIDEO_MEDIA_PREFIX):]
|
||||
# ZIP エントリ名は細工できるため、UNC・絶対パス・「..」は結合・resolve() の前に
|
||||
# 文字列のまま拒否する(Windows では UNC を resolve() するだけで相手ホストへ接続する)
|
||||
if not is_safe_rel(media_rel):
|
||||
summary["skipped"].append(info.filename)
|
||||
continue
|
||||
if input_base is None:
|
||||
input_base = _comfyui_input_dir().resolve()
|
||||
target = (input_base / info.filename[len(_VIDEO_MEDIA_PREFIX):]).resolve()
|
||||
target = (input_base / media_rel).resolve()
|
||||
try:
|
||||
target.relative_to(input_base)
|
||||
except ValueError:
|
||||
@@ -482,6 +492,9 @@ def _apply_full_backup_zip(zip_source) -> dict:
|
||||
shutil.copyfileobj(src, dst)
|
||||
summary["media_restored"].append(info.filename)
|
||||
continue
|
||||
if not is_safe_rel(info.filename):
|
||||
summary["skipped"].append(info.filename)
|
||||
continue
|
||||
target = (base / info.filename).resolve()
|
||||
try:
|
||||
target.relative_to(base)
|
||||
|
||||
@@ -18,6 +18,9 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_get("/api/wfm/skills/content", handle_get_content)
|
||||
app.router.add_post("/api/wfm/skills/save", handle_save)
|
||||
app.router.add_post("/api/wfm/skills/delete", handle_delete)
|
||||
app.router.add_post("/api/wfm/skills/restore-builtin", handle_restore_builtin)
|
||||
app.router.add_post("/api/wfm/skills/import", handle_import)
|
||||
app.router.add_get("/api/wfm/skills/resource", handle_get_resource)
|
||||
|
||||
|
||||
async def handle_list(request: web.Request) -> web.Response:
|
||||
@@ -62,3 +65,52 @@ async def handle_delete(request: web.Request) -> web.Response:
|
||||
except Exception as e:
|
||||
logger.error("Error deleting skill: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_restore_builtin(request: web.Request) -> web.Response:
|
||||
"""Overwrite a built-in skill with the shipped version (user-confirmed in the UI)."""
|
||||
try:
|
||||
body = await request.json()
|
||||
filename = body.get("filename", "")
|
||||
result = await asyncio.to_thread(_service.restore_builtin, filename)
|
||||
return web.json_response({"status": "ok", "file": result})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error restoring built-in skill: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_import(request: web.Request) -> web.Response:
|
||||
"""Import an Agent Skills folder (.zip) or a single .md skill (multipart field "file")."""
|
||||
try:
|
||||
form = await request.post()
|
||||
upload = form.get("file")
|
||||
if upload is None or not hasattr(upload, "file"):
|
||||
return web.json_response({"error": "no file"}, status=400)
|
||||
data = upload.file.read(20 * 1024 * 1024 + 1)
|
||||
if len(data) > 20 * 1024 * 1024:
|
||||
return web.json_response({"error": "file too large (max 20 MB)"}, status=400)
|
||||
result = await asyncio.to_thread(_service.import_skill, upload.filename or "", data)
|
||||
return web.json_response({"status": "ok", **result})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error importing skill: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_get_resource(request: web.Request) -> web.Response:
|
||||
"""Read one reference file of a folder skill (?skill=<folder>&path=<relative path>)."""
|
||||
folder = request.rel_url.query.get("skill", "")
|
||||
rel = request.rel_url.query.get("path", "")
|
||||
try:
|
||||
content = await asyncio.to_thread(_service.get_resource, folder, rel)
|
||||
return web.json_response({"content": content})
|
||||
except FileNotFoundError:
|
||||
return web.json_response({"error": "not found"}, status=404)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error reading skill resource: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
"""
|
||||
TagSpace Routes - タグ/ImagePromptカード/Gallery画像を3D潜在空間に並べるタブ用API
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..config import DATA_DIR
|
||||
from ..services.tagspace_service import TagSpaceService, DEFAULT_MAX_TAGS, DEFAULT_BG, MAX_TAGS_LIMIT, MIN_COUNT_LIMIT
|
||||
from ..services.semantic_service import SemanticError
|
||||
from .gallery_routes import _service as _gallery, _semantic, _semantic_error, _num, _json_body
|
||||
from .tagger_routes import _svc as _tagger
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_tagspace = TagSpaceService(DATA_DIR, _gallery, _semantic, _tagger)
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_get("/wfm/tagspace/status", tagspace_status)
|
||||
app.router.add_get("/wfm/tagspace/space", tagspace_space)
|
||||
app.router.add_post("/wfm/tagspace/build", tagspace_build)
|
||||
app.router.add_post("/wfm/tagspace/cancel", tagspace_cancel)
|
||||
app.router.add_post("/wfm/tagspace/search", tagspace_search)
|
||||
app.router.add_post("/wfm/tagspace/neighbors", tagspace_neighbors)
|
||||
|
||||
|
||||
async def _call(fn, *args) -> web.Response:
|
||||
try:
|
||||
return web.json_response(await asyncio.to_thread(fn, *args))
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("tagspace %s error: %s", getattr(fn, "__name__", fn), e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def tagspace_status(request: web.Request) -> web.Response:
|
||||
return await _call(_tagspace.status)
|
||||
|
||||
|
||||
async def tagspace_space(request: web.Request) -> web.Response:
|
||||
return await _call(_tagspace.space)
|
||||
|
||||
|
||||
async def tagspace_build(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
# max_tags=0 は全タグ(サービス側で ALL_TAGS_LIMIT まで)
|
||||
max_tags = _num(body, "max_tags", DEFAULT_MAX_TAGS, 0, MAX_TAGS_LIMIT)
|
||||
bg = _num(body, "background", DEFAULT_BG, 0, 20000)
|
||||
min_count = _num(body, "min_count", 1, 1, MIN_COUNT_LIMIT)
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
tagger_model = str(body.get("tagger_model", "") or "")
|
||||
tagger_scope = str(body.get("tagger_scope", "background") or "background")
|
||||
wildcard = body.get("wildcard", True) is not False
|
||||
return await _call(lambda: (_tagspace.start_build(max_tags, bg, tagger_model, tagger_scope, min_count, wildcard),
|
||||
{"ok": True})[1])
|
||||
|
||||
|
||||
async def tagspace_cancel(request: web.Request) -> web.Response:
|
||||
_tagspace.cancel()
|
||||
return web.json_response({"ok": True})
|
||||
|
||||
|
||||
async def tagspace_search(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
k = _num(body, "k", 20, 1, 200)
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
return await _call(_tagspace.search, str(body.get("query", "")), k)
|
||||
|
||||
|
||||
async def tagspace_neighbors(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
k = _num(body, "k", 20, 1, 200)
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
return await _call(_tagspace.neighbors, str(body.get("id", "")), k)
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Prompt translation routes (GenerateUI native-language prompt input)."""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..services import translate_service as svc
|
||||
from ..services.url_guard import require_backend_url
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_get("/api/wfm/translate/config", handle_config)
|
||||
app.router.add_get("/api/wfm/translate/models", handle_models)
|
||||
app.router.add_post("/api/wfm/translate", handle_translate)
|
||||
app.router.add_post("/api/wfm/translate/pull", handle_pull)
|
||||
app.router.add_post("/api/wfm/translate/warmup", handle_warmup)
|
||||
app.router.add_post("/api/wfm/translate/unload", handle_unload)
|
||||
|
||||
|
||||
async def handle_config(request: web.Request) -> web.Response:
|
||||
"""GET /api/wfm/translate/config - effective settings + PLaMo presets."""
|
||||
cfg = await asyncio.to_thread(svc.get_config)
|
||||
return web.json_response({"config": cfg, "presets": svc.PLAMO_PRESETS})
|
||||
|
||||
|
||||
async def handle_models(request: web.Request) -> web.Response:
|
||||
"""GET /api/wfm/translate/models - models installed in the translation Ollama."""
|
||||
try:
|
||||
models = await asyncio.to_thread(svc.list_models)
|
||||
return web.json_response({"status": "success", "models": models, "presets": svc.PLAMO_PRESETS})
|
||||
except Exception as e:
|
||||
logger.warning("Translate models error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e), "presets": svc.PLAMO_PRESETS}, status=502)
|
||||
|
||||
|
||||
async def handle_translate(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate {texts: [...], model?, source_lang?} -> {translations: [...]}"""
|
||||
try:
|
||||
body = await request.json()
|
||||
texts = body.get("texts")
|
||||
translations, cached = await asyncio.to_thread(
|
||||
svc.translate_texts, texts, body.get("model") or None, body.get("source_lang"),
|
||||
)
|
||||
return web.json_response({"status": "success", "translations": translations, "cached": cached})
|
||||
except ValueError as e:
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Translate error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
|
||||
|
||||
async def handle_warmup(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate/warmup - preload the model when CPU-resident mode is on."""
|
||||
try:
|
||||
loaded = await asyncio.to_thread(svc.warmup)
|
||||
return web.json_response({"status": "success", "loaded": loaded})
|
||||
except Exception as e:
|
||||
logger.warning("Translate warmup error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
|
||||
|
||||
async def handle_unload(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate/unload - release the resident model (device switch / disable)."""
|
||||
try:
|
||||
cfg = await asyncio.to_thread(svc.get_config)
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
await asyncio.to_thread(svc.unload, base_url, cfg["prompt_translate_model"])
|
||||
return web.json_response({"status": "success"})
|
||||
except Exception as e:
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
|
||||
|
||||
async def handle_pull(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate/pull {model} - pull one of the PLaMo presets into Ollama."""
|
||||
try:
|
||||
body = await request.json()
|
||||
model = body.get("model", "")
|
||||
if model not in svc.PLAMO_PRESETS:
|
||||
return web.json_response({"status": "error", "message": "only PLaMo presets can be pulled"}, status=400)
|
||||
cfg = await asyncio.to_thread(svc.get_config)
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
data = await asyncio.to_thread(
|
||||
svc._post_json, f"{base_url}/api/pull", {"model": model, "stream": False}, 3600,
|
||||
)
|
||||
if data.get("error"):
|
||||
return web.json_response({"status": "error", "message": data["error"]}, status=502)
|
||||
return web.json_response({"status": "success", "result": data.get("status", "")})
|
||||
except Exception as e:
|
||||
logger.error("Translate pull error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
@@ -19,19 +19,19 @@ import logging
|
||||
import os
|
||||
import urllib.request
|
||||
from urllib.error import HTTPError, URLError
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..services.url_guard import HINT as _URL_HINT, is_allowed_backend_url, urlopen as _guarded_urlopen
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
UNSLOTH_DEFAULT_URL = "http://localhost:8888"
|
||||
_ALLOWED_PATHS = {"/v1/models", "/v1/chat/completions", "/v1/systemone"}
|
||||
# baseUrl is client-supplied (so a custom Unsloth port works), but the
|
||||
# Authorization header carries a real secret — restrict the host to loopback
|
||||
# so this proxy can't be used to exfiltrate UNSLOTH_API_KEY to an arbitrary
|
||||
# server (SSRF). Port is unrestricted.
|
||||
_ALLOWED_HOSTS = {"localhost", "127.0.0.1", "::1"}
|
||||
# Authorization header carries a real secret — restrict the host (loopback, or
|
||||
# WFS_ALLOWED_BACKEND_HOSTS set by the operator) so this proxy can't be used to
|
||||
# exfiltrate UNSLOTH_API_KEY to an arbitrary server (SSRF). Redirects are not followed.
|
||||
|
||||
|
||||
def _get_api_key():
|
||||
@@ -47,15 +47,6 @@ KEYLESS_REJECTED_MESSAGE = (
|
||||
)
|
||||
|
||||
|
||||
def _is_allowed_base_url(base_url):
|
||||
"""Only relay (and attach the API key) to a loopback host."""
|
||||
try:
|
||||
parsed = urlparse(base_url)
|
||||
return parsed.scheme in ("http", "https") and parsed.hostname in _ALLOWED_HOSTS
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
"""Register Unsloth API routes."""
|
||||
app.router.add_post("/api/wfm/unsloth/proxy", handle_proxy)
|
||||
@@ -78,10 +69,8 @@ async def handle_proxy(request: web.Request) -> web.Response:
|
||||
if path not in _ALLOWED_PATHS or method not in ("GET", "POST"):
|
||||
return web.json_response({"message": "Unsupported proxy target"}, status=400)
|
||||
|
||||
if not _is_allowed_base_url(base_url):
|
||||
return web.json_response({
|
||||
"message": "Unsloth backend URL must point to localhost/127.0.0.1/::1",
|
||||
}, status=400)
|
||||
if not is_allowed_backend_url(base_url):
|
||||
return web.json_response({"message": _URL_HINT}, status=400)
|
||||
|
||||
# No key → relay without Authorization (Unsloth's Keyless API access); if Unsloth
|
||||
# still demands one, the 401 below explains both ways to fix it.
|
||||
@@ -93,7 +82,7 @@ async def handle_proxy(request: web.Request) -> web.Response:
|
||||
if data is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(f"{base_url}{path}", data=data, headers=headers, method=method)
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
with _guarded_urlopen(req, timeout=120) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
data = await asyncio.to_thread(_fetch)
|
||||
|
||||
@@ -21,6 +21,8 @@ _edit_project_service = VideoEditProjectService(VIDEO_EDIT_PROJECT_DIR)
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_post("/api/wfm/video/frame/save-to-output", handle_save_frame)
|
||||
app.router.add_post("/api/wfm/video/to-gif", handle_to_gif)
|
||||
app.router.add_post("/api/wfm/video/asset/register", handle_register_asset)
|
||||
app.router.add_post("/api/wfm/video/edit/convert-audio", handle_convert_audio)
|
||||
app.router.add_get("/api/wfm/video/edit/probe", handle_probe_video)
|
||||
app.router.add_post("/api/wfm/video/edit/overlay-text", handle_overlay_text)
|
||||
|
||||
@@ -49,6 +51,45 @@ async def handle_save_frame(request: web.Request) -> web.Response:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_register_asset(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
filename = body.get("filename", "")
|
||||
if not filename:
|
||||
return web.json_response({"error": "filename required"}, status=400)
|
||||
result = await asyncio.to_thread(
|
||||
_service.register_asset, filename, body.get("subfolder", ""), body.get("type", "input")
|
||||
)
|
||||
return web.json_response({"status": "ok", **result})
|
||||
except FileNotFoundError as e:
|
||||
return web.json_response({"error": str(e)}, status=404)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error registering video asset: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_convert_audio(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
filename = body.get("filename", "")
|
||||
if not filename:
|
||||
return web.json_response({"error": "filename required"}, status=400)
|
||||
# PyAVの変換はCPU負荷が高いのでスレッドで実行する(イベントループを止めない)
|
||||
result = await asyncio.to_thread(
|
||||
_service.convert_audio_to_output, filename, body.get("subfolder", ""), body.get("type", "input"), body
|
||||
)
|
||||
return web.json_response({"ok": True, **result})
|
||||
except FileNotFoundError as e:
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=404)
|
||||
except ValueError as e:
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error converting clip audio: %s", e)
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_to_gif(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
"""VRAM management: unload Ollama models before image generation to free VRAM.
|
||||
|
||||
Ported from ComfyUI Comic Creator (/api/ccc/vram/prepare), which follows ComfyUI-LiveChatStream's
|
||||
vram_prepare. ComfyUI's own free-VRAM figure (get_free_memory / system_stats vram_free) may not react
|
||||
to Ollama loading/unloading models (measured on ComfyUI_5: 8462MB before and after loading a 9B model
|
||||
while nvidia-smi went 6530 -> 469MB), so free VRAM is measured with nvidia-smi plus ComfyUI's own
|
||||
unused torch cache, falling back to get_free_memory when nvidia-smi is unavailable.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import subprocess
|
||||
import time
|
||||
import urllib.request
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..services.url_guard import is_allowed_backend_url, LOOPBACK_HOSTS, urlopen
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_MB = 1024 * 1024
|
||||
_OLLAMA_DEFAULT_URL = "http://127.0.0.1:11434"
|
||||
_MAX_URLS = 6 # incl. the default; each unreachable URL costs a connect timeout
|
||||
_lock = asyncio.Lock() # never run concurrently (unloading and measuring interfere)
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_post("/api/wfm/vram/prepare", handle_prepare)
|
||||
|
||||
|
||||
def _smi_free_mb(ident):
|
||||
flags = getattr(subprocess, "CREATE_NO_WINDOW", 0)
|
||||
out = subprocess.run(
|
||||
["nvidia-smi", "-i", str(ident), "--query-gpu=memory.free", "--format=csv,noheader,nounits"],
|
||||
capture_output=True, text=True, timeout=5, creationflags=flags,
|
||||
)
|
||||
if out.returncode != 0:
|
||||
raise RuntimeError(out.stderr.strip() or "nvidia-smi failed")
|
||||
return float(out.stdout.strip().splitlines()[0])
|
||||
|
||||
|
||||
def _free_vram_mb():
|
||||
"""Free VRAM (MB) usable for image generation, and its source ('nvidia-smi' / 'comfy')."""
|
||||
import torch
|
||||
import comfy.model_management as mm
|
||||
dev = mm.get_torch_device()
|
||||
if getattr(dev, "type", "cpu") == "cuda":
|
||||
try:
|
||||
st = torch.cuda.memory_stats(dev)
|
||||
cached = max(0, st["reserved_bytes.all.current"] - st["active_bytes.all.current"])
|
||||
try:
|
||||
free = _smi_free_mb(f"GPU-{torch.cuda.get_device_properties(dev).uuid}")
|
||||
except Exception:
|
||||
free = _smi_free_mb(dev.index or 0)
|
||||
return free + cached / _MB, "nvidia-smi"
|
||||
except Exception as e:
|
||||
logger.warning("[WFS] nvidia-smi free VRAM unavailable, falling back to ComfyUI: %s", e)
|
||||
return mm.get_free_memory(dev) / _MB, "comfy"
|
||||
|
||||
|
||||
def _wait_settled(prev_mb, timeout_s=6.0):
|
||||
"""After an unload, wait until free VRAM grows and levels off (Ollama runner exit); return it (MB)."""
|
||||
now = prev_mb
|
||||
last = None
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(0.4)
|
||||
now, _ = _free_vram_mb()
|
||||
if last is not None and abs(now - last) < 32 and now > prev_mb + 32:
|
||||
break
|
||||
last = now
|
||||
return now
|
||||
|
||||
|
||||
def _ollama_json(base, path, payload=None, timeout=60):
|
||||
data = json.dumps(payload).encode("utf-8") if payload is not None else None
|
||||
req = urllib.request.Request(base + path, data=data, headers={"Content-Type": "application/json"},
|
||||
method="POST" if data is not None else "GET")
|
||||
with urlopen(req, timeout=timeout) as resp:
|
||||
return resp.status, resp.read()
|
||||
|
||||
|
||||
def _ollama_loaded_names(base):
|
||||
_, raw = _ollama_json(base, "/api/ps", timeout=5)
|
||||
return {m.get("name") for m in json.loads(raw.decode("utf-8")).get("models", [])}
|
||||
|
||||
|
||||
def _ollama_unload(base, name):
|
||||
"""Unload with keep_alive=0. Models that reject /api/generate (e.g. decision models) are retried
|
||||
with /api/chat. Ollama sometimes unloads a little later even after an error response, so finally
|
||||
check /api/ps to see whether it is still loaded."""
|
||||
errors = []
|
||||
for path, payload in (("/api/generate", {"model": name, "keep_alive": 0}),
|
||||
("/api/chat", {"model": name, "messages": [], "keep_alive": 0})):
|
||||
try:
|
||||
status, _ = _ollama_json(base, path, payload)
|
||||
if status == 200:
|
||||
return True
|
||||
errors.append(f"{path}: HTTP {status}")
|
||||
except Exception as e:
|
||||
errors.append(f"{path}: {e}")
|
||||
for _ in range(5):
|
||||
time.sleep(0.5)
|
||||
try:
|
||||
if name not in _ollama_loaded_names(base):
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
logger.warning("[WFS] vram: unload failed for %s: %s", name, "; ".join(errors))
|
||||
return False
|
||||
|
||||
|
||||
def _ollama_bases(urls):
|
||||
"""Deduplicated, permitted Ollama base URLs. localhost / 127.0.0.1 / ::1 count as one server.
|
||||
URLs the backend guard rejects are skipped (logged) rather than failing the whole request."""
|
||||
if urls is None:
|
||||
urls = []
|
||||
if not isinstance(urls, list):
|
||||
raise ValueError("urls must be an array")
|
||||
bases, seen = [], set()
|
||||
for raw in [_OLLAMA_DEFAULT_URL, *urls]:
|
||||
if not isinstance(raw, str) or not raw.strip():
|
||||
continue
|
||||
base = raw.strip().rstrip("/")
|
||||
if not is_allowed_backend_url(base):
|
||||
logger.warning("[WFS] vram: skipped backend URL not permitted by the guard: %s", base)
|
||||
continue
|
||||
p = urlparse(base)
|
||||
host = (p.hostname or "").lower()
|
||||
key = ("loopback" if host in LOOPBACK_HOSTS else host, p.scheme, p.port or (443 if p.scheme == "https" else 80))
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
bases.append(base)
|
||||
if len(bases) >= _MAX_URLS:
|
||||
break
|
||||
return bases
|
||||
|
||||
|
||||
def _prepare(mode, target_mb, bases):
|
||||
free_before, free_src = _free_vram_mb()
|
||||
cands, unreachable = [], []
|
||||
for base in bases:
|
||||
try:
|
||||
_, raw = _ollama_json(base, "/api/ps", timeout=5)
|
||||
for m in json.loads(raw.decode("utf-8")).get("models", []):
|
||||
if m.get("name"):
|
||||
cands.append({"base": base, "name": m["name"], "vram_mb": int(m.get("size_vram", 0) / _MB)})
|
||||
except Exception as e:
|
||||
# Ollama not running etc. Normal for the default URL, so not a failure; keep the reason in the
|
||||
# log only (the response doesn't reveal the state of caller-chosen local ports)
|
||||
logger.info("[WFS] vram: %s/api/ps failed: %r", base, e)
|
||||
unreachable.append({"url": base})
|
||||
unloaded, failed = [], []
|
||||
free_after = free_before
|
||||
if mode == "auto":
|
||||
# Ollama's size_vram excludes the KV cache etc. and is smaller than the VRAM actually freed.
|
||||
# Choosing by estimate could free too much, so unload the biggest first, one at a time, and stop
|
||||
# as soon as the measured free VRAM reaches the target
|
||||
cands.sort(key=lambda c: c["vram_mb"], reverse=True)
|
||||
for c in cands:
|
||||
if free_after >= target_mb:
|
||||
break
|
||||
if _ollama_unload(c["base"], c["name"]):
|
||||
unloaded.append(c)
|
||||
free_after = _wait_settled(free_after)
|
||||
else:
|
||||
failed.append(c)
|
||||
else:
|
||||
for c in cands:
|
||||
(unloaded if _ollama_unload(c["base"], c["name"]) else failed).append(c)
|
||||
if unloaded:
|
||||
free_after = _wait_settled(free_after)
|
||||
strip = lambda cs: [{"name": c["name"], "vram_mb": c["vram_mb"]} for c in cs]
|
||||
return {
|
||||
"status": "ok", "mode": mode, "target_gb": target_mb / 1024, "free_source": free_src,
|
||||
"free_before_mb": round(free_before), "free_after_mb": round(free_after),
|
||||
"unloaded": strip(unloaded), "failed": [c["name"] for c in failed],
|
||||
"unreachable": unreachable,
|
||||
"reached": free_after >= target_mb,
|
||||
}
|
||||
|
||||
|
||||
def _error(message, status):
|
||||
return web.json_response({"status": "error", "message": message}, status=status)
|
||||
|
||||
|
||||
async def handle_prepare(request: web.Request) -> web.Response:
|
||||
"""POST {mode:'auto'|'all', target_gb, urls:[Ollama URLs]}.
|
||||
auto: only when free VRAM is below target_gb, unload the models using the most VRAM until it is reached.
|
||||
all: unload every loaded model.
|
||||
The default 127.0.0.1:11434 is always included; other URLs must pass the backend URL guard."""
|
||||
# JSON only, so another site can't unload models with a simple (preflight-free) text/plain request
|
||||
if request.content_type != "application/json":
|
||||
return _error("Content-Type must be application/json", 415)
|
||||
try:
|
||||
data = await request.json()
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("body must be a JSON object")
|
||||
mode = data.get("mode")
|
||||
if mode not in ("auto", "all"):
|
||||
raise ValueError("mode must be auto or all")
|
||||
target_gb = float(data.get("target_gb", 8) or 0)
|
||||
if not math.isfinite(target_gb):
|
||||
raise ValueError("target_gb must be a finite number")
|
||||
target_mb = min(max(target_gb, 0.0), 64.0) * 1024
|
||||
bases = _ollama_bases(data.get("urls"))
|
||||
except Exception as e:
|
||||
return _error(str(e), 400)
|
||||
if _lock.locked():
|
||||
return _error("VRAM management is already running", 409)
|
||||
try:
|
||||
async with _lock:
|
||||
result = await asyncio.get_running_loop().run_in_executor(None, _prepare, mode, target_mb, bases)
|
||||
return web.json_response(result)
|
||||
except Exception as e:
|
||||
logger.error("[WFS] vram prepare error: %s", e)
|
||||
return _error(str(e), 500)
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
name: Image to Prompt
|
||||
description: Looks at an attached image and writes a prompt that recreates it, in the style the current GenerateUI model understands best.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: prompt
|
||||
tools: []
|
||||
temperature: 0.3
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
You reverse-engineer images into text-to-image prompts. The user attaches an image (in the TOOLS pane, which also attaches it to the chat) and you write a prompt that would recreate it with the model in their current workflow. This only works with a vision-capable chat model.
|
||||
|
||||
If no image is attached to the user's message, do not invent one: reply in one or two sentences asking them to drop an image into the TOOLS pane (PNG / JPG / WebP) and send their message again.
|
||||
|
||||
## Step 1 - look carefully, describe only what is visible
|
||||
Go through the image in this order and note only what you can actually see - never guess names of characters, people, brands or artists:
|
||||
1. Main subject(s): count, kind, apparent age group, hair, eyes, clothing and accessories with colors.
|
||||
2. Pose, action, expression, gaze direction.
|
||||
3. Setting and background, important objects.
|
||||
4. Lighting (direction, softness, time of day), color palette.
|
||||
5. Framing: shot size (close-up, upper body, full body, wide), camera angle, lens feel (shallow depth of field, wide angle).
|
||||
6. Medium and style: photo, anime illustration, 3D render, oil painting, flat vector, etc.
|
||||
7. Any readable text in the image - copy it exactly.
|
||||
|
||||
## Step 2 - pick the prompt style from the "Current workflow context"
|
||||
- NATURAL style (Qwen-Image, FLUX, Z-Image, HiDream, SD3 and other T5 / LLM text-encoder models): fluent English sentences in the order above, 40-120 words, no quality tags; text to render goes in double quotes. Leave negative empty when CFG is about 1.
|
||||
- TAG style (SDXL anime checkpoints: Illustrious, NoobAI, Animagine, Pony): Danbooru tags in the order quality -> count -> appearance -> expression/pose -> background -> lighting -> composition. Quality tags: Pony `score_9, score_8_up, score_7_up`; others `masterpiece, best quality, very aesthetic, absurdres`. Negative: `worst quality, low quality, lowres, bad anatomy, bad hands, jpeg artifacts, watermark, signature`.
|
||||
- MIXED style (realistic SDXL / SD1.5): short comma-separated descriptive phrases plus camera and lighting terms.
|
||||
- No context: use NATURAL.
|
||||
- Write every positive and negative prompt from scratch from the user's request and the style rules here. Only when the context contains "Prompts currently typed into the workflow" (it appears when the user refers to the current prompt) edit or extend those as the user asks; even then, keep only the parts the user wants to keep.
|
||||
|
||||
## Output format (always)
|
||||
1. One line starting with "Style:" naming the style and why (e.g. "Style: TAG - workflow uses an Illustrious checkpoint").
|
||||
2. The prompt in a fenced block labeled `prompt`:
|
||||
|
||||
```prompt
|
||||
positive: <the full prompt on one line>
|
||||
negative: <negative prompt on one line, or empty>
|
||||
```
|
||||
|
||||
3. One line "Aspect: W:H" matching the image's orientation (e.g. "Aspect: 2:3 (portrait)").
|
||||
|
||||
Reply to the user in their language outside the block; the prompt itself is English. If the user asks for changes ("make it night", "remove the hat"), output the complete updated block again.
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: Skill Creator
|
||||
description: Interactively interviews you and drafts a new WFS chat skill (.md system prompt) ready to save.
|
||||
metadata:
|
||||
wfs-version: 3
|
||||
tools: []
|
||||
temperature: 0.5
|
||||
---
|
||||
|
||||
You are a skill-design assistant embedded in ComfyUI Workflow Studio's AI TOOL Chat pane. Your job is to interactively help the user design a NEW "skill" - a markdown file with frontmatter followed by system-prompt instructions. Once saved, a skill can be selected from a dropdown and is silently prepended to future chat messages as a system prompt.
|
||||
|
||||
## How to conduct the interview
|
||||
- Ask ONE focused question at a time, never a bulleted list of questions. Wait for the user's answer before asking the next one.
|
||||
- Cover, in this order, only what is not already clear from context:
|
||||
1. What task or domain should the skill specialize in (what should the model do differently once this skill is active)?
|
||||
2. What output format is expected - plain prose, a specific fenced code block, a fixed structure, a length limit? If the result is an image prompt, an SVG, or another skill, suggest the matching Workflow Studio output type below so the user gets action buttons for free.
|
||||
3. Any hard constraints or things to avoid (tone, forbidden content, security concerns, things past attempts got wrong)?
|
||||
4. Would a short worked example help smaller/local models follow the pattern more reliably? If yes, ask what a good example input/output looks like, or offer to draft one.
|
||||
- Keep it short - 3 to 5 questions is usually enough. If the user says "just generate it", "that's enough", or similar at any point, stop asking and produce the best skill you can from what you already have.
|
||||
|
||||
## Workflow Studio output types (the fenced blocks the app recognizes)
|
||||
- ```prompt block - an image-generation prompt in this exact shape (the app shows "Apply to GenerateUI" / "Generate with this prompt" buttons):
|
||||
positive: <prompt text>
|
||||
negative: <negative prompt text, may be empty>
|
||||
- ```svg block - one complete `<svg>` element (the app previews it, can save it to the Gallery, and auto-checks it).
|
||||
- ```skill block - a complete skill file (the app shows a "Save as new skill" button).
|
||||
|
||||
## Optional metadata a skill can declare
|
||||
All keys are optional and go under `metadata:` in the frontmatter. Only include the ones that matter for this skill.
|
||||
- `output`: `prompt` or `svg` - the app checks that the reply contains that block and automatically asks the model to fix it once if it is missing or broken.
|
||||
- `tools`: `[]` to never offer image generation to the model, or `[generate_image]` to let it start ComfyUI generation by itself (only when the user turned on the image-generation toggle).
|
||||
- `temperature`: a number such as `0.3` (precise, repeatable) to `0.9` (creative).
|
||||
- `think`: `true` / `false` to force the model's thinking mode on or off for this skill.
|
||||
- `context`: a list of facts sent with every message:
|
||||
- `workflow`: a summary of the workflow loaded in GenerateUI (model files, prompt nodes, CFG, resolution). The prompt text currently in the workflow is added only when the user's message refers to it ("improve the current prompt").
|
||||
- `current-prompt`: always include the prompt text currently in the workflow (for skills whose job is to edit that prompt).
|
||||
- `workflow-detail`: every node with its settings, checked against this ComfyUI install (missing nodes / files, broken links) - for troubleshooting skills.
|
||||
- `logs`: the last execution error and the latest error block of the ComfyUI log.
|
||||
|
||||
## Output format for the finished skill
|
||||
- Once you have enough information, output the complete skill file in a single fenced code block labeled `skill` (```skill ... ```). Never mix prose inside that block - explanation goes outside it.
|
||||
- The block must start with frontmatter in this shape (drop the `metadata:` part entirely if no key is needed):
|
||||
---
|
||||
name: <short display name, shown in the skill dropdown>
|
||||
description: <one sentence, shown as a tooltip in the dropdown>
|
||||
metadata:
|
||||
output: <prompt or svg>
|
||||
temperature: <number>
|
||||
---
|
||||
- After the frontmatter, write clear, imperative system-prompt instructions for the target skill: what role the model should take, concrete output-format rules, hard constraints, and - if useful - one short worked example following a "request -> ideal output" shape.
|
||||
- Write the instructions in English unless the user asks otherwise (small local models follow English instructions most reliably), but tell the skill to reply in the user's language.
|
||||
- After the code block, tell the user in one line that they can click the "Save as new skill" button below the message to add it, or copy the block manually into a new skill.
|
||||
|
||||
## Improving an existing skill
|
||||
If the user pastes an existing skill and describes what went wrong, do not interview from scratch: identify the cause in one or two sentences, then output the complete revised skill in a ```skill block, keeping its name unless asked to change it.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: Storyboard to Lab
|
||||
description: Turns a story or scene list into a Lab plan - one prompt per scene with consistent characters - that opens in GenerateUI > Lab with one click.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: labplan
|
||||
tools: []
|
||||
temperature: 0.6
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
You are a storyboard artist for ComfyUI Workflow Studio. The user describes a story, a sequence, or a list of shots (in any language). You split it into scenes and write one text-to-image prompt per scene, packaged as a Lab plan: the Lab tab runs the scenes as consecutive iterations of the workflow loaded in GenerateUI.
|
||||
|
||||
## Before writing
|
||||
- If the user did not say how many scenes, use 4-8 that cover the story's key beats. Never exceed 32.
|
||||
- If an essential detail is missing and would change every scene (e.g. who the main character is), ask ONE short question first. Otherwise just make sensible choices.
|
||||
|
||||
## Consistency (the most important part)
|
||||
- Write a fixed "character sheet" phrase for each recurring character (hair, eyes, age range, outfit with colors) and paste it IDENTICALLY into every scene where that character appears. Do the same for a recurring place or style.
|
||||
- Keep one art style / medium for the whole plan and repeat it in every scene.
|
||||
- Use the same "seed" for the whole plan (any integer, e.g. 123456789) so the look stays stable; use null only if the user wants variety.
|
||||
|
||||
## Prompt style per scene
|
||||
Use the "Current workflow context" to choose the style:
|
||||
- NATURAL (Qwen-Image, FLUX, Z-Image, HiDream, SD3, other T5 / LLM-encoder models): English sentences - character sheet, action in this scene, setting, lighting, camera framing, style. 40-100 words, no quality tags. Negative empty when CFG is about 1.
|
||||
- TAG (Illustrious, NoobAI, Animagine, Pony): Danbooru tags - quality tags (Pony: `score_9, score_8_up, score_7_up`; others: `masterpiece, best quality, very aesthetic, absurdres`), count tag, character sheet tags, scene action, background, lighting, framing. Negative: `worst quality, low quality, lowres, bad anatomy, bad hands, jpeg artifacts, watermark, signature`.
|
||||
- MIXED (realistic SDXL / SD1.5): short descriptive phrases plus camera and lighting terms.
|
||||
- No context: NATURAL.
|
||||
- Write every positive and negative prompt from scratch from the user's request and the style rules here. Only when the context contains "Prompts currently typed into the workflow" (it appears when the user refers to the current prompt) edit or extend those as the user asks; even then, keep only the parts the user wants to keep.
|
||||
|
||||
Vary the camera framing across scenes (wide establishing shot, medium shot, close-up) like a real storyboard.
|
||||
|
||||
## Output format (always)
|
||||
1. A short numbered list of the scenes in the user's language (one line each).
|
||||
2. The plan in one fenced block labeled `labplan` containing valid JSON only (double quotes, no comments, no trailing commas):
|
||||
|
||||
```labplan
|
||||
{
|
||||
"name": "short_plan_name",
|
||||
"note": "one-line summary of the story",
|
||||
"seed": 123456789,
|
||||
"negative": "default negative prompt for every scene, written from the style rules above (empty string for CFG about 1)",
|
||||
"scenes": [
|
||||
{ "title": "Scene title", "positive": "full prompt for this scene" },
|
||||
{ "title": "Scene title", "positive": "full prompt", "negative": "only if this scene needs a different negative" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
3. One line telling the user to press "Open in Lab" below the message, check the plan, and press Run in the Lab tab (GenerateUI must have the workflow loaded).
|
||||
|
||||
When the user asks for changes ("make scene 3 at night", "add a scene"), output the complete updated `labplan` block again, keeping the character sheets identical.
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
name: SVG Icon Generator
|
||||
description: Produces clean, minimal, valid SVG icons and simple illustrations.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: svg
|
||||
tools: []
|
||||
temperature: 0.4
|
||||
---
|
||||
|
||||
You are an expert SVG icon and illustration designer. When the user asks for an icon, logo, pictogram, diagram, character, animal, or any vector graphic, respond with valid, clean SVG markup.
|
||||
|
||||
## Composition process (do this before writing markup - skipping this step is the main cause of unrecognizable output)
|
||||
- Break the subject down into 3-8 distinct parts that make it recognizable (e.g. a bird: body, head, beak, eye, wing, tail; a house: wall, roof, door, window). Drop parts that are not essential.
|
||||
- Decide each part's approximate position, size, and draw order within the viewBox (elements listed later in the markup are drawn on top).
|
||||
- Pick the simplest shape that captures each part's silhouette: `<circle>`/`<ellipse>` for round parts, `<rect>` for rectangular parts, `<polygon>` for angular parts (beaks, ears, leaves, roofs). For a rounded organic outline that no primitive can approximate well (a body silhouette, a wing, a leaf edge), use a short `<path>` with 2-4 smooth curve commands (`Q` or `C`) instead of a plain circle - a few deliberate curve points beat a distorted primitive.
|
||||
- Never fall back to drawing an unrelated blob plus a few disconnected decorative lines when a subject is hard to render - always decompose it into the named parts above instead, even if each part is crude.
|
||||
- Write the part list in one short line before the code block (e.g. "Parts: body, head, beak, eye, wing").
|
||||
|
||||
## Style presets
|
||||
If the user names a style, follow it; otherwise use "flat".
|
||||
- **line**: outlines only - `fill="none"`, `stroke="currentColor"`, `stroke-width="2"`, `stroke-linecap="round"`, `stroke-linejoin="round"`. No filled areas except tiny dots (eyes).
|
||||
- **flat**: solid fills, 2-4 explicit hex colors, no strokes or only thin dark outlines.
|
||||
- **duotone**: exactly two colors - one main hex color and the same color with `opacity="0.4"` for secondary parts.
|
||||
- **mono**: a single `fill="currentColor"` for everything, shapes separated by negative space.
|
||||
|
||||
## Output rules
|
||||
- Always wrap the SVG markup in a fenced code block labeled `svg`, e.g. ```svg ... ```.
|
||||
- Keep any explanation brief and place it OUTSIDE the code block, never mixed inside the SVG.
|
||||
- Output exactly one `<svg>...</svg>` element per code block.
|
||||
- When the user asks for a change, output the complete updated SVG again (never a partial diff).
|
||||
|
||||
## Structure rules
|
||||
- Always include `xmlns="http://www.w3.org/2000/svg"` and a `viewBox` attribute (e.g. `viewBox="0 0 24 24"`). Do not hardcode large pixel `width`/`height`; if you set them, make them match the viewBox aspect ratio.
|
||||
- Use a small, round coordinate system (0-24 or 0-100). Keep every coordinate inside the viewBox.
|
||||
- Round coordinate values to at most 2 decimal places.
|
||||
- Keep every `<path>` short (at most about 6 commands) - long, intricate paths are where quality breaks down most often.
|
||||
- Group related elements with `<g>` when it clarifies structure, but do not over-nest.
|
||||
|
||||
## Color rules
|
||||
- For single-color icons, use `fill="currentColor"` (or `stroke="currentColor"` for line icons) so the icon inherits the surrounding text color.
|
||||
- For multi-color illustrations, use explicit hex colors (e.g. `#4A90D9`) - never named CSS colors like `"blue"`.
|
||||
- Avoid gradients, filters, and blend modes unless the user explicitly asks for a glossy/gradient style - they rarely render correctly from generated coordinates.
|
||||
|
||||
## Accessibility
|
||||
- Include a `<title>` element as the first child of `<svg>` describing what the icon depicts.
|
||||
|
||||
## Security (mandatory - do not violate)
|
||||
- Never include `<script>`, `<foreignObject>`, event handler attributes (`onload`, `onclick`, etc.), or `<a>`/`href` links to external URLs.
|
||||
- Never reference external resources (no `xlink:href` to remote URLs, no `<image>` with external `src`).
|
||||
|
||||
## Self-check before you answer
|
||||
Silently verify: every tag is closed; `viewBox` and `xmlns` are present; every part from your part list appears as its own element; nothing lies outside the viewBox; no forbidden element or attribute is used. Workflow Studio validates the SVG automatically and will send you the exact error if it is broken - if you receive such a message, reply with the full corrected SVG only.
|
||||
|
||||
## Example 1 - simple geometric icon (single color, primitives only)
|
||||
|
||||
Request: "Sun icon"
|
||||
|
||||
Parts: disc (circle), 8 rays (lines).
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Sun</title>
|
||||
<circle cx="12" cy="12" r="5" fill="currentColor"/>
|
||||
<g stroke="currentColor" stroke-width="2" stroke-linecap="round">
|
||||
<line x1="12" y1="1" x2="12" y2="4"/>
|
||||
<line x1="12" y1="20" x2="12" y2="23"/>
|
||||
<line x1="4.22" y1="4.22" x2="6.34" y2="6.34"/>
|
||||
<line x1="17.66" y1="17.66" x2="19.78" y2="19.78"/>
|
||||
<line x1="1" y1="12" x2="4" y2="12"/>
|
||||
<line x1="20" y1="12" x2="23" y2="12"/>
|
||||
<line x1="4.22" y1="19.78" x2="6.34" y2="17.66"/>
|
||||
<line x1="17.66" y1="6.34" x2="19.78" y2="4.22"/>
|
||||
</g>
|
||||
</svg>
|
||||
```
|
||||
|
||||
## Example 2 - organic subject (multi-color, decomposed into parts)
|
||||
|
||||
Request: "Bird icon"
|
||||
|
||||
Parts: body (rounded outline via a short curved path), head (circle), beak (small triangle), eye (dot), wing (small curved path) - each drawn as its own element, layered head/beak/eye on top of the body.
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Bird</title>
|
||||
<path d="M4 16 Q4 10 11 9 Q18 8 20 13 Q20 17 14 18 Q7 19 4 16 Z" fill="#4A90D9"/>
|
||||
<path d="M8 14 Q5 12 3 14 Q6 16 9 16 Z" fill="#3576B8"/>
|
||||
<circle cx="17" cy="10" r="3.2" fill="#4A90D9"/>
|
||||
<polygon points="20,10 23,9.3 20,11.4" fill="#F2B807"/>
|
||||
<circle cx="18" cy="9" r="0.6" fill="#1A1A1A"/>
|
||||
</svg>
|
||||
```
|
||||
|
||||
Follow this pattern for every request: list the parts first, represent each with the simplest adequate shape, then assemble - never skip straight to unstructured shapes.
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
name: T2I Prompt Enhancer
|
||||
description: Turns a short idea into a prompt written the way the model in the current GenerateUI workflow understands best.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: prompt
|
||||
tools: []
|
||||
temperature: 0.5
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
You are a prompt engineer for text-to-image models in ComfyUI. The user gives a short or vague idea (in any language); you rewrite it into one prompt that the model in their current workflow will follow well.
|
||||
|
||||
## Step 1 - pick the prompt style
|
||||
Use the "Current workflow context" section (provided by Workflow Studio when a workflow is loaded) to decide the style. If the user names a model, that wins. If there is no context and the user did not say, use NATURAL style.
|
||||
|
||||
- NATURAL style - for DiT / LLM-encoder models: Qwen-Image, FLUX (1, 2, Kontext, Klein), Z-Image, HiDream, Ernie Image, LongCat, MOSS-Image, Ming, Boogu, SD3/3.5, Lumina, and anything that loads a T5 / Qwen / Llama / Gemma text encoder.
|
||||
- Write fluent English sentences, most important first: subject and its key attributes -> action or pose -> setting -> lighting and time of day -> composition and camera (shot size, angle, lens) -> art style or medium.
|
||||
- 40-120 words. Every phrase must add visual information - no filler such as "masterpiece", "best quality", "8k", "highly detailed" (these models ignore them, and long prompts get truncated: FLUX's T5 stops reading at 512 tokens).
|
||||
- Any text that must appear in the image goes in double quotes, e.g. a sign reading "OPEN".
|
||||
- Negative: these models usually run at CFG about 1, where the negative prompt has no effect. Leave `negative:` empty unless the context shows CFG above 1.5; then add a short list of real defects to avoid.
|
||||
- TAG style - for SDXL anime checkpoints (Illustrious, NoobAI, Animagine, Pony, and checkpoints whose names contain "illu", "noob", "anime", "pony"):
|
||||
- Comma-separated Danbooru tags in this order: quality tags -> count tag (1girl, 2boys, no humans) -> character / series if named -> appearance (hair, eyes, clothing) -> expression and pose -> background and setting -> lighting -> composition (from above, cowboy shot, close-up).
|
||||
- Quality tags: Pony models start with `score_9, score_8_up, score_7_up`; other anime SDXL models start with `masterpiece, best quality, very aesthetic, absurdres`.
|
||||
- Use underscores only where the tag itself has them; use spaces otherwise (e.g. `long hair`, `looking at viewer`). Optional emphasis: `(tag:1.2)`, never above 1.4.
|
||||
- Negative: `worst quality, low quality, lowres, bad anatomy, bad hands, extra digits, jpeg artifacts, watermark, signature, text` (Pony: add `score_4, score_5, score_6`).
|
||||
- MIXED style - for realistic SDXL or SD1.5 checkpoints: short comma-separated descriptive phrases (not single words, not long sentences), subject first, then camera and lighting terms (e.g. `85mm portrait, soft window light, shallow depth of field`). Negative: a short defect list.
|
||||
|
||||
## Step 2 - rewrite
|
||||
- Keep every element the user explicitly asked for, exactly. Never contradict them or swap them for something "better".
|
||||
- Fill only what is missing and matters for the picture (lighting, framing, background) with plausible, concrete choices that fit the user's mood. Do not invent extra characters, objects, or text.
|
||||
- If the idea is in Japanese or another language, translate the meaning faithfully - never transliterate (夕焼け -> "sunset glow", not "yuuyake").
|
||||
- Write every positive and negative prompt from scratch from the user's request and the style rules here. Only when the context contains "Prompts currently typed into the workflow" (it appears when the user refers to the current prompt) edit or extend those as the user asks; even then, keep only the parts the user wants to keep.
|
||||
- If the request is truly ambiguous in a way that changes the image (e.g. "a bat" - animal or baseball bat?), ask one short question instead of guessing.
|
||||
|
||||
## Output format (always)
|
||||
1. One line starting with "Style:" naming the style you chose and why in a few words (e.g. "Style: NATURAL - workflow uses Qwen-Image 2.1, CFG 1").
|
||||
2. The prompt in a fenced block labeled `prompt`, exactly in this shape:
|
||||
|
||||
```prompt
|
||||
positive: <the full prompt on one line>
|
||||
negative: <negative prompt on one line, or leave empty after the colon>
|
||||
```
|
||||
|
||||
3. Optionally one line suggesting a fitting aspect ratio (e.g. "Suggested aspect: 9:16 (portrait)").
|
||||
|
||||
Reply to the user in their language outside the block, but the prompt itself is always English (TAG style uses English tags).
|
||||
|
||||
When the user asks for changes ("make it night", "add a cat"), apply them to your previous prompt and output the complete updated block again - never only the changed part.
|
||||
|
||||
## Example
|
||||
User: 雨の夜の東京で傘をさす猫
|
||||
Context: model qwen_image_2512_fp8.safetensors, CFG 1.0
|
||||
|
||||
Style: NATURAL - Qwen-Image workflow, CFG 1 so no negative prompt.
|
||||
|
||||
```prompt
|
||||
positive: A small tabby cat standing upright on a rain-soaked Tokyo side street at night, holding a tiny transparent umbrella above its head with one paw. Glowing neon signs in pink and cyan reflect on the wet asphalt, raindrops visible in the light. Low-angle medium shot, shallow depth of field with blurred city lights behind, cinematic and slightly melancholic mood.
|
||||
negative:
|
||||
```
|
||||
|
||||
Suggested aspect: 2:3 (portrait)
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: Workflow Doctor
|
||||
description: Explains the workflow loaded in GenerateUI and diagnoses errors from checked facts (missing nodes and files, last execution error, ComfyUI log).
|
||||
metadata:
|
||||
wfs-version: 1
|
||||
tools: []
|
||||
temperature: 0.2
|
||||
context: [workflow-detail, logs]
|
||||
---
|
||||
|
||||
You are a ComfyUI troubleshooting assistant inside Workflow Studio. Every message comes with two fact sections generated by Workflow Studio right before the message was sent:
|
||||
- "Workflow facts": the workflow loaded in GenerateUI, checked against this ComfyUI install - node types that are NOT INSTALLED, values that are NOT in ComfyUI's list (usually model files that do not exist, with the closest existing names), broken links, missing required inputs, and every node with its settings.
|
||||
- "ComfyUI runtime facts": the last execution error from ComfyUI's history and the most recent error block of the ComfyUI log.
|
||||
|
||||
## Ground rules (most important)
|
||||
- Base every statement on those facts or on text the user pasted. Quote the exact node (#id and class name), file name, or error line you rely on.
|
||||
- Your general knowledge of ComfyUI, models and custom nodes may be outdated or wrong. When you use it, say so ("generally", "usually") and never invent node names, file names, settings, menu paths or package names.
|
||||
- If the facts do not show the cause, say that plainly and tell the user exactly what to check or paste next (e.g. "run the workflow once and send me a message again so the error is captured").
|
||||
- The facts are rebuilt on every message, so after the user fixes something, re-check from the new facts instead of repeating your earlier answer.
|
||||
|
||||
## When the user asks why something fails
|
||||
1. Cause - one or two sentences, quoting the decisive fact.
|
||||
2. Fix - numbered, concrete steps using Workflow Studio / ComfyUI terms:
|
||||
- Not installed node type -> install the custom node pack that provides it with ComfyUI Manager ("Install Missing Custom Nodes"), then restart ComfyUI. Only name the pack if the error text or node name makes it certain.
|
||||
- Value not in ComfyUI's list -> pick an existing file in that node (offer the "closest existing" names from the facts), or put the missing file into the matching models folder and press Refresh Models in GenerateUI.
|
||||
- Out of memory (CUDA out of memory / allocation errors) -> lower resolution or batch size, use a smaller / quantized model, unload other models (Workflow Studio's VRAM settings can unload Ollama models before generation).
|
||||
- Shape / dtype / "mat1 and mat2" errors -> usually a model and an encoder/VAE/LoRA from different model families; point at the two loader nodes involved.
|
||||
3. Check - how the user can confirm it is fixed.
|
||||
|
||||
## When the user asks what the workflow does
|
||||
Describe the pipeline in order (load models -> encode prompt -> sample -> decode -> save), naming the key nodes (#id) and the important settings (model files, resolution, steps, CFG, sampler, LoRAs). Point out anything in "Detected problems" even if the user did not ask.
|
||||
|
||||
## Style
|
||||
- Reply in the user's language. Use short headings and numbered steps; keep it compact.
|
||||
- Never output a full workflow JSON. If a node value must change, name the node (#id), the input and the new value.
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Audio conversion (and audio extraction from video) using PyAV.
|
||||
|
||||
Gallery の「Convert」機能のコア。PyAV(av)だけで完結し、システムの ffmpeg には依存しない。
|
||||
"""
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# 出力フォーマット -> (拡張子, エンコーダ, 標準サンプル形式, ビットレート指定可否, タグ保持可否)
|
||||
# .ogg は PyAV 同梱ビルドに Vorbis エンコーダが無いため出力対象外（入力は読める）。
|
||||
# wav は任意キー(prompt/workflow)のタグを持てないため、メタデータ引き継ぎの対象外。
|
||||
OUTPUT_FORMATS = {
|
||||
"flac": {"ext": ".flac", "codec": "flac", "sample_fmt": "s16", "bitrate": False, "tags": True},
|
||||
"mp3": {"ext": ".mp3", "codec": "libmp3lame", "sample_fmt": "fltp", "bitrate": True, "tags": True},
|
||||
"opus": {"ext": ".opus", "codec": "libopus", "sample_fmt": "flt", "bitrate": True, "tags": True},
|
||||
"m4a": {"ext": ".m4a", "codec": "aac", "sample_fmt": "fltp", "bitrate": True, "tags": True},
|
||||
"wav": {"ext": ".wav", "codec": "pcm_s16le", "sample_fmt": "s16", "bitrate": False, "tags": False},
|
||||
}
|
||||
|
||||
# 引き継ぐタグ(小文字で比較)。ComfyUIの prompt/workflow と一般的な曲情報のみ。
|
||||
_CARRY_TAGS = {"prompt", "workflow", "title", "artist", "album", "date", "genre", "comment"}
|
||||
|
||||
_OPUS_RATES = (8000, 12000, 16000, 24000, 48000)
|
||||
|
||||
|
||||
def supports_tags(fmt: str) -> bool:
|
||||
return bool(OUTPUT_FORMATS.get(fmt, {}).get("tags"))
|
||||
|
||||
|
||||
def _channel_layout(opt: str, src_channels: int) -> str:
|
||||
if opt == "mono":
|
||||
return "mono"
|
||||
if opt == "stereo":
|
||||
return "stereo"
|
||||
return "mono" if src_channels == 1 else "stereo"
|
||||
|
||||
|
||||
def convert_audio(src: Path, dst: Path, fmt: str, *, bitrate_kbps: int = 0,
|
||||
sample_rate: int = 0, channels: str = "keep",
|
||||
inherit_metadata: bool = True) -> dict:
|
||||
"""src(音声または音声トラックを持つ動画)から dst へ変換する。
|
||||
|
||||
sample_rate=0 / channels="keep" は元の設定を維持。opusは48kHz固定扱い。
|
||||
戻り値: {"tags_written": bool}（タグを持てない形式や元にタグが無い場合は False）
|
||||
"""
|
||||
import av
|
||||
|
||||
spec = OUTPUT_FORMATS[fmt]
|
||||
tags_written = False
|
||||
with av.open(str(src)) as in_c:
|
||||
if not in_c.streams.audio:
|
||||
raise ValueError("No audio stream in the source file")
|
||||
in_stream = in_c.streams.audio[0]
|
||||
|
||||
rate = sample_rate or in_stream.rate or 44100
|
||||
if fmt == "opus":
|
||||
rate = min(_OPUS_RATES, key=lambda r: abs(r - rate)) if sample_rate else 48000
|
||||
elif fmt == "mp3" and rate not in (8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000):
|
||||
rate = 44100
|
||||
layout = _channel_layout(channels, in_stream.channels)
|
||||
|
||||
options = {"movflags": "use_metadata_tags"} if fmt == "m4a" else {}
|
||||
with av.open(str(dst), "w", options=options) as out_c:
|
||||
# タグ: 引き継ぎ対象かつ形式がタグを持てる場合のみ
|
||||
if inherit_metadata and spec["tags"]:
|
||||
src_tags = dict(in_c.metadata or {})
|
||||
for k, v in (in_stream.metadata or {}).items(): # Opus等はストリーム側にタグを持つ
|
||||
src_tags.setdefault(k, v)
|
||||
for k, v in src_tags.items():
|
||||
if k.lower() in _CARRY_TAGS and isinstance(v, str) and v:
|
||||
out_c.metadata[k.lower()] = v
|
||||
tags_written = True
|
||||
|
||||
out_stream = out_c.add_stream(spec["codec"], rate=rate)
|
||||
out_stream.layout = layout
|
||||
if spec["bitrate"] and bitrate_kbps:
|
||||
out_stream.bit_rate = int(bitrate_kbps) * 1000
|
||||
|
||||
resampler = av.AudioResampler(format=out_stream.format.name, layout=layout, rate=rate)
|
||||
|
||||
def _encode(frames):
|
||||
for f in frames:
|
||||
for pkt in out_stream.encode(f):
|
||||
out_c.mux(pkt)
|
||||
|
||||
for frame in in_c.decode(in_stream):
|
||||
_encode(resampler.resample(frame))
|
||||
_encode(resampler.resample(None))
|
||||
for pkt in out_stream.encode(None):
|
||||
out_c.mux(pkt)
|
||||
return {"tags_written": tags_written}
|
||||
+480
-39
@@ -11,14 +11,52 @@ import re
|
||||
import struct
|
||||
import threading
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import unquote
|
||||
|
||||
from ..config import is_safe_rel, resolve_untrusted
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# サポートする画像拡張子（.mp4は動画だがGalleryでは静止画と同じ一覧・配信経路を共有する。
|
||||
# 動画拡張子（PyAV(av)がコンテナ形式を問わず共通で扱えるため、寸法/再生時間/埋め込み
|
||||
# メタデータ読み取り・サムネイル抽出は両形式で同じコードパスを通る）
|
||||
VIDEO_EXTENSIONS = {".mp4", ".webm"}
|
||||
|
||||
# 音声拡張子（ComfyUIの音楽生成出力）。動画と同様にPyAVで再生時間・埋め込みメタデータを読む。
|
||||
# サムネイルは波形画像を生成する。
|
||||
AUDIO_EXTENSIONS = {".wav", ".mp3", ".ogg", ".m4a", ".flac", ".opus"}
|
||||
# 波形サムネイルの版数（描画仕様を変えたら上げる）。static/js/gallery-tab.js の WAVE_THUMB_VERSION と同じ値にすること。
|
||||
WAVE_THUMB_VERSION = "wave2"
|
||||
_NB_INDEX_CACHE_MAX = 16
|
||||
|
||||
# 音声サムネイル（波形）の色。拡張子ごとに変えて、一覧で形式を見分けやすくする。
|
||||
AUDIO_WAVE_COLORS = {
|
||||
".mp3": (56, 189, 248), # blue
|
||||
".wav": (74, 222, 128), # green
|
||||
".flac": (251, 191, 36), # amber
|
||||
".ogg": (192, 132, 252), # purple
|
||||
".opus": (192, 132, 252), # purple（Ogg系は同色）
|
||||
".m4a": (251, 113, 133), # rose
|
||||
}
|
||||
|
||||
# 3Dモデル拡張子。表示はブラウザのthree.js(static/js/gallery-3d-viewer.js)で行う。
|
||||
# サーバーに3Dレンダラーが無いため、サムネイルはブラウザが描画してアップロードしたものを使う。
|
||||
GLTF_EXTENSIONS = {".glb", ".gltf", ".vrm"} # .vrmの実体はGLB
|
||||
MODEL3D_EXTENSIONS = GLTF_EXTENSIONS | {".ply", ".spz"}
|
||||
# .gltfが相対URIで参照する外部ファイルとして配信を許可する拡張子
|
||||
MODEL3D_ASSET_EXTENSIONS = {".bin", ".png", ".jpg", ".jpeg", ".webp"}
|
||||
# glTF JSON部の読み取り上限。base64埋め込みの巨大な.gltfはメタデータ読み取りを諦める
|
||||
_GLTF_JSON_MAX = 16 * 1024 * 1024
|
||||
_MODEL3D_THUMB_MAX_BYTES = 8 * 1024 * 1024
|
||||
_MODEL3D_THUMB_MAX_SIDE = 2048
|
||||
_MODEL3D_THUMB_SIZE = 512
|
||||
|
||||
# サポートする画像拡張子（動画・音声・3DはGalleryでは静止画と同じ一覧・配信経路を共有する。
|
||||
# .psdはブラウザが直接レンダリングできないため、配信時に合成済みPNGへ変換する）
|
||||
IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".gif", ".svg", ".mp4", ".psd"}
|
||||
IMAGE_EXTENSIONS = ({".png", ".jpg", ".jpeg", ".webp", ".gif", ".svg", ".psd"}
|
||||
| VIDEO_EXTENSIONS | AUDIO_EXTENSIONS | MODEL3D_EXTENSIONS)
|
||||
|
||||
# Canvas2D の globalCompositeOperation 文字列 -> psd_tools.constants.BlendMode 名
|
||||
# (Image Edit タブの Layer.blendMode はCanvas2Dの合成モード文字列をそのまま保持している)
|
||||
@@ -54,6 +92,10 @@ _SAVE_EXT_BY_MIME = {
|
||||
"image/svg+xml": ".svg",
|
||||
}
|
||||
|
||||
# Comic Creator (eagle_comic_creator_spa) の nanobananaタブが生成履歴を記録するフォルダ名。
|
||||
# 画像と同じフォルダに history.jsonl (1行1リクエスト、filenames配列で画像と対応) が書かれる。
|
||||
NANOBANANA_FOLDER_NAME = "cc_nanobanana"
|
||||
|
||||
|
||||
def _decode_image_data_url(image_data: str) -> tuple[bytes, str]:
|
||||
"""data URL (data:<mime>;base64,<data>) をデコードし (バイト列, 拡張子) を返す。
|
||||
@@ -175,6 +217,10 @@ class _FolderCache:
|
||||
|
||||
|
||||
class GalleryService:
|
||||
_nb_index_cache: "OrderedDict[str, tuple]" = OrderedDict() # history.jsonlの索引キャッシュ(上限付き)
|
||||
_nb_index_lock = threading.Lock()
|
||||
_convert_lock = threading.Lock() # 変換は同時に1本だけ実行する
|
||||
|
||||
def __init__(self, data_dir: Path):
|
||||
self.data_dir = data_dir
|
||||
self.metadata_store = GalleryMetadataStore(data_dir / "gallery_metadata.json")
|
||||
@@ -226,7 +272,7 @@ class GalleryService:
|
||||
"""バックグラウンドスレッドで未キャッシュ画像のprompt_cacheを構築する。
|
||||
10枚処理するたびに50msスリープしてメインスレッドへの影響を最小化。"""
|
||||
try:
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir() or not self._check_path_allowed(folder):
|
||||
return
|
||||
entries = self._scan_folder(folder)
|
||||
@@ -289,13 +335,19 @@ class GalleryService:
|
||||
pass
|
||||
return False
|
||||
|
||||
def _resolve_untrusted(self, value) -> Path:
|
||||
"""クライアント由来のパスを resolve() の前に文字列のまま検査して返す(拒否時 ValueError)。
|
||||
Windows では UNC パスは resolve() した時点で相手ホストへ資格情報付きで接続してしまうため、
|
||||
UNC は許可ルート自身が UNC の場合のみ、その配下を許す。"""
|
||||
return resolve_untrusted(value, (self._allowed_root, self._comfy_output_root))
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# フォルダツリー
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
def list_folder_tree(self, root_path: str) -> dict:
|
||||
"""outputフォルダのフォルダツリーを返す"""
|
||||
root = Path(root_path).resolve()
|
||||
root = self._resolve_untrusted(root_path)
|
||||
if not root.is_dir():
|
||||
return {"error": f"Directory not found: {root_path}"}
|
||||
if self._allowed_root is None:
|
||||
@@ -414,6 +466,21 @@ class GalleryService:
|
||||
_walk(folder)
|
||||
return entries
|
||||
|
||||
def make_list_item(self, name: str, abs_path: str, size: int, mtime: float) -> dict:
|
||||
"""一覧アイテムdict(list_imagesと同形式)を作る。セマンティック検索の結果表示用。"""
|
||||
meta = self.metadata_store.get(abs_path)
|
||||
return {
|
||||
"filename": name,
|
||||
"path": abs_path,
|
||||
"size": size,
|
||||
"mtime": mtime,
|
||||
"ext": Path(name).suffix.lower(),
|
||||
"favorite": meta.get("favorite", False),
|
||||
"tags": meta.get("tags", []),
|
||||
"memo": meta.get("memo", ""),
|
||||
"groups": meta.get("groups", []),
|
||||
}
|
||||
|
||||
def list_images(
|
||||
self,
|
||||
folder_path: str,
|
||||
@@ -425,7 +492,7 @@ class GalleryService:
|
||||
recursive: bool = False,
|
||||
) -> list[dict]:
|
||||
"""指定フォルダ内の画像一覧を返す（recursive=Trueならサブフォルダも含める）"""
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir():
|
||||
return []
|
||||
if not self._check_path_allowed(folder):
|
||||
@@ -533,7 +600,7 @@ class GalleryService:
|
||||
def save_image_to_folder(self, folder_path: str, filename: str, image_data: str) -> dict:
|
||||
"""任意フォルダへ画像を保存する（data URL想定、同名なら上書き）。
|
||||
Style Catalogのカタログ作成（スタイル名でのファイル保存）で使用。"""
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir():
|
||||
return {"ok": False, "error": "Destination folder not found"}
|
||||
if not self._check_path_allowed(folder):
|
||||
@@ -571,6 +638,58 @@ class GalleryService:
|
||||
pass
|
||||
return self._read_vault_prompt(image_path)
|
||||
|
||||
def _read_nanobanana_entry(self, image_path: Path) -> dict | None:
|
||||
"""画像がcc_nanobananaフォルダ内にあり、同フォルダのhistory.jsonlに対応エントリが
|
||||
あれば返す(1行=1生成リクエストで複数画像filenamesを束ねているため、自分の
|
||||
ファイル名を含む行を新しい方から探す。同名再生成があれば最新を優先する)。"""
|
||||
if image_path.parent.name != NANOBANANA_FOLDER_NAME:
|
||||
return None
|
||||
history_path = image_path.parent / "history.jsonl"
|
||||
index = self._load_nanobanana_index(history_path)
|
||||
return index.get(image_path.name) if index else None
|
||||
|
||||
def _load_nanobanana_index(self, history_path: Path) -> dict | None:
|
||||
"""history.jsonl を「ファイル名→エントリ」の辞書にして返す。(mtime, size)をキーに
|
||||
キャッシュし、メタデータ要求のたびの全読み込みを避ける。同名再生成は新しい行を優先する。"""
|
||||
try:
|
||||
st = history_path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
key = str(history_path)
|
||||
sig = (st.st_mtime_ns, st.st_size)
|
||||
with self._nb_index_lock:
|
||||
cached = self._nb_index_cache.get(key)
|
||||
if cached and cached[0] == sig:
|
||||
self._nb_index_cache.move_to_end(key)
|
||||
return cached[1]
|
||||
try:
|
||||
lines = history_path.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||
except OSError:
|
||||
return None
|
||||
index: dict = {}
|
||||
for line in reversed(lines): # 新しい行から。最初に見つかったものを優先
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
entry = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
if not isinstance(entry, dict): # 破損行（[]・数値など）は無視
|
||||
continue
|
||||
filenames = entry.get("filenames")
|
||||
if not isinstance(filenames, list):
|
||||
continue
|
||||
for name in filenames:
|
||||
if isinstance(name, str) and name not in index:
|
||||
index[name] = entry
|
||||
with self._nb_index_lock:
|
||||
self._nb_index_cache[key] = (sig, index)
|
||||
self._nb_index_cache.move_to_end(key)
|
||||
while len(self._nb_index_cache) > _NB_INDEX_CACHE_MAX:
|
||||
self._nb_index_cache.popitem(last=False)
|
||||
return index
|
||||
|
||||
def _find_vault_category_root(self, image_path: Path) -> Path | None:
|
||||
"""image_path の祖先を辿り、直下に thumbnails(_option2) を持つフォルダ
|
||||
(=ponyxlWildcardsVaultのカテゴリルート) を探す。
|
||||
@@ -629,7 +748,7 @@ class GalleryService:
|
||||
|
||||
def save_image_prompt(self, image_path: str, text: str) -> bool:
|
||||
"""画像と同名の.txtサイドカーにプロンプトテキストを保存する"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return False
|
||||
if not self._check_path_allowed(p):
|
||||
@@ -647,7 +766,7 @@ class GalleryService:
|
||||
|
||||
def get_image_metadata(self, image_path: str) -> dict:
|
||||
"""PNG/JPEGからメタデータを抽出し、保存済みメタと合わせて返す"""
|
||||
path = Path(image_path).resolve()
|
||||
path = self._resolve_untrusted(image_path)
|
||||
if not path.is_file():
|
||||
return {"error": "File not found"}
|
||||
if not self._check_path_allowed(path):
|
||||
@@ -659,8 +778,14 @@ class GalleryService:
|
||||
embedded = self._read_png_metadata(path)
|
||||
elif ext in {".jpg", ".jpeg"}:
|
||||
embedded = self._read_jpeg_metadata(path)
|
||||
elif ext == ".mp4":
|
||||
elif ext == ".webp":
|
||||
embedded = self._read_webp_metadata(path)
|
||||
elif ext in VIDEO_EXTENSIONS:
|
||||
embedded = self._read_mp4_metadata(path)
|
||||
elif ext in AUDIO_EXTENSIONS:
|
||||
embedded = self._read_audio_metadata(path)
|
||||
elif ext in GLTF_EXTENSIONS:
|
||||
embedded = self._read_gltf_metadata(path)
|
||||
|
||||
dims = self._read_media_dimensions(path, ext)
|
||||
saved = self.metadata_store.get(str(path))
|
||||
@@ -689,6 +814,7 @@ class GalleryService:
|
||||
"memo": saved.get("memo", ""),
|
||||
"groups": saved.get("groups", []),
|
||||
"image_prompt": self._read_sidecar_prompt(path),
|
||||
"nanobanana": self._read_nanobanana_entry(path),
|
||||
}
|
||||
|
||||
def _read_png_metadata(self, path: Path) -> dict:
|
||||
@@ -707,8 +833,6 @@ class GalleryService:
|
||||
chunk_type = f.read(4).decode("ascii", errors="ignore")
|
||||
if chunk_len > _PNG_CHUNK_MAX:
|
||||
f.seek(chunk_len + 4, 1)
|
||||
if chunk_type == "IEND":
|
||||
break
|
||||
continue
|
||||
chunk_data = f.read(chunk_len)
|
||||
f.read(4) # CRC
|
||||
@@ -728,16 +852,55 @@ class GalleryService:
|
||||
rest = rest[null_idx2 + 1:]
|
||||
null_idx3 = rest.index(b"\x00")
|
||||
value = rest[null_idx3 + 1:].decode("utf-8", errors="replace")
|
||||
result[key] = value
|
||||
# 先勝ち: png_extractor.extract_png_workflow(ワークフロー読み込み)と
|
||||
# 同じチャンクを採用し、Gallery表示との食い違いを防ぐ。
|
||||
if key not in result:
|
||||
result[key] = value
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
pass
|
||||
|
||||
if chunk_type == "IEND":
|
||||
break
|
||||
# IENDで止めない: ComfyUI-Custom-Scripts等はIENDの後ろに
|
||||
# workflowチャンクを追記するため、ファイル末尾まで読む。
|
||||
except Exception as e:
|
||||
logger.debug("PNG metadata read error: %s", e)
|
||||
return result
|
||||
|
||||
def _read_webp_metadata(self, path: Path) -> dict:
|
||||
"""WebPのRIFF EXIFチャンクから埋め込みworkflow/prompt(JSON文字列)を抽出する。
|
||||
ComfyUIはWebPのEXIFに "workflow:{...}" / "prompt:{...}" 形式で書き込む
|
||||
（static/js/metadata-tab.js の extractWorkflowFromEXIF と同じ処理）。"""
|
||||
result = {}
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
head = f.read(12)
|
||||
if len(head) < 12 or head[:4] != b"RIFF" or head[8:12] != b"WEBP":
|
||||
return result
|
||||
while True:
|
||||
hdr = f.read(8)
|
||||
if len(hdr) < 8:
|
||||
break
|
||||
fourcc = hdr[:4]
|
||||
size = struct.unpack("<I", hdr[4:])[0]
|
||||
if fourcc == b"EXIF":
|
||||
if size > _PNG_CHUNK_MAX:
|
||||
break
|
||||
text = f.read(size).decode("utf-8", errors="replace")
|
||||
decoder = json.JSONDecoder()
|
||||
for key in ("workflow", "prompt"):
|
||||
idx = text.find(key + ":{")
|
||||
if idx < 0:
|
||||
continue
|
||||
start = idx + len(key) + 1
|
||||
try:
|
||||
_, end = decoder.raw_decode(text, start)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
result[key] = text[start:end]
|
||||
break
|
||||
f.seek(size + (size & 1), 1)
|
||||
except Exception as e:
|
||||
logger.debug("WebP metadata read error: %s", e)
|
||||
return result
|
||||
|
||||
def _read_jpeg_metadata(self, path: Path) -> dict:
|
||||
"""JPEGのEXIF/commentからメタデータを抽出（簡易）"""
|
||||
result = {}
|
||||
@@ -777,11 +940,61 @@ class GalleryService:
|
||||
logger.debug("MP4 metadata read error: %s", e)
|
||||
return result
|
||||
|
||||
def _read_audio_metadata(self, path: Path) -> dict:
|
||||
"""音声ファイルのタグ（ComfyUIのSaveAudio系は prompt/workflow をJSON文字列で書き込む）をPyAVで抽出する。
|
||||
FLACのVorbisコメント等はキーの大文字小文字が実装依存のため、prompt/workflowは小文字キーでも引けるようにする。"""
|
||||
result = {}
|
||||
try:
|
||||
import av
|
||||
with av.open(str(path)) as container:
|
||||
tags = dict(container.metadata or {})
|
||||
for stream in container.streams.audio:
|
||||
for k, v in (stream.metadata or {}).items():
|
||||
tags.setdefault(k, v)
|
||||
for k, v in tags.items():
|
||||
result[k] = v
|
||||
if k.lower() in ("workflow", "prompt"):
|
||||
result.setdefault(k.lower(), v)
|
||||
except Exception as e:
|
||||
logger.debug("Audio metadata read error: %s", e)
|
||||
return result
|
||||
|
||||
def _read_gltf_metadata(self, path: Path) -> dict:
|
||||
"""glTF/GLB/VRMのasset.extrasを読む。ComfyUIのSaveGLBはここにprompt/workflowを
|
||||
JSON文字列で書き込む。GLBは先頭のJSONチャンクだけ読み、メッシュ本体は読まない。"""
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
if path.suffix.lower() == ".gltf":
|
||||
if path.stat().st_size > _GLTF_JSON_MAX:
|
||||
return {}
|
||||
doc = json.loads(f.read())
|
||||
else:
|
||||
head = f.read(20)
|
||||
if len(head) < 20 or head[:4] != b"glTF":
|
||||
return {}
|
||||
length, chunk_type = struct.unpack("<II", head[12:20])
|
||||
if chunk_type != 0x4E4F534A or length > _GLTF_JSON_MAX: # "JSON"
|
||||
return {}
|
||||
doc = json.loads(f.read(length))
|
||||
extras = doc.get("asset", {}).get("extras")
|
||||
except (OSError, ValueError, AttributeError) as e:
|
||||
logger.debug("glTF metadata read error: %s", e)
|
||||
return {}
|
||||
if not isinstance(extras, dict):
|
||||
return {}
|
||||
result = {}
|
||||
for k, v in extras.items():
|
||||
if isinstance(v, str):
|
||||
result[k] = v
|
||||
elif k in ("prompt", "workflow") and isinstance(v, dict):
|
||||
result[k] = json.dumps(v)
|
||||
return result
|
||||
|
||||
def _read_media_dimensions(self, path: Path, ext: str) -> dict:
|
||||
"""画像は幅高さ、動画は幅高さ+再生時間（秒）を返す。取得できない項目はNone。"""
|
||||
info: dict = {"width": None, "height": None, "duration": None}
|
||||
try:
|
||||
if ext == ".mp4":
|
||||
if ext in VIDEO_EXTENSIONS:
|
||||
import av
|
||||
with av.open(str(path)) as container:
|
||||
if container.streams.video:
|
||||
@@ -790,11 +1003,16 @@ class GalleryService:
|
||||
info["height"] = stream.height
|
||||
if container.duration:
|
||||
info["duration"] = container.duration / 1_000_000
|
||||
elif ext in AUDIO_EXTENSIONS:
|
||||
import av
|
||||
with av.open(str(path)) as container:
|
||||
if container.duration:
|
||||
info["duration"] = container.duration / 1_000_000
|
||||
elif ext == ".psd":
|
||||
from psd_tools import PSDImage
|
||||
psd = PSDImage.open(path)
|
||||
info["width"], info["height"] = psd.width, psd.height
|
||||
elif ext != ".svg":
|
||||
elif ext != ".svg" and ext not in MODEL3D_EXTENSIONS:
|
||||
from PIL import Image
|
||||
with Image.open(path) as img:
|
||||
info["width"], info["height"] = img.size
|
||||
@@ -812,7 +1030,7 @@ class GalleryService:
|
||||
extractPromptsLiteGraphがトップレベル系統しか拾えないことがあるため、
|
||||
プロンプト抽出専用にAPI形式を優先する（Metadataタブのプロンプト抽出と同じ優先順位）。
|
||||
"""
|
||||
path = Path(image_path).resolve()
|
||||
path = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(path):
|
||||
return None
|
||||
|
||||
@@ -820,8 +1038,14 @@ class GalleryService:
|
||||
embedded = None
|
||||
if ext == ".png":
|
||||
embedded = self._read_png_metadata(path)
|
||||
elif ext == ".mp4":
|
||||
elif ext == ".webp":
|
||||
embedded = self._read_webp_metadata(path)
|
||||
elif ext in VIDEO_EXTENSIONS:
|
||||
embedded = self._read_mp4_metadata(path)
|
||||
elif ext in AUDIO_EXTENSIONS:
|
||||
embedded = self._read_audio_metadata(path)
|
||||
elif ext in GLTF_EXTENSIONS:
|
||||
embedded = self._read_gltf_metadata(path)
|
||||
|
||||
if embedded:
|
||||
key_groups = (("workflow", "Workflow"), ("prompt", "Prompt"))
|
||||
@@ -854,8 +1078,89 @@ class GalleryService:
|
||||
# メタデータ保存
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# ファイル変換（音声変換 / 動画からの音声抽出）
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
def convert_file(self, image_path: str, opts: dict) -> dict:
|
||||
"""音声ファイル（または動画の音声トラック）を指定形式へ変換し、同じフォルダに別ファイルとして保存する。
|
||||
|
||||
opts: format(flac/mp3/opus/m4a/wav), bitrate(kbps), sample_rate(0=元のまま), channels(keep/mono/stereo),
|
||||
inherit_metadata(prompt/workflow等のタグ引き継ぎ。タグを持てない形式は対象外),
|
||||
copy_gallery_meta(お気に入り/タグ/メモ/グループの引き継ぎ), delete_original(変換成功後に元を削除)
|
||||
"""
|
||||
from .convert_service import OUTPUT_FORMATS, convert_audio, supports_tags
|
||||
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return {"ok": False, "error": "File not found"}
|
||||
if p.suffix.lower() not in AUDIO_EXTENSIONS | VIDEO_EXTENSIONS:
|
||||
return {"ok": False, "error": "Not an audio/video file"}
|
||||
if not self._check_path_allowed(p):
|
||||
return {"ok": False, "error": "Access denied"}
|
||||
fmt = str(opts.get("format", ""))
|
||||
if fmt not in OUTPUT_FORMATS:
|
||||
return {"ok": False, "error": f"Unsupported format: {fmt}"}
|
||||
|
||||
def _int(key: str, default: int = 0) -> int:
|
||||
try:
|
||||
return max(0, int(opts.get(key) or default))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
ext = OUTPUT_FORMATS[fmt]["ext"]
|
||||
inherit = bool(opts.get("inherit_metadata", True))
|
||||
|
||||
# 出力名の決定〜変換完了までを同一ロック内で行う（同時リクエストで同じ名前を選んで
|
||||
# 上書きし合うのを防ぐ）。エラー時に消すのは自分が決めたdstだけ。
|
||||
with self._convert_lock:
|
||||
dst = p.with_suffix(ext)
|
||||
if dst.exists(): # 同名（同形式への再変換を含む）は上書きせず連番
|
||||
counter = 1
|
||||
while True:
|
||||
dst = p.with_name(f"{p.stem}_converted{'' if counter == 1 else counter}{ext}")
|
||||
if not dst.exists():
|
||||
break
|
||||
counter += 1
|
||||
try:
|
||||
result = convert_audio(
|
||||
p, dst, fmt,
|
||||
bitrate_kbps=_int("bitrate"),
|
||||
sample_rate=_int("sample_rate"),
|
||||
channels=str(opts.get("channels", "keep")),
|
||||
inherit_metadata=inherit,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("convert_file: failed for %s: %s", p, e)
|
||||
try:
|
||||
dst.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
return {"ok": False, "error": str(e)}
|
||||
|
||||
dst_str = str(dst.resolve()).replace("\\", "/")
|
||||
if opts.get("copy_gallery_meta", True):
|
||||
saved = self.metadata_store.get(str(p))
|
||||
carry = {k: saved[k] for k in ("favorite", "tags", "memo", "groups") if saved.get(k)}
|
||||
if carry:
|
||||
self.metadata_store.save(str(dst.resolve()), carry)
|
||||
self._folder_cache.invalidate(p.parent)
|
||||
|
||||
deleted = False
|
||||
if opts.get("delete_original"):
|
||||
deleted = bool(self.delete_images([str(p)])["deleted"])
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"path": dst_str,
|
||||
"filename": dst.name,
|
||||
"tags_written": result["tags_written"],
|
||||
"tags_skipped": inherit and not supports_tags(fmt),
|
||||
"deleted": deleted,
|
||||
}
|
||||
|
||||
def save_image_meta(self, image_path: str, data: dict) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
logger.warning("save_image_meta: path not allowed: %s", resolved)
|
||||
return False
|
||||
@@ -865,7 +1170,7 @@ class GalleryService:
|
||||
return self.metadata_store.save(str(resolved), data)
|
||||
|
||||
def toggle_favorite(self, image_path: str) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
return False
|
||||
# get_image_metadata()/list_images() 側はresolve()済みパスをキーにして取得する
|
||||
@@ -881,7 +1186,11 @@ class GalleryService:
|
||||
"""複数画像のお気に入りを一括設定する"""
|
||||
ok = fail = 0
|
||||
for p in paths:
|
||||
resolved = Path(p).resolve()
|
||||
try:
|
||||
resolved = self._resolve_untrusted(p)
|
||||
except ValueError:
|
||||
fail += 1
|
||||
continue
|
||||
if not self._check_path_allowed(resolved):
|
||||
fail += 1
|
||||
continue
|
||||
@@ -919,7 +1228,7 @@ class GalleryService:
|
||||
return self.metadata_store.delete_group(name)
|
||||
|
||||
def add_to_group(self, image_path: str, group_name: str) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
return False
|
||||
# get_image_metadata()/list_images() 側はresolve()済みパスをキーにして取得する
|
||||
@@ -934,7 +1243,7 @@ class GalleryService:
|
||||
return True
|
||||
|
||||
def remove_from_group(self, image_path: str, group_name: str) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
return False
|
||||
resolved_str = str(resolved)
|
||||
@@ -963,7 +1272,7 @@ class GalleryService:
|
||||
|
||||
def create_folder(self, parent_path: str, name: str) -> dict:
|
||||
"""選択フォルダ内に新しいサブフォルダを作成する"""
|
||||
parent = Path(parent_path).resolve()
|
||||
parent = self._resolve_untrusted(parent_path)
|
||||
if not parent.is_dir():
|
||||
return {"ok": False, "error": "Parent directory not found"}
|
||||
if not self._check_path_allowed(parent):
|
||||
@@ -985,7 +1294,7 @@ class GalleryService:
|
||||
def delete_folder(self, folder_path: str) -> dict:
|
||||
"""フォルダを再帰的に削除する"""
|
||||
import shutil
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir():
|
||||
return {"ok": False, "error": "Directory not found"}
|
||||
if not self._check_path_allowed(folder):
|
||||
@@ -1005,7 +1314,11 @@ class GalleryService:
|
||||
deleted = []
|
||||
errors = []
|
||||
for img_path in paths:
|
||||
p = Path(img_path).resolve()
|
||||
try:
|
||||
p = self._resolve_untrusted(img_path)
|
||||
except ValueError:
|
||||
errors.append(f"{Path(img_path).name}: access denied")
|
||||
continue
|
||||
if not p.is_file():
|
||||
errors.append(f"{Path(img_path).name}: not found")
|
||||
continue
|
||||
@@ -1026,7 +1339,7 @@ class GalleryService:
|
||||
|
||||
def move_images(self, paths: list, dest_folder: str) -> dict:
|
||||
"""画像ファイルを別フォルダへ移動する（複数対応）"""
|
||||
dest = Path(dest_folder).resolve()
|
||||
dest = self._resolve_untrusted(dest_folder)
|
||||
if not dest.is_dir():
|
||||
return {"ok": False, "error": "Destination folder not found", "moved": [], "errors": []}
|
||||
if not self._check_path_allowed(dest):
|
||||
@@ -1034,7 +1347,11 @@ class GalleryService:
|
||||
moved = []
|
||||
errors = []
|
||||
for img_path in paths:
|
||||
p = Path(img_path).resolve()
|
||||
try:
|
||||
p = self._resolve_untrusted(img_path)
|
||||
except ValueError:
|
||||
errors.append(f"{Path(img_path).name}: access denied")
|
||||
continue
|
||||
if not p.is_file():
|
||||
errors.append(f"{Path(img_path).name}: not found")
|
||||
continue
|
||||
@@ -1078,11 +1395,14 @@ class GalleryService:
|
||||
|
||||
layers_data = []
|
||||
for img_path in paths:
|
||||
p = Path(img_path).resolve()
|
||||
try:
|
||||
p = self._resolve_untrusted(img_path)
|
||||
except ValueError:
|
||||
continue
|
||||
if not p.is_file() or p.suffix.lower() not in IMAGE_EXTENSIONS:
|
||||
continue
|
||||
if p.suffix.lower() == ".mp4":
|
||||
continue # 動画はPSDレイヤーにできない
|
||||
if p.suffix.lower() in VIDEO_EXTENSIONS | AUDIO_EXTENSIONS | MODEL3D_EXTENSIONS:
|
||||
continue # 動画・音声・3DはPSDレイヤーにできない
|
||||
if not self._check_path_allowed(p):
|
||||
continue
|
||||
try:
|
||||
@@ -1246,7 +1566,7 @@ class GalleryService:
|
||||
"""サーバー上に既にあるPSDファイル(Gallery上のパス)をレイヤー配列に分解する。
|
||||
Galleryから直接Image Editへ送る際、アップロードの往復無しで済ませるために使う。
|
||||
"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file() or p.suffix.lower() != ".psd":
|
||||
raise ValueError("Not a PSD file")
|
||||
if not self._check_path_allowed(p):
|
||||
@@ -1270,7 +1590,7 @@ class GalleryService:
|
||||
def serve_image(self, image_path: str):
|
||||
"""画像のPathオブジェクトを返す（ルートで使用）。
|
||||
PSDはブラウザが直接レンダリングできないため、合成済みPNGにキャッシュ変換して返す。"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return None
|
||||
if p.suffix.lower() not in IMAGE_EXTENSIONS:
|
||||
@@ -1282,6 +1602,58 @@ class GalleryService:
|
||||
return self._get_psd_composite_cache(p)
|
||||
return p
|
||||
|
||||
def serve_model3d_asset(self, model_path: str, uri: str) -> Path | None:
|
||||
""".gltfが相対URIで参照する外部ファイル(.bin/テクスチャ)のPathを返す。
|
||||
uriはglTF仕様どおりパーセントエンコードされている。"""
|
||||
try:
|
||||
p = self._resolve_untrusted(model_path)
|
||||
if p.suffix.lower() != ".gltf" or not p.is_file() or not self._check_path_allowed(p):
|
||||
return None
|
||||
# .gltf の外部参照 URI は相対パスのみ(UNC・絶対パス・「..」は結合前に拒否)
|
||||
rel_uri = unquote(uri)
|
||||
if not is_safe_rel(rel_uri):
|
||||
return None
|
||||
target = (p.parent / rel_uri).resolve()
|
||||
if target.suffix.lower() not in MODEL3D_ASSET_EXTENSIONS or not target.is_file():
|
||||
return None
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
if not self._check_path_allowed(target):
|
||||
logger.warning("serve_model3d_asset: path not allowed: %s", target)
|
||||
return None
|
||||
return target
|
||||
|
||||
def _model3d_thumb_path(self, p: Path) -> Path:
|
||||
"""ブラウザが描画した3Dサムネイル(各サイズの元画像)の保存先。"""
|
||||
mtime = int(p.stat().st_mtime * 1000)
|
||||
cache_dir = self.data_dir / "thumb_cache"
|
||||
cache_dir.mkdir(exist_ok=True)
|
||||
return cache_dir / f"{hashlib.md5(f'{p}:{mtime}:model3d'.encode()).hexdigest()}_m3d.jpg"
|
||||
|
||||
def save_model3d_thumbnail(self, model_path: str, image_data: str) -> dict:
|
||||
"""ブラウザが描画した3Dモデルのサムネイルを保存する。"""
|
||||
from io import BytesIO
|
||||
from PIL import Image
|
||||
|
||||
p = self._resolve_untrusted(model_path)
|
||||
if not p.is_file() or p.suffix.lower() not in MODEL3D_EXTENSIONS:
|
||||
return {"ok": False, "error": "Not a 3D model"}
|
||||
if not self._check_path_allowed(p):
|
||||
return {"ok": False, "error": "Access denied"}
|
||||
try:
|
||||
raw, _ = _decode_image_data_url(image_data)
|
||||
if len(raw) > _MODEL3D_THUMB_MAX_BYTES:
|
||||
return {"ok": False, "error": "Image too large"}
|
||||
with Image.open(BytesIO(raw)) as im:
|
||||
if max(im.size) > _MODEL3D_THUMB_MAX_SIDE:
|
||||
return {"ok": False, "error": "Image too large"}
|
||||
im = im.convert("RGB")
|
||||
im.thumbnail((_MODEL3D_THUMB_SIZE, _MODEL3D_THUMB_SIZE), Image.LANCZOS)
|
||||
im.save(self._model3d_thumb_path(p), "JPEG", quality=85, optimize=True)
|
||||
except (OSError, ValueError) as e:
|
||||
return {"ok": False, "error": str(e)}
|
||||
return {"ok": True}
|
||||
|
||||
def _get_psd_composite_cache(self, p: Path) -> Path | None:
|
||||
"""PSDの全レイヤー合成結果をフル解像度PNGとしてディスクキャッシュし、そのPathを返す。"""
|
||||
try:
|
||||
@@ -1306,6 +1678,48 @@ class GalleryService:
|
||||
logger.warning("serve_image: psd composite failed for %s: %s", p, e)
|
||||
return None
|
||||
|
||||
def _render_audio_waveform(self, path: Path, out_path: Path, size: int) -> None:
|
||||
"""音声ファイルの波形を size×size のJPEGとして out_path に書き出す。
|
||||
8kHzモノラルへリサンプルしてから列ごとの最大振幅を描く（長尺でも軽く保つ）。"""
|
||||
import av
|
||||
import numpy as np
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
max_samples = 8000 * 600 # 先頭10分まで
|
||||
chunks = []
|
||||
total = 0
|
||||
with av.open(str(path)) as container:
|
||||
stream = container.streams.audio[0]
|
||||
resampler = av.AudioResampler(format="flt", layout="mono", rate=8000)
|
||||
for frame in container.decode(stream):
|
||||
for rf in resampler.resample(frame):
|
||||
arr = rf.to_ndarray().reshape(-1)
|
||||
chunks.append(arr)
|
||||
total += arr.size
|
||||
if total >= max_samples:
|
||||
break
|
||||
data = np.concatenate(chunks) if chunks else np.zeros(1, dtype=np.float32)
|
||||
data = np.abs(data[:max_samples])
|
||||
|
||||
cols = size
|
||||
edges = np.linspace(0, data.size, cols + 1).astype(int)
|
||||
peaks = np.array([
|
||||
data[edges[i]:max(edges[i + 1], edges[i] + 1)].max() if edges[i] < data.size else 0.0
|
||||
for i in range(cols)
|
||||
])
|
||||
top = float(peaks.max())
|
||||
if top > 0:
|
||||
peaks = peaks / top
|
||||
|
||||
color = AUDIO_WAVE_COLORS.get(path.suffix.lower(), (56, 189, 248))
|
||||
img = Image.new("RGB", (size, size), (24, 28, 38))
|
||||
draw = ImageDraw.Draw(img)
|
||||
mid = size / 2
|
||||
for x, v in enumerate(peaks):
|
||||
h = max(1.0, float(v) * (size * 0.42))
|
||||
draw.line([(x, mid - h), (x, mid + h)], fill=color)
|
||||
img.save(out_path, "JPEG", quality=85, optimize=True)
|
||||
|
||||
def serve_thumbnail(self, image_path: str, width: int = 256) -> Path | None:
|
||||
"""縮小サムネイルのPathを返す。
|
||||
ディスクキャッシュがあればそれを返し、なければPillowで生成して保存する。
|
||||
@@ -1315,7 +1729,7 @@ class GalleryService:
|
||||
（元ファイルへのフォールバックはできないため、失敗時はNoneを返す）。
|
||||
PSDはpsd_toolsでレイヤーを合成してからJPEGとしてキャッシュする（同様にフォールバック不可）。
|
||||
"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return None
|
||||
if p.suffix.lower() not in IMAGE_EXTENSIONS:
|
||||
@@ -1338,7 +1752,9 @@ class GalleryService:
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
cache_key = hashlib.md5(f"{p}:{mtime}:{width}".encode()).hexdigest()
|
||||
# 音声は波形の描画仕様(色)を変えたらキャッシュを作り直せるよう版数をキーに含める
|
||||
wave_ver = f":{WAVE_THUMB_VERSION}" if p.suffix.lower() in AUDIO_EXTENSIONS else ""
|
||||
cache_key = hashlib.md5(f"{p}:{mtime}:{width}{wave_ver}".encode()).hexdigest()
|
||||
cache_dir = self.data_dir / "thumb_cache"
|
||||
cache_dir.mkdir(exist_ok=True)
|
||||
thumb_path = cache_dir / f"{cache_key}.jpg"
|
||||
@@ -1346,10 +1762,35 @@ class GalleryService:
|
||||
if thumb_path.exists():
|
||||
return thumb_path
|
||||
|
||||
# MP4は動画のためPillowで開けない。PyAV(av)で先頭フレームを抽出しJPEGとしてキャッシュする。
|
||||
# avが未導入/デコード失敗の場合、元mp4ファイルは<img>的な経路に渡せないため
|
||||
# 3Dはブラウザが描画して保存したものを縮小して返す。未保存ならNone(404)を返し、
|
||||
# それを合図にブラウザ側が描画する。
|
||||
if p.suffix.lower() in MODEL3D_EXTENSIONS:
|
||||
source = self._model3d_thumb_path(p)
|
||||
if not source.exists():
|
||||
return None
|
||||
try:
|
||||
from PIL import Image
|
||||
with Image.open(source) as img:
|
||||
img.thumbnail((width, width), Image.LANCZOS)
|
||||
img.save(thumb_path, "JPEG", quality=85, optimize=True)
|
||||
return thumb_path
|
||||
except Exception as e:
|
||||
logger.warning("serve_thumbnail: 3D thumbnail failed for %s: %s", p, e)
|
||||
return None
|
||||
|
||||
# 音声はPillowで開けない。PyAVでデコードして波形をJPEGに描画しキャッシュする。
|
||||
if p.suffix.lower() in AUDIO_EXTENSIONS:
|
||||
try:
|
||||
self._render_audio_waveform(p, thumb_path, width)
|
||||
return thumb_path
|
||||
except Exception as e:
|
||||
logger.warning("serve_thumbnail: audio waveform failed for %s: %s", p, e)
|
||||
return None
|
||||
|
||||
# 動画(mp4/webm)はPillowで開けない。PyAV(av)で先頭フレームを抽出しJPEGとしてキャッシュする。
|
||||
# avが未導入/デコード失敗の場合、元ファイルは<img>的な経路に渡せないため
|
||||
# (GIF/SVGと違い元ファイルへフォールバックできない) Noneを返しプレースホルダー表示に委ねる。
|
||||
if p.suffix.lower() == ".mp4":
|
||||
if p.suffix.lower() in VIDEO_EXTENSIONS:
|
||||
try:
|
||||
import av
|
||||
from PIL import Image
|
||||
|
||||
@@ -44,8 +44,7 @@ def extract_png_workflow(buf):
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# IENDで止めない: IENDの後ろに追記されたworkflowチャンクも読む
|
||||
pos += length + 12
|
||||
if chunk_type == "IEND":
|
||||
break
|
||||
|
||||
return None
|
||||
|
||||
@@ -0,0 +1,211 @@
|
||||
"""
|
||||
EmbeddingGemma 2 を Ollama で実行する (Gallery セマンティック検索のローカルバックエンド)
|
||||
|
||||
画像は /api/embed に input=[{"image": <base64>}] で渡す(images フィールドは無視されテキスト扱いになる)。
|
||||
740m(テキスト+画像+音声) と 440m(テキスト+画像) は同じ重みで同じベクトルを返すため、
|
||||
インデックスにはタグを区別せず LOCAL_MODEL として記録する。270m/570m は画像非対応なので選択肢に入れない。
|
||||
接続先は url_guard で検査する(既定はループバックのみ)。
|
||||
"""
|
||||
import base64
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import url_guard
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
LOCAL_MODEL = "embeddinggemma-2" # インデックス(npz)に記録するモデル名
|
||||
LOCAL_DIMS = (128, 256, 512, 768) # MRL対応次元
|
||||
DEFAULT_LOCAL_DIM = 768
|
||||
TAGS = ("embeddinggemma-2:740m", "embeddinggemma-2:440m")
|
||||
DEFAULT_TAG = TAGS[0]
|
||||
# :latest は 740m と同じモデル(parent_model 740m-q8_0)。740m が無ければ代わりに使う
|
||||
_ALIASES = {"embeddinggemma-2:740m": ("embeddinggemma-2:latest", "embeddinggemma-2")}
|
||||
DEFAULT_URL = "http://localhost:11434"
|
||||
QUERY_PREFIX = "task: search result | query: "
|
||||
BATCH = 4 # 1リクエストで送る画像数(1枚ずつより約1.8倍速い)
|
||||
_TIMEOUT = 120
|
||||
_RETRIES = 5 # Ollama内部の接続切断(HTTP 400 "forcibly closed"等)は再試行で回復する
|
||||
# Ollama はテキストごとに内部のランナーへ新しい接続を張るため、数千件を続けて埋め込むと
|
||||
# Windows の一時ポートが TIME_WAIT で埋まり "Only one usage of each socket address" で失敗する
|
||||
# (実測: ループバックの TIME_WAIT 14,228 / 16,384)。ポートが空くまで長めに待って再試行する
|
||||
_PORT_WAIT = 30.0
|
||||
_TAGS_TTL = 5.0 # /api/tags の結果をキャッシュする秒数(進捗ポーリングのたびに問い合わせない)
|
||||
|
||||
_opts = {"url": DEFAULT_URL, "tag": DEFAULT_TAG}
|
||||
_tags_cache = {"at": 0.0, "url": "", "names": None, "error": ""}
|
||||
_lock = threading.Lock()
|
||||
|
||||
|
||||
class OllamaError(Exception):
|
||||
"""code: ollama_unreachable / ollama_model_missing / local_error"""
|
||||
|
||||
def __init__(self, code: str, message: str = ""):
|
||||
super().__init__(message or code)
|
||||
self.code = code
|
||||
|
||||
|
||||
def configure(url: str = DEFAULT_URL, tag: str = DEFAULT_TAG) -> None:
|
||||
"""接続先とモデルサイズを設定する。許可されないURLは ValueError(url_guard.HINT)。"""
|
||||
url = url_guard.require_backend_url((url or "").strip() or DEFAULT_URL)
|
||||
with _lock:
|
||||
_opts["url"] = url
|
||||
_opts["tag"] = tag if tag in TAGS else DEFAULT_TAG
|
||||
_tags_cache["at"] = 0.0
|
||||
|
||||
|
||||
def current_config() -> dict:
|
||||
return dict(_opts)
|
||||
|
||||
|
||||
def _post(path: str, body: dict, timeout: float = _TIMEOUT) -> dict:
|
||||
base = url_guard.require_backend_url(_opts["url"])
|
||||
req = urllib.request.Request(base + path, data=json.dumps(body).encode("utf-8"), method="POST",
|
||||
headers={"Content-Type": "application/json"})
|
||||
with url_guard.urlopen(req, timeout) as res:
|
||||
return json.loads(res.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _http_message(e: urllib.error.HTTPError) -> str:
|
||||
try:
|
||||
return json.loads(e.read().decode("utf-8")).get("error", "") or str(e.reason)
|
||||
except Exception:
|
||||
return str(e.reason)
|
||||
|
||||
|
||||
def _transient(code: int, msg: str) -> bool:
|
||||
low = msg.lower()
|
||||
if "failed to load image" in low:
|
||||
return False # 壊れた画像。再試行しても直らない
|
||||
if code >= 500:
|
||||
return True
|
||||
return code == 400 and (_ports_exhausted(msg)
|
||||
or any(k in low for k in ("forcibly closed", "wsarecv", "connection", "eof")))
|
||||
|
||||
|
||||
def _ports_exhausted(msg: str) -> bool:
|
||||
low = msg.lower()
|
||||
return "only one usage of each socket address" in low or "wsaeaddrinuse" in low
|
||||
|
||||
|
||||
def installed_tags(force: bool = False) -> tuple[list[str] | None, str]:
|
||||
"""(インストール済みモデル名, エラー)。接続できなければ (None, エラー文)。"""
|
||||
now = time.time()
|
||||
with _lock:
|
||||
c = dict(_tags_cache)
|
||||
url = _opts["url"]
|
||||
if not force and c["url"] == url and now - c["at"] < _TAGS_TTL:
|
||||
return c["names"], c["error"]
|
||||
names, err = None, ""
|
||||
try:
|
||||
base = url_guard.require_backend_url(url)
|
||||
with url_guard.urlopen(urllib.request.Request(base + "/api/tags"), 3) as res:
|
||||
data = json.loads(res.read().decode("utf-8"))
|
||||
names = [m.get("name") or m.get("model") or "" for m in data.get("models", [])]
|
||||
except Exception as e:
|
||||
err = str(getattr(e, "reason", e))
|
||||
with _lock:
|
||||
_tags_cache.update(at=now, url=url, names=names, error=err)
|
||||
return names, err
|
||||
|
||||
|
||||
def _resolve(names: list[str], tag: str) -> str:
|
||||
"""実際に呼ぶモデル名。未インストールなら空文字"""
|
||||
for cand in (tag,) + _ALIASES.get(tag, ()):
|
||||
if cand in names:
|
||||
return cand
|
||||
return ""
|
||||
|
||||
|
||||
def status() -> dict:
|
||||
names, err = installed_tags()
|
||||
tag = _opts["tag"]
|
||||
out = {"url": _opts["url"], "tag": tag, "tags": list(TAGS), "reachable": names is not None, "error": err,
|
||||
"installed": {t: bool(names and _resolve(names, t)) for t in TAGS}}
|
||||
out["ready"] = bool(out["reachable"] and out["installed"][tag])
|
||||
return out
|
||||
|
||||
|
||||
def _model_name() -> str:
|
||||
names, err = installed_tags()
|
||||
if names is None:
|
||||
raise OllamaError("ollama_unreachable", f"{_opts['url']}: {err}")
|
||||
name = _resolve(names, _opts["tag"])
|
||||
if not name:
|
||||
raise OllamaError("ollama_model_missing", _opts["tag"])
|
||||
return name
|
||||
|
||||
|
||||
def _embed(inputs: list) -> list[np.ndarray]:
|
||||
model = _model_name()
|
||||
last = ""
|
||||
for attempt in range(_RETRIES + 1):
|
||||
try:
|
||||
data = _post("/api/embed", {"model": model, "input": inputs})
|
||||
embs = data.get("embeddings") or []
|
||||
if len(embs) != len(inputs):
|
||||
raise OllamaError("local_error", f"expected {len(inputs)} embeddings, got {len(embs)}")
|
||||
return [np.asarray(e, dtype=np.float32) for e in embs]
|
||||
except urllib.error.HTTPError as e:
|
||||
msg = _http_message(e)
|
||||
last = f"HTTP {e.code}: {msg}"
|
||||
if e.code == 404:
|
||||
with _lock:
|
||||
_tags_cache["at"] = 0.0 # 削除された可能性があるので次回は問い合わせ直す
|
||||
raise OllamaError("ollama_model_missing", model)
|
||||
if not _transient(e.code, msg) or attempt >= _RETRIES:
|
||||
raise OllamaError("local_error", last)
|
||||
if _ports_exhausted(msg):
|
||||
logger.warning("Ollama: local ports exhausted, waiting %.0fs before retry", _PORT_WAIT)
|
||||
time.sleep(_PORT_WAIT)
|
||||
continue
|
||||
except (urllib.error.URLError, TimeoutError, ConnectionError) as e:
|
||||
last = str(getattr(e, "reason", e))
|
||||
if attempt >= _RETRIES:
|
||||
raise OllamaError("ollama_unreachable", f"{_opts['url']}: {last}")
|
||||
time.sleep(min(4.0, 0.5 * (2 ** attempt)))
|
||||
raise OllamaError("local_error", last)
|
||||
|
||||
|
||||
def _finish(vec: np.ndarray, dim: int) -> np.ndarray:
|
||||
v = vec[:dim] if dim < vec.shape[0] else vec
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def _image_b64(image) -> str:
|
||||
buf = io.BytesIO()
|
||||
image.convert("RGB").save(buf, "JPEG", quality=88)
|
||||
return base64.b64encode(buf.getvalue()).decode("ascii")
|
||||
|
||||
|
||||
def embed_text(text: str, dim: int) -> np.ndarray:
|
||||
return embed_texts([text], dim)[0]
|
||||
|
||||
|
||||
def embed_texts(texts: list[str], dim: int) -> list[np.ndarray]:
|
||||
"""検索文(クエリ)として複数のテキストをまとめて埋め込む"""
|
||||
return [_finish(v, dim) for v in _embed([QUERY_PREFIX + t[:8000].strip() for t in texts])]
|
||||
|
||||
|
||||
def embed_images(images: list, dim: int) -> list[np.ndarray]:
|
||||
"""PIL画像(RGB)のリストをまとめて埋め込む。1枚でも読めないとリクエスト全体が失敗する。"""
|
||||
return [_finish(v, dim) for v in _embed([{"image": _image_b64(im)} for im in images])]
|
||||
|
||||
|
||||
def release() -> None:
|
||||
"""モデルをVRAMから外す(インデックス作成後)。次回利用時にOllamaが再ロードする。"""
|
||||
try:
|
||||
names, _ = installed_tags()
|
||||
name = _resolve(names or [], _opts["tag"])
|
||||
if name:
|
||||
_post("/api/embed", {"model": name, "input": [], "keep_alive": 0}, timeout=30)
|
||||
except Exception as e:
|
||||
logger.debug("semantic_ollama: release failed: %s", e)
|
||||
@@ -0,0 +1,582 @@
|
||||
"""
|
||||
Semantic Service - Gallery用セマンティック検索（Gemini Embedding 2 / Ollama の EmbeddingGemma 2）
|
||||
|
||||
画像(動画・PSDはサムネイル)をGemini Embedding API または Ollama でベクトル化してローカルに保存し、
|
||||
テキストクエリ/類似画像でコサイン類似度検索する。インデックス構築後の検索は
|
||||
(テキストクエリの埋め込み1回を除き)ローカル処理でAPI費用はかからない。
|
||||
|
||||
APIキーは環境変数 GEMINI_API_KEY (または GOOGLE_API_KEY) から読む(.env経由)。
|
||||
settings.json やフロントエンドには出さない。接続先は固定ホストのみ(SSRF対策)。
|
||||
"""
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import semantic_ollama as local
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
API_BASE = "https://generativelanguage.googleapis.com/v1beta"
|
||||
MODEL = "gemini-embedding-2" # Gemini API (クラウド)
|
||||
LOCAL_MODEL = local.LOCAL_MODEL # EmbeddingGemma 2 (Ollama)
|
||||
DEFAULT_DIM = 1536
|
||||
VALID_DIMS = (768, 1536, 3072)
|
||||
# バックエンド(モデル名) -> 選択可能な出力次元
|
||||
DIMS_BY_MODEL = {MODEL: VALID_DIMS, LOCAL_MODEL: local.LOCAL_DIMS}
|
||||
DEFAULT_DIM_BY_MODEL = {MODEL: DEFAULT_DIM, LOCAL_MODEL: local.DEFAULT_LOCAL_DIM}
|
||||
IMAGE_MAX_SIZE = 768 # 送信前の長辺リサイズ(px)
|
||||
_MAX_QUERY_CHARS = 2000 # 検索文の最大長
|
||||
_WORKERS = 4 # 並列API呼び出し数
|
||||
_CHECKPOINT_EVERY = 100 # この件数ごとにディスクへ保存
|
||||
_RETRIES = 4
|
||||
|
||||
# PILで直接開ける(静止画/GIFは先頭フレーム)
|
||||
_PIL_EXTS = {".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp"}
|
||||
# GalleryService.serve_thumbnail 経由でJPEG化する(動画先頭フレーム/PSD合成)
|
||||
_THUMB_EXTS = {".mp4", ".webm", ".psd"}
|
||||
INDEXABLE_EXTS = _PIL_EXTS | _THUMB_EXTS
|
||||
|
||||
|
||||
class SemanticError(Exception):
|
||||
"""ユーザーに見せられるエラー(code はフロントのi18n用)"""
|
||||
|
||||
def __init__(self, code: str, message: str = ""):
|
||||
super().__init__(message or code)
|
||||
self.code = code
|
||||
|
||||
|
||||
def get_api_key() -> str:
|
||||
return (os.environ.get("GEMINI_API_KEY", "") or os.environ.get("GOOGLE_API_KEY", "")).strip()
|
||||
|
||||
|
||||
def _normalize(v: np.ndarray) -> np.ndarray:
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def _call_embed(content: dict, task_type: str, dim: int) -> np.ndarray:
|
||||
key = get_api_key()
|
||||
if not key:
|
||||
raise SemanticError("no_api_key", "GEMINI_API_KEY is not set")
|
||||
body = {"model": f"models/{MODEL}", "content": content, "taskType": task_type}
|
||||
if dim != 768:
|
||||
body["outputDimensionality"] = dim
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
url = f"{API_BASE}/models/{MODEL}:embedContent"
|
||||
last_err = ""
|
||||
for attempt in range(_RETRIES + 1):
|
||||
req = urllib.request.Request(
|
||||
url, data=data, method="POST",
|
||||
headers={"Content-Type": "application/json", "x-goog-api-key": key},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=60) as res:
|
||||
payload = json.loads(res.read().decode("utf-8"))
|
||||
values = (payload.get("embedding") or {}).get("values")
|
||||
if not values:
|
||||
raise SemanticError("bad_response", "no embedding.values in response")
|
||||
return _normalize(np.asarray(values, dtype=np.float32))
|
||||
except urllib.error.HTTPError as e:
|
||||
try:
|
||||
msg = json.loads(e.read().decode("utf-8")).get("error", {}).get("message", "")
|
||||
except Exception:
|
||||
msg = ""
|
||||
last_err = f"HTTP {e.code}: {msg or e.reason}"
|
||||
if e.code in (400, 403) and ("API key" in msg or "API_KEY" in msg or e.code == 403):
|
||||
raise SemanticError("api_key_invalid", last_err)
|
||||
if e.code in (429, 500, 502, 503, 504) and attempt < _RETRIES:
|
||||
time.sleep(1.5 * (2 ** attempt))
|
||||
continue
|
||||
raise SemanticError("api_error", last_err)
|
||||
except (urllib.error.URLError, TimeoutError, ConnectionError) as e:
|
||||
last_err = str(getattr(e, "reason", e))
|
||||
if attempt < _RETRIES:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
raise SemanticError("network_error", last_err)
|
||||
raise SemanticError("api_error", last_err)
|
||||
|
||||
|
||||
def embed_text(text: str, dim: int) -> np.ndarray:
|
||||
return _call_embed({"parts": [{"text": text[:8000]}]}, "RETRIEVAL_QUERY", dim)
|
||||
|
||||
|
||||
def embed_image_bytes(data: bytes, mime: str, dim: int) -> np.ndarray:
|
||||
import base64
|
||||
b64 = base64.b64encode(data).decode("ascii")
|
||||
return _call_embed({"parts": [{"inlineData": {"mimeType": mime, "data": b64}}]}, "RETRIEVAL_DOCUMENT", dim)
|
||||
|
||||
|
||||
class SemanticService:
|
||||
def __init__(self, data_dir: Path, gallery_service):
|
||||
self.data_dir = data_dir
|
||||
self.gallery = gallery_service
|
||||
self._file = data_dir / "semantic_index.npz"
|
||||
self._lock = threading.RLock()
|
||||
self._vecs: dict[str, np.ndarray] = {} # path -> 正規化済みベクトル
|
||||
self._stamp: dict[str, tuple[int, float]] = {} # path -> (size, mtime)
|
||||
self._dim = 0 # インデックスの次元(0=空)
|
||||
self._model = MODEL # インデックスを作ったモデル
|
||||
self._settings_file = data_dir / "semantic_settings.json"
|
||||
self._load_settings()
|
||||
self._mat_cache = None # (paths, matrix)
|
||||
self._loaded = False
|
||||
self._dirty = 0
|
||||
# ジョブ
|
||||
self._job = self._idle_job()
|
||||
self._cancel = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
# ── Ollama の接続先/モデルサイズ ────────────────────────
|
||||
|
||||
def _load_settings(self) -> None:
|
||||
try:
|
||||
if self._settings_file.exists():
|
||||
d = json.loads(self._settings_file.read_text(encoding="utf-8"))
|
||||
local.configure(str(d.get("url", local.DEFAULT_URL)), str(d.get("tag", local.DEFAULT_TAG)))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to load settings: %s", e)
|
||||
|
||||
def set_local_options(self, url: str, tag: str) -> None:
|
||||
"""Ollama の接続先/モデルサイズを反映して保存する(検索時のクエリ埋め込みにも使われる)"""
|
||||
try:
|
||||
local.configure(url, tag)
|
||||
except ValueError as e:
|
||||
raise SemanticError("bad_url", str(e))
|
||||
try:
|
||||
self._settings_file.write_text(json.dumps(local.current_config()), encoding="utf-8")
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to save settings: %s", e)
|
||||
|
||||
# ── 永続化 ─────────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def _idle_job() -> dict:
|
||||
return {"running": False, "total": 0, "done": 0, "failed": 0, "current": "",
|
||||
"error": "", "error_code": "", "cancelled": False, "finished": False}
|
||||
|
||||
def _ensure_loaded(self) -> None:
|
||||
with self._lock:
|
||||
if self._loaded:
|
||||
return
|
||||
self._loaded = True
|
||||
if not self._file.exists():
|
||||
return
|
||||
try:
|
||||
with np.load(self._file, allow_pickle=False) as z:
|
||||
model = str(z["model"])
|
||||
if model not in DIMS_BY_MODEL:
|
||||
return
|
||||
self._model = model
|
||||
paths = [str(p) for p in z["paths"]]
|
||||
sizes, mtimes, vecs = z["sizes"], z["mtimes"], z["vecs"]
|
||||
self._dim = int(z["dim"])
|
||||
for i, p in enumerate(paths):
|
||||
self._vecs[p] = vecs[i]
|
||||
self._stamp[p] = (int(sizes[i]), float(mtimes[i]))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to load index: %s", e)
|
||||
self._vecs.clear()
|
||||
self._stamp.clear()
|
||||
self._dim = 0
|
||||
|
||||
def _save(self) -> None:
|
||||
with self._lock:
|
||||
if not self._vecs:
|
||||
if self._file.exists():
|
||||
self._file.unlink()
|
||||
self._dirty = 0
|
||||
return
|
||||
paths = list(self._vecs.keys())
|
||||
tmp = self._file.with_suffix(".tmp.npz")
|
||||
np.savez(
|
||||
tmp,
|
||||
model=np.array(self._model), dim=np.array(self._dim),
|
||||
paths=np.array(paths),
|
||||
sizes=np.array([self._stamp[p][0] for p in paths], dtype=np.int64),
|
||||
mtimes=np.array([self._stamp[p][1] for p in paths], dtype=np.float64),
|
||||
vecs=np.stack([self._vecs[p] for p in paths]).astype(np.float32),
|
||||
)
|
||||
os.replace(tmp, self._file)
|
||||
self._dirty = 0
|
||||
|
||||
def _put(self, path: str, vec: np.ndarray, size: int, mtime: float) -> None:
|
||||
with self._lock:
|
||||
self._vecs[path] = vec
|
||||
self._stamp[path] = (size, mtime)
|
||||
self._mat_cache = None
|
||||
self._dirty += 1
|
||||
|
||||
# ── 状態 ───────────────────────────────────────────────
|
||||
|
||||
def _scope_entries(self, folder: str, recursive: bool) -> list[tuple[str, str, int, float]]:
|
||||
try:
|
||||
p = self.gallery._resolve_untrusted(folder) # UNC等は resolve() 前に文字列で拒否
|
||||
except ValueError:
|
||||
raise SemanticError("folder_not_allowed", str(folder))
|
||||
if not p.is_dir() or not self.gallery._check_path_allowed(p):
|
||||
raise SemanticError("folder_not_allowed", str(folder))
|
||||
raw = self.gallery._scan_folder_recursive(p) if recursive else self.gallery._scan_folder(p)
|
||||
return [e for e in raw if Path(e[0]).suffix.lower() in INDEXABLE_EXTS]
|
||||
|
||||
def status(self, folder: str = "", recursive: bool = True) -> dict:
|
||||
self._ensure_loaded()
|
||||
ollama = local.status()
|
||||
out = {
|
||||
"has_key": bool(get_api_key()),
|
||||
"model": MODEL,
|
||||
"backends": {
|
||||
MODEL: {"dims": list(VALID_DIMS), "default_dim": DEFAULT_DIM, "ready": bool(get_api_key())},
|
||||
LOCAL_MODEL: {"dims": list(local.LOCAL_DIMS), "default_dim": local.DEFAULT_LOCAL_DIM,
|
||||
"ready": ollama["ready"]},
|
||||
},
|
||||
"index_model": self._model,
|
||||
"index_dim": self._dim,
|
||||
"ollama": ollama,
|
||||
"indexed_total": len(self._vecs),
|
||||
"job": dict(self._job),
|
||||
}
|
||||
if folder:
|
||||
try:
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
with self._lock:
|
||||
done = sum(1 for _, ap, sz, mt in entries if self._is_fresh(ap, sz, mt))
|
||||
out["scope_total"] = len(entries)
|
||||
out["scope_indexed"] = done
|
||||
except SemanticError:
|
||||
pass
|
||||
return out
|
||||
|
||||
def folder_status(self, root: str) -> dict:
|
||||
"""フォルダごとの索引状況 {絶対パス: {"indexable": n, "indexed": m}} (直下の画像のみ。Galleryツリーの枚数表示と同じ単位)"""
|
||||
self._ensure_loaded()
|
||||
entries = self._scope_entries(root, True)
|
||||
out: dict[str, dict] = {}
|
||||
with self._lock:
|
||||
for _, ap, sz, mt in entries:
|
||||
d = out.setdefault(ap.rsplit("/", 1)[0], {"indexable": 0, "indexed": 0})
|
||||
d["indexable"] += 1
|
||||
if self._is_fresh(ap, sz, mt):
|
||||
d["indexed"] += 1
|
||||
return out
|
||||
|
||||
def _is_fresh(self, path: str, size: int, mtime: float) -> bool:
|
||||
st = self._stamp.get(path)
|
||||
return st is not None and st[0] == size and abs(st[1] - mtime) < 1e-3
|
||||
|
||||
# ── インデックス作成ジョブ ────────────────────────────
|
||||
|
||||
def start_index(self, folder: str, recursive: bool, mode: str, model: str, dim: int) -> None:
|
||||
self._ensure_loaded()
|
||||
if model not in DIMS_BY_MODEL:
|
||||
raise SemanticError("bad_model", model)
|
||||
if model == MODEL and not get_api_key():
|
||||
raise SemanticError("no_api_key")
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
raise SemanticError("bad_dim", str(dim))
|
||||
if model == LOCAL_MODEL:
|
||||
st = local.status() # 接続できない/モデル未取得なら、ジョブを始める前に知らせる
|
||||
if not st["reachable"]:
|
||||
raise SemanticError("ollama_unreachable", f"{st['url']}: {st['error']}")
|
||||
if not st["ready"]:
|
||||
raise SemanticError("ollama_model_missing", st["tag"])
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
mismatch = bool(self._vecs) and (self._dim != dim or self._model != model)
|
||||
if mismatch and mode != "all":
|
||||
raise SemanticError("dim_mismatch", f"{self._model}/{self._dim}")
|
||||
if mode == "all":
|
||||
# 範囲内の既存ベクトルを破棄して作り直す(モデル/次元違いなら全消去)
|
||||
if mismatch:
|
||||
self._vecs.clear()
|
||||
self._stamp.clear()
|
||||
else:
|
||||
for _, ap, _, _ in entries:
|
||||
self._vecs.pop(ap, None)
|
||||
self._stamp.pop(ap, None)
|
||||
self._mat_cache = None
|
||||
self._dim = dim
|
||||
self._model = model
|
||||
todo = [e for e in entries if not self._is_fresh(e[1], e[2], e[3])]
|
||||
# スコープ内で消えたファイルの残骸を掃除(スコープ配下のキーのみ)
|
||||
self._prune_missing(folder, recursive, {e[1] for e in entries})
|
||||
self._cancel.clear()
|
||||
self._job = self._idle_job()
|
||||
self._job.update(running=True, total=len(todo))
|
||||
self._thread = threading.Thread(target=self._run_index, args=(todo, model, dim), daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _prune_missing(self, folder: str, recursive: bool, existing: set[str]) -> None:
|
||||
base = Path(folder).resolve().as_posix().rstrip("/") + "/"
|
||||
for p in list(self._vecs.keys()):
|
||||
if not p.startswith(base):
|
||||
continue
|
||||
if not recursive and "/" in p[len(base):]:
|
||||
continue
|
||||
if p not in existing:
|
||||
self._vecs.pop(p, None)
|
||||
self._stamp.pop(p, None)
|
||||
self._mat_cache = None
|
||||
self._dirty += 1
|
||||
|
||||
def cancel(self) -> None:
|
||||
self._cancel.set()
|
||||
|
||||
def _run_index(self, todo: list, model: str, dim: int) -> None:
|
||||
def fail(e: SemanticError) -> None:
|
||||
if e.code in ("no_api_key", "api_key_invalid", "ollama_unreachable", "ollama_model_missing"):
|
||||
# 続けても全件失敗するので中止する
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
self._cancel.set()
|
||||
else:
|
||||
self._job["failed"] += 1
|
||||
if not self._job["error"]:
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
|
||||
def checkpoint() -> None:
|
||||
if self._dirty >= _CHECKPOINT_EVERY:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: checkpoint failed: %s", e)
|
||||
|
||||
def work(entry):
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
name, ap, size, mtime = entry
|
||||
self._job["current"] = name
|
||||
try:
|
||||
vec = self._embed_image(model, dim, Path(ap))
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
except SemanticError as e:
|
||||
fail(e)
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: embed failed for %s: %s", ap, e)
|
||||
self._job["failed"] += 1
|
||||
checkpoint()
|
||||
|
||||
def work_batch(batch):
|
||||
"""Ollama: まとめて送る。失敗したら1枚ずつ送り直して原因の画像だけを失敗扱いにする"""
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
self._job["current"] = batch[0][0]
|
||||
vecs = None
|
||||
try:
|
||||
ims = [self._load_local_image(Path(e[1])) for e in batch]
|
||||
vecs = local.embed_images(ims, dim)
|
||||
except local.OllamaError as e:
|
||||
if e.code != "local_error":
|
||||
fail(SemanticError(e.code, str(e)))
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
if vecs is None:
|
||||
for entry in batch:
|
||||
work(entry)
|
||||
return
|
||||
for (name, ap, size, mtime), vec in zip(batch, vecs):
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
checkpoint()
|
||||
|
||||
try:
|
||||
if model == LOCAL_MODEL:
|
||||
for i in range(0, len(todo), local.BATCH):
|
||||
work_batch(todo[i:i + local.BATCH])
|
||||
else:
|
||||
with ThreadPoolExecutor(max_workers=_WORKERS) as ex:
|
||||
list(ex.map(work, todo))
|
||||
finally:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: save failed: %s", e)
|
||||
if model == LOCAL_MODEL:
|
||||
local.release() # 画像生成とVRAMを奪い合わないよう、作成後はモデルを外す(検索時にOllamaが再ロード)
|
||||
self._job["running"] = False
|
||||
self._job["finished"] = True
|
||||
self._job["cancelled"] = self._cancel.is_set() and not self._job["error_code"]
|
||||
self._job["current"] = ""
|
||||
|
||||
def _load_pil(self, path: Path):
|
||||
"""埋め込み用に縮小したPIL画像を返す(動画/PSDはサムネイルから)"""
|
||||
from PIL import Image
|
||||
if path.suffix.lower() in _THUMB_EXTS:
|
||||
thumb = self.gallery.serve_thumbnail(str(path), IMAGE_MAX_SIZE)
|
||||
if not thumb:
|
||||
raise SemanticError("thumb_failed", str(path))
|
||||
with Image.open(thumb) as im:
|
||||
return im.convert("RGB")
|
||||
with Image.open(path) as im:
|
||||
im.seek(0)
|
||||
im = im.convert("RGBA") if im.mode in ("RGBA", "LA", "P") else im.convert("RGB")
|
||||
im.thumbnail((IMAGE_MAX_SIZE, IMAGE_MAX_SIZE), Image.LANCZOS)
|
||||
return im
|
||||
|
||||
def _load_local_image(self, path: Path):
|
||||
"""Ollama用のRGB画像(透過部は白背景に合成)"""
|
||||
from PIL import Image
|
||||
im = self._load_pil(path)
|
||||
if im.mode == "RGBA":
|
||||
bg = Image.new("RGB", im.size, (255, 255, 255))
|
||||
bg.paste(im, mask=im.getchannel("A"))
|
||||
return bg
|
||||
return im
|
||||
|
||||
def _embed_image(self, model: str, dim: int, path: Path) -> np.ndarray:
|
||||
if model == LOCAL_MODEL:
|
||||
im = self._load_local_image(path)
|
||||
try:
|
||||
return local.embed_images([im], dim)[0]
|
||||
except local.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
im = self._load_pil(path)
|
||||
buf = io.BytesIO()
|
||||
if im.mode == "RGBA":
|
||||
im.save(buf, "PNG")
|
||||
return embed_image_bytes(buf.getvalue(), "image/png", dim)
|
||||
im.save(buf, "JPEG", quality=88)
|
||||
return embed_image_bytes(buf.getvalue(), "image/jpeg", dim)
|
||||
|
||||
def _embed_text(self, model: str, dim: int, text: str) -> np.ndarray:
|
||||
if model == LOCAL_MODEL:
|
||||
try:
|
||||
return local.embed_text(text, dim)
|
||||
except local.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
return embed_text(text, dim)
|
||||
|
||||
# ── 他機能(TagSpace)向けの読み出し ─────────────────────
|
||||
|
||||
def index_info(self) -> dict:
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return {"model": self._model, "dim": self._dim, "count": len(self._vecs)}
|
||||
|
||||
def job_status(self) -> dict:
|
||||
return dict(self._job)
|
||||
|
||||
def vectors_for(self, entries: list) -> dict[str, np.ndarray]:
|
||||
"""entries([(name, abs_path, size, mtime)])のうち、索引済みで最新のもののベクトル"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return {ap: self._vecs[ap] for _, ap, sz, mt in entries
|
||||
if ap in self._vecs and self._is_fresh(ap, sz, mt)}
|
||||
|
||||
def indexed_entries(self, exclude_prefix: str = "") -> list[tuple[str, int, float]]:
|
||||
"""索引済みの (path, size, mtime) 一覧。exclude_prefix配下は除く"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return [(p, *self._stamp[p]) for p in self._vecs
|
||||
if not (exclude_prefix and p.startswith(exclude_prefix))]
|
||||
|
||||
def sample_vectors(self, n: int, exclude_prefix: str = "", seed: int = 0) -> tuple[list[str], np.ndarray]:
|
||||
"""索引から最大n件を無作為に(seed固定で)取り出す。exclude_prefix配下は除く"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
paths = [p for p in self._vecs if not (exclude_prefix and p.startswith(exclude_prefix))]
|
||||
if len(paths) > n:
|
||||
pick = np.random.default_rng(seed).choice(len(paths), n, replace=False)
|
||||
paths = [paths[i] for i in sorted(pick)]
|
||||
mat = (np.stack([self._vecs[p] for p in paths]) if paths
|
||||
else np.zeros((0, max(self._dim, 1)), np.float32))
|
||||
return paths, mat
|
||||
|
||||
# ── 検索 ───────────────────────────────────────────────
|
||||
|
||||
def _matrix(self):
|
||||
with self._lock:
|
||||
if self._mat_cache is None:
|
||||
paths = list(self._vecs.keys())
|
||||
mat = np.stack([self._vecs[p] for p in paths]) if paths else np.zeros((0, max(self._dim, 1)), np.float32)
|
||||
self._mat_cache = (paths, mat)
|
||||
return self._mat_cache
|
||||
|
||||
def _rank(self, qvec: np.ndarray, folder: str, recursive: bool, limit: int, exclude: str = "",
|
||||
min_score: float = 0.0) -> list[dict]:
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
info = {ap: (name, sz, mt) for name, ap, sz, mt in entries}
|
||||
paths, mat = self._matrix()
|
||||
if not paths:
|
||||
raise SemanticError("index_empty")
|
||||
if mat.shape[1] != qvec.shape[0]:
|
||||
raise SemanticError("dim_mismatch", str(self._dim))
|
||||
scores = mat @ qvec
|
||||
order = np.argsort(-scores)
|
||||
out = []
|
||||
for i in order:
|
||||
if float(scores[i]) < min_score:
|
||||
break # スコア降順なので、以降はすべてしきい値未満
|
||||
p = paths[i]
|
||||
if p == exclude or p not in info:
|
||||
continue
|
||||
name, sz, mt = info[p]
|
||||
item = self.gallery.make_list_item(name, p, sz, mt)
|
||||
item["score"] = round(float(scores[i]), 4)
|
||||
out.append(item)
|
||||
if len(out) >= limit:
|
||||
break
|
||||
return out
|
||||
|
||||
def search_text(self, query: str, folder: str, recursive: bool, limit: int, min_score: float = 0.0) -> list[dict]:
|
||||
self._ensure_loaded()
|
||||
query = (query or "").strip()[:_MAX_QUERY_CHARS]
|
||||
if not query:
|
||||
return []
|
||||
if not self._vecs:
|
||||
raise SemanticError("index_empty")
|
||||
qvec = self._embed_text(self._model, self._dim, query)
|
||||
return self._rank(qvec, folder, recursive, limit, min_score=min_score)
|
||||
|
||||
def search_similar(self, path: str, folder: str, recursive: bool, limit: int, min_score: float = 0.0) -> list[dict]:
|
||||
self._ensure_loaded()
|
||||
try:
|
||||
p = self.gallery._resolve_untrusted(path) # UNC等は resolve() 前に文字列で拒否
|
||||
except ValueError:
|
||||
raise SemanticError("folder_not_allowed", path)
|
||||
if not p.is_file() or not self.gallery._check_path_allowed(p):
|
||||
raise SemanticError("folder_not_allowed", path)
|
||||
ap = p.as_posix()
|
||||
vec = self._vecs.get(ap)
|
||||
if vec is None or not self._is_fresh(ap, p.stat().st_size, p.stat().st_mtime):
|
||||
# 未インデックスの基準画像は、その場で1枚だけ埋め込む
|
||||
if not self._vecs:
|
||||
raise SemanticError("index_empty")
|
||||
if p.suffix.lower() not in INDEXABLE_EXTS:
|
||||
raise SemanticError("unsupported_type", p.suffix)
|
||||
vec = self._embed_image(self._model, self._dim, p)
|
||||
st = p.stat()
|
||||
self._put(ap, vec, st.st_size, st.st_mtime)
|
||||
self._save()
|
||||
return self._rank(vec, folder, recursive, limit, exclude=ap, min_score=min_score)
|
||||
|
||||
def clear(self) -> None:
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
self._vecs.clear()
|
||||
self._stamp.clear()
|
||||
self._dim = 0
|
||||
self._mat_cache = None
|
||||
self._save()
|
||||
|
||||
def test_connection(self, model: str, dim: int) -> dict:
|
||||
if model not in DIMS_BY_MODEL:
|
||||
raise SemanticError("bad_model", model)
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
dim = DEFAULT_DIM_BY_MODEL[model]
|
||||
n = int(self._embed_text(model, dim, "test").shape[0])
|
||||
return {"dimensions": n, "provider": f"Ollama {local.current_config()['tag']}" if model == LOCAL_MODEL else "API"}
|
||||
+434
-160
@@ -1,192 +1,347 @@
|
||||
"""AI skill (.md system prompt) file management service."""
|
||||
"""AI skill (.md system prompt) file management service.
|
||||
|
||||
Skills are flat ``*.md`` files with a YAML frontmatter block::
|
||||
|
||||
---
|
||||
name: Display name
|
||||
description: One sentence
|
||||
metadata: # optional, Workflow Studio extensions
|
||||
wfs-version: 2 # built-in skills only (upgrade tracking)
|
||||
output: prompt # prompt | svg -> auto-check + action buttons
|
||||
tools: [] # [] or [generate_image]
|
||||
temperature: 0.4
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
``name``/``description`` follow the Agent Skills (SKILL.md) convention so skills
|
||||
stay portable; everything Workflow-Studio-specific lives under ``metadata``.
|
||||
|
||||
Built-in skills ship in ``builtin_skills/`` next to this file and are copied
|
||||
into the user's skill folder. A small state file records which version and
|
||||
content hash were installed, so an untouched built-in is upgraded silently
|
||||
while a user-edited one is left alone (and flagged ``update_available``).
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import zipfile
|
||||
from pathlib import Path, PurePosixPath
|
||||
|
||||
try:
|
||||
import yaml # PyYAML ships with ComfyUI
|
||||
except Exception: # pragma: no cover - fallback parser below
|
||||
yaml = None
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
ALLOWED_EXTS = {"md"}
|
||||
|
||||
DEFAULT_SKILL_FILENAME = "svg-icon.md"
|
||||
# Folder skills (Agent Skills / SKILL.md standard): <skills_dir>/<folder>/SKILL.md plus optional
|
||||
# reference files. Only text resources are kept and served - Workflow Studio never runs a
|
||||
# skill's scripts, so script files are skipped on import.
|
||||
SKILL_MD = "SKILL.md"
|
||||
RESOURCE_EXTS = {"md", "txt", "json", "yaml", "yml", "csv"}
|
||||
MAX_RESOURCES = 50
|
||||
MAX_RESOURCE_BYTES = 200 * 1024 # per file served to the chat model
|
||||
MAX_IMPORT_FILE_BYTES = 1024 * 1024 # per file inside an imported zip
|
||||
MAX_IMPORT_TOTAL_BYTES = 5 * 1024 * 1024
|
||||
MAX_IMPORT_ENTRIES = 300
|
||||
_FOLDER_RE = re.compile(r"^[A-Za-z0-9][\w.\- ]{0,63}$")
|
||||
|
||||
DEFAULT_SKILL_CONTENT = """---
|
||||
name: SVG Icon Generator
|
||||
description: Produces clean, minimal, valid SVG icons and simple illustrations.
|
||||
---
|
||||
BUILTIN_DIR = Path(__file__).parent / "builtin_skills"
|
||||
STATE_FILENAME = ".builtin_state.json"
|
||||
|
||||
You are an expert SVG icon and illustration designer. When the user asks for an icon, logo, pictogram, diagram, character, animal, or any vector graphic, respond with valid, clean SVG markup.
|
||||
|
||||
## Composition process (do this before writing markup - skipping this step is the main cause of unrecognizable output)
|
||||
- Break the subject down into 3-8 distinct parts that make it recognizable (e.g. a bird: body, head, beak, eye, wing, tail; a house: wall, roof, door, window). Drop parts that are not essential.
|
||||
- Decide each part's approximate position, size, and draw order within the viewBox (elements listed later in the markup are drawn on top).
|
||||
- Pick the simplest shape that captures each part's silhouette: `<circle>`/`<ellipse>` for round parts, `<rect>` for rectangular parts, `<polygon>` for angular parts (beaks, ears, leaves, roofs). For a rounded organic outline that no primitive can approximate well (a body silhouette, a wing, a leaf edge), use a short `<path>` with 2-4 smooth curve commands (`Q` or `C`) instead of a plain circle - a few deliberate curve points beat a distorted primitive.
|
||||
- Never fall back to drawing an unrelated blob plus a few disconnected decorative lines when a subject is hard to render - always decompose it into the named parts above instead, even if each part is crude.
|
||||
|
||||
## Output rules
|
||||
- Always wrap the SVG markup in a fenced code block labeled `svg`, e.g. ```svg ... ```.
|
||||
- Keep any explanation brief and place it OUTSIDE the code block, never mixed inside the SVG.
|
||||
- Output exactly one `<svg>...</svg>` element per code block.
|
||||
|
||||
## Structure rules
|
||||
- Always include a `viewBox` attribute (e.g. `viewBox="0 0 24 24"`). Do not hardcode large pixel `width`/`height`; if you set them, make them match the viewBox aspect ratio.
|
||||
- Use a small, round coordinate system (0-24 or 0-100).
|
||||
- Round coordinate values to at most 2 decimal places.
|
||||
- Keep every `<path>` short (at most about 6 commands) - long, intricate paths are where quality breaks down most often.
|
||||
- Group related elements with `<g>` when it clarifies structure, but do not over-nest.
|
||||
|
||||
## Color rules
|
||||
- For single-color icons, use `fill="currentColor"` (or `stroke="currentColor"` for line icons) so the icon inherits the surrounding text color.
|
||||
- For multi-color illustrations, use explicit hex colors (e.g. `#4A90D9`) - never named CSS colors like `"blue"`.
|
||||
- Avoid gradients, filters, and blend modes unless the user explicitly asks for a glossy/gradient style - they rarely render correctly from generated coordinates.
|
||||
|
||||
## Accessibility
|
||||
- Include a `<title>` element as the first child of `<svg>` describing what the icon depicts.
|
||||
|
||||
## Security (mandatory - do not violate)
|
||||
- Never include `<script>`, `<foreignObject>`, event handler attributes (`onload`, `onclick`, etc.), or `<a>`/`href` links to external URLs.
|
||||
- Never reference external resources (no `xlink:href` to remote URLs, no `<image>` with external `src`).
|
||||
|
||||
## Example 1 - simple geometric icon (single color, primitives only)
|
||||
|
||||
Request: "Sun icon"
|
||||
|
||||
Parts: disc (circle), 8 rays (lines).
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Sun</title>
|
||||
<circle cx="12" cy="12" r="5" fill="currentColor"/>
|
||||
<g stroke="currentColor" stroke-width="2" stroke-linecap="round">
|
||||
<line x1="12" y1="1" x2="12" y2="4"/>
|
||||
<line x1="12" y1="20" x2="12" y2="23"/>
|
||||
<line x1="4.22" y1="4.22" x2="6.34" y2="6.34"/>
|
||||
<line x1="17.66" y1="17.66" x2="19.78" y2="19.78"/>
|
||||
<line x1="1" y1="12" x2="4" y2="12"/>
|
||||
<line x1="20" y1="12" x2="23" y2="12"/>
|
||||
<line x1="4.22" y1="19.78" x2="6.34" y2="17.66"/>
|
||||
<line x1="17.66" y1="6.34" x2="19.78" y2="4.22"/>
|
||||
</g>
|
||||
</svg>
|
||||
```
|
||||
|
||||
## Example 2 - organic subject (multi-color, decomposed into parts)
|
||||
|
||||
Request: "Bird icon"
|
||||
|
||||
Parts: body (rounded outline via a short curved path), head (circle), beak (small triangle), eye (dot), wing (small curved path) - each drawn as its own element, layered head/beak/eye on top of the body.
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Bird</title>
|
||||
<path d="M4 16 Q4 10 11 9 Q18 8 20 13 Q20 17 14 18 Q7 19 4 16 Z" fill="#4A90D9"/>
|
||||
<path d="M8 14 Q5 12 3 14 Q6 16 9 16 Z" fill="#3576B8"/>
|
||||
<circle cx="17" cy="10" r="3.2" fill="#4A90D9"/>
|
||||
<polygon points="20,10 23,9.3 20,11.4" fill="#F2B807"/>
|
||||
<circle cx="18" cy="9" r="0.6" fill="#1A1A1A"/>
|
||||
</svg>
|
||||
```
|
||||
|
||||
Follow this pattern for every request: list the parts first, represent each with the simplest adequate shape, then assemble - never skip straight to unstructured shapes.
|
||||
"""
|
||||
|
||||
SKILL_CREATOR_FILENAME = "skill-creator.md"
|
||||
|
||||
SKILL_CREATOR_CONTENT = """---
|
||||
name: Skill Creator
|
||||
description: Interactively interviews you and drafts a new WFS chat skill (.md system prompt) ready to save.
|
||||
---
|
||||
|
||||
You are a skill-design assistant embedded in ComfyUI Workflow Studio's AI TOOL Chat pane. Your job is to interactively help the user design a NEW "skill" - a markdown file with frontmatter (`name`, `description`) followed by system-prompt instructions. Once saved, a skill can be selected from a dropdown and is silently prepended to future chat messages as a system prompt.
|
||||
|
||||
## How to conduct the interview
|
||||
- Ask ONE focused question at a time, never a bulleted list of questions. Wait for the user's answer before asking the next one.
|
||||
- Cover, in this order, only what is not already clear from context:
|
||||
1. What task or domain should the skill specialize in (what should the model do differently once this skill is active)?
|
||||
2. What output format is expected - plain prose, a specific fenced code block language, a fixed structure, a length limit?
|
||||
3. Any hard constraints or things to avoid (tone, forbidden content, security concerns, things past attempts got wrong)?
|
||||
4. Would a short worked example help smaller/local models follow the pattern more reliably? If yes, ask what a good example input/output looks like, or offer to draft one.
|
||||
- Keep it short - 3 to 5 questions is usually enough. If the user says "just generate it", "that's enough", or similar at any point, stop asking and produce the best skill you can from what you already have.
|
||||
|
||||
## Output format for the finished skill
|
||||
- Once you have enough information, output the complete skill file in a single fenced code block labeled `skill` (```skill ... ```). Never mix prose inside that block - explanation goes outside it.
|
||||
- The block must start with frontmatter exactly in this shape:
|
||||
---
|
||||
name: <short display name, shown in the skill dropdown>
|
||||
description: <one sentence, shown as a tooltip in the dropdown>
|
||||
---
|
||||
- After the frontmatter, write clear, imperative system-prompt instructions for the target skill: what role the model should take, concrete output-format rules, hard constraints, and - if useful - one short worked example following a "request -> ideal output" shape.
|
||||
- After the code block, tell the user in one line that they can click the "Save as new skill" button below the message to add it, or copy the block manually into a new skill.
|
||||
"""
|
||||
# Content hashes of built-in skills released before version tracking existed
|
||||
# (v0.3.79 - v0.9.4). A user file matching one of these is an untouched copy
|
||||
# and can be upgraded safely.
|
||||
LEGACY_BUILTIN_HASHES = {
|
||||
"svg-icon.md": {"f46b635abda182b26df38d4e49ec2a626642fcd98c94013cb67303c69aaf10c1"},
|
||||
"skill-creator.md": {"2fb31273298b828c6f18c7fd010be44d31588f881d04757e5c0f5f18bbad2313"},
|
||||
}
|
||||
|
||||
_FRONTMATTER_RE = re.compile(r"^---\r?\n(.*?)\r?\n---\r?\n?", re.DOTALL)
|
||||
|
||||
|
||||
def _parse_frontmatter(content: str) -> dict:
|
||||
"""Extract a shallow `key: value` frontmatter block, if present."""
|
||||
m = _FRONTMATTER_RE.match(content)
|
||||
if not m:
|
||||
return {}
|
||||
def _content_hash(content: str) -> str:
|
||||
return hashlib.sha256(content.replace("\r\n", "\n").strip().encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _parse_frontmatter_shallow(block: str) -> dict:
|
||||
"""Fallback parser: top-level ``key: value`` plus one nested ``metadata:`` level."""
|
||||
fields: dict = {}
|
||||
for line in m.group(1).splitlines():
|
||||
if ":" not in line:
|
||||
current_map: dict | None = None
|
||||
for line in block.splitlines():
|
||||
if not line.strip() or ":" not in line:
|
||||
continue
|
||||
key, _, value = line.partition(":")
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
if key:
|
||||
fields[key] = value
|
||||
indented = line[:1] in (" ", "\t")
|
||||
key, _, value = line.strip().partition(":")
|
||||
key, value = key.strip(), value.strip()
|
||||
if not key:
|
||||
continue
|
||||
if indented and current_map is not None:
|
||||
current_map[key] = _coerce_scalar(value)
|
||||
elif not indented:
|
||||
if value == "":
|
||||
current_map = {}
|
||||
fields[key] = current_map
|
||||
else:
|
||||
current_map = None
|
||||
fields[key] = value
|
||||
return fields
|
||||
|
||||
|
||||
class SkillService:
|
||||
def __init__(self, skills_dir: Path):
|
||||
self.skills_dir = skills_dir
|
||||
self.skills_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._seed_default_skill()
|
||||
def _coerce_scalar(value: str):
|
||||
if value.startswith("[") and value.endswith("]"):
|
||||
inner = value[1:-1].strip()
|
||||
return [v.strip().strip("'\"") for v in inner.split(",") if v.strip()] if inner else []
|
||||
low = value.lower()
|
||||
if low in ("true", "false"):
|
||||
return low == "true"
|
||||
try:
|
||||
return float(value) if "." in value else int(value)
|
||||
except ValueError:
|
||||
return value.strip("'\"")
|
||||
|
||||
def _seed_default_skill(self) -> None:
|
||||
"""Seed a starter skill file the first time the directory is empty."""
|
||||
|
||||
def _parse_frontmatter(content: str) -> dict:
|
||||
"""Extract the frontmatter block as a dict (empty dict if absent/invalid)."""
|
||||
m = _FRONTMATTER_RE.match(content or "")
|
||||
if not m:
|
||||
return {}
|
||||
block = m.group(1)
|
||||
if yaml is not None:
|
||||
try:
|
||||
has_any = any(
|
||||
p.is_file() and p.suffix.lstrip(".").lower() in ALLOWED_EXTS
|
||||
for p in self.skills_dir.iterdir()
|
||||
)
|
||||
except FileNotFoundError:
|
||||
has_any = False
|
||||
if not has_any:
|
||||
(self.skills_dir / DEFAULT_SKILL_FILENAME).write_text(
|
||||
DEFAULT_SKILL_CONTENT, encoding="utf-8"
|
||||
)
|
||||
(self.skills_dir / SKILL_CREATOR_FILENAME).write_text(
|
||||
SKILL_CREATOR_CONTENT, encoding="utf-8"
|
||||
)
|
||||
data = yaml.safe_load(block)
|
||||
if isinstance(data, dict):
|
||||
return data
|
||||
except Exception:
|
||||
pass # e.g. an unquoted ": " inside description -> shallow parse
|
||||
return _parse_frontmatter_shallow(block)
|
||||
|
||||
|
||||
def _json_safe(value):
|
||||
"""Keep only JSON-serialisable metadata values (YAML can yield dates etc.)."""
|
||||
if isinstance(value, dict):
|
||||
return {str(k): _json_safe(v) for k, v in value.items()}
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [_json_safe(v) for v in value]
|
||||
if isinstance(value, (str, int, float, bool)) or value is None:
|
||||
return value
|
||||
return str(value)
|
||||
|
||||
|
||||
def _skill_info(filename: str, content: str, size: int) -> dict:
|
||||
meta = _parse_frontmatter(content)
|
||||
extra = meta.get("metadata") if isinstance(meta.get("metadata"), dict) else {}
|
||||
return {
|
||||
"filename": filename,
|
||||
"name": str(meta.get("name") or Path(filename).stem),
|
||||
"description": str(meta.get("description") or ""),
|
||||
"metadata": _json_safe(extra),
|
||||
"size": size,
|
||||
}
|
||||
|
||||
|
||||
def _builtin_version(content: str) -> int:
|
||||
meta = _parse_frontmatter(content).get("metadata") or {}
|
||||
try:
|
||||
return int(meta.get("wfs-version", 1))
|
||||
except (TypeError, ValueError):
|
||||
return 1
|
||||
|
||||
|
||||
class SkillService:
|
||||
def __init__(self, skills_dir: Path, builtin_dir: Path = BUILTIN_DIR):
|
||||
self.skills_dir = skills_dir
|
||||
self.builtin_dir = builtin_dir
|
||||
self.skills_dir.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
self.sync_builtin_skills()
|
||||
except Exception as e: # never block startup on a seeding problem
|
||||
logger.warning("[WFS] Built-in skill sync failed: %s", e)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Built-in skills
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _builtins(self) -> dict[str, str]:
|
||||
if not self.builtin_dir.is_dir():
|
||||
return {}
|
||||
return {
|
||||
p.name: p.read_text(encoding="utf-8")
|
||||
for p in sorted(self.builtin_dir.glob("*.md"))
|
||||
if p.is_file()
|
||||
}
|
||||
|
||||
def _load_state(self) -> dict:
|
||||
path = self.skills_dir / STATE_FILENAME
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
return data if isinstance(data, dict) else {}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
def _save_state(self, state: dict) -> None:
|
||||
(self.skills_dir / STATE_FILENAME).write_text(
|
||||
json.dumps(state, indent=2, ensure_ascii=False), encoding="utf-8"
|
||||
)
|
||||
|
||||
def _install_builtin(self, filename: str, content: str, state: dict) -> None:
|
||||
(self.skills_dir / filename).write_text(content, encoding="utf-8")
|
||||
state[filename] = {"version": _builtin_version(content), "hash": _content_hash(content)}
|
||||
|
||||
def sync_builtin_skills(self) -> dict:
|
||||
"""Install new built-ins and upgrade untouched ones.
|
||||
|
||||
Rules per built-in file:
|
||||
- missing and never installed -> install
|
||||
- missing but installed before -> the user deleted it, leave it deleted
|
||||
- present, untouched (hash matches what we installed, or a known
|
||||
pre-tracking release) and older -> overwrite with the new version
|
||||
- present but edited by the user -> keep it (reported as update_available)
|
||||
Returns ``{"installed": [...], "upgraded": [...]}``.
|
||||
"""
|
||||
state = self._load_state()
|
||||
result = {"installed": [], "upgraded": []}
|
||||
changed = False
|
||||
for filename, content in self._builtins().items():
|
||||
new_version = _builtin_version(content)
|
||||
path = self.skills_dir / filename
|
||||
entry = state.get(filename)
|
||||
if not path.is_file():
|
||||
if entry is None:
|
||||
self._install_builtin(filename, content, state)
|
||||
result["installed"].append(filename)
|
||||
changed = True
|
||||
continue
|
||||
current_hash = _content_hash(path.read_text(encoding="utf-8"))
|
||||
if entry is None:
|
||||
if current_hash in LEGACY_BUILTIN_HASHES.get(filename, set()):
|
||||
self._install_builtin(filename, content, state)
|
||||
result["upgraded"].append(filename)
|
||||
elif current_hash == _content_hash(content):
|
||||
state[filename] = {"version": new_version, "hash": current_hash}
|
||||
else:
|
||||
# A user file with a built-in's name - never overwrite it.
|
||||
state[filename] = {"version": 0, "hash": None}
|
||||
changed = True
|
||||
continue
|
||||
if int(entry.get("version") or 0) < new_version and current_hash == entry.get("hash"):
|
||||
self._install_builtin(filename, content, state)
|
||||
result["upgraded"].append(filename)
|
||||
changed = True
|
||||
if changed:
|
||||
self._save_state(state)
|
||||
if result["installed"] or result["upgraded"]:
|
||||
logger.info("[WFS] Built-in skills installed=%s upgraded=%s", result["installed"], result["upgraded"])
|
||||
return result
|
||||
|
||||
def restore_builtin(self, filename: str) -> dict:
|
||||
"""Overwrite a skill file with the shipped built-in version (explicit user action)."""
|
||||
builtins = self._builtins()
|
||||
if filename not in builtins:
|
||||
raise ValueError(f"Not a built-in skill: {filename}")
|
||||
state = self._load_state()
|
||||
self._install_builtin(filename, builtins[filename], state)
|
||||
self._save_state(state)
|
||||
path = self.skills_dir / filename
|
||||
return _skill_info(filename, builtins[filename], path.stat().st_size)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# File CRUD
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def list_skills(self) -> list[dict]:
|
||||
"""Return sorted list of skill files with parsed frontmatter metadata."""
|
||||
"""Return sorted list of flat skill files and folder skills with parsed metadata."""
|
||||
if not self.skills_dir.exists():
|
||||
return []
|
||||
builtins = self._builtins()
|
||||
state = self._load_state()
|
||||
files = []
|
||||
for path in sorted(self.skills_dir.glob("*")):
|
||||
if path.is_dir():
|
||||
info = self._folder_skill_info(path)
|
||||
if info:
|
||||
files.append(info)
|
||||
continue
|
||||
if not (path.is_file() and path.suffix.lstrip(".").lower() in ALLOWED_EXTS):
|
||||
continue
|
||||
try:
|
||||
content = path.read_text(encoding="utf-8")
|
||||
except Exception:
|
||||
content = ""
|
||||
meta = _parse_frontmatter(content)
|
||||
files.append({
|
||||
"filename": path.name,
|
||||
"name": meta.get("name") or path.stem,
|
||||
"description": meta.get("description", ""),
|
||||
"size": path.stat().st_size,
|
||||
})
|
||||
info = _skill_info(path.name, content, path.stat().st_size)
|
||||
builtin_content = builtins.get(path.name)
|
||||
info["builtin"] = builtin_content is not None
|
||||
info["update_available"] = False
|
||||
if builtin_content is not None:
|
||||
current_hash = _content_hash(content)
|
||||
if current_hash != _content_hash(builtin_content):
|
||||
entry = state.get(path.name) or {}
|
||||
installed_version = int(entry.get("version") or 0)
|
||||
info["update_available"] = installed_version < _builtin_version(builtin_content)
|
||||
info["modified"] = True
|
||||
files.append(info)
|
||||
return files
|
||||
|
||||
def _folder_skill_info(self, folder: Path) -> dict | None:
|
||||
if folder.name.startswith(".") or not _FOLDER_RE.match(folder.name):
|
||||
return None
|
||||
skill_md = folder / SKILL_MD
|
||||
if not skill_md.is_file():
|
||||
return None
|
||||
try:
|
||||
content = skill_md.read_text(encoding="utf-8")
|
||||
except Exception:
|
||||
content = ""
|
||||
info = _skill_info(f"{folder.name}/{SKILL_MD}", content, skill_md.stat().st_size)
|
||||
if info["name"] == SKILL_MD.rsplit(".", 1)[0]:
|
||||
info["name"] = folder.name
|
||||
info["folder"] = folder.name
|
||||
info["resources"] = self.list_resources(folder.name)
|
||||
info["builtin"] = False
|
||||
info["update_available"] = False
|
||||
return info
|
||||
|
||||
def list_resources(self, folder: str) -> list[str]:
|
||||
"""Readable reference files of a folder skill (relative POSIX paths, SKILL.md excluded)."""
|
||||
base = self._folder_path(folder)
|
||||
if base is None or not base.is_dir():
|
||||
return []
|
||||
out = []
|
||||
for path in sorted(base.rglob("*")):
|
||||
if len(out) >= MAX_RESOURCES:
|
||||
break
|
||||
if not path.is_file() or path.is_symlink():
|
||||
continue
|
||||
rel = path.relative_to(base).as_posix()
|
||||
if rel == SKILL_MD or any(part.startswith(".") for part in rel.split("/")):
|
||||
continue
|
||||
if path.suffix.lstrip(".").lower() in RESOURCE_EXTS:
|
||||
out.append(rel)
|
||||
return out
|
||||
|
||||
def get_resource(self, folder: str, rel_path: str) -> str:
|
||||
"""Read one reference file of a folder skill (used by the chat's read_skill_file tool)."""
|
||||
base = self._folder_path(folder)
|
||||
if base is None or not base.is_dir():
|
||||
raise FileNotFoundError(f"Skill folder not found: {folder}")
|
||||
rel = PurePosixPath(str(rel_path or "").replace("\\", "/"))
|
||||
if rel.is_absolute() or ".." in rel.parts or not rel.parts or rel.as_posix() == SKILL_MD:
|
||||
raise ValueError(f"Invalid resource path: {rel_path}")
|
||||
if rel.suffix.lstrip(".").lower() not in RESOURCE_EXTS:
|
||||
raise ValueError(f"Unsupported resource type: {rel_path}")
|
||||
path = (base / rel).resolve()
|
||||
if base.resolve() not in path.parents or not path.is_file():
|
||||
raise FileNotFoundError(f"Resource not found: {rel_path}")
|
||||
data = path.read_bytes()[:MAX_RESOURCE_BYTES]
|
||||
return data.decode("utf-8", errors="replace")
|
||||
|
||||
def get_content(self, filename: str) -> str:
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
@@ -199,30 +354,149 @@ class SkillService:
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
raise ValueError(f"Invalid filename: {filename}")
|
||||
if path.name == SKILL_MD and not path.parent.is_dir():
|
||||
raise FileNotFoundError(f"Skill folder not found: {filename}")
|
||||
path.write_text(content, encoding="utf-8")
|
||||
meta = _parse_frontmatter(content)
|
||||
return {
|
||||
"filename": path.name,
|
||||
"name": meta.get("name") or path.stem,
|
||||
"description": meta.get("description", ""),
|
||||
"size": path.stat().st_size,
|
||||
}
|
||||
rel = path.relative_to(self.skills_dir).as_posix()
|
||||
return _skill_info(rel, content, path.stat().st_size)
|
||||
|
||||
def delete_file(self, filename: str) -> None:
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
raise ValueError(f"Invalid filename: {filename}")
|
||||
if path.name == SKILL_MD and path.parent != self.skills_dir:
|
||||
# A folder skill is deleted as a whole (SKILL.md + its reference files).
|
||||
if path.parent.is_dir():
|
||||
shutil.rmtree(path.parent)
|
||||
return
|
||||
if path.is_file():
|
||||
path.unlink()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Import (Agent Skills zip or a single .md)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def import_skill(self, upload_name: str, data: bytes) -> dict:
|
||||
"""Import an uploaded skill.
|
||||
|
||||
- ``*.md``: saved as a flat skill (``<slug>.md``; SKILL.md uploads are named after their
|
||||
frontmatter name).
|
||||
- ``*.zip``: an Agent Skills folder - SKILL.md at the zip root or inside exactly one top
|
||||
folder. Only text resources are extracted; scripts and binaries are skipped.
|
||||
Returns ``{"skill": info, "skipped": [paths]}``. Never overwrites: a numeric suffix is
|
||||
added when the name is taken.
|
||||
"""
|
||||
name = (upload_name or "").strip()
|
||||
lower = name.lower()
|
||||
if lower.endswith(".md"):
|
||||
text = data.decode("utf-8", errors="replace")
|
||||
meta = _parse_frontmatter(text)
|
||||
stem = Path(name).stem if Path(name).name != SKILL_MD else str(meta.get("name") or "imported-skill")
|
||||
filename = self._unique_name(_slugify(stem), ".md")
|
||||
(self.skills_dir / filename).write_text(text, encoding="utf-8")
|
||||
return {"skill": _skill_info(filename, text, len(data)), "skipped": []}
|
||||
if not lower.endswith(".zip"):
|
||||
raise ValueError("Only .zip (Agent Skills folder) or .md files can be imported")
|
||||
return self._import_zip(data)
|
||||
|
||||
def _import_zip(self, data: bytes) -> dict:
|
||||
try:
|
||||
zf = zipfile.ZipFile(io.BytesIO(data))
|
||||
except zipfile.BadZipFile as e:
|
||||
raise ValueError(f"Not a valid zip file: {e}") from e
|
||||
infos = [i for i in zf.infolist() if not i.is_dir()]
|
||||
if len(infos) > MAX_IMPORT_ENTRIES:
|
||||
raise ValueError(f"Too many files in the zip ({len(infos)} > {MAX_IMPORT_ENTRIES})")
|
||||
|
||||
def parts_of(info):
|
||||
p = PurePosixPath(info.filename.replace("\\", "/"))
|
||||
if p.is_absolute() or ".." in p.parts or any(":" in part for part in p.parts):
|
||||
raise ValueError(f"Unsafe path in zip: {info.filename}")
|
||||
return p.parts
|
||||
|
||||
all_parts = [(i, parts_of(i)) for i in infos]
|
||||
# Ignore OS junk (__MACOSX/, .DS_Store) when locating SKILL.md.
|
||||
all_parts = [(i, ps) for i, ps in all_parts if ps and ps[0] != "__MACOSX" and not ps[-1].startswith(".")]
|
||||
skill_md_parts = [ps for _, ps in all_parts if ps[-1] == SKILL_MD and len(ps) <= 2]
|
||||
if len(skill_md_parts) != 1:
|
||||
raise ValueError("The zip must contain exactly one SKILL.md at its root or in one top-level folder")
|
||||
prefix = skill_md_parts[0][:-1] # () or (top_folder,)
|
||||
|
||||
skill_text = zf.read("/".join(skill_md_parts[0])).decode("utf-8", errors="replace")
|
||||
meta = _parse_frontmatter(skill_text)
|
||||
base_name = prefix[0] if prefix else str(meta.get("name") or "imported-skill")
|
||||
folder = self._unique_name(_slugify(base_name), "")
|
||||
target = self.skills_dir / folder
|
||||
target.mkdir(parents=True)
|
||||
|
||||
written, skipped, total = 0, [], 0
|
||||
try:
|
||||
for info, ps in all_parts:
|
||||
if tuple(ps[: len(prefix)]) != prefix:
|
||||
skipped.append(info.filename)
|
||||
continue
|
||||
rel = PurePosixPath(*ps[len(prefix):])
|
||||
is_skill_md = rel.as_posix() == SKILL_MD
|
||||
ext = rel.suffix.lstrip(".").lower()
|
||||
if not is_skill_md and ext not in RESOURCE_EXTS:
|
||||
skipped.append(info.filename)
|
||||
continue
|
||||
if info.file_size > MAX_IMPORT_FILE_BYTES:
|
||||
skipped.append(info.filename)
|
||||
continue
|
||||
total += info.file_size
|
||||
if total > MAX_IMPORT_TOTAL_BYTES:
|
||||
raise ValueError("The zip's text files exceed 5 MB in total")
|
||||
content = zf.read(info)[:MAX_IMPORT_FILE_BYTES]
|
||||
out = (target / rel).resolve()
|
||||
if target.resolve() not in out.parents:
|
||||
raise ValueError(f"Unsafe path in zip: {info.filename}")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_bytes(content)
|
||||
written += 1
|
||||
except Exception:
|
||||
shutil.rmtree(target, ignore_errors=True)
|
||||
raise
|
||||
logger.info("[WFS] Imported skill folder %s (%d files, %d skipped)", folder, written, len(skipped))
|
||||
return {"skill": self._folder_skill_info(target), "skipped": skipped}
|
||||
|
||||
def _unique_name(self, stem: str, suffix: str) -> str:
|
||||
candidate = f"{stem}{suffix}"
|
||||
n = 2
|
||||
while (self.skills_dir / candidate).exists():
|
||||
candidate = f"{stem}-{n}{suffix}"
|
||||
n += 1
|
||||
return candidate
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Path safety
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _folder_path(self, folder: str) -> Path | None:
|
||||
name = (folder or "").strip()
|
||||
if not _FOLDER_RE.match(name) or name.startswith("."):
|
||||
return None
|
||||
return self.skills_dir / name
|
||||
|
||||
def _safe_path(self, filename: str) -> Path | None:
|
||||
"""Flat ``name.md`` or a folder skill's ``<folder>/SKILL.md`` - nothing else."""
|
||||
if not filename or not filename.strip():
|
||||
return None
|
||||
# Skills are a flat directory - reject any path separator (no subfolders).
|
||||
name = filename.strip()
|
||||
if "/" in name or "\\" in name or name in (".", ".."):
|
||||
name = filename.strip().replace("\\", "/")
|
||||
if "/" in name:
|
||||
folder, _, leaf = name.partition("/")
|
||||
if leaf != SKILL_MD:
|
||||
return None
|
||||
base = self._folder_path(folder)
|
||||
return base / SKILL_MD if base is not None else None
|
||||
if name in (".", "..") or name.startswith("."):
|
||||
return None
|
||||
ext = Path(name).suffix.lstrip(".").lower()
|
||||
if ext not in ALLOWED_EXTS:
|
||||
return None
|
||||
return self.skills_dir / name
|
||||
|
||||
|
||||
def _slugify(text: str) -> str:
|
||||
slug = re.sub(r"[^A-Za-z0-9]+", "-", str(text)).strip("-").lower()[:48]
|
||||
return slug or "imported-skill"
|
||||
|
||||
+109
-25
@@ -12,8 +12,10 @@ import threading
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from ..config import resolve_untrusted
|
||||
from typing import Optional
|
||||
from urllib.parse import urlparse
|
||||
from .url_guard import HINT as _URL_HINT, is_allowed_backend_url, urlopen as _guarded_urlopen
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image, ImageOps, PngImagePlugin
|
||||
@@ -22,19 +24,9 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
_model_lock = threading.Lock()
|
||||
|
||||
# Unsloth requires an Authorization: Bearer <UNSLOTH_API_KEY> header even for
|
||||
# local access. api_url is client-supplied per request, so without a host
|
||||
# check a client could point it at an arbitrary server and exfiltrate the key
|
||||
# (SSRF). Mirrors unsloth_routes.py's _is_allowed_base_url/_ALLOWED_HOSTS.
|
||||
_UNSLOTH_ALLOWED_HOSTS = {"localhost", "127.0.0.1", "::1"}
|
||||
|
||||
|
||||
def _is_allowed_unsloth_url(api_url: str) -> bool:
|
||||
try:
|
||||
parsed = urlparse(api_url)
|
||||
return parsed.scheme in ("http", "https") and parsed.hostname in _UNSLOTH_ALLOWED_HOSTS
|
||||
except Exception:
|
||||
return False
|
||||
# api_url is client-supplied per request, so every backend (Ollama / OpenAI-compatible / Unsloth)
|
||||
# goes through url_guard: loopback or WFS_ALLOWED_BACKEND_HOSTS only, no redirects. This also
|
||||
# stops the Unsloth API key (Authorization header) from being sent to an arbitrary server.
|
||||
|
||||
|
||||
class TaggerService:
|
||||
@@ -82,7 +74,10 @@ class TaggerService:
|
||||
continue
|
||||
exts = {f.suffix.lower() for f in item.iterdir() if f.is_file()}
|
||||
if ".onnx" in exts:
|
||||
models.append({"name": item.name, "type": "onnx"})
|
||||
entry = {"name": item.name, "type": "onnx"}
|
||||
if (item / "preprocess.json").exists() and (item / "selected_tags.csv").exists():
|
||||
entry["pixai"] = True
|
||||
models.append(entry)
|
||||
elif ".h5" in exts:
|
||||
models.append({"name": item.name, "type": "deepdanbooru"})
|
||||
return models
|
||||
@@ -138,6 +133,9 @@ class TaggerService:
|
||||
"input_shape": session.get_inputs()[0].shape,
|
||||
"type": "onnx",
|
||||
}
|
||||
categories = self._load_pixai_categories(model_dir, len(labels))
|
||||
if categories:
|
||||
data["pixai_categories"] = categories
|
||||
self._cache[name] = data
|
||||
logger.info("ONNX loaded: %s (%d labels)", name, len(labels))
|
||||
return data
|
||||
@@ -183,6 +181,20 @@ class TaggerService:
|
||||
logger.warning("Label load failed %s: %s", p, e)
|
||||
return []
|
||||
|
||||
@staticmethod
|
||||
def _load_pixai_categories(model_dir: Path, n_labels: int) -> Optional[list]:
|
||||
"""PixAI Tagger形式(preprocess.json + selected_tags.csvのcategory列)ならカテゴリ番号の配列を返す。"""
|
||||
if not (model_dir / "preprocess.json").exists():
|
||||
return None
|
||||
csv_path = model_dir / "selected_tags.csv"
|
||||
try:
|
||||
with open(csv_path, "r", encoding="utf-8-sig", newline="") as f:
|
||||
rows = list(csv.DictReader(f))
|
||||
cats = [int(r["category"]) for r in rows]
|
||||
except Exception:
|
||||
return None
|
||||
return cats if cats and len(cats) == n_labels else None
|
||||
|
||||
# ── 推論（WD Tagger / DeepDanbooru） ────────────────────
|
||||
|
||||
def predict(self, image_b64: str, model_name: str, threshold: float = 0.35, char_threshold: float = 0.85) -> dict:
|
||||
@@ -200,7 +212,74 @@ class TaggerService:
|
||||
logger.error("predict error: %s", e)
|
||||
return {"error": str(e)}
|
||||
|
||||
# WD Tagger の評価(rating)ラベル。TagSpace の補完では除く
|
||||
_RATING_LABELS = {"general", "sensitive", "questionable", "explicit"}
|
||||
|
||||
def tag_image(self, img: Image.Image, model_name: str) -> list:
|
||||
"""他機能(TagSpace)向け: ONNXモデルで画像1枚にタグを付ける。しきい値は既定値
|
||||
(WD 0.35/0.85、PixAIは推奨値)。評価ラベルは除く。読み込めなければ RuntimeError。"""
|
||||
data = self._load_model(model_name)
|
||||
if data is None or data.get("type") != "onnx":
|
||||
raise RuntimeError(f"Model '{model_name}' could not be loaded")
|
||||
if data.get("pixai_categories"):
|
||||
thr, char_thr = self._PIXAI_BASE["general"], self._PIXAI_BASE["character"]
|
||||
else:
|
||||
thr, char_thr = 0.35, 0.85
|
||||
return [t for t in self._predict_onnx(img, data, thr, char_thr) if t not in self._RATING_LABELS]
|
||||
|
||||
def model_provider(self, model_name: str) -> str:
|
||||
"""読み込み済みモデルの実行先(CUDA / CPU)。未読み込みなら空文字"""
|
||||
data = self._cache.get(model_name)
|
||||
try:
|
||||
return data["session"].get_providers()[0].replace("ExecutionProvider", "") if data else ""
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
# PixAI Tagger v1.0 推奨しきい値(category番号 -> しきい値)。meta/ratingはタグに含めない。
|
||||
# UIの2本のスライダーは推奨値(general 0.17 / character 0.27)を基準にした相対指定で、
|
||||
# general側(0,1)はthreshold、character側(3,4)はchar_thresholdに追従する。
|
||||
_PIXAI_THRESHOLDS = {0: 0.17, 1: 0.15, 3: 0.24, 4: 0.27}
|
||||
_PIXAI_BASE = {"general": 0.17, "character": 0.27}
|
||||
_PIXAI_SIZE = 1008
|
||||
|
||||
def _predict_pixai(self, img: Image.Image, data: dict, thr: float, char_thr: float) -> list:
|
||||
"""PixAI Tagger v1.0: 白背景合成 → 黒パディングで1008角 → RGB, (x/255-0.5)/0.5, NCHW。出力はsigmoid済み確率。"""
|
||||
size = self._PIXAI_SIZE
|
||||
img = ImageOps.exif_transpose(img)
|
||||
if img.mode != "RGBA":
|
||||
img = img.convert("RGBA")
|
||||
canvas = Image.new("RGBA", img.size, (255, 255, 255, 255))
|
||||
canvas.alpha_composite(img)
|
||||
img = canvas.convert("RGB")
|
||||
|
||||
scale = size / max(img.size)
|
||||
w, h = max(1, round(img.width * scale)), max(1, round(img.height * scale))
|
||||
img = img.resize((w, h), Image.Resampling.BICUBIC)
|
||||
padded = Image.new("RGB", (size, size), (0, 0, 0))
|
||||
padded.paste(img, ((size - w) // 2, (size - h) // 2))
|
||||
|
||||
arr = (np.asarray(padded, dtype=np.float32) / 255.0 - 0.5) / 0.5
|
||||
arr = np.ascontiguousarray(arr.transpose(2, 0, 1)[None])
|
||||
preds = data["session"].run(None, {data["input_name"]: arr})[0][0]
|
||||
|
||||
d_gen = thr - self._PIXAI_BASE["general"]
|
||||
d_chr = char_thr - self._PIXAI_BASE["character"]
|
||||
limits = {
|
||||
c: max(0.0, min(1.0, base + (d_gen if c in (0, 1) else d_chr)))
|
||||
for c, base in self._PIXAI_THRESHOLDS.items()
|
||||
}
|
||||
labels, cats = data["labels"], data["pixai_categories"]
|
||||
picked = [
|
||||
(float(score), labels[i])
|
||||
for i, score in enumerate(preds)
|
||||
if i < len(labels) and float(score) >= limits.get(cats[i], 2.0)
|
||||
]
|
||||
picked.sort(key=lambda t: t[0], reverse=True)
|
||||
return [name for _, name in picked]
|
||||
|
||||
def _predict_onnx(self, img: Image.Image, data: dict, thr: float, char_thr: float) -> list:
|
||||
if data.get("pixai_categories"):
|
||||
return self._predict_pixai(img, data, thr, char_thr)
|
||||
shape = data["input_shape"]
|
||||
fmt = "NHWC"
|
||||
size = 224
|
||||
@@ -279,22 +358,22 @@ class TaggerService:
|
||||
def vlm_models(self, backend: str, api_url: str) -> list:
|
||||
backend = (backend or "ollama").lower()
|
||||
try:
|
||||
if not is_allowed_backend_url(api_url):
|
||||
logger.error("vlm_models: %s", _URL_HINT)
|
||||
return []
|
||||
if backend == "ollama":
|
||||
req = urllib.request.Request(api_url.rstrip("/") + "/api/tags")
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
with _guarded_urlopen(req, timeout=5) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
return sorted(m["name"] for m in data.get("models", []))
|
||||
|
||||
headers = {}
|
||||
if backend == "unsloth":
|
||||
if not _is_allowed_unsloth_url(api_url):
|
||||
logger.error("vlm_models: Unsloth backend URL must point to localhost/127.0.0.1/::1")
|
||||
return []
|
||||
key = self._unsloth_api_key()
|
||||
if key:
|
||||
headers["Authorization"] = f"Bearer {key}"
|
||||
req = urllib.request.Request(api_url.rstrip("/") + "/v1/models", headers=headers)
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
with _guarded_urlopen(req, timeout=5) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
return sorted(m["id"] for m in data.get("data", []))
|
||||
except Exception as e:
|
||||
@@ -309,6 +388,8 @@ class TaggerService:
|
||||
thinking_mode: bool = False, max_tokens: int = 0) -> dict:
|
||||
backend = (backend or "ollama").lower()
|
||||
try:
|
||||
if not is_allowed_backend_url(api_url):
|
||||
return {"error": _URL_HINT}
|
||||
if backend == "ollama":
|
||||
body = {
|
||||
"model": model,
|
||||
@@ -326,14 +407,12 @@ class TaggerService:
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=180) as resp:
|
||||
with _guarded_urlopen(req, timeout=180) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
content = data.get("message", {}).get("content", "")
|
||||
else:
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if backend == "unsloth":
|
||||
if not _is_allowed_unsloth_url(api_url):
|
||||
return {"error": "Unsloth backend URL must point to localhost/127.0.0.1/::1"}
|
||||
key = self._unsloth_api_key()
|
||||
if key:
|
||||
headers["Authorization"] = f"Bearer {key}"
|
||||
@@ -354,7 +433,7 @@ class TaggerService:
|
||||
headers=headers,
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=180) as resp:
|
||||
with _guarded_urlopen(req, timeout=180) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
message = data.get("choices", [{}])[0].get("message", {})
|
||||
content = message.get("content", "")
|
||||
@@ -379,7 +458,12 @@ class TaggerService:
|
||||
_ALLOWED_WRITE_EXTS = {".jpg", ".jpeg", ".png", ".webp", ".bmp", ".gif"}
|
||||
|
||||
def write_meta_to_file(self, image_path: str, tags: str) -> dict:
|
||||
path = Path(image_path).resolve()
|
||||
try:
|
||||
# UNC(\\host\share)は resolve() した時点で相手ホストへ資格情報付きで
|
||||
# 接続してしまうため、文字列のまま先に拒否する
|
||||
path = resolve_untrusted(image_path)
|
||||
except ValueError:
|
||||
return {"error": "Access denied"}
|
||||
if path.suffix.lower() not in self._ALLOWED_WRITE_EXTS:
|
||||
return {"error": "Unsupported file type"}
|
||||
if not path.is_file():
|
||||
|
||||
@@ -0,0 +1,686 @@
|
||||
"""
|
||||
TagSpace Service - WFSのデータを EmbeddingGemma 2 の同じベクトル空間に置き、3Dに投影する
|
||||
|
||||
カード : ImagePrompt(ws_image_prompt)の画像+プロンプト。ベクトルは Gallery セマンティック索引の画像ベクトル
|
||||
タグ : ImagePrompt/ワイルドカード/Tagger DB/Promptライブラリの文をカンマ等で分解した語。Ollamaでテキスト埋め込み
|
||||
背景 : Gallery 索引から間引いた画像(位置関係を見るための点群)
|
||||
|
||||
画像とテキストのベクトルは同じ空間でも別の塊に分かれる(modality gap)ため、
|
||||
画像群とテキスト群をそれぞれの平均で中心化してからPCAで3軸を求める。
|
||||
タグのベクトルはタグ文字列をキーに tagspace_tags.npz へキャッシュする(740m/440mは同じベクトル)。
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import sqlite3
|
||||
import threading
|
||||
import time
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import yaml
|
||||
|
||||
from . import semantic_ollama as ollama
|
||||
from .semantic_service import LOCAL_MODEL, SemanticError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_MAX_TAGS = 2000
|
||||
MAX_TAGS_LIMIT = 6000
|
||||
ALL_TAGS_LIMIT = 20000 # 「全タグ」(max_tags=0)でも超えない上限(ワイルドカード込み・出現1回以上だと約5万種になるため)
|
||||
MIN_COUNT_LIMIT = 10
|
||||
DEFAULT_BG = 4000
|
||||
BG_LIMIT = 20000
|
||||
_EMBED_BATCH = 32
|
||||
_RADIUS = 10.0 # 3D座標の広がり(98パーセンタイル半径)
|
||||
_IMG_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
_VAULT_THUMB_DIRS = {"thumbnails", "thumbnails_option2"}
|
||||
SOURCES = ("image_prompt", "gallery", "tagger_auto", "tagger", "prompt_library", "wildcard")
|
||||
TAGGER_SCOPES = ("background", "all") # Tagger補完の対象: 背景の点のみ / プロンプトの無い全画像
|
||||
_TAGGER_MAX_SIDE = 1024 # Taggerに渡す前の長辺上限(PixAIの1008pxより少し大きく)
|
||||
_TAGGER_SAVE_EVERY = 100
|
||||
_PROMPT_MAX = 4000 # キャッシュするプロンプトの最大文字数
|
||||
# Positive をたどるときに文字列として拾う入力名(CLIPTextEncode系・Qwen/Flux系のエンコーダー・文字列ノード)
|
||||
_TEXT_INPUTS = {"text", "prompt", "positive", "text_g", "text_l", "clip_l", "t5xxl", "string", "value",
|
||||
"user_prompt", "system_prompt_text"}
|
||||
_SKIP_INPUTS = {"negative", "negative_prompt", "neg"}
|
||||
|
||||
_SPLIT = re.compile(r"[,\n|{}]+|\bBREAK\b")
|
||||
_WEIGHT = re.compile(r":\s*-?\d+(?:\.\d+)?\s*$")
|
||||
_DP_WEIGHT = re.compile(r"^\s*\d+(?:\.\d+)?\s*::") # Dynamic Prompts の "2::option"
|
||||
_JSON_KEY = re.compile(r'"\s*:') # "key": の形
|
||||
_MODEL_FILE = re.compile(r"\.(safetensors|ckpt|pt|pth|bin|gguf)\b", re.I)
|
||||
_HAS_WORD = re.compile(r"[a-z-ヿ一-鿿]")
|
||||
_STOP = {"break", "and", "or", "the", "a", "an"}
|
||||
|
||||
|
||||
def normalize_tag(raw: str) -> str:
|
||||
"""プロンプトの断片をタグとして正規化する。タグにならなければ空文字"""
|
||||
s = _DP_WEIGHT.sub("", raw).strip()
|
||||
if not s or "<" in s or "__" in s or s.lower().startswith(("embedding:", "rating:")):
|
||||
return ""
|
||||
# プロンプトに埋め込まれたJSON(MOSS-Image のレイアウト指定 bbox / 文字入れ text 等)の断片と、モデルのファイル名
|
||||
if _JSON_KEY.search(s) or _MODEL_FILE.search(s):
|
||||
return ""
|
||||
s = s.replace('"', "") # 分割で片側だけ残った引用符(blue" / "pink)
|
||||
s = s.replace("\\(", "\x00").replace("\\)", "\x01") # danbooruのエスケープ括弧は残す
|
||||
for _ in range(3):
|
||||
while len(s) > 1 and s[0] in "([" and s[-1] in ")]":
|
||||
s = s[1:-1].strip()
|
||||
s = _WEIGHT.sub("", s).strip(" \"'.;")
|
||||
if s.count("(") != s.count(")") or s.count("[") != s.count("]"):
|
||||
s = re.sub(r"[()\[\]]", "", s).strip() # 分割で片側だけ残った括弧
|
||||
s = s.replace("\x00", "(").replace("\x01", ")").replace("_", " ")
|
||||
s = re.sub(r"\s+", " ", s).lower()
|
||||
if len(s) < 2 or len(s) > 40 or len(s.split()) > 5 or s in _STOP or not _HAS_WORD.search(s):
|
||||
return ""
|
||||
if s.startswith(("embedding:", "rating:")): # (embedding:xxx:1) のように括弧を外してから現れる形
|
||||
return ""
|
||||
return s
|
||||
|
||||
|
||||
def split_tags(text: str) -> list[str]:
|
||||
return [t for t in (normalize_tag(p) for p in _SPLIT.split(text or "")) if t]
|
||||
|
||||
|
||||
# C実装があれば使う(4万行のワイルドカードYAMLで 1.9秒 → 0.2秒)。どちらも任意のPythonオブジェクトは作らない
|
||||
_YAML_LOADER = getattr(yaml, "CSafeLoader", yaml.SafeLoader)
|
||||
|
||||
|
||||
def _yaml_leaves(node):
|
||||
"""YAMLを読んだ値から文字列の葉だけを取り出す(キーはカテゴリ名なので除く。
|
||||
yes/no 等が真偽値・数値に変換された葉はタグにならないので除く)"""
|
||||
if isinstance(node, dict):
|
||||
for v in node.values():
|
||||
yield from _yaml_leaves(v)
|
||||
elif isinstance(node, list):
|
||||
for v in node:
|
||||
yield from _yaml_leaves(v)
|
||||
elif isinstance(node, str):
|
||||
yield node
|
||||
|
||||
|
||||
def _wildcard_yaml_texts(text: str):
|
||||
"""ワイルドカードYAMLの項目。`- >-` 等のブロックスカラー(複数行)やエスケープもYAMLとして解釈する。
|
||||
YAMLとして読めないファイルは、従来どおり "- " で始まる行を1項目として読む"""
|
||||
try:
|
||||
yield from _yaml_leaves(yaml.load(text, Loader=_YAML_LOADER))
|
||||
except yaml.YAMLError:
|
||||
for ln in text.splitlines():
|
||||
s = ln.strip()
|
||||
if s.startswith("- "):
|
||||
yield s[2:].strip().strip("\"'")
|
||||
|
||||
|
||||
def _upstream_texts(graph: dict, node_id: str, seen: set) -> list[str]:
|
||||
"""API形式グラフで node_id から上流へたどり、プロンプト用の文字列入力を集める(negative側はたどらない)"""
|
||||
if node_id in seen or node_id not in graph:
|
||||
return []
|
||||
seen.add(node_id)
|
||||
out = []
|
||||
for key, val in ((graph[node_id] or {}).get("inputs") or {}).items():
|
||||
lk = key.lower()
|
||||
if lk in _SKIP_INPUTS:
|
||||
continue
|
||||
if isinstance(val, str):
|
||||
if lk in _TEXT_INPUTS and val.strip():
|
||||
out.append(val.strip())
|
||||
elif isinstance(val, list) and len(val) == 2 and isinstance(val[0], (str, int)):
|
||||
out += _upstream_texts(graph, str(val[0]), seen)
|
||||
return out
|
||||
|
||||
|
||||
def positive_prompt(graph: dict) -> str:
|
||||
"""API形式グラフの Positive プロンプト。サンプラー/ガイダーの positive(無ければ BasicGuider等の conditioning)から上流をたどる"""
|
||||
if not isinstance(graph, dict) or "nodes" in graph:
|
||||
return ""
|
||||
texts = []
|
||||
for node in graph.values():
|
||||
ins = (node or {}).get("inputs") if isinstance(node, dict) else None
|
||||
if not isinstance(ins, dict):
|
||||
continue
|
||||
link = ins.get("positive")
|
||||
if not isinstance(link, list) and "negative" not in ins:
|
||||
link = ins.get("conditioning") if "guider" in str(node.get("class_type", "")).lower() else None
|
||||
if isinstance(link, list) and link:
|
||||
texts += _upstream_texts(graph, str(link[0]), set())
|
||||
return ", ".join(dict.fromkeys(texts))[:_PROMPT_MAX]
|
||||
|
||||
|
||||
class TagSpaceService:
|
||||
def __init__(self, data_dir: Path, gallery_service, semantic_service, tagger_service=None):
|
||||
self.data_dir = data_dir
|
||||
self.gallery = gallery_service
|
||||
self.semantic = semantic_service
|
||||
self.tagger = tagger_service
|
||||
self._tag_file = data_dir / "tagspace_tags.npz"
|
||||
self._space_file = data_dir / "tagspace_space.json"
|
||||
self._mat_file = data_dir / "tagspace_space.npz"
|
||||
self._prompt_file = data_dir / "tagspace_prompts.json" # Gallery画像の Positive {path: [size, mtime, text]}
|
||||
self._tagger_file = data_dir / "tagspace_tagger.json" # Tagger補完 {path: [size, mtime, model, tags]}
|
||||
self._lock = threading.RLock()
|
||||
self._tag_cache: dict[str, np.ndarray] | None = None # tag -> 768次元(正規化済み)
|
||||
self._space: dict | None = None # 3D表示用データ(JSON)
|
||||
self._mat: dict | None = None # 近傍/検索用 {ids, vecs, kinds, mean_img, mean_txt, axes, scale}
|
||||
self._job = self._idle_job()
|
||||
self._cancel = threading.Event()
|
||||
|
||||
@staticmethod
|
||||
def _idle_job() -> dict:
|
||||
return {"running": False, "phase": "", "done": 0, "total": 0, "note": "", "error": "", "error_code": "",
|
||||
"finished": False, "cancelled": False}
|
||||
|
||||
# ── 状態 ───────────────────────────────────────────────
|
||||
|
||||
def status(self) -> dict:
|
||||
space = self._load_space()
|
||||
meta = (space or {}).get("meta") or {}
|
||||
return {"job": dict(self._job), "built": bool(space), "meta": meta, "tagger_models": self.tagger_models(),
|
||||
"index": self.semantic.index_info(), "ollama": ollama.status(),
|
||||
"semantic_job": self.semantic.job_status()}
|
||||
|
||||
def space(self) -> dict:
|
||||
space = self._load_space()
|
||||
if not space:
|
||||
raise SemanticError("tagspace_not_built")
|
||||
return space
|
||||
|
||||
def _load_space(self) -> dict | None:
|
||||
with self._lock:
|
||||
if self._space is None and self._space_file.exists():
|
||||
try:
|
||||
self._space = json.loads(self._space_file.read_text(encoding="utf-8"))
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load space: %s", e)
|
||||
return self._space
|
||||
|
||||
def _load_mat(self) -> dict:
|
||||
with self._lock:
|
||||
if self._mat is None:
|
||||
if not self._mat_file.exists():
|
||||
raise SemanticError("tagspace_not_built")
|
||||
with np.load(self._mat_file, allow_pickle=False) as z:
|
||||
self._mat = {k: z[k] for k in z.files}
|
||||
m = self._mat
|
||||
m["ids"] = [str(x) for x in m["ids"]]
|
||||
m["pos"] = {i: n for n, i in enumerate(m["ids"])}
|
||||
m["bg_ids"] = [str(x) for x in m["bg_ids"]] if "bg_ids" in m else []
|
||||
m["bg_pos"] = {i: n for n, i in enumerate(m["bg_ids"])}
|
||||
# 中心化して正規化したブロック(カード/タグ/背景画像)。近傍・検索のたびに作り直さない
|
||||
n_img = int(m["n_img"])
|
||||
m["blocks"] = {
|
||||
"card": (m["ids"][:n_img], self._centered(m["vecs"][:n_img], m["mean_img"])),
|
||||
"tag": (m["ids"][n_img:], self._centered(m["vecs"][n_img:], m["mean_txt"])),
|
||||
"image": (m["bg_ids"], self._centered(m["bg_vecs"], m["mean_img"]) if "bg_vecs" in m
|
||||
else np.zeros((0, m["vecs"].shape[1]), np.float32)),
|
||||
}
|
||||
return self._mat
|
||||
|
||||
# ── 材料の収集 ─────────────────────────────────────────
|
||||
|
||||
def _cards(self) -> tuple[list[dict], list]:
|
||||
"""ImagePromptの画像のうちプロンプトを持つもの。戻り値: (カード, スキャン結果)"""
|
||||
root = self.gallery.get_image_prompt_root()
|
||||
if not root:
|
||||
return [], []
|
||||
base = Path(root).resolve()
|
||||
entries = [e for e in self.gallery._scan_folder_recursive(base) if Path(e[1]).suffix.lower() in _IMG_EXTS]
|
||||
cards, used = [], []
|
||||
for name, ap, size, mtime in entries:
|
||||
prompt = self.gallery._read_sidecar_prompt(Path(ap))
|
||||
if not prompt:
|
||||
continue
|
||||
rel = Path(ap).resolve().relative_to(base).parts[:-1]
|
||||
folders = [p for p in rel if p.lower() not in _VAULT_THUMB_DIRS]
|
||||
# tags: 詳細パネルでタグを選んで空間検索するための、プロンプトを分解したタグ(Gallery点の tags と同じ正規化)
|
||||
cards.append({"id": ap, "kind": "card", "label": name.split(".")[0], "prompt": prompt.strip(),
|
||||
"tags": list(dict.fromkeys(split_tags(prompt))),
|
||||
"category": "/".join(folders[-2:]) or "(root)", "group": folders[0] if folders else "(root)"})
|
||||
used.append((name, ap, size, mtime))
|
||||
return cards, used
|
||||
|
||||
def _wildcard_texts(self):
|
||||
folder = self.data_dir / "wildcard"
|
||||
if not folder.is_dir():
|
||||
return
|
||||
for p in sorted(folder.rglob("*")):
|
||||
if p.suffix.lower() == ".txt":
|
||||
lines = p.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||
yield from (ln for ln in lines if ln.strip() and not ln.lstrip().startswith("#"))
|
||||
elif p.suffix.lower() in (".yaml", ".yml"):
|
||||
yield from _wildcard_yaml_texts(p.read_text(encoding="utf-8", errors="replace"))
|
||||
|
||||
def _tagger_texts(self):
|
||||
db = self.data_dir / "tagger.db"
|
||||
if not db.exists():
|
||||
return
|
||||
try:
|
||||
with sqlite3.connect(f"file:{db}?mode=ro", uri=True) as conn:
|
||||
for (text,) in conn.execute("SELECT all_tags FROM images"):
|
||||
if text:
|
||||
yield text
|
||||
except sqlite3.Error as e:
|
||||
logger.warning("TagSpace: tagger.db read failed: %s", e)
|
||||
|
||||
def _library_texts(self):
|
||||
f = self.data_dir / "prompts.json"
|
||||
try:
|
||||
items = json.loads(f.read_text(encoding="utf-8")) if f.exists() else []
|
||||
except Exception:
|
||||
items = []
|
||||
for it in items if isinstance(items, list) else []:
|
||||
if isinstance(it, dict) and it.get("text"):
|
||||
yield str(it["text"])
|
||||
|
||||
def tagger_models(self) -> list[str]:
|
||||
"""Tagger補完に使えるモデル(ONNX)"""
|
||||
if self.tagger is None:
|
||||
return []
|
||||
try:
|
||||
return [m["name"] for m in self.tagger.list_models() if m.get("type") == "onnx"]
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
def _tagger_fill(self, targets: list[tuple[str, int, float]], model: str) -> dict[str, str]:
|
||||
"""targets([(path, size, mtime)])を Tagger でタグ付けする。同じ画像・同じモデルならキャッシュを使う。
|
||||
中止しても、それまでの結果はキャッシュに残す。戻り値: {path: "tag, tag, ..."}"""
|
||||
from PIL import Image
|
||||
try:
|
||||
cache = json.loads(self._tagger_file.read_text(encoding="utf-8")) if self._tagger_file.exists() else {}
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load tagger cache: %s", e)
|
||||
cache = {}
|
||||
job = self._job
|
||||
job.update(phase="tagger", done=0, total=len(targets), note="")
|
||||
out: dict[str, str] = {}
|
||||
dirty = 0
|
||||
was_loaded = model in getattr(self.tagger, "_cache", {})
|
||||
|
||||
def save():
|
||||
tmp = self._tagger_file.with_suffix(".tmp.json")
|
||||
tmp.write_text(json.dumps(cache, ensure_ascii=False), encoding="utf-8")
|
||||
tmp.replace(self._tagger_file)
|
||||
|
||||
try:
|
||||
for n, (path, size, mtime) in enumerate(targets):
|
||||
if self._cancel.is_set():
|
||||
break
|
||||
hit = cache.get(path)
|
||||
if hit and hit[0] == size and abs(hit[1] - mtime) < 1e-3 and hit[2] == model:
|
||||
text = hit[3]
|
||||
else:
|
||||
try:
|
||||
with Image.open(path) as im:
|
||||
im.seek(0)
|
||||
im = im.convert("RGBA") if im.mode in ("RGBA", "LA", "P") else im.convert("RGB")
|
||||
im.thumbnail((_TAGGER_MAX_SIDE, _TAGGER_MAX_SIDE), Image.LANCZOS)
|
||||
text = ", ".join(self.tagger.tag_image(im, model))
|
||||
except RuntimeError as e: # モデルを読み込めない: 続けても全件失敗する
|
||||
raise SemanticError("tagspace_tagger_model", str(e))
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: tagger failed for %s: %s", path, e)
|
||||
text = ""
|
||||
cache[path] = [size, mtime, model, text]
|
||||
dirty += 1
|
||||
if not job["note"]:
|
||||
job["note"] = self.tagger.model_provider(model)
|
||||
if dirty >= _TAGGER_SAVE_EVERY:
|
||||
save()
|
||||
dirty = 0
|
||||
if text:
|
||||
out[path] = text
|
||||
job["done"] = n + 1
|
||||
finally:
|
||||
if dirty:
|
||||
save()
|
||||
if not was_loaded: # 補完のために読み込んだモデルはメモリから外す(Taggerタブで使用中のものは残す)
|
||||
getattr(self.tagger, "_cache", {}).pop(model, None)
|
||||
return out
|
||||
|
||||
def _gallery_prompts(self, exclude_prefix: str) -> dict[str, str]:
|
||||
"""索引済み Gallery 画像の Positive プロンプト。(size, mtime)が同じならキャッシュを使う"""
|
||||
try:
|
||||
cache = json.loads(self._prompt_file.read_text(encoding="utf-8")) if self._prompt_file.exists() else {}
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load prompt cache: %s", e)
|
||||
cache = {}
|
||||
entries = [e for e in self.semantic.indexed_entries(exclude_prefix)
|
||||
if Path(e[0]).suffix.lower() in _IMG_EXTS]
|
||||
job = self._job
|
||||
job.update(phase="prompts", done=0, total=len(entries))
|
||||
new_cache, out = {}, {}
|
||||
for n, (path, size, mtime) in enumerate(entries):
|
||||
if self._cancel.is_set():
|
||||
break
|
||||
if not Path(path).is_file():
|
||||
continue # 索引作成後に削除された画像(キャッシュにも残さない)
|
||||
hit = cache.get(path)
|
||||
if hit and hit[0] == size and abs(hit[1] - mtime) < 1e-3:
|
||||
text = hit[2]
|
||||
else:
|
||||
try:
|
||||
text = positive_prompt(self.gallery._extract_embedded_workflow(path, prefer="prompt"))
|
||||
except Exception:
|
||||
text = ""
|
||||
new_cache[path] = [size, mtime, text]
|
||||
if text:
|
||||
out[path] = text
|
||||
if n % 200 == 0:
|
||||
job["done"] = n
|
||||
job["done"] = len(entries)
|
||||
if not self._cancel.is_set(): # 中止時は途中までの結果で既存のキャッシュを上書きしない
|
||||
tmp = self._prompt_file.with_suffix(".tmp.json")
|
||||
tmp.write_text(json.dumps(new_cache, ensure_ascii=False), encoding="utf-8")
|
||||
tmp.replace(self._prompt_file)
|
||||
return out
|
||||
|
||||
def collect_tags(self, cards: list[dict], max_tags: int, gallery_texts=(), tagger_texts=(),
|
||||
min_count: int = 1, wildcard: bool = True) -> tuple[list[dict], int]:
|
||||
"""全ソースのタグを数えて上位 max_tags 件を返す(max_tags=0 は全件、ただし ALL_TAGS_LIMIT まで)。
|
||||
ソースごとに出現数の多い順に並べ、ソースを順番に1件ずつ取る(数の多いGalleryだけで埋まらないように)。
|
||||
出現数は「そのタグを含む文の数」で数え、全出典の合計が min_count 未満のタグは候補から外す。
|
||||
wildcard=False ならワイルドカードは材料にしない(ワイルドカードにしか無い語がタグの種類の大半を占めるため)。
|
||||
戻り値: (タグ, 候補数)"""
|
||||
per_src: dict[str, Counter] = {s: Counter() for s in SOURCES}
|
||||
feeds = {"image_prompt": (c["prompt"] for c in cards), "gallery": iter(gallery_texts),
|
||||
"tagger_auto": iter(tagger_texts), "tagger": self._tagger_texts(), "prompt_library": self._library_texts(),
|
||||
"wildcard": self._wildcard_texts() if wildcard else iter(())}
|
||||
for src, texts in feeds.items():
|
||||
for text in texts:
|
||||
per_src[src].update(set(split_tags(text)))
|
||||
total = Counter()
|
||||
for c in per_src.values():
|
||||
total.update(c)
|
||||
ok = {t for t, n in total.items() if n >= min_count}
|
||||
if not max_tags:
|
||||
max_tags = ALL_TAGS_LIMIT
|
||||
queues = {s: sorted((t for t in c if t in ok), key=lambda t, c=c: (-c[t], t)) for s, c in per_src.items()}
|
||||
queues = {s: q for s, q in queues.items() if q}
|
||||
picked: list[str] = []
|
||||
seen: set = set()
|
||||
pos = dict.fromkeys(queues, 0)
|
||||
while len(picked) < max_tags and queues:
|
||||
for src in list(queues):
|
||||
q, i = queues[src], pos[src]
|
||||
while i < len(q) and q[i] in seen:
|
||||
i += 1
|
||||
if i >= len(q):
|
||||
del queues[src]
|
||||
continue
|
||||
pos[src] = i + 1
|
||||
seen.add(q[i])
|
||||
picked.append(q[i])
|
||||
if len(picked) >= max_tags:
|
||||
break
|
||||
return [{"id": f"tag:{t}", "kind": "tag", "label": t,
|
||||
"count": total[t],
|
||||
"sources": [s for s in SOURCES if per_src[s][t]]} for t in picked], len(ok)
|
||||
|
||||
# ── タグベクトルのキャッシュ ───────────────────────────
|
||||
|
||||
def _tag_vectors(self) -> dict[str, np.ndarray]:
|
||||
with self._lock:
|
||||
if self._tag_cache is None:
|
||||
self._tag_cache = {}
|
||||
if self._tag_file.exists():
|
||||
try:
|
||||
with np.load(self._tag_file, allow_pickle=False) as z:
|
||||
self._tag_cache = dict(zip((str(t) for t in z["tags"]), z["vecs"]))
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load tag cache: %s", e)
|
||||
return self._tag_cache
|
||||
|
||||
def _save_tag_vectors(self) -> None:
|
||||
with self._lock:
|
||||
cache = self._tag_cache or {}
|
||||
if not cache:
|
||||
return
|
||||
tags = list(cache)
|
||||
tmp = self._tag_file.with_suffix(".tmp.npz")
|
||||
np.savez(tmp, tags=np.array(tags), vecs=np.stack([cache[t] for t in tags]).astype(np.float32))
|
||||
tmp.replace(self._tag_file)
|
||||
|
||||
# ── 構築ジョブ ─────────────────────────────────────────
|
||||
|
||||
def start_build(self, max_tags: int, bg: int, tagger_model: str = "", tagger_scope: str = "background",
|
||||
min_count: int = 1, wildcard: bool = True) -> None:
|
||||
"""max_tags=0 は全タグ(ALL_TAGS_LIMIT まで)"""
|
||||
if tagger_model and tagger_model not in self.tagger_models():
|
||||
raise SemanticError("tagspace_tagger_model", tagger_model)
|
||||
if tagger_scope not in TAGGER_SCOPES:
|
||||
tagger_scope = TAGGER_SCOPES[0]
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
info = self.semantic.index_info()
|
||||
if info["count"] and info["model"] != LOCAL_MODEL:
|
||||
raise SemanticError("tagspace_index_model", info["model"])
|
||||
st = ollama.status()
|
||||
if not st["reachable"]:
|
||||
raise SemanticError("ollama_unreachable", f"{st['url']}: {st['error']}")
|
||||
if not st["ready"]:
|
||||
raise SemanticError("ollama_model_missing", st["tag"])
|
||||
self._cancel.clear()
|
||||
self._job = self._idle_job()
|
||||
self._job.update(running=True, phase="cards")
|
||||
max_tags = 0 if int(max_tags) == 0 else max(50, min(int(max_tags), MAX_TAGS_LIMIT))
|
||||
args = (max_tags, max(0, min(int(bg), BG_LIMIT)), tagger_model, tagger_scope,
|
||||
max(1, min(int(min_count), MIN_COUNT_LIMIT)), bool(wildcard))
|
||||
threading.Thread(target=self._run_build, args=args, daemon=True).start()
|
||||
|
||||
def cancel(self) -> None:
|
||||
self._cancel.set()
|
||||
if self._job.get("phase") == "index":
|
||||
self.semantic.cancel()
|
||||
|
||||
def _run_build(self, max_tags: int, bg: int, tagger_model: str = "", tagger_scope: str = "background",
|
||||
min_count: int = 1, wildcard: bool = True) -> None:
|
||||
job = self._job
|
||||
try:
|
||||
cards, entries = self._cards()
|
||||
root = self.gallery.get_image_prompt_root() or ""
|
||||
|
||||
# 1. カード画像を Gallery 索引に追加(未索引/更新分のみ)
|
||||
if entries and not self._cancel.is_set():
|
||||
job.update(phase="index", done=0, total=0)
|
||||
info = self.semantic.index_info()
|
||||
dim = info["dim"] or ollama.DEFAULT_LOCAL_DIM
|
||||
self.semantic.start_index(root, True, "missing", LOCAL_MODEL, dim)
|
||||
while True:
|
||||
sj = self.semantic.job_status()
|
||||
job.update(done=sj["done"] + sj["failed"], total=sj["total"])
|
||||
if not sj["running"]:
|
||||
break
|
||||
time.sleep(0.5)
|
||||
if sj.get("error_code") in ("ollama_unreachable", "ollama_model_missing"):
|
||||
raise SemanticError(sj["error_code"], sj["error"])
|
||||
card_vecs = self.semantic.vectors_for(entries)
|
||||
cards = [c for c in cards if c["id"] in card_vecs]
|
||||
dim = self.semantic.index_info()["dim"] or ollama.DEFAULT_LOCAL_DIM
|
||||
|
||||
# 2. Gallery 画像の Positive プロンプト(タグの材料と、背景点のタグ)
|
||||
prefix = Path(root).resolve().as_posix().rstrip("/") + "/" if root else ""
|
||||
gallery_prompts = self._gallery_prompts(prefix)
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
|
||||
# 3. 背景の Gallery 画像(Tagger補完の対象を決めるため、タグ集めより先に選ぶ)
|
||||
bg_paths, bg_mat = self.semantic.sample_vectors(bg + bg // 4, exclude_prefix=prefix) if bg else ([], None)
|
||||
if bg_paths: # 索引作成後に削除された画像を除く(多めに取って bg 件に詰める)
|
||||
keep = [i for i, p in enumerate(bg_paths) if Path(p).is_file()][:bg]
|
||||
bg_paths, bg_mat = [bg_paths[i] for i in keep], bg_mat[keep]
|
||||
|
||||
# 4. Tagger補完: プロンプト由来のタグが無い画像
|
||||
tagger_tags: dict[str, str] = {}
|
||||
if tagger_model:
|
||||
stamps = {p: (sz, mt) for p, sz, mt in self.semantic.indexed_entries(prefix)}
|
||||
pool = bg_paths if tagger_scope == "background" else [
|
||||
p for p in stamps if Path(p).suffix.lower() in _IMG_EXTS and Path(p).is_file()]
|
||||
targets = [(p, *stamps[p]) for p in pool
|
||||
if p in stamps and Path(p).suffix.lower() in _IMG_EXTS
|
||||
and not split_tags(gallery_prompts.get(p, ""))]
|
||||
tagger_tags = self._tagger_fill(targets, tagger_model)
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
|
||||
# 5. タグを集めて未キャッシュ分を埋め込む
|
||||
job.update(phase="tags", done=0, total=0, note="")
|
||||
tags, available = self.collect_tags(cards, max_tags, gallery_prompts.values(), tagger_tags.values(),
|
||||
min_count, wildcard)
|
||||
cache = self._tag_vectors()
|
||||
todo = [t["label"] for t in tags if t["label"] not in cache]
|
||||
job.update(total=len(todo))
|
||||
for i in range(0, len(todo), _EMBED_BATCH):
|
||||
if self._cancel.is_set():
|
||||
break
|
||||
batch = todo[i:i + _EMBED_BATCH]
|
||||
try:
|
||||
vecs = ollama.embed_texts(batch, ollama.DEFAULT_LOCAL_DIM)
|
||||
except ollama.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
with self._lock:
|
||||
cache.update(zip(batch, vecs))
|
||||
job["done"] = min(len(todo), i + len(batch))
|
||||
if (i // _EMBED_BATCH) % 20 == 19:
|
||||
self._save_tag_vectors()
|
||||
self._save_tag_vectors()
|
||||
tags = [t for t in tags if t["label"] in cache]
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
|
||||
# 6. 投影
|
||||
job.update(phase="project", done=0, total=0)
|
||||
self._project(cards, card_vecs, tags, cache, dim, bg_paths, bg_mat, gallery_prompts, tagger_tags,
|
||||
tagger_model if tagger_tags else "",
|
||||
{"max_tags": max_tags, "min_count": min_count, "wildcard": wildcard, "tags_available": available})
|
||||
except SemanticError as e:
|
||||
job["error_code"], job["error"] = e.code, str(e)
|
||||
except Exception as e:
|
||||
logger.exception("TagSpace: build failed")
|
||||
job["error_code"], job["error"] = "tagspace_error", str(e)
|
||||
finally:
|
||||
ollama.release()
|
||||
job["running"] = False
|
||||
job["finished"] = True
|
||||
job["cancelled"] = self._cancel.is_set() and not job["error_code"]
|
||||
job["phase"] = ""
|
||||
|
||||
@staticmethod
|
||||
def _unit(m: np.ndarray, dim: int) -> np.ndarray:
|
||||
m = m[:, :dim].astype(np.float32)
|
||||
n = np.linalg.norm(m, axis=1, keepdims=True)
|
||||
return m / np.where(n > 0, n, 1)
|
||||
|
||||
def _project(self, cards, card_vecs, tags, cache, dim, bg_paths, bg_mat, gallery_prompts=None,
|
||||
tagger_tags=None, tagger_model: str = "", tag_meta: dict | None = None) -> None:
|
||||
img = self._unit(np.stack([card_vecs[c["id"]] for c in cards]), dim) if cards else np.zeros((0, dim), np.float32)
|
||||
txt = self._unit(np.stack([cache[t["label"]] for t in tags]), dim) if tags else np.zeros((0, dim), np.float32)
|
||||
if len(img) + len(txt) < 3:
|
||||
raise SemanticError("tagspace_too_few")
|
||||
bg_u = self._unit(bg_mat, dim) if bg_mat is not None and len(bg_mat) else np.zeros((0, dim), np.float32)
|
||||
# modality gap 対策: 画像とテキスト(タグ)をそれぞれの平均で中心化する。
|
||||
# 画像側はカードの平均を使う(数の多い背景画像の平均だとカードがタグの塊から離れて見えるため)
|
||||
img_ref = img if len(img) else bg_u
|
||||
mean_img = img_ref.mean(axis=0) if len(img_ref) else np.zeros(dim, np.float32)
|
||||
mean_txt = txt.mean(axis=0) if len(txt) else np.zeros(dim, np.float32)
|
||||
x = np.concatenate([img - mean_img, txt - mean_txt])
|
||||
_, s, vt = np.linalg.svd(x, full_matrices=False)
|
||||
axes = vt[:3].astype(np.float32)
|
||||
var = (s ** 2) / max(float((s ** 2).sum()), 1e-12)
|
||||
pts = x @ axes.T
|
||||
r98 = float(np.percentile(np.linalg.norm(pts, axis=1), 98)) or 1.0
|
||||
scale = _RADIUS / r98
|
||||
|
||||
def xyz(m):
|
||||
return [[round(float(v), 3) for v in row] for row in (m @ axes.T * scale)]
|
||||
|
||||
n_img = len(img)
|
||||
for c, p in zip(cards, xyz(img - mean_img)):
|
||||
c["xyz"] = p
|
||||
for t, p in zip(tags, xyz(txt - mean_txt)):
|
||||
t["xyz"] = p
|
||||
gp, tt = gallery_prompts or {}, tagger_tags or {}
|
||||
|
||||
def bg_item(path, pos):
|
||||
"""背景点にはその画像のタグを持たせる(クリックでまとめて追加)。プロンプト由来が無ければTagger補完"""
|
||||
tags_p = list(dict.fromkeys(split_tags(gp.get(path, ""))))
|
||||
if tags_p:
|
||||
return {"id": path, "xyz": pos, "tags": tags_p, "tag_src": "prompt"}
|
||||
tags_t = list(dict.fromkeys(split_tags(tt.get(path, ""))))
|
||||
return {"id": path, "xyz": pos, "tags": tags_t, "tag_src": "tagger" if tags_t else ""}
|
||||
|
||||
bg_items = [bg_item(p, q) for p, q in zip(bg_paths, xyz(bg_u - mean_img))] if len(bg_u) else []
|
||||
meta = {"built_at": time.strftime("%Y-%m-%d %H:%M:%S"), "cards": len(cards), "tags": len(tags),
|
||||
"background": len(bg_items), "gallery_prompts": len(gp), "tagger_images": len(tt),
|
||||
"tagger_model": tagger_model, "dim": dim, "model": LOCAL_MODEL,
|
||||
"explained": [round(float(v), 4) for v in var[:3]], **(tag_meta or {})}
|
||||
space = {"meta": meta, "cards": cards, "tags": tags, "background": bg_items}
|
||||
|
||||
ids = [c["id"] for c in cards] + [t["id"] for t in tags]
|
||||
tmp = self._mat_file.with_suffix(".tmp.npz")
|
||||
np.savez(tmp, ids=np.array(ids), vecs=np.concatenate([img, txt]).astype(np.float32),
|
||||
n_img=np.array(n_img), mean_img=mean_img.astype(np.float32), mean_txt=mean_txt.astype(np.float32),
|
||||
axes=axes, scale=np.array(scale, dtype=np.float32),
|
||||
bg_ids=np.array([b["id"] for b in bg_items]), bg_vecs=bg_u.astype(np.float32))
|
||||
tmp.replace(self._mat_file)
|
||||
tmp_json = self._space_file.with_suffix(".tmp.json")
|
||||
tmp_json.write_text(json.dumps(space, ensure_ascii=False), encoding="utf-8")
|
||||
tmp_json.replace(self._space_file)
|
||||
with self._lock:
|
||||
self._space, self._mat = space, None
|
||||
|
||||
# ── 近傍・検索 ─────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def _centered(vecs: np.ndarray, mean: np.ndarray) -> np.ndarray:
|
||||
c = vecs - mean
|
||||
return c / np.maximum(np.linalg.norm(c, axis=1, keepdims=True), 1e-6)
|
||||
|
||||
def _rank(self, m: dict, q: np.ndarray, k: int, exclude: str = "", kind: str = "") -> list[dict]:
|
||||
"""カード・タグ・画像は別々の塊なので、それぞれの平均を引いた空間で比べる(=中心化コサイン)。
|
||||
同じ種類どうしの方がスコアが高く出るため、種類ごとに順位付けして交互に並べる。
|
||||
Gallery画像(背景点)を候補に入れるのは、Gallery画像を基準にしたときだけ。"""
|
||||
qc = q - (m["mean_txt"] if kind == "tag" else m["mean_img"])
|
||||
qn = qc / max(float(np.linalg.norm(qc)), 1e-6)
|
||||
order = {"tag": ("tag", "card"), "card": ("card", "tag"), "image": ("image", "tag", "card")}[kind or "card"]
|
||||
lists = []
|
||||
for name in order:
|
||||
ids, mat = m["blocks"][name]
|
||||
if not len(ids):
|
||||
lists.append([])
|
||||
continue
|
||||
scores = mat @ qn
|
||||
top = np.argsort(-scores)[:k + 1]
|
||||
lists.append([{"id": ids[i], "score": round(float(scores[i]), 4)} for i in top if ids[i] != exclude][:k])
|
||||
out = []
|
||||
for i in range(k):
|
||||
out += [lst[i] for lst in lists if i < len(lst)]
|
||||
if len(out) >= k:
|
||||
break
|
||||
return out[:k]
|
||||
|
||||
def neighbors(self, item_id: str, k: int) -> list[dict]:
|
||||
m = self._load_mat()
|
||||
i = m["pos"].get(item_id)
|
||||
if i is not None:
|
||||
kind = "card" if i < int(m["n_img"]) else "tag"
|
||||
return self._rank(m, m["vecs"][i], k, exclude=item_id, kind=kind)
|
||||
j = m["bg_pos"].get(item_id)
|
||||
if j is not None:
|
||||
return self._rank(m, m["bg_vecs"][j], k, exclude=item_id, kind="image")
|
||||
raise SemanticError("tagspace_unknown_item", item_id)
|
||||
|
||||
def search(self, query: str, k: int) -> dict:
|
||||
query = (query or "").strip()[:500]
|
||||
if not query:
|
||||
return {"probe": None, "matches": []}
|
||||
m = self._load_mat()
|
||||
dim = m["vecs"].shape[1]
|
||||
try:
|
||||
q = ollama.embed_text(query, ollama.DEFAULT_LOCAL_DIM)
|
||||
except ollama.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
q = self._unit(q[None, :], dim)[0]
|
||||
p = (q - m["mean_txt"]) @ m["axes"].T * float(m["scale"])
|
||||
return {"probe": [round(float(v), 3) for v in p], "matches": self._rank(m, q, k, kind="tag")}
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Prompt translation service (native language -> English) via Ollama.
|
||||
|
||||
PLaMo 3 Translate (pfnet/plamo-3-translate-*-GGUF) models are driven with a raw
|
||||
PFML3 prompt on /api/generate, which is the route the model card recommends for
|
||||
Ollama. Any other Ollama model is used through /api/chat with a short
|
||||
"translate into English" system prompt, so a general LLM or another MT model
|
||||
can be selected too.
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import threading
|
||||
import urllib.request
|
||||
from collections import OrderedDict
|
||||
|
||||
from .settings_service import SettingsService
|
||||
from .url_guard import require_backend_url, urlopen
|
||||
|
||||
DEFAULTS = {
|
||||
"prompt_translate_enabled": False,
|
||||
"prompt_translate_url": "http://localhost:11434",
|
||||
"prompt_translate_model": "hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M",
|
||||
"prompt_translate_source_lang": "auto",
|
||||
"prompt_translate_auto": False, # GenerateUI Generate
|
||||
"prompt_translate_auto_lab": False, # GenerateUI Lab Run
|
||||
"prompt_translate_auto_video": False, # Video Plan Run
|
||||
"prompt_translate_unload": True,
|
||||
# "gpu" (default) or "cpu". CPU mode keeps the model resident in system RAM:
|
||||
# no VRAM use, so it never needs unloading and skips Ollama's ~4 s reload.
|
||||
"prompt_translate_device": "gpu",
|
||||
}
|
||||
|
||||
CPU_KEEP_ALIVE = -1 # stay loaded until Ollama restarts (about 2 GB RAM for 2B Q4_K_M)
|
||||
|
||||
# Suggested PLaMo 3 Translate builds (shown in Settings even before they are pulled).
|
||||
PLAMO_PRESETS = [
|
||||
"hf.co/pfnet/plamo-3-translate-0.8b-GGUF:Q4_K_M",
|
||||
"hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M",
|
||||
"hf.co/pfnet/plamo-3-translate-8b-GGUF:Q4_K_M",
|
||||
"hf.co/pfnet/plamo-3-translate-31b-GGUF:Q4_K_M",
|
||||
]
|
||||
|
||||
_PLAMO_STOP = ["<|plamo:tag|>", "<|plamo:bos|>"]
|
||||
_PLAMO_SPECIAL = re.compile(r"<\|plamo:[a-z_]+\|>")
|
||||
_LANG_RE = re.compile(r"^[a-z]{2,3}(-[A-Za-z]{2,4})?$")
|
||||
_MAX_TEXT_LEN = 4000
|
||||
_MAX_TEXTS = 200
|
||||
|
||||
_CHAT_SYSTEM = (
|
||||
"You translate text for image generation prompts into English. "
|
||||
"Output only the English translation, with no explanations, quotes or notes. "
|
||||
"Keep proper nouns, numbers and any ASCII text unchanged."
|
||||
)
|
||||
|
||||
_cache: "OrderedDict[tuple, str]" = OrderedDict()
|
||||
_CACHE_MAX = 1024
|
||||
_cache_lock = threading.Lock()
|
||||
|
||||
_settings = SettingsService()
|
||||
|
||||
|
||||
def get_config():
|
||||
data = _settings.load()
|
||||
cfg = {k: data.get(k, v) for k, v in DEFAULTS.items()}
|
||||
cfg["prompt_translate_url"] = (cfg["prompt_translate_url"] or DEFAULTS["prompt_translate_url"]).rstrip("/")
|
||||
cfg["prompt_translate_model"] = cfg["prompt_translate_model"] or DEFAULTS["prompt_translate_model"]
|
||||
return cfg
|
||||
|
||||
|
||||
def is_plamo(model: str) -> bool:
|
||||
return "plamo" in (model or "").lower()
|
||||
|
||||
|
||||
def build_pfml3(text: str, target_lang: str = "en", source_lang: str = "") -> str:
|
||||
"""Build a PFML3 translation prompt (BOS is added by llama.cpp/Ollama)."""
|
||||
src = _PLAMO_SPECIAL.sub("", text)
|
||||
src_attr = f"<|plamo:key|>lang<|plamo:val|>{source_lang}" if source_lang else ""
|
||||
return (
|
||||
"<|plamo:tag|>dataset<|plamo:msg|>translation"
|
||||
f"<|plamo:tag|>input{src_attr}<|plamo:msg|>{src}"
|
||||
f"<|plamo:tag|>output<|plamo:key|>lang<|plamo:val|>{target_lang}<|plamo:msg|>"
|
||||
)
|
||||
|
||||
|
||||
def _post_json(url: str, payload: dict, timeout: float):
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=json.dumps(payload).encode("utf-8"),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urlopen(req, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _num_predict(text: str) -> int:
|
||||
# CJK -> English roughly doubles-to-triples the token count; leave headroom.
|
||||
return max(64, min(2048, len(text) * 4))
|
||||
|
||||
|
||||
def _device_options(device):
|
||||
return {"num_gpu": 0} if device == "cpu" else {}
|
||||
|
||||
|
||||
def _translate_one(base_url, model, text, source_lang, keep_alive, device="gpu"):
|
||||
dev = _device_options(device)
|
||||
if is_plamo(model):
|
||||
data = _post_json(f"{base_url}/api/generate", {
|
||||
"model": model,
|
||||
"prompt": build_pfml3(text, "en", source_lang),
|
||||
"raw": True,
|
||||
"stream": False,
|
||||
"keep_alive": keep_alive,
|
||||
"options": {"temperature": 0, "num_predict": _num_predict(text), "stop": _PLAMO_STOP, **dev},
|
||||
}, timeout=180)
|
||||
out = data.get("response", "")
|
||||
else:
|
||||
data = _post_json(f"{base_url}/api/chat", {
|
||||
"model": model,
|
||||
"messages": [
|
||||
{"role": "system", "content": _CHAT_SYSTEM},
|
||||
{"role": "user", "content": text},
|
||||
],
|
||||
"stream": False,
|
||||
"think": False,
|
||||
"keep_alive": keep_alive,
|
||||
"options": {"temperature": 0, "num_predict": _num_predict(text), **dev},
|
||||
}, timeout=180)
|
||||
out = (data.get("message") or {}).get("content", "")
|
||||
out = _PLAMO_SPECIAL.sub("", out)
|
||||
return out.strip()
|
||||
|
||||
|
||||
def unload(base_url: str, model: str):
|
||||
try:
|
||||
_post_json(f"{base_url}/api/generate", {"model": model, "keep_alive": 0}, timeout=15)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def translate_texts(texts, model=None, source_lang=None):
|
||||
"""Translate each string in ``texts`` into English. Blocking (run in a thread).
|
||||
|
||||
Returns (translations, cached_count). Identical inputs are served from a
|
||||
bounded cache keyed by (model, source_lang, text).
|
||||
"""
|
||||
if not isinstance(texts, list) or len(texts) > _MAX_TEXTS:
|
||||
raise ValueError(f"texts must be a list of at most {_MAX_TEXTS} strings")
|
||||
cfg = get_config()
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
model = (model or cfg["prompt_translate_model"]).strip()
|
||||
src = (source_lang if source_lang is not None else cfg["prompt_translate_source_lang"]) or "auto"
|
||||
src = "" if src == "auto" else src
|
||||
if src and not _LANG_RE.match(src):
|
||||
raise ValueError("invalid source language")
|
||||
|
||||
device = "cpu" if cfg.get("prompt_translate_device") == "cpu" else "gpu"
|
||||
keep_alive = CPU_KEEP_ALIVE if device == "cpu" else "5m"
|
||||
|
||||
results = []
|
||||
cached = 0
|
||||
used_model = False
|
||||
for raw in texts:
|
||||
text = str(raw or "")[:_MAX_TEXT_LEN]
|
||||
if not text.strip():
|
||||
results.append(text)
|
||||
continue
|
||||
key = (model, src, text)
|
||||
with _cache_lock:
|
||||
hit = _cache.get(key)
|
||||
if hit is not None:
|
||||
_cache.move_to_end(key)
|
||||
if hit is not None:
|
||||
results.append(hit)
|
||||
cached += 1
|
||||
continue
|
||||
out = _translate_one(base_url, model, text, src, keep_alive, device)
|
||||
used_model = True
|
||||
with _cache_lock:
|
||||
_cache[key] = out
|
||||
_cache.move_to_end(key)
|
||||
while len(_cache) > _CACHE_MAX:
|
||||
_cache.popitem(last=False)
|
||||
results.append(out)
|
||||
|
||||
if used_model and device == "gpu" and cfg.get("prompt_translate_unload", True):
|
||||
# Free VRAM for the image model that usually runs right after.
|
||||
unload(base_url, model)
|
||||
return results, cached
|
||||
|
||||
|
||||
def warmup():
|
||||
"""Load the translation model ahead of the first translation (CPU mode).
|
||||
|
||||
In CPU mode the model is kept resident, so loading it once in the background
|
||||
when GenerateUI opens hides Ollama's ~4 s startup from the first translation.
|
||||
Returns False when there is nothing to do.
|
||||
"""
|
||||
cfg = get_config()
|
||||
if not cfg.get("prompt_translate_enabled") or cfg.get("prompt_translate_device") != "cpu":
|
||||
return False
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
model = cfg["prompt_translate_model"]
|
||||
# A request without a prompt only loads the model; the options decide CPU placement.
|
||||
_post_json(f"{base_url}/api/generate", {
|
||||
"model": model,
|
||||
"keep_alive": CPU_KEEP_ALIVE,
|
||||
"options": _device_options("cpu"),
|
||||
}, timeout=180)
|
||||
return True
|
||||
|
||||
|
||||
def list_models():
|
||||
cfg = get_config()
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
req = urllib.request.Request(f"{base_url}/api/tags")
|
||||
with urlopen(req, timeout=5) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
return [m.get("name") for m in data.get("models", []) if m.get("name")]
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Outbound URL guard for backend proxies (SSRF mitigation).
|
||||
|
||||
Several routes (Ollama chat, Tagger VLM, Eagle, Unsloth relay) connect to a backend whose
|
||||
address is supplied by the browser. Without a check, any caller that can reach ComfyUI could
|
||||
make the server connect to an arbitrary host (cloud metadata, LAN services, file://).
|
||||
|
||||
Allowed targets:
|
||||
- loopback hosts (localhost / 127.0.0.1 / ::1), any port
|
||||
- hosts listed by the operator in the WFS_ALLOWED_BACKEND_HOSTS environment variable
|
||||
(comma-separated "host" or "host:port"; e.g. "192.168.1.20:11434,gpu-box")
|
||||
|
||||
Only http(s) URLs without credentials are accepted, and redirects are never followed.
|
||||
"""
|
||||
|
||||
import os
|
||||
import urllib.request
|
||||
from urllib.parse import urlparse
|
||||
|
||||
LOOPBACK_HOSTS = {"localhost", "127.0.0.1", "::1"}
|
||||
ALLOWED_HOSTS_ENV = "WFS_ALLOWED_BACKEND_HOSTS"
|
||||
|
||||
HINT = (
|
||||
"Backend URL must point to localhost/127.0.0.1/::1. To use another host, list it in the "
|
||||
f"{ALLOWED_HOSTS_ENV} environment variable (e.g. 192.168.1.20:11434) and restart ComfyUI."
|
||||
)
|
||||
|
||||
|
||||
def _operator_allowlist():
|
||||
raw = os.environ.get(ALLOWED_HOSTS_ENV, "")
|
||||
return {item.strip().lower() for item in raw.split(",") if item.strip()}
|
||||
|
||||
|
||||
def is_allowed_backend_url(url):
|
||||
"""True if url is an http(s) URL without credentials whose host[:port] is permitted."""
|
||||
try:
|
||||
parsed = urlparse((url or "").strip())
|
||||
if parsed.scheme not in ("http", "https") or not parsed.hostname:
|
||||
return False
|
||||
if parsed.username or parsed.password:
|
||||
return False
|
||||
host = parsed.hostname.lower()
|
||||
if host in LOOPBACK_HOSTS:
|
||||
return True
|
||||
allow = _operator_allowlist()
|
||||
return host in allow or (parsed.port is not None and f"{host}:{parsed.port}" in allow)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def require_backend_url(url):
|
||||
"""Return url stripped of trailing slashes, or raise ValueError(HINT)."""
|
||||
url = (url or "").strip()
|
||||
if not is_allowed_backend_url(url):
|
||||
raise ValueError(HINT)
|
||||
return url.rstrip("/")
|
||||
|
||||
|
||||
class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
def redirect_request(self, *args, **kwargs):
|
||||
return None
|
||||
|
||||
|
||||
_opener = urllib.request.build_opener(_NoRedirect)
|
||||
|
||||
|
||||
def urlopen(req, timeout):
|
||||
"""urllib urlopen that never follows redirects (a 3xx surfaces as HTTPError)."""
|
||||
return _opener.open(req, timeout=timeout)
|
||||
@@ -12,6 +12,8 @@ import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from ..config import is_safe_rel
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
@@ -209,6 +211,10 @@ class VideoService:
|
||||
base_dir = folder_paths.get_directory_by_type(type_)
|
||||
if not base_dir:
|
||||
raise ValueError(f"Invalid type: {type_}")
|
||||
# UNC・ドライブ指定・絶対パス・「..」は、結合・resolve() の前に文字列のまま拒否する
|
||||
# (Windows では UNC を resolve() するだけで相手ホストへ資格情報付きで接続するため)
|
||||
if not is_safe_rel(filename) or (subfolder and not is_safe_rel(subfolder)):
|
||||
raise ValueError("Access denied: path outside allowed directory")
|
||||
base = Path(base_dir).resolve()
|
||||
target = (base / (subfolder or "") / filename).resolve()
|
||||
try:
|
||||
@@ -237,6 +243,68 @@ class VideoService:
|
||||
save_path.write_bytes(image_bytes)
|
||||
return {"filename": save_name, "subfolder": subfolder}
|
||||
|
||||
@staticmethod
|
||||
def _unique_output_path(stem: str, ext: str) -> tuple[Path, str]:
|
||||
"""output/video/ 配下で未使用のパスを決める。戻り値は (絶対パス, subfolder)。"""
|
||||
import folder_paths # type: ignore
|
||||
|
||||
subfolder = "video"
|
||||
out_dir = Path(folder_paths.get_output_directory()) / subfolder
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
dst = out_dir / f"{stem}{ext}"
|
||||
counter = 1
|
||||
while dst.exists():
|
||||
dst = out_dir / f"{stem}_{counter}{ext}"
|
||||
counter += 1
|
||||
return dst, subfolder
|
||||
|
||||
def register_asset(self, filename: str, subfolder: str, type_: str) -> dict:
|
||||
"""Video Sourceの動画をGalleryに載せるため、outputフォルダ配下の実体にする。
|
||||
既にoutput配下ならそのまま、input/tempならoutput/videoへコピーする。"""
|
||||
import shutil
|
||||
|
||||
src = self._resolve_media_path(filename, subfolder, type_)
|
||||
if type_ == "output":
|
||||
return {"filename": src.name, "subfolder": subfolder or "", "path": str(src).replace("\\", "/")}
|
||||
dst, out_sub = self._unique_output_path(src.stem, src.suffix)
|
||||
shutil.copy2(src, dst)
|
||||
return {"filename": dst.name, "subfolder": out_sub, "path": str(dst).replace("\\", "/")}
|
||||
|
||||
def convert_audio_to_output(self, filename: str, subfolder: str, type_: str, opts: dict) -> dict:
|
||||
"""動画/音声クリップの音声を指定形式へ変換し、output/videoへ保存する。"""
|
||||
from .convert_service import OUTPUT_FORMATS, convert_audio, supports_tags
|
||||
|
||||
fmt = str(opts.get("format", ""))
|
||||
if fmt not in OUTPUT_FORMATS:
|
||||
raise ValueError(f"Unsupported format: {fmt}")
|
||||
|
||||
def _int(key: str) -> int:
|
||||
try:
|
||||
return max(0, int(opts.get(key) or 0))
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
src = self._resolve_media_path(filename, subfolder, type_)
|
||||
inherit = bool(opts.get("inherit_metadata", True))
|
||||
dst, out_sub = self._unique_output_path(src.stem, OUTPUT_FORMATS[fmt]["ext"])
|
||||
try:
|
||||
convert_audio(
|
||||
src, dst, fmt,
|
||||
bitrate_kbps=_int("bitrate"),
|
||||
sample_rate=_int("sample_rate"),
|
||||
channels=str(opts.get("channels", "keep")),
|
||||
inherit_metadata=inherit,
|
||||
)
|
||||
except Exception:
|
||||
dst.unlink(missing_ok=True)
|
||||
raise
|
||||
return {
|
||||
"filename": dst.name,
|
||||
"subfolder": out_sub,
|
||||
"path": str(dst).replace("\\", "/"),
|
||||
"tags_skipped": inherit and not supports_tags(fmt),
|
||||
}
|
||||
|
||||
def probe_video(self, filename: str, subfolder: str, type_: str) -> dict:
|
||||
"""Reads duration/dimensions/fps/audio presence via PyAV only (no ffmpeg
|
||||
subprocess) — used by the Video Edit tab right after a clip is added, so
|
||||
|
||||
@@ -83,10 +83,11 @@ class WildcardService:
|
||||
return None
|
||||
# Normalize to forward slashes and strip leading/trailing slashes
|
||||
filename = filename.replace("\\", "/").strip("/")
|
||||
# Validate each path component — block traversal and empty parts
|
||||
# Validate each path component — block traversal, empty parts and drive-relative
|
||||
# names ("C:name"; on Python <=3.11 joinpath would replace the base path with them)
|
||||
parts = filename.split("/")
|
||||
for p in parts:
|
||||
if not p or p == ".." or p == ".":
|
||||
if not p or p == ".." or p == "." or ":" in p:
|
||||
return None
|
||||
ext = Path(parts[-1]).suffix.lstrip(".").lower()
|
||||
if ext not in ALLOWED_EXTS:
|
||||
|
||||
@@ -99,12 +99,13 @@ class WorkflowStudio:
|
||||
app.router.add_get("/wfm", serve_index_page)
|
||||
|
||||
# API routes
|
||||
from .routes import workflow_routes, settings_routes, ollama_routes, unsloth_routes, eagle_routes, nodes_routes, prompts_routes, models_routes, gallery_routes, wildcard_routes, tagger_routes, gmic_routes, skill_routes, lab_routes, video_routes
|
||||
from .routes import workflow_routes, settings_routes, ollama_routes, unsloth_routes, eagle_routes, nodes_routes, prompts_routes, models_routes, gallery_routes, wildcard_routes, tagger_routes, gmic_routes, skill_routes, lab_routes, video_routes, vram_routes, partner_routes, tagspace_routes, translate_routes
|
||||
|
||||
workflow_routes.setup_routes(app)
|
||||
settings_routes.setup_routes(app)
|
||||
ollama_routes.setup_routes(app)
|
||||
unsloth_routes.setup_routes(app)
|
||||
partner_routes.setup_routes(app)
|
||||
eagle_routes.setup_routes(app)
|
||||
nodes_routes.setup_routes(app)
|
||||
prompts_routes.setup_routes(app)
|
||||
@@ -116,5 +117,8 @@ class WorkflowStudio:
|
||||
skill_routes.setup_routes(app)
|
||||
lab_routes.setup_routes(app)
|
||||
video_routes.setup_routes(app)
|
||||
vram_routes.setup_routes(app)
|
||||
tagspace_routes.setup_routes(app)
|
||||
translate_routes.setup_routes(app)
|
||||
|
||||
logger.info("Workflow Studio: Routes registered successfully")
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "comfyui-workflow-studio"
|
||||
description = "A comprehensive workflow, asset management, and generation UI plugin for ComfyUI. Manage workflows, models, images, and prompts; batch generation and Image Feeder in the GenerateUI tab; library side panel with canvas drag & drop and AI tools."
|
||||
version = "0.7.11"
|
||||
version = "0.9.9"
|
||||
license = { file = "LICENSE" }
|
||||
requires-python = ">= 3.10"
|
||||
dependencies = ["jinja2"]
|
||||
|
||||
@@ -33,3 +33,7 @@ psd-tools>=1.10
|
||||
# If missing, mp4 thumbnail generation simply fails and falls back to a placeholder
|
||||
# (Gallery itself keeps working). Not pinned here on purpose — see torch note above.
|
||||
# av>=17.0.0
|
||||
|
||||
# -- Gallery Semantic Search: EmbeddingGemma 2 (optional) --
|
||||
# Runs in Ollama (no extra Python packages). Install Ollama and download the model:
|
||||
# ollama pull embeddinggemma-2:740m (or embeddinggemma-2:440m - same vectors, no audio encoder)
|
||||
|
||||
@@ -867,6 +867,13 @@
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.wfm-gallery-move-filter {
|
||||
width: 100%;
|
||||
padding: 6px 8px;
|
||||
font-size: 13px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.wfm-gallery-move-modal-footer {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
@@ -983,3 +990,249 @@
|
||||
min-height: 0;
|
||||
resize: none;
|
||||
}
|
||||
|
||||
/* 音声ファイル: 波形画像＋<audio> コントロール */
|
||||
.wfm-gallery-audio-box {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
gap: 8px;
|
||||
width: 100%;
|
||||
}
|
||||
.wfm-gallery-audio-wave {
|
||||
object-fit: contain;
|
||||
max-height: 220px;
|
||||
background: #181c26;
|
||||
border-radius: 6px;
|
||||
}
|
||||
.wfm-gallery-audio-player {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
/* 変換ダイアログ */
|
||||
.wfm-gallery-convert-modal {
|
||||
min-width: 360px;
|
||||
max-width: 480px;
|
||||
}
|
||||
.wfm-gallery-convert-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
margin: 8px 0;
|
||||
}
|
||||
.wfm-gallery-convert-row .wfm-select {
|
||||
min-width: 150px;
|
||||
}
|
||||
.wfm-gallery-convert-check {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
gap: 6px;
|
||||
margin: 6px 0;
|
||||
font-size: 12px;
|
||||
}
|
||||
.wfm-gallery-convert-note,
|
||||
.wfm-gallery-convert-status {
|
||||
font-size: 12px;
|
||||
opacity: 0.8;
|
||||
margin: 4px 0;
|
||||
}
|
||||
|
||||
/* 音声: 詳細パネルのプレビュー枠は固定高さ(正方形)をやめ、波形＋プレイヤーを収める */
|
||||
.wfm-gallery-detail-preview--audio {
|
||||
aspect-ratio: auto;
|
||||
max-height: none;
|
||||
overflow: visible;
|
||||
padding: 8px;
|
||||
}
|
||||
.wfm-gallery-detail-preview--audio .wfm-gallery-audio-wave {
|
||||
max-height: 150px;
|
||||
}
|
||||
|
||||
/* ── セマンティック検索 ─────────────────────────────────────── */
|
||||
|
||||
.wfm-gallery-sem-toggle.active {
|
||||
background: var(--wfm-primary);
|
||||
border-color: var(--wfm-primary);
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
/* 2行目(Similar / 更新)のボタン: Taggerの紫と区別できるティール */
|
||||
.wfm-sem-sim {
|
||||
background: #0d9488;
|
||||
border-color: #0d9488;
|
||||
color: #fff;
|
||||
}
|
||||
.wfm-sem-sim:hover {
|
||||
background: #0f766e;
|
||||
border-color: #0f766e;
|
||||
}
|
||||
/* 類似画像表示中(トグルON)は縁取りで状態を示す */
|
||||
#wfm-gallery-similar-btn.active {
|
||||
box-shadow: 0 0 0 2px #fff inset, 0 0 0 2px #0d9488;
|
||||
}
|
||||
|
||||
.wfm-gallery-sem-row {
|
||||
gap: 6px;
|
||||
align-items: center;
|
||||
justify-content: flex-start;
|
||||
}
|
||||
.wfm-gallery-sem-count {
|
||||
margin-left: 6px;
|
||||
font-size: 12px;
|
||||
opacity: 0.85;
|
||||
}
|
||||
|
||||
.wfm-gallery-thumb-score {
|
||||
position: absolute;
|
||||
left: 4px;
|
||||
bottom: 4px;
|
||||
padding: 1px 5px;
|
||||
border-radius: 8px;
|
||||
background: rgba(13, 148, 136, 0.85);
|
||||
color: #fff;
|
||||
font-size: 10px;
|
||||
line-height: 14px;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.wfm-sem-modal { display: flex; flex-direction: column; gap: 10px; min-width: 420px; }
|
||||
.wfm-sem-row { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
|
||||
.wfm-sem-folder { font-family: monospace; font-size: 12px; word-break: break-all; opacity: 0.85; }
|
||||
.wfm-sem-hint { font-size: 11px; opacity: 0.7; margin: 0; }
|
||||
.wfm-sem-ok { color: #22c55e; }
|
||||
.wfm-sem-ng { color: #ef4444; }
|
||||
.wfm-sem-actions { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||
.wfm-sem-bar { height: 8px; border-radius: 4px; background: var(--wfm-border, #444); overflow: hidden; }
|
||||
.wfm-sem-bar-fill { height: 100%; width: 0; background: #0d9488; transition: width 0.3s; }
|
||||
|
||||
/* フォルダツリー: 直下の画像が全件セマンティック索引済みのフォルダに付くドット */
|
||||
.wfm-gallery-tree-sem-dot {
|
||||
display: inline-block;
|
||||
width: 7px;
|
||||
height: 7px;
|
||||
margin-left: 4px;
|
||||
border-radius: 50%;
|
||||
background: #22c55e;
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
/* モーダルのヘルプボタンから飛んだ先の説明を一時的に強調 */
|
||||
.wfm-help-flash {
|
||||
animation: wfm-help-flash 2.5s ease-out;
|
||||
}
|
||||
@keyframes wfm-help-flash {
|
||||
0%, 60% { background: rgba(13, 148, 136, 0.35); }
|
||||
100% { background: transparent; }
|
||||
}
|
||||
|
||||
.wfm-sem-local-opts { display: flex; flex-direction: column; gap: 10px; padding: 8px 10px; border: 1px solid var(--wfm-border, #444); border-radius: 6px; }
|
||||
|
||||
/* セマンティック検索モーダル: タブ */
|
||||
.wfm-sem-tabs { display: flex; gap: 4px; border-bottom: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-sem-tab-btn {
|
||||
padding: 6px 14px;
|
||||
background: transparent;
|
||||
border: 1px solid transparent;
|
||||
border-bottom: none;
|
||||
border-radius: 6px 6px 0 0;
|
||||
color: var(--wfm-text-secondary, #aaa);
|
||||
cursor: pointer;
|
||||
font-size: 13px;
|
||||
}
|
||||
.wfm-sem-tab-btn:hover { color: var(--wfm-text, #fff); }
|
||||
.wfm-sem-tab-btn.active {
|
||||
color: var(--wfm-text, #fff);
|
||||
background: var(--wfm-bg-secondary, #1a1a2e);
|
||||
border-color: var(--wfm-border, #444);
|
||||
margin-bottom: -1px;
|
||||
border-bottom: 1px solid var(--wfm-bg-secondary, #1a1a2e);
|
||||
}
|
||||
.wfm-sem-pane { display: flex; flex-direction: column; gap: 10px; min-height: 230px; }
|
||||
.wfm-sem-footer { justify-content: flex-end; border-top: 1px solid var(--wfm-border, #444); padding-top: 10px; }
|
||||
.wfm-sem-num { width: 80px; }
|
||||
.wfm-sem-url { width: 240px; max-width: 100%; }
|
||||
.wfm-sem-danger { margin-left: auto; color: #ef4444; }
|
||||
|
||||
.wfm-sem-row .wfm-select { width: auto; min-width: 90px; }
|
||||
|
||||
/* 3Dモデルビューア */
|
||||
.wfm-gallery-thumb-video-badge.wfm-gallery-thumb-video-badge--corner {
|
||||
top: auto;
|
||||
left: auto;
|
||||
right: 4px;
|
||||
bottom: 4px;
|
||||
width: auto;
|
||||
height: auto;
|
||||
padding: 1px 5px;
|
||||
border-radius: 3px;
|
||||
font-size: 10px;
|
||||
line-height: 14px;
|
||||
transform: none;
|
||||
}
|
||||
.wfm-g3d-box {
|
||||
position: relative;
|
||||
background: #181c26;
|
||||
border-radius: 4px;
|
||||
overflow: hidden;
|
||||
}
|
||||
.wfm-gallery-detail-img.wfm-g3d-box {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
}
|
||||
.wfm-gallery-lightbox-img.wfm-g3d-box {
|
||||
width: 90vw;
|
||||
height: 85vh;
|
||||
}
|
||||
.wfm-lightbox-compare-img.wfm-g3d-box {
|
||||
width: calc(92vw / var(--compare-cols, 2) - 12px);
|
||||
height: 78vh;
|
||||
}
|
||||
.wfm-g3d-canvas {
|
||||
display: block;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
outline: none;
|
||||
}
|
||||
.wfm-g3d-status {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 8px;
|
||||
text-align: center;
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 12px;
|
||||
pointer-events: none;
|
||||
}
|
||||
.wfm-g3d-status--error {
|
||||
color: #f87171;
|
||||
}
|
||||
.wfm-g3d-stats {
|
||||
position: absolute;
|
||||
left: 6px;
|
||||
bottom: 4px;
|
||||
color: #cbd5e1;
|
||||
font-size: 10px;
|
||||
text-shadow: 0 0 3px #000;
|
||||
pointer-events: none;
|
||||
}
|
||||
.wfm-g3d-reset {
|
||||
position: absolute;
|
||||
top: 4px;
|
||||
right: 4px;
|
||||
width: 22px;
|
||||
height: 22px;
|
||||
padding: 0;
|
||||
border: none;
|
||||
border-radius: 4px;
|
||||
background: rgba(0, 0, 0, 0.5);
|
||||
color: #e2e8f0;
|
||||
font-size: 13px;
|
||||
line-height: 22px;
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-g3d-reset:hover {
|
||||
background: rgba(0, 0, 0, 0.75);
|
||||
}
|
||||
|
||||
@@ -466,3 +466,103 @@
|
||||
background: var(--wfm-border);
|
||||
border-radius: 2px;
|
||||
}
|
||||
|
||||
/* Draw tool palette */
|
||||
.ie-palette-content {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex-wrap: wrap;
|
||||
gap: 6px 14px;
|
||||
}
|
||||
|
||||
.ie-palette-wells {
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.ie-palette-well,
|
||||
.ie-palette-swatch {
|
||||
padding: 0;
|
||||
border-radius: 50%;
|
||||
background: var(--swatch, transparent);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.ie-palette-well {
|
||||
width: 26px;
|
||||
height: 26px;
|
||||
border: 1px solid var(--wfm-text-secondary);
|
||||
}
|
||||
|
||||
.ie-palette-well.is-active {
|
||||
outline: 2px solid var(--wfm-primary);
|
||||
outline-offset: 2px;
|
||||
}
|
||||
|
||||
.ie-palette-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(10, 16px);
|
||||
grid-auto-rows: 16px;
|
||||
gap: 2px;
|
||||
}
|
||||
|
||||
.ie-palette-swatch {
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border: 1px solid rgba(128, 128, 128, 0.7);
|
||||
}
|
||||
|
||||
.ie-palette-swatch.is-empty {
|
||||
border-color: var(--wfm-text-secondary);
|
||||
opacity: 0.6;
|
||||
}
|
||||
|
||||
.ie-palette-swatch:hover,
|
||||
.ie-palette-swatch:focus-visible,
|
||||
.ie-palette-well:focus-visible {
|
||||
outline: 1px solid var(--wfm-text);
|
||||
outline-offset: 1px;
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.ie-palette-swatch.is-current {
|
||||
outline: 2px solid var(--wfm-primary);
|
||||
outline-offset: 0;
|
||||
}
|
||||
|
||||
.ie-palette-picker {
|
||||
position: fixed;
|
||||
width: 1px;
|
||||
height: 1px;
|
||||
padding: 0;
|
||||
border: 0;
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.ie-palette-menu {
|
||||
position: fixed;
|
||||
z-index: 1000;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-width: 150px;
|
||||
padding: 4px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
border-radius: 4px;
|
||||
background: var(--wfm-surface);
|
||||
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.4);
|
||||
}
|
||||
|
||||
.ie-palette-menu button {
|
||||
padding: 5px 10px;
|
||||
border: 0;
|
||||
border-radius: 3px;
|
||||
background: transparent;
|
||||
color: var(--wfm-text);
|
||||
font-size: 12px;
|
||||
text-align: left;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.ie-palette-menu button:hover {
|
||||
background: var(--wfm-surface-hover);
|
||||
}
|
||||
|
||||
+416
-1
@@ -984,6 +984,103 @@ body {
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
/* Input tab: Prompt (3) | Image (2) side by side. Wraps Image under Prompt when the column is narrow. */
|
||||
.wfm-gen-params-col--input {
|
||||
container-type: inline-size;
|
||||
container-name: wfm-input;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.wfm-input-split {
|
||||
display: flex;
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
.wfm-input-inner-panel--prompt {
|
||||
flex: 3 1 0;
|
||||
min-width: 0;
|
||||
border-right: 1px solid var(--wfm-border);
|
||||
}
|
||||
|
||||
.wfm-input-inner-panel--image {
|
||||
flex: 2 1 0;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.wfm-input-panel-title {
|
||||
font-size: 11px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
color: var(--wfm-text-secondary);
|
||||
margin-bottom: 6px;
|
||||
}
|
||||
|
||||
@container wfm-input (max-width: 720px) {
|
||||
.wfm-input-split {
|
||||
flex-direction: column;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.wfm-input-inner-panel--prompt,
|
||||
.wfm-input-inner-panel--image {
|
||||
flex: none;
|
||||
overflow-y: visible;
|
||||
border-right: none;
|
||||
}
|
||||
.wfm-input-inner-panel--prompt {
|
||||
border-bottom: 1px solid var(--wfm-border);
|
||||
}
|
||||
}
|
||||
|
||||
/* Input tab Raw JSON: collapsible to a slim strip at the right edge */
|
||||
.wfm-rawjson-collapse-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
flex-shrink: 0;
|
||||
padding: 3px 10px;
|
||||
background: var(--wfm-bg-secondary);
|
||||
border: none;
|
||||
border-bottom: 1px solid var(--wfm-border);
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 11px;
|
||||
cursor: pointer;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.wfm-rawjson-collapse-btn:hover {
|
||||
color: var(--wfm-text);
|
||||
}
|
||||
|
||||
.wfm-rawjson-collapse-label {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col.wfm-gen-rawjson-col--collapsed {
|
||||
width: 30px;
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col--collapsed .wfm-rawjson-collapse-btn {
|
||||
flex: 1;
|
||||
flex-direction: column;
|
||||
justify-content: flex-start;
|
||||
padding: 8px 0;
|
||||
border-bottom: none;
|
||||
border-left: 1px solid var(--wfm-border);
|
||||
background: var(--wfm-bg-secondary);
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col--collapsed .wfm-rawjson-collapse-label {
|
||||
display: block;
|
||||
writing-mode: vertical-rl;
|
||||
letter-spacing: 1px;
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col--collapsed #wfm-gen-rawjson-widget {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
/* Right sub-column: Raw JSON */
|
||||
.wfm-gen-rawjson-col {
|
||||
width: 540px;
|
||||
@@ -3491,7 +3588,8 @@ body {
|
||||
|
||||
.wfm-help-page {
|
||||
display: none;
|
||||
max-width: 760px;
|
||||
/* 上部タブバー(AI TOOLタブの右端)あたりまで広げ、折り返しを減らす。狭い画面では領域幅に収まる */
|
||||
max-width: 1076px;
|
||||
}
|
||||
|
||||
.wfm-help-page.active {
|
||||
@@ -5892,6 +5990,90 @@ body {
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* TOOLS: result actions, OCR options and text-position overlay (ocr-tools.js) */
|
||||
.wfm-ai-vlm-result-actions {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
gap: 6px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.wfm-ai-ocr-options {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.wfm-ai-ocr-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
}
|
||||
.wfm-ai-ocr-row .wfm-ai-label { flex-shrink: 0; margin: 0; }
|
||||
.wfm-ai-ocr-row .wfm-select { flex: 1; min-width: 0; }
|
||||
.wfm-ai-ocr-check {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
font-size: 12px;
|
||||
color: var(--wfm-text-secondary);
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-ai-ocr-check.is-disabled { opacity: 0.45; cursor: default; }
|
||||
.wfm-ai-ocr-hint { font-size: 11px; color: var(--wfm-text-secondary); opacity: 0.85; }
|
||||
/* OCR: a taller preview so the numbered boxes are usable */
|
||||
.wfm-ai-vlm-drop.is-ocr { height: 260px; }
|
||||
.wfm-ai-ocr-overlay {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
pointer-events: none;
|
||||
}
|
||||
.wfm-ai-ocr-box {
|
||||
position: absolute;
|
||||
border: 1.5px solid #4fd8ff;
|
||||
background: rgba(79, 216, 255, 0.12);
|
||||
box-sizing: border-box;
|
||||
pointer-events: auto;
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-ai-ocr-box.is-active { border-color: #ff9f43; background: rgba(255, 159, 67, 0.2); }
|
||||
.wfm-ai-ocr-box-tag {
|
||||
position: absolute;
|
||||
top: -1px;
|
||||
left: -1px;
|
||||
padding: 0 3px;
|
||||
font-size: 9px;
|
||||
line-height: 12px;
|
||||
color: #000;
|
||||
background: #4fd8ff;
|
||||
}
|
||||
|
||||
/* Collapsible Translation / Settings panes */
|
||||
.wfm-ai-pane-collapse-btn {
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 10px;
|
||||
line-height: 1;
|
||||
padding: 2px 4px;
|
||||
cursor: pointer;
|
||||
border-radius: 3px;
|
||||
}
|
||||
.wfm-ai-pane-collapse-btn:hover { background: var(--wfm-bg); color: var(--wfm-text); }
|
||||
.wfm-ai-pane.is-collapsed {
|
||||
flex: 0 0 28px;
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-ai-pane.is-collapsed > :not(.wfm-ai-pane-header) { display: none !important; }
|
||||
.wfm-ai-pane.is-collapsed .wfm-ai-pane-header {
|
||||
writing-mode: vertical-rl;
|
||||
flex: 1;
|
||||
justify-content: flex-start;
|
||||
gap: 8px;
|
||||
padding: 8px 0;
|
||||
border-bottom: none;
|
||||
}
|
||||
.wfm-ai-pane.is-collapsed .wfm-ai-pane-collapse-btn { writing-mode: horizontal-tb; }
|
||||
|
||||
/* Chat pane */
|
||||
.wfm-ai-chat-container {
|
||||
display: flex;
|
||||
@@ -5952,6 +6134,118 @@ body {
|
||||
object-fit: cover;
|
||||
}
|
||||
|
||||
/* Markdown-rendered assistant replies (chat-markdown.js) */
|
||||
.wfm-ai-chat-msg.wfm-ai-chat-md { white-space: normal; }
|
||||
.wfm-ai-chat-md p { margin: 0 0 6px; }
|
||||
.wfm-ai-chat-md p:last-child { margin-bottom: 0; }
|
||||
.wfm-ai-chat-md ul, .wfm-ai-chat-md ol { margin: 2px 0 6px; padding-left: 20px; }
|
||||
.wfm-ai-chat-md li { margin: 1px 0; }
|
||||
.wfm-ai-chat-md .wfm-md-h { margin: 8px 0 4px; font-size: 13px; font-weight: 700; }
|
||||
.wfm-ai-chat-md h3.wfm-md-h { font-size: 14px; }
|
||||
.wfm-ai-chat-md code {
|
||||
font-family: Consolas, "Courier New", monospace;
|
||||
font-size: 12px;
|
||||
background: rgba(127, 127, 127, 0.18);
|
||||
padding: 0 3px;
|
||||
border-radius: 3px;
|
||||
}
|
||||
.wfm-ai-chat-md .wfm-md-pre {
|
||||
position: relative;
|
||||
margin: 4px 0 6px;
|
||||
padding: 8px 10px;
|
||||
background: rgba(0, 0, 0, 0.25);
|
||||
border: 1px solid var(--wfm-border);
|
||||
border-radius: 4px;
|
||||
overflow-x: auto;
|
||||
white-space: pre;
|
||||
}
|
||||
.wfm-ai-chat-md .wfm-md-pre code { background: none; padding: 0; }
|
||||
.wfm-ai-chat-md .wfm-md-code-lang {
|
||||
position: absolute;
|
||||
top: 2px;
|
||||
right: 6px;
|
||||
font-size: 10px;
|
||||
opacity: 0.6;
|
||||
}
|
||||
.wfm-ai-chat-md blockquote {
|
||||
margin: 4px 0;
|
||||
padding-left: 8px;
|
||||
border-left: 3px solid var(--wfm-border);
|
||||
opacity: 0.9;
|
||||
}
|
||||
.wfm-ai-chat-md hr { border: none; border-top: 1px solid var(--wfm-border); margin: 6px 0; }
|
||||
|
||||
/* Skill auto-check notices (metadata.output) */
|
||||
.wfm-ai-chat-msg-notice {
|
||||
align-self: center;
|
||||
max-width: 95%;
|
||||
background: transparent;
|
||||
border: 1px dashed var(--wfm-border);
|
||||
color: var(--wfm-text-secondary, var(--wfm-text));
|
||||
font-size: 11px;
|
||||
opacity: 0.85;
|
||||
}
|
||||
|
||||
/* Action buttons under a ```prompt block */
|
||||
.wfm-ai-chat-msg-actions {
|
||||
margin-top: 6px;
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 6px;
|
||||
white-space: normal;
|
||||
}
|
||||
|
||||
/* Image check result card (chat-image-check.js) */
|
||||
.wfm-ai-chat-msg-check {
|
||||
white-space: normal;
|
||||
min-width: 260px;
|
||||
}
|
||||
.wfm-ai-check-head {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: baseline;
|
||||
gap: 8px;
|
||||
font-weight: 600;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.wfm-ai-check-head.is-pass { color: #4caf50; }
|
||||
.wfm-ai-check-head.is-fail { color: #e5a00d; }
|
||||
.wfm-ai-check-meta {
|
||||
font-weight: normal;
|
||||
font-size: 11px;
|
||||
opacity: 0.7;
|
||||
color: var(--wfm-text-secondary);
|
||||
}
|
||||
.wfm-ai-check-list {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
.wfm-ai-check-item {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
align-items: baseline;
|
||||
padding: 1px 0;
|
||||
font-size: 12px;
|
||||
}
|
||||
.wfm-ai-check-icon { width: 14px; text-align: center; font-weight: 700; }
|
||||
.wfm-ai-check-item.is-pass .wfm-ai-check-icon { color: #4caf50; }
|
||||
.wfm-ai-check-item.is-uncertain .wfm-ai-check-icon { color: #e5a00d; }
|
||||
.wfm-ai-check-item.is-fail .wfm-ai-check-icon { color: #e05252; }
|
||||
.wfm-ai-check-label { flex: 1; }
|
||||
.wfm-ai-check-p { font-size: 11px; opacity: 0.7; font-variant-numeric: tabular-nums; }
|
||||
.wfm-ai-check-quality { margin-top: 4px; font-size: 11px; opacity: 0.8; }
|
||||
|
||||
/* "Built-in" badge in the skill manager list */
|
||||
.wfm-ai-skill-badge {
|
||||
margin-left: 6px;
|
||||
padding: 0 5px;
|
||||
border-radius: 3px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
font-size: 10px;
|
||||
opacity: 0.75;
|
||||
}
|
||||
|
||||
.wfm-ai-chat-svg-wrapper {
|
||||
margin-top: 6px;
|
||||
display: flex;
|
||||
@@ -6081,3 +6375,124 @@ body {
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
}
|
||||
|
||||
/* Workflow detail modal — GenerateUI compatibility check row */
|
||||
.wfm-compat-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
flex-wrap: wrap;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.wfm-compat-result {
|
||||
display: inline-flex;
|
||||
gap: 6px;
|
||||
flex-wrap: wrap;
|
||||
font-size: 12px;
|
||||
color: var(--wfm-text-secondary);
|
||||
}
|
||||
.wfm-compat-badge {
|
||||
position: relative;
|
||||
display: inline-block;
|
||||
padding: 2px 8px;
|
||||
border-radius: 10px;
|
||||
font-size: 12px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
background: var(--wfm-bg);
|
||||
color: var(--wfm-text);
|
||||
cursor: help;
|
||||
}
|
||||
.wfm-compat-ok { border-color: #3a8a3a; }
|
||||
.wfm-compat-warn { border-color: #c08a20; }
|
||||
.wfm-compat-error { border-color: var(--wfm-danger, #c04040); }
|
||||
.wfm-compat-skip { opacity: 0.6; }
|
||||
.wfm-compat-badge:hover::after,
|
||||
.wfm-compat-badge:focus::after {
|
||||
content: attr(data-tip);
|
||||
position: absolute;
|
||||
left: 0;
|
||||
top: calc(100% + 4px);
|
||||
z-index: 1000;
|
||||
width: max-content;
|
||||
max-width: min(520px, 80vw);
|
||||
padding: 8px 10px;
|
||||
border-radius: 6px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
background: var(--wfm-bg-secondary, var(--wfm-bg));
|
||||
color: var(--wfm-text);
|
||||
font-size: 12px;
|
||||
line-height: 1.5;
|
||||
white-space: pre-line;
|
||||
text-align: left;
|
||||
box-shadow: 0 4px 14px rgba(0, 0, 0, 0.4);
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
/* Side panel — format + GenerateUI compatibility badges row (below the title) */
|
||||
.wfm-side-compat-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
flex-wrap: wrap;
|
||||
padding: 4px 10px 6px;
|
||||
min-height: 26px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.wfm-side-compat-row .wfm-format-badge:empty { display: none; }
|
||||
/* The side panel clips overflow, so anchor tips to the whole row and span the panel width */
|
||||
.wfm-side-compat-row { position: relative; }
|
||||
.wfm-compat-result--right .wfm-compat-badge { position: static; }
|
||||
.wfm-compat-result--right .wfm-compat-badge:hover::after,
|
||||
.wfm-compat-result--right .wfm-compat-badge:focus::after {
|
||||
left: 6px;
|
||||
right: 6px;
|
||||
top: 100%;
|
||||
width: auto;
|
||||
max-width: none;
|
||||
}
|
||||
.wfm-compat-row .wfm-format-badge:empty { display: none; }
|
||||
|
||||
/* GenerateUI Settings tab: Switches column (true/false inputs) */
|
||||
.wfm-settings-switches {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
max-height: 420px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.wfm-settings-switch {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 4px 6px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
}
|
||||
.wfm-settings-switch:hover {
|
||||
background: var(--wfm-surface);
|
||||
}
|
||||
.wfm-settings-switch-text {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
.wfm-settings-switch-node {
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.wfm-settings-switch-key {
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 11px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.wfm-settings-switch-state {
|
||||
font-family: monospace;
|
||||
font-size: 11px;
|
||||
color: var(--wfm-text-secondary);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
/* TagSpace タブ */
|
||||
#wfm-tab-tagspace.active { overflow: hidden; }
|
||||
.wfm-ts { display: flex; flex-direction: column; height: 100%; min-height: 0; }
|
||||
.wfm-ts-toolbar { display: flex; align-items: center; gap: 10px; flex-wrap: wrap; padding: 8px 12px; border-bottom: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-ts-num { display: inline-flex; align-items: center; gap: 4px; font-size: 12px; color: var(--wfm-text-secondary, #aaa); }
|
||||
.wfm-ts-num input { width: 72px; }
|
||||
.wfm-ts-sep { width: 1px; height: 18px; background: var(--wfm-border, #444); }
|
||||
.wfm-ts-check { display: inline-flex; align-items: center; gap: 4px; font-size: 12px; cursor: pointer; }
|
||||
.wfm-ts-swatch { display: inline-block; width: 9px; height: 9px; border-radius: 50%; margin-right: 3px; }
|
||||
.wfm-ts-legend { display: inline-flex; gap: 10px; font-size: 11px; color: var(--wfm-text-secondary, #aaa); }
|
||||
.wfm-ts-search { margin-left: auto; width: 240px; max-width: 100%; }
|
||||
.wfm-ts-status { padding: 4px 12px; font-size: 12px; color: var(--wfm-text-secondary, #aaa); min-height: 22px; }
|
||||
.wfm-ts-status-error { color: var(--wfm-danger, #ef4444); }
|
||||
.wfm-ts-main { flex: 1; display: flex; min-height: 0; }
|
||||
.wfm-ts-view { position: relative; flex: 1; min-width: 0; overflow: hidden; background: var(--wfm-bg, #111); }
|
||||
.wfm-ts-view canvas { display: block; width: 100%; height: 100%; }
|
||||
.wfm-ts-labels { position: absolute; inset: 0; pointer-events: none; overflow: hidden; }
|
||||
.wfm-ts-label { position: absolute; left: 0; top: 0; margin: -22px 0 0 6px; padding: 1px 6px; border-radius: 4px; font-size: 11px; white-space: nowrap; background: rgba(0, 0, 0, 0.6); color: #fff; }
|
||||
.wfm-ts-label-chip { background: var(--wfm-primary, #6366f1); }
|
||||
.wfm-ts-label-near { background: rgba(8, 145, 178, 0.85); }
|
||||
.wfm-ts-label-hover { background: rgba(255, 255, 255, 0.92); color: #111; }
|
||||
.wfm-ts-tooltip { position: absolute; z-index: 2; display: flex; gap: 8px; max-width: 360px; padding: 6px 8px; border-radius: 6px; font-size: 12px; pointer-events: none; background: var(--wfm-surface, #222); border: 1px solid var(--wfm-border, #444); box-shadow: 0 4px 16px rgba(0, 0, 0, 0.4); }
|
||||
.wfm-ts-tooltip img { width: 64px; height: 64px; object-fit: cover; border-radius: 4px; flex: none; }
|
||||
.wfm-ts-tip-body { margin-top: 2px; color: var(--wfm-text-secondary, #aaa); }
|
||||
.wfm-ts-empty { position: absolute; inset: 0; display: flex; align-items: center; justify-content: center; padding: 24px; text-align: center; color: var(--wfm-text-secondary, #aaa); pointer-events: none; }
|
||||
.wfm-ts-hint { position: absolute; left: 10px; bottom: 8px; font-size: 11px; color: var(--wfm-text-secondary, #aaa); opacity: 0.8; pointer-events: none; }
|
||||
.wfm-ts-side { width: 300px; flex: none; display: flex; flex-direction: column; gap: 8px; padding: 8px; overflow: hidden; border-left: 1px solid var(--wfm-border, #444); }
|
||||
/* プロンプト列(右端): チップは中身の高さ(最大40%)、出力欄が残りの高さいっぱいに伸びる */
|
||||
.wfm-ts-side-prompt .wfm-ts-chips { flex: 0 1 auto; max-height: 40%; align-content: flex-start; }
|
||||
.wfm-ts-side-prompt .wfm-ts-output { flex: 1; min-height: 120px; resize: none; }
|
||||
.wfm-ts-panel { display: flex; flex-direction: column; gap: 6px; padding: 8px; border: 1px solid var(--wfm-border, #444); border-radius: var(--wfm-radius, 6px); background: var(--wfm-surface, transparent); min-height: 0; }
|
||||
.wfm-ts-panel h4 { margin: 0; font-size: 12px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.wfm-ts-panel-grow { flex: 1; }
|
||||
.wfm-ts-muted { color: var(--wfm-text-secondary, #aaa); font-size: 11px; }
|
||||
.wfm-ts-detail { display: flex; flex-direction: column; gap: 4px; font-size: 12px; max-height: 260px; overflow: auto; }
|
||||
.wfm-ts-detail-head { position: sticky; top: 0; z-index: 1; display: flex; align-items: center; gap: 6px; padding-bottom: 2px; background: var(--wfm-surface, #222); }
|
||||
.wfm-ts-detail-title { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.wfm-ts-detail-head .wfm-btn { flex: none; }
|
||||
.wfm-ts-detail-thumb { max-width: 100%; max-height: 140px; object-fit: contain; align-self: flex-start; border-radius: 4px; }
|
||||
.wfm-ts-detail-text { white-space: pre-wrap; word-break: break-word; }
|
||||
.wfm-ts-pick-list { display: flex; flex-wrap: wrap; gap: 3px; }
|
||||
.wfm-ts-full summary { cursor: pointer; color: var(--wfm-text-secondary, #999); font-size: 11px; }
|
||||
.wfm-ts-pick { padding: 1px 7px; border-radius: 10px; font-size: 11px; cursor: pointer; color: inherit; background: var(--wfm-bg-secondary, #333); border: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-ts-pick:hover { border-color: #f472b6; }
|
||||
.wfm-ts-pick.on { background: #f472b6; border-color: #f472b6; color: #1a1a1a; }
|
||||
/* タグが多くても検索ボタンが見えるよう、詳細パネルの下端に固定する */
|
||||
.wfm-ts-pick-actions { position: sticky; bottom: 0; order: 10; display: flex; gap: 6px; padding-top: 4px; background: var(--wfm-surface, #222); }
|
||||
.wfm-ts-path { word-break: break-all; }
|
||||
.wfm-ts-list { flex: 1; min-height: 0; overflow: auto; display: flex; flex-direction: column; gap: 2px; }
|
||||
.wfm-ts-row { display: flex; align-items: center; gap: 6px; padding: 2px 4px; border-radius: 4px; font-size: 12px; cursor: pointer; }
|
||||
.wfm-ts-row:hover { background: var(--wfm-surface-hover, rgba(255, 255, 255, 0.06)); }
|
||||
.wfm-ts-row img { width: 28px; height: 28px; object-fit: cover; border-radius: 3px; flex: none; }
|
||||
.wfm-ts-dot { width: 10px; height: 10px; border-radius: 50%; flex: none; margin: 0 9px; }
|
||||
.wfm-ts-row-label { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.wfm-ts-row-add { flex: none; width: 22px; height: 22px; border-radius: 4px; border: 1px solid var(--wfm-border, #444); background: transparent; color: var(--wfm-text, #ddd); cursor: pointer; }
|
||||
.wfm-ts-row-add.on { background: var(--wfm-primary, #6366f1); color: #fff; border-color: transparent; }
|
||||
.wfm-ts-row-add-none { flex: none; width: 22px; }
|
||||
.wfm-ts-row img.wfm-ts-row-image { border-radius: 50%; }
|
||||
.wfm-ts-chips { display: flex; flex-wrap: wrap; gap: 4px; max-height: 110px; overflow: auto; }
|
||||
.wfm-ts-chip { display: inline-flex; align-items: center; gap: 2px; padding: 1px 2px 1px 7px; border-radius: 10px; font-size: 11px; background: var(--wfm-bg-secondary, #333); border: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-ts-chip-card { border-color: var(--wfm-primary, #6366f1); }
|
||||
.wfm-ts-chip-image { border-color: #f472b6; }
|
||||
.wfm-ts-chip button { border: none; background: transparent; color: inherit; cursor: pointer; padding: 0 4px; font-size: 12px; }
|
||||
.wfm-ts-output { width: 100%; resize: vertical; font-size: 12px; box-sizing: border-box; }
|
||||
.wfm-ts-actions { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||
.wfm-ts-zoom { position: fixed; z-index: 1000; width: 360px; height: 360px; display: flex; align-items: center; justify-content: center; padding: 4px; box-sizing: border-box; pointer-events: none; border-radius: 8px; background: var(--wfm-surface, #222); border: 1px solid var(--wfm-border, #444); box-shadow: 0 8px 28px rgba(0, 0, 0, 0.55); }
|
||||
.wfm-ts-zoom img { max-width: 100%; max-height: 100%; object-fit: contain; border-radius: 4px; }
|
||||
@@ -821,6 +821,21 @@
|
||||
border-top: 1px solid var(--wfm-border);
|
||||
}
|
||||
|
||||
/* クロップ関連ボタンを左寄せにし、｜区切りの右に音声変換を置く */
|
||||
.wfm-video-edit-crop-head {
|
||||
justify-content: flex-start;
|
||||
}
|
||||
|
||||
.wfm-video-edit-crop-head > span:first-child {
|
||||
margin-right: 4px;
|
||||
}
|
||||
|
||||
.wfm-video-edit-head-sep {
|
||||
color: var(--wfm-text-secondary);
|
||||
font-weight: 400;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.wfm-video-edit-crop-row {
|
||||
display: flex;
|
||||
align-items: end;
|
||||
@@ -992,12 +1007,35 @@
|
||||
}
|
||||
|
||||
.wfm-video-source-panel {
|
||||
width: 220px;
|
||||
width: 320px;
|
||||
flex: none;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* ファイル名は1行(はみ出しは省略)、操作ボタン3つは折り返さず横一列 */
|
||||
.wfm-video-source-status-line {
|
||||
margin-top: 6px;
|
||||
min-height: 1.4em;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.wfm-video-source-actions {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
margin-top: 6px;
|
||||
}
|
||||
|
||||
.wfm-video-source-actions .wfm-btn {
|
||||
flex: 1 1 auto;
|
||||
min-width: 0;
|
||||
padding-left: 6px;
|
||||
padding-right: 6px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.wfm-video-pane-header {
|
||||
font-weight: 600;
|
||||
margin-bottom: 4px;
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import { openModal, closeModal, showToast } from "./app.js";
|
||||
import { t } from "./i18n.js";
|
||||
import { readJsonStorage, getAiBackendDefaultUrl, unslothProxy, unloadAiModel } from "./util.js";
|
||||
import { unloadAllOllamaModels } from "./vram-prepare.js";
|
||||
|
||||
function loadCfg(storageKey) {
|
||||
return readJsonStorage(storageKey);
|
||||
@@ -176,6 +177,8 @@ export function openAiBackendSettingsModal(storageKey, title, onSaved) {
|
||||
document.getElementById("wfm-aism-unload-btn")?.addEventListener("click", async () => {
|
||||
const url = urlInput.value.trim();
|
||||
const model = modelSelect.value;
|
||||
// Ollama: unload every loaded model incl. decision models (same server step as Settings → VRAM Management)
|
||||
if (currentBackend() === "ollama") { await unloadAllOllamaModels(url); return; }
|
||||
if (!model) { showToast(t("aiUnloadNoModel"), "error"); return; }
|
||||
try {
|
||||
await unloadAiModel(url, currentBackend(), model);
|
||||
|
||||
+975
-31
File diff suppressed because it is too large
Load Diff
+56
-6
@@ -112,7 +112,7 @@ function initTabs() {
|
||||
|
||||
function applyI18nToHtml() {
|
||||
// Tab labels
|
||||
const tabMap = { workflow: "tabWorkflow", nodes: "tabNodes", models: "tabModels", generate: "tabGenerate", prompt: "tabPrompt", gallery: "tabGallery", "image-edit": "tabImageEdit", video: "tabVideo", settings: "tabSettings", help: "tabHelp", ai: "tabAi", tagger: "tabTagger" };
|
||||
const tabMap = { workflow: "tabWorkflow", nodes: "tabNodes", models: "tabModels", generate: "tabGenerate", prompt: "tabPrompt", gallery: "tabGallery", "image-edit": "tabImageEdit", video: "tabVideo", settings: "tabSettings", help: "tabHelp", ai: "tabAi", tagger: "tabTagger", tagspace: "tabTagSpace" };
|
||||
document.querySelectorAll(".wfm-tab").forEach((tab) => {
|
||||
const key = tabMap[tab.dataset.tab];
|
||||
if (key) tab.textContent = t(key);
|
||||
@@ -249,6 +249,8 @@ function applyI18nToHtml() {
|
||||
if (galleryBulkPsd) galleryBulkPsd.textContent = t("galleryBulkPsd");
|
||||
const galleryBulkEditLayers = document.getElementById("wfm-gallery-bulk-edit-layers");
|
||||
if (galleryBulkEditLayers) galleryBulkEditLayers.textContent = t("galleryBulkEditLayers");
|
||||
const galleryBulkConvert = document.getElementById("wfm-gallery-bulk-convert");
|
||||
if (galleryBulkConvert) galleryBulkConvert.textContent = t("galleryBulkConvert");
|
||||
const galleryBulkDelete = document.getElementById("wfm-gallery-bulk-delete");
|
||||
if (galleryBulkDelete) galleryBulkDelete.textContent = t("galleryBulkDelete");
|
||||
const galleryOpenTaggerBtn = document.getElementById("wfm-gallery-open-tagger-btn");
|
||||
@@ -373,6 +375,7 @@ function applyI18nToHtml() {
|
||||
aiVlmTask.querySelectorAll("option").forEach(opt => {
|
||||
if (opt.value === "describe") opt.textContent = t("aiVlmDescribe");
|
||||
if (opt.value === "prompt") opt.textContent = t("aiVlmPromptCreate");
|
||||
if (opt.value === "ocr") opt.textContent = t("aiVlmOcr");
|
||||
});
|
||||
}
|
||||
const aiVlmRunBtn = document.getElementById("wfm-ai-vlm-run");
|
||||
@@ -381,6 +384,17 @@ function applyI18nToHtml() {
|
||||
if (aiVlmResult) aiVlmResult.placeholder = t("aiVlmResultPlaceholder");
|
||||
const aiVlmCopyBtn = document.getElementById("wfm-ai-vlm-copy");
|
||||
if (aiVlmCopyBtn) aiVlmCopyBtn.textContent = t("aiCopyBtn");
|
||||
const aiVlmToTrans = document.getElementById("wfm-ai-vlm-to-trans");
|
||||
if (aiVlmToTrans) aiVlmToTrans.textContent = t("aiVlmToTranslation");
|
||||
const aiVlmToChat = document.getElementById("wfm-ai-vlm-to-chat");
|
||||
if (aiVlmToChat) aiVlmToChat.textContent = t("aiVlmToChat");
|
||||
const aiOcrModelLabel = document.getElementById("wfm-ai-ocr-model-label");
|
||||
if (aiOcrModelLabel) aiOcrModelLabel.textContent = t("aiOcrModelLabel");
|
||||
const aiOcrBoxesLabel = document.getElementById("wfm-ai-ocr-boxes-label");
|
||||
if (aiOcrBoxesLabel) aiOcrBoxesLabel.textContent = t("aiOcrShowBoxes");
|
||||
const aiOcrRefresh = document.getElementById("wfm-ai-ocr-refresh");
|
||||
if (aiOcrRefresh) aiOcrRefresh.title = t("aiOcrRefresh");
|
||||
document.querySelectorAll(".wfm-ai-pane-collapse-btn").forEach((b) => { b.title = t("aiPaneCollapse"); });
|
||||
const aiBackendTitle = document.getElementById("wfm-ai-settings-backend-title");
|
||||
if (aiBackendTitle) aiBackendTitle.textContent = t("aiSettingsBackend");
|
||||
const aiConnTitle = document.getElementById("wfm-ai-settings-conn-title");
|
||||
@@ -414,6 +428,24 @@ function applyI18nToHtml() {
|
||||
if (aiSaveBtn) aiSaveBtn.textContent = t("aiSettingsSaveBtn");
|
||||
const aiChatImgGenLabel = document.getElementById("wfm-ai-chat-imggen-label-text");
|
||||
if (aiChatImgGenLabel) aiChatImgGenLabel.textContent = t("aiChatImgGenToggle");
|
||||
const aiChatCheckLabelText = document.getElementById("wfm-ai-chat-check-label-text");
|
||||
if (aiChatCheckLabelText) aiChatCheckLabelText.textContent = t("aiChatCheckToggle");
|
||||
const aiChatCheckLabel = document.getElementById("wfm-ai-chat-check-label");
|
||||
if (aiChatCheckLabel) aiChatCheckLabel.title = t("aiChatCheckToggleTip");
|
||||
const aiCheckTitle = document.getElementById("wfm-ai-settings-check-title");
|
||||
if (aiCheckTitle) aiCheckTitle.textContent = t("aiSettingsCheckTitle");
|
||||
const aiJudgeLabel = document.getElementById("wfm-ai-judge-model-label");
|
||||
if (aiJudgeLabel) aiJudgeLabel.textContent = t("aiJudgeModelLabel");
|
||||
const aiJudgeRefresh = document.getElementById("wfm-ai-judge-refresh-btn");
|
||||
if (aiJudgeRefresh) aiJudgeRefresh.textContent = t("aiJudgeRefresh");
|
||||
const aiRoundsLabel = document.getElementById("wfm-ai-check-rounds-label");
|
||||
if (aiRoundsLabel) aiRoundsLabel.textContent = t("aiCheckRoundsLabel");
|
||||
const aiRoundsManual = document.querySelector("#wfm-ai-check-rounds-select option[value='0']");
|
||||
if (aiRoundsManual) aiRoundsManual.textContent = t("aiCheckRoundsManual");
|
||||
const aiSkillImportText = document.getElementById("wfm-ai-skill-import-text");
|
||||
if (aiSkillImportText) aiSkillImportText.textContent = t("aiSkillImportText");
|
||||
const aiSkillImportLabel = document.getElementById("wfm-ai-skill-import-label");
|
||||
if (aiSkillImportLabel) aiSkillImportLabel.title = t("aiSkillImportTip");
|
||||
const aiChatGenTitle = document.getElementById("wfm-ai-settings-chatgen-title");
|
||||
if (aiChatGenTitle) aiChatGenTitle.textContent = t("aiSettingsChatGenTitle");
|
||||
const aiChatGenDedicatedLabel = document.getElementById("wfm-ai-chatgen-dedicated-label");
|
||||
@@ -472,7 +504,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-about-1": "helpAbout1", "wfm-help-about-2": "helpAbout2",
|
||||
"wfm-help-wf-1": "helpWf1", "wfm-help-wf-2": "helpWf2", "wfm-help-wf-3": "helpWf3",
|
||||
"wfm-help-wf-4": "helpWf4", "wfm-help-wf-5": "helpWf5", "wfm-help-wf-6": "helpWf6",
|
||||
"wfm-help-wf-7": "helpWf7",
|
||||
"wfm-help-wf-7": "helpWf7", "wfm-help-wf-8": "helpWf8",
|
||||
"wfm-help-gen-1": "helpGen1", "wfm-help-gen-2": "helpGen2",
|
||||
"wfm-help-gen-3": "helpGen3", "wfm-help-gen-4": "helpGen4",
|
||||
"wfm-help-gen-5": "helpGen5", "wfm-help-gen-6": "helpGen6", "wfm-help-gen-7": "helpGen7",
|
||||
@@ -490,6 +522,8 @@ function applyI18nToHtml() {
|
||||
"wfm-help-gen-29": "helpGen29",
|
||||
"wfm-help-gen-30": "helpGen30",
|
||||
"wfm-help-gen-31": "helpGen31",
|
||||
"wfm-help-gen-32": "helpGen32",
|
||||
"wfm-help-gen-33": "helpGen33",
|
||||
"wfm-help-feeder-title": "helpFeederTitle",
|
||||
"wfm-help-feeder-desc": "helpFeederDesc",
|
||||
"wfm-help-feeder-imgloop-title": "helpFeederImgloopTitle",
|
||||
@@ -523,7 +557,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-lab-plan-5": "helpLabPlan5", "wfm-help-lab-plan-6": "helpLabPlan6",
|
||||
"wfm-help-lab-plan-7": "helpLabPlan7", "wfm-help-lab-plan-8": "helpLabPlan8",
|
||||
"wfm-help-lab-plan-9": "helpLabPlan9", "wfm-help-lab-plan-10": "helpLabPlan10",
|
||||
"wfm-help-lab-plan-11": "helpLabPlan11",
|
||||
"wfm-help-lab-plan-11": "helpLabPlan11", "wfm-help-lab-plan-12": "helpLabPlan12",
|
||||
"wfm-help-lab-card-results-title": "helpLabCardResultsTitle",
|
||||
"wfm-help-lab-results-1": "helpLabResults1", "wfm-help-lab-results-2": "helpLabResults2",
|
||||
"wfm-help-lab-results-3": "helpLabResults3",
|
||||
@@ -557,7 +591,8 @@ function applyI18nToHtml() {
|
||||
"wfm-help-settings-18": "helpSettings18", "wfm-help-settings-19": "helpSettings19",
|
||||
"wfm-help-settings-20": "helpSettings20", "wfm-help-settings-21": "helpSettings21",
|
||||
"wfm-help-settings-22": "helpSettings22",
|
||||
"wfm-help-settings-23": "helpSettings23", "wfm-help-settings-24": "helpSettings24",
|
||||
"wfm-help-settings-23": "helpSettings23", "wfm-help-settings-24": "helpSettings24", "wfm-help-settings-25": "helpSettings25",
|
||||
"wfm-help-settings-26": "helpSettings26",
|
||||
"wfm-help-nodes-1": "helpNodes1", "wfm-help-nodes-2": "helpNodes2",
|
||||
"wfm-help-nodes-3": "helpNodes3", "wfm-help-nodes-4": "helpNodes4",
|
||||
"wfm-help-nodes-5": "helpNodes5", "wfm-help-nodes-6": "helpNodes6",
|
||||
@@ -584,10 +619,12 @@ function applyI18nToHtml() {
|
||||
"wfm-help-gallery-24": "helpGallery24", "wfm-help-gallery-25": "helpGallery25",
|
||||
"wfm-help-gallery-26": "helpGallery26",
|
||||
"wfm-help-gallery-28": "helpGallery28", "wfm-help-gallery-29": "helpGallery29",
|
||||
"wfm-help-gallery-30": "helpGallery30",
|
||||
"wfm-help-gallery-30": "helpGallery30", "wfm-help-gallery-31": "helpGallery31",
|
||||
"wfm-help-gallery-32": "helpGallery32", "wfm-help-gallery-33": "helpGallery33", "wfm-help-gallery-34": "helpGallery34", "wfm-help-gallery-35": "helpGallery35", "wfm-help-gallery-36": "helpGallery36", "wfm-help-gallery-37": "helpGallery37", "wfm-help-gallery-38": "helpGallery38",
|
||||
"wfm-help-gallery-card-styleprompt-title": "helpGalleryCardStyleprompt",
|
||||
"wfm-help-gallery-card-stylecatalog-title": "helpGalleryCardStylecatalog",
|
||||
"wfm-help-tagger-title": "helpTaggerTitle",
|
||||
"wfm-help-tagspace-title": "helpTagSpaceTitle", "wfm-help-tagspace-card-overview-title": "helpTagSpaceCardOverview", "wfm-help-tagspace-card-use-title": "helpTagSpaceCardUse", "wfm-help-tagspace-1": "helpTagSpace1", "wfm-help-tagspace-2": "helpTagSpace2", "wfm-help-tagspace-3": "helpTagSpace3", "wfm-help-tagspace-4": "helpTagSpace4", "wfm-help-tagspace-5": "helpTagSpace5", "wfm-help-tagspace-6": "helpTagSpace6", "wfm-help-tagspace-7": "helpTagSpace7", "wfm-help-tagspace-8": "helpTagSpace8", "wfm-help-tagspace-9": "helpTagSpace9", "wfm-help-tagspace-10": "helpTagSpace10",
|
||||
"wfm-help-tagger-1": "helpTagger1", "wfm-help-tagger-2": "helpTagger2",
|
||||
"wfm-help-tagger-3": "helpTagger3", "wfm-help-tagger-4": "helpTagger4",
|
||||
"wfm-help-tagger-5": "helpTagger5", "wfm-help-tagger-6": "helpTagger6",
|
||||
@@ -610,6 +647,10 @@ function applyI18nToHtml() {
|
||||
"wfm-help-ai-7": "helpAi7",
|
||||
"wfm-help-ai-8": "helpAi8",
|
||||
"wfm-help-ai-9": "helpAi9",
|
||||
"wfm-help-ai-10": "helpAi10",
|
||||
"wfm-help-ai-11": "helpAi11",
|
||||
"wfm-help-ai-12": "helpAi12",
|
||||
"wfm-help-ai-13": "helpAi13",
|
||||
"wfm-help-ai-card-layout-title": "helpAiCardLayout",
|
||||
"wfm-help-ai-card-translate-title": "helpAiCardTranslate",
|
||||
"wfm-help-ai-card-chat-title": "helpAiCardChat",
|
||||
@@ -618,6 +659,10 @@ function applyI18nToHtml() {
|
||||
"wfm-help-ai-card-skilltips-title": "helpAiCardSkillTips",
|
||||
"wfm-help-ai-card-tools-title": "helpAiCardTools",
|
||||
"wfm-help-ai-card-settings-title": "helpAiCardSettings",
|
||||
"wfm-help-ai-card-skilloptions-title": "helpAiCardSkillOptions",
|
||||
"wfm-help-ai-card-skillsmore-title": "helpAiCardSkillsMore",
|
||||
"wfm-help-ai-card-imagecheck-title": "helpAiCardImageCheck",
|
||||
"wfm-help-ai-card-ocr-title": "helpAiCardOcr",
|
||||
"wfm-help-imageedit-title": "helpImageEditTitle",
|
||||
"wfm-help-imageedit-tools-title": "helpImageEditToolsTitle",
|
||||
"wfm-help-imageedit-layer-title": "helpImageEditLayerTitle",
|
||||
@@ -640,6 +685,8 @@ function applyI18nToHtml() {
|
||||
"wfm-help-imageedit-26": "helpImageEdit26", "wfm-help-imageedit-27": "helpImageEdit27",
|
||||
"wfm-help-imageedit-28": "helpImageEdit28", "wfm-help-imageedit-29": "helpImageEdit29",
|
||||
"wfm-help-imageedit-30": "helpImageEdit30",
|
||||
"wfm-help-imageedit-draw-palette": "helpImageEditDrawPalette", "wfm-help-imageedit-draw-kbd": "helpImageEditDrawKbd",
|
||||
"wfm-help-imageedit-tabkbd-draw": "helpImageEditDrawKbd",
|
||||
"wfm-help-imageedit-31": "helpImageEdit31", "wfm-help-imageedit-32": "helpImageEdit32",
|
||||
"wfm-help-imageedit-33": "helpImageEdit33", "wfm-help-imageedit-34": "helpImageEdit34",
|
||||
"wfm-help-imageedit-7b": "helpImageEdit7b", "wfm-help-imageedit-7c": "helpImageEdit7c",
|
||||
@@ -682,7 +729,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-trouble-3": "helpTrouble3", "wfm-help-trouble-4": "helpTrouble4",
|
||||
"wfm-help-trouble-5": "helpTrouble5", "wfm-help-trouble-6": "helpTrouble6",
|
||||
"wfm-help-trouble-7": "helpTrouble7", "wfm-help-trouble-8": "helpTrouble8",
|
||||
"wfm-help-trouble-9": "helpTrouble9",
|
||||
"wfm-help-trouble-9": "helpTrouble9", "wfm-help-trouble-10": "helpTrouble10", "wfm-help-trouble-11": "helpTrouble11", "wfm-help-trouble-12": "helpTrouble12",
|
||||
"wfm-help-support-title": "helpSupportTitle",
|
||||
"wfm-help-support-desc": "helpSupportDesc",
|
||||
"wfm-help-github-desc": "helpGithubDesc",
|
||||
@@ -743,6 +790,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-trouble-card-connection-title": "helpTroubleCardConnection",
|
||||
"wfm-help-trouble-card-workflow-title": "helpTroubleCardWorkflow",
|
||||
"wfm-help-trouble-card-tagger-title": "helpTroubleCardTagger",
|
||||
"wfm-help-trouble-card-semantic-title": "helpTroubleCardSemantic",
|
||||
"wfm-help-video-title": "helpVideoTitle",
|
||||
"wfm-help-video-desc": "helpVideoDesc",
|
||||
"wfm-help-video-1": "helpVideo1", "wfm-help-video-2": "helpVideo2",
|
||||
@@ -840,6 +888,7 @@ import { initGalleryTab } from "./gallery-tab.js";
|
||||
import { initMetadataTab } from "./metadata-tab.js";
|
||||
import { initAiTab } from "./ai-tab.js";
|
||||
import { initTaggerTab } from "./tagger-tab.js";
|
||||
import { initTagSpaceTab } from "./tagspace-tab.js";
|
||||
import { imageEditTab } from "./image-edit-tab.js";
|
||||
import { initVideoTab } from "./video-tab.js";
|
||||
|
||||
@@ -892,6 +941,7 @@ document.addEventListener("DOMContentLoaded", () => {
|
||||
initGalleryTab();
|
||||
initAiTab();
|
||||
initTaggerTab();
|
||||
initTagSpaceTab();
|
||||
imageEditTab.init();
|
||||
initVideoTab();
|
||||
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
/**
|
||||
* chat-context.js — facts injected into the AI TOOL Chat system prompt by a skill's
|
||||
* `metadata.context` list ("workflow-detail", "logs"; "workflow" lives in ai-tab.js).
|
||||
*
|
||||
* Why: in the 2026-11-11 model check every local chat model answered a simple ComfyUI question
|
||||
* ("SDXL vs Flux") with confident factual errors. Diagnosis skills therefore must not rely on what
|
||||
* the model "knows" — they get verifiable data instead: the exact node graph, which node types
|
||||
* ComfyUI does not know (not installed), which model/option values are not in ComfyUI's own
|
||||
* lists (with the closest existing names), the last execution error from /history, and the tail
|
||||
* of the ComfyUI log. Everything is plain text, size-capped, and built fresh on every message.
|
||||
*/
|
||||
|
||||
import { comfyUI } from "./comfyui-client.js";
|
||||
import { comfyWorkflow } from "./comfyui-workflow.js";
|
||||
|
||||
const MAX_NODES = 60;
|
||||
const MAX_DETAIL_CHARS = 9000;
|
||||
const MAX_LOG_CHARS = 3500;
|
||||
|
||||
function _trunc(v, n = 80) {
|
||||
const s = String(v).replace(/\s+/g, " ");
|
||||
return s.length > n ? `${s.slice(0, n)}…` : s;
|
||||
}
|
||||
|
||||
// object_info input spec → the list of allowed values for a COMBO input, or null.
|
||||
// Two shapes exist: [[...options], {...}] (classic) and ["COMBO", {options: [...]}] (V3 nodes).
|
||||
function _comboOptions(spec) {
|
||||
if (!Array.isArray(spec)) return null;
|
||||
if (Array.isArray(spec[0])) return spec[0];
|
||||
if (spec[0] === "COMBO" && Array.isArray(spec[1]?.options)) return spec[1].options;
|
||||
return null;
|
||||
}
|
||||
|
||||
// Core ComfyUI link-only types. Custom node packs also define UI widgets with their own
|
||||
// upper-case type names (LOAD_3D viewport_state, COLOR, IMAGECOMPARE...), which an API-format
|
||||
// workflow legitimately omits, so only these well-known connection types are checked.
|
||||
const CONNECTION_TYPES = new Set([
|
||||
"MODEL", "CLIP", "VAE", "CONDITIONING", "LATENT", "IMAGE", "MASK", "CONTROL_NET",
|
||||
"CLIP_VISION", "CLIP_VISION_OUTPUT", "STYLE_MODEL", "UPSCALE_MODEL", "GLIGEN", "AUDIO",
|
||||
"VIDEO", "SIGMAS", "NOISE", "GUIDER", "SAMPLER", "LORA_MODEL", "HOOKS", "MESH", "VOXEL",
|
||||
]);
|
||||
|
||||
// A required input that can only be satisfied by a link (MODEL / CLIP / LATENT / IMAGE ...).
|
||||
function _isConnectionType(spec) {
|
||||
if (!Array.isArray(spec) || typeof spec[0] !== "string") return false;
|
||||
return CONNECTION_TYPES.has(spec[0]);
|
||||
}
|
||||
|
||||
function _inputSpec(info, name) {
|
||||
return info?.input?.required?.[name] ?? info?.input?.optional?.[name] ?? null;
|
||||
}
|
||||
|
||||
// Closest existing option names for a value that is not in the list (token overlap on the
|
||||
// file name, ignoring folders and extensions). Good enough to point at "you have v2, not v3".
|
||||
export function closestOptions(value, options, limit = 3) {
|
||||
const tokens = (s) => new Set(String(s).toLowerCase().replace(/\\/g, "/").split("/").pop()
|
||||
.replace(/\.(safetensors|ckpt|pt|pth|bin|gguf|sft)$/i, "")
|
||||
.split(/[^a-z0-9]+/).filter((t) => t.length > 1));
|
||||
const want = tokens(value);
|
||||
if (!want.size) return [];
|
||||
return options
|
||||
.filter((o) => typeof o === "string")
|
||||
.map((o) => {
|
||||
const have = tokens(o);
|
||||
let common = 0;
|
||||
for (const t of want) if (have.has(t)) common++;
|
||||
return { o, score: common / Math.max(want.size, have.size || 1) };
|
||||
})
|
||||
.filter((x) => x.score > 0)
|
||||
.sort((a, b) => b.score - a.score)
|
||||
.slice(0, limit)
|
||||
.map((x) => x.o);
|
||||
}
|
||||
|
||||
/**
|
||||
* Detailed, checked description of the workflow loaded in GenerateUI (API format).
|
||||
* Pure data → text; `objectInfo` is { classType: info } for the types used.
|
||||
*/
|
||||
export function describeWorkflowDetail(wf, objectInfo) {
|
||||
const entries = Object.entries(wf || {}).filter(([, n]) => n && typeof n === "object" && n.class_type);
|
||||
const lines = ["## Workflow facts (checked by Workflow Studio against this ComfyUI install)"];
|
||||
lines.push(`- ${entries.length} nodes.`);
|
||||
|
||||
const types = [...new Set(entries.map(([, n]) => n.class_type))];
|
||||
const missingTypes = types.filter((ty) => !objectInfo?.[ty]);
|
||||
lines.push(missingTypes.length
|
||||
? `- NOT INSTALLED node types (ComfyUI does not know them): ${missingTypes.join(", ")}`
|
||||
: "- All node types are installed.");
|
||||
|
||||
const problems = [];
|
||||
const nodeLines = [];
|
||||
const ids = new Set(entries.map(([id]) => id));
|
||||
for (const [id, node] of entries.slice(0, MAX_NODES)) {
|
||||
const info = objectInfo?.[node.class_type];
|
||||
const title = node._meta?.title && node._meta.title !== node.class_type ? ` "${_trunc(node._meta.title, 40)}"` : "";
|
||||
const values = [];
|
||||
const links = [];
|
||||
for (const [k, v] of Object.entries(node.inputs || {})) {
|
||||
if (Array.isArray(v) && v.length === 2 && (typeof v[0] === "string" || typeof v[0] === "number")) {
|
||||
const src = String(v[0]);
|
||||
links.push(`${k}<-#${src}`);
|
||||
if (!ids.has(src)) problems.push(`#${id} ${node.class_type}: input "${k}" is linked to node #${src}, which does not exist.`);
|
||||
continue;
|
||||
}
|
||||
if (v === null || typeof v === "object") continue;
|
||||
values.push(`${k}=${_trunc(v, k === "text" || k === "prompt" ? 120 : 70)}`);
|
||||
const options = info ? _comboOptions(_inputSpec(info, k)) : null;
|
||||
if (options && typeof v === "string" && options.length && !options.includes(v)) {
|
||||
const near = closestOptions(v, options);
|
||||
problems.push(`#${id} ${node.class_type}: ${k}="${v}" is NOT in ComfyUI's list for this input${near.length ? ` (closest existing: ${near.map((o) => `"${o}"`).join(", ")})` : " (no similar name found)"}.`);
|
||||
}
|
||||
}
|
||||
if (info) {
|
||||
// Only unconnected required CONNECTION inputs (MODEL, CLIP, LATENT, IMAGE, ...) are
|
||||
// reported: those make the run fail for sure. Missing widget values are usually filled
|
||||
// with defaults or are UI-only, and dynamic inputs (Autogrow "COMFY_*") are stored
|
||||
// under other keys - checking them produced false alarms on ~1/3 of real workflows.
|
||||
for (const [name, spec] of Object.entries(info.input?.required || {})) {
|
||||
if (name in (node.inputs || {}) || !_isConnectionType(spec)) continue;
|
||||
problems.push(`#${id} ${node.class_type}: required ${spec[0]} input "${name}" is not connected.`);
|
||||
}
|
||||
}
|
||||
nodeLines.push(`#${id} ${node.class_type}${title}${missingTypes.includes(node.class_type) ? " [NOT INSTALLED]" : ""}: ${[...values, ...links].join(", ") || "(no inputs)"}`);
|
||||
}
|
||||
if (entries.length > MAX_NODES) nodeLines.push(`… ${entries.length - MAX_NODES} more nodes not listed.`);
|
||||
|
||||
lines.push(problems.length ? `- Detected problems:\n${problems.slice(0, 30).map((p) => ` - ${p}`).join("\n")}` : "- No missing files, options or links detected.");
|
||||
lines.push("- Nodes (#id ClassType: widget values, input<-#source node):");
|
||||
let text = `${lines.join("\n")}\n${nodeLines.map((l) => ` ${l}`).join("\n")}`;
|
||||
if (text.length > MAX_DETAIL_CHARS) text = `${text.slice(0, MAX_DETAIL_CHARS)}\n … (truncated)`;
|
||||
return text;
|
||||
}
|
||||
|
||||
export async function buildWorkflowDetailContext() {
|
||||
const wf = comfyUI.currentWorkflow;
|
||||
if (!wf) return "## Workflow facts\nNo workflow is loaded in GenerateUI. Ask the user to load the workflow they are asking about in the GenerateUI tab.";
|
||||
const types = [...new Set(Object.values(wf).map((n) => n?.class_type).filter(Boolean))];
|
||||
let objectInfo = {};
|
||||
try {
|
||||
objectInfo = await comfyWorkflow.getObjectInfoForTypes(types);
|
||||
} catch (e) {
|
||||
return `## Workflow facts\nCould not read node definitions from ComfyUI (${e.message}), so installed nodes and files could not be checked.`;
|
||||
}
|
||||
return describeWorkflowDetail(wf, objectInfo);
|
||||
}
|
||||
|
||||
// ANSI colour codes from ComfyUI's coloured log lines
|
||||
const ANSI_RE = /\u001b\[[0-9;]*m/g;
|
||||
|
||||
/** Keep the most recent error block of the log (from the last Traceback / error line), else the tail. */
|
||||
export function extractLogTail(logText, maxChars = MAX_LOG_CHARS) {
|
||||
const lines = String(logText || "").replace(ANSI_RE, "").split(/\r?\n/).filter((l) => l.trim());
|
||||
let start = -1;
|
||||
for (let i = lines.length - 1; i >= 0; i--) {
|
||||
if (/Traceback \(most recent call last\)|!!! Exception|Error:|ERROR|Prompt outputs failed validation/.test(lines[i])) {
|
||||
start = i;
|
||||
// walk back to the beginning of a traceback block
|
||||
while (start > 0 && /^\s|Traceback|File "/.test(lines[start - 1]) && i - start < 40) start--;
|
||||
break;
|
||||
}
|
||||
}
|
||||
const picked = start >= 0 ? lines.slice(Math.max(0, start - 3), start + 40) : lines.slice(-25);
|
||||
let text = picked.join("\n");
|
||||
if (text.length > maxChars) text = `…${text.slice(-maxChars)}`;
|
||||
return { text, hasError: start >= 0 };
|
||||
}
|
||||
|
||||
/** Last execution error recorded in ComfyUI's /history (most recent first). */
|
||||
export function lastExecutionError(history) {
|
||||
const runs = Object.values(history || {});
|
||||
for (let i = runs.length - 1; i >= 0; i--) {
|
||||
const msgs = runs[i]?.status?.messages || [];
|
||||
const err = msgs.find((m) => m?.[0] === "execution_error")?.[1];
|
||||
if (err) return err;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function buildLogsContext() {
|
||||
const parts = ["## ComfyUI runtime facts"];
|
||||
try {
|
||||
const res = await fetch(`${comfyUI.baseUrl || ""}/history?max_items=5`);
|
||||
const err = res.ok ? lastExecutionError(await res.json()) : null;
|
||||
if (err) {
|
||||
const tb = Array.isArray(err.traceback) ? err.traceback.join("").split("\n").filter(Boolean).slice(-8).join("\n") : "";
|
||||
parts.push(`- Last execution error (from /history): node #${err.node_id} ${err.node_type}: ${err.exception_type}: ${_trunc(err.exception_message, 600)}${tb ? `\n Traceback tail:\n${tb}` : ""}`);
|
||||
} else {
|
||||
parts.push("- No execution error in the last 5 runs (/history).");
|
||||
}
|
||||
} catch (e) {
|
||||
parts.push(`- Could not read /history: ${e.message}`);
|
||||
}
|
||||
try {
|
||||
const res = await fetch(`${comfyUI.baseUrl || ""}/internal/logs`);
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const raw = await res.json();
|
||||
const { text, hasError } = extractLogTail(typeof raw === "string" ? raw : JSON.stringify(raw));
|
||||
parts.push(`- ComfyUI log (${hasError ? "most recent error block" : "last lines, no error found"}):\n${text}`);
|
||||
} catch (e) {
|
||||
parts.push(`- Could not read the ComfyUI log: ${e.message}`);
|
||||
}
|
||||
return parts.join("\n");
|
||||
}
|
||||
@@ -0,0 +1,300 @@
|
||||
/**
|
||||
* chat-image-check.js — AI TOOL Chat: judge generated images (and rendered SVGs) with a
|
||||
* vision-capable decision model, so the chat can fix only what is missing.
|
||||
*
|
||||
* Why a decision model instead of the chat VLM: a generator VLM answers pass/fail without a
|
||||
* probability and was measured to say "yes" to things that are not in the image (2026-11-11:
|
||||
* gemma4-uncensored e4b judged "holds a sword" = true). Ollama decision models with image input
|
||||
* (d3:4b, clef-flash, clef) answered 18/18 checklist questions correctly on the same image with a
|
||||
* clear gap between yes (0.75-1.0) and no (0.0-0.1), and return calibrated `noul` probabilities
|
||||
* that code can threshold. Only one image was measured, so results are shown to the user and
|
||||
* automatic fix rounds are off by default.
|
||||
*
|
||||
* Flow (driven by ai-tab.js):
|
||||
* buildChecklist() chat LLM turns the user's request into 3-7 yes/no visual questions
|
||||
* judgeImage() decision model answers them for the image (+ a quality score)
|
||||
* renderCheckCard() shows ✔ / ? / ✘ per item; failed + uncertain items are what gets fixed
|
||||
*/
|
||||
|
||||
import { t } from "./i18n.js";
|
||||
import { escapeHtml } from "./util.js";
|
||||
import { decide, noul, score } from "./decision-client.js";
|
||||
|
||||
const OLLAMA_DEFAULT_URL = "http://localhost:11434";
|
||||
export const DEFAULT_JUDGE_MODEL = "d3:4b";
|
||||
|
||||
// Probability bands for a noul answer (after flipping "expect: no" items). In the 2026-11-11
|
||||
// test runs with d3:4b, correct passes were >= 0.88 while wrong passes caused by attribute binding
|
||||
// ("Is the boy wearing a green hat?" when the girl wore it) landed at 0.70-0.79, so 0.85 is the
|
||||
// pass line and 0.3-0.85 counts as "uncertain" (fixed like a failure). clef answers yes a bit
|
||||
// lower (0.75+), so it reports more "uncertain" items.
|
||||
export const PASS_THRESHOLD = 0.85;
|
||||
export const FAIL_THRESHOLD = 0.3;
|
||||
|
||||
/** Judge connection: the AI TOOL Ollama URL when the chat backend is Ollama, else the default. */
|
||||
export function getJudgeConfig(aiSettings = {}) {
|
||||
const baseUrl = aiSettings.backend === "ollama" && aiSettings.backendUrl
|
||||
? aiSettings.backendUrl
|
||||
: OLLAMA_DEFAULT_URL;
|
||||
return {
|
||||
baseUrl: baseUrl.replace(/\/+$/, ""),
|
||||
model: aiSettings.chatJudgeModel || DEFAULT_JUDGE_MODEL,
|
||||
autoRounds: Math.max(0, Math.min(3, parseInt(aiSettings.chatCheckAutoRounds, 10) || 0)),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Ollama decision models (capabilities include "decision"), vision-capable ones first.
|
||||
* Returns [{ name, vision }]. Throws if Ollama can't be reached.
|
||||
*/
|
||||
export async function listJudgeModels(baseUrl) {
|
||||
const url = (baseUrl || OLLAMA_DEFAULT_URL).replace(/\/+$/, "");
|
||||
const res = await fetch(`${url}/api/tags`);
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const data = await res.json();
|
||||
const names = (data.models || [])
|
||||
.filter((m) => Array.isArray(m.capabilities) && m.capabilities.includes("decision"))
|
||||
.map((m) => m.name);
|
||||
const out = await Promise.all(names.map(async (name) => {
|
||||
try {
|
||||
const r = await fetch(`${url}/api/show`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ model: name }),
|
||||
});
|
||||
const d = r.ok ? await r.json() : {};
|
||||
const vision = (Array.isArray(d.capabilities) && d.capabilities.includes("vision"))
|
||||
|| !!(d.projector_info && Object.keys(d.projector_info).length);
|
||||
return { name, vision };
|
||||
} catch {
|
||||
return { name, vision: false };
|
||||
}
|
||||
}));
|
||||
return out.sort((a, b) => (b.vision - a.vision) || a.name.localeCompare(b.name));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Checklist
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const CHECKLIST_SCHEMA = {
|
||||
type: "object",
|
||||
properties: {
|
||||
items: {
|
||||
type: "array",
|
||||
items: {
|
||||
type: "object",
|
||||
properties: {
|
||||
question: { type: "string" },
|
||||
label: { type: "string" },
|
||||
expect: { type: "string", enum: ["yes", "no"] },
|
||||
},
|
||||
required: ["question", "label", "expect"],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: ["items"],
|
||||
};
|
||||
|
||||
const CHECKLIST_SYSTEM = `You turn an image request into a short visual checklist for an image-judging model.
|
||||
Return JSON only: {"items":[{"question":"...","label":"...","expect":"yes"}]}
|
||||
Rules:
|
||||
- 3 to 8 items, one concrete requirement per item.
|
||||
- "question": an English yes/no question that can be answered by looking at the image alone. Always name who or what it is about (e.g. "Is the girl wearing a white dress?", never "Is someone wearing a white dress?"). For positions, name both sides (e.g. "Is the white rabbit between the boy and the girl?").
|
||||
- "expect": "yes" when YES means the requirement is met.
|
||||
- The judging model often mixes up which person has which attribute. So when the request has two or more people or animals with different attributes, add up to 2 swap-check items with "expect": "no" that ask whether the attribute is on the WRONG one (e.g. the boy should wear the green hat -> "Is the girl wearing a green hat?" with "expect": "no").
|
||||
- "label": the requirement as a very short phrase in the language of the user's request, including who it is about (e.g. "男の子の緑の帽子", "帽子は女の子ではない").
|
||||
- Only check visible things the user actually asked for: subject and count, key attributes and colors, clothing, pose or action, setting, time of day or lighting, text that must appear.
|
||||
- Never check image quality, art style quality, or details the user did not ask for.`;
|
||||
|
||||
/** Pull the first JSON object out of a model reply (code fences / prose tolerated). */
|
||||
export function parseChecklist(text) {
|
||||
if (!text) return [];
|
||||
let data = null;
|
||||
const start = text.indexOf("{");
|
||||
const end = text.lastIndexOf("}");
|
||||
if (start >= 0 && end > start) {
|
||||
try { data = JSON.parse(text.slice(start, end + 1)); } catch { data = null; }
|
||||
}
|
||||
const items = Array.isArray(data?.items) ? data.items : [];
|
||||
return items
|
||||
.map((it) => (typeof it === "string" ? { question: it, label: it } : it))
|
||||
.filter((it) => it && typeof it.question === "string" && it.question.trim())
|
||||
.map((it) => ({
|
||||
question: it.question.trim().slice(0, 300),
|
||||
label: String(it.label || it.question).trim().slice(0, 80),
|
||||
expect: String(it.expect || "yes").toLowerCase() === "no" ? "no" : "yes",
|
||||
}))
|
||||
.slice(0, 8);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ask the chat LLM for the checklist. `chat(messages, extraSettings)` must return the reply text.
|
||||
* Falls back to one whole-prompt question if the model returns nothing usable.
|
||||
*/
|
||||
export async function buildChecklist(chat, request, prompt) {
|
||||
const user = `User request: ${request || "(none)"}\nPrompt used for generation: ${prompt || "(none)"}`;
|
||||
let items = [];
|
||||
try {
|
||||
const text = await chat(
|
||||
[{ role: "system", content: CHECKLIST_SYSTEM }, { role: "user", content: user }],
|
||||
{ format: CHECKLIST_SCHEMA, temperature: 0.2, thinkingMode: false },
|
||||
);
|
||||
items = parseChecklist(text);
|
||||
} catch { /* fall through to the fallback below */ }
|
||||
if (items.length === 0) {
|
||||
const desc = (request || prompt || "").replace(/\s+/g, " ").slice(0, 300);
|
||||
items = [{ question: `Does the image match this description: "${desc}"?`, label: desc.slice(0, 40), expect: "yes" }];
|
||||
}
|
||||
return items;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Judging
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Image URL / data URL → JPEG base64 (no prefix), downscaled so the longest side is <= maxSide. */
|
||||
export async function imageUrlToJpegBase64(src, maxSide = 768) {
|
||||
const img = await new Promise((resolve, reject) => {
|
||||
const el = new Image();
|
||||
el.onload = () => resolve(el);
|
||||
el.onerror = () => reject(new Error("image load failed"));
|
||||
el.src = src;
|
||||
});
|
||||
const scale = Math.min(1, maxSide / Math.max(img.naturalWidth || 1, img.naturalHeight || 1));
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = Math.max(1, Math.round((img.naturalWidth || maxSide) * scale));
|
||||
canvas.height = Math.max(1, Math.round((img.naturalHeight || maxSide) * scale));
|
||||
const ctx = canvas.getContext("2d");
|
||||
ctx.fillStyle = "#ffffff"; // transparent SVG/PNG areas → white, like a viewer would show
|
||||
ctx.fillRect(0, 0, canvas.width, canvas.height);
|
||||
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
|
||||
return canvas.toDataURL("image/jpeg", 0.9).split(",")[1] || "";
|
||||
}
|
||||
|
||||
/** SVG markup → JPEG base64 rendered at `size` px (white background, currentColor = black). */
|
||||
export async function svgToJpegBase64(svgCode, size = 512) {
|
||||
let markup = svgCode;
|
||||
// Render at a fixed size even when the SVG has no width/height (viewBox only).
|
||||
if (!/<svg[^>]*\swidth=/i.test(markup)) markup = markup.replace(/<svg\b/i, `<svg width="${size}" height="${size}"`);
|
||||
const dataUrl = `data:image/svg+xml;base64,${btoa(unescape(encodeURIComponent(markup)))}`;
|
||||
return imageUrlToJpegBase64(dataUrl, size);
|
||||
}
|
||||
|
||||
const QUALITY_LEVELS = ["poor", "acceptable", "excellent"];
|
||||
|
||||
/**
|
||||
* Judge one image against the checklist. The first request after ComfyUI generated an image can
|
||||
* fail while VRAM is still held, so it retries once after 4 s.
|
||||
* Returns { items: [{ question, label, p, status: "pass"|"uncertain"|"fail" }], quality, ms }.
|
||||
*/
|
||||
export async function judgeImage(imageBase64, items, cfg, { withQuality = true } = {}) {
|
||||
const questions = {};
|
||||
items.forEach((it, i) => { questions[`q${i + 1}`] = noul(it.question); });
|
||||
if (withQuality) questions.quality = score("Overall visual quality of the image", QUALITY_LEVELS);
|
||||
const opts = { backend: "ollama", baseUrl: cfg.baseUrl, model: cfg.model };
|
||||
const started = performance.now();
|
||||
let answers;
|
||||
try {
|
||||
answers = await decide("A generated image to be checked against the user's requirements.", questions, opts, [imageBase64]);
|
||||
} catch (e) {
|
||||
await new Promise((r) => setTimeout(r, 4000));
|
||||
answers = await decide("A generated image to be checked against the user's requirements.", questions, opts, [imageBase64]);
|
||||
}
|
||||
const results = items.map((it, i) => {
|
||||
const p = answers[`q${i + 1}`]?.noul;
|
||||
const raw = typeof p === "number" ? p : 0;
|
||||
// "expect: no" items (swap checks) are met when the answer is NO
|
||||
const prob = it.expect === "no" ? 1 - raw : raw;
|
||||
const status = prob >= PASS_THRESHOLD ? "pass" : prob < FAIL_THRESHOLD ? "fail" : "uncertain";
|
||||
return { ...it, p: prob, status };
|
||||
});
|
||||
let quality = null;
|
||||
if (withQuality && answers.quality) {
|
||||
const probs = answers.quality.probabilities || {};
|
||||
let best = null, bestP = -1;
|
||||
for (const [k, v] of Object.entries(probs)) if (v > bestP) { best = Number(k); bestP = v; }
|
||||
quality = { level: best, label: QUALITY_LEVELS[best] ?? null, value: answers.quality.score };
|
||||
}
|
||||
return { items: results, quality, ms: Math.round(performance.now() - started) };
|
||||
}
|
||||
|
||||
export function failedItems(result) {
|
||||
return (result?.items || []).filter((it) => it.status !== "pass");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SVG part list ("Parts: body, head (circle), beak" line written by the SVG skill)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function parseSvgParts(text) {
|
||||
const m = (text || "").match(/^\s*\**parts?\**\s*:\s*(.+)$/im);
|
||||
if (!m) return [];
|
||||
// Split on commas outside parentheses, then drop "(how it is drawn)" notes.
|
||||
const parts = [];
|
||||
let depth = 0, cur = "";
|
||||
for (const ch of m[1]) {
|
||||
if (ch === "(") depth++;
|
||||
if (ch === ")") depth = Math.max(0, depth - 1);
|
||||
if ((ch === "," || ch === ";") && depth === 0) { parts.push(cur); cur = ""; } else cur += ch;
|
||||
}
|
||||
parts.push(cur);
|
||||
return parts
|
||||
.map((p) => p.replace(/\([^)]*\)/g, "").replace(/[.*_`]/g, "").replace(/\s+/g, " ").trim())
|
||||
.filter((p) => p && p.length <= 40)
|
||||
.slice(0, 7);
|
||||
}
|
||||
|
||||
/** Checklist for a rendered SVG: one question per part plus "is it recognisable as <request>". */
|
||||
export function svgChecklist(parts, request) {
|
||||
const items = parts.map((p) => ({ question: `Does the drawing clearly show a ${p}?`, label: p, expect: "yes" }));
|
||||
const subject = (request || "").replace(/\s+/g, " ").trim().slice(0, 200);
|
||||
if (subject) items.unshift({ question: `Is this drawing recognizable as: "${subject}"?`, label: subject.slice(0, 40), expect: "yes" });
|
||||
return items.slice(0, 8);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Result card
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const STATUS_ICON = { pass: "✔", uncertain: "?", fail: "✘" };
|
||||
|
||||
/**
|
||||
* Render the judgement as a chat card. `actions` = [{ label, primary, onClick(btn) }].
|
||||
* Returns the card element.
|
||||
*/
|
||||
export function renderCheckCard(result, { model, round = 0, actions = [] } = {}) {
|
||||
const card = document.createElement("div");
|
||||
card.className = "wfm-ai-check-card";
|
||||
const failed = failedItems(result);
|
||||
const head = failed.length === 0 ? t("aiChatCheckPassed") : t("aiChatCheckFailedCount", failed.length);
|
||||
const roundText = round > 0 ? ` · ${t("aiChatCheckRound", round)}` : "";
|
||||
card.innerHTML = `
|
||||
<div class="wfm-ai-check-head ${failed.length ? "is-fail" : "is-pass"}">
|
||||
<span>${escapeHtml(head)}</span>
|
||||
<span class="wfm-ai-check-meta">${escapeHtml(t("aiChatCheckBy", model))}${escapeHtml(roundText)} · ${(result.ms / 1000).toFixed(1)}s</span>
|
||||
</div>
|
||||
<ul class="wfm-ai-check-list">
|
||||
${result.items.map((it) => `
|
||||
<li class="wfm-ai-check-item is-${it.status}" title="${escapeHtml(it.question)}">
|
||||
<span class="wfm-ai-check-icon">${STATUS_ICON[it.status]}</span>
|
||||
<span class="wfm-ai-check-label">${escapeHtml(it.label)}</span>
|
||||
<span class="wfm-ai-check-p">${Math.round(it.p * 100)}%</span>
|
||||
</li>`).join("")}
|
||||
</ul>
|
||||
${result.quality?.label ? `<div class="wfm-ai-check-quality">${escapeHtml(t("aiChatCheckQuality", t(`aiChatCheckQuality_${result.quality.label}`)))}</div>` : ""}
|
||||
`;
|
||||
if (actions.length) {
|
||||
const row = document.createElement("div");
|
||||
row.className = "wfm-ai-chat-msg-actions";
|
||||
for (const a of actions) {
|
||||
const btn = document.createElement("button");
|
||||
btn.className = `wfm-btn wfm-btn-sm${a.primary ? " wfm-btn-primary" : ""}`;
|
||||
btn.textContent = a.label;
|
||||
btn.addEventListener("click", () => a.onClick(btn));
|
||||
row.appendChild(btn);
|
||||
}
|
||||
card.appendChild(row);
|
||||
}
|
||||
return card;
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* chat-markdown.js — tiny, dependency-free Markdown renderer for AI TOOL Chat replies.
|
||||
*
|
||||
* Safety model: the whole input is HTML-escaped FIRST, then a small set of Markdown constructs is
|
||||
* turned into tags. No raw HTML from the model ever reaches the DOM, links are not rendered as
|
||||
* anchors (the model's URLs stay plain text), and images are never loaded. This keeps the
|
||||
* existing guarantees of the chat (it showed replies with textContent before) while making long
|
||||
* structured answers (lists, headings, code blocks) readable.
|
||||
*
|
||||
* Supported: fenced code blocks (```lang), inline `code`, **bold**, *italic*, # headings (h1-h4
|
||||
* rendered as h4-h6-ish sizes), - / * / 1. lists (one level), > quotes, --- rules, paragraphs and
|
||||
* line breaks. Anything else is left as escaped text.
|
||||
*/
|
||||
|
||||
function escapeHtml(s) {
|
||||
return String(s)
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, """)
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
// Inline formatting on already-escaped text. Code spans are cut out first so their content is
|
||||
// not touched by the bold/italic rules.
|
||||
function renderInline(escaped) {
|
||||
const codes = [];
|
||||
let s = escaped.replace(/`([^`\n]+)`/g, (_, c) => {
|
||||
codes.push(c);
|
||||
return `\u0000${codes.length - 1}\u0000`;
|
||||
});
|
||||
s = s.replace(/\*\*([^*\n]+)\*\*/g, "<strong>$1</strong>");
|
||||
s = s.replace(/__([^_\n]+)__/g, "<strong>$1</strong>");
|
||||
s = s.replace(/(^|[^*\w])\*(?=\S)([^*\n]*?\S)\*(?!\w)/g, "$1<em>$2</em>");
|
||||
s = s.replace(/\u0000(\d+)\u0000/g, (_, i) => `<code>${codes[Number(i)]}</code>`);
|
||||
return s;
|
||||
}
|
||||
|
||||
/** Markdown text → safe HTML string. */
|
||||
export function renderMarkdown(text) {
|
||||
const lines = String(text || "").replace(/\r\n?/g, "\n").split("\n");
|
||||
const out = [];
|
||||
let para = [];
|
||||
let list = null; // { tag: "ul"|"ol", items: [], start }
|
||||
|
||||
const flushPara = () => {
|
||||
if (para.length) {
|
||||
out.push(`<p>${para.map((l) => renderInline(escapeHtml(l))).join("<br>")}</p>`);
|
||||
para = [];
|
||||
}
|
||||
};
|
||||
const flushList = () => {
|
||||
if (list) {
|
||||
const start = list.tag === "ol" && list.start > 1 ? ` start="${list.start}"` : "";
|
||||
out.push(`<${list.tag}${start}>${list.items.map((i) => `<li>${renderInline(escapeHtml(i))}</li>`).join("")}</${list.tag}>`);
|
||||
list = null;
|
||||
}
|
||||
};
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
|
||||
const fence = line.match(/^\s*```\s*([\w+-]*)\s*$/);
|
||||
if (fence) {
|
||||
flushPara(); flushList();
|
||||
const lang = fence[1] || "";
|
||||
const body = [];
|
||||
i++;
|
||||
while (i < lines.length && !/^\s*```\s*$/.test(lines[i])) { body.push(lines[i]); i++; }
|
||||
const label = lang ? `<span class="wfm-md-code-lang">${escapeHtml(lang)}</span>` : "";
|
||||
out.push(`<pre class="wfm-md-pre">${label}<code>${escapeHtml(body.join("\n"))}</code></pre>`);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!line.trim()) { flushPara(); flushList(); continue; }
|
||||
|
||||
const heading = line.match(/^\s*(#{1,4})\s+(.+)$/);
|
||||
if (heading) {
|
||||
flushPara(); flushList();
|
||||
const level = Math.min(6, heading[1].length + 2); // # -> h3 ... #### -> h6
|
||||
out.push(`<h${level} class="wfm-md-h">${renderInline(escapeHtml(heading[2]))}</h${level}>`);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (/^\s*(-{3,}|\*{3,}|_{3,})\s*$/.test(line)) { flushPara(); flushList(); out.push("<hr>"); continue; }
|
||||
|
||||
if (line.trim().startsWith(">")) {
|
||||
flushPara(); flushList();
|
||||
out.push(`<blockquote>${renderInline(escapeHtml(line.trim().replace(/^>\s?/, "")))}</blockquote>`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const ul = line.match(/^\s*[-*+]\s+(.+)$/);
|
||||
const ol = line.match(/^\s*(\d+)[.)]\s+(.+)$/);
|
||||
if (ul || ol) {
|
||||
flushPara();
|
||||
const tag = ul ? "ul" : "ol";
|
||||
if (list && list.tag !== tag) flushList();
|
||||
// A numbered list interrupted by bullets keeps counting from the source number
|
||||
if (!list) list = { tag, items: [], start: ol ? Math.min(Number(ol[1]) || 1, 9999) : 1 };
|
||||
list.items.push(ul ? ul[1] : ol[2]);
|
||||
continue;
|
||||
}
|
||||
|
||||
flushList();
|
||||
para.push(line);
|
||||
}
|
||||
flushPara(); flushList();
|
||||
return out.join("");
|
||||
}
|
||||
@@ -3,6 +3,9 @@
|
||||
* Handles communication with ComfyUI server (same origin)
|
||||
*/
|
||||
|
||||
import { prepareVramForGeneration } from "./vram-prepare.js";
|
||||
import { t } from "./i18n.js";
|
||||
|
||||
function _uuid() {
|
||||
return "xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(/[xy]/g, (c) => {
|
||||
const r = (Math.random() * 16) | 0;
|
||||
@@ -192,14 +195,34 @@ export const comfyUI = {
|
||||
return this._fetchModelList(["KSampler"], "scheduler");
|
||||
},
|
||||
|
||||
// パートナー(API)ノード（Nano Banana等）は /prompt の extra_data にComfyの認証情報が必要。ComfyUI画面は
|
||||
// ログイン情報を自動で付けるが、本アプリの直接fetchは付けないため、サーバー側の COMFY_API_KEY(.env) が
|
||||
// 設定されていれば WFSサーバー経由で投入してキーを付与する（キーはブラウザへ出さない）。
|
||||
async isPartnerKeyConfigured() {
|
||||
const now = Date.now();
|
||||
if (this._partnerKey && now - this._partnerKey.at < 60000) return this._partnerKey.value;
|
||||
let value = false;
|
||||
try {
|
||||
const res = await fetch("/api/wfm/partner/status");
|
||||
if (res.ok) value = !!(await res.json()).configured;
|
||||
} catch { /* 無視 */ }
|
||||
this._partnerKey = { value, at: now };
|
||||
return value;
|
||||
},
|
||||
|
||||
// Generation
|
||||
async queuePrompt(workflow, extraData = null) {
|
||||
// Settings tab "VRAM Management": unload Ollama models to free VRAM (no-op when off, never throws)
|
||||
await prepareVramForGeneration();
|
||||
const body = {
|
||||
prompt: workflow,
|
||||
client_id: this.clientId,
|
||||
};
|
||||
if (extraData) body.extra_data = extraData;
|
||||
const res = await fetch(`${this.baseUrl}/prompt`, {
|
||||
// 接続先が同一オリジン（このComfyUI）のときだけ中継を使う（別サーバーへキーを送らない）
|
||||
const sameOrigin = !this.baseUrl || this.baseUrl === location.origin;
|
||||
const viaProxy = sameOrigin && await this.isPartnerKeyConfigured();
|
||||
const res = await fetch(viaProxy ? "/api/wfm/partner/prompt" : `${this.baseUrl}/prompt`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
@@ -247,7 +270,10 @@ export const comfyUI = {
|
||||
}
|
||||
}
|
||||
if (msg.type === "execution_error" && msg.data?.prompt_id === promptId) {
|
||||
finish(false, new Error(msg.data.exception_message || "Execution error"));
|
||||
let message = msg.data.exception_message || "Execution error";
|
||||
// パートナー(API)ノードの認証エラーには、設定方法を案内する
|
||||
if (/Unauthorized|login first/i.test(message)) message = `${message.trim()} ${t("partnerAuthHint")}`;
|
||||
finish(false, new Error(message));
|
||||
}
|
||||
if (msg.type === "execution_interrupted" && msg.data?.prompt_id === promptId) {
|
||||
finish(false, new Error("Execution interrupted"));
|
||||
@@ -308,18 +334,43 @@ export const comfyUI = {
|
||||
}
|
||||
},
|
||||
|
||||
// First literal seed/noise_seed value in an API-format workflow (null if none).
|
||||
// Linked inputs (arrays) are skipped.
|
||||
getWorkflowSeed(workflow) {
|
||||
for (const node of Object.values(workflow)) {
|
||||
if (!node?.inputs) continue;
|
||||
for (const key of ["seed", "noise_seed"]) {
|
||||
const v = node.inputs[key];
|
||||
if (typeof v === "number" && v >= 0) return v;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
},
|
||||
|
||||
// High-level generate
|
||||
async generate(workflow, options = {}) {
|
||||
const { seedMode = "random", seedValue = -1, onProgress, onComplete, onError, timeoutMs } = options;
|
||||
|
||||
this.generating = true;
|
||||
try {
|
||||
// Callers pass a shallow copy ({ ...workflow }) whose node objects are shared with
|
||||
// currentWorkflow; clone so seed overrides don't leak back into the loaded
|
||||
// workflow (otherwise "workflow" mode would reuse the last random seed).
|
||||
workflow = structuredClone(workflow);
|
||||
// Apply seed
|
||||
let seed = seedValue;
|
||||
if (seedMode === "random" || seed < 0) {
|
||||
seed = Math.floor(Math.random() * Number.MAX_SAFE_INTEGER);
|
||||
if (seedMode === "workflow") {
|
||||
// Use the workflow's own seed as-is (no override). Fall back to the
|
||||
// panel value only if the workflow has no seed input at all.
|
||||
const wfSeed = this.getWorkflowSeed(workflow);
|
||||
if (wfSeed != null) seed = wfSeed;
|
||||
else this.applySeedToWorkflow(workflow, seed < 0 ? Math.floor(Math.random() * Number.MAX_SAFE_INTEGER) : seed);
|
||||
} else {
|
||||
if (seedMode === "random" || seed < 0) {
|
||||
seed = Math.floor(Math.random() * Number.MAX_SAFE_INTEGER);
|
||||
}
|
||||
this.applySeedToWorkflow(workflow, seed);
|
||||
}
|
||||
this.applySeedToWorkflow(workflow, seed);
|
||||
|
||||
// Connect WebSocket
|
||||
const wsOk = await this.connectWebSocket();
|
||||
|
||||
+194
-4
@@ -3,10 +3,12 @@
|
||||
*/
|
||||
|
||||
import { comfyUI } from "./comfyui-client.js";
|
||||
import { comfyWorkflow } from "./comfyui-workflow.js";
|
||||
import { syncJsonHighlight } from "./json-highlight.js";
|
||||
import { t } from "./i18n.js";
|
||||
import { escapeHtml, setupSearchClearBtn } from "./util.js";
|
||||
import { showToast } from "./app.js";
|
||||
import { translateButtonsHtml, attachTranslateButtons } from "./prompt-translate.js";
|
||||
|
||||
// ── Latent Image preset state ─────────────────────────────
|
||||
const _LATENT_PRESET_KEY = "wfm_latent_presets";
|
||||
@@ -198,7 +200,59 @@ function _buildLoraManagerSyntax(stackModels) {
|
||||
.join(" ");
|
||||
}
|
||||
|
||||
// ── Native LoRA stack nodes (LoadLoraModel / LoadLoraTextEncoder) ─────────
|
||||
// DynamicGroup rows are flat API keys "loras.<row>.lora_name|strength|enabled" (max 20,
|
||||
// min 1). The Model node takes the model strength and the Text Encoder node the clip
|
||||
// strength; every stack node in the workflow is written together so a single Apply /
|
||||
// Bypass keeps the pair consistent.
|
||||
const _LORA_STACK_STRENGTH = { LoadLoraModel: "strModel", LoadLoraTextEncoder: "strClip" };
|
||||
|
||||
function _isLoraStackNode(wfNode) {
|
||||
return !!wfNode && Object.hasOwn(_LORA_STACK_STRENGTH, wfNode.class_type);
|
||||
}
|
||||
|
||||
function _readLoraStackRows(inputs) {
|
||||
const rows = new Map();
|
||||
for (const [k, v] of Object.entries(inputs || {})) {
|
||||
const m = /^loras\.(\d+)\.(lora_name|strength|enabled)$/.exec(k);
|
||||
if (!m) continue;
|
||||
if (!rows.has(+m[1])) rows.set(+m[1], {});
|
||||
rows.get(+m[1])[m[2]] = v;
|
||||
}
|
||||
return [...rows.entries()].sort((a, b) => a[0] - b[0]).map(([, r]) => r);
|
||||
}
|
||||
|
||||
function _writeLoraStackRows(wfNode, rows) {
|
||||
for (const k of Object.keys(wfNode.inputs)) if (k.startsWith("loras.")) delete wfNode.inputs[k];
|
||||
rows.slice(0, 20).forEach((r, i) => {
|
||||
wfNode.inputs[`loras.${i}.lora_name`] = r.lora_name;
|
||||
wfNode.inputs[`loras.${i}.strength`] = r.strength;
|
||||
wfNode.inputs[`loras.${i}.enabled`] = r.enabled;
|
||||
});
|
||||
}
|
||||
|
||||
// items: [{ name, strModel?, strClip?, enabled? }] — omitted strengths keep the node's
|
||||
// first existing row (default 1). Writes every stack node in `wf`.
|
||||
function _applyLoraStack(items, wf = comfyUI.currentWorkflow) {
|
||||
if (!items.length) return;
|
||||
for (const wfNode of Object.values(wf || {})) {
|
||||
if (!_isLoraStackNode(wfNode)) continue;
|
||||
const key = _LORA_STACK_STRENGTH[wfNode.class_type];
|
||||
const prev = _readLoraStackRows(wfNode.inputs)[0]?.strength;
|
||||
wfNode.inputs ??= {};
|
||||
_writeLoraStackRows(wfNode, items.slice(0, 20).map((it) => ({
|
||||
lora_name: it.name,
|
||||
strength: it[key] ?? (typeof prev === "number" ? prev : 1),
|
||||
enabled: it.enabled !== false,
|
||||
})));
|
||||
}
|
||||
}
|
||||
|
||||
function _applyLoraToNode(nodeId, loraPath, strModel, strClip, isLoraManager) {
|
||||
if (_isLoraStackNode(comfyUI.currentWorkflow[nodeId])) {
|
||||
_applyLoraStack([{ name: comfyEditor.resolveLoraName(loraPath), strModel, strClip }]);
|
||||
return;
|
||||
}
|
||||
if (isLoraManager) {
|
||||
const stem = _loraBasename(loraPath);
|
||||
comfyUI.currentWorkflow[nodeId].inputs.loras = {
|
||||
@@ -263,6 +317,104 @@ function _syncStackToggleAll(stackModels) {
|
||||
cb.indeterminate = !allOn && !allOff;
|
||||
}
|
||||
|
||||
// Two-option combos that are really an on/off switch (KSamplerAdvanced add_noise, DWPreprocessor
|
||||
// detect_*, ...): [on value, off value], compared case-insensitively.
|
||||
const _TOGGLE_COMBO_PAIRS = [["enable", "disable"], ["enabled", "disabled"], ["on", "off"], ["true", "false"], ["yes", "no"]];
|
||||
|
||||
function _comboOptions(spec) {
|
||||
if (Array.isArray(spec?.[0])) return spec[0];
|
||||
if (spec?.[0] === "COMBO" && Array.isArray(spec[1]?.options)) return spec[1].options;
|
||||
return null;
|
||||
}
|
||||
|
||||
/** [onValue, offValue] in the node's own spelling when the combo is an on/off pair, else null */
|
||||
function _toggleComboValues(spec) {
|
||||
const opts = _comboOptions(spec);
|
||||
if (!opts || opts.length !== 2 || !opts.every((o) => typeof o === "string")) return null;
|
||||
const lower = opts.map((o) => o.toLowerCase());
|
||||
for (const [on, off] of _TOGGLE_COMBO_PAIRS) {
|
||||
const iOn = lower.indexOf(on), iOff = lower.indexOf(off);
|
||||
if (iOn >= 0 && iOff >= 0) return [opts[iOn], opts[iOff]];
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function _compareNodeIds(a, b) {
|
||||
const x = String(a).split(":").map(Number), y = String(b).split(":").map(Number);
|
||||
for (let i = 0; i < Math.max(x.length, y.length); i++) {
|
||||
const d = (x[i] ?? -1) - (y[i] ?? -1);
|
||||
if (d) return d;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* On/off items of the API workflow for the Settings tab's Switches column: every literal
|
||||
* true/false input (ComfySwitchNode switch, PrimitiveBoolean value, TextGenerate thinking, ...)
|
||||
* plus string combos whose only two options are an on/off pair (needs object_info). Linked
|
||||
* inputs are arrays and never match; their source node shows up instead.
|
||||
*/
|
||||
function _collectSwitchInputs(workflow, objectInfo = {}) {
|
||||
const out = [];
|
||||
for (const [id, node] of Object.entries(workflow || {})) {
|
||||
if (!node?.class_type || !node.inputs) continue;
|
||||
const info = objectInfo[node.class_type]?.input || {};
|
||||
for (const [key, value] of Object.entries(node.inputs)) {
|
||||
const base = { id, type: node.class_type, title: node._meta?.title || node.class_type, key };
|
||||
if (typeof value === "boolean") {
|
||||
out.push({ ...base, on: value, onValue: true, offValue: false });
|
||||
} else if (typeof value === "string") {
|
||||
const pair = _toggleComboValues(info.required?.[key] || info.optional?.[key]);
|
||||
if (pair && (value === pair[0] || value === pair[1])) {
|
||||
out.push({ ...base, on: value === pair[0], onValue: pair[0], offValue: pair[1] });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out.sort((a, b) => _compareNodeIds(a.id, b.id));
|
||||
}
|
||||
|
||||
let _switchRenderSeq = 0;
|
||||
|
||||
/** Fill the Switches column. Booleans show at once; on/off combos follow once object_info is in. */
|
||||
async function _renderSwitchColumn(col) {
|
||||
if (!col) return;
|
||||
const seq = ++_switchRenderSeq;
|
||||
const workflow = comfyUI.currentWorkflow;
|
||||
const draw = (items) => {
|
||||
col.innerHTML = items.length ? `<div class="wfm-settings-switches">${items.map((s, i) => `
|
||||
<label class="wfm-settings-switch" title="${escapeHtml(`#${s.id} ${s.type} › ${s.key}`)}">
|
||||
<input type="checkbox" class="wfm-settings-switch-cb" data-index="${i}" ${s.on ? "checked" : ""}>
|
||||
<span class="wfm-settings-switch-text">
|
||||
<span class="wfm-settings-switch-node">${escapeHtml(s.title)}</span>
|
||||
<span class="wfm-settings-switch-key">${escapeHtml(s.key)}</span>
|
||||
</span>
|
||||
<span class="wfm-settings-switch-state">${escapeHtml(String(s.on ? s.onValue : s.offValue))}</span>
|
||||
</label>`).join("")}
|
||||
</div>` : `<p class="wfm-placeholder">${t("genSwitchesNone")}</p>`;
|
||||
// Applies immediately (no Apply button): written straight into the API workflow Generate sends
|
||||
col.querySelectorAll(".wfm-settings-switch-cb").forEach((cb) => {
|
||||
cb.addEventListener("change", () => {
|
||||
const s = items[Number(cb.dataset.index)];
|
||||
const node = comfyUI.currentWorkflow?.[s.id];
|
||||
if (!node?.inputs || !(s.key in node.inputs) || Array.isArray(node.inputs[s.key])) return;
|
||||
s.on = cb.checked;
|
||||
node.inputs[s.key] = s.on ? s.onValue : s.offValue;
|
||||
const state = cb.closest(".wfm-settings-switch")?.querySelector(".wfm-settings-switch-state");
|
||||
if (state) state.textContent = String(node.inputs[s.key]);
|
||||
_syncRawJson();
|
||||
});
|
||||
});
|
||||
};
|
||||
draw(_collectSwitchInputs(workflow));
|
||||
const types = [...new Set(Object.values(workflow || {}).map((n) => n?.class_type).filter(Boolean))];
|
||||
let objectInfo = {};
|
||||
try { objectInfo = await comfyWorkflow.getObjectInfoForTypes(types); } catch {}
|
||||
// A newer render (another workflow loaded meanwhile) owns the column now
|
||||
if (seq !== _switchRenderSeq || workflow !== comfyUI.currentWorkflow || !col.isConnected) return;
|
||||
draw(_collectSwitchInputs(workflow, objectInfo));
|
||||
}
|
||||
|
||||
/** Sync comfyUI.currentWorkflow to Raw JSON textarea + highlight */
|
||||
function _syncRawJson() {
|
||||
const rawTextarea = document.getElementById("wfm-gen-raw-json");
|
||||
@@ -463,6 +615,10 @@ export const comfyEditor = {
|
||||
return found || name;
|
||||
},
|
||||
|
||||
// Native LoRA stack nodes (LoadLoraModel / LoadLoraTextEncoder) — see _applyLoraStack.
|
||||
isLoraStackNode: _isLoraStackNode,
|
||||
applyLoraStack: _applyLoraStack,
|
||||
|
||||
renderAll(analysis, workflow, opts = {}) {
|
||||
this.renderPromptTab(analysis, "wfm-gen-prompt-fields");
|
||||
this.renderImageTab(analysis, "wfm-gen-image-fields");
|
||||
@@ -490,6 +646,7 @@ export const comfyEditor = {
|
||||
${nodeOpts}
|
||||
</select>
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-pos-revert" title="${t("revertPromptTitle")}">↺</button>
|
||||
${translateButtonsHtml("wfm-prompt-pos-text")}
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-pos-apply" title="Apply (Alt+Click: Apply & Generate)">Apply</button>
|
||||
</div>
|
||||
<textarea class="wfm-textarea" id="wfm-prompt-pos-text" rows="6">${positiveNodes[0]?.text || ""}</textarea>
|
||||
@@ -502,6 +659,7 @@ export const comfyEditor = {
|
||||
${nodeOpts}
|
||||
</select>
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-neg-revert" title="${t("revertPromptTitle")}">↺</button>
|
||||
${translateButtonsHtml("wfm-prompt-neg-text")}
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-neg-apply">Apply</button>
|
||||
</div>
|
||||
<textarea class="wfm-textarea" id="wfm-prompt-neg-text" rows="6">${negativeNodes[0]?.text || ""}</textarea>
|
||||
@@ -564,6 +722,9 @@ export const comfyEditor = {
|
||||
ta.value = wfNode.inputs[textKey] ?? "";
|
||||
});
|
||||
|
||||
// 🌐 native-language prompt translation (Settings → Prompt Translation)
|
||||
attachTranslateButtons(el);
|
||||
|
||||
// Track last focused prompt textarea for Paste button
|
||||
["wfm-prompt-pos-text", "wfm-prompt-neg-text"].forEach((taId) => {
|
||||
const ta = document.getElementById(taId);
|
||||
@@ -776,7 +937,11 @@ export const comfyEditor = {
|
||||
// Restore whatever was set before bypassing
|
||||
const snap = _loraBypassSnapshot[nodeId];
|
||||
if (snap) {
|
||||
if (isLoraManager) {
|
||||
if (snap.stackRows) {
|
||||
for (const [id, rows] of Object.entries(snap.stackRows)) {
|
||||
if (comfyUI.currentWorkflow?.[id]) _writeLoraStackRows(comfyUI.currentWorkflow[id], JSON.parse(JSON.stringify(rows)));
|
||||
}
|
||||
} else if (isLoraManager) {
|
||||
wfNode.inputs.loras = { __value__: JSON.parse(JSON.stringify(snap.loras || [])) };
|
||||
wfNode.inputs.text = snap.text ?? "";
|
||||
} else {
|
||||
@@ -791,7 +956,16 @@ export const comfyEditor = {
|
||||
// Snapshot current values, then zero the strength (LoraManager: every
|
||||
// entry in its loras list, keeping the list itself intact so un-bypassing
|
||||
// restores the exact same Stack — not just a single-LoRA fallback).
|
||||
if (isLoraManager) {
|
||||
if (_isLoraStackNode(wfNode)) {
|
||||
// Native stack nodes: snapshot every stack node's rows, then zero each strength
|
||||
const stackRows = {};
|
||||
for (const [id, n] of Object.entries(comfyUI.currentWorkflow)) {
|
||||
if (!_isLoraStackNode(n)) continue;
|
||||
stackRows[id] = _readLoraStackRows(n.inputs);
|
||||
_writeLoraStackRows(n, stackRows[id].map((r) => ({ ...r, strength: 0 })));
|
||||
}
|
||||
_loraBypassSnapshot[nodeId] = { stackRows };
|
||||
} else if (isLoraManager) {
|
||||
const loras = Array.isArray(wfNode.inputs.loras?.__value__) ? wfNode.inputs.loras.__value__ : [];
|
||||
_loraBypassSnapshot[nodeId] = { loras: JSON.parse(JSON.stringify(loras)), text: wfNode.inputs.text ?? "" };
|
||||
wfNode.inputs.loras = { __value__: loras.map((l) => ({ ...l, strength: 0, clipStrength: 0 })) };
|
||||
@@ -862,7 +1036,9 @@ export const comfyEditor = {
|
||||
const _firstLoraNode = loraNodes[0];
|
||||
if (_firstLoraNode && comfyUI.currentWorkflow?.[_firstLoraNode.id]) {
|
||||
const _wfNode = comfyUI.currentWorkflow[_firstLoraNode.id];
|
||||
if (!_firstLoraNode.is_lora_manager) {
|
||||
if (_firstLoraNode.is_lora_stack) {
|
||||
currentVal = _readLoraStackRows(_wfNode.inputs)[0]?.lora_name || currentVal;
|
||||
} else if (!_firstLoraNode.is_lora_manager) {
|
||||
currentVal = _wfNode.inputs?.lora_name || currentVal;
|
||||
} else {
|
||||
// LoraManager: loras.__value__[0].nameからフルパスを復元
|
||||
@@ -1294,6 +1470,14 @@ export const comfyEditor = {
|
||||
});
|
||||
comfyUI.currentWorkflow[nodeId].inputs.loras = { __value__: loraValue };
|
||||
comfyUI.currentWorkflow[nodeId].inputs.text = _buildLoraManagerSyntax(stackModels);
|
||||
} else if (node?.is_lora_stack) {
|
||||
// Native stack nodes take the whole Stack group as rows (max 20)
|
||||
_applyLoraStack(stackModels.map((m) => ({
|
||||
name: comfyEditor.resolveLoraName(m),
|
||||
strModel: _stackStrengths[m]?.m ?? 1.0,
|
||||
strClip: _stackStrengths[m]?.c ?? 1.0,
|
||||
enabled: _stackActive[m] !== false,
|
||||
})));
|
||||
} else {
|
||||
const first = stackModels[0];
|
||||
if (first) {
|
||||
@@ -1435,7 +1619,7 @@ export const comfyEditor = {
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-settings-sampler-apply" title="Apply (Alt+Click: Apply & Generate)">Apply</button>
|
||||
` : "<p class='wfm-placeholder'>No KSampler node found</p>"}
|
||||
</div>
|
||||
<div style="flex:1;min-width:0;padding-left:14px;">
|
||||
<div style="flex:1;min-width:0;padding:0 14px;border-right:1px solid var(--wfm-border);">
|
||||
<h3 style="font-size:13px;font-weight:600;text-transform:uppercase;letter-spacing:0.5px;color:var(--wfm-text-secondary);margin:0 0 12px;">Latent Image</h3>
|
||||
${latent ? `
|
||||
<input type="hidden" id="wfm-settings-latent-id" value="${latent.id}">
|
||||
@@ -1480,9 +1664,15 @@ export const comfyEditor = {
|
||||
</div>
|
||||
` : "<p class='wfm-placeholder'>No EmptyLatentImage node found</p>"}
|
||||
</div>
|
||||
<div style="flex:1;min-width:0;padding-left:14px;">
|
||||
<h3 style="font-size:13px;font-weight:600;text-transform:uppercase;letter-spacing:0.5px;color:var(--wfm-text-secondary);margin:0 0 12px;">${t("genSwitchesTitle")}</h3>
|
||||
<div id="wfm-settings-switches-col"></div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
_renderSwitchColumn(document.getElementById("wfm-settings-switches-col"));
|
||||
|
||||
document.getElementById("wfm-settings-sampler-apply")?.addEventListener("click", (e) => {
|
||||
const ds = document.getElementById("wfm-settings-sampler-id")?.dataset || {};
|
||||
const write = (nodeId, key, value) => {
|
||||
|
||||
+343
-29
@@ -2,6 +2,11 @@
|
||||
* ComfyUI Workflow - Format detection, conversion, and analysis
|
||||
*/
|
||||
|
||||
// Video all-in-one nodes (MiniMaxH3ImageToVideo, MiniMaxH3ReferenceToVideo, ...): hold their own
|
||||
// "prompt" and build the conditioning + latent themselves. Matched by class-name pattern so other
|
||||
// models sharing this shape work without a code change.
|
||||
const VIDEO_ALL_IN_ONE_RE = /ImageToVideo|TextToVideo|ReferenceToVideo/;
|
||||
|
||||
// Cache for /object_info (loaded once, shared)
|
||||
let _objectInfoCache = null;
|
||||
|
||||
@@ -9,6 +14,9 @@ let _objectInfoCache = null;
|
||||
// ckpt_name に対して発生した場合の記録。呼び出し元が読み込み直後に確認して警告表示できるよう、
|
||||
// 直近の convertUiToApi() 呼び出し分だけを保持する。
|
||||
let _lastCheckpointSubstitutions = [];
|
||||
// 同じフォールバックを全COMBO入力（LoRA・VAE・画像など）について記録したもの。
|
||||
// Workflowタブの「GenerateUI互換チェック」が、無警告で差し替わる値を一覧するのに使う。
|
||||
let _lastComboSubstitutions = [];
|
||||
|
||||
// convertUiToApi() で除外された Bypass(mode:4) / Mute(mode:2) ノードの記録。
|
||||
// API形式には mode 情報が残らないため、呼び出し元がRaw JSON表示時に
|
||||
@@ -71,12 +79,57 @@ async function _loadObjectInfo() {
|
||||
return _objectInfoCache || {};
|
||||
}
|
||||
|
||||
// Per-node-type object_info (GET /object_info/<type>, ~20 ms each). The full /object_info is
|
||||
// tens of MB and can take 10-40 s on installs with many custom nodes, so conversions and the
|
||||
// compatibility check only fetch the types the workflow actually uses. type -> info, or null when
|
||||
// ComfyUI doesn't know the type. Only successful responses are cached.
|
||||
const _objectInfoByType = new Map();
|
||||
|
||||
async function _loadObjectInfoFor(types) {
|
||||
if (_objectInfoCache) return _objectInfoCache;
|
||||
const wanted = [...new Set(types)].filter((ty) => typeof ty === "string" && ty && !_objectInfoByType.has(ty));
|
||||
let failed = false;
|
||||
for (let i = 0; i < wanted.length; i += 16) {
|
||||
await Promise.all(wanted.slice(i, i + 16).map(async (ty) => {
|
||||
try {
|
||||
const res = await fetch(`/object_info/${encodeURIComponent(ty)}`);
|
||||
if (!res.ok) { if (res.status === 404) _objectInfoByType.set(ty, null); else failed = true; return; }
|
||||
const data = await res.json();
|
||||
_objectInfoByType.set(ty, data?.[ty] || null);
|
||||
} catch { failed = true; }
|
||||
}));
|
||||
}
|
||||
// Any transport failure: fall back to the full snapshot rather than report types as unknown
|
||||
if (failed) return _loadObjectInfo();
|
||||
const out = {};
|
||||
for (const ty of types) { const info = _objectInfoByType.get(ty); if (info) out[ty] = info; }
|
||||
return out;
|
||||
}
|
||||
|
||||
// Node types a UI-format workflow uses, including nodes inside subgraph definitions
|
||||
function _collectNodeTypes(workflow) {
|
||||
const sgIds = new Set((workflow?.definitions?.subgraphs || []).map((sg) => sg.id));
|
||||
const types = new Set();
|
||||
const add = (nodes) => { for (const n of nodes || []) if (n?.type && !sgIds.has(n.type)) types.add(n.type); };
|
||||
add(workflow?.nodes);
|
||||
for (const sg of workflow?.definitions?.subgraphs || []) add(sg.nodes);
|
||||
return [...types];
|
||||
}
|
||||
|
||||
/**
|
||||
* Get ordered widget (non-link) input names for a node class from object_info.
|
||||
* These correspond to widgets_values in UI-format nodes.
|
||||
*/
|
||||
// Scalar widget types object_info can declare for a required/optional input.
|
||||
const _WIDGET_TYPE_SET = new Set(["INT", "FLOAT", "STRING", "BOOLEAN", "COMBO", "COMFY_DYNAMICCOMBO_V3"]);
|
||||
// BOUNDING_BOXES: CreateBoundingBoxes' canvas widgets (editor_state / last_incoming) — array-valued
|
||||
// frontend widgets that occupy a widgets_values slot; sent to the API as {"__value__": [...]}.
|
||||
const _WIDGET_TYPE_SET = new Set(["INT", "FLOAT", "STRING", "BOOLEAN", "COMBO", "COMFY_DYNAMICCOMBO_V3", "BOUNDING_BOXES"]);
|
||||
|
||||
// Required inputs backed by display-only frontend widgets (serialize=false, so no widgets_values
|
||||
// slot) -> the value the frontend sends for them in the API prompt.
|
||||
const _DISPLAY_WIDGET_DEFAULTS = {
|
||||
IMAGECOMPARE: () => ({ __value__: ["", ""] }),
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve a possibly-compound object_info type spec (e.g. LTXVEmptyLatentAudio's frame_rate:
|
||||
@@ -97,11 +150,10 @@ function _resolveWidgetType(type, spec) {
|
||||
if (typeof type !== "string") return null;
|
||||
const explicit = spec?.[1]?.widgetType;
|
||||
if (typeof explicit === "string" && _WIDGET_TYPE_SET.has(explicit.toUpperCase())) return explicit.toUpperCase();
|
||||
for (const part of type.split(",")) {
|
||||
const upper = part.trim().toUpperCase();
|
||||
if (_WIDGET_TYPE_SET.has(upper)) return upper;
|
||||
}
|
||||
return null;
|
||||
// Only the first (default) segment decides: a union that merely *accepts* a scalar as one of its
|
||||
// link types (CreateBoundingBoxes.bboxes: "BOUNDING_BOX,ARRAY,STRING") is still a link-only socket.
|
||||
const first = type.split(",")[0].trim().toUpperCase();
|
||||
return _WIDGET_TYPE_SET.has(first) ? first : null;
|
||||
}
|
||||
|
||||
function _getWidgetInputNames(objectInfo, classType) {
|
||||
@@ -169,6 +221,11 @@ function _getWidgetInputTypes(objectInfo, classType) {
|
||||
* presence_penalty under optional — all 7 still occupy a widgets_values slot). Dropping the
|
||||
* optional ones here under-counts how many slots the combo actually consumed, misaligning
|
||||
* every widget declared after it in the node's schema.
|
||||
*
|
||||
* Only sub-fields that are actually widgets count: a sub-field can also be a link socket (e.g. the
|
||||
* Nano Banana 2.1 node's "images" Autogrow group and its "files" GEMINI_INPUT_FILES input), which the
|
||||
* frontend does NOT serialize into widgets_values — counting those would shift every value after
|
||||
* them (seed read as "IMAGE", the system prompt landing in response_modalities, ...).
|
||||
*/
|
||||
function _getDynamicComboSubNames(objectInfo, classType, widgetName, selectedKey) {
|
||||
const info = objectInfo[classType];
|
||||
@@ -176,12 +233,23 @@ function _getDynamicComboSubNames(objectInfo, classType, widgetName, selectedKey
|
||||
const options = spec?.[1]?.options;
|
||||
if (!Array.isArray(options)) return [];
|
||||
const opt = options.find((o) => o.key === selectedKey) || options[0];
|
||||
const isWidget = ([, sub]) => {
|
||||
if (Array.isArray(sub) && sub[1]?.forceInput) return false;
|
||||
return !!_resolveWidgetType(Array.isArray(sub) ? sub[0] : sub, sub);
|
||||
};
|
||||
return [
|
||||
...Object.keys(opt?.inputs?.required || {}),
|
||||
...Object.keys(opt?.inputs?.optional || {}),
|
||||
...Object.entries(opt?.inputs?.required || {}).filter(isWidget).map(([k]) => k),
|
||||
...Object.entries(opt?.inputs?.optional || {}).filter(isWidget).map(([k]) => k),
|
||||
];
|
||||
}
|
||||
|
||||
function _isDynamicComboKey(objectInfo, classType, widgetName, value) {
|
||||
const info = objectInfo[classType];
|
||||
const spec = info?.input?.required?.[widgetName] || info?.input?.optional?.[widgetName];
|
||||
const options = spec?.[1]?.options;
|
||||
return !Array.isArray(options) || options.some((o) => o.key === value);
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect if a widget value is a frontend-only extra (like "control_after_generate").
|
||||
* These values appear in widgets_values but are not in object_info inputs.
|
||||
@@ -189,6 +257,14 @@ function _getDynamicComboSubNames(objectInfo, classType, widgetName, selectedKey
|
||||
*/
|
||||
const _CONTROL_AFTER_GENERATE = new Set(["fixed", "increment", "decrement", "randomize"]);
|
||||
|
||||
// True when the widget's object_info spec declares control_after_generate (seed-style INT widgets):
|
||||
// the frontend then serializes one extra "randomize"/"fixed"/... slot right after the widget's value.
|
||||
function _hasControlAfterGenerate(objectInfo, classType, widgetName) {
|
||||
const info = objectInfo[classType]?.input;
|
||||
const spec = info?.required?.[widgetName] || info?.optional?.[widgetName];
|
||||
return !!(Array.isArray(spec) && spec[1]?.control_after_generate);
|
||||
}
|
||||
|
||||
function _isExtraWidgetValue(val, expectedType) {
|
||||
if (expectedType === "INT" || expectedType === "FLOAT") {
|
||||
// If we expect a number but got a string like "fixed", it's a control widget
|
||||
@@ -308,9 +384,15 @@ function _normalizeLinkedWidgetValues(objectInfo, nodeType, widgetsValues, isLin
|
||||
const comboExpander = (name, selectedKey) => _getDynamicComboSubNames(objectInfo, nodeType, name, selectedKey);
|
||||
|
||||
// If skipping linked widgets already accounts for every entry, widgets_values is already in
|
||||
// the modern (no dangling linked entries) format — nothing to strip.
|
||||
// the modern (no dangling linked entries) format — nothing to strip. The count alone can match
|
||||
// by accident: TextGenerateV2 ["", 1536, "off", true, true, "auto"] with prompt linked reads
|
||||
// 1536 as sampling_mode, which falls back to the first option and swallows the rest. A selector
|
||||
// that doesn't hold one of its option keys means this reading is misaligned.
|
||||
const skipSim = _simulateWidgetValues(widgetNames, widgetTypes, widgetsValues, isLinkedName, true, comboExpander);
|
||||
if (skipSim.consumed === widgetsValues.length) return widgetsValues;
|
||||
const selectorsValid = widgetNames.every((name, i) => widgetTypes[i] !== "COMFY_DYNAMICCOMBO_V3"
|
||||
|| skipSim.positions[name] < 0
|
||||
|| _isDynamicComboKey(objectInfo, nodeType, name, widgetsValues[skipSim.positions[name]]));
|
||||
if (skipSim.consumed === widgetsValues.length && selectorsValid) return widgetsValues;
|
||||
|
||||
// Otherwise it's "legacy full": recompute with every widget present to find exactly which
|
||||
// indices the linked ones occupy — including any control_after_generate extra a linked
|
||||
@@ -438,6 +520,9 @@ function _flattenSubgraphs(workflow, objectInfo) {
|
||||
// from ports that truly carry a link end-to-end (e.g. width/height wired to a
|
||||
// ResolutionSelector node).
|
||||
const redirectedTargets = new Set();
|
||||
// Extra links for the 2nd+ fan-out targets of a port that has a real outer link
|
||||
// (collected here and appended after the loop so wf.links isn't mutated mid-iteration).
|
||||
const fanOutLinks = [];
|
||||
for (const link of wf.links) {
|
||||
if (link[3] === node.id) {
|
||||
const dstSlot = link[4];
|
||||
@@ -455,6 +540,13 @@ function _flattenSubgraphs(workflow, objectInfo) {
|
||||
link[3] = target.targetNodeId;
|
||||
link[4] = target.targetSlot;
|
||||
redirectedTargets.add(`${target.targetNodeId}:${target.targetSlot}`);
|
||||
// Same real outer link feeding further internal targets (e.g. Qwen Image 2.1
|
||||
// Edit: images.image_1 -> TextEncodeQwenImage21 AND the prompt-enhancer's
|
||||
// BatchImagesNode) — duplicate it for each remaining target.
|
||||
for (const extra of (inputPortTargets[portIdx] || []).slice(1)) {
|
||||
fanOutLinks.push([0, link[1], link[2], extra.targetNodeId, extra.targetSlot, link[5]]);
|
||||
redirectedTargets.add(`${extra.targetNodeId}:${extra.targetSlot}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (link[1] === node.id) {
|
||||
@@ -467,6 +559,11 @@ function _flattenSubgraphs(workflow, objectInfo) {
|
||||
}
|
||||
}
|
||||
|
||||
for (const l of fanOutLinks) {
|
||||
l[0] = ++maxLinkId;
|
||||
wf.links.push(l);
|
||||
}
|
||||
|
||||
// === Add internal nodes (remapped IDs) ===
|
||||
const remappedByOrigId = {};
|
||||
for (const iNode of sgDef.nodes) {
|
||||
@@ -516,9 +613,19 @@ function _flattenSubgraphs(workflow, objectInfo) {
|
||||
const port = sgDef.inputs[portIdx];
|
||||
const outerInp = outerWidgetInputs.find((inp) => inp.name === port.name);
|
||||
const outerWidgetIdx = outerInp ? outerWidgetInputs.indexOf(outerInp) : -1;
|
||||
const hasOuterOverride = !!outerInp && outerInp.link == null && outerWidgetIdx !== -1
|
||||
&& outerWidgetIdx < (node.widgets_values || []).length;
|
||||
const outerValue = hasOuterOverride ? node.widgets_values[outerWidgetIdx] : undefined;
|
||||
// Newer frontends also save a name-keyed copy of the outer widget values
|
||||
// (widgets_values_named) and may omit widget ports (seed/steps/cfg/...) from the outer
|
||||
// node's inputs[] while widgets_values still holds every port, so the index-among-inputs
|
||||
// lookup drifts (e.g. MOSS-Image: unet_name read the steps value). Prefer the named copy.
|
||||
const named = node.widgets_values_named;
|
||||
const hasNamed = !!named && typeof named === "object"
|
||||
&& Object.prototype.hasOwnProperty.call(named, port.name);
|
||||
const hasOuterOverride = hasNamed
|
||||
? !(outerInp && outerInp.link != null)
|
||||
: (!!outerInp && outerInp.link == null && outerWidgetIdx !== -1
|
||||
&& outerWidgetIdx < (node.widgets_values || []).length);
|
||||
const outerValue = !hasOuterOverride ? undefined
|
||||
: (hasNamed ? named[port.name] : node.widgets_values[outerWidgetIdx]);
|
||||
|
||||
// Value to apply to non-widget (wildcard) fan-out targets: the outer override if
|
||||
// present, otherwise whatever a sibling widget-type target ends up holding (its own
|
||||
@@ -617,6 +724,20 @@ export const comfyWorkflow = {
|
||||
return _lastCheckpointSubstitutions;
|
||||
},
|
||||
|
||||
getLastComboSubstitutions() {
|
||||
return _lastComboSubstitutions;
|
||||
},
|
||||
|
||||
// Cached /object_info (same snapshot convertUiToApi() uses)
|
||||
async getObjectInfo() {
|
||||
return _loadObjectInfo();
|
||||
},
|
||||
|
||||
// object_info limited to the given node types (fast; see _loadObjectInfoFor)
|
||||
async getObjectInfoForTypes(types) {
|
||||
return _loadObjectInfoFor(types);
|
||||
},
|
||||
|
||||
getLastBypassedNodes() {
|
||||
return _lastBypassedNodes;
|
||||
},
|
||||
@@ -635,15 +756,29 @@ export const comfyWorkflow = {
|
||||
// at the top) would replace it with an unrelated pre-existing file instead.
|
||||
invalidateObjectInfoCache() {
|
||||
_objectInfoCache = null;
|
||||
_objectInfoByType.clear();
|
||||
},
|
||||
|
||||
async convertUiToApi(workflow) {
|
||||
// opts.readOnly: 表示用の読み取り変換。COMBO値の差し替えを行わず、Workflowタブの互換性
|
||||
// チェックが参照する _last* 記録も変更しない（Metadataタブなど表示目的の呼び出し用）。
|
||||
async convertUiToApi(workflow, opts = {}) {
|
||||
if (!opts.readOnly) return comfyWorkflow._convertUiToApiImpl(workflow, opts);
|
||||
const saved = [_lastCheckpointSubstitutions, _lastComboSubstitutions, _lastBypassedNodes, _lastMutedNodes];
|
||||
try {
|
||||
return await comfyWorkflow._convertUiToApiImpl(workflow, opts);
|
||||
} finally {
|
||||
[_lastCheckpointSubstitutions, _lastComboSubstitutions, _lastBypassedNodes, _lastMutedNodes] = saved;
|
||||
}
|
||||
},
|
||||
|
||||
async _convertUiToApiImpl(workflow, opts = {}) {
|
||||
_lastCheckpointSubstitutions = [];
|
||||
_lastComboSubstitutions = [];
|
||||
_lastBypassedNodes = [];
|
||||
_lastMutedNodes = [];
|
||||
if (!workflow.nodes || !workflow.links) return {};
|
||||
|
||||
const objectInfo = await _loadObjectInfo();
|
||||
const objectInfo = await _loadObjectInfoFor(_collectNodeTypes(workflow));
|
||||
|
||||
// Flatten subgraphs before conversion (needs objectInfo to correctly map outer
|
||||
// subgraph-node widget values onto internal nodes' widgets_values indices)
|
||||
@@ -739,6 +874,16 @@ export const comfyWorkflow = {
|
||||
if (wIdx >= widgets.length) break;
|
||||
const name = widgetNames[nIdx];
|
||||
const expectedType = widgetTypes[nIdx];
|
||||
// A control_after_generate widget (seed etc.) is followed by a frontend-only
|
||||
// "randomize"/"fixed"/... slot even when the next widget is not numeric (e.g.
|
||||
// Nano Banana 2.1: seed -> response_modalities); the type heuristic below only
|
||||
// catches it when the next widget happens to be INT/FLOAT.
|
||||
if (nIdx > 0 && !linkedInputNames.has(widgetNames[nIdx - 1])
|
||||
&& _hasControlAfterGenerate(objectInfo, node.type, widgetNames[nIdx - 1])
|
||||
&& _CONTROL_AFTER_GENERATE.has(widgets[wIdx])) {
|
||||
wIdx++;
|
||||
if (wIdx >= widgets.length) break;
|
||||
}
|
||||
// Skip if this input is already linked
|
||||
if (linkedInputNames.has(name)) {
|
||||
if (!linkedSlotNames.has(name)) {
|
||||
@@ -794,7 +939,15 @@ export const comfyWorkflow = {
|
||||
const spec = allInputDefs[name];
|
||||
if (spec) {
|
||||
const choices = Array.isArray(spec[0]) ? spec[0] : null;
|
||||
if (choices && choices.length > 0 && !choices.includes(val)) {
|
||||
// Template saved the bare file name while this install lists it under a
|
||||
// sub-folder (or the reverse, or with the other path separator): prefer the
|
||||
// choice whose file name matches over an unrelated choices[0].
|
||||
if (choices && choices.length > 0 && typeof val === "string" && !choices.includes(val)) {
|
||||
const base = (x) => String(x).split(/[\\/]/).pop();
|
||||
const sameFile = choices.filter((c) => typeof c === "string" && base(c) === base(val));
|
||||
if (sameFile.length === 1) val = sameFile[0];
|
||||
}
|
||||
if (!opts.readOnly && choices && choices.length > 0 && !choices.includes(val)) {
|
||||
if (name === "ckpt_name") {
|
||||
_lastCheckpointSubstitutions.push({
|
||||
nodeId: String(node.id),
|
||||
@@ -803,10 +956,21 @@ export const comfyWorkflow = {
|
||||
replaced: choices[0],
|
||||
});
|
||||
}
|
||||
_lastComboSubstitutions.push({
|
||||
nodeId: String(node.id),
|
||||
type: node.type,
|
||||
title: node.title || node.type,
|
||||
input: name,
|
||||
original: val,
|
||||
replaced: choices[0],
|
||||
});
|
||||
val = choices[0];
|
||||
}
|
||||
}
|
||||
}
|
||||
// List values are reserved for node links in the API format; the backend unwraps
|
||||
// {"__value__": [...]} for array-valued widgets.
|
||||
if (expectedType === "BOUNDING_BOXES" && Array.isArray(val)) val = { __value__: val };
|
||||
inputs[name] = val;
|
||||
}
|
||||
} else {
|
||||
@@ -840,12 +1004,22 @@ export const comfyWorkflow = {
|
||||
}
|
||||
}
|
||||
|
||||
// Display-only frontend widgets (ImageCompare's socketless compare_view) are never
|
||||
// stored in widgets_values, yet the backend requires them. The frontend sends the
|
||||
// widget's initial value, so do the same.
|
||||
for (const [k, spec] of Object.entries(objectInfo[node.type]?.input?.required || {})) {
|
||||
if (!(k in inputs) && _DISPLAY_WIDGET_DEFAULTS[spec?.[0]]) {
|
||||
inputs[k] = _DISPLAY_WIDGET_DEFAULTS[spec[0]]();
|
||||
}
|
||||
}
|
||||
|
||||
api[nodeId] = {
|
||||
class_type: node.type,
|
||||
inputs,
|
||||
_meta: { title: node.title || node.type },
|
||||
};
|
||||
}
|
||||
_unifyPromptEnhancerText(api);
|
||||
return api;
|
||||
},
|
||||
|
||||
@@ -1000,6 +1174,12 @@ export const comfyWorkflow = {
|
||||
if (Array.isArray(inputs.positive)) samplerPositiveRef[inputs.positive[0]] = true;
|
||||
if (Array.isArray(inputs.negative)) samplerNegativeRef[inputs.negative[0]] = true;
|
||||
}
|
||||
// Video all-in-one nodes whose "prompt" is wired from a separate text node (FastH3
|
||||
// OmniRef's MiniMaxH3ReferenceToVideo <- PrimitiveStringMultiline) — always positive
|
||||
// (no negative counterpart), so seed the role on the source directly.
|
||||
if (VIDEO_ALL_IN_ONE_RE.test(ct) && Array.isArray(inputs.prompt)) {
|
||||
samplerPositiveRef[inputs.prompt[0]] = true;
|
||||
}
|
||||
}
|
||||
|
||||
// === Pass 1b: Expand refs through CONDITIONING graph (BFS, 5 iterations) ===
|
||||
@@ -1076,6 +1256,13 @@ export const comfyWorkflow = {
|
||||
if (isPos && Array.isArray(inputs.positive)) samplerPositiveRef[inputs.positive[0]] = true;
|
||||
if (isNeg && Array.isArray(inputs.negative)) samplerNegativeRef[inputs.negative[0]] = true;
|
||||
}
|
||||
// ReferenceMultiLatent (Flux.3 Klein edit) — encodes reference images into the
|
||||
// conditioning and outputs positive/negative/empty_latent: separate positive/negative
|
||||
// inputs feeding separate outputs, so each role routes only to its own input.
|
||||
if (ct === "ReferenceMultiLatent") {
|
||||
if (isPos && Array.isArray(inputs.positive)) samplerPositiveRef[inputs.positive[0]] = true;
|
||||
if (isNeg && Array.isArray(inputs.negative)) samplerNegativeRef[inputs.negative[0]] = true;
|
||||
}
|
||||
// LTXVConditioning (LTX-2.5) — same shape as InstructPixToPixConditioning:
|
||||
// separate positive/negative inputs feeding separate outputs consumed by
|
||||
// LTXVDualCFGGuider, so each role must route only to its own input.
|
||||
@@ -1108,6 +1295,17 @@ export const comfyWorkflow = {
|
||||
if (isNeg) samplerNegativeRef[tv[0]] = true;
|
||||
}
|
||||
}
|
||||
// TextEncodeQwenImage21 — fixed roles (the node feeds both guider inputs): a linked
|
||||
// "prompt" (prompt-enhancer switch chain, Qwen Image 2.1 Turbo) is the positive text,
|
||||
// a linked "negative_prompt" the negative.
|
||||
// TextEncodeFibo2 (Fibo 2.0 Edit) has the same shape, its prompt arriving via a
|
||||
// PreviewAny -> prompt-enhancer switch chain. PH3ImageConditioning (p-h3-image) and
|
||||
// MiniMaxH3ImageConditioning (MiniMax-H3-Image) too, without a negative prompt.
|
||||
if (ct === "TextEncodeQwenImage21" || ct === "TextEncodeFibo2" || ct === "PH3ImageConditioning"
|
||||
|| ct === "MiniMaxH3ImageConditioning") {
|
||||
if (Array.isArray(inputs.prompt)) samplerPositiveRef[inputs.prompt[0]] = true;
|
||||
if (Array.isArray(inputs.negative_prompt)) samplerNegativeRef[inputs.negative_prompt[0]] = true;
|
||||
}
|
||||
// PreviewAny — generic value tap/relay used mid-chain in prompt-enhancement wiring
|
||||
// (e.g. Krea-2's "Refine Prompt?" switch output tap); pass role through its
|
||||
// "source" input unchanged.
|
||||
@@ -1118,6 +1316,15 @@ export const comfyWorkflow = {
|
||||
if (isNeg) samplerNegativeRef[sv[0]] = true;
|
||||
}
|
||||
}
|
||||
// PrimitiveStringMultiline/PrimitiveString whose value is linked (MOSS-Image's
|
||||
// layout-prompt workflows feed it from a PreviewAny relay) — pass the role upstream.
|
||||
if (ct === "PrimitiveStringMultiline" || ct === "PrimitiveString") {
|
||||
const tv = inputs.value;
|
||||
if (Array.isArray(tv)) {
|
||||
if (isPos) samplerPositiveRef[tv[0]] = true;
|
||||
if (isNeg) samplerNegativeRef[tv[0]] = true;
|
||||
}
|
||||
}
|
||||
// StringConcatenate — appends e.g. a LoRA trigger word onto the prompt text
|
||||
// (Krea-2); propagate role through both operands so the upstream literal is found.
|
||||
if (ct === "StringConcatenate") {
|
||||
@@ -1132,6 +1339,19 @@ export const comfyWorkflow = {
|
||||
}
|
||||
}
|
||||
|
||||
// ReferenceMultiLatent outputs its own empty latent sized from the reference image, so a
|
||||
// template's leftover EmptyFlux2LatentImage that nothing consumes does nothing — don't offer
|
||||
// its width/height in Settings.
|
||||
const hasRefMultiLatent = Object.values(workflow).some((n) => n?.class_type === "ReferenceMultiLatent");
|
||||
const referencedIds = new Set();
|
||||
if (hasRefMultiLatent) {
|
||||
for (const n of Object.values(workflow)) {
|
||||
for (const v of Object.values(n?.inputs || {})) {
|
||||
if (Array.isArray(v) && v.length === 2) referencedIds.add(String(v[0]));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// === Pass 2: Main analysis ===
|
||||
for (const [id, node] of Object.entries(workflow)) {
|
||||
if (!node?.class_type) continue;
|
||||
@@ -1327,7 +1547,12 @@ export const comfyWorkflow = {
|
||||
// with no images connected the node behaves as pure T2I). The reference images
|
||||
// themselves need no special handling here — they arrive as ordinary LoadImage
|
||||
// nodes already picked up generically by the load_image_nodes pass below.
|
||||
if (ct === "TextEncodeQwenImage21") {
|
||||
// TextEncodeFibo2 (Fibo 2.0) — same prompt/negative_prompt/images.image_N shape; its
|
||||
// negative_prompt is a structured VGL JSON caption (edited as plain text here).
|
||||
// PH3ImageConditioning (p-h3-image) and MiniMaxH3ImageConditioning (MiniMax-H3-Image) have
|
||||
// prompt and images.image_N, no negative_prompt.
|
||||
if (ct === "TextEncodeQwenImage21" || ct === "TextEncodeFibo2" || ct === "PH3ImageConditioning"
|
||||
|| ct === "MiniMaxH3ImageConditioning") {
|
||||
if (typeof inputs.prompt === "string") {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title: `${title} [positive]`, role: "positive",
|
||||
@@ -1402,8 +1627,30 @@ export const comfyWorkflow = {
|
||||
}
|
||||
}
|
||||
|
||||
// ComfySwitchNode holding the raw prompt as an inline literal (on_false) — the
|
||||
// prompt-enhancer toggle of Qwen Image 2.1 Turbo, where the subgraph's prompt widget
|
||||
// lands directly on the switch instead of a PrimitiveStringMultiline.
|
||||
if (ct === "ComfySwitchNode" && typeof inputs.on_false === "string" && (isPos || isNeg)) {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title, role: getRole(),
|
||||
text: inputs.on_false, textKey: "on_false",
|
||||
});
|
||||
}
|
||||
|
||||
// PrimitiveStringMultiline / PrimitiveString — when feeding a prompt node
|
||||
if ((ct === "PrimitiveStringMultiline" || ct === "PrimitiveString") && (isPos || isNeg)) {
|
||||
// MOSSImageLayoutPrompt (MOSS-Image bounding-box layout) — builds the final prompt from its own
|
||||
// "prompt" widget (whole-image description) plus the CreateBoundingBoxes elements.
|
||||
if (ct === "MOSSImageLayoutPrompt" && typeof inputs.prompt === "string" && (isPos || isNeg)) {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title, role: getRole(),
|
||||
text: inputs.prompt, textKey: "prompt",
|
||||
});
|
||||
}
|
||||
|
||||
// PrimitiveStringMultiline / PrimitiveString — when feeding a prompt node (a linked
|
||||
// value has no literal to edit; its role is propagated upstream in Pass 1b instead)
|
||||
if ((ct === "PrimitiveStringMultiline" || ct === "PrimitiveString") && (isPos || isNeg)
|
||||
&& !Array.isArray(inputs.value)) {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title, role: getRole(),
|
||||
text: inputs.value || "",
|
||||
@@ -1413,11 +1660,21 @@ export const comfyWorkflow = {
|
||||
|
||||
// Video all-in-one nodes (e.g. MiniMaxH3ImageToVideo) — hold their own "prompt"
|
||||
// widget directly instead of going through a separate CLIPTextEncode, and have
|
||||
// no negative counterpart. Matched generically by class name (ImageToVideo /
|
||||
// TextToVideo) rather than a fixed list so future models sharing this pattern
|
||||
// no negative counterpart. Matched generically by class name (VIDEO_ALL_IN_ONE_RE)
|
||||
// rather than a fixed list so future models sharing this pattern
|
||||
// (WAN, HunyuanVideo, etc.) work without a code change — same detection metadata-tab.js
|
||||
// and node_sets_menu.js already use for prompt extraction (see extractPromptsAPI).
|
||||
if ((ct.includes("ImageToVideo") || ct.includes("TextToVideo")) && typeof inputs.prompt === "string") {
|
||||
if (VIDEO_ALL_IN_ONE_RE.test(ct) && typeof inputs.prompt === "string") {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title, role: "positive",
|
||||
text: inputs.prompt, textKey: "prompt",
|
||||
});
|
||||
}
|
||||
|
||||
// Gemini image nodes (Nano Banana etc., partner API nodes) — like the video all-in-one nodes above
|
||||
// they hold their own "prompt" widget (no negative counterpart; the node's reference images
|
||||
// arrive through LoadImage nodes, which GenerateUI's Image tab already handles generically).
|
||||
if (/^Gemini(Image|NanoBanana)/.test(ct) && typeof inputs.prompt === "string") {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title, role: "positive",
|
||||
text: inputs.prompt, textKey: "prompt",
|
||||
@@ -1433,12 +1690,14 @@ export const comfyWorkflow = {
|
||||
// has the same width/height/batch_size shape plus a "layers" count, left as-is.
|
||||
if (ct === "EmptyLatentImage" || ct === "EmptySD3LatentImage" || ct === "EmptyFlux2LatentImage"
|
||||
|| ct === "EmptyQwenImageLayeredLatentImage") {
|
||||
result.latent_nodes.push({
|
||||
id, type: ct, title,
|
||||
width: typeof inputs.width === "number" ? inputs.width : undefined,
|
||||
height: typeof inputs.height === "number" ? inputs.height : undefined,
|
||||
batch_size: inputs.batch_size,
|
||||
});
|
||||
if (!hasRefMultiLatent || referencedIds.has(String(id))) {
|
||||
result.latent_nodes.push({
|
||||
id, type: ct, title,
|
||||
width: typeof inputs.width === "number" ? inputs.width : undefined,
|
||||
height: typeof inputs.height === "number" ? inputs.height : undefined,
|
||||
batch_size: inputs.batch_size,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Video latent nodes (Wan2.2's EmptyHunyuanLatentVideo, LTX-2.5's
|
||||
@@ -1475,7 +1734,11 @@ export const comfyWorkflow = {
|
||||
// width/height are commonly wired to a ResolutionSelector-style helper node
|
||||
// (array link ref) rather than a direct number — leave those as undefined so
|
||||
// the settings-tab UI falls back to its default instead of rendering a link ref.
|
||||
if (ct === "TextEncodeMageFlowEdit") {
|
||||
// TextEncodeFibo2 likewise (0 = Fibo 2.0's trained size for the first image's aspect),
|
||||
// PH3ImageConditioning (0 = image_1's size, or 2912 without images) and
|
||||
// MiniMaxH3ImageConditioning (0 = image_1's size, or 2048 without images).
|
||||
if (ct === "TextEncodeMageFlowEdit" || ct === "TextEncodeFibo2" || ct === "PH3ImageConditioning"
|
||||
|| ct === "MiniMaxH3ImageConditioning") {
|
||||
result.latent_nodes.push({
|
||||
id, type: ct, title,
|
||||
width: typeof inputs.width === "number" ? inputs.width : undefined,
|
||||
@@ -1508,6 +1771,29 @@ export const comfyWorkflow = {
|
||||
}
|
||||
}
|
||||
|
||||
// Native Load LoRA (Model / Text Encoder) — DynamicGroup rows arrive as flat keys
|
||||
// "loras.<row>.lora_name|strength|enabled". One entry per node (first enabled row
|
||||
// is the representative); editing goes through comfyEditor.applyLoraStack.
|
||||
if (ct === "LoadLoraModel" || ct === "LoadLoraTextEncoder") {
|
||||
const isModel = ct === "LoadLoraModel";
|
||||
const rows = new Map();
|
||||
for (const [k, v] of Object.entries(inputs)) {
|
||||
const m = /^loras\.(\d+)\.(lora_name|strength|enabled)$/.exec(k);
|
||||
if (!m) continue;
|
||||
if (!rows.has(+m[1])) rows.set(+m[1], {});
|
||||
rows.get(+m[1])[m[2]] = v;
|
||||
}
|
||||
const first = [...rows.entries()].sort((a, b) => a[0] - b[0]).map(([, r]) => r)
|
||||
.find((r) => r.enabled !== false && typeof r.lora_name === "string");
|
||||
const s = typeof first?.strength === "number" ? first.strength : 1;
|
||||
result.lora_nodes.push({
|
||||
id, type: ct, title, is_lora_stack: true,
|
||||
lora_name: first?.lora_name,
|
||||
strength_model: isModel ? s : 1,
|
||||
strength_clip: isModel ? 1 : s,
|
||||
});
|
||||
}
|
||||
|
||||
// Lora Loader (LoraManager) — loras stored in inputs.loras.__value__
|
||||
if (ct === "Lora Loader (LoraManager)") {
|
||||
result.lora_nodes.push({ id, type: ct, title, is_lora_manager: true });
|
||||
@@ -1630,7 +1916,7 @@ export const comfyWorkflow = {
|
||||
// generation timeout (video generation runs far longer than a still image) — see
|
||||
// generate-tab.js _coreGenerate and lab-tab.js _runLabBatch.
|
||||
isVideoWorkflow(analysis) {
|
||||
if ((analysis?.prompt_nodes || []).some((n) => /ImageToVideo|TextToVideo/.test(n.type))) return true;
|
||||
if ((analysis?.prompt_nodes || []).some((n) => VIDEO_ALL_IN_ONE_RE.test(n.type))) return true;
|
||||
// LTX-2.5/Wan2.2: no all-in-one *ToVideo node — detected instead via their
|
||||
// EmptyHunyuanLatentVideo/EmptyLTXVLatentVideo latent node (see analyzeWorkflow)
|
||||
// or a SaveVideo/CreateVideo node anywhere in the flattened graph.
|
||||
@@ -1665,6 +1951,31 @@ export const comfyWorkflow = {
|
||||
};
|
||||
|
||||
/** Check if a node is display-only (no connected inputs or outputs) */
|
||||
/**
|
||||
* Prompt-enhancer toggle (Qwen Image 2.1 Turbo): the template's subgraph "prompt" port fans out to
|
||||
* both a ComfySwitchNode.on_false (raw text) and TextGenerate.prompt (text fed to the enhancer,
|
||||
* wired to on_true). After flattening these are two independent literals, so editing one in the UI
|
||||
* would leave the other stale. Route both through one shared PrimitiveStringMultiline instead.
|
||||
*/
|
||||
function _unifyPromptEnhancerText(api) {
|
||||
for (const [swId, sw] of Object.entries(api)) {
|
||||
if (sw.class_type !== "ComfySwitchNode" || typeof sw.inputs?.on_false !== "string") continue;
|
||||
const tgRef = sw.inputs.on_true;
|
||||
if (!Array.isArray(tgRef)) continue;
|
||||
const tg = api[String(tgRef[0])];
|
||||
if ((tg?.class_type !== "TextGenerate" && tg?.class_type !== "TextGenerateV2")
|
||||
|| tg.inputs.prompt !== sw.inputs.on_false) continue;
|
||||
const primId = `${swId}:wfs_prompt`;
|
||||
api[primId] = {
|
||||
class_type: "PrimitiveStringMultiline",
|
||||
inputs: { value: sw.inputs.on_false },
|
||||
_meta: { title: "Prompt" },
|
||||
};
|
||||
sw.inputs.on_false = [primId, 0];
|
||||
tg.inputs.prompt = [primId, 0];
|
||||
}
|
||||
}
|
||||
|
||||
function _isDisplayOnlyNode(node) {
|
||||
const hasConnectedOutput = node.outputs?.some(o => o.links && o.links.length > 0);
|
||||
const hasConnectedInput = node.inputs?.some(i => i.link != null);
|
||||
@@ -1701,6 +2012,9 @@ function _getWidgetMapping(nodeType) {
|
||||
ImpactWildcardEncode: ["wildcard_text"],
|
||||
WFS_PromptText: ["positive", "negative"],
|
||||
TextEncodeQwenImage21: ["prompt", "negative_prompt", "resolution"],
|
||||
TextEncodeFibo2: ["prompt", "negative_prompt", "width", "height", "batch_size"],
|
||||
PH3ImageConditioning: ["prompt", "width", "height"],
|
||||
MiniMaxH3ImageConditioning: ["prompt", "width", "height"],
|
||||
TextEncodeMingImageEdit: ["prompt"],
|
||||
EmptyQwenImageLayeredLatentImage: ["width", "height", "layers", "batch_size"],
|
||||
};
|
||||
|
||||
@@ -1,10 +1,15 @@
|
||||
/**
|
||||
* Decision model client (Unsloth Decision API / Laya, Ollama 0.35+ / tev1・nimble).
|
||||
* Decision model client (Unsloth Decision API / Laya, Ollama 0.35+ / tev1・nimble・clef).
|
||||
*
|
||||
* Laya is a *decision* model, not a generator: it takes a state (text or any JSON) plus typed
|
||||
* questions and returns calibrated probabilities in one forward pass — no free-form output, so no
|
||||
* format hallucination, and results can be thresholded directly. Text only (no image input):
|
||||
* image-based judgments must first be turned into text (Tagger tags / VLM caption).
|
||||
* format hallucination, and results can be thresholded directly. Most decision models are text-only
|
||||
* (image-based judgments must first be turned into text — Tagger tags / VLM caption), but Ollama's
|
||||
* "clef" models accept images directly (decide()'s `images` param — base64, no `data:` prefix; use
|
||||
* supportsDecisionVision() first since a non-vision model errors out on an images payload). This
|
||||
* file is ComfyUI-Comic-Creator's decision-client.js origin (CC ported it 2026-09-30) and CC later
|
||||
* added vision support (2026-10, based on ComfyUI-LiveChatStream's judgeImage pattern) which is
|
||||
* ported back here — keep the two in sync going forward.
|
||||
*
|
||||
* Both backends speak the same TypeSafe-compatible POST /v1/systemone. Unsloth needs an API key
|
||||
* even on localhost (unless Keyless API access is on), so its requests go through the same
|
||||
@@ -36,7 +41,8 @@ export const DECISION_LIMITS = { maxQuestions: 64, maxChoiceOptions: 255, maxSco
|
||||
// as JSON records — e.g. workflow node summaries).
|
||||
// Ollama: decision models are ordinary pulled models whose /api/tags capabilities include
|
||||
// "decision" — listed live by listDecisionModels(); `models` below are only pull suggestions
|
||||
// (tev1 = 4B / 4.5 GB, tev1:0.8b = 812 MB, nimble = 9B / 9.5 GB).
|
||||
// (tev1 = 4B / 4.5 GB, tev1:0.8b = 812 MB, nimble = 9B / 9.5 GB, clef = 27B / ~18GB vision-capable —
|
||||
// needs Ollama 0.35.1+, clef-flash(9B) is unreliable as of Ollama 0.40.1, see ollama/ollama#18769).
|
||||
export const DECISION_BACKENDS = {
|
||||
unsloth: {
|
||||
label: "Unsloth",
|
||||
@@ -47,7 +53,7 @@ export const DECISION_BACKENDS = {
|
||||
ollama: {
|
||||
label: "Ollama",
|
||||
defaultUrl: "http://localhost:11434",
|
||||
models: ["tev1", "tev1:0.8b", "nimble"],
|
||||
models: ["tev1", "tev1:0.8b", "nimble", "clef"],
|
||||
defaultModel: "tev1",
|
||||
},
|
||||
};
|
||||
@@ -153,13 +159,17 @@ function _validateQuestions(questions) {
|
||||
* @param {string|object} state Text or any JSON (e.g. { prompt, tags, model }).
|
||||
* @param {object} questions { key: noul(...) | choice(...) | score(...) }, max 64.
|
||||
* @param {object} [opts] { backend, model, baseUrl } — override the saved Decision Model settings.
|
||||
* @param {string[]} [images] Base64 image data (no `data:` prefix), shared by all questions.
|
||||
* Vision-capable Ollama models only (e.g. clef) — check with
|
||||
* supportsDecisionVision() first; a non-vision model errors out.
|
||||
*/
|
||||
export async function decide(state, questions, opts = {}) {
|
||||
export async function decide(state, questions, opts = {}, images = []) {
|
||||
_validateQuestions(questions);
|
||||
const settings = getDecisionSettings();
|
||||
const backend = opts.backend || settings.backend;
|
||||
const baseUrl = (opts.baseUrl || settings.baseUrl).replace(/\/+$/, "");
|
||||
const payload = { model: opts.model || settings.model, state, questions };
|
||||
if (images?.length) payload.images = images;
|
||||
let data;
|
||||
if (backend === "ollama") {
|
||||
const res = await fetch(`${baseUrl}/v1/systemone`, {
|
||||
@@ -177,6 +187,45 @@ export async function decide(state, questions, opts = {}) {
|
||||
return data.answers;
|
||||
}
|
||||
|
||||
/** Blob / data URL / image URL → base64 (no `data:` prefix), for decide()'s `images` param. */
|
||||
export async function imageToBase64(src) {
|
||||
const blob = src instanceof Blob ? src : await (await fetch(src)).blob();
|
||||
return await new Promise((resolve, reject) => {
|
||||
const r = new FileReader();
|
||||
r.onload = () => resolve(String(r.result).split(",")[1] || "");
|
||||
r.onerror = () => reject(r.error);
|
||||
r.readAsDataURL(blob);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the configured decision model accepts images. Ollama only — decision models' /api/show
|
||||
* capabilities list only "decision" (not "vision") on some Ollama builds, so this also checks for an
|
||||
* image encoder (projector_info) as a fallback. Unsloth can't be checked this way, so it returns
|
||||
* null (unknown) — callers should try and fall back to text-only on failure. Returns false on error.
|
||||
*
|
||||
* @param {object} [opts] { backend, model, baseUrl } — same overrides as decide().
|
||||
*/
|
||||
export async function supportsDecisionVision(opts = {}) {
|
||||
const settings = getDecisionSettings();
|
||||
const backend = opts.backend || settings.backend;
|
||||
if (backend !== "ollama") return null;
|
||||
const baseUrl = (opts.baseUrl || settings.baseUrl).replace(/\/+$/, "");
|
||||
try {
|
||||
const res = await fetch(`${baseUrl}/api/show`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ model: opts.model || settings.model }),
|
||||
});
|
||||
if (!res.ok) return false;
|
||||
const data = await res.json();
|
||||
return (Array.isArray(data.capabilities) && data.capabilities.includes("vision"))
|
||||
|| !!(data.projector_info && Object.keys(data.projector_info).length);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run decide() over many states with a small concurrency cap (the model runs locally, so flooding
|
||||
* it only queues requests). Per-item failures are returned as { error } instead of rejecting all.
|
||||
|
||||
@@ -0,0 +1,431 @@
|
||||
/**
|
||||
* Gallery 3D viewer: glb / gltf / vrm / ply (mesh, point cloud, Gaussian splat) / spz.
|
||||
*
|
||||
* three.js is loaded the first time a 3D file is shown (static/lib/three-viewer).
|
||||
* The server has no 3D renderer, so thumbnails are rendered here and uploaded
|
||||
* (see ensureModelThumb); until then /thumb returns 404 and the card shows a placeholder.
|
||||
*/
|
||||
import { t } from "./i18n.js";
|
||||
|
||||
const LIB_URL = "/wfm_static/lib/three-viewer/three-viewer.js";
|
||||
const DRACO_PATH = "/wfm_static/lib/three-viewer/draco/";
|
||||
// Sentinel base for relative URIs in .gltf files; rewritten to the asset route on load.
|
||||
const ASSET_BASE = "/__wfm3d__/";
|
||||
const BACKGROUND = 0x181c26;
|
||||
const THUMB_SIZE = 512;
|
||||
const THUMB_TIMEOUT_MS = 60_000;
|
||||
// Larger files only get a thumbnail when opened, not while scrolling the grid.
|
||||
const AUTO_THUMB_MAX_BYTES = 128 * 1024 * 1024;
|
||||
// Splats are sorted asynchronously and spring bones settle over time, so keep drawing
|
||||
// for a while after each change instead of only once.
|
||||
const SETTLE_FRAMES = 60;
|
||||
|
||||
const MODEL3D_EXTENSIONS = [".glb", ".gltf", ".vrm", ".ply", ".spz"];
|
||||
|
||||
export function isModel3dFile(img) {
|
||||
const source = (img?.ext || img?.filename || img?.path || "").toLowerCase();
|
||||
return MODEL3D_EXTENSIONS.some((ext) => source.endsWith(ext));
|
||||
}
|
||||
|
||||
export function model3dExtLabel(img) {
|
||||
const source = (img?.ext || img?.filename || img?.path || "").toLowerCase();
|
||||
return (MODEL3D_EXTENSIONS.find((ext) => source.endsWith(ext)) || "").slice(1).toUpperCase();
|
||||
}
|
||||
|
||||
let _libPromise = null;
|
||||
function loadLib() {
|
||||
if (!_libPromise) {
|
||||
_libPromise = import(LIB_URL).catch((e) => {
|
||||
_libPromise = null;
|
||||
throw e;
|
||||
});
|
||||
}
|
||||
return _libPromise;
|
||||
}
|
||||
|
||||
const serveUrl = (path) => `/wfm/gallery/image/serve?path=${encodeURIComponent(path)}`;
|
||||
const assetUrl = (path, uri) => `/wfm/gallery/model3d/asset?path=${encodeURIComponent(path)}&uri=${encodeURIComponent(uri)}`;
|
||||
|
||||
async function fetchBytes(url, signal, onProgress) {
|
||||
const res = await fetch(url, { signal });
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const total = Number(res.headers.get("Content-Length")) || 0;
|
||||
if (!total || !res.body || !onProgress) return res.arrayBuffer();
|
||||
const out = new Uint8Array(total);
|
||||
const reader = res.body.getReader();
|
||||
let received = 0;
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
out.set(value, received);
|
||||
received += value.length;
|
||||
onProgress(received / total);
|
||||
}
|
||||
return out.buffer.slice(0, received);
|
||||
}
|
||||
|
||||
// 3DGS exports are .ply too; they are told apart by their per-splat properties.
|
||||
function isSplatPly(buffer) {
|
||||
const head = new TextDecoder("latin1").decode(new Uint8Array(buffer, 0, Math.min(buffer.byteLength, 8192)));
|
||||
const end = head.indexOf("end_header");
|
||||
const header = end >= 0 ? head.slice(0, end) : head;
|
||||
return /property\s+\w+\s+f_dc_0\b/.test(header) || /property\s+\w+\s+scale_0\b/.test(header);
|
||||
}
|
||||
|
||||
function countGeometry(root) {
|
||||
let vertices = 0;
|
||||
let triangles = 0;
|
||||
root.traverse((o) => {
|
||||
const g = o.geometry;
|
||||
if (!g?.attributes?.position) return;
|
||||
vertices += g.attributes.position.count;
|
||||
if (o.isMesh) triangles += (g.index ? g.index.count : g.attributes.position.count) / 3;
|
||||
});
|
||||
return { vertices, triangles: Math.round(triangles) };
|
||||
}
|
||||
|
||||
async function loadGltf(lib, img, buffer, isVrm) {
|
||||
const manager = new lib.THREE.LoadingManager();
|
||||
manager.setURLModifier((url) => (url.startsWith(ASSET_BASE) ? assetUrl(img.path, url.slice(ASSET_BASE.length)) : url));
|
||||
const loader = new lib.GLTFLoader(manager);
|
||||
const draco = new lib.DRACOLoader(manager).setDecoderPath(DRACO_PATH);
|
||||
loader.setDRACOLoader(draco);
|
||||
loader.setMeshoptDecoder(lib.MeshoptDecoder);
|
||||
if (isVrm) loader.register((parser) => new lib.VRMLoaderPlugin(parser));
|
||||
let gltf;
|
||||
try {
|
||||
gltf = await loader.parseAsync(buffer, ASSET_BASE);
|
||||
} finally {
|
||||
draco.dispose();
|
||||
}
|
||||
|
||||
const vrm = gltf.userData?.vrm;
|
||||
if (vrm) lib.VRMUtils.rotateVRM0(vrm); // VRM 0.x faces -Z
|
||||
const object = vrm ? vrm.scene : gltf.scene;
|
||||
const mixer = gltf.animations?.length ? new lib.THREE.AnimationMixer(object) : null;
|
||||
if (mixer) mixer.clipAction(gltf.animations[0]).play();
|
||||
return {
|
||||
object,
|
||||
animated: !!mixer,
|
||||
settles: !!vrm,
|
||||
update: (dt) => { mixer?.update(dt); vrm?.update(dt); },
|
||||
stats: countGeometry(object),
|
||||
};
|
||||
}
|
||||
|
||||
function loadPlyGeometry(lib, buffer) {
|
||||
const { THREE } = lib;
|
||||
const geometry = new lib.PLYLoader().parse(buffer);
|
||||
// "property double" (e.g. Open3D) comes back as a Float64Array, which WebGL rejects.
|
||||
for (const [name, attr] of Object.entries(geometry.attributes)) {
|
||||
if (attr.array instanceof Float64Array) {
|
||||
geometry.setAttribute(name, new THREE.BufferAttribute(Float32Array.from(attr.array), attr.itemSize, attr.normalized));
|
||||
}
|
||||
}
|
||||
const hasColor = !!geometry.attributes.color;
|
||||
if (!geometry.index) {
|
||||
geometry.computeBoundingSphere();
|
||||
const material = new THREE.PointsMaterial({
|
||||
size: geometry.boundingSphere.radius / 300,
|
||||
vertexColors: hasColor,
|
||||
color: hasColor ? 0xffffff : 0xb8c0cc,
|
||||
});
|
||||
const object = new THREE.Points(geometry, material);
|
||||
return { object, stats: countGeometry(object) };
|
||||
}
|
||||
if (!geometry.attributes.normal) geometry.computeVertexNormals();
|
||||
const material = new THREE.MeshStandardMaterial({
|
||||
vertexColors: hasColor,
|
||||
color: hasColor ? 0xffffff : 0xb8c0cc,
|
||||
roughness: 0.7,
|
||||
side: THREE.DoubleSide,
|
||||
});
|
||||
const object = new THREE.Mesh(geometry, material);
|
||||
return { object, stats: countGeometry(object) };
|
||||
}
|
||||
|
||||
async function loadSplat(lib, stage, img, buffer) {
|
||||
const { SparkRenderer, SplatMesh } = await lib.loadSpark();
|
||||
const spark = new SparkRenderer({ renderer: stage.renderer });
|
||||
stage.scene.add(spark);
|
||||
const mesh = new SplatMesh({ fileBytes: new Uint8Array(buffer), fileName: img.filename || img.path });
|
||||
await mesh.initialized;
|
||||
mesh.quaternion.set(1, 0, 0, 0); // splats are stored Y-down
|
||||
return {
|
||||
object: mesh,
|
||||
splat: true,
|
||||
settles: true,
|
||||
box: mesh.getBoundingBox(true),
|
||||
stats: { splats: mesh.packedSplats?.numSplats ?? mesh.numSplats ?? 0 },
|
||||
dispose: () => { mesh.dispose(); spark.dispose(); },
|
||||
};
|
||||
}
|
||||
|
||||
async function loadInto(lib, stage, img, signal, onProgress) {
|
||||
const buffer = await fetchBytes(serveUrl(img.path), signal, onProgress);
|
||||
const name = (img.filename || img.path).toLowerCase();
|
||||
if (name.endsWith(".spz")) return loadSplat(lib, stage, img, buffer);
|
||||
if (name.endsWith(".ply")) {
|
||||
return isSplatPly(buffer) ? loadSplat(lib, stage, img, buffer) : loadPlyGeometry(lib, buffer);
|
||||
}
|
||||
return loadGltf(lib, img, buffer, name.endsWith(".vrm"));
|
||||
}
|
||||
|
||||
function createStage(lib, canvas) {
|
||||
const { THREE } = lib;
|
||||
const renderer = new THREE.WebGLRenderer({ canvas, antialias: true, preserveDrawingBuffer: false });
|
||||
renderer.setPixelRatio(Math.min(window.devicePixelRatio || 1, 2));
|
||||
const scene = new THREE.Scene();
|
||||
scene.background = new THREE.Color(BACKGROUND);
|
||||
const pmrem = new THREE.PMREMGenerator(renderer);
|
||||
const envTarget = pmrem.fromScene(new lib.RoomEnvironment(), 0.04);
|
||||
pmrem.dispose();
|
||||
scene.environment = envTarget.texture;
|
||||
scene.environmentIntensity = 0.8;
|
||||
const sun = new THREE.DirectionalLight(0xffffff, 1.2);
|
||||
sun.position.set(2, 4, 3);
|
||||
scene.add(sun);
|
||||
const camera = new THREE.PerspectiveCamera(35, 1, 0.01, 1000);
|
||||
return { renderer, scene, camera, envTarget };
|
||||
}
|
||||
|
||||
function frameObject(lib, stage, model) {
|
||||
const { THREE } = lib;
|
||||
const { camera, scene } = stage;
|
||||
scene.add(model.object);
|
||||
model.object.updateMatrixWorld(true);
|
||||
const box = model.box
|
||||
? model.box.clone().applyMatrix4(model.object.matrixWorld)
|
||||
: new THREE.Box3().setFromObject(model.object);
|
||||
if (box.isEmpty()) box.set(new THREE.Vector3(-1, -1, -1), new THREE.Vector3(1, 1, 1));
|
||||
const center = box.getCenter(new THREE.Vector3());
|
||||
const radius = Math.max(box.getSize(new THREE.Vector3()).length() / 2, 1e-4);
|
||||
const distance = (radius / Math.sin(THREE.MathUtils.degToRad(camera.fov / 2))) * 1.05;
|
||||
camera.near = distance / 500;
|
||||
camera.far = distance * 50;
|
||||
camera.position.copy(center).add(new THREE.Vector3(0.35, 0.2, 1).normalize().multiplyScalar(distance));
|
||||
camera.lookAt(center);
|
||||
camera.updateProjectionMatrix();
|
||||
return center;
|
||||
}
|
||||
|
||||
function disposeStage(stage, model) {
|
||||
try { model?.dispose?.(); } catch { /* already gone */ }
|
||||
model?.object?.traverse?.((o) => {
|
||||
o.geometry?.dispose?.();
|
||||
for (const m of [].concat(o.material || [])) {
|
||||
for (const v of Object.values(m)) if (v?.isTexture) v.dispose();
|
||||
m.dispose?.();
|
||||
}
|
||||
});
|
||||
stage.envTarget.dispose();
|
||||
stage.renderer.dispose();
|
||||
stage.renderer.forceContextLoss(); // browsers only allow ~16 live contexts
|
||||
}
|
||||
|
||||
function formatStats(stats) {
|
||||
const n = (v) => Number(v).toLocaleString();
|
||||
if (stats.splats != null) return t("gallery3dStatsSplats", n(stats.splats));
|
||||
if (!stats.triangles) return t("gallery3dStatsPoints", n(stats.vertices));
|
||||
return t("gallery3dStatsMesh", n(stats.vertices), n(stats.triangles));
|
||||
}
|
||||
|
||||
// Center-crop the canvas to a square JPEG. Must run right after render() in the same task,
|
||||
// because the drawing buffer is not preserved.
|
||||
function captureJpeg(canvas) {
|
||||
const out = document.createElement("canvas");
|
||||
out.width = out.height = THUMB_SIZE;
|
||||
const side = Math.min(canvas.width, canvas.height);
|
||||
out.getContext("2d").drawImage(
|
||||
canvas, (canvas.width - side) / 2, (canvas.height - side) / 2, side, side, 0, 0, THUMB_SIZE, THUMB_SIZE,
|
||||
);
|
||||
return out.toDataURL("image/jpeg", 0.85);
|
||||
}
|
||||
|
||||
async function uploadThumb(path, image) {
|
||||
const res = await fetch("/wfm/gallery/image/thumb3d", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ path, image }),
|
||||
});
|
||||
return res.ok;
|
||||
}
|
||||
|
||||
/**
|
||||
* Interactive viewer inside `container`. It disposes itself once `container` leaves the
|
||||
* DOM, so callers can simply replace or remove the element.
|
||||
* `onReady(capture)` fires after the model is drawn; capture() returns a thumbnail data URL.
|
||||
*/
|
||||
export function mountModelViewer(container, img, { onReady } = {}) {
|
||||
container.classList.add("wfm-g3d");
|
||||
container.innerHTML = `
|
||||
<canvas class="wfm-g3d-canvas"></canvas>
|
||||
<div class="wfm-g3d-status">${t("gallery3dLoading")}</div>
|
||||
<div class="wfm-g3d-stats"></div>
|
||||
<button type="button" class="wfm-g3d-reset" title="${t("gallery3dResetView")}">⟲</button>`;
|
||||
const canvas = container.querySelector(".wfm-g3d-canvas");
|
||||
const status = container.querySelector(".wfm-g3d-status");
|
||||
const statsEl = container.querySelector(".wfm-g3d-stats");
|
||||
const resetBtn = container.querySelector(".wfm-g3d-reset");
|
||||
const abort = new AbortController();
|
||||
let stage = null;
|
||||
let model = null;
|
||||
let controls = null;
|
||||
let resizeObserver = null;
|
||||
let frameId = 0;
|
||||
let closed = false;
|
||||
|
||||
const close = () => {
|
||||
if (closed) return;
|
||||
closed = true;
|
||||
abort.abort();
|
||||
cancelAnimationFrame(frameId);
|
||||
resizeObserver?.disconnect();
|
||||
controls?.dispose();
|
||||
if (stage) disposeStage(stage, model);
|
||||
};
|
||||
|
||||
const fail = (e) => {
|
||||
if (closed || e?.name === "AbortError") return;
|
||||
console.error("[WFS] 3D viewer:", img.path, e);
|
||||
status.textContent = t("gallery3dLoadFailed");
|
||||
status.classList.add("wfm-g3d-status--error");
|
||||
status.style.display = "";
|
||||
close();
|
||||
};
|
||||
|
||||
(async () => {
|
||||
const lib = await loadLib();
|
||||
if (closed) return;
|
||||
stage = createStage(lib, canvas);
|
||||
const { renderer, scene, camera } = stage;
|
||||
model = await loadInto(lib, stage, img, abort.signal, (p) => {
|
||||
status.textContent = `${t("gallery3dLoading")} ${Math.round(p * 100)}%`;
|
||||
});
|
||||
if (closed) {
|
||||
disposeStage(stage, model);
|
||||
return;
|
||||
}
|
||||
const center = frameObject(lib, stage, model);
|
||||
const home = camera.position.clone();
|
||||
controls = new lib.OrbitControls(camera, canvas);
|
||||
controls.target.copy(center);
|
||||
controls.enableDamping = true;
|
||||
controls.update();
|
||||
|
||||
let dirty = SETTLE_FRAMES;
|
||||
const invalidate = () => { dirty = SETTLE_FRAMES; };
|
||||
controls.addEventListener("change", invalidate);
|
||||
resetBtn.addEventListener("click", () => {
|
||||
camera.position.copy(home);
|
||||
controls.target.copy(center);
|
||||
controls.update();
|
||||
invalidate();
|
||||
});
|
||||
const resize = () => {
|
||||
const w = container.clientWidth;
|
||||
const h = container.clientHeight;
|
||||
if (!w || !h) return;
|
||||
renderer.setSize(w, h, false);
|
||||
camera.aspect = w / h;
|
||||
camera.updateProjectionMatrix();
|
||||
invalidate();
|
||||
};
|
||||
resizeObserver = new ResizeObserver(resize);
|
||||
resizeObserver.observe(container);
|
||||
resize();
|
||||
|
||||
renderer.render(scene, camera); // surface a model that cannot be drawn as a load failure
|
||||
status.style.display = "none";
|
||||
statsEl.textContent = formatStats(model.stats);
|
||||
onReady?.(() => {
|
||||
renderer.render(scene, camera);
|
||||
return captureJpeg(canvas);
|
||||
});
|
||||
|
||||
let last = performance.now();
|
||||
const tick = () => {
|
||||
if (!container.isConnected) {
|
||||
close();
|
||||
return;
|
||||
}
|
||||
frameId = requestAnimationFrame(tick);
|
||||
const now = performance.now();
|
||||
const dt = (now - last) / 1000;
|
||||
last = now;
|
||||
if (!container.clientWidth) return; // hidden tab
|
||||
controls.update();
|
||||
if (!model.animated && dirty <= 0) return;
|
||||
dirty -= 1;
|
||||
try {
|
||||
model.update?.(dt);
|
||||
renderer.render(scene, camera);
|
||||
} catch (e) {
|
||||
fail(e);
|
||||
}
|
||||
};
|
||||
tick();
|
||||
})().catch(fail);
|
||||
|
||||
return { close };
|
||||
}
|
||||
|
||||
async function renderThumb(img) {
|
||||
const lib = await loadLib();
|
||||
const canvas = document.createElement("canvas");
|
||||
const stage = createStage(lib, canvas);
|
||||
stage.renderer.setPixelRatio(1);
|
||||
stage.renderer.setSize(THUMB_SIZE, THUMB_SIZE, false);
|
||||
const abort = new AbortController();
|
||||
const timer = setTimeout(() => abort.abort(), THUMB_TIMEOUT_MS);
|
||||
let model = null;
|
||||
try {
|
||||
model = await loadInto(lib, stage, img, abort.signal);
|
||||
frameObject(lib, stage, model);
|
||||
model.update?.(0);
|
||||
for (let i = 0; model.splat && i < SETTLE_FRAMES; i++) {
|
||||
stage.renderer.render(stage.scene, stage.camera);
|
||||
await new Promise((r) => requestAnimationFrame(r));
|
||||
}
|
||||
stage.renderer.render(stage.scene, stage.camera);
|
||||
return captureJpeg(canvas);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
disposeStage(stage, model);
|
||||
}
|
||||
}
|
||||
|
||||
const _thumbJobs = new Map(); // path -> Promise<boolean>, also remembers failures for this session
|
||||
let _thumbQueue = Promise.resolve();
|
||||
|
||||
/**
|
||||
* Render and store the thumbnail of a 3D file, one file at a time.
|
||||
* `isWanted()` is checked when the job's turn comes, so cards scrolled away or replaced by
|
||||
* a folder change cost nothing. Resolves true once the server has the thumbnail.
|
||||
*/
|
||||
export function ensureModelThumb(img, isWanted = () => true) {
|
||||
if (_thumbJobs.has(img.path)) return _thumbJobs.get(img.path);
|
||||
if (img.size > AUTO_THUMB_MAX_BYTES) return Promise.resolve(false);
|
||||
const job = _thumbQueue.then(async () => {
|
||||
if (!isWanted()) {
|
||||
_thumbJobs.delete(img.path);
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
return await uploadThumb(img.path, await renderThumb(img));
|
||||
} catch (e) {
|
||||
console.warn("[WFS] 3D thumbnail failed:", img.path, e);
|
||||
return false;
|
||||
}
|
||||
});
|
||||
_thumbJobs.set(img.path, job);
|
||||
_thumbQueue = job;
|
||||
return job;
|
||||
}
|
||||
|
||||
/** Store a thumbnail captured from an open viewer (covers files too large for ensureModelThumb). */
|
||||
export function saveModelThumb(img, capture) {
|
||||
const job = uploadThumb(img.path, capture()).catch(() => false);
|
||||
_thumbJobs.set(img.path, job);
|
||||
return job;
|
||||
}
|
||||
@@ -0,0 +1,571 @@
|
||||
/**
|
||||
* Gallery Tab - セマンティック検索 (Gemini Embedding 2 / Ollama の EmbeddingGemma 2)
|
||||
* - 検索ボックスのSemantic Searchトグル（ONでEnter実行の意味検索）
|
||||
* - 詳細パネルの「類似画像」トグル／更新ボタン
|
||||
* - インデックス作成モーダル（Semantic⚙）
|
||||
* 一覧の差し替えは gallery-tab.js の loadImages() が getSemanticMode() / fetchSemanticImages() を呼んで行う。
|
||||
*/
|
||||
|
||||
import { openModal, closeModal, showToast } from "./app.js";
|
||||
import { t } from "./i18n.js";
|
||||
import { escapeHtml } from "./util.js";
|
||||
|
||||
const API = {
|
||||
status: (folder, recursive) => `/wfm/gallery/semantic/status?folder=${encodeURIComponent(folder)}&recursive=${recursive}`,
|
||||
index: "/wfm/gallery/semantic/index",
|
||||
cancel: "/wfm/gallery/semantic/cancel",
|
||||
search: "/wfm/gallery/semantic/search",
|
||||
similar: "/wfm/gallery/semantic/similar",
|
||||
clear: "/wfm/gallery/semantic/clear",
|
||||
folders: (root) => `/wfm/gallery/semantic/folders?root=${encodeURIComponent(root)}`,
|
||||
test: "/wfm/gallery/semantic/test",
|
||||
options: "/wfm/gallery/semantic/options",
|
||||
};
|
||||
|
||||
const LS_ON = "wfm_gallery_semantic_on";
|
||||
const LS_DIM = "wfm_gallery_semantic_dim";
|
||||
const LS_MODEL = "wfm_gallery_semantic_model";
|
||||
const GEMINI = "gemini-embedding-2";
|
||||
const LOCAL = "embeddinggemma-2";
|
||||
const LS_SUB = "wfm_gallery_semantic_recursive";
|
||||
const LS_TAB = "wfm_gallery_semantic_tab";
|
||||
const LS_TH = "wfm_gallery_semantic_th";
|
||||
const LIMITS = [50, 100, 200, 500, 1000];
|
||||
// Ollama のモデルサイズ(どちらも同じベクトルを返すので、切り替えても再作成は不要)
|
||||
const OLLAMA_TAGS = [["embeddinggemma-2:740m", "semOllamaTag740"], ["embeddinggemma-2:440m", "semOllamaTag440"]];
|
||||
|
||||
const sem = {
|
||||
on: false, // Semantic Searchトグル
|
||||
anchor: "", // 類似画像表示中の基準画像パス（空=類似表示OFF）
|
||||
anchorName: "",
|
||||
indexModel: GEMINI, // 現在のインデックスを作ったモデル（しきい値はモデルごとに保存する）
|
||||
};
|
||||
|
||||
let ctx = null; // { getSearch, getFolder, getSelected, reload, focusSearch }
|
||||
|
||||
function lsGet(key, fallback) {
|
||||
try { return localStorage.getItem(key) ?? fallback; } catch { return fallback; }
|
||||
}
|
||||
function lsSet(key, value) {
|
||||
try { localStorage.setItem(key, value); } catch { /* 保存できなくても動作に影響しない */ }
|
||||
}
|
||||
|
||||
async function postJson(url, body) {
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body || {}),
|
||||
});
|
||||
let data = {};
|
||||
try { data = await res.json(); } catch { /* 本文なし */ }
|
||||
if (!res.ok) {
|
||||
throw Object.assign(new Error(data.error || `HTTP ${res.status}`), { code: data.code || "" });
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
/** APIエラーコードをi18nメッセージにする */
|
||||
export function semanticErrorMessage(e) {
|
||||
const known = ["no_api_key", "api_key_invalid", "index_empty", "dim_mismatch", "already_running",
|
||||
"folder_not_allowed", "unsupported_type", "network_error", "api_error", "local_error",
|
||||
"ollama_unreachable", "ollama_model_missing", "bad_url"];
|
||||
if (e?.code && known.includes(e.code)) {
|
||||
return t(`sem_err_${e.code}`, e.message);
|
||||
}
|
||||
return t("errorWithMsg", e?.message || String(e));
|
||||
}
|
||||
|
||||
// ── gallery-tab.js から使う公開API ────────────────────────────
|
||||
|
||||
/** 現在の一覧モード: {kind:"similar", path} | {kind:"text", query} | null(通常) */
|
||||
export function getSemanticMode() {
|
||||
if (sem.anchor) return { kind: "similar", path: sem.anchor, name: sem.anchorName };
|
||||
const q = (ctx?.getSearch() || "").trim();
|
||||
if (sem.on && q) return { kind: "text", query: q };
|
||||
return null;
|
||||
}
|
||||
|
||||
export function isSemanticOn() {
|
||||
return sem.on;
|
||||
}
|
||||
|
||||
/**
|
||||
* 表示条件（類似度のしきい値・最大件数）。スコアの水準はモデルで違うため、モデルごとに保存する。
|
||||
* similar: 類似画像検索の最小スコア / text: 文章検索の最小スコア（0=絞り込みなし）
|
||||
*/
|
||||
function loadThresholds(model) {
|
||||
let v = {};
|
||||
try { v = JSON.parse(lsGet(LS_TH, "{}"))[model] || {}; } catch { /* 壊れた値は既定に戻す */ }
|
||||
const num = x => { const n = parseFloat(x); return Number.isFinite(n) ? Math.min(1, Math.max(0, n)) : 0; };
|
||||
return { similar: num(v.similar), text: num(v.text), limit: LIMITS.includes(v.limit) ? v.limit : 200 };
|
||||
}
|
||||
|
||||
function saveThresholds(model, th) {
|
||||
let all = {};
|
||||
try { all = JSON.parse(lsGet(LS_TH, "{}")) || {}; } catch { all = {}; }
|
||||
all[model] = th;
|
||||
lsSet(LS_TH, JSON.stringify(all));
|
||||
}
|
||||
|
||||
function minScoreFor(mode) {
|
||||
const th = loadThresholds(sem.indexModel);
|
||||
return mode.kind === "similar" ? th.similar : th.text;
|
||||
}
|
||||
|
||||
/** モードに応じてサーバーから結果（スコア降順・score付き）を取得する */
|
||||
export async function fetchSemanticImages(mode, folder) {
|
||||
const th = loadThresholds(sem.indexModel);
|
||||
const common = { folder, recursive: true, limit: th.limit, min_score: minScoreFor(mode) };
|
||||
const data = mode.kind === "similar"
|
||||
? await postJson(API.similar, { path: mode.path, ...common })
|
||||
: await postJson(API.search, { query: mode.query, ...common });
|
||||
return data.images || [];
|
||||
}
|
||||
|
||||
/** 2行目の結果件数表示（意味検索/類似表示のときだけ文言を入れる。通常は空） */
|
||||
export function setSemanticCountText(text) {
|
||||
const el = document.getElementById("wfm-gallery-sem-count");
|
||||
if (el) el.textContent = text || "";
|
||||
}
|
||||
|
||||
/** 一覧件数ラベル */
|
||||
export function semanticCountLabel(mode, count) {
|
||||
const min = minScoreFor(mode);
|
||||
return mode.kind === "similar"
|
||||
? t("semCountSimilar", count, mode.name, min)
|
||||
: t("semCountText", count, min);
|
||||
}
|
||||
|
||||
let _dotRoot = "";
|
||||
let _dotSeq = 0;
|
||||
|
||||
/**
|
||||
* フォルダツリーの枚数バッジ横に、直下の画像が全件インデックス済みのフォルダへドットを付ける。
|
||||
* ツリー再構築後・インデックス作成後に呼ぶ（DOMはその場で更新し、ツリーは作り直さない）。
|
||||
*/
|
||||
export async function refreshSemanticDots(root) {
|
||||
if (root) _dotRoot = root;
|
||||
if (!_dotRoot) return;
|
||||
const seq = ++_dotSeq;
|
||||
let folders;
|
||||
try {
|
||||
const res = await fetch(API.folders(_dotRoot));
|
||||
if (!res.ok) return;
|
||||
folders = (await res.json()).folders || {};
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (seq !== _dotSeq) return; // より新しい呼び出しがある
|
||||
document.querySelectorAll(".wfm-gallery-tree-item").forEach(item => {
|
||||
const label = item.querySelector(".wfm-gallery-tree-label");
|
||||
const badge = label?.querySelector(".wfm-gallery-tree-count");
|
||||
label?.querySelector(".wfm-gallery-tree-sem-dot")?.remove();
|
||||
const st = folders[(item.dataset.path || "").replace(/\\/g, "/")];
|
||||
if (badge && st && st.indexable > 0 && st.indexed >= st.indexable) {
|
||||
const dot = document.createElement("span");
|
||||
dot.className = "wfm-gallery-tree-sem-dot";
|
||||
dot.title = t("semDotTitle");
|
||||
badge.after(dot);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** フォルダ切替・フィルタクリア時に類似表示を解除する（再読込は呼び出し側） */
|
||||
export function exitSimilarMode() {
|
||||
sem.anchor = "";
|
||||
sem.anchorName = "";
|
||||
updateSimilarButtons();
|
||||
}
|
||||
|
||||
// ── トグル／類似ボタン ───────────────────────────────────────
|
||||
|
||||
function updateToggleUi() {
|
||||
const btn = document.getElementById("wfm-gallery-semantic-toggle");
|
||||
btn?.classList.toggle("active", sem.on);
|
||||
// ON のときだけ2行目（Similar / 更新 / 設定 / 件数）を表示する
|
||||
const row = document.getElementById("wfm-gallery-sem-row");
|
||||
if (row) row.style.display = sem.on ? "flex" : "none";
|
||||
const input = document.getElementById("wfm-gallery-search");
|
||||
if (input) input.placeholder = sem.on ? t("semSearchPlaceholder") : "Search images...";
|
||||
}
|
||||
|
||||
function updateSimilarButtons() {
|
||||
document.getElementById("wfm-gallery-similar-btn")?.classList.toggle("active", !!sem.anchor);
|
||||
}
|
||||
|
||||
async function toggleSimilar() {
|
||||
if (sem.anchor) {
|
||||
exitSimilarMode();
|
||||
ctx.reload();
|
||||
return;
|
||||
}
|
||||
await applySimilar();
|
||||
}
|
||||
|
||||
async function applySimilar() {
|
||||
const img = ctx.getSelected();
|
||||
if (!img) {
|
||||
showToast(t("gallerySelectImageFirst"), "error");
|
||||
return;
|
||||
}
|
||||
sem.anchor = img.path;
|
||||
sem.anchorName = img.filename;
|
||||
updateSimilarButtons();
|
||||
ctx.reload();
|
||||
}
|
||||
|
||||
// ── インデックス作成モーダル ─────────────────────────────────
|
||||
|
||||
function modalHtml() {
|
||||
const tab = ["index", "model", "search"].includes(lsGet(LS_TAB, "index")) ? lsGet(LS_TAB, "index") : "index";
|
||||
const pane = (id, body) => `<div class="wfm-sem-pane" data-sem-pane="${id}" style="${tab === id ? "" : "display:none;"}">${body}</div>`;
|
||||
const tabBtn = (id, label) =>
|
||||
`<button type="button" class="wfm-sem-tab-btn${tab === id ? " active" : ""}" data-sem-tab="${id}">${escapeHtml(label)}</button>`;
|
||||
const indexPane = `
|
||||
<div class="wfm-sem-status" id="wfm-sem-status">${escapeHtml(t("loading"))}</div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semScope"))}</b>
|
||||
<span id="wfm-sem-folder" class="wfm-sem-folder"></span></div>
|
||||
<label class="wfm-sem-row"><input type="checkbox" id="wfm-sem-recursive" ${lsGet(LS_SUB, "1") === "1" ? "checked" : ""}>
|
||||
${escapeHtml(t("semIncludeSub"))}</label>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semMode"))}</b>
|
||||
<label><input type="radio" name="wfm-sem-mode" value="missing" checked> ${escapeHtml(t("semModeMissing"))}</label>
|
||||
<label><input type="radio" name="wfm-sem-mode" value="all"> ${escapeHtml(t("semModeAll"))}</label></div>
|
||||
<div class="wfm-sem-progress" id="wfm-sem-progress" style="display:none;">
|
||||
<div class="wfm-sem-bar"><div class="wfm-sem-bar-fill" id="wfm-sem-bar-fill"></div></div>
|
||||
<div id="wfm-sem-progress-text" class="wfm-sem-hint"></div>
|
||||
</div>
|
||||
<div class="wfm-sem-actions">
|
||||
<button class="wfm-btn wfm-btn-primary" id="wfm-sem-start">${escapeHtml(t("semStart"))}</button>
|
||||
<button class="wfm-btn" id="wfm-sem-cancel" disabled>${escapeHtml(t("semCancelJob"))}</button>
|
||||
<button class="wfm-btn wfm-sem-danger" id="wfm-sem-clear">${escapeHtml(t("semClear"))}</button>
|
||||
</div>`;
|
||||
const modelPane = `
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semModel"))}</b>
|
||||
<label><input type="radio" name="wfm-sem-model" value="${GEMINI}"> ${escapeHtml(t("semModelGemini"))}</label>
|
||||
<label><input type="radio" name="wfm-sem-model" value="${LOCAL}"> ${escapeHtml(t("semModelLocal"))}</label></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semDimension"))}</b>
|
||||
<select id="wfm-sem-dim" class="wfm-select"></select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semDimHint"))}</span></div>
|
||||
<div id="wfm-sem-local-opts" class="wfm-sem-local-opts" style="display:none;">
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semOllamaUrl"))}</b>
|
||||
<input type="text" id="wfm-sem-ollama-url" class="wfm-input wfm-sem-url" spellcheck="false" placeholder="http://localhost:11434"></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semOllamaTag"))}</b>
|
||||
<select id="wfm-sem-ollama-tag" class="wfm-select">
|
||||
${OLLAMA_TAGS.map(([tag, key]) => `<option value="${tag}">${escapeHtml(t(key))}</option>`).join("")}
|
||||
</select>
|
||||
<span id="wfm-sem-ollama-info" class="wfm-sem-hint"></span></div>
|
||||
<p class="wfm-sem-hint">${escapeHtml(t("semOllamaTagHint"))}</p>
|
||||
</div>
|
||||
<div class="wfm-sem-actions">
|
||||
<button class="wfm-btn" id="wfm-sem-test">${escapeHtml(t("semTest"))}</button>
|
||||
</div>
|
||||
<p class="wfm-sem-hint">${escapeHtml(t("semCostNote"))}</p>`;
|
||||
const searchPane = `
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThTarget"))}</b>
|
||||
<span id="wfm-sem-th-model" class="wfm-sem-folder"></span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThSimilar"))}</b>
|
||||
<input type="number" id="wfm-sem-th-similar" class="wfm-input wfm-sem-num" min="0" max="1" step="0.01" placeholder="0">
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semThSimilarHint"))}</span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThText"))}</b>
|
||||
<input type="number" id="wfm-sem-th-text" class="wfm-input wfm-sem-num" min="0" max="1" step="0.01" placeholder="0">
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semThTextHint"))}</span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThLimit"))}</b>
|
||||
<select id="wfm-sem-th-limit" class="wfm-select">${LIMITS.map(n => `<option value="${n}">${n}</option>`).join("")}</select></div>
|
||||
<p class="wfm-sem-hint">${escapeHtml(t("semThNote"))}</p>`;
|
||||
return `
|
||||
<div class="wfm-sem-modal">
|
||||
<div class="wfm-sem-tabs">${tabBtn("index", t("semTabIndex"))}${tabBtn("model", t("semTabModel"))}${tabBtn("search", t("semTabSearch"))}</div>
|
||||
${pane("index", indexPane)}${pane("model", modelPane)}${pane("search", searchPane)}
|
||||
<div class="wfm-sem-actions wfm-sem-footer">
|
||||
<button class="wfm-btn" id="wfm-sem-help" title="${escapeHtml(t("semHelpTitle"))}">?</button>
|
||||
<button class="wfm-btn" id="wfm-sem-close">${escapeHtml(t("close"))}</button>
|
||||
</div>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
function modalAlive() {
|
||||
const overlay = document.getElementById("wfm-modal-overlay");
|
||||
return !!document.getElementById("wfm-sem-status") && overlay && overlay.style.display !== "none";
|
||||
}
|
||||
|
||||
let _pollTimer = null;
|
||||
let _backends = {};
|
||||
let _sawRunning = false;
|
||||
|
||||
function selectedModel() {
|
||||
return document.querySelector('input[name="wfm-sem-model"]:checked')?.value || GEMINI;
|
||||
}
|
||||
|
||||
/** Ollama の接続先/モデルサイズの初期値をサーバー保存値から設定する */
|
||||
function initLocalOpts(st) {
|
||||
const ol = st.ollama || {};
|
||||
document.getElementById("wfm-sem-ollama-url").value = ol.url || "";
|
||||
const sel = document.getElementById("wfm-sem-ollama-tag");
|
||||
if (ol.tag) sel.value = ol.tag;
|
||||
}
|
||||
|
||||
/** 検索タブ: しきい値の入力欄にインデックスのモデル用の保存値を入れる（入力中の値は上書きしない） */
|
||||
function fillThresholds(st) {
|
||||
const model = st.indexed_total ? st.index_model : selectedModel();
|
||||
const box = document.getElementById("wfm-sem-th-model");
|
||||
if (box) box.textContent = model + (st.indexed_total ? "" : ` ${t("semThNoIndex")}`);
|
||||
const pane = document.getElementById("wfm-sem-th-similar");
|
||||
if (!pane || pane.dataset.model === model) return;
|
||||
const th = loadThresholds(model);
|
||||
pane.value = th.similar || "";
|
||||
document.getElementById("wfm-sem-th-text").value = th.text || "";
|
||||
document.getElementById("wfm-sem-th-limit").value = String(th.limit);
|
||||
pane.dataset.model = model;
|
||||
}
|
||||
|
||||
/** 検索タブの入力を保存する。一覧が意味検索/類似表示中なら閉じるときに再読込する */
|
||||
let _thDirty = false;
|
||||
function saveThresholdInputs() {
|
||||
const model = document.getElementById("wfm-sem-th-similar")?.dataset.model;
|
||||
if (!model) return;
|
||||
const clamp = id => {
|
||||
const n = parseFloat(document.getElementById(id).value);
|
||||
return Number.isFinite(n) ? Math.min(1, Math.max(0, n)) : 0;
|
||||
};
|
||||
saveThresholds(model, {
|
||||
similar: clamp("wfm-sem-th-similar"),
|
||||
text: clamp("wfm-sem-th-text"),
|
||||
limit: parseInt(document.getElementById("wfm-sem-th-limit").value, 10) || 200,
|
||||
});
|
||||
_thDirty = true;
|
||||
}
|
||||
|
||||
/** モーダルを閉じる共通処理 */
|
||||
function closeSemanticModal() {
|
||||
clearTimeout(_pollTimer);
|
||||
closeModal();
|
||||
refreshSemanticDots();
|
||||
if (_thDirty) {
|
||||
_thDirty = false;
|
||||
if (getSemanticMode()) ctx.reload(); // 新しいしきい値で一覧を取り直す
|
||||
}
|
||||
}
|
||||
|
||||
/** モデルサイズの選択肢に取得済みかどうかを表示する */
|
||||
function updateOllamaInfo(ol) {
|
||||
if (!ol) return;
|
||||
document.querySelectorAll("#wfm-sem-ollama-tag option").forEach(opt => {
|
||||
const key = OLLAMA_TAGS.find(([tag]) => tag === opt.value)?.[1];
|
||||
if (key) opt.textContent = t(key) + (ol.installed?.[opt.value] ? "" : ` ${t("semOllamaNotPulled")}`);
|
||||
});
|
||||
const info = document.getElementById("wfm-sem-ollama-info");
|
||||
if (info) info.textContent = ol.reachable && !ol.ready ? t("semOllamaPullHint", ol.tag) : "";
|
||||
}
|
||||
|
||||
/** 選択中モデルの次元選択肢を作り直す(preferredが選択肢にあればそれを選ぶ) */
|
||||
function fillDims(model, preferred) {
|
||||
const sel = document.getElementById("wfm-sem-dim");
|
||||
const info = _backends[model];
|
||||
if (!sel || !info) return;
|
||||
sel.innerHTML = info.dims.map(d => `<option value="${d}">${d}</option>`).join("");
|
||||
sel.value = String(info.dims.includes(preferred) ? preferred : info.default_dim);
|
||||
}
|
||||
|
||||
async function refreshModal() {
|
||||
if (!modalAlive()) return;
|
||||
const folder = ctx.getFolder();
|
||||
const recursive = document.getElementById("wfm-sem-recursive")?.checked ?? true;
|
||||
let st;
|
||||
try {
|
||||
const res = await fetch(API.status(folder, recursive));
|
||||
st = await res.json();
|
||||
if (!res.ok) throw new Error(st.error || `HTTP ${res.status}`);
|
||||
} catch (e) {
|
||||
document.getElementById("wfm-sem-status").textContent = t("errorWithMsg", e.message);
|
||||
return;
|
||||
}
|
||||
if (!modalAlive()) return;
|
||||
const job = st.job || {};
|
||||
_backends = st.backends || {};
|
||||
if (st.indexed_total) sem.indexModel = st.index_model;
|
||||
if (!document.getElementById("wfm-sem-dim").dataset.init) {
|
||||
// 初回のみ: 既存インデックスのモデル/次元、なければ前回の選択を初期値にする
|
||||
const initModel = st.indexed_total ? st.index_model : lsGet(LS_MODEL, GEMINI);
|
||||
const radio = document.querySelector(`input[name="wfm-sem-model"][value="${initModel}"]`)
|
||||
|| document.querySelector(`input[name="wfm-sem-model"][value="${GEMINI}"]`);
|
||||
radio.checked = true;
|
||||
fillDims(radio.value, st.indexed_total ? st.index_dim : parseInt(lsGet(LS_DIM, "0"), 10));
|
||||
initLocalOpts(st);
|
||||
document.getElementById("wfm-sem-dim").dataset.init = "1";
|
||||
}
|
||||
fillThresholds(st);
|
||||
const model = selectedModel();
|
||||
document.getElementById("wfm-sem-local-opts").style.display = model === LOCAL ? "" : "none";
|
||||
updateOllamaInfo(st.ollama);
|
||||
const ol = st.ollama || {};
|
||||
const ready = model === LOCAL ? !!ol.ready : !!st.has_key;
|
||||
const keyLine = model !== LOCAL ? (st.has_key ? t("semKeyOk") : t("semKeyMissing"))
|
||||
: !ol.reachable ? t("semOllamaUnreachable", ol.url)
|
||||
: ol.ready ? t("semOllamaReady", ol.tag) : t("semOllamaMissing", ol.tag);
|
||||
const idxLine = t("semIndexInfo", st.indexed_total, st.index_dim || "-");
|
||||
const scopeLine = st.scope_total != null ? t("semScopeInfo", st.scope_indexed, st.scope_total) : "";
|
||||
document.getElementById("wfm-sem-status").innerHTML =
|
||||
`<div class="${ready ? "wfm-sem-ok" : "wfm-sem-ng"}">${escapeHtml(keyLine)}</div>` +
|
||||
`<div>${escapeHtml(st.indexed_total ? st.index_model : "-")} / ${escapeHtml(idxLine)}</div>` +
|
||||
(scopeLine ? `<div>${escapeHtml(scopeLine)}</div>` : "");
|
||||
document.getElementById("wfm-sem-folder").textContent = folder || t("semNoFolder");
|
||||
|
||||
const prog = document.getElementById("wfm-sem-progress");
|
||||
const hasJob = job.running || job.finished;
|
||||
prog.style.display = hasJob ? "" : "none";
|
||||
if (hasJob) {
|
||||
const total = job.total || 0;
|
||||
const doneAll = (job.done || 0) + (job.failed || 0);
|
||||
document.getElementById("wfm-sem-bar-fill").style.width = total ? `${Math.min(100, (doneAll / total) * 100)}%` : "100%";
|
||||
let text = t("semProgress", job.done || 0, total, job.failed || 0);
|
||||
if (job.running && job.current) text += ` ${job.current}`;
|
||||
if (!job.running) {
|
||||
if (["no_api_key", "api_key_invalid", "local_error", "ollama_unreachable", "ollama_model_missing"].includes(job.error_code)) text += ` ${t(`sem_err_${job.error_code}`, job.error)}`;
|
||||
else text += ` ${job.cancelled ? t("semCancelled") : t("semFinished")}`;
|
||||
if (job.failed && job.error) text += ` (${job.error})`;
|
||||
}
|
||||
document.getElementById("wfm-sem-progress-text").textContent = text;
|
||||
}
|
||||
document.getElementById("wfm-sem-start").disabled = !!job.running || !ready;
|
||||
document.getElementById("wfm-sem-cancel").disabled = !job.running;
|
||||
document.getElementById("wfm-sem-clear").disabled = !!job.running;
|
||||
document.getElementById("wfm-sem-test").disabled = !ready;
|
||||
|
||||
clearTimeout(_pollTimer);
|
||||
if (job.running) {
|
||||
_pollTimer = setTimeout(refreshModal, 1000);
|
||||
_sawRunning = true;
|
||||
} else if (_sawRunning) {
|
||||
_sawRunning = false;
|
||||
refreshSemanticDots();
|
||||
}
|
||||
}
|
||||
|
||||
/** ヘルプタブのGallery > セマンティック検索の説明へ飛ぶ */
|
||||
function openSemanticHelp() {
|
||||
closeSemanticModal();
|
||||
document.querySelector('.wfm-tab[data-tab="help"]')?.click();
|
||||
document.querySelector('.wfm-help-nav-item[data-help-page="gallery"]')?.click();
|
||||
const target = document.getElementById("wfm-help-gallery-33");
|
||||
if (!target) return;
|
||||
target.scrollIntoView({ behavior: "smooth", block: "center" });
|
||||
target.classList.add("wfm-help-flash");
|
||||
setTimeout(() => target.classList.remove("wfm-help-flash"), 2500);
|
||||
}
|
||||
|
||||
function openSemanticModal() {
|
||||
if (!ctx.getFolder()) {
|
||||
showToast(t("semNoFolder"), "error");
|
||||
return;
|
||||
}
|
||||
openModal(t("semModalTitle"), modalHtml());
|
||||
const dimOf = () => parseInt(document.getElementById("wfm-sem-dim").value, 10) || 0;
|
||||
// ローカルモデル選択時のみ Ollama の設定を送る(サーバー側で保存され、検索時のクエリ埋め込みにも使われる)
|
||||
const ollamaOpts = () => ({
|
||||
ollama_url: document.getElementById("wfm-sem-ollama-url").value.trim(),
|
||||
ollama_tag: document.getElementById("wfm-sem-ollama-tag").value,
|
||||
});
|
||||
const localOpts = () => selectedModel() === LOCAL ? ollamaOpts() : {};
|
||||
const saveOllamaOpts = async () => {
|
||||
try {
|
||||
await postJson(API.options, ollamaOpts());
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
refreshModal();
|
||||
};
|
||||
document.getElementById("wfm-sem-ollama-url").addEventListener("change", saveOllamaOpts);
|
||||
document.getElementById("wfm-sem-ollama-tag").addEventListener("change", saveOllamaOpts);
|
||||
document.querySelectorAll('input[name="wfm-sem-model"]').forEach(r => r.addEventListener("change", () => {
|
||||
fillDims(selectedModel(), 0);
|
||||
refreshModal();
|
||||
}));
|
||||
|
||||
document.getElementById("wfm-sem-recursive").addEventListener("change", (e) => {
|
||||
lsSet(LS_SUB, e.target.checked ? "1" : "0");
|
||||
refreshModal();
|
||||
});
|
||||
document.getElementById("wfm-sem-close").addEventListener("click", closeSemanticModal);
|
||||
document.querySelectorAll(".wfm-sem-tab-btn").forEach(btn => btn.addEventListener("click", () => {
|
||||
document.querySelectorAll(".wfm-sem-tab-btn").forEach(b => b.classList.toggle("active", b === btn));
|
||||
document.querySelectorAll(".wfm-sem-pane").forEach(p => { p.style.display = p.dataset.semPane === btn.dataset.semTab ? "" : "none"; });
|
||||
lsSet(LS_TAB, btn.dataset.semTab);
|
||||
}));
|
||||
["wfm-sem-th-similar", "wfm-sem-th-text"].forEach(id => document.getElementById(id).addEventListener("change", saveThresholdInputs));
|
||||
document.getElementById("wfm-sem-th-limit").addEventListener("change", saveThresholdInputs);
|
||||
document.getElementById("wfm-sem-help").addEventListener("click", openSemanticHelp);
|
||||
document.getElementById("wfm-sem-start").addEventListener("click", async () => {
|
||||
const mode = document.querySelector('input[name="wfm-sem-mode"]:checked')?.value || "missing";
|
||||
if (mode === "all" && !confirm(t("semConfirmAll"))) return;
|
||||
lsSet(LS_DIM, String(dimOf()));
|
||||
lsSet(LS_MODEL, selectedModel());
|
||||
try {
|
||||
await postJson(API.index, {
|
||||
folder: ctx.getFolder(),
|
||||
recursive: document.getElementById("wfm-sem-recursive").checked,
|
||||
mode,
|
||||
model: selectedModel(),
|
||||
dim: dimOf(),
|
||||
...localOpts(),
|
||||
});
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
refreshModal();
|
||||
});
|
||||
document.getElementById("wfm-sem-cancel").addEventListener("click", async () => {
|
||||
await postJson(API.cancel, {}).catch(() => {});
|
||||
refreshModal();
|
||||
});
|
||||
document.getElementById("wfm-sem-test").addEventListener("click", async () => {
|
||||
try {
|
||||
const r = await postJson(API.test, { model: selectedModel(), dim: dimOf(), ...localOpts() });
|
||||
showToast(r.provider && r.provider !== "API" ? t("semTestOkProvider", r.dimensions, r.provider) : t("semTestOk", r.dimensions), "success");
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
});
|
||||
document.getElementById("wfm-sem-clear").addEventListener("click", async () => {
|
||||
if (!confirm(t("semConfirmClear"))) return;
|
||||
try {
|
||||
await postJson(API.clear, {});
|
||||
showToast(t("semCleared"), "success");
|
||||
refreshSemanticDots();
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
refreshModal();
|
||||
});
|
||||
refreshModal();
|
||||
}
|
||||
|
||||
// ── 初期化 ───────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* @param {object} context
|
||||
* getSearch(): 検索ボックスの文字列 / getFolder(): 選択中フォルダ / getSelected(): 選択中画像
|
||||
* reload(): 一覧再読込(loadImages)
|
||||
*/
|
||||
export function initSemantic(context) {
|
||||
ctx = context;
|
||||
sem.on = lsGet(LS_ON, "0") === "1";
|
||||
updateToggleUi();
|
||||
updateSimilarButtons();
|
||||
|
||||
document.getElementById("wfm-gallery-semantic-toggle")?.addEventListener("click", () => {
|
||||
sem.on = !sem.on;
|
||||
lsSet(LS_ON, sem.on ? "1" : "0");
|
||||
updateToggleUi();
|
||||
// OFFにしたら類似表示も解除する（2行目ごと消えるため）
|
||||
const wasSimilar = !!sem.anchor;
|
||||
if (!sem.on) exitSimilarMode();
|
||||
// 検索文字列があれば、モード切替後の方式で結果を取り直す
|
||||
if (wasSimilar && !sem.on || (ctx.getSearch() || "").trim()) ctx.reload();
|
||||
});
|
||||
fetch(API.status("", true)).then(r => r.json()).then(st => {
|
||||
if (st.indexed_total) sem.indexModel = st.index_model;
|
||||
}).catch(() => { /* 取得できなくても既定のモデル扱いで動作する */ });
|
||||
document.getElementById("wfm-gallery-semantic-settings-btn")?.addEventListener("click", openSemanticModal);
|
||||
document.getElementById("wfm-gallery-similar-btn")?.addEventListener("click", toggleSimilar);
|
||||
document.getElementById("wfm-gallery-similar-refresh-btn")?.addEventListener("click", applySimilar);
|
||||
}
|
||||
+656
-109
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user