Compare commits
@@ -2,6 +2,21 @@
|
||||
|
||||
Condensed version-by-version index of DEVLOG.md (each line: version + one-line summary of what changed and why). Full rationale, code details, and "How to apply" lessons live in the excluded DEVLOG.md itself; this index exists so graphify can extract a queryable semantic node per release without feeding the full log to the local Ollama model. Regenerate with `python tools/generate_doc_index.py devlog` whenever DEVLOG.md changes.
|
||||
|
||||
- **v0.9.9（2026-11-15）** — プロンプト翻訳（自国語 → 英語、PLaMo 3 Translate）
|
||||
- **v0.9.8（2026-11-14）** — セキュリティ: クライアントから受け取るパスを解決前に検証
|
||||
- **v0.9.6（2026-11-12）** — マニュアル v0.9.6版（ja/en）
|
||||
- **v0.9.5（2026-11-11）** — AI TOOL Chat: スキル v2（metadata・同梱スキルの版管理・プロンプトボタン）
|
||||
- **v0.9.4（2026-11-07）** — Gallery: 3Dモデル対応（GLB / glTF / VRM / PLY / SPZ）
|
||||
- **v0.9.3（2026-11-05）** — Image Edit: Drawツールのカラーパレット・ショートカット・スポイト（Comic Creator v1.67.0 から移植）
|
||||
- **v0.9.2（2026-11-03）** — Selectorイベントループ有効時に画像が壊れて届く不具合を修正
|
||||
- **v0.9.1（2026-11-03）** — TagSpace: 選んだタグだけを追加 / タグのカッコをエスケープして追加
|
||||
- **v0.9.0（2026-11-03）** — マニュアル（画像なし配布版）を v0.9.0 に改訂
|
||||
- **v0.8.14（2026-11-03）** — Fibo 2.0 Edit / FastH3 OmniRef / Pika-H3-Flex 対応（GenerateUI・Metadata・Videoタブ）
|
||||
- **v0.8.13（2026-10-27）** — Edit in Mask Editor One: 送信画像をノードへ永続化（Mask Editor One v0.1.12 対応）
|
||||
- **v0.8.12（2026-10-27）** — Comic Creator向け複数画像I2Iブリッジ + GenerateUI Inputタブ: Prompt/Image統合・Raw JSON折りたたみ
|
||||
- **v0.8.11（2026-10-26）** — Gallery: Send CC をレイアウト／Imageタブの2ボタンに分割
|
||||
- **v0.8.10（2026-10-23）** — Tagger: PixAI Tagger v1.0 対応
|
||||
- **v0.8.9（2026-10-20）** — Video: アセット登録ボタン / Editの音声変換 / ソースパネルのレイアウト
|
||||
- **v0.8.8（2026-10-19）** — MOSS-Image 対応（GenerateUI / Gallery / Metadata / Iタブ）
|
||||
- **v0.8.7（2026-10-16）** — Qwen Image 2.1 Turbo（ComfyUI公式テンプレート T2I / I2I）対応
|
||||
- **v0.8.6（2026-10-14）** — GenerateUI: パートナーAPIノード（Nano Banana 等）の実行に対応
|
||||
|
||||
@@ -2,17 +2,375 @@
|
||||
|
||||
---
|
||||
|
||||
## v0.9.9（2026-11-15）
|
||||
|
||||
### プロンプト翻訳（自国語 → 英語、PLaMo 3 Translate）
|
||||
|
||||
- 設定タブに「プロンプト翻訳」セクションを新設（既定オフ）。A1111 の prompt translator 拡張と同じく、英数字以外の文字を含む部分だけを翻訳し、カンマ・`(語句:1.2)`・`[ ]`・`{a|b}`・`<lora:...>`・`__wildcard__`・`embedding:`・`BREAK`/`AND` は保護する（`static/js/prompt-translate.js` の `splitPrompt`）。
|
||||
- 🌐ボタン（＋「原文」ボタン、Ctrl+Z も可）を GenerateUI の Positive/Negative、Lab プロンプトモーダル、Video Plan のブロックプロンプトに追加。
|
||||
- モデルは Ollama 経由。PLaMo は `/api/generate` に `raw:true` で PFML3 プロンプトを直接送る（モデルカード推奨、stop `<|plamo:tag|>`/`<|plamo:bos|>`）。PLaMo 以外の Ollama モデルは `/api/chat` ＋翻訳指示。PLaMo の4サイズはプリセットとして表示し、未取得なら「ダウンロード」（`/api/wfm/translate/pull`、プリセットのみ許可）。
|
||||
- 実行場所: GPU（1語句約0.1秒、翻訳後アンロード可）／CPU常駐（`num_gpu:0`＋`keep_alive:-1`、1語句約0.3秒、VRAM 0、GenerateUI表示時に `/warmup` で裏読み込み、デバイス・モデル切替や無効化時は `/unload`）。実測でコールド約4秒の大半は Ollama ランナー起動（CPUでも4.4秒）だったため、常駐でほぼ解消。
|
||||
- 「実行時に自動で翻訳する」: GenerateUI Generate（欄の文を置き換え）／Lab Run（各回のAPIワークフローのコピーを翻訳）／Video Plan Run（送信するプロンプトだけ翻訳）。Lab/Video はプランを原文のまま残す。失敗時は生成せず中断し、理由を進行状況欄に残す（終了トーストが上書きするため）。
|
||||
- サーバー側に翻訳結果の上限付きキャッシュ（1024件）。設定は `settings.json` の `prompt_translate_*`。
|
||||
- 実機確認: Z-Image で Generate 時自動翻訳と Lab Run（2枚）を実生成、Video Plan Run は `/prompt` 送信をブラウザで横取りして英訳の送信を確認。
|
||||
|
||||
### 修正: PLaMo を選んだときの各翻訳ボタン
|
||||
|
||||
- Prompt タブ AI アシスタント・AI TOOL タブ・サイドパネル A タブの翻訳は、指示文＋本文（または指示の system メッセージ）で送っていたため、PLaMo では指示文まで翻訳される／日本語⇔英語の自動切替になり ZH→EN・EN→ZH・自由入力が誤った言語で出ていた。PLaMo 系モデルのときは先頭 system に `[plamo-translate]` と `target_lang`/`source_lang` を書き本文だけを送る（モデルカードの Ollama/LM Studio 向け方式）。言語名（French・フランス語・français 等）を PLaMo の言語コードに変換する `static/js/plamo-chat.js` を追加（サイドパネルは import できないためインラインコピー）。
|
||||
- Prompt タブ AI アシスタントに応答待ちの表示を追加（初回はモデル読み込みで十数秒かかることがある）。
|
||||
|
||||
### 修正: Video Plan のプロンプト等がテンプレートの値のまま送られていた
|
||||
|
||||
- 新しいフロントエンドで保存したワークフローは、サブグラフのインスタンスノードに名前付きの値のコピー `widgets_values_named` を持ち、`convertUiToApi()` はそちらを優先する。`applyBlockToWorkflow` は `widgets_values` だけを書いていたため、Plan のプロンプト・Duration・Turbo 等が無視されていた（手元の動画テンプレート8本中7本）。書き込み後に `widgets_values_named` へ反映する `_syncNamedWidgetValues` を追加。修正後は12本中11本で反映（残る `minimaax_h3_fast.json` はプロンプトを LiveChatStream から配線で受けるため対象外）。
|
||||
|
||||
### ヘルプ
|
||||
|
||||
- 設定タブ（helpSettings26）と GenerateUI（helpGen33）にプロンプト翻訳を追加（en/ja/zh）。
|
||||
|
||||
### マニュアル v0.9.9版（ja/en）
|
||||
|
||||
- 新節 5-10「日本語のままプロンプトを書く（プロンプト翻訳）」（準備・モデルの大きさ・GPU/CPU常駐・🌐と原文・実行時の自動翻訳・Lab/Video Plan）。7-5・10-2 に PLaMo の注記、11-4 に Video Plan の🌐と v0.9.9 修正の注意、11-5 の表・12-3 のトラブル表・付録の用語集と参考リンクを追記。対象バージョンを v0.9.9 に更新し、全章MD・ZIPを `manual/rebuild_repo_edition.py` で再生成（v0.9.6版は削除）。
|
||||
- EPUB（画像付き、75枚＝新規3枚: 設定のプロンプト翻訳・生成UIで翻訳後・Labモーダル）を Google ドライブで公開。
|
||||
|
||||
## v0.9.8（2026-11-14）
|
||||
|
||||
### セキュリティ: クライアントから受け取るパスを解決前に検証
|
||||
|
||||
- `py/config.py` に `is_safe_rel` / `resolve_untrusted` を追加。文字列だけで正規化・配下チェックし、UNC・ドライブ相対・NUL 混入を最初の `resolve()`/`exists()` より前に拒否（Windows では UNC を解決した時点でアカウント資格情報付きで相手ホストへ接続するため）。UNC は許可ルート自体が UNC の場合のみ許可。G'MIC 結果・Gallery のパス受け取りに適用（ea44cf4）。
|
||||
- 追加の強化: 受け付ける形をドライブレター絶対パス（Windows）または許可された UNC に限定し、NT デバイス形式（`\??\...`、`\.\...`）等を拒否。Eagle の SVG パスで配下チェック前に `os.path.isfile` していた箇所を修正（f39227a）。
|
||||
- ※v0.9.8 はリリース時に README バッジと DEVLOG の更新が漏れていたため、v0.9.9 で追記。
|
||||
|
||||
## v0.9.6（2026-11-12）
|
||||
|
||||
### マニュアル v0.9.6版（ja/en）
|
||||
|
||||
- 5-4 Settingsタブを3列（スイッチ列の説明を追加）、5-9 対応表に MiniMax H3 Image と Compare Images 修正の注記、12-3 の Apply 注意にスイッチ列の例外を追記。対象バージョンを v0.9.6 に更新し、全章MD・ZIPを `manual/rebuild_repo_edition.py` で再生成（v0.9.5版は削除）。
|
||||
|
||||
### GenerateUI Settingsタブ: スイッチ列
|
||||
|
||||
- KSampler・Latent Imageの右に3列目「スイッチ」を追加。APIワークフロー内のリテラルな true/false 入力（ComfySwitchNode の switch、PrimitiveBoolean の value、TextGenerateV2 の thinking など）をノード名・入力名付きで一覧し、チェックボックスで即時切り替え（Apply不要、Raw JSONへ同期）。リンクされた入力は配列なので対象外で、配線元のPrimitiveBooleanが出る。
|
||||
- 選択肢がちょうど enable/disable・enabled/disabled・on/off・true/false・yes/no の2択のCOMBO（KSamplerAdvanced add_noise、DWPreprocessor detect_* など）も対象。object_info を型別取得してから追加描画し、値はそのノード自身の綴りの文字列で書き戻す。DynamicCombo（TextGenerate の sampling_mode）は子入力が変わるため対象外。
|
||||
- 保存済みワークフロー347本（App形式5本は読込対象外）をブラウザで順に読み込み、128本・426項目でオン→オフ→元に戻すの往復と Raw JSON 同期を自動検証、失敗・コンソールエラー0。
|
||||
- MiniMax H3 Image のT2I（サブグラフ内のプロンプト強化スイッチ）とEdit（PrimitiveBoolean「Enable PE?」）でブラウザ実操作を確認。
|
||||
|
||||
### MiniMax-H3-Image対応
|
||||
|
||||
- `MiniMaxH3ImageConditioning`（Comfy-Org の MiniMax H3 Image T2I/Edit テンプレート）をGenerateUI/Metadata/Iタブに追加。入力は `prompt`/`width`/`height`/`images.image_N` で `PH3ImageConditioning` と同形のため同じ扱い（negativeなし、promptはPreviewAny→プロンプト強化スイッチ経由で配線元のユーザープロンプトをpositiveに）。width/height=0は image_1 のサイズ、画像なしなら2048。
|
||||
- 修正前はプロンプト欄が空で、Metadataタブもプロンプトを読めなかった。T2I（プロンプト強化オフ、2048x2048）とEdit（プロンプト強化オン、2720x1536）で実生成確認。
|
||||
|
||||
### 修正: ImageCompareノードを含むワークフローで検証エラー
|
||||
|
||||
- `ImageCompare` の `compare_view`（表示専用の socketless ウィジェット、`widgets_values` に保存されない）がAPI変換で欠落し、`required_input_missing` になっていた。フロントエンドと同じ `{"__value__": ["", ""]}` を補う（`_DISPLAY_WIDGET_DEFAULTS`）。MiniMax H3 Image Edit テンプレートで発覚。
|
||||
|
||||
## v0.9.5（2026-11-11）
|
||||
|
||||
### AI TOOL Chat: スキル v2（metadata・同梱スキルの版管理・プロンプトボタン）
|
||||
|
||||
- スキルの frontmatter に `metadata:` を追加（`output` prompt/svg の自動チェック＋1回だけ再依頼、`tools`、`temperature`、`think`、`context`: workflow / current-prompt / workflow-detail / logs）。`name`/`description` は Agent Skills（SKILL.md）と互換のまま。
|
||||
- 同梱スキルを `py/services/builtin_skills/*.md` に分離し `wfs-version` で版管理。`ai_skills/.builtin_state.json` に版とハッシュを記録し、未編集なら自動更新、編集済みなら残して ↻ ボタンを表示。v0.3.79 版の未編集コピーはハッシュで判別して更新。
|
||||
- 同梱スキル6本: T2I Prompt Enhancer / Image to Prompt / Storyboard to Lab / Workflow Doctor（新規）、SVG Icon Generator v2、Skill Creator v3。
|
||||
- ```prompt ブロックに「GenerateUIに反映」「このプロンプトで生成」「コピー」。応答は依存なしの Markdown 表示（先に全体をエスケープ）。
|
||||
- ワークフロー要約から「今のプロンプト」を既定で外した。「参考のみ・写すな」と書いても gemma4 e4b は 6/6 回で古いネガティブ（pink wear）を新しいプロンプトへ混ぜたため。発言が今のプロンプトに触れたときと `context: [current-prompt]` のときだけ渡す。
|
||||
|
||||
### AI TOOL Chat: 生成画像のチェックと修正ループ（意思決定モデル）
|
||||
|
||||
- 新規 `static/js/chat-image-check.js`。チャットモデルが依頼を3〜8個の yes/no 項目にし（JSON Schema）、Ollama の画像対応意思決定モデル（既定 d3:4b）が確率で判定。✔≥0.85 / ? 0.3〜0.85 / ✘<0.3。
|
||||
- 実測: 生成VLM（gemma4 uncensored e4b）は「剣を持っている」を誤って合格（7/8）。d3:4b・clef-flash・clef は引っかけ含む18/18正解。属性の取り違え（帽子が女の子なのに「男の子が帽子」0.70〜0.79）があるため合格ラインを0.85にし、`expect: no` の逆向き確認項目を加えた。
|
||||
- 「足りない点を直して再生成」で未達項目だけを伝えて書き直し・再生成。自動修正0〜3回。SVG も描画して部位ごとにチェック。
|
||||
|
||||
### AI TOOL Chat: Workflow Doctor / Storyboard to Lab / フォルダ形式スキル
|
||||
|
||||
- 新規 `static/js/chat-context.js`: 未インストールのノード、ComfyUI の一覧に無いファイル・選択肢（近い名前の候補）、切れたリンク、未接続のコア型必須入力、`/history` の直近エラー、ComfyUI ログの最新エラー部分を文脈に入れる。手元77ワークフローで誤検出0（ウィジェット系の必須入力チェックは1/3で誤検出したため除外）。
|
||||
- ```labplan → Lab プラン（1シーン=1イテレーション、T2I、seed 固定）。`lab-tab.js` に `openLabPlanData()`。
|
||||
- スキルの zip 取り込み（SKILL.md 必須、テキストのみ、スクリプトは展開しない、zip-slip とサイズの検査）。参照ファイルは `read_skill_file` ツールで必要時に読む。
|
||||
- Lab: KSampler 列が seed / sampler / scheduler だけ指定のとき「ワークフローの設定のまま」と表示していた不具合を修正。
|
||||
|
||||
### AI TOOL: OCR・ペインの折りたたみ
|
||||
|
||||
- 新規 `static/js/ocr-tools.js`（Comic Creator の OCR タブから汎用部分を移植）。実測（日英混在・縦書きあり）: LightOnOCR-3 4b 1.1秒（位置付き7.9秒）100%、PaddleOCR-VL-manga 3.9秒100%、unlimited-ocr 14秒100%、PaddleOCR-VL 1.6 / GLM-OCR は縦書きを読み落とし94%。
|
||||
- 位置の表示（LightOnOCR / DeepSeek-OCR）、枠クリックで行選択、「翻訳へ送る」「Chatへ送る」。Translation / Settings ペインを細い帯にたためる。
|
||||
|
||||
### 修正: 非推奨の import（GitHub issue #3）
|
||||
|
||||
- `top_menu_extension.js` の `/scripts/ui/components/button.js`・`buttonGroup.js` の import は、a6f45ef で actionBarButtons に移行して以来呼ばれていない旧メニュー用コードだった。`window.comfyAPI` への置き換えではなく削除。`web/comfyui/*.bak` も削除。
|
||||
|
||||
### ドキュメント
|
||||
|
||||
- ヘルプ（AI TOOL を12カードに再編、Lab に Storyboard）、マニュアル v0.9.5（v0.9.2〜v0.9.4 の未反映分も追記）、README の TagSpace スクリーンショット、Registry アイコン用に `docs/thumb.png` を 800x400 に縮小。
|
||||
|
||||
**検証**: 各機能を ComfyUI_5（:8189）で実機確認（Chrome / Playwright）。単体テスト: スキルサービス35件、JS（解析・チェック・Markdown・OCR・文脈）約50件。
|
||||
|
||||
**未確認**: サイドパネル（A タブ）はスキル本文のみ対応（metadata・ボタン・チェックは SPA のみ）。判定の閾値は画像約5枚から決めた値。OpenAI 互換バックエンドでの `read_skill_file` 往復。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.4（2026-11-07）
|
||||
|
||||
### Gallery: 3Dモデル対応（GLB / glTF / VRM / PLY / SPZ）
|
||||
|
||||
- 新規 `static/js/gallery-3d-viewer.js`: three.jsのビューア。詳細パネル・ライトボックス・比較で開き、枠がDOMから外れると自分で破棄する（WebGLコンテキストを解放）。
|
||||
- 新規 `static/lib/three-viewer/`: three 0.187 / @pixiv/three-vrm 3.5.5 / @sparkjsdev/spark 2.4.0 のバンドル。Spark(2.8MB)は別チャンクで、スプラットを開くときだけ読む。再ビルド手順は同フォルダのREADME.txt。
|
||||
- サムネイル: サーバーに3Dレンダラーが無いため、`/thumb` の404を合図にブラウザが描画し `/wfm/gallery/image/thumb3d` へ保存する。1件ずつ処理し、128MB超は開いたときに保存する。
|
||||
- `.gltf` の外部ファイルは `/wfm/gallery/model3d/asset` で配信する（.bin/画像のみ、許可ルート内のみ）。
|
||||
- PLYは `f_dc_0`/`scale_0` の有無でスプラットを判定する。`double` 属性はFloat64ArrayになりWebGLが拒否するのでFloat32へ変換する。
|
||||
- `asset.extras` からSaveGLBのprompt/workflowを読む（GLBはJSONチャンクのみ読む）。
|
||||
- 3DファイルはImage Edit / GenerateUI / Send to LI node / Tagger / Feederへ送れないようにした。
|
||||
|
||||
### P-H3 Image対応
|
||||
|
||||
- `PH3ImageConditioning` をGenerateUI/Metadata/Iタブに追加（`TextEncodeFibo2` と同じ扱い、negativeなし）。`_unifyPromptEnhancerText` は `TextGenerateV2` も対象にした。
|
||||
|
||||
### 修正: TextGenerate系の値ずれ
|
||||
|
||||
- `_normalizeLinkedWidgetValues` が、リンク済みpromptの空枠を持つ形式を「枠なし形式」と誤判定していた。値が1つずれても、無効なDynamicComboキーがoptions[0]へフォールバックして枠数が偶然一致するため。Krea-2 / Ernie / LTX-2.5 などでも `sampling_mode:2048` のようにずれていた。選択キーの有効性も判定に加えた。
|
||||
|
||||
**検証**: 3Dは実サンプルで描画・自動サムネイル・詳細・拡大・破棄・送信ガード（Image Editボタン）を確認（SPZとスプラットPLYは自作サンプル）。パストラバーサルは全て404。変換は手元の640ワークフローを修正前後で比較し、差分21件は全てTextGenerate系。うち5種類は `widgets_values_named` 等で正しい値と確認した。P-H3は編集（1024x1024、約60秒）とT2I（3360x2496、約40秒）をGenerateUIから実生成（RTX 4070 12GB、モデルロード済み）。
|
||||
|
||||
**未確認**: 比較表示、テーブル表示のサムネイル、128MB超のファイル、実データのスプラット、サイドパネルIタブのP-H3、Draco/Meshopt圧縮のモデル。
|
||||
|
||||
**補足**: KTX2圧縮テクスチャは未対応。`.gltf` を移動・削除しても外部ファイルは追従しない。GenerateUIにプロンプト強化のオン/オフ切り替えは無い。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.3（2026-11-05）
|
||||
|
||||
### Image Edit: Drawツールのカラーパレット・ショートカット・スポイト（Comic Creator v1.67.0 から移植）
|
||||
|
||||
- 新規 `static/js/image-edit/DrawPalette.js`: Drawツールの2段目（`ie-tool-options`）に色1/色2、基本色20色、カスタム色20枠を表示する。右クリックで現在の色の登録・色の編集・削除（カスタム色）・初期の色に戻す（変更済みの基本色）。保存先は localStorage `ie_draw_palette_v1`。
|
||||
- `image-edit-tab.js`: 色の変更を `_setDrawColor` に集約し、モードは `_setDrawMode`、サイズは `_setDrawBrushSize` / `_stepDrawBrushSize` に分離。
|
||||
- ショートカット（Draw中のみ）: `D` Draw / `E` Erase / `X` 色1・色2切替 / `[` `]` サイズ増減（1〜10pxは1、10〜50pxは5、50〜200pxは10刻み）/ `B` ブラシ選択の開閉。ブラシ選択の表示中は D/E/X/[/] を無効にし、Escで閉じる。他のツールからのBはこれまでどおりDrawへの切り替え。
|
||||
- スポイト: WFSのDrawツールには無かったためCC版から追加。右パネルのColorの隣のボタンからキャンバスの色を拾う。Esc・再クリック・ツール切替・キャンバス外クリック・Closeで解除。
|
||||
- ブラシの初期色を赤から黒に変更（色1/色2 = 黒/白）。
|
||||
|
||||
**CC版との違い**
|
||||
|
||||
- 色の引き継ぎは、Drawツールの表示時ではなく `_initCanvases` で `DrawTool` を作り直す時点で行う。
|
||||
- スポイトは `ie-canvas-draw` から色を読むため、アクティブレイヤーのみの表示なら合成し直してから読む。
|
||||
- 右クリックメニューを開いたままツールを切り替えても残らないよう、`_renderToolOptions` で閉じる。
|
||||
|
||||
**検証（Playwright、ComfyUI :8189）**: パレットからの描画色、登録と再読み込み後の保持、重複登録の通知、右クリックメニュー、基本色の復元、Color入力との連動、各ショートカット、入力欄やCtrl/Alt同時押しで反応しないこと、サイズの刻み、ブラシ選択の開閉、ブラシ未登録時の通知、Newでの色の引き継ぎ、スポイトの取得と解除、Undo履歴が増えないこと、別レイヤーの色の取得を確認。コンソールエラーなし。
|
||||
|
||||
**未確認**: 「色を編集…」で開くブラウザ標準のカラーピッカーの表示と、日本語キーボードでの `[` `]` の実打鍵。
|
||||
|
||||
**補足**: 透明部分をスポイトすると黒を拾う（CCと同じ）。描いた線から拾うと値が1ずれることがある。READMEのバージョンバッジが0.8.11のままだったので0.9.3に更新した。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.2（2026-11-03）
|
||||
|
||||
### Selectorイベントループ有効時に画像が壊れて届く不具合を修正
|
||||
|
||||
- 症状: 設定「Selectorイベントループ」（`windows_selector_event_loop`）が有効だと、ギャラリーのプレビューと Download ボタンで取得した画像が壊れる（元ファイルと同サイズのまま、32KB 分のブロックが直後のブロックの内容に置き換わる）。出力フォルダのファイル自体は正常。
|
||||
- 原因: Selector 方式では aiohttp の `FileResponse` が asyncio の `_sendfile_fallback` を通る。これは 16KB のバッファを使い回しながら、送り切れなかった分をトランスポートにコピーせず積むため（Python 3.12）、送信が詰まると次の読み込みで上書きされる。ギャラリーは多数の画像を同時に読むので詰まりやすい。
|
||||
- 修正: `prestartup_script.py` で Selector に切り替えるとき `AIOHTTP_NOSENDFILE=1` も設定し、aiohttp がチャンクごとに新しい bytes を読んで送る経路を使うようにした（aiohttp 読み込み済みの場合はモジュール定数も上書き）。プロセス全体に効くので ComfyUI 本体の `/view` なども対象。
|
||||
- 検証: 受信を遅くしたクライアントで、修正前は 5/5 回破損、修正後は 5/5 回ハッシュ一致。再起動後の実サーバー（ギャラリー配信 URL）でも 5/5 回一致。
|
||||
- 壊れた応答は `Cache-Control: max-age=3600` でブラウザに残ることがあるので、更新後は再読み込み（Ctrl+F5）が必要。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.1（2026-11-03）
|
||||
|
||||
### TagSpace: 選んだタグだけを追加 / タグのカッコをエスケープして追加
|
||||
|
||||
- 選択中パネルでタグを1つ以上選んでいると、「追加」ボタンが「選択したタグを追加（N）」になり、選んだタグだけ（プロンプト中の順）を1つのチップ（名前は「rapunzel（2タグ）」）として追加する。選んでいなければ従来どおり全体（カードはプロンプト全文、Gallery点は全タグ）。3D画面のダブルクリックと一覧の＋は常に全体。
|
||||
- 不具合修正: タグは分解時にエスケープを外している（`rapunzel \(disney\)` → `rapunzel (disney)`）ため、タグ・Gallery点のチップをそのままプロンプトに入れると `(disney)` が強調指定になっていた。プロンプトに入れるときにカッコを `\(` `\)` に戻す（表示はそのまま）。カード全文の `\(...\)` と同じ形になるので、重複除去でも1つにまとまる。
|
||||
- 検証（Playwright）: rapunzel カードで未選択の「追加」→全文、2つ選んで「選択したタグを追加（2）」→ `rapunzel \(disney\), braided ponytail`、一覧の＋で同じタグを追加しても重複せず。
|
||||
- ヘルプ（tagspace-10 英/日/中）、README、マニュアル第11章 11-7（日英、全章まとめ・ZIP 再作成）。
|
||||
|
||||
### TagSpace: Gallery点から「Galleryで開く」
|
||||
|
||||
- 選択中パネル（Gallery点）に「Galleryで開く」ボタン。`gallery-tab.js` に `revealInGallery(path)` を新設（export）: ギャラリータブ／Output サブタブへ切り替え（初回はここで出力フォルダとツリーを読み込むので `state.folderTree` を待つ）→ 絞り込み（検索・お気に入り・タグ・グループ・類似表示）を「Clear filters」と同じ要領で解除 → ツリーを根から目的のフォルダまで矢印クリックで展開（子は展開時に描画される遅延描画のため）→ フォルダを選んで `await loadImages()` → 一覧で `[data-path]` の要素をクリック（通常クリックと同じ選択・詳細表示）して中央へスクロール。
|
||||
- サムネイル表示は50件ずつ描くので、目的の画像まで `_appendNextPage` で描き足し、無限スクロールの目印を末尾へ戻す（全件描いたら外す）。パスは `\` / 大文字小文字の違いを無視して比較。出力フォルダ外・一覧に無い画像は false を返し、TagSpace 側でトーストを出す。
|
||||
- 検証（Playwright）: 検索語を入れた状態から TagSpace の Gallery 点で押す → 0.3秒でギャラリーに切り替わり、ツリー「2026-06-07」選択・一覧で目的の画像が選択されて画面内・検索欄クリア・詳細表示。456枚のフォルダの453枚目でも 0.6秒で表示（全件描画・目印も解除）。出力フォルダ外／存在しないパスは false。コンソールエラーなし。
|
||||
- ヘルプ（tagspace-10 英/日/中）、README、マニュアル第11章 11-7（日英、全章まとめ・ZIP 再作成）。
|
||||
|
||||
---
|
||||
|
||||
## v0.9.0（2026-11-03）
|
||||
|
||||
### マニュアル（画像なし配布版）を v0.9.0 に改訂
|
||||
|
||||
- 日本語・英語の `docs/manual/` を v0.8.9 → v0.9.0 に更新（全章まとめ・ZIP・README を作り直し、v0.8.9 のファイルは削除）。
|
||||
- 反映した変更: TagSpace タブ（第11章に 11-7 を新設、第3章のタブ一覧・付録の用語集・第12章のトラブル表）、Gallery セマンティック検索の Ollama 化（第8章 8-6・第2章・第12章。ONNX の GPU 設定の記述を Tagger 側へ移動）、Tagger の PixAI 対応と GPU 確認（第11章 11-2・第12章）、GenerateUI の Input タブ横並び・Raw JSON 折りたたみ（第5章）、Fibo 2.0 Edit／FastH3 OmniRef／Pika-H3-Flex（第5章 5-9）、Video Plan の Reference Image（第11章 11-4）、Comic Creator からの複数参照画像の実行（第11章 11-6）。
|
||||
- 前回の全章まとめは v0.8.11 の章ファイル修正（Send CC）を取り込んでいなかった。今回は章ファイルから全章まとめを作り直したので解消。
|
||||
- 英語版の Raw JSON 折りたたみボタンは、実際の表示（»）ではなく ASCII の `>>` と書いた。
|
||||
|
||||
### TagSpace タブ新設（latenttag-3d の WFS 統合）
|
||||
|
||||
- **目的**: latenttag-3d（単独アプリは別途継続）を WFS に組み込み、WFS のデータだけでタグを視覚的に選んでプロンプトを作る。
|
||||
- **材料**: カード＝ImagePrompt（`ws_image_prompt`、`.txt` または ponyxlWildcardsVault の YAML から `_read_sidecar_prompt` でプロンプト解決、プロンプトのある951枚）／タグ＝ImagePrompt・ワイルドカード（`.txt` 行・YAML の `- ` 行）・Tagger DB・`prompts.json` の文を `, \n | { } BREAK` で分解し正規化（重み・括弧・Dynamic Prompts の `n::`・LoRA・`__wildcard__`・`rating:` を除去、`_`→空白、2〜40文字・5語以内）。出典の種類数→出現回数の順で上位 N（既定2000）／背景＝Gallery 索引から seed 固定で間引いた画像（既定4000）。
|
||||
- **ベクトル**: カードは Gallery セマンティック索引の画像ベクトルを流用（未索引分は構築時に `SemanticService.start_index(..., "missing")` で追加）。タグは Ollama でテキスト埋め込み（`task: search result | query:` 付き）し `tagspace_tags.npz` にキャッシュ。索引が Gemini のときはエラーにする。
|
||||
- **modality gap**: 同じ空間でも画像とテキストは別の塊になる。実測で、そのまま PCA すると第1・第2軸が画像/テキストの分離に使われた（|Δ平均|/標準偏差 2.99・4.43）。画像とテキストを別々の平均で中心化すると 0 になり、画像→近いタグも改善（例 `alley`: 中心化前 anime, twintails… → 後 street, neon lights, city lights, night）。
|
||||
- 画像側の平均を「カード＋背景」で取ると、数の多い背景に引っ張られてカードがタグの塊から離れた（中心間距離 5.06）。カードの平均に変えて 0.00 にした。
|
||||
- 近傍・検索も同じ中心化コサイン。同種どうしのスコアが高く出るため、カードとタグを別々に順位付けして交互に並べる。
|
||||
- **3D**: three.js r187（MIT）から使う部品＋OrbitControls だけを esbuild で束ねた `static/lib/three/three-tagspace.min.js`（559KB）を同梱し、タブを初めて開いたときに動的 import。カードは Sprite（サムネイルを6並列で順次読み込み）、タグ・背景は Points、選択/近傍/検索は重ね描き、ラベルは HTML オーバーレイ。タブが非表示の間は描画ループを止める。
|
||||
- **API**: `/wfm/tagspace/{status,space,build,cancel,search,neighbors}`（`tagspace_routes.py`。Gallery と同じ GalleryService/SemanticService を共有）。生成物は `tagspace_space.json`（表示用）・`tagspace_space.npz`（近傍/検索用）・`tagspace_tags.npz`（タグのベクトル）。
|
||||
- **検証**（ComfyUI_5 / 8189）: 初回構築 約2.5分（カード画像1,054枚の索引追加＋タグ2,000件）、キャッシュありの再構築 7秒。英語・日本語の検索、キャンバスのホバー/クリック（タグ・カード）、チップのトグル、重複除去、GenerateUI の Positive への追記（`wfs_sdxl_gallery.json`）、ワークフロー未読込時のトーストを確認。コンソールエラー0。
|
||||
- 注意: Git Bash の curl で日本語の JSON を送ると文字化けして空の検索になる（API 側の問題ではない）。ヘルプ（TagSpace ページ 英/日/中）と README を追加。マニュアルは未更新。
|
||||
- **追加: Gallery 画像のプロンプト由来タグ**（Tagger による補完は次の段階で検討）
|
||||
- 表示自体にタグは不要（背景点は画像ベクトルで配置）。目的は、自分の生成画像の語をタグの材料にすることと、Gallery 点をクリックしてその画像のタグを入れること。
|
||||
- 抽出: 埋め込みの API 形式グラフで、サンプラー/ガイダーの `positive`（無ければ Guider 系の `conditioning`）から上流をたどり、`text` / `prompt` / `clip_l` 等の文字列入力を集める。`negative` 系の入力はたどらない（ControlNet 等の両方を持つノード経由で negative が混ざるのを防ぐ）。UI 形式しか無い画像は対象外。
|
||||
- 実測: 2.9万枚中 64%（18,723枚）で取得、初回 約2〜3分・キャッシュ（`tagspace_prompts.json`、size/mtime 一致）で 0.7秒。典型的なネガティブ語が3つ以上混ざるのは5件で、いずれも利用者がプロンプト欄に画面テキストを貼っていたもの。区切りはカンマ等のみ（スペース区切りは「blue eyes」が分かれるため不採用）。
|
||||
- **出典の偏り**: タグを「出典の種類数→出現数」で選ぶと、Gallery が上位2000件中1982件を占めた。出典ごとに「そのタグを含む文の数」で並べ、出典を順番に1件ずつ取る方式に変更（Gallery 682・ImagePrompt 639・ワイルドカード 476・Tagger DB 199・Prompt ライブラリ 4）。
|
||||
- 背景点に `tags` を持たせ、クリックでまとめて1チップ（`kind: image`）。プロンプトが無い画像（背景4000中 約41%）は追加不可で、詳細に「タグなし」と表示。
|
||||
- バグ修正: 背景の点（`userData.items`）が `kind` を付ける前の生データだったため、ホバー/クリックで種類と名前が undefined になっていた。同じオブジェクトに `kind` を付けるよう修正。
|
||||
- バグ修正: セマンティック索引に削除済み画像が残っており、背景のサムネイルが 404 になった。背景は多めに取って実在するファイルだけに絞り、プロンプト読み取りでも削除済みは除外。
|
||||
- 既知: 画像編集モデルの指示文（例 `do not use the colors`）も5語以内ならタグとして残る。
|
||||
- 検証（ComfyUI_5 再起動後）: 構築148秒（プロンプト読み取り111秒）→キャッシュ有りで15〜16秒。背景4000点で存在しないファイル0件・サムネイル404は40件抽出で0件。Gallery 点のクリック（タグありは1チップで追加、タグなしは追加不可）と、ホバー331回でコンソールエラー0を確認。
|
||||
- **追加: Tagger 補完**（プロンプトを取り出せない Gallery 画像、2.9万枚中 10,652枚）
|
||||
- ツールバー「Tagger補完: [OFF/モデル] [Gallery点のみ/プロンプトの無い全画像]」。モデルは Tagger タブの ONNX モデル（`.onnx` の無い `wd-vit-tagger-v3` は出ない）。`TaggerService.tag_image()` を追加し、しきい値は既定値（WD 0.35/0.85、PixAI は推奨値 0.17/0.27）、評価ラベル（general/sensitive/questionable/explicit）は除外。
|
||||
- 背景点の選択をタグ集めより前に移し、「Gallery点のみ」はその中のプロンプト由来タグが無い画像だけを処理。結果は `tagspace_tagger.json`（path→size/mtime/model/tags）にキャッシュし、100件ごとと終了時に保存（中止しても残る）。補完のために読み込んだモデルは終了後に Tagger のキャッシュから外す（Tagger タブで読み込み済みなら残す）。
|
||||
- 新出典 `tagger_auto`（「Tagger補完」、黄緑）。背景点は `tag_src: prompt|tagger` を持ち、Tagger 由来はツールチップ/詳細に「（Tagger補完）」。進み具合に実行先（CUDA/CPU）を表示。
|
||||
- **環境の問題（再発）**: ComfyUI_5 の venv で `onnxruntime` 1.31.0（CPU）と `onnxruntime-gpu` 1.30.0 が共存し CUDA が無効。CPU では wd-v1-4-convnext 465ms/枚・wd-v1-4-vit 641ms/枚（全画像なら約1.5時間）。修復は ComfyUI 停止→`fix_onnxruntime_gpu.bat`（外側リポジトリ、v0.8.10 と同じ）。今回は未実施。
|
||||
- 検証（CPU）: Gallery点600・Gallery点のみ・convnext で 288枚を142秒（0.49秒/枚）、全600点にタグ（プロンプト312・Tagger 288・なし0）。`tagger_auto` を含むタグ571件（うち補完でしか出ないもの180件: 1boy, multiple girls, scenery 等）。キャッシュで再構築17秒。UI（選択欄の有効/無効、凡例、「（Tagger補完）」表示、クリックで1チップ）とコンソールエラー0を確認。確認後は Gallery点4000・補完OFFで作り直し済み。
|
||||
- GPU 修復後（利用者が `fix_onnxruntime_gpu.bat` を実行、`onnxruntime-gpu` 1.30.0 のみ・CUDA/TensorRT 有効）: Gallery点4000・Gallery点のみ・convnext で 1,918枚を約136秒（約0.07秒/枚、画像読み込み込み。CPU の約7倍）、全4000点にタグ（プロンプト2,082・Tagger 1,918）。全画像（10,652枚）なら約12〜13分の見込み。
|
||||
- **追加: Gallery 点の近傍**
|
||||
- 以前は Gallery 点を選んでも `select()` が近傍を問い合わせずに抜け、直前の一覧が残っていた（ファイル名とは無関係）。近傍用 `tagspace_space.npz` にも背景のベクトルが無かった。
|
||||
- 背景のベクトル（`bg_ids` / `bg_vecs`、4000×768 で約12MB）を保存し、カード/タグ/画像の中心化行列を読み込み時にキャッシュ。Gallery 点を基準にしたときだけ「画像→タグ→カード」を交互に並べる。カード・タグ・検索では画像を混ぜない（利用者の指定）。一覧の Gallery 画像は丸いサムネイル、タグの無い画像は「＋」なし。
|
||||
- 検証: `ComfyUI_00510_.png` → 似た画像（0.69〜0.71）・pink body suits 等のタグ・カードが交互、57ms。タグ/カード/検索の一覧は card・tag のみ。画面でも Tagger 補完の Gallery 点で確認。
|
||||
- 未解明: 再起動直後の最初の構築が146秒かかり `tagspace_tagger.json` が書き換えられた（2回目以降は13秒・書き換えなし、ログに警告なし）。再現したら段階ごとの時間を記録して調べる。
|
||||
- **追加: 表示と操作の改善**
|
||||
- 発光: 放射グラデーションのテクスチャ（明るい芯＋にじみ）を加算合成（`blending: 2`。同梱サブセットに `AdditiveBlending` が無いため数値で指定）、`depthWrite: false`、点サイズ×2.4。背景は夜空色 `#05070d`。Gallery点2万点だと重なりで中心が白く飛んだので、発光時は Gallery点の不透明度を 0.55→0.3。
|
||||
- またたき: `PointsMaterial.onBeforeCompile` で頂点シェーダーに `twinkle`（位相・速さ。添字から決まる疑似乱数で再構築しても同じ）と時刻 uniform を差し込み、`vColor` を 0.15〜1.25 倍（`pow(0.5+0.5 sin, 2)` で暗い時間を長く・ピークを鋭く）。タグ・カード・Gallery点のみで、選択・近傍の強調は瞬かない。検証: 0.9秒で変化した画素 約2.6万（オフ時0）。
|
||||
- 拡大表示: 「選択中」の画像と「近く」の一覧の画像に `data-ts-zoom`。マウスを乗せるとサイド列の左に 360px で表示（読み込み済みの小画像を先に出し 720px に差し替え、`pointer-events: none`）。
|
||||
- 追加/削除ボタンを「選択中」の名前の行（`position: sticky`）へ移動。本文をスクロールしても押せる。
|
||||
- クリックは選択のみ（`e.detail > 1` は無視）、ダブルクリックで追加/削除。「追加」ボタンと一覧の＋は従来どおり。ヒント・空欄の案内・ヘルプ（英/日/中）・README を更新し、ヘルプに表示の項目（tagspace-8）を追加。
|
||||
|
||||
- **追加: ワイルドカード YAML をパーサーで読む**
|
||||
- 従来は「`- ` で始まる行」を1項目として読んでいたため、`- >-` 等のブロックスカラー（本文が次行以降）は `>-` だけになり本文を取りこぼしていた（billions_of_all_in_one.yaml、corn-flakes-doll.yaml）。二重引用符内の `’` もエスケープのまま残っていた。
|
||||
- PyYAML（ComfyUI 本体の依存）の `CSafeLoader`（無ければ `SafeLoader`）で読み、文字列の葉だけを取る（キー＝カテゴリ名、yes/no・数値に変換された葉は除く）。YAML として読めないファイルは従来の行読みにフォールバック。4.2万行で 0.19 秒（純 Python の safe_load は 1.86 秒）。
|
||||
- 実測（ワイルドカード7ファイル）: 異なるタグ 28,861 → 30,130（+1,269。female / wings / dress / lace 等）。他5ファイルは不変。
|
||||
- 実機（ComfyUI_5 再起動後、2000タグ・背景20000で再構築）: ワイルドカード由来 748 → 762、入れ替わり22件（male / benevolent / pointed ears 等が入り、出典を順番に取る方式なので他出典の下位が抜ける）。female の出現数 164 → 208 等。
|
||||
- **追加: 全タグ・最低出現回数・ワイルドカードのオン/オフ**
|
||||
- 背景: 「全部の点を表示したい」。タグの種類は全出典で約4.8万、うち約3.7万がワイルドカードにしか無い語（1回だけの語が大半）。ワイルドカードを除くと約1.1万。
|
||||
- ツールバーに「全タグ」（`max_tags=0`、上限 `ALL_TAGS_LIMIT`=20,000）、「最低出現回数」（1/2/3/5。全出典合計の出現数で候補から外す）、「ワイルドカード」（材料に含めるか）。meta に `max_tags` / `min_count` / `wildcard` / `tags_available` を記録し、一部だけ使ったときは状態表示に「（候補 N）」。既定値（ワイルドカードあり・最低1回）は従来と同じ結果（2000タグの集合・出現数・出典が完全一致を確認）。
|
||||
- **正規化の追加**: 全タグにすると、MOSS-Image のレイアウト指定（`{"bbox":[...],"label":"text"}`）や文字入れ指定の JSON 断片が Gallery プロンプトから大量に入り、`bbox":-8` 等 234件が空間の外に孤立した塊を作った。`"key":` を含む断片とモデルのファイル名（`.safetensors` 等）を除外し、片側だけ残った `"` は取り除く。括弧を外した後の `embedding:` も除外。ワイルドカードなし全タグ 11,406 → 10,435（2回以上 7,383 → 6,843）。
|
||||
- **Ollama の一時ポート枯渇**: 9,056件を連続で埋め込んだところ 6,880件目で `HTTP 400 ... dial tcp 127.0.0.1:NNNNN: connectex: Only one usage of each socket address` で失敗。Ollama はテキストごとにランナーへ新しい接続を張るため、ループバックの TIME_WAIT が 14,228 / 16,384 に達していた。このエラーを一時的扱いにし、30秒待って再試行（最大5回、Windows の TIME_WAIT は約2分で解放）。単体テスト（2回失敗→3回目で回復、失敗が続けば6回で諦める）で確認。実構築では以後枯渇せず（新規5,203件で TIME_WAIT 最大 11,968）、本番経路での再試行は未発生。
|
||||
- 実測（ComfyUI_5、Gallery点20,000・Tagger補完あり）: ワイルドカードなし全タグ 79秒（新規埋め込み 2,656件）、最低2回 22秒、ワイルドカードあり全タグ・最低2回（14,188件）127秒（新規 5,203件）。空間データ 9.4MB（2000タグ）→ 10.8MB（11,406タグ）。ブラウザで約3.2万点を表示し、毎フレームのホバー判定込みで 60fps・JSヒープ 35MB。PCA 寄与率は 10.5% → 7.9%（タグの種類が増えて軸が分散）。
|
||||
- **追加: Gallery点のタグを選んで空間検索**
|
||||
- 選択中パネルで Gallery 画像のプロンプト（またはTagger補完）のタグを、文ではなくタグごとのボタンで表示。クリックで選択/解除（複数可、`state.picked` に画像IDと Set で保持し、別の要素を選ぶとリセット）。「選択したタグで空間検索（N）」で、選んだタグをプロンプト中の順に `, ` でつないだ文を既存の `runSearch()` に渡す（検索欄にも入れる）。
|
||||
- タグが多いとパネル（最大260px）からボタンがはみ出したため、ボタン行を `position: sticky; bottom: 0` で下端に固定。
|
||||
- 検証（Playwright）: 33タグの画像で2つ選択→ボタン有効化→「brown hair, east asian」で検索し、east asian 0.67 / brown hair 0.67 / dark brown hair 0.62 等が並ぶ。検索後も選択を保持。15タグの画像でパネルがスクロールする状態でもボタンが見えることを確認。
|
||||
- **追加: カードのタグも選んで空間検索**
|
||||
- カードはプロンプトを文で持つだけだったので、構築時に `split_tags`（Gallery点と同じ正規化）で `tags` を付ける。詳細パネルの表示処理を `tagPicker()` に共通化し、カードはタグのボタン＋「プロンプト全文」（`<details>`）＋カテゴリ。タグが無いカード・作り直し前の空間（`tags` 無し）は従来どおり全文表示。
|
||||
- 検証（ComfyUI_5 再起動・再構築後）: 951枚中948枚にタグ（平均3.4個）。無いのは `zero two \(darling in the franxx\)` のように1語が5語を超えるもの等3枚。「resentful」カードで angry / evil smile を選んで検索 → evil smile 0.89、evil grin 0.84、arrogant smile 0.64。
|
||||
- **修正: ズーム時に広い範囲で同じ点が反応する（ホバー・クリック）**
|
||||
- 原因: `Raycaster.params.Points.threshold` が3D空間の固定値 0.12 で、しかも結果はカメラに近い順に並ぶのに先頭を採用していた。ズームして点群に入ると、手前の1点が画面の広い範囲で「当たり幅の内側」になり続ける。
|
||||
- 修正（`pick()`）: 候補は広めに取り（注視点の3倍の奥行きで 8px になる幅と、描かれた点の3D上の半径の大きい方）、各点を画面へ投影してカーソルとの距離（px）が最小のものを選ぶ。点はカーソルから 8px 以内、拡大で大きく描かれた点は見た目の半径（発光時は芯）まで。カードのサムネイル（Sprite）は従来どおり当たれば候補。
|
||||
- 検証（Playwright、ホイールで大きくズーム、10×10 の格子でホバー）: 修正前 100か所すべてで反応し2種類（boke 55回）→ 修正後（適度なズーム）73か所で反応し72種類、点の無い場所は無反応。4か所のクリックもそれぞれ別の要素を選択。カメラがカードの中に入るほどズームするとサムネイルが画面全体を覆うので、そのカードが反応するのは正しい。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.14（2026-11-03）
|
||||
|
||||
### Fibo 2.0 Edit / FastH3 OmniRef / Pika-H3-Flex 対応（GenerateUI・Metadata・Videoタブ）
|
||||
|
||||
- **GenerateUI解析**（`comfyui-workflow.js`）: `TextEncodeFibo2` を `TextEncodeQwenImage21` と同じ固定ロール扱いに（prompt は PreviewAny → プロンプト強化スイッチ経由でリンクされるので配線元へ positive を伝播、negative_prompt は VGL JSON のままプロンプト欄に出す、width/height/batch_size を設定タブへ。0 は自動サイズ）。動画オールインワンノードの判定を `VIDEO_ALL_IN_ONE_RE`（`ImageToVideo|TextToVideo|ReferenceToVideo`）に集約し、`MiniMaxH3ReferenceToVideo` の prompt が別ノード（PrimitiveStringMultiline）から配線されていれば配線元を positive にする。
|
||||
- **Metadata / サイドパネルIタブ**: 動画ノードの prompt がリンクなら配線元テキストを解決。Fibo2 は MageFlow/Boogu と同じく1ノードで positive/negative を分けて取り出す。
|
||||
- **Videoタブ（Plan）**: 既存の MiniMax H3 / LTX-2.5 / Wan2.2 はサブグラフの公開入力1ノードに書く方式だが、FastH3 系はサブグラフ無しのフラット構成。新ファミリー `fasth3` として、各入力の**配線元ノード**を書き込み先にした（prompt → PrimitiveStringMultiline、duration → PrimitiveFloat（length が未配線なら自ノードの length へ 24fps・17k+5 グリッドのフレーム数で書く）、解像度 → ResolutionSelector）。First Image / Last Image 欄を **Reference Image / Reference Image 2**（`ref_images.ref_image_0` / `ref_image_1`）として流用し、Asset タブのボタン表記・トーストも切り替える。
|
||||
- **未指定の参照画像は接続を外す**: FastH3 系のみ、None / Clear の参照画像はテンプレートの LoadImage が繋がっていても切断し、使われなくなった LoadImage はノードごと除く（外さないと UI 上は未指定なのにテンプレートの画像で生成される）。他ファミリーは従来通り。
|
||||
- **chain_previous**: Reference Image 2 は「前ブロックの最終フレーム」として扱わず、常に生成された最終フレームを抽出して次ブロックの1枚目の参照画像にする。
|
||||
- **罠**: FastH3 テンプレートの PrimitiveString/Float・ResolutionSelector・LoadImage は `inputs[]` に widget 定義を持たない形式で保存されており、`widgetIndex` が -1 を返して何も書けなかった。ノード型ごとの既定の並び順にフォールバックするようにした。
|
||||
- **ノード側の仕様**: `MiniMaxH3ReferenceToVideo` は `ref_images.values()` を順に使うだけなので、1枚目が欠けて2枚目だけでも動く（その画像が `<Picture 1>` になる）。
|
||||
- **検証**: Node 上で実コード＋ ComfyUI_5（8189）の object_info を使い、3種のワークフローの変換・解析・抽出と、参照画像の差し替え／注入／切断を確認（既存 minimax / ltx25 テンプレートは結果不変）。Playwright で Video タブを実操作（ラベル・既定値・Clear・＋Block・ドロップ・Asset 表記）。Pika-H3-Flex を Plan から実生成（2ブロック×2秒・0.2MP → 352×608・56フレーム・音声付き、約160秒）し、chain_previous でブロック1の最終フレームがブロック2の参照画像になって自然に繋がることを確認。
|
||||
- **未対応**: 参照画像の3枚目以降・参照動画・参照音声はワークフロー側の設定のまま。Pika は ManualSigmas のため GenerateUI の Steps は効かない（linked 表示）。
|
||||
|
||||
|
||||
### Gallery セマンティック検索: EmbeddingGemma 2 を ONNX から Ollama へ移行
|
||||
|
||||
- **背景**: latenttag-3d（Ollama の `embeddinggemma-2` でタグを埋め込む）と Gallery 索引を共通の空間で使いたい。ONNX 版（`semantic_local.py`、q4）は onnxruntime-gpu の入れ替え問題も抱えていた。
|
||||
- **互換性の実測**: 同じ入力で ONNX 版と Ollama 版（Q8_0）を比較。テキスト cos 0.985〜0.993、画像（12枚）平均 0.967（最小 0.923）。既存索引に Ollama の検索文を当てた上位10件の一致は 8〜9/10。→ npz の `model` 名は `embeddinggemma-2` のまま、既存索引（2.9万枚）をそのまま使う。
|
||||
- **740m / 440m**: 740m=テキスト+画像+音声、440m=テキスト+画像。画像・テキストとも cos 1.0（同じベクトル）なので、切り替えても再作成不要。`:latest` は 740m と同じ（parent_model 740m-q8_0）で、740m 未取得時の代わりに使う。270m / 570m は画像非対応のため選択肢に入れない。
|
||||
- **Ollama の画像埋め込みの罠**: `/api/embed` に `images` フィールドを付けても**エラーにならず無視され、テキストだけの埋め込みが返る**。正しくは `input: [{"image": "<base64>"}]`。`[{"images": ...}]` は 400、壊れた画像は 500 "Failed to load image or audio file"、未取得モデルは 404。`input: []` + `keep_alive: 0` でアンロードできる。
|
||||
- **実装**: `semantic_ollama.py` を新設し `semantic_local.py` を削除。画像は4枚ずつ送信（1枚ずつより約1.8倍速い）し、バッチが失敗したら1枚ずつ送り直して原因の画像だけを失敗扱いにする。Ollama 内部の接続切断（400 "forcibly closed" 等）と 5xx は最大5回再試行。接続不可・モデル未取得はジョブ開始前に検出し、作成中に起きたら中止。作成後は VRAM から外す。URL は `url_guard` で検査。設定は `semantic_settings.json`（`url` / `tag`）、保存用に `/wfm/gallery/semantic/options` を追加。
|
||||
- **UI**: モデル設定タブのデバイス/CPUスレッド/画像精度を廃止し、Ollama URL・モデルサイズ（未取得表示つき）に置き換え。ヘルプ（helpGallery33/35/36・helpTrouble10/11 英/日/中）、README、requirements.txt を更新。マニュアル（v0.8.9版）は未更新。
|
||||
- **実機確認**（ComfyUI_5 / 8189、再起動 13:17 > 反映 13:13 をプロセス起動時刻で確認）: 既存索引 29,356件をそのまま読み込み。テキスト検索 0.9秒、類似画像 約0.6秒（4件）。URL ガードで `169.254.169.254` を拒否。モデル設定の 740m⇔440m 切替が保存され状態欄に反映。`ponyxl/scenes/city`（未索引9枚）を「新規・変更のみ」で作成→約1.7秒・失敗0（4+4+1枚のバッチ）、作成後 `/api/ps` が空（アンロード済み）、追加した `alley.jpeg` の類似上位は同フォルダの画像（0.88〜0.93）。コンソールエラーなし。
|
||||
- 確認中に Python の urllib から `/similar` が1回120秒タイムアウト、`/folders` が1回 WinError 10054 になった。いずれも再試行や curl では毎回0.6秒前後で正常で、変更していない `/folders` でも起きたため今回の変更とは別の現象と判断（このPCで以前から出ている接続切断と同種の可能性）。
|
||||
- **残作業**: 旧 ONNX モデル（`user/default/Workflow-Studio/models/embeddinggemma-2`、302MB）は削除可能。`onnxruntime` は Tagger で使うため依存に残す。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.13（2026-10-27）
|
||||
|
||||
### Edit in Mask Editor One: 送信画像をノードへ永続化（Mask Editor One v0.1.12 対応）
|
||||
|
||||
- **背景**: Mask Editor One v0.1.12（PR #4）が、背景画像を `input/mask_editor/` へアップロードして `node.properties.bg_image` に保存し、ワークフロー切り替え／リロード時の `onConfigure` で復元するようになった。一方 `window.wfmOpenMaskEditorForNode`（`web/comfyui/node_sets_menu.js`）は `node._bgDataUrl` / `node._bgImg` を直接差し替えるだけで `properties` を更新しないため、そのままだとリロード時に**以前の画像が復元され、`/mask_editor/store_image` のサーバーキャッシュを上書きする**（WFSが送った画像が消える）。
|
||||
- **修正**: 画像送信後に `node._persistBgDataUrl(imageDataUrl, "wfs_image.png")` を呼んで `properties.bg_image` を同期（Mask Editor One 側が公開する関数。旧バージョンでは `typeof` チェックで無視）。
|
||||
- **layer_data**: 既存マスクを引き継ぐ際の `layer_data` に、送信画像の `width` / `height` を付与。ノードのマスクプレビュー復元はこのサイズでレイヤーを作る（無い場合は復元済みの背景画像サイズ、それも無ければスキップ）。付与順は「`_bgImg` ロード後」に変更。
|
||||
- **How to apply**: 外部から `MaskEditorOne` ノードの画像を差し替えるコードは、`node._bgDataUrl` を書き換えたら必ず `node._persistBgDataUrl()` で参照も同期すること。`applyImageAndMaskToMaskEditorOneNode`（API形式ワークフローへ書き込む経路）はグラフ上のノードを触らないので影響なし。
|
||||
- **検証**: ComfyUI_5（8189）で、画像A保存→WFS相当でB送信→シリアライズ／復元で、背景がB（300×600）・マスクプレビューも300×600になること、`width/height` なしの `layer_data` でも背景サイズで復元されることを確認。
|
||||
- ヘルプ（`helpImageEditMask14` 英/日/中）とREADMEを更新。マニュアル（v0.8.9版）は未更新。
|
||||
|
||||
|
||||
---
|
||||
|
||||
## v0.8.12（2026-10-27）
|
||||
|
||||
### Comic Creator向け複数画像I2Iブリッジ + GenerateUI Inputタブ: Prompt/Image統合・Raw JSON折りたたみ
|
||||
|
||||
- **背景**: Comic Creator v1.65.0 の Nanobanana+ タブ（ローカルモード）が、Flux.3 Image Klein / Qwen Image 2.1 等の Image Edit 系ワークフローを複数の参照画像つきで実行したい、という要望。既存の `_wfmReceiveI2IRunRequest` は画像1枚（slot 0固定）だった。
|
||||
- **複数画像ブリッジ**: `gallery-tab.js` に `window._wfmReceiveMultiImageI2IRunRequest(imageBlobs, params, workflowData, workflowFilename)` を追加（ワークフロー読込→`image-edit-tab.js` の `runI2IExternalMulti` へ委譲）。`InpaintI2IActions.js` の `_runI2IWithImages` が `imageBlobs[i]` を i 番目のLoadImageへ `applyImageToSlot(file, i, ...)` で割り当て（ノード数超過分は無視）。戻り値は `{ ok, url, slotCount, usedImages }`。
|
||||
- **生成パラメータ**: width/height/steps/cfg/denoise/sampler_name/scheduler は既存の `comfyEditor.setGenerationParams` を再利用（指定時のみ書き込み。未指定=ワークフローの値のまま）。シードは `generate(workflow, { seedValue })` で固定。
|
||||
- **How to apply（限界）**: `setGenerationParams` はKSampler系・解析済みlatentノードにしか書けない。出力サイズを入力画像から決める構成（例: Flux.3 Image Klein の `ReferenceMultiLatent`）では `EmptyFlux2LatentImage` が解析対象外になり、width/height は反映されない（Steps/CFG/シード/プロンプトは反映）。
|
||||
- **Inputタブ（GenerateUI）**: Prompt / Image の内部タブを廃止し、同一画面に左右（Prompt:Image = 3:2）で並べた。親カラム（`.wfm-gen-params-col--input`）をCSSコンテナにして、幅が720px以下になるとImageがPromptの下へ折り返す（`@container`）。JSのタブ切替ハンドラは削除（`gallery-tab.js` の `[data-input-tab="image"]` クリックは `?.` のため無害）。
|
||||
- **Raw JSONの折りたたみ（Inputのみ）**: 右カラムの上端に »/« ボタン。折りたたむと幅30pxの縦書き「Raw JSON」の帯だけが右端に残り、Prompt/Imageが広がる。状態は `localStorage`（`wfm_gen_rawjson_collapsed`）。Model/Settings の Raw JSON は従来どおり。
|
||||
- 落とし穴: `.wfm-gen-rawjson-col`（幅540px）より前に書いた `--collapsed { width:30px }` は詳細度が同じで負ける。`.wfm-gen-rawjson-col.wfm-gen-rawjson-col--collapsed` と2クラスで指定する。
|
||||
- **検証**: Comic Creator内のiframe（ComfyUI_5、8189）で、横並び（757px : 514px）・折りたたみ（540→30px）・狭幅での折り返しと、Comic Creator経由の実生成（Qwen2.1 1枚/2枚入力、Flux.3 Klein）を確認。
|
||||
- ヘルプ（helpGen4 英/日/中・`index.html`）とREADMEを更新。マニュアル（v0.8.9版）は未更新。
|
||||
|
||||
|
||||
---
|
||||
|
||||
## v0.8.11（2026-10-26）
|
||||
|
||||
### Gallery: Send CC をレイアウト／Imageタブの2ボタンに分割
|
||||
|
||||
- **背景**: これまで「Send CC」はレイアウトタブ（選択コマ／オーバーレイ）専用で、Imageタブへ送れるのはComic Creatorの「I2Iへ送る」を押した直後（`window.parent._ccI2ITargetMode === 'image'`）の戻り経路だけだった。送り先を自分で選べるようにしたい、という要望。
|
||||
- **UI**（`templates/index.html`）: `wfm-gallery-send-cc-btn` を「Send CC Layout」に改名し、隣に `wfm-gallery-send-cc-image-btn`「Send CC Image」を追加。どちらもComic Creator内にiframe埋め込みされたときだけ表示（従来どおり）。
|
||||
- **Send CC Layout**（`gallery-tab.js`）: 従来の経路（画像は `insertImageFromUrl`、MP4は `insertVideoFromUrl`）のまま。`_ccI2ITargetMode` による分岐は撤去（Imageタブへは新ボタンで送る）。
|
||||
- **Send CC Image**: Comic Creator側の新設ブリッジ `window.parent.sendImageToImageTab(url, name)` を呼ぶ。ブリッジは Imageタブへ `switchTab` してから `_ccImageTab.loadFromUrl` で読み込む（作業中のレイヤーがあれば新規レイヤー、無ければ新規キャンバス）。動画はImageタブ未対応のため従来どおりトーストで弾く。ブリッジが無い（古いComic Creator）／失敗時は「Comic Creator内から開いてください」のトースト。
|
||||
- **連携の前提**: Comic Creator側の `window.sendImageToImageTab`（`14-integrations.js`、feature/ocr-text-insert ブランチ）が必要。
|
||||
- **検証**: Comic Creator内のGalleryで両ボタンの表示と、画像選択→Send CC Image→Imageタブへの読み込み（トースト表示）を実機（ComfyUI_5）で確認。Send CC Layoutの実クリックは未検証（処理は従来のまま）。
|
||||
- ヘルプ（helpGallery37、日英中）、README、マニュアル第8章・第11章（日英）を更新。
|
||||
- 教訓: ComfyUI_5への同期は変更したファイルだけでなく、事前に実行側とHEADが同一であることを `diff --strip-trailing-cr` で確認してからコピーする。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.10（2026-10-23）
|
||||
|
||||
### Tagger: PixAI Tagger v1.0 対応
|
||||
|
||||
- **対象**: コミュニティ変換の `Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8`（FP32/FP16/INT8、sigmoid込みの確率出力、`selected_tags.csv`にcategory列）。モデルフォルダに`preprocess.json`と`selected_tags.csv`があり、行数が出力数(30,877)と一致すればPixAI形式とみなす（`TaggerService._load_pixai_categories`）。`list_models`は該当モデルに`pixai: true`を付ける。
|
||||
- **前処理**（`_predict_pixai`）: 透過は白合成 → 長辺1008にBICUBICで縮小 → 黒で1008角にセンタリング → RGBのまま`(x/255-0.5)/0.5` → NCHW。WD系(白パディング/BGR/0-255)とは別経路。
|
||||
- **タグ抽出**: general(0)/style(1)/copyright(3)/character(4)のみ採用（meta/ratingは除外）、確率の高い順。推奨しきい値は general 0.17 / style 0.15 / copyright 0.24 / character 0.27。UIのThreshold(general,style)とChar Thr.(copyright,character)は推奨値を基準にした相対指定（スライダー=0.17/0.27で推奨値そのもの）。
|
||||
- **UI**（`tagger-tab.js`）: PixAIモデル選択でスライダーが0.17/0.27へ、WD系へ戻すと0.35/0.85へ切替（Single/Batch両方）。手動で変えた値は上書きしない。
|
||||
- **性能**: CUDA(onnxruntime-gpu)で約0.4秒/枚（FP16・1008px）、CPUでは約21秒/枚。
|
||||
- 教訓: `onnxruntime`と`onnxruntime-gpu`が同じvenvに共存すると、後から入ったCPU版が同じ`onnxruntime/`フォルダを上書きしてCUDAが無言で無効になる（`get_available_providers()`にCUDAが出ない）。insightface/rtmlib/qwen-tts/onnxruntime-genaiが`onnxruntime`を要求するため再発しやすい。直し方は両方uninstallして`onnxruntime-gpu`のみ入れ直し。
|
||||
- ヘルプ（helpTagger1/2/4、日英中）を更新。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.9（2026-10-20）
|
||||
|
||||
### Video: アセット登録ボタン / Editの音声変換 / ソースパネルのレイアウト
|
||||
|
||||
- **アセット登録**: 動画ソースパネルに「アセット登録」ボタンを追加（`video-tab.js`）。ローカルファイルは`comfyUI.uploadImage`でinputへ上げ、バックエンド`POST /api/wfm/video/asset/register`（`VideoService.register_asset`）でoutput/videoへコピーしてから`__Video Assets__`グループへ追加する。Galleryはoutputフォルダしか一覧しないためコピーが必要。すでにoutput配下のソース（Asset選択・生成結果）はコピーせずタグ付けのみ。同名はアンダースコア+連番で回避。登録後はAssetタブを更新。
|
||||
- **Editの音声変換**: クロップ行のボタンをクロップ関連（編集/キャンセル/リセット）左寄せ→`|`区切り→「音声変換...」の並びに変更。Galleryの変換モーダルを`openAudioConvertDialog()`として共通化・export（`convertOne`コールバック方式。`allowDelete`/`allowCopyMeta`で項目の出し分け）。Gallery側の挙動は不変。Editのクリップはinputフォルダにあるため`POST /api/wfm/video/edit/convert-audio`（`VideoService.convert_audio_to_output`、PyAVを`asyncio.to_thread`で実行）がoutput/videoへ保存する。音声付き動画クリップのみ対象、読み込み中のクリップはトーストで案内。
|
||||
- **ソースパネルのレイアウト**: パネル幅を220→320pxに拡大（左右プレビューはflex:1で自動的に縮む）、ファイル名を専用の1行（省略表示）にし、ボタン3つを折り返さず横一列に。
|
||||
- ヘルプ（helpVideo5/helpVideo11、日英中）を更新。
|
||||
- 教訓: i18n.jsの英語文字列内に`"`を入れると全体が構文エラーになる（`node --check`は.mjsコピーで確認）。
|
||||
|
||||
## v0.8.8（2026-10-19）
|
||||
|
||||
### MOSS-Image 対応（GenerateUI / Gallery / Metadata / Iタブ）
|
||||
|
||||
ユーザー提供の5本（Base/SFT × T2I/バウンディングボックス、manga 8stepのLoRA構成）で、`convertUiToApi`→`analyzeWorkflow`を実測して直した。
|
||||
ユーザー提供の5本（公式テンプレートのBase/SFT × T2I/バウンディングボックス4本と、公式ではない自作のmanga 8step LoRA構成1本）で、`convertUiToApi`→`analyzeWorkflow`を実測して直した。
|
||||
|
||||
- **サブグラフの外側ウィジェット値のズレ**: 新フロントエンドは外側ノードに`widgets_values_named`（名前キーの値）も保存し、`inputs[]`にはseed/steps/cfg/sampler/schedulerが載らない一方`widgets_values`は全ポートを持つ。従来の「`inputs[]`内の位置」参照だとunet_nameに25、vae_nameに`er_sde`が入っていた。`_flattenSubgraphs()`で`widgets_values_named`を優先（無い場合は従来どおり）。Iタブ用の複製（`node_sets_menu.js`の`_wfmFlattenSubgraphs`）にも同じ修正。
|
||||
- **`CreateBoundingBoxes`の変換**: `bboxes`の型`BOUNDING_BOX,ARRAY,STRING`が共用体の途中にSTRINGを含むため入力欄と誤判定され、本来の入力欄`editor_state`/`last_incoming`（型`BOUNDING_BOXES`）が読まれていなかった。共用体型は先頭セグメントだけで判定し、`BOUNDING_BOXES`を入力欄型に追加。配列値はAPIでリンク扱いされるため`{"__value__": [...]}`で送る（バックエンドが展開）。Flux.3 Klein 4種にも同ノードがあり、変換前は検証エラーになる出力だったものが直った。
|
||||
- **プロンプト検出**: `MOSSImageLayoutPrompt`の`prompt`をpositiveとして検出（`textKey: prompt`）。リンクされた`PrimitiveStringMultiline`/`PrimitiveString`はロールを上流へ伝播し、リンクのままの値は編集対象にしない（従来は配列がtextに入っていた）。
|
||||
- **Gallery Promptタブ**: APIの`prompt`チャンクが無くサブグラフ入りのUI形式のみの画像は、変換してから抽出（Metadataタブと同じ扱い）。通常の生成画像は従来どおりAPI形式を使う。
|
||||
- 実機(8189)でbboxes版とmanga版の生成成功を確認。ワークフロー357件で変換結果を新旧比較し、差分はFlux.3 Klein 4件のCreateBoundingBoxesのみ。
|
||||
- 実機(8189)で公式bboxes版と自作manga版の生成成功を確認。ワークフロー357件で変換結果を新旧比較し、差分はFlux.3 Klein 4件のCreateBoundingBoxesのみ。
|
||||
|
||||
### 変換チェック（GenerateUI Check）の初回待ち時間を短縮
|
||||
|
||||
|
||||
+21
-15
@@ -27,7 +27,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Built-in AI tools (translation and more)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Features (tab -> feature names)
|
||||
|
||||
@@ -38,22 +38,22 @@ Thumbnail / Table views, Thumbnail side panel, Badge filtering, Search, Side pan
|
||||
One-click capture, Auto-save as thumbnail, Embedded workflow metadata, Auto-import
|
||||
|
||||
### GenerateUI Tab (v0.3.5)
|
||||
6-tab layout, Save button, Input tab, Model tab, Settings tab, Always-visible Raw JSON, Bypass/Mute node handling, One-click generation, Seed control, Style selector, Create Catalog / Catalog buttons, Batch type selector, Batch tab, UI-to-API conversion, Video workflow support, Eagle integration, Qwen Image 2.1 support, Ming Image support
|
||||
6-tab layout, Save button, Input tab, Multi-image I2I for Comic Creator, Model tab, Settings tab, Always-visible Raw JSON, Bypass/Mute node handling, One-click generation, Seed control, Style selector, Create Catalog / Catalog buttons, Batch type selector, Batch tab, UI-to-API conversion, Video workflow support, Eagle integration, Qwen Image 2.1 support, Ming Image support, P-H3 Image support, MiniMax-H3-Image support
|
||||
|
||||
### Feeder subtab (v0.3.5 / v0.3.42)
|
||||
ImageFeeder node control, Image library, Selection management, Presets, Continuous Run loop, Index sync, Seed, WFS_GalleryFeeder node, Node & group selector, Image grid, After Gen modes, Run / Stop controls
|
||||
|
||||
### Lab subtab (v0.3.87 – v0.5.2)
|
||||
Setting / Results / Plan JSON sub-panels, Per-column keyframes, Model column — Checkpoint / LoRA Single / VAE, Empty state when no matching node exists, Wildcards, Prompt cell extras, Keyframe #1 live-reflect, Bypass checkbox, LoRA node bypass & prompt injection, T2I workflow support, Use generated image for next, Workflow recall, Plan files, Plan Load, Plan JSON tab, Save index image to Output on Run, Index images are self-contained plan files, Eagle integration, Select vs. edit, and keyframe reordering, PG — apply a Prompt tab preset group, Video workflow support, Use generated image for next, now video-aware
|
||||
### Lab subtab (v0.3.87 – v0.9.5)
|
||||
Setting / Results / Plan JSON sub-panels, Per-column keyframes, Model column — Checkpoint / LoRA Single / VAE, Empty state when no matching node exists, Wildcards, Prompt cell extras, Keyframe #1 live-reflect, Bypass checkbox, LoRA node bypass & prompt injection, T2I workflow support, Use generated image for next, Workflow recall, Plan files, Plan Load, Plan JSON tab, Save index image to Output on Run, Index images are self-contained plan files, Eagle integration, Select vs. edit, and keyframe reordering, PG — apply a Prompt tab preset group, Video workflow support, Use generated image for next, now video-aware, Plans from a story
|
||||
|
||||
### Prompt Tab
|
||||
3-column layout, AI chat assistant, Image attachment, Translation, Prompt presets, Preset Manager, Group management, Clipboard copy, GenUI Set, Wildcard/Style tab bar, Wildcard input toolbar, Wildcard file manager, Style manager, Form / Table toggle, Table view — Presets / Presets Group / Wildcards / Style
|
||||
|
||||
### Settings Tab
|
||||
2-column layout, Collapsible sections, Theme selection, Theme customization, Workflows directory, Gallery output directory, Eagle connection, CivitAI Host, CivitAI API Key, Default workflow, Default Checkpoint, VRAM Management (Ollama), Video Playback Volume, Data Management, Text Size, RAW JSON Colors, GenerateUI Model Tab Highlight, Wildcard Integration, G'MIC-Qt Integration, Language
|
||||
2-column layout, Collapsible sections, Theme selection, Theme customization, Workflows directory, Gallery output directory, Eagle connection, CivitAI Host, CivitAI API Key, Default workflow, Default Checkpoint, VRAM Management (Ollama), Video Playback Volume, Data Management, Text Size, RAW JSON Colors, GenerateUI Model Tab Highlight, Wildcard Integration, G'MIC-Qt Integration, Prompt Translation (native language → English), Language
|
||||
|
||||
### Gallery Tab (v0.3.44)
|
||||
Output / ImagePrompt / Style_Catalog / Metadata sub-tabs, Image browser, Folder tree root label, Thumbnail / Table views, Folder management, File operations, Download, MP4 video support, MP4 embedded metadata, WebM support, Embedded metadata reading fixes, Audio support, File conversion, Move To folder filter, Semantic Search, Audio waveform colors, Info tab: dimensions & duration, Multi-select, Image Compare, Prompt search, Server-side filtering, Group management, Thumbnail F button, Favorites, Detail panel, Prompt tab, Nanobanana tab, Gemini / Nano Banana nodes, MOSS-Image in GenerateUI, Faster GenerateUI Check / conversion, Partner API nodes in GenerateUI, Workflow viewer, GenUI button, Image Edit button, Send GenUI Image button, Send to LI node button, Send to LV node button, Send CC button, Search clear (✕), Clear all filters (✕ Clear), Workflow auto-save, Output folder configurable, SVG file support, PSD file support, Performance
|
||||
Output / ImagePrompt / Style_Catalog / Metadata sub-tabs, Image browser, Folder tree root label, Thumbnail / Table views, Folder management, File operations, Download, MP4 video support, MP4 embedded metadata, WebM support, Embedded metadata reading fixes, Audio support, File conversion, Move To folder filter, Semantic Search, Audio waveform colors, Info tab: dimensions & duration, Multi-select, Image Compare, Prompt search, Server-side filtering, Group management, Thumbnail F button, Favorites, Detail panel, Prompt tab, Nanobanana tab, Gemini / Nano Banana nodes, MOSS-Image in GenerateUI, Faster GenerateUI Check / conversion, Partner API nodes in GenerateUI, Workflow viewer, GenUI button, Image Edit button, Send GenUI Image button, Send to LI node button, Send to LV node button, Send CC Layout / Send CC Image buttons, Search clear (✕), Clear all filters (✕ Clear), Workflow auto-save, Output folder configurable, SVG file support, PSD file support, 3D model support, Performance
|
||||
|
||||
### ImagePrompt Gallery subtab (v0.3.94, renamed from Style/Prompt in v0.3.95)
|
||||
Visual prompt library, 3-column layout, Prompt builder (right pane), Plain-text prompt storage, ponyxlWildcardsVault format support, Seed data importer
|
||||
@@ -73,14 +73,17 @@ Model Browser, Thumbnail / Table views, Table column sort, Search & Filter, User
|
||||
### Tagger Tab (v0.3.38)
|
||||
3 sub-tabs, Model support, Threshold sliders, VLM tagging, Single tab, Single output options, Batch tab, DB tab, Dependencies
|
||||
|
||||
### TagSpace Tab (v0.9.0)
|
||||
Choose tags visually in a 3D latent space, Build / Update, Choosing tags, Explore, Search with a card's or Gallery image's tags, Make a prompt, Tagger fill, Glow / Twinkle, Hover preview, Requirements
|
||||
|
||||
### Image Edit Tab (v0.3.65)
|
||||
Layer-based image editor, Loading images, Open PSD button, Close button, New button, Tools, Layer panel, Duplicate layer (⧉), Layer lock, Text quality, Export, Canvas navigation, Undo / Redo
|
||||
|
||||
### Video Tab (v0.5.0 – v0.7.4)
|
||||
Dedicated video generation UI, independent of GenerateUI, Plan subtab, Base settings row, Edit subtab, Edit: Multi-track, Edit: Audio / BGM, Edit: Text overlays, Edit: Crop, Edit: Undo / Redo, Edit project persistence, Asset subtab, Project subtab, Two center previews, First Frame / Last Frame are both optional, Semantic node lookup, not hardcoded IDs, Field highlight color, Load in Video, Frame tab, GIF tab, Playback volume
|
||||
Dedicated video generation UI, independent of GenerateUI, Plan subtab, Base settings row, Edit subtab, Edit: Multi-track, Edit: Audio / BGM, Edit: Text overlays, Edit: Crop, Edit: Convert audio, Edit: Undo / Redo, Edit project persistence, Asset subtab, Project subtab, Two center previews, Register as Asset, First Frame / Last Frame are both optional, Semantic node lookup, not hardcoded IDs, Field highlight color, Load in Video, Frame tab, GIF tab, Playback volume
|
||||
|
||||
### AI TOOL Tab (v0.3.14)
|
||||
4-pane layout, Translation pane, Chat pane, Chat pane — image generation via Tool Calling, Chat pane — generate_image extra parameters, TOOLS pane (VLM), TOOLS pane — shared Chat attachment, TOOLS pane (Wildcards), Chat pane — image-to-image (I2I), Chat pane — SVG generation, no ComfyUI workflow involved, Chat pane — Skills, Settings pane, Unsloth backend, Decision API client, Settings pane — Chat Image Generation, Settings pane — Chat I2I Generation, Settings pane — Generation, Model unload, Settings shared, Backend support, URL security, Lemonade's other endpoints (not integrated)
|
||||
### AI TOOL Tab (v0.3.14 – v0.9.5)
|
||||
4-pane layout, Translation pane, Chat pane, Chat pane — image generation via Tool Calling, Chat pane — generate_image extra parameters, TOOLS pane (VLM), TOOLS pane — shared Chat attachment, TOOLS pane (Wildcards), Chat pane — image-to-image (I2I), Chat pane — SVG generation, no ComfyUI workflow involved, Chat pane — Skills, Chat pane — Skills v2, Built-in skills, Chat pane — Image check & refine loop, Chat pane — Workflow Doctor, Chat pane — Storyboard to Lab, Chat pane — Agent Skills folders, TOOLS pane — OCR, Collapsible Translation / Settings panes, Settings pane, Unsloth backend, Decision API client, Settings pane — Chat Image Generation, Settings pane — Chat I2I Generation, Settings pane — Generation, Model unload, Settings shared, Backend support, URL security, Lemonade's other endpoints (not integrated)
|
||||
|
||||
### Workflow Studio Library (ComfyUI Side Panel) (v0.3.9)
|
||||
Tab layout (W / N / P / M / I / A), W — Workflows tab, N — Nodes tab, M — Models tab, P — Prompts tab, I — Information tab, A — AI TOOL tab, Drag & drop workflows, Drag & drop nodes, Drag & drop prompts, Send to Canvas, Copy prompts, Double-click, Search, ⚙ Theme settings
|
||||
@@ -125,10 +128,12 @@ Samples for the Feeder tab: `ws_sdxl_image_feeder_i2i` / `ws_dwpose_image_feeder
|
||||
|
||||
## Manual
|
||||
|
||||
A beginner-oriented manual (covers v0.8.3, text-only edition without screenshots) is available in [`docs/manual/`](docs/manual/):
|
||||
A beginner-oriented manual (covers v0.9.9, text-only edition without screenshots) is available in [`docs/manual/`](docs/manual/):
|
||||
|
||||
- English: [`docs/manual/en/`](docs/manual/en/) — ZIP: `Workflow_Studio_Manual_v0.8.3_en.zip`
|
||||
- 日本語: [`docs/manual/`](docs/manual/) — ZIP: `Workflow_Studio_Manual_v0.8.3.zip`
|
||||
- English: [`docs/manual/en/`](docs/manual/en/) — ZIP: `Workflow_Studio_Manual_v0.9.9_en.zip`
|
||||
- 日本語: [`docs/manual/`](docs/manual/) — ZIP: `Workflow_Studio_Manual_v0.9.9.zip`
|
||||
|
||||
An EPUB edition with screenshots (Japanese and English, v0.9.9) is available on [Google Drive](https://drive.google.com/drive/folders/14mOHcnLYrFgNrOo6fy7ELyFjjWrAoU-_?usp=sharing).
|
||||
|
||||
---
|
||||
|
||||
@@ -166,11 +171,11 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
|
||||
### Optional
|
||||
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features; also runs PLaMo 3 Translate for Settings → Prompt Translation (`ollama pull hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M`, or the Download button there)
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** (no key; `onnxruntime`, `tokenizers` and `huggingface_hub` — normally already present with ComfyUI; the model downloads on first use). `onnxruntime-gpu` makes the local model ≈45× faster on NVIDIA GPUs
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** in [Ollama](https://ollama.com) (no key; `ollama pull embeddinggemma-2:740m` or `embeddinggemma-2:440m`)
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
@@ -210,7 +215,8 @@ ComfyUI-Workflow-Studio/
|
||||
│ │ ├── settings_routes.py # Settings API
|
||||
│ │ ├── ollama_routes.py # Ollama proxy API
|
||||
│ │ ├── eagle_routes.py # Eagle integration API
|
||||
│ │ └── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ ├── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ └── tagspace_routes.py # TagSpace tab API (build / space / search / neighbors)
|
||||
│ └── services/
|
||||
│ ├── workflow_service.py # Workflow file operations
|
||||
│ ├── nodes_service.py # Node metadata & node sets
|
||||
|
||||
@@ -23,7 +23,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Built-in AI tools (translation and more)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Screenshots
|
||||
|
||||
@@ -55,9 +55,9 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
| :--------------------------------: | :------------------------------------------: |
|
||||
|  |  |
|
||||
|
||||
| Gallery Tab - Semantic Search ("sunset city street") |
|
||||
| :-----------------------------------------------------------------: |
|
||||
|  |
|
||||
| Gallery Tab - Semantic Search ("sunset city street") | TagSpace Tab - Search ("sunset city street") |
|
||||
| :-----------------------------------------------------------------: | :-----------------------------------------: |
|
||||
|  |  |
|
||||
|
||||
---
|
||||
|
||||
@@ -99,7 +99,8 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- **6-tab layout** — Input / Model / Settings / Feeder / Batch / **Lab** tabs; Input, Model, and Settings each include a Raw JSON column on the right for instant preview and direct editing
|
||||
- **Save button** — located at the right end of the subtab row; opens a filename dialog (default: current workflow name) and saves the current workflow as a `.json` file to the Workflow tab via the import API; hidden while the Batch or Lab subtab is active (v0.4.6), since those work on workflow copies / their own Plan files rather than the currently loaded workflow
|
||||
- **Default-workflow snapshot fix** (v0.4.8) — overwriting the loaded workflow under a filename that's currently set as the auto-load **Default workflow** (Settings tab, see below) now also refreshes that cached snapshot; previously only the file on disk was updated, so the old (pre-edit) version — including any checkpoint no longer installed — kept reappearing after every page reload until the default was re-set by hand
|
||||
- **Input tab** — Prompt and Image inner tabs (drag-and-drop upload); Prompt tab shows Positive Prompt and Negative Prompt textareas, plus an **Embeddings selector** at the bottom (Filter + Select + Weight input + Paste button); Paste inserts `(embedding:Name:weight)` at the cursor position of the last focused textarea (defaults to Positive when neither is focused); Raw JSON (540px) in the right column
|
||||
- **Input tab** — Prompt (left) and Image (right, about 3:2) side by side on one screen (drag-and-drop upload; Image wraps under Prompt when the area is narrow, about 720px or less); Prompt area shows Positive Prompt and Negative Prompt textareas, plus an **Embeddings selector** at the bottom (Filter + Select + Weight input + Paste button); Paste inserts `(embedding:Name:weight)` at the cursor position of the last focused textarea (defaults to Positive when neither is focused); Raw JSON (540px) in the right column, collapsible to a slim strip with the » button (Input tab only; remembered)
|
||||
- **Multi-image I2I for Comic Creator** (v0.8.12) — new iframe bridge `window._wfmReceiveMultiImageI2IRunRequest(imageBlobs, params, workflowData?, workflowFilename?)` used by Comic Creator's Nanobanana+ tab (Local mode): image *i* goes to the workflow's *i*-th LoadImage node (extra images are ignored), and optional width / height / steps / cfg / denoise / sampler / scheduler / seed are written through `setGenerationParams`; returns `{ ok, url, slotCount, usedImages }`
|
||||
- **Revert button** (v0.3.70) — a ↺ button next to each Apply button reverts the textarea to the node's currently applied value (the reverse of Apply)
|
||||
- **Emphasis weight editing** (v0.3.70) — select text (or just place the cursor on a word/parenthesis block) and press Ctrl+↑/↓ to adjust `(text:weight)` by ±0.05, A1111/native-ComfyUI style; parentheses are removed automatically when the weight returns to 1.0
|
||||
- **Mask slot for inpainting** (v0.3.71) — LoadImage slots whose MASK output feeds an inpaint node (e.g. `VAEEncodeForInpaint`) show an additional Mask drop zone; when a mask is set, image + mask are composited into a single RGBA upload (mask baked into the alpha channel) matching ComfyUI's native alpha→MASK extraction — no extra nodes required
|
||||
@@ -132,6 +133,8 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- **Eagle integration** — auto-save generated images to [Eagle](https://eagle.cool/) with metadata
|
||||
- **Qwen Image 2.1 support** (v0.7.3) — `TextEncodeQwenImage21` (the unified T2I/Edit encoder used by Comfy-Org's Qwen-Image-2.1 workflow templates) is now recognized: its `prompt`/`negative_prompt` widgets populate the Input tab's Positive/Negative Prompt textareas and are editable like any other prompt node; its `resolution` (a pixel-budget int, not width/height) is exposed for Raw JSON editing. No extra work was needed for the model's up to 10 reference images (`image_1`…`image_10`, used for image editing) — each one arrives as an ordinary LoadImage node, so it already gets its own card in the Input tab's Image sub-panel via the existing generic LoadImage detection
|
||||
- **Ming Image support** (v0.7.5) — `TextEncodeMingImageEdit` (single `prompt` widget, up to 8 optional `images.image_N` reference inputs; role taken from the sampler/guider wiring, so the empty-prompt negative instance in layer-decomposition workflows shows as Negative) and `EmptyQwenImageLayeredLatentImage` (Latent panel; `layers` kept as-is) are recognized in GenerateUI, and Gallery/Metadata/sidepanel I tab now show their prompts as Positive. The Sampler field is disabled (N/A) when the SAMPLER input comes from a node without `sampler_name` (e.g. `SamplerLCM`). **Fix:** top-level nodes whose widgets are converted to wired inputs (e.g. `EmptyLatentImage` width/height fed by `GetImageSize`) keep those values in `widgets_values`; they were not stripped outside subgraphs, shifting later values (`batch_size` became `1024`, KSampler `steps`/`cfg` misread). Also: `BasicGuider`'s `conditioning` is treated as positive, and a linked text input wins over the stale widget text in UI-format metadata
|
||||
- **P-H3 Image support** (v0.9.4) — the `PH3ImageConditioning` templates (Image Edit and T2I) load and run from GenerateUI, with the prompt shown on the Input tab and read back by the Metadata tab. Width and height follow the workflow's wiring, and the prompt upsampler runs as set in the workflow
|
||||
- **MiniMax-H3-Image support** (v0.9.6) — the `MiniMaxH3ImageConditioning` templates (Image Edit and T2I) load and run from GenerateUI the same way, with the prompt shown on the Input tab and read back by the Metadata tab and the sidepanel I tab
|
||||
|
||||
</details>
|
||||
|
||||
@@ -164,7 +167,7 @@ Two independent modes selectable via **[Image Loop] / [Gallery]** toggle buttons
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>Lab subtab (v0.3.87 – v0.5.2)</h3></summary>
|
||||
<summary><h3>Lab subtab (v0.3.87 – v0.9.5)</h3></summary>
|
||||
|
||||
An experimental batch generator: runs the workflow currently loaded in GenerateUI N times, letting Model (Checkpoint/LoRA Single/VAE) / Prompt / KSampler each change independently starting at a chosen iteration — unlike the Batch tab's single-axis queue, every column keeps its own list of change points. Never mutates the loaded workflow; every iteration runs on a fresh clone. Works with both I2I and T2I workflows (v0.3.90).
|
||||
|
||||
@@ -190,6 +193,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **PG — apply a Prompt tab preset group** (v0.4.8) — a **PG** button on the Prompt column header opens a picker for the Prompt tab's Table-view preset groups (the same groups the Batch tab's checklist uses); choosing one replaces the entire Prompt column with one keyframe per preset in that group's order, starting at #1 (#1 is overwritten, not appended after). If the group has more presets than the current Batch count, Batch count is raised automatically to fit (capped at 50, the Batch input's own limit); anything still beyond that shows the existing "keyframes beyond Batch count" warning instead of being dropped
|
||||
- **Video workflow support** (v0.5.2) — loading a MiniMax H3 (or other all-in-one video-generation) workflow into GenerateUI and running Lab now works end to end: the Prompt column writes to its `prompt` widget, the KSampler column auto-hides (same "no matching node" treatment as Model sub-columns) since the sampler wired internally is a fixed part of the model's own preset rather than something meant to be tuned per keyframe, the Results grid plays mp4 results inline, and the contact-sheet index image draws each video result's last frame instead of failing silently onto a black square
|
||||
- **Use generated image for next, now video-aware** (v0.5.2) — when a Run's output is a video, the "Use generated image for next" chain extracts its last frame client-side (canvas) and uploads it as the next iteration's source image, instead of trying to feed the raw mp4 into a LoadImage node
|
||||
- **Plans from a story** (v0.9.5) — AI TOOL Chat's **Storyboard to Lab** skill loads a ready plan here via **Open in Lab**; KSampler cells now show every value that is set (Step / CFG / Denoise / sampler / scheduler / Seed) instead of "(workflow default)" when only the seed or sampler is set
|
||||
|
||||
</details>
|
||||
|
||||
@@ -253,6 +257,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **GenerateUI Model Tab Highlight** (v0.3.98) — pick the color used to mark Checkpoint/VAE/Diffusion Model/Text Encoder/ControlNet/Hypernetwork labels and the LoRA column header in GenerateUI's Model tab when the loaded workflow actually contains that type of node; default bright green; changes apply immediately, Reset Defaults restores it; saved to `localStorage` under `wfm_settings.modelTabActiveColor`; the same color and mechanism also drives the Video tab's field highlighting (v0.5.4, see above)
|
||||
- **Wildcard Integration** — link the WFS wildcard directory to ComfyUI-Impact-Pack's `wildcards/` directory (directory junction on Windows, symlink on other OS); existing WFS wildcard files are automatically migrated; requires [ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack)
|
||||
- **G'MIC-Qt Integration** — set the path to `gmic_qt.exe` for the Image Edit Filter tool; download G'MIC-Qt Standalone from `gmic.eu/download.html` (Windows: `gmic_qt-win64.zip`), extract anywhere, then enter the full path to `gmic_qt.exe` and click Save
|
||||
- **Prompt Translation (native language → English)** (v0.9.9) — off by default. Adds a 🌐 button to the GenerateUI Positive/Negative prompts, the Lab prompt modal and the Video Plan block prompt that translates only the parts written in your own language into English with a local Ollama model, keeping prompt syntax as it is (commas, `(word:1.2)`, `[ ]`, `{a|b}`, `<lora:...>`, `__wildcards__`, `embedding:name`, `BREAK`); "Original" restores the text. Model: [PLaMo 3 Translate](https://huggingface.co/pfnet/plamo-3-translate-2b-GGUF) 0.8B / 2B / 8B / 31B (Download button pulls it into Ollama; free for personal and non-commercial use under the PLaMo Community License) or any other Ollama model. Source language auto-detect or one of 51 languages; run on GPU (about 0.1 s per phrase, optionally unloaded after each translation) or **CPU kept loaded** (about 0.3 s per phrase, no VRAM, preloaded when GenerateUI opens). **Translate automatically when running**: GenerateUI Generate (replaces the prompt text), Lab Run and Video Plan Run (translate only what is sent; the plan keeps your text); if translation fails nothing is generated. Server routes: `POST /api/wfm/translate`, `GET /api/wfm/translate/config|models`, `POST /api/wfm/translate/pull|warmup|unload`
|
||||
- **Language** — English / Japanese / Chinese
|
||||
|
||||
</details>
|
||||
@@ -276,9 +281,9 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **File conversion** (v0.8.0) — a **Convert...** button in the Gallery bulk bar (shown when the selection contains audio or video) converts audio files to FLAC / MP3 / Opus / M4A / WAV, or extracts the audio track of videos, using PyAV (no system ffmpeg needed); bitrate, sample rate (44.1 / 48 kHz) and channels (mono / stereo) are selectable, output goes next to the source as a new file (`name.ext`, or `name_converted.ext` if taken). ComfyUI `prompt` / `workflow` tags (and title / artist / album tags) are carried over to formats that can hold them — WAV cannot, so they are skipped there; favorite / tags / memo / groups can be carried over, and the original can optionally be deleted after a successful conversion (with a confirmation). `.ogg` is readable but not offered as an output (the bundled PyAV has no Vorbis encoder)
|
||||
- **Move To folder filter** (v0.8.0) — the Move To dialog (detail panel and bulk bar) has a filter box that narrows the folder list by name or path (space = AND), Enter moves to the first match; the move toast now reads "Moved N file(s)" since audio/video files are moved too
|
||||
- **Semantic Search** (v0.8.2) — find images by meaning instead of keywords. The **S** button next to the search box turns it on: the search box then takes a free-text description (e.g. "sunset city street"), Enter searches, and results are ordered by similarity with a score badge on each thumbnail. Turning **S** on also shows a second toolbar row (left-aligned): **Similar** (lists images similar to the selected one; selecting another image keeps the list, **↻** re-runs it for the newly selected image, pressing Similar again returns to the normal list), the **⚙** settings button, and the result count. Searches cover the selected folder and its subfolders; sort and tag/group filters do not apply to semantic results
|
||||
- **Index window (⚙)** — three tabs. **Index**: target folder (optionally with subfolders), "new/changed only" or "rebuild all", progress bar, stop, delete all. **Model**: choose with radio buttons between **Gemini Embedding 2** (API; set `GEMINI_API_KEY` in the plugin's `.env` and restart ComfyUI — the free tier exists but content sent on it may be used by Google to improve its products, beyond it the API is paid) and **EmbeddingGemma 2** (runs locally, free, no key; the ~300 MB q4 ONNX model is downloaded on first use), plus vector size (Gemini 768 / 1536 / 3072, EmbeddingGemma 2 128 / 256 / 512 / 768), and for the local model the device (auto / CPU / GPU CUDA), CPU thread count (default: half the cores) and image precision (high / standard ≈2.5× faster / light ≈5× faster). **Search**: minimum score for similar-image results (e.g. `0.6` shows only images scoring 0.6 or higher), minimum score for text search, and maximum results — stored per model because score levels differ between models
|
||||
- **Index window (⚙)** — three tabs. **Index**: target folder (optionally with subfolders), "new/changed only" or "rebuild all", progress bar, stop, delete all. **Model**: choose with radio buttons between **Gemini Embedding 2** (API; set `GEMINI_API_KEY` in the plugin's `.env` and restart ComfyUI — the free tier exists but content sent on it may be used by Google to improve its products, beyond it the API is paid) and **EmbeddingGemma 2** (runs locally in Ollama, free, no key), plus vector size (Gemini 768 / 1536 / 3072, EmbeddingGemma 2 128 / 256 / 512 / 768), and for EmbeddingGemma 2 the Ollama URL and the model size (740m or 440m). **Search**: minimum score for similar-image results (e.g. `0.6` shows only images scoring 0.6 or higher), minimum score for text search, and maximum results — stored per model because score levels differ between models
|
||||
- **Index state in the folder tree** — a green dot next to a folder's image count means every image directly in that folder is indexed; the index is one shared store, so folders can be indexed one at a time. Images, plus videos / PSD via their thumbnails (audio is skipped), are embedded; changing the model, vector size or image precision requires "Rebuild all"
|
||||
- **Local GPU** — EmbeddingGemma 2 is a port of Transformers.js' pipeline onto `onnxruntime` (no `transformers` support needed). On an RTX 4070 it indexes ≈0.13 s per image with CUDA versus ≈6 s on the CPU (≈45× faster, identical vectors); to use the GPU, install `onnxruntime-gpu` in place of `onnxruntime` (`pip uninstall onnxruntime onnxruntime-gpu` then `pip install onnxruntime-gpu`). DirectML is not supported (its vision model fails). The model is released from memory when indexing finishes
|
||||
- **EmbeddingGemma 2 in Ollama** — download the model with `ollama pull embeddinggemma-2:740m` (text + image + audio) or `ollama pull embeddinggemma-2:440m` (text + image, smaller). Both return identical vectors, so you can switch sizes without rebuilding (`:latest` is the same as 740m and is used when 740m is missing; 270m / 570m cannot embed images). Images are sent 4 per request (≈0.1–0.2 s per image on an NVIDIA GPU) and retried automatically when Ollama drops a connection. The model is unloaded from VRAM when indexing finishes. Indexes built with the earlier built-in ONNX runtime keep working (cos ≈0.97 with the Ollama vectors); use "rebuild all" to regenerate them exactly. Other Ollama hosts must be listed in `WFS_ALLOWED_BACKEND_HOSTS`
|
||||
- **Audio waveform colors** (v0.8.0) — waveform thumbnails are colored per format (MP3 blue, WAV green, FLAC amber, OGG/Opus purple, M4A rose)
|
||||
- **Info tab: dimensions & duration** (v0.5.1) — the detail panel's Info tab now also shows image dimensions (width×height) or video duration, in addition to filename, size, date, tags, and memo
|
||||
- **Multi-select** — Ctrl+click to select multiple images; Bulk Bar appears for batch operations: select group → "Add to Group" / "Remove from Group"; Favorite All / Unfavorite All; Compare (2–4 images); Move To...; **Export** (downloads all selected images as a ZIP file); **Create PSD** (v0.6.0, combines the selected images into one layered PSD file, positioned top-left on a canvas sized to the largest image); **Edit as Layers** (v0.6.0, opens all selected images in the Image Edit tab at once, each as its own layer); Delete File
|
||||
@@ -292,7 +297,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Prompt tab** (v0.4.7) — selected by default (the most frequently checked piece of info shouldn't need an extra click); lists every positive/negative prompt candidate found in the embedded workflow with POS/NEG badges, reusing the exact same list→click→full-text UI as the Metadata tab; extracts from the API-format embedding when available rather than the UI-format one, since a workflow with independent top-level and subgraph sampling chains (e.g. Krea-2) can otherwise have some of its prompts missed; hidden entirely for images in the `cc_nanobanana` folder (see Nanobanana tab below), since they carry no embedded ComfyUI workflow to extract a prompt from
|
||||
- **Nanobanana tab** (v0.7.14) — shown only for images in a `cc_nanobanana` folder (written by ComfyUI Comic Creator's Nanobanana tab) that have a matching entry in that folder's `history.jsonl` generation log; shows engine, model, image size, seed, timestamp, and the prompt/negative prompt actually used — the same info is also surfaced in the Metadata tab (see below) when opened from here via the **Metadata** button
|
||||
- **Gemini / Nano Banana nodes** (v0.8.5) — images generated by ComfyUI's native Nano Banana nodes (`GeminiNanoBanana2V2` etc., e.g. via ComfyUI Comic Creator's "ComfyUI native node" engine) are readable like any other ComfyUI image: the Metadata tab and the side-panel I tab show the prompt (in the prompt list, including the appended `NEGATIVE_PROMPT:` text), the model name (under Diffusion Model, e.g. `Gemini Nano Banana 2.1`) and the seed; these images need no history file and are not special-cased like the `cc_nanobanana` folder (API-format workflows only)
|
||||
- **MOSS-Image in GenerateUI** (v0.8.8) — the MOSS-Image templates (Base / SFT, T2I and the bounding-box layout versions, plus the 8-step distillation LoRA setup) load and run from GenerateUI. The `MOSSImageLayoutPrompt` text is edited as the positive prompt, `CreateBoundingBoxes` converts correctly (the boxes themselves are still drawn on ComfyUI's canvas), and the Gallery Prompt tab, Metadata tab and side-panel I tab read the prompt, models and sampler settings of MOSS images. Newer frontends save subgraph widget values by name (`widgets_values_named`); these are now read correctly, so model names and sampler values no longer shift
|
||||
- **MOSS-Image in GenerateUI** (v0.8.8) — the official MOSS-Image templates (Base / SFT, T2I and the bounding-box layout versions) load and run from GenerateUI; a custom (non-official) manga workflow using MOSS-Image Base with the 8-step distillation LoRA was also tested and works. The `MOSSImageLayoutPrompt` text is edited as the positive prompt, `CreateBoundingBoxes` converts correctly (the boxes themselves are still drawn on ComfyUI's canvas), and the Gallery Prompt tab, Metadata tab and side-panel I tab read the prompt, models and sampler settings of MOSS images. Newer frontends save subgraph widget values by name (`widgets_values_named`); these are now read correctly, so model names and sampler values no longer shift
|
||||
- **Faster GenerateUI Check / conversion** (v0.8.8) — the first conversion or check used to download the full `/object_info` (tens of MB, 40+ s on installs with many custom nodes); it now fetches only the node types the workflow uses (about 0.25 s) and caches them per type
|
||||
- **Partner API nodes in GenerateUI** (v0.8.6) — workflows containing ComfyUI's partner API nodes (Nano Banana / Gemini image nodes, etc.) can be run from GenerateUI. These nodes need Comfy credentials in the `/prompt` request's `extra_data`, which ComfyUI's own screen adds automatically but a separate page cannot: add `COMFY_API_KEY=...` (a Comfy API key issued at platform.comfy.org) to the plugin's `.env` and restart ComfyUI. With it set, GenerateUI queues prompts through the plugin server (`/api/wfm/partner/prompt`), which adds the key only when the workflow contains an API node (`api_node` in `/object_info`) and only for this ComfyUI instance; the key never reaches the browser, and usage is billed as Comfy credits. Without a key, an authentication error now shows how to set it up. A Gemini image node's prompt shows in the Prompt tab (reference images go through `LoadImage` in the Image tab); sample: `ws_nano_banana_2_1_t2i`. Also fixed: UI-format → API-format conversion misaligned `widgets_values` for nodes whose DynamicCombo option has socket sub-inputs (Nano Banana 2.1's `images` / `files`) or a widget right after a `control_after_generate` seed; Gallery now shows the Prompt tab for `cc_nanobanana`-folder images that carry embedded metadata (images made through Comic Creator's ComfyUI native node engine)
|
||||
- **Workflow viewer** — Metadata tab displays workflow JSON from PNG embedded data (`prompt` / `workflow` keys) or from workflow saved by the Generate UI tab; **Copy & Send Canvas** button copies the JSON to the clipboard and sends the workflow directly to the ComfyUI canvas (UI and API formats both supported)
|
||||
@@ -301,13 +306,14 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Send GenUI Image button** — toolbar button (next to Image Edit); uploads the selected image to ComfyUI and sets it as the input for the first LoadImage node in the GenerateUI tab; automatically switches to GenerateUI → Input → Image; requires a workflow with at least one LoadImage node to be loaded
|
||||
- **Send to LI node button** (v0.5.2) — toolbar button (next to Send GenUI Image); uploads the selected image and writes its filename into the `image` widget of the node currently selected on the ComfyUI canvas (falling back to the first LoadImage-type node found) — same destination and cross-window bridge as the Image Edit tab's **Send to LI node** button (renamed from "Send to Workflow" in this same release), so both entry points now share one consistent name; requires Workflow Studio to have been opened from ComfyUI's top menu button
|
||||
- **Send to LV node button** (v0.7.14) — toolbar button (next to Send to LI node), the video equivalent: uploads the selected video and writes its filename into the `file` widget of the node currently selected on the ComfyUI canvas, restricted to ComfyUI's native **Load Video** node (`LoadVideo`) since `file` as a widget name isn't unique enough to search for generically the way `image` is; only enabled when the selected item is a video (mp4/webm); same cross-window bridge and launch requirement as Send to LI node
|
||||
- **Send CC button** (v0.3.69) — toolbar button, shown only when this Gallery tab is embedded as an iframe inside ComfyUI Comic Creator (a separate custom node); sends the selected image directly into Comic Creator's currently selected panel/overlay; when the selected file is an MP4 video (v0.6.3), it is routed to Comic Creator's Layout-tab Video tool instead — inserted as a new video object in the selected panel (playable, capturable to a still frame) rather than as a flattened image; sending a video while the Image tab is the intended destination is not yet supported and shows an info toast instead
|
||||
- **Send CC Layout / Send CC Image buttons** (v0.3.69, split in v0.8.11) — toolbar buttons, shown only when this Gallery tab is embedded as an iframe inside ComfyUI Comic Creator (a separate custom node). **Send CC Layout** sends the selected image directly into Comic Creator's Layout tab (the currently selected panel/overlay); when the selected file is an MP4 video (v0.6.3), it is routed to the Layout-tab Video tool instead — inserted as a new video object in the selected panel (playable, capturable to a still frame) rather than as a flattened image. **Send CC Image** (v0.8.11) switches Comic Creator to its Image tab and loads the selected image there — as a new layer if a document is already open, otherwise as a new canvas; videos are not supported there and show an info toast. Requires a Comic Creator version that provides `window.sendImageToImageTab` (the one released together with v0.8.11).
|
||||
- **Search clear (✕)** — inline ✕ button appears inside the search box whenever text is entered; click to clear immediately
|
||||
- **Clear all filters (✕ Clear)** — toolbar button next to the view-mode (Thumbnail/Table) switch (v0.5.2, moved from the toolbar's rightmost position so it sits with the other display-affecting controls); resets search text, tag filter, group filter, and favorites filter in one click (sort order is preserved)
|
||||
- **Workflow auto-save** — images generated from the Generate UI tab have their workflow automatically saved to gallery metadata
|
||||
- **Output folder configurable** — set the scanned output folder from Settings tab
|
||||
- **SVG file support** (v0.3.77) — `.svg` files are browsable, previewable (detail panel and lightbox), and included in favorites/tags/groups/move/delete/export alongside raster formats; served as-is with no rasterized thumbnail, since `<img>` scales vector graphics natively at any size
|
||||
- **PSD file support** (v0.6.0) — `.psd` files are browsable alongside other formats; thumbnails and full-size previews show every layer merged together (hidden layers excluded), rendered server-side via `psd-tools` and disk-cached; the underlying PSD file itself is never modified — only the Gallery preview is a flattened rendering
|
||||
- **3D model support** (v0.9.4) — `.glb` / `.gltf` / `.vrm` / `.ply` (mesh, point cloud or Gaussian splat) / `.spz` files are browsable alongside images with a **3D** badge and open in a built-in viewer in the detail panel, lightbox and compare view (drag to rotate, right-drag to pan, wheel to zoom). Thumbnails are rendered by your browser the first time a file appears and then kept on the server (files over 128 MB get one when opened). Workflows saved by ComfyUI's SaveGLB are read like an image's. A `.gltf` using separate `.bin` or texture files needs them to stay in place: Move and Delete only affect the `.gltf` itself. KTX2-compressed textures are not supported
|
||||
- **Performance** — server-side 256px JPEG thumbnail generation with disk cache (`data/thumb_cache/`); infinite-scroll paging (50 images per page, IntersectionObserver); folder-level mtime cache (60s TTL); bulk operations use single-request API endpoints
|
||||
|
||||
</details>
|
||||
@@ -403,7 +409,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
<summary><h3>Tagger Tab (v0.3.38)</h3></summary>
|
||||
|
||||
- **3 sub-tabs** — Single / Batch / DB for single-image tagging, folder batch processing, and tag database management
|
||||
- **Model support** — WD Tagger (ONNX, NCHW/NHWC auto-detect), SwinV2 (ONNX), DeepDanbooru (.h5, requires TensorFlow); place each model in its own subfolder under `ComfyUI/models/tagger/<model-name>/` containing the `.onnx` + `selected_tags.csv` (or `.h5` + label file)
|
||||
- **Model support** — WD Tagger (ONNX, NCHW/NHWC auto-detect), SwinV2 (ONNX), DeepDanbooru (.h5, requires TensorFlow); place each model in its own subfolder under `ComfyUI/models/tagger/<model-name>/` containing the `.onnx` + `selected_tags.csv` (or `.h5` + label file). **PixAI Tagger v1.0** (v0.8.10) is supported via the community ONNX conversion [Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8](https://huggingface.co/Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8): put one of `model.onnx` / `model_fp16.onnx` / `model_int8.onnx` (only one `.onnx` per folder) together with `selected_tags.csv` and `preprocess.json` in a subfolder. It is detected by `preprocess.json` and uses its own preprocessing (1008px, black padding, RGB, mean/std 0.5; the model already outputs probabilities); tags come from the general / style / copyright / character categories (meta and rating are left out). Selecting it switches the sliders to its recommended defaults (Threshold 0.17 / Char Thr. 0.27); the sliders then offset the recommended per-category values (Threshold moves general and style, Char Thr. moves character and copyright). FP16 needs a GPU to be fast: with both `onnxruntime` and `onnxruntime-gpu` installed the CPU package overwrites the GPU one and CUDA is silently unavailable, so keep only `onnxruntime-gpu`
|
||||
- **Threshold sliders** — General threshold (default 0.35) and Character threshold for `character:` prefix tags (default 0.85); sliders update the display value in real time
|
||||
- **VLM tagging** (v0.3.38, multi-backend: v0.5.4) — optional vision model alongside WD Tagger, running on Ollama, LM Studio, Lemonade, or Unsloth through the same unified backend used by the Prompt tab; results from both models are merged into one comma-separated tag string
|
||||
- **⚙ Settings modal** (v0.5.4) — a Settings button (Single and Batch tabs) opens the same modal component as the Prompt tab's AI Assistant (Backend / Connection / Model / Generation / Unload), but saved under its own `localStorage` key so Tagger's VLM backend/model choice is independent of the Prompt tab's; **Thinking mode** and **Max tokens** apply to VLM requests too, and the **Unload** button frees the selected model (Ollama/LM Studio/Lemonade; Unsloth unsupported) — replaces the old Tagger-only direct Ollama URL/model fields
|
||||
@@ -415,6 +421,22 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>TagSpace Tab (v0.9.0)</h3></summary>
|
||||
|
||||
- **Choose tags visually in a 3D latent space** — WFS's own data is placed in the EmbeddingGemma 2 vector space and shown in 3D (three.js, bundled): **Cards** (ImagePrompt images with their prompt, drawn as thumbnails), **Tags** (words split out of ImagePrompt prompts, the positive prompts embedded in Gallery images, wildcards, the Tagger DB and the Prompt library, colored by source) and **Gallery points** (a sample of indexed Gallery images as a faint background, each carrying the tags from its own prompt). Nearby items have similar meanings
|
||||
- **Build / Update** — collects the cards, adds card images that are not yet indexed to the Gallery semantic index, reads the positive prompt embedded in each indexed Gallery image (followed upstream from the sampler's `positive` input, never the negative; cached in `tagspace_prompts.json`, deleted images are skipped), embeds new tags with Ollama (cached in `tagspace_tags.npz`) and projects everything to 3D with PCA. Prompts are split only at commas (and `|` `{}` `BREAK`), not at spaces; phrases over 5 words are dropped. **Tags** sets how many tags are kept — each source is ranked by how many prompts use a tag and the sources take turns, so the many Gallery prompts do not crowd out the others; **Gallery points** sets how many Gallery images are shown. Images and text form two separate clusters in the same space (modality gap), so each is centered on its own mean (images: the cards) before PCA, which makes cards and their matching tags overlap. First build: about 2.5 min for ~1,000 cards plus about 2 min to read the prompts of 29,000 Gallery images (64% had a readable positive prompt); rebuilds with caches: about 15 s
|
||||
- **Choosing tags** — **All tags** uses every candidate tag instead of the Tags number (up to 20,000); **Min uses** drops tags used in fewer prompts than that (all sources combined); **Wildcards** turns wildcard files on or off as a tag source. Words found only in wildcards are most of the distinct tags (about 37,000 of 47,600 in testing), so All tags without wildcards gives about 10,400 tags (about 6,800 with Min uses 2); ~32,000 points still render at 60 fps. Wildcard YAML is read with a YAML parser, so multi-line (`- >-`) items are included; JSON fragments embedded in prompts (e.g. MOSS-Image bbox layouts) and model file names are not turned into tags
|
||||
- **Explore** — drag to rotate, right-drag to pan, wheel to zoom; hover shows name, thumbnail and prompt/sources; layer checkboxes for Cards / Thumbnails / Tags / Gallery, and source checkboxes (ImagePrompt / Gallery prompts / Tagger DB / Prompt library / Wildcards) that show or hide tags by origin — a tag stays visible while any of its sources is on, and hidden tags are left out of the nearby/search lists. The search box (Enter, any language) places a marker for the text and lists the nearest items
|
||||
- **Search with a card's or Gallery image's tags** — a selected card or Gallery point lists its prompt tags as buttons; pick some and search the space with them (joined in prompt order); while tags are picked, the Add button becomes "Add selected (N)" and adds only those tags. A selected Gallery point also has **Open in Gallery**, which opens its folder in the Gallery tab with the image selected (filters are cleared)
|
||||
- **Make a prompt** — click a card or tag to select it and list nearby items (clicking does not change the prompt); double-click it, press **Add** next to its name under Selection, or press **+** in the list to add it (again to remove); cards and tags are ranked separately and interleaved, so a card also shows its closest tags. Adding a Gallery point adds all tags from that image's prompt as one chip (images without readable tags cannot be added); clicking one lists similar Gallery images together with nearby tags and cards (cards, tags and searches never list Gallery images). The prompt joins the chips with duplicates removed; **Add to GenerateUI (Positive)** appends it to the loaded workflow's positive prompt, **Copy** puts it on the clipboard
|
||||
- **Tagger fill** — Gallery images without a readable prompt (about a third) can be tagged with a Tagger model (ONNX models from the Tagger tab) during Build / Update — either only the sampled Gallery points or every indexed image without prompt tags. Default thresholds, rating labels dropped, results cached per image and model in `tagspace_tagger.json` (stopping keeps finished images). The tags form the **Tagger fill** source and become the tags of those Gallery points. The status shows CUDA/CPU: on the CPU it is about 0.5 s per image, so make sure `onnxruntime-gpu` is active before tagging everything
|
||||
- **Glow / Twinkle** — points are drawn as glowing stars on a night-sky background (Gallery points dimmed so dense areas don't wash out); **Twinkle** makes tags, cards and Gallery points twinkle slowly with individual rhythms (selection/nearby highlights stay steady). Both are toolbar toggles; turning Glow off restores the plain dots
|
||||
- **Hover preview** — hovering the image under Selection or a thumbnail in the Nearby list shows an enlarged preview without changing the selection; the Add / Remove button sits on Selection's title row so it stays reachable while the text scrolls
|
||||
- **Requirements** — Ollama with EmbeddingGemma 2 (Gallery > Semantic settings > Model); a Gallery semantic index built with Gemini must be rebuilt with EmbeddingGemma 2 first
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>Image Edit Tab (v0.3.65)</h3></summary>
|
||||
|
||||
@@ -425,11 +447,11 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **New button** — create a blank canvas with custom dimensions (WxH prompt); clears all existing layers and automatically adds an empty Layer 1 (v0.3.70)
|
||||
- **Tools** — Select (V), Draw (B), Text (T), Shape (S), Fill (G), Mask (🎭), Blur (≈), BG Remove (⬚), Filter (★), Inpaint (🩹); tool options bar updates per active tool
|
||||
- **Select** — click to select; drag to move; drag corner handles to resize; drag circle handle to rotate; Flip H / Flip V / Rotate angle in the options bar; double-click a text layer to re-edit its content
|
||||
- **Draw** — freehand brush; options: color, brush size, blend mode; paints directly onto the active draw layer while all other layers remain visible; brush cursor shown as a size-accurate circle; strokes can start from outside the canvas margin and continue past the edge; **pen tablet support** (v0.6.2) — brush size and opacity respond to pen pressure (mouse behaves exactly as before); the canvas listens for Pointer Events so pen, touch, and mouse all work through the same code path
|
||||
- **Draw** — freehand brush; options: color, brush size, blend mode; paints directly onto the active draw layer while all other layers remain visible; brush cursor shown as a size-accurate circle; strokes can start from outside the canvas margin and continue past the edge; **pen tablet support** (v0.6.2) — brush size and opacity respond to pen pressure (mouse behaves exactly as before); the canvas listens for Pointer Events so pen, touch, and mouse all work through the same code path; **color palette** on the second row — Color 1/Color 2, 20 basic colors, and 20 custom slots (registered and edited colors are saved in the browser); **eyedropper** button next to Color picks a color from the canvas (`Esc` cancels); shortcuts while Draw is active: `D` Draw, `E` Erase, `X` switch Color 1/2, `[` `]` brush size, `B` open or close the brush picker (`Esc` also closes it)
|
||||
- **Text** — click to place; configure font, size, bold/italic, align, and color; placed as an exact-size text layer sized to the measured bounding box; double-click to re-edit
|
||||
- **Shape** — drag to draw geometric shapes (Rect / Ellipse / Line / FreeLine); options: shape type, Rounded toggle (Rect/Ellipse), Fill color, Stroke color + width, Opacity; each committed shape becomes an independent draw layer; Stroke None hidden for Line/FreeLine (stroke always active)
|
||||
- **Fill** (v0.6.2) — bucket fill; click a contiguous same-color region on the active layer to flood-fill it (scanline-based, ported from the Comic Creator plugin's Image tab); **Properties pane**: Mode (Solid / Gradient) — Solid: Color picker; Gradient: Shape (Linear / Radial), a draggable color-stop Ramp (click a stop to select it, drag to reposition, ＋/－ to add/remove stops, color swatch to edit the selected stop's color), and a Direction dial (drag to set angle/radius); both Tolerance (0–255, color-match threshold) and Opacity apply to either mode; Ramp and Direction dial are pointer-event based with pointer capture, so pen tablets work the same as mouse
|
||||
- **Mask** — 7 sub-tools for painting masks onto a dedicated mask layer; click **Mask** on the second row of the Layers header (renamed from "M" and moved to its own row in v0.6.0) to add a mask layer (Mask tool activates automatically); clicking a mask layer in the list also switches to the Mask tool; **Tool Options bar**: sub-tool buttons (Paint / Color / Alpha / Text / Vector / Shape / SAM3), Invert checkbox, Overlay color picker (default red), Blur slider (0–50 px); the overlay is only shown for the currently active mask layer, other mask layers stay hidden until selected (v0.3.70); SAM3 requires Mask Editor One (disabled when not installed); Color / Alpha / Text / Vector / Shape work without Mask Editor One; **Properties pane — Paint**: Mode (Add / Erase), Size (1–200 px), Hardness (0–100%); brush size responds to pen pressure on a tablet (v0.6.2, same as the Draw tool); **MASK EDITOR ONE section** (Paint sub-tool, visible when installed): Select opens the ABR stamp-brush library; ✕ clears back to Circle; when an image brush is active, Hardness is disabled and Spacing (5–100%), Angle (0–359°), Sz Jitter (0–100%), Rot. Jitter (on/off) appear; **Edit in Mask Editor One →** (v0.4.0) — sends the current canvas composite (plus the active mask layer's content, if one is selected) to a MaskEditorOne node on the ComfyUI canvas — the currently selected node if it is one, otherwise the first MaskEditorOne node found in the graph (same selection logic as **Send to LI node**) — and opens its full interactive editor (paint, SAM3, BiRefNet, shapes, etc.) in the ComfyUI browser tab; clicking Apply there imports the result back here as a new mask layer; requires Workflow Studio to have been launched from ComfyUI's top-menu button and at least one MaskEditorOne node to exist on the ComfyUI canvas; **Properties pane — Color**: click canvas to flood-select pixels by color similarity; Mode (Add / Subtract), Tolerance (0–255), Feather % (0–100); **Properties pane — Alpha**: extract the active layer's transparency as a mask; Threshold, Invert checkbox, Extract Alpha button; **Properties pane — Text**: click canvas to open a stamp overlay — enter text, set Font / Size / Bold / Italic / Align / Mode (Add / Erase), Stamp or Ctrl+Enter to apply; multi-line supported; **Properties pane — Vector**: click to add Catmull-Rom spline points; Enter (or click first point) closes and fills the polygon; Backspace removes the last point; Esc resets; Mode (Add / Erase); **Properties pane — Shape**: drag to draw Rect or Ellipse; hold Shift for square/circle; blue dashed outline preview while dragging; Mode (Add / Erase); **Properties pane — SAM3**: enter a text prompt, set Max candidates (3/6/9/12), click Segment; select candidate thumbnails (multi-select), Apply Selected (N) applies with current Mode; **Mask layer A/S operation**: A/S toggle — Add (green) composites normally, Subtract (red) cuts the mask from below; **✂ clipping mask button** clips the layer below to the painted area; brush cursor shown as size-accurate circle; strokes can start from outside the canvas margin and continue past the edge
|
||||
- **Mask** — 7 sub-tools for painting masks onto a dedicated mask layer; click **Mask** on the second row of the Layers header (renamed from "M" and moved to its own row in v0.6.0) to add a mask layer (Mask tool activates automatically); clicking a mask layer in the list also switches to the Mask tool; **Tool Options bar**: sub-tool buttons (Paint / Color / Alpha / Text / Vector / Shape / SAM3), Invert checkbox, Overlay color picker (default red), Blur slider (0–50 px); the overlay is only shown for the currently active mask layer, other mask layers stay hidden until selected (v0.3.70); SAM3 requires Mask Editor One (disabled when not installed); Color / Alpha / Text / Vector / Shape work without Mask Editor One; **Properties pane — Paint**: Mode (Add / Erase), Size (1–200 px), Hardness (0–100%); brush size responds to pen pressure on a tablet (v0.6.2, same as the Draw tool); **MASK EDITOR ONE section** (Paint sub-tool, visible when installed): Select opens the ABR stamp-brush library; ✕ clears back to Circle; when an image brush is active, Hardness is disabled and Spacing (5–100%), Angle (0–359°), Sz Jitter (0–100%), Rot. Jitter (on/off) appear; **Edit in Mask Editor One →** (v0.4.0) — sends the current canvas composite (plus the active mask layer's content, if one is selected) to a MaskEditorOne node on the ComfyUI canvas — the currently selected node if it is one, otherwise the first MaskEditorOne node found in the graph (same selection logic as **Send to LI node**) — and opens its full interactive editor (paint, SAM3, BiRefNet, shapes, etc.) in the ComfyUI browser tab; clicking Apply there imports the result back here as a new mask layer; requires Workflow Studio to have been launched from ComfyUI's top-menu button and at least one MaskEditorOne node to exist on the ComfyUI canvas; **Properties pane — Color**: click canvas to flood-select pixels by color similarity; Mode (Add / Subtract), Tolerance (0–255), Feather % (0–100); **Properties pane — Alpha**: extract the active layer's transparency as a mask; Threshold, Invert checkbox, Extract Alpha button; **Properties pane — Text**: click canvas to open a stamp overlay — enter text, set Font / Size / Bold / Italic / Align / Mode (Add / Erase), Stamp or Ctrl+Enter to apply; multi-line supported; **Properties pane — Vector**: click to add Catmull-Rom spline points; Enter (or click first point) closes and fills the polygon; Backspace removes the last point; Esc resets; Mode (Add / Erase); **Properties pane — Shape**: drag to draw Rect or Ellipse; hold Shift for square/circle; blue dashed outline preview while dragging; Mode (Add / Erase); **Properties pane — SAM3**: enter a text prompt, set Max candidates (3/6/9/12), click Segment; select candidate thumbnails (multi-select), Apply Selected (N) applies with current Mode; **Mask layer A/S operation**: A/S toggle — Add (green) composites normally, Subtract (red) cuts the mask from below; **✂ clipping mask button** clips the layer below to the painted area; brush cursor shown as size-accurate circle; strokes can start from outside the canvas margin and continue past the edge Since v0.8.13 the sent image is also saved to the node's `node.properties.bg_image` (via `node._persistBgDataUrl`, Mask Editor One v0.1.12 or later; ignored on older versions) and the mask layer data carries the image `width`/`height`, so the image and the mask preview survive a ComfyUI page reload or workflow tab switch instead of an older image being restored.
|
||||
- **Blur** — **Whole Blur**: Gaussian blur to the entire active layer with intensity slider (1–50 px); **Whole Mosaic**: pixelation mosaic with block-size slider (5–100 px); **Rect Blur / Rect Mosaic**: enable the toggle then drag a rectangle on the canvas to apply blur or mosaic to that region only (blue preview for blur, orange for mosaic); Rect Blur and Rect Mosaic are mutually exclusive; all operations support Undo
|
||||
- **BG Remove** — remove the background from the active layer; **Lightweight (@imgly)**: runs entirely in the browser via CDN (no server required; ~40 MB model downloaded on first use); **BiRefNet**: high-quality removal via Mask Editor One's Python backend (requires Mask Editor One installed and `birefnet.safetensors` in `ComfyUI/models/background_removal/`); **New Layer** option (default on) adds the result as a new layer above the original; when off, replaces the active layer in-place
|
||||
- **Filter (G'MIC)** — apply G'MIC-Qt filter effects to the active image layer; click **Edit with G'MIC** to launch G'MIC-Qt GUI; select a filter and click OK — a "G'MIC-Qt filter output" window appears; close it to save the result back as the active layer; requires G'MIC-Qt Standalone installed and path configured in Settings → G'MIC-Qt Executable Path
|
||||
@@ -447,7 +469,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
<details>
|
||||
<summary><h3>Video Tab (v0.5.0 – v0.7.4)</h3></summary>
|
||||
|
||||
- **Dedicated video generation UI, independent of GenerateUI** — a purpose-built tab for MiniMax H3 (Image to Video), LTX-2.5 (Text / Image / First & Last Frame to Video), and Wan2.2 (Text to Video) workflows (LTX-2.5/Wan2.2: v0.5.4); does not share state with the GenerateUI tab, and model selection (UNet/CLIP/VAE/Audio VAE) is fixed to the loaded workflow's own defaults rather than editable here
|
||||
- **Dedicated video generation UI, independent of GenerateUI** — a purpose-built tab for MiniMax H3 (Image to Video), LTX-2.5 (Text / Image / First & Last Frame to Video), Wan2.2 (Text to Video), and FastH3 OmniRef (MiniMax H3 Reference to Video) workflows (LTX-2.5/Wan2.2: v0.5.4); does not share state with the GenerateUI tab, and model selection (UNet/CLIP/VAE/Audio VAE) is fixed to the loaded workflow's own defaults rather than editable here
|
||||
- **Plan subtab** (center panel, batch redesign in v0.5.5) — a **plan** is a sequence of timeline blocks, each a fully self-contained generation job with its own Prompt, Duration, and First/Last Image; split the timeline with **＋ Split** / **＋ Block** / **Delete**, then **Run** to generate every block in order (Pause/Cancel work mid-run) — a plan with a single block behaves exactly like the old one-shot form. First Image can be None, an explicit image, or "use previous block's last frame" (chaining — an explicit Last Image on the previous block always wins over extracting one from its generated output). Save / Save As / Clear a plan (`ws_videoplan_*.json`, with a PNG contact-sheet thumbnail that embeds the full plan data so dropping the image back on works too)
|
||||
- **Base settings row** (topbar, v0.5.7) — Aspect Ratio / Megapixels / Multiple / Duration (seconds) / Noise Seed on one row — Duration always reflects the currently selected block (each block keeps its own value), the rest are plan-wide — plus a second row of model-specific fields: Width / Height (become directly editable for workflows with no Resolution Selector node, e.g. Wan2.2 and LTX-2.5's First & Last Frame to Video), Turbo Mode / Strength / Steps (MiniMax H3's turbo LoRA — Turbo Mode doubles as Wan2.2's own turbo toggle, sharing one field), and Prompt Enhance (LTX-2.5 only). Every field in both rows is enabled only when the loaded workflow actually has it — see Field highlight color below (one exception: Wan2.2's Text-to-Video variant doesn't expose Noise Seed at all, so it alone leaves that field disabled)
|
||||
- **Edit subtab** (v0.7.0) — a lightweight clip timeline for trimming and concatenating video/image clips outside of generation: add clips via the Video Source panel's **Add to Edit** button or the Asset subtab's **Send to Edit** button, drag blocks to reorder (or use the ◀/▶ toolbar buttons), Duplicate/Delete the selected clip, Clear to empty the timeline. The trim panel for a selected video clip shows a Clipchamp-style scrubber — a time ruler, a draggable playhead synced to the Source preview, and draggable in/out handles with time badges (image clips get a simple hold-duration field instead); the playhead's own time badge sits a row above the in/out handle badges (v0.7.2) so it stays legible even when the playhead lands right on a trim point, and the scrubber has extra headroom/height and spacing from the clip name above it (v0.7.2). **▶ Preview** plays every ready clip back-to-back (trimmed to its in/out points) in the result preview, and **Export** builds and runs the actual ComfyUI graph (trim → crop → concatenate if more than one clip → save), tagging the result into Video Temp automatically. Multi-clip exports keep each clip's own audio (v0.7.4 — previously they were always silent). **Clear** empties the clips and also resets the Audio section to its defaults (v0.7.4)
|
||||
@@ -455,15 +477,17 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Edit: Audio / BGM** (v0.7.4) — the export panel's **Audio** section sets the soundtrack: **Keep original audio** (with its own volume in dB) keeps every clip's audio aligned to its segment even across multiple clips (image clips contribute silence), and one **BGM** file is laid over the whole video with its own volume and start offset (a longer BGM is cut to the video length). Pick the BGM with **BGM...**, by dropping an audio/video file onto the Audio section, or with **Set as BGM** on a video in the Asset subtab (that video's audio track is used; videos without audio are rejected). Built entirely inside the export graph with ComfyUI core audio nodes (EmptyAudio / AudioMerge / AudioConcat / LoadAudio / TrimAudioDuration / AudioAdjustVolume) passed to ConcatenateVideo's `complete_audio` — no extra dependencies. ▶ Preview plays the BGM alongside the clips
|
||||
- **Edit: Text overlays** (v0.7.4) — each clip can carry any number of text overlays (**+ Add text** under the trim panel): text, start/end seconds within the clip, size as % of frame height, color, 9-position anchor, outline, and an optional background box. They follow the clip when it's reordered, are previewed live over the Source/Result panes, and are burned in on Export as a final pass (PyAV decode → Pillow → H.264 encode, frame by frame so memory stays flat; the soundtrack is remuxed untouched). Japanese/CJK text renders with a system CJK font (Meiryo / Yu Gothic / Noto Sans JP on Windows)
|
||||
- **Edit: Crop** (v0.7.4) — **Edit crop** puts a draggable/corner-resizable rectangle over the Source preview (optionally locked to 16:9 / 9:16 / 1:1 / 4:3 / 3:4); **Done** keeps it, **Cancel** restores the crop from before editing, and switching to another subtab/tab ends editing like Done. Export applies it with ComfyUI core `VideoCrop`; the output size is the first video clip's cropped size, and other clips of a different size are center-fit to it. Text overlay previews are drawn relative to the cropped frame
|
||||
- **Edit: Convert audio** (v0.8.9) — a **Convert audio...** button sits next to the crop buttons (separated by `|`); it converts the selected video clip's audio track to FLAC / MP3 / Opus / M4A / WAV with the same dialog as Gallery's **Convert...** (bitrate, sample rate, channels), and saves the result into the output/video folder so it shows up in Gallery. Delete-original and carry-over-gallery-meta options are not offered here. Server route: `POST /api/wfm/video/edit/convert-audio`
|
||||
- **Edit: Undo / Redo** (v0.7.4) — **↶ / ↷** in the timeline toolbar (or Ctrl+Z / Ctrl+Y / Ctrl+Shift+Z outside text fields) step through edits to clips, trims, crops, text and audio settings; a deleted clip comes back without re-uploading. History resets when a project is loaded
|
||||
- **Edit project persistence** (v0.7.2) — **Save** / **Save As** / **Load** buttons in the Edit subtab's export panel persist the timeline itself (clip order, trim points, and which uploaded server-side file each clip refers to — plus crops, text overlays and audio/BGM settings since v0.7.4) as a `ws_videoeditproj_*.json` project file; Load re-fetches and re-probes each clip from its saved server reference, so the original files need to still exist
|
||||
- **Asset subtab** (center panel, next to Plan/Edit — moved here from the sidebar in v0.7.0) — browses generated videos/images through three groups: **Video Assets** (a manually-curated keeper group you organize yourself, on your own schedule), **Video Temp** (every plan run's output lands here automatically regardless of whether the plan itself was saved — a **Clear Video Temp** button in the Gallery tab's detail panel empties it in one click), and **All Video Assets** (the union of both); a filename search box, a video/image type filter dropdown (All Types / Video Only / Image Only, v0.7.2), and a thumbnail-grid/table (▦/☰) view toggle sit above the list (v0.7.0). Selecting an item shows it in the center preview and lets you tag/memo it, **Send to Edit** (queues it as a clip in the Edit subtab), **Set as BGM** for videos (v0.7.4 — uses its audio track as the Edit BGM), jump straight to the full Gallery tab, or — for images only — **Set as First Image** / **Set as Last Image** (v0.7.0, uploads it straight into the currently-selected Plan block and switches to the Plan subtab)
|
||||
- **Project subtab** (sidebar) — lists saved plan files while the center Plan subtab is active, or saved Edit projects while the center Edit subtab is active (v0.7.2 — thumbnail/block-or-clip-count/last-updated) — click one to load it into the matching editor, or delete it with the × button; the panel collapses back down when its own tab is clicked again, and switching Plan↔Edit while it's open refreshes the list in place
|
||||
- **Two center previews** (v0.5.5) — the left preview shows either the video currently selected in the Asset subtab or whatever's dropped/picked in the **Video Source** panel to its left (**Clear** resets it to empty); the right preview always shows the most recently generated clip; both preview frames are a fixed height (v0.7.0) so switching sources never shifts the Plan/Edit/Asset panel below them. The Frame/GIF tools act on whichever preview was touched most recently
|
||||
- **Register as Asset** (v0.8.9) — a **Register as Asset** button in the Video Source panel (next to **Add to Edit** and **Clear**) puts the loaded video into the **Video Assets** group so it appears in the Asset subtab: local files are uploaded and copied into output/video (Gallery only lists the output folder), sources already in output are just tagged. The Video Source panel was also widened so the file name stays on one line and the three buttons fit in a row. Server route: `POST /api/wfm/video/asset/register`
|
||||
- **First Frame / Last Frame are both optional** — leaving First Image empty runs the workflow text-only (the node is simply left unconnected — a "bypass"); workflows that don't even include a LoadImage node for First Frame are supported out of the box (Wan2.2's Text-to-Video graph has neither slot at all), and a LoadImage node is only injected into the graph on the fly if an image is actually supplied
|
||||
- **Semantic node lookup, not hardcoded IDs** (generalized in v0.5.4) — detects which of the three families is loaded from its subgraph's internal node types (or, for Wan2.2, the subgraph's own name — it has no distinctive internal node type), then locates its Resolution Selector and LoadImage node(s) by the subgraph's *public input label* (`prompt`, `duration`, `width`, `first_frame`, `last_frame`) rather than a fixed slot index — so it keeps working even though the three families expose those inputs under different internal names; a workflow no longer needs to match any bundled reference file's exact node IDs, and no fallback test workflow is auto-loaded on tab open
|
||||
- **Field highlight color** — Prompt / Duration / First Image / Last Image / Aspect Ratio / Megapixels / Multiple / Noise Seed / Width / Height / Turbo Mode / Strength / Steps / Prompt Enhance labels turn the same highlight color as the GenerateUI Model tab (shared setting, customizable in the Settings tab — see below) when the loaded workflow actually exposes that field; e.g. a Wan2.2 workflow (no Resolution Selector node) leaves Aspect Ratio / Megapixels / Multiple unhighlighted while Width / Height light up instead
|
||||
- **Load in Video** (Workflow tab) — load any workflow containing a MiniMax H3 Image-to-Video node, an LTX-2.5 Text/Image/First&Last-Frame-to-Video node, or a Wan2.2 Text-to-Video node straight into this tab from the Workflow tab's side-panel toolbar (next to Load in GenerateUI) — starts a fresh single-block plan seeded from the workflow's own current values; unsupported workflows show an error toast instead of loading
|
||||
- **Load in Video** (Workflow tab) — load any workflow containing a MiniMax H3 Image-to-Video node, an LTX-2.5 Text/Image/First&Last-Frame-to-Video node, a Wan2.2 Text-to-Video node, or a FastH3 OmniRef (MiniMax H3 Reference-to-Video) node — for FastH3 / Pika-H3-Flex the First Image and Last Image fields become Reference Image (ref_image_0) and Reference Image 2 (ref_image_1); leaving either one empty (None / Clear) disconnects that reference input even if the workflow had an image wired to it; Reference Image 2 is never used as the previous block's last frame for "Use previous block's last frame" (the actually generated last frame is used instead) straight into this tab from the Workflow tab's side-panel toolbar (next to Load in GenerateUI) — starts a fresh single-block plan seeded from the workflow's own current values; unsupported workflows show an error toast instead of loading
|
||||
- **Frame tab** (right panel) — captures the active preview's current playback position as a still frame: play/seek to the position you want, click **Capture Frame** (client-side canvas, no server round-trip), then **Save to Output** to write it as a PNG into ComfyUI's own Output folder (auto-numbered, same pattern as other generated images); an optional **Add to Video Assets group** checkbox (v0.7.0) also tags the saved frame into Gallery's Video Assets group
|
||||
- **GIF tab** (right panel) — converts the active preview to an animated GIF; set Start/End (seconds), FPS, and an optional Max Width, then **Convert to GIF** — the server decodes the video (PyAV) and encodes an animated GIF (Pillow), both dependencies ComfyUI itself already requires, so no ffmpeg binary or extra install is needed; the result is saved into ComfyUI's Output folder and previewed inline
|
||||
- **Playback volume** — the preview player's volume is saved to the Settings tab and shared across every video player in the plugin (Video tab, Gallery tab), so it no longer resets to maximum each time a different video is opened
|
||||
@@ -471,7 +495,7 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><h3>AI TOOL Tab (v0.3.14)</h3></summary>
|
||||
<summary><h3>AI TOOL Tab (v0.3.14 – v0.9.5)</h3></summary>
|
||||
|
||||
- **4-pane layout** — Translation | Chat | TOOLS | Settings; all panes always visible simultaneously; no sub-tab switching required
|
||||
- **Translation pane** — translate text between Japanese, English, Chinese, or a custom Free language using Ollama, LM Studio, Lemonade, or Unsloth; language selectors with ⇄ swap button (swaps both language selectors and text content); selections saved automatically
|
||||
@@ -485,6 +509,14 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Chat pane — image-to-image (I2I)** (v0.3.74) — when an image is attached and `generate_image` is invoked, the attachment is uploaded and swapped into the target workflow's LoadImage node instead of running text-to-image; the target workflow is the one currently loaded in GenerateUI, or a dedicated I2I workflow configured in the Settings pane
|
||||
- **Chat pane — SVG generation, no ComfyUI workflow involved** (v0.3.77) — the LLM can also produce SVG graphics directly as text; if a reply contains SVG code (wrapped in a markdown code fence or written directly as a bare `<svg>...</svg>`), it's rendered inline as a preview and a "Save to Gallery" button writes it as a `.svg` file to the Gallery tab's output folder; unlike the Tool Calling image generation flow above, this is plain text pattern matching, so it works with any model regardless of tool-calling support
|
||||
- **Chat pane — Skills** (v0.3.79) — select a `.md` skill file from a dropdown above the message list to prepend its instructions to the conversation as a system prompt (not shown in the visible chat log); a pencil button opens an in-pane manager (list + inline editor, same pattern as the Prompt tab's wildcard file manager) to create, edit, or delete skill files stored in `user/default/Workflow-Studio/ai_skills/`; a file's `---`-delimited frontmatter (`name`/`description`) is shown in the dropdown; two starter skills are bundled — **SVG Icon Generator** (explicitly decomposes the subject into parts before drawing, which markedly improves recognizability from small local models) and **Skill Creator** (interviews you conversationally and drafts a new skill as a ```skill code block; a "Save as new skill" button appears under such replies and opens the manager pre-filled with the draft)
|
||||
- **Chat pane — Skills v2** (v0.9.5) — skills keep the portable `name`/`description` frontmatter (Agent Skills / SKILL.md style) and add optional `metadata:` keys: `output` (`prompt` or `svg` — the reply is checked and, if the model tried to output one but it is broken, it is asked once to fix it), `tools` (`[]` = never start image generation), `temperature`, `think`, and `context` (`workflow` summary, `current-prompt`, `workflow-detail`, `logs`). Six built-in skills ship in `py/services/builtin_skills/` with a version number: untouched copies are upgraded automatically, edited ones keep your text and show a ↻ button. A ```` ```prompt ```` block in a reply gets **Apply to GenerateUI** / **Generate with this prompt** / **Copy** buttons, and replies are rendered as Markdown (HTML-escaped first)
|
||||
- **Built-in skills** (v0.9.5) — **T2I Prompt Enhancer** (writes natural sentences for Qwen-Image / FLUX / Z-Image, Danbooru tags for Illustrious / Pony, phrases for realistic SDXL, based on the workflow loaded in GenerateUI), **Image to Prompt** (recreates an image attached in TOOLS), **Storyboard to Lab**, **Workflow Doctor**, **SVG Icon Generator** v2 (line/flat/duotone/mono styles) and **Skill Creator** v3. The prompt text currently in GenerateUI is only passed to the model when your message refers to it, because small models copied an old negative prompt into new ones even when told not to
|
||||
- **Chat pane — Image check & refine loop** (v0.9.5) — with **Check images** on, the chat model turns the request into 3–8 yes/no checks and an Ollama decision model with image input (`d3:4b` by default; `clef-flash` / `clef` also work) answers them with probabilities: ✔ ≥ 85%, ? 30–85%, ✘ < 30%. Swap checks ("the hat is not on the girl") catch the attribute mix-ups the judge otherwise passes. **Fix missing items and regenerate** sends only the unmet items back for a rewritten prompt; 0–3 automatic rounds are configurable. SVG replies are rendered and checked part by part the same way
|
||||
- **Chat pane — Workflow Doctor** (v0.9.5) — diagnoses the workflow loaded in GenerateUI from facts checked on every message: node types that are not installed, model files or options missing from ComfyUI's lists (with the closest existing names), broken links, unconnected core-type required inputs, the last execution error from `/history`, and the latest error block of the ComfyUI log (no false positives across 77 local workflows)
|
||||
- **Chat pane — Storyboard to Lab** (v0.9.5) — a ```` ```labplan ```` reply becomes a Lab plan (one scene per iteration, T2I mode, identical character descriptions, fixed seed); **Open in Lab** loads it into GenerateUI > Lab
|
||||
- **Chat pane — Agent Skills folders** (v0.9.5) — **Import** in the skill manager adds a `.zip` containing `SKILL.md` (or a single `.md`); only text files are kept, scripts are never extracted or run, and zip-slip / size limits apply. Reference files are listed to the model and read on demand through a `read_skill_file` tool (progressive disclosure; needs a tool-capable model)
|
||||
- **TOOLS pane — OCR** (v0.9.5) — **OCR (extract text)** reads text with an Ollama OCR model (LightOnOCR-3 by default; PaddleOCR-VL, GLM-OCR and DeepSeek-OCR are listed too). Ported from Comic Creator's OCR tab: 1600 px white-background JPEG, temperature 0, grayscale retry on empty / looping output. **Show text positions** (LightOnOCR / DeepSeek-OCR) overlays numbered boxes; clicking one selects its line. **To Translation** / **To Chat** pass any TOOLS result on
|
||||
- **Collapsible Translation / Settings panes** (v0.9.5) — the arrow in each header folds the pane into a thin strip (click the strip to reopen); the state is remembered
|
||||
- **Settings pane** — choose backend (Ollama / LM Studio / Lemonade / Unsloth), set the API URL, test connection, select a model (with refresh button), and configure Free language names for translation source and destination
|
||||
- **Unsloth backend** (v0.3.99) — Unsloth Desktop's OpenAI-compatible API requires an API key even for local access unless its **Keyless API access → "Chat and inference"** setting is on; unlike the other backends it's proxied server-side (`py/routes/unsloth_routes.py`, `POST /api/wfm/unsloth/proxy`) so the key never reaches the browser: set `UNSLOTH_API_KEY` in a `.env` file in the plugin folder (copy `.env.example`, loaded via `python-dotenv` in `prestartup_script.py`) and restart ComfyUI — or, since v0.7.7, leave `UNSLOTH_API_KEY` unset and turn on Keyless API access → "Chat and inference" in Unsloth Desktop (requests are then relayed without an `Authorization` header, still only to localhost; that Unsloth setting also opens the API to your private LAN, and "Allow tools" should stay off). A 401 with no key set returns a message explaining both options; also requires **"Auto model switching (OpenAI API)"** to be turned on in the Unsloth Desktop app's own Settings screen — with it off, requests fail with HTTP 404 unless a model was already manually loaded in the Unsloth Studio UI
|
||||
- **Decision API client** (v0.7.5, foundation) — `static/js/decision-client.js` calls Unsloth Desktop's Decision API (`/v1/systemone`, TypeSafe-compatible Laya decision models) through the same key-attaching server proxy: `decide(state, questions)` / `decideMany`, question builders `noul`/`choice`/`score`, and `isYes`/`pickChoice`/`pickScore` helpers that threshold on `probabilities` (as Unsloth recommends) and flag low-probability answers as not confident. Requires **Decision API → Serve requests** enabled in Unsloth Desktop. **Ollama 0.35+ backend** (v0.7.7) — Ollama's own `/v1/systemone` (decision models `tev1` 4B / `tev1:0.8b` 812 MB / `nimble` 9B / `clef` 27B vision-capable, pulled with `ollama pull`; no key, called directly from the browser); the model list shows only installed models whose `capabilities` include `decision`, with a ↻ refresh button and a pull hint when none are installed. **Image input** (v0.7.14, ported from ComfyUI Comic Creator) — most decision models are text-only, but Ollama's `clef` accepts images directly: `decide()` takes an optional 4th `images` argument (base64, no `data:` prefix), `imageToBase64()` converts a Blob/data URL/image URL, and `supportsDecisionVision()` checks whether the currently-configured model accepts them (via `/api/show`'s `capabilities`/`projector_info`, since not every Ollama build reports `vision` for decision models) — call it before passing images, since a non-vision model errors out on them; confirmed working end to end against `clef`. Connection settings live in the Settings tab's own **Decision Model** section (v0.7.6): backend (Unsloth / Ollama), URL, model and default probability threshold, stored under `wfm_decision_settings` independently of the AI TOOL / Tagger backends, with a Test button (one yes/no question). No feature uses it yet
|
||||
@@ -576,10 +608,12 @@ Samples for the Feeder tab: `ws_sdxl_image_feeder_i2i` / `ws_dwpose_image_feeder
|
||||
|
||||
## Manual
|
||||
|
||||
A beginner-oriented manual (covers v0.8.3, text-only edition without screenshots) is available in [`docs/manual/`](docs/manual/):
|
||||
A beginner-oriented manual (covers v0.9.9, text-only edition without screenshots) is available in [`docs/manual/`](docs/manual/):
|
||||
|
||||
- English: [`docs/manual/en/`](docs/manual/en/) — ZIP: `Workflow_Studio_Manual_v0.8.3_en.zip`
|
||||
- 日本語: [`docs/manual/`](docs/manual/) — ZIP: `Workflow_Studio_Manual_v0.8.3.zip`
|
||||
- English: [`docs/manual/en/`](docs/manual/en/) — ZIP: `Workflow_Studio_Manual_v0.9.9_en.zip`
|
||||
- 日本語: [`docs/manual/`](docs/manual/) — ZIP: `Workflow_Studio_Manual_v0.9.9.zip`
|
||||
|
||||
An EPUB edition with screenshots (Japanese and English, v0.9.9) is available on [Google Drive](https://drive.google.com/drive/folders/14mOHcnLYrFgNrOo6fy7ELyFjjWrAoU-_?usp=sharing).
|
||||
|
||||
---
|
||||
|
||||
@@ -617,11 +651,11 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
|
||||
### Optional
|
||||
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features
|
||||
- **[Ollama](https://ollama.com/)** — for AI chat assistant, translation, and VLM features; also runs PLaMo 3 Translate for Settings → Prompt Translation (`ollama pull hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M`, or the Download button there)
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** (no key; `onnxruntime`, `tokenizers` and `huggingface_hub` — normally already present with ComfyUI; the model downloads on first use). `onnxruntime-gpu` makes the local model ≈45× faster on NVIDIA GPUs
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** in [Ollama](https://ollama.com) (no key; `ollama pull embeddinggemma-2:740m` or `embeddinggemma-2:440m`)
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
@@ -661,7 +695,8 @@ ComfyUI-Workflow-Studio/
|
||||
│ │ ├── settings_routes.py # Settings API
|
||||
│ │ ├── ollama_routes.py # Ollama proxy API
|
||||
│ │ ├── eagle_routes.py # Eagle integration API
|
||||
│ │ └── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ ├── gallery_routes.py # Gallery tab API (Output + ImagePrompt + Style_Catalog sub-tabs)
|
||||
│ │ └── tagspace_routes.py # TagSpace tab API (build / space / search / neighbors)
|
||||
│ └── services/
|
||||
│ ├── workflow_service.py # Workflow file operations
|
||||
│ ├── nodes_service.py # Node metadata & node sets
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 942 KiB |
@@ -2,12 +2,12 @@
|
||||
|
||||
ノードが苦手でも大丈夫 ― 画像生成をもっと手軽に、もっと整理して
|
||||
|
||||
- 対象バージョン：Workflow Studio v0.8.3
|
||||
- 対象バージョン：Workflow Studio v0.9.9
|
||||
- 著者：ketle-man
|
||||
- © 2026 ketle-man
|
||||
|
||||
このマニュアルは、Kindle版をもとにした **画像なしのテキスト版** です。
|
||||
章ごとのファイルは `chapters/` フォルダ、全章をまとめたファイルは `Workflow_Studio_Manual_v0.8.3.md` です。
|
||||
章ごとのファイルは `chapters/` フォルダ、全章をまとめたファイルは `Workflow_Studio_Manual_v0.9.9.md` です。
|
||||
画面の例は省略しているため、実際の画面はWorkflow Studioのヘルプタブとあわせてご覧ください。
|
||||
|
||||
## 目次
|
||||
@@ -29,6 +29,4 @@
|
||||
- [13_付録](chapters/13_付録.md)
|
||||
- [14_おわりに](chapters/14_おわりに.md)
|
||||
|
||||
全章まとめ：[Workflow_Studio_Manual_v0.8.3.md](Workflow_Studio_Manual_v0.8.3.md)
|
||||
|
||||
English edition: [en/README.md](en/README.md)
|
||||
全章まとめ：[Workflow_Studio_Manual_v0.9.9.md](Workflow_Studio_Manual_v0.9.9.md)
|
||||
|
||||
Binary file not shown.
+706
-30
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -83,6 +83,7 @@ Workflow Studioは、そうして作り上げたワークフローを **素早
|
||||
- **画像編集（Image Edit）** － レイヤー付きの簡易編集、マスクを使った部分的な描き直し（インペイント）
|
||||
- **ローカルAI連携** － 日本語からのプロンプト翻訳、画像からのプロンプト・タグ作成
|
||||
- **Tagger・Video** － 画像からのタグ自動生成、動画の生成と簡単な編集
|
||||
- **TagSpace** － 手持ちのプロンプト・タグ・生成画像を「意味の近さ」で3Dの空間に並べ、眺めながらタグを選ぶ
|
||||
|
||||
---
|
||||
|
||||
@@ -128,7 +129,7 @@ ComfyUIの「自由さ」はそのままに、「分かりやすさ」と「手
|
||||
|
||||
## 本書の対象バージョンと環境
|
||||
|
||||
- **Workflow Studio**：v0.8.3
|
||||
- **Workflow Studio**：v0.9.9
|
||||
- **ComfyUI**：本書執筆時点の最新版
|
||||
- **OS**：Windows（画面の例はWindowsのものです）
|
||||
|
||||
|
||||
@@ -19,7 +19,8 @@ Workflow Studioの基本機能は、ComfyUIが動いていれば追加の準備
|
||||
### あとから必要に応じて
|
||||
次のものは、特定の機能を使うときだけ必要です。最初は気にしなくてかまいません。
|
||||
|
||||
- **Ollama / LM Studio など** － プロンプトの翻訳やAIチャットなどのAI補助機能（第10章）
|
||||
- **Ollama / LM Studio など** － プロンプトの翻訳やAIチャットなどのAI補助機能（第10章）。
|
||||
ギャラリーのセマンティック検索をローカルで行うとき（第8章）と、TagSpaceタブ（第11章）には **Ollama** が必要です
|
||||
- **Eagle** － 生成画像を画像管理ソフトEagleに自動保存する機能（第8章）
|
||||
- **追加のPythonパッケージ** － Tagger（画像からのタグ生成）やPSD書き出しなど（第11章）
|
||||
|
||||
|
||||
@@ -24,9 +24,10 @@ Workflow Studioを開くと、画面の上部にタブが横一列に並んで
|
||||
| Image Edit | Image Edit | 画像の簡易編集・マスク作成 | | 第11章 |
|
||||
| Video | Video | 動画生成・動画編集 | | 第11章 |
|
||||
| Tagger | Tagger | 画像からタグ（プロンプト）を自動生成 | | 第11章 |
|
||||
| TagSpace | TagSpace | タグ・プロンプト・生成画像を意味の近さで3D表示し、タグを選ぶ | | 第11章 |
|
||||
| 設定 | Settings | 言語・テーマ・フォルダなどの設定 | | 本章・第11章 |
|
||||
| ヘルプ | Help | 機能一覧・ショートカット・トラブル対処 | | 本章 |
|
||||
| AI TOOL | AI TOOL | ローカルAIによる翻訳・チャット・画像解析 | | 第10章 |
|
||||
| AI TOOL | AI TOOL | ローカルAIによる翻訳・チャット・画像解析・文字起こし（OCR） | | 第10章 |
|
||||
|
||||
### タブは「整理する」「作る」「補助する」の3グループで考える
|
||||
タブが多くて迷ったときは、次の3つのグループに分けて考えると分かりやすくなります。
|
||||
@@ -35,7 +36,7 @@ Workflow Studioを開くと、画面の上部にタブが横一列に並んで
|
||||
手持ちの素材を一覧にして、探しやすくするタブです。
|
||||
- **作る** － 生成UI／Image Edit／Video
|
||||
実際に画像や動画を作ったり、手を加えたりするタブです。
|
||||
- **補助する** － AI TOOL／Tagger／設定／ヘルプ
|
||||
- **補助する** － AI TOOL／Tagger／TagSpace／設定／ヘルプ
|
||||
他のタブでの作業を助けたり、全体の設定をしたりするタブです。
|
||||
|
||||
> **ポイント**
|
||||
|
||||
@@ -257,8 +257,8 @@ Workflow Studioで探して、ComfyUIで細かく編集する、という使い
|
||||
> △ や × でも生成自体はできることが多いので、「生成UIだけで完結しにくい」という目安として見てください。
|
||||
|
||||
> **ポイント**
|
||||
> ワークフロータブを開いて最初のチェックは、ComfyUIからノードの情報を取得するため、10秒ほどかかることがあります。
|
||||
> 2回目以降はすぐに結果が出ます。
|
||||
> チェックは、そのワークフローで使っているノードの情報だけをComfyUIから取得して行います（v0.8.8〜）。
|
||||
> 以前は最初の1回に数十秒かかることがありましたが、今は一瞬で結果が出ます。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -72,6 +72,9 @@ Input・Model・Settingsの右側には、ワークフローの中身がJSON形
|
||||
フォームで値を変えると、ここにも反映されます。
|
||||
慣れないうちは、ここは「見るだけ」にしておきましょう。
|
||||
|
||||
Inputサブタブでは、Raw JSONの上端にある **»** ボタンで、Raw JSONを右端の細い帯に折りたためます（v0.8.12〜）。
|
||||
プロンプトや画像の欄が広くなります。もう一度押すと元に戻り、折りたたんだ状態は次回も記憶されます。
|
||||
|
||||
---
|
||||
|
||||
## 5-3 はじめての1枚を生成する
|
||||
@@ -79,7 +82,7 @@ Input・Model・Settingsの右側には、ワークフローの中身がJSON形
|
||||
いよいよ生成です。最低限、次の3つを確認・設定します。
|
||||
|
||||
1. どのモデルで描くか（Model）
|
||||
2. 何を描くか（Input → Prompt）
|
||||
2. 何を描くか（Input）
|
||||
3. 生成する（Generate）
|
||||
|
||||
### 手順1：モデルを選ぶ（Model）
|
||||
@@ -95,6 +98,9 @@ Input・Model・Settingsの右側には、ワークフローの中身がJSON形
|
||||
> - 読み込んだときに「このワークフローのCheckpoint「…」が見つからず、無警告で別のモデルに差し替えられました」と表示される
|
||||
> （この場合、手元のモデルの一覧の先頭のものに自動で差し替わっています）
|
||||
>
|
||||
> なお v0.8.7〜は、フォルダ名が違うだけで同じファイル名のモデルが手元に1つだけあれば、先頭のものではなく、そのモデルに自動で差し替わります。
|
||||
> （公式テンプレートはフォルダ名なしでモデル名を保存していることが多く、以前は無関係なモデルに差し替わることがありました）
|
||||
>
|
||||
> どちらの場合も、一覧から使いたいモデルを選び直して、Apply を押してください。
|
||||
> 毎回選び直すのが面倒なときは、設定タブの **デフォルトCheckpoint**（第11章）が便利です。
|
||||
|
||||
@@ -109,7 +115,9 @@ Input・Model・Settingsの右側には、ワークフローの中身がJSON形
|
||||
> 緑色でない項目は、変更してもこのワークフローには影響しません。
|
||||
|
||||
### 手順2：プロンプトを書く（Input）
|
||||
1. サブタブの **Input** を開き、その中の **Prompt** を選びます。
|
||||
1. サブタブの **Input** を開きます。
|
||||
左側にプロンプト（Prompt）の欄、右側に入力画像（Image）の欄が並んでいます（v0.8.12〜。それまではPrompt／Imageを切り替える形でした）。
|
||||
画面の幅が狭いときは、Imageの欄はプロンプトの下に回ります。
|
||||
2. **Positive Prompt** の欄に、描いてほしい内容を英語で入力します。
|
||||
|
||||
```
|
||||
@@ -160,12 +168,13 @@ Input・Model・Settingsの右側には、ワークフローの中身がJSON形
|
||||
|
||||
1枚目が生成できたら、次は **Settings** サブタブで、画像の仕上がりを左右する設定を見てみましょう。
|
||||
|
||||
Settingsタブは、左右2つのブロックに分かれています。
|
||||
Settingsタブは、3つの列に分かれています。
|
||||
|
||||
- **KSampler（左）** － 画像をどのように描くかの設定
|
||||
- **Latent Image（右）** － 画像の大きさと枚数の設定
|
||||
- **Latent Image（中央）** － 画像の大きさと枚数の設定
|
||||
- **スイッチ（右、v0.9.6〜）** － ワークフローの中のオン・オフの切り替え
|
||||
|
||||
ここでも、値を変えたら各ブロックの **Apply** を押すのを忘れないようにしましょう。
|
||||
ここでも、KSamplerとLatent Imageは、値を変えたら各ブロックの **Apply** を押すのを忘れないようにしましょう（スイッチの列はApply不要です）。
|
||||
|
||||
### KSampler の項目
|
||||
| 項目 | 意味 | 初心者向けの目安 |
|
||||
@@ -192,6 +201,23 @@ Settingsタブは、左右2つのブロックに分かれています。
|
||||
> モデルが学習したサイズから大きく外れると、人物が二重になったり、構図が崩れたりしやすくなります。
|
||||
> 縦長・横長にしたいときも、合計のピクセル数がおおよそ同じになるように調整しましょう（例：SDXLなら 832×1216 など）。
|
||||
|
||||
### スイッチの項目（v0.9.6〜）
|
||||
ワークフローの中にある「オン・オフ」の設定が、ノード名と項目名つきで一覧になります。
|
||||
チェックボックスをクリックすると、その場で切り替わります。**Apply を押す必要はありません**。切り替えた値は右側の Raw JSON にもすぐ反映されます。
|
||||
|
||||
一覧に出るのは、次の2種類です。
|
||||
|
||||
- 値が true／false の項目 － プロンプト強化（PE）のオン・オフを切り替えるスイッチや「Boolean」ノード、TextGenerate の thinking など
|
||||
- 選択肢が2つだけのオン・オフの項目 － enable／disable、on／off、yes／no など。KSamplerAdvanced の add_noise、DWPreprocessor の detect_hand（手を検出するか）など
|
||||
|
||||
各行の右端には、いまの値（true／false、enable／disable など）が表示されます。
|
||||
行にマウスを乗せると、ノードの番号と種類を確認できます。
|
||||
|
||||
> **ポイント**
|
||||
> 別のノードから線でつながっている項目は、一覧に出ません。代わりに、つながっている元のノード（たとえば「Enable PE?」という名前の Boolean ノード）が出るので、そちらを切り替えてください。
|
||||
> ノードに名前を付けていないワークフローでは、ノードの種類名（ComfySwitchNode など）で表示されます。
|
||||
> オン・オフの項目が1つも無いワークフローでは「このワークフローに true/false の項目はありません」と表示されます。
|
||||
|
||||
### シードの管理（生成パネル）
|
||||
シードは、生成パネルの **Seed** 欄と、その下のプルダウンで管理します。
|
||||
|
||||
@@ -224,6 +250,6 @@ Settingsタブは、左右2つのブロックに分かれています。
|
||||
> **この章（前半）のまとめ**
|
||||
> - ワークフロータブで選んで「生成UIに読み込み」を押すと、生成UIで編集できる
|
||||
> - 基本は Input（プロンプト）・Model（モデル）・Settings（細かい設定）の3つのサブタブ
|
||||
> - モデルやSettingsの値を変えたら必ず **Apply**。押し忘れると変更前の値で生成される（プロンプトは自動で反映）
|
||||
> - モデルやSettingsの値を変えたら必ず **Apply**。押し忘れると変更前の値で生成される（プロンプトとSettingsのスイッチ列は自動で反映）
|
||||
> - Generate で生成。Alt＋Apply なら反映と生成を一度に行える
|
||||
> - シードを Fixed にすると、気に入った絵をもとに条件を変えて比べられる。ワークフローのシードをそのまま使うなら Workflow
|
||||
|
||||
@@ -163,6 +163,8 @@ Labでは、「何回目の生成から、どの設定に切り替えるか」
|
||||
4. 生成回数（Batch）を決めます。
|
||||
5. Model・Prompt・KSampler の各列で **＋** を押してキーフレームを追加し、
|
||||
キーフレームをダブルクリックして、「何回目から」「どの設定に」するかを入力します。
|
||||
KSampler列のセルには、指定した項目（Step・CFG・Denoise・サンプラー・スケジューラー・Seed）が表示されます。
|
||||
何も指定していないセルは「（ワークフローの設定のまま）」と表示されます。
|
||||
6. **Run** を押します。
|
||||
|
||||
結果は **Results** パネルに一覧表示されます。
|
||||
@@ -172,9 +174,185 @@ Labでは、「何回目の生成から、どの設定に切り替えるか」
|
||||
少しずつ変化していく連続画像を作れます。
|
||||
- **Plan Save** － 設定（プラン）を保存できます。保存すると、結果をまとめた一覧画像も作られ、
|
||||
その画像をドロップすれば同じプランを読み込み直せます。
|
||||
- **物語からプランを作る**（v0.9.5〜） － AI TOOLのChatで **Storyboard to Lab** スキルを使うと、
|
||||
物語をシーンごとのプロンプトに分けたプランを作り、**Labで開く** ボタンでここに読み込めます（第10章 10-5）。
|
||||
|
||||
---
|
||||
|
||||
## 5-8 パートナーAPI（Nano Banana）を使う
|
||||
|
||||
ComfyUIには、Googleの画像生成モデル **Nano Banana**（Gemini画像ノード）のように、
|
||||
クラウドのサービスに生成を任せる **パートナーAPIノード** があります。
|
||||
v0.8.6〜は、こうしたノードを含むワークフローも生成UIで実行できます。
|
||||
|
||||
> **注意**
|
||||
> パートナーAPIノードは、自分のパソコンのGPUではなくクラウドで動き、**Comfyクレジット（有料）を消費** します。
|
||||
> BatchやLabなどで繰り返し実行すると、回数ぶんのクレジットが続けて消費されます。
|
||||
|
||||
### 準備：Comfy APIキーを設定する
|
||||
1. platform.comfy.org でComfy APIキーを発行します。
|
||||
2. プラグインのフォルダにある `.env` に、`COMFY_API_KEY=発行したキー` という行を追加します。
|
||||
（`.env` がなければ `.env.example` をコピーして作ります）
|
||||
3. ComfyUIを再起動します。`.env` は起動時にしか読み込まれません。
|
||||
|
||||
キーはサーバー側でだけ使われ、ブラウザには送られません。
|
||||
キーを設定していない状態で実行すると、「パートナーAPIノードの認証に失敗しました」という案内が出ます。
|
||||
|
||||
### 使い方
|
||||
1. ワークフロータブで、Nano Bananaのサンプル（`ws_nano_banana_2_1_t2i`）を生成UIに読み込みます。
|
||||
2. **Input** サブタブの左側（Prompt）でプロンプトを編集し、**Apply** を押します。
|
||||
3. 参考画像を使うときは、**Input** サブタブの右側（Image）の画像カードで指定します。
|
||||
4. **Generate** を押します。
|
||||
|
||||
> **ポイント**
|
||||
> ComfyUI本体の画面でログインして実行する方法もあります。その場合は `.env` の設定は要りません。
|
||||
|
||||
---
|
||||
|
||||
## 5-9 対応ワークフローが増えました（v0.8.4〜）
|
||||
|
||||
生成UIで使えるワークフローが、次のように増えています。使い方は、ほかのワークフローと同じです。
|
||||
|
||||
| 対応したもの | 内容 |
|
||||
| --- | --- |
|
||||
| Qwen Image 2.1 Turbo | ComfyUI公式テンプレートの文章から画像（T2I）・画像から画像（I2I）。プロンプトエンハンサー（プロンプトを自動で補う機能）の入力欄にも、編集が反映される |
|
||||
| Flux.3 Klein | 公式テンプレート4種（KleinとKlein SDのT2IとImage Edit） |
|
||||
| MOSS-Image | Base／SFTのT2Iと、枠で位置を指定するバウンディングボックス版 |
|
||||
| Fibo 2.0 Edit（v0.8.14〜） | 画像編集のワークフロー。プロンプト強化（PE）を使う構成でも、プロンプト欄の編集がPEのオン・オフどちらにも反映される。ネガティブの欄にはVGL形式（JSON）の説明文がそのまま入る。幅・高さ（Settings）は0で自動 |
|
||||
| FastH3 OmniRef／Pika-H3-Flex（v0.8.14〜） | MiniMax H3の「参照画像から動画」（Reference to Video）。動画のワークフローとして扱われ、Videoタブ（第11章）でも使える |
|
||||
| ネイティブLoRAスタック | ComfyUI v0.40のLoRAをまとめて読み込むノード（LoadLoraModel／LoadLoraTextEncoder）。Apply・バイパス・Metadata表示に対応 |
|
||||
| P-H3 Image（v0.9.4〜） | 画像編集とT2Iのテンプレート。プロンプトはInputタブに表示され、Metadataタブでも読める。ネガティブの欄は無い。幅・高さはワークフローのつなぎ方に従い、プロンプトを補う機能（プロンプトアップサンプラー）もワークフローの設定どおりに動く |
|
||||
| MiniMax H3 Image（v0.9.6〜） | ComfyUI公式の画像編集とT2Iのテンプレート。P-H3 Imageと同じく、プロンプトはInputタブに表示され、MetadataタブとサイドパネルのIタブでも読める。ネガティブの欄は無い。プロンプト強化（PE）のオン・オフは、Settingsタブのスイッチの列（5-4）で切り替えられる |
|
||||
|
||||
> **注意**
|
||||
> ネイティブLoRAスタックのノードは、保存形式が **API形式のワークフローのみ** 対応です。
|
||||
> ComfyUI本体の画面がこのノードの行追加にまだ対応していないためで、UI形式のワークフローは対象外です。
|
||||
|
||||
> **ポイント**
|
||||
> 2枚の画像をスライダーで見比べる **Compare Images**（ImageCompare）ノードを含むワークフローは、以前は生成UIから実行すると、このノードだけエラーになっていました（v0.9.6で修正）。
|
||||
|
||||
> **ポイント**
|
||||
> 画像のメタデータを見るMetadataタブとサイドパネルの **I** タブでも、Nano Bananaで作った画像を読めます（v0.8.5〜）。
|
||||
> プロンプトは、Positive／Negativeに分けずに全文が1件で表示されます。
|
||||
|
||||
## 5-10 日本語のままプロンプトを書く（プロンプト翻訳、v0.9.9〜）
|
||||
|
||||
多くの画像生成モデルは、英語のプロンプトで学習されています。
|
||||
**プロンプト翻訳** をオンにすると、プロンプトを日本語で書き、ボタン1つで英語に直せます。
|
||||
Stable Diffusion web UI（A1111）の翻訳拡張機能と同じように、プロンプトの中の **日本語の部分だけ** を翻訳します。
|
||||
|
||||
翻訳には、パソコンで動くAI（**Ollama**）と翻訳専用のモデル **PLaMo 3 Translate** を使います。
|
||||
インターネット上の翻訳サービスにプロンプトを送ることはありません。
|
||||
|
||||
> **事前に必要なもの**
|
||||
> **Ollama** をインストールして起動しておきます（第10章 10-1）。翻訳モデルのダウンロードは、設定画面からできます（下記）。
|
||||
|
||||
### 準備：設定でオンにする
|
||||
1. **設定** タブの **プロンプト翻訳（自国語 → 英語）** を開きます。
|
||||
2. **生成UIでプロンプト翻訳を使う** にチェックを入れます。
|
||||
3. **翻訳モデル** で、使うモデルを選びます。まだOllamaに入っていないモデルには **（未取得）** と付いています。選んで **ダウンロード** を押すと、Ollamaに取り込まれます。
|
||||
4. **テスト** を押して、見本のプロンプトが英語になれば準備完了です。
|
||||
|
||||
PLaMo 3 Translate には4つの大きさがあります。
|
||||
|
||||
| モデル | 大きさ | 特徴 |
|
||||
| --- | --- | --- |
|
||||
| 0.8B | 約0.6GB | 最速。短いタグの翻訳なら十分 |
|
||||
| 2B | 約1.8GB | おすすめ。文章の翻訳も正確 |
|
||||
| 8B | 約5.3GB | より高品質 |
|
||||
| 31B | 約19.5GB | 最高品質。メモリに余裕がある人向け |
|
||||
|
||||
> **ポイント**
|
||||
> 0.8Bは、文章の翻訳で意味がずれることがあります（例：「カメラは後ろから追いかける」が「her camera following behind」＝「彼女のカメラが後ろについてくる」になった）。
|
||||
> 動画のプロンプトのように文章で書くことが多いなら、2B以上を選びましょう。
|
||||
|
||||
設定のほかの項目は、次のとおりです。
|
||||
|
||||
| 項目 | 内容 |
|
||||
| --- | --- |
|
||||
| 入力する言語 | ふつうは **自動判定** のままでよい。日本語だけで書くなら「日本語」を選ぶと確実 |
|
||||
| Ollama URL | Ollamaのアドレス。空欄なら `http://localhost:11434` |
|
||||
| 翻訳モデルの実行場所 | **GPU** か **CPUで常駐** か（下記） |
|
||||
| 実行時に自動で翻訳する | Generateなどを押したときに自動で翻訳する（下記） |
|
||||
|
||||
PLaMo以外のOllamaのモデル（ふだんチャットに使っているモデルなど）も、**その他のOllamaモデル** から選べます。その場合は「英語に翻訳して」という指示を付けて送ります。
|
||||
|
||||
### 翻訳モデルをどこで動かすか
|
||||
| 実行場所 | 1語句あたりの時間 | 向いている人 |
|
||||
| --- | --- | --- |
|
||||
| GPU（1回の翻訳が最速・VRAMを使う） | 約0.1秒 | GPUのメモリに余裕がある |
|
||||
| CPUで常駐（VRAMを使わず、再読み込みの待ちなし） | 約0.2〜0.3秒 | 画像生成にGPUのメモリをすべて使いたい |
|
||||
|
||||
GPUのときは、**翻訳後に翻訳モデルをVRAMから解放する** をオンにしておくと、画像生成の前にGPUのメモリを空けられます（初期値はオン）。
|
||||
ただし、翻訳のたびにモデルの読み込み直しで **約4秒** 待つことになります。
|
||||
|
||||
**CPUで常駐** にすると、モデルをメインメモリに置いたままにします（2Bで約2.7GB）。
|
||||
GPUのメモリを使わず、生成UIにワークフローを読み込んだときに裏で読み込んでおくので、最初の翻訳から待たされません。
|
||||
|
||||
> **ポイント**
|
||||
> Ollamaは、ほかのモデル（AI TOOLのチャット用など）を読み込むときに、常駐させた翻訳モデルを外すことがあります。
|
||||
> その場合は、次の翻訳のときに自動で読み込み直します（その1回だけ数秒かかります）。
|
||||
|
||||
### 使い方：🌐ボタンで翻訳する
|
||||
プロンプト翻訳をオンにすると、InputタブのPositive Prompt・Negative Promptの **↺** ボタンの右に **🌐** ボタンが表示されます。
|
||||
|
||||
1. プロンプト欄に、日本語で書きます。英語のタグと混ぜてもかまいません。
|
||||
|
||||
```
|
||||
masterpiece, 夕焼けの海辺に立つ少女、(長い黒髪:1.2), <lora:test_lora:0.8>, {猫耳|狐耳}, 水彩画風
|
||||
```
|
||||
|
||||
2. **🌐** を押します。
|
||||
3. 日本語の部分だけが英語になります。
|
||||
|
||||
```
|
||||
masterpiece, A girl standing by the seaside at sunset, (long black hair:1.2), <lora:test_lora:0.8>, {cat ears|fox ears}, watercolor style
|
||||
```
|
||||
|
||||
次のようなプロンプトの書き方（記法）は、翻訳せずにそのまま残します。
|
||||
|
||||
- カンマ・改行
|
||||
- 重み付け `(語句:1.2)`、`[ ]`
|
||||
- ランダムに選ぶ `{A|B}`、ワイルドカード `__名前__`
|
||||
- LoRAの指定 `<lora:名前:0.8>`、`embedding:名前`
|
||||
- `BREAK`、`AND`
|
||||
|
||||
翻訳したあとに **原文** ボタンが表示されます。押すと、翻訳する前の文に戻ります（Ctrl＋Zでも戻せます）。
|
||||
|
||||
> **注意**
|
||||
> 翻訳されるのは、英数字以外の文字を含む部分だけです。英語だけの部分は、そのまま残ります。
|
||||
> 反対に、é などのアクセント付きの文字を含む英単語は、翻訳の対象になります。
|
||||
|
||||
### 実行時に自動で翻訳する
|
||||
設定の **実行時に自動で翻訳する** で、どの実行のときに自動で翻訳するかを、チェックボックスで選べます。
|
||||
|
||||
| チェックボックス | 動き |
|
||||
| --- | --- |
|
||||
| 生成UI － Generate | Generateを押すと、Positive／Negativeの欄の日本語を英語に置き換えてから生成する |
|
||||
| 生成UI Lab － Run | Labの実行で、送る内容だけを翻訳する。Labのプランには日本語が残る |
|
||||
| Video Plan － Run | Videoタブ Planの実行で、送るプロンプトだけを翻訳する。プランには日本語が残る |
|
||||
|
||||
LabとVideo Planでは、プランを日本語のまま残すので、あとから日本語で書き直せます。
|
||||
生成した画像や動画の情報（メタデータ）には、実際に送った英語のプロンプトが記録されます。
|
||||
|
||||
> **注意**
|
||||
> 自動翻訳がオンのときに翻訳に失敗すると（Ollamaが起動していないなど）、日本語のまま生成しないよう、生成を止めます。
|
||||
> LabとVideo Planでは、止まった理由が進行状況の欄に表示されます。
|
||||
|
||||
### Labのモーダルと、Video Planでも使える
|
||||
🌐ボタンは、次の欄にもあります。使い方は同じです。
|
||||
|
||||
- Lab（5-7）のプロンプト列のセルをダブルクリックして開く画面の、Positive・Negativeの欄
|
||||
- Videoタブ Planサブタブの、ブロックの **Prompt** の欄（第11章 11-4）
|
||||
|
||||
> **ポイント**
|
||||
> PLaMo 3 Translate は、個人の利用や趣味、研究などの非商用の利用なら無料で使えます（PLaMoコミュニティライセンス）。
|
||||
> 商用で使う場合は、ライセンスの条件を確認してください。
|
||||
|
||||
> **この章（後半）のまとめ**
|
||||
> - Batchサブタブで、モデル・プロンプト・サンプラーなどを1種類ずつ変えながらまとめて生成できる
|
||||
> - Batch Queueの列のチェックで種類を選び、Generateで開始。終わったらチェックを外す
|
||||
> - Feederは、たくさんの画像を1枚ずつワークフローに送り込む機能。専用ノードが必要
|
||||
> - Labは、複数の設定を別々のタイミングで切り替えられる上級者向けの機能
|
||||
> - パートナーAPIノード（Nano Bananaなど）は、`.env` の `COMFY_API_KEY` を設定すれば実行できる。クレジットが消費される
|
||||
> - プロンプト翻訳をオンにすると、日本語で書いたプロンプトを🌐ボタンで英語にできる。記法はそのまま残り、Generate・Lab・Video Planの実行時に自動で翻訳することもできる
|
||||
|
||||
@@ -227,6 +227,10 @@ Workflow Studioにはたくさんのスタイルが付属しています。
|
||||
|
||||
英語のプロンプトの意味を知りたいときは、反対に **EN→JA** で日本語に訳せます。
|
||||
|
||||
> **ポイント**
|
||||
> 生成UIのプロンプト欄で直接翻訳したいときは、第5章 5-10 の **プロンプト翻訳**（🌐ボタン）が便利です。重み付けやLoRAの指定を残したまま、日本語の部分だけを英語にできます。
|
||||
> AI Assistantの翻訳にも、翻訳専用のモデル **PLaMo 3 Translate** を選べます（v0.9.9〜。以前は指示文まで翻訳されたり、ZH→ENで日本語が出たりしていました）。
|
||||
|
||||
### AIと相談しながらプロンプトを作る
|
||||
入力欄に質問を書いて **Send**（または Enter キー）を押すと、AIとチャットできます。
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
ComfyUIで生成した画像には、生成に使ったワークフローやプロンプトの情報（**メタデータ**）が埋め込まれています。
|
||||
ギャラリータブでは、この情報を読み取って、同じ設定をすぐに再現できます。
|
||||
また、動画（MP4・WebM）や音声ファイル（WAV・MP3・OGG・M4A・FLAC・Opus）も、画像と同じ一覧で扱えます。
|
||||
また、動画（MP4・WebM）や音声ファイル（WAV・MP3・OGG・M4A・FLAC・Opus）、3Dモデル（GLB・glTF・VRM・PLY・SPZ）も、画像と同じ一覧で扱えます。
|
||||
|
||||
---
|
||||
|
||||
@@ -135,8 +135,9 @@ ComfyUIで生成した画像には、生成に使ったワークフローやプ
|
||||
|
||||
### 方法2：ComfyUIのノード画面で再現する（コピー＆キャンバスへ送る）
|
||||
1. ギャラリーで、再現したい画像をクリックします。
|
||||
2. 詳細パネルの **メタデータ** タブを開きます。
|
||||
3. **コピー＆キャンバスへ送る**（Copy & Send Canvas）を押します。
|
||||
2. 詳細パネルの **JSON** タブを開きます。
|
||||
3. **Copy & Send Canvas**（コピー＆キャンバスへ送る）を押します。
|
||||
（このボタンは英語表示のままです。ワークフローが埋め込まれた画像でだけ押せます）
|
||||
|
||||
ComfyUIのノード画面に、その画像を作ったワークフローが読み込まれます。
|
||||
ノード画面で細かく手を加えたいときは、こちらを使います。
|
||||
@@ -153,7 +154,8 @@ ComfyUIのノード画面に、その画像を作ったワークフローが読
|
||||
| Send GenUI Image | 選んだ画像を、生成UIのワークフローの入力画像（LoadImageノード）にセットする |
|
||||
| Send to LI node | 選んだ画像を、ComfyUIのノード画面で選んでいる画像読み込みノードにセットする |
|
||||
| Image Edit | 選んだ画像を Image Edit タブで開く（第11章） |
|
||||
| Send CC | 選んだ画像を Comic Creator のコマに送る（Comic Creatorの中でギャラリーを開いたときだけ表示。第11章 11-6） |
|
||||
| Send CC Layout | 選んだ画像を Comic Creator のレイアウトタブ（選択中のコマ／オーバーレイ）に送る（Comic Creatorの中でギャラリーを開いたときだけ表示。第11章 11-6） |
|
||||
| Send CC Image | 選んだ画像を Comic Creator の Image タブに送る（作業中のレイヤーがあれば新規レイヤー、なければ新規キャンバス。動画は未対応。表示条件は Send CC Layout と同じ） |
|
||||
|
||||
I2I（画像から画像を作る）ワークフローで、生成した画像をさらに描き直していくときに便利です。
|
||||
|
||||
@@ -238,7 +240,7 @@ Eagleで画像を管理している方は、試してみてください。
|
||||
| モデル | 特徴 | 準備 |
|
||||
| --- | --- | --- |
|
||||
| Gemini Embedding 2 | Googleのサービスで計算する。速い。画像をGoogleに送る | APIキーを取得し、プラグインの `.env` に `GEMINI_API_KEY=キー` と書いて、ComfyUIを再起動する |
|
||||
| EmbeddingGemma 2 | パソコンの中で計算する（ローカル）。外部に送らない | 追加の準備は不要。初回だけモデルが自動でダウンロードされる |
|
||||
| EmbeddingGemma 2 | パソコンの中で、**Ollama** を使って計算する（ローカル・無料）。外部に送らない | Ollamaをインストールして起動し、`ollama pull embeddinggemma-2:740m`（または軽量版の `embeddinggemma-2:440m`）を実行する。モデル設定タブで **Ollama URL**（通常は変更不要）と **モデルサイズ** を選ぶ |
|
||||
|
||||
> **注意：Geminiを使うとき**
|
||||
> 画像がGoogleのサービスに送られます。無料枠で送った内容は、Googleの製品改善に使われる場合があります。
|
||||
@@ -250,13 +252,14 @@ Eagleで画像を管理している方は、試してみてください。
|
||||
2. 「新規のみ」か「全件再作成」を選んで開始します。進み具合が表示され、途中で停止もできます。
|
||||
3. インデックスは全フォルダで共通の1つです。フォルダごとに少しずつ作っても構いません。
|
||||
|
||||
モデルやベクトル次元、画像の精度を変えたときは、**全件再作成** が必要です。
|
||||
モデルやベクトル次元を変えたときは、**全件再作成** が必要です。
|
||||
|
||||
### 速さの目安と設定
|
||||
- ローカルのEmbeddingGemma 2は、初期設定ではCPUで動き、1枚あたり数秒かかります。
|
||||
**画像の精度** を「標準」や「軽量」にすると速くなります（変えたら全件再作成）。
|
||||
**CPUスレッド数** を減らすと、画像生成中のパソコンの負荷が軽くなります。
|
||||
- NVIDIAのGPUがあれば、ComfyUIのPythonに `onnxruntime-gpu` を入れると、1枚あたり約0.1秒になります（対処は第12章 12-5）。
|
||||
- ローカルのEmbeddingGemma 2は、v0.8.14からOllamaで動くようになりました（それまでの、ComfyUIのPythonで動かす方式は廃止）。
|
||||
NVIDIAのGPUなら1枚あたり約0.1〜0.2秒です。インデックスを作り終えると、モデルはGPUメモリから自動で降ろされます。
|
||||
- インデックスを作るときと、検索するときは、Ollamaが起動している必要があります。
|
||||
- モデルサイズの740m（テキスト＋画像＋音声）と440m（テキスト＋画像・軽量）は同じ結果を返すので、切り替えても全件再作成は不要です。
|
||||
v0.8.13以前に作ったインデックスも、そのまま使えます。
|
||||
- **検索設定** タブで、類似画像・文章検索それぞれの「最小スコア」と最大表示件数を決められます。
|
||||
スコアの水準はモデルによって違うため、モデルごとに保存されます。
|
||||
|
||||
@@ -294,6 +297,30 @@ ComfyUIで音楽や効果音を作る機会が増えたため、ギャラリー
|
||||
> **ポイント：Videoタブでの活用**
|
||||
> Video の Asset サブタブでは、音声ファイルを「BGMに設定」したり、「サウンドとして追加」したりできます（第11章 11-4）。
|
||||
|
||||
---
|
||||
|
||||
## 8-8 3Dモデルを見る（v0.9.4〜）
|
||||
|
||||
ComfyUIで3Dモデルを作るワークフロー（画像から3Dなど）の結果も、ギャラリーで確かめられます。
|
||||
対応しているのは、GLB・glTF・VRM・PLY（メッシュ／点群／ガウシアンスプラット）・SPZ です。
|
||||
|
||||
- 一覧には「**3D**」のバッジ付きで並びます。サムネイルは、最初に一覧に出たときにブラウザが描いてサーバーに保存します
|
||||
（128MBを超えるファイルは、開いたときに保存します）。
|
||||
- 選ぶと、右の詳細パネルの内蔵ビューアで立体として表示されます。ダブルクリックで大きく表示できます。
|
||||
|
||||
| 操作 | 動き |
|
||||
| --- | --- |
|
||||
| ドラッグ | 回転する |
|
||||
| 右ドラッグ | 位置を動かす |
|
||||
| マウスホイール | 拡大・縮小 |
|
||||
| ⟲（リセット）ボタン | 最初の向きに戻す |
|
||||
|
||||
- ComfyUIの **SaveGLB** で保存したモデルは、画像と同じようにワークフローを読み取れます（8-2・8-3）。
|
||||
|
||||
> **注意**
|
||||
> - 3Dファイルは、Image Edit・生成UI・Tagger・Feederには送れません。
|
||||
> - `.gltf` の中には、別の `.bin` や画像ファイルを参照しているものがあります。ギャラリーで移動・削除しても動くのは `.gltf` 本体だけなので、参照しているファイルの場所は変えないでください。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ギャラリーのOutputで、生成した画像をフォルダごとに見返せる。プロンプトの言葉でも検索できる
|
||||
> - 詳細パネルのプロンプトタブで、画像の生成に使ったプロンプトがすぐ分かる
|
||||
@@ -302,3 +329,4 @@ ComfyUIで音楽や効果音を作る機会が増えたため、ギャラリー
|
||||
> - メタデータサブタブに画像をドロップすると、モデル・LoRA・プロンプトを詳しく調べられる
|
||||
> - Sボタンで、文章の意味や「似た画像」から探せる（セマンティック検索）
|
||||
> - 音声ファイルも一覧・再生・変換できる
|
||||
> - 3Dモデル（GLB・glTF・VRM・PLY・SPZ）も、ギャラリーで回して確かめられる（8-8）
|
||||
|
||||
@@ -7,6 +7,9 @@ Workflow Studioは、パソコンの中で動く **ローカルAI（大規模言
|
||||
- AIとチャットしながら、プロンプトのアイデアを出してもらう
|
||||
- 画像を見せて、その画像の説明やプロンプト、タグを作ってもらう
|
||||
- ワイルドカード用の候補リストを作ってもらう
|
||||
- **スキル** を選んで、プロンプト作り・ストーリーボード・不具合の相談など、得意分野を切り替える（v0.9.5〜）
|
||||
- 生成した画像が依頼どおりかをAIにチェックさせて、足りない点だけ直す（v0.9.5〜）
|
||||
- 画像に写っている文字を読み取る（OCR、v0.9.5〜）
|
||||
|
||||
インターネット上のサービスではなく、自分のパソコンの中で動くので、
|
||||
**入力した内容や画像が外部に送られない** のが大きな特長です。
|
||||
@@ -69,10 +72,14 @@ Workflow Studio本体の **AI TOOL** タブを開きます。
|
||||
| 部分 | できること |
|
||||
| --- | --- |
|
||||
| 翻訳 | 日本語・英語・中国語などの翻訳 |
|
||||
| Chat | AIとの会話。条件がそろえば、会話から画像生成も |
|
||||
| TOOLS | 画像の読み取り（VLM）、ワイルドカード作成 |
|
||||
| Chat | AIとの会話。スキルで得意分野を切り替えられる。条件がそろえば、会話から画像生成も |
|
||||
| TOOLS | 画像の読み取り（VLM）、文字起こし（OCR）、ワイルドカード作成 |
|
||||
| 設定 | 使うAIソフト・モデルの設定 |
|
||||
|
||||
> **ポイント：翻訳と設定はたためる（v0.9.5〜）**
|
||||
> 翻訳と設定の見出しの右端にある矢印（◀／▶）を押すと、その部分が細い帯にたたまれ、空いた幅をChatとTOOLSが使います。
|
||||
> 帯のどこかをクリックすると元に戻ります。たたんだ状態は、次に開いたときも保たれます。
|
||||
|
||||
### 最初に：設定でAIソフトとつなぐ
|
||||
1. **設定** の **バックエンド** で、使うソフト（例：Ollama）を選びます。
|
||||
2. **接続設定** のアドレスを確認します。Ollamaを標準のまま使っている場合は、変更不要です。
|
||||
@@ -96,10 +103,18 @@ Workflow Studio本体の **AI TOOL** タブを開きます。
|
||||
2. 入力欄に文章を書き、**翻訳** を押します。
|
||||
3. 結果を **コピー** して、生成UIやプリセットに貼り付けます。
|
||||
|
||||
> **ポイント**
|
||||
> TOOLSで読み取った文字（10-8のOCRなど）は、TOOLSの **翻訳へ送る** で、この入力欄にそのまま入れられます。
|
||||
|
||||
> **ポイント**
|
||||
> 小さなモデルは、翻訳せずに元の文章をそのまま返してしまうことがあります。
|
||||
> その場合は「モデルが原文をそのまま返しました」と表示されるので、別のモデルを試してください。
|
||||
|
||||
> **ポイント**
|
||||
> 翻訳専用のモデル **PLaMo 3 Translate**（第5章 5-10）を、設定のモデルに選ぶこともできます（v0.9.9〜）。
|
||||
> 日本語・英語・中国語のほか、Free（自由入力）では French・フランス語・français のような言語名で、PLaMoが対応する言語を指定できます。PLaMoが対応していない言語（ポルトガル語など）を翻訳先にすると、エラーになります。
|
||||
> WS LibraryのAタブの翻訳も同じです。
|
||||
|
||||
### Chat
|
||||
入力欄に話しかけて、**Enter** で送信します（**Shift＋Enter** で改行）。
|
||||
会話の流れを覚えているので、「もっと明るい雰囲気にして」のような続けての依頼もできます。
|
||||
@@ -110,6 +125,10 @@ Workflow Studio本体の **AI TOOL** タブを開きます。
|
||||
> チャットで「猫の絵を描いて」と頼むだけで、生成UIに読み込んだワークフローで画像が生成され、チャット内に表示されます。
|
||||
> この機能は、入力欄の横の **画像生成を許可する** のチェックでオン・オフできます。
|
||||
|
||||
> **ポイント**
|
||||
> Chatの上にあるドロップダウンで **スキル** を選ぶと、AIの得意分野を切り替えられます（10-5）。
|
||||
> AIの返事は、見出し・箇条書き・コードの枠を整えて表示されます（v0.9.5〜）。
|
||||
|
||||
### TOOLS － 画像を読み取る（VLM）
|
||||
画像入力に対応したモデル（VLM）を使うと、画像の内容をAIに読み取らせることができます。
|
||||
|
||||
@@ -122,8 +141,16 @@ Workflow Studio本体の **AI TOOL** タブを開きます。
|
||||
| プロンプト作成 | 画像を再現するためのプロンプトを作る |
|
||||
| Create tags（英語表示のまま） | 画像の要素をタグ形式で書き出す |
|
||||
| Create wildcards（英語表示のまま） | カテゴリー名と件数を指定して、ワイルドカードの候補を作る（画像は不要） |
|
||||
| 文字起こし（OCR） | 画像に写っている文字を読み取る（10-8） |
|
||||
|
||||
3. **実行** を押します。結果はコピーして使えます。
|
||||
3. **実行** を押します。
|
||||
4. 結果の下のボタンで、結果を使います（v0.9.5〜）。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| 翻訳へ送る | 翻訳の入力欄に入れる（翻訳がたたまれていれば開く） |
|
||||
| Chatへ送る | Chatの入力欄に追加する |
|
||||
| コピー | クリップボードにコピーする |
|
||||
|
||||
> **例**
|
||||
> 気に入ったイラストの雰囲気をまねしたいとき、その画像から「プロンプト作成」をして、
|
||||
@@ -190,9 +217,227 @@ ComfyUI（とWorkflow Studio）にアクセスできるようになります。
|
||||
> `--listen` は、家庭内LANなど **信頼できるネットワークでだけ** 使ってください。
|
||||
> カフェや公共のWi-Fiでは使わないようにしましょう。
|
||||
|
||||
---
|
||||
|
||||
## 10-5 スキル － Chatの得意分野を切り替える（v0.9.5〜）
|
||||
|
||||
**スキル** は、Chatに「どんな役割で、どんな形で答えるか」を教える指示書です。
|
||||
Chatの上のドロップダウン（最初は「-- スキルなし --」）で選ぶと、そのスキルの指示が毎回の会話に付けて送られます
|
||||
（指示の文章はチャットの画面には表示されません）。
|
||||
|
||||
### 最初から入っているスキル
|
||||
Workflow Studioには、次の6つのスキルが入っています（名前は英語表示のままです）。
|
||||
|
||||
| スキル | 何をするか | 結果の使い方 |
|
||||
| --- | --- | --- |
|
||||
| T2I Prompt Enhancer | 短いアイデアを、生成UIのワークフローのモデルに合った書き方のプロンプトにする | プロンプトの下のボタン |
|
||||
| Image to Prompt | TOOLSに入れた画像を再現するプロンプトを書く（画像を読み取れるモデルが必要） | プロンプトの下のボタン |
|
||||
| Storyboard to Lab | 物語をシーンごとのプロンプトに分け、Labで連続生成できる形にする | **Labで開く** |
|
||||
| Workflow Doctor | ワークフローの説明や、エラーの原因の相談にのる（10-7） | 文章で答える |
|
||||
| SVG Icon Generator | 簡単なアイコンやイラストをSVG（ベクター画像）で描く | プレビューと **Galleryに保存** |
|
||||
| Skill Creator | 質問に答えていくと、新しいスキルを作ってくれる（10-9） | **スキルとして保存** |
|
||||
|
||||
### プロンプトを作って、そのまま生成する（T2I Prompt Enhancer）
|
||||
1. 生成UIに、使うワークフローを読み込みます。スキルはこのワークフローのモデルを見て、プロンプトの書き方を決めます。
|
||||
2. AI TOOLのChatで、ドロップダウンから **T2I Prompt Enhancer** を選びます。
|
||||
3. 「雨の夜の東京、ネオン、傘を持つ猫」のように、日本語で作りたい絵を書いて送信します。
|
||||
4. 返事に、選んだ書き方（`Style:` の行）と、英語のプロンプトの枠が表示されます。その下のボタンで使います。
|
||||
|
||||
| ボタン | 動き |
|
||||
| --- | --- |
|
||||
| GenerateUIに反映 | 生成UIのポジティブ／ネガティブの欄に書き込む |
|
||||
| このプロンプトで生成 | その場で生成し、チャットに画像を表示する |
|
||||
| コピー | ポジティブプロンプトをコピーする |
|
||||
|
||||
モデルに合わせた書き方は、次のとおりです。
|
||||
|
||||
| 生成UIのモデル | プロンプトの書き方 |
|
||||
| --- | --- |
|
||||
| Qwen-Image、FLUX、Z-Image など | 英語の文章（ネガティブは、CFGが1前後なら空にする） |
|
||||
| Illustrious、NoobAI、Pony など（SDXLのアニメ系） | Danbooruのタグ（品質タグから始める） |
|
||||
| 写実系のSDXL、SD1.5 | 短い言葉をカンマで並べる |
|
||||
|
||||
> **ポイント**
|
||||
> 「もっと夜らしく」「猫を2匹に」のように続けて頼むと、直したプロンプト全体を出し直します。
|
||||
|
||||
> **ポイント：今入っているプロンプトは渡さない**
|
||||
> 生成UIに今入っているプロンプトは、ふだんはAIに渡しません。渡すと、小さなモデルが古いネガティブプロンプトなどを新しいプロンプトに混ぜてしまうためです。
|
||||
> 「**今のプロンプトを改善して**」のように、今のプロンプトに触れて頼んだときだけ渡します。
|
||||
|
||||
### ストーリーボードから連続生成する（Storyboard to Lab）
|
||||
1. 生成UIに、文章から画像を作る（T2I）ワークフローを読み込みます。
|
||||
2. Chatで **Storyboard to Lab** を選び、「茶トラの子猫が朝目覚めて、庭で蝶を追いかけ、夕方に縁側で昼寝する。3シーンで」のように頼みます。
|
||||
3. 返事に、シーンの一覧と **Labで開く（3シーン）** ボタンが表示されます。ボタンを押します。
|
||||
4. 生成UIの **Lab**（第5章 5-7）が開き、まだ保存していないプランとして読み込まれます。
|
||||
T2Iモードになり、Batchがシーンの数に、Prompt列がシーンごとのキーフレームになります。KSampler列には、全シーン共通のシード（`Seed …`）が入ります。
|
||||
5. 内容を確かめて **Run** を押します。残したいときは **Plan Save** で保存します。
|
||||
|
||||
> **ポイント**
|
||||
> 登場人物の見た目（髪・服の色など）は、全シーンで同じ文章が入るように書かれ、シードも固定されます。
|
||||
> そのため、シーンが変わっても同じキャラクターに見えやすくなります。
|
||||
|
||||
### 返事の自動チェック
|
||||
プロンプトやSVGを出すスキルでは、Workflow Studioが返事の形を自動で確かめます。
|
||||
プロンプトの枠が崩れている、SVGが閉じていない、といった問題があると、
|
||||
点線の枠で「**自動チェック：…**」と表示し、AIに1回だけ直させます。
|
||||
説明だけを頼んだときなど、プロンプトやSVGを出していない返事はそのままにします。
|
||||
|
||||
### スキルの管理（鉛筆ボタン）
|
||||
ドロップダウンの右の鉛筆ボタンで、スキルの一覧が開きます。
|
||||
|
||||
- **＋ 新規** で新しいスキルを書く、各行の鉛筆ボタンで編集・削除する、**取り込み** で他のスキルを追加する（10-9）
|
||||
- 最初から入っているスキルには **同梱** と表示されます。
|
||||
編集していない同梱スキルは、Workflow Studioを更新すると自動で新しい版になります。
|
||||
編集したスキルは自動では置き換わらず、新しい版があると **↻（更新）** ボタンが表示されます。
|
||||
- スキルのファイルは `ComfyUI/user/default/Workflow-Studio/ai_skills/` に保存されます。
|
||||
|
||||
> **注意**
|
||||
> ↻（更新）で新しい版に置き換えると、そのスキルに加えた編集は消えます。
|
||||
|
||||
---
|
||||
|
||||
## 10-6 生成した画像をチェックして直す（v0.9.5〜）
|
||||
|
||||
チャットから生成した画像が、頼んだとおりになっているかをAIに確かめさせる機能です。
|
||||
足りないところがあれば、その点だけを直したプロンプトで描き直せます。
|
||||
|
||||
### 準備：判定用のモデル
|
||||
チェックには、ふつうのチャット用モデルではなく、Ollamaの **意思決定モデル**（はい／いいえを確率で答えるモデル）を使います。
|
||||
画像を読み取れるもの（作者の環境では `d3:4b`、`clef-flash`、`clef` で動作を確認）を、`ollama pull` で用意します。
|
||||
|
||||
> **ポイント：なぜ専用のモデルを使うのか**
|
||||
> 作者の検証では、チャット用のモデルに画像を見せて「剣を持っているか」と聞くと、持っていないのに「はい」と答えることがありました。
|
||||
> 意思決定モデルは「はい」の確率を数字で返すので、あいまいな答えを見分けられます。
|
||||
|
||||
1. AI TOOLの **設定** にある **CHAT 画像チェック** で、**判定モデル（Ollamaの意思決定モデル）** を選びます（一覧が空なら **更新**）。
|
||||
2. **自動修正の回数** を選びます。最初は **0（ボタンで修正）** がおすすめです。
|
||||
3. **設定を保存** を押します。
|
||||
|
||||
### 使い方
|
||||
1. Chatの入力欄の下にある **画像をチェック** にチェックを入れます。
|
||||
2. チャットから画像を生成します（AIの画像生成、または **このプロンプトで生成**）。
|
||||
3. 画像の下に結果のカードが表示されます。AIが頼んだ内容を3〜8個の確認項目に分け、1つずつ確かめた結果です。
|
||||
|
||||
| 表示 | 意味 |
|
||||
| --- | --- |
|
||||
| ✔（満たしている） | 「はい」の確率が85％以上 |
|
||||
| ?（はっきりしない） | 30〜85％ |
|
||||
| ✘（満たしていない） | 30％未満 |
|
||||
|
||||
4. ✘や?があるときは **足りない点を直して再生成** を押します。満たしていない項目だけをAIに伝え、プロンプトを書き直して描き直します。
|
||||
|
||||
自動修正の回数を1〜3にすると、ボタンを押さなくても、その回数まで自動でくり返します。
|
||||
|
||||
> **ポイント：SVGもチェックできる**
|
||||
> SVG Icon Generatorで描いたSVGも、画像にしてから「頼んだものに見えるか」「部品（くちばし・翼など）が描けているか」をチェックします。
|
||||
> **分かりにくい部分を描き直させる** で直させられます。
|
||||
|
||||
> **注意**
|
||||
> - チェックは完ぺきではありません。とくに「誰が何を身につけているか」の取り違えに弱いため、
|
||||
> 人物が複数いるときは「帽子は女の子ではない」のような逆向きの確認項目を自動で加えて補っています。
|
||||
> - 描き直しの上手さは、チャット用モデルの力しだいです。直らないときは、頼み方を変えるか、モデルを変えてください。
|
||||
> - 判定モデルもGPUのメモリを使います（`d3:4b` で約6GB）。読み込んだあとは、1回のチェックが数秒です。
|
||||
|
||||
---
|
||||
|
||||
## 10-7 ワークフローの不具合を相談する（Workflow Doctor、v0.9.5〜）
|
||||
|
||||
**Workflow Doctor** は、生成UIに読み込んだワークフローの説明や、エラーの原因の相談にのるスキルです。
|
||||
AIの知識だけに頼らず、送信するたびにWorkflow Studioが実際に確かめた次の内容をAIに渡します。
|
||||
|
||||
| 確かめる内容 | 例 |
|
||||
| --- | --- |
|
||||
| インストールされていないノード | ComfyUIが知らないノードの名前 |
|
||||
| ComfyUIの一覧に無いファイル・設定値 | 見つからないモデルやLoRA（似た名前の候補も） |
|
||||
| 線のつながり | つながっていない入力、存在しないノードへの線 |
|
||||
| 直近の実行エラー | ComfyUIの履歴に残ったエラーの内容 |
|
||||
| ComfyUIのログ | ログに出た最新のエラーの部分 |
|
||||
|
||||
### 使い方
|
||||
1. 相談したいワークフローを生成UIに読み込みます（エラーの相談なら、一度実行してエラーを出しておきます）。
|
||||
2. Chatで **Workflow Doctor** を選びます。
|
||||
3. 「実行する前に問題がないか確認して」「さっきのエラーの原因は？」のように聞きます。
|
||||
|
||||
> **例**
|
||||
> LoRAのファイルを別のフォルダに移したワークフローで聞くと、
|
||||
> 「`temp\` のフォルダに指定されたファイルが見つからない。`Illustrious\` に同じ名前のファイルがある」と、直し方まで答えました。
|
||||
|
||||
> **ポイント**
|
||||
> 確かめる内容は、送信のたびに作り直されます。直したあとは、もう一度聞いてみてください。
|
||||
|
||||
---
|
||||
|
||||
## 10-8 画像の文字を読み取る（OCR、v0.9.5〜）
|
||||
|
||||
TOOLSで、画像に写っている文字を読み取れます。読み取った文字は、そのまま翻訳やChatに送れます。
|
||||
|
||||
### 準備：OCR用のモデル
|
||||
OCRには、チャット用とは別の、文字を読むのが得意なモデルを使います。Ollamaで用意してください。
|
||||
作者の環境で、横書きと縦書きの日本語・英語がまじった画像を読ませた結果は次のとおりです。
|
||||
|
||||
| モデル | 時間 | 読み取り |
|
||||
| --- | --- | --- |
|
||||
| LightOnOCR-3（`maternion/LightOnOCR-3:4b`） | 約1秒（位置つきは約6〜8秒） | すべて読めた |
|
||||
| PaddleOCR-VL（マンガ向けの版） | 約4秒 | すべて読めた |
|
||||
| DeepSeek-OCR系 | 約14秒 | すべて読めた（いつも位置つき） |
|
||||
| PaddleOCR-VL 1.6、GLM-OCR | 約11〜15秒 | 縦書きを読み落とした |
|
||||
|
||||
### 使い方
|
||||
1. TOOLSに画像をドロップします。
|
||||
2. 作業で **文字起こし（OCR）** を選びます。
|
||||
3. **OCRモデル** を選びます。名前に「ocr」を含むOllamaのモデルが、向いている順に並びます（一覧が空なら ↻ を押します）。
|
||||
4. **実行** を押します。読み取った文字が、結果の欄に1行ずつ表示されます。
|
||||
|
||||
> **ポイント：文字の位置を表示**
|
||||
> **文字の位置を表示** にチェックを入れると、画像の上に番号つきの枠が重なり、どこの文字かが分かります。
|
||||
> 枠をクリックすると、結果の欄の同じ行が選ばれます。
|
||||
> 位置を返せるのは LightOnOCR と DeepSeek-OCR系だけです。ほかのモデルでは、このチェックは選べません。
|
||||
|
||||
> **ポイント**
|
||||
> 結果が空だったり、同じ行がくり返されたりしたときは、自動で白黒（グレースケール）にして1回だけ読み直します。
|
||||
|
||||
---
|
||||
|
||||
## 10-9 スキルを自作・取り込む（上級、v0.9.5〜）
|
||||
|
||||
### Skill Creatorで作る
|
||||
Chatで **Skill Creator** を選び、「〇〇が得意なスキルを作りたい」と話しかけます。
|
||||
AIが1つずつ質問してくるので答えていくと、スキルの下書きができます。
|
||||
返事の下の **スキルとして保存** を押すと、スキルの一覧に下書きが入った状態で開くので、確かめて保存します。
|
||||
「このスキルの、ここがうまくいかない」と相談して、今あるスキルを直してもらうこともできます。
|
||||
|
||||
### スキルのファイルの中身
|
||||
スキルは、テキストファイル（`.md`）です。先頭の `---` で囲んだ部分に名前と説明を書き、その下にAIへの指示を書きます。
|
||||
先頭の部分の `metadata:` には、次の設定も書けます（書かなくてもかまいません）。
|
||||
|
||||
| 設定 | 意味 |
|
||||
| --- | --- |
|
||||
| `output` | `prompt` か `svg`。返事の形を自動でチェックし、ボタンを付ける |
|
||||
| `tools` | `[]` にすると、AIが自分で画像生成を始めない |
|
||||
| `temperature` | 答えのばらつき（0.3なら安定、0.9なら自由） |
|
||||
| `think` | 考える（Thinking）モードを、このスキルだけオン／オフにする |
|
||||
| `context` | AIに渡す情報。`workflow`（ワークフローの要約）、`current-prompt`（今のプロンプト）、`workflow-detail`／`logs`（Workflow Doctorと同じ確認内容） |
|
||||
|
||||
### ほかのスキルを取り込む
|
||||
スキルの一覧の **取り込み** で、次のファイルを追加できます。
|
||||
|
||||
- `.md` のスキル1つ
|
||||
- `SKILL.md` を含むフォルダをZIPにしたもの（**Agent Skills** という、多くのAIツールで共通の形式）
|
||||
|
||||
フォルダの中のテキストファイル（参考資料など）も一緒に取り込まれ、AIが必要になったときだけ読みに行きます
|
||||
（ツール呼び出しに対応したモデルが必要です）。
|
||||
|
||||
> **注意**
|
||||
> - 取り込むのはテキストファイルだけです。スクリプトなどのプログラムは取り込まず、実行もしません。
|
||||
> - スキルの指示は、そのままAIに渡されます。他の人が作ったスキルは、中身を読んでから使ってください。
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ローカルAI（Ollamaなど）を用意すると、翻訳・チャット・画像の読み取りができる。入力内容は外部に送られない
|
||||
> - AI TOOLタブの設定で、バックエンド・アドレス・モデルを選び、接続テストをする
|
||||
> - プロンプトタブとTaggerタブには、別に歯車ボタンの設定がある
|
||||
> - AIと画像生成はVRAMを取り合う。「モデルをアンロード」や「VRAM調整（Ollama）」で解放する
|
||||
> - 標準では自分のパソコンのAIにだけつながる。`--listen` は信頼できるネットワークだけで
|
||||
> - **スキル** で得意分野を切り替える。T2I Prompt Enhancer でプロンプトを作り、ボタンで生成UIに反映・生成できる
|
||||
> - **画像をチェック** をオンにすると、意思決定モデルが生成画像を確かめ、足りない点だけ直して描き直せる
|
||||
> - Workflow Doctor は、実際に確かめた事実をもとにワークフローの不具合を相談にのる
|
||||
> - TOOLSの **文字起こし（OCR）** で画像の文字を読み取り、翻訳やChatへ送れる
|
||||
|
||||
@@ -54,6 +54,9 @@ Taggerを使うには、次の準備が必要です。
|
||||
|
||||
1. **タグ付け用のモデル**（WD Tagger など）を、ComfyUIの `models/tagger/（モデル名）/` フォルダに置きます。
|
||||
フォルダには、`.onnx` ファイルと `selected_tags.csv` を入れます。
|
||||
**PixAI Tagger v1.0**（v0.8.10〜）も使えます。Hugging Faceのコミュニティ変換版（`Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8`）から、
|
||||
`.onnx` を1つ（GPUなら `model_fp16.onnx` がおすすめ）と、`selected_tags.csv`・`preprocess.json` を同じフォルダに入れると、自動で判別されます。
|
||||
PixAIのモデルを選ぶと、しきい値のスライダーが、PixAIの推奨値（0.17／0.27）に切り替わります。
|
||||
2. **追加のPythonパッケージ** をインストールします。
|
||||
Workflow Studioのフォルダにある `requirements.txt` を使います。
|
||||
|
||||
@@ -69,6 +72,10 @@ Taggerを使うには、次の準備が必要です。
|
||||
> 必ず、**ComfyUIが使っているPython** にインストールしてください。
|
||||
> 分からない場合は、ComfyUI Managerの機能でパッケージをインストールする方法もあります。
|
||||
|
||||
> **注意：GPUで動いているか確認する**
|
||||
> Taggerは、NVIDIAのGPUで動かすと1枚あたり1秒かからないのに対し、CPUで動くとPixAIでは1枚あたり20秒ほどかかります。
|
||||
> とても遅いときは、GPU用のパッケージが正しく入っているか確認してください（第12章 12-5）。
|
||||
|
||||
> **ポイント**
|
||||
> ローカルAI（第10章）の画像読み取り（VLM）と組み合わせて、両方の結果をまとめたタグを作ることもできます。
|
||||
|
||||
@@ -96,6 +103,22 @@ Taggerを使うには、次の準備が必要です。
|
||||
- 編集した画像は、**Save PNG**（ダウンロード）、**Save to Gallery**（ギャラリーに保存）などで保存します。
|
||||
- **Ctrl＋Z** で元に戻す、**Ctrl＋Y** でやり直しができます。
|
||||
|
||||
### Drawの色とショートカット（v0.9.3〜）
|
||||
**Draw** を選ぶと、ツールの2段目に **カラーパレット** が表示されます。
|
||||
|
||||
- **色1／色2**、基本色20色、カスタム色20枠が並びます。色をクリックすると、その色で描けます。
|
||||
- **＋** か空いている枠をクリックすると、今の色をカスタム色に登録します。色の上で右クリックすると、登録・編集・削除ができます（基本色も変えられ、最初の色に戻せます）。
|
||||
登録した色は、ブラウザごとに保存されます。
|
||||
- 右側の **Color** の隣にある **スポイト** ボタンで、キャンバスの色を拾えます（Escで取り消し）。
|
||||
- 描き始めの色は黒（色1）と白（色2）です。
|
||||
|
||||
| キー（Draw中） | 動き |
|
||||
| --- | --- |
|
||||
| D／E | 描く（Draw）／消す（Erase） |
|
||||
| X | 色1と色2を入れ替える |
|
||||
| ［ ／ ］ | ブラシを小さく／大きくする（実際は半角の `[` `]`） |
|
||||
| B | ブラシの選択を開く・閉じる（Escでも閉じる） |
|
||||
|
||||
### インペイント（部分の描き直し）
|
||||
Image Editのいちばん便利な使い方が、**インペイント** です。
|
||||
「顔だけ描き直したい」「手が崩れたところだけ直したい」というときに使います。
|
||||
@@ -119,6 +142,7 @@ Image Editのいちばん便利な使い方が、**インペイント** です
|
||||
- **MiniMax H3**（画像から動画）
|
||||
- **LTX-2.5**（文章から動画、画像から動画、最初と最後の画像を指定して動画）
|
||||
- **Wan2.2**（文章から動画）
|
||||
- **FastH3 OmniRef／Pika-H3-Flex**（MiniMax H3の参照画像から動画。v0.8.14〜）
|
||||
|
||||
対応するワークフローを、ワークフロータブの **Videoに読み込み** で読み込んで使います（第4章 4-6）。
|
||||
|
||||
@@ -131,6 +155,40 @@ Image Editのいちばん便利な使い方が、**インペイント** です
|
||||
|
||||
そのほか、動画の好きな場面を静止画として保存する **Frame** や、動画をGIFアニメに変換する **GIF** の機能もあります。
|
||||
|
||||
> **ポイント**
|
||||
> Planサブタブのブロックの **Prompt** の欄にも、プロンプト翻訳の **🌐** ボタンがあります（第5章 5-10）。
|
||||
> 設定の **Video Plan － Run** をオンにすると、日本語のプロンプトのまま実行して、送る内容だけを英語にできます。
|
||||
|
||||
> **注意**
|
||||
> v0.9.8までは、新しいComfyUIで保存したワークフローを使うと、Planで入力したプロンプト・長さ（Duration）・Turboの設定が反映されず、ワークフローに保存されている値のまま生成されていました（v0.9.9で修正）。
|
||||
> 以前と同じPlanを実行しても、Planの値が反映されるようになったため、結果が変わることがあります。
|
||||
|
||||
### アセット登録と音声変換（v0.8.9〜）
|
||||
- **アセット登録** － 左側の動画ソースパネルのボタンです。パソコン内の動画を、outputフォルダ（`output/video`）へコピーして「Video Assets」グループに登録します。
|
||||
登録すると、Assetサブタブの一覧に出て、Editなどで使えるようになります。
|
||||
すでにoutputフォルダにある動画（Assetで選んだものや生成結果）は、コピーせずにグループへの登録だけを行います。同じ名前があるときは、名前の後ろに連番が付きます。
|
||||
- **音声変換...** － Editサブタブの、クロップのボタンの横にあるボタンです。選んだ動画クリップの音声だけを、FLAC・MP3・Opus・M4A・WAVに変換して `output/video` に保存します。
|
||||
ギャラリーの変換と同じ画面です。音声が入っている動画クリップだけが対象です。
|
||||
- 音声変換のボタンを押すと、次の画面が開きます。**出力形式**（FLACなど）、**サンプルレート**、**チャンネル** を選び、**変換** を押します。
|
||||
「埋め込みのprompt/workflowタグを引き継ぐ」にチェックを入れると、タグを持つ形式（FLACなど）で、元の動画の情報を引き継げます。
|
||||
|
||||
- 動画ソースパネルは少し広くなり、ファイル名が1行で表示され、ボタンも横一列に並ぶようになりました。
|
||||
|
||||
### 参照画像から動画（FastH3 OmniRef／Pika-H3-Flex）（v0.8.14〜）
|
||||
このタイプのワークフローは、最初と最後の画像ではなく、**参照画像** をもとに動画を作ります。
|
||||
そのため、Planサブタブの画像の欄は、次のように名前が変わります。
|
||||
|
||||
| 欄 | このタイプでの意味 |
|
||||
| --- | --- |
|
||||
| Reference Image | 1枚目の参照画像（ふだんの First Image の欄） |
|
||||
| Reference Image 2 (optional) | 2枚目の参照画像（ふだんの Last Image の欄） |
|
||||
|
||||
- プロンプト・長さ（秒）・縦横比などは、ほかの動画ワークフローと同じように使えます。
|
||||
- 画像の欄を **None** や **Clear** で空にしたブロックは、ワークフローに参照画像が設定されていても、その画像を使わずに生成します。
|
||||
「＋ Block」で追加したブロックは画像が空の状態で始まるので、参照画像が必要なブロックでは、それぞれ画像を指定してください。
|
||||
- 画像の選び方で **Use previous block's last frame**（前のブロックの最後のフレームを使う）を選ぶと、前のブロックで実際に生成された動画の最後のフレームが、そのブロックの1枚目の参照画像になります。
|
||||
- Assetサブタブのボタンも「Reference Imageに設定」「Reference Image 2に設定」に変わります。
|
||||
|
||||
> **注意**
|
||||
> 動画生成は、画像生成よりもはるかに多くのGPUメモリと時間を使います。
|
||||
> 使うモデルの必要スペックを確認してから試しましょう。
|
||||
@@ -155,6 +213,7 @@ Image Editのいちばん便利な使い方が、**インペイント** です
|
||||
| CivitAI APIキー | Civitaiの情報取得用（省略可） | 第6章 |
|
||||
| Eagle 連携 (オプション) | 生成画像をEagleに自動保存 | 第8章 |
|
||||
| VRAM調整（Ollama） | 生成前にAIモデルをメモリから降ろす | 第10章 |
|
||||
| プロンプト翻訳（自国語 → 英語） | 日本語のプロンプトを🌐ボタンや実行時の自動翻訳で英語にする | 第5章 5-10 |
|
||||
| データ管理 | 設定やデータのバックアップ・復元 | 下記 |
|
||||
|
||||
### デフォルトワークフロー
|
||||
@@ -198,11 +257,12 @@ Workflow Studioとは別にインストールするソフトですが、両方
|
||||
両方をインストールすると、次のような連携ができます。
|
||||
|
||||
- **Comic Creatorの中でWorkflow Studioのギャラリーを使う**
|
||||
Comic Creatorの中にWorkflow Studioのギャラリーが表示され、生成した画像を選んで **Send CC** ボタンでコマに送れます。
|
||||
Comic Creatorの中にWorkflow Studioのギャラリーが表示され、生成した画像を選んで **Send CC Layout**（レイアウトタブのコマ／オーバーレイへ）または **Send CC Image**（Imageタブへ）ボタンで送れます。
|
||||
（Send CCボタンは、Comic Creatorの中で開いたギャラリーにだけ表示されます）
|
||||
- **Comic Creatorから画像を生成する**
|
||||
Comic Creatorの画面から、Workflow Studio（生成UI）を通して、コマに入れる画像を生成したり（T2I）、
|
||||
描いたラフや配置した画像をもとに描き直したり（I2I・インペイント）できます。
|
||||
Comic CreatorのNanobanana+タブ（ローカルモード）からは、複数の参照画像を使う画像編集のワークフロー（Flux.3 Klein、Qwen Image 2.1 など）も実行できます（v0.8.12〜）。
|
||||
|
||||
「Workflow Studioで画像を作る → Comic Creatorでマンガにまとめる」という流れで、
|
||||
画像生成からマンガの完成までを、ComfyUIの中だけで行えます。
|
||||
@@ -224,6 +284,74 @@ Workflow Studioとは別にインストールするソフトですが、両方
|
||||
|
||||
---
|
||||
|
||||
## 11-7 TagSpace － タグとプロンプトを「意味の地図」から選ぶ（v0.9.0〜）
|
||||
|
||||
**TagSpace** タブは、Workflow Studioの中にあるプロンプトやタグ、生成した画像を、
|
||||
**意味の近さ** にしたがって3Dの空間に並べて表示する機能です。
|
||||
近くにあるものほど意味が近いので、「この雰囲気に近いタグは何だろう」と眺めながら、タグを選んでプロンプトを作れます。
|
||||
|
||||
### 空間に並ぶもの
|
||||
| 種類 | 中身 | 表示 |
|
||||
| --- | --- | --- |
|
||||
| カード | ギャラリーの **ImagePrompt**（第8章 8-4）の見本画像と、そのプロンプト | サムネイル |
|
||||
| タグ | ImagePromptのプロンプト・生成画像に埋め込まれたプロンプト・ワイルドカード・Tagger DB・Promptライブラリの文を、カンマで区切った語 | 出典ごとに色分けした点 |
|
||||
| Gallery点 | ギャラリーの生成画像の一部（それぞれ、自分のプロンプトのタグを持つ） | 薄い点 |
|
||||
|
||||
### 準備
|
||||
ギャラリーのセマンティック検索（第8章 8-6）と同じく、**Ollama** と **EmbeddingGemma 2** が必要です。
|
||||
ギャラリーのセマンティック検索の索引を、Gemini Embedding 2で作っている場合は、先にEmbeddingGemma 2で作り直してください。
|
||||
|
||||
### 空間を作る
|
||||
ツールバーで設定を選び、**作成 / 更新** を押します。
|
||||
|
||||
| 項目 | 内容 |
|
||||
| --- | --- |
|
||||
| タグ数 | 空間に入れるタグの数。出典ごとに、よく使われている順に交互に選ぶ |
|
||||
| 全タグ | オンにすると、タグ数の代わりに候補のタグをすべて使う（上限2万件） |
|
||||
| 最低出現回数 | この回数未満しか使われていないタグを除く。2にすると、1回だけの表記ゆれなどが消える |
|
||||
| ワイルドカード | ワイルドカードをタグの材料に含めるかどうか |
|
||||
| Gallery点数 | 表示する生成画像の数（0で表示なし） |
|
||||
| Tagger補完 | プロンプトを取り出せない生成画像に、Taggerのモデルでタグを付ける（OFF／モデル名。「Gallery点のみ」か「プロンプトの無い全画像」を選ぶ） |
|
||||
|
||||
- 1回目は、画像のプロンプトの読み取りやタグの準備に数分かかります。2回目からは、前回の結果を使うので数秒〜数十秒で終わります。
|
||||
- 状態表示に、カード・タグ・Gallery点の数が出ます。候補の一部だけを使ったときは「（候補 N）」と出ます。
|
||||
|
||||
> **ポイント**
|
||||
> タグの種類の大半は、ワイルドカードにしか出てこない語です。
|
||||
> 「ワイルドカード」をオフにして「全タグ」をオンにすると、自分が実際に使ってきたタグだけを全部並べられます（検証環境では約1万種）。
|
||||
|
||||
> **注意**
|
||||
> Tagger補完は、CPUで動くと1枚あたり0.5秒ほどかかります。「プロンプトの無い全画像」を選ぶ前に、TaggerがGPUで動いているか確認してください（第12章 12-5）。
|
||||
|
||||
### 見る・探す
|
||||
- **ドラッグ** で回転、**右ドラッグ** で移動、**ホイール** でズームします。
|
||||
- 点にカーソルを合わせると、名前・サムネイル・プロンプトや出典が表示されます。
|
||||
- ツールバーのチェックボックスで、カード・サムネイル・タグ・Galleryの表示や、**発光**（夜空に光る星のような表示）・**またたき** を切り替えます。
|
||||
出典ごとのチェックボックスで、タグを出典ごとに表示・非表示にできます。
|
||||
- 右上の検索欄に言葉や文章を入れて **Enter** を押すと（日本語も可）、その意味の位置に白い印が付き、近いカードやタグが一覧に出ます。
|
||||
|
||||
### タグを選んでプロンプトを作る
|
||||
1. カードやタグをクリックすると選択され、右の **近くの要素** に、意味の近いカード・タグが並びます。
|
||||
（Gallery点をクリックすると、似ている生成画像も並びます）
|
||||
2. プロンプトに入れたいものを **ダブルクリック** するか、**選択中** の **追加**、または一覧の **＋** を押します。
|
||||
カードはプロンプト全体、タグはタグそのもの、Gallery点はその画像のタグが、右端の **プロンプト** 欄に入ります。
|
||||
3. **GenerateUIへ追記（Positive）** で、生成UIに読み込んだワークフローのポジティブプロンプトに追記します（先に生成UIでワークフローを読み込んでおきます）。
|
||||
**コピー** でクリップボードにコピーすることもできます。
|
||||
|
||||
### カードや生成画像のタグで探す
|
||||
カードやGallery点を選ぶと、**選択中** に、そのプロンプトのタグがボタンで並びます。
|
||||
タグをクリックして選び（もう一度で解除）、**選択したタグで空間検索** を押すと、選んだタグで空間を検索します。
|
||||
「この画像の、髪型と服装だけに近いものを探したい」といったときに便利です。
|
||||
タグを選んでいる間は、**選択中** の **追加** ボタンが **選択したタグを追加（N）** に変わり、選んだタグだけをプロンプト欄に追加できます。
|
||||
タグを選んでいなければ、これまでどおり全体（カードはプロンプト全文、Gallery点はすべてのタグ）が追加されます。
|
||||
|
||||
Gallery点を選んでいるときは、**選択中** に **Galleryで開く** ボタンも表示されます。
|
||||
押すとギャラリータブに切り替わり、その画像のフォルダが選ばれ、画像一覧でその画像が選択された状態になります（右の詳細も開きます）。
|
||||
画像が一覧に必ず出るよう、ギャラリーの検索・お気に入り・タグ・グループの絞り込みは解除されます。
|
||||
カードのプロンプト全文は、**プロンプト全文** を開くと読めます。
|
||||
|
||||
---
|
||||
|
||||
> **この章のまとめ**
|
||||
> - ノードタブでインストール済みのノードを一覧・整理でき、WS Libraryで使える
|
||||
> - Taggerは画像からタグを自動生成する。モデルと追加パッケージの準備が必要
|
||||
@@ -231,3 +359,4 @@ Workflow Studioとは別にインストールするソフトですが、両方
|
||||
> - Videoタブで、対応する動画生成モデルを使った動画作りと簡単な編集ができる
|
||||
> - 設定タブの「デフォルトCheckpoint」と「データ管理（バックアップ）」は特に便利
|
||||
> - 同じ作者の Comic Creator と組み合わせると、生成した画像でマンガを作れる
|
||||
> - TagSpaceでは、手持ちのタグ・プロンプト・生成画像を意味の近さで3Dに並べ、眺めながらタグを選んでプロンプトを作れる
|
||||
|
||||
@@ -72,6 +72,7 @@ Workflow Studioを更新（アップデート）したあとは、必ずComfyUI
|
||||
### 設定を変えたのに、結果が変わらない
|
||||
いちばん多い原因は、**Apply の押し忘れ** です。
|
||||
生成UIでは、モデルやSettings（ステップ数・サイズなど）の値を変えたあと、**Apply** を押さないとワークフローに反映されません（第5章 5-3）。
|
||||
ただし、Settingsタブの右端のスイッチの列は、クリックした時点で反映されます（第5章 5-4）。
|
||||
右側の **Raw JSON** を見ると、実際に反映されている値を確認できます。
|
||||
|
||||
### 読み込んだワークフローとは違う内容で生成される
|
||||
@@ -82,10 +83,20 @@ Batchサブタブを開き、Batch Queueの列のチェックをすべてオフ
|
||||
### 思っていたのと違うモデルで生成される
|
||||
ワークフローに保存されているモデルが手元にないと、別のモデルに自動で差し替えられることがあります。
|
||||
その場合は「Checkpoint「…」が見つからず、無警告で別のモデルに差し替えられました」と表示されます。
|
||||
（v0.8.7〜は、フォルダ名だけが違う同名のモデルが1つだけ手元にあれば、それに自動で差し替わります）
|
||||
生成UIの **Model** サブタブで、使いたいモデルを選び直して **Apply** を押してください。
|
||||
|
||||
いつも同じモデルを使いたい場合は、設定タブの **デフォルトCheckpoint** を使うと便利です（第11章 11-5）。
|
||||
|
||||
### プロンプト翻訳の🌐ボタンが出ない・翻訳できない
|
||||
| 症状 | 原因 | 対処 |
|
||||
| --- | --- | --- |
|
||||
| 🌐ボタンが表示されない | プロンプト翻訳がオフになっている | 設定タブの **プロンプト翻訳（自国語 → 英語）** で **生成UIでプロンプト翻訳を使う** にチェックを入れる（第5章 5-10） |
|
||||
| 「翻訳エラー」と出る | Ollamaが起動していない、または翻訳モデルがOllamaに無い | Ollamaを起動する。設定の **翻訳モデル** に **（未取得）** と出ていれば **ダウンロード** を押す |
|
||||
| 押しても何も変わらない（「翻訳する部分がありません」） | 英語（英数字）だけのプロンプトになっている | 翻訳されるのは英数字以外の文字を含む部分だけ。英語の部分はそのまま残る |
|
||||
| 自動翻訳がオンのとき、生成が始まらない | 翻訳に失敗したので、日本語のまま生成しないよう止めた | 上の「翻訳エラー」と同じ対処をする。LabとVideo Planでは、理由が進行状況の欄に出る |
|
||||
| 訳がおかしい | 小さいモデル（0.8B）は、文章で意味がずれることがある | 設定の **翻訳モデル** で2B以上を選ぶ |
|
||||
|
||||
### 同じ設定なのに、同じ画像にならない
|
||||
生成パネルのシードのモードが **Random** になっていませんか？
|
||||
同じ画像を作るには、シードを **Workflow**（ワークフローのシードをそのまま使う）か **Fixed**（Seed欄の値で固定）にする必要があります。
|
||||
@@ -134,6 +145,21 @@ Ollamaの場合は `ollama pull （モデル名）` でモデルをダウンロ
|
||||
安全のため、標準では同じパソコンの中のAIソフトにしかつながりません。
|
||||
環境変数 `WFS_ALLOWED_BACKEND_HOSTS` の設定が必要です（第10章 10-4）。
|
||||
|
||||
### 画像チェックの判定モデルや、OCRモデルが一覧に出てこない
|
||||
- 判定モデルの一覧には、Ollamaの **意思決定モデル** のうち、画像を読み取れるものだけが出ます。
|
||||
ふつうのチャット用モデルは出ません。Ollamaを新しい版に更新し、意思決定モデルを `ollama pull` してから **更新** を押します（第10章 10-6）。
|
||||
- OCRモデルの一覧には、名前に「ocr」を含むOllamaのモデルだけが出ます。取得後に ↻ を押します（第10章 10-8）。
|
||||
- どちらも、AI TOOLのバックエンドがOllamaならそのアドレス、そうでなければ `http://localhost:11434` のOllamaを見に行きます。
|
||||
|
||||
### 画像チェックで、何度直しても満たされない
|
||||
- 描き直しの上手さは、チャット用モデルの力しだいです。頼み方を短く具体的にするか、別のモデルを試してください。
|
||||
- チェック自体もまちがえることがあります。画像を自分の目でも確かめてください。
|
||||
- 自動修正の回数を上げすぎると、生成の時間とGPUのメモリを使い続けます。最初は「0（ボタンで修正）」がおすすめです。
|
||||
|
||||
### AIがスキルの資料を読まない・チャットから画像を生成しない
|
||||
取り込んだスキルの参考資料を読む機能と、チャットからの画像生成は、
|
||||
**ツール呼び出し（Tool Calling）に対応したモデル** でないと動きません。対応しているモデルに切り替えてください。
|
||||
|
||||
---
|
||||
|
||||
## 12-5 そのほかの機能のトラブル
|
||||
@@ -142,6 +168,11 @@ Ollamaの場合は `ollama pull （モデル名）` でモデルをダウンロ
|
||||
- `ComfyUI/models/tagger/（モデル名）/` フォルダに、`.onnx` ファイル（と `selected_tags.csv`）が入っているか確認します。
|
||||
- 追加のPythonパッケージ（onnxruntime）が、**ComfyUIが使っているPython** にインストールされているか確認します（第11章 11-2）。
|
||||
|
||||
### Taggerがとても遅い（GPUが使われない）
|
||||
ComfyUIのPythonに、CPU版の `onnxruntime` とGPU版の `onnxruntime-gpu` が両方入っていると、GPUが使われなくなります。
|
||||
（ほかのカスタムノードのインストールで、CPU版が入ることがあります）
|
||||
ComfyUIを停止し、ComfyUIのPythonで `pip uninstall -y onnxruntime onnxruntime-gpu` のあと `pip install onnxruntime-gpu` を実行して、ComfyUIを再起動してください（DirectMLは非対応）。
|
||||
|
||||
### PSD関連の機能がエラーになる
|
||||
追加のPythonパッケージ（psd-tools）が必要です。
|
||||
エラーメッセージに表示されるインストール用のコマンドを、ComfyUIが使っているPythonで実行してから、ComfyUIを再起動してください。
|
||||
@@ -149,9 +180,31 @@ Ollamaの場合は `ollama pull （モデル名）` でモデルをダウンロ
|
||||
### セマンティック検索（Gallery）がうまく動かない
|
||||
| 症状 | 原因 | 対処 |
|
||||
| --- | --- | --- |
|
||||
| GPUが検出されない（設定に「GPU未検出」と出る、実行がCPUになる） | ComfyUIのPythonに入っているonnxruntimeがCPU版 | ComfyUIを停止し、ComfyUIのPythonで `pip uninstall -y onnxruntime onnxruntime-gpu` のあと `pip install onnxruntime-gpu` を実行して、ComfyUIを再起動する（DirectMLは非対応） |
|
||||
| 「Ollamaに接続できません」と出る | Ollamaが起動していない、またはOllama URLが違う | Ollamaを起動する（インデックス作成中と検索時に必要）。歯車ボタンのモデル設定タブでOllama URLを確認する |
|
||||
| 「Ollamaのモデルが未取得です」と出る | EmbeddingGemma 2をダウンロードしていない | `ollama pull embeddinggemma-2:740m`（または `embeddinggemma-2:440m`）を実行する |
|
||||
| インデックス作成が「connection was forcibly closed」などでときどき失敗する | セキュリティソフト（AdGuardなど）がOllamaの通信に割り込んでいる | Ollamaをフィルタの対象から外す。失敗した画像は、次回の「新規・変更された画像のみ」で作り直される |
|
||||
| Geminiで「GEMINI_API_KEYが未設定」と出る・キーが拒否される | `.env` にキーが無い、または行頭に `#` が付いている | プラグインの `.env`（`.env.example` をコピー）に `GEMINI_API_KEY=キー` と書き、ComfyUIを再起動する。`.env` は起動時にしか読み込まれない |
|
||||
| 検索やSimilarの結果が出ない | 選んだフォルダにインデックス済みの画像が無い |歯車ボタンでインデックスを作る。緑の点があるフォルダを選ぶ。モデルや精度を変えたときは「全件再作成」 |
|
||||
| 検索やSimilarの結果が出ない | 選んだフォルダにインデックス済みの画像が無い | 歯車ボタンでインデックスを作る。緑の点があるフォルダを選ぶ。モデルやベクトル次元を変えたときは「全件再作成」 |
|
||||
|
||||
### TagSpaceの空間が作れない
|
||||
| 症状 | 原因 | 対処 |
|
||||
| --- | --- | --- |
|
||||
| 「OllamaのEmbeddingGemma 2が必要です」と出る | Ollamaが起動していない、またはモデルが無い | セマンティック検索と同じ準備をする（上の表） |
|
||||
| 「Galleryのセマンティック索引は … で作られています」と出る | ギャラリーの索引をGeminiで作っている | ギャラリーのセマンティック設定でEmbeddingGemma 2を選び、「全件再作成」する |
|
||||
| 作成にとても時間がかかる | 1回目は、画像のプロンプトの読み取りと、新しいタグの準備に時間がかかる | 2回目からは速くなる。Tagger補完を使うときは、TaggerがGPUで動いているか確認する（上の「Taggerがとても遅い」） |
|
||||
|
||||
### パートナーAPI（Nano Banana）が「認証に失敗しました」と出る
|
||||
`.env` にComfy APIキーが無いのが原因です。プラグインの `.env` に `COMFY_API_KEY=キー`（platform.comfy.orgで発行）を書き、ComfyUIを再起動してください（第5章 5-8）。
|
||||
`.env` は起動時にしか読み込まれません。
|
||||
|
||||
### ComfyUIが固まる（どのタブも反応しなくなる）
|
||||
Windowsで、プレビュー画像の多いモデルタブの一覧を切り替えたあとなどに、ComfyUIが反応しなくなる（生成もできなくなる）場合があります。
|
||||
設定タブの **Windowsの安定性（イベントループ）** で、**WindowsでSelectorイベントループを使用する** をオンにして、ComfyUIを再起動してください。
|
||||
|
||||
### ギャラリーの画像の一部が、別の画像のように壊れて見える
|
||||
上の **Selectorイベントループ** をオンにしているとき、v0.9.1までは、ギャラリーのプレビューや Download で受け取った画像の一部が壊れることがありました
|
||||
（出力フォルダのファイル自体は正常です）。v0.9.2で直っています。
|
||||
Workflow Studioを更新したあと、壊れた画像がブラウザに残っていることがあるので、**Ctrl＋F5** で再読み込みしてください。
|
||||
|
||||
### ギャラリーに画像が出てこない
|
||||
- ギャラリーの **Refresh** を押します。
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
### 日本語
|
||||
| 用語 | 意味 | 本書 |
|
||||
| --- | --- | --- |
|
||||
| 意思決定モデル | 質問に「はい」の確率などを数字で答えるAIモデル。画像チェックの判定に使う | 第10章 |
|
||||
| インペイント | 画像の一部（マスクした部分）だけを描き直すこと | 第11章 |
|
||||
| カスタムノード | ComfyUIに後から追加するノード・拡張機能。Workflow Studioもその1つ | 第1章 |
|
||||
| キャンバス | ComfyUIのノード画面で、ノードを並べる作業スペース | 第3章 |
|
||||
@@ -15,6 +16,7 @@
|
||||
| サンプラー | 画像を描くときのアルゴリズム（euler など） | 第5章 |
|
||||
| シード | 画像のもとになる乱数の種。同じなら、ほぼ同じ画像になる | 第5章 |
|
||||
| セマンティック検索 | 文章や画像の「意味」で画像を探す検索。ギャラリーのSボタンで使う | 第8章 |
|
||||
| スキル | AI TOOLのChatに、役割や答え方を教える指示書。選ぶと得意分野が切り替わる | 第10章 |
|
||||
| スケジューラー | ステップごとの描き込み量の配分のしかた | 第5章 |
|
||||
| スタイル | プロンプトに付け足す雰囲気のテンプレート | 第7章 |
|
||||
| ステップ数 | 描き込みをくり返す回数 | 第5章 |
|
||||
@@ -50,8 +52,11 @@
|
||||
| I2I（Image to Image） | 画像をもとに、新しい画像を作ること | 第5章 |
|
||||
| JSON | ワークフローの保存に使われるデータ形式 | 第1章 |
|
||||
| LoRA | 絵柄やキャラクターなどを追加で覚えさせる小さなモデル | 第6章 |
|
||||
| OCR | 画像に写っている文字を読み取ること | 第10章 |
|
||||
| Ollama | ローカルAIを手軽に動かせるソフト | 第10章 |
|
||||
| PLaMo 3 Translate | Preferred Networks（PFN）の翻訳専用AIモデル。プロンプト翻訳で使う | 第5章 |
|
||||
| T2I（Text to Image） | 文章（プロンプト）から画像を作ること | 第5章 |
|
||||
| TagSpace | タグ・プロンプト・生成画像を意味の近さで3Dの空間に並べて表示するタブ | 第11章 |
|
||||
| VAE | 画像の色合いや細部を仕上げるモデル | 第6章 |
|
||||
| VLM | 画像を読み取れるAIモデル | 第10章 |
|
||||
| VRAM | GPU（グラフィックボード）のメモリ | 第10章 |
|
||||
@@ -99,6 +104,10 @@
|
||||
| マウスホイール | 拡大・縮小 |
|
||||
| Space＋ドラッグ | 表示位置を移動する |
|
||||
| Delete | 選んでいるレイヤーを削除する（レイヤーが2つ以上あるとき） |
|
||||
| D／E（Draw中） | 描く／消す |
|
||||
| X（Draw中） | 色1と色2を入れ替える |
|
||||
| ［／］（Draw中） | ブラシを小さく／大きくする（実際は半角の `[` `]`） |
|
||||
| B（Draw中） | ブラシの選択を開く・閉じる |
|
||||
|
||||
---
|
||||
|
||||
@@ -119,6 +128,7 @@
|
||||
| --- | --- | --- |
|
||||
| Civitai（https://civitai.com/） | モデルの入手・情報取得 | 第6章 |
|
||||
| Ollama（https://ollama.com/） | ローカルAI | 第10章 |
|
||||
| PLaMo 3 Translate（https://huggingface.co/pfnet/plamo-3-translate-2b-GGUF） | プロンプト翻訳の翻訳モデル | 第5章 |
|
||||
| LM Studio（https://lmstudio.ai/） | ローカルAI | 第10章 |
|
||||
| Lemonade Server（https://lemonade-server.ai/） | ローカルAI | 第10章 |
|
||||
| Eagle（https://eagle.cool/） | 画像管理ソフト | 第8章 |
|
||||
|
||||
@@ -2,12 +2,12 @@
|
||||
|
||||
Don't Be Afraid of Nodes - Generate Images More Easily and Keep Them Organized
|
||||
|
||||
- Covers: Workflow Studio v0.8.3
|
||||
- Covers: Workflow Studio v0.9.9
|
||||
- Author: ketle-man
|
||||
- (c) 2026 ketle-man
|
||||
|
||||
This is the **text-only edition** (without screenshots) of the Kindle book.
|
||||
Chapter files are in the `chapters/` folder; the whole book in one file is `Workflow_Studio_Manual_v0.8.3_en.md`.
|
||||
Chapter files are in the `chapters/` folder; the whole book in one file is `Workflow_Studio_Manual_v0.9.9_en.md`.
|
||||
Screen examples are omitted, so please use the book together with Workflow Studio's Help tab.
|
||||
|
||||
## Contents
|
||||
@@ -29,4 +29,4 @@ Screen examples are omitted, so please use the book together with Workflow Studi
|
||||
- [13_Appendix](chapters/13_Appendix.md)
|
||||
- [14_Afterword](chapters/14_Afterword.md)
|
||||
|
||||
Whole book: [Workflow_Studio_Manual_v0.8.3_en.md](Workflow_Studio_Manual_v0.8.3_en.md)
|
||||
Whole book: [Workflow_Studio_Manual_v0.9.9_en.md](Workflow_Studio_Manual_v0.9.9_en.md)
|
||||
|
||||
Binary file not shown.
+692
-28
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -83,6 +83,7 @@ Together with the supporting features, it makes moving between the two screens s
|
||||
- **Image Edit** - simple layer-based editing, and redrawing parts with a mask (inpainting)
|
||||
- **Local AI integration** - translating prompts from Japanese, creating prompts and tags from images
|
||||
- **Tagger and Video** - automatic tag generation from images, and video generation with simple editing
|
||||
- **TagSpace** - lays out your own prompts, tags and generated images in a 3D space by closeness of meaning, so you can browse it and pick tags
|
||||
|
||||
---
|
||||
|
||||
@@ -128,7 +129,7 @@ It is written so that you can follow it if you know basic computer operations (c
|
||||
|
||||
## Version and environment covered
|
||||
|
||||
- **Workflow Studio**: v0.8.3
|
||||
- **Workflow Studio**: v0.9.9
|
||||
- **ComfyUI**: the latest version at the time of writing
|
||||
- **OS**: Windows (screen examples are from Windows)
|
||||
|
||||
|
||||
@@ -19,7 +19,8 @@ The basic features of Workflow Studio work with no extra preparation as long as
|
||||
### Later, as needed
|
||||
The following are needed only when you use particular features. You can ignore them at first.
|
||||
|
||||
- **Ollama, LM Studio and similar** - AI support features such as prompt translation and AI chat (Chapter 10)
|
||||
- **Ollama, LM Studio and similar** - AI support features such as prompt translation and AI chat (Chapter 10).
|
||||
**Ollama** is also needed for local semantic search in the Gallery (Chapter 8) and for the TagSpace tab (Chapter 11)
|
||||
- **Eagle** - a feature that automatically saves generated images to the image-management software Eagle (Chapter 8)
|
||||
- **Extra Python packages** - Tagger (tag generation from images), PSD export and the like (Chapter 11)
|
||||
|
||||
|
||||
@@ -24,9 +24,10 @@ It is enough to learn the tabs marked with a star under "Often used" first.
|
||||
| Image Edit | Simple image editing, making masks | | Chapter 11 |
|
||||
| Video | Video generation and video editing | | Chapter 11 |
|
||||
| Tagger | Automatically generating tags (prompts) from images | | Chapter 11 |
|
||||
| TagSpace | Showing tags, prompts and generated images in 3D by closeness of meaning, and picking tags | | Chapter 11 |
|
||||
| Settings | Language, theme, folders and other settings | | This chapter, Chapter 11 |
|
||||
| Help | Feature list, shortcuts, troubleshooting | | This chapter |
|
||||
| AI TOOL | Translation, chat and image analysis with local AI | | Chapter 10 |
|
||||
| AI TOOL | Translation, chat, image analysis and text reading (OCR) with local AI | | Chapter 10 |
|
||||
|
||||
### Think of the tabs in three groups: "organize", "create" and "support"
|
||||
If the number of tabs makes you lose your way, it helps to think of them in these three groups.
|
||||
@@ -35,7 +36,7 @@ If the number of tabs makes you lose your way, it helps to think of them in thes
|
||||
Tabs that list your materials so they are easy to find.
|
||||
- **Create** - GenerateUI / Image Edit / Video
|
||||
Tabs for actually making images and videos, or touching them up.
|
||||
- **Support** - AI TOOL / Tagger / Settings / Help
|
||||
- **Support** - AI TOOL / Tagger / TagSpace / Settings / Help
|
||||
Tabs that help with work in other tabs, or hold overall settings.
|
||||
|
||||
> **Tip**
|
||||
|
||||
@@ -257,7 +257,8 @@ Hovering the mouse over a badge shows the reason for the judgment.
|
||||
> Even with a warning or cross, generation itself is often possible, so treat it as a rough indication of "hard to finish using GenerateUI alone".
|
||||
|
||||
> **Tip**
|
||||
> The first check after opening the Workflow tab can take about 10 seconds, because it fetches node information from ComfyUI.
|
||||
> The check fetches from ComfyUI only the information of the nodes that workflow uses (v0.8.8 and later).
|
||||
> It used to take tens of seconds the first time, but now the result appears almost instantly.
|
||||
> From the second time on, results appear immediately.
|
||||
|
||||
---
|
||||
|
||||
@@ -72,6 +72,9 @@ To the right of Input, Model and Settings, the contents of the workflow are show
|
||||
When you change values in the form, they are reflected here too.
|
||||
Until you are used to it, treat this as "view only".
|
||||
|
||||
On the Input subtab, the **>>** button at the top of the Raw JSON folds it into a thin strip at the right edge (v0.8.12 and later),
|
||||
which gives the prompt and image areas more room. Press it again to unfold it; the folded state is remembered.
|
||||
|
||||
---
|
||||
|
||||
## 5-3 Generating your first image
|
||||
@@ -79,7 +82,7 @@ Until you are used to it, treat this as "view only".
|
||||
Now, to generate. At minimum, you check and set these three things.
|
||||
|
||||
1. Which model to draw with (Model)
|
||||
2. What to draw (Input -> Prompt)
|
||||
2. What to draw (Input)
|
||||
3. Generate (Generate)
|
||||
|
||||
### Step 1: Choose a model (Model)
|
||||
@@ -95,6 +98,9 @@ Now, to generate. At minimum, you check and set these three things.
|
||||
> - When loading, a message says the workflow's Checkpoint "..." was not found and was replaced with another model without warning
|
||||
> (in this case it has been replaced automatically with the first model in the list you have)
|
||||
>
|
||||
> From v0.8.7, if you have exactly one model with the same file name that differs only in its folder, it is replaced with that model instead of the first one.
|
||||
> (Official templates often save model names without a folder, and the old behavior could swap in an unrelated model.)
|
||||
>
|
||||
> In either case, choose the model you want to use from the list and press Apply.
|
||||
> If choosing again each time is a nuisance, the **Default Checkpoint** in the Settings tab (Chapter 11) is handy.
|
||||
|
||||
@@ -109,7 +115,9 @@ Now, to generate. At minimum, you check and set these three things.
|
||||
> Items that are not green have no effect on this workflow even if you change them.
|
||||
|
||||
### Step 2: Write a prompt (Input)
|
||||
1. Open the **Input** subtab and choose **Prompt** inside it.
|
||||
1. Open the **Input** subtab.
|
||||
The prompt area (Prompt) is on the left and the input image area (Image) is on the right (v0.8.12 and later; before that you switched between Prompt and Image).
|
||||
When the screen is narrow, the Image area moves below the prompts.
|
||||
2. In the **Positive Prompt** field, enter in English what you want drawn.
|
||||
|
||||
```
|
||||
@@ -160,12 +168,13 @@ If you save it with a name, it is added as a new workflow in the Workflow tab.
|
||||
|
||||
Once the first image is out, next look at the **Settings** subtab, where settings that decide how the result turns out are found.
|
||||
|
||||
The Settings tab is divided into two blocks, left and right.
|
||||
The Settings tab is divided into three columns.
|
||||
|
||||
- **KSampler (left)** - settings for how the image is drawn
|
||||
- **Latent Image (right)** - settings for the size and number of images
|
||||
- **Latent Image (middle)** - settings for the size and number of images
|
||||
- **Switches (right, v0.9.6+)** - the on/off settings inside the workflow
|
||||
|
||||
Here too, don't forget to press **Apply** on each block after changing values.
|
||||
Here too, for KSampler and Latent Image, don't forget to press **Apply** on each block after changing values (the Switches column needs no Apply).
|
||||
|
||||
### KSampler items
|
||||
| Item | Meaning | Rule of thumb for beginners |
|
||||
@@ -192,6 +201,23 @@ Here too, don't forget to press **Apply** on each block after changing values.
|
||||
> If you stray far from the size the model learned, people tend to come out doubled or the composition falls apart.
|
||||
> Even when you want portrait or landscape, adjust so the total pixel count stays about the same (for example 832x1216 for SDXL).
|
||||
|
||||
### Switches items (v0.9.6+)
|
||||
The "on/off" settings inside the workflow are listed with their node name and item name.
|
||||
Clicking a checkbox switches it right away. **You don't need to press Apply**. The new value also shows up in the Raw JSON on the right at once.
|
||||
|
||||
Two kinds of items are listed:
|
||||
|
||||
- Items whose value is true/false - for example the switch that turns the prompt enhancer (PE) on and off, "Boolean" nodes, and TextGenerate's thinking
|
||||
- On/off items with only two choices - enable/disable, on/off, yes/no and so on. For example KSamplerAdvanced's add_noise and DWPreprocessor's detect_hand (whether to detect hands)
|
||||
|
||||
The right end of each row shows the current value (true/false, enable/disable and so on).
|
||||
Hover over a row to see the node's number and type.
|
||||
|
||||
> **Tip**
|
||||
> Items wired from another node are not listed. Instead, the source node they are wired from (for example a Boolean node named "Enable PE?") is listed, so switch that one.
|
||||
> In workflows where nodes have no names, the node type name (such as ComfySwitchNode) is shown.
|
||||
> If the workflow has no on/off items at all, "No true/false settings in this workflow" is shown.
|
||||
|
||||
### Managing the seed (generate panel)
|
||||
The seed is managed with the **Seed** field in the generate panel and the pull-down below it.
|
||||
|
||||
@@ -224,6 +250,6 @@ This is the shortest path to getting used to image generation.
|
||||
> **Summary of this chapter (first half)**
|
||||
> - Select in the Workflow tab and press "Load in GenerateUI" to edit in GenerateUI
|
||||
> - The basics are the three subtabs Input (prompt), Model (model) and Settings (detailed settings)
|
||||
> - After changing the model or Settings values, always press **Apply**. If you forget, it generates with the values from before the change (prompts are reflected automatically)
|
||||
> - After changing the model or Settings values, always press **Apply**. If you forget, it generates with the values from before the change (prompts and the Settings Switches column are reflected automatically)
|
||||
> - Generate to produce. Alt + Apply reflects and generates in one go
|
||||
> - If you set the seed to Fixed, you can keep a picture you like and compare by changing conditions. To use the workflow's seed as it is, use Workflow
|
||||
|
||||
@@ -163,6 +163,8 @@ Until the next keyframe arrives, the previous setting is carried over.
|
||||
4. Decide the number of generations (Batch).
|
||||
5. In each of the Model, Prompt and KSampler columns, press **+** to add a keyframe,
|
||||
then double-click the keyframe and enter "from which run" and "to which setting".
|
||||
Cells in the KSampler column show the items you set (Step, CFG, Denoise, sampler, scheduler, Seed).
|
||||
A cell with nothing set shows "(workflow default)".
|
||||
6. Press **Run**.
|
||||
|
||||
Results are listed in the **Results** panel.
|
||||
@@ -172,9 +174,180 @@ Results are listed in the **Results** panel.
|
||||
You can make a series of images that change little by little.
|
||||
- **Plan Save** - lets you save the settings (the plan). When saved, an overview image summarizing the results is also made,
|
||||
and dropping that image lets you load the same plan again.
|
||||
- **Making a plan from a story** (v0.9.5+) - with the **Storyboard to Lab** skill in the AI TOOL chat,
|
||||
a story is split into one prompt per scene and the **Open in Lab** button loads the plan here (Chapter 10, section 10-5).
|
||||
|
||||
---
|
||||
|
||||
## 5-8 Using partner APIs (Nano Banana)
|
||||
|
||||
ComfyUI has **partner API nodes**, such as Google's **Nano Banana** image model (the Gemini image nodes), that hand the generation over to a cloud service.
|
||||
From v0.8.6, GenerateUI can also run workflows that contain these nodes.
|
||||
|
||||
> **Caution**
|
||||
> Partner API nodes run in the cloud, not on your PC's GPU, and **consume Comfy credits (paid)**.
|
||||
> If you run them repeatedly with Batch, Lab and so on, credits are consumed once per run.
|
||||
|
||||
### Preparation: set your Comfy API key
|
||||
1. Issue a Comfy API key at platform.comfy.org.
|
||||
2. Add the line `COMFY_API_KEY=your key` to the `.env` file in the plugin folder.
|
||||
(If there is no `.env`, copy `.env.example` to make one.)
|
||||
3. Restart ComfyUI. `.env` is read only at startup.
|
||||
|
||||
The key is used only on the server side and is never sent to the browser.
|
||||
If you run without a key, a message says that partner API authentication failed and how to set it up.
|
||||
|
||||
### How to use
|
||||
1. In the Workflow tab, load the Nano Banana sample (`ws_nano_banana_2_1_t2i`) into GenerateUI.
|
||||
2. Edit the prompt on the left side (Prompt) of the **Input** subtab and press **Apply**.
|
||||
3. To use reference images, specify them on the image cards on the right side (Image) of the **Input** subtab.
|
||||
4. Press **Generate**.
|
||||
|
||||
> **Tip**
|
||||
> You can also run it from ComfyUI's own screen while logged in. In that case you do not need the `.env` setting.
|
||||
|
||||
---
|
||||
|
||||
## 5-9 More workflows are supported (v0.8.4 and later)
|
||||
|
||||
The workflows you can use in GenerateUI have increased as follows. They are used the same way as other workflows.
|
||||
|
||||
- **Qwen Image 2.1 Turbo** - ComfyUI's official templates for text to image (T2I) and image to image (I2I). Edits also carry over to the prompt enhancer's input (the feature that expands your prompt automatically).
|
||||
- **Flux.3 Klein** - the four official templates (T2I and Image Edit, for Klein and Klein SD).
|
||||
- **MOSS-Image** - Base/SFT T2I, and the bounding-box versions that specify positions with frames.
|
||||
- **Fibo 2.0 Edit** (v0.8.14 and later) - an image-editing workflow. With the prompt enhancer (PE) setup, edits in the prompt area apply whether PE is on or off. The negative area holds a VGL-format (JSON) description as is. Width and height (Settings) of 0 mean automatic.
|
||||
- **FastH3 OmniRef / Pika-H3-Flex** (v0.8.14 and later) - MiniMax H3 reference to video. Treated as video workflows, and also usable in the Video tab (Chapter 11).
|
||||
- **Native LoRA stack** - ComfyUI v0.40's nodes that load several LoRAs at once (LoadLoraModel / LoadLoraTextEncoder). Apply, bypass and the Metadata display are supported.
|
||||
- **P-H3 Image** (v0.9.4+) - the image-edit and T2I templates. The prompt is shown on the Input tab and can be read by the Metadata tab; there is no negative prompt field. Width and height follow the workflow's wiring, and the prompt upsampler runs as set in the workflow.
|
||||
- **MiniMax H3 Image** (v0.9.6+) - ComfyUI's official image-edit and T2I templates. As with P-H3 Image, the prompt is shown on the Input tab and can be read by the Metadata tab and the sidepanel I tab; there is no negative prompt field. The prompt enhancer (PE) can be turned on and off in the Settings tab's Switches column (5-4).
|
||||
|
||||
> **Caution**
|
||||
> The native LoRA stack nodes are supported **only in API-format workflows**.
|
||||
> ComfyUI's own screen cannot yet add rows to this node, so UI-format workflows are not covered.
|
||||
|
||||
> **Tip**
|
||||
> Workflows containing the **Compare Images** (ImageCompare) node, which compares two images with a slider, used to fail on that node when run from GenerateUI (fixed in v0.9.6).
|
||||
|
||||
> **Tip**
|
||||
> The Metadata tab and the side panel's **I** tab can also read images made with Nano Banana (v0.8.5 and later).
|
||||
> The prompt appears as one whole entry, not split into Positive/Negative.
|
||||
|
||||
## 5-10 Writing prompts in your own language (Prompt Translation, v0.9.9 and later)
|
||||
|
||||
Most image generation models were trained on English prompts.
|
||||
With **Prompt Translation** turned on, you can write a prompt in your own language and turn it into English with one button.
|
||||
Like the translation extensions for Stable Diffusion web UI (A1111), it translates **only the parts written in your own language**.
|
||||
|
||||
Translation uses AI running on your PC (**Ollama**) and the translation model **PLaMo 3 Translate**, which supports 51 languages besides English (Japanese, Chinese, Korean, French, German, Spanish, Russian, Thai, Vietnamese and more).
|
||||
Your prompts are not sent to an online translation service.
|
||||
|
||||
> **What you need first**
|
||||
> Install and start **Ollama** (Chapter 10, section 10-1). The translation model can be downloaded from the Settings screen (see below).
|
||||
|
||||
### Preparation: turn it on in Settings
|
||||
1. In the **Settings** tab, open **Prompt Translation (native language -> English)**.
|
||||
2. Check **Enable prompt translation in GenerateUI**.
|
||||
3. Choose the model in **Translation model**. Models not yet in Ollama are marked **(not downloaded)**; choose one and press **Download** to pull it into Ollama.
|
||||
4. Press **Test**. When the sample prompt comes back in English, you are ready.
|
||||
|
||||
PLaMo 3 Translate comes in four sizes.
|
||||
|
||||
| Model | Size | Notes |
|
||||
| --- | --- | --- |
|
||||
| 0.8B | about 0.6 GB | Fastest. Enough for short tags |
|
||||
| 2B | about 1.8 GB | Recommended. Translates sentences accurately too |
|
||||
| 8B | about 5.3 GB | Higher quality |
|
||||
| 31B | about 19.5 GB | Highest quality, for PCs with plenty of memory |
|
||||
|
||||
> **Tip**
|
||||
> 0.8B sometimes shifts the meaning of a sentence (for example, "the camera follows from behind" in Japanese became "her camera following behind").
|
||||
> If you often write prompts as sentences, such as video prompts, choose 2B or larger.
|
||||
|
||||
The other settings are:
|
||||
|
||||
| Setting | What it does |
|
||||
| --- | --- |
|
||||
| Source language | Usually leave it at **Auto-detect**. If you always write in one language, choosing it makes detection certain |
|
||||
| Ollama URL | Ollama's address. Blank means `http://localhost:11434` |
|
||||
| Where to run the translation model | **GPU** or **CPU, kept loaded** (see below) |
|
||||
| Translate automatically when running | Translates automatically when you press Generate and so on (see below) |
|
||||
|
||||
Other Ollama models (such as the one you use for chat) can also be chosen under **Other Ollama models**. They are sent the text with a "translate into English" instruction.
|
||||
|
||||
### Where to run the translation model
|
||||
| Where | Time per phrase | Good for |
|
||||
| --- | --- | --- |
|
||||
| GPU (fastest per translation, uses VRAM) | about 0.1 s | GPUs with VRAM to spare |
|
||||
| CPU, kept loaded (no VRAM, no reload wait) | about 0.2-0.3 s | Keeping all VRAM for image generation |
|
||||
|
||||
On GPU, turning on **Unload the translation model from VRAM after translating** frees VRAM before image generation (on by default).
|
||||
However, every translation then waits **about 4 seconds** while the model loads again.
|
||||
|
||||
**CPU, kept loaded** keeps the model in system RAM (about 2.7 GB for 2B).
|
||||
It uses no VRAM and is loaded in the background when a workflow is loaded into GenerateUI, so even the first translation does not wait.
|
||||
|
||||
> **Tip**
|
||||
> When Ollama loads another model (for example the AI TOOL chat model), it may drop the resident translation model.
|
||||
> It then loads again automatically at the next translation (that one translation takes a few seconds).
|
||||
|
||||
### Using it: translate with the globe button
|
||||
With Prompt Translation on, a **globe (🌐)** button appears to the right of the **↺** button of Positive Prompt and Negative Prompt on the Input tab.
|
||||
|
||||
1. Write the prompt in your own language. Mixing in English tags is fine.
|
||||
|
||||
```
|
||||
masterpiece, 夕焼けの海辺に立つ少女、(長い黒髪:1.2), <lora:test_lora:0.8>, {猫耳|狐耳}, 水彩画風
|
||||
```
|
||||
|
||||
2. Press the globe button.
|
||||
3. Only the parts in your language become English.
|
||||
|
||||
```
|
||||
masterpiece, A girl standing by the seaside at sunset, (long black hair:1.2), <lora:test_lora:0.8>, {cat ears|fox ears}, watercolor style
|
||||
```
|
||||
|
||||
The following prompt syntax is kept as it is, without translating:
|
||||
|
||||
- Commas and line breaks
|
||||
- Weights `(word:1.2)` and `[ ]`
|
||||
- Random choice `{A|B}` and wildcards `__name__`
|
||||
- LoRA `<lora:name:0.8>` and `embedding:name`
|
||||
- `BREAK` and `AND`
|
||||
|
||||
After translating, an **Original** button appears. Press it to go back to the text before translation (Ctrl+Z also works).
|
||||
|
||||
> **Note**
|
||||
> Only parts that contain letters outside ASCII are translated; English-only parts stay as they are.
|
||||
> Conversely, an English word that contains an accented letter such as é is translated too.
|
||||
|
||||
### Translating automatically when running
|
||||
Under **Translate automatically when running** in Settings, checkboxes choose when to translate automatically.
|
||||
|
||||
- **GenerateUI - Generate**: pressing Generate replaces the text of the Positive/Negative fields with the translation, then generates
|
||||
- **GenerateUI Lab - Run**: a Lab run translates only what is sent. The Lab plan keeps your text
|
||||
- **Video Plan - Run**: a Video tab Plan run translates only the prompts that are sent. The plan keeps your text
|
||||
|
||||
Because Lab and Video Plan keep your text, you can keep editing the plan in your own language.
|
||||
The information saved with the generated images and videos (metadata) records the English prompt that was actually sent.
|
||||
|
||||
> **Note**
|
||||
> If translation fails while automatic translation is on (for example, Ollama is not running), generation stops so that untranslated text is not sent.
|
||||
> For Lab and Video Plan, the reason is shown in the progress line.
|
||||
|
||||
### Also in the Lab modal and Video Plan
|
||||
The globe button is also on these fields, and works the same way:
|
||||
|
||||
- The Positive and Negative fields of the screen that opens when you double-click a cell in Lab's prompt column (5-7)
|
||||
- The **Prompt** field of a block in the Video tab's Plan subtab (Chapter 11, section 11-4)
|
||||
|
||||
> **Tip**
|
||||
> PLaMo 3 Translate is free for personal use, hobbies, research and other non-commercial use (PLaMo Community License).
|
||||
> Check the license terms before using it commercially.
|
||||
|
||||
> **Summary of this chapter (second half)**
|
||||
> - With the Batch subtab, you can generate in bulk while changing the model, prompt, sampler and so on, one kind at a time
|
||||
> - Choose the kind with the Batch Queue column checkbox and start with Generate. Turn the check off when done
|
||||
> - Feeder is a feature that feeds many images one at a time into a workflow. A dedicated node is required
|
||||
> - Lab is an advanced feature that can switch several settings at separate timings
|
||||
> - Partner API nodes (Nano Banana and others) run once `COMFY_API_KEY` is set in `.env`. They consume credits
|
||||
> - With Prompt Translation on, the globe button turns a prompt written in your own language into English. Prompt syntax is kept, and Generate, Lab and Video Plan runs can translate automatically
|
||||
|
||||
@@ -223,6 +223,10 @@ With the buttons shown below the translation result, you can use the result.
|
||||
|
||||
When you want to know the meaning of an English prompt, you can conversely translate it with **EN->JA**.
|
||||
|
||||
> **Tip**
|
||||
> To translate directly in GenerateUI's prompt fields, **Prompt Translation** (the globe button) in Chapter 5, section 5-10 is handy. It turns only the parts in your language into English and keeps weights and LoRA tags.
|
||||
> The AI Assistant's translation can also use the translation model **PLaMo 3 Translate** (v0.9.9 and later; before that, the instruction text was translated too, and ZH->EN came out in Japanese).
|
||||
|
||||
### Making prompts while consulting the AI
|
||||
Write a question in the input box and press **Send** (or the Enter key) to chat with the AI.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ and to solve "how did I make this image?".
|
||||
|
||||
Images generated in ComfyUI have information about the workflow and prompt used for generation (**metadata**) embedded in them.
|
||||
The Gallery tab reads this information so that you can reproduce the same settings right away.
|
||||
Videos (MP4 and WebM) and audio files (WAV, MP3, OGG, M4A, FLAC and Opus) can also be handled in the same list as images.
|
||||
Videos (MP4 and WebM), audio files (WAV, MP3, OGG, M4A, FLAC and Opus) and 3D models (GLB, glTF, VRM, PLY and SPZ) can also be handled in the same list as images.
|
||||
|
||||
---
|
||||
|
||||
@@ -153,7 +153,8 @@ You can also use a Gallery image as an "input image" for generation.
|
||||
| Send GenUI Image | Sets the chosen image as the input image (LoadImage node) of the GenerateUI workflow |
|
||||
| Send to LI node | Sets the chosen image on the image-loading node currently selected on ComfyUI's node screen |
|
||||
| Image Edit | Opens the chosen image in the Image Edit tab (Chapter 11) |
|
||||
| Send CC | Sends the chosen image to a panel in Comic Creator (shown only when the Gallery is opened inside Comic Creator; Chapter 11, section 11-6) |
|
||||
| Send CC Layout | Sends the chosen image to Comic Creator's Layout tab (the selected panel / overlay) (shown only when the Gallery is opened inside Comic Creator; Chapter 11, section 11-6) |
|
||||
| Send CC Image | Sends the chosen image to Comic Creator's Image tab (as a new layer if a document is open, otherwise as a new canvas; videos are not supported; shown under the same condition as Send CC Layout) |
|
||||
|
||||
Handy when redrawing a generated image further with an I2I (image to image) workflow.
|
||||
|
||||
@@ -238,7 +239,7 @@ You can also bring up a list of **similar images** to the one you choose.
|
||||
| Model | Characteristics | Preparation |
|
||||
| --- | --- | --- |
|
||||
| Gemini Embedding 2 | Computed on Google's service. Fast. Images are sent to Google | Get an API key, write `GEMINI_API_KEY=key` in the plugin's `.env`, and restart ComfyUI |
|
||||
| EmbeddingGemma 2 | Computed inside your computer (local). Nothing is sent outside | No extra preparation. The model is downloaded automatically only the first time |
|
||||
| EmbeddingGemma 2 | Computed inside your computer with **Ollama** (local, free). Nothing is sent outside | Install and start Ollama, then run `ollama pull embeddinggemma-2:740m` (or the lighter `embeddinggemma-2:440m`). On the Model tab, choose the **Ollama URL** (usually unchanged) and the **Model size** |
|
||||
|
||||
> **Caution: when using Gemini**
|
||||
> Images are sent to Google's service. Content sent under the free tier may be used by Google to improve its products.
|
||||
@@ -250,13 +251,14 @@ You can also bring up a list of **similar images** to the one you choose.
|
||||
2. Choose "new only" or "rebuild all" and start. Progress is shown, and you can stop partway.
|
||||
3. The index is one shared across all folders. You may build it a little at a time, folder by folder.
|
||||
|
||||
When you change the model, the vector dimension or the image precision, a **rebuild of everything** is needed.
|
||||
When you change the model or the vector dimension, a **rebuild of everything** is needed.
|
||||
|
||||
### Speed guidelines and settings
|
||||
- The local EmbeddingGemma 2 runs on the CPU by default and takes a few seconds per image.
|
||||
Setting the **image precision** to "standard" or "light" makes it faster (rebuild everything after changing it).
|
||||
Reducing the **number of CPU threads** lightens the computer's load while generating images.
|
||||
- If you have an NVIDIA GPU, installing `onnxruntime-gpu` into ComfyUI's Python brings it to about 0.1 second per image (the fix is in Chapter 12, section 12-5).
|
||||
- Since v0.8.14 the local EmbeddingGemma 2 runs in Ollama (the earlier way of running it in ComfyUI's Python was removed).
|
||||
On an NVIDIA GPU it takes about 0.1 to 0.2 second per image. When indexing finishes, the model is unloaded from GPU memory automatically.
|
||||
- Ollama must be running while you build the index and while you search.
|
||||
- The 740m (text + image + audio) and 440m (text + image, lighter) model sizes return the same results, so switching does not need a rebuild.
|
||||
Indexes made with v0.8.13 or earlier keep working as they are.
|
||||
- In the **Search** tab, you can decide the "minimum score" for similar-image and text search, and the maximum number of results.
|
||||
Because the score level differs by model, it is saved per model.
|
||||
|
||||
@@ -294,6 +296,30 @@ When you choose a video, only the audio track is extracted and saved.
|
||||
> **Tip: use in the Video tab**
|
||||
> In the Asset subtab of Video, you can set an audio file as "Set as BGM" or "Add as sound" (Chapter 11, section 11-4).
|
||||
|
||||
---
|
||||
|
||||
## 8-8 Viewing 3D models (v0.9.4+)
|
||||
|
||||
The results of ComfyUI workflows that make 3D models (such as image to 3D) can also be checked in the Gallery.
|
||||
Supported formats are GLB, glTF, VRM, PLY (mesh, point cloud or Gaussian splat) and SPZ.
|
||||
|
||||
- They are listed with a "**3D**" badge. The thumbnail is drawn by your browser the first time the file appears in the list and is then kept on the server
|
||||
(files over 128 MB get one when opened).
|
||||
- When selected, the file is shown in 3D in the built-in viewer of the detail panel on the right. Double-click to enlarge it.
|
||||
|
||||
| Action | What it does |
|
||||
| --- | --- |
|
||||
| Drag | Rotate |
|
||||
| Right-drag | Move (pan) |
|
||||
| Mouse wheel | Zoom in / out |
|
||||
| ⟲ (reset) button | Return to the starting view |
|
||||
|
||||
- For a model saved with ComfyUI's **SaveGLB**, the workflow can be read just like an image's (sections 8-2 and 8-3).
|
||||
|
||||
> **Caution**
|
||||
> - 3D files cannot be sent to Image Edit, GenerateUI, Tagger or Feeder.
|
||||
> - Some `.gltf` files refer to separate `.bin` or image files. Moving or deleting in the Gallery only affects the `.gltf` itself, so do not change where those files are.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - In the Gallery's Output, you can look back at generated images folder by folder. You can also search by words in the prompt
|
||||
> - The detail panel's Prompt tab shows right away the prompt used to generate the image
|
||||
@@ -302,3 +328,4 @@ When you choose a video, only the audio track is extracted and saved.
|
||||
> - If you drop an image onto the Metadata subtab, you can examine the models, LoRAs and prompt in detail
|
||||
> - With the S button you can search by the meaning of a sentence or by "similar images" (semantic search)
|
||||
> - Audio files can also be listed, played and converted
|
||||
> - 3D models (GLB, glTF, VRM, PLY, SPZ) can also be turned around and checked in the Gallery (8-8)
|
||||
|
||||
@@ -7,6 +7,9 @@ If you prepare local AI, you can do things like the following.
|
||||
- Chat with the AI and have it come up with prompt ideas
|
||||
- Show it an image and have it make a description, prompt or tags for that image
|
||||
- Have it make candidate lists for wildcards
|
||||
- Pick a **skill** to switch what it is good at: writing prompts, storyboards, troubleshooting and more (v0.9.5+)
|
||||
- Have it check whether a generated image matches your request and fix only what is missing (v0.9.5+)
|
||||
- Read the text shown in an image (OCR, v0.9.5+)
|
||||
|
||||
Because it runs inside your own computer rather than as a service on the internet,
|
||||
a big feature is that **what you type and the images you use are not sent outside**.
|
||||
@@ -69,10 +72,14 @@ The screen is divided into four parts, all shown at the same time.
|
||||
| Part | What you can do |
|
||||
| --- | --- |
|
||||
| Translate | Translation between languages such as Japanese, English and Chinese |
|
||||
| Chat | Conversation with the AI. If the conditions are met, generating images from the conversation too |
|
||||
| TOOLS | Reading images (VLM), making wildcards |
|
||||
| Chat | Conversation with the AI. Skills switch what it is good at. If the conditions are met, generating images from the conversation too |
|
||||
| TOOLS | Reading images (VLM), reading text (OCR), making wildcards |
|
||||
| Settings | Settings for which AI software and model to use |
|
||||
|
||||
> **Tip: Translate and Settings can be collapsed (v0.9.5+)**
|
||||
> Press the arrow (◀ / ▶) at the right end of the Translate or Settings header to collapse that part into a thin strip, giving Chat and TOOLS more room.
|
||||
> Click anywhere on the strip to open it again. The collapsed state is kept the next time you open the tab.
|
||||
|
||||
### First: connect to the AI software in the settings
|
||||
1. In **Backend** under the settings, choose the software to use (for example Ollama).
|
||||
2. Check the address under **Connection**. If you use Ollama with its defaults, no change is needed.
|
||||
@@ -96,10 +103,18 @@ The screen is divided into four parts, all shown at the same time.
|
||||
2. Write the text in the input box and press **Translate**.
|
||||
3. **Copy** the result and paste it into GenerateUI or a preset.
|
||||
|
||||
> **Tip**
|
||||
> Text read in TOOLS (such as OCR in 10-8) can be put straight into this input box with **To Translation** in TOOLS.
|
||||
|
||||
> **Tip**
|
||||
> Small models sometimes return the original text as it is without translating.
|
||||
> In that case a message says the model returned the original text, so try another model.
|
||||
|
||||
> **Tip**
|
||||
> You can also choose the translation model **PLaMo 3 Translate** (Chapter 5, section 5-10) as the model in Settings (v0.9.9 and later).
|
||||
> Besides Japanese, English and Chinese, **Free** accepts language names such as French, フランス語 or français for any language PLaMo supports. A target language PLaMo does not support (such as Portuguese) gives an error.
|
||||
> The same applies to the translation in WS Library's A tab.
|
||||
|
||||
### Chat
|
||||
Talk in the input box and send with **Enter** (**Shift + Enter** for a line break).
|
||||
It remembers the flow of the conversation, so follow-up requests such as "make it a brighter mood" also work.
|
||||
@@ -110,6 +125,10 @@ It remembers the flow of the conversation, so follow-up requests such as "make i
|
||||
> just asking in chat "draw a cat" generates an image with the workflow loaded in GenerateUI, and it is shown inside the chat.
|
||||
> This feature can be turned on or off with the **Allow image generation** check next to the input box.
|
||||
|
||||
> **Tip**
|
||||
> Choose a **skill** in the dropdown above the chat to switch what the AI is good at (10-5).
|
||||
> The AI's replies are shown with headings, lists and code boxes formatted (v0.9.5+).
|
||||
|
||||
### TOOLS - reading images (VLM)
|
||||
With a model that supports image input (a VLM), you can have the AI read the contents of an image.
|
||||
|
||||
@@ -122,8 +141,16 @@ With a model that supports image input (a VLM), you can have the AI read the con
|
||||
| Create prompt | Makes a prompt for reproducing the image |
|
||||
| Create tags | Writes out the elements of the image in tag format |
|
||||
| Create wildcards | Makes candidate wildcard lists by specifying a category name and count (no image needed) |
|
||||
| OCR (extract text) | Reads the text shown in the image (10-8) |
|
||||
|
||||
3. Press **Run**. You can copy the result and use it.
|
||||
3. Press **Run**.
|
||||
4. Use the result with the buttons under it (v0.9.5+).
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| To Translation | Puts it into the Translate input box (opens Translate if it is collapsed) |
|
||||
| To Chat | Adds it to the chat input box |
|
||||
| Copy | Copies it to the clipboard |
|
||||
|
||||
> **Example**
|
||||
> When you want to imitate the mood of an illustration you like, run "Create prompt" on that image
|
||||
@@ -190,9 +217,226 @@ In that case, anyone who can access it can use your AI software and images.
|
||||
> Use `--listen` **only on a trusted network**, such as a home LAN.
|
||||
> Do not use it in cafes or on public Wi-Fi.
|
||||
|
||||
---
|
||||
|
||||
## 10-5 Skills - switching what the chat is good at (v0.9.5+)
|
||||
|
||||
A **skill** is a set of instructions that tells the chat what role to play and how to answer.
|
||||
When you pick one in the dropdown above the chat (it starts as "-- No skill --"), its instructions are sent along with every message
|
||||
(the instructions themselves are not shown in the chat).
|
||||
|
||||
### Built-in skills
|
||||
Workflow Studio comes with the following six skills.
|
||||
|
||||
| Skill | What it does | How you use the result |
|
||||
| --- | --- | --- |
|
||||
| T2I Prompt Enhancer | Turns a short idea into a prompt written the way the model in your GenerateUI workflow understands best | Buttons under the prompt |
|
||||
| Image to Prompt | Writes a prompt that recreates the image you put in TOOLS (needs a model that can read images) | Buttons under the prompt |
|
||||
| Storyboard to Lab | Splits a story into one prompt per scene, ready for continuous generation in Lab | **Open in Lab** |
|
||||
| Workflow Doctor | Explains the workflow and helps you find the cause of errors (10-7) | Answers in text |
|
||||
| SVG Icon Generator | Draws simple icons and illustrations as SVG (vector images) | Preview and **Save to Gallery** |
|
||||
| Skill Creator | Asks you questions and builds a new skill (10-9) | **Save as new skill** |
|
||||
|
||||
### Making a prompt and generating with it (T2I Prompt Enhancer)
|
||||
1. Load the workflow you want to use into GenerateUI. The skill looks at that workflow's model to decide how to write the prompt.
|
||||
2. In the AI TOOL chat, choose **T2I Prompt Enhancer** in the dropdown.
|
||||
3. Describe the picture in your own words, for example "a cat holding an umbrella in rainy Tokyo at night, neon", and send it.
|
||||
4. The reply shows which writing style was chosen (the `Style:` line) and the English prompt in a box. Use the buttons under it.
|
||||
|
||||
| Button | What it does |
|
||||
| --- | --- |
|
||||
| Apply to GenerateUI | Writes the positive / negative prompt into GenerateUI |
|
||||
| Generate with this prompt | Generates right away and shows the image in the chat |
|
||||
| Copy | Copies the positive prompt |
|
||||
|
||||
The writing style depends on the model:
|
||||
|
||||
| Model in GenerateUI | How the prompt is written |
|
||||
| --- | --- |
|
||||
| Qwen-Image, FLUX, Z-Image, etc. | English sentences (the negative is left empty when CFG is about 1) |
|
||||
| Illustrious, NoobAI, Pony, etc. (SDXL anime models) | Danbooru tags (starting with quality tags) |
|
||||
| Realistic SDXL, SD1.5 | Short phrases separated by commas |
|
||||
|
||||
> **Tip**
|
||||
> Follow-up requests such as "make it more night-like" or "two cats" give you the whole updated prompt again.
|
||||
|
||||
> **Tip: the prompt currently in GenerateUI is not sent**
|
||||
> Normally the prompt already typed into GenerateUI is not passed to the AI, because small models tend to mix it (for example an old negative prompt) into the new prompt.
|
||||
> It is passed only when your message refers to it, such as "**improve the current prompt**".
|
||||
|
||||
### Generating a storyboard in a row (Storyboard to Lab)
|
||||
1. Load a text-to-image (T2I) workflow into GenerateUI.
|
||||
2. In the chat, choose **Storyboard to Lab** and ask, for example, "A tabby kitten wakes up in the morning, chases a butterfly in the garden, and naps on the veranda in the evening. 3 scenes."
|
||||
3. The reply shows the scene list and an **Open in Lab (3 scenes)** button. Press it.
|
||||
4. GenerateUI's **Lab** (Chapter 5, section 5-7) opens with the plan loaded as an unsaved plan.
|
||||
T2I mode is on, Batch is the number of scenes, and the Prompt column has one keyframe per scene. The KSampler column holds a seed shared by all scenes (`Seed ...`).
|
||||
5. Check it and press **Run**. To keep it, use **Plan Save**.
|
||||
|
||||
> **Tip**
|
||||
> The characters' looks (hair, clothing colors and so on) are written with the same words in every scene, and the seed is fixed,
|
||||
> so the same character is easier to keep across scenes.
|
||||
|
||||
### Automatic check of replies
|
||||
For skills that output a prompt or an SVG, Workflow Studio checks the shape of the reply automatically.
|
||||
If, for example, the prompt box is broken or the SVG is not closed, a dashed notice "**Auto-check: ...**" appears and the AI is asked once to fix it.
|
||||
Replies that do not try to output a prompt or SVG (for example when you only asked for an explanation) are left alone.
|
||||
|
||||
### Managing skills (pencil button)
|
||||
The pencil button to the right of the dropdown opens the skill list.
|
||||
|
||||
- **+ New** writes a new skill, the pencil on each row edits or deletes it, and **Import** adds other skills (10-9)
|
||||
- Skills that ship with Workflow Studio are marked **built-in**.
|
||||
Built-in skills you have not edited are updated automatically when Workflow Studio is updated.
|
||||
An edited one is not replaced automatically; when a newer version exists, a **↻ (update)** button appears.
|
||||
- Skill files are saved in `ComfyUI/user/default/Workflow-Studio/ai_skills/`.
|
||||
|
||||
> **Caution**
|
||||
> Replacing a skill with the newer version via ↻ (update) discards your edits to that skill.
|
||||
|
||||
---
|
||||
|
||||
## 10-6 Checking generated images and fixing them (v0.9.5+)
|
||||
|
||||
This feature has the AI check whether an image generated from the chat matches what you asked for.
|
||||
If something is missing, you can redraw with a prompt that fixes just that.
|
||||
|
||||
### Preparation: a judging model
|
||||
The check does not use a normal chat model but an Ollama **decision model** (a model that answers yes/no questions with a probability).
|
||||
Get one that can read images with `ollama pull` (the author confirmed `d3:4b`, `clef-flash` and `clef`).
|
||||
|
||||
> **Tip: why a separate model**
|
||||
> In the author's tests, a chat model shown an image sometimes answered "yes" to "is the person holding a sword?" when there was no sword.
|
||||
> A decision model returns the probability of "yes" as a number, so uncertain answers can be told apart.
|
||||
|
||||
1. In AI TOOL's **Settings**, under **Chat Image Check**, choose the **Judge model (Ollama decision model)** (press **Refresh** if the list is empty).
|
||||
2. Choose **Automatic fix rounds**. **0 (fix by button)** is recommended at first.
|
||||
3. Press **Save Settings**.
|
||||
|
||||
### How to use it
|
||||
1. Tick **Check images** below the chat input.
|
||||
2. Generate an image from the chat (the AI's image generation, or **Generate with this prompt**).
|
||||
3. A result card appears under the image. The AI splits your request into 3 to 8 check items and checks them one by one.
|
||||
|
||||
| Mark | Meaning |
|
||||
| --- | --- |
|
||||
| ✔ (met) | Probability of "yes" is 85% or more |
|
||||
| ? (unclear) | 30-85% |
|
||||
| ✘ (not met) | Less than 30% |
|
||||
|
||||
4. If there is a ✘ or ?, press **Fix missing items and regenerate**. Only the unmet items are sent to the AI, which rewrites the prompt and draws again.
|
||||
|
||||
If you set Automatic fix rounds to 1-3, this repeats automatically up to that number of times.
|
||||
|
||||
> **Tip: SVGs can be checked too**
|
||||
> SVGs drawn with SVG Icon Generator are turned into an image and checked for "does it look like what you asked for" and "are its parts (beak, wings and so on) drawn".
|
||||
> **Ask to redraw unclear parts** asks the AI to fix them.
|
||||
|
||||
> **Caution**
|
||||
> - The check is not perfect. It is especially weak at "who is wearing what",
|
||||
> so when there are several people it automatically adds reverse check items such as "the hat is not on the girl" to compensate.
|
||||
> - How well the redraw works depends on the chat model. If it does not improve, change your request or the model.
|
||||
> - The judging model also uses GPU memory (about 6 GB for `d3:4b`). Once loaded, a check takes a few seconds.
|
||||
|
||||
---
|
||||
|
||||
## 10-7 Asking about workflow problems (Workflow Doctor, v0.9.5+)
|
||||
|
||||
**Workflow Doctor** is a skill that explains the workflow loaded in GenerateUI and helps you find the cause of errors.
|
||||
Instead of relying only on the AI's knowledge, every time you send a message Workflow Studio passes it the following facts it has actually checked.
|
||||
|
||||
| What is checked | Example |
|
||||
| --- | --- |
|
||||
| Nodes that are not installed | Names of nodes ComfyUI does not know |
|
||||
| Files and values not in ComfyUI's lists | Models or LoRAs that cannot be found (with similar names) |
|
||||
| Connections | Inputs that are not connected, links to nodes that do not exist |
|
||||
| The last execution error | The error recorded in ComfyUI's history |
|
||||
| ComfyUI's log | The latest error part of the log |
|
||||
|
||||
### How to use it
|
||||
1. Load the workflow you want to ask about into GenerateUI (for an error, run it once so the error is recorded).
|
||||
2. In the chat, choose **Workflow Doctor**.
|
||||
3. Ask, for example, "check this workflow for problems before I run it" or "what caused the last error?".
|
||||
|
||||
> **Example**
|
||||
> For a workflow whose LoRA file had been moved to another folder, it answered
|
||||
> "the file specified in the `temp\` folder cannot be found; a file with the same name exists in `Illustrious\`", including how to fix it.
|
||||
|
||||
> **Tip**
|
||||
> The checked facts are rebuilt every time you send a message. After fixing something, just ask again.
|
||||
|
||||
---
|
||||
|
||||
## 10-8 Reading text in images (OCR, v0.9.5+)
|
||||
|
||||
In TOOLS you can read the text shown in an image. The text you read can be sent straight to translation or the chat.
|
||||
|
||||
### Preparation: an OCR model
|
||||
OCR uses a model that is good at reading text, separate from the chat model. Prepare one in Ollama.
|
||||
These are the author's results on an image mixing horizontal and vertical Japanese and English text.
|
||||
|
||||
| Model | Time | Result |
|
||||
| --- | --- | --- |
|
||||
| LightOnOCR-3 (`maternion/LightOnOCR-3:4b`) | About 1 s (6-8 s with positions) | Read everything |
|
||||
| PaddleOCR-VL (manga version) | About 4 s | Read everything |
|
||||
| DeepSeek-OCR family | About 14 s | Read everything (always with positions) |
|
||||
| PaddleOCR-VL 1.6, GLM-OCR | About 11-15 s | Missed the vertical text |
|
||||
|
||||
### How to use it
|
||||
1. Drop an image into TOOLS.
|
||||
2. Choose **OCR (extract text)** as the task.
|
||||
3. Choose the **OCR model**. Ollama models whose name contains "ocr" are listed, best first (press ↻ if the list is empty).
|
||||
4. Press **Run**. The text is shown in the result box, one line each.
|
||||
|
||||
> **Tip: Show text positions**
|
||||
> Tick **Show text positions** to draw numbered boxes over the image so you can see where each text is.
|
||||
> Clicking a box selects the same line in the result box.
|
||||
> Only LightOnOCR and the DeepSeek-OCR family can return positions; for other models this check cannot be selected.
|
||||
|
||||
> **Tip**
|
||||
> If the result is empty or the same line repeats, it is automatically read once more in grayscale.
|
||||
|
||||
---
|
||||
|
||||
## 10-9 Making and importing skills (advanced, v0.9.5+)
|
||||
|
||||
### Making one with Skill Creator
|
||||
In the chat, choose **Skill Creator** and say something like "I want a skill that is good at ...".
|
||||
The AI asks questions one at a time; answer them and you get a draft skill.
|
||||
Press **Save as new skill** under the reply; the skill list opens with the draft filled in, so check it and save.
|
||||
You can also ask it to improve an existing skill, for example "this part of this skill does not work well".
|
||||
|
||||
### What a skill file contains
|
||||
A skill is a text file (`.md`). Write the name and description in the part enclosed by `---` at the top, and the instructions for the AI below it.
|
||||
Under `metadata:` in the top part you can also write these settings (all optional).
|
||||
|
||||
| Setting | Meaning |
|
||||
| --- | --- |
|
||||
| `output` | `prompt` or `svg`. Checks the shape of the reply automatically and adds the buttons |
|
||||
| `tools` | `[]` stops the AI from starting image generation on its own |
|
||||
| `temperature` | How varied the answers are (0.3 stable, 0.9 free) |
|
||||
| `think` | Turns Thinking mode on or off just for this skill |
|
||||
| `context` | Information passed to the AI: `workflow` (workflow summary), `current-prompt` (the current prompt), `workflow-detail` / `logs` (the same checks as Workflow Doctor) |
|
||||
|
||||
### Importing other skills
|
||||
**Import** in the skill list adds:
|
||||
|
||||
- a single `.md` skill
|
||||
- a ZIP of a folder containing `SKILL.md` (the **Agent Skills** format shared by many AI tools)
|
||||
|
||||
Text files in the folder (reference material and so on) are imported too, and the AI reads them only when it needs them
|
||||
(this needs a model that supports tool calling).
|
||||
|
||||
> **Caution**
|
||||
> - Only text files are imported. Programs such as scripts are not imported and never run.
|
||||
> - A skill's instructions are passed to the AI as they are. Read a skill made by someone else before using it.
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - If you prepare local AI (Ollama and so on), you can translate, chat and read images. What you type is not sent outside
|
||||
> - In the AI TOOL tab's settings, choose the backend, address and model, and test the connection
|
||||
> - The Prompt tab and Tagger tab have separate gear-button settings
|
||||
> - AI and image generation compete for VRAM. Free it with "Unload Model" or "VRAM Management (Ollama)"
|
||||
> - By default it connects only to AI on your own computer. Use `--listen` only on a trusted network
|
||||
> - **Skills** switch what the chat is good at. T2I Prompt Enhancer writes a prompt that buttons apply to GenerateUI or generate with
|
||||
> - With **Check images** on, a decision model checks the generated image and you can redraw fixing only what is missing
|
||||
> - Workflow Doctor helps with workflow problems based on facts it has actually checked
|
||||
> - **OCR (extract text)** in TOOLS reads the text in an image and sends it to translation or the chat
|
||||
|
||||
@@ -54,6 +54,9 @@ To use Tagger, you need the following preparation.
|
||||
|
||||
1. Put a **model for tagging** (such as WD Tagger) in ComfyUI's `models/tagger/(model name)/` folder.
|
||||
Put a `.onnx` file and `selected_tags.csv` in the folder.
|
||||
**PixAI Tagger v1.0** (v0.8.10 and later) also works. From the community conversion on Hugging Face (`Mexes/pixai-tagger-v1.0-onnx-fp32-fp16-int8`),
|
||||
put one `.onnx` file (`model_fp16.onnx` is recommended for a GPU) together with `selected_tags.csv` and `preprocess.json` in one folder, and it is detected automatically.
|
||||
When you choose a PixAI model, the threshold sliders switch to PixAI's recommended values (0.17 / 0.27).
|
||||
2. Install the **extra Python packages**.
|
||||
Use the `requirements.txt` in the Workflow Studio folder.
|
||||
|
||||
@@ -69,6 +72,10 @@ To use Tagger, you need the following preparation.
|
||||
> Always install into **the Python that ComfyUI uses**.
|
||||
> If you are not sure, there is also a method of installing packages with a ComfyUI Manager feature.
|
||||
|
||||
> **Caution: check that it runs on the GPU**
|
||||
> On an NVIDIA GPU, Tagger takes less than a second per image, but on the CPU, PixAI takes about 20 seconds per image.
|
||||
> If it is very slow, check that the GPU package is installed correctly (Chapter 12, section 12-5).
|
||||
|
||||
> **Tip**
|
||||
> You can also combine it with the local AI image reading (VLM) from Chapter 10 to make tags that combine both results.
|
||||
|
||||
@@ -96,6 +103,22 @@ You can put text on a generated image, or make a mask for redrawing part of it.
|
||||
- Save the edited image with **Save PNG** (download), **Save to Gallery** and so on.
|
||||
- Undo with **Ctrl + Z**, redo with **Ctrl + Y**.
|
||||
|
||||
### Draw colors and shortcuts (v0.9.3+)
|
||||
When you choose **Draw**, a **color palette** appears in the second row of the tool options.
|
||||
|
||||
- It shows **Color 1 / Color 2**, 20 basic colors and 20 custom slots. Click a color to paint with it.
|
||||
- Click **+** or an empty slot to register the current color as a custom color. Right-click a color to register, edit or delete it (basic colors can be changed too, and returned to their original color).
|
||||
Registered colors are saved per browser.
|
||||
- The **eyedropper** button next to **Color** on the right picks a color from the canvas (Esc cancels).
|
||||
- The starting colors are black (Color 1) and white (Color 2).
|
||||
|
||||
| Key (while Draw is active) | What it does |
|
||||
| --- | --- |
|
||||
| D / E | Draw / Erase |
|
||||
| X | Swap Color 1 and Color 2 |
|
||||
| [ / ] | Make the brush smaller / larger |
|
||||
| B | Open or close the brush picker (Esc also closes it) |
|
||||
|
||||
### Inpainting (redrawing part of an image)
|
||||
The handiest use of Image Edit is **inpainting**.
|
||||
It is for when "I want to redraw only the face" or "I want to fix only the place where the hand broke".
|
||||
@@ -119,6 +142,7 @@ The **Video** tab is a dedicated screen for making short videos from images or t
|
||||
- **MiniMax H3** (image to video)
|
||||
- **LTX-2.5** (text to video, image to video, and video from specified first and last images)
|
||||
- **Wan2.2** (text to video)
|
||||
- **FastH3 OmniRef / Pika-H3-Flex** (MiniMax H3 reference to video, v0.8.14 and later)
|
||||
|
||||
You load a matching workflow with **Load in Video** in the Workflow tab and use it (Chapter 4, section 4-6).
|
||||
|
||||
@@ -131,6 +155,38 @@ You load a matching workflow with **Load in Video** in the Workflow tab and use
|
||||
|
||||
There are also the **Frame** feature, which saves a favorite scene of a video as a still image, and **GIF**, which converts a video to an animated GIF.
|
||||
|
||||
> **Tip**
|
||||
> The **Prompt** field of a Plan block also has the Prompt Translation globe button (Chapter 5, section 5-10).
|
||||
> With **Video Plan - Run** turned on in Settings, you can run the plan with prompts in your own language and send only English.
|
||||
|
||||
> **Note**
|
||||
> Up to v0.9.8, with workflows saved by newer ComfyUI versions, the prompt, length (Duration) and Turbo settings entered in Plan were not applied, and the values saved in the workflow were used instead (fixed in v0.9.9).
|
||||
> Because Plan values are applied now, running the same plan as before may give different results.
|
||||
|
||||
### Register as Asset and audio conversion (v0.8.9 and later)
|
||||
- **Register as Asset** - a button in the Video Source panel on the left. It copies a video from your PC into the output folder (`output/video`) and registers it in the "Video Assets" group.
|
||||
Once registered, it appears in the Asset subtab list and can be used in Edit and elsewhere.
|
||||
Videos already in the output folder (ones chosen in Asset or generation results) are only tagged into the group, not copied. If the name already exists, a number is added after it.
|
||||
- **Convert audio...** - a button next to the crop buttons in the Edit subtab. It converts only the audio of the selected video clip to FLAC, MP3, Opus, M4A or WAV and saves it into `output/video`.
|
||||
It is the same dialog as Gallery's Convert. Only video clips that contain audio are covered.
|
||||
- Pressing the audio-conversion button opens the dialog below. Choose the **output format** (such as FLAC), the **sample rate** and the **channels**, then press **Convert**.
|
||||
If you check the option to carry over the embedded prompt/workflow tags, the original video's information is kept in formats that can hold tags (such as FLAC).
|
||||
|
||||
- The Video Source panel is a little wider, the file name shows on one line, and the buttons sit in a single row.
|
||||
|
||||
### Reference to video (FastH3 OmniRef / Pika-H3-Flex) (v0.8.14 and later)
|
||||
These workflows make a video from **reference images** instead of a first and last frame,
|
||||
so the image fields in the Plan subtab are renamed:
|
||||
|
||||
- **Reference Image** - the first reference image (the usual First Image field)
|
||||
- **Reference Image 2 (optional)** - the second reference image (the usual Last Image field)
|
||||
|
||||
- Prompt, duration (seconds), aspect ratio and so on work the same as for other video workflows.
|
||||
- A block whose image field is left empty with **None** or **Clear** is generated without that image, even if the workflow has a reference image wired in.
|
||||
Blocks added with "+ Block" start with empty images, so set an image on each block that needs a reference.
|
||||
- Choosing **Use previous block's last frame** makes the last frame of the video actually generated by the previous block the first reference image of that block.
|
||||
- The Asset subtab buttons also change to "Set as Reference Image" and "Set as Reference Image 2".
|
||||
|
||||
> **Caution**
|
||||
> Video generation uses far more GPU memory and time than image generation.
|
||||
> Check the required specs of the model before trying it.
|
||||
@@ -155,6 +211,7 @@ Here we mainly introduce items not covered in earlier chapters.
|
||||
| CivitAI API Key | For fetching Civitai information (optional) | Chapter 6 |
|
||||
| Eagle integration (optional) | Automatically save generated images to Eagle | Chapter 8 |
|
||||
| VRAM Management (Ollama) | Unload AI models from memory before generation | Chapter 10 |
|
||||
| Prompt Translation (native language -> English) | Turn prompts in your own language into English with the globe button or automatic translation when running | Chapter 5, 5-10 |
|
||||
| Data Management | Backing up and restoring settings and data | Below |
|
||||
|
||||
### Default Workflow
|
||||
@@ -198,11 +255,12 @@ It is software you install separately from Workflow Studio, but if you install b
|
||||
If you install both, the following integrations are possible.
|
||||
|
||||
- **Use Workflow Studio's Gallery inside Comic Creator**
|
||||
Workflow Studio's Gallery appears inside Comic Creator, and you can choose generated images and send them to panels with the **Send CC** button.
|
||||
(The Send CC button is shown only in a Gallery opened inside Comic Creator)
|
||||
Workflow Studio's Gallery appears inside Comic Creator, and you can choose generated images and send them to panels with **Send CC Layout** (Layout tab panel / overlay) or to the Image tab with **Send CC Image**.
|
||||
(The Send CC buttons are shown only in a Gallery opened inside Comic Creator)
|
||||
- **Generate images from Comic Creator**
|
||||
From the Comic Creator screen, through Workflow Studio (GenerateUI), you can generate images to put in panels (T2I),
|
||||
or redraw based on a rough you drew or an image you placed (I2I and inpainting).
|
||||
From Comic Creator's Nanobanana+ tab (local mode), you can also run image-editing workflows that use several reference images (Flux.3 Klein, Qwen Image 2.1 and so on) (v0.8.12 and later).
|
||||
|
||||
With the flow "make images in Workflow Studio -> put them together as manga in Comic Creator",
|
||||
you can go from image generation to finished manga entirely inside ComfyUI.
|
||||
@@ -224,6 +282,70 @@ you can go from image generation to finished manga entirely inside ComfyUI.
|
||||
|
||||
---
|
||||
|
||||
## 11-7 TagSpace - picking tags and prompts from a "map of meaning" (v0.9.0 and later)
|
||||
|
||||
The **TagSpace** tab lays out the prompts, tags and generated images inside Workflow Studio in a 3D space
|
||||
according to **closeness of meaning**. Things close together mean similar things, so you can browse the space
|
||||
("which tags are close to this mood?") and pick tags to build a prompt.
|
||||
|
||||
### What is in the space
|
||||
- **Cards** - the sample images of the Gallery's **ImagePrompt** (Chapter 8, section 8-4) with their prompts, shown as thumbnails
|
||||
- **Tags** - words taken (split at commas) from ImagePrompt prompts, prompts embedded in generated images, wildcards, the Tagger DB and the Prompt library, shown as dots colored by source
|
||||
- **Gallery points** - some of your generated images, shown as faint dots (each carries the tags of its own prompt)
|
||||
|
||||
### Preparation
|
||||
As with the Gallery's semantic search (Chapter 8, section 8-6), **Ollama** and **EmbeddingGemma 2** are needed.
|
||||
If the Gallery's semantic index was made with Gemini Embedding 2, rebuild it with EmbeddingGemma 2 first.
|
||||
|
||||
### Building the space
|
||||
Choose the settings in the toolbar and press **Build / Update**.
|
||||
|
||||
- **Tags** - how many tags to include. The sources take turns, each in order of how often a tag is used
|
||||
- **All tags** - when on, every candidate tag is used instead of the Tags number (up to 20,000)
|
||||
- **Min uses** - leaves out tags used fewer times than this. 2 removes one-off spelling variants and the like
|
||||
- **Wildcards** - whether wildcards are used as a tag source
|
||||
- **Gallery points** - how many generated images to show (0 = none)
|
||||
- **Tagger fill** - tags generated images whose prompt cannot be read, using a Tagger model (OFF / a model name; choose "Gallery points only" or "All images without a prompt")
|
||||
|
||||
- The first build takes a few minutes to read image prompts and prepare tags. Later builds reuse the previous results and finish in seconds to tens of seconds.
|
||||
- The status line shows the numbers of cards, tags and Gallery points. When only part of the candidates is used, "(of N candidates)" is shown.
|
||||
|
||||
> **Tip**
|
||||
> Most distinct tags are words that only appear in wildcards.
|
||||
> Turning "Wildcards" off and "All tags" on lays out every tag you have actually used (about 10,000 in testing).
|
||||
|
||||
> **Caution**
|
||||
> Tagger fill takes about 0.5 second per image on the CPU. Before choosing "All images without a prompt", check that Tagger runs on the GPU (Chapter 12, section 12-5).
|
||||
|
||||
### Looking and searching
|
||||
- **Drag** to rotate, **right-drag** to pan, and use the **wheel** to zoom.
|
||||
- Hovering over a point shows its name, thumbnail and prompt or sources.
|
||||
- The toolbar checkboxes switch Cards, Thumbnails, Tags and Gallery on or off, as well as **Glow** (stars shining in a night sky) and **Twinkle**.
|
||||
The checkboxes for each source show or hide tags by source.
|
||||
- Type words or a sentence in the search box at the top right and press **Enter** (any language). A white mark is placed at that meaning, and nearby cards and tags are listed.
|
||||
|
||||
### Picking tags to build a prompt
|
||||
1. Click a card or tag to select it. Cards and tags close in meaning are listed under **Nearby** on the right.
|
||||
(Clicking a Gallery point also lists similar generated images)
|
||||
2. **Double-click** what you want in the prompt, or press **Add** in **Selection**, or the **+** in the list.
|
||||
A card adds its whole prompt, a tag adds the tag itself, and a Gallery point adds that image's tags to the **Prompt** area at the far right.
|
||||
3. **Add to GenerateUI (Positive)** appends it to the positive prompt of the workflow loaded in GenerateUI (load a workflow in GenerateUI first).
|
||||
You can also **Copy** it to the clipboard.
|
||||
|
||||
### Searching with the tags of a card or generated image
|
||||
When you select a card or Gallery point, its prompt tags are shown as buttons under **Selection**.
|
||||
Click tags to choose them (click again to deselect) and press **Search with selected** to search the space with them.
|
||||
This helps when you want to find things close to, say, only the hairstyle and clothing of an image.
|
||||
While tags are selected, the **Add** button in **Selection** becomes **Add selected (N)** and adds only those tags to the prompt area.
|
||||
With no tags selected it adds the whole item as before (a card's whole prompt, or all tags of a Gallery point).
|
||||
|
||||
When a Gallery point is selected, **Selection** also shows an **Open in Gallery** button.
|
||||
It switches to the Gallery tab, selects the image's folder, and selects the image in the list (its details open on the right).
|
||||
So that the image is sure to appear, the Gallery's search, favorite, tag and group filters are cleared.
|
||||
A card's whole prompt can be read under **Full prompt**.
|
||||
|
||||
---
|
||||
|
||||
> **Summary of this chapter**
|
||||
> - In the Nodes tab you can list and organize installed nodes, and use them in the WS Library
|
||||
> - Tagger automatically generates tags from images. A model and extra packages must be prepared
|
||||
@@ -231,3 +353,4 @@ you can go from image generation to finished manga entirely inside ComfyUI.
|
||||
> - In the Video tab, you can make videos with supported video-generation models and do simple editing
|
||||
> - "Default Checkpoint" and "Data Management (backup)" in the Settings tab are especially handy
|
||||
> - Combined with Comic Creator by the same author, you can make manga from generated images
|
||||
> - TagSpace lays out your tags, prompts and generated images in 3D by closeness of meaning, so you can browse it and pick tags for a prompt
|
||||
|
||||
@@ -72,6 +72,7 @@ This means **the custom nodes the workflow needs are not installed**.
|
||||
### You changed a setting but the result did not change
|
||||
The most common cause is **forgetting to press Apply**.
|
||||
In GenerateUI, after changing the model or Settings values (steps, size and so on), the change is not reflected in the workflow unless you press **Apply** (Chapter 5, section 5-3).
|
||||
The Switches column at the right of the Settings tab is the exception: it takes effect the moment you click (Chapter 5, section 5-4).
|
||||
Looking at the **Raw JSON** on the right lets you check the values actually reflected.
|
||||
|
||||
### It generates something different from the loaded workflow
|
||||
@@ -82,10 +83,20 @@ Open the Batch subtab and turn off all the Batch Queue column checkboxes.
|
||||
### It generates with a different model than you expected
|
||||
If the model saved in the workflow is not on your computer, it may be replaced automatically with another model.
|
||||
In that case a message says the Checkpoint "..." was not found and was replaced with another model without warning.
|
||||
(From v0.8.7, if you have exactly one model with the same name that differs only in its folder, it is replaced with that one.)
|
||||
In the **Model** subtab of GenerateUI, choose the model you want to use again and press **Apply**.
|
||||
|
||||
If you always want to use the same model, the **Default Checkpoint** in the Settings tab is handy (Chapter 11, section 11-5).
|
||||
|
||||
### The Prompt Translation globe button does not appear or translation fails
|
||||
| Symptom | Cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| No globe button | Prompt Translation is off | In the Settings tab, open **Prompt Translation (native language -> English)** and check **Enable prompt translation in GenerateUI** (Chapter 5, section 5-10) |
|
||||
| "Translation error" appears | Ollama is not running, or the translation model is not in Ollama | Start Ollama. If **Translation model** shows **(not downloaded)**, press **Download** |
|
||||
| Nothing changes ("Nothing to translate") | The prompt is English (ASCII) only | Only parts with letters outside ASCII are translated; English parts stay as they are |
|
||||
| Generation does not start with automatic translation on | Translation failed, so generation stopped instead of sending untranslated text | Same fix as "Translation error". In Lab and Video Plan the reason is shown in the progress line |
|
||||
| The translation is off | Small models (0.8B) can shift the meaning of sentences | Choose 2B or larger in **Translation model** |
|
||||
|
||||
### It does not become the same image even with the same settings
|
||||
Is the seed mode in the generate panel set to **Random**?
|
||||
To make the same image, you need to set the seed to **Workflow** (use the workflow's seed as it is) or **Fixed** (fix it to the value in the Seed field).
|
||||
@@ -134,6 +145,21 @@ For Ollama, download the model with `ollama pull (model name)`, then press **Ref
|
||||
For safety, by default it connects only to AI software inside the same computer.
|
||||
You need to set the environment variable `WFS_ALLOWED_BACKEND_HOSTS` (Chapter 10, section 10-4).
|
||||
|
||||
### The judge model for image checks, or the OCR model, does not appear in the list
|
||||
- The judge model list shows only Ollama **decision models** that can read images; normal chat models are not shown.
|
||||
Update Ollama, `ollama pull` a decision model, then press **Refresh** (Chapter 10, section 10-6).
|
||||
- The OCR model list shows only Ollama models whose name contains "ocr". Press ↻ after getting one (Chapter 10, section 10-8).
|
||||
- Both look at Ollama at the AI TOOL backend address when the backend is Ollama, otherwise at `http://localhost:11434`.
|
||||
|
||||
### Image checks never pass no matter how often it fixes
|
||||
- How well the redraw works depends on the chat model. Make your request shorter and more concrete, or try another model.
|
||||
- The check itself can be wrong too. Look at the image yourself as well.
|
||||
- Too many automatic fix rounds keep using generation time and GPU memory. "0 (fix by button)" is recommended at first.
|
||||
|
||||
### The AI does not read a skill's reference files, or does not generate images from the chat
|
||||
Reading the reference files of an imported skill and generating images from the chat only work with
|
||||
**a model that supports tool calling**. Switch to such a model.
|
||||
|
||||
---
|
||||
|
||||
## 12-5 Troubles with other features
|
||||
@@ -142,6 +168,11 @@ You need to set the environment variable `WFS_ALLOWED_BACKEND_HOSTS` (Chapter 10
|
||||
- Check that a `.onnx` file (and `selected_tags.csv`) is in the `ComfyUI/models/tagger/(model name)/` folder.
|
||||
- Check that the extra Python package (onnxruntime) is installed in **the Python that ComfyUI uses** (Chapter 11, section 11-2).
|
||||
|
||||
### Tagger is very slow (the GPU is not used)
|
||||
If ComfyUI's Python has both the CPU package `onnxruntime` and the GPU package `onnxruntime-gpu`, the GPU stops being used.
|
||||
(Installing other custom nodes can bring in the CPU package.)
|
||||
Stop ComfyUI, then in ComfyUI's Python run `pip uninstall -y onnxruntime onnxruntime-gpu` followed by `pip install onnxruntime-gpu`, and restart ComfyUI (DirectML is not supported).
|
||||
|
||||
### PSD-related features give an error
|
||||
An extra Python package (psd-tools) is needed.
|
||||
Run the installation command shown in the error message in the Python that ComfyUI uses, then restart ComfyUI.
|
||||
@@ -149,9 +180,31 @@ Run the installation command shown in the error message in the Python that Comfy
|
||||
### Semantic search (Gallery) does not work well
|
||||
| Symptom | Cause | What to do |
|
||||
| --- | --- | --- |
|
||||
| The GPU is not detected (settings say no GPU detected, or runs on the CPU) | The onnxruntime in ComfyUI's Python is the CPU version | Stop ComfyUI, then in ComfyUI's Python run `pip uninstall -y onnxruntime onnxruntime-gpu` followed by `pip install onnxruntime-gpu`, and restart ComfyUI (DirectML is not supported) |
|
||||
| "Cannot reach Ollama" is shown | Ollama is not running, or the Ollama URL is wrong | Start Ollama (it is needed while indexing and searching). Check the Ollama URL on the gear button's Model tab |
|
||||
| "The Ollama model is not downloaded" is shown | EmbeddingGemma 2 has not been downloaded | Run `ollama pull embeddinggemma-2:740m` (or `embeddinggemma-2:440m`) |
|
||||
| Indexing sometimes fails with "connection was forcibly closed" or similar | Security software (such as AdGuard) is intercepting Ollama's traffic | Exclude Ollama from filtering. Failed images are processed again by the next "New / changed images only" run |
|
||||
| Gemini says "GEMINI_API_KEY is not set", or the key is rejected | There is no key in `.env`, or the line starts with `#` | Write `GEMINI_API_KEY=key` in the plugin's `.env` (copy `.env.example`) and restart ComfyUI. `.env` is only read at startup |
|
||||
| Search or Similar gives no results | The selected folder has no indexed images | Build the index with the gear button. Choose a folder with a green dot. When you change the model or precision, "rebuild all" |
|
||||
| Search or Similar gives no results | The selected folder has no indexed images | Build the index with the gear button. Choose a folder with a green dot. When you change the model or vector dimension, "rebuild all" |
|
||||
|
||||
### The TagSpace space cannot be built
|
||||
| Symptom | Cause | What to do |
|
||||
| --- | --- | --- |
|
||||
| "EmbeddingGemma 2 in Ollama is required" is shown | Ollama is not running, or the model is missing | Prepare the same way as for semantic search (table above) |
|
||||
| "The Gallery semantic index was made with ..." is shown | The Gallery index was made with Gemini | Choose EmbeddingGemma 2 in the Gallery's semantic settings and rebuild all |
|
||||
| Building takes a very long time | The first build reads image prompts and prepares new tags | Later builds are faster. When using Tagger fill, check that Tagger runs on the GPU ("Tagger is very slow" above) |
|
||||
|
||||
### A partner API (Nano Banana) says authentication failed
|
||||
The cause is a missing Comfy API key in `.env`. Write `COMFY_API_KEY=your key` (issued at platform.comfy.org) in the plugin's `.env` and restart ComfyUI (Chapter 5, section 5-8).
|
||||
`.env` is read only at startup.
|
||||
|
||||
### ComfyUI freezes (no tab responds)
|
||||
On Windows, ComfyUI can stop responding (generation included), for example after switching a Models tab list with many preview images.
|
||||
In the Settings tab, under **Windows Stability (Event Loop)**, turn on **Use the Selector event loop on Windows (restart required)** and restart ComfyUI.
|
||||
|
||||
### Part of a Gallery image looks broken, as if mixed with another image
|
||||
With the **Selector event loop** above turned on, up to v0.9.1 part of an image received by the Gallery preview or Download could be corrupted
|
||||
(the file in the output folder itself is fine). This is fixed in v0.9.2.
|
||||
After updating Workflow Studio, broken images may remain in the browser cache, so reload with **Ctrl + F5**.
|
||||
|
||||
### Images do not appear in the Gallery
|
||||
- Press **Refresh** in the Gallery.
|
||||
|
||||
@@ -16,6 +16,7 @@ The main terms that appear in this book, in alphabetical order.
|
||||
| Civitai | A sharing site for image-generation models | Chapter 6 |
|
||||
| ComfyUI Manager | A management tool for searching for and installing custom nodes | Chapter 2 |
|
||||
| Custom node | A node or extension added to ComfyUI afterwards. Workflow Studio is one of them | Chapter 1 |
|
||||
| Decision model | An AI model that answers questions with numbers such as the probability of "yes". Used to judge image checks | Chapter 10 |
|
||||
| Denoise | How strongly to redraw the original image | Chapter 5 |
|
||||
| Embedding | A small file that teaches a particular word a meaning | Chapter 6 |
|
||||
| Hash value | A "fingerprint"-like value computed from a file's contents. Used to look up models on Civitai | Chapter 6 |
|
||||
@@ -31,7 +32,9 @@ The main terms that appear in this book, in alphabetical order.
|
||||
| Negative prompt | A prompt where you write what you do not want drawn | Chapter 7 |
|
||||
| Node | A "box" arranged on ComfyUI's screen. It handles one process | Chapter 1 |
|
||||
| Node set | Several nodes and their connections saved together | Chapter 9 |
|
||||
| OCR | Reading the text shown in an image | Chapter 10 |
|
||||
| Ollama | Software for running local AI easily | Chapter 10 |
|
||||
| PLaMo 3 Translate | A translation AI model by Preferred Networks (PFN), used by Prompt Translation | Chapter 5 |
|
||||
| Port number | A number for distinguishing which software to communicate with inside one computer | Chapter 2 |
|
||||
| Positive prompt | A prompt where you write what you want drawn | Chapter 7 |
|
||||
| Preset | A pair of positive and negative prompts saved together | Chapter 7 |
|
||||
@@ -41,9 +44,11 @@ The main terms that appear in this book, in alphabetical order.
|
||||
| Scheduler | How the drawing amount is distributed across steps | Chapter 5 |
|
||||
| Seed | The seed of random numbers an image grows from. If the same, you get almost the same image | Chapter 5 |
|
||||
| Semantic search | Searching images by the "meaning" of a sentence or image. Used with the Gallery's S button | Chapter 8 |
|
||||
| Skill | Instructions that tell the AI TOOL chat its role and how to answer. Picking one switches what it is good at | Chapter 10 |
|
||||
| Steps | The number of times drawing is repeated | Chapter 5 |
|
||||
| Style | A mood template added to a prompt | Chapter 7 |
|
||||
| T2I (Text to Image) | Making an image from text (a prompt) | Chapter 5 |
|
||||
| TagSpace | A tab that lays out tags, prompts and generated images in 3D by closeness of meaning | Chapter 11 |
|
||||
| Trigger word | A word to put in the prompt to bring out a LoRA's effect | Chapter 6 |
|
||||
| VAE | The model that finishes an image's colors and fine details | Chapter 6 |
|
||||
| VLM | An AI model that can read images | Chapter 10 |
|
||||
@@ -94,6 +99,10 @@ The main terms that appear in this book, in alphabetical order.
|
||||
| Mouse wheel | Zoom in and out |
|
||||
| Space + drag | Move the view position |
|
||||
| Delete | Delete the selected layer (when there are two or more layers) |
|
||||
| D / E (while Draw is active) | Draw / Erase |
|
||||
| X (while Draw is active) | Swap Color 1 and Color 2 |
|
||||
| [ / ] (while Draw is active) | Make the brush smaller / larger |
|
||||
| B (while Draw is active) | Open or close the brush picker |
|
||||
|
||||
---
|
||||
|
||||
@@ -114,6 +123,7 @@ The main terms that appear in this book, in alphabetical order.
|
||||
| --- | --- | --- |
|
||||
| Civitai (https://civitai.com/) | Getting models, fetching information | Chapter 6 |
|
||||
| Ollama (https://ollama.com/) | Local AI | Chapter 10 |
|
||||
| PLaMo 3 Translate (https://huggingface.co/pfnet/plamo-3-translate-2b-GGUF) | Translation model for Prompt Translation | Chapter 5 |
|
||||
| LM Studio (https://lmstudio.ai/) | Local AI | Chapter 10 |
|
||||
| Lemonade Server (https://lemonade-server.ai/) | Local AI | Chapter 10 |
|
||||
| Eagle (https://eagle.cool/) | Image-management software | Chapter 8 |
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.7 MiB After Width: | Height: | Size: 422 KiB |
@@ -52,6 +52,15 @@ def _maybe_use_selector_event_loop():
|
||||
print("[Workflow Studio] Using WindowsSelectorEventLoopPolicy (windows_selector_event_loop enabled)")
|
||||
except Exception as e:
|
||||
print(f"[WARNING] Workflow Studio: failed to switch to selector event loop: {e}")
|
||||
return
|
||||
# Selector方式ではloop.sendfileがasyncioの_sendfile_fallbackになるが、これは16KBバッファを
|
||||
# 使い回しつつ未送信分をコピーせずトランスポートに積むため、送信が詰まると後続ブロックで
|
||||
# 上書きされ画像等が壊れて届く(Python 3.12で確認)。aiohttpのsendfileを無効化し、
|
||||
# チャンクごとに新しいbytesを読む自前フォールバックを使わせる
|
||||
os.environ["AIOHTTP_NOSENDFILE"] = "1"
|
||||
fileresponse = sys.modules.get("aiohttp.web_fileresponse")
|
||||
if fileresponse is not None:
|
||||
fileresponse.NOSENDFILE = True
|
||||
|
||||
|
||||
_maybe_use_selector_event_loop()
|
||||
|
||||
@@ -1,9 +1,66 @@
|
||||
import os
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ─── パス検査（クライアント由来のパスを resolve() する前に文字列のまま検査する）───
|
||||
# Windows では UNC パス(\\host\share)は resolve()・exists() を呼んだ時点で相手ホストへ
|
||||
# Windows アカウントの資格情報付きで接続してしまうため、チェックはパス操作より先に行う。
|
||||
|
||||
_DRIVE_RE = re.compile(r"^[A-Za-z]:")
|
||||
|
||||
|
||||
def is_unc(value) -> bool:
|
||||
"""ネットワーク共有/デバイスパス(\\\\host\\share, //host/share, \\\\?\\ 等)か。"""
|
||||
return str(value or "").replace("/", "\\").startswith("\\\\")
|
||||
|
||||
|
||||
def is_safe_rel(rel) -> bool:
|
||||
"""相対パスとして安全か。UNC・ドライブ指定・絶対パス・「..」・空セグメントを
|
||||
結合・resolve() より前に文字列のまま拒否する。"""
|
||||
s = str(rel or "")
|
||||
if not s or "\x00" in s:
|
||||
return False
|
||||
if s.startswith(("/", "\\")) or _DRIVE_RE.match(s):
|
||||
return False
|
||||
return not any(part in ("", "..") for part in re.split(r"[/\\]", s))
|
||||
|
||||
|
||||
def resolve_untrusted(value, roots=()) -> Path:
|
||||
"""クライアント由来の絶対パス文字列を検査してから resolve() して返す。
|
||||
|
||||
ファイルシステムに触れない文字列処理(normpath)だけで「..」を正規化・拒否し、
|
||||
UNC は roots のうち UNC のルート配下に文字列上一致する場合のみ許可する
|
||||
(ギャラリーのルートをネットワーク共有に置いている構成だけを通すため)。
|
||||
拒否時は ValueError。"""
|
||||
s = str(value or "")
|
||||
if not s or "\x00" in s:
|
||||
raise ValueError("Access denied: invalid path")
|
||||
p = Path(os.path.normpath(s))
|
||||
if ".." in p.parts:
|
||||
raise ValueError("Access denied: path traversal")
|
||||
if is_unc(s):
|
||||
for root in roots:
|
||||
if root is None or not is_unc(str(root)):
|
||||
continue
|
||||
try:
|
||||
p.relative_to(Path(root)) # 文字列上の照合(解決しない)
|
||||
return p.resolve()
|
||||
except (ValueError, OSError):
|
||||
continue
|
||||
raise ValueError("Access denied: network path")
|
||||
# 受け付ける形はドライブレター絶対パス(Windows)または / 始まり(その他OS)に限る。
|
||||
# ルート相対(\foo)・ドライブ相対(C:foo)・NT デバイス形式(\??\, \.\ 等)・相対パスは拒否する
|
||||
if os.name == "nt":
|
||||
if not re.match(r"^[A-Za-z]:[\\/]", s):
|
||||
raise ValueError("Access denied: invalid path")
|
||||
elif not s.startswith("/"):
|
||||
raise ValueError("Access denied: invalid path")
|
||||
return p.resolve()
|
||||
|
||||
|
||||
# Plugin root directory
|
||||
PLUGIN_DIR = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ import urllib.error
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..config import is_safe_rel
|
||||
from ..services.url_guard import require_backend_url, urlopen as _guarded_urlopen
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -38,6 +39,11 @@ def _resolve_local_path(filename, subfolder, type_name):
|
||||
if not filename:
|
||||
return None
|
||||
|
||||
# subfolder の UNC・ドライブ指定・絶対パス・「..」は、結合より前に文字列のまま拒否する
|
||||
# (別ドライブ/UNC だと下の commonpath が ValueError を投げるため、その前に弾く)
|
||||
if subfolder and not is_safe_rel(subfolder):
|
||||
return None
|
||||
|
||||
full_dir = os.path.abspath(os.path.join(base_dir, subfolder or ""))
|
||||
# Windowsはパス比較で大文字小文字を区別しないため normcase で正規化してから比較する
|
||||
if os.path.commonpath((os.path.normcase(full_dir), os.path.normcase(base_dir))) != os.path.normcase(base_dir):
|
||||
@@ -62,9 +68,9 @@ def _resolve_absolute_svg_path(abs_path):
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
# isfile() も UNC(\\host\share)だと相手ホストへ資格情報付きで接続してしまうため、
|
||||
# 文字列処理(abspath/commonpath)だけで ComfyUI 管理下と確認できるまでファイルシステムに触れない
|
||||
abs_path = os.path.abspath(abs_path)
|
||||
if not os.path.isfile(abs_path):
|
||||
return None
|
||||
|
||||
for type_name in ("output", "temp", "input"):
|
||||
base_dir = folder_paths.get_directory_by_type(type_name)
|
||||
@@ -74,9 +80,9 @@ def _resolve_absolute_svg_path(abs_path):
|
||||
try:
|
||||
# Windowsはパス比較で大文字小文字を区別しないため normcase で正規化してから比較する
|
||||
if os.path.commonpath((os.path.normcase(abs_path), os.path.normcase(base_dir))) == os.path.normcase(base_dir):
|
||||
return abs_path
|
||||
return abs_path if os.path.isfile(abs_path) else None
|
||||
except ValueError:
|
||||
# Windowsで異なるドライブレターの場合 commonpath が例外を投げる
|
||||
# Windowsで異なるドライブレターやUNCの場合 commonpath が例外を投げる
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
@@ -59,7 +59,10 @@ _init_allowed_root()
|
||||
|
||||
# Windowsのレジストリ依存で音声のMIMEが返らない/不統一なため明示登録する（<audio>の再生に必要）
|
||||
for _ext, _mime in ((".wav", "audio/wav"), (".mp3", "audio/mpeg"), (".ogg", "audio/ogg"),
|
||||
(".opus", "audio/ogg"), (".m4a", "audio/mp4"), (".flac", "audio/flac")):
|
||||
(".opus", "audio/ogg"), (".m4a", "audio/mp4"), (".flac", "audio/flac"),
|
||||
(".glb", "model/gltf-binary"), (".vrm", "model/gltf-binary"), (".gltf", "model/gltf+json"),
|
||||
(".ply", "application/octet-stream"), (".spz", "application/octet-stream"),
|
||||
(".bin", "application/octet-stream")):
|
||||
mimetypes.add_type(_mime, _ext)
|
||||
|
||||
|
||||
@@ -73,11 +76,14 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_post("/wfm/gallery/semantic/similar", semantic_similar)
|
||||
app.router.add_post("/wfm/gallery/semantic/clear", semantic_clear)
|
||||
app.router.add_post("/wfm/gallery/semantic/test", semantic_test)
|
||||
app.router.add_post("/wfm/gallery/semantic/options", semantic_options)
|
||||
app.router.add_get("/wfm/gallery/images", list_images)
|
||||
app.router.add_get("/wfm/gallery/image/meta", get_image_meta)
|
||||
app.router.add_get("/wfm/gallery/image/workflow", get_image_workflow)
|
||||
app.router.add_get("/wfm/gallery/image/serve", serve_image)
|
||||
app.router.add_get("/wfm/gallery/image/thumb", serve_thumb)
|
||||
app.router.add_post("/wfm/gallery/image/thumb3d", save_model3d_thumb)
|
||||
app.router.add_get("/wfm/gallery/model3d/asset", serve_model3d_asset)
|
||||
app.router.add_post("/wfm/gallery/image/meta", save_image_meta)
|
||||
app.router.add_get("/wfm/gallery/image-prompt/root", get_image_prompt_root)
|
||||
app.router.add_post("/wfm/gallery/image/image-prompt", save_image_prompt)
|
||||
@@ -112,7 +118,7 @@ def setup_routes(app: web.Application):
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# セマンティック検索 (Gemini Embedding 2)
|
||||
# セマンティック検索 (Gemini Embedding 2 / Ollama EmbeddingGemma 2)
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _semantic_error(e: SemanticError) -> web.Response:
|
||||
@@ -164,9 +170,9 @@ async def semantic_folders(request: web.Request) -> web.Response:
|
||||
|
||||
|
||||
def _apply_local_options(body: dict) -> None:
|
||||
"""リクエストに実行デバイス/スレッド数が含まれていれば反映・保存する"""
|
||||
if "device" in body:
|
||||
_semantic.set_local_options(str(body.get("device", "auto")), _num(body, "threads", 0, 0, 256))
|
||||
"""リクエストに Ollama の接続先/モデルサイズが含まれていれば反映・保存する"""
|
||||
if "ollama_url" in body:
|
||||
_semantic.set_local_options(str(body.get("ollama_url", "")), str(body.get("ollama_tag", "")))
|
||||
|
||||
|
||||
async def semantic_index(request: web.Request) -> web.Response:
|
||||
@@ -180,7 +186,6 @@ async def semantic_index(request: web.Request) -> web.Response:
|
||||
"all" if body.get("mode") == "all" else "missing",
|
||||
str(body.get("model", SEMANTIC_DEFAULT_MODEL)),
|
||||
_num(body, "dim", DEFAULT_DIM, 0, 100000),
|
||||
_num(body, "soft", 280, 0, 100000),
|
||||
)
|
||||
return web.json_response({"ok": True})
|
||||
except SemanticError as e:
|
||||
@@ -386,6 +391,37 @@ async def serve_thumb(request: web.Request) -> web.Response:
|
||||
})
|
||||
|
||||
|
||||
async def save_model3d_thumb(request: web.Request) -> web.Response:
|
||||
"""POST /wfm/gallery/image/thumb3d — ブラウザが描画した3Dモデルのサムネイルを保存する"""
|
||||
try:
|
||||
body = await request.json()
|
||||
path = body.get("path", "")
|
||||
image = body.get("image", "")
|
||||
if not path or not image:
|
||||
return web.json_response({"ok": False, "error": "path and image required"}, status=400)
|
||||
loop = asyncio.get_event_loop()
|
||||
result = await loop.run_in_executor(None, _service.save_model3d_thumbnail, path, image)
|
||||
return web.json_response(result, status=200 if result.get("ok") else 400)
|
||||
except Exception as e:
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def serve_model3d_asset(request: web.Request) -> web.Response:
|
||||
"""GET /wfm/gallery/model3d/asset — .gltfが参照する外部ファイル(.bin/テクスチャ)を配信する"""
|
||||
path = request.rel_url.query.get("path", "")
|
||||
uri = request.rel_url.query.get("uri", "")
|
||||
if not path or not uri:
|
||||
return web.Response(status=400)
|
||||
asset = _service.serve_model3d_asset(path, uri)
|
||||
if asset is None:
|
||||
return web.Response(status=404)
|
||||
content_type, _ = mimetypes.guess_type(str(asset))
|
||||
return web.FileResponse(asset, headers={
|
||||
"Content-Type": content_type or "application/octet-stream",
|
||||
"Cache-Control": "max-age=3600",
|
||||
})
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# バルク操作
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
@@ -848,3 +884,16 @@ async def get_psd_layers(request: web.Request) -> web.Response:
|
||||
except Exception as e:
|
||||
logger.error("Error reading PSD layers: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_options(request: web.Request) -> web.Response:
|
||||
"""Ollama の接続先/モデルサイズを保存して、新しい状態を返す"""
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
await asyncio.to_thread(_apply_local_options, body)
|
||||
return web.json_response(await asyncio.to_thread(_semantic.status))
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_options error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from aiohttp import web
|
||||
|
||||
from ..services.settings_service import SettingsService
|
||||
from ..config import DATA_DIR
|
||||
from ..config import DATA_DIR, resolve_untrusted
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -153,17 +153,20 @@ async def handle_result(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
result_path = body.get("result_path", "")
|
||||
if not result_path or not os.path.exists(result_path):
|
||||
return web.json_response({"error": f"Result file not found: {result_path}"}, status=404)
|
||||
|
||||
# Path traversal guard: only allow files inside gmic_temp
|
||||
# Path traversal guard: only allow files inside gmic_temp.
|
||||
# UNC(\\host\share)は exists()/resolve() を呼んだ時点で相手ホストへ資格情報付きで
|
||||
# 接続してしまうため、パス操作より先に文字列のまま検査する(resolve_untrusted)。
|
||||
allowed_dir = (Path(DATA_DIR) / "gmic_temp").resolve()
|
||||
try:
|
||||
Path(result_path).resolve().relative_to(allowed_dir)
|
||||
resolved = resolve_untrusted(result_path)
|
||||
resolved.relative_to(allowed_dir)
|
||||
except ValueError:
|
||||
return web.json_response({"error": "Access denied: path outside gmic_temp"}, status=403)
|
||||
if not resolved.is_file():
|
||||
return web.json_response({"error": f"Result file not found: {result_path}"}, status=404)
|
||||
|
||||
with open(result_path, "rb") as f:
|
||||
with open(resolved, "rb") as f:
|
||||
img_bytes = f.read()
|
||||
|
||||
ext = os.path.splitext(result_path)[1].lower()
|
||||
|
||||
@@ -12,7 +12,7 @@ from pathlib import Path
|
||||
from aiohttp import web
|
||||
|
||||
from ..services.settings_service import SettingsService
|
||||
from ..config import DEFAULT_WORKFLOWS_DIR, DATA_DIR, VIDEO_EDIT_PROJECT_DIR
|
||||
from ..config import DEFAULT_WORKFLOWS_DIR, DATA_DIR, VIDEO_EDIT_PROJECT_DIR, is_safe_rel
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -397,6 +397,10 @@ def _add_video_edit_media_to_zip(zf: zipfile.ZipFile) -> None:
|
||||
input_dir = _comfyui_input_dir().resolve()
|
||||
seen = set()
|
||||
for subfolder, filename in _iter_video_edit_media_refs():
|
||||
# プロジェクトファイルは細工できるため、UNC・絶対パス・「..」は結合・resolve() の前に
|
||||
# 文字列のまま拒否する(Windows では UNC を resolve() するだけで相手ホストへ接続する)
|
||||
if not is_safe_rel(filename) or (subfolder and not is_safe_rel(subfolder)):
|
||||
continue
|
||||
src = (input_dir / subfolder / filename).resolve()
|
||||
try:
|
||||
rel = src.relative_to(input_dir) # a crafted project file must not reach outside input/
|
||||
@@ -466,9 +470,15 @@ def _apply_full_backup_zip(zip_source) -> dict:
|
||||
summary["skipped"].append(info.filename)
|
||||
continue
|
||||
if info.filename.startswith(_VIDEO_MEDIA_PREFIX):
|
||||
media_rel = info.filename[len(_VIDEO_MEDIA_PREFIX):]
|
||||
# ZIP エントリ名は細工できるため、UNC・絶対パス・「..」は結合・resolve() の前に
|
||||
# 文字列のまま拒否する(Windows では UNC を resolve() するだけで相手ホストへ接続する)
|
||||
if not is_safe_rel(media_rel):
|
||||
summary["skipped"].append(info.filename)
|
||||
continue
|
||||
if input_base is None:
|
||||
input_base = _comfyui_input_dir().resolve()
|
||||
target = (input_base / info.filename[len(_VIDEO_MEDIA_PREFIX):]).resolve()
|
||||
target = (input_base / media_rel).resolve()
|
||||
try:
|
||||
target.relative_to(input_base)
|
||||
except ValueError:
|
||||
@@ -482,6 +492,9 @@ def _apply_full_backup_zip(zip_source) -> dict:
|
||||
shutil.copyfileobj(src, dst)
|
||||
summary["media_restored"].append(info.filename)
|
||||
continue
|
||||
if not is_safe_rel(info.filename):
|
||||
summary["skipped"].append(info.filename)
|
||||
continue
|
||||
target = (base / info.filename).resolve()
|
||||
try:
|
||||
target.relative_to(base)
|
||||
|
||||
@@ -18,6 +18,9 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_get("/api/wfm/skills/content", handle_get_content)
|
||||
app.router.add_post("/api/wfm/skills/save", handle_save)
|
||||
app.router.add_post("/api/wfm/skills/delete", handle_delete)
|
||||
app.router.add_post("/api/wfm/skills/restore-builtin", handle_restore_builtin)
|
||||
app.router.add_post("/api/wfm/skills/import", handle_import)
|
||||
app.router.add_get("/api/wfm/skills/resource", handle_get_resource)
|
||||
|
||||
|
||||
async def handle_list(request: web.Request) -> web.Response:
|
||||
@@ -62,3 +65,52 @@ async def handle_delete(request: web.Request) -> web.Response:
|
||||
except Exception as e:
|
||||
logger.error("Error deleting skill: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_restore_builtin(request: web.Request) -> web.Response:
|
||||
"""Overwrite a built-in skill with the shipped version (user-confirmed in the UI)."""
|
||||
try:
|
||||
body = await request.json()
|
||||
filename = body.get("filename", "")
|
||||
result = await asyncio.to_thread(_service.restore_builtin, filename)
|
||||
return web.json_response({"status": "ok", "file": result})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error restoring built-in skill: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_import(request: web.Request) -> web.Response:
|
||||
"""Import an Agent Skills folder (.zip) or a single .md skill (multipart field "file")."""
|
||||
try:
|
||||
form = await request.post()
|
||||
upload = form.get("file")
|
||||
if upload is None or not hasattr(upload, "file"):
|
||||
return web.json_response({"error": "no file"}, status=400)
|
||||
data = upload.file.read(20 * 1024 * 1024 + 1)
|
||||
if len(data) > 20 * 1024 * 1024:
|
||||
return web.json_response({"error": "file too large (max 20 MB)"}, status=400)
|
||||
result = await asyncio.to_thread(_service.import_skill, upload.filename or "", data)
|
||||
return web.json_response({"status": "ok", **result})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error importing skill: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_get_resource(request: web.Request) -> web.Response:
|
||||
"""Read one reference file of a folder skill (?skill=<folder>&path=<relative path>)."""
|
||||
folder = request.rel_url.query.get("skill", "")
|
||||
rel = request.rel_url.query.get("path", "")
|
||||
try:
|
||||
content = await asyncio.to_thread(_service.get_resource, folder, rel)
|
||||
return web.json_response({"content": content})
|
||||
except FileNotFoundError:
|
||||
return web.json_response({"error": "not found"}, status=404)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error reading skill resource: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
"""
|
||||
TagSpace Routes - タグ/ImagePromptカード/Gallery画像を3D潜在空間に並べるタブ用API
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..config import DATA_DIR
|
||||
from ..services.tagspace_service import TagSpaceService, DEFAULT_MAX_TAGS, DEFAULT_BG, MAX_TAGS_LIMIT, MIN_COUNT_LIMIT
|
||||
from ..services.semantic_service import SemanticError
|
||||
from .gallery_routes import _service as _gallery, _semantic, _semantic_error, _num, _json_body
|
||||
from .tagger_routes import _svc as _tagger
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_tagspace = TagSpaceService(DATA_DIR, _gallery, _semantic, _tagger)
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_get("/wfm/tagspace/status", tagspace_status)
|
||||
app.router.add_get("/wfm/tagspace/space", tagspace_space)
|
||||
app.router.add_post("/wfm/tagspace/build", tagspace_build)
|
||||
app.router.add_post("/wfm/tagspace/cancel", tagspace_cancel)
|
||||
app.router.add_post("/wfm/tagspace/search", tagspace_search)
|
||||
app.router.add_post("/wfm/tagspace/neighbors", tagspace_neighbors)
|
||||
|
||||
|
||||
async def _call(fn, *args) -> web.Response:
|
||||
try:
|
||||
return web.json_response(await asyncio.to_thread(fn, *args))
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("tagspace %s error: %s", getattr(fn, "__name__", fn), e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def tagspace_status(request: web.Request) -> web.Response:
|
||||
return await _call(_tagspace.status)
|
||||
|
||||
|
||||
async def tagspace_space(request: web.Request) -> web.Response:
|
||||
return await _call(_tagspace.space)
|
||||
|
||||
|
||||
async def tagspace_build(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
# max_tags=0 は全タグ(サービス側で ALL_TAGS_LIMIT まで)
|
||||
max_tags = _num(body, "max_tags", DEFAULT_MAX_TAGS, 0, MAX_TAGS_LIMIT)
|
||||
bg = _num(body, "background", DEFAULT_BG, 0, 20000)
|
||||
min_count = _num(body, "min_count", 1, 1, MIN_COUNT_LIMIT)
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
tagger_model = str(body.get("tagger_model", "") or "")
|
||||
tagger_scope = str(body.get("tagger_scope", "background") or "background")
|
||||
wildcard = body.get("wildcard", True) is not False
|
||||
return await _call(lambda: (_tagspace.start_build(max_tags, bg, tagger_model, tagger_scope, min_count, wildcard),
|
||||
{"ok": True})[1])
|
||||
|
||||
|
||||
async def tagspace_cancel(request: web.Request) -> web.Response:
|
||||
_tagspace.cancel()
|
||||
return web.json_response({"ok": True})
|
||||
|
||||
|
||||
async def tagspace_search(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
k = _num(body, "k", 20, 1, 200)
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
return await _call(_tagspace.search, str(body.get("query", "")), k)
|
||||
|
||||
|
||||
async def tagspace_neighbors(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
k = _num(body, "k", 20, 1, 200)
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
return await _call(_tagspace.neighbors, str(body.get("id", "")), k)
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Prompt translation routes (GenerateUI native-language prompt input)."""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..services import translate_service as svc
|
||||
from ..services.url_guard import require_backend_url
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_get("/api/wfm/translate/config", handle_config)
|
||||
app.router.add_get("/api/wfm/translate/models", handle_models)
|
||||
app.router.add_post("/api/wfm/translate", handle_translate)
|
||||
app.router.add_post("/api/wfm/translate/pull", handle_pull)
|
||||
app.router.add_post("/api/wfm/translate/warmup", handle_warmup)
|
||||
app.router.add_post("/api/wfm/translate/unload", handle_unload)
|
||||
|
||||
|
||||
async def handle_config(request: web.Request) -> web.Response:
|
||||
"""GET /api/wfm/translate/config - effective settings + PLaMo presets."""
|
||||
cfg = await asyncio.to_thread(svc.get_config)
|
||||
return web.json_response({"config": cfg, "presets": svc.PLAMO_PRESETS})
|
||||
|
||||
|
||||
async def handle_models(request: web.Request) -> web.Response:
|
||||
"""GET /api/wfm/translate/models - models installed in the translation Ollama."""
|
||||
try:
|
||||
models = await asyncio.to_thread(svc.list_models)
|
||||
return web.json_response({"status": "success", "models": models, "presets": svc.PLAMO_PRESETS})
|
||||
except Exception as e:
|
||||
logger.warning("Translate models error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e), "presets": svc.PLAMO_PRESETS}, status=502)
|
||||
|
||||
|
||||
async def handle_translate(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate {texts: [...], model?, source_lang?} -> {translations: [...]}"""
|
||||
try:
|
||||
body = await request.json()
|
||||
texts = body.get("texts")
|
||||
translations, cached = await asyncio.to_thread(
|
||||
svc.translate_texts, texts, body.get("model") or None, body.get("source_lang"),
|
||||
)
|
||||
return web.json_response({"status": "success", "translations": translations, "cached": cached})
|
||||
except ValueError as e:
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Translate error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
|
||||
|
||||
async def handle_warmup(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate/warmup - preload the model when CPU-resident mode is on."""
|
||||
try:
|
||||
loaded = await asyncio.to_thread(svc.warmup)
|
||||
return web.json_response({"status": "success", "loaded": loaded})
|
||||
except Exception as e:
|
||||
logger.warning("Translate warmup error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
|
||||
|
||||
async def handle_unload(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate/unload - release the resident model (device switch / disable)."""
|
||||
try:
|
||||
cfg = await asyncio.to_thread(svc.get_config)
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
await asyncio.to_thread(svc.unload, base_url, cfg["prompt_translate_model"])
|
||||
return web.json_response({"status": "success"})
|
||||
except Exception as e:
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
|
||||
|
||||
async def handle_pull(request: web.Request) -> web.Response:
|
||||
"""POST /api/wfm/translate/pull {model} - pull one of the PLaMo presets into Ollama."""
|
||||
try:
|
||||
body = await request.json()
|
||||
model = body.get("model", "")
|
||||
if model not in svc.PLAMO_PRESETS:
|
||||
return web.json_response({"status": "error", "message": "only PLaMo presets can be pulled"}, status=400)
|
||||
cfg = await asyncio.to_thread(svc.get_config)
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
data = await asyncio.to_thread(
|
||||
svc._post_json, f"{base_url}/api/pull", {"model": model, "stream": False}, 3600,
|
||||
)
|
||||
if data.get("error"):
|
||||
return web.json_response({"status": "error", "message": data["error"]}, status=502)
|
||||
return web.json_response({"status": "success", "result": data.get("status", "")})
|
||||
except Exception as e:
|
||||
logger.error("Translate pull error: %s", e)
|
||||
return web.json_response({"status": "error", "message": str(e)}, status=502)
|
||||
@@ -21,6 +21,8 @@ _edit_project_service = VideoEditProjectService(VIDEO_EDIT_PROJECT_DIR)
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_post("/api/wfm/video/frame/save-to-output", handle_save_frame)
|
||||
app.router.add_post("/api/wfm/video/to-gif", handle_to_gif)
|
||||
app.router.add_post("/api/wfm/video/asset/register", handle_register_asset)
|
||||
app.router.add_post("/api/wfm/video/edit/convert-audio", handle_convert_audio)
|
||||
app.router.add_get("/api/wfm/video/edit/probe", handle_probe_video)
|
||||
app.router.add_post("/api/wfm/video/edit/overlay-text", handle_overlay_text)
|
||||
|
||||
@@ -49,6 +51,45 @@ async def handle_save_frame(request: web.Request) -> web.Response:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_register_asset(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
filename = body.get("filename", "")
|
||||
if not filename:
|
||||
return web.json_response({"error": "filename required"}, status=400)
|
||||
result = await asyncio.to_thread(
|
||||
_service.register_asset, filename, body.get("subfolder", ""), body.get("type", "input")
|
||||
)
|
||||
return web.json_response({"status": "ok", **result})
|
||||
except FileNotFoundError as e:
|
||||
return web.json_response({"error": str(e)}, status=404)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error registering video asset: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_convert_audio(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
filename = body.get("filename", "")
|
||||
if not filename:
|
||||
return web.json_response({"error": "filename required"}, status=400)
|
||||
# PyAVの変換はCPU負荷が高いのでスレッドで実行する(イベントループを止めない)
|
||||
result = await asyncio.to_thread(
|
||||
_service.convert_audio_to_output, filename, body.get("subfolder", ""), body.get("type", "input"), body
|
||||
)
|
||||
return web.json_response({"ok": True, **result})
|
||||
except FileNotFoundError as e:
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=404)
|
||||
except ValueError as e:
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
logger.error("Error converting clip audio: %s", e)
|
||||
return web.json_response({"ok": False, "error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def handle_to_gif(request: web.Request) -> web.Response:
|
||||
try:
|
||||
body = await request.json()
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
name: Image to Prompt
|
||||
description: Looks at an attached image and writes a prompt that recreates it, in the style the current GenerateUI model understands best.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: prompt
|
||||
tools: []
|
||||
temperature: 0.3
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
You reverse-engineer images into text-to-image prompts. The user attaches an image (in the TOOLS pane, which also attaches it to the chat) and you write a prompt that would recreate it with the model in their current workflow. This only works with a vision-capable chat model.
|
||||
|
||||
If no image is attached to the user's message, do not invent one: reply in one or two sentences asking them to drop an image into the TOOLS pane (PNG / JPG / WebP) and send their message again.
|
||||
|
||||
## Step 1 - look carefully, describe only what is visible
|
||||
Go through the image in this order and note only what you can actually see - never guess names of characters, people, brands or artists:
|
||||
1. Main subject(s): count, kind, apparent age group, hair, eyes, clothing and accessories with colors.
|
||||
2. Pose, action, expression, gaze direction.
|
||||
3. Setting and background, important objects.
|
||||
4. Lighting (direction, softness, time of day), color palette.
|
||||
5. Framing: shot size (close-up, upper body, full body, wide), camera angle, lens feel (shallow depth of field, wide angle).
|
||||
6. Medium and style: photo, anime illustration, 3D render, oil painting, flat vector, etc.
|
||||
7. Any readable text in the image - copy it exactly.
|
||||
|
||||
## Step 2 - pick the prompt style from the "Current workflow context"
|
||||
- NATURAL style (Qwen-Image, FLUX, Z-Image, HiDream, SD3 and other T5 / LLM text-encoder models): fluent English sentences in the order above, 40-120 words, no quality tags; text to render goes in double quotes. Leave negative empty when CFG is about 1.
|
||||
- TAG style (SDXL anime checkpoints: Illustrious, NoobAI, Animagine, Pony): Danbooru tags in the order quality -> count -> appearance -> expression/pose -> background -> lighting -> composition. Quality tags: Pony `score_9, score_8_up, score_7_up`; others `masterpiece, best quality, very aesthetic, absurdres`. Negative: `worst quality, low quality, lowres, bad anatomy, bad hands, jpeg artifacts, watermark, signature`.
|
||||
- MIXED style (realistic SDXL / SD1.5): short comma-separated descriptive phrases plus camera and lighting terms.
|
||||
- No context: use NATURAL.
|
||||
- Write every positive and negative prompt from scratch from the user's request and the style rules here. Only when the context contains "Prompts currently typed into the workflow" (it appears when the user refers to the current prompt) edit or extend those as the user asks; even then, keep only the parts the user wants to keep.
|
||||
|
||||
## Output format (always)
|
||||
1. One line starting with "Style:" naming the style and why (e.g. "Style: TAG - workflow uses an Illustrious checkpoint").
|
||||
2. The prompt in a fenced block labeled `prompt`:
|
||||
|
||||
```prompt
|
||||
positive: <the full prompt on one line>
|
||||
negative: <negative prompt on one line, or empty>
|
||||
```
|
||||
|
||||
3. One line "Aspect: W:H" matching the image's orientation (e.g. "Aspect: 2:3 (portrait)").
|
||||
|
||||
Reply to the user in their language outside the block; the prompt itself is English. If the user asks for changes ("make it night", "remove the hat"), output the complete updated block again.
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: Skill Creator
|
||||
description: Interactively interviews you and drafts a new WFS chat skill (.md system prompt) ready to save.
|
||||
metadata:
|
||||
wfs-version: 3
|
||||
tools: []
|
||||
temperature: 0.5
|
||||
---
|
||||
|
||||
You are a skill-design assistant embedded in ComfyUI Workflow Studio's AI TOOL Chat pane. Your job is to interactively help the user design a NEW "skill" - a markdown file with frontmatter followed by system-prompt instructions. Once saved, a skill can be selected from a dropdown and is silently prepended to future chat messages as a system prompt.
|
||||
|
||||
## How to conduct the interview
|
||||
- Ask ONE focused question at a time, never a bulleted list of questions. Wait for the user's answer before asking the next one.
|
||||
- Cover, in this order, only what is not already clear from context:
|
||||
1. What task or domain should the skill specialize in (what should the model do differently once this skill is active)?
|
||||
2. What output format is expected - plain prose, a specific fenced code block, a fixed structure, a length limit? If the result is an image prompt, an SVG, or another skill, suggest the matching Workflow Studio output type below so the user gets action buttons for free.
|
||||
3. Any hard constraints or things to avoid (tone, forbidden content, security concerns, things past attempts got wrong)?
|
||||
4. Would a short worked example help smaller/local models follow the pattern more reliably? If yes, ask what a good example input/output looks like, or offer to draft one.
|
||||
- Keep it short - 3 to 5 questions is usually enough. If the user says "just generate it", "that's enough", or similar at any point, stop asking and produce the best skill you can from what you already have.
|
||||
|
||||
## Workflow Studio output types (the fenced blocks the app recognizes)
|
||||
- ```prompt block - an image-generation prompt in this exact shape (the app shows "Apply to GenerateUI" / "Generate with this prompt" buttons):
|
||||
positive: <prompt text>
|
||||
negative: <negative prompt text, may be empty>
|
||||
- ```svg block - one complete `<svg>` element (the app previews it, can save it to the Gallery, and auto-checks it).
|
||||
- ```skill block - a complete skill file (the app shows a "Save as new skill" button).
|
||||
|
||||
## Optional metadata a skill can declare
|
||||
All keys are optional and go under `metadata:` in the frontmatter. Only include the ones that matter for this skill.
|
||||
- `output`: `prompt` or `svg` - the app checks that the reply contains that block and automatically asks the model to fix it once if it is missing or broken.
|
||||
- `tools`: `[]` to never offer image generation to the model, or `[generate_image]` to let it start ComfyUI generation by itself (only when the user turned on the image-generation toggle).
|
||||
- `temperature`: a number such as `0.3` (precise, repeatable) to `0.9` (creative).
|
||||
- `think`: `true` / `false` to force the model's thinking mode on or off for this skill.
|
||||
- `context`: a list of facts sent with every message:
|
||||
- `workflow`: a summary of the workflow loaded in GenerateUI (model files, prompt nodes, CFG, resolution). The prompt text currently in the workflow is added only when the user's message refers to it ("improve the current prompt").
|
||||
- `current-prompt`: always include the prompt text currently in the workflow (for skills whose job is to edit that prompt).
|
||||
- `workflow-detail`: every node with its settings, checked against this ComfyUI install (missing nodes / files, broken links) - for troubleshooting skills.
|
||||
- `logs`: the last execution error and the latest error block of the ComfyUI log.
|
||||
|
||||
## Output format for the finished skill
|
||||
- Once you have enough information, output the complete skill file in a single fenced code block labeled `skill` (```skill ... ```). Never mix prose inside that block - explanation goes outside it.
|
||||
- The block must start with frontmatter in this shape (drop the `metadata:` part entirely if no key is needed):
|
||||
---
|
||||
name: <short display name, shown in the skill dropdown>
|
||||
description: <one sentence, shown as a tooltip in the dropdown>
|
||||
metadata:
|
||||
output: <prompt or svg>
|
||||
temperature: <number>
|
||||
---
|
||||
- After the frontmatter, write clear, imperative system-prompt instructions for the target skill: what role the model should take, concrete output-format rules, hard constraints, and - if useful - one short worked example following a "request -> ideal output" shape.
|
||||
- Write the instructions in English unless the user asks otherwise (small local models follow English instructions most reliably), but tell the skill to reply in the user's language.
|
||||
- After the code block, tell the user in one line that they can click the "Save as new skill" button below the message to add it, or copy the block manually into a new skill.
|
||||
|
||||
## Improving an existing skill
|
||||
If the user pastes an existing skill and describes what went wrong, do not interview from scratch: identify the cause in one or two sentences, then output the complete revised skill in a ```skill block, keeping its name unless asked to change it.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: Storyboard to Lab
|
||||
description: Turns a story or scene list into a Lab plan - one prompt per scene with consistent characters - that opens in GenerateUI > Lab with one click.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: labplan
|
||||
tools: []
|
||||
temperature: 0.6
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
You are a storyboard artist for ComfyUI Workflow Studio. The user describes a story, a sequence, or a list of shots (in any language). You split it into scenes and write one text-to-image prompt per scene, packaged as a Lab plan: the Lab tab runs the scenes as consecutive iterations of the workflow loaded in GenerateUI.
|
||||
|
||||
## Before writing
|
||||
- If the user did not say how many scenes, use 4-8 that cover the story's key beats. Never exceed 32.
|
||||
- If an essential detail is missing and would change every scene (e.g. who the main character is), ask ONE short question first. Otherwise just make sensible choices.
|
||||
|
||||
## Consistency (the most important part)
|
||||
- Write a fixed "character sheet" phrase for each recurring character (hair, eyes, age range, outfit with colors) and paste it IDENTICALLY into every scene where that character appears. Do the same for a recurring place or style.
|
||||
- Keep one art style / medium for the whole plan and repeat it in every scene.
|
||||
- Use the same "seed" for the whole plan (any integer, e.g. 123456789) so the look stays stable; use null only if the user wants variety.
|
||||
|
||||
## Prompt style per scene
|
||||
Use the "Current workflow context" to choose the style:
|
||||
- NATURAL (Qwen-Image, FLUX, Z-Image, HiDream, SD3, other T5 / LLM-encoder models): English sentences - character sheet, action in this scene, setting, lighting, camera framing, style. 40-100 words, no quality tags. Negative empty when CFG is about 1.
|
||||
- TAG (Illustrious, NoobAI, Animagine, Pony): Danbooru tags - quality tags (Pony: `score_9, score_8_up, score_7_up`; others: `masterpiece, best quality, very aesthetic, absurdres`), count tag, character sheet tags, scene action, background, lighting, framing. Negative: `worst quality, low quality, lowres, bad anatomy, bad hands, jpeg artifacts, watermark, signature`.
|
||||
- MIXED (realistic SDXL / SD1.5): short descriptive phrases plus camera and lighting terms.
|
||||
- No context: NATURAL.
|
||||
- Write every positive and negative prompt from scratch from the user's request and the style rules here. Only when the context contains "Prompts currently typed into the workflow" (it appears when the user refers to the current prompt) edit or extend those as the user asks; even then, keep only the parts the user wants to keep.
|
||||
|
||||
Vary the camera framing across scenes (wide establishing shot, medium shot, close-up) like a real storyboard.
|
||||
|
||||
## Output format (always)
|
||||
1. A short numbered list of the scenes in the user's language (one line each).
|
||||
2. The plan in one fenced block labeled `labplan` containing valid JSON only (double quotes, no comments, no trailing commas):
|
||||
|
||||
```labplan
|
||||
{
|
||||
"name": "short_plan_name",
|
||||
"note": "one-line summary of the story",
|
||||
"seed": 123456789,
|
||||
"negative": "default negative prompt for every scene, written from the style rules above (empty string for CFG about 1)",
|
||||
"scenes": [
|
||||
{ "title": "Scene title", "positive": "full prompt for this scene" },
|
||||
{ "title": "Scene title", "positive": "full prompt", "negative": "only if this scene needs a different negative" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
3. One line telling the user to press "Open in Lab" below the message, check the plan, and press Run in the Lab tab (GenerateUI must have the workflow loaded).
|
||||
|
||||
When the user asks for changes ("make scene 3 at night", "add a scene"), output the complete updated `labplan` block again, keeping the character sheets identical.
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
name: SVG Icon Generator
|
||||
description: Produces clean, minimal, valid SVG icons and simple illustrations.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: svg
|
||||
tools: []
|
||||
temperature: 0.4
|
||||
---
|
||||
|
||||
You are an expert SVG icon and illustration designer. When the user asks for an icon, logo, pictogram, diagram, character, animal, or any vector graphic, respond with valid, clean SVG markup.
|
||||
|
||||
## Composition process (do this before writing markup - skipping this step is the main cause of unrecognizable output)
|
||||
- Break the subject down into 3-8 distinct parts that make it recognizable (e.g. a bird: body, head, beak, eye, wing, tail; a house: wall, roof, door, window). Drop parts that are not essential.
|
||||
- Decide each part's approximate position, size, and draw order within the viewBox (elements listed later in the markup are drawn on top).
|
||||
- Pick the simplest shape that captures each part's silhouette: `<circle>`/`<ellipse>` for round parts, `<rect>` for rectangular parts, `<polygon>` for angular parts (beaks, ears, leaves, roofs). For a rounded organic outline that no primitive can approximate well (a body silhouette, a wing, a leaf edge), use a short `<path>` with 2-4 smooth curve commands (`Q` or `C`) instead of a plain circle - a few deliberate curve points beat a distorted primitive.
|
||||
- Never fall back to drawing an unrelated blob plus a few disconnected decorative lines when a subject is hard to render - always decompose it into the named parts above instead, even if each part is crude.
|
||||
- Write the part list in one short line before the code block (e.g. "Parts: body, head, beak, eye, wing").
|
||||
|
||||
## Style presets
|
||||
If the user names a style, follow it; otherwise use "flat".
|
||||
- **line**: outlines only - `fill="none"`, `stroke="currentColor"`, `stroke-width="2"`, `stroke-linecap="round"`, `stroke-linejoin="round"`. No filled areas except tiny dots (eyes).
|
||||
- **flat**: solid fills, 2-4 explicit hex colors, no strokes or only thin dark outlines.
|
||||
- **duotone**: exactly two colors - one main hex color and the same color with `opacity="0.4"` for secondary parts.
|
||||
- **mono**: a single `fill="currentColor"` for everything, shapes separated by negative space.
|
||||
|
||||
## Output rules
|
||||
- Always wrap the SVG markup in a fenced code block labeled `svg`, e.g. ```svg ... ```.
|
||||
- Keep any explanation brief and place it OUTSIDE the code block, never mixed inside the SVG.
|
||||
- Output exactly one `<svg>...</svg>` element per code block.
|
||||
- When the user asks for a change, output the complete updated SVG again (never a partial diff).
|
||||
|
||||
## Structure rules
|
||||
- Always include `xmlns="http://www.w3.org/2000/svg"` and a `viewBox` attribute (e.g. `viewBox="0 0 24 24"`). Do not hardcode large pixel `width`/`height`; if you set them, make them match the viewBox aspect ratio.
|
||||
- Use a small, round coordinate system (0-24 or 0-100). Keep every coordinate inside the viewBox.
|
||||
- Round coordinate values to at most 2 decimal places.
|
||||
- Keep every `<path>` short (at most about 6 commands) - long, intricate paths are where quality breaks down most often.
|
||||
- Group related elements with `<g>` when it clarifies structure, but do not over-nest.
|
||||
|
||||
## Color rules
|
||||
- For single-color icons, use `fill="currentColor"` (or `stroke="currentColor"` for line icons) so the icon inherits the surrounding text color.
|
||||
- For multi-color illustrations, use explicit hex colors (e.g. `#4A90D9`) - never named CSS colors like `"blue"`.
|
||||
- Avoid gradients, filters, and blend modes unless the user explicitly asks for a glossy/gradient style - they rarely render correctly from generated coordinates.
|
||||
|
||||
## Accessibility
|
||||
- Include a `<title>` element as the first child of `<svg>` describing what the icon depicts.
|
||||
|
||||
## Security (mandatory - do not violate)
|
||||
- Never include `<script>`, `<foreignObject>`, event handler attributes (`onload`, `onclick`, etc.), or `<a>`/`href` links to external URLs.
|
||||
- Never reference external resources (no `xlink:href` to remote URLs, no `<image>` with external `src`).
|
||||
|
||||
## Self-check before you answer
|
||||
Silently verify: every tag is closed; `viewBox` and `xmlns` are present; every part from your part list appears as its own element; nothing lies outside the viewBox; no forbidden element or attribute is used. Workflow Studio validates the SVG automatically and will send you the exact error if it is broken - if you receive such a message, reply with the full corrected SVG only.
|
||||
|
||||
## Example 1 - simple geometric icon (single color, primitives only)
|
||||
|
||||
Request: "Sun icon"
|
||||
|
||||
Parts: disc (circle), 8 rays (lines).
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Sun</title>
|
||||
<circle cx="12" cy="12" r="5" fill="currentColor"/>
|
||||
<g stroke="currentColor" stroke-width="2" stroke-linecap="round">
|
||||
<line x1="12" y1="1" x2="12" y2="4"/>
|
||||
<line x1="12" y1="20" x2="12" y2="23"/>
|
||||
<line x1="4.22" y1="4.22" x2="6.34" y2="6.34"/>
|
||||
<line x1="17.66" y1="17.66" x2="19.78" y2="19.78"/>
|
||||
<line x1="1" y1="12" x2="4" y2="12"/>
|
||||
<line x1="20" y1="12" x2="23" y2="12"/>
|
||||
<line x1="4.22" y1="19.78" x2="6.34" y2="17.66"/>
|
||||
<line x1="17.66" y1="6.34" x2="19.78" y2="4.22"/>
|
||||
</g>
|
||||
</svg>
|
||||
```
|
||||
|
||||
## Example 2 - organic subject (multi-color, decomposed into parts)
|
||||
|
||||
Request: "Bird icon"
|
||||
|
||||
Parts: body (rounded outline via a short curved path), head (circle), beak (small triangle), eye (dot), wing (small curved path) - each drawn as its own element, layered head/beak/eye on top of the body.
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Bird</title>
|
||||
<path d="M4 16 Q4 10 11 9 Q18 8 20 13 Q20 17 14 18 Q7 19 4 16 Z" fill="#4A90D9"/>
|
||||
<path d="M8 14 Q5 12 3 14 Q6 16 9 16 Z" fill="#3576B8"/>
|
||||
<circle cx="17" cy="10" r="3.2" fill="#4A90D9"/>
|
||||
<polygon points="20,10 23,9.3 20,11.4" fill="#F2B807"/>
|
||||
<circle cx="18" cy="9" r="0.6" fill="#1A1A1A"/>
|
||||
</svg>
|
||||
```
|
||||
|
||||
Follow this pattern for every request: list the parts first, represent each with the simplest adequate shape, then assemble - never skip straight to unstructured shapes.
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
name: T2I Prompt Enhancer
|
||||
description: Turns a short idea into a prompt written the way the model in the current GenerateUI workflow understands best.
|
||||
metadata:
|
||||
wfs-version: 2
|
||||
output: prompt
|
||||
tools: []
|
||||
temperature: 0.5
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
You are a prompt engineer for text-to-image models in ComfyUI. The user gives a short or vague idea (in any language); you rewrite it into one prompt that the model in their current workflow will follow well.
|
||||
|
||||
## Step 1 - pick the prompt style
|
||||
Use the "Current workflow context" section (provided by Workflow Studio when a workflow is loaded) to decide the style. If the user names a model, that wins. If there is no context and the user did not say, use NATURAL style.
|
||||
|
||||
- NATURAL style - for DiT / LLM-encoder models: Qwen-Image, FLUX (1, 2, Kontext, Klein), Z-Image, HiDream, Ernie Image, LongCat, MOSS-Image, Ming, Boogu, SD3/3.5, Lumina, and anything that loads a T5 / Qwen / Llama / Gemma text encoder.
|
||||
- Write fluent English sentences, most important first: subject and its key attributes -> action or pose -> setting -> lighting and time of day -> composition and camera (shot size, angle, lens) -> art style or medium.
|
||||
- 40-120 words. Every phrase must add visual information - no filler such as "masterpiece", "best quality", "8k", "highly detailed" (these models ignore them, and long prompts get truncated: FLUX's T5 stops reading at 512 tokens).
|
||||
- Any text that must appear in the image goes in double quotes, e.g. a sign reading "OPEN".
|
||||
- Negative: these models usually run at CFG about 1, where the negative prompt has no effect. Leave `negative:` empty unless the context shows CFG above 1.5; then add a short list of real defects to avoid.
|
||||
- TAG style - for SDXL anime checkpoints (Illustrious, NoobAI, Animagine, Pony, and checkpoints whose names contain "illu", "noob", "anime", "pony"):
|
||||
- Comma-separated Danbooru tags in this order: quality tags -> count tag (1girl, 2boys, no humans) -> character / series if named -> appearance (hair, eyes, clothing) -> expression and pose -> background and setting -> lighting -> composition (from above, cowboy shot, close-up).
|
||||
- Quality tags: Pony models start with `score_9, score_8_up, score_7_up`; other anime SDXL models start with `masterpiece, best quality, very aesthetic, absurdres`.
|
||||
- Use underscores only where the tag itself has them; use spaces otherwise (e.g. `long hair`, `looking at viewer`). Optional emphasis: `(tag:1.2)`, never above 1.4.
|
||||
- Negative: `worst quality, low quality, lowres, bad anatomy, bad hands, extra digits, jpeg artifacts, watermark, signature, text` (Pony: add `score_4, score_5, score_6`).
|
||||
- MIXED style - for realistic SDXL or SD1.5 checkpoints: short comma-separated descriptive phrases (not single words, not long sentences), subject first, then camera and lighting terms (e.g. `85mm portrait, soft window light, shallow depth of field`). Negative: a short defect list.
|
||||
|
||||
## Step 2 - rewrite
|
||||
- Keep every element the user explicitly asked for, exactly. Never contradict them or swap them for something "better".
|
||||
- Fill only what is missing and matters for the picture (lighting, framing, background) with plausible, concrete choices that fit the user's mood. Do not invent extra characters, objects, or text.
|
||||
- If the idea is in Japanese or another language, translate the meaning faithfully - never transliterate (夕焼け -> "sunset glow", not "yuuyake").
|
||||
- Write every positive and negative prompt from scratch from the user's request and the style rules here. Only when the context contains "Prompts currently typed into the workflow" (it appears when the user refers to the current prompt) edit or extend those as the user asks; even then, keep only the parts the user wants to keep.
|
||||
- If the request is truly ambiguous in a way that changes the image (e.g. "a bat" - animal or baseball bat?), ask one short question instead of guessing.
|
||||
|
||||
## Output format (always)
|
||||
1. One line starting with "Style:" naming the style you chose and why in a few words (e.g. "Style: NATURAL - workflow uses Qwen-Image 2.1, CFG 1").
|
||||
2. The prompt in a fenced block labeled `prompt`, exactly in this shape:
|
||||
|
||||
```prompt
|
||||
positive: <the full prompt on one line>
|
||||
negative: <negative prompt on one line, or leave empty after the colon>
|
||||
```
|
||||
|
||||
3. Optionally one line suggesting a fitting aspect ratio (e.g. "Suggested aspect: 9:16 (portrait)").
|
||||
|
||||
Reply to the user in their language outside the block, but the prompt itself is always English (TAG style uses English tags).
|
||||
|
||||
When the user asks for changes ("make it night", "add a cat"), apply them to your previous prompt and output the complete updated block again - never only the changed part.
|
||||
|
||||
## Example
|
||||
User: 雨の夜の東京で傘をさす猫
|
||||
Context: model qwen_image_2512_fp8.safetensors, CFG 1.0
|
||||
|
||||
Style: NATURAL - Qwen-Image workflow, CFG 1 so no negative prompt.
|
||||
|
||||
```prompt
|
||||
positive: A small tabby cat standing upright on a rain-soaked Tokyo side street at night, holding a tiny transparent umbrella above its head with one paw. Glowing neon signs in pink and cyan reflect on the wet asphalt, raindrops visible in the light. Low-angle medium shot, shallow depth of field with blurred city lights behind, cinematic and slightly melancholic mood.
|
||||
negative:
|
||||
```
|
||||
|
||||
Suggested aspect: 2:3 (portrait)
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: Workflow Doctor
|
||||
description: Explains the workflow loaded in GenerateUI and diagnoses errors from checked facts (missing nodes and files, last execution error, ComfyUI log).
|
||||
metadata:
|
||||
wfs-version: 1
|
||||
tools: []
|
||||
temperature: 0.2
|
||||
context: [workflow-detail, logs]
|
||||
---
|
||||
|
||||
You are a ComfyUI troubleshooting assistant inside Workflow Studio. Every message comes with two fact sections generated by Workflow Studio right before the message was sent:
|
||||
- "Workflow facts": the workflow loaded in GenerateUI, checked against this ComfyUI install - node types that are NOT INSTALLED, values that are NOT in ComfyUI's list (usually model files that do not exist, with the closest existing names), broken links, missing required inputs, and every node with its settings.
|
||||
- "ComfyUI runtime facts": the last execution error from ComfyUI's history and the most recent error block of the ComfyUI log.
|
||||
|
||||
## Ground rules (most important)
|
||||
- Base every statement on those facts or on text the user pasted. Quote the exact node (#id and class name), file name, or error line you rely on.
|
||||
- Your general knowledge of ComfyUI, models and custom nodes may be outdated or wrong. When you use it, say so ("generally", "usually") and never invent node names, file names, settings, menu paths or package names.
|
||||
- If the facts do not show the cause, say that plainly and tell the user exactly what to check or paste next (e.g. "run the workflow once and send me a message again so the error is captured").
|
||||
- The facts are rebuilt on every message, so after the user fixes something, re-check from the new facts instead of repeating your earlier answer.
|
||||
|
||||
## When the user asks why something fails
|
||||
1. Cause - one or two sentences, quoting the decisive fact.
|
||||
2. Fix - numbered, concrete steps using Workflow Studio / ComfyUI terms:
|
||||
- Not installed node type -> install the custom node pack that provides it with ComfyUI Manager ("Install Missing Custom Nodes"), then restart ComfyUI. Only name the pack if the error text or node name makes it certain.
|
||||
- Value not in ComfyUI's list -> pick an existing file in that node (offer the "closest existing" names from the facts), or put the missing file into the matching models folder and press Refresh Models in GenerateUI.
|
||||
- Out of memory (CUDA out of memory / allocation errors) -> lower resolution or batch size, use a smaller / quantized model, unload other models (Workflow Studio's VRAM settings can unload Ollama models before generation).
|
||||
- Shape / dtype / "mat1 and mat2" errors -> usually a model and an encoder/VAE/LoRA from different model families; point at the two loader nodes involved.
|
||||
3. Check - how the user can confirm it is fixed.
|
||||
|
||||
## When the user asks what the workflow does
|
||||
Describe the pipeline in order (load models -> encode prompt -> sample -> decode -> save), naming the key nodes (#id) and the important settings (model files, resolution, steps, CFG, sampler, LoRAs). Point out anything in "Detected problems" even if the user did not ask.
|
||||
|
||||
## Style
|
||||
- Reply in the user's language. Use short headings and numbered steps; keep it compact.
|
||||
- Never output a full workflow JSON. If a node value must change, name the node (#id), the input and the new value.
|
||||
+167
-27
@@ -14,6 +14,9 @@ import time
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import unquote
|
||||
|
||||
from ..config import is_safe_rel, resolve_untrusted
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -38,9 +41,22 @@ AUDIO_WAVE_COLORS = {
|
||||
".m4a": (251, 113, 133), # rose
|
||||
}
|
||||
|
||||
# サポートする画像拡張子（動画・音声はGalleryでは静止画と同じ一覧・配信経路を共有する。
|
||||
# 3Dモデル拡張子。表示はブラウザのthree.js(static/js/gallery-3d-viewer.js)で行う。
|
||||
# サーバーに3Dレンダラーが無いため、サムネイルはブラウザが描画してアップロードしたものを使う。
|
||||
GLTF_EXTENSIONS = {".glb", ".gltf", ".vrm"} # .vrmの実体はGLB
|
||||
MODEL3D_EXTENSIONS = GLTF_EXTENSIONS | {".ply", ".spz"}
|
||||
# .gltfが相対URIで参照する外部ファイルとして配信を許可する拡張子
|
||||
MODEL3D_ASSET_EXTENSIONS = {".bin", ".png", ".jpg", ".jpeg", ".webp"}
|
||||
# glTF JSON部の読み取り上限。base64埋め込みの巨大な.gltfはメタデータ読み取りを諦める
|
||||
_GLTF_JSON_MAX = 16 * 1024 * 1024
|
||||
_MODEL3D_THUMB_MAX_BYTES = 8 * 1024 * 1024
|
||||
_MODEL3D_THUMB_MAX_SIDE = 2048
|
||||
_MODEL3D_THUMB_SIZE = 512
|
||||
|
||||
# サポートする画像拡張子（動画・音声・3DはGalleryでは静止画と同じ一覧・配信経路を共有する。
|
||||
# .psdはブラウザが直接レンダリングできないため、配信時に合成済みPNGへ変換する）
|
||||
IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".gif", ".svg", ".psd"} | VIDEO_EXTENSIONS | AUDIO_EXTENSIONS
|
||||
IMAGE_EXTENSIONS = ({".png", ".jpg", ".jpeg", ".webp", ".gif", ".svg", ".psd"}
|
||||
| VIDEO_EXTENSIONS | AUDIO_EXTENSIONS | MODEL3D_EXTENSIONS)
|
||||
|
||||
# Canvas2D の globalCompositeOperation 文字列 -> psd_tools.constants.BlendMode 名
|
||||
# (Image Edit タブの Layer.blendMode はCanvas2Dの合成モード文字列をそのまま保持している)
|
||||
@@ -256,7 +272,7 @@ class GalleryService:
|
||||
"""バックグラウンドスレッドで未キャッシュ画像のprompt_cacheを構築する。
|
||||
10枚処理するたびに50msスリープしてメインスレッドへの影響を最小化。"""
|
||||
try:
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir() or not self._check_path_allowed(folder):
|
||||
return
|
||||
entries = self._scan_folder(folder)
|
||||
@@ -319,13 +335,19 @@ class GalleryService:
|
||||
pass
|
||||
return False
|
||||
|
||||
def _resolve_untrusted(self, value) -> Path:
|
||||
"""クライアント由来のパスを resolve() の前に文字列のまま検査して返す(拒否時 ValueError)。
|
||||
Windows では UNC パスは resolve() した時点で相手ホストへ資格情報付きで接続してしまうため、
|
||||
UNC は許可ルート自身が UNC の場合のみ、その配下を許す。"""
|
||||
return resolve_untrusted(value, (self._allowed_root, self._comfy_output_root))
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# フォルダツリー
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
def list_folder_tree(self, root_path: str) -> dict:
|
||||
"""outputフォルダのフォルダツリーを返す"""
|
||||
root = Path(root_path).resolve()
|
||||
root = self._resolve_untrusted(root_path)
|
||||
if not root.is_dir():
|
||||
return {"error": f"Directory not found: {root_path}"}
|
||||
if self._allowed_root is None:
|
||||
@@ -470,7 +492,7 @@ class GalleryService:
|
||||
recursive: bool = False,
|
||||
) -> list[dict]:
|
||||
"""指定フォルダ内の画像一覧を返す（recursive=Trueならサブフォルダも含める）"""
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir():
|
||||
return []
|
||||
if not self._check_path_allowed(folder):
|
||||
@@ -578,7 +600,7 @@ class GalleryService:
|
||||
def save_image_to_folder(self, folder_path: str, filename: str, image_data: str) -> dict:
|
||||
"""任意フォルダへ画像を保存する（data URL想定、同名なら上書き）。
|
||||
Style Catalogのカタログ作成（スタイル名でのファイル保存）で使用。"""
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir():
|
||||
return {"ok": False, "error": "Destination folder not found"}
|
||||
if not self._check_path_allowed(folder):
|
||||
@@ -726,7 +748,7 @@ class GalleryService:
|
||||
|
||||
def save_image_prompt(self, image_path: str, text: str) -> bool:
|
||||
"""画像と同名の.txtサイドカーにプロンプトテキストを保存する"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return False
|
||||
if not self._check_path_allowed(p):
|
||||
@@ -744,7 +766,7 @@ class GalleryService:
|
||||
|
||||
def get_image_metadata(self, image_path: str) -> dict:
|
||||
"""PNG/JPEGからメタデータを抽出し、保存済みメタと合わせて返す"""
|
||||
path = Path(image_path).resolve()
|
||||
path = self._resolve_untrusted(image_path)
|
||||
if not path.is_file():
|
||||
return {"error": "File not found"}
|
||||
if not self._check_path_allowed(path):
|
||||
@@ -762,6 +784,8 @@ class GalleryService:
|
||||
embedded = self._read_mp4_metadata(path)
|
||||
elif ext in AUDIO_EXTENSIONS:
|
||||
embedded = self._read_audio_metadata(path)
|
||||
elif ext in GLTF_EXTENSIONS:
|
||||
embedded = self._read_gltf_metadata(path)
|
||||
|
||||
dims = self._read_media_dimensions(path, ext)
|
||||
saved = self.metadata_store.get(str(path))
|
||||
@@ -935,6 +959,37 @@ class GalleryService:
|
||||
logger.debug("Audio metadata read error: %s", e)
|
||||
return result
|
||||
|
||||
def _read_gltf_metadata(self, path: Path) -> dict:
|
||||
"""glTF/GLB/VRMのasset.extrasを読む。ComfyUIのSaveGLBはここにprompt/workflowを
|
||||
JSON文字列で書き込む。GLBは先頭のJSONチャンクだけ読み、メッシュ本体は読まない。"""
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
if path.suffix.lower() == ".gltf":
|
||||
if path.stat().st_size > _GLTF_JSON_MAX:
|
||||
return {}
|
||||
doc = json.loads(f.read())
|
||||
else:
|
||||
head = f.read(20)
|
||||
if len(head) < 20 or head[:4] != b"glTF":
|
||||
return {}
|
||||
length, chunk_type = struct.unpack("<II", head[12:20])
|
||||
if chunk_type != 0x4E4F534A or length > _GLTF_JSON_MAX: # "JSON"
|
||||
return {}
|
||||
doc = json.loads(f.read(length))
|
||||
extras = doc.get("asset", {}).get("extras")
|
||||
except (OSError, ValueError, AttributeError) as e:
|
||||
logger.debug("glTF metadata read error: %s", e)
|
||||
return {}
|
||||
if not isinstance(extras, dict):
|
||||
return {}
|
||||
result = {}
|
||||
for k, v in extras.items():
|
||||
if isinstance(v, str):
|
||||
result[k] = v
|
||||
elif k in ("prompt", "workflow") and isinstance(v, dict):
|
||||
result[k] = json.dumps(v)
|
||||
return result
|
||||
|
||||
def _read_media_dimensions(self, path: Path, ext: str) -> dict:
|
||||
"""画像は幅高さ、動画は幅高さ+再生時間（秒）を返す。取得できない項目はNone。"""
|
||||
info: dict = {"width": None, "height": None, "duration": None}
|
||||
@@ -957,7 +1012,7 @@ class GalleryService:
|
||||
from psd_tools import PSDImage
|
||||
psd = PSDImage.open(path)
|
||||
info["width"], info["height"] = psd.width, psd.height
|
||||
elif ext != ".svg":
|
||||
elif ext != ".svg" and ext not in MODEL3D_EXTENSIONS:
|
||||
from PIL import Image
|
||||
with Image.open(path) as img:
|
||||
info["width"], info["height"] = img.size
|
||||
@@ -975,7 +1030,7 @@ class GalleryService:
|
||||
extractPromptsLiteGraphがトップレベル系統しか拾えないことがあるため、
|
||||
プロンプト抽出専用にAPI形式を優先する（Metadataタブのプロンプト抽出と同じ優先順位）。
|
||||
"""
|
||||
path = Path(image_path).resolve()
|
||||
path = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(path):
|
||||
return None
|
||||
|
||||
@@ -989,6 +1044,8 @@ class GalleryService:
|
||||
embedded = self._read_mp4_metadata(path)
|
||||
elif ext in AUDIO_EXTENSIONS:
|
||||
embedded = self._read_audio_metadata(path)
|
||||
elif ext in GLTF_EXTENSIONS:
|
||||
embedded = self._read_gltf_metadata(path)
|
||||
|
||||
if embedded:
|
||||
key_groups = (("workflow", "Workflow"), ("prompt", "Prompt"))
|
||||
@@ -1034,7 +1091,7 @@ class GalleryService:
|
||||
"""
|
||||
from .convert_service import OUTPUT_FORMATS, convert_audio, supports_tags
|
||||
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return {"ok": False, "error": "File not found"}
|
||||
if p.suffix.lower() not in AUDIO_EXTENSIONS | VIDEO_EXTENSIONS:
|
||||
@@ -1103,7 +1160,7 @@ class GalleryService:
|
||||
}
|
||||
|
||||
def save_image_meta(self, image_path: str, data: dict) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
logger.warning("save_image_meta: path not allowed: %s", resolved)
|
||||
return False
|
||||
@@ -1113,7 +1170,7 @@ class GalleryService:
|
||||
return self.metadata_store.save(str(resolved), data)
|
||||
|
||||
def toggle_favorite(self, image_path: str) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
return False
|
||||
# get_image_metadata()/list_images() 側はresolve()済みパスをキーにして取得する
|
||||
@@ -1129,7 +1186,11 @@ class GalleryService:
|
||||
"""複数画像のお気に入りを一括設定する"""
|
||||
ok = fail = 0
|
||||
for p in paths:
|
||||
resolved = Path(p).resolve()
|
||||
try:
|
||||
resolved = self._resolve_untrusted(p)
|
||||
except ValueError:
|
||||
fail += 1
|
||||
continue
|
||||
if not self._check_path_allowed(resolved):
|
||||
fail += 1
|
||||
continue
|
||||
@@ -1167,7 +1228,7 @@ class GalleryService:
|
||||
return self.metadata_store.delete_group(name)
|
||||
|
||||
def add_to_group(self, image_path: str, group_name: str) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
return False
|
||||
# get_image_metadata()/list_images() 側はresolve()済みパスをキーにして取得する
|
||||
@@ -1182,7 +1243,7 @@ class GalleryService:
|
||||
return True
|
||||
|
||||
def remove_from_group(self, image_path: str, group_name: str) -> bool:
|
||||
resolved = Path(image_path).resolve()
|
||||
resolved = self._resolve_untrusted(image_path)
|
||||
if not self._check_path_allowed(resolved):
|
||||
return False
|
||||
resolved_str = str(resolved)
|
||||
@@ -1211,7 +1272,7 @@ class GalleryService:
|
||||
|
||||
def create_folder(self, parent_path: str, name: str) -> dict:
|
||||
"""選択フォルダ内に新しいサブフォルダを作成する"""
|
||||
parent = Path(parent_path).resolve()
|
||||
parent = self._resolve_untrusted(parent_path)
|
||||
if not parent.is_dir():
|
||||
return {"ok": False, "error": "Parent directory not found"}
|
||||
if not self._check_path_allowed(parent):
|
||||
@@ -1233,7 +1294,7 @@ class GalleryService:
|
||||
def delete_folder(self, folder_path: str) -> dict:
|
||||
"""フォルダを再帰的に削除する"""
|
||||
import shutil
|
||||
folder = Path(folder_path).resolve()
|
||||
folder = self._resolve_untrusted(folder_path)
|
||||
if not folder.is_dir():
|
||||
return {"ok": False, "error": "Directory not found"}
|
||||
if not self._check_path_allowed(folder):
|
||||
@@ -1253,7 +1314,11 @@ class GalleryService:
|
||||
deleted = []
|
||||
errors = []
|
||||
for img_path in paths:
|
||||
p = Path(img_path).resolve()
|
||||
try:
|
||||
p = self._resolve_untrusted(img_path)
|
||||
except ValueError:
|
||||
errors.append(f"{Path(img_path).name}: access denied")
|
||||
continue
|
||||
if not p.is_file():
|
||||
errors.append(f"{Path(img_path).name}: not found")
|
||||
continue
|
||||
@@ -1274,7 +1339,7 @@ class GalleryService:
|
||||
|
||||
def move_images(self, paths: list, dest_folder: str) -> dict:
|
||||
"""画像ファイルを別フォルダへ移動する（複数対応）"""
|
||||
dest = Path(dest_folder).resolve()
|
||||
dest = self._resolve_untrusted(dest_folder)
|
||||
if not dest.is_dir():
|
||||
return {"ok": False, "error": "Destination folder not found", "moved": [], "errors": []}
|
||||
if not self._check_path_allowed(dest):
|
||||
@@ -1282,7 +1347,11 @@ class GalleryService:
|
||||
moved = []
|
||||
errors = []
|
||||
for img_path in paths:
|
||||
p = Path(img_path).resolve()
|
||||
try:
|
||||
p = self._resolve_untrusted(img_path)
|
||||
except ValueError:
|
||||
errors.append(f"{Path(img_path).name}: access denied")
|
||||
continue
|
||||
if not p.is_file():
|
||||
errors.append(f"{Path(img_path).name}: not found")
|
||||
continue
|
||||
@@ -1326,11 +1395,14 @@ class GalleryService:
|
||||
|
||||
layers_data = []
|
||||
for img_path in paths:
|
||||
p = Path(img_path).resolve()
|
||||
try:
|
||||
p = self._resolve_untrusted(img_path)
|
||||
except ValueError:
|
||||
continue
|
||||
if not p.is_file() or p.suffix.lower() not in IMAGE_EXTENSIONS:
|
||||
continue
|
||||
if p.suffix.lower() in VIDEO_EXTENSIONS | AUDIO_EXTENSIONS:
|
||||
continue # 動画・音声はPSDレイヤーにできない
|
||||
if p.suffix.lower() in VIDEO_EXTENSIONS | AUDIO_EXTENSIONS | MODEL3D_EXTENSIONS:
|
||||
continue # 動画・音声・3DはPSDレイヤーにできない
|
||||
if not self._check_path_allowed(p):
|
||||
continue
|
||||
try:
|
||||
@@ -1494,7 +1566,7 @@ class GalleryService:
|
||||
"""サーバー上に既にあるPSDファイル(Gallery上のパス)をレイヤー配列に分解する。
|
||||
Galleryから直接Image Editへ送る際、アップロードの往復無しで済ませるために使う。
|
||||
"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file() or p.suffix.lower() != ".psd":
|
||||
raise ValueError("Not a PSD file")
|
||||
if not self._check_path_allowed(p):
|
||||
@@ -1518,7 +1590,7 @@ class GalleryService:
|
||||
def serve_image(self, image_path: str):
|
||||
"""画像のPathオブジェクトを返す（ルートで使用）。
|
||||
PSDはブラウザが直接レンダリングできないため、合成済みPNGにキャッシュ変換して返す。"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return None
|
||||
if p.suffix.lower() not in IMAGE_EXTENSIONS:
|
||||
@@ -1530,6 +1602,58 @@ class GalleryService:
|
||||
return self._get_psd_composite_cache(p)
|
||||
return p
|
||||
|
||||
def serve_model3d_asset(self, model_path: str, uri: str) -> Path | None:
|
||||
""".gltfが相対URIで参照する外部ファイル(.bin/テクスチャ)のPathを返す。
|
||||
uriはglTF仕様どおりパーセントエンコードされている。"""
|
||||
try:
|
||||
p = self._resolve_untrusted(model_path)
|
||||
if p.suffix.lower() != ".gltf" or not p.is_file() or not self._check_path_allowed(p):
|
||||
return None
|
||||
# .gltf の外部参照 URI は相対パスのみ(UNC・絶対パス・「..」は結合前に拒否)
|
||||
rel_uri = unquote(uri)
|
||||
if not is_safe_rel(rel_uri):
|
||||
return None
|
||||
target = (p.parent / rel_uri).resolve()
|
||||
if target.suffix.lower() not in MODEL3D_ASSET_EXTENSIONS or not target.is_file():
|
||||
return None
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
if not self._check_path_allowed(target):
|
||||
logger.warning("serve_model3d_asset: path not allowed: %s", target)
|
||||
return None
|
||||
return target
|
||||
|
||||
def _model3d_thumb_path(self, p: Path) -> Path:
|
||||
"""ブラウザが描画した3Dサムネイル(各サイズの元画像)の保存先。"""
|
||||
mtime = int(p.stat().st_mtime * 1000)
|
||||
cache_dir = self.data_dir / "thumb_cache"
|
||||
cache_dir.mkdir(exist_ok=True)
|
||||
return cache_dir / f"{hashlib.md5(f'{p}:{mtime}:model3d'.encode()).hexdigest()}_m3d.jpg"
|
||||
|
||||
def save_model3d_thumbnail(self, model_path: str, image_data: str) -> dict:
|
||||
"""ブラウザが描画した3Dモデルのサムネイルを保存する。"""
|
||||
from io import BytesIO
|
||||
from PIL import Image
|
||||
|
||||
p = self._resolve_untrusted(model_path)
|
||||
if not p.is_file() or p.suffix.lower() not in MODEL3D_EXTENSIONS:
|
||||
return {"ok": False, "error": "Not a 3D model"}
|
||||
if not self._check_path_allowed(p):
|
||||
return {"ok": False, "error": "Access denied"}
|
||||
try:
|
||||
raw, _ = _decode_image_data_url(image_data)
|
||||
if len(raw) > _MODEL3D_THUMB_MAX_BYTES:
|
||||
return {"ok": False, "error": "Image too large"}
|
||||
with Image.open(BytesIO(raw)) as im:
|
||||
if max(im.size) > _MODEL3D_THUMB_MAX_SIDE:
|
||||
return {"ok": False, "error": "Image too large"}
|
||||
im = im.convert("RGB")
|
||||
im.thumbnail((_MODEL3D_THUMB_SIZE, _MODEL3D_THUMB_SIZE), Image.LANCZOS)
|
||||
im.save(self._model3d_thumb_path(p), "JPEG", quality=85, optimize=True)
|
||||
except (OSError, ValueError) as e:
|
||||
return {"ok": False, "error": str(e)}
|
||||
return {"ok": True}
|
||||
|
||||
def _get_psd_composite_cache(self, p: Path) -> Path | None:
|
||||
"""PSDの全レイヤー合成結果をフル解像度PNGとしてディスクキャッシュし、そのPathを返す。"""
|
||||
try:
|
||||
@@ -1605,7 +1729,7 @@ class GalleryService:
|
||||
（元ファイルへのフォールバックはできないため、失敗時はNoneを返す）。
|
||||
PSDはpsd_toolsでレイヤーを合成してからJPEGとしてキャッシュする（同様にフォールバック不可）。
|
||||
"""
|
||||
p = Path(image_path).resolve()
|
||||
p = self._resolve_untrusted(image_path)
|
||||
if not p.is_file():
|
||||
return None
|
||||
if p.suffix.lower() not in IMAGE_EXTENSIONS:
|
||||
@@ -1638,6 +1762,22 @@ class GalleryService:
|
||||
if thumb_path.exists():
|
||||
return thumb_path
|
||||
|
||||
# 3Dはブラウザが描画して保存したものを縮小して返す。未保存ならNone(404)を返し、
|
||||
# それを合図にブラウザ側が描画する。
|
||||
if p.suffix.lower() in MODEL3D_EXTENSIONS:
|
||||
source = self._model3d_thumb_path(p)
|
||||
if not source.exists():
|
||||
return None
|
||||
try:
|
||||
from PIL import Image
|
||||
with Image.open(source) as img:
|
||||
img.thumbnail((width, width), Image.LANCZOS)
|
||||
img.save(thumb_path, "JPEG", quality=85, optimize=True)
|
||||
return thumb_path
|
||||
except Exception as e:
|
||||
logger.warning("serve_thumbnail: 3D thumbnail failed for %s: %s", p, e)
|
||||
return None
|
||||
|
||||
# 音声はPillowで開けない。PyAVでデコードして波形をJPEGに描画しキャッシュする。
|
||||
if p.suffix.lower() in AUDIO_EXTENSIONS:
|
||||
try:
|
||||
|
||||
@@ -1,239 +0,0 @@
|
||||
"""
|
||||
EmbeddingGemma 2 ローカル実行 (onnx-community/embeddinggemma-2-ONNX)
|
||||
|
||||
onnxruntime + tokenizers + huggingface_hub だけで動かす(transformers の対応を待たない)。
|
||||
前処理は Transformers.js の Gemma4ImageProcessor / EmbeddingGemma2Processor の移植:
|
||||
画像 -> アスペクト比維持でリサイズ(48px倍数, 最大2520パッチ) -> 16pxパッチ化 -> vision_encoder
|
||||
-> image_features を text model の <|image|> トークン位置に差し込み -> sentence_embedding(768)
|
||||
モデルは初回のみ固定リポジトリからダウンロードする(約300MB, q4)。
|
||||
"""
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
REPO_ID = "onnx-community/embeddinggemma-2-ONNX"
|
||||
LOCAL_MODEL = "embeddinggemma-2"
|
||||
LOCAL_DIMS = (128, 256, 512, 768) # MRL対応次元
|
||||
DEFAULT_LOCAL_DIM = 768
|
||||
# 量子化精度 -> ファイル名サフィックス
|
||||
DTYPES = {"q4": "_q4", "q8": "_quantized", "fp16": "_fp16", "fp32": ""}
|
||||
DEFAULT_DTYPE = "q4"
|
||||
|
||||
_PATCH = 16
|
||||
_POOL = 3
|
||||
SOFT_TOKENS = (70, 140, 280) # 画像1枚あたりのソフトトークン数(多いほど高精度・低速)
|
||||
DEFAULT_SOFT = 280
|
||||
_MAX_TOKENS = 1024 # テキスト入力のトークン数上限(画像用プロンプトは約290トークン)
|
||||
_MAX_IMAGE_SIDE = 2048 # 画像の長辺上限(超える場合は縮小してから処理)
|
||||
_IMAGE_TOKEN = "<|image|>"
|
||||
_BOI = "<|image>"
|
||||
_EOI = "<image|>"
|
||||
|
||||
DEVICES = ("auto", "cpu", "cuda", "dml")
|
||||
_PROVIDER_BY_DEVICE = {"cuda": "CUDAExecutionProvider", "dml": "DmlExecutionProvider"}
|
||||
|
||||
_lock = threading.Lock()
|
||||
_runtime = None # ((dtype, device, threads), _Runtime)
|
||||
|
||||
|
||||
_opts = {"device": "auto", "threads": 0} # 実行デバイス/CPUスレッド数(configureで更新)
|
||||
|
||||
|
||||
def configure(device: str = "auto", threads: int = 0) -> None:
|
||||
"""ローカル推論の実行設定。次回のensure_runtimeから反映(device/threads変更時はセッション再作成)。"""
|
||||
_opts["device"] = device if device in DEVICES else "auto"
|
||||
_opts["threads"] = max(0, min(int(threads), (os.cpu_count() or 1) * 2))
|
||||
|
||||
|
||||
def current_config() -> dict:
|
||||
return dict(_opts)
|
||||
|
||||
|
||||
def release() -> None:
|
||||
"""セッションを破棄してVRAM/RAMを解放する(インデックス作成後など)。次回利用時に再ロードされる。"""
|
||||
global _runtime
|
||||
with _lock:
|
||||
_runtime = None
|
||||
|
||||
|
||||
def active_provider() -> str:
|
||||
"""直近にロードしたセッションの実行プロバイダ(未ロードなら空)"""
|
||||
rt = _runtime[1] if _runtime else None
|
||||
return rt.providers[0].replace("ExecutionProvider", "") if rt else ""
|
||||
|
||||
|
||||
def available_devices() -> dict:
|
||||
"""onnxruntime が使える実行デバイス。GPU用ビルド(onnxruntime-gpu / -directml)が無ければcpuのみ。"""
|
||||
try:
|
||||
import onnxruntime as ort
|
||||
prov = ort.get_available_providers()
|
||||
version = ort.__version__
|
||||
except Exception:
|
||||
return {"cpu": True, "cuda": False, "dml": False, "ort_version": ""}
|
||||
return {"cpu": True, "cuda": "CUDAExecutionProvider" in prov, "dml": "DmlExecutionProvider" in prov,
|
||||
"ort_version": version, "cpu_count": os.cpu_count() or 1}
|
||||
|
||||
|
||||
def default_threads() -> int:
|
||||
"""既定のCPUスレッド数: 画像生成と同居しても邪魔になりにくいよう、コアの半分(最低1)"""
|
||||
return max(1, (os.cpu_count() or 2) // 2)
|
||||
|
||||
|
||||
def _target_size(h: int, w: int, max_patches: int) -> tuple[int, int]:
|
||||
"""Gemma4ImageProcessor の get_aspect_ratio_preserving_size"""
|
||||
factor = math.sqrt(max_patches * _PATCH ** 2 / (h * w))
|
||||
side = _POOL * _PATCH
|
||||
th = int(math.floor(factor * h / side)) * side
|
||||
tw = int(math.floor(factor * w / side)) * side
|
||||
max_side = (max_patches // _POOL ** 2) * side
|
||||
if th == 0 and tw == 0:
|
||||
raise ValueError("image too small")
|
||||
if th == 0:
|
||||
th, tw = side, min((w // h) * side, max_side)
|
||||
elif tw == 0:
|
||||
tw, th = side, min((h // w) * side, max_side)
|
||||
return th, tw
|
||||
|
||||
|
||||
class _Runtime:
|
||||
def __init__(self, model_dir: Path, dtype: str, device: str = "auto", threads: int = 0):
|
||||
import onnxruntime as ort
|
||||
from tokenizers import Tokenizer
|
||||
|
||||
sfx = DTYPES[dtype]
|
||||
so = ort.SessionOptions()
|
||||
so.log_severity_level = 3 # CUDA使用時のMemcpy警告などを抑止
|
||||
if threads > 0:
|
||||
so.intra_op_num_threads = threads
|
||||
so.inter_op_num_threads = 1
|
||||
avail = ort.get_available_providers()
|
||||
if device in _PROVIDER_BY_DEVICE:
|
||||
providers = [_PROVIDER_BY_DEVICE[device]] if _PROVIDER_BY_DEVICE[device] in avail else []
|
||||
elif device == "cpu":
|
||||
providers = []
|
||||
else: # auto: 使えるGPUを優先
|
||||
providers = [p for p in ("CUDAExecutionProvider", "DmlExecutionProvider") if p in avail]
|
||||
if "DmlExecutionProvider" in providers:
|
||||
# DirectMLは並列実行/メモリパターン最適化に非対応
|
||||
so.enable_mem_pattern = False
|
||||
so.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL
|
||||
providers.append("CPUExecutionProvider")
|
||||
self.providers = providers
|
||||
|
||||
def make(name):
|
||||
path = str(model_dir / "onnx" / f"{name}{sfx}.onnx")
|
||||
try:
|
||||
return ort.InferenceSession(path, so, providers=providers)
|
||||
except Exception as e:
|
||||
if providers == ["CPUExecutionProvider"]:
|
||||
raise
|
||||
logger.warning("EmbeddingGemma2: %s failed on %s (%s); falling back to CPU", name, providers[0], e)
|
||||
self.providers = ["CPUExecutionProvider"]
|
||||
return ort.InferenceSession(path, so, providers=["CPUExecutionProvider"])
|
||||
|
||||
self.text = make("model")
|
||||
self.vision = make("vision_encoder")
|
||||
self.tok = Tokenizer.from_file(str(model_dir / "tokenizer.json"))
|
||||
self.tok.no_truncation()
|
||||
self.tok.no_padding()
|
||||
|
||||
def _run_text(self, text: str, image_features: np.ndarray | None) -> np.ndarray:
|
||||
ids = np.asarray([self.tok.encode(text).ids[:_MAX_TOKENS]], dtype=np.int64)
|
||||
feeds = {
|
||||
"input_ids": ids,
|
||||
"attention_mask": np.ones_like(ids),
|
||||
"image_features": image_features if image_features is not None else np.zeros((0, 512), np.float32),
|
||||
"video_features": np.zeros((0, 512), np.float32),
|
||||
"audio_features": np.zeros((0, 512), np.float32),
|
||||
}
|
||||
return self.text.run(["sentence_embedding"], feeds)[0][0]
|
||||
|
||||
def embed_text(self, text: str, prefix: str) -> np.ndarray:
|
||||
return self._run_text(f"{prefix}{text.strip()}", None)
|
||||
|
||||
def embed_image(self, image, soft_tokens: int = DEFAULT_SOFT) -> np.ndarray:
|
||||
from PIL import Image
|
||||
im = image.convert("RGB")
|
||||
if im.width < 1 or im.height < 1:
|
||||
raise ValueError("empty image")
|
||||
if max(im.size) > _MAX_IMAGE_SIDE: # 巨大画像でメモリを使い切らないよう先に縮小
|
||||
im.thumbnail((_MAX_IMAGE_SIDE, _MAX_IMAGE_SIDE), Image.LANCZOS)
|
||||
max_patches = soft_tokens * _POOL ** 2
|
||||
th, tw = _target_size(im.height, im.width, max_patches)
|
||||
if (th, tw) != (im.height, im.width):
|
||||
im = im.resize((tw, th), Image.BICUBIC)
|
||||
arr = np.asarray(im, dtype=np.float32) * (1.0 / 255.0) # H,W,3
|
||||
ph, pw = th // _PATCH, tw // _PATCH
|
||||
n = ph * pw
|
||||
# (pH, dy, pW, dx, C) -> (pH, pW, dy, dx, C) -> (n, 768)
|
||||
patches = arr.reshape(ph, _PATCH, pw, _PATCH, 3).transpose(0, 2, 1, 3, 4).reshape(n, _PATCH * _PATCH * 3)
|
||||
# パディング無し(n個ちょうど)で渡す: 計算量が減り、パディングは出力に影響しない
|
||||
pixel = patches[None].astype(np.float32)
|
||||
pos = np.zeros((1, n, 2), dtype=np.int64)
|
||||
rows, cols = np.divmod(np.arange(n), pw)
|
||||
pos[0, :, 0], pos[0, :, 1] = cols, rows
|
||||
feats = self.vision.run(["image_features"], {"pixel_values": pixel, "pixel_position_ids": pos})[0]
|
||||
text = f"{_BOI}{_IMAGE_TOKEN * (n // _POOL ** 2)}{_EOI}"
|
||||
return self._run_text(text, feats.astype(np.float32))
|
||||
|
||||
|
||||
def model_dir(data_dir: Path) -> Path:
|
||||
return data_dir / "models" / LOCAL_MODEL
|
||||
|
||||
|
||||
def is_downloaded(data_dir: Path, dtype: str = DEFAULT_DTYPE) -> bool:
|
||||
d, sfx = model_dir(data_dir), DTYPES.get(dtype, "_q4")
|
||||
need = [d / "tokenizer.json"] + [d / "onnx" / f"{n}{sfx}.onnx{e}" for n in ("model", "vision_encoder") for e in ("", "_data")]
|
||||
return all(p.exists() for p in need)
|
||||
|
||||
|
||||
def ensure_runtime(data_dir: Path, dtype: str = DEFAULT_DTYPE, progress=None,
|
||||
device: str | None = None, threads: int | None = None) -> _Runtime:
|
||||
"""モデルをダウンロード(未取得時)してセッションを返す。dtype/device/threads変更時は作り直す。"""
|
||||
global _runtime
|
||||
if dtype not in DTYPES:
|
||||
dtype = DEFAULT_DTYPE
|
||||
device = _opts["device"] if device is None else device
|
||||
threads = _opts["threads"] if threads is None else threads
|
||||
if device not in DEVICES:
|
||||
device = "auto"
|
||||
key = (dtype, device, int(threads))
|
||||
with _lock:
|
||||
if _runtime and _runtime[0] == key:
|
||||
return _runtime[1]
|
||||
d, sfx = model_dir(data_dir), DTYPES[dtype]
|
||||
if not is_downloaded(data_dir, dtype):
|
||||
if progress:
|
||||
progress("downloading")
|
||||
from huggingface_hub import hf_hub_download
|
||||
files = ["tokenizer.json"] + [f"onnx/{n}{sfx}.onnx{e}" for n in ("model", "vision_encoder") for e in ("", "_data")]
|
||||
for f in files:
|
||||
hf_hub_download(REPO_ID, f, local_dir=str(d))
|
||||
if progress:
|
||||
progress("loading")
|
||||
_runtime = (key, _Runtime(d, dtype, device, int(threads)))
|
||||
return _runtime[1]
|
||||
|
||||
|
||||
def _finish(vec: np.ndarray, dim: int) -> np.ndarray:
|
||||
v = np.asarray(vec, dtype=np.float32)
|
||||
if dim < v.shape[0]:
|
||||
v = v[:dim]
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def embed_text(data_dir: Path, text: str, dim: int, dtype: str = DEFAULT_DTYPE) -> np.ndarray:
|
||||
rt = ensure_runtime(data_dir, dtype)
|
||||
return _finish(rt.embed_text(text[:8000], "task: search result | query: "), dim)
|
||||
|
||||
|
||||
def embed_image(data_dir: Path, image, dim: int, soft: int = DEFAULT_SOFT, dtype: str = DEFAULT_DTYPE) -> np.ndarray:
|
||||
rt = ensure_runtime(data_dir, dtype)
|
||||
return _finish(rt.embed_image(image, soft if soft in SOFT_TOKENS else DEFAULT_SOFT), dim)
|
||||
@@ -0,0 +1,211 @@
|
||||
"""
|
||||
EmbeddingGemma 2 を Ollama で実行する (Gallery セマンティック検索のローカルバックエンド)
|
||||
|
||||
画像は /api/embed に input=[{"image": <base64>}] で渡す(images フィールドは無視されテキスト扱いになる)。
|
||||
740m(テキスト+画像+音声) と 440m(テキスト+画像) は同じ重みで同じベクトルを返すため、
|
||||
インデックスにはタグを区別せず LOCAL_MODEL として記録する。270m/570m は画像非対応なので選択肢に入れない。
|
||||
接続先は url_guard で検査する(既定はループバックのみ)。
|
||||
"""
|
||||
import base64
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import url_guard
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
LOCAL_MODEL = "embeddinggemma-2" # インデックス(npz)に記録するモデル名
|
||||
LOCAL_DIMS = (128, 256, 512, 768) # MRL対応次元
|
||||
DEFAULT_LOCAL_DIM = 768
|
||||
TAGS = ("embeddinggemma-2:740m", "embeddinggemma-2:440m")
|
||||
DEFAULT_TAG = TAGS[0]
|
||||
# :latest は 740m と同じモデル(parent_model 740m-q8_0)。740m が無ければ代わりに使う
|
||||
_ALIASES = {"embeddinggemma-2:740m": ("embeddinggemma-2:latest", "embeddinggemma-2")}
|
||||
DEFAULT_URL = "http://localhost:11434"
|
||||
QUERY_PREFIX = "task: search result | query: "
|
||||
BATCH = 4 # 1リクエストで送る画像数(1枚ずつより約1.8倍速い)
|
||||
_TIMEOUT = 120
|
||||
_RETRIES = 5 # Ollama内部の接続切断(HTTP 400 "forcibly closed"等)は再試行で回復する
|
||||
# Ollama はテキストごとに内部のランナーへ新しい接続を張るため、数千件を続けて埋め込むと
|
||||
# Windows の一時ポートが TIME_WAIT で埋まり "Only one usage of each socket address" で失敗する
|
||||
# (実測: ループバックの TIME_WAIT 14,228 / 16,384)。ポートが空くまで長めに待って再試行する
|
||||
_PORT_WAIT = 30.0
|
||||
_TAGS_TTL = 5.0 # /api/tags の結果をキャッシュする秒数(進捗ポーリングのたびに問い合わせない)
|
||||
|
||||
_opts = {"url": DEFAULT_URL, "tag": DEFAULT_TAG}
|
||||
_tags_cache = {"at": 0.0, "url": "", "names": None, "error": ""}
|
||||
_lock = threading.Lock()
|
||||
|
||||
|
||||
class OllamaError(Exception):
|
||||
"""code: ollama_unreachable / ollama_model_missing / local_error"""
|
||||
|
||||
def __init__(self, code: str, message: str = ""):
|
||||
super().__init__(message or code)
|
||||
self.code = code
|
||||
|
||||
|
||||
def configure(url: str = DEFAULT_URL, tag: str = DEFAULT_TAG) -> None:
|
||||
"""接続先とモデルサイズを設定する。許可されないURLは ValueError(url_guard.HINT)。"""
|
||||
url = url_guard.require_backend_url((url or "").strip() or DEFAULT_URL)
|
||||
with _lock:
|
||||
_opts["url"] = url
|
||||
_opts["tag"] = tag if tag in TAGS else DEFAULT_TAG
|
||||
_tags_cache["at"] = 0.0
|
||||
|
||||
|
||||
def current_config() -> dict:
|
||||
return dict(_opts)
|
||||
|
||||
|
||||
def _post(path: str, body: dict, timeout: float = _TIMEOUT) -> dict:
|
||||
base = url_guard.require_backend_url(_opts["url"])
|
||||
req = urllib.request.Request(base + path, data=json.dumps(body).encode("utf-8"), method="POST",
|
||||
headers={"Content-Type": "application/json"})
|
||||
with url_guard.urlopen(req, timeout) as res:
|
||||
return json.loads(res.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _http_message(e: urllib.error.HTTPError) -> str:
|
||||
try:
|
||||
return json.loads(e.read().decode("utf-8")).get("error", "") or str(e.reason)
|
||||
except Exception:
|
||||
return str(e.reason)
|
||||
|
||||
|
||||
def _transient(code: int, msg: str) -> bool:
|
||||
low = msg.lower()
|
||||
if "failed to load image" in low:
|
||||
return False # 壊れた画像。再試行しても直らない
|
||||
if code >= 500:
|
||||
return True
|
||||
return code == 400 and (_ports_exhausted(msg)
|
||||
or any(k in low for k in ("forcibly closed", "wsarecv", "connection", "eof")))
|
||||
|
||||
|
||||
def _ports_exhausted(msg: str) -> bool:
|
||||
low = msg.lower()
|
||||
return "only one usage of each socket address" in low or "wsaeaddrinuse" in low
|
||||
|
||||
|
||||
def installed_tags(force: bool = False) -> tuple[list[str] | None, str]:
|
||||
"""(インストール済みモデル名, エラー)。接続できなければ (None, エラー文)。"""
|
||||
now = time.time()
|
||||
with _lock:
|
||||
c = dict(_tags_cache)
|
||||
url = _opts["url"]
|
||||
if not force and c["url"] == url and now - c["at"] < _TAGS_TTL:
|
||||
return c["names"], c["error"]
|
||||
names, err = None, ""
|
||||
try:
|
||||
base = url_guard.require_backend_url(url)
|
||||
with url_guard.urlopen(urllib.request.Request(base + "/api/tags"), 3) as res:
|
||||
data = json.loads(res.read().decode("utf-8"))
|
||||
names = [m.get("name") or m.get("model") or "" for m in data.get("models", [])]
|
||||
except Exception as e:
|
||||
err = str(getattr(e, "reason", e))
|
||||
with _lock:
|
||||
_tags_cache.update(at=now, url=url, names=names, error=err)
|
||||
return names, err
|
||||
|
||||
|
||||
def _resolve(names: list[str], tag: str) -> str:
|
||||
"""実際に呼ぶモデル名。未インストールなら空文字"""
|
||||
for cand in (tag,) + _ALIASES.get(tag, ()):
|
||||
if cand in names:
|
||||
return cand
|
||||
return ""
|
||||
|
||||
|
||||
def status() -> dict:
|
||||
names, err = installed_tags()
|
||||
tag = _opts["tag"]
|
||||
out = {"url": _opts["url"], "tag": tag, "tags": list(TAGS), "reachable": names is not None, "error": err,
|
||||
"installed": {t: bool(names and _resolve(names, t)) for t in TAGS}}
|
||||
out["ready"] = bool(out["reachable"] and out["installed"][tag])
|
||||
return out
|
||||
|
||||
|
||||
def _model_name() -> str:
|
||||
names, err = installed_tags()
|
||||
if names is None:
|
||||
raise OllamaError("ollama_unreachable", f"{_opts['url']}: {err}")
|
||||
name = _resolve(names, _opts["tag"])
|
||||
if not name:
|
||||
raise OllamaError("ollama_model_missing", _opts["tag"])
|
||||
return name
|
||||
|
||||
|
||||
def _embed(inputs: list) -> list[np.ndarray]:
|
||||
model = _model_name()
|
||||
last = ""
|
||||
for attempt in range(_RETRIES + 1):
|
||||
try:
|
||||
data = _post("/api/embed", {"model": model, "input": inputs})
|
||||
embs = data.get("embeddings") or []
|
||||
if len(embs) != len(inputs):
|
||||
raise OllamaError("local_error", f"expected {len(inputs)} embeddings, got {len(embs)}")
|
||||
return [np.asarray(e, dtype=np.float32) for e in embs]
|
||||
except urllib.error.HTTPError as e:
|
||||
msg = _http_message(e)
|
||||
last = f"HTTP {e.code}: {msg}"
|
||||
if e.code == 404:
|
||||
with _lock:
|
||||
_tags_cache["at"] = 0.0 # 削除された可能性があるので次回は問い合わせ直す
|
||||
raise OllamaError("ollama_model_missing", model)
|
||||
if not _transient(e.code, msg) or attempt >= _RETRIES:
|
||||
raise OllamaError("local_error", last)
|
||||
if _ports_exhausted(msg):
|
||||
logger.warning("Ollama: local ports exhausted, waiting %.0fs before retry", _PORT_WAIT)
|
||||
time.sleep(_PORT_WAIT)
|
||||
continue
|
||||
except (urllib.error.URLError, TimeoutError, ConnectionError) as e:
|
||||
last = str(getattr(e, "reason", e))
|
||||
if attempt >= _RETRIES:
|
||||
raise OllamaError("ollama_unreachable", f"{_opts['url']}: {last}")
|
||||
time.sleep(min(4.0, 0.5 * (2 ** attempt)))
|
||||
raise OllamaError("local_error", last)
|
||||
|
||||
|
||||
def _finish(vec: np.ndarray, dim: int) -> np.ndarray:
|
||||
v = vec[:dim] if dim < vec.shape[0] else vec
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def _image_b64(image) -> str:
|
||||
buf = io.BytesIO()
|
||||
image.convert("RGB").save(buf, "JPEG", quality=88)
|
||||
return base64.b64encode(buf.getvalue()).decode("ascii")
|
||||
|
||||
|
||||
def embed_text(text: str, dim: int) -> np.ndarray:
|
||||
return embed_texts([text], dim)[0]
|
||||
|
||||
|
||||
def embed_texts(texts: list[str], dim: int) -> list[np.ndarray]:
|
||||
"""検索文(クエリ)として複数のテキストをまとめて埋め込む"""
|
||||
return [_finish(v, dim) for v in _embed([QUERY_PREFIX + t[:8000].strip() for t in texts])]
|
||||
|
||||
|
||||
def embed_images(images: list, dim: int) -> list[np.ndarray]:
|
||||
"""PIL画像(RGB)のリストをまとめて埋め込む。1枚でも読めないとリクエスト全体が失敗する。"""
|
||||
return [_finish(v, dim) for v in _embed([{"image": _image_b64(im)} for im in images])]
|
||||
|
||||
|
||||
def release() -> None:
|
||||
"""モデルをVRAMから外す(インデックス作成後)。次回利用時にOllamaが再ロードする。"""
|
||||
try:
|
||||
names, _ = installed_tags()
|
||||
name = _resolve(names or [], _opts["tag"])
|
||||
if name:
|
||||
_post("/api/embed", {"model": name, "input": [], "keep_alive": 0}, timeout=30)
|
||||
except Exception as e:
|
||||
logger.debug("semantic_ollama: release failed: %s", e)
|
||||
+145
-76
@@ -1,7 +1,7 @@
|
||||
"""
|
||||
Semantic Service - Gallery用セマンティック検索（Gemini Embedding 2）
|
||||
Semantic Service - Gallery用セマンティック検索（Gemini Embedding 2 / Ollama の EmbeddingGemma 2）
|
||||
|
||||
画像(動画・PSDはサムネイル)をGemini Embedding APIでベクトル化してローカルに保存し、
|
||||
画像(動画・PSDはサムネイル)をGemini Embedding API または Ollama でベクトル化してローカルに保存し、
|
||||
テキストクエリ/類似画像でコサイン類似度検索する。インデックス構築後の検索は
|
||||
(テキストクエリの埋め込み1回を除き)ローカル処理でAPI費用はかからない。
|
||||
|
||||
@@ -21,13 +21,13 @@ from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import semantic_local as local
|
||||
from . import semantic_ollama as local
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
API_BASE = "https://generativelanguage.googleapis.com/v1beta"
|
||||
MODEL = "gemini-embedding-2" # Gemini API (クラウド)
|
||||
LOCAL_MODEL = local.LOCAL_MODEL # EmbeddingGemma 2 (ローカルONNX)
|
||||
LOCAL_MODEL = local.LOCAL_MODEL # EmbeddingGemma 2 (Ollama)
|
||||
DEFAULT_DIM = 1536
|
||||
VALID_DIMS = (768, 1536, 3072)
|
||||
# バックエンド(モデル名) -> 選択可能な出力次元
|
||||
@@ -126,7 +126,6 @@ class SemanticService:
|
||||
self._stamp: dict[str, tuple[int, float]] = {} # path -> (size, mtime)
|
||||
self._dim = 0 # インデックスの次元(0=空)
|
||||
self._model = MODEL # インデックスを作ったモデル
|
||||
self._soft = local.DEFAULT_SOFT # ローカルモデルの画像ソフトトークン数(インデックスと一致させる)
|
||||
self._settings_file = data_dir / "semantic_settings.json"
|
||||
self._load_settings()
|
||||
self._mat_cache = None # (paths, matrix)
|
||||
@@ -137,21 +136,22 @@ class SemanticService:
|
||||
self._cancel = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
# ── ローカル推論の実行設定(デバイス/スレッド数) ─────────────
|
||||
# ── Ollama の接続先/モデルサイズ ────────────────────────
|
||||
|
||||
def _load_settings(self) -> None:
|
||||
try:
|
||||
if self._settings_file.exists():
|
||||
d = json.loads(self._settings_file.read_text(encoding="utf-8"))
|
||||
local.configure(str(d.get("device", "auto")), int(d.get("threads", 0)))
|
||||
return
|
||||
local.configure(str(d.get("url", local.DEFAULT_URL)), str(d.get("tag", local.DEFAULT_TAG)))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to load settings: %s", e)
|
||||
local.configure("auto", local.default_threads())
|
||||
|
||||
def set_local_options(self, device: str, threads: int) -> None:
|
||||
"""実行デバイス/CPUスレッド数を反映して保存する(検索時のクエリ埋め込みにも使われる)"""
|
||||
local.configure(device, threads)
|
||||
def set_local_options(self, url: str, tag: str) -> None:
|
||||
"""Ollama の接続先/モデルサイズを反映して保存する(検索時のクエリ埋め込みにも使われる)"""
|
||||
try:
|
||||
local.configure(url, tag)
|
||||
except ValueError as e:
|
||||
raise SemanticError("bad_url", str(e))
|
||||
try:
|
||||
self._settings_file.write_text(json.dumps(local.current_config()), encoding="utf-8")
|
||||
except Exception as e:
|
||||
@@ -177,7 +177,6 @@ class SemanticService:
|
||||
if model not in DIMS_BY_MODEL:
|
||||
return
|
||||
self._model = model
|
||||
self._soft = int(z["soft"]) if "soft" in z.files else local.DEFAULT_SOFT
|
||||
paths = [str(p) for p in z["paths"]]
|
||||
sizes, mtimes, vecs = z["sizes"], z["mtimes"], z["vecs"]
|
||||
self._dim = int(z["dim"])
|
||||
@@ -201,7 +200,7 @@ class SemanticService:
|
||||
tmp = self._file.with_suffix(".tmp.npz")
|
||||
np.savez(
|
||||
tmp,
|
||||
model=np.array(self._model), dim=np.array(self._dim), soft=np.array(self._soft),
|
||||
model=np.array(self._model), dim=np.array(self._dim),
|
||||
paths=np.array(paths),
|
||||
sizes=np.array([self._stamp[p][0] for p in paths], dtype=np.int64),
|
||||
mtimes=np.array([self._stamp[p][1] for p in paths], dtype=np.float64),
|
||||
@@ -220,7 +219,10 @@ class SemanticService:
|
||||
# ── 状態 ───────────────────────────────────────────────
|
||||
|
||||
def _scope_entries(self, folder: str, recursive: bool) -> list[tuple[str, str, int, float]]:
|
||||
p = Path(folder).resolve()
|
||||
try:
|
||||
p = self.gallery._resolve_untrusted(folder) # UNC等は resolve() 前に文字列で拒否
|
||||
except ValueError:
|
||||
raise SemanticError("folder_not_allowed", str(folder))
|
||||
if not p.is_dir() or not self.gallery._check_path_allowed(p):
|
||||
raise SemanticError("folder_not_allowed", str(folder))
|
||||
raw = self.gallery._scan_folder_recursive(p) if recursive else self.gallery._scan_folder(p)
|
||||
@@ -228,24 +230,18 @@ class SemanticService:
|
||||
|
||||
def status(self, folder: str = "", recursive: bool = True) -> dict:
|
||||
self._ensure_loaded()
|
||||
ollama = local.status()
|
||||
out = {
|
||||
"has_key": bool(get_api_key()),
|
||||
"model": MODEL,
|
||||
"backends": {
|
||||
MODEL: {"dims": list(VALID_DIMS), "default_dim": DEFAULT_DIM, "ready": bool(get_api_key())},
|
||||
LOCAL_MODEL: {"dims": list(local.LOCAL_DIMS), "default_dim": local.DEFAULT_LOCAL_DIM,
|
||||
"ready": True, "downloaded": local.is_downloaded(self.data_dir)},
|
||||
"ready": ollama["ready"]},
|
||||
},
|
||||
"index_model": self._model,
|
||||
"index_dim": self._dim,
|
||||
"index_soft": self._soft,
|
||||
"local": {
|
||||
"devices": local.available_devices(),
|
||||
"options": local.current_config(),
|
||||
"default_threads": local.default_threads(),
|
||||
"soft_tokens": list(local.SOFT_TOKENS),
|
||||
"default_soft": local.DEFAULT_SOFT,
|
||||
},
|
||||
"ollama": ollama,
|
||||
"indexed_total": len(self._vecs),
|
||||
"job": dict(self._job),
|
||||
}
|
||||
@@ -279,8 +275,7 @@ class SemanticService:
|
||||
|
||||
# ── インデックス作成ジョブ ────────────────────────────
|
||||
|
||||
def start_index(self, folder: str, recursive: bool, mode: str, model: str, dim: int,
|
||||
soft: int = local.DEFAULT_SOFT) -> None:
|
||||
def start_index(self, folder: str, recursive: bool, mode: str, model: str, dim: int) -> None:
|
||||
self._ensure_loaded()
|
||||
if model not in DIMS_BY_MODEL:
|
||||
raise SemanticError("bad_model", model)
|
||||
@@ -288,15 +283,19 @@ class SemanticService:
|
||||
raise SemanticError("no_api_key")
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
raise SemanticError("bad_dim", str(dim))
|
||||
if model != LOCAL_MODEL or soft not in local.SOFT_TOKENS:
|
||||
soft = local.DEFAULT_SOFT # ソフトトークン数はローカルモデルのみの設定
|
||||
if model == LOCAL_MODEL:
|
||||
st = local.status() # 接続できない/モデル未取得なら、ジョブを始める前に知らせる
|
||||
if not st["reachable"]:
|
||||
raise SemanticError("ollama_unreachable", f"{st['url']}: {st['error']}")
|
||||
if not st["ready"]:
|
||||
raise SemanticError("ollama_model_missing", st["tag"])
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
mismatch = bool(self._vecs) and (self._dim != dim or self._model != model or self._soft != soft)
|
||||
mismatch = bool(self._vecs) and (self._dim != dim or self._model != model)
|
||||
if mismatch and mode != "all":
|
||||
raise SemanticError("dim_mismatch", f"{self._model}/{self._dim}/{self._soft}")
|
||||
raise SemanticError("dim_mismatch", f"{self._model}/{self._dim}")
|
||||
if mode == "all":
|
||||
# 範囲内の既存ベクトルを破棄して作り直す(モデル/次元違いなら全消去)
|
||||
if mismatch:
|
||||
@@ -309,14 +308,13 @@ class SemanticService:
|
||||
self._mat_cache = None
|
||||
self._dim = dim
|
||||
self._model = model
|
||||
self._soft = soft
|
||||
todo = [e for e in entries if not self._is_fresh(e[1], e[2], e[3])]
|
||||
# スコープ内で消えたファイルの残骸を掃除(スコープ配下のキーのみ)
|
||||
self._prune_missing(folder, recursive, {e[1] for e in entries})
|
||||
self._cancel.clear()
|
||||
self._job = self._idle_job()
|
||||
self._job.update(running=True, total=len(todo))
|
||||
self._thread = threading.Thread(target=self._run_index, args=(todo, model, dim, soft), daemon=True)
|
||||
self._thread = threading.Thread(target=self._run_index, args=(todo, model, dim), daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _prune_missing(self, folder: str, recursive: bool, existing: set[str]) -> None:
|
||||
@@ -335,52 +333,78 @@ class SemanticService:
|
||||
def cancel(self) -> None:
|
||||
self._cancel.set()
|
||||
|
||||
def _run_index(self, todo: list, model: str, dim: int, soft: int = local.DEFAULT_SOFT) -> None:
|
||||
def work(entry):
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
name, ap, size, mtime = entry
|
||||
self._job["current"] = name
|
||||
try:
|
||||
vec = self._embed_image(model, dim, Path(ap), soft)
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
except SemanticError as e:
|
||||
if e.code in ("no_api_key", "api_key_invalid"):
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
self._cancel.set()
|
||||
else:
|
||||
self._job["failed"] += 1
|
||||
if not self._job["error"]:
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: embed failed for %s: %s", ap, e)
|
||||
def _run_index(self, todo: list, model: str, dim: int) -> None:
|
||||
def fail(e: SemanticError) -> None:
|
||||
if e.code in ("no_api_key", "api_key_invalid", "ollama_unreachable", "ollama_model_missing"):
|
||||
# 続けても全件失敗するので中止する
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
self._cancel.set()
|
||||
else:
|
||||
self._job["failed"] += 1
|
||||
if not self._job["error"]:
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
|
||||
def checkpoint() -> None:
|
||||
if self._dirty >= _CHECKPOINT_EVERY:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: checkpoint failed: %s", e)
|
||||
|
||||
def work(entry):
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
name, ap, size, mtime = entry
|
||||
self._job["current"] = name
|
||||
try:
|
||||
vec = self._embed_image(model, dim, Path(ap))
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
except SemanticError as e:
|
||||
fail(e)
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: embed failed for %s: %s", ap, e)
|
||||
self._job["failed"] += 1
|
||||
checkpoint()
|
||||
|
||||
def work_batch(batch):
|
||||
"""Ollama: まとめて送る。失敗したら1枚ずつ送り直して原因の画像だけを失敗扱いにする"""
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
self._job["current"] = batch[0][0]
|
||||
vecs = None
|
||||
try:
|
||||
ims = [self._load_local_image(Path(e[1])) for e in batch]
|
||||
vecs = local.embed_images(ims, dim)
|
||||
except local.OllamaError as e:
|
||||
if e.code != "local_error":
|
||||
fail(SemanticError(e.code, str(e)))
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
if vecs is None:
|
||||
for entry in batch:
|
||||
work(entry)
|
||||
return
|
||||
for (name, ap, size, mtime), vec in zip(batch, vecs):
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
checkpoint()
|
||||
|
||||
try:
|
||||
if model == LOCAL_MODEL:
|
||||
# 初回はモデルのダウンロード/ロードを先に行う(失敗したらジョブ全体を中止)
|
||||
try:
|
||||
local.ensure_runtime(self.data_dir, progress=lambda ph: self._job.update(current=f"({ph} model...)"))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: local model load failed: %s", e)
|
||||
self._job["error_code"], self._job["error"] = "local_error", str(e)
|
||||
self._cancel.set()
|
||||
workers = 1 if model == LOCAL_MODEL else _WORKERS # ローカルはORT側がマルチスレッド
|
||||
with ThreadPoolExecutor(max_workers=workers) as ex:
|
||||
list(ex.map(work, todo))
|
||||
for i in range(0, len(todo), local.BATCH):
|
||||
work_batch(todo[i:i + local.BATCH])
|
||||
else:
|
||||
with ThreadPoolExecutor(max_workers=_WORKERS) as ex:
|
||||
list(ex.map(work, todo))
|
||||
finally:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: save failed: %s", e)
|
||||
if model == LOCAL_MODEL:
|
||||
local.release() # 画像生成とVRAM/RAMを奪い合わないよう、作成後はモデルを解放(検索時に再ロード)
|
||||
local.release() # 画像生成とVRAMを奪い合わないよう、作成後はモデルを外す(検索時にOllamaが再ロード)
|
||||
self._job["running"] = False
|
||||
self._job["finished"] = True
|
||||
self._job["cancelled"] = self._cancel.is_set() and not self._job["error_code"]
|
||||
@@ -401,18 +425,24 @@ class SemanticService:
|
||||
im.thumbnail((IMAGE_MAX_SIZE, IMAGE_MAX_SIZE), Image.LANCZOS)
|
||||
return im
|
||||
|
||||
def _embed_image(self, model: str, dim: int, path: Path, soft: int | None = None) -> np.ndarray:
|
||||
def _load_local_image(self, path: Path):
|
||||
"""Ollama用のRGB画像(透過部は白背景に合成)"""
|
||||
from PIL import Image
|
||||
im = self._load_pil(path)
|
||||
if im.mode == "RGBA":
|
||||
bg = Image.new("RGB", im.size, (255, 255, 255))
|
||||
bg.paste(im, mask=im.getchannel("A"))
|
||||
return bg
|
||||
return im
|
||||
|
||||
def _embed_image(self, model: str, dim: int, path: Path) -> np.ndarray:
|
||||
if model == LOCAL_MODEL:
|
||||
if im.mode == "RGBA": # 透過部は白背景に合成
|
||||
bg = Image.new("RGB", im.size, (255, 255, 255))
|
||||
bg.paste(im, mask=im.getchannel("A"))
|
||||
im = bg
|
||||
im = self._load_local_image(path)
|
||||
try:
|
||||
return local.embed_image(self.data_dir, im, dim, self._soft if soft is None else soft)
|
||||
except Exception as e:
|
||||
raise SemanticError("local_error", str(e))
|
||||
return local.embed_images([im], dim)[0]
|
||||
except local.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
im = self._load_pil(path)
|
||||
buf = io.BytesIO()
|
||||
if im.mode == "RGBA":
|
||||
im.save(buf, "PNG")
|
||||
@@ -423,11 +453,47 @@ class SemanticService:
|
||||
def _embed_text(self, model: str, dim: int, text: str) -> np.ndarray:
|
||||
if model == LOCAL_MODEL:
|
||||
try:
|
||||
return local.embed_text(self.data_dir, text, dim)
|
||||
except Exception as e:
|
||||
raise SemanticError("local_error", str(e))
|
||||
return local.embed_text(text, dim)
|
||||
except local.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
return embed_text(text, dim)
|
||||
|
||||
# ── 他機能(TagSpace)向けの読み出し ─────────────────────
|
||||
|
||||
def index_info(self) -> dict:
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return {"model": self._model, "dim": self._dim, "count": len(self._vecs)}
|
||||
|
||||
def job_status(self) -> dict:
|
||||
return dict(self._job)
|
||||
|
||||
def vectors_for(self, entries: list) -> dict[str, np.ndarray]:
|
||||
"""entries([(name, abs_path, size, mtime)])のうち、索引済みで最新のもののベクトル"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return {ap: self._vecs[ap] for _, ap, sz, mt in entries
|
||||
if ap in self._vecs and self._is_fresh(ap, sz, mt)}
|
||||
|
||||
def indexed_entries(self, exclude_prefix: str = "") -> list[tuple[str, int, float]]:
|
||||
"""索引済みの (path, size, mtime) 一覧。exclude_prefix配下は除く"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return [(p, *self._stamp[p]) for p in self._vecs
|
||||
if not (exclude_prefix and p.startswith(exclude_prefix))]
|
||||
|
||||
def sample_vectors(self, n: int, exclude_prefix: str = "", seed: int = 0) -> tuple[list[str], np.ndarray]:
|
||||
"""索引から最大n件を無作為に(seed固定で)取り出す。exclude_prefix配下は除く"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
paths = [p for p in self._vecs if not (exclude_prefix and p.startswith(exclude_prefix))]
|
||||
if len(paths) > n:
|
||||
pick = np.random.default_rng(seed).choice(len(paths), n, replace=False)
|
||||
paths = [paths[i] for i in sorted(pick)]
|
||||
mat = (np.stack([self._vecs[p] for p in paths]) if paths
|
||||
else np.zeros((0, max(self._dim, 1)), np.float32))
|
||||
return paths, mat
|
||||
|
||||
# ── 検索 ───────────────────────────────────────────────
|
||||
|
||||
def _matrix(self):
|
||||
@@ -476,7 +542,10 @@ class SemanticService:
|
||||
|
||||
def search_similar(self, path: str, folder: str, recursive: bool, limit: int, min_score: float = 0.0) -> list[dict]:
|
||||
self._ensure_loaded()
|
||||
p = Path(path).resolve()
|
||||
try:
|
||||
p = self.gallery._resolve_untrusted(path) # UNC等は resolve() 前に文字列で拒否
|
||||
except ValueError:
|
||||
raise SemanticError("folder_not_allowed", path)
|
||||
if not p.is_file() or not self.gallery._check_path_allowed(p):
|
||||
raise SemanticError("folder_not_allowed", path)
|
||||
ap = p.as_posix()
|
||||
@@ -510,4 +579,4 @@ class SemanticService:
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
dim = DEFAULT_DIM_BY_MODEL[model]
|
||||
n = int(self._embed_text(model, dim, "test").shape[0])
|
||||
return {"dimensions": n, "provider": local.active_provider() if model == LOCAL_MODEL else "API"}
|
||||
return {"dimensions": n, "provider": f"Ollama {local.current_config()['tag']}" if model == LOCAL_MODEL else "API"}
|
||||
|
||||
+434
-160
@@ -1,192 +1,347 @@
|
||||
"""AI skill (.md system prompt) file management service."""
|
||||
"""AI skill (.md system prompt) file management service.
|
||||
|
||||
Skills are flat ``*.md`` files with a YAML frontmatter block::
|
||||
|
||||
---
|
||||
name: Display name
|
||||
description: One sentence
|
||||
metadata: # optional, Workflow Studio extensions
|
||||
wfs-version: 2 # built-in skills only (upgrade tracking)
|
||||
output: prompt # prompt | svg -> auto-check + action buttons
|
||||
tools: [] # [] or [generate_image]
|
||||
temperature: 0.4
|
||||
think: false
|
||||
context: [workflow]
|
||||
---
|
||||
|
||||
``name``/``description`` follow the Agent Skills (SKILL.md) convention so skills
|
||||
stay portable; everything Workflow-Studio-specific lives under ``metadata``.
|
||||
|
||||
Built-in skills ship in ``builtin_skills/`` next to this file and are copied
|
||||
into the user's skill folder. A small state file records which version and
|
||||
content hash were installed, so an untouched built-in is upgraded silently
|
||||
while a user-edited one is left alone (and flagged ``update_available``).
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import zipfile
|
||||
from pathlib import Path, PurePosixPath
|
||||
|
||||
try:
|
||||
import yaml # PyYAML ships with ComfyUI
|
||||
except Exception: # pragma: no cover - fallback parser below
|
||||
yaml = None
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
ALLOWED_EXTS = {"md"}
|
||||
|
||||
DEFAULT_SKILL_FILENAME = "svg-icon.md"
|
||||
# Folder skills (Agent Skills / SKILL.md standard): <skills_dir>/<folder>/SKILL.md plus optional
|
||||
# reference files. Only text resources are kept and served - Workflow Studio never runs a
|
||||
# skill's scripts, so script files are skipped on import.
|
||||
SKILL_MD = "SKILL.md"
|
||||
RESOURCE_EXTS = {"md", "txt", "json", "yaml", "yml", "csv"}
|
||||
MAX_RESOURCES = 50
|
||||
MAX_RESOURCE_BYTES = 200 * 1024 # per file served to the chat model
|
||||
MAX_IMPORT_FILE_BYTES = 1024 * 1024 # per file inside an imported zip
|
||||
MAX_IMPORT_TOTAL_BYTES = 5 * 1024 * 1024
|
||||
MAX_IMPORT_ENTRIES = 300
|
||||
_FOLDER_RE = re.compile(r"^[A-Za-z0-9][\w.\- ]{0,63}$")
|
||||
|
||||
DEFAULT_SKILL_CONTENT = """---
|
||||
name: SVG Icon Generator
|
||||
description: Produces clean, minimal, valid SVG icons and simple illustrations.
|
||||
---
|
||||
BUILTIN_DIR = Path(__file__).parent / "builtin_skills"
|
||||
STATE_FILENAME = ".builtin_state.json"
|
||||
|
||||
You are an expert SVG icon and illustration designer. When the user asks for an icon, logo, pictogram, diagram, character, animal, or any vector graphic, respond with valid, clean SVG markup.
|
||||
|
||||
## Composition process (do this before writing markup - skipping this step is the main cause of unrecognizable output)
|
||||
- Break the subject down into 3-8 distinct parts that make it recognizable (e.g. a bird: body, head, beak, eye, wing, tail; a house: wall, roof, door, window). Drop parts that are not essential.
|
||||
- Decide each part's approximate position, size, and draw order within the viewBox (elements listed later in the markup are drawn on top).
|
||||
- Pick the simplest shape that captures each part's silhouette: `<circle>`/`<ellipse>` for round parts, `<rect>` for rectangular parts, `<polygon>` for angular parts (beaks, ears, leaves, roofs). For a rounded organic outline that no primitive can approximate well (a body silhouette, a wing, a leaf edge), use a short `<path>` with 2-4 smooth curve commands (`Q` or `C`) instead of a plain circle - a few deliberate curve points beat a distorted primitive.
|
||||
- Never fall back to drawing an unrelated blob plus a few disconnected decorative lines when a subject is hard to render - always decompose it into the named parts above instead, even if each part is crude.
|
||||
|
||||
## Output rules
|
||||
- Always wrap the SVG markup in a fenced code block labeled `svg`, e.g. ```svg ... ```.
|
||||
- Keep any explanation brief and place it OUTSIDE the code block, never mixed inside the SVG.
|
||||
- Output exactly one `<svg>...</svg>` element per code block.
|
||||
|
||||
## Structure rules
|
||||
- Always include a `viewBox` attribute (e.g. `viewBox="0 0 24 24"`). Do not hardcode large pixel `width`/`height`; if you set them, make them match the viewBox aspect ratio.
|
||||
- Use a small, round coordinate system (0-24 or 0-100).
|
||||
- Round coordinate values to at most 2 decimal places.
|
||||
- Keep every `<path>` short (at most about 6 commands) - long, intricate paths are where quality breaks down most often.
|
||||
- Group related elements with `<g>` when it clarifies structure, but do not over-nest.
|
||||
|
||||
## Color rules
|
||||
- For single-color icons, use `fill="currentColor"` (or `stroke="currentColor"` for line icons) so the icon inherits the surrounding text color.
|
||||
- For multi-color illustrations, use explicit hex colors (e.g. `#4A90D9`) - never named CSS colors like `"blue"`.
|
||||
- Avoid gradients, filters, and blend modes unless the user explicitly asks for a glossy/gradient style - they rarely render correctly from generated coordinates.
|
||||
|
||||
## Accessibility
|
||||
- Include a `<title>` element as the first child of `<svg>` describing what the icon depicts.
|
||||
|
||||
## Security (mandatory - do not violate)
|
||||
- Never include `<script>`, `<foreignObject>`, event handler attributes (`onload`, `onclick`, etc.), or `<a>`/`href` links to external URLs.
|
||||
- Never reference external resources (no `xlink:href` to remote URLs, no `<image>` with external `src`).
|
||||
|
||||
## Example 1 - simple geometric icon (single color, primitives only)
|
||||
|
||||
Request: "Sun icon"
|
||||
|
||||
Parts: disc (circle), 8 rays (lines).
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Sun</title>
|
||||
<circle cx="12" cy="12" r="5" fill="currentColor"/>
|
||||
<g stroke="currentColor" stroke-width="2" stroke-linecap="round">
|
||||
<line x1="12" y1="1" x2="12" y2="4"/>
|
||||
<line x1="12" y1="20" x2="12" y2="23"/>
|
||||
<line x1="4.22" y1="4.22" x2="6.34" y2="6.34"/>
|
||||
<line x1="17.66" y1="17.66" x2="19.78" y2="19.78"/>
|
||||
<line x1="1" y1="12" x2="4" y2="12"/>
|
||||
<line x1="20" y1="12" x2="23" y2="12"/>
|
||||
<line x1="4.22" y1="19.78" x2="6.34" y2="17.66"/>
|
||||
<line x1="17.66" y1="6.34" x2="19.78" y2="4.22"/>
|
||||
</g>
|
||||
</svg>
|
||||
```
|
||||
|
||||
## Example 2 - organic subject (multi-color, decomposed into parts)
|
||||
|
||||
Request: "Bird icon"
|
||||
|
||||
Parts: body (rounded outline via a short curved path), head (circle), beak (small triangle), eye (dot), wing (small curved path) - each drawn as its own element, layered head/beak/eye on top of the body.
|
||||
|
||||
```svg
|
||||
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
|
||||
<title>Bird</title>
|
||||
<path d="M4 16 Q4 10 11 9 Q18 8 20 13 Q20 17 14 18 Q7 19 4 16 Z" fill="#4A90D9"/>
|
||||
<path d="M8 14 Q5 12 3 14 Q6 16 9 16 Z" fill="#3576B8"/>
|
||||
<circle cx="17" cy="10" r="3.2" fill="#4A90D9"/>
|
||||
<polygon points="20,10 23,9.3 20,11.4" fill="#F2B807"/>
|
||||
<circle cx="18" cy="9" r="0.6" fill="#1A1A1A"/>
|
||||
</svg>
|
||||
```
|
||||
|
||||
Follow this pattern for every request: list the parts first, represent each with the simplest adequate shape, then assemble - never skip straight to unstructured shapes.
|
||||
"""
|
||||
|
||||
SKILL_CREATOR_FILENAME = "skill-creator.md"
|
||||
|
||||
SKILL_CREATOR_CONTENT = """---
|
||||
name: Skill Creator
|
||||
description: Interactively interviews you and drafts a new WFS chat skill (.md system prompt) ready to save.
|
||||
---
|
||||
|
||||
You are a skill-design assistant embedded in ComfyUI Workflow Studio's AI TOOL Chat pane. Your job is to interactively help the user design a NEW "skill" - a markdown file with frontmatter (`name`, `description`) followed by system-prompt instructions. Once saved, a skill can be selected from a dropdown and is silently prepended to future chat messages as a system prompt.
|
||||
|
||||
## How to conduct the interview
|
||||
- Ask ONE focused question at a time, never a bulleted list of questions. Wait for the user's answer before asking the next one.
|
||||
- Cover, in this order, only what is not already clear from context:
|
||||
1. What task or domain should the skill specialize in (what should the model do differently once this skill is active)?
|
||||
2. What output format is expected - plain prose, a specific fenced code block language, a fixed structure, a length limit?
|
||||
3. Any hard constraints or things to avoid (tone, forbidden content, security concerns, things past attempts got wrong)?
|
||||
4. Would a short worked example help smaller/local models follow the pattern more reliably? If yes, ask what a good example input/output looks like, or offer to draft one.
|
||||
- Keep it short - 3 to 5 questions is usually enough. If the user says "just generate it", "that's enough", or similar at any point, stop asking and produce the best skill you can from what you already have.
|
||||
|
||||
## Output format for the finished skill
|
||||
- Once you have enough information, output the complete skill file in a single fenced code block labeled `skill` (```skill ... ```). Never mix prose inside that block - explanation goes outside it.
|
||||
- The block must start with frontmatter exactly in this shape:
|
||||
---
|
||||
name: <short display name, shown in the skill dropdown>
|
||||
description: <one sentence, shown as a tooltip in the dropdown>
|
||||
---
|
||||
- After the frontmatter, write clear, imperative system-prompt instructions for the target skill: what role the model should take, concrete output-format rules, hard constraints, and - if useful - one short worked example following a "request -> ideal output" shape.
|
||||
- After the code block, tell the user in one line that they can click the "Save as new skill" button below the message to add it, or copy the block manually into a new skill.
|
||||
"""
|
||||
# Content hashes of built-in skills released before version tracking existed
|
||||
# (v0.3.79 - v0.9.4). A user file matching one of these is an untouched copy
|
||||
# and can be upgraded safely.
|
||||
LEGACY_BUILTIN_HASHES = {
|
||||
"svg-icon.md": {"f46b635abda182b26df38d4e49ec2a626642fcd98c94013cb67303c69aaf10c1"},
|
||||
"skill-creator.md": {"2fb31273298b828c6f18c7fd010be44d31588f881d04757e5c0f5f18bbad2313"},
|
||||
}
|
||||
|
||||
_FRONTMATTER_RE = re.compile(r"^---\r?\n(.*?)\r?\n---\r?\n?", re.DOTALL)
|
||||
|
||||
|
||||
def _parse_frontmatter(content: str) -> dict:
|
||||
"""Extract a shallow `key: value` frontmatter block, if present."""
|
||||
m = _FRONTMATTER_RE.match(content)
|
||||
if not m:
|
||||
return {}
|
||||
def _content_hash(content: str) -> str:
|
||||
return hashlib.sha256(content.replace("\r\n", "\n").strip().encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _parse_frontmatter_shallow(block: str) -> dict:
|
||||
"""Fallback parser: top-level ``key: value`` plus one nested ``metadata:`` level."""
|
||||
fields: dict = {}
|
||||
for line in m.group(1).splitlines():
|
||||
if ":" not in line:
|
||||
current_map: dict | None = None
|
||||
for line in block.splitlines():
|
||||
if not line.strip() or ":" not in line:
|
||||
continue
|
||||
key, _, value = line.partition(":")
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
if key:
|
||||
fields[key] = value
|
||||
indented = line[:1] in (" ", "\t")
|
||||
key, _, value = line.strip().partition(":")
|
||||
key, value = key.strip(), value.strip()
|
||||
if not key:
|
||||
continue
|
||||
if indented and current_map is not None:
|
||||
current_map[key] = _coerce_scalar(value)
|
||||
elif not indented:
|
||||
if value == "":
|
||||
current_map = {}
|
||||
fields[key] = current_map
|
||||
else:
|
||||
current_map = None
|
||||
fields[key] = value
|
||||
return fields
|
||||
|
||||
|
||||
class SkillService:
|
||||
def __init__(self, skills_dir: Path):
|
||||
self.skills_dir = skills_dir
|
||||
self.skills_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._seed_default_skill()
|
||||
def _coerce_scalar(value: str):
|
||||
if value.startswith("[") and value.endswith("]"):
|
||||
inner = value[1:-1].strip()
|
||||
return [v.strip().strip("'\"") for v in inner.split(",") if v.strip()] if inner else []
|
||||
low = value.lower()
|
||||
if low in ("true", "false"):
|
||||
return low == "true"
|
||||
try:
|
||||
return float(value) if "." in value else int(value)
|
||||
except ValueError:
|
||||
return value.strip("'\"")
|
||||
|
||||
def _seed_default_skill(self) -> None:
|
||||
"""Seed a starter skill file the first time the directory is empty."""
|
||||
|
||||
def _parse_frontmatter(content: str) -> dict:
|
||||
"""Extract the frontmatter block as a dict (empty dict if absent/invalid)."""
|
||||
m = _FRONTMATTER_RE.match(content or "")
|
||||
if not m:
|
||||
return {}
|
||||
block = m.group(1)
|
||||
if yaml is not None:
|
||||
try:
|
||||
has_any = any(
|
||||
p.is_file() and p.suffix.lstrip(".").lower() in ALLOWED_EXTS
|
||||
for p in self.skills_dir.iterdir()
|
||||
)
|
||||
except FileNotFoundError:
|
||||
has_any = False
|
||||
if not has_any:
|
||||
(self.skills_dir / DEFAULT_SKILL_FILENAME).write_text(
|
||||
DEFAULT_SKILL_CONTENT, encoding="utf-8"
|
||||
)
|
||||
(self.skills_dir / SKILL_CREATOR_FILENAME).write_text(
|
||||
SKILL_CREATOR_CONTENT, encoding="utf-8"
|
||||
)
|
||||
data = yaml.safe_load(block)
|
||||
if isinstance(data, dict):
|
||||
return data
|
||||
except Exception:
|
||||
pass # e.g. an unquoted ": " inside description -> shallow parse
|
||||
return _parse_frontmatter_shallow(block)
|
||||
|
||||
|
||||
def _json_safe(value):
|
||||
"""Keep only JSON-serialisable metadata values (YAML can yield dates etc.)."""
|
||||
if isinstance(value, dict):
|
||||
return {str(k): _json_safe(v) for k, v in value.items()}
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [_json_safe(v) for v in value]
|
||||
if isinstance(value, (str, int, float, bool)) or value is None:
|
||||
return value
|
||||
return str(value)
|
||||
|
||||
|
||||
def _skill_info(filename: str, content: str, size: int) -> dict:
|
||||
meta = _parse_frontmatter(content)
|
||||
extra = meta.get("metadata") if isinstance(meta.get("metadata"), dict) else {}
|
||||
return {
|
||||
"filename": filename,
|
||||
"name": str(meta.get("name") or Path(filename).stem),
|
||||
"description": str(meta.get("description") or ""),
|
||||
"metadata": _json_safe(extra),
|
||||
"size": size,
|
||||
}
|
||||
|
||||
|
||||
def _builtin_version(content: str) -> int:
|
||||
meta = _parse_frontmatter(content).get("metadata") or {}
|
||||
try:
|
||||
return int(meta.get("wfs-version", 1))
|
||||
except (TypeError, ValueError):
|
||||
return 1
|
||||
|
||||
|
||||
class SkillService:
|
||||
def __init__(self, skills_dir: Path, builtin_dir: Path = BUILTIN_DIR):
|
||||
self.skills_dir = skills_dir
|
||||
self.builtin_dir = builtin_dir
|
||||
self.skills_dir.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
self.sync_builtin_skills()
|
||||
except Exception as e: # never block startup on a seeding problem
|
||||
logger.warning("[WFS] Built-in skill sync failed: %s", e)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Built-in skills
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _builtins(self) -> dict[str, str]:
|
||||
if not self.builtin_dir.is_dir():
|
||||
return {}
|
||||
return {
|
||||
p.name: p.read_text(encoding="utf-8")
|
||||
for p in sorted(self.builtin_dir.glob("*.md"))
|
||||
if p.is_file()
|
||||
}
|
||||
|
||||
def _load_state(self) -> dict:
|
||||
path = self.skills_dir / STATE_FILENAME
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
return data if isinstance(data, dict) else {}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
def _save_state(self, state: dict) -> None:
|
||||
(self.skills_dir / STATE_FILENAME).write_text(
|
||||
json.dumps(state, indent=2, ensure_ascii=False), encoding="utf-8"
|
||||
)
|
||||
|
||||
def _install_builtin(self, filename: str, content: str, state: dict) -> None:
|
||||
(self.skills_dir / filename).write_text(content, encoding="utf-8")
|
||||
state[filename] = {"version": _builtin_version(content), "hash": _content_hash(content)}
|
||||
|
||||
def sync_builtin_skills(self) -> dict:
|
||||
"""Install new built-ins and upgrade untouched ones.
|
||||
|
||||
Rules per built-in file:
|
||||
- missing and never installed -> install
|
||||
- missing but installed before -> the user deleted it, leave it deleted
|
||||
- present, untouched (hash matches what we installed, or a known
|
||||
pre-tracking release) and older -> overwrite with the new version
|
||||
- present but edited by the user -> keep it (reported as update_available)
|
||||
Returns ``{"installed": [...], "upgraded": [...]}``.
|
||||
"""
|
||||
state = self._load_state()
|
||||
result = {"installed": [], "upgraded": []}
|
||||
changed = False
|
||||
for filename, content in self._builtins().items():
|
||||
new_version = _builtin_version(content)
|
||||
path = self.skills_dir / filename
|
||||
entry = state.get(filename)
|
||||
if not path.is_file():
|
||||
if entry is None:
|
||||
self._install_builtin(filename, content, state)
|
||||
result["installed"].append(filename)
|
||||
changed = True
|
||||
continue
|
||||
current_hash = _content_hash(path.read_text(encoding="utf-8"))
|
||||
if entry is None:
|
||||
if current_hash in LEGACY_BUILTIN_HASHES.get(filename, set()):
|
||||
self._install_builtin(filename, content, state)
|
||||
result["upgraded"].append(filename)
|
||||
elif current_hash == _content_hash(content):
|
||||
state[filename] = {"version": new_version, "hash": current_hash}
|
||||
else:
|
||||
# A user file with a built-in's name - never overwrite it.
|
||||
state[filename] = {"version": 0, "hash": None}
|
||||
changed = True
|
||||
continue
|
||||
if int(entry.get("version") or 0) < new_version and current_hash == entry.get("hash"):
|
||||
self._install_builtin(filename, content, state)
|
||||
result["upgraded"].append(filename)
|
||||
changed = True
|
||||
if changed:
|
||||
self._save_state(state)
|
||||
if result["installed"] or result["upgraded"]:
|
||||
logger.info("[WFS] Built-in skills installed=%s upgraded=%s", result["installed"], result["upgraded"])
|
||||
return result
|
||||
|
||||
def restore_builtin(self, filename: str) -> dict:
|
||||
"""Overwrite a skill file with the shipped built-in version (explicit user action)."""
|
||||
builtins = self._builtins()
|
||||
if filename not in builtins:
|
||||
raise ValueError(f"Not a built-in skill: {filename}")
|
||||
state = self._load_state()
|
||||
self._install_builtin(filename, builtins[filename], state)
|
||||
self._save_state(state)
|
||||
path = self.skills_dir / filename
|
||||
return _skill_info(filename, builtins[filename], path.stat().st_size)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# File CRUD
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def list_skills(self) -> list[dict]:
|
||||
"""Return sorted list of skill files with parsed frontmatter metadata."""
|
||||
"""Return sorted list of flat skill files and folder skills with parsed metadata."""
|
||||
if not self.skills_dir.exists():
|
||||
return []
|
||||
builtins = self._builtins()
|
||||
state = self._load_state()
|
||||
files = []
|
||||
for path in sorted(self.skills_dir.glob("*")):
|
||||
if path.is_dir():
|
||||
info = self._folder_skill_info(path)
|
||||
if info:
|
||||
files.append(info)
|
||||
continue
|
||||
if not (path.is_file() and path.suffix.lstrip(".").lower() in ALLOWED_EXTS):
|
||||
continue
|
||||
try:
|
||||
content = path.read_text(encoding="utf-8")
|
||||
except Exception:
|
||||
content = ""
|
||||
meta = _parse_frontmatter(content)
|
||||
files.append({
|
||||
"filename": path.name,
|
||||
"name": meta.get("name") or path.stem,
|
||||
"description": meta.get("description", ""),
|
||||
"size": path.stat().st_size,
|
||||
})
|
||||
info = _skill_info(path.name, content, path.stat().st_size)
|
||||
builtin_content = builtins.get(path.name)
|
||||
info["builtin"] = builtin_content is not None
|
||||
info["update_available"] = False
|
||||
if builtin_content is not None:
|
||||
current_hash = _content_hash(content)
|
||||
if current_hash != _content_hash(builtin_content):
|
||||
entry = state.get(path.name) or {}
|
||||
installed_version = int(entry.get("version") or 0)
|
||||
info["update_available"] = installed_version < _builtin_version(builtin_content)
|
||||
info["modified"] = True
|
||||
files.append(info)
|
||||
return files
|
||||
|
||||
def _folder_skill_info(self, folder: Path) -> dict | None:
|
||||
if folder.name.startswith(".") or not _FOLDER_RE.match(folder.name):
|
||||
return None
|
||||
skill_md = folder / SKILL_MD
|
||||
if not skill_md.is_file():
|
||||
return None
|
||||
try:
|
||||
content = skill_md.read_text(encoding="utf-8")
|
||||
except Exception:
|
||||
content = ""
|
||||
info = _skill_info(f"{folder.name}/{SKILL_MD}", content, skill_md.stat().st_size)
|
||||
if info["name"] == SKILL_MD.rsplit(".", 1)[0]:
|
||||
info["name"] = folder.name
|
||||
info["folder"] = folder.name
|
||||
info["resources"] = self.list_resources(folder.name)
|
||||
info["builtin"] = False
|
||||
info["update_available"] = False
|
||||
return info
|
||||
|
||||
def list_resources(self, folder: str) -> list[str]:
|
||||
"""Readable reference files of a folder skill (relative POSIX paths, SKILL.md excluded)."""
|
||||
base = self._folder_path(folder)
|
||||
if base is None or not base.is_dir():
|
||||
return []
|
||||
out = []
|
||||
for path in sorted(base.rglob("*")):
|
||||
if len(out) >= MAX_RESOURCES:
|
||||
break
|
||||
if not path.is_file() or path.is_symlink():
|
||||
continue
|
||||
rel = path.relative_to(base).as_posix()
|
||||
if rel == SKILL_MD or any(part.startswith(".") for part in rel.split("/")):
|
||||
continue
|
||||
if path.suffix.lstrip(".").lower() in RESOURCE_EXTS:
|
||||
out.append(rel)
|
||||
return out
|
||||
|
||||
def get_resource(self, folder: str, rel_path: str) -> str:
|
||||
"""Read one reference file of a folder skill (used by the chat's read_skill_file tool)."""
|
||||
base = self._folder_path(folder)
|
||||
if base is None or not base.is_dir():
|
||||
raise FileNotFoundError(f"Skill folder not found: {folder}")
|
||||
rel = PurePosixPath(str(rel_path or "").replace("\\", "/"))
|
||||
if rel.is_absolute() or ".." in rel.parts or not rel.parts or rel.as_posix() == SKILL_MD:
|
||||
raise ValueError(f"Invalid resource path: {rel_path}")
|
||||
if rel.suffix.lstrip(".").lower() not in RESOURCE_EXTS:
|
||||
raise ValueError(f"Unsupported resource type: {rel_path}")
|
||||
path = (base / rel).resolve()
|
||||
if base.resolve() not in path.parents or not path.is_file():
|
||||
raise FileNotFoundError(f"Resource not found: {rel_path}")
|
||||
data = path.read_bytes()[:MAX_RESOURCE_BYTES]
|
||||
return data.decode("utf-8", errors="replace")
|
||||
|
||||
def get_content(self, filename: str) -> str:
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
@@ -199,30 +354,149 @@ class SkillService:
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
raise ValueError(f"Invalid filename: {filename}")
|
||||
if path.name == SKILL_MD and not path.parent.is_dir():
|
||||
raise FileNotFoundError(f"Skill folder not found: {filename}")
|
||||
path.write_text(content, encoding="utf-8")
|
||||
meta = _parse_frontmatter(content)
|
||||
return {
|
||||
"filename": path.name,
|
||||
"name": meta.get("name") or path.stem,
|
||||
"description": meta.get("description", ""),
|
||||
"size": path.stat().st_size,
|
||||
}
|
||||
rel = path.relative_to(self.skills_dir).as_posix()
|
||||
return _skill_info(rel, content, path.stat().st_size)
|
||||
|
||||
def delete_file(self, filename: str) -> None:
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
raise ValueError(f"Invalid filename: {filename}")
|
||||
if path.name == SKILL_MD and path.parent != self.skills_dir:
|
||||
# A folder skill is deleted as a whole (SKILL.md + its reference files).
|
||||
if path.parent.is_dir():
|
||||
shutil.rmtree(path.parent)
|
||||
return
|
||||
if path.is_file():
|
||||
path.unlink()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Import (Agent Skills zip or a single .md)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def import_skill(self, upload_name: str, data: bytes) -> dict:
|
||||
"""Import an uploaded skill.
|
||||
|
||||
- ``*.md``: saved as a flat skill (``<slug>.md``; SKILL.md uploads are named after their
|
||||
frontmatter name).
|
||||
- ``*.zip``: an Agent Skills folder - SKILL.md at the zip root or inside exactly one top
|
||||
folder. Only text resources are extracted; scripts and binaries are skipped.
|
||||
Returns ``{"skill": info, "skipped": [paths]}``. Never overwrites: a numeric suffix is
|
||||
added when the name is taken.
|
||||
"""
|
||||
name = (upload_name or "").strip()
|
||||
lower = name.lower()
|
||||
if lower.endswith(".md"):
|
||||
text = data.decode("utf-8", errors="replace")
|
||||
meta = _parse_frontmatter(text)
|
||||
stem = Path(name).stem if Path(name).name != SKILL_MD else str(meta.get("name") or "imported-skill")
|
||||
filename = self._unique_name(_slugify(stem), ".md")
|
||||
(self.skills_dir / filename).write_text(text, encoding="utf-8")
|
||||
return {"skill": _skill_info(filename, text, len(data)), "skipped": []}
|
||||
if not lower.endswith(".zip"):
|
||||
raise ValueError("Only .zip (Agent Skills folder) or .md files can be imported")
|
||||
return self._import_zip(data)
|
||||
|
||||
def _import_zip(self, data: bytes) -> dict:
|
||||
try:
|
||||
zf = zipfile.ZipFile(io.BytesIO(data))
|
||||
except zipfile.BadZipFile as e:
|
||||
raise ValueError(f"Not a valid zip file: {e}") from e
|
||||
infos = [i for i in zf.infolist() if not i.is_dir()]
|
||||
if len(infos) > MAX_IMPORT_ENTRIES:
|
||||
raise ValueError(f"Too many files in the zip ({len(infos)} > {MAX_IMPORT_ENTRIES})")
|
||||
|
||||
def parts_of(info):
|
||||
p = PurePosixPath(info.filename.replace("\\", "/"))
|
||||
if p.is_absolute() or ".." in p.parts or any(":" in part for part in p.parts):
|
||||
raise ValueError(f"Unsafe path in zip: {info.filename}")
|
||||
return p.parts
|
||||
|
||||
all_parts = [(i, parts_of(i)) for i in infos]
|
||||
# Ignore OS junk (__MACOSX/, .DS_Store) when locating SKILL.md.
|
||||
all_parts = [(i, ps) for i, ps in all_parts if ps and ps[0] != "__MACOSX" and not ps[-1].startswith(".")]
|
||||
skill_md_parts = [ps for _, ps in all_parts if ps[-1] == SKILL_MD and len(ps) <= 2]
|
||||
if len(skill_md_parts) != 1:
|
||||
raise ValueError("The zip must contain exactly one SKILL.md at its root or in one top-level folder")
|
||||
prefix = skill_md_parts[0][:-1] # () or (top_folder,)
|
||||
|
||||
skill_text = zf.read("/".join(skill_md_parts[0])).decode("utf-8", errors="replace")
|
||||
meta = _parse_frontmatter(skill_text)
|
||||
base_name = prefix[0] if prefix else str(meta.get("name") or "imported-skill")
|
||||
folder = self._unique_name(_slugify(base_name), "")
|
||||
target = self.skills_dir / folder
|
||||
target.mkdir(parents=True)
|
||||
|
||||
written, skipped, total = 0, [], 0
|
||||
try:
|
||||
for info, ps in all_parts:
|
||||
if tuple(ps[: len(prefix)]) != prefix:
|
||||
skipped.append(info.filename)
|
||||
continue
|
||||
rel = PurePosixPath(*ps[len(prefix):])
|
||||
is_skill_md = rel.as_posix() == SKILL_MD
|
||||
ext = rel.suffix.lstrip(".").lower()
|
||||
if not is_skill_md and ext not in RESOURCE_EXTS:
|
||||
skipped.append(info.filename)
|
||||
continue
|
||||
if info.file_size > MAX_IMPORT_FILE_BYTES:
|
||||
skipped.append(info.filename)
|
||||
continue
|
||||
total += info.file_size
|
||||
if total > MAX_IMPORT_TOTAL_BYTES:
|
||||
raise ValueError("The zip's text files exceed 5 MB in total")
|
||||
content = zf.read(info)[:MAX_IMPORT_FILE_BYTES]
|
||||
out = (target / rel).resolve()
|
||||
if target.resolve() not in out.parents:
|
||||
raise ValueError(f"Unsafe path in zip: {info.filename}")
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_bytes(content)
|
||||
written += 1
|
||||
except Exception:
|
||||
shutil.rmtree(target, ignore_errors=True)
|
||||
raise
|
||||
logger.info("[WFS] Imported skill folder %s (%d files, %d skipped)", folder, written, len(skipped))
|
||||
return {"skill": self._folder_skill_info(target), "skipped": skipped}
|
||||
|
||||
def _unique_name(self, stem: str, suffix: str) -> str:
|
||||
candidate = f"{stem}{suffix}"
|
||||
n = 2
|
||||
while (self.skills_dir / candidate).exists():
|
||||
candidate = f"{stem}-{n}{suffix}"
|
||||
n += 1
|
||||
return candidate
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Path safety
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _folder_path(self, folder: str) -> Path | None:
|
||||
name = (folder or "").strip()
|
||||
if not _FOLDER_RE.match(name) or name.startswith("."):
|
||||
return None
|
||||
return self.skills_dir / name
|
||||
|
||||
def _safe_path(self, filename: str) -> Path | None:
|
||||
"""Flat ``name.md`` or a folder skill's ``<folder>/SKILL.md`` - nothing else."""
|
||||
if not filename or not filename.strip():
|
||||
return None
|
||||
# Skills are a flat directory - reject any path separator (no subfolders).
|
||||
name = filename.strip()
|
||||
if "/" in name or "\\" in name or name in (".", ".."):
|
||||
name = filename.strip().replace("\\", "/")
|
||||
if "/" in name:
|
||||
folder, _, leaf = name.partition("/")
|
||||
if leaf != SKILL_MD:
|
||||
return None
|
||||
base = self._folder_path(folder)
|
||||
return base / SKILL_MD if base is not None else None
|
||||
if name in (".", "..") or name.startswith("."):
|
||||
return None
|
||||
ext = Path(name).suffix.lstrip(".").lower()
|
||||
if ext not in ALLOWED_EXTS:
|
||||
return None
|
||||
return self.skills_dir / name
|
||||
|
||||
|
||||
def _slugify(text: str) -> str:
|
||||
slug = re.sub(r"[^A-Za-z0-9]+", "-", str(text)).strip("-").lower()[:48]
|
||||
return slug or "imported-skill"
|
||||
|
||||
@@ -12,6 +12,8 @@ import threading
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from ..config import resolve_untrusted
|
||||
from typing import Optional
|
||||
from .url_guard import HINT as _URL_HINT, is_allowed_backend_url, urlopen as _guarded_urlopen
|
||||
|
||||
@@ -72,7 +74,10 @@ class TaggerService:
|
||||
continue
|
||||
exts = {f.suffix.lower() for f in item.iterdir() if f.is_file()}
|
||||
if ".onnx" in exts:
|
||||
models.append({"name": item.name, "type": "onnx"})
|
||||
entry = {"name": item.name, "type": "onnx"}
|
||||
if (item / "preprocess.json").exists() and (item / "selected_tags.csv").exists():
|
||||
entry["pixai"] = True
|
||||
models.append(entry)
|
||||
elif ".h5" in exts:
|
||||
models.append({"name": item.name, "type": "deepdanbooru"})
|
||||
return models
|
||||
@@ -128,6 +133,9 @@ class TaggerService:
|
||||
"input_shape": session.get_inputs()[0].shape,
|
||||
"type": "onnx",
|
||||
}
|
||||
categories = self._load_pixai_categories(model_dir, len(labels))
|
||||
if categories:
|
||||
data["pixai_categories"] = categories
|
||||
self._cache[name] = data
|
||||
logger.info("ONNX loaded: %s (%d labels)", name, len(labels))
|
||||
return data
|
||||
@@ -173,6 +181,20 @@ class TaggerService:
|
||||
logger.warning("Label load failed %s: %s", p, e)
|
||||
return []
|
||||
|
||||
@staticmethod
|
||||
def _load_pixai_categories(model_dir: Path, n_labels: int) -> Optional[list]:
|
||||
"""PixAI Tagger形式(preprocess.json + selected_tags.csvのcategory列)ならカテゴリ番号の配列を返す。"""
|
||||
if not (model_dir / "preprocess.json").exists():
|
||||
return None
|
||||
csv_path = model_dir / "selected_tags.csv"
|
||||
try:
|
||||
with open(csv_path, "r", encoding="utf-8-sig", newline="") as f:
|
||||
rows = list(csv.DictReader(f))
|
||||
cats = [int(r["category"]) for r in rows]
|
||||
except Exception:
|
||||
return None
|
||||
return cats if cats and len(cats) == n_labels else None
|
||||
|
||||
# ── 推論（WD Tagger / DeepDanbooru） ────────────────────
|
||||
|
||||
def predict(self, image_b64: str, model_name: str, threshold: float = 0.35, char_threshold: float = 0.85) -> dict:
|
||||
@@ -190,7 +212,74 @@ class TaggerService:
|
||||
logger.error("predict error: %s", e)
|
||||
return {"error": str(e)}
|
||||
|
||||
# WD Tagger の評価(rating)ラベル。TagSpace の補完では除く
|
||||
_RATING_LABELS = {"general", "sensitive", "questionable", "explicit"}
|
||||
|
||||
def tag_image(self, img: Image.Image, model_name: str) -> list:
|
||||
"""他機能(TagSpace)向け: ONNXモデルで画像1枚にタグを付ける。しきい値は既定値
|
||||
(WD 0.35/0.85、PixAIは推奨値)。評価ラベルは除く。読み込めなければ RuntimeError。"""
|
||||
data = self._load_model(model_name)
|
||||
if data is None or data.get("type") != "onnx":
|
||||
raise RuntimeError(f"Model '{model_name}' could not be loaded")
|
||||
if data.get("pixai_categories"):
|
||||
thr, char_thr = self._PIXAI_BASE["general"], self._PIXAI_BASE["character"]
|
||||
else:
|
||||
thr, char_thr = 0.35, 0.85
|
||||
return [t for t in self._predict_onnx(img, data, thr, char_thr) if t not in self._RATING_LABELS]
|
||||
|
||||
def model_provider(self, model_name: str) -> str:
|
||||
"""読み込み済みモデルの実行先(CUDA / CPU)。未読み込みなら空文字"""
|
||||
data = self._cache.get(model_name)
|
||||
try:
|
||||
return data["session"].get_providers()[0].replace("ExecutionProvider", "") if data else ""
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
# PixAI Tagger v1.0 推奨しきい値(category番号 -> しきい値)。meta/ratingはタグに含めない。
|
||||
# UIの2本のスライダーは推奨値(general 0.17 / character 0.27)を基準にした相対指定で、
|
||||
# general側(0,1)はthreshold、character側(3,4)はchar_thresholdに追従する。
|
||||
_PIXAI_THRESHOLDS = {0: 0.17, 1: 0.15, 3: 0.24, 4: 0.27}
|
||||
_PIXAI_BASE = {"general": 0.17, "character": 0.27}
|
||||
_PIXAI_SIZE = 1008
|
||||
|
||||
def _predict_pixai(self, img: Image.Image, data: dict, thr: float, char_thr: float) -> list:
|
||||
"""PixAI Tagger v1.0: 白背景合成 → 黒パディングで1008角 → RGB, (x/255-0.5)/0.5, NCHW。出力はsigmoid済み確率。"""
|
||||
size = self._PIXAI_SIZE
|
||||
img = ImageOps.exif_transpose(img)
|
||||
if img.mode != "RGBA":
|
||||
img = img.convert("RGBA")
|
||||
canvas = Image.new("RGBA", img.size, (255, 255, 255, 255))
|
||||
canvas.alpha_composite(img)
|
||||
img = canvas.convert("RGB")
|
||||
|
||||
scale = size / max(img.size)
|
||||
w, h = max(1, round(img.width * scale)), max(1, round(img.height * scale))
|
||||
img = img.resize((w, h), Image.Resampling.BICUBIC)
|
||||
padded = Image.new("RGB", (size, size), (0, 0, 0))
|
||||
padded.paste(img, ((size - w) // 2, (size - h) // 2))
|
||||
|
||||
arr = (np.asarray(padded, dtype=np.float32) / 255.0 - 0.5) / 0.5
|
||||
arr = np.ascontiguousarray(arr.transpose(2, 0, 1)[None])
|
||||
preds = data["session"].run(None, {data["input_name"]: arr})[0][0]
|
||||
|
||||
d_gen = thr - self._PIXAI_BASE["general"]
|
||||
d_chr = char_thr - self._PIXAI_BASE["character"]
|
||||
limits = {
|
||||
c: max(0.0, min(1.0, base + (d_gen if c in (0, 1) else d_chr)))
|
||||
for c, base in self._PIXAI_THRESHOLDS.items()
|
||||
}
|
||||
labels, cats = data["labels"], data["pixai_categories"]
|
||||
picked = [
|
||||
(float(score), labels[i])
|
||||
for i, score in enumerate(preds)
|
||||
if i < len(labels) and float(score) >= limits.get(cats[i], 2.0)
|
||||
]
|
||||
picked.sort(key=lambda t: t[0], reverse=True)
|
||||
return [name for _, name in picked]
|
||||
|
||||
def _predict_onnx(self, img: Image.Image, data: dict, thr: float, char_thr: float) -> list:
|
||||
if data.get("pixai_categories"):
|
||||
return self._predict_pixai(img, data, thr, char_thr)
|
||||
shape = data["input_shape"]
|
||||
fmt = "NHWC"
|
||||
size = 224
|
||||
@@ -369,7 +458,12 @@ class TaggerService:
|
||||
_ALLOWED_WRITE_EXTS = {".jpg", ".jpeg", ".png", ".webp", ".bmp", ".gif"}
|
||||
|
||||
def write_meta_to_file(self, image_path: str, tags: str) -> dict:
|
||||
path = Path(image_path).resolve()
|
||||
try:
|
||||
# UNC(\\host\share)は resolve() した時点で相手ホストへ資格情報付きで
|
||||
# 接続してしまうため、文字列のまま先に拒否する
|
||||
path = resolve_untrusted(image_path)
|
||||
except ValueError:
|
||||
return {"error": "Access denied"}
|
||||
if path.suffix.lower() not in self._ALLOWED_WRITE_EXTS:
|
||||
return {"error": "Unsupported file type"}
|
||||
if not path.is_file():
|
||||
|
||||
@@ -0,0 +1,686 @@
|
||||
"""
|
||||
TagSpace Service - WFSのデータを EmbeddingGemma 2 の同じベクトル空間に置き、3Dに投影する
|
||||
|
||||
カード : ImagePrompt(ws_image_prompt)の画像+プロンプト。ベクトルは Gallery セマンティック索引の画像ベクトル
|
||||
タグ : ImagePrompt/ワイルドカード/Tagger DB/Promptライブラリの文をカンマ等で分解した語。Ollamaでテキスト埋め込み
|
||||
背景 : Gallery 索引から間引いた画像(位置関係を見るための点群)
|
||||
|
||||
画像とテキストのベクトルは同じ空間でも別の塊に分かれる(modality gap)ため、
|
||||
画像群とテキスト群をそれぞれの平均で中心化してからPCAで3軸を求める。
|
||||
タグのベクトルはタグ文字列をキーに tagspace_tags.npz へキャッシュする(740m/440mは同じベクトル)。
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import sqlite3
|
||||
import threading
|
||||
import time
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import yaml
|
||||
|
||||
from . import semantic_ollama as ollama
|
||||
from .semantic_service import LOCAL_MODEL, SemanticError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_MAX_TAGS = 2000
|
||||
MAX_TAGS_LIMIT = 6000
|
||||
ALL_TAGS_LIMIT = 20000 # 「全タグ」(max_tags=0)でも超えない上限(ワイルドカード込み・出現1回以上だと約5万種になるため)
|
||||
MIN_COUNT_LIMIT = 10
|
||||
DEFAULT_BG = 4000
|
||||
BG_LIMIT = 20000
|
||||
_EMBED_BATCH = 32
|
||||
_RADIUS = 10.0 # 3D座標の広がり(98パーセンタイル半径)
|
||||
_IMG_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
_VAULT_THUMB_DIRS = {"thumbnails", "thumbnails_option2"}
|
||||
SOURCES = ("image_prompt", "gallery", "tagger_auto", "tagger", "prompt_library", "wildcard")
|
||||
TAGGER_SCOPES = ("background", "all") # Tagger補完の対象: 背景の点のみ / プロンプトの無い全画像
|
||||
_TAGGER_MAX_SIDE = 1024 # Taggerに渡す前の長辺上限(PixAIの1008pxより少し大きく)
|
||||
_TAGGER_SAVE_EVERY = 100
|
||||
_PROMPT_MAX = 4000 # キャッシュするプロンプトの最大文字数
|
||||
# Positive をたどるときに文字列として拾う入力名(CLIPTextEncode系・Qwen/Flux系のエンコーダー・文字列ノード)
|
||||
_TEXT_INPUTS = {"text", "prompt", "positive", "text_g", "text_l", "clip_l", "t5xxl", "string", "value",
|
||||
"user_prompt", "system_prompt_text"}
|
||||
_SKIP_INPUTS = {"negative", "negative_prompt", "neg"}
|
||||
|
||||
_SPLIT = re.compile(r"[,\n|{}]+|\bBREAK\b")
|
||||
_WEIGHT = re.compile(r":\s*-?\d+(?:\.\d+)?\s*$")
|
||||
_DP_WEIGHT = re.compile(r"^\s*\d+(?:\.\d+)?\s*::") # Dynamic Prompts の "2::option"
|
||||
_JSON_KEY = re.compile(r'"\s*:') # "key": の形
|
||||
_MODEL_FILE = re.compile(r"\.(safetensors|ckpt|pt|pth|bin|gguf)\b", re.I)
|
||||
_HAS_WORD = re.compile(r"[a-z-ヿ一-鿿]")
|
||||
_STOP = {"break", "and", "or", "the", "a", "an"}
|
||||
|
||||
|
||||
def normalize_tag(raw: str) -> str:
|
||||
"""プロンプトの断片をタグとして正規化する。タグにならなければ空文字"""
|
||||
s = _DP_WEIGHT.sub("", raw).strip()
|
||||
if not s or "<" in s or "__" in s or s.lower().startswith(("embedding:", "rating:")):
|
||||
return ""
|
||||
# プロンプトに埋め込まれたJSON(MOSS-Image のレイアウト指定 bbox / 文字入れ text 等)の断片と、モデルのファイル名
|
||||
if _JSON_KEY.search(s) or _MODEL_FILE.search(s):
|
||||
return ""
|
||||
s = s.replace('"', "") # 分割で片側だけ残った引用符(blue" / "pink)
|
||||
s = s.replace("\\(", "\x00").replace("\\)", "\x01") # danbooruのエスケープ括弧は残す
|
||||
for _ in range(3):
|
||||
while len(s) > 1 and s[0] in "([" and s[-1] in ")]":
|
||||
s = s[1:-1].strip()
|
||||
s = _WEIGHT.sub("", s).strip(" \"'.;")
|
||||
if s.count("(") != s.count(")") or s.count("[") != s.count("]"):
|
||||
s = re.sub(r"[()\[\]]", "", s).strip() # 分割で片側だけ残った括弧
|
||||
s = s.replace("\x00", "(").replace("\x01", ")").replace("_", " ")
|
||||
s = re.sub(r"\s+", " ", s).lower()
|
||||
if len(s) < 2 or len(s) > 40 or len(s.split()) > 5 or s in _STOP or not _HAS_WORD.search(s):
|
||||
return ""
|
||||
if s.startswith(("embedding:", "rating:")): # (embedding:xxx:1) のように括弧を外してから現れる形
|
||||
return ""
|
||||
return s
|
||||
|
||||
|
||||
def split_tags(text: str) -> list[str]:
|
||||
return [t for t in (normalize_tag(p) for p in _SPLIT.split(text or "")) if t]
|
||||
|
||||
|
||||
# C実装があれば使う(4万行のワイルドカードYAMLで 1.9秒 → 0.2秒)。どちらも任意のPythonオブジェクトは作らない
|
||||
_YAML_LOADER = getattr(yaml, "CSafeLoader", yaml.SafeLoader)
|
||||
|
||||
|
||||
def _yaml_leaves(node):
|
||||
"""YAMLを読んだ値から文字列の葉だけを取り出す(キーはカテゴリ名なので除く。
|
||||
yes/no 等が真偽値・数値に変換された葉はタグにならないので除く)"""
|
||||
if isinstance(node, dict):
|
||||
for v in node.values():
|
||||
yield from _yaml_leaves(v)
|
||||
elif isinstance(node, list):
|
||||
for v in node:
|
||||
yield from _yaml_leaves(v)
|
||||
elif isinstance(node, str):
|
||||
yield node
|
||||
|
||||
|
||||
def _wildcard_yaml_texts(text: str):
|
||||
"""ワイルドカードYAMLの項目。`- >-` 等のブロックスカラー(複数行)やエスケープもYAMLとして解釈する。
|
||||
YAMLとして読めないファイルは、従来どおり "- " で始まる行を1項目として読む"""
|
||||
try:
|
||||
yield from _yaml_leaves(yaml.load(text, Loader=_YAML_LOADER))
|
||||
except yaml.YAMLError:
|
||||
for ln in text.splitlines():
|
||||
s = ln.strip()
|
||||
if s.startswith("- "):
|
||||
yield s[2:].strip().strip("\"'")
|
||||
|
||||
|
||||
def _upstream_texts(graph: dict, node_id: str, seen: set) -> list[str]:
|
||||
"""API形式グラフで node_id から上流へたどり、プロンプト用の文字列入力を集める(negative側はたどらない)"""
|
||||
if node_id in seen or node_id not in graph:
|
||||
return []
|
||||
seen.add(node_id)
|
||||
out = []
|
||||
for key, val in ((graph[node_id] or {}).get("inputs") or {}).items():
|
||||
lk = key.lower()
|
||||
if lk in _SKIP_INPUTS:
|
||||
continue
|
||||
if isinstance(val, str):
|
||||
if lk in _TEXT_INPUTS and val.strip():
|
||||
out.append(val.strip())
|
||||
elif isinstance(val, list) and len(val) == 2 and isinstance(val[0], (str, int)):
|
||||
out += _upstream_texts(graph, str(val[0]), seen)
|
||||
return out
|
||||
|
||||
|
||||
def positive_prompt(graph: dict) -> str:
|
||||
"""API形式グラフの Positive プロンプト。サンプラー/ガイダーの positive(無ければ BasicGuider等の conditioning)から上流をたどる"""
|
||||
if not isinstance(graph, dict) or "nodes" in graph:
|
||||
return ""
|
||||
texts = []
|
||||
for node in graph.values():
|
||||
ins = (node or {}).get("inputs") if isinstance(node, dict) else None
|
||||
if not isinstance(ins, dict):
|
||||
continue
|
||||
link = ins.get("positive")
|
||||
if not isinstance(link, list) and "negative" not in ins:
|
||||
link = ins.get("conditioning") if "guider" in str(node.get("class_type", "")).lower() else None
|
||||
if isinstance(link, list) and link:
|
||||
texts += _upstream_texts(graph, str(link[0]), set())
|
||||
return ", ".join(dict.fromkeys(texts))[:_PROMPT_MAX]
|
||||
|
||||
|
||||
class TagSpaceService:
|
||||
def __init__(self, data_dir: Path, gallery_service, semantic_service, tagger_service=None):
|
||||
self.data_dir = data_dir
|
||||
self.gallery = gallery_service
|
||||
self.semantic = semantic_service
|
||||
self.tagger = tagger_service
|
||||
self._tag_file = data_dir / "tagspace_tags.npz"
|
||||
self._space_file = data_dir / "tagspace_space.json"
|
||||
self._mat_file = data_dir / "tagspace_space.npz"
|
||||
self._prompt_file = data_dir / "tagspace_prompts.json" # Gallery画像の Positive {path: [size, mtime, text]}
|
||||
self._tagger_file = data_dir / "tagspace_tagger.json" # Tagger補完 {path: [size, mtime, model, tags]}
|
||||
self._lock = threading.RLock()
|
||||
self._tag_cache: dict[str, np.ndarray] | None = None # tag -> 768次元(正規化済み)
|
||||
self._space: dict | None = None # 3D表示用データ(JSON)
|
||||
self._mat: dict | None = None # 近傍/検索用 {ids, vecs, kinds, mean_img, mean_txt, axes, scale}
|
||||
self._job = self._idle_job()
|
||||
self._cancel = threading.Event()
|
||||
|
||||
@staticmethod
|
||||
def _idle_job() -> dict:
|
||||
return {"running": False, "phase": "", "done": 0, "total": 0, "note": "", "error": "", "error_code": "",
|
||||
"finished": False, "cancelled": False}
|
||||
|
||||
# ── 状態 ───────────────────────────────────────────────
|
||||
|
||||
def status(self) -> dict:
|
||||
space = self._load_space()
|
||||
meta = (space or {}).get("meta") or {}
|
||||
return {"job": dict(self._job), "built": bool(space), "meta": meta, "tagger_models": self.tagger_models(),
|
||||
"index": self.semantic.index_info(), "ollama": ollama.status(),
|
||||
"semantic_job": self.semantic.job_status()}
|
||||
|
||||
def space(self) -> dict:
|
||||
space = self._load_space()
|
||||
if not space:
|
||||
raise SemanticError("tagspace_not_built")
|
||||
return space
|
||||
|
||||
def _load_space(self) -> dict | None:
|
||||
with self._lock:
|
||||
if self._space is None and self._space_file.exists():
|
||||
try:
|
||||
self._space = json.loads(self._space_file.read_text(encoding="utf-8"))
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load space: %s", e)
|
||||
return self._space
|
||||
|
||||
def _load_mat(self) -> dict:
|
||||
with self._lock:
|
||||
if self._mat is None:
|
||||
if not self._mat_file.exists():
|
||||
raise SemanticError("tagspace_not_built")
|
||||
with np.load(self._mat_file, allow_pickle=False) as z:
|
||||
self._mat = {k: z[k] for k in z.files}
|
||||
m = self._mat
|
||||
m["ids"] = [str(x) for x in m["ids"]]
|
||||
m["pos"] = {i: n for n, i in enumerate(m["ids"])}
|
||||
m["bg_ids"] = [str(x) for x in m["bg_ids"]] if "bg_ids" in m else []
|
||||
m["bg_pos"] = {i: n for n, i in enumerate(m["bg_ids"])}
|
||||
# 中心化して正規化したブロック(カード/タグ/背景画像)。近傍・検索のたびに作り直さない
|
||||
n_img = int(m["n_img"])
|
||||
m["blocks"] = {
|
||||
"card": (m["ids"][:n_img], self._centered(m["vecs"][:n_img], m["mean_img"])),
|
||||
"tag": (m["ids"][n_img:], self._centered(m["vecs"][n_img:], m["mean_txt"])),
|
||||
"image": (m["bg_ids"], self._centered(m["bg_vecs"], m["mean_img"]) if "bg_vecs" in m
|
||||
else np.zeros((0, m["vecs"].shape[1]), np.float32)),
|
||||
}
|
||||
return self._mat
|
||||
|
||||
# ── 材料の収集 ─────────────────────────────────────────
|
||||
|
||||
def _cards(self) -> tuple[list[dict], list]:
|
||||
"""ImagePromptの画像のうちプロンプトを持つもの。戻り値: (カード, スキャン結果)"""
|
||||
root = self.gallery.get_image_prompt_root()
|
||||
if not root:
|
||||
return [], []
|
||||
base = Path(root).resolve()
|
||||
entries = [e for e in self.gallery._scan_folder_recursive(base) if Path(e[1]).suffix.lower() in _IMG_EXTS]
|
||||
cards, used = [], []
|
||||
for name, ap, size, mtime in entries:
|
||||
prompt = self.gallery._read_sidecar_prompt(Path(ap))
|
||||
if not prompt:
|
||||
continue
|
||||
rel = Path(ap).resolve().relative_to(base).parts[:-1]
|
||||
folders = [p for p in rel if p.lower() not in _VAULT_THUMB_DIRS]
|
||||
# tags: 詳細パネルでタグを選んで空間検索するための、プロンプトを分解したタグ(Gallery点の tags と同じ正規化)
|
||||
cards.append({"id": ap, "kind": "card", "label": name.split(".")[0], "prompt": prompt.strip(),
|
||||
"tags": list(dict.fromkeys(split_tags(prompt))),
|
||||
"category": "/".join(folders[-2:]) or "(root)", "group": folders[0] if folders else "(root)"})
|
||||
used.append((name, ap, size, mtime))
|
||||
return cards, used
|
||||
|
||||
def _wildcard_texts(self):
|
||||
folder = self.data_dir / "wildcard"
|
||||
if not folder.is_dir():
|
||||
return
|
||||
for p in sorted(folder.rglob("*")):
|
||||
if p.suffix.lower() == ".txt":
|
||||
lines = p.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||
yield from (ln for ln in lines if ln.strip() and not ln.lstrip().startswith("#"))
|
||||
elif p.suffix.lower() in (".yaml", ".yml"):
|
||||
yield from _wildcard_yaml_texts(p.read_text(encoding="utf-8", errors="replace"))
|
||||
|
||||
def _tagger_texts(self):
|
||||
db = self.data_dir / "tagger.db"
|
||||
if not db.exists():
|
||||
return
|
||||
try:
|
||||
with sqlite3.connect(f"file:{db}?mode=ro", uri=True) as conn:
|
||||
for (text,) in conn.execute("SELECT all_tags FROM images"):
|
||||
if text:
|
||||
yield text
|
||||
except sqlite3.Error as e:
|
||||
logger.warning("TagSpace: tagger.db read failed: %s", e)
|
||||
|
||||
def _library_texts(self):
|
||||
f = self.data_dir / "prompts.json"
|
||||
try:
|
||||
items = json.loads(f.read_text(encoding="utf-8")) if f.exists() else []
|
||||
except Exception:
|
||||
items = []
|
||||
for it in items if isinstance(items, list) else []:
|
||||
if isinstance(it, dict) and it.get("text"):
|
||||
yield str(it["text"])
|
||||
|
||||
def tagger_models(self) -> list[str]:
|
||||
"""Tagger補完に使えるモデル(ONNX)"""
|
||||
if self.tagger is None:
|
||||
return []
|
||||
try:
|
||||
return [m["name"] for m in self.tagger.list_models() if m.get("type") == "onnx"]
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
def _tagger_fill(self, targets: list[tuple[str, int, float]], model: str) -> dict[str, str]:
|
||||
"""targets([(path, size, mtime)])を Tagger でタグ付けする。同じ画像・同じモデルならキャッシュを使う。
|
||||
中止しても、それまでの結果はキャッシュに残す。戻り値: {path: "tag, tag, ..."}"""
|
||||
from PIL import Image
|
||||
try:
|
||||
cache = json.loads(self._tagger_file.read_text(encoding="utf-8")) if self._tagger_file.exists() else {}
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load tagger cache: %s", e)
|
||||
cache = {}
|
||||
job = self._job
|
||||
job.update(phase="tagger", done=0, total=len(targets), note="")
|
||||
out: dict[str, str] = {}
|
||||
dirty = 0
|
||||
was_loaded = model in getattr(self.tagger, "_cache", {})
|
||||
|
||||
def save():
|
||||
tmp = self._tagger_file.with_suffix(".tmp.json")
|
||||
tmp.write_text(json.dumps(cache, ensure_ascii=False), encoding="utf-8")
|
||||
tmp.replace(self._tagger_file)
|
||||
|
||||
try:
|
||||
for n, (path, size, mtime) in enumerate(targets):
|
||||
if self._cancel.is_set():
|
||||
break
|
||||
hit = cache.get(path)
|
||||
if hit and hit[0] == size and abs(hit[1] - mtime) < 1e-3 and hit[2] == model:
|
||||
text = hit[3]
|
||||
else:
|
||||
try:
|
||||
with Image.open(path) as im:
|
||||
im.seek(0)
|
||||
im = im.convert("RGBA") if im.mode in ("RGBA", "LA", "P") else im.convert("RGB")
|
||||
im.thumbnail((_TAGGER_MAX_SIDE, _TAGGER_MAX_SIDE), Image.LANCZOS)
|
||||
text = ", ".join(self.tagger.tag_image(im, model))
|
||||
except RuntimeError as e: # モデルを読み込めない: 続けても全件失敗する
|
||||
raise SemanticError("tagspace_tagger_model", str(e))
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: tagger failed for %s: %s", path, e)
|
||||
text = ""
|
||||
cache[path] = [size, mtime, model, text]
|
||||
dirty += 1
|
||||
if not job["note"]:
|
||||
job["note"] = self.tagger.model_provider(model)
|
||||
if dirty >= _TAGGER_SAVE_EVERY:
|
||||
save()
|
||||
dirty = 0
|
||||
if text:
|
||||
out[path] = text
|
||||
job["done"] = n + 1
|
||||
finally:
|
||||
if dirty:
|
||||
save()
|
||||
if not was_loaded: # 補完のために読み込んだモデルはメモリから外す(Taggerタブで使用中のものは残す)
|
||||
getattr(self.tagger, "_cache", {}).pop(model, None)
|
||||
return out
|
||||
|
||||
def _gallery_prompts(self, exclude_prefix: str) -> dict[str, str]:
|
||||
"""索引済み Gallery 画像の Positive プロンプト。(size, mtime)が同じならキャッシュを使う"""
|
||||
try:
|
||||
cache = json.loads(self._prompt_file.read_text(encoding="utf-8")) if self._prompt_file.exists() else {}
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load prompt cache: %s", e)
|
||||
cache = {}
|
||||
entries = [e for e in self.semantic.indexed_entries(exclude_prefix)
|
||||
if Path(e[0]).suffix.lower() in _IMG_EXTS]
|
||||
job = self._job
|
||||
job.update(phase="prompts", done=0, total=len(entries))
|
||||
new_cache, out = {}, {}
|
||||
for n, (path, size, mtime) in enumerate(entries):
|
||||
if self._cancel.is_set():
|
||||
break
|
||||
if not Path(path).is_file():
|
||||
continue # 索引作成後に削除された画像(キャッシュにも残さない)
|
||||
hit = cache.get(path)
|
||||
if hit and hit[0] == size and abs(hit[1] - mtime) < 1e-3:
|
||||
text = hit[2]
|
||||
else:
|
||||
try:
|
||||
text = positive_prompt(self.gallery._extract_embedded_workflow(path, prefer="prompt"))
|
||||
except Exception:
|
||||
text = ""
|
||||
new_cache[path] = [size, mtime, text]
|
||||
if text:
|
||||
out[path] = text
|
||||
if n % 200 == 0:
|
||||
job["done"] = n
|
||||
job["done"] = len(entries)
|
||||
if not self._cancel.is_set(): # 中止時は途中までの結果で既存のキャッシュを上書きしない
|
||||
tmp = self._prompt_file.with_suffix(".tmp.json")
|
||||
tmp.write_text(json.dumps(new_cache, ensure_ascii=False), encoding="utf-8")
|
||||
tmp.replace(self._prompt_file)
|
||||
return out
|
||||
|
||||
def collect_tags(self, cards: list[dict], max_tags: int, gallery_texts=(), tagger_texts=(),
|
||||
min_count: int = 1, wildcard: bool = True) -> tuple[list[dict], int]:
|
||||
"""全ソースのタグを数えて上位 max_tags 件を返す(max_tags=0 は全件、ただし ALL_TAGS_LIMIT まで)。
|
||||
ソースごとに出現数の多い順に並べ、ソースを順番に1件ずつ取る(数の多いGalleryだけで埋まらないように)。
|
||||
出現数は「そのタグを含む文の数」で数え、全出典の合計が min_count 未満のタグは候補から外す。
|
||||
wildcard=False ならワイルドカードは材料にしない(ワイルドカードにしか無い語がタグの種類の大半を占めるため)。
|
||||
戻り値: (タグ, 候補数)"""
|
||||
per_src: dict[str, Counter] = {s: Counter() for s in SOURCES}
|
||||
feeds = {"image_prompt": (c["prompt"] for c in cards), "gallery": iter(gallery_texts),
|
||||
"tagger_auto": iter(tagger_texts), "tagger": self._tagger_texts(), "prompt_library": self._library_texts(),
|
||||
"wildcard": self._wildcard_texts() if wildcard else iter(())}
|
||||
for src, texts in feeds.items():
|
||||
for text in texts:
|
||||
per_src[src].update(set(split_tags(text)))
|
||||
total = Counter()
|
||||
for c in per_src.values():
|
||||
total.update(c)
|
||||
ok = {t for t, n in total.items() if n >= min_count}
|
||||
if not max_tags:
|
||||
max_tags = ALL_TAGS_LIMIT
|
||||
queues = {s: sorted((t for t in c if t in ok), key=lambda t, c=c: (-c[t], t)) for s, c in per_src.items()}
|
||||
queues = {s: q for s, q in queues.items() if q}
|
||||
picked: list[str] = []
|
||||
seen: set = set()
|
||||
pos = dict.fromkeys(queues, 0)
|
||||
while len(picked) < max_tags and queues:
|
||||
for src in list(queues):
|
||||
q, i = queues[src], pos[src]
|
||||
while i < len(q) and q[i] in seen:
|
||||
i += 1
|
||||
if i >= len(q):
|
||||
del queues[src]
|
||||
continue
|
||||
pos[src] = i + 1
|
||||
seen.add(q[i])
|
||||
picked.append(q[i])
|
||||
if len(picked) >= max_tags:
|
||||
break
|
||||
return [{"id": f"tag:{t}", "kind": "tag", "label": t,
|
||||
"count": total[t],
|
||||
"sources": [s for s in SOURCES if per_src[s][t]]} for t in picked], len(ok)
|
||||
|
||||
# ── タグベクトルのキャッシュ ───────────────────────────
|
||||
|
||||
def _tag_vectors(self) -> dict[str, np.ndarray]:
|
||||
with self._lock:
|
||||
if self._tag_cache is None:
|
||||
self._tag_cache = {}
|
||||
if self._tag_file.exists():
|
||||
try:
|
||||
with np.load(self._tag_file, allow_pickle=False) as z:
|
||||
self._tag_cache = dict(zip((str(t) for t in z["tags"]), z["vecs"]))
|
||||
except Exception as e:
|
||||
logger.warning("TagSpace: failed to load tag cache: %s", e)
|
||||
return self._tag_cache
|
||||
|
||||
def _save_tag_vectors(self) -> None:
|
||||
with self._lock:
|
||||
cache = self._tag_cache or {}
|
||||
if not cache:
|
||||
return
|
||||
tags = list(cache)
|
||||
tmp = self._tag_file.with_suffix(".tmp.npz")
|
||||
np.savez(tmp, tags=np.array(tags), vecs=np.stack([cache[t] for t in tags]).astype(np.float32))
|
||||
tmp.replace(self._tag_file)
|
||||
|
||||
# ── 構築ジョブ ─────────────────────────────────────────
|
||||
|
||||
def start_build(self, max_tags: int, bg: int, tagger_model: str = "", tagger_scope: str = "background",
|
||||
min_count: int = 1, wildcard: bool = True) -> None:
|
||||
"""max_tags=0 は全タグ(ALL_TAGS_LIMIT まで)"""
|
||||
if tagger_model and tagger_model not in self.tagger_models():
|
||||
raise SemanticError("tagspace_tagger_model", tagger_model)
|
||||
if tagger_scope not in TAGGER_SCOPES:
|
||||
tagger_scope = TAGGER_SCOPES[0]
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
info = self.semantic.index_info()
|
||||
if info["count"] and info["model"] != LOCAL_MODEL:
|
||||
raise SemanticError("tagspace_index_model", info["model"])
|
||||
st = ollama.status()
|
||||
if not st["reachable"]:
|
||||
raise SemanticError("ollama_unreachable", f"{st['url']}: {st['error']}")
|
||||
if not st["ready"]:
|
||||
raise SemanticError("ollama_model_missing", st["tag"])
|
||||
self._cancel.clear()
|
||||
self._job = self._idle_job()
|
||||
self._job.update(running=True, phase="cards")
|
||||
max_tags = 0 if int(max_tags) == 0 else max(50, min(int(max_tags), MAX_TAGS_LIMIT))
|
||||
args = (max_tags, max(0, min(int(bg), BG_LIMIT)), tagger_model, tagger_scope,
|
||||
max(1, min(int(min_count), MIN_COUNT_LIMIT)), bool(wildcard))
|
||||
threading.Thread(target=self._run_build, args=args, daemon=True).start()
|
||||
|
||||
def cancel(self) -> None:
|
||||
self._cancel.set()
|
||||
if self._job.get("phase") == "index":
|
||||
self.semantic.cancel()
|
||||
|
||||
def _run_build(self, max_tags: int, bg: int, tagger_model: str = "", tagger_scope: str = "background",
|
||||
min_count: int = 1, wildcard: bool = True) -> None:
|
||||
job = self._job
|
||||
try:
|
||||
cards, entries = self._cards()
|
||||
root = self.gallery.get_image_prompt_root() or ""
|
||||
|
||||
# 1. カード画像を Gallery 索引に追加(未索引/更新分のみ)
|
||||
if entries and not self._cancel.is_set():
|
||||
job.update(phase="index", done=0, total=0)
|
||||
info = self.semantic.index_info()
|
||||
dim = info["dim"] or ollama.DEFAULT_LOCAL_DIM
|
||||
self.semantic.start_index(root, True, "missing", LOCAL_MODEL, dim)
|
||||
while True:
|
||||
sj = self.semantic.job_status()
|
||||
job.update(done=sj["done"] + sj["failed"], total=sj["total"])
|
||||
if not sj["running"]:
|
||||
break
|
||||
time.sleep(0.5)
|
||||
if sj.get("error_code") in ("ollama_unreachable", "ollama_model_missing"):
|
||||
raise SemanticError(sj["error_code"], sj["error"])
|
||||
card_vecs = self.semantic.vectors_for(entries)
|
||||
cards = [c for c in cards if c["id"] in card_vecs]
|
||||
dim = self.semantic.index_info()["dim"] or ollama.DEFAULT_LOCAL_DIM
|
||||
|
||||
# 2. Gallery 画像の Positive プロンプト(タグの材料と、背景点のタグ)
|
||||
prefix = Path(root).resolve().as_posix().rstrip("/") + "/" if root else ""
|
||||
gallery_prompts = self._gallery_prompts(prefix)
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
|
||||
# 3. 背景の Gallery 画像(Tagger補完の対象を決めるため、タグ集めより先に選ぶ)
|
||||
bg_paths, bg_mat = self.semantic.sample_vectors(bg + bg // 4, exclude_prefix=prefix) if bg else ([], None)
|
||||
if bg_paths: # 索引作成後に削除された画像を除く(多めに取って bg 件に詰める)
|
||||
keep = [i for i, p in enumerate(bg_paths) if Path(p).is_file()][:bg]
|
||||
bg_paths, bg_mat = [bg_paths[i] for i in keep], bg_mat[keep]
|
||||
|
||||
# 4. Tagger補完: プロンプト由来のタグが無い画像
|
||||
tagger_tags: dict[str, str] = {}
|
||||
if tagger_model:
|
||||
stamps = {p: (sz, mt) for p, sz, mt in self.semantic.indexed_entries(prefix)}
|
||||
pool = bg_paths if tagger_scope == "background" else [
|
||||
p for p in stamps if Path(p).suffix.lower() in _IMG_EXTS and Path(p).is_file()]
|
||||
targets = [(p, *stamps[p]) for p in pool
|
||||
if p in stamps and Path(p).suffix.lower() in _IMG_EXTS
|
||||
and not split_tags(gallery_prompts.get(p, ""))]
|
||||
tagger_tags = self._tagger_fill(targets, tagger_model)
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
|
||||
# 5. タグを集めて未キャッシュ分を埋め込む
|
||||
job.update(phase="tags", done=0, total=0, note="")
|
||||
tags, available = self.collect_tags(cards, max_tags, gallery_prompts.values(), tagger_tags.values(),
|
||||
min_count, wildcard)
|
||||
cache = self._tag_vectors()
|
||||
todo = [t["label"] for t in tags if t["label"] not in cache]
|
||||
job.update(total=len(todo))
|
||||
for i in range(0, len(todo), _EMBED_BATCH):
|
||||
if self._cancel.is_set():
|
||||
break
|
||||
batch = todo[i:i + _EMBED_BATCH]
|
||||
try:
|
||||
vecs = ollama.embed_texts(batch, ollama.DEFAULT_LOCAL_DIM)
|
||||
except ollama.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
with self._lock:
|
||||
cache.update(zip(batch, vecs))
|
||||
job["done"] = min(len(todo), i + len(batch))
|
||||
if (i // _EMBED_BATCH) % 20 == 19:
|
||||
self._save_tag_vectors()
|
||||
self._save_tag_vectors()
|
||||
tags = [t for t in tags if t["label"] in cache]
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
|
||||
# 6. 投影
|
||||
job.update(phase="project", done=0, total=0)
|
||||
self._project(cards, card_vecs, tags, cache, dim, bg_paths, bg_mat, gallery_prompts, tagger_tags,
|
||||
tagger_model if tagger_tags else "",
|
||||
{"max_tags": max_tags, "min_count": min_count, "wildcard": wildcard, "tags_available": available})
|
||||
except SemanticError as e:
|
||||
job["error_code"], job["error"] = e.code, str(e)
|
||||
except Exception as e:
|
||||
logger.exception("TagSpace: build failed")
|
||||
job["error_code"], job["error"] = "tagspace_error", str(e)
|
||||
finally:
|
||||
ollama.release()
|
||||
job["running"] = False
|
||||
job["finished"] = True
|
||||
job["cancelled"] = self._cancel.is_set() and not job["error_code"]
|
||||
job["phase"] = ""
|
||||
|
||||
@staticmethod
|
||||
def _unit(m: np.ndarray, dim: int) -> np.ndarray:
|
||||
m = m[:, :dim].astype(np.float32)
|
||||
n = np.linalg.norm(m, axis=1, keepdims=True)
|
||||
return m / np.where(n > 0, n, 1)
|
||||
|
||||
def _project(self, cards, card_vecs, tags, cache, dim, bg_paths, bg_mat, gallery_prompts=None,
|
||||
tagger_tags=None, tagger_model: str = "", tag_meta: dict | None = None) -> None:
|
||||
img = self._unit(np.stack([card_vecs[c["id"]] for c in cards]), dim) if cards else np.zeros((0, dim), np.float32)
|
||||
txt = self._unit(np.stack([cache[t["label"]] for t in tags]), dim) if tags else np.zeros((0, dim), np.float32)
|
||||
if len(img) + len(txt) < 3:
|
||||
raise SemanticError("tagspace_too_few")
|
||||
bg_u = self._unit(bg_mat, dim) if bg_mat is not None and len(bg_mat) else np.zeros((0, dim), np.float32)
|
||||
# modality gap 対策: 画像とテキスト(タグ)をそれぞれの平均で中心化する。
|
||||
# 画像側はカードの平均を使う(数の多い背景画像の平均だとカードがタグの塊から離れて見えるため)
|
||||
img_ref = img if len(img) else bg_u
|
||||
mean_img = img_ref.mean(axis=0) if len(img_ref) else np.zeros(dim, np.float32)
|
||||
mean_txt = txt.mean(axis=0) if len(txt) else np.zeros(dim, np.float32)
|
||||
x = np.concatenate([img - mean_img, txt - mean_txt])
|
||||
_, s, vt = np.linalg.svd(x, full_matrices=False)
|
||||
axes = vt[:3].astype(np.float32)
|
||||
var = (s ** 2) / max(float((s ** 2).sum()), 1e-12)
|
||||
pts = x @ axes.T
|
||||
r98 = float(np.percentile(np.linalg.norm(pts, axis=1), 98)) or 1.0
|
||||
scale = _RADIUS / r98
|
||||
|
||||
def xyz(m):
|
||||
return [[round(float(v), 3) for v in row] for row in (m @ axes.T * scale)]
|
||||
|
||||
n_img = len(img)
|
||||
for c, p in zip(cards, xyz(img - mean_img)):
|
||||
c["xyz"] = p
|
||||
for t, p in zip(tags, xyz(txt - mean_txt)):
|
||||
t["xyz"] = p
|
||||
gp, tt = gallery_prompts or {}, tagger_tags or {}
|
||||
|
||||
def bg_item(path, pos):
|
||||
"""背景点にはその画像のタグを持たせる(クリックでまとめて追加)。プロンプト由来が無ければTagger補完"""
|
||||
tags_p = list(dict.fromkeys(split_tags(gp.get(path, ""))))
|
||||
if tags_p:
|
||||
return {"id": path, "xyz": pos, "tags": tags_p, "tag_src": "prompt"}
|
||||
tags_t = list(dict.fromkeys(split_tags(tt.get(path, ""))))
|
||||
return {"id": path, "xyz": pos, "tags": tags_t, "tag_src": "tagger" if tags_t else ""}
|
||||
|
||||
bg_items = [bg_item(p, q) for p, q in zip(bg_paths, xyz(bg_u - mean_img))] if len(bg_u) else []
|
||||
meta = {"built_at": time.strftime("%Y-%m-%d %H:%M:%S"), "cards": len(cards), "tags": len(tags),
|
||||
"background": len(bg_items), "gallery_prompts": len(gp), "tagger_images": len(tt),
|
||||
"tagger_model": tagger_model, "dim": dim, "model": LOCAL_MODEL,
|
||||
"explained": [round(float(v), 4) for v in var[:3]], **(tag_meta or {})}
|
||||
space = {"meta": meta, "cards": cards, "tags": tags, "background": bg_items}
|
||||
|
||||
ids = [c["id"] for c in cards] + [t["id"] for t in tags]
|
||||
tmp = self._mat_file.with_suffix(".tmp.npz")
|
||||
np.savez(tmp, ids=np.array(ids), vecs=np.concatenate([img, txt]).astype(np.float32),
|
||||
n_img=np.array(n_img), mean_img=mean_img.astype(np.float32), mean_txt=mean_txt.astype(np.float32),
|
||||
axes=axes, scale=np.array(scale, dtype=np.float32),
|
||||
bg_ids=np.array([b["id"] for b in bg_items]), bg_vecs=bg_u.astype(np.float32))
|
||||
tmp.replace(self._mat_file)
|
||||
tmp_json = self._space_file.with_suffix(".tmp.json")
|
||||
tmp_json.write_text(json.dumps(space, ensure_ascii=False), encoding="utf-8")
|
||||
tmp_json.replace(self._space_file)
|
||||
with self._lock:
|
||||
self._space, self._mat = space, None
|
||||
|
||||
# ── 近傍・検索 ─────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def _centered(vecs: np.ndarray, mean: np.ndarray) -> np.ndarray:
|
||||
c = vecs - mean
|
||||
return c / np.maximum(np.linalg.norm(c, axis=1, keepdims=True), 1e-6)
|
||||
|
||||
def _rank(self, m: dict, q: np.ndarray, k: int, exclude: str = "", kind: str = "") -> list[dict]:
|
||||
"""カード・タグ・画像は別々の塊なので、それぞれの平均を引いた空間で比べる(=中心化コサイン)。
|
||||
同じ種類どうしの方がスコアが高く出るため、種類ごとに順位付けして交互に並べる。
|
||||
Gallery画像(背景点)を候補に入れるのは、Gallery画像を基準にしたときだけ。"""
|
||||
qc = q - (m["mean_txt"] if kind == "tag" else m["mean_img"])
|
||||
qn = qc / max(float(np.linalg.norm(qc)), 1e-6)
|
||||
order = {"tag": ("tag", "card"), "card": ("card", "tag"), "image": ("image", "tag", "card")}[kind or "card"]
|
||||
lists = []
|
||||
for name in order:
|
||||
ids, mat = m["blocks"][name]
|
||||
if not len(ids):
|
||||
lists.append([])
|
||||
continue
|
||||
scores = mat @ qn
|
||||
top = np.argsort(-scores)[:k + 1]
|
||||
lists.append([{"id": ids[i], "score": round(float(scores[i]), 4)} for i in top if ids[i] != exclude][:k])
|
||||
out = []
|
||||
for i in range(k):
|
||||
out += [lst[i] for lst in lists if i < len(lst)]
|
||||
if len(out) >= k:
|
||||
break
|
||||
return out[:k]
|
||||
|
||||
def neighbors(self, item_id: str, k: int) -> list[dict]:
|
||||
m = self._load_mat()
|
||||
i = m["pos"].get(item_id)
|
||||
if i is not None:
|
||||
kind = "card" if i < int(m["n_img"]) else "tag"
|
||||
return self._rank(m, m["vecs"][i], k, exclude=item_id, kind=kind)
|
||||
j = m["bg_pos"].get(item_id)
|
||||
if j is not None:
|
||||
return self._rank(m, m["bg_vecs"][j], k, exclude=item_id, kind="image")
|
||||
raise SemanticError("tagspace_unknown_item", item_id)
|
||||
|
||||
def search(self, query: str, k: int) -> dict:
|
||||
query = (query or "").strip()[:500]
|
||||
if not query:
|
||||
return {"probe": None, "matches": []}
|
||||
m = self._load_mat()
|
||||
dim = m["vecs"].shape[1]
|
||||
try:
|
||||
q = ollama.embed_text(query, ollama.DEFAULT_LOCAL_DIM)
|
||||
except ollama.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
q = self._unit(q[None, :], dim)[0]
|
||||
p = (q - m["mean_txt"]) @ m["axes"].T * float(m["scale"])
|
||||
return {"probe": [round(float(v), 3) for v in p], "matches": self._rank(m, q, k, kind="tag")}
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Prompt translation service (native language -> English) via Ollama.
|
||||
|
||||
PLaMo 3 Translate (pfnet/plamo-3-translate-*-GGUF) models are driven with a raw
|
||||
PFML3 prompt on /api/generate, which is the route the model card recommends for
|
||||
Ollama. Any other Ollama model is used through /api/chat with a short
|
||||
"translate into English" system prompt, so a general LLM or another MT model
|
||||
can be selected too.
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import threading
|
||||
import urllib.request
|
||||
from collections import OrderedDict
|
||||
|
||||
from .settings_service import SettingsService
|
||||
from .url_guard import require_backend_url, urlopen
|
||||
|
||||
DEFAULTS = {
|
||||
"prompt_translate_enabled": False,
|
||||
"prompt_translate_url": "http://localhost:11434",
|
||||
"prompt_translate_model": "hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M",
|
||||
"prompt_translate_source_lang": "auto",
|
||||
"prompt_translate_auto": False, # GenerateUI Generate
|
||||
"prompt_translate_auto_lab": False, # GenerateUI Lab Run
|
||||
"prompt_translate_auto_video": False, # Video Plan Run
|
||||
"prompt_translate_unload": True,
|
||||
# "gpu" (default) or "cpu". CPU mode keeps the model resident in system RAM:
|
||||
# no VRAM use, so it never needs unloading and skips Ollama's ~4 s reload.
|
||||
"prompt_translate_device": "gpu",
|
||||
}
|
||||
|
||||
CPU_KEEP_ALIVE = -1 # stay loaded until Ollama restarts (about 2 GB RAM for 2B Q4_K_M)
|
||||
|
||||
# Suggested PLaMo 3 Translate builds (shown in Settings even before they are pulled).
|
||||
PLAMO_PRESETS = [
|
||||
"hf.co/pfnet/plamo-3-translate-0.8b-GGUF:Q4_K_M",
|
||||
"hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M",
|
||||
"hf.co/pfnet/plamo-3-translate-8b-GGUF:Q4_K_M",
|
||||
"hf.co/pfnet/plamo-3-translate-31b-GGUF:Q4_K_M",
|
||||
]
|
||||
|
||||
_PLAMO_STOP = ["<|plamo:tag|>", "<|plamo:bos|>"]
|
||||
_PLAMO_SPECIAL = re.compile(r"<\|plamo:[a-z_]+\|>")
|
||||
_LANG_RE = re.compile(r"^[a-z]{2,3}(-[A-Za-z]{2,4})?$")
|
||||
_MAX_TEXT_LEN = 4000
|
||||
_MAX_TEXTS = 200
|
||||
|
||||
_CHAT_SYSTEM = (
|
||||
"You translate text for image generation prompts into English. "
|
||||
"Output only the English translation, with no explanations, quotes or notes. "
|
||||
"Keep proper nouns, numbers and any ASCII text unchanged."
|
||||
)
|
||||
|
||||
_cache: "OrderedDict[tuple, str]" = OrderedDict()
|
||||
_CACHE_MAX = 1024
|
||||
_cache_lock = threading.Lock()
|
||||
|
||||
_settings = SettingsService()
|
||||
|
||||
|
||||
def get_config():
|
||||
data = _settings.load()
|
||||
cfg = {k: data.get(k, v) for k, v in DEFAULTS.items()}
|
||||
cfg["prompt_translate_url"] = (cfg["prompt_translate_url"] or DEFAULTS["prompt_translate_url"]).rstrip("/")
|
||||
cfg["prompt_translate_model"] = cfg["prompt_translate_model"] or DEFAULTS["prompt_translate_model"]
|
||||
return cfg
|
||||
|
||||
|
||||
def is_plamo(model: str) -> bool:
|
||||
return "plamo" in (model or "").lower()
|
||||
|
||||
|
||||
def build_pfml3(text: str, target_lang: str = "en", source_lang: str = "") -> str:
|
||||
"""Build a PFML3 translation prompt (BOS is added by llama.cpp/Ollama)."""
|
||||
src = _PLAMO_SPECIAL.sub("", text)
|
||||
src_attr = f"<|plamo:key|>lang<|plamo:val|>{source_lang}" if source_lang else ""
|
||||
return (
|
||||
"<|plamo:tag|>dataset<|plamo:msg|>translation"
|
||||
f"<|plamo:tag|>input{src_attr}<|plamo:msg|>{src}"
|
||||
f"<|plamo:tag|>output<|plamo:key|>lang<|plamo:val|>{target_lang}<|plamo:msg|>"
|
||||
)
|
||||
|
||||
|
||||
def _post_json(url: str, payload: dict, timeout: float):
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=json.dumps(payload).encode("utf-8"),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urlopen(req, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _num_predict(text: str) -> int:
|
||||
# CJK -> English roughly doubles-to-triples the token count; leave headroom.
|
||||
return max(64, min(2048, len(text) * 4))
|
||||
|
||||
|
||||
def _device_options(device):
|
||||
return {"num_gpu": 0} if device == "cpu" else {}
|
||||
|
||||
|
||||
def _translate_one(base_url, model, text, source_lang, keep_alive, device="gpu"):
|
||||
dev = _device_options(device)
|
||||
if is_plamo(model):
|
||||
data = _post_json(f"{base_url}/api/generate", {
|
||||
"model": model,
|
||||
"prompt": build_pfml3(text, "en", source_lang),
|
||||
"raw": True,
|
||||
"stream": False,
|
||||
"keep_alive": keep_alive,
|
||||
"options": {"temperature": 0, "num_predict": _num_predict(text), "stop": _PLAMO_STOP, **dev},
|
||||
}, timeout=180)
|
||||
out = data.get("response", "")
|
||||
else:
|
||||
data = _post_json(f"{base_url}/api/chat", {
|
||||
"model": model,
|
||||
"messages": [
|
||||
{"role": "system", "content": _CHAT_SYSTEM},
|
||||
{"role": "user", "content": text},
|
||||
],
|
||||
"stream": False,
|
||||
"think": False,
|
||||
"keep_alive": keep_alive,
|
||||
"options": {"temperature": 0, "num_predict": _num_predict(text), **dev},
|
||||
}, timeout=180)
|
||||
out = (data.get("message") or {}).get("content", "")
|
||||
out = _PLAMO_SPECIAL.sub("", out)
|
||||
return out.strip()
|
||||
|
||||
|
||||
def unload(base_url: str, model: str):
|
||||
try:
|
||||
_post_json(f"{base_url}/api/generate", {"model": model, "keep_alive": 0}, timeout=15)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def translate_texts(texts, model=None, source_lang=None):
|
||||
"""Translate each string in ``texts`` into English. Blocking (run in a thread).
|
||||
|
||||
Returns (translations, cached_count). Identical inputs are served from a
|
||||
bounded cache keyed by (model, source_lang, text).
|
||||
"""
|
||||
if not isinstance(texts, list) or len(texts) > _MAX_TEXTS:
|
||||
raise ValueError(f"texts must be a list of at most {_MAX_TEXTS} strings")
|
||||
cfg = get_config()
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
model = (model or cfg["prompt_translate_model"]).strip()
|
||||
src = (source_lang if source_lang is not None else cfg["prompt_translate_source_lang"]) or "auto"
|
||||
src = "" if src == "auto" else src
|
||||
if src and not _LANG_RE.match(src):
|
||||
raise ValueError("invalid source language")
|
||||
|
||||
device = "cpu" if cfg.get("prompt_translate_device") == "cpu" else "gpu"
|
||||
keep_alive = CPU_KEEP_ALIVE if device == "cpu" else "5m"
|
||||
|
||||
results = []
|
||||
cached = 0
|
||||
used_model = False
|
||||
for raw in texts:
|
||||
text = str(raw or "")[:_MAX_TEXT_LEN]
|
||||
if not text.strip():
|
||||
results.append(text)
|
||||
continue
|
||||
key = (model, src, text)
|
||||
with _cache_lock:
|
||||
hit = _cache.get(key)
|
||||
if hit is not None:
|
||||
_cache.move_to_end(key)
|
||||
if hit is not None:
|
||||
results.append(hit)
|
||||
cached += 1
|
||||
continue
|
||||
out = _translate_one(base_url, model, text, src, keep_alive, device)
|
||||
used_model = True
|
||||
with _cache_lock:
|
||||
_cache[key] = out
|
||||
_cache.move_to_end(key)
|
||||
while len(_cache) > _CACHE_MAX:
|
||||
_cache.popitem(last=False)
|
||||
results.append(out)
|
||||
|
||||
if used_model and device == "gpu" and cfg.get("prompt_translate_unload", True):
|
||||
# Free VRAM for the image model that usually runs right after.
|
||||
unload(base_url, model)
|
||||
return results, cached
|
||||
|
||||
|
||||
def warmup():
|
||||
"""Load the translation model ahead of the first translation (CPU mode).
|
||||
|
||||
In CPU mode the model is kept resident, so loading it once in the background
|
||||
when GenerateUI opens hides Ollama's ~4 s startup from the first translation.
|
||||
Returns False when there is nothing to do.
|
||||
"""
|
||||
cfg = get_config()
|
||||
if not cfg.get("prompt_translate_enabled") or cfg.get("prompt_translate_device") != "cpu":
|
||||
return False
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
model = cfg["prompt_translate_model"]
|
||||
# A request without a prompt only loads the model; the options decide CPU placement.
|
||||
_post_json(f"{base_url}/api/generate", {
|
||||
"model": model,
|
||||
"keep_alive": CPU_KEEP_ALIVE,
|
||||
"options": _device_options("cpu"),
|
||||
}, timeout=180)
|
||||
return True
|
||||
|
||||
|
||||
def list_models():
|
||||
cfg = get_config()
|
||||
base_url = require_backend_url(cfg["prompt_translate_url"])
|
||||
req = urllib.request.Request(f"{base_url}/api/tags")
|
||||
with urlopen(req, timeout=5) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
return [m.get("name") for m in data.get("models", []) if m.get("name")]
|
||||
@@ -12,6 +12,8 @@ import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from ..config import is_safe_rel
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
@@ -209,6 +211,10 @@ class VideoService:
|
||||
base_dir = folder_paths.get_directory_by_type(type_)
|
||||
if not base_dir:
|
||||
raise ValueError(f"Invalid type: {type_}")
|
||||
# UNC・ドライブ指定・絶対パス・「..」は、結合・resolve() の前に文字列のまま拒否する
|
||||
# (Windows では UNC を resolve() するだけで相手ホストへ資格情報付きで接続するため)
|
||||
if not is_safe_rel(filename) or (subfolder and not is_safe_rel(subfolder)):
|
||||
raise ValueError("Access denied: path outside allowed directory")
|
||||
base = Path(base_dir).resolve()
|
||||
target = (base / (subfolder or "") / filename).resolve()
|
||||
try:
|
||||
@@ -237,6 +243,68 @@ class VideoService:
|
||||
save_path.write_bytes(image_bytes)
|
||||
return {"filename": save_name, "subfolder": subfolder}
|
||||
|
||||
@staticmethod
|
||||
def _unique_output_path(stem: str, ext: str) -> tuple[Path, str]:
|
||||
"""output/video/ 配下で未使用のパスを決める。戻り値は (絶対パス, subfolder)。"""
|
||||
import folder_paths # type: ignore
|
||||
|
||||
subfolder = "video"
|
||||
out_dir = Path(folder_paths.get_output_directory()) / subfolder
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
dst = out_dir / f"{stem}{ext}"
|
||||
counter = 1
|
||||
while dst.exists():
|
||||
dst = out_dir / f"{stem}_{counter}{ext}"
|
||||
counter += 1
|
||||
return dst, subfolder
|
||||
|
||||
def register_asset(self, filename: str, subfolder: str, type_: str) -> dict:
|
||||
"""Video Sourceの動画をGalleryに載せるため、outputフォルダ配下の実体にする。
|
||||
既にoutput配下ならそのまま、input/tempならoutput/videoへコピーする。"""
|
||||
import shutil
|
||||
|
||||
src = self._resolve_media_path(filename, subfolder, type_)
|
||||
if type_ == "output":
|
||||
return {"filename": src.name, "subfolder": subfolder or "", "path": str(src).replace("\\", "/")}
|
||||
dst, out_sub = self._unique_output_path(src.stem, src.suffix)
|
||||
shutil.copy2(src, dst)
|
||||
return {"filename": dst.name, "subfolder": out_sub, "path": str(dst).replace("\\", "/")}
|
||||
|
||||
def convert_audio_to_output(self, filename: str, subfolder: str, type_: str, opts: dict) -> dict:
|
||||
"""動画/音声クリップの音声を指定形式へ変換し、output/videoへ保存する。"""
|
||||
from .convert_service import OUTPUT_FORMATS, convert_audio, supports_tags
|
||||
|
||||
fmt = str(opts.get("format", ""))
|
||||
if fmt not in OUTPUT_FORMATS:
|
||||
raise ValueError(f"Unsupported format: {fmt}")
|
||||
|
||||
def _int(key: str) -> int:
|
||||
try:
|
||||
return max(0, int(opts.get(key) or 0))
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
src = self._resolve_media_path(filename, subfolder, type_)
|
||||
inherit = bool(opts.get("inherit_metadata", True))
|
||||
dst, out_sub = self._unique_output_path(src.stem, OUTPUT_FORMATS[fmt]["ext"])
|
||||
try:
|
||||
convert_audio(
|
||||
src, dst, fmt,
|
||||
bitrate_kbps=_int("bitrate"),
|
||||
sample_rate=_int("sample_rate"),
|
||||
channels=str(opts.get("channels", "keep")),
|
||||
inherit_metadata=inherit,
|
||||
)
|
||||
except Exception:
|
||||
dst.unlink(missing_ok=True)
|
||||
raise
|
||||
return {
|
||||
"filename": dst.name,
|
||||
"subfolder": out_sub,
|
||||
"path": str(dst).replace("\\", "/"),
|
||||
"tags_skipped": inherit and not supports_tags(fmt),
|
||||
}
|
||||
|
||||
def probe_video(self, filename: str, subfolder: str, type_: str) -> dict:
|
||||
"""Reads duration/dimensions/fps/audio presence via PyAV only (no ffmpeg
|
||||
subprocess) — used by the Video Edit tab right after a clip is added, so
|
||||
|
||||
@@ -83,10 +83,11 @@ class WildcardService:
|
||||
return None
|
||||
# Normalize to forward slashes and strip leading/trailing slashes
|
||||
filename = filename.replace("\\", "/").strip("/")
|
||||
# Validate each path component — block traversal and empty parts
|
||||
# Validate each path component — block traversal, empty parts and drive-relative
|
||||
# names ("C:name"; on Python <=3.11 joinpath would replace the base path with them)
|
||||
parts = filename.split("/")
|
||||
for p in parts:
|
||||
if not p or p == ".." or p == ".":
|
||||
if not p or p == ".." or p == "." or ":" in p:
|
||||
return None
|
||||
ext = Path(parts[-1]).suffix.lstrip(".").lower()
|
||||
if ext not in ALLOWED_EXTS:
|
||||
|
||||
@@ -99,7 +99,7 @@ class WorkflowStudio:
|
||||
app.router.add_get("/wfm", serve_index_page)
|
||||
|
||||
# API routes
|
||||
from .routes import workflow_routes, settings_routes, ollama_routes, unsloth_routes, eagle_routes, nodes_routes, prompts_routes, models_routes, gallery_routes, wildcard_routes, tagger_routes, gmic_routes, skill_routes, lab_routes, video_routes, vram_routes, partner_routes
|
||||
from .routes import workflow_routes, settings_routes, ollama_routes, unsloth_routes, eagle_routes, nodes_routes, prompts_routes, models_routes, gallery_routes, wildcard_routes, tagger_routes, gmic_routes, skill_routes, lab_routes, video_routes, vram_routes, partner_routes, tagspace_routes, translate_routes
|
||||
|
||||
workflow_routes.setup_routes(app)
|
||||
settings_routes.setup_routes(app)
|
||||
@@ -118,5 +118,7 @@ class WorkflowStudio:
|
||||
lab_routes.setup_routes(app)
|
||||
video_routes.setup_routes(app)
|
||||
vram_routes.setup_routes(app)
|
||||
tagspace_routes.setup_routes(app)
|
||||
translate_routes.setup_routes(app)
|
||||
|
||||
logger.info("Workflow Studio: Routes registered successfully")
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "comfyui-workflow-studio"
|
||||
description = "A comprehensive workflow, asset management, and generation UI plugin for ComfyUI. Manage workflows, models, images, and prompts; batch generation and Image Feeder in the GenerateUI tab; library side panel with canvas drag & drop and AI tools."
|
||||
version = "0.8.8"
|
||||
version = "0.9.9"
|
||||
license = { file = "LICENSE" }
|
||||
requires-python = ">= 3.10"
|
||||
dependencies = ["jinja2"]
|
||||
|
||||
+3
-9
@@ -34,12 +34,6 @@ psd-tools>=1.10
|
||||
# (Gallery itself keeps working). Not pinned here on purpose — see torch note above.
|
||||
# av>=17.0.0
|
||||
|
||||
# -- Gallery Semantic Search: local EmbeddingGemma 2 (optional) --
|
||||
# Uses onnxruntime (above) + tokenizers + huggingface_hub (both normally pre-installed with ComfyUI's
|
||||
# transformers). The model (~300 MB, q4) is downloaded to the data folder on first use.
|
||||
# GPU (NVIDIA/CUDA): replace onnxruntime with onnxruntime-gpu
|
||||
# pip uninstall onnxruntime onnxruntime-gpu
|
||||
# pip install onnxruntime-gpu
|
||||
# Local EmbeddingGemma 2 then indexes ~45x faster than on the CPU. DirectML is not supported.
|
||||
# tokenizers
|
||||
# huggingface_hub
|
||||
# -- Gallery Semantic Search: EmbeddingGemma 2 (optional) --
|
||||
# Runs in Ollama (no extra Python packages). Install Ollama and download the model:
|
||||
# ollama pull embeddinggemma-2:740m (or embeddinggemma-2:440m - same vectors, no audio encoder)
|
||||
|
||||
@@ -1151,6 +1151,88 @@
|
||||
.wfm-sem-pane { display: flex; flex-direction: column; gap: 10px; min-height: 230px; }
|
||||
.wfm-sem-footer { justify-content: flex-end; border-top: 1px solid var(--wfm-border, #444); padding-top: 10px; }
|
||||
.wfm-sem-num { width: 80px; }
|
||||
.wfm-sem-url { width: 240px; max-width: 100%; }
|
||||
.wfm-sem-danger { margin-left: auto; color: #ef4444; }
|
||||
|
||||
.wfm-sem-row .wfm-select { width: auto; min-width: 90px; }
|
||||
|
||||
/* 3Dモデルビューア */
|
||||
.wfm-gallery-thumb-video-badge.wfm-gallery-thumb-video-badge--corner {
|
||||
top: auto;
|
||||
left: auto;
|
||||
right: 4px;
|
||||
bottom: 4px;
|
||||
width: auto;
|
||||
height: auto;
|
||||
padding: 1px 5px;
|
||||
border-radius: 3px;
|
||||
font-size: 10px;
|
||||
line-height: 14px;
|
||||
transform: none;
|
||||
}
|
||||
.wfm-g3d-box {
|
||||
position: relative;
|
||||
background: #181c26;
|
||||
border-radius: 4px;
|
||||
overflow: hidden;
|
||||
}
|
||||
.wfm-gallery-detail-img.wfm-g3d-box {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
}
|
||||
.wfm-gallery-lightbox-img.wfm-g3d-box {
|
||||
width: 90vw;
|
||||
height: 85vh;
|
||||
}
|
||||
.wfm-lightbox-compare-img.wfm-g3d-box {
|
||||
width: calc(92vw / var(--compare-cols, 2) - 12px);
|
||||
height: 78vh;
|
||||
}
|
||||
.wfm-g3d-canvas {
|
||||
display: block;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
outline: none;
|
||||
}
|
||||
.wfm-g3d-status {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 8px;
|
||||
text-align: center;
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 12px;
|
||||
pointer-events: none;
|
||||
}
|
||||
.wfm-g3d-status--error {
|
||||
color: #f87171;
|
||||
}
|
||||
.wfm-g3d-stats {
|
||||
position: absolute;
|
||||
left: 6px;
|
||||
bottom: 4px;
|
||||
color: #cbd5e1;
|
||||
font-size: 10px;
|
||||
text-shadow: 0 0 3px #000;
|
||||
pointer-events: none;
|
||||
}
|
||||
.wfm-g3d-reset {
|
||||
position: absolute;
|
||||
top: 4px;
|
||||
right: 4px;
|
||||
width: 22px;
|
||||
height: 22px;
|
||||
padding: 0;
|
||||
border: none;
|
||||
border-radius: 4px;
|
||||
background: rgba(0, 0, 0, 0.5);
|
||||
color: #e2e8f0;
|
||||
font-size: 13px;
|
||||
line-height: 22px;
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-g3d-reset:hover {
|
||||
background: rgba(0, 0, 0, 0.75);
|
||||
}
|
||||
|
||||
@@ -466,3 +466,103 @@
|
||||
background: var(--wfm-border);
|
||||
border-radius: 2px;
|
||||
}
|
||||
|
||||
/* Draw tool palette */
|
||||
.ie-palette-content {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex-wrap: wrap;
|
||||
gap: 6px 14px;
|
||||
}
|
||||
|
||||
.ie-palette-wells {
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.ie-palette-well,
|
||||
.ie-palette-swatch {
|
||||
padding: 0;
|
||||
border-radius: 50%;
|
||||
background: var(--swatch, transparent);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.ie-palette-well {
|
||||
width: 26px;
|
||||
height: 26px;
|
||||
border: 1px solid var(--wfm-text-secondary);
|
||||
}
|
||||
|
||||
.ie-palette-well.is-active {
|
||||
outline: 2px solid var(--wfm-primary);
|
||||
outline-offset: 2px;
|
||||
}
|
||||
|
||||
.ie-palette-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(10, 16px);
|
||||
grid-auto-rows: 16px;
|
||||
gap: 2px;
|
||||
}
|
||||
|
||||
.ie-palette-swatch {
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border: 1px solid rgba(128, 128, 128, 0.7);
|
||||
}
|
||||
|
||||
.ie-palette-swatch.is-empty {
|
||||
border-color: var(--wfm-text-secondary);
|
||||
opacity: 0.6;
|
||||
}
|
||||
|
||||
.ie-palette-swatch:hover,
|
||||
.ie-palette-swatch:focus-visible,
|
||||
.ie-palette-well:focus-visible {
|
||||
outline: 1px solid var(--wfm-text);
|
||||
outline-offset: 1px;
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.ie-palette-swatch.is-current {
|
||||
outline: 2px solid var(--wfm-primary);
|
||||
outline-offset: 0;
|
||||
}
|
||||
|
||||
.ie-palette-picker {
|
||||
position: fixed;
|
||||
width: 1px;
|
||||
height: 1px;
|
||||
padding: 0;
|
||||
border: 0;
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.ie-palette-menu {
|
||||
position: fixed;
|
||||
z-index: 1000;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-width: 150px;
|
||||
padding: 4px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
border-radius: 4px;
|
||||
background: var(--wfm-surface);
|
||||
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.4);
|
||||
}
|
||||
|
||||
.ie-palette-menu button {
|
||||
padding: 5px 10px;
|
||||
border: 0;
|
||||
border-radius: 3px;
|
||||
background: transparent;
|
||||
color: var(--wfm-text);
|
||||
font-size: 12px;
|
||||
text-align: left;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.ie-palette-menu button:hover {
|
||||
background: var(--wfm-surface-hover);
|
||||
}
|
||||
|
||||
@@ -984,6 +984,103 @@ body {
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
/* Input tab: Prompt (3) | Image (2) side by side. Wraps Image under Prompt when the column is narrow. */
|
||||
.wfm-gen-params-col--input {
|
||||
container-type: inline-size;
|
||||
container-name: wfm-input;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.wfm-input-split {
|
||||
display: flex;
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
.wfm-input-inner-panel--prompt {
|
||||
flex: 3 1 0;
|
||||
min-width: 0;
|
||||
border-right: 1px solid var(--wfm-border);
|
||||
}
|
||||
|
||||
.wfm-input-inner-panel--image {
|
||||
flex: 2 1 0;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.wfm-input-panel-title {
|
||||
font-size: 11px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
color: var(--wfm-text-secondary);
|
||||
margin-bottom: 6px;
|
||||
}
|
||||
|
||||
@container wfm-input (max-width: 720px) {
|
||||
.wfm-input-split {
|
||||
flex-direction: column;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.wfm-input-inner-panel--prompt,
|
||||
.wfm-input-inner-panel--image {
|
||||
flex: none;
|
||||
overflow-y: visible;
|
||||
border-right: none;
|
||||
}
|
||||
.wfm-input-inner-panel--prompt {
|
||||
border-bottom: 1px solid var(--wfm-border);
|
||||
}
|
||||
}
|
||||
|
||||
/* Input tab Raw JSON: collapsible to a slim strip at the right edge */
|
||||
.wfm-rawjson-collapse-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
flex-shrink: 0;
|
||||
padding: 3px 10px;
|
||||
background: var(--wfm-bg-secondary);
|
||||
border: none;
|
||||
border-bottom: 1px solid var(--wfm-border);
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 11px;
|
||||
cursor: pointer;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.wfm-rawjson-collapse-btn:hover {
|
||||
color: var(--wfm-text);
|
||||
}
|
||||
|
||||
.wfm-rawjson-collapse-label {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col.wfm-gen-rawjson-col--collapsed {
|
||||
width: 30px;
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col--collapsed .wfm-rawjson-collapse-btn {
|
||||
flex: 1;
|
||||
flex-direction: column;
|
||||
justify-content: flex-start;
|
||||
padding: 8px 0;
|
||||
border-bottom: none;
|
||||
border-left: 1px solid var(--wfm-border);
|
||||
background: var(--wfm-bg-secondary);
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col--collapsed .wfm-rawjson-collapse-label {
|
||||
display: block;
|
||||
writing-mode: vertical-rl;
|
||||
letter-spacing: 1px;
|
||||
}
|
||||
|
||||
.wfm-gen-rawjson-col--collapsed #wfm-gen-rawjson-widget {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
/* Right sub-column: Raw JSON */
|
||||
.wfm-gen-rawjson-col {
|
||||
width: 540px;
|
||||
@@ -5893,6 +5990,90 @@ body {
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* TOOLS: result actions, OCR options and text-position overlay (ocr-tools.js) */
|
||||
.wfm-ai-vlm-result-actions {
|
||||
display: flex;
|
||||
justify-content: flex-end;
|
||||
gap: 6px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.wfm-ai-ocr-options {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.wfm-ai-ocr-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
}
|
||||
.wfm-ai-ocr-row .wfm-ai-label { flex-shrink: 0; margin: 0; }
|
||||
.wfm-ai-ocr-row .wfm-select { flex: 1; min-width: 0; }
|
||||
.wfm-ai-ocr-check {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
font-size: 12px;
|
||||
color: var(--wfm-text-secondary);
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-ai-ocr-check.is-disabled { opacity: 0.45; cursor: default; }
|
||||
.wfm-ai-ocr-hint { font-size: 11px; color: var(--wfm-text-secondary); opacity: 0.85; }
|
||||
/* OCR: a taller preview so the numbered boxes are usable */
|
||||
.wfm-ai-vlm-drop.is-ocr { height: 260px; }
|
||||
.wfm-ai-ocr-overlay {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
pointer-events: none;
|
||||
}
|
||||
.wfm-ai-ocr-box {
|
||||
position: absolute;
|
||||
border: 1.5px solid #4fd8ff;
|
||||
background: rgba(79, 216, 255, 0.12);
|
||||
box-sizing: border-box;
|
||||
pointer-events: auto;
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-ai-ocr-box.is-active { border-color: #ff9f43; background: rgba(255, 159, 67, 0.2); }
|
||||
.wfm-ai-ocr-box-tag {
|
||||
position: absolute;
|
||||
top: -1px;
|
||||
left: -1px;
|
||||
padding: 0 3px;
|
||||
font-size: 9px;
|
||||
line-height: 12px;
|
||||
color: #000;
|
||||
background: #4fd8ff;
|
||||
}
|
||||
|
||||
/* Collapsible Translation / Settings panes */
|
||||
.wfm-ai-pane-collapse-btn {
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 10px;
|
||||
line-height: 1;
|
||||
padding: 2px 4px;
|
||||
cursor: pointer;
|
||||
border-radius: 3px;
|
||||
}
|
||||
.wfm-ai-pane-collapse-btn:hover { background: var(--wfm-bg); color: var(--wfm-text); }
|
||||
.wfm-ai-pane.is-collapsed {
|
||||
flex: 0 0 28px;
|
||||
cursor: pointer;
|
||||
}
|
||||
.wfm-ai-pane.is-collapsed > :not(.wfm-ai-pane-header) { display: none !important; }
|
||||
.wfm-ai-pane.is-collapsed .wfm-ai-pane-header {
|
||||
writing-mode: vertical-rl;
|
||||
flex: 1;
|
||||
justify-content: flex-start;
|
||||
gap: 8px;
|
||||
padding: 8px 0;
|
||||
border-bottom: none;
|
||||
}
|
||||
.wfm-ai-pane.is-collapsed .wfm-ai-pane-collapse-btn { writing-mode: horizontal-tb; }
|
||||
|
||||
/* Chat pane */
|
||||
.wfm-ai-chat-container {
|
||||
display: flex;
|
||||
@@ -5953,6 +6134,118 @@ body {
|
||||
object-fit: cover;
|
||||
}
|
||||
|
||||
/* Markdown-rendered assistant replies (chat-markdown.js) */
|
||||
.wfm-ai-chat-msg.wfm-ai-chat-md { white-space: normal; }
|
||||
.wfm-ai-chat-md p { margin: 0 0 6px; }
|
||||
.wfm-ai-chat-md p:last-child { margin-bottom: 0; }
|
||||
.wfm-ai-chat-md ul, .wfm-ai-chat-md ol { margin: 2px 0 6px; padding-left: 20px; }
|
||||
.wfm-ai-chat-md li { margin: 1px 0; }
|
||||
.wfm-ai-chat-md .wfm-md-h { margin: 8px 0 4px; font-size: 13px; font-weight: 700; }
|
||||
.wfm-ai-chat-md h3.wfm-md-h { font-size: 14px; }
|
||||
.wfm-ai-chat-md code {
|
||||
font-family: Consolas, "Courier New", monospace;
|
||||
font-size: 12px;
|
||||
background: rgba(127, 127, 127, 0.18);
|
||||
padding: 0 3px;
|
||||
border-radius: 3px;
|
||||
}
|
||||
.wfm-ai-chat-md .wfm-md-pre {
|
||||
position: relative;
|
||||
margin: 4px 0 6px;
|
||||
padding: 8px 10px;
|
||||
background: rgba(0, 0, 0, 0.25);
|
||||
border: 1px solid var(--wfm-border);
|
||||
border-radius: 4px;
|
||||
overflow-x: auto;
|
||||
white-space: pre;
|
||||
}
|
||||
.wfm-ai-chat-md .wfm-md-pre code { background: none; padding: 0; }
|
||||
.wfm-ai-chat-md .wfm-md-code-lang {
|
||||
position: absolute;
|
||||
top: 2px;
|
||||
right: 6px;
|
||||
font-size: 10px;
|
||||
opacity: 0.6;
|
||||
}
|
||||
.wfm-ai-chat-md blockquote {
|
||||
margin: 4px 0;
|
||||
padding-left: 8px;
|
||||
border-left: 3px solid var(--wfm-border);
|
||||
opacity: 0.9;
|
||||
}
|
||||
.wfm-ai-chat-md hr { border: none; border-top: 1px solid var(--wfm-border); margin: 6px 0; }
|
||||
|
||||
/* Skill auto-check notices (metadata.output) */
|
||||
.wfm-ai-chat-msg-notice {
|
||||
align-self: center;
|
||||
max-width: 95%;
|
||||
background: transparent;
|
||||
border: 1px dashed var(--wfm-border);
|
||||
color: var(--wfm-text-secondary, var(--wfm-text));
|
||||
font-size: 11px;
|
||||
opacity: 0.85;
|
||||
}
|
||||
|
||||
/* Action buttons under a ```prompt block */
|
||||
.wfm-ai-chat-msg-actions {
|
||||
margin-top: 6px;
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 6px;
|
||||
white-space: normal;
|
||||
}
|
||||
|
||||
/* Image check result card (chat-image-check.js) */
|
||||
.wfm-ai-chat-msg-check {
|
||||
white-space: normal;
|
||||
min-width: 260px;
|
||||
}
|
||||
.wfm-ai-check-head {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: baseline;
|
||||
gap: 8px;
|
||||
font-weight: 600;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.wfm-ai-check-head.is-pass { color: #4caf50; }
|
||||
.wfm-ai-check-head.is-fail { color: #e5a00d; }
|
||||
.wfm-ai-check-meta {
|
||||
font-weight: normal;
|
||||
font-size: 11px;
|
||||
opacity: 0.7;
|
||||
color: var(--wfm-text-secondary);
|
||||
}
|
||||
.wfm-ai-check-list {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
.wfm-ai-check-item {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
align-items: baseline;
|
||||
padding: 1px 0;
|
||||
font-size: 12px;
|
||||
}
|
||||
.wfm-ai-check-icon { width: 14px; text-align: center; font-weight: 700; }
|
||||
.wfm-ai-check-item.is-pass .wfm-ai-check-icon { color: #4caf50; }
|
||||
.wfm-ai-check-item.is-uncertain .wfm-ai-check-icon { color: #e5a00d; }
|
||||
.wfm-ai-check-item.is-fail .wfm-ai-check-icon { color: #e05252; }
|
||||
.wfm-ai-check-label { flex: 1; }
|
||||
.wfm-ai-check-p { font-size: 11px; opacity: 0.7; font-variant-numeric: tabular-nums; }
|
||||
.wfm-ai-check-quality { margin-top: 4px; font-size: 11px; opacity: 0.8; }
|
||||
|
||||
/* "Built-in" badge in the skill manager list */
|
||||
.wfm-ai-skill-badge {
|
||||
margin-left: 6px;
|
||||
padding: 0 5px;
|
||||
border-radius: 3px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
font-size: 10px;
|
||||
opacity: 0.75;
|
||||
}
|
||||
|
||||
.wfm-ai-chat-svg-wrapper {
|
||||
margin-top: 6px;
|
||||
display: flex;
|
||||
@@ -6158,3 +6451,48 @@ body {
|
||||
max-width: none;
|
||||
}
|
||||
.wfm-compat-row .wfm-format-badge:empty { display: none; }
|
||||
|
||||
/* GenerateUI Settings tab: Switches column (true/false inputs) */
|
||||
.wfm-settings-switches {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
max-height: 420px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.wfm-settings-switch {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 4px 6px;
|
||||
border: 1px solid var(--wfm-border);
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
}
|
||||
.wfm-settings-switch:hover {
|
||||
background: var(--wfm-surface);
|
||||
}
|
||||
.wfm-settings-switch-text {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
.wfm-settings-switch-node {
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.wfm-settings-switch-key {
|
||||
color: var(--wfm-text-secondary);
|
||||
font-size: 11px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.wfm-settings-switch-state {
|
||||
font-family: monospace;
|
||||
font-size: 11px;
|
||||
color: var(--wfm-text-secondary);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
/* TagSpace タブ */
|
||||
#wfm-tab-tagspace.active { overflow: hidden; }
|
||||
.wfm-ts { display: flex; flex-direction: column; height: 100%; min-height: 0; }
|
||||
.wfm-ts-toolbar { display: flex; align-items: center; gap: 10px; flex-wrap: wrap; padding: 8px 12px; border-bottom: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-ts-num { display: inline-flex; align-items: center; gap: 4px; font-size: 12px; color: var(--wfm-text-secondary, #aaa); }
|
||||
.wfm-ts-num input { width: 72px; }
|
||||
.wfm-ts-sep { width: 1px; height: 18px; background: var(--wfm-border, #444); }
|
||||
.wfm-ts-check { display: inline-flex; align-items: center; gap: 4px; font-size: 12px; cursor: pointer; }
|
||||
.wfm-ts-swatch { display: inline-block; width: 9px; height: 9px; border-radius: 50%; margin-right: 3px; }
|
||||
.wfm-ts-legend { display: inline-flex; gap: 10px; font-size: 11px; color: var(--wfm-text-secondary, #aaa); }
|
||||
.wfm-ts-search { margin-left: auto; width: 240px; max-width: 100%; }
|
||||
.wfm-ts-status { padding: 4px 12px; font-size: 12px; color: var(--wfm-text-secondary, #aaa); min-height: 22px; }
|
||||
.wfm-ts-status-error { color: var(--wfm-danger, #ef4444); }
|
||||
.wfm-ts-main { flex: 1; display: flex; min-height: 0; }
|
||||
.wfm-ts-view { position: relative; flex: 1; min-width: 0; overflow: hidden; background: var(--wfm-bg, #111); }
|
||||
.wfm-ts-view canvas { display: block; width: 100%; height: 100%; }
|
||||
.wfm-ts-labels { position: absolute; inset: 0; pointer-events: none; overflow: hidden; }
|
||||
.wfm-ts-label { position: absolute; left: 0; top: 0; margin: -22px 0 0 6px; padding: 1px 6px; border-radius: 4px; font-size: 11px; white-space: nowrap; background: rgba(0, 0, 0, 0.6); color: #fff; }
|
||||
.wfm-ts-label-chip { background: var(--wfm-primary, #6366f1); }
|
||||
.wfm-ts-label-near { background: rgba(8, 145, 178, 0.85); }
|
||||
.wfm-ts-label-hover { background: rgba(255, 255, 255, 0.92); color: #111; }
|
||||
.wfm-ts-tooltip { position: absolute; z-index: 2; display: flex; gap: 8px; max-width: 360px; padding: 6px 8px; border-radius: 6px; font-size: 12px; pointer-events: none; background: var(--wfm-surface, #222); border: 1px solid var(--wfm-border, #444); box-shadow: 0 4px 16px rgba(0, 0, 0, 0.4); }
|
||||
.wfm-ts-tooltip img { width: 64px; height: 64px; object-fit: cover; border-radius: 4px; flex: none; }
|
||||
.wfm-ts-tip-body { margin-top: 2px; color: var(--wfm-text-secondary, #aaa); }
|
||||
.wfm-ts-empty { position: absolute; inset: 0; display: flex; align-items: center; justify-content: center; padding: 24px; text-align: center; color: var(--wfm-text-secondary, #aaa); pointer-events: none; }
|
||||
.wfm-ts-hint { position: absolute; left: 10px; bottom: 8px; font-size: 11px; color: var(--wfm-text-secondary, #aaa); opacity: 0.8; pointer-events: none; }
|
||||
.wfm-ts-side { width: 300px; flex: none; display: flex; flex-direction: column; gap: 8px; padding: 8px; overflow: hidden; border-left: 1px solid var(--wfm-border, #444); }
|
||||
/* プロンプト列(右端): チップは中身の高さ(最大40%)、出力欄が残りの高さいっぱいに伸びる */
|
||||
.wfm-ts-side-prompt .wfm-ts-chips { flex: 0 1 auto; max-height: 40%; align-content: flex-start; }
|
||||
.wfm-ts-side-prompt .wfm-ts-output { flex: 1; min-height: 120px; resize: none; }
|
||||
.wfm-ts-panel { display: flex; flex-direction: column; gap: 6px; padding: 8px; border: 1px solid var(--wfm-border, #444); border-radius: var(--wfm-radius, 6px); background: var(--wfm-surface, transparent); min-height: 0; }
|
||||
.wfm-ts-panel h4 { margin: 0; font-size: 12px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.wfm-ts-panel-grow { flex: 1; }
|
||||
.wfm-ts-muted { color: var(--wfm-text-secondary, #aaa); font-size: 11px; }
|
||||
.wfm-ts-detail { display: flex; flex-direction: column; gap: 4px; font-size: 12px; max-height: 260px; overflow: auto; }
|
||||
.wfm-ts-detail-head { position: sticky; top: 0; z-index: 1; display: flex; align-items: center; gap: 6px; padding-bottom: 2px; background: var(--wfm-surface, #222); }
|
||||
.wfm-ts-detail-title { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.wfm-ts-detail-head .wfm-btn { flex: none; }
|
||||
.wfm-ts-detail-thumb { max-width: 100%; max-height: 140px; object-fit: contain; align-self: flex-start; border-radius: 4px; }
|
||||
.wfm-ts-detail-text { white-space: pre-wrap; word-break: break-word; }
|
||||
.wfm-ts-pick-list { display: flex; flex-wrap: wrap; gap: 3px; }
|
||||
.wfm-ts-full summary { cursor: pointer; color: var(--wfm-text-secondary, #999); font-size: 11px; }
|
||||
.wfm-ts-pick { padding: 1px 7px; border-radius: 10px; font-size: 11px; cursor: pointer; color: inherit; background: var(--wfm-bg-secondary, #333); border: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-ts-pick:hover { border-color: #f472b6; }
|
||||
.wfm-ts-pick.on { background: #f472b6; border-color: #f472b6; color: #1a1a1a; }
|
||||
/* タグが多くても検索ボタンが見えるよう、詳細パネルの下端に固定する */
|
||||
.wfm-ts-pick-actions { position: sticky; bottom: 0; order: 10; display: flex; gap: 6px; padding-top: 4px; background: var(--wfm-surface, #222); }
|
||||
.wfm-ts-path { word-break: break-all; }
|
||||
.wfm-ts-list { flex: 1; min-height: 0; overflow: auto; display: flex; flex-direction: column; gap: 2px; }
|
||||
.wfm-ts-row { display: flex; align-items: center; gap: 6px; padding: 2px 4px; border-radius: 4px; font-size: 12px; cursor: pointer; }
|
||||
.wfm-ts-row:hover { background: var(--wfm-surface-hover, rgba(255, 255, 255, 0.06)); }
|
||||
.wfm-ts-row img { width: 28px; height: 28px; object-fit: cover; border-radius: 3px; flex: none; }
|
||||
.wfm-ts-dot { width: 10px; height: 10px; border-radius: 50%; flex: none; margin: 0 9px; }
|
||||
.wfm-ts-row-label { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.wfm-ts-row-add { flex: none; width: 22px; height: 22px; border-radius: 4px; border: 1px solid var(--wfm-border, #444); background: transparent; color: var(--wfm-text, #ddd); cursor: pointer; }
|
||||
.wfm-ts-row-add.on { background: var(--wfm-primary, #6366f1); color: #fff; border-color: transparent; }
|
||||
.wfm-ts-row-add-none { flex: none; width: 22px; }
|
||||
.wfm-ts-row img.wfm-ts-row-image { border-radius: 50%; }
|
||||
.wfm-ts-chips { display: flex; flex-wrap: wrap; gap: 4px; max-height: 110px; overflow: auto; }
|
||||
.wfm-ts-chip { display: inline-flex; align-items: center; gap: 2px; padding: 1px 2px 1px 7px; border-radius: 10px; font-size: 11px; background: var(--wfm-bg-secondary, #333); border: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-ts-chip-card { border-color: var(--wfm-primary, #6366f1); }
|
||||
.wfm-ts-chip-image { border-color: #f472b6; }
|
||||
.wfm-ts-chip button { border: none; background: transparent; color: inherit; cursor: pointer; padding: 0 4px; font-size: 12px; }
|
||||
.wfm-ts-output { width: 100%; resize: vertical; font-size: 12px; box-sizing: border-box; }
|
||||
.wfm-ts-actions { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||
.wfm-ts-zoom { position: fixed; z-index: 1000; width: 360px; height: 360px; display: flex; align-items: center; justify-content: center; padding: 4px; box-sizing: border-box; pointer-events: none; border-radius: 8px; background: var(--wfm-surface, #222); border: 1px solid var(--wfm-border, #444); box-shadow: 0 8px 28px rgba(0, 0, 0, 0.55); }
|
||||
.wfm-ts-zoom img { max-width: 100%; max-height: 100%; object-fit: contain; border-radius: 4px; }
|
||||
@@ -821,6 +821,21 @@
|
||||
border-top: 1px solid var(--wfm-border);
|
||||
}
|
||||
|
||||
/* クロップ関連ボタンを左寄せにし、｜区切りの右に音声変換を置く */
|
||||
.wfm-video-edit-crop-head {
|
||||
justify-content: flex-start;
|
||||
}
|
||||
|
||||
.wfm-video-edit-crop-head > span:first-child {
|
||||
margin-right: 4px;
|
||||
}
|
||||
|
||||
.wfm-video-edit-head-sep {
|
||||
color: var(--wfm-text-secondary);
|
||||
font-weight: 400;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.wfm-video-edit-crop-row {
|
||||
display: flex;
|
||||
align-items: end;
|
||||
@@ -992,12 +1007,35 @@
|
||||
}
|
||||
|
||||
.wfm-video-source-panel {
|
||||
width: 220px;
|
||||
width: 320px;
|
||||
flex: none;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* ファイル名は1行(はみ出しは省略)、操作ボタン3つは折り返さず横一列 */
|
||||
.wfm-video-source-status-line {
|
||||
margin-top: 6px;
|
||||
min-height: 1.4em;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.wfm-video-source-actions {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
margin-top: 6px;
|
||||
}
|
||||
|
||||
.wfm-video-source-actions .wfm-btn {
|
||||
flex: 1 1 auto;
|
||||
min-width: 0;
|
||||
padding-left: 6px;
|
||||
padding-right: 6px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.wfm-video-pane-header {
|
||||
font-weight: 600;
|
||||
margin-bottom: 4px;
|
||||
|
||||
+972
-31
File diff suppressed because it is too large
Load Diff
+48
-3
@@ -112,7 +112,7 @@ function initTabs() {
|
||||
|
||||
function applyI18nToHtml() {
|
||||
// Tab labels
|
||||
const tabMap = { workflow: "tabWorkflow", nodes: "tabNodes", models: "tabModels", generate: "tabGenerate", prompt: "tabPrompt", gallery: "tabGallery", "image-edit": "tabImageEdit", video: "tabVideo", settings: "tabSettings", help: "tabHelp", ai: "tabAi", tagger: "tabTagger" };
|
||||
const tabMap = { workflow: "tabWorkflow", nodes: "tabNodes", models: "tabModels", generate: "tabGenerate", prompt: "tabPrompt", gallery: "tabGallery", "image-edit": "tabImageEdit", video: "tabVideo", settings: "tabSettings", help: "tabHelp", ai: "tabAi", tagger: "tabTagger", tagspace: "tabTagSpace" };
|
||||
document.querySelectorAll(".wfm-tab").forEach((tab) => {
|
||||
const key = tabMap[tab.dataset.tab];
|
||||
if (key) tab.textContent = t(key);
|
||||
@@ -375,6 +375,7 @@ function applyI18nToHtml() {
|
||||
aiVlmTask.querySelectorAll("option").forEach(opt => {
|
||||
if (opt.value === "describe") opt.textContent = t("aiVlmDescribe");
|
||||
if (opt.value === "prompt") opt.textContent = t("aiVlmPromptCreate");
|
||||
if (opt.value === "ocr") opt.textContent = t("aiVlmOcr");
|
||||
});
|
||||
}
|
||||
const aiVlmRunBtn = document.getElementById("wfm-ai-vlm-run");
|
||||
@@ -383,6 +384,17 @@ function applyI18nToHtml() {
|
||||
if (aiVlmResult) aiVlmResult.placeholder = t("aiVlmResultPlaceholder");
|
||||
const aiVlmCopyBtn = document.getElementById("wfm-ai-vlm-copy");
|
||||
if (aiVlmCopyBtn) aiVlmCopyBtn.textContent = t("aiCopyBtn");
|
||||
const aiVlmToTrans = document.getElementById("wfm-ai-vlm-to-trans");
|
||||
if (aiVlmToTrans) aiVlmToTrans.textContent = t("aiVlmToTranslation");
|
||||
const aiVlmToChat = document.getElementById("wfm-ai-vlm-to-chat");
|
||||
if (aiVlmToChat) aiVlmToChat.textContent = t("aiVlmToChat");
|
||||
const aiOcrModelLabel = document.getElementById("wfm-ai-ocr-model-label");
|
||||
if (aiOcrModelLabel) aiOcrModelLabel.textContent = t("aiOcrModelLabel");
|
||||
const aiOcrBoxesLabel = document.getElementById("wfm-ai-ocr-boxes-label");
|
||||
if (aiOcrBoxesLabel) aiOcrBoxesLabel.textContent = t("aiOcrShowBoxes");
|
||||
const aiOcrRefresh = document.getElementById("wfm-ai-ocr-refresh");
|
||||
if (aiOcrRefresh) aiOcrRefresh.title = t("aiOcrRefresh");
|
||||
document.querySelectorAll(".wfm-ai-pane-collapse-btn").forEach((b) => { b.title = t("aiPaneCollapse"); });
|
||||
const aiBackendTitle = document.getElementById("wfm-ai-settings-backend-title");
|
||||
if (aiBackendTitle) aiBackendTitle.textContent = t("aiSettingsBackend");
|
||||
const aiConnTitle = document.getElementById("wfm-ai-settings-conn-title");
|
||||
@@ -416,6 +428,24 @@ function applyI18nToHtml() {
|
||||
if (aiSaveBtn) aiSaveBtn.textContent = t("aiSettingsSaveBtn");
|
||||
const aiChatImgGenLabel = document.getElementById("wfm-ai-chat-imggen-label-text");
|
||||
if (aiChatImgGenLabel) aiChatImgGenLabel.textContent = t("aiChatImgGenToggle");
|
||||
const aiChatCheckLabelText = document.getElementById("wfm-ai-chat-check-label-text");
|
||||
if (aiChatCheckLabelText) aiChatCheckLabelText.textContent = t("aiChatCheckToggle");
|
||||
const aiChatCheckLabel = document.getElementById("wfm-ai-chat-check-label");
|
||||
if (aiChatCheckLabel) aiChatCheckLabel.title = t("aiChatCheckToggleTip");
|
||||
const aiCheckTitle = document.getElementById("wfm-ai-settings-check-title");
|
||||
if (aiCheckTitle) aiCheckTitle.textContent = t("aiSettingsCheckTitle");
|
||||
const aiJudgeLabel = document.getElementById("wfm-ai-judge-model-label");
|
||||
if (aiJudgeLabel) aiJudgeLabel.textContent = t("aiJudgeModelLabel");
|
||||
const aiJudgeRefresh = document.getElementById("wfm-ai-judge-refresh-btn");
|
||||
if (aiJudgeRefresh) aiJudgeRefresh.textContent = t("aiJudgeRefresh");
|
||||
const aiRoundsLabel = document.getElementById("wfm-ai-check-rounds-label");
|
||||
if (aiRoundsLabel) aiRoundsLabel.textContent = t("aiCheckRoundsLabel");
|
||||
const aiRoundsManual = document.querySelector("#wfm-ai-check-rounds-select option[value='0']");
|
||||
if (aiRoundsManual) aiRoundsManual.textContent = t("aiCheckRoundsManual");
|
||||
const aiSkillImportText = document.getElementById("wfm-ai-skill-import-text");
|
||||
if (aiSkillImportText) aiSkillImportText.textContent = t("aiSkillImportText");
|
||||
const aiSkillImportLabel = document.getElementById("wfm-ai-skill-import-label");
|
||||
if (aiSkillImportLabel) aiSkillImportLabel.title = t("aiSkillImportTip");
|
||||
const aiChatGenTitle = document.getElementById("wfm-ai-settings-chatgen-title");
|
||||
if (aiChatGenTitle) aiChatGenTitle.textContent = t("aiSettingsChatGenTitle");
|
||||
const aiChatGenDedicatedLabel = document.getElementById("wfm-ai-chatgen-dedicated-label");
|
||||
@@ -493,6 +523,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-gen-30": "helpGen30",
|
||||
"wfm-help-gen-31": "helpGen31",
|
||||
"wfm-help-gen-32": "helpGen32",
|
||||
"wfm-help-gen-33": "helpGen33",
|
||||
"wfm-help-feeder-title": "helpFeederTitle",
|
||||
"wfm-help-feeder-desc": "helpFeederDesc",
|
||||
"wfm-help-feeder-imgloop-title": "helpFeederImgloopTitle",
|
||||
@@ -526,7 +557,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-lab-plan-5": "helpLabPlan5", "wfm-help-lab-plan-6": "helpLabPlan6",
|
||||
"wfm-help-lab-plan-7": "helpLabPlan7", "wfm-help-lab-plan-8": "helpLabPlan8",
|
||||
"wfm-help-lab-plan-9": "helpLabPlan9", "wfm-help-lab-plan-10": "helpLabPlan10",
|
||||
"wfm-help-lab-plan-11": "helpLabPlan11",
|
||||
"wfm-help-lab-plan-11": "helpLabPlan11", "wfm-help-lab-plan-12": "helpLabPlan12",
|
||||
"wfm-help-lab-card-results-title": "helpLabCardResultsTitle",
|
||||
"wfm-help-lab-results-1": "helpLabResults1", "wfm-help-lab-results-2": "helpLabResults2",
|
||||
"wfm-help-lab-results-3": "helpLabResults3",
|
||||
@@ -561,6 +592,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-settings-20": "helpSettings20", "wfm-help-settings-21": "helpSettings21",
|
||||
"wfm-help-settings-22": "helpSettings22",
|
||||
"wfm-help-settings-23": "helpSettings23", "wfm-help-settings-24": "helpSettings24", "wfm-help-settings-25": "helpSettings25",
|
||||
"wfm-help-settings-26": "helpSettings26",
|
||||
"wfm-help-nodes-1": "helpNodes1", "wfm-help-nodes-2": "helpNodes2",
|
||||
"wfm-help-nodes-3": "helpNodes3", "wfm-help-nodes-4": "helpNodes4",
|
||||
"wfm-help-nodes-5": "helpNodes5", "wfm-help-nodes-6": "helpNodes6",
|
||||
@@ -588,10 +620,11 @@ function applyI18nToHtml() {
|
||||
"wfm-help-gallery-26": "helpGallery26",
|
||||
"wfm-help-gallery-28": "helpGallery28", "wfm-help-gallery-29": "helpGallery29",
|
||||
"wfm-help-gallery-30": "helpGallery30", "wfm-help-gallery-31": "helpGallery31",
|
||||
"wfm-help-gallery-32": "helpGallery32", "wfm-help-gallery-33": "helpGallery33", "wfm-help-gallery-34": "helpGallery34", "wfm-help-gallery-35": "helpGallery35", "wfm-help-gallery-36": "helpGallery36",
|
||||
"wfm-help-gallery-32": "helpGallery32", "wfm-help-gallery-33": "helpGallery33", "wfm-help-gallery-34": "helpGallery34", "wfm-help-gallery-35": "helpGallery35", "wfm-help-gallery-36": "helpGallery36", "wfm-help-gallery-37": "helpGallery37", "wfm-help-gallery-38": "helpGallery38",
|
||||
"wfm-help-gallery-card-styleprompt-title": "helpGalleryCardStyleprompt",
|
||||
"wfm-help-gallery-card-stylecatalog-title": "helpGalleryCardStylecatalog",
|
||||
"wfm-help-tagger-title": "helpTaggerTitle",
|
||||
"wfm-help-tagspace-title": "helpTagSpaceTitle", "wfm-help-tagspace-card-overview-title": "helpTagSpaceCardOverview", "wfm-help-tagspace-card-use-title": "helpTagSpaceCardUse", "wfm-help-tagspace-1": "helpTagSpace1", "wfm-help-tagspace-2": "helpTagSpace2", "wfm-help-tagspace-3": "helpTagSpace3", "wfm-help-tagspace-4": "helpTagSpace4", "wfm-help-tagspace-5": "helpTagSpace5", "wfm-help-tagspace-6": "helpTagSpace6", "wfm-help-tagspace-7": "helpTagSpace7", "wfm-help-tagspace-8": "helpTagSpace8", "wfm-help-tagspace-9": "helpTagSpace9", "wfm-help-tagspace-10": "helpTagSpace10",
|
||||
"wfm-help-tagger-1": "helpTagger1", "wfm-help-tagger-2": "helpTagger2",
|
||||
"wfm-help-tagger-3": "helpTagger3", "wfm-help-tagger-4": "helpTagger4",
|
||||
"wfm-help-tagger-5": "helpTagger5", "wfm-help-tagger-6": "helpTagger6",
|
||||
@@ -614,6 +647,10 @@ function applyI18nToHtml() {
|
||||
"wfm-help-ai-7": "helpAi7",
|
||||
"wfm-help-ai-8": "helpAi8",
|
||||
"wfm-help-ai-9": "helpAi9",
|
||||
"wfm-help-ai-10": "helpAi10",
|
||||
"wfm-help-ai-11": "helpAi11",
|
||||
"wfm-help-ai-12": "helpAi12",
|
||||
"wfm-help-ai-13": "helpAi13",
|
||||
"wfm-help-ai-card-layout-title": "helpAiCardLayout",
|
||||
"wfm-help-ai-card-translate-title": "helpAiCardTranslate",
|
||||
"wfm-help-ai-card-chat-title": "helpAiCardChat",
|
||||
@@ -622,6 +659,10 @@ function applyI18nToHtml() {
|
||||
"wfm-help-ai-card-skilltips-title": "helpAiCardSkillTips",
|
||||
"wfm-help-ai-card-tools-title": "helpAiCardTools",
|
||||
"wfm-help-ai-card-settings-title": "helpAiCardSettings",
|
||||
"wfm-help-ai-card-skilloptions-title": "helpAiCardSkillOptions",
|
||||
"wfm-help-ai-card-skillsmore-title": "helpAiCardSkillsMore",
|
||||
"wfm-help-ai-card-imagecheck-title": "helpAiCardImageCheck",
|
||||
"wfm-help-ai-card-ocr-title": "helpAiCardOcr",
|
||||
"wfm-help-imageedit-title": "helpImageEditTitle",
|
||||
"wfm-help-imageedit-tools-title": "helpImageEditToolsTitle",
|
||||
"wfm-help-imageedit-layer-title": "helpImageEditLayerTitle",
|
||||
@@ -644,6 +685,8 @@ function applyI18nToHtml() {
|
||||
"wfm-help-imageedit-26": "helpImageEdit26", "wfm-help-imageedit-27": "helpImageEdit27",
|
||||
"wfm-help-imageedit-28": "helpImageEdit28", "wfm-help-imageedit-29": "helpImageEdit29",
|
||||
"wfm-help-imageedit-30": "helpImageEdit30",
|
||||
"wfm-help-imageedit-draw-palette": "helpImageEditDrawPalette", "wfm-help-imageedit-draw-kbd": "helpImageEditDrawKbd",
|
||||
"wfm-help-imageedit-tabkbd-draw": "helpImageEditDrawKbd",
|
||||
"wfm-help-imageedit-31": "helpImageEdit31", "wfm-help-imageedit-32": "helpImageEdit32",
|
||||
"wfm-help-imageedit-33": "helpImageEdit33", "wfm-help-imageedit-34": "helpImageEdit34",
|
||||
"wfm-help-imageedit-7b": "helpImageEdit7b", "wfm-help-imageedit-7c": "helpImageEdit7c",
|
||||
@@ -845,6 +888,7 @@ import { initGalleryTab } from "./gallery-tab.js";
|
||||
import { initMetadataTab } from "./metadata-tab.js";
|
||||
import { initAiTab } from "./ai-tab.js";
|
||||
import { initTaggerTab } from "./tagger-tab.js";
|
||||
import { initTagSpaceTab } from "./tagspace-tab.js";
|
||||
import { imageEditTab } from "./image-edit-tab.js";
|
||||
import { initVideoTab } from "./video-tab.js";
|
||||
|
||||
@@ -897,6 +941,7 @@ document.addEventListener("DOMContentLoaded", () => {
|
||||
initGalleryTab();
|
||||
initAiTab();
|
||||
initTaggerTab();
|
||||
initTagSpaceTab();
|
||||
imageEditTab.init();
|
||||
initVideoTab();
|
||||
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
/**
|
||||
* chat-context.js — facts injected into the AI TOOL Chat system prompt by a skill's
|
||||
* `metadata.context` list ("workflow-detail", "logs"; "workflow" lives in ai-tab.js).
|
||||
*
|
||||
* Why: in the 2026-11-11 model check every local chat model answered a simple ComfyUI question
|
||||
* ("SDXL vs Flux") with confident factual errors. Diagnosis skills therefore must not rely on what
|
||||
* the model "knows" — they get verifiable data instead: the exact node graph, which node types
|
||||
* ComfyUI does not know (not installed), which model/option values are not in ComfyUI's own
|
||||
* lists (with the closest existing names), the last execution error from /history, and the tail
|
||||
* of the ComfyUI log. Everything is plain text, size-capped, and built fresh on every message.
|
||||
*/
|
||||
|
||||
import { comfyUI } from "./comfyui-client.js";
|
||||
import { comfyWorkflow } from "./comfyui-workflow.js";
|
||||
|
||||
const MAX_NODES = 60;
|
||||
const MAX_DETAIL_CHARS = 9000;
|
||||
const MAX_LOG_CHARS = 3500;
|
||||
|
||||
function _trunc(v, n = 80) {
|
||||
const s = String(v).replace(/\s+/g, " ");
|
||||
return s.length > n ? `${s.slice(0, n)}…` : s;
|
||||
}
|
||||
|
||||
// object_info input spec → the list of allowed values for a COMBO input, or null.
|
||||
// Two shapes exist: [[...options], {...}] (classic) and ["COMBO", {options: [...]}] (V3 nodes).
|
||||
function _comboOptions(spec) {
|
||||
if (!Array.isArray(spec)) return null;
|
||||
if (Array.isArray(spec[0])) return spec[0];
|
||||
if (spec[0] === "COMBO" && Array.isArray(spec[1]?.options)) return spec[1].options;
|
||||
return null;
|
||||
}
|
||||
|
||||
// Core ComfyUI link-only types. Custom node packs also define UI widgets with their own
|
||||
// upper-case type names (LOAD_3D viewport_state, COLOR, IMAGECOMPARE...), which an API-format
|
||||
// workflow legitimately omits, so only these well-known connection types are checked.
|
||||
const CONNECTION_TYPES = new Set([
|
||||
"MODEL", "CLIP", "VAE", "CONDITIONING", "LATENT", "IMAGE", "MASK", "CONTROL_NET",
|
||||
"CLIP_VISION", "CLIP_VISION_OUTPUT", "STYLE_MODEL", "UPSCALE_MODEL", "GLIGEN", "AUDIO",
|
||||
"VIDEO", "SIGMAS", "NOISE", "GUIDER", "SAMPLER", "LORA_MODEL", "HOOKS", "MESH", "VOXEL",
|
||||
]);
|
||||
|
||||
// A required input that can only be satisfied by a link (MODEL / CLIP / LATENT / IMAGE ...).
|
||||
function _isConnectionType(spec) {
|
||||
if (!Array.isArray(spec) || typeof spec[0] !== "string") return false;
|
||||
return CONNECTION_TYPES.has(spec[0]);
|
||||
}
|
||||
|
||||
function _inputSpec(info, name) {
|
||||
return info?.input?.required?.[name] ?? info?.input?.optional?.[name] ?? null;
|
||||
}
|
||||
|
||||
// Closest existing option names for a value that is not in the list (token overlap on the
|
||||
// file name, ignoring folders and extensions). Good enough to point at "you have v2, not v3".
|
||||
export function closestOptions(value, options, limit = 3) {
|
||||
const tokens = (s) => new Set(String(s).toLowerCase().replace(/\\/g, "/").split("/").pop()
|
||||
.replace(/\.(safetensors|ckpt|pt|pth|bin|gguf|sft)$/i, "")
|
||||
.split(/[^a-z0-9]+/).filter((t) => t.length > 1));
|
||||
const want = tokens(value);
|
||||
if (!want.size) return [];
|
||||
return options
|
||||
.filter((o) => typeof o === "string")
|
||||
.map((o) => {
|
||||
const have = tokens(o);
|
||||
let common = 0;
|
||||
for (const t of want) if (have.has(t)) common++;
|
||||
return { o, score: common / Math.max(want.size, have.size || 1) };
|
||||
})
|
||||
.filter((x) => x.score > 0)
|
||||
.sort((a, b) => b.score - a.score)
|
||||
.slice(0, limit)
|
||||
.map((x) => x.o);
|
||||
}
|
||||
|
||||
/**
|
||||
* Detailed, checked description of the workflow loaded in GenerateUI (API format).
|
||||
* Pure data → text; `objectInfo` is { classType: info } for the types used.
|
||||
*/
|
||||
export function describeWorkflowDetail(wf, objectInfo) {
|
||||
const entries = Object.entries(wf || {}).filter(([, n]) => n && typeof n === "object" && n.class_type);
|
||||
const lines = ["## Workflow facts (checked by Workflow Studio against this ComfyUI install)"];
|
||||
lines.push(`- ${entries.length} nodes.`);
|
||||
|
||||
const types = [...new Set(entries.map(([, n]) => n.class_type))];
|
||||
const missingTypes = types.filter((ty) => !objectInfo?.[ty]);
|
||||
lines.push(missingTypes.length
|
||||
? `- NOT INSTALLED node types (ComfyUI does not know them): ${missingTypes.join(", ")}`
|
||||
: "- All node types are installed.");
|
||||
|
||||
const problems = [];
|
||||
const nodeLines = [];
|
||||
const ids = new Set(entries.map(([id]) => id));
|
||||
for (const [id, node] of entries.slice(0, MAX_NODES)) {
|
||||
const info = objectInfo?.[node.class_type];
|
||||
const title = node._meta?.title && node._meta.title !== node.class_type ? ` "${_trunc(node._meta.title, 40)}"` : "";
|
||||
const values = [];
|
||||
const links = [];
|
||||
for (const [k, v] of Object.entries(node.inputs || {})) {
|
||||
if (Array.isArray(v) && v.length === 2 && (typeof v[0] === "string" || typeof v[0] === "number")) {
|
||||
const src = String(v[0]);
|
||||
links.push(`${k}<-#${src}`);
|
||||
if (!ids.has(src)) problems.push(`#${id} ${node.class_type}: input "${k}" is linked to node #${src}, which does not exist.`);
|
||||
continue;
|
||||
}
|
||||
if (v === null || typeof v === "object") continue;
|
||||
values.push(`${k}=${_trunc(v, k === "text" || k === "prompt" ? 120 : 70)}`);
|
||||
const options = info ? _comboOptions(_inputSpec(info, k)) : null;
|
||||
if (options && typeof v === "string" && options.length && !options.includes(v)) {
|
||||
const near = closestOptions(v, options);
|
||||
problems.push(`#${id} ${node.class_type}: ${k}="${v}" is NOT in ComfyUI's list for this input${near.length ? ` (closest existing: ${near.map((o) => `"${o}"`).join(", ")})` : " (no similar name found)"}.`);
|
||||
}
|
||||
}
|
||||
if (info) {
|
||||
// Only unconnected required CONNECTION inputs (MODEL, CLIP, LATENT, IMAGE, ...) are
|
||||
// reported: those make the run fail for sure. Missing widget values are usually filled
|
||||
// with defaults or are UI-only, and dynamic inputs (Autogrow "COMFY_*") are stored
|
||||
// under other keys - checking them produced false alarms on ~1/3 of real workflows.
|
||||
for (const [name, spec] of Object.entries(info.input?.required || {})) {
|
||||
if (name in (node.inputs || {}) || !_isConnectionType(spec)) continue;
|
||||
problems.push(`#${id} ${node.class_type}: required ${spec[0]} input "${name}" is not connected.`);
|
||||
}
|
||||
}
|
||||
nodeLines.push(`#${id} ${node.class_type}${title}${missingTypes.includes(node.class_type) ? " [NOT INSTALLED]" : ""}: ${[...values, ...links].join(", ") || "(no inputs)"}`);
|
||||
}
|
||||
if (entries.length > MAX_NODES) nodeLines.push(`… ${entries.length - MAX_NODES} more nodes not listed.`);
|
||||
|
||||
lines.push(problems.length ? `- Detected problems:\n${problems.slice(0, 30).map((p) => ` - ${p}`).join("\n")}` : "- No missing files, options or links detected.");
|
||||
lines.push("- Nodes (#id ClassType: widget values, input<-#source node):");
|
||||
let text = `${lines.join("\n")}\n${nodeLines.map((l) => ` ${l}`).join("\n")}`;
|
||||
if (text.length > MAX_DETAIL_CHARS) text = `${text.slice(0, MAX_DETAIL_CHARS)}\n … (truncated)`;
|
||||
return text;
|
||||
}
|
||||
|
||||
export async function buildWorkflowDetailContext() {
|
||||
const wf = comfyUI.currentWorkflow;
|
||||
if (!wf) return "## Workflow facts\nNo workflow is loaded in GenerateUI. Ask the user to load the workflow they are asking about in the GenerateUI tab.";
|
||||
const types = [...new Set(Object.values(wf).map((n) => n?.class_type).filter(Boolean))];
|
||||
let objectInfo = {};
|
||||
try {
|
||||
objectInfo = await comfyWorkflow.getObjectInfoForTypes(types);
|
||||
} catch (e) {
|
||||
return `## Workflow facts\nCould not read node definitions from ComfyUI (${e.message}), so installed nodes and files could not be checked.`;
|
||||
}
|
||||
return describeWorkflowDetail(wf, objectInfo);
|
||||
}
|
||||
|
||||
// ANSI colour codes from ComfyUI's coloured log lines
|
||||
const ANSI_RE = /\u001b\[[0-9;]*m/g;
|
||||
|
||||
/** Keep the most recent error block of the log (from the last Traceback / error line), else the tail. */
|
||||
export function extractLogTail(logText, maxChars = MAX_LOG_CHARS) {
|
||||
const lines = String(logText || "").replace(ANSI_RE, "").split(/\r?\n/).filter((l) => l.trim());
|
||||
let start = -1;
|
||||
for (let i = lines.length - 1; i >= 0; i--) {
|
||||
if (/Traceback \(most recent call last\)|!!! Exception|Error:|ERROR|Prompt outputs failed validation/.test(lines[i])) {
|
||||
start = i;
|
||||
// walk back to the beginning of a traceback block
|
||||
while (start > 0 && /^\s|Traceback|File "/.test(lines[start - 1]) && i - start < 40) start--;
|
||||
break;
|
||||
}
|
||||
}
|
||||
const picked = start >= 0 ? lines.slice(Math.max(0, start - 3), start + 40) : lines.slice(-25);
|
||||
let text = picked.join("\n");
|
||||
if (text.length > maxChars) text = `…${text.slice(-maxChars)}`;
|
||||
return { text, hasError: start >= 0 };
|
||||
}
|
||||
|
||||
/** Last execution error recorded in ComfyUI's /history (most recent first). */
|
||||
export function lastExecutionError(history) {
|
||||
const runs = Object.values(history || {});
|
||||
for (let i = runs.length - 1; i >= 0; i--) {
|
||||
const msgs = runs[i]?.status?.messages || [];
|
||||
const err = msgs.find((m) => m?.[0] === "execution_error")?.[1];
|
||||
if (err) return err;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function buildLogsContext() {
|
||||
const parts = ["## ComfyUI runtime facts"];
|
||||
try {
|
||||
const res = await fetch(`${comfyUI.baseUrl || ""}/history?max_items=5`);
|
||||
const err = res.ok ? lastExecutionError(await res.json()) : null;
|
||||
if (err) {
|
||||
const tb = Array.isArray(err.traceback) ? err.traceback.join("").split("\n").filter(Boolean).slice(-8).join("\n") : "";
|
||||
parts.push(`- Last execution error (from /history): node #${err.node_id} ${err.node_type}: ${err.exception_type}: ${_trunc(err.exception_message, 600)}${tb ? `\n Traceback tail:\n${tb}` : ""}`);
|
||||
} else {
|
||||
parts.push("- No execution error in the last 5 runs (/history).");
|
||||
}
|
||||
} catch (e) {
|
||||
parts.push(`- Could not read /history: ${e.message}`);
|
||||
}
|
||||
try {
|
||||
const res = await fetch(`${comfyUI.baseUrl || ""}/internal/logs`);
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const raw = await res.json();
|
||||
const { text, hasError } = extractLogTail(typeof raw === "string" ? raw : JSON.stringify(raw));
|
||||
parts.push(`- ComfyUI log (${hasError ? "most recent error block" : "last lines, no error found"}):\n${text}`);
|
||||
} catch (e) {
|
||||
parts.push(`- Could not read the ComfyUI log: ${e.message}`);
|
||||
}
|
||||
return parts.join("\n");
|
||||
}
|
||||
@@ -0,0 +1,300 @@
|
||||
/**
|
||||
* chat-image-check.js — AI TOOL Chat: judge generated images (and rendered SVGs) with a
|
||||
* vision-capable decision model, so the chat can fix only what is missing.
|
||||
*
|
||||
* Why a decision model instead of the chat VLM: a generator VLM answers pass/fail without a
|
||||
* probability and was measured to say "yes" to things that are not in the image (2026-11-11:
|
||||
* gemma4-uncensored e4b judged "holds a sword" = true). Ollama decision models with image input
|
||||
* (d3:4b, clef-flash, clef) answered 18/18 checklist questions correctly on the same image with a
|
||||
* clear gap between yes (0.75-1.0) and no (0.0-0.1), and return calibrated `noul` probabilities
|
||||
* that code can threshold. Only one image was measured, so results are shown to the user and
|
||||
* automatic fix rounds are off by default.
|
||||
*
|
||||
* Flow (driven by ai-tab.js):
|
||||
* buildChecklist() chat LLM turns the user's request into 3-7 yes/no visual questions
|
||||
* judgeImage() decision model answers them for the image (+ a quality score)
|
||||
* renderCheckCard() shows ✔ / ? / ✘ per item; failed + uncertain items are what gets fixed
|
||||
*/
|
||||
|
||||
import { t } from "./i18n.js";
|
||||
import { escapeHtml } from "./util.js";
|
||||
import { decide, noul, score } from "./decision-client.js";
|
||||
|
||||
const OLLAMA_DEFAULT_URL = "http://localhost:11434";
|
||||
export const DEFAULT_JUDGE_MODEL = "d3:4b";
|
||||
|
||||
// Probability bands for a noul answer (after flipping "expect: no" items). In the 2026-11-11
|
||||
// test runs with d3:4b, correct passes were >= 0.88 while wrong passes caused by attribute binding
|
||||
// ("Is the boy wearing a green hat?" when the girl wore it) landed at 0.70-0.79, so 0.85 is the
|
||||
// pass line and 0.3-0.85 counts as "uncertain" (fixed like a failure). clef answers yes a bit
|
||||
// lower (0.75+), so it reports more "uncertain" items.
|
||||
export const PASS_THRESHOLD = 0.85;
|
||||
export const FAIL_THRESHOLD = 0.3;
|
||||
|
||||
/** Judge connection: the AI TOOL Ollama URL when the chat backend is Ollama, else the default. */
|
||||
export function getJudgeConfig(aiSettings = {}) {
|
||||
const baseUrl = aiSettings.backend === "ollama" && aiSettings.backendUrl
|
||||
? aiSettings.backendUrl
|
||||
: OLLAMA_DEFAULT_URL;
|
||||
return {
|
||||
baseUrl: baseUrl.replace(/\/+$/, ""),
|
||||
model: aiSettings.chatJudgeModel || DEFAULT_JUDGE_MODEL,
|
||||
autoRounds: Math.max(0, Math.min(3, parseInt(aiSettings.chatCheckAutoRounds, 10) || 0)),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Ollama decision models (capabilities include "decision"), vision-capable ones first.
|
||||
* Returns [{ name, vision }]. Throws if Ollama can't be reached.
|
||||
*/
|
||||
export async function listJudgeModels(baseUrl) {
|
||||
const url = (baseUrl || OLLAMA_DEFAULT_URL).replace(/\/+$/, "");
|
||||
const res = await fetch(`${url}/api/tags`);
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const data = await res.json();
|
||||
const names = (data.models || [])
|
||||
.filter((m) => Array.isArray(m.capabilities) && m.capabilities.includes("decision"))
|
||||
.map((m) => m.name);
|
||||
const out = await Promise.all(names.map(async (name) => {
|
||||
try {
|
||||
const r = await fetch(`${url}/api/show`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ model: name }),
|
||||
});
|
||||
const d = r.ok ? await r.json() : {};
|
||||
const vision = (Array.isArray(d.capabilities) && d.capabilities.includes("vision"))
|
||||
|| !!(d.projector_info && Object.keys(d.projector_info).length);
|
||||
return { name, vision };
|
||||
} catch {
|
||||
return { name, vision: false };
|
||||
}
|
||||
}));
|
||||
return out.sort((a, b) => (b.vision - a.vision) || a.name.localeCompare(b.name));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Checklist
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const CHECKLIST_SCHEMA = {
|
||||
type: "object",
|
||||
properties: {
|
||||
items: {
|
||||
type: "array",
|
||||
items: {
|
||||
type: "object",
|
||||
properties: {
|
||||
question: { type: "string" },
|
||||
label: { type: "string" },
|
||||
expect: { type: "string", enum: ["yes", "no"] },
|
||||
},
|
||||
required: ["question", "label", "expect"],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: ["items"],
|
||||
};
|
||||
|
||||
const CHECKLIST_SYSTEM = `You turn an image request into a short visual checklist for an image-judging model.
|
||||
Return JSON only: {"items":[{"question":"...","label":"...","expect":"yes"}]}
|
||||
Rules:
|
||||
- 3 to 8 items, one concrete requirement per item.
|
||||
- "question": an English yes/no question that can be answered by looking at the image alone. Always name who or what it is about (e.g. "Is the girl wearing a white dress?", never "Is someone wearing a white dress?"). For positions, name both sides (e.g. "Is the white rabbit between the boy and the girl?").
|
||||
- "expect": "yes" when YES means the requirement is met.
|
||||
- The judging model often mixes up which person has which attribute. So when the request has two or more people or animals with different attributes, add up to 2 swap-check items with "expect": "no" that ask whether the attribute is on the WRONG one (e.g. the boy should wear the green hat -> "Is the girl wearing a green hat?" with "expect": "no").
|
||||
- "label": the requirement as a very short phrase in the language of the user's request, including who it is about (e.g. "男の子の緑の帽子", "帽子は女の子ではない").
|
||||
- Only check visible things the user actually asked for: subject and count, key attributes and colors, clothing, pose or action, setting, time of day or lighting, text that must appear.
|
||||
- Never check image quality, art style quality, or details the user did not ask for.`;
|
||||
|
||||
/** Pull the first JSON object out of a model reply (code fences / prose tolerated). */
|
||||
export function parseChecklist(text) {
|
||||
if (!text) return [];
|
||||
let data = null;
|
||||
const start = text.indexOf("{");
|
||||
const end = text.lastIndexOf("}");
|
||||
if (start >= 0 && end > start) {
|
||||
try { data = JSON.parse(text.slice(start, end + 1)); } catch { data = null; }
|
||||
}
|
||||
const items = Array.isArray(data?.items) ? data.items : [];
|
||||
return items
|
||||
.map((it) => (typeof it === "string" ? { question: it, label: it } : it))
|
||||
.filter((it) => it && typeof it.question === "string" && it.question.trim())
|
||||
.map((it) => ({
|
||||
question: it.question.trim().slice(0, 300),
|
||||
label: String(it.label || it.question).trim().slice(0, 80),
|
||||
expect: String(it.expect || "yes").toLowerCase() === "no" ? "no" : "yes",
|
||||
}))
|
||||
.slice(0, 8);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ask the chat LLM for the checklist. `chat(messages, extraSettings)` must return the reply text.
|
||||
* Falls back to one whole-prompt question if the model returns nothing usable.
|
||||
*/
|
||||
export async function buildChecklist(chat, request, prompt) {
|
||||
const user = `User request: ${request || "(none)"}\nPrompt used for generation: ${prompt || "(none)"}`;
|
||||
let items = [];
|
||||
try {
|
||||
const text = await chat(
|
||||
[{ role: "system", content: CHECKLIST_SYSTEM }, { role: "user", content: user }],
|
||||
{ format: CHECKLIST_SCHEMA, temperature: 0.2, thinkingMode: false },
|
||||
);
|
||||
items = parseChecklist(text);
|
||||
} catch { /* fall through to the fallback below */ }
|
||||
if (items.length === 0) {
|
||||
const desc = (request || prompt || "").replace(/\s+/g, " ").slice(0, 300);
|
||||
items = [{ question: `Does the image match this description: "${desc}"?`, label: desc.slice(0, 40), expect: "yes" }];
|
||||
}
|
||||
return items;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Judging
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Image URL / data URL → JPEG base64 (no prefix), downscaled so the longest side is <= maxSide. */
|
||||
export async function imageUrlToJpegBase64(src, maxSide = 768) {
|
||||
const img = await new Promise((resolve, reject) => {
|
||||
const el = new Image();
|
||||
el.onload = () => resolve(el);
|
||||
el.onerror = () => reject(new Error("image load failed"));
|
||||
el.src = src;
|
||||
});
|
||||
const scale = Math.min(1, maxSide / Math.max(img.naturalWidth || 1, img.naturalHeight || 1));
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = Math.max(1, Math.round((img.naturalWidth || maxSide) * scale));
|
||||
canvas.height = Math.max(1, Math.round((img.naturalHeight || maxSide) * scale));
|
||||
const ctx = canvas.getContext("2d");
|
||||
ctx.fillStyle = "#ffffff"; // transparent SVG/PNG areas → white, like a viewer would show
|
||||
ctx.fillRect(0, 0, canvas.width, canvas.height);
|
||||
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
|
||||
return canvas.toDataURL("image/jpeg", 0.9).split(",")[1] || "";
|
||||
}
|
||||
|
||||
/** SVG markup → JPEG base64 rendered at `size` px (white background, currentColor = black). */
|
||||
export async function svgToJpegBase64(svgCode, size = 512) {
|
||||
let markup = svgCode;
|
||||
// Render at a fixed size even when the SVG has no width/height (viewBox only).
|
||||
if (!/<svg[^>]*\swidth=/i.test(markup)) markup = markup.replace(/<svg\b/i, `<svg width="${size}" height="${size}"`);
|
||||
const dataUrl = `data:image/svg+xml;base64,${btoa(unescape(encodeURIComponent(markup)))}`;
|
||||
return imageUrlToJpegBase64(dataUrl, size);
|
||||
}
|
||||
|
||||
const QUALITY_LEVELS = ["poor", "acceptable", "excellent"];
|
||||
|
||||
/**
|
||||
* Judge one image against the checklist. The first request after ComfyUI generated an image can
|
||||
* fail while VRAM is still held, so it retries once after 4 s.
|
||||
* Returns { items: [{ question, label, p, status: "pass"|"uncertain"|"fail" }], quality, ms }.
|
||||
*/
|
||||
export async function judgeImage(imageBase64, items, cfg, { withQuality = true } = {}) {
|
||||
const questions = {};
|
||||
items.forEach((it, i) => { questions[`q${i + 1}`] = noul(it.question); });
|
||||
if (withQuality) questions.quality = score("Overall visual quality of the image", QUALITY_LEVELS);
|
||||
const opts = { backend: "ollama", baseUrl: cfg.baseUrl, model: cfg.model };
|
||||
const started = performance.now();
|
||||
let answers;
|
||||
try {
|
||||
answers = await decide("A generated image to be checked against the user's requirements.", questions, opts, [imageBase64]);
|
||||
} catch (e) {
|
||||
await new Promise((r) => setTimeout(r, 4000));
|
||||
answers = await decide("A generated image to be checked against the user's requirements.", questions, opts, [imageBase64]);
|
||||
}
|
||||
const results = items.map((it, i) => {
|
||||
const p = answers[`q${i + 1}`]?.noul;
|
||||
const raw = typeof p === "number" ? p : 0;
|
||||
// "expect: no" items (swap checks) are met when the answer is NO
|
||||
const prob = it.expect === "no" ? 1 - raw : raw;
|
||||
const status = prob >= PASS_THRESHOLD ? "pass" : prob < FAIL_THRESHOLD ? "fail" : "uncertain";
|
||||
return { ...it, p: prob, status };
|
||||
});
|
||||
let quality = null;
|
||||
if (withQuality && answers.quality) {
|
||||
const probs = answers.quality.probabilities || {};
|
||||
let best = null, bestP = -1;
|
||||
for (const [k, v] of Object.entries(probs)) if (v > bestP) { best = Number(k); bestP = v; }
|
||||
quality = { level: best, label: QUALITY_LEVELS[best] ?? null, value: answers.quality.score };
|
||||
}
|
||||
return { items: results, quality, ms: Math.round(performance.now() - started) };
|
||||
}
|
||||
|
||||
export function failedItems(result) {
|
||||
return (result?.items || []).filter((it) => it.status !== "pass");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SVG part list ("Parts: body, head (circle), beak" line written by the SVG skill)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function parseSvgParts(text) {
|
||||
const m = (text || "").match(/^\s*\**parts?\**\s*:\s*(.+)$/im);
|
||||
if (!m) return [];
|
||||
// Split on commas outside parentheses, then drop "(how it is drawn)" notes.
|
||||
const parts = [];
|
||||
let depth = 0, cur = "";
|
||||
for (const ch of m[1]) {
|
||||
if (ch === "(") depth++;
|
||||
if (ch === ")") depth = Math.max(0, depth - 1);
|
||||
if ((ch === "," || ch === ";") && depth === 0) { parts.push(cur); cur = ""; } else cur += ch;
|
||||
}
|
||||
parts.push(cur);
|
||||
return parts
|
||||
.map((p) => p.replace(/\([^)]*\)/g, "").replace(/[.*_`]/g, "").replace(/\s+/g, " ").trim())
|
||||
.filter((p) => p && p.length <= 40)
|
||||
.slice(0, 7);
|
||||
}
|
||||
|
||||
/** Checklist for a rendered SVG: one question per part plus "is it recognisable as <request>". */
|
||||
export function svgChecklist(parts, request) {
|
||||
const items = parts.map((p) => ({ question: `Does the drawing clearly show a ${p}?`, label: p, expect: "yes" }));
|
||||
const subject = (request || "").replace(/\s+/g, " ").trim().slice(0, 200);
|
||||
if (subject) items.unshift({ question: `Is this drawing recognizable as: "${subject}"?`, label: subject.slice(0, 40), expect: "yes" });
|
||||
return items.slice(0, 8);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Result card
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const STATUS_ICON = { pass: "✔", uncertain: "?", fail: "✘" };
|
||||
|
||||
/**
|
||||
* Render the judgement as a chat card. `actions` = [{ label, primary, onClick(btn) }].
|
||||
* Returns the card element.
|
||||
*/
|
||||
export function renderCheckCard(result, { model, round = 0, actions = [] } = {}) {
|
||||
const card = document.createElement("div");
|
||||
card.className = "wfm-ai-check-card";
|
||||
const failed = failedItems(result);
|
||||
const head = failed.length === 0 ? t("aiChatCheckPassed") : t("aiChatCheckFailedCount", failed.length);
|
||||
const roundText = round > 0 ? ` · ${t("aiChatCheckRound", round)}` : "";
|
||||
card.innerHTML = `
|
||||
<div class="wfm-ai-check-head ${failed.length ? "is-fail" : "is-pass"}">
|
||||
<span>${escapeHtml(head)}</span>
|
||||
<span class="wfm-ai-check-meta">${escapeHtml(t("aiChatCheckBy", model))}${escapeHtml(roundText)} · ${(result.ms / 1000).toFixed(1)}s</span>
|
||||
</div>
|
||||
<ul class="wfm-ai-check-list">
|
||||
${result.items.map((it) => `
|
||||
<li class="wfm-ai-check-item is-${it.status}" title="${escapeHtml(it.question)}">
|
||||
<span class="wfm-ai-check-icon">${STATUS_ICON[it.status]}</span>
|
||||
<span class="wfm-ai-check-label">${escapeHtml(it.label)}</span>
|
||||
<span class="wfm-ai-check-p">${Math.round(it.p * 100)}%</span>
|
||||
</li>`).join("")}
|
||||
</ul>
|
||||
${result.quality?.label ? `<div class="wfm-ai-check-quality">${escapeHtml(t("aiChatCheckQuality", t(`aiChatCheckQuality_${result.quality.label}`)))}</div>` : ""}
|
||||
`;
|
||||
if (actions.length) {
|
||||
const row = document.createElement("div");
|
||||
row.className = "wfm-ai-chat-msg-actions";
|
||||
for (const a of actions) {
|
||||
const btn = document.createElement("button");
|
||||
btn.className = `wfm-btn wfm-btn-sm${a.primary ? " wfm-btn-primary" : ""}`;
|
||||
btn.textContent = a.label;
|
||||
btn.addEventListener("click", () => a.onClick(btn));
|
||||
row.appendChild(btn);
|
||||
}
|
||||
card.appendChild(row);
|
||||
}
|
||||
return card;
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* chat-markdown.js — tiny, dependency-free Markdown renderer for AI TOOL Chat replies.
|
||||
*
|
||||
* Safety model: the whole input is HTML-escaped FIRST, then a small set of Markdown constructs is
|
||||
* turned into tags. No raw HTML from the model ever reaches the DOM, links are not rendered as
|
||||
* anchors (the model's URLs stay plain text), and images are never loaded. This keeps the
|
||||
* existing guarantees of the chat (it showed replies with textContent before) while making long
|
||||
* structured answers (lists, headings, code blocks) readable.
|
||||
*
|
||||
* Supported: fenced code blocks (```lang), inline `code`, **bold**, *italic*, # headings (h1-h4
|
||||
* rendered as h4-h6-ish sizes), - / * / 1. lists (one level), > quotes, --- rules, paragraphs and
|
||||
* line breaks. Anything else is left as escaped text.
|
||||
*/
|
||||
|
||||
function escapeHtml(s) {
|
||||
return String(s)
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, """)
|
||||
.replace(/'/g, "'");
|
||||
}
|
||||
|
||||
// Inline formatting on already-escaped text. Code spans are cut out first so their content is
|
||||
// not touched by the bold/italic rules.
|
||||
function renderInline(escaped) {
|
||||
const codes = [];
|
||||
let s = escaped.replace(/`([^`\n]+)`/g, (_, c) => {
|
||||
codes.push(c);
|
||||
return `\u0000${codes.length - 1}\u0000`;
|
||||
});
|
||||
s = s.replace(/\*\*([^*\n]+)\*\*/g, "<strong>$1</strong>");
|
||||
s = s.replace(/__([^_\n]+)__/g, "<strong>$1</strong>");
|
||||
s = s.replace(/(^|[^*\w])\*(?=\S)([^*\n]*?\S)\*(?!\w)/g, "$1<em>$2</em>");
|
||||
s = s.replace(/\u0000(\d+)\u0000/g, (_, i) => `<code>${codes[Number(i)]}</code>`);
|
||||
return s;
|
||||
}
|
||||
|
||||
/** Markdown text → safe HTML string. */
|
||||
export function renderMarkdown(text) {
|
||||
const lines = String(text || "").replace(/\r\n?/g, "\n").split("\n");
|
||||
const out = [];
|
||||
let para = [];
|
||||
let list = null; // { tag: "ul"|"ol", items: [], start }
|
||||
|
||||
const flushPara = () => {
|
||||
if (para.length) {
|
||||
out.push(`<p>${para.map((l) => renderInline(escapeHtml(l))).join("<br>")}</p>`);
|
||||
para = [];
|
||||
}
|
||||
};
|
||||
const flushList = () => {
|
||||
if (list) {
|
||||
const start = list.tag === "ol" && list.start > 1 ? ` start="${list.start}"` : "";
|
||||
out.push(`<${list.tag}${start}>${list.items.map((i) => `<li>${renderInline(escapeHtml(i))}</li>`).join("")}</${list.tag}>`);
|
||||
list = null;
|
||||
}
|
||||
};
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
|
||||
const fence = line.match(/^\s*```\s*([\w+-]*)\s*$/);
|
||||
if (fence) {
|
||||
flushPara(); flushList();
|
||||
const lang = fence[1] || "";
|
||||
const body = [];
|
||||
i++;
|
||||
while (i < lines.length && !/^\s*```\s*$/.test(lines[i])) { body.push(lines[i]); i++; }
|
||||
const label = lang ? `<span class="wfm-md-code-lang">${escapeHtml(lang)}</span>` : "";
|
||||
out.push(`<pre class="wfm-md-pre">${label}<code>${escapeHtml(body.join("\n"))}</code></pre>`);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!line.trim()) { flushPara(); flushList(); continue; }
|
||||
|
||||
const heading = line.match(/^\s*(#{1,4})\s+(.+)$/);
|
||||
if (heading) {
|
||||
flushPara(); flushList();
|
||||
const level = Math.min(6, heading[1].length + 2); // # -> h3 ... #### -> h6
|
||||
out.push(`<h${level} class="wfm-md-h">${renderInline(escapeHtml(heading[2]))}</h${level}>`);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (/^\s*(-{3,}|\*{3,}|_{3,})\s*$/.test(line)) { flushPara(); flushList(); out.push("<hr>"); continue; }
|
||||
|
||||
if (line.trim().startsWith(">")) {
|
||||
flushPara(); flushList();
|
||||
out.push(`<blockquote>${renderInline(escapeHtml(line.trim().replace(/^>\s?/, "")))}</blockquote>`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const ul = line.match(/^\s*[-*+]\s+(.+)$/);
|
||||
const ol = line.match(/^\s*(\d+)[.)]\s+(.+)$/);
|
||||
if (ul || ol) {
|
||||
flushPara();
|
||||
const tag = ul ? "ul" : "ol";
|
||||
if (list && list.tag !== tag) flushList();
|
||||
// A numbered list interrupted by bullets keeps counting from the source number
|
||||
if (!list) list = { tag, items: [], start: ol ? Math.min(Number(ol[1]) || 1, 9999) : 1 };
|
||||
list.items.push(ul ? ul[1] : ol[2]);
|
||||
continue;
|
||||
}
|
||||
|
||||
flushList();
|
||||
para.push(line);
|
||||
}
|
||||
flushPara(); flushList();
|
||||
return out.join("");
|
||||
}
|
||||
+112
-1
@@ -3,10 +3,12 @@
|
||||
*/
|
||||
|
||||
import { comfyUI } from "./comfyui-client.js";
|
||||
import { comfyWorkflow } from "./comfyui-workflow.js";
|
||||
import { syncJsonHighlight } from "./json-highlight.js";
|
||||
import { t } from "./i18n.js";
|
||||
import { escapeHtml, setupSearchClearBtn } from "./util.js";
|
||||
import { showToast } from "./app.js";
|
||||
import { translateButtonsHtml, attachTranslateButtons } from "./prompt-translate.js";
|
||||
|
||||
// ── Latent Image preset state ─────────────────────────────
|
||||
const _LATENT_PRESET_KEY = "wfm_latent_presets";
|
||||
@@ -315,6 +317,104 @@ function _syncStackToggleAll(stackModels) {
|
||||
cb.indeterminate = !allOn && !allOff;
|
||||
}
|
||||
|
||||
// Two-option combos that are really an on/off switch (KSamplerAdvanced add_noise, DWPreprocessor
|
||||
// detect_*, ...): [on value, off value], compared case-insensitively.
|
||||
const _TOGGLE_COMBO_PAIRS = [["enable", "disable"], ["enabled", "disabled"], ["on", "off"], ["true", "false"], ["yes", "no"]];
|
||||
|
||||
function _comboOptions(spec) {
|
||||
if (Array.isArray(spec?.[0])) return spec[0];
|
||||
if (spec?.[0] === "COMBO" && Array.isArray(spec[1]?.options)) return spec[1].options;
|
||||
return null;
|
||||
}
|
||||
|
||||
/** [onValue, offValue] in the node's own spelling when the combo is an on/off pair, else null */
|
||||
function _toggleComboValues(spec) {
|
||||
const opts = _comboOptions(spec);
|
||||
if (!opts || opts.length !== 2 || !opts.every((o) => typeof o === "string")) return null;
|
||||
const lower = opts.map((o) => o.toLowerCase());
|
||||
for (const [on, off] of _TOGGLE_COMBO_PAIRS) {
|
||||
const iOn = lower.indexOf(on), iOff = lower.indexOf(off);
|
||||
if (iOn >= 0 && iOff >= 0) return [opts[iOn], opts[iOff]];
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function _compareNodeIds(a, b) {
|
||||
const x = String(a).split(":").map(Number), y = String(b).split(":").map(Number);
|
||||
for (let i = 0; i < Math.max(x.length, y.length); i++) {
|
||||
const d = (x[i] ?? -1) - (y[i] ?? -1);
|
||||
if (d) return d;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* On/off items of the API workflow for the Settings tab's Switches column: every literal
|
||||
* true/false input (ComfySwitchNode switch, PrimitiveBoolean value, TextGenerate thinking, ...)
|
||||
* plus string combos whose only two options are an on/off pair (needs object_info). Linked
|
||||
* inputs are arrays and never match; their source node shows up instead.
|
||||
*/
|
||||
function _collectSwitchInputs(workflow, objectInfo = {}) {
|
||||
const out = [];
|
||||
for (const [id, node] of Object.entries(workflow || {})) {
|
||||
if (!node?.class_type || !node.inputs) continue;
|
||||
const info = objectInfo[node.class_type]?.input || {};
|
||||
for (const [key, value] of Object.entries(node.inputs)) {
|
||||
const base = { id, type: node.class_type, title: node._meta?.title || node.class_type, key };
|
||||
if (typeof value === "boolean") {
|
||||
out.push({ ...base, on: value, onValue: true, offValue: false });
|
||||
} else if (typeof value === "string") {
|
||||
const pair = _toggleComboValues(info.required?.[key] || info.optional?.[key]);
|
||||
if (pair && (value === pair[0] || value === pair[1])) {
|
||||
out.push({ ...base, on: value === pair[0], onValue: pair[0], offValue: pair[1] });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out.sort((a, b) => _compareNodeIds(a.id, b.id));
|
||||
}
|
||||
|
||||
let _switchRenderSeq = 0;
|
||||
|
||||
/** Fill the Switches column. Booleans show at once; on/off combos follow once object_info is in. */
|
||||
async function _renderSwitchColumn(col) {
|
||||
if (!col) return;
|
||||
const seq = ++_switchRenderSeq;
|
||||
const workflow = comfyUI.currentWorkflow;
|
||||
const draw = (items) => {
|
||||
col.innerHTML = items.length ? `<div class="wfm-settings-switches">${items.map((s, i) => `
|
||||
<label class="wfm-settings-switch" title="${escapeHtml(`#${s.id} ${s.type} › ${s.key}`)}">
|
||||
<input type="checkbox" class="wfm-settings-switch-cb" data-index="${i}" ${s.on ? "checked" : ""}>
|
||||
<span class="wfm-settings-switch-text">
|
||||
<span class="wfm-settings-switch-node">${escapeHtml(s.title)}</span>
|
||||
<span class="wfm-settings-switch-key">${escapeHtml(s.key)}</span>
|
||||
</span>
|
||||
<span class="wfm-settings-switch-state">${escapeHtml(String(s.on ? s.onValue : s.offValue))}</span>
|
||||
</label>`).join("")}
|
||||
</div>` : `<p class="wfm-placeholder">${t("genSwitchesNone")}</p>`;
|
||||
// Applies immediately (no Apply button): written straight into the API workflow Generate sends
|
||||
col.querySelectorAll(".wfm-settings-switch-cb").forEach((cb) => {
|
||||
cb.addEventListener("change", () => {
|
||||
const s = items[Number(cb.dataset.index)];
|
||||
const node = comfyUI.currentWorkflow?.[s.id];
|
||||
if (!node?.inputs || !(s.key in node.inputs) || Array.isArray(node.inputs[s.key])) return;
|
||||
s.on = cb.checked;
|
||||
node.inputs[s.key] = s.on ? s.onValue : s.offValue;
|
||||
const state = cb.closest(".wfm-settings-switch")?.querySelector(".wfm-settings-switch-state");
|
||||
if (state) state.textContent = String(node.inputs[s.key]);
|
||||
_syncRawJson();
|
||||
});
|
||||
});
|
||||
};
|
||||
draw(_collectSwitchInputs(workflow));
|
||||
const types = [...new Set(Object.values(workflow || {}).map((n) => n?.class_type).filter(Boolean))];
|
||||
let objectInfo = {};
|
||||
try { objectInfo = await comfyWorkflow.getObjectInfoForTypes(types); } catch {}
|
||||
// A newer render (another workflow loaded meanwhile) owns the column now
|
||||
if (seq !== _switchRenderSeq || workflow !== comfyUI.currentWorkflow || !col.isConnected) return;
|
||||
draw(_collectSwitchInputs(workflow, objectInfo));
|
||||
}
|
||||
|
||||
/** Sync comfyUI.currentWorkflow to Raw JSON textarea + highlight */
|
||||
function _syncRawJson() {
|
||||
const rawTextarea = document.getElementById("wfm-gen-raw-json");
|
||||
@@ -546,6 +646,7 @@ export const comfyEditor = {
|
||||
${nodeOpts}
|
||||
</select>
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-pos-revert" title="${t("revertPromptTitle")}">↺</button>
|
||||
${translateButtonsHtml("wfm-prompt-pos-text")}
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-pos-apply" title="Apply (Alt+Click: Apply & Generate)">Apply</button>
|
||||
</div>
|
||||
<textarea class="wfm-textarea" id="wfm-prompt-pos-text" rows="6">${positiveNodes[0]?.text || ""}</textarea>
|
||||
@@ -558,6 +659,7 @@ export const comfyEditor = {
|
||||
${nodeOpts}
|
||||
</select>
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-neg-revert" title="${t("revertPromptTitle")}">↺</button>
|
||||
${translateButtonsHtml("wfm-prompt-neg-text")}
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-prompt-neg-apply">Apply</button>
|
||||
</div>
|
||||
<textarea class="wfm-textarea" id="wfm-prompt-neg-text" rows="6">${negativeNodes[0]?.text || ""}</textarea>
|
||||
@@ -620,6 +722,9 @@ export const comfyEditor = {
|
||||
ta.value = wfNode.inputs[textKey] ?? "";
|
||||
});
|
||||
|
||||
// 🌐 native-language prompt translation (Settings → Prompt Translation)
|
||||
attachTranslateButtons(el);
|
||||
|
||||
// Track last focused prompt textarea for Paste button
|
||||
["wfm-prompt-pos-text", "wfm-prompt-neg-text"].forEach((taId) => {
|
||||
const ta = document.getElementById(taId);
|
||||
@@ -1514,7 +1619,7 @@ export const comfyEditor = {
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-settings-sampler-apply" title="Apply (Alt+Click: Apply & Generate)">Apply</button>
|
||||
` : "<p class='wfm-placeholder'>No KSampler node found</p>"}
|
||||
</div>
|
||||
<div style="flex:1;min-width:0;padding-left:14px;">
|
||||
<div style="flex:1;min-width:0;padding:0 14px;border-right:1px solid var(--wfm-border);">
|
||||
<h3 style="font-size:13px;font-weight:600;text-transform:uppercase;letter-spacing:0.5px;color:var(--wfm-text-secondary);margin:0 0 12px;">Latent Image</h3>
|
||||
${latent ? `
|
||||
<input type="hidden" id="wfm-settings-latent-id" value="${latent.id}">
|
||||
@@ -1559,9 +1664,15 @@ export const comfyEditor = {
|
||||
</div>
|
||||
` : "<p class='wfm-placeholder'>No EmptyLatentImage node found</p>"}
|
||||
</div>
|
||||
<div style="flex:1;min-width:0;padding-left:14px;">
|
||||
<h3 style="font-size:13px;font-weight:600;text-transform:uppercase;letter-spacing:0.5px;color:var(--wfm-text-secondary);margin:0 0 12px;">${t("genSwitchesTitle")}</h3>
|
||||
<div id="wfm-settings-switches-col"></div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
_renderSwitchColumn(document.getElementById("wfm-settings-switches-col"));
|
||||
|
||||
document.getElementById("wfm-settings-sampler-apply")?.addEventListener("click", (e) => {
|
||||
const ds = document.getElementById("wfm-settings-sampler-id")?.dataset || {};
|
||||
const write = (nodeId, key, value) => {
|
||||
|
||||
@@ -2,6 +2,11 @@
|
||||
* ComfyUI Workflow - Format detection, conversion, and analysis
|
||||
*/
|
||||
|
||||
// Video all-in-one nodes (MiniMaxH3ImageToVideo, MiniMaxH3ReferenceToVideo, ...): hold their own
|
||||
// "prompt" and build the conditioning + latent themselves. Matched by class-name pattern so other
|
||||
// models sharing this shape work without a code change.
|
||||
const VIDEO_ALL_IN_ONE_RE = /ImageToVideo|TextToVideo|ReferenceToVideo/;
|
||||
|
||||
// Cache for /object_info (loaded once, shared)
|
||||
let _objectInfoCache = null;
|
||||
|
||||
@@ -120,6 +125,12 @@ function _collectNodeTypes(workflow) {
|
||||
// frontend widgets that occupy a widgets_values slot; sent to the API as {"__value__": [...]}.
|
||||
const _WIDGET_TYPE_SET = new Set(["INT", "FLOAT", "STRING", "BOOLEAN", "COMBO", "COMFY_DYNAMICCOMBO_V3", "BOUNDING_BOXES"]);
|
||||
|
||||
// Required inputs backed by display-only frontend widgets (serialize=false, so no widgets_values
|
||||
// slot) -> the value the frontend sends for them in the API prompt.
|
||||
const _DISPLAY_WIDGET_DEFAULTS = {
|
||||
IMAGECOMPARE: () => ({ __value__: ["", ""] }),
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve a possibly-compound object_info type spec (e.g. LTXVEmptyLatentAudio's frame_rate:
|
||||
* "FLOAT,INT", a MultiType declaration meaning "widget defaults to FLOAT but also accepts an INT
|
||||
@@ -232,6 +243,13 @@ function _getDynamicComboSubNames(objectInfo, classType, widgetName, selectedKey
|
||||
];
|
||||
}
|
||||
|
||||
function _isDynamicComboKey(objectInfo, classType, widgetName, value) {
|
||||
const info = objectInfo[classType];
|
||||
const spec = info?.input?.required?.[widgetName] || info?.input?.optional?.[widgetName];
|
||||
const options = spec?.[1]?.options;
|
||||
return !Array.isArray(options) || options.some((o) => o.key === value);
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect if a widget value is a frontend-only extra (like "control_after_generate").
|
||||
* These values appear in widgets_values but are not in object_info inputs.
|
||||
@@ -366,9 +384,15 @@ function _normalizeLinkedWidgetValues(objectInfo, nodeType, widgetsValues, isLin
|
||||
const comboExpander = (name, selectedKey) => _getDynamicComboSubNames(objectInfo, nodeType, name, selectedKey);
|
||||
|
||||
// If skipping linked widgets already accounts for every entry, widgets_values is already in
|
||||
// the modern (no dangling linked entries) format — nothing to strip.
|
||||
// the modern (no dangling linked entries) format — nothing to strip. The count alone can match
|
||||
// by accident: TextGenerateV2 ["", 1536, "off", true, true, "auto"] with prompt linked reads
|
||||
// 1536 as sampling_mode, which falls back to the first option and swallows the rest. A selector
|
||||
// that doesn't hold one of its option keys means this reading is misaligned.
|
||||
const skipSim = _simulateWidgetValues(widgetNames, widgetTypes, widgetsValues, isLinkedName, true, comboExpander);
|
||||
if (skipSim.consumed === widgetsValues.length) return widgetsValues;
|
||||
const selectorsValid = widgetNames.every((name, i) => widgetTypes[i] !== "COMFY_DYNAMICCOMBO_V3"
|
||||
|| skipSim.positions[name] < 0
|
||||
|| _isDynamicComboKey(objectInfo, nodeType, name, widgetsValues[skipSim.positions[name]]));
|
||||
if (skipSim.consumed === widgetsValues.length && selectorsValid) return widgetsValues;
|
||||
|
||||
// Otherwise it's "legacy full": recompute with every widget present to find exactly which
|
||||
// indices the linked ones occupy — including any control_after_generate extra a linked
|
||||
@@ -980,6 +1004,15 @@ export const comfyWorkflow = {
|
||||
}
|
||||
}
|
||||
|
||||
// Display-only frontend widgets (ImageCompare's socketless compare_view) are never
|
||||
// stored in widgets_values, yet the backend requires them. The frontend sends the
|
||||
// widget's initial value, so do the same.
|
||||
for (const [k, spec] of Object.entries(objectInfo[node.type]?.input?.required || {})) {
|
||||
if (!(k in inputs) && _DISPLAY_WIDGET_DEFAULTS[spec?.[0]]) {
|
||||
inputs[k] = _DISPLAY_WIDGET_DEFAULTS[spec[0]]();
|
||||
}
|
||||
}
|
||||
|
||||
api[nodeId] = {
|
||||
class_type: node.type,
|
||||
inputs,
|
||||
@@ -1141,6 +1174,12 @@ export const comfyWorkflow = {
|
||||
if (Array.isArray(inputs.positive)) samplerPositiveRef[inputs.positive[0]] = true;
|
||||
if (Array.isArray(inputs.negative)) samplerNegativeRef[inputs.negative[0]] = true;
|
||||
}
|
||||
// Video all-in-one nodes whose "prompt" is wired from a separate text node (FastH3
|
||||
// OmniRef's MiniMaxH3ReferenceToVideo <- PrimitiveStringMultiline) — always positive
|
||||
// (no negative counterpart), so seed the role on the source directly.
|
||||
if (VIDEO_ALL_IN_ONE_RE.test(ct) && Array.isArray(inputs.prompt)) {
|
||||
samplerPositiveRef[inputs.prompt[0]] = true;
|
||||
}
|
||||
}
|
||||
|
||||
// === Pass 1b: Expand refs through CONDITIONING graph (BFS, 5 iterations) ===
|
||||
@@ -1259,7 +1298,11 @@ export const comfyWorkflow = {
|
||||
// TextEncodeQwenImage21 — fixed roles (the node feeds both guider inputs): a linked
|
||||
// "prompt" (prompt-enhancer switch chain, Qwen Image 2.1 Turbo) is the positive text,
|
||||
// a linked "negative_prompt" the negative.
|
||||
if (ct === "TextEncodeQwenImage21") {
|
||||
// TextEncodeFibo2 (Fibo 2.0 Edit) has the same shape, its prompt arriving via a
|
||||
// PreviewAny -> prompt-enhancer switch chain. PH3ImageConditioning (p-h3-image) and
|
||||
// MiniMaxH3ImageConditioning (MiniMax-H3-Image) too, without a negative prompt.
|
||||
if (ct === "TextEncodeQwenImage21" || ct === "TextEncodeFibo2" || ct === "PH3ImageConditioning"
|
||||
|| ct === "MiniMaxH3ImageConditioning") {
|
||||
if (Array.isArray(inputs.prompt)) samplerPositiveRef[inputs.prompt[0]] = true;
|
||||
if (Array.isArray(inputs.negative_prompt)) samplerNegativeRef[inputs.negative_prompt[0]] = true;
|
||||
}
|
||||
@@ -1504,7 +1547,12 @@ export const comfyWorkflow = {
|
||||
// with no images connected the node behaves as pure T2I). The reference images
|
||||
// themselves need no special handling here — they arrive as ordinary LoadImage
|
||||
// nodes already picked up generically by the load_image_nodes pass below.
|
||||
if (ct === "TextEncodeQwenImage21") {
|
||||
// TextEncodeFibo2 (Fibo 2.0) — same prompt/negative_prompt/images.image_N shape; its
|
||||
// negative_prompt is a structured VGL JSON caption (edited as plain text here).
|
||||
// PH3ImageConditioning (p-h3-image) and MiniMaxH3ImageConditioning (MiniMax-H3-Image) have
|
||||
// prompt and images.image_N, no negative_prompt.
|
||||
if (ct === "TextEncodeQwenImage21" || ct === "TextEncodeFibo2" || ct === "PH3ImageConditioning"
|
||||
|| ct === "MiniMaxH3ImageConditioning") {
|
||||
if (typeof inputs.prompt === "string") {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title: `${title} [positive]`, role: "positive",
|
||||
@@ -1612,11 +1660,11 @@ export const comfyWorkflow = {
|
||||
|
||||
// Video all-in-one nodes (e.g. MiniMaxH3ImageToVideo) — hold their own "prompt"
|
||||
// widget directly instead of going through a separate CLIPTextEncode, and have
|
||||
// no negative counterpart. Matched generically by class name (ImageToVideo /
|
||||
// TextToVideo) rather than a fixed list so future models sharing this pattern
|
||||
// no negative counterpart. Matched generically by class name (VIDEO_ALL_IN_ONE_RE)
|
||||
// rather than a fixed list so future models sharing this pattern
|
||||
// (WAN, HunyuanVideo, etc.) work without a code change — same detection metadata-tab.js
|
||||
// and node_sets_menu.js already use for prompt extraction (see extractPromptsAPI).
|
||||
if ((ct.includes("ImageToVideo") || ct.includes("TextToVideo")) && typeof inputs.prompt === "string") {
|
||||
if (VIDEO_ALL_IN_ONE_RE.test(ct) && typeof inputs.prompt === "string") {
|
||||
result.prompt_nodes.push({
|
||||
id, type: ct, title, role: "positive",
|
||||
text: inputs.prompt, textKey: "prompt",
|
||||
@@ -1686,7 +1734,11 @@ export const comfyWorkflow = {
|
||||
// width/height are commonly wired to a ResolutionSelector-style helper node
|
||||
// (array link ref) rather than a direct number — leave those as undefined so
|
||||
// the settings-tab UI falls back to its default instead of rendering a link ref.
|
||||
if (ct === "TextEncodeMageFlowEdit") {
|
||||
// TextEncodeFibo2 likewise (0 = Fibo 2.0's trained size for the first image's aspect),
|
||||
// PH3ImageConditioning (0 = image_1's size, or 2912 without images) and
|
||||
// MiniMaxH3ImageConditioning (0 = image_1's size, or 2048 without images).
|
||||
if (ct === "TextEncodeMageFlowEdit" || ct === "TextEncodeFibo2" || ct === "PH3ImageConditioning"
|
||||
|| ct === "MiniMaxH3ImageConditioning") {
|
||||
result.latent_nodes.push({
|
||||
id, type: ct, title,
|
||||
width: typeof inputs.width === "number" ? inputs.width : undefined,
|
||||
@@ -1864,7 +1916,7 @@ export const comfyWorkflow = {
|
||||
// generation timeout (video generation runs far longer than a still image) — see
|
||||
// generate-tab.js _coreGenerate and lab-tab.js _runLabBatch.
|
||||
isVideoWorkflow(analysis) {
|
||||
if ((analysis?.prompt_nodes || []).some((n) => /ImageToVideo|TextToVideo/.test(n.type))) return true;
|
||||
if ((analysis?.prompt_nodes || []).some((n) => VIDEO_ALL_IN_ONE_RE.test(n.type))) return true;
|
||||
// LTX-2.5/Wan2.2: no all-in-one *ToVideo node — detected instead via their
|
||||
// EmptyHunyuanLatentVideo/EmptyLTXVLatentVideo latent node (see analyzeWorkflow)
|
||||
// or a SaveVideo/CreateVideo node anywhere in the flattened graph.
|
||||
@@ -1911,7 +1963,8 @@ function _unifyPromptEnhancerText(api) {
|
||||
const tgRef = sw.inputs.on_true;
|
||||
if (!Array.isArray(tgRef)) continue;
|
||||
const tg = api[String(tgRef[0])];
|
||||
if (tg?.class_type !== "TextGenerate" || tg.inputs.prompt !== sw.inputs.on_false) continue;
|
||||
if ((tg?.class_type !== "TextGenerate" && tg?.class_type !== "TextGenerateV2")
|
||||
|| tg.inputs.prompt !== sw.inputs.on_false) continue;
|
||||
const primId = `${swId}:wfs_prompt`;
|
||||
api[primId] = {
|
||||
class_type: "PrimitiveStringMultiline",
|
||||
@@ -1959,6 +2012,9 @@ function _getWidgetMapping(nodeType) {
|
||||
ImpactWildcardEncode: ["wildcard_text"],
|
||||
WFS_PromptText: ["positive", "negative"],
|
||||
TextEncodeQwenImage21: ["prompt", "negative_prompt", "resolution"],
|
||||
TextEncodeFibo2: ["prompt", "negative_prompt", "width", "height", "batch_size"],
|
||||
PH3ImageConditioning: ["prompt", "width", "height"],
|
||||
MiniMaxH3ImageConditioning: ["prompt", "width", "height"],
|
||||
TextEncodeMingImageEdit: ["prompt"],
|
||||
EmptyQwenImageLayeredLatentImage: ["width", "height", "layers", "batch_size"],
|
||||
};
|
||||
|
||||
@@ -0,0 +1,431 @@
|
||||
/**
|
||||
* Gallery 3D viewer: glb / gltf / vrm / ply (mesh, point cloud, Gaussian splat) / spz.
|
||||
*
|
||||
* three.js is loaded the first time a 3D file is shown (static/lib/three-viewer).
|
||||
* The server has no 3D renderer, so thumbnails are rendered here and uploaded
|
||||
* (see ensureModelThumb); until then /thumb returns 404 and the card shows a placeholder.
|
||||
*/
|
||||
import { t } from "./i18n.js";
|
||||
|
||||
const LIB_URL = "/wfm_static/lib/three-viewer/three-viewer.js";
|
||||
const DRACO_PATH = "/wfm_static/lib/three-viewer/draco/";
|
||||
// Sentinel base for relative URIs in .gltf files; rewritten to the asset route on load.
|
||||
const ASSET_BASE = "/__wfm3d__/";
|
||||
const BACKGROUND = 0x181c26;
|
||||
const THUMB_SIZE = 512;
|
||||
const THUMB_TIMEOUT_MS = 60_000;
|
||||
// Larger files only get a thumbnail when opened, not while scrolling the grid.
|
||||
const AUTO_THUMB_MAX_BYTES = 128 * 1024 * 1024;
|
||||
// Splats are sorted asynchronously and spring bones settle over time, so keep drawing
|
||||
// for a while after each change instead of only once.
|
||||
const SETTLE_FRAMES = 60;
|
||||
|
||||
const MODEL3D_EXTENSIONS = [".glb", ".gltf", ".vrm", ".ply", ".spz"];
|
||||
|
||||
export function isModel3dFile(img) {
|
||||
const source = (img?.ext || img?.filename || img?.path || "").toLowerCase();
|
||||
return MODEL3D_EXTENSIONS.some((ext) => source.endsWith(ext));
|
||||
}
|
||||
|
||||
export function model3dExtLabel(img) {
|
||||
const source = (img?.ext || img?.filename || img?.path || "").toLowerCase();
|
||||
return (MODEL3D_EXTENSIONS.find((ext) => source.endsWith(ext)) || "").slice(1).toUpperCase();
|
||||
}
|
||||
|
||||
let _libPromise = null;
|
||||
function loadLib() {
|
||||
if (!_libPromise) {
|
||||
_libPromise = import(LIB_URL).catch((e) => {
|
||||
_libPromise = null;
|
||||
throw e;
|
||||
});
|
||||
}
|
||||
return _libPromise;
|
||||
}
|
||||
|
||||
const serveUrl = (path) => `/wfm/gallery/image/serve?path=${encodeURIComponent(path)}`;
|
||||
const assetUrl = (path, uri) => `/wfm/gallery/model3d/asset?path=${encodeURIComponent(path)}&uri=${encodeURIComponent(uri)}`;
|
||||
|
||||
async function fetchBytes(url, signal, onProgress) {
|
||||
const res = await fetch(url, { signal });
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const total = Number(res.headers.get("Content-Length")) || 0;
|
||||
if (!total || !res.body || !onProgress) return res.arrayBuffer();
|
||||
const out = new Uint8Array(total);
|
||||
const reader = res.body.getReader();
|
||||
let received = 0;
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
out.set(value, received);
|
||||
received += value.length;
|
||||
onProgress(received / total);
|
||||
}
|
||||
return out.buffer.slice(0, received);
|
||||
}
|
||||
|
||||
// 3DGS exports are .ply too; they are told apart by their per-splat properties.
|
||||
function isSplatPly(buffer) {
|
||||
const head = new TextDecoder("latin1").decode(new Uint8Array(buffer, 0, Math.min(buffer.byteLength, 8192)));
|
||||
const end = head.indexOf("end_header");
|
||||
const header = end >= 0 ? head.slice(0, end) : head;
|
||||
return /property\s+\w+\s+f_dc_0\b/.test(header) || /property\s+\w+\s+scale_0\b/.test(header);
|
||||
}
|
||||
|
||||
function countGeometry(root) {
|
||||
let vertices = 0;
|
||||
let triangles = 0;
|
||||
root.traverse((o) => {
|
||||
const g = o.geometry;
|
||||
if (!g?.attributes?.position) return;
|
||||
vertices += g.attributes.position.count;
|
||||
if (o.isMesh) triangles += (g.index ? g.index.count : g.attributes.position.count) / 3;
|
||||
});
|
||||
return { vertices, triangles: Math.round(triangles) };
|
||||
}
|
||||
|
||||
async function loadGltf(lib, img, buffer, isVrm) {
|
||||
const manager = new lib.THREE.LoadingManager();
|
||||
manager.setURLModifier((url) => (url.startsWith(ASSET_BASE) ? assetUrl(img.path, url.slice(ASSET_BASE.length)) : url));
|
||||
const loader = new lib.GLTFLoader(manager);
|
||||
const draco = new lib.DRACOLoader(manager).setDecoderPath(DRACO_PATH);
|
||||
loader.setDRACOLoader(draco);
|
||||
loader.setMeshoptDecoder(lib.MeshoptDecoder);
|
||||
if (isVrm) loader.register((parser) => new lib.VRMLoaderPlugin(parser));
|
||||
let gltf;
|
||||
try {
|
||||
gltf = await loader.parseAsync(buffer, ASSET_BASE);
|
||||
} finally {
|
||||
draco.dispose();
|
||||
}
|
||||
|
||||
const vrm = gltf.userData?.vrm;
|
||||
if (vrm) lib.VRMUtils.rotateVRM0(vrm); // VRM 0.x faces -Z
|
||||
const object = vrm ? vrm.scene : gltf.scene;
|
||||
const mixer = gltf.animations?.length ? new lib.THREE.AnimationMixer(object) : null;
|
||||
if (mixer) mixer.clipAction(gltf.animations[0]).play();
|
||||
return {
|
||||
object,
|
||||
animated: !!mixer,
|
||||
settles: !!vrm,
|
||||
update: (dt) => { mixer?.update(dt); vrm?.update(dt); },
|
||||
stats: countGeometry(object),
|
||||
};
|
||||
}
|
||||
|
||||
function loadPlyGeometry(lib, buffer) {
|
||||
const { THREE } = lib;
|
||||
const geometry = new lib.PLYLoader().parse(buffer);
|
||||
// "property double" (e.g. Open3D) comes back as a Float64Array, which WebGL rejects.
|
||||
for (const [name, attr] of Object.entries(geometry.attributes)) {
|
||||
if (attr.array instanceof Float64Array) {
|
||||
geometry.setAttribute(name, new THREE.BufferAttribute(Float32Array.from(attr.array), attr.itemSize, attr.normalized));
|
||||
}
|
||||
}
|
||||
const hasColor = !!geometry.attributes.color;
|
||||
if (!geometry.index) {
|
||||
geometry.computeBoundingSphere();
|
||||
const material = new THREE.PointsMaterial({
|
||||
size: geometry.boundingSphere.radius / 300,
|
||||
vertexColors: hasColor,
|
||||
color: hasColor ? 0xffffff : 0xb8c0cc,
|
||||
});
|
||||
const object = new THREE.Points(geometry, material);
|
||||
return { object, stats: countGeometry(object) };
|
||||
}
|
||||
if (!geometry.attributes.normal) geometry.computeVertexNormals();
|
||||
const material = new THREE.MeshStandardMaterial({
|
||||
vertexColors: hasColor,
|
||||
color: hasColor ? 0xffffff : 0xb8c0cc,
|
||||
roughness: 0.7,
|
||||
side: THREE.DoubleSide,
|
||||
});
|
||||
const object = new THREE.Mesh(geometry, material);
|
||||
return { object, stats: countGeometry(object) };
|
||||
}
|
||||
|
||||
async function loadSplat(lib, stage, img, buffer) {
|
||||
const { SparkRenderer, SplatMesh } = await lib.loadSpark();
|
||||
const spark = new SparkRenderer({ renderer: stage.renderer });
|
||||
stage.scene.add(spark);
|
||||
const mesh = new SplatMesh({ fileBytes: new Uint8Array(buffer), fileName: img.filename || img.path });
|
||||
await mesh.initialized;
|
||||
mesh.quaternion.set(1, 0, 0, 0); // splats are stored Y-down
|
||||
return {
|
||||
object: mesh,
|
||||
splat: true,
|
||||
settles: true,
|
||||
box: mesh.getBoundingBox(true),
|
||||
stats: { splats: mesh.packedSplats?.numSplats ?? mesh.numSplats ?? 0 },
|
||||
dispose: () => { mesh.dispose(); spark.dispose(); },
|
||||
};
|
||||
}
|
||||
|
||||
async function loadInto(lib, stage, img, signal, onProgress) {
|
||||
const buffer = await fetchBytes(serveUrl(img.path), signal, onProgress);
|
||||
const name = (img.filename || img.path).toLowerCase();
|
||||
if (name.endsWith(".spz")) return loadSplat(lib, stage, img, buffer);
|
||||
if (name.endsWith(".ply")) {
|
||||
return isSplatPly(buffer) ? loadSplat(lib, stage, img, buffer) : loadPlyGeometry(lib, buffer);
|
||||
}
|
||||
return loadGltf(lib, img, buffer, name.endsWith(".vrm"));
|
||||
}
|
||||
|
||||
function createStage(lib, canvas) {
|
||||
const { THREE } = lib;
|
||||
const renderer = new THREE.WebGLRenderer({ canvas, antialias: true, preserveDrawingBuffer: false });
|
||||
renderer.setPixelRatio(Math.min(window.devicePixelRatio || 1, 2));
|
||||
const scene = new THREE.Scene();
|
||||
scene.background = new THREE.Color(BACKGROUND);
|
||||
const pmrem = new THREE.PMREMGenerator(renderer);
|
||||
const envTarget = pmrem.fromScene(new lib.RoomEnvironment(), 0.04);
|
||||
pmrem.dispose();
|
||||
scene.environment = envTarget.texture;
|
||||
scene.environmentIntensity = 0.8;
|
||||
const sun = new THREE.DirectionalLight(0xffffff, 1.2);
|
||||
sun.position.set(2, 4, 3);
|
||||
scene.add(sun);
|
||||
const camera = new THREE.PerspectiveCamera(35, 1, 0.01, 1000);
|
||||
return { renderer, scene, camera, envTarget };
|
||||
}
|
||||
|
||||
function frameObject(lib, stage, model) {
|
||||
const { THREE } = lib;
|
||||
const { camera, scene } = stage;
|
||||
scene.add(model.object);
|
||||
model.object.updateMatrixWorld(true);
|
||||
const box = model.box
|
||||
? model.box.clone().applyMatrix4(model.object.matrixWorld)
|
||||
: new THREE.Box3().setFromObject(model.object);
|
||||
if (box.isEmpty()) box.set(new THREE.Vector3(-1, -1, -1), new THREE.Vector3(1, 1, 1));
|
||||
const center = box.getCenter(new THREE.Vector3());
|
||||
const radius = Math.max(box.getSize(new THREE.Vector3()).length() / 2, 1e-4);
|
||||
const distance = (radius / Math.sin(THREE.MathUtils.degToRad(camera.fov / 2))) * 1.05;
|
||||
camera.near = distance / 500;
|
||||
camera.far = distance * 50;
|
||||
camera.position.copy(center).add(new THREE.Vector3(0.35, 0.2, 1).normalize().multiplyScalar(distance));
|
||||
camera.lookAt(center);
|
||||
camera.updateProjectionMatrix();
|
||||
return center;
|
||||
}
|
||||
|
||||
function disposeStage(stage, model) {
|
||||
try { model?.dispose?.(); } catch { /* already gone */ }
|
||||
model?.object?.traverse?.((o) => {
|
||||
o.geometry?.dispose?.();
|
||||
for (const m of [].concat(o.material || [])) {
|
||||
for (const v of Object.values(m)) if (v?.isTexture) v.dispose();
|
||||
m.dispose?.();
|
||||
}
|
||||
});
|
||||
stage.envTarget.dispose();
|
||||
stage.renderer.dispose();
|
||||
stage.renderer.forceContextLoss(); // browsers only allow ~16 live contexts
|
||||
}
|
||||
|
||||
function formatStats(stats) {
|
||||
const n = (v) => Number(v).toLocaleString();
|
||||
if (stats.splats != null) return t("gallery3dStatsSplats", n(stats.splats));
|
||||
if (!stats.triangles) return t("gallery3dStatsPoints", n(stats.vertices));
|
||||
return t("gallery3dStatsMesh", n(stats.vertices), n(stats.triangles));
|
||||
}
|
||||
|
||||
// Center-crop the canvas to a square JPEG. Must run right after render() in the same task,
|
||||
// because the drawing buffer is not preserved.
|
||||
function captureJpeg(canvas) {
|
||||
const out = document.createElement("canvas");
|
||||
out.width = out.height = THUMB_SIZE;
|
||||
const side = Math.min(canvas.width, canvas.height);
|
||||
out.getContext("2d").drawImage(
|
||||
canvas, (canvas.width - side) / 2, (canvas.height - side) / 2, side, side, 0, 0, THUMB_SIZE, THUMB_SIZE,
|
||||
);
|
||||
return out.toDataURL("image/jpeg", 0.85);
|
||||
}
|
||||
|
||||
async function uploadThumb(path, image) {
|
||||
const res = await fetch("/wfm/gallery/image/thumb3d", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ path, image }),
|
||||
});
|
||||
return res.ok;
|
||||
}
|
||||
|
||||
/**
|
||||
* Interactive viewer inside `container`. It disposes itself once `container` leaves the
|
||||
* DOM, so callers can simply replace or remove the element.
|
||||
* `onReady(capture)` fires after the model is drawn; capture() returns a thumbnail data URL.
|
||||
*/
|
||||
export function mountModelViewer(container, img, { onReady } = {}) {
|
||||
container.classList.add("wfm-g3d");
|
||||
container.innerHTML = `
|
||||
<canvas class="wfm-g3d-canvas"></canvas>
|
||||
<div class="wfm-g3d-status">${t("gallery3dLoading")}</div>
|
||||
<div class="wfm-g3d-stats"></div>
|
||||
<button type="button" class="wfm-g3d-reset" title="${t("gallery3dResetView")}">⟲</button>`;
|
||||
const canvas = container.querySelector(".wfm-g3d-canvas");
|
||||
const status = container.querySelector(".wfm-g3d-status");
|
||||
const statsEl = container.querySelector(".wfm-g3d-stats");
|
||||
const resetBtn = container.querySelector(".wfm-g3d-reset");
|
||||
const abort = new AbortController();
|
||||
let stage = null;
|
||||
let model = null;
|
||||
let controls = null;
|
||||
let resizeObserver = null;
|
||||
let frameId = 0;
|
||||
let closed = false;
|
||||
|
||||
const close = () => {
|
||||
if (closed) return;
|
||||
closed = true;
|
||||
abort.abort();
|
||||
cancelAnimationFrame(frameId);
|
||||
resizeObserver?.disconnect();
|
||||
controls?.dispose();
|
||||
if (stage) disposeStage(stage, model);
|
||||
};
|
||||
|
||||
const fail = (e) => {
|
||||
if (closed || e?.name === "AbortError") return;
|
||||
console.error("[WFS] 3D viewer:", img.path, e);
|
||||
status.textContent = t("gallery3dLoadFailed");
|
||||
status.classList.add("wfm-g3d-status--error");
|
||||
status.style.display = "";
|
||||
close();
|
||||
};
|
||||
|
||||
(async () => {
|
||||
const lib = await loadLib();
|
||||
if (closed) return;
|
||||
stage = createStage(lib, canvas);
|
||||
const { renderer, scene, camera } = stage;
|
||||
model = await loadInto(lib, stage, img, abort.signal, (p) => {
|
||||
status.textContent = `${t("gallery3dLoading")} ${Math.round(p * 100)}%`;
|
||||
});
|
||||
if (closed) {
|
||||
disposeStage(stage, model);
|
||||
return;
|
||||
}
|
||||
const center = frameObject(lib, stage, model);
|
||||
const home = camera.position.clone();
|
||||
controls = new lib.OrbitControls(camera, canvas);
|
||||
controls.target.copy(center);
|
||||
controls.enableDamping = true;
|
||||
controls.update();
|
||||
|
||||
let dirty = SETTLE_FRAMES;
|
||||
const invalidate = () => { dirty = SETTLE_FRAMES; };
|
||||
controls.addEventListener("change", invalidate);
|
||||
resetBtn.addEventListener("click", () => {
|
||||
camera.position.copy(home);
|
||||
controls.target.copy(center);
|
||||
controls.update();
|
||||
invalidate();
|
||||
});
|
||||
const resize = () => {
|
||||
const w = container.clientWidth;
|
||||
const h = container.clientHeight;
|
||||
if (!w || !h) return;
|
||||
renderer.setSize(w, h, false);
|
||||
camera.aspect = w / h;
|
||||
camera.updateProjectionMatrix();
|
||||
invalidate();
|
||||
};
|
||||
resizeObserver = new ResizeObserver(resize);
|
||||
resizeObserver.observe(container);
|
||||
resize();
|
||||
|
||||
renderer.render(scene, camera); // surface a model that cannot be drawn as a load failure
|
||||
status.style.display = "none";
|
||||
statsEl.textContent = formatStats(model.stats);
|
||||
onReady?.(() => {
|
||||
renderer.render(scene, camera);
|
||||
return captureJpeg(canvas);
|
||||
});
|
||||
|
||||
let last = performance.now();
|
||||
const tick = () => {
|
||||
if (!container.isConnected) {
|
||||
close();
|
||||
return;
|
||||
}
|
||||
frameId = requestAnimationFrame(tick);
|
||||
const now = performance.now();
|
||||
const dt = (now - last) / 1000;
|
||||
last = now;
|
||||
if (!container.clientWidth) return; // hidden tab
|
||||
controls.update();
|
||||
if (!model.animated && dirty <= 0) return;
|
||||
dirty -= 1;
|
||||
try {
|
||||
model.update?.(dt);
|
||||
renderer.render(scene, camera);
|
||||
} catch (e) {
|
||||
fail(e);
|
||||
}
|
||||
};
|
||||
tick();
|
||||
})().catch(fail);
|
||||
|
||||
return { close };
|
||||
}
|
||||
|
||||
async function renderThumb(img) {
|
||||
const lib = await loadLib();
|
||||
const canvas = document.createElement("canvas");
|
||||
const stage = createStage(lib, canvas);
|
||||
stage.renderer.setPixelRatio(1);
|
||||
stage.renderer.setSize(THUMB_SIZE, THUMB_SIZE, false);
|
||||
const abort = new AbortController();
|
||||
const timer = setTimeout(() => abort.abort(), THUMB_TIMEOUT_MS);
|
||||
let model = null;
|
||||
try {
|
||||
model = await loadInto(lib, stage, img, abort.signal);
|
||||
frameObject(lib, stage, model);
|
||||
model.update?.(0);
|
||||
for (let i = 0; model.splat && i < SETTLE_FRAMES; i++) {
|
||||
stage.renderer.render(stage.scene, stage.camera);
|
||||
await new Promise((r) => requestAnimationFrame(r));
|
||||
}
|
||||
stage.renderer.render(stage.scene, stage.camera);
|
||||
return captureJpeg(canvas);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
disposeStage(stage, model);
|
||||
}
|
||||
}
|
||||
|
||||
const _thumbJobs = new Map(); // path -> Promise<boolean>, also remembers failures for this session
|
||||
let _thumbQueue = Promise.resolve();
|
||||
|
||||
/**
|
||||
* Render and store the thumbnail of a 3D file, one file at a time.
|
||||
* `isWanted()` is checked when the job's turn comes, so cards scrolled away or replaced by
|
||||
* a folder change cost nothing. Resolves true once the server has the thumbnail.
|
||||
*/
|
||||
export function ensureModelThumb(img, isWanted = () => true) {
|
||||
if (_thumbJobs.has(img.path)) return _thumbJobs.get(img.path);
|
||||
if (img.size > AUTO_THUMB_MAX_BYTES) return Promise.resolve(false);
|
||||
const job = _thumbQueue.then(async () => {
|
||||
if (!isWanted()) {
|
||||
_thumbJobs.delete(img.path);
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
return await uploadThumb(img.path, await renderThumb(img));
|
||||
} catch (e) {
|
||||
console.warn("[WFS] 3D thumbnail failed:", img.path, e);
|
||||
return false;
|
||||
}
|
||||
});
|
||||
_thumbJobs.set(img.path, job);
|
||||
_thumbQueue = job;
|
||||
return job;
|
||||
}
|
||||
|
||||
/** Store a thumbnail captured from an open viewer (covers files too large for ensureModelThumb). */
|
||||
export function saveModelThumb(img, capture) {
|
||||
const job = uploadThumb(img.path, capture()).catch(() => false);
|
||||
_thumbJobs.set(img.path, job);
|
||||
return job;
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Gallery Tab - セマンティック検索 (Gemini Embedding 2)
|
||||
* Gallery Tab - セマンティック検索 (Gemini Embedding 2 / Ollama の EmbeddingGemma 2)
|
||||
* - 検索ボックスのSemantic Searchトグル（ONでEnter実行の意味検索）
|
||||
* - 詳細パネルの「類似画像」トグル／更新ボタン
|
||||
* - インデックス作成モーダル（Semantic⚙）
|
||||
@@ -19,18 +19,20 @@ const API = {
|
||||
clear: "/wfm/gallery/semantic/clear",
|
||||
folders: (root) => `/wfm/gallery/semantic/folders?root=${encodeURIComponent(root)}`,
|
||||
test: "/wfm/gallery/semantic/test",
|
||||
options: "/wfm/gallery/semantic/options",
|
||||
};
|
||||
|
||||
const LS_ON = "wfm_gallery_semantic_on";
|
||||
const LS_DIM = "wfm_gallery_semantic_dim";
|
||||
const LS_MODEL = "wfm_gallery_semantic_model";
|
||||
const LS_SOFT = "wfm_gallery_semantic_soft";
|
||||
const GEMINI = "gemini-embedding-2";
|
||||
const LOCAL = "embeddinggemma-2";
|
||||
const LS_SUB = "wfm_gallery_semantic_recursive";
|
||||
const LS_TAB = "wfm_gallery_semantic_tab";
|
||||
const LS_TH = "wfm_gallery_semantic_th";
|
||||
const LIMITS = [50, 100, 200, 500, 1000];
|
||||
// Ollama のモデルサイズ(どちらも同じベクトルを返すので、切り替えても再作成は不要)
|
||||
const OLLAMA_TAGS = [["embeddinggemma-2:740m", "semOllamaTag740"], ["embeddinggemma-2:440m", "semOllamaTag440"]];
|
||||
|
||||
const sem = {
|
||||
on: false, // Semantic Searchトグル
|
||||
@@ -65,7 +67,8 @@ async function postJson(url, body) {
|
||||
/** APIエラーコードをi18nメッセージにする */
|
||||
export function semanticErrorMessage(e) {
|
||||
const known = ["no_api_key", "api_key_invalid", "index_empty", "dim_mismatch", "already_running",
|
||||
"folder_not_allowed", "unsupported_type", "network_error", "api_error", "local_error"];
|
||||
"folder_not_allowed", "unsupported_type", "network_error", "api_error", "local_error",
|
||||
"ollama_unreachable", "ollama_model_missing", "bad_url"];
|
||||
if (e?.code && known.includes(e.code)) {
|
||||
return t(`sem_err_${e.code}`, e.message);
|
||||
}
|
||||
@@ -244,23 +247,14 @@ function modalHtml() {
|
||||
<select id="wfm-sem-dim" class="wfm-select"></select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semDimHint"))}</span></div>
|
||||
<div id="wfm-sem-local-opts" class="wfm-sem-local-opts" style="display:none;">
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semDevice"))}</b>
|
||||
<select id="wfm-sem-device" class="wfm-select">
|
||||
<option value="auto">${escapeHtml(t("semDeviceAuto"))}</option>
|
||||
<option value="cpu">CPU</option>
|
||||
<option value="cuda">GPU (CUDA)</option>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semOllamaUrl"))}</b>
|
||||
<input type="text" id="wfm-sem-ollama-url" class="wfm-input wfm-sem-url" spellcheck="false" placeholder="http://localhost:11434"></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semOllamaTag"))}</b>
|
||||
<select id="wfm-sem-ollama-tag" class="wfm-select">
|
||||
${OLLAMA_TAGS.map(([tag, key]) => `<option value="${tag}">${escapeHtml(t(key))}</option>`).join("")}
|
||||
</select>
|
||||
<span id="wfm-sem-device-info" class="wfm-sem-hint"></span></div>
|
||||
<div class="wfm-sem-row" id="wfm-sem-threads-row"><b>${escapeHtml(t("semThreads"))}</b>
|
||||
<select id="wfm-sem-threads" class="wfm-select"></select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semThreadsHint"))}</span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semSoft"))}</b>
|
||||
<select id="wfm-sem-soft" class="wfm-select">
|
||||
<option value="280">${escapeHtml(t("semSoftHigh"))}</option>
|
||||
<option value="140">${escapeHtml(t("semSoftStd"))}</option>
|
||||
<option value="70">${escapeHtml(t("semSoftLight"))}</option>
|
||||
</select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semSoftHint"))}</span></div>
|
||||
<span id="wfm-sem-ollama-info" class="wfm-sem-hint"></span></div>
|
||||
<p class="wfm-sem-hint">${escapeHtml(t("semOllamaTagHint"))}</p>
|
||||
</div>
|
||||
<div class="wfm-sem-actions">
|
||||
<button class="wfm-btn" id="wfm-sem-test">${escapeHtml(t("semTest"))}</button>
|
||||
@@ -302,19 +296,12 @@ function selectedModel() {
|
||||
return document.querySelector('input[name="wfm-sem-model"]:checked')?.value || GEMINI;
|
||||
}
|
||||
|
||||
/** ローカルモデルの実行設定(デバイス/スレッド/画像精度)の初期値をサーバー保存値から設定する */
|
||||
/** Ollama の接続先/モデルサイズの初期値をサーバー保存値から設定する */
|
||||
function initLocalOpts(st) {
|
||||
const lo = st.local || {};
|
||||
const devSel = document.getElementById("wfm-sem-device");
|
||||
devSel.value = (lo.options && lo.options.device) || "auto";
|
||||
const cores = lo.devices?.cpu_count || 4;
|
||||
const cur = (lo.options && lo.options.threads) || lo.default_threads || Math.max(1, Math.floor(cores / 2));
|
||||
const set = new Set([1, 2, 4, 6, 8, 12, 16, 24, 32].filter(n => n <= cores).concat([cores, cur, lo.default_threads || 1]));
|
||||
document.getElementById("wfm-sem-threads").innerHTML = [...set].sort((a, b) => a - b)
|
||||
.map(n => `<option value="${n}">${n}${n === cores ? ` (${t("semThreadsAll")})` : ""}</option>`).join("");
|
||||
document.getElementById("wfm-sem-threads").value = String(cur);
|
||||
const soft = st.indexed_total && st.index_model === LOCAL ? st.index_soft : parseInt(lsGet(LS_SOFT, String(lo.default_soft || 280)), 10);
|
||||
document.getElementById("wfm-sem-soft").value = String([70, 140, 280].includes(soft) ? soft : 280);
|
||||
const ol = st.ollama || {};
|
||||
document.getElementById("wfm-sem-ollama-url").value = ol.url || "";
|
||||
const sel = document.getElementById("wfm-sem-ollama-tag");
|
||||
if (ol.tag) sel.value = ol.tag;
|
||||
}
|
||||
|
||||
/** 検索タブ: しきい値の入力欄にインデックスのモデル用の保存値を入れる（入力中の値は上書きしない） */
|
||||
@@ -359,20 +346,15 @@ function closeSemanticModal() {
|
||||
}
|
||||
}
|
||||
|
||||
/** 検出した実行デバイスの表示(GPU版onnxruntimeが無いときの案内を含む) */
|
||||
function updateDeviceInfo(lo) {
|
||||
const dev = lo?.devices;
|
||||
if (!dev) return;
|
||||
const info = document.getElementById("wfm-sem-device-info");
|
||||
const cudaOpt = document.querySelector('#wfm-sem-device option[value="cuda"]');
|
||||
if (cudaOpt) cudaOpt.disabled = !dev.cuda;
|
||||
if (!dev.cuda && document.getElementById("wfm-sem-device").value === "cuda") {
|
||||
document.getElementById("wfm-sem-device").value = "auto";
|
||||
}
|
||||
if (info) info.textContent = dev.cuda ? t("semDeviceCudaOk", dev.ort_version) : t("semDeviceNoGpu", dev.ort_version);
|
||||
const dv = document.getElementById("wfm-sem-device").value;
|
||||
const useCpu = dv === "cpu" || (dv === "auto" && !dev.cuda);
|
||||
document.getElementById("wfm-sem-threads-row").style.display = useCpu ? "" : "none";
|
||||
/** モデルサイズの選択肢に取得済みかどうかを表示する */
|
||||
function updateOllamaInfo(ol) {
|
||||
if (!ol) return;
|
||||
document.querySelectorAll("#wfm-sem-ollama-tag option").forEach(opt => {
|
||||
const key = OLLAMA_TAGS.find(([tag]) => tag === opt.value)?.[1];
|
||||
if (key) opt.textContent = t(key) + (ol.installed?.[opt.value] ? "" : ` ${t("semOllamaNotPulled")}`);
|
||||
});
|
||||
const info = document.getElementById("wfm-sem-ollama-info");
|
||||
if (info) info.textContent = ol.reachable && !ol.ready ? t("semOllamaPullHint", ol.tag) : "";
|
||||
}
|
||||
|
||||
/** 選択中モデルの次元選択肢を作り直す(preferredが選択肢にあればそれを選ぶ) */
|
||||
@@ -414,11 +396,12 @@ async function refreshModal() {
|
||||
fillThresholds(st);
|
||||
const model = selectedModel();
|
||||
document.getElementById("wfm-sem-local-opts").style.display = model === LOCAL ? "" : "none";
|
||||
updateDeviceInfo(st.local);
|
||||
const ready = model === LOCAL ? true : !!st.has_key;
|
||||
const keyLine = model === LOCAL
|
||||
? (_backends[LOCAL]?.downloaded ? t("semLocalReady") : t("semLocalNeedDownload"))
|
||||
: (st.has_key ? t("semKeyOk") : t("semKeyMissing"));
|
||||
updateOllamaInfo(st.ollama);
|
||||
const ol = st.ollama || {};
|
||||
const ready = model === LOCAL ? !!ol.ready : !!st.has_key;
|
||||
const keyLine = model !== LOCAL ? (st.has_key ? t("semKeyOk") : t("semKeyMissing"))
|
||||
: !ol.reachable ? t("semOllamaUnreachable", ol.url)
|
||||
: ol.ready ? t("semOllamaReady", ol.tag) : t("semOllamaMissing", ol.tag);
|
||||
const idxLine = t("semIndexInfo", st.indexed_total, st.index_dim || "-");
|
||||
const scopeLine = st.scope_total != null ? t("semScopeInfo", st.scope_indexed, st.scope_total) : "";
|
||||
document.getElementById("wfm-sem-status").innerHTML =
|
||||
@@ -437,7 +420,7 @@ async function refreshModal() {
|
||||
let text = t("semProgress", job.done || 0, total, job.failed || 0);
|
||||
if (job.running && job.current) text += ` ${job.current}`;
|
||||
if (!job.running) {
|
||||
if (["no_api_key", "api_key_invalid", "local_error"].includes(job.error_code)) text += ` ${t(`sem_err_${job.error_code}`, job.error)}`;
|
||||
if (["no_api_key", "api_key_invalid", "local_error", "ollama_unreachable", "ollama_model_missing"].includes(job.error_code)) text += ` ${t(`sem_err_${job.error_code}`, job.error)}`;
|
||||
else text += ` ${job.cancelled ? t("semCancelled") : t("semFinished")}`;
|
||||
if (job.failed && job.error) text += ` (${job.error})`;
|
||||
}
|
||||
@@ -477,13 +460,22 @@ function openSemanticModal() {
|
||||
}
|
||||
openModal(t("semModalTitle"), modalHtml());
|
||||
const dimOf = () => parseInt(document.getElementById("wfm-sem-dim").value, 10) || 0;
|
||||
// ローカルモデル選択時のみ実行設定を送る(サーバー側で保存され、検索時のクエリ埋め込みにも使われる)
|
||||
const localOpts = () => selectedModel() === LOCAL ? {
|
||||
device: document.getElementById("wfm-sem-device").value,
|
||||
threads: parseInt(document.getElementById("wfm-sem-threads").value, 10) || 0,
|
||||
soft: parseInt(document.getElementById("wfm-sem-soft").value, 10) || 280,
|
||||
} : {};
|
||||
document.getElementById("wfm-sem-device").addEventListener("change", () => refreshModal());
|
||||
// ローカルモデル選択時のみ Ollama の設定を送る(サーバー側で保存され、検索時のクエリ埋め込みにも使われる)
|
||||
const ollamaOpts = () => ({
|
||||
ollama_url: document.getElementById("wfm-sem-ollama-url").value.trim(),
|
||||
ollama_tag: document.getElementById("wfm-sem-ollama-tag").value,
|
||||
});
|
||||
const localOpts = () => selectedModel() === LOCAL ? ollamaOpts() : {};
|
||||
const saveOllamaOpts = async () => {
|
||||
try {
|
||||
await postJson(API.options, ollamaOpts());
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
refreshModal();
|
||||
};
|
||||
document.getElementById("wfm-sem-ollama-url").addEventListener("change", saveOllamaOpts);
|
||||
document.getElementById("wfm-sem-ollama-tag").addEventListener("change", saveOllamaOpts);
|
||||
document.querySelectorAll('input[name="wfm-sem-model"]').forEach(r => r.addEventListener("change", () => {
|
||||
fillDims(selectedModel(), 0);
|
||||
refreshModal();
|
||||
@@ -507,7 +499,6 @@ function openSemanticModal() {
|
||||
if (mode === "all" && !confirm(t("semConfirmAll"))) return;
|
||||
lsSet(LS_DIM, String(dimOf()));
|
||||
lsSet(LS_MODEL, selectedModel());
|
||||
lsSet(LS_SOFT, document.getElementById("wfm-sem-soft").value);
|
||||
try {
|
||||
await postJson(API.index, {
|
||||
folder: ctx.getFolder(),
|
||||
|
||||
+293
-106
@@ -15,6 +15,7 @@ import { callLLM, loadAiSettings, isValidBackendUrl } from "./ai-tab.js";
|
||||
import { initImagePromptTab, activateImagePromptTab } from "./image-prompt-tab.js";
|
||||
import { initStyleCatalogTab, activateStyleCatalogTab } from "./style-catalog-tab.js";
|
||||
import { applyStoredVideoVolume } from "./settings-tab.js";
|
||||
import { isModel3dFile, model3dExtLabel, mountModelViewer, ensureModelThumb, saveModelThumb } from "./gallery-3d-viewer.js";
|
||||
import { initSemantic, refreshSemanticDots, getSemanticMode, isSemanticOn, fetchSemanticImages, semanticCountLabel, setSemanticCountText, semanticErrorMessage, exitSimilarMode } from "./gallery-semantic.js";
|
||||
|
||||
// ── 定数 ─────────────────────────────────────────────────────
|
||||
@@ -80,9 +81,11 @@ export function isAudioFile(img) {
|
||||
}
|
||||
|
||||
// プレビュー/ライトボックス/比較用のメディア要素HTML。音声は波形画像＋<audio controls>。
|
||||
// 3Dは空の枠だけ返し、挿入後に mountModelViewers() がビューアを起動する。
|
||||
function mediaElementHtml(img, { imgClass, videoAttrs = "controls", audioAttrs = "controls", title = "" }) {
|
||||
const src = API.serveImage(img.path);
|
||||
const titleAttr = title ? ` title="${title}"` : "";
|
||||
if (isModel3dFile(img)) return `<div class="${imgClass} wfm-g3d-box" data-path="${escapeHtml(img.path)}"${titleAttr}></div>`;
|
||||
if (isVideoFile(img)) return `<video src="${src}" class="${imgClass}" ${videoAttrs}${titleAttr}></video>`;
|
||||
if (isAudioFile(img)) {
|
||||
return `<div class="wfm-gallery-audio-box">
|
||||
@@ -93,6 +96,56 @@ function mediaElementHtml(img, { imgClass, videoAttrs = "controls", audioAttrs =
|
||||
return `<img src="${src}" class="${imgClass}" alt="${escapeHtml(img.filename)}"${titleAttr}>`;
|
||||
}
|
||||
|
||||
// root内の3D枠でビューアを起動する。枠がDOMから外れるとビューアは自分で破棄される。
|
||||
function mountModelViewers(root, imgs) {
|
||||
root.querySelectorAll(".wfm-g3d-box").forEach((box) => {
|
||||
const img = imgs.find((im) => im.path === box.dataset.path);
|
||||
if (!img) return;
|
||||
mountModelViewer(box, img, {
|
||||
onReady: (capture) => {
|
||||
// グリッドの自動描画はサイズ上限があるため、開いたときにも保存しておく
|
||||
if (_modelThumbMissing.delete(img.path)) {
|
||||
saveModelThumb(img, capture).then((ok) => { if (ok) refreshModelThumb(img.path); });
|
||||
}
|
||||
},
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// サムネイルが無く、開いたときに保存する3Dファイルのパス
|
||||
const _modelThumbMissing = new Set();
|
||||
|
||||
function refreshModelThumb(path) {
|
||||
document.querySelectorAll(".wfm-gallery-thumb-img[data-model3d]").forEach((el) => {
|
||||
if (el.dataset.model3d === path) el.dispatchEvent(new CustomEvent("wfm-model3d-thumb"));
|
||||
});
|
||||
}
|
||||
|
||||
// サムネイル未作成の3Dファイル: プレースホルダーを出し、描画できたら差し替える
|
||||
function setupModelThumb(card, imgEl, img) {
|
||||
imgEl.dataset.model3d = img.path;
|
||||
const placeholder = document.createElement("div");
|
||||
placeholder.className = "wfm-gallery-thumb-placeholder";
|
||||
placeholder.textContent = model3dExtLabel(img);
|
||||
const show = () => {
|
||||
placeholder.remove();
|
||||
imgEl.style.display = "";
|
||||
imgEl.src = `${API.thumb(img.path)}&t=${Date.now()}`;
|
||||
};
|
||||
imgEl.addEventListener("wfm-model3d-thumb", show);
|
||||
imgEl.onerror = () => {
|
||||
if (placeholder.isConnected) return;
|
||||
imgEl.style.display = "none";
|
||||
card.insertBefore(placeholder, imgEl.nextSibling);
|
||||
_modelThumbMissing.add(img.path);
|
||||
ensureModelThumb(img, () => card.isConnected).then((ok) => {
|
||||
if (!ok) return;
|
||||
_modelThumbMissing.delete(img.path);
|
||||
refreshModelThumb(img.path);
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
// ComfyUI Comic Creater からiframe越しに画像を受け取り、Generate UIのImage入力スロットへ直接セットする（I2I連携）。
|
||||
// Comic Creater側の「I2Iへ送る」ボタンから
|
||||
// iframe.contentWindow._wfmReceiveImageForI2I(blob, name, workflowData?, workflowFilename?) として呼ばれる。
|
||||
@@ -176,6 +229,26 @@ window._wfmReceiveI2IRunRequest = async (imageBlob, params, workflowData, workfl
|
||||
}
|
||||
};
|
||||
|
||||
// 複数画像版のI2I実行（Comic Creater の Nanobanana+ タブ「Local」用）。
|
||||
// iframe.contentWindow._wfmReceiveMultiImageI2IRunRequest(imageBlobs, params, workflowData?, workflowFilename?)
|
||||
// imageBlobs[i] がワークフロー内 i 番目の LoadImage ノードへ割り当てられる。結果: { ok, url, slotCount, usedImages }
|
||||
window._wfmReceiveMultiImageI2IRunRequest = async (imageBlobs, params, workflowData, workflowFilename) => {
|
||||
try {
|
||||
if (workflowData) {
|
||||
if (!comfyUI.connected) {
|
||||
await comfyUI.checkConnection();
|
||||
}
|
||||
if (comfyUI.connected && (!comfyEditor.models.checkpoints || comfyEditor.models.checkpoints.length === 0)) {
|
||||
await comfyEditor.loadModelLists();
|
||||
}
|
||||
await loadWorkflowIntoEditor(workflowData, workflowFilename || "workflow.json");
|
||||
}
|
||||
return await window._wfmImageEditTab.runI2IExternalMulti(imageBlobs, params);
|
||||
} catch (e) {
|
||||
return { ok: false, message: e.message };
|
||||
}
|
||||
};
|
||||
|
||||
// ComfyUI Comic Creater からiframe越しにコマの設定（シーン・要素・セリフ等）を受け取り、
|
||||
// AIタブの設定タブと同じLLM接続設定（Ollama/LM Studio）で画像生成プロンプトの下書きを
|
||||
// 生成して返す（半自動マンガ作成の「L」ボタン連携用）。
|
||||
@@ -720,13 +793,16 @@ function createThumbCard(img) {
|
||||
placeholder.textContent = "?";
|
||||
card.insertBefore(placeholder, imgEl.nextSibling);
|
||||
};
|
||||
const isModel3d = isModel3dFile(img);
|
||||
if (isModel3d) setupModelThumb(card, imgEl, img);
|
||||
card.appendChild(imgEl);
|
||||
|
||||
// 動画ファイルは再生アイコンバッジを重ねて表示（サムネイル自体は先頭フレームのJPEG）
|
||||
if (isVideoFile(img) || isAudioFile(img)) {
|
||||
if (isVideoFile(img) || isAudioFile(img) || isModel3d) {
|
||||
const playBadge = document.createElement("div");
|
||||
playBadge.className = "wfm-gallery-thumb-video-badge";
|
||||
playBadge.textContent = isAudioFile(img) ? "♪" : "▶";
|
||||
playBadge.textContent = isModel3d ? "3D" : isAudioFile(img) ? "♪" : "▶";
|
||||
playBadge.classList.toggle("wfm-gallery-thumb-video-badge--corner", isModel3d);
|
||||
card.appendChild(playBadge);
|
||||
}
|
||||
|
||||
@@ -759,7 +835,7 @@ function createThumbCard(img) {
|
||||
e.stopPropagation();
|
||||
await toggleFeederGroupInPlace(img, feederBtn);
|
||||
});
|
||||
card.appendChild(feederBtn);
|
||||
if (!isModel3d) card.appendChild(feederBtn); // Feederノードは画像しか読めない
|
||||
|
||||
// クリック: 詳細表示 / Ctrl+クリック: 複数選択 / Shift+クリック: 範囲選択 / Alt+クリック: Metadataタブで開く
|
||||
card.addEventListener("click", (e) => {
|
||||
@@ -849,6 +925,18 @@ function createTable(images) {
|
||||
|
||||
const tdThumb = document.createElement("td");
|
||||
tdThumb.innerHTML = `<img src="${API.thumb(img.path, 128)}" class="wfm-gallery-table-thumb" loading="lazy" alt="">`;
|
||||
if (isModel3dFile(img)) {
|
||||
const thumbEl = tdThumb.firstElementChild;
|
||||
thumbEl.onerror = () => {
|
||||
thumbEl.style.visibility = "hidden";
|
||||
ensureModelThumb(img, () => thumbEl.isConnected).then((ok) => {
|
||||
if (!ok) return;
|
||||
thumbEl.onerror = null;
|
||||
thumbEl.style.visibility = "";
|
||||
thumbEl.src = `${API.thumb(img.path, 128)}&t=${Date.now()}`;
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
const tdName = document.createElement("td");
|
||||
tdName.className = "wfm-gallery-table-name";
|
||||
@@ -1054,8 +1142,9 @@ async function loadImageDetail(img) {
|
||||
${mediaElementHtml(img, { imgClass: "wfm-gallery-detail-img", title: "Double-click to enlarge" })}
|
||||
</div>
|
||||
`;
|
||||
const previewMediaEl = preview.querySelector(isVideo ? "video" : "img");
|
||||
const previewMediaEl = preview.querySelector(isModel3dFile(img) ? ".wfm-g3d-box" : isVideo ? "video" : "img");
|
||||
previewMediaEl.addEventListener("dblclick", () => openLightbox(img));
|
||||
mountModelViewers(preview, [img]);
|
||||
if (isVideo) applyStoredVideoVolume(previewMediaEl);
|
||||
if (isAudio) applyStoredVideoVolume(preview.querySelector("audio"));
|
||||
|
||||
@@ -1850,6 +1939,13 @@ function imageByPath(path) {
|
||||
return map.get(path);
|
||||
}
|
||||
|
||||
// 画像を前提とする送信先へ3Dファイルを渡さない
|
||||
function rejectModel3d() {
|
||||
if (!isModel3dFile(state.selectedImage)) return false;
|
||||
showToast(t("gallery3dNotImage"), "info");
|
||||
return true;
|
||||
}
|
||||
|
||||
function isConvertible(img) {
|
||||
return !!img && (isAudioFile(img) || isVideoFile(img));
|
||||
}
|
||||
@@ -1864,13 +1960,38 @@ function openConvertModal(paths) {
|
||||
showToast(t("convertNoTarget"), "info");
|
||||
return;
|
||||
}
|
||||
const hasVideo = targets.some(isVideoFile);
|
||||
openAudioConvertDialog({
|
||||
count: targets.length,
|
||||
hasVideo: targets.some(isVideoFile),
|
||||
convertOne: async (opts, i) => {
|
||||
const res = await fetch(API.convert, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ path: targets[i].path, ...opts }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!data.ok) throw new Error(data.error || `HTTP ${res.status}`);
|
||||
if (data.deleted) state.selectedImages.delete(targets[i].path);
|
||||
return data;
|
||||
},
|
||||
onDone: async () => {
|
||||
await loadImages();
|
||||
updateBulkBar();
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* 音声変換ダイアログ（Gallery・Video Edit共通）。
|
||||
* convertOne(opts, i): i件目を変換し、失敗時はthrowする。結果は {tags_skipped?} を含むオブジェクト。
|
||||
* allowDelete / allowCopyMeta=false の場合は該当チェックを出さず、optsにもfalseで渡す。
|
||||
*/
|
||||
export function openAudioConvertDialog({ count, hasVideo, convertOne, onDone, allowDelete = true, allowCopyMeta = true }) {
|
||||
const overlay = document.createElement("div");
|
||||
overlay.className = "wfm-gallery-lightbox";
|
||||
overlay.innerHTML = `
|
||||
<div class="wfm-gallery-move-modal wfm-gallery-convert-modal">
|
||||
<div class="wfm-gallery-move-modal-title">${escapeHtml(t("convertTitle", targets.length))}</div>
|
||||
<div class="wfm-gallery-move-modal-title">${escapeHtml(t("convertTitle", count))}</div>
|
||||
${hasVideo ? `<div class="wfm-gallery-convert-note">${escapeHtml(t("convertVideoNote"))}</div>` : ""}
|
||||
<label class="wfm-gallery-convert-row">${escapeHtml(t("convertFormat"))}
|
||||
<select id="wfm-convert-format" class="wfm-select">
|
||||
@@ -1895,8 +2016,8 @@ function openConvertModal(paths) {
|
||||
</select>
|
||||
</label>
|
||||
<label class="wfm-gallery-convert-check"><input type="checkbox" id="wfm-convert-inherit" checked> ${escapeHtml(t("convertInherit"))}</label>
|
||||
<label class="wfm-gallery-convert-check"><input type="checkbox" id="wfm-convert-copymeta" checked> ${escapeHtml(t("convertCopyMeta"))}</label>
|
||||
<label class="wfm-gallery-convert-check"><input type="checkbox" id="wfm-convert-delete"> ${escapeHtml(t("convertDeleteOriginal"))}</label>
|
||||
${allowCopyMeta ? `<label class="wfm-gallery-convert-check"><input type="checkbox" id="wfm-convert-copymeta" checked> ${escapeHtml(t("convertCopyMeta"))}</label>` : ""}
|
||||
${allowDelete ? `<label class="wfm-gallery-convert-check"><input type="checkbox" id="wfm-convert-delete"> ${escapeHtml(t("convertDeleteOriginal"))}</label>` : ""}
|
||||
<div class="wfm-gallery-convert-status" id="wfm-convert-status"></div>
|
||||
<div class="wfm-gallery-move-modal-footer">
|
||||
<button id="wfm-convert-run" class="wfm-btn wfm-btn-primary">${escapeHtml(t("convertRun"))}</button>
|
||||
@@ -1925,8 +2046,8 @@ function openConvertModal(paths) {
|
||||
q("#wfm-convert-cancel").addEventListener("click", () => { if (!running) overlay.remove(); });
|
||||
overlay.addEventListener("click", (e) => { if (e.target === overlay && !running) overlay.remove(); });
|
||||
q("#wfm-convert-run").addEventListener("click", async () => {
|
||||
const deleteOriginal = q("#wfm-convert-delete").checked;
|
||||
if (deleteOriginal && !window.confirm(t("convertConfirmDelete", targets.length))) return;
|
||||
const deleteOriginal = !!q("#wfm-convert-delete")?.checked;
|
||||
if (deleteOriginal && !window.confirm(t("convertConfirmDelete", count))) return;
|
||||
running = true;
|
||||
q("#wfm-convert-run").disabled = true;
|
||||
q("#wfm-convert-cancel").disabled = true;
|
||||
@@ -1936,33 +2057,25 @@ function openConvertModal(paths) {
|
||||
sample_rate: Number(q("#wfm-convert-rate").value),
|
||||
channels: q("#wfm-convert-channels").value,
|
||||
inherit_metadata: q("#wfm-convert-inherit").checked,
|
||||
copy_gallery_meta: q("#wfm-convert-copymeta").checked,
|
||||
copy_gallery_meta: !!q("#wfm-convert-copymeta")?.checked,
|
||||
delete_original: deleteOriginal,
|
||||
};
|
||||
let ok = 0, fail = 0, tagsSkipped = false;
|
||||
for (let i = 0; i < targets.length; i++) {
|
||||
q("#wfm-convert-status").textContent = t("convertProgress", i + 1, targets.length);
|
||||
for (let i = 0; i < count; i++) {
|
||||
q("#wfm-convert-status").textContent = t("convertProgress", i + 1, count);
|
||||
try {
|
||||
const res = await fetch(API.convert, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ path: targets[i].path, ...opts }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!data.ok) throw new Error(data.error || `HTTP ${res.status}`);
|
||||
const data = await convertOne(opts, i);
|
||||
ok++;
|
||||
if (data.tags_skipped) tagsSkipped = true;
|
||||
if (data.deleted) state.selectedImages.delete(targets[i].path);
|
||||
if (data?.tags_skipped) tagsSkipped = true;
|
||||
} catch (e) {
|
||||
fail++;
|
||||
console.warn("[Gallery] convert failed:", targets[i].filename, e);
|
||||
console.warn("[Gallery] convert failed:", i, e);
|
||||
}
|
||||
}
|
||||
overlay.remove();
|
||||
showToast(t("convertDone", ok, fail), fail ? "error" : "success");
|
||||
if (tagsSkipped && opts.inherit_metadata) showToast(t("convertTagsSkipped"), "info");
|
||||
await loadImages();
|
||||
updateBulkBar();
|
||||
await onDone?.({ ok, fail });
|
||||
});
|
||||
|
||||
document.body.appendChild(overlay);
|
||||
@@ -2067,6 +2180,7 @@ function openLightbox(img) {
|
||||
}
|
||||
});
|
||||
document.body.appendChild(overlay);
|
||||
mountModelViewers(overlay, [img]);
|
||||
if (isVideoFile(img)) applyStoredVideoVolume(overlay.querySelector("video"));
|
||||
if (isAudioFile(img)) applyStoredVideoVolume(overlay.querySelector("audio"));
|
||||
}
|
||||
@@ -2081,7 +2195,7 @@ function openCompare(paths) {
|
||||
overlay.className = "wfm-gallery-lightbox wfm-lightbox-compare";
|
||||
|
||||
const itemsHtml = imgs.map(img => {
|
||||
const mediaHtml = isAudioFile(img)
|
||||
const mediaHtml = isAudioFile(img) || isModel3dFile(img)
|
||||
? mediaElementHtml(img, { imgClass: "wfm-lightbox-compare-img" })
|
||||
: isVideoFile(img)
|
||||
? `<video src="${API.serveImage(img.path)}" class="wfm-lightbox-compare-img" controls></video>`
|
||||
@@ -2106,9 +2220,89 @@ function openCompare(paths) {
|
||||
}
|
||||
});
|
||||
document.body.appendChild(overlay);
|
||||
mountModelViewers(overlay, imgs);
|
||||
overlay.querySelectorAll("video, audio").forEach(applyStoredVideoVolume);
|
||||
}
|
||||
|
||||
// ── 他のタブから画像を表示する ────────────────────────────────
|
||||
|
||||
const _normPath = (p) => (p || "").replace(/\\/g, "/").replace(/\/+$/, "").toLowerCase();
|
||||
|
||||
/**
|
||||
* Output サブタブで path の画像を表示する(TagSpace の Gallery 点から呼ぶ)。
|
||||
* 絞り込み(検索・お気に入り・タグ・グループ・類似表示)を解除し、フォルダツリーを
|
||||
* 親から順に展開してフォルダを選び、一覧でその画像を選択して詳細を開く。
|
||||
* 戻り値: 表示できたら true(出力フォルダ外・一覧に無いときは false)
|
||||
*/
|
||||
export async function revealInGallery(path) {
|
||||
const wait = (ms) => new Promise((r) => setTimeout(r, ms));
|
||||
document.querySelector('.wfm-tab[data-tab="gallery"]')?.click(); // 初回はここで出力フォルダとツリーを読み込む
|
||||
document.querySelector('.wfm-gallery-subtab-btn[data-gallery-subtab="output"]')?.click();
|
||||
for (let i = 0; i < 100 && !state.folderTree; i++) await wait(100);
|
||||
if (!state.folderTree) return false;
|
||||
|
||||
const target = _normPath(path);
|
||||
const folder = target.slice(0, target.lastIndexOf("/"));
|
||||
// ツリーの根から目的のフォルダまでのノードを集める
|
||||
const chain = [];
|
||||
let node = state.folderTree;
|
||||
if (folder !== _normPath(node.abs_path) && !folder.startsWith(_normPath(node.abs_path) + "/")) return false;
|
||||
while (node) {
|
||||
chain.push(node);
|
||||
if (_normPath(node.abs_path) === folder) break;
|
||||
node = (node.children || []).find((c) => folder === _normPath(c.abs_path) || folder.startsWith(_normPath(c.abs_path) + "/"));
|
||||
}
|
||||
const leaf = chain[chain.length - 1];
|
||||
if (_normPath(leaf.abs_path) !== folder) return false;
|
||||
|
||||
// 絞り込みの解除(「Clear filters」ボタンと同じ。再読み込みは下で1回だけ行う)
|
||||
clearTimeout(_searchDebounceTimer);
|
||||
state.search = "";
|
||||
state.favoriteOnly = false;
|
||||
state.tagFilter = "";
|
||||
state.groupFilter = "";
|
||||
const searchInput = document.getElementById("wfm-gallery-search");
|
||||
if (searchInput) searchInput.value = "";
|
||||
const tagSel = document.getElementById("wfm-gallery-tag-filter");
|
||||
if (tagSel) tagSel.value = "";
|
||||
const groupSel = document.getElementById("wfm-gallery-group-filter");
|
||||
if (groupSel) groupSel.value = "";
|
||||
document.getElementById("wfm-gallery-fav-btn")?.classList.remove("active");
|
||||
exitSimilarMode();
|
||||
|
||||
// ツリーを親から順に展開(子ノードは展開時に描画される)
|
||||
const itemOf = (abs) => [...document.querySelectorAll(".wfm-gallery-tree-item")].find((el) => _normPath(el.dataset.path) === _normPath(abs));
|
||||
for (const n of chain.slice(0, -1)) {
|
||||
const arrow = itemOf(n.abs_path)?.querySelector(".wfm-gallery-tree-arrow");
|
||||
if (arrow && arrow.dataset.expanded !== "true" && arrow.style.visibility !== "hidden") arrow.click();
|
||||
}
|
||||
const item = itemOf(leaf.abs_path);
|
||||
document.querySelectorAll(".wfm-gallery-tree-item.selected").forEach((el) => el.classList.remove("selected"));
|
||||
item?.classList.add("selected");
|
||||
item?.scrollIntoView({ block: "nearest" });
|
||||
state.currentFolder = leaf.abs_path;
|
||||
state.selectedImages.clear();
|
||||
state.lastSelectionIndex = -1;
|
||||
updateBulkBar();
|
||||
await loadImages();
|
||||
|
||||
const idx = state.images.findIndex((img) => _normPath(img.path) === target);
|
||||
if (idx === -1) return false;
|
||||
// サムネイル表示は50件ずつ描くので、目的の画像まで描き足す
|
||||
const grid = document.getElementById("wfm-gallery-grid");
|
||||
if (state.viewMode === "thumb" && _renderedCount <= idx) {
|
||||
while (_renderedCount <= idx) _appendNextPage(grid);
|
||||
// 無限スクロールの目印を末尾へ戻す(全件描いたら外す)。_attachScrollSentinel と同じ扱い
|
||||
if (_renderedCount >= state.images.length) _disconnectScrollObserver();
|
||||
else { const s = document.getElementById("wfm-gallery-scroll-sentinel"); if (s) grid.appendChild(s); }
|
||||
}
|
||||
const el = [...grid.querySelectorAll("[data-path]")].find((e) => _normPath(e.dataset.path) === target);
|
||||
if (!el) return false;
|
||||
el.click(); // 通常のクリックと同じ選択・詳細表示
|
||||
el.scrollIntoView({ block: "center" });
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── outputパス取得 ────────────────────────────────────────────
|
||||
|
||||
async function detectOutputPath() {
|
||||
@@ -2191,6 +2385,7 @@ function bindEvents() {
|
||||
showToast("Please select an image first", "info");
|
||||
return;
|
||||
}
|
||||
if (rejectModel3d()) return;
|
||||
if (!window._wfmImageEditTab) return;
|
||||
const path = state.selectedImage.path;
|
||||
const name = (state.selectedImage.filename || "gallery-image").replace(/\.[^.]+$/, "");
|
||||
@@ -2220,6 +2415,7 @@ function bindEvents() {
|
||||
showToast(t("gallerySelectImageFirst"), "info");
|
||||
return;
|
||||
}
|
||||
if (rejectModel3d()) return;
|
||||
if (!comfyUI.currentAnalysis) {
|
||||
showToast(t("noWorkflowLoaded"), "info");
|
||||
return;
|
||||
@@ -2257,6 +2453,7 @@ function bindEvents() {
|
||||
showToast(t("gallerySelectImageFirst"), "info");
|
||||
return;
|
||||
}
|
||||
if (rejectModel3d()) return;
|
||||
if (!window.opener || typeof window.opener.wfmSendImageToSelectedNode !== "function") {
|
||||
showToast(t("sendToLiNodeNoOpener"), "error");
|
||||
return;
|
||||
@@ -2306,93 +2503,82 @@ function bindEvents() {
|
||||
}
|
||||
});
|
||||
|
||||
// 選択画像を ComfyUI Comic Creator の選択コマ／オーバーレイへ送信
|
||||
// （ComfyUI Comic CreaterからこのGalleryタブがiframe埋め込みされている場合のみ表示・動作する）
|
||||
// 選択画像を ComfyUI Comic Creator へ送信（ComfyUI Comic CreaterからこのGalleryタブがiframe埋め込みされている場合のみ表示・動作する）
|
||||
// Send CC Layout: レイアウトタブの選択コマ／オーバーレイへ（動画はComic Creatorの動画ツールへ）
|
||||
// Send CC Image : Imageタブへ（作業中のレイヤーがあれば新規レイヤー、無ければ新規キャンバス）。動画は未対応
|
||||
const sendCcBtn = document.getElementById("wfm-gallery-send-cc-btn");
|
||||
if (sendCcBtn) {
|
||||
let embedded = false;
|
||||
const sendCcImageBtn = document.getElementById("wfm-gallery-send-cc-image-btn");
|
||||
let ccEmbedded = false;
|
||||
try {
|
||||
ccEmbedded = !!(window.parent && window.parent !== window);
|
||||
} catch (e) {
|
||||
ccEmbedded = false;
|
||||
}
|
||||
if (ccEmbedded) {
|
||||
if (sendCcBtn) sendCcBtn.style.display = "";
|
||||
if (sendCcImageBtn) sendCcImageBtn.style.display = "";
|
||||
}
|
||||
|
||||
// window.parent のメンバーを安全に取り出す（クロスオリジン等で参照できなければnull）
|
||||
const getCcParentFn = (name) => {
|
||||
try {
|
||||
embedded = !!(window.parent && window.parent !== window);
|
||||
} catch (e) {
|
||||
embedded = false;
|
||||
if (window.parent && window.parent !== window && typeof window.parent[name] === "function") return window.parent[name];
|
||||
} catch (e) { /* ignore */ }
|
||||
return null;
|
||||
};
|
||||
|
||||
sendCcBtn?.addEventListener("click", () => {
|
||||
if (!state.selectedImage) {
|
||||
showToast(t("gallerySelectImageFirst"), "info");
|
||||
return;
|
||||
}
|
||||
if (embedded) sendCcBtn.style.display = "";
|
||||
const url = API.serveImage(state.selectedImage.path);
|
||||
|
||||
sendCcBtn.addEventListener("click", () => {
|
||||
if (!state.selectedImage) {
|
||||
showToast(t("gallerySelectImageFirst"), "info");
|
||||
return;
|
||||
}
|
||||
const url = API.serveImage(state.selectedImage.path);
|
||||
const isVideo = isVideoFile(state.selectedImage);
|
||||
|
||||
// I2I送信元がImageタブだった場合はそちらへ画像を読み込ませる（_ccI2ITargetModeはComic Creater側が
|
||||
// 「I2Iへ送る」ボタン押下時にセットするフラグ。undefined/'layout'なら従来通りコマ/オーバーレイへ挿入）
|
||||
let targetMode = null;
|
||||
try {
|
||||
targetMode = window.parent?._ccI2ITargetMode || null;
|
||||
} catch (e) {
|
||||
targetMode = null;
|
||||
}
|
||||
if (targetMode === "image") {
|
||||
// Imageタブは動画未対応（Comic Creator側の動画ツールはレイアウトタブのみ対応）
|
||||
if (isVideo) {
|
||||
showToast(t("galleryVideoNotSupportedInImageTab"), "info");
|
||||
return;
|
||||
}
|
||||
let imageTab = null;
|
||||
try {
|
||||
if (window.parent && window.parent !== window && typeof window.parent._ccImageTab?.loadFromUrl === "function") {
|
||||
imageTab = window.parent._ccImageTab;
|
||||
}
|
||||
} catch (e) {
|
||||
imageTab = null;
|
||||
}
|
||||
if (!imageTab) {
|
||||
showToast(t("galleryCCNotAvailable"), "info");
|
||||
return;
|
||||
}
|
||||
imageTab.loadFromUrl(url, (state.selectedImage.filename || "cc-image").replace(/\.[^.]+$/, ""));
|
||||
showToast(t("gallerySentCC"), "success");
|
||||
return;
|
||||
}
|
||||
|
||||
// レイアウトタブのコマ/オーバーレイへ（デフォルト経路）。動画（.mp4）はComic Creator側の
|
||||
// 動画ツール専用の受け口（insertVideoFromUrl）へ、それ以外は従来通り画像として送る
|
||||
if (isVideo) {
|
||||
let insertVideoFn = null;
|
||||
try {
|
||||
if (window.parent && window.parent !== window && typeof window.parent.insertVideoFromUrl === "function") {
|
||||
insertVideoFn = window.parent.insertVideoFromUrl;
|
||||
}
|
||||
} catch (e) {
|
||||
insertVideoFn = null;
|
||||
}
|
||||
if (!insertVideoFn) {
|
||||
showToast(t("galleryCCNotAvailable"), "info");
|
||||
return;
|
||||
}
|
||||
insertVideoFn(url, state.selectedImage.filename || "video.mp4");
|
||||
showToast(t("gallerySentCC"), "success");
|
||||
return;
|
||||
}
|
||||
|
||||
let insertFn = null;
|
||||
try {
|
||||
if (window.parent && window.parent !== window && typeof window.parent.insertImageFromUrl === "function") {
|
||||
insertFn = window.parent.insertImageFromUrl;
|
||||
}
|
||||
} catch (e) {
|
||||
insertFn = null;
|
||||
}
|
||||
if (!insertFn) {
|
||||
// 動画（.mp4）はComic Creator側の動画ツール専用の受け口（insertVideoFromUrl）へ、それ以外は画像として送る
|
||||
if (isVideoFile(state.selectedImage)) {
|
||||
const insertVideoFn = getCcParentFn("insertVideoFromUrl");
|
||||
if (!insertVideoFn) {
|
||||
showToast(t("galleryCCNotAvailable"), "info");
|
||||
return;
|
||||
}
|
||||
insertFn(url);
|
||||
insertVideoFn(url, state.selectedImage.filename || "video.mp4");
|
||||
showToast(t("gallerySentCC"), "success");
|
||||
});
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const insertFn = getCcParentFn("insertImageFromUrl");
|
||||
if (!insertFn) {
|
||||
showToast(t("galleryCCNotAvailable"), "info");
|
||||
return;
|
||||
}
|
||||
insertFn(url);
|
||||
showToast(t("gallerySentCC"), "success");
|
||||
});
|
||||
|
||||
sendCcImageBtn?.addEventListener("click", async () => {
|
||||
if (!state.selectedImage) {
|
||||
showToast(t("gallerySelectImageFirst"), "info");
|
||||
return;
|
||||
}
|
||||
// Imageタブは動画未対応
|
||||
if (isVideoFile(state.selectedImage)) {
|
||||
showToast(t("galleryVideoNotSupportedInImageTab"), "info");
|
||||
return;
|
||||
}
|
||||
const sendFn = getCcParentFn("sendImageToImageTab");
|
||||
if (!sendFn) {
|
||||
showToast(t("galleryCCNotAvailable"), "info");
|
||||
return;
|
||||
}
|
||||
const url = API.serveImage(state.selectedImage.path);
|
||||
try {
|
||||
const ok = await sendFn(url, (state.selectedImage.filename || "cc-image").replace(/\.[^.]+$/, ""));
|
||||
showToast(t(ok === false ? "galleryCCNotAvailable" : "gallerySentCC"), ok === false ? "info" : "success");
|
||||
} catch (e) {
|
||||
console.error("[gallery] Send CC Image failed", e);
|
||||
showToast(t("galleryCCNotAvailable"), "info");
|
||||
}
|
||||
});
|
||||
|
||||
// 検索（連続キー入力のたびにリクエストを飛ばすと、50枚程度のフォルダでも
|
||||
// ファイルI/Oの重いサーバー処理が積み重なって読み込みが遅延するためdebounceする）
|
||||
@@ -2580,7 +2766,7 @@ function bindEvents() {
|
||||
if (!window._wfmImageEditTab) return;
|
||||
|
||||
const allPaths = [...state.selectedImages];
|
||||
const paths = allPaths.filter(p => !isVideoFile({ path: p }) && !isAudioFile({ path: p }));
|
||||
const paths = allPaths.filter(p => !isVideoFile({ path: p }) && !isAudioFile({ path: p }) && !isModel3dFile({ path: p }));
|
||||
if (paths.length === 0) {
|
||||
showToast(t("editLayersNoImages"), "error");
|
||||
return;
|
||||
@@ -2641,6 +2827,7 @@ function bindEvents() {
|
||||
showToast(t("gallerySelectImageFirst"), "error");
|
||||
return;
|
||||
}
|
||||
if (rejectModel3d()) return;
|
||||
const { openImageInTaggerTab } = await import("./tagger-tab.js");
|
||||
openImageInTaggerTab(state.selectedImage);
|
||||
});
|
||||
|
||||
@@ -12,6 +12,7 @@ import { initFeederTab, refreshFeederNodeList } from "./feeder-tab.js";
|
||||
import { initLabTab, refreshLabLiveDefaults } from "./lab-tab.js";
|
||||
import { getSettings, readJsonStorage, escapeHtml, getEagleSettings, saveToEagle } from "./util.js";
|
||||
import { isVideoFilename } from "./video-utils.js";
|
||||
import { autoTranslateBeforeGenerate } from "./prompt-translate.js";
|
||||
|
||||
// ============================================
|
||||
// Gallery Metadata - ワークフロー保存
|
||||
@@ -2032,6 +2033,22 @@ async function handleGenerate(workflowOverride = null, genOptions = {}) {
|
||||
}
|
||||
if (comfyUI.generating) return;
|
||||
|
||||
// 設定「プロンプト翻訳」で生成前の自動翻訳がオンなら、Positive/Negative欄の自国語部分を英語へ置き換えてから同期する。
|
||||
// 失敗時は未翻訳のまま生成しないよう中断する。
|
||||
if (!workflowOverride) {
|
||||
const genBtnEl = document.getElementById("wfm-gen-generate-btn");
|
||||
if (genBtnEl) genBtnEl.disabled = true;
|
||||
try {
|
||||
await autoTranslateBeforeGenerate();
|
||||
} catch (err) {
|
||||
if (genBtnEl) genBtnEl.disabled = false;
|
||||
const progressText = document.getElementById("wfm-gen-progress-text");
|
||||
if (progressText) progressText.textContent = "";
|
||||
showToast(t("promptTranslateError", err.message), "error");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// オーバーライド実行時は GenerateUI 側の表示中プロンプトを別ワークフローへ誤って書き込まないよう同期をスキップする
|
||||
if (!workflowOverride) comfyEditor.syncToWorkflow();
|
||||
|
||||
@@ -2217,16 +2234,25 @@ export async function initGenerateTab() {
|
||||
comfyEditor.renderLoraPane(comfyUI.currentAnalysis, "wfm-gen-lora-fields");
|
||||
});
|
||||
|
||||
// Input inner tab (Prompt / Image)
|
||||
document.querySelectorAll(".wfm-input-inner-tab").forEach((btn) => {
|
||||
btn.addEventListener("click", () => {
|
||||
const target = btn.dataset.inputTab;
|
||||
document.querySelectorAll(".wfm-input-inner-tab").forEach((b) => b.classList.remove("active"));
|
||||
btn.classList.add("active");
|
||||
document.querySelectorAll(".wfm-input-inner-panel").forEach((p) => p.style.display = "none");
|
||||
document.getElementById(`wfm-input-panel-${target}`)?.style.setProperty("display", "");
|
||||
// Input tab: Raw JSON column collapse / expand (state remembered in localStorage)
|
||||
{
|
||||
const col = document.getElementById("wfm-gen-rawjson-col-input");
|
||||
const btn = document.getElementById("wfm-gen-rawjson-collapse-btn");
|
||||
const KEY = "wfm_gen_rawjson_collapsed";
|
||||
const apply = (collapsed) => {
|
||||
col?.classList.toggle("wfm-gen-rawjson-col--collapsed", collapsed);
|
||||
const icon = btn?.querySelector(".wfm-rawjson-collapse-icon");
|
||||
if (icon) icon.textContent = collapsed ? "«" : "»";
|
||||
};
|
||||
let stored = false;
|
||||
try { stored = localStorage.getItem(KEY) === "1"; } catch { /* ignore */ }
|
||||
apply(stored);
|
||||
btn?.addEventListener("click", () => {
|
||||
const next = !col.classList.contains("wfm-gen-rawjson-col--collapsed");
|
||||
apply(next);
|
||||
try { localStorage.setItem(KEY, next ? "1" : "0"); } catch { /* ignore */ }
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Initial placement: move to first active tab (input)
|
||||
moveRawJsonToTab("input");
|
||||
|
||||
+834
-117
File diff suppressed because one or more lines are too long
+144
-26
@@ -6,6 +6,7 @@
|
||||
|
||||
import { LayerManager, Layer } from "./image-edit/LayerManager.js";
|
||||
import { DrawTool } from "./image-edit/DrawTool.js";
|
||||
import { DrawPalette } from "./image-edit/DrawPalette.js";
|
||||
import { TextTool, TEXT_FONTS } from "./image-edit/TextTool.js";
|
||||
import { SelectTool } from "./image-edit/SelectTool.js";
|
||||
import { ShapeTool } from "./image-edit/ShapeTool.js";
|
||||
@@ -20,10 +21,22 @@ import { MaskEditorOneBridge } from "./image-edit/MaskEditorOneBridge.js";
|
||||
import { InpaintI2IActions } from "./image-edit/InpaintI2IActions.js";
|
||||
import { FileExport } from "./image-edit/FileExport.js";
|
||||
import { showToast } from "./app.js";
|
||||
import { t as tr } from "./i18n.js";
|
||||
import { comfyUI } from "./comfyui-client.js";
|
||||
import { comfyEditor } from "./comfyui-editor.js";
|
||||
import { comfyWorkflow } from "./comfyui-workflow.js";
|
||||
|
||||
// Draw Tool: スポイト（Eyedropper）アイコン。ボタンとカスタムカーソルの両方で使い回す。
|
||||
const EYEDROPPER_ICON_PATH = "M20.71 5.63l-2.34-2.34a1 1 0 0 0-1.41 0L14.7 5.55 12 8.24 3 17.24V21h3.76l9-9 2.69-2.69 2.26-2.26a1 1 0 0 0 0-1.41zM6.34 19H5v-1.34l8.06-8.06 1.41 1.41L6.34 19z";
|
||||
const EYEDROPPER_ICON_SVG =
|
||||
`<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="14" height="14" fill="currentColor"><path d="${EYEDROPPER_ICON_PATH}"/></svg>`;
|
||||
|
||||
function _eyedropperCursorCss() {
|
||||
const svg = `<svg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24'>`
|
||||
+ `<path d='${EYEDROPPER_ICON_PATH}' fill='%23ffffff' stroke='%23000000' stroke-width='1'/></svg>`;
|
||||
return `url("data:image/svg+xml,${svg}") 2 22, crosshair`;
|
||||
}
|
||||
|
||||
const TOOL_DEFS = [
|
||||
{ id: "select", icon: "▲", label: "Select", ready: true },
|
||||
{ id: "draw", icon: "✏", label: "Draw", ready: true },
|
||||
@@ -49,6 +62,11 @@ class ImageEditTab {
|
||||
this._layerMgr = null;
|
||||
this._activeTool = "select";
|
||||
this._drawTool = new DrawTool(null);
|
||||
this._eyedropperActive = false; // Drawツールのスポイト（色抽出）モード
|
||||
this._drawPalette = new DrawPalette({
|
||||
onColorChange: hex => this._setDrawColor(hex),
|
||||
toast: showToast,
|
||||
});
|
||||
this._textTool = new TextTool(null);
|
||||
this._selectTool = new SelectTool();
|
||||
this._zoom = 1.0;
|
||||
@@ -196,6 +214,7 @@ class ImageEditTab {
|
||||
_updateBrushCursor(e) {
|
||||
const el = this._brushCursorEl;
|
||||
if (!el) return;
|
||||
this._lastBrushCursorEvent = e; // キー操作でサイズを変えたときに再描画するため保持
|
||||
const tool = this._activeTool;
|
||||
const size = tool === "draw"
|
||||
? this._drawTool?.brushSize
|
||||
@@ -226,6 +245,79 @@ class ImageEditTab {
|
||||
if (this._brushCursorEl) this._brushCursorEl.style.display = "none";
|
||||
}
|
||||
|
||||
// ── Draw: スポイト（Eyedropper） ─────────────────
|
||||
|
||||
_setEyedropperActive(active) {
|
||||
this._eyedropperActive = active;
|
||||
const drawCanvas = document.getElementById("ie-canvas-draw");
|
||||
const overlay = document.getElementById("ie-canvas-overlay");
|
||||
const cursor = active ? _eyedropperCursorCss() : "";
|
||||
if (drawCanvas) drawCanvas.style.cursor = cursor;
|
||||
if (overlay) overlay.style.cursor = cursor;
|
||||
if (active) this._hideBrushCursor();
|
||||
document.getElementById("ie-draw-eyedropper")?.classList.toggle("ie-opt-active", active);
|
||||
}
|
||||
|
||||
_pickColorAt(x, y) {
|
||||
const canvas = document.getElementById("ie-canvas-draw");
|
||||
const px = Math.floor(x), py = Math.floor(y);
|
||||
if (!canvas || px < 0 || py < 0 || px >= canvas.width || py >= canvas.height) {
|
||||
this._setEyedropperActive(false);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
// 見えている色を拾うため、アクティブレイヤーのみの表示なら合成し直す
|
||||
if (!this._compositeMode) this._updateCompositeView();
|
||||
const [r, g, b] = canvas.getContext("2d").getImageData(px, py, 1, 1).data;
|
||||
const hex = "#" + [r, g, b].map(v => v.toString(16).padStart(2, "0")).join("");
|
||||
this._setDrawColor(hex);
|
||||
this._drawPalette.syncColor(hex);
|
||||
} catch {
|
||||
showToast(tr("imageEditPixelColorFail"), "error");
|
||||
}
|
||||
this._setEyedropperActive(false);
|
||||
}
|
||||
|
||||
_setDrawMode(mode) {
|
||||
this._drawTool.mode = mode;
|
||||
this._drawTool._stamp = null;
|
||||
this._renderDrawProps();
|
||||
}
|
||||
|
||||
_setDrawBrushSize(size) {
|
||||
this._drawTool.brushSize = size;
|
||||
this._drawTool._stamp = null;
|
||||
const slider = document.getElementById("ie-draw-size");
|
||||
if (slider) slider.value = size;
|
||||
const label = document.getElementById("ie-draw-size-lbl");
|
||||
if (label) label.textContent = size + "px";
|
||||
if (this._lastBrushCursorEvent && !this._eyedropperActive) {
|
||||
this._updateBrushCursor(this._lastBrushCursorEvent);
|
||||
}
|
||||
}
|
||||
|
||||
/** [ / ] キー: サイズが大きいほど刻みを粗くする */
|
||||
_stepDrawBrushSize(direction) {
|
||||
const size = this._drawTool.brushSize;
|
||||
const base = direction < 0 ? size - 1 : size;
|
||||
const step = base < 10 ? 1 : base < 50 ? 5 : 10;
|
||||
this._setDrawBrushSize(Math.min(200, Math.max(1, size + direction * step)));
|
||||
}
|
||||
|
||||
/** Drawツール中のBキー: ブラシ選択モーダルを開閉する */
|
||||
_toggleDrawBrushPicker() {
|
||||
if (this._abrAvailable) this._openAbrBrushPickerForDraw();
|
||||
else showToast(tr("imageEditNoAbrBrushes"), "info");
|
||||
}
|
||||
|
||||
_setDrawColor(hex) {
|
||||
this._drawTool.color = hex;
|
||||
this._drawTool._stamp = null;
|
||||
this._drawTool._imgStamp = null;
|
||||
const input = document.getElementById("ie-draw-color");
|
||||
if (input) input.value = hex;
|
||||
}
|
||||
|
||||
// ── ツールボタン ──────────────────────────────
|
||||
|
||||
_setupToolButtons() {
|
||||
@@ -240,6 +332,7 @@ class ImageEditTab {
|
||||
|
||||
_setActiveTool(toolId) {
|
||||
this._hideBrushCursor();
|
||||
if (this._eyedropperActive) this._setEyedropperActive(false);
|
||||
if (this._activeTool === "draw") this._drawTool?.deactivate();
|
||||
if (this._activeTool === "text") this._textTool?.deactivate();
|
||||
if (this._activeTool === "select") this._selectTool?.deactivate();
|
||||
@@ -306,6 +399,7 @@ class ImageEditTab {
|
||||
_renderToolOptions(toolId) {
|
||||
const el = document.getElementById("ie-tool-options");
|
||||
if (!el) return;
|
||||
this._drawPalette.closeMenu();
|
||||
el.innerHTML = "";
|
||||
|
||||
if (toolId === "select" && this._selectTool) {
|
||||
@@ -355,6 +449,7 @@ class ImageEditTab {
|
||||
|
||||
} else if (toolId === "draw" && this._drawTool) {
|
||||
el.innerHTML = "";
|
||||
this._drawPalette.render(el);
|
||||
this._renderDrawProps();
|
||||
|
||||
} else if (toolId === "fill" && this._fillTool) {
|
||||
@@ -873,18 +968,22 @@ class ImageEditTab {
|
||||
<div class="ie-props-row">
|
||||
<label>Mode</label>
|
||||
<div style="display:flex;gap:4px;">
|
||||
<button class="wfm-btn wfm-btn-sm${t.mode === "draw" ? " ie-opt-active" : ""}" id="ie-draw-mode-draw" style="flex:1;">Draw</button>
|
||||
<button class="wfm-btn wfm-btn-sm${t.mode === "erase" ? " ie-opt-active" : ""}" id="ie-draw-mode-erase" style="flex:1;">Erase</button>
|
||||
<button class="wfm-btn wfm-btn-sm${t.mode === "draw" ? " ie-opt-active" : ""}" id="ie-draw-mode-draw" title="Draw (D)" style="flex:1;">Draw</button>
|
||||
<button class="wfm-btn wfm-btn-sm${t.mode === "erase" ? " ie-opt-active" : ""}" id="ie-draw-mode-erase" title="Erase (E)" style="flex:1;">Erase</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="ie-props-row">
|
||||
<label>Color</label>
|
||||
<input type="color" id="ie-draw-color" value="${t.color}"
|
||||
style="width:36px;height:24px;padding:0;border:1px solid var(--wfm-border);cursor:pointer;border-radius:3px;flex-shrink:0;">
|
||||
<div style="display:flex;gap:4px;align-items:center;">
|
||||
<input type="color" id="ie-draw-color" value="${t.color}"
|
||||
style="width:36px;height:24px;padding:0;border:1px solid var(--wfm-border);cursor:pointer;border-radius:3px;flex-shrink:0;">
|
||||
<button class="wfm-btn wfm-btn-sm${this._eyedropperActive ? " ie-opt-active" : ""}" id="ie-draw-eyedropper"
|
||||
title="Eyedropper" style="flex-shrink:0;width:auto;height:24px;padding:2px 6px;display:flex;align-items:center;justify-content:center;">${EYEDROPPER_ICON_SVG}</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="ie-props-row">
|
||||
<label>Size</label>
|
||||
<input type="range" id="ie-draw-size" min="1" max="200" value="${t.brushSize}">
|
||||
<label title="[ / ]">Size</label>
|
||||
<input type="range" id="ie-draw-size" min="1" max="200" value="${t.brushSize}" title="[ / ]">
|
||||
<span id="ie-draw-size-lbl">${t.brushSize}px</span>
|
||||
</div>
|
||||
<div class="ie-props-row">
|
||||
@@ -906,7 +1005,7 @@ class ImageEditTab {
|
||||
<label>Brush</label>
|
||||
<span style="font-size:11px;color:var(--wfm-text-secondary);flex:1;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;"
|
||||
title="${t.brushName ?? "Circle"}">${t.brushName ?? "Circle"}</span>
|
||||
<button class="wfm-btn wfm-btn-sm" id="ie-draw-select-brush" style="font-size:10px;padding:1px 6px;flex-shrink:0;" ${this._abrAvailable ? "" : "disabled"}>Select</button>
|
||||
<button class="wfm-btn wfm-btn-sm" id="ie-draw-select-brush" title="B" style="font-size:10px;padding:1px 6px;flex-shrink:0;" ${this._abrAvailable ? "" : "disabled"}>Select</button>
|
||||
${t.brushImage ? `<button class="wfm-btn wfm-btn-sm" id="ie-draw-clear-brush" style="font-size:10px;padding:1px 5px;flex-shrink:0;">✕</button>` : ""}
|
||||
</div>
|
||||
${t.brushImage ? `
|
||||
@@ -933,25 +1032,17 @@ class ImageEditTab {
|
||||
</div>` : ""}
|
||||
`;
|
||||
|
||||
document.getElementById("ie-draw-mode-draw")?.addEventListener("click", () => {
|
||||
this._drawTool.mode = "draw";
|
||||
this._drawTool._stamp = null;
|
||||
this._renderDrawProps();
|
||||
});
|
||||
document.getElementById("ie-draw-mode-erase")?.addEventListener("click", () => {
|
||||
this._drawTool.mode = "erase";
|
||||
this._drawTool._stamp = null;
|
||||
this._renderDrawProps();
|
||||
});
|
||||
document.getElementById("ie-draw-mode-draw")?.addEventListener("click", () => this._setDrawMode("draw"));
|
||||
document.getElementById("ie-draw-mode-erase")?.addEventListener("click", () => this._setDrawMode("erase"));
|
||||
document.getElementById("ie-draw-color")?.addEventListener("input", e => {
|
||||
this._drawTool.color = e.target.value;
|
||||
this._drawTool._stamp = null;
|
||||
this._drawTool._imgStamp = null;
|
||||
this._setDrawColor(e.target.value);
|
||||
this._drawPalette.syncColor(e.target.value);
|
||||
});
|
||||
document.getElementById("ie-draw-eyedropper")?.addEventListener("click", () => {
|
||||
this._setEyedropperActive(!this._eyedropperActive);
|
||||
});
|
||||
document.getElementById("ie-draw-size")?.addEventListener("input", e => {
|
||||
this._drawTool.brushSize = parseInt(e.target.value);
|
||||
document.getElementById("ie-draw-size-lbl").textContent = e.target.value + "px";
|
||||
this._drawTool._stamp = null;
|
||||
this._setDrawBrushSize(parseInt(e.target.value));
|
||||
});
|
||||
document.getElementById("ie-draw-hard")?.addEventListener("input", e => {
|
||||
this._drawTool.hardness = parseInt(e.target.value) / 100;
|
||||
@@ -1345,6 +1436,10 @@ class ImageEditTab {
|
||||
}
|
||||
|
||||
// Comic Creator等、同一オリジンiframe越しの外部呼び出し専用エントリポイント（I2I版）。
|
||||
async runI2IExternalMulti(imageBlobs, opts) {
|
||||
return this._inpaint.runI2IExternalMulti(imageBlobs, opts);
|
||||
}
|
||||
|
||||
async runI2IExternal(imageBlob, opts) {
|
||||
return this._inpaint.runI2IExternal(imageBlob, opts);
|
||||
}
|
||||
@@ -1457,7 +1552,9 @@ class ImageEditTab {
|
||||
}
|
||||
// Draw/Mask: update brush cursor and continue stroke outside canvas
|
||||
const tool = this._activeTool;
|
||||
if (tool === "draw" || tool === "mask") {
|
||||
if (this._eyedropperActive) {
|
||||
this._hideBrushCursor();
|
||||
} else if (tool === "draw" || tool === "mask") {
|
||||
this._updateBrushCursor(e);
|
||||
const isDrawing = tool === "draw"
|
||||
? this._drawTool?._drawing
|
||||
@@ -1510,6 +1607,11 @@ class ImageEditTab {
|
||||
// PointerEvent.pressure: pen -> actual pressure, mouse -> 0.5 while a button is held.
|
||||
const pressure = e.pressure || 0.5;
|
||||
|
||||
if (this._eyedropperActive) {
|
||||
this._pickColorAt(pos.x, pos.y);
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._activeTool === "draw" && this._drawTool) {
|
||||
const activeLayer = this._layerMgr.activeLayer;
|
||||
if (!activeLayer) return;
|
||||
@@ -1777,6 +1879,7 @@ class ImageEditTab {
|
||||
|
||||
// 現在のドキュメントを閉じ、未保存の変更を破棄してプレースホルダー表示に戻す
|
||||
_closeDocument() {
|
||||
if (this._eyedropperActive) this._setEyedropperActive(false);
|
||||
if (!this._layerMgr) return;
|
||||
if (!confirm("Close the current document? Unsaved changes will be lost.")) return;
|
||||
|
||||
@@ -1844,6 +1947,8 @@ class ImageEditTab {
|
||||
this._layerMgr.on("change", () => this._refreshLayerList());
|
||||
|
||||
this._drawTool = new DrawTool(null);
|
||||
// 新規ドキュメントでも選択中の色を引き継ぐ（パレットを色の正とする）
|
||||
this._drawTool.color = this._drawPalette.currentColor;
|
||||
this._drawTool.onChange(() => {
|
||||
// layer.canvas に直接描くため sync 不要
|
||||
this._updateCompositeView();
|
||||
@@ -2454,6 +2559,9 @@ class ImageEditTab {
|
||||
const wrap = document.getElementById("ie-canvas-wrap");
|
||||
if (wrap && !this._panning) wrap.style.cursor = "grab";
|
||||
}
|
||||
if (e.key === "Escape" && this._eyedropperActive) this._setEyedropperActive(false);
|
||||
const brushPicker = document.getElementById("wfm-abr-picker-overlay");
|
||||
if (e.key === "Escape") brushPicker?.remove();
|
||||
if (e.ctrlKey && e.key === "z") { e.preventDefault(); this._undo(); }
|
||||
if (e.ctrlKey && e.key === "y") { e.preventDefault(); this._redo(); }
|
||||
if (this._activeTool === "mask" && this._maskSubtool === "vector") {
|
||||
@@ -2461,10 +2569,20 @@ class ImageEditTab {
|
||||
}
|
||||
if (!e.ctrlKey && !e.target.closest("input, textarea, select")) {
|
||||
if (e.key === "v") this._setActiveTool("select");
|
||||
if (e.key === "b") this._setActiveTool("draw");
|
||||
if (e.key === "b") {
|
||||
if (this._activeTool === "draw") this._toggleDrawBrushPicker();
|
||||
else this._setActiveTool("draw");
|
||||
}
|
||||
if (e.key === "t") this._setActiveTool("text");
|
||||
if (e.key === "s") this._setActiveTool("shape");
|
||||
if (e.key === "g") this._setActiveTool("fill");
|
||||
if (this._activeTool === "draw" && !brushPicker && !e.altKey && !e.metaKey) {
|
||||
if (e.key === "d") this._setDrawMode("draw");
|
||||
if (e.key === "e") this._setDrawMode("erase");
|
||||
if (e.key === "x") this._drawPalette.toggleWell();
|
||||
if (e.key === "[") this._stepDrawBrushSize(-1);
|
||||
if (e.key === "]") this._stepDrawBrushSize(1);
|
||||
}
|
||||
// Delete/Backspaceで選択オブジェクト削除
|
||||
if ((e.key === "Delete" || e.key === "Backspace") && this._activeTool === "select") {
|
||||
const layer = this._selectTool?.getSelectedLayer();
|
||||
@@ -2925,7 +3043,7 @@ class ImageEditTab {
|
||||
return null;
|
||||
};
|
||||
|
||||
const currentColor = this._drawTool?.color ?? "#ff0000";
|
||||
const currentColor = this._drawTool?.color ?? "#000000";
|
||||
const hex = currentColor.replace("#", "");
|
||||
const cr = parseInt(hex.slice(0, 2), 16);
|
||||
const cg = parseInt(hex.slice(2, 4), 16);
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
/**
|
||||
* Image Edit Tab - Draw Tool color palette
|
||||
* Drawツールの2段目（ie-tool-options）に並べるカラーパレット。
|
||||
* 色1/色2の切替、基本色、カスタム色の登録を担当する。
|
||||
* 基本色・カスタム色はユーザー設定として localStorage に保存する（作品データには含めない）。
|
||||
*/
|
||||
|
||||
import { t } from "../i18n.js";
|
||||
|
||||
const STORAGE_KEY = "ie_draw_palette_v1";
|
||||
const CUSTOM_SLOTS = 20;
|
||||
|
||||
const DEFAULT_BASE = [
|
||||
"#000000", "#7f7f7f", "#880015", "#ed1c24", "#ff7f27", "#fff200", "#22b14c", "#00a2e8", "#3f48cc", "#a349a4",
|
||||
"#ffffff", "#c3c3c3", "#b97a57", "#ffaec9", "#ffc90e", "#efe4b0", "#b5e61d", "#99d9ea", "#7092be", "#c8bfe7",
|
||||
];
|
||||
|
||||
function normalizeHex(value) {
|
||||
if (typeof value !== "string") return null;
|
||||
const hex = value.toLowerCase();
|
||||
return /^#[0-9a-f]{6}$/.test(hex) ? hex : null;
|
||||
}
|
||||
|
||||
function loadPalette() {
|
||||
let saved = null;
|
||||
try { saved = JSON.parse(localStorage.getItem(STORAGE_KEY)); } catch { /* 破損時は初期値 */ }
|
||||
return {
|
||||
base: DEFAULT_BASE.map((c, i) => normalizeHex(saved?.base?.[i]) ?? c),
|
||||
custom: Array.from({ length: CUSTOM_SLOTS }, (_, i) => normalizeHex(saved?.custom?.[i])),
|
||||
};
|
||||
}
|
||||
|
||||
export class DrawPalette {
|
||||
/**
|
||||
* @param {object} opts
|
||||
* @param {(hex: string) => void} opts.onColorChange ブラシ色を変更する
|
||||
* @param {(msg: string, type?: string) => void} opts.toast
|
||||
*/
|
||||
constructor({ onColorChange, toast }) {
|
||||
this._onColorChange = onColorChange;
|
||||
this._toast = toast;
|
||||
this._wells = ["#000000", "#ffffff"];
|
||||
this._activeWell = 0;
|
||||
Object.assign(this, loadPalette());
|
||||
this._content = null;
|
||||
this._picker = null;
|
||||
this._pickerTarget = null;
|
||||
this._menu = null;
|
||||
this._closeMenuOnOutside = e => { if (!this._menu?.contains(e.target)) this.closeMenu(); };
|
||||
this._closeMenuOnEscape = e => { if (e.key === "Escape") this.closeMenu(); };
|
||||
}
|
||||
|
||||
get currentColor() { return this._wells[this._activeWell]; }
|
||||
|
||||
/** ie-tool-options へパレットを描画する */
|
||||
render(container) {
|
||||
this.closeMenu();
|
||||
|
||||
const root = document.createElement("div");
|
||||
root.className = "ie-palette";
|
||||
this._content = document.createElement("div");
|
||||
this._content.className = "ie-palette-content";
|
||||
this._picker = document.createElement("input");
|
||||
this._picker.type = "color";
|
||||
this._picker.className = "ie-palette-picker";
|
||||
this._picker.tabIndex = -1;
|
||||
root.append(this._content, this._picker);
|
||||
container.appendChild(root);
|
||||
|
||||
root.addEventListener("click", e => this._onClick(e));
|
||||
root.addEventListener("contextmenu", e => this._onContextMenu(e));
|
||||
this._picker.addEventListener("input", () => this._onPickerInput());
|
||||
this._refresh();
|
||||
}
|
||||
|
||||
/** スポイト・カラー入力などパレット外で変わった色を反映する */
|
||||
syncColor(color) {
|
||||
const hex = normalizeHex(color);
|
||||
if (!hex) return;
|
||||
this._wells[this._activeWell] = hex;
|
||||
this._refresh();
|
||||
}
|
||||
|
||||
/** 色1/色2を切り替える */
|
||||
toggleWell() {
|
||||
this._selectWell(1 - this._activeWell);
|
||||
}
|
||||
|
||||
closeMenu() {
|
||||
if (!this._menu) return;
|
||||
this._menu.remove();
|
||||
this._menu = null;
|
||||
document.removeEventListener("pointerdown", this._closeMenuOnOutside, true);
|
||||
document.removeEventListener("keydown", this._closeMenuOnEscape, true);
|
||||
}
|
||||
|
||||
_refresh() {
|
||||
if (!this._content?.isConnected) return;
|
||||
const wells = this._wells.map((color, i) => {
|
||||
const label = t(i === 0 ? "imageEditPaletteColor1" : "imageEditPaletteColor2");
|
||||
return `<button type="button" class="ie-palette-well${i === this._activeWell ? " is-active" : ""}"
|
||||
data-well="${i}" style="--swatch:${color}" title="${label} (X): ${color}" aria-label="${label}"></button>`;
|
||||
}).join("");
|
||||
this._content.innerHTML = `
|
||||
<div class="ie-opt-group ie-palette-wells">${wells}</div>
|
||||
<div class="ie-opt-group">
|
||||
<label>${t("imageEditPaletteBase")}</label>
|
||||
<div class="ie-palette-grid">${this._swatches("base")}</div>
|
||||
</div>
|
||||
<div class="ie-opt-group">
|
||||
<label>${t("imageEditPaletteCustom")}</label>
|
||||
<div class="ie-palette-grid">${this._swatches("custom")}</div>
|
||||
<button type="button" class="wfm-btn wfm-btn-sm" data-palette-add
|
||||
title="${t("imageEditPaletteAddTitle")}">+</button>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
_swatches(kind) {
|
||||
return this[kind].map((color, i) => {
|
||||
const attrs = `data-kind="${kind}" data-index="${i}"`;
|
||||
if (!color) {
|
||||
const title = t("imageEditPaletteEmptyTitle");
|
||||
return `<button type="button" class="ie-palette-swatch is-empty" ${attrs} title="${title}" aria-label="${title}"></button>`;
|
||||
}
|
||||
const current = color === this.currentColor ? " is-current" : "";
|
||||
const title = `${color} — ${t("imageEditPaletteSwatchTitle")}`;
|
||||
return `<button type="button" class="ie-palette-swatch${current}" ${attrs}
|
||||
style="--swatch:${color}" title="${title}" aria-label="${color}"></button>`;
|
||||
}).join("");
|
||||
}
|
||||
|
||||
_onClick(e) {
|
||||
const well = e.target.closest("[data-well]");
|
||||
if (well) {
|
||||
this._selectWell(Number(well.dataset.well));
|
||||
return;
|
||||
}
|
||||
if (e.target.closest("[data-palette-add]")) {
|
||||
this._addCurrent();
|
||||
return;
|
||||
}
|
||||
const swatch = e.target.closest("[data-kind]");
|
||||
if (!swatch) return;
|
||||
const { kind, index } = swatch.dataset;
|
||||
const color = this[kind][index];
|
||||
if (color) this._applyColor(color);
|
||||
else this._setSlot(kind, index, this.currentColor);
|
||||
}
|
||||
|
||||
_onContextMenu(e) {
|
||||
const swatch = e.target.closest("[data-kind]");
|
||||
if (!swatch) return;
|
||||
e.preventDefault();
|
||||
this._openMenu(swatch, e.clientX, e.clientY);
|
||||
}
|
||||
|
||||
_selectWell(index) {
|
||||
this._activeWell = index;
|
||||
this._applyColor(this.currentColor);
|
||||
}
|
||||
|
||||
_applyColor(color) {
|
||||
this._wells[this._activeWell] = color;
|
||||
this._onColorChange(color);
|
||||
this._refresh();
|
||||
}
|
||||
|
||||
_addCurrent() {
|
||||
const color = this.currentColor;
|
||||
if (this.custom.includes(color)) {
|
||||
this._toast(t("imageEditPaletteExists"), "info");
|
||||
return;
|
||||
}
|
||||
const index = this.custom.indexOf(null);
|
||||
if (index < 0) {
|
||||
this._toast(t("imageEditPaletteFull"), "error");
|
||||
return;
|
||||
}
|
||||
this._setSlot("custom", index, color);
|
||||
}
|
||||
|
||||
_setSlot(kind, index, color) {
|
||||
this[kind][index] = color;
|
||||
try {
|
||||
localStorage.setItem(STORAGE_KEY, JSON.stringify({ base: this.base, custom: this.custom }));
|
||||
} catch { /* 保存できなくてもこのセッション中は使える */ }
|
||||
this._refresh();
|
||||
}
|
||||
|
||||
_openMenu(swatch, x, y) {
|
||||
this.closeMenu();
|
||||
const { kind, index } = swatch.dataset;
|
||||
const color = this[kind][index];
|
||||
const items = [
|
||||
["imageEditPaletteRegisterCurrent", () => this._setSlot(kind, index, this.currentColor)],
|
||||
[color ? "imageEditPaletteEditColor" : "imageEditPalettePickColor", () => this._openPicker(kind, index, x, y)],
|
||||
];
|
||||
if (kind === "custom" && color) {
|
||||
items.push(["imageEditPaletteDelete", () => this._setSlot(kind, index, null)]);
|
||||
}
|
||||
if (kind === "base" && color !== DEFAULT_BASE[index]) {
|
||||
items.push(["imageEditPaletteResetBase", () => this._setSlot(kind, index, DEFAULT_BASE[index])]);
|
||||
}
|
||||
|
||||
const menu = document.createElement("div");
|
||||
menu.className = "ie-palette-menu";
|
||||
for (const [key, action] of items) {
|
||||
const button = document.createElement("button");
|
||||
button.type = "button";
|
||||
button.textContent = t(key);
|
||||
button.addEventListener("click", () => { this.closeMenu(); action(); });
|
||||
menu.appendChild(button);
|
||||
}
|
||||
this._content.closest("#wfm-tab-image-edit").appendChild(menu);
|
||||
menu.style.left = `${Math.min(x, window.innerWidth - menu.offsetWidth - 4)}px`;
|
||||
menu.style.top = `${Math.min(y, window.innerHeight - menu.offsetHeight - 4)}px`;
|
||||
this._menu = menu;
|
||||
document.addEventListener("pointerdown", this._closeMenuOnOutside, true);
|
||||
document.addEventListener("keydown", this._closeMenuOnEscape, true);
|
||||
}
|
||||
|
||||
_openPicker(kind, index, x, y) {
|
||||
this._pickerTarget = { kind, index };
|
||||
this._picker.value = this[kind][index] ?? this.currentColor;
|
||||
this._picker.style.left = `${x}px`;
|
||||
this._picker.style.top = `${y}px`;
|
||||
this._picker.click();
|
||||
}
|
||||
|
||||
_onPickerInput() {
|
||||
const { kind, index } = this._pickerTarget;
|
||||
this._setSlot(kind, index, this._picker.value);
|
||||
this._applyColor(this._picker.value);
|
||||
}
|
||||
}
|
||||
@@ -12,7 +12,7 @@ export class DrawTool {
|
||||
this.brushSize = 20;
|
||||
this.hardness = 0.8; // 0.0 = very soft, 1.0 = hard edge
|
||||
this.opacity = 1.0;
|
||||
this.color = "#ff0000";
|
||||
this.color = "#000000";
|
||||
this.mode = "draw"; // "draw" | "erase"
|
||||
this.spacing = 0.25; // stamp interval as fraction of brushSize (image brush only)
|
||||
this.angle = 0; // degrees [0, 359]
|
||||
|
||||
@@ -287,6 +287,56 @@ export class InpaintI2IActions {
|
||||
return resultImg?.src || null;
|
||||
}
|
||||
|
||||
// 複数画像版: imageBlobs[i] をワークフロー内の i 番目の LoadImage ノードへ割り当てて実行する
|
||||
// （Comic Creator の Nanobanana+ タブ「Local」用。LoadImage 数を超える分は無視する）。
|
||||
// denoise は指定されたときだけ反映する（Image Edit系ワークフローはdenoiseを持たないことが多い）。
|
||||
// width/height/steps/cfg/seed は指定時のみ反映（未指定=ワークフロー既定のまま）。
|
||||
// seed が 0以上なら固定シード、未指定/負ならGenerateUIのシード設定に従う。
|
||||
async _runI2IWithImages(imageBlobs, { positive, negative, denoise, width, height, steps, cfg, sampler_name, scheduler, seed, overrideOpts = {}, onStatus } = {}) {
|
||||
const setStatus = onStatus || (() => {});
|
||||
|
||||
const analysisForLookup = overrideOpts.analysis || comfyUI.currentAnalysis;
|
||||
if (!analysisForLookup) throw new Error("No workflow loaded in GenerateUI");
|
||||
const slotCount = analysisForLookup.load_image_nodes?.length || 0;
|
||||
if (imageBlobs.length > 0 && slotCount === 0) {
|
||||
throw new Error("No LoadImage node found in the workflow");
|
||||
}
|
||||
|
||||
setStatus("Uploading...");
|
||||
const used = Math.min(imageBlobs.length, slotCount);
|
||||
for (let i = 0; i < used; i++) {
|
||||
const file = new File([imageBlobs[i]], `i2i_${Math.random().toString(36).slice(2, 10)}.png`, { type: "image/png" });
|
||||
await comfyEditor.applyImageToSlot(file, i, overrideOpts);
|
||||
}
|
||||
|
||||
comfyEditor.setPromptText("positive", positive, overrideOpts);
|
||||
if (negative) comfyEditor.setPromptText("negative", negative, overrideOpts);
|
||||
if (width != null || height != null || steps != null || cfg != null || denoise != null || sampler_name || scheduler) {
|
||||
comfyEditor.setGenerationParams({ width, height, steps, cfg, denoise, sampler_name, scheduler, ...overrideOpts });
|
||||
}
|
||||
|
||||
if (!window._wfmGenerateTab?.generate) throw new Error("GenerateUI is not ready yet");
|
||||
|
||||
setStatus("Generating...");
|
||||
const genOptions = (seed != null && seed >= 0) ? { seedValue: seed } : {};
|
||||
await window._wfmGenerateTab.generate(overrideOpts.workflow || undefined, genOptions);
|
||||
|
||||
const resultImg = document.getElementById("wfm-gen-result-img");
|
||||
return { url: resultImg?.src || null, slotCount, usedImages: used };
|
||||
}
|
||||
|
||||
async runI2IExternalMulti(imageBlobs, { positive = "", negative = "", denoise = null, width, height, steps, cfg, sampler_name, scheduler, seed } = {}) {
|
||||
if (this.i2iExternalRunning) throw new Error("I2I is already running");
|
||||
this.i2iExternalRunning = true;
|
||||
try {
|
||||
const r = await this._runI2IWithImages(imageBlobs || [], { positive, negative, denoise, width, height, steps, cfg, sampler_name, scheduler, seed });
|
||||
if (!r.url) throw new Error("No result image produced");
|
||||
return { ok: true, url: r.url, slotCount: r.slotCount, usedImages: r.usedImages };
|
||||
} finally {
|
||||
this.i2iExternalRunning = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Comic Creator等、同一オリジンiframe越しの外部呼び出し専用エントリポイント（I2I版）。
|
||||
// runExternal と対になる、マスク不要のシンプルなI2I実行。
|
||||
async runI2IExternal(imageBlob, { positive = "", negative = "", denoise = 1.0 } = {}) {
|
||||
|
||||
+47
-4
@@ -19,6 +19,7 @@ import { syncJsonHighlight, syncScroll } from "./json-highlight.js";
|
||||
import { _expandWildcardsInWorkflow } from "./generate-tab.js";
|
||||
import { pmGroups, getPresetsInGroup } from "./prompt-presets.js";
|
||||
import { isVideoFilename, extractLastFrameBlob } from "./video-utils.js";
|
||||
import { translateButtonsHtml, attachTranslateButtons, isAutoTranslateOn, translateWorkflowPrompts } from "./prompt-translate.js";
|
||||
|
||||
// Checkpoint/LoRA/VAE are three fully independent keyframe timelines — same as
|
||||
// Prompt/KSampler — each with its own +/-, its own atIteration sequence, its own
|
||||
@@ -167,6 +168,15 @@ export function refreshLabLiveDefaults() {
|
||||
_renderAllColumns();
|
||||
}
|
||||
|
||||
// Loads plan data built elsewhere (AI TOOL Chat's ```labplan block → Storyboard) into the Lab
|
||||
// tab as an unsaved plan, then switches to GenerateUI > Lab so the user can review and Run it.
|
||||
// The user saves it with Lab's own Save button (which also builds the index thumbnail).
|
||||
export function openLabPlanData(data) {
|
||||
_applyPlanData(null, data);
|
||||
document.querySelector('.wfm-tab[data-tab="generate"]')?.click();
|
||||
document.querySelector('.wfm-gen-subtab-btn[data-subtab="lab"]')?.click();
|
||||
}
|
||||
|
||||
function _initDropZone() {
|
||||
const dropZone = document.getElementById("wfm-lab-drop-zone");
|
||||
const fileInput = document.getElementById("wfm-lab-file-input");
|
||||
@@ -679,12 +689,18 @@ function _cellLabel(col, kf) {
|
||||
return `<span class="wfm-lab-cell-prompt-line">P: ${escapeHtml(p)}</span><span class="wfm-lab-cell-prompt-line">N: ${escapeHtml(n)}</span>`;
|
||||
}
|
||||
if (col === "ksampler") {
|
||||
// Same six fields _isEmptyValue checks — a keyframe that only sets the seed (e.g. a
|
||||
// Storyboard plan's fixed seed) or the sampler/scheduler used to read "(workflow
|
||||
// default)" even though Run does apply it.
|
||||
const v = kf.value || {};
|
||||
if (v.steps == null && v.cfg == null && v.denoise == null) return t("labUseWorkflowDefault");
|
||||
const parts = [];
|
||||
if (v.steps != null) parts.push(`Step${v.steps}`);
|
||||
if (v.cfg != null) parts.push(`CFG${v.cfg}`);
|
||||
if (v.denoise != null) parts.push(`Denoise${v.denoise}`);
|
||||
if (v.sampler_name) parts.push(v.sampler_name);
|
||||
if (v.scheduler) parts.push(v.scheduler);
|
||||
if (v.seed != null) parts.push(`Seed ${v.seed}`);
|
||||
if (!parts.length) return t("labUseWorkflowDefault");
|
||||
return escapeHtml(parts.join(" / "));
|
||||
}
|
||||
return "";
|
||||
@@ -819,11 +835,17 @@ function _openCellModal(col, idx) {
|
||||
<button type="button" id="wfm-lab-modal-clear" class="wfm-btn wfm-btn-sm">${t("labClearPrompt")}</button>
|
||||
</div>
|
||||
<div class="wfm-lab-modal-row">
|
||||
<label>Positive</label>
|
||||
<div style="display:flex;align-items:center;gap:6px;">
|
||||
<label style="flex:1;">Positive</label>
|
||||
${translateButtonsHtml("wfm-lab-modal-positive")}
|
||||
</div>
|
||||
<textarea id="wfm-lab-modal-positive" class="wfm-input" rows="3">${escapeHtml(v.positive || "")}</textarea>
|
||||
</div>
|
||||
<div class="wfm-lab-modal-row">
|
||||
<label>Negative</label>
|
||||
<div style="display:flex;align-items:center;gap:6px;">
|
||||
<label style="flex:1;">Negative</label>
|
||||
${translateButtonsHtml("wfm-lab-modal-negative")}
|
||||
</div>
|
||||
<textarea id="wfm-lab-modal-negative" class="wfm-input" rows="3">${escapeHtml(v.negative || "")}</textarea>
|
||||
</div>
|
||||
<div class="wfm-lab-modal-checkbox">
|
||||
@@ -900,7 +922,10 @@ function _openCellModal(col, idx) {
|
||||
<button id="wfm-lab-modal-save" class="wfm-btn wfm-btn-primary">${t("save")}</button>
|
||||
</div>`;
|
||||
openModal(`${col} #${idx + 1}`, html);
|
||||
if (col === "prompt") _wireLabPromptModalExtras(idx);
|
||||
if (col === "prompt") {
|
||||
_wireLabPromptModalExtras(idx);
|
||||
attachTranslateButtons(document.getElementById("wfm-lab-modal-fields") || document);
|
||||
}
|
||||
if (col === "checkpoint" || col === "vae") _wireLabValueFilter(col);
|
||||
if (col === "lora") _wireLabLoraFilter();
|
||||
|
||||
@@ -1437,6 +1462,10 @@ async function _runLabBatch() {
|
||||
// mid-batch (see comfyWorkflow.isVideoWorkflow and generate-tab.js's own use of this).
|
||||
const isVideoWf = comfyWorkflow.isVideoWorkflow(comfyUI.currentAnalysis);
|
||||
|
||||
// Settings -> Prompt Translation -> "Lab Run": translate each iteration's prompts
|
||||
// (the plan itself keeps the original text).
|
||||
const autoTranslate = await isAutoTranslateOn("lab");
|
||||
|
||||
let completed = 0, failed = 0;
|
||||
let prevLoraInjection = null;
|
||||
try {
|
||||
@@ -1450,6 +1479,20 @@ async function _runLabBatch() {
|
||||
const { workflow, seed, loraInjection } = _buildWorkflowForIteration(i, chainRef, prevLoraInjection);
|
||||
prevLoraInjection = loraInjection;
|
||||
const workflowExpanded = await _expandWildcardsInWorkflow(workflow);
|
||||
if (autoTranslate) {
|
||||
try {
|
||||
if (progressText) progressText.textContent = `[${i}/${_lab.batchCount}] ${t("promptTranslateRunning")}`;
|
||||
await translateWorkflowPrompts(workflowExpanded, comfyUI.currentAnalysis.prompt_nodes);
|
||||
} catch (err) {
|
||||
// Do not send untranslated text - stop the batch like GenerateUI does.
|
||||
// The end-of-batch "stopped" toast replaces this one, so also keep the
|
||||
// reason in the progress line.
|
||||
showToast(t("promptTranslateError", err.message), "error");
|
||||
if (progressText) progressText.textContent = t("promptTranslateError", err.message);
|
||||
_aborted = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const { images } = await comfyUI.generate(workflowExpanded, {
|
||||
|
||||
@@ -624,9 +624,12 @@ function extractPromptsAPI(wf) {
|
||||
// 自ノードの"prompt"入力に直接テキストを持つ(positive/negativeの区別も無い)ため、
|
||||
// isTextEncoderNode判定を通さずtextMapへ直接投入する(fallbackのtextsに載る)。
|
||||
// Gemini系の画像生成ノード(Nano Banana等)も同様に自ノードの"prompt"入力を持つ。
|
||||
if (typeof n.inputs?.prompt === "string" && /ImageToVideo|TextToVideo|^Gemini(Image|NanoBanana)/i.test(ct)) {
|
||||
textMap.set(id, n.inputs.prompt);
|
||||
continue;
|
||||
// FastH3 OmniRef(MiniMaxH3ReferenceToVideo)のように"prompt"が別ノードから配線される場合はリンク先を解決する。
|
||||
if (/ImageToVideo|TextToVideo|ReferenceToVideo|^Gemini(Image|NanoBanana)/i.test(ct)) {
|
||||
const pv = n.inputs?.prompt;
|
||||
const txt = typeof pv === "string" ? pv : Array.isArray(pv) ? resolveLinkedText(wf, pv[0], pv[1] ?? 0) : null;
|
||||
if (txt) textMap.set(id, txt);
|
||||
if (pv !== undefined) continue;
|
||||
}
|
||||
if (!isTextEncoderNode(ct)) continue;
|
||||
// "prompt" — TextEncodeQwenImageEdit(Plus)/TextEncodeBooguEdit and similar Image Edit
|
||||
@@ -667,8 +670,12 @@ function extractPromptsAPI(wf) {
|
||||
// negative_prompt directly, so textMap (single text per node) can't represent both
|
||||
// roles.
|
||||
const srcNode = wf[String(val[0])];
|
||||
if (srcNode?.class_type === "TextEncodeMageFlowEdit" || srcNode?.class_type === "TextEncodeBooguEdit") {
|
||||
const text = isPos ? srcNode.inputs?.prompt : srcNode.inputs?.negative_prompt;
|
||||
// TextEncodeFibo2(Fibo 2.0)も同形。promptはPreviewAny→プロンプト強化スイッチ経由でリンクされる。
|
||||
if (srcNode?.class_type === "TextEncodeMageFlowEdit" || srcNode?.class_type === "TextEncodeBooguEdit"
|
||||
|| srcNode?.class_type === "TextEncodeFibo2" || srcNode?.class_type === "PH3ImageConditioning"
|
||||
|| srcNode?.class_type === "MiniMaxH3ImageConditioning") {
|
||||
const raw = isPos ? srcNode.inputs?.prompt : srcNode.inputs?.negative_prompt;
|
||||
const text = Array.isArray(raw) ? resolveLinkedText(wf, raw[0], raw[1] ?? 0) : raw;
|
||||
if (text && typeof text === "string") { if (isPos) pos.add(text); else neg.add(text); }
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
/**
|
||||
* ocr-tools.js — AI TOOL > TOOLS: OCR (text extraction) with Ollama OCR models.
|
||||
*
|
||||
* Ported from ComfyUI Comic Creator's OCR tab (static/js/ocr-core.js + main/29-ocr-tab.js),
|
||||
* keeping only the general-purpose part: read the text of one image, optionally with boxes.
|
||||
* What carried over from CC's measurements:
|
||||
* - Send a JPEG whose long side is <= 1600 px with transparent areas filled white.
|
||||
* - temperature 0; retry once on HTTP 500 (the runner can drop right after a model swap).
|
||||
* - LightOnOCR sometimes returns nothing or loops on dense images; a grayscale copy often works,
|
||||
* so an empty / repetitive result is retried once in grayscale.
|
||||
* - Per-model prompts: PaddleOCR-VL "OCR:", GLM-OCR "Text Recognition:".
|
||||
*
|
||||
* Measured in WFS (2026-11-11, mixed JA/EN test page with a vertical column):
|
||||
* LightOnOCR-3 4b text-only 1.1 s / boxes 7.9 s, 100 % PaddleOCR-VL-manga 3.9 s, 100 %
|
||||
* unlimited-ocr (DeepSeek-OCR) 14 s, 100 %, always emits boxes
|
||||
* PaddleOCR-VL 1.6 11 s and GLM-OCR 15 s missed the vertical column (94 %).
|
||||
* Hence LightOnOCR is preferred, and boxes are only requested when the user asks for them.
|
||||
*/
|
||||
|
||||
export const SEND_MAX_SIDE = 1600;
|
||||
|
||||
// Preference order when picking the default OCR model from what Ollama has installed.
|
||||
const MODEL_PREFERENCE = [/lighton/i, /paddleocr.*manga/i, /paddleocr/i, /deepseek.*ocr|unlimited-ocr/i, /glm-ocr/i, /ocr/i];
|
||||
|
||||
export const BOX_PROMPT = "Detect all text regions and output each as text with bounding box [x1,y1,x2,y2] normalized to 0-1000.";
|
||||
const PLAIN_PROMPT = "Extract all text from this image.";
|
||||
|
||||
/** Installed Ollama models that look like OCR models, best first. */
|
||||
export async function listOcrModels(baseUrl) {
|
||||
const res = await fetch(`${baseUrl.replace(/\/+$/, "")}/api/tags`);
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const names = ((await res.json()).models || []).map((m) => m.name).filter((n) => /ocr/i.test(n));
|
||||
const rank = (n) => { const i = MODEL_PREFERENCE.findIndex((re) => re.test(n)); return i < 0 ? 99 : i; };
|
||||
return names.sort((a, b) => rank(a) - rank(b) || a.localeCompare(b));
|
||||
}
|
||||
|
||||
/** Can this model return text positions? (LightOnOCR on request, DeepSeek-OCR always) */
|
||||
export function supportsBoxes(model) {
|
||||
return /lighton|unlimited-ocr|deepseek.*ocr/i.test(String(model || ""));
|
||||
}
|
||||
|
||||
export function ocrPrompt(model, withBoxes) {
|
||||
const m = String(model || "");
|
||||
if (/lighton/i.test(m)) return withBoxes ? BOX_PROMPT : PLAIN_PROMPT;
|
||||
if (/paddle/i.test(m)) return "OCR:";
|
||||
if (/glm-ocr/i.test(m)) return "Text Recognition:";
|
||||
if (/unlimited-ocr|deepseek.*ocr/i.test(m)) return "Free OCR.";
|
||||
return PLAIN_PROMPT;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Parsing
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const clamp01 = (v) => Math.min(1, Math.max(0, v));
|
||||
|
||||
function makeBox(label, x1, y1, x2, y2, text) {
|
||||
const a = clamp01(Math.min(x1, x2)), b = clamp01(Math.min(y1, y2));
|
||||
const c = clamp01(Math.max(x1, x2)), d = clamp01(Math.max(y1, y2));
|
||||
if (c - a < 0.004 || d - b < 0.004) return null;
|
||||
return { label: label || "text", x1: a, y1: b, x2: c, y2: d, text: cleanLine(text) };
|
||||
}
|
||||
|
||||
// Markdown decorations the OCR models add (headings, bold, rules)
|
||||
function cleanLine(s) {
|
||||
return String(s || "")
|
||||
.replace(/^\s*#{1,6}\s*/gm, "")
|
||||
.replace(/\*\*([^*]+)\*\*/g, "$1")
|
||||
.replace(/^\s*(-{3,}|\*{3,}|_{3,})\s*$/gm, "")
|
||||
.trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Boxes from either syntax (coordinates normalized 0-1000, x1,y1,x2,y2):
|
||||
* LightOnOCR ` text`
|
||||
* DeepSeek-OCR `label [x1, y1, x2, y2]text` (one per line)
|
||||
* Returns [{ label, x1, y1, x2, y2, text }] with 0-1 coordinates.
|
||||
*/
|
||||
export function parseOcrBoxes(raw) {
|
||||
const s = String(raw || "");
|
||||
const out = [];
|
||||
const md = s.split(/(?=!\[[^\]]*\]\(\s*\d+\s*,\s*\d+\s*,\s*\d+\s*,\s*\d+\s*\))/);
|
||||
for (const chunk of md) {
|
||||
const m = chunk.match(/^!\[([^\]]*)\]\(\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*\)([\s\S]*)$/);
|
||||
if (!m) continue;
|
||||
const box = makeBox(m[1], +m[2] / 1000, +m[3] / 1000, +m[4] / 1000, +m[5] / 1000, m[6]);
|
||||
if (box) out.push(box);
|
||||
}
|
||||
if (out.length) return out;
|
||||
for (const line of s.split(/\r?\n/)) {
|
||||
const m = line.match(/^\s*([a-z_]+)?\s*\[\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*,\s*(\d+)\s*\]\s*(.*)$/i);
|
||||
if (!m) continue;
|
||||
const box = makeBox(m[1], +m[2] / 1000, +m[3] / 1000, +m[4] / 1000, +m[5] / 1000, m[6]);
|
||||
if (box) out.push(box);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Plain text of an OCR reply: box tokens and Markdown decorations removed, blank runs collapsed. */
|
||||
export function ocrPlainText(raw, boxes = null) {
|
||||
const list = boxes || parseOcrBoxes(raw);
|
||||
if (list.length) return list.map((b) => b.text).filter(Boolean).join("\n");
|
||||
// Model special tokens (<|ref|> etc.) and Markdown decorations removed, line by line;
|
||||
// blank lines between paragraphs are kept but never more than one in a row.
|
||||
return String(raw || "").replace(/<\|[^|]*\|>/g, "")
|
||||
.split(/\r?\n/).map((l) => cleanLine(l))
|
||||
.join("\n").replace(/\n{3,}/g, "\n\n").trim();
|
||||
}
|
||||
|
||||
// Empty, or the same line over and over (LightOnOCR's failure mode on dense images)
|
||||
export function ocrLooksFailed(text) {
|
||||
const lines = String(text || "").split(/\n/).map((l) => l.trim()).filter(Boolean);
|
||||
if (!lines.length) return true;
|
||||
return lines.length >= 30 && new Set(lines).size / lines.length < 0.5;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Image + model call
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** base64 image → JPEG base64 (long side <= maxSide, white background, optional grayscale). */
|
||||
export async function prepareOcrImage(base64, mimeType, { maxSide = SEND_MAX_SIDE, gray = false } = {}) {
|
||||
const img = await new Promise((resolve, reject) => {
|
||||
const el = new Image();
|
||||
el.onload = () => resolve(el);
|
||||
el.onerror = () => reject(new Error("image load failed"));
|
||||
el.src = `data:${mimeType || "image/png"};base64,${base64}`;
|
||||
});
|
||||
const k = Math.min(1, maxSide / Math.max(img.naturalWidth || 1, img.naturalHeight || 1));
|
||||
const cv = document.createElement("canvas");
|
||||
cv.width = Math.max(1, Math.round(img.naturalWidth * k));
|
||||
cv.height = Math.max(1, Math.round(img.naturalHeight * k));
|
||||
const cx = cv.getContext("2d");
|
||||
cx.fillStyle = "#fff";
|
||||
cx.fillRect(0, 0, cv.width, cv.height);
|
||||
cx.drawImage(img, 0, 0, cv.width, cv.height);
|
||||
if (gray) {
|
||||
const d = cx.getImageData(0, 0, cv.width, cv.height), p = d.data;
|
||||
for (let i = 0; i < p.length; i += 4) {
|
||||
const y = (p[i] * 299 + p[i + 1] * 587 + p[i + 2] * 114) / 1000;
|
||||
p[i] = p[i + 1] = p[i + 2] = y;
|
||||
}
|
||||
cx.putImageData(d, 0, 0);
|
||||
}
|
||||
return cv.toDataURL("image/jpeg", 0.92).split(",")[1];
|
||||
}
|
||||
|
||||
async function ollamaOcrChat(baseUrl, model, prompt, b64, retry = true) {
|
||||
const res = await fetch(`${baseUrl.replace(/\/+$/, "")}/api/chat`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model, stream: false, think: false,
|
||||
messages: [{ role: "user", content: prompt, images: [b64] }],
|
||||
options: { temperature: 0, num_predict: 4096 },
|
||||
}),
|
||||
});
|
||||
if (res.status === 500 && retry) {
|
||||
await new Promise((r) => setTimeout(r, 800));
|
||||
return ollamaOcrChat(baseUrl, model, prompt, b64, false);
|
||||
}
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status} ${(await res.text().catch(() => "")).slice(0, 200)}`);
|
||||
return (await res.json()).message?.content || "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Read one image. Returns { text, boxes, retriedGray, ms }.
|
||||
* withBoxes only takes effect for models that support positions (supportsBoxes).
|
||||
*/
|
||||
export async function runOcr({ baseUrl, model, base64, mimeType, withBoxes = false }) {
|
||||
const started = performance.now();
|
||||
const prompt = ocrPrompt(model, withBoxes && supportsBoxes(model));
|
||||
const read = async (gray) => {
|
||||
const raw = await ollamaOcrChat(baseUrl, model, prompt, await prepareOcrImage(base64, mimeType, { gray }));
|
||||
// Only boxes with text: the result shows one line per box, so box #n == line #n
|
||||
// (image regions without text would shift the numbering).
|
||||
const all = parseOcrBoxes(raw);
|
||||
const boxes = all.filter((b) => b.text);
|
||||
return { boxes, text: all.length ? boxes.map((b) => b.text).join("\n") : ocrPlainText(raw) };
|
||||
};
|
||||
let r = await read(false);
|
||||
let retriedGray = false;
|
||||
if (ocrLooksFailed(r.text)) {
|
||||
const g = await read(true);
|
||||
if (!ocrLooksFailed(g.text) || g.text.length > r.text.length) { r = g; retriedGray = true; }
|
||||
}
|
||||
return { ...r, retriedGray, ms: Math.round(performance.now() - started) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Draw boxes over an <img> shown with object-fit: contain inside `container`.
|
||||
* Each box gets a small numbered tag; clicking it calls onPick(index).
|
||||
*/
|
||||
export function renderOcrOverlay(container, img, boxes, onPick) {
|
||||
container.innerHTML = "";
|
||||
if (!img || !boxes?.length || !img.naturalWidth) { container.style.display = "none"; return; }
|
||||
const cw = img.clientWidth, ch = img.clientHeight;
|
||||
const k = Math.min(cw / img.naturalWidth, ch / img.naturalHeight);
|
||||
const w = img.naturalWidth * k, h = img.naturalHeight * k;
|
||||
const ox = img.offsetLeft + (cw - w) / 2, oy = img.offsetTop + (ch - h) / 2;
|
||||
boxes.forEach((b, i) => {
|
||||
const el = document.createElement("div");
|
||||
el.className = "wfm-ai-ocr-box";
|
||||
el.style.left = `${ox + b.x1 * w}px`;
|
||||
el.style.top = `${oy + b.y1 * h}px`;
|
||||
el.style.width = `${(b.x2 - b.x1) * w}px`;
|
||||
el.style.height = `${(b.y2 - b.y1) * h}px`;
|
||||
el.title = b.text;
|
||||
const tag = document.createElement("span");
|
||||
tag.className = "wfm-ai-ocr-box-tag";
|
||||
tag.textContent = String(i + 1);
|
||||
el.appendChild(tag);
|
||||
el.addEventListener("click", (e) => { e.stopPropagation(); onPick?.(i); });
|
||||
container.appendChild(el);
|
||||
});
|
||||
container.style.display = "";
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
/**
|
||||
* PLaMo Translate over a chat API (Ollama /api/chat, LM Studio, ...).
|
||||
*
|
||||
* PLaMo Translate is a translation-only model. Its chat template treats the user
|
||||
* message as source text and a system message that does not start with
|
||||
* "[plamo-translate]" as mere context, so instruction prompts are translated or
|
||||
* ignored and the direction falls back to Japanese <-> English auto-switching.
|
||||
* Per the model card, chat clients pass the languages in a leading
|
||||
* "[plamo-translate]" system message and send the text alone.
|
||||
*
|
||||
* Keep in sync with the inline copy in web/comfyui/node_sets_menu.js (side panel A tab),
|
||||
* which cannot import from static/.
|
||||
*/
|
||||
|
||||
// Languages PLaMo 3 Translate supports.
|
||||
export const PLAMO_LANGS = [
|
||||
"ar", "az", "be", "bg", "bn", "ca", "ceb", "de", "en", "es", "et", "eu", "fa", "fr", "gu", "he",
|
||||
"hi", "hr", "hu", "hy", "id", "it", "ja", "jv", "ka", "kk", "km", "ko", "lt", "lv", "ml", "mr",
|
||||
"ms", "my", "ne", "nl", "pa", "pl", "ru", "si", "sk", "sl", "sr", "sw", "ta", "te", "th", "tl",
|
||||
"ur", "uz", "vi", "zh",
|
||||
];
|
||||
|
||||
export const PLAMO_STOP = ["<|plamo:tag|>", "<|plamo:bos|>"];
|
||||
|
||||
export function isPlamoModel(model) {
|
||||
return /plamo/i.test(model || "");
|
||||
}
|
||||
|
||||
let _nameMap = null;
|
||||
function _buildNameMap() {
|
||||
const map = new Map();
|
||||
const locales = ["en", "ja", "zh", typeof navigator !== "undefined" ? navigator.language : "en"];
|
||||
for (const code of PLAMO_LANGS) {
|
||||
map.set(code, code);
|
||||
for (const loc of [...locales, code]) { // `code` adds the endonym (français, 한국어, ...)
|
||||
try {
|
||||
const name = new Intl.DisplayNames([loc], { type: "language" }).of(code);
|
||||
if (name) map.set(name.toLowerCase(), code);
|
||||
} catch { /* Intl.DisplayNames unavailable */ }
|
||||
}
|
||||
}
|
||||
// Common spellings Intl does not produce
|
||||
[["chinese", "zh"], ["中文", "zh"], ["中国語", "zh"], ["filipino", "tl"], ["tagalog", "tl"], ["persian", "fa"], ["farsi", "fa"]]
|
||||
.forEach(([k, v]) => map.set(k, v));
|
||||
return map;
|
||||
}
|
||||
|
||||
/** "ja" / "Japanese" / "日本語" / "French" -> PLaMo language code, or "" when unknown. */
|
||||
export function plamoLangCode(nameOrCode) {
|
||||
const key = (nameOrCode || "").trim().toLowerCase();
|
||||
if (!key) return "";
|
||||
if (!_nameMap) _nameMap = _buildNameMap();
|
||||
return _nameMap.get(key) || _nameMap.get(key.split(/[-_]/)[0]) || "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Chat messages for translating ``text`` with PLaMo. ``src``/``dst`` are codes or
|
||||
* language names; an unknown source is left to auto-detection. Throws when the
|
||||
* target language is not one PLaMo supports.
|
||||
*/
|
||||
export function buildPlamoMessages(text, src, dst) {
|
||||
const dstCode = plamoLangCode(dst);
|
||||
if (!dstCode) throw new Error(`PLaMo Translate does not support the target language "${dst}"`);
|
||||
const srcCode = plamoLangCode(src);
|
||||
const header = `[plamo-translate]\ntarget_lang=${dstCode}` + (srcCode && srcCode !== dstCode ? `\nsource_lang=${srcCode}` : "");
|
||||
return [
|
||||
{ role: "system", content: header },
|
||||
{ role: "user", content: text },
|
||||
];
|
||||
}
|
||||
|
||||
export function cleanPlamoOutput(text) {
|
||||
return (text || "").replace(/<\|plamo:[a-z_]+\|>/g, "").trim();
|
||||
}
|
||||
@@ -46,30 +46,37 @@ function _stripThinkingTags(text) {
|
||||
}
|
||||
|
||||
// Ollamaは `think` (bool) と `options.num_predict`、OpenAI互換は `max_tokens` で制御
|
||||
function _applyGenOptions(body, backend, cfg) {
|
||||
// extra: { temperature, stop, noThink } — 翻訳専用モデル(PLaMo)向けの追加指定
|
||||
function _applyGenOptions(body, backend, cfg, extra = {}) {
|
||||
if (backend === "ollama") {
|
||||
body.think = !!cfg.thinkingMode;
|
||||
if (cfg.maxTokens > 0) body.options = { ...(body.options || {}), num_predict: cfg.maxTokens };
|
||||
} else if (cfg.maxTokens > 0) {
|
||||
body.max_tokens = cfg.maxTokens;
|
||||
if (!extra.noThink) body.think = !!cfg.thinkingMode;
|
||||
const opts = { ...(body.options || {}) };
|
||||
if (cfg.maxTokens > 0) opts.num_predict = cfg.maxTokens;
|
||||
if (extra.temperature !== undefined) opts.temperature = extra.temperature;
|
||||
if (extra.stop) opts.stop = extra.stop;
|
||||
if (Object.keys(opts).length) body.options = opts;
|
||||
} else {
|
||||
if (cfg.maxTokens > 0) body.max_tokens = cfg.maxTokens;
|
||||
if (extra.temperature !== undefined) body.temperature = extra.temperature;
|
||||
if (extra.stop) body.stop = extra.stop;
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
async function chatWithAi(model, messages) {
|
||||
async function chatWithAi(model, messages, extra = {}) {
|
||||
const cfg = getPromptAiConfig();
|
||||
const { backend, url } = cfg;
|
||||
if (backend === "ollama") {
|
||||
const res = await fetch(`${url}/api/chat`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(_applyGenOptions({ model, messages, stream: false }, backend, cfg)),
|
||||
body: JSON.stringify(_applyGenOptions({ model, messages, stream: false }, backend, cfg, extra)),
|
||||
});
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const text = (await res.json()).message?.content || "";
|
||||
return cfg.thinkingMode ? text : _stripThinkingTags(text);
|
||||
} else if (backend === "unsloth") {
|
||||
const body = _applyGenOptions({ model, messages: _toOpenAiMessages(messages), stream: false }, backend, cfg);
|
||||
const body = _applyGenOptions({ model, messages: _toOpenAiMessages(messages), stream: false }, backend, cfg, extra);
|
||||
const data = await unslothProxy(url, "/v1/chat/completions", "POST", body);
|
||||
const msg = data.choices?.[0]?.message;
|
||||
// Unslothはreasoningを別フィールドで返すため<think>タグに畳んで表示に混ぜる
|
||||
@@ -80,7 +87,7 @@ async function chatWithAi(model, messages) {
|
||||
const res = await fetch(`${url}/v1/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(_applyGenOptions({ model, messages: _toOpenAiMessages(messages), stream: false }, backend, cfg)),
|
||||
body: JSON.stringify(_applyGenOptions({ model, messages: _toOpenAiMessages(messages), stream: false }, backend, cfg, extra)),
|
||||
});
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const text = (await res.json()).choices?.[0]?.message?.content || "";
|
||||
@@ -141,6 +148,25 @@ function addChatMessage(role, content) {
|
||||
chat.scrollTop = chat.scrollHeight;
|
||||
}
|
||||
|
||||
// 応答待ちの間だけ表示するプレースホルダ（初回はモデル読み込みで十数秒かかることがある）
|
||||
function _addPendingMessage(text) {
|
||||
const chat = document.getElementById("wfm-ollama-chat");
|
||||
if (!chat) return null;
|
||||
const msg = document.createElement("div");
|
||||
msg.className = "wfm-ollama-msg assistant wfm-ollama-pending";
|
||||
const roleLabel = document.createElement("div");
|
||||
roleLabel.className = "wfm-ollama-msg-role";
|
||||
roleLabel.textContent = "AI";
|
||||
const contentDiv = document.createElement("div");
|
||||
contentDiv.className = "wfm-ollama-msg-content";
|
||||
contentDiv.style.opacity = "0.6";
|
||||
contentDiv.textContent = text;
|
||||
msg.append(roleLabel, contentDiv);
|
||||
chat.appendChild(msg);
|
||||
chat.scrollTop = chat.scrollHeight;
|
||||
return msg;
|
||||
}
|
||||
|
||||
function clearChat() {
|
||||
ollamaState.chatHistory = [];
|
||||
ollamaState.attachedImage = null;
|
||||
@@ -243,13 +269,16 @@ async function sendMessage() {
|
||||
const sendBtn = document.getElementById("wfm-ollama-send-btn");
|
||||
if (sendBtn) sendBtn.disabled = true;
|
||||
|
||||
const pending = _addPendingMessage("…");
|
||||
try {
|
||||
const reply = await chatWithAi(model, ollamaState.chatHistory);
|
||||
pending?.remove();
|
||||
addChatMessage("assistant", reply);
|
||||
ollamaState.chatHistory.push({ role: "assistant", content: reply });
|
||||
ollamaState.attachedImage = null;
|
||||
updateAttachmentDisplay();
|
||||
} catch (err) {
|
||||
pending?.remove();
|
||||
showToast(t("error") + ": " + err.message, "error");
|
||||
} finally {
|
||||
if (sendBtn) sendBtn.disabled = false;
|
||||
@@ -281,7 +310,24 @@ async function sendTranslate(direction) {
|
||||
.map((id) => document.getElementById(id));
|
||||
btns.forEach((b) => { if (b) b.disabled = true; });
|
||||
|
||||
const pending = _addPendingMessage(t("aiTranslating"));
|
||||
try {
|
||||
// PLaMo Translate is a translation-only model: anything in the user message is
|
||||
// treated as source text, so an instruction prompt gets translated too, and without
|
||||
// an explicit target it only switches between Japanese and English (ZH→EN / EN→ZH
|
||||
// came out in Japanese). Per its model card, Ollama/LM Studio clients pass the
|
||||
// languages in a leading "[plamo-translate]" system message and send the text alone.
|
||||
if (/plamo/i.test(model)) {
|
||||
const [src, dst] = { ja2en: ["ja", "en"], en2ja: ["en", "ja"], zh2en: ["zh", "en"], en2zh: ["en", "zh"] }[direction];
|
||||
const reply = await chatWithAi(model, [
|
||||
{ role: "system", content: `[plamo-translate]\ntarget_lang=${dst}\nsource_lang=${src}` },
|
||||
{ role: "user", content: text },
|
||||
], { temperature: 0, stop: ["<|plamo:tag|>", "<|plamo:bos|>"], noThink: true });
|
||||
pending?.remove();
|
||||
addChatMessage("assistant", reply.replace(/<\|plamo:[a-z_]+\|>/g, "").trim());
|
||||
return;
|
||||
}
|
||||
|
||||
const prompts = {
|
||||
ja2en: `Translate the following Japanese text into English. Output only the translated text, nothing else.\n\n${text}`,
|
||||
en2ja: `以下の英語のテキストを日本語に翻訳してください。翻訳結果のみを出力し、他の説明は不要です。\n\n${text}`,
|
||||
@@ -291,8 +337,10 @@ async function sendTranslate(direction) {
|
||||
const prompt = prompts[direction];
|
||||
|
||||
const reply = await chatWithAi(model, [{ role: "user", content: prompt }]);
|
||||
pending?.remove();
|
||||
addChatMessage("assistant", reply);
|
||||
} catch (err) {
|
||||
pending?.remove();
|
||||
showToast(t("error") + ": " + err.message, "error");
|
||||
} finally {
|
||||
btns.forEach((b) => { if (b) b.disabled = false; });
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
/**
|
||||
* Settings tab section for Prompt Translate (see prompt-translate.js).
|
||||
* Values live in the server settings.json so the backend can read them.
|
||||
*/
|
||||
|
||||
import { showToast } from "./app.js";
|
||||
import { t, getLang } from "./i18n.js";
|
||||
import { escapeHtml } from "./util.js";
|
||||
import { setTranslateConfig, translatePrompt } from "./prompt-translate.js";
|
||||
|
||||
const DEFAULT_URL = "http://localhost:11434";
|
||||
const DEFAULT_MODEL = "hf.co/pfnet/plamo-3-translate-2b-GGUF:Q4_K_M";
|
||||
|
||||
// Languages PLaMo 3 Translate supports (English is the target, so it is left out).
|
||||
const SOURCE_LANGS = [
|
||||
"ja", "zh", "ko", "ar", "az", "be", "bg", "bn", "ca", "ceb", "de", "es", "et", "eu", "fa", "fr",
|
||||
"gu", "he", "hi", "hr", "hu", "hy", "id", "it", "jv", "ka", "kk", "km", "lt", "lv", "ml", "mr",
|
||||
"ms", "my", "ne", "nl", "pa", "pl", "ru", "si", "sk", "sl", "sr", "sw", "ta", "te", "th", "tl",
|
||||
"ur", "uz", "vi",
|
||||
];
|
||||
|
||||
const SAMPLE_TEXT = {
|
||||
ja: "夕焼けの海辺に立つ少女、(長い黒髪:1.2), masterpiece",
|
||||
zh: "站在夕阳海边的少女, (长长的黑发:1.2), masterpiece",
|
||||
};
|
||||
|
||||
function _langName(code) {
|
||||
try {
|
||||
const dn = new Intl.DisplayNames([getLang() || "en"], { type: "language" });
|
||||
return dn.of(code) || code;
|
||||
} catch {
|
||||
return code;
|
||||
}
|
||||
}
|
||||
|
||||
export function promptTranslateSettingsHtml(s) {
|
||||
const enabled = !!s.prompt_translate_enabled;
|
||||
const src = s.prompt_translate_source_lang || "auto";
|
||||
const unload = s.prompt_translate_unload !== false;
|
||||
const cpu = s.prompt_translate_device === "cpu";
|
||||
return `
|
||||
<details class="wfm-settings-section" id="wfm-pt-settings">
|
||||
<summary class="wfm-settings-summary">${t("promptTranslateSection")}</summary>
|
||||
<div class="wfm-form-group">
|
||||
<label style="display:flex;align-items:center;gap:6px;cursor:pointer;">
|
||||
<input type="checkbox" id="wfm-pt-enabled" ${enabled ? "checked" : ""}>
|
||||
<span>${t("promptTranslateEnable")}</span>
|
||||
</label>
|
||||
<small style="color:var(--wfm-text-secondary);font-size:11px;display:block;margin-top:4px;">
|
||||
${t("promptTranslateHint")}
|
||||
</small>
|
||||
</div>
|
||||
<div id="wfm-pt-body" style="${enabled ? "" : "opacity:0.55;"}">
|
||||
<div class="wfm-form-group">
|
||||
<label>${t("promptTranslateAutoHeading")}</label>
|
||||
<label style="display:flex;align-items:center;gap:6px;cursor:pointer;">
|
||||
<input type="checkbox" id="wfm-pt-auto" ${s.prompt_translate_auto ? "checked" : ""}>
|
||||
<span>${t("promptTranslateAuto")}</span>
|
||||
</label>
|
||||
<label style="display:flex;align-items:center;gap:6px;cursor:pointer;margin-top:2px;">
|
||||
<input type="checkbox" id="wfm-pt-auto-lab" ${s.prompt_translate_auto_lab ? "checked" : ""}>
|
||||
<span>${t("promptTranslateAutoLab")}</span>
|
||||
</label>
|
||||
<label style="display:flex;align-items:center;gap:6px;cursor:pointer;margin-top:2px;">
|
||||
<input type="checkbox" id="wfm-pt-auto-video" ${s.prompt_translate_auto_video ? "checked" : ""}>
|
||||
<span>${t("promptTranslateAutoVideo")}</span>
|
||||
</label>
|
||||
<small style="color:var(--wfm-text-secondary);font-size:11px;display:block;margin-top:4px;">
|
||||
${t("promptTranslateAutoHint")}
|
||||
</small>
|
||||
</div>
|
||||
<div class="wfm-form-group">
|
||||
<label>${t("promptTranslateDevice")}</label>
|
||||
<select class="wfm-select" id="wfm-pt-device" style="width:100%;">
|
||||
<option value="gpu" ${cpu ? "" : "selected"}>${t("promptTranslateDeviceGpu")}</option>
|
||||
<option value="cpu" ${cpu ? "selected" : ""}>${t("promptTranslateDeviceCpu")}</option>
|
||||
</select>
|
||||
<label id="wfm-pt-unload-label" style="display:flex;align-items:center;gap:6px;cursor:pointer;margin-top:6px;${cpu ? "opacity:0.5;" : ""}">
|
||||
<input type="checkbox" id="wfm-pt-unload" ${unload ? "checked" : ""} ${cpu ? "disabled" : ""}>
|
||||
<span>${t("promptTranslateUnload")}</span>
|
||||
</label>
|
||||
<small style="color:var(--wfm-text-secondary);font-size:11px;display:block;margin-top:4px;">
|
||||
${t("promptTranslateDeviceHint")}
|
||||
</small>
|
||||
</div>
|
||||
<div class="wfm-form-group">
|
||||
<label>${t("promptTranslateSourceLang")}</label>
|
||||
<select class="wfm-select" id="wfm-pt-source-lang" style="width:100%;">
|
||||
<option value="auto" ${src === "auto" ? "selected" : ""}>${t("promptTranslateSourceAuto")}</option>
|
||||
${SOURCE_LANGS.map((c) => `<option value="${c}" ${src === c ? "selected" : ""}>${escapeHtml(_langName(c))} (${c})</option>`).join("")}
|
||||
</select>
|
||||
</div>
|
||||
<div class="wfm-form-group">
|
||||
<label>${t("promptTranslateUrl")}</label>
|
||||
<div style="display:flex;gap:8px;">
|
||||
<input type="text" class="wfm-input" id="wfm-pt-url" value="${escapeHtml(s.prompt_translate_url || "")}" placeholder="${DEFAULT_URL}">
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-pt-url-save">${t("save")}</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="wfm-form-group">
|
||||
<label>${t("promptTranslateModel")}</label>
|
||||
<div style="display:flex;gap:8px;">
|
||||
<select class="wfm-select" id="wfm-pt-model" style="flex:1;min-width:0;">
|
||||
<option value="${escapeHtml(s.prompt_translate_model || DEFAULT_MODEL)}" selected>${escapeHtml(s.prompt_translate_model || DEFAULT_MODEL)}</option>
|
||||
</select>
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-pt-model-refresh">${t("refresh")}</button>
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-pt-model-pull" style="display:none;">${t("promptTranslatePull")}</button>
|
||||
</div>
|
||||
<small id="wfm-pt-model-hint" style="color:var(--wfm-text-secondary);font-size:11px;display:block;margin-top:4px;">
|
||||
${t("promptTranslateModelHint")}
|
||||
</small>
|
||||
</div>
|
||||
<div class="wfm-form-group">
|
||||
<div style="display:flex;gap:8px;align-items:center;">
|
||||
<button class="wfm-btn wfm-btn-sm" id="wfm-pt-test">${t("test")}</button>
|
||||
<span id="wfm-pt-test-status" style="font-size:12px;word-break:break-all;"></span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</details>`;
|
||||
}
|
||||
|
||||
export function initPromptTranslateSettings(serverSettings, saveServerSettings) {
|
||||
const $ = (id) => document.getElementById(id);
|
||||
let installed = [];
|
||||
let presets = [];
|
||||
|
||||
const save = async (patch, toast = true) => {
|
||||
try {
|
||||
const res = await saveServerSettings(patch);
|
||||
if (res?.error) throw new Error(res.error);
|
||||
Object.assign(serverSettings, patch);
|
||||
setTranslateConfig(patch);
|
||||
if (toast) showToast(t("promptTranslateSaved"), "success");
|
||||
return true;
|
||||
} catch (err) {
|
||||
showToast(t("errorWithMsg", err.message), "error");
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
const currentModel = () => serverSettings.prompt_translate_model || DEFAULT_MODEL;
|
||||
|
||||
const renderModelOptions = () => {
|
||||
const sel = $("wfm-pt-model");
|
||||
if (!sel) return;
|
||||
const cur = currentModel();
|
||||
const plamoInstalled = installed.filter((m) => /plamo/i.test(m));
|
||||
// Embedding-only models cannot generate text, so they are not offered.
|
||||
const others = installed.filter((m) => !/plamo/i.test(m) && !/embed|minilm/i.test(m));
|
||||
const missingPresets = presets.filter((p) => !installed.includes(p));
|
||||
const opt = (v, label) => `<option value="${escapeHtml(v)}" ${v === cur ? "selected" : ""}>${escapeHtml(label || v)}</option>`;
|
||||
let html = "";
|
||||
if (plamoInstalled.length) html += `<optgroup label="PLaMo 3 Translate">${plamoInstalled.map((m) => opt(m)).join("")}</optgroup>`;
|
||||
if (missingPresets.length) html += `<optgroup label="${escapeHtml(t("promptTranslateNotPulled"))}">${missingPresets.map((m) => opt(m, `${m} ${t("promptTranslateNotPulledMark")}`)).join("")}</optgroup>`;
|
||||
if (others.length) html += `<optgroup label="${escapeHtml(t("promptTranslateOtherModels"))}">${others.map((m) => opt(m)).join("")}</optgroup>`;
|
||||
if (![...installed, ...presets].includes(cur)) html = opt(cur) + html;
|
||||
sel.innerHTML = html;
|
||||
updatePullButton();
|
||||
};
|
||||
|
||||
const updatePullButton = () => {
|
||||
const btn = $("wfm-pt-model-pull");
|
||||
if (!btn) return;
|
||||
const m = currentModel();
|
||||
btn.style.display = presets.includes(m) && !installed.includes(m) ? "" : "none";
|
||||
};
|
||||
|
||||
const refreshModels = async (quiet = false) => {
|
||||
try {
|
||||
const res = await fetch("/api/wfm/translate/models");
|
||||
const data = await res.json();
|
||||
presets = data.presets || [];
|
||||
if (data.status !== "success") throw new Error(data.message || `HTTP ${res.status}`);
|
||||
installed = data.models || [];
|
||||
} catch (err) {
|
||||
installed = [];
|
||||
if (!quiet) showToast(t("promptTranslateModelsError", err.message), "error");
|
||||
}
|
||||
renderModelOptions();
|
||||
};
|
||||
|
||||
// CPU mode keeps the model resident: preload it now, and release it when leaving CPU mode
|
||||
// or turning the feature off so it does not sit in RAM unused.
|
||||
const postTranslate = (path) => fetch(`/api/wfm/translate/${path}`, { method: "POST" }).catch(() => {});
|
||||
const isCpu = () => serverSettings.prompt_translate_device === "cpu";
|
||||
|
||||
$("wfm-pt-device")?.addEventListener("change", async (e) => {
|
||||
const device = e.target.value === "cpu" ? "cpu" : "gpu";
|
||||
const prev = isCpu() ? "cpu" : "gpu";
|
||||
if (!(await save({ prompt_translate_device: device }))) { e.target.value = prev; return; }
|
||||
const cpu = device === "cpu";
|
||||
const cb = $("wfm-pt-unload");
|
||||
if (cb) cb.disabled = cpu;
|
||||
const lbl = $("wfm-pt-unload-label");
|
||||
if (lbl) lbl.style.opacity = cpu ? "0.5" : "";
|
||||
// Ollama reloads the model for the new placement; drop the old instance first.
|
||||
await postTranslate("unload");
|
||||
if (cpu) postTranslate("warmup");
|
||||
});
|
||||
|
||||
$("wfm-pt-enabled")?.addEventListener("change", async (e) => {
|
||||
const on = !!e.target.checked;
|
||||
if (!(await save({ prompt_translate_enabled: on }))) { e.target.checked = !on; return; }
|
||||
if (isCpu()) postTranslate(on ? "warmup" : "unload");
|
||||
const body = $("wfm-pt-body");
|
||||
if (body) body.style.opacity = on ? "" : "0.55";
|
||||
});
|
||||
[["wfm-pt-auto", "prompt_translate_auto"], ["wfm-pt-auto-lab", "prompt_translate_auto_lab"], ["wfm-pt-auto-video", "prompt_translate_auto_video"]]
|
||||
.forEach(([id, key]) => {
|
||||
$(id)?.addEventListener("change", async (e) => {
|
||||
const on = !!e.target.checked;
|
||||
if (!(await save({ [key]: on }))) e.target.checked = !on;
|
||||
});
|
||||
});
|
||||
$("wfm-pt-unload")?.addEventListener("change", async (e) => {
|
||||
const on = !!e.target.checked;
|
||||
if (!(await save({ prompt_translate_unload: on }))) e.target.checked = !on;
|
||||
});
|
||||
$("wfm-pt-source-lang")?.addEventListener("change", (e) => save({ prompt_translate_source_lang: e.target.value }));
|
||||
$("wfm-pt-url-save")?.addEventListener("click", async () => {
|
||||
const url = ($("wfm-pt-url")?.value || "").trim();
|
||||
if (await save({ prompt_translate_url: url })) refreshModels();
|
||||
});
|
||||
$("wfm-pt-model")?.addEventListener("change", async (e) => {
|
||||
const cpu = isCpu() && serverSettings.prompt_translate_enabled;
|
||||
if (cpu) await postTranslate("unload"); // release the previous resident model
|
||||
await save({ prompt_translate_model: e.target.value });
|
||||
if (cpu) postTranslate("warmup");
|
||||
updatePullButton();
|
||||
});
|
||||
$("wfm-pt-model-refresh")?.addEventListener("click", () => refreshModels());
|
||||
|
||||
$("wfm-pt-model-pull")?.addEventListener("click", async (e) => {
|
||||
const btn = e.currentTarget;
|
||||
const model = currentModel();
|
||||
btn.disabled = true;
|
||||
const label = btn.textContent;
|
||||
btn.textContent = t("promptTranslatePulling");
|
||||
try {
|
||||
const res = await fetch("/api/wfm/translate/pull", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ model }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (data.status !== "success") throw new Error(data.message || `HTTP ${res.status}`);
|
||||
showToast(t("promptTranslatePulled"), "success");
|
||||
await refreshModels(true);
|
||||
} catch (err) {
|
||||
showToast(t("errorWithMsg", err.message), "error");
|
||||
} finally {
|
||||
btn.disabled = false;
|
||||
btn.textContent = label;
|
||||
}
|
||||
});
|
||||
|
||||
$("wfm-pt-test")?.addEventListener("click", async (e) => {
|
||||
const btn = e.currentTarget;
|
||||
const status = $("wfm-pt-test-status");
|
||||
const sample = SAMPLE_TEXT[getLang()] || SAMPLE_TEXT.ja;
|
||||
btn.disabled = true;
|
||||
if (status) { status.style.color = "var(--wfm-text-secondary)"; status.textContent = t("promptTranslateRunning"); }
|
||||
const t0 = performance.now();
|
||||
try {
|
||||
const out = await translatePrompt(sample);
|
||||
const sec = ((performance.now() - t0) / 1000).toFixed(1);
|
||||
if (status) { status.style.color = "var(--wfm-success, #4caf50)"; status.textContent = `${sample} → ${out} (${sec}s)`; }
|
||||
} catch (err) {
|
||||
if (status) { status.style.color = "var(--wfm-danger, #f44336)"; status.textContent = err.message; }
|
||||
} finally {
|
||||
btn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
// Fill the model list when the section is first opened (avoids an Ollama call on every Settings visit).
|
||||
const section = $("wfm-pt-settings");
|
||||
section?.addEventListener("toggle", () => { if (section.open && !installed.length) refreshModels(true); });
|
||||
}
|
||||
@@ -0,0 +1,265 @@
|
||||
/**
|
||||
* Prompt Translate — type GenerateUI prompts in your own language and translate
|
||||
* them into English with a local Ollama model (PLaMo 3 Translate by default).
|
||||
*
|
||||
* Works like the A1111 prompt-translator extensions: only the parts of a prompt
|
||||
* that contain non-ASCII letters are sent to the model. Prompt syntax such as
|
||||
* commas, (weights:1.2), [ ], {a|b}, <lora:...>, __wildcards__,
|
||||
* embedding:name and BREAK/AND is kept as it is, so a mixed prompt like
|
||||
* "masterpiece, (赤いドレスの女性:1.2), <lora:foo:0.8>" only translates the Japanese part.
|
||||
*/
|
||||
|
||||
import { showToast } from "./app.js";
|
||||
import { t } from "./i18n.js";
|
||||
|
||||
let _config = null;
|
||||
let _configPromise = null;
|
||||
|
||||
// textarea -> { original, translated } for the "restore original" button
|
||||
const _history = new WeakMap();
|
||||
|
||||
export async function loadTranslateConfig(force = false) {
|
||||
if (_config && !force) return _config;
|
||||
if (!_configPromise || force) {
|
||||
_configPromise = fetch("/api/wfm/translate/config")
|
||||
.then((r) => r.json())
|
||||
.then((d) => { _config = d.config || {}; return _config; })
|
||||
.catch(() => { _config = {}; return _config; })
|
||||
.finally(() => { _configPromise = null; });
|
||||
}
|
||||
return _configPromise;
|
||||
}
|
||||
|
||||
export function getTranslateConfig() {
|
||||
return _config || {};
|
||||
}
|
||||
|
||||
export function isTranslateEnabled() {
|
||||
return !!_config?.prompt_translate_enabled;
|
||||
}
|
||||
|
||||
/** Called by the Settings tab after saving, so open UIs update without a reload. */
|
||||
export function setTranslateConfig(patch) {
|
||||
_config = { ...(_config || {}), ...patch };
|
||||
updateTranslateButtons();
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------
|
||||
// Prompt splitting
|
||||
// ------------------------------------------------------------
|
||||
|
||||
// Spans that are never translated: prompt syntax and delimiters.
|
||||
const PROTECTED_RE = /\b(?:BREAK|AND)\b|<[^<>\n]*>|__[^\s_][^\s]*?__|embedding:[^\s,()[\]{}|]+|:\s*-?\d+(?:\.\d+)?|[,\n()[\]{}|]/g;
|
||||
// A piece needs translation when it contains a non-ASCII letter (CJK, Cyrillic, accented Latin, ...).
|
||||
const NEEDS_TRANSLATION_RE = /(?![\x00-\x7F])\p{L}/u;
|
||||
|
||||
/** Split a prompt into [{ text, translate }] pieces. Joining all .text gives back the input. */
|
||||
export function splitPrompt(text) {
|
||||
const pieces = [];
|
||||
let last = 0;
|
||||
for (const m of text.matchAll(PROTECTED_RE)) {
|
||||
if (m.index > last) pieces.push(text.slice(last, m.index));
|
||||
pieces.push({ keep: m[0] });
|
||||
last = m.index + m[0].length;
|
||||
}
|
||||
if (last < text.length) pieces.push(text.slice(last));
|
||||
return pieces.map((p) => {
|
||||
if (typeof p !== "string") return { text: p.keep, translate: false };
|
||||
return { text: p, translate: NEEDS_TRANSLATION_RE.test(p) };
|
||||
});
|
||||
}
|
||||
|
||||
export function needsTranslation(text) {
|
||||
return splitPrompt(text || "").some((p) => p.translate);
|
||||
}
|
||||
|
||||
// Keep the surrounding whitespace of a piece and tidy the model output for prompt use.
|
||||
function _rebuild(source, translated) {
|
||||
const lead = source.match(/^\s*/)[0];
|
||||
const trail = source.match(/\s*$/)[0];
|
||||
const core = source.trim();
|
||||
let out = (translated || "").replace(/\s*\n+\s*/g, " ").trim();
|
||||
// A tag such as "赤いドレス" should not become "Red dress." — drop the final period
|
||||
// unless the source itself was a full sentence.
|
||||
if (!/[。．.!?！？]$/.test(core)) out = out.replace(/[.。]+$/, "");
|
||||
// "少女、(" -> "girl, (" rather than "girl,(" — a Japanese comma needs no space, an English one does.
|
||||
if (!trail && /[,;:]$/.test(out)) out += " ";
|
||||
return lead + out + trail;
|
||||
}
|
||||
|
||||
/** Translate a whole prompt string. Throws on backend errors. */
|
||||
export async function translatePrompt(text) {
|
||||
const pieces = splitPrompt(text || "");
|
||||
const cores = [...new Set(pieces.filter((p) => p.translate).map((p) => p.text.trim()))];
|
||||
if (cores.length === 0) return text;
|
||||
|
||||
const res = await fetch("/api/wfm/translate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ texts: cores }),
|
||||
});
|
||||
const data = await res.json().catch(() => ({}));
|
||||
if (!res.ok || data.status !== "success") {
|
||||
throw new Error(data.message || `HTTP ${res.status}`);
|
||||
}
|
||||
const map = new Map(cores.map((c, i) => [c, data.translations[i] ?? c]));
|
||||
return pieces.map((p) => (p.translate ? _rebuild(p.text, map.get(p.text.trim())) : p.text)).join("");
|
||||
}
|
||||
|
||||
// Replace textarea text while keeping the browser's Ctrl+Z history where possible.
|
||||
function _setTextareaValue(ta, value) {
|
||||
ta.focus();
|
||||
ta.select();
|
||||
let ok = false;
|
||||
try { ok = document.execCommand("insertText", false, value); } catch { ok = false; }
|
||||
if (!ok || ta.value !== value) ta.value = value;
|
||||
ta.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate one textarea in place. Returns true when the text changed.
|
||||
* The original text is remembered for the restore button.
|
||||
*/
|
||||
export async function translateTextarea(ta) {
|
||||
if (!ta || ta.disabled) return false;
|
||||
const original = ta.value;
|
||||
if (!needsTranslation(original)) return false;
|
||||
const translated = await translatePrompt(original);
|
||||
if (translated === original) return false;
|
||||
_setTextareaValue(ta, translated);
|
||||
_history.set(ta, { original, translated });
|
||||
_updateRestoreButton(ta);
|
||||
return true;
|
||||
}
|
||||
|
||||
export function restoreTextarea(ta) {
|
||||
const h = ta && _history.get(ta);
|
||||
if (!h || ta.disabled) return;
|
||||
_setTextareaValue(ta, h.original);
|
||||
_history.delete(ta);
|
||||
_updateRestoreButton(ta);
|
||||
}
|
||||
|
||||
const PROMPT_TEXTAREAS = ["wfm-prompt-pos-text", "wfm-prompt-neg-text"];
|
||||
|
||||
/**
|
||||
* GenerateUI hook: when translation and "translate before Generate" are both on,
|
||||
* translate the Positive/Negative prompt textareas. Throws if translation fails so
|
||||
* the caller can stop instead of sending untranslated text.
|
||||
*/
|
||||
export async function autoTranslateBeforeGenerate() {
|
||||
if (!(await isAutoTranslateOn("generate"))) return false;
|
||||
const targets = PROMPT_TEXTAREAS.map((id) => document.getElementById(id))
|
||||
.filter((ta) => ta && needsTranslation(ta.value));
|
||||
if (targets.length === 0) return false;
|
||||
const progress = document.getElementById("wfm-gen-progress-text");
|
||||
if (progress) progress.textContent = t("promptTranslateRunning");
|
||||
for (const ta of targets) await translateTextarea(ta);
|
||||
if (progress) progress.textContent = "";
|
||||
return true;
|
||||
}
|
||||
|
||||
// Settings key for each "translate automatically when running" checkbox.
|
||||
const AUTO_KEYS = {
|
||||
generate: "prompt_translate_auto",
|
||||
lab: "prompt_translate_auto_lab",
|
||||
video: "prompt_translate_auto_video",
|
||||
};
|
||||
|
||||
/** True when translation is enabled and the auto checkbox for ``kind`` (generate/lab/video) is on. */
|
||||
export async function isAutoTranslateOn(kind) {
|
||||
await loadTranslateConfig();
|
||||
return isTranslateEnabled() && !!_config[AUTO_KEYS[kind]];
|
||||
}
|
||||
|
||||
/**
|
||||
* Lab Run hook: translate the prompt texts inside one iteration's API-format workflow
|
||||
* copy, in place. The Lab plan's own keyframe values stay in the user's language;
|
||||
* repeated phrases come from the server-side cache, so later iterations cost ~nothing.
|
||||
* Throws on translation failure so the caller can stop the batch.
|
||||
*/
|
||||
export async function translateWorkflowPrompts(workflow, promptNodes) {
|
||||
for (const n of promptNodes || []) {
|
||||
const inputs = workflow?.[n.id]?.inputs;
|
||||
const key = n.textKey || "text";
|
||||
if (inputs && typeof inputs[key] === "string" && needsTranslation(inputs[key])) {
|
||||
inputs[key] = await translatePrompt(inputs[key]);
|
||||
}
|
||||
}
|
||||
return workflow;
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------
|
||||
// Buttons in the GenerateUI Prompt tab
|
||||
// ------------------------------------------------------------
|
||||
|
||||
function _updateRestoreButton(ta) {
|
||||
const btn = document.querySelector(`.wfm-pt-restore[data-target="${ta.id}"]`);
|
||||
if (!btn) return;
|
||||
const h = _history.get(ta);
|
||||
btn.style.display = isTranslateEnabled() && h && ta.value === h.translated ? "" : "none";
|
||||
}
|
||||
|
||||
export function updateTranslateButtons() {
|
||||
const on = isTranslateEnabled();
|
||||
document.querySelectorAll(".wfm-pt-translate").forEach((b) => { b.style.display = on ? "" : "none"; });
|
||||
document.querySelectorAll(".wfm-pt-restore").forEach((b) => {
|
||||
const ta = document.getElementById(b.dataset.target);
|
||||
if (ta) _updateRestoreButton(ta);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Wire the 🌐 / restore buttons produced by translateButtonsHtml() inside ``root``
|
||||
* (GenerateUI prompt tab, Lab prompt modal, Video Plan block prompt, ...).
|
||||
*/
|
||||
export function attachTranslateButtons(root = document) {
|
||||
root.querySelectorAll(".wfm-pt-translate").forEach((btn) => {
|
||||
btn.addEventListener("click", async () => {
|
||||
const ta = document.getElementById(btn.dataset.target);
|
||||
if (!ta) return;
|
||||
if (!needsTranslation(ta.value)) {
|
||||
showToast(t("promptTranslateNothing"), "info");
|
||||
return;
|
||||
}
|
||||
btn.disabled = true;
|
||||
const label = btn.textContent;
|
||||
btn.textContent = "…";
|
||||
try {
|
||||
await translateTextarea(ta);
|
||||
} catch (err) {
|
||||
showToast(t("promptTranslateError", err.message), "error");
|
||||
} finally {
|
||||
btn.disabled = false;
|
||||
btn.textContent = label;
|
||||
}
|
||||
});
|
||||
});
|
||||
root.querySelectorAll(".wfm-pt-restore").forEach((btn) => {
|
||||
btn.addEventListener("click", () => restoreTextarea(document.getElementById(btn.dataset.target)));
|
||||
});
|
||||
root.querySelectorAll(".wfm-pt-restore").forEach((btn) => {
|
||||
const ta = document.getElementById(btn.dataset.target);
|
||||
ta?.addEventListener("input", () => _updateRestoreButton(ta));
|
||||
});
|
||||
loadTranslateConfig().then(() => {
|
||||
updateTranslateButtons();
|
||||
_warmupOnce();
|
||||
});
|
||||
}
|
||||
|
||||
// CPU-resident mode: load the model in the background once per page so the first 🌐
|
||||
// does not pay Ollama's ~4 s startup. The server ignores this unless CPU mode is on.
|
||||
let _warmedUp = false;
|
||||
function _warmupOnce() {
|
||||
if (_warmedUp || !isTranslateEnabled() || _config.prompt_translate_device !== "cpu") return;
|
||||
_warmedUp = true;
|
||||
fetch("/api/wfm/translate/warmup", { method: "POST" }).catch(() => {});
|
||||
}
|
||||
|
||||
/** HTML for the two buttons placed next to a prompt textarea's Apply button. */
|
||||
export function translateButtonsHtml(textareaId) {
|
||||
const on = isTranslateEnabled();
|
||||
return `<button class="wfm-btn wfm-btn-sm wfm-pt-translate" data-target="${textareaId}" title="${t("promptTranslateBtnTitle")}" style="${on ? "" : "display:none;"}">🌐</button>`
|
||||
+ `<button class="wfm-btn wfm-btn-sm wfm-pt-restore" data-target="${textareaId}" title="${t("promptTranslateRestoreTitle")}" style="display:none;">${t("promptTranslateRestore")}</button>`;
|
||||
}
|
||||
@@ -9,6 +9,7 @@ import { t, getLang, getSummaryLang, setLang, setSummaryLang, getLanguageOptions
|
||||
import { getSettings, readJsonStorage, escapeHtml } from "./util.js";
|
||||
import { DECISION_BACKENDS, getDecisionSettings, saveDecisionSettings, testDecisionConnection, listDecisionModels } from "./decision-client.js";
|
||||
import { initVramSettings } from "./vram-prepare.js";
|
||||
import { initPromptTranslateSettings, promptTranslateSettingsHtml } from "./prompt-translate-settings.js";
|
||||
|
||||
const SETTINGS_KEY = "wfm_settings";
|
||||
|
||||
@@ -772,6 +773,9 @@ export async function initSettingsTab() {
|
||||
<button class="wfm-btn wfm-btn-primary wfm-btn-sm" id="wfm-settings-default-ckpt-save">${t("save")}</button>
|
||||
</details>
|
||||
|
||||
<!-- Prompt Translation (native language -> English, PLaMo 3 Translate via Ollama) -->
|
||||
${promptTranslateSettingsHtml(serverSettings)}
|
||||
|
||||
<!-- CivitAI Settings -->
|
||||
<details class="wfm-settings-section">
|
||||
<summary class="wfm-settings-summary">${t("civitaiApiKeySetting")}</summary>
|
||||
@@ -1228,6 +1232,9 @@ export async function initSettingsTab() {
|
||||
}
|
||||
});
|
||||
|
||||
// --- Prompt translation ---
|
||||
initPromptTranslateSettings(serverSettings, saveServerSettings);
|
||||
|
||||
// --- Windows selector event loop (needs ComfyUI restart) ---
|
||||
document.getElementById("wfm-settings-selector-loop")?.addEventListener("change", async (e) => {
|
||||
const enabled = !!e.target.checked;
|
||||
|
||||
@@ -199,6 +199,35 @@ function _updateVlmBackendIndicators() {
|
||||
|
||||
// ── モデル一覧ロード ─────────────────────────────────────────────
|
||||
|
||||
// PixAI Tagger はWD系と推奨しきい値が違う(general 0.17 / character 0.27)ため、
|
||||
// モデル切替時にスライダーが各モデルの既定値のままなら、新モデルの既定値へ切り替える。
|
||||
const _WD_DEFAULTS = { thr: 0.35, chr: 0.85 };
|
||||
const _PIXAI_DEFAULTS = { thr: 0.17, chr: 0.27 };
|
||||
let _pixaiModels = new Set();
|
||||
let _thresholdDefaultsBound = false;
|
||||
|
||||
function _setupModelThresholdDefaults() {
|
||||
if (_thresholdDefaultsBound) return;
|
||||
_thresholdDefaultsBound = true;
|
||||
[
|
||||
["wfm-tagger-model", "wfm-tagger-threshold", "wfm-tagger-char-threshold"],
|
||||
["wfm-tagger-batch-model", "wfm-tagger-batch-threshold", "wfm-tagger-batch-char-threshold"],
|
||||
].forEach(([modelId, thrId, chrId]) => {
|
||||
const sel = document.getElementById(modelId);
|
||||
if (!sel) return;
|
||||
sel.addEventListener("change", () => {
|
||||
const to = _pixaiModels.has(sel.value) ? _PIXAI_DEFAULTS : _WD_DEFAULTS;
|
||||
const from = to === _PIXAI_DEFAULTS ? _WD_DEFAULTS : _PIXAI_DEFAULTS;
|
||||
[[thrId, from.thr, to.thr], [chrId, from.chr, to.chr]].forEach(([id, oldDef, newDef]) => {
|
||||
const slider = document.getElementById(id);
|
||||
if (!slider || Math.abs(parseFloat(slider.value) - oldDef) > 0.005) return;
|
||||
slider.value = newDef;
|
||||
slider.dispatchEvent(new Event("input"));
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function _loadModels() {
|
||||
try {
|
||||
const res = await fetch("/wfm/tagger/models");
|
||||
@@ -210,6 +239,8 @@ async function _loadModels() {
|
||||
sel.innerHTML = `<option value="">${t("taggerNoModel")}</option>` +
|
||||
models.map(m => `<option value="${m.name}">[${m.type}] ${m.name}</option>`).join("");
|
||||
});
|
||||
_pixaiModels = new Set(models.filter(m => m.pixai).map(m => m.name));
|
||||
_setupModelThresholdDefaults();
|
||||
} catch (e) {
|
||||
console.warn("tagger: _loadModels error", e);
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -13,7 +13,7 @@ import { setSourcePreview } from "./video-preview.js";
|
||||
import { showToast } from "./app.js";
|
||||
import { t } from "./i18n.js";
|
||||
import { addClipFromFile, setBgmFromFile, addPipFromFile, addSoundFromFile } from "./video-edit-tab.js";
|
||||
import { setBlockImageFromFile } from "./video-plan-tab.js";
|
||||
import { setBlockImageFromFile, isReferenceImagePlan } from "./video-plan-tab.js";
|
||||
|
||||
// Sentinel for the "All Video Assets" option — not a real backend group, since
|
||||
// Gallery's group filter only ever matches one group at a time. Selecting it
|
||||
@@ -249,8 +249,8 @@ function _renderDetail(img) {
|
||||
<button type="button" class="wfm-btn wfm-btn-sm" id="wfm-video-asset-set-bgm" style="width:100%;margin-top:${isAudioFile(img) ? "12" : "6"}px;display:${isVideoFile(img) || isAudioFile(img) ? "block" : "none"};">${t("videoAssetSetAsBgm")}</button>
|
||||
<button type="button" class="wfm-btn wfm-btn-sm" id="wfm-video-asset-add-sound" style="width:100%;margin-top:6px;display:${isAudioFile(img) ? "block" : "none"};">${t("videoAssetAddAsSound")}</button>
|
||||
<div id="wfm-video-asset-set-plan-image-row" style="display:${isVideoFile(img) || isAudioFile(img) ? "none" : "flex"};gap:6px;margin-top:6px;">
|
||||
<button type="button" class="wfm-btn wfm-btn-sm" id="wfm-video-asset-set-first" style="flex:1;">${t("videoAssetSetAsFirst")}</button>
|
||||
<button type="button" class="wfm-btn wfm-btn-sm" id="wfm-video-asset-set-last" style="flex:1;">${t("videoAssetSetAsLast")}</button>
|
||||
<button type="button" class="wfm-btn wfm-btn-sm" id="wfm-video-asset-set-first" style="flex:1;">${t(isReferenceImagePlan() ? "videoAssetSetAsRef1" : "videoAssetSetAsFirst")}</button>
|
||||
<button type="button" class="wfm-btn wfm-btn-sm" id="wfm-video-asset-set-last" style="flex:1;">${t(isReferenceImagePlan() ? "videoAssetSetAsRef2" : "videoAssetSetAsLast")}</button>
|
||||
</div>
|
||||
<button type="button" class="wfm-btn wfm-btn-primary wfm-btn-sm" id="wfm-video-asset-open-gallery" style="width:100%;margin-top:6px;">Open in Gallery</button>
|
||||
`;
|
||||
@@ -353,7 +353,11 @@ function _renderDetail(img) {
|
||||
const file = new File([blob], img.filename, { type: blob.type || "image/png" });
|
||||
await setBlockImageFromFile(which, file);
|
||||
document.querySelector('.wfm-video-center-panel .wfm-video-subtab-btn[data-video-subtab="plan"]')?.click();
|
||||
showToast(t(which === "first" ? "videoAssetSetAsFirstDone" : "videoAssetSetAsLastDone", img.filename), "success");
|
||||
const ref = isReferenceImagePlan();
|
||||
const doneKey = which === "first"
|
||||
? (ref ? "videoAssetSetAsRef1Done" : "videoAssetSetAsFirstDone")
|
||||
: (ref ? "videoAssetSetAsRef2Done" : "videoAssetSetAsLastDone");
|
||||
showToast(t(doneKey, img.filename), "success");
|
||||
} catch (err) {
|
||||
showToast(t("errorWithMsg", err.message), "error");
|
||||
}
|
||||
|
||||
@@ -83,7 +83,8 @@ import {
|
||||
setSourcePreview, setResultPreview, getActivePreviewVideoElement, getResultPreviewVideoElement,
|
||||
getPreviewPaneElements,
|
||||
} from "./video-preview.js";
|
||||
import { VTEMP_GROUP, ensureVideoGroup } from "./gallery-tab.js";
|
||||
import { VTEMP_GROUP, ensureVideoGroup, openAudioConvertDialog } from "./gallery-tab.js";
|
||||
import { escapeHtml } from "./util.js";
|
||||
|
||||
const _s = {
|
||||
clips: [], // { id, name, file, kind:"video"|"image", serverRef:{filename,subfolder,type}|null, duration, width, height, fps, hasAudio, trimStart, trimEnd, crop:{x,y,w,h}(0..1)|null, probing, error }
|
||||
@@ -1057,13 +1058,13 @@ function _renderCropSection(clip) {
|
||||
.map((k) => `<option value="${k}">${k === "free" ? t("videoEditCropFree") : k}</option>`)
|
||||
.join("");
|
||||
host.innerHTML = `
|
||||
<div class="wfm-video-edit-section-head">
|
||||
<div class="wfm-video-edit-section-head wfm-video-edit-crop-head">
|
||||
<span>${t("videoEditCrop")}</span>
|
||||
<span style="display:flex;gap:6px;">
|
||||
<button type="button" class="wfm-btn wfm-btn-xs" id="wfm-video-edit-crop-toggle"></button>
|
||||
<button type="button" class="wfm-btn wfm-btn-xs" id="wfm-video-edit-crop-cancel" style="display:${_cropEditing ? "" : "none"};">${t("videoEditCropCancel")}</button>
|
||||
<button type="button" class="wfm-btn wfm-btn-xs" id="wfm-video-edit-crop-reset">${t("videoEditCropReset")}</button>
|
||||
</span>
|
||||
<button type="button" class="wfm-btn wfm-btn-xs" id="wfm-video-edit-crop-toggle"></button>
|
||||
<button type="button" class="wfm-btn wfm-btn-xs" id="wfm-video-edit-crop-cancel" style="display:${_cropEditing ? "" : "none"};">${t("videoEditCropCancel")}</button>
|
||||
<button type="button" class="wfm-btn wfm-btn-xs" id="wfm-video-edit-crop-reset">${t("videoEditCropReset")}</button>
|
||||
<span class="wfm-video-edit-head-sep" aria-hidden="true">|</span>
|
||||
<button type="button" class="wfm-btn wfm-btn-xs" id="wfm-video-edit-audio-convert" title="${escapeHtml(t("videoEditAudioConvertTitle"))}">${t("videoEditAudioConvert")}</button>
|
||||
</div>
|
||||
<div class="wfm-video-edit-crop-row">
|
||||
<label>${t("videoEditCropAspect")}<select class="wfm-input" id="wfm-video-edit-crop-aspect">${aspectOptions}</select></label>
|
||||
@@ -1089,6 +1090,7 @@ function _renderCropSection(clip) {
|
||||
clip.crop = _cropEditing ? { x: 0, y: 0, w: 1, h: 1 } : null;
|
||||
_commitCrop(clip);
|
||||
});
|
||||
document.getElementById("wfm-video-edit-audio-convert").addEventListener("click", () => _openClipAudioConvert(clip));
|
||||
aspect.addEventListener("change", () => {
|
||||
_cropAspect = aspect.value;
|
||||
if (clip.crop) _applyCropAspect(clip);
|
||||
@@ -1097,6 +1099,31 @@ function _renderCropSection(clip) {
|
||||
_updateCropInfo(clip);
|
||||
}
|
||||
|
||||
// Audio conversion for the selected clip: reuses Gallery's convert dialog, but the
|
||||
// clip lives in ComfyUI's input folder, so the backend writes the result into
|
||||
// output/video (visible in Gallery) instead of next to the source.
|
||||
function _openClipAudioConvert(clip) {
|
||||
if (clip.probing || !clip.serverRef) { showToast(t("videoEditClipNotReady"), "info"); return; }
|
||||
if (clip.kind !== "video" || !clip.hasAudio) { showToast(t("convertNoTarget"), "info"); return; }
|
||||
openAudioConvertDialog({
|
||||
count: 1,
|
||||
hasVideo: true,
|
||||
allowDelete: false,
|
||||
allowCopyMeta: false,
|
||||
convertOne: async (opts) => {
|
||||
const res = await fetch("/api/wfm/video/edit/convert-audio", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ ...clip.serverRef, ...opts }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (!data.ok) throw new Error(data.error || `HTTP ${res.status}`);
|
||||
showToast(t("videoEditAudioConvertSaved", data.filename), "success");
|
||||
return data;
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Leaves crop-edit mode. keep=true ("Done", or leaving the Edit subtab /
|
||||
// selecting another clip) keeps the adjusted rect; keep=false ("Cancel")
|
||||
// puts back the crop the clip had when editing started.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user