Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fb0c8cc979 | ||
|
|
bc436a4435 | ||
|
|
4ed361b78a | ||
|
|
821bee4b6e | ||
|
|
63c3321aa6 | ||
|
|
cbedb973aa | ||
|
|
4ec1861790 | ||
|
|
47408506ba | ||
|
|
333c5e5021 |
@@ -14,3 +14,8 @@
|
||||
# 3. Restart ComfyUI
|
||||
|
||||
UNSLOTH_API_KEY=
|
||||
|
||||
# ComfyUI-Workflow-Studio - Gallery Semantic Search (Gemini Embedding 2)
|
||||
# Get a key at https://aistudio.google.com (free tier available).
|
||||
# Same steps as above: copy to ".env", paste the key, restart ComfyUI.
|
||||
GEMINI_API_KEY=
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
Condensed version-by-version index of DEVLOG.md (each line: version + one-line summary of what changed and why). Full rationale, code details, and "How to apply" lessons live in the excluded DEVLOG.md itself; this index exists so graphify can extract a queryable semantic node per release without feeding the full log to the local Ollama model. Regenerate with `python tools/generate_doc_index.py devlog` whenever DEVLOG.md changes.
|
||||
|
||||
- **v0.8.2(2026-10-12)** — Gallery: セマンティック検索(Gemini Embedding 2 / EmbeddingGemma 2)
|
||||
- **v0.8.1(2026-10-11)** — バグ修正
|
||||
- **v0.8.0(2026-10-11)** — Gallery: 音声ファイル対応(wav / mp3 / ogg / m4a / flac / opus)
|
||||
- **v0.7.14(2026-10-09)** — Comic Creator連携: nanobananaタブの生成履歴をGallery/Metadataタブに表示
|
||||
- **v0.7.13(2026-10-06)** — VRAM調整(Ollama)— Comic Creatorから移植
|
||||
|
||||
@@ -2,6 +2,57 @@
|
||||
|
||||
---
|
||||
|
||||
## v0.8.2(2026-10-12)
|
||||
|
||||
### Gallery: セマンティック検索(Gemini Embedding 2 / EmbeddingGemma 2)
|
||||
|
||||
画像を文章の意味で探す検索と、選択画像に類似した画像の一覧表示を追加した。
|
||||
|
||||
**UI**
|
||||
- 検索ボックス横に「S」ボタン。ONで検索ボックスが意味検索になる(Enterで実行、スコア順、サムネイルにスコアバッジ)。OFFは従来のファイル名/タグ/プロンプト検索。
|
||||
- Sを入れるとツールバーに2行目が現れ、`Similar`(トグル)・`↻`(更新)・`⚙`(設定)・検索結果件数を左詰めで表示する。Similarは、類似一覧の中で別の画像を選んでも一覧はそのままで、↻でその画像の類似を再表示、Similar再押下で通常表示に戻る。フォルダ切替・フィルタクリア・S OFFでも解除。Similar/↻は当初詳細パネルに置いていたが2行目へ移設(Taggerの紫と区別するためティール)。
|
||||
- ⚙のモーダルを3タブ化: インデックス(対象フォルダ・サブフォルダ・新規のみ/全件再作成・進捗・停止・全削除)/ モデル設定(モデルのラジオボタン・次元・ローカルのデバイス/CPUスレッド/画像精度・接続テスト)/ 検索設定(類似画像と文章検索の最小スコア、最大表示件数)。「?」でヘルプの該当項目へジャンプ。
|
||||
- 最小スコア(例: 0.6以上)はモデルごとに保存(スコアの水準がモデルで違うため。少数画像の実測では、類似画像はEmbeddingGemma 2で約0.6〜0.8、Geminiで約0.6、文章検索はそれぞれ約0.6〜0.7・約0.3)。
|
||||
- フォルダツリーの枚数バッジの右に緑のドット = そのフォルダ直下の画像が全件インデックス済み(`GET /wfm/gallery/semantic/folders`)。インデックスは全フォルダ共通の1つで、フォルダ単位で少しずつ作れる。
|
||||
- ヘルプページの幅を760→1076px(上部タブバーのAI TOOLタブの右端に揃える)。
|
||||
|
||||
**バックエンド**
|
||||
- `py/services/semantic_service.py`: ベクトルを `semantic_index.npz` に保存(パス→正規化ベクトル、サイズ/mtimeで鮮度判定、モデル/次元/画像精度をヘッダに保持)。インデックス作成はバックグラウンドスレッド(進捗ポーリング・停止可)、検索はコサイン類似度(`mat @ q`)、未インデックスの基準画像はその場で1枚だけ埋め込む。動画/PSDはサムネイルから、音声は対象外。
|
||||
- Gemini: `gemini-embedding-2`(`x-goog-api-key`ヘッダ、接続先固定、429/5xxは指数バックオフで再試行)。APIキーは `.env` の `GEMINI_API_KEY`(UNSLOTHと同じく settings.json・フロントには出さない)。
|
||||
- EmbeddingGemma 2: `py/services/semantic_local.py`。onnx-community/embeddinggemma-2-ONNX を `onnxruntime` + `tokenizers` + `huggingface_hub` だけで実行(`transformers` の対応を待たない)。Transformers.jsの `Gemma4ImageProcessor` / `EmbeddingGemma2Processor` を移植(アスペクト比維持リサイズ → 16pxパッチ化 → vision_encoder → `<|image|>` 位置へ `image_features` を差し込み → `sentence_embedding`)。クエリは `task: search result | query: ` プレフィックス。MRL(128/256/512/768)で切り詰め+再正規化。
|
||||
- 実行デバイス: 自動(CUDAがあればGPU)/ CPU / GPU(CUDA)。CPUスレッド数の既定はコアの半分。設定は `semantic_settings.json` に保存。画像精度はソフトトークン数 280/140/70(高精度/標準/軽量、CPUで約2.5倍/約5倍速)でインデックスに記録し、変えたら全件再作成が必要。インデックス作成後はセッションを解放しVRAM/RAMを画像生成に譲る。
|
||||
|
||||
**実測(RTX 4070 / 32コア)**
|
||||
- 768×576の画像1枚あたり: CUDA 0.13秒(標準 0.07秒)、CPU 4スレッド 5.8秒(標準 2.3秒、軽量 1.1秒)。CPUとCUDAのベクトルは一致(cos 1.0)。実機で122枚を約20秒で作成。
|
||||
- DirectMLはvision_encoderのReshapeノードで実行時エラーとなり非対応(テキストモデルのみ動作)。WindowsMLは未検証。
|
||||
- 環境メモ: ComfyUI_5のvenvには `onnxruntime` と `onnxruntime-gpu`(ともに1.30.0)が同居し、CPU版が勝っていたため、両方を削除して `onnxruntime-gpu==1.30.0` のみを入れ直した(`pip check` は `onnxruntime` 名での依存を警告するが動作に影響なし)。
|
||||
|
||||
**安全性(セキュリティレビュー指摘への対応)**
|
||||
- 数値パラメータ(dim / soft / threads / limit / min_score)を共通関数で範囲内に丸め、数値でない値は500ではなく400(`bad_param`)で返す。スレッド数はコア数の2倍まで。
|
||||
- ローカルモデルへの入力を制限: テキスト1024トークン、検索文2000文字、画像は長辺2048pxを超えたら先に縮小。
|
||||
|
||||
**その他**
|
||||
- `.env.example` に `GEMINI_API_KEY`、`requirements.txt` にGPU用の手順コメントを追加。ヘルプ(3言語)・READMEを更新。
|
||||
- Gemini Embedding 2には無料枠があるが、無料枠で送った内容はGoogleの製品改善に使われる場合がある(2026年10月時点の公式料金: 画像1枚あたり約$0.00012)。
|
||||
|
||||
## v0.8.1(2026-10-11)
|
||||
|
||||
v0.8.0 のコードレビュー指摘(#1〜#10)への対応。
|
||||
|
||||
### バグ修正
|
||||
- Nodes: 検索結果が0件のとき `renderToken` を進めておらず、描画途中のカードが「No nodes found」の下に追加され続けていた。
|
||||
- Metadata: `convertUiToApi()` に `{ readOnly: true }` を追加。表示用の変換ではCOMBO値の無警告差し替え(サンプラー名の誤表示)を行わず、Workflowタブの互換性チェックが参照する `_last*` 記録も更新しない。
|
||||
- Metadata: 画像を連続で開いたとき、先に始めた処理の結果が後から描画されて混ざる問題を、リクエストID(`_loadSeq`)で防止。
|
||||
- Metadata: サンプラー設定の抽出を改善。Flux/SD3系(RandomNoise / BasicScheduler / KSamplerSelect / CFGGuider)の設定を統合し、複数KSamplerでは設定が最も多いノード(同数なら後ろ)を採用。
|
||||
- Gallery: `history.jsonl` に dict でない行・`filenames` が list でない行があっても、フォルダ内の全画像でメタ取得が失敗しないようにした。
|
||||
- Gallery: 同じファイルを同時に変換すると出力名が衝突して上書きされる問題を、出力名の決定を `_convert_lock` 内に移して解消。
|
||||
- Gallery: PNGチャンクの読み取りを先勝ちに統一(`png_extractor` と同じ)。ワークフロー読み込みとの食い違いを防止。
|
||||
|
||||
### パフォーマンス・保守性
|
||||
- Gallery: 一括選択の変換対象判定を O(選択数×画像数) から O(選択数) に(パス→画像Mapのキャッシュ + `some` で打ち切り)。12,780枚の全選択で約33ms。
|
||||
- Gallery: `history.jsonl` を `(mtime_ns, size)` キーの索引としてキャッシュ(`OrderedDict`、上限16件)。
|
||||
- 音声・動画の拡張子判定を `isAudioFile()` / `isVideoFile()` に統一し、波形サムネイルの版数を `WAVE_THUMB_VERSION` 定数に集約(JS/Python)。
|
||||
|
||||
## v0.8.0(2026-10-11)
|
||||
|
||||
### Gallery: 音声ファイル対応(wav / mp3 / ogg / m4a / flac / opus)
|
||||
|
||||
+4
-2
@@ -11,6 +11,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Workflow, model, image, and prompt management
|
||||
- AI-powered prompt writing assistance, translation, and tag generation
|
||||
- File drop & Gallery-linked metadata viewing — inspect and reuse generation settings instantly
|
||||
- Semantic image search in the Gallery — find images by describing them, or list images similar to a selected one
|
||||
|
||||
**⚡ GenerateUI** — a productivity-focused generation tab connected to every other tab
|
||||
|
||||
@@ -26,7 +27,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Built-in AI tools (translation and more)
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Features (tab -> feature names)
|
||||
|
||||
@@ -52,7 +53,7 @@ Setting / Results / Plan JSON sub-panels, Per-column keyframes, Model column —
|
||||
2-column layout, Collapsible sections, Theme selection, Theme customization, Workflows directory, Gallery output directory, Eagle connection, CivitAI Host, CivitAI API Key, Default workflow, Default Checkpoint, VRAM Management (Ollama), Video Playback Volume, Data Management, Text Size, RAW JSON Colors, GenerateUI Model Tab Highlight, Wildcard Integration, G'MIC-Qt Integration, Language
|
||||
|
||||
### Gallery Tab (v0.3.44)
|
||||
Output / ImagePrompt / Style_Catalog / Metadata sub-tabs, Image browser, Folder tree root label, Thumbnail / Table views, Folder management, File operations, Download, MP4 video support, MP4 embedded metadata, WebM support, Embedded metadata reading fixes, Audio support, File conversion, Move To folder filter, Audio waveform colors, Info tab: dimensions & duration, Multi-select, Image Compare, Prompt search, Server-side filtering, Group management, Thumbnail F button, Favorites, Detail panel, Prompt tab, Nanobanana tab, Workflow viewer, GenUI button, Image Edit button, Send GenUI Image button, Send to LI node button, Send to LV node button, Send CC button, Search clear (✕), Clear all filters (✕ Clear), Workflow auto-save, Output folder configurable, SVG file support, PSD file support, Performance
|
||||
Output / ImagePrompt / Style_Catalog / Metadata sub-tabs, Image browser, Folder tree root label, Thumbnail / Table views, Folder management, File operations, Download, MP4 video support, MP4 embedded metadata, WebM support, Embedded metadata reading fixes, Audio support, File conversion, Move To folder filter, Semantic Search, Audio waveform colors, Info tab: dimensions & duration, Multi-select, Image Compare, Prompt search, Server-side filtering, Group management, Thumbnail F button, Favorites, Detail panel, Prompt tab, Nanobanana tab, Workflow viewer, GenUI button, Image Edit button, Send GenUI Image button, Send to LI node button, Send to LV node button, Send CC button, Search clear (✕), Clear all filters (✕ Clear), Workflow auto-save, Output folder configurable, SVG file support, PSD file support, Performance
|
||||
|
||||
### ImagePrompt Gallery subtab (v0.3.94, renamed from Style/Prompt in v0.3.95)
|
||||
Visual prompt library, 3-column layout, Prompt builder (right pane), Plain-text prompt storage, ponyxlWildcardsVault format support, Seed data importer
|
||||
@@ -158,6 +159,7 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** (no key; `onnxruntime`, `tokenizers` and `huggingface_hub` — normally already present with ComfyUI; the model downloads on first use). `onnxruntime-gpu` makes the local model ≈45× faster on NVIDIA GPUs
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
|
||||
@@ -7,6 +7,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Workflow, model, image, and prompt management
|
||||
- AI-powered prompt writing assistance, translation, and tag generation
|
||||
- File drop & Gallery-linked metadata viewing — inspect and reuse generation settings instantly
|
||||
- Semantic image search in the Gallery — find images by describing them, or list images similar to a selected one
|
||||
|
||||
**⚡ GenerateUI** — a productivity-focused generation tab connected to every other tab
|
||||
|
||||
@@ -22,7 +23,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
|
||||
- Built-in AI tools (translation and more)
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Screenshots
|
||||
|
||||
@@ -268,6 +269,10 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **Audio support** (v0.8.0) — `.wav` / `.mp3` / `.ogg` / `.m4a` / `.flac` / `.opus` files are browsable alongside images and videos (for ComfyUI music generation output): thumbnails are waveform images generated server-side (PyAV, cached), the detail panel / lightbox / Compare show the waveform with a built-in audio player (shared volume setting with videos), the Info tab shows the duration, and the Prompt / JSON tabs read the `prompt` / `workflow` tags that SaveAudio-type nodes embed (FLAC / MP3 / Opus); favorites, tags, groups, move, delete, and export work as for other files, while image-only actions (PSD export, Edit Layers) skip audio; in the Video tab's Asset sub-tab, audio files added to the Video Assets group appear with an "Audio Only" filter, an inline player, and **Set as BGM** / **Add as sound** buttons that feed the Edit sub-tab
|
||||
- **File conversion** (v0.8.0) — a **Convert...** button in the Gallery bulk bar (shown when the selection contains audio or video) converts audio files to FLAC / MP3 / Opus / M4A / WAV, or extracts the audio track of videos, using PyAV (no system ffmpeg needed); bitrate, sample rate (44.1 / 48 kHz) and channels (mono / stereo) are selectable, output goes next to the source as a new file (`name.ext`, or `name_converted.ext` if taken). ComfyUI `prompt` / `workflow` tags (and title / artist / album tags) are carried over to formats that can hold them — WAV cannot, so they are skipped there; favorite / tags / memo / groups can be carried over, and the original can optionally be deleted after a successful conversion (with a confirmation). `.ogg` is readable but not offered as an output (the bundled PyAV has no Vorbis encoder)
|
||||
- **Move To folder filter** (v0.8.0) — the Move To dialog (detail panel and bulk bar) has a filter box that narrows the folder list by name or path (space = AND), Enter moves to the first match; the move toast now reads "Moved N file(s)" since audio/video files are moved too
|
||||
- **Semantic Search** (v0.8.2) — find images by meaning instead of keywords. The **S** button next to the search box turns it on: the search box then takes a free-text description (e.g. "sunset city street"), Enter searches, and results are ordered by similarity with a score badge on each thumbnail. Turning **S** on also shows a second toolbar row (left-aligned): **Similar** (lists images similar to the selected one; selecting another image keeps the list, **↻** re-runs it for the newly selected image, pressing Similar again returns to the normal list), the **⚙** settings button, and the result count. Searches cover the selected folder and its subfolders; sort and tag/group filters do not apply to semantic results
|
||||
- **Index window (⚙)** — three tabs. **Index**: target folder (optionally with subfolders), "new/changed only" or "rebuild all", progress bar, stop, delete all. **Model**: choose with radio buttons between **Gemini Embedding 2** (API; set `GEMINI_API_KEY` in the plugin's `.env` and restart ComfyUI — the free tier exists but content sent on it may be used by Google to improve its products, beyond it the API is paid) and **EmbeddingGemma 2** (runs locally, free, no key; the ~300 MB q4 ONNX model is downloaded on first use), plus vector size (Gemini 768 / 1536 / 3072, EmbeddingGemma 2 128 / 256 / 512 / 768), and for the local model the device (auto / CPU / GPU CUDA), CPU thread count (default: half the cores) and image precision (high / standard ≈2.5× faster / light ≈5× faster). **Search**: minimum score for similar-image results (e.g. `0.6` shows only images scoring 0.6 or higher), minimum score for text search, and maximum results — stored per model because score levels differ between models
|
||||
- **Index state in the folder tree** — a green dot next to a folder's image count means every image directly in that folder is indexed; the index is one shared store, so folders can be indexed one at a time. Images, plus videos / PSD via their thumbnails (audio is skipped), are embedded; changing the model, vector size or image precision requires "Rebuild all"
|
||||
- **Local GPU** — EmbeddingGemma 2 is a port of Transformers.js' pipeline onto `onnxruntime` (no `transformers` support needed). On an RTX 4070 it indexes ≈0.13 s per image with CUDA versus ≈6 s on the CPU (≈45× faster, identical vectors); to use the GPU, install `onnxruntime-gpu` in place of `onnxruntime` (`pip uninstall onnxruntime onnxruntime-gpu` then `pip install onnxruntime-gpu`). DirectML is not supported (its vision model fails). The model is released from memory when indexing finishes
|
||||
- **Audio waveform colors** (v0.8.0) — waveform thumbnails are colored per format (MP3 blue, WAV green, FLAC amber, OGG/Opus purple, M4A rose)
|
||||
- **Info tab: dimensions & duration** (v0.5.1) — the detail panel's Info tab now also shows image dimensions (width×height) or video duration, in addition to filename, size, date, tags, and memo
|
||||
- **Multi-select** — Ctrl+click to select multiple images; Bulk Bar appears for batch operations: select group → "Add to Group" / "Remove from Group"; Favorite All / Unfavorite All; Compare (2–4 images); Move To...; **Export** (downloads all selected images as a ZIP file); **Create PSD** (v0.6.0, combines the selected images into one layered PSD file, positioned top-left on a canvas sized to the largest image); **Edit as Layers** (v0.6.0, opens all selected images in the Image Edit tab at once, each as its own layer); Delete File
|
||||
@@ -595,6 +600,7 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** (no key; `onnxruntime`, `tokenizers` and `huggingface_hub` — normally already present with ComfyUI; the model downloads on first use). `onnxruntime-gpu` makes the local model ≈45× faster on NVIDIA GPUs
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
|
||||
@@ -63,7 +63,7 @@ if ENV_FILE.exists():
|
||||
except ImportError:
|
||||
print(
|
||||
"[WARNING] Workflow Studio: .env file found but python-dotenv is not installed "
|
||||
"(pip install -r requirements.txt). UNSLOTH_API_KEY will not be loaded from .env."
|
||||
"(pip install -r requirements.txt). UNSLOTH_API_KEY / GEMINI_API_KEY will not be loaded from .env."
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"[WARNING] Workflow Studio: failed to load .env file: {e}")
|
||||
|
||||
@@ -10,10 +10,12 @@ from aiohttp import web
|
||||
|
||||
from ..config import DATA_DIR
|
||||
from ..services.gallery_service import GalleryService
|
||||
from ..services.semantic_service import SemanticService, SemanticError, DEFAULT_DIM, MODEL as SEMANTIC_DEFAULT_MODEL
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_service = GalleryService(DATA_DIR)
|
||||
_semantic = SemanticService(DATA_DIR, _service)
|
||||
|
||||
|
||||
def _init_allowed_root() -> None:
|
||||
@@ -63,6 +65,14 @@ for _ext, _mime in ((".wav", "audio/wav"), (".mp3", "audio/mpeg"), (".ogg", "aud
|
||||
|
||||
def setup_routes(app: web.Application):
|
||||
app.router.add_get("/wfm/gallery/folders", list_folders)
|
||||
app.router.add_get("/wfm/gallery/semantic/status", semantic_status)
|
||||
app.router.add_get("/wfm/gallery/semantic/folders", semantic_folders)
|
||||
app.router.add_post("/wfm/gallery/semantic/index", semantic_index)
|
||||
app.router.add_post("/wfm/gallery/semantic/cancel", semantic_cancel)
|
||||
app.router.add_post("/wfm/gallery/semantic/search", semantic_search)
|
||||
app.router.add_post("/wfm/gallery/semantic/similar", semantic_similar)
|
||||
app.router.add_post("/wfm/gallery/semantic/clear", semantic_clear)
|
||||
app.router.add_post("/wfm/gallery/semantic/test", semantic_test)
|
||||
app.router.add_get("/wfm/gallery/images", list_images)
|
||||
app.router.add_get("/wfm/gallery/image/meta", get_image_meta)
|
||||
app.router.add_get("/wfm/gallery/image/workflow", get_image_workflow)
|
||||
@@ -101,6 +111,144 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_get("/wfm/gallery/image/psd-layers", get_psd_layers)
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# セマンティック検索 (Gemini Embedding 2)
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _semantic_error(e: SemanticError) -> web.Response:
|
||||
# code はフロントのi18n用。message は詳細(APIキー等の秘密は含まない)
|
||||
return web.json_response({"error": str(e), "code": e.code}, status=400)
|
||||
|
||||
|
||||
def _num(body: dict, key: str, default, lo, hi, cast=int):
|
||||
"""リクエストの数値パラメータを検証して範囲内に収める。数値でなければ400相当のSemanticErrorにする"""
|
||||
raw = body.get(key, default)
|
||||
if raw is None or raw == "":
|
||||
raw = default
|
||||
try:
|
||||
v = cast(raw)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
raise SemanticError("bad_param", key)
|
||||
if v != v: # NaN
|
||||
raise SemanticError("bad_param", key)
|
||||
return max(lo, min(v, hi))
|
||||
|
||||
|
||||
async def _json_body(request: web.Request) -> dict:
|
||||
try:
|
||||
body = await request.json()
|
||||
return body if isinstance(body, dict) else {}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
async def semantic_status(request: web.Request) -> web.Response:
|
||||
folder = request.rel_url.query.get("folder", "")
|
||||
recursive = request.rel_url.query.get("recursive", "true") == "true"
|
||||
try:
|
||||
return web.json_response(await asyncio.to_thread(_semantic.status, folder, recursive))
|
||||
except Exception as e:
|
||||
logger.error("semantic_status error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_folders(request: web.Request) -> web.Response:
|
||||
root = request.rel_url.query.get("root", "")
|
||||
try:
|
||||
return web.json_response({"folders": await asyncio.to_thread(_semantic.folder_status, root)})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_folders error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
def _apply_local_options(body: dict) -> None:
|
||||
"""リクエストに実行デバイス/スレッド数が含まれていれば反映・保存する"""
|
||||
if "device" in body:
|
||||
_semantic.set_local_options(str(body.get("device", "auto")), _num(body, "threads", 0, 0, 256))
|
||||
|
||||
|
||||
async def semantic_index(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
_apply_local_options(body)
|
||||
await asyncio.to_thread(
|
||||
_semantic.start_index,
|
||||
str(body.get("folder", "")),
|
||||
bool(body.get("recursive", True)),
|
||||
"all" if body.get("mode") == "all" else "missing",
|
||||
str(body.get("model", SEMANTIC_DEFAULT_MODEL)),
|
||||
_num(body, "dim", DEFAULT_DIM, 0, 100000),
|
||||
_num(body, "soft", 280, 0, 100000),
|
||||
)
|
||||
return web.json_response({"ok": True})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_index error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_cancel(request: web.Request) -> web.Response:
|
||||
_semantic.cancel()
|
||||
return web.json_response({"ok": True})
|
||||
|
||||
|
||||
async def semantic_search(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
limit = _num(body, "limit", 200, 1, 1000)
|
||||
min_score = _num(body, "min_score", 0.0, -1.0, 1.0, float)
|
||||
images = await asyncio.to_thread(
|
||||
_semantic.search_text, str(body.get("query", "")), str(body.get("folder", "")),
|
||||
bool(body.get("recursive", True)), limit, min_score,
|
||||
)
|
||||
return web.json_response({"images": images, "total": len(images)})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_search error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_similar(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
limit = _num(body, "limit", 200, 1, 1000)
|
||||
min_score = _num(body, "min_score", 0.0, -1.0, 1.0, float)
|
||||
images = await asyncio.to_thread(
|
||||
_semantic.search_similar, str(body.get("path", "")), str(body.get("folder", "")),
|
||||
bool(body.get("recursive", True)), limit, min_score,
|
||||
)
|
||||
return web.json_response({"images": images, "total": len(images)})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_similar error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_clear(request: web.Request) -> web.Response:
|
||||
try:
|
||||
await asyncio.to_thread(_semantic.clear)
|
||||
return web.json_response({"ok": True})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
|
||||
|
||||
async def semantic_test(request: web.Request) -> web.Response:
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
_apply_local_options(body)
|
||||
res = await asyncio.to_thread(_semantic.test_connection, str(body.get("model", SEMANTIC_DEFAULT_MODEL)), _num(body, "dim", DEFAULT_DIM, 0, 100000))
|
||||
return web.json_response({"ok": True, **res})
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# フォルダツリー
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -11,6 +11,7 @@ import re
|
||||
import struct
|
||||
import threading
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -23,6 +24,9 @@ VIDEO_EXTENSIONS = {".mp4", ".webm"}
|
||||
# 音声拡張子(ComfyUIの音楽生成出力)。動画と同様にPyAVで再生時間・埋め込みメタデータを読む。
|
||||
# サムネイルは波形画像を生成する。
|
||||
AUDIO_EXTENSIONS = {".wav", ".mp3", ".ogg", ".m4a", ".flac", ".opus"}
|
||||
# 波形サムネイルの版数(描画仕様を変えたら上げる)。static/js/gallery-tab.js の WAVE_THUMB_VERSION と同じ値にすること。
|
||||
WAVE_THUMB_VERSION = "wave2"
|
||||
_NB_INDEX_CACHE_MAX = 16
|
||||
|
||||
# 音声サムネイル(波形)の色。拡張子ごとに変えて、一覧で形式を見分けやすくする。
|
||||
AUDIO_WAVE_COLORS = {
|
||||
@@ -197,6 +201,8 @@ class _FolderCache:
|
||||
|
||||
|
||||
class GalleryService:
|
||||
_nb_index_cache: "OrderedDict[str, tuple]" = OrderedDict() # history.jsonlの索引キャッシュ(上限付き)
|
||||
_nb_index_lock = threading.Lock()
|
||||
_convert_lock = threading.Lock() # 変換は同時に1本だけ実行する
|
||||
|
||||
def __init__(self, data_dir: Path):
|
||||
@@ -438,6 +444,21 @@ class GalleryService:
|
||||
_walk(folder)
|
||||
return entries
|
||||
|
||||
def make_list_item(self, name: str, abs_path: str, size: int, mtime: float) -> dict:
|
||||
"""一覧アイテムdict(list_imagesと同形式)を作る。セマンティック検索の結果表示用。"""
|
||||
meta = self.metadata_store.get(abs_path)
|
||||
return {
|
||||
"filename": name,
|
||||
"path": abs_path,
|
||||
"size": size,
|
||||
"mtime": mtime,
|
||||
"ext": Path(name).suffix.lower(),
|
||||
"favorite": meta.get("favorite", False),
|
||||
"tags": meta.get("tags", []),
|
||||
"memo": meta.get("memo", ""),
|
||||
"groups": meta.get("groups", []),
|
||||
}
|
||||
|
||||
def list_images(
|
||||
self,
|
||||
folder_path: str,
|
||||
@@ -602,14 +623,29 @@ class GalleryService:
|
||||
if image_path.parent.name != NANOBANANA_FOLDER_NAME:
|
||||
return None
|
||||
history_path = image_path.parent / "history.jsonl"
|
||||
if not history_path.is_file():
|
||||
index = self._load_nanobanana_index(history_path)
|
||||
return index.get(image_path.name) if index else None
|
||||
|
||||
def _load_nanobanana_index(self, history_path: Path) -> dict | None:
|
||||
"""history.jsonl を「ファイル名→エントリ」の辞書にして返す。(mtime, size)をキーに
|
||||
キャッシュし、メタデータ要求のたびの全読み込みを避ける。同名再生成は新しい行を優先する。"""
|
||||
try:
|
||||
st = history_path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
key = str(history_path)
|
||||
sig = (st.st_mtime_ns, st.st_size)
|
||||
with self._nb_index_lock:
|
||||
cached = self._nb_index_cache.get(key)
|
||||
if cached and cached[0] == sig:
|
||||
self._nb_index_cache.move_to_end(key)
|
||||
return cached[1]
|
||||
try:
|
||||
lines = history_path.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||
except OSError:
|
||||
return None
|
||||
filename = image_path.name
|
||||
for line in reversed(lines):
|
||||
index: dict = {}
|
||||
for line in reversed(lines): # 新しい行から。最初に見つかったものを優先
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
@@ -617,9 +653,20 @@ class GalleryService:
|
||||
entry = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
if filename in (entry.get("filenames") or []):
|
||||
return entry
|
||||
return None
|
||||
if not isinstance(entry, dict): # 破損行([]・数値など)は無視
|
||||
continue
|
||||
filenames = entry.get("filenames")
|
||||
if not isinstance(filenames, list):
|
||||
continue
|
||||
for name in filenames:
|
||||
if isinstance(name, str) and name not in index:
|
||||
index[name] = entry
|
||||
with self._nb_index_lock:
|
||||
self._nb_index_cache[key] = (sig, index)
|
||||
self._nb_index_cache.move_to_end(key)
|
||||
while len(self._nb_index_cache) > _NB_INDEX_CACHE_MAX:
|
||||
self._nb_index_cache.popitem(last=False)
|
||||
return index
|
||||
|
||||
def _find_vault_category_root(self, image_path: Path) -> Path | None:
|
||||
"""image_path の祖先を辿り、直下に thumbnails(_option2) を持つフォルダ
|
||||
@@ -781,7 +828,10 @@ class GalleryService:
|
||||
rest = rest[null_idx2 + 1:]
|
||||
null_idx3 = rest.index(b"\x00")
|
||||
value = rest[null_idx3 + 1:].decode("utf-8", errors="replace")
|
||||
result[key] = value
|
||||
# 先勝ち: png_extractor.extract_png_workflow(ワークフロー読み込み)と
|
||||
# 同じチャンクを採用し、Gallery表示との食い違いを防ぐ。
|
||||
if key not in result:
|
||||
result[key] = value
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
pass
|
||||
# IENDで止めない: ComfyUI-Custom-Scripts等はIENDの後ろに
|
||||
@@ -1002,18 +1052,20 @@ class GalleryService:
|
||||
return default
|
||||
|
||||
ext = OUTPUT_FORMATS[fmt]["ext"]
|
||||
dst = p.with_suffix(ext)
|
||||
if dst.exists(): # 同名(同形式への再変換を含む)は上書きせず連番
|
||||
counter = 1
|
||||
while True:
|
||||
dst = p.with_name(f"{p.stem}_converted{'' if counter == 1 else counter}{ext}")
|
||||
if not dst.exists():
|
||||
break
|
||||
counter += 1
|
||||
|
||||
inherit = bool(opts.get("inherit_metadata", True))
|
||||
try:
|
||||
with self._convert_lock:
|
||||
|
||||
# 出力名の決定〜変換完了までを同一ロック内で行う(同時リクエストで同じ名前を選んで
|
||||
# 上書きし合うのを防ぐ)。エラー時に消すのは自分が決めたdstだけ。
|
||||
with self._convert_lock:
|
||||
dst = p.with_suffix(ext)
|
||||
if dst.exists(): # 同名(同形式への再変換を含む)は上書きせず連番
|
||||
counter = 1
|
||||
while True:
|
||||
dst = p.with_name(f"{p.stem}_converted{'' if counter == 1 else counter}{ext}")
|
||||
if not dst.exists():
|
||||
break
|
||||
counter += 1
|
||||
try:
|
||||
result = convert_audio(
|
||||
p, dst, fmt,
|
||||
bitrate_kbps=_int("bitrate"),
|
||||
@@ -1021,13 +1073,13 @@ class GalleryService:
|
||||
channels=str(opts.get("channels", "keep")),
|
||||
inherit_metadata=inherit,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("convert_file: failed for %s: %s", p, e)
|
||||
try:
|
||||
dst.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
return {"ok": False, "error": str(e)}
|
||||
except Exception as e:
|
||||
logger.warning("convert_file: failed for %s: %s", p, e)
|
||||
try:
|
||||
dst.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
return {"ok": False, "error": str(e)}
|
||||
|
||||
dst_str = str(dst.resolve()).replace("\\", "/")
|
||||
if opts.get("copy_gallery_meta", True):
|
||||
@@ -1577,7 +1629,7 @@ class GalleryService:
|
||||
return None
|
||||
|
||||
# 音声は波形の描画仕様(色)を変えたらキャッシュを作り直せるよう版数をキーに含める
|
||||
wave_ver = ":wave2" if p.suffix.lower() in AUDIO_EXTENSIONS else ""
|
||||
wave_ver = f":{WAVE_THUMB_VERSION}" if p.suffix.lower() in AUDIO_EXTENSIONS else ""
|
||||
cache_key = hashlib.md5(f"{p}:{mtime}:{width}{wave_ver}".encode()).hexdigest()
|
||||
cache_dir = self.data_dir / "thumb_cache"
|
||||
cache_dir.mkdir(exist_ok=True)
|
||||
|
||||
@@ -0,0 +1,239 @@
|
||||
"""
|
||||
EmbeddingGemma 2 ローカル実行 (onnx-community/embeddinggemma-2-ONNX)
|
||||
|
||||
onnxruntime + tokenizers + huggingface_hub だけで動かす(transformers の対応を待たない)。
|
||||
前処理は Transformers.js の Gemma4ImageProcessor / EmbeddingGemma2Processor の移植:
|
||||
画像 -> アスペクト比維持でリサイズ(48px倍数, 最大2520パッチ) -> 16pxパッチ化 -> vision_encoder
|
||||
-> image_features を text model の <|image|> トークン位置に差し込み -> sentence_embedding(768)
|
||||
モデルは初回のみ固定リポジトリからダウンロードする(約300MB, q4)。
|
||||
"""
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
REPO_ID = "onnx-community/embeddinggemma-2-ONNX"
|
||||
LOCAL_MODEL = "embeddinggemma-2"
|
||||
LOCAL_DIMS = (128, 256, 512, 768) # MRL対応次元
|
||||
DEFAULT_LOCAL_DIM = 768
|
||||
# 量子化精度 -> ファイル名サフィックス
|
||||
DTYPES = {"q4": "_q4", "q8": "_quantized", "fp16": "_fp16", "fp32": ""}
|
||||
DEFAULT_DTYPE = "q4"
|
||||
|
||||
_PATCH = 16
|
||||
_POOL = 3
|
||||
SOFT_TOKENS = (70, 140, 280) # 画像1枚あたりのソフトトークン数(多いほど高精度・低速)
|
||||
DEFAULT_SOFT = 280
|
||||
_MAX_TOKENS = 1024 # テキスト入力のトークン数上限(画像用プロンプトは約290トークン)
|
||||
_MAX_IMAGE_SIDE = 2048 # 画像の長辺上限(超える場合は縮小してから処理)
|
||||
_IMAGE_TOKEN = "<|image|>"
|
||||
_BOI = "<|image>"
|
||||
_EOI = "<image|>"
|
||||
|
||||
DEVICES = ("auto", "cpu", "cuda", "dml")
|
||||
_PROVIDER_BY_DEVICE = {"cuda": "CUDAExecutionProvider", "dml": "DmlExecutionProvider"}
|
||||
|
||||
_lock = threading.Lock()
|
||||
_runtime = None # ((dtype, device, threads), _Runtime)
|
||||
|
||||
|
||||
_opts = {"device": "auto", "threads": 0} # 実行デバイス/CPUスレッド数(configureで更新)
|
||||
|
||||
|
||||
def configure(device: str = "auto", threads: int = 0) -> None:
|
||||
"""ローカル推論の実行設定。次回のensure_runtimeから反映(device/threads変更時はセッション再作成)。"""
|
||||
_opts["device"] = device if device in DEVICES else "auto"
|
||||
_opts["threads"] = max(0, min(int(threads), (os.cpu_count() or 1) * 2))
|
||||
|
||||
|
||||
def current_config() -> dict:
|
||||
return dict(_opts)
|
||||
|
||||
|
||||
def release() -> None:
|
||||
"""セッションを破棄してVRAM/RAMを解放する(インデックス作成後など)。次回利用時に再ロードされる。"""
|
||||
global _runtime
|
||||
with _lock:
|
||||
_runtime = None
|
||||
|
||||
|
||||
def active_provider() -> str:
|
||||
"""直近にロードしたセッションの実行プロバイダ(未ロードなら空)"""
|
||||
rt = _runtime[1] if _runtime else None
|
||||
return rt.providers[0].replace("ExecutionProvider", "") if rt else ""
|
||||
|
||||
|
||||
def available_devices() -> dict:
|
||||
"""onnxruntime が使える実行デバイス。GPU用ビルド(onnxruntime-gpu / -directml)が無ければcpuのみ。"""
|
||||
try:
|
||||
import onnxruntime as ort
|
||||
prov = ort.get_available_providers()
|
||||
version = ort.__version__
|
||||
except Exception:
|
||||
return {"cpu": True, "cuda": False, "dml": False, "ort_version": ""}
|
||||
return {"cpu": True, "cuda": "CUDAExecutionProvider" in prov, "dml": "DmlExecutionProvider" in prov,
|
||||
"ort_version": version, "cpu_count": os.cpu_count() or 1}
|
||||
|
||||
|
||||
def default_threads() -> int:
|
||||
"""既定のCPUスレッド数: 画像生成と同居しても邪魔になりにくいよう、コアの半分(最低1)"""
|
||||
return max(1, (os.cpu_count() or 2) // 2)
|
||||
|
||||
|
||||
def _target_size(h: int, w: int, max_patches: int) -> tuple[int, int]:
|
||||
"""Gemma4ImageProcessor の get_aspect_ratio_preserving_size"""
|
||||
factor = math.sqrt(max_patches * _PATCH ** 2 / (h * w))
|
||||
side = _POOL * _PATCH
|
||||
th = int(math.floor(factor * h / side)) * side
|
||||
tw = int(math.floor(factor * w / side)) * side
|
||||
max_side = (max_patches // _POOL ** 2) * side
|
||||
if th == 0 and tw == 0:
|
||||
raise ValueError("image too small")
|
||||
if th == 0:
|
||||
th, tw = side, min((w // h) * side, max_side)
|
||||
elif tw == 0:
|
||||
tw, th = side, min((h // w) * side, max_side)
|
||||
return th, tw
|
||||
|
||||
|
||||
class _Runtime:
|
||||
def __init__(self, model_dir: Path, dtype: str, device: str = "auto", threads: int = 0):
|
||||
import onnxruntime as ort
|
||||
from tokenizers import Tokenizer
|
||||
|
||||
sfx = DTYPES[dtype]
|
||||
so = ort.SessionOptions()
|
||||
so.log_severity_level = 3 # CUDA使用時のMemcpy警告などを抑止
|
||||
if threads > 0:
|
||||
so.intra_op_num_threads = threads
|
||||
so.inter_op_num_threads = 1
|
||||
avail = ort.get_available_providers()
|
||||
if device in _PROVIDER_BY_DEVICE:
|
||||
providers = [_PROVIDER_BY_DEVICE[device]] if _PROVIDER_BY_DEVICE[device] in avail else []
|
||||
elif device == "cpu":
|
||||
providers = []
|
||||
else: # auto: 使えるGPUを優先
|
||||
providers = [p for p in ("CUDAExecutionProvider", "DmlExecutionProvider") if p in avail]
|
||||
if "DmlExecutionProvider" in providers:
|
||||
# DirectMLは並列実行/メモリパターン最適化に非対応
|
||||
so.enable_mem_pattern = False
|
||||
so.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL
|
||||
providers.append("CPUExecutionProvider")
|
||||
self.providers = providers
|
||||
|
||||
def make(name):
|
||||
path = str(model_dir / "onnx" / f"{name}{sfx}.onnx")
|
||||
try:
|
||||
return ort.InferenceSession(path, so, providers=providers)
|
||||
except Exception as e:
|
||||
if providers == ["CPUExecutionProvider"]:
|
||||
raise
|
||||
logger.warning("EmbeddingGemma2: %s failed on %s (%s); falling back to CPU", name, providers[0], e)
|
||||
self.providers = ["CPUExecutionProvider"]
|
||||
return ort.InferenceSession(path, so, providers=["CPUExecutionProvider"])
|
||||
|
||||
self.text = make("model")
|
||||
self.vision = make("vision_encoder")
|
||||
self.tok = Tokenizer.from_file(str(model_dir / "tokenizer.json"))
|
||||
self.tok.no_truncation()
|
||||
self.tok.no_padding()
|
||||
|
||||
def _run_text(self, text: str, image_features: np.ndarray | None) -> np.ndarray:
|
||||
ids = np.asarray([self.tok.encode(text).ids[:_MAX_TOKENS]], dtype=np.int64)
|
||||
feeds = {
|
||||
"input_ids": ids,
|
||||
"attention_mask": np.ones_like(ids),
|
||||
"image_features": image_features if image_features is not None else np.zeros((0, 512), np.float32),
|
||||
"video_features": np.zeros((0, 512), np.float32),
|
||||
"audio_features": np.zeros((0, 512), np.float32),
|
||||
}
|
||||
return self.text.run(["sentence_embedding"], feeds)[0][0]
|
||||
|
||||
def embed_text(self, text: str, prefix: str) -> np.ndarray:
|
||||
return self._run_text(f"{prefix}{text.strip()}", None)
|
||||
|
||||
def embed_image(self, image, soft_tokens: int = DEFAULT_SOFT) -> np.ndarray:
|
||||
from PIL import Image
|
||||
im = image.convert("RGB")
|
||||
if im.width < 1 or im.height < 1:
|
||||
raise ValueError("empty image")
|
||||
if max(im.size) > _MAX_IMAGE_SIDE: # 巨大画像でメモリを使い切らないよう先に縮小
|
||||
im.thumbnail((_MAX_IMAGE_SIDE, _MAX_IMAGE_SIDE), Image.LANCZOS)
|
||||
max_patches = soft_tokens * _POOL ** 2
|
||||
th, tw = _target_size(im.height, im.width, max_patches)
|
||||
if (th, tw) != (im.height, im.width):
|
||||
im = im.resize((tw, th), Image.BICUBIC)
|
||||
arr = np.asarray(im, dtype=np.float32) * (1.0 / 255.0) # H,W,3
|
||||
ph, pw = th // _PATCH, tw // _PATCH
|
||||
n = ph * pw
|
||||
# (pH, dy, pW, dx, C) -> (pH, pW, dy, dx, C) -> (n, 768)
|
||||
patches = arr.reshape(ph, _PATCH, pw, _PATCH, 3).transpose(0, 2, 1, 3, 4).reshape(n, _PATCH * _PATCH * 3)
|
||||
# パディング無し(n個ちょうど)で渡す: 計算量が減り、パディングは出力に影響しない
|
||||
pixel = patches[None].astype(np.float32)
|
||||
pos = np.zeros((1, n, 2), dtype=np.int64)
|
||||
rows, cols = np.divmod(np.arange(n), pw)
|
||||
pos[0, :, 0], pos[0, :, 1] = cols, rows
|
||||
feats = self.vision.run(["image_features"], {"pixel_values": pixel, "pixel_position_ids": pos})[0]
|
||||
text = f"{_BOI}{_IMAGE_TOKEN * (n // _POOL ** 2)}{_EOI}"
|
||||
return self._run_text(text, feats.astype(np.float32))
|
||||
|
||||
|
||||
def model_dir(data_dir: Path) -> Path:
|
||||
return data_dir / "models" / LOCAL_MODEL
|
||||
|
||||
|
||||
def is_downloaded(data_dir: Path, dtype: str = DEFAULT_DTYPE) -> bool:
|
||||
d, sfx = model_dir(data_dir), DTYPES.get(dtype, "_q4")
|
||||
need = [d / "tokenizer.json"] + [d / "onnx" / f"{n}{sfx}.onnx{e}" for n in ("model", "vision_encoder") for e in ("", "_data")]
|
||||
return all(p.exists() for p in need)
|
||||
|
||||
|
||||
def ensure_runtime(data_dir: Path, dtype: str = DEFAULT_DTYPE, progress=None,
|
||||
device: str | None = None, threads: int | None = None) -> _Runtime:
|
||||
"""モデルをダウンロード(未取得時)してセッションを返す。dtype/device/threads変更時は作り直す。"""
|
||||
global _runtime
|
||||
if dtype not in DTYPES:
|
||||
dtype = DEFAULT_DTYPE
|
||||
device = _opts["device"] if device is None else device
|
||||
threads = _opts["threads"] if threads is None else threads
|
||||
if device not in DEVICES:
|
||||
device = "auto"
|
||||
key = (dtype, device, int(threads))
|
||||
with _lock:
|
||||
if _runtime and _runtime[0] == key:
|
||||
return _runtime[1]
|
||||
d, sfx = model_dir(data_dir), DTYPES[dtype]
|
||||
if not is_downloaded(data_dir, dtype):
|
||||
if progress:
|
||||
progress("downloading")
|
||||
from huggingface_hub import hf_hub_download
|
||||
files = ["tokenizer.json"] + [f"onnx/{n}{sfx}.onnx{e}" for n in ("model", "vision_encoder") for e in ("", "_data")]
|
||||
for f in files:
|
||||
hf_hub_download(REPO_ID, f, local_dir=str(d))
|
||||
if progress:
|
||||
progress("loading")
|
||||
_runtime = (key, _Runtime(d, dtype, device, int(threads)))
|
||||
return _runtime[1]
|
||||
|
||||
|
||||
def _finish(vec: np.ndarray, dim: int) -> np.ndarray:
|
||||
v = np.asarray(vec, dtype=np.float32)
|
||||
if dim < v.shape[0]:
|
||||
v = v[:dim]
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def embed_text(data_dir: Path, text: str, dim: int, dtype: str = DEFAULT_DTYPE) -> np.ndarray:
|
||||
rt = ensure_runtime(data_dir, dtype)
|
||||
return _finish(rt.embed_text(text[:8000], "task: search result | query: "), dim)
|
||||
|
||||
|
||||
def embed_image(data_dir: Path, image, dim: int, soft: int = DEFAULT_SOFT, dtype: str = DEFAULT_DTYPE) -> np.ndarray:
|
||||
rt = ensure_runtime(data_dir, dtype)
|
||||
return _finish(rt.embed_image(image, soft if soft in SOFT_TOKENS else DEFAULT_SOFT), dim)
|
||||
@@ -0,0 +1,513 @@
|
||||
"""
|
||||
Semantic Service - Gallery用セマンティック検索(Gemini Embedding 2)
|
||||
|
||||
画像(動画・PSDはサムネイル)をGemini Embedding APIでベクトル化してローカルに保存し、
|
||||
テキストクエリ/類似画像でコサイン類似度検索する。インデックス構築後の検索は
|
||||
(テキストクエリの埋め込み1回を除き)ローカル処理でAPI費用はかからない。
|
||||
|
||||
APIキーは環境変数 GEMINI_API_KEY (または GOOGLE_API_KEY) から読む(.env経由)。
|
||||
settings.json やフロントエンドには出さない。接続先は固定ホストのみ(SSRF対策)。
|
||||
"""
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import semantic_local as local
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
API_BASE = "https://generativelanguage.googleapis.com/v1beta"
|
||||
MODEL = "gemini-embedding-2" # Gemini API (クラウド)
|
||||
LOCAL_MODEL = local.LOCAL_MODEL # EmbeddingGemma 2 (ローカルONNX)
|
||||
DEFAULT_DIM = 1536
|
||||
VALID_DIMS = (768, 1536, 3072)
|
||||
# バックエンド(モデル名) -> 選択可能な出力次元
|
||||
DIMS_BY_MODEL = {MODEL: VALID_DIMS, LOCAL_MODEL: local.LOCAL_DIMS}
|
||||
DEFAULT_DIM_BY_MODEL = {MODEL: DEFAULT_DIM, LOCAL_MODEL: local.DEFAULT_LOCAL_DIM}
|
||||
IMAGE_MAX_SIZE = 768 # 送信前の長辺リサイズ(px)
|
||||
_MAX_QUERY_CHARS = 2000 # 検索文の最大長
|
||||
_WORKERS = 4 # 並列API呼び出し数
|
||||
_CHECKPOINT_EVERY = 100 # この件数ごとにディスクへ保存
|
||||
_RETRIES = 4
|
||||
|
||||
# PILで直接開ける(静止画/GIFは先頭フレーム)
|
||||
_PIL_EXTS = {".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp"}
|
||||
# GalleryService.serve_thumbnail 経由でJPEG化する(動画先頭フレーム/PSD合成)
|
||||
_THUMB_EXTS = {".mp4", ".webm", ".psd"}
|
||||
INDEXABLE_EXTS = _PIL_EXTS | _THUMB_EXTS
|
||||
|
||||
|
||||
class SemanticError(Exception):
|
||||
"""ユーザーに見せられるエラー(code はフロントのi18n用)"""
|
||||
|
||||
def __init__(self, code: str, message: str = ""):
|
||||
super().__init__(message or code)
|
||||
self.code = code
|
||||
|
||||
|
||||
def get_api_key() -> str:
|
||||
return (os.environ.get("GEMINI_API_KEY", "") or os.environ.get("GOOGLE_API_KEY", "")).strip()
|
||||
|
||||
|
||||
def _normalize(v: np.ndarray) -> np.ndarray:
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def _call_embed(content: dict, task_type: str, dim: int) -> np.ndarray:
|
||||
key = get_api_key()
|
||||
if not key:
|
||||
raise SemanticError("no_api_key", "GEMINI_API_KEY is not set")
|
||||
body = {"model": f"models/{MODEL}", "content": content, "taskType": task_type}
|
||||
if dim != 768:
|
||||
body["outputDimensionality"] = dim
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
url = f"{API_BASE}/models/{MODEL}:embedContent"
|
||||
last_err = ""
|
||||
for attempt in range(_RETRIES + 1):
|
||||
req = urllib.request.Request(
|
||||
url, data=data, method="POST",
|
||||
headers={"Content-Type": "application/json", "x-goog-api-key": key},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=60) as res:
|
||||
payload = json.loads(res.read().decode("utf-8"))
|
||||
values = (payload.get("embedding") or {}).get("values")
|
||||
if not values:
|
||||
raise SemanticError("bad_response", "no embedding.values in response")
|
||||
return _normalize(np.asarray(values, dtype=np.float32))
|
||||
except urllib.error.HTTPError as e:
|
||||
try:
|
||||
msg = json.loads(e.read().decode("utf-8")).get("error", {}).get("message", "")
|
||||
except Exception:
|
||||
msg = ""
|
||||
last_err = f"HTTP {e.code}: {msg or e.reason}"
|
||||
if e.code in (400, 403) and ("API key" in msg or "API_KEY" in msg or e.code == 403):
|
||||
raise SemanticError("api_key_invalid", last_err)
|
||||
if e.code in (429, 500, 502, 503, 504) and attempt < _RETRIES:
|
||||
time.sleep(1.5 * (2 ** attempt))
|
||||
continue
|
||||
raise SemanticError("api_error", last_err)
|
||||
except (urllib.error.URLError, TimeoutError, ConnectionError) as e:
|
||||
last_err = str(getattr(e, "reason", e))
|
||||
if attempt < _RETRIES:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
raise SemanticError("network_error", last_err)
|
||||
raise SemanticError("api_error", last_err)
|
||||
|
||||
|
||||
def embed_text(text: str, dim: int) -> np.ndarray:
|
||||
return _call_embed({"parts": [{"text": text[:8000]}]}, "RETRIEVAL_QUERY", dim)
|
||||
|
||||
|
||||
def embed_image_bytes(data: bytes, mime: str, dim: int) -> np.ndarray:
|
||||
import base64
|
||||
b64 = base64.b64encode(data).decode("ascii")
|
||||
return _call_embed({"parts": [{"inlineData": {"mimeType": mime, "data": b64}}]}, "RETRIEVAL_DOCUMENT", dim)
|
||||
|
||||
|
||||
class SemanticService:
|
||||
def __init__(self, data_dir: Path, gallery_service):
|
||||
self.data_dir = data_dir
|
||||
self.gallery = gallery_service
|
||||
self._file = data_dir / "semantic_index.npz"
|
||||
self._lock = threading.RLock()
|
||||
self._vecs: dict[str, np.ndarray] = {} # path -> 正規化済みベクトル
|
||||
self._stamp: dict[str, tuple[int, float]] = {} # path -> (size, mtime)
|
||||
self._dim = 0 # インデックスの次元(0=空)
|
||||
self._model = MODEL # インデックスを作ったモデル
|
||||
self._soft = local.DEFAULT_SOFT # ローカルモデルの画像ソフトトークン数(インデックスと一致させる)
|
||||
self._settings_file = data_dir / "semantic_settings.json"
|
||||
self._load_settings()
|
||||
self._mat_cache = None # (paths, matrix)
|
||||
self._loaded = False
|
||||
self._dirty = 0
|
||||
# ジョブ
|
||||
self._job = self._idle_job()
|
||||
self._cancel = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
# ── ローカル推論の実行設定(デバイス/スレッド数) ─────────────
|
||||
|
||||
def _load_settings(self) -> None:
|
||||
try:
|
||||
if self._settings_file.exists():
|
||||
d = json.loads(self._settings_file.read_text(encoding="utf-8"))
|
||||
local.configure(str(d.get("device", "auto")), int(d.get("threads", 0)))
|
||||
return
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to load settings: %s", e)
|
||||
local.configure("auto", local.default_threads())
|
||||
|
||||
def set_local_options(self, device: str, threads: int) -> None:
|
||||
"""実行デバイス/CPUスレッド数を反映して保存する(検索時のクエリ埋め込みにも使われる)"""
|
||||
local.configure(device, threads)
|
||||
try:
|
||||
self._settings_file.write_text(json.dumps(local.current_config()), encoding="utf-8")
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to save settings: %s", e)
|
||||
|
||||
# ── 永続化 ─────────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def _idle_job() -> dict:
|
||||
return {"running": False, "total": 0, "done": 0, "failed": 0, "current": "",
|
||||
"error": "", "error_code": "", "cancelled": False, "finished": False}
|
||||
|
||||
def _ensure_loaded(self) -> None:
|
||||
with self._lock:
|
||||
if self._loaded:
|
||||
return
|
||||
self._loaded = True
|
||||
if not self._file.exists():
|
||||
return
|
||||
try:
|
||||
with np.load(self._file, allow_pickle=False) as z:
|
||||
model = str(z["model"])
|
||||
if model not in DIMS_BY_MODEL:
|
||||
return
|
||||
self._model = model
|
||||
self._soft = int(z["soft"]) if "soft" in z.files else local.DEFAULT_SOFT
|
||||
paths = [str(p) for p in z["paths"]]
|
||||
sizes, mtimes, vecs = z["sizes"], z["mtimes"], z["vecs"]
|
||||
self._dim = int(z["dim"])
|
||||
for i, p in enumerate(paths):
|
||||
self._vecs[p] = vecs[i]
|
||||
self._stamp[p] = (int(sizes[i]), float(mtimes[i]))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to load index: %s", e)
|
||||
self._vecs.clear()
|
||||
self._stamp.clear()
|
||||
self._dim = 0
|
||||
|
||||
def _save(self) -> None:
|
||||
with self._lock:
|
||||
if not self._vecs:
|
||||
if self._file.exists():
|
||||
self._file.unlink()
|
||||
self._dirty = 0
|
||||
return
|
||||
paths = list(self._vecs.keys())
|
||||
tmp = self._file.with_suffix(".tmp.npz")
|
||||
np.savez(
|
||||
tmp,
|
||||
model=np.array(self._model), dim=np.array(self._dim), soft=np.array(self._soft),
|
||||
paths=np.array(paths),
|
||||
sizes=np.array([self._stamp[p][0] for p in paths], dtype=np.int64),
|
||||
mtimes=np.array([self._stamp[p][1] for p in paths], dtype=np.float64),
|
||||
vecs=np.stack([self._vecs[p] for p in paths]).astype(np.float32),
|
||||
)
|
||||
os.replace(tmp, self._file)
|
||||
self._dirty = 0
|
||||
|
||||
def _put(self, path: str, vec: np.ndarray, size: int, mtime: float) -> None:
|
||||
with self._lock:
|
||||
self._vecs[path] = vec
|
||||
self._stamp[path] = (size, mtime)
|
||||
self._mat_cache = None
|
||||
self._dirty += 1
|
||||
|
||||
# ── 状態 ───────────────────────────────────────────────
|
||||
|
||||
def _scope_entries(self, folder: str, recursive: bool) -> list[tuple[str, str, int, float]]:
|
||||
p = Path(folder).resolve()
|
||||
if not p.is_dir() or not self.gallery._check_path_allowed(p):
|
||||
raise SemanticError("folder_not_allowed", str(folder))
|
||||
raw = self.gallery._scan_folder_recursive(p) if recursive else self.gallery._scan_folder(p)
|
||||
return [e for e in raw if Path(e[0]).suffix.lower() in INDEXABLE_EXTS]
|
||||
|
||||
def status(self, folder: str = "", recursive: bool = True) -> dict:
|
||||
self._ensure_loaded()
|
||||
out = {
|
||||
"has_key": bool(get_api_key()),
|
||||
"model": MODEL,
|
||||
"backends": {
|
||||
MODEL: {"dims": list(VALID_DIMS), "default_dim": DEFAULT_DIM, "ready": bool(get_api_key())},
|
||||
LOCAL_MODEL: {"dims": list(local.LOCAL_DIMS), "default_dim": local.DEFAULT_LOCAL_DIM,
|
||||
"ready": True, "downloaded": local.is_downloaded(self.data_dir)},
|
||||
},
|
||||
"index_model": self._model,
|
||||
"index_dim": self._dim,
|
||||
"index_soft": self._soft,
|
||||
"local": {
|
||||
"devices": local.available_devices(),
|
||||
"options": local.current_config(),
|
||||
"default_threads": local.default_threads(),
|
||||
"soft_tokens": list(local.SOFT_TOKENS),
|
||||
"default_soft": local.DEFAULT_SOFT,
|
||||
},
|
||||
"indexed_total": len(self._vecs),
|
||||
"job": dict(self._job),
|
||||
}
|
||||
if folder:
|
||||
try:
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
with self._lock:
|
||||
done = sum(1 for _, ap, sz, mt in entries if self._is_fresh(ap, sz, mt))
|
||||
out["scope_total"] = len(entries)
|
||||
out["scope_indexed"] = done
|
||||
except SemanticError:
|
||||
pass
|
||||
return out
|
||||
|
||||
def folder_status(self, root: str) -> dict:
|
||||
"""フォルダごとの索引状況 {絶対パス: {"indexable": n, "indexed": m}} (直下の画像のみ。Galleryツリーの枚数表示と同じ単位)"""
|
||||
self._ensure_loaded()
|
||||
entries = self._scope_entries(root, True)
|
||||
out: dict[str, dict] = {}
|
||||
with self._lock:
|
||||
for _, ap, sz, mt in entries:
|
||||
d = out.setdefault(ap.rsplit("/", 1)[0], {"indexable": 0, "indexed": 0})
|
||||
d["indexable"] += 1
|
||||
if self._is_fresh(ap, sz, mt):
|
||||
d["indexed"] += 1
|
||||
return out
|
||||
|
||||
def _is_fresh(self, path: str, size: int, mtime: float) -> bool:
|
||||
st = self._stamp.get(path)
|
||||
return st is not None and st[0] == size and abs(st[1] - mtime) < 1e-3
|
||||
|
||||
# ── インデックス作成ジョブ ────────────────────────────
|
||||
|
||||
def start_index(self, folder: str, recursive: bool, mode: str, model: str, dim: int,
|
||||
soft: int = local.DEFAULT_SOFT) -> None:
|
||||
self._ensure_loaded()
|
||||
if model not in DIMS_BY_MODEL:
|
||||
raise SemanticError("bad_model", model)
|
||||
if model == MODEL and not get_api_key():
|
||||
raise SemanticError("no_api_key")
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
raise SemanticError("bad_dim", str(dim))
|
||||
if model != LOCAL_MODEL or soft not in local.SOFT_TOKENS:
|
||||
soft = local.DEFAULT_SOFT # ソフトトークン数はローカルモデルのみの設定
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
mismatch = bool(self._vecs) and (self._dim != dim or self._model != model or self._soft != soft)
|
||||
if mismatch and mode != "all":
|
||||
raise SemanticError("dim_mismatch", f"{self._model}/{self._dim}/{self._soft}")
|
||||
if mode == "all":
|
||||
# 範囲内の既存ベクトルを破棄して作り直す(モデル/次元違いなら全消去)
|
||||
if mismatch:
|
||||
self._vecs.clear()
|
||||
self._stamp.clear()
|
||||
else:
|
||||
for _, ap, _, _ in entries:
|
||||
self._vecs.pop(ap, None)
|
||||
self._stamp.pop(ap, None)
|
||||
self._mat_cache = None
|
||||
self._dim = dim
|
||||
self._model = model
|
||||
self._soft = soft
|
||||
todo = [e for e in entries if not self._is_fresh(e[1], e[2], e[3])]
|
||||
# スコープ内で消えたファイルの残骸を掃除(スコープ配下のキーのみ)
|
||||
self._prune_missing(folder, recursive, {e[1] for e in entries})
|
||||
self._cancel.clear()
|
||||
self._job = self._idle_job()
|
||||
self._job.update(running=True, total=len(todo))
|
||||
self._thread = threading.Thread(target=self._run_index, args=(todo, model, dim, soft), daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _prune_missing(self, folder: str, recursive: bool, existing: set[str]) -> None:
|
||||
base = Path(folder).resolve().as_posix().rstrip("/") + "/"
|
||||
for p in list(self._vecs.keys()):
|
||||
if not p.startswith(base):
|
||||
continue
|
||||
if not recursive and "/" in p[len(base):]:
|
||||
continue
|
||||
if p not in existing:
|
||||
self._vecs.pop(p, None)
|
||||
self._stamp.pop(p, None)
|
||||
self._mat_cache = None
|
||||
self._dirty += 1
|
||||
|
||||
def cancel(self) -> None:
|
||||
self._cancel.set()
|
||||
|
||||
def _run_index(self, todo: list, model: str, dim: int, soft: int = local.DEFAULT_SOFT) -> None:
|
||||
def work(entry):
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
name, ap, size, mtime = entry
|
||||
self._job["current"] = name
|
||||
try:
|
||||
vec = self._embed_image(model, dim, Path(ap), soft)
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
except SemanticError as e:
|
||||
if e.code in ("no_api_key", "api_key_invalid"):
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
self._cancel.set()
|
||||
else:
|
||||
self._job["failed"] += 1
|
||||
if not self._job["error"]:
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: embed failed for %s: %s", ap, e)
|
||||
self._job["failed"] += 1
|
||||
if self._dirty >= _CHECKPOINT_EVERY:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: checkpoint failed: %s", e)
|
||||
|
||||
try:
|
||||
if model == LOCAL_MODEL:
|
||||
# 初回はモデルのダウンロード/ロードを先に行う(失敗したらジョブ全体を中止)
|
||||
try:
|
||||
local.ensure_runtime(self.data_dir, progress=lambda ph: self._job.update(current=f"({ph} model...)"))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: local model load failed: %s", e)
|
||||
self._job["error_code"], self._job["error"] = "local_error", str(e)
|
||||
self._cancel.set()
|
||||
workers = 1 if model == LOCAL_MODEL else _WORKERS # ローカルはORT側がマルチスレッド
|
||||
with ThreadPoolExecutor(max_workers=workers) as ex:
|
||||
list(ex.map(work, todo))
|
||||
finally:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: save failed: %s", e)
|
||||
if model == LOCAL_MODEL:
|
||||
local.release() # 画像生成とVRAM/RAMを奪い合わないよう、作成後はモデルを解放(検索時に再ロード)
|
||||
self._job["running"] = False
|
||||
self._job["finished"] = True
|
||||
self._job["cancelled"] = self._cancel.is_set() and not self._job["error_code"]
|
||||
self._job["current"] = ""
|
||||
|
||||
def _load_pil(self, path: Path):
|
||||
"""埋め込み用に縮小したPIL画像を返す(動画/PSDはサムネイルから)"""
|
||||
from PIL import Image
|
||||
if path.suffix.lower() in _THUMB_EXTS:
|
||||
thumb = self.gallery.serve_thumbnail(str(path), IMAGE_MAX_SIZE)
|
||||
if not thumb:
|
||||
raise SemanticError("thumb_failed", str(path))
|
||||
with Image.open(thumb) as im:
|
||||
return im.convert("RGB")
|
||||
with Image.open(path) as im:
|
||||
im.seek(0)
|
||||
im = im.convert("RGBA") if im.mode in ("RGBA", "LA", "P") else im.convert("RGB")
|
||||
im.thumbnail((IMAGE_MAX_SIZE, IMAGE_MAX_SIZE), Image.LANCZOS)
|
||||
return im
|
||||
|
||||
def _embed_image(self, model: str, dim: int, path: Path, soft: int | None = None) -> np.ndarray:
|
||||
from PIL import Image
|
||||
im = self._load_pil(path)
|
||||
if model == LOCAL_MODEL:
|
||||
if im.mode == "RGBA": # 透過部は白背景に合成
|
||||
bg = Image.new("RGB", im.size, (255, 255, 255))
|
||||
bg.paste(im, mask=im.getchannel("A"))
|
||||
im = bg
|
||||
try:
|
||||
return local.embed_image(self.data_dir, im, dim, self._soft if soft is None else soft)
|
||||
except Exception as e:
|
||||
raise SemanticError("local_error", str(e))
|
||||
buf = io.BytesIO()
|
||||
if im.mode == "RGBA":
|
||||
im.save(buf, "PNG")
|
||||
return embed_image_bytes(buf.getvalue(), "image/png", dim)
|
||||
im.save(buf, "JPEG", quality=88)
|
||||
return embed_image_bytes(buf.getvalue(), "image/jpeg", dim)
|
||||
|
||||
def _embed_text(self, model: str, dim: int, text: str) -> np.ndarray:
|
||||
if model == LOCAL_MODEL:
|
||||
try:
|
||||
return local.embed_text(self.data_dir, text, dim)
|
||||
except Exception as e:
|
||||
raise SemanticError("local_error", str(e))
|
||||
return embed_text(text, dim)
|
||||
|
||||
# ── 検索 ───────────────────────────────────────────────
|
||||
|
||||
def _matrix(self):
|
||||
with self._lock:
|
||||
if self._mat_cache is None:
|
||||
paths = list(self._vecs.keys())
|
||||
mat = np.stack([self._vecs[p] for p in paths]) if paths else np.zeros((0, max(self._dim, 1)), np.float32)
|
||||
self._mat_cache = (paths, mat)
|
||||
return self._mat_cache
|
||||
|
||||
def _rank(self, qvec: np.ndarray, folder: str, recursive: bool, limit: int, exclude: str = "",
|
||||
min_score: float = 0.0) -> list[dict]:
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
info = {ap: (name, sz, mt) for name, ap, sz, mt in entries}
|
||||
paths, mat = self._matrix()
|
||||
if not paths:
|
||||
raise SemanticError("index_empty")
|
||||
if mat.shape[1] != qvec.shape[0]:
|
||||
raise SemanticError("dim_mismatch", str(self._dim))
|
||||
scores = mat @ qvec
|
||||
order = np.argsort(-scores)
|
||||
out = []
|
||||
for i in order:
|
||||
if float(scores[i]) < min_score:
|
||||
break # スコア降順なので、以降はすべてしきい値未満
|
||||
p = paths[i]
|
||||
if p == exclude or p not in info:
|
||||
continue
|
||||
name, sz, mt = info[p]
|
||||
item = self.gallery.make_list_item(name, p, sz, mt)
|
||||
item["score"] = round(float(scores[i]), 4)
|
||||
out.append(item)
|
||||
if len(out) >= limit:
|
||||
break
|
||||
return out
|
||||
|
||||
def search_text(self, query: str, folder: str, recursive: bool, limit: int, min_score: float = 0.0) -> list[dict]:
|
||||
self._ensure_loaded()
|
||||
query = (query or "").strip()[:_MAX_QUERY_CHARS]
|
||||
if not query:
|
||||
return []
|
||||
if not self._vecs:
|
||||
raise SemanticError("index_empty")
|
||||
qvec = self._embed_text(self._model, self._dim, query)
|
||||
return self._rank(qvec, folder, recursive, limit, min_score=min_score)
|
||||
|
||||
def search_similar(self, path: str, folder: str, recursive: bool, limit: int, min_score: float = 0.0) -> list[dict]:
|
||||
self._ensure_loaded()
|
||||
p = Path(path).resolve()
|
||||
if not p.is_file() or not self.gallery._check_path_allowed(p):
|
||||
raise SemanticError("folder_not_allowed", path)
|
||||
ap = p.as_posix()
|
||||
vec = self._vecs.get(ap)
|
||||
if vec is None or not self._is_fresh(ap, p.stat().st_size, p.stat().st_mtime):
|
||||
# 未インデックスの基準画像は、その場で1枚だけ埋め込む
|
||||
if not self._vecs:
|
||||
raise SemanticError("index_empty")
|
||||
if p.suffix.lower() not in INDEXABLE_EXTS:
|
||||
raise SemanticError("unsupported_type", p.suffix)
|
||||
vec = self._embed_image(self._model, self._dim, p)
|
||||
st = p.stat()
|
||||
self._put(ap, vec, st.st_size, st.st_mtime)
|
||||
self._save()
|
||||
return self._rank(vec, folder, recursive, limit, exclude=ap, min_score=min_score)
|
||||
|
||||
def clear(self) -> None:
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
self._vecs.clear()
|
||||
self._stamp.clear()
|
||||
self._dim = 0
|
||||
self._mat_cache = None
|
||||
self._save()
|
||||
|
||||
def test_connection(self, model: str, dim: int) -> dict:
|
||||
if model not in DIMS_BY_MODEL:
|
||||
raise SemanticError("bad_model", model)
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
dim = DEFAULT_DIM_BY_MODEL[model]
|
||||
n = int(self._embed_text(model, dim, "test").shape[0])
|
||||
return {"dimensions": n, "provider": local.active_provider() if model == LOCAL_MODEL else "API"}
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "comfyui-workflow-studio"
|
||||
description = "A comprehensive workflow, asset management, and generation UI plugin for ComfyUI. Manage workflows, models, images, and prompts; batch generation and Image Feeder in the GenerateUI tab; library side panel with canvas drag & drop and AI tools."
|
||||
version = "0.8.0"
|
||||
version = "0.8.2"
|
||||
license = { file = "LICENSE" }
|
||||
requires-python = ">= 3.10"
|
||||
dependencies = ["jinja2"]
|
||||
|
||||
@@ -33,3 +33,13 @@ psd-tools>=1.10
|
||||
# If missing, mp4 thumbnail generation simply fails and falls back to a placeholder
|
||||
# (Gallery itself keeps working). Not pinned here on purpose — see torch note above.
|
||||
# av>=17.0.0
|
||||
|
||||
# -- Gallery Semantic Search: local EmbeddingGemma 2 (optional) --
|
||||
# Uses onnxruntime (above) + tokenizers + huggingface_hub (both normally pre-installed with ComfyUI's
|
||||
# transformers). The model (~300 MB, q4) is downloaded to the data folder on first use.
|
||||
# GPU (NVIDIA/CUDA): replace onnxruntime with onnxruntime-gpu
|
||||
# pip uninstall onnxruntime onnxruntime-gpu
|
||||
# pip install onnxruntime-gpu
|
||||
# Local EmbeddingGemma 2 then indexes ~45x faster than on the CPU. DirectML is not supported.
|
||||
# tokenizers
|
||||
# huggingface_hub
|
||||
|
||||
@@ -1048,3 +1048,109 @@
|
||||
.wfm-gallery-detail-preview--audio .wfm-gallery-audio-wave {
|
||||
max-height: 150px;
|
||||
}
|
||||
|
||||
/* ── セマンティック検索 ─────────────────────────────────────── */
|
||||
|
||||
.wfm-gallery-sem-toggle.active {
|
||||
background: var(--wfm-primary);
|
||||
border-color: var(--wfm-primary);
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
/* 2行目(Similar / 更新)のボタン: Taggerの紫と区別できるティール */
|
||||
.wfm-sem-sim {
|
||||
background: #0d9488;
|
||||
border-color: #0d9488;
|
||||
color: #fff;
|
||||
}
|
||||
.wfm-sem-sim:hover {
|
||||
background: #0f766e;
|
||||
border-color: #0f766e;
|
||||
}
|
||||
/* 類似画像表示中(トグルON)は縁取りで状態を示す */
|
||||
#wfm-gallery-similar-btn.active {
|
||||
box-shadow: 0 0 0 2px #fff inset, 0 0 0 2px #0d9488;
|
||||
}
|
||||
|
||||
.wfm-gallery-sem-row {
|
||||
gap: 6px;
|
||||
align-items: center;
|
||||
justify-content: flex-start;
|
||||
}
|
||||
.wfm-gallery-sem-count {
|
||||
margin-left: 6px;
|
||||
font-size: 12px;
|
||||
opacity: 0.85;
|
||||
}
|
||||
|
||||
.wfm-gallery-thumb-score {
|
||||
position: absolute;
|
||||
left: 4px;
|
||||
bottom: 4px;
|
||||
padding: 1px 5px;
|
||||
border-radius: 8px;
|
||||
background: rgba(13, 148, 136, 0.85);
|
||||
color: #fff;
|
||||
font-size: 10px;
|
||||
line-height: 14px;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.wfm-sem-modal { display: flex; flex-direction: column; gap: 10px; min-width: 420px; }
|
||||
.wfm-sem-row { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
|
||||
.wfm-sem-folder { font-family: monospace; font-size: 12px; word-break: break-all; opacity: 0.85; }
|
||||
.wfm-sem-hint { font-size: 11px; opacity: 0.7; margin: 0; }
|
||||
.wfm-sem-ok { color: #22c55e; }
|
||||
.wfm-sem-ng { color: #ef4444; }
|
||||
.wfm-sem-actions { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||
.wfm-sem-bar { height: 8px; border-radius: 4px; background: var(--wfm-border, #444); overflow: hidden; }
|
||||
.wfm-sem-bar-fill { height: 100%; width: 0; background: #0d9488; transition: width 0.3s; }
|
||||
|
||||
/* フォルダツリー: 直下の画像が全件セマンティック索引済みのフォルダに付くドット */
|
||||
.wfm-gallery-tree-sem-dot {
|
||||
display: inline-block;
|
||||
width: 7px;
|
||||
height: 7px;
|
||||
margin-left: 4px;
|
||||
border-radius: 50%;
|
||||
background: #22c55e;
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
/* モーダルのヘルプボタンから飛んだ先の説明を一時的に強調 */
|
||||
.wfm-help-flash {
|
||||
animation: wfm-help-flash 2.5s ease-out;
|
||||
}
|
||||
@keyframes wfm-help-flash {
|
||||
0%, 60% { background: rgba(13, 148, 136, 0.35); }
|
||||
100% { background: transparent; }
|
||||
}
|
||||
|
||||
.wfm-sem-local-opts { display: flex; flex-direction: column; gap: 10px; padding: 8px 10px; border: 1px solid var(--wfm-border, #444); border-radius: 6px; }
|
||||
|
||||
/* セマンティック検索モーダル: タブ */
|
||||
.wfm-sem-tabs { display: flex; gap: 4px; border-bottom: 1px solid var(--wfm-border, #444); }
|
||||
.wfm-sem-tab-btn {
|
||||
padding: 6px 14px;
|
||||
background: transparent;
|
||||
border: 1px solid transparent;
|
||||
border-bottom: none;
|
||||
border-radius: 6px 6px 0 0;
|
||||
color: var(--wfm-text-secondary, #aaa);
|
||||
cursor: pointer;
|
||||
font-size: 13px;
|
||||
}
|
||||
.wfm-sem-tab-btn:hover { color: var(--wfm-text, #fff); }
|
||||
.wfm-sem-tab-btn.active {
|
||||
color: var(--wfm-text, #fff);
|
||||
background: var(--wfm-bg-secondary, #1a1a2e);
|
||||
border-color: var(--wfm-border, #444);
|
||||
margin-bottom: -1px;
|
||||
border-bottom: 1px solid var(--wfm-bg-secondary, #1a1a2e);
|
||||
}
|
||||
.wfm-sem-pane { display: flex; flex-direction: column; gap: 10px; min-height: 230px; }
|
||||
.wfm-sem-footer { justify-content: flex-end; border-top: 1px solid var(--wfm-border, #444); padding-top: 10px; }
|
||||
.wfm-sem-num { width: 80px; }
|
||||
.wfm-sem-danger { margin-left: auto; color: #ef4444; }
|
||||
|
||||
.wfm-sem-row .wfm-select { width: auto; min-width: 90px; }
|
||||
|
||||
+2
-1
@@ -3491,7 +3491,8 @@ body {
|
||||
|
||||
.wfm-help-page {
|
||||
display: none;
|
||||
max-width: 760px;
|
||||
/* 上部タブバー(AI TOOLタブの右端)あたりまで広げ、折り返しを減らす。狭い画面では領域幅に収まる */
|
||||
max-width: 1076px;
|
||||
}
|
||||
|
||||
.wfm-help-page.active {
|
||||
|
||||
+1
-1
@@ -587,7 +587,7 @@ function applyI18nToHtml() {
|
||||
"wfm-help-gallery-26": "helpGallery26",
|
||||
"wfm-help-gallery-28": "helpGallery28", "wfm-help-gallery-29": "helpGallery29",
|
||||
"wfm-help-gallery-30": "helpGallery30", "wfm-help-gallery-31": "helpGallery31",
|
||||
"wfm-help-gallery-32": "helpGallery32",
|
||||
"wfm-help-gallery-32": "helpGallery32", "wfm-help-gallery-33": "helpGallery33", "wfm-help-gallery-34": "helpGallery34", "wfm-help-gallery-35": "helpGallery35", "wfm-help-gallery-36": "helpGallery36",
|
||||
"wfm-help-gallery-card-styleprompt-title": "helpGalleryCardStyleprompt",
|
||||
"wfm-help-gallery-card-stylecatalog-title": "helpGalleryCardStylecatalog",
|
||||
"wfm-help-tagger-title": "helpTaggerTitle",
|
||||
|
||||
@@ -649,7 +649,19 @@ export const comfyWorkflow = {
|
||||
_objectInfoCache = null;
|
||||
},
|
||||
|
||||
async convertUiToApi(workflow) {
|
||||
// opts.readOnly: 表示用の読み取り変換。COMBO値の差し替えを行わず、Workflowタブの互換性
|
||||
// チェックが参照する _last* 記録も変更しない(Metadataタブなど表示目的の呼び出し用)。
|
||||
async convertUiToApi(workflow, opts = {}) {
|
||||
if (!opts.readOnly) return comfyWorkflow._convertUiToApiImpl(workflow, opts);
|
||||
const saved = [_lastCheckpointSubstitutions, _lastComboSubstitutions, _lastBypassedNodes, _lastMutedNodes];
|
||||
try {
|
||||
return await comfyWorkflow._convertUiToApiImpl(workflow, opts);
|
||||
} finally {
|
||||
[_lastCheckpointSubstitutions, _lastComboSubstitutions, _lastBypassedNodes, _lastMutedNodes] = saved;
|
||||
}
|
||||
},
|
||||
|
||||
async _convertUiToApiImpl(workflow, opts = {}) {
|
||||
_lastCheckpointSubstitutions = [];
|
||||
_lastComboSubstitutions = [];
|
||||
_lastBypassedNodes = [];
|
||||
@@ -807,7 +819,7 @@ export const comfyWorkflow = {
|
||||
const spec = allInputDefs[name];
|
||||
if (spec) {
|
||||
const choices = Array.isArray(spec[0]) ? spec[0] : null;
|
||||
if (choices && choices.length > 0 && !choices.includes(val)) {
|
||||
if (!opts.readOnly && choices && choices.length > 0 && !choices.includes(val)) {
|
||||
if (name === "ckpt_name") {
|
||||
_lastCheckpointSubstitutions.push({
|
||||
nodeId: String(node.id),
|
||||
|
||||
@@ -0,0 +1,580 @@
|
||||
/**
|
||||
* Gallery Tab - セマンティック検索 (Gemini Embedding 2)
|
||||
* - 検索ボックスのSemantic Searchトグル(ONでEnter実行の意味検索)
|
||||
* - 詳細パネルの「類似画像」トグル/更新ボタン
|
||||
* - インデックス作成モーダル(Semantic⚙)
|
||||
* 一覧の差し替えは gallery-tab.js の loadImages() が getSemanticMode() / fetchSemanticImages() を呼んで行う。
|
||||
*/
|
||||
|
||||
import { openModal, closeModal, showToast } from "./app.js";
|
||||
import { t } from "./i18n.js";
|
||||
import { escapeHtml } from "./util.js";
|
||||
|
||||
const API = {
|
||||
status: (folder, recursive) => `/wfm/gallery/semantic/status?folder=${encodeURIComponent(folder)}&recursive=${recursive}`,
|
||||
index: "/wfm/gallery/semantic/index",
|
||||
cancel: "/wfm/gallery/semantic/cancel",
|
||||
search: "/wfm/gallery/semantic/search",
|
||||
similar: "/wfm/gallery/semantic/similar",
|
||||
clear: "/wfm/gallery/semantic/clear",
|
||||
folders: (root) => `/wfm/gallery/semantic/folders?root=${encodeURIComponent(root)}`,
|
||||
test: "/wfm/gallery/semantic/test",
|
||||
};
|
||||
|
||||
const LS_ON = "wfm_gallery_semantic_on";
|
||||
const LS_DIM = "wfm_gallery_semantic_dim";
|
||||
const LS_MODEL = "wfm_gallery_semantic_model";
|
||||
const LS_SOFT = "wfm_gallery_semantic_soft";
|
||||
const GEMINI = "gemini-embedding-2";
|
||||
const LOCAL = "embeddinggemma-2";
|
||||
const LS_SUB = "wfm_gallery_semantic_recursive";
|
||||
const LS_TAB = "wfm_gallery_semantic_tab";
|
||||
const LS_TH = "wfm_gallery_semantic_th";
|
||||
const LIMITS = [50, 100, 200, 500, 1000];
|
||||
|
||||
const sem = {
|
||||
on: false, // Semantic Searchトグル
|
||||
anchor: "", // 類似画像表示中の基準画像パス(空=類似表示OFF)
|
||||
anchorName: "",
|
||||
indexModel: GEMINI, // 現在のインデックスを作ったモデル(しきい値はモデルごとに保存する)
|
||||
};
|
||||
|
||||
let ctx = null; // { getSearch, getFolder, getSelected, reload, focusSearch }
|
||||
|
||||
function lsGet(key, fallback) {
|
||||
try { return localStorage.getItem(key) ?? fallback; } catch { return fallback; }
|
||||
}
|
||||
function lsSet(key, value) {
|
||||
try { localStorage.setItem(key, value); } catch { /* 保存できなくても動作に影響しない */ }
|
||||
}
|
||||
|
||||
async function postJson(url, body) {
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body || {}),
|
||||
});
|
||||
let data = {};
|
||||
try { data = await res.json(); } catch { /* 本文なし */ }
|
||||
if (!res.ok) {
|
||||
throw Object.assign(new Error(data.error || `HTTP ${res.status}`), { code: data.code || "" });
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
/** APIエラーコードをi18nメッセージにする */
|
||||
export function semanticErrorMessage(e) {
|
||||
const known = ["no_api_key", "api_key_invalid", "index_empty", "dim_mismatch", "already_running",
|
||||
"folder_not_allowed", "unsupported_type", "network_error", "api_error", "local_error"];
|
||||
if (e?.code && known.includes(e.code)) {
|
||||
return t(`sem_err_${e.code}`, e.message);
|
||||
}
|
||||
return t("errorWithMsg", e?.message || String(e));
|
||||
}
|
||||
|
||||
// ── gallery-tab.js から使う公開API ────────────────────────────
|
||||
|
||||
/** 現在の一覧モード: {kind:"similar", path} | {kind:"text", query} | null(通常) */
|
||||
export function getSemanticMode() {
|
||||
if (sem.anchor) return { kind: "similar", path: sem.anchor, name: sem.anchorName };
|
||||
const q = (ctx?.getSearch() || "").trim();
|
||||
if (sem.on && q) return { kind: "text", query: q };
|
||||
return null;
|
||||
}
|
||||
|
||||
export function isSemanticOn() {
|
||||
return sem.on;
|
||||
}
|
||||
|
||||
/**
|
||||
* 表示条件(類似度のしきい値・最大件数)。スコアの水準はモデルで違うため、モデルごとに保存する。
|
||||
* similar: 類似画像検索の最小スコア / text: 文章検索の最小スコア(0=絞り込みなし)
|
||||
*/
|
||||
function loadThresholds(model) {
|
||||
let v = {};
|
||||
try { v = JSON.parse(lsGet(LS_TH, "{}"))[model] || {}; } catch { /* 壊れた値は既定に戻す */ }
|
||||
const num = x => { const n = parseFloat(x); return Number.isFinite(n) ? Math.min(1, Math.max(0, n)) : 0; };
|
||||
return { similar: num(v.similar), text: num(v.text), limit: LIMITS.includes(v.limit) ? v.limit : 200 };
|
||||
}
|
||||
|
||||
function saveThresholds(model, th) {
|
||||
let all = {};
|
||||
try { all = JSON.parse(lsGet(LS_TH, "{}")) || {}; } catch { all = {}; }
|
||||
all[model] = th;
|
||||
lsSet(LS_TH, JSON.stringify(all));
|
||||
}
|
||||
|
||||
function minScoreFor(mode) {
|
||||
const th = loadThresholds(sem.indexModel);
|
||||
return mode.kind === "similar" ? th.similar : th.text;
|
||||
}
|
||||
|
||||
/** モードに応じてサーバーから結果(スコア降順・score付き)を取得する */
|
||||
export async function fetchSemanticImages(mode, folder) {
|
||||
const th = loadThresholds(sem.indexModel);
|
||||
const common = { folder, recursive: true, limit: th.limit, min_score: minScoreFor(mode) };
|
||||
const data = mode.kind === "similar"
|
||||
? await postJson(API.similar, { path: mode.path, ...common })
|
||||
: await postJson(API.search, { query: mode.query, ...common });
|
||||
return data.images || [];
|
||||
}
|
||||
|
||||
/** 2行目の結果件数表示(意味検索/類似表示のときだけ文言を入れる。通常は空) */
|
||||
export function setSemanticCountText(text) {
|
||||
const el = document.getElementById("wfm-gallery-sem-count");
|
||||
if (el) el.textContent = text || "";
|
||||
}
|
||||
|
||||
/** 一覧件数ラベル */
|
||||
export function semanticCountLabel(mode, count) {
|
||||
const min = minScoreFor(mode);
|
||||
return mode.kind === "similar"
|
||||
? t("semCountSimilar", count, mode.name, min)
|
||||
: t("semCountText", count, min);
|
||||
}
|
||||
|
||||
let _dotRoot = "";
|
||||
let _dotSeq = 0;
|
||||
|
||||
/**
|
||||
* フォルダツリーの枚数バッジ横に、直下の画像が全件インデックス済みのフォルダへドットを付ける。
|
||||
* ツリー再構築後・インデックス作成後に呼ぶ(DOMはその場で更新し、ツリーは作り直さない)。
|
||||
*/
|
||||
export async function refreshSemanticDots(root) {
|
||||
if (root) _dotRoot = root;
|
||||
if (!_dotRoot) return;
|
||||
const seq = ++_dotSeq;
|
||||
let folders;
|
||||
try {
|
||||
const res = await fetch(API.folders(_dotRoot));
|
||||
if (!res.ok) return;
|
||||
folders = (await res.json()).folders || {};
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (seq !== _dotSeq) return; // より新しい呼び出しがある
|
||||
document.querySelectorAll(".wfm-gallery-tree-item").forEach(item => {
|
||||
const label = item.querySelector(".wfm-gallery-tree-label");
|
||||
const badge = label?.querySelector(".wfm-gallery-tree-count");
|
||||
label?.querySelector(".wfm-gallery-tree-sem-dot")?.remove();
|
||||
const st = folders[(item.dataset.path || "").replace(/\\/g, "/")];
|
||||
if (badge && st && st.indexable > 0 && st.indexed >= st.indexable) {
|
||||
const dot = document.createElement("span");
|
||||
dot.className = "wfm-gallery-tree-sem-dot";
|
||||
dot.title = t("semDotTitle");
|
||||
badge.after(dot);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** フォルダ切替・フィルタクリア時に類似表示を解除する(再読込は呼び出し側) */
|
||||
export function exitSimilarMode() {
|
||||
sem.anchor = "";
|
||||
sem.anchorName = "";
|
||||
updateSimilarButtons();
|
||||
}
|
||||
|
||||
// ── トグル/類似ボタン ───────────────────────────────────────
|
||||
|
||||
function updateToggleUi() {
|
||||
const btn = document.getElementById("wfm-gallery-semantic-toggle");
|
||||
btn?.classList.toggle("active", sem.on);
|
||||
// ON のときだけ2行目(Similar / 更新 / 設定 / 件数)を表示する
|
||||
const row = document.getElementById("wfm-gallery-sem-row");
|
||||
if (row) row.style.display = sem.on ? "flex" : "none";
|
||||
const input = document.getElementById("wfm-gallery-search");
|
||||
if (input) input.placeholder = sem.on ? t("semSearchPlaceholder") : "Search images...";
|
||||
}
|
||||
|
||||
function updateSimilarButtons() {
|
||||
document.getElementById("wfm-gallery-similar-btn")?.classList.toggle("active", !!sem.anchor);
|
||||
}
|
||||
|
||||
async function toggleSimilar() {
|
||||
if (sem.anchor) {
|
||||
exitSimilarMode();
|
||||
ctx.reload();
|
||||
return;
|
||||
}
|
||||
await applySimilar();
|
||||
}
|
||||
|
||||
async function applySimilar() {
|
||||
const img = ctx.getSelected();
|
||||
if (!img) {
|
||||
showToast(t("gallerySelectImageFirst"), "error");
|
||||
return;
|
||||
}
|
||||
sem.anchor = img.path;
|
||||
sem.anchorName = img.filename;
|
||||
updateSimilarButtons();
|
||||
ctx.reload();
|
||||
}
|
||||
|
||||
// ── インデックス作成モーダル ─────────────────────────────────
|
||||
|
||||
function modalHtml() {
|
||||
const tab = ["index", "model", "search"].includes(lsGet(LS_TAB, "index")) ? lsGet(LS_TAB, "index") : "index";
|
||||
const pane = (id, body) => `<div class="wfm-sem-pane" data-sem-pane="${id}" style="${tab === id ? "" : "display:none;"}">${body}</div>`;
|
||||
const tabBtn = (id, label) =>
|
||||
`<button type="button" class="wfm-sem-tab-btn${tab === id ? " active" : ""}" data-sem-tab="${id}">${escapeHtml(label)}</button>`;
|
||||
const indexPane = `
|
||||
<div class="wfm-sem-status" id="wfm-sem-status">${escapeHtml(t("loading"))}</div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semScope"))}</b>
|
||||
<span id="wfm-sem-folder" class="wfm-sem-folder"></span></div>
|
||||
<label class="wfm-sem-row"><input type="checkbox" id="wfm-sem-recursive" ${lsGet(LS_SUB, "1") === "1" ? "checked" : ""}>
|
||||
${escapeHtml(t("semIncludeSub"))}</label>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semMode"))}</b>
|
||||
<label><input type="radio" name="wfm-sem-mode" value="missing" checked> ${escapeHtml(t("semModeMissing"))}</label>
|
||||
<label><input type="radio" name="wfm-sem-mode" value="all"> ${escapeHtml(t("semModeAll"))}</label></div>
|
||||
<div class="wfm-sem-progress" id="wfm-sem-progress" style="display:none;">
|
||||
<div class="wfm-sem-bar"><div class="wfm-sem-bar-fill" id="wfm-sem-bar-fill"></div></div>
|
||||
<div id="wfm-sem-progress-text" class="wfm-sem-hint"></div>
|
||||
</div>
|
||||
<div class="wfm-sem-actions">
|
||||
<button class="wfm-btn wfm-btn-primary" id="wfm-sem-start">${escapeHtml(t("semStart"))}</button>
|
||||
<button class="wfm-btn" id="wfm-sem-cancel" disabled>${escapeHtml(t("semCancelJob"))}</button>
|
||||
<button class="wfm-btn wfm-sem-danger" id="wfm-sem-clear">${escapeHtml(t("semClear"))}</button>
|
||||
</div>`;
|
||||
const modelPane = `
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semModel"))}</b>
|
||||
<label><input type="radio" name="wfm-sem-model" value="${GEMINI}"> ${escapeHtml(t("semModelGemini"))}</label>
|
||||
<label><input type="radio" name="wfm-sem-model" value="${LOCAL}"> ${escapeHtml(t("semModelLocal"))}</label></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semDimension"))}</b>
|
||||
<select id="wfm-sem-dim" class="wfm-select"></select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semDimHint"))}</span></div>
|
||||
<div id="wfm-sem-local-opts" class="wfm-sem-local-opts" style="display:none;">
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semDevice"))}</b>
|
||||
<select id="wfm-sem-device" class="wfm-select">
|
||||
<option value="auto">${escapeHtml(t("semDeviceAuto"))}</option>
|
||||
<option value="cpu">CPU</option>
|
||||
<option value="cuda">GPU (CUDA)</option>
|
||||
</select>
|
||||
<span id="wfm-sem-device-info" class="wfm-sem-hint"></span></div>
|
||||
<div class="wfm-sem-row" id="wfm-sem-threads-row"><b>${escapeHtml(t("semThreads"))}</b>
|
||||
<select id="wfm-sem-threads" class="wfm-select"></select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semThreadsHint"))}</span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semSoft"))}</b>
|
||||
<select id="wfm-sem-soft" class="wfm-select">
|
||||
<option value="280">${escapeHtml(t("semSoftHigh"))}</option>
|
||||
<option value="140">${escapeHtml(t("semSoftStd"))}</option>
|
||||
<option value="70">${escapeHtml(t("semSoftLight"))}</option>
|
||||
</select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semSoftHint"))}</span></div>
|
||||
</div>
|
||||
<div class="wfm-sem-actions">
|
||||
<button class="wfm-btn" id="wfm-sem-test">${escapeHtml(t("semTest"))}</button>
|
||||
</div>
|
||||
<p class="wfm-sem-hint">${escapeHtml(t("semCostNote"))}</p>`;
|
||||
const searchPane = `
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThTarget"))}</b>
|
||||
<span id="wfm-sem-th-model" class="wfm-sem-folder"></span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThSimilar"))}</b>
|
||||
<input type="number" id="wfm-sem-th-similar" class="wfm-input wfm-sem-num" min="0" max="1" step="0.01" placeholder="0">
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semThSimilarHint"))}</span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThText"))}</b>
|
||||
<input type="number" id="wfm-sem-th-text" class="wfm-input wfm-sem-num" min="0" max="1" step="0.01" placeholder="0">
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semThTextHint"))}</span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semThLimit"))}</b>
|
||||
<select id="wfm-sem-th-limit" class="wfm-select">${LIMITS.map(n => `<option value="${n}">${n}</option>`).join("")}</select></div>
|
||||
<p class="wfm-sem-hint">${escapeHtml(t("semThNote"))}</p>`;
|
||||
return `
|
||||
<div class="wfm-sem-modal">
|
||||
<div class="wfm-sem-tabs">${tabBtn("index", t("semTabIndex"))}${tabBtn("model", t("semTabModel"))}${tabBtn("search", t("semTabSearch"))}</div>
|
||||
${pane("index", indexPane)}${pane("model", modelPane)}${pane("search", searchPane)}
|
||||
<div class="wfm-sem-actions wfm-sem-footer">
|
||||
<button class="wfm-btn" id="wfm-sem-help" title="${escapeHtml(t("semHelpTitle"))}">?</button>
|
||||
<button class="wfm-btn" id="wfm-sem-close">${escapeHtml(t("close"))}</button>
|
||||
</div>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
function modalAlive() {
|
||||
const overlay = document.getElementById("wfm-modal-overlay");
|
||||
return !!document.getElementById("wfm-sem-status") && overlay && overlay.style.display !== "none";
|
||||
}
|
||||
|
||||
let _pollTimer = null;
|
||||
let _backends = {};
|
||||
let _sawRunning = false;
|
||||
|
||||
function selectedModel() {
|
||||
return document.querySelector('input[name="wfm-sem-model"]:checked')?.value || GEMINI;
|
||||
}
|
||||
|
||||
/** ローカルモデルの実行設定(デバイス/スレッド/画像精度)の初期値をサーバー保存値から設定する */
|
||||
function initLocalOpts(st) {
|
||||
const lo = st.local || {};
|
||||
const devSel = document.getElementById("wfm-sem-device");
|
||||
devSel.value = (lo.options && lo.options.device) || "auto";
|
||||
const cores = lo.devices?.cpu_count || 4;
|
||||
const cur = (lo.options && lo.options.threads) || lo.default_threads || Math.max(1, Math.floor(cores / 2));
|
||||
const set = new Set([1, 2, 4, 6, 8, 12, 16, 24, 32].filter(n => n <= cores).concat([cores, cur, lo.default_threads || 1]));
|
||||
document.getElementById("wfm-sem-threads").innerHTML = [...set].sort((a, b) => a - b)
|
||||
.map(n => `<option value="${n}">${n}${n === cores ? ` (${t("semThreadsAll")})` : ""}</option>`).join("");
|
||||
document.getElementById("wfm-sem-threads").value = String(cur);
|
||||
const soft = st.indexed_total && st.index_model === LOCAL ? st.index_soft : parseInt(lsGet(LS_SOFT, String(lo.default_soft || 280)), 10);
|
||||
document.getElementById("wfm-sem-soft").value = String([70, 140, 280].includes(soft) ? soft : 280);
|
||||
}
|
||||
|
||||
/** 検索タブ: しきい値の入力欄にインデックスのモデル用の保存値を入れる(入力中の値は上書きしない) */
|
||||
function fillThresholds(st) {
|
||||
const model = st.indexed_total ? st.index_model : selectedModel();
|
||||
const box = document.getElementById("wfm-sem-th-model");
|
||||
if (box) box.textContent = model + (st.indexed_total ? "" : ` ${t("semThNoIndex")}`);
|
||||
const pane = document.getElementById("wfm-sem-th-similar");
|
||||
if (!pane || pane.dataset.model === model) return;
|
||||
const th = loadThresholds(model);
|
||||
pane.value = th.similar || "";
|
||||
document.getElementById("wfm-sem-th-text").value = th.text || "";
|
||||
document.getElementById("wfm-sem-th-limit").value = String(th.limit);
|
||||
pane.dataset.model = model;
|
||||
}
|
||||
|
||||
/** 検索タブの入力を保存する。一覧が意味検索/類似表示中なら閉じるときに再読込する */
|
||||
let _thDirty = false;
|
||||
function saveThresholdInputs() {
|
||||
const model = document.getElementById("wfm-sem-th-similar")?.dataset.model;
|
||||
if (!model) return;
|
||||
const clamp = id => {
|
||||
const n = parseFloat(document.getElementById(id).value);
|
||||
return Number.isFinite(n) ? Math.min(1, Math.max(0, n)) : 0;
|
||||
};
|
||||
saveThresholds(model, {
|
||||
similar: clamp("wfm-sem-th-similar"),
|
||||
text: clamp("wfm-sem-th-text"),
|
||||
limit: parseInt(document.getElementById("wfm-sem-th-limit").value, 10) || 200,
|
||||
});
|
||||
_thDirty = true;
|
||||
}
|
||||
|
||||
/** モーダルを閉じる共通処理 */
|
||||
function closeSemanticModal() {
|
||||
clearTimeout(_pollTimer);
|
||||
closeModal();
|
||||
refreshSemanticDots();
|
||||
if (_thDirty) {
|
||||
_thDirty = false;
|
||||
if (getSemanticMode()) ctx.reload(); // 新しいしきい値で一覧を取り直す
|
||||
}
|
||||
}
|
||||
|
||||
/** 検出した実行デバイスの表示(GPU版onnxruntimeが無いときの案内を含む) */
|
||||
function updateDeviceInfo(lo) {
|
||||
const dev = lo?.devices;
|
||||
if (!dev) return;
|
||||
const info = document.getElementById("wfm-sem-device-info");
|
||||
const cudaOpt = document.querySelector('#wfm-sem-device option[value="cuda"]');
|
||||
if (cudaOpt) cudaOpt.disabled = !dev.cuda;
|
||||
if (!dev.cuda && document.getElementById("wfm-sem-device").value === "cuda") {
|
||||
document.getElementById("wfm-sem-device").value = "auto";
|
||||
}
|
||||
if (info) info.textContent = dev.cuda ? t("semDeviceCudaOk", dev.ort_version) : t("semDeviceNoGpu", dev.ort_version);
|
||||
const dv = document.getElementById("wfm-sem-device").value;
|
||||
const useCpu = dv === "cpu" || (dv === "auto" && !dev.cuda);
|
||||
document.getElementById("wfm-sem-threads-row").style.display = useCpu ? "" : "none";
|
||||
}
|
||||
|
||||
/** 選択中モデルの次元選択肢を作り直す(preferredが選択肢にあればそれを選ぶ) */
|
||||
function fillDims(model, preferred) {
|
||||
const sel = document.getElementById("wfm-sem-dim");
|
||||
const info = _backends[model];
|
||||
if (!sel || !info) return;
|
||||
sel.innerHTML = info.dims.map(d => `<option value="${d}">${d}</option>`).join("");
|
||||
sel.value = String(info.dims.includes(preferred) ? preferred : info.default_dim);
|
||||
}
|
||||
|
||||
async function refreshModal() {
|
||||
if (!modalAlive()) return;
|
||||
const folder = ctx.getFolder();
|
||||
const recursive = document.getElementById("wfm-sem-recursive")?.checked ?? true;
|
||||
let st;
|
||||
try {
|
||||
const res = await fetch(API.status(folder, recursive));
|
||||
st = await res.json();
|
||||
if (!res.ok) throw new Error(st.error || `HTTP ${res.status}`);
|
||||
} catch (e) {
|
||||
document.getElementById("wfm-sem-status").textContent = t("errorWithMsg", e.message);
|
||||
return;
|
||||
}
|
||||
if (!modalAlive()) return;
|
||||
const job = st.job || {};
|
||||
_backends = st.backends || {};
|
||||
if (st.indexed_total) sem.indexModel = st.index_model;
|
||||
if (!document.getElementById("wfm-sem-dim").dataset.init) {
|
||||
// 初回のみ: 既存インデックスのモデル/次元、なければ前回の選択を初期値にする
|
||||
const initModel = st.indexed_total ? st.index_model : lsGet(LS_MODEL, GEMINI);
|
||||
const radio = document.querySelector(`input[name="wfm-sem-model"][value="${initModel}"]`)
|
||||
|| document.querySelector(`input[name="wfm-sem-model"][value="${GEMINI}"]`);
|
||||
radio.checked = true;
|
||||
fillDims(radio.value, st.indexed_total ? st.index_dim : parseInt(lsGet(LS_DIM, "0"), 10));
|
||||
initLocalOpts(st);
|
||||
document.getElementById("wfm-sem-dim").dataset.init = "1";
|
||||
}
|
||||
fillThresholds(st);
|
||||
const model = selectedModel();
|
||||
document.getElementById("wfm-sem-local-opts").style.display = model === LOCAL ? "" : "none";
|
||||
updateDeviceInfo(st.local);
|
||||
const ready = model === LOCAL ? true : !!st.has_key;
|
||||
const keyLine = model === LOCAL
|
||||
? (_backends[LOCAL]?.downloaded ? t("semLocalReady") : t("semLocalNeedDownload"))
|
||||
: (st.has_key ? t("semKeyOk") : t("semKeyMissing"));
|
||||
const idxLine = t("semIndexInfo", st.indexed_total, st.index_dim || "-");
|
||||
const scopeLine = st.scope_total != null ? t("semScopeInfo", st.scope_indexed, st.scope_total) : "";
|
||||
document.getElementById("wfm-sem-status").innerHTML =
|
||||
`<div class="${ready ? "wfm-sem-ok" : "wfm-sem-ng"}">${escapeHtml(keyLine)}</div>` +
|
||||
`<div>${escapeHtml(st.indexed_total ? st.index_model : "-")} / ${escapeHtml(idxLine)}</div>` +
|
||||
(scopeLine ? `<div>${escapeHtml(scopeLine)}</div>` : "");
|
||||
document.getElementById("wfm-sem-folder").textContent = folder || t("semNoFolder");
|
||||
|
||||
const prog = document.getElementById("wfm-sem-progress");
|
||||
const hasJob = job.running || job.finished;
|
||||
prog.style.display = hasJob ? "" : "none";
|
||||
if (hasJob) {
|
||||
const total = job.total || 0;
|
||||
const doneAll = (job.done || 0) + (job.failed || 0);
|
||||
document.getElementById("wfm-sem-bar-fill").style.width = total ? `${Math.min(100, (doneAll / total) * 100)}%` : "100%";
|
||||
let text = t("semProgress", job.done || 0, total, job.failed || 0);
|
||||
if (job.running && job.current) text += ` ${job.current}`;
|
||||
if (!job.running) {
|
||||
if (["no_api_key", "api_key_invalid", "local_error"].includes(job.error_code)) text += ` ${t(`sem_err_${job.error_code}`, job.error)}`;
|
||||
else text += ` ${job.cancelled ? t("semCancelled") : t("semFinished")}`;
|
||||
if (job.failed && job.error) text += ` (${job.error})`;
|
||||
}
|
||||
document.getElementById("wfm-sem-progress-text").textContent = text;
|
||||
}
|
||||
document.getElementById("wfm-sem-start").disabled = !!job.running || !ready;
|
||||
document.getElementById("wfm-sem-cancel").disabled = !job.running;
|
||||
document.getElementById("wfm-sem-clear").disabled = !!job.running;
|
||||
document.getElementById("wfm-sem-test").disabled = !ready;
|
||||
|
||||
clearTimeout(_pollTimer);
|
||||
if (job.running) {
|
||||
_pollTimer = setTimeout(refreshModal, 1000);
|
||||
_sawRunning = true;
|
||||
} else if (_sawRunning) {
|
||||
_sawRunning = false;
|
||||
refreshSemanticDots();
|
||||
}
|
||||
}
|
||||
|
||||
/** ヘルプタブのGallery > セマンティック検索の説明へ飛ぶ */
|
||||
function openSemanticHelp() {
|
||||
closeSemanticModal();
|
||||
document.querySelector('.wfm-tab[data-tab="help"]')?.click();
|
||||
document.querySelector('.wfm-help-nav-item[data-help-page="gallery"]')?.click();
|
||||
const target = document.getElementById("wfm-help-gallery-33");
|
||||
if (!target) return;
|
||||
target.scrollIntoView({ behavior: "smooth", block: "center" });
|
||||
target.classList.add("wfm-help-flash");
|
||||
setTimeout(() => target.classList.remove("wfm-help-flash"), 2500);
|
||||
}
|
||||
|
||||
function openSemanticModal() {
|
||||
if (!ctx.getFolder()) {
|
||||
showToast(t("semNoFolder"), "error");
|
||||
return;
|
||||
}
|
||||
openModal(t("semModalTitle"), modalHtml());
|
||||
const dimOf = () => parseInt(document.getElementById("wfm-sem-dim").value, 10) || 0;
|
||||
// ローカルモデル選択時のみ実行設定を送る(サーバー側で保存され、検索時のクエリ埋め込みにも使われる)
|
||||
const localOpts = () => selectedModel() === LOCAL ? {
|
||||
device: document.getElementById("wfm-sem-device").value,
|
||||
threads: parseInt(document.getElementById("wfm-sem-threads").value, 10) || 0,
|
||||
soft: parseInt(document.getElementById("wfm-sem-soft").value, 10) || 280,
|
||||
} : {};
|
||||
document.getElementById("wfm-sem-device").addEventListener("change", () => refreshModal());
|
||||
document.querySelectorAll('input[name="wfm-sem-model"]').forEach(r => r.addEventListener("change", () => {
|
||||
fillDims(selectedModel(), 0);
|
||||
refreshModal();
|
||||
}));
|
||||
|
||||
document.getElementById("wfm-sem-recursive").addEventListener("change", (e) => {
|
||||
lsSet(LS_SUB, e.target.checked ? "1" : "0");
|
||||
refreshModal();
|
||||
});
|
||||
document.getElementById("wfm-sem-close").addEventListener("click", closeSemanticModal);
|
||||
document.querySelectorAll(".wfm-sem-tab-btn").forEach(btn => btn.addEventListener("click", () => {
|
||||
document.querySelectorAll(".wfm-sem-tab-btn").forEach(b => b.classList.toggle("active", b === btn));
|
||||
document.querySelectorAll(".wfm-sem-pane").forEach(p => { p.style.display = p.dataset.semPane === btn.dataset.semTab ? "" : "none"; });
|
||||
lsSet(LS_TAB, btn.dataset.semTab);
|
||||
}));
|
||||
["wfm-sem-th-similar", "wfm-sem-th-text"].forEach(id => document.getElementById(id).addEventListener("change", saveThresholdInputs));
|
||||
document.getElementById("wfm-sem-th-limit").addEventListener("change", saveThresholdInputs);
|
||||
document.getElementById("wfm-sem-help").addEventListener("click", openSemanticHelp);
|
||||
document.getElementById("wfm-sem-start").addEventListener("click", async () => {
|
||||
const mode = document.querySelector('input[name="wfm-sem-mode"]:checked')?.value || "missing";
|
||||
if (mode === "all" && !confirm(t("semConfirmAll"))) return;
|
||||
lsSet(LS_DIM, String(dimOf()));
|
||||
lsSet(LS_MODEL, selectedModel());
|
||||
lsSet(LS_SOFT, document.getElementById("wfm-sem-soft").value);
|
||||
try {
|
||||
await postJson(API.index, {
|
||||
folder: ctx.getFolder(),
|
||||
recursive: document.getElementById("wfm-sem-recursive").checked,
|
||||
mode,
|
||||
model: selectedModel(),
|
||||
dim: dimOf(),
|
||||
...localOpts(),
|
||||
});
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
refreshModal();
|
||||
});
|
||||
document.getElementById("wfm-sem-cancel").addEventListener("click", async () => {
|
||||
await postJson(API.cancel, {}).catch(() => {});
|
||||
refreshModal();
|
||||
});
|
||||
document.getElementById("wfm-sem-test").addEventListener("click", async () => {
|
||||
try {
|
||||
const r = await postJson(API.test, { model: selectedModel(), dim: dimOf(), ...localOpts() });
|
||||
showToast(r.provider && r.provider !== "API" ? t("semTestOkProvider", r.dimensions, r.provider) : t("semTestOk", r.dimensions), "success");
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
});
|
||||
document.getElementById("wfm-sem-clear").addEventListener("click", async () => {
|
||||
if (!confirm(t("semConfirmClear"))) return;
|
||||
try {
|
||||
await postJson(API.clear, {});
|
||||
showToast(t("semCleared"), "success");
|
||||
refreshSemanticDots();
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
refreshModal();
|
||||
});
|
||||
refreshModal();
|
||||
}
|
||||
|
||||
// ── 初期化 ───────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* @param {object} context
|
||||
* getSearch(): 検索ボックスの文字列 / getFolder(): 選択中フォルダ / getSelected(): 選択中画像
|
||||
* reload(): 一覧再読込(loadImages)
|
||||
*/
|
||||
export function initSemantic(context) {
|
||||
ctx = context;
|
||||
sem.on = lsGet(LS_ON, "0") === "1";
|
||||
updateToggleUi();
|
||||
updateSimilarButtons();
|
||||
|
||||
document.getElementById("wfm-gallery-semantic-toggle")?.addEventListener("click", () => {
|
||||
sem.on = !sem.on;
|
||||
lsSet(LS_ON, sem.on ? "1" : "0");
|
||||
updateToggleUi();
|
||||
// OFFにしたら類似表示も解除する(2行目ごと消えるため)
|
||||
const wasSimilar = !!sem.anchor;
|
||||
if (!sem.on) exitSimilarMode();
|
||||
// 検索文字列があれば、モード切替後の方式で結果を取り直す
|
||||
if (wasSimilar && !sem.on || (ctx.getSearch() || "").trim()) ctx.reload();
|
||||
});
|
||||
fetch(API.status("", true)).then(r => r.json()).then(st => {
|
||||
if (st.indexed_total) sem.indexModel = st.index_model;
|
||||
}).catch(() => { /* 取得できなくても既定のモデル扱いで動作する */ });
|
||||
document.getElementById("wfm-gallery-semantic-settings-btn")?.addEventListener("click", openSemanticModal);
|
||||
document.getElementById("wfm-gallery-similar-btn")?.addEventListener("click", toggleSimilar);
|
||||
document.getElementById("wfm-gallery-similar-refresh-btn")?.addEventListener("click", applySimilar);
|
||||
}
|
||||
@@ -15,6 +15,7 @@ import { callLLM, loadAiSettings, isValidBackendUrl } from "./ai-tab.js";
|
||||
import { initImagePromptTab, activateImagePromptTab } from "./image-prompt-tab.js";
|
||||
import { initStyleCatalogTab, activateStyleCatalogTab } from "./style-catalog-tab.js";
|
||||
import { applyStoredVideoVolume } from "./settings-tab.js";
|
||||
import { initSemantic, refreshSemanticDots, getSemanticMode, isSemanticOn, fetchSemanticImages, semanticCountLabel, setSemanticCountText, semanticErrorMessage, exitSimilarMode } from "./gallery-semantic.js";
|
||||
|
||||
// ── 定数 ─────────────────────────────────────────────────────
|
||||
|
||||
@@ -25,7 +26,7 @@ const API = {
|
||||
imageWorkflow: (path) => `/wfm/gallery/image/workflow?path=${encodeURIComponent(path)}`,
|
||||
serveImage: (path) => `/wfm/gallery/image/serve?path=${encodeURIComponent(path)}`,
|
||||
// 音声の波形サムネイルは描画仕様を変えた際にブラウザのHTTPキャッシュ(24h)を回避するため版数を付ける
|
||||
thumb: (path, w = 256) => `/wfm/gallery/image/thumb?path=${encodeURIComponent(path)}&w=${w}${/\.(wav|mp3|ogg|m4a|flac|opus)$/i.test(path) ? "&v=wave2" : ""}`,
|
||||
thumb: (path, w = 256) => `/wfm/gallery/image/thumb?path=${encodeURIComponent(path)}&w=${w}${isAudioFile({ path }) ? `&v=${WAVE_THUMB_VERSION}` : ""}`,
|
||||
bulkFavorite: "/wfm/gallery/bulk/favorite",
|
||||
bulkGroup: "/wfm/gallery/bulk/group",
|
||||
convert: "/wfm/gallery/convert",
|
||||
@@ -68,6 +69,9 @@ export function isVideoFile(img) {
|
||||
return VIDEO_EXTENSIONS.some(ext => source.endsWith(ext));
|
||||
}
|
||||
|
||||
// 音声の波形サムネイルの版数。py/services/gallery_service.py の WAVE_THUMB_VERSION と同じ値にすること。
|
||||
const WAVE_THUMB_VERSION = "wave2";
|
||||
|
||||
// 音声ファイル(ComfyUIの音楽生成出力)。サムネイルはサーバー生成の波形画像、再生は<audio>。
|
||||
const AUDIO_EXTENSIONS = [".wav", ".mp3", ".ogg", ".m4a", ".flac", ".opus"];
|
||||
export function isAudioFile(img) {
|
||||
@@ -450,6 +454,7 @@ async function loadFolderTree() {
|
||||
state.folderTree = data;
|
||||
tree.innerHTML = "";
|
||||
renderTreeNode(data, tree, 0, true);
|
||||
refreshSemanticDots(state.outputRoot);
|
||||
|
||||
if (isFirstLoad) {
|
||||
// 初回のみ root を自動選択
|
||||
@@ -505,6 +510,7 @@ function renderTreeNode(node, container, depth, isRoot) {
|
||||
state.selectedImages.clear();
|
||||
state.lastSelectionIndex = -1;
|
||||
updateBulkBar();
|
||||
exitSimilarMode();
|
||||
loadImages();
|
||||
// Delete Folder ボタン: rootは削除不可
|
||||
const delBtn = document.getElementById("wfm-gallery-folder-delete-btn");
|
||||
@@ -588,15 +594,37 @@ async function loadImages() {
|
||||
params.recursive = "true";
|
||||
}
|
||||
|
||||
// セマンティック検索/類似画像モード: サーバーがスコア順で返すためソート・各フィルタは適用しない
|
||||
const semMode = getSemanticMode();
|
||||
|
||||
try {
|
||||
if (semMode) {
|
||||
const images = await fetchSemanticImages(semMode, state.currentFolder);
|
||||
if (controller.signal.aborted) return;
|
||||
state.images = images;
|
||||
state.lastSelectionIndex = -1;
|
||||
document.getElementById("wfm-gallery-count").textContent = `${images.length} images`;
|
||||
setSemanticCountText(semanticCountLabel(semMode, images.length));
|
||||
renderImages();
|
||||
return;
|
||||
}
|
||||
const images = (await apiFetch(API.images(params), { signal: controller.signal })).images || [];
|
||||
state.images = images;
|
||||
state.lastSelectionIndex = -1;
|
||||
document.getElementById("wfm-gallery-count").textContent = `${state.images.length} images`;
|
||||
setSemanticCountText("");
|
||||
renderImages();
|
||||
updateTagFilter(state.images);
|
||||
} catch (e) {
|
||||
if (e.name === "AbortError") return;
|
||||
if (semMode) {
|
||||
state.images = [];
|
||||
document.getElementById("wfm-gallery-count").textContent = "";
|
||||
setSemanticCountText("");
|
||||
grid.innerHTML = `<p class="wfm-placeholder">${escapeHtml(semanticErrorMessage(e))}</p>`;
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
return;
|
||||
}
|
||||
grid.innerHTML = `<p class="wfm-placeholder">Error: ${escapeHtml(e.message)}</p>`;
|
||||
} finally {
|
||||
if (_loadImagesAbortController === controller) {
|
||||
@@ -702,6 +730,14 @@ function createThumbCard(img) {
|
||||
card.appendChild(playBadge);
|
||||
}
|
||||
|
||||
// セマンティック検索/類似画像の結果にはコサイン類似度を表示
|
||||
if (img.score != null) {
|
||||
const scoreBadge = document.createElement("div");
|
||||
scoreBadge.className = "wfm-gallery-thumb-score";
|
||||
scoreBadge.textContent = img.score.toFixed(3);
|
||||
card.appendChild(scoreBadge);
|
||||
}
|
||||
|
||||
// お気に入りトグルボタン(カード右上)
|
||||
const favBtn = document.createElement("button");
|
||||
favBtn.className = `wfm-gallery-thumb-fav-btn${img.favorite ? " active" : ""}`;
|
||||
@@ -823,7 +859,7 @@ function createTable(images) {
|
||||
tdSize.textContent = formatBytes(img.size);
|
||||
|
||||
const tdDate = document.createElement("td");
|
||||
tdDate.textContent = formatDate(img.mtime);
|
||||
tdDate.textContent = formatDate(img.mtime) + (img.score != null ? ` (${img.score.toFixed(3)})` : "");
|
||||
|
||||
const tdTags = document.createElement("td");
|
||||
tdTags.innerHTML = (img.tags || []).map(tag => `<span class="wfm-gallery-tag-badge">${escapeHtml(tag)}</span>`).join("");
|
||||
@@ -905,7 +941,7 @@ function updateBulkBar() {
|
||||
// Convert ボタンは音声/動画を含む選択時のみ表示
|
||||
const convertBtn = document.getElementById("wfm-gallery-bulk-convert");
|
||||
if (convertBtn) {
|
||||
convertBtn.style.display = count > 0 && convertTargets([...state.selectedImages]).length > 0 ? "" : "none";
|
||||
convertBtn.style.display = count > 0 && [...state.selectedImages].some(p => isConvertible(imageByPath(p))) ? "" : "none";
|
||||
convertBtn.textContent = t("galleryBulkConvert");
|
||||
}
|
||||
// Compare ボタンは 2〜4 枚選択時のみ表示
|
||||
@@ -1790,10 +1826,24 @@ const CONVERT_FORMATS = {
|
||||
wav: { bitrate: false },
|
||||
};
|
||||
|
||||
// パス→画像のMap。state.imagesは再代入/filterで新しい配列になるため、配列ごとにキャッシュする
|
||||
// (選択のたびに O(選択数 × 画像数) の線形探索をしないため)。
|
||||
const _imageByPathCache = new WeakMap();
|
||||
function imageByPath(path) {
|
||||
let map = _imageByPathCache.get(state.images);
|
||||
if (!map) {
|
||||
map = new Map(state.images.map(i => [i.path, i]));
|
||||
_imageByPathCache.set(state.images, map);
|
||||
}
|
||||
return map.get(path);
|
||||
}
|
||||
|
||||
function isConvertible(img) {
|
||||
return !!img && (isAudioFile(img) || isVideoFile(img));
|
||||
}
|
||||
|
||||
function convertTargets(paths) {
|
||||
return paths
|
||||
.map(p => state.images.find(i => i.path === p))
|
||||
.filter(img => img && (isAudioFile(img) || isVideoFile(img)));
|
||||
return paths.map(imageByPath).filter(isConvertible);
|
||||
}
|
||||
|
||||
function openConvertModal(paths) {
|
||||
@@ -2337,8 +2387,17 @@ function bindEvents() {
|
||||
document.getElementById("wfm-gallery-search")?.addEventListener("input", (e) => {
|
||||
state.search = e.target.value;
|
||||
clearTimeout(_searchDebounceTimer);
|
||||
// Semantic Search ON中は入力のたびにAPIを呼ばず、Enterで実行する(クエリ埋め込みにAPI呼び出しが要るため)。
|
||||
// 全文消去した時だけ通常一覧へ即復帰する。
|
||||
if (isSemanticOn() && state.search.trim()) return;
|
||||
_searchDebounceTimer = setTimeout(loadImages, SEARCH_DEBOUNCE_MS);
|
||||
});
|
||||
document.getElementById("wfm-gallery-search")?.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Enter" && isSemanticOn() && !e.isComposing) {
|
||||
clearTimeout(_searchDebounceTimer);
|
||||
loadImages();
|
||||
}
|
||||
});
|
||||
setupSearchClearBtn("wfm-gallery-search", "wfm-gallery-search-clear-btn", () => {
|
||||
clearTimeout(_searchDebounceTimer);
|
||||
state.search = "";
|
||||
@@ -2509,7 +2568,7 @@ function bindEvents() {
|
||||
if (!window._wfmImageEditTab) return;
|
||||
|
||||
const allPaths = [...state.selectedImages];
|
||||
const paths = allPaths.filter(p => !/\.(mp4|webm|wav|mp3|ogg|m4a|flac|opus)$/i.test(p));
|
||||
const paths = allPaths.filter(p => !isVideoFile({ path: p }) && !isAudioFile({ path: p }));
|
||||
if (paths.length === 0) {
|
||||
showToast(t("editLayersNoImages"), "error");
|
||||
return;
|
||||
@@ -2636,7 +2695,15 @@ function bindEvents() {
|
||||
if (groupFilter) groupFilter.value = "";
|
||||
const favBtn = document.getElementById("wfm-gallery-fav-btn");
|
||||
if (favBtn) favBtn.classList.remove("active");
|
||||
exitSimilarMode();
|
||||
loadImages();
|
||||
});
|
||||
}
|
||||
|
||||
initSemantic({
|
||||
getSearch: () => state.search,
|
||||
getFolder: () => state.currentFolder,
|
||||
getSelected: () => state.selectedImage,
|
||||
reload: loadImages,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -605,6 +605,75 @@ const LANGUAGES = {
|
||||
gallerySentCC: "Sent to ComfyUI Comic Creator",
|
||||
galleryVideoNotSupportedInImageTab: "Sending a video to the Image tab is not supported yet",
|
||||
galleryNoEmbeddedWorkflow: "No ComfyUI workflow embedded in this image.",
|
||||
semSearchPlaceholder: "Describe it (Enter)...",
|
||||
semModalTitle: "Semantic Search Index",
|
||||
semCountText: (n, min) => `${n} results (semantic${min > 0 ? `, score >= ${min}` : ""})`,
|
||||
semCountSimilar: (n, name, min) => `${n} similar to "${name}"${min > 0 ? ` (score >= ${min})` : ""}`,
|
||||
semTabIndex: "Index",
|
||||
semTabModel: "Model",
|
||||
semTabSearch: "Search",
|
||||
semThTarget: "Applies to:",
|
||||
semThNoIndex: "(no index yet; shown for the selected model)",
|
||||
semThSimilar: "Similar images - minimum score:",
|
||||
semThSimilarHint: "e.g. 0.6 shows only images scoring 0.6 or higher. 0 = no filter",
|
||||
semThText: "Text search - minimum score:",
|
||||
semThTextHint: "0 = no filter",
|
||||
semThLimit: "Max results:",
|
||||
semThNote: "Score levels differ by model (in a small test, similar images scored about 0.6-0.8 with EmbeddingGemma 2 and about 0.6 with Gemini, while text queries scored about 0.6-0.7 and about 0.3), so the value is saved per model. The current semantic/similar list is refreshed with the new value when you close this window.",
|
||||
semDevice: "Device:",
|
||||
semDeviceAuto: "Auto (GPU if available)",
|
||||
semDeviceCudaOk: (v) => `GPU (CUDA) detected - onnxruntime ${v}`,
|
||||
semDeviceNoGpu: (v) => `No GPU detected (onnxruntime ${v}, CPU only). To use the GPU, see the help ("?")`,
|
||||
semThreads: "CPU threads:",
|
||||
semThreadsAll: "all",
|
||||
semThreadsHint: "Fewer threads = lighter on the PC while generating images (slower indexing)",
|
||||
semSoft: "Image precision:",
|
||||
semSoftHigh: "High (slowest)",
|
||||
semSoftStd: "Standard (about 2.5x faster)",
|
||||
semSoftLight: "Light (about 5x faster)",
|
||||
semSoftHint: "Changing it requires \"Rebuild all\"",
|
||||
semTestOkProvider: (dim, prov) => `Connection OK (vector size ${dim}, running on ${prov})`,
|
||||
semDotTitle: "Semantic index: all images in this folder are indexed",
|
||||
semHelpTitle: "Open the help for Semantic Search",
|
||||
semModel: "Embedding model:",
|
||||
semModelGemini: "Gemini Embedding 2 (API)",
|
||||
semModelLocal: "EmbeddingGemma 2 (local, free)",
|
||||
semLocalReady: "EmbeddingGemma 2: model downloaded (runs locally, no API key needed)",
|
||||
semLocalNeedDownload: "EmbeddingGemma 2: the model (about 300 MB) will be downloaded on first use",
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2 error: ${m}`,
|
||||
semScope: "Target folder:",
|
||||
semNoFolder: "Select a folder in the Gallery first",
|
||||
semIncludeSub: "Include subfolders",
|
||||
semDimension: "Vector size:",
|
||||
semDimHint: "Larger = more precise but heavier. Changing the model or size requires \"Rebuild all\".",
|
||||
semMode: "Mode:",
|
||||
semModeMissing: "New / changed images only",
|
||||
semModeAll: "Rebuild all (target folder)",
|
||||
semStart: "Start indexing",
|
||||
semCancelJob: "Stop",
|
||||
semTest: "Test connection",
|
||||
semClear: "Delete all index data",
|
||||
semCostNote: "Gemini Embedding 2 has a free tier, but content sent on the free tier may be used by Google to improve its products; beyond the free tier it is paid (about $0.00012 per image as of Oct 2026). Indexing sends each image (resized) to the API; EmbeddingGemma 2 runs locally on the CPU (about 2-3 s per image). Searches reuse the saved vectors (only the text query is sent). Videos/PSD use a thumbnail; audio is skipped.",
|
||||
semKeyOk: "GEMINI_API_KEY: set",
|
||||
semKeyMissing: "GEMINI_API_KEY is not set — add GEMINI_API_KEY=... to the plugin's .env file and restart ComfyUI",
|
||||
semIndexInfo: (n, dim) => `indexed images: ${n} (vector size: ${dim})`,
|
||||
semScopeInfo: (done, total) => `Target folder: ${done} / ${total} indexed`,
|
||||
semProgress: (done, total, failed) => `${done} / ${total} done, ${failed} failed`,
|
||||
semCancelled: "Stopped",
|
||||
semFinished: "Finished",
|
||||
semConfirmAll: "Delete the existing vectors of the target folder and rebuild them? (API charges apply)",
|
||||
semConfirmClear: "Delete ALL semantic search index data?",
|
||||
semCleared: "Index deleted",
|
||||
semTestOk: (dim) => `Connection OK (vector size ${dim})`,
|
||||
sem_err_no_api_key: "GEMINI_API_KEY is not set (.env)",
|
||||
sem_err_api_key_invalid: (m) => `Gemini API key rejected: ${m}`,
|
||||
sem_err_index_empty: "No index yet — build one with the ⚙ button (second toolbar row, shown when S is ON) first",
|
||||
sem_err_dim_mismatch: "Model, vector size or image precision differs from the existing index — choose \"Rebuild all\"",
|
||||
sem_err_already_running: "Indexing is already running",
|
||||
sem_err_folder_not_allowed: "Folder is not allowed",
|
||||
sem_err_unsupported_type: "This file type cannot be searched",
|
||||
sem_err_network_error: (m) => `Network error: ${m}`,
|
||||
sem_err_api_error: (m) => `Gemini API error: ${m}`,
|
||||
galleryTabOutput: "Output",
|
||||
galleryTabImagePrompt: "ImagePrompt",
|
||||
galleryTabStyleCatalog: "Style_Catalog",
|
||||
@@ -1010,6 +1079,10 @@ const LANGUAGES = {
|
||||
helpGallery11: "Supports PNG (tEXt/iTXt), JPEG, WebP, GIF, MP4, WebM, PSD, and audio (WAV, MP3, OGG, M4A, FLAC, Opus — waveform thumbnail, built-in player, prompt/workflow tags written by ComfyUI's SaveAudio nodes); workflow extraction reads ComfyUI prompt/workflow keys embedded in in PNG metadata (including chunks appended after IEND by ComfyUI-Custom-Scripts), WebP EXIF, or MP4/WebM container tags (as written by SaveVideo, e.g. MiniMax H3 output); the Prompt tab also reads A1111 / Forge / Fooocus parameters text when no workflow is embedded. Audio waveform thumbnails are colored by format (MP3 blue, WAV green, FLAC amber, OGG/Opus purple, M4A rose) so formats can be told apart at a glance; the detail panel shows the waveform with the player below it.",
|
||||
helpGallery31: "Send to LV node button (toolbar, next to Send to LI node): the video equivalent — uploads the selected video and writes its filename into the file widget of the currently selected node, which must be ComfyUI's native Load Video node; only enabled when the selected item is a video (mp4/webm); requires Workflow Studio to have been opened from ComfyUI's top menu button",
|
||||
helpGallery32: "Detail panel — Prompt tab (selected by default): lists positive/negative prompts found in the embedded workflow; hidden for images in a cc_nanobanana folder, since those carry no embedded ComfyUI workflow — a Nanobanana tab (v0.7.14) appears there instead, shown only when that folder's history.jsonl generation log (written by ComfyUI Comic Creator's Nanobanana tab) has a matching entry for the selected image: engine, model, image size, seed, timestamp, and the prompt/negative prompt actually used",
|
||||
helpGallery33: "Semantic Search (v0.8.2): the S button next to the search box turns semantic search on. When ON, the search box finds images by meaning using Gemini Embedding 2 or the local EmbeddingGemma 2 (e.g. 'sunset city street') - press Enter to search; results are ordered by similarity with a score badge. When OFF it is the normal filename/tag/prompt search. Turning S on also shows a second toolbar row with Similar, the update button, the settings button and the result count. The settings button (gear) opens a window to build/update/delete the index for the selected folder (optionally with subfolders); choose the model with the radio buttons (Gemini needs GEMINI_API_KEY in the plugin's .env + a ComfyUI restart; EmbeddingGemma 2 runs locally and downloads its model on first use). Switching model or vector size needs 'Rebuild all'. Searches cover the selected folder tree; sort and tag/group filters do not apply to semantic results",
|
||||
helpGallery34: "Similar toggle and update button (second toolbar row, shown while S is ON): Similar shows images similar to the selected one (an image not yet indexed is embedded on the fly). Selecting another image keeps the list as is; press the update button to show images similar to the newly selected one, or press Similar again to return to the normal list. Changing folder, Clear filters or turning S off also leaves similar mode",
|
||||
helpGallery35: "EmbeddingGemma 2 performance: by default it runs on the CPU with half of the cores (about 6 s per image at full precision). Image precision trades accuracy for speed (Standard is about 2.5x and Light about 5x faster; vectors differ, so changing it needs 'Rebuild all'); fewer CPU threads keep the PC lighter while generating images. To use an NVIDIA GPU, install onnxruntime-gpu into ComfyUI's Python (pip uninstall onnxruntime, then pip install onnxruntime-gpu; the CUDA runtime/cuDNN that ComfyUI's PyTorch uses is picked up) and restart ComfyUI: indexing then takes about 0.1 s per image (measured on an RTX 4070, about 45x faster than the CPU) and the results are identical to the CPU's. DirectML is not supported (its vision model fails to run). The model is released from memory when indexing finishes and reloaded on the next search",
|
||||
helpGallery36: "Modal tabs and the similarity threshold: the Semantic settings window has three tabs - Index (target folder, rebuild mode, progress, delete all), Model (Gemini / EmbeddingGemma 2, vector size, device, CPU threads, image precision, connection test) and Search. On the Search tab you can set a minimum score for similar-image results (e.g. 0.6 shows only images scoring 0.6 or higher), a minimum score for text search, and the maximum number of results. Scores differ by model, so each model keeps its own values; the list header shows the active threshold",
|
||||
helpGallery28: "Create PSD bulk button: combines all selected images into a single layered PSD file and downloads it — each image becomes its own layer (named after the file), positioned top-left on a canvas sized to the largest selected image; the last selected image ends up on top Convert... bulk button (shown when audio/video files are selected): converts audio files — or extracts the audio track of videos — to FLAC / MP3 / Opus / M4A / WAV (bitrate, sample rate, channels selectable) into the same folder as new files; prompt/workflow tags are carried over for formats that can hold them (WAV cannot), favorite/tags/memo/groups can be carried over, and the originals can optionally be deleted afterwards.",
|
||||
helpGallery29: "Edit as Layers bulk button: switches to the Image Edit tab and opens all selected images at once, each as an independent layer (video files in the selection are skipped); the last selected image ends up on top",
|
||||
helpGallery30: "PSD files: thumbnails and full-size previews show all layers merged together (hidden layers excluded); the underlying PSD file itself is left untouched — only the preview shown in Gallery is a flattened rendering",
|
||||
@@ -2234,6 +2307,75 @@ const LANGUAGES = {
|
||||
gallerySentCC: "ComfyUI Comic Creatorへ送信しました",
|
||||
galleryVideoNotSupportedInImageTab: "動画のImageタブへの送信は未対応です",
|
||||
galleryNoEmbeddedWorkflow: "この画像にはComfyUIワークフローが埋め込まれていません。",
|
||||
semSearchPlaceholder: "内容を文章で入力(Enter)...",
|
||||
semModalTitle: "セマンティック検索インデックス",
|
||||
semCountText: (n, min) => `${n}件(セマンティック検索${min > 0 ? `・スコア${min}以上` : ""})`,
|
||||
semCountSimilar: (n, name, min) => `「${name}」に類似 ${n}件${min > 0 ? `(スコア${min}以上)` : ""}`,
|
||||
semTabIndex: "インデックス",
|
||||
semTabModel: "モデル設定",
|
||||
semTabSearch: "検索設定",
|
||||
semThTarget: "適用対象:",
|
||||
semThNoIndex: "(インデックス未作成のため、選択中のモデル用)",
|
||||
semThSimilar: "類似画像 - 最小スコア:",
|
||||
semThSimilarHint: "例: 0.6 なら類似度0.6以上の画像だけを表示します。0=絞り込みなし",
|
||||
semThText: "文章検索 - 最小スコア:",
|
||||
semThTextHint: "0=絞り込みなし",
|
||||
semThLimit: "最大表示件数:",
|
||||
semThNote: "スコアの水準はモデルによって違うため(少数画像での実測: 類似画像はEmbeddingGemma 2で約0.6〜0.8、Geminiで約0.6、文章検索はそれぞれ約0.6〜0.7、約0.3)、値はモデルごとに保存されます。閉じたときに、意味検索/類似表示中の一覧へ反映されます。",
|
||||
semDevice: "実行デバイス:",
|
||||
semDeviceAuto: "自動(GPUが使えればGPU)",
|
||||
semDeviceCudaOk: (v) => `GPU (CUDA) を検出 - onnxruntime ${v}`,
|
||||
semDeviceNoGpu: (v) => `GPU未検出(onnxruntime ${v}・CPUのみ)。GPUを使うには「?」のヘルプを参照`,
|
||||
semThreads: "CPUスレッド数:",
|
||||
semThreadsAll: "全コア",
|
||||
semThreadsHint: "少ないほど画像生成中のPC負荷が軽くなります(作成は遅くなります)",
|
||||
semSoft: "画像の精度:",
|
||||
semSoftHigh: "高精度(最も遅い)",
|
||||
semSoftStd: "標準(約2.5倍速)",
|
||||
semSoftLight: "軽量(約5倍速)",
|
||||
semSoftHint: "変更時は「全件再作成」が必要です",
|
||||
semTestOkProvider: (dim, prov) => `接続OK(ベクトル次元 ${dim}、実行: ${prov})`,
|
||||
semDotTitle: "セマンティック索引: このフォルダの画像は全件作成済み",
|
||||
semHelpTitle: "セマンティック検索のヘルプを開く",
|
||||
semModel: "埋め込みモデル:",
|
||||
semModelGemini: "Gemini Embedding 2(API)",
|
||||
semModelLocal: "EmbeddingGemma 2(ローカル・無料)",
|
||||
semLocalReady: "EmbeddingGemma 2: モデル取得済み(ローカル実行・APIキー不要)",
|
||||
semLocalNeedDownload: "EmbeddingGemma 2: 初回利用時にモデル(約300MB)をダウンロードします",
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2 エラー: ${m}`,
|
||||
semScope: "対象フォルダ:",
|
||||
semNoFolder: "先にGalleryでフォルダを選択してください",
|
||||
semIncludeSub: "サブフォルダを含める",
|
||||
semDimension: "ベクトル次元:",
|
||||
semDimHint: "大きいほど高精度ですが重くなります。モデルや次元の変更時は「全件再作成」が必要です。",
|
||||
semMode: "モード:",
|
||||
semModeMissing: "新規・変更された画像のみ",
|
||||
semModeAll: "全件再作成(対象フォルダ)",
|
||||
semStart: "インデックス作成開始",
|
||||
semCancelJob: "停止",
|
||||
semTest: "接続テスト",
|
||||
semClear: "インデックスを全削除",
|
||||
semCostNote: "Gemini Embedding 2には無料枠がありますが、無料枠で送信した内容はGoogleの製品改善に利用される場合があります。無料枠を超えると有料です(2026年10月現在、画像1枚あたり約$0.00012)。インデックス作成時に各画像(縮小)をAPIへ送信します。EmbeddingGemma 2はローカル(CPU)で実行します(1枚あたり約2〜3秒)。検索は保存済みベクトルを使います(送信されるのは検索文のみ)。動画/PSDはサムネイルを使用、音声は対象外です。",
|
||||
semKeyOk: "GEMINI_API_KEY: 設定済み",
|
||||
semKeyMissing: "GEMINI_API_KEY が未設定です — プラグインの .env に GEMINI_API_KEY=... を追記し、ComfyUIを再起動してください",
|
||||
semIndexInfo: (n, dim) => `インデックス済み: ${n}枚(ベクトル次元: ${dim})`,
|
||||
semScopeInfo: (done, total) => `対象フォルダ: ${done} / ${total} 枚作成済み`,
|
||||
semProgress: (done, total, failed) => `${done} / ${total} 完了、失敗 ${failed}`,
|
||||
semCancelled: "停止しました",
|
||||
semFinished: "完了",
|
||||
semConfirmAll: "対象フォルダの既存ベクトルを破棄して作り直します。よろしいですか?(API料金が発生します)",
|
||||
semConfirmClear: "セマンティック検索のインデックスを全て削除しますか?",
|
||||
semCleared: "インデックスを削除しました",
|
||||
semTestOk: (dim) => `接続OK(ベクトル次元 ${dim})`,
|
||||
sem_err_no_api_key: "GEMINI_API_KEY が未設定です(.env)",
|
||||
sem_err_api_key_invalid: (m) => `Gemini APIキーが拒否されました: ${m}`,
|
||||
sem_err_index_empty: "インデックスがありません — 先に⚙ボタン(SをONにすると出るツールバー2行目)から作成してください",
|
||||
sem_err_dim_mismatch: "既存インデックスとモデル・ベクトル次元・画像精度のいずれかが異なります — 「全件再作成」を選んでください",
|
||||
sem_err_already_running: "インデックス作成が既に実行中です",
|
||||
sem_err_folder_not_allowed: "このフォルダは許可されていません",
|
||||
sem_err_unsupported_type: "この種類のファイルは検索対象外です",
|
||||
sem_err_network_error: (m) => `ネットワークエラー: ${m}`,
|
||||
sem_err_api_error: (m) => `Gemini APIエラー: ${m}`,
|
||||
galleryTabOutput: "Output",
|
||||
galleryTabImagePrompt: "ImagePrompt",
|
||||
galleryTabStyleCatalog: "Style_Catalog",
|
||||
@@ -2639,6 +2781,10 @@ const LANGUAGES = {
|
||||
helpGallery11: "PNG (tEXt/iTXt)、JPEG、WebP、GIF、MP4、WebM、PSD、音声(WAV・MP3・OGG・M4A・FLAC・Opus:波形サムネイル・内蔵プレイヤー・SaveAudio系ノードが書き込むprompt/workflowタグ)対応;PNG形式はComfyUIの埋め込みJSON(prompt / workflow キー、IEND後ろに追記されたチャンクも対象)、WebP形式はEXIF内のworkflow/prompt、MP4/WebM形式はSaveVideoが書き込むコンテナタグ(同キー、MiniMax H3出力等)からワークフロー抽出が可能;ワークフローが無い画像はA1111 / Forge / Fooocusのparametersからプロンプトを表示。音声の波形サムネイルは形式ごとに色分けされます(MP3=青、WAV=緑、FLAC=琥珀、OGG/Opus=紫、M4A=ローズ)。詳細パネルは波形の下にプレイヤーを表示します",
|
||||
helpGallery31: "Send to LV nodeボタン(ツールバー、Send to LI nodeボタン横):動画版のSend to LI node — 選択動画をアップロードし、そのファイル名をComfyUIキャンバス上で選択中のノード(ネイティブのLoad Videoノードである必要がある)のfileウィジェットへ書き込む;選択中の項目が動画(mp4/webm)の時のみ有効;ComfyUIのトップメニューボタンからWorkflow Studioを起動している必要がある",
|
||||
helpGallery32: "詳細パネル — Promptタブ(既定で選択):埋め込みワークフローから見つかったpositive/negativeプロンプトを一覧表示;cc_nanobananaフォルダの画像はComfyUIワークフローを埋め込んでいないため非表示になり、代わりにNanobananaタブ(v0.7.14)が表示される — 同フォルダのhistory.jsonl生成履歴(ComfyUI Comic CreatorのNanobananaタブが書き込む)に選択画像の対応エントリがある場合のみ表示:engine・model・画像サイズ・seed・日時・実際に使用されたプロンプト/ネガティブプロンプト",
|
||||
helpGallery33: "Semantic Search(v0.8.2):検索ボックス横の「S」ボタンで意味検索をON/OFFします。ONにすると検索ボックスがGemini Embedding 2またはローカルのEmbeddingGemma 2による意味検索になります(例:「夕暮れの街並み」)。Enterで実行し、結果は類似度順でスコアバッジ付き。OFFの時は従来のファイル名/タグ/プロンプト検索です。ONにするとツールバーに2行目が現れ、Similar・更新ボタン・設定ボタン・結果件数が左詰めで表示されます。設定ボタン(歯車)のウィンドウで、選択中フォルダ(サブフォルダ含む指定可)のインデックスを作成/更新/削除します。モデルはラジオボタンで選択します(Geminiはプラグインの.envにGEMINI_API_KEYの設定+ComfyUI再起動が必要、EmbeddingGemma 2はローカル実行で初回にモデルを自動ダウンロード)。モデルやベクトル次元を変えるときは「全件再作成」が必要です。検索範囲は選択中フォルダ配下で、意味検索の結果にはソートやタグ/グループフィルタは適用されません",
|
||||
helpGallery34: "Similarトグルと更新ボタン(ツールバー2行目、SがONの間に表示):Similarで選択中の画像に類似した画像を一覧表示します(未インデックスの画像はその場で埋め込み)。類似一覧の中の別の画像を選択しても一覧はそのままで、更新ボタンを押すとその画像の類似画像を表示、Similarをもう一度押すと通常の一覧に戻ります。フォルダ切替・フィルタクリア・SをOFFにした場合も類似表示は解除されます",
|
||||
helpGallery35: "EmbeddingGemma 2の速度:既定ではCPU(コアの半分)で動作し、高精度設定で1枚あたり約6秒です。「画像の精度」は精度と速度の引き換えで(標準で約2.5倍、軽量で約5倍速。ベクトルが変わるため変更時は「全件再作成」が必要)、CPUスレッド数を減らすと画像生成中のPC負荷が軽くなります。NVIDIA GPUを使うにはComfyUIのPythonにonnxruntime-gpuを入れ(pip uninstall onnxruntime のあと pip install onnxruntime-gpu。ComfyUIのPyTorchが使うCUDAランタイム/cuDNNが利用されます)、ComfyUIを再起動してください:1枚あたり約0.1秒(RTX 4070で実測、CPUの約45倍速)で、結果はCPUと同一です。DirectMLは非対応です(画像モデルが動作しません)。モデルはインデックス作成の完了後にメモリから解放され、次回の検索時に再ロードされます",
|
||||
helpGallery36: "モーダルのタブとしきい値:セマンティック設定ウィンドウは3つのタブ構成です。インデックス(対象フォルダ・再作成モード・進捗・全削除)、モデル設定(Gemini / EmbeddingGemma 2・ベクトル次元・デバイス・CPUスレッド数・画像精度・接続テスト)、検索設定。検索設定タブでは、類似画像結果の最小スコア(例:0.6なら類似度0.6以上のみ表示)、文章検索の最小スコア、最大表示件数を指定できます。スコアの水準はモデルで違うため値はモデルごとに保存され、一覧の見出しに適用中のしきい値が表示されます",
|
||||
helpGallery28: "「PSD作成」一括操作ボタン:選択中の全画像を1つのレイヤー付きPSDファイルにまとめてダウンロードする — 各画像はファイル名を冠した独立レイヤーとなり、選択画像中の最大サイズに合わせたキャンバスに左上基準で配置される;最後に選択した画像が最前面になる 変換...一括ボタン(音声/動画を選択すると表示):音声ファイル(動画は音声トラックを抽出)をFLAC / MP3 / Opus / M4A / WAVへ変換して同じフォルダに別ファイルで保存します(ビットレート・サンプルレート・チャンネル指定可)。prompt/workflowタグはタグを持てる形式で引き継がれ(WAVは対象外)、お気に入り・タグ・メモ・グループの引き継ぎや、変換後の元ファイル削除も選べます。",
|
||||
helpGallery29: "「レイヤーとして編集」一括操作ボタン:Image Editタブに切り替え、選択中の全画像を一度に開き、それぞれ独立したレイヤーとして展開する(選択中に動画ファイルが含まれる場合はスキップされる);最後に選択した画像が最前面になる",
|
||||
helpGallery30: "PSDファイル:サムネイル・フルサイズプレビューは全レイヤーを合成した結果を表示する(非表示レイヤーは除外される);元のPSDファイル自体は変更されず、Gallery上のプレビューのみが合成済みの表示になる",
|
||||
@@ -3863,6 +4009,75 @@ const LANGUAGES = {
|
||||
gallerySentCC: "已发送至ComfyUI Comic Creator",
|
||||
galleryVideoNotSupportedInImageTab: "暂不支持将视频发送到Image标签",
|
||||
galleryNoEmbeddedWorkflow: "此图像中未嵌入ComfyUI工作流。",
|
||||
semSearchPlaceholder: "用文字描述(Enter)...",
|
||||
semModalTitle: "语义搜索索引",
|
||||
semCountText: (n, min) => `${n}个结果(语义搜索${min > 0 ? `,分数≥${min}` : ""})`,
|
||||
semCountSimilar: (n, name, min) => `与"${name}"相似的 ${n}张${min > 0 ? `(分数≥${min})` : ""}`,
|
||||
semTabIndex: "索引",
|
||||
semTabModel: "模型设置",
|
||||
semTabSearch: "搜索设置",
|
||||
semThTarget: "适用对象:",
|
||||
semThNoIndex: "(尚无索引,针对所选模型)",
|
||||
semThSimilar: "相似图像 - 最低分数:",
|
||||
semThSimilarHint: "例如 0.6 仅显示相似度 0.6 以上的图像。0=不过滤",
|
||||
semThText: "文字搜索 - 最低分数:",
|
||||
semThTextHint: "0=不过滤",
|
||||
semThLimit: "最大显示数量:",
|
||||
semThNote: "不同模型的分数水平不同(少量图像的实测:相似图像在EmbeddingGemma 2约0.6-0.8,Gemini约0.6;文字搜索分别约0.6-0.7和约0.3),因此按模型分别保存。关闭窗口时会应用到正在显示的语义搜索/相似列表。",
|
||||
semDevice: "运行设备:",
|
||||
semDeviceAuto: "自动(可用则用GPU)",
|
||||
semDeviceCudaOk: (v) => `检测到GPU (CUDA) - onnxruntime ${v}`,
|
||||
semDeviceNoGpu: (v) => `未检测到GPU(onnxruntime ${v},仅CPU)。使用GPU请参见“?”帮助`,
|
||||
semThreads: "CPU线程数:",
|
||||
semThreadsAll: "全部核心",
|
||||
semThreadsHint: "线程越少,生成图像时对电脑的负载越轻(创建会变慢)",
|
||||
semSoft: "图像精度:",
|
||||
semSoftHigh: "高精度(最慢)",
|
||||
semSoftStd: "标准(约快2.5倍)",
|
||||
semSoftLight: "轻量(约快5倍)",
|
||||
semSoftHint: "更改后需要“全部重建”",
|
||||
semTestOkProvider: (dim, prov) => `连接正常(向量维度 ${dim},运行于 ${prov})`,
|
||||
semDotTitle: "语义索引:此文件夹中的图像已全部创建",
|
||||
semHelpTitle: "打开语义搜索的帮助",
|
||||
semModel: "嵌入模型:",
|
||||
semModelGemini: "Gemini Embedding 2(API)",
|
||||
semModelLocal: "EmbeddingGemma 2(本地・免费)",
|
||||
semLocalReady: "EmbeddingGemma 2:模型已下载(本地运行,无需API密钥)",
|
||||
semLocalNeedDownload: "EmbeddingGemma 2:首次使用时将下载模型(约300MB)",
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2错误:${m}`,
|
||||
semScope: "目标文件夹:",
|
||||
semNoFolder: "请先在Gallery中选择文件夹",
|
||||
semIncludeSub: "包含子文件夹",
|
||||
semDimension: "向量维度:",
|
||||
semDimHint: "越大越精确但越重。更改模型或维度后需要“全部重建”。",
|
||||
semMode: "模式:",
|
||||
semModeMissing: "仅新增/已更改的图像",
|
||||
semModeAll: "全部重建(目标文件夹)",
|
||||
semStart: "开始创建索引",
|
||||
semCancelJob: "停止",
|
||||
semTest: "连接测试",
|
||||
semClear: "删除全部索引",
|
||||
semCostNote: "Gemini Embedding 2有免费额度,但免费额度内发送的内容可能被Google用于改进其产品;超出免费额度后需付费(截至2026年10月,每张图像约$0.00012)。创建索引时会将每张图像(缩小后)发送到API;EmbeddingGemma 2在本地CPU运行(每张约2-3秒)。搜索使用已保存的向量(仅发送搜索文字)。视频/PSD使用缩略图,音频不在范围内。",
|
||||
semKeyOk: "GEMINI_API_KEY:已设置",
|
||||
semKeyMissing: "未设置GEMINI_API_KEY — 请在插件的 .env 中添加 GEMINI_API_KEY=... 并重启ComfyUI",
|
||||
semIndexInfo: (n, dim) => `已索引:${n}张(向量维度:${dim})`,
|
||||
semScopeInfo: (done, total) => `目标文件夹:已创建 ${done} / ${total}`,
|
||||
semProgress: (done, total, failed) => `完成 ${done} / ${total},失败 ${failed}`,
|
||||
semCancelled: "已停止",
|
||||
semFinished: "已完成",
|
||||
semConfirmAll: "将丢弃目标文件夹的现有向量并重建。确定吗?(会产生API费用)",
|
||||
semConfirmClear: "确定删除全部语义搜索索引吗?",
|
||||
semCleared: "索引已删除",
|
||||
semTestOk: (dim) => `连接正常(向量维度 ${dim})`,
|
||||
sem_err_no_api_key: "未设置GEMINI_API_KEY (.env)",
|
||||
sem_err_api_key_invalid: (m) => `Gemini API密钥被拒绝:${m}`,
|
||||
sem_err_index_empty: "尚无索引 — 请先通过⚙按钮(开启S后出现的第二行)创建",
|
||||
sem_err_dim_mismatch: "模型、向量维度或图像精度与现有索引不同 — 请选择“全部重建”",
|
||||
sem_err_already_running: "索引创建已在运行",
|
||||
sem_err_folder_not_allowed: "不允许该文件夹",
|
||||
sem_err_unsupported_type: "不支持搜索此类文件",
|
||||
sem_err_network_error: (m) => `网络错误:${m}`,
|
||||
sem_err_api_error: (m) => `Gemini API错误:${m}`,
|
||||
galleryTabOutput: "Output",
|
||||
galleryTabImagePrompt: "ImagePrompt",
|
||||
galleryTabStyleCatalog: "Style_Catalog",
|
||||
@@ -4268,6 +4483,10 @@ const LANGUAGES = {
|
||||
helpGallery11: "支持PNG(tEXt/iTXt)、JPEG、WebP、GIF、MP4、WebM、PSD以及音频(WAV、MP3、OGG、M4A、FLAC、Opus:波形缩略图、内置播放器、SaveAudio系节点写入的prompt/workflow标签);PNG格式读取ComfyUI嵌入的JSON(prompt / workflow键,包括IEND之后追加的块),WebP格式读取EXIF中的workflow/prompt,MP4/WebM格式读取SaveVideo写入的容器标签(同名键,如MiniMax H3输出);无工作流的图像会从A1111 / Forge / Fooocus的parameters显示提示词。音频波形缩略图按格式着色(MP3蓝、WAV绿、FLAC琥珀、OGG/Opus紫、M4A玫红),详情面板在波形下方显示播放器",
|
||||
helpGallery31: "Send to LV node按钮(工具栏,Send to LI node按钮旁):视频版的Send to LI node — 上传选中的视频并将其文件名写入ComfyUI画布上当前选中节点(必须是原生的Load Video节点)的file部件;仅当选中项为视频(mp4/webm)时可用;需要通过ComfyUI顶部菜单按钮打开Workflow Studio",
|
||||
helpGallery32: "详情面板 — Prompt标签(默认选中):列出从嵌入工作流中找到的正/负向提示词;cc_nanobanana文件夹中的图像未嵌入ComfyUI工作流,因此该标签会被隐藏,取而代之显示Nanobanana标签(v0.7.14)— 仅当该文件夹的history.jsonl生成记录(由ComfyUI Comic Creator的Nanobanana标签写入)中有选中图像的对应记录时才显示:engine、model、图像尺寸、seed、时间戳以及实际使用的提示词/负向提示词",
|
||||
helpGallery33: "Semantic Search(v0.8.2):搜索框旁的“S”按钮用于开启/关闭语义搜索。开启后,搜索框使用Gemini Embedding 2或本地EmbeddingGemma 2按含义查找图像(例如“黄昏的城市街道”),按Enter执行,结果按相似度排序并显示分数标记。关闭时为原有的文件名/标签/提示词搜索。开启后工具栏会出现第二行,左对齐显示Similar、更新按钮、设置按钮和结果数量。设置按钮(齿轮)会打开窗口,用于创建/更新/删除所选文件夹(可含子文件夹)的索引;模型通过单选按钮选择(Gemini需在插件的.env中设置GEMINI_API_KEY并重启ComfyUI;EmbeddingGemma 2在本地运行,首次使用时自动下载模型)。更改模型或向量维度时需要“全部重建”。搜索范围为所选文件夹及其下级,语义搜索结果不应用排序和标签/分组筛选",
|
||||
helpGallery34: "Similar切换按钮和更新按钮(工具栏第二行,S开启时显示):Similar显示与所选图像相似的图像(未索引的图像会即时嵌入)。在相似列表中选择其他图像时列表保持不变,按更新按钮显示与新选图像相似的图像,再次按Similar则返回普通列表。切换文件夹、清除筛选或关闭S也会退出相似模式",
|
||||
helpGallery35: "EmbeddingGemma 2的速度:默认在CPU(一半核心)上运行,高精度设置下每张约6秒。“图像精度”是在精度与速度之间取舍(标准约快2.5倍,轻量约快5倍;向量会改变,因此更改后需要“全部重建”),减少CPU线程数可降低生成图像时对电脑的负载。要使用NVIDIA GPU,请在ComfyUI的Python中安装onnxruntime-gpu(先pip uninstall onnxruntime,再pip install onnxruntime-gpu;会使用ComfyUI的PyTorch所用的CUDA运行库/cuDNN)并重启ComfyUI:每张约0.1秒(在RTX 4070上实测,约为CPU的45倍),结果与CPU相同。不支持DirectML(其图像模型无法运行)。索引创建完成后模型会从内存中释放,下次搜索时重新加载",
|
||||
helpGallery36: "弹窗标签页与相似度阈值:语义设置窗口分为三个标签页——索引(目标文件夹、重建模式、进度、全部删除)、模型设置(Gemini / EmbeddingGemma 2、向量维度、设备、CPU线程数、图像精度、连接测试)和搜索设置。在搜索设置中可指定相似图像结果的最低分数(例如0.6仅显示相似度0.6以上的图像)、文字搜索的最低分数以及最大显示数量。不同模型的分数水平不同,因此每个模型分别保存,列表标题会显示当前应用的阈值",
|
||||
helpGallery28: "「创建PSD」批量操作按钮:将所有选中的图像合并为一个带图层的PSD文件并下载 — 每张图像成为独立图层(以文件名命名),以左上对齐方式放置在按所选最大图像尺寸创建的画布上;最后选中的图像位于最上层 转换...批量按钮(选中音频/视频时显示):将音频文件(视频则提取音轨)转换为FLAC / MP3 / Opus / M4A / WAV并保存为同文件夹内的新文件(可选比特率、采样率、声道);prompt/workflow标签会在支持标签的格式中继承(WAV除外),也可继承收藏/标签/备注/分组,并可选择转换后删除原文件。",
|
||||
helpGallery29: "「作为图层编辑」批量操作按钮:切换到Image Edit标签并一次性打开所有选中的图像,每张作为独立图层(所选内容中的视频文件会被跳过);最后选中的图像位于最上层",
|
||||
helpGallery30: "PSD文件:缩略图与完整预览显示所有图层合并后的效果(隐藏图层不参与合并);原始PSD文件本身不会被修改,Gallery中显示的仅是合并渲染后的预览",
|
||||
|
||||
@@ -275,17 +275,44 @@ function isPromptStylerNode(ct) { return ct.includes("PromptStyler"); }
|
||||
const SAMPLER_SETTING_KEYS = ["seed", "noise_seed", "steps", "cfg", "sampler_name", "scheduler", "denoise"];
|
||||
function extractSamplerSettings(apiWf) {
|
||||
if (!apiWf || typeof apiWf !== "object" || Array.isArray(apiWf.nodes)) return null;
|
||||
for (const n of Object.values(apiWf)) {
|
||||
if (!n || typeof n !== "object" || !isSamplerNode(n.class_type ?? "")) continue;
|
||||
const pick = (inputs, keys) => {
|
||||
const out = {};
|
||||
for (const key of keys) {
|
||||
const v = inputs?.[key];
|
||||
if (v !== undefined && !Array.isArray(v)) out[key] = v;
|
||||
}
|
||||
return out;
|
||||
};
|
||||
// Flux/SD3系はRandomNoise+BasicScheduler+KSamplerSelect+CFGGuider等に設定が分散するため、
|
||||
// 補助ノードから不足分を補う。
|
||||
const AUX = {
|
||||
RandomNoise: ["noise_seed"],
|
||||
BasicScheduler: ["steps", "scheduler", "denoise"],
|
||||
KSamplerSelect: ["sampler_name"],
|
||||
CFGGuider: ["cfg"],
|
||||
DualCFGGuider: ["cfg"],
|
||||
};
|
||||
// 設定キーを最も多く持つサンプラーを主とし、同数なら後ろのノード(hires-fix等の最終パス)を優先する
|
||||
let main = null, mainCount = 0;
|
||||
const aux = {};
|
||||
for (const [id, n] of Object.entries(apiWf)) {
|
||||
if (!n || typeof n !== "object") continue;
|
||||
const ct = n.class_type ?? "";
|
||||
const inputs = n.inputs ?? {};
|
||||
const settings = {};
|
||||
for (const key of SAMPLER_SETTING_KEYS) {
|
||||
const v = inputs[key];
|
||||
if (v !== undefined && !Array.isArray(v)) settings[key] = v;
|
||||
if (AUX[ct]) {
|
||||
const got = pick(inputs, AUX[ct]);
|
||||
for (const [k, v] of Object.entries(got)) if (!(k in aux)) aux[k] = v;
|
||||
continue;
|
||||
}
|
||||
if (!isSamplerNode(ct)) continue;
|
||||
const settings = pick(inputs, SAMPLER_SETTING_KEYS);
|
||||
const count = Object.keys(settings).length;
|
||||
if (count > 0 && count >= mainCount) {
|
||||
main = settings; mainCount = count;
|
||||
}
|
||||
if (Object.keys(settings).length > 0) return settings;
|
||||
}
|
||||
return null;
|
||||
const merged = { ...aux, ...(main ?? {}) };
|
||||
return Object.keys(merged).length > 0 ? merged : null;
|
||||
}
|
||||
|
||||
// CLIPTextEncodeEditPlus (model-and-prompt-from-metadata) の encode() と同じ結合ルール。
|
||||
@@ -754,7 +781,7 @@ export async function extractAllMetadata(file) {
|
||||
// 正しく解決するため、そちらを経由してから抽出する。通常のワークフローには影響しない。
|
||||
if (Array.isArray(wf?.nodes)) {
|
||||
try {
|
||||
const converted = await comfyWorkflow.convertUiToApi(wf);
|
||||
const converted = await comfyWorkflow.convertUiToApi(wf, { readOnly: true });
|
||||
if (converted && Object.keys(converted).length > 0) {
|
||||
apiWf = converted;
|
||||
if (wf.definitions?.subgraphs?.length > 0) wf = converted;
|
||||
@@ -1039,8 +1066,11 @@ export function initMetadataTab() {
|
||||
}
|
||||
}
|
||||
|
||||
// 連続で画像を開いたとき、先に始めた処理が後から描画して結果が混ざるのを防ぐ
|
||||
let _loadSeq = 0;
|
||||
async function handleFile(file, path) {
|
||||
if (!file) return;
|
||||
const seq = ++_loadSeq;
|
||||
if (file.size > MAX_FILE_SIZE) {
|
||||
fileInfo.textContent = t("metaFileTooLarge");
|
||||
fileInfo.style.color = "var(--wfm-warning)";
|
||||
@@ -1052,6 +1082,7 @@ export function initMetadataTab() {
|
||||
clearAll();
|
||||
|
||||
const serverMeta = await fetchServerMeta(path);
|
||||
if (seq !== _loadSeq) return;
|
||||
|
||||
// Show preview for images
|
||||
const isImage = file.type.startsWith("image/") || file.name.toLowerCase().match(/\.(png|webp|jpg|jpeg)$/);
|
||||
@@ -1074,6 +1105,7 @@ export function initMetadataTab() {
|
||||
dropLabel.style.display = "flex";
|
||||
}
|
||||
|
||||
if (seq !== _loadSeq) return;
|
||||
const dims = (serverMeta?.width && serverMeta?.height) ? serverMeta : naturalDims;
|
||||
renderNanobanana(serverMeta?.nanobanana ?? null);
|
||||
|
||||
@@ -1081,6 +1113,7 @@ export function initMetadataTab() {
|
||||
try {
|
||||
meta = await extractAllMetadata(file);
|
||||
} catch (err) {
|
||||
if (seq !== _loadSeq) return;
|
||||
console.error("[MetadataTab]", err);
|
||||
fileInfo.textContent = t("metaParseError");
|
||||
fileInfo.style.color = "var(--wfm-danger)";
|
||||
@@ -1088,6 +1121,7 @@ export function initMetadataTab() {
|
||||
return;
|
||||
}
|
||||
|
||||
if (seq !== _loadSeq) return;
|
||||
if (!meta) {
|
||||
// nanobanana(Gemini API経由)の画像はComfyUIワークフローを埋め込んでいないため
|
||||
// extractAllMetadataはnullを返す。その場合でもnanobanana履歴があれば
|
||||
|
||||
@@ -322,6 +322,7 @@ function renderNodeGrid() {
|
||||
if (countEl) countEl.textContent = `${filtered.length} / ${state.allNodes.length}`;
|
||||
|
||||
if (filtered.length === 0) {
|
||||
state.renderToken++; // 実行中の分割描画を止める(止めないと古いカードが追加され続ける)
|
||||
grid.innerHTML = `<p class="wfm-placeholder">${t("noNodesFound")}</p>`;
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1809,6 +1809,10 @@
|
||||
<li id="wfm-help-gallery-31"><b>Send to LV node</b> button (toolbar, next to Send to LI node): the video equivalent — uploads the selected video and writes its filename into the file widget of the currently selected node, which must be ComfyUI's native <b>Load Video</b> node; only enabled when the selected item is a video (mp4/webm); requires Workflow Studio to have been opened from ComfyUI's top menu button</li>
|
||||
<li id="wfm-help-gallery-28"><b>Create PSD</b> bulk button: combines all selected images into a single layered PSD file and downloads it — each image becomes its own layer (named after the file), positioned top-left on a canvas sized to the largest selected image; the last selected image ends up on top Convert... bulk button (shown when audio/video files are selected): converts audio files — or extracts the audio track of videos — to FLAC / MP3 / Opus / M4A / WAV (bitrate, sample rate, channels selectable) into the same folder as new files; prompt/workflow tags are carried over for formats that can hold them (WAV cannot), favorite/tags/memo/groups can be carried over, and the originals can optionally be deleted afterwards.</li>
|
||||
<li id="wfm-help-gallery-29"><b>Edit as Layers</b> bulk button: switches to the Image Edit tab and opens all selected images at once, each as an independent layer (video files in the selection are skipped); the last selected image ends up on top</li>
|
||||
<li id="wfm-help-gallery-33">Semantic Search (v0.8.2): the S button next to the search box turns semantic search on. When ON, the search box finds images by meaning using Gemini Embedding 2 or the local EmbeddingGemma 2 (e.g. 'sunset city street') - press Enter to search; results are ordered by similarity with a score badge. When OFF it is the normal filename/tag/prompt search. Turning S on also shows a second toolbar row with Similar, the update button, the settings button and the result count. The settings button (gear) opens a window to build/update/delete the index for the selected folder (optionally with subfolders); choose the model with the radio buttons (Gemini needs GEMINI_API_KEY in the plugin's .env + a ComfyUI restart; EmbeddingGemma 2 runs locally and downloads its model on first use). Switching model or vector size needs 'Rebuild all'. Searches cover the selected folder tree; sort and tag/group filters do not apply to semantic results</li>
|
||||
<li id="wfm-help-gallery-34">Similar toggle and update button (second toolbar row, shown while S is ON): Similar shows images similar to the selected one (an image not yet indexed is embedded on the fly). Selecting another image keeps the list as is; press the update button to show images similar to the newly selected one, or press Similar again to return to the normal list. Changing folder, Clear filters or turning S off also leaves similar mode</li>
|
||||
<li id="wfm-help-gallery-35">EmbeddingGemma 2 performance: by default it runs on the CPU with half of the cores (about 6 s per image at full precision). Image precision trades accuracy for speed (Standard is about 2.5x and Light about 5x faster; vectors differ, so changing it needs 'Rebuild all'); fewer CPU threads keep the PC lighter while generating images. To use an NVIDIA GPU, install onnxruntime-gpu into ComfyUI's Python (pip uninstall onnxruntime, then pip install onnxruntime-gpu; the CUDA runtime/cuDNN that ComfyUI's PyTorch uses is picked up) and restart ComfyUI: indexing then takes about 0.1 s per image (measured on an RTX 4070, about 45x faster than the CPU) and the results are identical to the CPU's. DirectML is not supported (its vision model fails to run). The model is released from memory when indexing finishes and reloaded on the next search</li>
|
||||
<li id="wfm-help-gallery-36">Modal tabs and the similarity threshold: the Semantic settings window has three tabs - Index (target folder, rebuild mode, progress, delete all), Model (Gemini / EmbeddingGemma 2, vector size, device, CPU threads, image precision, connection test) and Search. On the Search tab you can set a minimum score for similar-image results (e.g. 0.6 shows only images scoring 0.6 or higher), a minimum score for text search, and the maximum number of results. Scores differ by model, so each model keeps its own values; the list header shows the active threshold</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="wfm-help-card">
|
||||
@@ -2265,6 +2269,7 @@
|
||||
<input type="text" id="wfm-gallery-search" class="wfm-search-input" placeholder="Search images...">
|
||||
<button class="wfm-search-clear-btn" id="wfm-gallery-search-clear-btn" title="Clear search">✕</button>
|
||||
</div>
|
||||
<button id="wfm-gallery-semantic-toggle" class="wfm-btn wfm-btn-sm wfm-gallery-sem-toggle" title="Semantic Search: when ON, the search box finds images by meaning (press Enter to search) and a second row with Similar / settings appears.">S</button>
|
||||
<select id="wfm-gallery-tag-filter" class="wfm-select wfm-toolbar-select">
|
||||
<option value="">All Tags</option>
|
||||
</select>
|
||||
@@ -2291,6 +2296,13 @@
|
||||
<button id="wfm-gallery-send-lv-node-btn" class="wfm-btn wfm-btn-sm" title="Upload and write selected video's filename into the selected node's file widget (native Load Video node)">Send to LV node</button>
|
||||
<button id="wfm-gallery-send-cc-btn" class="wfm-btn wfm-btn-sm" title="Send selected image to ComfyUI Comic Creator's selected panel" style="display:none;">Send CC</button>
|
||||
</div>
|
||||
<!-- Semantic Search ON時の2行目: 類似画像 / 更新 / 設定 / 結果件数 -->
|
||||
<div id="wfm-gallery-sem-row" class="wfm-toolbar-row wfm-gallery-sem-row" style="display:none;">
|
||||
<button id="wfm-gallery-similar-btn" class="wfm-btn wfm-btn-sm wfm-sem-sim" title="Show images similar to the selected one (toggle)">Similar</button>
|
||||
<button id="wfm-gallery-similar-refresh-btn" class="wfm-btn wfm-btn-sm wfm-sem-sim" title="Update: show images similar to the currently selected image">↻</button>
|
||||
<button id="wfm-gallery-semantic-settings-btn" class="wfm-btn wfm-btn-sm" title="Semantic Search settings (index / model / search)">⚙</button>
|
||||
<span id="wfm-gallery-sem-count" class="wfm-gallery-sem-count"></span>
|
||||
</div>
|
||||
<!-- 複数選択時の一括操作バー -->
|
||||
<div id="wfm-gallery-bulk-bar" class="wfm-gallery-bulk-bar" style="display:none;">
|
||||
<span id="wfm-gallery-bulk-count" class="wfm-gallery-bulk-count"></span>
|
||||
|
||||
Reference in New Issue
Block a user