feat: run Gallery semantic search's EmbeddingGemma 2 in Ollama instead of ONNX
- new semantic_ollama.py replaces semantic_local.py (ONNX runtime removed)
- images sent as input=[{"image": b64}] (the "images" field is silently ignored),
4 per request, per-image fallback on batch failure, retries on dropped connections
- model size selectable: 740m / 440m (identical vectors, no rebuild); :latest used as 740m
- npz model name unchanged, existing indexes keep working (cos ~0.97 vs ONNX vectors)
- Ollama URL checked by url_guard; settings saved via /wfm/gallery/semantic/options
- model is unloaded from VRAM after indexing (keep_alive 0)
- settings UI: device/threads/image precision replaced by Ollama URL + model size
- help (helpGallery33/35/36, helpTrouble10/11 en/ja/zh), README, requirements, DEVLOG updated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ketle-man
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
f43ad515bf
commit
ea66f2565d
@@ -2,6 +2,7 @@
|
||||
|
||||
Condensed version-by-version index of DEVLOG.md (each line: version + one-line summary of what changed and why). Full rationale, code details, and "How to apply" lessons live in the excluded DEVLOG.md itself; this index exists so graphify can extract a queryable semantic node per release without feeding the full log to the local Ollama model. Regenerate with `python tools/generate_doc_index.py devlog` whenever DEVLOG.md changes.
|
||||
|
||||
- **未リリース（2026-11-02）** — Gallery セマンティック検索: EmbeddingGemma 2 を ONNX から Ollama へ移行
|
||||
- **v0.8.13（2026-10-27）** — Edit in Mask Editor One: 送信画像をノードへ永続化（Mask Editor One v0.1.12 対応）
|
||||
- **v0.8.12（2026-10-27）** — Comic Creator向け複数画像I2Iブリッジ + GenerateUI Inputタブ: Prompt/Image統合・Raw JSON折りたたみ
|
||||
- **v0.8.11（2026-10-26）** — Gallery: Send CC をレイアウト／Imageタブの2ボタンに分割
|
||||
|
||||
@@ -2,6 +2,20 @@
|
||||
|
||||
---
|
||||
|
||||
## 未リリース（2026-11-02）
|
||||
|
||||
### Gallery セマンティック検索: EmbeddingGemma 2 を ONNX から Ollama へ移行
|
||||
|
||||
- **背景**: latenttag-3d（Ollama の `embeddinggemma-2` でタグを埋め込む）と Gallery 索引を共通の空間で使いたい。ONNX 版（`semantic_local.py`、q4）は onnxruntime-gpu の入れ替え問題も抱えていた。
|
||||
- **互換性の実測**: 同じ入力で ONNX 版と Ollama 版（Q8_0）を比較。テキスト cos 0.985〜0.993、画像（12枚）平均 0.967（最小 0.923）。既存索引に Ollama の検索文を当てた上位10件の一致は 8〜9/10。→ npz の `model` 名は `embeddinggemma-2` のまま、既存索引（2.9万枚）をそのまま使う。
|
||||
- **740m / 440m**: 740m=テキスト+画像+音声、440m=テキスト+画像。画像・テキストとも cos 1.0（同じベクトル）なので、切り替えても再作成不要。`:latest` は 740m と同じ（parent_model 740m-q8_0）で、740m 未取得時の代わりに使う。270m / 570m は画像非対応のため選択肢に入れない。
|
||||
- **Ollama の画像埋め込みの罠**: `/api/embed` に `images` フィールドを付けても**エラーにならず無視され、テキストだけの埋め込みが返る**。正しくは `input: [{"image": "<base64>"}]`。`[{"images": ...}]` は 400、壊れた画像は 500 "Failed to load image or audio file"、未取得モデルは 404。`input: []` + `keep_alive: 0` でアンロードできる。
|
||||
- **実装**: `semantic_ollama.py` を新設し `semantic_local.py` を削除。画像は4枚ずつ送信（1枚ずつより約1.8倍速い）し、バッチが失敗したら1枚ずつ送り直して原因の画像だけを失敗扱いにする。Ollama 内部の接続切断（400 "forcibly closed" 等）と 5xx は最大5回再試行。接続不可・モデル未取得はジョブ開始前に検出し、作成中に起きたら中止。作成後は VRAM から外す。URL は `url_guard` で検査。設定は `semantic_settings.json`（`url` / `tag`）、保存用に `/wfm/gallery/semantic/options` を追加。
|
||||
- **UI**: モデル設定タブのデバイス/CPUスレッド/画像精度を廃止し、Ollama URL・モデルサイズ（未取得表示つき）に置き換え。ヘルプ（helpGallery33/35/36・helpTrouble10/11 英/日/中）、README、requirements.txt を更新。マニュアル（v0.8.9版）は未更新。
|
||||
- **残作業**: ComfyUI 再起動後の実機確認（検索・類似・インデックス作成）が未実施。旧 ONNX モデル（`user/default/Workflow-Studio/models/embeddinggemma-2`、302MB）は削除可能。`onnxruntime` は Tagger で使うため依存に残す。
|
||||
|
||||
---
|
||||
|
||||
## v0.8.13（2026-10-27）
|
||||
|
||||
### Edit in Mask Editor One: 送信画像をノードへ永続化（Mask Editor One v0.1.12 対応）
|
||||
|
||||
+1
-1
@@ -170,7 +170,7 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** (no key; `onnxruntime`, `tokenizers` and `huggingface_hub` — normally already present with ComfyUI; the model downloads on first use). `onnxruntime-gpu` makes the local model ≈45× faster on NVIDIA GPUs
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** in [Ollama](https://ollama.com) (no key; `ollama pull embeddinggemma-2:740m` or `embeddinggemma-2:440m`)
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
|
||||
@@ -277,9 +277,9 @@ An experimental batch generator: runs the workflow currently loaded in GenerateU
|
||||
- **File conversion** (v0.8.0) — a **Convert...** button in the Gallery bulk bar (shown when the selection contains audio or video) converts audio files to FLAC / MP3 / Opus / M4A / WAV, or extracts the audio track of videos, using PyAV (no system ffmpeg needed); bitrate, sample rate (44.1 / 48 kHz) and channels (mono / stereo) are selectable, output goes next to the source as a new file (`name.ext`, or `name_converted.ext` if taken). ComfyUI `prompt` / `workflow` tags (and title / artist / album tags) are carried over to formats that can hold them — WAV cannot, so they are skipped there; favorite / tags / memo / groups can be carried over, and the original can optionally be deleted after a successful conversion (with a confirmation). `.ogg` is readable but not offered as an output (the bundled PyAV has no Vorbis encoder)
|
||||
- **Move To folder filter** (v0.8.0) — the Move To dialog (detail panel and bulk bar) has a filter box that narrows the folder list by name or path (space = AND), Enter moves to the first match; the move toast now reads "Moved N file(s)" since audio/video files are moved too
|
||||
- **Semantic Search** (v0.8.2) — find images by meaning instead of keywords. The **S** button next to the search box turns it on: the search box then takes a free-text description (e.g. "sunset city street"), Enter searches, and results are ordered by similarity with a score badge on each thumbnail. Turning **S** on also shows a second toolbar row (left-aligned): **Similar** (lists images similar to the selected one; selecting another image keeps the list, **↻** re-runs it for the newly selected image, pressing Similar again returns to the normal list), the **⚙** settings button, and the result count. Searches cover the selected folder and its subfolders; sort and tag/group filters do not apply to semantic results
|
||||
- **Index window (⚙)** — three tabs. **Index**: target folder (optionally with subfolders), "new/changed only" or "rebuild all", progress bar, stop, delete all. **Model**: choose with radio buttons between **Gemini Embedding 2** (API; set `GEMINI_API_KEY` in the plugin's `.env` and restart ComfyUI — the free tier exists but content sent on it may be used by Google to improve its products, beyond it the API is paid) and **EmbeddingGemma 2** (runs locally, free, no key; the ~300 MB q4 ONNX model is downloaded on first use), plus vector size (Gemini 768 / 1536 / 3072, EmbeddingGemma 2 128 / 256 / 512 / 768), and for the local model the device (auto / CPU / GPU CUDA), CPU thread count (default: half the cores) and image precision (high / standard ≈2.5× faster / light ≈5× faster). **Search**: minimum score for similar-image results (e.g. `0.6` shows only images scoring 0.6 or higher), minimum score for text search, and maximum results — stored per model because score levels differ between models
|
||||
- **Index window (⚙)** — three tabs. **Index**: target folder (optionally with subfolders), "new/changed only" or "rebuild all", progress bar, stop, delete all. **Model**: choose with radio buttons between **Gemini Embedding 2** (API; set `GEMINI_API_KEY` in the plugin's `.env` and restart ComfyUI — the free tier exists but content sent on it may be used by Google to improve its products, beyond it the API is paid) and **EmbeddingGemma 2** (runs locally in Ollama, free, no key), plus vector size (Gemini 768 / 1536 / 3072, EmbeddingGemma 2 128 / 256 / 512 / 768), and for EmbeddingGemma 2 the Ollama URL and the model size (740m or 440m). **Search**: minimum score for similar-image results (e.g. `0.6` shows only images scoring 0.6 or higher), minimum score for text search, and maximum results — stored per model because score levels differ between models
|
||||
- **Index state in the folder tree** — a green dot next to a folder's image count means every image directly in that folder is indexed; the index is one shared store, so folders can be indexed one at a time. Images, plus videos / PSD via their thumbnails (audio is skipped), are embedded; changing the model, vector size or image precision requires "Rebuild all"
|
||||
- **Local GPU** — EmbeddingGemma 2 is a port of Transformers.js' pipeline onto `onnxruntime` (no `transformers` support needed). On an RTX 4070 it indexes ≈0.13 s per image with CUDA versus ≈6 s on the CPU (≈45× faster, identical vectors); to use the GPU, install `onnxruntime-gpu` in place of `onnxruntime` (`pip uninstall onnxruntime onnxruntime-gpu` then `pip install onnxruntime-gpu`). DirectML is not supported (its vision model fails). The model is released from memory when indexing finishes
|
||||
- **EmbeddingGemma 2 in Ollama** — download the model with `ollama pull embeddinggemma-2:740m` (text + image + audio) or `ollama pull embeddinggemma-2:440m` (text + image, smaller). Both return identical vectors, so you can switch sizes without rebuilding (`:latest` is the same as 740m and is used when 740m is missing; 270m / 570m cannot embed images). Images are sent 4 per request (≈0.1–0.2 s per image on an NVIDIA GPU) and retried automatically when Ollama drops a connection. The model is unloaded from VRAM when indexing finishes. Indexes built with the earlier built-in ONNX runtime keep working (cos ≈0.97 with the Ollama vectors); use "rebuild all" to regenerate them exactly. Other Ollama hosts must be listed in `WFS_ALLOWED_BACKEND_HOSTS`
|
||||
- **Audio waveform colors** (v0.8.0) — waveform thumbnails are colored per format (MP3 blue, WAV green, FLAC amber, OGG/Opus purple, M4A rose)
|
||||
- **Info tab: dimensions & duration** (v0.5.1) — the detail panel's Info tab now also shows image dimensions (width×height) or video duration, in addition to filename, size, date, tags, and memo
|
||||
- **Multi-select** — Ctrl+click to select multiple images; Bulk Bar appears for batch operations: select group → "Add to Group" / "Remove from Group"; Favorite All / Unfavorite All; Compare (2–4 images); Move To...; **Export** (downloads all selected images as a ZIP file); **Create PSD** (v0.6.0, combines the selected images into one layered PSD file, positioned top-left on a canvas sized to the largest image); **Edit as Layers** (v0.6.0, opens all selected images in the Image Edit tab at once, each as its own layer); Delete File
|
||||
@@ -624,7 +624,7 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
|
||||
- **[LM Studio](https://lmstudio.ai/)** — alternative backend for translation and VLM (OpenAI-compatible API)
|
||||
- **[Lemonade Server](https://lemonade-server.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API)
|
||||
- **[Unsloth](https://unsloth.ai/)** — alternative backend for translation, chat, and VLM (OpenAI-compatible API); needs an API key set via `UNSLOTH_API_KEY` in a `.env` file, or Unsloth's Keyless API access → "Chat and inference" turned on — `pip install -r requirements.txt` for `python-dotenv`
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** (no key; `onnxruntime`, `tokenizers` and `huggingface_hub` — normally already present with ComfyUI; the model downloads on first use). `onnxruntime-gpu` makes the local model ≈45× faster on NVIDIA GPUs
|
||||
- **Gemini API key / EmbeddingGemma 2** (v0.8.2) — only for the Gallery's Semantic Search: either set `GEMINI_API_KEY` in the plugin's `.env` (key from [Google AI Studio](https://aistudio.google.com); free tier available, paid beyond it), or use the local **EmbeddingGemma 2** in [Ollama](https://ollama.com) (no key; `ollama pull embeddinggemma-2:740m` or `embeddinggemma-2:440m`)
|
||||
- **[Eagle](https://eagle.cool/)** — for auto-saving generated images with metadata
|
||||
- **[comfyui-mask-editor-one](https://github.com/ketle-man/comfyui-mask-editor-one) (v0.1.9+)** — enables BiRefNet background removal, SAM3 text-prompt segmentation, and ABR stamp-brush library in the Image Edit Mask tool; `birefnet.safetensors` must be placed in `ComfyUI/models/background_removal/` for BiRefNet
|
||||
- **[psd-tools](https://pypi.org/project/psd-tools/)** (v0.6.0) — enables all PSD (Photoshop) features: Gallery bulk "Create PSD" export, Gallery browsing/thumbnails for `.psd` files, and Image Edit's Save PSD / Open PSD buttons; `pip install -r requirements.txt` (ships prebuilt wheels, no C compiler required); without it, PSD-related actions show an error toast with the install command instead of failing silently
|
||||
|
||||
@@ -73,6 +73,7 @@ def setup_routes(app: web.Application):
|
||||
app.router.add_post("/wfm/gallery/semantic/similar", semantic_similar)
|
||||
app.router.add_post("/wfm/gallery/semantic/clear", semantic_clear)
|
||||
app.router.add_post("/wfm/gallery/semantic/test", semantic_test)
|
||||
app.router.add_post("/wfm/gallery/semantic/options", semantic_options)
|
||||
app.router.add_get("/wfm/gallery/images", list_images)
|
||||
app.router.add_get("/wfm/gallery/image/meta", get_image_meta)
|
||||
app.router.add_get("/wfm/gallery/image/workflow", get_image_workflow)
|
||||
@@ -112,7 +113,7 @@ def setup_routes(app: web.Application):
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# セマンティック検索 (Gemini Embedding 2)
|
||||
# セマンティック検索 (Gemini Embedding 2 / Ollama EmbeddingGemma 2)
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _semantic_error(e: SemanticError) -> web.Response:
|
||||
@@ -164,9 +165,9 @@ async def semantic_folders(request: web.Request) -> web.Response:
|
||||
|
||||
|
||||
def _apply_local_options(body: dict) -> None:
|
||||
"""リクエストに実行デバイス/スレッド数が含まれていれば反映・保存する"""
|
||||
if "device" in body:
|
||||
_semantic.set_local_options(str(body.get("device", "auto")), _num(body, "threads", 0, 0, 256))
|
||||
"""リクエストに Ollama の接続先/モデルサイズが含まれていれば反映・保存する"""
|
||||
if "ollama_url" in body:
|
||||
_semantic.set_local_options(str(body.get("ollama_url", "")), str(body.get("ollama_tag", "")))
|
||||
|
||||
|
||||
async def semantic_index(request: web.Request) -> web.Response:
|
||||
@@ -180,7 +181,6 @@ async def semantic_index(request: web.Request) -> web.Response:
|
||||
"all" if body.get("mode") == "all" else "missing",
|
||||
str(body.get("model", SEMANTIC_DEFAULT_MODEL)),
|
||||
_num(body, "dim", DEFAULT_DIM, 0, 100000),
|
||||
_num(body, "soft", 280, 0, 100000),
|
||||
)
|
||||
return web.json_response({"ok": True})
|
||||
except SemanticError as e:
|
||||
@@ -848,3 +848,16 @@ async def get_psd_layers(request: web.Request) -> web.Response:
|
||||
except Exception as e:
|
||||
logger.error("Error reading PSD layers: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
|
||||
async def semantic_options(request: web.Request) -> web.Response:
|
||||
"""Ollama の接続先/モデルサイズを保存して、新しい状態を返す"""
|
||||
body = await _json_body(request)
|
||||
try:
|
||||
await asyncio.to_thread(_apply_local_options, body)
|
||||
return web.json_response(await asyncio.to_thread(_semantic.status))
|
||||
except SemanticError as e:
|
||||
return _semantic_error(e)
|
||||
except Exception as e:
|
||||
logger.error("semantic_options error: %s", e)
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
|
||||
@@ -1,239 +0,0 @@
|
||||
"""
|
||||
EmbeddingGemma 2 ローカル実行 (onnx-community/embeddinggemma-2-ONNX)
|
||||
|
||||
onnxruntime + tokenizers + huggingface_hub だけで動かす(transformers の対応を待たない)。
|
||||
前処理は Transformers.js の Gemma4ImageProcessor / EmbeddingGemma2Processor の移植:
|
||||
画像 -> アスペクト比維持でリサイズ(48px倍数, 最大2520パッチ) -> 16pxパッチ化 -> vision_encoder
|
||||
-> image_features を text model の <|image|> トークン位置に差し込み -> sentence_embedding(768)
|
||||
モデルは初回のみ固定リポジトリからダウンロードする(約300MB, q4)。
|
||||
"""
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
REPO_ID = "onnx-community/embeddinggemma-2-ONNX"
|
||||
LOCAL_MODEL = "embeddinggemma-2"
|
||||
LOCAL_DIMS = (128, 256, 512, 768) # MRL対応次元
|
||||
DEFAULT_LOCAL_DIM = 768
|
||||
# 量子化精度 -> ファイル名サフィックス
|
||||
DTYPES = {"q4": "_q4", "q8": "_quantized", "fp16": "_fp16", "fp32": ""}
|
||||
DEFAULT_DTYPE = "q4"
|
||||
|
||||
_PATCH = 16
|
||||
_POOL = 3
|
||||
SOFT_TOKENS = (70, 140, 280) # 画像1枚あたりのソフトトークン数(多いほど高精度・低速)
|
||||
DEFAULT_SOFT = 280
|
||||
_MAX_TOKENS = 1024 # テキスト入力のトークン数上限(画像用プロンプトは約290トークン)
|
||||
_MAX_IMAGE_SIDE = 2048 # 画像の長辺上限(超える場合は縮小してから処理)
|
||||
_IMAGE_TOKEN = "<|image|>"
|
||||
_BOI = "<|image>"
|
||||
_EOI = "<image|>"
|
||||
|
||||
DEVICES = ("auto", "cpu", "cuda", "dml")
|
||||
_PROVIDER_BY_DEVICE = {"cuda": "CUDAExecutionProvider", "dml": "DmlExecutionProvider"}
|
||||
|
||||
_lock = threading.Lock()
|
||||
_runtime = None # ((dtype, device, threads), _Runtime)
|
||||
|
||||
|
||||
_opts = {"device": "auto", "threads": 0} # 実行デバイス/CPUスレッド数(configureで更新)
|
||||
|
||||
|
||||
def configure(device: str = "auto", threads: int = 0) -> None:
|
||||
"""ローカル推論の実行設定。次回のensure_runtimeから反映(device/threads変更時はセッション再作成)。"""
|
||||
_opts["device"] = device if device in DEVICES else "auto"
|
||||
_opts["threads"] = max(0, min(int(threads), (os.cpu_count() or 1) * 2))
|
||||
|
||||
|
||||
def current_config() -> dict:
|
||||
return dict(_opts)
|
||||
|
||||
|
||||
def release() -> None:
|
||||
"""セッションを破棄してVRAM/RAMを解放する(インデックス作成後など)。次回利用時に再ロードされる。"""
|
||||
global _runtime
|
||||
with _lock:
|
||||
_runtime = None
|
||||
|
||||
|
||||
def active_provider() -> str:
|
||||
"""直近にロードしたセッションの実行プロバイダ(未ロードなら空)"""
|
||||
rt = _runtime[1] if _runtime else None
|
||||
return rt.providers[0].replace("ExecutionProvider", "") if rt else ""
|
||||
|
||||
|
||||
def available_devices() -> dict:
|
||||
"""onnxruntime が使える実行デバイス。GPU用ビルド(onnxruntime-gpu / -directml)が無ければcpuのみ。"""
|
||||
try:
|
||||
import onnxruntime as ort
|
||||
prov = ort.get_available_providers()
|
||||
version = ort.__version__
|
||||
except Exception:
|
||||
return {"cpu": True, "cuda": False, "dml": False, "ort_version": ""}
|
||||
return {"cpu": True, "cuda": "CUDAExecutionProvider" in prov, "dml": "DmlExecutionProvider" in prov,
|
||||
"ort_version": version, "cpu_count": os.cpu_count() or 1}
|
||||
|
||||
|
||||
def default_threads() -> int:
|
||||
"""既定のCPUスレッド数: 画像生成と同居しても邪魔になりにくいよう、コアの半分(最低1)"""
|
||||
return max(1, (os.cpu_count() or 2) // 2)
|
||||
|
||||
|
||||
def _target_size(h: int, w: int, max_patches: int) -> tuple[int, int]:
|
||||
"""Gemma4ImageProcessor の get_aspect_ratio_preserving_size"""
|
||||
factor = math.sqrt(max_patches * _PATCH ** 2 / (h * w))
|
||||
side = _POOL * _PATCH
|
||||
th = int(math.floor(factor * h / side)) * side
|
||||
tw = int(math.floor(factor * w / side)) * side
|
||||
max_side = (max_patches // _POOL ** 2) * side
|
||||
if th == 0 and tw == 0:
|
||||
raise ValueError("image too small")
|
||||
if th == 0:
|
||||
th, tw = side, min((w // h) * side, max_side)
|
||||
elif tw == 0:
|
||||
tw, th = side, min((h // w) * side, max_side)
|
||||
return th, tw
|
||||
|
||||
|
||||
class _Runtime:
|
||||
def __init__(self, model_dir: Path, dtype: str, device: str = "auto", threads: int = 0):
|
||||
import onnxruntime as ort
|
||||
from tokenizers import Tokenizer
|
||||
|
||||
sfx = DTYPES[dtype]
|
||||
so = ort.SessionOptions()
|
||||
so.log_severity_level = 3 # CUDA使用時のMemcpy警告などを抑止
|
||||
if threads > 0:
|
||||
so.intra_op_num_threads = threads
|
||||
so.inter_op_num_threads = 1
|
||||
avail = ort.get_available_providers()
|
||||
if device in _PROVIDER_BY_DEVICE:
|
||||
providers = [_PROVIDER_BY_DEVICE[device]] if _PROVIDER_BY_DEVICE[device] in avail else []
|
||||
elif device == "cpu":
|
||||
providers = []
|
||||
else: # auto: 使えるGPUを優先
|
||||
providers = [p for p in ("CUDAExecutionProvider", "DmlExecutionProvider") if p in avail]
|
||||
if "DmlExecutionProvider" in providers:
|
||||
# DirectMLは並列実行/メモリパターン最適化に非対応
|
||||
so.enable_mem_pattern = False
|
||||
so.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL
|
||||
providers.append("CPUExecutionProvider")
|
||||
self.providers = providers
|
||||
|
||||
def make(name):
|
||||
path = str(model_dir / "onnx" / f"{name}{sfx}.onnx")
|
||||
try:
|
||||
return ort.InferenceSession(path, so, providers=providers)
|
||||
except Exception as e:
|
||||
if providers == ["CPUExecutionProvider"]:
|
||||
raise
|
||||
logger.warning("EmbeddingGemma2: %s failed on %s (%s); falling back to CPU", name, providers[0], e)
|
||||
self.providers = ["CPUExecutionProvider"]
|
||||
return ort.InferenceSession(path, so, providers=["CPUExecutionProvider"])
|
||||
|
||||
self.text = make("model")
|
||||
self.vision = make("vision_encoder")
|
||||
self.tok = Tokenizer.from_file(str(model_dir / "tokenizer.json"))
|
||||
self.tok.no_truncation()
|
||||
self.tok.no_padding()
|
||||
|
||||
def _run_text(self, text: str, image_features: np.ndarray | None) -> np.ndarray:
|
||||
ids = np.asarray([self.tok.encode(text).ids[:_MAX_TOKENS]], dtype=np.int64)
|
||||
feeds = {
|
||||
"input_ids": ids,
|
||||
"attention_mask": np.ones_like(ids),
|
||||
"image_features": image_features if image_features is not None else np.zeros((0, 512), np.float32),
|
||||
"video_features": np.zeros((0, 512), np.float32),
|
||||
"audio_features": np.zeros((0, 512), np.float32),
|
||||
}
|
||||
return self.text.run(["sentence_embedding"], feeds)[0][0]
|
||||
|
||||
def embed_text(self, text: str, prefix: str) -> np.ndarray:
|
||||
return self._run_text(f"{prefix}{text.strip()}", None)
|
||||
|
||||
def embed_image(self, image, soft_tokens: int = DEFAULT_SOFT) -> np.ndarray:
|
||||
from PIL import Image
|
||||
im = image.convert("RGB")
|
||||
if im.width < 1 or im.height < 1:
|
||||
raise ValueError("empty image")
|
||||
if max(im.size) > _MAX_IMAGE_SIDE: # 巨大画像でメモリを使い切らないよう先に縮小
|
||||
im.thumbnail((_MAX_IMAGE_SIDE, _MAX_IMAGE_SIDE), Image.LANCZOS)
|
||||
max_patches = soft_tokens * _POOL ** 2
|
||||
th, tw = _target_size(im.height, im.width, max_patches)
|
||||
if (th, tw) != (im.height, im.width):
|
||||
im = im.resize((tw, th), Image.BICUBIC)
|
||||
arr = np.asarray(im, dtype=np.float32) * (1.0 / 255.0) # H,W,3
|
||||
ph, pw = th // _PATCH, tw // _PATCH
|
||||
n = ph * pw
|
||||
# (pH, dy, pW, dx, C) -> (pH, pW, dy, dx, C) -> (n, 768)
|
||||
patches = arr.reshape(ph, _PATCH, pw, _PATCH, 3).transpose(0, 2, 1, 3, 4).reshape(n, _PATCH * _PATCH * 3)
|
||||
# パディング無し(n個ちょうど)で渡す: 計算量が減り、パディングは出力に影響しない
|
||||
pixel = patches[None].astype(np.float32)
|
||||
pos = np.zeros((1, n, 2), dtype=np.int64)
|
||||
rows, cols = np.divmod(np.arange(n), pw)
|
||||
pos[0, :, 0], pos[0, :, 1] = cols, rows
|
||||
feats = self.vision.run(["image_features"], {"pixel_values": pixel, "pixel_position_ids": pos})[0]
|
||||
text = f"{_BOI}{_IMAGE_TOKEN * (n // _POOL ** 2)}{_EOI}"
|
||||
return self._run_text(text, feats.astype(np.float32))
|
||||
|
||||
|
||||
def model_dir(data_dir: Path) -> Path:
|
||||
return data_dir / "models" / LOCAL_MODEL
|
||||
|
||||
|
||||
def is_downloaded(data_dir: Path, dtype: str = DEFAULT_DTYPE) -> bool:
|
||||
d, sfx = model_dir(data_dir), DTYPES.get(dtype, "_q4")
|
||||
need = [d / "tokenizer.json"] + [d / "onnx" / f"{n}{sfx}.onnx{e}" for n in ("model", "vision_encoder") for e in ("", "_data")]
|
||||
return all(p.exists() for p in need)
|
||||
|
||||
|
||||
def ensure_runtime(data_dir: Path, dtype: str = DEFAULT_DTYPE, progress=None,
|
||||
device: str | None = None, threads: int | None = None) -> _Runtime:
|
||||
"""モデルをダウンロード(未取得時)してセッションを返す。dtype/device/threads変更時は作り直す。"""
|
||||
global _runtime
|
||||
if dtype not in DTYPES:
|
||||
dtype = DEFAULT_DTYPE
|
||||
device = _opts["device"] if device is None else device
|
||||
threads = _opts["threads"] if threads is None else threads
|
||||
if device not in DEVICES:
|
||||
device = "auto"
|
||||
key = (dtype, device, int(threads))
|
||||
with _lock:
|
||||
if _runtime and _runtime[0] == key:
|
||||
return _runtime[1]
|
||||
d, sfx = model_dir(data_dir), DTYPES[dtype]
|
||||
if not is_downloaded(data_dir, dtype):
|
||||
if progress:
|
||||
progress("downloading")
|
||||
from huggingface_hub import hf_hub_download
|
||||
files = ["tokenizer.json"] + [f"onnx/{n}{sfx}.onnx{e}" for n in ("model", "vision_encoder") for e in ("", "_data")]
|
||||
for f in files:
|
||||
hf_hub_download(REPO_ID, f, local_dir=str(d))
|
||||
if progress:
|
||||
progress("loading")
|
||||
_runtime = (key, _Runtime(d, dtype, device, int(threads)))
|
||||
return _runtime[1]
|
||||
|
||||
|
||||
def _finish(vec: np.ndarray, dim: int) -> np.ndarray:
|
||||
v = np.asarray(vec, dtype=np.float32)
|
||||
if dim < v.shape[0]:
|
||||
v = v[:dim]
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def embed_text(data_dir: Path, text: str, dim: int, dtype: str = DEFAULT_DTYPE) -> np.ndarray:
|
||||
rt = ensure_runtime(data_dir, dtype)
|
||||
return _finish(rt.embed_text(text[:8000], "task: search result | query: "), dim)
|
||||
|
||||
|
||||
def embed_image(data_dir: Path, image, dim: int, soft: int = DEFAULT_SOFT, dtype: str = DEFAULT_DTYPE) -> np.ndarray:
|
||||
rt = ensure_runtime(data_dir, dtype)
|
||||
return _finish(rt.embed_image(image, soft if soft in SOFT_TOKENS else DEFAULT_SOFT), dim)
|
||||
@@ -0,0 +1,192 @@
|
||||
"""
|
||||
EmbeddingGemma 2 を Ollama で実行する (Gallery セマンティック検索のローカルバックエンド)
|
||||
|
||||
画像は /api/embed に input=[{"image": <base64>}] で渡す(images フィールドは無視されテキスト扱いになる)。
|
||||
740m(テキスト+画像+音声) と 440m(テキスト+画像) は同じ重みで同じベクトルを返すため、
|
||||
インデックスにはタグを区別せず LOCAL_MODEL として記録する。270m/570m は画像非対応なので選択肢に入れない。
|
||||
接続先は url_guard で検査する(既定はループバックのみ)。
|
||||
"""
|
||||
import base64
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import url_guard
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
LOCAL_MODEL = "embeddinggemma-2" # インデックス(npz)に記録するモデル名
|
||||
LOCAL_DIMS = (128, 256, 512, 768) # MRL対応次元
|
||||
DEFAULT_LOCAL_DIM = 768
|
||||
TAGS = ("embeddinggemma-2:740m", "embeddinggemma-2:440m")
|
||||
DEFAULT_TAG = TAGS[0]
|
||||
# :latest は 740m と同じモデル(parent_model 740m-q8_0)。740m が無ければ代わりに使う
|
||||
_ALIASES = {"embeddinggemma-2:740m": ("embeddinggemma-2:latest", "embeddinggemma-2")}
|
||||
DEFAULT_URL = "http://localhost:11434"
|
||||
QUERY_PREFIX = "task: search result | query: "
|
||||
BATCH = 4 # 1リクエストで送る画像数(1枚ずつより約1.8倍速い)
|
||||
_TIMEOUT = 120
|
||||
_RETRIES = 5 # Ollama内部の接続切断(HTTP 400 "forcibly closed"等)は再試行で回復する
|
||||
_TAGS_TTL = 5.0 # /api/tags の結果をキャッシュする秒数(進捗ポーリングのたびに問い合わせない)
|
||||
|
||||
_opts = {"url": DEFAULT_URL, "tag": DEFAULT_TAG}
|
||||
_tags_cache = {"at": 0.0, "url": "", "names": None, "error": ""}
|
||||
_lock = threading.Lock()
|
||||
|
||||
|
||||
class OllamaError(Exception):
|
||||
"""code: ollama_unreachable / ollama_model_missing / local_error"""
|
||||
|
||||
def __init__(self, code: str, message: str = ""):
|
||||
super().__init__(message or code)
|
||||
self.code = code
|
||||
|
||||
|
||||
def configure(url: str = DEFAULT_URL, tag: str = DEFAULT_TAG) -> None:
|
||||
"""接続先とモデルサイズを設定する。許可されないURLは ValueError(url_guard.HINT)。"""
|
||||
url = url_guard.require_backend_url((url or "").strip() or DEFAULT_URL)
|
||||
with _lock:
|
||||
_opts["url"] = url
|
||||
_opts["tag"] = tag if tag in TAGS else DEFAULT_TAG
|
||||
_tags_cache["at"] = 0.0
|
||||
|
||||
|
||||
def current_config() -> dict:
|
||||
return dict(_opts)
|
||||
|
||||
|
||||
def _post(path: str, body: dict, timeout: float = _TIMEOUT) -> dict:
|
||||
base = url_guard.require_backend_url(_opts["url"])
|
||||
req = urllib.request.Request(base + path, data=json.dumps(body).encode("utf-8"), method="POST",
|
||||
headers={"Content-Type": "application/json"})
|
||||
with url_guard.urlopen(req, timeout) as res:
|
||||
return json.loads(res.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _http_message(e: urllib.error.HTTPError) -> str:
|
||||
try:
|
||||
return json.loads(e.read().decode("utf-8")).get("error", "") or str(e.reason)
|
||||
except Exception:
|
||||
return str(e.reason)
|
||||
|
||||
|
||||
def _transient(code: int, msg: str) -> bool:
|
||||
low = msg.lower()
|
||||
if "failed to load image" in low:
|
||||
return False # 壊れた画像。再試行しても直らない
|
||||
if code >= 500:
|
||||
return True
|
||||
return code == 400 and any(k in low for k in ("forcibly closed", "wsarecv", "connection", "eof"))
|
||||
|
||||
|
||||
def installed_tags(force: bool = False) -> tuple[list[str] | None, str]:
|
||||
"""(インストール済みモデル名, エラー)。接続できなければ (None, エラー文)。"""
|
||||
now = time.time()
|
||||
with _lock:
|
||||
c = dict(_tags_cache)
|
||||
url = _opts["url"]
|
||||
if not force and c["url"] == url and now - c["at"] < _TAGS_TTL:
|
||||
return c["names"], c["error"]
|
||||
names, err = None, ""
|
||||
try:
|
||||
base = url_guard.require_backend_url(url)
|
||||
with url_guard.urlopen(urllib.request.Request(base + "/api/tags"), 3) as res:
|
||||
data = json.loads(res.read().decode("utf-8"))
|
||||
names = [m.get("name") or m.get("model") or "" for m in data.get("models", [])]
|
||||
except Exception as e:
|
||||
err = str(getattr(e, "reason", e))
|
||||
with _lock:
|
||||
_tags_cache.update(at=now, url=url, names=names, error=err)
|
||||
return names, err
|
||||
|
||||
|
||||
def _resolve(names: list[str], tag: str) -> str:
|
||||
"""実際に呼ぶモデル名。未インストールなら空文字"""
|
||||
for cand in (tag,) + _ALIASES.get(tag, ()):
|
||||
if cand in names:
|
||||
return cand
|
||||
return ""
|
||||
|
||||
|
||||
def status() -> dict:
|
||||
names, err = installed_tags()
|
||||
tag = _opts["tag"]
|
||||
out = {"url": _opts["url"], "tag": tag, "tags": list(TAGS), "reachable": names is not None, "error": err,
|
||||
"installed": {t: bool(names and _resolve(names, t)) for t in TAGS}}
|
||||
out["ready"] = bool(out["reachable"] and out["installed"][tag])
|
||||
return out
|
||||
|
||||
|
||||
def _model_name() -> str:
|
||||
names, err = installed_tags()
|
||||
if names is None:
|
||||
raise OllamaError("ollama_unreachable", f"{_opts['url']}: {err}")
|
||||
name = _resolve(names, _opts["tag"])
|
||||
if not name:
|
||||
raise OllamaError("ollama_model_missing", _opts["tag"])
|
||||
return name
|
||||
|
||||
|
||||
def _embed(inputs: list) -> list[np.ndarray]:
|
||||
model = _model_name()
|
||||
last = ""
|
||||
for attempt in range(_RETRIES + 1):
|
||||
try:
|
||||
data = _post("/api/embed", {"model": model, "input": inputs})
|
||||
embs = data.get("embeddings") or []
|
||||
if len(embs) != len(inputs):
|
||||
raise OllamaError("local_error", f"expected {len(inputs)} embeddings, got {len(embs)}")
|
||||
return [np.asarray(e, dtype=np.float32) for e in embs]
|
||||
except urllib.error.HTTPError as e:
|
||||
msg = _http_message(e)
|
||||
last = f"HTTP {e.code}: {msg}"
|
||||
if e.code == 404:
|
||||
with _lock:
|
||||
_tags_cache["at"] = 0.0 # 削除された可能性があるので次回は問い合わせ直す
|
||||
raise OllamaError("ollama_model_missing", model)
|
||||
if not _transient(e.code, msg) or attempt >= _RETRIES:
|
||||
raise OllamaError("local_error", last)
|
||||
except (urllib.error.URLError, TimeoutError, ConnectionError) as e:
|
||||
last = str(getattr(e, "reason", e))
|
||||
if attempt >= _RETRIES:
|
||||
raise OllamaError("ollama_unreachable", f"{_opts['url']}: {last}")
|
||||
time.sleep(min(4.0, 0.5 * (2 ** attempt)))
|
||||
raise OllamaError("local_error", last)
|
||||
|
||||
|
||||
def _finish(vec: np.ndarray, dim: int) -> np.ndarray:
|
||||
v = vec[:dim] if dim < vec.shape[0] else vec
|
||||
n = float(np.linalg.norm(v))
|
||||
return v / n if n > 0 else v
|
||||
|
||||
|
||||
def _image_b64(image) -> str:
|
||||
buf = io.BytesIO()
|
||||
image.convert("RGB").save(buf, "JPEG", quality=88)
|
||||
return base64.b64encode(buf.getvalue()).decode("ascii")
|
||||
|
||||
|
||||
def embed_text(text: str, dim: int) -> np.ndarray:
|
||||
return _finish(_embed([QUERY_PREFIX + text[:8000].strip()])[0], dim)
|
||||
|
||||
|
||||
def embed_images(images: list, dim: int) -> list[np.ndarray]:
|
||||
"""PIL画像(RGB)のリストをまとめて埋め込む。1枚でも読めないとリクエスト全体が失敗する。"""
|
||||
return [_finish(v, dim) for v in _embed([{"image": _image_b64(im)} for im in images])]
|
||||
|
||||
|
||||
def release() -> None:
|
||||
"""モデルをVRAMから外す(インデックス作成後)。次回利用時にOllamaが再ロードする。"""
|
||||
try:
|
||||
names, _ = installed_tags()
|
||||
name = _resolve(names or [], _opts["tag"])
|
||||
if name:
|
||||
_post("/api/embed", {"model": name, "input": [], "keep_alive": 0}, timeout=30)
|
||||
except Exception as e:
|
||||
logger.debug("semantic_ollama: release failed: %s", e)
|
||||
+101
-74
@@ -1,7 +1,7 @@
|
||||
"""
|
||||
Semantic Service - Gallery用セマンティック検索（Gemini Embedding 2）
|
||||
Semantic Service - Gallery用セマンティック検索（Gemini Embedding 2 / Ollama の EmbeddingGemma 2）
|
||||
|
||||
画像(動画・PSDはサムネイル)をGemini Embedding APIでベクトル化してローカルに保存し、
|
||||
画像(動画・PSDはサムネイル)をGemini Embedding API または Ollama でベクトル化してローカルに保存し、
|
||||
テキストクエリ/類似画像でコサイン類似度検索する。インデックス構築後の検索は
|
||||
(テキストクエリの埋め込み1回を除き)ローカル処理でAPI費用はかからない。
|
||||
|
||||
@@ -21,13 +21,13 @@ from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from . import semantic_local as local
|
||||
from . import semantic_ollama as local
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
API_BASE = "https://generativelanguage.googleapis.com/v1beta"
|
||||
MODEL = "gemini-embedding-2" # Gemini API (クラウド)
|
||||
LOCAL_MODEL = local.LOCAL_MODEL # EmbeddingGemma 2 (ローカルONNX)
|
||||
LOCAL_MODEL = local.LOCAL_MODEL # EmbeddingGemma 2 (Ollama)
|
||||
DEFAULT_DIM = 1536
|
||||
VALID_DIMS = (768, 1536, 3072)
|
||||
# バックエンド(モデル名) -> 選択可能な出力次元
|
||||
@@ -126,7 +126,6 @@ class SemanticService:
|
||||
self._stamp: dict[str, tuple[int, float]] = {} # path -> (size, mtime)
|
||||
self._dim = 0 # インデックスの次元(0=空)
|
||||
self._model = MODEL # インデックスを作ったモデル
|
||||
self._soft = local.DEFAULT_SOFT # ローカルモデルの画像ソフトトークン数(インデックスと一致させる)
|
||||
self._settings_file = data_dir / "semantic_settings.json"
|
||||
self._load_settings()
|
||||
self._mat_cache = None # (paths, matrix)
|
||||
@@ -137,21 +136,22 @@ class SemanticService:
|
||||
self._cancel = threading.Event()
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
# ── ローカル推論の実行設定(デバイス/スレッド数) ─────────────
|
||||
# ── Ollama の接続先/モデルサイズ ────────────────────────
|
||||
|
||||
def _load_settings(self) -> None:
|
||||
try:
|
||||
if self._settings_file.exists():
|
||||
d = json.loads(self._settings_file.read_text(encoding="utf-8"))
|
||||
local.configure(str(d.get("device", "auto")), int(d.get("threads", 0)))
|
||||
return
|
||||
local.configure(str(d.get("url", local.DEFAULT_URL)), str(d.get("tag", local.DEFAULT_TAG)))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: failed to load settings: %s", e)
|
||||
local.configure("auto", local.default_threads())
|
||||
|
||||
def set_local_options(self, device: str, threads: int) -> None:
|
||||
"""実行デバイス/CPUスレッド数を反映して保存する(検索時のクエリ埋め込みにも使われる)"""
|
||||
local.configure(device, threads)
|
||||
def set_local_options(self, url: str, tag: str) -> None:
|
||||
"""Ollama の接続先/モデルサイズを反映して保存する(検索時のクエリ埋め込みにも使われる)"""
|
||||
try:
|
||||
local.configure(url, tag)
|
||||
except ValueError as e:
|
||||
raise SemanticError("bad_url", str(e))
|
||||
try:
|
||||
self._settings_file.write_text(json.dumps(local.current_config()), encoding="utf-8")
|
||||
except Exception as e:
|
||||
@@ -177,7 +177,6 @@ class SemanticService:
|
||||
if model not in DIMS_BY_MODEL:
|
||||
return
|
||||
self._model = model
|
||||
self._soft = int(z["soft"]) if "soft" in z.files else local.DEFAULT_SOFT
|
||||
paths = [str(p) for p in z["paths"]]
|
||||
sizes, mtimes, vecs = z["sizes"], z["mtimes"], z["vecs"]
|
||||
self._dim = int(z["dim"])
|
||||
@@ -201,7 +200,7 @@ class SemanticService:
|
||||
tmp = self._file.with_suffix(".tmp.npz")
|
||||
np.savez(
|
||||
tmp,
|
||||
model=np.array(self._model), dim=np.array(self._dim), soft=np.array(self._soft),
|
||||
model=np.array(self._model), dim=np.array(self._dim),
|
||||
paths=np.array(paths),
|
||||
sizes=np.array([self._stamp[p][0] for p in paths], dtype=np.int64),
|
||||
mtimes=np.array([self._stamp[p][1] for p in paths], dtype=np.float64),
|
||||
@@ -228,24 +227,18 @@ class SemanticService:
|
||||
|
||||
def status(self, folder: str = "", recursive: bool = True) -> dict:
|
||||
self._ensure_loaded()
|
||||
ollama = local.status()
|
||||
out = {
|
||||
"has_key": bool(get_api_key()),
|
||||
"model": MODEL,
|
||||
"backends": {
|
||||
MODEL: {"dims": list(VALID_DIMS), "default_dim": DEFAULT_DIM, "ready": bool(get_api_key())},
|
||||
LOCAL_MODEL: {"dims": list(local.LOCAL_DIMS), "default_dim": local.DEFAULT_LOCAL_DIM,
|
||||
"ready": True, "downloaded": local.is_downloaded(self.data_dir)},
|
||||
"ready": ollama["ready"]},
|
||||
},
|
||||
"index_model": self._model,
|
||||
"index_dim": self._dim,
|
||||
"index_soft": self._soft,
|
||||
"local": {
|
||||
"devices": local.available_devices(),
|
||||
"options": local.current_config(),
|
||||
"default_threads": local.default_threads(),
|
||||
"soft_tokens": list(local.SOFT_TOKENS),
|
||||
"default_soft": local.DEFAULT_SOFT,
|
||||
},
|
||||
"ollama": ollama,
|
||||
"indexed_total": len(self._vecs),
|
||||
"job": dict(self._job),
|
||||
}
|
||||
@@ -279,8 +272,7 @@ class SemanticService:
|
||||
|
||||
# ── インデックス作成ジョブ ────────────────────────────
|
||||
|
||||
def start_index(self, folder: str, recursive: bool, mode: str, model: str, dim: int,
|
||||
soft: int = local.DEFAULT_SOFT) -> None:
|
||||
def start_index(self, folder: str, recursive: bool, mode: str, model: str, dim: int) -> None:
|
||||
self._ensure_loaded()
|
||||
if model not in DIMS_BY_MODEL:
|
||||
raise SemanticError("bad_model", model)
|
||||
@@ -288,15 +280,19 @@ class SemanticService:
|
||||
raise SemanticError("no_api_key")
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
raise SemanticError("bad_dim", str(dim))
|
||||
if model != LOCAL_MODEL or soft not in local.SOFT_TOKENS:
|
||||
soft = local.DEFAULT_SOFT # ソフトトークン数はローカルモデルのみの設定
|
||||
if model == LOCAL_MODEL:
|
||||
st = local.status() # 接続できない/モデル未取得なら、ジョブを始める前に知らせる
|
||||
if not st["reachable"]:
|
||||
raise SemanticError("ollama_unreachable", f"{st['url']}: {st['error']}")
|
||||
if not st["ready"]:
|
||||
raise SemanticError("ollama_model_missing", st["tag"])
|
||||
with self._lock:
|
||||
if self._job["running"]:
|
||||
raise SemanticError("already_running")
|
||||
entries = self._scope_entries(folder, recursive)
|
||||
mismatch = bool(self._vecs) and (self._dim != dim or self._model != model or self._soft != soft)
|
||||
mismatch = bool(self._vecs) and (self._dim != dim or self._model != model)
|
||||
if mismatch and mode != "all":
|
||||
raise SemanticError("dim_mismatch", f"{self._model}/{self._dim}/{self._soft}")
|
||||
raise SemanticError("dim_mismatch", f"{self._model}/{self._dim}")
|
||||
if mode == "all":
|
||||
# 範囲内の既存ベクトルを破棄して作り直す(モデル/次元違いなら全消去)
|
||||
if mismatch:
|
||||
@@ -309,14 +305,13 @@ class SemanticService:
|
||||
self._mat_cache = None
|
||||
self._dim = dim
|
||||
self._model = model
|
||||
self._soft = soft
|
||||
todo = [e for e in entries if not self._is_fresh(e[1], e[2], e[3])]
|
||||
# スコープ内で消えたファイルの残骸を掃除(スコープ配下のキーのみ)
|
||||
self._prune_missing(folder, recursive, {e[1] for e in entries})
|
||||
self._cancel.clear()
|
||||
self._job = self._idle_job()
|
||||
self._job.update(running=True, total=len(todo))
|
||||
self._thread = threading.Thread(target=self._run_index, args=(todo, model, dim, soft), daemon=True)
|
||||
self._thread = threading.Thread(target=self._run_index, args=(todo, model, dim), daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _prune_missing(self, folder: str, recursive: bool, existing: set[str]) -> None:
|
||||
@@ -335,52 +330,78 @@ class SemanticService:
|
||||
def cancel(self) -> None:
|
||||
self._cancel.set()
|
||||
|
||||
def _run_index(self, todo: list, model: str, dim: int, soft: int = local.DEFAULT_SOFT) -> None:
|
||||
def work(entry):
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
name, ap, size, mtime = entry
|
||||
self._job["current"] = name
|
||||
try:
|
||||
vec = self._embed_image(model, dim, Path(ap), soft)
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
except SemanticError as e:
|
||||
if e.code in ("no_api_key", "api_key_invalid"):
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
self._cancel.set()
|
||||
else:
|
||||
self._job["failed"] += 1
|
||||
if not self._job["error"]:
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: embed failed for %s: %s", ap, e)
|
||||
def _run_index(self, todo: list, model: str, dim: int) -> None:
|
||||
def fail(e: SemanticError) -> None:
|
||||
if e.code in ("no_api_key", "api_key_invalid", "ollama_unreachable", "ollama_model_missing"):
|
||||
# 続けても全件失敗するので中止する
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
self._cancel.set()
|
||||
else:
|
||||
self._job["failed"] += 1
|
||||
if not self._job["error"]:
|
||||
self._job["error_code"], self._job["error"] = e.code, str(e)
|
||||
|
||||
def checkpoint() -> None:
|
||||
if self._dirty >= _CHECKPOINT_EVERY:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: checkpoint failed: %s", e)
|
||||
|
||||
def work(entry):
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
name, ap, size, mtime = entry
|
||||
self._job["current"] = name
|
||||
try:
|
||||
vec = self._embed_image(model, dim, Path(ap))
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
except SemanticError as e:
|
||||
fail(e)
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: embed failed for %s: %s", ap, e)
|
||||
self._job["failed"] += 1
|
||||
checkpoint()
|
||||
|
||||
def work_batch(batch):
|
||||
"""Ollama: まとめて送る。失敗したら1枚ずつ送り直して原因の画像だけを失敗扱いにする"""
|
||||
if self._cancel.is_set():
|
||||
return
|
||||
self._job["current"] = batch[0][0]
|
||||
vecs = None
|
||||
try:
|
||||
ims = [self._load_local_image(Path(e[1])) for e in batch]
|
||||
vecs = local.embed_images(ims, dim)
|
||||
except local.OllamaError as e:
|
||||
if e.code != "local_error":
|
||||
fail(SemanticError(e.code, str(e)))
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
if vecs is None:
|
||||
for entry in batch:
|
||||
work(entry)
|
||||
return
|
||||
for (name, ap, size, mtime), vec in zip(batch, vecs):
|
||||
self._put(ap, vec, size, mtime)
|
||||
self._job["done"] += 1
|
||||
checkpoint()
|
||||
|
||||
try:
|
||||
if model == LOCAL_MODEL:
|
||||
# 初回はモデルのダウンロード/ロードを先に行う(失敗したらジョブ全体を中止)
|
||||
try:
|
||||
local.ensure_runtime(self.data_dir, progress=lambda ph: self._job.update(current=f"({ph} model...)"))
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: local model load failed: %s", e)
|
||||
self._job["error_code"], self._job["error"] = "local_error", str(e)
|
||||
self._cancel.set()
|
||||
workers = 1 if model == LOCAL_MODEL else _WORKERS # ローカルはORT側がマルチスレッド
|
||||
with ThreadPoolExecutor(max_workers=workers) as ex:
|
||||
list(ex.map(work, todo))
|
||||
for i in range(0, len(todo), local.BATCH):
|
||||
work_batch(todo[i:i + local.BATCH])
|
||||
else:
|
||||
with ThreadPoolExecutor(max_workers=_WORKERS) as ex:
|
||||
list(ex.map(work, todo))
|
||||
finally:
|
||||
try:
|
||||
self._save()
|
||||
except Exception as e:
|
||||
logger.warning("SemanticService: save failed: %s", e)
|
||||
if model == LOCAL_MODEL:
|
||||
local.release() # 画像生成とVRAM/RAMを奪い合わないよう、作成後はモデルを解放(検索時に再ロード)
|
||||
local.release() # 画像生成とVRAMを奪い合わないよう、作成後はモデルを外す(検索時にOllamaが再ロード)
|
||||
self._job["running"] = False
|
||||
self._job["finished"] = True
|
||||
self._job["cancelled"] = self._cancel.is_set() and not self._job["error_code"]
|
||||
@@ -401,18 +422,24 @@ class SemanticService:
|
||||
im.thumbnail((IMAGE_MAX_SIZE, IMAGE_MAX_SIZE), Image.LANCZOS)
|
||||
return im
|
||||
|
||||
def _embed_image(self, model: str, dim: int, path: Path, soft: int | None = None) -> np.ndarray:
|
||||
def _load_local_image(self, path: Path):
|
||||
"""Ollama用のRGB画像(透過部は白背景に合成)"""
|
||||
from PIL import Image
|
||||
im = self._load_pil(path)
|
||||
if im.mode == "RGBA":
|
||||
bg = Image.new("RGB", im.size, (255, 255, 255))
|
||||
bg.paste(im, mask=im.getchannel("A"))
|
||||
return bg
|
||||
return im
|
||||
|
||||
def _embed_image(self, model: str, dim: int, path: Path) -> np.ndarray:
|
||||
if model == LOCAL_MODEL:
|
||||
if im.mode == "RGBA": # 透過部は白背景に合成
|
||||
bg = Image.new("RGB", im.size, (255, 255, 255))
|
||||
bg.paste(im, mask=im.getchannel("A"))
|
||||
im = bg
|
||||
im = self._load_local_image(path)
|
||||
try:
|
||||
return local.embed_image(self.data_dir, im, dim, self._soft if soft is None else soft)
|
||||
except Exception as e:
|
||||
raise SemanticError("local_error", str(e))
|
||||
return local.embed_images([im], dim)[0]
|
||||
except local.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
im = self._load_pil(path)
|
||||
buf = io.BytesIO()
|
||||
if im.mode == "RGBA":
|
||||
im.save(buf, "PNG")
|
||||
@@ -423,9 +450,9 @@ class SemanticService:
|
||||
def _embed_text(self, model: str, dim: int, text: str) -> np.ndarray:
|
||||
if model == LOCAL_MODEL:
|
||||
try:
|
||||
return local.embed_text(self.data_dir, text, dim)
|
||||
except Exception as e:
|
||||
raise SemanticError("local_error", str(e))
|
||||
return local.embed_text(text, dim)
|
||||
except local.OllamaError as e:
|
||||
raise SemanticError(e.code, str(e))
|
||||
return embed_text(text, dim)
|
||||
|
||||
# ── 検索 ───────────────────────────────────────────────
|
||||
@@ -510,4 +537,4 @@ class SemanticService:
|
||||
if dim not in DIMS_BY_MODEL[model]:
|
||||
dim = DEFAULT_DIM_BY_MODEL[model]
|
||||
n = int(self._embed_text(model, dim, "test").shape[0])
|
||||
return {"dimensions": n, "provider": local.active_provider() if model == LOCAL_MODEL else "API"}
|
||||
return {"dimensions": n, "provider": f"Ollama {local.current_config()['tag']}" if model == LOCAL_MODEL else "API"}
|
||||
|
||||
+3
-9
@@ -34,12 +34,6 @@ psd-tools>=1.10
|
||||
# (Gallery itself keeps working). Not pinned here on purpose — see torch note above.
|
||||
# av>=17.0.0
|
||||
|
||||
# -- Gallery Semantic Search: local EmbeddingGemma 2 (optional) --
|
||||
# Uses onnxruntime (above) + tokenizers + huggingface_hub (both normally pre-installed with ComfyUI's
|
||||
# transformers). The model (~300 MB, q4) is downloaded to the data folder on first use.
|
||||
# GPU (NVIDIA/CUDA): replace onnxruntime with onnxruntime-gpu
|
||||
# pip uninstall onnxruntime onnxruntime-gpu
|
||||
# pip install onnxruntime-gpu
|
||||
# Local EmbeddingGemma 2 then indexes ~45x faster than on the CPU. DirectML is not supported.
|
||||
# tokenizers
|
||||
# huggingface_hub
|
||||
# -- Gallery Semantic Search: EmbeddingGemma 2 (optional) --
|
||||
# Runs in Ollama (no extra Python packages). Install Ollama and download the model:
|
||||
# ollama pull embeddinggemma-2:740m (or embeddinggemma-2:440m - same vectors, no audio encoder)
|
||||
|
||||
@@ -1151,6 +1151,7 @@
|
||||
.wfm-sem-pane { display: flex; flex-direction: column; gap: 10px; min-height: 230px; }
|
||||
.wfm-sem-footer { justify-content: flex-end; border-top: 1px solid var(--wfm-border, #444); padding-top: 10px; }
|
||||
.wfm-sem-num { width: 80px; }
|
||||
.wfm-sem-url { width: 240px; max-width: 100%; }
|
||||
.wfm-sem-danger { margin-left: auto; color: #ef4444; }
|
||||
|
||||
.wfm-sem-row .wfm-select { width: auto; min-width: 90px; }
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Gallery Tab - セマンティック検索 (Gemini Embedding 2)
|
||||
* Gallery Tab - セマンティック検索 (Gemini Embedding 2 / Ollama の EmbeddingGemma 2)
|
||||
* - 検索ボックスのSemantic Searchトグル（ONでEnter実行の意味検索）
|
||||
* - 詳細パネルの「類似画像」トグル／更新ボタン
|
||||
* - インデックス作成モーダル（Semantic⚙）
|
||||
@@ -19,18 +19,20 @@ const API = {
|
||||
clear: "/wfm/gallery/semantic/clear",
|
||||
folders: (root) => `/wfm/gallery/semantic/folders?root=${encodeURIComponent(root)}`,
|
||||
test: "/wfm/gallery/semantic/test",
|
||||
options: "/wfm/gallery/semantic/options",
|
||||
};
|
||||
|
||||
const LS_ON = "wfm_gallery_semantic_on";
|
||||
const LS_DIM = "wfm_gallery_semantic_dim";
|
||||
const LS_MODEL = "wfm_gallery_semantic_model";
|
||||
const LS_SOFT = "wfm_gallery_semantic_soft";
|
||||
const GEMINI = "gemini-embedding-2";
|
||||
const LOCAL = "embeddinggemma-2";
|
||||
const LS_SUB = "wfm_gallery_semantic_recursive";
|
||||
const LS_TAB = "wfm_gallery_semantic_tab";
|
||||
const LS_TH = "wfm_gallery_semantic_th";
|
||||
const LIMITS = [50, 100, 200, 500, 1000];
|
||||
// Ollama のモデルサイズ(どちらも同じベクトルを返すので、切り替えても再作成は不要)
|
||||
const OLLAMA_TAGS = [["embeddinggemma-2:740m", "semOllamaTag740"], ["embeddinggemma-2:440m", "semOllamaTag440"]];
|
||||
|
||||
const sem = {
|
||||
on: false, // Semantic Searchトグル
|
||||
@@ -65,7 +67,8 @@ async function postJson(url, body) {
|
||||
/** APIエラーコードをi18nメッセージにする */
|
||||
export function semanticErrorMessage(e) {
|
||||
const known = ["no_api_key", "api_key_invalid", "index_empty", "dim_mismatch", "already_running",
|
||||
"folder_not_allowed", "unsupported_type", "network_error", "api_error", "local_error"];
|
||||
"folder_not_allowed", "unsupported_type", "network_error", "api_error", "local_error",
|
||||
"ollama_unreachable", "ollama_model_missing", "bad_url"];
|
||||
if (e?.code && known.includes(e.code)) {
|
||||
return t(`sem_err_${e.code}`, e.message);
|
||||
}
|
||||
@@ -244,23 +247,14 @@ function modalHtml() {
|
||||
<select id="wfm-sem-dim" class="wfm-select"></select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semDimHint"))}</span></div>
|
||||
<div id="wfm-sem-local-opts" class="wfm-sem-local-opts" style="display:none;">
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semDevice"))}</b>
|
||||
<select id="wfm-sem-device" class="wfm-select">
|
||||
<option value="auto">${escapeHtml(t("semDeviceAuto"))}</option>
|
||||
<option value="cpu">CPU</option>
|
||||
<option value="cuda">GPU (CUDA)</option>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semOllamaUrl"))}</b>
|
||||
<input type="text" id="wfm-sem-ollama-url" class="wfm-input wfm-sem-url" spellcheck="false" placeholder="http://localhost:11434"></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semOllamaTag"))}</b>
|
||||
<select id="wfm-sem-ollama-tag" class="wfm-select">
|
||||
${OLLAMA_TAGS.map(([tag, key]) => `<option value="${tag}">${escapeHtml(t(key))}</option>`).join("")}
|
||||
</select>
|
||||
<span id="wfm-sem-device-info" class="wfm-sem-hint"></span></div>
|
||||
<div class="wfm-sem-row" id="wfm-sem-threads-row"><b>${escapeHtml(t("semThreads"))}</b>
|
||||
<select id="wfm-sem-threads" class="wfm-select"></select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semThreadsHint"))}</span></div>
|
||||
<div class="wfm-sem-row"><b>${escapeHtml(t("semSoft"))}</b>
|
||||
<select id="wfm-sem-soft" class="wfm-select">
|
||||
<option value="280">${escapeHtml(t("semSoftHigh"))}</option>
|
||||
<option value="140">${escapeHtml(t("semSoftStd"))}</option>
|
||||
<option value="70">${escapeHtml(t("semSoftLight"))}</option>
|
||||
</select>
|
||||
<span class="wfm-sem-hint">${escapeHtml(t("semSoftHint"))}</span></div>
|
||||
<span id="wfm-sem-ollama-info" class="wfm-sem-hint"></span></div>
|
||||
<p class="wfm-sem-hint">${escapeHtml(t("semOllamaTagHint"))}</p>
|
||||
</div>
|
||||
<div class="wfm-sem-actions">
|
||||
<button class="wfm-btn" id="wfm-sem-test">${escapeHtml(t("semTest"))}</button>
|
||||
@@ -302,19 +296,12 @@ function selectedModel() {
|
||||
return document.querySelector('input[name="wfm-sem-model"]:checked')?.value || GEMINI;
|
||||
}
|
||||
|
||||
/** ローカルモデルの実行設定(デバイス/スレッド/画像精度)の初期値をサーバー保存値から設定する */
|
||||
/** Ollama の接続先/モデルサイズの初期値をサーバー保存値から設定する */
|
||||
function initLocalOpts(st) {
|
||||
const lo = st.local || {};
|
||||
const devSel = document.getElementById("wfm-sem-device");
|
||||
devSel.value = (lo.options && lo.options.device) || "auto";
|
||||
const cores = lo.devices?.cpu_count || 4;
|
||||
const cur = (lo.options && lo.options.threads) || lo.default_threads || Math.max(1, Math.floor(cores / 2));
|
||||
const set = new Set([1, 2, 4, 6, 8, 12, 16, 24, 32].filter(n => n <= cores).concat([cores, cur, lo.default_threads || 1]));
|
||||
document.getElementById("wfm-sem-threads").innerHTML = [...set].sort((a, b) => a - b)
|
||||
.map(n => `<option value="${n}">${n}${n === cores ? ` (${t("semThreadsAll")})` : ""}</option>`).join("");
|
||||
document.getElementById("wfm-sem-threads").value = String(cur);
|
||||
const soft = st.indexed_total && st.index_model === LOCAL ? st.index_soft : parseInt(lsGet(LS_SOFT, String(lo.default_soft || 280)), 10);
|
||||
document.getElementById("wfm-sem-soft").value = String([70, 140, 280].includes(soft) ? soft : 280);
|
||||
const ol = st.ollama || {};
|
||||
document.getElementById("wfm-sem-ollama-url").value = ol.url || "";
|
||||
const sel = document.getElementById("wfm-sem-ollama-tag");
|
||||
if (ol.tag) sel.value = ol.tag;
|
||||
}
|
||||
|
||||
/** 検索タブ: しきい値の入力欄にインデックスのモデル用の保存値を入れる（入力中の値は上書きしない） */
|
||||
@@ -359,20 +346,15 @@ function closeSemanticModal() {
|
||||
}
|
||||
}
|
||||
|
||||
/** 検出した実行デバイスの表示(GPU版onnxruntimeが無いときの案内を含む) */
|
||||
function updateDeviceInfo(lo) {
|
||||
const dev = lo?.devices;
|
||||
if (!dev) return;
|
||||
const info = document.getElementById("wfm-sem-device-info");
|
||||
const cudaOpt = document.querySelector('#wfm-sem-device option[value="cuda"]');
|
||||
if (cudaOpt) cudaOpt.disabled = !dev.cuda;
|
||||
if (!dev.cuda && document.getElementById("wfm-sem-device").value === "cuda") {
|
||||
document.getElementById("wfm-sem-device").value = "auto";
|
||||
}
|
||||
if (info) info.textContent = dev.cuda ? t("semDeviceCudaOk", dev.ort_version) : t("semDeviceNoGpu", dev.ort_version);
|
||||
const dv = document.getElementById("wfm-sem-device").value;
|
||||
const useCpu = dv === "cpu" || (dv === "auto" && !dev.cuda);
|
||||
document.getElementById("wfm-sem-threads-row").style.display = useCpu ? "" : "none";
|
||||
/** モデルサイズの選択肢に取得済みかどうかを表示する */
|
||||
function updateOllamaInfo(ol) {
|
||||
if (!ol) return;
|
||||
document.querySelectorAll("#wfm-sem-ollama-tag option").forEach(opt => {
|
||||
const key = OLLAMA_TAGS.find(([tag]) => tag === opt.value)?.[1];
|
||||
if (key) opt.textContent = t(key) + (ol.installed?.[opt.value] ? "" : ` ${t("semOllamaNotPulled")}`);
|
||||
});
|
||||
const info = document.getElementById("wfm-sem-ollama-info");
|
||||
if (info) info.textContent = ol.reachable && !ol.ready ? t("semOllamaPullHint", ol.tag) : "";
|
||||
}
|
||||
|
||||
/** 選択中モデルの次元選択肢を作り直す(preferredが選択肢にあればそれを選ぶ) */
|
||||
@@ -414,11 +396,12 @@ async function refreshModal() {
|
||||
fillThresholds(st);
|
||||
const model = selectedModel();
|
||||
document.getElementById("wfm-sem-local-opts").style.display = model === LOCAL ? "" : "none";
|
||||
updateDeviceInfo(st.local);
|
||||
const ready = model === LOCAL ? true : !!st.has_key;
|
||||
const keyLine = model === LOCAL
|
||||
? (_backends[LOCAL]?.downloaded ? t("semLocalReady") : t("semLocalNeedDownload"))
|
||||
: (st.has_key ? t("semKeyOk") : t("semKeyMissing"));
|
||||
updateOllamaInfo(st.ollama);
|
||||
const ol = st.ollama || {};
|
||||
const ready = model === LOCAL ? !!ol.ready : !!st.has_key;
|
||||
const keyLine = model !== LOCAL ? (st.has_key ? t("semKeyOk") : t("semKeyMissing"))
|
||||
: !ol.reachable ? t("semOllamaUnreachable", ol.url)
|
||||
: ol.ready ? t("semOllamaReady", ol.tag) : t("semOllamaMissing", ol.tag);
|
||||
const idxLine = t("semIndexInfo", st.indexed_total, st.index_dim || "-");
|
||||
const scopeLine = st.scope_total != null ? t("semScopeInfo", st.scope_indexed, st.scope_total) : "";
|
||||
document.getElementById("wfm-sem-status").innerHTML =
|
||||
@@ -437,7 +420,7 @@ async function refreshModal() {
|
||||
let text = t("semProgress", job.done || 0, total, job.failed || 0);
|
||||
if (job.running && job.current) text += ` ${job.current}`;
|
||||
if (!job.running) {
|
||||
if (["no_api_key", "api_key_invalid", "local_error"].includes(job.error_code)) text += ` ${t(`sem_err_${job.error_code}`, job.error)}`;
|
||||
if (["no_api_key", "api_key_invalid", "local_error", "ollama_unreachable", "ollama_model_missing"].includes(job.error_code)) text += ` ${t(`sem_err_${job.error_code}`, job.error)}`;
|
||||
else text += ` ${job.cancelled ? t("semCancelled") : t("semFinished")}`;
|
||||
if (job.failed && job.error) text += ` (${job.error})`;
|
||||
}
|
||||
@@ -477,13 +460,22 @@ function openSemanticModal() {
|
||||
}
|
||||
openModal(t("semModalTitle"), modalHtml());
|
||||
const dimOf = () => parseInt(document.getElementById("wfm-sem-dim").value, 10) || 0;
|
||||
// ローカルモデル選択時のみ実行設定を送る(サーバー側で保存され、検索時のクエリ埋め込みにも使われる)
|
||||
const localOpts = () => selectedModel() === LOCAL ? {
|
||||
device: document.getElementById("wfm-sem-device").value,
|
||||
threads: parseInt(document.getElementById("wfm-sem-threads").value, 10) || 0,
|
||||
soft: parseInt(document.getElementById("wfm-sem-soft").value, 10) || 280,
|
||||
} : {};
|
||||
document.getElementById("wfm-sem-device").addEventListener("change", () => refreshModal());
|
||||
// ローカルモデル選択時のみ Ollama の設定を送る(サーバー側で保存され、検索時のクエリ埋め込みにも使われる)
|
||||
const ollamaOpts = () => ({
|
||||
ollama_url: document.getElementById("wfm-sem-ollama-url").value.trim(),
|
||||
ollama_tag: document.getElementById("wfm-sem-ollama-tag").value,
|
||||
});
|
||||
const localOpts = () => selectedModel() === LOCAL ? ollamaOpts() : {};
|
||||
const saveOllamaOpts = async () => {
|
||||
try {
|
||||
await postJson(API.options, ollamaOpts());
|
||||
} catch (e) {
|
||||
showToast(semanticErrorMessage(e), "error");
|
||||
}
|
||||
refreshModal();
|
||||
};
|
||||
document.getElementById("wfm-sem-ollama-url").addEventListener("change", saveOllamaOpts);
|
||||
document.getElementById("wfm-sem-ollama-tag").addEventListener("change", saveOllamaOpts);
|
||||
document.querySelectorAll('input[name="wfm-sem-model"]').forEach(r => r.addEventListener("change", () => {
|
||||
fillDims(selectedModel(), 0);
|
||||
refreshModal();
|
||||
@@ -507,7 +499,6 @@ function openSemanticModal() {
|
||||
if (mode === "all" && !confirm(t("semConfirmAll"))) return;
|
||||
lsSet(LS_DIM, String(dimOf()));
|
||||
lsSet(LS_MODEL, selectedModel());
|
||||
lsSet(LS_SOFT, document.getElementById("wfm-sem-soft").value);
|
||||
try {
|
||||
await postJson(API.index, {
|
||||
folder: ctx.getFolder(),
|
||||
|
||||
+66
-69
@@ -620,32 +620,31 @@ const LANGUAGES = {
|
||||
semThTextHint: "0 = no filter",
|
||||
semThLimit: "Max results:",
|
||||
semThNote: "Score levels differ by model (in a small test, similar images scored about 0.6-0.8 with EmbeddingGemma 2 and about 0.6 with Gemini, while text queries scored about 0.6-0.7 and about 0.3), so the value is saved per model. The current semantic/similar list is refreshed with the new value when you close this window.",
|
||||
semDevice: "Device:",
|
||||
semDeviceAuto: "Auto (GPU if available)",
|
||||
semDeviceCudaOk: (v) => `GPU (CUDA) detected - onnxruntime ${v}`,
|
||||
semDeviceNoGpu: (v) => `No GPU detected (onnxruntime ${v}, CPU only). To use the GPU, see the help ("?")`,
|
||||
semThreads: "CPU threads:",
|
||||
semThreadsAll: "all",
|
||||
semThreadsHint: "Fewer threads = lighter on the PC while generating images (slower indexing)",
|
||||
semSoft: "Image precision:",
|
||||
semSoftHigh: "High (slowest)",
|
||||
semSoftStd: "Standard (about 2.5x faster)",
|
||||
semSoftLight: "Light (about 5x faster)",
|
||||
semSoftHint: "Changing it requires \"Rebuild all\"",
|
||||
semTestOkProvider: (dim, prov) => `Connection OK (vector size ${dim}, running on ${prov})`,
|
||||
semDotTitle: "Semantic index: all images in this folder are indexed",
|
||||
semHelpTitle: "Open the help for Semantic Search",
|
||||
semModel: "Embedding model:",
|
||||
semModelGemini: "Gemini Embedding 2 (API)",
|
||||
semModelLocal: "EmbeddingGemma 2 (local, free)",
|
||||
semLocalReady: "EmbeddingGemma 2: model downloaded (runs locally, no API key needed)",
|
||||
semLocalNeedDownload: "EmbeddingGemma 2: the model (about 300 MB) will be downloaded on first use",
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2 error: ${m}`,
|
||||
semModelLocal: "EmbeddingGemma 2 (Ollama, local, free)",
|
||||
semOllamaUrl: "Ollama URL:",
|
||||
semOllamaTag: "Model size:",
|
||||
semOllamaTag740: "740m (text + image + audio, 1.3 GB)",
|
||||
semOllamaTag440: "440m (text + image, 671 MB, lighter)",
|
||||
semOllamaTagHint: "Both sizes return identical vectors, so switching does not need \"Rebuild all\". 440m leaves out the audio encoder (not used here) and uses less memory.",
|
||||
semOllamaNotPulled: "(not downloaded)",
|
||||
semOllamaPullHint: (tag) => `Download it with: ollama pull ${tag}`,
|
||||
semOllamaReady: (tag) => `Ollama: ${tag} ready (runs locally, no API key needed)`,
|
||||
semOllamaMissing: (tag) => `Ollama is running but ${tag} is not downloaded — run "ollama pull ${tag}"`,
|
||||
semOllamaUnreachable: (url) => `Cannot reach Ollama at ${url} — start Ollama or check the URL`,
|
||||
sem_err_ollama_unreachable: (m) => `Cannot reach Ollama: ${m}`,
|
||||
sem_err_ollama_model_missing: (m) => `The Ollama model is not downloaded — run "ollama pull ${m}"`,
|
||||
sem_err_bad_url: (m) => `URL not allowed: ${m}`,
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2 (Ollama) error: ${m}`,
|
||||
semScope: "Target folder:",
|
||||
semNoFolder: "Select a folder in the Gallery first",
|
||||
semIncludeSub: "Include subfolders",
|
||||
semDimension: "Vector size:",
|
||||
semDimHint: "Larger = more precise but heavier. Changing the model or size requires \"Rebuild all\".",
|
||||
semDimHint: "Larger = more precise but heavier. Changing the model (Gemini / EmbeddingGemma 2) or size requires \"Rebuild all\".",
|
||||
semMode: "Mode:",
|
||||
semModeMissing: "New / changed images only",
|
||||
semModeAll: "Rebuild all (target folder)",
|
||||
@@ -653,7 +652,7 @@ const LANGUAGES = {
|
||||
semCancelJob: "Stop",
|
||||
semTest: "Test connection",
|
||||
semClear: "Delete all index data",
|
||||
semCostNote: "Gemini Embedding 2 has a free tier, but content sent on the free tier may be used by Google to improve its products; beyond the free tier it is paid (about $0.00012 per image as of Oct 2026). Indexing sends each image (resized) to the API; EmbeddingGemma 2 runs locally on the CPU (about 2-3 s per image). Searches reuse the saved vectors (only the text query is sent). Videos/PSD use a thumbnail; audio is skipped.",
|
||||
semCostNote: "Gemini Embedding 2 has a free tier, but content sent on the free tier may be used by Google to improve its products; beyond the free tier it is paid (about $0.00012 per image as of Oct 2026). Indexing sends each image (resized) to the API; EmbeddingGemma 2 runs locally in Ollama (about 0.1-0.2 s per image with an NVIDIA GPU). Searches reuse the saved vectors (only the text query is sent). Videos/PSD use a thumbnail; audio is skipped.",
|
||||
semKeyOk: "GEMINI_API_KEY: set",
|
||||
semKeyMissing: "GEMINI_API_KEY is not set — add GEMINI_API_KEY=... to the plugin's .env file and restart ComfyUI",
|
||||
semIndexInfo: (n, dim) => `indexed images: ${n} (vector size: ${dim})`,
|
||||
@@ -1082,10 +1081,10 @@ const LANGUAGES = {
|
||||
helpGallery31: "Send to LV node button (toolbar, next to Send to LI node): the video equivalent — uploads the selected video and writes its filename into the file widget of the currently selected node, which must be ComfyUI's native Load Video node; only enabled when the selected item is a video (mp4/webm); requires Workflow Studio to have been opened from ComfyUI's top menu button",
|
||||
helpGallery37: "Send CC Layout / Send CC Image buttons (toolbar, shown only when this Gallery is opened inside ComfyUI Comic Creator; v0.8.11): Send CC Layout sends the selected image to the Layout tab's selected panel / overlay (an MP4 goes to the Video tool); Send CC Image switches to Comic Creator's Image tab and loads the selected image there (as a new layer if a document is open, otherwise as a new canvas; videos are not supported)",
|
||||
helpGallery32: "Detail panel — Prompt tab (selected by default): lists positive/negative prompts found in the embedded workflow; hidden for images in a cc_nanobanana folder, since those carry no embedded ComfyUI workflow — a Nanobanana tab (v0.7.14) appears there instead, shown only when that folder's history.jsonl generation log (written by ComfyUI Comic Creator's Nanobanana tab) has a matching entry for the selected image: engine, model, image size, seed, timestamp, and the prompt/negative prompt actually used",
|
||||
helpGallery33: "Semantic Search (v0.8.2): the S button next to the search box turns semantic search on. When ON, the search box finds images by meaning using Gemini Embedding 2 or the local EmbeddingGemma 2 (e.g. 'sunset city street') - press Enter to search; results are ordered by similarity with a score badge. When OFF it is the normal filename/tag/prompt search. Turning S on also shows a second toolbar row with Similar, the update button, the settings button and the result count. The settings button (gear) opens a window to build/update/delete the index for the selected folder (optionally with subfolders); choose the model with the radio buttons (Gemini needs GEMINI_API_KEY in the plugin's .env + a ComfyUI restart; EmbeddingGemma 2 runs locally and downloads its model on first use). Switching model or vector size needs 'Rebuild all'. Searches cover the selected folder tree; sort and tag/group filters do not apply to semantic results",
|
||||
helpGallery33: "Semantic Search (v0.8.2): the S button next to the search box turns semantic search on. When ON, the search box finds images by meaning using Gemini Embedding 2 or EmbeddingGemma 2 running in Ollama (e.g. 'sunset city street') - press Enter to search; results are ordered by similarity with a score badge. When OFF it is the normal filename/tag/prompt search. Turning S on also shows a second toolbar row with Similar, the update button, the settings button and the result count. The settings button (gear) opens a window to build/update/delete the index for the selected folder (optionally with subfolders); choose the model with the radio buttons (Gemini needs GEMINI_API_KEY in the plugin's .env + a ComfyUI restart; EmbeddingGemma 2 needs Ollama running locally with the model downloaded: ollama pull embeddinggemma-2:740m or embeddinggemma-2:440m). Switching model or vector size needs 'Rebuild all'. Searches cover the selected folder tree; sort and tag/group filters do not apply to semantic results",
|
||||
helpGallery34: "Similar toggle and update button (second toolbar row, shown while S is ON): Similar shows images similar to the selected one (an image not yet indexed is embedded on the fly). Selecting another image keeps the list as is; press the update button to show images similar to the newly selected one, or press Similar again to return to the normal list. Changing folder, Clear filters or turning S off also leaves similar mode",
|
||||
helpGallery35: "EmbeddingGemma 2 performance: by default it runs on the CPU with half of the cores (about 6 s per image at full precision). Image precision trades accuracy for speed (Standard is about 2.5x and Light about 5x faster; vectors differ, so changing it needs 'Rebuild all'); fewer CPU threads keep the PC lighter while generating images. To use an NVIDIA GPU, install onnxruntime-gpu into ComfyUI's Python (pip uninstall onnxruntime, then pip install onnxruntime-gpu; the CUDA runtime/cuDNN that ComfyUI's PyTorch uses is picked up) and restart ComfyUI: indexing then takes about 0.1 s per image (measured on an RTX 4070, about 45x faster than the CPU) and the results are identical to the CPU's. DirectML is not supported (its vision model fails to run). The model is released from memory when indexing finishes and reloaded on the next search",
|
||||
helpGallery36: "Modal tabs and the similarity threshold: the Semantic settings window has three tabs - Index (target folder, rebuild mode, progress, delete all), Model (Gemini / EmbeddingGemma 2, vector size, device, CPU threads, image precision, connection test) and Search. On the Search tab you can set a minimum score for similar-image results (e.g. 0.6 shows only images scoring 0.6 or higher), a minimum score for text search, and the maximum number of results. Scores differ by model, so each model keeps its own values; the list header shows the active threshold",
|
||||
helpGallery35: "EmbeddingGemma 2 runs in Ollama (the built-in ONNX runtime was removed, so onnxruntime-gpu is no longer needed for it). On the Model tab set the Ollama URL (default http://localhost:11434; other hosts must be listed in WFS_ALLOWED_BACKEND_HOSTS) and the model size: 740m (text + image + audio) or 440m (text + image, smaller). Both return identical vectors, so you can switch sizes without rebuilding; the ':latest' tag is the same as 740m and is used when 740m is not downloaded. 270m and 570m cannot embed images and are not offered. Indexes made with the earlier ONNX version keep working (the vectors match closely; use 'Rebuild all' to regenerate them exactly). Images are sent 4 at a time (about 0.1-0.2 s per image with an NVIDIA GPU). The model is unloaded from VRAM when indexing finishes and Ollama reloads it on the next search",
|
||||
helpGallery36: "Modal tabs and the similarity threshold: the Semantic settings window has three tabs - Index (target folder, rebuild mode, progress, delete all), Model (Gemini / EmbeddingGemma 2, vector size, Ollama URL, model size, connection test) and Search. On the Search tab you can set a minimum score for similar-image results (e.g. 0.6 shows only images scoring 0.6 or higher), a minimum score for text search, and the maximum number of results. Scores differ by model, so each model keeps its own values; the list header shows the active threshold",
|
||||
helpGallery28: "Create PSD bulk button: combines all selected images into a single layered PSD file and downloads it — each image becomes its own layer (named after the file), positioned top-left on a canvas sized to the largest selected image; the last selected image ends up on top Convert... bulk button (shown when audio/video files are selected): converts audio files — or extracts the audio track of videos — to FLAC / MP3 / Opus / M4A / WAV (bitrate, sample rate, channels selectable) into the same folder as new files; prompt/workflow tags are carried over for formats that can hold them (WAV cannot), favorite/tags/memo/groups can be carried over, and the originals can optionally be deleted afterwards.",
|
||||
helpGallery29: "Edit as Layers bulk button: switches to the Image Edit tab and opens all selected images at once, each as an independent layer (video files in the selection are skipped); the last selected image ends up on top",
|
||||
helpGallery30: "PSD files: thumbnails and full-size previews show all layers merged together (hidden layers excluded); the underlying PSD file itself is left untouched — only the preview shown in Gallery is a flattened rendering",
|
||||
@@ -1312,8 +1311,8 @@ const LANGUAGES = {
|
||||
helpTrouble7: "Tagger tab shows no models: verify the model folder exists at ComfyUI/models/tagger/<model-name>/ and contains a .onnx or .h5 file; if onnxruntime is not installed, run: python_embeded\\python.exe -m pip install onnxruntime",
|
||||
helpTrouble8: "Model tab shows a checkpoint you didn't expect (e.g. always the same model regardless of the loaded workflow): the checkpoint saved in the workflow file is not installed, so it was silently replaced with another available model during loading — an error toast reports the original (missing) model name when this happens. Select the correct checkpoint manually in the Model tab, or enable \"Default Checkpoint\" in Settings to force a known-good model on every load.",
|
||||
helpTrouble9: "Generate runs a different workflow than the one just loaded: a Batch type checkbox (Checkpoint/LoRA/Prompt/Workflow/Sampler/Scheduler) is left ON in the Batch subtab. While any Batch type is active, the Generate button runs the batch selection instead of the currently loaded workflow. Open the Batch subtab and turn off the active batch type checkbox.",
|
||||
helpTrouble10: "GPU not detected for EmbeddingGemma 2 (Semantic settings > Model says 'No GPU detected', or the connection test reports CPU): the onnxruntime in ComfyUI's Python is the CPU build, so 'CUDAExecutionProvider' is missing. Fix: stop ComfyUI (files in use cannot be replaced), then in ComfyUI's Python run 'pip uninstall -y onnxruntime onnxruntime-gpu' followed by 'pip install --no-deps onnxruntime-gpu', and start ComfyUI again. It can silently return to the CPU build later: other packages that require 'onnxruntime' (e.g. insightface, rtmlib, qwen-tts) make pip install the CPU build over the GPU files whenever they are installed or updated, so if the GPU disappears after adding a custom node, repeat the fix. Requires an NVIDIA GPU with the CUDA runtime/cuDNN that ComfyUI's PyTorch uses; DirectML is not supported. Without a GPU the CPU is used automatically (slower)",
|
||||
helpTrouble11: "Semantic Search with Gemini says 'GEMINI_API_KEY is not set' or the key is rejected: add GEMINI_API_KEY=your-key (no leading #) to the plugin's .env file (copy .env.example), then restart ComfyUI - the .env file is only read at startup. Get a key at Google AI Studio. If you do not want to use the API, choose EmbeddingGemma 2 (local, no key) in Semantic settings > Model",
|
||||
helpTrouble10: "EmbeddingGemma 2 says 'Cannot reach Ollama' or 'not downloaded': start Ollama (it must be running while indexing and searching) and check the Ollama URL in Semantic settings > Model; download the model with 'ollama pull embeddinggemma-2:740m' (or embeddinggemma-2:440m). If indexing fails now and then with 'connection was forcibly closed' errors, security software that filters per-app traffic (e.g. AdGuard) may be interrupting ollama.exe - exclude Ollama from filtering. Failed requests are retried automatically; images that still fail are counted as failed and are picked up again by the next 'New / changed images only' run",
|
||||
helpTrouble11: "Semantic Search with Gemini says 'GEMINI_API_KEY is not set' or the key is rejected: add GEMINI_API_KEY=your-key (no leading #) to the plugin's .env file (copy .env.example), then restart ComfyUI - the .env file is only read at startup. Get a key at Google AI Studio. If you do not want to use the API, choose EmbeddingGemma 2 (Ollama, local, no key) in Semantic settings > Model",
|
||||
helpTrouble12: "Semantic Search or Similar shows no results: results only include images that are already indexed inside the selected folder (a green dot in the folder tree means the folder is fully indexed) - build the index from the gear button first, and select a folder that contains indexed images. After changing the model, vector size or image precision, choose 'Rebuild all'. A minimum score set in Semantic settings > Search can also hide everything; set it to 0 to check. The first search after indexing takes a few seconds because the local model is reloaded",
|
||||
helpSupportTitle: "Support",
|
||||
helpSupportDesc: "If you find this plugin useful, please consider supporting its development!",
|
||||
@@ -2335,32 +2334,31 @@ const LANGUAGES = {
|
||||
semThTextHint: "0=絞り込みなし",
|
||||
semThLimit: "最大表示件数:",
|
||||
semThNote: "スコアの水準はモデルによって違うため（少数画像での実測: 類似画像はEmbeddingGemma 2で約0.6〜0.8、Geminiで約0.6、文章検索はそれぞれ約0.6〜0.7、約0.3）、値はモデルごとに保存されます。閉じたときに、意味検索/類似表示中の一覧へ反映されます。",
|
||||
semDevice: "実行デバイス:",
|
||||
semDeviceAuto: "自動（GPUが使えればGPU）",
|
||||
semDeviceCudaOk: (v) => `GPU (CUDA) を検出 - onnxruntime ${v}`,
|
||||
semDeviceNoGpu: (v) => `GPU未検出（onnxruntime ${v}・CPUのみ）。GPUを使うには「?」のヘルプを参照`,
|
||||
semThreads: "CPUスレッド数:",
|
||||
semThreadsAll: "全コア",
|
||||
semThreadsHint: "少ないほど画像生成中のPC負荷が軽くなります（作成は遅くなります）",
|
||||
semSoft: "画像の精度:",
|
||||
semSoftHigh: "高精度（最も遅い）",
|
||||
semSoftStd: "標準（約2.5倍速）",
|
||||
semSoftLight: "軽量（約5倍速）",
|
||||
semSoftHint: "変更時は「全件再作成」が必要です",
|
||||
semTestOkProvider: (dim, prov) => `接続OK（ベクトル次元 ${dim}、実行: ${prov}）`,
|
||||
semDotTitle: "セマンティック索引: このフォルダの画像は全件作成済み",
|
||||
semHelpTitle: "セマンティック検索のヘルプを開く",
|
||||
semModel: "埋め込みモデル:",
|
||||
semModelGemini: "Gemini Embedding 2（API）",
|
||||
semModelLocal: "EmbeddingGemma 2（ローカル・無料）",
|
||||
semLocalReady: "EmbeddingGemma 2: モデル取得済み（ローカル実行・APIキー不要）",
|
||||
semLocalNeedDownload: "EmbeddingGemma 2: 初回利用時にモデル（約300MB）をダウンロードします",
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2 エラー: ${m}`,
|
||||
semModelLocal: "EmbeddingGemma 2（Ollama・ローカル・無料）",
|
||||
semOllamaUrl: "Ollama URL:",
|
||||
semOllamaTag: "モデルサイズ:",
|
||||
semOllamaTag740: "740m（テキスト＋画像＋音声・1.3GB）",
|
||||
semOllamaTag440: "440m（テキスト＋画像・671MB・軽量）",
|
||||
semOllamaTagHint: "どちらのサイズも同じベクトルを返すため、切り替えても「全件再作成」は不要です。440mは音声エンコーダー（ここでは未使用）を省いた版で、メモリ使用量が少なくなります。",
|
||||
semOllamaNotPulled: "（未取得）",
|
||||
semOllamaPullHint: (tag) => `取得コマンド: ollama pull ${tag}`,
|
||||
semOllamaReady: (tag) => `Ollama: ${tag} 利用可能（ローカル実行・APIキー不要）`,
|
||||
semOllamaMissing: (tag) => `Ollamaは起動していますが ${tag} が未取得です —「ollama pull ${tag}」を実行してください`,
|
||||
semOllamaUnreachable: (url) => `Ollama（${url}）に接続できません — Ollamaを起動するか、URLを確認してください`,
|
||||
sem_err_ollama_unreachable: (m) => `Ollamaに接続できません: ${m}`,
|
||||
sem_err_ollama_model_missing: (m) => `Ollamaのモデルが未取得です —「ollama pull ${m}」を実行してください`,
|
||||
sem_err_bad_url: (m) => `許可されていないURLです: ${m}`,
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2（Ollama）エラー: ${m}`,
|
||||
semScope: "対象フォルダ:",
|
||||
semNoFolder: "先にGalleryでフォルダを選択してください",
|
||||
semIncludeSub: "サブフォルダを含める",
|
||||
semDimension: "ベクトル次元:",
|
||||
semDimHint: "大きいほど高精度ですが重くなります。モデルや次元の変更時は「全件再作成」が必要です。",
|
||||
semDimHint: "大きいほど高精度ですが重くなります。モデル（Gemini / EmbeddingGemma 2）や次元の変更時は「全件再作成」が必要です。",
|
||||
semMode: "モード:",
|
||||
semModeMissing: "新規・変更された画像のみ",
|
||||
semModeAll: "全件再作成（対象フォルダ）",
|
||||
@@ -2368,7 +2366,7 @@ const LANGUAGES = {
|
||||
semCancelJob: "停止",
|
||||
semTest: "接続テスト",
|
||||
semClear: "インデックスを全削除",
|
||||
semCostNote: "Gemini Embedding 2には無料枠がありますが、無料枠で送信した内容はGoogleの製品改善に利用される場合があります。無料枠を超えると有料です（2026年10月現在、画像1枚あたり約$0.00012）。インデックス作成時に各画像(縮小)をAPIへ送信します。EmbeddingGemma 2はローカル(CPU)で実行します(1枚あたり約2〜3秒)。検索は保存済みベクトルを使います(送信されるのは検索文のみ)。動画/PSDはサムネイルを使用、音声は対象外です。",
|
||||
semCostNote: "Gemini Embedding 2には無料枠がありますが、無料枠で送信した内容はGoogleの製品改善に利用される場合があります。無料枠を超えると有料です（2026年10月現在、画像1枚あたり約$0.00012）。インデックス作成時に各画像(縮小)をAPIへ送信します。EmbeddingGemma 2はローカルのOllamaで実行します(NVIDIA GPUで1枚あたり約0.1〜0.2秒)。検索は保存済みベクトルを使います(送信されるのは検索文のみ)。動画/PSDはサムネイルを使用、音声は対象外です。",
|
||||
semKeyOk: "GEMINI_API_KEY: 設定済み",
|
||||
semKeyMissing: "GEMINI_API_KEY が未設定です — プラグインの .env に GEMINI_API_KEY=... を追記し、ComfyUIを再起動してください",
|
||||
semIndexInfo: (n, dim) => `インデックス済み: ${n}枚（ベクトル次元: ${dim}）`,
|
||||
@@ -2797,10 +2795,10 @@ const LANGUAGES = {
|
||||
helpGallery31: "Send to LV nodeボタン（ツールバー、Send to LI nodeボタン横）：動画版のSend to LI node — 選択動画をアップロードし、そのファイル名をComfyUIキャンバス上で選択中のノード（ネイティブのLoad Videoノードである必要がある）のfileウィジェットへ書き込む；選択中の項目が動画(mp4/webm)の時のみ有効；ComfyUIのトップメニューボタンからWorkflow Studioを起動している必要がある",
|
||||
helpGallery37: "Send CC Layout / Send CC Imageボタン（ツールバー、ComfyUI Comic Creator内でこのGalleryを開いた場合のみ表示；v0.8.11）：Send CC Layoutは選択画像をレイアウトタブの選択中のコマ／オーバーレイへ送る（MP4は動画ツールへ）；Send CC ImageはComic CreatorをImageタブへ切り替え、選択画像をそこへ読み込む（作業中のドキュメントがあれば新規レイヤー、なければ新規キャンバス；動画は未対応）",
|
||||
helpGallery32: "詳細パネル — Promptタブ（既定で選択）：埋め込みワークフローから見つかったpositive/negativeプロンプトを一覧表示；cc_nanobananaフォルダの画像はComfyUIワークフローを埋め込んでいないため非表示になり、代わりにNanobananaタブ（v0.7.14）が表示される — 同フォルダのhistory.jsonl生成履歴（ComfyUI Comic CreatorのNanobananaタブが書き込む）に選択画像の対応エントリがある場合のみ表示：engine・model・画像サイズ・seed・日時・実際に使用されたプロンプト/ネガティブプロンプト",
|
||||
helpGallery33: "Semantic Search（v0.8.2）：検索ボックス横の「S」ボタンで意味検索をON/OFFします。ONにすると検索ボックスがGemini Embedding 2またはローカルのEmbeddingGemma 2による意味検索になります（例：「夕暮れの街並み」）。Enterで実行し、結果は類似度順でスコアバッジ付き。OFFの時は従来のファイル名/タグ/プロンプト検索です。ONにするとツールバーに2行目が現れ、Similar・更新ボタン・設定ボタン・結果件数が左詰めで表示されます。設定ボタン（歯車）のウィンドウで、選択中フォルダ（サブフォルダ含む指定可）のインデックスを作成/更新/削除します。モデルはラジオボタンで選択します（Geminiはプラグインの.envにGEMINI_API_KEYの設定＋ComfyUI再起動が必要、EmbeddingGemma 2はローカル実行で初回にモデルを自動ダウンロード）。モデルやベクトル次元を変えるときは「全件再作成」が必要です。検索範囲は選択中フォルダ配下で、意味検索の結果にはソートやタグ/グループフィルタは適用されません",
|
||||
helpGallery33: "Semantic Search（v0.8.2）：検索ボックス横の「S」ボタンで意味検索をON/OFFします。ONにすると検索ボックスがGemini Embedding 2、またはOllamaで動かすEmbeddingGemma 2による意味検索になります（例：「夕暮れの街並み」）。Enterで実行し、結果は類似度順でスコアバッジ付き。OFFの時は従来のファイル名/タグ/プロンプト検索です。ONにするとツールバーに2行目が現れ、Similar・更新ボタン・設定ボタン・結果件数が左詰めで表示されます。設定ボタン（歯車）のウィンドウで、選択中フォルダ（サブフォルダ含む指定可）のインデックスを作成/更新/削除します。モデルはラジオボタンで選択します（Geminiはプラグインの.envにGEMINI_API_KEYの設定＋ComfyUI再起動が必要、EmbeddingGemma 2はローカルでOllamaを起動し、モデルを取得しておく必要があります：ollama pull embeddinggemma-2:740m または embeddinggemma-2:440m）。モデルやベクトル次元を変えるときは「全件再作成」が必要です。検索範囲は選択中フォルダ配下で、意味検索の結果にはソートやタグ/グループフィルタは適用されません",
|
||||
helpGallery34: "Similarトグルと更新ボタン（ツールバー2行目、SがONの間に表示）：Similarで選択中の画像に類似した画像を一覧表示します（未インデックスの画像はその場で埋め込み）。類似一覧の中の別の画像を選択しても一覧はそのままで、更新ボタンを押すとその画像の類似画像を表示、Similarをもう一度押すと通常の一覧に戻ります。フォルダ切替・フィルタクリア・SをOFFにした場合も類似表示は解除されます",
|
||||
helpGallery35: "EmbeddingGemma 2の速度：既定ではCPU（コアの半分）で動作し、高精度設定で1枚あたり約6秒です。「画像の精度」は精度と速度の引き換えで（標準で約2.5倍、軽量で約5倍速。ベクトルが変わるため変更時は「全件再作成」が必要）、CPUスレッド数を減らすと画像生成中のPC負荷が軽くなります。NVIDIA GPUを使うにはComfyUIのPythonにonnxruntime-gpuを入れ（pip uninstall onnxruntime のあと pip install onnxruntime-gpu。ComfyUIのPyTorchが使うCUDAランタイム/cuDNNが利用されます）、ComfyUIを再起動してください：1枚あたり約0.1秒（RTX 4070で実測、CPUの約45倍速）で、結果はCPUと同一です。DirectMLは非対応です（画像モデルが動作しません）。モデルはインデックス作成の完了後にメモリから解放され、次回の検索時に再ロードされます",
|
||||
helpGallery36: "モーダルのタブとしきい値：セマンティック設定ウィンドウは3つのタブ構成です。インデックス（対象フォルダ・再作成モード・進捗・全削除）、モデル設定（Gemini / EmbeddingGemma 2・ベクトル次元・デバイス・CPUスレッド数・画像精度・接続テスト）、検索設定。検索設定タブでは、類似画像結果の最小スコア（例：0.6なら類似度0.6以上のみ表示）、文章検索の最小スコア、最大表示件数を指定できます。スコアの水準はモデルで違うため値はモデルごとに保存され、一覧の見出しに適用中のしきい値が表示されます",
|
||||
helpGallery35: "EmbeddingGemma 2はOllamaで実行します（内蔵のONNX実行は廃止したため、そのためのonnxruntime-gpuは不要になりました）。モデル設定タブでOllama URL（既定はhttp://localhost:11434。他のホストはWFS_ALLOWED_BACKEND_HOSTSへの登録が必要）とモデルサイズを指定します：740m（テキスト＋画像＋音声）または440m（テキスト＋画像・軽量）。どちらも同じベクトルを返すため、サイズを切り替えても再作成は不要です。「:latest」タグは740mと同じもので、740mが未取得なら代わりに使われます。270mと570mは画像を扱えないため選択肢にありません。以前のONNX版で作ったインデックスもそのまま使えます（ベクトルはほぼ一致します。完全に作り直したい場合は「全件再作成」）。画像は4枚ずつまとめて送信します（NVIDIA GPUで1枚あたり約0.1〜0.2秒）。モデルはインデックス作成の完了後にVRAMから外され、次回の検索時にOllamaが再ロードします",
|
||||
helpGallery36: "モーダルのタブとしきい値：セマンティック設定ウィンドウは3つのタブ構成です。インデックス（対象フォルダ・再作成モード・進捗・全削除）、モデル設定（Gemini / EmbeddingGemma 2・ベクトル次元・Ollama URL・モデルサイズ・接続テスト）、検索設定。検索設定タブでは、類似画像結果の最小スコア（例：0.6なら類似度0.6以上のみ表示）、文章検索の最小スコア、最大表示件数を指定できます。スコアの水準はモデルで違うため値はモデルごとに保存され、一覧の見出しに適用中のしきい値が表示されます",
|
||||
helpGallery28: "「PSD作成」一括操作ボタン：選択中の全画像を1つのレイヤー付きPSDファイルにまとめてダウンロードする — 各画像はファイル名を冠した独立レイヤーとなり、選択画像中の最大サイズに合わせたキャンバスに左上基準で配置される；最後に選択した画像が最前面になる 変換...一括ボタン（音声/動画を選択すると表示）：音声ファイル（動画は音声トラックを抽出）をFLAC / MP3 / Opus / M4A / WAVへ変換して同じフォルダに別ファイルで保存します（ビットレート・サンプルレート・チャンネル指定可）。prompt/workflowタグはタグを持てる形式で引き継がれ（WAVは対象外）、お気に入り・タグ・メモ・グループの引き継ぎや、変換後の元ファイル削除も選べます。",
|
||||
helpGallery29: "「レイヤーとして編集」一括操作ボタン：Image Editタブに切り替え、選択中の全画像を一度に開き、それぞれ独立したレイヤーとして展開する（選択中に動画ファイルが含まれる場合はスキップされる）；最後に選択した画像が最前面になる",
|
||||
helpGallery30: "PSDファイル：サムネイル・フルサイズプレビューは全レイヤーを合成した結果を表示する（非表示レイヤーは除外される）；元のPSDファイル自体は変更されず、Gallery上のプレビューのみが合成済みの表示になる",
|
||||
@@ -3027,8 +3025,8 @@ const LANGUAGES = {
|
||||
helpTrouble7: "Taggerタブにモデルが表示されない：ComfyUI/models/tagger/<モデル名>/ フォルダの存在と .onnx または .h5 ファイルの有無を確認；onnxruntimeが未インストールの場合は python_embeded\\python.exe -m pip install onnxruntime を実行",
|
||||
helpTrouble8: "ModelタブのCheckpointが想定と違うモデルになっている（どのワークフローを読み込んでも同じモデルになる等）：ワークフローファイルに保存されたCheckpointが実際にはインストールされておらず、読み込み時に無警告で別のインストール済みモデルに差し替えられています。この場合、元の（見つからなかった）モデル名を含むエラートーストが表示されます。Modelタブで正しいCheckpointを手動選択するか、設定タブの「デフォルトCheckpoint」を有効にして、読み込み時に常に動作確認済みのモデルを強制適用してください。",
|
||||
helpTrouble9: "Generateを押すと、読み込んだワークフローとは別のワークフローで実行される：Batchサブタブでバッチタイプ（Checkpoint/LoRA/Prompt/Workflow/Sampler/Scheduler）のチェックボックスがONのまま残っています。いずれかのバッチタイプが有効な間は、Generateボタンは現在読み込まれているワークフローではなくバッチ選択された内容を実行します。Batchサブタブを開き、有効になっているバッチタイプのチェックボックスをOFFにしてください。",
|
||||
helpTrouble10: "EmbeddingGemma 2でGPUが検出されない（セマンティック設定のモデル設定タブに「GPU未検出」と出る、または接続テストの実行がCPUになる）：ComfyUIのPythonに入っているonnxruntimeがCPU版で、CUDAExecutionProviderが無い状態です。対処：ComfyUIを停止し（使用中のファイルは置き換えられません）、ComfyUIのPythonで「pip uninstall -y onnxruntime onnxruntime-gpu」のあと「pip install --no-deps onnxruntime-gpu」を実行して、ComfyUIを起動し直します。後からCPU版に戻ることがあります：「onnxruntime」を名前で要求する他のパッケージ（insightface・rtmlib・qwen-ttsなど）をインストール・更新するたびに、pipがGPU版のファイルの上にCPU版を入れ直すためです。カスタムノードを追加した後にGPUが使えなくなったら、同じ手順をやり直してください。NVIDIA GPUと、ComfyUIのPyTorchが使うCUDAランタイム/cuDNNが必要です（DirectMLは非対応）。GPUが無い場合は自動でCPUが使われます（低速）",
|
||||
helpTrouble11: "GeminiでSemantic Searchを使うと「GEMINI_API_KEYが未設定」と出る、またはキーが拒否される：プラグインの.env（.env.exampleをコピー）にGEMINI_API_KEY=キー（行頭に#を付けない）を追記し、ComfyUIを再起動してください。.envは起動時にしか読み込まれません。キーはGoogle AI Studioで取得できます。APIを使いたくない場合は、セマンティック設定のモデル設定でEmbeddingGemma 2（ローカル・キー不要）を選びます",
|
||||
helpTrouble10: "EmbeddingGemma 2で「Ollamaに接続できません」「未取得です」と出る：Ollamaを起動し（インデックス作成中と検索時に起動している必要があります）、セマンティック設定のモデル設定タブでOllama URLを確認してください。モデルは「ollama pull embeddinggemma-2:740m」（または embeddinggemma-2:440m）で取得します。インデックス作成が「connection was forcibly closed」などで時々失敗する場合は、アプリ単位で通信をフィルタするセキュリティソフト（AdGuardなど）がollama.exeの通信に割り込んでいる可能性があります。Ollamaをフィルタ対象から外してください。失敗したリクエストは自動で再試行され、それでも失敗した画像は失敗件数に数えられ、次回の「新規・変更された画像のみ」で再処理されます",
|
||||
helpTrouble11: "GeminiでSemantic Searchを使うと「GEMINI_API_KEYが未設定」と出る、またはキーが拒否される：プラグインの.env（.env.exampleをコピー）にGEMINI_API_KEY=キー（行頭に#を付けない）を追記し、ComfyUIを再起動してください。.envは起動時にしか読み込まれません。キーはGoogle AI Studioで取得できます。APIを使いたくない場合は、セマンティック設定のモデル設定でEmbeddingGemma 2（Ollama・ローカル・キー不要）を選びます",
|
||||
helpTrouble12: "意味検索やSimilarの結果が出ない：結果には選択中フォルダ内でインデックス済みの画像だけが入ります（フォルダツリーの緑のドットは全件済み）。先に歯車ボタンからインデックスを作成し、インデックス済みの画像があるフォルダを選んでください。モデル・ベクトル次元・画像精度を変えたときは「全件再作成」が必要です。セマンティック設定の検索設定タブの最小スコアで全件が隠れている場合もあるので、0にして確認してください。インデックス作成直後の最初の検索は、ローカルモデルの再ロードで数秒かかります",
|
||||
helpSupportTitle: "サポート",
|
||||
helpSupportDesc: "このプラグインが役に立ったら、開発を支援していただけると嬉しいです！",
|
||||
@@ -4050,32 +4048,31 @@ const LANGUAGES = {
|
||||
semThTextHint: "0=不过滤",
|
||||
semThLimit: "最大显示数量：",
|
||||
semThNote: "不同模型的分数水平不同（少量图像的实测：相似图像在EmbeddingGemma 2约0.6-0.8，Gemini约0.6；文字搜索分别约0.6-0.7和约0.3），因此按模型分别保存。关闭窗口时会应用到正在显示的语义搜索/相似列表。",
|
||||
semDevice: "运行设备：",
|
||||
semDeviceAuto: "自动（可用则用GPU）",
|
||||
semDeviceCudaOk: (v) => `检测到GPU (CUDA) - onnxruntime ${v}`,
|
||||
semDeviceNoGpu: (v) => `未检测到GPU（onnxruntime ${v}，仅CPU）。使用GPU请参见“?”帮助`,
|
||||
semThreads: "CPU线程数：",
|
||||
semThreadsAll: "全部核心",
|
||||
semThreadsHint: "线程越少，生成图像时对电脑的负载越轻（创建会变慢）",
|
||||
semSoft: "图像精度：",
|
||||
semSoftHigh: "高精度（最慢）",
|
||||
semSoftStd: "标准（约快2.5倍）",
|
||||
semSoftLight: "轻量（约快5倍）",
|
||||
semSoftHint: "更改后需要“全部重建”",
|
||||
semTestOkProvider: (dim, prov) => `连接正常（向量维度 ${dim}，运行于 ${prov}）`,
|
||||
semDotTitle: "语义索引：此文件夹中的图像已全部创建",
|
||||
semHelpTitle: "打开语义搜索的帮助",
|
||||
semModel: "嵌入模型：",
|
||||
semModelGemini: "Gemini Embedding 2（API）",
|
||||
semModelLocal: "EmbeddingGemma 2（本地・免费）",
|
||||
semLocalReady: "EmbeddingGemma 2：模型已下载（本地运行，无需API密钥）",
|
||||
semLocalNeedDownload: "EmbeddingGemma 2：首次使用时将下载模型（约300MB）",
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2错误：${m}`,
|
||||
semModelLocal: "EmbeddingGemma 2（Ollama・本地・免费）",
|
||||
semOllamaUrl: "Ollama URL：",
|
||||
semOllamaTag: "模型大小：",
|
||||
semOllamaTag740: "740m（文本＋图像＋音频・1.3GB）",
|
||||
semOllamaTag440: "440m（文本＋图像・671MB・轻量）",
|
||||
semOllamaTagHint: "两种大小返回相同的向量，因此切换后无需“全部重建”。440m去掉了音频编码器（此处未使用），占用内存更少。",
|
||||
semOllamaNotPulled: "（未下载）",
|
||||
semOllamaPullHint: (tag) => `下载命令：ollama pull ${tag}`,
|
||||
semOllamaReady: (tag) => `Ollama：${tag} 可用（本地运行，无需API密钥）`,
|
||||
semOllamaMissing: (tag) => `Ollama已启动，但尚未下载 ${tag} — 请执行“ollama pull ${tag}”`,
|
||||
semOllamaUnreachable: (url) => `无法连接Ollama（${url}）— 请启动Ollama或检查URL`,
|
||||
sem_err_ollama_unreachable: (m) => `无法连接Ollama：${m}`,
|
||||
sem_err_ollama_model_missing: (m) => `尚未下载Ollama模型 — 请执行“ollama pull ${m}”`,
|
||||
sem_err_bad_url: (m) => `不允许的URL：${m}`,
|
||||
sem_err_local_error: (m) => `EmbeddingGemma 2（Ollama）错误：${m}`,
|
||||
semScope: "目标文件夹：",
|
||||
semNoFolder: "请先在Gallery中选择文件夹",
|
||||
semIncludeSub: "包含子文件夹",
|
||||
semDimension: "向量维度：",
|
||||
semDimHint: "越大越精确但越重。更改模型或维度后需要“全部重建”。",
|
||||
semDimHint: "越大越精确但越重。更改模型（Gemini / EmbeddingGemma 2）或维度后需要“全部重建”。",
|
||||
semMode: "模式：",
|
||||
semModeMissing: "仅新增/已更改的图像",
|
||||
semModeAll: "全部重建（目标文件夹）",
|
||||
@@ -4083,7 +4080,7 @@ const LANGUAGES = {
|
||||
semCancelJob: "停止",
|
||||
semTest: "连接测试",
|
||||
semClear: "删除全部索引",
|
||||
semCostNote: "Gemini Embedding 2有免费额度，但免费额度内发送的内容可能被Google用于改进其产品；超出免费额度后需付费（截至2026年10月，每张图像约$0.00012）。创建索引时会将每张图像（缩小后）发送到API；EmbeddingGemma 2在本地CPU运行（每张约2-3秒）。搜索使用已保存的向量（仅发送搜索文字）。视频/PSD使用缩略图，音频不在范围内。",
|
||||
semCostNote: "Gemini Embedding 2有免费额度，但免费额度内发送的内容可能被Google用于改进其产品；超出免费额度后需付费（截至2026年10月，每张图像约$0.00012）。创建索引时会将每张图像（缩小后）发送到API；EmbeddingGemma 2在本地Ollama中运行（使用NVIDIA GPU时每张约0.1-0.2秒）。搜索使用已保存的向量（仅发送搜索文字）。视频/PSD使用缩略图，音频不在范围内。",
|
||||
semKeyOk: "GEMINI_API_KEY：已设置",
|
||||
semKeyMissing: "未设置GEMINI_API_KEY — 请在插件的 .env 中添加 GEMINI_API_KEY=... 并重启ComfyUI",
|
||||
semIndexInfo: (n, dim) => `已索引：${n}张（向量维度：${dim}）`,
|
||||
@@ -4512,10 +4509,10 @@ const LANGUAGES = {
|
||||
helpGallery31: "Send to LV node按钮（工具栏，Send to LI node按钮旁）：视频版的Send to LI node — 上传选中的视频并将其文件名写入ComfyUI画布上当前选中节点（必须是原生的Load Video节点）的file部件；仅当选中项为视频(mp4/webm)时可用；需要通过ComfyUI顶部菜单按钮打开Workflow Studio",
|
||||
helpGallery37: "Send CC Layout / Send CC Image按钮（工具栏，仅在ComfyUI Comic Creator内打开此图库时显示；v0.8.11）：Send CC Layout将选中的图像发送到排版标签页中选中的分格/覆盖层（MP4发送到视频工具）；Send CC Image切换到Comic Creator的Image标签页并在其中加载选中的图像（已有文档时作为新图层，否则作为新画布；不支持视频）",
|
||||
helpGallery32: "详情面板 — Prompt标签（默认选中）：列出从嵌入工作流中找到的正/负向提示词；cc_nanobanana文件夹中的图像未嵌入ComfyUI工作流，因此该标签会被隐藏，取而代之显示Nanobanana标签（v0.7.14）— 仅当该文件夹的history.jsonl生成记录（由ComfyUI Comic Creator的Nanobanana标签写入）中有选中图像的对应记录时才显示：engine、model、图像尺寸、seed、时间戳以及实际使用的提示词/负向提示词",
|
||||
helpGallery33: "Semantic Search（v0.8.2）：搜索框旁的“S”按钮用于开启/关闭语义搜索。开启后，搜索框使用Gemini Embedding 2或本地EmbeddingGemma 2按含义查找图像（例如“黄昏的城市街道”），按Enter执行，结果按相似度排序并显示分数标记。关闭时为原有的文件名/标签/提示词搜索。开启后工具栏会出现第二行，左对齐显示Similar、更新按钮、设置按钮和结果数量。设置按钮（齿轮）会打开窗口，用于创建/更新/删除所选文件夹（可含子文件夹）的索引；模型通过单选按钮选择（Gemini需在插件的.env中设置GEMINI_API_KEY并重启ComfyUI；EmbeddingGemma 2在本地运行，首次使用时自动下载模型）。更改模型或向量维度时需要“全部重建”。搜索范围为所选文件夹及其下级，语义搜索结果不应用排序和标签/分组筛选",
|
||||
helpGallery33: "Semantic Search（v0.8.2）：搜索框旁的“S”按钮用于开启/关闭语义搜索。开启后，搜索框使用Gemini Embedding 2或在Ollama中运行的EmbeddingGemma 2按含义查找图像（例如“黄昏的城市街道”），按Enter执行，结果按相似度排序并显示分数标记。关闭时为原有的文件名/标签/提示词搜索。开启后工具栏会出现第二行，左对齐显示Similar、更新按钮、设置按钮和结果数量。设置按钮（齿轮）会打开窗口，用于创建/更新/删除所选文件夹（可含子文件夹）的索引；模型通过单选按钮选择（Gemini需在插件的.env中设置GEMINI_API_KEY并重启ComfyUI；EmbeddingGemma 2需要在本地运行Ollama并下载模型：ollama pull embeddinggemma-2:740m 或 embeddinggemma-2:440m）。更改模型或向量维度时需要“全部重建”。搜索范围为所选文件夹及其下级，语义搜索结果不应用排序和标签/分组筛选",
|
||||
helpGallery34: "Similar切换按钮和更新按钮（工具栏第二行，S开启时显示）：Similar显示与所选图像相似的图像（未索引的图像会即时嵌入）。在相似列表中选择其他图像时列表保持不变，按更新按钮显示与新选图像相似的图像，再次按Similar则返回普通列表。切换文件夹、清除筛选或关闭S也会退出相似模式",
|
||||
helpGallery35: "EmbeddingGemma 2的速度：默认在CPU（一半核心）上运行，高精度设置下每张约6秒。“图像精度”是在精度与速度之间取舍（标准约快2.5倍，轻量约快5倍；向量会改变，因此更改后需要“全部重建”），减少CPU线程数可降低生成图像时对电脑的负载。要使用NVIDIA GPU，请在ComfyUI的Python中安装onnxruntime-gpu（先pip uninstall onnxruntime，再pip install onnxruntime-gpu；会使用ComfyUI的PyTorch所用的CUDA运行库/cuDNN）并重启ComfyUI：每张约0.1秒（在RTX 4070上实测，约为CPU的45倍），结果与CPU相同。不支持DirectML（其图像模型无法运行）。索引创建完成后模型会从内存中释放，下次搜索时重新加载",
|
||||
helpGallery36: "弹窗标签页与相似度阈值：语义设置窗口分为三个标签页——索引（目标文件夹、重建模式、进度、全部删除）、模型设置（Gemini / EmbeddingGemma 2、向量维度、设备、CPU线程数、图像精度、连接测试）和搜索设置。在搜索设置中可指定相似图像结果的最低分数（例如0.6仅显示相似度0.6以上的图像）、文字搜索的最低分数以及最大显示数量。不同模型的分数水平不同，因此每个模型分别保存，列表标题会显示当前应用的阈值",
|
||||
helpGallery35: "EmbeddingGemma 2在Ollama中运行（已移除内置的ONNX运行方式，因此不再需要为其安装onnxruntime-gpu）。在模型设置标签中指定Ollama URL（默认http://localhost:11434；其他主机需登记到WFS_ALLOWED_BACKEND_HOSTS）和模型大小：740m（文本＋图像＋音频）或440m（文本＋图像，更小）。两者返回相同的向量，因此切换大小无需重建；“:latest”标签与740m相同，未下载740m时会自动使用。270m和570m无法处理图像，因此不在选项中。用旧版ONNX创建的索引可以继续使用（向量几乎一致；如需完全重新生成请使用“全部重建”）。图像每4张一起发送（使用NVIDIA GPU时每张约0.1-0.2秒）。索引创建完成后模型会从显存中卸载，下次搜索时由Ollama重新加载",
|
||||
helpGallery36: "弹窗标签页与相似度阈值：语义设置窗口分为三个标签页——索引（目标文件夹、重建模式、进度、全部删除）、模型设置（Gemini / EmbeddingGemma 2、向量维度、Ollama URL、模型大小、连接测试）和搜索设置。在搜索设置中可指定相似图像结果的最低分数（例如0.6仅显示相似度0.6以上的图像）、文字搜索的最低分数以及最大显示数量。不同模型的分数水平不同，因此每个模型分别保存，列表标题会显示当前应用的阈值",
|
||||
helpGallery28: "「创建PSD」批量操作按钮：将所有选中的图像合并为一个带图层的PSD文件并下载 — 每张图像成为独立图层（以文件名命名），以左上对齐方式放置在按所选最大图像尺寸创建的画布上；最后选中的图像位于最上层 转换...批量按钮（选中音频/视频时显示）：将音频文件（视频则提取音轨）转换为FLAC / MP3 / Opus / M4A / WAV并保存为同文件夹内的新文件（可选比特率、采样率、声道）；prompt/workflow标签会在支持标签的格式中继承（WAV除外），也可继承收藏/标签/备注/分组，并可选择转换后删除原文件。",
|
||||
helpGallery29: "「作为图层编辑」批量操作按钮：切换到Image Edit标签并一次性打开所有选中的图像，每张作为独立图层（所选内容中的视频文件会被跳过）；最后选中的图像位于最上层",
|
||||
helpGallery30: "PSD文件：缩略图与完整预览显示所有图层合并后的效果（隐藏图层不参与合并）；原始PSD文件本身不会被修改，Gallery中显示的仅是合并渲染后的预览",
|
||||
@@ -4742,8 +4739,8 @@ const LANGUAGES = {
|
||||
helpTrouble7: "Tagger标签不显示模型：确认ComfyUI/models/tagger/<模型名>/文件夹存在且包含.onnx或.h5文件；如未安装onnxruntime，请运行：python_embeded\\python.exe -m pip install onnxruntime",
|
||||
helpTrouble8: "Model 标签页显示的 Checkpoint 与预期不符（例如无论加载哪个工作流都显示相同模型）：工作流文件中保存的 Checkpoint 实际未安装，加载时已被静默替换为其他已安装模型，此时会弹出包含原始（未找到）模型名称的错误提示。请在 Model 标签页手动选择正确的 Checkpoint，或在设置标签页启用\"默认 Checkpoint\"，在每次加载时强制应用一个已验证可用的模型。",
|
||||
helpTrouble9: "点击 Generate 后，执行的是与刚加载的工作流不同的工作流：Batch 子标签页中的某个批量类型（Checkpoint/LoRA/Prompt/Workflow/Sampler/Scheduler）复选框仍处于开启状态。只要有任意批量类型处于启用状态，Generate 按钮执行的就是批量选择的内容，而不是当前加载的工作流。请打开 Batch 子标签页，关闭已启用的批量类型复选框。",
|
||||
helpTrouble10: "EmbeddingGemma 2未检测到GPU（语义设置的模型设置标签显示“未检测到GPU”，或连接测试显示在CPU上运行）：ComfyUI的Python中的onnxruntime是CPU版，缺少CUDAExecutionProvider。解决：先停止ComfyUI（正在使用的文件无法替换），然后在ComfyUI的Python中执行“pip uninstall -y onnxruntime onnxruntime-gpu”，再执行“pip install --no-deps onnxruntime-gpu”，之后重新启动ComfyUI。之后它可能悄悄变回CPU版：按名称要求“onnxruntime”的其他软件包（如insightface、rtmlib、qwen-tts）在安装或更新时，pip会在GPU版文件上覆盖安装CPU版；如果添加自定义节点后GPU失效，请重复上述步骤。需要NVIDIA GPU以及ComfyUI的PyTorch所使用的CUDA运行库/cuDNN（不支持DirectML）。没有GPU时会自动使用CPU（较慢）",
|
||||
helpTrouble11: "使用Gemini的语义搜索提示“未设置GEMINI_API_KEY”或密钥被拒绝：在插件的.env文件（复制.env.example）中添加GEMINI_API_KEY=密钥（行首不要加#），然后重启ComfyUI——.env只在启动时读取。密钥可在Google AI Studio获取。如果不想使用API，请在语义设置的模型设置中选择EmbeddingGemma 2（本地，无需密钥）",
|
||||
helpTrouble10: "EmbeddingGemma 2提示“无法连接Ollama”或“尚未下载”：请启动Ollama（创建索引和搜索时都需要运行），并在语义设置的模型设置标签中确认Ollama URL；用“ollama pull embeddinggemma-2:740m”（或 embeddinggemma-2:440m）下载模型。如果创建索引时偶尔出现“connection was forcibly closed”等错误而失败，可能是按应用过滤流量的安全软件（如AdGuard）干扰了ollama.exe的通信，请将Ollama排除在过滤之外。失败的请求会自动重试；仍然失败的图像计入失败数，下次执行“仅新增/已更改的图像”时会重新处理",
|
||||
helpTrouble11: "使用Gemini的语义搜索提示“未设置GEMINI_API_KEY”或密钥被拒绝：在插件的.env文件（复制.env.example）中添加GEMINI_API_KEY=密钥（行首不要加#），然后重启ComfyUI——.env只在启动时读取。密钥可在Google AI Studio获取。如果不想使用API，请在语义设置的模型设置中选择EmbeddingGemma 2（Ollama・本地，无需密钥）",
|
||||
helpTrouble12: "语义搜索或Similar没有结果：结果只包含所选文件夹内已建立索引的图像（文件夹树中的绿点表示该文件夹已全部建立索引）。请先通过齿轮按钮创建索引，并选择包含已索引图像的文件夹。更改模型、向量维度或图像精度后需要“全部重建”。语义设置搜索标签中的最低分数也可能隐藏全部结果，请设为0以确认。创建索引后的首次搜索需要几秒钟，因为要重新加载本地模型",
|
||||
helpSupportTitle: "支持",
|
||||
helpSupportDesc: "如果您觉得这个插件有用，请考虑支持它的开发！",
|
||||
|
||||
@@ -1817,10 +1817,10 @@
|
||||
<li id="wfm-help-gallery-37"><b>Send CC Layout / Send CC Image</b> buttons (toolbar, shown only when this Gallery is opened inside ComfyUI Comic Creator; v0.8.11): Send CC Layout sends the selected image to the Layout tab's selected panel / overlay (an MP4 goes to the Video tool); Send CC Image switches to Comic Creator's Image tab and loads the selected image there (as a new layer if a document is open, otherwise as a new canvas; videos are not supported)</li>
|
||||
<li id="wfm-help-gallery-28"><b>Create PSD</b> bulk button: combines all selected images into a single layered PSD file and downloads it — each image becomes its own layer (named after the file), positioned top-left on a canvas sized to the largest selected image; the last selected image ends up on top Convert... bulk button (shown when audio/video files are selected): converts audio files — or extracts the audio track of videos — to FLAC / MP3 / Opus / M4A / WAV (bitrate, sample rate, channels selectable) into the same folder as new files; prompt/workflow tags are carried over for formats that can hold them (WAV cannot), favorite/tags/memo/groups can be carried over, and the originals can optionally be deleted afterwards.</li>
|
||||
<li id="wfm-help-gallery-29"><b>Edit as Layers</b> bulk button: switches to the Image Edit tab and opens all selected images at once, each as an independent layer (video files in the selection are skipped); the last selected image ends up on top</li>
|
||||
<li id="wfm-help-gallery-33">Semantic Search (v0.8.2): the S button next to the search box turns semantic search on. When ON, the search box finds images by meaning using Gemini Embedding 2 or the local EmbeddingGemma 2 (e.g. 'sunset city street') - press Enter to search; results are ordered by similarity with a score badge. When OFF it is the normal filename/tag/prompt search. Turning S on also shows a second toolbar row with Similar, the update button, the settings button and the result count. The settings button (gear) opens a window to build/update/delete the index for the selected folder (optionally with subfolders); choose the model with the radio buttons (Gemini needs GEMINI_API_KEY in the plugin's .env + a ComfyUI restart; EmbeddingGemma 2 runs locally and downloads its model on first use). Switching model or vector size needs 'Rebuild all'. Searches cover the selected folder tree; sort and tag/group filters do not apply to semantic results</li>
|
||||
<li id="wfm-help-gallery-33">Semantic Search (v0.8.2): the S button next to the search box turns semantic search on. When ON, the search box finds images by meaning using Gemini Embedding 2 or EmbeddingGemma 2 running in Ollama (e.g. 'sunset city street') - press Enter to search; results are ordered by similarity with a score badge. When OFF it is the normal filename/tag/prompt search. Turning S on also shows a second toolbar row with Similar, the update button, the settings button and the result count. The settings button (gear) opens a window to build/update/delete the index for the selected folder (optionally with subfolders); choose the model with the radio buttons (Gemini needs GEMINI_API_KEY in the plugin's .env + a ComfyUI restart; EmbeddingGemma 2 needs Ollama running locally with the model downloaded: ollama pull embeddinggemma-2:740m or embeddinggemma-2:440m). Switching model or vector size needs 'Rebuild all'. Searches cover the selected folder tree; sort and tag/group filters do not apply to semantic results</li>
|
||||
<li id="wfm-help-gallery-34">Similar toggle and update button (second toolbar row, shown while S is ON): Similar shows images similar to the selected one (an image not yet indexed is embedded on the fly). Selecting another image keeps the list as is; press the update button to show images similar to the newly selected one, or press Similar again to return to the normal list. Changing folder, Clear filters or turning S off also leaves similar mode</li>
|
||||
<li id="wfm-help-gallery-35">EmbeddingGemma 2 performance: by default it runs on the CPU with half of the cores (about 6 s per image at full precision). Image precision trades accuracy for speed (Standard is about 2.5x and Light about 5x faster; vectors differ, so changing it needs 'Rebuild all'); fewer CPU threads keep the PC lighter while generating images. To use an NVIDIA GPU, install onnxruntime-gpu into ComfyUI's Python (pip uninstall onnxruntime, then pip install onnxruntime-gpu; the CUDA runtime/cuDNN that ComfyUI's PyTorch uses is picked up) and restart ComfyUI: indexing then takes about 0.1 s per image (measured on an RTX 4070, about 45x faster than the CPU) and the results are identical to the CPU's. DirectML is not supported (its vision model fails to run). The model is released from memory when indexing finishes and reloaded on the next search</li>
|
||||
<li id="wfm-help-gallery-36">Modal tabs and the similarity threshold: the Semantic settings window has three tabs - Index (target folder, rebuild mode, progress, delete all), Model (Gemini / EmbeddingGemma 2, vector size, device, CPU threads, image precision, connection test) and Search. On the Search tab you can set a minimum score for similar-image results (e.g. 0.6 shows only images scoring 0.6 or higher), a minimum score for text search, and the maximum number of results. Scores differ by model, so each model keeps its own values; the list header shows the active threshold</li>
|
||||
<li id="wfm-help-gallery-35">EmbeddingGemma 2 runs in Ollama (the built-in ONNX runtime was removed, so onnxruntime-gpu is no longer needed for it). On the Model tab set the Ollama URL (default http://localhost:11434; other hosts must be listed in WFS_ALLOWED_BACKEND_HOSTS) and the model size: 740m (text + image + audio) or 440m (text + image, smaller). Both return identical vectors, so you can switch sizes without rebuilding; the ':latest' tag is the same as 740m and is used when 740m is not downloaded. 270m and 570m cannot embed images and are not offered. Indexes made with the earlier ONNX version keep working (the vectors match closely; use 'Rebuild all' to regenerate them exactly). Images are sent 4 at a time (about 0.1-0.2 s per image with an NVIDIA GPU). The model is unloaded from VRAM when indexing finishes and Ollama reloads it on the next search</li>
|
||||
<li id="wfm-help-gallery-36">Modal tabs and the similarity threshold: the Semantic settings window has three tabs - Index (target folder, rebuild mode, progress, delete all), Model (Gemini / EmbeddingGemma 2, vector size, Ollama URL, model size, connection test) and Search. On the Search tab you can set a minimum score for similar-image results (e.g. 0.6 shows only images scoring 0.6 or higher), a minimum score for text search, and the maximum number of results. Scores differ by model, so each model keeps its own values; the list header shows the active threshold</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="wfm-help-card">
|
||||
@@ -2217,8 +2217,8 @@
|
||||
<div class="wfm-help-card">
|
||||
<h3 id="wfm-help-trouble-card-semantic-title">Semantic Search (Gallery)</h3>
|
||||
<ul>
|
||||
<li id="wfm-help-trouble-10">GPU not detected for EmbeddingGemma 2 (Semantic settings > Model says 'No GPU detected', or the connection test reports CPU): the onnxruntime in ComfyUI's Python is the CPU build, so 'CUDAExecutionProvider' is missing. Fix: stop ComfyUI (files in use cannot be replaced), then in ComfyUI's Python run 'pip uninstall -y onnxruntime onnxruntime-gpu' followed by 'pip install --no-deps onnxruntime-gpu', and start ComfyUI again. It can silently return to the CPU build later: other packages that require 'onnxruntime' (e.g. insightface, rtmlib, qwen-tts) make pip install the CPU build over the GPU files whenever they are installed or updated, so if the GPU disappears after adding a custom node, repeat the fix. Requires an NVIDIA GPU with the CUDA runtime/cuDNN that ComfyUI's PyTorch uses; DirectML is not supported. Without a GPU the CPU is used automatically (slower)</li>
|
||||
<li id="wfm-help-trouble-11">Semantic Search with Gemini says 'GEMINI_API_KEY is not set' or the key is rejected: add GEMINI_API_KEY=your-key (no leading #) to the plugin's .env file (copy .env.example), then restart ComfyUI - the .env file is only read at startup. Get a key at Google AI Studio. If you do not want to use the API, choose EmbeddingGemma 2 (local, no key) in Semantic settings > Model</li>
|
||||
<li id="wfm-help-trouble-10">EmbeddingGemma 2 says 'Cannot reach Ollama' or 'not downloaded': start Ollama (it must be running while indexing and searching) and check the Ollama URL in Semantic settings > Model; download the model with 'ollama pull embeddinggemma-2:740m' (or embeddinggemma-2:440m). If indexing fails now and then with 'connection was forcibly closed' errors, security software that filters per-app traffic (e.g. AdGuard) may be interrupting ollama.exe - exclude Ollama from filtering. Failed requests are retried automatically; images that still fail are counted as failed and are picked up again by the next 'New / changed images only' run</li>
|
||||
<li id="wfm-help-trouble-11">Semantic Search with Gemini says 'GEMINI_API_KEY is not set' or the key is rejected: add GEMINI_API_KEY=your-key (no leading #) to the plugin's .env file (copy .env.example), then restart ComfyUI - the .env file is only read at startup. Get a key at Google AI Studio. If you do not want to use the API, choose EmbeddingGemma 2 (Ollama, local, no key) in Semantic settings > Model</li>
|
||||
<li id="wfm-help-trouble-12">Semantic Search or Similar shows no results: results only include images that are already indexed inside the selected folder (a green dot in the folder tree means the folder is fully indexed) - build the index from the gear button first, and select a folder that contains indexed images. After changing the model, vector size or image precision, choose 'Rebuild all'. A minimum score set in Semantic settings > Search can also hide everything; set it to 0 to check. The first search after indexing takes a few seconds because the local model is reloaded</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user