feat(scan): list unmatched and unscanned models, show each scan's result, explain why

The scan page now says which models are unmatched or not scanned yet (type, folder,
why Civitai had nothing) and what each scan did, model by model; rows open the model,
scan it, look it up again or search Civitai. The progress box shows the counts and,
when floating, a way to the result. The activity log records a scan when it ends with
its result, and model edits (from -> to) and deletions (files moved) in detail.

- scraper.py reports one JSON line per changed model (--report-file); a failing model
  no longer ends the scan. Unmatched info keeps why (not_found / network / offline);
  online scans look network/offline ones up again. After two models in a row without
  an answer the rest of the scan infers from the files (--civitai-down for the next
  folders) instead of waiting out every timeout.
- civitai_client.py: the network side moved out of scraper.py; tells a 404 from
  Civitai being unreachable (CivitaiUnreachable).
- api/scan_report.py: ScanJob turns a scan's reports into one result, kept as the last
  scan (/anomalous/last_scan) and logged; /anomalous/scan_model opens a listed model.
- /anomalous/scan_all takes picked or listed models as `targets`, so one scan has one
  result; scan_summary lists unmatched and new models with their reason.
- Activity middleware takes details a handler attaches (request[ACTIVITY_DETAIL]).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DemonGatanjieu
2026-10-02 02:52:17 +08:00
co-authored by Claude Opus 5.5
parent e689cc3dac
commit e466e9dfcf
25 changed files with 1508 additions and 628 deletions
+15 -6
View File
@@ -87,9 +87,14 @@ covered by the `styles.css` manifest.
whole models folder with its subfolders for the models page's type chips.
- `api/scanner.py` and `scraper.py` own scan orchestration and enrichment; both scan
routes map request fields to scraper switches in one place (`_scraper_flags`).
`civitai_client.py` is the scraper's network side (version by hash, model page, media),
telling "Civitai does not know the file" from "Civitai could not be asked".
`api/scan_report.py` turns the scraper's per-model report lines into one result per
scan: kept as the last scan (`/anomalous/last_scan`), recorded in the activity log, each
model located so the browser can open it (`/anomalous/scan_model`).
`api/scan_summary.py` counts models matched on Civitai, unmatched and not yet scanned
for the scan page, by the same sidecar rules the scraper uses (`sidecar_info`,
`is_unmatched` in `model_identity.py`).
for the scan page and lists the last two, by the same sidecar rules the scraper uses
(`sidecar_info`, `is_unmatched`, `unmatched_reason` in `model_identity.py`).
- `api/workflow_schema.py`, `api/recipe_schema.py`, `api/recipe_images.py`, and
`api/recipe_store.py` own recipe validation/shaping, images, CRUD, history, and
integrity receipts; `api/recipes.py` is the HTTP facade. `api/recipe_constants.py`
@@ -282,7 +287,8 @@ covered by the `styles.css` manifest.
The activity log (docs/decisions AD-018): `api/activity_log.py` keeps the entries
(newest first, bounded, in the ComfyUI user folder), records this plugin's and
Anomalous_TTS's successful write requests in a middleware, and serves
`/anomalous/activity`; `activity_canvas.js` records canvas changes by comparing a
`/anomalous/activity` (a handler may attach what it changed, `request[ACTIVITY_DETAIL]`;
scans record themselves when they end); `activity_canvas.js` records canvas changes by comparing a
snapshot taken when the user presses inside Anomalous with the canvas when they
next press or type outside it (`activity_diff.js` holds the pure snapshot and
difference); `activity_log.js` is the client and the words for each entry;
@@ -293,9 +299,12 @@ covered by the `styles.css` manifest.
including Esc on the workspace panel (`nbPanel`, below the header and right of the
rail; the list column steps aside while it is open).
`ui_scan_page.js` is the scan page (a tool page like the doctor): counts, scan and
look-up-again buttons, advanced options. `scan_runner.js` starts and follows scans
(every folder, picked models, or one model from its card) and refreshes node
drop-downs, hashes and the grid afterwards; folder visibility/order lives in
look-up-again buttons, advanced options; `ui_scan_lists.js` renders its last-scan result
and the unmatched / not-scanned lists, whose rows open a model or scan it;
`scan_results.js` holds the words for scan results, shared with the activity page.
`scan_runner.js` starts and follows scans (every folder, picked or listed models, or one
model from its card), shows the result and refreshes node drop-downs, hashes and the
grid afterwards; folder visibility/order lives in
`ui_folder_manager.js`; and help content lives in `ui_help.js`.
`scan_progress.js` owns the scan progress panel
(`updateScanProgress` / `finishScanProgress` / `failScanProgress`): inside the scan
+3
View File
@@ -11,6 +11,8 @@
- **Deleting goes to the Recycle Bin (删除进回收站)**: deleting a model (with its info and covers), an output image, generated audio, a recipe (with its versions and images), a material, a prompt note or a parameter note, replacing or resetting a cover, and the duplicate copy a file rename finds, all move the files to the system Recycle Bin instead of erasing them. On a USB stick or network drive, which has no Recycle Bin, nothing is deleted and the message says to delete it by hand. The confirmations say where things go. An unused route that could erase every `.civitai.info` in the library (with the names you set) is gone.
- **Scan page (扫描页)**: the rail's scan button and Home's "Scan model folders" open a page instead of the old seven-switch dialog. It shows how many models are matched on Civitai, not matched, and not scanned yet; one button scans the new ones, with progress in the page (it runs in the background, so the window can be closed). Offline scanning, renaming files on disk, fetching everything again, picking models and the Civitai API key are under **Advanced**; the two risky switches turn themselves off after each scan. The "record model fingerprints in workflows" switch moved to Settings → Model settings.
- **Unmatched models can be looked up again (没匹配上的模型可以重新查找)**: a model that Civitai did not know, or that was scanned while Civitai could not be reached, used to stay "local only" for good unless every model was fetched again (which also replaced your own covers). **Look up the N unmatched models again** retries just those, keeps your covers, and reuses the saved hash when the file is unchanged, so it takes seconds instead of re-hashing gigabytes.
- **Which models, and what a scan did (是哪些模型、扫描做了什么)**: on the scan page, **Not matched** and **Not scanned yet** open a list of those models, with their type and folder; unmatched ones say why (Civitai does not have the file, Civitai could not be reached then, or it was an offline scan) and the base model inferred. Each row opens the model (Back returns to the scan page), scans it or looks it up again, or searches Civitai for its name. After a scan the page shows its result: when, how long, how many matched, inferred, failed, covers downloaded and files renamed, and each model it changed with what happened; the progress box says the same and, when it floats over the canvas, has **Show the result**. A model's card scan looks an unmatched model up again and its message says what it found.
- **Online and offline in one scan (联网和离线在同一次扫描里)**: a scan asks Civitai first and infers the base model from the file for what Civitai does not know, as before; now it also remembers why. When Civitai stops answering (two models in a row), the rest of the scan infers from the files right away instead of waiting out every timeout, and the next online scan looks those models up again by itself, as it does for models from an offline scan. Models Civitai answered "not found" for wait for **Look up again**. A model that fails (a locked file, say) no longer ends the scan of the others; it is listed as failed with the reason.
- **Your edits come first (你改的永远优先)**: names, notes and download links you set in the model editor now have a file of their own next to the model (`<model>.anomalous.json`) that scans never write, and they win over Civitai's information, which wins over what is inferred from the file. Edits saved by earlier versions keep working. Editing a model that was never scanned used to make it look scanned, so scans skipped it for good; it is now scanned like any new model.
- **Covers you chose are never replaced (你换的封面不会被替换)**: a cover counts as Civitai's only while it is the very image the scan downloaded; any other cover is yours. **Fetch scanned models again** now keeps your covers (and names and notes), updates only Civitai's own, and asks before it starts.
- **Where a model's information comes from (信息来源标注)**: cards look as before for models matched on Civitai; a base model inferred from the file shows as "≈ SDXL" with a dashed outline, and a model not scanned yet says so, each with a tooltip on what to do. The detail header adds the source (Civitai, inferred from the file, not scanned) and which fields you edited.
@@ -22,6 +24,7 @@
### 🕘 Activity log (操作记录)
- **See what Anomalous changed (看 Anomalous 改了什么)**: the new **Activity** page on the rail lists, by day, every change the plugin made: on the canvas, which node settings changed (from → to), which nodes were added or removed, and which workflow was opened; in files, models, covers, recipes, materials, notes, presets, images, audio and imported characters. Filter by canvas or files; open a canvas entry for its details and **Find on canvas** to jump to the node. Your own edits on the canvas are not recorded as the plugin's. The home page shows the latest few.
- **More detail (记录更详细)**: a scan is recorded when it ends, with what it found and each model it changed (open one from the entry), instead of a bare "scanned models" when it started. Editing a model records each field from → to, and deleting one lists the files moved to the Recycle Bin.
- It only records; nothing is undone for you. **Clear** removes the log itself, never your changes. The newest 500 entries (at most 2 MB) are kept in ComfyUI's user folder.
### 🎙️ Audio & Voice Studio (音频与声音工作台)
+3 -1
View File
@@ -1,6 +1,6 @@
from . import (
activity_log, audio_catalog, folder_types, gallery_routes, materials, media_routes, model_catalog,
model_media, model_metadata, model_resolution, model_type_listing, recipe_packages, recipes, scan_summary,
model_media, model_metadata, model_resolution, model_type_listing, recipe_packages, recipes, scan_report, scan_summary,
translation_routes, version_manager,
)
from .scanner import *
@@ -106,6 +106,8 @@ def setup_routes(app):
app.router.add_post('/anomalous/resolve_paths_to_previews', model_catalog.api_resolve_paths_to_previews)
app.router.add_post('/anomalous/scan_missing_models', api_scan_missing_models)
app.router.add_get('/anomalous/scan_summary', scan_summary.api_scan_summary)
app.router.add_get('/anomalous/last_scan', scan_report.api_last_scan)
app.router.add_get('/anomalous/scan_model', scan_report.api_scan_model)
# Audio & Voice Studio Routes
app.router.add_get('/anomalous/audio_stream', audio_catalog.api_serve_audio)
+22 -4
View File
@@ -26,14 +26,13 @@ MAX_LOG_BYTES = 2 * 1024 * 1024 # long canvas entries could otherwise outgrow t
MAX_VALUE_CHARS = 400
MAX_CHANGES = 60
# Write routes worth a line in the log → the action id the browser has words for.
# Write routes worth a line in the log → the action id the browser has words for. Scans are
# recorded when they end, with what they found (api/scan_report.py), not here.
ROUTE_ACTIONS = {
'/anomalous/delete_model': 'model_delete',
'/anomalous/update_metadata': 'model_edit',
'/anomalous/set_custom_cover': 'model_cover',
'/anomalous/upload_custom_cover': 'model_cover',
'/anomalous/scan': 'scan',
'/anomalous/scan_all': 'scan',
'/anomalous/save_recipe': 'recipe_save',
'/anomalous/update_recipe': 'recipe_edit',
'/anomalous/delete_recipe': 'recipe_delete',
@@ -63,6 +62,10 @@ ROUTE_ACTIONS = {
TARGET_FIELDS = ('name', 'character', 'filename', 'target_filename', 'recipe_filename',
'notebook_filename', 'new_name', 'path', 'file', 'target')
# What a handler attaches to its request for the log: request[ACTIVITY_DETAIL] = {fields, files}.
# web.RequestKey is newer than the aiohttp some ComfyUI installs have; a plain key works there.
ACTIVITY_DETAIL = web.RequestKey("activity_detail", dict) if hasattr(web, "RequestKey") else "activity_detail"
_lock = threading.RLock()
_entries = None
@@ -173,6 +176,20 @@ async def _request_body(request):
return {}
def _clean_detail(detail):
"""What a handler attached for the log (request[ACTIVITY_DETAIL]): the fields it changed
({field, before, after}) and the files it moved, values shortened."""
if not isinstance(detail, dict):
return None
cleaned = {}
if isinstance(detail.get('fields'), list):
cleaned['fields'] = [{key: _short(item[key]) for key in ('field', 'before', 'after') if key in item}
for item in detail['fields'][:MAX_CHANGES] if isinstance(item, dict)]
if isinstance(detail.get('files'), list):
cleaned['files'] = [_short(name) for name in detail['files'][:MAX_CHANGES]]
return {key: value for key, value in cleaned.items() if value} or None
@web.middleware
async def activity_middleware(request, handler):
response = await handler(request)
@@ -180,7 +197,8 @@ async def activity_middleware(request, handler):
if action and not _failed(response):
try:
target = _target_of(await _request_body(request))
await asyncio.get_running_loop().run_in_executor(None, add_entry, 'file', action, target)
detail = _clean_detail(request.get(ACTIVITY_DETAIL))
await asyncio.get_running_loop().run_in_executor(None, add_entry, 'file', action, target, detail)
except Exception: # noqa: BLE001 - the log must never fail the action it records
logging.getLogger(__name__).warning('Anomalous activity log: could not record %s', request.path, exc_info=True)
return response
+6 -4
View File
@@ -11,9 +11,9 @@ from aiohttp import web
import folder_paths
import struct
try:
from ..model_identity import USER_INFO_SUFFIX, is_unmatched, read_json, sidecar_file_hash, sidecar_info
from ..model_identity import USER_INFO_SUFFIX, is_unmatched, read_json, sidecar_file_hash, sidecar_info, unmatched_reason
except ImportError:
from model_identity import USER_INFO_SUFFIX, is_unmatched, read_json, sidecar_file_hash, sidecar_info
from model_identity import USER_INFO_SUFFIX, is_unmatched, read_json, sidecar_file_hash, sidecar_info, unmatched_reason
# Fields the model editor sets: <model>.anomalous.json key -> metadata key.
USER_FIELDS = {"custom_name": "custom_name", "custom_notes": "custom_notes", "source_url": "source_url"}
@@ -102,7 +102,8 @@ def _read_metadata(file_path):
"custom_name": "",
"custom_notes": "",
# "civitai" (matched), "local" (only what the file tells: no match, or offline) or "" (not scanned)
"info_source": "",
"info_source": "",
"unmatched_reason": "", # for "local": not_found | network | offline | "" (scanned before this was kept)
"user_fields": [],
}
@@ -163,7 +164,8 @@ def _read_metadata(file_path):
scan = sidecar_info(base_path)
if scan is not None:
metadata["info_source"] = "local" if is_unmatched(scan) else "civitai"
metadata["info_source"] = "local" if is_unmatched(scan) else "civitai"
metadata["unmatched_reason"] = unmatched_reason(scan)
# The user's own layer comes last and wins, also when a field was cleared on purpose ("").
user = read_json(base_path + USER_INFO_SUFFIX) or {}
for key, field in USER_FIELDS.items():
+11
View File
@@ -14,6 +14,8 @@ from .model_constants import (
CIVITAI_BACKUP_SUFFIXES, MEDIA_EXTENSIONS, MODEL_EXTENSIONS,
PREVIEW_SUFFIXES, SIDECAR_SUFFIXES,
)
from .activity_log import ACTIVITY_DETAIL
from .metadata import get_metadata
from .model_catalog import _resolve_paths_to_model_info_sync
from .trash import move_to_trash, trash_failure
from .utils import atomic_write_json
@@ -117,6 +119,7 @@ async def api_delete_model(request):
except Exception as e:
return web.json_response({"status": "error", "message": trash_failure(e)})
deleted_files = [os.path.basename(path) for path in moved]
request[ACTIVITY_DETAIL] = {"files": deleted_files}
# 前端期待的成功状态是 "success" 而不是 "ok"
return web.json_response({
@@ -194,6 +197,7 @@ async def api_update_metadata(request):
if os.path.exists(user_file) and read_json(user_file) is None:
return web.json_response({"status": "error", "message": f"{os.path.basename(user_file)} is unreadable; nothing was changed."})
user_data = read_json(user_file) or {"format": 1}
before = get_metadata(file_path) # what was shown, for the activity log
if 'custom_name' in data:
user_data["custom_name"] = str(custom_name)
if 'custom_notes' in data:
@@ -231,6 +235,13 @@ async def api_update_metadata(request):
elif os.path.exists(new_file_path) and new_file_path != file_path:
return web.json_response({"status": "error", "message": "A file with the target physical name already exists."})
sent = [key for key in ("custom_name", "custom_notes") if key in data]
sent += ["source_url"] if custom_source_url is not None else []
fields = [{"field": key, "before": before.get(key) or "", "after": user_data[key]}
for key in sent if user_data[key] != (before.get(key) or "")]
if new_filename != filename:
fields.append({"field": "filename", "before": filename, "after": new_filename})
request[ACTIVITY_DETAIL] = {"fields": fields}
response_data = {
"status": "success",
"new_filename": new_filename,
+166
View File
@@ -0,0 +1,166 @@
"""What a scan did, model by model.
The scraper appends one JSON line per model it changed to a report file (--report-file),
plus events: "civitai_down" when Civitai stopped answering, "done" with the count of models
left as they were. When a scan ends, ScanJob turns the lines of all its folders into one
result: kept as the last scan (GET /anomalous/last_scan, shown on the scan page) and recorded
in the activity log, with each model located by type, path index and relative path so the
browser can open it (GET /anomalous/scan_model).
"""
import asyncio
import json
import os
import time
from aiohttp import web
import folder_paths
from .activity_log import MAX_CHANGES, add_entry, log_path
from .folder_types import get_active_folder_types
from .model_catalog import _model_info_for_path
from .path_utils import atomic_write_json, resolve_folder_subdir
MAX_FILES = 2000 # models kept in the last scan's result
FILE_FIELDS = ("status", "reason", "name", "base", "cover", "error")
def last_scan_path():
return os.path.join(os.path.dirname(log_path()), "last_scan.json")
def locate_root(directory):
"""(type, path_idx) of the models folder that is `directory`, shown types first; None if none."""
target = os.path.normcase(os.path.realpath(directory))
active = get_active_folder_types()
types = active + [name for name in folder_paths.folder_names_and_paths if name not in active]
for folder_type in types:
try:
paths = folder_paths.get_folder_paths(folder_type) or []
except Exception:
continue
for path_idx, path in enumerate(paths):
if os.path.normcase(os.path.realpath(path)) == target:
return folder_type, path_idx
return None
def read_report(path):
"""(model entries, events) from a report file; both empty when there is none."""
entries, events = [], []
try:
with open(path, encoding="utf-8") as source:
for line in source:
try:
item = json.loads(line)
except ValueError:
continue
if isinstance(item, dict):
(events if "event" in item else entries).append(item)
except OSError:
pass
return entries, events
class ScanJob:
"""One scan from start to end: what to call it, its options, and its folders' reports."""
def __init__(self, kind, data):
self.kind = kind # "all" | "picked" | "one"
self.options = {key: bool(data.get(key)) for key in ("offline_only", "retry_unmatched", "force_overwrite")}
self.started = time.time()
self.files = []
self.unchanged = 0
self.civitai_down = False
self.errors = []
def add_folder(self, report_file, base_dir, located=None):
"""Reads (and removes) one scraper run's report. `located`: (type, path_idx) of base_dir."""
located = located or locate_root(base_dir) or ("", 0)
entries, events = read_report(report_file)
try:
os.remove(report_file)
except OSError:
pass
for event in events:
if event.get("event") == "done":
self.unchanged += int(event.get("unchanged") or 0)
elif event.get("event") == "civitai_down":
self.civitai_down = True
for entry in entries:
path = entry.get("renamed_to") if entry.get("renamed_to") and entry.get("status") != "failed" else entry.get("file")
if not isinstance(path, str):
continue
item = {key: entry[key] for key in FILE_FIELDS if entry.get(key)}
item.update(type=located[0], path_idx=located[1], filename=os.path.basename(path),
rel=os.path.relpath(path, base_dir).replace(os.sep, "/"))
if entry.get("renamed_to") and item["status"] != "failed":
item["renamed_from"] = os.path.basename(entry.get("file") or "")
if entry.get("duplicate_of"):
item["duplicate"] = True
self.files.append(item)
def add_error(self, message):
self.errors.append(str(message)[:300])
def result(self):
files = self.files
counts = {
"matched": sum(item["status"] == "matched" for item in files),
"inferred": sum(item["status"] == "inferred" for item in files),
"failed": sum(item["status"] == "failed" for item in files),
"covers": sum(item.get("cover") == "new" for item in files),
"renamed": sum(bool(item.get("renamed_from")) for item in files),
"unchanged": self.unchanged,
}
return {
"kind": self.kind, "options": self.options, "started": self.started,
"finished": time.time(), "counts": counts, "civitai_down": self.civitai_down,
"errors": self.errors[:10], "files": files[:MAX_FILES],
}
def finish(self):
"""Keeps the result as the last scan and records it in the activity log. Never raises:
the scan itself is done, the record of it is a convenience."""
result = self.result()
try:
atomic_write_json(last_scan_path(), result)
except Exception as error: # noqa: BLE001
print(f"[Anomalous Browser] Could not keep the scan result: {error}")
try:
target = result["files"][0]["filename"] if self.kind == "one" and result["files"] else ""
detail = {"scan": {key: result[key] for key in ("kind", "options", "counts", "civitai_down", "errors")},
"files": result["files"][:MAX_CHANGES], "total": len(self.files),
"seconds": round(result["finished"] - result["started"])}
add_entry("file", "scan_done", target, detail)
except Exception as error: # noqa: BLE001
print(f"[Anomalous Browser] Could not record the scan in the activity log: {error}")
return result
async def api_last_scan(request):
"""GET /anomalous/last_scan - the last scan's result, or {} before the first one."""
def read():
try:
with open(last_scan_path(), encoding="utf-8") as source:
data = json.load(source)
return data if isinstance(data, dict) else {}
except (OSError, ValueError):
return {}
return web.json_response(await asyncio.to_thread(read))
async def api_scan_model(request):
"""GET /anomalous/scan_model?type&path_idx&rel - one model as the grid lists it, to open it."""
rel = request.query.get("rel", "").replace("\\", "/").strip("/")
subfolder, filename = os.path.split(rel)
try:
path_idx = int(request.query.get("path_idx", 0))
base_dir, target_dir = resolve_folder_subdir(request.query.get("type", ""), path_idx, "/" + subfolder)
except (ValueError, KeyError, TypeError):
return web.json_response({"error": "Invalid model location"}, status=400)
file_path = os.path.join(target_dir, filename)
if not filename or filename in (".", "..") or not os.path.isfile(file_path):
return web.json_response({"error": "Model not found"}, status=404)
model = await asyncio.to_thread(_model_info_for_path, request.query.get("type"), path_idx, base_dir, file_path)
return web.json_response({"model": model})
+29 -12
View File
@@ -1,4 +1,6 @@
"""What the scan page shows before a scan: how many models are matched on Civitai, unmatched, or new."""
"""What the scan page shows before a scan: how many models are matched on Civitai, unmatched or
new, and which ones the last two are (located like the scan report's models, so the page can
open them)."""
import asyncio
import os
@@ -6,18 +8,25 @@ import os
from aiohttp import web
from .folder_types import get_active_scan_paths
from .scan_report import locate_root
try:
from ..model_identity import is_unmatched, sidecar_info
from ..model_identity import is_unmatched, sidecar_info, unmatched_reason
except ImportError: # loaded outside the package (tests)
from model_identity import is_unmatched, sidecar_info
from model_identity import is_unmatched, sidecar_info, unmatched_reason
MAX_LISTED = 2000 # models listed per kind; the counts stay exact
def _summarize():
"""Counts the files the scanner handles (.safetensors in the active scan folders), each once."""
"""Counts the files the scanner handles (.safetensors in the active scan folders), each once.
`pending`: unmatched models Civitai was never asked about (offline, or unreachable then),
which the next online scan looks up."""
seen = set()
total = matched = unmatched = 0
total = matched = 0
unmatched, new = [], []
for base_dir in get_active_scan_paths():
located = locate_root(base_dir) or ("", 0)
for root, _dirs, files in os.walk(base_dir):
for filename in files:
if not filename.endswith(".safetensors"):
@@ -29,15 +38,23 @@ def _summarize():
seen.add(real)
total += 1
info = sidecar_info(os.path.splitext(path)[0])
if info is None:
continue
if is_unmatched(info):
unmatched += 1
else:
if info is not None and not is_unmatched(info):
matched += 1
return {"total": total, "matched": matched, "unmatched": unmatched, "new": total - matched - unmatched}
continue
item = {"type": located[0], "path_idx": located[1], "filename": filename,
"rel": os.path.relpath(path, base_dir).replace(os.sep, "/")}
if info is None:
new.append(item)
else:
item.update(reason=unmatched_reason(info), base=info.get("baseModel") or "")
unmatched.append(item)
return {
"total": total, "matched": matched, "unmatched": len(unmatched), "new": len(new),
"pending": sum(item["reason"] in ("offline", "network") for item in unmatched),
"unmatched_models": unmatched[:MAX_LISTED], "new_models": new[:MAX_LISTED],
}
async def api_scan_summary(request):
"""GET /anomalous/scan_summary - {total, matched, unmatched, new}."""
"""GET /anomalous/scan_summary - counts, and the unmatched and new models."""
return web.json_response(await asyncio.to_thread(_summarize))
+105 -86
View File
@@ -13,14 +13,17 @@ import uuid
from aiohttp import web
import folder_paths
import struct
from .folder_types import get_active_folder_types, get_active_scan_paths
from .folder_types import get_active_scan_paths
from .path_utils import resolve_folder_subdir
from .scan_report import ScanJob, locate_root
try:
from ..model_policies import is_physical_rename_protected
except ImportError:
from model_policies import is_physical_rename_protected
PLUGIN_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
SCRAPER_PATH = os.path.join(PLUGIN_DIR, "scraper.py")
SCAN_SESSION_ID = uuid.uuid4().hex
SCAN_MARKER_LOCK = threading.Lock()
ACTIVE_SCAN_MARKERS = set()
@@ -278,82 +281,75 @@ async def api_scan_status(request):
return web.json_response(data)
async def api_scan_folder(request):
"""Launches the scraper in the background for a specific directory."""
"""Launches the scraper in the background for one folder (a model card's scan)."""
folder_type = request.query.get('type', 'checkpoints')
subfolder = request.query.get('subfolder', '/')
try:
path_idx = int(request.query.get('path_idx', 0))
except:
except (TypeError, ValueError):
path_idx = 0
try:
base_dir, target_dir = resolve_folder_subdir(folder_type, path_idx, subfolder)
except (ValueError, KeyError):
return web.json_response({"status": "error", "message": "Invalid folder type"})
if not os.path.exists(target_dir):
return web.json_response({"status": "error", "message": "Directory does not exist"})
# Get the scraper path relative to this script
plugin_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
scraper_path = os.path.join(plugin_dir, "scraper.py")
if not os.path.exists(scraper_path):
if not os.path.exists(SCRAPER_PATH):
return web.json_response({"status": "error", "message": "scraper.py not found in extension directory"})
print(f"[Anomalous Browser] Starting background scan for: {target_dir}")
try:
data = await request.json()
except ValueError:
data = None
if not isinstance(data, dict):
return web.json_response({"status": "error", "message": "Invalid request body"}, status=400)
flags = _scraper_flags(data, False, not is_physical_rename_protected(
folder_type=folder_type,
folder_path=target_dir,
))
target_files_list = data.get("target_files", [])
if not target_files_list:
target_files_str = request.query.get('target_files', '')
target_files_list = [f.strip() for f in target_files_str.split(',')] if target_files_str else []
claimed = False
try:
marker_file = os.path.join(target_dir, '.scan_in_progress')
progress_file = os.path.join(target_dir, '.scan_progress.json')
targets_file = os.path.join(target_dir, '.scan_targets.json')
claimed, recovered = _claim_scan_marker(
marker_file,
"folder",
(progress_file, targets_file),
)
report_file = os.path.join(target_dir, '.scan_report.jsonl')
artifacts = (progress_file, targets_file, report_file)
claimed, recovered = _claim_scan_marker(marker_file, "folder", artifacts)
if not claimed:
return web.json_response({"status": "error", "message": "Scan already in progress"}, status=409)
if target_files_list:
with open(targets_file, 'w', encoding='utf-8') as f:
__import__('json').dump(target_files_list, f)
json.dump(target_files_list, f)
job = ScanJob("one" if len(target_files_list) == 1 else "picked", data)
def run_bg():
try:
cmd = [sys.executable, scraper_path, target_dir, "--folder-type", folder_type, *flags]
cmd.extend(["--progress-file", progress_file])
process = subprocess.Popen(cmd, cwd=plugin_dir)
cmd = [sys.executable, SCRAPER_PATH, target_dir, "--folder-type", folder_type, *flags]
cmd.extend(["--progress-file", progress_file, "--report-file", report_file])
process = subprocess.Popen(cmd, cwd=PLUGIN_DIR)
_update_scan_marker(marker_file, worker_pid=process.pid)
return_code = process.wait()
job.add_folder(report_file, base_dir, (folder_type, path_idx))
if return_code != 0:
job.add_error(f"Scanner exited with code {return_code}")
result_file = os.path.join(target_dir, '.scan_result.json')
with open(result_file, 'w', encoding='utf-8') as f:
json.dump({"success": 0, "fail": 1, "error": f"Scanner exited with code {return_code}"}, f)
finally:
_clear_folder_caches()
_release_scan_marker(marker_file, (progress_file, targets_file))
job.finish() # before the marker goes: whoever sees the scan end finds its result
_release_scan_marker(marker_file, artifacts)
threading.Thread(target=run_bg, daemon=True).start()
return web.json_response({
"status": "ok",
"message": "Scan started in background. Check console for details.",
@@ -364,85 +360,122 @@ async def api_scan_folder(request):
_release_scan_marker(marker_file, (progress_file, targets_file))
return web.json_response({"status": "error", "message": str(e)})
def _global_scan_plan(data):
"""The folders a scan-page scan runs through: [(scan dir, (type, path_idx) or None, base dir,
file names or None, folder type for the rename policy)]. `data["targets"]`, when given, is
the picked models: [{type, path_idx, subfolder, files}]; otherwise every active folder."""
targets = data.get("targets")
if not targets:
plan = []
for base_dir in get_active_scan_paths():
if os.path.exists(base_dir):
plan.append((base_dir, locate_root(base_dir), base_dir, None, _protected_type_for_path(base_dir)))
return plan
if not isinstance(targets, list):
raise ValueError("targets must be a list")
plan = []
for target in targets:
files = target.get("files") if isinstance(target, dict) else None
if not isinstance(files, list) or not all(isinstance(name, str) for name in files):
raise ValueError("each target needs a list of file names")
path_idx = int(target.get("path_idx", 0))
base_dir, target_dir = resolve_folder_subdir(target.get("type"), path_idx, target.get("subfolder") or "/")
if files and os.path.isdir(target_dir):
plan.append((target_dir, (target["type"], path_idx), base_dir, files, target["type"]))
return plan
async def api_scan_all(request):
plugin_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
scraper_path = os.path.join(plugin_dir, "scraper.py")
marker_file = os.path.join(plugin_dir, '.global_scan_in_progress')
progress_file = os.path.join(plugin_dir, '.global_scan_progress.json')
"""Scans every active models folder, or the picked models (`targets`), one folder after another."""
marker_file = os.path.join(PLUGIN_DIR, '.global_scan_in_progress')
progress_file = os.path.join(PLUGIN_DIR, '.global_scan_progress.json')
targets_file = os.path.join(PLUGIN_DIR, '.global_scan_targets.json')
report_file = os.path.join(PLUGIN_DIR, '.global_scan_report.jsonl')
artifacts = (progress_file, targets_file, report_file)
try:
data = await request.json()
except ValueError:
data = None
if not isinstance(data, dict):
return web.json_response({"status": "error", "message": "Invalid request body"}, status=400)
try:
plan = _global_scan_plan(data) if data.get("targets") else None
except (ValueError, KeyError, TypeError) as e:
return web.json_response({"status": "error", "message": f"Invalid targets: {e}"}, status=400)
picked = sum(len(files) for _dir, _loc, _base, files, _type in plan or ())
job = ScanJob("all" if plan is None else "one" if picked == 1 else "picked", data)
claimed = False
try:
claimed, recovered = _claim_scan_marker(marker_file, "global", (progress_file,))
claimed, recovered = _claim_scan_marker(marker_file, "global", artifacts)
if not claimed:
return web.json_response({"status": "error", "message": "Global scan already in progress"}, status=409)
def run_global_bg():
try:
paths_to_scan = get_active_scan_paths()
paths_to_scan = [path for path in paths_to_scan if os.path.exists(path)]
_update_scan_state(
marker_file,
phase="preparing",
folder_total=len(paths_to_scan),
folder_current=0,
folder="",
)
for folder_index, base_dir in enumerate(paths_to_scan, 1):
if not os.path.exists(base_dir): continue
folders = plan if plan is not None else _global_scan_plan(data)
_update_scan_state(marker_file, phase="preparing", folder_total=len(folders), folder_current=0, folder="")
for folder_index, (scan_dir, located, base_dir, files, policy_type) in enumerate(folders, 1):
try:
print(f"[Anomalous Browser] Global scan processing: {base_dir}")
print(f"[Anomalous Browser] Global scan processing: {scan_dir}")
_remove_file(progress_file)
_update_scan_state(
marker_file,
phase="enumerating",
folder_total=len(paths_to_scan),
folder_total=len(folders),
folder_current=folder_index,
folder=os.path.basename(os.path.normpath(base_dir)) or base_dir,
folder=os.path.basename(os.path.normpath(scan_dir)) or scan_dir,
error="",
)
cmd = [sys.executable, scraper_path, base_dir]
protected_type = _protected_type_for_path(base_dir)
if protected_type:
cmd.extend(["--folder-type", protected_type])
cmd = [sys.executable, SCRAPER_PATH, scan_dir]
if policy_type:
cmd.extend(["--folder-type", policy_type])
cmd.extend(_scraper_flags(data, True, not is_physical_rename_protected(
folder_type=protected_type,
folder_path=base_dir,
folder_type=policy_type,
folder_path=scan_dir,
)))
cmd.extend(["--progress-file", progress_file])
process = subprocess.Popen(cmd, cwd=plugin_dir)
if files is not None:
with open(targets_file, 'w', encoding='utf-8') as f:
json.dump(files, f)
cmd.extend(["--targets-file", targets_file])
if job.civitai_down: # an earlier folder found Civitai unreachable
cmd.append("--civitai-down")
cmd.extend(["--progress-file", progress_file, "--report-file", report_file])
process = subprocess.Popen(cmd, cwd=PLUGIN_DIR)
_update_scan_marker(marker_file, worker_pid=process.pid)
return_code = process.wait()
_update_scan_marker(marker_file, worker_pid=0)
job.add_folder(report_file, base_dir, located)
if return_code != 0:
message = f"Scanner exited with code {return_code}: {base_dir}"
message = f"Scanner exited with code {return_code}: {scan_dir}"
_update_scan_state(marker_file, error=message)
job.add_error(message)
print(f"[Anomalous Browser] {message}")
except Exception as e:
_update_scan_state(marker_file, error=str(e))
print(f"[Anomalous Browser] Global scan error on {base_dir}: {e}")
job.add_error(f"{scan_dir}: {e}")
print(f"[Anomalous Browser] Global scan error on {scan_dir}: {e}")
finally:
_clear_folder_caches()
_release_scan_marker(marker_file, (progress_file,))
job.finish() # before the marker goes: whoever sees the scan end finds its result
_release_scan_marker(marker_file, artifacts)
threading.Thread(target=run_global_bg, daemon=True).start()
return web.json_response({"status": "ok", "message": "Global scan started", "recovered": recovered})
except Exception as e:
if claimed:
_release_scan_marker(marker_file, (progress_file,))
_release_scan_marker(marker_file, artifacts)
return web.json_response({"status": "error", "message": str(e)})
async def api_global_scan_status(request):
plugin_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
marker_file = os.path.join(plugin_dir, '.global_scan_in_progress')
progress_file = os.path.join(plugin_dir, '.global_scan_progress.json')
marker_file = os.path.join(PLUGIN_DIR, '.global_scan_in_progress')
progress_file = os.path.join(PLUGIN_DIR, '.global_scan_progress.json')
return web.json_response(_scan_status_payload(marker_file, progress_file, (progress_file,)))
GLOBAL_SCAN_STATE = {
"scanning": False,
"total": 0,
@@ -465,7 +498,7 @@ async def api_scan_missing_models(request):
sys.path.insert(0, plugin_dir)
try:
from scraper import calculate_sha256, fetch_civitai_info, infer_base_model_from_header, computed_file_identity
from scraper import CivitaiUnreachable, calculate_sha256, computed_file_identity, fetch_civitai_info, local_record
except ImportError:
return web.json_response({"status": "error", "message": "Failed to load scraper module"})
@@ -510,27 +543,13 @@ async def api_scan_missing_models(request):
try:
file_hash = calculate_sha256(file_path)
civitai_data = fetch_civitai_info(file_hash)
try:
civitai_data, reason = fetch_civitai_info(file_hash), "not_found"
except CivitaiUnreachable:
civitai_data, reason = None, "network"
if not civitai_data: # what the file itself tells
civitai_data = local_record(file_path, os.path.dirname(file_path), file_hash, reason)
# Fallback 3: Local Offline Inference
if not civitai_data:
inferred_base = infer_base_model_from_header(file_path)
if inferred_base == 'Unknown':
inferred_base = ""
civitai_data = {
"id": -1,
"modelId": -1,
"name": os.path.splitext(filename)[0],
"baseModel": inferred_base,
"description": "<p>Automatically inferred by Anomalous Local Engine.</p>",
"model": {
"name": os.path.splitext(filename)[0],
"type": "LORA" if "lora" in file_path.lower() else "Checkpoint"
},
"files": [{"hashes": {"SHA256": file_hash}}]
}
civitai_data["anomalous_file_identity"] = computed_file_identity(file_path, file_hash)
# Save info file
+107
View File
@@ -0,0 +1,107 @@
"""Civitai over the network, for the scanner: a model version by file hash, a model page's
description, and preview media. The API key comes from the plugin's config.json.
fetch_civitai_info tells the two failures apart: None means Civitai answered that it does not
know the file; CivitaiUnreachable means it could not be asked (no connection, timeouts, errors).
"""
import json
import os
import time
import urllib.error
import urllib.request
from typing import Dict, Optional
USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
class CivitaiUnreachable(Exception):
"""Civitai did not answer (network, timeout or server error) after the retries."""
def _load_api_key():
plugin_dir = os.path.dirname(os.path.abspath(__file__))
for config_path in (os.path.join(plugin_dir, "api", "config.json"), os.path.join(plugin_dir, "config.json")):
if not os.path.exists(config_path):
continue
try:
with open(config_path, 'r', encoding='utf-8') as f:
key = json.load(f).get("CIVITAI_API_KEY", "")
if isinstance(key, str) and key.strip():
return key.strip()
except Exception as e:
print(f"[-] 读取 config.json 失败: {e}")
return None
CIVITAI_API_KEY = _load_api_key()
def _headers():
headers = {"User-Agent": USER_AGENT}
if CIVITAI_API_KEY:
headers["Authorization"] = f"Bearer {CIVITAI_API_KEY}"
return headers
def fetch_civitai_info(file_hash: str, max_retries: int = 3) -> Optional[Dict]:
"""Civitai's record of the model version with this file hash; None when Civitai does not
know the file (404). Raises CivitaiUnreachable when it could not be asked."""
url = f"https://civitai.com/api/v1/model-versions/by-hash/{file_hash}"
for attempt in range(max_retries):
try:
with urllib.request.urlopen(urllib.request.Request(url, headers=_headers()), timeout=15) as response:
return json.loads(response.read().decode('utf-8'))
except urllib.error.HTTPError as e:
if e.code == 404:
print(f"\033[93m[Skip] 模型 Hash {file_hash} 未在 Civitai 找到 (404)。\033[0m")
return None
print(f"[-] 请求异常,状态码: {e.code} (尝试 {attempt+1}/{max_retries})")
except urllib.error.URLError as e:
print(f"[-] 网络请求超时或异常: {e.reason} (尝试 {attempt+1}/{max_retries})")
except Exception as e:
print(f"[-] 未知异常: {e} (尝试 {attempt+1}/{max_retries})")
if attempt < max_retries - 1:
time.sleep(2)
print(f"\033[93m[Skip] 模型 Hash {file_hash} 网络重试失败。\033[0m")
raise CivitaiUnreachable(file_hash)
def fetch_model_description(model_id) -> str:
"""The model page's description (the version record only has the version's); '' when unavailable."""
try:
request = urllib.request.Request(f"https://civitai.com/api/v1/models/{model_id}", headers=_headers())
with urllib.request.urlopen(request, timeout=10) as response:
return json.loads(response.read().decode('utf-8')).get("description") or ""
except Exception as e:
print(f"[-] 获取模型主页详细说明失败: {e}")
return ""
def download_media(url: str, base_path: str, max_retries: int = 3):
"""Downloads an image or video to base_path + the extension its type calls for; the path, or None."""
for attempt in range(max_retries):
try:
with urllib.request.urlopen(urllib.request.Request(url, headers=_headers()), timeout=15) as response:
content_type = response.headers.get("Content-Type", "").lower()
ext = ".png"
if "video/mp4" in content_type: ext = ".mp4"
elif "video/webm" in content_type: ext = ".webm"
elif "image/jpeg" in content_type: ext = ".jpg"
elif "image/webp" in content_type: ext = ".webp"
elif url.endswith(".mp4"): ext = ".mp4"
final_path = base_path + ext
with open(final_path, 'wb') as f:
while True:
chunk = response.read(8192)
if not chunk:
break
f.write(chunk)
return final_path
except urllib.error.HTTPError as e:
print(f"[-] 媒体下载失败,状态码: {e.code} (尝试 {attempt+1}/{max_retries})")
except urllib.error.URLError as e:
print(f"[-] 媒体下载网络异常: {e.reason} (尝试 {attempt+1}/{max_retries})")
except Exception:
pass
return None
+16
View File
@@ -128,6 +128,22 @@ file count, current index, and filename. Status responses merge worker progress
with parent-owned folder progress. The frontend can reconstruct this state by
polling after its UI has been reopened.
`scraper.py --report-file` appends one JSON line per model it changed (`matched`,
`inferred` with its reason, `failed` with the error, renames and covers) and events
(`civitai_down`, `done` with the unchanged count). One model's error is reported and
the scan goes on. `ScanJob` in `api/scan_report.py` reads each folder's report when its
worker exits and, before the scan's marker is released, writes the scan's result to
`user/anomalous/last_scan.json` and the activity log, so whoever sees the scan end can
read its result. The scan page's picked or listed models go to `/anomalous/scan_all` as
`targets` (`--targets-file` per folder), so one scan has one result.
An unmatched model's `.info` (`id` -1) keeps why: `anomalous_unmatched_reason` is
`not_found` (Civitai answered 404), `network` (no answer after the retries) or
`offline`. Online scans look up `network` and `offline` models again; `not_found` ones
wait for "look up again". After `CIVITAI_DOWN_AFTER` models in a row without an answer
a scan stops asking (`--civitai-down` carries this to the next folders) and infers the
rest from the files, instead of waiting for every timeout.
## Metadata and cache behavior
The output gallery keeps one ordered directory snapshot for at most ten seconds
+11
View File
@@ -48,6 +48,17 @@ def is_unmatched(info):
return isinstance(info, dict) and info.get("id") == -1
# Why an unmatched model has no Civitai record: Civitai does not know the file; Civitai could
# not be reached; the scan was offline. Scans before this was recorded leave it empty.
UNMATCHED_REASONS = ("not_found", "network", "offline")
def unmatched_reason(info):
"""One of UNMATCHED_REASONS for an unmatched model's info, else ""."""
reason = info.get("anomalous_unmatched_reason") if is_unmatched(info) else ""
return reason if reason in UNMATCHED_REASONS else ""
def normalise_sha256(value):
if not isinstance(value, str) or not re.fullmatch(r"[0-9a-fA-F]{64}", value.strip()):
return ""
+227 -304
View File
@@ -16,14 +16,12 @@ import json
import hashlib
import re
import urllib.request
import urllib.error
import urllib.parse
import argparse
import shutil
from typing import Dict, Optional
from model_policies import is_physical_rename_protected
from recycle_bin import move_to_trash
from model_identity import USER_INFO_SUFFIX, computed_file_identity, is_unmatched, scan_info_path, sidecar_file_hash, sidecar_info as read_local_info
from model_identity import USER_INFO_SUFFIX, computed_file_identity, is_unmatched, scan_info_path, sidecar_file_hash, sidecar_info as read_local_info, unmatched_reason
from civitai_client import CIVITAI_API_KEY, CivitaiUnreachable, download_media, fetch_civitai_info, fetch_model_description
# Fixed tuples avoid rebuilding long extension lists for every scanned model.
@@ -89,30 +87,6 @@ def count_scan_files(target_folder, target_files_basenames, unmatched_only=False
)
return total
# ==============================================================================
# CIVITAI API 配置读取
# 请在插件目录 (Anomalous_Model_Browser) 下新建 config.json 文件:
# { "CIVITAI_API_KEY": "你的KEY" }
# ==============================================================================
CIVITAI_API_KEY = None
plugin_dir = os.path.dirname(os.path.abspath(__file__))
config_paths = [os.path.join(plugin_dir, "api", "config.json"), os.path.join(plugin_dir, "config.json")]
for config_path in config_paths:
if not os.path.exists(config_path):
continue
try:
with open(config_path, 'r', encoding='utf-8') as f:
cfg = json.load(f)
configured_key = cfg.get("CIVITAI_API_KEY", "")
if isinstance(configured_key, str) and configured_key.strip():
CIVITAI_API_KEY = configured_key.strip()
break
except Exception as e:
print(f"[-] 读取 config.json 失败: {e}")
if not CIVITAI_API_KEY:
print("[!] 未配置 Civitai API Key。部分限制级模型或将无法获取图片。")
def calculate_sha256(file_path: str) -> str:
"""计算文件的 SHA256 哈希值 (用于 Civitai 匹配)"""
sha256_hash = hashlib.sha256()
@@ -167,71 +141,205 @@ def sanitize_filename(name: str) -> str:
name = re.sub(r'[\\/*?:"<>|#]', "", name)
return name.strip(' .')
def fetch_civitai_info(file_hash: str, max_retries: int = 3) -> Optional[Dict]:
"""向 Civitai API 获取模型信息,支持重试机制"""
url = f"https://civitai.com/api/v1/model-versions/by-hash/{file_hash}"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}
if CIVITAI_API_KEY:
headers["Authorization"] = f"Bearer {CIVITAI_API_KEY}"
for attempt in range(max_retries):
try:
req = urllib.request.Request(url, headers=headers)
with urllib.request.urlopen(req, timeout=15) as response:
return json.loads(response.read().decode('utf-8'))
except urllib.error.HTTPError as e:
if e.code == 404:
print(f"\033[93m[Skip] 模型 Hash {file_hash} 未在 Civitai 找到 (404),已跳过。\033[0m")
return None
print(f"[-] 请求异常,状态码: {e.code} (尝试 {attempt+1}/{max_retries})")
except urllib.error.URLError as e:
print(f"[-] 网络请求超时或异常: {e.reason} (尝试 {attempt+1}/{max_retries})")
except Exception as e:
print(f"[-] 未知异常: {e} (尝试 {attempt+1}/{max_retries})")
if attempt < max_retries - 1:
time.sleep(2)
CIVITAI_DOWN_AFTER = 2 # models in a row Civitai did not answer for before the rest of the run stays offline
print(f"\033[93m[Skip] 模型 Hash {file_hash} 网络重试失败,已跳过该文件。\033[0m")
return None
def download_media(url: str, base_path: str, max_retries: int = 3):
"""下载图片或视频并自动识别扩展名"""
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
def write_report(args, **entry):
"""One line of the scan report (--report-file): a model's outcome, or an event of the run."""
if not args.report_file:
return
try:
with open(args.report_file, 'a', encoding='utf-8') as f:
f.write(json.dumps(entry, ensure_ascii=False) + "\n")
except OSError as e:
print(f"[-] 扫描记录写入失败: {e}")
def local_record(file_path, root, file_hash, reason):
"""What the file alone tells (id -1), with why Civitai's record is missing."""
inferred_base = infer_base_model_from_header(file_path)
stem = os.path.splitext(os.path.basename(file_path))[0]
print(f"[*] 使用本地哈希重建基础元数据 ({os.path.basename(file_path)})")
return {
"id": -1,
"modelId": -1,
"name": stem,
"baseModel": "" if inferred_base == 'Unknown' else inferred_base,
"description": "<p>Automatically inferred by Anomalous Local Engine.</p>",
"model": {"name": stem, "type": "LORA" if "lora" in root.lower() else "Checkpoint"},
"files": [{"hashes": {"SHA256": file_hash}}],
"anomalous_unmatched_reason": reason,
}
if CIVITAI_API_KEY:
headers["Authorization"] = f"Bearer {CIVITAI_API_KEY}"
for attempt in range(max_retries):
try:
req = urllib.request.Request(url, headers=headers)
with urllib.request.urlopen(req, timeout=15) as response:
content_type = response.headers.get("Content-Type", "").lower()
ext = ".png" # default
if "video/mp4" in content_type: ext = ".mp4"
elif "video/webm" in content_type: ext = ".webm"
elif "image/jpeg" in content_type: ext = ".jpg"
elif "image/webp" in content_type: ext = ".webp"
elif url.endswith(".mp4"): ext = ".mp4"
final_path = base_path + ext
with open(final_path, 'wb') as f:
while True:
chunk = response.read(8192)
if not chunk:
break
f.write(chunk)
return final_path
except urllib.error.HTTPError as e:
print(f"[-] 媒体下载失败,状态码: {e.code} (尝试 {attempt+1}/{max_retries})")
except urllib.error.URLError as e:
print(f"[-] 媒体下载网络异常: {e.reason} (尝试 {attempt+1}/{max_retries})")
except Exception as e:
pass
return None
def ask_civitai(args, state, file_hash):
"""(Civitai's record or None, why it is missing: "not_found" | "network" | "offline" | "")."""
if args.offline_only:
return None, "offline"
if state["civitai_down"]:
return None, "network"
try:
data = fetch_civitai_info(file_hash)
except CivitaiUnreachable:
state["network_failures"] += 1
if state["network_failures"] >= CIVITAI_DOWN_AFTER:
state["civitai_down"] = True
print("\033[93m[!] 连续几个模型都连不上 Civitai,本次其余模型改为从文件推断,之后可以“重新联网查找”。\033[0m")
write_report(args, event="civitai_down")
return None, "network"
state["network_failures"] = 0
return data, ("" if data else "not_found")
def scan_one(args, state, rename_log, root, filename, previous):
"""Scans one model; returns its report entry, or None when nothing about it changed."""
file_path = os.path.join(root, filename)
old_base = os.path.splitext(file_path)[0]
online = not args.offline_only and not state["civitai_down"]
# A .civitai.info holding only edits from older versions is not a scan result.
info_exists = scan_info_path(old_base) is not None
# Models Civitai was never asked about (an offline scan, no connection then) are looked up
# by every online scan; models Civitai did not know wait for "look up again".
unasked = is_unmatched(previous) and unmatched_reason(previous) in ("offline", "network")
if args.force_overwrite or args.retry_unmatched or (online and unasked):
info_exists = False
preview_exists = args.skip_media or any(os.path.exists(old_base + ext) for ext in COVER_SUFFIXES)
needs_rename = bool(args.virtual_rename or (not args.skip_rename and args.physical_rename))
if info_exists and preview_exists and not needs_rename:
print(f"[*] 已跳过 (信息满足要求): {filename}")
return None
print(f"\n---> 处理文件: {filename} (位于 {root})")
civitai_data = None
file_hash = None
reason = ""
if info_exists: # a rename, or a missing cover fetched from the saved record: no new lookup
civitai_data = read_local_info(old_base)
if civitai_data:
print(f"[*] 使用本地已有的信息")
if not civitai_data:
if is_unmatched(previous): # the hash from last time, if the file is unchanged
file_hash = sidecar_file_hash(previous, file_path, (previous.get("files") or [{}])[0])[0]
file_hash = file_hash or calculate_sha256(file_path)
civitai_data, reason = ask_civitai(args, state, file_hash)
# What was read from disk is not fetched again: only fresh Civitai data needs the model
# page's description and the cover.
fresh = civitai_data is not None and file_hash is not None
if not civitai_data:
if args.skip_local_metadata:
print(f"[*] Civitai 获取失败,且禁用了本地元数据解析。跳过 {filename}")
return {"status": "failed", "error": "no_local_metadata", "reason": reason}
civitai_data = local_record(file_path, root, file_hash, reason)
entry = {"status": "matched" if fresh else "inferred" if reason else "unchanged"}
if reason:
entry["reason"] = reason
model_id = civitai_data.get("modelId")
if fresh and model_id and model_id != -1 and not state["civitai_down"]:
description = fetch_model_description(model_id)
if description:
civitai_data["description"] = description
if not isinstance(civitai_data.get("model"), dict):
civitai_data["model"] = {}
civitai_data["model"]["description"] = description
model_name = sanitize_filename(civitai_data.get("model", {}).get("name", "UnknownModel"))
version_name = sanitize_filename(civitai_data.get("name", "UnknownVersion"))
if fresh:
entry["name"] = f"{civitai_data.get('model', {}).get('name', '')} · {civitai_data.get('name', '')}".strip(" ·")
entry["base"] = civitai_data.get("baseModel") or ""
new_filename = f"{model_name}_{version_name}.safetensors"
new_file_path = os.path.join(root, new_filename)
new_base = os.path.splitext(new_file_path)[0]
info_data = civitai_data
if file_hash:
info_data["anomalous_file_identity"] = computed_file_identity(file_path, file_hash)
# Civitai's name for the card; a name set in the model editor lives in
# <model>.anomalous.json and is shown instead.
if args.virtual_rename:
info_data["anomalous_custom_name"] = f"{model_name}_{version_name}"
if not args.dry_run:
with open(old_base + ".info", 'w', encoding='utf-8') as f:
json.dump(info_data, f, ensure_ascii=True, indent=4)
print(f"[+] 写入纯净版 Civitai 描述 -> .info")
else:
print(f"[Dry-Run] 拟生成标准描述信息 -> .info")
media_url = None
if not args.skip_media:
media_url = next((img.get("url") for img in civitai_data.get("images") or [] if img.get("url")), None)
civitai_covers, user_covers = cover_owners(old_base)
# Download when the data is fresh, or when the model has no cover at all.
if media_url and not state["civitai_down"] and (fresh or not (civitai_covers or user_covers)):
if not args.dry_run:
print(f"[*] 正在下载预览媒体...")
saved_path = download_media(media_url, old_base + ".civitai_bak")
if saved_path:
print(f"[+] 媒体下载成功 -> {os.path.basename(saved_path)}")
entry["cover"] = "kept" if user_covers else "new"
if user_covers:
# The user's cover always stays; Civitai's image is kept as the backup.
print(f"[*] 保留你自己的封面,Civitai 封面存为备份")
else:
ext = os.path.splitext(saved_path)[1]
preview_path = old_base + (ext if ext.startswith('.preview.') else f".preview{ext}")
# Earlier copies of Civitai's image under another name give way to the new one.
older = [c for c in civitai_covers if c != preview_path]
try:
if older:
move_to_trash(*older)
except Exception as e:
print(f"[-] 旧的 Civitai 封面没有移走: {e}")
shutil.copy2(saved_path, preview_path)
else:
print(f"[Dry-Run] 拟下载预览媒体...")
if args.skip_rename or not args.physical_rename:
print(f"[*] 物理重命名已跳过。仅保存 .info 及其可能包含的虚拟重命名。")
elif file_path != new_file_path and new_filename != filename:
if os.path.exists(new_file_path):
print(f"[*] 目标文件名已存在,正在验证内容是否完全相同: {filename}")
try:
files_identical = os.path.getsize(file_path) == os.path.getsize(new_file_path)
if files_identical:
files_identical = calculate_sha256(file_path) == calculate_sha256(new_file_path)
except OSError:
files_identical = False
if not files_identical:
print(f"\033[91m[-] 目标名称冲突但文件内容不同,已保留两个模型并跳过重命名: {filename}\033[0m")
return {**entry, "status": "failed", "error": "name_conflict", "renamed_to": new_file_path}
if not args.dry_run:
try:
print(f"[*] Hash 一致,把已确认的重复副本移到回收站: {filename}")
move_to_trash(file_path, *[old_base + ext for ext in SIDECAR_SUFFIXES])
entry["duplicate_of"] = new_file_path
except Exception as e:
print(f"[-] 重复副本没有移走,两份都保留 (可能文件被占用或这个盘没有回收站): {e}")
return {**entry, "status": "failed", "error": "duplicate_kept"}
else:
print(f"[Dry-Run] Hash 一致,拟删除重复副本及其附属文件: {filename}")
elif not args.dry_run:
os.rename(file_path, new_file_path)
rename_log[file_path] = new_file_path
for ext in SIDECAR_SUFFIXES:
if os.path.exists(old_base + ext):
os.replace(old_base + ext, new_base + ext)
print(f"[+] 物理重命名完成: {filename} ==> {new_filename}")
entry["renamed_to"] = new_file_path
else:
print(f"[Dry-Run] 拟物理重命名文件: {filename} ==> {new_filename}")
print(f"[Dry-Run] 拟连带重命名附属文件 (.info / .png 等)")
else:
print("[*] 文件名已符合规范,无需重命名。")
changed = entry["status"] != "unchanged" or set(entry) & {"cover", "renamed_to", "duplicate_of"}
return entry if changed else None
def main():
parser = argparse.ArgumentParser(description="ComfyUI 模型 Civitai 嗅探与重命名工具")
@@ -249,6 +357,9 @@ def main():
parser.add_argument("--target-files", type=str, default="", help="仅扫描逗号分隔的具体文件(相对路径)")
parser.add_argument("--folder-type", default="", help="由 ComfyUI 传入的模型目录类型,用于执行安全策略")
parser.add_argument("--progress-file", default="", help=argparse.SUPPRESS)
parser.add_argument("--targets-file", default="", help=argparse.SUPPRESS) # JSON list of file names
parser.add_argument("--report-file", default="", help=argparse.SUPPRESS) # one JSON line per changed model
parser.add_argument("--civitai-down", action="store_true", help=argparse.SUPPRESS) # an earlier folder could not reach it
args = parser.parse_args()
if args.offline_only:
# Offline means no network at all: previews and model pages come from Civitai.
@@ -303,22 +414,21 @@ def main():
# 模式二:正常嗅探与重命名
# ==========================
rename_log = {}
success_count = 0
fail_count = 0
if os.path.exists(backup_log_path):
with open(backup_log_path, 'r', encoding='utf-8') as f:
rename_log = json.load(f)
target_files_basenames = []
target_file_path = os.path.join(target_folder, '.scan_targets.json')
if os.path.exists(target_file_path):
for targets_path in (os.path.join(target_folder, '.scan_targets.json'), args.targets_file):
if not targets_path or not os.path.exists(targets_path):
continue
try:
with open(target_file_path, 'r', encoding='utf-8') as f:
target_files_basenames = [os.path.basename(t.strip()) for t in __import__('json').load(f)]
os.remove(target_file_path)
except:
with open(targets_path, 'r', encoding='utf-8') as f:
target_files_basenames += [os.path.basename(t.strip()) for t in json.load(f)]
if targets_path != args.targets_file: # the scanner removes its own file
os.remove(targets_path)
except (OSError, ValueError, AttributeError):
pass
if args.target_files:
target_files_basenames.extend([os.path.basename(t.strip()) for t in args.target_files.split(',')])
@@ -327,22 +437,25 @@ def main():
print("==================================================")
print("[警告]: 当前处于 Dry-Run (空跑) 模式,不会修改系统中的任何文件!")
print("==================================================")
if not CIVITAI_API_KEY and not args.offline_only:
print("[!] 未配置 Civitai API Key。部分限制级模型或将无法获取图片。")
write_scan_progress(args.progress_file, "enumerating")
total_files = count_scan_files(target_folder, target_files_basenames, args.retry_unmatched)
write_scan_progress(args.progress_file, "scanning", total_files)
current_file = 0
last_progress_write = 0.0
state = {"network_failures": 0, "civitai_down": args.civitai_down}
counts = {"success": 0, "fail": 0, "unchanged": 0}
for root, _, files in os.walk(target_folder):
for filename in files:
if not filename.endswith(".safetensors"):
continue
if target_files_basenames and filename not in target_files_basenames:
continue
previous_info = read_local_info(os.path.join(root, os.path.splitext(filename)[0])) if args.retry_unmatched else None
if args.retry_unmatched and not is_unmatched(previous_info):
previous = read_local_info(os.path.join(root, os.path.splitext(filename)[0]))
if args.retry_unmatched and not is_unmatched(previous):
continue
current_file += 1
@@ -351,218 +464,28 @@ def main():
write_scan_progress(args.progress_file, "scanning", total_files, current_file, filename)
last_progress_write = now
file_path = os.path.join(root, filename)
old_base = os.path.splitext(file_path)[0]
# A .civitai.info holding only edits from older versions is not a scan result.
info_exists = scan_info_path(old_base) is not None
if args.force_overwrite or args.retry_unmatched:
info_exists = False
preview_exists = args.skip_media
if not preview_exists:
for ext in COVER_SUFFIXES:
if os.path.exists(old_base + ext):
preview_exists = True
break
needs_rename = False
if not args.skip_rename and args.physical_rename:
needs_rename = True
elif args.virtual_rename:
needs_rename = True
if info_exists and preview_exists and not needs_rename:
print(f"[*] 已跳过 (信息满足要求): {filename}")
try:
entry = scan_one(args, state, rename_log, root, filename, previous)
except Exception as e: # one model's trouble does not end the scan of the others
print(f"\033[91m[-] 处理失败,已跳过: {filename}: {e}\033[0m")
entry = {"status": "failed", "error": str(e)[:300]}
if entry is None:
counts["unchanged"] += 1
continue
print(f"\n---> 处理文件: {filename} (位于 {root})")
civitai_data = None
file_hash = None
if info_exists and needs_rename:
info_path = scan_info_path(old_base)
try:
with open(info_path, 'r', encoding='utf-8') as f:
civitai_data = json.load(f)
print(f"[*] 本地信息存在,直接进入重命名流程")
except:
pass
if not civitai_data:
if previous_info: # --retry-unmatched: the hash from last time, if the file is unchanged
file_hash = sidecar_file_hash(previous_info, file_path, (previous_info.get("files") or [{}])[0])[0]
file_hash = file_hash or calculate_sha256(file_path)
if not args.offline_only:
civitai_data = fetch_civitai_info(file_hash)
counts["fail" if entry["status"] == "failed" else "success"] += 1
write_report(args, file=os.path.join(root, filename), **entry)
# Fallback 3: Local Offline Inference (if Civitai still fails or offline_only)
if not civitai_data:
if args.skip_local_metadata:
print(f"[*] Civitai 获取失败,且禁用了本地元数据解析。跳过 {filename}")
fail_count += 1
continue
# 尝试离线推断底模
inferred_base = infer_base_model_from_header(file_path)
if inferred_base == 'Unknown':
inferred_base = ""
print(f"[*] 使用本地哈希重建基础元数据 ({filename})")
civitai_data = {
"id": -1,
"modelId": -1,
"name": os.path.splitext(filename)[0],
"baseModel": inferred_base,
"description": "<p>Automatically inferred by Anomalous Local Engine.</p>",
"model": {
"name": os.path.splitext(filename)[0],
"type": "LORA" if "lora" in root.lower() else "Checkpoint"
},
"files": [{"hashes": {"SHA256": file_hash}}]
}
# --- 额外获取模型主页的说明文字 ---
# What was read from disk is not fetched again: only fresh Civitai data needs the
# model page's description and the cover.
fresh = file_hash is not None and not args.offline_only
model_id = civitai_data.get("modelId")
if fresh and model_id and model_id != -1:
try:
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}
if CIVITAI_API_KEY:
headers["Authorization"] = f"Bearer {CIVITAI_API_KEY}"
req = urllib.request.Request(f"https://civitai.com/api/v1/models/{model_id}", headers=headers)
with urllib.request.urlopen(req, timeout=10) as m_resp:
m_data = json.loads(m_resp.read().decode('utf-8'))
if "description" in m_data and m_data["description"]:
civitai_data["description"] = m_data["description"]
if "model" not in civitai_data or not isinstance(civitai_data["model"], dict):
civitai_data["model"] = {}
civitai_data["model"]["description"] = m_data["description"]
except Exception as e:
print(f"[-] 获取模型主页详细说明失败: {e}")
# ----------------------------------
model_name = sanitize_filename(civitai_data.get("model", {}).get("name", "UnknownModel"))
version_name = sanitize_filename(civitai_data.get("name", "UnknownVersion"))
new_filename = f"{model_name}_{version_name}.safetensors"
new_file_path = os.path.join(root, new_filename)
new_base = os.path.splitext(new_file_path)[0]
# ==========================================
# 兼容性大刀阔斧改革:直接保存全宇宙最原汁原味的格式
# ==========================================
info_data = civitai_data
if file_hash:
info_data["anomalous_file_identity"] = computed_file_identity(file_path, file_hash)
# Civitai's name for the card; a name set in the model editor lives in
# <model>.anomalous.json and is shown instead.
if args.virtual_rename:
info_data["anomalous_custom_name"] = f"{model_name}_{version_name}"
if not args.dry_run:
info_path = old_base + ".info"
with open(info_path, 'w', encoding='utf-8') as f:
json.dump(info_data, f, ensure_ascii=True, indent=4)
print(f"[+] 写入纯净版 Civitai 描述 -> .info")
else:
print(f"[Dry-Run] 拟生成标准描述信息 -> .info")
media_url = None
images = civitai_data.get("images", [])
if images and len(images) > 0:
for img_obj in images:
if not args.skip_media:
if img_obj.get("url"):
media_url = img_obj.get("url")
break
civitai_covers, user_covers = cover_owners(old_base)
# Download when the data is fresh, or when the model has no cover at all.
if media_url and not args.skip_media and (fresh or not (civitai_covers or user_covers)):
if not args.dry_run:
print(f"[*] 正在下载预览媒体...")
saved_path = download_media(media_url, old_base + ".civitai_bak")
if saved_path:
print(f"[+] 媒体下载成功 -> {os.path.basename(saved_path)}")
if user_covers:
# The user's cover always stays; Civitai's image is kept as the backup.
print(f"[*] 保留你自己的封面,Civitai 封面存为备份")
else:
ext = os.path.splitext(saved_path)[1]
preview_path = old_base + (ext if ext.startswith('.preview.') else f".preview{ext}")
# Earlier copies of Civitai's image under another name give way to the new one.
older = [c for c in civitai_covers if c != preview_path]
try:
if older:
move_to_trash(*older)
except Exception as e:
print(f"[-] 旧的 Civitai 封面没有移走: {e}")
shutil.copy2(saved_path, preview_path)
else:
print(f"[Dry-Run] 拟下载预览媒体...")
if args.skip_rename or not args.physical_rename:
print(f"[*] 物理重命名已跳过。仅保存 .info 及其可能包含的虚拟重命名。")
success_count += 1
elif file_path != new_file_path and new_filename != filename:
if os.path.exists(new_file_path):
print(f"[*] 目标文件名已存在,正在验证内容是否完全相同: {filename}")
try:
files_identical = os.path.getsize(file_path) == os.path.getsize(new_file_path)
if files_identical:
files_identical = calculate_sha256(file_path) == calculate_sha256(new_file_path)
except OSError:
files_identical = False
if not files_identical:
print(f"\033[91m[-] 目标名称冲突但文件内容不同,已保留两个模型并跳过重命名: {filename}\033[0m")
fail_count += 1
elif not args.dry_run:
try:
print(f"[*] Hash 一致,把已确认的重复副本移到回收站: {filename}")
move_to_trash(file_path, *[old_base + ext for ext in SIDECAR_SUFFIXES])
success_count += 1
except Exception as e:
print(f"[-] 重复副本没有移走,两份都保留 (可能文件被占用或这个盘没有回收站): {e}")
fail_count += 1
else:
print(f"[Dry-Run] Hash 一致,拟删除重复副本及其附属文件: {filename}")
else:
if not args.dry_run:
os.rename(file_path, new_file_path)
rename_log[file_path] = new_file_path
for ext in SIDECAR_SUFFIXES:
old_ext = old_base + ext
new_ext = new_base + ext
if os.path.exists(old_ext):
os.replace(old_ext, new_ext)
print(f"[+] 物理重命名完成: {filename} ==> {new_filename}")
success_count += 1
else:
print(f"[Dry-Run] 拟物理重命名文件: {filename} ==> {new_filename}")
print(f"[Dry-Run] 拟连带重命名附属文件 (.info / .png 等)")
else:
print("[*] 文件名已符合规范,无需重命名。")
success_count += 1
# Save scan results
write_scan_progress(args.progress_file, "complete", total_files, total_files)
write_report(args, event="done", unchanged=counts["unchanged"], civitai_down=state["civitai_down"])
if not args.dry_run:
result_path = os.path.join(target_folder, ".scan_result.json")
try:
with open(result_path, 'w', encoding='utf-8') as f:
json.dump({"success": success_count, "fail": fail_count}, f)
json.dump({"success": counts["success"], "fail": counts["fail"]}, f)
except Exception as e:
print(f"[-] 保存统计结果失败: {e}")
if not args.dry_run and rename_log:
with open(backup_log_path, 'w', encoding='utf-8') as f:
json.dump(rename_log, f, ensure_ascii=True, indent=4)
+33 -1
View File
@@ -5,13 +5,14 @@
*/
import { translate as t } from './locales.js';
import { countsLine, fileOutcome, scanScope, STATUS_MARKS } from './scan_results.js';
// Server action id → locale key.
const ACTION_KEYS = {
model_delete: 'activityModelDelete',
model_edit: 'activityModelEdit',
model_cover: 'activityModelCover',
scan: 'activityScan',
scan: 'activityScan', // entries from before scans were recorded with their result
recipe_save: 'activityRecipeSave',
recipe_edit: 'activityRecipeEdit',
recipe_delete: 'activityRecipeDelete',
@@ -70,10 +71,41 @@ export function entrySummary(entry) {
if (opened) return t('activityCanvasOpened', { name: opened.node || '—', count: opened.after || '?' });
return t('activityCanvasChanged', { count: entry.detail?.total || changes.length });
}
if (entry.action === 'scan_done') {
const scan = entry.detail?.scan || {};
return t('activityScanDone', { scope: scanScope(scan, entry.target), counts: countsLine(scan.counts) });
}
const words = t(ACTION_KEYS[entry.action] || 'activityUnknown');
return entry.target ? t('activityWithTarget', { action: words, target: entry.target }) : words;
}
const FIELD_KEYS = {
custom_name: 'modelSourceFieldName', custom_notes: 'modelSourceFieldNotes',
source_url: 'modelSourceFieldLink', filename: 'activityFieldFilename',
};
/**
* The lines a file entry opens to: [{text, model?}], `model` being a scanned model the line
* can open ({type, path_idx, rel, filename}). Empty for entries without details.
*/
export function fileDetailLines(entry) {
const detail = entry.detail || {};
const empty = t('activityEmptyValue');
const lines = [];
if (detail.scan?.civitai_down) lines.push({ text: t('scanResultCivitaiDown') });
for (const error of detail.scan?.errors || []) lines.push({ text: error });
for (const item of detail.files || []) {
if (typeof item === 'string') lines.push({ text: t('activityFileMoved', { name: item }) });
else lines.push({ text: `${STATUS_MARKS[item.status] || '·'} ${item.filename}: ${fileOutcome(item)}`, model: item.rel ? item : null });
}
for (const field of detail.fields || []) {
lines.push({ text: t('activityFieldChange', {
field: t(FIELD_KEYS[field.field] || field.field), before: field.before || empty, after: field.after || empty,
}) });
}
return lines;
}
/** One line per canvas change (not for `opened`). */
export function changeLine(change) {
const empty = t('activityEmptyValue');
+118 -14
View File
@@ -289,19 +289,18 @@ export const i18n = {
collapseAll: '收起全部',
expandAll: '展开全部',
scanPageTitle: "扫描模型",
scanPageLead: "读取模型文件夹里的 .safetensors 模型,按文件哈希从 Civitai 获取封面、触发词和底模。已经扫过的模型不会重复下载。",
scanPageLead: "读取模型文件夹里的 .safetensors 模型,按文件哈希从 Civitai 获取封面、触发词和底模;Civitai 上找不到的,从模型文件推断底模。已经扫过的模型不会重复下载。",
scanPageTotal: "模型总数",
scanPageMatched: "已匹配 Civitai",
scanPageUnmatched: "没匹配上",
scanPageNew: "还没扫",
scanPageSummaryFailed: "暂时读不到模型数量,仍然可以扫描。",
scanPageScanNew: "扫描 {count} 个新模型",
scanPageScanNew: "扫描 {count} 个待查的模型",
scanPageScanAgain: "再检查一遍",
scanPageScanPicked: "扫描挑选的 {count} 个模型",
scanPageScanning: "正在扫描…",
scanPageRetry: "重新联网查找 {count} 个没匹配上的模型",
scanPageHint: "扫描在后台进行,可以关掉窗口继续用 ComfyUI,进度会显示在右下角。",
scanPageHintRetry: "“没匹配上”是上次扫描时 Civitai 没找到,或者当时连不上网。重新查找不会改动你的封面,文件没变时也不用重新计算哈希。扫描在后台进行,可以关掉窗口。",
scanPageRetry: "全部重新联网查找({count} 个没匹配上的)",
scanPageHint: "联网扫描先查 Civitai,查不到的从文件推断;连不上 Civitai 时会自动改成离线推断,下次再补查。扫描在后台进行,可以关掉窗口继续用 ComfyUI。",
scanPageHintPicked: "只扫描在高级选项里挑选的模型。",
scanPageAdvanced: "高级选项",
scanPageScope: "扫描哪些模型",
@@ -312,7 +311,7 @@ export const i18n = {
scanPageNothingPicked: "还没有挑选模型。",
scanPageHow: "怎么扫描",
scanPageOffline: "离线扫描",
scanPageOfflineHelp: "不联网,只从模型文件推断底模,不会有封面和触发词。网络不通时用。",
scanPageOfflineHelp: "不联网,只从模型文件推断底模,没有封面和触发词。之后联网扫描会自动补查这些模型。",
scanPageDisplayName: "用 Civitai 上的名称作为显示名",
scanPageDisplayNameHelp: "只改卡片上显示的名字,不改文件。你在编辑窗口里改过的名字会保留。",
scanPageFileRename: "同时修改磁盘上的文件名",
@@ -332,11 +331,60 @@ export const i18n = {
scanPageApiKeyCleared: "API Key 已清除。",
scanPageApiKeyFailed: "保存失败:{error}",
scanPageStartFailed: "扫描没能开始:{error}",
scanReasonNotFound: "Civitai 上没有这个文件",
scanReasonNetwork: "当时连不上 Civitai",
scanReasonOffline: "离线扫描,还没查 Civitai",
scanReasonUnknown: "Civitai 上没找到",
scanErrorNameConflict: "目标文件名已被另一个不同的模型占用,没有改名",
scanErrorDuplicateKept: "发现完全相同的副本,但没能移到回收站,两份都保留了",
scanCountMatched: "新匹配 {count}",
scanCountInferred: "从文件推断 {count}",
scanCountFailed: "失败 {count}",
scanCountCovers: "下载封面 {count}",
scanCountRenamed: "改名 {count}",
scanCountNothing: "没有需要更新的模型",
scanFileMatched: "已匹配 Civitai:{name}",
scanFileInferredBase: "{reason},底模从文件推断为 {base}",
scanFileInferred: "{reason},文件里也没读出底模",
scanFileFailed: "没有完成:{error}",
scanFileUpdated: "已更新",
scanFileRenamed: "原文件名 {from}",
scanFileDuplicate: "重复副本已移到回收站",
scanFileCoverKept: "保留了你的封面,Civitai 封面存为备份",
scanScopeOne: "扫描 {name}",
scanScopePicked: "扫描挑选的模型",
scanScopeRetry: "重新联网查找",
scanScopeAll: "扫描全部模型",
scanCivitaiDownShort: "中途连不上 Civitai",
scanOneUnchanged: "这个模型的信息已经是最新的",
scanOneTitle: "扫描:{name}",
scanModelGone: "找不到 {name},可能已被移动或删除。",
scanRowOpen: "查看",
scanRowRetry: "重新查找",
scanRowSearch: "去 Civitai 搜",
scanRowScan: "扫描",
scanShowAll: "显示全部 {count} 个",
scanResultTitle: "上次扫描 · {when} · {scope} · 用时 {duration}",
scanSeconds: "{count} 秒",
scanMinutes: "{count} 分钟",
scanResultOffline: "这是离线扫描:没有查 Civitai,下次联网扫描会补查这些模型。",
scanResultCivitaiDown: "扫描中途连不上 Civitai,之后的模型先从文件推断了底模。网络恢复后,下次扫描会自动补查它们。",
scanResultNothing: "没有模型需要更新({count} 个已是最新)。",
scanResultUnchanged: "另有 {count} 个模型已是最新,没有改动。",
scanListNew: "还没扫 · {count}",
scanListUnmatched: "没匹配上 · {count}",
scanListNewHint: "这些模型还没扫描过,卡片上标着“未扫描”。可以逐个扫描,也可以用上面的按钮一起扫。",
scanListUnmatchedHint: "Civitai 上没找到这些模型,底模是从文件推断的(卡片上的“≈”)。“当时连不上”和“离线扫描”的,下次联网扫描会自动补查;Civitai 上确实没有的(比如自己训练、合并或从别处下载的),可以去 Civitai 搜一下,或在编辑窗口里填上下载地址。",
scanListUnmatchedHintOffline: "Civitai 上没找到这些模型,底模是从文件推断的。离线扫描时不能重新查找。",
scanPageShowList: "查看是哪些模型",
scanPageScanNewOffline: "离线扫描 {count} 个新模型",
scanPageHintOffline: "离线扫描只读模型文件推断底模,不联网,没有封面和触发词。之后联网扫描会自动补查这些模型。",
scanProgressShowResult: "查看结果",
modelSourceLabel: "来源",
modelSourceCivitai: "Civitai",
modelSourceLocal: "从文件推断",
modelSourceLocalShort: "本地",
modelSourceLocalHint: "底模是从模型文件推断的:Civitai 上没找到这个模型,或者扫描时连不上 Civitai。可以在扫描页“重新联网查找”。",
modelSourceLocalHint: "{reason},底模是从模型文件推断的。可以在扫描页重新联网查找。",
modelSourceNotScanned: "未扫描",
modelSourceNotScannedHint: "还没扫描:到扫描页补全封面、触发词和底模。",
modelSourceEdited: "你改过:{fields}",
@@ -860,6 +908,10 @@ export const i18n = {
activityModelEdit: "修改模型信息",
activityModelCover: "更换模型封面",
activityScan: "扫描模型",
activityScanDone: "{scope}:{counts}",
activityFieldFilename: "文件名",
activityFileMoved: "移到回收站:{name}",
activityFieldChange: "{field}:{before} → {after}",
activityRecipeSave: "保存工作流配方",
activityRecipeEdit: "修改配方",
activityRecipeDelete: "删除配方",
@@ -2121,19 +2173,18 @@ export const i18n = {
collapseAll: 'Collapse All',
expandAll: 'Expand All',
scanPageTitle: "Scan models",
scanPageLead: "Reads the .safetensors models in your model folders and fetches covers, trigger words and base models from Civitai by file hash. Models already scanned are not downloaded again.",
scanPageLead: "Reads the .safetensors models in your model folders and fetches covers, trigger words and base models from Civitai by file hash; for models Civitai does not know, the base model is inferred from the file. Models already scanned are not downloaded again.",
scanPageTotal: "Models",
scanPageMatched: "Matched on Civitai",
scanPageUnmatched: "Not matched",
scanPageNew: "Not scanned yet",
scanPageSummaryFailed: "Couldn't count the models right now; you can still scan.",
scanPageScanNew: "Scan {count} new models",
scanPageScanNew: "Scan {count} models not looked up yet",
scanPageScanAgain: "Check again",
scanPageScanPicked: "Scan the {count} picked models",
scanPageScanning: "Scanning…",
scanPageRetry: "Look up the {count} unmatched models again",
scanPageHint: "The scan runs in the background: you can close this window and keep using ComfyUI; progress shows at the bottom right.",
scanPageHintRetry: "“Not matched” models were not found on Civitai last time, or Civitai could not be reached then. Looking them up again keeps your covers and, for unchanged files, reuses the saved hash. Scans run in the background; you can close this window.",
scanPageRetry: "Look up all {count} unmatched models again",
scanPageHint: "An online scan asks Civitai first and infers from the file what Civitai does not know; when Civitai cannot be reached it falls back to the file and looks again next time. Scans run in the background; you can close this window and keep using ComfyUI.",
scanPageHintPicked: "Only the models picked under Advanced are scanned.",
scanPageAdvanced: "Advanced",
scanPageScope: "Which models",
@@ -2144,7 +2195,7 @@ export const i18n = {
scanPageNothingPicked: "No models picked yet.",
scanPageHow: "How to scan",
scanPageOffline: "Offline scan",
scanPageOfflineHelp: "No internet: only the base model is inferred from the file; no covers or trigger words. For when you are offline.",
scanPageOfflineHelp: "No internet: only the base model is inferred from the file; no covers or trigger words. The next online scan looks these models up.",
scanPageDisplayName: "Use Civitai's name as the display name",
scanPageDisplayNameHelp: "Only the name shown on cards; files keep their names. Names you set in the model editor are kept.",
scanPageFileRename: "Also rename the files on disk",
@@ -2164,11 +2215,60 @@ export const i18n = {
scanPageApiKeyCleared: "API key cleared.",
scanPageApiKeyFailed: "Could not save: {error}",
scanPageStartFailed: "The scan could not start: {error}",
scanReasonNotFound: "Civitai does not know this file",
scanReasonNetwork: "Civitai could not be reached then",
scanReasonOffline: "Offline scan, Civitai not asked yet",
scanReasonUnknown: "Not found on Civitai",
scanErrorNameConflict: "a different model already has the target file name; not renamed",
scanErrorDuplicateKept: "an identical copy could not be moved to the Recycle Bin; both kept",
scanCountMatched: "{count} matched",
scanCountInferred: "{count} inferred from the file",
scanCountFailed: "{count} failed",
scanCountCovers: "{count} covers downloaded",
scanCountRenamed: "{count} renamed",
scanCountNothing: "No model needed an update",
scanFileMatched: "Matched on Civitai: {name}",
scanFileInferredBase: "{reason}; base model inferred from the file: {base}",
scanFileInferred: "{reason}; the file does not tell its base model either",
scanFileFailed: "Not done: {error}",
scanFileUpdated: "Updated",
scanFileRenamed: "was {from}",
scanFileDuplicate: "duplicate copy moved to the Recycle Bin",
scanFileCoverKept: "your cover kept, Civitai’s saved as the backup",
scanScopeOne: "Scanned {name}",
scanScopePicked: "Scanned the picked models",
scanScopeRetry: "Looked up again",
scanScopeAll: "Scanned all models",
scanCivitaiDownShort: "Civitai stopped answering",
scanOneUnchanged: "This model’s information is already up to date",
scanOneTitle: "Scanning: {name}",
scanModelGone: "{name} is not there any more; it may have been moved or deleted.",
scanRowOpen: "Open",
scanRowRetry: "Look up again",
scanRowSearch: "Search Civitai",
scanRowScan: "Scan",
scanShowAll: "Show all {count}",
scanResultTitle: "Last scan · {when} · {scope} · took {duration}",
scanSeconds: "{count} s",
scanMinutes: "{count} min",
scanResultOffline: "This was an offline scan: Civitai was not asked; the next online scan looks these models up.",
scanResultCivitaiDown: "Civitai stopped answering during the scan, so the base model of the models after that was inferred from their files. The next scan looks them up once the connection is back.",
scanResultNothing: "No model needed an update ({count} already up to date).",
scanResultUnchanged: "{count} more models were already up to date.",
scanListNew: "Not scanned · {count}",
scanListUnmatched: "Not matched · {count}",
scanListNewHint: "These models have not been scanned; their cards say “Not scanned”. Scan them one by one, or all with the button above.",
scanListUnmatchedHint: "Civitai did not find these models; their base model was inferred from the file (“≈” on the card). Those Civitai could not be asked about (no connection, offline scan) are looked up by the next online scan. For models Civitai really does not have (your own, merges, from elsewhere), search Civitai or set the download link in the model editor.",
scanListUnmatchedHintOffline: "Civitai did not find these models; their base model was inferred from the file. Looking them up again needs an online scan.",
scanPageShowList: "Show which models",
scanPageScanNewOffline: "Scan {count} new models offline",
scanPageHintOffline: "An offline scan only reads the files to infer the base model: no internet, no covers or trigger words. The next online scan looks these models up.",
scanProgressShowResult: "Show the result",
modelSourceLabel: "Source",
modelSourceCivitai: "Civitai",
modelSourceLocal: "Inferred from the file",
modelSourceLocalShort: "Local",
modelSourceLocalHint: "The base model was inferred from the file: Civitai does not know this model, or could not be reached during the scan. Try \"Look up again\" on the scan page.",
modelSourceLocalHint: "{reason}; the base model was inferred from the file. You can look it up again on the scan page.",
modelSourceNotScanned: "Not scanned",
modelSourceNotScannedHint: "Not scanned yet: the scan page fills in cover, trigger words and base model.",
modelSourceEdited: "Edited by you: {fields}",
@@ -2692,6 +2792,10 @@ export const i18n = {
activityModelEdit: "Edited model info",
activityModelCover: "Changed model cover",
activityScan: "Scanned models",
activityScanDone: "{scope}: {counts}",
activityFieldFilename: "file name",
activityFileMoved: "Moved to the Recycle Bin: {name}",
activityFieldChange: "{field}: {before} → {after}",
activityRecipeSave: "Saved workflow recipe",
activityRecipeEdit: "Edited recipe",
activityRecipeDelete: "Deleted recipe",
+3 -2
View File
@@ -7,6 +7,7 @@
import { translate as t } from './locales.js';
import { escapeHtml } from './safe_dom.js';
import { reasonText } from './scan_results.js';
const FIELD_KEYS = { custom_name: 'modelSourceFieldName', custom_notes: 'modelSourceFieldNotes', source_url: 'modelSourceFieldLink' };
@@ -32,7 +33,7 @@ export function renderSourceBadge(model) {
if (meta.info_source === 'local') {
badge.classList.add('is-inferred', ...(baseModel ? [colourClass(baseModel)] : []));
badge.textContent = `≈ ${baseModel || t('modelSourceLocalShort')}`;
badge.title = t('modelSourceLocalHint');
badge.title = t('modelSourceLocalHint', { reason: reasonText(meta.unmatched_reason) });
} else if (!meta.info_source && scannable(model)) {
badge.classList.add('is-unscanned');
badge.textContent = t('modelSourceNotScanned');
@@ -57,6 +58,6 @@ export function renderSourceLine(span, model, meta) {
const edited = (meta.user_fields || []).map(field => t(FIELD_KEYS[field] || field));
if (edited.length) parts.push(escapeHtml(t('modelSourceEdited', { fields: edited.join(t('modelSourceListSep')) })));
span.innerHTML = parts.join('<span class="anomalous-source-sep"> · </span>');
span.title = meta.info_source === 'local' ? t('modelSourceLocalHint')
span.title = meta.info_source === 'local' ? t('modelSourceLocalHint', { reason: reasonText(meta.unmatched_reason) })
: !meta.info_source && scannable(model) ? t('modelSourceNotScannedHint') : '';
}
+18 -5
View File
@@ -138,17 +138,18 @@ export function updateScanProgress(status, titleText = '') {
}
function closeLater(panel) {
function closeLater(panel, ms = 4000) {
if (closeTimer) clearTimeout(closeTimer);
if (host) return; // on the scan page the result stays until the page is left
closeTimer = setTimeout(() => {
if (panel === current) removePanel();
closeTimer = null;
}, 4000);
}, ms);
}
export function finishScanProgress(detailText = '') {
/** The scan has ended. `action` ({label, onClick}): a button while the box floats, e.g. to the result. */
export function finishScanProgress(detailText = '', action = null) {
const panel = ensurePanel();
panel.classList.remove('is-error');
panel.classList.add('is-complete');
@@ -166,8 +167,20 @@ export function finishScanProgress(detailText = '') {
panel.querySelector('.anomalous-scan-progress-detail').textContent = translate('scanProgressFinished');
}
}
panel.querySelector('.anomalous-scan-progress-item').textContent = '';
closeLater(panel);
const item = panel.querySelector('.anomalous-scan-progress-item');
item.textContent = '';
if (action && !host) {
const open = document.createElement('button');
open.type = 'button';
open.className = 'anomalous-scan-progress-action';
open.textContent = action.label;
open.onclick = () => {
removePanel();
action.onClick();
};
item.appendChild(open);
}
closeLater(panel, action && !host ? 12000 : 4000);
}
+68
View File
@@ -0,0 +1,68 @@
/**
* Words for what a scan found (the result GET /anomalous/last_scan returns, also kept in the
* activity log): a model's outcome, why an unmatched model has no Civitai record, a scan's
* counts. Shared by the scan page, the activity page and the progress box. No DOM.
*/
import { translate as t } from './locales.js';
const REASON_KEYS = { not_found: 'scanReasonNotFound', network: 'scanReasonNetwork', offline: 'scanReasonOffline' };
const ERROR_KEYS = { name_conflict: 'scanErrorNameConflict', duplicate_kept: 'scanErrorDuplicateKept' };
const COUNT_KEYS = [
['matched', 'scanCountMatched'], ['inferred', 'scanCountInferred'], ['failed', 'scanCountFailed'],
['covers', 'scanCountCovers'], ['renamed', 'scanCountRenamed'],
];
/** The mark before a model's line. */
export const STATUS_MARKS = { matched: '✓', inferred: '≈', failed: '✕', unchanged: '·' };
/** Why an unmatched model has no Civitai record. */
export function reasonText(reason) {
return t(REASON_KEYS[reason] || 'scanReasonUnknown');
}
/** Models whose lookup did not happen (offline, or no connection): the next online scan asks. */
export function isPending(reason) {
return reason === 'offline' || reason === 'network';
}
/** "3 matched · 1 inferred …", the parts that are not zero. */
export function countsLine(counts = {}) {
const parts = COUNT_KEYS.filter(([key]) => counts[key]).map(([key, label]) => t(label, { count: counts[key] }));
return parts.length ? parts.join(' · ') : t('scanCountNothing');
}
/** What happened to one model in a scan. */
export function fileOutcome(item) {
let text;
if (item.status === 'matched') text = t('scanFileMatched', { name: item.name || item.filename });
else if (item.status === 'inferred') {
text = item.base ? t('scanFileInferredBase', { reason: reasonText(item.reason), base: item.base })
: t('scanFileInferred', { reason: reasonText(item.reason) });
} else if (item.status === 'failed') text = t('scanFileFailed', { error: ERROR_KEYS[item.error] ? t(ERROR_KEYS[item.error]) : item.error || '?' });
else text = t('scanFileUpdated');
if (item.renamed_from) text += ` · ${t('scanFileRenamed', { from: item.renamed_from })}`;
if (item.duplicate) text += ` · ${t('scanFileDuplicate')}`;
if (item.cover === 'kept') text += ` · ${t('scanFileCoverKept')}`;
return text;
}
/** Which models a scan covered: all, picked, or one (`target`: its file name). */
export function scanScope(scan, target = '') {
if (scan?.kind === 'one') return t('scanScopeOne', { name: target || '?' });
if (scan?.kind === 'picked') return t('scanScopePicked');
return t(scan?.options?.retry_unmatched ? 'scanScopeRetry' : 'scanScopeAll');
}
/** The progress box's line when a scan has ended. */
export function resultLine(result) {
if (!result?.counts) return '';
if (result.kind === 'one') return oneModelLine(result);
return result.civitai_down ? `${countsLine(result.counts)} · ${t('scanCivitaiDownShort')}` : countsLine(result.counts);
}
/** After one model's scan (its card's radar button). */
export function oneModelLine(result) {
const item = result?.files?.[0];
return item ? fileOutcome(item) : t('scanOneUnchanged');
}
+89 -147
View File
@@ -1,13 +1,15 @@
/**
* Starting scans and following them: a scan of every model folder (or of picked models)
* from the scan page, and the precision scan of one model from its card. Progress goes
* to scan_progress.js; when a scan ends, node drop-downs, hashes and the model grid are
* refreshed. No page DOM here: ui_scan_page.js renders the page.
* Starting scans and following them: every model folder, picked models or listed models from
* the scan page (all through /anomalous/scan_all), and one model from its card. Progress goes
* to scan_progress.js; when a scan ends its result (GET /anomalous/last_scan) is shown, and
* node drop-downs, hashes and the model grid are refreshed. No page DOM here: ui_scan_page.js
* renders the page.
*/
import { app } from '../../../scripts/app.js';
import { translate as t } from './locales.js';
import { updateScanProgress, finishScanProgress, failScanProgress } from './scan_progress.js';
import { oneModelLine, resultLine } from './scan_results.js';
import { showWorkbenchToast } from './ui_prompt_toast.js';
const POLL_MS = 2000;
@@ -50,6 +52,31 @@ export function scanRequestBody(options) {
};
}
/** Listed models ({type, path_idx, rel}) as the scan's `targets`, one per folder. */
export function targetsForItems(items) {
const folders = new Map();
for (const item of items) {
const cut = item.rel.lastIndexOf('/');
const subfolder = cut < 0 ? '/' : `/${item.rel.slice(0, cut)}`;
const key = `${item.type}|${item.path_idx}|${subfolder}`;
if (!folders.has(key)) folders.set(key, { type: item.type, path_idx: item.path_idx, subfolder, files: [] });
folders.get(key).files.push(item.filename);
}
return [...folders.values()];
}
/** The model picker's selection (Map "type|path_idx|subfolder" -> Set of file names) as `targets`. */
function targetsForSelection(selection) {
const targets = [];
for (const [folderKey, files] of selection.entries()) {
const [type, pathIdx, ...rest] = folderKey.split('|');
if (type && pathIdx !== undefined && rest.length && files.size) {
targets.push({ type, path_idx: Number(pathIdx), subfolder: rest.join('|'), files: [...files] });
}
}
return targets;
}
async function postJson(url, body) {
const res = await fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
const data = await res.json().catch(() => ({}));
@@ -57,13 +84,22 @@ async function postJson(url, body) {
return data;
}
/** The last scan's result ({} before the first, or when it cannot be read). */
export async function fetchLastScan() {
try {
return await (await fetch('/anomalous/last_scan')).json();
} catch {
return {};
}
}
/** Polls a status URL until the scan there stops; resolves with the last status. */
function followScan(statusUrl, extra = {}) {
function followScan(statusUrl, extra = {}, everyMs = POLL_MS, title = '') {
return new Promise((resolve, reject) => {
const poll = setInterval(async () => {
try {
const status = await (await fetch(statusUrl)).json();
updateScanProgress({ ...status, ...extra });
updateScanProgress({ ...status, ...extra }, title);
if (!status.scanning) {
clearInterval(poll);
resolve(status);
@@ -72,7 +108,7 @@ function followScan(statusUrl, extra = {}) {
clearInterval(poll);
reject(error);
}
}, POLL_MS);
}, everyMs);
});
}
@@ -87,141 +123,54 @@ async function afterScan(owner, autoFix) {
owner.loadModels?.();
}
/** Picked models: one folder after the other (`selection`: Map "type|path_idx|subfolder" -> Set of file names). */
async function scanSelection(selection, body) {
const folders = [...selection.entries()].filter(([, files]) => files.size > 0);
let current = 0;
updateScanProgress({ scanning: true, phase: 'preparing', folder_total: folders.length, folder_current: 0 });
let last = {};
for (const [folderKey, files] of folders) {
current += 1;
const [type, pathIdx, ...rest] = folderKey.split('|');
const subfolder = rest.join('|');
if (!type || pathIdx === undefined || !rest.length) continue;
const params = new URLSearchParams({ type, path_idx: pathIdx, subfolder });
try {
await postJson(`/anomalous/scan?${params}`, { ...body, target_files: [...files] });
last = await followScan(`/anomalous/scan_status?${params}`, { folder_total: folders.length, folder_current: current, folder: subfolder });
} catch (error) {
console.error('[AMB] Scan failed for folder:', folderKey, error);
last = { error: String(error.message || error) };
}
}
return last;
/** The ended scan's line, with a way to its result when the progress box floats. */
export function showScanResult(owner, result) {
finishScanProgress(resultLine(result), {
label: t('scanProgressShowResult'),
onClick: () => {
owner.show?.();
owner.openScanPage?.();
},
});
}
/**
* Runs one scan from the scan page: every active model folder, or `options.selection`.
* Resolves when it has ended (true) or could not start (false, with the reason shown).
* Runs one scan from the scan page: every active model folder, `options.selection` (the model
* picker) or `options.targets` (listed models). Resolves with the scan's result when it has
* ended, or null when it could not start (the reason shown).
*/
export async function startScan(owner, options) {
if (running) return false;
if (running) return null;
running = true;
setActiveScanButtonState(true);
const body = scanRequestBody(options);
const targets = options.targets || (options.selection ? targetsForSelection(options.selection) : null);
if (targets) body.targets = targets;
try {
let status;
if (options.selection) {
status = await scanSelection(options.selection, body);
} else {
const data = await postJson('/anomalous/scan_all', body);
updateScanProgress({ scanning: true, phase: 'preparing', recovered: data.recovered });
status = await followScan('/anomalous/global_scan_status');
}
const data = await postJson('/anomalous/scan_all', body);
updateScanProgress({ scanning: true, phase: 'preparing', recovered: data.recovered });
const status = await followScan('/anomalous/global_scan_status');
const result = await fetchLastScan();
if (status.interrupted) failScanProgress(t('scanProgressInterrupted'));
else finishScanProgress();
else showScanResult(owner, result);
await afterScan(owner, options.autoFix);
return true;
return result;
} catch (error) {
failScanProgress(t('scanPageStartFailed', { error: String(error.message || error) }));
return false;
return null;
} finally {
running = false;
setActiveScanButtonState(false);
}
}
export async function formatScanCompletionToast(model) {
const isZh = window.anomalous_browser_lang === 'zh';
const fallbackName = model.name || model.filename;
try {
const findRes = await fetch('/anomalous/find_model?search=' + encodeURIComponent(model.filename));
if (findRes.ok) {
const updated = await findRes.json();
const target = updated?.model || updated || model;
if (target && target.metadata) {
const meta = target.metadata;
const isCivitai = Boolean(
(meta.id && meta.id !== -1) ||
(meta.modelId && meta.modelId !== -1) ||
(meta.model_id && meta.model_id !== -1) ||
(meta.version_id && meta.version_id !== -1) ||
meta.civitai_url
);
const hasPreview = Boolean(target.preview_url);
const baseModel = meta.baseModel;
if (isCivitai && hasPreview) {
return isZh ? `✓ 已从 Civitai 获取封面与模型信息!` : `✓ Civitai cover & metadata fetched!`;
}
if (isCivitai && !hasPreview) {
return isZh ? `✓ 已匹配到 Civitai 信息(线上未提供封面)` : `✓ Civitai metadata matched (no cover online)`;
}
if (!isCivitai) {
if (baseModel) {
return isZh
? `ℹ️ 非 Civitai 模型:已识别底模为 [${baseModel}]`
: `ℹ️ Non-Civitai model: inferred base model [${baseModel}]`;
}
return isZh
? `ℹ️ 未在 Civitai 匹配到此模型`
: `ℹ️ No Civitai match found for this model`;
}
}
}
} catch {
// fallback to standard text
}
return isZh ? `✓ 模型 [${fallbackName}] 扫描完成!` : `✓ Model [${fallbackName}] scanned!`;
}
function pollDirectScanStatus(params, titleText, model, onComplete) {
const statusUrl = '/anomalous/scan_status?' + params.toString();
const poll = setInterval(async () => {
try {
const statusRes = await fetch(statusUrl);
const statusData = await statusRes.json();
updateScanProgress(statusData, titleText);
if (!statusData.scanning) {
clearInterval(poll);
if (statusData.interrupted) {
failScanProgress(t('scanProgressInterrupted'));
showWorkbenchToast(window.anomalous_browser_lang === 'zh' ? '扫描被中断' : 'Scan interrupted');
} else {
const toastMsg = await formatScanCompletionToast(model);
finishScanProgress(toastMsg);
showWorkbenchToast(toastMsg);
}
onComplete(true);
}
} catch (err) {
clearInterval(poll);
failScanProgress(String(err));
onComplete(false);
}
}, 1200);
return poll;
}
/** The radar button on a model card: scan just that model, in its own folder. */
export async function triggerDirectModelScan(model, triggerBtn = null, browserInstance = null) {
const browser = browserInstance || this || {};
if (!model || !model.filename) return;
const modelLabel = model.name || model.filename;
const isZh = window.anomalous_browser_lang === 'zh';
const titleText = isZh ? `精准扫描: ${modelLabel}` : `Scanning: ${modelLabel}`;
const titleText = t('scanOneTitle', { name: modelLabel });
if (triggerBtn) {
triggerBtn.classList.add('anomalous-radar-spinning');
@@ -254,45 +203,38 @@ export async function triggerDirectModelScan(model, triggerBtn = null, browserIn
path_idx: browser.currentPathIdx || 0,
subfolder: browser.currentSubfolder || '/',
});
const reqBody = {
// A model that was looked up without an answer is looked up again.
await postJson(`/anomalous/scan?${params}`, {
target_files: [model.filename],
offline_only: false,
skip_rename: true,
virtual_rename: false,
physical_rename: false,
force_overwrite: false
};
const res = await fetch('/anomalous/scan?' + params.toString(), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reqBody)
force_overwrite: false,
retry_unmatched: model.metadata?.info_source === 'local',
});
const data = await res.json();
if (data.status === 'ok') {
showWorkbenchToast(isZh ? `开始精准扫描: ${modelLabel}` : `Scanning model: ${modelLabel}`);
pollDirectScanStatus(params, titleText, model, async () => {
resetBtn();
if (typeof browser.loadModels === 'function') {
browser.loadModels();
}
try {
if (app?.refreshComboInNodes) await app.refreshComboInNodes();
if (window.anomalous_reload_hashes) await window.anomalous_reload_hashes();
} catch (e) {
console.warn('[AMB] Error reloading hashes or combo nodes:', e);
}
});
showWorkbenchToast(titleText);
const status = await followScan(`/anomalous/scan_status?${params}`, {}, 1200, titleText);
resetBtn();
if (status.interrupted) {
failScanProgress(t('scanProgressInterrupted'));
showWorkbenchToast(t('scanProgressInterrupted'));
} else {
resetBtn();
const errMsg = data.message || (isZh ? '扫描启动失败' : 'Failed to start scan');
failScanProgress(errMsg);
showWorkbenchToast(errMsg);
const line = oneModelLine(await fetchLastScan());
finishScanProgress(line);
showWorkbenchToast(line);
}
browser.loadModels?.();
try {
if (app?.refreshComboInNodes) await app.refreshComboInNodes();
if (window.anomalous_reload_hashes) await window.anomalous_reload_hashes();
} catch (e) {
console.warn('[AMB] Error reloading hashes or combo nodes:', e);
}
} catch (e) {
resetBtn();
failScanProgress(String(e));
showWorkbenchToast(String(e));
const message = String(e.message || e) || t('scanPageStartFailed', { error: '' });
failScanProgress(message);
showWorkbenchToast(message);
}
}
+37 -20
View File
@@ -1,12 +1,13 @@
/**
* The activity page (everything Anomalous changed, by day, with filters, details and
* "find on canvas") and the home page's "recent" list. Reads through activity_log.js;
* each render cancels the previous one's request.
* The activity page (everything Anomalous changed, by day, with filters, details,
* "find on canvas" and "open" for scanned models) and the home page's "recent" list.
* Reads through activity_log.js; each render cancels the previous one's request.
*/
import { app } from "../../../scripts/app.js";
import { translate as t } from './locales.js';
import { changeLine, clearActivity, dayLabel, entrySummary, fetchActivity, timeLabel } from './activity_log.js';
import { changeLine, clearActivity, dayLabel, entrySummary, fetchActivity, fileDetailLines, timeLabel } from './activity_log.js';
import { openListedModel } from './ui_scan_lists.js';
import { changeNodeId } from './activity_diff.js';
import { anomalousConfirm } from './ui_dialog.js';
@@ -45,34 +46,50 @@ function locateNode(change, note) {
app.canvas.setDirty?.(true, true);
}
function renderEntry(entry, { compact = false } = {}) {
/** What an entry opens to: [{text, label?, onClick?}]. */
function entryLines(entry, owner, note) {
if (entry.source !== 'canvas') {
return fileDetailLines(entry).map(line => (line.model && owner ? {
text: line.text,
label: t('scanRowOpen'),
onClick: () => openListedModel(owner, line.model, () => owner.goTo('activity')),
} : line));
}
return (entry.detail?.changes || []).filter(change => change.kind !== 'opened').map(change => ({
text: changeLine(change),
...(change.kind !== 'removed' ? { label: t('activityLocate'), onClick: () => locateNode(change, note) } : {}),
}));
}
function renderEntry(entry, { compact = false, owner = null } = {}) {
const row = el('div', `anomalous-activity-entry is-${entry.source === 'canvas' ? 'canvas' : 'file'}`);
const head = el('div', 'anomalous-activity-head');
const summary = entrySummary(entry);
head.append(el('span', 'anomalous-activity-time', timeLabel(entry.time)),
el('span', 'anomalous-activity-source', t(entry.source === 'canvas' ? 'activityCanvas' : 'activityFiles')),
el('span', 'anomalous-activity-summary', entrySummary(entry)));
el('span', 'anomalous-activity-summary', summary));
head.title = summary;
row.appendChild(head);
const changes = (entry.detail?.changes || []).filter(change => change.kind !== 'opened');
if (compact || !changes.length) return row;
if (compact) return row;
const note = el('div', 'anomalous-activity-note');
const lines = entryLines(entry, owner, note);
if (!lines.length) return row;
// Canvas entries open to one line per change.
// An entry with details opens to one line each.
head.classList.add('is-expandable');
head.tabIndex = 0;
head.setAttribute('role', 'button');
head.setAttribute('aria-expanded', 'false');
const details = el('div', 'anomalous-activity-details');
details.hidden = true;
const note = el('div', 'anomalous-activity-note');
for (const change of changes) {
for (const { text, label, onClick } of lines) {
const line = el('div', 'anomalous-activity-change');
line.appendChild(el('span', 'anomalous-activity-change-text', changeLine(change)));
line.title = changeLine(change);
if (change.kind !== 'removed') {
line.appendChild(button('anomalous-activity-locate', t('activityLocate'), () => locateNode(change, note)));
}
line.appendChild(el('span', 'anomalous-activity-change-text', text));
line.title = text;
if (onClick) line.appendChild(button('anomalous-activity-locate', label, onClick));
details.appendChild(line);
}
const shown = changes.length;
const shown = entry.source === 'canvas' ? lines.length : (entry.detail?.files || []).length;
const total = entry.detail?.total || shown;
if (total > shown) details.appendChild(el('div', 'anomalous-activity-more', t('activityMore', { count: total - shown })));
details.appendChild(note);
@@ -91,7 +108,7 @@ function renderEntry(entry, { compact = false } = {}) {
return row;
}
function renderList(list, entries) {
function renderList(list, entries, owner) {
list.replaceChildren();
let day = null;
for (const entry of entries) {
@@ -100,7 +117,7 @@ function renderList(list, entries) {
day = label;
list.appendChild(el('div', 'anomalous-activity-day', label));
}
list.appendChild(renderEntry(entry));
list.appendChild(renderEntry(entry, { owner }));
}
}
@@ -120,7 +137,7 @@ export function renderActivityPage(owner, panel) {
fetchActivity({ limit: PAGE_SIZE, source: filter, signal })
.then(entries => {
if (signal.aborted) return;
if (entries.length) renderList(list, entries);
if (entries.length) renderList(list, entries, owner);
else list.replaceChildren(el('div', 'anomalous-activity-empty', t('activityEmpty')));
})
.catch((error) => {
+178
View File
@@ -0,0 +1,178 @@
/**
* The scan page's model lists: the last scan's result (what each model got, why Civitai had
* nothing) and the models still unmatched or not scanned, from GET /anomalous/scan_summary.
* Every row opens its model; rows offer "scan" / "look up again" and a Civitai search.
* The page passes what the buttons do (`actions`); this module only renders.
*/
import { translate as t } from './locales.js';
import { dayLabel, timeLabel } from './activity_log.js';
import { countsLine, fileOutcome, isPending, reasonText, scanScope, STATUS_MARKS } from './scan_results.js';
import { typeLabel } from './ui_model_types.js';
const SHOWN_ROWS = 100; // more behind "show all"
const RESULT_ROWS = 8;
function el(tag, className, text) {
const node = document.createElement(tag);
if (className) node.className = className;
if (text !== undefined) node.textContent = text;
return node;
}
function button(className, label, onClick) {
const node = el('button', className, label);
node.type = 'button';
node.onclick = onClick;
return node;
}
/** Opens a listed model's detail; its back button returns to `onBack` (the scan page). */
export async function openListedModel(owner, item, onBack) {
const params = new URLSearchParams({ type: item.type, path_idx: item.path_idx, rel: item.rel });
const data = await fetch(`/anomalous/scan_model?${params}`).then(res => res.json()).catch(() => ({}));
if (!data.model) {
alert(t('scanModelGone', { name: item.filename }));
return;
}
owner.hideAllPanels();
owner.historyStack = [];
owner.currentType = data.model.type;
owner.currentPathIdx = data.model.path_idx;
owner.currentSubfolder = data.model.subfolder || '/';
owner.currentDetailModel = data.model;
owner.recipeModelReturn = onBack; // set after hideAllPanels, which clears it
owner.showDetail(data.model);
}
/** Civitai's model search for a file name, to find a model by hand. */
function searchCivitai(item) {
const query = item.filename.replace(/\.safetensors$/i, '').replace(/[_-]+/g, ' ');
window.open(`https://civitai.com/search/models?query=${encodeURIComponent(query)}`, '_blank', 'noopener');
}
function where(item) {
const folder = item.rel.includes('/') ? item.rel.slice(0, item.rel.lastIndexOf('/')) : '';
return [item.type ? typeLabel({ type: item.type }) : '', folder].filter(Boolean).join(' · ');
}
/** One model: mark, file name, where it is, what about it, and its buttons. */
function row(item, detail, buttons, mark = '') {
const line = el('div', `anomalous-scan-row${item.status ? ` is-${item.status}` : ''}`);
if (mark) line.append(el('span', 'anomalous-scan-row-mark', mark));
const copy = el('div', 'anomalous-scan-row-copy');
const name = el('strong', 'anomalous-scan-row-name', item.filename);
name.title = item.rel;
copy.append(name, el('small', 'anomalous-scan-row-where', where(item)));
if (detail) copy.append(el('small', 'anomalous-scan-row-detail', detail));
const actions = el('div', 'anomalous-scan-row-actions');
for (const [label, onClick, disabled] of buttons) {
const node = button('anomalous-scan-row-btn', label, onClick);
node.disabled = Boolean(disabled);
actions.append(node);
}
line.append(copy, actions);
return line;
}
/** Buttons for an unmatched model. */
function unmatchedButtons(item, actions) {
return [
[t('scanRowOpen'), () => actions.open(item)],
...(actions.offline ? [] : [[t('scanRowRetry'), () => actions.scan([item], { retry: true }), actions.busy]]),
[t('scanRowSearch'), () => searchCivitai(item)],
];
}
/** Rows into `list`, the first `limit`, then a button for the rest. */
function fillRows(list, items, render, limit) {
const shown = items.slice(0, limit);
list.replaceChildren(...shown.map(render));
if (items.length > shown.length) {
list.append(button('anomalous-scan-more', t('scanShowAll', { count: items.length }), () => fillRows(list, items, render, Infinity)));
}
}
/** The last scan: when, what, the counts, and each model it changed. Null before the first scan. */
export function renderLastScan(result, actions) {
if (!result?.counts) return null;
const box = el('section', 'anomalous-scan-card anomalous-scan-result');
const seconds = Math.max(0, Math.round((result.finished || 0) - (result.started || 0)));
const target = result.kind === 'one' ? result.files?.[0]?.filename : '';
box.append(
el('div', 'anomalous-scan-card-title', t('scanResultTitle', {
when: `${dayLabel(result.finished)} ${timeLabel(result.finished)}`,
scope: scanScope(result, target),
duration: seconds < 60 ? t('scanSeconds', { count: seconds }) : t('scanMinutes', { count: Math.round(seconds / 60) }),
})),
el('div', 'anomalous-scan-result-counts', countsLine(result.counts)),
);
if (result.options?.offline_only) box.append(el('p', 'anomalous-scan-muted', t('scanResultOffline')));
if (result.civitai_down) box.append(el('p', 'anomalous-scan-note is-warn', t('scanResultCivitaiDown')));
for (const error of result.errors || []) box.append(el('p', 'anomalous-scan-note is-error', error));
const files = result.files || [];
if (!files.length) {
box.append(el('p', 'anomalous-scan-muted', t('scanResultNothing', { count: result.counts.unchanged || 0 })));
return box;
}
const list = el('div', 'anomalous-scan-rows');
const render = item => row(item, fileOutcome(item),
item.status === 'inferred' ? unmatchedButtons(item, actions) : [[t('scanRowOpen'), () => actions.open(item)]],
STATUS_MARKS[item.status] || '');
fillRows(list, files, render, RESULT_ROWS);
box.append(list);
if (result.counts.unchanged) box.append(el('p', 'anomalous-scan-muted', t('scanResultUnchanged', { count: result.counts.unchanged })));
return box;
}
/** The models not scanned yet or unmatched, one tab each; `state.tab` remembers the open one. */
export function renderModelLists(summary, actions, state) {
const unmatched = [...(summary?.unmatched_models || [])]
.sort((a, b) => Number(isPending(b.reason)) - Number(isPending(a.reason)));
const tabs = [['new', t('scanListNew', { count: summary?.new || 0 }), summary?.new_models || []],
['unmatched', t('scanListUnmatched', { count: summary?.unmatched || 0 }), unmatched]];
if (!tabs.some(([, , items]) => items.length)) return null;
if (!tabs.some(([key, , items]) => key === state.tab && items.length)) state.tab = tabs.find(([, , items]) => items.length)[0];
const box = el('section', 'anomalous-scan-card anomalous-scan-lists');
box.id = 'anomalous-scan-lists';
const bar = el('div', 'anomalous-scan-tabs');
bar.setAttribute('role', 'tablist');
const hint = el('p', 'anomalous-scan-muted');
const list = el('div', 'anomalous-scan-rows');
const show = (key) => {
state.tab = key;
bar.querySelectorAll('button').forEach(tab => tab.setAttribute('aria-selected', String(tab.dataset.tab === key)));
const items = tabs.find(([tabKey]) => tabKey === key)[2];
if (key === 'new') {
hint.textContent = t('scanListNewHint');
fillRows(list, items, item => row(item, '', [
[t('scanRowOpen'), () => actions.open(item)],
[t('scanRowScan'), () => actions.scan([item]), actions.busy],
]), SHOWN_ROWS);
} else {
hint.textContent = t(actions.offline ? 'scanListUnmatchedHintOffline' : 'scanListUnmatchedHint');
fillRows(list, items, item => row(item, [reasonText(item.reason), item.base ? `≈ ${item.base}` : ''].filter(Boolean).join(' · '),
unmatchedButtons(item, actions), '≈'), SHOWN_ROWS);
}
};
for (const [key, label, items] of tabs) {
const tab = button('anomalous-scan-tab', label, () => show(key));
tab.dataset.tab = key;
tab.setAttribute('role', 'tab');
tab.disabled = !items.length;
bar.append(tab);
}
box.append(bar, hint, list);
show(state.tab);
return box;
}
/** A count on the page that opens its list (unmatched, not scanned). */
export function showList(page, state, key) {
state.tab = key;
const lists = page.querySelector('#anomalous-scan-lists');
lists?.querySelector(`[data-tab="${key}"]`)?.click();
lists?.scrollIntoView({ behavior: 'smooth', block: 'start' });
}
+52 -17
View File
@@ -1,14 +1,16 @@
/**
* The scan page: how many models are matched on Civitai, unmatched or new
* The scan page: how many models are matched on Civitai, unmatched or not scanned
* (GET /anomalous/scan_summary), one button to scan, one to look unmatched models up
* again, progress in the page, and the rarer choices folded under "Advanced".
* again, progress in the page, the last scan's result and the unmatched / not scanned
* models (ui_scan_lists.js), and the rarer choices folded under "Advanced".
* Scans run through scan_runner.js; the progress box is scan_progress.js's panel,
* hosted here while the page is shown.
*/
import { translate as t } from './locales.js';
import { isScanRunning, startScan } from './scan_runner.js';
import { isScanRunning, startScan, targetsForItems } from './scan_runner.js';
import { setScanProgressHost } from './scan_progress.js';
import { openListedModel, renderLastScan, renderModelLists, showList } from './ui_scan_lists.js';
// Choices kept while the browser is open. The two risky ones go back off after each scan.
const options = {
@@ -20,6 +22,7 @@ const options = {
forceOverwrite: false,
autoFix: true,
};
const listState = { tab: '' }; // the open list: 'new' | 'unmatched'
function el(tag, className, text) {
const node = document.createElement(tag);
@@ -50,8 +53,14 @@ async function fetchJson(url) {
}
}
function stat(value, label, tone) {
const box = el('div', `anomalous-scan-stat${tone ? ` is-${tone}` : ''}`);
/** A count; with `onClick` it opens the list of those models. */
function stat(value, label, tone, onClick) {
const box = el(onClick ? 'button' : 'div', `anomalous-scan-stat${tone ? ` is-${tone}` : ''}`);
if (onClick) {
box.type = 'button';
box.onclick = onClick;
box.title = t('scanPageShowList');
}
box.append(el('strong', 'anomalous-scan-stat-value', value ?? '–'), el('span', 'anomalous-scan-stat-label', label));
return box;
}
@@ -144,26 +153,45 @@ async function saveApiKey() {
}
}
/** A scan from this page. `extra`: { retryUnmatched } for every unmatched model, or
* { targets, retryUnmatched } for the models of a list row. */
async function run(owner, panel, extra = {}) {
if (options.scope === 'picked' && !extra.retryUnmatched && pickedCount() === 0) {
const whole = !extra.retryUnmatched && !extra.targets;
if (whole && options.scope === 'picked' && pickedCount() === 0) {
alert(t('scanPageNothingPicked'));
return;
}
if (options.forceOverwrite && !extra.retryUnmatched && !confirm(t('scanPageForceConfirm'))) return;
const selection = options.scope === 'picked' && !extra.retryUnmatched ? options.selection : null;
if (whole && options.forceOverwrite && !confirm(t('scanPageForceConfirm'))) return;
const selection = whole && options.scope === 'picked' ? options.selection : null;
const started = startScan(owner, { ...options, ...extra, selection });
renderScanPage(owner, panel); // buttons show "scanning"
await started;
options.physicalRename = false;
options.forceOverwrite = false;
if (whole) {
options.physicalRename = false;
options.forceOverwrite = false;
}
if (panel.isConnected && panel.style.display !== 'none') renderScanPage(owner, panel);
}
/** What the list rows' buttons do. */
function listActions(owner, panel, busy) {
return {
busy,
offline: options.offline,
open: item => openListedModel(owner, item, () => openScanPage(owner)),
scan: (items, { retry = false } = {}) => run(owner, panel, { targets: targetsForItems(items), retryUnmatched: retry }),
};
}
/** Renders the page into `panel` (again after each scan, so the counts are fresh). */
export async function renderScanPage(owner, panel) {
const token = (panel._scanRender = (panel._scanRender || 0) + 1);
const [summary, config] = await Promise.all([fetchJson('/anomalous/scan_summary'), fetchJson('/anomalous/config')]);
const [summary, config, last] = await Promise.all([
fetchJson('/anomalous/scan_summary'), fetchJson('/anomalous/config'), fetchJson('/anomalous/last_scan'),
]);
if (token !== panel._scanRender) return;
// Closed or left meanwhile: the progress box keeps floating instead of hiding in the page.
if (panel.style.display === 'none' || !owner.modal?.classList.contains('visible')) return;
const page = el('div', 'anomalous-scan-page');
page.append(el('h1', 'anomalous-scan-title', t('scanPageTitle')), el('p', 'anomalous-scan-lead', t('scanPageLead')));
@@ -172,8 +200,10 @@ export async function renderScanPage(owner, panel) {
stats.append(
stat(summary?.total, t('scanPageTotal')),
stat(summary?.matched, t('scanPageMatched'), 'ok'),
stat(summary?.unmatched, t('scanPageUnmatched'), summary?.unmatched ? 'warn' : ''),
stat(summary?.new, t('scanPageNew'), summary?.new ? 'new' : ''),
stat(summary?.unmatched, t('scanPageUnmatched'), summary?.unmatched ? 'warn' : '',
summary?.unmatched ? () => showList(page, listState, 'unmatched') : null),
stat(summary?.new, t('scanPageNew'), summary?.new ? 'new' : '',
summary?.new ? () => showList(page, listState, 'new') : null),
);
page.append(stats);
if (!summary) page.append(el('p', 'anomalous-scan-muted', t('scanPageSummaryFailed')));
@@ -181,9 +211,11 @@ export async function renderScanPage(owner, panel) {
const busy = isScanRunning();
const actions = el('div', 'anomalous-scan-actions');
const picked = options.scope === 'picked';
// An online scan also looks up the models earlier scans could not ask Civitai about.
const waiting = (summary?.new || 0) + (options.offline ? 0 : summary?.pending || 0);
const label = busy ? t('scanPageScanning')
: picked ? t('scanPageScanPicked', { count: pickedCount() })
: summary?.new ? t('scanPageScanNew', { count: summary.new })
: waiting ? t(options.offline ? 'scanPageScanNewOffline' : 'scanPageScanNew', { count: waiting })
: t('scanPageScanAgain');
const primary = button('anomalous-scan-primary', label, () => run(owner, panel));
primary.disabled = busy;
@@ -194,12 +226,15 @@ export async function renderScanPage(owner, panel) {
actions.append(retry);
}
page.append(actions);
const hint = picked ? t('scanPageHintPicked')
: summary?.unmatched && !options.offline ? t('scanPageHintRetry') : t('scanPageHint');
const hint = picked ? t('scanPageHintPicked') : t(options.offline ? 'scanPageHintOffline' : 'scanPageHint');
page.append(el('p', 'anomalous-scan-muted', hint));
const progressHost = el('div', 'anomalous-scan-progress-host');
page.append(progressHost, renderAdvanced(owner, panel, Boolean(config?.has_api_key)));
page.append(progressHost);
const rowActions = listActions(owner, panel, busy);
const result = renderLastScan(last, rowActions);
const lists = renderModelLists(summary, rowActions, listState);
page.append(...[result, lists].filter(Boolean), renderAdvanced(owner, panel, Boolean(config?.has_api_key)));
panel.replaceChildren(page);
setScanProgressHost(progressHost);
// A scan that ends elsewhere (a card's radar, another tab) refreshes the counts too.
+5 -5
View File
@@ -1,11 +1,11 @@
/**
* Follows scans started anywhere (the wizard, a single model, another tab): shows their
* progress, turns the rail's scan button into a spinner, and reloads the models when done.
* Follows scans started anywhere (the scan page, a single model, another tab): shows their
* progress and result, turns the rail's scan button into a spinner, and reloads the models when done.
*/
import { translate as t } from './locales.js';
import { updateScanProgress, finishScanProgress, failScanProgress } from './scan_progress.js';
import { setScanButtonState } from './scan_runner.js';
import { updateScanProgress, failScanProgress } from './scan_progress.js';
import { fetchLastScan, setScanButtonState, showScanResult } from './scan_runner.js';
const POLL_MS = 3000;
@@ -33,7 +33,7 @@ export function watchScans(owner) {
} else if (!active && scanning) {
scanning = false;
if (button) setScanButtonState(button, false);
finishScanProgress();
showScanResult(owner, await fetchLastScan());
owner.loadModels();
owner.onScanFinished?.();
if (window.anomalous_reload_hashes) await window.anomalous_reload_hashes();
+16
View File
@@ -903,6 +903,22 @@ html:not(.anomalous-floating-entry-enabled) [data-setting-id="Anomalous.ModelBro
font-size: 11px;
}
.anomalous-scan-progress-action {
margin-top: 8px;
padding: 4px 12px;
border: 1px solid rgba(138, 180, 248, 0.45);
border-radius: 999px;
background: transparent;
color: #d7dde8;
font-size: 12px;
cursor: pointer;
}
.anomalous-scan-progress-action:hover {
background: rgba(138, 180, 248, 0.14);
color: #ffffff;
}
.anomalous-scan-progress.is-complete .anomalous-scan-progress-fill {
background: #81c995;
}
+170
View File
@@ -44,6 +44,18 @@
border: 1px solid rgba(255, 255, 255, 0.08);
border-radius: 12px;
background: rgba(255, 255, 255, 0.03);
font: inherit;
text-align: left;
}
/* Unmatched and not scanned open their list. */
button.anomalous-scan-stat {
cursor: pointer;
}
button.anomalous-scan-stat:hover {
border-color: rgba(165, 180, 252, 0.4);
background: rgba(255, 255, 255, 0.06);
}
.anomalous-scan-stat-value {
@@ -228,7 +240,165 @@
border-top: 1px solid rgba(255, 255, 255, 0.06);
}
/* The scan page's cards: the last scan's result and the model lists. */
.anomalous-scan-card {
display: flex;
flex-direction: column;
gap: 8px;
padding: 14px 16px;
border: 1px solid rgba(255, 255, 255, 0.08);
border-radius: 12px;
background: rgba(255, 255, 255, 0.02);
}
.anomalous-scan-card-title {
color: #9ca3af;
font-size: 12.5px;
}
.anomalous-scan-result-counts {
color: #f3f4f6;
font-size: 15px;
font-weight: 600;
}
.anomalous-scan-note {
margin: 0;
padding: 8px 10px;
border-radius: 8px;
font-size: 12.5px;
line-height: 1.55;
}
.anomalous-scan-note.is-warn {
background: rgba(252, 211, 77, 0.08);
color: #fcd34d;
}
.anomalous-scan-note.is-error {
background: rgba(248, 113, 113, 0.08);
color: #fca5a5;
}
.anomalous-scan-tabs {
display: flex;
flex-wrap: wrap;
gap: 6px;
}
.anomalous-scan-tab {
padding: 5px 14px;
border: 1px solid rgba(255, 255, 255, 0.1);
border-radius: 999px;
background: transparent;
color: #9ca3af;
font-size: 13px;
cursor: pointer;
}
.anomalous-scan-tab[aria-selected="true"] {
border-color: rgba(165, 180, 252, 0.45);
background: rgba(165, 180, 252, 0.12);
color: #e5e7eb;
}
.anomalous-scan-tab:disabled {
opacity: 0.45;
cursor: default;
}
.anomalous-scan-rows {
display: flex;
flex-direction: column;
}
.anomalous-scan-row {
display: flex;
align-items: center;
gap: 10px;
padding: 8px 0;
border-top: 1px solid rgba(255, 255, 255, 0.05);
}
.anomalous-scan-row-mark {
flex-shrink: 0;
width: 16px;
color: #fcd34d;
text-align: center;
}
.anomalous-scan-row.is-matched .anomalous-scan-row-mark { color: #86efac; }
.anomalous-scan-row.is-failed .anomalous-scan-row-mark { color: #fca5a5; }
.anomalous-scan-row.is-unchanged .anomalous-scan-row-mark { color: #9ca3af; }
.anomalous-scan-row-copy {
display: flex;
flex: 1;
flex-direction: column;
gap: 2px;
min-width: 0;
}
.anomalous-scan-row-name,
.anomalous-scan-row-where,
.anomalous-scan-row-detail {
overflow: hidden;
white-space: nowrap;
text-overflow: ellipsis;
}
.anomalous-scan-row-name {
color: #e5e7eb;
font-size: 13px;
font-weight: 600;
}
.anomalous-scan-row-where {
color: #6b7280;
font-size: 11.5px;
}
.anomalous-scan-row-detail {
color: #9ca3af;
font-size: 12px;
}
.anomalous-scan-row-actions {
display: flex;
flex-shrink: 0;
gap: 6px;
}
.anomalous-scan-row-btn,
.anomalous-scan-more {
padding: 3px 10px;
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: 999px;
background: transparent;
color: #d1d5db;
font-size: 12px;
cursor: pointer;
}
.anomalous-scan-row-btn:hover:not(:disabled),
.anomalous-scan-more:hover {
border-color: rgba(165, 180, 252, 0.45);
color: #ffffff;
}
.anomalous-scan-row-btn:disabled {
opacity: 0.45;
cursor: default;
}
.anomalous-scan-more {
align-self: flex-start;
margin-top: 8px;
}
@media (max-width: 560px) {
.anomalous-scan-stats { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.anomalous-scan-page { padding: 24px 16px 32px; }
.anomalous-scan-row { flex-wrap: wrap; }
.anomalous-scan-row-actions { width: 100%; padding-left: 26px; }
}