diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 2db64df..813e70f 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -423,7 +423,9 @@ covered by the `styles.css` manifest. the page keeps its picks as items (none: the button scans everything). `scan_runner.js` starts and follows scans (every folder, picked or listed models, or one model from its card), shows the result and refreshes node drop-downs, hashes and the - grid afterwards; folder visibility/order lives in + grid afterwards; before an online scan `scan_connection.js` checks whether Civitai can be + reached and, when not, asks: scan from the files, set a proxy, or not now; + folder visibility/order lives in `ui_folder_manager.js`. `scan_progress.js` owns the scan progress panel (`updateScanProgress` / `finishScanProgress` / `failScanProgress`): inside the scan diff --git a/CHANGELOG.md b/CHANGELOG.md index df2b1ed..760a28a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,8 @@ - On a rented cloud server (AutoDL, RunPod…) the page is always opened from another device, so since v1.58.1 the plugin only let it look: scans, downloads, moves and saves were refused. Now there are two ways: open ComfyUI through an **SSH tunnel** (the address starts with 127.0.0.1; it counts as the computer ComfyUI runs on, nothing to change), or make an empty file named **`allow_remote_changes`** in the plugin folder on the server and restart ComfyUI. With the file, every device that can open the page may change things; the page can neither make nor remove it, and the MCP endpoint stays for this computer only. How, and the risk: [docs/guides/cloud-server.md](docs/guides/cloud-server.md). - **Civitai's other address (civitai.red) when civitai.com does not answer**: some networks reach only the second, a cloud server in mainland China among them (tested on AutoDL: civitai.com timed out, civitai.red answered in about a second). Scans, import and Model Check's lookups and **Settings → Network → Test** ask civitai.red then, so models still get their names, trigger words and base models; the test says which address answered. The covers are on Civitai's picture server, which such networks often cannot reach either: after two covers in a row that would not download, a scan fetches no more covers instead of waiting on each. - **Models that are links (模型是快捷方式/链接时)**: on a cloud server such as AutoDL every model in the models folder is a link into the platform's shared model disk. Scans wrote the info beside the link, but the cards read it from the disk behind it: a scanned model showed as not scanned, under the disk's file name (`6b4679…`) instead of its own, and editing, deleting, opening its folder or finding it from a saved workflow could be refused as "outside the models folder". A linked model is now taken as named, with its covers and info beside the link; edits, renames and deletes act on the link, never on the shared disk; moving one in Tidy moves the link instead of copying the model disk's file onto the server. Names leaving the folder (`..`) are refused as before. +- **Before an online scan, the plugin checks whether Civitai can be reached**, on both its addresses. When it cannot, it asks instead of waiting on every model: **Scan from the files** (base models found, no covers or trigger words; a later scan fills them in), **Set a proxy** (opens Settings → Models → Network), or Cancel. A check that reached Civitai is trusted for five minutes. +- **Models linked from another tool keep their info**: a model that is a link with no info beside it shows the info beside the file it links to (A1111's Civitai Helper leaves `.civitai.info` there). Only read there; covers, edits and scans stay beside the link. - The **View only** chip beside the page title can be pressed: it says what is refused and the two ways above, with a **Copy** button for the one line to paste into the server's terminal (the plugin's real path on that server already filled in), so there is no folder to find. With the file there it reads **Remote changes on**, in yellow, and its dialog gives the line that turns it off again. ## v1.60.0 (Model Import, Tidy, Parameter Compare, Workflows Rebuilt, Base Model Filter, Settings Tabs) — unreleased diff --git a/api/metadata.py b/api/metadata.py index de400d1..87614f9 100644 --- a/api/metadata.py +++ b/api/metadata.py @@ -89,8 +89,18 @@ def _positive_civitai_id(value): return None return parsed if parsed > 0 else None +def _info_base(file_path, base_path): + """Where the model's info files are read: beside the model as named; for a link with none + beside it, beside the file it links to, where another tool (A1111's Civitai Helper) may + have left them. Only read there: covers, edits and scans stay beside the link.""" + if os.path.islink(file_path) and not any(os.path.exists(base_path + suffix) for suffix in (".info", ".civitai.info")): + return os.path.splitext(os.path.realpath(file_path))[0] + return base_path + + def _read_metadata(file_path): base_path = os.path.splitext(file_path)[0] + info_base = _info_base(file_path, base_path) metadata = { "name": os.path.basename(base_path), "description": "", @@ -115,7 +125,7 @@ def _read_metadata(file_path): "not_drawing": False, # not scanned yet, and its header looks like no drawing model } - info_files = [f"{base_path}.info", f"{base_path}.civitai.info"] + info_files = [f"{info_base}.info", f"{info_base}.civitai.info"] for info_file in info_files: if os.path.exists(info_file): try: @@ -173,7 +183,7 @@ def _read_metadata(file_path): except Exception: pass - scan = sidecar_info(base_path) + scan = sidecar_info(info_base) if scan is not None: metadata["info_source"] = "local" if is_unmatched(scan) else "civitai" metadata["unmatched_reason"] = unmatched_reason(scan) diff --git a/docs/architecture/backend.md b/docs/architecture/backend.md index c9f3034..9c8078a 100644 --- a/docs/architecture/backend.md +++ b/docs/architecture/backend.md @@ -29,7 +29,8 @@ are written beside the link. So the folder part is resolved and must stay inside (through links too), and the file is taken as named: `model_file_within` is the folder joined with a bare `require_filename` name, `get_metadata` reads the sidecars beside the path it is given (`abspath`, not `realpath`), and edits, renames and deletes act on the link, never on the -disk behind it. Nothing in the plugin writes into a model file. Tidy's moves and the +disk behind it. Only when a link has no info beside it are the info files beside the file it +links to read (`_info_base`): another tool (A1111's Civitai Helper) may have left them there. Nothing in the plugin writes into a model file. Tidy's moves and the import's `move_into` tell a rename from a copy by where the link is (`lstat`), so moving a linked model renames the link instead of copying the model disk's file onto the server. diff --git a/docs/architecture/frontend.md b/docs/architecture/frontend.md index fa59e76..1606c1a 100644 --- a/docs/architecture/frontend.md +++ b/docs/architecture/frontend.md @@ -57,7 +57,7 @@ The Prompt Studio shortcut (its command and setting ids still say "materials") i native ComfyUI command/keybinding; a deferred window-key fallback invokes the same command path only when the host did not open the studio, so handled shortcuts are not executed twice. `ui_browser_navigation.js` owns shared panel hiding, recoverable detail cleanup, -and workspace return. `ui_scan_page.js` owns the scan page and `scan_runner.js` scan launch and polling, +and workspace return. `ui_scan_page.js` owns the scan page and `scan_runner.js` scan launch and polling (an online scan first goes through `scan_connection.js`: Civitai reachable, or the user picks scanning from the files), `ui_folder_manager.js` is the settings page's Model folders view (grouping mode, visibility and order, saved on each change). Public entry functions remain browser-instance methods so existing actions share current browser state. diff --git a/tests/test_model_links.py b/tests/test_model_links.py index 2f16a89..36b7430 100644 --- a/tests/test_model_links.py +++ b/tests/test_model_links.py @@ -41,6 +41,20 @@ class ModelLinkTests(unittest.TestCase): with self.resolve: self.assertEqual(metadata.get_metadata(str(self.model))["name"], "style") # not the disk's file name + def test_info_left_beside_the_linked_file_is_read_when_none_is_beside_the_link(self): + # Another tool (A1111's Civitai Helper) kept its info beside the real file. + disk = Path(self.temp.name, "a1111") + disk.mkdir() + (disk / "style.civitai.info").write_text(json.dumps({"id": 9, "modelId": 4, "name": "from a1111"}), encoding="utf-8") + Path(str(self.model)[:-12] + ".info").unlink() + real = os.path.realpath + to_a1111 = mock.patch("os.path.realpath", side_effect=lambda p: str(disk / "style.safetensors") if os.path.basename(p) == "style.safetensors" else real(p)) + with to_a1111, mock.patch("os.path.islink", side_effect=lambda p: os.path.basename(p) == "style.safetensors"): + self.assertEqual(metadata.get_metadata(str(self.model))["name"], "from a1111") + metadata.clear_metadata_cache() + Path(str(self.model)[:-12] + ".info").write_text(json.dumps({"id": 7, "modelId": 3, "name": "v1"}), encoding="utf-8") + self.assertEqual(metadata.get_metadata(str(self.model))["name"], "v1") # beside the link wins + def test_a_model_named_in_its_folder_stays_the_link_but_cannot_leave_the_folder(self): folder = str(self.root / "sub") with self.resolve: diff --git a/web/modules/locales.js b/web/modules/locales.js index 58828d4..a7b06dc 100644 --- a/web/modules/locales.js +++ b/web/modules/locales.js @@ -240,6 +240,11 @@ export const i18n = { scanPickBackToAll: "清空，改回扫描全部", scanPageLeadShort: "给模型配上封面、说明和底模；扫过的不会重复下载。", scanCountingTitle: "正在清点你的模型…", + scanCheckingNet: "正在检查能不能连上 Civitai…", + scanNetDownTitle: "连不上 Civitai", + scanNetDownText: "原因：{why}\n\n可以先只从模型文件推断：能认出底模（SDXL、Flux 这类），没有封面和触发词；以后能连上时再扫一次，会自动补上。\n\n也可以先去「设置 → 模型 → 网络」填代理。", + scanNetFromFiles: "从文件推断", + scanNetSetProxy: "去设置代理", scanCountingHint: "模型很多，或者放在网络盘上时，第一次要等一会儿；之后再打开会快很多。", scanPageAboutLink: "扫描是怎么回事？", scanHeroWaiting: "还有 {count} 个模型待查", @@ -2821,6 +2826,11 @@ export const i18n = { scanPickBackToAll: "Clear, scan everything again", scanPageLeadShort: "Gives your models a cover, a description and a base model; models already scanned are not downloaded again.", scanCountingTitle: "Counting your models…", + scanCheckingNet: "Checking whether Civitai can be reached…", + scanNetDownTitle: "Civitai cannot be reached", + scanNetDownText: "Why: {why}\n\nYou can scan from the model files only: base models (SDXL, Flux…) are found, without covers or trigger words; scan again once Civitai can be reached and the rest is filled in.\n\nOr set a proxy first in Settings → Models → Network.", + scanNetFromFiles: "Scan from the files", + scanNetSetProxy: "Set a proxy", scanCountingHint: "With many models, or models on a network drive, the first time takes a while; opening the page again is much quicker.", scanPageAboutLink: "How scanning works", scanHeroWaiting: "{count} {count|model|models} still to look up", diff --git a/web/modules/scan_connection.js b/web/modules/scan_connection.js new file mode 100644 index 0000000..dc87e31 --- /dev/null +++ b/web/modules/scan_connection.js @@ -0,0 +1,42 @@ +/** + * Before an online scan: can Civitai be reached (GET /anomalous/civitai_check, both its + * addresses)? When not, the user is asked instead of the scan waiting on every model: scan + * from the files only (an offline scan; a later online scan fills in the rest), set a proxy + * first (Settings → Models, Network), or not now. A check that reached Civitai is trusted for + * a few minutes, so scans in a row do not wait on it. + */ + +import { translate as t } from './locales.js'; +import { failureWords } from './net_failure.js'; +import { anomalousConfirm } from './ui_dialog.js'; +import { showWorkbenchToast } from './ui_prompt_toast.js'; +import { openSettingsPage } from './ui_settings_page.js'; + +const TRUSTED_MS = 5 * 60 * 1000; +let reachedAt = 0; + +/** 'online' (Civitai answers, or this cannot be told), 'offline' (scan from the files), or null (not now). */ +export async function onlineOrAsk(owner) { + if (Date.now() - reachedAt < TRUSTED_MS) return 'online'; + showWorkbenchToast(t('scanCheckingNet')); + let answer = null; + try { + const res = await fetch('/anomalous/civitai_check'); + answer = res.ok ? await res.json() : null; + } catch { + answer = null; + } + if (!answer || answer.reachable) { + if (answer) reachedAt = Date.now(); + return 'online'; + } + const choice = await anomalousConfirm(t('scanNetDownText', { why: failureWords(answer.failure) }), t('scanNetDownTitle'), { + okLabel: t('scanNetFromFiles'), okClass: 'anomalous-btn-primary', noLabel: t('scanNetSetProxy'), + }); + if (choice === true) return 'offline'; + if (choice === false) { + owner.show?.(); + openSettingsPage(owner, { tab: 'models' }); + } + return null; +} diff --git a/web/modules/scan_runner.js b/web/modules/scan_runner.js index 3af0e3e..54b5f0f 100644 --- a/web/modules/scan_runner.js +++ b/web/modules/scan_runner.js @@ -2,8 +2,9 @@ * Starting scans and following them: every model folder, picked models or listed models from * the scan page (all through /anomalous/scan_all), and one model from its card. Progress goes * to scan_progress.js; when a scan ends its result (GET /anomalous/last_scan) is shown, and - * node drop-downs, hashes and the model grid are refreshed. No page DOM here: ui_scan_page.js - * renders the page. + * node drop-downs, hashes and the model grid are refreshed. An online scan first asks whether + * Civitai can be reached (scan_connection.js) and, when not, whether to scan from the files. + * No page DOM here: ui_scan_page.js renders the page. */ import { app } from '../../../scripts/app.js'; @@ -13,6 +14,7 @@ import { oneModelLine, resultLine } from './scan_results.js'; import { showWorkbenchToast } from './ui_prompt_toast.js'; import { checkWorkflowModels, fixWorkflowModels } from './model_check.js'; import { updateDoctorBanner } from './ui_doctor_banner.js'; +import { onlineOrAsk } from './scan_connection.js'; const POLL_MS = 2000; const SCAN_RADAR_ICON_SVG = ``; @@ -139,9 +141,11 @@ export async function startScan(owner, options) { if (running) return null; running = true; setActiveScanButtonState(true); - const body = scanRequestBody(options); - if (options.targets) body.targets = options.targets; try { + const how = options.offline ? 'offline' : await onlineOrAsk(owner); + if (!how) return null; + const body = scanRequestBody({ ...options, offline: how === 'offline' }); + if (options.targets) body.targets = options.targets; const data = await postJson('/anomalous/scan_all', body); updateScanProgress({ scanning: true, phase: 'preparing', recovered: data.recovered }); const status = await followScan('/anomalous/global_scan_status'); @@ -166,6 +170,8 @@ export async function triggerDirectModelScan(model, triggerBtn = null, browserIn const modelLabel = model.name || model.filename; const titleText = t('scanOneTitle', { name: modelLabel }); + const how = await onlineOrAsk(browser); + if (!how) return; if (triggerBtn) { triggerBtn.classList.add('anomalous-radar-spinning'); @@ -201,12 +207,12 @@ export async function triggerDirectModelScan(model, triggerBtn = null, browserIn // A model that was looked up without an answer is looked up again. await postJson(`/anomalous/scan?${params}`, { target_files: [model.filename], - offline_only: false, + offline_only: how === 'offline', skip_rename: true, virtual_rename: false, physical_rename: false, force_overwrite: false, - retry_unmatched: model.metadata?.info_source === 'local', + retry_unmatched: how === 'online' && model.metadata?.info_source === 'local', }); showWorkbenchToast(titleText); const status = await followScan(`/anomalous/scan_status?${params}`, {}, 1200, titleText); diff --git a/web/modules/ui_dialog.js b/web/modules/ui_dialog.js index cacf697..5042611 100644 --- a/web/modules/ui_dialog.js +++ b/web/modules/ui_dialog.js @@ -184,7 +184,7 @@ export function anomalousConfirm(message, title = 'Anomalous', options = {}) { const okBtn = document.createElement('button'); okBtn.textContent = options.okLabel || t('dialogOk'); - okBtn.className = 'anomalous-btn-danger'; + okBtn.className = options.okClass || 'anomalous-btn-danger'; // most confirmations delete or replace okBtn.style.padding = '8px 24px'; const noBtn = options.noLabel ? document.createElement('button') : null;