Compare commits
89
Commits
+3
-4
@@ -77,10 +77,9 @@ AI_SYSTEM_TEMPLATE.md
|
||||
# AI Agents Directory
|
||||
.agents/
|
||||
|
||||
# Test files
|
||||
test_*.js
|
||||
test*.mjs
|
||||
tests/
|
||||
# Scratch test files at the top level (the plugin's tests live in tests/, see docs/guides/testing.md)
|
||||
/test_*.js
|
||||
/test*.mjs
|
||||
|
||||
# Local diagnostic/test artifacts
|
||||
grep_doctor.txt
|
||||
|
||||
@@ -8,10 +8,10 @@
|
||||
以下几条每次提交都要做到，没有“这次改动小”的例外；其余各节是怎么做好的指引。
|
||||
|
||||
1. **结构和文档同步。** 按下表更新文档，与代码放在同一次提交里。新增、删除、改名、拆分源码文件一律算改了模块职责：职责清单同时是文件目录，每个源码文件都要有一行。
|
||||
2. **跑结构检查。** `node tools/check_structure.mjs`，失败不提交。它检查每个源码文件都登记在架构文档里、每个前端模块都能从入口引用到、每个前端文件都能按浏览器的方式解析（`node --check` 对 .js 文件会漏掉浏览器报错的写法），并执行规模阈值（第 2 节）：新文件不能一开始就超线，已超线的文件比上次提交只能变短或持平。
|
||||
2. **跑结构检查。** `node tools/check_structure.mjs`，失败不提交。它检查每个源码文件都登记在架构文档里、每个前端模块都能从入口引用到、每个前端文件都能按浏览器的方式解析（`node --check` 对 .js 文件会漏掉浏览器报错的写法）、CSS 里没有按 `style` 属性匹配的 `:has()`（会让拖动画布掉帧，见 [lessons.md](docs/architecture/lessons.md)），并执行规模阈值（第 2 节）：新文件不能一开始就超线，已超线的文件比上次提交只能变短或持平。
|
||||
3. **不留旧实现。** 被取代的模块、样式、文案 key、文档说明同一次提交删掉（第 6 节）。
|
||||
4. **按风险验证**（第 7 节），只提交本任务的文件，没有明确同意不推送。收尾时工作区干净，或在交付说明里列出每个有意留下的未提交文件。
|
||||
5. **测试产物不进仓库。** 截图、录屏、日志、原始测试数据、审计附件放在 `.agents/`（已忽略），不用 `git add -f` 绕过忽略规则。提交的文件里不写本机路径、用户名或 `file:///` 链接，文档之间用相对链接。
|
||||
5. **测试产物不进仓库。** 测试代码放 `tests/`；截图、录屏、日志、原始测试数据、审计附件放在 `.agents/`（已忽略，只留本机），不用 `git add -f` 绕过忽略规则。提交的文件里不写本机路径、用户名或 `file:///` 链接，文档之间用相对链接。
|
||||
6. **交付说明**写清：改了什么和为什么、验证结果和没验证到的地方、遗留问题、提交号；用了规范例外就写原因和影响。
|
||||
|
||||
| 改了…… | 更新 |
|
||||
@@ -20,8 +20,8 @@
|
||||
| 数据流、接口、持久化格式、安全边界、跨系统规则 | 对应的 `docs/architecture/` 专题；系统图或阅读地图变了才改 `ARCHITECTURE.md` 入口 |
|
||||
| 用户能感觉到的行为 | `CHANGELOG.md`；使用方法变了同步 `README.md` 或 `docs/guides/` |
|
||||
| 长期有效的产品或架构取舍 | `docs/decisions/` |
|
||||
| 反复出现的坑、严重故障 | `.agents/logs/ai_lessons.md` |
|
||||
| 开发规范本身 | 只改本文件；`CLAUDE.md`、`GEMINI.md` 只做阅读入口 |
|
||||
| 反复出现的坑、严重故障 | `docs/architecture/lessons.md` |
|
||||
| 开发规范本身 | 只改本文件 |
|
||||
|
||||
普通修复、样式微调不写进架构文档；没有变化的边界不写“本次未改变”。
|
||||
|
||||
@@ -137,11 +137,10 @@ ComfyUI 的内存、显存和 CPU 是留给出图的。插件闲着的时候应
|
||||
| 修改范围 | 至少验证 |
|
||||
| --- | --- |
|
||||
| 导入、导出、模块拆分 | JS 语法、模块链接，以及受影响按钮的真实执行路径 |
|
||||
| 工坊状态与生命周期 | `node --experimental-vm-modules tests/prompt_ui_lifecycle.mjs` |
|
||||
| 词卡读取、分页 | `node tests/prompt_material_source.mjs` |
|
||||
| 翻译桥接 | `node tests/translation_service_contracts.mjs`，异步节点写入另跑工坊生命周期测试 |
|
||||
| 素材应用、笔记隔离 | `node --experimental-vm-modules tests/material_workspace_flow.mjs` |
|
||||
| 素材提示词协议、配方筛选、拖拽清理 | `node --experimental-vm-modules tests/studio_contracts.mjs` |
|
||||
| 提示词工坊、词块 | `node tests/prompt_tags.mjs`、`node tests/prompt_boxes.mjs` |
|
||||
| 素材提示词、节点参数 | `node tests/material_prompt_cross_node.mjs`、`node tests/node_material_actions.mjs`、`node tests/node_parameter_sets.mjs` |
|
||||
| 搭配、往画布插节点、撤销 | `node tests/combo_slots.mjs`、`node tests/graph_splice.mjs`、`node tests/canvas_undo.mjs` |
|
||||
| 提交前全部跑一遍 | `node tools/run_tests.mjs`（说明见 `docs/guides/testing.md`） |
|
||||
| CSS、布局、交互 | 实际浏览器验证受影响状态；涉及宿主行为时验证 ComfyUI |
|
||||
| 文件修改、导入、存储 | 对应 Python 测试，至少覆盖成功、冲突和中途失败保留旧数据 |
|
||||
| 仅文档 | 检查链接、命令、实际代码一致性和 Git 差异 |
|
||||
@@ -150,7 +149,7 @@ ComfyUI 的内存、显存和 CPU 是留给出图的。插件闲着的时候应
|
||||
- 修复故障时，优先补充能复现原故障的行为测试。不能只通过源码字符串“存在某段代码”来证明修复。
|
||||
- UX 明确替换旧交互时，更新测试到新的用户流程，并保留原本要保护的行为；不要仅删除失败断言或重新塞回废弃功能。
|
||||
- 新增应随仓库交付的回归测试要检查 `.gitignore`，确认确实进入版本控制。
|
||||
- 历史本地 Python 测试可能不在 Git 中。使用合适环境运行相关测试，例如 `python -m unittest discover -s tests -p 'test_*.py'`，核对实际发现的数量；零项测试不算通过。
|
||||
- Python 测试要在 ComfyUI 的 Python 环境里跑（`node tools/run_tests.mjs` 会自己找），核对实际发现的数量；零项测试不算通过。
|
||||
- 无法运行真实宿主、网络服务或完整测试时，准确说明缺口；不要将模拟画布、语法检查或已有通过数量称为本轮宿主验证。
|
||||
|
||||
## 8. 文档与交付
|
||||
|
||||
+71
-12
@@ -9,7 +9,6 @@ by default.
|
||||
|
||||
Before implementation, follow [AGENTS.md](AGENTS.md), the canonical development
|
||||
and maintenance rules; its section 0 is required for every commit.
|
||||
[GEMINI.md](GEMINI.md) and [CLAUDE.md](CLAUDE.md) are reading entry points only.
|
||||
|
||||
Saved prompts and saved node values are material files (`anomalous_materials`); there is no
|
||||
Material Library page any more. Prompt Studio lists the saved prompts (reading each body
|
||||
@@ -25,7 +24,7 @@ and Current node lists the saved values. See the material contract below.
|
||||
| Saved prompts and node values (material files), image parameter details, keeping an output image | [`docs/architecture/material-library.md`](docs/architecture/material-library.md) |
|
||||
| Model Check, provenance hashes, missing-model recovery, or model scanning | [`docs/architecture/model-resolution.md`](docs/architecture/model-resolution.md) |
|
||||
| Why a current product boundary exists | [`docs/decisions/README.md`](docs/decisions/README.md) |
|
||||
| Recurring implementation mistakes and post-mortems | [`.agents/logs/ai_lessons.md`](.agents/logs/ai_lessons.md) |
|
||||
| Recurring implementation mistakes and post-mortems | [`docs/architecture/lessons.md`](docs/architecture/lessons.md) |
|
||||
|
||||
`README.md` is user-facing documentation and `CHANGELOG.md` is user-facing
|
||||
release history. Neither is the source of truth for internal architecture.
|
||||
@@ -91,7 +90,11 @@ covered by the `styles.css` manifest.
|
||||
- `api/scanner.py` and `scraper.py` own scan orchestration and enrichment; both scan
|
||||
routes map request fields to scraper switches in one place (`_scraper_flags`).
|
||||
`civitai_client.py` is the scraper's network side (version by hash, model page, media),
|
||||
telling "Civitai does not know the file" from "Civitai could not be asked".
|
||||
telling "Civitai does not know the file" from "Civitai could not be asked" (and why).
|
||||
`net_proxy.py` is the proxy every request of the plugin goes through (Settings → Network,
|
||||
`NETWORK_PROXY` in api/config.json; empty = urllib's own: HTTPS_PROXY or the Windows proxy),
|
||||
handed to the scan process as HTTP(S)_PROXY, and the names of why a request failed;
|
||||
`api/network_check.py` (`GET /anomalous/civitai_check`, this computer only) tests Civitai through it.
|
||||
`api/scan_report.py` turns the scraper's per-model report lines into one result per
|
||||
scan: kept as the last scan (`/anomalous/last_scan`), recorded in the activity log, each
|
||||
model located so the browser can open it (`/anomalous/scan_model`).
|
||||
@@ -101,6 +104,36 @@ covered by the `styles.css` manifest.
|
||||
for the scan page and lists the last two, by the same sidecar rules the scraper uses
|
||||
(`sidecar_info`, `is_unmatched`, `unmatched_reason` in `model_identity.py`); it also
|
||||
lists the model files in other formats, which scans never read, so the page can say so.
|
||||
- `api/mcp_server.py` serves `/anomalous/mcp`, the Model Context Protocol endpoint AI apps
|
||||
connect to (Streamable HTTP; the 2026-07-28 stateless protocol and the earlier
|
||||
`initialize`-based ones; this computer only, Host and Origin checked against DNS
|
||||
rebinding). `api/mcp_tools.py` holds the reading tools (summarised views over the same
|
||||
stores the pages use: models, scans, output images, combos, recipes, saved prompts,
|
||||
generated audio, activity log) and `call_tool`; `api/mcp_actions.py` the acting ones:
|
||||
canvas actions done by the open page, scans and Anomalous TTS speech through this
|
||||
server's own routes. `api/mcp_bridge.py` hands a canvas action to the page over
|
||||
ComfyUI's websocket and waits for its answer; `web/modules/mcp_bridge.js` is the page
|
||||
side, doing it with Anomalous's canvas code (one Ctrl+Z step each, logged as the AI's).
|
||||
User guide: `docs/guides/mcp.md`.
|
||||
- `api/backup.py` is Settings → Backup (`/anomalous/backup/*`): one .zip of the user's stores
|
||||
(recipes, combos, materials, parameters, optionally ComfyUI's saved workflows), the plugin
|
||||
settings without the Civitai key, and each model's user layer and own cover; putting it
|
||||
back adds what is missing, replaces differing files only on request (Recycle Bin first)
|
||||
and finds models again by SHA-256, else by folder, path and size. Its model list (every
|
||||
model's folder, path, size, SHA-256, link) names the models a new computer lacks.
|
||||
`ui_backup.js` is its export and import dialogs; `ui_backup_downloads.js` downloads the
|
||||
lacking models again through Model Check's downloads, into the path they had.
|
||||
- `api/lora_info.py` reads a LoRA's safetensors header to tell whether it trains the text
|
||||
encoder (`/anomalous/lora_info`), so inserting it wires the CLIP line only then.
|
||||
- `api/model_download.py` downloads the models a workflow misses (Model Check's
|
||||
"Download", `/anomalous/download/*`): one at a time into `<file>.part`, continued after
|
||||
an interruption, checked against the SHA-256 before it gets its name, never over an
|
||||
existing file; also where downloads go by default (`download_settings.json`).
|
||||
`api/download_sources.py` finds the one exact file: Civitai by hash, a Civitai version
|
||||
link, or a Hugging Face / GitHub file link the workflow carries, else a file of the same
|
||||
name in ComfyUI-Manager's model list (`api/manager_catalog.py`), checked by the hash.
|
||||
`api/hf_card.py` gives a model downloaded from Hugging Face its model card's example image
|
||||
as cover and its trigger words and example prompts as notes (never over the user's).
|
||||
- `api/workflow_schema.py`, `api/recipe_schema.py`, `api/recipe_images.py`, and
|
||||
`api/recipe_store.py` own recipe validation/shaping, images, CRUD, history, and
|
||||
integrity receipts; `api/recipes.py` is the HTTP facade. `api/recipe_constants.py`
|
||||
@@ -323,11 +356,12 @@ covered by the `styles.css` manifest.
|
||||
`ui_settings_hub.js` owns the display preferences (view mode, scale, atmosphere, window
|
||||
layout) and their application, the gear and language redraws; `ui_settings_page.js`
|
||||
is the settings page the gear opens (a tool page with Back): look and language, model
|
||||
cards and memory with the card image cache, folders, workflow fingerprints, opening
|
||||
mode and window, help; `ui_feedback_dialog.js` is the feedback window (Home and the
|
||||
cards and memory with the card image cache, folders, workflow fingerprints, downloads,
|
||||
backup (`ui_backup.js`), opening mode and window, help; `ui_feedback_dialog.js` is the feedback window (Home and the
|
||||
settings page open it): one text box, then `feedback.js` opens a GitHub issue with it in
|
||||
the browser's language, the environment folded at the end when attached (versions and
|
||||
hardware only, never a path or ComfyUI's command line), or copies the environment;
|
||||
hardware only, never a path or ComfyUI's command line), or copies the environment, and
|
||||
`openRepoPage()` opens the project page for a Star (on a press only; nothing is fetched);
|
||||
`ui_rail_tools.js` owns the rail's tool buttons (scan, doctor, current node);
|
||||
`ui_browser_navigation.js` owns shared panel hiding/cleanup and workspace return,
|
||||
including Esc on the workspace panel (`nbPanel`, below the header and right of the
|
||||
@@ -338,7 +372,9 @@ covered by the `styles.css` manifest.
|
||||
list page from `ui_scan_lists.js` (not scanned, unmatched, other formats, the last scan's
|
||||
models) with Back, a button for the whole list (scan these / look them up again) and rows
|
||||
that open a model, whose Back returns to the list, or scan it;
|
||||
`scan_results.js` holds the words for scan results, shared with the activity page.
|
||||
`scan_results.js` holds the words for scan results, shared with the activity page;
|
||||
`net_failure.js` the words for why Civitai could not be reached (net_proxy.py's codes), for
|
||||
the scan results and Settings → Network's connection test.
|
||||
`scan_runner.js` starts and follows scans (every folder, picked or listed models, or one
|
||||
model from its card), shows the result and refreshes node drop-downs, hashes and the
|
||||
grid afterwards; folder visibility/order lives in
|
||||
@@ -350,7 +386,9 @@ covered by the `styles.css` manifest.
|
||||
shortcuts (the second keeps its old "materials" ids), their fallback when ComfyUI's keybinding does not fire, and the
|
||||
settings control that opens ComfyUI's keybinding editor.
|
||||
- `ui_model_sources.js` renders the Models page's Sources view (the last type chip): where each model is downloaded, for the open workflow or every model, with each link editable and saved in place; `model_source_links.js` owns its data and actions (collecting the workflow's models, resolving them here, saving a link to the model's information and the workflow, the canvas note and the clipboard list).
|
||||
- `ui_gallery_card.js` builds one output image's gallery card (viewer, workbench, cover pick, drag, star, delete);
|
||||
- `ui_gallery_card.js` builds one output's gallery card (viewer, workbench, cover pick, drag, star, delete; a video's card
|
||||
plays in the viewer and has no workbench, star or menu); `gallery_filter.js` holds what the gallery lists (All / Pictures /
|
||||
Videos and the formats hidden in Settings, per browser) and `ui_gallery_filter.js` draws both controls;
|
||||
`gallery_card_menu.js` is its right-click menu (insert as a Load Image node reading the output file, parameters);
|
||||
`ui_keep_menu.js` is the star's keep menu (also the workbench's Keep): the whole workflow, the combo or the prompts,
|
||||
each saying where it goes, opening what was kept already or removing it (to the Recycle Bin through the recipe,
|
||||
@@ -359,6 +397,14 @@ covered by the `styles.css` manifest.
|
||||
- `ui_combos.js` renders the Combos page's list (搭配: a main model, LoRAs and a prompt, formerly Prompt Notes): search,
|
||||
New, and a card per combo with its model's cover and Put on canvas; a card opens the combo's editor, and dragged onto the
|
||||
canvas (`material_drag.js`) becomes a new group of nodes where it is dropped. Combos keep the notes' files.
|
||||
A combo can instead hold a node structure (`data.kind === 'nodes'`): `combo_structure.js`
|
||||
captures the canvas's picked nodes (subgraphs aside) with the links between them and puts
|
||||
them back (other boxes stored by name and restored untouched, slots = model drop-downs and
|
||||
text boxes, the folder a drop-down lists read from ComfyUI's own `/models` lists, inputs fed
|
||||
from unpicked nodes noted to wire);
|
||||
`combo_slots.js` holds its rules that need no canvas; `ui_combo_structure.js` is the save
|
||||
dialog, the slot editor, the slot model picker and "New"; `ui_translation_peek.js` is the
|
||||
read-only translation panel beside its prompts.
|
||||
- `detached_graph.js` lays a workflow on a graph of its own, off the canvas, under an id of its own: ComfyUI keeps widget
|
||||
values by graph id, so a copy under the canvas's id would write its values onto the canvas.
|
||||
- `recipe_save.js` saves Workflow Recipes for the canvas save and for an output image's workflow (the gallery's keep menu,
|
||||
@@ -366,11 +412,19 @@ covered by the `styles.css` manifest.
|
||||
- `ui_apply_receipt.js` is the receipt of values written to a node from a panel (Current node's parameters) with its Undo. `node_material_actions.js` owns prompt envelope extraction (`extractMaterialPromptEnvelope`) and the node writes shared by those panels and prompt drops.
|
||||
- `ui_update_guide.js` and `update_guide_data.js` own the non-intrusive update guide modal (accessible via header button `#anomalous-update-notice-btn`, the Help modal and Home's "What's new"; the current guide's ID and steps live in `update_guide_data.js`) with full bilingual localization. `ui_spotlight_tour.js` provides the interactive spotlight mask tour (`startSpotlightTour`), gliding smooth focal box highlights across the rail's pages and tools with directional tooltip cards and keyboard navigation; steps whose target is not on screen are skipped. The browser tour goes in the order the pages are used (rail, Gallery and what its ☆ keeps, Workflows, Combos, Prompts, then the tools), with its text in `locales.js`. Other views pass their own `steps` (text from locale keys) and an optional `onClose`; the GPT-SoVITS import window does.
|
||||
- `tool_registry.js` holds the tool icons shared by the rail and Home.
|
||||
- The rail's tool slots (`ui_rail_tools.js`, fixed) hold scan, doctor and current node; settings sits at the rail's bottom. The rail's Prompts entry opens Prompt Studio beside the canvas (the browser folds away and comes back when it closes). There is no toolbox: the other tools open from their pages (workflow share codes on Workflows, Model Sources as a view of Models, opened from the doctor too, translation in Current node).
|
||||
- The rail's tool slots (`ui_rail_tools.js`, fixed) hold scan, doctor and current node; settings sits at the rail's bottom. The rail's Prompts entry is Prompt Studio's page (`ui_prompt_composer.js`); its "Beside the canvas" moves the studio into a drawer by the canvas with the browser folded away, and "Back to the window" returns it. There is no toolbox: the other tools open from their pages (share and import on Workflows, Model Sources as a view of Models, opened from the doctor too, translation in Current node).
|
||||
- `ui_share.js` is share and import (the Workflows page's ⇅): the canvas workflow as a share
|
||||
code in Chinese characters or letters, and one place to paste a code or workflow JSON or drop
|
||||
a workflow file, an image, a recipe package or a backup .zip. `ui_recipe_package.js` exports
|
||||
a recipe package from a card or the detail's More menu and takes one in as a new card. `share_code.js` is the AMB2 code (lean workflow
|
||||
checked by rebuilding it, else the whole workflow; AMB0/AMB1 still open); details in
|
||||
`docs/architecture/frontend.md`.
|
||||
- `ui_model_types.js` owns the models page's type chips (one per models folder, with its count) and
|
||||
`owner.modelScope`, what the grid lists: a whole type, or one list folder shown as a crumb. The
|
||||
grid's cards set `currentType/PathIdx/Subfolder` to their own model's folder (`focusModel`),
|
||||
which the editor, the scanner and "add to canvas" read.
|
||||
which the editor, the scanner and "add to canvas" read. `ui_model_search.js` is the models
|
||||
search in the header (shown on the models page through the shell's `data-page`): the grid
|
||||
lists the models of its type matching every word (`owner.modelQuery`).
|
||||
- `model_source.js` shows where a model's information came from: the card badge (marked only
|
||||
when inferred from the file, ≈, or not scanned yet) and the detail header's source line with
|
||||
the fields the user set.
|
||||
@@ -381,7 +435,7 @@ covered by the `styles.css` manifest.
|
||||
safe docked sidebar preservation upon node addition, and multi-type node dispatch; `ui_detail.js`
|
||||
coordinates detail display (the file card with Show in folder in `model_location.js`), `ui_model_editor.js` owns metadata editing, and
|
||||
`ui_model_selector.js` owns advanced selection. `ui_gallery.js` (cards in
|
||||
`ui_gallery_card.js`) and `ui_gallery_detail.js` own generated-image browsing and workbench lifecycle,
|
||||
`ui_gallery_card.js`, filters in `gallery_filter.js` / `ui_gallery_filter.js`) and `ui_gallery_detail.js` own generated-output browsing and workbench lifecycle,
|
||||
with stage interaction in `ui_image_stage.js` and metadata tabs in
|
||||
`ui_image_inspector.js`.
|
||||
- `ui_recipes.js` / `ui_recipe_detail.js` and `ui_notebooks.js`
|
||||
@@ -462,7 +516,12 @@ covered by the `styles.css` manifest.
|
||||
say "doctor") and puts a found file into its node; `ui_doctor.js` is the
|
||||
Model Check page and `ui_doctor_banner.js` the bar over the canvas when an opened workflow
|
||||
misses models (it also owns the check-on-open preference that Settings → Workflows and the
|
||||
bar's "Don't show again" set). `ui_node_assistant.js` owns the current-node panel (model actions, LoRA
|
||||
bar's "Don't show again" set). `model_download.js` finds a missing model's source,
|
||||
decides where its file goes (the download settings, `{base}` = its base model's folder),
|
||||
follows the downloads and puts each finished file into its nodes (the placement rules
|
||||
are `download_places.js`, without imports so they are tested alone); `ui_model_download.js`
|
||||
is the page's Download buttons, progress and the dialog that shows and changes where
|
||||
each file goes. `ui_node_assistant.js` owns the current-node panel (model actions, LoRA
|
||||
insertion, model cards and history), `ui_node_model_picker.js` owns native combo
|
||||
replacement, `ui_node_parameters.js` renders the panel's parameters section, and
|
||||
`node_parameter_sets.js` merges the node type's saved values (material files and recipe
|
||||
|
||||
@@ -1,5 +1,94 @@
|
||||
# 📈 Anomalous Model Browser Changelog
|
||||
|
||||
## v1.59.0 Beta (Model Downloads, Backup, Share Codes, Node Combos, MCP, Light Theme) — 2026-10-17
|
||||
|
||||
### ⬇️ Download missing models from Model Check (模型检查一键下载缺失模型)
|
||||
- **Download** next to a missing model, and **Download all** for every one, when its source is certain: Civitai found the very file by the workflow's fingerprint, or the workflow carries a link to the file (a shared workflow's model links, or the download links in ComfyUI's own templates on Hugging Face). A file name alone is never enough, and a link to a different file than the workflow's is not offered.
|
||||
- **You see where it goes before it starts**: the dialog shows, per file, its size, source and base model, the full path it will be saved to, and the free disk space. The models folder (when you have several, e.g. through extra_model_paths) and the subfolder can be changed right there; the folder you pick is remembered per type.
|
||||
- **Your folder habits (按你的文件夹习惯放)**: Settings → Model downloads sets the default. *Where the workflow says* (default) puts the file in the subfolder the workflow names, so nothing in the workflow changes; a model without one goes into the folder set below (`Downloads`). *A set folder* always uses that folder, and the node is moved to the new place (Ctrl+Z undoes it). Write `{base}` in the folder to sort by base model (SDXL, Flux, Illustrious…); a folder of that name you already have is used, in whatever spelling.
|
||||
- **Safe by default**: it downloads into a temporary file and checks the fingerprint before giving it its name, so a broken or wrong download never shows up in your models; an existing file is never replaced; an interrupted download continues where it stopped. `.ckpt` / `.pt` files get a warning (they can run code), and a source without a fingerprint says it cannot be checked. Your Civitai API key (needed for some models; Settings → Model downloads, or the scan page) is only sent to Civitai.
|
||||
- Progress shows in the row with Cancel; when a file is done, the nodes using it get it, a short message says so, the new files are scanned for their information and cover, and the activity log records the download (source, size, whether the fingerprint matched).
|
||||
- **Base models, VAEs, text encoders… from ComfyUI-Manager's list (Manager 模型清单)**: models that are not on Civitai (Flux's VAE, T5 and CLIP text encoders, upscalers, ControlNets…) are now looked up by file name in the model list of the ComfyUI-Manager installed here, which has a few hundred of them, mostly on Hugging Face. When the workflow carries the model's fingerprint, only the listed file with that fingerprint is offered, so it is certainly the same file; without one, a single listed file of that name is offered as **Download by name**, with a warning, and never in Download all.
|
||||
- **Hugging Face first with the mirror (开镜像时优先抱脸)**: when a model is on Civitai and the very same file (same fingerprint) is on Hugging Face too, through the workflow's link or ComfyUI-Manager's list, it now downloads from Hugging Face while the China mirror is on: usually much faster there, and no Civitai login. Civitai's name, base model and page are still shown.
|
||||
- **Covers and prompts from Hugging Face (抱脸模型的封面和示例提示词)**: a model downloaded from Hugging Face gets its model card's first example image as its cover, and its trigger words, example prompts and base model in its notes, with the repository as its link. Nothing you or a scan already set is replaced. Hugging Face cannot be searched by fingerprint, so this is for files downloaded from it.
|
||||
- **Text encoders, CLIP vision and upscalers shown by default (文本编码器等默认显示)**: these folders are now shown and scanned by default, also where an earlier version saved them as hidden (once; you can hide them again in Settings → Model folders). Scanned models get fingerprints, so workflows you share carry them for T5, CLIP and the like too and Model Check can find the very file for whoever opens them.
|
||||
- **One site down does not stop the others (一个网站连不上不影响其他)**: when Civitai does not answer (common in mainland China), the rest of the lookup no longer gives up: it stops asking Civitai and still asks Hugging Face (through the mirror) and the others.
|
||||
- **AI apps can download too (AI 也能下载)**: three new MCP tools: `download_missing_models` (what Model Check's Download all does, for the open workflow), `download_model` (one model from a Civitai version link, a Hugging Face or GitHub file link, or a Civitai hash, into the folder of its type) and `download_status`. Files go where Settings → Model downloads says, are checked the same way, and are never written over an existing file.
|
||||
- **Hugging Face through the China mirror (Hugging Face 国内镜像)**: Settings → Model downloads can send Hugging Face downloads through hf-mirror.com, which is on by default for the Chinese interface. When a download from Hugging Face itself fails, the row suggests the mirror.
|
||||
|
||||
### 🌐 A proxy for the plugin, and why Civitai cannot be reached (代理设置与连不上的原因)
|
||||
- **Settings → Network → Proxy address**: the plugin's own requests (scans, model downloads, translation, update checks) go through the proxy written here, e.g. `127.0.0.1:7890`. Empty uses the system proxy as before. For a VPN app in PAC mode or one that only proxies the browser: the browser reached Civitai, the scan did not. Only this plugin uses it; ComfyUI and other node packs are untouched.
|
||||
- **Test connection** right below says whether Civitai answers now, and if not, why.
|
||||
- **A scan that cannot reach Civitai says why**: no answer (often no proxy), refused (wrong proxy port, VPN off), the address not found, a failed secure connection, or an error from Civitai, with what to try.
|
||||
|
||||
### 💾 Backup and restore (备份与恢复)
|
||||
- **Settings → Backup → Export** packs your Workflow Recipes, combos, materials and saved prompts, parameter sets, the plugin settings and the names, notes and covers you gave your models into one .zip, optionally with ComfyUI's own saved workflows. Keep it on a cloud drive or a USB stick. The Civitai API key is never included.
|
||||
- **Import** first lists what is in the backup next to what this computer has (new, the same, different), and the models it finds again by file hash, even after they were renamed or moved. By default it only adds what is missing; "take the backup's" replaces differing files and puts this computer's in the Recycle Bin. A model's name and notes are filled in where empty, and your own cover takes the place of the Civitai one. Nothing is deleted.
|
||||
- Models' scan results (.info: names, trigger words, base models from Civitai) are in the backup too, so they survive Civitai removing a model; Civitai's covers can be added (they can be hundreds of MB). Both are put back only where this computer has none.
|
||||
- **Download what the new computer lacks**: the backup also keeps a model list (each model's place, size, fingerprint and link). Importing says how many of those models this computer lacks; **Find downloads…** looks them up like Model Check (Civitai by hash, the saved link, Hugging Face), shows sizes and sources, and downloads the ticked ones one by one into the folder and file name they had, each checked against its fingerprint and scanned after.
|
||||
|
||||
### 🔗 Short share codes, one place to import (短分享码与统一导入)
|
||||
- **Share codes are 5–10 times shorter (AMB2)**: only what a workflow needs is kept — node types, values, links, positions, titles, groups and the models' fingerprints — and ComfyUI fills in the rest when it opens. A Chinese-character spelling ("汉字码") is the shortest; a letter spelling ("字母码") posts anywhere, for chats or comment sections whose word filters stop the characters. Before a code is given, it is opened once and compared with the canvas; a workflow it cannot hold exactly (subgraphs, for one) is shared whole. Nodes from node packs keep their links on a computer that has not installed the pack yet. AMB0 and AMB1 codes still open.
|
||||
- **Share and import in one place**: the Workflows page's ⇅ makes the code as soon as it opens (with the fingerprint and download-link notes), and takes anything in: a pasted share code or workflow JSON, or a dropped or chosen workflow file, an image with a workflow, or a backup .zip.
|
||||
|
||||
### 📦 Recipe packages are back (配方包重新开放)
|
||||
- A recipe card's export button (and Export in the recipe's More menu) makes a **recipe package**: one .zip with the recipe's workflow, name, notes, tags, parameters, cover and model thumbnails (its versions only when ticked), to give to someone. Dropped into the ⇅ of their Workflows page, it becomes a new recipe card, optionally opened on the canvas; a recipe of the same name is never replaced (the new one is kept as "Name (2)"). Packages are checked for size, paths and checksums before anything is written, and a failed import leaves nothing half done. Send it as a file in chat apps.
|
||||
|
||||
### 🧱 Combos from your own nodes (按自己的节点结构做搭配)
|
||||
- A combo is no longer only "a main model, LoRAs and a prompt". Pick nodes on the canvas (a Flux UNet with its two text encoders and VAE, a video model, a node pack's loader…), choose **New → From the nodes picked on the canvas**, and the combo keeps those nodes and how they are wired. Its editor then lists the **slots**: each model drop-down (a node with several has several) to change with a picker that shows that folder's models with covers, filtered by base model, and each prompt box to write in. **Put on canvas** lays the nodes out wired as saved, with your choices in them.
|
||||
- Only certain things are kept: nodes that pick model files and nodes that hold text, and the links between them. Other picked nodes (samplers, latents…) are left out and their ports stay open for you to wire, as before. Every other value of the kept nodes (strengths, CLIP type…) is stored by name and put back untouched. When the dialog saves, you see what is kept and what is not, and can rename slots or untick the ones that should stay as they are.
|
||||
- A node combo's page is grouped into Models, LoRAs and Prompts, in the order they flow (loaders first, LoRAs from the model outwards, the positive prompt before the negative); "LoRA 1" is the one next to the model. Each model shows its cover (looked up once for models chosen on the canvas), a two-line name, and its node's saved values (strength, CLIP strength, type) as small read-only tags. A whole row opens the picker; prompt boxes grow with their text. The top bar of the combo editor no longer floats 20 px down with the page showing above it.
|
||||
- Each slot names the node it is on, and its node pack when that is not ComfyUI's own. A prompt has a small "译" button: a read-only panel beside it shows the translation (English for Chinese, Chinese for English); Prompt Studio stays the place to edit prompts.
|
||||
- Nodes holding several models (Load CLIP Triple, LoRA stacks of rgthree, Comfyroll and Easy-Use, Easy-Use's full loader): each model is its own slot, and each saved value goes to the row with its number (strength_02 under the second LoRA); more than four values fold behind "+N". Empty rows of a stack (None) start unticked when saving, so a ten-row stack does not make ten empty slots. rgthree's Power Lora Loader draws its own rows instead of drop-downs and is not kept.
|
||||
- **Right-click → Save as combo**: the canvas's node menu saves the picked nodes (or the node you right-clicked, when it is not among them) the same way, without opening the browser first. It shows only when there is a model or text node to keep.
|
||||
- Every other value of a saved node stays as it was saved: the combo always puts down exactly what you saved, so a Flux combo cannot fall back to a node's default CLIP type or strength. For other values, save another combo.
|
||||
- On another computer, missing nodes are named with their node pack before anything is placed; a box or link a node pack has since renamed is reported after placing, never guessed. **New → Same structure as …** starts another combo from a structure you already saved.
|
||||
- **Save as combo keeps every picked node**, not only the model and text nodes: an upscale model's loader with the node that uses it and the Save Image after it, a ControlNet's nodes, any small group of nodes that work together. The models and prompts in it can still be changed in the combo; the other nodes are listed under **Nodes put back as saved** with their values, and go back on the canvas exactly as saved (change them on the canvas and save the combo again).
|
||||
- A link coming in from a node that was not picked cannot be kept; the save dialog, the combo and its card (**1 input to wire**) say which inputs to connect after putting it down. Subgraphs cannot be kept yet. A combo with no models shows its nodes' names on the card. Combos saved before work as they did.
|
||||
|
||||
### 🤖 AI apps can use your library (MCP，让 AI 用你的库)
|
||||
- While ComfyUI runs, AI apps that speak MCP (Claude Code, Claude Desktop, Cursor, Cherry Studio, VS Code…) can connect to `http://127.0.0.1:8188/anomalous/mcp` and ask about your library: models with their Civitai information, trigger words and your notes, which models are unmatched and why, output images and the settings that made them, combos, workflow recipes, saved prompts, generated speech and the activity log. A model's cover or an image's thumbnail can be attached for apps that look at pictures.
|
||||
- It can also act on the workflow open in ComfyUI: rewrite or extend the prompts, swap a node's model, add a LoRA (it goes after the model loader and the LoRAs already there; one that also trains the text encoder takes the CLIP line too, from the checkpoint or, in a UNet workflow, from its CLIP loader, and everything those fed, such as both prompts, moves behind it; one that does not, like most Flux and Anima LoRAs, patches the model only), place one of your combos, check and put back missing models, and run the workflow; and start scans, or have an Anomalous TTS voice read a line. Every canvas change is one Ctrl+Z step, shows a note on the page and is in the activity log marked "AI". Nothing deletes or renames files; AI apps ask before calling a tool that changes something.
|
||||
- Only this computer can connect, and requests from web pages, other hosts or through a proxy are refused. It costs no memory while nobody is connected. Setup for each app: `docs/guides/mcp.md`.
|
||||
|
||||
### ☀️ Light theme (浅色主题)
|
||||
- **Settings → Appearance → Theme**: **Follow ComfyUI** (the default), **Dark**, **Light** or **Abyssal Scarlet**. Following ComfyUI switches with ComfyUI's color palette at once, light with a light palette and dark with a dark one. The same choice is in ComfyUI's settings under Anomalous Model Browser (it replaces the Abyssal Scarlet on/off switch; Abyssal Scarlet stays chosen if it was on).
|
||||
- Every page, the Prompt Studio drawer and the dialogs follow the theme. The image viewer stays dark, like ComfyUI's own, and so do the labels and buttons over pictures (a model card's name, a recipe's cover labels, the gallery's View parameters button), so their text reads on any picture. Grey hint and secondary text is darker in the light theme so it reads clearly. The dark themes look as before.
|
||||
- The "all models are here" banner on a recipe now reads green instead of pink, and the voice page's **Import character** button has dark text on its light blue in the dark themes.
|
||||
|
||||
### 🎯 Opens where your node is; number keys for the pages (按节点打开、数字键切页)
|
||||
- **Opening follows the selected node**: open the browser with a text node selected on the canvas and Prompt Studio opens on that node; with a KSampler, a model loader or any other node, Current node opens. With nothing selected it reopens the page you used last, as before. It can be turned off in Settings → Opening and window.
|
||||
- **Prompts is a page like the others (提示词变成普通页面)**: the rail's Prompts entry (and 7) now shows Prompt Studio inside the browser window instead of folding the window away. **Beside the canvas** on the page moves it next to the canvas as before, to see the canvas while you edit (its side and width are remembered); **Back to the window** returns it.
|
||||
- **Prompt Studio's own shortcut is gone** (Ctrl+Shift+L, and its row in ComfyUI's settings and the Extensions menu): it opens from the rail, with 7 while the browser is open, or by opening the browser with a text node selected. The browser's shortcut (Ctrl+Shift+M) stays.
|
||||
- **1–9 go to the pages on the left** while the browser is open (Home, Activity, Models, Gallery, Workflows, Combos, Prompts, Voices, Audio library). Numbers typed into a box stay in the box, and the rail's tooltips show each number. W/S were not used because W is ComfyUI's own Workflows key.
|
||||
|
||||
### 🔎 Model search in the header (模型搜索放到顶栏)
|
||||
- The models page's search sits in the header now, instead of inside the folder list you had to open first. It matches the name, file name, folder, trigger words, base model and notes, every word you type, in the type you are looking at; Esc clears it. It also searches all the models of the type, where the old box only filtered the cards on screen.
|
||||
|
||||
### 🧩 Inserting a LoRA wires it properly (插入 LoRA 连线更聪明)
|
||||
- **Current node → Insert LoRA before / after** now works on the usual workflows it refused before: a checkpoint whose CLIP feeds both prompts (both now hang behind the LoRA), and UNet / Flux / Anima workflows with their own CLIP loader. Whether the CLIP line goes through the LoRA is decided by the LoRA file itself: one that also trains the text encoder (most SD 1.5 / SDXL / Illustrious LoRAs) takes it, from the checkpoint or the CLIP loader; one that does not (most Flux and Anima LoRAs) is inserted as a model-only LoRA loader. Inserting before a sampler moves only that sampler's model, and the CLIP of the prompts wired into it.
|
||||
|
||||
### 🌐 Your own DeepL key for translation (DeepL 翻译密钥)
|
||||
- **Settings → Translation → DeepL key**: prompts are translated with Google's free endpoint (MyMemory when Google cannot be reached); with your own DeepL API key, DeepL comes first. Saving the key translates one sentence to show it works, or says why not (a key DeepL refuses, a used-up month). The Free plan gives 500,000 characters a month. The key stays on this computer and is never put in a backup.
|
||||
- A DeepL key saved by hand in `config.json` before did not work any more: DeepL stopped taking the key as a request parameter in 2025. It now goes in the Authorization header.
|
||||
|
||||
### 🎬 Videos in the gallery (图库显示视频)
|
||||
- **The gallery shows your video outputs** (MP4, WebM, MOV, MKV) next to the pictures: each card shows the video's first frame with a ▶ badge naming its format and plays it muted while the pointer is on it (or always, as Settings → Video covers says for model covers); clicking it plays the video with sound in the viewer, filling the window, and a double-click goes full screen. Closing the viewer (×, a click beside it or Esc) stops it. Videos are deleted (to the Recycle Bin) like pictures, and a video dragged onto the canvas becomes a Load Video node; View parameters, the star and the right-click menu stay with pictures, which carry the parameters. A first contribution from outside: thanks to [@namnamjeopjeop](https://github.com/namnamjeopjeop) (cocorang) and [@wkdtjs](https://github.com/wkdtjs) (PR #21).
|
||||
- **All / Images / Videos (全部 / 图片 / 视频)** beside the gallery's search, shown once your output folder has a video. **Settings → Gallery** hides formats you never want to see (GIF, say); click a format again to show it. Both are remembered in this browser. The recipe cover picker and AI apps (MCP) still see pictures only.
|
||||
|
||||
### 🧪 Tests in the repository (测试公开)
|
||||
- The plugin's tests are now part of the repository: `tests/` (Python tests for the backend, `.mjs` tests for the frontend rules), `node tools/run_tests.mjs` runs them all with the structure check, and [docs/guides/testing.md](docs/guides/testing.md) says how to run and write them. They need no network or GPU and only touch temporary folders. Developer notes no longer point at tests that were removed earlier.
|
||||
|
||||
### ⭐ A Star for the project (点个 Star)
|
||||
- Home's "Found a problem, or have an idea?" band ends with **⭐ Star on GitHub**, and Settings → Help and feedback has **Star it on GitHub**: both open the project page, where the ☆ Star button is. Nothing pops up and nothing is fetched; it is there for whoever wants it.
|
||||
|
||||
### ⚠️ Damaged model files are named as such (模型文件损坏会直接说出来)
|
||||
- A model whose file header cannot be read (a download cut short, or another tool that rewrote the header and failed) used to look like a model Civitai simply does not know, inferred from the file. A scan now checks the header of every model it cannot match and names these **文件损坏**: a red badge on the card, a line in the details, a warning on the scan page, and first place in its unmatched list with advice to download the model again (then **Look up again**). Only the header is read. Models scanned before keep their old reason until they are looked up again.
|
||||
|
||||
### Fixes
|
||||
- **Ctrl+Z after a scan (扫描后的 Ctrl+Z)**: after any scan, the first Ctrl+Z used to seem to do nothing: the workflow's saved fingerprints had changed, which ComfyUI counted as a step. A change of fingerprints alone no longer takes a step.
|
||||
- **Esc in a model's Outputs window (作品窗口里按 Esc)**: Esc closes the window of pictures made with a model; it used to close the model's details behind it and leave the window open.
|
||||
|
||||
## v1.58.3 Beta (Fix: canvas speed with other extensions) — 2026-10-15
|
||||
|
||||
### 🐢 The canvas stays smooth with other extensions (画布不再被其他插件拖慢)
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
# Claude 工作入口
|
||||
|
||||
本仓库的开发与维护规范以 AGENTS.md 为唯一正文，第 0 节是每次提交都必须做到的要求；这里只做入口，不另写规则。
|
||||
|
||||
@AGENTS.md
|
||||
@@ -1,11 +0,0 @@
|
||||
# Gemini 工作入口
|
||||
|
||||
在本插件中修改任何文件前，先阅读并遵守根目录的 [AGENTS.md](AGENTS.md)。它是本仓库开发与维护规范的唯一正文，适用于 UX 调整、修复和重构；第 0 节是每次提交都必须做到的要求。
|
||||
|
||||
随后阅读 [ARCHITECTURE.md](ARCHITECTURE.md)，按其中的阅读地图加载与当前任务有关的专题。
|
||||
|
||||
不要只凭当前任务描述直接追加实现。先确认现有责任模块、状态来源、关闭清理和样式规则；完成后按规范验证并报告结果。
|
||||
|
||||
如果当前工具没有加载本文件，任务发起者应明确要求 Gemini 阅读 `GEMINI.md` 和 `AGENTS.md`。文件的存在不代表工具已实际读取，更不代表检查已自动执行。
|
||||
|
||||
前端设计任务按 AGENTS.md 第 4 节和 [前端架构](docs/architecture/frontend.md) 进行，不使用历史视觉建议作为当前任务单。
|
||||
@@ -9,9 +9,10 @@
|
||||
|
||||
[](https://github.com/ltdrdata/ComfyUI-Manager)
|
||||
[](LICENSE)
|
||||
[](CHANGELOG.md)
|
||||
[](https://www.bilibili.com/video/BV1a1bv68EuA/)
|
||||
[](https://youtu.be/hAvsj7uiaCw)
|
||||
[](CHANGELOG.md)
|
||||
[](https://www.bilibili.com/video/BV1boHx61Enc/)
|
||||
[](https://www.bilibili.com/video/BV1bkpZ6uEZQ/)
|
||||
[](https://www.reddit.com/r/comfyui/s/n38ihf3DLf)
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -19,7 +20,7 @@
|
||||
|
||||
<br/>
|
||||
|
||||
<img src="docs/images/models.webp" width="900" alt="Models with their Civitai covers, base model and real names">
|
||||
<img src="docs/images/en-models.webp" width="900" alt="Models with their Civitai covers, base model and real names">
|
||||
|
||||
</div>
|
||||
|
||||
@@ -29,43 +30,56 @@
|
||||
|
||||
> **Anomalous Model Browser** takes care of the chores around ComfyUI in one window: browse your models with their Civitai covers and trigger words, look back through your outputs, keep workflows you have tuned, keep model combos and prompts, repair workflows whose models are missing, and, with the companion node pack [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS), give your characters a voice. An activity log shows everything the plugin changed.
|
||||
|
||||
### 🎬 Video Walkthrough & Demos
|
||||
* 📺 **YouTube**: [Watch Quick Walkthrough on YouTube](https://youtu.be/hAvsj7uiaCw)
|
||||
* 📺 **Bilibili**: [Watch Video Demo on Bilibili (在哔哩哔哩观看)](https://www.bilibili.com/video/BV1a1bv68EuA/)
|
||||
### 🎬 Videos and discussion
|
||||
* 📺 **Bilibili** (Chinese): [Video demo](https://www.bilibili.com/video/BV1boHx61Enc/) · [Character voices demo](https://www.bilibili.com/video/BV1bkpZ6uEZQ/)
|
||||
* 💬 **Reddit**: [The post on r/comfyui](https://www.reddit.com/r/comfyui/s/n38ihf3DLf)
|
||||
|
||||
### 👀 See it in action
|
||||
### 💡 What it helps with
|
||||
|
||||
*The pictures show the Chinese interface; English is one switch away in Settings.*
|
||||
**1. "Which LoRA was that again?"** A file called `download (3).safetensors` tells you nothing. Press **Scan** once: every model is looked up on Civitai by its file hash and gets its real name, cover, trigger words and base model. From then on you pick models by their covers.
|
||||
|
||||
**1. Scan once, then pick models by their covers.** Press **Scan** and every model is looked up on Civitai by its file hash, so even a file called `下载 (3).safetensors` gets its real name, cover, trigger words and base model.
|
||||
<p align="center"><img src="docs/images/en-scan.webp" width="520" alt="Scan results: each file matched on Civitai"></p>
|
||||
|
||||
<p align="center"><img src="docs/images/scan.webp" width="520" alt="Scan results: each file matched on Civitai"></p>
|
||||
**2. Someone else's workflow opens with red nodes.** Model Check says which models are missing. The ones you already have under another name or in another folder go back in with one press; the rest can be downloaded from Civitai or Hugging Face, and each download is checked against the model's fingerprint.
|
||||
|
||||
**2. Open someone else's workflow without hunting for models.** When models are missing, a bar over the canvas says so. One press puts back the ones you already have under another name or in another folder; the red outlines go away.
|
||||
<p align="center"><img src="docs/images/en-model-check.webp" width="760" alt="Model Check puts back three missing models with one press"></p>
|
||||
|
||||
<p align="center"><img src="docs/images/model-check.webp" width="760" alt="Model Check puts back three missing models with one press"></p>
|
||||
**3. "How did I make this picture?"** Search your outputs by prompt words, model or LoRA, then drag the picture onto the canvas: its workflow comes back.
|
||||
|
||||
**3. Find an old image and get its workflow back.** Search the gallery by prompt words, model or LoRA, then drag the image onto the canvas.
|
||||
<p align="center"><img src="docs/images/en-gallery.webp" width="760" alt="Gallery search, then drag an image onto the canvas"></p>
|
||||
|
||||
<p align="center"><img src="docs/images/gallery.webp" width="760" alt="Gallery search, then drag an image onto the canvas"></p>
|
||||
**4. The same model, LoRAs and prompt, again and again.** Pick those nodes on the canvas, right-click → **Save as combo**, and they are kept as a card beside the canvas. Next time press **Put on canvas** (or drag the card) and the wired group comes back where you click; nothing already on the canvas changes. Models and prompts in a combo can be changed later.
|
||||
|
||||
**4. Keep a model + LoRA + prompt combo and lay it down in one move.** Drag a combo card onto empty canvas: a wired group of nodes appears, and nothing already on the canvas changes.
|
||||
<p align="center"><img src="docs/images/en-combo.webp" width="760" alt="Picked nodes saved as a combo, then put back on the canvas as wired nodes"></p>
|
||||
|
||||
<p align="center"><img src="docs/images/combo.webp" width="760" alt="A combo card dragged onto the canvas becomes wired nodes"></p>
|
||||
**5. Long prompts are hard to edit.** Select a prompt node and Prompt Studio shows its text as tags: click to weight, drag to reorder, type to add (Chinese and other languages become English).
|
||||
|
||||
**5. Edit prompts as tags.** Select a prompt node and Prompt Studio opens its text as tags: click to weight, drag to reorder, type to add (Chinese and other languages become English).
|
||||
<p align="center"><img src="docs/images/en-prompt-studio.webp" width="760" alt="Prompt Studio editing a prompt node as tags"></p>
|
||||
|
||||
<p align="center"><img src="docs/images/prompt-studio.webp" width="760" alt="Prompt Studio editing a prompt node as tags"></p>
|
||||
**6. Characters that talk.** With the companion node pack [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS): GPT-SoVITS characters with a reference clip per emotion, and a script page to voice them.
|
||||
|
||||
**6. See what the plugin changed, and undo it.** Every change the plugin makes is listed in Activity, with **Undo** and **Find on canvas**.
|
||||
<p align="center"><img src="docs/images/en-voices.webp" width="760" alt="Character voices page"></p>
|
||||
|
||||
<p align="center"><img src="docs/images/activity.webp" width="560" alt="Activity log with an undo button"></p>
|
||||
**Changed your mind?** Everything the plugin changes is listed in **Activity**, with **Undo** and **Find on canvas**.
|
||||
|
||||
**7. Give characters a voice** (with [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS)): GPT-SoVITS characters with a reference clip per emotion, and a script page to voice them.
|
||||
<p align="center"><img src="docs/images/en-activity.webp" width="620" alt="Activity log with an undo button"></p>
|
||||
|
||||
<p align="center"><img src="docs/images/voices.webp" width="760" alt="Character voices page"></p>
|
||||
**Light on your computer**: the plugin itself loads no models, so it takes no graphics memory from image generation; the browser shows small thumbnails instead of full images, and its pictures are released a minute and a half after you close it. The voice models of Anomalous_TTS give their graphics memory back whenever ComfyUI needs it for image generation, and all of it on **Unload models**.
|
||||
|
||||
### 🌟 What it does
|
||||
### 📦 Install
|
||||
|
||||
1. In **ComfyUI Manager**, search `Anomalous Model Browser`, install it and restart ComfyUI. Or clone it into `custom_nodes`:
|
||||
```bash
|
||||
cd custom_nodes
|
||||
git clone https://github.com/DemonGatanjieu/Anomalous_Model_Browser.git
|
||||
```
|
||||
2. Press **Shift + B** to open it. On **Home**, **Scan model folders** fills in covers and trigger words, and **Tour the interface** points at each part of the window.
|
||||
3. Optional: for character voices, also install [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS) (search `Anomalous TTS` in ComfyUI Manager).
|
||||
|
||||
Scanning needs to reach Civitai. Without it, scan offline: base models are read from the files, and a later online scan fills in the rest (see **Network and backups** below).
|
||||
|
||||
<details>
|
||||
<summary><b>📖 Every page and tool, in detail</b></summary>
|
||||
|
||||
Every page has an icon on the rail at the left of the window.
|
||||
|
||||
@@ -75,9 +89,9 @@ Every page has an icon on the rail at the left of the window.
|
||||
| **🕘 Activity** | Everything the plugin changed, by day: on the canvas (which node settings changed from what to what, nodes added or removed, workflows opened) and in files (models, covers, recipes, materials, notes, images, audio). **Find on canvas** jumps to the node; a canvas change can be undone with **Undo** until something changes it again, the rest is recorded only. Your own edits are not listed. |
|
||||
| **📦 Models** | Your models with covers, trigger words and base model. Chips at the top switch between types (Checkpoint, LoRA, VAE…) and list a whole type, subfolders included; the folder list narrows it to one folder. **+** adds a loader node to the canvas; edit a model's name, notes and cover, or scan just that model. |
|
||||
| **🖼️ Gallery** | Your ComfyUI `output` folder. Search by prompt, model, LoRA, seed, file name or model hash; open an image to see the parameters it was made with, or drag it onto the canvas to get its workflow back. **☆** keeps a good one as a whole workflow, a combo (model, LoRAs, prompt) or just its prompts. |
|
||||
| **🪡 Workflows** | Workflow recipes: save a whole workflow or a part of one together with its models, cover, notes and parameters. Each card shows whether the models are on this computer; drag it onto the canvas to load it. Versions can be compared and restored. |
|
||||
| **🪡 Workflows** | Workflow recipes: save a whole workflow or a part of one together with its models, cover, notes and parameters. Each card shows whether the models are on this computer; drag it onto the canvas to load it. Versions can be compared and restored. A card's export button makes a **recipe package** (.zip) to give to someone; dropped into their ⇅, it becomes a recipe card. |
|
||||
| **🧩 Combos** | A main model, a few LoRAs and a prompt you like, kept together. **Put on canvas** (or dragging the card onto the canvas) lays them out as a wired group of nodes; nothing already on the canvas changes. |
|
||||
| **✍️ Prompts** | Opens Prompt Studio beside the canvas. Select a prompt node and its prompt boxes appear as tags, edited in place: click a tag to make it stronger or weaker, double-click to edit, drag to reorder, type to add (Chinese, Japanese and other languages are translated to English), and show each tag's meaning in Chinese, Japanese, Korean and other languages. Click a card (a few common ones, plus your saved prompts) to add it, or drag it straight onto a prompt box on the canvas (outlined green for positive, red for negative) or onto empty canvas for a new prompt node. With no prompt node selected you write a draft. |
|
||||
| **✍️ Prompts** | Prompt Studio, a page of the browser; **Beside the canvas** moves it next to the canvas to see both. Select a prompt node and its prompt boxes appear as tags, edited in place: click a tag to make it stronger or weaker, double-click to edit, drag to reorder, type to add (Chinese, Japanese and other languages are translated to English), and show each tag's meaning in Chinese, Japanese, Korean and other languages. Click a card (a few common ones, plus your saved prompts) to add it, or drag it straight onto a prompt box on the canvas (outlined green for positive, red for negative) or onto empty canvas for a new prompt node. With no prompt node selected you write a draft. |
|
||||
| **🎙️ Voices / 🎧 Audio** | With [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS) installed: import GPT-SoVITS characters, pick a reference clip per emotion, fix pronunciations, write a script under Voice-over on the same page and generate it right there; generated audio is listed in the audio gallery. Without it, the page explains how to install it and nothing else depends on it. |
|
||||
|
||||
Tools on the rail:
|
||||
@@ -85,31 +99,30 @@ Tools on the rail:
|
||||
* **Scan** reads your model folders and fetches covers, trigger words and base models from Civitai (by file hash, no extra Python packages).
|
||||
* **Model Check** (formerly Model Doctor) checks each workflow you open: when models are missing, a bar over the canvas says so and puts back, with one press, the ones you have under another name or folder (recognised by hash and file size); its page lists the rest with what you can do.
|
||||
* **Current node** (formerly Node Assistant) shows the model of the node you select, lets you swap it from the covers or insert a LoRA, and lists every set of values saved for that kind of node with exactly what applying it would change; saved values can be deleted there.
|
||||
* **Each tool on its page**: workflow share codes (import / export) on Workflows, Model Sources (where each model can be downloaded) on Models and in Model Check, translation on the prompt boxes in Current node.
|
||||
* **Each tool on its page**: share and import on Workflows (⇅: a short share code of the canvas workflow, in Chinese characters or letters; paste a code, or drop a workflow file, an image or a backup .zip), Model Sources (where each model can be downloaded) on Models and in Model Check, translation on the prompt boxes in Current node.
|
||||
* **Settings** (at the bottom): language, font size, thumbnails, video covers, which model folders to show, and feedback. Home and Settings have **Report a problem**, which opens a GitHub issue with your versions and GPU already filled in.
|
||||
|
||||
**Light on your computer**: the plugin itself loads no models, so it takes no graphics memory from image generation; the browser shows small thumbnails instead of full images, and its pictures are released a minute and a half after you close it. The voice models of Anomalous_TTS give their graphics memory back whenever ComfyUI needs it for image generation, and all of it on **Unload models**.
|
||||
**AI apps (MCP)**: Claude, Cursor, Cherry Studio and other apps that support MCP can use your library through `http://127.0.0.1:8188/anomalous/mcp`: answer questions about your models, outputs and saved prompts, and change the open workflow for you (prompts, models, LoRAs, combos, missing models), each change one Ctrl+Z step. Local only; setup in [docs/guides/mcp.md](docs/guides/mcp.md).
|
||||
|
||||
### 📦 Quick Installation
|
||||
</details>
|
||||
|
||||
1. Open your terminal in the ComfyUI `custom_nodes` directory:
|
||||
```bash
|
||||
cd custom_nodes
|
||||
git clone https://github.com/DemonGatanjieu/Anomalous_Model_Browser.git
|
||||
```
|
||||
2. Restart ComfyUI. *(Alternatively, install via **ComfyUI Manager** by searching for `Anomalous Model Browser`)*
|
||||
3. For character voices, also install [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS) the same way. It is optional.
|
||||
4. **Updating**: click the **!** button at the top right of the browser; it shows the current version, and **Check for updates / switch version** opens the version panel. Update to the newest release, go back to an earlier one or return to the latest, then restart ComfyUI. The plugin never checks for updates on its own.
|
||||
<details>
|
||||
<summary><b>⚙️ Opening, shortcuts and updating</b></summary>
|
||||
|
||||
Open the browser with **Shift + B** (again to close), the floating button on the canvas, or **Extensions → Anomalous Model Browser** in ComfyUI's menu. **ComfyUI Settings → Anomalous Model Browser → Interface** changes the shortcut, the language and how the browser is opened (floating button, a button next to Run, or the Extensions menu only).
|
||||
|
||||
The first time you open it, start on **Home**: **Scan model folders** fills in covers and trigger words, and **Tour the interface** points at each part of the window.
|
||||
**Updating**: click the **!** button at the top right of the browser; it shows the current version, and **Check for updates / switch version** opens the version panel. Update to the newest release, go back to an earlier one or return to the latest, then restart ComfyUI. The plugin never checks for updates on its own.
|
||||
|
||||
> [!NOTE]
|
||||
> **Network:** Scan looks models up on Civitai and prompt translation uses Google Translate, so both need to reach those sites (from mainland China usually through a proxy). Without them, pick **Offline scan**: base models are read from the files, without covers or trigger words, and a later online scan fills in the rest. Browsing models, outputs, workflows and combos needs no network.
|
||||
</details>
|
||||
|
||||
> [!WARNING]
|
||||
> **Keep a copy of your own data:** Workflow Recipes, combos, saved prompts and node values, and parameter presets live in `workflows/anomalous_recipes`, `workflows/anomalous_notebooks`, `workflows/anomalous_materials` and `workflows/anomalous_parameters` in your ComfyUI user folder. A copy of them before updating the plugin is a good habit.
|
||||
<details>
|
||||
<summary><b>🌐 Network and backups</b></summary>
|
||||
|
||||
**Network:** Scan looks models up on Civitai and prompt translation uses Google Translate, so both need to reach those sites (from mainland China usually through a proxy). Without them, pick **Offline scan**: base models are read from the files, without covers or trigger words, and a later online scan fills in the rest. Downloading missing models needs the network too; for Hugging Face, **Settings → Model downloads** can use the hf-mirror.com mirror (Civitai has none). Browsing models, outputs, workflows and combos needs no network.
|
||||
|
||||
**Keep a copy of your own data:** **Settings → Backup → Export** packs Workflow Recipes, combos, saved prompts and node values, parameter presets, and the names, notes and covers you gave your models into one .zip; keep it on a cloud drive or a USB stick, and **Import** puts it back, also on another computer. (They live in `workflows/anomalous_recipes`, `workflows/anomalous_notebooks`, `workflows/anomalous_materials` and `workflows/anomalous_parameters` in your ComfyUI user folder.)
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
@@ -124,41 +137,59 @@ The first time you open it, start on **Home**: **Scan model folders** fills in c
|
||||
>
|
||||
> 插件改动过什么，都能在“操作记录”里查到。
|
||||
|
||||
### 🎬 视频演示与教程
|
||||
* 📺 **哔哩哔哩 (Bilibili)**：[在 B 站观看快速上手与使用演示](https://www.bilibili.com/video/BV1a1bv68EuA/)
|
||||
* 📺 **YouTube**：[在 YouTube 观看视频演示](https://youtu.be/hAvsj7uiaCw)
|
||||
### 🎬 视频与讨论
|
||||
* 📺 **哔哩哔哩 (Bilibili)**：[功能演示](https://www.bilibili.com/video/BV1boHx61Enc/) · [角色配音演示](https://www.bilibili.com/video/BV1bkpZ6uEZQ/)
|
||||
* 💬 **Reddit**：[r/comfyui 上的介绍帖](https://www.reddit.com/r/comfyui/s/n38ihf3DLf)
|
||||
|
||||
### 👀 看图上手
|
||||
### 💡 它能帮你解决什么
|
||||
|
||||
**1. 扫描一次，以后看封面挑模型。** 点 **扫描**，每个模型都按文件哈希去 C 站认一遍。就算文件叫 `下载 (3).safetensors`，也能显示出它真正的名字、封面、触发词和底模。
|
||||
**1. 模型太多，认不出谁是谁。** 文件叫 `下载 (3).safetensors`，根本看不出是什么。点一次 **扫描**，每个模型都按文件哈希去 C 站认一遍，显示出它真正的名字、封面、触发词和底模。以后看封面挑模型就行。
|
||||
|
||||
<p align="center"><img src="docs/images/scan.webp" width="520" alt="扫描结果：每个文件都在 C 站认出来了"></p>
|
||||
|
||||
**2. 打开别人的工作流，不用自己找模型。** 缺模型时画布上方会提示。你电脑里其实有、只是改了名或换了文件夹的，点一下就换上，红框随之消失。
|
||||
**2. 打开别人的工作流，一片红框。** 模型检查会告诉你缺哪些模型。你电脑里其实有、只是改了名或换了文件夹的，点一下就换上；真没有的可以从 C 站或 Hugging Face 下载，下载完按模型指纹核对过才用。
|
||||
|
||||
<p align="center"><img src="docs/images/model-check.webp" width="760" alt="模型检查一键换上 3 个缺失的模型"></p>
|
||||
|
||||
**3. 找到以前的图，拿回当时的工作流。** 在图库里按提示词、模型或 LoRA 搜，再把图拖到画布上。
|
||||
**3. 这张图当时是怎么出的？** 在图库里按提示词、模型或 LoRA 搜，再把图拖到画布上，当时的工作流就回来了。
|
||||
|
||||
<p align="center"><img src="docs/images/gallery.webp" width="760" alt="图库搜索后把图拖到画布"></p>
|
||||
|
||||
**4. 存下“模型 + LoRA + 提示词”的搭配，一拖就用。** 把搭配卡片拖到画布空白处，就放出一组连好线的节点，画布上原有的东西不动。
|
||||
**4. 每次都要重新配一遍模型、LoRA 和提示词。** 存成一个搭配。把卡片拖到画布空白处，就放出一组连好线的节点，画布上原有的东西不动。
|
||||
|
||||
<p align="center"><img src="docs/images/combo.webp" width="760" alt="搭配卡片拖到画布变成连好线的节点"></p>
|
||||
|
||||
**5. 像搭积木一样改提示词。** 选中提示词节点，提示词工坊把它拆成词块：点一下调权重，拖动换位置，输入就加（中文会自动译成英文）。
|
||||
**5. 提示词太长，改起来麻烦。** 选中提示词节点，提示词工坊把它拆成词块：点一下调权重，拖动换位置，输入就加（中文会自动译成英文）。
|
||||
|
||||
<p align="center"><img src="docs/images/prompt-studio.webp" width="760" alt="提示词工坊按词块编辑提示词节点"></p>
|
||||
|
||||
**6. 插件改过什么都查得到，还能撤销。** 插件做的每一处改动都列在“操作记录”里，带 **撤销** 和 **在画布上找到**。
|
||||
|
||||
<p align="center"><img src="docs/images/activity.webp" width="560" alt="操作记录和撤销按钮"></p>
|
||||
|
||||
**7. 给角色配音**（需要 [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS)）：导入 GPT-SoVITS 角色，每种情绪一段参考音频，在配音页写台词直接生成。
|
||||
**6. 想让角色开口说话。** 配合配套节点包 [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS)：导入 GPT-SoVITS 角色，每种情绪一段参考音频，在配音页写台词直接生成。
|
||||
|
||||
<p align="center"><img src="docs/images/voices.webp" width="760" alt="角色语音页面"></p>
|
||||
|
||||
### 🌟 能做什么
|
||||
**改错了也不怕。** 插件做的每一处改动都列在“操作记录”里，带 **撤销** 和 **在画布上找到**。
|
||||
|
||||
<p align="center"><img src="docs/images/activity.webp" width="560" alt="操作记录和撤销按钮"></p>
|
||||
|
||||
**不占你的资源**：
|
||||
- 插件本身不加载任何模型，不和生图抢显存。
|
||||
- 浏览时显示的是小缩略图，不加载原图；关闭窗口一分半后，图片也会释放掉。
|
||||
- Anomalous_TTS 的语音模型在 ComfyUI 生图需要显存时会自动让出，点 **卸载模型** 时全部释放。
|
||||
|
||||
### 📦 安装
|
||||
|
||||
1. 在 **ComfyUI Manager** 里搜索 `Anomalous Model Browser`，安装后重启 ComfyUI。也可以在 `custom_nodes` 目录下手动安装：
|
||||
```bash
|
||||
cd custom_nodes
|
||||
git clone https://github.com/DemonGatanjieu/Anomalous_Model_Browser.git
|
||||
```
|
||||
2. 按 **Shift + B** 打开。在 **主页** 点 **扫描模型文件夹** 补全封面和触发词；点 **界面导览** 会逐一指给你看窗口的各个部分。
|
||||
3. 想用角色配音的话，再装 [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS)（在 ComfyUI Manager 搜索 `Anomalous TTS`）。不装也不影响其他功能。
|
||||
|
||||
扫描要连 C 站。连不上也能用：选离线扫描，只从模型文件推断底模，以后能连上时再扫一次会自动补上（见下面的“网络和备份”）。
|
||||
|
||||
<details>
|
||||
<summary><b>📖 每个页面和工具的详细说明</b></summary>
|
||||
|
||||
窗口左侧的图标栏，每个页面一个图标。
|
||||
|
||||
@@ -168,9 +199,9 @@ The first time you open it, start on **Home**: **Scan model folders** fills in c
|
||||
| **🕘 记录** | 按天列出插件做过的每一处改动。<br>画布上：哪个节点的设置从什么改成了什么、加了或删了哪些节点、打开了哪个工作流。<br>文件上：模型、封面、配方、素材、笔记、图片、音频。<br>**在画布上找到** 可以直接跳到那个节点；画布上的改动在没被再改过之前可以点 **撤销**，其余只记录。你自己在画布上的修改不会记进来。 |
|
||||
| **📦 模型** | 模型带封面、触发词和底模信息。<br>顶部标签切换类型（Checkpoint、LoRA、VAE…），连同子文件夹一起列出；左侧文件夹列表可以只看某个文件夹。<br>**+** 一键在画布上创建加载节点；也可以改名、写备注、换封面，或只扫描这一个模型。 |
|
||||
| **🖼️ 图库** | 读取 ComfyUI 的 `output` 文件夹。<br>可以按提示词、模型、LoRA、seed、文件名或模型哈希搜索。<br>点开图片能看到生成参数；拖到画布上可以还原当时的工作流。<br>觉得好就点 **☆**：存成整个工作流、搭配（模型 + LoRA + 提示词），或者只存提示词。 |
|
||||
| **🪡 工作流** | 工作流配方：把完整工作流或其中一段，连同模型、封面、备注和参数一起存下来。<br>卡片会显示这台电脑上模型是否齐全；拖到画布上即可载入。<br>可以比较、恢复历史版本。 |
|
||||
| **🪡 工作流** | 工作流配方：把完整工作流或其中一段，连同模型、封面、备注和参数一起存下来。<br>卡片会显示这台电脑上模型是否齐全；拖到画布上即可载入。<br>可以比较、恢复历史版本。<br>卡片上的导出按钮能打成**配方包**（压缩包）发给别人，对方拖进 ⇅ 就多一张配方卡片。 |
|
||||
| **🧩 搭配** | 把常用的主模型、几个 LoRA 和一段提示词存成一个搭配。<br>按 **放到画布**（或把卡片拖到画布上），就放出一组连好线的节点；画布上原有的东西不动。 |
|
||||
| **✍️ 提示词** | 在画布旁边打开提示词工坊。选中一个提示词节点，它的提示词框就以词块的样子出现在工坊里，直接改：点一下调权重，双击改字，拖动换位置，输入就加（中文、日文等会自动译成英文），还能在每个词下面显示它的中文、日文、韩文等语言的意思。<br>点词卡（几张常用的，加上你存的）加进去，或者直接拖到画布的提示词框里（正面绿框、负面红框），拖到空白处就新建一个提示词节点。没选中提示词节点时写的是草稿。 |
|
||||
| **✍️ 提示词** | 提示词工坊，浏览器里的一个页面；点“停到画布旁”可以移到画布旁边，边看画布边改。选中一个提示词节点，它的提示词框就以词块的样子出现在工坊里，直接改：点一下调权重，双击改字，拖动换位置，输入就加（中文、日文等会自动译成英文），还能在每个词下面显示它的中文、日文、韩文等语言的意思。<br>点词卡（几张常用的，加上你存的）加进去，或者直接拖到画布的提示词框里（正面绿框、负面红框），拖到空白处就新建一个提示词节点。没选中提示词节点时写的是草稿。 |
|
||||
| **🎙️ 角色语音 / 🎧 音频库** | 装了 [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS) 后可用：<br>导入 GPT-SoVITS 角色，给每种情绪选参考音频，修正读音；切到“配音”写台词并直接生成。<br>生成的音频都在音频库里。没装时，页面会说明怎么安装，其他功能不受影响。 |
|
||||
|
||||
图标栏上的工具：
|
||||
@@ -178,40 +209,41 @@ The first time you open it, start on **Home**: **Scan model folders** fills in c
|
||||
* **扫描**：读取模型文件夹，按文件哈希从 C 站获取封面、触发词和底模。不需要额外安装 Python 包。
|
||||
* **模型检查**（原“模型医生”）：打开工作流时自动检查，缺模型就在画布上方提示；本地改过名、换过文件夹的同一个文件（按哈希和文件大小认）一键换上，其余的在检查页里自己挑或去 Civitai 找。
|
||||
* **当前节点**（原“节点助手”）：选中画布上的节点，查看它用的模型。可以看图换模型、插入 LoRA；存过的同类节点参数列在一起，每条写明会改哪几项，一键套用；存下的参数也在这里删除。
|
||||
* **工具回到各自的页面**：工作流分享码（导入导出）在“工作流”页，模型来源（每个模型去哪下载）在“模型”页和模型检查里，翻译在“当前节点”的提示词框上。
|
||||
* **工具回到各自的页面**：分享与导入在“工作流”页（⇅：把画布上的工作流变成一段短短的分享码，有汉字码和字母码；也能粘贴分享码，或拖入工作流文件、图片、备份压缩包），模型来源（每个模型去哪下载）在“模型”页和模型检查里，翻译在“当前节点”的提示词框上。
|
||||
* **设置**（最下面）：语言、字号、缩略图、视频封面、显示哪些模型文件夹，以及提交反馈。主页和设置里的 **报告问题** 会打开一个已经填好版本和显卡信息的 GitHub 页面。
|
||||
|
||||
**不占你的资源**：
|
||||
- 插件本身不加载任何模型，不和生图抢显存。
|
||||
- 浏览时显示的是小缩略图，不加载原图；关闭窗口一分半后，图片也会释放掉。
|
||||
- Anomalous_TTS 的语音模型在 ComfyUI 生图需要显存时会自动让出，点 **卸载模型** 时全部释放。
|
||||
**接入 AI 软件（MCP）**：Claude、Cursor、Cherry Studio 等支持 MCP 的 AI 软件，可以通过 `http://127.0.0.1:8188/anomalous/mcp` 查你的模型、出图和存下的提示词，回答相关问题；也能帮你改当前工作流（提示词、模型、LoRA、搭配、缺失的模型），每次改动都能用 Ctrl+Z 撤销。只限本机。设置方法见 [docs/guides/mcp.md](docs/guides/mcp.md)。
|
||||
|
||||
### 📦 快速安装
|
||||
</details>
|
||||
|
||||
1. 在 ComfyUI 的 `custom_nodes` 目录下打开终端执行：
|
||||
```bash
|
||||
cd custom_nodes
|
||||
git clone https://github.com/DemonGatanjieu/Anomalous_Model_Browser.git
|
||||
```
|
||||
2. 重启 ComfyUI 即可使用。（*也可以直接在 **ComfyUI Manager** 搜索 `Anomalous Model Browser` 安装*）
|
||||
3. 想用角色配音的话，用同样的方法再装 [Anomalous_TTS](https://github.com/DemonGatanjieu/Anomalous_TTS)。不装也不影响其他功能。
|
||||
4. **更新插件**：
|
||||
- 点窗口右上角的 **!**，可以看到当前版本；
|
||||
- 点 **检查更新 / 切换版本** 打开版本面板，可以更新到最新版、回退到以前的版本，或回到最新版；
|
||||
- 换完版本后重启 ComfyUI。插件不会自己联网检查更新。
|
||||
<details>
|
||||
<summary><b>⚙️ 打开方式、快捷键和更新</b></summary>
|
||||
|
||||
打开方式有三种：按 **Shift + B**（再按一次关闭）、点击画布上的悬浮按钮，或者从 ComfyUI 菜单 **扩展 → Anomalous Model Browser** 打开。在 **ComfyUI 设置 → Anomalous Model Browser → 界面** 里，可以修改快捷键、插件语言和打开方式（悬浮按钮、运行按钮旁的按钮，或只用扩展菜单）。
|
||||
|
||||
第一次打开时会停在 **主页**：点 **扫描模型文件夹** 补全封面和触发词；点 **界面导览** 会逐一指给你看窗口的各个部分。
|
||||
**更新插件**：
|
||||
- 点窗口右上角的 **!**，可以看到当前版本；
|
||||
- 点 **检查更新 / 切换版本** 打开版本面板，可以更新到最新版、回退到以前的版本，或回到最新版；
|
||||
- 换完版本后重启 ComfyUI。插件不会自己联网检查更新。
|
||||
|
||||
> [!NOTE]
|
||||
> **关于网络：** 扫描模型要连 Civitai（C 站）查封面和触发词，提示词翻译默认用谷歌翻译，这两样在国内一般要开网络代理才能用。不开代理也行：扫描时选 **离线扫描**，只从模型文件推断底模，没有封面和触发词，以后能连上时再扫一次会自动补上。浏览模型、图库、工作流和搭配都不用联网。
|
||||
</details>
|
||||
|
||||
> [!WARNING]
|
||||
> **备份自己的数据：** 工作流配方、搭配、存下的提示词和节点参数、参数预设存在 ComfyUI 用户目录下的 `workflows/anomalous_recipes`、`workflows/anomalous_notebooks`、`workflows/anomalous_materials` 与 `workflows/anomalous_parameters` 文件夹。更新插件前顺手备份一份，更安心。
|
||||
<details>
|
||||
<summary><b>🌐 网络和备份</b></summary>
|
||||
|
||||
**关于网络：** 扫描模型要连 Civitai（C 站）查封面和触发词，提示词翻译默认用谷歌翻译，这两样在国内一般要开网络代理才能用。不开代理也行：扫描时选 **离线扫描**，只从模型文件推断底模，没有封面和触发词，以后能连上时再扫一次会自动补上。下载缺失的模型也要联网：Hugging Face 的模型可以在 **设置 → 模型下载** 里打开“Hugging Face 用国内镜像”，不用代理；C 站的没有镜像。浏览模型、图库、工作流和搭配都不用联网。
|
||||
|
||||
**备份自己的数据：** **设置 → 备份 → 导出**，把工作流配方、搭配、存下的提示词和节点参数、参数预设，以及你给模型改的名字、备注和封面打成一个压缩包，放到云端硬盘或 U 盘里；换电脑或重装后用 **导入** 恢复。（它们存在 ComfyUI 用户目录下的 `workflows/anomalous_recipes`、`workflows/anomalous_notebooks`、`workflows/anomalous_materials` 与 `workflows/anomalous_parameters`。）
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
### 🧪 Tests (测试)
|
||||
|
||||
The plugin's tests are in `tests/`; run them all with `node tools/run_tests.mjs` from the plugin folder. See [docs/guides/testing.md](docs/guides/testing.md).
|
||||
插件的测试在 `tests/` 里，在插件文件夹运行 `node tools/run_tests.mjs` 一次跑完。改代码或提 PR 之前跑一遍，说明见 [docs/guides/testing.md](docs/guides/testing.md)。
|
||||
|
||||
### 📝 License & Branding (开源与品牌声明)
|
||||
|
||||
* **Code License (代码授权)**: The source code is released under the [MIT License](LICENSE). 本项目源代码基于 MIT 许可证开源，可自由使用、修改与分发。
|
||||
|
||||
+11
-1
@@ -1,7 +1,8 @@
|
||||
from . import (
|
||||
activity_log, audio_catalog, folder_types, gallery_routes, materials, media_routes, model_catalog,
|
||||
model_folder, model_media, model_metadata, model_resolution, model_type_listing, kept_images, node_material, recipe_packages, recipes, scan_report, scan_summary,
|
||||
local_guard, translation_routes, version_manager,
|
||||
local_guard, lora_info, mcp_server, model_download, translation_routes, version_manager, backup,
|
||||
network_check,
|
||||
)
|
||||
from .scanner import *
|
||||
from .config import *
|
||||
@@ -33,6 +34,10 @@ def setup_routes(app):
|
||||
if hasattr(app, 'middlewares') and local_guard.local_only_middleware not in app.middlewares:
|
||||
app.middlewares.append(local_guard.local_only_middleware)
|
||||
app.router.add_get('/anomalous/access', local_guard.api_access)
|
||||
try:
|
||||
folder_types.upgrade_folder_defaults()
|
||||
except Exception as error:
|
||||
print(f"[Anomalous Browser] Could not update the shown model folders: {error}")
|
||||
app.router.add_get('/anomalous/folders', model_catalog.api_get_folders)
|
||||
app.router.add_get('/anomalous/all_folder_types', folder_types.api_get_all_folder_types)
|
||||
app.router.add_get('/anomalous/models', model_catalog.api_get_models)
|
||||
@@ -114,6 +119,7 @@ def setup_routes(app):
|
||||
app.router.add_get('/anomalous/scan_summary', scan_summary.api_scan_summary)
|
||||
app.router.add_get('/anomalous/last_scan', scan_report.api_last_scan)
|
||||
app.router.add_get('/anomalous/scan_model', scan_report.api_scan_model)
|
||||
app.router.add_get('/anomalous/lora_info', lora_info.api_lora_info)
|
||||
|
||||
# Audio & Voice Studio Routes
|
||||
app.router.add_get('/anomalous/audio_stream', audio_catalog.api_serve_audio)
|
||||
@@ -123,3 +129,7 @@ def setup_routes(app):
|
||||
app.router.add_post('/anomalous/save_audio_preview', audio_catalog.api_save_audio_preview)
|
||||
version_manager.register_routes(app)
|
||||
activity_log.register_routes(app)
|
||||
mcp_server.register_routes(app)
|
||||
model_download.register_routes(app)
|
||||
network_check.register_routes(app)
|
||||
backup.register_routes(app)
|
||||
|
||||
+4
-1
@@ -248,7 +248,8 @@ async def api_get_activity(request):
|
||||
|
||||
|
||||
async def api_post_activity(request):
|
||||
"""Canvas changes from the browser: {changes: [...], total, workflow}."""
|
||||
"""Canvas changes from the browser: {changes: [...], total, workflow, via?}; via "mcp" marks
|
||||
changes an AI app made through the MCP endpoint."""
|
||||
try:
|
||||
body = await request.json()
|
||||
except ValueError:
|
||||
@@ -260,6 +261,8 @@ async def api_post_activity(request):
|
||||
return web.json_response({'error': 'No recognizable changes'}, status=400)
|
||||
total = body.get('total') if isinstance(body.get('total'), int) else len(changes)
|
||||
detail = {'changes': changes, 'total': max(total, len(changes))}
|
||||
if body.get('via') == 'mcp':
|
||||
detail['via'] = 'mcp'
|
||||
entry = await asyncio.get_running_loop().run_in_executor(
|
||||
None, add_entry, 'canvas', 'canvas', body.get('workflow') if isinstance(body.get('workflow'), str) else '', detail)
|
||||
return web.json_response({'entry': entry})
|
||||
|
||||
+598
@@ -0,0 +1,598 @@
|
||||
"""Backups of the user's own data (Settings -> Backup): one .zip to keep anywhere (a cloud
|
||||
drive, a USB stick), and putting it back on this or another computer.
|
||||
|
||||
A backup holds the plugin's stores (Workflow Recipes, combos, materials and saved prompts,
|
||||
parameter sets), optionally ComfyUI's own saved workflows, the plugin settings without the
|
||||
Civitai key, what the user set on models: <model>.anomalous.json (name, notes, link) and
|
||||
the user's cover (any cover that is not a copy of Civitai's <model>.civitai_bak.*), what a
|
||||
scan found (<model>.info; Civitai's cover only when asked, it can be hundreds of MB), and the
|
||||
model list (each model's folder, path, size, SHA-256 and link) so the models a new computer
|
||||
lacks can be downloaded again. Models are found again by SHA-256, else by models folder, path
|
||||
and size, so they match after a move.
|
||||
|
||||
Putting back never erases: a file missing here is added; a file that differs stays unless
|
||||
"replace" is chosen, and then the one here goes to the Recycle Bin first. A model's user file
|
||||
is merged field by field (empty fields are filled; "replace" takes the backup's values), and
|
||||
the backup's cover takes the place of a Civitai cover (whose copy stays as civitai_bak).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
import time
|
||||
import uuid
|
||||
import zipfile
|
||||
|
||||
import folder_paths
|
||||
from aiohttp import web
|
||||
|
||||
from .activity_log import add_entry
|
||||
from .folder_types import config_path
|
||||
from .material_store import get_materials_dir
|
||||
from .metadata import USER_FIELDS, clear_metadata_cache, get_metadata
|
||||
from .model_constants import CIVITAI_BACKUP_SUFFIXES, MEDIA_EXTENSIONS, PREVIEW_SUFFIXES, RESOLVABLE_MODEL_TYPES, is_model_file
|
||||
from .model_download import settings_path as download_settings_path
|
||||
from .notebooks import get_notebooks_dir
|
||||
from .parameters import _invalidate_parameter_notebooks_cache, get_parameters_dir
|
||||
from .path_utils import atomic_write_json, require_filename, resolve_within
|
||||
from .recipe_store import get_recipes_dir
|
||||
from .trash import TrashUnavailable, move_to_trash, trash_failure
|
||||
from . import version_manager
|
||||
|
||||
try:
|
||||
from ..model_identity import USER_INFO_SUFFIX, read_json, scan_info_path
|
||||
except ImportError:
|
||||
from model_identity import USER_INFO_SUFFIX, read_json, scan_info_path
|
||||
|
||||
FORMAT = 1
|
||||
APP = "Anomalous Model Browser"
|
||||
ACTIVE_COVER_SUFFIXES = PREVIEW_SUFFIXES + MEDIA_EXTENSIONS # as scraper.py: <model>.preview.png or <model>.png
|
||||
SECRET_KEYS = {"CIVITAI_API_KEY", "DEEPL_API_KEY"} # never leave this computer
|
||||
SCAN_INFO_SUFFIXES = (".info", ".civitai.info")
|
||||
SKIP_FILES = {".legacy_imported.json"} # this computer's own migration marker
|
||||
MAX_BACKUP_BYTES = 4 * 1024 ** 3 # unpacked size of a backup that is still read
|
||||
MAX_MEMBERS = 200_000
|
||||
KEEP_EXPORTS = 3 # finished exports kept in the temp folder for downloading
|
||||
CHUNK = 1024 * 1024
|
||||
|
||||
PART_ORDER = ("recipes", "combos", "materials", "parameters", "comfy_workflows")
|
||||
|
||||
|
||||
def _comfy_workflows_dir():
|
||||
return os.path.join(folder_paths.get_user_directory(), "default", "workflows")
|
||||
|
||||
|
||||
def part_dirs():
|
||||
"""Each kept store and where it lives on this computer."""
|
||||
return {
|
||||
"recipes": get_recipes_dir(),
|
||||
"combos": get_notebooks_dir(),
|
||||
"materials": get_materials_dir(),
|
||||
"parameters": get_parameters_dir(),
|
||||
"comfy_workflows": _comfy_workflows_dir(),
|
||||
}
|
||||
|
||||
|
||||
def work_dir():
|
||||
path = os.path.join(folder_paths.get_temp_directory(), "anomalous_backups")
|
||||
os.makedirs(path, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def _sha256_file(path):
|
||||
digest = hashlib.sha256()
|
||||
with open(path, "rb") as handle:
|
||||
for block in iter(lambda: handle.read(CHUNK), b""):
|
||||
digest.update(block)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _same_bytes(a, b):
|
||||
try:
|
||||
return os.path.getsize(a) == os.path.getsize(b) and _sha256_file(a) == _sha256_file(b)
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _store_files(directory):
|
||||
"""(path, '/'-relative path) of every file kept in a store."""
|
||||
if not os.path.isdir(directory):
|
||||
return
|
||||
for root, dirs, files in os.walk(directory):
|
||||
dirs[:] = sorted(d for d in dirs if d != "__pycache__")
|
||||
for name in sorted(files):
|
||||
if name in SKIP_FILES or name.endswith(".tmp"):
|
||||
continue
|
||||
path = os.path.join(root, name)
|
||||
yield path, os.path.relpath(path, directory).replace(os.sep, "/")
|
||||
|
||||
|
||||
def _model_roots():
|
||||
"""(models folder type, folder) once per folder: clip and text_encoders often share one."""
|
||||
seen = set()
|
||||
for folder_type in RESOLVABLE_MODEL_TYPES:
|
||||
try:
|
||||
roots = folder_paths.get_folder_paths(folder_type) or []
|
||||
except Exception: # noqa: BLE001 - a type this ComfyUI does not know
|
||||
continue
|
||||
for root in roots:
|
||||
key = os.path.normcase(os.path.abspath(root))
|
||||
if key in seen or not os.path.isdir(root):
|
||||
continue
|
||||
seen.add(key)
|
||||
yield folder_type, root
|
||||
|
||||
|
||||
def _models():
|
||||
"""(type, root, path, '/'-relative path) of every model file."""
|
||||
for folder_type, root in _model_roots():
|
||||
for walk_root, dirs, files in os.walk(root):
|
||||
dirs.sort()
|
||||
for name in sorted(files):
|
||||
if is_model_file(name):
|
||||
path = os.path.join(walk_root, name)
|
||||
yield folder_type, root, path, os.path.relpath(path, root).replace(os.sep, "/")
|
||||
|
||||
|
||||
def cover_owners(base):
|
||||
"""(Civitai's, the user's) covers of a model: Civitai's are copies of <model>.civitai_bak.*."""
|
||||
backups = [base + suffix for suffix in CIVITAI_BACKUP_SUFFIXES if os.path.isfile(base + suffix)]
|
||||
covers = [base + suffix for suffix in ACTIVE_COVER_SUFFIXES if os.path.isfile(base + suffix)]
|
||||
civitai = [cover for cover in covers if any(_same_bytes(cover, backup) for backup in backups)]
|
||||
return civitai, [cover for cover in covers if cover not in civitai]
|
||||
|
||||
|
||||
def _model_facts(path):
|
||||
"""(SHA-256, download page) from the model's sidecars; nothing is computed."""
|
||||
try:
|
||||
meta = get_metadata(path) or {}
|
||||
except Exception: # noqa: BLE001 - an unreadable sidecar only means "match by path"
|
||||
return "", ""
|
||||
return str(meta.get("hash") or "").lower(), str(meta.get("source_url") or meta.get("civitai_url") or "")
|
||||
|
||||
|
||||
def _model_hash(path):
|
||||
return _model_facts(path)[0]
|
||||
|
||||
|
||||
def _settings_for_backup():
|
||||
data = read_json(config_path()) or {}
|
||||
return {key: value for key, value in data.items() if key not in SECRET_KEYS}
|
||||
|
||||
|
||||
def _version():
|
||||
try:
|
||||
return version_manager.current_state().get("label") or ""
|
||||
except Exception: # noqa: BLE001 - a copy that is no git checkout
|
||||
return ""
|
||||
|
||||
|
||||
# ---------- export ----------
|
||||
|
||||
def build_backup(include_models=True, include_comfy=True, include_library=True, include_scan=True, include_civitai_covers=False):
|
||||
"""Writes the backup .zip into the temp folder: (path, manifest)."""
|
||||
stamp = time.strftime("%Y%m%d-%H%M%S")
|
||||
path = os.path.join(work_dir(), f"AMB-backup-{stamp}.zip")
|
||||
manifest = {"format": FORMAT, "app": APP, "version": _version(), "created": time.time(), "parts": {}, "settings": False, "models": []}
|
||||
parts = [part for part in PART_ORDER if include_comfy or part != "comfy_workflows"]
|
||||
dirs = part_dirs()
|
||||
temporary = path + ".tmp"
|
||||
try:
|
||||
with zipfile.ZipFile(temporary, "w", zipfile.ZIP_DEFLATED, allowZip64=True) as archive:
|
||||
for part in parts:
|
||||
count = 0
|
||||
for file_path, rel in _store_files(dirs[part]):
|
||||
archive.write(file_path, f"stores/{part}/{rel}")
|
||||
count += 1
|
||||
manifest["parts"][part] = count
|
||||
archive.writestr("settings/config.json", json.dumps(_settings_for_backup(), ensure_ascii=False))
|
||||
if os.path.isfile(download_settings_path()):
|
||||
archive.write(download_settings_path(), "settings/download_settings.json")
|
||||
manifest["settings"] = True
|
||||
if include_library:
|
||||
manifest["library"] = []
|
||||
if include_models or include_library:
|
||||
for folder_type, _root, model_path, rel in _models():
|
||||
if include_library:
|
||||
digest, page = _model_facts(model_path)
|
||||
manifest["library"].append({"type": folder_type, "rel": rel, "size": os.path.getsize(model_path),
|
||||
"sha256": digest, "url": page})
|
||||
if not include_models:
|
||||
continue
|
||||
base = os.path.splitext(model_path)[0]
|
||||
user_file = base + USER_INFO_SUFFIX
|
||||
user_data = read_json(user_file) if os.path.isfile(user_file) else None
|
||||
civitai, covers = cover_owners(base)
|
||||
scan_info = scan_info_path(base) if include_scan else None
|
||||
backups = [base + suffix for suffix in CIVITAI_BACKUP_SUFFIXES if os.path.isfile(base + suffix)]
|
||||
civitai_cover = civitai[0] if include_civitai_covers and civitai and backups else None
|
||||
if not user_data and not covers and not scan_info and not civitai_cover:
|
||||
continue
|
||||
entry = {"id": len(manifest["models"]), "type": folder_type, "rel": rel,
|
||||
"size": os.path.getsize(model_path), "sha256": _model_hash(model_path), "user": bool(user_data), "cover": ""}
|
||||
folder = f"models/{entry['id']}/"
|
||||
if user_data:
|
||||
archive.writestr(folder + "user.json", json.dumps(user_data, ensure_ascii=False))
|
||||
if covers:
|
||||
entry["cover"] = covers[0][len(base):]
|
||||
archive.write(covers[0], f"{folder}cover{entry['cover']}")
|
||||
if scan_info:
|
||||
entry["info"] = scan_info[len(base):]
|
||||
archive.write(scan_info, f"{folder}scan{entry['info']}")
|
||||
if civitai_cover:
|
||||
# One copy: it is both the cover and Civitai's saved image (<model>.civitai_bak.*).
|
||||
entry["civitai_cover"], entry["civitai_bak"] = civitai_cover[len(base):], backups[0][len(base):]
|
||||
archive.write(backups[0], f"{folder}civitai{entry['civitai_bak']}")
|
||||
manifest["models"].append(entry)
|
||||
archive.writestr("manifest.json", json.dumps(manifest, ensure_ascii=False, indent=1))
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.remove(temporary)
|
||||
_prune_exports()
|
||||
return path, manifest
|
||||
|
||||
|
||||
def _prune_exports():
|
||||
"""Keeps the newest finished exports; older ones are this plugin's own temp files."""
|
||||
exports = sorted((name for name in os.listdir(work_dir()) if name.startswith("AMB-backup-") and name.endswith(".zip")), reverse=True)
|
||||
for name in exports[KEEP_EXPORTS:]:
|
||||
try:
|
||||
os.remove(os.path.join(work_dir(), name))
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# ---------- reading a backup ----------
|
||||
|
||||
class BackupError(ValueError):
|
||||
"""A file that is no backup of this plugin, or one that cannot be read safely."""
|
||||
|
||||
|
||||
def _safe_rel(rel):
|
||||
parts = rel.split("/")
|
||||
if not rel or rel.startswith("/") or "\\" in rel or ":" in rel or any(part in ("", ".", "..") for part in parts):
|
||||
raise BackupError(f"unsafe path in the backup: {rel}")
|
||||
return parts
|
||||
|
||||
|
||||
def open_backup(path):
|
||||
"""The zip and its manifest, after checking it is a backup and safe to unpack."""
|
||||
try:
|
||||
archive = zipfile.ZipFile(path)
|
||||
except (zipfile.BadZipFile, OSError) as error:
|
||||
raise BackupError("not a zip file") from error
|
||||
try:
|
||||
infos = archive.infolist()
|
||||
if len(infos) > MAX_MEMBERS or sum(info.file_size for info in infos) > MAX_BACKUP_BYTES:
|
||||
raise BackupError("the backup is too large")
|
||||
try:
|
||||
manifest = json.loads(archive.read("manifest.json"))
|
||||
except (KeyError, ValueError) as error:
|
||||
raise BackupError("no manifest: not a backup of this plugin") from error
|
||||
if not isinstance(manifest, dict) or manifest.get("app") != APP or manifest.get("format") != FORMAT:
|
||||
raise BackupError("not a backup of this plugin, or one from a newer version")
|
||||
for info in infos:
|
||||
if not info.is_dir():
|
||||
_safe_rel(info.filename)
|
||||
return archive, manifest
|
||||
except Exception:
|
||||
archive.close()
|
||||
raise
|
||||
|
||||
|
||||
def _store_members(archive):
|
||||
"""(part, '/'-relative path, ZipInfo) of each kept store file in the backup."""
|
||||
for info in archive.infolist():
|
||||
name = info.filename
|
||||
if info.is_dir() or not name.startswith("stores/"):
|
||||
continue
|
||||
bits = name.split("/", 2)
|
||||
if len(bits) == 3 and bits[1] in PART_ORDER:
|
||||
_, part, rel = bits
|
||||
_safe_rel(rel)
|
||||
yield part, rel, info
|
||||
|
||||
|
||||
def _member_sha256(archive, info):
|
||||
digest = hashlib.sha256()
|
||||
with archive.open(info) as handle:
|
||||
for block in iter(lambda: handle.read(CHUNK), b""):
|
||||
digest.update(block)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _file_state(archive, info, target):
|
||||
if not os.path.isfile(target):
|
||||
return "new"
|
||||
if os.path.getsize(target) == info.file_size and _sha256_file(target) == _member_sha256(archive, info):
|
||||
return "same"
|
||||
return "differs"
|
||||
|
||||
|
||||
def _local_models():
|
||||
"""This computer's models: by SHA-256 and by (type, path) -> (path, size)."""
|
||||
by_hash, by_rel = {}, {}
|
||||
for folder_type, _root, path, rel in _models():
|
||||
size = os.path.getsize(path)
|
||||
digest = _model_hash(path)
|
||||
if digest:
|
||||
by_hash.setdefault(digest, path)
|
||||
by_rel[(folder_type, rel.lower())] = (path, size)
|
||||
return by_hash, by_rel
|
||||
|
||||
|
||||
def _match(entry, by_hash, by_rel):
|
||||
digest = str(entry.get("sha256") or "").lower()
|
||||
if digest and digest in by_hash:
|
||||
return by_hash[digest]
|
||||
found = by_rel.get((entry.get("type"), str(entry.get("rel") or "").lower()))
|
||||
return found[0] if found and found[1] == entry.get("size") else None
|
||||
|
||||
|
||||
def inspect_backup(path):
|
||||
"""What putting the backup back would do, before anything is written."""
|
||||
archive, manifest = open_backup(path)
|
||||
with archive:
|
||||
dirs = part_dirs()
|
||||
parts = {}
|
||||
for part, rel, info in _store_members(archive):
|
||||
counts = parts.setdefault(part, {"new": 0, "same": 0, "differs": 0})
|
||||
counts[_file_state(archive, info, resolve_within(dirs[part], *rel.split("/")))] += 1
|
||||
models = {"total": len(manifest.get("models") or []), "matched": 0, "missing": []}
|
||||
library = manifest.get("library") if isinstance(manifest.get("library"), list) else None
|
||||
summary = {"created": manifest.get("created"), "version": manifest.get("version") or "",
|
||||
"parts": {part: parts[part] for part in PART_ORDER if part in parts},
|
||||
"settings": bool(manifest.get("settings")), "models": models}
|
||||
if models["total"] or library:
|
||||
by_hash, by_rel = _local_models()
|
||||
for entry in manifest.get("models") or []:
|
||||
if _match(entry, by_hash, by_rel):
|
||||
models["matched"] += 1
|
||||
else:
|
||||
models["missing"].append(entry.get("rel") or "")
|
||||
if library is not None:
|
||||
# The models the old computer had and this one lacks: what can be downloaded again
|
||||
# (two copies of one file there are one download here).
|
||||
missing, seen = [], set()
|
||||
for entry in library:
|
||||
if not isinstance(entry, dict) or _match(entry, by_hash, by_rel):
|
||||
continue
|
||||
digest = str(entry.get("sha256") or "").lower()
|
||||
if digest and digest in seen:
|
||||
continue
|
||||
seen.add(digest)
|
||||
missing.append({key: entry.get(key) for key in ("type", "rel", "size", "sha256", "url")})
|
||||
summary["library"] = {"total": len(library), "missing": missing}
|
||||
return summary
|
||||
|
||||
|
||||
# ---------- putting back ----------
|
||||
|
||||
def _write_member(archive, info, target):
|
||||
os.makedirs(os.path.dirname(target), exist_ok=True)
|
||||
fd, temporary = tempfile.mkstemp(prefix=".restore-", suffix=".tmp", dir=os.path.dirname(target))
|
||||
try:
|
||||
with os.fdopen(fd, "wb") as output, archive.open(info) as source:
|
||||
shutil.copyfileobj(source, output, CHUNK)
|
||||
os.replace(temporary, target)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.remove(temporary)
|
||||
|
||||
|
||||
def _restore_stores(archive, wanted, replace, result):
|
||||
dirs = part_dirs()
|
||||
for part, rel, info in _store_members(archive):
|
||||
if part not in wanted:
|
||||
continue
|
||||
target = resolve_within(dirs[part], *rel.split("/"))
|
||||
state = _file_state(archive, info, target)
|
||||
if state == "same" or (state == "differs" and not replace):
|
||||
result["skipped"] += 1
|
||||
continue
|
||||
if state == "differs":
|
||||
try:
|
||||
move_to_trash(target)
|
||||
except (TrashUnavailable, OSError) as error:
|
||||
result["failed"].append(f"{part}/{rel}: {trash_failure(error)}")
|
||||
continue
|
||||
result["replaced"] += 1
|
||||
else:
|
||||
result["added"] += 1
|
||||
_write_member(archive, info, target)
|
||||
|
||||
|
||||
def _restore_scan(archive, entry, base, folder, replace, result):
|
||||
"""What a scan found: the scan file where there is none (or replaced on request), and
|
||||
Civitai's cover where the model has no cover at all."""
|
||||
suffix = str(entry.get("info") or "")
|
||||
if suffix:
|
||||
if suffix not in SCAN_INFO_SUFFIXES:
|
||||
raise BackupError(f"unknown scan file name: {suffix}")
|
||||
info = archive.getinfo(folder + "scan" + suffix)
|
||||
local = scan_info_path(base)
|
||||
if not local or (replace and _sha256_file(local) != _member_sha256(archive, info)):
|
||||
if local:
|
||||
move_to_trash(local)
|
||||
_write_member(archive, info, base + suffix)
|
||||
result["model_infos"] += 1
|
||||
cover, saved = str(entry.get("civitai_cover") or ""), str(entry.get("civitai_bak") or "")
|
||||
if cover and saved:
|
||||
if cover not in ACTIVE_COVER_SUFFIXES or saved not in CIVITAI_BACKUP_SUFFIXES:
|
||||
raise BackupError(f"unknown cover name: {cover}")
|
||||
info = archive.getinfo(folder + "civitai" + saved)
|
||||
civitai, mine = cover_owners(base)
|
||||
has_saved = any(os.path.isfile(base + suffix) for suffix in CIVITAI_BACKUP_SUFFIXES)
|
||||
if not has_saved:
|
||||
_write_member(archive, info, base + saved)
|
||||
if not civitai and not mine and not entry.get("cover"):
|
||||
_write_member(archive, info, base + cover)
|
||||
result["model_civitai_covers"] += 1
|
||||
|
||||
|
||||
def _restore_model(archive, entry, model_path, replace, result):
|
||||
base = os.path.splitext(model_path)[0]
|
||||
folder = f"models/{int(entry['id'])}/"
|
||||
_restore_scan(archive, entry, base, folder, replace, result)
|
||||
if entry.get("user"):
|
||||
backup = json.loads(archive.read(folder + "user.json"))
|
||||
user_file = base + USER_INFO_SUFFIX
|
||||
local = read_json(user_file) if os.path.isfile(user_file) else None
|
||||
merged = dict(local or {"format": 1})
|
||||
for key in USER_FIELDS:
|
||||
value = str(backup.get(key) or "").strip()
|
||||
if value and (replace or not str(merged.get(key) or "").strip()):
|
||||
merged[key] = backup[key]
|
||||
if merged != (local or {"format": 1}):
|
||||
if local and replace:
|
||||
move_to_trash(user_file) # the old values stay recoverable
|
||||
atomic_write_json(user_file, merged)
|
||||
result["model_notes"] += 1
|
||||
suffix = str(entry.get("cover") or "")
|
||||
if suffix:
|
||||
if suffix not in ACTIVE_COVER_SUFFIXES:
|
||||
raise BackupError(f"unknown cover name: {suffix}")
|
||||
info = archive.getinfo(folder + "cover" + suffix)
|
||||
civitai, mine = cover_owners(base)
|
||||
member_hash = _member_sha256(archive, info)
|
||||
if any(_sha256_file(cover) == member_hash for cover in mine):
|
||||
return
|
||||
if mine and not replace:
|
||||
result["covers_kept"] += 1
|
||||
return
|
||||
for cover in mine + civitai: # Civitai's own copy stays as <model>.civitai_bak.*
|
||||
move_to_trash(cover)
|
||||
_write_member(archive, info, base + suffix)
|
||||
result["model_covers"] += 1
|
||||
|
||||
|
||||
def _restore_settings(archive, result):
|
||||
names = set(archive.namelist())
|
||||
if "settings/config.json" in names:
|
||||
backup = json.loads(archive.read("settings/config.json"))
|
||||
local = read_json(config_path()) or {}
|
||||
merged = {**{key: value for key, value in backup.items() if key not in SECRET_KEYS},
|
||||
**{key: local[key] for key in SECRET_KEYS if key in local}}
|
||||
atomic_write_json(config_path(), merged)
|
||||
result["settings"] = True
|
||||
if "settings/download_settings.json" in names:
|
||||
os.makedirs(os.path.dirname(download_settings_path()), exist_ok=True)
|
||||
atomic_write_json(download_settings_path(), json.loads(archive.read("settings/download_settings.json")))
|
||||
|
||||
|
||||
def apply_backup(path, parts=PART_ORDER, models=True, settings=False, replace=False):
|
||||
"""Puts the backup back as chosen; returns the counts for the user and the activity log."""
|
||||
archive, manifest = open_backup(path)
|
||||
result = {"added": 0, "replaced": 0, "skipped": 0, "model_notes": 0, "model_covers": 0, "model_infos": 0,
|
||||
"model_civitai_covers": 0, "covers_kept": 0, "models_missing": 0, "settings": False, "failed": []}
|
||||
with archive:
|
||||
_restore_stores(archive, set(parts) & set(PART_ORDER), replace, result)
|
||||
if models and manifest.get("models"):
|
||||
by_hash, by_rel = _local_models()
|
||||
for entry in manifest["models"]:
|
||||
model_path = _match(entry, by_hash, by_rel)
|
||||
if not model_path:
|
||||
result["models_missing"] += 1
|
||||
continue
|
||||
try:
|
||||
_restore_model(archive, entry, model_path, replace, result)
|
||||
except (TrashUnavailable, OSError, KeyError, ValueError) as error:
|
||||
result["failed"].append(f"{entry.get('rel')}: {trash_failure(error) if isinstance(error, (TrashUnavailable, OSError)) else error}")
|
||||
clear_metadata_cache()
|
||||
if settings and manifest.get("settings"):
|
||||
_restore_settings(archive, result)
|
||||
_invalidate_parameter_notebooks_cache()
|
||||
return result
|
||||
|
||||
|
||||
# ---------- routes ----------
|
||||
|
||||
def _error(message, status=400):
|
||||
return web.json_response({"status": "error", "error": str(message)}, status=status)
|
||||
|
||||
|
||||
async def api_export(request):
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception: # noqa: BLE001 - no body: the defaults
|
||||
body = {}
|
||||
path, manifest = await asyncio.to_thread(build_backup, body.get("models", True) is not False,
|
||||
body.get("comfy_workflows", True) is not False, body.get("library", True) is not False,
|
||||
body.get("scan", True) is not False, body.get("civitai_covers") is True)
|
||||
counts = {**manifest["parts"], "models": len(manifest["models"]), "library": len(manifest.get("library") or [])}
|
||||
await asyncio.to_thread(add_entry, "file", "backup_export", os.path.basename(path), {"backup": counts})
|
||||
return web.json_response({"status": "success", "name": os.path.basename(path), "size": os.path.getsize(path), "counts": counts})
|
||||
|
||||
|
||||
async def api_file(request):
|
||||
try:
|
||||
name = require_filename(request.query.get("name", ""))
|
||||
if not name.startswith("AMB-backup-") or not name.endswith(".zip"):
|
||||
raise ValueError("not a backup")
|
||||
path = resolve_within(work_dir(), name)
|
||||
except ValueError as error:
|
||||
return _error(error)
|
||||
if not os.path.isfile(path):
|
||||
return _error("This backup is gone; export it again.", 404)
|
||||
return web.FileResponse(path, headers={"Content-Disposition": f'attachment; filename="{name}"'})
|
||||
|
||||
|
||||
async def api_inspect(request):
|
||||
"""Receives the chosen .zip, keeps it in the temp folder and says what putting it back would do."""
|
||||
try:
|
||||
reader = await request.multipart()
|
||||
field = await reader.next()
|
||||
except (AssertionError, ValueError): # not a file upload at all
|
||||
return _error("no file")
|
||||
if field is None or field.name != "file":
|
||||
return _error("no file")
|
||||
token = uuid.uuid4().hex
|
||||
path = os.path.join(work_dir(), f"incoming-{token}.zip")
|
||||
size = 0
|
||||
with open(path, "wb") as output:
|
||||
while chunk := await field.read_chunk(CHUNK):
|
||||
size += len(chunk)
|
||||
if size > MAX_BACKUP_BYTES:
|
||||
output.close()
|
||||
os.remove(path)
|
||||
return _error("the backup is too large")
|
||||
output.write(chunk)
|
||||
try:
|
||||
summary = await asyncio.to_thread(inspect_backup, path)
|
||||
except BackupError as error:
|
||||
os.remove(path)
|
||||
return _error(error)
|
||||
return web.json_response({"status": "success", "token": token, **summary})
|
||||
|
||||
|
||||
async def api_apply(request):
|
||||
try:
|
||||
body = await request.json()
|
||||
token = str(body.get("token") or "")
|
||||
if not token.isalnum():
|
||||
raise ValueError("bad token")
|
||||
path = resolve_within(work_dir(), f"incoming-{token}.zip")
|
||||
except Exception as error: # noqa: BLE001
|
||||
return _error(error)
|
||||
if not os.path.isfile(path):
|
||||
return _error("This backup is gone; choose the file again.", 404)
|
||||
parts = [part for part in body.get("parts") or [] if part in PART_ORDER]
|
||||
try:
|
||||
result = await asyncio.to_thread(apply_backup, path, parts, body.get("models") is not False,
|
||||
body.get("settings") is True, body.get("replace") is True)
|
||||
except BackupError as error:
|
||||
return _error(error)
|
||||
os.remove(path)
|
||||
await asyncio.to_thread(add_entry, "file", "backup_import", str(body.get("name") or ""), {"backup": {**result, "failed": result["failed"][:20]}})
|
||||
return web.json_response({"status": "success", **result})
|
||||
|
||||
|
||||
def register_routes(app):
|
||||
app.router.add_post("/anomalous/backup/export", api_export)
|
||||
app.router.add_get("/anomalous/backup/file", api_file)
|
||||
app.router.add_post("/anomalous/backup/inspect", api_inspect)
|
||||
app.router.add_post("/anomalous/backup/apply", api_apply)
|
||||
+55
-28
@@ -9,6 +9,11 @@ from aiohttp import web
|
||||
import folder_paths
|
||||
import struct
|
||||
|
||||
try:
|
||||
from ..net_proxy import CONFIG_KEY as PROXY_KEY, configured as configured_proxy, normalize as normalize_proxy
|
||||
except ImportError: # loaded outside the package (tests)
|
||||
from net_proxy import CONFIG_KEY as PROXY_KEY, configured as configured_proxy, normalize as normalize_proxy
|
||||
|
||||
async def api_get_config(request):
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
config_path = os.path.join(script_dir, "config.json")
|
||||
@@ -20,20 +25,27 @@ async def api_get_config(request):
|
||||
cfg = json.load(f)
|
||||
has_key = bool(cfg.get("CIVITAI_API_KEY", "").strip())
|
||||
folder_types = cfg.get("folder_types_config", [])
|
||||
except:
|
||||
pass
|
||||
if not has_key:
|
||||
legacy_config_path = os.path.join(os.path.dirname(script_dir), "config.json")
|
||||
if os.path.exists(legacy_config_path):
|
||||
try:
|
||||
with open(legacy_config_path, 'r', encoding='utf-8') as f:
|
||||
legacy_cfg = json.load(f)
|
||||
has_key = bool(legacy_cfg.get("CIVITAI_API_KEY", "").strip())
|
||||
except Exception:
|
||||
pass
|
||||
return web.json_response({"has_api_key": has_key, "folder_types_config": folder_types})
|
||||
except:
|
||||
pass
|
||||
if not has_key:
|
||||
legacy_config_path = os.path.join(os.path.dirname(script_dir), "config.json")
|
||||
if os.path.exists(legacy_config_path):
|
||||
try:
|
||||
with open(legacy_config_path, 'r', encoding='utf-8') as f:
|
||||
legacy_cfg = json.load(f)
|
||||
has_key = bool(legacy_cfg.get("CIVITAI_API_KEY", "").strip())
|
||||
except Exception:
|
||||
pass
|
||||
deepl_key = ""
|
||||
try:
|
||||
with open(config_path, 'r', encoding='utf-8') as f:
|
||||
deepl_key = json.load(f).get("DEEPL_API_KEY", "")
|
||||
except (OSError, ValueError, AttributeError):
|
||||
pass
|
||||
return web.json_response({"has_api_key": has_key, "has_deepl_key": bool(isinstance(deepl_key, str) and deepl_key.strip()),
|
||||
"proxy": configured_proxy(), "folder_types_config": folder_types})
|
||||
|
||||
async def api_save_config(request):
|
||||
async def api_save_config(request):
|
||||
try:
|
||||
data = await request.json()
|
||||
api_key = data.get("api_key")
|
||||
@@ -50,17 +62,32 @@ async def api_save_config(request):
|
||||
except:
|
||||
pass
|
||||
|
||||
if api_key is not None and not isinstance(api_key, str):
|
||||
return web.json_response({"status": "error", "message": "Invalid API key"}, status=400)
|
||||
if folder_types_config is not None and not isinstance(folder_types_config, list):
|
||||
return web.json_response({"status": "error", "message": "Invalid folder configuration"}, status=400)
|
||||
if "physical_folders_config" in data and not isinstance(data.get("physical_folders_config"), list):
|
||||
return web.json_response({"status": "error", "message": "Invalid physical folder configuration"}, status=400)
|
||||
if "folder_view_mode" in data and data.get("folder_view_mode") not in {"abstract", "physical"}:
|
||||
return web.json_response({"status": "error", "message": "Invalid folder view mode"}, status=400)
|
||||
|
||||
if api_key is not None:
|
||||
cfg["CIVITAI_API_KEY"] = api_key.strip()
|
||||
deepl_key = data.get("deepl_key")
|
||||
if api_key is not None and not isinstance(api_key, str):
|
||||
return web.json_response({"status": "error", "message": "Invalid API key"}, status=400)
|
||||
if deepl_key is not None and not isinstance(deepl_key, str):
|
||||
return web.json_response({"status": "error", "message": "Invalid DeepL key"}, status=400)
|
||||
proxy = None
|
||||
if "proxy" in data: # Settings -> Network (net_proxy.py); "" = the system's proxy
|
||||
try:
|
||||
proxy = normalize_proxy(data.get("proxy"))
|
||||
except ValueError as error:
|
||||
return web.json_response({"status": "error", "message": "Invalid proxy", "code": str(error)}, status=400)
|
||||
if folder_types_config is not None and not isinstance(folder_types_config, list):
|
||||
return web.json_response({"status": "error", "message": "Invalid folder configuration"}, status=400)
|
||||
if "physical_folders_config" in data and not isinstance(data.get("physical_folders_config"), list):
|
||||
return web.json_response({"status": "error", "message": "Invalid physical folder configuration"}, status=400)
|
||||
if "folder_view_mode" in data and data.get("folder_view_mode") not in {"abstract", "physical"}:
|
||||
return web.json_response({"status": "error", "message": "Invalid folder view mode"}, status=400)
|
||||
|
||||
if api_key is not None:
|
||||
cfg["CIVITAI_API_KEY"] = api_key.strip()
|
||||
|
||||
if deepl_key is not None: # Settings -> Translation (translation_routes.py)
|
||||
cfg["DEEPL_API_KEY"] = deepl_key.strip()
|
||||
|
||||
if proxy is not None:
|
||||
cfg[PROXY_KEY] = proxy
|
||||
|
||||
if folder_types_config is not None:
|
||||
cfg["folder_types_config"] = folder_types_config
|
||||
@@ -71,10 +98,10 @@ async def api_save_config(request):
|
||||
if "folder_view_mode" in data:
|
||||
cfg["folder_view_mode"] = data.get("folder_view_mode")
|
||||
|
||||
temp_path = config_path + ".tmp"
|
||||
with open(temp_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(cfg, f, indent=4)
|
||||
os.replace(temp_path, config_path)
|
||||
temp_path = config_path + ".tmp"
|
||||
with open(temp_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(cfg, f, indent=4)
|
||||
os.replace(temp_path, config_path)
|
||||
|
||||
return web.json_response({"status": "ok"})
|
||||
except Exception as e:
|
||||
|
||||
@@ -0,0 +1,354 @@
|
||||
"""Where a model the open workflow is missing can be downloaded from, for Model Check's
|
||||
"Download" (api/model_download.py fetches it).
|
||||
|
||||
Sources that name one exact file: Civitai by the file's hash (the workflow's fingerprint), a
|
||||
Civitai model version or file link, or a direct file link on Hugging Face (or its mirror) or
|
||||
GitHub, as a shared workflow carries them (AMB's model links, ComfyUI's own `properties.models`).
|
||||
Then ComfyUI-Manager's model list by file name (manager_catalog.py): with the fingerprint, only
|
||||
the entry whose file has it; without, the one entry of that name, marked `by_name` because a
|
||||
name is no proof (the page asks before it downloads one). Each answer says the file's size and
|
||||
SHA-256 when the site gives them, so the download can be checked.
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
try:
|
||||
from ..civitai_client import USER_AGENT, _load_api_key
|
||||
from ..net_proxy import urlopen
|
||||
except ImportError: # loaded outside the package (tests)
|
||||
from civitai_client import USER_AGENT, _load_api_key
|
||||
from net_proxy import urlopen
|
||||
from . import manager_catalog
|
||||
|
||||
CIVITAI_HOSTS = ("civitai.com", "civitai.red")
|
||||
HF_HOSTS = ("huggingface.co", "hf-mirror.com")
|
||||
HF_MIRROR = "hf-mirror.com" # Hugging Face's mirror for mainland China (it sends other networks back)
|
||||
OTHER_HOSTS = ("github.com",)
|
||||
TIMEOUT = 15
|
||||
SHA256 = re.compile(r"^[0-9a-fA-F]{64}$")
|
||||
|
||||
|
||||
class SourceUnreachable(Exception):
|
||||
"""The site did not answer (network, timeout, server error)."""
|
||||
|
||||
|
||||
def host_of(url):
|
||||
try:
|
||||
parsed = urllib.parse.urlsplit(str(url).strip())
|
||||
except ValueError:
|
||||
return ""
|
||||
return (parsed.hostname or "").lower() if parsed.scheme == "https" else ""
|
||||
|
||||
|
||||
def _site(host, sites):
|
||||
return any(host == site or host.endswith("." + site) for site in sites)
|
||||
|
||||
|
||||
def allowed_download_url(url):
|
||||
"""Whether `url` is an https link on a site downloads may start from."""
|
||||
host = host_of(url)
|
||||
return bool(host) and _site(host, CIVITAI_HOSTS + HF_HOSTS + OTHER_HOSTS)
|
||||
|
||||
|
||||
def is_civitai(url):
|
||||
return _site(host_of(url), CIVITAI_HOSTS)
|
||||
|
||||
|
||||
def civitai_headers():
|
||||
headers = {"User-Agent": USER_AGENT}
|
||||
key = _load_api_key()
|
||||
if key:
|
||||
headers["Authorization"] = f"Bearer {key}"
|
||||
return headers
|
||||
|
||||
|
||||
def _get_json(url, headers):
|
||||
"""The JSON at `url`; None on 404; SourceUnreachable when it could not be asked."""
|
||||
try:
|
||||
with urlopen(urllib.request.Request(url, headers=headers), timeout=TIMEOUT) as response:
|
||||
return json.loads(response.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as error:
|
||||
if error.code in (400, 404):
|
||||
return None
|
||||
raise SourceUnreachable(f"HTTP {error.code}") from error
|
||||
except (urllib.error.URLError, OSError, ValueError) as error:
|
||||
raise SourceUnreachable(str(error)) from error
|
||||
|
||||
|
||||
def _civitai_page(version):
|
||||
domain = "civitai.red" if (version.get("model") or {}).get("nsfw") or (version.get("nsfwLevel") or 1) > 1 else "civitai.com"
|
||||
model_id = version.get("modelId")
|
||||
return f"https://{domain}/models/{model_id}?modelVersionId={version.get('id')}" if model_id else ""
|
||||
|
||||
|
||||
def _pick_civitai_file(files, file_hash="", name=""):
|
||||
"""The version's file with this hash (SHA-256 or a prefix such as AutoV2; None if it has
|
||||
none such). Without a hash: the file of this name, else the version's primary model file —
|
||||
the link names that version, and people rename their files."""
|
||||
files = [item for item in files or [] if isinstance(item, dict) and item.get("downloadUrl")]
|
||||
wanted = str(file_hash or "").upper()
|
||||
if wanted:
|
||||
for item in files:
|
||||
hashes = {str(value).upper() for value in (item.get("hashes") or {}).values()}
|
||||
sha = str((item.get("hashes") or {}).get("SHA256") or "").upper()
|
||||
if wanted in hashes or (len(wanted) >= 10 and sha.startswith(wanted)):
|
||||
return item
|
||||
return None
|
||||
if name:
|
||||
for item in files:
|
||||
if str(item.get("name", "")).lower() == name.lower():
|
||||
return item
|
||||
models = [item for item in files if item.get("type", "Model") == "Model"]
|
||||
return next((item for item in models if item.get("primary")), models[0] if models else None)
|
||||
|
||||
|
||||
def _from_civitai_version(version, file_hash="", name=""):
|
||||
picked = _pick_civitai_file(version.get("files"), file_hash, name)
|
||||
if not picked:
|
||||
return None
|
||||
sha = str((picked.get("hashes") or {}).get("SHA256") or "")
|
||||
return {
|
||||
"found": True,
|
||||
"source": "civitai",
|
||||
"download_url": picked["downloadUrl"],
|
||||
"file_name": picked.get("name") or "",
|
||||
"size": int(float(picked.get("sizeKB") or 0) * 1024),
|
||||
"sha256": sha.lower() if SHA256.match(sha) else "",
|
||||
"format": ((picked.get("metadata") or {}).get("format") or ""),
|
||||
"base_model": version.get("baseModel") or "",
|
||||
"model_name": (version.get("model") or {}).get("name") or "",
|
||||
"version_name": version.get("name") or "",
|
||||
"page": _civitai_page(version),
|
||||
}
|
||||
|
||||
|
||||
def civitai_by_hash(file_hash):
|
||||
version = _get_json(f"https://civitai.com/api/v1/model-versions/by-hash/{urllib.parse.quote(file_hash)}", civitai_headers())
|
||||
return _from_civitai_version(version, file_hash) if version else None
|
||||
|
||||
|
||||
def _civitai_version_id(url):
|
||||
parsed = urllib.parse.urlsplit(url)
|
||||
query = urllib.parse.parse_qs(parsed.query)
|
||||
if query.get("modelVersionId"):
|
||||
return query["modelVersionId"][0]
|
||||
match = re.search(r"/api/download/models/(\d+)", parsed.path) or re.search(r"/model-versions/(\d+)", parsed.path)
|
||||
return match.group(1) if match else ""
|
||||
|
||||
|
||||
def civitai_by_link(url, name, file_hash=""):
|
||||
"""A Civitai link that names a model version: its file of this hash or name."""
|
||||
version_id = _civitai_version_id(url)
|
||||
if not version_id.isdigit():
|
||||
return None
|
||||
version = _get_json(f"https://civitai.com/api/v1/model-versions/{version_id}", civitai_headers())
|
||||
return _from_civitai_version(version, file_hash, name) if version else None
|
||||
|
||||
|
||||
class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
def redirect_request(self, *args, **kwargs):
|
||||
return None
|
||||
|
||||
|
||||
def _hf_file_url(url, host=""):
|
||||
"""https://<hf>/<repo>/resolve/<rev>/<path> for a blob or resolve link to a file, else '';
|
||||
on `host` (Hugging Face or its mirror) when given."""
|
||||
parsed = urllib.parse.urlsplit(url)
|
||||
parts = [part for part in parsed.path.split("/") if part]
|
||||
for marker in ("resolve", "blob"):
|
||||
if marker in parts:
|
||||
at = parts.index(marker)
|
||||
if at >= 2 and len(parts) > at + 2:
|
||||
parts[at] = "resolve"
|
||||
return urllib.parse.urlunsplit(("https", host or parsed.netloc, "/" + "/".join(parts), "", ""))
|
||||
return ""
|
||||
|
||||
|
||||
def _head(url):
|
||||
"""(headers, redirected) of a HEAD request that does not follow redirects; None on 404,
|
||||
"gated" on 401/403."""
|
||||
opener = urllib.request.build_opener(_NoRedirect)
|
||||
request = urllib.request.Request(url, method="HEAD", headers={"User-Agent": USER_AGENT})
|
||||
try:
|
||||
return opener.open(request, timeout=TIMEOUT).headers, False
|
||||
except urllib.error.HTTPError as error:
|
||||
if error.code in (401, 403):
|
||||
return "gated"
|
||||
if error.code == 404:
|
||||
return None
|
||||
if error.code not in (301, 302, 303, 307, 308):
|
||||
raise SourceUnreachable(f"HTTP {error.code}") from error
|
||||
return error.headers, True
|
||||
except (urllib.error.URLError, OSError) as error:
|
||||
raise SourceUnreachable(str(error)) from error
|
||||
|
||||
|
||||
def hugging_face_file(url, host=""):
|
||||
"""A Hugging Face file link, fetched from `host` (huggingface.co or the mirror; the link's
|
||||
own when empty): its size and SHA-256 from the site's headers."""
|
||||
file_url = _hf_file_url(url, host)
|
||||
if not file_url:
|
||||
return None
|
||||
answer = _head(file_url)
|
||||
if isinstance(answer, tuple) and answer[1] and not answer[0].get("x-linked-size"):
|
||||
# Sent on to the other Hugging Face host (the mirror does that outside China): ask there.
|
||||
location = urllib.parse.urljoin(file_url, answer[0].get("Location") or "")
|
||||
if _site(host_of(location), HF_HOSTS) and host_of(location) != host_of(file_url):
|
||||
answer = _head(location)
|
||||
if answer is None:
|
||||
return None
|
||||
if answer == "gated":
|
||||
# The repository's page, where the user logs in and accepts its terms.
|
||||
parts = urllib.parse.urlsplit(file_url).path.strip("/").split("/")
|
||||
return {"found": False, "reason": "gated", "page": f"https://huggingface.co/{'/'.join(parts[:2])}"}
|
||||
headers, redirected = answer
|
||||
etag = str(headers.get("x-linked-etag") or headers.get("etag") or "").strip('"').replace("W/", "").strip('"')
|
||||
# A redirect's own length is not the file's: without x-linked-size (small files kept
|
||||
# outside large-file storage) the size is unknown.
|
||||
size = headers.get("x-linked-size") or (0 if redirected else headers.get("content-length")) or 0
|
||||
return {
|
||||
"found": True,
|
||||
"source": "huggingface",
|
||||
"mirror": host_of(file_url) == HF_MIRROR,
|
||||
"download_url": file_url,
|
||||
"file_name": urllib.parse.unquote(file_url.rsplit("/", 1)[-1]),
|
||||
"size": int(size) if str(size).isdigit() else 0,
|
||||
"sha256": etag.lower() if SHA256.match(etag) else "",
|
||||
"format": "",
|
||||
"base_model": "",
|
||||
"model_name": "",
|
||||
"version_name": "",
|
||||
"page": url,
|
||||
}
|
||||
|
||||
|
||||
def plain_link(url):
|
||||
"""A direct file link on GitHub (a release asset, or a file in a repository, fetched raw): no
|
||||
size or hash to check against."""
|
||||
parsed = urllib.parse.urlsplit(url)
|
||||
path = parsed.path
|
||||
if "/blob/" in path: # the page about the file; its raw form is the file
|
||||
path = path.replace("/blob/", "/raw/", 1)
|
||||
if "/releases/download/" not in path and "/raw/" not in path:
|
||||
return None
|
||||
return {
|
||||
"found": True, "source": "link", "download_url": urllib.parse.urlunsplit(("https", parsed.netloc, path, "", "")),
|
||||
"file_name": urllib.parse.unquote(path.rsplit("/", 1)[-1]), "size": 0, "sha256": "",
|
||||
"format": "", "base_model": "", "model_name": "", "version_name": "", "page": url,
|
||||
}
|
||||
|
||||
|
||||
def same_file(expected, sha256):
|
||||
"""Whether a fingerprint (full SHA-256 or a prefix of 10+, AutoV2) names this SHA-256."""
|
||||
expected, sha256 = str(expected or "").lower(), str(sha256 or "").lower()
|
||||
return len(expected) >= 10 and sha256.startswith(expected)
|
||||
|
||||
|
||||
def _is_sha_prefix(value):
|
||||
return len(value) >= 10 and bool(re.fullmatch(r"[0-9a-fA-F]+", value))
|
||||
|
||||
|
||||
class _Sites:
|
||||
"""Which sites stopped answering during one lookup, so the next models skip only those
|
||||
(offline in mainland China, Civitai fails while Hugging Face through its mirror works)."""
|
||||
|
||||
def __init__(self, down=None):
|
||||
self.down = down if down is not None else set()
|
||||
self.missed = False
|
||||
|
||||
def ask(self, site, call, *args):
|
||||
if site in self.down:
|
||||
self.missed = True
|
||||
return None
|
||||
try:
|
||||
return call(*args)
|
||||
except SourceUnreachable:
|
||||
self.down.add(site)
|
||||
self.missed = True
|
||||
return None
|
||||
|
||||
|
||||
def _by_link(sites, url, name, file_hash, hf_host):
|
||||
host = host_of(url)
|
||||
if _site(host, CIVITAI_HOSTS):
|
||||
return sites.ask("civitai", civitai_by_link, url, name, file_hash)
|
||||
if _site(host, HF_HOSTS):
|
||||
return sites.ask("huggingface", hugging_face_file, url, hf_host)
|
||||
if _site(host, OTHER_HOSTS):
|
||||
return plain_link(url)
|
||||
return None
|
||||
|
||||
|
||||
def _from_manager_list(sites, name, folder_type, file_hash, hf_host):
|
||||
"""A file of this name in ComfyUI-Manager's model list. With the workflow's fingerprint,
|
||||
only an entry whose file has that SHA-256 (several of the same name are each asked); without,
|
||||
the one entry of that name, marked `by_name` (nothing can tell it is the same file)."""
|
||||
entries = manager_catalog.candidates(name, folder_type)
|
||||
if not entries:
|
||||
return None
|
||||
if _is_sha_prefix(file_hash):
|
||||
for entry in entries[:6]:
|
||||
found = _by_link(sites, entry["url"], name, "", hf_host)
|
||||
if found and found.get("found") and same_file(file_hash, found.get("sha256")):
|
||||
return {**found, "via": "manager_list", "list_name": entry["name"], "by_name": False}
|
||||
return None
|
||||
if len(entries) != 1:
|
||||
return None # several files of that name: which one is unknowable
|
||||
found = _by_link(sites, entries[0]["url"], name, "", hf_host)
|
||||
if found and found.get("found"):
|
||||
return {**found, "via": "manager_list", "list_name": entries[0]["name"], "by_name": True,
|
||||
"base_model": found.get("base_model") or entries[0]["base"]}
|
||||
return found
|
||||
|
||||
|
||||
def _same_file_on_hugging_face(sites, civitai, url, name, folder_type, hf_host):
|
||||
"""With the mirror on (mainland China, where Civitai is slow or out of reach and some of its
|
||||
files need a login), the same file (same SHA-256) on Hugging Face, from the workflow's link
|
||||
or ComfyUI-Manager's list, is fetched there instead. Civitai's name, base model and page stay."""
|
||||
sha = civitai.get("sha256") or ""
|
||||
if not SHA256.match(sha):
|
||||
return civitai
|
||||
other = None
|
||||
if url and _site(host_of(url), HF_HOSTS):
|
||||
other = _by_link(sites, url, name, "", hf_host)
|
||||
if not (other and other.get("found") and same_file(sha, other.get("sha256"))):
|
||||
other = None
|
||||
if other is None and name:
|
||||
other = _from_manager_list(sites, name, folder_type, sha, hf_host)
|
||||
if not other:
|
||||
return civitai
|
||||
keep = {key: civitai[key] for key in ("base_model", "model_name", "version_name", "page") if civitai.get(key)}
|
||||
return {**other, **keep, "via": other.get("via", ""), "also_on": "civitai"}
|
||||
|
||||
|
||||
def find_source(file_hash="", url="", name="", hf_mirror=False, folder_type="", down=None):
|
||||
"""Where one missing model can come from: {found, source, download_url, file_name, size,
|
||||
sha256, base_model, model_name, version_name, page}, or {found: False, reason}. Asked in
|
||||
turn: Civitai by the fingerprint, the workflow's link, ComfyUI-Manager's model list by file
|
||||
name. With `hf_mirror`, Hugging Face files come from its mirror (and without, never from it).
|
||||
`down` (a set kept across one lookup) names the sites that stopped answering."""
|
||||
sites = _Sites(down)
|
||||
hf_host = HF_MIRROR if hf_mirror else "huggingface.co"
|
||||
looked = False
|
||||
found = None
|
||||
if file_hash:
|
||||
looked = True
|
||||
found = sites.ask("civitai", civitai_by_hash, file_hash)
|
||||
if not found and url and host_of(url):
|
||||
looked = True
|
||||
found = _by_link(sites, url, name, file_hash, hf_host)
|
||||
if not (found and found.get("found")) and name:
|
||||
listed = _from_manager_list(sites, name, folder_type, file_hash, hf_host)
|
||||
looked = looked or listed is not None or bool(manager_catalog.candidates(name, folder_type))
|
||||
found = listed or found
|
||||
if found and found.get("found") and found.get("source") == "civitai" and hf_mirror:
|
||||
found = _same_file_on_hugging_face(sites, found, url, name, folder_type, hf_host)
|
||||
if found:
|
||||
return found
|
||||
if sites.missed:
|
||||
return {"found": False, "reason": "network"}
|
||||
return {"found": False, "reason": "not_found" if looked else "no_source"}
|
||||
+41
-4
@@ -1,4 +1,9 @@
|
||||
"""Configured model-folder visibility and scan-scope helpers."""
|
||||
"""Configured model-folder visibility and scan-scope helpers.
|
||||
|
||||
Shown folders are also the ones scans read, so a workflow can carry each model's fingerprint
|
||||
only for them. Text encoders, CLIP vision and upscalers are shown by default since
|
||||
2026-10 (FOLDER_DEFAULTS_VERSION 2); an older config.json gets them turned on once.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
@@ -6,12 +11,44 @@ import os
|
||||
from aiohttp import web
|
||||
import folder_paths
|
||||
|
||||
from .path_utils import atomic_write_json
|
||||
|
||||
DEFAULT_TYPES = ['checkpoints', 'loras', 'diffusion_models', 'unet', 'controlnet', 'vae',
|
||||
'text_encoders', 'clip', 'clip_vision', 'upscale_models']
|
||||
# Added to the defaults in version 2 (the components a workflow's fingerprints need most).
|
||||
ADDED_IN_2 = {'text_encoders', 'clip', 'clip_vision', 'upscale_models'}
|
||||
FOLDER_DEFAULTS_VERSION = 2
|
||||
|
||||
DEFAULT_PHYSICAL_FOLDER_NAMES = {
|
||||
'checkpoints', 'loras', 'unet', 'diffusion_models', 'controlnet', 'vae'
|
||||
'checkpoints', 'loras', 'unet', 'diffusion_models', 'controlnet', 'vae',
|
||||
'text_encoders', 'clip', 'clip_vision', 'upscale_models',
|
||||
}
|
||||
|
||||
|
||||
def config_path():
|
||||
return os.path.join(os.path.dirname(os.path.abspath(__file__)), "config.json")
|
||||
|
||||
|
||||
def upgrade_folder_defaults(path=None):
|
||||
"""Once per config.json: the folders added to the defaults are shown (and scanned) even
|
||||
where an earlier version saved them as hidden. True when the file changed."""
|
||||
path = path or config_path()
|
||||
try:
|
||||
with open(path, encoding='utf-8') as source:
|
||||
cfg = json.load(source)
|
||||
except (OSError, ValueError):
|
||||
return False # no config yet: the defaults apply as they are
|
||||
if not isinstance(cfg, dict) or int(cfg.get("folder_defaults_version") or 1) >= FOLDER_DEFAULTS_VERSION:
|
||||
return False
|
||||
for key in ("folder_types_config", "physical_folders_config"):
|
||||
for item in cfg.get(key) or []:
|
||||
if isinstance(item, dict) and item.get("type") in ADDED_IN_2:
|
||||
item["visible"] = True
|
||||
cfg["folder_defaults_version"] = FOLDER_DEFAULTS_VERSION
|
||||
atomic_write_json(path, cfg)
|
||||
return True
|
||||
|
||||
|
||||
def get_folder_view_mode():
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
config_path = os.path.join(script_dir, "config.json")
|
||||
@@ -149,7 +186,7 @@ def get_active_folder_types():
|
||||
config_path = os.path.join(script_dir, "config.json")
|
||||
|
||||
all_types = list(folder_paths.folder_names_and_paths.keys())
|
||||
default_types = ['checkpoints', 'loras', 'diffusion_models', 'unet', 'controlnet', 'vae']
|
||||
default_types = DEFAULT_TYPES
|
||||
|
||||
try:
|
||||
if os.path.exists(config_path):
|
||||
@@ -215,7 +252,7 @@ async def api_get_all_folder_types(request):
|
||||
else:
|
||||
# Abstract mode
|
||||
all_types = list(folder_paths.folder_names_and_paths.keys())
|
||||
default_types = ['checkpoints', 'loras', 'diffusion_models', 'controlnet', 'vae']
|
||||
default_types = DEFAULT_TYPES
|
||||
configured_types = set()
|
||||
try:
|
||||
if os.path.exists(config_path):
|
||||
|
||||
+55
-10
@@ -9,13 +9,22 @@ from aiohttp import web
|
||||
import folder_paths
|
||||
|
||||
from .image_search import filter_images
|
||||
from .media_routes import CARD_THUMBNAIL_STATIC_EXTENSIONS, _build_card_thumbnail
|
||||
from .media_routes import (
|
||||
CARD_POSTER_VIDEO_EXTENSIONS,
|
||||
CARD_THUMBNAIL_STATIC_EXTENSIONS,
|
||||
_build_card_thumbnail,
|
||||
_build_video_poster,
|
||||
)
|
||||
from .path_utils import require_filename, resolve_within
|
||||
from .trash import move_to_trash, trash_failure
|
||||
|
||||
|
||||
GALLERY_SNAPSHOT_SECONDS = 10
|
||||
MAX_CACHED_GALLERY_IMAGES = 50_000
|
||||
GALLERY_IMAGE_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.webp', '.gif', '.avif'}
|
||||
# Videos a browser plays in the viewer; not .avi (Chrome and Edge cannot).
|
||||
GALLERY_VIDEO_EXTENSIONS = {'.mp4', '.webm', '.mov', '.mkv'}
|
||||
GALLERY_MEDIA = ("all", "image", "video")
|
||||
_gallery_snapshot_lock = threading.Lock()
|
||||
_gallery_snapshot = {"root": None, "expires": 0, "images": []}
|
||||
|
||||
@@ -24,7 +33,8 @@ def _collect_gallery_images(output_dir):
|
||||
images = []
|
||||
for root, _, files in os.walk(output_dir):
|
||||
for filename in files:
|
||||
if os.path.splitext(filename)[1].lower() not in {'.png', '.jpg', '.jpeg', '.webp', '.gif'}:
|
||||
ext = os.path.splitext(filename)[1].lower()
|
||||
if ext not in GALLERY_IMAGE_EXTENSIONS and ext not in GALLERY_VIDEO_EXTENSIONS:
|
||||
continue
|
||||
try:
|
||||
mtime = os.path.getmtime(os.path.join(root, filename))
|
||||
@@ -34,7 +44,9 @@ def _collect_gallery_images(output_dir):
|
||||
images.append({
|
||||
"filename": filename,
|
||||
"subfolder": "" if subfolder == "." else subfolder.replace(os.sep, '/'),
|
||||
"type": "output", "mtime": mtime,
|
||||
"type": "output",
|
||||
"mtime": mtime,
|
||||
"media_type": "video" if ext in GALLERY_VIDEO_EXTENSIONS else "image",
|
||||
})
|
||||
images.sort(key=lambda item: (item['mtime'], item['subfolder'], item['filename']), reverse=True)
|
||||
return images
|
||||
@@ -58,33 +70,61 @@ def _invalidate_gallery_snapshot():
|
||||
_gallery_snapshot.update(root=None, expires=0, images=[])
|
||||
|
||||
|
||||
def _gallery_listing(output_dir, refresh, query):
|
||||
def _format(filename):
|
||||
"""The format name the gallery's settings use: the extension, with .jpeg counted as jpg."""
|
||||
ext = os.path.splitext(filename)[1].lower().lstrip('.')
|
||||
return 'jpg' if ext == 'jpeg' else ext
|
||||
|
||||
|
||||
def _gallery_listing(output_dir, refresh, query, media="image", hidden=frozenset()):
|
||||
"""Outputs, newest first. `media`: "image" (the default, for callers that show pictures),
|
||||
"video" or "all"; `hidden`: format names left out (see _format)."""
|
||||
images = _gallery_images(output_dir, refresh)
|
||||
if media != "all" or hidden:
|
||||
images = [item for item in images
|
||||
if media in ("all", item["media_type"]) and _format(item["filename"]) not in hidden]
|
||||
return filter_images(os.path.realpath(output_dir), images, query) if query else images
|
||||
|
||||
|
||||
def _gallery_page_source(output_dir, refresh, query, media, hidden):
|
||||
"""The listing, and whether the output folder has any video (the gallery offers its
|
||||
Pictures / Videos switch only then)."""
|
||||
images = _gallery_listing(output_dir, refresh, query, media, hidden)
|
||||
has_videos = any(item["media_type"] == "video" for item in _gallery_images(output_dir))
|
||||
return images, has_videos
|
||||
|
||||
|
||||
async def api_get_gallery_images(request):
|
||||
"""GET /anomalous/gallery_images?page&limit&refresh&term=..&term=.. (or q) - Output images, optionally searched by embedded generation info."""
|
||||
"""GET /anomalous/gallery_images?page&limit&refresh&term=..&term=.. (or q)&media=all|image|video&hide=mkv,gif
|
||||
- Outputs (pictures only unless `media` says otherwise), optionally searched by embedded
|
||||
generation info, without the `hide` formats."""
|
||||
try:
|
||||
page = max(1, int(request.query.get('page', 1)))
|
||||
limit = min(200, max(1, int(request.query.get('limit', 50))))
|
||||
# Repeated `term` params keep phrases whole; `q` is split on whitespace.
|
||||
terms = [term for term in request.query.getall('term', []) if term.strip()]
|
||||
query = terms or request.query.get('q', '').strip()
|
||||
images = await asyncio.to_thread(_gallery_listing, folder_paths.get_output_directory(), request.query.get('refresh') == '1', query)
|
||||
media = request.query.get('media', 'image')
|
||||
if media not in GALLERY_MEDIA:
|
||||
raise ValueError(media)
|
||||
hidden = frozenset(part.strip().lower().lstrip('.') for part in request.query.get('hide', '').split(','))
|
||||
images, has_videos = await asyncio.to_thread(
|
||||
_gallery_page_source, folder_paths.get_output_directory(), request.query.get('refresh') == '1',
|
||||
query, media, hidden - {''})
|
||||
total = len(images)
|
||||
start = (page - 1) * limit
|
||||
return web.json_response({"images": images[start:start + limit], "total": total,
|
||||
"page": page, "pages": (total + limit - 1) // limit})
|
||||
"page": page, "pages": (total + limit - 1) // limit,
|
||||
"has_videos": has_videos})
|
||||
except (TypeError, ValueError):
|
||||
return web.json_response({"error": "Invalid pagination"}, status=400)
|
||||
return web.json_response({"error": "Invalid page or filter"}, status=400)
|
||||
except OSError:
|
||||
return web.json_response({"error": "Could not list output images"}, status=500)
|
||||
|
||||
|
||||
async def api_output_thumbnail(request):
|
||||
"""GET /anomalous/output_thumbnail?filename&subfolder - the gallery grid's 512 px WebP of an
|
||||
output image (cached like model cards); the original stays at ComfyUI's /view."""
|
||||
output image/video (cached like model cards); the original stays at ComfyUI's /view."""
|
||||
output_dir = folder_paths.get_output_directory()
|
||||
try:
|
||||
filename = require_filename(request.query.get('filename'))
|
||||
@@ -94,8 +134,13 @@ async def api_output_thumbnail(request):
|
||||
if not os.path.isfile(file_path):
|
||||
return web.json_response({"status": "error", "message": "File not found"}, status=404)
|
||||
served = file_path
|
||||
if os.path.splitext(file_path)[1].lower() in CARD_THUMBNAIL_STATIC_EXTENSIONS:
|
||||
ext = os.path.splitext(file_path)[1].lower()
|
||||
if ext in CARD_THUMBNAIL_STATIC_EXTENSIONS:
|
||||
served = await asyncio.to_thread(_build_card_thumbnail, file_path)
|
||||
elif ext in CARD_POSTER_VIDEO_EXTENSIONS:
|
||||
poster = await asyncio.to_thread(_build_video_poster, file_path)
|
||||
if poster:
|
||||
served = poster
|
||||
# Revalidate (cheap 304s): an output file can be replaced under the same name.
|
||||
return web.FileResponse(served, headers={'Cache-Control': 'no-cache'})
|
||||
|
||||
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
"""What a Hugging Face repository says about a model downloaded from it (model_download.py):
|
||||
the model card's first example image becomes its cover, and its trigger words, example prompts
|
||||
and base model go into its notes, with the repository as its link. Hugging Face cannot be asked
|
||||
by fingerprint, so this works only for files downloaded from it (the repository is known then).
|
||||
|
||||
Nothing the user or a scan already has is replaced: a cover only when the model has none, notes
|
||||
and link only when empty (in the user's layer, <model>.anomalous.json).
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from .model_constants import MEDIA_EXTENSIONS, PREVIEW_SUFFIXES
|
||||
from .path_utils import atomic_write_json
|
||||
|
||||
try:
|
||||
from ..model_identity import USER_INFO_SUFFIX, read_json
|
||||
from ..civitai_client import USER_AGENT
|
||||
from ..net_proxy import urlopen
|
||||
except ImportError: # loaded outside the package (tests)
|
||||
from model_identity import USER_INFO_SUFFIX, read_json
|
||||
from civitai_client import USER_AGENT
|
||||
from net_proxy import urlopen
|
||||
|
||||
TIMEOUT = 15
|
||||
MAX_COVER_BYTES = 20 * 1024 * 1024
|
||||
IMAGE_TYPES = {"image/png": ".png", "image/jpeg": ".jpg", "image/webp": ".webp", "image/gif": ".gif"}
|
||||
IMAGE_FILE = (".png", ".jpg", ".jpeg", ".webp")
|
||||
HF_HOSTS = ("huggingface.co", "hf-mirror.com")
|
||||
MAX_PROMPTS = 3
|
||||
|
||||
|
||||
def repo_of(url):
|
||||
"""("org/name", host) of a Hugging Face file link, or None."""
|
||||
parsed = urllib.parse.urlsplit(url)
|
||||
parts = [part for part in parsed.path.split("/") if part]
|
||||
if (parsed.hostname or "") not in HF_HOSTS or "resolve" not in parts or parts.index("resolve") < 2:
|
||||
return None
|
||||
return "/".join(parts[:2]), parsed.hostname
|
||||
|
||||
|
||||
def _get(url, limit=None):
|
||||
request = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
|
||||
with urlopen(request, timeout=TIMEOUT) as response:
|
||||
data = response.read(limit + 1 if limit else -1)
|
||||
return data, str(response.headers.get("Content-Type", "")).split(";")[0].strip().lower()
|
||||
|
||||
|
||||
def _file_url(repo, host, path):
|
||||
"""A link to a file of the repository, on `host` (the mirror when the download used it)."""
|
||||
if path.startswith(("https://", "http://")):
|
||||
parsed = urllib.parse.urlsplit(path)
|
||||
if parsed.hostname not in HF_HOSTS:
|
||||
return ""
|
||||
return urllib.parse.urlunsplit(("https", host, parsed.path, parsed.query, ""))
|
||||
return f"https://{host}/{repo}/resolve/main/{urllib.parse.quote(path.lstrip('/'))}"
|
||||
|
||||
|
||||
def _as_list(value):
|
||||
if isinstance(value, str):
|
||||
return [value] if value.strip() else []
|
||||
return [str(item) for item in value if str(item).strip()] if isinstance(value, list) else []
|
||||
|
||||
|
||||
def read_card(repo, host):
|
||||
"""{images, prompts, trigger, base} from the repository's model card (empty when it has none)."""
|
||||
data, _type = _get(f"https://{host}/api/models/{repo}?expand[]=cardData&expand[]=siblings")
|
||||
info = json.loads(data.decode("utf-8"))
|
||||
card = info.get("cardData") or {}
|
||||
widgets = [item for item in card.get("widget") or [] if isinstance(item, dict)]
|
||||
images = [(item.get("output") or {}).get("url") for item in widgets]
|
||||
images = [url for url in (_file_url(repo, host, str(path)) for path in images if path) if url]
|
||||
if not images: # no examples in the card: the repository's own pictures
|
||||
files = [item.get("rfilename", "") for item in info.get("siblings") or [] if isinstance(item, dict)]
|
||||
images = [_file_url(repo, host, name) for name in files if name.lower().endswith(IMAGE_FILE)][:1]
|
||||
prompts = [str(item.get("text")).strip() for item in widgets if isinstance(item.get("text"), str) and item["text"].strip()]
|
||||
return {"images": images, "prompts": prompts[:MAX_PROMPTS],
|
||||
"trigger": _as_list(card.get("instance_prompt")), "base": _as_list(card.get("base_model"))}
|
||||
|
||||
|
||||
def save_cover(image_url, base_path):
|
||||
"""The image as the model's cover (<model>.preview.<ext>) when it has none yet; True if saved."""
|
||||
if any(os.path.exists(base_path + suffix) for suffix in PREVIEW_SUFFIXES + MEDIA_EXTENSIONS):
|
||||
return False
|
||||
data, content_type = _get(image_url, MAX_COVER_BYTES)
|
||||
extension = IMAGE_TYPES.get(content_type)
|
||||
if not extension or len(data) > MAX_COVER_BYTES:
|
||||
return False
|
||||
target = base_path + ".preview" + extension
|
||||
temporary = target + ".part"
|
||||
with open(temporary, "wb") as output:
|
||||
output.write(data)
|
||||
os.replace(temporary, target)
|
||||
return True
|
||||
|
||||
|
||||
def notes_text(repo, card):
|
||||
lines = [f"Hugging Face: {repo}"]
|
||||
if card["base"]:
|
||||
lines.append("Base model: " + ", ".join(card["base"]))
|
||||
if card["trigger"]:
|
||||
lines.append("Trigger words: " + ", ".join(card["trigger"]))
|
||||
if card["prompts"]:
|
||||
lines.append("Example prompts:")
|
||||
lines.extend("- " + prompt[:400] for prompt in card["prompts"])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def save_notes(base_path, repo, card):
|
||||
"""The card's words as the model's notes and the repository as its link, where both are empty."""
|
||||
path = base_path + USER_INFO_SUFFIX
|
||||
if os.path.exists(path) and read_json(path) is None:
|
||||
return False # unreadable: leave it to the user
|
||||
user = read_json(path) or {"format": 1}
|
||||
changed = False
|
||||
if not str(user.get("custom_notes") or "").strip():
|
||||
user["custom_notes"] = notes_text(repo, card)
|
||||
changed = True
|
||||
if not str(user.get("source_url") or "").strip():
|
||||
user["source_url"] = f"https://huggingface.co/{repo}"
|
||||
changed = True
|
||||
if changed:
|
||||
atomic_write_json(path, user)
|
||||
return changed
|
||||
|
||||
|
||||
def enrich(model_path, download_url):
|
||||
"""Cover and notes for a model just downloaded from Hugging Face: {cover, notes}."""
|
||||
located = repo_of(download_url)
|
||||
if not located:
|
||||
return {"cover": False, "notes": False}
|
||||
repo, host = located
|
||||
base_path = os.path.splitext(model_path)[0]
|
||||
card = read_card(repo, host)
|
||||
cover = False
|
||||
for image in card["images"][:2]:
|
||||
try:
|
||||
cover = save_cover(image, base_path)
|
||||
break
|
||||
except OSError:
|
||||
continue
|
||||
return {"cover": cover, "notes": save_notes(base_path, repo, card)}
|
||||
@@ -27,6 +27,7 @@ LOCAL_GETS = frozenset({
|
||||
"/anomalous/backup/file",
|
||||
"/anomalous/version/releases",
|
||||
"/anomalous/import/folders",
|
||||
"/anomalous/civitai_check",
|
||||
})
|
||||
# Routes that answer other devices themselves (MCP replies in JSON-RPC).
|
||||
OWN_CHECK = frozenset({"/anomalous/mcp"})
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
"""What a LoRA file trains, read from its safetensors header: whether it also has text-encoder
|
||||
weights, so inserting it wires the CLIP line too (graph_splice.js planLoraInsertion).
|
||||
|
||||
GET /anomalous/lora_info?name=<the loader's lora_name> -> {text_encoder}.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import struct
|
||||
|
||||
from aiohttp import web
|
||||
import folder_paths
|
||||
|
||||
# Key prefixes of text-encoder weights: kohya lora_te / lora_te1 / lora_te2, te1 / te2, diffusers text_encoder.
|
||||
TEXT_ENCODER_KEYS = ("lora_te", "te_", "te1", "te2", "text_encoder")
|
||||
MAX_HEADER_BYTES = 100 * 1024 * 1024
|
||||
|
||||
|
||||
def trains_text_encoder(path):
|
||||
"""Whether the LoRA at `path` has text-encoder weights. Other formats, or a header that
|
||||
cannot be read, count as yes: the CLIP line then goes through it, which costs nothing."""
|
||||
if not str(path).lower().endswith(".safetensors"):
|
||||
return True
|
||||
try:
|
||||
with open(path, "rb") as source:
|
||||
size = struct.unpack("<Q", source.read(8))[0]
|
||||
if size > MAX_HEADER_BYTES:
|
||||
return True
|
||||
keys = json.loads(source.read(size))
|
||||
except (OSError, ValueError, struct.error):
|
||||
return True
|
||||
return any(key.startswith(TEXT_ENCODER_KEYS) for key in keys if key != "__metadata__")
|
||||
|
||||
|
||||
async def api_lora_info(request):
|
||||
name = request.query.get("name", "")
|
||||
path = folder_paths.get_full_path("loras", name) if name else None # only names ComfyUI lists
|
||||
if not path:
|
||||
return web.json_response({"error": "Unknown LoRA"}, status=404)
|
||||
return web.json_response({"text_encoder": await asyncio.to_thread(trains_text_encoder, path)})
|
||||
@@ -0,0 +1,98 @@
|
||||
"""Where ComfyUI-Manager's model list says a model file comes from, for Model Check's downloads
|
||||
(download_sources.find_source). The list (model-list.json, a few hundred common models: base
|
||||
models, VAEs, text encoders, upscalers, ControlNets… mostly on Hugging Face) is read from the
|
||||
Manager installed here; without it there is nothing to find. A list entry is only a file of the
|
||||
same name: download_sources checks it against the workflow's fingerprint, or offers it as such.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
|
||||
import folder_paths
|
||||
|
||||
# The list's model types and save paths -> ComfyUI folder types.
|
||||
TYPE_FOLDERS = {
|
||||
"checkpoint": "checkpoints", "checkpoints": "checkpoints", "lora": "loras", "loras": "loras",
|
||||
"vae": "vae", "taesd": "vae_approx", "vae_approx": "vae_approx", "clip": "text_encoders",
|
||||
"text_encoders": "text_encoders", "clip_vision": "clip_vision", "unet": "diffusion_models",
|
||||
"diffusion_model": "diffusion_models", "diffusion_models": "diffusion_models", "controlnet": "controlnet",
|
||||
"upscale": "upscale_models", "upscale_models": "upscale_models", "embeddings": "embeddings",
|
||||
"embedding": "embeddings", "gligen": "gligen", "style_models": "style_models",
|
||||
}
|
||||
# ComfyUI's older names for the same folders.
|
||||
SAME_FOLDER = {"unet": "diffusion_models", "clip": "text_encoders"}
|
||||
|
||||
_cache = {"path": "", "mtime": 0.0, "index": {}}
|
||||
|
||||
|
||||
def list_path():
|
||||
"""The Manager's model-list.json on this computer, or ''."""
|
||||
roots = []
|
||||
try:
|
||||
roots = list(folder_paths.get_folder_paths("custom_nodes") or [])
|
||||
except Exception:
|
||||
pass
|
||||
for root in roots:
|
||||
try:
|
||||
names = sorted(os.listdir(root))
|
||||
except OSError:
|
||||
continue
|
||||
for name in names:
|
||||
path = os.path.join(root, name, "model-list.json")
|
||||
if "manager" in name.lower() and os.path.isfile(path):
|
||||
return path
|
||||
try: # the Manager installed as a Python package
|
||||
spec = importlib.util.find_spec("comfyui_manager")
|
||||
for location in (spec.submodule_search_locations or []) if spec else []:
|
||||
path = os.path.join(location, "model-list.json")
|
||||
if os.path.isfile(path):
|
||||
return path
|
||||
except (ImportError, ValueError):
|
||||
pass
|
||||
return ""
|
||||
|
||||
|
||||
def _folder(entry):
|
||||
path = str(entry.get("save_path") or "").replace("\\", "/").strip("/")
|
||||
first = path.split("/")[0].lower() if path and path != "default" else ""
|
||||
return TYPE_FOLDERS.get(first) or TYPE_FOLDERS.get(str(entry.get("type") or "").lower(), "")
|
||||
|
||||
|
||||
def _index():
|
||||
path = list_path()
|
||||
if not path:
|
||||
return {}
|
||||
try:
|
||||
mtime = os.path.getmtime(path)
|
||||
except OSError:
|
||||
return {}
|
||||
if path != _cache["path"] or mtime != _cache["mtime"]:
|
||||
index = {}
|
||||
try:
|
||||
with open(path, encoding="utf-8") as source:
|
||||
models = json.load(source).get("models") or []
|
||||
except (OSError, ValueError, AttributeError):
|
||||
models = []
|
||||
for entry in models:
|
||||
if not isinstance(entry, dict) or not entry.get("filename") or not str(entry.get("url", "")).startswith("https://"):
|
||||
continue
|
||||
index.setdefault(str(entry["filename"]).lower(), []).append({
|
||||
"name": str(entry.get("name") or entry["filename"]), "url": entry["url"],
|
||||
"folder": _folder(entry), "base": str(entry.get("base") or ""),
|
||||
})
|
||||
_cache.update(path=path, mtime=mtime, index=index)
|
||||
return _cache["index"]
|
||||
|
||||
|
||||
def candidates(file_name, folder_type=""):
|
||||
"""The list's entries for a file of this name (any case) that fit `folder_type`, one per URL."""
|
||||
wanted = SAME_FOLDER.get(folder_type, folder_type)
|
||||
seen, found = set(), []
|
||||
for entry in _index().get(os.path.basename(str(file_name)).lower(), []):
|
||||
if wanted and entry["folder"] and SAME_FOLDER.get(entry["folder"], entry["folder"]) != wanted:
|
||||
continue
|
||||
if entry["url"] not in seen:
|
||||
seen.add(entry["url"])
|
||||
found.append(entry)
|
||||
return found
|
||||
@@ -0,0 +1,305 @@
|
||||
"""The MCP tools that act: on the open canvas, through the page (mcp_bridge.py and
|
||||
web/modules/mcp_bridge.js, which use Anomalous's own canvas code, so every change is one
|
||||
Ctrl+Z step and lands in the activity log marked as the AI's), and on this ComfyUI server
|
||||
through its own routes (scans, Anomalous TTS speech), so they behave exactly as from the UI.
|
||||
|
||||
Nothing here deletes or renames files. Tools that overwrite something on the canvas say so
|
||||
(destructiveHint); AI apps ask the user before calling tools that are not read-only.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import ssl
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from .lora_info import trains_text_encoder
|
||||
from .mcp_bridge import PageRefused, PageUnavailable, ask_page
|
||||
from .mcp_tools import MAX_LIMIT, ToolError, _resolve_model, _schema
|
||||
|
||||
SPEAK_WAIT_SECONDS = 120
|
||||
HISTORY_POLL_SECONDS = 1.5
|
||||
|
||||
|
||||
def _page(action, args=None, timeout=30):
|
||||
try:
|
||||
return ask_page(action, args, timeout)
|
||||
except (PageUnavailable, PageRefused) as error:
|
||||
raise ToolError(str(error))
|
||||
|
||||
|
||||
def _rel(base_dir, file_path):
|
||||
return os.path.relpath(file_path, base_dir).replace(os.sep, "/")
|
||||
|
||||
|
||||
def _model_ref(model_id):
|
||||
"""{type, path} of a model, as a node's drop-down names it (relative to its folder)."""
|
||||
folder_type, _idx, base_dir, file_path = _resolve_model(model_id)
|
||||
return {"type": folder_type, "path": _rel(base_dir, file_path)}
|
||||
|
||||
|
||||
def _local(ctx, method, path, body=None, timeout=20):
|
||||
"""(status, JSON) of one request to this ComfyUI server's own routes."""
|
||||
data = json.dumps(body).encode("utf-8") if body is not None else None
|
||||
request = urllib.request.Request(ctx["base_url"] + path, data=data, method=method,
|
||||
headers={"Content-Type": "application/json"})
|
||||
# Loopback only (mcp_server checked the host); a self-signed certificate is the user's own.
|
||||
context = ssl._create_unverified_context() if ctx["base_url"].startswith("https:") else None
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout, context=context) as response:
|
||||
return response.status, json.loads(response.read() or b"null")
|
||||
except urllib.error.HTTPError as error:
|
||||
try:
|
||||
return error.code, json.loads(error.read() or b"null")
|
||||
except ValueError:
|
||||
return error.code, None
|
||||
except (urllib.error.URLError, OSError) as error:
|
||||
raise ToolError(f"Could not reach ComfyUI's own server: {error}")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- canvas (through the page)
|
||||
|
||||
def describe_canvas(ctx):
|
||||
return _page("describe_canvas")
|
||||
|
||||
|
||||
def set_prompt(ctx, text, role="positive", node_id=None, mode="replace"):
|
||||
if role not in ("positive", "negative") or mode not in ("replace", "append"):
|
||||
raise ToolError("role is positive or negative; mode is replace or append.")
|
||||
return _page("set_prompt", {"text": str(text), "role": role, "node_id": node_id, "mode": mode})
|
||||
|
||||
|
||||
def set_model(ctx, node_id, model_id):
|
||||
return _page("set_model", {"node_id": node_id, "model": _model_ref(model_id)})
|
||||
|
||||
|
||||
def add_lora(ctx, model_id, after_node_id=None, strength_model=1.0, strength_clip=1.0):
|
||||
folder_type, _idx, base_dir, file_path = _resolve_model(model_id)
|
||||
if folder_type != "loras":
|
||||
raise ToolError("add_lora takes a model from the loras folder.")
|
||||
return _page("add_lora", {"model": {"type": folder_type, "path": _rel(base_dir, file_path)},
|
||||
"text_encoder": trains_text_encoder(file_path), "after_node_id": after_node_id,
|
||||
"strength_model": float(strength_model), "strength_clip": float(strength_clip)})
|
||||
|
||||
|
||||
def place_combo(ctx, name):
|
||||
return _page("place_combo", {"name": str(name)})
|
||||
|
||||
|
||||
def check_workflow_models(ctx):
|
||||
return _page("check_models", timeout=90)
|
||||
|
||||
|
||||
def fix_workflow_models(ctx):
|
||||
return _page("fix_models", timeout=90)
|
||||
|
||||
|
||||
def download_missing_models(ctx, names=None):
|
||||
return _page("download_missing", {"names": [str(name) for name in names or []]}, timeout=120)
|
||||
|
||||
|
||||
def download_model(ctx, type, url="", hash="", folder=None, name=""):
|
||||
if not (url or hash):
|
||||
raise ToolError("Give url (a Civitai, Hugging Face or GitHub link to the file or model version) or hash.")
|
||||
return _page("download_model", {"url": str(url), "hash": str(hash), "type": str(type),
|
||||
"folder": None if folder is None else str(folder), "name": str(name or "")}, timeout=90)
|
||||
|
||||
|
||||
def download_status(ctx):
|
||||
status, data = _local(ctx, "GET", "/anomalous/download/status")
|
||||
if status != 200:
|
||||
raise ToolError(f"Download status unavailable ({status}).")
|
||||
jobs = []
|
||||
for job in (data or {}).get("jobs", []):
|
||||
total = job.get("total") or 0
|
||||
jobs.append({"file": job.get("rel"), "type": job.get("type"), "state": job.get("state"), "error": job.get("error") or None,
|
||||
"percent": round(job["received"] * 100 / total) if total else None, "size": total or None,
|
||||
"checked": job.get("verified"), "for_model": job.get("value") or None})
|
||||
return {"downloads": jobs, "meaning": "state: queued, running, verifying, done (checked: its fingerprint matched), "
|
||||
"failed (error says why; starting it again continues it), cancelled."}
|
||||
|
||||
|
||||
def run_workflow(ctx, batch=1):
|
||||
batch = max(1, min(8, int(batch)))
|
||||
return _page("run_workflow", {"batch": batch})
|
||||
|
||||
|
||||
def open_in_anomalous(ctx, model_id):
|
||||
folder_type, path_idx, base_dir, file_path = _resolve_model(model_id)
|
||||
return _page("open_model", {"type": folder_type, "path_idx": path_idx, "rel": _rel(base_dir, file_path)})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- this server
|
||||
|
||||
def scan_models(ctx, what="new", model_ids=None):
|
||||
body = {"offline_only": False, "skip_rename": False, "virtual_rename": True, "physical_rename": False,
|
||||
"force_overwrite": False, "retry_unmatched": what == "retry_unmatched"}
|
||||
if what == "models":
|
||||
folders = {}
|
||||
for model_id in model_ids or []:
|
||||
folder_type, path_idx, base_dir, file_path = _resolve_model(model_id)
|
||||
subfolder, _, filename = _rel(base_dir, file_path).rpartition("/")
|
||||
folders.setdefault((folder_type, path_idx, "/" + subfolder), []).append(filename)
|
||||
if not folders:
|
||||
raise ToolError("what=models needs model_ids from search_models.")
|
||||
body["targets"] = [{"type": t, "path_idx": i, "subfolder": s, "files": files} for (t, i, s), files in folders.items()]
|
||||
elif what not in ("new", "retry_unmatched"):
|
||||
raise ToolError("what is new, retry_unmatched or models.")
|
||||
status, data = _local(ctx, "POST", "/anomalous/scan_all", body)
|
||||
if status == 409:
|
||||
raise ToolError("A scan is already running; call scan_report later to see it.")
|
||||
if status != 200 or (data or {}).get("status") != "ok":
|
||||
raise ToolError(f"The scan did not start: {(data or {}).get('message') or status}")
|
||||
return {"started": True, "note": "Scans run in the background and only read .safetensors files. "
|
||||
"Call scan_report in a minute or two for what it found."}
|
||||
|
||||
|
||||
def _tts_characters(ctx):
|
||||
status, data = _local(ctx, "GET", "/anomalous_tts/characters")
|
||||
if status == 404:
|
||||
raise ToolError("Anomalous TTS is not installed, so there are no voices.")
|
||||
if status != 200 or not isinstance(data, dict):
|
||||
raise ToolError(f"Could not read the voices ({status}).")
|
||||
return data.get("characters") or []
|
||||
|
||||
|
||||
def list_voices(ctx):
|
||||
return {"voices": [{"character": item.get("name"), "language": item.get("language") or "",
|
||||
"emotions": sorted((item.get("emotions") or {}).keys()),
|
||||
**({"problem": item.get("error") or item.get("settings_error")}
|
||||
if item.get("error") or item.get("settings_error") else {})}
|
||||
for item in _tts_characters(ctx)]}
|
||||
|
||||
|
||||
def _output_prefix(character):
|
||||
"""Where the audio page saves a character's lines (audio_script.js ttsOutputPrefix)."""
|
||||
parts = [re.sub(r'[<>:"\\|?*\x00-\x1f]', "_", part).strip() or "_" for part in str(character).split("/")]
|
||||
parts = ["_" if re.fullmatch(r"\.+", part) else part for part in parts]
|
||||
return f"audio/{'/'.join(parts)}/{parts[-1]}"
|
||||
|
||||
|
||||
def speak(ctx, character, text, seed=None, language="auto"):
|
||||
names = {item.get("name") for item in _tts_characters(ctx)}
|
||||
if character not in names:
|
||||
raise ToolError(f"No voice called {character!r}; list_voices names them.")
|
||||
if not str(text).strip():
|
||||
raise ToolError("text is empty.")
|
||||
seed = int(seed) if seed is not None else int(time.time() * 1000) % 2**31
|
||||
prompt = {"1": {"class_type": "AnomalousTTS_CharacterSpeech",
|
||||
"inputs": {"character": character, "text": str(text), "seed": seed, "language": language}},
|
||||
"2": {"class_type": "SaveAudio", "inputs": {"audio": ["1", 0], "filename_prefix": _output_prefix(character)}}}
|
||||
status, data = _local(ctx, "POST", "/prompt", {"prompt": prompt, "client_id": "anomalous-mcp"})
|
||||
prompt_id = (data or {}).get("prompt_id")
|
||||
if status != 200 or not prompt_id:
|
||||
errors = (data or {}).get("node_errors") or (data or {}).get("error") or status
|
||||
raise ToolError(f"ComfyUI refused the speech job: {json.dumps(errors, ensure_ascii=False)[:500]}")
|
||||
deadline = time.monotonic() + SPEAK_WAIT_SECONDS
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(HISTORY_POLL_SECONDS)
|
||||
_status, history = _local(ctx, "GET", f"/history/{prompt_id}")
|
||||
record = (history or {}).get(prompt_id) or {}
|
||||
state = record.get("status") or {}
|
||||
if state.get("status_str") == "error":
|
||||
messages = [m[1].get("exception_message") for m in state.get("messages") or [] if m and m[0] == "execution_error"]
|
||||
raise ToolError(f"Speech failed: {messages[0] if messages else 'see the ComfyUI console'}")
|
||||
if state.get("completed"):
|
||||
files = [f"{item.get('subfolder')}/{item.get('filename')}".replace("\\", "/").strip("/")
|
||||
for output in (record.get("outputs") or {}).values() for item in output.get("audio") or []]
|
||||
return {"done": True, "seed": seed, "files": files,
|
||||
"note": "Saved under ComfyUI's output folder; it shows in Anomalous's audio gallery."}
|
||||
return {"done": False, "prompt_id": prompt_id, "seed": seed,
|
||||
"note": "Still generating (ComfyUI may be busy with other jobs); search_audio will list it when done."}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- registry
|
||||
|
||||
_NODE = {"type": "integer", "description": "A node id from describe_canvas."}
|
||||
_MODEL = {"type": "string", "description": "model_id from search_models."}
|
||||
|
||||
|
||||
def _action(name, title, description, run, properties=None, required=(), destructive=False, open_world=False,
|
||||
read_only=False):
|
||||
return name, {"run": run, "context": True, "spec": {
|
||||
"name": name, "title": title, "description": description,
|
||||
"inputSchema": _schema(properties, required),
|
||||
"annotations": {"title": title, "readOnlyHint": read_only, "destructiveHint": destructive,
|
||||
"idempotentHint": read_only, "openWorldHint": open_world},
|
||||
}}
|
||||
|
||||
|
||||
ACTIONS = dict([
|
||||
_action("describe_canvas", "Read the canvas",
|
||||
"The workflow open in ComfyUI now: each node's id, type, title, settings, which prompt box is "
|
||||
"positive or negative, and where its inputs come from. Call it before changing the canvas.",
|
||||
describe_canvas, read_only=True),
|
||||
_action("set_prompt", "Write a prompt",
|
||||
"Writes text into the workflow's positive or negative prompt box (or the one on node_id). "
|
||||
"mode=append adds to what is there. One Ctrl+Z step.",
|
||||
set_prompt, {"text": {"type": "string"}, "role": {"type": "string", "enum": ["positive", "negative"], "default": "positive"},
|
||||
"node_id": _NODE, "mode": {"type": "string", "enum": ["replace", "append"], "default": "replace"}},
|
||||
["text"], destructive=True),
|
||||
_action("set_model", "Swap a node's model",
|
||||
"Puts a model into a loader node (checkpoint, LoRA, VAE…) on the canvas, replacing the one it had.",
|
||||
set_model, {"node_id": _NODE, "model_id": _MODEL}, ["node_id", "model_id"], destructive=True),
|
||||
_action("add_lora", "Add a LoRA",
|
||||
"Inserts a LoRA loader after node after_node_id (default: the workflow's main model loader, past "
|
||||
"the LoRAs already on it) and moves everything that node fed behind it. LoRAs that train the text "
|
||||
"encoder also take the CLIP line (from the same node, or a UNet workflow's CLIP loader); others "
|
||||
"patch the model only.",
|
||||
add_lora, {"model_id": _MODEL, "after_node_id": _NODE,
|
||||
"strength_model": {"type": "number", "default": 1.0}, "strength_clip": {"type": "number", "default": 1.0}},
|
||||
["model_id"]),
|
||||
_action("place_combo", "Place a combo",
|
||||
"Adds one of the user's combos (list_combos) to the canvas as a new wired group: model loader, LoRAs, "
|
||||
"positive and negative prompt, or a node-structure combo's own nodes and links. Nothing already there "
|
||||
"changes; ports to nodes outside the combo (sampler…) stay open.",
|
||||
place_combo, {"name": {"type": "string", "description": "Combo name from list_combos."}}, ["name"]),
|
||||
_action("check_workflow_models", "Check missing models",
|
||||
"Which models the open workflow needs that this computer does not have under that name, and which "
|
||||
"of them Anomalous can put back (same file found by hash, or a likely candidate).",
|
||||
check_workflow_models, read_only=True),
|
||||
_action("fix_workflow_models", "Fix missing models",
|
||||
"Puts back every missing model that Anomalous found with certainty (same file by hash). "
|
||||
"Likely candidates are left for the user. One Ctrl+Z step.",
|
||||
fix_workflow_models, destructive=True),
|
||||
_action("download_missing_models", "Download missing models",
|
||||
"Downloads the models the open workflow is missing whose source is certain: Civitai found by the "
|
||||
"workflow's fingerprint, or a file link the workflow carries. A file name alone is never used. Files go "
|
||||
"where the user's download settings say and are checked against the fingerprint before use; finished "
|
||||
"ones are put into their nodes. names limits it to those files. Large files take a while.",
|
||||
download_missing_models, {"names": {"type": "array", "items": {"type": "string"}, "maxItems": MAX_LIMIT,
|
||||
"description": "Only these model file names (default: all missing)."}},
|
||||
open_world=True),
|
||||
_action("download_model", "Download a model",
|
||||
"Downloads one model into this computer's models folder of `type` (loras, checkpoints, vae, "
|
||||
"diffusion_models, text_encoders, upscale_models…): from url (a Civitai model-version link, or a Hugging "
|
||||
"Face / GitHub link to the file) or by its Civitai hash. folder is a subfolder inside that type's folder "
|
||||
"(default: the user's download settings); name the file name (default: the site's). Never replaces a file.",
|
||||
download_model, {"type": {"type": "string"}, "url": {"type": "string"}, "hash": {"type": "string"},
|
||||
"folder": {"type": "string"}, "name": {"type": "string"}}, ["type"], open_world=True),
|
||||
_action("download_status", "Downloads", "How the downloads of this ComfyUI run are going.", download_status,
|
||||
read_only=True),
|
||||
_action("run_workflow", "Run the workflow",
|
||||
"Queues the open workflow in ComfyUI, like pressing Run. Uses the GPU; results appear in the output folder.",
|
||||
run_workflow, {"batch": {"type": "integer", "minimum": 1, "maximum": 8, "default": 1}}),
|
||||
_action("open_in_anomalous", "Show a model",
|
||||
"Opens Anomalous Model Browser on this model's page, so the user sees its cover and details.",
|
||||
open_in_anomalous, {"model_id": _MODEL}, ["model_id"]),
|
||||
_action("scan_models", "Scan models",
|
||||
"Starts a scan that fetches covers, trigger words and base models from Civitai (by file hash) for "
|
||||
"models not scanned yet (new), looks unmatched ones up again (retry_unmatched), or scans model_ids.",
|
||||
scan_models, {"what": {"type": "string", "enum": ["new", "retry_unmatched", "models"], "default": "new"},
|
||||
"model_ids": {"type": "array", "items": {"type": "string"}, "maxItems": MAX_LIMIT}},
|
||||
open_world=True),
|
||||
_action("list_voices", "Voices",
|
||||
"The Anomalous TTS characters (GPT-SoVITS voices) with their language and emotions.", list_voices,
|
||||
read_only=True),
|
||||
_action("speak", "Speak a line",
|
||||
"Has an Anomalous TTS character read text and saves the audio. Emotion tags like {happy} switch "
|
||||
"emotion mid-text; waits up to two minutes.",
|
||||
speak, {"character": {"type": "string", "description": "From list_voices."}, "text": {"type": "string"},
|
||||
"seed": {"type": "integer"}, "language": {"type": "string", "default": "auto"}},
|
||||
["character", "text"]),
|
||||
])
|
||||
@@ -0,0 +1,78 @@
|
||||
"""The way from an MCP tool to the open ComfyUI page, for what only the page can do (the
|
||||
canvas lives in the browser).
|
||||
|
||||
`ask_page` sends {id, action, args} as the "anomalous.mcp" event over ComfyUI's websocket
|
||||
to every open ComfyUI page and waits. One page claims it (the first claim wins, visible
|
||||
pages claim first: web/modules/mcp_bridge.js), does the action with Anomalous's own canvas
|
||||
code and posts the outcome back; `claim` and `resolve` take those posts
|
||||
(routes in mcp_server.py). With no page open, or none answering in time, the tool fails
|
||||
with words the AI can pass on.
|
||||
"""
|
||||
|
||||
import threading
|
||||
import uuid
|
||||
|
||||
TIMEOUT_SECONDS = 30
|
||||
MAX_WAITING = 20
|
||||
|
||||
_lock = threading.Lock()
|
||||
_waiting = {} # id -> {"event": Event, "claimed": bool, "outcome": dict | None}
|
||||
|
||||
|
||||
class PageUnavailable(Exception):
|
||||
"""No ComfyUI page took or finished the action."""
|
||||
|
||||
|
||||
class PageRefused(Exception):
|
||||
"""The page tried and said why it could not (missing node, wrong model type…)."""
|
||||
|
||||
|
||||
def _server():
|
||||
from server import PromptServer # ComfyUI's; imported late so tests can run without it
|
||||
return PromptServer.instance
|
||||
|
||||
|
||||
def ask_page(action, args=None, timeout=TIMEOUT_SECONDS):
|
||||
"""Runs `action` in the open ComfyUI page; its result value. Call from a worker thread."""
|
||||
server = _server()
|
||||
if not getattr(server, "sockets", None):
|
||||
raise PageUnavailable("ComfyUI is not open in a browser on this computer. Open it, then try again.")
|
||||
request_id = uuid.uuid4().hex
|
||||
entry = {"event": threading.Event(), "claimed": False, "outcome": None}
|
||||
with _lock:
|
||||
if len(_waiting) >= MAX_WAITING:
|
||||
raise PageUnavailable("Too many canvas actions are waiting; try again in a moment.")
|
||||
_waiting[request_id] = entry
|
||||
try:
|
||||
server.send_sync("anomalous.mcp", {"id": request_id, "action": action, "args": args or {}})
|
||||
if not entry["event"].wait(timeout):
|
||||
raise PageUnavailable("The ComfyUI page did not answer in time. Is it open and not frozen? "
|
||||
"If the action was taken late, check the canvas before repeating it.")
|
||||
finally:
|
||||
with _lock:
|
||||
_waiting.pop(request_id, None)
|
||||
outcome = entry["outcome"] or {}
|
||||
if not outcome.get("ok"):
|
||||
raise PageRefused(str(outcome.get("error") or "The page could not do that."))
|
||||
return outcome.get("value")
|
||||
|
||||
|
||||
def claim(request_id):
|
||||
"""True for the first page that asks to do request `request_id`."""
|
||||
with _lock:
|
||||
entry = _waiting.get(request_id)
|
||||
if entry is None or entry["claimed"]:
|
||||
return False
|
||||
entry["claimed"] = True
|
||||
return True
|
||||
|
||||
|
||||
def resolve(request_id, outcome):
|
||||
"""The claiming page's outcome: {ok, value} or {ok: false, error}. False when nobody waits."""
|
||||
with _lock:
|
||||
entry = _waiting.get(request_id)
|
||||
if entry is None or not entry["claimed"] or entry["outcome"] is not None:
|
||||
return False
|
||||
entry["outcome"] = outcome if isinstance(outcome, dict) else {"ok": False, "error": "Malformed answer"}
|
||||
entry["event"].set()
|
||||
return True
|
||||
@@ -0,0 +1,201 @@
|
||||
"""MCP endpoint: AI apps (Claude, Cursor, Cherry Studio…) use Anomalous's library as tools.
|
||||
|
||||
POST /anomalous/mcp speaks the Model Context Protocol's Streamable HTTP transport, both eras:
|
||||
- modern (2026-07-28): stateless; every request carries its protocol version in `_meta` and
|
||||
the MCP-Protocol-Version / Mcp-Method / Mcp-Name headers, which must match the body;
|
||||
- legacy (2025-03-26 to 2025-11-25): an `initialize` handshake first. No session is kept
|
||||
(sessions are optional there), so both eras are served the same stateless way.
|
||||
Only this computer may connect: the peer must be loopback, no proxy header may be present,
|
||||
and Host and Origin (when sent) must name a loopback host, which stops DNS rebinding.
|
||||
Answers are always one JSON object; there are no server-initiated messages. The tools are
|
||||
in mcp_tools.py (reading) and mcp_actions.py (acting); POST /anomalous/mcp/bridge is where
|
||||
the open ComfyUI page claims and answers canvas actions (mcp_bridge.py).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import urllib.parse
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from . import mcp_bridge
|
||||
from .local_guard import is_local as _is_local, loopback_host as _loopback_host
|
||||
from .mcp_actions import ACTIONS
|
||||
from .mcp_tools import TOOLS as READ_TOOLS, call_tool
|
||||
|
||||
TOOLS = {**READ_TOOLS, **ACTIONS}
|
||||
|
||||
MODERN_VERSIONS = ("2026-07-28",)
|
||||
LEGACY_VERSIONS = ("2025-11-25", "2025-06-18", "2025-03-26")
|
||||
SERVER_INFO = {"name": "anomalous-model-browser", "title": "Anomalous Model Browser", "version": "1.0.0"}
|
||||
INSTRUCTIONS = (
|
||||
"Tools over the user's ComfyUI library as Anomalous Model Browser keeps it: model files with "
|
||||
"their Civitai information, output images with their generation settings, combos (a model, "
|
||||
"LoRAs and a prompt), workflow recipes, saved prompts, generated audio, the scan status and "
|
||||
"the activity log; and tools that act on the workflow open in ComfyUI (read it with "
|
||||
"describe_canvas first), scan models, or have an Anomalous TTS voice speak. Canvas changes "
|
||||
"are single Ctrl+Z steps and are logged as the AI's; nothing deletes files. Ids returned by "
|
||||
"one tool (model_id, image, node ids) are what the others take. For what no tool does, tell "
|
||||
"the user where in Anomalous to do it (the rail on the left: Models, Gallery, Workflows, "
|
||||
"Combos, Prompts, Voices)."
|
||||
)
|
||||
LIST_TTL_MS = 300000
|
||||
# Requests run tools in threads; a few at a time is plenty for one person's AI app.
|
||||
_slots = asyncio.Semaphore(4)
|
||||
|
||||
|
||||
PARSE_ERROR, INVALID_REQUEST, METHOD_NOT_FOUND, INVALID_PARAMS = -32700, -32600, -32601, -32602
|
||||
HEADER_MISMATCH, UNSUPPORTED_VERSION = -32020, -32022
|
||||
|
||||
|
||||
def _origin_ok(request):
|
||||
origin = request.headers.get("Origin")
|
||||
return origin is None or _loopback_host(urllib.parse.urlsplit(origin).hostname)
|
||||
|
||||
|
||||
def _decode_header(value):
|
||||
"""Mcp-Name values outside plain ASCII come as =?base64?…?=."""
|
||||
if value and value.startswith("=?base64?") and value.endswith("?="):
|
||||
try:
|
||||
return base64.b64decode(value[9:-2], validate=True).decode("utf-8")
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
return None
|
||||
return value
|
||||
|
||||
|
||||
def _error(id_, code, message, status=200, data=None):
|
||||
error = {"code": code, "message": message}
|
||||
if data is not None:
|
||||
error["data"] = data
|
||||
body = {"jsonrpc": "2.0", "error": error}
|
||||
if id_ is not None:
|
||||
body["id"] = id_
|
||||
return web.json_response(body, status=status)
|
||||
|
||||
|
||||
def _result(id_, result, modern):
|
||||
if modern:
|
||||
result = {"resultType": "complete", **result}
|
||||
return web.json_response({"jsonrpc": "2.0", "id": id_, "result": result})
|
||||
|
||||
|
||||
def _tool_list(modern):
|
||||
result = {"tools": [tool["spec"] for tool in TOOLS.values()]}
|
||||
if modern:
|
||||
result.update(ttlMs=LIST_TTL_MS, cacheScope="public")
|
||||
return result
|
||||
|
||||
|
||||
def _initialize(params):
|
||||
requested = params.get("protocolVersion")
|
||||
return {
|
||||
"protocolVersion": requested if requested in LEGACY_VERSIONS else LEGACY_VERSIONS[0],
|
||||
"capabilities": {"tools": {"listChanged": False}},
|
||||
"serverInfo": SERVER_INFO,
|
||||
"instructions": INSTRUCTIONS,
|
||||
}
|
||||
|
||||
|
||||
def _discover():
|
||||
return {
|
||||
"supportedVersions": list(MODERN_VERSIONS + LEGACY_VERSIONS),
|
||||
"capabilities": {"tools": {"listChanged": False}},
|
||||
"_meta": {"io.modelcontextprotocol/serverInfo": SERVER_INFO},
|
||||
"instructions": INSTRUCTIONS,
|
||||
"ttlMs": LIST_TTL_MS,
|
||||
"cacheScope": "public",
|
||||
}
|
||||
|
||||
|
||||
def _modern_header_problem(request, method, params, version):
|
||||
"""Why the mirrored headers do not match the body (2026-07-28), or None."""
|
||||
if request.headers.get("MCP-Protocol-Version") != version:
|
||||
return "MCP-Protocol-Version header missing or not the body's protocol version"
|
||||
if request.headers.get("Mcp-Method") != method:
|
||||
return "Mcp-Method header missing or not the body's method"
|
||||
if method in ("tools/call", "prompts/get", "resources/read"):
|
||||
name = params.get("uri") if method == "resources/read" else params.get("name")
|
||||
if _decode_header(request.headers.get("Mcp-Name")) != name:
|
||||
return "Mcp-Name header missing or not the body's name"
|
||||
return None
|
||||
|
||||
|
||||
async def _call(params, request):
|
||||
name = params.get("name")
|
||||
arguments = params.get("arguments") or {}
|
||||
if name not in TOOLS or not isinstance(arguments, dict):
|
||||
return None
|
||||
ctx = {"base_url": f"{request.scheme}://{request.host}"} # loopback, checked in api_mcp
|
||||
async with _slots:
|
||||
result = await asyncio.to_thread(call_tool, TOOLS, name, arguments, ctx)
|
||||
return result
|
||||
|
||||
|
||||
async def api_mcp(request):
|
||||
"""POST /anomalous/mcp - one JSON-RPC request or notification per POST."""
|
||||
if request.method != "POST":
|
||||
return web.Response(status=405, headers={"Allow": "POST"})
|
||||
if not _is_local(request) or not _origin_ok(request):
|
||||
return _error(None, INVALID_REQUEST, "Only this computer may use the Anomalous MCP endpoint.", status=403)
|
||||
try:
|
||||
body = await request.json()
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
return _error(None, PARSE_ERROR, "Body is not JSON", status=400)
|
||||
if not isinstance(body, dict) or body.get("jsonrpc") != "2.0" or not isinstance(body.get("method"), str):
|
||||
return _error(None, INVALID_REQUEST, "Expected one JSON-RPC 2.0 request (batches are not supported)", status=400)
|
||||
if "id" not in body:
|
||||
return web.Response(status=202) # notifications (initialized, cancelled) need no answer
|
||||
id_, method = body["id"], body["method"]
|
||||
params = body.get("params") if isinstance(body.get("params"), dict) else {}
|
||||
|
||||
if method == "initialize":
|
||||
return _result(id_, _initialize(params), modern=False)
|
||||
meta = params.get("_meta") if isinstance(params.get("_meta"), dict) else {}
|
||||
version = meta.get("io.modelcontextprotocol/protocolVersion")
|
||||
modern = version is not None
|
||||
if modern:
|
||||
if version not in MODERN_VERSIONS:
|
||||
return _error(id_, UNSUPPORTED_VERSION, "Unsupported protocol version", status=400,
|
||||
data={"supported": list(MODERN_VERSIONS + LEGACY_VERSIONS), "requested": version})
|
||||
problem = _modern_header_problem(request, method, params, version)
|
||||
if problem:
|
||||
return _error(id_, HEADER_MISMATCH, f"Header mismatch: {problem}", status=400)
|
||||
else:
|
||||
header = request.headers.get("MCP-Protocol-Version")
|
||||
if header is not None and header not in LEGACY_VERSIONS:
|
||||
return _error(id_, INVALID_REQUEST, f"Unsupported protocol version {header}", status=400,
|
||||
data={"supported": list(MODERN_VERSIONS + LEGACY_VERSIONS), "requested": header})
|
||||
|
||||
if method == "server/discover" and modern:
|
||||
return _result(id_, _discover(), modern)
|
||||
if method == "ping":
|
||||
return _result(id_, {}, modern)
|
||||
if method == "tools/list":
|
||||
return _result(id_, _tool_list(modern), modern)
|
||||
if method == "tools/call":
|
||||
result = await _call(params, request)
|
||||
if result is None:
|
||||
return _error(id_, INVALID_PARAMS, f"Unknown tool or bad arguments: {params.get('name')}")
|
||||
return _result(id_, result, modern)
|
||||
return _error(id_, METHOD_NOT_FOUND, f"Method not found: {method}", status=404 if modern else 200)
|
||||
|
||||
|
||||
async def api_mcp_bridge(request):
|
||||
"""POST /anomalous/mcp/bridge - the open page claims a canvas action ({id, claim: true} ->
|
||||
{granted}) and then answers it ({id, outcome: {ok, value} | {ok: false, error}})."""
|
||||
if not _is_local(request) or not _origin_ok(request):
|
||||
return web.json_response({"error": "This computer only"}, status=403)
|
||||
try:
|
||||
body = await request.json()
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
body = None
|
||||
if not isinstance(body, dict) or not isinstance(body.get("id"), str):
|
||||
return web.json_response({"error": "Expected {id, claim} or {id, outcome}"}, status=400)
|
||||
if body.get("claim") is True:
|
||||
return web.json_response({"granted": mcp_bridge.claim(body["id"])})
|
||||
return web.json_response({"accepted": mcp_bridge.resolve(body["id"], body.get("outcome"))})
|
||||
|
||||
|
||||
def register_routes(app):
|
||||
app.router.add_route("*", "/anomalous/mcp", api_mcp)
|
||||
app.router.add_post("/anomalous/mcp/bridge", api_mcp_bridge)
|
||||
@@ -0,0 +1,492 @@
|
||||
"""The MCP endpoint's (mcp_server.py) reading tools: views of the library (mcp_actions.py acts).
|
||||
|
||||
`call_tool` runs any tool of a registry, these and mcp_actions.py's.
|
||||
|
||||
Each tool is a plain function over the same stores the browser's pages use (model catalog
|
||||
and sidecars, output gallery, combos, workflow recipes, saved prompts, generated audio, scan
|
||||
results, activity log). They never write. Results are small, already-summarised JSON (an AI
|
||||
reads them), with ids the detail tools take back: model_id "type:path_idx:relative/path" and
|
||||
image "subfolder/file.png" under ComfyUI's output folder. A tool's mistake (bad id, missing
|
||||
file) comes back as a tool error the AI can read and correct, never as an exception.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import datetime
|
||||
import html
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
|
||||
import folder_paths
|
||||
|
||||
from .activity_log import list_entries
|
||||
from .audio_catalog import _gallery_page as _audio_page
|
||||
from .gallery_routes import _gallery_listing
|
||||
from .image_search import read_png_text
|
||||
from .kept_images import _kept_images
|
||||
from .material_store import _list_materials, _read_material, get_materials_dir
|
||||
from .media_routes import _build_card_thumbnail
|
||||
from .metadata import get_metadata
|
||||
from .model_catalog import _iter_search_models
|
||||
from .model_constants import MEDIA_EXTENSIONS, MODEL_EXTENSIONS, PREVIEW_SUFFIXES
|
||||
from .notebooks import _list_notebooks
|
||||
from .path_utils import resolve_folder_subdir, resolve_within
|
||||
from .recipe_store import _list_recipes, get_recipes_dir
|
||||
from .scan_report import last_scan_path
|
||||
from .scan_summary import _summarize
|
||||
|
||||
MAX_LIMIT = 50
|
||||
MAX_TEXT = 1500
|
||||
MAX_IMAGE_BYTES = 1_500_000
|
||||
IMAGE_TYPES = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".webp": "image/webp"}
|
||||
SOURCES = {"civitai": "civitai", "local": "inferred_from_file", "": "not_scanned"}
|
||||
|
||||
|
||||
class ToolError(Exception):
|
||||
"""A problem the AI can fix (bad id, nothing there); shown to it as the tool's result."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- helpers
|
||||
|
||||
def _text(value, limit=MAX_TEXT):
|
||||
"""Plain text from HTML-ish Civitai descriptions, shortened."""
|
||||
value = html.unescape(re.sub(r"<[^>]+>", " ", str(value or "")))
|
||||
value = re.sub(r"\s+", " ", value).strip()
|
||||
return value if len(value) <= limit else value[:limit] + "…"
|
||||
|
||||
|
||||
def _when(seconds):
|
||||
try:
|
||||
return datetime.datetime.fromtimestamp(float(seconds)).isoformat(timespec="seconds")
|
||||
except (TypeError, ValueError, OverflowError, OSError):
|
||||
return None
|
||||
|
||||
|
||||
def _limit(value, default=20):
|
||||
try:
|
||||
return max(1, min(MAX_LIMIT, int(value)))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def _terms(query):
|
||||
return [term for term in str(query or "").lower().split() if term]
|
||||
|
||||
|
||||
def _matches(terms, *texts):
|
||||
haystack = " ".join(str(text or "") for text in texts).lower()
|
||||
return all(term in haystack for term in terms)
|
||||
|
||||
|
||||
def _model_id(folder_type, path_idx, base_dir, file_path):
|
||||
return f"{folder_type}:{path_idx}:{os.path.relpath(file_path, base_dir).replace(os.sep, '/')}"
|
||||
|
||||
|
||||
def _resolve_model(model_id):
|
||||
"""(type, path_idx, base_dir, file_path) for a model_id from search_models."""
|
||||
parts = str(model_id or "").split(":", 2)
|
||||
if len(parts) != 3:
|
||||
raise ToolError("model_id looks like 'loras:0:folder/file.safetensors'; take it from search_models.")
|
||||
folder_type, path_idx, rel = parts
|
||||
rel = rel.replace("\\", "/").strip("/")
|
||||
subfolder, filename = os.path.split(rel)
|
||||
try:
|
||||
path_idx = int(path_idx)
|
||||
base_dir, target_dir = resolve_folder_subdir(folder_type, path_idx, "/" + subfolder)
|
||||
file_path = resolve_within(target_dir, filename)
|
||||
except (ValueError, KeyError, TypeError):
|
||||
raise ToolError(f"No models folder for {model_id}.")
|
||||
if not filename.lower().endswith(MODEL_EXTENSIONS) or not os.path.isfile(file_path):
|
||||
raise ToolError(f"No model file at {model_id}; it may have been moved or renamed.")
|
||||
return folder_type, path_idx, base_dir, file_path
|
||||
|
||||
|
||||
def _resolve_image(image):
|
||||
"""The output image at `image` ('subfolder/file.png'), checked to stay inside the output folder."""
|
||||
rel = str(image or "").replace("\\", "/").strip("/")
|
||||
try:
|
||||
path = resolve_within(folder_paths.get_output_directory(), rel)
|
||||
except ValueError:
|
||||
raise ToolError("image is a path under ComfyUI's output folder, as search_images returns it.")
|
||||
if os.path.splitext(path)[1].lower() not in IMAGE_TYPES or not os.path.isfile(path):
|
||||
raise ToolError(f"No output image at {image}.")
|
||||
return rel, path
|
||||
|
||||
|
||||
def _image_content(path):
|
||||
"""An MCP image item: the 512 px thumbnail the gallery uses, or None when too big or unreadable."""
|
||||
try:
|
||||
thumb = _build_card_thumbnail(path)
|
||||
with open(thumb, "rb") as source:
|
||||
data = source.read(MAX_IMAGE_BYTES + 1)
|
||||
except OSError:
|
||||
return None
|
||||
if len(data) > MAX_IMAGE_BYTES:
|
||||
return None
|
||||
mime = IMAGE_TYPES.get(os.path.splitext(thumb)[1].lower(), "image/webp")
|
||||
return {"type": "image", "data": base64.b64encode(data).decode("ascii"), "mimeType": mime}
|
||||
|
||||
|
||||
def _kept_lookup():
|
||||
return {(item["subfolder"], item["filename"]): {kind: (value or {}).get("name") for kind, value in item.items()
|
||||
if kind in ("recipe", "combo", "prompt") and value}
|
||||
for item in _kept_images()}
|
||||
|
||||
|
||||
def _model_brief(folder_type, path_idx, base_dir, file_path, meta):
|
||||
rel_dir = os.path.relpath(os.path.dirname(file_path), base_dir).replace(os.sep, "/")
|
||||
try:
|
||||
size_mb = round(os.path.getsize(file_path) / 1048576, 1)
|
||||
except OSError:
|
||||
size_mb = None
|
||||
return {
|
||||
"model_id": _model_id(folder_type, path_idx, base_dir, file_path),
|
||||
"name": meta.get("custom_name") or meta.get("name") or os.path.basename(file_path),
|
||||
"file": os.path.basename(file_path),
|
||||
"type": folder_type,
|
||||
"folder": "" if rel_dir == "." else rel_dir,
|
||||
"base_model": meta.get("baseModel") or "",
|
||||
"source": SOURCES.get(meta.get("info_source", ""), "not_scanned"),
|
||||
"trigger_words": list(meta.get("trainedWords") or [])[:8],
|
||||
"size_mb": size_mb,
|
||||
}
|
||||
|
||||
|
||||
def _each_model():
|
||||
"""Every model file once (folders registered twice, like unet and diffusion_models, count once).
|
||||
ComfyUI also registers custom_nodes, whose node packs carry files of their own: not the library."""
|
||||
seen = set()
|
||||
for folder_type, path_idx, base_dir, file_path in _iter_search_models():
|
||||
if folder_type == "custom_nodes" or not file_path.lower().endswith(MODEL_EXTENSIONS):
|
||||
continue
|
||||
real = os.path.normcase(os.path.realpath(file_path))
|
||||
if real in seen:
|
||||
continue
|
||||
seen.add(real)
|
||||
yield folder_type, path_idx, base_dir, file_path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- models and scans
|
||||
|
||||
def search_models(query="", type="", base_model="", source="", limit=20):
|
||||
terms, wanted_base, found = _terms(query), str(base_model or "").lower(), []
|
||||
total = 0
|
||||
for folder_type, path_idx, base_dir, file_path in _each_model():
|
||||
if type and folder_type != type:
|
||||
continue
|
||||
meta = get_metadata(file_path)
|
||||
if wanted_base and wanted_base not in str(meta.get("baseModel") or "").lower():
|
||||
continue
|
||||
if source and SOURCES.get(meta.get("info_source", ""), "not_scanned") != source:
|
||||
continue
|
||||
if not _matches(terms, os.path.relpath(file_path, base_dir), meta.get("name"), meta.get("custom_name"),
|
||||
meta.get("baseModel"), " ".join(meta.get("trainedWords") or [])):
|
||||
continue
|
||||
total += 1
|
||||
if len(found) < _limit(limit):
|
||||
found.append(_model_brief(folder_type, path_idx, base_dir, file_path, meta))
|
||||
return {"total_matches": total, "models": found}
|
||||
|
||||
|
||||
def get_model(model_id, include_cover=False):
|
||||
folder_type, path_idx, base_dir, file_path = _resolve_model(model_id)
|
||||
meta = get_metadata(file_path)
|
||||
detail = _model_brief(folder_type, path_idx, base_dir, file_path, meta)
|
||||
detail.update({
|
||||
"trigger_words": list(meta.get("trainedWords") or []),
|
||||
"civitai_name": meta.get("name") or "",
|
||||
"civitai_url": meta.get("civitai_url") or "",
|
||||
"download_link": meta.get("source_url") or "",
|
||||
"user_notes": meta.get("custom_notes") or "",
|
||||
"fields_set_by_user": list(meta.get("user_fields") or []),
|
||||
"description": _text(meta.get("description")),
|
||||
"sha256": meta.get("hash") or "",
|
||||
})
|
||||
if meta.get("info_source") == "local":
|
||||
detail["why_not_on_civitai"] = meta.get("unmatched_reason") or "unknown"
|
||||
extra = []
|
||||
if include_cover:
|
||||
base = os.path.splitext(file_path)[0]
|
||||
cover = next((base + suffix for suffix in PREVIEW_SUFFIXES + MEDIA_EXTENSIONS
|
||||
if os.path.splitext(suffix)[1].lower() in IMAGE_TYPES and os.path.isfile(base + suffix)), None)
|
||||
item = _image_content(cover) if cover else None
|
||||
detail["cover"] = "attached" if item else "none"
|
||||
extra = [item] if item else []
|
||||
return detail, extra
|
||||
|
||||
|
||||
def library_overview():
|
||||
by_type, by_base = {}, {}
|
||||
for folder_type, _idx, _base, file_path in _each_model():
|
||||
by_type[folder_type] = by_type.get(folder_type, 0) + 1
|
||||
base = get_metadata(file_path).get("baseModel") or "unknown"
|
||||
by_base[base] = by_base.get(base, 0) + 1
|
||||
summary = _summarize()
|
||||
return {
|
||||
"models_by_type": dict(sorted(by_type.items(), key=lambda item: -item[1])),
|
||||
"models_by_base_model": dict(sorted(by_base.items(), key=lambda item: -item[1])[:20]),
|
||||
"scan": {key: summary[key] for key in ("total", "matched", "unmatched", "new", "pending")},
|
||||
"combos": len(_list_notebooks()),
|
||||
"workflow_recipes": len(_list_recipes(get_recipes_dir())),
|
||||
"saved_prompts": sum(1 for item in _list_materials(get_materials_dir()) if item.get("kind") == "prompt_plan"),
|
||||
}
|
||||
|
||||
|
||||
def scan_report():
|
||||
summary = _summarize()
|
||||
try:
|
||||
with open(last_scan_path(), encoding="utf-8") as source:
|
||||
last = json.load(source)
|
||||
except (OSError, ValueError):
|
||||
last = None
|
||||
report = {
|
||||
"counts": {key: summary[key] for key in ("total", "matched", "unmatched", "new", "pending")},
|
||||
"meaning": "matched: found on Civitai; unmatched: base model inferred from the file (reason says why); "
|
||||
"new: never scanned; pending: unmatched only because Civitai was not asked (offline or no connection).",
|
||||
"unmatched_models": summary["unmatched_models"][:MAX_LIMIT],
|
||||
"not_scanned_models": summary["new_models"][:MAX_LIMIT],
|
||||
}
|
||||
if isinstance(last, dict) and last.get("counts"):
|
||||
report["last_scan"] = {"finished": _when(last.get("finished")), "kind": last.get("kind"),
|
||||
"counts": last.get("counts"), "civitai_unreachable": bool(last.get("civitai_down")),
|
||||
"models": (last.get("files") or [])[:30]}
|
||||
return report
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- outputs
|
||||
|
||||
def search_images(query="", limit=20, page=1):
|
||||
images = _gallery_listing(folder_paths.get_output_directory(), False, str(query or "").strip())
|
||||
limit = _limit(limit)
|
||||
try:
|
||||
page = max(1, int(page))
|
||||
except (TypeError, ValueError):
|
||||
page = 1
|
||||
kept = _kept_lookup()
|
||||
chosen = images[(page - 1) * limit:page * limit]
|
||||
return {
|
||||
"total_matches": len(images), "page": page,
|
||||
"images": [{"image": "/".join(part for part in (item["subfolder"], item["filename"]) if part),
|
||||
"created": _when(item.get("mtime")),
|
||||
**({"kept_as": kept[(item["subfolder"], item["filename"])]}
|
||||
if (item["subfolder"], item["filename"]) in kept else {})}
|
||||
for item in chosen],
|
||||
}
|
||||
|
||||
|
||||
def get_image_info(image, include_image=False):
|
||||
rel, path = _resolve_image(image)
|
||||
info = {"image": rel, "created": _when(os.path.getmtime(path))}
|
||||
chunks = read_png_text(path) if path.lower().endswith(".png") else {}
|
||||
try:
|
||||
prompt = json.loads(chunks.get("prompt") or "null")
|
||||
except ValueError:
|
||||
prompt = None
|
||||
if isinstance(prompt, dict):
|
||||
nodes, models = [], set()
|
||||
for node_id, node in list(prompt.items())[:80]:
|
||||
if not isinstance(node, dict):
|
||||
continue
|
||||
inputs = {}
|
||||
for key, value in (node.get("inputs") or {}).items():
|
||||
if isinstance(value, str):
|
||||
inputs[key] = _text(value, 2000) if len(value) > 2000 else value
|
||||
if value.lower().endswith(MODEL_EXTENSIONS):
|
||||
models.add(value.replace("\\", "/"))
|
||||
elif isinstance(value, (int, float, bool)):
|
||||
inputs[key] = value
|
||||
nodes.append({"id": node_id, "type": node.get("class_type"), "inputs": inputs})
|
||||
info.update(models_used=sorted(models), nodes=nodes)
|
||||
elif chunks.get("parameters"):
|
||||
info["parameters"] = _text(chunks["parameters"], 4000)
|
||||
else:
|
||||
info["note"] = "No generation settings are stored in this image."
|
||||
kept = _kept_lookup().get(tuple(rel.rsplit("/", 1)) if "/" in rel else ("", rel))
|
||||
if kept:
|
||||
info["kept_as"] = kept
|
||||
extra = []
|
||||
if include_image:
|
||||
item = _image_content(path)
|
||||
info["image_attached"] = bool(item)
|
||||
extra = [item] if item else []
|
||||
return info, extra
|
||||
|
||||
|
||||
def search_audio(query="", limit=20):
|
||||
items, total, _more = _audio_page(folder_paths.get_output_directory(), 1, _limit(limit), str(query or "").lower().strip())
|
||||
return {"total_matches": total, "audio": [{
|
||||
"file": "/".join(part for part in (item["subfolder"], item["filename"]) if part),
|
||||
"created": _when(item.get("mtime")),
|
||||
"speech": (item.get("generation") or {}).get("speech") or "",
|
||||
"voice_sample": (item.get("generation") or {}).get("sample") or "",
|
||||
"seed": (item.get("generation") or {}).get("seed"),
|
||||
} for item in items]}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- what the user kept
|
||||
|
||||
def _model_name(model):
|
||||
if isinstance(model, dict):
|
||||
return model.get("filename") or model.get("name") or ""
|
||||
return str(model or "")
|
||||
|
||||
|
||||
def list_combos(query="", limit=20):
|
||||
terms, found = _terms(query), []
|
||||
for note in _list_notebooks():
|
||||
data = note.get("data") or {}
|
||||
if data.get("kind") == "nodes": # its own node structure: model and text slots by their names
|
||||
structure = data.get("structure") or {}
|
||||
values = data.get("values") or {}
|
||||
slots = structure.get("slots") or []
|
||||
models = [{"slot": s.get("label") or s.get("widget"), "folder": s.get("folder") or "",
|
||||
"file": os.path.basename(str(values.get(s["id"]) or "").replace("\\", "/"))}
|
||||
for s in slots if s.get("kind") == "model"]
|
||||
texts = [{"slot": s.get("label") or s.get("widget"), "text": _text(values.get(s["id"]), 400)}
|
||||
for s in slots if s.get("kind") == "text"]
|
||||
combo = {"name": note.get("name"), "kind": "node structure", "models": models, "texts": texts,
|
||||
"nodes": [n.get("name") or n.get("type") for n in structure.get("nodes") or []],
|
||||
"main_model": models[0]["file"] if models else "", "prompt": texts[0]["text"] if texts else ""}
|
||||
loras = [model["file"] for model in models[1:]] + [text["text"] for text in texts[1:]]
|
||||
else:
|
||||
loras = [_model_name(lora) for lora in data.get("loras") or []]
|
||||
combo = {"name": note.get("name"), "base_model": data.get("baseModel") or "",
|
||||
"main_model": _model_name(data.get("mainModel")), "loras": loras,
|
||||
"prompt": _text(data.get("promptEn"), 800)}
|
||||
if not _matches(terms, combo["name"], combo["main_model"], combo["prompt"], " ".join(loras)):
|
||||
continue
|
||||
found.append(combo)
|
||||
return {"total_matches": len(found), "combos": found[:_limit(limit)]}
|
||||
|
||||
|
||||
def list_workflows(query="", limit=20):
|
||||
terms, found = _terms(query), []
|
||||
for recipe in _list_recipes(get_recipes_dir()):
|
||||
data = recipe.get("data") or {}
|
||||
params = json.dumps(data.get("params") or {}, ensure_ascii=False)
|
||||
if not _matches(terms, data.get("name"), " ".join(data.get("tags") or []), data.get("notes"), params):
|
||||
continue
|
||||
found.append({"name": data.get("name") or recipe["filename"], "tags": data.get("tags") or [],
|
||||
"notes": _text(data.get("notes"), 500), "saved": _when((data.get("timestamp") or 0) / 1000),
|
||||
"settings": params if len(params) <= MAX_TEXT else params[:MAX_TEXT] + "…"})
|
||||
return {"total_matches": len(found), "workflows": found[:_limit(limit)]}
|
||||
|
||||
|
||||
def list_saved_prompts(query="", limit=20):
|
||||
terms, found = _terms(query), []
|
||||
directory = get_materials_dir()
|
||||
for summary in _list_materials(directory):
|
||||
if summary.get("kind") != "prompt_plan" or summary.get("moved_to_recipe"):
|
||||
continue
|
||||
try:
|
||||
plan = _read_material(os.path.join(directory, summary["filename"])).get("plan") or {}
|
||||
except (OSError, ValueError, KeyError):
|
||||
continue
|
||||
prompt = {"name": summary.get("name"), "tags": summary.get("tags") or [],
|
||||
"positive": _text(plan.get("positive"), 1200), "negative": _text(plan.get("negative"), 600)}
|
||||
if _matches(terms, prompt["name"], " ".join(prompt["tags"]), prompt["positive"], prompt["negative"]):
|
||||
found.append(prompt)
|
||||
return {"total_matches": len(found), "prompts": found[:_limit(limit)]}
|
||||
|
||||
|
||||
def recent_activity(limit=20, source=""):
|
||||
entries = list_entries(_limit(limit), source if source in ("canvas", "file") else None)
|
||||
return {"entries": [{"time": _when(entry.get("time")), "source": entry.get("source"), "action": entry.get("action"),
|
||||
"target": entry.get("target") or "", "detail": entry.get("detail") or {}} for entry in entries]}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- registry
|
||||
|
||||
def _schema(properties=None, required=()):
|
||||
schema = {"type": "object", "properties": properties or {}, "additionalProperties": False}
|
||||
if required:
|
||||
schema["required"] = list(required)
|
||||
return schema
|
||||
|
||||
|
||||
_QUERY = {"type": "string", "description": "Words that must all appear (case-insensitive); empty lists everything."}
|
||||
_LIMIT = {"type": "integer", "minimum": 1, "maximum": MAX_LIMIT, "default": 20}
|
||||
|
||||
|
||||
def _tool(name, title, description, run, properties=None, required=()):
|
||||
return name, {"run": run, "spec": {
|
||||
"name": name, "title": title, "description": description,
|
||||
"inputSchema": _schema(properties, required),
|
||||
"annotations": {"title": title, "readOnlyHint": True, "destructiveHint": False,
|
||||
"idempotentHint": True, "openWorldHint": False},
|
||||
}}
|
||||
|
||||
|
||||
TOOLS = dict([
|
||||
_tool("library_overview", "Library overview",
|
||||
"How many models of each type and base model the user has, how many are scanned, and how many "
|
||||
"combos, workflow recipes and saved prompts they kept. A good first call.", library_overview),
|
||||
_tool("search_models", "Search models",
|
||||
"Find model files (checkpoints, LoRAs, VAEs…) by name, trigger word or base model. Returns model_id for get_model.",
|
||||
search_models, {
|
||||
"query": _QUERY,
|
||||
"type": {"type": "string", "description": "ComfyUI models folder, e.g. checkpoints, loras, vae, diffusion_models."},
|
||||
"base_model": {"type": "string", "description": "Part of the base model name, e.g. SDXL, Pony, Flux, SD 1.5."},
|
||||
"source": {"type": "string", "enum": ["civitai", "inferred_from_file", "not_scanned"],
|
||||
"description": "Where the model's information came from."},
|
||||
"limit": _LIMIT}),
|
||||
_tool("get_model", "Model details",
|
||||
"Everything known about one model: Civitai name and link, trigger words, base model, description, "
|
||||
"the user's own name and notes. include_cover attaches its cover picture.",
|
||||
get_model, {"model_id": {"type": "string", "description": "From search_models."},
|
||||
"include_cover": {"type": "boolean", "default": False}}, ["model_id"]),
|
||||
_tool("scan_report", "Scan status",
|
||||
"Which models are matched on Civitai, which are not and why, which were never scanned, and what the last scan did.",
|
||||
scan_report),
|
||||
_tool("search_images", "Search output images",
|
||||
"Find images ComfyUI generated, newest first, by words in their prompt, model or settings (e.g. a LoRA "
|
||||
"name or 'seed:123'). Returns image paths for get_image_info.",
|
||||
search_images, {"query": _QUERY, "limit": _LIMIT, "page": {"type": "integer", "minimum": 1, "default": 1}}),
|
||||
_tool("get_image_info", "Image generation settings",
|
||||
"The nodes and settings that produced an output image (prompts, models, seed, sampler…) and whether the "
|
||||
"user kept it as a recipe, combo or prompt. include_image attaches a 512 px copy.",
|
||||
get_image_info, {"image": {"type": "string", "description": "From search_images, e.g. 'ComfyUI_00012_.png'."},
|
||||
"include_image": {"type": "boolean", "default": False}}, ["image"]),
|
||||
_tool("list_combos", "Combos",
|
||||
"The user's combos (搭配): a main model, LoRAs and a prompt kept together, or (kind \"node "
|
||||
"structure\") model nodes and text nodes saved from the canvas with their links, listed by slot.", list_combos,
|
||||
{"query": _QUERY, "limit": _LIMIT}),
|
||||
_tool("list_workflows", "Workflow recipes",
|
||||
"The user's saved workflow recipes with tags, notes and their main settings.", list_workflows,
|
||||
{"query": _QUERY, "limit": _LIMIT}),
|
||||
_tool("list_saved_prompts", "Saved prompts",
|
||||
"Prompts the user saved in the Prompt Studio, with positive and negative text.", list_saved_prompts,
|
||||
{"query": _QUERY, "limit": _LIMIT}),
|
||||
_tool("search_audio", "Generated audio",
|
||||
"Speech the user generated (Anomalous TTS), newest first, with its text, voice and seed.", search_audio,
|
||||
{"query": _QUERY, "limit": _LIMIT}),
|
||||
_tool("recent_activity", "Activity log",
|
||||
"What Anomalous changed recently, newest first: canvas edits it made and files it saved, scanned or deleted.",
|
||||
recent_activity, {"limit": _LIMIT, "source": {"type": "string", "enum": ["canvas", "file"]}}),
|
||||
])
|
||||
|
||||
|
||||
def call_tool(tools, name, arguments, ctx=None):
|
||||
"""An MCP tools/call result for tool `name` of `tools` (run in a worker thread). Tools marked
|
||||
`context` get `ctx` ({base_url} of this server) first."""
|
||||
tool = tools[name]
|
||||
spec = tool["spec"]["inputSchema"]
|
||||
unknown = set(arguments) - set(spec["properties"])
|
||||
missing = [key for key in spec.get("required", []) if key not in arguments]
|
||||
if unknown or missing:
|
||||
return _failure(f"Unknown arguments {sorted(unknown)}" if unknown else f"Missing arguments {missing}")
|
||||
try:
|
||||
value = tool["run"](ctx, **arguments) if tool.get("context") else tool["run"](**arguments)
|
||||
except ToolError as error:
|
||||
return _failure(str(error))
|
||||
except Exception as error: # noqa: BLE001 - an AI app gets a readable failure, never a dropped request
|
||||
logging.getLogger(__name__).warning("Anomalous MCP tool %s failed", name, exc_info=True)
|
||||
return _failure(f"{name} failed: {error}")
|
||||
data, extra = value if isinstance(value, tuple) else (value, [])
|
||||
return {"content": [{"type": "text", "text": json.dumps(data, ensure_ascii=False)}, *extra],
|
||||
"structuredContent": data, "isError": False}
|
||||
|
||||
|
||||
def _failure(message):
|
||||
return {"content": [{"type": "text", "text": message}], "isError": True}
|
||||
+3
-2
@@ -26,7 +26,7 @@ from .path_utils import require_filename, resolve_folder_subdir, resolve_within
|
||||
CARD_THUMBNAIL_EDGE = 512
|
||||
CARD_THUMBNAIL_CACHE_LIMIT = 256 * 1024 * 1024
|
||||
CARD_THUMBNAIL_STATIC_EXTENSIONS = {'.png', '.jpg', '.jpeg', '.webp', '.avif'}
|
||||
CARD_POSTER_VIDEO_EXTENSIONS = {'.mp4', '.webm', '.mov'}
|
||||
CARD_POSTER_VIDEO_EXTENSIONS = {'.mp4', '.webm', '.mov', '.avi', '.mkv'}
|
||||
CARD_WARM_QUEUE_LIMIT = 512
|
||||
_thumbnail_cleanup_lock = threading.Lock()
|
||||
_thumbnail_generation_slots = threading.BoundedSemaphore(2)
|
||||
@@ -302,7 +302,8 @@ async def api_serve_image(request):
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime',
|
||||
'.avi': 'video/x-msvideo'
|
||||
'.avi': 'video/x-msvideo',
|
||||
'.mkv': 'video/x-matroska'
|
||||
}
|
||||
content_type = content_types.get(ext)
|
||||
if content_type is None:
|
||||
|
||||
@@ -0,0 +1,437 @@
|
||||
"""Downloading the models the open workflow is missing (Model Check's "Download").
|
||||
|
||||
POST /anomalous/download/lookup {items: [{key, hash, url, value, types}]} -> where each can come
|
||||
from (download_sources.find_source) and the model folders of its type: [{index, path, free,
|
||||
subfolders}].
|
||||
POST /anomalous/download/start {download_url, type, root, rel, sha256, size, ...} -> {id}
|
||||
GET /anomalous/download/status -> {jobs}
|
||||
POST /anomalous/download/cancel {id}
|
||||
GET/POST /anomalous/download/settings -> where downloads go ({place, folder}), kept in
|
||||
user/anomalous/download_settings.json.
|
||||
|
||||
One download runs at a time, into "<file>.part" next to where the file goes; an interrupted one
|
||||
continues from there. The finished file is checked against the SHA-256 the workflow or the site
|
||||
gives, and only then gets its name; a file that is already there is never replaced. The Civitai
|
||||
key goes only to Civitai (not along a redirect to its file storage).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
from aiohttp import web
|
||||
import folder_paths
|
||||
|
||||
from . import hf_card
|
||||
from .activity_log import add_entry, log_path
|
||||
from .download_sources import (
|
||||
SHA256, USER_AGENT, allowed_download_url, find_source, host_of, is_civitai, same_file, _load_api_key,
|
||||
)
|
||||
from .path_utils import atomic_write_json, resolve_within
|
||||
|
||||
try:
|
||||
from ..net_proxy import urlopen
|
||||
except ImportError: # loaded outside the package (tests)
|
||||
from net_proxy import urlopen
|
||||
|
||||
MODEL_EXTENSIONS = (".safetensors", ".ckpt", ".pt", ".pth", ".bin", ".sft", ".gguf")
|
||||
CHUNK = 1024 * 1024
|
||||
SPACE_MARGIN = 200 * 1024 * 1024
|
||||
MAX_SUBFOLDERS = 300
|
||||
KEEP_FINISHED = 50
|
||||
# hf_mirror None: not chosen yet (the page takes the mirror for a Chinese interface).
|
||||
DEFAULT_SETTINGS = {"place": "workflow", "folder": "Downloads", "hf_mirror": None}
|
||||
|
||||
_lock = threading.Lock()
|
||||
_jobs = {} # id -> job dict, in the order started
|
||||
_queue = []
|
||||
_worker = None
|
||||
|
||||
|
||||
class DownloadError(Exception):
|
||||
def __init__(self, code, message=""):
|
||||
super().__init__(message or code)
|
||||
self.code = code
|
||||
|
||||
|
||||
async def _body(request):
|
||||
"""The request's JSON object, or {} when it is not one (each route then refuses it)."""
|
||||
try:
|
||||
data = await request.json()
|
||||
except ValueError:
|
||||
return {}
|
||||
return data if isinstance(data, dict) else {}
|
||||
|
||||
|
||||
# --- settings ---------------------------------------------------------------------------
|
||||
|
||||
def settings_path():
|
||||
return os.path.join(os.path.dirname(log_path()), "download_settings.json")
|
||||
|
||||
|
||||
def read_settings():
|
||||
settings = dict(DEFAULT_SETTINGS)
|
||||
try:
|
||||
with open(settings_path(), encoding="utf-8") as source:
|
||||
saved = json.load(source)
|
||||
if saved.get("place") in ("workflow", "folder"):
|
||||
settings["place"] = saved["place"]
|
||||
if isinstance(saved.get("folder"), str):
|
||||
settings["folder"] = saved["folder"]
|
||||
if isinstance(saved.get("hf_mirror"), bool):
|
||||
settings["hf_mirror"] = saved["hf_mirror"]
|
||||
except (OSError, ValueError, AttributeError):
|
||||
pass
|
||||
return settings
|
||||
|
||||
|
||||
def clean_folder(folder):
|
||||
"""A relative folder ('' = the type's own folder), '/' separated; ValueError if it climbs out."""
|
||||
parts = [part.strip() for part in str(folder or "").replace("\\", "/").split("/") if part.strip()]
|
||||
for part in parts:
|
||||
if part in (".", "..") or ":" in part or any(char in part for char in '<>"|?*'):
|
||||
raise ValueError("Invalid folder")
|
||||
return "/".join(parts)
|
||||
|
||||
|
||||
async def api_settings(request):
|
||||
if request.method == "POST":
|
||||
data = await _body(request)
|
||||
settings = read_settings()
|
||||
if data.get("place") in ("workflow", "folder"):
|
||||
settings["place"] = data["place"]
|
||||
if isinstance(data.get("hf_mirror"), bool):
|
||||
settings["hf_mirror"] = data["hf_mirror"]
|
||||
if "folder" in data:
|
||||
try:
|
||||
# {base} is filled with the model's base model when a download starts.
|
||||
settings["folder"] = clean_folder(data.get("folder"))
|
||||
except ValueError:
|
||||
return web.json_response({"error": "Invalid folder"}, status=400)
|
||||
atomic_write_json(settings_path(), settings)
|
||||
return web.json_response(settings)
|
||||
return web.json_response(read_settings())
|
||||
|
||||
|
||||
# --- lookup -----------------------------------------------------------------------------
|
||||
|
||||
def folder_type_for(types):
|
||||
known = getattr(folder_paths, "folder_names_and_paths", {})
|
||||
for name in types or []:
|
||||
if isinstance(name, str) and name in known:
|
||||
return name
|
||||
return ""
|
||||
|
||||
|
||||
def _subfolders(root):
|
||||
found = []
|
||||
for current, dirs, _files in os.walk(root):
|
||||
dirs[:] = sorted(name for name in dirs if not name.startswith("."))
|
||||
depth = os.path.relpath(current, root).count(os.sep)
|
||||
for name in dirs:
|
||||
found.append(os.path.relpath(os.path.join(current, name), root).replace(os.sep, "/"))
|
||||
if len(found) >= MAX_SUBFOLDERS:
|
||||
return found
|
||||
if depth >= 1:
|
||||
dirs[:] = []
|
||||
return found
|
||||
|
||||
|
||||
def roots_of(folder_type):
|
||||
"""The model folders of a type: [{index, path, exists, free, subfolders}]."""
|
||||
roots = []
|
||||
for index, path in enumerate(folder_paths.get_folder_paths(folder_type) or []):
|
||||
exists = os.path.isdir(path)
|
||||
probe = path if exists else os.path.dirname(path)
|
||||
try:
|
||||
free = shutil.disk_usage(probe).free if os.path.isdir(probe) else 0
|
||||
except OSError:
|
||||
free = 0
|
||||
roots.append({"index": index, "path": path, "exists": exists, "free": free,
|
||||
"subfolders": _subfolders(path) if exists else []})
|
||||
return roots
|
||||
|
||||
|
||||
def lookup_item(item, hf_mirror=False, down=None):
|
||||
"""Where one item can come from; `down` (kept across one lookup) names the sites that
|
||||
stopped answering, so the next items skip only those."""
|
||||
file_hash = str(item.get("hash") or "").strip()
|
||||
name = os.path.basename(str(item.get("value") or "").replace("\\", "/"))
|
||||
found = find_source(file_hash, str(item.get("url") or "").strip(), name, hf_mirror,
|
||||
folder_type_for(item.get("types")), down)
|
||||
if found.get("found") and file_hash and found.get("sha256") and len(file_hash) >= 10 \
|
||||
and all(c in "0123456789abcdefABCDEF" for c in file_hash) and not same_file(file_hash, found["sha256"]):
|
||||
# The link names another file than the one the workflow was saved with.
|
||||
return {"found": False, "reason": "different_file", "page": found.get("page", "")}
|
||||
return found
|
||||
|
||||
|
||||
async def api_lookup(request):
|
||||
"""Items already looked up (`known`) only get their type and folders. Once a site does not
|
||||
answer, the rest do not ask it (offline: no wait per model); the other sites still are."""
|
||||
data = await _body(request)
|
||||
items, hf_mirror = data.get("items"), data.get("hf_mirror") is True
|
||||
if not isinstance(items, list):
|
||||
return web.json_response({"error": "items must be a list"}, status=400)
|
||||
results, types, down = [], {}, set()
|
||||
for item in items[:100]:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
folder_type = folder_type_for(item.get("types"))
|
||||
if item.get("known"):
|
||||
result = {"found": False, "reason": "known"}
|
||||
else:
|
||||
result = await asyncio.to_thread(lookup_item, item, hf_mirror, down)
|
||||
result.update(key=item.get("key"), type=folder_type)
|
||||
if folder_type and folder_type not in types:
|
||||
types[folder_type] = await asyncio.to_thread(roots_of, folder_type)
|
||||
results.append(result)
|
||||
return web.json_response({"results": results, "roots": types, "settings": read_settings()})
|
||||
|
||||
|
||||
# --- downloads --------------------------------------------------------------------------
|
||||
|
||||
def _public(job):
|
||||
public = {key: job[key] for key in ("id", "key", "state", "error", "received", "total", "rel", "type",
|
||||
"root", "value", "file", "source", "verified")}
|
||||
public["host"] = host_of(job["url"])
|
||||
return public
|
||||
|
||||
|
||||
def plan_job(data):
|
||||
"""A queued job from a start request, or ValueError / DownloadError saying why not."""
|
||||
url = str(data.get("download_url") or "")
|
||||
if not allowed_download_url(url):
|
||||
raise DownloadError("bad_source")
|
||||
folder_type = folder_type_for([data.get("type")])
|
||||
if not folder_type:
|
||||
raise DownloadError("bad_type")
|
||||
paths = folder_paths.get_folder_paths(folder_type) or []
|
||||
root_index = data.get("root", 0)
|
||||
if not isinstance(root_index, int) or not 0 <= root_index < len(paths):
|
||||
raise DownloadError("bad_root")
|
||||
rel = clean_folder(data.get("rel"))
|
||||
if not rel.lower().endswith(MODEL_EXTENSIONS):
|
||||
raise DownloadError("bad_name")
|
||||
root = os.path.realpath(paths[root_index])
|
||||
dest = resolve_within(root, *rel.split("/"))
|
||||
if os.path.exists(dest):
|
||||
raise DownloadError("exists")
|
||||
size = int(data.get("size") or 0)
|
||||
probe = root
|
||||
while probe and not os.path.isdir(probe):
|
||||
probe = os.path.dirname(probe)
|
||||
if size and probe and shutil.disk_usage(probe).free < size + SPACE_MARGIN:
|
||||
raise DownloadError("no_space")
|
||||
sha256 = str(data.get("sha256") or "").lower()
|
||||
with _lock:
|
||||
if any(job["dest"] == dest and job["state"] in ("queued", "running", "verifying") for job in _jobs.values()):
|
||||
raise DownloadError("already")
|
||||
return {
|
||||
"id": uuid.uuid4().hex[:12], "key": str(data.get("key") or ""), "state": "queued", "error": "",
|
||||
"received": 0, "total": size, "rel": rel, "type": folder_type, "root": root_index,
|
||||
"value": str(data.get("value") or ""), "file": os.path.basename(dest), "dest": dest, "url": url,
|
||||
"sha256": sha256 if SHA256.match(sha256) else "", "expected": str(data.get("hash") or "").lower(),
|
||||
"source": str(data.get("source") or ""), "page": str(data.get("page") or ""),
|
||||
"verified": False, "cancel": False, "finished": 0,
|
||||
}
|
||||
|
||||
|
||||
def _open(job, start):
|
||||
headers = {"User-Agent": USER_AGENT}
|
||||
if start:
|
||||
headers["Range"] = f"bytes={start}-"
|
||||
request = urllib.request.Request(job["url"], headers=headers)
|
||||
key = _load_api_key() if is_civitai(job["url"]) else None
|
||||
if key: # not carried along the redirect to the file storage
|
||||
request.add_unredirected_header("Authorization", f"Bearer {key}")
|
||||
try:
|
||||
return urlopen(request, timeout=60)
|
||||
except urllib.error.HTTPError as error:
|
||||
if error.code in (401, 403):
|
||||
raise DownloadError("needs_key" if is_civitai(job["url"]) and not key else "forbidden") from error
|
||||
if error.code == 404:
|
||||
raise DownloadError("gone") from error
|
||||
if error.code == 416 and start:
|
||||
raise DownloadError("restart") from error
|
||||
raise DownloadError("network", f"HTTP {error.code}") from error
|
||||
except (urllib.error.URLError, OSError) as error:
|
||||
raise DownloadError("network", str(error)) from error
|
||||
|
||||
|
||||
def _hash_existing(path, digest):
|
||||
with open(path, "rb") as source:
|
||||
for chunk in iter(lambda: source.read(CHUNK), b""):
|
||||
digest.update(chunk)
|
||||
|
||||
|
||||
def _fetch(job):
|
||||
part = job["dest"] + ".part"
|
||||
os.makedirs(os.path.dirname(job["dest"]), exist_ok=True)
|
||||
digest = hashlib.sha256()
|
||||
start = os.path.getsize(part) if os.path.isfile(part) else 0
|
||||
try:
|
||||
response = _open(job, start)
|
||||
except DownloadError as error:
|
||||
if error.code != "restart":
|
||||
raise
|
||||
start, response = 0, _open(job, 0)
|
||||
with response:
|
||||
if start and response.status != 206:
|
||||
start = 0 # the site sends the whole file again
|
||||
if "text/html" in str(response.headers.get("Content-Type", "")).lower():
|
||||
raise DownloadError("not_a_file") # a login or error page, not the model
|
||||
if start:
|
||||
_hash_existing(part, digest)
|
||||
length = int(response.headers.get("Content-Length") or 0)
|
||||
job["total"] = start + length if length else job["total"]
|
||||
job["received"] = start
|
||||
with open(part, "ab" if start else "wb") as target:
|
||||
while True:
|
||||
if job["cancel"]:
|
||||
raise DownloadError("cancelled")
|
||||
chunk = response.read(CHUNK)
|
||||
if not chunk:
|
||||
break
|
||||
target.write(chunk)
|
||||
digest.update(chunk)
|
||||
job["received"] += len(chunk)
|
||||
return part, digest.hexdigest()
|
||||
|
||||
|
||||
def _finish(job, part, sha256):
|
||||
job["state"] = "verifying"
|
||||
if job["total"] and job["received"] != job["total"]:
|
||||
raise DownloadError("network", "incomplete")
|
||||
expected = job["sha256"] or (job["expected"] if len(job["expected"]) >= 10 else "")
|
||||
if expected and not same_file(expected, sha256):
|
||||
os.remove(part) # our own unfinished download, not a file of the user's
|
||||
raise DownloadError("hash_mismatch")
|
||||
job["verified"] = bool(expected)
|
||||
if os.path.exists(job["dest"]):
|
||||
raise DownloadError("exists")
|
||||
os.replace(part, job["dest"])
|
||||
if host_of(job["url"]) in ("huggingface.co", "hf-mirror.com"):
|
||||
try: # the model card's example image as cover, its words as notes (never over the user's)
|
||||
job["card"] = hf_card.enrich(job["dest"], job["url"])
|
||||
except Exception as error:
|
||||
print(f"[Anomalous Browser] Download: the Hugging Face model card was not read: {error}")
|
||||
# ComfyUI's file lists and the browser's model metadata see the new file.
|
||||
for cache in (getattr(folder_paths, "filename_list_cache", None), getattr(folder_paths, "cache_helper", None)):
|
||||
try:
|
||||
cache.clear()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
from .metadata import clear_metadata_cache
|
||||
clear_metadata_cache()
|
||||
except Exception as error:
|
||||
print(f"[Anomalous Browser] Download: could not clear the model cache: {error}")
|
||||
|
||||
|
||||
def _record(job, sha256):
|
||||
add_entry("file", "model_download", job["rel"], {"download": {
|
||||
"type": job["type"], "path_idx": job["root"], "rel": job["rel"], "file": job["file"],
|
||||
"size": job["received"], "sha256": sha256, "verified": job["verified"],
|
||||
"source": job["source"], "page": job["page"], "value": job["value"], "card": job.get("card"),
|
||||
}})
|
||||
|
||||
|
||||
def run_job(job):
|
||||
job["state"] = "running"
|
||||
try:
|
||||
part, sha256 = _fetch(job)
|
||||
_finish(job, part, sha256)
|
||||
job["state"] = "done"
|
||||
try:
|
||||
_record(job, sha256)
|
||||
except Exception as error:
|
||||
print(f"[Anomalous Browser] Download: could not record it in the activity log: {error}")
|
||||
except DownloadError as error:
|
||||
job["state"] = "cancelled" if error.code == "cancelled" else "failed"
|
||||
job["error"] = error.code
|
||||
if error.code == "cancelled" and os.path.isfile(job["dest"] + ".part"):
|
||||
os.remove(job["dest"] + ".part") # our own unfinished download
|
||||
if error.code not in ("cancelled",):
|
||||
print(f"[Anomalous Browser] Download of {job['rel']} failed: {error}")
|
||||
except Exception as error:
|
||||
job["state"], job["error"] = "failed", "unknown"
|
||||
print(f"[Anomalous Browser] Download of {job['rel']} failed: {error}")
|
||||
job["finished"] = time.time()
|
||||
|
||||
|
||||
def _work():
|
||||
global _worker
|
||||
while True:
|
||||
with _lock:
|
||||
job = next((item for item in _queue if item["state"] == "queued"), None)
|
||||
if job is None:
|
||||
_queue.clear()
|
||||
_worker = None
|
||||
return
|
||||
_queue.remove(job)
|
||||
run_job(job)
|
||||
|
||||
|
||||
def enqueue(job):
|
||||
global _worker
|
||||
with _lock:
|
||||
_jobs[job["id"]] = job
|
||||
_queue.append(job)
|
||||
finished = [item for item in _jobs.values() if item["finished"]]
|
||||
for old in finished[:-KEEP_FINISHED]:
|
||||
_jobs.pop(old["id"], None)
|
||||
if _worker is None:
|
||||
_worker = threading.Thread(target=_work, daemon=True, name="anomalous-download")
|
||||
_worker.start()
|
||||
|
||||
|
||||
async def api_start(request):
|
||||
data = await _body(request)
|
||||
if not data:
|
||||
return web.json_response({"error": "bad_request"}, status=400)
|
||||
try:
|
||||
job = await asyncio.to_thread(plan_job, data)
|
||||
except DownloadError as error:
|
||||
return web.json_response({"error": error.code}, status=409 if error.code in ("exists", "already") else 400)
|
||||
except (ValueError, TypeError) as error:
|
||||
return web.json_response({"error": "bad_path", "message": str(error)}, status=400)
|
||||
enqueue(job)
|
||||
return web.json_response(_public(job))
|
||||
|
||||
|
||||
async def api_status(request):
|
||||
with _lock:
|
||||
jobs = [_public(job) for job in _jobs.values()]
|
||||
return web.json_response({"jobs": jobs})
|
||||
|
||||
|
||||
async def api_cancel(request):
|
||||
data = await _body(request)
|
||||
with _lock:
|
||||
job = _jobs.get(str(data.get("id") or ""))
|
||||
if not job:
|
||||
return web.json_response({"error": "unknown"}, status=404)
|
||||
if job["state"] == "queued":
|
||||
job["state"], job["error"], job["finished"] = "cancelled", "cancelled", time.time()
|
||||
if job in _queue:
|
||||
_queue.remove(job)
|
||||
else:
|
||||
job["cancel"] = True
|
||||
return web.json_response(_public(job))
|
||||
|
||||
|
||||
def register_routes(app):
|
||||
app.router.add_post("/anomalous/download/lookup", api_lookup)
|
||||
app.router.add_post("/anomalous/download/start", api_start)
|
||||
app.router.add_get("/anomalous/download/status", api_status)
|
||||
app.router.add_post("/anomalous/download/cancel", api_cancel)
|
||||
app.router.add_get("/anomalous/download/settings", api_settings)
|
||||
app.router.add_post("/anomalous/download/settings", api_settings)
|
||||
@@ -0,0 +1,52 @@
|
||||
"""GET /anomalous/civitai_check: whether this computer reaches Civitai now, through the proxy in
|
||||
Settings -> Network when one is set (net_proxy.py); for Settings' "Test" button and the first-run
|
||||
setup. One small request with the key, if one is set, so a key Civitai refuses is told apart.
|
||||
It goes online, so it answers this computer only (local_guard LOCAL_GETS)."""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from .download_sources import civitai_headers
|
||||
|
||||
try:
|
||||
from ..net_proxy import configured, failure, urlopen
|
||||
except ImportError: # loaded outside the package (tests)
|
||||
from net_proxy import configured, failure, urlopen
|
||||
|
||||
PROBE_URL = "https://civitai.com/api/v1/models?limit=1"
|
||||
TIMEOUT = 8
|
||||
|
||||
|
||||
def check_civitai():
|
||||
"""{reachable, key ('set' | 'none' | 'refused'), ms, error, failure (net_proxy.failure), proxy}."""
|
||||
headers = civitai_headers()
|
||||
key = "set" if "Authorization" in headers else "none"
|
||||
started = time.monotonic()
|
||||
|
||||
def answer(reachable, key, error="", why=""):
|
||||
return {"reachable": reachable, "key": key, "ms": round((time.monotonic() - started) * 1000),
|
||||
"error": error, "failure": why, "proxy": configured()}
|
||||
try:
|
||||
with urlopen(urllib.request.Request(PROBE_URL, headers=headers), timeout=TIMEOUT) as response:
|
||||
response.read(1024)
|
||||
return answer(True, key)
|
||||
except urllib.error.HTTPError as error:
|
||||
# Civitai answered: the network is fine; 401/403 with a key means the key is refused.
|
||||
if error.code in (401, 403) and key == "set":
|
||||
return answer(True, "refused", f"HTTP {error.code}", failure(error))
|
||||
return answer(error.code < 500, key, f"HTTP {error.code}", failure(error))
|
||||
except (urllib.error.URLError, OSError, ValueError) as error:
|
||||
reason = getattr(error, "reason", None) or error
|
||||
return answer(False, key, str(reason)[:200], failure(error))
|
||||
|
||||
|
||||
async def api_civitai_check(request):
|
||||
return web.json_response(await asyncio.to_thread(check_civitai))
|
||||
|
||||
|
||||
def register_routes(app):
|
||||
app.router.add_get("/anomalous/civitai_check", api_civitai_check)
|
||||
+45
-80
@@ -1,4 +1,7 @@
|
||||
"""Bounded import/export for self-contained Workflow Recipe packages."""
|
||||
"""Recipe packages (配方包): one Workflow Recipe as a .zip to give to someone else — the recipe,
|
||||
its cover, its models' thumbnails and, when asked, its versions — and taking one in as a new
|
||||
recipe card. Bounded: entry names, count, sizes, compression ratio and checksums are checked
|
||||
before anything is written; an import never replaces a recipe (a taken name gets " (2)")."""
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
@@ -9,6 +12,7 @@ import shutil
|
||||
import stat
|
||||
import tempfile
|
||||
import time
|
||||
import urllib.parse
|
||||
import uuid
|
||||
import zipfile
|
||||
|
||||
@@ -19,10 +23,6 @@ from .recipe_store import get_recipes_dir
|
||||
from .utils import require_filename, resolve_within
|
||||
|
||||
|
||||
# Release gates: package transfers stay closed until validation is complete.
|
||||
# Keep the format implementation for compatibility testing.
|
||||
RECIPE_PACKAGE_EXPORT_ENABLED = False
|
||||
RECIPE_PACKAGE_IMPORT_ENABLED = False
|
||||
PACKAGE_VERSION = 1
|
||||
MAX_UPLOAD_BYTES = 32 * 1024 * 1024
|
||||
MAX_ENTRY_COUNT = 256
|
||||
@@ -134,6 +134,18 @@ def _cover_asset_id(recipe):
|
||||
return require_filename(asset_id)
|
||||
|
||||
|
||||
def _picture_summary(recipe):
|
||||
"""What a package shows the receiver: (has a cover, how many model thumbnails). The cover
|
||||
is usually the small image inside recipe.json (`thumbnail`), not an asset file."""
|
||||
thumbnail = recipe.get("thumbnail")
|
||||
has_cover = bool(_cover_asset_id(recipe)) or (
|
||||
isinstance(thumbnail, str) and thumbnail.lower().startswith(recipe_schema.SAFE_THUMBNAIL_PREFIXES))
|
||||
references = recipe.get("params", {}).get("model_references", [])
|
||||
models = {_asset_id_from_reference(item) for item in references if isinstance(item, dict)} if isinstance(references, list) else set()
|
||||
models.discard(None)
|
||||
return has_cover, len(models)
|
||||
|
||||
|
||||
def _validate_manifest(manifest, archive, names):
|
||||
if not isinstance(manifest, dict) or manifest.get("package_version") != PACKAGE_VERSION:
|
||||
raise ValueError("Unsupported recipe package version")
|
||||
@@ -212,7 +224,7 @@ def _recipe_history_files(recipes_dir, filename):
|
||||
|
||||
|
||||
def _build_export(raw_recipe, recipes_dir, filename, options):
|
||||
include_snapshots = options.get("include_snapshots") is True
|
||||
include_snapshots = options.get("include_snapshots", True) is True
|
||||
include_history = options.get("include_history") is True
|
||||
include_identity = options.get("include_identity", True) is True
|
||||
include_model_notes = options.get("include_model_notes", True) is True
|
||||
@@ -372,42 +384,26 @@ def _write_staged_json(path, value):
|
||||
|
||||
|
||||
def _commit_import(record, payload):
|
||||
"""Writes the inspected package as a new recipe (never over one): (filename, name)."""
|
||||
recipes_dir = get_recipes_dir()
|
||||
package_recipe = json.loads(json.dumps(record["recipe"], ensure_ascii=False))
|
||||
collision = payload.get("collision", "rename")
|
||||
name_override = payload.get("name")
|
||||
if name_override is not None:
|
||||
if not isinstance(name_override, str) or not name_override.strip():
|
||||
raise ValueError("Invalid imported recipe name")
|
||||
package_recipe["name"] = name_override.strip()
|
||||
|
||||
existing_names = _existing_recipe_names(recipes_dir)
|
||||
target_filename = None
|
||||
if collision == "replace":
|
||||
target_filename = require_filename(payload.get("target_filename", ""))
|
||||
if not target_filename.endswith(".json"):
|
||||
raise ValueError("Invalid replacement target")
|
||||
if not os.path.isfile(resolve_within(recipes_dir, target_filename)):
|
||||
raise FileNotFoundError
|
||||
elif collision != "rename":
|
||||
raise ValueError("Unsupported collision choice")
|
||||
else:
|
||||
package_recipe["name"] = _unique_name(package_recipe.get("name", ""), existing_names)
|
||||
package_recipe["name"] = _unique_name(package_recipe.get("name", ""), _existing_recipe_names(recipes_dir))
|
||||
|
||||
normalized = recipe_schema._normalise_recipe(package_recipe)
|
||||
normalized["presentation"]["imported"] = True
|
||||
filename = target_filename or f"recipe_{int(time.time())}_{uuid.uuid4().hex[:8]}.json"
|
||||
filename = f"recipe_{int(time.time())}_{uuid.uuid4().hex[:8]}.json"
|
||||
normalized = recipe_schema._enrich_recipe(normalized)
|
||||
|
||||
staging = tempfile.mkdtemp(prefix=".recipe-import-", dir=recipes_dir)
|
||||
final_path = resolve_within(recipes_dir, filename)
|
||||
final_assets = recipe_images._recipe_assets_dir(recipes_dir, filename)
|
||||
final_history = recipe_store._history_dir(recipes_dir, filename)
|
||||
backup_recipe = None
|
||||
backup_assets = None
|
||||
backup_history = None
|
||||
installed_assets = False
|
||||
installed_history = False
|
||||
installed = []
|
||||
try:
|
||||
archive, _ = _validate_zip(record["raw"])
|
||||
with archive:
|
||||
@@ -419,54 +415,33 @@ def _commit_import(record, payload):
|
||||
for name in record["history_names"]:
|
||||
history = _parse_json(_read_zip_entry(archive, name), "historical recipe")
|
||||
_write_staged_json(os.path.join(history_stage, require_filename(name.split("/", 1)[1])), history)
|
||||
|
||||
if collision == "replace":
|
||||
backup_recipe = os.path.join(staging, "old-recipe.json")
|
||||
os.replace(final_path, backup_recipe)
|
||||
if os.path.isdir(final_assets):
|
||||
backup_assets = os.path.join(staging, "old-assets")
|
||||
shutil.move(final_assets, backup_assets)
|
||||
if record["history_names"] and os.path.isdir(final_history):
|
||||
backup_history = os.path.join(staging, "old-history")
|
||||
shutil.move(final_history, backup_history)
|
||||
|
||||
os.replace(os.path.join(staging, "recipe.json"), final_path)
|
||||
# The images first, the recipe last: a recipe card never shows without its pictures.
|
||||
staged_assets = os.path.join(staging, "assets")
|
||||
if os.path.isdir(staged_assets):
|
||||
if os.listdir(staged_assets):
|
||||
os.makedirs(os.path.dirname(final_assets), exist_ok=True)
|
||||
os.replace(staged_assets, final_assets)
|
||||
installed_assets = True
|
||||
installed.append(final_assets)
|
||||
if record["history_names"]:
|
||||
history_stage = os.path.join(staging, "history")
|
||||
if os.path.isdir(final_history):
|
||||
shutil.rmtree(final_history)
|
||||
os.replace(history_stage, final_history)
|
||||
installed_history = True
|
||||
os.makedirs(os.path.dirname(final_history), exist_ok=True)
|
||||
os.replace(os.path.join(staging, "history"), final_history)
|
||||
installed.append(final_history)
|
||||
os.replace(os.path.join(staging, "recipe.json"), final_path)
|
||||
except Exception:
|
||||
if collision == "replace" and backup_recipe and os.path.isfile(backup_recipe):
|
||||
if installed_assets and os.path.isdir(final_assets):
|
||||
shutil.rmtree(final_assets, ignore_errors=True)
|
||||
if installed_history and os.path.isdir(final_history):
|
||||
shutil.rmtree(final_history, ignore_errors=True)
|
||||
if os.path.isfile(final_path):
|
||||
os.remove(final_path)
|
||||
os.replace(backup_recipe, final_path)
|
||||
if backup_assets and os.path.isdir(backup_assets):
|
||||
shutil.move(backup_assets, final_assets)
|
||||
if backup_history and os.path.isdir(backup_history):
|
||||
shutil.move(backup_history, final_history)
|
||||
for path in installed: # this import's own new folders; nothing of the user's
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
raise
|
||||
finally:
|
||||
shutil.rmtree(staging, ignore_errors=True)
|
||||
return filename, normalized.get("name", "")
|
||||
|
||||
|
||||
def _download_name(recipe, filename):
|
||||
"""The .zip's name for the browser: the recipe's name, safe on every system."""
|
||||
name = "".join("_" if char in '<>:"/\\|?*' or ord(char) < 32 else char for char in str(recipe.get("name") or "")).strip(" .")
|
||||
return f"{name or os.path.splitext(filename)[0]}.anomalous-recipe.zip"
|
||||
|
||||
|
||||
async def api_export_recipe_package(request):
|
||||
if not RECIPE_PACKAGE_EXPORT_ENABLED:
|
||||
return web.json_response({
|
||||
"status": "error",
|
||||
"code": "recipe_export_disabled",
|
||||
"message": "Recipe package export is temporarily unavailable",
|
||||
}, status=503)
|
||||
try:
|
||||
payload = await request.json()
|
||||
filename = require_filename(payload.get("filename", ""))
|
||||
@@ -481,19 +456,13 @@ async def api_export_recipe_package(request):
|
||||
return web.json_response({"status": "error", "message": "Recipe not found"}, status=404)
|
||||
except OSError:
|
||||
return web.json_response({"status": "error", "message": "Could not export recipe"}, status=500)
|
||||
return web.Response(
|
||||
body=package,
|
||||
content_type="application/zip",
|
||||
headers={"Content-Disposition": f'attachment; filename="{os.path.splitext(filename)[0]}.anomalous-recipe.zip"'},
|
||||
)
|
||||
name = _download_name(recipe, filename)
|
||||
return web.Response(body=package, content_type="application/zip", headers={
|
||||
"Content-Disposition": f"attachment; filename=\"recipe.anomalous-recipe.zip\"; filename*=UTF-8''{urllib.parse.quote(name)}",
|
||||
})
|
||||
|
||||
|
||||
async def api_import_recipe_package_inspect(request):
|
||||
if not RECIPE_PACKAGE_IMPORT_ENABLED:
|
||||
return web.json_response({
|
||||
"status": "error", "code": "recipe_import_disabled",
|
||||
"message": "Recipe package import is temporarily unavailable",
|
||||
}, status=503)
|
||||
try:
|
||||
raw = await request.content.read(MAX_UPLOAD_BYTES + 1)
|
||||
report = await asyncio.to_thread(_inspect_package, raw)
|
||||
@@ -507,17 +476,15 @@ async def api_import_recipe_package_inspect(request):
|
||||
"token": token,
|
||||
"recipe": {"name": report["recipe"].get("name", ""), "tags": report["recipe"].get("tags", [])},
|
||||
"asset_count": len(report["asset_names"]),
|
||||
"has_cover": _picture_summary(report["recipe"])[0],
|
||||
"model_pictures": _picture_summary(report["recipe"])[1],
|
||||
"history_count": len(report["history_names"]),
|
||||
"existing_names": _existing_recipe_names(get_recipes_dir()),
|
||||
"imported_name": _unique_name(report["recipe"].get("name", ""), _existing_recipe_names(get_recipes_dir())),
|
||||
})
|
||||
|
||||
|
||||
async def api_import_recipe_package_commit(request):
|
||||
if not RECIPE_PACKAGE_IMPORT_ENABLED:
|
||||
return web.json_response({
|
||||
"status": "error", "code": "recipe_import_disabled",
|
||||
"message": "Recipe package import is temporarily unavailable",
|
||||
}, status=503)
|
||||
try:
|
||||
payload = await request.json()
|
||||
token = payload.get("token")
|
||||
@@ -529,8 +496,6 @@ async def api_import_recipe_package_commit(request):
|
||||
filename, name = await asyncio.to_thread(_commit_import, record, payload)
|
||||
except (AttributeError, TypeError, ValueError, json.JSONDecodeError):
|
||||
return web.json_response({"status": "error", "message": "Could not commit recipe package"}, status=400)
|
||||
except FileNotFoundError:
|
||||
return web.json_response({"status": "error", "message": "Replacement recipe not found"}, status=404)
|
||||
except OSError:
|
||||
return web.json_response({"status": "error", "message": "Could not commit recipe package"}, status=500)
|
||||
return web.json_response({"status": "success", "filename": filename, "name": name})
|
||||
|
||||
+4
-1
@@ -72,6 +72,7 @@ class ScanJob:
|
||||
self.files = []
|
||||
self.unchanged = 0
|
||||
self.civitai_down = False
|
||||
self.civitai_error = "" # why Civitai could not be reached (net_proxy.failure), the first one
|
||||
self.errors = []
|
||||
|
||||
def add_folder(self, report_file, base_dir, located=None):
|
||||
@@ -87,6 +88,7 @@ class ScanJob:
|
||||
self.unchanged += int(event.get("unchanged") or 0)
|
||||
elif event.get("event") == "civitai_down":
|
||||
self.civitai_down = True
|
||||
self.civitai_error = self.civitai_error or str(event.get("reason") or "")[:40]
|
||||
for entry in entries:
|
||||
path = entry.get("renamed_to") if entry.get("renamed_to") and entry.get("status") != "failed" else entry.get("file")
|
||||
if not isinstance(path, str):
|
||||
@@ -116,6 +118,7 @@ class ScanJob:
|
||||
return {
|
||||
"kind": self.kind, "options": self.options, "started": self.started,
|
||||
"finished": time.time(), "counts": counts, "civitai_down": self.civitai_down,
|
||||
"civitai_error": self.civitai_error,
|
||||
"errors": self.errors[:10], "files": files[:MAX_FILES],
|
||||
}
|
||||
|
||||
@@ -129,7 +132,7 @@ class ScanJob:
|
||||
print(f"[Anomalous Browser] Could not keep the scan result: {error}")
|
||||
try:
|
||||
target = result["files"][0]["filename"] if self.kind == "one" and result["files"] else ""
|
||||
detail = {"scan": {key: result[key] for key in ("kind", "options", "counts", "civitai_down", "errors")},
|
||||
detail = {"scan": {key: result[key] for key in ("kind", "options", "counts", "civitai_down", "civitai_error", "errors")},
|
||||
"files": result["files"][:MAX_CHANGES], "total": len(self.files),
|
||||
"seconds": round(result["finished"] - result["started"])}
|
||||
add_entry("file", "scan_done", target, detail)
|
||||
|
||||
@@ -61,6 +61,7 @@ def _summarize():
|
||||
return {
|
||||
"total": total, "matched": matched, "unmatched": len(unmatched), "new": len(new),
|
||||
"pending": sum(item["reason"] in ("offline", "network") for item in unmatched),
|
||||
"damaged": sum(item["reason"] == "damaged" for item in unmatched),
|
||||
"unmatched_models": unmatched[:MAX_LISTED], "new_models": new[:MAX_LISTED],
|
||||
"skipped": len(skipped), "skipped_models": skipped[:MAX_LISTED],
|
||||
}
|
||||
|
||||
+4
-2
@@ -17,8 +17,10 @@ from .path_utils import resolve_folder_subdir
|
||||
from .scan_report import ScanJob, locate_root
|
||||
try:
|
||||
from ..model_policies import is_physical_rename_protected
|
||||
from ..net_proxy import child_env
|
||||
except ImportError:
|
||||
from model_policies import is_physical_rename_protected
|
||||
from net_proxy import child_env
|
||||
|
||||
|
||||
PLUGIN_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
@@ -334,7 +336,7 @@ async def api_scan_folder(request):
|
||||
try:
|
||||
cmd = [sys.executable, SCRAPER_PATH, target_dir, "--folder-type", folder_type, *flags]
|
||||
cmd.extend(["--progress-file", progress_file, "--report-file", report_file])
|
||||
process = subprocess.Popen(cmd, cwd=PLUGIN_DIR)
|
||||
process = subprocess.Popen(cmd, cwd=PLUGIN_DIR, env=child_env())
|
||||
_update_scan_marker(marker_file, worker_pid=process.pid)
|
||||
return_code = process.wait()
|
||||
job.add_folder(report_file, base_dir, (folder_type, path_idx))
|
||||
@@ -442,7 +444,7 @@ async def api_scan_all(request):
|
||||
if job.civitai_down: # an earlier folder found Civitai unreachable
|
||||
cmd.append("--civitai-down")
|
||||
cmd.extend(["--progress-file", progress_file, "--report-file", report_file])
|
||||
process = subprocess.Popen(cmd, cwd=PLUGIN_DIR)
|
||||
process = subprocess.Popen(cmd, cwd=PLUGIN_DIR, env=child_env())
|
||||
_update_scan_marker(marker_file, worker_pid=process.pid)
|
||||
return_code = process.wait()
|
||||
_update_scan_marker(marker_file, worker_pid=0)
|
||||
|
||||
+53
-27
@@ -1,16 +1,24 @@
|
||||
"""Bounded translation-provider HTTP route."""
|
||||
"""Prompt translation (POST /anomalous/translate): DeepL when the user saved a key in Settings
|
||||
-> Translation, else Google's free endpoint, else MyMemory. When DeepL was asked and failed, the
|
||||
answer says why (`deepl_error`) while another provider still translates."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
try:
|
||||
from ..net_proxy import urlopen
|
||||
except ImportError: # loaded outside the package (tests)
|
||||
from net_proxy import urlopen
|
||||
|
||||
|
||||
_DEEPL_TARGETS = {
|
||||
"zh": "ZH", "zh-CN": "ZH", "zh-TW": "ZH-HANT", "en": "EN", "ja": "JA", "ko": "KO",
|
||||
"zh": "ZH-HANS", "zh-CN": "ZH-HANS", "zh-TW": "ZH-HANT", "en": "EN-US", "ja": "JA", "ko": "KO",
|
||||
"fr": "FR", "de": "DE", "es": "ES", "ru": "RU", "pt": "PT-BR",
|
||||
}
|
||||
|
||||
@@ -20,18 +28,43 @@ def _translate_with_deepl(text, tl, deepl_key):
|
||||
d_tl = _DEEPL_TARGETS.get(tl)
|
||||
if not d_tl:
|
||||
return None
|
||||
url = "https://api-free.deepl.com/v2/translate" if ":fx" in deepl_key else "https://api.deepl.com/v2/translate"
|
||||
payload = urllib.parse.urlencode({
|
||||
"auth_key": deepl_key,
|
||||
"text": text,
|
||||
"target_lang": d_tl
|
||||
}).encode('utf-8')
|
||||
req = urllib.request.Request(url, data=payload)
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
# A Free account's key ends in ":fx". The key goes in the Authorization header: DeepL no
|
||||
# longer takes it as an auth_key parameter (2025).
|
||||
url = "https://api-free.deepl.com/v2/translate" if deepl_key.endswith(":fx") else "https://api.deepl.com/v2/translate"
|
||||
payload = json.dumps({"text": [text], "target_lang": d_tl}).encode("utf-8")
|
||||
req = urllib.request.Request(url, data=payload, method="POST", headers={
|
||||
"Authorization": f"DeepL-Auth-Key {deepl_key}",
|
||||
"Content-Type": "application/json",
|
||||
})
|
||||
with urlopen(req, timeout=8) as resp:
|
||||
result = json.loads(resp.read().decode('utf-8'))
|
||||
return result["translations"][0]["text"]
|
||||
|
||||
|
||||
def _deepl_error(error):
|
||||
"""Why DeepL did not translate, for Settings: bad_key, quota, busy, or the error's own words."""
|
||||
if isinstance(error, urllib.error.HTTPError):
|
||||
if error.code in (401, 403):
|
||||
return "bad_key"
|
||||
if error.code == 456:
|
||||
return "quota"
|
||||
if error.code == 429:
|
||||
return "busy"
|
||||
return f"HTTP {error.code}"
|
||||
return str(error) or type(error).__name__
|
||||
|
||||
|
||||
def _deepl_key():
|
||||
"""The DeepL key saved in Settings -> Translation (config.json), or ""."""
|
||||
config_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "config.json")
|
||||
try:
|
||||
with open(config_path, 'r', encoding='utf-8') as f:
|
||||
key = json.load(f).get("DEEPL_API_KEY", "")
|
||||
except (OSError, ValueError, AttributeError):
|
||||
return ""
|
||||
return key.strip() if isinstance(key, str) else ""
|
||||
|
||||
|
||||
def _translate_with_google(text, tl):
|
||||
clients = ["dict-chrome-ex", "gtx"]
|
||||
last_err = None
|
||||
@@ -42,7 +75,7 @@ def _translate_with_google(text, tl):
|
||||
req = urllib.request.Request(url, headers={
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
|
||||
})
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
with urlopen(req, timeout=5) as resp:
|
||||
result = json.loads(resp.read().decode('utf-8'))
|
||||
if result and isinstance(result, list) and len(result) > 0 and isinstance(result[0], list):
|
||||
translated_text = "".join([part[0] for part in result[0] if part and len(part) > 0 and part[0]])
|
||||
@@ -65,7 +98,7 @@ def _translate_with_mymemory(text, tl):
|
||||
req = urllib.request.Request(url, headers={
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
|
||||
})
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
with urlopen(req, timeout=5) as resp:
|
||||
data = json.loads(resp.read().decode('utf-8'))
|
||||
res = data.get("responseData", {}).get("translatedText")
|
||||
if res and res != text:
|
||||
@@ -92,31 +125,24 @@ async def api_translate(request):
|
||||
if not text:
|
||||
return web.json_response({"translated": "", "status": "success"})
|
||||
|
||||
# 1. Try DeepL if configured
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
config_path = os.path.join(script_dir, "config.json")
|
||||
deepl_key = ""
|
||||
if os.path.exists(config_path):
|
||||
try:
|
||||
with open(config_path, 'r', encoding='utf-8') as f:
|
||||
cfg = json.load(f)
|
||||
deepl_key = cfg.get("DEEPL_API_KEY", "").strip()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 1. DeepL, when a key is saved
|
||||
deepl_key = _deepl_key()
|
||||
deepl_error = None
|
||||
if deepl_key:
|
||||
try:
|
||||
translated = await asyncio.to_thread(_translate_with_deepl, text, tl, deepl_key)
|
||||
if translated:
|
||||
return web.json_response({"translated": translated, "status": "success", "engine": "deepl"})
|
||||
except Exception as e:
|
||||
print(f"[Anomalous] DeepL translate failed: {e}")
|
||||
deepl_error = _deepl_error(e)
|
||||
print(f"[Anomalous] DeepL translate failed: {deepl_error}")
|
||||
why = {"deepl_error": deepl_error} if deepl_error else {}
|
||||
|
||||
# 2. Try Google Translate (client=dict-chrome-ex)
|
||||
try:
|
||||
translated = await asyncio.to_thread(_translate_with_google, text, tl)
|
||||
if translated:
|
||||
return web.json_response({"translated": translated, "status": "success", "engine": "google"})
|
||||
return web.json_response({"translated": translated, "status": "success", "engine": "google", **why})
|
||||
except Exception as e:
|
||||
print(f"[Anomalous] Google translate failed: {e}")
|
||||
|
||||
@@ -124,7 +150,7 @@ async def api_translate(request):
|
||||
try:
|
||||
translated = await asyncio.to_thread(_translate_with_mymemory, text, tl)
|
||||
if translated:
|
||||
return web.json_response({"translated": translated, "status": "success", "engine": "mymemory"})
|
||||
return web.json_response({"translated": translated, "status": "success", "engine": "mymemory", **why})
|
||||
except Exception as e:
|
||||
print(f"[Anomalous] MyMemory translate failed: {e}")
|
||||
raise e
|
||||
|
||||
@@ -20,6 +20,11 @@ import urllib.request
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
try:
|
||||
from ..net_proxy import urlopen
|
||||
except ImportError: # loaded outside the package (tests)
|
||||
from net_proxy import urlopen
|
||||
|
||||
PLUGIN_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
STATE_FILE = os.path.join(PLUGIN_DIR, ".anomalous_version.local.json")
|
||||
# A version whose tree contains this file has the version panel; older ones do not.
|
||||
@@ -124,7 +129,7 @@ def _fetch_github_releases(owner, repo):
|
||||
"Accept": "application/vnd.github+json",
|
||||
"User-Agent": "Anomalous-Model-Browser",
|
||||
})
|
||||
with urllib.request.urlopen(request, timeout=NETWORK_TIMEOUT) as response:
|
||||
with urlopen(request, timeout=NETWORK_TIMEOUT) as response:
|
||||
payload = json.loads(response.read().decode("utf-8"))
|
||||
releases = []
|
||||
for item in payload if isinstance(payload, list) else []:
|
||||
|
||||
+17
-6
@@ -2,7 +2,8 @@
|
||||
description, and preview media. The API key comes from the plugin's config.json.
|
||||
|
||||
fetch_civitai_info tells the two failures apart: None means Civitai answered that it does not
|
||||
know the file; CivitaiUnreachable means it could not be asked (no connection, timeouts, errors).
|
||||
know the file; CivitaiUnreachable means it could not be asked (no connection, timeouts, errors),
|
||||
and carries why (net_proxy.failure). Requests go through Settings' proxy (net_proxy.py).
|
||||
"""
|
||||
|
||||
import json
|
||||
@@ -12,11 +13,17 @@ import urllib.error
|
||||
import urllib.request
|
||||
from typing import Dict, Optional
|
||||
|
||||
try:
|
||||
from .net_proxy import failure, urlopen
|
||||
except ImportError: # the scan process runs this as a top-level module
|
||||
from net_proxy import failure, urlopen
|
||||
|
||||
USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
|
||||
|
||||
|
||||
class CivitaiUnreachable(Exception):
|
||||
"""Civitai did not answer (network, timeout or server error) after the retries."""
|
||||
"""Civitai did not answer (network, timeout or server error) after the retries; args[0] says
|
||||
why ('timeout', 'refused', 'http:503' …)."""
|
||||
|
||||
|
||||
def _load_api_key():
|
||||
@@ -48,30 +55,34 @@ def fetch_civitai_info(file_hash: str, max_retries: int = 3) -> Optional[Dict]:
|
||||
"""Civitai's record of the model version with this file hash; None when Civitai does not
|
||||
know the file (404). Raises CivitaiUnreachable when it could not be asked."""
|
||||
url = f"https://civitai.com/api/v1/model-versions/by-hash/{file_hash}"
|
||||
why = "other"
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
with urllib.request.urlopen(urllib.request.Request(url, headers=_headers()), timeout=15) as response:
|
||||
with urlopen(urllib.request.Request(url, headers=_headers()), timeout=15) as response:
|
||||
return json.loads(response.read().decode('utf-8'))
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
print(f"\033[93m[Skip] 模型 Hash {file_hash} 未在 Civitai 找到 (404)。\033[0m")
|
||||
return None
|
||||
why = failure(e)
|
||||
print(f"[-] 请求异常，状态码: {e.code} (尝试 {attempt+1}/{max_retries})")
|
||||
except urllib.error.URLError as e:
|
||||
why = failure(e)
|
||||
print(f"[-] 网络请求超时或异常: {e.reason} (尝试 {attempt+1}/{max_retries})")
|
||||
except Exception as e:
|
||||
why = failure(e)
|
||||
print(f"[-] 未知异常: {e} (尝试 {attempt+1}/{max_retries})")
|
||||
if attempt < max_retries - 1:
|
||||
time.sleep(2)
|
||||
print(f"\033[93m[Skip] 模型 Hash {file_hash} 网络重试失败。\033[0m")
|
||||
raise CivitaiUnreachable(file_hash)
|
||||
raise CivitaiUnreachable(why)
|
||||
|
||||
|
||||
def fetch_model_description(model_id) -> str:
|
||||
"""The model page's description (the version record only has the version's); '' when unavailable."""
|
||||
try:
|
||||
request = urllib.request.Request(f"https://civitai.com/api/v1/models/{model_id}", headers=_headers())
|
||||
with urllib.request.urlopen(request, timeout=10) as response:
|
||||
with urlopen(request, timeout=10) as response:
|
||||
return json.loads(response.read().decode('utf-8')).get("description") or ""
|
||||
except Exception as e:
|
||||
print(f"[-] 获取模型主页详细说明失败: {e}")
|
||||
@@ -82,7 +93,7 @@ def download_media(url: str, base_path: str, max_retries: int = 3):
|
||||
"""Downloads an image or video to base_path + the extension its type calls for; the path, or None."""
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
with urllib.request.urlopen(urllib.request.Request(url, headers=_headers()), timeout=15) as response:
|
||||
with urlopen(urllib.request.Request(url, headers=_headers()), timeout=15) as response:
|
||||
content_type = response.headers.get("Content-Type", "").lower()
|
||||
ext = ".png"
|
||||
if "video/mp4" in content_type: ext = ".mp4"
|
||||
|
||||
@@ -21,4 +21,4 @@ open only the topic needed for the current task.
|
||||
These documents describe current contracts rather than implementation history.
|
||||
For exact changes, use Git history. For user-visible releases, use
|
||||
`CHANGELOG.md`. For recurring implementation traps, use
|
||||
`.agents/logs/ai_lessons.md`.
|
||||
[`lessons.md`](lessons.md).
|
||||
|
||||
@@ -9,9 +9,8 @@ recipe persistence, or filesystem-facing routes.
|
||||
package. `api/__init__.py` registers `aiohttp` routes, all prefixed with
|
||||
`/anomalous/`. Changes to Python modules require a full ComfyUI restart; a
|
||||
frontend reload alone does not replace registered handlers or module state.
|
||||
Route registration uses explicit module references. `tests/test_route_manifest.py`
|
||||
locks the HTTP method/path pairs so internal ownership changes cannot silently
|
||||
drop, duplicate, or rename an endpoint.
|
||||
Route registration uses explicit module references, so moving a handler between
|
||||
modules cannot silently drop, duplicate, or rename an endpoint.
|
||||
|
||||
The backend owns filesystem authority. A browser-supplied path, filename,
|
||||
category, output reference, recipe asset, or archive member is untrusted until
|
||||
@@ -145,7 +144,9 @@ polling after its UI has been reopened.
|
||||
|
||||
`scraper.py --report-file` appends one JSON line per model it changed (`matched`,
|
||||
`inferred` with its reason, `failed` with the error, renames and covers) and events
|
||||
(`civitai_down`, `done` with the unchanged count). One model's error is reported and
|
||||
(`civitai_down` with `reason`, net_proxy.py's name for the failure, and `done` with the
|
||||
unchanged count; the result carries the first reason as `civitai_error`). The scan process
|
||||
gets Settings' proxy as HTTP(S)_PROXY (`net_proxy.child_env`). One model's error is reported and
|
||||
the scan goes on. `ScanJob` in `api/scan_report.py` reads each folder's report when its
|
||||
worker exits and, before the scan's marker is released, writes the scan's result to
|
||||
`user/anomalous/last_scan.json` and the activity log, so whoever sees the scan end can
|
||||
@@ -153,12 +154,120 @@ read its result. The scan page's picked or listed models go to `/anomalous/scan_
|
||||
`targets` (`--targets-file` per folder), so one scan has one result.
|
||||
|
||||
An unmatched model's `.info` (`id` -1) keeps why: `anomalous_unmatched_reason` is
|
||||
`not_found` (Civitai answered 404), `network` (no answer after the retries) or
|
||||
`offline`. Online scans look up `network` and `offline` models again; `not_found` ones
|
||||
wait for "look up again". After `CIVITAI_DOWN_AFTER` models in a row without an answer
|
||||
`not_found` (Civitai answered 404), `network` (no answer after the retries),
|
||||
`offline`, or `damaged`: Civitai had no record and the safetensors header does not read
|
||||
whole (`model_identity.safetensors_damage`: too short, a header longer than the file, not
|
||||
JSON, or a tensor ending past the file's end; only the header is read). A damaged header
|
||||
changes the file's hash, so the file is to be downloaded again; the scan summary counts these
|
||||
(`damaged`). Online scans look up `network` and `offline` models again; `not_found` and
|
||||
`damaged` ones wait for "look up again". After `CIVITAI_DOWN_AFTER` models in a row without an answer
|
||||
a scan stops asking (`--civitai-down` carries this to the next folders) and infers the
|
||||
rest from the files, instead of waiting for every timeout.
|
||||
|
||||
## MCP endpoint
|
||||
|
||||
`POST /anomalous/mcp` (`api/mcp_server.py`) answers one JSON-RPC message per POST with
|
||||
one JSON object; GET and DELETE get 405, and nothing is sent unasked. A request whose
|
||||
`params._meta` carries `io.modelcontextprotocol/protocolVersion` is served as 2026-07-28:
|
||||
the `MCP-Protocol-Version`, `Mcp-Method` and (for `tools/call`) `Mcp-Name` headers must
|
||||
match the body, or 400 with -32020; an unknown version gets 400 with -32022 and the
|
||||
supported list. `initialize` selects the legacy shape (2025-03-26 to 2025-11-25); no
|
||||
session id is minted, so both eras are stateless. Unknown methods are -32601 (404 for
|
||||
modern requests).
|
||||
|
||||
The peer must be loopback with no proxy header (`Forwarded`, `X-Forwarded-*`,
|
||||
`X-Real-IP`), and the Host and any Origin must name a loopback host; anything else is
|
||||
403. ComfyUI's own origin middleware also applies.
|
||||
|
||||
Tools (`api/mcp_tools.py`) only read and run in worker threads, four at a time. Model ids
|
||||
are `type:path_idx:relative/path` and images are paths under the output folder; both are
|
||||
resolved with `resolve_folder_subdir` / `resolve_within`, so an id cannot leave its root.
|
||||
A tool's own failure (bad id, missing file, unexpected error) is a result with
|
||||
`isError: true`, not a protocol error. Pictures attached on request are the gallery's
|
||||
512 px thumbnails (at most 1.5 MB).
|
||||
|
||||
Acting tools (`api/mcp_actions.py`) never delete or rename files. Canvas actions go
|
||||
through `mcp_bridge.ask_page`: it sends `{id, action, args}` as the `anomalous.mcp`
|
||||
websocket event to every open page and waits (30 s, 90 s for model checks; at most 20
|
||||
waiting). The first page to POST `{id, claim: true}` to `/anomalous/mcp/bridge` gets it
|
||||
(hidden tabs wait 1.5 s so a visible one wins) and posts `{id, outcome}` back. No page,
|
||||
no answer in time, or the page's refusal becomes a tool error. Scans and speech call this
|
||||
server's own `/anomalous/scan_all`, `/anomalous/download/status`, `/anomalous_tts/characters`, `/prompt` and
|
||||
`/history` over loopback, so they run exactly as from the UI (speech waits up to two
|
||||
minutes for the file). Downloads go through the page (`model_download.js` downloadMissing /
|
||||
downloadFrom), so the placement rules and the put-in after a download are the UI's own. Canvas changes are posted to the activity log with `via: "mcp"`.
|
||||
|
||||
## Backups
|
||||
|
||||
`api/backup.py` packs the user's data into one .zip (`POST /anomalous/backup/export`, then
|
||||
`GET /anomalous/backup/file`) in the temp folder (the newest three are kept): the stores
|
||||
under `stores/<part>/` (recipes, combos, materials, parameters, and `comfy_workflows` =
|
||||
`user/default/workflows` when chosen; the notebooks' legacy-migration marker stays out),
|
||||
`settings/config.json` without `CIVITAI_API_KEY` plus `download_settings.json`, and for each
|
||||
model with a user layer or a user cover `models/<id>/user.json` and `cover<suffix>` (and,
|
||||
unless left out, its scan file `scan<.info|.civitai.info>`; Civitai's cover only when asked,
|
||||
stored once as `civitai<.civitai_bak suffix>`), listed
|
||||
in `manifest.json` with type, relative path, size and SHA-256 (from its sidecars; none is
|
||||
computed). The manifest's `library` lists every model the same way plus its download page;
|
||||
inspecting a backup returns the ones this computer lacks, which the frontend looks up and
|
||||
downloads with the model download routes (into the same folder and path). A cover is the user's when it is not byte-identical to `<model>.civitai_bak.*`,
|
||||
as in the scanner.
|
||||
|
||||
`POST /anomalous/backup/inspect` receives the .zip into the temp folder, refuses anything
|
||||
that is not this plugin's backup, too large, or has an unsafe member path, and reports per
|
||||
store how many files are new, the same or different, and how many models it finds here.
|
||||
`POST /anomalous/backup/apply` then writes only the chosen parts: new files are added, the
|
||||
same skipped, different ones replaced only with `replace` (the file here goes to the
|
||||
Recycle Bin first). Models are found by SHA-256, else by models folder, path and equal
|
||||
size. A scan file is written where the model has none (replaced only with `replace`);
|
||||
Civitai's cover where it has no cover at all, with its `civitai_bak` copy so it stays
|
||||
Civitai's. Their user layer is merged field by field (empty fields filled; `replace` takes the
|
||||
backup's, the old file to the Recycle Bin); the backup's cover replaces a Civitai cover
|
||||
(whose `civitai_bak` copy stays) but not another user cover unless `replace`. Settings
|
||||
are written only when chosen, keeping this computer's Civitai key. Export and import are
|
||||
activity entries (`backup_export`, `backup_import`).
|
||||
|
||||
## Model downloads
|
||||
|
||||
`api/model_download.py` serves Model Check's downloads. `POST /anomalous/download/lookup`
|
||||
asks `api/download_sources.py` where each missing model comes from: Civitai by the
|
||||
workflow's hash (the version's file with that SHA-256 or AutoV2), else a link the
|
||||
workflow carries — a Civitai link naming a model version (the file of that name, else
|
||||
its primary file), a Hugging Face or hf-mirror file link (size and SHA-256 from the
|
||||
site's headers), a GitHub release asset or raw file — else ComfyUI-Manager's model list
|
||||
(`api/manager_catalog.py`, the installed Manager's `model-list.json`, entries of the same
|
||||
file name fitting the node's folder type). With the workflow's hash only a list entry
|
||||
whose file has that SHA-256 counts (several of one name are each asked); without one,
|
||||
a single entry of that name is offered marked `by_name`, never in "Download all" or
|
||||
the MCP's download_missing_models. A link whose file has another SHA-256 than the
|
||||
workflow's hash is refused (`different_file`). A site that stops answering is skipped
|
||||
for the rest of the lookup (`down`), the others are still asked, so Civitai failing in
|
||||
mainland China does not stop Hugging Face through its mirror. It also returns each type's model folders with free
|
||||
space and subfolders (two levels).
|
||||
|
||||
`POST /anomalous/download/start` takes the source, type, root index and a relative path
|
||||
and refuses: a non-https link or one off Civitai, Hugging Face, hf-mirror or GitHub; a
|
||||
path leaving the root (`resolve_within`) or not ending in a model extension; a file that
|
||||
exists; not enough space. Jobs run one at a time in a worker thread into
|
||||
`<file>.part`; a new start for the same file continues it with a Range request. The
|
||||
Civitai key is sent to Civitai only, as an unredirected header, so it never reaches the
|
||||
file storage the download redirects to. An HTML answer (a login page) fails as
|
||||
`not_a_file`, 401/403 as `needs_key` or `forbidden`. The finished file must match the
|
||||
source's SHA-256 (else the workflow's hash) or it is removed as `hash_mismatch`; then it
|
||||
is renamed into place, ComfyUI's file-list caches are cleared and the activity log gets
|
||||
a `model_download` entry. A file from Hugging Face first gets its repository's model card
|
||||
(`api/hf_card.py`, `/api/models/<repo>` on the same host): the first example image (else
|
||||
the repository's first picture) as `<model>.preview.<ext>` when the model has no cover, and
|
||||
its base model, trigger words (`instance_prompt`) and example prompts as notes plus the
|
||||
repository as link in the user layer, only where those are empty. `GET /anomalous/download/status` lists the jobs of this run,
|
||||
`POST /anomalous/download/cancel` stops one (its `.part` goes). The default place is
|
||||
`user/anomalous/download_settings.json` (`{place: workflow | folder, folder, hf_mirror}`).
|
||||
With `hf_mirror` (the lookup's flag; the page sends the setting, or for a Chinese interface
|
||||
true while it is unset) Hugging Face files are looked up and fetched on hf-mirror.com, and
|
||||
without it on huggingface.co, whichever host the link names. The mirror sends networks
|
||||
outside China back to huggingface.co; the lookup follows such a redirect once for the size
|
||||
and SHA-256.
|
||||
|
||||
## Metadata and cache behavior
|
||||
|
||||
The output gallery keeps one ordered directory snapshot for at most ten seconds
|
||||
@@ -234,5 +343,5 @@ Keep product wording precise: deletion cleans sidecars; rename migrates them.
|
||||
a one-field object.
|
||||
- Large input collections use a JSON `POST` body. Do not place them in a query
|
||||
string or pass them as command-line arguments on Windows.
|
||||
- The private local `tests/` directory is ignored and is not imported by runtime
|
||||
code or shipped in the installable plugin.
|
||||
- `tests/` holds the plugin's tests (how to run them: `docs/guides/testing.md`,
|
||||
`tools/run_tests.mjs`). Runtime code never imports it.
|
||||
|
||||
+105
-24
@@ -5,11 +5,17 @@ localization, media, or explicit canvas mutations.
|
||||
|
||||
## Bootstrap and module ownership
|
||||
|
||||
The verified AMB0/AMB1 workflow share-code Import / Export Center is available
|
||||
from the Workflows page's top bar (⇅) through `window.AMB_WorkflowShare.showUnifiedModal()`. Both directions
|
||||
are enabled by explicit product decision. It is independent of the paused Recipe
|
||||
package import/export. Image/workflow hash
|
||||
injection, host saving, and ordinary image downloads remain unchanged.
|
||||
Share and import is the Workflows page's ⇅ (`ui_share.js`). Its share code
|
||||
(`share_code.js`, no DOM) is AMB2: the workflow's lean form — node id, type, widget values,
|
||||
position to 10 px, and only what differs from a fresh node of the type (`LiteGraph.createNode`
|
||||
serialized as the template; a node pack's node always keeps its ports, so its links survive on
|
||||
a computer still lacking the pack), links by slot, groups, the `anomalous_*` extras — deflated
|
||||
and spelled as Chinese characters (14 bits each, U+4E00..U+8DFF, a first byte marking the
|
||||
padding) or base64url. Encoding rebuilds the workflow from the lean form and compares
|
||||
(`sameWorkflow`); subgraphs, native reroutes or any difference make the code hold the whole
|
||||
workflow. AMB0/AMB1 codes still decode. The same dialog imports a pasted code or workflow JSON,
|
||||
a dropped or chosen workflow file or image (`app.handleFile`), or a backup .zip
|
||||
(`ui_backup.js`). It is independent of the paused Recipe package import/export.
|
||||
|
||||
ComfyUI loads JavaScript in the extension `WEB_DIRECTORY` as ES modules.
|
||||
`web/main.js` registers `Anomalous.ModelBrowser` and coordinates host hooks.
|
||||
@@ -68,7 +74,9 @@ The main surfaces are:
|
||||
- Grid and model detail: model browsing plus coordinated detail display in
|
||||
`ui_detail.js`, metadata editing in `ui_model_editor.js`, and advanced selection
|
||||
in `ui_model_selector.js`.
|
||||
- Gallery: generated outputs and the full-screen Image Detail Studio Workbench.
|
||||
- Gallery: generated outputs (pictures and videos; All / Pictures / Videos beside the search
|
||||
and the hidden formats in Settings, `gallery_filter.js` + `ui_gallery_filter.js`) and the
|
||||
full-screen Image Detail Studio Workbench, for pictures only.
|
||||
`ui_gallery_card.js` builds each output's card; its star and the workbench's Keep open
|
||||
`ui_keep_menu.js`, which keeps the image as a Workflow Recipe, a combo or its prompts
|
||||
(`image_keep.js`; `GET /anomalous/kept_images` says what each image was kept as).
|
||||
@@ -79,14 +87,41 @@ The main surfaces are:
|
||||
`notebook_canvas.js`; Put on canvas lays a combo out as a new wired group of nodes (Esc while it
|
||||
follows the pointer removes it; the activity log holds back while `owner.placingCombo` is set) and
|
||||
never changes nodes already on the canvas (values of an existing node are changed in
|
||||
Current node).
|
||||
Current node). A combo with a node structure (`combo_structure.js`, `ui_combo_structure.js`)
|
||||
keeps every picked node but subgraphs, and the links between them. Its slots are the model
|
||||
drop-downs (a drop-down of model files) and text boxes (a multiline text box by the node's
|
||||
definition); a node without slots goes back exactly as saved. A link from a node that was not
|
||||
picked cannot be kept: its input is stored in `structure.open`, and the save dialog, the
|
||||
editor and the card say it needs wiring. Slots are per box, so a node with
|
||||
several model drop-downs has several; a box fed by a link is none. Each slot's models folder
|
||||
is the one whose files hold all the drop-down's options. Putting it down needs every node
|
||||
type installed (missing ones and their packs are named first), restores the other boxes by
|
||||
name in their order, fills the slots by name, then links by port name and type; a slot or
|
||||
link that cannot be restored is reported after placing, never guessed. Values other than
|
||||
the slots cannot be edited: a combo puts down exactly what was saved. The canvas's node menu
|
||||
(`getNodeMenuItems`, browser_entry.js) offers "Save as combo" for the picked nodes, or the
|
||||
right-clicked node when it is not among them, when any of them is not a subgraph.
|
||||
Nodes are kept in flow order (`chainOrder`, combo_slots.js) so slot numbers read along the
|
||||
chain; the editor groups slots into models, LoRAs and prompts in that order, renumbers names
|
||||
it made itself, and lists each model node's other saved values read-only (on a node with
|
||||
several models, by the number in the box name); nodes without slots follow under "put back as
|
||||
saved" with their saved values, read-only. Prompts get a read-only translation panel
|
||||
(translation_service.js); empty optional slots start unticked in the save dialog.
|
||||
- Prompts: the rail entry opens Prompt Studio (below); there is no Material Library page.
|
||||
Saved node values are listed, applied and deleted in Current node.
|
||||
- Current node (Node Assistant)/Model Check: selected-node actions, saved parameters, and
|
||||
missing-model recovery. `model_check.js` decides each workflow model's state and
|
||||
applies fixes; `ui_doctor.js` (the page) and `ui_doctor_banner.js` (the bar shown
|
||||
after a workflow opens, unless its "Don't show again" or Settings → Workflows turned the
|
||||
check on opening off) only render it and call it on a press.
|
||||
check on opening off) only render it and call it on a press. A missing model with a
|
||||
certain source gets Download (`ui_model_download.js`; `model_download.js` holds the
|
||||
logic): the dialog names, per file, the type's folder (a choice when there are several,
|
||||
remembered per type in `localStorage`) and the subfolder from Settings → Model downloads
|
||||
— where the workflow says (so the node needs no change) or a set folder, `{base}` there
|
||||
being the model's base model, matched to an existing folder in any spelling — and the
|
||||
file keeps the workflow's name. When a download ends, the model lists reload, nodes
|
||||
still naming that model get the file (one Ctrl+Z step each), and once nothing is
|
||||
downloading the new files are scanned in one targeted scan.
|
||||
`ui_node_assistant.js` owns the current-node panel and model history, `ui_node_model_picker.js`
|
||||
owns the native-widget model replacer, and `ui_node_parameters.js` with
|
||||
`node_parameter_sets.js` own the panel's one list of saved values for the node type;
|
||||
@@ -104,12 +139,25 @@ button, and `Extensions -> Anomalous Model Browser` command. Entry presentations
|
||||
are mutually exclusive and reuse the same browser instance. The Extensions
|
||||
command remains available as a recovery path in every mode.
|
||||
|
||||
Native commands own the default `Shift + B` browser binding (Ctrl + Shift + M before v1.58.2: Chrome
|
||||
and Edge take it for switching profiles) and the
|
||||
`Ctrl + Shift + L` Prompt Studio binding. Shortcut customization delegates
|
||||
A native command owns the default `Shift + B` browser binding (Ctrl + Shift + M before v1.58.2: Chrome and Edge
|
||||
take it for switching profiles). Prompt Studio has
|
||||
no shortcut of its own (removed 2026-10-07): it opens from the rail, the 7 key, or by opening
|
||||
the browser with a text node selected. Shortcut customization delegates
|
||||
to ComfyUI's command/keybinding panel and recorder; the plugin does not install
|
||||
a parallel global keyboard listener or maintain a second shortcut preference.
|
||||
|
||||
Opening the browser from the trigger, the action-bar button, the menu or its shortcut
|
||||
(`show({ followSelection: true })`) follows the node selected on the canvas unless Settings
|
||||
→ Opening and window turns it off (`shell_open_rules.js`, a localStorage preference): a node
|
||||
with prompt boxes and no model drop-down opens Prompt Studio, which targets that node; any
|
||||
other node opens Current node; nothing selected reopens the page used last. Other callers
|
||||
(Prompt Studio closing, the doctor banner, MCP actions) open the browser as it was.
|
||||
|
||||
While the browser is visible, the plain keys 1–9 go to the rail's pages from the top
|
||||
(`ui_shell_nav.js`; ComfyUI binds no plain digits). They are ignored in text fields, while
|
||||
an input method is composing, and while a dialog is open; the rail's tooltips show each
|
||||
number. This is in-page navigation, not a second shortcut for opening anything.
|
||||
|
||||
Language changes replace the registered Interface setting descriptors with
|
||||
freshly translated copies. ComfyUI's reactive settings tree therefore updates
|
||||
the open category heading, labels, tooltips, custom controls, and combo options
|
||||
@@ -213,9 +261,14 @@ groups, and rolls back everything it created on failure. It never calls
|
||||
until they have a complete remapping path. Complete recipes use
|
||||
`loadGraphData` intentionally so ComfyUI owns new-workflow canvas creation.
|
||||
|
||||
`graph_splice.js` handles deliberate MODEL/CLIP insertion. It analyzes declared
|
||||
port types rather than slot indexes or display names. Ambiguous downstream
|
||||
fan-out is rejected until a user can choose a branch. Picker candidates come
|
||||
`graph_splice.js` decides where a LoRA loader goes (`planLoraInsertion`) and wires it
|
||||
(`spliceLora`), for Current node's insert buttons and the MCP `add_lora` alike. It reads
|
||||
declared port types, never slot indexes or display names. After a node, every link of its
|
||||
MODEL output moves behind the LoRA; before one, only that node's MODEL input. The CLIP
|
||||
line goes through only when the LoRA file trains the text encoder (`api/lora_info.py`
|
||||
reads its header): from the same node, a UNet workflow's one CLIP source, or, before a
|
||||
sampler, the source the prompts wired into it share. Otherwise a model-only LoRA loader
|
||||
is used. A failed connection removes the node and restores every link. Picker candidates come
|
||||
from the target widget's native combo values; metadata may filter or decorate
|
||||
that set but may not introduce foreign category values.
|
||||
|
||||
@@ -240,10 +293,14 @@ browsing, editing, or already stored data.
|
||||
|
||||
The studio edits prompts where they are: the prompt boxes of the prompt node last
|
||||
selected on the canvas, or, with no such node, a positive and a negative draft
|
||||
(`owner.promptStudioDraft`, kept while the page stays open). It opens from the rail's
|
||||
Prompts entry, the Prompt Studio shortcut and Current node's "Edit in Prompt Studio";
|
||||
the browser folds away while it is open and comes back when it closes (not when
|
||||
another studio replaces it). There is no assembly board, block draft or plan loading.
|
||||
(`owner.promptStudioDraft`, kept while the page stays open). It is the browser's Prompts
|
||||
page (`owner.promptPanel`, `ui_prompt_composer.js`): the rail's Prompts entry, the 7 key,
|
||||
opening the browser with a text node selected, Current node's "Edit in Prompt Studio" and
|
||||
a kept prompt all go there. "Beside the canvas" moves it into a drawer docked by the canvas
|
||||
(side and width remembered) with the browser folded away; "Back to the window" returns it to
|
||||
the page and Close or Esc closes the drawer. Only one studio is shown at a time; the page's
|
||||
studio stops when another page is shown or the browser closes (its 400 ms poll with it) and
|
||||
is made again when the page is shown. There is no assembly board, block draft or plan loading.
|
||||
|
||||
The target (`ui_prompt_target.js`) is polled every 400 ms. Selecting another node with
|
||||
prompt boxes switches to it; empty canvas or a node without boxes keeps the current
|
||||
@@ -266,8 +323,12 @@ language picked in the top bar (one of `GLOSS_LANGUAGES`, none by default, remem
|
||||
local storage with the view; an older "Chinese on" setting reads as Simplified Chinese).
|
||||
They are asked for in batches, kept for the session per language, and a failed lookup
|
||||
leaves that box's meanings in that language blank instead of retrying. The backend
|
||||
skips DeepL for a target it does not take and lets MyMemory detect a source that is
|
||||
neither English nor Chinese.
|
||||
(`api/translation_routes.py`) asks DeepL first when Settings → Translation has a key
|
||||
(`DEEPL_API_KEY` in config.json, sent in the Authorization header, never in a backup),
|
||||
then Google's free endpoint, then MyMemory; when DeepL failed, the answer carries
|
||||
`deepl_error` (bad_key, quota, busy or the error) beside the other provider's translation,
|
||||
which Settings shows after a key is saved. It skips DeepL for a target it does not take and
|
||||
lets MyMemory detect a source that is neither English nor Chinese.
|
||||
|
||||
The cards are three built-in ones and the saved prompts (prompt-kind material files,
|
||||
`prompt_material_source.js`). The saved list is asked for on open, every 30 seconds
|
||||
@@ -313,12 +374,19 @@ provider-specific validation belongs to the backend translation route.
|
||||
`styles.css` is the single external entry and ordered import manifest for current
|
||||
visual values. Its `web/styles/00-*.css` through `10-*.css` children preserve the
|
||||
original cascade order; every child import carries the same cache version so an
|
||||
entry-cache hit cannot leave stale child rules. `tests/css_bundle_order.mjs`
|
||||
guards the unique ordered list and byte-for-byte reconstructed bundle. Shared `--amb-*`
|
||||
entry-cache hit cannot leave stale child rules. Shared `--amb-*`
|
||||
tokens express surfaces, text, borders and control shapes; theme overrides must
|
||||
be scoped to `.theme-abyssal-scarlet` rather than changing unrelated surfaces.
|
||||
be scoped to `.theme-abyssal-scarlet` or `html.amb-theme-light` rather than changing
|
||||
unrelated surfaces.
|
||||
|
||||
Colours come from the tokens in `00-foundation-models.css`, which both themes define:
|
||||
Themes (`interface_settings.js`, Settings → Appearance → Theme and the ComfyUI setting
|
||||
`Anomalous.ModelBrowser.Theme`): follow ComfyUI (default), dark, light, or Abyssal Scarlet.
|
||||
Following ComfyUI reads `dark-theme` on `<html>`, which ComfyUI sets for a dark palette and
|
||||
removes for a light one, and repaints when it changes. The light theme is the class
|
||||
`amb-theme-light` on `<html>`; Abyssal Scarlet keeps `theme-abyssal-scarlet`. An older
|
||||
Abyssal Scarlet switch (`anomalous_theme_abyssal_scarlet`) reads as the Abyssal choice.
|
||||
|
||||
Colours come from the tokens in `00-foundation-models.css`, which every theme defines:
|
||||
surfaces (`--amb-bg-page`, `-panel`, `-card`, `-card-hover`, `-input`), text
|
||||
(`--amb-text-main`, `-soft`, `-muted`, `-dim`), borders (`--amb-border`, `-strong`,
|
||||
`-hover`), the primary button (`--amb-btn-primary-*`) and one accent, `--amb-link`, for
|
||||
@@ -327,6 +395,19 @@ transparent)`. CSS injected from JS and inline styles use the same tokens. Liter
|
||||
colours stay only where the colour is the meaning: status (red, green, amber), prompt
|
||||
roles, voice emotions, canvas node colours and translucent shadows or overlays.
|
||||
|
||||
Tokens added for the light theme keep the dark themes exactly as they were:
|
||||
`rgba(var(--amb-ink-rgb), a)` for tints over the page (white on dark, black on light),
|
||||
`--amb-danger/-warn/-ok` and their `-text`/`-soft` shades for status text, `--amb-bg-tooltip`,
|
||||
`--amb-bg-float`, `--amb-bg-dialog`, `--amb-bg-sunken`, `--amb-scrim` and `--amb-note-*`.
|
||||
Older rules and inline styles that wrote a dark colour now write
|
||||
`var(--amb-lt-<role>, <that colour>)`: the `--amb-lt-*` tokens exist only in the light theme,
|
||||
so every other theme falls back to the original colour. Use them when a dark literal cannot
|
||||
become a shared token. Inside the plugin's windows ComfyUI's own palette variables
|
||||
(`--bg-color`, `--fg-color`, `--input-text` …) are mapped to the plugin's tokens whenever the
|
||||
two themes differ. Surfaces over images (card labels, cover pills) stay dark, and the image
|
||||
viewer (`.anomalous-workbench-overlay`, `.anomalous-gallery-viewer`) gets the dark tokens back
|
||||
in the light theme.
|
||||
|
||||
Studio drawer rules keep the source deck at the screen edge and the assembly
|
||||
track next to the canvas. Common geometry is shared between dock directions;
|
||||
direction-specific rules set column order and separators. The removed embedded
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
# 踩过的坑（Lessons）
|
||||
|
||||
反复出现的坑和严重故障。每条写清：现象、根因、怎么发现的、以后怎么做。
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-26 · v1.57.2：一条 `:has()` 样式让拖动画布掉帧（严重）
|
||||
|
||||
**现象**：B 站用户反馈装了插件后拖动工作流 10% low 只有二十几帧；和 ComfyUI Prompt Assistant 一起装时，大工作流只剩个位数帧甚至卡死。插件界面关着也一样。
|
||||
|
||||
**根因**：`web/styles/01-shell-tools.css`（b6acd09，2026-09-19，进了 v1.57 / v1.57.1）
|
||||
|
||||
```css
|
||||
body:has(#anomalous-toolbox-modal[style*="display: flex"]) #anomalous-sidebar-tooltip-bubble, …
|
||||
```
|
||||
|
||||
`:has()` 里放属性选择器（这里是 `[style*=…]`），浏览器在页面上任何元素的该属性变化时，都要重新判断 `body:has(...)`。ComfyUI 平移、缩放画布时每一帧都在改每个 DOM 控件（文本框）的 `style`，于是每帧都要付出和页面元素数量成正比的样式计算。Prompt Assistant 给每个文本框加一组按钮，页面元素多十几倍，开销跟着放大。两个插件本身没有冲突。
|
||||
|
||||
**实测**（真 ComfyUI 前端 1.53.6 + Chromium，300 节点工作流平移 120 帧，样式重算合计）：
|
||||
|
||||
| 情况 | 修复前 | 修复后 |
|
||||
| --- | --- | --- |
|
||||
| 不装插件 | 60 ms | — |
|
||||
| 只装 AMB | 690 ms | 63 ms |
|
||||
| AMB + Prompt Assistant | 2300 ms（约 19 ms/帧，超过 60 帧的一整帧） | 86 ms |
|
||||
| 只装 Prompt Assistant | 80 ms | — |
|
||||
|
||||
逐条删规则二分：只删掉这 7 条 `:has()` 就回到基线，其中罪魁是带 `[style*=…]` 的那条；`button:has(.x)` 等其他 `:has` 测不出开销。
|
||||
|
||||
**修复**：删掉这条规则。它是多余的：工具箱和设置窗口打开前都会调用 `hideTooltipImmediately()`，`showSharedTooltip` 在窗口开着时也会拒绝显示。
|
||||
|
||||
**以后怎么做**
|
||||
- 不要用 `:has()` 或 `[style*="display: …"]` 去判断界面开关状态。要表示“开着”，由 JS 加减一个 class（作用在自己的容器上），或者直接在 JS 里处理。
|
||||
- 全局生效的昂贵选择器（`body:has`、`html:has`、`*`、未限定作用域的 `button`、`textarea`）会作用在 ComfyUI 的每一帧上。新增前先问：它会不会在画布拖动时被反复判断？
|
||||
- 性能问题要实测，别只看代码猜：
|
||||
- 装一个真的 ComfyUI（`--cpu` 即可），用 Playwright 在空白处拖动画布；
|
||||
- 用 CDP 的 `Performance.getMetrics` 比较 `RecalcStyleDuration`；
|
||||
- 再在页面里删掉可疑规则做二分（遍历 `document.styleSheets`，对插件自己的样式表 `deleteRule`）。
|
||||
- 没有显卡时总帧时间不准，但样式计算的倍数可信。
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-26 · 修复只在本机 main，发布版没带上
|
||||
|
||||
**现象**：素材库“选中节点时卡片错位”（应用按钮重复、图标重复成 ⚡ ⚡、标签日期挤在一起）的修复 01fc925、9e761d0 早在 9 月 24 日就做了，但只进了 `feature/audio-engines`（通过“Merge main … into feature”）。GitHub 上的 `main` 并没有，发布版 v1.57.1 的用户一直带着这个问题。
|
||||
|
||||
**根因**：修复提交在本机 main 上，只把 main 合进了 feature，本机 main 从未推送。
|
||||
|
||||
**以后怎么做**
|
||||
- 发布或回答“修了没有”之前，用 `git branch -r --contains <提交>` 确认修复在远端的哪个分支，别只看本机。
|
||||
- 本机 main 有未推送提交时，先推送或说明，再做合并。
|
||||
- 这次已把两个提交 cherry-pick 进远端 main（e1ac84f、4e8603b），随 v1.57.2 发布。
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-26 · 发布热修复的检查项
|
||||
|
||||
- 改了 CSS 或 JS，要同步改缓存参数：`web/styles.css` 里对应的 `@import …?v=`、`browser_entry.js` 的 `styles.css?v=`、`main.js` 引用 `browser_entry.js` 的 `?v=`。不然用户刷新后浏览器可能还用旧文件。feature 分支已去掉部分 `?v=`，合并时以各分支现状为准。
|
||||
- `CHANGELOG.md` 是 CRLF、LF 混合换行。用脚本编辑时按原字节读写（`newline=''`），否则整份文件都会显示成改动。
|
||||
- 云端会话能推分支，推不了标签，GitHub 工具也不能创建 Release。发布要在网页上完成：Draft a new release → 填新标签 → Target 选 `main` 并核对提交号 → 粘贴说明 → Publish。插件的版本面板只列正式 Release，用 ComfyUI Manager 更新的用户则直接拉 main 最新提交。
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-27 · 拆文件漏了 import，保存工作流在发布版里整个坏掉
|
||||
|
||||
**现象**：v1.57 到 v1.57.2，工作流页“保存当前工作流”只要工作流里有模型加载器就提示“保存工作流配方失败”。B 站用户反馈“就没成功过”。素材库走的是另一条路，所以能存。
|
||||
|
||||
**根因**：`refactor(recipes): separate schema images and storage`（6ad975c）把函数搬进 `recipe_images.py`，但它调用的 `_resolve_exact_model_reference` 和 `hashlib` 留在了旧文件里。同一轮拆分还让 `media_routes.py` 少了 `time`、`tempfile`：卡片缩略图出错后被 `except Exception` 吞掉，静默退回原图。另外 `model_metadata.py` 少了 `_resolve_paths_to_model_info_sync`。Python 只在那一行真正执行时才报 NameError，导入和其他测试都通过。唯一的保存测试关掉了模型预览快照，正好绕开了出错的那一行。
|
||||
|
||||
**以后怎么做**
|
||||
- 拆模块后跑 `tests/test_backend_names.py`：后端文件读到却从没绑定的名字，它都会列出来。
|
||||
- 行为测试要走默认路径。`recipe_payload` 默认关着快照，新测试 `test_saving_with_preview_snapshots_captures_the_model_cover` 专门走默认的“开着快照”。
|
||||
- 真实按钮至少点一次：在隔离的 ComfyUI 里点“保存当前工作流”，一眼就能看出问题。
|
||||
|
||||
---
|
||||
|
||||
## 2026-10-12 · 页面往下滚后，点“返回”点到了下面的模型（第二次）
|
||||
|
||||
**现象**：从画布节点存的搭配（节点搭配），页面往下滚一点后，点“‹ 全部搭配”打开的是被滚到标题下面的模型封面。普通搭配和搭配列表没事。
|
||||
|
||||
**根因**：和 2026-10-09 修过的是同一个 Chrome 行为：滚动框里有 `overflow: hidden` 的元素（这里是模型封面的圆角框）或 `content-visibility: auto` 时，滚出可见区域的部分在点击判断里仍算“在那里”，盖住了上面的标题按钮。上次只给当时有问题的两个滚动框加了 `clip-path: inset(0)`，之后新做的节点搭配页面又放进了带圆角封面的行。
|
||||
|
||||
**怎么发现的**：在隔离的 ComfyUI 里把节点存成搭配，把编辑区分 12 步滚到底，每一步用 `document.elementFromPoint` 取标题里每个按钮中心点的元素，看是不是按钮自己；滚到 140px 起就变成了封面 `IMG`。
|
||||
|
||||
**以后怎么做**
|
||||
- 标题、标签栏下面的滚动框一律加 `clip-path: inset(0)`（它对点击判断也生效）。`overflow` 本身裁不住这种情况。
|
||||
- 新页面或新内容放进滚动框后，用上面的办法扫一遍：每个滚动框分步滚到底，检查每个可见按钮都能拿到自己的点击。
|
||||
|
||||
---
|
||||
|
||||
## 2026-10-08 · ComfyUI-Manager 审核 Anomalous_TTS：三类安全口子
|
||||
|
||||
**现象**：Anomalous_TTS 申请收录进 ComfyUI-Manager（Comfy-Org/ComfyUI-Manager#3342），维护者两轮审核退回，先后指出：
|
||||
|
||||
1. **下载来的东西被当代码执行**：中文多音字模型从个人 ModelScope 账号的 `master` 下载，不核对指纹，解压后执行包里的 `config.py`；英文词典读 pickle。而网页请求还能改“去哪个文件夹找这些文件”，等于网页能决定运行什么代码。
|
||||
2. **网页请求决定删改哪里**：`POST /anomalous_tts/storage` 从请求里取新的存放位置，跨盘移动后对旧位置 `rmtree`。只检查“是不是本机请求”不够：本机网页也不该决定在哪里建、删文件夹。
|
||||
3. **按请求里的路径读任意文件**：导入预览、导入检查会返回请求点名的任何文件内容。
|
||||
|
||||
**根因**：这些接口最初是为了方便（在界面里选文件夹、试听本机音频、一键挪角色），没有把“网页请求是不可信的输入”当成前提；“只接受本机请求”被当成了足够的保护。
|
||||
|
||||
**怎么改的**（接口版本 13）：配置文件当数据解析，词典从文本建，不读写 pickle，测试拦住 `pickle.load` / `exec_module` / `eval(`；下载锁定版本并先核对 SHA-256 再解压；存放位置、角色文件夹、底模来源只从用户手写的设置文件读；导入只收上传的文件，没有接口再接受本机路径。
|
||||
|
||||
**以后怎么做**
|
||||
- 两个仓库的规范都写了安全底线（本仓库 AGENTS.md 第 5 节“安全底线”，Anomalous_TTS 的 ARCHITECTURE.md“不能破的规矩”）。新接口按它写，审查时逐条对照。
|
||||
- 本仓库的现状（2026-10-09 统计）：55 个会写文件的接口，只有 MCP、模型导入、整理三处检查了本机请求；删除模型、改模型信息、开始下载、导入备份等在 `--listen` 时局域网里的其他设备也能调用。没有执行下载内容、pickle、eval 的代码。2026-10-12 的 v1.58.1 一次补齐：`api/local_guard.py` 一个中间件把关 `/anomalous/` 下所有会改东西的请求，只接受本机（见 backend.md“Requests from other devices”）。
|
||||
|
||||
---
|
||||
|
||||
## 2026-10-13 · v1.58.0–v1.58.2：同一种 `:has()` 又掉帧（第二次）
|
||||
|
||||
**现象**：#22（Lindemellis）：216 个节点的工作流，开着 Prompt Assistant 时一次画布重绘 465–770 ms，拖动只有 2–3 帧。只删掉一条规则就回到 53 ms。
|
||||
|
||||
**根因**：图标栏（46facb9，2026-09-29，随 v1.58.0 发布）在 `web/styles/15-shell-rail.css` 里又写了
|
||||
|
||||
```css
|
||||
#anomalous-container.anomalous-has-rail:has(> .anomalous-nb-modal[style*="display: flex"]) #anomalous-sidebar-wrapper
|
||||
```
|
||||
|
||||
和上面 v1.57.2 那条是同一种写法：`:has()` 里按 `style` 属性匹配。v1.57.2 修好三天后就写回来了，因为经验只写在这份文档里，没有东西拦着；当时的性能实测也只跑了 main 和音频分支，没跑图标栏所在的分支。
|
||||
|
||||
**修复**：工作区开没开由 JS 在容器上加减 `anomalous-workspace-open`，CSS 只看这个 class（#22 的修复）。
|
||||
|
||||
**以后怎么做**
|
||||
- `node tools/check_structure.mjs` 现在会拒绝 `:has()` 里出现 `[style…]`，报出文件和行号。
|
||||
- 同一个坑第二次出现，就把它做成检查，不再只补一段经验。
|
||||
@@ -152,6 +152,12 @@ press puts them in: the bar's or the doctor page's for hash matches, the page's
|
||||
per-row button for a size-only candidate. A scan with "fix the open workflow"
|
||||
on puts hash matches in when it ends.
|
||||
|
||||
A model that is not here can be downloaded from the page when its source names one
|
||||
exact file: Civitai by the workflow's hash, or a file link the workflow carries. The
|
||||
download is checked against the SHA-256 before it gets its name, so a wrong file never
|
||||
lands in a models folder; the file keeps the workflow's name, and nodes are pointed at
|
||||
it only when it went to another folder than the workflow says.
|
||||
|
||||
Provenance-rich workflows skip the redundant full filename-to-hash cache refresh.
|
||||
Legacy workflows without injected provenance may refresh it for compatibility.
|
||||
This is an I/O optimization only and does not change identity evidence.
|
||||
|
||||
@@ -4,24 +4,26 @@ Read this document for recipe schemas, cards/detail behavior, package handling,
|
||||
result galleries, Parameter Notebooks, prompt roles, and the recipe-powered Node
|
||||
Assistant.
|
||||
|
||||
## Package import/export availability
|
||||
## Recipe packages
|
||||
|
||||
Recipe package import and export are temporarily closed pending validation.
|
||||
Topbar/card/detail buttons are disabled with a localized explanation. The registered
|
||||
`POST /anomalous/export_recipe_package` route returns HTTP 503 with code
|
||||
`recipe_export_disabled` before reading request JSON or recipe files.
|
||||
`RECIPE_PACKAGE_EXPORT_ENABLED` in `api/recipe_packages.py` is a release gate,
|
||||
not a user setting. `RECIPE_PACKAGE_IMPORT_ENABLED` likewise gates both
|
||||
`POST /anomalous/import_recipe_package_inspect` and
|
||||
`POST /anomalous/import_recipe_package_commit`: each returns HTTP 503 with code
|
||||
`recipe_import_disabled` before reading uploaded bytes, request JSON, inspection
|
||||
tokens or writing files. Package format helpers remain for compatibility tests;
|
||||
local saves are unaffected.
|
||||
A recipe package (配方包) gives one recipe to someone else; a backup (`api/backup.py`) is
|
||||
for the user's own data and a share code (`share_code.js`) for a workflow alone. Reopened
|
||||
by the user's decision on 2026-10-07 after round-trip and failure tests.
|
||||
|
||||
Reopening requires an explicit release decision, package round-trip and failure
|
||||
validation, then restoring the frontend export action and enabling the backend
|
||||
gates together. Do not reopen package transfers as a side effect of UX work. Workflow
|
||||
share-code import/export is a separate, verified feature on the Workflows page's top bar (⇅).
|
||||
`POST /anomalous/export_recipe_package` (`api/recipe_packages.py`) returns a .zip named
|
||||
after the recipe: `recipe.json` (its `source_image` cleared), the cover and the models'
|
||||
thumbnails as WebP assets (`include_snapshots`, on by default), its versions only with
|
||||
`include_history`, and `manifest.json` listing every entry with size and SHA-256. The card's
|
||||
export button and the detail's More menu open `ui_recipe_package.js`.
|
||||
|
||||
Importing goes through the Workflows page's ⇅ (`ui_share.js`): a dropped .zip is tried as a
|
||||
recipe package, else as a backup. `POST /anomalous/import_recipe_package_inspect` checks
|
||||
entry names, count, sizes, compression ratio, the manifest's checksums, the schema and that
|
||||
every referenced asset is a WebP and nothing unreferenced is there, then keeps it for ten
|
||||
minutes under a token. `POST /anomalous/import_recipe_package_commit` always writes a new
|
||||
recipe: a taken name gets " (2)", nothing is replaced. Assets and versions are moved in
|
||||
first and the recipe file last; a failure removes this import's own new folders and leaves
|
||||
no half recipe.
|
||||
|
||||
## Product and data model
|
||||
|
||||
|
||||
@@ -42,12 +42,6 @@
|
||||
|
||||
## 验证
|
||||
|
||||
在插件根目录运行：
|
||||
|
||||
```powershell
|
||||
node --experimental-vm-modules tests/update_guide.mjs
|
||||
```
|
||||
|
||||
`update_guide.mjs` 加载真实模块，检查双语配置、版本记忆、重看、翻页、关闭清理、
|
||||
父窗口关闭、存储失败及扫描图标替换后的标签。它不能代替实际浏览器的布局与焦点验证。
|
||||
更新指南没有单独的自动测试。改动后在浏览器里实际打开 **!** 指南，检查双语、
|
||||
翻页、重看、关闭和焦点；全部测试的跑法见 `docs/guides/testing.md`。
|
||||
只有同时修改工坊生命周期或导入导出行为时，才追加对应测试；纯引导样式不运行无关测试。
|
||||
|
||||
@@ -105,7 +105,7 @@ Notebook application.
|
||||
Architecture documents change only when an owner, data flow, interface contract,
|
||||
persistence format, security boundary, or critical invariant changes. Ordinary
|
||||
fixes do not append “unchanged” entries. Git is the implementation history;
|
||||
`CHANGELOG.md` is user-facing release history; `.agents/logs/ai_lessons.md` holds
|
||||
`CHANGELOG.md` is user-facing release history; `docs/architecture/lessons.md` holds
|
||||
only durable recurring traps.
|
||||
|
||||
## AD-015 — Recipe model mutations update all identity representations
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
# 让 AI 读你的模型库（MCP）
|
||||
|
||||
支持 MCP 的 AI 软件（Claude、Cursor、Cherry Studio、VS Code 等）连上以后，就能直接查你的 ComfyUI 资料：有哪些模型和触发词、某张图当时用的什么参数、存过哪些搭配和提示词，然后据此回答你的问题或者帮你写提示词。
|
||||
|
||||
它还能动手：改当前工作流的提示词、换模型、加 LoRA、摆上你的搭配、找回缺失的模型、运行工作流、扫描模型，以及让 Anomalous TTS 的角色读台词。
|
||||
|
||||
- **每一步都能撤销**：画布上的改动都是一步，按 Ctrl+Z 就回去；
|
||||
- **都有记录**：改动会写进“操作记录”，标着“AI”，页面右上角也会提示一句；
|
||||
- **不删文件**：不会删除或改名任何文件。
|
||||
|
||||
AI 软件在调用会改东西的工具前，一般会先问你同不同意。
|
||||
|
||||
English summary at the end.
|
||||
|
||||
---
|
||||
|
||||
## 1. 地址
|
||||
|
||||
ComfyUI 开着的时候，插件自动提供：
|
||||
|
||||
```
|
||||
http://127.0.0.1:8188/anomalous/mcp
|
||||
```
|
||||
|
||||
`8188` 换成你 ComfyUI 实际的端口。不需要装别的东西，也不占内存和显存：没人连接的时候它什么也不做。
|
||||
|
||||
**只有运行 ComfyUI 的这台电脑能连。** 别的电脑、经过反向代理的请求、网页里的脚本都会被拒绝。
|
||||
|
||||
## 2. 在 AI 软件里添加
|
||||
|
||||
**Cherry Studio**：设置 → MCP → 添加 → 快速创建，类型选 **可流式传输的 HTTP（streamableHttp）**，URL 填上面的地址。需要较新的版本。
|
||||
|
||||
**Claude Code**：
|
||||
|
||||
```bash
|
||||
claude mcp add --transport http anomalous http://127.0.0.1:8188/anomalous/mcp
|
||||
```
|
||||
|
||||
**Cursor**：在 `~/.cursor/mcp.json` 里加：
|
||||
|
||||
```json
|
||||
{ "mcpServers": { "anomalous": { "url": "http://127.0.0.1:8188/anomalous/mcp" } } }
|
||||
```
|
||||
|
||||
**VS Code（Copilot）**：在项目的 `.vscode/mcp.json` 里加：
|
||||
|
||||
```json
|
||||
{ "servers": { "anomalous": { "type": "http", "url": "http://127.0.0.1:8188/anomalous/mcp" } } }
|
||||
```
|
||||
|
||||
**Claude 桌面版**：它的配置文件只能启动本地程序，要借 `mcp-remote` 转一下（需要装 Node.js）。在 `claude_desktop_config.json` 里加：
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"anomalous": {
|
||||
"command": "npx",
|
||||
"args": ["mcp-remote", "http://127.0.0.1:8188/anomalous/mcp", "--transport", "http-only"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 3. 可以这样问
|
||||
|
||||
- “我有哪些 Flux 的 LoRA？各自的触发词是什么？”
|
||||
- “找找用过『某某』这个 LoRA 的出图，看看当时的种子和采样器。”
|
||||
- “哪些模型还没在 C 站匹配上？为什么？”
|
||||
- “我上周存的那个搭配用的是哪个底模？”
|
||||
- “给当前工作流加上那个赛博朋克 LoRA，强度 0.7，再把触发词写进正向提示词。”
|
||||
- “把我的搭配『夜景』摆到画布上。”
|
||||
- “这个工作流缺哪些模型？能找回的都找回来。”
|
||||
- “用阿罗娜读一下：老师，欢迎回来！”
|
||||
|
||||
改画布需要 ComfyUI 在这台电脑的浏览器里开着（就是你平时用的那个页面）。开了好几个标签页的话，由当前正在看的那个来执行。
|
||||
|
||||
## 4. 能查什么
|
||||
|
||||
| 工具 | 查什么 |
|
||||
|---|---|
|
||||
| library_overview | 各类模型的数量、底模分布、扫描情况，存了多少搭配、工作流和提示词 |
|
||||
| search_models / get_model | 按名字、触发词、底模、来源找模型；看某个模型的 C 站信息、描述、你写的备注，可以附上封面 |
|
||||
| scan_report | 哪些模型匹配上了 C 站，哪些没有、原因是什么，哪些还没扫，上次扫描做了什么 |
|
||||
| search_images / get_image_info | 按提示词、模型、参数找出图；看某张图的全部生成参数，可以附上缩略图 |
|
||||
| list_combos | 你的搭配（主模型 + LoRA + 提示词） |
|
||||
| list_workflows | 你的工作流配方，带标签、备注和主要设置 |
|
||||
| list_saved_prompts | 提示词工坊里存的提示词 |
|
||||
| search_audio | 用 Anomalous TTS 生成过的语音，带台词和角色 |
|
||||
| recent_activity | 插件最近改过什么（操作记录） |
|
||||
|
||||
能动手的：
|
||||
|
||||
| 工具 | 做什么 |
|
||||
|---|---|
|
||||
| describe_canvas | 读当前工作流：每个节点的编号、类型、设置，哪个是正向/反向提示词框 |
|
||||
| set_prompt | 改写或追加正向/反向提示词 |
|
||||
| set_model | 换某个加载器节点里的模型 |
|
||||
| add_lora | 插入一个 LoRA，默认接在主模型加载器和已有的 LoRA 后面。LoRA 文件里练过文本编码器的（SD1.5、SDXL、Illustrious 的大多是），CLIP 线也接上：从 checkpoint 接，或者从 UNet 工作流里单独的 CLIP 加载器接，正反向提示词一起改接到它后面；没练过的（多数 Flux、Anima 的 LoRA）只接模型线 |
|
||||
| place_combo | 把一个搭配作为一组新节点摆到画布上，原有节点不动 |
|
||||
| check_workflow_models / fix_workflow_models | 检查缺失的模型；把能确定找回的（同一个文件）放回去，“可能是”的留给你在模型检查里确认 |
|
||||
| run_workflow | 像按“运行”一样把工作流加入队列 |
|
||||
| open_in_anomalous | 在插件里打开某个模型的详情页给你看 |
|
||||
| scan_models | 扫描新模型、重新查找没匹配上的，或扫描指定的模型（会联网访问 Civitai） |
|
||||
| download_missing_models | 下载当前工作流缺的模型：只下能确定来源的（C 站按指纹找到、或工作流带着文件链接），放在设置里“模型下载”说的位置，核对指纹后自动换上 |
|
||||
| download_model | 按链接（C 站模型版本、Hugging Face / GitHub 文件）或 C 站指纹下载一个模型到指定类型的文件夹，不覆盖已有文件 |
|
||||
| download_status | 看下载进度和结果 |
|
||||
| list_voices / speak | 列出 Anomalous TTS 的角色；让某个角色读一段台词并保存（最多等两分钟） |
|
||||
|
||||
## 5. 注意
|
||||
|
||||
- **隐私**：如果 AI 软件用的是云端模型，查询结果（模型名、提示词、图片的生成参数）会发给那家 AI 服务商。附上封面或缩略图时，图片也会一起发过去。介意的话可以用本地模型，或者不让 AI 附图。
|
||||
- 附图时会用到插件自己的缩略图缓存，和浏览图库时一样，不动你的原图。
|
||||
- 改模型的名字和备注、删除东西、物理改文件名，这些 AI 做不了，得在插件里自己点。
|
||||
|
||||
---
|
||||
|
||||
## English
|
||||
|
||||
While ComfyUI runs, Anomalous serves an MCP endpoint at `http://127.0.0.1:8188/anomalous/mcp` (use your ComfyUI port). Add it as a Streamable HTTP server in your AI app (Claude Code: `claude mcp add --transport http anomalous <url>`; Cursor and VS Code: a `url` / `"type": "http"` entry; Claude Desktop: through `npx mcp-remote <url> --transport http-only`). Both the 2026-07-28 protocol and the earlier `initialize`-based versions are served.
|
||||
|
||||
Reading tools: library overview, model search and details (optionally with the cover), scan status, output-image search and generation settings (optionally with a thumbnail), combos, workflow recipes, saved prompts, generated audio and the activity log. Acting tools: read the open workflow, write prompts, swap a node's model, add a LoRA, place a combo, check and fix missing models, run the workflow, open a model in Anomalous, start scans, list Anomalous TTS voices and have one speak. Canvas actions need ComfyUI open in a browser on this computer; each is one Ctrl+Z step and is logged as the AI's. Nothing deletes or renames files. Only this computer may connect; requests through a proxy, from other hosts or from web pages are refused. When your AI app uses a cloud model, what the tools return (and any attached picture) is sent to that provider.
|
||||
@@ -0,0 +1,60 @@
|
||||
# 测试怎么跑
|
||||
|
||||
`tests/` 里是插件自己的测试：Python 测试检查后端（扫描、文件读写、回收站、模型下载、MCP 等），`.mjs` 测试检查前端里不依赖浏览器的规则（提示词词块、搭配的空位、往画布插 LoRA、撤销等）。改代码或提 PR 之前跑一遍，就知道有没有弄坏别的地方。
|
||||
|
||||
English summary at the end.
|
||||
|
||||
---
|
||||
|
||||
## 1. 准备
|
||||
|
||||
- 插件要装在一个 ComfyUI 的 `custom_nodes` 里。Python 测试会用到 ComfyUI 自己的模块（`folder_paths`、`server` 等）和它带的包（aiohttp、Pillow、av）。
|
||||
- 装好 [Node.js](https://nodejs.org/)（18 或更新）。
|
||||
- 不需要联网，不需要显卡，也不会碰你真实的模型库：测试只在临时文件夹里建假文件，跑完就删。
|
||||
|
||||
## 2. 一条命令全跑
|
||||
|
||||
在插件文件夹（`custom_nodes/Anomalous_Model_Browser`）里打开终端：
|
||||
|
||||
```bash
|
||||
node tools/run_tests.mjs
|
||||
```
|
||||
|
||||
它依次跑结构检查、每个 `tests/*.mjs`、全部 Python 测试，最后写 `All passed.` 或者列出失败的项目。
|
||||
|
||||
Python 用哪个：
|
||||
|
||||
- 设置了环境变量 `PYTHON` 就用它；
|
||||
- 否则用便携版 ComfyUI 的 `python_embeded`（就在 ComfyUI 文件夹旁边）；
|
||||
- 都没有就用系统的 `python`。用 venv 或 conda 装的 ComfyUI，先激活那个环境再跑。
|
||||
|
||||
## 3. 只跑一部分
|
||||
|
||||
```bash
|
||||
# 结构检查：每个源码文件都登记在架构文档里、前端模块都能被引用到、文件长度不超线、CSS 没有按 style 属性匹配的 :has()
|
||||
node tools/check_structure.mjs
|
||||
|
||||
# 一个前端测试
|
||||
node tests/combo_slots.mjs
|
||||
|
||||
# 全部 Python 测试（便携版 ComfyUI 的写法；其他安装方式换成自己的 python）
|
||||
../../../python_embeded/python.exe -B -m unittest discover -s tests -p "test_*.py"
|
||||
|
||||
# 一个 Python 测试文件
|
||||
../../../python_embeded/python.exe -B -m unittest discover -s tests -p "test_model_download.py"
|
||||
```
|
||||
|
||||
## 4. 写新测试
|
||||
|
||||
- 前端规则测试放 `tests/<名字>.mjs`，用 `node:assert`，直接导入 `web/modules/` 里的真实模块，只替换浏览器和 ComfyUI 这些外部部分。
|
||||
- 后端测试放 `tests/test_<名字>.py`，用 `unittest`。文件一律建在 `tempfile` 的临时文件夹里，网络请求用 `unittest.mock` 换掉。
|
||||
- 测试里不要写自己电脑上的真实路径、真实模型名或个人信息。
|
||||
- 修 bug 时，先写一个能重现这个 bug 的测试。
|
||||
|
||||
---
|
||||
|
||||
## English summary
|
||||
|
||||
`tests/` holds the plugin's own tests: Python tests for the backend and `.mjs` tests for the browser-free frontend rules. They need the plugin inside a ComfyUI `custom_nodes` folder (the Python tests import ComfyUI's modules) and Node.js 18 or newer; no network, no GPU, and they only touch temporary folders.
|
||||
|
||||
Run everything from the plugin folder with `node tools/run_tests.mjs`: the structure check, every `tests/*.mjs`, then all Python tests with `$PYTHON`, the portable build's `python_embeded`, or `python`. Single parts: `node tools/check_structure.mjs`, `node tests/<name>.mjs`, `python -B -m unittest discover -s tests -p "test_*.py"`. New tests import the real modules and replace only outside systems; keep real paths and personal data out of them.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 449 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 295 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 69 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 117 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 165 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
+38
-2
@@ -52,8 +52,44 @@ def is_unmatched(info):
|
||||
|
||||
|
||||
# Why an unmatched model has no Civitai record: Civitai does not know the file; Civitai could
|
||||
# not be reached; the scan was offline. Scans before this was recorded leave it empty.
|
||||
UNMATCHED_REASONS = ("not_found", "network", "offline")
|
||||
# not be reached; the scan was offline; the file's header is damaged (safetensors_damage), so
|
||||
# neither Civitai nor the header can tell what it is. Scans before this was recorded leave it empty.
|
||||
UNMATCHED_REASONS = ("not_found", "network", "offline", "damaged")
|
||||
MAX_HEADER_BYTES = 100 * 1024 * 1024
|
||||
|
||||
|
||||
def safetensors_damage(file_path):
|
||||
"""What is wrong with a .safetensors file's header, or "" when it reads whole: "short" (no
|
||||
room for the header's length), "header_size" (longer than the file), "header" (not the JSON
|
||||
of a tensor list) or "truncated" (a tensor ends past the file's end: a download cut short).
|
||||
Reads only the header. A file that cannot be opened now says nothing ("")."""
|
||||
try:
|
||||
size = os.path.getsize(file_path)
|
||||
with open(file_path, "rb") as handle:
|
||||
head = handle.read(8)
|
||||
if len(head) < 8:
|
||||
return "short"
|
||||
header_size = struct.unpack("<Q", head)[0]
|
||||
if not header_size or header_size > MAX_HEADER_BYTES or 8 + header_size > size:
|
||||
return "header_size"
|
||||
header = json.loads(handle.read(header_size).decode("utf-8"))
|
||||
except OSError:
|
||||
return ""
|
||||
except ValueError: # bad UTF-8 or JSON
|
||||
return "header"
|
||||
if not isinstance(header, dict):
|
||||
return "header"
|
||||
data_size = size - 8 - header_size
|
||||
for name, tensor in header.items():
|
||||
if name == "__metadata__":
|
||||
continue
|
||||
offsets = tensor.get("data_offsets") if isinstance(tensor, dict) else None
|
||||
if not (isinstance(offsets, list) and len(offsets) == 2 and all(isinstance(value, int) for value in offsets)
|
||||
and 0 <= offsets[0] <= offsets[1]):
|
||||
return "header"
|
||||
if offsets[1] > data_size:
|
||||
return "truncated"
|
||||
return ""
|
||||
|
||||
|
||||
def unmatched_reason(info):
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"""The proxy the plugin's own web requests go through, and why a request failed.
|
||||
|
||||
Settings -> Network keeps a proxy address in api/config.json (NETWORK_PROXY). When it is set,
|
||||
every request of the plugin (Civitai, Hugging Face, GitHub, the translators) goes through it,
|
||||
and the scan process gets it as HTTP(S)_PROXY. When it is empty, urllib finds the proxy itself:
|
||||
HTTPS_PROXY, else the Windows proxy (not a PAC script, not a browser extension). ComfyUI's own
|
||||
requests and other node packs are not touched.
|
||||
|
||||
`failure()` names why a request failed in a few fixed words the pages explain. No imports from
|
||||
ComfyUI or the api package, so the scan process (scraper.py) uses it too.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import socket
|
||||
import ssl
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
CONFIG_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "api", "config.json")
|
||||
CONFIG_KEY = "NETWORK_PROXY"
|
||||
_ADDRESS = re.compile(r"^(?:(https?)://)?([A-Za-z0-9.\-]+|\[[0-9A-Fa-f:]+\]):(\d{1,5})/?$", re.IGNORECASE)
|
||||
|
||||
|
||||
def normalize(text) -> str:
|
||||
"""'' or 'http://host:port'. ValueError('socks') for a SOCKS address (urllib cannot use
|
||||
one; proxy apps also offer an HTTP port), ValueError('format') for anything else."""
|
||||
value = str(text or "").strip()
|
||||
if not value:
|
||||
return ""
|
||||
if value.lower().startswith("socks"):
|
||||
raise ValueError("socks")
|
||||
match = _ADDRESS.match(value)
|
||||
if not match or not 0 < int(match.group(3)) < 65536:
|
||||
raise ValueError("format")
|
||||
return f"{(match.group(1) or 'http').lower()}://{match.group(2)}:{int(match.group(3))}"
|
||||
|
||||
|
||||
def configured() -> str:
|
||||
"""The saved proxy address, or '' (none saved, or the file unreadable)."""
|
||||
try:
|
||||
with open(CONFIG_PATH, encoding="utf-8") as f:
|
||||
return normalize(json.load(f).get(CONFIG_KEY, ""))
|
||||
except (OSError, ValueError, AttributeError):
|
||||
return ""
|
||||
|
||||
|
||||
def urlopen(request, timeout, context=None):
|
||||
"""urllib.request.urlopen through the saved proxy, when there is one."""
|
||||
proxy = configured()
|
||||
if not proxy:
|
||||
if context is None:
|
||||
return urllib.request.urlopen(request, timeout=timeout)
|
||||
return urllib.request.urlopen(request, timeout=timeout, context=context)
|
||||
handlers = [urllib.request.ProxyHandler({"http": proxy, "https": proxy})]
|
||||
if context is not None:
|
||||
handlers.append(urllib.request.HTTPSHandler(context=context))
|
||||
return urllib.request.build_opener(*handlers).open(request, timeout=timeout)
|
||||
|
||||
|
||||
def child_env() -> dict:
|
||||
"""The environment for a process of ours (the scan) that should use the same proxy."""
|
||||
env = dict(os.environ)
|
||||
proxy = configured()
|
||||
if proxy:
|
||||
env.update(HTTP_PROXY=proxy, HTTPS_PROXY=proxy, http_proxy=proxy, https_proxy=proxy)
|
||||
return env
|
||||
|
||||
|
||||
def failure(error) -> str:
|
||||
"""'timeout' | 'refused' | 'dns' | 'ssl' | 'reset' | 'proxy' | 'http:<code>' | 'other'."""
|
||||
if isinstance(error, urllib.error.HTTPError):
|
||||
return f"http:{error.code}"
|
||||
reason = error.reason if isinstance(error, urllib.error.URLError) else error
|
||||
if isinstance(reason, (socket.timeout, TimeoutError)):
|
||||
return "timeout"
|
||||
if isinstance(reason, ConnectionRefusedError):
|
||||
return "refused"
|
||||
if isinstance(reason, socket.gaierror):
|
||||
return "dns"
|
||||
if isinstance(reason, (ssl.SSLError, ssl.CertificateError)):
|
||||
return "ssl"
|
||||
if isinstance(reason, (ConnectionResetError, ConnectionAbortedError)):
|
||||
return "reset"
|
||||
if "tunnel connection failed" in str(reason).lower():
|
||||
return "proxy"
|
||||
if "timed out" in str(reason).lower():
|
||||
return "timeout"
|
||||
return "other"
|
||||
+9
-3
@@ -19,7 +19,7 @@ import argparse
|
||||
import shutil
|
||||
from model_policies import is_physical_rename_protected
|
||||
from recycle_bin import move_to_trash
|
||||
from model_identity import USER_INFO_SUFFIX, computed_file_identity, file_sha256, infer_base_model_from_header, is_unmatched, scan_info_path, sidecar_file_hash, sidecar_info as read_local_info, unmatched_reason
|
||||
from model_identity import USER_INFO_SUFFIX, computed_file_identity, file_sha256, infer_base_model_from_header, is_unmatched, safetensors_damage, scan_info_path, sidecar_file_hash, sidecar_info as read_local_info, unmatched_reason
|
||||
from civitai_client import CIVITAI_API_KEY, CivitaiUnreachable, download_media, fetch_civitai_info, fetch_model_description
|
||||
|
||||
|
||||
@@ -136,12 +136,12 @@ def ask_civitai(args, state, file_hash):
|
||||
return None, "network"
|
||||
try:
|
||||
data = fetch_civitai_info(file_hash)
|
||||
except CivitaiUnreachable:
|
||||
except CivitaiUnreachable as unreachable:
|
||||
state["network_failures"] += 1
|
||||
if state["network_failures"] >= CIVITAI_DOWN_AFTER:
|
||||
state["civitai_down"] = True
|
||||
print("\033[93m[!] 连续几个模型都连不上 Civitai，本次其余模型改为从文件推断，之后可以“重新联网查找”。\033[0m")
|
||||
write_report(args, event="civitai_down")
|
||||
write_report(args, event="civitai_down", reason=str(unreachable))
|
||||
return None, "network"
|
||||
state["network_failures"] = 0
|
||||
return data, ("" if data else "not_found")
|
||||
@@ -181,6 +181,12 @@ def scan_one(args, state, rename_log, root, filename, previous):
|
||||
file_hash = sidecar_file_hash(previous, file_path, (previous.get("files") or [{}])[0])[0]
|
||||
file_hash = file_hash or calculate_sha256(file_path)
|
||||
civitai_data, reason = ask_civitai(args, state, file_hash)
|
||||
# A damaged header changes the file's hash and hides its tensors: say so instead of
|
||||
# "Civitai does not know it", so the user downloads it again.
|
||||
damage = "" if civitai_data else safetensors_damage(file_path)
|
||||
if damage:
|
||||
print(f"\033[91m[-] 文件头损坏 ({damage})，文件可能不完整或被改坏了，建议重新下载: {filename}\033[0m")
|
||||
reason = "damaged"
|
||||
|
||||
# What was read from disk is not fetched again: only fresh Civitai data needs the model
|
||||
# page's description and the cover.
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
// node tests/activity_diff.mjs — canvas snapshots and their difference for the activity log.
|
||||
import assert from 'node:assert/strict';
|
||||
import { changeNodeId, diffSnapshots, snapshotGraph } from '../web/modules/activity_diff.js';
|
||||
|
||||
const graph = (nodes) => ({ _nodes: nodes });
|
||||
const node = (id, title, widgets) => ({ id, type: title, title, widgets: Object.entries(widgets).map(([name, value]) => ({ name, value })) });
|
||||
|
||||
const before = snapshotGraph(graph([
|
||||
node(3, 'KSampler', { steps: 20, cfg: 7, seed: 1 }),
|
||||
node(4, 'CheckpointLoaderSimple', { ckpt_name: 'a.safetensors' }),
|
||||
node(9, 'Note', { text: 'hello' }),
|
||||
]));
|
||||
const after = snapshotGraph(graph([
|
||||
node(3, 'KSampler', { steps: 30, cfg: 7, seed: 1 }),
|
||||
node(4, 'CheckpointLoaderSimple', { ckpt_name: 'b.safetensors' }),
|
||||
node(12, 'LoraLoader', { lora_name: 'x.safetensors', strength_model: 1 }),
|
||||
]));
|
||||
|
||||
assert.deepEqual(diffSnapshots(before, after).map(c => [c.kind, c.node, c.widget, c.before, c.after]), [
|
||||
['changed', 'KSampler #3', 'steps', '20', '30'],
|
||||
['changed', 'CheckpointLoaderSimple #4', 'ckpt_name', 'a.safetensors', 'b.safetensors'],
|
||||
['added', 'LoraLoader #12', undefined, undefined, undefined],
|
||||
['removed', 'Note #9', undefined, undefined, undefined],
|
||||
]);
|
||||
assert.deepEqual(diffSnapshots(before, before), []);
|
||||
|
||||
// Buttons and nameless widgets hold no value.
|
||||
const withButton = snapshotGraph(graph([{ id: 1, type: 'X', title: 'X', widgets: [{ name: 'run', type: 'button', value: 1 }, { value: 5 }] }]));
|
||||
assert.deepEqual(withButton.get('1').widgets, {});
|
||||
|
||||
// Object values compare by content; a widget that appears counts as a change.
|
||||
const a = snapshotGraph(graph([node(1, 'Img', { image: { name: 'a.png' } })]));
|
||||
const b = snapshotGraph(graph([node(1, 'Img', { image: { name: 'a.png' }, extra: 2 })]));
|
||||
assert.deepEqual(diffSnapshots(a, b).map(c => [c.widget, c.before, c.after]), [['extra', '', '2']]);
|
||||
|
||||
assert.equal(changeNodeId({ node: 'KSampler #12' }), '12');
|
||||
assert.equal(changeNodeId({ node: 'Title with # sign #7' }), '7');
|
||||
assert.equal(changeNodeId({ node: 'no id' }), null);
|
||||
console.log('activity_diff: ok');
|
||||
@@ -0,0 +1,70 @@
|
||||
// Run: node tests/audio_engines.mjs (fetch is stubbed; no ComfyUI needed)
|
||||
import assert from 'node:assert/strict';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
import path from 'node:path';
|
||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
|
||||
const calls = [];
|
||||
const character = {
|
||||
name: '阿罗娜/日配', aliases: ['阿罗娜'], language: 'ja', settings: { format: 1, keep: 1 },
|
||||
counts: { gpt: 1, sovits: 1, audio: 2 },
|
||||
reference: { audio: 'ref/a.wav', text: 'こんにちは', language: 'ja', source: 'auto' },
|
||||
emotions: { 开心: { audio: 'ref/a.开心.wav', text: '', language: 'ja', source: 'filename' } },
|
||||
};
|
||||
globalThis.fetch = async (url) => {
|
||||
calls.push(url);
|
||||
const json = (data, status = 200) => ({ ok: status < 400, status, json: async () => data });
|
||||
if (url.startsWith('/object_info/')) {
|
||||
const cls = decodeURIComponent(url.slice('/object_info/'.length));
|
||||
return json(cls === 'AnomalousTTS_CharacterSpeech' ? { [cls]: {} } : {});
|
||||
}
|
||||
if (url.startsWith('/anomalous_tts/characters?name=')) {
|
||||
return json({ format: 2, character: { ...character, audio: ['ref/a.wav', 'ref/a.开心.wav'] } });
|
||||
}
|
||||
if (url.startsWith('/anomalous_tts/characters')) return json({ format: 2, characters: [character, { name: 'broken', error: 'x' }] });
|
||||
return json(null, 404);
|
||||
};
|
||||
|
||||
const engines = await import(pathToFileURL(path.join(root, 'web/modules/audio_engines.js')).href);
|
||||
|
||||
// Summary list → voice groups; main first, filename emotions kept.
|
||||
const first = await engines.loadVoices();
|
||||
assert.equal(first.installed, true);
|
||||
assert.equal(first.error, '');
|
||||
const arona = first.groups.find(g => g.character === '阿罗娜/日配');
|
||||
assert.deepEqual(arona.slices.map(s => s.emotion), ['main', '开心']);
|
||||
assert.equal(arona.has_main, true);
|
||||
assert.equal(first.groups.find(g => g.character === 'broken').has_main, false);
|
||||
|
||||
// Re-rendering (sidebar click, studio + sidebar together) does not hit the server again.
|
||||
const before = calls.length;
|
||||
await Promise.all([engines.loadVoices(), engines.loadVoices(), engines.isTtsInstalled()]);
|
||||
assert.equal(calls.length, before, `unexpected requests: ${calls.slice(before)}`);
|
||||
assert.equal(await engines.isTtsInstalled(), true);
|
||||
assert.ok(!calls.some(url => url.includes('F5TTS')), 'no F5-TTS probe');
|
||||
|
||||
// Refresh asks the node to rescan once; later loads use its cache again.
|
||||
engines.invalidateEngineCache({ rescan: true });
|
||||
await engines.loadVoices();
|
||||
assert.ok(calls.includes('/anomalous_tts/characters?refresh=1'));
|
||||
engines.invalidateEngineCache();
|
||||
await engines.loadVoices();
|
||||
assert.equal(calls.at(-1), '/anomalous_tts/characters');
|
||||
|
||||
// File lists come from the per-character detail.
|
||||
const detail = await engines.fetchGptSovitsCharacter('阿罗娜/日配');
|
||||
assert.deepEqual(detail.audio, ['ref/a.wav', 'ref/a.开心.wav']);
|
||||
assert.equal(calls.at(-1), `/anomalous_tts/characters?name=${encodeURIComponent('阿罗娜/日配')}`);
|
||||
|
||||
// Saving keeps fields Anomalous does not know.
|
||||
const merged = engines.mergeGptSovitsSettings({ format: 1, keep: 1, reference: { audio: 'old.wav', language: 'ja' } }, {
|
||||
main: { audio: 'ref/a.wav', text: '' },
|
||||
emotions: [{ name: '开心', audio: 'ref/a.开心.wav', text: 'x' }, { name: 'main', audio: 'y.wav' }],
|
||||
});
|
||||
assert.deepEqual(merged, {
|
||||
format: 1, keep: 1,
|
||||
reference: { audio: 'ref/a.wav', language: 'ja' },
|
||||
emotions: { 开心: { audio: 'ref/a.开心.wav', text: 'x' } },
|
||||
});
|
||||
|
||||
console.log('audio_engines: ok');
|
||||
@@ -0,0 +1,37 @@
|
||||
// Run: node tests/audio_node_targets.mjs (pure rules, no DOM)
|
||||
import assert from 'node:assert/strict';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
import path from 'node:path';
|
||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const { planVoiceDrop, targetForNode } =
|
||||
await import(pathToFileURL(path.join(root, 'web/modules/audio_node_targets.js')).href);
|
||||
|
||||
const ttsNode = (values = ['阿罗娜/日配数据集制', '普拉娜']) => ({
|
||||
type: 'AnomalousTTS_CharacterSpeech',
|
||||
widgets: [{ name: 'character', value: '', options: { values } }, { name: 'text', value: '' }],
|
||||
});
|
||||
const group = { character: '阿罗娜/日配数据集制', node_value: '阿罗娜/日配数据集制', has_main: true };
|
||||
|
||||
// A character name goes into `character`, in the node's own spelling.
|
||||
const plan = planVoiceDrop(ttsNode(), group);
|
||||
assert.equal(plan.ok, true);
|
||||
assert.equal(plan.value, '阿罗娜/日配数据集制');
|
||||
assert.equal(plan.widget.name, 'character');
|
||||
assert.equal(planVoiceDrop(ttsNode(['阿罗娜\\日配数据集制']), group).value, '阿罗娜\\日配数据集制', 'Windows spelling');
|
||||
assert.equal(targetForNode(ttsNode()).speechWidget, 'text');
|
||||
assert.equal(targetForNode({ comfyClass: 'AnomalousTTS_CharacterSpeech' })?.id, 'anomalous-tts');
|
||||
|
||||
// Anything not listed is refused, even with a matching widget name; Load Audio and F5-TTS included.
|
||||
const lookalike = { type: 'SomeOtherTTS', widgets: [{ name: 'character', options: { values: [group.node_value] } }] };
|
||||
for (const node of [lookalike, { type: 'LoadAudio', widgets: [] }, { type: 'F5TTSAudio', widgets: [] }]) {
|
||||
assert.equal(targetForNode(node), null, node.type);
|
||||
assert.equal(planVoiceDrop(node, group).reason, 'unsupported', node.type);
|
||||
}
|
||||
assert.equal(planVoiceDrop(null, group).reason, 'unsupported');
|
||||
|
||||
// Refusals that explain themselves.
|
||||
assert.equal(planVoiceDrop(ttsNode(['普拉娜']), group).reason, 'notListed');
|
||||
assert.equal(planVoiceDrop(ttsNode(['普拉娜']), group).path, group.node_value);
|
||||
assert.equal(planVoiceDrop(ttsNode(), { ...group, has_main: false }).reason, 'noMainVoice');
|
||||
assert.equal(planVoiceDrop({ type: 'AnomalousTTS_CharacterSpeech', widgets: [] }, group).reason, 'unsupported');
|
||||
console.log('audio node target rules OK');
|
||||
@@ -0,0 +1,85 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { buildScriptPackage, buildTtsPrompt, comboValueForPath, composeScript, joinSegments, parseTaggedSpeech, splitScriptLines, splitSegmentAt, ttsOutputPrefix, usableEmotions } from '../web/modules/audio_script.js';
|
||||
|
||||
// Sentence splitting keeps decimals and abbreviations without trailing space.
|
||||
assert.deepEqual(splitScriptLines('老师好！今天3.5点开会。\nOK. Next line?'), ['老师好！', '今天3.5点开会。', 'OK.', 'Next line?']);
|
||||
assert.deepEqual(splitScriptLines(' \n\n '), []);
|
||||
// Ellipses never split; stacked marks and closing quotes stay with their sentence;
|
||||
// punctuation-only pieces are attached to a neighbour instead of becoming a card.
|
||||
assert.deepEqual(splitScriptLines('嗯……好吧。'), ['嗯……好吧。']);
|
||||
assert.deepEqual(splitScriptLines('我……我不知道。那就这样吧！'), ['我……我不知道。', '那就这样吧！']);
|
||||
assert.deepEqual(splitScriptLines('真的吗？！」他说。'), ['真的吗？！」', '他说。']);
|
||||
assert.deepEqual(splitScriptLines('你好。\n……\n再见。'), ['你好。……', '再见。']);
|
||||
assert.deepEqual(splitScriptLines('……你好。'), ['……你好。']);
|
||||
assert.deepEqual(splitScriptLines('Wait... what? OK.'), ['Wait... what?', 'OK.']);
|
||||
assert.deepEqual(splitScriptLines('（笑）好的。'), ['（笑）好的。']);
|
||||
assert.deepEqual(splitScriptLines('好的。\n！！', 'line'), ['好的。！！']);
|
||||
// Manual split at the caret.
|
||||
assert.deepEqual(splitSegmentAt('早上好！今天开会。', 4), ['早上好！', '今天开会。']);
|
||||
assert.equal(splitSegmentAt('早上好！', 0), null);
|
||||
assert.equal(splitSegmentAt('早上好！', 4), null);
|
||||
// Line mode keeps a whole paragraph line as one segment.
|
||||
assert.deepEqual(splitScriptLines('老师好！今天开会。\n\n好开心！', 'line'), ['老师好！今天开会。', '好开心！']);
|
||||
assert.equal(joinSegments('老师好！', '今天开会。'), '老师好！今天开会。');
|
||||
assert.equal(joinSegments('Hello.', 'World'), 'Hello. World');
|
||||
assert.equal(joinSegments('', 'x'), 'x');
|
||||
|
||||
// Once a line is tagged, untagged lines must be pinned to {main}; tags are single-braced.
|
||||
assert.equal(composeScript([
|
||||
{ text: '早上好', voice: null },
|
||||
{ text: '好开心', voice: { tag: '{happy}' } },
|
||||
{ text: '继续', voice: null },
|
||||
]), '{main} 早上好\n{happy} 好开心\n{main} 继续');
|
||||
assert.equal(composeScript([{ text: 'a', voice: null }, { text: 'b', voice: null }]), 'a\nb');
|
||||
|
||||
const slice = emotion => ({ emotion });
|
||||
const arona = { group: 'gpt_sovits:Arona', node_value: 'Arona', has_main: true, slices: [slice('main'), slice('happy')] };
|
||||
assert.deepEqual(usableEmotions(arona), ['main', 'happy']);
|
||||
|
||||
// A bundle names the character and tags each line with its emotion.
|
||||
assert.deepEqual(buildScriptPackage([
|
||||
{ text: '早上好', emotion: 'main' },
|
||||
{ text: ' 好开心 ', emotion: 'happy' },
|
||||
{ text: ' ', emotion: 'happy' },
|
||||
], arona), { sample: 'Arona', speech: '{main} 早上好\n{happy} 好开心', lineCount: 2 });
|
||||
// All-main scripts stay plain text.
|
||||
assert.equal(buildScriptPackage([{ text: 'a', emotion: 'main' }, { text: 'b' }], arona).speech, 'a\nb');
|
||||
assert.equal(buildScriptPackage([{ text: 'a', emotion: 'main' }], null).error, 'scriptDirectorPickCharacter');
|
||||
assert.equal(buildScriptPackage([{ text: 'a', emotion: 'sad' }], arona).error, 'scriptDirectorVoiceUnusable');
|
||||
assert.equal(buildScriptPackage([{ text: 'a', emotion: 'main' }], { ...arona, has_main: false }).error, 'scriptDirectorMainMissing');
|
||||
assert.equal(buildScriptPackage([{ text: 'a', emotion: 'main' }], { ...arona, slices: [slice('happy')] }).error, 'scriptDirectorMainMissing');
|
||||
assert.equal(buildScriptPackage([{ text: ' ', emotion: 'main' }], arona).error, 'scriptDirectorEmpty');
|
||||
|
||||
// Combo values keep the node's own separator; unknown files are not written.
|
||||
const windowsCombo = { options: { values: ['阿罗娜\\日配', '普拉娜'] } };
|
||||
assert.equal(comboValueForPath(windowsCombo, '阿罗娜/日配'), '阿罗娜\\日配');
|
||||
assert.equal(comboValueForPath(windowsCombo, 'Mika'), null);
|
||||
assert.equal(comboValueForPath({}, 'audio/x.wav'), 'audio/x.wav');
|
||||
assert.equal(comboValueForPath(windowsCombo, ''), null);
|
||||
|
||||
// Saved speech reads back into segments at its tags.
|
||||
assert.deepEqual(parseTaggedSpeech('开场\n{happy} 哇！\n{main} 好'), [
|
||||
{ emotion: 'main', text: '开场' }, { emotion: 'happy', text: '哇！' }, { emotion: 'main', text: '好' },
|
||||
]);
|
||||
assert.deepEqual(parseTaggedSpeech(composeScript([{ text: 'a', voice: { tag: '{sad}' } }, { text: 'b', voice: null }])),
|
||||
[{ emotion: 'sad', text: 'a' }, { emotion: 'main', text: 'b' }]);
|
||||
assert.deepEqual(parseTaggedSpeech(''), []);
|
||||
|
||||
// GPT-SoVITS direct runs: a retaken line gets [take:N] after its emotion tag, only when asked for.
|
||||
const ttsGroup = { node_value: '阿罗娜/日配', slices: [{ emotion: 'main' }, { emotion: '开心' }] };
|
||||
const retaken = [{ text: '一。', emotion: 'main', take: 1 }, { text: '二。', emotion: '开心', take: 3 }, { text: '三。', emotion: 'main', take: 2 }];
|
||||
assert.equal(buildScriptPackage(retaken, ttsGroup, { takes: true }).speech, '{main} 一。\n{开心} [take:3]二。\n{main} [take:2]三。');
|
||||
assert.equal(buildScriptPackage(retaken, ttsGroup).speech, '{main} 一。\n{开心} 二。\n{main} 三。');
|
||||
assert.equal(composeScript([{ text: 'a', voice: null, take: 2 }]), '[take:2]a');
|
||||
|
||||
// The output folder follows the character, with characters Windows refuses replaced.
|
||||
assert.equal(ttsOutputPrefix('阿罗娜/日配'), 'audio/阿罗娜/日配/日配');
|
||||
assert.equal(ttsOutputPrefix('a:b?/..'), 'audio/a_b_/_/_');
|
||||
const prompt = buildTtsPrompt({ character: '阿罗娜/日配', speech: '{main} 一。', seed: 5, language: 'zh', speed: 1.1 });
|
||||
assert.deepEqual(prompt[1], { class_type: 'AnomalousTTS_CharacterSpeech', inputs: { character: '阿罗娜/日配', text: '{main} 一。', seed: 5, language: 'zh', speed: 1.1 } });
|
||||
assert.deepEqual(prompt[2].inputs, { audio: ['1', 0], filename_prefix: 'audio/阿罗娜/日配/日配' });
|
||||
assert.equal(buildTtsPrompt({ character: 'x', speech: 'a', seed: 1, top_k: 10, temperature: 0.8 })[1].inputs.top_k, 10);
|
||||
assert.deepEqual(buildTtsPrompt({ character: 'x', speech: 'a', seed: 1, preview: true })[2], { class_type: 'PreviewAudio', inputs: { audio: ['1', 0] } });
|
||||
assert.equal('preview' in buildTtsPrompt({ character: 'x', speech: 'a', seed: 1, preview: true })[1].inputs, false);
|
||||
|
||||
console.log('audio script rules OK');
|
||||
@@ -0,0 +1,106 @@
|
||||
// node --experimental-vm-modules tests/canvas_undo.mjs — undo from the activity log.
|
||||
import assert from 'node:assert/strict';
|
||||
import { snapshotGraph } from '../web/modules/activity_diff.js';
|
||||
import { canUndo, keepUndo, undoEntry } from '../web/modules/canvas_undo.js';
|
||||
|
||||
globalThis.LiteGraph = { registered_node_types: {} };
|
||||
|
||||
function makeNode(id, type, widgets, inputs = []) {
|
||||
return {
|
||||
id, type, title: type, inputs, outputs: [],
|
||||
widgets: Object.entries(widgets).map(([name, value]) => ({ name, value })),
|
||||
};
|
||||
}
|
||||
function makeGraph(nodes) {
|
||||
const graph = {
|
||||
_nodes: nodes,
|
||||
getNodeById: id => graph._nodes.find(node => String(node.id) === String(id)) || null,
|
||||
remove: node => { graph._nodes = graph._nodes.filter(item => item !== node); },
|
||||
steps: 0,
|
||||
};
|
||||
return graph;
|
||||
}
|
||||
function makeApp(graph, key = 'wf-1') {
|
||||
const tracker = { captureCanvasState() { graph.steps += 1; } };
|
||||
return { graph, canvas: { graph }, extensionManager: { workflow: { activeWorkflow: { key, changeTracker: tracker } } } };
|
||||
}
|
||||
|
||||
// A value change and an added unconnected node: undone together, one Ctrl+Z step.
|
||||
{
|
||||
const sampler = makeNode(3, 'KSampler', { seed: 5, steps: 20, sampler_name: 'euler' });
|
||||
const graph = makeGraph([sampler]);
|
||||
const app = makeApp(graph);
|
||||
const before = snapshotGraph(graph);
|
||||
sampler.widgets[0].value = 6; // a seed changed meanwhile stays
|
||||
sampler.widgets[1].value = 30;
|
||||
sampler.widgets[2].value = 'dpmpp_2m';
|
||||
graph._nodes.push(makeNode(7, 'CLIPTextEncode', { text: 'a cat' }));
|
||||
keepUndo('e1', 'wf-1', before, snapshotGraph(graph));
|
||||
assert.equal(canUndo(app, { id: 'e1' }), true);
|
||||
undoEntry(app, { id: 'e1' });
|
||||
assert.equal(sampler.widgets[0].value, 6);
|
||||
assert.equal(sampler.widgets[1].value, 20);
|
||||
assert.equal(typeof sampler.widgets[1].value, 'number');
|
||||
assert.equal(sampler.widgets[2].value, 'euler');
|
||||
assert.equal(graph.getNodeById(7), null);
|
||||
assert.equal(graph.steps, 1);
|
||||
assert.equal(canUndo(app, { id: 'e1' }), false, 'an entry is undone once');
|
||||
}
|
||||
|
||||
// Changed again since, another workflow open, or the added node wired up: no undo.
|
||||
{
|
||||
const sampler = makeNode(3, 'Steps', { steps: 20 });
|
||||
const graph = makeGraph([sampler]);
|
||||
const before = snapshotGraph(graph);
|
||||
sampler.widgets[0].value = 30;
|
||||
keepUndo('e2', 'wf-1', before, snapshotGraph(graph));
|
||||
assert.equal(canUndo(makeApp(graph, 'wf-2'), { id: 'e2' }), false);
|
||||
sampler.widgets[0].value = 31;
|
||||
assert.equal(canUndo(makeApp(graph), { id: 'e2' }), false);
|
||||
assert.throws(() => undoEntry(makeApp(graph), { id: 'e2' }), /activityUndoGone/);
|
||||
|
||||
const graph2 = makeGraph([]);
|
||||
const before2 = snapshotGraph(graph2);
|
||||
const added = makeNode(5, 'LoraLoader', { lora_name: 'x' });
|
||||
graph2._nodes.push(added);
|
||||
keepUndo('e3', 'wf-1', before2, snapshotGraph(graph2));
|
||||
added.outputs = [{ links: [11] }];
|
||||
assert.equal(canUndo(makeApp(graph2), { id: 'e3' }), false);
|
||||
}
|
||||
|
||||
// Removed nodes or rewired inputs are never kept; nor is an entry without an id.
|
||||
{
|
||||
const a = makeNode(1, 'A', { v: 1 });
|
||||
const b = makeNode(2, 'B', { v: 1 }, [{ link: null }]);
|
||||
const graph = makeGraph([a, b]);
|
||||
const before = snapshotGraph(graph);
|
||||
graph._nodes = [b];
|
||||
keepUndo('e4', 'wf-1', before, snapshotGraph(graph));
|
||||
assert.equal(canUndo(makeApp(graph), { id: 'e4' }), false);
|
||||
|
||||
const graph2 = makeGraph([makeNode(1, 'A', { v: 1 }), b]);
|
||||
const before2 = snapshotGraph(graph2);
|
||||
b.inputs[0].link = 4;
|
||||
b.widgets[0].value = 2;
|
||||
keepUndo('e5', 'wf-1', before2, snapshotGraph(graph2));
|
||||
assert.equal(canUndo(makeApp(graph2), { id: 'e5' }), false);
|
||||
assert.equal(canUndo(makeApp(graph2), {}), false);
|
||||
}
|
||||
// A choice this computer no longer has, or only a seed changed: nothing to offer.
|
||||
{
|
||||
const loader = makeNode(4, 'CheckpointLoaderSimple', { ckpt_name: 'a.safetensors' });
|
||||
loader.widgets[0].options = { values: ['b.safetensors'] };
|
||||
const graph = makeGraph([loader]);
|
||||
const before = snapshotGraph(graph);
|
||||
loader.widgets[0].value = 'b.safetensors';
|
||||
keepUndo('e6', 'wf-1', before, snapshotGraph(graph));
|
||||
assert.equal(canUndo(makeApp(graph), { id: 'e6' }), false);
|
||||
|
||||
const seeded = makeNode(8, 'Noise', { noise_seed: 1 });
|
||||
const graph2 = makeGraph([seeded]);
|
||||
const before2 = snapshotGraph(graph2);
|
||||
seeded.widgets[0].value = 2;
|
||||
keepUndo('e7', 'wf-1', before2, snapshotGraph(graph2));
|
||||
assert.equal(canUndo(makeApp(graph2), { id: 'e7' }), false);
|
||||
}
|
||||
console.log('canvas_undo: ok');
|
||||
@@ -0,0 +1,80 @@
|
||||
// The rules of combo structures that need no canvas (web/modules/combo_slots.js).
|
||||
import assert from 'node:assert/strict';
|
||||
import { chainOrder, findPort, fedByLink, fixedNodes, folderFor, isModelWidget, missingTypes, packOf, structureKey, structureSummary } from '../web/modules/combo_slots.js';
|
||||
|
||||
const combo = (value, values) => ({ type: 'combo', value, options: { values } });
|
||||
|
||||
// Model drop-downs: by the files they list, not by their name.
|
||||
assert.equal(isModelWidget(combo('ae.safetensors', ['ae.safetensors', 'other.sft'])), true);
|
||||
assert.equal(isModelWidget(combo('None', ['None', 'Flux\\a.safetensors'])), true); // a LoRA stack row left empty
|
||||
assert.equal(isModelWidget(combo('flux', ['sdxl', 'sd3', 'flux'])), false); // CLIP type: not a model
|
||||
assert.equal(isModelWidget(combo('x.gguf', () => ['x.gguf'])), true); // options given as a function
|
||||
assert.equal(isModelWidget({ type: 'text', value: 'a.safetensors' }), false);
|
||||
|
||||
// A box turned into an input and fed by a link is no slot.
|
||||
const node = { inputs: [{ name: 'lora_name', widget: { name: 'lora_name' }, link: 7 }, { name: 'model', link: 3 }] };
|
||||
assert.equal(fedByLink(node, { name: 'lora_name' }), true);
|
||||
assert.equal(fedByLink(node, { name: 'strength' }), false);
|
||||
|
||||
// The models folder: the one whose files hold every option ("None" aside).
|
||||
const lists = new Map([
|
||||
['loras', new Set(['Flux/a.safetensors', 'b.safetensors'])],
|
||||
['clip', new Set(['t5.safetensors', 'clip_l.safetensors'])],
|
||||
['text_encoders', new Set(['t5.safetensors', 'clip_l.safetensors'])],
|
||||
['checkpoints', new Set(['sdxl.safetensors'])],
|
||||
['my_loras', new Set(['Flux/a.safetensors', 'b.safetensors', 'c.safetensors'])],
|
||||
]);
|
||||
assert.equal(folderFor(['None', 'Flux\\a.safetensors', 'b.safetensors'], 'None', lists), 'loras'); // known folder before a custom one
|
||||
assert.equal(folderFor(['t5.safetensors', 'clip_l.safetensors'], 't5.safetensors', lists), 'text_encoders'); // clip is the same folder
|
||||
assert.equal(folderFor(['c.safetensors'], 'c.safetensors', lists), 'my_loras');
|
||||
assert.equal(folderFor(['nowhere.safetensors'], 'nowhere.safetensors', lists), '');
|
||||
assert.equal(folderFor([], 'sdxl.safetensors', lists), 'checkpoints'); // no options: the value decides
|
||||
assert.equal(folderFor([], 'None', lists), '');
|
||||
lists.set('vae', new Set(['ae.safetensors']));
|
||||
assert.equal(folderFor(['ae.safetensors', 'pixel_space', 'taesd'], 'ae.safetensors', lists), 'vae'); // built-in choices are no files
|
||||
|
||||
// Link ends: same name and type, else the old place, else the one port of that type.
|
||||
const outputs = [{ name: 'MODEL', type: 'MODEL' }, { name: 'CLIP', type: 'CLIP' }, { name: 'VAE', type: 'VAE' }];
|
||||
assert.equal(findPort(outputs, 'CLIP', 1, 'CLIP'), 1);
|
||||
assert.equal(findPort([{ name: 'VAE', type: 'VAE' }, { name: 'CLIP', type: 'CLIP' }], 'CLIP', 0, 'CLIP'), 1); // moved
|
||||
assert.equal(findPort([{ name: 'clip_out', type: 'CLIP' }], 'CLIP', 1, 'CLIP'), 0); // renamed, one of its type
|
||||
assert.equal(findPort([{ name: 'a', type: 'CLIP' }, { name: 'b', type: 'CLIP' }], 'CLIP', 5, 'CLIP'), -1); // two: no guess
|
||||
assert.equal(findPort(outputs, 'MODEL', 0, 'LATENT'), -1); // wrong type
|
||||
|
||||
// Node packs and missing node types (LiteGraph's registry).
|
||||
globalThis.LiteGraph = { registered_node_types: {
|
||||
UnetLoaderGGUF: { nodeData: { python_module: 'custom_nodes.ComfyUI-GGUF' } },
|
||||
CLIPTextEncode: { nodeData: { python_module: 'nodes' } },
|
||||
} };
|
||||
assert.equal(packOf('UnetLoaderGGUF'), 'ComfyUI-GGUF');
|
||||
assert.equal(packOf('CLIPTextEncode'), '');
|
||||
const structure = { nodes: [{ type: 'UnetLoaderGGUF' }, { type: 'Missing', pack: 'Some-Pack' }, { type: 'Missing', pack: 'Some-Pack' }], links: [], slots: [] };
|
||||
assert.deepEqual(missingTypes(structure), [{ type: 'Missing', pack: 'Some-Pack' }]);
|
||||
|
||||
// Same structure, whatever the values.
|
||||
const one = { nodes: [{ type: 'A', key: 'n1' }], links: [{ from: 'n1', fromName: 'MODEL', to: 'n2', toName: 'model' }], slots: [{ node: 'n1', widget: 'x' }] };
|
||||
assert.equal(structureKey(one), structureKey(JSON.parse(JSON.stringify(one))));
|
||||
assert.notEqual(structureKey(one), structureKey({ ...one, slots: [] }));
|
||||
|
||||
// Repeated nodes are counted in the summary.
|
||||
assert.equal(structureSummary({ nodes: [{ name: 'UNet' }, { name: 'Text' }, { name: 'LoRA' }, { name: 'Text' }, { title: 'Mine', name: 'LoRA' }] }),
|
||||
'UNet + Text ×2 + LoRA + Mine');
|
||||
|
||||
// Flow order: loaders first, a LoRA chain from the model outwards, ties as given, loops last.
|
||||
assert.deepEqual(chainOrder(['p1', 'p2', 'l1', 'l2', 'l3', 'l4', 'unet', 'clip'],
|
||||
[['l3', 'l1'], ['l1', 'p1'], ['l1', 'p2'], ['unet', 'l2'], ['clip', 'l2'], ['l4', 'l3'], ['l2', 'l4']]),
|
||||
['unet', 'clip', 'l2', 'l4', 'l3', 'l1', 'p1', 'p2']);
|
||||
assert.deepEqual(chainOrder(['a', 'b', 'c'], [['a', 'b'], ['b', 'a'], ['x', 'c']]), ['c', 'a', 'b']);
|
||||
|
||||
// Nodes without slots are put back as saved, listed in flow order: an upscale model's loader with a slot,
|
||||
// the node using it and a saver without.
|
||||
const upscale = {
|
||||
nodes: [{ key: 'n1', type: 'SaveImage' }, { key: 'n2', type: 'ImageUpscaleWithModel' }, { key: 'n3', type: 'UpscaleModelLoader' }],
|
||||
links: [{ from: 'n3', to: 'n2' }, { from: 'n2', to: 'n1' }],
|
||||
slots: [{ node: 'n3', widget: 'model_name' }],
|
||||
};
|
||||
assert.deepEqual(fixedNodes(upscale).map(item => item.type), ['ImageUpscaleWithModel', 'SaveImage']);
|
||||
assert.deepEqual(fixedNodes({ ...upscale, slots: [] }).map(item => item.key), ['n3', 'n2', 'n1']);
|
||||
assert.deepEqual(fixedNodes({}), []);
|
||||
|
||||
console.log('combo_slots: ok');
|
||||
@@ -0,0 +1,25 @@
|
||||
// node tests/detached_graph.mjs — a workflow read off the canvas never takes the canvas's graph id.
|
||||
import assert from 'node:assert/strict';
|
||||
import { detachedGraph } from '../web/modules/detached_graph.js';
|
||||
|
||||
const configured = [];
|
||||
globalThis.LiteGraph = { LGraph: class { configure(data) { configured.push(data); } } };
|
||||
|
||||
const workflow = { id: '3675a66f-fd88-4261-ab21-5b285bd54a02', nodes: [{ id: 3, widgets_values: [20] }] };
|
||||
detachedGraph(workflow);
|
||||
detachedGraph(workflow);
|
||||
const [first, second] = configured;
|
||||
assert.match(first.id, /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/);
|
||||
assert.notEqual(first.id, workflow.id);
|
||||
assert.notEqual(first.id, second.id);
|
||||
assert.deepEqual(first.nodes, workflow.nodes);
|
||||
assert.equal(workflow.id, '3675a66f-fd88-4261-ab21-5b285bd54a02'); // the caller's workflow is left as it was
|
||||
|
||||
// Without crypto.randomUUID (a page over plain http on another computer) the id is made by hand.
|
||||
const realCrypto = globalThis.crypto;
|
||||
Object.defineProperty(globalThis, 'crypto', { value: { getRandomValues: array => realCrypto.getRandomValues(array) }, configurable: true });
|
||||
detachedGraph(workflow);
|
||||
assert.match(configured[2].id, /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/);
|
||||
Object.defineProperty(globalThis, 'crypto', { value: realCrypto, configurable: true });
|
||||
|
||||
console.log('detached_graph ok');
|
||||
@@ -0,0 +1,38 @@
|
||||
// Where a downloaded model goes (web/modules/download_places.js).
|
||||
import assert from 'node:assert/strict';
|
||||
import { baseFolder, destinationFor, expandFolder, unsafeFormat } from '../web/modules/download_places.js';
|
||||
|
||||
const entry = (value) => ({ value });
|
||||
const workflow = { place: 'workflow', folder: 'Downloads' };
|
||||
const fixed = { place: 'folder', folder: 'Downloads' };
|
||||
|
||||
// Where the workflow says: its own subfolder (Windows separators too), so the node needs no change.
|
||||
assert.equal(destinationFor(entry('Flux\\style\\ink.safetensors'), {}, workflow), 'Flux/style/ink.safetensors');
|
||||
// A bare name goes into the settings' folder; an empty folder means the type's folder itself.
|
||||
assert.equal(destinationFor(entry('ink.safetensors'), {}, workflow), 'Downloads/ink.safetensors');
|
||||
assert.equal(destinationFor(entry('ink.safetensors'), {}, { place: 'workflow', folder: '' }), 'ink.safetensors');
|
||||
// A set folder wins over the workflow's; the file keeps the workflow's name.
|
||||
assert.equal(destinationFor(entry('Flux/ink.safetensors'), {}, fixed), 'Downloads/ink.safetensors');
|
||||
|
||||
// {base}: the usual folder name of the base model, or the folder you already have for it.
|
||||
assert.equal(baseFolder('SDXL 1.0'), 'SDXL');
|
||||
assert.equal(baseFolder('Flux.1 D'), 'Flux');
|
||||
assert.equal(baseFolder('SD 1.5'), 'SD1.5');
|
||||
assert.equal(baseFolder('Illustrious', ['illustrious', 'pony']), 'illustrious');
|
||||
assert.equal(baseFolder('SD 1.5', ['sd15']), 'sd15');
|
||||
assert.equal(baseFolder('Some: New/Model'), 'Some New Model');
|
||||
assert.equal(baseFolder(''), '');
|
||||
const subfolders = ['Characters', 'Characters/SDXL', 'Downloads', 'Downloads/flux-models', 'NoobAI'];
|
||||
assert.equal(expandFolder('{base}', 'Flux.1 D', subfolders), 'Flux'); // the nested one is not beside it
|
||||
assert.equal(expandFolder('Downloads/{base}', 'Flux.1 D', subfolders), 'Downloads/Flux');
|
||||
assert.equal(expandFolder('downloads/{base}', 'SDXL 1.0', ['Downloads', 'Downloads/sd-xl']), 'downloads/sd-xl');
|
||||
assert.equal(expandFolder('{base}', 'NoobAI', subfolders), 'NoobAI');
|
||||
assert.equal(expandFolder('{base}/loras', '', subfolders), 'loras'); // unknown base: the part goes
|
||||
assert.equal(destinationFor(entry('a.safetensors'), { base_model: 'Pony' }, { place: 'folder', folder: '{base}' }), 'Pony/a.safetensors');
|
||||
|
||||
assert.equal(unsafeFormat('x.ckpt'), true);
|
||||
assert.equal(unsafeFormat('x.PT'), true);
|
||||
assert.equal(unsafeFormat('x.safetensors'), false);
|
||||
assert.equal(unsafeFormat('x.gguf'), false);
|
||||
|
||||
console.log('download_places: ok');
|
||||
@@ -0,0 +1,193 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
normalizeFloatingTriggerSize,
|
||||
normalizeFloatingTriggerStyle,
|
||||
normalizeEntryMode,
|
||||
isValidSavedTriggerPosition,
|
||||
normalizeSavedTriggerPosition,
|
||||
clampFloatingTriggerPosition,
|
||||
loadSavedTriggerPosition,
|
||||
saveTriggerPosition,
|
||||
clearSavedTriggerPosition
|
||||
} from '../web/modules/entry_controls.js';
|
||||
|
||||
test('normalizeFloatingTriggerSize returns default for invalid values', () => {
|
||||
assert.equal(normalizeFloatingTriggerSize('small'), 'small');
|
||||
assert.equal(normalizeFloatingTriggerSize('medium'), 'medium');
|
||||
assert.equal(normalizeFloatingTriggerSize('large'), 'large');
|
||||
assert.equal(normalizeFloatingTriggerSize('invalid'), 'medium');
|
||||
assert.equal(normalizeFloatingTriggerSize(null), 'medium');
|
||||
assert.equal(normalizeFloatingTriggerSize(undefined), 'medium');
|
||||
});
|
||||
|
||||
test('normalizeFloatingTriggerStyle returns default for invalid values', () => {
|
||||
assert.equal(normalizeFloatingTriggerStyle('icon'), 'icon');
|
||||
assert.equal(normalizeFloatingTriggerStyle('pill'), 'pill');
|
||||
assert.equal(normalizeFloatingTriggerStyle('unknown'), 'icon');
|
||||
assert.equal(normalizeFloatingTriggerStyle(null), 'icon');
|
||||
});
|
||||
|
||||
test('normalizeEntryMode returns default for invalid values', () => {
|
||||
assert.equal(normalizeEntryMode('floating'), 'floating');
|
||||
assert.equal(normalizeEntryMode('topbar'), 'topbar');
|
||||
assert.equal(normalizeEntryMode('menu'), 'menu');
|
||||
assert.equal(normalizeEntryMode('other'), 'floating');
|
||||
});
|
||||
|
||||
test('isValidSavedTriggerPosition validates finite numeric coordinates and rejects invalid/null values', () => {
|
||||
// Nullish or empty
|
||||
assert.equal(isValidSavedTriggerPosition(null, null), false);
|
||||
assert.equal(isValidSavedTriggerPosition('100px', null), false);
|
||||
assert.equal(isValidSavedTriggerPosition(null, '100px'), false);
|
||||
assert.equal(isValidSavedTriggerPosition('', ''), false);
|
||||
assert.equal(isValidSavedTriggerPosition('NaN', 'NaN'), false);
|
||||
|
||||
// Negative or non-finite coordinates are rejected
|
||||
assert.equal(isValidSavedTriggerPosition('-10px', '50px'), false);
|
||||
assert.equal(isValidSavedTriggerPosition('50px', '-10px'), false);
|
||||
assert.equal(isValidSavedTriggerPosition('Infinity', '100'), false);
|
||||
|
||||
// Coordinates in dock area (<70px) are rejected as invalid legacy/dirty positions
|
||||
assert.equal(isValidSavedTriggerPosition('0px', '0px'), false);
|
||||
assert.equal(isValidSavedTriggerPosition('50px', '100px'), false);
|
||||
assert.equal(isValidSavedTriggerPosition('69px', '80px'), false);
|
||||
|
||||
// Valid non-negative finite canvas positions outside dock
|
||||
assert.equal(isValidSavedTriggerPosition('70px', '80px'), true);
|
||||
assert.equal(isValidSavedTriggerPosition('100px', '200px'), true);
|
||||
assert.equal(isValidSavedTriggerPosition('500', '300'), true);
|
||||
assert.equal(isValidSavedTriggerPosition('1800px', '900px'), true);
|
||||
});
|
||||
|
||||
test('normalizeSavedTriggerPosition preserves precise user drag coordinates without distortion', () => {
|
||||
const pos1 = normalizeSavedTriggerPosition('75px', '48px');
|
||||
assert.deepEqual(pos1, { x: 75, y: 48 });
|
||||
|
||||
const pos2 = normalizeSavedTriggerPosition('120px', '80px');
|
||||
assert.deepEqual(pos2, { x: 120, y: 80 });
|
||||
|
||||
const pos3 = normalizeSavedTriggerPosition('500px', '400px');
|
||||
assert.deepEqual(pos3, { x: 500, y: 400 });
|
||||
|
||||
// Invalid / null coordinates return null
|
||||
assert.equal(normalizeSavedTriggerPosition(null, null), null);
|
||||
assert.equal(normalizeSavedTriggerPosition('invalid', 'invalid'), null);
|
||||
assert.equal(normalizeSavedTriggerPosition('30px', '100px'), null);
|
||||
});
|
||||
|
||||
test('clampFloatingTriggerPosition calculates safe top-left fallback positions when coordinates are absent', () => {
|
||||
const pos = clampFloatingTriggerPosition({
|
||||
x: null,
|
||||
y: null,
|
||||
width: 60,
|
||||
height: 60,
|
||||
viewportWidth: 1920,
|
||||
viewportHeight: 1080,
|
||||
margin: 0
|
||||
});
|
||||
// Falls back cleanly to top-left safe zone (80, 80)
|
||||
assert.equal(pos.x, 80);
|
||||
assert.equal(pos.y, 80);
|
||||
});
|
||||
|
||||
test('clampFloatingTriggerPosition provides safe fallback viewport when viewportWidth/Height is unmeasured', () => {
|
||||
const pos = clampFloatingTriggerPosition({
|
||||
x: null,
|
||||
y: null,
|
||||
width: 60,
|
||||
height: 60,
|
||||
viewportWidth: 0,
|
||||
viewportHeight: 0,
|
||||
margin: 0
|
||||
});
|
||||
// Uses safe fallback top-left position (80, 80)
|
||||
assert.equal(pos.x, 80);
|
||||
assert.equal(pos.y, 80);
|
||||
});
|
||||
|
||||
test('clampFloatingTriggerPosition clamps coordinates within viewport boundaries', () => {
|
||||
// Coordinate beyond viewport bounds
|
||||
const overflowPos = clampFloatingTriggerPosition({
|
||||
x: '3000px',
|
||||
y: '2000px',
|
||||
width: 60,
|
||||
height: 60,
|
||||
viewportWidth: 1000,
|
||||
viewportHeight: 800,
|
||||
margin: 30
|
||||
});
|
||||
assert.equal(overflowPos.x, 940); // 1000 - 60
|
||||
assert.equal(overflowPos.y, 740); // 800 - 60
|
||||
|
||||
// Coordinate with negative values clamped to min safe boundary (70, 0)
|
||||
const negativePos = clampFloatingTriggerPosition({
|
||||
x: '-50px',
|
||||
y: '-100px',
|
||||
width: 60,
|
||||
height: 60,
|
||||
viewportWidth: 1000,
|
||||
viewportHeight: 800,
|
||||
margin: 30
|
||||
});
|
||||
assert.equal(negativePos.x, 70);
|
||||
assert.equal(negativePos.y, 0);
|
||||
|
||||
// Coordinate within valid range
|
||||
const validPos = clampFloatingTriggerPosition({
|
||||
x: '400px',
|
||||
y: '300px',
|
||||
width: 60,
|
||||
height: 60,
|
||||
viewportWidth: 1000,
|
||||
viewportHeight: 800,
|
||||
margin: 30
|
||||
});
|
||||
assert.equal(validPos.x, 400);
|
||||
assert.equal(validPos.y, 300);
|
||||
});
|
||||
|
||||
test('saveTriggerPosition, loadSavedTriggerPosition and clearSavedTriggerPosition provide robust roundtrip persistence', () => {
|
||||
const mockStorage = new Map();
|
||||
const storageAdapter = {
|
||||
getItem: k => mockStorage.get(k) || null,
|
||||
setItem: (k, v) => mockStorage.set(k, String(v)),
|
||||
removeItem: k => mockStorage.delete(k)
|
||||
};
|
||||
|
||||
// 1. Initial empty state returns null
|
||||
assert.equal(loadSavedTriggerPosition(storageAdapter), null);
|
||||
|
||||
// 2. User drags and saves position at (949, 588)
|
||||
saveTriggerPosition({ x: 949.4, y: 588.6 }, storageAdapter);
|
||||
|
||||
// 3. Roundtrip load retrieves exact integer coordinates
|
||||
const restored = loadSavedTriggerPosition(storageAdapter);
|
||||
assert.deepEqual(restored, { x: 949, y: 589 });
|
||||
|
||||
// 4. Clamping restored coordinates in 1920x1080 viewport stays strictly at (949, 589)
|
||||
const clamped = clampFloatingTriggerPosition({
|
||||
x: restored.x,
|
||||
y: restored.y,
|
||||
width: 60,
|
||||
height: 60,
|
||||
viewportWidth: 1920,
|
||||
viewportHeight: 1080
|
||||
});
|
||||
assert.equal(clamped.x, 949);
|
||||
assert.equal(clamped.y, 589);
|
||||
|
||||
// 5. Clear storage flushes memory
|
||||
clearSavedTriggerPosition(storageAdapter);
|
||||
assert.equal(loadSavedTriggerPosition(storageAdapter), null);
|
||||
|
||||
// 6. Dirty legacy dock coordinates (< 70px) are rejected and purged automatically
|
||||
storageAdapter.setItem('anomalous_trigger_pos_v3', JSON.stringify({ x: 0, y: 449 }));
|
||||
assert.equal(loadSavedTriggerPosition(storageAdapter), null);
|
||||
assert.equal(storageAdapter.getItem('anomalous_trigger_pos_v3'), null);
|
||||
|
||||
// 7. Attempting to save coordinates inside sidebar dock is ignored
|
||||
saveTriggerPosition({ x: 50, y: 300 }, storageAdapter);
|
||||
assert.equal(storageAdapter.getItem('anomalous_trigger_pos_v3'), null);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
// Where a LoRA goes and how it is wired (web/modules/graph_splice.js), on a small stand-in graph.
|
||||
import assert from 'node:assert/strict';
|
||||
import { chainEnd, loraInsertionCapabilities, mainModelLoaders, planLoraInsertion, spliceLora } from '../web/modules/graph_splice.js';
|
||||
|
||||
const SLOTS = {
|
||||
CheckpointLoaderSimple: [[], ['MODEL', 'CLIP', 'VAE']],
|
||||
UNETLoader: [[], ['MODEL']],
|
||||
CLIPLoader: [[], ['CLIP']],
|
||||
CLIPTextEncode: [['CLIP'], ['CONDITIONING']],
|
||||
KSampler: [['MODEL', 'CONDITIONING', 'CONDITIONING', 'LATENT'], ['LATENT']],
|
||||
LoraLoader: [['MODEL', 'CLIP'], ['MODEL', 'CLIP']],
|
||||
LoraLoaderModelOnly: [['MODEL'], ['MODEL']],
|
||||
};
|
||||
|
||||
class Graph {
|
||||
constructor() { this._nodes = []; this.links = {}; this.nextLink = 1; this.nextNode = 1; }
|
||||
getNodeById(id) { return this._nodes.find(node => node.id === id) || null; }
|
||||
getLink(id) { return this.links[id] || null; }
|
||||
add(node) { node.id ??= this.nextNode++; node.graph = this; this._nodes.push(node); }
|
||||
remove(node) {
|
||||
node.inputs.forEach((input, slot) => input.link != null && this.unlink(input.link));
|
||||
node.outputs.forEach(output => [...output.links].forEach(id => this.unlink(id)));
|
||||
this._nodes = this._nodes.filter(item => item !== node);
|
||||
}
|
||||
unlink(id) {
|
||||
const link = this.links[id];
|
||||
if (!link) return;
|
||||
const origin = this.getNodeById(link.origin_id);
|
||||
origin.outputs[link.origin_slot].links = origin.outputs[link.origin_slot].links.filter(item => item !== id);
|
||||
this.getNodeById(link.target_id).inputs[link.target_slot].link = null;
|
||||
delete this.links[id];
|
||||
}
|
||||
}
|
||||
|
||||
function node(graph, type, failConnect = false) {
|
||||
const [inputs, outputs] = SLOTS[type];
|
||||
const item = {
|
||||
type, mode: 0,
|
||||
inputs: inputs.map(t => ({ type: t, link: null })),
|
||||
outputs: outputs.map(t => ({ type: t, links: [] })),
|
||||
connect(slot, target, targetSlot) {
|
||||
if (failConnect) return null;
|
||||
const g = this.graph;
|
||||
if (target.inputs[targetSlot].link != null) g.unlink(target.inputs[targetSlot].link);
|
||||
const id = g.nextLink++;
|
||||
g.links[id] = { id, origin_id: this.id, origin_slot: slot, target_id: target.id, target_slot: targetSlot };
|
||||
this.outputs[slot].links.push(id);
|
||||
target.inputs[targetSlot].link = id;
|
||||
return g.links[id];
|
||||
},
|
||||
};
|
||||
if (graph) graph.add(item);
|
||||
return item;
|
||||
}
|
||||
|
||||
const source = (graph, target, slot) => {
|
||||
const link = graph.getLink(target.inputs[slot].link);
|
||||
return link ? `${graph.getNodeById(link.origin_id).type}#${link.origin_id}` : null;
|
||||
};
|
||||
|
||||
// A checkpoint whose CLIP feeds both prompts.
|
||||
function checkpointWorkflow() {
|
||||
const g = new Graph();
|
||||
const ckpt = node(g, 'CheckpointLoaderSimple');
|
||||
const pos = node(g, 'CLIPTextEncode');
|
||||
const neg = node(g, 'CLIPTextEncode');
|
||||
const sampler = node(g, 'KSampler');
|
||||
ckpt.connect(1, pos, 0); ckpt.connect(1, neg, 0); ckpt.connect(0, sampler, 0);
|
||||
pos.connect(0, sampler, 1); neg.connect(0, sampler, 2);
|
||||
return { g, ckpt, pos, neg, sampler };
|
||||
}
|
||||
|
||||
{
|
||||
const { g, ckpt, pos, neg, sampler } = checkpointWorkflow();
|
||||
const plan = planLoraInsertion(g, ckpt, 'after', { textEncoder: true });
|
||||
assert.equal(plan.clip, true);
|
||||
const lora = spliceLora({ graph: g, plan, anchorNode: ckpt, insertedNode: node(null, 'LoraLoader') });
|
||||
assert.equal(source(g, lora, 0), `CheckpointLoaderSimple#${ckpt.id}`);
|
||||
assert.equal(source(g, lora, 1), `CheckpointLoaderSimple#${ckpt.id}`);
|
||||
for (const prompt of [pos, neg]) assert.equal(source(g, prompt, 0), `LoraLoader#${lora.id}`, 'both prompts move behind the LoRA');
|
||||
assert.equal(source(g, sampler, 0), `LoraLoader#${lora.id}`);
|
||||
// The next one goes after the LoRA already there; a model-only LoRA leaves CLIP alone.
|
||||
assert.equal(chainEnd(g, mainModelLoaders(g)[0], 'MODEL'), lora);
|
||||
const plan2 = planLoraInsertion(g, lora, 'after', { textEncoder: false });
|
||||
const lora2 = spliceLora({ graph: g, plan: plan2, anchorNode: lora, insertedNode: node(null, 'LoraLoaderModelOnly') });
|
||||
assert.equal(source(g, sampler, 0), `LoraLoaderModelOnly#${lora2.id}`);
|
||||
assert.equal(source(g, pos, 0), `LoraLoader#${lora.id}`);
|
||||
}
|
||||
|
||||
{
|
||||
// A UNet with its own CLIP loader: the CLIP line comes from the CLIP loader.
|
||||
const g = new Graph();
|
||||
const unet = node(g, 'UNETLoader');
|
||||
const clip = node(g, 'CLIPLoader');
|
||||
const pos = node(g, 'CLIPTextEncode');
|
||||
const neg = node(g, 'CLIPTextEncode');
|
||||
const sampler = node(g, 'KSampler');
|
||||
clip.connect(0, pos, 0); clip.connect(0, neg, 0); unet.connect(0, sampler, 0);
|
||||
pos.connect(0, sampler, 1); neg.connect(0, sampler, 2);
|
||||
const plan = planLoraInsertion(g, unet, 'after', { textEncoder: true });
|
||||
const lora = spliceLora({ graph: g, plan, anchorNode: unet, insertedNode: node(null, 'LoraLoader') });
|
||||
assert.equal(source(g, lora, 0), `UNETLoader#${unet.id}`);
|
||||
assert.equal(source(g, lora, 1), `CLIPLoader#${clip.id}`);
|
||||
assert.equal(source(g, neg, 0), `LoraLoader#${lora.id}`);
|
||||
assert.equal(planLoraInsertion(g, unet, 'after', { textEncoder: false }).clip, false);
|
||||
}
|
||||
|
||||
{
|
||||
// Before a sampler: only its own model input moves; CLIP through the prompts wired into it.
|
||||
const { g, ckpt, pos, neg, sampler } = checkpointWorkflow();
|
||||
const caps = loraInsertionCapabilities(g, sampler);
|
||||
assert.equal(caps.before.supported, true);
|
||||
assert.equal(caps.after.supported, false); // a sampler has no MODEL output
|
||||
const plan = planLoraInsertion(g, sampler, 'before', { textEncoder: true });
|
||||
assert.equal(plan.clip, true);
|
||||
const lora = spliceLora({ graph: g, plan, anchorNode: sampler, insertedNode: node(null, 'LoraLoader') });
|
||||
assert.equal(source(g, sampler, 0), `LoraLoader#${lora.id}`);
|
||||
assert.equal(source(g, pos, 0), `LoraLoader#${lora.id}`);
|
||||
assert.equal(source(g, neg, 0), `LoraLoader#${lora.id}`);
|
||||
assert.equal(source(g, lora, 1), `CheckpointLoaderSimple#${ckpt.id}`);
|
||||
}
|
||||
|
||||
{
|
||||
// A connection that fails leaves the graph as it was.
|
||||
const { g, ckpt, pos, sampler } = checkpointWorkflow();
|
||||
const plan = planLoraInsertion(g, ckpt, 'after', { textEncoder: true });
|
||||
const broken = node(null, 'LoraLoader', true);
|
||||
assert.throws(() => spliceLora({ graph: g, plan, anchorNode: ckpt, insertedNode: broken }), /Failed to connect/);
|
||||
assert.equal(g._nodes.includes(broken), false);
|
||||
assert.equal(source(g, pos, 0), `CheckpointLoaderSimple#${ckpt.id}`);
|
||||
assert.equal(source(g, sampler, 0), `CheckpointLoaderSimple#${ckpt.id}`);
|
||||
assert.equal(planLoraInsertion(g, pos, 'after').supported, false); // a prompt has no MODEL output
|
||||
}
|
||||
|
||||
console.log('graph_splice: ok');
|
||||
@@ -0,0 +1,61 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { renderDetailedNodeCards } from '../web/modules/material_inspector.js';
|
||||
|
||||
// Minimal DOM double: exercise allocation and toggle/copy handlers without a browser dependency.
|
||||
class Element {
|
||||
constructor(tag) {
|
||||
this.tagName = tag;
|
||||
this.children = [];
|
||||
this.listeners = {};
|
||||
this.dataset = {};
|
||||
this.classList = { toggle() {} };
|
||||
this.open = false;
|
||||
}
|
||||
appendChild(child) { this.children.push(child); return child; }
|
||||
append(...children) { this.children.push(...children); }
|
||||
prepend(...children) { this.children.unshift(...children); }
|
||||
setAttribute() {}
|
||||
querySelector(selector) { return this.children.find(child => child.className?.split(' ').includes(selector.slice(1))) || null; }
|
||||
addEventListener(event, callback) { (this.listeners[event] ||= []).push(callback); }
|
||||
toggle(open) { this.open = open; this.listeners.toggle?.forEach(callback => callback()); }
|
||||
}
|
||||
globalThis.document = { createElement: tag => new Element(tag) };
|
||||
globalThis.window = { anomalous_browser_lang: 'zh' };
|
||||
const descendants = node => [node, ...node.children.flatMap(descendants)];
|
||||
const byClass = (node, className) => descendants(node).filter(child => child.className === className);
|
||||
let propertiesRead = 0;
|
||||
const blocks = Array.from({ length: 2000 }, (_, index) => ({
|
||||
node_id: index, type: 'KSampler', widgets_values: [123, 'fixed', 20, 7, 'euler', 'normal', 1],
|
||||
get properties() { propertiesRead++; return { long: 'x'.repeat(16000) }; },
|
||||
}));
|
||||
blocks.unshift({ node_id: 'prompt', type: 'CLIPTextEncode', widgets_values: [' exact prompt\n'] });
|
||||
const list = renderDetailedNodeCards(new Element('main'), blocks);
|
||||
assert.equal(list.children.length, 2001);
|
||||
assert.equal(list.children.filter(card => card.open).length, 1);
|
||||
assert.equal(byClass(list, 'anomalous-material-node-parameter-content').length, 1);
|
||||
assert.equal(propertiesRead, 0, 'closed nodes must not render/format their properties');
|
||||
assert.equal(byClass(list.children[0], 'anomalous-material-prompt-body')[0].textContent, ' exact prompt\n');
|
||||
|
||||
const sampler = list.children[1];
|
||||
sampler.toggle(true);
|
||||
assert.equal(byClass(sampler, 'anomalous-material-node-parameter-content').length, 1);
|
||||
assert.equal(propertiesRead, 1);
|
||||
sampler.toggle(false);
|
||||
sampler.toggle(true);
|
||||
assert.equal(propertiesRead, 1, 'reopening must reuse the rendered content');
|
||||
assert.equal(byClass(sampler, 'anomalous-material-node-parameter-content').length, 1);
|
||||
|
||||
const selectable = renderDetailedNodeCards(new Element('main'), blocks.slice(0, 2), { selectable: true });
|
||||
assert.equal(selectable.children.filter(card => card.open).length, 0, 'workbench selection cards stay closed');
|
||||
assert.equal(byClass(selectable, 'anomalous-material-node-parameter-content').length, 0);
|
||||
|
||||
const copy = byClass(list.children[0], 'anomalous-material-copy-prompt-btn')[0];
|
||||
Object.defineProperty(globalThis, 'navigator', { configurable: true, value: {
|
||||
clipboard: { writeText: async () => { throw new Error('denied'); } },
|
||||
} });
|
||||
await copy.onclick({ stopPropagation() {} });
|
||||
assert.equal(copy.textContent, '复制失败，请重试');
|
||||
navigator.clipboard.writeText = async value => assert.equal(value, ' exact prompt\n');
|
||||
await copy.onclick({ stopPropagation() {} });
|
||||
assert.notEqual(copy.textContent, '复制失败，请重试');
|
||||
console.log('material inspector: 2001 cards, one initial body, lazy toggles and clipboard feedback OK');
|
||||
@@ -0,0 +1,543 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
extractMaterialPromptEnvelope,
|
||||
fillPrompt as dispatchPromptInjection,
|
||||
applyNodeMaterialValues,
|
||||
isModelFilePath,
|
||||
sanitizePromptText,
|
||||
PROMPT_EXTRACTORS,
|
||||
} from '../web/modules/node_material_actions.js';
|
||||
import { promptBoxes, typeTakesPrompt } from '../web/modules/prompt_boxes.js';
|
||||
|
||||
// A saved block's values into a node of its type (the library's block apply, removed with the
|
||||
// Material Library page), to exercise applyNodeMaterialValues' value and hash rules.
|
||||
function applyMaterialBlock(app, node, block, workflowHashes) {
|
||||
if (node?.type !== block?.type || !Array.isArray(block.widgets_values)
|
||||
|| block.widgets_values.length > (node.widgets?.length || 0)) throw new Error('materialNoCompatibleValues');
|
||||
return applyNodeMaterialValues(app, node, block.widgets_values.map((value, index) => ({ index, value })),
|
||||
{ sourceNodeId: block.node_id, workflowHashes });
|
||||
}
|
||||
|
||||
const multiline = ['STRING', { multiline: true }];
|
||||
globalThis.LiteGraph = { registered_node_types: {
|
||||
'easy a1111Loader': { nodeData: { input: { required: { positive: multiline, negative: multiline } } } },
|
||||
'easy fullLoader': { nodeData: { input: { required: { positive: multiline, negative: multiline } } } },
|
||||
'easy positive': { nodeData: { input: { required: { positive: multiline } } } },
|
||||
'easy negative': { nodeData: { input: { required: { negative: multiline } } } },
|
||||
CLIPTextEncode: { nodeData: { input: { required: { text: multiline, clip: ['CLIP'] } } } },
|
||||
LoraLoader: { nodeData: { input: { required: { lora_name: [[]], strength_model: ['FLOAT'] } } } },
|
||||
} };
|
||||
|
||||
|
||||
function createEasyA1111LoaderFixture() {
|
||||
const widgets = [
|
||||
{ name: 'ckpt_name', value: 'v1-5-pruned-emaonly.safetensors', type: 'combo' },
|
||||
{ name: 'vae_name', value: 'vae-ft-mse-840000-ema_pruned.safetensors', type: 'combo' },
|
||||
{ name: 'clip_skip', value: -1, type: 'number' },
|
||||
{ name: 'lora_name', value: 'None', type: 'combo' },
|
||||
{ name: 'lora_model_strength', value: 1.0, type: 'number' },
|
||||
{ name: 'lora_clip_strength', value: 1.0, type: 'number' },
|
||||
{ name: 'positive', value: 'initial positive prompt', type: 'customtext', options: { multiline: true } },
|
||||
{ name: 'negative', value: 'initial negative prompt', type: 'customtext', options: { multiline: true } },
|
||||
{ name: 'empty_latent_width', value: 512, type: 'number' },
|
||||
{ name: 'empty_latent_height', value: 768, type: 'number' },
|
||||
{ name: 'batch_size', value: 1, type: 'number' },
|
||||
{ name: 'seed', value: 123456789, type: 'number' },
|
||||
{ name: 'control_after_generate', value: 'fixed', type: 'combo' },
|
||||
{ name: 'steps', value: 25, type: 'number' },
|
||||
{ name: 'cfg', value: 7.5, type: 'number' },
|
||||
{ name: 'sampler_name', value: 'euler_ancestral', type: 'combo' },
|
||||
{ name: 'scheduler', value: 'karras', type: 'combo' },
|
||||
];
|
||||
const node = {
|
||||
id: 10,
|
||||
type: 'easy a1111Loader',
|
||||
title: 'Easy A1111 Loader',
|
||||
widgets,
|
||||
widgets_values: widgets.map(w => w.value),
|
||||
};
|
||||
const calls = [];
|
||||
node.onWidgetChanged = (index, value, old, widget) => {
|
||||
calls.push({ index, value, old, name: widget.name });
|
||||
};
|
||||
const graph = {
|
||||
getNodeById: id => (id === node.id ? node : null),
|
||||
beforeChange: () => calls.push('before'),
|
||||
afterChange: () => calls.push('after'),
|
||||
change() {},
|
||||
setDirtyCanvas() {},
|
||||
};
|
||||
const app = {
|
||||
graph,
|
||||
canvas: { selected_nodes: { 10: node }, setDirty() {} },
|
||||
};
|
||||
return { node, graph, app, calls };
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 1: promptBoxes on easy a1111Loader (boxes by definition, roles by name)
|
||||
// ============================================================================
|
||||
{
|
||||
const { node } = createEasyA1111LoaderFixture();
|
||||
assert.deepEqual(promptBoxes(node).map(box => [box.index, box.name, box.role]), [[6, 'positive', 'positive'], [7, 'negative', 'negative']]);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 2: Strategy ① - Dual-Slot Pair Injection into easy a1111Loader
|
||||
// ============================================================================
|
||||
{
|
||||
const { node, app, calls } = createEasyA1111LoaderFixture();
|
||||
const envelope = {
|
||||
hasPrompt: true,
|
||||
positive: '1girl, cyberpunk city, neon lights, masterpiece',
|
||||
negative: 'bad hands, low quality, deformed, blurry',
|
||||
singleText: '1girl, cyberpunk city, neon lights, masterpiece',
|
||||
primaryRole: 'both',
|
||||
};
|
||||
|
||||
const result = dispatchPromptInjection(app, node, envelope);
|
||||
assert.equal(result.widgets, 2);
|
||||
|
||||
// Verify positive and negative injected into their respective slots
|
||||
assert.equal(node.widgets[6].value, '1girl, cyberpunk city, neon lights, masterpiece');
|
||||
assert.equal(node.widgets[7].value, 'bad hands, low quality, deformed, blurry');
|
||||
assert.equal(node.widgets_values[6], '1girl, cyberpunk city, neon lights, masterpiece');
|
||||
assert.equal(node.widgets_values[7], 'bad hands, low quality, deformed, blurry');
|
||||
|
||||
// Verify other widgets completely untouched
|
||||
assert.equal(node.widgets[0].value, 'v1-5-pruned-emaonly.safetensors');
|
||||
assert.equal(node.widgets[11].value, 123456789);
|
||||
|
||||
// Verify atomic undo
|
||||
result.undo();
|
||||
assert.equal(node.widgets[6].value, 'initial positive prompt');
|
||||
assert.equal(node.widgets[7].value, 'initial negative prompt');
|
||||
assert.equal(node.widgets_values[6], 'initial positive prompt');
|
||||
assert.equal(node.widgets_values[7], 'initial negative prompt');
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 3: Strategy ② - Role-Matched Negative Prompt Injection (Fixes index 0 bug)
|
||||
// ============================================================================
|
||||
{
|
||||
const { node, app } = createEasyA1111LoaderFixture();
|
||||
const envelope = {
|
||||
hasPrompt: true,
|
||||
positive: '',
|
||||
negative: 'ugly, bad anatomy, bad feet, lowres',
|
||||
singleText: 'ugly, bad anatomy, bad feet, lowres',
|
||||
primaryRole: 'negative',
|
||||
};
|
||||
|
||||
const result = dispatchPromptInjection(app, node, envelope);
|
||||
assert.equal(result.widgets, 1);
|
||||
|
||||
// Positive must REMAIN unchanged!
|
||||
assert.equal(node.widgets[6].value, 'initial positive prompt');
|
||||
assert.equal(node.widgets_values[6], 'initial positive prompt');
|
||||
|
||||
// Negative must be accurately updated!
|
||||
assert.equal(node.widgets[7].value, 'ugly, bad anatomy, bad feet, lowres');
|
||||
assert.equal(node.widgets_values[7], 'ugly, bad anatomy, bad feet, lowres');
|
||||
|
||||
// Undo restores negative
|
||||
result.undo();
|
||||
assert.equal(node.widgets[7].value, 'initial negative prompt');
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 4: Single-Slot Role Nodes (easy positive & easy negative)
|
||||
// ============================================================================
|
||||
{
|
||||
// easy positive
|
||||
const posWidgets = [{ name: 'positive', value: 'old pos' }];
|
||||
const posNode = {
|
||||
id: 20,
|
||||
type: 'easy positive',
|
||||
title: 'Easy Positive',
|
||||
widgets: posWidgets,
|
||||
widgets_values: ['old pos'],
|
||||
};
|
||||
const appPos = {
|
||||
graph: { getNodeById: id => id === 20 ? posNode : null, beforeChange() {}, afterChange() {}, change() {}, setDirtyCanvas() {} },
|
||||
canvas: { selected_nodes: { 20: posNode }, setDirty() {} },
|
||||
};
|
||||
|
||||
assert.deepEqual(promptBoxes(posNode).map(box => [box.index, box.role]), [[0, 'positive']]);
|
||||
|
||||
const dualEnvelope = {
|
||||
hasPrompt: true,
|
||||
positive: 'beautiful scenery, sunset',
|
||||
negative: 'clouds, rain',
|
||||
singleText: 'beautiful scenery, sunset',
|
||||
primaryRole: 'both',
|
||||
};
|
||||
|
||||
dispatchPromptInjection(appPos, posNode, dualEnvelope);
|
||||
assert.equal(posNode.widgets[0].value, 'beautiful scenery, sunset');
|
||||
|
||||
// easy negative
|
||||
const negWidgets = [{ name: 'negative', value: 'old neg' }];
|
||||
const negNode = {
|
||||
id: 21,
|
||||
type: 'easy negative',
|
||||
title: 'Easy Negative',
|
||||
widgets: negWidgets,
|
||||
widgets_values: ['old neg'],
|
||||
};
|
||||
const appNeg = {
|
||||
graph: { getNodeById: id => id === 21 ? negNode : null, beforeChange() {}, afterChange() {}, change() {}, setDirtyCanvas() {} },
|
||||
canvas: { selected_nodes: { 21: negNode }, setDirty() {} },
|
||||
};
|
||||
|
||||
assert.deepEqual(promptBoxes(negNode).map(box => [box.index, box.role]), [[0, 'negative']]);
|
||||
|
||||
dispatchPromptInjection(appNeg, negNode, dualEnvelope);
|
||||
assert.equal(negNode.widgets[0].value, 'clouds, rain');
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 5: Standard CLIPTextEncode (General Slot)
|
||||
// ============================================================================
|
||||
{
|
||||
const clipWidgets = [{ name: 'text', value: 'old clip text' }];
|
||||
const clipNode = {
|
||||
id: 30,
|
||||
type: 'CLIPTextEncode',
|
||||
title: 'CLIP Text Encode (Prompt)',
|
||||
widgets: clipWidgets,
|
||||
widgets_values: ['old clip text'],
|
||||
};
|
||||
const appClip = {
|
||||
graph: { getNodeById: id => id === 30 ? clipNode : null, beforeChange() {}, afterChange() {}, change() {}, setDirtyCanvas() {} },
|
||||
canvas: { selected_nodes: { 30: clipNode }, setDirty() {} },
|
||||
};
|
||||
|
||||
assert.deepEqual(promptBoxes(clipNode).map(box => [box.index, box.role]), [[0, '']]);
|
||||
|
||||
// Single positive
|
||||
dispatchPromptInjection(appClip, clipNode, {
|
||||
hasPrompt: true,
|
||||
positive: 'cyberpunk girl',
|
||||
negative: '',
|
||||
singleText: 'cyberpunk girl',
|
||||
primaryRole: 'positive',
|
||||
});
|
||||
assert.equal(clipNode.widgets[0].value, 'cyberpunk girl');
|
||||
|
||||
// Single negative
|
||||
dispatchPromptInjection(appClip, clipNode, {
|
||||
hasPrompt: true,
|
||||
positive: '',
|
||||
negative: 'worst quality',
|
||||
singleText: 'worst quality',
|
||||
primaryRole: 'negative',
|
||||
});
|
||||
assert.equal(clipNode.widgets[0].value, 'worst quality');
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 6: extractMaterialPromptEnvelope Extraction Correctness
|
||||
// ============================================================================
|
||||
{
|
||||
// A: prompt_plan with positive and negative
|
||||
const planMaterial = {
|
||||
kind: 'prompt_plan',
|
||||
plan: {
|
||||
positive: 'masterpiece, 8k',
|
||||
negative: 'low quality, worst quality',
|
||||
parts: [],
|
||||
},
|
||||
};
|
||||
const envPlan = extractMaterialPromptEnvelope(planMaterial);
|
||||
assert.equal(envPlan.hasPrompt, true);
|
||||
assert.equal(envPlan.primaryRole, 'both');
|
||||
assert.equal(envPlan.positive, 'masterpiece, 8k');
|
||||
assert.equal(envPlan.negative, 'low quality, worst quality');
|
||||
|
||||
// B: prompt_text note marked as negative
|
||||
const noteMaterial = {
|
||||
kind: 'prompt_text',
|
||||
name: '反向提示词常用',
|
||||
note: {
|
||||
promptEn: 'bad hands, blurry',
|
||||
promptZh: '坏手，模糊',
|
||||
},
|
||||
};
|
||||
const envNote = extractMaterialPromptEnvelope(noteMaterial);
|
||||
assert.equal(envNote.hasPrompt, true);
|
||||
assert.equal(envNote.primaryRole, 'negative');
|
||||
assert.equal(envNote.negative, 'bad hands, blurry');
|
||||
|
||||
// C: image material with node_blocks and prompt_roles
|
||||
const imagePayload = {
|
||||
node_blocks: [
|
||||
{ node_id: 6, type: 'CLIPTextEncode', widgets_values: ['high quality, 1girl'] },
|
||||
{ node_id: 7, type: 'CLIPTextEncode', widgets_values: ['ugly, deformed'] },
|
||||
],
|
||||
prompt_roles: {
|
||||
'6': { role: 'positive', source: 'automatic' },
|
||||
'7': { role: 'negative', source: 'automatic' },
|
||||
},
|
||||
};
|
||||
const envImage = extractMaterialPromptEnvelope({}, imagePayload);
|
||||
assert.equal(envImage.hasPrompt, true);
|
||||
assert.equal(envImage.primaryRole, 'both');
|
||||
assert.equal(envImage.positive, 'high quality, 1girl');
|
||||
assert.equal(envImage.negative, 'ugly, deformed');
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 7: Regression test - Same-type block application remains strictly isolated
|
||||
// ============================================================================
|
||||
{
|
||||
const ksamplerNode = {
|
||||
id: 50,
|
||||
type: 'KSampler',
|
||||
widgets: [
|
||||
{ name: 'seed', value: 100 },
|
||||
{ name: 'steps', value: 20 },
|
||||
{ name: 'cfg', value: 8 },
|
||||
],
|
||||
widgets_values: [100, 20, 8],
|
||||
};
|
||||
const appKS = {
|
||||
graph: { getNodeById: id => id === 50 ? ksamplerNode : null, beforeChange() {}, afterChange() {}, change() {}, setDirtyCanvas() {} },
|
||||
canvas: { selected_nodes: { 50: ksamplerNode }, setDirty() {} },
|
||||
};
|
||||
const ksamplerBlock = {
|
||||
node_id: 3,
|
||||
type: 'KSampler',
|
||||
widgets_values: [999, 35, 7.5],
|
||||
};
|
||||
|
||||
// Applying same-type block
|
||||
const result = applyMaterialBlock(appKS, ksamplerNode, ksamplerBlock);
|
||||
assert.equal(ksamplerNode.widgets_values[0], 100); // seed is volatile
|
||||
assert.equal(ksamplerNode.widgets_values[1], 35);
|
||||
assert.equal(ksamplerNode.widgets_values[2], 7.5);
|
||||
|
||||
// Mismatched node type must throw
|
||||
const otherBlock = {
|
||||
node_id: 4,
|
||||
type: 'DifferentNode',
|
||||
widgets_values: [1, 2, 3],
|
||||
};
|
||||
assert.throws(() => applyMaterialBlock(appKS, ksamplerNode, otherBlock), /materialNoCompatibleValues/);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 8: isModelFilePath & isPromptNodeType filtering rules
|
||||
// ============================================================================
|
||||
{
|
||||
assert.equal(isModelFilePath('Tools\\Aesthetic Quality Modifiers - Masterpiece_v5.0 [anima-preview-3].safetensors'), true);
|
||||
assert.equal(isModelFilePath('2D_Anime/Anima/Style - Chen bin鬼针草 [Anima]_Anima v4.0.safetensors'), true);
|
||||
assert.equal(isModelFilePath('Anima\\qwen_3_06b_base.safetensors'), true);
|
||||
assert.equal(isModelFilePath('model.ckpt'), true);
|
||||
assert.equal(isModelFilePath('vae.pt'), true);
|
||||
assert.equal(isModelFilePath('model.gguf'), true);
|
||||
|
||||
// Genuine prompts must NOT be flagged as model paths
|
||||
assert.equal(isModelFilePath('masterpiece, 1girl, cyberpunk city, neon lights'), false);
|
||||
assert.equal(isModelFilePath('bad hands, lowres, blurry'), false);
|
||||
assert.equal(isModelFilePath(''), false);
|
||||
|
||||
// A type takes a prompt when its definition has a multiline text input.
|
||||
assert.equal(typeTakesPrompt('CLIPTextEncode'), true);
|
||||
assert.equal(typeTakesPrompt('easy positive'), true);
|
||||
assert.equal(typeTakesPrompt('LoraLoader'), false);
|
||||
assert.equal(typeTakesPrompt('NotInstalledNode'), false);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 9: Realistic workflow snapshot with multiple LoRA / UNET / VAE blocks
|
||||
// (Strictly prevents model filenames from leaking into prompt text boxes)
|
||||
// ============================================================================
|
||||
{
|
||||
// Simulating material_1789723104
|
||||
const workflowPayload = {
|
||||
node_blocks: [
|
||||
{ node_id: 27, type: 'LoraLoader', widgets_values: ['Tools\\Aesthetic Quality Modifiers - Masterpiece_v5.0 [anima-preview-3].safetensors', 1.2, 1] },
|
||||
{ node_id: 32, type: 'LoraLoader', widgets_values: ['2D_Anime\\Anima\\Style - Chen bin鬼针草 [Anima]_Anima v4.0.safetensors', 1, 1] },
|
||||
{ node_id: 26, type: 'LoraLoader', widgets_values: ['2D_Anime\\Anima\\over-thigh-thighhighs_anima-base-v1.0.safetensors', 1, 1] },
|
||||
{ node_id: 1, type: 'UNETLoader', widgets_values: ['Anima\\Kirazuri (Anima)_v4.0 [anima-base-1].safetensors', 'default'] },
|
||||
{ node_id: 8, type: 'VAELoader', widgets_values: ['Anima\\qwenImageGGUF_vae.safetensors'] },
|
||||
{ node_id: 22, type: 'CLIPTextEncode', widgets_values: ['masterpiece, high quality anime illustration, young adult woman'] },
|
||||
{ node_id: 6, type: 'CLIPTextEncode', widgets_values: ['mature woman, older woman, onee-san, child, loli, bad hands'] },
|
||||
],
|
||||
prompt_roles: {
|
||||
'22': { role: 'positive', source: 'automatic' },
|
||||
'6': { role: 'negative', source: 'automatic' },
|
||||
},
|
||||
prompt_groups: {
|
||||
positive: ['masterpiece, high quality anime illustration, young adult woman'],
|
||||
negative: ['mature woman, older woman, onee-san, child, loli, bad hands'],
|
||||
},
|
||||
};
|
||||
|
||||
const envelope = extractMaterialPromptEnvelope({}, workflowPayload);
|
||||
assert.equal(envelope.hasPrompt, true);
|
||||
assert.equal(envelope.primaryRole, 'both');
|
||||
|
||||
// CRITICAL: positive prompt must NOT contain ANY .safetensors file paths!
|
||||
assert.equal(envelope.positive, 'masterpiece, high quality anime illustration, young adult woman');
|
||||
assert.equal(envelope.positive.includes('.safetensors'), false);
|
||||
|
||||
// Negative prompt must be purely the negative prompt
|
||||
assert.equal(envelope.negative, 'mature woman, older woman, onee-san, child, loli, bad hands');
|
||||
assert.equal(envelope.negative.includes('.safetensors'), false);
|
||||
|
||||
// Now inject into easy fullLoader fixture
|
||||
const fullLoaderWidgets = [
|
||||
{ name: 'ckpt_name', value: 'model.safetensors', type: 'combo' },
|
||||
{ name: 'config_name', value: 'Default', type: 'combo' },
|
||||
{ name: 'vae_name', value: 'Baked VAE', type: 'combo' },
|
||||
{ name: 'clip_skip', value: -2, type: 'number' },
|
||||
{ name: 'lora_name', value: 'None', type: 'combo' },
|
||||
{ name: 'resolution', value: '512 x 512', type: 'combo' },
|
||||
{ name: 'positive', value: 'old positive', type: 'customtext', options: { multiline: true } },
|
||||
{ name: 'positive_token_normalization', value: 'none', type: 'combo' },
|
||||
{ name: 'positive_weight_interpretation', value: 'comfy', type: 'combo' },
|
||||
{ name: 'negative', value: 'old negative', type: 'customtext', options: { multiline: true } },
|
||||
{ name: 'negative_token_normalization', value: 'none', type: 'combo' },
|
||||
{ name: 'negative_weight_interpretation', value: 'comfy', type: 'combo' },
|
||||
{ name: 'batch_size', value: 1, type: 'number' },
|
||||
{ name: 'a1111_prompt_style', value: false, type: 'combo' },
|
||||
];
|
||||
const fullLoaderNode = {
|
||||
id: 99,
|
||||
type: 'easy fullLoader',
|
||||
title: '简易加载器 (完整版)',
|
||||
widgets: fullLoaderWidgets,
|
||||
widgets_values: fullLoaderWidgets.map(w => w.value),
|
||||
};
|
||||
const appFull = {
|
||||
graph: { getNodeById: id => id === 99 ? fullLoaderNode : null, beforeChange() {}, afterChange() {}, change() {}, setDirtyCanvas() {} },
|
||||
canvas: { selected_nodes: { 99: fullLoaderNode }, setDirty() {} },
|
||||
};
|
||||
|
||||
assert.deepEqual(promptBoxes(fullLoaderNode).map(box => [box.index, box.role]), [[6, 'positive'], [9, 'negative']]);
|
||||
|
||||
dispatchPromptInjection(appFull, fullLoaderNode, envelope);
|
||||
|
||||
// Verify positive textarea (index 6) has ONLY the positive prompt!
|
||||
assert.equal(fullLoaderNode.widgets[6].value, 'masterpiece, high quality anime illustration, young adult woman');
|
||||
assert.equal(fullLoaderNode.widgets[6].value.includes('.safetensors'), false);
|
||||
|
||||
// Verify negative textarea (index 9) has ONLY the negative prompt!
|
||||
assert.equal(fullLoaderNode.widgets[9].value, 'mature woman, older woman, onee-san, child, loli, bad hands');
|
||||
assert.equal(fullLoaderNode.widgets[9].value.includes('.safetensors'), false);
|
||||
|
||||
// Verify non-prompt widgets (ckpt_name, resolution, etc.) are untouched!
|
||||
assert.equal(fullLoaderNode.widgets[0].value, 'model.safetensors');
|
||||
assert.equal(fullLoaderNode.widgets[4].value, 'None');
|
||||
assert.equal(fullLoaderNode.widgets[5].value, '512 x 512');
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 10: sanitizePromptText unit guardrails
|
||||
// ============================================================================
|
||||
{
|
||||
assert.equal(sanitizePromptText(null), '');
|
||||
assert.equal(sanitizePromptText(undefined), '');
|
||||
assert.equal(sanitizePromptText(12345), '');
|
||||
assert.equal(sanitizePromptText(' '), '');
|
||||
assert.equal(sanitizePromptText('model.safetensors'), '');
|
||||
assert.equal(sanitizePromptText('checkpoints/v1-5-pruned.ckpt'), '');
|
||||
assert.equal(sanitizePromptText('loras/anime_style.safetensors\r\nsub/model.safetensors'), '');
|
||||
assert.equal(sanitizePromptText('vae.pt'), '');
|
||||
assert.equal(sanitizePromptText('onnx_model.onnx'), '');
|
||||
assert.equal(sanitizePromptText('llm.gguf'), '');
|
||||
assert.equal(sanitizePromptText(' 1girl, beautiful lighting, cinematic shot '), '1girl, beautiful lighting, cinematic shot');
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 11: prompt envelopes of plans, notes and loader blocks
|
||||
// ============================================================================
|
||||
{
|
||||
assert.equal(Array.isArray(PROMPT_EXTRACTORS), true);
|
||||
assert.equal(PROMPT_EXTRACTORS.length, 5);
|
||||
|
||||
// 11.1 Prompt Plan SSOT
|
||||
const planMaterial = {
|
||||
kind: 'prompt_plan',
|
||||
plan: {
|
||||
positive: 'cyberpunk street, neon glow',
|
||||
negative: 'blurry, low quality',
|
||||
},
|
||||
};
|
||||
const envPlan = extractMaterialPromptEnvelope(planMaterial);
|
||||
assert.equal(envPlan.positive, 'cyberpunk street, neon glow');
|
||||
assert.equal(envPlan.negative, 'blurry, low quality');
|
||||
|
||||
// 11.2 Negative Prompt Note SSOT
|
||||
const negNoteMaterial = {
|
||||
kind: 'prompt_note_bundle',
|
||||
name: 'Negative Base Prompts',
|
||||
tags: ['negative'],
|
||||
note: {
|
||||
promptEn: 'worst quality, bad anatomy, deformed limbs',
|
||||
},
|
||||
};
|
||||
const envNote = extractMaterialPromptEnvelope(negNoteMaterial);
|
||||
assert.equal(envNote.primaryRole, 'negative');
|
||||
assert.equal(envNote.negative, 'worst quality, bad anatomy, deformed limbs');
|
||||
|
||||
// 11.3 Loader blocks with model file: info must NEVER contain .safetensors
|
||||
const loaderMaterial = {
|
||||
kind: 'recipe_parameter_selection',
|
||||
node_blocks: [
|
||||
{
|
||||
node_id: 1,
|
||||
type: 'easy fullLoader',
|
||||
widgets_values: ['epicrealism.safetensors', 'baked_vae.safetensors'],
|
||||
},
|
||||
{
|
||||
node_id: 2,
|
||||
type: 'CLIPTextEncode',
|
||||
promptRole: 'positive',
|
||||
widgets_values: ['photorealistic portrait, 8k'],
|
||||
},
|
||||
],
|
||||
};
|
||||
const envLoader = extractMaterialPromptEnvelope(loaderMaterial);
|
||||
assert.equal(envLoader.positive, 'photorealistic portrait, 8k');
|
||||
assert.equal(envLoader.positive.includes('.safetensors'), false);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Test 12: Prompt text never crosses roles
|
||||
// ============================================================================
|
||||
{
|
||||
const makeNode = (id, type, name) => {
|
||||
const node = { id, type, title: type, widgets: [{ name, value: 'keep me' }], widgets_values: ['keep me'] };
|
||||
const app = {
|
||||
graph: { getNodeById: nodeId => nodeId === id ? node : null, beforeChange() {}, afterChange() {}, change() {}, setDirtyCanvas() {} },
|
||||
canvas: { selected_nodes: { [id]: node }, setDirty() {} },
|
||||
};
|
||||
return { node, app };
|
||||
};
|
||||
const negativeOnly = { hasPrompt: true, positive: '', negative: 'lowres, blurry', singleText: 'lowres, blurry', primaryRole: 'negative' };
|
||||
const positiveOnly = { hasPrompt: true, positive: '1girl, garden', negative: '', singleText: '1girl, garden', primaryRole: 'positive' };
|
||||
|
||||
const pos = makeNode(30, 'easy positive', 'positive');
|
||||
assert.throws(() => dispatchPromptInjection(pos.app, pos.node, negativeOnly), /materialNoCompatibleValues/);
|
||||
assert.equal(pos.node.widgets[0].value, 'keep me', 'negative text must not land in a positive-only node');
|
||||
|
||||
const neg = makeNode(31, 'easy negative', 'negative');
|
||||
assert.throws(() => dispatchPromptInjection(neg.app, neg.node, positiveOnly), /materialNoCompatibleValues/);
|
||||
assert.equal(neg.node.widgets[0].value, 'keep me', 'positive text must not land in a negative-only node');
|
||||
|
||||
// Role-neutral CLIPTextEncode still accepts either side.
|
||||
const clip = makeNode(32, 'CLIPTextEncode', 'text');
|
||||
dispatchPromptInjection(clip.app, clip.node, negativeOnly);
|
||||
assert.equal(clip.node.widgets[0].value, 'lowres, blurry');
|
||||
|
||||
// Two-slot loader receives a one-sided envelope only in the matching slot.
|
||||
const { node: loader, app: loaderApp } = createEasyA1111LoaderFixture();
|
||||
dispatchPromptInjection(loaderApp, loader, negativeOnly);
|
||||
assert.equal(loader.widgets[6].value, 'initial positive prompt');
|
||||
assert.equal(loader.widgets[7].value, 'lowres, blurry');
|
||||
}
|
||||
|
||||
console.log('✔ All Cross-Node Prompt Injection Protocol (CNPIP) tests passed!');
|
||||
@@ -0,0 +1,93 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { applyNodeMaterialValues, selectedMaterialNode } from '../web/modules/node_material_actions.js';
|
||||
import { promptBoxes } from '../web/modules/prompt_boxes.js';
|
||||
import { composePromptPlan } from '../web/modules/prompt_composition.js';
|
||||
|
||||
// A saved block's values into a node of its type (the library's block apply, removed with the
|
||||
// Material Library page), to exercise applyNodeMaterialValues' value and hash rules.
|
||||
function applyMaterialBlock(app, node, block, workflowHashes) {
|
||||
if (node?.type !== block?.type || !Array.isArray(block.widgets_values)
|
||||
|| block.widgets_values.length > (node.widgets?.length || 0)) throw new Error('materialNoCompatibleValues');
|
||||
return applyNodeMaterialValues(app, node, block.widgets_values.map((value, index) => ({ index, value })),
|
||||
{ sourceNodeId: block.node_id, workflowHashes });
|
||||
}
|
||||
|
||||
function fixture() {
|
||||
const node = { id: 7, type: 'KSampler', pos: [100, 20], inputs: [{ link: 88 }],
|
||||
widgets: [{ name: 'seed', value: 123 }, { name: 'steps', value: 20 }, { name: 'sampler_name', value: 'euler', options: { values: ['euler', 'heun'] } }],
|
||||
widgets_values: [123, 20, 'euler'] };
|
||||
const calls = [];
|
||||
node.onWidgetChanged = (name, value, old, widget) => { assert.equal(widget.name, name); calls.push([node.widgets.indexOf(widget), value, old]); };
|
||||
const graph = { extra: { keep: true, anomalous_hashes: { '7_old': 'old', '8_other': 'other' } },
|
||||
getNodeById: id => id === node.id ? node : null,
|
||||
beforeChange: () => calls.push('before'), afterChange: () => calls.push('after'), change() {}, setDirtyCanvas() {} };
|
||||
const app = { graph, canvas: { selected_nodes: { 7: node }, setDirty() {} } };
|
||||
const block = { node_id: 2, type: 'KSampler', widgets_values: [999, 30, 'heun'] };
|
||||
return { node, graph, app, block, calls };
|
||||
}
|
||||
{
|
||||
const { node, graph, app, block, calls } = fixture();
|
||||
assert.equal(selectedMaterialNode(app), node);
|
||||
const before = structuredClone({ values: node.widgets_values, extra: graph.extra, pos: node.pos, inputs: node.inputs });
|
||||
const result = applyMaterialBlock(app, node, block, { '2_model': { hash: 'abc' }, '22_secret': 'ignore' });
|
||||
assert.equal(result.widgets, 2);
|
||||
assert.deepEqual(node.widgets_values, [123, 30, 'heun']);
|
||||
assert.deepEqual(graph.extra.anomalous_hashes, { '7_model': { hash: 'abc' }, '8_other': 'other' });
|
||||
assert.deepEqual(calls, ['before', [1, 30, 20], [2, 'heun', 'euler'], 'after']);
|
||||
result.undo();
|
||||
assert.deepEqual({ values: node.widgets_values, extra: graph.extra, pos: node.pos, inputs: node.inputs }, before);
|
||||
assert.throws(() => result.undo(), /materialUndoChanged/);
|
||||
}
|
||||
for (const invalid of [ { type: 'Other' }, { widgets_values: [99, 30, 'foreign'] }, { widgets_values: [99, 'bad', 'heun'] }, { widgets_values: [99, NaN, 'heun'] }, { widgets_values: [99, 30, 'heun', 4] } ]) {
|
||||
const { app, node, block, graph, calls } = fixture();
|
||||
const before = JSON.stringify([node.widgets_values, graph.extra]);
|
||||
assert.throws(() => applyMaterialBlock(app, node, { ...block, ...invalid }, {}));
|
||||
assert.equal(JSON.stringify([node.widgets_values, graph.extra]), before);
|
||||
assert.deepEqual(calls, [], 'validation must finish before graph mutation');
|
||||
}
|
||||
{
|
||||
const { app, node, block, graph, calls } = fixture();
|
||||
node.widgets[2].callback = () => { throw new Error('host hook failed'); };
|
||||
const before = JSON.stringify([node.widgets_values, graph.extra]);
|
||||
assert.throws(() => applyMaterialBlock(app, node, block, { '2_new': 'new' }), /host hook/);
|
||||
assert.deepEqual([node.widgets_values, graph.extra], JSON.parse(before));
|
||||
assert.deepEqual(node.widgets.map(w => w.value), node.widgets_values);
|
||||
assert.equal(calls.at(-1), 'after');
|
||||
}
|
||||
for (const change of ['widget', 'serialized', 'hash', 'graph']) {
|
||||
const { app, node, graph, block } = fixture();
|
||||
const result = applyMaterialBlock(app, node, block, {});
|
||||
if (change === 'widget') node.widgets[1].value = 55;
|
||||
if (change === 'serialized') node.widgets_values[1] = 55;
|
||||
if (change === 'hash') graph.extra.anomalous_hashes['7_other'] = 'new';
|
||||
if (change === 'graph') app.graph = { ...graph };
|
||||
assert.throws(() => result.undo(), /materialUndoChanged/);
|
||||
}
|
||||
{
|
||||
const { app, node, block, graph } = fixture();
|
||||
delete graph.extra;
|
||||
const result = applyMaterialBlock(app, node, block, { '2_model': 'new' });
|
||||
result.undo(); assert.equal(Object.hasOwn(graph, 'extra'), false);
|
||||
app.canvas.selected_nodes.other = {};
|
||||
assert.equal(selectedMaterialNode(app), null);
|
||||
app.canvas.selected_nodes = { 7: { ...node } };
|
||||
assert.equal(selectedMaterialNode(app), null);
|
||||
assert.throws(() => applyNodeMaterialValues(app, { ...node }, [{ index: 1, value: 30 }]), /materialTargetChanged/);
|
||||
}
|
||||
{
|
||||
const { app, node } = fixture();
|
||||
node.type = 'CLIPTextEncode';
|
||||
node.widgets = [{ name: 'text', value: 'old' }, { name: 'text_l', value: '' }, { name: 'model', value: 'not a prompt' }];
|
||||
globalThis.LiteGraph = { registered_node_types: { CLIPTextEncode: { nodeData: { input: { required: { text: ['STRING', { multiline: true }], text_l: ['STRING', { multiline: true }], model: ['STRING', {}] } } } } } };
|
||||
assert.deepEqual(promptBoxes(node).map(box => [box.index, box.name]), [[0, 'text'], [1, 'text_l']]);
|
||||
const result = applyNodeMaterialValues(app, node, [{ index: 0, value: 'new\n(weights:1.2)' }]);
|
||||
assert.equal(node.widgets[1].value, ''); result.undo(); assert.equal(node.widgets[0].value, 'old');
|
||||
}
|
||||
const plan = { parts: [
|
||||
{ enabled: true, positive: ' (quality:1.2), BREAK\n', negative: 'bad' },
|
||||
{ enabled: false, positive: 'disabled', negative: 'disabled' },
|
||||
{ enabled: true, positive: 'style', negative: 'bad' },
|
||||
], positive: 'scene', negative: ' ' };
|
||||
assert.deepEqual(composePromptPlan(plan), { positive: ' (quality:1.2), BREAK\n\nstyle\nscene', negative: 'bad\nbad' });
|
||||
assert.deepEqual(composePromptPlan({ parts: [] }), { positive: '', negative: '' });
|
||||
console.log('node materials: validation, scoped hashes, rollback, guarded undo and prompt composition OK');
|
||||
@@ -0,0 +1,59 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { applyParameterChanges, loadParameterSets, parameterChanges } from '../web/modules/node_parameter_sets.js';
|
||||
|
||||
// Both sources, with the same values saved in a material and in a recipe parameter set.
|
||||
const shared = [123, 'fixed', 30, 5.5, 'euler', 'karras', 1];
|
||||
globalThis.fetch = async (url) => ({
|
||||
ok: true,
|
||||
json: async () => (String(url).includes('/materials/')
|
||||
? { materials: [
|
||||
{ name: 'Slow', timestamp: 10, blocks: [{ title: 'KSampler', widgets_values: shared }] },
|
||||
{ name: 'Fast', timestamp: 30, blocks: [{ title: 'KSampler', widgets_values: [1, 'fixed', 8, 2, 'lcm', 'sgm_uniform', 1] }] },
|
||||
] }
|
||||
: { groups: [
|
||||
{ recipe_filename: 'r.json', recipe_name: 'Portrait', notebooks: [{ name: 'Slow set', timestamp: 20, nodes: [{ title: 'KSampler', widgets_values: shared }] }] },
|
||||
{ recipe_filename: 'unbound', notebooks: [{ name: 'Loose', timestamp: 99, nodes: [{ widgets_values: [9] }] }] },
|
||||
{ recipe_filename: 'gone.json', recipe_name: 'gone.json', notebooks: [{ name: 'Old', timestamp: 98, nodes: [{ widgets_values: [8] }] }] },
|
||||
] }),
|
||||
});
|
||||
|
||||
{
|
||||
const entries = await loadParameterSets('KSampler');
|
||||
assert.deepEqual(entries.map(entry => entry.name), ['Fast', 'Slow']);
|
||||
assert.deepEqual(entries[1].sources.map(source => [source.kind, source.label]), [['material', 'Slow'], ['recipe', 'Portrait']]);
|
||||
assert.equal(entries[1].timestamp, 20);
|
||||
}
|
||||
|
||||
// What would change: seeds, model files and missing choices stay.
|
||||
const widget = (name, value, values) => ({ name, value, ...(values ? { options: { values } } : {}) });
|
||||
function sampler() {
|
||||
return {
|
||||
id: 3, type: 'KSampler',
|
||||
widgets: [widget('seed', 5), widget('control_after_generate', 'fixed', ['fixed', 'randomize']), widget('steps', 20),
|
||||
widget('cfg', 7), widget('sampler_name', 'euler', ['euler', 'dpmpp_2m']), widget('scheduler', 'normal', ['normal', 'karras']), widget('denoise', 1)],
|
||||
};
|
||||
}
|
||||
{
|
||||
const node = sampler();
|
||||
const changes = parameterChanges(node, [123, 'randomize', 30, 5.5, 'euler', 'beta57', 1]);
|
||||
assert.deepEqual(changes.map(c => [c.label, c.from, c.to, c.keep]), [
|
||||
['seed', 5, 123, 'seed'], ['control_after_generate', 'fixed', 'randomize', 'seed'], ['steps', 20, 30, ''], ['cfg', 7, 5.5, ''], ['scheduler', 'normal', 'beta57', 'missing'],
|
||||
]);
|
||||
const loader = { id: 4, type: 'CheckpointLoaderSimple', widgets: [widget('ckpt_name', 'a.safetensors', ['a.safetensors', 'b.safetensors'])] };
|
||||
assert.equal(parameterChanges(loader, ['b.safetensors'])[0].keep, 'model');
|
||||
assert.deepEqual(parameterChanges(node, ['not a number', 'fixed', 20]), []); // a value of another type is not this input
|
||||
}
|
||||
|
||||
// Applying writes only the changes that are made, as one undoable step.
|
||||
{
|
||||
const node = sampler();
|
||||
node.widgets_values = node.widgets.map(w => w.value);
|
||||
const graph = { getNodeById: id => (id === 3 ? node : null), beforeChange() {}, afterChange() {}, change() {}, setDirtyCanvas() {} };
|
||||
const app = { graph, canvas: { setDirty() {} } };
|
||||
const result = applyParameterChanges(app, node, parameterChanges(node, [123, 'fixed', 30, 5.5, 'euler', 'beta57', 1]));
|
||||
assert.deepEqual(node.widgets.map(w => w.value), [5, 'fixed', 30, 5.5, 'euler', 'normal', 1]);
|
||||
result.undo();
|
||||
assert.deepEqual(node.widgets.map(w => w.value), [5, 'fixed', 20, 7, 'euler', 'normal', 1]);
|
||||
}
|
||||
|
||||
console.log('node parameter sets: merged sources, kept values and undoable apply OK');
|
||||
@@ -0,0 +1,100 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { partnerBox, planPromptFill, promptBoxes, promptNameRole, typeTakesPrompt, wiredRole } from '../web/modules/prompt_boxes.js';
|
||||
|
||||
const multiline = ['STRING', { multiline: true }];
|
||||
globalThis.LiteGraph = { registered_node_types: {
|
||||
CLIPTextEncode: { nodeData: { input: { required: { text: multiline, clip: ['CLIP'] } } } },
|
||||
ConditioningCombine: { nodeData: { input: { required: { conditioning_1: ['CONDITIONING'], conditioning_2: ['CONDITIONING'] } } } },
|
||||
KSampler: { nodeData: { input: { required: { seed: ['INT'] } } } },
|
||||
BasicGuider: { nodeData: { input: { required: { conditioning: ['CONDITIONING'] } } } },
|
||||
SaveImage: { nodeData: { input: { required: { filename_prefix: ['STRING', {}] } } } },
|
||||
'easy a1111Loader': { nodeData: { input: { required: { positive: multiline, negative: multiline } } } },
|
||||
} };
|
||||
|
||||
// A tiny graph: nodes with outputs/inputs and a Map of links, like ComfyUI's.
|
||||
function makeGraph() {
|
||||
const nodes = [];
|
||||
const links = new Map();
|
||||
let nextLink = 1;
|
||||
const graph = {
|
||||
_nodes: nodes,
|
||||
getLink: id => links.get(id),
|
||||
getNodeById: id => nodes.find(node => node.id === id) || null,
|
||||
};
|
||||
const add = (id, type, inputs = [], widgets = []) => {
|
||||
const node = { id, type, graph, inputs: inputs.map(name => ({ name })), outputs: [{ links: [] }], widgets };
|
||||
nodes.push(node);
|
||||
return node;
|
||||
};
|
||||
const connect = (from, to, inputName) => {
|
||||
const id = nextLink++;
|
||||
links.set(id, { target_id: to.id, target_slot: to.inputs.findIndex(input => input.name === inputName) });
|
||||
from.outputs[0].links.push(id);
|
||||
};
|
||||
return { add, connect };
|
||||
}
|
||||
|
||||
const text = value => ({ name: 'text', value });
|
||||
|
||||
// Boxes come from the definition: a single-line STRING is not a prompt box.
|
||||
{
|
||||
const { add } = makeGraph();
|
||||
assert.equal(promptBoxes(add(1, 'SaveImage', [], [{ name: 'filename_prefix', value: 'ComfyUI' }])).length, 0);
|
||||
assert.equal(typeTakesPrompt('SaveImage'), false);
|
||||
assert.equal(typeTakesPrompt('CLIPTextEncode'), true);
|
||||
assert.equal(promptNameRole('negative_prompt'), 'negative');
|
||||
assert.equal(promptNameRole('text_g'), '');
|
||||
}
|
||||
|
||||
// Roles follow the wiring, also through a combine node; titles do not matter.
|
||||
{
|
||||
const { add, connect } = makeGraph();
|
||||
const pos = add(1, 'CLIPTextEncode', [], [text('a cat')]);
|
||||
const neg = add(2, 'CLIPTextEncode', [], [text('blurry')]);
|
||||
neg.title = 'Positive (misnamed)';
|
||||
const style = add(3, 'CLIPTextEncode', [], [text('watercolor')]);
|
||||
const combine = add(4, 'ConditioningCombine', ['conditioning_1', 'conditioning_2']);
|
||||
const sampler = add(5, 'KSampler', ['positive', 'negative']);
|
||||
connect(pos, combine, 'conditioning_1');
|
||||
connect(style, combine, 'conditioning_2');
|
||||
connect(combine, sampler, 'positive');
|
||||
connect(neg, sampler, 'negative');
|
||||
assert.equal(promptBoxes(pos)[0].role, 'positive');
|
||||
assert.equal(promptBoxes(style)[0].role, 'positive');
|
||||
assert.equal(promptBoxes(neg)[0].role, 'negative');
|
||||
assert.deepEqual([...wiredRole(neg).samplers], [5]);
|
||||
|
||||
// The negative box on the same sampler is the partner of a positive one.
|
||||
assert.equal(partnerBox(pos, 'positive').node, neg);
|
||||
assert.equal(partnerBox(neg, 'negative'), null); // two positive boxes reach it: no single partner
|
||||
|
||||
const loose = add(6, 'CLIPTextEncode', [], [text('')]);
|
||||
assert.equal(promptBoxes(loose)[0].role, '');
|
||||
assert.equal(partnerBox(loose, 'positive'), null);
|
||||
}
|
||||
|
||||
// A guider's conditioning input counts as positive.
|
||||
{
|
||||
const { add, connect } = makeGraph();
|
||||
const encode = add(1, 'CLIPTextEncode', [], [text('a cat')]);
|
||||
connect(encode, add(2, 'BasicGuider', ['conditioning']), 'conditioning');
|
||||
assert.equal(promptBoxes(encode)[0].role, 'positive');
|
||||
}
|
||||
|
||||
// Filling: each box takes its own role's text; a chosen box takes its role's text or the main one.
|
||||
{
|
||||
const { add } = makeGraph();
|
||||
const loader = add(1, 'easy a1111Loader', [], [{ name: 'positive', value: '' }, { name: 'negative', value: '' }]);
|
||||
const both = { positive: 'a cat', negative: 'blurry', singleText: 'a cat', primaryRole: 'both' };
|
||||
const negOnly = { positive: '', negative: 'blurry', singleText: 'blurry', primaryRole: 'negative' };
|
||||
assert.deepEqual(planPromptFill(loader, both).map(e => [e.index, e.value]), [[0, 'a cat'], [1, 'blurry']]);
|
||||
assert.deepEqual(planPromptFill(loader, negOnly).map(e => [e.index, e.value]), [[1, 'blurry']]);
|
||||
const [posBox, negBox] = promptBoxes(loader);
|
||||
assert.deepEqual(planPromptFill(loader, both, negBox).map(e => [e.index, e.value]), [[1, 'blurry']]);
|
||||
// The user chose the positive box for a negative-only prompt: it gets that text.
|
||||
assert.deepEqual(planPromptFill(loader, negOnly, posBox).map(e => [e.index, e.value]), [[0, 'blurry']]);
|
||||
const loose = add(2, 'CLIPTextEncode', [], [text('')]);
|
||||
assert.deepEqual(planPromptFill(loose, both).map(e => e.value), ['a cat']);
|
||||
}
|
||||
|
||||
console.log('prompt boxes: definition, wiring roles, partners and filling OK');
|
||||
@@ -0,0 +1,41 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { insertTags, joinPrompt, moveTag, promptTagsOf, removeTag, replaceTag, splitPrompt, tagWeight, withWeight } from '../web/modules/prompt_tags.js';
|
||||
|
||||
const texts = value => splitPrompt(value).items.map(item => item.text);
|
||||
|
||||
// Round trip keeps every character.
|
||||
for (const value of ['', 'a', 'a, b', ' a ,b,\n\nc\n', 'masterpiece, (red hair:1.2), <lora:x:0.8>, c', 'a，b、c', ', a, , b,']) {
|
||||
assert.equal(joinPrompt(splitPrompt(value)), value, JSON.stringify(value));
|
||||
}
|
||||
assert.deepEqual(texts('a, (b, c:1.2), <lora:x, y:0.5>\nd，e,'), ['a', '(b, c:1.2)', '<lora:x, y:0.5>', 'd', 'e']);
|
||||
const backslash = String.fromCharCode(92);
|
||||
assert.deepEqual(texts(`artist ${backslash}(x, y${backslash}), z`), [`artist ${backslash}(x`, `y${backslash})`, 'z']);
|
||||
assert.deepEqual(promptTagsOf(',, a ,, b ,'), ['a', 'b']);
|
||||
|
||||
// Weights.
|
||||
assert.deepEqual(tagWeight('(red hair:1.2)'), { core: 'red hair', weight: 1.2 });
|
||||
assert.deepEqual(tagWeight('red hair'), { core: 'red hair', weight: 1 });
|
||||
assert.equal(withWeight('red hair', 1.1), '(red hair:1.1)');
|
||||
assert.equal(withWeight('(red hair:1.1)', 1), 'red hair');
|
||||
assert.equal(withWeight('(red hair:1.95)', 5), '(red hair:2)');
|
||||
|
||||
// Insert: at the end the text's ending stays last; duplicates by words are skipped.
|
||||
assert.deepEqual(insertTags('a, b', ['c']), { value: 'a, b, c', added: 1, skipped: 0 });
|
||||
assert.deepEqual(insertTags('a, b,\n', ['c']), { value: 'a, b, c,\n', added: 1, skipped: 0 });
|
||||
assert.deepEqual(insertTags('', ['a', 'b']), { value: 'a, b', added: 2, skipped: 0 });
|
||||
assert.deepEqual(insertTags('a, (B:1.2)', ['b', 'c', 'c']), { value: 'a, (B:1.2), c', added: 1, skipped: 2 });
|
||||
assert.equal(insertTags('a, b', ['x'], 0).value, 'x, a, b');
|
||||
assert.equal(insertTags('a,\nb', ['x'], 1).value, 'a,\nx, b');
|
||||
|
||||
// Remove, replace, move.
|
||||
assert.equal(removeTag('a, b, c', 1), 'a, c');
|
||||
assert.equal(removeTag('a, b, c,\n', 2), 'a, b,\n');
|
||||
assert.equal(removeTag('a', 0), '');
|
||||
assert.equal(replaceTag('a, b, c', 1, '(b:1.2)'), 'a, (b:1.2), c');
|
||||
assert.equal(replaceTag('a, b, c', 1, ' '), 'a, c');
|
||||
assert.equal(moveTag('a, b, c', 0, 3), 'b, c, a');
|
||||
assert.equal(moveTag('a, b, c', 2, 0), 'c, a, b');
|
||||
assert.equal(moveTag('a, b, c', 0, 1), 'a, b, c');
|
||||
assert.equal(moveTag('a, b, c,\n', 0, 3), 'b, c, a,\n');
|
||||
|
||||
console.log('prompt_tags ok');
|
||||
@@ -0,0 +1,453 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
extractRecipeMetadata,
|
||||
applyRecipeWidgetChanges,
|
||||
extractRecipeParameterChoices,
|
||||
extractRecipeParameterChoicesFromMetadata,
|
||||
} from '../web/modules/recipe_parser.js';
|
||||
import {
|
||||
deriveNodeModelSpecs,
|
||||
deriveRecipeModelReferences,
|
||||
recipeReferenceKey,
|
||||
} from '../web/modules/recipe_identity.js';
|
||||
|
||||
// Live nodes carry ComfyUI's definitions; prompt boxes are found by them.
|
||||
globalThis.LiteGraph = { registered_node_types: {
|
||||
'easy a1111Loader': { nodeData: { input: { required: { positive: ['STRING', { multiline: true }], negative: ['STRING', { multiline: true }] } } } },
|
||||
} };
|
||||
|
||||
// --- Test 1: easy a1111Loader Full Parameter Parsing & None Filtering ---
|
||||
{
|
||||
const easyLoaderNode = {
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
title: 'Easy A1111 Loader',
|
||||
widgets: [
|
||||
{ name: 'ckpt_name', value: 'v1-5-pruned-emaonly.safetensors', type: 'combo' },
|
||||
{ name: 'vae_name', value: 'vae-ft-mse-840000-ema_pruned.safetensors', type: 'combo' },
|
||||
{ name: 'clip_skip', value: -1, type: 'number' },
|
||||
{ name: 'lora_name', value: 'None', type: 'combo' },
|
||||
{ name: 'lora_model_strength', value: 1.0, type: 'number' },
|
||||
{ name: 'lora_clip_strength', value: 1.0, type: 'number' },
|
||||
{ name: 'positive', value: 'a beautiful cyberpunk cat, neon rain', type: 'customtext', options: { multiline: true } },
|
||||
{ name: 'negative', value: 'ugly, deformed, blurry, low quality', type: 'customtext', options: { multiline: true } },
|
||||
{ name: 'empty_latent_width', value: 512, type: 'number' },
|
||||
{ name: 'empty_latent_height', value: 768, type: 'number' },
|
||||
{ name: 'batch_size', value: 1, type: 'number' },
|
||||
{ name: 'seed', value: 123456789, type: 'number' },
|
||||
{ name: 'control_after_generate', value: 'fixed', type: 'combo' },
|
||||
{ name: 'steps', value: 25, type: 'number' },
|
||||
{ name: 'cfg', value: 7.5, type: 'number' },
|
||||
{ name: 'sampler_name', value: 'euler_ancestral', type: 'combo' },
|
||||
{ name: 'scheduler', value: 'karras', type: 'combo' },
|
||||
{ name: 'denoise', value: 1.0, type: 'number' },
|
||||
],
|
||||
inputs: [],
|
||||
};
|
||||
|
||||
const graph = {
|
||||
_nodes: [easyLoaderNode],
|
||||
links: new Map(),
|
||||
};
|
||||
|
||||
const metadata = extractRecipeMetadata(graph);
|
||||
|
||||
// Verify Base Model
|
||||
assert.equal(metadata.baseModel, 'v1-5-pruned-emaonly.safetensors');
|
||||
assert.deepEqual(metadata.baseModels, ['v1-5-pruned-emaonly.safetensors']);
|
||||
|
||||
// Verify LoRA 'None' was filtered out
|
||||
assert.deepEqual(metadata.loras, [], 'LoRA value "None" must be filtered out');
|
||||
|
||||
// Verify Embedded Prompts (pure text without synthetic prefixes)
|
||||
assert.deepEqual(metadata.promptPositive, ['a beautiful cyberpunk cat, neon rain']);
|
||||
assert.deepEqual(metadata.promptNegative, ['ugly, deformed, blurry, low quality']);
|
||||
|
||||
// Verify Sampler Parameters
|
||||
assert.equal(metadata.steps, 25);
|
||||
assert.equal(metadata.cfg, 7.5);
|
||||
assert.equal(metadata.sampler_name, 'euler_ancestral');
|
||||
assert.equal(metadata.scheduler, 'karras');
|
||||
assert.equal(metadata.denoise, 1.0);
|
||||
assert.equal(metadata.seed, 123456789);
|
||||
|
||||
// Verify Latent Resolution
|
||||
assert.deepEqual(metadata.resolution, { width: 512, height: 768 });
|
||||
|
||||
// Verify node summary preserved role and all widgets (>16 widgets)
|
||||
const summary = metadata.nodes.find((n) => n.id === 1);
|
||||
assert.ok(summary);
|
||||
assert.equal(summary.role, 'both');
|
||||
assert.equal(summary.roleSource, 'embedded');
|
||||
assert.ok(summary.widgets.length >= 18, 'all 18 widgets should be preserved in summary');
|
||||
|
||||
console.log('✔ easy a1111Loader full parameter parsing OK');
|
||||
}
|
||||
|
||||
// --- Test 2: Multi-LoRA Slot Parsing (easy loraStack) & None Filtering ---
|
||||
{
|
||||
const loraStackNode = {
|
||||
id: 2,
|
||||
type: 'easy loraStack',
|
||||
title: 'Easy LoRA Stack',
|
||||
widgets: [
|
||||
{ name: 'toggle', value: true },
|
||||
{ name: 'mode', value: 'simple' },
|
||||
{ name: 'num_loras', value: 3 },
|
||||
{ name: 'lora_1_name', value: 'detail_tweaker_v1.safetensors' },
|
||||
{ name: 'lora_1_strength', value: 0.8 },
|
||||
{ name: 'lora_1_model_strength', value: 0.8 },
|
||||
{ name: 'lora_1_clip_strength', value: 0.8 },
|
||||
{ name: 'lora_2_name', value: 'None' },
|
||||
{ name: 'lora_2_strength', value: 1.0 },
|
||||
{ name: 'lora_3_name', value: 'anime_screencap_v2.safetensors' },
|
||||
{ name: 'lora_3_strength', value: 1.2 },
|
||||
{ name: 'lora_3_model_strength', value: 1.2 },
|
||||
{ name: 'lora_3_clip_strength', value: 1.0 },
|
||||
],
|
||||
inputs: [],
|
||||
};
|
||||
|
||||
const graph = {
|
||||
_nodes: [loraStackNode],
|
||||
links: new Map(),
|
||||
};
|
||||
|
||||
const metadata = extractRecipeMetadata(graph);
|
||||
|
||||
assert.equal(metadata.loras.length, 2, 'Should capture 2 LoRAs, skipping "None"');
|
||||
assert.equal(metadata.loras[0].name, 'detail_tweaker_v1.safetensors');
|
||||
assert.equal(metadata.loras[0].strength_model, 0.8);
|
||||
assert.equal(metadata.loras[1].name, 'anime_screencap_v2.safetensors');
|
||||
assert.equal(metadata.loras[1].strength_model, 1.2);
|
||||
assert.equal(metadata.loras[1].strength_clip, 1.0);
|
||||
|
||||
console.log('✔ multi-lora slot parsing & None filtering OK');
|
||||
}
|
||||
|
||||
// --- Test 3: Precedence Invariant: External CLIPTextEncode > Embedded Prompts ---
|
||||
{
|
||||
const loaderNode = {
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
title: 'Loader',
|
||||
widgets: [
|
||||
{ name: 'ckpt_name', value: 'model.safetensors' },
|
||||
{ name: 'positive', value: 'embedded loader positive text' },
|
||||
{ name: 'negative', value: 'embedded loader negative text' },
|
||||
],
|
||||
inputs: [],
|
||||
};
|
||||
|
||||
const externalPosNode = {
|
||||
id: 10,
|
||||
type: 'CLIPTextEncode',
|
||||
title: 'External Positive',
|
||||
widgets: [{ name: 'text', value: 'external linked positive prompt' }],
|
||||
inputs: [],
|
||||
};
|
||||
|
||||
const samplerNode = {
|
||||
id: 20,
|
||||
type: 'KSampler',
|
||||
title: 'Sampler',
|
||||
widgets: [
|
||||
{ name: 'seed', value: 1 },
|
||||
{ name: 'control_after_generate', value: 'fixed' },
|
||||
{ name: 'steps', value: 20 },
|
||||
{ name: 'cfg', value: 7 },
|
||||
{ name: 'sampler_name', value: 'euler' },
|
||||
{ name: 'scheduler', value: 'normal' },
|
||||
{ name: 'denoise', value: 1 },
|
||||
],
|
||||
inputs: [
|
||||
{ name: 'positive', link: 101 },
|
||||
{ name: 'negative', link: null }, // no negative linked
|
||||
],
|
||||
};
|
||||
|
||||
const links = new Map([
|
||||
[101, [101, 10, 0, 20, 0, 'CONDITIONING']],
|
||||
]);
|
||||
|
||||
const graph = {
|
||||
_nodes: [loaderNode, externalPosNode, samplerNode],
|
||||
links,
|
||||
};
|
||||
|
||||
const metadata = extractRecipeMetadata(graph);
|
||||
|
||||
// Positive should come from the external node (precedence)
|
||||
assert.deepEqual(metadata.promptPositive, ['external linked positive prompt']);
|
||||
|
||||
// Negative should fall back to the loader's embedded negative
|
||||
assert.deepEqual(metadata.promptNegative, ['embedded loader negative text']);
|
||||
|
||||
console.log('✔ external link > embedded fallback precedence OK');
|
||||
}
|
||||
|
||||
// --- Test 4: Model Specs & Model Reference Derivation for All-in-One Nodes ---
|
||||
{
|
||||
// Live node specs discovery
|
||||
const liveNode = {
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
widgets: [
|
||||
{ name: 'ckpt_name', value: 'epicrealism.safetensors' },
|
||||
{ name: 'vae_name', value: 'vae-ft-mse.safetensors' },
|
||||
{ name: 'clip_skip', value: -1 },
|
||||
{ name: 'lora_name', value: 'add_detail.safetensors' },
|
||||
],
|
||||
};
|
||||
|
||||
const liveSpecs = deriveNodeModelSpecs(liveNode);
|
||||
assert.deepEqual(liveSpecs, [
|
||||
[0, 'checkpoint', 'ckpt_name'],
|
||||
[1, 'vae', 'vae_name'],
|
||||
[3, 'lora', 'lora_name'],
|
||||
]);
|
||||
|
||||
// Serialized node specs discovery (no widgets array)
|
||||
const serializedNode = {
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
widgets_values: [
|
||||
'epicrealism.safetensors',
|
||||
'vae-ft-mse.safetensors',
|
||||
-1,
|
||||
'None', // Unselected LoRA slot
|
||||
],
|
||||
};
|
||||
|
||||
const serializedSpecs = deriveNodeModelSpecs(serializedNode);
|
||||
assert.deepEqual(serializedSpecs, [
|
||||
[0, 'checkpoint', 'ckpt_name'],
|
||||
[1, 'vae', 'vae_name'],
|
||||
[3, 'lora', 'lora_name'],
|
||||
]);
|
||||
|
||||
// Derive references from recipe workflow
|
||||
const recipe = {
|
||||
params: { baseModel: 'epicrealism.safetensors' },
|
||||
workflow: {
|
||||
nodes: [serializedNode],
|
||||
extra: {
|
||||
anomalous_hashes: {
|
||||
'1_epicrealism.safetensors': { hash: 'a'.repeat(64), size: 2000000000 },
|
||||
'1_vae-ft-mse.safetensors': { hash: 'b'.repeat(64), size: 300000000 },
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
const refs = deriveRecipeModelReferences(recipe);
|
||||
assert.equal(refs.length, 2, 'Should derive Checkpoint and VAE, skipping None LoRA');
|
||||
|
||||
// Reference 1: Checkpoint
|
||||
assert.equal(refs[0].node_id, 1);
|
||||
assert.equal(refs[0].widget_index, 0);
|
||||
assert.equal(refs[0].category, 'checkpoint');
|
||||
assert.equal(refs[0].saved_value, 'epicrealism.safetensors');
|
||||
assert.equal(refs[0].identity.status, 'verified');
|
||||
assert.equal(refs[0].identity.sha256, 'a'.repeat(64));
|
||||
|
||||
// Reference 2: VAE
|
||||
assert.equal(refs[1].node_id, 1);
|
||||
assert.equal(refs[1].widget_index, 1);
|
||||
assert.equal(refs[1].category, 'vae');
|
||||
assert.equal(refs[1].saved_value, 'vae-ft-mse.safetensors');
|
||||
assert.equal(refs[1].identity.status, 'verified');
|
||||
assert.equal(refs[1].identity.sha256, 'b'.repeat(64));
|
||||
|
||||
// Verify Composite Keys isolate the two models on the same node
|
||||
const key1 = recipeReferenceKey(refs[0]);
|
||||
const key2 = recipeReferenceKey(refs[1]);
|
||||
assert.notEqual(key1, key2, 'Composite keys must be distinct for different widgets on same node');
|
||||
|
||||
console.log('✔ model specs & composite key OK');
|
||||
}
|
||||
|
||||
// --- Test 5: syncCommonRecipeMetadata & applyRecipeWidgetChanges on All-in-One Node ---
|
||||
{
|
||||
const params = {
|
||||
baseModel: 'v1-5-pruned-emaonly.safetensors',
|
||||
baseModels: ['v1-5-pruned-emaonly.safetensors'],
|
||||
steps: 20,
|
||||
cfg: 7,
|
||||
promptPositive: ['initial positive prompt'],
|
||||
promptNegative: ['initial negative prompt'],
|
||||
nodes: [
|
||||
{
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
widgets: [
|
||||
{ name: 'ckpt_name', value: 'v1-5-pruned-emaonly.safetensors', index: 0 },
|
||||
{ name: 'positive', value: 'initial positive prompt', index: 6 },
|
||||
{ name: 'steps', value: 20, index: 13 },
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const workflow = {
|
||||
nodes: [
|
||||
{
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
widgets_values: [
|
||||
'v1-5-pruned-emaonly.safetensors', 'vae.safetensors', -1, 'None',
|
||||
1, 1, 'initial positive prompt', 'initial negative prompt',
|
||||
512, 512, 1, 1, 'fixed', 20, 7, 'euler', 'normal', 1,
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
// 1. Edit ckpt_name on easy a1111Loader
|
||||
applyRecipeWidgetChanges(params, workflow, [
|
||||
{
|
||||
nodeId: 1,
|
||||
nodeType: 'easy a1111Loader',
|
||||
widgetIndex: 0,
|
||||
widgetName: 'ckpt_name',
|
||||
previousValue: 'v1-5-pruned-emaonly.safetensors',
|
||||
value: 'v2-1-768.safetensors',
|
||||
},
|
||||
]);
|
||||
assert.equal(params.baseModel, 'v2-1-768.safetensors');
|
||||
assert.equal(workflow.nodes[0].widgets_values[0], 'v2-1-768.safetensors');
|
||||
|
||||
// 2. Edit steps on easy a1111Loader
|
||||
applyRecipeWidgetChanges(params, workflow, [
|
||||
{
|
||||
nodeId: 1,
|
||||
nodeType: 'easy a1111Loader',
|
||||
widgetIndex: 13,
|
||||
widgetName: 'steps',
|
||||
previousValue: 20,
|
||||
value: 30,
|
||||
},
|
||||
]);
|
||||
assert.equal(params.steps, 30);
|
||||
assert.equal(workflow.nodes[0].widgets_values[13], 30);
|
||||
|
||||
// 3. Edit positive prompt on easy a1111Loader
|
||||
applyRecipeWidgetChanges(params, workflow, [
|
||||
{
|
||||
nodeId: 1,
|
||||
nodeType: 'easy a1111Loader',
|
||||
widgetIndex: 6,
|
||||
widgetName: 'positive',
|
||||
previousValue: 'initial positive prompt',
|
||||
value: 'updated cyberpunk city positive prompt',
|
||||
},
|
||||
]);
|
||||
assert.deepEqual(params.promptPositive, ['updated cyberpunk city positive prompt']);
|
||||
assert.equal(workflow.nodes[0].widgets_values[6], 'updated cyberpunk city positive prompt');
|
||||
|
||||
console.log('✔ syncCommonRecipeMetadata & applyRecipeWidgetChanges on all-in-one nodes OK');
|
||||
}
|
||||
|
||||
// --- Test 6: Parameter Choices Extraction from All-in-One Nodes ---
|
||||
{
|
||||
const node = {
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
title: 'AllInOne',
|
||||
widgets: [
|
||||
{ name: 'ckpt_name', value: 'model.safetensors' },
|
||||
{ name: 'positive', value: 'prompt' },
|
||||
{ name: 'steps', value: 25 },
|
||||
],
|
||||
};
|
||||
|
||||
const graph = {
|
||||
_nodes: [node],
|
||||
};
|
||||
|
||||
const choices = extractRecipeParameterChoices(graph);
|
||||
assert.equal(choices.length, 3);
|
||||
assert.equal(choices[0].widgetName, 'ckpt_name');
|
||||
assert.equal(choices[1].widgetName, 'positive');
|
||||
assert.equal(choices[2].widgetName, 'steps');
|
||||
|
||||
const metadataChoices = extractRecipeParameterChoicesFromMetadata({
|
||||
nodes: [
|
||||
{
|
||||
id: 1,
|
||||
type: 'easy a1111Loader',
|
||||
widgets: [
|
||||
{ name: 'ckpt_name', value: 'model.safetensors', index: 0 },
|
||||
{ name: 'positive', value: 'prompt', index: 1 },
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
assert.equal(metadataChoices.length, 2);
|
||||
assert.equal(metadataChoices[0].widgetName, 'ckpt_name');
|
||||
assert.equal(metadataChoices[1].widgetName, 'positive');
|
||||
|
||||
console.log('✔ parameter choices extraction OK');
|
||||
}
|
||||
|
||||
// --- Test 7: Regressions - unrelated widgets must not rewrite the recipe summary ---
|
||||
{
|
||||
// Extraction: an upscaler's model_name is not a base model; duplicate LoRAs keep one entry per loader.
|
||||
const graph = {
|
||||
_nodes: [
|
||||
{ id: 1, type: 'CheckpointLoaderSimple', widgets: [{ name: 'ckpt_name', value: 'base.safetensors' }], inputs: [] },
|
||||
{ id: 2, type: 'UpscaleModelLoader', widgets: [{ name: 'model_name', value: 'RealESRGAN_x4.safetensors' }], inputs: [] },
|
||||
{ id: 3, type: 'LoraLoader', widgets: [{ name: 'lora_name', value: 'style.safetensors' }, { name: 'strength_model', value: 0.5 }, { name: 'strength_clip', value: 0.5 }], inputs: [] },
|
||||
{ id: 4, type: 'LoraLoader', widgets: [{ name: 'lora_name', value: 'style.safetensors' }, { name: 'strength_model', value: 0.9 }, { name: 'strength_clip', value: 0.9 }], inputs: [] },
|
||||
{ id: 5, type: 'Seed (rgthree)', widgets: [{ name: 'seed', value: 42 }], inputs: [] },
|
||||
],
|
||||
links: new Map(),
|
||||
};
|
||||
const metadata = extractRecipeMetadata(graph);
|
||||
assert.deepEqual(metadata.baseModels, ['base.safetensors']);
|
||||
assert.equal(metadata.loras.length, 2, 'each LoraLoader keeps its own entry');
|
||||
assert.equal(metadata.seed, null, 'a standalone seed node is not a sampler');
|
||||
|
||||
// Sync: the same rules apply to edits.
|
||||
const params = {
|
||||
baseModel: 'base.safetensors',
|
||||
baseModels: ['base.safetensors'],
|
||||
seed: 1,
|
||||
loras: [{ name: 'a.safetensors', strength_model: 1, strength_clip: 1 }, { name: 'b.safetensors', strength_model: 1, strength_clip: 1 }],
|
||||
nodes: [
|
||||
{ id: 2, type: 'UpscaleModelLoader', widgets: [{ name: 'model_name', value: 'x4.safetensors', index: 0 }] },
|
||||
{ id: 5, type: 'Seed (rgthree)', widgets: [{ name: 'seed', value: 1, index: 0 }] },
|
||||
{ id: 7, type: 'LoraLoader', widgets: [{ name: 'lora_name', value: 'b.safetensors', index: 0 }, { name: 'strength_model', value: 1, index: 1 }] },
|
||||
{ id: 8, type: 'SomeCustomNode', widgets: [{ name: 'lora_strength', value: 1, index: 0 }] },
|
||||
],
|
||||
};
|
||||
const workflow = { nodes: [
|
||||
{ id: 2, type: 'UpscaleModelLoader', widgets_values: ['x4.safetensors'] },
|
||||
{ id: 5, type: 'Seed (rgthree)', widgets_values: [1] },
|
||||
{ id: 7, type: 'LoraLoader', widgets_values: ['b.safetensors', 1, 1] },
|
||||
{ id: 8, type: 'SomeCustomNode', widgets_values: [1] },
|
||||
] };
|
||||
applyRecipeWidgetChanges(params, workflow, [
|
||||
{ nodeId: 2, nodeType: 'UpscaleModelLoader', widgetIndex: 0, widgetName: 'model_name', previousValue: 'x4.safetensors', value: 'x8.safetensors' },
|
||||
{ nodeId: 5, nodeType: 'Seed (rgthree)', widgetIndex: 0, widgetName: 'seed', previousValue: 1, value: 999 },
|
||||
{ nodeId: 7, nodeType: 'LoraLoader', widgetIndex: 1, widgetName: 'strength_model', previousValue: 1, value: 0.3 },
|
||||
{ nodeId: 8, nodeType: 'SomeCustomNode', widgetIndex: 0, widgetName: 'lora_strength', previousValue: 1, value: 0 },
|
||||
]);
|
||||
assert.equal(params.baseModel, 'base.safetensors', 'upscaler model_name must not replace the base model');
|
||||
assert.equal(params.seed, 1, 'a seed widget outside samplers must not replace the recipe seed');
|
||||
assert.equal(params.loras[0].strength_model, 1, 'unrelated LoRA entry untouched');
|
||||
assert.equal(params.loras[1].strength_model, 0.3, 'strength applies to the edited loader\'s LoRA');
|
||||
assert.equal(workflow.nodes[1].widgets_values[0], 999, 'the workflow itself is still edited');
|
||||
|
||||
// Model references: only verified layouts; no file-name guessing, no placeholder values.
|
||||
assert.deepEqual(deriveNodeModelSpecs({ type: 'SomeCustomLoader', widgets_values: ['sdxl_0.9vae.safetensors'] }), []);
|
||||
assert.deepEqual(deriveNodeModelSpecs('CheckpointLoaderSimple'), [[0, 'checkpoint', 'checkpoint']]);
|
||||
const refs = deriveRecipeModelReferences({ workflow: { nodes: [
|
||||
{ id: 9, type: 'easy fullLoader', widgets_values: ['sdxl_0.9vae.safetensors', 'Default', 'Baked VAE', -2, 'None'] },
|
||||
] } });
|
||||
assert.deepEqual(refs.map(r => [r.category, r.saved_value]), [['checkpoint', 'sdxl_0.9vae.safetensors']]);
|
||||
|
||||
console.log('✔ regressions: unrelated widgets, duplicate LoRAs, placeholder values OK');
|
||||
}
|
||||
|
||||
console.log('\nAll 7 All-in-One Loader semantic compatibility test suites PASSED!\n');
|
||||
@@ -0,0 +1,169 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
applyRecipeWidgetChanges,
|
||||
captureRecipeDraft,
|
||||
analyseRecipeScope,
|
||||
extractRecipeMetadata,
|
||||
extractRecipeParameterChoicesFromMetadata,
|
||||
} from '../web/modules/recipe_parser.js';
|
||||
|
||||
const longNegative = 'negative prompt '.repeat(40).trim();
|
||||
const workflow = {
|
||||
nodes: [
|
||||
{ id: 1, type: 'CLIPTextEncode', title: 'positive', widgets_values: ['a full positive prompt'] },
|
||||
{ id: 2, type: 'CLIPTextEncode', title: 'negative', widgets_values: [longNegative] },
|
||||
{ id: 3, type: 'KSampler', title: 'Sampler', widgets_values: [42, 'fixed', 24, 6.5, 'euler', 'normal', 1] },
|
||||
],
|
||||
links: [
|
||||
[10, 1, 0, 3, 1, 'CONDITIONING'],
|
||||
[11, 2, 0, 3, 2, 'CONDITIONING'],
|
||||
],
|
||||
};
|
||||
|
||||
const graph = {
|
||||
_nodes: [
|
||||
{
|
||||
id: 1,
|
||||
type: 'CLIPTextEncode',
|
||||
title: 'positive',
|
||||
widgets: [{ name: 'text', value: 'a full positive prompt', type: 'text' }],
|
||||
inputs: [],
|
||||
},
|
||||
{
|
||||
id: 2,
|
||||
type: 'CLIPTextEncode',
|
||||
title: 'negative',
|
||||
widgets: [{ name: 'text', value: longNegative, type: 'text' }],
|
||||
inputs: [],
|
||||
},
|
||||
{
|
||||
id: 3,
|
||||
type: 'KSampler',
|
||||
title: 'Sampler',
|
||||
widgets: [
|
||||
{ name: 'seed', value: 42 }, { name: 'control_after_generate', value: 'fixed' },
|
||||
{ name: 'steps', value: 24 }, { name: 'cfg', value: 6.5 },
|
||||
{ name: 'sampler_name', value: 'euler' }, { name: 'scheduler', value: 'normal' },
|
||||
{ name: 'denoise', value: 1 },
|
||||
],
|
||||
inputs: [
|
||||
{ name: 'positive', link: 10 },
|
||||
{ name: 'negative', link: 11 },
|
||||
],
|
||||
},
|
||||
],
|
||||
links: new Map(workflow.links.map((link) => [link[0], link])),
|
||||
serializeCalls: 0,
|
||||
serialize() {
|
||||
this.serializeCalls += 1;
|
||||
return JSON.parse(JSON.stringify(workflow));
|
||||
},
|
||||
};
|
||||
|
||||
const draft = captureRecipeDraft(graph);
|
||||
assert.equal(graph.serializeCalls, 1, 'recipe capture serializes exactly once');
|
||||
assert.deepEqual(draft.workflow, workflow, 'draft keeps the authoritative serialized workflow');
|
||||
assert.equal(draft.stats.nodeCount, 3);
|
||||
assert.equal(draft.stats.linkCount, 2);
|
||||
|
||||
const completeScope = analyseRecipeScope({
|
||||
_nodes: [{
|
||||
constructor: { nodeData: { output_node: true, input: { required: { images: ['IMAGE', {}] }, optional: {} } } },
|
||||
inputs: [{ name: 'images', link: 1 }],
|
||||
widgets: [],
|
||||
}],
|
||||
});
|
||||
assert.equal(completeScope.scope, 'complete', 'a runnable graph with an output node is a complete recipe');
|
||||
|
||||
const partialScope = analyseRecipeScope({
|
||||
_nodes: [{
|
||||
constructor: { nodeData: { output_node: false, input: { required: { model: ['MODEL', {}] }, optional: {} } } },
|
||||
inputs: [{ name: 'model', link: null }],
|
||||
widgets: [],
|
||||
}],
|
||||
});
|
||||
assert.equal(partialScope.scope, 'partial', 'an open required connection makes a partial recipe');
|
||||
assert.equal(partialScope.boundaryInputCount, 1);
|
||||
|
||||
const nestedOutput = {
|
||||
constructor: { nodeData: { output_node: true, input: { required: {}, optional: {} } } },
|
||||
inputs: [],
|
||||
widgets: [],
|
||||
};
|
||||
const subgraphScope = analyseRecipeScope({
|
||||
_nodes: [{
|
||||
constructor: { nodeData: { output_node: false, input: { required: {}, optional: {} } } },
|
||||
inputs: [],
|
||||
widgets: [],
|
||||
getInnerNodes: () => [nestedOutput],
|
||||
}],
|
||||
});
|
||||
assert.equal(subgraphScope.scope, 'complete', 'output nodes inside a subgraph close the complete workflow');
|
||||
|
||||
const metadata = extractRecipeMetadata(graph);
|
||||
assert.equal(metadata.nodes.find((node) => node.id === 1).role, 'positive', 'positive prompt follows the linked conditioning input');
|
||||
assert.equal(metadata.nodes.find((node) => node.id === 2).role, 'negative', 'negative prompt follows the linked input');
|
||||
const negativeSummary = metadata.nodes.find((node) => node.id === 2).widgets[0].value;
|
||||
assert.equal(negativeSummary.length, 320, 'generic metadata remains a bounded browsing summary');
|
||||
|
||||
const editable = extractRecipeParameterChoicesFromMetadata(metadata, workflow);
|
||||
const negativeChoice = editable.find((choice) => choice.nodeId === 2 && choice.widgetName === 'text');
|
||||
assert.equal(negativeChoice.value, longNegative, 'editing choices resolve the full workflow value');
|
||||
|
||||
applyRecipeWidgetChanges(metadata, workflow, [{
|
||||
...negativeChoice,
|
||||
previousValue: longNegative,
|
||||
value: 'replacement negative prompt',
|
||||
}]);
|
||||
assert.equal(workflow.nodes[1].widgets_values[0], 'replacement negative prompt');
|
||||
assert.deepEqual(metadata.promptNegative, ['replacement negative prompt']);
|
||||
assert.equal(workflow.nodes[0].widgets_values[0], 'a full positive prompt', 'only the selected widget changes');
|
||||
|
||||
const conservativeGraph = {
|
||||
_nodes: [
|
||||
{
|
||||
id: 20,
|
||||
type: 'CLIPTextEncode',
|
||||
title: 'negative-looking but disconnected',
|
||||
widgets: [{ name: 'text', value: 'do not guess me', type: 'text' }],
|
||||
inputs: [],
|
||||
},
|
||||
{
|
||||
id: 21,
|
||||
type: 'ThirdPartyCLIPTextEncode',
|
||||
title: 'negative',
|
||||
widgets: [{ name: 'text', value: 'third party prompt', type: 'text' }],
|
||||
inputs: [],
|
||||
},
|
||||
],
|
||||
links: new Map(),
|
||||
};
|
||||
const conservativeMetadata = extractRecipeMetadata(conservativeGraph);
|
||||
assert.equal(conservativeMetadata.nodes.find((node) => node.id === 20).role, 'unknown', 'disconnected official text is not guessed from its title');
|
||||
assert.equal(conservativeMetadata.nodes.find((node) => node.id === 21).role, undefined, 'third-party text nodes are not automatically classified');
|
||||
assert.deepEqual(conservativeMetadata.promptPositive, [], 'unresolved text is not inserted into positive prompts');
|
||||
assert.deepEqual(conservativeMetadata.promptNegative, [], 'unresolved text is not inserted into negative prompts');
|
||||
|
||||
const sharedText = {
|
||||
id: 30,
|
||||
type: 'CLIPTextEncode',
|
||||
widgets: [{ name: 'text', value: 'shared conditioning', type: 'text' }],
|
||||
inputs: [],
|
||||
};
|
||||
const sharedSampler = {
|
||||
id: 31,
|
||||
type: 'KSampler',
|
||||
widgets: [],
|
||||
inputs: [{ name: 'positive', link: 30 }, { name: 'negative', link: 31 }],
|
||||
};
|
||||
const sharedGraph = {
|
||||
_nodes: [sharedText, sharedSampler],
|
||||
links: new Map([
|
||||
[30, [30, 30, 0, 31, 1, 'CONDITIONING']],
|
||||
[31, [31, 30, 0, 31, 2, 'CONDITIONING']],
|
||||
]),
|
||||
};
|
||||
const sharedMetadata = extractRecipeMetadata(sharedGraph);
|
||||
assert.equal(sharedMetadata.nodes.find((node) => node.id === 30).role, 'both', 'one official text node may supply both roles');
|
||||
|
||||
console.log('recipe_parser_roundtrip: ok');
|
||||
@@ -0,0 +1,34 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
findWorkflowHashRecord,
|
||||
mergeRecipeHashRecords,
|
||||
remapWorkflowHashRecords,
|
||||
} from '../web/modules/recipe_provenance.js';
|
||||
|
||||
const hash = 'a'.repeat(64);
|
||||
const workflow = {
|
||||
nodes: [{ id: 4, type: 'LoraLoader', widgets_values: ['styles\\detail.safetensors', 0.8, 0.8] }],
|
||||
extra: {
|
||||
anomalous_hashes: {
|
||||
'4_styles\\detail.safetensors': { hash, size: 12345 },
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
assert.deepEqual(
|
||||
findWorkflowHashRecord(workflow, 4, 'styles/detail.safetensors'),
|
||||
{ hash, size: 12345 },
|
||||
'path separators do not break recipe provenance lookup',
|
||||
);
|
||||
|
||||
const remapped = remapWorkflowHashRecords(workflow, new Map([['4', 19]]));
|
||||
assert.deepEqual(remapped['19_styles/detail.safetensors'], { hash, size: 12345 });
|
||||
assert.deepEqual(remapped['19_styles\\detail.safetensors'], { hash, size: 12345 });
|
||||
|
||||
const graph = { extra: { anomalous_hashes: { '1_host.safetensors': { hash: 'b'.repeat(64), size: 9 } } } };
|
||||
const mergedCount = mergeRecipeHashRecords(graph, workflow, new Map([['4', 19]]));
|
||||
assert.equal(mergedCount, 2);
|
||||
assert.ok(graph.extra.anomalous_hashes['1_host.safetensors'], 'host provenance is preserved');
|
||||
assert.ok(graph.extra.anomalous_hashes['19_styles/detail.safetensors'], 'appended node gets remapped provenance');
|
||||
|
||||
console.log('recipe provenance roundtrip: ok');
|
||||
@@ -0,0 +1,82 @@
|
||||
import assert from 'node:assert/strict';
|
||||
|
||||
|
||||
class TestClassList {
|
||||
constructor() { this.values = new Set(); }
|
||||
add(...values) { values.forEach(value => this.values.add(value)); }
|
||||
remove(...values) { values.forEach(value => this.values.delete(value)); }
|
||||
toggle(value, force) {
|
||||
if (force) this.values.add(value);
|
||||
else this.values.delete(value);
|
||||
}
|
||||
contains(value) { return this.values.has(value); }
|
||||
}
|
||||
|
||||
|
||||
class TestElement {
|
||||
constructor(tagName) {
|
||||
this.tagName = tagName;
|
||||
this.children = [];
|
||||
this.attributes = new Map();
|
||||
this.classList = new TestClassList();
|
||||
this.style = {};
|
||||
this.textContent = '';
|
||||
this.id = '';
|
||||
}
|
||||
set className(value) {
|
||||
this.classList = new TestClassList();
|
||||
String(value).split(/\s+/).filter(Boolean).forEach(name => this.classList.add(name));
|
||||
}
|
||||
appendChild(child) { this.children.push(child); return child; }
|
||||
remove() { this.removed = true; }
|
||||
setAttribute(name, value) { this.attributes.set(name, String(value)); }
|
||||
removeAttribute(name) { this.attributes.delete(name); }
|
||||
querySelector(selector) {
|
||||
const className = selector.startsWith('.') ? selector.slice(1) : '';
|
||||
for (const child of this.children) {
|
||||
if (className && child.classList.contains(className)) return child;
|
||||
const nested = child.querySelector(selector);
|
||||
if (nested) return nested;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
const body = new TestElement('body');
|
||||
globalThis.document = {
|
||||
body,
|
||||
createElement: tagName => new TestElement(tagName),
|
||||
getElementById(id) {
|
||||
const visit = element => {
|
||||
if (element.id === id) return element;
|
||||
for (const child of element.children) {
|
||||
const found = visit(child);
|
||||
if (found) return found;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
return visit(body);
|
||||
},
|
||||
};
|
||||
globalThis.anomalous_browser_lang = 'en';
|
||||
|
||||
const { updateScanProgress, finishScanProgress } = await import('../web/modules/scan_progress.js');
|
||||
updateScanProgress({ scanning: true, phase: 'enumerating' });
|
||||
const panel = document.getElementById('anomalous-scan-progress');
|
||||
assert.ok(panel);
|
||||
assert.equal(panel.querySelector('.anomalous-scan-progress-fill').classList.contains('is-indeterminate'), true);
|
||||
|
||||
updateScanProgress({ scanning: true, phase: 'scanning', total: 4, current: 2, filename: 'model.safetensors' });
|
||||
assert.equal(panel.querySelector('.anomalous-scan-progress-fill').style.width, '50%');
|
||||
assert.match(panel.querySelector('.anomalous-scan-progress-item').textContent, /2\/4/);
|
||||
|
||||
finishScanProgress('Custom finish message');
|
||||
assert.equal(panel.querySelector('.anomalous-scan-progress-detail').textContent, 'Custom finish message');
|
||||
assert.equal(panel.classList.contains('is-complete'), true);
|
||||
|
||||
// Calling without arguments preserves existing custom detail
|
||||
finishScanProgress();
|
||||
assert.equal(panel.querySelector('.anomalous-scan-progress-detail').textContent, 'Custom finish message');
|
||||
|
||||
console.log('scan progress smoke OK');
|
||||
@@ -0,0 +1,123 @@
|
||||
// Workflow share codes (web/modules/share_code.js): the two spellings, the lean form and its
|
||||
// self-check, the fall back to the whole workflow, and the old AMB0/AMB1 codes.
|
||||
import assert from 'node:assert/strict';
|
||||
import { deflateRawSync } from 'node:zlib';
|
||||
import {
|
||||
decodeShareCode, encodeShareCode, fromHanzi, fromLetters, leanWorkflow, rebuildWorkflow, sameWorkflow, toHanzi, toLetters,
|
||||
} from '../web/modules/share_code.js';
|
||||
|
||||
// Bytes survive both spellings at every length.
|
||||
for (let length = 0; length < 64; length += 1) {
|
||||
const bytes = Uint8Array.from({ length }, (_, index) => (index * 97 + length * 31) % 256);
|
||||
assert.deepEqual(fromHanzi(toHanzi(bytes)), bytes, `hanzi ${length}`);
|
||||
assert.deepEqual(fromLetters(toLetters(bytes)), bytes, `letters ${length}`);
|
||||
}
|
||||
assert.throws(() => fromHanzi('abc'));
|
||||
|
||||
// Fresh nodes of each type, as the canvas would make them.
|
||||
const port = (name, type, extra = {}) => ({ name, type, ...extra });
|
||||
const TEMPLATES = {
|
||||
CheckpointLoaderSimple: { inputs: [], outputs: [port('MODEL', 'MODEL', { links: null }), port('CLIP', 'CLIP', { links: null }), port('VAE', 'VAE', { links: null })], size: [315, 98] },
|
||||
CLIPTextEncode: { inputs: [port('clip', 'CLIP', { link: null }), port('text', 'STRING', { widget: { name: 'text' }, link: null })], outputs: [port('CONDITIONING', 'CONDITIONING', { links: null })], size: [400, 200] },
|
||||
KSampler: {
|
||||
inputs: [port('model', 'MODEL', { link: null }), port('positive', 'CONDITIONING', { link: null }), port('negative', 'CONDITIONING', { link: null })],
|
||||
outputs: [port('LATENT', 'LATENT', { links: null })], size: [315, 262],
|
||||
},
|
||||
'Lora Loader Stack (rgthree)': { inputs: [port('model', 'MODEL', { link: null }), port('clip', 'CLIP', { link: null })], outputs: [port('MODEL', 'MODEL', { links: null }), port('CLIP', 'CLIP', { links: null })], size: [340, 200] },
|
||||
};
|
||||
const CORE = new Set(['CheckpointLoaderSimple', 'CLIPTextEncode', 'KSampler']);
|
||||
const here = { templateOf: type => TEMPLATES[type] || null, isCore: type => CORE.has(type) };
|
||||
const nowhere = { templateOf: () => null, isCore: () => false }; // a computer without these node packs
|
||||
|
||||
const node = (id, type, widgets, pos, more = {}) => {
|
||||
const template = TEMPLATES[type];
|
||||
return {
|
||||
id, type, pos, size: template.size, flags: {}, order: id, mode: 0,
|
||||
inputs: template.inputs.map(input => ({ ...input })), outputs: template.outputs.map(output => ({ ...output, links: [] })),
|
||||
properties: { 'Node name for S&R': type }, widgets_values: widgets, ...more,
|
||||
};
|
||||
};
|
||||
function workflow() {
|
||||
const nodes = [
|
||||
node(4, 'CheckpointLoaderSimple', ['sd_xl_base_1.0.safetensors'], [26, 474]),
|
||||
node(6, 'CLIPTextEncode', ['a photo of a cat, masterpiece, best quality'], [415, 186], { title: 'Positive' }),
|
||||
node(7, 'CLIPTextEncode', ['blurry, lowres'], [413, 389]),
|
||||
node(3, 'KSampler', [156680208700286, 'randomize', 20, 8, 'euler', 'normal', 1], [863, 186]),
|
||||
node(10, 'Lora Loader Stack (rgthree)', ['cute.safetensors', 0.8, 'None', 1, 'None', 1, 'None', 1], [200, 700], { mode: 4 }),
|
||||
];
|
||||
const links = [[1, 4, 1, 6, 0, 'CLIP'], [2, 4, 1, 7, 0, 'CLIP'], [3, 6, 0, 3, 1, 'CONDITIONING'], [4, 7, 0, 3, 2, 'CONDITIONING'], [5, 4, 0, 10, 0, 'MODEL'], [6, 10, 0, 3, 0, 'MODEL']];
|
||||
const byId = new Map(nodes.map(item => [item.id, item]));
|
||||
for (const [id, from, fromSlot, to, toSlot] of links) {
|
||||
byId.get(from).outputs[fromSlot].links.push(id);
|
||||
byId.get(to).inputs[toSlot].link = id;
|
||||
}
|
||||
return {
|
||||
last_node_id: 10, last_link_id: 6, nodes, links,
|
||||
groups: [{ title: 'Prompts', bounding: [400, 150, 450, 450], color: '#3f789e', font_size: 24 }],
|
||||
config: {}, version: 0.4,
|
||||
extra: { ds: { scale: 1, offset: [0, 0] }, anomalous_hashes: { '4_sd_xl_base_1.0.safetensors': { hash: 'ab'.repeat(32), size: 6938078334 } } },
|
||||
};
|
||||
}
|
||||
|
||||
// The lean form rebuilds the same workflow; only differences from a fresh node are kept.
|
||||
const original = workflow();
|
||||
const lean = leanWorkflow(original, here);
|
||||
const rebuilt = rebuildWorkflow(lean, here);
|
||||
assert.ok(sameWorkflow(original, rebuilt));
|
||||
assert.equal(lean.n.find(item => item[0] === 3).length, 4); // a plain KSampler: no extras at all
|
||||
assert.ok(lean.n.find(item => item[0] === 10)[4].i, 'a node pack node keeps its ports');
|
||||
assert.equal(rebuilt.nodes.find(item => item.id === 6).title, 'Positive');
|
||||
assert.equal(rebuilt.nodes.find(item => item.id === 10).mode, 4);
|
||||
assert.deepEqual(rebuilt.groups[0].bounding, [400, 150, 450, 450]);
|
||||
assert.equal(rebuilt.extra.anomalous_hashes['4_sd_xl_base_1.0.safetensors'].hash, 'ab'.repeat(32));
|
||||
assert.equal(rebuilt.extra.ds, undefined);
|
||||
|
||||
// A workflow as ComfyUI saves it leaves out the values of nodes without any: still the same.
|
||||
const saved = rebuildWorkflow(lean, here);
|
||||
for (const item of saved.nodes) if (!item.widgets_values.length) delete item.widgets_values;
|
||||
assert.ok(sameWorkflow(saved, rebuilt) && sameWorkflow(rebuilt, saved));
|
||||
|
||||
// Codes: both spellings open to the same workflow, much shorter than the old full code.
|
||||
const code = await encodeShareCode(original, here);
|
||||
assert.equal(code.whole, false);
|
||||
assert.ok(code.hanzi.startsWith('AMB2-') && code.letters.startsWith('AMB2-'));
|
||||
const old = 'AMB0-' + Buffer.from(deflateRawSync(JSON.stringify(original))).toString('base64');
|
||||
assert.ok(code.hanzi.length * 3 < old.length, `${code.hanzi.length} vs ${old.length}`);
|
||||
assert.ok(code.hanzi.length < code.letters.length);
|
||||
for (const spelling of [code.hanzi, code.letters]) assert.ok(sameWorkflow(original, await decodeShareCode(spelling, here)));
|
||||
// A chat that wraps the text or adds spaces does not break it.
|
||||
const wrapped = code.hanzi.replace(/(.{20})/gu, '$1\n ');
|
||||
assert.ok(sameWorkflow(original, await decodeShareCode(wrapped, here)));
|
||||
|
||||
// On a computer without the node pack, its node still has its ports and links.
|
||||
const elsewhere = await decodeShareCode(code.hanzi, { templateOf: type => (CORE.has(type) ? TEMPLATES[type] : null) });
|
||||
const stack = elsewhere.nodes.find(item => item.id === 10);
|
||||
assert.deepEqual(stack.inputs.map(input => input.link), [5, null]);
|
||||
assert.deepEqual(stack.outputs[0].links, [6]);
|
||||
assert.ok(sameWorkflow(original, elsewhere));
|
||||
|
||||
// A core node whose ports differ from a fresh one (an extra input) keeps them.
|
||||
const grown = workflow();
|
||||
grown.nodes.find(item => item.id === 3).inputs.push({ name: 'latent_image', type: 'LATENT', link: null });
|
||||
const grownLean = leanWorkflow(grown, here);
|
||||
assert.ok(grownLean.n.find(item => item[0] === 3)[4].i);
|
||||
assert.ok(sameWorkflow(grown, rebuildWorkflow(grownLean, here)));
|
||||
|
||||
// What the lean form cannot hold goes whole: a subgraph, or nodes it cannot rebuild.
|
||||
const withSubgraph = { ...workflow(), definitions: { subgraphs: [{ id: 'x' }] } };
|
||||
const wholeCode = await encodeShareCode(withSubgraph, here);
|
||||
assert.equal(wholeCode.whole, true);
|
||||
assert.deepEqual((await decodeShareCode(wholeCode.letters, here)).definitions, withSubgraph.definitions);
|
||||
const unknownHere = await encodeShareCode(workflow(), nowhere); // every node keeps its ports: still lean
|
||||
assert.equal(unknownHere.whole, false);
|
||||
|
||||
// Old codes still open; AMB1 is laid out by link depth.
|
||||
assert.ok(sameWorkflow(original, await decodeShareCode(old, here)));
|
||||
const skeleton = workflow();
|
||||
skeleton.nodes.forEach((item) => { delete item.pos; });
|
||||
const amb1 = await decodeShareCode('AMB1-' + Buffer.from(deflateRawSync(JSON.stringify(skeleton))).toString('base64'), here);
|
||||
assert.deepEqual(amb1.nodes.find(item => item.id === 3).pos, [800, 0]);
|
||||
await assert.rejects(decodeShareCode('hello', here));
|
||||
await assert.rejects(decodeShareCode('AMB2-' + '一'.repeat(3) + 'x', here));
|
||||
|
||||
console.log(`share_code: ok (hanzi ${code.hanzi.length}, letters ${code.letters.length}, old ${old.length})`);
|
||||
@@ -0,0 +1,37 @@
|
||||
// Which page opening the browser shows for the selected node, and the rail's number keys
|
||||
// (web/modules/shell_open_rules.js).
|
||||
import assert from 'node:assert/strict';
|
||||
import { pageForNode, railPageForKey } from '../web/modules/shell_open_rules.js';
|
||||
|
||||
const kinds = {
|
||||
takesPrompt: node => Boolean(node.prompt),
|
||||
hasModel: node => Boolean(node.model),
|
||||
};
|
||||
|
||||
// Opening the browser: a text node opens Prompt Studio, any other node Current node.
|
||||
assert.equal(pageForNode(null, kinds), null); // nothing selected: the page used last
|
||||
assert.equal(pageForNode({ prompt: true }, kinds), 'prompts'); // CLIP Text Encode
|
||||
assert.equal(pageForNode({}, kinds), 'assistant'); // KSampler
|
||||
assert.equal(pageForNode({ model: true }, kinds), 'assistant'); // a loader
|
||||
assert.equal(pageForNode({ prompt: true, model: true }, kinds), 'assistant'); // a loader with prompt boxes: its models first
|
||||
|
||||
// Number keys: 1-9 from the top of the rail.
|
||||
const pages = ['home', 'activity', 'models', 'gallery', 'recipes', 'combos', 'prompts', 'voices', 'audio-gallery'].map(page => ({ page }));
|
||||
const canvas = { closest: () => null };
|
||||
const field = { closest: selector => (selector.includes('textarea') ? {} : null) };
|
||||
const key = (code, extra = {}) => ({ code, key: code.slice(-1), target: canvas, ...extra });
|
||||
|
||||
assert.equal(railPageForKey(key('Digit1'), pages), 'home');
|
||||
assert.equal(railPageForKey(key('Digit7'), pages), 'prompts');
|
||||
assert.equal(railPageForKey(key('Numpad9'), pages), 'audio-gallery');
|
||||
assert.equal(railPageForKey(key('Digit0'), pages), null);
|
||||
assert.equal(railPageForKey(key('KeyW'), pages), null); // W stays ComfyUI's (the Workflows sidebar)
|
||||
assert.equal(railPageForKey(key('Digit3', { target: field }), pages), null); // typing a number into a box
|
||||
assert.equal(railPageForKey(key('Digit3', { ctrlKey: true }), pages), null);
|
||||
assert.equal(railPageForKey(key('Digit3', { shiftKey: true }), pages), null);
|
||||
assert.equal(railPageForKey(key('Digit3', { isComposing: true }), pages), null); // a Chinese input method choosing a word
|
||||
assert.equal(railPageForKey(key('Digit3', { repeat: true }), pages), null);
|
||||
assert.equal(railPageForKey({ code: '', key: '4', target: canvas }, pages), 'gallery'); // no code: by the key
|
||||
assert.equal(railPageForKey(key('Digit9'), pages.slice(0, 3)), null); // fewer pages than keys
|
||||
|
||||
console.log('shell_open_rules: ok');
|
||||
@@ -0,0 +1,101 @@
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
from aiohttp import web
|
||||
from aiohttp.test_utils import make_mocked_request
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import activity_log # noqa: E402
|
||||
|
||||
|
||||
class ActivityLogTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.directory = tempfile.TemporaryDirectory()
|
||||
self.path = str(Path(self.directory.name, 'activity_log.json'))
|
||||
self.patch = mock.patch.object(activity_log, 'log_path', return_value=self.path)
|
||||
self.patch.start()
|
||||
activity_log._entries = None
|
||||
|
||||
def tearDown(self):
|
||||
self.patch.stop()
|
||||
activity_log._entries = None
|
||||
self.directory.cleanup()
|
||||
|
||||
def test_entries_are_newest_first_bounded_and_saved(self):
|
||||
with mock.patch.object(activity_log, 'MAX_ENTRIES', 3):
|
||||
for index in range(5):
|
||||
activity_log.add_entry('file', 'recipe_save', f'r{index}')
|
||||
self.assertEqual([e['target'] for e in activity_log.list_entries()], ['r4', 'r3', 'r2'])
|
||||
activity_log._entries = None # read back from disk
|
||||
self.assertEqual([e['target'] for e in activity_log.list_entries()], ['r4', 'r3', 'r2'])
|
||||
self.assertEqual([e['target'] for e in activity_log.list_entries(source='canvas')], [])
|
||||
activity_log.clear()
|
||||
self.assertEqual(json.loads(Path(self.path).read_text()), [])
|
||||
|
||||
def test_log_stays_under_its_byte_limit(self):
|
||||
with mock.patch.object(activity_log, 'MAX_LOG_BYTES', 20_000):
|
||||
for index in range(200):
|
||||
activity_log.add_entry('file', 'note_save', f'{index}-' + 'x' * 300)
|
||||
self.assertLessEqual(Path(self.path).stat().st_size, 20_000)
|
||||
self.assertTrue(activity_log.list_entries()[0]['target'].startswith('199-'))
|
||||
|
||||
def test_long_values_are_shortened(self):
|
||||
entry = activity_log.add_entry('file', 'note_save', 'x' * 1000)
|
||||
self.assertLessEqual(len(entry['target']), activity_log.MAX_VALUE_CHARS + 1)
|
||||
|
||||
def test_a_change_late_in_a_long_text_still_shows(self):
|
||||
before = ', '.join(f'tag{i}' for i in range(200)) # far longer than MAX_VALUE_CHARS
|
||||
after = before.replace('tag180', 'tag180, extra')
|
||||
item = activity_log._clean_changes([{'kind': 'changed', 'node': 'n', 'widget': 'text',
|
||||
'before': before, 'after': after}])[0]
|
||||
self.assertNotEqual(item['before'], item['after'])
|
||||
self.assertIn('extra', item['after'])
|
||||
self.assertTrue(item['before'].startswith('…'))
|
||||
self.assertLessEqual(len(item['after']), activity_log.MAX_VALUE_CHARS + 2)
|
||||
|
||||
def test_failure_responses_are_not_recorded(self):
|
||||
ok = web.json_response({'success': True})
|
||||
self.assertFalse(activity_log._failed(ok))
|
||||
for bad in (web.json_response({'status': 'error'}), web.json_response({'success': False}),
|
||||
web.json_response({'ok': True}, status=400)):
|
||||
self.assertTrue(activity_log._failed(bad))
|
||||
|
||||
def test_middleware_records_successful_writes_only(self):
|
||||
async def run(path, status, payload):
|
||||
request = make_mocked_request('POST', path, headers={'Content-Type': 'application/json'})
|
||||
request._read_bytes = json.dumps(payload).encode()
|
||||
|
||||
async def handler(_request):
|
||||
await _request.json()
|
||||
return web.json_response({'success': status < 300}, status=status)
|
||||
await activity_log.activity_middleware(request, handler)
|
||||
|
||||
asyncio.run(run('/anomalous/delete_recipe', 200, {'filename': 'folder/My Recipe.json'}))
|
||||
asyncio.run(run('/anomalous/delete_recipe', 500, {'filename': 'broken.json'}))
|
||||
asyncio.run(run('/anomalous/folders', 200, {'name': 'not a write route'}))
|
||||
entries = activity_log.list_entries()
|
||||
self.assertEqual([(e['action'], e['target']) for e in entries], [('recipe_delete', 'My Recipe.json')])
|
||||
|
||||
def test_canvas_post_keeps_known_change_kinds(self):
|
||||
async def post(body):
|
||||
request = make_mocked_request('POST', '/anomalous/activity', headers={'Content-Type': 'application/json'})
|
||||
request._read_bytes = json.dumps(body).encode()
|
||||
return await activity_log.api_post_activity(request)
|
||||
|
||||
bad = asyncio.run(post({'changes': [{'kind': 'script', 'node': 'x'}]}))
|
||||
self.assertEqual(bad.status, 400)
|
||||
good = asyncio.run(post({'changes': [{'kind': 'changed', 'node': 'KSampler #3', 'widget': 'steps',
|
||||
'before': '20', 'after': '30'}], 'total': 4, 'workflow': 'a.json'}))
|
||||
self.assertEqual(good.status, 200)
|
||||
entry = activity_log.list_entries()[0]
|
||||
self.assertEqual((entry['source'], entry['target'], entry['detail']['total']), ('canvas', 'a.json', 4))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,164 @@
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import audio_catalog
|
||||
|
||||
|
||||
class _Request:
|
||||
def __init__(self, form=None, body=None, query=None):
|
||||
self._form = form or {}
|
||||
self._body = body
|
||||
self.query = query or {}
|
||||
|
||||
async def post(self):
|
||||
return self._form
|
||||
|
||||
async def json(self):
|
||||
return self._body
|
||||
|
||||
|
||||
def _run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
def _body(response):
|
||||
return json.loads(response.body.decode('utf-8'))
|
||||
|
||||
|
||||
def _vorbis_block(tags):
|
||||
vendor = b'test'
|
||||
body = len(vendor).to_bytes(4, 'little') + vendor + len(tags).to_bytes(4, 'little')
|
||||
for key, value in tags.items():
|
||||
entry = f'{key}={value}'.encode('utf-8')
|
||||
body += len(entry).to_bytes(4, 'little') + entry
|
||||
return body
|
||||
|
||||
|
||||
def _write_flac(path, tags):
|
||||
streaminfo = b'\x00' + (34).to_bytes(3, 'big') + b'\x00' * 34
|
||||
comment = _vorbis_block(tags)
|
||||
block = bytes([0x80 | 4]) + len(comment).to_bytes(3, 'big') + comment
|
||||
Path(path).write_bytes(b'fLaC' + streaminfo + block + b'\xff\xf8audio')
|
||||
|
||||
|
||||
def _ogg_page(packet):
|
||||
lacing = [255] * (len(packet) // 255) + [len(packet) % 255]
|
||||
return b'OggS' + b'\x00' * 22 + bytes([len(lacing)]) + bytes(lacing) + packet
|
||||
|
||||
|
||||
def _write_opus(path, tags):
|
||||
Path(path).write_bytes(_ogg_page(b'OpusHead' + b'\x01' * 11) + _ogg_page(b'OpusTags' + _vorbis_block(tags)))
|
||||
|
||||
|
||||
PROMPT = json.dumps({
|
||||
'14': {'class_type': 'PreviewAudio', 'inputs': {'audio': ['16', 0]}},
|
||||
'16': {'class_type': 'F5TTSAudio', 'inputs': {'sample': 'F5-TTS\\Arona.wav', 'speech': '{happy} 早上好！\n{main} 开始吧。', 'seed': 7}},
|
||||
}, ensure_ascii=False)
|
||||
|
||||
|
||||
class AudioGalleryTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory()
|
||||
root = Path(self._tmp.name)
|
||||
self.output_dir = root / 'output'
|
||||
self.temp_dir = root / 'temp'
|
||||
self.output_dir.mkdir()
|
||||
self.temp_dir.mkdir()
|
||||
for name, target in (('get_output_directory', self.output_dir), ('get_temp_directory', self.temp_dir), ('get_input_directory', root / 'input')):
|
||||
patcher = mock.patch.object(audio_catalog.folder_paths, name, return_value=str(target))
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
audio_catalog._info_cache.clear()
|
||||
|
||||
def tearDown(self):
|
||||
self._tmp.cleanup()
|
||||
|
||||
def test_generation_info_is_read_from_flac_and_opus_comments(self):
|
||||
(self.output_dir / 'audio').mkdir()
|
||||
_write_flac(self.output_dir / 'audio' / 'a.flac', {'prompt': PROMPT, 'workflow': '{}'})
|
||||
_write_opus(self.output_dir / 'b.opus', {'PROMPT': PROMPT})
|
||||
(self.output_dir / 'c.mp3').write_bytes(b'ID3')
|
||||
body = _body(_run(audio_catalog.api_get_audio_gallery(_Request(query={}))))
|
||||
by_name = {item['filename']: item for item in body['audios']}
|
||||
self.assertEqual(set(by_name), {'a.flac', 'b.opus', 'c.mp3'})
|
||||
for name in ('a.flac', 'b.opus'):
|
||||
info = by_name[name]['generation']
|
||||
self.assertEqual(info['speech'], '{happy} 早上好！\n{main} 开始吧。')
|
||||
self.assertEqual(info['sample'], 'F5-TTS/Arona.wav')
|
||||
self.assertEqual(info['seed'], 7)
|
||||
self.assertIsNone(by_name['c.mp3']['generation'])
|
||||
|
||||
def test_gallery_search_matches_speech_text_and_file_names(self):
|
||||
_write_flac(self.output_dir / 'first.flac', {'prompt': PROMPT})
|
||||
(self.output_dir / 'second.wav').write_bytes(b'RIFF')
|
||||
hits = _body(_run(audio_catalog.api_get_audio_gallery(_Request(query={'q': '开始吧'}))))
|
||||
self.assertEqual([item['filename'] for item in hits['audios']], ['first.flac'])
|
||||
self.assertEqual(hits['total'], 1)
|
||||
by_name = _body(_run(audio_catalog.api_get_audio_gallery(_Request(query={'q': 'SECOND'}))))
|
||||
self.assertEqual([item['filename'] for item in by_name['audios']], ['second.wav'])
|
||||
|
||||
def test_previews_list_temp_audio_and_save_copies_without_overwriting(self):
|
||||
_write_flac(self.temp_dir / 'ComfyUI_temp_abcde_00001.flac', {'prompt': PROMPT})
|
||||
(self.temp_dir / 'note.txt').write_text('x')
|
||||
previews = _body(_run(audio_catalog.api_get_audio_previews(_Request())))['audios']
|
||||
self.assertEqual([p['filename'] for p in previews], ['ComfyUI_temp_abcde_00001.flac'])
|
||||
self.assertIn('type=temp', previews[0]['audio_url'])
|
||||
self.assertEqual(previews[0]['generation']['seed'], 7)
|
||||
|
||||
request = {'filename': 'ComfyUI_temp_abcde_00001.flac', 'subfolder': ''}
|
||||
first = _body(_run(audio_catalog.api_save_audio_preview(_Request(body=request))))
|
||||
second = _body(_run(audio_catalog.api_save_audio_preview(_Request(body=request))))
|
||||
self.assertNotEqual(first['audio']['filename'], second['audio']['filename'])
|
||||
saved = sorted(p.name for p in (self.output_dir / 'audio').iterdir())
|
||||
self.assertEqual(len(saved), 2)
|
||||
self.assertEqual(first['audio']['generation']['speech'], '{happy} 早上好！\n{main} 开始吧。')
|
||||
self.assertTrue((self.temp_dir / 'ComfyUI_temp_abcde_00001.flac').exists(), 'the preview itself is untouched')
|
||||
|
||||
def test_preview_save_rejects_escape_and_non_audio(self):
|
||||
(self.temp_dir / 'note.txt').write_text('x')
|
||||
escape = _run(audio_catalog.api_save_audio_preview(_Request(body={'filename': 'x.flac', 'subfolder': '../output'})))
|
||||
self.assertEqual(escape.status, 403)
|
||||
text = _run(audio_catalog.api_save_audio_preview(_Request(body={'filename': 'note.txt', 'subfolder': ''})))
|
||||
self.assertEqual(text.status, 403)
|
||||
missing = _run(audio_catalog.api_save_audio_preview(_Request(body={'filename': 'gone.flac', 'subfolder': ''})))
|
||||
self.assertEqual(missing.status, 404)
|
||||
self.assertFalse((self.output_dir / 'audio').exists() and any((self.output_dir / 'audio').iterdir()))
|
||||
|
||||
def test_gallery_delete_only_removes_audio_inside_output(self):
|
||||
(self.output_dir / 'image.png').write_bytes(b'png')
|
||||
(self.output_dir / 'take.wav').write_bytes(b'wav')
|
||||
refused = _run(audio_catalog.api_delete_audio_gallery(_Request(body={'filename': 'image.png', 'subfolder': ''})))
|
||||
self.assertEqual(refused.status, 415)
|
||||
self.assertTrue((self.output_dir / 'image.png').exists())
|
||||
escape = _run(audio_catalog.api_delete_audio_gallery(_Request(body={'filename': 'x.wav', 'subfolder': '../input'})))
|
||||
self.assertEqual(escape.status, 403)
|
||||
sent = []
|
||||
with mock.patch.object(audio_catalog, 'move_to_trash', side_effect=lambda *paths: sent.extend(paths)):
|
||||
ok = _run(audio_catalog.api_delete_audio_gallery(_Request(body={'filename': 'take.wav', 'subfolder': ''})))
|
||||
self.assertEqual(ok.status, 200)
|
||||
self.assertEqual([os.path.basename(path) for path in sent], ['take.wav']) # to the Recycle Bin
|
||||
|
||||
def test_gallery_rejects_bad_paging_values(self):
|
||||
response = _run(audio_catalog.api_get_audio_gallery(_Request(query={'page': 'x', 'limit': '-3'})))
|
||||
body = _body(response)
|
||||
self.assertEqual((body['page'], body['total']), (1, 0))
|
||||
|
||||
def test_stream_accepts_temp_and_rejects_unknown_roots(self):
|
||||
_write_flac(self.temp_dir / 'p.flac', {})
|
||||
ok = _run(audio_catalog.api_serve_audio(_Request(query={'path': 'p.flac', 'type': 'temp'})))
|
||||
self.assertEqual(ok.status, 200)
|
||||
for root in ('models', 'input'):
|
||||
bad = _run(audio_catalog.api_serve_audio(_Request(query={'path': 'p.flac', 'type': root})))
|
||||
self.assertEqual(bad.status, 400, root)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Every name a backend module reads must be bound somewhere in that module.
|
||||
|
||||
Splitting a module can leave a function calling a helper, or a standard library module,
|
||||
that was only imported in the old file. Python reports that only when the line runs, so a
|
||||
save or thumbnail route fails for users while imports and other tests still pass. This
|
||||
check is deliberately scope-blind: it catches missing imports, not every local-name slip.
|
||||
"""
|
||||
|
||||
import ast
|
||||
import builtins
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
PLUGIN_DIR = Path(__file__).resolve().parents[1]
|
||||
ALWAYS_BOUND = set(dir(builtins)) | {"__file__", "__name__", "__doc__", "__package__", "__spec__", "__path__"}
|
||||
|
||||
|
||||
def _backend_files():
|
||||
yield from sorted(PLUGIN_DIR.glob("*.py"))
|
||||
yield from sorted((PLUGIN_DIR / "api").glob("*.py"))
|
||||
|
||||
|
||||
def _arguments(args):
|
||||
names = [a.arg for a in args.posonlyargs + args.args + args.kwonlyargs]
|
||||
names += [a.arg for a in (args.vararg, args.kwarg) if a]
|
||||
return names
|
||||
|
||||
|
||||
def _bound_names(tree, path):
|
||||
bound = set()
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
bound.add(node.name)
|
||||
bound.update(_arguments(node.args))
|
||||
elif isinstance(node, ast.Lambda):
|
||||
bound.update(_arguments(node.args))
|
||||
elif isinstance(node, ast.ClassDef):
|
||||
bound.add(node.name)
|
||||
elif isinstance(node, (ast.Import, ast.ImportFrom)):
|
||||
for alias in node.names:
|
||||
if alias.name == "*":
|
||||
bound |= _star_names(node, path)
|
||||
else:
|
||||
bound.add((alias.asname or alias.name).split(".")[0])
|
||||
elif isinstance(node, ast.Name) and isinstance(node.ctx, (ast.Store, ast.Del)):
|
||||
bound.add(node.id)
|
||||
elif isinstance(node, ast.ExceptHandler) and node.name:
|
||||
bound.add(node.name)
|
||||
elif isinstance(node, (ast.Global, ast.Nonlocal)):
|
||||
bound.update(node.names)
|
||||
elif isinstance(node, (ast.MatchAs, ast.MatchStar)) and node.name:
|
||||
bound.add(node.name)
|
||||
return bound
|
||||
|
||||
|
||||
def _star_names(node, path):
|
||||
"""Top-level names of a sibling module imported with ``from .x import *``."""
|
||||
if node.level != 1 or not node.module:
|
||||
raise AssertionError(f"{path.name}: only sibling star imports are checked")
|
||||
target = path.parent / f"{node.module}.py"
|
||||
tree = ast.parse(target.read_text(encoding="utf-8"))
|
||||
return {name for name in _bound_names(tree, target) if not name.startswith("_")}
|
||||
|
||||
|
||||
class BackendNamesTests(unittest.TestCase):
|
||||
def test_every_read_name_is_bound(self):
|
||||
problems = []
|
||||
for path in _backend_files():
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"))
|
||||
bound = _bound_names(tree, path) | ALWAYS_BOUND
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Name) and isinstance(node.ctx, ast.Load) and node.id not in bound:
|
||||
problems.append(f"{path.relative_to(PLUGIN_DIR)}:{node.lineno} {node.id}")
|
||||
self.assertEqual(problems, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,245 @@
|
||||
"""Backups (Settings -> Backup): exporting the user's data and putting it back on another
|
||||
computer. Two fake computers in temporary folders; nothing outside them is touched."""
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import zipfile
|
||||
from contextlib import ExitStack
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
from api import backup # noqa: E402
|
||||
|
||||
PARTS = ("recipes", "combos", "materials", "parameters", "comfy_workflows")
|
||||
|
||||
|
||||
class Computer:
|
||||
"""A user folder, a models folder and settings of one computer; `hashes` stands for the scan."""
|
||||
|
||||
def __init__(self, root):
|
||||
self.root = root
|
||||
self.stores = {part: os.path.join(root, "user", part) for part in PARTS}
|
||||
self.models = os.path.join(root, "models", "loras")
|
||||
self.config = os.path.join(root, "config.json")
|
||||
self.download_settings = os.path.join(root, "download_settings.json")
|
||||
self.temp = os.path.join(root, "temp")
|
||||
self.trash = os.path.join(root, "trash")
|
||||
for path in [*self.stores.values(), self.models, self.temp, self.trash]:
|
||||
os.makedirs(path, exist_ok=True)
|
||||
self.hashes = {}
|
||||
|
||||
def write(self, path, data):
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "wb" if isinstance(data, bytes) else "w", **({} if isinstance(data, bytes) else {"encoding": "utf-8"})) as handle:
|
||||
handle.write(data)
|
||||
return path
|
||||
|
||||
def model(self, rel, data=b"weights", sha=""):
|
||||
path = self.write(os.path.join(self.models, rel), data)
|
||||
if sha:
|
||||
self.hashes[os.path.realpath(path)] = sha
|
||||
return os.path.splitext(path)[0]
|
||||
|
||||
def trashed(self):
|
||||
return sorted(os.listdir(self.trash))
|
||||
|
||||
def patches(self):
|
||||
def to_trash(path):
|
||||
shutil.move(path, os.path.join(self.trash, os.path.basename(path)))
|
||||
|
||||
stack = ExitStack()
|
||||
stack.enter_context(mock.patch.object(backup, "part_dirs", return_value=self.stores))
|
||||
stack.enter_context(mock.patch.object(backup, "work_dir", return_value=self.temp))
|
||||
stack.enter_context(mock.patch.object(backup, "config_path", return_value=self.config))
|
||||
stack.enter_context(mock.patch.object(backup, "download_settings_path", return_value=self.download_settings))
|
||||
stack.enter_context(mock.patch.object(backup.folder_paths, "get_folder_paths", side_effect=lambda t: [self.models] if t == "loras" else []))
|
||||
stack.enter_context(mock.patch.object(backup, "get_metadata", side_effect=lambda p: {"hash": self.hashes.get(os.path.realpath(p), "")}))
|
||||
stack.enter_context(mock.patch.object(backup, "move_to_trash", side_effect=to_trash))
|
||||
stack.enter_context(mock.patch.object(backup, "add_entry"))
|
||||
stack.enter_context(mock.patch.object(backup, "_version", return_value="v-test"))
|
||||
return stack
|
||||
|
||||
|
||||
class BackupTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.a = Computer(os.path.join(self.tmp.name, "a"))
|
||||
self.b = Computer(os.path.join(self.tmp.name, "b"))
|
||||
a = self.a
|
||||
a.write(os.path.join(a.stores["recipes"], "recipe_1.json"), '{"name": "portrait"}')
|
||||
a.write(os.path.join(a.stores["materials"], ".assets", "m1", "preview.webp"), b"img")
|
||||
a.write(os.path.join(a.stores["combos"], "Flux.json"), '{"name": "Flux"}')
|
||||
a.write(os.path.join(a.stores["combos"], ".legacy_imported.json"), "{}")
|
||||
a.write(os.path.join(a.stores["comfy_workflows"], "mine.json"), "{}")
|
||||
a.write(a.config, json.dumps({"CIVITAI_API_KEY": "secret-a", "folder_view_mode": "abstract"}))
|
||||
base = a.model("Style/cute.safetensors", sha="ab" * 32)
|
||||
a.write(base + ".anomalous.json", json.dumps({"format": 1, "custom_name": "Cute", "custom_notes": "trigger: cute"}))
|
||||
a.write(base + ".preview.png", b"my own cover")
|
||||
civitai = a.model("plain.safetensors", sha="cd" * 32)
|
||||
a.write(civitai + ".preview.png", b"civitai image")
|
||||
a.write(civitai + ".civitai_bak.png", b"civitai image")
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def export(self, **options):
|
||||
with self.a.patches():
|
||||
path, manifest = backup.build_backup(**options)
|
||||
return path, manifest
|
||||
|
||||
def put_back(self, path, **options):
|
||||
with self.b.patches():
|
||||
summary = backup.inspect_backup(path)
|
||||
result = backup.apply_backup(path, **{"parts": PARTS, **options})
|
||||
return summary, result
|
||||
|
||||
def test_export_keeps_stores_and_the_users_own_model_data(self):
|
||||
path, manifest = self.export()
|
||||
names = zipfile.ZipFile(path).namelist()
|
||||
self.assertIn("stores/recipes/recipe_1.json", names)
|
||||
self.assertIn("stores/materials/.assets/m1/preview.webp", names)
|
||||
self.assertNotIn("stores/combos/.legacy_imported.json", names) # this computer's own marker
|
||||
self.assertEqual(manifest["parts"]["comfy_workflows"], 1)
|
||||
# Only the model with the user's name, notes and cover; a Civitai cover is no user data.
|
||||
self.assertEqual([m["rel"] for m in manifest["models"]], ["Style/cute.safetensors"])
|
||||
self.assertEqual(manifest["models"][0]["cover"], ".preview.png")
|
||||
settings = json.loads(zipfile.ZipFile(path).read("settings/config.json"))
|
||||
self.assertNotIn("CIVITAI_API_KEY", settings) # the key never leaves the computer
|
||||
|
||||
def test_scan_results_and_civitai_covers_come_back_where_missing(self):
|
||||
civitai = os.path.splitext(os.path.join(self.a.models, "plain.safetensors"))[0]
|
||||
self.a.write(civitai + ".info", '{"id": 7, "name": "Plain"}')
|
||||
path, manifest = self.export(include_civitai_covers=True)
|
||||
plain = [m for m in manifest["models"] if m["rel"] == "plain.safetensors"][0]
|
||||
self.assertEqual((plain["info"], plain["civitai_cover"], plain["civitai_bak"]), (".info", ".preview.png", ".civitai_bak.png"))
|
||||
base = self.b.model("plain.safetensors", sha="cd" * 32)
|
||||
_summary, result = self.put_back(path)
|
||||
self.assertEqual((result["model_infos"], result["model_civitai_covers"]), (1, 1))
|
||||
self.assertEqual(Path(base + ".info").read_text(encoding="utf-8"), '{"id": 7, "name": "Plain"}')
|
||||
self.assertEqual(Path(base + ".preview.png").read_bytes(), b"civitai image")
|
||||
self.assertEqual(Path(base + ".civitai_bak.png").read_bytes(), b"civitai image") # still Civitai's, not the user's
|
||||
self.b.write(base + ".info", '{"id": 7, "name": "Newer scan"}')
|
||||
_summary, kept = self.put_back(path)
|
||||
self.assertEqual(kept["model_infos"], 0)
|
||||
self.assertIn("Newer scan", Path(base + ".info").read_text(encoding="utf-8"))
|
||||
_path, small = self.export() # Civitai's covers only when asked
|
||||
self.assertNotIn("civitai_cover", [m for m in small["models"] if m["rel"] == "plain.safetensors"][0])
|
||||
|
||||
def test_the_model_list_names_what_a_new_computer_lacks(self):
|
||||
path, manifest = self.export()
|
||||
self.assertEqual(sorted(m["rel"] for m in manifest["library"]), ["Style/cute.safetensors", "plain.safetensors"])
|
||||
self.a.model("copy/cute_again.safetensors", sha="ab" * 32) # a second copy: one download
|
||||
path, manifest = self.export()
|
||||
self.b.model("somewhere/plain_renamed.safetensors", sha="cd" * 32) # this one is here under another name
|
||||
with self.b.patches():
|
||||
summary = backup.inspect_backup(path)
|
||||
self.assertEqual(summary["library"]["total"], 3)
|
||||
self.assertEqual([(m["rel"], m["sha256"]) for m in summary["library"]["missing"]], [("Style/cute.safetensors", "ab" * 32)])
|
||||
_path, without = self.export(include_library=False)
|
||||
self.assertNotIn("library", without)
|
||||
|
||||
def test_comfy_workflows_can_be_left_out(self):
|
||||
_path, manifest = self.export(include_comfy=False)
|
||||
self.assertNotIn("comfy_workflows", manifest["parts"])
|
||||
|
||||
def test_a_new_computer_gets_everything_and_models_are_found_by_hash(self):
|
||||
path, _ = self.export()
|
||||
b = self.b
|
||||
base = b.model("Moved/elsewhere/cute_renamed.safetensors", sha="ab" * 32) # moved and renamed
|
||||
b.write(b.config, json.dumps({"CIVITAI_API_KEY": "secret-b"}))
|
||||
summary, result = self.put_back(path, settings=True)
|
||||
self.assertEqual(summary["parts"]["recipes"], {"new": 1, "same": 0, "differs": 0})
|
||||
self.assertEqual(summary["models"]["matched"], 1)
|
||||
self.assertEqual(result["added"], 4)
|
||||
self.assertTrue(os.path.isfile(os.path.join(b.stores["materials"], ".assets", "m1", "preview.webp")))
|
||||
user = json.loads(Path(base + ".anomalous.json").read_text(encoding="utf-8"))
|
||||
self.assertEqual(user["custom_name"], "Cute")
|
||||
self.assertEqual(Path(base + ".preview.png").read_bytes(), b"my own cover")
|
||||
config = json.loads(Path(b.config).read_text(encoding="utf-8"))
|
||||
self.assertEqual(config["CIVITAI_API_KEY"], "secret-b") # this computer's key stays
|
||||
self.assertEqual(config["folder_view_mode"], "abstract")
|
||||
|
||||
def test_differing_files_stay_unless_replace_and_then_go_to_the_recycle_bin(self):
|
||||
path, _ = self.export()
|
||||
b = self.b
|
||||
mine = b.write(os.path.join(b.stores["recipes"], "recipe_1.json"), '{"name": "changed here"}')
|
||||
_summary, kept = self.put_back(path)
|
||||
self.assertEqual(Path(mine).read_text(encoding="utf-8"), '{"name": "changed here"}')
|
||||
self.assertEqual(kept["skipped"], 1)
|
||||
_summary, replaced = self.put_back(path, replace=True)
|
||||
self.assertEqual(replaced["replaced"], 1)
|
||||
self.assertEqual(Path(mine).read_text(encoding="utf-8"), '{"name": "portrait"}')
|
||||
self.assertIn("recipe_1.json", b.trashed())
|
||||
_summary, again = self.put_back(path, replace=True)
|
||||
self.assertEqual((again["added"], again["replaced"]), (0, 0)) # all the same now
|
||||
|
||||
def test_user_fields_are_filled_where_empty_and_replaced_only_on_request(self):
|
||||
path, _ = self.export()
|
||||
base = self.b.model("cute.safetensors", sha="ab" * 32)
|
||||
self.b.write(base + ".anomalous.json", json.dumps({"format": 1, "custom_name": "Mine"}))
|
||||
self.put_back(path)
|
||||
user = json.loads(Path(base + ".anomalous.json").read_text(encoding="utf-8"))
|
||||
self.assertEqual((user["custom_name"], user["custom_notes"]), ("Mine", "trigger: cute"))
|
||||
self.put_back(path, replace=True)
|
||||
user = json.loads(Path(base + ".anomalous.json").read_text(encoding="utf-8"))
|
||||
self.assertEqual(user["custom_name"], "Cute")
|
||||
self.assertIn("cute.anomalous.json", self.b.trashed())
|
||||
|
||||
def test_the_users_cover_replaces_civitais_but_not_another_user_cover(self):
|
||||
path, _ = self.export()
|
||||
b = self.b
|
||||
base = b.model("cute.safetensors", sha="ab" * 32)
|
||||
b.write(base + ".png", b"civitai here")
|
||||
b.write(base + ".civitai_bak.png", b"civitai here")
|
||||
self.put_back(path)
|
||||
self.assertEqual(Path(base + ".preview.png").read_bytes(), b"my own cover")
|
||||
self.assertFalse(os.path.exists(base + ".png"))
|
||||
self.assertTrue(os.path.exists(base + ".civitai_bak.png")) # Civitai's copy stays
|
||||
b.write(base + ".preview.png", b"another of mine")
|
||||
_summary, result = self.put_back(path)
|
||||
self.assertEqual(result["covers_kept"], 1)
|
||||
self.assertEqual(Path(base + ".preview.png").read_bytes(), b"another of mine")
|
||||
|
||||
def test_without_a_hash_a_model_needs_the_same_path_and_size(self):
|
||||
path, _ = self.export()
|
||||
with zipfile.ZipFile(path) as archive:
|
||||
manifest = json.loads(archive.read("manifest.json"))
|
||||
manifest["models"][0]["sha256"] = ""
|
||||
rewritten = os.path.join(self.b.temp, "nohash.zip")
|
||||
with zipfile.ZipFile(path) as source, zipfile.ZipFile(rewritten, "w") as target:
|
||||
for info in source.infolist():
|
||||
data = json.dumps(manifest) if info.filename == "manifest.json" else source.read(info)
|
||||
target.writestr(info.filename, data)
|
||||
self.b.model("Style/cute.safetensors", data=b"other size!")
|
||||
summary, result = self.put_back(rewritten)
|
||||
self.assertEqual((summary["models"]["matched"], result["models_missing"]), (0, 1))
|
||||
self.b.model("Style/cute.safetensors", data=b"weights")
|
||||
summary, _ = self.put_back(rewritten)
|
||||
self.assertEqual(summary["models"]["matched"], 1)
|
||||
|
||||
def test_unsafe_or_foreign_files_are_refused(self):
|
||||
evil = os.path.join(self.b.temp, "evil.zip")
|
||||
with zipfile.ZipFile(evil, "w") as archive:
|
||||
archive.writestr("manifest.json", json.dumps({"app": backup.APP, "format": backup.FORMAT}))
|
||||
archive.writestr("stores/recipes/../../escape.json", "{}")
|
||||
with self.b.patches(), self.assertRaises(backup.BackupError):
|
||||
backup.inspect_backup(evil)
|
||||
foreign = os.path.join(self.b.temp, "foreign.zip")
|
||||
with zipfile.ZipFile(foreign, "w") as archive:
|
||||
archive.writestr("readme.txt", "hello")
|
||||
with self.b.patches(), self.assertRaises(backup.BackupError):
|
||||
backup.inspect_backup(foreign)
|
||||
text = os.path.join(self.b.temp, "text.zip")
|
||||
Path(text).write_text("not a zip", encoding="utf-8")
|
||||
with self.b.patches(), self.assertRaises(backup.BackupError):
|
||||
backup.inspect_backup(text)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Card images: kept in the user folder, posters for video covers, background warm-up."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
from PIL import Image # noqa: E402
|
||||
|
||||
from api import media_routes # noqa: E402
|
||||
|
||||
|
||||
def write_video(path, frames=3):
|
||||
import av
|
||||
with av.open(path, mode='w') as container:
|
||||
stream = container.add_stream('mpeg4', rate=10)
|
||||
stream.width, stream.height, stream.pix_fmt = 640, 360, 'yuv420p'
|
||||
for index in range(frames):
|
||||
image = Image.new('RGB', (640, 360), (200, 40 * index, 30))
|
||||
frame = av.VideoFrame.from_image(image)
|
||||
for packet in stream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
|
||||
|
||||
class CardImageTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.root = self.tmp.name
|
||||
self.user = os.path.join(self.root, 'user')
|
||||
patcher = mock.patch.object(media_routes.folder_paths, 'get_user_directory', return_value=self.user)
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
|
||||
def cache_files(self):
|
||||
directory = os.path.join(self.user, 'anomalous', 'cache', 'card_thumbnails')
|
||||
return sorted(os.listdir(directory)) if os.path.isdir(directory) else []
|
||||
|
||||
def test_thumbnail_is_kept_in_the_user_folder(self):
|
||||
cover = os.path.join(self.root, 'model.preview.png')
|
||||
Image.new('RGB', (1200, 1600), (10, 20, 30)).save(cover)
|
||||
served = media_routes._build_card_thumbnail(cover)
|
||||
self.assertTrue(served.startswith(os.path.join(self.user, 'anomalous', 'cache')))
|
||||
with Image.open(served) as image:
|
||||
self.assertEqual(max(image.size), media_routes.CARD_THUMBNAIL_EDGE)
|
||||
self.assertEqual(media_routes._build_card_thumbnail(cover), served)
|
||||
self.assertEqual(media_routes.thumbnail_cache_usage()[0], 1)
|
||||
self.assertEqual(media_routes.clear_thumbnail_cache(), 1)
|
||||
self.assertTrue(os.path.isfile(cover)) # the cover itself is never touched
|
||||
|
||||
def test_video_cover_gets_a_poster(self):
|
||||
video = os.path.join(self.root, 'model.mp4')
|
||||
write_video(video)
|
||||
poster = media_routes._build_video_poster(video)
|
||||
self.assertIsNotNone(poster)
|
||||
with Image.open(poster) as image:
|
||||
self.assertEqual(image.size, (512, 288))
|
||||
broken = os.path.join(self.root, 'broken.mp4')
|
||||
with open(broken, 'wb') as handle:
|
||||
handle.write(b'not a video')
|
||||
self.assertIsNone(media_routes._build_video_poster(broken))
|
||||
|
||||
def test_warm_up_makes_the_images_in_the_background(self):
|
||||
cover = os.path.join(self.root, 'a.png')
|
||||
Image.new('RGB', (900, 900)).save(cover)
|
||||
video = os.path.join(self.root, 'b.mp4')
|
||||
write_video(video)
|
||||
media_routes.queue_card_images([cover, video, os.path.join(self.root, 'c.txt'), cover])
|
||||
deadline = time.time() + 20
|
||||
while media_routes._warm_thread is not None and time.time() < deadline:
|
||||
time.sleep(0.05)
|
||||
self.assertEqual(len(self.cache_files()), 2)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,87 @@
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import metadata, models, recipes
|
||||
from model_identity import computed_file_identity, safetensors_damage
|
||||
|
||||
|
||||
def safetensors_bytes(tensors, payload=b'\0' * 16, header_size=None):
|
||||
header = json.dumps({'__metadata__': {'x': 'y'}, **tensors}).encode()
|
||||
return struct.pack('<Q', len(header) if header_size is None else header_size) + header + payload
|
||||
|
||||
|
||||
class FileIdentityTests(unittest.TestCase):
|
||||
def test_damaged_headers_are_told_apart_from_whole_files(self):
|
||||
tensor = lambda end: {'w': {'dtype': 'F32', 'shape': [4], 'data_offsets': [0, end]}}
|
||||
cases = {
|
||||
'whole': (safetensors_bytes(tensor(16)), ''),
|
||||
'short': (b'abc', 'short'),
|
||||
'header longer than the file': (safetensors_bytes(tensor(16), header_size=10 ** 6), 'header_size'),
|
||||
'not json': (struct.pack('<Q', 4) + b'{{{{' + b'\0' * 8, 'header'),
|
||||
'offsets not numbers': (safetensors_bytes({'w': {'data_offsets': ['a', 'b']}}), 'header'),
|
||||
'cut short': (safetensors_bytes(tensor(64)), 'truncated'),
|
||||
}
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
for name, (content, expected) in cases.items():
|
||||
with self.subTest(name):
|
||||
path = Path(directory, 'm.safetensors')
|
||||
path.write_bytes(content)
|
||||
self.assertEqual(safetensors_damage(str(path)), expected)
|
||||
self.assertEqual(safetensors_damage(str(Path(directory, 'missing.safetensors'))), '')
|
||||
|
||||
def test_header_digests_are_not_file_identity(self):
|
||||
for key in ('modelspec.hash.blake3', 'modelspec.hash.sha256', 'modelspec.hash_sha256'):
|
||||
with self.subTest(key=key), tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory, 'fixture.safetensors')
|
||||
header = json.dumps({'__metadata__': {key: 'a' * 64}}).encode()
|
||||
path.write_bytes(struct.pack('<Q', len(header)) + header + b'payload')
|
||||
self.assertFalse(metadata.get_metadata(str(path))['hash'])
|
||||
with mock.patch.object(recipes, '_resolve_exact_model_reference', return_value={'path': str(path)}):
|
||||
identity, _ = recipes._identity_for_reference(path.name)
|
||||
self.assertEqual(identity['status'], 'unverified')
|
||||
result = models._resolve_from_candidates([{'type': 'vae', 'filename': path.name, 'path': str(path), 'size': path.stat().st_size}], 'a' * 64, path.stat().st_size, require_hash=True)
|
||||
self.assertFalse(result['found'])
|
||||
|
||||
def test_legacy_offline_hash_requires_reverification_and_keeps_notes(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory, 'fixture.safetensors')
|
||||
path.write_bytes(b'payload')
|
||||
info = path.with_suffix('.info')
|
||||
info.write_text(json.dumps({'id': -1, 'anomalous_custom_notes': 'Keep me', 'files': [{'hashes': {'SHA256': 'a' * 64}}]}))
|
||||
self.assertFalse(metadata.get_metadata(str(path))['hash'])
|
||||
digest = hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
models._compute_and_save_fallback_info(str(path), digest)
|
||||
self.assertEqual(metadata.get_metadata(str(path))['hash'], digest)
|
||||
self.assertEqual(json.loads(info.read_text())['anomalous_custom_notes'], 'Keep me')
|
||||
|
||||
def test_computed_record_invalidated_after_file_changes(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory, 'fixture.safetensors')
|
||||
path.write_bytes(b'payload')
|
||||
digest = hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
path.with_suffix('.info').write_text(json.dumps({'anomalous_file_identity': computed_file_identity(path, digest)}))
|
||||
self.assertEqual(metadata.get_metadata(str(path))['hash'], digest)
|
||||
path.write_bytes(b'changed payload')
|
||||
self.assertFalse(metadata.get_metadata(str(path))['hash'])
|
||||
|
||||
def test_offline_scan_persists_full_file_sha256(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory, 'fixture.safetensors')
|
||||
header = json.dumps({'__metadata__': {'modelspec.hash.blake3': 'a' * 64}}).encode()
|
||||
path.write_bytes(struct.pack('<Q', len(header)) + header + b'payload')
|
||||
result = subprocess.run([sys.executable, str(PLUGIN / 'scraper.py'), directory, '--offline-only', '--skip-media', '--skip-rename'], capture_output=True)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(metadata.get_metadata(str(path))['hash'], hashlib.sha256(path.read_bytes()).hexdigest())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,111 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
from PIL import Image, PngImagePlugin
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import image_search, model_resolution
|
||||
|
||||
|
||||
def _png(path, **texts):
|
||||
info = PngImagePlugin.PngInfo()
|
||||
for key, value in texts.items():
|
||||
info.add_text(key, value)
|
||||
Image.new('RGB', (4, 4)).save(path, pnginfo=info)
|
||||
|
||||
|
||||
PROMPT = json.dumps({
|
||||
'3': {'class_type': 'KSampler', 'inputs': {'seed': 123456789, 'steps': 25, 'model': ['4', 0]}},
|
||||
'4': {'class_type': 'CheckpointLoaderSimple', 'inputs': {'ckpt_name': '2D_Anime\\MiaoMiao RealSkin.safetensors'}},
|
||||
'6': {'class_type': 'CLIPTextEncode', 'inputs': {'text': '1girl, 赛博朋克 city, neon'}},
|
||||
})
|
||||
WORKFLOW = json.dumps({'extra': {'anomalous_hashes': {
|
||||
'4_2D_Anime\\MiaoMiao RealSkin.safetensors': {'hash': '667B5E3BEDE5A2FD53BB6F0EA5B93A89453D6855FA2E5D95ACDE93CF543C4F7C', 'size': 1},
|
||||
}}})
|
||||
|
||||
|
||||
class ImageSearchTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self._tmp.name)
|
||||
(self.root / 'sub').mkdir()
|
||||
_png(self.root / 'a.png', prompt=PROMPT, workflow=WORKFLOW)
|
||||
_png(self.root / 'sub' / 'b.png', prompt=json.dumps({'1': {'class_type': 'CheckpointLoaderSimple', 'inputs': {'ckpt_name': 'renamed_copy.safetensors'}}}))
|
||||
_png(self.root / 'c.png', parameters='a cat in rain\nSteps: 20, Seed: 42, Model hash: abcdef1234, Model: dreamshaper')
|
||||
Image.new('RGB', (4, 4)).save(self.root / 'plain.jpg')
|
||||
self.images = [
|
||||
{'filename': 'a.png', 'subfolder': ''},
|
||||
{'filename': 'b.png', 'subfolder': 'sub'},
|
||||
{'filename': 'c.png', 'subfolder': ''},
|
||||
{'filename': 'plain.jpg', 'subfolder': ''},
|
||||
]
|
||||
image_search._record_cache.clear()
|
||||
image_search._model_index.update(expires=0, entries={})
|
||||
patcher = mock.patch.object(model_resolution, 'collect_model_hash_index', return_value={
|
||||
'renamed_copy.safetensors': {'hash': '667b5e3bede5a2fd53bb6f0ea5b93a89453d6855fa2e5d95acde93cf543c4f7c'},
|
||||
})
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
|
||||
def tearDown(self):
|
||||
self._tmp.cleanup()
|
||||
|
||||
def names(self, query):
|
||||
return [image['filename'] for image in image_search.filter_images(str(self.root), self.images, query)]
|
||||
|
||||
def test_prompt_text_models_and_seed(self):
|
||||
self.assertEqual(self.names('赛博朋克'), ['a.png'])
|
||||
self.assertEqual(self.names('miaomiao realskin'), ['a.png'], 'all terms must match')
|
||||
self.assertEqual(self.names('123456789'), ['a.png'])
|
||||
self.assertEqual(self.names('seed:123456789'), ['a.png'])
|
||||
self.assertEqual(self.names('neon cat'), [])
|
||||
|
||||
def test_search_blocks_keep_phrases_whole(self):
|
||||
# A list is one phrase per block; a plain string still splits on spaces.
|
||||
self.assertEqual(self.names(['city, neon']), ['a.png'])
|
||||
self.assertEqual(self.names(['neon city']), [])
|
||||
self.assertEqual(self.names('neon city'), ['a.png'])
|
||||
self.assertEqual(self.names(['cat in rain', 'Seed: 42']), ['c.png'])
|
||||
self.assertEqual(image_search.parse_query([' A b ', 'a b', '']), ['a b'])
|
||||
|
||||
def test_a1111_parameters_and_file_names(self):
|
||||
self.assertEqual(self.names('cat'), ['c.png'])
|
||||
self.assertEqual(self.names('abcdef1234'), ['c.png'], 'A1111 model hash text')
|
||||
self.assertEqual(self.names('plain'), ['plain.jpg'])
|
||||
|
||||
def test_hash_matches_recorded_hashes_and_local_renamed_files(self):
|
||||
# a.png recorded the hash; b.png only names a local file whose scanned hash matches.
|
||||
self.assertEqual(self.names('667B5E3B'), ['a.png', 'b.png'])
|
||||
self.assertEqual(self.names('667b5e3bede5a2fd'), ['a.png', 'b.png'])
|
||||
self.assertEqual(self.names('deadbeef'), [])
|
||||
|
||||
def test_records_are_cached_by_mtime(self):
|
||||
self.names('cat')
|
||||
with mock.patch.object(image_search, 'build_record', side_effect=AssertionError('re-read')):
|
||||
images = [dict(image, mtime=image_search._record_cache[str(self.root / image['subfolder'] / image['filename'])][0])
|
||||
for image in self.images]
|
||||
self.assertEqual([i['filename'] for i in image_search.filter_images(str(self.root), images, 'cat')], ['c.png'])
|
||||
|
||||
def test_only_text_chunks_before_pixels_are_read(self):
|
||||
chunks = image_search.read_png_text(self.root / 'a.png')
|
||||
self.assertEqual(set(chunks), {'prompt', 'workflow'})
|
||||
self.assertIn('赛博朋克', json.loads(chunks['prompt'])['6']['inputs']['text'])
|
||||
|
||||
|
||||
class ModelResolutionImportTests(unittest.TestCase):
|
||||
def test_model_scan_uses_model_extensions(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
Path(directory, 'model.safetensors').write_bytes(b'x')
|
||||
Path(directory, 'notes.txt').write_text('x')
|
||||
with mock.patch.object(model_resolution.folder_paths, 'get_folder_paths', return_value=[directory]):
|
||||
candidates = model_resolution._collect_resolution_candidates(['checkpoints'])
|
||||
self.assertEqual([c['filename'] for c in candidates], ['model.safetensors'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,88 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import kept_images, materials, notebooks
|
||||
|
||||
IMAGE = {"type": "output", "filename": "a.png", "subfolder": "day"}
|
||||
|
||||
|
||||
class KeptImagesTests(unittest.IsolatedAsyncioTestCase):
|
||||
def setUp(self):
|
||||
temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
root = Path(temporary.name)
|
||||
self.output = root / "output"
|
||||
(self.output / "day").mkdir(parents=True)
|
||||
self.recipes = root / "recipes"
|
||||
self.recipes.mkdir()
|
||||
self.materials = root / "materials"
|
||||
self.materials.mkdir()
|
||||
self.notebooks = root / "notebooks"
|
||||
self.notebooks.mkdir()
|
||||
(self.notebooks / ".legacy_imported.json").write_text("{}", encoding="utf-8")
|
||||
for target, name, value in (
|
||||
(materials.folder_paths, "get_output_directory", lambda: str(self.output)),
|
||||
(materials, "get_materials_dir", lambda: str(self.materials)),
|
||||
(kept_images, "get_materials_dir", lambda: str(self.materials)),
|
||||
(kept_images, "get_recipes_dir", lambda: str(self.recipes)),
|
||||
(notebooks, "get_notebooks_dir", lambda: str(self.notebooks)),
|
||||
):
|
||||
patcher = mock.patch.object(target, name, value)
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
|
||||
async def _post(self, handler, payload):
|
||||
async def body():
|
||||
return payload
|
||||
response = await handler(SimpleNamespace(json=body))
|
||||
return response.status, json.loads(response.text)
|
||||
|
||||
async def _kept(self):
|
||||
response = await kept_images.api_kept_images(SimpleNamespace(query={}))
|
||||
return json.loads(response.text)["images"]
|
||||
|
||||
async def test_nothing_kept(self):
|
||||
self.assertEqual(await self._kept(), [])
|
||||
|
||||
async def test_prompt_plan_remembers_its_image(self):
|
||||
plan = {"version": 2, "positive": "1girl", "negative": "lowres", "parts": [
|
||||
{"name": "x", "category": "subject", "role": "positive", "positive": "1girl", "negative": "", "enabled": True}]}
|
||||
status, saved = await self._post(materials.api_save_prompt_plan, {"name": "From a", "plan": plan, "source_image": IMAGE})
|
||||
self.assertEqual(status, 200)
|
||||
self.assertEqual(await self._kept(), [{"subfolder": "day", "filename": "a.png", "recipe": None, "combo": None,
|
||||
"prompt": {"filename": saved["filename"], "name": "From a"}}])
|
||||
# The same prompts again come back as the one saved, not a copy.
|
||||
status, again = await self._post(materials.api_save_prompt_plan, {"name": "Again", "plan": plan, "source_image": IMAGE})
|
||||
self.assertEqual((status, again["status"], again["filename"]), (409, "duplicate", saved["filename"]))
|
||||
|
||||
async def test_prompt_plan_rejects_an_image_outside_output(self):
|
||||
plan = {"version": 2, "positive": "1girl", "negative": "", "parts": []}
|
||||
for image in ({"type": "input", "filename": "a.png"}, {"type": "output", "filename": "../a.png"},
|
||||
{"type": "output", "filename": "a.png", "subfolder": "../.."}):
|
||||
status, _ = await self._post(materials.api_save_prompt_plan, {"name": "x", "plan": plan, "source_image": image})
|
||||
self.assertEqual(status, 400, image)
|
||||
|
||||
async def test_combo_and_recipe_are_listed_by_their_image(self):
|
||||
status, _ = await self._post(notebooks.api_save_notebook, {
|
||||
"filename": "Combo a.json", "data": {"name": "Combo a", "mainModel": None, "loras": [], "source_image": IMAGE}})
|
||||
self.assertEqual(status, 200)
|
||||
(self.recipes / "recipe_1.json").write_text(json.dumps({
|
||||
"name": "Recipe a", "workflow": {"nodes": [], "links": []}, "source_image": IMAGE}), encoding="utf-8")
|
||||
(self.recipes / "recipe_2.json").write_text(json.dumps({
|
||||
"name": "Other", "workflow": {"nodes": [], "links": []}}), encoding="utf-8")
|
||||
images = await self._kept()
|
||||
self.assertEqual(len(images), 1)
|
||||
self.assertEqual(images[0]["combo"], {"filename": "Combo a.json", "name": "Combo a"})
|
||||
self.assertEqual(images[0]["recipe"]["filename"], "recipe_1.json")
|
||||
self.assertIsNone(images[0]["prompt"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,86 @@
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import materials, utils
|
||||
|
||||
|
||||
def material(name, node_type='KSampler'):
|
||||
return {'name': name, 'workflow': {'nodes': [{'id': 1, 'type': node_type, 'widgets_values': [1]}]}, 'timestamp': 1}
|
||||
|
||||
|
||||
class ListingIndexTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
materials._cached_material_summary.cache_clear()
|
||||
utils._invalidate_gallery_snapshot()
|
||||
|
||||
def test_warm_summary_avoids_workflow_reads_and_detects_file_changes(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory, 'one.json')
|
||||
path.write_text(json.dumps(material('first')))
|
||||
with mock.patch.object(materials, '_read_material', wraps=materials._read_material) as read:
|
||||
first = materials._list_materials(directory)
|
||||
first[0]['name'] = 'mutated caller'
|
||||
self.assertEqual(materials._list_materials(directory)[0]['name'], 'first')
|
||||
self.assertEqual(read.call_count, 1)
|
||||
path.write_text(json.dumps(material('changed name')))
|
||||
self.assertEqual(materials._list_materials(directory)[0]['name'], 'changed name')
|
||||
self.assertEqual(read.call_count, 2)
|
||||
path.unlink()
|
||||
self.assertEqual(materials._list_materials(directory), [])
|
||||
|
||||
def test_node_lookup_reads_only_matching_records_after_index_warmup(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
Path(directory, 'one.json').write_text(json.dumps(material('one', 'KSampler')))
|
||||
Path(directory, 'two.json').write_text(json.dumps(material('two', 'CLIPTextEncode')))
|
||||
materials._list_materials(directory)
|
||||
request = type('Request', (), {'query': {'type': 'KSampler'}})()
|
||||
with mock.patch.object(materials, 'get_materials_dir', return_value=directory), mock.patch.object(materials, '_read_material', wraps=materials._read_material) as read:
|
||||
response = asyncio.run(materials.api_get_materials_by_node_type(request))
|
||||
self.assertEqual(len(json.loads(response.text)['materials']), 1)
|
||||
self.assertEqual(read.call_count, 1)
|
||||
|
||||
def test_gallery_pagination_reuses_snapshot_and_refresh_sees_new_output(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
Path(directory, 'one.png').write_bytes(b'image')
|
||||
with mock.patch.object(utils, '_collect_gallery_images', wraps=utils._collect_gallery_images) as collect:
|
||||
self.assertEqual(len(utils._gallery_images(directory)), 1)
|
||||
Path(directory, 'two.png').write_bytes(b'image')
|
||||
self.assertEqual(len(utils._gallery_images(directory)), 1)
|
||||
self.assertEqual(collect.call_count, 1)
|
||||
self.assertEqual(len(utils._gallery_images(directory, refresh=True)), 2)
|
||||
utils._invalidate_gallery_snapshot()
|
||||
utils._gallery_images(directory)
|
||||
self.assertEqual(collect.call_count, 3)
|
||||
|
||||
def test_gallery_ttl_and_root_change_invalidate_snapshot(self):
|
||||
with tempfile.TemporaryDirectory() as first, tempfile.TemporaryDirectory() as second:
|
||||
with mock.patch.object(utils, '_collect_gallery_images', wraps=utils._collect_gallery_images) as collect:
|
||||
utils._gallery_images(first)
|
||||
utils._gallery_images(second)
|
||||
utils._gallery_snapshot['expires'] = 0
|
||||
utils._gallery_images(second)
|
||||
self.assertEqual(collect.call_count, 3)
|
||||
|
||||
|
||||
def test_gallery_lists_pictures_unless_asked_and_hides_formats(self):
|
||||
from api import gallery_routes
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
for name in ('a.png', 'b.jpeg', 'c.mp4', 'd.mkv', 'e.avi', 'f.txt'):
|
||||
Path(directory, name).write_bytes(b'x')
|
||||
names = lambda items: sorted(item['filename'] for item in items)
|
||||
self.assertEqual(names(gallery_routes._gallery_listing(directory, True, '')), ['a.png', 'b.jpeg'])
|
||||
self.assertEqual(names(gallery_routes._gallery_listing(directory, False, '', 'video')), ['c.mp4', 'd.mkv'])
|
||||
self.assertEqual(names(gallery_routes._gallery_listing(directory, False, '', 'all', frozenset({'jpg', 'mkv'}))),
|
||||
['a.png', 'c.mp4'])
|
||||
images, has_videos = gallery_routes._gallery_page_source(directory, False, '', 'image', frozenset())
|
||||
self.assertEqual((len(images), has_videos), (2, True))
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Only this computer may change things: one middleware in front of every /anomalous/ route."""
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from aiohttp import web
|
||||
from aiohttp.test_utils import make_mocked_request
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
from api import local_guard
|
||||
|
||||
|
||||
def ask(method, path, remote="127.0.0.1", host="127.0.0.1:8188", **headers):
|
||||
"""Runs the middleware for one request; the handler answers 200 "done"."""
|
||||
async def handler(request):
|
||||
return web.Response(text="done")
|
||||
|
||||
request = make_mocked_request(method, path, headers={"Host": host, **headers},
|
||||
transport=_Transport(remote))
|
||||
response = asyncio.run(local_guard.local_only_middleware(request, handler))
|
||||
return response.status, response.text
|
||||
|
||||
|
||||
class _Transport:
|
||||
def __init__(self, remote):
|
||||
self.remote = remote
|
||||
|
||||
def get_extra_info(self, name, default=None):
|
||||
return (self.remote, 50000) if name == "peername" else default
|
||||
|
||||
|
||||
class LocalGuardTests(unittest.TestCase):
|
||||
def test_changes_from_another_device_are_refused(self):
|
||||
for path in ("/anomalous/delete_model", "/anomalous/backup/export", "/anomalous/translate",
|
||||
"/anomalous/save_config", "/anomalous/version/switch", "/anomalous/download/start"):
|
||||
status, text = ask("POST", path, remote="192.168.1.20", host="192.168.1.5:8188")
|
||||
self.assertEqual(status, 403, path)
|
||||
self.assertEqual(json.loads(text)["code"], "local_only")
|
||||
|
||||
def test_the_lan_address_counts_as_another_device_even_from_this_computer(self):
|
||||
self.assertEqual(ask("POST", "/anomalous/delete_model", remote="127.0.0.1", host="192.168.1.5:8188")[0], 403)
|
||||
self.assertEqual(ask("POST", "/anomalous/delete_model", host="evil.example:8188")[0], 403) # DNS rebinding
|
||||
self.assertEqual(ask("POST", "/anomalous/delete_model", **{"X-Forwarded-For": "8.8.8.8"})[0], 403) # a proxy
|
||||
|
||||
def test_this_computer_may_change_things(self):
|
||||
for host in ("127.0.0.1:8188", "localhost:8188", "[::1]:8188"):
|
||||
self.assertEqual(ask("POST", "/anomalous/delete_model", host=host)[0], 200, host)
|
||||
|
||||
def test_other_devices_may_look(self):
|
||||
lan = {"remote": "192.168.1.20", "host": "192.168.1.5:8188"}
|
||||
for path in ("/anomalous/models", "/anomalous/image", "/anomalous/gallery_images", "/anomalous/recipes"):
|
||||
self.assertEqual(ask("GET", path, **lan)[0], 200, path)
|
||||
for path in local_guard.OPEN_POSTS:
|
||||
self.assertEqual(ask("POST", path, **lan)[0], 200, path)
|
||||
for path in local_guard.LOCAL_GETS:
|
||||
self.assertEqual(ask("GET", path, **lan)[0], 403, path)
|
||||
self.assertEqual(ask("POST", "/prompt", **lan)[0], 200) # not ours to decide
|
||||
self.assertEqual(ask("POST", "/anomalous/mcp", **lan)[0], 200) # answers in JSON-RPC itself
|
||||
|
||||
def test_the_page_learns_whether_it_may_change_things(self):
|
||||
async def run(remote, host):
|
||||
request = make_mocked_request("GET", "/anomalous/access", headers={"Host": host}, transport=_Transport(remote))
|
||||
return json.loads((await local_guard.api_access(request)).text)
|
||||
|
||||
self.assertEqual(asyncio.run(run("127.0.0.1", "127.0.0.1:8188")), {"local": True})
|
||||
self.assertEqual(asyncio.run(run("192.168.1.20", "192.168.1.5:8188")), {"local": False})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,75 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
from PIL import Image
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import materials
|
||||
|
||||
|
||||
class MaterialBrowsingTests(unittest.TestCase):
|
||||
def test_filters_pagination_and_tag_edit_preserve_workflow(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
workflow = {'nodes': [{'id': 1, 'type': 'KSampler', 'widgets_values': [123]}]}
|
||||
for i in range(55):
|
||||
Path(directory, f'{i}.json').write_text(json.dumps({'name': f'Material {i}', 'workflow': workflow, 'kind': 'image_workflow_snapshot', 'tags': ['Portrait'] if i % 2 else ['景色'], 'timestamp': i}))
|
||||
first = materials._query_materials(directory, {'category': 'workflow', 'page': '1'})
|
||||
second = materials._query_materials(directory, {'category': 'workflow', 'page': '2'})
|
||||
self.assertEqual((len(first['materials']), len(second['materials']), first['total']), (48, 7, 55))
|
||||
self.assertFalse({m['filename'] for m in first['materials']} & {m['filename'] for m in second['materials']})
|
||||
self.assertEqual(materials._query_materials(directory, {'category': 'workflow', 'tag': 'portrait'})['total'], 27)
|
||||
# Whole workflows are recipes now: "all" (the pieces) does not list them.
|
||||
self.assertEqual(materials._query_materials(directory, {})['total'], 0)
|
||||
self.assertEqual(materials._query_materials(directory, {'category': 'workflow', 'q': 'ksampler'})['total'], 55)
|
||||
self.assertEqual(materials._query_materials(directory, {'kind': 'image_node_selection'})['total'], 0)
|
||||
materials._update_material_details(directory, '0.json', 'Renamed', ['new'])
|
||||
self.assertEqual(materials._query_materials(directory, {'category': 'workflow', 'q': 'renamed', 'tag': 'new'})['total'], 1)
|
||||
self.assertEqual(json.loads(Path(directory, '0.json').read_text())['workflow'], workflow)
|
||||
# One moved to the recipes is not listed again; its file stays whole.
|
||||
materials._store._mark_material_moved(directory, '0.json', 'recipe_1.json')
|
||||
self.assertEqual(materials._query_materials(directory, {'category': 'workflow'})['total'], 54)
|
||||
self.assertEqual(json.loads(Path(directory, '0.json').read_text())['workflow'], workflow)
|
||||
|
||||
def test_duplicate_warning_uses_image_and_selected_scope(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
source = Path(directory, 'image.png'); Image.new('RGB', (8, 8)).save(source)
|
||||
store = Path(directory, 'library'); store.mkdir()
|
||||
record = {'name': 'one', 'kind': 'image_node_selection', 'workflow': {'nodes': [{'id': 1, 'type': 'KSampler'}, {'id': 2, 'type': 'CLIPTextEncode'}]}, 'selection': {'node_ids': ['1', '2']}}
|
||||
self.assertIsNone(materials._persist_material(str(store), 'one.json', str(source), record))
|
||||
record['selection']['node_ids'].reverse()
|
||||
duplicate = materials._persist_material(str(store), 'two.json', str(source), record)
|
||||
self.assertEqual(duplicate['filename'], 'one.json')
|
||||
self.assertFalse((store/'two.json').exists())
|
||||
self.assertFalse((store/'.assets/two').exists())
|
||||
self.assertIsNone(materials._persist_material(str(store), 'two.json', str(source), record, allow_duplicate=True))
|
||||
record['selection']['node_ids'] = ['1']
|
||||
self.assertIsNone(materials._persist_material(str(store), 'three.json', str(source), record))
|
||||
|
||||
def test_tags_are_bounded_and_case_insensitive_duplicates_removed(self):
|
||||
self.assertEqual(materials._normalise_material_tags([' Portrait ', 'portrait', '景色']), ['Portrait', '景色'])
|
||||
for tags in ('bad', ['x' * 61], ['x'] * 21, [123]):
|
||||
with self.assertRaises(ValueError):
|
||||
materials._normalise_material_tags(tags)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
||||
|
||||
class MaterialsByNodeTypeTests(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_moved_workflow_is_not_offered_again(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
workflow = {'nodes': [{'id': 1, 'type': 'KSampler', 'widgets_values': [123, 'fixed', 20]}]}
|
||||
for name in ('kept', 'moved'):
|
||||
Path(directory, f'{name}.json').write_text(json.dumps({'name': name, 'workflow': workflow, 'kind': 'image_workflow_snapshot'}))
|
||||
materials._store._mark_material_moved(directory, 'moved.json', 'recipe_1.json')
|
||||
with mock.patch.object(materials, 'get_materials_dir', lambda: directory):
|
||||
response = await materials.api_get_materials_by_node_type(SimpleNamespace(query={'type': 'KSampler'}))
|
||||
self.assertEqual([item['name'] for item in json.loads(response.text)['materials']], ['kept'])
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import materials
|
||||
|
||||
|
||||
class MaterialDetailTests(unittest.IsolatedAsyncioTestCase):
|
||||
def setUp(self):
|
||||
self.directory = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.directory.cleanup)
|
||||
self.path = Path(self.directory.name, 'fixture.json')
|
||||
self.workflow = {
|
||||
'nodes': [
|
||||
{'id': 1, 'type': 'CLIPTextEncode', 'widgets_values': [' exact prompt\n']},
|
||||
{'id': 2, 'type': 'KSampler', 'widgets_values': [123, 'fixed', 20, 7, 'euler', 'normal', 1], 'properties': {'custom': 'kept'}, 'mode': 0},
|
||||
],
|
||||
'links': [[1, 1, 0, 2, 1, 'CONDITIONING']],
|
||||
'extra': {'anomalous_hashes': {'2_model': {'sha256': 'a' * 64}}},
|
||||
}
|
||||
self.record = {'kind': 'image_workflow_snapshot', 'name': 'Fixture', 'tags': ['tag'],
|
||||
'capabilities': ['open_workflow', 'apply_node_parameters'],
|
||||
'workflow': self.workflow, 'source': {'image': {'filename': 'source.png'}},
|
||||
'model_references': [{'saved_value': 'model.safetensors'}]}
|
||||
self.write_record()
|
||||
|
||||
def write_record(self):
|
||||
self.path.write_text(json.dumps(self.record), encoding='utf-8')
|
||||
|
||||
async def request(self, **query):
|
||||
with mock.patch.object(materials, 'get_materials_dir', return_value=self.directory.name):
|
||||
return await materials.api_get_material_full(SimpleNamespace(query={'filename': 'fixture.json', **query}))
|
||||
|
||||
async def test_detail_has_exact_blocks_without_workflow_or_disk_changes(self):
|
||||
original = self.path.read_bytes()
|
||||
response = await self.request(include_workflow='0')
|
||||
self.assertEqual(response.status, 200)
|
||||
payload = json.loads(response.body)
|
||||
self.assertNotIn('workflow', payload['data'])
|
||||
self.assertEqual(payload['data']['model_references'], self.record['model_references'])
|
||||
self.assertEqual(payload['data']['source'], self.record['source'])
|
||||
self.assertEqual(payload['node_blocks'][0]['widgets_values'], [' exact prompt\n'])
|
||||
self.assertEqual(payload['node_blocks'][1]['properties'], {'custom': 'kept'})
|
||||
self.assertEqual(payload['node_blocks'][1]['volatile_widget_indexes'], [0])
|
||||
self.assertEqual(self.path.read_bytes(), original)
|
||||
|
||||
async def test_open_has_exact_workflow_and_does_not_build_duplicate_blocks(self):
|
||||
with mock.patch.object(materials, '_material_node_blocks', side_effect=AssertionError('unneeded node blocks')):
|
||||
response = await self.request(include_workflow='1')
|
||||
payload = json.loads(response.body)
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(payload['data']['workflow'], self.workflow)
|
||||
self.assertEqual(payload['node_blocks'], [])
|
||||
|
||||
async def test_legacy_full_snapshot_response_remains_compatible(self):
|
||||
payload = json.loads((await self.request()).body)
|
||||
self.assertEqual(payload['data']['workflow'], self.workflow)
|
||||
self.assertEqual(len(payload['node_blocks']), 2)
|
||||
self.assertEqual(payload['node_blocks'][1]['widgets_values'][0], 123)
|
||||
|
||||
async def test_selected_material_never_sends_hidden_workflow(self):
|
||||
self.record.update(kind='image_node_selection', selection={'scope': 'nodes', 'node_ids': [2]})
|
||||
# Even an inconsistent capability must not expose the source graph.
|
||||
self.write_record()
|
||||
for query in ({}, {'include_workflow': '0'}):
|
||||
payload = json.loads((await self.request(**query)).body)
|
||||
self.assertNotIn('workflow', payload['data'])
|
||||
self.assertEqual([block['node_id'] for block in payload['node_blocks']], [2])
|
||||
self.assertNotIn('exact prompt', json.dumps(payload))
|
||||
self.assertEqual((await self.request(include_workflow='1')).status, 403)
|
||||
|
||||
async def test_invalid_requests_are_rejected(self):
|
||||
self.assertEqual((await self.request(include_workflow='yes')).status, 400)
|
||||
self.assertEqual((await self.request(filename='../fixture.json')).status, 400)
|
||||
self.assertEqual((await self.request(filename='missing.json')).status, 404)
|
||||
|
||||
def test_selection_filters_before_copying_values_and_keeps_occurrences(self):
|
||||
class UnusedValues(list):
|
||||
def __deepcopy__(self, memo):
|
||||
raise AssertionError('copied a hidden node')
|
||||
record = {'workflow': {'nodes': [
|
||||
{'id': 1, 'type': 'KSampler', 'widgets_values': UnusedValues([999])},
|
||||
{'id': 2, 'type': 'KSampler', 'widgets_values': [123]},
|
||||
]}, 'selection': {'scope': 'nodes', 'node_ids': [2]}}
|
||||
blocks = materials._material_node_blocks(record)
|
||||
self.assertEqual(blocks[0]['occurrence'], 2)
|
||||
self.assertEqual(blocks[0]['widgets_values'], [123])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,323 @@
|
||||
"""The MCP endpoint: both protocol eras, who may connect, and the read-only tools."""
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from aiohttp.test_utils import make_mocked_request
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
from api import lora_info, mcp_actions, mcp_bridge, mcp_server, mcp_tools # noqa: E402
|
||||
|
||||
MODERN = "2026-07-28"
|
||||
|
||||
|
||||
def meta(version=MODERN):
|
||||
return {"io.modelcontextprotocol/protocolVersion": version,
|
||||
"io.modelcontextprotocol/clientInfo": {"name": "test", "version": "1"},
|
||||
"io.modelcontextprotocol/clientCapabilities": {}}
|
||||
|
||||
|
||||
def post(body, headers=None, remote="127.0.0.1", method="POST"):
|
||||
"""Runs the endpoint on one request; returns (status, parsed body or None)."""
|
||||
transport = mock.Mock()
|
||||
transport.get_extra_info.side_effect = lambda key, default=None: (remote, 5555) if key == "peername" else default
|
||||
request = make_mocked_request(method, "/anomalous/mcp", transport=transport,
|
||||
headers={"Host": "127.0.0.1:8188", "Content-Type": "application/json", **(headers or {})})
|
||||
request.json = lambda: asyncio.sleep(0, body)
|
||||
response = asyncio.run(mcp_server.api_mcp(request))
|
||||
text = getattr(response, "text", None)
|
||||
return response.status, json.loads(text) if text else None
|
||||
|
||||
|
||||
def modern_headers(method, name=None):
|
||||
headers = {"MCP-Protocol-Version": MODERN, "Mcp-Method": method}
|
||||
if name is not None:
|
||||
headers["Mcp-Name"] = name
|
||||
return headers
|
||||
|
||||
|
||||
class ProtocolTests(unittest.TestCase):
|
||||
def test_legacy_clients_initialize_then_list_tools(self):
|
||||
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "initialize",
|
||||
"params": {"protocolVersion": "2025-06-18", "capabilities": {}, "clientInfo": {"name": "c", "version": "1"}}})
|
||||
self.assertEqual(status, 200)
|
||||
self.assertEqual(body["result"]["protocolVersion"], "2025-06-18")
|
||||
self.assertIn("tools", body["result"]["capabilities"])
|
||||
self.assertEqual(post({"jsonrpc": "2.0", "method": "notifications/initialized"})[0], 202)
|
||||
status, body = post({"jsonrpc": "2.0", "id": 2, "method": "tools/list"}, {"MCP-Protocol-Version": "2025-06-18"})
|
||||
names = [tool["name"] for tool in body["result"]["tools"]]
|
||||
self.assertIn("search_models", names)
|
||||
self.assertNotIn("resultType", body["result"]) # legacy shape
|
||||
reading = [tool for tool in body["result"]["tools"] if tool["name"] in mcp_tools.TOOLS]
|
||||
self.assertTrue(reading and all(tool["annotations"]["readOnlyHint"] for tool in reading))
|
||||
# An unknown legacy version is answered with the latest one we speak.
|
||||
status, body = post({"jsonrpc": "2.0", "id": 3, "method": "initialize", "params": {"protocolVersion": "2030-01-01"}})
|
||||
self.assertEqual(body["result"]["protocolVersion"], "2025-11-25")
|
||||
|
||||
def test_modern_requests_are_stateless_and_headers_must_match(self):
|
||||
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {"_meta": meta()}},
|
||||
modern_headers("tools/list"))
|
||||
self.assertEqual(status, 200)
|
||||
self.assertEqual(body["result"]["resultType"], "complete")
|
||||
self.assertEqual(body["result"]["cacheScope"], "public")
|
||||
status, body = post({"jsonrpc": "2.0", "id": 2, "method": "server/discover", "params": {"_meta": meta()}},
|
||||
modern_headers("server/discover"))
|
||||
self.assertIn(MODERN, body["result"]["supportedVersions"])
|
||||
# Header missing, or naming another tool than the body.
|
||||
status, body = post({"jsonrpc": "2.0", "id": 3, "method": "tools/list", "params": {"_meta": meta()}})
|
||||
self.assertEqual((status, body["error"]["code"]), (400, -32020))
|
||||
call = {"jsonrpc": "2.0", "id": 4, "method": "tools/call",
|
||||
"params": {"name": "scan_report", "arguments": {}, "_meta": meta()}}
|
||||
status, body = post(call, modern_headers("tools/call", "search_models"))
|
||||
self.assertEqual((status, body["error"]["code"]), (400, -32020))
|
||||
# A base64-encoded Mcp-Name is decoded before comparing.
|
||||
with mock.patch.object(mcp_tools, "_summarize", return_value=dict.fromkeys(
|
||||
("total", "matched", "unmatched", "new", "pending"), 0) | {"unmatched_models": [], "new_models": []}), \
|
||||
mock.patch.object(mcp_tools, "last_scan_path", return_value="no such file"):
|
||||
status, body = post(call, modern_headers("tools/call", "=?base64?c2Nhbl9yZXBvcnQ=?="))
|
||||
self.assertEqual(status, 200)
|
||||
self.assertFalse(body["result"]["isError"])
|
||||
|
||||
def test_versions_and_methods_we_do_not_speak(self):
|
||||
status, body = post({"jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {"_meta": meta("2030-01-01")}},
|
||||
{"MCP-Protocol-Version": "2030-01-01", "Mcp-Method": "tools/list"})
|
||||
self.assertEqual((status, body["error"]["code"]), (400, -32022))
|
||||
self.assertIn(MODERN, body["error"]["data"]["supported"])
|
||||
status, body = post({"jsonrpc": "2.0", "id": 2, "method": "resources/list", "params": {"_meta": meta()}},
|
||||
modern_headers("resources/list"))
|
||||
self.assertEqual((status, body["error"]["code"]), (404, -32601))
|
||||
self.assertEqual(post({}, method="GET")[0], 405)
|
||||
self.assertEqual(post([{"jsonrpc": "2.0", "id": 1, "method": "ping"}])[0], 400) # no batches
|
||||
|
||||
def test_only_this_computer_may_connect(self):
|
||||
ping = {"jsonrpc": "2.0", "id": 1, "method": "ping"}
|
||||
self.assertEqual(post(ping)[0], 200)
|
||||
self.assertEqual(post(ping, remote="192.168.1.20")[0], 403)
|
||||
self.assertEqual(post(ping, {"X-Forwarded-For": "8.8.8.8"})[0], 403) # behind a proxy
|
||||
self.assertEqual(post(ping, {"Origin": "http://evil.example"})[0], 403) # a web page
|
||||
self.assertEqual(post(ping, {"Host": "evil.example:8188"})[0], 403) # DNS rebinding
|
||||
self.assertEqual(post(ping, {"Origin": "http://localhost:8188"})[0], 200)
|
||||
|
||||
|
||||
class ToolTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(os.path.realpath(self.temp.name)) # the tools resolve short Windows names
|
||||
|
||||
def tearDown(self):
|
||||
self.temp.cleanup()
|
||||
|
||||
def test_search_and_detail_of_models(self):
|
||||
(self.root / "sub").mkdir()
|
||||
lora = self.root / "sub" / "neon_city.safetensors"
|
||||
lora.write_bytes(b"x")
|
||||
(self.root / "plain.ckpt").write_bytes(b"x")
|
||||
files = [("loras", 0, str(self.root), str(lora)), ("loras", 0, str(self.root), str(self.root / "plain.ckpt"))]
|
||||
metas = {str(lora): {"name": "Neon City", "baseModel": "SDXL 1.0", "trainedWords": ["neon"], "info_source": "civitai",
|
||||
"description": "<p>Bright <b>lights</b></p>", "custom_notes": "mine"},
|
||||
str(self.root / "plain.ckpt"): {"info_source": ""}}
|
||||
with mock.patch.object(mcp_tools, "_iter_search_models", return_value=files), \
|
||||
mock.patch.object(mcp_tools, "get_metadata", side_effect=lambda path: metas[path]), \
|
||||
mock.patch.object(mcp_tools, "resolve_folder_subdir", side_effect=lambda t, i, s: (str(self.root), str(self.root / s.strip("/")))):
|
||||
found = mcp_tools.call_tool(mcp_tools.TOOLS, "search_models", {"query": "neon sdxl"})["structuredContent"]
|
||||
self.assertEqual(found["total_matches"], 1)
|
||||
model = found["models"][0]
|
||||
self.assertEqual((model["model_id"], model["source"]), ("loras:0:sub/neon_city.safetensors", "civitai"))
|
||||
self.assertEqual(mcp_tools.call_tool(mcp_tools.TOOLS, "search_models", {"source": "not_scanned"})["structuredContent"]["total_matches"], 1)
|
||||
detail = mcp_tools.call_tool(mcp_tools.TOOLS, "get_model", {"model_id": model["model_id"]})["structuredContent"]
|
||||
self.assertEqual((detail["description"], detail["user_notes"]), ("Bright lights", "mine"))
|
||||
for bad in ("loras:0:../../secret.safetensors", "loras:0:sub/missing.safetensors", "nonsense"):
|
||||
result = mcp_tools.call_tool(mcp_tools.TOOLS, "get_model", {"model_id": bad})
|
||||
self.assertTrue(result["isError"], bad)
|
||||
|
||||
def test_image_settings_come_from_the_png_and_stay_inside_output(self):
|
||||
from PIL import Image, PngImagePlugin
|
||||
prompt = {"3": {"class_type": "KSampler", "inputs": {"seed": 42, "model": ["4", 0]}},
|
||||
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "sdxl/base.safetensors"}}}
|
||||
info = PngImagePlugin.PngInfo()
|
||||
info.add_text("prompt", json.dumps(prompt))
|
||||
Image.new("RGB", (8, 8)).save(self.root / "out.png", pnginfo=info)
|
||||
with mock.patch.object(mcp_tools.folder_paths, "get_output_directory", return_value=str(self.root)), \
|
||||
mock.patch.object(mcp_tools, "_kept_images", return_value=[{"subfolder": "", "filename": "out.png",
|
||||
"recipe": None, "combo": {"name": "C"}, "prompt": None}]):
|
||||
data = mcp_tools.call_tool(mcp_tools.TOOLS, "get_image_info", {"image": "out.png"})["structuredContent"]
|
||||
self.assertEqual(data["models_used"], ["sdxl/base.safetensors"])
|
||||
sampler = next(node for node in data["nodes"] if node["type"] == "KSampler")
|
||||
self.assertEqual(sampler["inputs"], {"seed": 42}) # links are left out
|
||||
self.assertEqual(data["kept_as"], {"combo": "C"})
|
||||
self.assertTrue(mcp_tools.call_tool(mcp_tools.TOOLS, "get_image_info", {"image": "../out.png"})["isError"])
|
||||
|
||||
def test_bad_arguments_are_tool_errors(self):
|
||||
self.assertTrue(mcp_tools.call_tool(mcp_tools.TOOLS, "get_model", {})["isError"])
|
||||
self.assertTrue(mcp_tools.call_tool(mcp_tools.TOOLS, "search_models", {"delete": True})["isError"])
|
||||
|
||||
|
||||
class BridgeTests(unittest.TestCase):
|
||||
"""A canvas action waits for the one page that claims it."""
|
||||
|
||||
def server(self, sockets=True):
|
||||
sent = []
|
||||
server = mock.Mock(sockets={"a": object()} if sockets else {})
|
||||
server.send_sync.side_effect = lambda event, data: sent.append(data)
|
||||
return server, sent
|
||||
|
||||
def test_first_claim_wins_and_its_outcome_is_returned(self):
|
||||
server, sent = self.server()
|
||||
|
||||
def page():
|
||||
while not sent:
|
||||
pass
|
||||
request_id = sent[0]["id"]
|
||||
self.assertTrue(mcp_bridge.claim(request_id))
|
||||
self.assertFalse(mcp_bridge.claim(request_id)) # a second tab
|
||||
self.assertTrue(mcp_bridge.resolve(request_id, {"ok": True, "value": {"nodes": 3}}))
|
||||
|
||||
import threading
|
||||
with mock.patch.object(mcp_bridge, "_server", return_value=server):
|
||||
thread = threading.Thread(target=page)
|
||||
thread.start()
|
||||
value = mcp_bridge.ask_page("describe_canvas", timeout=5)
|
||||
thread.join()
|
||||
self.assertEqual(value, {"nodes": 3})
|
||||
self.assertEqual(sent[0]["action"], "describe_canvas")
|
||||
self.assertFalse(mcp_bridge.resolve(sent[0]["id"], {"ok": True})) # nobody waits any more
|
||||
|
||||
def test_no_page_or_no_answer_is_a_tool_error(self):
|
||||
server, _sent = self.server(sockets=False)
|
||||
with mock.patch.object(mcp_bridge, "_server", return_value=server):
|
||||
result = mcp_tools.call_tool(mcp_server.TOOLS, "describe_canvas", {}, {"base_url": "http://127.0.0.1:1"})
|
||||
self.assertTrue(result["isError"])
|
||||
self.assertIn("not open", result["content"][0]["text"])
|
||||
server, _sent = self.server()
|
||||
with mock.patch.object(mcp_bridge, "_server", return_value=server), mock.patch.object(mcp_bridge, "TIMEOUT_SECONDS", 0.05):
|
||||
with self.assertRaises(mcp_bridge.PageUnavailable):
|
||||
mcp_bridge.ask_page("describe_canvas", timeout=0.05)
|
||||
self.assertEqual(mcp_bridge._waiting, {})
|
||||
|
||||
def test_a_refusal_from_the_page_reaches_the_ai(self):
|
||||
with mock.patch.object(mcp_actions, "ask_page", side_effect=mcp_bridge.PageRefused("Node 9 has no model drop-down.")), mock.patch.object(mcp_actions, "_resolve_model", return_value=("loras", 0, "/m", "/m/a.safetensors")):
|
||||
result = mcp_tools.call_tool(mcp_server.TOOLS, "set_model", {"node_id": 9, "model_id": "loras:0:a.safetensors"}, {})
|
||||
self.assertTrue(result["isError"])
|
||||
self.assertIn("Node 9", result["content"][0]["text"])
|
||||
|
||||
def test_bridge_route_is_local_only(self):
|
||||
transport = mock.Mock()
|
||||
transport.get_extra_info.side_effect = lambda key, default=None: ("10.0.0.5", 1) if key == "peername" else default
|
||||
request = make_mocked_request("POST", "/anomalous/mcp/bridge", transport=transport, headers={"Host": "127.0.0.1:8188"})
|
||||
request.json = lambda: asyncio.sleep(0, {"id": "x", "claim": True})
|
||||
self.assertEqual(asyncio.run(mcp_server.api_mcp_bridge(request)).status, 403)
|
||||
|
||||
|
||||
class ServerActionTests(unittest.TestCase):
|
||||
"""Actions that go through this server's own routes."""
|
||||
CTX = {"base_url": "http://127.0.0.1:8188"}
|
||||
|
||||
def test_scanning_picked_models_groups_them_by_folder(self):
|
||||
calls = []
|
||||
models = {"loras:0:a/x.safetensors": ("loras", 0, "/m", "/m/a/x.safetensors"),
|
||||
"loras:0:a/y.safetensors": ("loras", 0, "/m", "/m/a/y.safetensors")}
|
||||
with mock.patch.object(mcp_actions, "_resolve_model", side_effect=lambda mid: models[mid]), mock.patch.object(mcp_actions.os.path, "relpath", side_effect=lambda f, b: f[len(b) + 1:]), mock.patch.object(mcp_actions, "_local", side_effect=lambda ctx, m, p, body=None, **k: calls.append(body) or (200, {"status": "ok"})):
|
||||
result = mcp_tools.call_tool(mcp_server.TOOLS, "scan_models", {"what": "models", "model_ids": list(models)}, self.CTX)
|
||||
self.assertFalse(result["isError"])
|
||||
self.assertEqual(calls[0]["targets"], [{"type": "loras", "path_idx": 0, "subfolder": "/a", "files": ["x.safetensors", "y.safetensors"]}])
|
||||
with mock.patch.object(mcp_actions, "_local", return_value=(409, {"status": "error"})):
|
||||
busy = mcp_tools.call_tool(mcp_server.TOOLS, "scan_models", {}, self.CTX)
|
||||
self.assertIn("already running", busy["content"][0]["text"])
|
||||
|
||||
def test_speak_queues_the_tts_node_and_waits_for_the_file(self):
|
||||
posted = []
|
||||
|
||||
def local(ctx, method, path, body=None, **kwargs):
|
||||
if path == "/anomalous_tts/characters":
|
||||
return 200, {"characters": [{"name": "Arona", "language": "ja", "emotions": {"happy": {}}}]}
|
||||
if path == "/prompt":
|
||||
posted.append(body)
|
||||
return 200, {"prompt_id": "p1"}
|
||||
return 200, {"p1": {"status": {"completed": True},
|
||||
"outputs": {"2": {"audio": [{"subfolder": "audio/Arona", "filename": "Arona_00001.flac"}]}}}}
|
||||
|
||||
with mock.patch.object(mcp_actions, "_local", side_effect=local), mock.patch.object(mcp_actions, "HISTORY_POLL_SECONDS", 0):
|
||||
data = mcp_tools.call_tool(mcp_server.TOOLS, "speak", {"character": "Arona", "text": "Sensei!", "seed": 7}, self.CTX)["structuredContent"]
|
||||
missing = mcp_tools.call_tool(mcp_server.TOOLS, "speak", {"character": "Nobody", "text": "hi"}, self.CTX)
|
||||
self.assertEqual(data["files"], ["audio/Arona/Arona_00001.flac"])
|
||||
node = posted[0]["prompt"]["1"]
|
||||
self.assertEqual((node["class_type"], node["inputs"]["text"], node["inputs"]["seed"]), ("AnomalousTTS_CharacterSpeech", "Sensei!", 7))
|
||||
self.assertEqual(posted[0]["prompt"]["2"]["inputs"]["filename_prefix"], "audio/Arona/Arona")
|
||||
self.assertTrue(missing["isError"])
|
||||
|
||||
def test_acting_tools_are_marked_and_none_deletes(self):
|
||||
for name, tool in mcp_server.TOOLS.items():
|
||||
annotations = tool["spec"]["annotations"]
|
||||
self.assertNotIn("delete", name)
|
||||
if name in mcp_actions.ACTIONS and name not in ("describe_canvas", "check_workflow_models", "list_voices", "download_status"):
|
||||
self.assertFalse(annotations["readOnlyHint"], name)
|
||||
|
||||
|
||||
class NodeComboTests(unittest.TestCase):
|
||||
def test_a_node_structure_combo_is_listed_by_slot(self):
|
||||
note = {"name": "Flux", "data": {"kind": "nodes", "values": {"s1": "Flux\\a.safetensors", "s2": "a cat"},
|
||||
"structure": {"nodes": [{"type": "UNETLoader", "name": "UNet"}, {"type": "CLIPTextEncode"}],
|
||||
"slots": [{"id": "s1", "kind": "model", "folder": "diffusion_models", "label": "Main"},
|
||||
{"id": "s2", "kind": "text", "label": "Positive"}]}}}
|
||||
with mock.patch.object(mcp_tools, "_list_notebooks", return_value=[note]):
|
||||
result = mcp_tools.list_combos("cat")
|
||||
combo = result["combos"][0]
|
||||
self.assertEqual(combo["models"], [{"slot": "Main", "folder": "diffusion_models", "file": "a.safetensors"}])
|
||||
self.assertEqual(combo["texts"], [{"slot": "Positive", "text": "a cat"}])
|
||||
self.assertEqual(combo["nodes"], ["UNet", "CLIPTextEncode"])
|
||||
|
||||
|
||||
class DownloadToolTests(unittest.TestCase):
|
||||
def test_status_is_summarised(self):
|
||||
jobs = {"jobs": [{"rel": "Flux/a.safetensors", "type": "loras", "state": "running", "error": "", "received": 50,
|
||||
"total": 200, "verified": False, "value": "Flux\a.safetensors"}]}
|
||||
with mock.patch.object(mcp_actions, "_local", return_value=(200, jobs)):
|
||||
result = mcp_actions.download_status({"base_url": "http://127.0.0.1:8188"})
|
||||
self.assertEqual(result["downloads"][0]["percent"], 25)
|
||||
self.assertEqual(result["downloads"][0]["file"], "Flux/a.safetensors")
|
||||
|
||||
def test_download_model_needs_a_source_and_goes_through_the_page(self):
|
||||
with self.assertRaises(mcp_tools.ToolError):
|
||||
mcp_actions.download_model({}, "loras")
|
||||
with mock.patch.object(mcp_actions, "ask_page", return_value={"started": True}) as ask:
|
||||
mcp_actions.download_model({}, "loras", url="https://civitai.com/models/1?modelVersionId=2")
|
||||
action, args, _timeout = ask.call_args.args
|
||||
self.assertEqual((action, args["type"], args["folder"]), ("download_model", "loras", None))
|
||||
with mock.patch.object(mcp_actions, "ask_page", return_value={"started": []}) as ask:
|
||||
mcp_actions.download_missing_models({}, ["a.safetensors"])
|
||||
self.assertEqual(ask.call_args.args[:2], ("download_missing", {"names": ["a.safetensors"]}))
|
||||
|
||||
|
||||
class LoraHeaderTests(unittest.TestCase):
|
||||
"""add_lora wires the CLIP line only for LoRAs that train the text encoder."""
|
||||
|
||||
def lora(self, keys):
|
||||
import struct
|
||||
header = json.dumps({"__metadata__": {}, **{key: {"dtype": "F16", "shape": [1], "data_offsets": [0, 2]} for key in keys}}).encode()
|
||||
path = Path(self.temp.name) / f"l{len(keys)}_{keys[0][:8]}.safetensors"
|
||||
path.write_bytes(struct.pack("<Q", len(header)) + header + b"\0\0")
|
||||
return str(path)
|
||||
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
|
||||
def tearDown(self):
|
||||
self.temp.cleanup()
|
||||
|
||||
def test_text_encoder_weights_are_told_from_the_header(self):
|
||||
self.assertTrue(lora_info.trains_text_encoder(self.lora(["lora_te_text_model.x.alpha", "lora_unet_a.alpha"])))
|
||||
self.assertTrue(lora_info.trains_text_encoder(self.lora(["lora_te1_x.alpha", "lora_te2_x.alpha"])))
|
||||
self.assertFalse(lora_info.trains_text_encoder(self.lora(["lora_unet_double_blocks.alpha"]))) # Flux
|
||||
self.assertFalse(lora_info.trains_text_encoder(self.lora(["diffusion_model.blocks.0.lora_A.weight"]))) # Anima
|
||||
self.assertTrue(lora_info.trains_text_encoder(str(Path(self.temp.name) / "old.ckpt"))) # unknown: assume it does
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,455 @@
|
||||
"""Model Check's downloads: finding the one exact file, where it may go, and fetching it safely.
|
||||
No network: every site answer is faked; files go to a temporary folder."""
|
||||
import asyncio
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
import urllib.error
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from aiohttp.test_utils import make_mocked_request
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
from api import download_sources, hf_card, model_download # noqa: E402
|
||||
|
||||
DATA = b"model bytes " * 1000
|
||||
SHA = hashlib.sha256(DATA).hexdigest()
|
||||
|
||||
|
||||
def civitai_version(sha=SHA, name="cool.safetensors"):
|
||||
return {
|
||||
"id": 456, "modelId": 123, "name": "v1", "baseModel": "SDXL 1.0", "model": {"name": "Cool", "nsfw": False},
|
||||
"files": [
|
||||
{"name": "other.pt", "sizeKB": 1, "downloadUrl": "https://civitai.com/api/download/models/456?type=Training",
|
||||
"hashes": {"SHA256": "AB" * 32}},
|
||||
{"name": name, "sizeKB": len(DATA) / 1024, "primary": True, "downloadUrl": "https://civitai.com/api/download/models/456",
|
||||
"hashes": {"SHA256": sha.upper(), "AutoV2": sha[:10].upper()}, "metadata": {"format": "SafeTensor"}},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class SourceTests(unittest.TestCase):
|
||||
def test_civitai_by_hash_picks_the_file_with_that_hash(self):
|
||||
with mock.patch.object(download_sources, "_get_json", return_value=civitai_version()):
|
||||
found = download_sources.find_source(SHA[:10]) # AutoV2
|
||||
self.assertTrue(found["found"])
|
||||
self.assertEqual(found["download_url"], "https://civitai.com/api/download/models/456")
|
||||
self.assertEqual(found["sha256"], SHA)
|
||||
self.assertEqual(found["base_model"], "SDXL 1.0")
|
||||
self.assertEqual(found["page"], "https://civitai.com/models/123?modelVersionId=456")
|
||||
|
||||
def test_a_version_without_that_hash_is_no_source(self):
|
||||
with mock.patch.object(download_sources, "_get_json", return_value=civitai_version(sha="cd" * 32)):
|
||||
self.assertFalse(download_sources.find_source(SHA)["found"])
|
||||
|
||||
def test_a_civitai_link_needs_a_version_and_takes_the_file_of_that_name(self):
|
||||
asked = []
|
||||
def fake(url, headers):
|
||||
asked.append(url)
|
||||
return civitai_version()
|
||||
with mock.patch.object(download_sources, "_get_json", side_effect=fake):
|
||||
found = download_sources.find_source(url="https://civitai.com/models/123?modelVersionId=456", name="cool.safetensors")
|
||||
self.assertTrue(found["found"])
|
||||
self.assertTrue(asked[-1].endswith("/model-versions/456"))
|
||||
# A model page without a version could mean any version: not a source.
|
||||
self.assertFalse(download_sources.find_source(url="https://civitai.com/models/123", name="cool.safetensors")["found"])
|
||||
# A renamed file: the version's primary model file (the hash, when there is one, still decides).
|
||||
renamed = download_sources.find_source(url="https://civitai.com/models/123?modelVersionId=456", name="renamed.safetensors")
|
||||
self.assertEqual(renamed["file_name"], "cool.safetensors")
|
||||
self.assertFalse(download_sources.find_source(url="https://civitai.com/models/123?modelVersionId=456",
|
||||
file_hash="cd" * 32)["found"])
|
||||
|
||||
def test_a_file_name_alone_is_never_a_source(self):
|
||||
self.assertEqual(download_sources.find_source("", "", "cool.safetensors"), {"found": False, "reason": "no_source"})
|
||||
|
||||
def test_hugging_face_links_become_file_links_with_size_and_hash(self):
|
||||
self.assertEqual(download_sources._hf_file_url("https://huggingface.co/org/repo/blob/main/split/vae/ae.safetensors?download=true"),
|
||||
"https://huggingface.co/org/repo/resolve/main/split/vae/ae.safetensors")
|
||||
self.assertEqual(download_sources._hf_file_url("https://huggingface.co/org/repo"), "")
|
||||
headers = {"x-linked-size": str(len(DATA)), "x-linked-etag": f'"{SHA}"'}
|
||||
redirect = urllib.error.HTTPError("u", 302, "Found", headers, None)
|
||||
opener = mock.Mock()
|
||||
opener.open.side_effect = redirect
|
||||
with mock.patch.object(download_sources.urllib.request, "build_opener", return_value=opener):
|
||||
found = download_sources.find_source(url="https://hf-mirror.com/org/repo/resolve/main/ae.safetensors")
|
||||
self.assertEqual((found["source"], found["size"], found["sha256"]), ("huggingface", len(DATA), SHA))
|
||||
|
||||
def test_the_mirror_setting_picks_the_host_and_follows_its_redirect(self):
|
||||
asked = []
|
||||
def head(url):
|
||||
asked.append(url)
|
||||
if "hf-mirror.com" in url: # outside China the mirror sends you back
|
||||
return {"Location": url.replace("hf-mirror.com", "huggingface.co")}, True
|
||||
return {"x-linked-size": str(len(DATA)), "x-linked-etag": f'"{SHA}"'}, True
|
||||
link = "https://huggingface.co/org/repo/blob/main/ae.safetensors"
|
||||
with mock.patch.object(download_sources, "_head", side_effect=head):
|
||||
found = download_sources.find_source(url=link, hf_mirror=True)
|
||||
self.assertEqual(found["download_url"], "https://hf-mirror.com/org/repo/resolve/main/ae.safetensors")
|
||||
self.assertTrue(found["mirror"])
|
||||
self.assertEqual((found["size"], found["sha256"]), (len(DATA), SHA))
|
||||
self.assertEqual(len(asked), 2)
|
||||
# Without the mirror, a mirror link is fetched from Hugging Face itself.
|
||||
found = download_sources.find_source(url=link.replace("huggingface.co", "hf-mirror.com"))
|
||||
self.assertEqual(found["download_url"], "https://huggingface.co/org/repo/resolve/main/ae.safetensors")
|
||||
self.assertFalse(found["mirror"])
|
||||
|
||||
def test_only_https_links_on_known_sites_may_be_downloaded(self):
|
||||
allowed = download_sources.allowed_download_url
|
||||
self.assertTrue(allowed("https://civitai.com/api/download/models/1"))
|
||||
self.assertTrue(allowed("https://huggingface.co/a/b/resolve/main/x.safetensors"))
|
||||
self.assertFalse(allowed("http://civitai.com/api/download/models/1"))
|
||||
self.assertFalse(allowed("https://civitai.com.evil.example/x"))
|
||||
self.assertFalse(allowed("https://example.com/x.safetensors"))
|
||||
self.assertFalse(allowed("file:///etc/passwd"))
|
||||
|
||||
def test_network_failure_is_told_apart(self):
|
||||
with mock.patch.object(download_sources, "_get_json", side_effect=download_sources.SourceUnreachable("down")):
|
||||
self.assertEqual(download_sources.find_source(SHA)["reason"], "network")
|
||||
|
||||
def test_lookup_refuses_a_link_to_another_file(self):
|
||||
with mock.patch.object(model_download, "find_source", return_value={"found": True, "sha256": "cd" * 32, "page": "p"}):
|
||||
self.assertEqual(model_download.lookup_item({"hash": SHA, "url": "https://huggingface.co/a/b/resolve/main/x.safetensors"})["reason"],
|
||||
"different_file")
|
||||
|
||||
|
||||
class FakeResponse(io.BytesIO):
|
||||
def __init__(self, data, status=200, headers=None):
|
||||
super().__init__(data)
|
||||
self.status = status
|
||||
self.headers = {"Content-Length": str(len(data)), "Content-Type": "application/octet-stream", **(headers or {})}
|
||||
|
||||
|
||||
class DownloadTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.root = os.path.realpath(self.temp.name)
|
||||
self.patches = [
|
||||
mock.patch.object(model_download.folder_paths, "folder_names_and_paths", {"loras": ([self.root], set())}),
|
||||
mock.patch.object(model_download.folder_paths, "get_folder_paths", side_effect=lambda t: [self.root] if t == "loras" else []),
|
||||
mock.patch.object(model_download, "add_entry"),
|
||||
mock.patch.object(model_download, "settings_path", return_value=os.path.join(self.root, "settings.json")),
|
||||
]
|
||||
for patch in self.patches:
|
||||
patch.start()
|
||||
self.add_entry = model_download.add_entry
|
||||
|
||||
def tearDown(self):
|
||||
for patch in reversed(self.patches):
|
||||
patch.stop()
|
||||
self.temp.cleanup()
|
||||
|
||||
def job(self, **changes):
|
||||
data = {"download_url": "https://civitai.com/api/download/models/456", "type": "loras", "root": 0,
|
||||
"rel": "Flux/cool.safetensors", "sha256": SHA, "size": len(DATA), "value": "Flux\\cool.safetensors", "source": "civitai"}
|
||||
data.update(changes)
|
||||
return model_download.plan_job(data)
|
||||
|
||||
def run_with(self, job, responses):
|
||||
calls = []
|
||||
def fake(request, timeout=0):
|
||||
calls.append(request)
|
||||
answer = responses.pop(0)
|
||||
if isinstance(answer, Exception):
|
||||
raise answer
|
||||
return answer
|
||||
with mock.patch.object(model_download.urllib.request, "urlopen", side_effect=fake):
|
||||
model_download.run_job(job)
|
||||
return calls
|
||||
|
||||
def test_where_a_file_may_go(self):
|
||||
refused = lambda **c: self.assertRaises((model_download.DownloadError, ValueError), self.job, **c)
|
||||
refused(rel="../outside.safetensors")
|
||||
refused(rel="C:/Windows/x.safetensors")
|
||||
refused(rel="Flux/cool.txt")
|
||||
refused(download_url="https://example.com/cool.safetensors")
|
||||
refused(type="checkpoints")
|
||||
refused(root=3)
|
||||
os.makedirs(os.path.join(self.root, "Flux"))
|
||||
Path(self.root, "Flux", "cool.safetensors").write_bytes(b"mine")
|
||||
with self.assertRaises(model_download.DownloadError) as caught:
|
||||
self.job()
|
||||
self.assertEqual(caught.exception.code, "exists")
|
||||
with mock.patch.object(model_download.shutil, "disk_usage", return_value=mock.Mock(free=10)):
|
||||
with self.assertRaises(model_download.DownloadError) as caught:
|
||||
self.job(rel="other.safetensors")
|
||||
self.assertEqual(caught.exception.code, "no_space")
|
||||
|
||||
def test_a_download_is_checked_then_named_and_recorded(self):
|
||||
job = self.job()
|
||||
self.run_with(job, [FakeResponse(DATA)])
|
||||
self.assertEqual((job["state"], job["error"], job["verified"]), ("done", "", True))
|
||||
dest = Path(self.root, "Flux", "cool.safetensors")
|
||||
self.assertEqual(dest.read_bytes(), DATA)
|
||||
self.assertFalse(Path(str(dest) + ".part").exists())
|
||||
source, action, target, detail = self.add_entry.call_args.args
|
||||
self.assertEqual((source, action, target), ("file", "model_download", "Flux/cool.safetensors"))
|
||||
self.assertEqual(detail["download"]["sha256"], SHA)
|
||||
|
||||
def test_a_wrong_file_never_lands(self):
|
||||
job = self.job(sha256="cd" * 32)
|
||||
self.run_with(job, [FakeResponse(DATA)])
|
||||
self.assertEqual((job["state"], job["error"]), ("failed", "hash_mismatch"))
|
||||
self.assertEqual(os.listdir(os.path.join(self.root, "Flux")), [])
|
||||
self.add_entry.assert_not_called()
|
||||
|
||||
def test_the_workflow_hash_checks_when_the_site_gives_none(self):
|
||||
job = self.job(sha256="", hash=SHA[:10])
|
||||
self.run_with(job, [FakeResponse(DATA)])
|
||||
self.assertEqual((job["state"], job["verified"]), ("done", True))
|
||||
|
||||
def test_an_interrupted_download_continues(self):
|
||||
os.makedirs(os.path.join(self.root, "Flux"))
|
||||
Path(self.root, "Flux", "cool.safetensors.part").write_bytes(DATA[:5000])
|
||||
job = self.job()
|
||||
calls = self.run_with(job, [FakeResponse(DATA[5000:], status=206)])
|
||||
self.assertEqual(calls[0].get_header("Range"), "bytes=5000-")
|
||||
self.assertEqual(job["state"], "done")
|
||||
self.assertEqual(Path(self.root, "Flux", "cool.safetensors").read_bytes(), DATA)
|
||||
|
||||
def test_a_site_that_ignores_the_range_sends_it_all_again(self):
|
||||
os.makedirs(os.path.join(self.root, "Flux"))
|
||||
Path(self.root, "Flux", "cool.safetensors.part").write_bytes(b"junk")
|
||||
job = self.job()
|
||||
self.run_with(job, [FakeResponse(DATA, status=200)])
|
||||
self.assertEqual(Path(self.root, "Flux", "cool.safetensors").read_bytes(), DATA)
|
||||
|
||||
def test_a_login_page_is_not_a_model(self):
|
||||
job = self.job()
|
||||
self.run_with(job, [FakeResponse(b"<html>", headers={"Content-Type": "text/html; charset=utf-8"})])
|
||||
self.assertEqual((job["state"], job["error"]), ("failed", "not_a_file"))
|
||||
self.assertFalse(Path(self.root, "Flux", "cool.safetensors").exists())
|
||||
|
||||
def test_civitai_login_needed(self):
|
||||
job = self.job()
|
||||
with mock.patch.object(model_download, "_load_api_key", return_value=None):
|
||||
self.run_with(job, [urllib.error.HTTPError("u", 401, "no", {}, None)])
|
||||
self.assertEqual(job["error"], "needs_key")
|
||||
|
||||
def test_the_key_goes_to_civitai_only_and_not_along_redirects(self):
|
||||
with mock.patch.object(model_download, "_load_api_key", return_value="secret"):
|
||||
calls = self.run_with(self.job(), [FakeResponse(DATA)])
|
||||
self.assertEqual(calls[0].unredirected_hdrs.get("Authorization"), "Bearer secret")
|
||||
self.assertNotIn("Authorization", calls[0].headers)
|
||||
calls = self.run_with(self.job(rel="b.safetensors", download_url="https://huggingface.co/a/b/resolve/main/b.safetensors"),
|
||||
[FakeResponse(DATA)])
|
||||
self.assertIsNone(calls[0].unredirected_hdrs.get("Authorization"))
|
||||
|
||||
def test_a_network_failure_keeps_what_came(self):
|
||||
job = self.job()
|
||||
self.run_with(job, [urllib.error.URLError("reset")])
|
||||
self.assertEqual((job["state"], job["error"]), ("failed", "network"))
|
||||
|
||||
def test_cancel_removes_the_unfinished_file(self):
|
||||
job = self.job()
|
||||
job["cancel"] = True
|
||||
self.run_with(job, [FakeResponse(DATA)])
|
||||
self.assertEqual(job["state"], "cancelled")
|
||||
self.assertFalse(Path(self.root, "Flux", "cool.safetensors.part").exists())
|
||||
|
||||
def test_settings(self):
|
||||
self.assertEqual(model_download.read_settings(), {"place": "workflow", "folder": "Downloads", "hf_mirror": None})
|
||||
def call(body):
|
||||
request = make_mocked_request("POST", "/anomalous/download/settings")
|
||||
request.json = lambda: asyncio.sleep(0, body)
|
||||
response = asyncio.run(model_download.api_settings(request))
|
||||
return response.status, json.loads(response.text)
|
||||
self.assertEqual(call({"place": "folder", "folder": "\\Models\\{base}\\", "hf_mirror": True})[1],
|
||||
{"place": "folder", "folder": "Models/{base}", "hf_mirror": True})
|
||||
self.assertEqual(call({"folder": "../up"})[0], 400)
|
||||
self.assertEqual(model_download.read_settings()["folder"], "Models/{base}")
|
||||
|
||||
def test_a_site_that_stops_answering_is_skipped_but_the_others_are_asked(self):
|
||||
civitai = []
|
||||
def offline(url, headers):
|
||||
civitai.append(url)
|
||||
raise download_sources.SourceUnreachable("down")
|
||||
def head(url):
|
||||
return {"x-linked-size": str(len(DATA)), "x-linked-etag": f'"{SHA}"'}, True
|
||||
listed = [{"name": "Flux VAE", "url": "https://huggingface.co/bfl/flux/resolve/main/ae.safetensors", "folder": "vae", "base": "FLUX.1"}]
|
||||
request = make_mocked_request("POST", "/anomalous/download/lookup")
|
||||
items = [{"key": str(i), "hash": SHA, "types": ["loras"], "value": "a.safetensors"} for i in range(3)]
|
||||
items.append({"key": "3", "hash": SHA, "types": ["vae"], "value": "vae\\ae.safetensors"})
|
||||
items.append({"key": "4", "types": ["loras"], "value": "b.safetensors"})
|
||||
request.json = lambda: asyncio.sleep(0, {"items": items, "hf_mirror": True})
|
||||
with mock.patch.object(download_sources, "_get_json", side_effect=offline), \
|
||||
mock.patch.object(download_sources, "_head", side_effect=head), \
|
||||
mock.patch.object(download_sources.manager_catalog, "candidates",
|
||||
side_effect=lambda name, folder="": listed if name == "ae.safetensors" else []):
|
||||
body = json.loads(asyncio.run(model_download.api_lookup(request)).text)
|
||||
self.assertEqual(len(civitai), 1) # Civitai asked once, not once per model
|
||||
reasons = [r.get("reason") for r in body["results"]]
|
||||
self.assertEqual(reasons[:3], ["network"] * 3)
|
||||
self.assertTrue(body["results"][3]["found"]) # Hugging Face (mirror) through the Manager's list
|
||||
self.assertEqual(body["results"][3]["download_url"], "https://hf-mirror.com/bfl/flux/resolve/main/ae.safetensors")
|
||||
self.assertEqual(reasons[4], "no_source")
|
||||
self.assertEqual(body["roots"]["loras"][0]["path"], self.root)
|
||||
|
||||
def test_one_at_a_time_through_the_queue(self):
|
||||
order = []
|
||||
def fake_run(job):
|
||||
order.append(job["rel"])
|
||||
job["finished"] = time.time()
|
||||
with mock.patch.object(model_download, "run_job", side_effect=fake_run):
|
||||
model_download.enqueue(self.job(rel="a.safetensors"))
|
||||
model_download.enqueue(self.job(rel="b.safetensors"))
|
||||
for _ in range(100):
|
||||
if model_download._worker is None:
|
||||
break
|
||||
time.sleep(0.01)
|
||||
self.assertEqual(order, ["a.safetensors", "b.safetensors"])
|
||||
|
||||
|
||||
class ManagerListTests(unittest.TestCase):
|
||||
"""ComfyUI-Manager's model list as a source by file name, checked by the fingerprint."""
|
||||
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
path = os.path.join(self.temp.name, "model-list.json")
|
||||
hf = "https://huggingface.co/{}/resolve/main/{}"
|
||||
models = [
|
||||
{"name": "FLUX VAE", "type": "VAE", "save_path": "vae/FLUX1", "filename": "ae.safetensors", "url": hf.format("bfl/one", "ae.safetensors"), "base": "FLUX.1"},
|
||||
{"name": "CN A", "type": "controlnet", "save_path": "controlnet/SDXL", "filename": "diffusion_pytorch_model.safetensors", "url": hf.format("a/cn", "diffusion_pytorch_model.safetensors")},
|
||||
{"name": "CN B", "type": "controlnet", "save_path": "controlnet/SDXL", "filename": "diffusion_pytorch_model.safetensors", "url": hf.format("b/cn", "diffusion_pytorch_model.safetensors")},
|
||||
{"name": "UNet", "type": "diffusion_model", "save_path": "diffusion_models/x", "filename": "diffusion_pytorch_model.safetensors", "url": hf.format("c/unet", "diffusion_pytorch_model.safetensors")},
|
||||
{"name": "Upscaler", "type": "upscale", "save_path": "default", "filename": "4x-UltraSharp.pth", "url": hf.format("k/u", "4x-UltraSharp.pth")},
|
||||
{"name": "Elsewhere", "type": "VAE", "save_path": "default", "filename": "x.pth", "url": "http://example.com/x.pth"},
|
||||
]
|
||||
Path(path).write_text(json.dumps({"models": models}), encoding="utf-8")
|
||||
self.patch = mock.patch.object(download_sources.manager_catalog, "list_path", return_value=path)
|
||||
self.patch.start()
|
||||
self.civitai = mock.patch.object(download_sources, "_get_json", return_value=None) # Civitai: not known
|
||||
self.civitai.start()
|
||||
self.addCleanup(self.civitai.stop)
|
||||
self.shas = {"bfl/one": SHA, "a/cn": "aa" * 32, "b/cn": SHA, "c/unet": SHA, "k/u": "bb" * 32}
|
||||
|
||||
def tearDown(self):
|
||||
self.patch.stop()
|
||||
self.temp.cleanup()
|
||||
|
||||
def head(self, url):
|
||||
repo = "/".join(url.split("/")[3:5])
|
||||
return {"x-linked-size": "10", "x-linked-etag": f'"{self.shas[repo]}"'}, True
|
||||
|
||||
def test_candidates_fit_the_folder_type(self):
|
||||
candidates = download_sources.manager_catalog.candidates
|
||||
self.assertEqual([c["name"] for c in candidates("DIFFUSION_PYTORCH_MODEL.safetensors", "controlnet")], ["CN A", "CN B"])
|
||||
self.assertEqual([c["name"] for c in candidates("diffusion_pytorch_model.safetensors", "unet")], ["UNet"])
|
||||
self.assertEqual([c["name"] for c in candidates("Sub/4x-ultrasharp.pth", "upscale_models")], ["Upscaler"])
|
||||
self.assertEqual(candidates("x.pth"), []) # not https
|
||||
self.assertEqual(candidates("ae.safetensors", "loras"), [])
|
||||
|
||||
def test_with_a_fingerprint_only_the_matching_file(self):
|
||||
with mock.patch.object(download_sources, "_head", side_effect=self.head):
|
||||
found = download_sources.find_source(SHA, name="diffusion_pytorch_model.safetensors", folder_type="controlnet")
|
||||
self.assertEqual((found["via"], found["list_name"], found["by_name"]), ("manager_list", "CN B", False))
|
||||
self.assertFalse(download_sources.find_source("cc" * 32, name="ae.safetensors", folder_type="vae")["found"])
|
||||
|
||||
def test_without_one_only_a_single_entry_and_marked(self):
|
||||
with mock.patch.object(download_sources, "_head", side_effect=self.head):
|
||||
found = download_sources.find_source(name="ae.safetensors", folder_type="vae")
|
||||
self.assertTrue(found["found"] and found["by_name"])
|
||||
self.assertEqual(found["base_model"], "FLUX.1")
|
||||
# Two different files of that name: not offered.
|
||||
self.assertFalse(download_sources.find_source(name="diffusion_pytorch_model.safetensors", folder_type="controlnet")["found"])
|
||||
|
||||
|
||||
class HuggingFaceFirstTests(unittest.TestCase):
|
||||
"""With the mirror on, the same file on Hugging Face is fetched there instead of Civitai."""
|
||||
|
||||
def test_same_file_comes_from_hugging_face_with_the_mirror(self):
|
||||
listed = [{"name": "T5", "url": "https://huggingface.co/c/t5/resolve/main/t5.safetensors", "folder": "text_encoders", "base": ""}]
|
||||
head = lambda url: ({"x-linked-size": str(len(DATA)), "x-linked-etag": f'"{SHA}"'}, True)
|
||||
with mock.patch.object(download_sources, "_get_json", return_value=civitai_version(name="t5.safetensors")), \
|
||||
mock.patch.object(download_sources, "_head", side_effect=head), \
|
||||
mock.patch.object(download_sources.manager_catalog, "candidates", return_value=listed):
|
||||
found = download_sources.find_source(SHA, name="t5.safetensors", hf_mirror=True, folder_type="text_encoders")
|
||||
self.assertEqual(found["download_url"], "https://hf-mirror.com/c/t5/resolve/main/t5.safetensors")
|
||||
self.assertEqual((found["also_on"], found["base_model"], found["page"]),
|
||||
("civitai", "SDXL 1.0", "https://civitai.com/models/123?modelVersionId=456"))
|
||||
# Without the mirror, Civitai as before.
|
||||
self.assertEqual(download_sources.find_source(SHA, name="t5.safetensors", folder_type="text_encoders")["source"], "civitai")
|
||||
other = lambda url: ({"x-linked-size": "1", "x-linked-etag": f'"{"dd" * 32}"'}, True)
|
||||
with mock.patch.object(download_sources, "_get_json", return_value=civitai_version(name="t5.safetensors")), \
|
||||
mock.patch.object(download_sources, "_head", side_effect=other), \
|
||||
mock.patch.object(download_sources.manager_catalog, "candidates", return_value=listed):
|
||||
# A different file on Hugging Face: Civitai's stays.
|
||||
self.assertEqual(download_sources.find_source(SHA, name="t5.safetensors", hf_mirror=True)["source"], "civitai")
|
||||
|
||||
|
||||
class ModelCardTests(unittest.TestCase):
|
||||
"""A file downloaded from Hugging Face gets its model card's picture and words."""
|
||||
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.model = os.path.join(self.temp.name, "style.safetensors")
|
||||
Path(self.model).write_bytes(b"x")
|
||||
self.base = os.path.splitext(self.model)[0]
|
||||
|
||||
def tearDown(self):
|
||||
self.temp.cleanup()
|
||||
|
||||
def fake_get(self, url, limit=None):
|
||||
if "/api/models/" in url:
|
||||
card = {"cardData": {"base_model": "black-forest-labs/FLUX.1-dev", "instance_prompt": "frstingln",
|
||||
"widget": [{"text": "a frosted cake", "output": {"url": "images/a b.png"}}]},
|
||||
"siblings": [{"rfilename": "images/a b.png"}, {"rfilename": "style.safetensors"}]}
|
||||
return json.dumps(card).encode(), "application/json"
|
||||
self.asked_image = url
|
||||
return b"\x89PNG picture", "image/png"
|
||||
|
||||
def test_cover_and_notes_from_the_card(self):
|
||||
with mock.patch.object(hf_card, "_get", side_effect=self.fake_get):
|
||||
result = hf_card.enrich(self.model, "https://hf-mirror.com/alv/frost/resolve/main/style.safetensors")
|
||||
self.assertEqual(result, {"cover": True, "notes": True})
|
||||
self.assertEqual(self.asked_image, "https://hf-mirror.com/alv/frost/resolve/main/images/a%20b.png") # same host
|
||||
self.assertTrue(os.path.exists(self.base + ".preview.png"))
|
||||
user = json.loads(Path(self.base + ".anomalous.json").read_text(encoding="utf-8"))
|
||||
self.assertIn("Trigger words: frstingln", user["custom_notes"])
|
||||
self.assertIn("- a frosted cake", user["custom_notes"])
|
||||
self.assertEqual(user["source_url"], "https://huggingface.co/alv/frost")
|
||||
|
||||
def test_nothing_the_user_has_is_replaced(self):
|
||||
Path(self.base + ".preview.jpg").write_bytes(b"mine")
|
||||
Path(self.base + ".anomalous.json").write_text(json.dumps({"format": 1, "custom_notes": "my notes"}), encoding="utf-8")
|
||||
with mock.patch.object(hf_card, "_get", side_effect=self.fake_get):
|
||||
result = hf_card.enrich(self.model, "https://huggingface.co/alv/frost/resolve/main/style.safetensors")
|
||||
self.assertEqual(result["cover"], False)
|
||||
self.assertFalse(os.path.exists(self.base + ".preview.png"))
|
||||
user = json.loads(Path(self.base + ".anomalous.json").read_text(encoding="utf-8"))
|
||||
self.assertEqual(user["custom_notes"], "my notes") # kept; only the empty link is filled
|
||||
self.assertEqual(user["source_url"], "https://huggingface.co/alv/frost")
|
||||
|
||||
def test_not_a_hugging_face_file(self):
|
||||
self.assertEqual(hf_card.enrich(self.model, "https://civitai.com/api/download/models/1"), {"cover": False, "notes": False})
|
||||
|
||||
|
||||
class FolderDefaultsTests(unittest.TestCase):
|
||||
def test_older_configs_get_the_new_defaults_once(self):
|
||||
from api import folder_types
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
path = os.path.join(temp, "config.json")
|
||||
Path(path).write_text(json.dumps({"CIVITAI_API_KEY": "k", "folder_types_config": [
|
||||
{"type": "loras", "visible": True}, {"type": "text_encoders", "visible": False},
|
||||
{"type": "upscale_models", "visible": False}, {"type": "embeddings", "visible": False}]}), encoding="utf-8")
|
||||
self.assertTrue(folder_types.upgrade_folder_defaults(path))
|
||||
cfg = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||
shown = {item["type"]: item["visible"] for item in cfg["folder_types_config"]}
|
||||
self.assertEqual(shown, {"loras": True, "text_encoders": True, "upscale_models": True, "embeddings": False})
|
||||
self.assertEqual((cfg["CIVITAI_API_KEY"], cfg["folder_defaults_version"]), ("k", 2))
|
||||
# Hidden again by the user afterwards: stays hidden.
|
||||
cfg["folder_types_config"][1]["visible"] = False
|
||||
Path(path).write_text(json.dumps(cfg), encoding="utf-8")
|
||||
self.assertFalse(folder_types.upgrade_folder_defaults(path))
|
||||
self.assertFalse(folder_types.upgrade_folder_defaults(os.path.join(temp, "none.json")))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,128 @@
|
||||
"""The user's layer comes first: editor edits, covers and what a scan may touch."""
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
import scraper
|
||||
from api import metadata as metadata_module, model_metadata, scan_summary
|
||||
from api.metadata import get_metadata
|
||||
|
||||
CIVITAI = {"id": 7, "modelId": 3, "name": "v2", "model": {"name": "Found"},
|
||||
"images": [{"url": "https://example.invalid/cover.png"}]}
|
||||
|
||||
|
||||
def write(path, data):
|
||||
Path(path).write_text(json.dumps(data), encoding="utf-8")
|
||||
|
||||
|
||||
class Request(dict): # aiohttp requests also hold values for the middleware
|
||||
def __init__(self, body):
|
||||
super().__init__()
|
||||
self.body = body
|
||||
|
||||
async def json(self):
|
||||
return self.body
|
||||
|
||||
|
||||
class LayerTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.dir = Path(self.temp.name)
|
||||
self.model = self.dir / "m.safetensors"
|
||||
self.model.write_bytes(b"model")
|
||||
self.base = str(self.dir / "m")
|
||||
metadata_module.clear_metadata_cache()
|
||||
|
||||
def tearDown(self):
|
||||
self.temp.cleanup()
|
||||
|
||||
def scan(self, *flags, cover=b"new civitai image"):
|
||||
"""Runs the scraper with Civitai and downloads faked; returns what went to the network."""
|
||||
calls = {"fetch": 0, "download": 0}
|
||||
|
||||
def fetch(_hash):
|
||||
calls["fetch"] += 1
|
||||
return json.loads(json.dumps(CIVITAI))
|
||||
|
||||
def download(_url, base_path):
|
||||
calls["download"] += 1
|
||||
Path(base_path + ".png").write_bytes(cover)
|
||||
return base_path + ".png"
|
||||
|
||||
with mock.patch.object(sys, "argv", ["scraper.py", str(self.dir), *flags]), \
|
||||
mock.patch.object(scraper, "fetch_civitai_info", side_effect=fetch), \
|
||||
mock.patch.object(scraper, "download_media", side_effect=download), \
|
||||
mock.patch.object(scraper, "calculate_sha256", side_effect=lambda p: "b" * 64 if p == str(self.model) else open(p, "rb").read().hex()), \
|
||||
mock.patch.object(scraper.urllib.request, "urlopen", side_effect=OSError("no network in tests")):
|
||||
scraper.main()
|
||||
metadata_module.clear_metadata_cache()
|
||||
return calls
|
||||
|
||||
def test_an_edited_but_unscanned_model_still_counts_as_new_and_gets_scanned(self):
|
||||
write(self.base + ".civitai.info", {"anomalous_custom_name": "Mine"}) # how older versions saved edits
|
||||
with mock.patch.object(scan_summary, "get_active_scan_paths", return_value=[str(self.dir)]):
|
||||
self.assertEqual(scan_summary._summarize()["new"], 1)
|
||||
self.assertEqual(self.scan("--virtual-rename")["fetch"], 1)
|
||||
meta = get_metadata(str(self.model))
|
||||
self.assertEqual(meta["info_source"], "civitai")
|
||||
self.assertEqual(meta["custom_name"], "Mine") # the old edit still wins over Civitai's name
|
||||
|
||||
def test_user_layer_wins_and_an_emptied_field_stays_empty(self):
|
||||
write(self.base + ".info", {"id": 7, "modelId": 3, "name": "v2", "anomalous_custom_name": "Found_v2"})
|
||||
write(self.base + ".anomalous.json", {"format": 1, "custom_name": "Mine", "custom_notes": ""})
|
||||
meta = get_metadata(str(self.model))
|
||||
self.assertEqual(meta["custom_name"], "Mine")
|
||||
self.assertEqual(meta["custom_notes"], "")
|
||||
self.assertEqual(sorted(meta["user_fields"]), ["custom_name", "custom_notes"])
|
||||
self.assertEqual(meta["info_source"], "civitai")
|
||||
|
||||
def test_info_source_tells_local_inference_and_unscanned_apart(self):
|
||||
self.assertEqual(get_metadata(str(self.model))["info_source"], "")
|
||||
write(self.base + ".info", {"id": -1, "modelId": -1, "name": "m"})
|
||||
metadata_module.clear_metadata_cache()
|
||||
self.assertEqual(get_metadata(str(self.model))["info_source"], "local")
|
||||
|
||||
def test_editor_writes_only_the_user_layer(self):
|
||||
write(self.base + ".civitai.info", {"id": 7, "modelId": 3, "name": "from another tool"})
|
||||
before = Path(self.base + ".civitai.info").read_text(encoding="utf-8")
|
||||
with mock.patch.object(model_metadata, "resolve_folder_subdir", return_value=(str(self.dir), str(self.dir))):
|
||||
asyncio.run(model_metadata.api_update_metadata(Request({
|
||||
"type": "loras", "filename": "m.safetensors", "custom_name": "Mine", "custom_notes": "note"})))
|
||||
self.assertEqual(Path(self.base + ".civitai.info").read_text(encoding="utf-8"), before)
|
||||
user = json.loads(Path(self.base + ".anomalous.json").read_text(encoding="utf-8"))
|
||||
self.assertEqual((user["custom_name"], user["custom_notes"]), ("Mine", "note"))
|
||||
|
||||
def test_a_scan_does_not_refetch_what_is_on_disk(self):
|
||||
write(self.base + ".info", {**CIVITAI, "anomalous_custom_name": "Found_v2"})
|
||||
Path(self.base + ".civitai_bak.png").write_bytes(b"old civitai image")
|
||||
Path(self.base + ".preview.png").write_bytes(b"old civitai image")
|
||||
self.assertEqual(self.scan("--virtual-rename"), {"fetch": 0, "download": 0})
|
||||
|
||||
def test_fetching_again_keeps_the_users_cover(self):
|
||||
write(self.base + ".info", CIVITAI)
|
||||
Path(self.base + ".civitai_bak.png").write_bytes(b"old civitai image")
|
||||
Path(self.base + ".preview.png").write_bytes(b"my own picture")
|
||||
self.scan("--force-overwrite", "--virtual-rename")
|
||||
self.assertEqual(Path(self.base + ".preview.png").read_bytes(), b"my own picture")
|
||||
self.assertEqual(Path(self.base + ".civitai_bak.png").read_bytes(), b"new civitai image")
|
||||
|
||||
def test_fetching_again_replaces_an_unchanged_civitai_cover(self):
|
||||
write(self.base + ".info", CIVITAI)
|
||||
Path(self.base + ".civitai_bak.png").write_bytes(b"old civitai image")
|
||||
Path(self.base + ".preview.png").write_bytes(b"old civitai image")
|
||||
with mock.patch.object(scraper, "move_to_trash") as trash:
|
||||
self.scan("--force-overwrite")
|
||||
trash.assert_not_called() # same file name: overwritten in place, nothing to move
|
||||
self.assertEqual(Path(self.base + ".preview.png").read_bytes(), b"new civitai image")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,115 @@
|
||||
"""The plugin's proxy (net_proxy.py): which addresses are taken, that requests really go through
|
||||
the saved proxy (a proxy on this computer answers them; nothing leaves it), that the scan process
|
||||
gets it, and the names of failures the pages explain."""
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import ssl
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
import net_proxy # noqa: E402
|
||||
|
||||
|
||||
class Proxy(http.server.BaseHTTPRequestHandler):
|
||||
"""Answers every request with the URL it was asked for, as a forwarding proxy sees it."""
|
||||
seen = []
|
||||
|
||||
def do_GET(self):
|
||||
Proxy.seen.append(self.path)
|
||||
body = json.dumps({"asked": self.path}).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
|
||||
class NetProxyTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.config = os.path.join(self.temp.name, "config.json")
|
||||
patcher = mock.patch.object(net_proxy, "CONFIG_PATH", self.config)
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
|
||||
def save(self, value):
|
||||
with open(self.config, "w", encoding="utf-8") as f:
|
||||
json.dump({"CIVITAI_API_KEY": "kept", net_proxy.CONFIG_KEY: value}, f)
|
||||
|
||||
def test_addresses(self):
|
||||
self.assertEqual(net_proxy.normalize(""), "")
|
||||
self.assertEqual(net_proxy.normalize(" 127.0.0.1:7890 "), "http://127.0.0.1:7890")
|
||||
self.assertEqual(net_proxy.normalize("HTTP://localhost:10809/"), "http://localhost:10809")
|
||||
self.assertEqual(net_proxy.normalize("https://proxy.lan:8443"), "https://proxy.lan:8443")
|
||||
self.assertEqual(net_proxy.normalize("[::1]:7897"), "http://[::1]:7897")
|
||||
for bad, why in (("socks5://127.0.0.1:7891", "socks"), ("127.0.0.1", "format"), ("127.0.0.1:0", "format"),
|
||||
("127.0.0.1:70000", "format"), ("http://a b:1", "format"), ("http://u:p@host:1", "format")):
|
||||
with self.assertRaises(ValueError) as caught:
|
||||
net_proxy.normalize(bad)
|
||||
self.assertEqual(str(caught.exception), why, bad)
|
||||
|
||||
def test_saved_address_is_read_and_a_bad_one_ignored(self):
|
||||
self.assertEqual(net_proxy.configured(), "") # no file
|
||||
self.save("127.0.0.1:7890")
|
||||
self.assertEqual(net_proxy.configured(), "http://127.0.0.1:7890")
|
||||
self.save("socks5://127.0.0.1:7891")
|
||||
self.assertEqual(net_proxy.configured(), "")
|
||||
|
||||
def test_requests_go_through_the_saved_proxy(self):
|
||||
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Proxy)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
self.addCleanup(server.server_close)
|
||||
self.addCleanup(server.shutdown) # runs first
|
||||
self.save(f"127.0.0.1:{server.server_address[1]}")
|
||||
Proxy.seen.clear()
|
||||
with net_proxy.urlopen(urllib.request.Request("http://civitai.invalid/api/v1/models?limit=1"), timeout=5) as response:
|
||||
self.assertEqual(json.loads(response.read())["asked"], "http://civitai.invalid/api/v1/models?limit=1")
|
||||
self.assertEqual(Proxy.seen, ["http://civitai.invalid/api/v1/models?limit=1"])
|
||||
|
||||
def test_the_scan_process_gets_the_proxy(self):
|
||||
with mock.patch.dict(os.environ, {"HTTPS_PROXY": ""}, clear=False):
|
||||
self.assertEqual(net_proxy.child_env().get("HTTPS_PROXY"), "") # nothing saved: unchanged
|
||||
self.save("127.0.0.1:7890")
|
||||
env = net_proxy.child_env()
|
||||
self.assertEqual((env["HTTP_PROXY"], env["HTTPS_PROXY"], env["https_proxy"]), ("http://127.0.0.1:7890",) * 3)
|
||||
|
||||
def test_a_proxy_that_is_not_running_is_named_refused(self):
|
||||
with socket.socket() as probe: # a port nothing listens on
|
||||
probe.bind(("127.0.0.1", 0))
|
||||
port = probe.getsockname()[1]
|
||||
self.save(f"127.0.0.1:{port}")
|
||||
with self.assertRaises(urllib.error.URLError) as caught:
|
||||
net_proxy.urlopen(urllib.request.Request("http://civitai.invalid/"), timeout=5)
|
||||
self.assertEqual(net_proxy.failure(caught.exception), "refused")
|
||||
|
||||
def test_failure_names(self):
|
||||
cases = [
|
||||
(urllib.error.HTTPError("u", 503, "x", {}, None), "http:503"),
|
||||
(urllib.error.URLError(socket.timeout("timed out")), "timeout"),
|
||||
(TimeoutError(), "timeout"),
|
||||
(urllib.error.URLError(socket.gaierror(11001, "getaddrinfo failed")), "dns"),
|
||||
(urllib.error.URLError(ssl.SSLError(1, "bad handshake")), "ssl"),
|
||||
(urllib.error.URLError(ConnectionResetError(10054, "reset")), "reset"),
|
||||
(urllib.error.URLError(OSError("Tunnel connection failed: 403 Forbidden")), "proxy"),
|
||||
(urllib.error.URLError("something else"), "other"),
|
||||
]
|
||||
for error, name in cases:
|
||||
self.assertEqual(net_proxy.failure(error), name, error)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,60 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import materials, node_material
|
||||
|
||||
|
||||
class NodeMaterialTests(unittest.IsolatedAsyncioTestCase):
|
||||
def setUp(self):
|
||||
temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
self.root = Path(temporary.name)
|
||||
patcher = mock.patch.object(materials, 'get_materials_dir', return_value=str(self.root))
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
self.node = {'type': 'KSampler', 'title': 'Sampler', 'widgets_values': [123, 'fixed', 30, 5.5, 'euler', 'karras', 1]}
|
||||
|
||||
async def save(self, **body):
|
||||
request = SimpleNamespace(json=mock.AsyncMock(return_value={'node': self.node, **body}))
|
||||
response = await node_material.api_save_node_material(request)
|
||||
return response.status, json.loads(response.body)
|
||||
|
||||
async def test_saved_node_reads_back_as_a_parameter_block(self):
|
||||
status, data = await self.save(name='Slow and sharp')
|
||||
self.assertEqual(status, 200)
|
||||
self.assertEqual(data['material']['kind'], 'node_parameter_selection')
|
||||
self.assertEqual(data['material']['node_types'], ['KSampler'])
|
||||
result = materials._query_materials(str(self.root), {'category': 'params'})
|
||||
self.assertEqual([item['name'] for item in result['materials']], ['Slow and sharp'])
|
||||
record = materials._read_material(self.root / data['filename'])
|
||||
blocks = materials._material_node_blocks(record, include_values=True)
|
||||
self.assertEqual([(block['type'], block['widgets_values']) for block in blocks],
|
||||
[('KSampler', self.node['widgets_values'])])
|
||||
|
||||
async def test_same_values_are_saved_once(self):
|
||||
self.assertEqual((await self.save())[0], 200)
|
||||
status, data = await self.save(name='Another name')
|
||||
self.assertEqual(status, 409)
|
||||
self.assertEqual(data['name'], 'Sampler') # the title is the default name
|
||||
self.node = {**self.node, 'widgets_values': [123, 'fixed', 31, 5.5, 'euler', 'karras', 1]}
|
||||
self.assertEqual((await self.save())[0], 200)
|
||||
|
||||
async def test_bad_input_saves_nothing(self):
|
||||
for node in [None, {'type': '', 'widgets_values': [1]}, {'type': 'KSampler', 'widgets_values': []},
|
||||
{'type': 'KSampler', 'widgets_values': 'x'}, {'type': 'K' * 201, 'widgets_values': [1]}]:
|
||||
self.node = node
|
||||
self.assertEqual((await self.save())[0], 400)
|
||||
self.node = {'type': 'KSampler', 'widgets_values': [float('nan')]}
|
||||
self.assertEqual((await self.save())[0], 400)
|
||||
self.assertEqual(list(self.root.iterdir()), [])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,61 @@
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
PLUGIN_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(PLUGIN_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(PLUGIN_ROOT))
|
||||
|
||||
from model_policies import is_physical_rename_protected, requires_hash_for_model_recovery
|
||||
|
||||
|
||||
class PhysicalRenamePolicyTests(unittest.TestCase):
|
||||
def test_protected_model_types(self):
|
||||
for folder_type in ("vae", "vae_approx", "clip", "text_encoders", "clip_vision"):
|
||||
self.assertTrue(is_physical_rename_protected(folder_type=folder_type))
|
||||
|
||||
def test_regular_model_types_remain_renameable(self):
|
||||
for folder_type in ("checkpoints", "loras", "unet", "diffusion_models", "controlnet"):
|
||||
self.assertFalse(is_physical_rename_protected(folder_type=folder_type))
|
||||
|
||||
def test_foundation_recovery_requires_hash(self):
|
||||
self.assertTrue(requires_hash_for_model_recovery(("clip", "text_encoders")))
|
||||
self.assertTrue(requires_hash_for_model_recovery(("vae",)))
|
||||
self.assertFalse(requires_hash_for_model_recovery(("checkpoints",)))
|
||||
self.assertFalse(requires_hash_for_model_recovery(()))
|
||||
|
||||
def test_standalone_scraper_path_guard_handles_nested_foundation_folders(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
protected = Path(temp_dir) / "models" / "vae" / "Anima"
|
||||
regular = Path(temp_dir) / "models" / "checkpoints" / "Anime"
|
||||
self.assertTrue(is_physical_rename_protected(folder_path=protected))
|
||||
self.assertFalse(is_physical_rename_protected(folder_path=regular))
|
||||
|
||||
def test_standalone_scraper_refuses_protected_physical_rename(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
protected = Path(temp_dir) / "models" / "vae"
|
||||
protected.mkdir(parents=True)
|
||||
result = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(PLUGIN_ROOT / "scraper.py"),
|
||||
str(protected),
|
||||
"--dry-run",
|
||||
"--physical-rename",
|
||||
"--folder-type",
|
||||
"vae",
|
||||
],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn("不执行物理重命名", result.stdout)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,89 @@
|
||||
import copy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import materials
|
||||
|
||||
|
||||
class PromptPlanTests(unittest.IsolatedAsyncioTestCase):
|
||||
def setUp(self):
|
||||
temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
self.root = Path(temporary.name)
|
||||
patcher = mock.patch.object(materials, 'get_materials_dir', return_value=str(self.root))
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
self.plan = {'parts': [
|
||||
{'name': '默认词', 'category': 'general', 'enabled': True, 'positive': ' (quality:1.2)\n', 'negative': 'bad'},
|
||||
{'name': '模型', 'category': 'specific', 'enabled': False, 'positive': 'style', 'negative': ''}],
|
||||
'positive': '本次内容', 'negative': ''}
|
||||
|
||||
async def save(self, **changes):
|
||||
return await materials.api_save_prompt_plan(SimpleNamespace(json=mock.AsyncMock(return_value={
|
||||
'name': '测试方案', 'tags': ['模型'], 'plan': self.plan, **changes})))
|
||||
|
||||
async def test_roundtrip_dedup_and_reopen(self):
|
||||
response = await self.save()
|
||||
self.assertEqual(response.status, 200)
|
||||
filename = json.loads(response.body)['filename']
|
||||
record = materials._read_material(self.root / filename)
|
||||
self.assertEqual(record['plan'], self.plan)
|
||||
self.assertNotIn('workflow', record)
|
||||
self.assertNotIn('image', record)
|
||||
self.assertEqual((await self.save(name='Same content')).status, 409)
|
||||
self.assertEqual((await self.save(allow_duplicate=True)).status, 200)
|
||||
detail = json.loads(materials._material_detail_response(self.root / filename, '0').body)
|
||||
self.assertEqual(detail['data']['plan'], self.plan)
|
||||
self.assertEqual(detail['node_blocks'], [])
|
||||
self.assertEqual(detail['workflow_hashes'], {})
|
||||
self.assertEqual(materials._material_detail_response(self.root / filename, '1').status, 403)
|
||||
self.assertEqual(materials._query_materials(str(self.root), {'kind': 'prompt_plan'})['total'], 2)
|
||||
self.assertEqual(materials._query_materials(str(self.root), {'node_type': 'CLIPTextEncode'})['total'], 0)
|
||||
|
||||
async def test_invalid_data_never_publishes(self):
|
||||
for plan in [None, [], {'parts': {}}, {'positive': 12}, {'parts': [{}]},
|
||||
{'parts': [{'category': 'general', 'enabled': 'yes'}]},
|
||||
{'parts': [{'category': 'general', 'name': 'x' * 121}]},
|
||||
{'parts': [self.plan['parts'][0]] * 101}]:
|
||||
self.assertEqual((await self.save(plan=plan)).status, 400, plan)
|
||||
for changes in [{'name': ''}, {'name': []}, {'allow_duplicate': 'yes'}, {'tags': 12}]:
|
||||
self.assertEqual((await self.save(**changes)).status, 400)
|
||||
with mock.patch.object(materials, 'MAX_NOTEBOOK_BYTES', 20):
|
||||
self.assertEqual((await self.save()).status, 400)
|
||||
with mock.patch.object(materials, '_atomic_write_json', side_effect=OSError('disk full')):
|
||||
self.assertEqual((await self.save()).status, 500)
|
||||
self.assertEqual(list(self.root.iterdir()), [])
|
||||
|
||||
async def test_snapshot_excludes_unknown_fields_and_is_independent(self):
|
||||
plan = copy.deepcopy(self.plan)
|
||||
plan['workflow'] = {'nodes': ['injected']}
|
||||
plan['parts'][0]['source_path'] = '../unused.json'
|
||||
response = await self.save(plan=plan)
|
||||
filename = json.loads(response.body)['filename']
|
||||
plan['parts'][0]['positive'] = 'later change'
|
||||
self.assertEqual(materials._read_material(self.root / filename)['plan'], self.plan)
|
||||
|
||||
async def test_node_type_filter_and_scoped_hashes(self):
|
||||
workflow = {'nodes': [{'id': 1, 'type': 'CLIPTextEncode', 'widgets_values': ['hello']},
|
||||
{'id': 2, 'type': 'KSampler', 'widgets_values': [123, 20]}],
|
||||
'links': [], 'extra': {'anomalous_hashes': {'1_a': 'a', '2_b': 'b', '11_hidden': 'c'}}}
|
||||
record = {'schema_version': 1, 'kind': 'image_node_selection', 'name': 'Node', 'workflow': workflow,
|
||||
'selection': {'scope': 'nodes', 'node_ids': [1]}, 'capabilities': ['apply_node_parameters']}
|
||||
(self.root / 'node.json').write_text(json.dumps(record), encoding='utf-8')
|
||||
self.assertEqual(materials._query_materials(str(self.root), {'node_type': 'CLIPTextEncode'})['total'], 1)
|
||||
self.assertEqual(materials._query_materials(str(self.root), {'node_type': 'cliptextencode'})['total'], 0)
|
||||
self.assertEqual(materials._query_materials(str(self.root), {'node_type': 'KSampler'})['total'], 0)
|
||||
result = json.loads(materials._material_detail_response(self.root / 'node.json', '0').body)
|
||||
self.assertEqual(result['workflow_hashes'], {'1_a': 'a'})
|
||||
self.assertNotIn('workflow', result['data'])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,40 @@
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
from api import recipe_schema
|
||||
|
||||
|
||||
class RecipeModelReferenceTests(unittest.TestCase):
|
||||
def _references(self, nodes):
|
||||
with mock.patch.object(recipe_schema, '_identity_for_reference', return_value=({"status": "unavailable"}, None)):
|
||||
return recipe_schema._build_model_references({"workflow": {"nodes": nodes}, "params": {}})
|
||||
|
||||
def test_all_in_one_loaders_use_verified_layouts_and_skip_placeholders(self):
|
||||
refs = self._references([
|
||||
{"id": 1, "type": "easy a1111Loader", "widgets_values": ["base.safetensors", "Baked VAE", -2, "style.safetensors"]},
|
||||
{"id": 2, "type": "easy fullLoader", "widgets_values": ["full.safetensors", "Default", "vae.safetensors", -2, "None"]},
|
||||
])
|
||||
self.assertEqual(
|
||||
[(r["node_id"], r["widget_index"], r["category"], r["widget_name"], r["saved_value"]) for r in refs],
|
||||
[
|
||||
(1, 0, "checkpoint", "ckpt_name", "base.safetensors"),
|
||||
(1, 3, "lora", "lora_name", "style.safetensors"),
|
||||
(2, 0, "checkpoint", "ckpt_name", "full.safetensors"),
|
||||
(2, 2, "vae", "vae_name", "vae.safetensors"),
|
||||
],
|
||||
)
|
||||
|
||||
def test_native_labels_and_unknown_nodes_are_unchanged(self):
|
||||
refs = self._references([
|
||||
{"id": 3, "type": "CheckpointLoaderSimple", "widgets_values": ["sdxl.safetensors"]},
|
||||
{"id": 4, "type": "SomeCustomLoader", "widgets_values": ["other_vae.safetensors"]},
|
||||
])
|
||||
self.assertEqual([(r["node_id"], r["category"], r["widget_name"]) for r in refs], [(3, "checkpoint", "checkpoint")])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,529 @@
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
import io
|
||||
import json
|
||||
import hashlib
|
||||
import tempfile
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from PIL import Image, PngImagePlugin
|
||||
|
||||
PLUGIN_DIR = Path(__file__).resolve().parents[1]
|
||||
COMFY_ROOT = PLUGIN_DIR.parents[1]
|
||||
sys.path.insert(0, str(COMFY_ROOT))
|
||||
sys.path.insert(0, str(PLUGIN_DIR))
|
||||
|
||||
from api import recipes
|
||||
from api import models
|
||||
from api import parameters
|
||||
from api import recipe_packages
|
||||
from api import materials
|
||||
|
||||
|
||||
def recipe_payload(workflow):
|
||||
return {
|
||||
"name": "Round-trip fixture",
|
||||
"tags": ["test"],
|
||||
"notes": "",
|
||||
"params": {"nodes": [], "pinned": []},
|
||||
"workflow": workflow,
|
||||
"workflow_scope": "complete",
|
||||
"thumbnail": None,
|
||||
"source_image": None,
|
||||
"presentation": {"save_model_preview_snapshots": False},
|
||||
}
|
||||
|
||||
|
||||
def valid_workflow():
|
||||
return {
|
||||
"nodes": [
|
||||
{"id": 1, "type": "CLIPTextEncode", "widgets_values": ["prompt"]},
|
||||
{"id": 2, "type": "KSampler", "widgets_values": [1, "fixed", 20, 7, "euler", "normal", 1]},
|
||||
],
|
||||
"links": [[1, 1, 0, 2, 1, "CONDITIONING"]],
|
||||
"groups": [],
|
||||
}
|
||||
|
||||
|
||||
class MaterialSnapshotTests(unittest.TestCase):
|
||||
def test_image_snapshot_requires_full_ui_workflow(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
info = PngImagePlugin.PngInfo()
|
||||
info.add_text("prompt", json.dumps({"1": {"class_type": "KSampler", "inputs": {"seed": 1}}}))
|
||||
Image.new("RGB", (12, 12), "white").save(Path(directory, "prompt-only.png"), pnginfo=info)
|
||||
previous_output_dir = materials.folder_paths.get_output_directory
|
||||
materials.folder_paths.get_output_directory = lambda: directory
|
||||
try:
|
||||
with self.assertRaises(ValueError):
|
||||
materials._inspect_source_image({"type": "output", "filename": "prompt-only.png", "subfolder": ""})
|
||||
finally:
|
||||
materials.folder_paths.get_output_directory = previous_output_dir
|
||||
|
||||
def test_node_blocks_preserve_exact_values_and_mark_sampler_seed(self):
|
||||
workflow = valid_workflow()
|
||||
workflow["nodes"][1]["properties"] = {"preview": "detailed"}
|
||||
workflow["nodes"][1]["mode"] = 0
|
||||
blocks = materials._node_blocks(workflow, include_values=True)
|
||||
sampler = next(block for block in blocks if block["type"] == "KSampler")
|
||||
self.assertEqual(sampler["widgets_values"][0], 1)
|
||||
self.assertEqual(sampler["volatile_widget_indexes"], [0])
|
||||
self.assertEqual(sampler["properties"], {"preview": "detailed"})
|
||||
self.assertEqual(sampler["mode"], 0)
|
||||
|
||||
def test_material_summary_excludes_full_workflow(self):
|
||||
material = {
|
||||
"id": "fixture",
|
||||
"kind": "image_workflow_snapshot",
|
||||
"name": "Fixture",
|
||||
"timestamp": 1,
|
||||
"workflow": valid_workflow(),
|
||||
"image": {"preview_asset_id": "preview.webp", "source_asset_id": "source.png"},
|
||||
"capabilities": ["open_workflow"],
|
||||
}
|
||||
summary = materials._material_summary("material_fixture.json", material)
|
||||
self.assertNotIn("workflow", summary)
|
||||
self.assertEqual(summary["node_count"], 2)
|
||||
self.assertEqual(summary["node_types"], ["CLIPTextEncode", "KSampler"])
|
||||
|
||||
def test_selected_material_exposes_only_selected_node_blocks(self):
|
||||
material = {
|
||||
"workflow": valid_workflow(),
|
||||
"selection": {"scope": "nodes", "node_ids": [2]},
|
||||
}
|
||||
blocks = materials._material_node_blocks(material, include_values=True)
|
||||
self.assertEqual([block["node_id"] for block in blocks], [2])
|
||||
self.assertEqual(blocks[0]["type"], "KSampler")
|
||||
|
||||
def test_selected_node_ids_are_validated_and_deduplicated(self):
|
||||
self.assertEqual(materials._normalise_selected_node_ids(valid_workflow(), ["2", 2, 1]), [2, 1])
|
||||
with self.assertRaises(ValueError):
|
||||
materials._normalise_selected_node_ids(valid_workflow(), [99])
|
||||
|
||||
|
||||
class RecipeWorkflowValidationTests(unittest.TestCase):
|
||||
def test_workflow_scope_round_trips_and_legacy_defaults_complete(self):
|
||||
payload = recipe_payload(valid_workflow())
|
||||
payload["workflow_scope"] = "partial"
|
||||
self.assertEqual(recipes._normalise_recipe(payload)["workflow_scope"], "partial")
|
||||
payload.pop("workflow_scope")
|
||||
self.assertEqual(recipes._normalise_recipe(payload)["workflow_scope"], "complete")
|
||||
|
||||
def test_invalid_workflow_scope_is_rejected(self):
|
||||
payload = recipe_payload(valid_workflow())
|
||||
payload["workflow_scope"] = "selection"
|
||||
with self.assertRaises(ValueError):
|
||||
recipes._normalise_recipe(payload)
|
||||
|
||||
def test_valid_workflow_returns_integrity_receipt(self):
|
||||
recipe = recipes._normalise_recipe(recipe_payload(valid_workflow()))
|
||||
receipt = recipes._recipe_receipt(recipe, "recipe_fixture.json")
|
||||
self.assertEqual(receipt["node_count"], 2)
|
||||
self.assertEqual(receipt["link_count"], 1)
|
||||
self.assertEqual(receipt["group_count"], 0)
|
||||
self.assertTrue(receipt["workflow_fingerprint"] is None)
|
||||
|
||||
def test_duplicate_node_ids_are_rejected(self):
|
||||
workflow = valid_workflow()
|
||||
workflow["nodes"].append({"id": 1, "type": "PreviewImage", "widgets_values": []})
|
||||
with self.assertRaises(ValueError):
|
||||
recipes._normalise_recipe(recipe_payload(workflow))
|
||||
|
||||
def test_dangling_link_is_rejected(self):
|
||||
workflow = valid_workflow()
|
||||
workflow["links"] = [[1, 99, 0, 2, 1, "CONDITIONING"]]
|
||||
with self.assertRaises(ValueError):
|
||||
recipes._normalise_recipe(recipe_payload(workflow))
|
||||
|
||||
def test_nodes_must_be_a_list(self):
|
||||
workflow = valid_workflow()
|
||||
workflow["nodes"] = {}
|
||||
with self.assertRaises(ValueError):
|
||||
recipes._normalise_recipe(recipe_payload(workflow))
|
||||
|
||||
def test_structural_fingerprint_ignores_sampler_seed_and_batch_size(self):
|
||||
first = valid_workflow()
|
||||
first["nodes"][1]["properties"] = {"batch_size": 1, "seed": 1}
|
||||
second = json.loads(json.dumps(first))
|
||||
second["nodes"][1]["widgets_values"][0] = 987654321
|
||||
second["nodes"][1]["properties"] = {"batch_size": 4, "seed": 987654321}
|
||||
self.assertEqual(
|
||||
recipes._workflow_fingerprint(first),
|
||||
recipes._workflow_fingerprint(second),
|
||||
)
|
||||
|
||||
def test_structural_fingerprint_keeps_generation_parameters(self):
|
||||
first = valid_workflow()
|
||||
second = json.loads(json.dumps(first))
|
||||
second["nodes"][1]["widgets_values"][2] = 28
|
||||
self.assertNotEqual(
|
||||
recipes._workflow_fingerprint(first),
|
||||
recipes._workflow_fingerprint(second),
|
||||
)
|
||||
|
||||
def test_gallery_uses_embedded_workflow_and_structural_fingerprint(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
info = PngImagePlugin.PngInfo()
|
||||
workflow = valid_workflow()
|
||||
embedded_workflow = json.loads(json.dumps(workflow))
|
||||
embedded_workflow["nodes"][0]["widgets_values"] = ["a different prompt"]
|
||||
embedded_workflow["nodes"][1]["widgets_values"][2] = 42
|
||||
info.add_text("workflow", json.dumps(embedded_workflow))
|
||||
Image.new("RGB", (12, 12), "white").save(Path(directory, "match.png"), pnginfo=info)
|
||||
previous_output_dir = recipes.folder_paths.get_output_directory
|
||||
recipes.folder_paths.get_output_directory = lambda: directory
|
||||
try:
|
||||
matches, scanned = recipes._recipe_gallery_images(recipes._workflow_node_signature(workflow)["value"])
|
||||
finally:
|
||||
recipes.folder_paths.get_output_directory = previous_output_dir
|
||||
self.assertEqual(scanned, 1)
|
||||
self.assertEqual([item["filename"] for item in matches], ["match.png"])
|
||||
|
||||
def test_gallery_comparison_reports_parameter_difference_after_node_match(self):
|
||||
recipe = valid_workflow()
|
||||
image_workflow = json.loads(json.dumps(recipe))
|
||||
image_workflow["nodes"][1]["widgets_values"][2] = 42
|
||||
comparison = recipes._gallery_parameter_diff(recipe, {"workflow": image_workflow, "prompt": None})
|
||||
self.assertEqual(
|
||||
recipes._workflow_node_signature(recipe),
|
||||
recipes._workflow_node_signature(image_workflow),
|
||||
)
|
||||
self.assertTrue(any(change["type"] == "KSampler" for change in comparison["changes"]))
|
||||
|
||||
def test_parameter_signature_reads_api_prompt_inputs(self):
|
||||
first = {
|
||||
"1": {"class_type": "KSampler", "inputs": {"steps": 20, "seed": 1}},
|
||||
}
|
||||
second = {
|
||||
"1": {"class_type": "KSampler", "inputs": {"steps": 28, "seed": 1}},
|
||||
}
|
||||
self.assertNotEqual(recipes._parameter_signature(first), recipes._parameter_signature(second))
|
||||
|
||||
|
||||
class RecipeIdentityBoundaryTests(unittest.TestCase):
|
||||
def test_save_time_verification_computes_supported_model_hash(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
model_path = Path(directory, "model.safetensors")
|
||||
model_path.write_bytes(b"recipe verification fixture")
|
||||
previous_resolver = recipes._resolve_exact_model_reference
|
||||
recipes._resolve_exact_model_reference = lambda value: {
|
||||
"path": str(model_path), "folder_type": "checkpoints", "path_index": 0,
|
||||
}
|
||||
try:
|
||||
identity = recipes._computed_identity_for_reference("model.safetensors")
|
||||
finally:
|
||||
recipes._resolve_exact_model_reference = previous_resolver
|
||||
self.assertEqual(identity["status"], "verified")
|
||||
self.assertEqual(identity["sha256"], hashlib.sha256(b"recipe verification fixture").hexdigest())
|
||||
|
||||
def test_foundation_models_are_in_save_time_verification_categories(self):
|
||||
self.assertIn("vae", recipes.VERIFIABLE_RECIPE_MODEL_CATEGORIES)
|
||||
self.assertIn("text_encoder", recipes.VERIFIABLE_RECIPE_MODEL_CATEGORIES)
|
||||
self.assertIn("lora", recipes.VERIFIABLE_RECIPE_MODEL_CATEGORIES)
|
||||
|
||||
def test_filename_alone_never_resolves_identity(self):
|
||||
candidates = [{"type": "checkpoints", "filename": "nested/model.safetensors", "size": 10, "hashes": set()}]
|
||||
result = models._resolve_from_candidates(candidates, filename_query="nested/model.safetensors")
|
||||
self.assertFalse(result["found"])
|
||||
|
||||
def test_hash_and_size_resolve_without_filename_evidence(self):
|
||||
candidates = [{"type": "checkpoints", "filename": "other/model.safetensors", "size": 10, "hashes": {"A" * 64}}]
|
||||
result = models._resolve_from_candidates(candidates, target_hash="A" * 64, target_size=10, filename_query="missing/model.safetensors")
|
||||
self.assertTrue(result["found"])
|
||||
self.assertTrue(result["matched_by_hash"])
|
||||
self.assertTrue(result["matched_by_size"])
|
||||
|
||||
def test_enrichment_preserves_imported_identity(self):
|
||||
original_identity = {"status": "verified", "sha256": "B" * 64, "size": 10, "provenance": "package"}
|
||||
original_preview = {"snapshot_asset_id": "historical.webp", "media_type": "image/webp"}
|
||||
recipe = {
|
||||
"workflow": {"nodes": [{"id": 1, "type": "CheckpointLoaderSimple", "widgets_values": ["model.safetensors"]}], "links": [], "groups": []},
|
||||
"params": {"baseModel": "model.safetensors", "model_references": [{
|
||||
"node_id": 1, "widget_index": 0, "category": "checkpoint", "saved_value": "model.safetensors",
|
||||
"identity": original_identity,
|
||||
"preview": original_preview,
|
||||
}]},
|
||||
}
|
||||
previous_resolver = recipes._identity_for_reference
|
||||
recipes._identity_for_reference = lambda value: {"status": "unavailable"}
|
||||
try:
|
||||
enriched = recipes._enrich_recipe(recipe)
|
||||
finally:
|
||||
recipes._identity_for_reference = previous_resolver
|
||||
self.assertEqual(enriched["params"]["model_references"][0]["identity"], original_identity)
|
||||
self.assertEqual(enriched["params"]["model_references"][0]["preview"], original_preview)
|
||||
|
||||
def test_enrichment_preserves_bounded_model_note(self):
|
||||
recipe = {
|
||||
"workflow": {"nodes": [{"id": 1, "type": "CheckpointLoaderSimple", "widgets_values": ["model.safetensors"]}], "links": [], "groups": []},
|
||||
"params": {"model_references": [{
|
||||
"node_id": 1, "widget_index": 0, "category": "checkpoint", "saved_value": "model.safetensors",
|
||||
"identity": {"status": "unverified"},
|
||||
"user_note": " Recommended CFG 5.5 ",
|
||||
}]},
|
||||
}
|
||||
previous_resolver = recipes._identity_for_reference
|
||||
recipes._identity_for_reference = lambda value: {"status": "unavailable"}
|
||||
try:
|
||||
enriched = recipes._enrich_recipe(recipe)
|
||||
finally:
|
||||
recipes._identity_for_reference = previous_resolver
|
||||
self.assertEqual(enriched["params"]["model_references"][0]["user_note"], "Recommended CFG 5.5")
|
||||
self.assertEqual(enriched["schema_version"], 7)
|
||||
|
||||
|
||||
class RecipePreviewSnapshotTests(unittest.TestCase):
|
||||
def test_saving_with_preview_snapshots_captures_the_model_cover(self):
|
||||
"""The default save path: a loader node, its cover next to the model, snapshots on."""
|
||||
with tempfile.TemporaryDirectory() as models_dir, tempfile.TemporaryDirectory() as recipes_dir:
|
||||
Path(models_dir, "model.safetensors").write_bytes(b"not a real model")
|
||||
Image.new("RGB", (64, 48), "red").save(Path(models_dir, "model.preview.png"))
|
||||
payload = recipe_payload({
|
||||
"nodes": [{"id": 1, "type": "CheckpointLoaderSimple", "widgets_values": ["model.safetensors"]}],
|
||||
"links": [],
|
||||
"groups": [],
|
||||
})
|
||||
payload["presentation"] = {"save_model_preview_snapshots": True}
|
||||
folder_paths = recipes.folder_paths
|
||||
previous_names = folder_paths.folder_names_and_paths
|
||||
previous_get = folder_paths.get_folder_paths
|
||||
folder_paths.folder_names_and_paths = {"checkpoints": ([models_dir], {".safetensors"})}
|
||||
folder_paths.get_folder_paths = lambda folder_type: [models_dir] if folder_type == "checkpoints" else []
|
||||
try:
|
||||
recipe = recipes._normalise_recipe(payload)
|
||||
enriched = recipes._enrich_recipe(recipe, recipes_dir, "recipe_fixture.json", True, False, False)
|
||||
finally:
|
||||
folder_paths.folder_names_and_paths = previous_names
|
||||
folder_paths.get_folder_paths = previous_get
|
||||
preview = enriched["params"]["model_references"][0]["preview"]
|
||||
self.assertEqual(preview["media_type"], "image/webp")
|
||||
self.assertTrue(Path(recipes_dir, ".assets", "recipe_fixture", preview["snapshot_asset_id"]).is_file())
|
||||
|
||||
|
||||
class RecipePackageBoundaryTests(unittest.TestCase):
|
||||
def _package_recipe(self, asset_id):
|
||||
recipe = recipe_payload(valid_workflow())
|
||||
recipe["source_image"] = {"type": "output", "filename": "local.png", "subfolder": "renders"}
|
||||
recipe["params"]["model_references"] = [{
|
||||
"node_id": 1,
|
||||
"widget_index": 0,
|
||||
"category": "checkpoint",
|
||||
"saved_value": "model.safetensors",
|
||||
"preview": {"snapshot_asset_id": asset_id, "media_type": "image/webp"},
|
||||
}]
|
||||
return recipe
|
||||
|
||||
def test_export_unions_history_assets_and_removes_local_source(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
filename = "recipe_fixture.json"
|
||||
assets_dir = Path(recipes._recipe_assets_dir(directory, filename, create=True))
|
||||
assets_dir.joinpath("current.webp").write_bytes(b"RIFF0000WEBP")
|
||||
assets_dir.joinpath("historical.webp").write_bytes(b"RIFF1111WEBP")
|
||||
history_dir = Path(recipes._history_dir(directory, filename, create=True))
|
||||
history = self._package_recipe("historical.webp")
|
||||
history_dir.joinpath("version_1.json").write_text(json.dumps(history), encoding="utf-8")
|
||||
current = self._package_recipe("current.webp")
|
||||
package = recipe_packages._build_export(current, directory, filename, {
|
||||
"include_snapshots": True,
|
||||
"include_history": True,
|
||||
"include_identity": True,
|
||||
})
|
||||
with zipfile.ZipFile(io.BytesIO(package)) as archive:
|
||||
self.assertIn("assets/current.webp", archive.namelist())
|
||||
self.assertIn("assets/historical.webp", archive.namelist())
|
||||
exported = json.loads(archive.read("recipe.json"))
|
||||
historical_exported = json.loads(archive.read("history/version_1.json"))
|
||||
self.assertIsNone(exported["source_image"])
|
||||
self.assertIsNone(historical_exported["source_image"])
|
||||
|
||||
def test_export_can_remove_personal_model_notes(self):
|
||||
recipe = self._package_recipe("model-preview.webp")
|
||||
recipe["params"]["model_references"][0]["user_note"] = "private tuning note"
|
||||
exported = recipe_packages._sanitize_recipe_for_export(
|
||||
recipe,
|
||||
include_snapshots=False,
|
||||
include_identity=True,
|
||||
include_model_notes=False,
|
||||
)
|
||||
self.assertNotIn("user_note", exported["params"]["model_references"][0])
|
||||
self.assertEqual(recipe["params"]["model_references"][0]["user_note"], "private tuning note")
|
||||
|
||||
def test_the_receiver_is_told_about_the_cover_and_the_model_thumbnails(self):
|
||||
recipe = self._package_recipe("model-preview.webp")
|
||||
recipe["thumbnail"] = "data:image/jpeg;base64,AAAA"
|
||||
self.assertEqual(recipe_packages._picture_summary(recipe), (True, 1))
|
||||
recipe["thumbnail"] = None
|
||||
recipe["params"]["model_references"][0].pop("preview", None)
|
||||
self.assertEqual(recipe_packages._picture_summary(recipe), (False, 0))
|
||||
|
||||
def _import_into(self, directory, package, fail_on=None):
|
||||
record = recipe_packages._inspect_package(package)
|
||||
record["raw"] = package
|
||||
previous_get_dir = recipe_packages.get_recipes_dir
|
||||
previous_replace = recipe_packages.os.replace
|
||||
calls = {"count": 0}
|
||||
|
||||
def replace(source, target):
|
||||
calls["count"] += 1
|
||||
if fail_on and str(target).endswith(fail_on):
|
||||
raise OSError("simulated commit failure")
|
||||
return previous_replace(source, target)
|
||||
|
||||
recipe_packages.get_recipes_dir = lambda: directory
|
||||
recipe_packages.os.replace = replace
|
||||
try:
|
||||
return recipe_packages._commit_import(record, {})
|
||||
finally:
|
||||
recipe_packages.get_recipes_dir = previous_get_dir
|
||||
recipe_packages.os.replace = previous_replace
|
||||
|
||||
def _shared_package(self, source_dir):
|
||||
filename = "recipe_shared.json"
|
||||
assets_dir = Path(recipes._recipe_assets_dir(source_dir, filename, create=True))
|
||||
assets_dir.joinpath("cover-result.webp").write_bytes(b"RIFF0000WEBP")
|
||||
assets_dir.joinpath("model-preview.webp").write_bytes(b"RIFF1111WEBP")
|
||||
recipe = self._package_recipe("model-preview.webp")
|
||||
recipe["name"] = "Portrait"
|
||||
recipe["presentation"]["cover_asset_id"] = "cover-result.webp"
|
||||
return recipe_packages._build_export(recipe, source_dir, filename, {})
|
||||
|
||||
def test_import_makes_a_new_card_and_never_replaces_one(self):
|
||||
with tempfile.TemporaryDirectory() as source, tempfile.TemporaryDirectory() as directory:
|
||||
package = self._shared_package(source)
|
||||
mine = recipe_payload(valid_workflow())
|
||||
mine["name"] = "Portrait"
|
||||
Path(directory, "mine.json").write_text(json.dumps(mine), encoding="utf-8")
|
||||
filename, name = self._import_into(directory, package)
|
||||
self.assertEqual(name, "Portrait (2)")
|
||||
self.assertEqual(json.loads(Path(directory, "mine.json").read_text(encoding="utf-8"))["name"], "Portrait")
|
||||
imported = json.loads(Path(directory, filename).read_text(encoding="utf-8"))
|
||||
self.assertTrue(imported["presentation"]["imported"])
|
||||
assets = Path(recipes._recipe_assets_dir(directory, filename))
|
||||
self.assertTrue(assets.joinpath("cover-result.webp").is_file())
|
||||
self.assertTrue(assets.joinpath("model-preview.webp").is_file()) # model thumbnails by default
|
||||
_, again = self._import_into(directory, package)
|
||||
self.assertEqual(again, "Portrait (3)")
|
||||
|
||||
def test_a_failed_import_leaves_nothing_behind(self):
|
||||
with tempfile.TemporaryDirectory() as source, tempfile.TemporaryDirectory() as directory:
|
||||
package = self._shared_package(source)
|
||||
with self.assertRaises(OSError):
|
||||
self._import_into(directory, package, fail_on=".json")
|
||||
self.assertEqual([name for name in os.listdir(directory) if name.endswith(".json")], [])
|
||||
assets_root = Path(directory, ".assets")
|
||||
self.assertEqual(list(assets_root.iterdir()) if assets_root.exists() else [], [])
|
||||
self.assertFalse([name for name in os.listdir(directory) if name.startswith(".recipe-import-")])
|
||||
|
||||
def test_a_tampered_package_is_refused(self):
|
||||
with tempfile.TemporaryDirectory() as source:
|
||||
package = self._shared_package(source)
|
||||
with zipfile.ZipFile(io.BytesIO(package)) as archive:
|
||||
entries = {name: archive.read(name) for name in archive.namelist()}
|
||||
entries["recipe.json"] = entries["recipe.json"].replace(b"Portrait", b"Changed!")
|
||||
tampered = io.BytesIO()
|
||||
with zipfile.ZipFile(tampered, "w") as archive:
|
||||
for name, data in entries.items():
|
||||
archive.writestr(name, data)
|
||||
with self.assertRaises(ValueError):
|
||||
recipe_packages._inspect_package(tampered.getvalue())
|
||||
escaping = io.BytesIO()
|
||||
with zipfile.ZipFile(escaping, "w") as archive:
|
||||
for name, data in entries.items():
|
||||
archive.writestr(name, data)
|
||||
archive.writestr("../evil.json", b"{}")
|
||||
with self.assertRaises(ValueError):
|
||||
recipe_packages._inspect_package(escaping.getvalue())
|
||||
|
||||
def test_export_includes_gallery_cover_without_model_snapshots(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
filename = "recipe_fixture.json"
|
||||
assets_dir = Path(recipes._recipe_assets_dir(directory, filename, create=True))
|
||||
assets_dir.joinpath("cover-result.webp").write_bytes(b"RIFF0000WEBP")
|
||||
recipe = self._package_recipe("model-preview.webp")
|
||||
recipe["presentation"]["cover_asset_id"] = "cover-result.webp"
|
||||
package = recipe_packages._build_export(recipe, directory, filename, {
|
||||
"include_snapshots": False,
|
||||
"include_history": False,
|
||||
"include_identity": True,
|
||||
})
|
||||
with zipfile.ZipFile(io.BytesIO(package)) as archive:
|
||||
self.assertIn("assets/cover-result.webp", archive.namelist())
|
||||
self.assertNotIn("assets/model-preview.webp", archive.namelist())
|
||||
|
||||
|
||||
class LegacyPromptRoleTests(unittest.TestCase):
|
||||
def test_manual_role_override_has_priority(self):
|
||||
node = {"id": 7, "type": "CLIPTextEncode", "widgets_values": ["negative prompt"]}
|
||||
params = {
|
||||
"promptRoleOverrides": {"7": {"role": "positive", "nodeType": "CLIPTextEncode"}},
|
||||
"promptNegative": ["negative prompt"],
|
||||
}
|
||||
self.assertEqual(parameters._prompt_role_for_node(node, params), "positive")
|
||||
|
||||
def test_stale_manual_role_type_is_ignored(self):
|
||||
node = {"id": 7, "type": "CLIPTextEncode", "widgets_values": ["negative prompt"]}
|
||||
params = {
|
||||
"promptRoleOverrides": {"7": {"role": "positive", "nodeType": "OtherNode"}},
|
||||
"promptNegative": ["negative prompt"],
|
||||
}
|
||||
self.assertEqual(parameters._prompt_role_for_node(node, params), "negative")
|
||||
|
||||
def test_legacy_parameter_node_recovers_negative_role(self):
|
||||
node = {"id": 7, "type": "CLIPTextEncode", "widgets_values": ["negative prompt"]}
|
||||
params = {"promptPositive": ["positive prompt"], "promptNegative": ["negative prompt"]}
|
||||
self.assertEqual(parameters._prompt_role_for_node(node, params), "negative")
|
||||
|
||||
def test_ambiguous_prompt_is_not_guessed(self):
|
||||
node = {"id": 7, "type": "CLIPTextEncode", "widgets_values": ["shared prompt"]}
|
||||
params = {"promptPositive": ["shared prompt"], "promptNegative": ["shared prompt"]}
|
||||
self.assertIsNone(parameters._prompt_role_for_node(node, params))
|
||||
|
||||
|
||||
class ParameterNotebookRenameTests(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_rename_parameter_notebook(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
nb_filename = "params_test_12345.json"
|
||||
nb_path = os.path.join(temp_dir, nb_filename)
|
||||
initial_data = {
|
||||
"name": "Original Name",
|
||||
"recipe_filename": "recipe_1.json",
|
||||
"workflow": valid_workflow(),
|
||||
"timestamp": 1000
|
||||
}
|
||||
with open(nb_path, "w", encoding="utf-8") as f:
|
||||
json.dump(initial_data, f)
|
||||
|
||||
# Mock get_parameters_dir
|
||||
orig_get_dir = parameters.get_parameters_dir
|
||||
parameters.get_parameters_dir = lambda: temp_dir
|
||||
try:
|
||||
class DummyRequest:
|
||||
def __init__(self, payload):
|
||||
self._payload = payload
|
||||
async def json(self):
|
||||
return self._payload
|
||||
|
||||
# Test successful rename
|
||||
req = DummyRequest({"filename": nb_filename, "name": "Renamed Notebook"})
|
||||
res = await parameters.api_rename_parameter(req)
|
||||
self.assertEqual(res.status, 200)
|
||||
|
||||
with open(nb_path, "r", encoding="utf-8") as f:
|
||||
updated_data = json.load(f)
|
||||
self.assertEqual(updated_data["name"], "Renamed Notebook")
|
||||
|
||||
# Test invalid filename rejection
|
||||
bad_req = DummyRequest({"filename": "../evil.json", "name": "Evil"})
|
||||
bad_res = await parameters.api_rename_parameter(bad_req)
|
||||
self.assertEqual(bad_res.status, 400)
|
||||
|
||||
# Test empty name rejection
|
||||
empty_req = DummyRequest({"filename": nb_filename, "name": " "})
|
||||
empty_res = await parameters.api_rename_parameter(empty_req)
|
||||
self.assertEqual(empty_res.status, 400)
|
||||
finally:
|
||||
parameters.get_parameters_dir = orig_get_dir
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,124 @@
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
import recycle_bin
|
||||
from api import material_store, model_metadata
|
||||
|
||||
|
||||
class FakeTrash:
|
||||
"""Stands in for the Recycle Bin: records what was sent and removes it, like a move would."""
|
||||
|
||||
def __init__(self, fail=None):
|
||||
self.calls = []
|
||||
self.fail = fail
|
||||
|
||||
def __call__(self, *paths):
|
||||
if self.fail:
|
||||
raise self.fail
|
||||
self.calls.append([os.path.basename(p) for p in paths if os.path.lexists(p)])
|
||||
for path in paths:
|
||||
if os.path.isdir(path):
|
||||
import shutil
|
||||
shutil.rmtree(path)
|
||||
elif os.path.lexists(path):
|
||||
os.remove(path)
|
||||
return [p for p in paths]
|
||||
|
||||
|
||||
class Request(dict): # aiohttp requests also hold values for the middleware
|
||||
def __init__(self, body):
|
||||
super().__init__()
|
||||
self.body = body
|
||||
|
||||
async def json(self):
|
||||
return self.body
|
||||
|
||||
|
||||
def body(response):
|
||||
return json.loads(response.body)
|
||||
|
||||
|
||||
class RecycleBinTests(unittest.TestCase):
|
||||
def test_only_existing_paths_go_together_as_absolute_paths(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
Path(directory, 'a.safetensors').write_bytes(b'x')
|
||||
with mock.patch.object(recycle_bin.sys, 'platform', 'win32'), \
|
||||
mock.patch.object(recycle_bin, '_windows_recycle') as recycle:
|
||||
moved = recycle_bin.move_to_trash(os.path.join(directory, 'a.safetensors'), os.path.join(directory, 'gone.png'), '')
|
||||
recycle.assert_called_once()
|
||||
self.assertEqual(moved, [os.path.abspath(os.path.join(directory, 'a.safetensors'))])
|
||||
self.assertEqual(recycle_bin.move_to_trash(), [])
|
||||
|
||||
def test_network_and_removable_drives_are_refused_not_deleted(self):
|
||||
with mock.patch.object(recycle_bin, '_windows_fixed_drive', return_value=False):
|
||||
with self.assertRaises(recycle_bin.TrashUnavailable):
|
||||
recycle_bin._windows_recycle([r'\\server\share\model.safetensors'])
|
||||
|
||||
|
||||
class ModelDeleteTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.temp.name
|
||||
for name in ('m.safetensors', 'm.info', 'm.preview.png', 'other.safetensors'):
|
||||
Path(self.dir, name).write_bytes(b'x')
|
||||
self.patch = mock.patch.object(model_metadata, 'resolve_folder_subdir', return_value=(self.dir, self.dir))
|
||||
self.patch.start()
|
||||
|
||||
def tearDown(self):
|
||||
self.patch.stop()
|
||||
self.temp.cleanup()
|
||||
|
||||
def delete(self, trash):
|
||||
with mock.patch.object(model_metadata, 'move_to_trash', trash):
|
||||
return body(asyncio.run(model_metadata.api_delete_model(Request({'filename': 'm.safetensors', 'type': 'loras'}))))
|
||||
|
||||
def test_model_and_its_sidecars_go_to_the_recycle_bin_together(self):
|
||||
trash = FakeTrash()
|
||||
result = self.delete(trash)
|
||||
self.assertEqual(result['status'], 'success')
|
||||
self.assertEqual(len(trash.calls), 1)
|
||||
self.assertEqual(sorted(trash.calls[0]), ['m.info', 'm.preview.png', 'm.safetensors'])
|
||||
self.assertTrue(Path(self.dir, 'other.safetensors').exists())
|
||||
|
||||
def test_sidecars_stay_when_another_model_shares_the_name(self):
|
||||
Path(self.dir, 'm.ckpt').write_bytes(b'x')
|
||||
trash = FakeTrash()
|
||||
result = self.delete(trash)
|
||||
self.assertEqual(trash.calls, [['m.safetensors']])
|
||||
self.assertTrue(result['sidecars_preserved'])
|
||||
|
||||
def test_nothing_is_deleted_without_a_recycle_bin(self):
|
||||
result = self.delete(FakeTrash(fail=recycle_bin.TrashUnavailable('no bin')))
|
||||
self.assertEqual(result['status'], 'error')
|
||||
self.assertIn('回收站', result['message'])
|
||||
self.assertTrue(Path(self.dir, 'm.safetensors').exists())
|
||||
|
||||
|
||||
class MaterialDeleteTests(unittest.TestCase):
|
||||
def test_material_and_its_images_go_together(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory, 'one.json')
|
||||
path.write_text(json.dumps({'name': 'one', 'workflow': {'nodes': []}, 'timestamp': 1}))
|
||||
assets = Path(material_store._material_assets_dir(directory, 'one.json'))
|
||||
assets.mkdir(parents=True)
|
||||
(assets / 'image.png').write_bytes(b'png')
|
||||
trash = FakeTrash()
|
||||
with mock.patch.object(material_store, 'move_to_trash', trash), \
|
||||
mock.patch.object(material_store, '_read_material', return_value={}):
|
||||
material_store._delete_material(directory, 'one.json', str(path))
|
||||
self.assertEqual(len(trash.calls), 1)
|
||||
self.assertIn('one.json', trash.calls[0])
|
||||
self.assertFalse(path.exists())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,197 @@
|
||||
"""What a scan reports per model, why a model is unmatched, and the scan's result and log entry."""
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from aiohttp.test_utils import make_mocked_request
|
||||
|
||||
PLUGIN = Path(__file__).resolve().parents[1]
|
||||
sys.path[:0] = [str(PLUGIN), str(PLUGIN.parents[1])]
|
||||
|
||||
import scraper
|
||||
from api import activity_log, model_metadata, scan_report, scan_summary, scanner
|
||||
from model_identity import computed_file_identity
|
||||
|
||||
def digest(name):
|
||||
return hashlib.sha256(name.encode()).hexdigest()
|
||||
|
||||
|
||||
FOUND = {"id": 7, "modelId": 3, "name": "v2", "baseModel": "SDXL 1.0", "model": {"name": "Found"}, "images": []}
|
||||
|
||||
|
||||
class ScanReportTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.dir = Path(self.temp.name)
|
||||
self.report = self.dir / "report.jsonl"
|
||||
|
||||
def tearDown(self):
|
||||
self.temp.cleanup()
|
||||
|
||||
def model(self, name, info=None):
|
||||
path = self.dir / f"{name}.safetensors"
|
||||
header = json.dumps({"__metadata__": {"name": name}, "w": {"dtype": "F32", "shape": [1], "data_offsets": [0, 4]}}).encode()
|
||||
path.write_bytes(len(header).to_bytes(8, "little") + header + b"\0" * 4) # a whole file (not damaged)
|
||||
if info is not None:
|
||||
Path(str(path)[:-12] + ".info").write_text(json.dumps(info), encoding="utf-8")
|
||||
return path
|
||||
|
||||
def info(self, name):
|
||||
return json.loads((self.dir / f"{name}.info").read_text(encoding="utf-8"))
|
||||
|
||||
def scan(self, fetch, *flags):
|
||||
"""Runs the scraper (no real network); returns (models Civitai was asked about, report lines)."""
|
||||
asked = []
|
||||
|
||||
def ask(file_hash):
|
||||
asked.append(file_hash)
|
||||
return fetch(file_hash)
|
||||
|
||||
argv = ["scraper.py", str(self.dir), "--report-file", str(self.report), *flags]
|
||||
with mock.patch.object(sys, "argv", argv), \
|
||||
mock.patch.object(scraper, "fetch_civitai_info", side_effect=ask), \
|
||||
mock.patch.object(scraper, "calculate_sha256", side_effect=lambda p: digest(Path(p).stem)), \
|
||||
mock.patch.object(scraper.urllib.request, "urlopen", side_effect=OSError("no network in tests")):
|
||||
scraper.main()
|
||||
lines = [json.loads(line) for line in self.report.read_text(encoding="utf-8").splitlines()]
|
||||
self.report.unlink()
|
||||
return asked, lines
|
||||
|
||||
def test_each_model_reports_its_outcome_and_unmatched_ones_keep_why(self):
|
||||
self.model("aaa")
|
||||
self.model("bbb")
|
||||
asked, lines = self.scan(lambda h: dict(FOUND) if h == digest("aaa") else None)
|
||||
self.assertEqual(len(asked), 2)
|
||||
by_file = {Path(line["file"]).stem: line for line in lines if "file" in line}
|
||||
self.assertEqual(by_file["aaa"]["status"], "matched")
|
||||
self.assertEqual(by_file["aaa"]["name"], "Found · v2")
|
||||
self.assertEqual((by_file["bbb"]["status"], by_file["bbb"]["reason"]), ("inferred", "not_found"))
|
||||
self.assertEqual(self.info("bbb")["anomalous_unmatched_reason"], "not_found")
|
||||
self.assertEqual(lines[-1], {"event": "done", "unchanged": 0, "civitai_down": False})
|
||||
|
||||
def test_when_civitai_stops_answering_the_rest_is_inferred_without_asking(self):
|
||||
for name in ("m1", "m2", "m3", "m4"):
|
||||
self.model(name)
|
||||
|
||||
def down(_hash):
|
||||
raise scraper.CivitaiUnreachable("timeout")
|
||||
|
||||
asked, lines = self.scan(down)
|
||||
self.assertEqual(len(asked), scraper.CIVITAI_DOWN_AFTER)
|
||||
self.assertIn({"event": "civitai_down", "reason": "timeout"}, lines) # why, for the scan page
|
||||
reasons = [line.get("reason") for line in lines if "file" in line]
|
||||
self.assertEqual(reasons, ["network"] * 4)
|
||||
|
||||
def test_online_scans_look_up_models_civitai_was_never_asked_about(self):
|
||||
for name, reason in (("off", "offline"), ("net", "network"), ("gone", "not_found")):
|
||||
path = self.model(name)
|
||||
Path(str(path)[:-12] + ".info").write_text(json.dumps({
|
||||
"id": -1, "modelId": -1, "anomalous_unmatched_reason": reason,
|
||||
"anomalous_file_identity": computed_file_identity(str(path), digest(name))}), encoding="utf-8")
|
||||
asked, lines = self.scan(lambda h: dict(FOUND))
|
||||
self.assertEqual(sorted(asked), sorted([digest("net"), digest("off")]))
|
||||
self.assertEqual(self.info("gone")["id"], -1) # waits for "look up again"
|
||||
self.assertEqual(self.info("off")["id"], 7)
|
||||
# An offline scan leaves them alone.
|
||||
asked, _ = self.scan(lambda h: dict(FOUND), "--offline-only")
|
||||
self.assertEqual(asked, [])
|
||||
|
||||
def test_one_models_failure_does_not_end_the_scan(self):
|
||||
self.model("bad")
|
||||
self.model("good")
|
||||
|
||||
def fetch(file_hash):
|
||||
if file_hash == digest("bad"):
|
||||
raise PermissionError("locked")
|
||||
return dict(FOUND)
|
||||
|
||||
_asked, lines = self.scan(fetch)
|
||||
by_file = {Path(line["file"]).stem: line for line in lines if "file" in line}
|
||||
self.assertEqual(by_file["bad"]["status"], "failed")
|
||||
self.assertIn("locked", by_file["bad"]["error"])
|
||||
self.assertEqual(by_file["good"]["status"], "matched")
|
||||
|
||||
def test_job_result_locates_models_and_is_kept_and_logged(self):
|
||||
sub = self.dir / "sub"
|
||||
sub.mkdir()
|
||||
self.report.write_text("\n".join(json.dumps(line) for line in (
|
||||
{"file": str(sub / "a.safetensors"), "status": "matched", "name": "A · v1", "cover": "new"},
|
||||
{"file": str(self.dir / "b.safetensors"), "status": "inferred", "reason": "network", "base": "Flux.1 D"},
|
||||
{"file": str(self.dir / "old.safetensors"), "status": "unchanged", "renamed_to": str(self.dir / "New_v1.safetensors")},
|
||||
{"event": "civitai_down", "reason": "refused"},
|
||||
{"event": "done", "unchanged": 5},
|
||||
)), encoding="utf-8")
|
||||
log = self.dir / "activity_log.json"
|
||||
with mock.patch.object(scan_report, "log_path", return_value=str(log)), \
|
||||
mock.patch.object(activity_log, "log_path", return_value=str(log)):
|
||||
activity_log._entries = None
|
||||
job = scan_report.ScanJob("all", {"offline_only": False})
|
||||
job.add_folder(str(self.report), str(self.dir), ("loras", 0))
|
||||
result = job.finish()
|
||||
entry = activity_log.list_entries()[0]
|
||||
activity_log._entries = None
|
||||
self.assertFalse(self.report.exists())
|
||||
self.assertEqual(result["counts"], {"matched": 1, "inferred": 1, "failed": 0, "covers": 1, "renamed": 1, "unchanged": 5})
|
||||
self.assertTrue(result["civitai_down"])
|
||||
self.assertEqual(result["civitai_error"], "refused")
|
||||
first, _, renamed = result["files"]
|
||||
self.assertEqual((first["type"], first["rel"], first["filename"]), ("loras", "sub/a.safetensors", "a.safetensors"))
|
||||
self.assertEqual((renamed["filename"], renamed["renamed_from"]), ("New_v1.safetensors", "old.safetensors"))
|
||||
self.assertEqual(json.loads((self.dir / "last_scan.json").read_text(encoding="utf-8"))["counts"]["matched"], 1)
|
||||
self.assertEqual(entry["action"], "scan_done")
|
||||
self.assertEqual(entry["detail"]["scan"]["counts"]["inferred"], 1)
|
||||
self.assertEqual(len(entry["detail"]["files"]), 3)
|
||||
|
||||
def test_summary_lists_unmatched_and_new_models_with_why(self):
|
||||
self.model("new")
|
||||
self.model("gone", {"id": -1, "anomalous_unmatched_reason": "not_found", "baseModel": "SD 1.5"})
|
||||
self.model("off", {"id": -1, "anomalous_unmatched_reason": "offline"})
|
||||
self.model("found", {"id": 3})
|
||||
with mock.patch.object(scan_summary, "get_active_scan_paths", return_value=[str(self.dir)]), \
|
||||
mock.patch.object(scan_summary, "locate_root", return_value=("loras", 1)):
|
||||
summary = scan_summary._summarize()
|
||||
self.assertEqual((summary["total"], summary["matched"], summary["unmatched"], summary["new"], summary["pending"]), (4, 1, 2, 1, 1))
|
||||
self.assertEqual(summary["new_models"], [{"type": "loras", "path_idx": 1, "filename": "new.safetensors", "rel": "new.safetensors"}])
|
||||
reasons = {item["filename"]: (item["reason"], item["base"]) for item in summary["unmatched_models"]}
|
||||
self.assertEqual(reasons, {"gone.safetensors": ("not_found", "SD 1.5"), "off.safetensors": ("offline", "")})
|
||||
|
||||
|
||||
class EditDetailTests(unittest.TestCase):
|
||||
def test_editing_a_model_logs_what_changed_from_what(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
model = Path(temp) / "m.safetensors"
|
||||
model.write_bytes(b"m")
|
||||
(Path(temp) / "m.anomalous.json").write_text(json.dumps({"custom_name": "Old"}), encoding="utf-8")
|
||||
request = make_mocked_request("POST", "/anomalous/update_metadata")
|
||||
body = {"type": "loras", "filename": "m.safetensors", "custom_name": "New", "custom_notes": ""}
|
||||
request.json = lambda: asyncio.sleep(0, body)
|
||||
with mock.patch.object(model_metadata, "resolve_folder_subdir", return_value=(temp, temp)):
|
||||
asyncio.run(model_metadata.api_update_metadata(request))
|
||||
self.assertEqual(request[activity_log.ACTIVITY_DETAIL], {"fields": [{"field": "custom_name", "before": "Old", "after": "New"}]})
|
||||
self.assertEqual(activity_log._clean_detail({"fields": [{"field": "custom_notes", "after": "x" * 900}]})["fields"][0]["after"][-1], "…")
|
||||
|
||||
|
||||
class ScanPlanTests(unittest.TestCase):
|
||||
def test_picked_models_scan_their_own_folders_and_bad_targets_are_refused(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
sub = str(Path(temp) / "sub")
|
||||
Path(sub).mkdir()
|
||||
resolve = lambda _type, _idx, subfolder: (temp, sub if subfolder == "/sub" else temp)
|
||||
with mock.patch.object(scanner, "resolve_folder_subdir", side_effect=resolve):
|
||||
plan = scanner._global_scan_plan({"targets": [
|
||||
{"type": "loras", "path_idx": 0, "subfolder": "/sub", "files": ["a.safetensors"]},
|
||||
{"type": "loras", "path_idx": 0, "subfolder": "/", "files": []}, # nothing picked there
|
||||
]})
|
||||
self.assertEqual(plan, [(sub, ("loras", 0), temp, ["a.safetensors"], "loras")])
|
||||
for bad in ({"targets": "loras"}, {"targets": [{"type": "loras", "files": "a.safetensors"}]}):
|
||||
with self.assertRaises(ValueError):
|
||||
scanner._global_scan_plan(bad)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,128 @@
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
PLUGIN_DIR = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(PLUGIN_DIR.parents[1]))
|
||||
sys.path.insert(0, str(PLUGIN_DIR))
|
||||
|
||||
import scraper
|
||||
from api import metadata, scan_summary, scanner
|
||||
from model_identity import computed_file_identity
|
||||
|
||||
DIGEST = "a" * 64
|
||||
|
||||
|
||||
def write_info(base, data):
|
||||
Path(base + ".info").write_text(json.dumps(data), encoding="utf-8")
|
||||
|
||||
|
||||
class ScanRetryTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
root = Path(self.temp.name)
|
||||
(root / "sub").mkdir()
|
||||
self.unmatched = root / "sub" / "unmatched.safetensors"
|
||||
self.matched = root / "matched.safetensors"
|
||||
self.new = root / "new.safetensors"
|
||||
for path in (self.unmatched, self.matched, self.new):
|
||||
path.write_bytes(b"model bytes")
|
||||
write_info(str(self.unmatched)[:-12], {
|
||||
"id": -1, "modelId": -1, "name": "unmatched", "files": [{"hashes": {"SHA256": DIGEST}}],
|
||||
"anomalous_file_identity": computed_file_identity(str(self.unmatched), DIGEST),
|
||||
})
|
||||
write_info(str(self.matched)[:-12], {"id": 5, "modelId": 2, "name": "v1"})
|
||||
Path(str(self.matched)[:-12] + ".preview.png").write_bytes(b"png")
|
||||
|
||||
def tearDown(self):
|
||||
self.temp.cleanup()
|
||||
|
||||
def run_scraper(self, *flags):
|
||||
fetched = []
|
||||
found = {"id": 7, "modelId": 3, "name": "v2", "model": {"name": "Found"}, "images": []}
|
||||
argv = ["scraper.py", self.temp.name, *flags]
|
||||
with mock.patch.object(sys, "argv", argv), \
|
||||
mock.patch.object(scraper, "fetch_civitai_info", side_effect=lambda h: fetched.append(h) or dict(found)), \
|
||||
mock.patch.object(scraper, "calculate_sha256", side_effect=AssertionError("hash recomputed")), \
|
||||
mock.patch.object(scraper.urllib.request, "urlopen", side_effect=OSError("offline test")):
|
||||
scraper.main()
|
||||
return fetched
|
||||
|
||||
def test_retry_refetches_only_unmatched_models_with_the_saved_hash(self):
|
||||
self.assertEqual(scraper.count_scan_files(self.temp.name, [], True), 1)
|
||||
fetched = self.run_scraper("--retry-unmatched", "--virtual-rename")
|
||||
self.assertEqual(fetched, [DIGEST])
|
||||
info = json.loads(Path(str(self.unmatched)[:-12] + ".info").read_text(encoding="utf-8"))
|
||||
self.assertEqual(info["id"], 7)
|
||||
self.assertEqual(json.loads(Path(str(self.matched)[:-12] + ".info").read_text(encoding="utf-8"))["id"], 5)
|
||||
self.assertFalse(Path(str(self.new)[:-12] + ".info").exists())
|
||||
|
||||
def test_models_found_offline_keep_their_name(self):
|
||||
# Named after the file as model and version: renaming doubled the name on every scan.
|
||||
for flags in (["--offline-only"], ["--force-overwrite"]):
|
||||
with mock.patch.object(sys, "argv", ["scraper.py", self.temp.name, "--physical-rename", "--virtual-rename", *flags]), \
|
||||
mock.patch.object(scraper, "fetch_civitai_info", return_value=None), \
|
||||
mock.patch.object(scraper, "calculate_sha256", return_value=DIGEST):
|
||||
scraper.main()
|
||||
self.assertTrue(self.new.exists())
|
||||
info = json.loads(Path(str(self.new)[:-12] + ".info").read_text(encoding="utf-8"))
|
||||
self.assertNotIn("anomalous_custom_name", info)
|
||||
self.assertEqual(metadata._read_metadata(str(self.new))["name"], "new")
|
||||
|
||||
def test_damaged_files_are_named_as_such(self):
|
||||
# "model bytes" is no safetensors header; a whole file next to it is only unknown to Civitai.
|
||||
header = json.dumps({"w": {"dtype": "F32", "shape": [1], "data_offsets": [0, 4]}}).encode()
|
||||
whole = Path(self.temp.name, "whole.safetensors")
|
||||
whole.write_bytes(len(header).to_bytes(8, "little") + header + b"\0" * 4)
|
||||
with mock.patch.object(sys, "argv", ["scraper.py", self.temp.name]), \
|
||||
mock.patch.object(scraper, "fetch_civitai_info", return_value=None), \
|
||||
mock.patch.object(scraper, "calculate_sha256", return_value=DIGEST):
|
||||
scraper.main()
|
||||
reason = lambda path: json.loads(Path(str(path)[:-12] + ".info").read_text(encoding="utf-8"))["anomalous_unmatched_reason"]
|
||||
self.assertEqual(reason(self.new), "damaged")
|
||||
self.assertEqual(reason(whole), "not_found")
|
||||
with mock.patch.object(scan_summary, "get_active_scan_paths", return_value=[self.temp.name]):
|
||||
self.assertEqual(scan_summary._summarize()["damaged"], 1)
|
||||
|
||||
def test_changed_file_is_hashed_again(self):
|
||||
self.unmatched.write_bytes(b"other bytes, other size")
|
||||
with mock.patch.object(scraper, "write_report") as report:
|
||||
self.run_scraper("--retry-unmatched")
|
||||
failed = [call.kwargs for call in report.call_args_list if call.kwargs.get("status") == "failed"]
|
||||
self.assertEqual([entry["error"] for entry in failed], ["hash recomputed"])
|
||||
|
||||
def test_summary_counts_each_kind_once(self):
|
||||
with mock.patch.object(scan_summary, "get_active_scan_paths", return_value=[self.temp.name, self.temp.name]):
|
||||
summary = scan_summary._summarize()
|
||||
counts = {key: summary[key] for key in ("total", "matched", "unmatched", "new")}
|
||||
self.assertEqual(counts, {"total": 3, "matched": 1, "unmatched": 1, "new": 1})
|
||||
|
||||
def test_summary_lists_model_files_in_other_formats_outside_hidden_folders(self):
|
||||
root = Path(self.temp.name)
|
||||
(root / "voice").mkdir()
|
||||
(root / "voice" / "speaker.pth").write_bytes(b"x")
|
||||
(root / ".staging").mkdir()
|
||||
(root / ".staging" / "half.ckpt").write_bytes(b"x")
|
||||
(root / "notes.txt").write_bytes(b"x")
|
||||
with mock.patch.object(scan_summary, "get_active_scan_paths", return_value=[self.temp.name]):
|
||||
summary = scan_summary._summarize()
|
||||
self.assertEqual(summary["total"], 3)
|
||||
self.assertEqual(summary["skipped"], 1)
|
||||
self.assertEqual(summary["skipped_models"][0]["rel"], "voice/speaker.pth")
|
||||
|
||||
def test_both_routes_map_request_fields_to_the_same_switches(self):
|
||||
data = {"offline_only": True, "virtual_rename": True, "retry_unmatched": True, "physical_rename": True}
|
||||
self.assertEqual(
|
||||
scanner._scraper_flags(data, False, False),
|
||||
["--offline-only", "--virtual-rename", "--retry-unmatched"],
|
||||
)
|
||||
self.assertIn("--skip-rename", scanner._scraper_flags({}, True, True))
|
||||
self.assertIn("--physical-rename", scanner._scraper_flags({"physical_rename": True}, False, True))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,167 @@
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
|
||||
PLUGIN_DIR = Path(__file__).resolve().parents[1]
|
||||
COMFY_ROOT = PLUGIN_DIR.parents[1]
|
||||
sys.path.insert(0, str(COMFY_ROOT))
|
||||
sys.path.insert(0, str(PLUGIN_DIR))
|
||||
|
||||
from api import scanner
|
||||
from scraper import count_scan_files, write_scan_progress
|
||||
|
||||
|
||||
class ScannerRecoveryTests(unittest.TestCase):
|
||||
class Request:
|
||||
query = {"type": "checkpoints", "path_idx": "0", "subfolder": "/"}
|
||||
|
||||
async def json(self):
|
||||
return {"skip_rename": True, "offline_only": True}
|
||||
|
||||
def test_legacy_marker_is_recovered_before_new_scan(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
marker = Path(temp_dir) / ".scan_in_progress"
|
||||
progress = Path(temp_dir) / ".scan_progress.json"
|
||||
marker.write_text("1", encoding="utf-8")
|
||||
progress.write_text("{}", encoding="utf-8")
|
||||
|
||||
claimed, recovered = scanner._claim_scan_marker(
|
||||
str(marker),
|
||||
"folder",
|
||||
(str(progress),),
|
||||
)
|
||||
|
||||
self.assertTrue(claimed)
|
||||
self.assertTrue(recovered)
|
||||
self.assertEqual(json.loads(marker.read_text(encoding="utf-8"))["version"], 2)
|
||||
scanner._release_scan_marker(str(marker), (str(progress),))
|
||||
self.assertFalse(marker.exists())
|
||||
|
||||
def test_active_marker_still_rejects_duplicate_scan(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
marker = str(Path(temp_dir) / ".global_scan_in_progress")
|
||||
first_claim, _ = scanner._claim_scan_marker(marker, "global")
|
||||
second_claim, _ = scanner._claim_scan_marker(marker, "global")
|
||||
scanner._release_scan_marker(marker)
|
||||
|
||||
self.assertTrue(first_claim)
|
||||
self.assertFalse(second_claim)
|
||||
|
||||
def test_dead_owner_marker_reports_interruption_and_cleans_artifacts(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
marker = Path(temp_dir) / ".scan_in_progress"
|
||||
progress = Path(temp_dir) / ".scan_progress.json"
|
||||
targets = Path(temp_dir) / ".scan_targets.json"
|
||||
marker.write_text(json.dumps({
|
||||
"version": 2,
|
||||
"session_id": "previous-session",
|
||||
"owner_pid": 999999999,
|
||||
"worker_pid": 0,
|
||||
}), encoding="utf-8")
|
||||
progress.write_text("{}", encoding="utf-8")
|
||||
targets.write_text("[]", encoding="utf-8")
|
||||
|
||||
status = scanner._scan_status_payload(
|
||||
str(marker),
|
||||
str(progress),
|
||||
(str(progress), str(targets)),
|
||||
)
|
||||
|
||||
self.assertFalse(status["scanning"])
|
||||
self.assertTrue(status["interrupted"])
|
||||
self.assertFalse(marker.exists())
|
||||
self.assertFalse(progress.exists())
|
||||
self.assertFalse(targets.exists())
|
||||
|
||||
def test_orphan_worker_remains_locked_only_while_process_is_alive(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
marker = Path(temp_dir) / ".scan_in_progress"
|
||||
progress = Path(temp_dir) / ".scan_progress.json"
|
||||
process = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(30)"])
|
||||
try:
|
||||
marker.write_text(json.dumps({
|
||||
"version": 2,
|
||||
"session_id": "previous-session",
|
||||
"owner_pid": 999999999,
|
||||
"worker_pid": process.pid,
|
||||
}), encoding="utf-8")
|
||||
progress.write_text("{}", encoding="utf-8")
|
||||
|
||||
active = scanner._scan_status_payload(str(marker), str(progress), (str(progress),))
|
||||
self.assertTrue(active["scanning"])
|
||||
self.assertFalse(active["interrupted"])
|
||||
finally:
|
||||
process.terminate()
|
||||
process.wait(timeout=5)
|
||||
|
||||
recovered = scanner._scan_status_payload(str(marker), str(progress), (str(progress),))
|
||||
self.assertFalse(recovered["scanning"])
|
||||
self.assertTrue(recovered["interrupted"])
|
||||
self.assertFalse(marker.exists())
|
||||
|
||||
def test_scraper_progress_uses_real_selected_file_count(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
root = Path(temp_dir)
|
||||
(root / "a.safetensors").write_bytes(b"a")
|
||||
(root / "b.safetensors").write_bytes(b"b")
|
||||
(root / "ignore.txt").write_text("x", encoding="utf-8")
|
||||
progress = root / ".scan_progress.json"
|
||||
|
||||
self.assertEqual(count_scan_files(str(root), []), 2)
|
||||
self.assertEqual(count_scan_files(str(root), ["b.safetensors"]), 1)
|
||||
write_scan_progress(str(progress), "scanning", 2, 1, "a.safetensors")
|
||||
payload = json.loads(progress.read_text(encoding="utf-8"))
|
||||
self.assertEqual(payload["current"], 1)
|
||||
self.assertEqual(payload["total"], 2)
|
||||
|
||||
def test_scraper_cli_publishes_completion_for_empty_folder(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
progress = Path(temp_dir) / ".scan_progress.json"
|
||||
result = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(PLUGIN_DIR / "scraper.py"),
|
||||
temp_dir,
|
||||
"--dry-run",
|
||||
"--progress-file",
|
||||
str(progress),
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
check=False,
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
payload = json.loads(progress.read_text(encoding="utf-8"))
|
||||
self.assertEqual(payload["phase"], "complete")
|
||||
self.assertEqual(payload["total"], 0)
|
||||
|
||||
def test_folder_scan_endpoint_completes_and_releases_marker(self):
|
||||
# The scan's result is kept in the ComfyUI user folder: patched until the worker is done.
|
||||
with tempfile.TemporaryDirectory() as temp_dir, \
|
||||
mock.patch.object(scanner, "resolve_folder_subdir", return_value=(temp_dir, temp_dir)), \
|
||||
mock.patch.object(scanner.ScanJob, "finish") as finish:
|
||||
response = asyncio.run(scanner.api_scan_folder(self.Request()))
|
||||
payload = json.loads(response.text)
|
||||
self.assertEqual(payload["status"], "ok")
|
||||
|
||||
marker = Path(temp_dir) / ".scan_in_progress"
|
||||
deadline = time.monotonic() + 5
|
||||
while marker.exists() and time.monotonic() < deadline:
|
||||
time.sleep(0.02)
|
||||
|
||||
self.assertFalse(marker.exists())
|
||||
self.assertTrue((Path(temp_dir) / ".scan_result.json").exists())
|
||||
finish.assert_called_once()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user