Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
71b55bc630 | ||
|
|
4c9cb7f5ea | ||
|
|
56d1072e58 | ||
|
|
e71d0ca6b6 | ||
|
|
6f442bf423 | ||
|
|
bf2fa3f50e | ||
|
|
2e9322d5c4 | ||
|
|
a1377cec36 | ||
|
|
fa98d0644d | ||
|
|
043403d09e | ||
|
|
7e11d9cae2 | ||
|
|
3295ff8b20 | ||
|
|
9640ca86f8 |
@@ -0,0 +1,21 @@
|
||||
# Left out of the Comfy Registry package (comfy node publish). Git clones still
|
||||
# have everything. tests/test_package_contents.py holds these rules to what
|
||||
# ComfyUI needs at run time; keep each one anchored to the root with a leading /.
|
||||
|
||||
# Development folders
|
||||
/tests/
|
||||
/tools/
|
||||
/docs/
|
||||
/.github/
|
||||
|
||||
# Screenshots used only by README-legacy.md on GitHub
|
||||
/images/
|
||||
|
||||
# Development files
|
||||
/TESTING.md
|
||||
/Makefile
|
||||
/requirements-dev.txt
|
||||
/.coveragerc
|
||||
/.flake8
|
||||
/.pre-commit-config.yaml
|
||||
/.gitignore
|
||||
@@ -17,6 +17,11 @@ updates:
|
||||
prefix: "deps(pip)"
|
||||
include: "scope"
|
||||
rebase-strategy: "auto"
|
||||
# These floors follow ComfyUI's requirements.txt, never the newest release.
|
||||
ignore:
|
||||
- dependency-name: "Pillow"
|
||||
- dependency-name: "aiohttp"
|
||||
- dependency-name: "numpy"
|
||||
# Optional: group non-breaking updates to reduce PR noise
|
||||
groups:
|
||||
python-minor-and-patch:
|
||||
|
||||
@@ -80,7 +80,7 @@ jobs:
|
||||
# pulled them from Ubuntu's apt mirror on every run, and three runs (#349, #378, #382) crawled at
|
||||
# ~26 kB/s for 11-12 minutes until the job hit its limit after every test had passed.
|
||||
container:
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble@sha256:47b510dd9230bf0b9bfc9b20102241e577ea7a99334ab4c3ca19f19770e9b8ea
|
||||
options: --user 1001
|
||||
timeout-minutes: 24
|
||||
steps:
|
||||
|
||||
@@ -53,7 +53,7 @@ jobs:
|
||||
# same pinned Playwright image as local runs (tag = @playwright/test in tools/vendor/package-lock.json;
|
||||
# tests/vendor/browser-image.test.mjs checks it). Browsers and fonts come from the image.
|
||||
container:
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble@sha256:47b510dd9230bf0b9bfc9b20102241e577ea7a99334ab4c3ca19f19770e9b8ea
|
||||
options: --user 1001
|
||||
timeout-minutes: 20
|
||||
# Public-repo ubuntu-latest runners have 4 vCPUs: three Playwright workers, each with its own test
|
||||
@@ -121,7 +121,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
# Same image as the shards: the raw files name sources by the container's workspace path.
|
||||
container:
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble@sha256:47b510dd9230bf0b9bfc9b20102241e577ea7a99334ab4c3ca19f19770e9b8ea
|
||||
options: --user 1001
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
@@ -192,7 +192,7 @@ jobs:
|
||||
# same pinned Playwright image as local runs (tag = @playwright/test in tools/vendor/package-lock.json;
|
||||
# tests/vendor/browser-image.test.mjs checks it). Browsers and fonts come from the image.
|
||||
container:
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble
|
||||
image: mcr.microsoft.com/playwright:v1.64.1-noble@sha256:47b510dd9230bf0b9bfc9b20102241e577ea7a99334ab4c3ca19f19770e9b8ea
|
||||
options: --user 1001
|
||||
timeout-minutes: 20
|
||||
# Public-repo ubuntu-latest runners have 4 vCPUs: three Playwright workers, each with its own test
|
||||
|
||||
@@ -12,10 +12,10 @@ Prompt Manager saves your prompts as you generate, links each image back to the
|
||||
|
||||
## What's new: rebuilt from the ground up
|
||||
|
||||
- **A new UI.** One app, the workspace, replaces the separate dashboard, gallery and metadata pages. The old pages still work if you want them.
|
||||
- **A new UI.** One app, the workspace, replaces the separate dashboard, gallery and metadata pages.
|
||||
- **Classic and Nodes 2.0.** The nodes work in both ComfyUI renderers. CI starts a real ComfyUI and checks both.
|
||||
- **Tested end to end.** Unit tests, browser tests in Chromium, Firefox, WebKit and touch emulation, and tests against a real ComfyUI.
|
||||
- **Mac, Linux and Windows.** The Python suite runs on all three, on Python 3.12, 3.13 and 3.14.
|
||||
- **Linux and Windows.** The Python suite runs on both, on Python 3.12, 3.13 and 3.14. macOS should work but is not tested.
|
||||
|
||||
## See it
|
||||
|
||||
@@ -212,9 +212,9 @@ Set `PROMPT_MANAGER_DATA_DIR` to an absolute folder to keep the library somewher
|
||||
- **Older images:** a one-time notice offers a metadata scan that links them to prompts and negatives. Scans don't count as new runs.
|
||||
- **Going back isn't supported.** Workflows are now stored once, and an older version shows new records without their workflow or prompt graph. Back up first in System > Backup & export. See [the old README's changelog](README-legacy.md#changelog) for the manual way back.
|
||||
|
||||
## The old pages still work
|
||||
## The old pages will be removed
|
||||
|
||||
They stay at their URLs, and System's Overview section links to them:
|
||||
They still work at their URLs for now and will be removed in a future release. Everything they do is moving to the workspace.
|
||||
|
||||
| Page | URL |
|
||||
| --- | --- |
|
||||
|
||||
+207
-20
@@ -77,7 +77,7 @@ PM_TEST_WORKERS=8 npm run --prefix tools/vendor test:coverage # whole
|
||||
npm run --prefix tools/vendor test:coverage:shard -- 1/4 # node units, viewer suite, workflows 1/4
|
||||
npm run --prefix tools/vendor test:coverage:shard -- 2/4 # workflows 2/4 (likewise 3/4, 4/4)
|
||||
npm run --prefix tools/vendor coverage:merge # report and baseline check of every raw file
|
||||
npm run --prefix tools/vendor test:workflows -- --project webkit --shard 2/2
|
||||
npm run --prefix tools/vendor test:workflows -- --project webkit --shard 2/3
|
||||
```
|
||||
|
||||
`coverage:merge` refuses an empty raw directory; it does not count shards, so run
|
||||
@@ -111,8 +111,11 @@ distributions, browser system dependencies may need manual installation. For a
|
||||
quick browser run, use `test:workflows -- --project chromium`.
|
||||
|
||||
CI runs the browser jobs inside the pinned image
|
||||
`mcr.microsoft.com/playwright:v<@playwright/test version>-noble`
|
||||
`mcr.microsoft.com/playwright:v<@playwright/test version>-noble@sha256:<digest>`
|
||||
(`tests/vendor/browser-image.test.mjs` keeps the tag in step with the lockfile).
|
||||
The digest is part of the pin, because Microsoft re-pushes tags: `v1.64.1-noble`
|
||||
moved from Node 24 to Node 26 on 2026-11-01 (#609). To bump it, change the tag and
|
||||
digest together and check the new image's Node against the Node 24 guard.
|
||||
Concept comparisons measure geometry, and geometry follows the fonts a browser
|
||||
falls back to: the concept names Inter, which neither the image nor a stock
|
||||
runner ships. On a plain `ubuntu-latest` runner the fallback is DejaVu Sans,
|
||||
@@ -190,8 +193,8 @@ waiting for the `frontend coverage` jobs.
|
||||
|
||||
The `viewerjs` check runs on every PR, including backend-only changes. It is an
|
||||
aggregate job. It requires the four `frontend coverage (shard N/4)` jobs and
|
||||
`frontend coverage (merge)` to succeed on every event. The four
|
||||
`page workflows (firefox|webkit, shard N/2)` jobs run only in the merge queue,
|
||||
`frontend coverage (merge)` to succeed on every event. The six
|
||||
`page workflows (firefox|webkit, shard N/3)` jobs run only in the merge queue,
|
||||
on `main` and on `workflow_dispatch` (see "Pull requests and the merge queue").
|
||||
`viewerjs` requires them to be skipped on a pull request and to succeed
|
||||
everywhere else. Any other failed, cancelled or skipped job fails it. Every browser job runs with `PM_TEST_WORKERS: 3` (the public
|
||||
@@ -665,6 +668,10 @@ Browser tests: `workspace-stale-source.spec.mjs` (`/__test__/sources/reverse` an
|
||||
`/__test__/scope-key/rotate`, both undone by every reset); the manual-Refresh
|
||||
recovery specs in `workspace-images.spec.mjs` answer every rescope 409.
|
||||
|
||||
## Registry package
|
||||
|
||||
`comfy node publish` zips the tracked files minus `.comfyignore`, so a new development file or folder at the root ships until it is listed there. `tests/test_package_contents.py` keeps each rule anchored to the root, checks that development paths are left out and that nothing ComfyUI needs at run time is. Git clones are unaffected.
|
||||
|
||||
## Windows portability preflight
|
||||
|
||||
Before changing paths, SQLite, streams, restore or cleanup:
|
||||
@@ -1886,7 +1893,8 @@ Inspect plan keeps D60–D66 (plan D60 → D83, D61 → D84, D65 → D88, D68
|
||||
engines in the mock's order with legacy labels and sizes (`WD14 SwinV2 · ~400
|
||||
MB`, `WD14 ViT · ~400 MB`, `GGUF JoyCaption (Recommended) · ~4.5 GB` in the
|
||||
mock's Ollama slot, `HuggingFace JoyCaption · ~16 GB`); the instructions start
|
||||
from the server's `default_prompt`.
|
||||
from the chosen engine's default saved instructions (#431, D204; before, the
|
||||
server's `default_prompt`).
|
||||
- **D91** (plan D68, ruling T3-bulk): a **Suggestions** number field (1–100) for
|
||||
every model, prefilled from `autotag.defaults.suggestion_limit` (10); a model
|
||||
change keeps it.
|
||||
@@ -2727,7 +2735,7 @@ the grid) and D43 (sorted chips, checked against the concept with its chips sort
|
||||
D44 needs no mapping: the comparison runs with no Refine filter active, so the button
|
||||
reads "Filters" as the concept's does. Nothing else follows the grid: the legacy
|
||||
"Current prompt library" link is removed from the shelf; the legacy Library stays at
|
||||
`/prompt_manager/admin` by direct URL, linked only from System (#525, D165)
|
||||
`/prompt_manager/admin` by direct URL (#525, #620)
|
||||
(`tests/browser/workspace.spec.mjs`). `library comparison detects
|
||||
mutations` injects `.card-footer { padding: 2px }` and removes `.card-meta .chip.red`.
|
||||
|
||||
@@ -2978,7 +2986,8 @@ change: ComfyUI needs a restart.
|
||||
A Pair save, by a run or the node's Save pair button, writes the category and tags to the positive prompt only when that
|
||||
save creates the prompt row (maintainer ruling). An existing prompt never changes on a run: empty, cleared or set by a person, including one first seen as
|
||||
another pair's negative. The negative stays empty, the saved pair keeps its own metadata, and `save_prompt_pair` writes
|
||||
no prompt metadata unless called with `fill_new_positive`. No backfill; optional repair is #550.
|
||||
no prompt metadata unless called with `fill_new_positive`. No backfill on a run; the opt-in repair for prompts older
|
||||
Pair runs saved is System › Maintenance › "Pair category and tags" (#550, "Pair category and tags repair" below).
|
||||
Tests: `tests/test_prompt_pair_node.py`, `tests/test_api_pairs.py`, `tests/comfyui/pair-node.spec.mjs`. A Python change: ComfyUI needs a restart.
|
||||
|
||||
### Delete… on the archived shelf (#492)
|
||||
@@ -3234,7 +3243,7 @@ npm run --prefix tools/vendor test:workflows -- history-concept.spec.mjs \
|
||||
|
||||
`#/inspect` ("Inspect a file") is a native workspace route; it no longer hands off to
|
||||
the legacy metadata page, which stays at its URL
|
||||
(`/prompt_manager/gallery.html`) linked only from System (#525, D165). The page is the concept's `#metadata`
|
||||
(`/prompt_manager/gallery.html`) at its URL only (#525, #620). The page is the concept's `#metadata`
|
||||
(app.js `renderMetadata` a740–748, selection a1715–1729; style.css 709–722, 1180–1194,
|
||||
1543–1545, 1559–1564) in the concept's own classes (`.surface`, `.file-drop`,
|
||||
`.spaced-20`, `.local-file-preview`, all from the synced `concept.css`), with no
|
||||
@@ -4307,6 +4316,12 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
primary stays the concept's plain button; the plan it opens has the primary.
|
||||
`duplicates dialog matches the concept` and `duplicates comparison detects mutations`
|
||||
(system-maintenance-concept.spec) pin the mapping.
|
||||
- **D192** (Maintenance, #550, beyond the concept): Database adds "Pair category and
|
||||
tags" after "Unused prompt components", a plain Review plan row (no danger mark: it
|
||||
deletes nothing) with the line "Fill an empty category or tags on prompts older Pair
|
||||
node runs saved, from their saved pairs. Only empty fields of never-edited prompts."
|
||||
`system-maintenance-concept.spec.mjs` maps it (`mapPairRow`) as a copy of Optimize
|
||||
database's row.
|
||||
- **D102** (Maintenance, S7, Q5, #222): Metadata cache clears only the browser entries
|
||||
Prompt Manager owns as a cache (`web/js/cache-ownership.js`). The inventory of every
|
||||
storage writer has none (see "Browser cache ownership" below), so the row's line adds
|
||||
@@ -4606,9 +4621,9 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
engine and policy (read-only) and opens "Review tag output images" in the concept's
|
||||
record-changing review (its `maintenancePlan` shell): `h3` "Tag output images", the
|
||||
`dl.meta-grid` Model, Existing tags, Images, Already auto-tagged, Images to tag
|
||||
("about N"), Time, Settings ("General <g> · Character <c>" or "Template of N
|
||||
characters", then " · up to L tags per image" with the preview's saved
|
||||
`suggestion_limit`) and Backup, the estimate sentence ("Nothing is written until you
|
||||
("about N"), Time, Settings ("General <g> · Character <c>", then " · up to L tags
|
||||
per image" with the preview's saved `suggestion_limit`; for a language model, since
|
||||
D205, an "Instructions" row and "Up to L tags per image") and Backup, the estimate sentence ("Nothing is written until you
|
||||
start. The number to tag is an estimate: images changed since they were tagged are
|
||||
tagged again, and missing files are skipped. Activity reports what was tagged.") in the
|
||||
`p.small`, the D55 backup box and Cancel / "Start tagging", which starts the
|
||||
@@ -4630,6 +4645,74 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
Cancel-only review matches the concept review shell` (system-tagging-concept.spec) and
|
||||
the batch tests of workspace-system-tagging.spec; the mutation check renames each grid
|
||||
label in turn.
|
||||
- **D202** (Auto-tagging, #431, beyond the concept): a "Skipped tags" block
|
||||
(`.setting-block`, `data-setting="tag-skipped"`, `data-key="autotag_skipped_tags"`)
|
||||
follows "Suggestions per image": the label and description in the concept's
|
||||
setting-row tokens ("Suggested tags that contain one of these words are hidden.
|
||||
Words are separated by spaces, _ or -; other characters such as ( ) belong to the
|
||||
word."), an "Add a word or words" input (`#tag-skipped-input`) with Add (Enter adds
|
||||
too), the entries as removable chips in the Tag image dialog's D83 chip markup (each
|
||||
with its count, title "Hides N tags", or muted "none" with "Hides no known tag"), and
|
||||
a `role="status"` preview line: "**Hides N tags from suggestions**: <up to 12>[ and K
|
||||
more]. Tags already on images are not changed." ("Checked against library tags only;
|
||||
no WD14 label list is downloaded." before the last sentence without a label list;
|
||||
"No tags are skipped." for an empty list; "Checking…" while waiting; "The preview
|
||||
could not be read." on failure). The client refuses an entry with "Enter a word.",
|
||||
"No commas or line breaks.", "At most 50 characters.", "Already in the list." or "The
|
||||
list is full (200)." under the input. The Tag image dialog's chip hint adds "N skipped
|
||||
tags were not suggested." when the server hid any (the static mock's copy). Pinned by
|
||||
the comparison's mapping (system-tagging-concept.spec: the block's tokens are the
|
||||
concept row's), `skipped tags add, preview, remove, save and reload (D202)`, `skipped
|
||||
tags preview checks library tags only …` (workspace-system-tagging.spec) and
|
||||
`suggestions omit skipped tags and say how many were hidden` (workspace-tagging.spec).
|
||||
- **D205** (Prompt library, #620, beyond the concept): the sort menu appends Recently used, Most used and Name Z–A
|
||||
after Oldest first. These are the old Dashboard's sorts, kept before its removal because the community asked for
|
||||
them. A prompt never run sorts last in both use orders.
|
||||
- **D203** (Auto-tagging, #431, beyond the concept): "Saved instructions" replaces the
|
||||
concept's "LLM tag template" field (the mock's state 2): a `.setting-block`
|
||||
(`data-setting="tag-instructions"`, `data-key="autotag_instructions"`) with the label,
|
||||
"Instructions for the language-model engines. Built-in can’t be changed; duplicate it
|
||||
to make your own. N of 20 saved · Built‑in is not counted.", a `role="listbox"` list
|
||||
(Built-in first with a "Read-only" chip, then the saved entries; "Default for GGUF" /
|
||||
"Default for HuggingFace" chips), "+ New" ("New instructions", "(2)" if taken),
|
||||
"Duplicate" ("<name> copy"), "Delete" (`.danger`, disabled for Built-in), "At most 20
|
||||
saved instructions." at the cap, and the editor beside the list (under it below 700px):
|
||||
"Name" ("N of 60") and "Text" ("N of 4,000 characters"), read-only for Built-in with
|
||||
"Built-in instructions can’t be changed. Duplicate them to make your own.", else the
|
||||
entry's problem ("The name “X” is already used.", "The text must not be empty."…).
|
||||
Then two concept select rows, "Default for GGUF JoyCaption" and "Default for
|
||||
HuggingFace JoyCaption" ("The Tag image dialog and batches start from this.",
|
||||
`data-key="autotag_default_instructions"`), and the note a deleted default leaves
|
||||
("GGUF now uses Built-in."; nothing is written until Save preferences). Pinned by the
|
||||
comparison's mapping (system-tagging-concept.spec: the block in the concept row's
|
||||
tokens, the two rows on the concept's select row) and `saved instructions: new, edit,
|
||||
per-engine default, delete, save and reload (D203)`, `a refused instructions save names
|
||||
the entry and highlights the block (D203)` (workspace-system-tagging.spec).
|
||||
- **D204** (Tag image and Auto-tag selected dialogs, #431, beyond the concept): for a
|
||||
language-model engine a "Saved instructions" select (`#image-tag-instructions-pick`,
|
||||
`#bulk-tag-instructions-pick`) sits above "Tagging instructions" (the mock's state 3),
|
||||
on the concept's select field: Built-in, then the saved entries (`GET /autotag/models`
|
||||
`instructions`, full text), the engine's default marked "<name> · default". It starts
|
||||
at that default; a pick replaces the text. The text stays editable for the run: while
|
||||
it differs from the picked entry the select shows a disabled "Edited for this run" and
|
||||
a muted line under the textarea reads "Edits apply to this run only. Manage saved
|
||||
instructions in System › Auto-tagging." Choosing an entry again replaces the edit.
|
||||
Switching between GGUF and HF starts from the new engine's default and drops an edit;
|
||||
WD14 hides both. A preset (Preview tagging review, the bulk review) shows the entry
|
||||
with its text, else "Edited for this run". When the read fails the select holds only
|
||||
Built-in with the client's copy (`BUILTIN_INSTRUCTIONS`, pinned equal to
|
||||
`DEFAULT_TAG_TEMPLATE` by workspace-image-tagging-state.test). Request bodies are
|
||||
unchanged (`options.instructions` is the textarea's text); there is no "Save as new".
|
||||
The workspace no longer reads `default_prompt`. Pinned by the tagging comparison's
|
||||
mapping (tagging-canonical-comparison.mjs: the select before the instructions),
|
||||
`the saved instructions picker fills, edits one-off and resets per engine (D204)`
|
||||
(workspace-tagging.spec) and `Auto-tag selected picks saved instructions and sends the
|
||||
text shown (D204)` (workspace-tagging-bulk.spec).
|
||||
- **D205** (Auto-tagging, #431; amends D134): for a language-model engine the batch
|
||||
review adds a read-only "Instructions" row with the drafted engine default's name
|
||||
before "Settings", which reads "Up to L tags per image"; the batch and Preview tagging
|
||||
review send that entry's text. WD14 is unchanged. Pinned by `the batch review names the
|
||||
language model's default instructions (D205)` (workspace-system-tagging.spec).
|
||||
- **D158** (Sources, #247 D-Q4): the concept has no switch for prompts from workflows
|
||||
without a PromptManager node; the maintainer chose a visible switch over a config key.
|
||||
A `.setting-row` in the concept's toggle markup follows "Watch for new outputs":
|
||||
@@ -4861,6 +4944,47 @@ replaced the database since the review; the plan is dropped). The job:
|
||||
prompt ids only), logs the backup by its file name, and prunes safety backups to the
|
||||
newest 10. With no backup the log and the status line say "No safety backup".
|
||||
|
||||
#### Pair category and tags repair (#550)
|
||||
|
||||
`pair_metadata` ("Pair category and tags", D192; `database/pair_metadata_repair.py`) fills
|
||||
the empty category and the empty tag set of prompts that Pair node runs before #546
|
||||
saved, from their saved pairs. It is reviewed and applied like the operations above and
|
||||
follows the safety backup policy like them (no forced backup). Rules
|
||||
(`docs/egm/post-3.4.1/550-design.md`):
|
||||
|
||||
- Considered: prompts the Pair node created as a positive: their earliest pair (lowest
|
||||
id, either role, archived included) has them as the positive and its `created_at` is
|
||||
within 2 s of theirs. A prompt first saved by the PromptManager node, or first as a
|
||||
negative, is out of scope and not counted.
|
||||
- Sources: the prompt's non-archived pairs where it is the positive; pairs with a blank
|
||||
category or no tags are ignored for that field. Category values are trimmed and
|
||||
case-sensitive, tag sets compared by tag id; a field fills when exactly one value is
|
||||
left. A whitespace-only category counts as empty.
|
||||
- Kept, per field, in this order: archived prompt, edited prompt (`updated_at` not within
|
||||
2 s of `created_at`; NULL or unreadable counts as edited), pairs that disagree, a tag
|
||||
set over 20 tags. A prompt can fill one field and keep the other.
|
||||
- The plan lists protected rows first, then fills, then a "Record" row, within the 50
|
||||
detail rows; at most 20,000 prompts per plan ("review again for the rest").
|
||||
- The fingerprint covers each considered prompt's computed result (values, tag ids and
|
||||
names, source pair ids), never a pair's title, notes, rating, run count or use
|
||||
stamps, so a Pair run between review and apply still applies.
|
||||
- The apply's writes are guarded (`category` still blank, no `prompt_tags` row) and a
|
||||
guard that hits nothing rolls the whole transaction back as `plan-changed`. It never
|
||||
writes `prompt_pairs`, `prompt_pair_tags`, `updated_at`, a tag or Library state.
|
||||
- The job lists 200 changed rows ("Prompt 12: category “night”; tags a, b"); its
|
||||
Download report lists every one (`truncated: false`, `Job.record_counted(complete=True)`).
|
||||
Protected lists one row per kept prompt in the review's wording and order ("Prompt 7:
|
||||
kept, pairs disagree on category: Adult, asian"), also complete in the report.
|
||||
- Activity shows a row with no path as a record: its detail as plain text, never an open
|
||||
or locate action, and the filter reads the detail. A maintenance job of records (every
|
||||
operation but Missing-file links) says "No records in this group.", "No listed record
|
||||
matches this filter." and "Filter records…" (`groupCopy`); file jobs keep "files".
|
||||
|
||||
Tests: `tests/test_maintenance_pair_metadata.py`, `tests/test_api_maintenance_pair_metadata.py`,
|
||||
`tests/test_jobs.py`, `tests/js/workspace-system-maintenance.test.js`,
|
||||
`tests/browser/workspace-system-maintenance.spec.mjs` (hook `/__test__/system/pair-metadata`).
|
||||
A Python change: ComfyUI needs a restart.
|
||||
|
||||
The legacy restore (`POST /prompt_manager/restore`) takes a backup of any size the disks
|
||||
have room for, so a real library's safety backup restores: the upload, its staged copy
|
||||
and the swap (the upload twice in the temporary folder, once more plus the current
|
||||
@@ -5215,16 +5339,16 @@ server-side tagging configuration, `autotag.defaults` in config.json: `model_typ
|
||||
(`wd14-swinv2`, `wd14-vit`, `gguf`, `hf`), `existing` (`skip`, `merge`,
|
||||
`replace_after_review`), `general_threshold` and `character_threshold` (0–1 in
|
||||
steps of 0.05, #369),
|
||||
`template` (non-blank, at most 4,000 characters and 16 KiB) and `suggestion_limit`
|
||||
(1–100). They are host drafts (`autotag_model_type`, `autotag_existing`,
|
||||
`autotag_general_threshold`, `autotag_character_threshold`, `autotag_template`,
|
||||
`autotag_suggestion_limit`) saved by the one Save row through `POST
|
||||
`suggestion_limit` (1–100), and since #431 `skipped_tags`, `instructions` and
|
||||
`default_instructions` (below). They are host drafts (`autotag_model_type`, `autotag_existing`,
|
||||
`autotag_general_threshold`, `autotag_character_threshold`, `autotag_suggestion_limit`,
|
||||
`autotag_skipped_tags`, `autotag_instructions`, `autotag_default_instructions`) saved by the one Save row through `POST
|
||||
/prompt_manager/settings` with `expected_revision`; a bad value is refused with a 400
|
||||
naming the setting by its row label ("General-tag confidence must be between 0 and
|
||||
1."), nothing is written, the Save row shows that sentence (`invalid-setting`, set by
|
||||
`settingRefusal` in system.mjs only for a save that carried `autotag_*` keys; every
|
||||
other section's plain 400 keeps the generic status) and the control is highlighted.
|
||||
The template's limit counts code points, as Python's `len` does. A hand-edited bad value
|
||||
Instruction text limits count code points, as Python's `len` does. A hand-edited bad value
|
||||
falls back to its default on load, one key at a time. Every page shows the thresholds in
|
||||
steps of 0.05 (the System rows, the Tag image dialogs, the legacy sliders), so a saved
|
||||
threshold off that grid is refused ("General-tag confidence must be a multiple of
|
||||
@@ -5235,8 +5359,70 @@ stored exactly on the grid (#369). A per-request threshold keeps its own check:
|
||||
`validate_options` and starts the `autotag` job titled "Tag output images (legacy page)";
|
||||
the legacy `/autotag/single` only requires a number. `GET /autotag/models` reports the
|
||||
saved values (`wd14_general_threshold`, `wd14_character_threshold`, `suggestion_limit`,
|
||||
`default_prompt`, `model_type`, `existing`), so the Tag image and Auto-tag selected
|
||||
dialogs start from them; a preset (the bulk review, Preview tagging review) wins.
|
||||
`default_prompt`, `model_type`, `existing`, and since #431 `instructions` and
|
||||
`default_instructions`), so the Tag image and Auto-tag selected dialogs start from them;
|
||||
a preset (the bulk review, Preview tagging review) wins.
|
||||
|
||||
**Saved instructions** (#431, D203, D205): `autotag.defaults.instructions` holds the
|
||||
saved entries `[{id, name, text}]` in order (at most 20; `id` 12 lowercase hex made by
|
||||
the client, unique, `builtin` reserved; `name` trimmed, 1–60 characters, no control
|
||||
character, unique case-folded, not "Built-in"; `text` as `validate_template`), and
|
||||
`autotag.defaults.default_instructions` is `{gguf, hf}`, each `builtin` or a saved id.
|
||||
Built-in (`{id: "builtin", name: "Built-in", text: DEFAULT_TAG_TEMPLATE}`) is never
|
||||
stored; readers put it first (`AutotagConfig.all_instructions`), so a later release's
|
||||
built-in text reaches everyone who uses it. Refusals (400, the Save row shows them and
|
||||
highlights the editor): "Saved instructions: at most 20 entries besides Built-in.",
|
||||
"Saved instructions: entry N has an invalid id.", "… repeats the id of entry M.", "…
|
||||
needs a name.", "… name must be at most 60 characters.", "… name must not contain a line
|
||||
break.", "… name “X” is already used.", "Saved instructions: the name “Built-in” is
|
||||
reserved.", "Saved instructions: entry N text must be …", "Default for GGUF JoyCaption
|
||||
must be one of the saved instructions." (and HuggingFace). A save carrying either key
|
||||
without `expected_revision` is refused ("Reload the page and try again."); a save
|
||||
carrying one is paired with the other from the running configuration and both are
|
||||
written, plus the old `template` key as the GGUF default's text (deleted when that
|
||||
default is Built-in), so an older build reads that default as its one template. `POST`
|
||||
with `autotag_template` is refused ("LLM tag template was replaced by Saved instructions.
|
||||
Reload the page."); `GET` keeps returning it (the GGUF default's text, read-only) for one
|
||||
release and adds `autotag_builtin_instructions` (Built-in's text, read-only). Load: each
|
||||
key alone (a damaged list → Built-in only, a default naming nothing → Built-in, each with
|
||||
one warning), then the in-memory migration of the old `template`: with no `instructions`
|
||||
key, a template other than Built-in becomes "Custom template" (id = first 12 hex of its
|
||||
SHA-256), the default of both engines unless `default_instructions` is stored; with the
|
||||
key, a template matching no entry (edited on an older build) is appended as "Custom
|
||||
template (n)" with one INFO line (a full list: one WARNING, nothing imported). Nothing is
|
||||
written at load; the next save that carries either key writes the list.
|
||||
`AutotagConfig.default_instructions_text(engine)` feeds the legacy batch and single
|
||||
blank-prompt fallbacks (by the request's engine) and `default_prompt`. Tests:
|
||||
`tests/test_saved_instructions.py`, `tests/test_saved_instructions_routes.py`,
|
||||
`tests/js/workspace-system-instructions.test.js`.
|
||||
|
||||
**Skipped tags** (#431, D202): `autotag.defaults.skipped_tags`, host draft
|
||||
`autotag_skipped_tags`, a list of at most 200 entries, each trimmed, 1–50 code points,
|
||||
no comma or control character, at least one word, no two with the same words (refused
|
||||
with "Skipped tags: at most 200 entries.", "Skipped tags: entry N is empty.", "… must be
|
||||
at most 50 characters.", "… must not contain a comma or a line break.", "… repeats entry
|
||||
M."); a save carrying it without `expected_revision` is refused ("Reload the page and try
|
||||
again."). A damaged saved list falls back to none with one warning. Words are the
|
||||
NFC, case-folded text split on runs of space, `_` and `-` (`tag_words`); an entry hides
|
||||
a suggested tag whose words contain the entry's words consecutively (`SkipMatcher`:
|
||||
`blurry` hides `very_blurry` and `blurry_background`, not `unblurry`; `(fate)` hides
|
||||
`saber_(fate)`, `fate` does not). `generate_tag_suggestions(skip=)` drops a match after
|
||||
the name check and before the dedupe and the limit, so skipped tags use up no slot
|
||||
(WD14 fills from lower scores; a language model may return fewer). Every path filters:
|
||||
`/image-annotations/suggest` (compiled per request; the answer adds `skipped`, the
|
||||
number of distinct names it hid), `/autotag/single`, and the `image-tags` (Auto-tag
|
||||
selected) and `autotag` (System batch, legacy `/autotag/start`) jobs, which snapshot the
|
||||
list at their start and log "Skipped tags: N entries" or "Skipped tags: none"; a save
|
||||
during a run does not change it. Hand-typed and stored tags are never filtered; the
|
||||
language models' hard-coded exclude list (`_parse_tags`) runs first. `POST
|
||||
/prompt_manager/autotag/skipped-tags/preview {entries}` (validated as a save) answers
|
||||
`count`, `sample` (12, library tags first, each part sorted case-folded),
|
||||
`sample_limit`, `per_entry [{entry, count}]` and `sources {library, wd14}` over the
|
||||
library's tags plus the category 0/4 names of each downloaded WD14 `selected_tags.csv`
|
||||
(read as UTF-8, cached by mtime and size; a missing list is skipped). It loads no model,
|
||||
opens no image, writes nothing and is `409 busy` while an exclusive job runs. Tests:
|
||||
`tests/test_skipped_tags.py`, `tests/test_skipped_tags_routes.py`, the settings and
|
||||
config tests, `tests/js/workspace-system-skipped-tags.test.js`.
|
||||
|
||||
`GET /prompt_manager/autotag/models` reads the disk in the executor, never loads or
|
||||
downloads a model and carries no absolute path: per engine `name`, `label`,
|
||||
@@ -7309,7 +7495,7 @@ Deploying this backend needs a ComfyUI restart (Python and route changes).
|
||||
|
||||
`#/tags` ("Tags & categories", page title "Tags") is a native workspace route; it no
|
||||
longer hands off to the legacy tag manager. That page stays at
|
||||
`/prompt_manager/admin#/tags` linked only from System (#525, D165); tag search and sort, the untagged
|
||||
`/prompt_manager/admin#/tags` at its URL only (#525, #620); tag search and sort, the untagged
|
||||
entry (D149) and multi-tag OR (the Library's Tag match, D150) moved to the workspace
|
||||
(#444); bulk prompt delete stays on the legacy admin page (bulk tags: the Prompt
|
||||
library's selection bar, D151). A prompt's category is changed in the Prompt library
|
||||
@@ -7441,7 +7627,8 @@ The legacy pages (`/prompt_manager/admin`, `admin#/tags`, `gallery`, `gallery.ht
|
||||
their URLs, and admin's Metadata and Gallery buttons still open their legacy pages (maintainer
|
||||
ruling; pinned by `legacy-system-links.spec.mjs`).
|
||||
|
||||
- **D165** (beyond the concept, maintainer ruling 2026-10-30): System's Overview section ends
|
||||
- **D165 retired** (#620): the old pages show their removal notice since 3.5.1, so System no longer links them and the load failure below offers a reload and ComfyUI. The original ruling:
|
||||
**D165** (beyond the concept, maintainer ruling 2026-10-30): System's Overview section ends
|
||||
with one small line, "Legacy pages: Dashboard · Tags · Gallery · Metadata viewer"
|
||||
(`.legacy-pages`, new tabs), outside `.settings-layout`. The concept has no such line;
|
||||
`mapOverviewConcept` adds it to the page tree.
|
||||
|
||||
+5
-2
@@ -461,8 +461,11 @@ except Exception as e:
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
||||
|
||||
# One startup line. ASCII only: a Windows console using a legacy code page
|
||||
# cannot print emoji, and a UnicodeEncodeError here would abort loading the node.
|
||||
# One startup line, printed at import on purpose: it is the package's banner on
|
||||
# stdout, as ComfyUI's other custom nodes print theirs. Not a log record: the
|
||||
# console log handler hides INFO by default, so a logger would drop it. ASCII
|
||||
# only: a Windows console using a legacy code page cannot print emoji, and a
|
||||
# UnicodeEncodeError here would abort loading the node.
|
||||
print(
|
||||
f"\033[94m[ComfyUI-PromptManager] Version:\033[0m {get_version()}"
|
||||
f" - {len(NODE_CLASS_MAPPINGS)} tools loaded"
|
||||
|
||||
@@ -291,13 +291,6 @@ class AnnotationOperations:
|
||||
conn.execute(_BUMP_INDEX)
|
||||
return conn.execute(_REVISION, (byte_id,)).fetchone()[0]
|
||||
|
||||
def _current_set(self, conn, sha256, byte_id, revision) -> dict:
|
||||
return {
|
||||
"sha256": sha256,
|
||||
"tags": self._byte_tags(conn, byte_id),
|
||||
"tag_revision": revision,
|
||||
}
|
||||
|
||||
def read_image_tags(self, locator: str, sha256: str) -> dict:
|
||||
"""The stored set of these exact bytes, or an empty set at revision 0.
|
||||
|
||||
|
||||
+20
-25
@@ -16,18 +16,17 @@ check, and none of them carries a server path.
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import ntpath
|
||||
import os
|
||||
import sqlite3
|
||||
from pathlib import PurePosixPath, PureWindowsPath
|
||||
from urllib.parse import quote
|
||||
|
||||
try:
|
||||
from ..utils.hashing import register_prompt_functions
|
||||
from .models import normalize_image_path
|
||||
from .sqlite_uri import readonly_uri
|
||||
except ImportError:
|
||||
from utils.hashing import register_prompt_functions
|
||||
from database.models import normalize_image_path
|
||||
from database.sqlite_uri import readonly_uri
|
||||
|
||||
logger = logging.getLogger("prompt_manager.database.health")
|
||||
|
||||
@@ -89,25 +88,6 @@ def _counted(lines):
|
||||
return _result(count, detail)
|
||||
|
||||
|
||||
def readonly_uri(db_path, *, windows=None):
|
||||
"""The ``mode=ro`` SQLite URI of *db_path*, on this platform or the given one.
|
||||
|
||||
pathlib spells a UNC path ``\\\\server\\share\\x.db`` as
|
||||
``file://server/share/x.db``, a remote authority SQLite refuses to open.
|
||||
With an empty authority (``file:////server/share/x.db``) SQLite hands
|
||||
Windows the ``//server/share`` path, which it opens as the UNC path.
|
||||
"""
|
||||
if windows is None:
|
||||
windows = os.name == "nt"
|
||||
db_path = os.path.abspath(db_path)
|
||||
if not windows:
|
||||
return PurePosixPath(db_path).as_uri() + "?mode=ro"
|
||||
path = PureWindowsPath(db_path)
|
||||
if ntpath.splitdrive(str(path))[0].startswith("\\\\"):
|
||||
return "file:" + quote("//" + path.as_posix()) + "?mode=ro"
|
||||
return path.as_uri() + "?mode=ro"
|
||||
|
||||
|
||||
def _integrity(conn, _roots):
|
||||
rows = [
|
||||
row[0] for row in conn.execute(f"PRAGMA integrity_check({INTEGRITY_LIMIT})")
|
||||
@@ -199,6 +179,23 @@ def link_states(rows, roots, readable):
|
||||
check's ``missing_files`` and the maintenance missing-link plan both
|
||||
judge links this way, so their counts agree.
|
||||
"""
|
||||
for row, owner, state in _placed_links(rows, roots, readable):
|
||||
if state is None:
|
||||
state = "present" if os.path.exists(row[1]) else "missing"
|
||||
yield row, owner, state
|
||||
|
||||
|
||||
def held_links(rows, roots, readable):
|
||||
"""``(row, owner, state)`` for each link ``link_states`` holds back
|
||||
(``unavailable`` or ``outside``), without looking at any file (#586)."""
|
||||
for row, owner, state in _placed_links(rows, roots, readable):
|
||||
if state is not None:
|
||||
yield row, owner, state
|
||||
|
||||
|
||||
def _placed_links(rows, roots, readable):
|
||||
"""``link_states`` before any file is looked at: state None means the
|
||||
link lies under a readable source."""
|
||||
prefixes = [_root_prefix(root) for root in roots]
|
||||
for row in rows:
|
||||
owner = _image_owner(row[1], row[3], prefixes)
|
||||
@@ -206,10 +203,8 @@ def link_states(rows, roots, readable):
|
||||
yield row, None, "outside"
|
||||
elif not readable[owner]:
|
||||
yield row, owner, "unavailable"
|
||||
elif os.path.exists(row[1]):
|
||||
yield row, owner, "present"
|
||||
else:
|
||||
yield row, owner, "missing"
|
||||
yield row, owner, None
|
||||
|
||||
|
||||
def _unavailable_line(unavailable, *, partial):
|
||||
|
||||
@@ -97,6 +97,9 @@ _ORDERS = {
|
||||
"newest": ("COALESCE(julianday(p.created_at),0)", "DESC"),
|
||||
"oldest": ("COALESCE(julianday(p.created_at),0)", "ASC"),
|
||||
"name": (_NAME, "ASC"),
|
||||
"name-desc": (_NAME, "DESC"),
|
||||
"most-used": ("COALESCE(p.run_count,0)", "DESC"),
|
||||
"last-used": ("COALESCE(julianday(p.last_used_at),0)", "DESC"),
|
||||
"rating": ("COALESCE(p.rating,0)", "DESC"),
|
||||
}
|
||||
# Pair archive scopes for variant and image reads, as conditions on alias pp.
|
||||
|
||||
@@ -47,11 +47,11 @@ import sqlite3
|
||||
import time
|
||||
import types
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Callable, Dict, List, Optional, Tuple
|
||||
|
||||
from .models import BUSY_TIMEOUT_SECONDS, PromptModel
|
||||
from .sqlite_uri import readonly_uri
|
||||
|
||||
try:
|
||||
from ..utils import library_location as ll
|
||||
@@ -322,7 +322,7 @@ def _verify_copy(temp: str, counts, documents) -> None:
|
||||
"verify-failed",
|
||||
f"{MESSAGES['verify-failed']} {detail}" if detail else None,
|
||||
)
|
||||
uri = Path(os.path.abspath(temp)).as_uri() + "?mode=ro"
|
||||
uri = readonly_uri(temp)
|
||||
try:
|
||||
with contextlib.closing(sqlite3.connect(uri, uri=True)) as conn:
|
||||
copied = _describe(conn)
|
||||
|
||||
+95
-24
@@ -2,7 +2,8 @@
|
||||
|
||||
Four operations, each the native owner of a legacy ``/maintenance`` one and
|
||||
built on the same queries, so a preview counts what the legacy cleanup would
|
||||
change:
|
||||
change (``workflow_storage`` and #550's ``pair_metadata``, which fills old
|
||||
Pair prompts' empty category and tags, live in their own modules):
|
||||
|
||||
- ``duplicate_prompts``: prompts whose text is equal ignoring case and outer
|
||||
spaces (``DUPLICATE_PROMPTS_SQL``, legacy ``cleanup_duplicates``); each
|
||||
@@ -42,7 +43,7 @@ from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from .execution_contexts import LIVE_HINT_CTES
|
||||
from .health import BATCH_ROWS, link_states, readable_roots, readonly_uri
|
||||
from .health import BATCH_ROWS, held_links, link_states, readable_roots, readonly_uri
|
||||
from .image_removal_schema import TABLE as MARKS_TABLE
|
||||
from .image_removal_schema import clear_marks, has_marks_table
|
||||
from .pairs import PairDependencyError
|
||||
@@ -60,6 +61,9 @@ MAINTENANCE_OPERATIONS = (
|
||||
"missing_links",
|
||||
# Applied as its own batched job (database/workflow_storage.py).
|
||||
"workflow_storage",
|
||||
# #550: fill old Pair prompts' empty category and tags
|
||||
# (database/pair_metadata_repair.py).
|
||||
"pair_metadata",
|
||||
)
|
||||
MAX_DETAILS = 50 # detail rows a plan lists
|
||||
MAX_RESULT_ROWS = 200 # changed records an apply describes (the job lists 200)
|
||||
@@ -126,6 +130,16 @@ _UNLINKED_PROTECTIONS_SQL = (
|
||||
+ "), 0) FROM prompts p WHERE NOT EXISTS "
|
||||
"(SELECT 1 FROM generated_images gi WHERE gi.prompt_id = p.id)"
|
||||
)
|
||||
# Each prompt no image links to, with which protections keep it (#586); the
|
||||
# ones with none are filtered out in Python, so each predicate runs once.
|
||||
_UNLINKED_KEPT_SQL = (
|
||||
"WITH "
|
||||
+ LIVE_HINT_CTES
|
||||
+ " SELECT p.id, "
|
||||
+ ", ".join(f"({predicate})" for predicate, _many, _one in PROTECTIONS)
|
||||
+ " FROM prompts p WHERE NOT EXISTS "
|
||||
"(SELECT 1 FROM generated_images gi WHERE gi.prompt_id = p.id) ORDER BY p.id"
|
||||
)
|
||||
|
||||
# Duplicate prompt groups, as the legacy cleanup finds them (case-insensitive
|
||||
# text; GROUP_CONCAT order is sorted by created_at afterwards).
|
||||
@@ -155,6 +169,14 @@ _PROMPT_STATE_SQL = (
|
||||
"ORDER BY id",
|
||||
)
|
||||
|
||||
# Why the missing-link cleanup holds a link back: (one, many) (#586).
|
||||
HELD_LINK_REASONS = {
|
||||
"unavailable": (
|
||||
"link under an unavailable source",
|
||||
"links under unavailable sources",
|
||||
),
|
||||
"outside": ("link outside the image sources", "links outside the image sources"),
|
||||
}
|
||||
NO_FILES_DELETED = "No image file is deleted."
|
||||
LINKS_ONLY = "Database links only; no image file is deleted."
|
||||
UNUSED_DETAIL = "No image, saved pair, Library state or recorded run uses it"
|
||||
@@ -168,6 +190,7 @@ CONFLICT_REASONS = {
|
||||
"groups have their own Library titles",
|
||||
),
|
||||
}
|
||||
PAIRED_DETAIL = "Kept: belongs to a saved pair"
|
||||
CONFLICT_DETAILS = {
|
||||
"collision": "Kept: two copies link the same image",
|
||||
"title": "Kept: a copy has its own Library title",
|
||||
@@ -409,6 +432,35 @@ def _source_name(root, index):
|
||||
return os.path.basename(str(root).rstrip("\\/")) or f"Source {index + 1}"
|
||||
|
||||
|
||||
def _kept_duplicates(paired, conflicted):
|
||||
"""``{prompt_id, reasons}`` for each prompt of a group kept whole (#586).
|
||||
|
||||
The reasons are the review's details without "Kept: ".
|
||||
"""
|
||||
held = [(ids, PAIRED_DETAIL) for ids in paired] + [
|
||||
(ids, CONFLICT_DETAILS[reason]) for ids, reason in conflicted
|
||||
]
|
||||
return [
|
||||
{"prompt_id": prompt_id, "reasons": [detail.removeprefix("Kept: ")]}
|
||||
for ids, detail in held
|
||||
for prompt_id in ids
|
||||
]
|
||||
|
||||
|
||||
def _kept_unused(conn):
|
||||
"""``{prompt_id, reasons}`` for each prompt the unused prune keeps (#586).
|
||||
|
||||
The reasons are the review's, one per protection that applies, in the
|
||||
order the plan names them.
|
||||
"""
|
||||
kept = []
|
||||
for row in conn.execute(_UNLINKED_KEPT_SQL):
|
||||
reasons = [one for flag, (_p, _m, one) in zip(row[1:], PROTECTIONS) if flag]
|
||||
if reasons:
|
||||
kept.append({"prompt_id": row[0], "reasons": reasons})
|
||||
return kept
|
||||
|
||||
|
||||
class MaintenanceOperations:
|
||||
"""Mixin for PromptDatabase: maintenance previews and their apply."""
|
||||
|
||||
@@ -533,6 +585,10 @@ class MaintenanceOperations:
|
||||
lambda conn: (self._link_rows(conn), _mark_rows(conn)),
|
||||
)
|
||||
return self._plan_missing_links(rows, roots, _stale_marks(marks, roots))
|
||||
if operation == "pair_metadata":
|
||||
from .pair_metadata_repair import plan_pair_metadata
|
||||
|
||||
return _read_snapshot(self.model.db_path, plan_pair_metadata)
|
||||
reader = (
|
||||
self._plan_duplicate_prompts
|
||||
if operation == "duplicate_prompts"
|
||||
@@ -584,7 +640,7 @@ class MaintenanceOperations:
|
||||
+ [
|
||||
{
|
||||
"label": _excerpt(texts.get(group[0])),
|
||||
"detail": "Kept: belongs to a saved pair",
|
||||
"detail": PAIRED_DETAIL,
|
||||
}
|
||||
for group in paired
|
||||
]
|
||||
@@ -662,6 +718,24 @@ class MaintenanceOperations:
|
||||
],
|
||||
)
|
||||
|
||||
def kept_links(self, roots):
|
||||
"""``{image_path, owner, reason}`` for each link the missing-link
|
||||
cleanup holds back under ``roots`` (blocking, reads only, no file
|
||||
opened); ``owner`` indexes ``roots`` (None outside every source).
|
||||
|
||||
``reason`` is the review's (``HELD_LINK_REASONS``), one link each.
|
||||
"""
|
||||
roots = [str(root) for root in roots]
|
||||
rows = _read_snapshot(self.model.db_path, self._link_rows)
|
||||
return [
|
||||
{
|
||||
"image_path": row[1],
|
||||
"owner": owner,
|
||||
"reason": HELD_LINK_REASONS[state][0],
|
||||
}
|
||||
for row, owner, state in held_links(rows, roots, readable_roots(roots))
|
||||
]
|
||||
|
||||
@staticmethod
|
||||
def _links_fingerprint(candidates, stale=()):
|
||||
fingerprint = _Fingerprint("missing_links")
|
||||
@@ -681,23 +755,11 @@ class MaintenanceOperations:
|
||||
unavailable += 1
|
||||
elif state == "outside":
|
||||
outside += 1
|
||||
reasons = []
|
||||
if unavailable:
|
||||
reasons.append(
|
||||
_plural(
|
||||
unavailable,
|
||||
"link under an unavailable source",
|
||||
"links under unavailable sources",
|
||||
)
|
||||
)
|
||||
if outside:
|
||||
reasons.append(
|
||||
_plural(
|
||||
outside,
|
||||
"link outside the image sources",
|
||||
"links outside the image sources",
|
||||
)
|
||||
)
|
||||
reasons = [
|
||||
_plural(count, *HELD_LINK_REASONS[state])
|
||||
for state, count in (("unavailable", unavailable), ("outside", outside))
|
||||
if count
|
||||
]
|
||||
count = len(candidates)
|
||||
parts = []
|
||||
if count:
|
||||
@@ -800,7 +862,11 @@ class MaintenanceOperations:
|
||||
records changed and up to ``MAX_RESULT_ROWS`` descriptions of them
|
||||
(``{prompt_id[, kept_id]}`` or ``{image_path}``). ``marks`` are a
|
||||
missing-links plan's stale removal marks; that apply also returns
|
||||
``marks``, how many it forgot.
|
||||
``marks``, how many it forgot. ``pair_metadata`` returns every filled
|
||||
prompt, uncapped (its job keeps the complete record, #550). The
|
||||
prompt operations also return ``kept``: every prompt the plan keeps,
|
||||
``{prompt_id, reasons}`` in the review's wording (#550, #586); the
|
||||
links a missing-links plan keeps are read by ``kept_links``.
|
||||
"""
|
||||
if operation not in MAINTENANCE_OPERATIONS or operation == "workflow_storage":
|
||||
raise ValueError(f"Unknown maintenance operation {operation!r}")
|
||||
@@ -815,6 +881,10 @@ class MaintenanceOperations:
|
||||
"unused_components": self._apply_unused_components,
|
||||
}
|
||||
with self._pair_transaction() as conn:
|
||||
if operation == "pair_metadata":
|
||||
from .pair_metadata_repair import apply_pair_metadata
|
||||
|
||||
return apply_pair_metadata(conn, fingerprint, targets)
|
||||
if operation == "missing_links":
|
||||
return self._apply_missing_links(
|
||||
conn, fingerprint, targets, roots, list(marks)
|
||||
@@ -822,10 +892,11 @@ class MaintenanceOperations:
|
||||
return appliers[operation](conn, fingerprint, targets, roots)
|
||||
|
||||
def _apply_duplicate_prompts(self, conn, fingerprint, targets, _roots):
|
||||
groups, _paired, _conflicted, current = self._duplicates_fingerprint(conn)
|
||||
groups, paired, conflicted, current = self._duplicates_fingerprint(conn)
|
||||
if current != fingerprint or groups != targets:
|
||||
raise PlanChanged("duplicate_prompts")
|
||||
changed, rows = 0, []
|
||||
kept = _kept_duplicates(paired, conflicted)
|
||||
for sorted_ids in groups:
|
||||
removed, _moved = self._merge_duplicate_group(conn, sorted_ids, strict=True)
|
||||
changed += removed
|
||||
@@ -833,7 +904,7 @@ class MaintenanceOperations:
|
||||
{"prompt_id": duplicate_id, "kept_id": sorted_ids[0]}
|
||||
for duplicate_id in sorted_ids[1:]
|
||||
)
|
||||
return {"changed": changed, "rows": rows[:MAX_RESULT_ROWS]}
|
||||
return {"changed": changed, "rows": rows[:MAX_RESULT_ROWS], "kept": kept}
|
||||
|
||||
def _apply_unused_components(self, conn, fingerprint, targets, _roots):
|
||||
ids, current = self._unused_fingerprint(conn)
|
||||
@@ -845,7 +916,7 @@ class MaintenanceOperations:
|
||||
{"prompt_id": prompt_id, "text": _excerpt(texts.get(prompt_id))}
|
||||
for prompt_id in ids[:MAX_RESULT_ROWS]
|
||||
]
|
||||
return {"changed": changed, "rows": rows}
|
||||
return {"changed": changed, "rows": rows, "kept": _kept_unused(conn)}
|
||||
|
||||
def _apply_missing_links(self, conn, fingerprint, targets, roots, marks):
|
||||
rows = []
|
||||
|
||||
+3
-4
@@ -14,6 +14,7 @@ from contextlib import closing
|
||||
from pathlib import Path
|
||||
from typing import Optional, Set, Tuple
|
||||
|
||||
from .sqlite_uri import readonly_uri
|
||||
from .managed_connection import (
|
||||
ManagedConnection,
|
||||
await_closed,
|
||||
@@ -1199,7 +1200,7 @@ class PromptModel:
|
||||
if header != SQLITE_HEADER:
|
||||
return False, "Not a SQLite database file"
|
||||
|
||||
uri = Path(os.path.abspath(path)).as_uri() + "?mode=ro"
|
||||
uri = readonly_uri(path)
|
||||
try:
|
||||
conn = sqlite3.connect(uri, uri=True)
|
||||
except sqlite3.Error as e:
|
||||
@@ -1306,9 +1307,7 @@ class PromptModel:
|
||||
replacement (for example, incompatible page sizes in WAL mode),
|
||||
propagate the error rather than copy files under other open owners.
|
||||
"""
|
||||
source = sqlite3.connect(
|
||||
Path(src_path).resolve().as_uri() + "?mode=ro", uri=True
|
||||
)
|
||||
source = sqlite3.connect(readonly_uri(Path(src_path).resolve()), uri=True)
|
||||
try:
|
||||
target = self.get_connection()
|
||||
# The target is written under backup's step too: mark it in use
|
||||
|
||||
@@ -0,0 +1,452 @@
|
||||
"""Pair category and tags repair (#550): the ``pair_metadata`` maintenance plan.
|
||||
|
||||
Before #546 a Pair node run saved its category and tags on the saved pair
|
||||
only; the positive prompt it created stayed Uncategorized and untagged. This
|
||||
plan fills the **empty** category and the **empty** tag set of those prompts
|
||||
from their saved pairs (``docs/egm/post-3.4.1/550-design.md``):
|
||||
|
||||
- Considered: prompts the Pair node created as a positive (their earliest
|
||||
pair, in either role, has them as the positive and its ``created_at`` is
|
||||
within ``PAIR_CREATED_TOLERANCE`` of theirs) whose category is blank
|
||||
and/or that have no tags.
|
||||
- Sources: the prompt's non-archived pairs where it is the positive. Pairs
|
||||
with a blank category or no tags are ignored for that field. A field fills
|
||||
when exactly one value remains (category: trimmed, case-sensitive; tags:
|
||||
the tag-id set).
|
||||
- Kept (protected), in this order: archived prompt, edited prompt
|
||||
(``updated_at`` not within the tolerance of ``created_at``), pairs that
|
||||
disagree, a tag set over ``MAX_LIBRARY_TAGS``.
|
||||
|
||||
The apply runs in the caller's ``BEGIN IMMEDIATE`` transaction, recomputes
|
||||
the fingerprint, then writes guarded statements (the field must still be
|
||||
empty) that never touch a pair, a negative or ``updated_at``. A guard that
|
||||
hits nothing raises ``PlanChanged`` so the whole transaction rolls back.
|
||||
"""
|
||||
|
||||
import datetime
|
||||
|
||||
from . import maintenance as _maintenance
|
||||
from .library_commands import MAX_LIBRARY_TAGS
|
||||
from .maintenance import (
|
||||
MAX_DETAILS,
|
||||
MaintenanceDraft,
|
||||
PlanChanged,
|
||||
_excerpt,
|
||||
_Fingerprint,
|
||||
_marks,
|
||||
_plural,
|
||||
)
|
||||
|
||||
OPERATION = "pair_metadata"
|
||||
# Seconds: the Pair save stamps its new prompt and its pair with separate
|
||||
# utc_now_iso() calls in one transaction (microseconds apart).
|
||||
PAIR_CREATED_TOLERANCE = 2.0
|
||||
# Fill prompts one plan applies; the rest are applied by the next review.
|
||||
# Bounds the complete record a job keeps in memory (about 100 bytes a row).
|
||||
MAX_PLAN_FILLS = 20000
|
||||
DETAIL_VALUES = 3
|
||||
DETAIL_TAGS = 5
|
||||
MEDIA_LABEL = "No image file or saved pair is changed."
|
||||
EMPTY_LABEL = "No Pair prompts are missing a category or tags"
|
||||
RECORD_NOTE = (
|
||||
"The job's report lists every filled field. Download it to keep it: "
|
||||
"Activity forgets jobs after a restart."
|
||||
)
|
||||
# Protected classes in precedence order: (key, detail, reason one, many).
|
||||
EDITED_DETAIL = (
|
||||
"Kept: edited after it was saved; its empty category or tags may be intentional"
|
||||
)
|
||||
PROTECTED = (
|
||||
(
|
||||
"archived",
|
||||
"Kept: archived in the Library",
|
||||
"archived in the Library",
|
||||
"archived in the Library",
|
||||
),
|
||||
(
|
||||
"edited",
|
||||
EDITED_DETAIL,
|
||||
"edited after it was saved",
|
||||
"edited after they were saved",
|
||||
),
|
||||
("conflict", None, "where pairs disagree", "where pairs disagree"),
|
||||
(
|
||||
"too-many-tags",
|
||||
f"Kept: its pair has more than {MAX_LIBRARY_TAGS} tags",
|
||||
f"whose pair has more than {MAX_LIBRARY_TAGS} tags",
|
||||
f"whose pair has more than {MAX_LIBRARY_TAGS} tags",
|
||||
),
|
||||
)
|
||||
_PROTECTED_ORDER = [key for key, *_rest in PROTECTED]
|
||||
# Detail rows: what the user can act on (a conflict) first.
|
||||
DETAIL_ORDER = ("conflict", "edited", "archived", "too-many-tags")
|
||||
|
||||
_CANDIDATES_SQL = """
|
||||
SELECT p.id, p.hash, p.text, p.created_at, p.updated_at,
|
||||
COALESCE(TRIM(p.category), '') = '' AS blank_category,
|
||||
NOT EXISTS (SELECT 1 FROM prompt_tags pt WHERE pt.prompt_id = p.id)
|
||||
AS untagged,
|
||||
EXISTS (SELECT 1 FROM prompt_archive pa WHERE pa.prompt_id = p.id)
|
||||
AS archived,
|
||||
(SELECT CASE WHEN pp.positive_prompt_id = p.id
|
||||
THEN pp.created_at END
|
||||
FROM prompt_pairs pp
|
||||
WHERE pp.positive_prompt_id = p.id OR pp.negative_prompt_id = p.id
|
||||
ORDER BY pp.id LIMIT 1) AS first_pair_at
|
||||
FROM prompts p
|
||||
WHERE EXISTS (SELECT 1 FROM prompt_pairs pp WHERE pp.positive_prompt_id = p.id)
|
||||
AND (COALESCE(TRIM(p.category), '') = ''
|
||||
OR NOT EXISTS (SELECT 1 FROM prompt_tags pt WHERE pt.prompt_id = p.id))
|
||||
ORDER BY p.id
|
||||
"""
|
||||
_SOURCES_SQL = (
|
||||
"SELECT id, positive_prompt_id, TRIM(COALESCE(category, '')) FROM prompt_pairs "
|
||||
"WHERE archived = 0 AND positive_prompt_id IN ({marks}) ORDER BY id"
|
||||
)
|
||||
_PAIR_TAGS_SQL = (
|
||||
"SELECT ppt.pair_id, t.id, t.name FROM prompt_pair_tags ppt "
|
||||
"JOIN tags t ON t.id = ppt.tag_id WHERE ppt.pair_id IN ({marks}) "
|
||||
"ORDER BY ppt.pair_id, t.id"
|
||||
)
|
||||
|
||||
|
||||
def parse_stamp(value):
|
||||
"""A stored timestamp as an aware UTC datetime, or None when unreadable."""
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
return None
|
||||
try:
|
||||
stamp = datetime.datetime.fromisoformat(value.strip())
|
||||
except ValueError:
|
||||
return None
|
||||
if stamp.tzinfo is None:
|
||||
stamp = stamp.replace(tzinfo=datetime.timezone.utc)
|
||||
return stamp
|
||||
|
||||
|
||||
def _close(first, second):
|
||||
"""Whether two stored stamps parse and lie within the tolerance."""
|
||||
a, b = parse_stamp(first), parse_stamp(second)
|
||||
if a is None or b is None:
|
||||
return False
|
||||
return abs((a - b).total_seconds()) <= PAIR_CREATED_TOLERANCE
|
||||
|
||||
|
||||
def _batches(ids):
|
||||
size = _maintenance.ID_BATCH
|
||||
for start in range(0, len(ids), size):
|
||||
yield ids[start : start + size]
|
||||
|
||||
|
||||
def _sources(conn, ids):
|
||||
"""``{prompt_id: [(pair_id, category, ((tag_id, name), ...)), ...]}``."""
|
||||
pairs = {}
|
||||
for batch in _batches(ids):
|
||||
for pair_id, prompt_id, category in conn.execute(
|
||||
_SOURCES_SQL.format(marks=_marks(batch)), batch
|
||||
):
|
||||
pairs.setdefault(prompt_id, []).append([pair_id, category, []])
|
||||
by_pair = {pair[0]: pair for group in pairs.values() for pair in group}
|
||||
pair_ids = sorted(by_pair)
|
||||
for batch in _batches(pair_ids):
|
||||
for pair_id, tag_id, name in conn.execute(
|
||||
_PAIR_TAGS_SQL.format(marks=_marks(batch)), batch
|
||||
):
|
||||
by_pair[pair_id][2].append((tag_id, name))
|
||||
return {
|
||||
prompt_id: [(pid, category, tuple(tags)) for pid, category, tags in group]
|
||||
for prompt_id, group in pairs.items()
|
||||
}
|
||||
|
||||
|
||||
def _field(values, guard):
|
||||
"""``(class, value)`` of one field from its distinct values, or None."""
|
||||
if not values:
|
||||
return None
|
||||
if guard:
|
||||
return (guard, None)
|
||||
if len(values) > 1:
|
||||
return ("conflict", sorted(values))
|
||||
return ("fill", next(iter(values)))
|
||||
|
||||
|
||||
def _classify(row, sources):
|
||||
"""The result of one considered prompt, or None when nothing applies."""
|
||||
guard = None
|
||||
if row["archived"]:
|
||||
guard = "archived"
|
||||
elif not _close(row["updated_at"], row["created_at"]):
|
||||
guard = "edited"
|
||||
category = tags = None
|
||||
tag_pair = None
|
||||
if row["blank_category"]:
|
||||
category = _field({c for _p, c, _t in sources if c}, guard)
|
||||
if row["untagged"]:
|
||||
sets = {}
|
||||
for pair_id, _category, pair_tags in sources:
|
||||
if pair_tags:
|
||||
sets.setdefault(frozenset(pair_tags), pair_id)
|
||||
tags = _field(set(sets), guard)
|
||||
if tags and tags[0] == "fill":
|
||||
tag_pair = sets[tags[1]]
|
||||
if len(tags[1]) > MAX_LIBRARY_TAGS:
|
||||
tags = ("too-many-tags", None)
|
||||
else:
|
||||
tags = ("fill", tuple(sorted(tags[1])))
|
||||
elif tags and tags[0] == "conflict":
|
||||
tags = ("conflict", [sorted(s) for s in sorted(sets, key=sorted)])
|
||||
if category is None and tags is None:
|
||||
return None
|
||||
return {
|
||||
"id": row["id"],
|
||||
"hash": row["hash"],
|
||||
"text": row["text"],
|
||||
"category": category,
|
||||
"tags": tags,
|
||||
"tag_pair": tag_pair if tags and tags[0] == "fill" else None,
|
||||
"pairs": [pair_id for pair_id, _c, _t in sources],
|
||||
}
|
||||
|
||||
|
||||
def _state(conn):
|
||||
"""``(results, fingerprint)``: every considered prompt's result, by id."""
|
||||
considered = []
|
||||
for row in conn.execute(_CANDIDATES_SQL):
|
||||
row = dict(
|
||||
zip(
|
||||
(
|
||||
"id",
|
||||
"hash",
|
||||
"text",
|
||||
"created_at",
|
||||
"updated_at",
|
||||
"blank_category",
|
||||
"untagged",
|
||||
"archived",
|
||||
"first_pair_at",
|
||||
),
|
||||
tuple(row),
|
||||
)
|
||||
)
|
||||
if _close(row["created_at"], row["first_pair_at"]):
|
||||
considered.append(row)
|
||||
sources = _sources(conn, [row["id"] for row in considered])
|
||||
fingerprint = _Fingerprint(OPERATION)
|
||||
results = []
|
||||
for row in considered:
|
||||
result = _classify(row, sources.get(row["id"], []))
|
||||
if result is None:
|
||||
continue
|
||||
results.append(result)
|
||||
fingerprint.add(
|
||||
[
|
||||
result["id"],
|
||||
result["hash"],
|
||||
result["category"],
|
||||
result["tags"],
|
||||
result["tag_pair"],
|
||||
result["pairs"],
|
||||
]
|
||||
)
|
||||
return results, fingerprint.hexdigest()
|
||||
|
||||
|
||||
def _fills(result):
|
||||
return (
|
||||
bool(result["category"] and result["category"][0] == "fill"),
|
||||
bool(result["tags"] and result["tags"][0] == "fill"),
|
||||
)
|
||||
|
||||
|
||||
def _targets(results):
|
||||
targets = []
|
||||
for result in results:
|
||||
category, tags = _fills(result)
|
||||
if category or tags:
|
||||
targets.append([result["id"], category, tags])
|
||||
return targets[:MAX_PLAN_FILLS]
|
||||
|
||||
|
||||
def _protected_classes(result):
|
||||
"""The protected classes of a result's fields, in precedence order."""
|
||||
found = {
|
||||
field[0]
|
||||
for field in (result["category"], result["tags"])
|
||||
if field and field[0] != "fill"
|
||||
}
|
||||
return [key for key in _PROTECTED_ORDER if key in found]
|
||||
|
||||
|
||||
def _more(values, limit):
|
||||
shown = ", ".join(values[:limit])
|
||||
return shown + (", …" if len(values) > limit else "")
|
||||
|
||||
|
||||
def _fill_detail(result):
|
||||
parts = []
|
||||
category, tags = _fills(result)
|
||||
if category:
|
||||
parts.append(f"category “{result['category'][1]}”")
|
||||
if tags:
|
||||
names = sorted(name for _id, name in result["tags"][1])
|
||||
noun = "tag" if len(names) == 1 else "tags"
|
||||
parts.append(f"{len(names)} {noun}: {_more(names, DETAIL_TAGS)}")
|
||||
pairs = [str(pair_id) for pair_id in result["pairs"]]
|
||||
label = "pair" if len(pairs) == 1 else "pairs"
|
||||
return f"Fills {' and '.join(parts)} ({label} {_more(pairs, DETAIL_VALUES)})"
|
||||
|
||||
|
||||
def _protected_detail(result, key):
|
||||
if key != "conflict":
|
||||
return next(detail for k, detail, *_r in PROTECTED if k == key)
|
||||
if result["category"] and result["category"][0] == "conflict":
|
||||
values = result["category"][1]
|
||||
return f"Kept: pairs disagree on category: {_more(values, DETAIL_VALUES)}"
|
||||
return "Kept: pairs disagree on tags"
|
||||
|
||||
|
||||
def _kept(results):
|
||||
"""One row per kept prompt, its review reasons without "Kept: " (#550)."""
|
||||
rows = []
|
||||
for result in results:
|
||||
classes = set(_protected_classes(result))
|
||||
if classes:
|
||||
rows.append(
|
||||
{
|
||||
"prompt_id": result["id"],
|
||||
"reasons": [
|
||||
_protected_detail(result, key).removeprefix("Kept: ")
|
||||
for key in DETAIL_ORDER
|
||||
if key in classes
|
||||
],
|
||||
}
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def _details(results, targets):
|
||||
rows = []
|
||||
for key in DETAIL_ORDER:
|
||||
for result in results:
|
||||
if key in _protected_classes(result):
|
||||
rows.append(
|
||||
{
|
||||
"label": _excerpt(result["text"]),
|
||||
"detail": _protected_detail(result, key),
|
||||
}
|
||||
)
|
||||
planned = {target[0] for target in targets}
|
||||
for result in results:
|
||||
if result["id"] in planned:
|
||||
rows.append(
|
||||
{"label": _excerpt(result["text"]), "detail": _fill_detail(result)}
|
||||
)
|
||||
if not targets:
|
||||
return rows[:MAX_DETAILS]
|
||||
return rows[: MAX_DETAILS - 1] + [{"label": "Record", "detail": RECORD_NOTE}]
|
||||
|
||||
|
||||
def _scope_label(targets, total):
|
||||
if not targets:
|
||||
return EMPTY_LABEL
|
||||
categories = sum(1 for _id, category, _tags in targets if category)
|
||||
tag_sets = sum(1 for _id, _category, tags in targets if tags)
|
||||
label = (
|
||||
f"{_plural(len(targets), 'prompt')} to fill: "
|
||||
f"{_plural(categories, 'category', 'categories')}, "
|
||||
f"{_plural(tag_sets, 'tag set')}"
|
||||
)
|
||||
if total > len(targets):
|
||||
label += (
|
||||
f" (the first {len(targets):,} of {total:,}; review again for the rest)"
|
||||
)
|
||||
return label
|
||||
|
||||
|
||||
def plan_pair_metadata(conn):
|
||||
"""The reviewed ``pair_metadata`` plan on ``conn``'s snapshot."""
|
||||
results, fingerprint = _state(conn)
|
||||
targets = _targets(results)
|
||||
total = sum(1 for result in results if any(_fills(result)))
|
||||
counts = {key: 0 for key in _PROTECTED_ORDER}
|
||||
held = 0
|
||||
for result in results:
|
||||
classes = _protected_classes(result)
|
||||
held += bool(classes)
|
||||
for key in classes:
|
||||
counts[key] += 1
|
||||
reasons = [
|
||||
f"{counts[key]:,} {one if counts[key] == 1 else many}"
|
||||
for key, _detail, one, many in PROTECTED
|
||||
if counts[key]
|
||||
]
|
||||
return MaintenanceDraft(
|
||||
operation=OPERATION,
|
||||
fingerprint=fingerprint,
|
||||
targets=targets,
|
||||
scope={"count": len(targets), "label": _scope_label(targets, total)},
|
||||
protected={"count": held, "reasons": reasons},
|
||||
media={"deletes_files": False, "label": MEDIA_LABEL},
|
||||
details=_details(results, targets),
|
||||
)
|
||||
|
||||
|
||||
def _fill_categories(conn, fills):
|
||||
"""Set each blank category; raises PlanChanged when one is no longer blank."""
|
||||
cursor = conn.executemany(
|
||||
"UPDATE prompts SET category = ? WHERE id = ? "
|
||||
"AND COALESCE(TRIM(category), '') = ''",
|
||||
fills,
|
||||
)
|
||||
if cursor.rowcount != len(fills):
|
||||
raise PlanChanged(OPERATION)
|
||||
|
||||
|
||||
def _fill_tags(conn, fills, expected):
|
||||
"""Copy each source pair's tags onto its untagged prompt."""
|
||||
cursor = conn.executemany(
|
||||
"INSERT INTO prompt_tags(prompt_id, tag_id) "
|
||||
"SELECT ?, ppt.tag_id FROM prompt_pair_tags ppt "
|
||||
"JOIN tags t ON t.id = ppt.tag_id WHERE ppt.pair_id = ? "
|
||||
"AND NOT EXISTS (SELECT 1 FROM prompt_tags WHERE prompt_id = ?)",
|
||||
fills,
|
||||
)
|
||||
if cursor.rowcount != expected:
|
||||
raise PlanChanged(OPERATION)
|
||||
|
||||
|
||||
def apply_pair_metadata(conn, fingerprint, targets):
|
||||
"""Apply the reviewed fills on ``conn`` (inside BEGIN IMMEDIATE).
|
||||
|
||||
Returns ``{changed, categories, tag_sets, rows, kept}``; ``rows`` lists
|
||||
every filled prompt (``{prompt_id, category?, tags?}``) and ``kept`` every
|
||||
prompt the plan keeps (``{prompt_id, reasons}``, in the review's wording
|
||||
and order), both never capped.
|
||||
"""
|
||||
results, current = _state(conn)
|
||||
if current != fingerprint or _targets(results) != targets:
|
||||
raise PlanChanged(OPERATION)
|
||||
by_id = {result["id"]: result for result in results}
|
||||
categories, tags, rows = [], [], []
|
||||
tag_count = 0
|
||||
for prompt_id, fill_category, fill_tags in targets:
|
||||
result = by_id[prompt_id]
|
||||
row = {"prompt_id": prompt_id}
|
||||
if fill_category:
|
||||
row["category"] = result["category"][1]
|
||||
categories.append((row["category"], prompt_id))
|
||||
if fill_tags:
|
||||
row["tags"] = sorted(name for _id, name in result["tags"][1])
|
||||
tags.append((prompt_id, result["tag_pair"], prompt_id))
|
||||
tag_count += len(result["tags"][1])
|
||||
rows.append(row)
|
||||
if categories:
|
||||
_fill_categories(conn, categories)
|
||||
if tags:
|
||||
_fill_tags(conn, tags, tag_count)
|
||||
return {
|
||||
"changed": len(rows),
|
||||
"categories": len(categories),
|
||||
"tag_sets": len(tags),
|
||||
"rows": rows,
|
||||
"kept": _kept(results),
|
||||
}
|
||||
@@ -42,6 +42,7 @@ from typing import Dict, Optional
|
||||
from . import workflow_documents as wd
|
||||
from .execution_history_schema import rotate_history_epoch
|
||||
from .safety_backup import take_safety_backup
|
||||
from .sqlite_uri import readonly_uri
|
||||
|
||||
logger = logging.getLogger("prompt_manager.database.restore_stage")
|
||||
|
||||
@@ -114,7 +115,7 @@ def file_digest(path: str) -> str:
|
||||
|
||||
|
||||
def _readonly(path: str) -> sqlite3.Connection:
|
||||
return sqlite3.connect(Path(path).resolve().as_uri() + "?mode=ro", uri=True)
|
||||
return sqlite3.connect(readonly_uri(Path(path).resolve()), uri=True)
|
||||
|
||||
|
||||
def _schema(path: str) -> list:
|
||||
|
||||
@@ -31,7 +31,15 @@ MAX_STATE_BYTES = 4096
|
||||
MAX_TAGS = 20
|
||||
MAX_TAG_LENGTH = 50
|
||||
_IMAGE_SORTS = ("newest", "oldest", "name", "name-desc", "rating")
|
||||
_LIBRARY_SORTS = ("newest", "oldest", "name", "rating")
|
||||
_LIBRARY_SORTS = (
|
||||
"newest",
|
||||
"oldest",
|
||||
"name",
|
||||
"name-desc",
|
||||
"most-used",
|
||||
"last-used",
|
||||
"rating",
|
||||
)
|
||||
_MEDIA = ("all", "image", "video")
|
||||
# How a Library view's Tags match (#444): every tag, any of them, or parents
|
||||
# without tags ("untagged" never goes with tags; the listing refuses both).
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
"""The read-only SQLite URI of a path, safe for Windows network shares."""
|
||||
|
||||
import ntpath
|
||||
import os
|
||||
from pathlib import PurePosixPath, PureWindowsPath
|
||||
from urllib.parse import quote
|
||||
|
||||
|
||||
def readonly_uri(db_path, *, windows=None):
|
||||
"""The ``mode=ro`` SQLite URI of *db_path*, on this platform or the given one.
|
||||
|
||||
pathlib spells a UNC path ``\\\\server\\share\\x.db`` as
|
||||
``file://server/share/x.db``, a remote authority SQLite refuses to open.
|
||||
With an empty authority (``file:////server/share/x.db``) SQLite hands
|
||||
Windows the ``//server/share`` path, which it opens as the UNC path.
|
||||
"""
|
||||
if windows is None:
|
||||
windows = os.name == "nt"
|
||||
db_path = os.path.abspath(db_path)
|
||||
if not windows:
|
||||
return PurePosixPath(db_path).as_uri() + "?mode=ro"
|
||||
path = PureWindowsPath(db_path)
|
||||
if ntpath.splitdrive(str(path))[0].startswith("\\\\"):
|
||||
return "file:" + quote("//" + path.as_posix()) + "?mode=ro"
|
||||
return path.as_uri() + "?mode=ro"
|
||||
@@ -112,20 +112,22 @@ QUERY_PLAN_STATEMENTS = {
|
||||
|
||||
|
||||
def _rewrite_row(conn, image_id, row, verified, checked):
|
||||
"""Store the row's inline texts as documents; returns (changed, protected, size).
|
||||
"""Store the row's inline texts as documents; returns (changed, kept, size).
|
||||
|
||||
``kept`` names the columns whose value stays inline (not text).
|
||||
|
||||
``verified`` maps document ids already read back in this batch to their
|
||||
text, and ``checked`` holds the reused ids ``store_text`` already
|
||||
checked, so a document shared by many rows is decompressed once.
|
||||
"""
|
||||
changes, replaced = {}, []
|
||||
protected = size = 0
|
||||
changes, replaced, kept = {}, [], []
|
||||
size = 0
|
||||
for index, (column, reference) in enumerate(TEXT_COLUMNS):
|
||||
value, current = row[index], row[index + len(TEXT_COLUMNS)]
|
||||
if value is None:
|
||||
continue
|
||||
if not storable(value):
|
||||
protected += 1
|
||||
kept.append(column)
|
||||
continue
|
||||
document_id = store_text(conn, value, checked)
|
||||
if document_id not in verified:
|
||||
@@ -143,7 +145,7 @@ def _rewrite_row(conn, image_id, row, verified, checked):
|
||||
[*changes.values(), image_id],
|
||||
)
|
||||
release_documents(conn, replaced)
|
||||
return bool(changes), protected, size
|
||||
return bool(changes), kept, size
|
||||
|
||||
|
||||
def migrate_batch(
|
||||
@@ -160,9 +162,10 @@ def migrate_batch(
|
||||
savepoint: a row that cannot be rewritten (``_row_error``) is rolled
|
||||
back alone, stays inline and is listed in ``errors``; anything else rolls
|
||||
the whole batch back and propagates. Returns ``{last_id, rows,
|
||||
protected, errors, done}``: the last row looked at, rows rewritten,
|
||||
inline values kept (not text), the ids kept by an error and whether
|
||||
there was nothing left to look at.
|
||||
protected, kept, errors, done}``: the last row looked at, rows
|
||||
rewritten, inline values kept (not text) and each of them as ``(image
|
||||
id, column)`` (#586), the ids kept by an error and whether there was
|
||||
nothing left to look at.
|
||||
"""
|
||||
if conn.in_transaction:
|
||||
raise RuntimeError("The migration needs its own transaction")
|
||||
@@ -174,6 +177,7 @@ def migrate_batch(
|
||||
"last_id": after_id,
|
||||
"rows": 0,
|
||||
"protected": 0,
|
||||
"kept": [],
|
||||
"errors": [],
|
||||
"done": not ids,
|
||||
}
|
||||
@@ -182,7 +186,7 @@ def migrate_batch(
|
||||
conn.execute("SAVEPOINT workflow_row")
|
||||
try:
|
||||
row = conn.execute(ROW_SQL, (image_id,)).fetchone()
|
||||
changed, protected, row_size = _rewrite_row(
|
||||
changed, kept, row_size = _rewrite_row(
|
||||
conn, image_id, row, verified, checked
|
||||
)
|
||||
except Exception as error:
|
||||
@@ -194,11 +198,12 @@ def migrate_batch(
|
||||
checked.clear()
|
||||
logger.warning(f"Image record {image_id} kept inline: {error}")
|
||||
result["errors"].append(image_id)
|
||||
changed, protected, row_size = False, 0, 0
|
||||
changed, kept, row_size = False, [], 0
|
||||
else:
|
||||
conn.execute("RELEASE workflow_row")
|
||||
result["rows"] += int(changed)
|
||||
result["protected"] += protected
|
||||
result["protected"] += len(kept)
|
||||
result["kept"].extend((image_id, column) for column in kept)
|
||||
result["last_id"] = image_id
|
||||
size += row_size
|
||||
if size >= max_bytes or _clock() - started >= max_seconds:
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 85 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 34 KiB |
@@ -3,7 +3,6 @@ PromptManager: Main custom node implementation that extends CLIPTextEncode
|
||||
with persistent prompt storage and search capabilities.
|
||||
"""
|
||||
|
||||
import time
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
|
||||
if __package__:
|
||||
@@ -259,19 +258,6 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
if not save_to_library:
|
||||
return result # #90: no metadata registration, no history text
|
||||
|
||||
# Register with ComfyUI integration for standard metadata compatibility
|
||||
node_id = f"promptmanager_{int(time.time() * 1000)}"
|
||||
self.comfyui_integration.register_prompt(
|
||||
node_id,
|
||||
encoding_text.strip(),
|
||||
{
|
||||
"category": category.strip() if category else None,
|
||||
"tags": extended_tags,
|
||||
"prompt_id": prompt_id,
|
||||
"prepend_text": prepend_text.strip() if prepend_text else None,
|
||||
"append_text": append_text.strip() if append_text else None,
|
||||
},
|
||||
)
|
||||
record_runtime_context(
|
||||
type(self).__name__, text, prepend_text, append_text, encoding_text
|
||||
)
|
||||
|
||||
@@ -19,7 +19,6 @@ except ImportError:
|
||||
|
||||
try:
|
||||
from .database.operations import PromptDatabase
|
||||
from .utils.comfyui_integration import get_comfyui_integration
|
||||
from .utils.image_monitor import get_image_monitor
|
||||
from .utils.prompt_tracker import get_prompt_tracker
|
||||
from .utils.hashing import generate_prompt_hash
|
||||
@@ -38,7 +37,6 @@ except ImportError:
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from database.operations import PromptDatabase
|
||||
from utils.comfyui_integration import get_comfyui_integration
|
||||
from utils.image_monitor import get_image_monitor
|
||||
from utils.prompt_tracker import get_prompt_tracker
|
||||
from utils.hashing import generate_prompt_hash
|
||||
@@ -86,7 +84,6 @@ class PromptManagerBase:
|
||||
self.db = PromptDatabase()
|
||||
self.prompt_tracker = get_prompt_tracker(self.db)
|
||||
self.image_monitor = get_image_monitor(self.db, self.prompt_tracker)
|
||||
self.comfyui_integration = get_comfyui_integration()
|
||||
|
||||
self._start_gallery_system()
|
||||
self.logger.debug(f"{self.__class__.__name__} node initialization completed")
|
||||
@@ -433,18 +430,6 @@ class PromptManagerBase:
|
||||
self.logger.error(f"Error searching prompts: {e}")
|
||||
return []
|
||||
|
||||
def search_prompts_api(self, search_text: str = "") -> List[Dict[str, Any]]:
|
||||
"""API method for JavaScript UI to search prompts."""
|
||||
return self._search_prompts(search_text=search_text)
|
||||
|
||||
def get_recent_prompts_api(self, limit: int = 20) -> List[Dict[str, Any]]:
|
||||
"""API method for JavaScript UI to get recent prompts."""
|
||||
try:
|
||||
return self.db.get_recent_prompts(limit=limit)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error getting recent prompts: {e}")
|
||||
return []
|
||||
|
||||
def _start_gallery_system(self):
|
||||
"""Initialize and start the gallery monitoring system."""
|
||||
try:
|
||||
@@ -455,13 +440,6 @@ class PromptManagerBase:
|
||||
self.logger.error(f"Failed to start gallery system: {e}")
|
||||
self.logger.warning("Gallery features will be disabled")
|
||||
|
||||
def get_gallery_status(self) -> Dict[str, Any]:
|
||||
"""Get status of the gallery system."""
|
||||
return {
|
||||
"image_monitor": self.image_monitor.get_status(),
|
||||
"prompt_tracker": self.prompt_tracker.get_status(),
|
||||
}
|
||||
|
||||
def cleanup_gallery_system(self):
|
||||
"""Clean up gallery system resources.
|
||||
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import time
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
|
||||
if __package__:
|
||||
@@ -173,19 +172,6 @@ class PromptManagerPair(PromptManagerFocused):
|
||||
self.logger.warning(f"Could not save or track paired prompt: {error}")
|
||||
if is_positive:
|
||||
self._mark_untracked(prompt, unique_id)
|
||||
try:
|
||||
self.comfyui_integration.register_prompt(
|
||||
f"promptmanager_pair_{time.time_ns()}",
|
||||
positive,
|
||||
{
|
||||
"prompt_id": prompt_id,
|
||||
"pair_id": pair_id if pair_used else None,
|
||||
"negative_text": negative_text,
|
||||
"category": category.strip() or None,
|
||||
},
|
||||
)
|
||||
except Exception as error:
|
||||
self.logger.warning(f"Could not register paired metadata: {error}")
|
||||
record_runtime_context(
|
||||
type(self).__name__,
|
||||
text,
|
||||
|
||||
@@ -3,7 +3,6 @@ PromptManagerText: A text-only version of PromptManager that outputs STRING
|
||||
without CLIP encoding, while maintaining all database and search features.
|
||||
"""
|
||||
|
||||
import time
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
|
||||
if __package__:
|
||||
@@ -230,19 +229,6 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
if not save_to_library:
|
||||
return result # #90: no metadata registration, no history text
|
||||
|
||||
# Register with ComfyUI integration for standard metadata compatibility
|
||||
node_id = f"promptmanagertext_{int(time.time() * 1000)}"
|
||||
self.comfyui_integration.register_prompt(
|
||||
node_id,
|
||||
final_text.strip(),
|
||||
{
|
||||
"category": category.strip() if category else None,
|
||||
"tags": extended_tags,
|
||||
"prompt_id": prompt_id,
|
||||
"prepend_text": prepend_text.strip() if prepend_text else None,
|
||||
"append_text": append_text.strip() if append_text else None,
|
||||
},
|
||||
)
|
||||
record_runtime_context(
|
||||
type(self).__name__, text, prepend_text, append_text, final_text
|
||||
)
|
||||
|
||||
@@ -89,7 +89,6 @@ from .source_rescope import SourceRescopeMixin
|
||||
|
||||
try:
|
||||
from ...database.operations import PromptDatabase
|
||||
from ...utils.file_prompt import parse_comfyui_prompt
|
||||
from ...utils.logging_config import get_logger
|
||||
from ...utils.native_memory import release_free_memory
|
||||
from ...utils.png_text import png_text
|
||||
@@ -101,7 +100,6 @@ except ImportError:
|
||||
0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
)
|
||||
from database.operations import PromptDatabase
|
||||
from utils.file_prompt import parse_comfyui_prompt
|
||||
from utils.logging_config import get_logger
|
||||
from utils.native_memory import release_free_memory
|
||||
from utils.png_text import png_text
|
||||
@@ -254,23 +252,6 @@ async def _read_json_body(request, max_bytes=JSON_BODY_MAX_BYTES):
|
||||
return data, None
|
||||
|
||||
|
||||
class _LongJobGuard:
|
||||
"""Async context manager handed out by ``_acquire_long_job``."""
|
||||
|
||||
def __init__(self, lock):
|
||||
self._lock = lock
|
||||
|
||||
async def __aenter__(self):
|
||||
# The caller checked ``locked()`` on this same loop iteration, so this
|
||||
# acquire never waits.
|
||||
await self._lock.acquire()
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
self._lock.release()
|
||||
return False
|
||||
|
||||
|
||||
# ── Gzip compression middleware ────────────────────────────────────────
|
||||
_GZIP_MIN_SIZE = 1024 # Only compress bodies larger than 1 KB
|
||||
_GZIP_TYPES = frozenset(
|
||||
@@ -426,10 +407,6 @@ class PromptManagerAPI(
|
||||
self._workspace_card_cache = {}
|
||||
# Favorite keys of each root's cached scan (images._favorite_listing).
|
||||
self._favorite_listing_cache = {}
|
||||
# Single-flight locks for long jobs, keyed by job family; see
|
||||
# _acquire_long_job(). Created lazily so no lock binds to an event
|
||||
# loop before the server's loop exists.
|
||||
self._long_jobs: dict = {}
|
||||
# This session's jobs (py/api/jobs.py): the metadata scan, and later
|
||||
# every other long operation the Activity page tracks.
|
||||
self._jobs = JobRegistry(self.logger)
|
||||
@@ -486,26 +463,6 @@ class PromptManagerAPI(
|
||||
_public_error = staticmethod(_public_error)
|
||||
_read_json_body = staticmethod(_read_json_body)
|
||||
|
||||
def _acquire_long_job(self, name):
|
||||
"""Claim the single-flight slot for a long job family, or None when busy.
|
||||
|
||||
Usage in a handler::
|
||||
|
||||
job = self._acquire_long_job("duplicates")
|
||||
if job is None:
|
||||
return self._long_job_busy_response("Duplicate scan")
|
||||
async with job:
|
||||
...
|
||||
|
||||
Families are free-form strings ("duplicates", "thumbnails",
|
||||
"autotag", ...); the lock is released when the ``async with`` block
|
||||
exits, including on exceptions. Must be called from the event loop.
|
||||
"""
|
||||
lock = self._long_jobs.setdefault(name, asyncio.Lock())
|
||||
if lock.locked():
|
||||
return None
|
||||
return _LongJobGuard(lock)
|
||||
|
||||
@staticmethod
|
||||
def _long_job_busy_response(label):
|
||||
"""409 envelope for a long job that is already running."""
|
||||
@@ -1079,7 +1036,3 @@ class PromptManagerAPI(
|
||||
f"Unreadable image {os.path.basename(str(image_path))}: {e}"
|
||||
)
|
||||
return {}
|
||||
|
||||
def _parse_comfyui_prompt(self, metadata):
|
||||
"""Parse ComfyUI and A1111 prompt data from metadata."""
|
||||
return parse_comfyui_prompt(metadata)
|
||||
|
||||
+3
-1
@@ -30,8 +30,10 @@ from .jobs import (
|
||||
)
|
||||
|
||||
try:
|
||||
from ...database.sqlite_uri import readonly_uri
|
||||
from ...utils.parallel import map_parallel
|
||||
except ImportError:
|
||||
from database.sqlite_uri import readonly_uri
|
||||
from utils.parallel import map_parallel
|
||||
|
||||
try:
|
||||
@@ -399,7 +401,7 @@ def _inspect_restore_upload(path):
|
||||
Returns:
|
||||
None when acceptable, otherwise a human-readable reason (blocking).
|
||||
"""
|
||||
uri = Path(os.path.abspath(path)).as_uri() + "?mode=ro"
|
||||
uri = readonly_uri(path)
|
||||
try:
|
||||
conn = sqlite3.connect(uri, uri=True)
|
||||
except sqlite3.Error as e:
|
||||
|
||||
@@ -102,11 +102,14 @@ def model_refusal(status: dict, model_type: str, model_name: str):
|
||||
return None
|
||||
|
||||
|
||||
def run_suggestions(service, capture_path: str, model_type: str, options: dict):
|
||||
def run_suggestions(
|
||||
service, capture_path: str, model_type: str, options: dict, skip=None
|
||||
):
|
||||
"""Blocking: load (or swap to) *model_type*, then suggest for the capture.
|
||||
|
||||
The caller holds the model gate for the whole call; nothing is ever
|
||||
downloaded (``load_model`` refuses a model that is not on disk).
|
||||
downloaded (``load_model`` refuses a model that is not on disk). ``skip``
|
||||
is the request's or job's compiled Skipped tags (#431).
|
||||
"""
|
||||
if not service.is_model_loaded() or service.get_loaded_model_type() != model_type:
|
||||
service.load_model(model_type, options["use_gpu"])
|
||||
@@ -116,6 +119,7 @@ def run_suggestions(service, capture_path: str, model_type: str, options: dict):
|
||||
prompt=options.get("instructions"),
|
||||
general_threshold=options.get("general_threshold"),
|
||||
character_threshold=options.get("character_threshold"),
|
||||
skip=skip,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -35,6 +35,7 @@ from .media_target import media_target as _target
|
||||
from .images import _unavailable_link, _verified_title
|
||||
from .prompts import bad_request
|
||||
from .. import model_gate
|
||||
from ..tagging_defaults import SkipMatcher
|
||||
|
||||
try:
|
||||
from ...database.annotations import (
|
||||
@@ -640,6 +641,8 @@ class AnnotationRoutesMixin:
|
||||
owner = model_gate.new_owner("image-annotations")
|
||||
if not model_gate.try_acquire(owner):
|
||||
return model_busy_response()
|
||||
# #431: the Skipped tags as they are when the request is accepted.
|
||||
skip = SkipMatcher(AutotagConfig.SKIPPED_TAGS).counting()
|
||||
try:
|
||||
# From here the pin is the gate's to end: the worker ends it,
|
||||
# or ``on_skipped`` does when the worker never starts.
|
||||
@@ -653,6 +656,7 @@ class AnnotationRoutesMixin:
|
||||
capture,
|
||||
model_type,
|
||||
options,
|
||||
skip,
|
||||
on_skipped=functools.partial(store.release_use, token),
|
||||
)
|
||||
except MissingBackendPackages as exc:
|
||||
@@ -677,6 +681,8 @@ class AnnotationRoutesMixin:
|
||||
"model_type": model_type,
|
||||
"options": options,
|
||||
"suggestions": suggestion_rows(pairs),
|
||||
# How many distinct suggestions the Skipped tags hid (#431).
|
||||
"skipped": len(skip.hidden),
|
||||
},
|
||||
headers={"Cache-Control": "no-store"},
|
||||
)
|
||||
@@ -732,10 +738,10 @@ def _byte_range(request: web.Request, size: int):
|
||||
return start, stop
|
||||
|
||||
|
||||
def _pinned_suggestions(store, token, service, capture, model_type, options):
|
||||
def _pinned_suggestions(store, token, service, capture, model_type, options, skip=None):
|
||||
"""Worker thread: infer on the capture, then end its pin (even on error)."""
|
||||
try:
|
||||
return run_suggestions(service, capture.path, model_type, options)
|
||||
return run_suggestions(service, capture.path, model_type, options, skip)
|
||||
finally:
|
||||
store.release_use(token)
|
||||
|
||||
|
||||
+120
-14
@@ -40,6 +40,12 @@ BATCH_PREVIEW_BODY_MAX = 1024
|
||||
APPLY_BODY_MAX = 64 * 1024 # POST /autotag/apply: one image and its tags
|
||||
_HEX64 = re.compile(r"[0-9a-f]{64}")
|
||||
BATCH_JOB_BODY_MAX = 32 * 1024
|
||||
# Skipped tags preview (#431): 200 entries of 50 characters, with room for JSON.
|
||||
SKIPPED_PREVIEW_BODY_MAX = 64 * 1024
|
||||
SKIPPED_PREVIEW_LEASE = "Skipped tags preview"
|
||||
SKIPPED_PREVIEW_SAMPLE = 12
|
||||
# WD14 label lists the preview found missing or unreadable (logged once each).
|
||||
_MISSING_LABEL_LISTS = set()
|
||||
REVIEW_REQUIRED = (
|
||||
"Replacing tags needs a review of each image. Use Preview tagging review, "
|
||||
"or choose Skip or Merge for a batch run."
|
||||
@@ -72,6 +78,12 @@ MODEL_DOWNLOAD_BODY_MAX = 1024
|
||||
_download_titles = {}
|
||||
|
||||
|
||||
def _instruction_engine(model_type: str) -> str:
|
||||
"""The language-model engine whose default instructions ``model_type``
|
||||
reads: itself, or GGUF for a WD14 engine (which takes none)."""
|
||||
return model_type if model_type in ("gguf", "hf") else "gguf"
|
||||
|
||||
|
||||
def _holder_label(holder: Optional[str]) -> str:
|
||||
if not holder:
|
||||
return ANOTHER_OPERATION
|
||||
@@ -247,6 +259,10 @@ class AutotagRoutesMixin:
|
||||
async def preview_autotag_batch_route(request):
|
||||
return await self.preview_autotag_batch(request)
|
||||
|
||||
@routes.post("/prompt_manager/autotag/skipped-tags/preview")
|
||||
async def preview_skipped_tags_route(request):
|
||||
return await self.preview_skipped_tags(request)
|
||||
|
||||
@routes.post("/prompt_manager/autotag/start")
|
||||
async def start_autotag_route(request):
|
||||
return await self.start_autotag(request)
|
||||
@@ -284,7 +300,15 @@ class AutotagRoutesMixin:
|
||||
"success": True,
|
||||
"models": public_models(status, loaded_type),
|
||||
# The saved tagging defaults (#328, System › Auto-tagging).
|
||||
"default_prompt": AutotagConfig.TEMPLATE,
|
||||
# The saved engine's default instructions (GGUF for a
|
||||
# WD14 engine); kept one release for cached pages (#431).
|
||||
"default_prompt": AutotagConfig.default_instructions_text(
|
||||
_instruction_engine(AutotagConfig.MODEL_TYPE)
|
||||
),
|
||||
# #431: Built-in first, with every text, and each
|
||||
# language-model engine's default.
|
||||
"instructions": AutotagConfig.all_instructions(),
|
||||
"default_instructions": dict(AutotagConfig.DEFAULT_INSTRUCTIONS),
|
||||
"model_loaded": loaded,
|
||||
"loaded_model_type": loaded_type,
|
||||
"wd14_general_threshold": AutotagConfig.GENERAL_THRESHOLD,
|
||||
@@ -539,7 +563,10 @@ class AutotagRoutesMixin:
|
||||
else:
|
||||
prompt = params.get("prompt")
|
||||
template = prompt if isinstance(prompt, str) and prompt.strip() else None
|
||||
options = {"instructions": template or AutotagConfig.TEMPLATE}
|
||||
options = {
|
||||
"instructions": template
|
||||
or AutotagConfig.default_instructions_text(model_type)
|
||||
}
|
||||
try:
|
||||
_model, options = validate_options(
|
||||
model_type,
|
||||
@@ -699,6 +726,86 @@ class AutotagRoutesMixin:
|
||||
return None, "existing must be one of " + ", ".join(EXISTING_POLICIES)
|
||||
return MODELS[model_type], None
|
||||
|
||||
async def preview_skipped_tags(self, request):
|
||||
"""Which known tags a Skipped tags list would hide (#431); no write.
|
||||
|
||||
``{entries}`` is checked as a save checks it (the same 400 sentence).
|
||||
The known tags are the library's plus the general and character names
|
||||
of each downloaded WD14 label list; no model is loaded and no image is
|
||||
opened. Refused (409 ``busy``) while an exclusive job runs.
|
||||
"""
|
||||
from ..tagging_defaults import validate_skipped_tags
|
||||
|
||||
data, error = await self._batch_body(
|
||||
request, ("entries",), SKIPPED_PREVIEW_BODY_MAX
|
||||
)
|
||||
if error is not None:
|
||||
return error
|
||||
try:
|
||||
entries = validate_skipped_tags(data["entries"])
|
||||
except ValueError as exc:
|
||||
return web.json_response({"success": False, "error": str(exc)}, status=400)
|
||||
try:
|
||||
async with self._jobs.lease(SKIPPED_PREVIEW_LEASE):
|
||||
result = await self._run_in_executor(self._skipped_preview, entries)
|
||||
except JobBusy as busy:
|
||||
return busy_response(busy)
|
||||
except Exception:
|
||||
self.logger.exception("The Skipped tags preview failed")
|
||||
return web.json_response(
|
||||
{"success": False, "error": _INTERNAL_ERROR}, status=500
|
||||
)
|
||||
return web.json_response({"success": True, **result})
|
||||
|
||||
def _skipped_preview(self, entries):
|
||||
"""Worker thread: the preview's counts over the known tags."""
|
||||
from ..autotag import MODELS, get_autotag_service
|
||||
from ..tagging_defaults import (
|
||||
WD14_MODEL_TYPES,
|
||||
SkipMatcher,
|
||||
wd14_label_names,
|
||||
)
|
||||
|
||||
library = self.db.get_all_tags()
|
||||
known = {} # case-folded -> (0 library / 1 label list, spelling)
|
||||
for name in library:
|
||||
known.setdefault(name.casefold(), (0, name))
|
||||
lists = []
|
||||
models_dir = getattr(get_autotag_service(), "models_dir", None)
|
||||
for model_type in sorted(WD14_MODEL_TYPES, key=lambda t: MODELS[t]["name"]):
|
||||
if models_dir is None:
|
||||
break
|
||||
path = Path(models_dir) / MODELS[model_type]["subdir"] / "selected_tags.csv"
|
||||
try:
|
||||
names = wd14_label_names(path)
|
||||
except (OSError, ValueError) as exc:
|
||||
if path not in _MISSING_LABEL_LISTS: # logged once per list
|
||||
_MISSING_LABEL_LISTS.add(path)
|
||||
self.logger.debug(f"No WD14 label list for {model_type}: {exc}")
|
||||
continue
|
||||
_MISSING_LABEL_LISTS.discard(path)
|
||||
lists.append(MODELS[model_type]["name"])
|
||||
for name in names:
|
||||
known.setdefault(name.casefold(), (1, name))
|
||||
matchers = [(entry, SkipMatcher([entry])) for entry in entries]
|
||||
whole = SkipMatcher(entries)
|
||||
hidden = [item for item in known.values() if whole.matches(item[1])]
|
||||
hidden.sort(key=lambda item: (item[0], item[1].casefold()))
|
||||
per_entry = [
|
||||
{
|
||||
"entry": entry,
|
||||
"count": sum(1 for _, name in hidden if matcher.matches(name)),
|
||||
}
|
||||
for entry, matcher in matchers
|
||||
]
|
||||
return {
|
||||
"count": len(hidden),
|
||||
"sample": [name for _, name in hidden[:SKIPPED_PREVIEW_SAMPLE]],
|
||||
"sample_limit": SKIPPED_PREVIEW_SAMPLE,
|
||||
"per_entry": per_entry,
|
||||
"sources": {"library": len(library), "wd14": lists},
|
||||
}
|
||||
|
||||
async def preview_autotag_batch(self, request):
|
||||
"""Review Tag output images: counts, model, backup and disk; no write.
|
||||
|
||||
@@ -836,16 +943,6 @@ class AutotagRoutesMixin:
|
||||
return refusal.response
|
||||
return web.json_response({"success": True, "job": job.summary()}, status=202)
|
||||
|
||||
def _gated_runner(self, owner):
|
||||
"""Executor calls under the model gate *owner* holds (plain when None)."""
|
||||
if owner is None:
|
||||
return self._run_in_executor
|
||||
|
||||
async def run(func, *args, **kwargs):
|
||||
return await model_gate.run(owner, func, *args, **kwargs)
|
||||
|
||||
return run
|
||||
|
||||
def _resolve_allowed_image_path(self, raw_path: str) -> Optional[str]:
|
||||
"""Return the real path when it is a media file inside an output directory."""
|
||||
if Path(str(raw_path)).suffix.lower() not in _MEDIA_EXTENSIONS:
|
||||
@@ -1002,7 +1099,10 @@ class AutotagRoutesMixin:
|
||||
else:
|
||||
prompt = data.get("prompt")
|
||||
template = prompt if isinstance(prompt, str) and prompt.strip() else None
|
||||
options = {"instructions": template or AutotagConfig.TEMPLATE}
|
||||
options = {
|
||||
"instructions": template
|
||||
or AutotagConfig.default_instructions_text(model_type)
|
||||
}
|
||||
use_gpu = data.get("use_gpu", True)
|
||||
options["use_gpu"] = use_gpu if isinstance(use_gpu, bool) else True
|
||||
return validate_options(
|
||||
@@ -1096,6 +1196,8 @@ class AutotagRoutesMixin:
|
||||
|
||||
async def _suggest_single(self, target, model_type, options, base):
|
||||
from ..autotag import MODELS, MissingBackendPackages, get_autotag_service
|
||||
from ..config import AutotagConfig
|
||||
from ..tagging_defaults import SkipMatcher
|
||||
from .annotation_suggest import model_refusal
|
||||
from .annotations import _pinned_suggestions, _settled, _submit
|
||||
|
||||
@@ -1112,6 +1214,8 @@ class AutotagRoutesMixin:
|
||||
owner = model_gate.new_owner("legacy-single")
|
||||
if not model_gate.try_acquire(owner):
|
||||
return model_busy_response()
|
||||
# #431: the legacy review hides the Skipped tags too (19A).
|
||||
skip = SkipMatcher(AutotagConfig.SKIPPED_TAGS)
|
||||
store = self._capture_store()
|
||||
try:
|
||||
epoch = await self._run_in_executor(self.db.get_execution_epoch)
|
||||
@@ -1151,6 +1255,7 @@ class AutotagRoutesMixin:
|
||||
capture,
|
||||
model_type,
|
||||
options,
|
||||
skip,
|
||||
on_skipped=functools.partial(store.release_use, capture.token),
|
||||
)
|
||||
except MissingBackendPackages as exc:
|
||||
@@ -1306,7 +1411,8 @@ class AutotagRoutesMixin:
|
||||
)
|
||||
|
||||
model_type = service.get_loaded_model_type()
|
||||
service.unload_model()
|
||||
# Off the loop; a dropped request keeps the gate until it ends.
|
||||
await model_gate.run(owner, service.unload_model)
|
||||
finally:
|
||||
model_gate.release(owner)
|
||||
|
||||
|
||||
@@ -45,7 +45,6 @@ import os
|
||||
import sqlite3
|
||||
import tempfile
|
||||
from contextlib import closing
|
||||
from pathlib import Path
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
@@ -56,6 +55,7 @@ from .maintenance_plans import PlanExpired, PlanUsed
|
||||
from .restore_plans import RestorePlanStore
|
||||
|
||||
try:
|
||||
from ...database.sqlite_uri import readonly_uri
|
||||
from ...database import restore_stage
|
||||
from ...database.operations import PromptDatabase
|
||||
from ...database.safety_backup import (
|
||||
@@ -66,6 +66,7 @@ try:
|
||||
safety_temp,
|
||||
)
|
||||
except ImportError:
|
||||
from database.sqlite_uri import readonly_uri
|
||||
from database import restore_stage
|
||||
from database.operations import PromptDatabase
|
||||
from database.safety_backup import (
|
||||
@@ -186,7 +187,7 @@ def snapshot_corruption(path):
|
||||
with open(path, "rb") as handle:
|
||||
if handle.read(len(SQLITE_HEADER)) != SQLITE_HEADER:
|
||||
return "Not a SQLite database file"
|
||||
uri = Path(os.path.abspath(path)).as_uri() + "?mode=ro"
|
||||
uri = readonly_uri(path)
|
||||
with closing(sqlite3.connect(uri, uri=True)) as conn:
|
||||
row = conn.execute("PRAGMA integrity_check").fetchone()
|
||||
except (OSError, sqlite3.Error) as error:
|
||||
|
||||
@@ -65,6 +65,7 @@ from .duplicate_review import ( # noqa: F401 (CHANGED, OUTSIDE_SOURCE: public)
|
||||
NOT_ATTEMPTED,
|
||||
OUTSIDE_SOURCE,
|
||||
RECORDS_FAILED,
|
||||
REVIEW_KEPT_DETAILS,
|
||||
ROW_DETAILS,
|
||||
SAME_FILE,
|
||||
SOURCES_CHANGED,
|
||||
@@ -434,6 +435,7 @@ class DuplicateFilesMixin:
|
||||
{
|
||||
"targets": review["targets"],
|
||||
"kept": review["kept"],
|
||||
"held": review["held"],
|
||||
"protected": review["public"]["protected"]["count"],
|
||||
},
|
||||
)
|
||||
@@ -446,7 +448,7 @@ class DuplicateFilesMixin:
|
||||
job hashes each file again right before it unlinks it.
|
||||
"""
|
||||
listed = self._listed_sources()
|
||||
targets, kept, protected = [], [], {}
|
||||
targets, kept, protected, held = [], [], {}, []
|
||||
for group, keepers in selection.values():
|
||||
removals = [r for r in group["files"] if r not in keepers]
|
||||
for record in removals:
|
||||
@@ -467,6 +469,7 @@ class DuplicateFilesMixin:
|
||||
problem = _bound_problem(record, group, keeper)
|
||||
if problem:
|
||||
protected[problem] = protected.get(problem, 0) + 1
|
||||
held.append(self._copy_row(record, REVIEW_KEPT_DETAILS[problem]))
|
||||
continue
|
||||
reviewed.append(self._target(group, record, keeper))
|
||||
if reviewed:
|
||||
@@ -488,6 +491,7 @@ class DuplicateFilesMixin:
|
||||
return {
|
||||
"targets": targets,
|
||||
"kept": kept,
|
||||
"held": held,
|
||||
"fingerprint": _fingerprint(targets),
|
||||
"public": _public_plan(targets, protected, records),
|
||||
}
|
||||
@@ -584,6 +588,13 @@ class DuplicateFilesMixin:
|
||||
|
||||
async def _remove_duplicates(self, job, db, plan, backup):
|
||||
targets = plan.payload["targets"]
|
||||
# #586: every copy the review protected, with its reason, and the
|
||||
# whole group in the report.
|
||||
held = plan.payload["held"]
|
||||
job.record_counted("kept", len(held), held, complete=True)
|
||||
if held:
|
||||
copies = _plural(len(held), "copy", "copies")
|
||||
job.log_line(f"Kept {copies} the review protected")
|
||||
for row in plan.payload["kept"]:
|
||||
job.record("kept", row, f"= {row['root']}/{row['path']} · {KEPT_COPY}")
|
||||
job.update(total=len(targets), status_line="Removing duplicate files")
|
||||
|
||||
@@ -70,6 +70,18 @@ PROTECTED_REASONS = {
|
||||
"files kept: their kept copy changed or is gone",
|
||||
),
|
||||
}
|
||||
# The job's row for a copy the review protected (#586): the review's reason.
|
||||
REVIEW_KEPT_DETAILS = {
|
||||
MAYBE_SAME: "Kept: may be the same file reached through another mount or share",
|
||||
NO_FILE_ID: "Kept: can't confirm this is a separate file on this drive",
|
||||
SAME: "Kept: the same file as its kept copy",
|
||||
OUTSIDE_SOURCE: "Kept: not inside a configured source",
|
||||
LINK: "Kept: a link, not a regular file",
|
||||
CHANGED_CONTENT: "Kept: changed since the search",
|
||||
GONE: "Kept: no longer exists",
|
||||
UNREADABLE: "Kept: could not be read",
|
||||
KEEPER: "Kept: its kept copy changed or is gone",
|
||||
}
|
||||
# Job outcome rows (never an OS error text).
|
||||
CHANGED = "The file changed since the review; it was kept."
|
||||
KEEPER_CHANGED = "The kept copy changed or is gone; this file was kept."
|
||||
|
||||
@@ -39,6 +39,7 @@ from typing import NamedTuple
|
||||
from aiohttp import web
|
||||
|
||||
from .. import model_gate
|
||||
from ..tagging_defaults import SkipMatcher
|
||||
from .admin import _plural, _time_left
|
||||
from .annotation_suggest import (
|
||||
OptionError,
|
||||
@@ -170,6 +171,8 @@ class _Run:
|
||||
owner: object
|
||||
service: object = None
|
||||
roots: dict = field(default_factory=dict)
|
||||
# The job's Skipped tags, compiled once at its start (#431).
|
||||
skip: object = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -183,6 +186,9 @@ class _Spec:
|
||||
options: dict | None = None
|
||||
# A listing filter the start resolves into ``targets`` (#392).
|
||||
filter: dict | None = None
|
||||
# The Skipped tags when the job was requested (#431); a later save does
|
||||
# not change a running job.
|
||||
skip: tuple = ()
|
||||
|
||||
@property
|
||||
def title(self) -> str:
|
||||
@@ -238,7 +244,11 @@ def _parse_auto(targets, data):
|
||||
)
|
||||
except OptionError as exc:
|
||||
return None, bad_request(str(exc))
|
||||
return _Spec("auto", targets, model_type=model_type, options=options), None
|
||||
skip = tuple(AutotagConfig.SKIPPED_TAGS)
|
||||
return (
|
||||
_Spec("auto", targets, model_type=model_type, options=options, skip=skip),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def parse_image_tag_request(data: dict):
|
||||
@@ -329,6 +339,8 @@ class AutotagRun:
|
||||
keep_in_memory: bool
|
||||
backup_on: bool
|
||||
owner: object
|
||||
# The Skipped tags when the job was started (#431).
|
||||
skip: tuple = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -400,6 +412,14 @@ def _load_model(service, model_type: str, use_gpu: bool) -> None:
|
||||
raise RuntimeError(f"{model_type} could not be loaded")
|
||||
|
||||
|
||||
def _skipped_tags_line(entries: tuple) -> str:
|
||||
"""The job log's one line about the Skipped tags it runs with (#431)."""
|
||||
if not entries:
|
||||
return "Skipped tags: none"
|
||||
count = len(entries)
|
||||
return f"Skipped tags: {count} {'entry' if count == 1 else 'entries'}"
|
||||
|
||||
|
||||
def _refusal(status: int, code: str, message: str) -> Refusal:
|
||||
response = web.json_response(
|
||||
{"success": False, "code": code, "error": message}, status=status
|
||||
@@ -507,6 +527,7 @@ class ImageTagJobsMixin:
|
||||
released when the task ends, also before its runner ever starts.
|
||||
"""
|
||||
from ..autotag import MODELS, get_autotag_service
|
||||
from ..config import AutotagConfig
|
||||
|
||||
backup_on = self._safety_backup_policy() == "on"
|
||||
status = await self._run_in_executor(get_autotag_service().get_models_status)
|
||||
@@ -529,7 +550,13 @@ class ImageTagJobsMixin:
|
||||
response = model_busy_response()
|
||||
return None, Refusal(409, "model-busy", MODEL_BUSY_ERROR, response)
|
||||
run = AutotagRun(
|
||||
model_type, existing, options, keep_in_memory, backup_on, owner
|
||||
model_type,
|
||||
existing,
|
||||
options,
|
||||
keep_in_memory,
|
||||
backup_on,
|
||||
owner,
|
||||
skip=tuple(AutotagConfig.SKIPPED_TAGS),
|
||||
)
|
||||
admitted = False
|
||||
try:
|
||||
@@ -598,7 +625,9 @@ class ImageTagJobsMixin:
|
||||
return _batch_cancelled(job)
|
||||
job.publish({"type": "phase", "phase": "model"})
|
||||
job.update(status_line="Loading the tagging model")
|
||||
spec = _Spec("auto", (), model_type=run.model_type, options=run.options)
|
||||
spec = _Spec(
|
||||
"auto", (), model_type=run.model_type, options=run.options, skip=run.skip
|
||||
)
|
||||
service = await self._prepare_tagging_model(job, spec, run.owner)
|
||||
if service is None:
|
||||
return "failed"
|
||||
@@ -624,6 +653,7 @@ class ImageTagJobsMixin:
|
||||
f"Tagging started · {job.scope.get('label') or run.model_type} · "
|
||||
f"{_plural(total, 'image')} · {policy}"
|
||||
)
|
||||
job.log_line(_skipped_tags_line(spec.skip))
|
||||
if job.cancelled:
|
||||
return _batch_cancelled(job)
|
||||
remaining = await self._autotag_prepass(job, targets, run.existing == "skip")
|
||||
@@ -633,7 +663,9 @@ class ImageTagJobsMixin:
|
||||
spec = dataclasses.replace(
|
||||
spec, targets=tuple(item.target for item in remaining)
|
||||
)
|
||||
loop_run = _Run(job, spec, epoch, run.owner, service)
|
||||
loop_run = _Run(
|
||||
job, spec, epoch, run.owner, service, skip=SkipMatcher(spec.skip)
|
||||
)
|
||||
if not await self._tag_each_file(loop_run, done_before=done):
|
||||
job.update(status_line=STORAGE_RESTORED)
|
||||
job.log_line(f"Tagging stopped · {STORAGE_RESTORED}")
|
||||
@@ -739,6 +771,9 @@ class ImageTagJobsMixin:
|
||||
started += f" · {spec.model_type}"
|
||||
job.log_line(started)
|
||||
run = _Run(job, spec, epoch, owner)
|
||||
if spec.mode == "auto":
|
||||
job.log_line(_skipped_tags_line(spec.skip))
|
||||
run.skip = SkipMatcher(spec.skip)
|
||||
try:
|
||||
if spec.mode == "auto":
|
||||
run.service = await self._prepare_tagging_model(job, spec, owner)
|
||||
@@ -894,6 +929,7 @@ class ImageTagJobsMixin:
|
||||
capture,
|
||||
spec.model_type,
|
||||
spec.options,
|
||||
run.skip,
|
||||
on_skipped=functools.partial(store.release_use, capture.token),
|
||||
)
|
||||
except Exception:
|
||||
|
||||
+39
-25
@@ -204,6 +204,8 @@ class Job:
|
||||
self.total = 0
|
||||
self.counters = dict.fromkeys(groups, 0)
|
||||
self.outcomes = {group: [] for group in groups}
|
||||
# Groups whose report lists every row, beyond MAX_OUTCOME_ROWS (#550).
|
||||
self.complete = {}
|
||||
self.log = []
|
||||
self._cancel_requested = False
|
||||
self.last_event = None
|
||||
@@ -286,42 +288,48 @@ class Job:
|
||||
index = row.get("root_index")
|
||||
if isinstance(index, bool) or not isinstance(index, int) or index < 0:
|
||||
index = None
|
||||
shaped = {
|
||||
"path": path,
|
||||
"root": str(row.get("root") or ""),
|
||||
"root_index": index,
|
||||
"detail": str(row.get("detail") or ""),
|
||||
}
|
||||
if group in self.complete: # a group kept whole for the report (#586)
|
||||
self.complete[group].append(dict(shaped))
|
||||
if len(rows) < MAX_OUTCOME_ROWS:
|
||||
rows.append(
|
||||
{
|
||||
"path": path,
|
||||
"root": str(row.get("root") or ""),
|
||||
"root_index": index,
|
||||
"detail": str(row.get("detail") or ""),
|
||||
}
|
||||
)
|
||||
rows.append(shaped)
|
||||
self.log_line(log_line)
|
||||
|
||||
def record_counted(self, group, count, rows=()):
|
||||
def record_counted(self, group, count, rows=(), *, complete=False):
|
||||
"""Count ``count`` outcomes in ``group`` at once, listing ``rows``.
|
||||
|
||||
For outcomes that need no log line each (a thumbnail job's already
|
||||
fresh previews): the counter is exact, ``rows`` (``record``'s row
|
||||
shape) are listed up to MAX_OUTCOME_ROWS, and nothing is logged.
|
||||
``complete`` also keeps every row for the report (#550: a record of
|
||||
each change that must not be cut short; the caller bounds it).
|
||||
"""
|
||||
if group not in self.counters:
|
||||
raise ValueError(f"Unknown outcome group {group!r} for {self.kind}")
|
||||
self.counters[group] += max(0, int(count))
|
||||
listed = self.outcomes[group]
|
||||
kept = self.complete.setdefault(group, []) if complete else None
|
||||
for row in rows:
|
||||
if len(listed) >= MAX_OUTCOME_ROWS:
|
||||
if kept is None and len(listed) >= MAX_OUTCOME_ROWS:
|
||||
break
|
||||
index = row.get("root_index")
|
||||
if isinstance(index, bool) or not isinstance(index, int) or index < 0:
|
||||
index = None
|
||||
listed.append(
|
||||
{
|
||||
"path": _relative_path(row.get("path")),
|
||||
"root": str(row.get("root") or ""),
|
||||
"root_index": index,
|
||||
"detail": str(row.get("detail") or ""),
|
||||
}
|
||||
)
|
||||
shaped = {
|
||||
"path": _relative_path(row.get("path")),
|
||||
"root": str(row.get("root") or ""),
|
||||
"root_index": index,
|
||||
"detail": str(row.get("detail") or ""),
|
||||
}
|
||||
if kept is not None:
|
||||
kept.append(shaped)
|
||||
if len(listed) < MAX_OUTCOME_ROWS:
|
||||
listed.append(dict(shaped))
|
||||
|
||||
def finish(self, state="finished"):
|
||||
"""End the job in ``state`` and release every subscriber.
|
||||
@@ -406,8 +414,9 @@ class Job:
|
||||
"""The downloadable report: the summary, every outcome group and the log.
|
||||
|
||||
Each group carries its exact count, its listed rows (at most
|
||||
MAX_OUTCOME_ROWS) and whether the list is cut short. Rows and log lines
|
||||
already hold root-relative paths only.
|
||||
MAX_OUTCOME_ROWS, or every row of a ``complete`` group) and whether
|
||||
the list is cut short. Rows and log lines already hold root-relative
|
||||
paths only.
|
||||
"""
|
||||
summary = self.summary()
|
||||
summary.pop("listed", None)
|
||||
@@ -417,16 +426,21 @@ class Job:
|
||||
"generated_at": self._clock(),
|
||||
"job": summary,
|
||||
"outcomes": {
|
||||
group: {
|
||||
"count": self.counters.get(group, len(rows)),
|
||||
"rows": [dict(row) for row in rows],
|
||||
"truncated": self.counters.get(group, len(rows)) > len(rows),
|
||||
}
|
||||
group: self._report_group(group, rows)
|
||||
for group, rows in self.outcomes.items()
|
||||
},
|
||||
"log": list(self.log),
|
||||
}
|
||||
|
||||
def _report_group(self, group, listed):
|
||||
rows = self.complete.get(group, listed)
|
||||
count = self.counters.get(group, len(rows))
|
||||
return {
|
||||
"count": count,
|
||||
"rows": [dict(row) for row in rows],
|
||||
"truncated": count > len(rows),
|
||||
}
|
||||
|
||||
def report_filename(self):
|
||||
"""The download name: prompt-manager-KIND-YYYYMMDD-HHMM.json (local time)."""
|
||||
stamp = time.strftime("%Y%m%d-%H%M", time.localtime(self._clock()))
|
||||
|
||||
+3
-1
@@ -377,7 +377,9 @@ class LibraryRoutesMixin:
|
||||
[parent["id"] for parent in result["items"]]
|
||||
)
|
||||
result["next_cursor"] = _encode(
|
||||
result.pop("next_key"), scope, name=options["sort"] == "name"
|
||||
result.pop("next_key"),
|
||||
scope,
|
||||
name=options["sort"] in ("name", "name-desc"),
|
||||
)
|
||||
return result, records
|
||||
|
||||
|
||||
+23
-2
@@ -68,6 +68,23 @@ def _count(count, singular, plural):
|
||||
return f"{count} {singular if count == 1 else plural}"
|
||||
|
||||
|
||||
# Each kept entry's row in the job (#586), in the review's "Kept: ..." words.
|
||||
KEPT_DETAILS = {
|
||||
"active": "Kept: the active log",
|
||||
"link": "Kept: a link",
|
||||
"other": "Kept: another file",
|
||||
}
|
||||
|
||||
|
||||
def _kept_files(rows):
|
||||
"""``{name, kind}`` of each entry the cleanup keeps, in folder order."""
|
||||
return [
|
||||
{"name": row["name"], "kind": row["kind"]}
|
||||
for row in rows
|
||||
if row["kind"] in KEPT_DETAILS
|
||||
]
|
||||
|
||||
|
||||
def _protected_reasons(rows):
|
||||
"""The plan's Protected reasons: what the cleanup keeps, zero counts left out."""
|
||||
active = sum(1 for row in rows if row["kind"] == "active")
|
||||
@@ -129,7 +146,7 @@ class LogCleanupMixin:
|
||||
token = store.issue(
|
||||
ROTATED_LOGS,
|
||||
json.dumps(files, sort_keys=True),
|
||||
{"files": files, "protected": protected},
|
||||
{"files": files, "protected": protected, "kept": _kept_files(rows)},
|
||||
(),
|
||||
)
|
||||
except ScopeTooLarge:
|
||||
@@ -198,7 +215,11 @@ class LogCleanupMixin:
|
||||
job.record(
|
||||
"errors", {"path": name, "detail": detail}, f"! {name} · {detail}"
|
||||
)
|
||||
job.record_counted("protected", plan.payload["protected"])
|
||||
kept = [
|
||||
{"path": row["name"], "detail": KEPT_DETAILS[row["kind"]]}
|
||||
for row in plan.payload["kept"]
|
||||
]
|
||||
job.record_counted("protected", len(kept), kept, complete=True)
|
||||
removed, failed = len(result["removed"]), len(result["failed"])
|
||||
line = f"Deleted {_count(removed, 'rotated log copy', 'rotated log copies')}"
|
||||
if failed:
|
||||
|
||||
+95
-6
@@ -1,8 +1,9 @@
|
||||
"""Maintenance routes: reviewed, read-only plans (#249 System S6).
|
||||
|
||||
``POST /prompt_manager/maintenance/preview {operation, options:{}}`` reviews
|
||||
one operation (``duplicate_prompts``, ``optimize``, ``unused_components`` or
|
||||
``missing_links``; ``database/maintenance.py``; ``duplicate_files``,
|
||||
one operation (``duplicate_prompts``, ``optimize``, ``unused_components``,
|
||||
``missing_links`` or ``pair_metadata``, #550; ``database/maintenance.py``;
|
||||
``duplicate_files``,
|
||||
``py/api/duplicate_files.py``; ``rotated_logs``, the rotated log copies,
|
||||
``py/api/log_cleanup.py``, which has its own apply job) off the event loop and
|
||||
answers a ``Plan``: ``{success, token, operation, expires_at, scope:{count,
|
||||
@@ -97,7 +98,11 @@ MAINTENANCE_TITLES = {
|
||||
"unused_components": "Unused prompt components",
|
||||
"missing_links": "Missing-file links",
|
||||
"workflow_storage": "Store each workflow once",
|
||||
# #550, beyond the concept: fills old Pair prompts' empty category and tags.
|
||||
"pair_metadata": "Pair category and tags",
|
||||
}
|
||||
# Operations whose job keeps every changed row for its report (#550).
|
||||
COMPLETE_RECORD = frozenset({"pair_metadata"})
|
||||
_OPERATION_ERROR = "operation must be one of: " + ", ".join(PREVIEW_OPERATIONS)
|
||||
MAINTENANCE_GROUPS = ("changed", "protected", "errors")
|
||||
MAX_TOKEN_LENGTH = 128
|
||||
@@ -281,9 +286,27 @@ def short_space_message(short):
|
||||
)
|
||||
|
||||
|
||||
def _changed_line(operation, count, marks=0):
|
||||
def _pair_metadata_line(result):
|
||||
count = result["changed"]
|
||||
noun = (
|
||||
"prompt from its saved pairs"
|
||||
if count == 1
|
||||
else "prompts from their saved pairs"
|
||||
)
|
||||
categories = result.get("categories", 0)
|
||||
tag_sets = result.get("tag_sets", 0)
|
||||
return (
|
||||
f"Filled {count:,} {noun} "
|
||||
f"({categories:,} {'category' if categories == 1 else 'categories'}, "
|
||||
f"{tag_sets:,} {'tag set' if tag_sets == 1 else 'tag sets'})"
|
||||
)
|
||||
|
||||
|
||||
def _changed_line(operation, count, marks=0, result=None):
|
||||
if operation == "optimize":
|
||||
return "Optimized the database"
|
||||
if operation == "pair_metadata":
|
||||
return _pair_metadata_line(result or {"changed": count})
|
||||
if operation == "duplicate_prompts":
|
||||
noun = "duplicate prompt" if count == 1 else "duplicate prompts"
|
||||
return f"Merged {count:,} {noun}"
|
||||
@@ -668,13 +691,67 @@ class MaintenanceRoutesMixin:
|
||||
rows = await self._run_in_executor(
|
||||
self._maintenance_rows, plan.operation, result["rows"]
|
||||
)
|
||||
job.record_counted("changed", result["changed"], rows)
|
||||
job.record_counted("protected", payload["protected"])
|
||||
line = _changed_line(plan.operation, result["changed"], result.get("marks", 0))
|
||||
job.record_counted(
|
||||
"changed",
|
||||
result["changed"],
|
||||
rows,
|
||||
complete=plan.operation in COMPLETE_RECORD,
|
||||
)
|
||||
kept = await self._run_in_executor(self._kept_rows, db, plan, result)
|
||||
if kept is None:
|
||||
job.record_counted("protected", payload["protected"]) # none kept
|
||||
else:
|
||||
# #550, #586: each kept record with its reason, like the changed
|
||||
# ones, and every one of them in the report.
|
||||
job.record_counted("protected", len(kept), kept, complete=True)
|
||||
line = _changed_line(
|
||||
plan.operation, result["changed"], result.get("marks", 0), result
|
||||
)
|
||||
job.log_line(line)
|
||||
job.update(progress=100, status_line=f"{line} · {backup_note(backup)}")
|
||||
return None
|
||||
|
||||
def _kept_rows(self, db, plan, result):
|
||||
"""Outcome rows for the records the apply kept (blocking), or None.
|
||||
|
||||
Prompts are records (no path); a missing-links plan's kept links
|
||||
name their file, relative to its output root, so Activity can open
|
||||
and locate it (#586); one under a source that is gone keeps its path
|
||||
under that source's label. None for an operation that keeps nothing
|
||||
(Optimize).
|
||||
"""
|
||||
if plan.operation == "missing_links":
|
||||
roots = _scan_roots(self._get_all_output_dirs())
|
||||
judged = plan.payload["roots"]
|
||||
rows = []
|
||||
for item in db.kept_links(judged):
|
||||
path, label, index = _scan_location(item["image_path"], roots)
|
||||
if index is None and item["owner"] is not None:
|
||||
# A source that is gone is not listed: name the file
|
||||
# under it, with no listing index to open it by.
|
||||
source = judged[item["owner"]]
|
||||
where = [(os.path.abspath(source), self._public_path(source), None)]
|
||||
path, label, index = _scan_location(item["image_path"], where)
|
||||
rows.append(
|
||||
{
|
||||
"path": path,
|
||||
"root": label,
|
||||
"root_index": index,
|
||||
"detail": f"Kept: {item['reason']}",
|
||||
}
|
||||
)
|
||||
return rows
|
||||
if "kept" not in result:
|
||||
return None
|
||||
return [
|
||||
{
|
||||
"path": "",
|
||||
"detail": f"Prompt {item['prompt_id']}: kept, "
|
||||
+ "; ".join(item["reasons"]),
|
||||
}
|
||||
for item in result["kept"]
|
||||
]
|
||||
|
||||
def _maintenance_rows(self, operation, described):
|
||||
"""Outcome rows for the changed records (root-relative paths only)."""
|
||||
if operation == "missing_links":
|
||||
@@ -691,6 +768,8 @@ class MaintenanceRoutesMixin:
|
||||
}
|
||||
)
|
||||
return rows
|
||||
if operation == "pair_metadata":
|
||||
return [{"path": "", "detail": _filled_detail(item)} for item in described]
|
||||
if operation == "duplicate_prompts":
|
||||
return [
|
||||
{
|
||||
@@ -708,6 +787,16 @@ class MaintenanceRoutesMixin:
|
||||
]
|
||||
|
||||
|
||||
def _filled_detail(item):
|
||||
"""``Prompt 12: category “portrait”; tags a, b``: one filled prompt (#550)."""
|
||||
parts = []
|
||||
if "category" in item:
|
||||
parts.append(f"category “{item['category']}”")
|
||||
if item.get("tags"):
|
||||
parts.append("tags " + ", ".join(item["tags"]))
|
||||
return f"Prompt {item['prompt_id']}: " + "; ".join(parts)
|
||||
|
||||
|
||||
def backup_note(backup):
|
||||
"""The status line's backup part: the file name, or that there is none."""
|
||||
return f"Safety backup {backup}" if backup else NO_BACKUP
|
||||
|
||||
+33
-11
@@ -69,6 +69,8 @@ _state_lock = threading.Lock()
|
||||
_parsed = preview_location.ParsedLocation()
|
||||
_available = True
|
||||
_problem = None
|
||||
# The newest check whose answer was published.
|
||||
_published_order = 0
|
||||
_keys = {}
|
||||
_key_cache = {}
|
||||
_clock = time.monotonic
|
||||
@@ -167,10 +169,13 @@ def refresh():
|
||||
if parsed.mode != FOLDER_MODE or parsed.problem is not None:
|
||||
return parsed.mode == SOURCE_MODE
|
||||
try:
|
||||
available, problem, marker = _probe.check(parsed.path, parsed.id)
|
||||
(available, problem, marker), order = _probe.ordered_check(
|
||||
parsed.path, parsed.id
|
||||
)
|
||||
except (OSError, TimeoutError, RuntimeError):
|
||||
available, problem, marker = False, "unreachable", None
|
||||
_set_state(parsed, available, problem, marker)
|
||||
order = _probe.latest_order()
|
||||
_set_state(parsed, available, problem, marker, order)
|
||||
return available
|
||||
|
||||
|
||||
@@ -207,11 +212,16 @@ def mark_invalid():
|
||||
_available, _problem = False, "invalid"
|
||||
|
||||
|
||||
def _set_state(parsed, available, problem, marker):
|
||||
global _available, _problem
|
||||
def _set_state(parsed, available, problem, marker, order):
|
||||
"""Publish a check's answer, unless a check that started later already
|
||||
published (two callers can finish out of order, #611)."""
|
||||
global _available, _problem, _published_order
|
||||
with _state_lock:
|
||||
if _parsed is not parsed:
|
||||
return # the setting changed meanwhile
|
||||
if order < _published_order:
|
||||
return
|
||||
_published_order = order
|
||||
_available, _problem = available, problem
|
||||
keys = dict((marker or {}).get("keys") or {})
|
||||
if keys != _keys:
|
||||
@@ -261,14 +271,25 @@ class _Probe:
|
||||
self._stop = threading.Event()
|
||||
self._thread = None
|
||||
self._checking = None
|
||||
self._order = 0
|
||||
|
||||
def check(self, path, expected_id):
|
||||
"""``folder_state`` within IO_BUDGET; raises TimeoutError.
|
||||
"""``folder_state`` within IO_BUDGET; see :meth:`ordered_check`."""
|
||||
return self.ordered_check(path, expected_id)[0]
|
||||
|
||||
The answer comes from a check that started after this call: one
|
||||
already running may have looked before what the caller just did, so
|
||||
it is waited for (never run twice at once; a stuck one times out here)
|
||||
and followed by a new one."""
|
||||
def latest_order(self):
|
||||
"""The order number of the check started last."""
|
||||
with self._lock:
|
||||
return self._order
|
||||
|
||||
def ordered_check(self, path, expected_id):
|
||||
"""``(folder_state, order)`` within IO_BUDGET; raises TimeoutError.
|
||||
|
||||
``order`` grows with each check started, so answers can be published
|
||||
in the order they looked. The answer comes from a check that started
|
||||
after this call: one already running may have looked before what the
|
||||
caller just did, so it is waited for (never run twice at once; a stuck
|
||||
one times out here) and followed by a new one."""
|
||||
key = (path, expected_id)
|
||||
with self._lock:
|
||||
earlier = self._running(key)
|
||||
@@ -284,7 +305,7 @@ class _Probe:
|
||||
self._checking = None
|
||||
if "error" in pending[2]:
|
||||
raise pending[2]["error"]
|
||||
return pending[2]["value"]
|
||||
return pending[2]["value"], pending[2]["order"]
|
||||
|
||||
def _running(self, key):
|
||||
"""The check of *key* still running (lock held), else None."""
|
||||
@@ -300,7 +321,8 @@ class _Probe:
|
||||
|
||||
def _start_check(self, path, expected_id):
|
||||
"""Start one check on its own thread (lock held)."""
|
||||
result = {}
|
||||
self._order += 1
|
||||
result = {"order": self._order}
|
||||
done = threading.Event()
|
||||
|
||||
def run():
|
||||
|
||||
+9
-1
@@ -727,7 +727,15 @@ class PromptRoutesMixin:
|
||||
async def save_prompt(self, request):
|
||||
"""Save a new prompt with metadata and duplicate detection."""
|
||||
try:
|
||||
data = await request.json()
|
||||
data, error = await self._read_json_body(request)
|
||||
if error is not None:
|
||||
return error
|
||||
for field in ("text", "category", "notes"):
|
||||
if data.get(field) is not None and not isinstance(data[field], str):
|
||||
return web.json_response(
|
||||
{"success": False, "error": f"{field} must be text"},
|
||||
status=400,
|
||||
)
|
||||
|
||||
text = (data.get("text") or "").strip()
|
||||
if not text:
|
||||
|
||||
+114
-13
@@ -55,9 +55,25 @@ _SETTINGS_FILE_KEYS = {
|
||||
"AUTOTAG_EXISTING": ("autotag", "defaults", "existing"),
|
||||
"AUTOTAG_GENERAL_THRESHOLD": ("autotag", "defaults", "general_threshold"),
|
||||
"AUTOTAG_CHARACTER_THRESHOLD": ("autotag", "defaults", "character_threshold"),
|
||||
"AUTOTAG_TEMPLATE": ("autotag", "defaults", "template"),
|
||||
"AUTOTAG_SUGGESTION_LIMIT": ("autotag", "defaults", "suggestion_limit"),
|
||||
"AUTOTAG_SKIPPED_TAGS": ("autotag", "defaults", "skipped_tags"),
|
||||
# #431: the saved instructions and each engine's default, always written
|
||||
# together (server-side pairing).
|
||||
"AUTOTAG_INSTRUCTIONS": ("autotag", "defaults", "instructions"),
|
||||
"AUTOTAG_DEFAULT_INSTRUCTIONS": ("autotag", "defaults", "default_instructions"),
|
||||
}
|
||||
# The old single template: written only as the GGUF default's mirror for a
|
||||
# downgrade (#431 B1.1), deleted when that default is Built-in.
|
||||
_TEMPLATE_MIRROR_KEY = ("autotag", "defaults", "template")
|
||||
TEMPLATE_REPLACED_ERROR = (
|
||||
"LLM tag template was replaced by Saved instructions. Reload the page."
|
||||
)
|
||||
# Whole-list settings (#431): a save without ``expected_revision`` would merge
|
||||
# a stale copy over a list changed elsewhere, so it is refused.
|
||||
_REVISION_REQUIRED = frozenset(
|
||||
{"autotag_skipped_tags", "autotag_instructions", "autotag_default_instructions"}
|
||||
)
|
||||
REVISION_REQUIRED_ERROR = "Reload the page and try again."
|
||||
# The thumbnail settings a running Convert outdated job fixes (#317).
|
||||
_THUMBNAIL_SETTINGS = ("THUMBNAIL_FORMAT", "THUMBNAIL_QUALITY", "THUMBNAIL_EDGE")
|
||||
# Attributes that live on GalleryConfig; the others on PromptManagerConfig.
|
||||
@@ -138,13 +154,20 @@ def _settings_mutation(updates, new_roots):
|
||||
"""The config commit for a settings save: only the submitted keys change.
|
||||
|
||||
Every other section and key (integrations and the CivitAI key, database,
|
||||
unknown sections) is kept as it is on disk.
|
||||
unknown sections) is kept as it is on disk. A save of the instructions
|
||||
also writes the old ``template`` key as the GGUF default's mirror.
|
||||
"""
|
||||
|
||||
def mutate(config):
|
||||
for name, value in updates.items():
|
||||
*sections, key = _SETTINGS_FILE_KEYS[name]
|
||||
_set_nested(config, sections, key, value)
|
||||
_set_nested(config, sections, key, _jsonable(value))
|
||||
if "AUTOTAG_INSTRUCTIONS" in updates:
|
||||
*sections, key = _TEMPLATE_MIRROR_KEY
|
||||
mirror = _mirror_text(
|
||||
updates["AUTOTAG_INSTRUCTIONS"], updates["AUTOTAG_DEFAULT_INSTRUCTIONS"]
|
||||
)
|
||||
_set_nested(config, sections, key, _DELETE if mirror is None else mirror)
|
||||
if new_roots is not None:
|
||||
gallery = _section(config, "gallery")
|
||||
monitoring = _section(gallery, "monitoring")
|
||||
@@ -156,10 +179,37 @@ def _settings_mutation(updates, new_roots):
|
||||
return mutate
|
||||
|
||||
|
||||
# ``_set_nested``'s value that removes the key instead of setting it.
|
||||
_DELETE = object()
|
||||
|
||||
|
||||
def _jsonable(value):
|
||||
"""A runtime value as config.json holds it (tuples become lists)."""
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [_jsonable(item) for item in value]
|
||||
if isinstance(value, dict):
|
||||
return {key: _jsonable(item) for key, item in value.items()}
|
||||
return value
|
||||
|
||||
|
||||
def _mirror_text(entries, defaults):
|
||||
"""The GGUF default's text, or None when it is Built-in (#431 B1.1)."""
|
||||
from ..tagging_defaults import BUILTIN_INSTRUCTIONS_ID
|
||||
|
||||
for item in entries:
|
||||
if item["id"] == defaults["gguf"] != BUILTIN_INSTRUCTIONS_ID:
|
||||
return item["text"]
|
||||
return None
|
||||
|
||||
|
||||
def _set_nested(config, sections, key, value):
|
||||
"""``config[s1][s2]...[key] = value``, each section a fresh dict."""
|
||||
"""``config[s1][s2]...[key] = value``, each section a fresh dict;
|
||||
``_DELETE`` removes the key."""
|
||||
if not sections:
|
||||
config[key] = value
|
||||
if value is _DELETE:
|
||||
config.pop(key, None)
|
||||
else:
|
||||
config[key] = value
|
||||
return
|
||||
merged = _section(config, sections[0])
|
||||
_set_nested(merged, sections[1:], key, value)
|
||||
@@ -177,7 +227,7 @@ def _validate_autotag_settings(data):
|
||||
MAX_SUGGESTION_LIMIT,
|
||||
MODEL_TYPES,
|
||||
validate_saved_threshold,
|
||||
validate_template,
|
||||
validate_skipped_tags,
|
||||
)
|
||||
|
||||
updates = {}
|
||||
@@ -206,12 +256,7 @@ def _validate_autotag_settings(data):
|
||||
raise ValueError(f"{error}.") from None
|
||||
updates[key.upper()] = value
|
||||
if "autotag_template" in data:
|
||||
try:
|
||||
updates["AUTOTAG_TEMPLATE"] = validate_template(
|
||||
data["autotag_template"], "LLM tag template"
|
||||
)
|
||||
except ValueError as error:
|
||||
raise ValueError(f"{error}.") from None
|
||||
raise ValueError(TEMPLATE_REPLACED_ERROR)
|
||||
if "autotag_suggestion_limit" in data:
|
||||
value = data["autotag_suggestion_limit"]
|
||||
if type(value) is not int or not 1 <= value <= MAX_SUGGESTION_LIMIT:
|
||||
@@ -220,9 +265,41 @@ def _validate_autotag_settings(data):
|
||||
f"{MAX_SUGGESTION_LIMIT}."
|
||||
)
|
||||
updates["AUTOTAG_SUGGESTION_LIMIT"] = value
|
||||
if "autotag_skipped_tags" in data:
|
||||
updates["AUTOTAG_SKIPPED_TAGS"] = tuple(
|
||||
validate_skipped_tags(data["autotag_skipped_tags"])
|
||||
)
|
||||
updates.update(_validate_instruction_settings(data))
|
||||
return updates
|
||||
|
||||
|
||||
def _validate_instruction_settings(data):
|
||||
"""Server-side pairing (#431): a save that carries either instruction key
|
||||
gets the other from the running configuration (which holds the migrated
|
||||
list), the defaults are checked against the merged list, and both are
|
||||
written. Empty when the save carries neither."""
|
||||
from ..config import AutotagConfig
|
||||
from ..tagging_defaults import (
|
||||
validate_default_instructions,
|
||||
validate_instructions,
|
||||
)
|
||||
|
||||
if not {"autotag_instructions", "autotag_default_instructions"} & set(data):
|
||||
return {}
|
||||
if "autotag_instructions" in data:
|
||||
entries = validate_instructions(data["autotag_instructions"])
|
||||
else:
|
||||
entries = [dict(item) for item in AutotagConfig.INSTRUCTIONS]
|
||||
defaults = data.get(
|
||||
"autotag_default_instructions", dict(AutotagConfig.DEFAULT_INSTRUCTIONS)
|
||||
)
|
||||
defaults = validate_default_instructions(defaults, entries)
|
||||
return {
|
||||
"AUTOTAG_INSTRUCTIONS": tuple(entries),
|
||||
"AUTOTAG_DEFAULT_INSTRUCTIONS": defaults,
|
||||
}
|
||||
|
||||
|
||||
async def _read_json(request):
|
||||
"""Parse a JSON object body through the package's capped reader."""
|
||||
from . import _read_json_body
|
||||
@@ -250,6 +327,7 @@ class SettingsRoutesMixin:
|
||||
"""Get current settings."""
|
||||
try:
|
||||
from ..config import AutotagConfig, PromptManagerConfig, GalleryConfig
|
||||
from ..tagging_defaults import DEFAULT_TAG_TEMPLATE
|
||||
|
||||
root_paths = [
|
||||
self._public_path(d) for d in GalleryConfig.MONITORING_DIRECTORIES
|
||||
@@ -288,8 +366,21 @@ class SettingsRoutesMixin:
|
||||
"autotag_character_threshold": (
|
||||
AutotagConfig.CHARACTER_THRESHOLD
|
||||
),
|
||||
"autotag_template": AutotagConfig.TEMPLATE,
|
||||
# Read-only for one release (#431): a page cached
|
||||
# from before the upgrade still renders.
|
||||
"autotag_template": AutotagConfig.default_instructions_text(
|
||||
"gguf"
|
||||
),
|
||||
"autotag_instructions": [
|
||||
dict(item) for item in AutotagConfig.INSTRUCTIONS
|
||||
],
|
||||
"autotag_default_instructions": dict(
|
||||
AutotagConfig.DEFAULT_INSTRUCTIONS
|
||||
),
|
||||
# Built-in's text, which System shows read-only.
|
||||
"autotag_builtin_instructions": DEFAULT_TAG_TEMPLATE,
|
||||
"autotag_suggestion_limit": AutotagConfig.SUGGESTION_LIMIT,
|
||||
"autotag_skipped_tags": list(AutotagConfig.SKIPPED_TAGS),
|
||||
"cpu_count": PromptManagerConfig.max_worker_threads(),
|
||||
"gallery_root_paths": root_paths,
|
||||
"gallery_root_path": root_paths[0] if root_paths else "",
|
||||
@@ -393,6 +484,10 @@ class SettingsRoutesMixin:
|
||||
{"success": False, "error": "expected_revision must be a string"},
|
||||
status=400,
|
||||
)
|
||||
if expected_revision is None and _REVISION_REQUIRED & set(data):
|
||||
return web.json_response(
|
||||
{"success": False, "error": REVISION_REQUIRED_ERROR}, status=400
|
||||
)
|
||||
try:
|
||||
updates = self._validate_settings(data)
|
||||
except ValueError as ve:
|
||||
@@ -522,6 +617,12 @@ class SettingsRoutesMixin:
|
||||
if name in _AUTOTAG_SETTINGS:
|
||||
setattr(AutotagConfig, name[len(_AUTOTAG_PREFIX) :], value)
|
||||
continue
|
||||
if "AUTOTAG_INSTRUCTIONS" in updates:
|
||||
# What config.json now holds as the old key (the mirror).
|
||||
AutotagConfig._LEGACY_TEMPLATE = AutotagConfig.mirror_template()
|
||||
for name, value in updates.items():
|
||||
if name in _AUTOTAG_SETTINGS:
|
||||
continue
|
||||
owner = GalleryConfig if name in _GALLERY_SETTINGS else PromptManagerConfig
|
||||
setattr(owner, name, value)
|
||||
restart_fields = []
|
||||
|
||||
@@ -632,10 +632,6 @@ class AutoThumbnailer:
|
||||
"""True once the worker has nothing queued or in hand."""
|
||||
return self._wait_for(lambda: not self._busy, timeout)
|
||||
|
||||
def wait_batch(self, size, timeout=None):
|
||||
"""True once the worker holds a batch of *size* files."""
|
||||
return self._wait_for(lambda: self._batch == size, timeout)
|
||||
|
||||
|
||||
class _PreviewGate:
|
||||
"""Admission of view requests on the event loop (#249 I2).
|
||||
@@ -967,15 +963,6 @@ class AutoThumbnailMixin:
|
||||
return "queued"
|
||||
return "skipped"
|
||||
|
||||
def _capture_target(self, image_path):
|
||||
"""``(src, dst, is_video)`` of a new output's preview now, or None."""
|
||||
located = self._capture_source(image_path)
|
||||
if located is None:
|
||||
return None
|
||||
src, root, rel, is_video = located
|
||||
dst = DeferredPreview(root, rel).resolve()
|
||||
return None if dst is None else (src, dst, is_video)
|
||||
|
||||
def _capture_source(self, image_path):
|
||||
"""``(src, root, rel, is_video)`` of a media file under a current
|
||||
root (not a preview, a link or in a trash folder), or None."""
|
||||
|
||||
@@ -90,7 +90,7 @@ class ThumbnailOptions:
|
||||
}
|
||||
|
||||
|
||||
def thumbnail_options(raw, *, from_query=False):
|
||||
def thumbnail_options(raw):
|
||||
"""``(ThumbnailOptions, None)`` from a body or query, or ``(None, message)``.
|
||||
|
||||
An absent mode is ``missing``; the settings are the server's now.
|
||||
@@ -136,7 +136,7 @@ class ThumbnailJobsMixin:
|
||||
``mode=prune`` lists the removed sources' thumbnails (#320)."""
|
||||
if request.query.get("mode") == "prune":
|
||||
return await self.get_prune_plan()
|
||||
options, error = thumbnail_options(request.query, from_query=True)
|
||||
options, error = thumbnail_options(request.query)
|
||||
if error:
|
||||
return _invalid(error)
|
||||
try:
|
||||
|
||||
@@ -59,6 +59,9 @@ REWRITE_FAILED = (
|
||||
)
|
||||
RECLAIM_FAILED = "Reclaim space failed; the database is unchanged."
|
||||
ROW_KEPT = "could not be rewritten; it keeps its inline text and reads as before"
|
||||
# A protected value's row (#586), in the review's "not text; kept as is".
|
||||
VALUE_KEPT = "stored value not text; kept as is"
|
||||
KEPT_COLUMNS = {"workflow_data": "workflow", "prompt_metadata": "prompt graph"}
|
||||
|
||||
|
||||
NOTHING_LEFT = "Every image record already stores its workflow once"
|
||||
@@ -177,7 +180,19 @@ class WorkflowStorageRoutesMixin:
|
||||
protected += batch["protected"]
|
||||
errors += len(batch["errors"])
|
||||
job.record_counted("changed", batch["rows"])
|
||||
job.record_counted("protected", batch["protected"])
|
||||
job.record_counted(
|
||||
"protected",
|
||||
batch["protected"],
|
||||
[
|
||||
{
|
||||
"path": "",
|
||||
"detail": f"Image record {image_id}: "
|
||||
f"{KEPT_COLUMNS.get(column, column)}, {VALUE_KEPT}",
|
||||
}
|
||||
for image_id, column in batch["kept"]
|
||||
],
|
||||
complete=True,
|
||||
)
|
||||
for image_id in batch["errors"]:
|
||||
job.record(
|
||||
"errors",
|
||||
|
||||
+10
-10
@@ -30,9 +30,9 @@ except ImportError:
|
||||
# The tag template and thresholds live with the saved tagging defaults
|
||||
# (#328), so config.py never imports PIL.
|
||||
try:
|
||||
from .tagging_defaults import DEFAULT_TAG_TEMPLATE
|
||||
from .tagging_defaults import DEFAULT_TAG_TEMPLATE, read_wd14_label_rows
|
||||
except ImportError:
|
||||
from py.tagging_defaults import DEFAULT_TAG_TEMPLATE
|
||||
from py.tagging_defaults import DEFAULT_TAG_TEMPLATE, read_wd14_label_rows
|
||||
|
||||
try:
|
||||
from ..utils import exit_signal
|
||||
@@ -559,7 +559,6 @@ class AutoTagService:
|
||||
# A missing onnxruntime is refused by load_model before this runs
|
||||
# (#424); a ModuleNotFoundError here becomes the same refusal.
|
||||
import onnxruntime as ort
|
||||
import csv
|
||||
|
||||
config = MODELS[model_type]
|
||||
model_dir = self.models_dir / config["subdir"]
|
||||
@@ -576,12 +575,8 @@ class AutoTagService:
|
||||
|
||||
session = ort.InferenceSession(str(model_path), providers=providers)
|
||||
|
||||
# Read tag list from CSV
|
||||
tags_list = []
|
||||
with open(tags_path, "r", encoding="utf-8") as f:
|
||||
reader = csv.DictReader(f)
|
||||
for row in reader:
|
||||
tags_list.append(row)
|
||||
# Read tag list from CSV (the reader the Skipped tags preview shares)
|
||||
tags_list = read_wd14_label_rows(tags_path)
|
||||
|
||||
self.logger.info(
|
||||
f"WD14 model loaded ({len(tags_list)} tags, "
|
||||
@@ -992,6 +987,7 @@ class AutoTagService:
|
||||
prompt: Optional[str] = None,
|
||||
general_threshold: Optional[float] = None,
|
||||
character_threshold: Optional[float] = None,
|
||||
skip: Any = None,
|
||||
) -> List[Tuple[str, Optional[float]]]:
|
||||
"""The top ``limit`` suggestions as ``(name, confidence)`` pairs.
|
||||
|
||||
@@ -1000,7 +996,9 @@ class AutoTagService:
|
||||
``None``. Names are distinct case-insensitively (first one wins).
|
||||
A name Save would reject (over 50 characters, a control character
|
||||
such as a newline, a comma) is skipped before the dedupe and the
|
||||
``limit`` count. ``prompt`` is used for this call only;
|
||||
``limit`` count. A name ``skip`` matches (a ``SkipMatcher``, the
|
||||
Skipped tags compiled for this request or job, #431) is dropped next,
|
||||
so it uses up no slot. ``prompt`` is used for this call only;
|
||||
``custom_prompt`` is never changed.
|
||||
"""
|
||||
if type(limit) is not int or not 1 <= limit <= MAX_SUGGESTION_LIMIT:
|
||||
@@ -1017,6 +1015,8 @@ class AutoTagService:
|
||||
name = validate_vocabulary_tag_name(raw)
|
||||
except ValueError:
|
||||
continue
|
||||
if skip is not None and skip.matches(name):
|
||||
continue
|
||||
key = name.casefold()
|
||||
if key in seen:
|
||||
continue
|
||||
|
||||
+183
-66
@@ -30,6 +30,7 @@ routes = server_instance.routes
|
||||
# Extension info
|
||||
extension_uri = None # Will be set in __init__.py
|
||||
|
||||
import hashlib # noqa: E402
|
||||
import os # noqa: E402
|
||||
from typing import Any, Dict, List, Optional, Tuple # noqa: E402
|
||||
|
||||
@@ -77,27 +78,6 @@ def _canonical_path(path: str) -> str:
|
||||
return os.path.normcase(os.path.realpath(path))
|
||||
|
||||
|
||||
def write_private_json(path: str, data: Any) -> None:
|
||||
"""Write ``data`` as JSON to ``path`` readable by the current user only.
|
||||
|
||||
config.json can hold an API key (integrations.lora_manager), so it is
|
||||
created with mode 0600 and, on POSIX, an existing more open file is
|
||||
tightened too. Windows has no POSIX modes; there the chmod is a best
|
||||
effort that only affects the read-only bit. Parent directories are
|
||||
created as needed. Raises OSError when the file cannot be written.
|
||||
"""
|
||||
import json
|
||||
|
||||
parent_dir = os.path.dirname(path)
|
||||
if parent_dir:
|
||||
os.makedirs(parent_dir, exist_ok=True)
|
||||
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
if os.name == "posix":
|
||||
os.chmod(path, 0o600)
|
||||
|
||||
|
||||
def _lexical_path(path: str) -> str:
|
||||
"""``path`` normalised for comparison without touching the filesystem.
|
||||
|
||||
@@ -587,8 +567,15 @@ try:
|
||||
MODEL_TYPES,
|
||||
WD14_MODEL_TYPES,
|
||||
validate_saved_threshold,
|
||||
validate_skipped_tags,
|
||||
validate_template,
|
||||
validate_threshold,
|
||||
BUILTIN_INSTRUCTIONS_ID,
|
||||
INSTRUCTION_ENGINES,
|
||||
MAX_SAVED_INSTRUCTIONS,
|
||||
builtin_instructions,
|
||||
validate_default_instructions,
|
||||
validate_instructions,
|
||||
)
|
||||
except ImportError:
|
||||
from py.tagging_defaults import ( # noqa: F401 - re-exported
|
||||
@@ -600,8 +587,15 @@ except ImportError:
|
||||
MODEL_TYPES,
|
||||
WD14_MODEL_TYPES,
|
||||
validate_saved_threshold,
|
||||
validate_skipped_tags,
|
||||
validate_template,
|
||||
validate_threshold,
|
||||
BUILTIN_INSTRUCTIONS_ID,
|
||||
INSTRUCTION_ENGINES,
|
||||
MAX_SAVED_INSTRUCTIONS,
|
||||
builtin_instructions,
|
||||
validate_default_instructions,
|
||||
validate_instructions,
|
||||
)
|
||||
|
||||
|
||||
@@ -617,6 +611,31 @@ def _valid_existing(value: Any) -> str:
|
||||
return value
|
||||
|
||||
|
||||
def _valid_skipped_tags(value: Any) -> Tuple[str, ...]:
|
||||
return tuple(validate_skipped_tags(value))
|
||||
|
||||
|
||||
def _reason(error: ValueError) -> str:
|
||||
"""A settings sentence as a load warning's reason ("entry 2 is empty")."""
|
||||
text = str(error).rstrip(".")
|
||||
for prefix in ("Skipped tags", "Saved instructions"):
|
||||
if text.startswith(prefix):
|
||||
return text.removeprefix(prefix).lstrip(": ")
|
||||
return text
|
||||
|
||||
|
||||
def _text_id(text: str) -> str:
|
||||
"""A migrated entry's id: deterministic, so every load gives the same."""
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()[:12]
|
||||
|
||||
|
||||
def _fallback_warning(key: str, error: ValueError, default: Any) -> str:
|
||||
"""The one load warning of a saved tagging default that fails its check."""
|
||||
if key == "skipped_tags":
|
||||
return f"autotag.defaults.skipped_tags {_reason(error)}; using no skipped tags"
|
||||
return f"autotag.defaults.{key} {error}; using {default!r}"
|
||||
|
||||
|
||||
def _valid_suggestion_limit(value: Any) -> int:
|
||||
if type(value) is not int or not 1 <= value <= MAX_SUGGESTION_LIMIT:
|
||||
raise ValueError(f"must be an integer from 1 to {MAX_SUGGESTION_LIMIT}")
|
||||
@@ -637,8 +656,18 @@ class AutotagConfig:
|
||||
EXISTING = "skip"
|
||||
GENERAL_THRESHOLD = 0.35
|
||||
CHARACTER_THRESHOLD = 0.85
|
||||
TEMPLATE = DEFAULT_TAG_TEMPLATE
|
||||
SUGGESTION_LIMIT = DEFAULT_SUGGESTION_LIMIT
|
||||
SKIPPED_TAGS: Tuple[str, ...] = ()
|
||||
# #431: the saved instructions (Built-in is never stored) and the default
|
||||
# of each language-model engine, by id.
|
||||
INSTRUCTIONS: Tuple[Dict[str, str], ...] = ()
|
||||
DEFAULT_INSTRUCTIONS: Dict[str, str] = {
|
||||
"gguf": BUILTIN_INSTRUCTIONS_ID,
|
||||
"hf": BUILTIN_INSTRUCTIONS_ID,
|
||||
}
|
||||
# The old single template as read at load (None when absent or invalid);
|
||||
# only the migration reads it. Saves keep it as the GGUF default's mirror.
|
||||
_LEGACY_TEMPLATE: Optional[str] = None
|
||||
|
||||
# config.json key -> (attribute, validator, default)
|
||||
_KEYS = {
|
||||
@@ -654,35 +683,155 @@ class AutotagConfig:
|
||||
lambda v: validate_saved_threshold("value", v),
|
||||
0.85,
|
||||
),
|
||||
"template": ("TEMPLATE", validate_template, DEFAULT_TAG_TEMPLATE),
|
||||
"suggestion_limit": (
|
||||
"SUGGESTION_LIMIT",
|
||||
_valid_suggestion_limit,
|
||||
DEFAULT_SUGGESTION_LIMIT,
|
||||
),
|
||||
# #431: words whose suggested tags are never offered.
|
||||
"skipped_tags": ("SKIPPED_TAGS", _valid_skipped_tags, ()),
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def suggestion_limit(cls) -> int:
|
||||
return cls.SUGGESTION_LIMIT
|
||||
|
||||
@staticmethod
|
||||
def valid_suggestion_limit(value: Any) -> bool:
|
||||
return (
|
||||
type(value) is int and 1 <= value <= MAX_SUGGESTION_LIMIT
|
||||
) # bool, float and str are rejected
|
||||
@classmethod
|
||||
def all_instructions(cls) -> List[Dict[str, str]]:
|
||||
"""Built-in first, then the saved entries (copies)."""
|
||||
return [builtin_instructions()] + [dict(item) for item in cls.INSTRUCTIONS]
|
||||
|
||||
@classmethod
|
||||
def default_instructions_text(cls, engine: str) -> str:
|
||||
"""The text of ``engine``'s (``gguf`` or ``hf``) default instructions."""
|
||||
if engine not in INSTRUCTION_ENGINES:
|
||||
raise ValueError(f"{engine} takes no instructions")
|
||||
wanted = cls.DEFAULT_INSTRUCTIONS.get(engine, BUILTIN_INSTRUCTIONS_ID)
|
||||
for item in cls.INSTRUCTIONS:
|
||||
if item["id"] == wanted:
|
||||
return item["text"]
|
||||
return DEFAULT_TAG_TEMPLATE
|
||||
|
||||
@classmethod
|
||||
def mirror_template(cls) -> Optional[str]:
|
||||
"""What an older build reads as its one template (#431 B1.1): the
|
||||
GGUF default's text, or None (no key) when that default is Built-in."""
|
||||
if cls.DEFAULT_INSTRUCTIONS.get("gguf") == BUILTIN_INSTRUCTIONS_ID:
|
||||
return None
|
||||
return cls.default_instructions_text("gguf")
|
||||
|
||||
@classmethod
|
||||
def get_config(cls) -> Dict[str, Any]:
|
||||
"""The saved defaults; the template only when it is not the built-in
|
||||
one, so a later built-in template still reaches who never set one."""
|
||||
"""The saved defaults; ``template`` only as the GGUF default's mirror."""
|
||||
defaults = {
|
||||
key: getattr(cls, attr) for key, (attr, _v, _d) in cls._KEYS.items()
|
||||
}
|
||||
if defaults["template"] == DEFAULT_TAG_TEMPLATE:
|
||||
del defaults["template"]
|
||||
defaults["skipped_tags"] = list(defaults["skipped_tags"])
|
||||
defaults["instructions"] = [dict(item) for item in cls.INSTRUCTIONS]
|
||||
defaults["default_instructions"] = dict(cls.DEFAULT_INSTRUCTIONS)
|
||||
mirror = cls.mirror_template()
|
||||
if mirror is not None:
|
||||
defaults["template"] = mirror
|
||||
return {"defaults": defaults}
|
||||
|
||||
@classmethod
|
||||
def _load_instructions(cls, defaults: Dict[str, Any]) -> None:
|
||||
"""Each instruction key alone, then the cross-key default check, then
|
||||
the in-memory migration of the old template (#431). Nothing is
|
||||
written here, so the revision never changes at load."""
|
||||
cls._LEGACY_TEMPLATE = None
|
||||
if "template" in defaults:
|
||||
try:
|
||||
cls._LEGACY_TEMPLATE = validate_template(defaults["template"])
|
||||
except ValueError as error:
|
||||
config_logger.warning(
|
||||
f"autotag.defaults.template {error}; using the built-in one"
|
||||
)
|
||||
entries: List[Dict[str, str]] = []
|
||||
damaged = False
|
||||
if "instructions" in defaults:
|
||||
try:
|
||||
entries = validate_instructions(defaults["instructions"])
|
||||
except ValueError as error:
|
||||
damaged = True
|
||||
config_logger.warning(
|
||||
f"autotag.defaults.instructions {_reason(error)}; "
|
||||
"using Built-in only"
|
||||
)
|
||||
chosen = {engine: BUILTIN_INSTRUCTIONS_ID for engine in INSTRUCTION_ENGINES}
|
||||
stored = defaults.get("default_instructions")
|
||||
if damaged:
|
||||
# Both defaults fall back with the list (its one warning says so);
|
||||
# the old template is migrated as on a first upgrade.
|
||||
pass
|
||||
elif "default_instructions" in defaults and (
|
||||
not isinstance(stored, dict) or set(stored) != set(INSTRUCTION_ENGINES)
|
||||
):
|
||||
config_logger.warning(
|
||||
"autotag.defaults.default_instructions must name gguf and hf; "
|
||||
"using Built-in"
|
||||
)
|
||||
elif isinstance(stored, dict):
|
||||
ids = {item["id"] for item in entries}
|
||||
for engine in INSTRUCTION_ENGINES:
|
||||
value = stored[engine]
|
||||
if isinstance(value, str) and (
|
||||
value == BUILTIN_INSTRUCTIONS_ID or value in ids
|
||||
):
|
||||
chosen[engine] = value
|
||||
else:
|
||||
config_logger.warning(
|
||||
f"autotag.defaults.default_instructions.{engine} names no "
|
||||
"saved instructions; using Built-in"
|
||||
)
|
||||
cls._migrate_template(
|
||||
entries,
|
||||
chosen,
|
||||
first=damaged or "instructions" not in defaults,
|
||||
keep_defaults=not damaged and "default_instructions" in defaults,
|
||||
)
|
||||
cls.INSTRUCTIONS = tuple(entries)
|
||||
cls.DEFAULT_INSTRUCTIONS = chosen
|
||||
|
||||
@classmethod
|
||||
def _migrate_template(cls, entries, chosen, *, first, keep_defaults) -> None:
|
||||
"""Bring a template saved by an older build in as a named entry."""
|
||||
legacy = cls._LEGACY_TEMPLATE
|
||||
if legacy is None or legacy == DEFAULT_TAG_TEMPLATE:
|
||||
return
|
||||
if first:
|
||||
# The first upgrade: it becomes "Custom template", the default for
|
||||
# both engines unless defaults are stored already (20A).
|
||||
entries.append(
|
||||
{"id": _text_id(legacy), "name": "Custom template", "text": legacy}
|
||||
)
|
||||
if not keep_defaults:
|
||||
for engine in INSTRUCTION_ENGINES:
|
||||
chosen[engine] = entries[-1]["id"]
|
||||
return
|
||||
if any(item["text"] == legacy for item in entries):
|
||||
return # the mirror of a saved default
|
||||
# Edited on an older build after an upgrade (B1.2): import it.
|
||||
if len(entries) >= MAX_SAVED_INSTRUCTIONS:
|
||||
config_logger.warning(
|
||||
"The tagging template edited on an older version was not imported: "
|
||||
f"the saved instructions are full ({MAX_SAVED_INSTRUCTIONS})"
|
||||
)
|
||||
return
|
||||
names = {item["name"].casefold() for item in entries}
|
||||
number = 2
|
||||
while f"custom template ({number})" in names:
|
||||
number += 1
|
||||
name = f"Custom template ({number})"
|
||||
ids = {item["id"] for item in entries}
|
||||
new_id, n = _text_id(legacy), 2
|
||||
while new_id in ids:
|
||||
new_id, n = _text_id(f"{legacy}\n{n}"), n + 1
|
||||
entries.append({"id": new_id, "name": name, "text": legacy})
|
||||
config_logger.info(
|
||||
"Imported the tagging template edited on an older version as " f"'{name}'"
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def update_config(cls, new_config: Any) -> None:
|
||||
if not isinstance(new_config, dict):
|
||||
@@ -698,12 +847,9 @@ class AutotagConfig:
|
||||
try:
|
||||
setattr(cls, attr, validate(defaults[key]))
|
||||
except ValueError as error:
|
||||
config_logger.warning(
|
||||
f"autotag.defaults.{key} {error}; using {default!r}"
|
||||
if key != "template"
|
||||
else f"autotag.defaults.template {error}; using the built-in one"
|
||||
)
|
||||
config_logger.warning(_fallback_warning(key, error, default))
|
||||
setattr(cls, attr, default)
|
||||
cls._load_instructions(defaults)
|
||||
|
||||
|
||||
class PromptManagerConfig:
|
||||
@@ -890,35 +1036,6 @@ class PromptManagerConfig:
|
||||
else:
|
||||
config_logger.info(f"Config file not found: {config_path}, using defaults")
|
||||
|
||||
@classmethod
|
||||
def save_to_file(cls, config_path: Optional[str] = None):
|
||||
"""Save the current configuration to a JSON file.
|
||||
|
||||
Serializes the complete configuration (including gallery settings) to
|
||||
a JSON file. Creates the directory structure if it doesn't exist.
|
||||
|
||||
Args:
|
||||
config_path (str): Path where the JSON configuration file should be saved.
|
||||
Parent directories will be created if they don't exist.
|
||||
Defaults to :meth:`get_config_path`.
|
||||
|
||||
Raises:
|
||||
The method handles all exceptions internally and logs errors rather
|
||||
than propagating them.
|
||||
|
||||
Example:
|
||||
PromptManagerConfig.save_to_file('backup_config.json')
|
||||
PromptManagerConfig.save_to_file('/etc/comfyui/prompt_manager.json')
|
||||
"""
|
||||
if config_path is None:
|
||||
config_path = cls.get_config_path()
|
||||
|
||||
try:
|
||||
write_private_json(config_path, cls.get_config())
|
||||
config_logger.info(f"Saved configuration to {config_path}")
|
||||
except Exception as e:
|
||||
config_logger.error(f"Error saving config to {config_path}: {e}")
|
||||
|
||||
@classmethod
|
||||
def update_config(cls, new_config: Dict[str, Any]):
|
||||
"""Update configuration attributes from a dictionary.
|
||||
|
||||
+233
-1
@@ -4,8 +4,13 @@ Server-free on purpose: ``py.api`` must import without ComfyUI's ``server``
|
||||
module, which ``py/config.py`` needs; config re-exports all of this.
|
||||
"""
|
||||
|
||||
import csv
|
||||
import math
|
||||
from typing import Any
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import unicodedata
|
||||
from typing import Any, Dict, Iterable, List, Set, Tuple
|
||||
|
||||
# The tagging engines (``py/autotag.py`` MODELS) and the options they take.
|
||||
WD14_MODEL_TYPES = frozenset({"wd14-swinv2", "wd14-vit"})
|
||||
@@ -88,3 +93,230 @@ def validate_template(value: Any, name: str = "instructions") -> str:
|
||||
if len(encoded) > MAX_INSTRUCTION_BYTES:
|
||||
raise ValueError(f"{name} {'are' if name == 'instructions' else 'is'} too long")
|
||||
return value
|
||||
|
||||
|
||||
# Skipped tags (#431): words whose suggested tags are never offered.
|
||||
MAX_SKIPPED_TAGS = 200
|
||||
MAX_SKIPPED_TAG_CHARS = 50
|
||||
_WORD_SEPARATORS = re.compile(r"[ _-]+")
|
||||
_FORBIDDEN_IN_ENTRY = re.compile(r"[,\x00-\x1f\x7f-\x9f\u2028\u2029]")
|
||||
|
||||
|
||||
def tag_words(text: str) -> Tuple[str, ...]:
|
||||
"""The words of a tag or a skipped entry: NFC, case-folded, split on runs
|
||||
of space, ``_`` and ``-``. Every other character belongs to a word."""
|
||||
folded = unicodedata.normalize("NFC", text).casefold()
|
||||
return tuple(word for word in _WORD_SEPARATORS.split(folded) if word)
|
||||
|
||||
|
||||
def validate_skipped_tags(value: Any) -> List[str]:
|
||||
"""The Skipped tags list, each entry trimmed, or ValueError with the
|
||||
sentence the System page shows (entries are numbered from 1)."""
|
||||
if not isinstance(value, list):
|
||||
raise ValueError("Skipped tags must be a list of words.")
|
||||
if len(value) > MAX_SKIPPED_TAGS:
|
||||
raise ValueError(f"Skipped tags: at most {MAX_SKIPPED_TAGS} entries.")
|
||||
entries: List[str] = []
|
||||
seen: Dict[Tuple[str, ...], int] = {}
|
||||
for number, raw in enumerate(value, 1):
|
||||
if not isinstance(raw, str):
|
||||
raise ValueError(f"Skipped tags: entry {number} must be text.")
|
||||
entry = raw.strip()
|
||||
if not entry:
|
||||
raise ValueError(f"Skipped tags: entry {number} is empty.")
|
||||
if len(entry) > MAX_SKIPPED_TAG_CHARS:
|
||||
raise ValueError(
|
||||
f"Skipped tags: entry {number} must be at most "
|
||||
f"{MAX_SKIPPED_TAG_CHARS} characters."
|
||||
)
|
||||
if _FORBIDDEN_IN_ENTRY.search(entry):
|
||||
raise ValueError(
|
||||
f"Skipped tags: entry {number} must not contain a comma "
|
||||
"or a line break."
|
||||
)
|
||||
words = tag_words(entry)
|
||||
if not words:
|
||||
raise ValueError(f"Skipped tags: entry {number} is empty.")
|
||||
if words in seen:
|
||||
raise ValueError(
|
||||
f"Skipped tags: entry {number} repeats entry {seen[words]}."
|
||||
)
|
||||
seen[words] = number
|
||||
entries.append(entry)
|
||||
return entries
|
||||
|
||||
|
||||
class SkipMatcher:
|
||||
"""Whether a suggested tag contains a skipped entry's words, consecutive
|
||||
and whole (#431, 17B). Compiled once per request or job; the cost of a
|
||||
check depends on the tag's words, not on the number of entries."""
|
||||
|
||||
def __init__(self, entries: Iterable[str] = ()):
|
||||
self._entries = frozenset(filter(None, (tag_words(e) for e in entries)))
|
||||
self._longest = max((len(words) for words in self._entries), default=0)
|
||||
|
||||
def __bool__(self) -> bool:
|
||||
return bool(self._entries)
|
||||
|
||||
def matches(self, name: str) -> bool:
|
||||
if not self._entries:
|
||||
return False
|
||||
words = tag_words(name)
|
||||
for start in range(len(words)):
|
||||
stop = min(len(words), start + self._longest)
|
||||
for end in range(start + 1, stop + 1):
|
||||
if words[start:end] in self._entries:
|
||||
return True
|
||||
return False
|
||||
|
||||
def counting(self) -> "CountingSkip":
|
||||
"""A per-request view that also remembers which names it hid."""
|
||||
return CountingSkip(self)
|
||||
|
||||
|
||||
class CountingSkip:
|
||||
"""A :class:`SkipMatcher` that records the distinct names it hid, so the
|
||||
Tag image dialog can say how many skipped tags were not suggested."""
|
||||
|
||||
def __init__(self, matcher: SkipMatcher):
|
||||
self._matcher = matcher
|
||||
self.hidden: Set[str] = set()
|
||||
|
||||
def matches(self, name: str) -> bool:
|
||||
if self._matcher.matches(name):
|
||||
self.hidden.add(name.casefold())
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
# A WD14 label list's tags the engine can suggest: general (0) and character (4).
|
||||
WD14_SUGGESTED_CATEGORIES = frozenset({"0", "4"})
|
||||
_label_cache: Dict[str, Tuple[Tuple[int, int], Tuple[str, ...]]] = {}
|
||||
_label_cache_lock = threading.Lock()
|
||||
|
||||
|
||||
def read_wd14_label_rows(path) -> List[Dict[str, str]]:
|
||||
"""Every row of a WD14 ``selected_tags.csv``, read as UTF-8 and closed
|
||||
before returning (the handle never outlives the call, for Windows)."""
|
||||
with open(path, "r", encoding="utf-8", newline="") as handle:
|
||||
return list(csv.DictReader(handle))
|
||||
|
||||
|
||||
def wd14_label_names(path) -> Tuple[str, ...]:
|
||||
"""The names a WD14 label list can suggest (categories 0 and 4), cached
|
||||
by ``(mtime_ns, size)``. OSError (missing) and ValueError (unreadable)
|
||||
propagate; the caller skips that list."""
|
||||
key = os.fspath(path)
|
||||
stat = os.stat(key)
|
||||
stamp = (stat.st_mtime_ns, stat.st_size)
|
||||
with _label_cache_lock:
|
||||
cached = _label_cache.get(key)
|
||||
if cached is not None and cached[0] == stamp:
|
||||
return cached[1]
|
||||
try:
|
||||
rows = read_wd14_label_rows(key)
|
||||
except csv.Error as exc: # a damaged list (a NUL byte, an oversized field)
|
||||
raise ValueError(f"unreadable label list: {exc}") from None
|
||||
names = tuple(
|
||||
name
|
||||
for row in rows
|
||||
if row.get("category", "0") in WD14_SUGGESTED_CATEGORIES
|
||||
and (name := (row.get("name") or "").strip())
|
||||
)
|
||||
with _label_cache_lock:
|
||||
_label_cache[key] = (stamp, names)
|
||||
return names
|
||||
|
||||
|
||||
# Saved instructions (#431): named language-model instructions with a default
|
||||
# per engine. Built-in is never stored; readers put it first.
|
||||
BUILTIN_INSTRUCTIONS_ID = "builtin"
|
||||
BUILTIN_INSTRUCTIONS_NAME = "Built-in"
|
||||
MAX_SAVED_INSTRUCTIONS = 20
|
||||
MAX_INSTRUCTION_NAME_CHARS = 60
|
||||
INSTRUCTION_ENGINES = ("gguf", "hf")
|
||||
ENGINE_LABELS = {"gguf": "GGUF JoyCaption", "hf": "HuggingFace JoyCaption"}
|
||||
_INSTRUCTION_ID = re.compile(r"[0-9a-f]{12}")
|
||||
_CONTROL = re.compile(r"[\x00-\x1f\x7f-\x9f
]")
|
||||
|
||||
|
||||
def builtin_instructions() -> Dict[str, str]:
|
||||
"""The read-only Built-in entry, always the shipped template."""
|
||||
return {
|
||||
"id": BUILTIN_INSTRUCTIONS_ID,
|
||||
"name": BUILTIN_INSTRUCTIONS_NAME,
|
||||
"text": DEFAULT_TAG_TEMPLATE,
|
||||
}
|
||||
|
||||
|
||||
def _instruction_entry(raw: Any, number: int) -> Dict[str, str]:
|
||||
prefix = f"Saved instructions: entry {number}"
|
||||
if (
|
||||
not isinstance(raw, dict)
|
||||
or set(raw) != {"id", "name", "text"}
|
||||
or not isinstance(raw["id"], str)
|
||||
or _INSTRUCTION_ID.fullmatch(raw["id"]) is None
|
||||
):
|
||||
raise ValueError(f"{prefix} has an invalid id.")
|
||||
name = raw["name"].strip() if isinstance(raw["name"], str) else ""
|
||||
if not name:
|
||||
raise ValueError(f"{prefix} needs a name.")
|
||||
if len(name) > MAX_INSTRUCTION_NAME_CHARS:
|
||||
raise ValueError(
|
||||
f"{prefix} name must be at most {MAX_INSTRUCTION_NAME_CHARS} characters."
|
||||
)
|
||||
if _CONTROL.search(name):
|
||||
raise ValueError(f"{prefix} name must not contain a line break.")
|
||||
if name.casefold() == BUILTIN_INSTRUCTIONS_NAME.casefold():
|
||||
raise ValueError("Saved instructions: the name “Built-in” is reserved.")
|
||||
try:
|
||||
text = validate_template(raw["text"], f"{prefix} text")
|
||||
except ValueError as error:
|
||||
raise ValueError(f"{error}.") from None
|
||||
return {"id": raw["id"], "name": name, "text": text}
|
||||
|
||||
|
||||
def validate_instructions(value: Any) -> List[Dict[str, str]]:
|
||||
"""The saved entries ``[{id, name, text}]`` in order (names trimmed), or
|
||||
ValueError with the sentence System shows (Built-in is not counted)."""
|
||||
if not isinstance(value, list):
|
||||
raise ValueError("Saved instructions must be a list.")
|
||||
if len(value) > MAX_SAVED_INSTRUCTIONS:
|
||||
raise ValueError(
|
||||
f"Saved instructions: at most {MAX_SAVED_INSTRUCTIONS} entries besides "
|
||||
"Built-in."
|
||||
)
|
||||
entries: List[Dict[str, str]] = []
|
||||
ids: Dict[str, int] = {}
|
||||
names: Dict[str, int] = {}
|
||||
for number, raw in enumerate(value, 1):
|
||||
item = _instruction_entry(raw, number)
|
||||
if item["id"] in ids:
|
||||
raise ValueError(
|
||||
f"Saved instructions: entry {number} repeats the id of entry "
|
||||
f"{ids[item['id']]}."
|
||||
)
|
||||
key = item["name"].casefold()
|
||||
if key in names:
|
||||
raise ValueError(
|
||||
f"Saved instructions: entry {number} name “{item['name']}” is "
|
||||
"already used."
|
||||
)
|
||||
ids[item["id"]] = number
|
||||
names[key] = number
|
||||
entries.append(item)
|
||||
return entries
|
||||
|
||||
|
||||
def validate_default_instructions(value: Any, entries) -> Dict[str, str]:
|
||||
"""``{gguf: id, hf: id}``, each Built-in or an id of ``entries``."""
|
||||
if not isinstance(value, dict) or set(value) != set(INSTRUCTION_ENGINES):
|
||||
raise ValueError("Default instructions must name GGUF and HuggingFace.")
|
||||
ids = {BUILTIN_INSTRUCTIONS_ID} | {item["id"] for item in entries}
|
||||
for engine in INSTRUCTION_ENGINES:
|
||||
if not isinstance(value[engine], str) or value[engine] not in ids:
|
||||
raise ValueError(
|
||||
f"Default for {ENGINE_LABELS[engine]} must be one of the saved "
|
||||
"instructions."
|
||||
)
|
||||
return {engine: value[engine] for engine in INSTRUCTION_ENGINES}
|
||||
|
||||
+3
-2
@@ -1,10 +1,11 @@
|
||||
[project]
|
||||
name = "promptmanager"
|
||||
description = "A powerful ComfyUI custom node that extends the standard text encoder with persistent prompt storage, advanced search capabilities, and an automatic image gallery system using SQLite."
|
||||
version = "3.4.1"
|
||||
version = "3.6.0"
|
||||
license = {file = "LICENSE"}
|
||||
requires-python = ">=3.12"
|
||||
dependencies = ["watchdog>=2.1.0", "Pillow>=8.0.0"]
|
||||
dependencies = ["watchdog>=2.1.0", "Pillow>=10.1.0"]
|
||||
classifiers = ["Operating System :: OS Independent"]
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=84.2.1"]
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
# Runtime dependencies
|
||||
# Runtime dependencies (Pillow, aiohttp and numpy floors match ComfyUI requirements.txt)
|
||||
watchdog>=6.0.0
|
||||
Pillow>=12.3.0
|
||||
aiohttp>=3.14.5
|
||||
Pillow>=10.1.0
|
||||
aiohttp>=3.11.8
|
||||
|
||||
# Tests and coverage
|
||||
coverage>=7.16.2
|
||||
numpy>=2.2.6 # WD14 engine tests
|
||||
coverage>=7.17.0
|
||||
numpy>=1.25.0 # WD14 engine tests
|
||||
|
||||
# Code quality
|
||||
black==26.10.1
|
||||
flake8==7.4.1
|
||||
flake8==7.5.0
|
||||
|
||||
# Security scanning
|
||||
bandit[toml]>=1.9.4
|
||||
|
||||
# Pre-commit hooks
|
||||
pre-commit>=4.6.2
|
||||
pre-commit>=4.7.0
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Runtime dependencies for PromptManager.
|
||||
# aiohttp, numpy and torch come from ComfyUI itself and are not listed here.
|
||||
watchdog>=2.1.0 # Output directory monitoring
|
||||
Pillow>=8.0.0 # Image metadata extraction and thumbnails
|
||||
Pillow>=10.1.0 # Image metadata extraction and thumbnails
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.5 MiB |
@@ -0,0 +1,47 @@
|
||||
// #615: each old page says it will be removed and links to where its job lives now. The page keeps working, and the
|
||||
// notice takes its own room: nothing is pushed off a page whose layout fills the window.
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
|
||||
const PAGES = Object.freeze([
|
||||
{ path: '/prompt_manager/admin', section: 'library', heading: 'Prompt library' },
|
||||
{ path: '/prompt_manager/gallery', section: 'images', heading: 'Images' },
|
||||
{ path: '/prompt_manager/gallery.html', section: 'inspect', heading: 'Inspect a file' },
|
||||
]);
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await expect(await request.post('/__test__/reset', {})).toBeOK();
|
||||
await page.addInitScript(() => {
|
||||
localStorage.setItem('pm_update_notice_dismissed', '3.2.0');
|
||||
localStorage.setItem('gallerySettings', JSON.stringify({ checkThumbnailsAtStartup: false }));
|
||||
});
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
});
|
||||
|
||||
for (const { path, section, heading } of PAGES) {
|
||||
test(`${path} says it will be removed and links to ${section}`, async ({ page }) => {
|
||||
await page.goto(path);
|
||||
const notice = page.locator('#legacy-removal-notice');
|
||||
await expect(notice).toBeVisible();
|
||||
await expect(notice).toContainText('This page will be removed in a future release.');
|
||||
await notice.getByRole('link', { name: 'Open the workspace' }).click();
|
||||
await expect(page).toHaveURL(new RegExp(`/prompt_manager/workspace#/${section}$`));
|
||||
await expect(page.locator('#workspace-main h1')).toHaveText(heading);
|
||||
});
|
||||
}
|
||||
|
||||
test('the notice leaves the metadata page filling the window', async ({ page }) => {
|
||||
await page.goto('/prompt_manager/gallery.html');
|
||||
await expect(page.locator('#legacy-removal-notice')).toBeVisible();
|
||||
const { bottom, height, scroll } = await page.locator('#image-area').evaluate(area => ({
|
||||
bottom: area.getBoundingClientRect().bottom,
|
||||
height: window.innerHeight,
|
||||
scroll: document.documentElement.scrollHeight,
|
||||
}));
|
||||
expect(bottom).toBeLessThanOrEqual(height);
|
||||
expect(scroll).toBeLessThanOrEqual(height);
|
||||
});
|
||||
@@ -21,8 +21,9 @@ export function mapLibraryConcept(reference) {
|
||||
// Save view is the concept's own (saved views, #249 D16 retired).
|
||||
const toolbar={...library.toolbar};
|
||||
toolbar.items=toolbar.items.map(item=>{
|
||||
// D14: the Library appends its Oldest first after the concept's sorts; D153 (#447) drops "Highest rated".
|
||||
if(item.id==='sort')return {...item,options:[...item.options.filter(option=>option!=='Highest rated'),'Oldest first']};
|
||||
// D14: the Library appends its Oldest first after the concept's sorts; D153 (#447) drops "Highest rated"; D205
|
||||
// (#620) appends the old Dashboard's three sorts.
|
||||
if(item.id==='sort')return {...item,options:[...item.options.filter(option=>option!=='Highest rated'),'Oldest first','Recently used','Most used','Name Z–A']};
|
||||
// D36: the category list is the Library's real list, sorted alphabetically; the concept lists prompt order.
|
||||
if(item.id==='category-filter')return {...item,options:[item.options[0],...item.options.slice(1).sort(nocase)]};
|
||||
return item;
|
||||
|
||||
+19
-10
@@ -88,6 +88,9 @@ class TestModel:
|
||||
suggestions: list | None = None
|
||||
# The engine's model table (``py/autotag.py`` MODELS), set at startup.
|
||||
models_config: dict = {}
|
||||
# Where the WD14 label lists live (the Skipped tags preview, #431), set at
|
||||
# startup; the tagging fixture writes or removes a small list there.
|
||||
models_dir = None
|
||||
|
||||
def get_models_status(self) -> dict:
|
||||
from py.autotag import package_fields
|
||||
@@ -118,21 +121,26 @@ class TestModel:
|
||||
return ["generated", Path(image_path).parent.parent.name]
|
||||
|
||||
def generate_tag_suggestions(
|
||||
self, image_path: str, *, limit: int, **kwargs: object
|
||||
self, image_path: str, *, limit: int, skip=None, **kwargs: object
|
||||
) -> list[tuple[str, float | None]]:
|
||||
"""Fixed scored suggestions (WD14) or unscored ones (language models)."""
|
||||
"""Fixed scored suggestions (WD14) or unscored ones (language models),
|
||||
without the ones ``skip`` hides before the limit, as the engine does
|
||||
(#431)."""
|
||||
scored = self.loaded is not None and self.loaded.startswith("wd14")
|
||||
if self.suggestions is not None:
|
||||
return [
|
||||
pairs = [
|
||||
(name, confidence if scored else None)
|
||||
for name, confidence in self.suggestions[:limit]
|
||||
for name, confidence in self.suggestions
|
||||
]
|
||||
names = ["sandstone", "desert", "arch", "sky", "rock", "dune", "canyon"]
|
||||
names += [f"tag_{index:02d}" for index in range(100)]
|
||||
return [
|
||||
(name, round(0.98 - index * 0.008, 3) if scored else None)
|
||||
for index, name in enumerate(names[:limit])
|
||||
]
|
||||
else:
|
||||
names = ["sandstone", "desert", "arch", "sky", "rock", "dune", "canyon"]
|
||||
names += [f"tag_{index:02d}" for index in range(100)]
|
||||
pairs = [
|
||||
(name, round(0.98 - index * 0.008, 3) if scored else None)
|
||||
for index, name in enumerate(names)
|
||||
]
|
||||
kept = [pair for pair in pairs if skip is None or not skip.matches(pair[0])]
|
||||
return kept[:limit]
|
||||
|
||||
def download_model(self, model_type: str, progress_callback=None) -> bool:
|
||||
raise AssertionError("browser tests never download a model")
|
||||
@@ -236,6 +244,7 @@ def create_app(root: Path) -> web.Application:
|
||||
|
||||
TestModel.models_config = autotag.MODELS
|
||||
model = TestModel()
|
||||
model.models_dir = root / "models"
|
||||
autotag.get_autotag_service = lambda: model
|
||||
api = PromptManagerAPI()
|
||||
api.add_routes(routes)
|
||||
|
||||
@@ -167,8 +167,6 @@ const IMAGES_COUNT_URL = '/prompt_manager/images/output?mode=workspace&limit=1';
|
||||
|
||||
/** The concept's shell and Overview with the recorded deviations applied, given the server's real overview. */
|
||||
function mapOverviewConcept(reference, overview, images) {
|
||||
// D165: the discreet legacy-pages line follows the settings layout.
|
||||
reference = { ...reference, tree: [...reference.tree, 'p.legacy-pages.small'] };
|
||||
const relabel = { 'Database integrity': 'Not checked', 'Positive prompts': number(overview.library.positive_prompts),
|
||||
'Negative variants': number(overview.library.negative_variants), Images: number(images) };
|
||||
const rows = {
|
||||
@@ -265,7 +263,7 @@ const UNMAPPED_PATHS = [
|
||||
'columns.surfaces.0.rows.length', 'columns.surfaces.0.tree.4', 'columns.surfaces.0.tree.length',
|
||||
'columns.surfaces.1.rows.0.description', 'columns.surfaces.1.rows.1.title', 'columns.surfaces.1.rows.2.description',
|
||||
'heading.actions.0', 'heading.actions.length', 'metrics.items.3.value', 'nav.label',
|
||||
'save.button.cursor', 'save.button.opacity', 'save.status.text', 'tree.3', 'tree.length',
|
||||
'save.button.cursor', 'save.button.opacity', 'save.status.text',
|
||||
].sort();
|
||||
|
||||
test('system comparison detects mutations', async ({ page, request }) => {
|
||||
|
||||
@@ -64,11 +64,31 @@ function measureMaintenance(mainSelector) {
|
||||
};
|
||||
}
|
||||
|
||||
/** D102: the Metadata cache row says why its review is unavailable and its button is aria-disabled (D93's dimmed style). */
|
||||
function mapMaintenanceConcept(reference) {
|
||||
/**
|
||||
* D192 (#550, beyond the concept): Database adds "Pair category and tags" after "Unused prompt components", shaped like
|
||||
* the concept's plain Review plan row (Optimize database's).
|
||||
*/
|
||||
function mapPairRow(reference) {
|
||||
return {
|
||||
...reference,
|
||||
surfaces: reference.surfaces.map(surface => ({ ...surface, rows: surface.rows.map(row => row.title !== 'Metadata cache' ? row : {
|
||||
surfaces: reference.surfaces.map(surface => {
|
||||
const plain = surface.rows.find(row => row.title === 'Optimize database');
|
||||
const at = surface.rows.findIndex(row => row.title === 'Unused prompt components');
|
||||
if (!plain || at < 0) return surface;
|
||||
return { ...surface, tree: [...surface.tree, 'div.operation'], rows: [...surface.rows.slice(0, at + 1),
|
||||
{ ...plain, title: 'Pair category and tags', description: 'Fill an empty category or tags on prompts older ' +
|
||||
'Pair node runs saved, from their saved pairs. Only empty fields of never-edited prompts.' },
|
||||
...surface.rows.slice(at + 1)] };
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
/** D102: the Metadata cache row says why its review is unavailable and its button is aria-disabled (D93's dimmed style). */
|
||||
function mapMaintenanceConcept(reference) {
|
||||
const mapped = mapPairRow(reference);
|
||||
return {
|
||||
...mapped,
|
||||
surfaces: mapped.surfaces.map(surface => ({ ...surface, rows: surface.rows.map(row => row.title !== 'Metadata cache' ? row : {
|
||||
...row, description: `${row.description} Nothing is cached in this browser.`,
|
||||
actions: row.actions.map(action => ({ ...action, opacity: '0.55', cursor: 'not-allowed' })) }) })),
|
||||
};
|
||||
@@ -209,8 +229,8 @@ test('maintenance comparison detects mutations', async ({ page, request }) => {
|
||||
const expected = mapMaintenanceConcept(concept);
|
||||
const production = await productionMaintenance(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the section differs only on the D102 row.
|
||||
expect(diffInventory(concept, production).map(item => item.path).sort()).toEqual([
|
||||
// Mapped only for the D192 row, the section differs only on the D102 row.
|
||||
expect(diffInventory(mapPairRow(concept), production).map(item => item.path).sort()).toEqual([
|
||||
'surfaces.1.rows.3.actions.0.cursor', 'surfaces.1.rows.3.actions.0.opacity', 'surfaces.1.rows.3.description']);
|
||||
const measure = () => page.evaluate(measureMaintenance, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
|
||||
@@ -33,6 +33,10 @@ test.afterEach(async ({ page, request, browserName }) => {
|
||||
const PREVIEW_URL = '/prompt_manager/autotag/batch/preview';
|
||||
const MODEL_OPTIONS = ['WD14 SwinV2 · ~400 MB', 'WD14 ViT · ~400 MB', 'GGUF JoyCaption · ~4.5 GB', 'HuggingFace JoyCaption · ~16 GB'];
|
||||
const SUGGESTIONS = 'From 1 to 100: the most suggestions Generate shows for one image.';
|
||||
const SKIPPED_DESCRIPTION = 'Suggested tags that contain one of these words are hidden. Words are separated by spaces, _ or -; '
|
||||
+ 'other characters such as ( ) belong to the word.';
|
||||
const DEFAULT_DESCRIPTION = 'The Tag image dialog and batches start from this.';
|
||||
const INSTRUCTIONS_DESCRIPTION = 'Instructions for the language-model engines. Built-in can’t be changed; duplicate it to make your own.';
|
||||
const BATCH = 'Tag each output image from its own pixels with the model above. Prompts are not changed. Review the scope first.';
|
||||
|
||||
/** Runs in the page: Auto-tagging's surface under `mainSelector`: rows, field, actions, operation, geometry and tokens. */
|
||||
@@ -62,7 +66,7 @@ function measureTagging(mainSelector) {
|
||||
text: [...row.firstElementChild.children].map(node => [name(node), text(node), node.localName === 'label' ? node.htmlFor : null]),
|
||||
control: control(row), flexWrap: getComputedStyle(row).flexWrap,
|
||||
controlBelow: row.lastElementChild.getBoundingClientRect().top >= row.firstElementChild.getBoundingClientRect().bottom - 0.5 })),
|
||||
field: { tree: [...field.children].map(name), label: [text(field.querySelector('label')), field.querySelector('label').htmlFor],
|
||||
field: field && { tree: [...field.children].map(name), label: [text(field.querySelector('label')), field.querySelector('label').htmlFor],
|
||||
textarea: { id: field.querySelector('textarea').id, value: field.querySelector('textarea').value,
|
||||
...css(field.querySelector('textarea'), 'fontSize', 'minHeight', 'backgroundColor', 'color', 'borderTopColor') },
|
||||
...css(field, 'display', 'rowGap', 'fontSize', 'marginTop', 'marginBottom') },
|
||||
@@ -70,6 +74,31 @@ function measureTagging(mainSelector) {
|
||||
...css(actions, 'display', 'flexWrap', 'columnGap', 'justifyContent'),
|
||||
primary: css(actions.querySelector('.ca-primary'), 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft'),
|
||||
plain: css(actions.querySelector('button:not(.ca-primary)'), 'backgroundColor', 'color', 'fontSize', 'borderTopColor') },
|
||||
// D202 (#431): the Skipped tags block (the concept has none).
|
||||
skipped: (block => block && { tree: [...block.children].map(name), key: block.dataset.key,
|
||||
text: [...block.firstElementChild.children].map(node => [node.localName, text(node), node.localName === 'label' ? node.htmlFor : null]),
|
||||
add: [...block.querySelector('.skip-add').children].map(node => [name(node), node.id, node.getAttribute('placeholder'), text(node)]),
|
||||
preview: [text(block.querySelector('.skip-preview')), block.querySelector('.skip-preview').getAttribute('role')],
|
||||
...css(block, 'paddingTop', 'paddingBottom', 'borderTopWidth', 'borderTopColor'),
|
||||
label: css(block.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
|
||||
description: css(block.querySelector('.setting-block-text p'), 'fontSize', 'color', 'marginTop'),
|
||||
input: css(block.querySelector('input'), 'fontSize', 'backgroundColor', 'color', 'borderTopColor') })(
|
||||
surface.querySelector(':scope > .setting-block[data-setting="tag-skipped"]')),
|
||||
// D203 (#431): Saved instructions (the concept has a template field instead).
|
||||
instructions: (block => block && { tree: [...block.children].map(name), key: block.dataset.key,
|
||||
text: [...block.firstElementChild.children].map(node => [node.localName, text(node)]),
|
||||
list: [...block.querySelectorAll('[role="option"]')].map(option => [text(option.firstElementChild),
|
||||
[...option.querySelectorAll('.badges > .chip')].map(chip => [text(chip), chip.className]), option.getAttribute('aria-selected')]),
|
||||
listRole: block.querySelector('.instruction-list').getAttribute('role'),
|
||||
actions: [...block.querySelectorAll('.instruction-actions > button')].map(item => [text(item), item.className, item.disabled]),
|
||||
editor: [...block.querySelectorAll('.instruction-editor .field')].map(item => [text(item.querySelector('label > span')),
|
||||
item.querySelector('input, textarea').value, item.querySelector('input, textarea').readOnly]),
|
||||
note: text(block.querySelector('#instruction-note')),
|
||||
layout: getComputedStyle(block.querySelector('.instructions-layout')).gridTemplateColumns.split(' ').length,
|
||||
...css(block, 'paddingTop', 'paddingBottom', 'borderTopWidth', 'borderTopColor'),
|
||||
label: css(block.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
|
||||
textarea: css(block.querySelector('textarea'), 'fontSize', 'backgroundColor', 'borderTopColor') })(
|
||||
surface.querySelector(':scope > .setting-block[data-setting="tag-instructions"]')),
|
||||
operations: [...surface.querySelectorAll(':scope > .operation')].map(row => ({ tree: [...row.children].map(name),
|
||||
title: text(row.querySelector('h3')), description: text(row.querySelector('p')),
|
||||
actions: [...row.querySelectorAll('.row-actions > button')].map(item => [text(item), item.className]) })),
|
||||
@@ -103,13 +132,21 @@ function measureTagging(mainSelector) {
|
||||
* The concept's Auto-tagging with the recorded deviations applied, for the fixture (all engines on disk, none loaded).
|
||||
* `phone` is a viewport at the concept's 700px breakpoint or below.
|
||||
*/
|
||||
function mapTaggingConcept(reference, { template, phone }) {
|
||||
function mapTaggingConcept(reference, { builtin, saved, phone }) {
|
||||
const [model, existing, general, character] = reference.rows;
|
||||
const [preview] = reference.actions.buttons;
|
||||
// D203: Built-in first, then the saved entries, in each default select.
|
||||
const names = ['Built-in', ...saved.map(entry => entry.name)];
|
||||
const defaultRow = (engine, long) => ({ ...existing,
|
||||
text: [['label', `Default for ${long}`, `instruction-default-${engine}`], ['p', DEFAULT_DESCRIPTION, null]],
|
||||
control: { ...existing.control, id: `instruction-default-${engine}`, options: names.map(label => [label, false]),
|
||||
selected: names[1] } });
|
||||
return {
|
||||
...reference,
|
||||
// D133 adds a row after the thresholds; D134 adds the operation after the actions.
|
||||
tree: [...reference.tree.slice(0, 6), 'div.setting-row', ...reference.tree.slice(6), 'div.operation'],
|
||||
// D133 adds a row after the thresholds, D202 the Skipped tags block after it; D203 replaces the template field
|
||||
// with Saved instructions, the two default rows and their note; D134 adds the operation after the actions.
|
||||
tree: [...reference.tree.slice(0, 6), 'div.setting-row', 'div.setting-block', 'div.setting-block', 'div.setting-row',
|
||||
'div.setting-row', 'p.instruction-reset.small', reference.tree[7], 'div.operation'],
|
||||
rows: [
|
||||
// D132 (OQ5, OQ6): the four real engines with their download size, and the chosen one's live status.
|
||||
{ ...model, text: [model.text[0], ['p', 'Downloaded. Not loaded.', null]],
|
||||
@@ -120,16 +157,49 @@ function mapTaggingConcept(reference, { template, phone }) {
|
||||
// D133: the persisted suggestion count, on the concept's number row.
|
||||
{ ...general, text: [['label', 'Suggestions per image', 'tag-suggestions'], ['p', SUGGESTIONS, null]],
|
||||
control: { ...general.control, id: 'tag-suggestions', value: '10' } },
|
||||
// D203: the default of each language-model engine (the fixture's saved entry is both), on the concept's
|
||||
// select row.
|
||||
defaultRow('gguf', 'GGUF JoyCaption'), defaultRow('hf', 'HuggingFace JoyCaption'),
|
||||
],
|
||||
// The saved template (the server's default until one is saved) replaces the concept's sample text.
|
||||
field: { ...reference.field, textarea: { ...reference.field.textarea, value: template } },
|
||||
// D203: Saved instructions in the concept's setting-row tokens (as D202), Built-in selected and read-only.
|
||||
instructions: { tree: ['div.setting-block-text', 'div.instructions-layout'], key: 'autotag_instructions',
|
||||
text: [['label', 'Saved instructions'], ['p', `${INSTRUCTIONS_DESCRIPTION} ${saved.length} of 20 saved · Built\u2011in is not counted.`]],
|
||||
list: [['Built-in', [['Read-only', 'chip']], 'true'],
|
||||
...saved.map((entry, index) => [entry.name, index === 0
|
||||
? [['Default for GGUF', 'chip blue'], ['Default for HuggingFace', 'chip blue']] : [], 'false'])],
|
||||
listRole: 'listbox',
|
||||
actions: [['+ New', '', false], ['Duplicate', '', false], ['Delete', 'danger', true]],
|
||||
editor: [['Name', 'Built-in', true], ['Text', builtin, true]],
|
||||
note: 'Built-in instructions can’t be changed. Duplicate them to make your own.',
|
||||
layout: phone ? 1 : 2,
|
||||
paddingTop: reference.row.paddingTop, paddingBottom: reference.row.paddingBottom,
|
||||
borderTopWidth: reference.row.borderTopWidth, borderTopColor: reference.row.borderTopColor,
|
||||
label: reference.label,
|
||||
textarea: { fontSize: reference.field.textarea.fontSize, backgroundColor: reference.field.textarea.backgroundColor,
|
||||
borderTopColor: reference.field.textarea.borderTopColor } },
|
||||
// D202 (#431): the Skipped tags block in the concept's setting-row tokens: its label and description as a row's,
|
||||
// the row's padding and rule above, the concept's field input; an empty list says so.
|
||||
skipped: { tree: ['div.setting-block-text', 'div.skip-add', 'p.skip-problem.small', 'div.skip-chips.tags', 'p.skip-preview.small'],
|
||||
key: 'autotag_skipped_tags',
|
||||
text: [['label', 'Skipped tags', 'tag-skipped-input'], ['p', SKIPPED_DESCRIPTION, null]],
|
||||
add: [['input', 'tag-skipped-input', 'Add a word or words', ''], ['button', 'tag-skipped-add', null, 'Add']],
|
||||
preview: ['No tags are skipped.', 'status'],
|
||||
paddingTop: reference.row.paddingTop, paddingBottom: reference.row.paddingBottom,
|
||||
borderTopWidth: reference.row.borderTopWidth, borderTopColor: reference.row.borderTopColor,
|
||||
label: reference.label,
|
||||
description: { fontSize: reference.description.fontSize, color: reference.description.color,
|
||||
marginTop: reference.description.marginTop },
|
||||
input: { fontSize: '12px', backgroundColor: reference.field.textarea.backgroundColor, color: reference.field.textarea.color,
|
||||
borderTopColor: reference.field.textarea.borderTopColor } },
|
||||
// D203: no template field.
|
||||
field: null,
|
||||
// D132: Download (for an engine already on disk, "Download again") and Unload replace the two simulated jobs.
|
||||
actions: { ...reference.actions, buttons: [preview, ['Download again', ''], ['Unload', '']] },
|
||||
// D134: the prompt-tag batch, reviewed first.
|
||||
operations: [{ tree: ['div', 'div.row-actions'], title: 'Tag output images', description: BATCH, actions: [['Review', '']] }],
|
||||
// D132 (#398): every select shows its whole chosen option at every width (at 390 the concept's 120px cap cuts
|
||||
// them; production lets each grow past it).
|
||||
selectsFit: reference.selectsFit.map(() => true),
|
||||
selectsFit: [...reference.selectsFit, true, true].map(() => true),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -153,7 +223,9 @@ async function productionTagging(page, theme) {
|
||||
async function savedTemplate(request, page) {
|
||||
const response = await request.get('/prompt_manager/settings');
|
||||
await expect(response).toBeOK();
|
||||
return { template: (await response.json()).settings.autotag_template, phone: page.viewportSize().width <= 700 };
|
||||
const { settings } = await response.json();
|
||||
return { builtin: settings.autotag_builtin_instructions, saved: settings.autotag_instructions,
|
||||
phone: page.viewportSize().width <= 700 };
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
@@ -177,14 +249,15 @@ for (const theme of THEMES) for (const width of [390, 320]) {
|
||||
await page.locator('#workspace-main #tag-existing').selectOption('replace_after_review');
|
||||
await page.locator('#workspace-main section.surface').screenshot({ path: testInfo.outputPath(`production-tagging-selects-${theme}-${width}.png`) });
|
||||
const selects = page.locator('#workspace-main section.surface > .setting-row > select');
|
||||
await expect(selects).toHaveCount(2);
|
||||
for (let index = 0; index < 2; index += 1) {
|
||||
// D203 adds the two per-engine default selects.
|
||||
await expect(selects).toHaveCount(4);
|
||||
for (let index = 0; index < 4; index += 1) {
|
||||
const select = selects.nth(index);
|
||||
const values = await select.evaluate(node => [...node.options].map(option => option.value));
|
||||
for (const value of values) {
|
||||
await select.selectOption(value);
|
||||
const measured = await page.evaluate(measureTagging, '#workspace-main');
|
||||
expect(measured.selectsFit, `${await select.getAttribute('id')}=${value}`).toEqual([true, true]);
|
||||
expect(measured.selectsFit, `${await select.getAttribute('id')}=${value}`).toEqual([true, true, true, true]);
|
||||
expect(measured.pageScroll, value).toBe(false);
|
||||
}
|
||||
}
|
||||
@@ -198,12 +271,16 @@ test('auto-tagging comparison detects mutations', async ({ page, request }) => {
|
||||
const expected = mapTaggingConcept(concept, saved);
|
||||
const production = await productionTagging(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the section differs only where D132, D133 and D134 are recorded (and the saved template's text).
|
||||
expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 3).join('.'))
|
||||
// Unmapped, the section differs only where D132, D133, D134, D202 and D203 are recorded (and the saved template's text).
|
||||
// (The concept has no Skipped tags block, so every path under it is one D202 difference.)
|
||||
expect(diffInventory(concept, production).map(item => (/^(skipped|instructions|field)\b/.test(item.path) ? item.path.split('.')[0]
|
||||
: item.path.split('.').slice(0, 3).join('.')))
|
||||
.filter((path, index, all) => all.indexOf(path) === index).sort()).toEqual([
|
||||
'actions.buttons.1', 'actions.buttons.2', 'field.textarea.value', 'operations.0.actions', 'operations.0.description',
|
||||
'operations.0.title', 'operations.0.tree', 'operations.length', 'rows.0.control', 'rows.0.text', 'rows.4.control', 'rows.4.controlBelow',
|
||||
'rows.4.flexWrap', 'rows.4.text', 'rows.4.tree', 'rows.length', 'tree.6', 'tree.7', 'tree.8', 'tree.9', 'tree.length'].sort());
|
||||
'actions.buttons.1', 'actions.buttons.2', 'field', 'instructions', 'operations.0.actions', 'operations.0.description',
|
||||
'operations.0.title', 'operations.0.tree', 'operations.length', 'rows.0.control', 'rows.0.text',
|
||||
...[4, 5, 6].flatMap(row => ['control', 'controlBelow', 'flexWrap', 'text', 'tree'].map(key => `rows.${row}.${key}`)),
|
||||
'rows.length', 'selectsFit.2', 'selectsFit.3', 'selectsFit.length', 'skipped',
|
||||
...Array.from({ length: 8 }, (_, index) => `tree.${index + 6}`), 'tree.length'].sort());
|
||||
const measure = () => page.evaluate(measureTagging, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
// A removed row.
|
||||
@@ -254,8 +331,8 @@ test('auto-tagging at 390 wraps only the engine row', async ({ page, request })
|
||||
const concept = await conceptTagging(page, 'dark');
|
||||
const production = await productionTagging(page, 'dark');
|
||||
expect(diffInventory(mapTaggingConcept(concept, await savedTemplate(request, page)), production)).toEqual([]);
|
||||
expect(production.rows.map(row => row.flexWrap)).toEqual(['wrap', 'nowrap', 'nowrap', 'nowrap', 'nowrap']);
|
||||
expect(production.selectsFit).toEqual([true, true]);
|
||||
expect(production.rows.map(row => row.flexWrap)).toEqual(['wrap', 'nowrap', 'nowrap', 'nowrap', 'nowrap', 'nowrap', 'nowrap']);
|
||||
expect(production.selectsFit).toEqual([true, true, true, true]);
|
||||
});
|
||||
|
||||
/** Runs in the page: the open review dialog: its title, body and the concept's tokens. */
|
||||
|
||||
@@ -92,6 +92,8 @@ export function cardActionsInventory(card) {
|
||||
};
|
||||
}
|
||||
|
||||
// #431: the fixture's saved "Describe the image" entry, the default of both engines (workspace_tagging_fixture.py).
|
||||
export const FIXTURE_INSTRUCTIONS_ID='d35c71be0a1e';
|
||||
// D84 (plan D61): the legacy engines' values, labels and sizes, GGUF JoyCaption in the concept's Ollama slot.
|
||||
const MODEL_LABELS=Object.freeze({'wd14-swinv2':['wd14-swinv2','WD14 SwinV2 · ~400 MB'],'wd14-vit':['wd14-vit','WD14 ViT · ~400 MB'],
|
||||
ollama:['gguf','GGUF JoyCaption (Recommended) · ~4.5 GB'],huggingface:['hf','HuggingFace JoyCaption · ~16 GB']});
|
||||
@@ -106,17 +108,24 @@ const CONCEPT_LABELS=Object.freeze({'WD14 SwinV2 · fast classifier':'WD14 SwinV
|
||||
* (`chip`: a concept `.chip` inventory, the heading's size chip) around the row's own text (12px) in the checked
|
||||
* row's colours (accent border, `--soft-blue`);
|
||||
* - D84 (plan D61): the legacy engine labels and values, and the server's default instructions (`instructions`);
|
||||
* - D204 (#431): a "Saved instructions" select (`picker`: its value and options) before the instructions, on the
|
||||
* concept's select field tokens;
|
||||
* - D91 (plan D68): a Suggestions count field after the model's own fields, on the concept's number field tokens
|
||||
* (`numberField`: the concept's General confidence field) with the configured count (`limit`);
|
||||
* - D29: the prototype's "Sample suggestions from …" status reads "Suggestions from …".
|
||||
* The opened state is compared unmapped: both show the concept's on-screen "Generate suggestions with the selected
|
||||
* model before approving." (its renderTagModelOptions, a1062–1076, replaces the markup's sentence at once).
|
||||
*/
|
||||
export function mapConceptTagging(reference,{instructions,limit=10,chip,numberField}) {
|
||||
export function mapConceptTagging(reference,{instructions,limit=10,chip,numberField,
|
||||
picker={value:FIXTURE_INSTRUCTIONS_ID,options:[['builtin','Built-in'],[FIXTURE_INSTRUCTIONS_ID,`${instructions} · default`]]}}) {
|
||||
const [select,...own]=reference.modelPanel.fields;
|
||||
// D204 (#431): a "Saved instructions" select on the concept's select field, before the instructions.
|
||||
const pickField={...select,label:'Saved instructions',control:{...select.control,id:'image-tag-instructions-pick',
|
||||
value:picker.value,options:picker.options}};
|
||||
const suggestionsField={...numberField,label:'Suggestions',control:{...numberField.control,id:'image-suggestion-limit',
|
||||
value:String(limit),min:'1',max:'100',step:'1'}};
|
||||
const mappedOwn=own.map(field=>field.control.id==='image-tag-instructions'?{...field,control:{...field.control,value:instructions}}:field);
|
||||
const mappedOwn=own.flatMap(field=>field.control.id==='image-tag-instructions'
|
||||
?[pickField,{...field,control:{...field.control,value:instructions}}]:[field]);
|
||||
const rows=reference.suggestions.rows.map(row=>({...row,
|
||||
control:row.control?.kind==='checkbox'&&row.control.checked?{kind:'remove',label:`Remove ${row.name}`}:row.control,
|
||||
borderTopLeftRadius:chip.borderTopLeftRadius}));
|
||||
|
||||
@@ -191,7 +191,7 @@ test('library toolbar matches the concept and tags live in Refine this view',asy
|
||||
expect(await toolbar.evaluate(node=>[...node.children].map(child=>child.tagName.toLowerCase()+(child.id?`#${child.id}`:'')+(child.className?`.${child.className.split(' ').filter(name=>!name.startsWith('ca-')).join('.')}`:'')))).toEqual(['label.search-field','select#category-filter','select#sort','button','button']);
|
||||
await expect(toolbar.locator('#search')).toHaveAttribute('placeholder','Search prompts, tags, filenames…');
|
||||
// D153 (#447): no "Highest rated".
|
||||
await expect(toolbar.locator('#sort option')).toHaveText(['Recently added','Name A–Z','Oldest first']);
|
||||
await expect(toolbar.locator('#sort option')).toHaveText(['Recently added','Name A–Z','Oldest first','Recently used','Most used','Name Z–A']);
|
||||
await expect(toolbar.getByRole('button')).toHaveText(['Save view','Filters']);
|
||||
await expect(page.getByRole('button',{name:'Refresh library',exact:true})).toHaveCount(0);
|
||||
await expect(page.getByRole('textbox',{name:'Tags',exact:true})).toHaveCount(0);
|
||||
|
||||
@@ -205,7 +205,7 @@ test('a view stored under older rules still lists and opens with its usable valu
|
||||
return (await response.json()).id;
|
||||
};
|
||||
const imagesId = await raw({ page: 'images', name: 'Old cats', state: JSON.stringify({ q: 'cat', sort: 'shuffle', layout: 'grid' }) });
|
||||
await raw({ page: 'library', name: '', state: JSON.stringify({ q: 'dog', sort: 'name-desc' }) });
|
||||
await raw({ page: 'library', name: '', state: JSON.stringify({ q: 'dog', sort: 'shuffle' }) });
|
||||
expect(await listed(request)).toEqual([
|
||||
{ id: imagesId, page: 'images', name: 'Old cats', state: { ...IMAGE_DEFAULTS, q: 'cat' } },
|
||||
expect.objectContaining({ page: 'library', name: 'Saved view', state: { q: 'dog', category: '', sort: 'newest', tags: [], tag_mode: 'all', folder: '' } }),
|
||||
|
||||
@@ -64,7 +64,9 @@ test('the section shows the concept rows; Metadata cache follows Q5 (D102)', asy
|
||||
item.querySelector('.row-actions > button').textContent, item.querySelector('.row-actions > button').className]),
|
||||
]))).toEqual([
|
||||
['Database', [['Integrity & relationship checks', 'Check', ''], ['Duplicate prompt text', 'Review plan', ''],
|
||||
['Optimize database', 'Review plan', ''], ['Unused prompt components', 'Review plan', 'danger']]],
|
||||
['Optimize database', 'Review plan', ''], ['Unused prompt components', 'Review plan', 'danger'],
|
||||
// D192 (#550): beyond the concept.
|
||||
['Pair category and tags', 'Review plan', '']]],
|
||||
['Images & cache', [['Missing-file links', 'Review plan', 'danger'], ['Duplicate image files', 'Find duplicates', ''],
|
||||
['Thumbnail cache', 'Review options', ''], ['Metadata cache', 'Review plan', '']]],
|
||||
]);
|
||||
@@ -177,6 +179,70 @@ test("plan dialog shows the server's scope, protected, backup policy and media l
|
||||
expect(await serverJson(request, '/__test__/system/digest')).toEqual(before);
|
||||
});
|
||||
|
||||
test('D192: Pair category and tags reviews old Pair prompts, keeps an edited one and fills the other (#550)',
|
||||
async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/pair-metadata')).toBeOK();
|
||||
const answers = [];
|
||||
page.on('response', response => { if (response.url().endsWith(PREVIEW_URL)) answers.push(response.json()); });
|
||||
await openMaintenance(page);
|
||||
const pairRow = row(page, 'Pair category and tags');
|
||||
await expect(pairRow.locator('p')).toHaveText('Fill an empty category or tags on prompts older Pair node runs saved, ' +
|
||||
'from their saved pairs. Only empty fields of never-edited prompts.');
|
||||
await pairRow.getByRole('button', { name: 'Review plan', exact: true }).click();
|
||||
const plan = dialog(page);
|
||||
await expect(plan.locator('#modal-body > h3')).toHaveText('Pair category and tags');
|
||||
const answer = await answers[0];
|
||||
expect(answer.operation).toBe('pair_metadata');
|
||||
expect(await gridOf(page)).toEqual([
|
||||
['Scope', '1 prompt to fill: 1 category, 1 tag set'],
|
||||
['Protected', '1 edited after it was saved'],
|
||||
['Backup', `Create and verify before changes (${answer.backup.destination_label})`],
|
||||
['Media on disk', 'No image file or saved pair is changed.'],
|
||||
]);
|
||||
await expect(plan.locator('details > summary')).toHaveText('Listed records (3)');
|
||||
await plan.locator('details > summary').click();
|
||||
// Protected rows come first, the Record row last.
|
||||
const lines = (await plan.locator('details > pre').textContent()).split('\n');
|
||||
expect(lines).toHaveLength(3);
|
||||
expect(lines[0]).toBe('S550 harbor · Kept: edited after it was saved; its empty category or tags may be intentional');
|
||||
expect(lines[1]).toMatch(/^S550 lantern · Fills category “night” and 2 tags: s550-glow, s550-warm \(pair \d+\)$/);
|
||||
expect(lines[2]).toMatch(/^Record · The job's report lists every filled field\. Download it to keep it/);
|
||||
await plan.getByRole('button', { name: 'Apply reviewed operation', exact: true }).click();
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
|
||||
const job = await ended(request, page.url().split('/').pop());
|
||||
expect([job.title, job.state, job.counters.changed, job.counters.protected])
|
||||
.toEqual(['Pair category and tags', 'finished', 1, 1]);
|
||||
const main = page.locator('#workspace-main');
|
||||
await expect(main.locator('h1')).toHaveText('Pair category and tags');
|
||||
await expect(main.getByRole('button', { name: 'Download report', exact: true })).toBeVisible();
|
||||
// #550 live test: the Changed group lists the path-less record rows as plain text, never an open action.
|
||||
const results = main.getByRole('region', { name: 'Results', exact: true });
|
||||
await results.locator('.outcome-list > button').filter({ hasText: 'Changed' }).click();
|
||||
const records = results.locator('.outcome-files > .file-row');
|
||||
await expect(records).toHaveCount(1);
|
||||
await expect(records.first()).toHaveText(/^Prompt \d+: category “night”; tags s550-glow, s550-warm$/);
|
||||
expect(await records.first().evaluate(node => node.localName)).toBe('div');
|
||||
await expect(results.locator('.outcome-files button')).toHaveCount(0);
|
||||
await expect(results.getByRole('searchbox', { name: 'Filter results' })).toHaveAttribute('placeholder', 'Filter records…');
|
||||
// Protected lists each kept prompt with the review's reason (#550 live re-check).
|
||||
await results.locator('.outcome-list > button').filter({ hasText: 'Protected' }).click();
|
||||
await expect(records).toHaveCount(1);
|
||||
await expect(records.first()).toHaveText(
|
||||
/^Prompt \d+: kept, edited after it was saved; its empty category or tags may be intentional$/);
|
||||
// An empty group says records, not files.
|
||||
await results.locator('.outcome-list > button').filter({ hasText: 'Errors' }).click();
|
||||
await expect(results.locator('.outcome-files > p.empty-state')).toHaveText('No records in this group.');
|
||||
const library = await serverJson(request, '/prompt_manager/library');
|
||||
const lantern = library.items.find(item => item.text === 'S550 lantern');
|
||||
const harbor = library.items.find(item => item.text === 'S550 harbor');
|
||||
const filled = (await serverJson(request, `/prompt_manager/library/${lantern.id}?archive=all`)).positive;
|
||||
expect([filled.category, [...filled.tags].sort()]).toEqual(['night', ['s550-glow', 's550-warm']]);
|
||||
expect((await serverJson(request, `/prompt_manager/library/${harbor.id}?archive=all`)).positive.category).toBeFalsy();
|
||||
await page.goto(`${workspace}#/library`);
|
||||
await expect(page.locator(`article[data-prompt-id="${lantern.id}"] .card-title > .chip`)).toHaveText('night');
|
||||
});
|
||||
|
||||
test('a plan with nothing to do shows only Close, never Apply', async ({ page, request }) => {
|
||||
const applies = [];
|
||||
page.on('request', item => { if (item.url().endsWith(APPLY_URL)) applies.push(item.url()); });
|
||||
|
||||
@@ -112,13 +112,14 @@ test('model options are the four real engines with size and the description show
|
||||
await expect(status(page)).toHaveText('Unsaved changes.');
|
||||
// The other rows keep the concept's copy and the saved values (D133 adds Suggestions per image).
|
||||
expect(await surface(page).locator(':scope > .setting-row label').allTextContents()).toEqual(['Tagging model', 'Existing tags',
|
||||
'General-tag confidence', 'Character-tag confidence', 'Suggestions per image']);
|
||||
'General-tag confidence', 'Character-tag confidence', 'Suggestions per image', 'Default for GGUF JoyCaption',
|
||||
'Default for HuggingFace JoyCaption']);
|
||||
await expect(page.locator('#tag-existing')).toHaveValue('skip');
|
||||
await expect(page.locator('#threshold-general')).toHaveValue('0.35');
|
||||
await expect(page.locator('#threshold-character')).toHaveValue('0.85');
|
||||
await expect(page.locator('#tag-suggestions')).toHaveValue('10');
|
||||
const settings = await serverJson(request, SETTINGS_URL);
|
||||
await expect(page.locator('#tag-template')).toHaveValue(settings.settings.autotag_template);
|
||||
// D203 (#431): Saved instructions replace the template field.
|
||||
await expect(page.locator('#tag-template')).toHaveCount(0);
|
||||
// Unload is offered only while a model is loaded (none is).
|
||||
await expect(actionOf(page, 'Unload')).toBeDisabled();
|
||||
});
|
||||
@@ -418,3 +419,156 @@ test('a model that is not downloaded cannot be reviewed for a batch; the button
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
expect(page.modelCalls.filter(call => call.includes('/batch/preview'))).toEqual([]);
|
||||
});
|
||||
|
||||
// D202 (#431): Skipped tags: add (button and Enter), the preview over the library's tags and the WD14 label list, the
|
||||
// per-chip counts, remove, Save preferences and a reload; a list the server refuses highlights the block.
|
||||
const SKIPPED_PREVIEW = '/prompt_manager/autotag/skipped-tags/preview';
|
||||
const skippedBlock = page => surface(page).locator(':scope > .setting-block[data-setting="tag-skipped"]');
|
||||
const skippedInput = page => page.locator('#tag-skipped-input');
|
||||
const skippedChips = page => skippedBlock(page).locator('.skip-chips .chip.tag-suggestion');
|
||||
const skippedPreview = page => skippedBlock(page).locator('.skip-preview');
|
||||
|
||||
test('skipped tags add, preview, remove, save and reload (D202)', async ({ page, request }) => {
|
||||
await openTagging(page);
|
||||
await expect(skippedBlock(page).locator('label')).toHaveText('Skipped tags');
|
||||
await expect(skippedBlock(page)).toHaveAttribute('data-key', 'autotag_skipped_tags');
|
||||
await expect(skippedPreview(page)).toHaveText('No tags are skipped.');
|
||||
await expect(skippedPreview(page)).toHaveAttribute('role', 'status');
|
||||
// The label list on disk (blurry, blurry_background, signature, …) and the library tags (warm_light, …).
|
||||
let answer = page.waitForResponse(item => item.url().endsWith(SKIPPED_PREVIEW));
|
||||
await skippedInput(page).fill('blurry');
|
||||
await skippedInput(page).press('Enter');
|
||||
expect((await answer).status()).toBe(200);
|
||||
await expect(skippedInput(page)).toHaveValue('');
|
||||
await expect(skippedPreview(page)).toHaveText('Hides 2 tags from suggestions: blurry, blurry_background. '
|
||||
+ 'Tags already on images are not changed.');
|
||||
answer = page.waitForResponse(item => item.url().endsWith(SKIPPED_PREVIEW));
|
||||
await skippedInput(page).fill('light');
|
||||
await page.getByRole('button', { name: 'Add', exact: true }).click();
|
||||
await answer;
|
||||
await expect(skippedPreview(page)).toHaveText('Hides 3 tags from suggestions: warm_light, blurry, blurry_background. '
|
||||
+ 'Tags already on images are not changed.');
|
||||
await expect(skippedChips(page)).toHaveText(['blurry2×', 'light1×']);
|
||||
await expect(skippedChips(page).first()).toHaveAttribute('title', 'Hides 2 tags');
|
||||
// An entry that hides nothing is muted; a duplicate is not added and the line under the input says so.
|
||||
answer = page.waitForResponse(item => item.url().endsWith(SKIPPED_PREVIEW));
|
||||
await skippedInput(page).fill('zebra');
|
||||
await skippedInput(page).press('Enter');
|
||||
await answer;
|
||||
await expect(skippedChips(page).last()).toHaveClass(/\bnone\b/);
|
||||
await expect(skippedChips(page).last()).toHaveAttribute('title', 'Hides no known tag');
|
||||
await skippedInput(page).fill('Blurry');
|
||||
await skippedInput(page).press('Enter');
|
||||
await expect(page.locator('#tag-skipped-problem')).toHaveText('Already in the list.');
|
||||
await expect(skippedInput(page)).toHaveAttribute('aria-invalid', 'true');
|
||||
await skippedInput(page).fill('a, b');
|
||||
await skippedInput(page).press('Enter');
|
||||
await expect(page.locator('#tag-skipped-problem')).toHaveText('No commas or line breaks.');
|
||||
// Remove zebra, then save.
|
||||
await skippedChips(page).last().getByRole('button', { name: 'Remove zebra', exact: true }).click();
|
||||
await expect(skippedChips(page)).toHaveCount(2);
|
||||
await expect(status(page)).toHaveText('Unsaved changes.');
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).settings.autotag_skipped_tags).toEqual(['blurry', 'light']);
|
||||
await openTagging(page);
|
||||
await expect(skippedChips(page)).toHaveText(['blurry2×', 'light1×']);
|
||||
// Removing a chip and adding it back is no change.
|
||||
await skippedChips(page).last().getByRole('button', { name: 'Remove light', exact: true }).click();
|
||||
await expect(status(page)).toHaveText('Unsaved changes.');
|
||||
await skippedInput(page).fill('light');
|
||||
await skippedInput(page).press('Enter');
|
||||
await expect(status(page)).toHaveText('No unsaved changes.');
|
||||
});
|
||||
|
||||
test('skipped tags preview checks library tags only without a label list, and a refused save highlights it', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/tagging/defaults', { data: { general: 0.35, character: 0.85, limit: 10,
|
||||
labels: false } })).toBeOK();
|
||||
await openTagging(page);
|
||||
const answer = page.waitForResponse(item => item.url().endsWith(SKIPPED_PREVIEW));
|
||||
await skippedInput(page).fill('blurry');
|
||||
await skippedInput(page).press('Enter');
|
||||
await answer;
|
||||
await expect(skippedPreview(page)).toHaveText('Hides 0 tags from suggestions. Checked against library tags only; no WD14 '
|
||||
+ 'label list is downloaded. Tags already on images are not changed.');
|
||||
// The server is the authority: a list it refuses keeps the draft, says why and highlights the block.
|
||||
await page.route(`**${SETTINGS_URL}`, route => (route.request().method() === 'POST'
|
||||
? route.fulfill({ status: 400, contentType: 'application/json',
|
||||
body: JSON.stringify({ success: false, error: 'Skipped tags: entry 1 is empty.' }) })
|
||||
: route.fallback()));
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Skipped tags: entry 1 is empty.');
|
||||
await expect(skippedBlock(page)).toHaveAttribute('aria-invalid', 'true');
|
||||
await page.unrouteAll({ behavior: 'ignoreErrors' });
|
||||
});
|
||||
|
||||
// D203 (#431): Saved instructions: Built-in read-only, New, edit, per-engine defaults, Delete resets a default, Save
|
||||
// preferences and a reload; the server pairs the two keys. D205: the batch review names the engine's default.
|
||||
const instructionOption = (page, name) => page.locator('#instruction-list [role="option"]').filter({ has: page.locator(`span:text-is("${name}")`) });
|
||||
|
||||
test('saved instructions: new, edit, per-engine default, delete, save and reload (D203)', async ({ page, request }) => {
|
||||
await openTagging(page);
|
||||
const settings = (await serverJson(request, SETTINGS_URL)).settings;
|
||||
await expect(page.locator('#instructions-count')).toHaveText('1 of 20 saved · Built\u2011in is not counted.');
|
||||
await expect(page.locator('#instruction-list [role="option"] > span:first-child')).toHaveText(['Built-in', 'Describe the image']);
|
||||
await expect(page.locator('#instruction-name')).toHaveValue('Built-in');
|
||||
await expect(page.locator('#instruction-text')).toHaveValue(settings.autotag_builtin_instructions);
|
||||
await expect(page.locator('#instruction-text')).toHaveAttribute('readonly', '');
|
||||
await expect(page.locator('#instruction-delete')).toBeDisabled();
|
||||
await expect(page.locator('#instruction-default-gguf')).toHaveValue(settings.autotag_instructions[0].id);
|
||||
// + New, then name and text it, and make it HuggingFace's default.
|
||||
await page.locator('#instruction-new').click();
|
||||
await expect(page.locator('#instruction-name')).toHaveValue('New instructions');
|
||||
await expect(page.locator('#instruction-note')).toHaveText('The text must not be empty.');
|
||||
await page.locator('#instruction-name').fill('Photo realism');
|
||||
await page.locator('#instruction-text').fill('Tag camera angle and lighting.');
|
||||
await expect(page.locator('#instruction-text').locator('xpath=..').locator('label .muted')).toHaveText('30 of 4,000 characters');
|
||||
await page.locator('#instruction-default-hf').selectOption({ label: 'Photo realism' });
|
||||
await expect(instructionOption(page, 'Photo realism').locator('.chip')).toHaveText(['Default for HuggingFace']);
|
||||
await expect(status(page)).toHaveText('Unsaved changes.');
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
let saved = (await serverJson(request, SETTINGS_URL)).settings;
|
||||
expect(saved.autotag_instructions.map(entry => [entry.name, entry.text])).toEqual([['Describe the image', 'Describe the image'],
|
||||
['Photo realism', 'Tag camera angle and lighting.']]);
|
||||
expect(saved.autotag_default_instructions.hf).toBe(saved.autotag_instructions[1].id);
|
||||
// A reload shows the saved list; deleting GGUF's default resets it to Built-in and says so.
|
||||
await openTagging(page);
|
||||
await instructionOption(page, 'Describe the image').click();
|
||||
await page.locator('#instruction-delete').click();
|
||||
await expect(page.locator('#instruction-reset')).toHaveText('GGUF now uses Built-in.');
|
||||
await expect(page.locator('#instruction-default-gguf')).toHaveValue('builtin');
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
saved = (await serverJson(request, SETTINGS_URL)).settings;
|
||||
expect(saved.autotag_instructions.map(entry => entry.name)).toEqual(['Photo realism']);
|
||||
expect(saved.autotag_default_instructions.gguf).toBe('builtin');
|
||||
expect(saved.autotag_template).toBe(saved.autotag_builtin_instructions);
|
||||
});
|
||||
|
||||
test('a refused instructions save names the entry and highlights the block (D203)', async ({ page }) => {
|
||||
await openTagging(page);
|
||||
await page.locator('#instruction-duplicate').click();
|
||||
await page.locator('#instruction-name').fill('describe the IMAGE');
|
||||
await expect(page.locator('#instruction-note')).toHaveText('The name “describe the IMAGE” is already used.');
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved instructions: entry 2 name “describe the IMAGE” is already used.');
|
||||
await expect(surface(page).locator('.setting-block[data-setting="tag-instructions"]')).toHaveAttribute('aria-invalid', 'true');
|
||||
});
|
||||
|
||||
test('the batch review names the language model\'s default instructions (D205)', async ({ page }) => {
|
||||
await openTagging(page);
|
||||
await page.locator('#tag-model').selectOption('gguf');
|
||||
await page.locator('#tag-existing').selectOption('merge');
|
||||
const answer = page.waitForResponse(item => item.url().endsWith(PREVIEW_URL));
|
||||
await reviewButton(page).click();
|
||||
await answer;
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Review tag output images');
|
||||
const grid = await gridOf(page);
|
||||
expect(grid.slice(-3)).toEqual([['Instructions', 'Describe the image'], ['Settings', 'Up to 10 tags per image'],
|
||||
grid.at(-1)]);
|
||||
// The batch sends the default's text.
|
||||
const start = page.waitForRequest(item => item.url().endsWith('/prompt_manager/jobs/autotag'));
|
||||
await dialog(page).getByRole('button', { name: 'Start tagging', exact: true }).click();
|
||||
expect((await start).postDataJSON().options).toEqual({ instructions: 'Describe the image' });
|
||||
});
|
||||
|
||||
@@ -341,3 +341,21 @@ test('a running scan makes the start report busy', async ({ page, request }) =>
|
||||
await request.post(`/__test__/jobs/${scan.job.id}/finish`, { data: { state: 'finished' } });
|
||||
}
|
||||
});
|
||||
|
||||
// D204 (#431): Auto-tag selected has the same picker; the job sends the text the dialog shows.
|
||||
test('Auto-tag selected picks saved instructions and sends the text shown (D204)', async ({ page }) => {
|
||||
await select(page, FIRST);
|
||||
await barButton(page, 'Auto-tag selected').click();
|
||||
await expect(title(page)).toHaveText('Auto-tag selected images');
|
||||
const body = modal(page).locator('#modal-body');
|
||||
await body.getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('gguf');
|
||||
const pick = body.getByRole('combobox', { name: 'Saved instructions', exact: true });
|
||||
await expect(pick).toHaveAttribute('id', 'bulk-tag-instructions-pick');
|
||||
await expect(pick.locator('option')).toHaveText(['Built-in', 'Describe the image · default']);
|
||||
const text = body.getByRole('textbox', { name: 'Tagging instructions', exact: true });
|
||||
await text.fill('Bulk one-off.');
|
||||
await expect(pick.locator('option:checked')).toHaveText('Edited for this run');
|
||||
const started = jobStart(page);
|
||||
await actionButtons(page).filter({ hasText: 'Start auto-tagging' }).click();
|
||||
expect((await started).postDataJSON().options.instructions).toBe('Bulk one-off.');
|
||||
});
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { saveImageTags } from './concept-inventory.mjs';
|
||||
import { taggingInventory } from './tagging-canonical-comparison.mjs';
|
||||
import { FIXTURE_INSTRUCTIONS_ID, taggingInventory } from './tagging-canonical-comparison.mjs';
|
||||
|
||||
const images = '/prompt_manager/workspace#/images';
|
||||
const MODULE = '/prompt_manager/js/workspace/image-tagging.mjs';
|
||||
@@ -849,3 +849,61 @@ test('a failed listing re-read after a save says so and keeps the cards (M2)', a
|
||||
await expect(page.locator('#gallery-continuation [aria-disabled="true"]')).toHaveCount(0);
|
||||
await page.unroute(failing);
|
||||
});
|
||||
|
||||
// #431: the saved Skipped tags hide matching suggestions before the limit (WD14 fills it from lower scores), and the
|
||||
// chip hint says how many were not suggested.
|
||||
test('suggestions omit skipped tags and say how many were hidden', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/tagging/defaults', { data: { general: 0.35, character: 0.85, limit: 4,
|
||||
skipped: ['desert', 'rock'] } })).toBeOK();
|
||||
await openTag(page);
|
||||
const answer = await (await generated(page)).json();
|
||||
expect(answer.skipped).toBe(2);
|
||||
await expect(chips(page).locator(':scope > span:first-child')).toHaveText(['sandstone', 'arch', 'sky', 'dune']);
|
||||
await expect(dialog(page).locator('#tag-suggestions > p.small')).toHaveText(`${CHIP_HINT} 2 skipped tags were not suggested.`);
|
||||
});
|
||||
|
||||
// D204 (#431): the Saved instructions select above Tagging instructions: the engine default, a pick, a one-off edit
|
||||
// ("Edited for this run" and the muted note), an engine switch resets, WD14 hides both; Generate sends the text shown.
|
||||
test('the saved instructions picker fills, edits one-off and resets per engine (D204)', async ({ page, request }) => {
|
||||
const settings = await (await request.get('/prompt_manager/settings')).json();
|
||||
const builtin = settings.settings.autotag_builtin_instructions;
|
||||
await openTag(page);
|
||||
const options = dialog(page).locator('#image-tag-model-options');
|
||||
const pick = options.getByRole('combobox', { name: 'Saved instructions', exact: true });
|
||||
const text = options.getByRole('textbox', { name: 'Tagging instructions', exact: true });
|
||||
const note = options.locator('.instructions-edited');
|
||||
await expect(pick).toHaveCount(0);
|
||||
await model(page).selectOption('gguf');
|
||||
await expect(pick.locator('option')).toHaveText(['Built-in', 'Describe the image · default']);
|
||||
await expect(pick).toHaveAttribute('id', 'image-tag-instructions-pick');
|
||||
await expect(text).toHaveValue('Describe the image');
|
||||
await expect(note).toBeHidden();
|
||||
// A keyboard pick (the select focused) keeps focus on the select the fields are re-rendered with.
|
||||
await pick.focus();
|
||||
await pick.selectOption('builtin');
|
||||
await expect(text).toHaveValue(builtin);
|
||||
await expect(pick).toBeFocused();
|
||||
await text.fill('Only the sky.');
|
||||
await expect(text).toHaveAttribute('aria-describedby', 'image-tag-instructions-pick-note');
|
||||
await expect(pick.locator('option:checked')).toHaveText('Edited for this run');
|
||||
await expect(pick.locator('option[value="edited"]')).toBeDisabled();
|
||||
await expect(note).toHaveText('Edits apply to this run only. Manage saved instructions in System › Auto-tagging.');
|
||||
await expect(text).toBeFocused();
|
||||
// Generate sends the edited text; the saved entries are unchanged.
|
||||
const sent = page.waitForRequest(item => item.url().endsWith('/image-annotations/suggest'));
|
||||
await generate(page).click();
|
||||
expect((await sent).postDataJSON().options.instructions).toBe('Only the sky.');
|
||||
// Choosing an entry replaces the edit; switching engines starts from the new engine's default.
|
||||
await pick.selectOption('builtin');
|
||||
await expect(text).toHaveValue(builtin);
|
||||
await expect(note).toBeHidden();
|
||||
await text.fill('Edited again.');
|
||||
await model(page).selectOption('hf');
|
||||
await expect(text).toHaveValue('Describe the image');
|
||||
await expect(pick).toHaveValue(FIXTURE_INSTRUCTIONS_ID);
|
||||
await model(page).selectOption('wd14-vit');
|
||||
await expect(pick).toHaveCount(0);
|
||||
await expect(text).toHaveCount(0);
|
||||
expect((await (await request.get('/prompt_manager/settings')).json()).settings.autotag_instructions)
|
||||
.toEqual(settings.settings.autotag_instructions);
|
||||
});
|
||||
|
||||
@@ -135,34 +135,11 @@ test('/prompt_manager/web lands on the workspace Library and Back does not bounc
|
||||
await expect(main.getByRole('heading', { level: 1, name: 'Overview', exact: true })).toBeVisible();
|
||||
});
|
||||
|
||||
// D165 (#525, maintainer ruling): a discreet line under System is the workspace's only visible way to the legacy pages.
|
||||
test('System Overview links the legacy pages discreetly, and each one loads (#525)', async ({ page, context }) => {
|
||||
// #620: the old pages are on their way out (notice since 3.5.1), so the workspace no longer links them.
|
||||
test('System no longer links the old pages (#620)', async ({ page }) => {
|
||||
await page.goto(`${workspace}#/system/overview`);
|
||||
const line = page.locator('#workspace-main .legacy-pages');
|
||||
await expect(line).toBeVisible();
|
||||
await expect(line).toContainText('Legacy pages');
|
||||
const links = line.getByRole('link');
|
||||
await expect(links).toHaveCount(4);
|
||||
for (const [name, url, ready] of [
|
||||
['Dashboard', '/prompt_manager/admin', '#addPromptBtn'],
|
||||
['Tags', '/prompt_manager/admin#/tags', '#addPromptBtn'],
|
||||
['Gallery', '/prompt_manager/gallery', 'body'],
|
||||
['Metadata viewer', '/prompt_manager/gallery.html', 'body'],
|
||||
]) {
|
||||
const opened = context.waitForEvent('page');
|
||||
await line.getByRole('link', { name, exact: true }).click();
|
||||
const tab = await opened;
|
||||
await tab.waitForLoadState('domcontentloaded');
|
||||
const target = new URL(tab.url());
|
||||
expect(target.pathname + target.hash).toBe(url);
|
||||
await expect(tab.locator(ready).first()).toBeAttached();
|
||||
await tab.close();
|
||||
}
|
||||
// Only on System's Overview section, and gone with the page.
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Appearance', exact: true }).click();
|
||||
await expect(line).toBeHidden();
|
||||
await page.goto(`${workspace}#/library`);
|
||||
await expect(page.locator('#workspace-main .legacy-pages')).toHaveCount(0);
|
||||
await expect(page.locator('#workspace-main h1')).toHaveText('System');
|
||||
await expect(page.locator('#workspace-main a[href*="/admin"], #workspace-main a[href*="/gallery"]')).toHaveCount(0);
|
||||
});
|
||||
|
||||
const bootFailure = page => page.locator('#workspace-boot-failure');
|
||||
@@ -175,7 +152,7 @@ test('a healthy workspace never shows the load failure message (#525)', async ({
|
||||
});
|
||||
|
||||
for (const [label, pattern] of [['the entry module', '**/js/workspace/index.mjs*'], ['a dependency', '**/js/workspace/state.mjs*']]) {
|
||||
test(`when ${label} fails to load the page still offers the legacy pages (#525)`, async ({ page }) => {
|
||||
test(`when ${label} fails to load the page offers a reload and ComfyUI (#525, #620)`, async ({ page }) => {
|
||||
const pageErrors = [];
|
||||
page.on('pageerror', error => pageErrors.push(error.message));
|
||||
await page.route(pattern, route => route.abort());
|
||||
@@ -183,19 +160,17 @@ for (const [label, pattern] of [['the entry module', '**/js/workspace/index.mjs*
|
||||
const failure = bootFailure(page);
|
||||
await expect(failure).toBeVisible();
|
||||
await expect(failure).toHaveAttribute('role', 'alert');
|
||||
const dashboard = failure.getByRole('link', { name: 'Dashboard', exact: true });
|
||||
await expect(dashboard).toBeVisible();
|
||||
await expect(failure.getByRole('link')).toHaveCount(4);
|
||||
await dashboard.click();
|
||||
await expect(page).toHaveURL(/\/prompt_manager\/admin$/);
|
||||
await expect(failure.getByRole('link', { name: 'Reload the page', exact: true })).toHaveAttribute('href', 'workspace');
|
||||
await expect(failure.getByRole('link', { name: 'ComfyUI', exact: true })).toHaveAttribute('href', '../');
|
||||
await expect(failure.getByRole('link')).toHaveCount(2);
|
||||
expect(pageErrors).toEqual([]);
|
||||
});
|
||||
}
|
||||
|
||||
test('when a module throws while starting the page still offers the legacy pages (#525)', async ({ page }) => {
|
||||
test('when a module throws while starting the page still offers a reload (#525, #620)', async ({ page }) => {
|
||||
await page.route('**/js/workspace/state.mjs*', route => route.fulfill({ contentType: 'text/javascript', body: "throw new Error('boot failure');" }));
|
||||
await page.goto(`${workspace}#/library`);
|
||||
await expect(bootFailure(page).getByRole('link', { name: 'Dashboard', exact: true })).toBeVisible();
|
||||
await expect(bootFailure(page).getByRole('link', { name: 'Reload the page', exact: true })).toBeVisible();
|
||||
// The broken module's own error is this test's premise, not a failure of the page.
|
||||
page.workspaceErrors.length = 0;
|
||||
});
|
||||
@@ -569,7 +544,7 @@ test('System is native: the concept shell, section navigation, sections and the
|
||||
// D92: no "Sample configuration" chip.
|
||||
await expect(main.locator('.heading .chip')).toHaveCount(0);
|
||||
expect(await main.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`)))
|
||||
.toEqual(['div.heading', 'div.section-nav', 'div.settings-layout', 'p.small legacy-pages']);
|
||||
.toEqual(['div.heading', 'div.section-nav', 'div.settings-layout']);
|
||||
const nav = main.getByRole('group', { name: 'System sections', exact: true });
|
||||
await expect(nav.getByRole('button')).toHaveText(SYSTEM_SECTION_LABELS);
|
||||
await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true');
|
||||
|
||||
@@ -41,6 +41,11 @@ read. These per-server hooks only change what a System test needs on top:
|
||||
- ``POST /__test__/system/duplicate-prompts`` (S7) adds two prompts whose
|
||||
text differs only in case and spacing (a duplicate prompt text group);
|
||||
``{"again": true}`` adds a third, so a reviewed plan no longer matches;
|
||||
- ``POST /__test__/system/pair-metadata`` (#550) saves two saved pairs the
|
||||
way the Pair node did before #546 (the pair holds the category and tags,
|
||||
its new positive prompt none): ``S550 lantern`` (category ``night``, tags
|
||||
``s550-glow``, ``s550-warm``) fills; ``S550 harbor`` was edited after it was
|
||||
saved, so the plan keeps it;
|
||||
- ``POST /__test__/system/duplicate-files`` (S7) writes one PNG's bytes three
|
||||
times: ``output/s7-dupes/same.png``, ``input/s7-dupes/same.png`` (the same
|
||||
relative path in the other source) and ``output/s7-dupes/copy.png``;
|
||||
@@ -344,6 +349,25 @@ def register_system_commands(app, routes, api, root: Path):
|
||||
conn.commit()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/pair-metadata")
|
||||
async def pair_metadata(request):
|
||||
def seed():
|
||||
api.db.save_prompt_pair(
|
||||
"S550 lantern",
|
||||
"S550 blurry",
|
||||
category="night",
|
||||
tags=["s550-glow", "s550-warm"],
|
||||
)
|
||||
api.db.save_prompt_pair("S550 harbor", category="night")
|
||||
with api.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompts SET updated_at='2099-01-01T00:00:00+00:00' "
|
||||
"WHERE text='S550 harbor'"
|
||||
)
|
||||
|
||||
await api._run_in_executor(seed)
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/duplicate-files")
|
||||
async def duplicate_files(request):
|
||||
body = await request.json() if request.can_read_body else {}
|
||||
|
||||
@@ -226,6 +226,7 @@ def register_tagging_commands(
|
||||
return download["result"]
|
||||
|
||||
_set_defaults(model, DEFAULTS) # the server starts on the standard ones
|
||||
_write_labels(model, True)
|
||||
|
||||
@routes.post("/__test__/tagging/reset")
|
||||
async def reset(request: web.Request) -> web.Response:
|
||||
@@ -244,6 +245,7 @@ def register_tagging_commands(
|
||||
model.loaded = None
|
||||
restore_download()
|
||||
_set_defaults(model, DEFAULTS)
|
||||
await api._run_in_executor(_write_labels, model, True)
|
||||
# A test that ends with the dialog open leaves its capture behind; drop
|
||||
# every capture so the next test starts below the store's limits.
|
||||
await api._run_in_executor(api.close_annotation_captures)
|
||||
@@ -353,13 +355,22 @@ def register_tagging_commands(
|
||||
|
||||
``{"general": g, "character": c, "limit": n}``: the WD14 thresholds
|
||||
(in [0, 1]) and the persisted suggestion count (1..100), so a test can
|
||||
tell the server's values from the dialog's built-in ones. Reset
|
||||
restores the standard 0.35 / 0.85 / 10.
|
||||
tell the server's values from the dialog's built-in ones. Optional
|
||||
``skipped`` sets the saved Skipped tags (#431) and ``labels`` (true or
|
||||
false) writes or removes the fake WD14 SwinV2 label list the Skipped
|
||||
tags preview reads. Reset restores the standard 0.35 / 0.85 / 10, no
|
||||
skipped tags and the label list.
|
||||
"""
|
||||
data = await request.json() if request.can_read_body else None
|
||||
if request.query or not _valid_defaults(data):
|
||||
raise web.HTTPBadRequest()
|
||||
_set_defaults(model, (data["general"], data["character"], data["limit"]))
|
||||
if "skipped" in data:
|
||||
from py.config import AutotagConfig
|
||||
|
||||
AutotagConfig.SKIPPED_TAGS = tuple(data["skipped"])
|
||||
if "labels" in data:
|
||||
await api._run_in_executor(_write_labels, model, data["labels"])
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.get("/__test__/tagging/captures")
|
||||
@@ -380,10 +391,15 @@ DOWNLOAD_HOLD_LIMIT_S = 60
|
||||
DEFAULTS = (0.35, 0.85, 10)
|
||||
|
||||
|
||||
# The saved "Describe the image" entry's id (#431).
|
||||
FIXTURE_INSTRUCTIONS_ID = "d35c71be0a1e"
|
||||
|
||||
|
||||
def _set_defaults(model, values) -> None:
|
||||
"""Set the saved tagging defaults (``AutotagConfig``, #328); a reset also
|
||||
puts back the saved model, batch policy and template (the fake model's
|
||||
short instructions, which the concept comparisons show)."""
|
||||
puts back the saved model, batch policy, skipped tags and instructions
|
||||
(the fake model's short instructions, which the concept comparisons
|
||||
show)."""
|
||||
from py.config import AutotagConfig
|
||||
|
||||
general, character, limit = values
|
||||
@@ -393,7 +409,48 @@ def _set_defaults(model, values) -> None:
|
||||
if values == DEFAULTS:
|
||||
AutotagConfig.MODEL_TYPE = "wd14-swinv2"
|
||||
AutotagConfig.EXISTING = "skip"
|
||||
AutotagConfig.TEMPLATE = model.default_prompt
|
||||
# #431: Built-in is always the shipped text, so the fake model's short
|
||||
# instructions are a saved entry, the default of both engines (the
|
||||
# concept comparisons show it).
|
||||
AutotagConfig.INSTRUCTIONS = (
|
||||
{
|
||||
"id": FIXTURE_INSTRUCTIONS_ID,
|
||||
"name": model.default_prompt,
|
||||
"text": model.default_prompt,
|
||||
},
|
||||
)
|
||||
AutotagConfig.DEFAULT_INSTRUCTIONS = {
|
||||
"gguf": FIXTURE_INSTRUCTIONS_ID,
|
||||
"hf": FIXTURE_INSTRUCTIONS_ID,
|
||||
}
|
||||
AutotagConfig._LEGACY_TEMPLATE = None
|
||||
AutotagConfig.SKIPPED_TAGS = ()
|
||||
|
||||
|
||||
# The fake WD14 SwinV2 label list (general 0, character 4, rating 9): the
|
||||
# names the Skipped tags preview checks beside the library's tags (#431).
|
||||
FAKE_LABELS = (
|
||||
("blurry", "0"),
|
||||
("blurry_background", "0"),
|
||||
("signature", "0"),
|
||||
("watermark", "0"),
|
||||
("t-shirt", "0"),
|
||||
("half-closed_eyes", "0"),
|
||||
("saber_(fate)", "4"),
|
||||
("general", "9"),
|
||||
)
|
||||
|
||||
|
||||
def _write_labels(model, present: bool) -> None:
|
||||
"""Write (or remove) the fake label list under the fake models folder."""
|
||||
path = Path(model.models_dir) / "WD14-SwinV2" / "selected_tags.csv"
|
||||
if not present:
|
||||
path.unlink(missing_ok=True)
|
||||
return
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
lines = ["tag_id,name,category,count"]
|
||||
lines += [f"{i},{name},{cat},1" for i, (name, cat) in enumerate(FAKE_LABELS)]
|
||||
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def _valid_defaults(data: object) -> bool:
|
||||
@@ -402,9 +459,20 @@ def _valid_defaults(data: object) -> bool:
|
||||
and not isinstance(value, bool)
|
||||
and 0 <= value <= 1
|
||||
)
|
||||
optional = {"skipped", "labels"}
|
||||
if isinstance(data, dict) and "skipped" in data:
|
||||
try:
|
||||
from py.tagging_defaults import validate_skipped_tags
|
||||
|
||||
validate_skipped_tags(data["skipped"])
|
||||
except ValueError:
|
||||
return False
|
||||
if isinstance(data, dict) and not isinstance(data.get("labels", True), bool):
|
||||
return False
|
||||
return (
|
||||
isinstance(data, dict)
|
||||
and set(data) == {"general", "character", "limit"}
|
||||
and {"general", "character", "limit"} <= set(data)
|
||||
and set(data) <= {"general", "character", "limit"} | optional
|
||||
and unit(data["general"])
|
||||
and unit(data["character"])
|
||||
and type(data["limit"]) is int
|
||||
|
||||
@@ -105,6 +105,7 @@
|
||||
"POST /prompt_manager/autotag/batch/preview",
|
||||
"POST /prompt_manager/autotag/download/{model_type}",
|
||||
"POST /prompt_manager/autotag/single",
|
||||
"POST /prompt_manager/autotag/skipped-tags/preview",
|
||||
"POST /prompt_manager/autotag/start",
|
||||
"POST /prompt_manager/autotag/unload",
|
||||
"POST /prompt_manager/bulk/category",
|
||||
@@ -187,10 +188,14 @@
|
||||
"monitor.scheduled",
|
||||
"revision",
|
||||
"settings",
|
||||
"settings.autotag_builtin_instructions",
|
||||
"settings.autotag_character_threshold",
|
||||
"settings.autotag_default_instructions",
|
||||
"settings.autotag_existing",
|
||||
"settings.autotag_general_threshold",
|
||||
"settings.autotag_instructions",
|
||||
"settings.autotag_model_type",
|
||||
"settings.autotag_skipped_tags",
|
||||
"settings.autotag_suggestion_limit",
|
||||
"settings.autotag_template",
|
||||
"settings.cpu_count",
|
||||
|
||||
@@ -255,7 +255,8 @@ test('All is offered only with two or more eligible nodes and is never selected
|
||||
test('Apply to All previews every node first, then sends one after another in node id order',async()=>{
|
||||
const {createSendSession,ALL_NODES}=await source;const s=await allSession(createSendSession);
|
||||
s.session.selectTarget(ALL_NODES);await s.session.apply();
|
||||
assert.deepEqual(order(s.calls),['preview 2','preview 9','preview 10','apply 2','preview 9','apply 9','preview 10','apply 10']);
|
||||
// Each node takes a fresh ticket, the first too: the host keeps only its latest 100 (#593).
|
||||
assert.deepEqual(order(s.calls),['preview 2','preview 9','preview 10','preview 2','apply 2','preview 9','apply 9','preview 10','apply 10']);
|
||||
assert.equal(s.session.state.phase,'success');assert.equal(s.session.state.message,'Sent to 3 of 3 nodes in Flow - ComfyUI.');
|
||||
await s.session.apply();assert.equal(s.calls.filter(call=>call[0]==='apply').length,3);s.session.dispose();
|
||||
});
|
||||
@@ -363,3 +364,48 @@ test('a changed prepend or append after the review refuses that node',async()=>{
|
||||
assert.deepEqual(s.calls.filter(call=>call[0]==='apply').map(call=>call[1]),['2','10']);
|
||||
assert.match(s.session.state.message,/^Sent to 2 of 3 nodes in Flow - ComfyUI\. #9 Node 9: Its prompt changed since the preview\./);s.session.dispose();
|
||||
});
|
||||
|
||||
// #593: Send to All through the real Toolkit bridge and the real adapter, as the review bot reproduced it.
|
||||
function realWindow(count) {
|
||||
const listeners=new Set();
|
||||
const channelFactory=()=>{const own=new Set();return{
|
||||
addEventListener(type,listener){if(type==='message'){own.add(listener);listeners.add(listener);}},removeEventListener(type,listener){listeners.delete(listener);},
|
||||
postMessage(data){for(const listener of [...listeners])if(!own.has(listener))queueMicrotask(()=>listener({data:structuredClone(data),origin:'https://comfy.example'}));},close(){},
|
||||
};};
|
||||
// Like LiteGraph, a widget write moves the version of the graph it is in.
|
||||
const graph=()=>{const made={_version:1,setDirtyCanvas(){made._version++;}};made._nodes=Array.from({length:count},(_,index)=>{
|
||||
const widgets=['text','prepend_text','append_text'].map(name=>({name,value:name==='text'?'old':'',inputEl:{value:name==='text'?'old':''}}));
|
||||
return {id:index+1,type:'PromptManager',title:`Node ${index+1}`,widgets,widgets_values:widgets.map(widget=>widget.value),inputs:[],setDirtyCanvas(){}};
|
||||
});return made;};
|
||||
return {channelFactory,graph};
|
||||
}
|
||||
async function realSend(count,{afterFirstApply}={}) {
|
||||
const {createSendSession,ALL_NODES}=await source;
|
||||
const {createBridgeHost,createBridgeClient}=await import('../../web/js/shared/toolkit/prompt-bridge.js');
|
||||
const {createPromptAdapter}=await import('../../web/js/prompt-bridge-adapter.js');
|
||||
const win=realWindow(count);const app={graph:win.graph(),canvas:{}};
|
||||
const adapter=createPromptAdapter(app,{runtime:{}});let applied=0;
|
||||
const watched={...adapter,apply(snapshot){const result=adapter.apply(snapshot);if(++applied===1)afterFirstApply?.(app,win);return result;}};
|
||||
const options={namespace:'/prompt_manager/',origin:'https://comfy.example',channelFactory:win.channelFactory};
|
||||
const host=createBridgeHost({...options,label:'Flow - ComfyUI',adapter:watched});
|
||||
const client=createBridgeClient(options);
|
||||
const session=createSendSession({client,roles:{positive:'new'},onChange:()=>{}});
|
||||
try {
|
||||
await session.discover();session.selectTarget(ALL_NODES);await session.apply();
|
||||
return {state:session.state,app};
|
||||
} finally { session.dispose();host.dispose(); }
|
||||
}
|
||||
test('All refuses the later nodes when another workflow is loaded after the first send (#593)',async()=>{
|
||||
let first;
|
||||
const {state,app}=await realSend(3,{afterFirstApply:(target,win)=>{first=target.graph;target.graph=win.graph();target.graph._version=first._version;}});
|
||||
// The new workflow has the same ids, text and composition, and even the same graph version: only node identity differs.
|
||||
assert.equal(first._nodes[0].widgets[0].value,'new');
|
||||
assert.deepEqual(app.graph._nodes.map(node=>node.widgets[0].value),['old','old','old']);
|
||||
assert.equal(state.phase,'error');
|
||||
assert.match(state.message,/^Sent to 1 of 3 nodes in Flow - ComfyUI\. #2 Node 2: The workflow changed since this window was found\./);
|
||||
});
|
||||
test('All sends to every node of a window with more than 100 of them (#593)',async()=>{
|
||||
const {state,app}=await realSend(101);
|
||||
assert.equal(state.message,'Sent to 101 of 101 nodes in Flow - ComfyUI.');
|
||||
assert.ok(app.graph._nodes.every(node=>node.widgets[0].value==='new'));
|
||||
});
|
||||
|
||||
@@ -53,8 +53,12 @@ function observed(overrides = {}, capture = {}) {
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
// #431: the saved instructions; the fixture's entry is the default of both engines.
|
||||
const SAVED_ENTRY = { id: '0123456789ab', name: 'Saved', text: DEFAULT_PROMPT };
|
||||
const models = (overrides = {}) => ({
|
||||
success: true, models: { 'wd14-swinv2': { downloaded: true } }, default_prompt: DEFAULT_PROMPT,
|
||||
success: true, models: { 'wd14-swinv2': { downloaded: true } },
|
||||
instructions: [{ id: 'builtin', name: 'Built-in', text: 'Built-in text.' }, SAVED_ENTRY],
|
||||
default_instructions: { gguf: SAVED_ENTRY.id, hf: SAVED_ENTRY.id },
|
||||
wd14_general_threshold: 0.35, wd14_character_threshold: 0.85, suggestion_limit: 10, ...overrides,
|
||||
});
|
||||
const SANDSTONE = [
|
||||
@@ -244,10 +248,13 @@ test('dialog starts on the saved model unless a preset or the user chose one', a
|
||||
});
|
||||
|
||||
test('an unreadable or malformed models read falls back to the documented defaults', async () => {
|
||||
for (const route of [() => fail(500, null), () => response(200, models({ default_prompt: 4, suggestion_limit: 0, wd14_general_threshold: 2, wd14_character_threshold: 'x' }))]) {
|
||||
for (const route of [() => fail(500, null), () => response(200, models({ instructions: 4, suggestion_limit: 0, wd14_general_threshold: 2, wd14_character_threshold: 'x' }))]) {
|
||||
const { c } = await opened({ [`GET ${MODELS}`]: route });
|
||||
assert.equal(c.state.phase, 'ready');
|
||||
assert.deepEqual({ ...c.state.options }, { general_threshold: 0.35, character_threshold: 0.85, instructions: '', suggestion_limit: 10 });
|
||||
// #431: the client's Built-in copy (D204's read failure).
|
||||
const { BUILTIN_INSTRUCTIONS } = await load();
|
||||
assert.deepEqual({ ...c.state.options }, { general_threshold: 0.35, character_threshold: 0.85, instructions: BUILTIN_INSTRUCTIONS,
|
||||
suggestion_limit: 10 });
|
||||
}
|
||||
});
|
||||
|
||||
@@ -936,3 +943,99 @@ test('#515: text typed before Retry after a failed first observation follows the
|
||||
assert.equal(c.state.manual, 'Minimal, warm_light, dunes');
|
||||
assert.equal(c.state.statusText, '');
|
||||
});
|
||||
|
||||
test('the chip hint says how many skipped tags were not suggested (#431)', async () => {
|
||||
const { skippedNote } = await load();
|
||||
assert.equal(skippedNote(0), '');
|
||||
assert.equal(skippedNote(undefined), '');
|
||||
assert.equal(skippedNote(1), ' 1 skipped tag was not suggested.');
|
||||
assert.equal(skippedNote(2), ' 2 skipped tags were not suggested.');
|
||||
for (const [skipped, hint] of [[2, 'Remove anything you do not want to add. 2 skipped tags were not suggested.'],
|
||||
[0, 'Remove anything you do not want to add.'], ['3', 'Remove anything you do not want to add.']]) {
|
||||
const { c } = await opened({ [`POST ${SUGGEST}`]: () => response(200, { ...suggested(), skipped }) });
|
||||
await c.generate();
|
||||
assert.equal(c.state.chipHint, hint, String(skipped));
|
||||
}
|
||||
});
|
||||
|
||||
// D204 (#431): the Saved instructions picker of both dialogs.
|
||||
const PHOTO_ENTRY = { id: '0123456789ac', name: 'Photo', text: 'Photo tags.' };
|
||||
const picker = (overrides = {}) => ({ [`GET ${MODELS}`]: () => response(200, models({ model_type: 'gguf',
|
||||
instructions: [{ id: 'builtin', name: 'Built-in', text: 'Built-in text.' }, SAVED_ENTRY, PHOTO_ENTRY],
|
||||
default_instructions: { gguf: SAVED_ENTRY.id, hf: PHOTO_ENTRY.id }, ...overrides })) });
|
||||
|
||||
test('the picker starts at the engine default; a pick replaces the text and an edit is one-off (D204)', async () => {
|
||||
const { c } = await opened(picker());
|
||||
assert.deepEqual([c.state.instructionId, c.state.options.instructions, c.state.instructionsEdited], [SAVED_ENTRY.id, DEFAULT_PROMPT, false]);
|
||||
const revision = c.state.optionsRevision;
|
||||
c.pickInstructions(PHOTO_ENTRY.id);
|
||||
assert.deepEqual([c.state.instructionId, c.state.options.instructions, c.state.instructionsEdited], [PHOTO_ENTRY.id, 'Photo tags.', false]);
|
||||
assert.equal(c.state.optionsRevision, revision + 1, 'a pick re-renders the textarea');
|
||||
c.setOption('instructions', 'Photo tags, please.');
|
||||
assert.equal(c.state.instructionsEdited, true);
|
||||
assert.equal(c.state.optionsRevision, revision + 1, 'typing never re-renders');
|
||||
// Choosing an entry again replaces the edit without asking; an unknown id changes nothing.
|
||||
c.pickInstructions(PHOTO_ENTRY.id);
|
||||
assert.deepEqual([c.state.options.instructions, c.state.instructionsEdited], ['Photo tags.', false]);
|
||||
c.pickInstructions('nope');
|
||||
assert.equal(c.state.instructionId, PHOTO_ENTRY.id);
|
||||
// Switching engines resets to the new engine's default and drops a one-off edit; WD14 is never "edited".
|
||||
c.setOption('instructions', 'edited');
|
||||
c.setModel('hf');
|
||||
assert.deepEqual([c.state.instructionId, c.state.options.instructions, c.state.instructionsEdited], [PHOTO_ENTRY.id, 'Photo tags.', false]);
|
||||
c.setModel('wd14-vit');
|
||||
assert.equal(c.state.instructionsEdited, false);
|
||||
c.setModel('gguf');
|
||||
assert.deepEqual([c.state.instructionId, c.state.options.instructions], [SAVED_ENTRY.id, DEFAULT_PROMPT]);
|
||||
c.dispose();
|
||||
});
|
||||
|
||||
test('a preset shows the entry with its text, else "Edited for this run" (D204)', async () => {
|
||||
const { c } = await opened(picker(), { preset: { model: 'hf', options: { instructions: DEFAULT_PROMPT } } });
|
||||
assert.deepEqual([c.state.instructionId, c.state.instructionsEdited], [SAVED_ENTRY.id, false]);
|
||||
const { c: other } = await opened(picker(), { preset: { model: 'hf', options: { instructions: 'Something else' } } });
|
||||
assert.deepEqual([other.state.instructionId, other.state.options.instructions, other.state.instructionsEdited],
|
||||
[PHOTO_ENTRY.id, 'Something else', true]);
|
||||
for (const controller of [c, other]) controller.dispose();
|
||||
});
|
||||
|
||||
test('a models read without instructions offers Built-in only with the client copy (D204)', async () => {
|
||||
const { BUILTIN_INSTRUCTIONS, readChoices } = await load();
|
||||
assert.equal(BUILTIN_INSTRUCTIONS.startsWith('Generate only comma-separated Danbooru tags'), true);
|
||||
for (const value of [null, {}, { instructions: [] }, { instructions: [{ id: 'x', name: 'X', text: 'y' }] }]) {
|
||||
assert.deepEqual(readChoices(value).entries.map(entry => entry.id), ['builtin'], JSON.stringify(value));
|
||||
}
|
||||
// A default naming no entry is Built-in.
|
||||
assert.deepEqual({ ...readChoices({ instructions: [{ id: 'builtin', name: 'Built-in', text: 't' }], default_instructions: { gguf: 'gone' } }).defaults },
|
||||
{ gguf: 'builtin', hf: 'builtin' });
|
||||
const { c } = await opened(picker({ instructions: 'broken' }));
|
||||
assert.deepEqual([c.state.instructionId, c.state.options.instructions], ['builtin', BUILTIN_INSTRUCTIONS]);
|
||||
c.dispose();
|
||||
});
|
||||
|
||||
test('the bulk options pick and reset like the Tag image dialog (D204)', async () => {
|
||||
const { createTaggingOptionsController } = await load();
|
||||
const options = createTaggingOptionsController({ api: harness(defaultRoutes(picker())).api });
|
||||
await options.open();
|
||||
assert.deepEqual([options.state.model, options.state.instructionId, options.state.options.instructions], ['gguf', SAVED_ENTRY.id, DEFAULT_PROMPT]);
|
||||
options.pickInstructions(PHOTO_ENTRY.id);
|
||||
assert.equal(options.request().options.instructions, 'Photo tags.');
|
||||
options.setOption('instructions', 'One-off.');
|
||||
assert.equal(options.state.instructionsEdited, true);
|
||||
assert.equal(options.request().options.instructions, 'One-off.');
|
||||
options.setModel('hf');
|
||||
assert.deepEqual([options.state.instructionId, options.state.instructionsEdited], [PHOTO_ENTRY.id, false]);
|
||||
options.pickInstructions('nope');
|
||||
options.dispose();
|
||||
options.pickInstructions(SAVED_ENTRY.id);
|
||||
assert.equal(options.state.instructionId, PHOTO_ENTRY.id);
|
||||
});
|
||||
|
||||
test('the client Built-in copy equals the server DEFAULT_TAG_TEMPLATE (#431)', async () => {
|
||||
const { BUILTIN_INSTRUCTIONS } = await load();
|
||||
const source = require('node:fs').readFileSync(require('node:path').join(__dirname, '../../py/tagging_defaults.py'), 'utf8');
|
||||
const block = source.slice(source.indexOf('DEFAULT_TAG_TEMPLATE = ('), source.indexOf('\n)\n', source.indexOf('DEFAULT_TAG_TEMPLATE = (')));
|
||||
const text = [...block.matchAll(/"((?:[^"\\]|\\.)*)"/g)].map(([, part]) => JSON.parse(`"${part}"`)).join('');
|
||||
assert.ok(text.length > 100);
|
||||
assert.equal(BUILTIN_INSTRUCTIONS, text);
|
||||
});
|
||||
|
||||
@@ -180,7 +180,9 @@ test('the bulk options start from the server defaults and wait for them before S
|
||||
assert.equal(options.state.canStart, false);
|
||||
assert.equal(options.state.canGenerate, false);
|
||||
const opening = options.open();
|
||||
answer({ wd14_general_threshold: 0.5, wd14_character_threshold: 0.9, default_prompt: 'Say it', suggestion_limit: 7,
|
||||
answer({ wd14_general_threshold: 0.5, wd14_character_threshold: 0.9, suggestion_limit: 7,
|
||||
instructions: [{ id: 'builtin', name: 'Built-in', text: 'Built-in text.' }, { id: '0123456789ab', name: 'Say', text: 'Say it' }],
|
||||
default_instructions: { gguf: '0123456789ab', hf: 'builtin' },
|
||||
models: { 'wd14-swinv2': { downloaded: true }, 'wd14-vit': { downloaded: false }, gguf: { downloaded: true } } });
|
||||
await opening;
|
||||
assert.equal(options.state.canStart, true);
|
||||
|
||||
@@ -143,7 +143,7 @@ test('the shelf no longer sorts by rating (D153)',async()=>{
|
||||
const {createLibraryStream}=await source;const s=createLibraryStream({api:{get:async()=>page([])}});
|
||||
await assert.rejects(s.reset({sort:'rating'}),/Invalid library sort/);
|
||||
assert.throws(()=>createLibraryStream({api:{get:async()=>page([])},restore:{filters:{sort:'rating'},state:{}}}),/Invalid library sort/);
|
||||
for(const sort of ['newest','oldest','name']) assert.equal(await s.reset({sort}),0);s.dispose();
|
||||
for(const sort of ['newest','oldest','name','name-desc','most-used','last-used']) assert.equal(await s.reset({sort}),0);s.dispose();
|
||||
});
|
||||
test('Refine › Folder nests each folder under its exact parent, by path segment (#444 live B1)',async()=>{
|
||||
const {libraryFolderOptions}=await source;assert.equal(typeof libraryFolderOptions,'function');
|
||||
|
||||
@@ -32,7 +32,7 @@ test('a saved state always holds every page key and keeps only values its page a
|
||||
{ ...full, q: '', folder: '', tags: [] });
|
||||
assert.deepEqual(normalizeViewState('images', { ...full, category: 'bad\u0007', tags: [' '] }), { ...full, category: '', tags: [] });
|
||||
// Library has no source or media, and no Name Z–A; its folder (#444) is the Library's own.
|
||||
assert.deepEqual(normalizeViewState('library', { ...full, sort: 'name-desc' }), { q: 'castle', category: 'Portraits', sort: 'newest', tags: ['moody', 'fog'], tag_mode: 'all', folder: 'a/b' });
|
||||
assert.deepEqual(normalizeViewState('library', { ...full, sort: 'rating' }), { q: 'castle', category: 'Portraits', sort: 'newest', tags: ['moody', 'fog'], tag_mode: 'all', folder: 'a/b' });
|
||||
assert.deepEqual(normalizeViewState('library', { sort: 'oldest' }), { ...LIBRARY_DEFAULTS, sort: 'oldest' });
|
||||
// The returned tags are a copy.
|
||||
const tags = ['one'];
|
||||
@@ -244,5 +244,7 @@ test('a view saved under the retired Highest rated sort opens on Recently added
|
||||
assert.equal(imagesFilters({ sort: 'rating' }).sort, 'newest');
|
||||
assert.equal(libraryFilters({ sort: 'rating' }).sort, 'newest');
|
||||
assert.equal(normalizeViewState('images', { sort: 'name-desc' }).sort, 'name-desc');
|
||||
assert.equal(normalizeViewState('library', { sort: 'oldest' }).sort, 'oldest');
|
||||
for (const sort of ['oldest', 'name-desc', 'most-used', 'last-used']) {
|
||||
assert.equal(normalizeViewState('library', { sort }).sort, sort);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -93,13 +93,6 @@ test('every Overview card opens its own native page, never a legacy one (#525)',
|
||||
}
|
||||
});
|
||||
|
||||
test('the legacy pages System links are the four that stay at their URLs (#525 D165)', async () => {
|
||||
const { LEGACY_PAGES } = await source;
|
||||
assert.deepEqual(LEGACY_PAGES.map(page => page.href), ['/prompt_manager/admin', '/prompt_manager/admin#/tags',
|
||||
'/prompt_manager/gallery', '/prompt_manager/gallery.html']);
|
||||
for (const page of LEGACY_PAGES) assert.ok(page.label);
|
||||
});
|
||||
|
||||
test('a job of the Activity page is its own deep link', async () => {
|
||||
const { resolveRoute, routeTitle, isHandoff } = await source;
|
||||
const id = '0123456789abcdef'.repeat(2);
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
// System › Auto-tagging "Saved instructions" (#431, D203): Built-in read-only, New, Duplicate, Delete (a deleted default
|
||||
// resets to Built-in), the caps, the editor drafts and the per-engine default selects.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const instructionsModule = import('../../web/js/workspace/system-instructions.mjs');
|
||||
const stateModule = import('../../web/js/workspace/system-state.mjs');
|
||||
const coreModule = import('../../web/js/shared/toolkit/core.js');
|
||||
|
||||
/** A minimal DOM: elements with children, attributes, listeners (aborted by their signal) and focus. */
|
||||
function fakeDOM() {
|
||||
const previous = global.document;
|
||||
class Element {
|
||||
constructor(tag) {
|
||||
this.tagName = tag.toUpperCase();
|
||||
this.children = [];
|
||||
this.attributes = {};
|
||||
this.listeners = [];
|
||||
this.dataset = {};
|
||||
this.className = '';
|
||||
this.id = '';
|
||||
this.value = '';
|
||||
this.title = '';
|
||||
this.disabled = false;
|
||||
this._text = '';
|
||||
}
|
||||
get localName() { return this.tagName.toLowerCase(); }
|
||||
get textContent() { return this._text + this.children.map(child => (typeof child === 'string' ? child : child.textContent)).join(''); }
|
||||
set textContent(text) { this.children = []; this._text = String(text); }
|
||||
append(...nodes) { for (const node of nodes) { if (typeof node !== 'string') node.parentNode = this; this.children.push(node); } }
|
||||
replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); }
|
||||
querySelectorAll(selector) {
|
||||
assert.equal(selector, 'button');
|
||||
const found = [];
|
||||
const walk = node => { for (const child of node.children) if (typeof child !== 'string') { if (child.tagName === 'BUTTON') found.push(child); walk(child); } };
|
||||
walk(this);
|
||||
return found;
|
||||
}
|
||||
focus() { global.document.activeElement = this; }
|
||||
setAttribute(name, value) { this.attributes[name] = String(value); }
|
||||
getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; }
|
||||
removeAttribute(name) { delete this.attributes[name]; }
|
||||
addEventListener(type, callback, options = {}) {
|
||||
if (options.signal?.aborted) return;
|
||||
const entry = { type, callback };
|
||||
this.listeners.push(entry);
|
||||
options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); });
|
||||
}
|
||||
dispatch(type, event = {}) {
|
||||
return Promise.all(this.listeners.filter(item => item.type === type).map(({ callback }) => callback({ type, target: this, ...event })));
|
||||
}
|
||||
}
|
||||
global.document = { createElement: tag => new Element(tag), activeElement: null };
|
||||
const walk = (node, found = []) => { found.push(node); node.children.forEach(child => typeof child !== 'string' && walk(child, found)); return found; };
|
||||
return { find: (root, predicate) => walk(root).find(predicate), restore() { global.document = previous; } };
|
||||
}
|
||||
|
||||
|
||||
const PHOTO = { id: '0123456789ab', name: 'Custom template', text: 'Photo tags.' };
|
||||
const ANIME = { id: '0123456789ac', name: 'Anime', text: 'Anime tags.' };
|
||||
const HOST = { autotag_builtin_instructions: 'Built-in text.', autotag_instructions: [PHOTO, ANIME],
|
||||
autotag_default_instructions: { gguf: PHOTO.id, hf: PHOTO.id } };
|
||||
|
||||
async function mounted({ host = HOST } = {}) {
|
||||
const { savedInstructionsBlock } = await instructionsModule;
|
||||
const { createSystemDrafts } = await stateModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const drafts = createSystemDrafts({
|
||||
readBrowser: () => ({}), readRuntime: () => ({}), readHost: async () => ({ values: host, revision: 'sha256:one' }),
|
||||
writeBrowser: async () => ({}), writeHost: async () => ({ success: true, persisted: true }), writeRuntime: async () => ({}),
|
||||
});
|
||||
const scope = createScope();
|
||||
let counter = 0;
|
||||
const random = bytes => bytes.map(() => (counter += 1));
|
||||
const block = savedInstructionsBlock({ scope, drafts, random });
|
||||
const root = global.document.createElement('div');
|
||||
root.append(...block.nodes);
|
||||
scope.cleanup(drafts.subscribe(block.repaint));
|
||||
block.repaint();
|
||||
await drafts.reloadHost();
|
||||
const byId = id => dom.find(root, node => node.id === id);
|
||||
const options = () => byId('instruction-list').children.map(option => [option.children[0].textContent,
|
||||
option.children[1]?.children.map(badge => badge.textContent) ?? [], option.getAttribute('aria-selected')]);
|
||||
const click = id => byId(id).dispatch('click');
|
||||
// A click on an option reaches the list's one listener (as it bubbles in a browser).
|
||||
const pick = name => byId('instruction-list').dispatch('click', { target: byId('instruction-list').children
|
||||
.find(option => option.children[0].textContent === name).children[0] });
|
||||
return { dom, drafts, scope, block, byId, options, click, pick, root };
|
||||
}
|
||||
|
||||
test('helpers: Built-in first, defaults, free names, ids and the server sentences', async () => {
|
||||
const { allEntries, defaultEntry, freeName, newId, listProblem, entryNote } = await instructionsModule;
|
||||
assert.deepEqual(allEntries(HOST).map(entry => entry.name), ['Built-in', 'Custom template', 'Anime']);
|
||||
assert.equal(allEntries(HOST)[0].text, 'Built-in text.');
|
||||
assert.equal(defaultEntry(HOST, 'hf').text, 'Photo tags.');
|
||||
assert.equal(defaultEntry({ ...HOST, autotag_default_instructions: { gguf: 'gone', hf: 'builtin' } }, 'gguf').name, 'Built-in');
|
||||
assert.equal(freeName('New instructions', allEntries(HOST)), 'New instructions');
|
||||
assert.equal(freeName('anime', allEntries(HOST)), 'anime (2)');
|
||||
assert.equal(freeName('Anime', [...allEntries(HOST), { name: 'Anime (2)' }]), 'Anime (3)');
|
||||
let calls = 0;
|
||||
const id = newId([{ id: '000000000000' }], bytes => { calls += 1; return bytes.fill(calls === 1 ? 0 : 171); });
|
||||
assert.equal(id, 'abababababab');
|
||||
assert.match(newId([]), /^[0-9a-f]{12}$/);
|
||||
const list = entries => listProblem({ autotag_instructions: entries });
|
||||
assert.equal(list([PHOTO]), null);
|
||||
assert.equal(list([{ ...PHOTO, name: ' ' }]), 'Saved instructions: entry 1 needs a name.');
|
||||
assert.equal(list([PHOTO, { ...ANIME, name: 'x'.repeat(61) }]), 'Saved instructions: entry 2 name must be at most 60 characters.');
|
||||
assert.equal(list([{ ...PHOTO, name: 'a\nb' }]), 'Saved instructions: entry 1 name must not contain a line break.');
|
||||
assert.equal(list([{ ...PHOTO, name: 'built-in' }]), 'Saved instructions: the name “Built-in” is reserved.');
|
||||
assert.equal(list([PHOTO, { ...ANIME, name: ' custom TEMPLATE ' }]), 'Saved instructions: entry 2 name “custom TEMPLATE” is already used.');
|
||||
assert.equal(list([{ ...PHOTO, text: ' ' }]), 'Saved instructions: entry 1 text must be non-empty text.');
|
||||
assert.equal(list([{ ...PHOTO, text: 'x'.repeat(4001) }]), 'Saved instructions: entry 1 text must be at most 4000 characters.');
|
||||
const long = { ...PHOTO, text: 'x'.repeat(4001) };
|
||||
assert.equal(entryNote(long, [long]), 'The text must be at most 4,000 characters.');
|
||||
});
|
||||
|
||||
test('the list shows Built-in read-only first, the saved entries and their default chips', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
assert.deepEqual(view.options(), [['Built-in', ['Read-only'], 'true'],
|
||||
['Custom template', ['Default for GGUF', 'Default for HuggingFace'], 'false'], ['Anime', [], 'false']]);
|
||||
assert.equal(view.byId('instructions-count').textContent, '2 of 20 saved · Built‑in is not counted.');
|
||||
assert.equal(view.byId('instruction-name').value, 'Built-in');
|
||||
assert.equal(view.byId('instruction-name').readOnly, true);
|
||||
assert.equal(view.byId('instruction-text').value, 'Built-in text.');
|
||||
assert.equal(view.byId('instruction-text').readOnly, true);
|
||||
assert.equal(view.byId('instruction-note').textContent, 'Built-in instructions can’t be changed. Duplicate them to make your own.');
|
||||
assert.equal(view.byId('instruction-delete').disabled, true);
|
||||
assert.equal(view.byId('instruction-list').getAttribute('role'), 'listbox');
|
||||
assert.equal(view.block.nodes[0].dataset.key, 'autotag_instructions');
|
||||
assert.deepEqual(view.byId('instruction-default-gguf').children.map(option => option.textContent), ['Built-in', 'Custom template', 'Anime']);
|
||||
assert.equal(view.byId('instruction-default-gguf').value, PHOTO.id);
|
||||
assert.equal(view.byId('instruction-default-hf').dataset.key, 'autotag_default_instructions');
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('editing a saved entry drafts the list; its counts and note follow', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
await view.pick('Anime');
|
||||
assert.equal(view.byId('instruction-name').readOnly, false);
|
||||
assert.equal(view.byId('instruction-delete').disabled, false);
|
||||
view.byId('instruction-name').value = 'Anime tags';
|
||||
await view.byId('instruction-name').dispatch('input');
|
||||
view.byId('instruction-text').value = 'Tags, anime.';
|
||||
await view.byId('instruction-text').dispatch('input');
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_instructions'), [PHOTO, { ...ANIME, name: 'Anime tags', text: 'Tags, anime.' }]);
|
||||
assert.equal(view.byId('instruction-name').parentNode.children[0].children[1].textContent, '10 of 60');
|
||||
assert.equal(view.byId('instruction-text').parentNode.children[0].children[1].textContent, '12 of 4,000 characters');
|
||||
view.byId('instruction-name').value = 'custom template';
|
||||
await view.byId('instruction-name').dispatch('input');
|
||||
assert.equal(view.byId('instruction-note').textContent, 'The name “custom template” is already used.');
|
||||
// Built-in is never edited.
|
||||
await view.pick('Built-in');
|
||||
view.byId('instruction-text').value = 'changed';
|
||||
await view.byId('instruction-text').dispatch('input');
|
||||
assert.equal(view.drafts.get('host', 'autotag_instructions')[0], PHOTO);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('New and Duplicate append an entry with a fresh id and a free name; the caps disable them', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
await view.click('instruction-new');
|
||||
const added = view.drafts.get('host', 'autotag_instructions').at(-1);
|
||||
assert.match(added.id, /^[0-9a-f]{12}$/);
|
||||
assert.deepEqual([added.name, added.text], ['New instructions', '']);
|
||||
assert.equal(view.options().at(-1)[2], 'true');
|
||||
assert.equal(view.byId('instruction-note').textContent, 'The text must not be empty.');
|
||||
await view.pick('Built-in');
|
||||
await view.click('instruction-duplicate');
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_instructions').at(-1).name, 'Built-in copy');
|
||||
assert.equal(view.drafts.get('host', 'autotag_instructions').at(-1).text, 'Built-in text.');
|
||||
const full = await mounted({ host: { ...HOST, autotag_instructions: Array.from({ length: 20 },
|
||||
(_, i) => ({ id: i.toString(16).padStart(12, '0'), name: `n${i}`, text: 't' })) } });
|
||||
try {
|
||||
assert.equal(full.byId('instruction-new').disabled, true);
|
||||
assert.equal(full.byId('instruction-duplicate').disabled, true);
|
||||
assert.equal(full.byId('instruction-cap').hidden, false);
|
||||
await full.click('instruction-new');
|
||||
assert.equal(full.drafts.get('host', 'autotag_instructions').length, 20);
|
||||
} finally { full.scope.dispose(); }
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('deleting a default resets that engine to Built-in and says so; a select change clears the note', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
await view.pick('Custom template');
|
||||
await view.click('instruction-delete');
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_instructions'), [ANIME]);
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_default_instructions'), { gguf: 'builtin', hf: 'builtin' });
|
||||
assert.equal(view.byId('instruction-reset').textContent, 'GGUF and HuggingFace now use Built-in.');
|
||||
assert.equal(view.options()[1][2], 'true', 'the next entry is selected');
|
||||
view.byId('instruction-default-hf').value = ANIME.id;
|
||||
await view.byId('instruction-default-hf').dispatch('change');
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_default_instructions'), { gguf: 'builtin', hf: ANIME.id });
|
||||
assert.equal(view.byId('instruction-reset').textContent, '');
|
||||
await view.click('instruction-delete');
|
||||
assert.equal(view.byId('instruction-reset').textContent, 'HuggingFace now uses Built-in.');
|
||||
assert.deepEqual(view.options().map(([name]) => name), ['Built-in']);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('arrow keys move the selection; before the read everything is disabled', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
const list = view.byId('instruction-list');
|
||||
await list.dispatch('keydown', { key: 'ArrowDown' });
|
||||
assert.equal(view.byId('instruction-name').value, 'Custom template');
|
||||
await list.dispatch('keydown', { key: 'End' });
|
||||
assert.equal(view.byId('instruction-name').value, 'Anime');
|
||||
await list.dispatch('keydown', { key: 'ArrowDown' });
|
||||
assert.equal(view.byId('instruction-name').value, 'Anime');
|
||||
await list.dispatch('keydown', { key: 'Home' });
|
||||
assert.equal(view.byId('instruction-name').value, 'Built-in');
|
||||
await list.dispatch('keydown', { key: 'a' });
|
||||
assert.equal(list.getAttribute('aria-activedescendant'), 'instruction-option-builtin');
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
const unread = await mounted({ host: {} });
|
||||
try {
|
||||
assert.equal(unread.byId('instruction-new').disabled, true);
|
||||
assert.equal(unread.byId('instruction-default-gguf').disabled, true);
|
||||
} finally { unread.scope.dispose(); unread.dom.restore(); }
|
||||
});
|
||||
|
||||
test('the reset note goes once the defaults are saved; a long name is cut to fit its copy suffix', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
await view.pick('Custom template');
|
||||
await view.click('instruction-delete');
|
||||
assert.equal(view.byId('instruction-reset').textContent, 'GGUF and HuggingFace now use Built-in.');
|
||||
await view.drafts.save();
|
||||
assert.equal(view.byId('instruction-reset').textContent, '');
|
||||
await view.pick('Anime');
|
||||
view.byId('instruction-name').value = 'x'.repeat(60);
|
||||
await view.byId('instruction-name').dispatch('input');
|
||||
await view.click('instruction-duplicate');
|
||||
const copy = view.drafts.get('host', 'autotag_instructions').at(-1).name;
|
||||
assert.equal(copy, `${'x'.repeat(55)} copy`);
|
||||
await view.click('instruction-duplicate');
|
||||
assert.equal([...view.drafts.get('host', 'autotag_instructions').at(-1).name].length <= 60, true);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('option clicks are one listener on the list, not one per repaint', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
const list = view.byId('instruction-list');
|
||||
const before = list.listeners.length;
|
||||
for (let i = 0; i < 5; i += 1) await view.pick('Anime');
|
||||
assert.equal(list.listeners.length, before);
|
||||
assert.equal(view.byId('instruction-list').children.every(option => option.listeners.length === 0), true);
|
||||
assert.equal(view.byId('instruction-name').value, 'Anime');
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
@@ -21,8 +21,23 @@ test('the section lists the concept rows in its order, labels and danger marks',
|
||||
assert.deepEqual(MAINTENANCE_SURFACES.map(([heading]) => heading), surfaces.map(text => text.replace('&', '&')));
|
||||
const production = MAINTENANCE_SURFACES.flatMap(([, rows]) => rows.map(([title, description, label, , danger = false]) =>
|
||||
[title, description, label, danger]));
|
||||
assert.deepEqual(production, concept.map(([title, ...rest]) => [title.replace('&', '&'), ...rest]));
|
||||
assert.equal(production.length, 8);
|
||||
// D192 (#550, beyond the concept): "Pair category and tags" follows "Unused prompt components" in Database.
|
||||
const expected = concept.map(([title, ...rest]) => [title.replace('&', '&'), ...rest]);
|
||||
expected.splice(expected.findIndex(([title]) => title === 'Unused prompt components') + 1, 0, PAIR_ROW);
|
||||
assert.deepEqual(production, expected);
|
||||
assert.equal(production.length, 9);
|
||||
});
|
||||
|
||||
const PAIR_ROW = Object.freeze(['Pair category and tags', 'Fill an empty category or tags on prompts older Pair node runs ' +
|
||||
'saved, from their saved pairs. Only empty fields of never-edited prompts.', 'Review plan', false]);
|
||||
|
||||
test('D192: the Pair category and tags row reviews the pair_metadata plan under its title (#550)', async () => {
|
||||
const { MAINTENANCE_SURFACES } = await maintenance;
|
||||
const { OPERATION_TITLES } = await review;
|
||||
const [, database] = MAINTENANCE_SURFACES[0];
|
||||
const row = database.find(([title]) => title === PAIR_ROW[0]);
|
||||
assert.deepEqual(row, [PAIR_ROW[0], PAIR_ROW[1], PAIR_ROW[2], 'pair_metadata']);
|
||||
assert.equal(OPERATION_TITLES.pair_metadata, 'Pair category and tags');
|
||||
});
|
||||
|
||||
const PLAN = Object.freeze({
|
||||
@@ -210,3 +225,40 @@ test('a rotated-log cleanup the log overtook offers Review again (S12)', async (
|
||||
assert.deepEqual(reviewAgainOf({ kind: 'maintenance', state: 'failed', scope: { operation: 'rotated_logs', failure: 'plan-changed' } }),
|
||||
{ operation: 'rotated_logs' });
|
||||
});
|
||||
|
||||
// #550 live test: a records operation's rows have no path, only a detail ("Prompt 12: category …").
|
||||
const RECORD_JOB = Object.freeze({
|
||||
id: 'a'.repeat(32), kind: 'maintenance', title: 'Pair category and tags', state: 'finished',
|
||||
admission: 'records', cancellable: false, progress: 100, processed: 0, total: 0, status_line: 'Filled 2 prompts',
|
||||
scope: { operation: 'pair_metadata' }, counters: { changed: 2, protected: 0, errors: 0 },
|
||||
started_at: 1, finished_at: 2,
|
||||
});
|
||||
|
||||
test('#550: path-less record rows are kept with their detail; file rows are unchanged', async () => {
|
||||
const { parseDetail, filterRows } = await jobView;
|
||||
const parsed = parseDetail({ ...RECORD_JOB, outcomes: { changed: [
|
||||
{ path: '', root: '', root_index: null, detail: 'Prompt 2325: category “asian”; tags asian' },
|
||||
{ path: '', root: '', root_index: null, detail: '' },
|
||||
{ path: 'a/b.png', root: 'output', root_index: 0, detail: 'Link removed' },
|
||||
] }, log: [] });
|
||||
assert.deepEqual(parsed.outcomes.changed, [
|
||||
{ path: '', root: '', root_index: null, detail: 'Prompt 2325: category “asian”; tags asian', record: true },
|
||||
{ path: 'a/b.png', root: 'output', root_index: 0, detail: 'Link removed', record: false },
|
||||
]);
|
||||
// The filter reads a record's detail and a file's path.
|
||||
assert.deepEqual(filterRows(parsed.outcomes.changed, 'asian').map(row => row.detail),
|
||||
['Prompt 2325: category “asian”; tags asian']);
|
||||
assert.deepEqual(filterRows(parsed.outcomes.changed, 'a/b').map(row => row.path), ['a/b.png']);
|
||||
});
|
||||
|
||||
test('#550: a records operation says records, not files, for an empty group and a filter', async () => {
|
||||
const { groupCopy, EMPTY_GROUP, NO_MATCH } = await jobView;
|
||||
assert.deepEqual(groupCopy(RECORD_JOB), { empty: 'No records in this group.', noMatch: 'No listed record matches this filter.',
|
||||
placeholder: 'Filter records…' });
|
||||
for (const operation of ['duplicate_prompts', 'unused_components']) {
|
||||
assert.equal(groupCopy({ ...RECORD_JOB, scope: { operation } }).empty, 'No records in this group.');
|
||||
}
|
||||
const files = { empty: EMPTY_GROUP, noMatch: NO_MATCH, placeholder: 'Filter by file path…' };
|
||||
assert.deepEqual(groupCopy({ ...RECORD_JOB, scope: { operation: 'missing_links' } }), files);
|
||||
assert.deepEqual(groupCopy({ ...RECORD_JOB, kind: 'scan', scope: {} }), files);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,247 @@
|
||||
// System › Auto-tagging "Skipped tags" (#431, D202): the add input, the chips, the client checks and the preview line.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const skippedModule = import('../../web/js/workspace/system-skipped-tags.mjs');
|
||||
const stateModule = import('../../web/js/workspace/system-state.mjs');
|
||||
const coreModule = import('../../web/js/shared/toolkit/core.js');
|
||||
|
||||
/** A minimal DOM: elements with children, attributes, listeners (aborted by their signal) and focus. */
|
||||
function fakeDOM() {
|
||||
const previous = global.document;
|
||||
class Element {
|
||||
constructor(tag) {
|
||||
this.tagName = tag.toUpperCase();
|
||||
this.children = [];
|
||||
this.attributes = {};
|
||||
this.listeners = [];
|
||||
this.dataset = {};
|
||||
this.className = '';
|
||||
this.id = '';
|
||||
this.value = '';
|
||||
this.title = '';
|
||||
this.disabled = false;
|
||||
this._text = '';
|
||||
}
|
||||
get localName() { return this.tagName.toLowerCase(); }
|
||||
get textContent() { return this._text + this.children.map(child => (typeof child === 'string' ? child : child.textContent)).join(''); }
|
||||
set textContent(text) { this.children = []; this._text = String(text); }
|
||||
append(...nodes) { for (const node of nodes) { if (typeof node !== 'string') node.parentNode = this; this.children.push(node); } }
|
||||
replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); }
|
||||
querySelectorAll(selector) {
|
||||
assert.equal(selector, 'button');
|
||||
const found = [];
|
||||
const walk = node => { for (const child of node.children) if (typeof child !== 'string') { if (child.tagName === 'BUTTON') found.push(child); walk(child); } };
|
||||
walk(this);
|
||||
return found;
|
||||
}
|
||||
focus() { global.document.activeElement = this; }
|
||||
setAttribute(name, value) { this.attributes[name] = String(value); }
|
||||
getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; }
|
||||
removeAttribute(name) { delete this.attributes[name]; }
|
||||
addEventListener(type, callback, options = {}) {
|
||||
if (options.signal?.aborted) return;
|
||||
const entry = { type, callback };
|
||||
this.listeners.push(entry);
|
||||
options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); });
|
||||
}
|
||||
dispatch(type, event = {}) {
|
||||
return Promise.all(this.listeners.filter(item => item.type === type).map(({ callback }) => callback({ type, target: this, ...event })));
|
||||
}
|
||||
}
|
||||
global.document = { createElement: tag => new Element(tag), activeElement: null };
|
||||
const walk = (node, found = []) => { found.push(node); node.children.forEach(child => typeof child !== 'string' && walk(child, found)); return found; };
|
||||
return { find: (root, predicate) => walk(root).find(predicate), restore() { global.document = previous; } };
|
||||
}
|
||||
|
||||
const tick = () => new Promise(resolve => setTimeout(resolve, 5));
|
||||
|
||||
async function mounted({ saved = ['blurry'], answer, host = true } = {}) {
|
||||
const { skippedTagsBlock } = await skippedModule;
|
||||
const { createSystemDrafts } = await stateModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const drafts = createSystemDrafts({
|
||||
readBrowser: () => ({}), readRuntime: () => ({}),
|
||||
readHost: async () => ({ values: host ? { autotag_skipped_tags: saved } : {}, revision: 'sha256:one' }),
|
||||
writeBrowser: async () => ({}), writeHost: async () => ({ success: true, persisted: true }), writeRuntime: async () => ({}),
|
||||
});
|
||||
const scope = createScope();
|
||||
const posts = [];
|
||||
const api = {
|
||||
post: (url, body, options) => {
|
||||
posts.push({ url, body, signal: options.signal });
|
||||
if (answer) return answer(body, options);
|
||||
return Promise.resolve({ success: true, count: body.entries.length, sample: body.entries, sample_limit: 12,
|
||||
per_entry: body.entries.map((entry, index) => ({ entry, count: index })), sources: { library: 3, wd14: ['WD14 SwinV2'] } });
|
||||
},
|
||||
};
|
||||
const block = skippedTagsBlock({ api, scope, drafts, delay: 1 });
|
||||
scope.cleanup(drafts.subscribe(block.repaint));
|
||||
block.repaint();
|
||||
await drafts.reloadHost();
|
||||
const byId = id => dom.find(block.node, node => node.id === id);
|
||||
const chips = () => dom.find(block.node, node => node.className === 'tags skip-chips');
|
||||
const preview = () => dom.find(block.node, node => node.className === 'small skip-preview');
|
||||
const add = async text => { byId('tag-skipped-input').value = text; await byId('tag-skipped-add').dispatch('click'); };
|
||||
return { dom, drafts, scope, block, byId, chips, preview, posts, add };
|
||||
}
|
||||
|
||||
test('words split like the server: NFC, lower case, space, _ and - only', async () => {
|
||||
const { entryWords } = await skippedModule;
|
||||
assert.deepEqual(entryWords('Half-closed_eyes'), ['half', 'closed', 'eyes']);
|
||||
assert.deepEqual(entryWords('saber_(fate)'), ['saber', '(fate)']);
|
||||
assert.deepEqual(entryWords(' _ - '), []);
|
||||
assert.deepEqual(entryWords('CAFÉ'), ['café']);
|
||||
});
|
||||
|
||||
test('the client checks say why an entry is not added', async () => {
|
||||
const { entryProblem, SKIPPED_COPY } = await skippedModule;
|
||||
assert.equal(entryProblem(' sky ', ['blurry']), '');
|
||||
assert.equal(entryProblem(' ', []), SKIPPED_COPY.empty);
|
||||
assert.equal(entryProblem('_--', []), 'Enter a word.');
|
||||
assert.equal(entryProblem('a, b', []), 'No commas or line breaks.');
|
||||
assert.equal(entryProblem('a\nb', []), 'No commas or line breaks.');
|
||||
assert.equal(entryProblem('x'.repeat(51), []), 'At most 50 characters.');
|
||||
assert.equal(entryProblem('🏜'.repeat(50), []), '');
|
||||
assert.equal(entryProblem('Blurry-Background', ['blurry background']), 'Already in the list.');
|
||||
assert.equal(entryProblem('sky', Array.from({ length: 200 }, (_, i) => `w${i}`)), 'The list is full (200).');
|
||||
});
|
||||
|
||||
test('the preview line states the count, the sample and the sources', async () => {
|
||||
const { previewText, chipCount } = await skippedModule;
|
||||
assert.deepEqual(previewText({ count: 15, sample: ['blurry', 'signature'], sources: { wd14: ['WD14 SwinV2'] } }), {
|
||||
lead: 'Hides 15 tags from suggestions', rest: ': blurry, signature and 13 more. Tags already on images are not changed.' });
|
||||
assert.deepEqual(previewText({ count: 1, sample: ['blurry'], sources: { wd14: [] } }), {
|
||||
lead: 'Hides 1 tag from suggestions',
|
||||
rest: ': blurry. Checked against library tags only; no WD14 label list is downloaded. Tags already on images are not changed.' });
|
||||
assert.deepEqual(previewText({ count: 0, sample: [], sources: { wd14: ['WD14 ViT'] } }), {
|
||||
lead: 'Hides 0 tags from suggestions', rest: '. Tags already on images are not changed.' });
|
||||
assert.deepEqual(previewText(null).lead, 'Hides 0 tags from suggestions');
|
||||
assert.deepEqual(chipCount(3), { title: 'Hides 3 tags', text: '3' });
|
||||
assert.deepEqual(chipCount(1), { title: 'Hides 1 tag', text: '1' });
|
||||
assert.deepEqual(chipCount(0), { title: 'Hides no known tag', text: 'none' });
|
||||
assert.equal(chipCount(undefined), null);
|
||||
});
|
||||
|
||||
test('the block shows the saved list, previews it and carries its setting keys', async () => {
|
||||
const view = await mounted({ saved: ['blurry', 'signature'] });
|
||||
try {
|
||||
assert.equal(view.block.node.dataset.setting, 'tag-skipped');
|
||||
assert.equal(view.block.node.dataset.key, 'autotag_skipped_tags');
|
||||
assert.equal(view.preview().textContent, 'Checking…');
|
||||
await tick();
|
||||
assert.deepEqual(view.posts.map(post => [post.url, post.body]),
|
||||
[['/prompt_manager/autotag/skipped-tags/preview', { entries: ['blurry', 'signature'] }]]);
|
||||
assert.equal(view.preview().textContent, 'Hides 2 tags from suggestions: blurry, signature. Tags already on images are not changed.');
|
||||
const chips = view.chips().children;
|
||||
assert.deepEqual(chips.map(chip => [chip.className, chip.title, chip.textContent]), [
|
||||
['chip tag-suggestion none', 'Hides no known tag', 'blurrynone×'],
|
||||
['chip tag-suggestion', 'Hides 1 tag', 'signature1×']]);
|
||||
assert.equal(chips[1].children[2].getAttribute('aria-label'), 'Remove signature');
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('add and remove draft the list; a list set back is clean; Enter adds', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
await view.add(' closed eyes ');
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_skipped_tags'), ['blurry', 'closed eyes']);
|
||||
assert.equal(view.byId('tag-skipped-input').value, '');
|
||||
assert.equal(view.drafts.dirty(), true);
|
||||
view.byId('tag-skipped-input').value = 'sky';
|
||||
let prevented = false;
|
||||
await view.byId('tag-skipped-input').dispatch('keydown', { key: 'Enter', preventDefault: () => { prevented = true; } });
|
||||
assert.equal(prevented, true);
|
||||
await view.byId('tag-skipped-input').dispatch('keydown', { key: 'a' });
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_skipped_tags'), ['blurry', 'closed eyes', 'sky']);
|
||||
// Remove the two added chips: the list equals the saved one and nothing is unsaved (sameDraft).
|
||||
const removeButton = index => view.chips().children[index].children.at(-1);
|
||||
removeButton(2).focus();
|
||||
await removeButton(2).dispatch('click');
|
||||
assert.equal(global.document.activeElement, removeButton(1), 'focus moves to the neighbour');
|
||||
await removeButton(1).dispatch('click');
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_skipped_tags'), ['blurry']);
|
||||
assert.equal(view.drafts.dirty(), false);
|
||||
await tick();
|
||||
// Only the last list of a quick series is asked for.
|
||||
assert.deepEqual(view.posts.map(post => post.body.entries), [['blurry']]);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('a refused entry is not added and the line under the input says why', async () => {
|
||||
const view = await mounted();
|
||||
try {
|
||||
const problem = () => view.byId('tag-skipped-problem').textContent;
|
||||
await view.add('Blurry');
|
||||
assert.equal(problem(), 'Already in the list.');
|
||||
assert.equal(view.byId('tag-skipped-input').getAttribute('aria-invalid'), 'true');
|
||||
assert.equal(view.byId('tag-skipped-input').value, 'Blurry');
|
||||
await view.byId('tag-skipped-input').dispatch('input');
|
||||
assert.equal(problem(), '');
|
||||
assert.equal(view.byId('tag-skipped-input').getAttribute('aria-invalid'), null);
|
||||
await view.add('a,b');
|
||||
assert.equal(problem(), 'No commas or line breaks.');
|
||||
await view.add(' ');
|
||||
assert.equal(problem(), 'Enter a word.');
|
||||
assert.deepEqual(view.drafts.get('host', 'autotag_skipped_tags'), ['blurry']);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('a full list disables Add and says so', async () => {
|
||||
const view = await mounted({ saved: Array.from({ length: 200 }, (_, i) => `w${i}`) });
|
||||
try {
|
||||
assert.equal(view.byId('tag-skipped-add').disabled, true);
|
||||
assert.equal(view.byId('tag-skipped-problem').textContent, 'The list is full (200).');
|
||||
await view.chips().children[0].children.at(-1).dispatch('click');
|
||||
assert.equal(view.byId('tag-skipped-add').disabled, false);
|
||||
assert.equal(view.byId('tag-skipped-problem').textContent, '');
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('an empty list says no tags are skipped without a request', async () => {
|
||||
const view = await mounted({ saved: [] });
|
||||
try {
|
||||
await tick();
|
||||
assert.equal(view.preview().textContent, 'No tags are skipped.');
|
||||
assert.deepEqual(view.posts, []);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('a failed preview says so; a newer list aborts the older request', async () => {
|
||||
let calls = 0;
|
||||
const view = await mounted({ answer: (body, options) => {
|
||||
calls += 1;
|
||||
if (calls === 1) return new Promise((_resolve, reject) => options.signal.addEventListener('abort', () => reject(new Error('aborted'))));
|
||||
return Promise.reject(Object.assign(new Error('busy'), { code: 'busy' }));
|
||||
} });
|
||||
try {
|
||||
await tick();
|
||||
assert.equal(view.posts.length, 1);
|
||||
await view.add('sky');
|
||||
assert.equal(view.posts[0].signal.aborted, true);
|
||||
await tick();
|
||||
assert.equal(view.preview().textContent, 'The preview could not be read.');
|
||||
// The list stays editable.
|
||||
assert.equal(view.byId('tag-skipped-input').disabled, false);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('before the settings are read the controls are disabled', async () => {
|
||||
const view = await mounted({ host: false });
|
||||
try {
|
||||
assert.equal(view.byId('tag-skipped-input').disabled, true);
|
||||
assert.equal(view.byId('tag-skipped-add').disabled, true);
|
||||
await view.add('sky');
|
||||
assert.equal(view.drafts.get('host', 'autotag_skipped_tags'), undefined);
|
||||
} finally { view.scope.dispose(); view.dom.restore(); }
|
||||
});
|
||||
|
||||
test('leaving the section aborts a request in flight and the pending timer', async () => {
|
||||
const view = await mounted({ answer: (_body, options) => new Promise((_r, reject) => options.signal.addEventListener('abort', () => reject(new Error('x')))) });
|
||||
await tick();
|
||||
assert.equal(view.posts.length, 1);
|
||||
view.scope.dispose();
|
||||
assert.equal(view.posts[0].signal.aborted, true);
|
||||
view.dom.restore();
|
||||
});
|
||||
@@ -402,11 +402,12 @@ test('a partial host save keeps what saved, adopts its revision and fails only t
|
||||
assert.equal(drafts.statusText(), 'missing can\'t be used: not found. Remove it or reconnect it, then save.');
|
||||
// The next save carries the revision the saved part produced: no false "changed elsewhere".
|
||||
domains.outcome.host = null;
|
||||
drafts.set('host', 'gallery_root_paths', ['a']);
|
||||
// (#431: a list equal to the saved one is no change, so a different one is drafted.)
|
||||
drafts.set('host', 'gallery_root_paths', ['a', 'b']);
|
||||
const second = await drafts.save();
|
||||
assert.equal(second.host, 'saved');
|
||||
assert.equal(domains.calls.at(-1)[2].expected_revision, 'sha256:lora');
|
||||
assert.deepEqual(domains.calls.at(-1)[1], { gallery_root_paths: ['a'] });
|
||||
assert.deepEqual(domains.calls.at(-1)[1], { gallery_root_paths: ['a', 'b'] });
|
||||
// Without a server reason, the status says some were saved and the highlighted ones were not.
|
||||
domains.outcome.host = () => { throw Object.assign(new Error('boom'), { partial: { saved: ['x'], revision: 'sha256:three' } }); };
|
||||
drafts.set('host', 'x', 1);
|
||||
@@ -429,3 +430,39 @@ test('runtime adapter maps config.level to runtime_level and back', async () =>
|
||||
assert.deepEqual(runtimeBody({ runtime_level: 'WARNING' }), { level: 'WARNING' });
|
||||
assert.deepEqual(runtimeBody({ runtime_level: 'CRITICAL', console_logging: true }), { level: 'CRITICAL', console_logging: true });
|
||||
});
|
||||
|
||||
test('sameDraft compares arrays and plain objects by value, keys in any order (#431)', async () => {
|
||||
const { sameDraft } = await source;
|
||||
assert.equal(sameDraft(1, 1), true);
|
||||
assert.equal(sameDraft(Number.NaN, Number.NaN), true);
|
||||
assert.equal(sameDraft(0, -0), false);
|
||||
assert.equal(sameDraft(['a', 'b'], ['a', 'b']), true);
|
||||
assert.equal(sameDraft(['a', 'b'], ['b', 'a']), false);
|
||||
assert.equal(sameDraft([], []), true);
|
||||
assert.equal(sameDraft([], {}), false);
|
||||
assert.equal(sameDraft({ gguf: 'x', hf: 'y' }, { hf: 'y', gguf: 'x' }), true);
|
||||
assert.equal(sameDraft([{ id: 'a', name: 'n' }], [{ name: 'n', id: 'a' }]), true);
|
||||
assert.equal(sameDraft({ a: 1 }, { a: 2 }), false);
|
||||
assert.equal(sameDraft('a', ['a']), false);
|
||||
assert.equal(sameDraft(null, {}), false);
|
||||
assert.equal(sameDraft(new Date(0), new Date(0)), false);
|
||||
});
|
||||
|
||||
test('a list draft set back to the saved list is clean, and a save cleans it (#431)', async () => {
|
||||
const domains = adapters({ host: { autotag_skipped_tags: ['blurry'] } });
|
||||
const drafts = await loaded(domains);
|
||||
drafts.set('host', 'autotag_skipped_tags', ['blurry', 'signature']);
|
||||
assert.equal(drafts.dirty('host'), true);
|
||||
// Removing the added chip builds a new array equal to the saved one.
|
||||
drafts.set('host', 'autotag_skipped_tags', ['blurry']);
|
||||
assert.equal(drafts.dirty('host'), false);
|
||||
drafts.set('host', 'autotag_skipped_tags', ['signature']);
|
||||
await drafts.save();
|
||||
assert.equal(drafts.dirty('host'), false);
|
||||
assert.deepEqual(drafts.saved('host').autotag_skipped_tags, ['signature']);
|
||||
// A reload whose list equals the draft's (a new array) keeps nothing dirty.
|
||||
drafts.set('host', 'autotag_skipped_tags', ['sky']);
|
||||
domains.readHost = async () => ({ values: { autotag_skipped_tags: ['sky'] }, revision: 'sha256:three' });
|
||||
await drafts.load('host');
|
||||
assert.equal(drafts.dirty('host'), false);
|
||||
});
|
||||
|
||||
@@ -42,6 +42,8 @@ function fakeDOM() {
|
||||
this.children.push(node);
|
||||
}
|
||||
}
|
||||
replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); }
|
||||
focus() { global.document.activeElement = this; }
|
||||
remove() { if (this.parentNode) this.parentNode.children = this.parentNode.children.filter(child => child !== this); this.parentNode = null; }
|
||||
setAttribute(name, value) { this.attributes[name] = String(value); }
|
||||
getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; }
|
||||
@@ -65,7 +67,9 @@ function fakeDOM() {
|
||||
}
|
||||
|
||||
const HOST = { autotag_model_type: 'wd14-swinv2', autotag_existing: 'skip', autotag_general_threshold: 0.35,
|
||||
autotag_character_threshold: 0.85, autotag_suggestion_limit: 10, autotag_template: 'Describe the image as tags.' };
|
||||
autotag_character_threshold: 0.85, autotag_suggestion_limit: 10, autotag_builtin_instructions: 'Built-in tags.',
|
||||
autotag_instructions: [{ id: '0123456789ab', name: 'Describe', text: 'Describe the image as tags.' }],
|
||||
autotag_default_instructions: { gguf: '0123456789ab', hf: 'builtin' } };
|
||||
const entry = (label, size, extra = {}) => ({ name: label, label, size_label: size, downloaded: true, loaded: false, ...extra });
|
||||
const MODELS = {
|
||||
success: true,
|
||||
@@ -124,7 +128,8 @@ test('the section keeps the concept surface, rows, labels and actions in order',
|
||||
for (const option of ['Skip already tagged', 'Merge with existing', 'Replace after review']) assert.ok(body.includes(option) && SOURCE.includes(`'${option}'`));
|
||||
assert.ok(body.includes('<h2>Auto-tagging</h2>') && SOURCE.includes("'h2', 'Auto-tagging'"));
|
||||
assert.ok(body.includes('One tagging configuration for images and prompt-library actions. Every result can be reviewed before applying.'));
|
||||
assert.ok(body.includes('LLM tag template') && SOURCE.includes("'LLM tag template'"));
|
||||
// D203 (#431): Saved instructions replace the concept's template field.
|
||||
assert.ok(body.includes('LLM tag template') && !SOURCE.includes("'LLM tag template'"));
|
||||
assert.ok(body.includes('"Preview tagging review", "tag-review", "", "ca-primary"'));
|
||||
// Prototype copy is never shipped.
|
||||
for (const sample of ['Simulate download', 'Simulate unload', 'Ollama']) assert.ok(!SOURCE.includes(sample), sample);
|
||||
@@ -167,27 +172,25 @@ test('controls draft their host keys; numbers as numbers, other text as typed',
|
||||
const { SETTING_KEYS, coerceNumber, draftValue } = await tagging;
|
||||
assert.deepEqual(SETTING_KEYS, { 'tag-model': 'autotag_model_type', 'tag-existing': 'autotag_existing',
|
||||
'threshold-general': 'autotag_general_threshold', 'threshold-character': 'autotag_character_threshold',
|
||||
'tag-suggestions': 'autotag_suggestion_limit', 'tag-template': 'autotag_template' });
|
||||
'tag-suggestions': 'autotag_suggestion_limit' });
|
||||
assert.deepEqual(['0.4', ' 1.5 ', '15', '', 'abc', '1e-1'].map(coerceNumber), [0.4, 1.5, 15, '', 'abc', 0.1]);
|
||||
assert.equal(draftValue('autotag_suggestion_limit', '15'), 15);
|
||||
assert.equal(draftValue('autotag_template', ' 12 '), ' 12 ');
|
||||
assert.equal(draftValue('autotag_existing', ' 12 '), ' 12 ');
|
||||
assert.equal(draftValue('autotag_existing', undefined), '');
|
||||
const { dom, drafts, scope, byId } = await mounted();
|
||||
try {
|
||||
assert.equal(byId('tag-model').value, 'wd14-swinv2');
|
||||
assert.equal(byId('threshold-general').value, '0.35');
|
||||
assert.equal(byId('tag-template').value, HOST.autotag_template);
|
||||
assert.deepEqual([byId('threshold-general').min, byId('threshold-general').max, byId('threshold-general').step], ['0', '1', '0.05']);
|
||||
assert.deepEqual([byId('tag-suggestions').min, byId('tag-suggestions').max], ['1', '100']);
|
||||
byId('tag-existing').value = 'merge'; await byId('tag-existing').dispatch('change');
|
||||
byId('threshold-general').value = '0.4'; await byId('threshold-general').dispatch('change');
|
||||
byId('tag-suggestions').value = '15'; await byId('tag-suggestions').dispatch('change');
|
||||
byId('tag-template').value = 'Tags only.'; await byId('tag-template').dispatch('input');
|
||||
assert.deepEqual(drafts.changes('host'), { autotag_existing: 'merge', autotag_general_threshold: 0.4,
|
||||
autotag_suggestion_limit: 15, autotag_template: 'Tags only.' });
|
||||
autotag_suggestion_limit: 15 });
|
||||
// Every control names its stored key, so a refused save highlights it (D92).
|
||||
for (const id of Object.keys(SETTING_KEYS)) assert.equal(byId(id).dataset.key, SETTING_KEYS[id]);
|
||||
assert.equal(byId('tag-template').dataset.setting, 'tag-template');
|
||||
assert.equal(byId('tag-template'), undefined);
|
||||
} finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); }
|
||||
});
|
||||
|
||||
@@ -201,7 +204,8 @@ test('opening the section reads the model status once; the select waits for it',
|
||||
assert.deepEqual(byId('tag-model').options.map(option => option.textContent),
|
||||
['WD14 SwinV2 · ~400 MB', 'WD14 ViT · ~400 MB', 'GGUF JoyCaption · ~4.5 GB', 'HuggingFace JoyCaption · ~16 GB']);
|
||||
assert.deepEqual(surface.children.map(node => node.className),
|
||||
['', '', 'setting-row', 'setting-row', 'setting-row', 'setting-row', 'setting-row', 'field', 'row-actions', 'operation']);
|
||||
['', '', 'setting-row', 'setting-row', 'setting-row', 'setting-row', 'setting-row', 'setting-block', 'setting-block', 'setting-row',
|
||||
'setting-row', 'small instruction-reset', 'row-actions', 'operation']);
|
||||
} finally { scope.dispose(); dom.restore(); }
|
||||
const failed = await mounted({ modelsFail: true });
|
||||
try {
|
||||
@@ -302,7 +306,9 @@ test('Preview tagging review opens the Tag image dialog with the section’s val
|
||||
assert.deepEqual(taggingPreset(HOST), { model: 'wd14-swinv2', options: { general_threshold: 0.35, character_threshold: 0.85,
|
||||
suggestion_limit: 10 } });
|
||||
assert.deepEqual(taggingPreset({ ...HOST, autotag_model_type: 'gguf' }), { model: 'gguf',
|
||||
options: { instructions: HOST.autotag_template, suggestion_limit: 10 } });
|
||||
options: { instructions: 'Describe the image as tags.', suggestion_limit: 10 } });
|
||||
// D205 (#431): each language-model engine reads its own drafted default; Built-in's text comes from the server.
|
||||
assert.deepEqual(taggingPreset({ ...HOST, autotag_model_type: 'hf' }).options.instructions, 'Built-in tags.');
|
||||
assert.equal(draftProblem(HOST), null);
|
||||
assert.deepEqual(draftProblem({ ...HOST, autotag_general_threshold: 1.5 }),
|
||||
{ key: 'autotag_general_threshold', message: 'General-tag confidence must be between 0 and 1.' });
|
||||
@@ -319,15 +325,17 @@ test('Preview tagging review opens the Tag image dialog with the section’s val
|
||||
assert.equal(taggingPreset({ ...HOST, autotag_general_threshold: 0.1 + 0.2 }).options.general_threshold, 0.3);
|
||||
assert.equal(draftProblem({ ...HOST, autotag_suggestion_limit: 2.5 }).key, 'autotag_suggestion_limit');
|
||||
assert.equal(draftProblem({ ...HOST, autotag_suggestion_limit: 0 }, { limit: false }), null);
|
||||
assert.deepEqual(draftProblem({ ...HOST, autotag_model_type: 'hf', autotag_template: ' ' }),
|
||||
{ key: 'autotag_template', message: 'LLM tag template must be non-empty text.' });
|
||||
assert.deepEqual(draftProblem({ ...HOST, autotag_model_type: 'hf', autotag_template: 'x'.repeat(4001) }),
|
||||
{ key: 'autotag_template', message: 'LLM tag template must be at most 4000 characters.' });
|
||||
// #431: a saved entry the server would refuse holds back a language-model run, with the server's sentence.
|
||||
const entries = list => ({ ...HOST, autotag_model_type: 'hf', autotag_instructions: list });
|
||||
assert.deepEqual(draftProblem(entries([{ id: '0123456789ab', name: 'A', text: ' ' }])),
|
||||
{ key: 'autotag_instructions', message: 'Saved instructions: entry 1 text must be non-empty text.' });
|
||||
assert.deepEqual(draftProblem(entries([{ id: '0123456789ab', name: 'A', text: 'x'.repeat(4001) }])),
|
||||
{ key: 'autotag_instructions', message: 'Saved instructions: entry 1 text must be at most 4000 characters.' });
|
||||
// Characters are code points, as Python's len counts them: 4000 emoji (8000 UTF-16 units) are allowed.
|
||||
assert.equal(draftProblem({ ...HOST, autotag_model_type: 'hf', autotag_template: '🏜'.repeat(4000) }), null);
|
||||
assert.equal(draftProblem({ ...HOST, autotag_model_type: 'hf', autotag_template: '🏜'.repeat(4001) }).key, 'autotag_template');
|
||||
// A WD14 run never reads the template, nor a language model the thresholds.
|
||||
assert.equal(draftProblem({ ...HOST, autotag_template: '' }), null);
|
||||
assert.equal(draftProblem(entries([{ id: '0123456789ab', name: 'A', text: '🏜'.repeat(4000) }])), null);
|
||||
assert.equal(draftProblem(entries([{ id: '0123456789ab', name: 'A', text: '🏜'.repeat(4001) }])).key, 'autotag_instructions');
|
||||
// A WD14 run never reads the instructions, nor a language model the thresholds.
|
||||
assert.equal(draftProblem({ ...HOST, autotag_instructions: [{ id: '0123456789ab', name: '', text: '' }] }), null);
|
||||
assert.equal(draftProblem({ ...HOST, autotag_model_type: 'gguf', autotag_general_threshold: 7 }), null);
|
||||
|
||||
const none = await mounted({ image: null });
|
||||
@@ -372,9 +380,12 @@ test('the batch review grid, body and Cancel-only reasons (D134, #379, CUJ 4–5
|
||||
['Settings', 'General 0.35 · Character 0.85 · up to 10 tags per image'], ['Backup', 'Create and verify before changes']]);
|
||||
const llm = { ...HOST, autotag_model_type: 'gguf' };
|
||||
assert.deepEqual(batchGrid({ ...PLAN, model: { type: 'gguf' }, backup: { policy: 'off' }, estimate_seconds: null,
|
||||
estimate_device: null, suggestion_limit: 4 }, llm).slice(-3),
|
||||
estimate_device: null, suggestion_limit: 4 }, llm).slice(-4),
|
||||
[['Time', 'Language-model engines take longer per image. Activity shows the time left once tagging starts.'],
|
||||
['Settings', 'Template of 27 characters · up to 4 tags per image'], ['Backup', 'No safety backup (the policy is off)']]);
|
||||
['Instructions', 'Describe'], ['Settings', 'Up to 4 tags per image'], ['Backup', 'No safety backup (the policy is off)']]);
|
||||
// D205 (#431): HF shows its own default by name.
|
||||
assert.deepEqual(batchGrid({ ...PLAN, model: { type: 'hf' } }, { ...llm, autotag_model_type: 'hf' }).find(([term]) => term === 'Instructions'),
|
||||
['Instructions', 'Built-in']);
|
||||
assert.equal(batchGrid({}, HOST)[2][1], '0');
|
||||
assert.equal(batchGrid({}, HOST)[6][1], 'General 0.35 · Character 0.85');
|
||||
assert.equal(existingLabel('odd'), 'odd');
|
||||
@@ -382,7 +393,7 @@ test('the batch review grid, body and Cancel-only reasons (D134, #379, CUJ 4–5
|
||||
options: { general_threshold: 0.35, character_threshold: 0.85 } });
|
||||
// #369: a threshold a hair off the grid is sent snapped to it.
|
||||
assert.equal(batchBody({ ...HOST, autotag_character_threshold: 0.1 + 0.2 }).options.character_threshold, 0.3);
|
||||
assert.deepEqual(batchBody(llm), { model_type: 'gguf', existing: 'skip', options: { instructions: HOST.autotag_template } });
|
||||
assert.deepEqual(batchBody(llm), { model_type: 'gguf', existing: 'skip', options: { instructions: 'Describe the image as tags.' } });
|
||||
assert.equal(batchCloseNote(PLAN), null);
|
||||
assert.equal(batchCloseNote({ ...PLAN, review_required: true }), REPLACE_NOTE);
|
||||
assert.equal(REPLACE_NOTE, 'Replacing tags needs a review of each image. Use Preview tagging review, or choose Skip or Merge for a batch run.');
|
||||
|
||||
@@ -73,6 +73,7 @@ class FakeTagger:
|
||||
self.calls = []
|
||||
self.read_paths = []
|
||||
self.read_bytes = []
|
||||
self.skips = []
|
||||
self.download_model = MagicMock(side_effect=AssertionError("downloaded"))
|
||||
self.load_error = None
|
||||
self.block = None # (entered, proceed) events
|
||||
@@ -115,8 +116,10 @@ class FakeTagger:
|
||||
prompt=None,
|
||||
general_threshold=None,
|
||||
character_threshold=None,
|
||||
skip=None,
|
||||
):
|
||||
self.read_paths.append(image_path)
|
||||
self.skips.append(skip)
|
||||
with open(image_path, "rb") as handle:
|
||||
self.read_bytes.append(handle.read())
|
||||
self.calls.append(
|
||||
@@ -132,9 +135,11 @@ class FakeTagger:
|
||||
entered, proceed = self.block
|
||||
entered.set()
|
||||
proceed.wait(WAIT)
|
||||
# As the engine does (#431): a skipped tag uses up no slot.
|
||||
kept = [pair for pair in SCORED if skip is None or not skip.matches(pair[0])]
|
||||
if self.loaded.startswith("wd14"):
|
||||
return list(SCORED[:limit])
|
||||
return [(name, None) for name, _ in SCORED[:limit]]
|
||||
return list(kept[:limit])
|
||||
return [(name, None) for name, _ in kept[:limit]]
|
||||
|
||||
|
||||
class SuggestTestCase(AnnotationAPITestCase):
|
||||
|
||||
@@ -1144,8 +1144,10 @@ class TestEnvelopes(AnnotationAPITestCase):
|
||||
self.assertEqual(
|
||||
sorted(await self.json_of(resp)),
|
||||
[
|
||||
"default_instructions", # #431 adds the saved instructions
|
||||
"default_prompt",
|
||||
"existing", # #328 adds the saved batch policy and model
|
||||
"instructions",
|
||||
"loaded_model_type",
|
||||
"model_loaded",
|
||||
"model_type",
|
||||
|
||||
@@ -129,8 +129,10 @@ class FakeAutotagService:
|
||||
prompt=None,
|
||||
general_threshold=None,
|
||||
character_threshold=None,
|
||||
skip=None,
|
||||
):
|
||||
"""The suggestion entry point (#379): the fixed tags, at most ``limit``."""
|
||||
"""The suggestion entry point (#379): the fixed tags, at most ``limit``,
|
||||
without the ones ``skip`` hides (#431)."""
|
||||
self.suggest_calls += 1
|
||||
tags = self.generate_tags(
|
||||
image_path,
|
||||
@@ -138,6 +140,7 @@ class FakeAutotagService:
|
||||
general_threshold=general_threshold,
|
||||
character_threshold=character_threshold,
|
||||
)
|
||||
tags = [name for name in tags if skip is None or not skip.matches(name)]
|
||||
return [(name, None) for name in tags[:limit]]
|
||||
|
||||
|
||||
@@ -729,6 +732,32 @@ class TestAutotagErrorBodiesHideServerPaths(AutotagAPITestCase):
|
||||
|
||||
class TestUnloadAutotag(AutotagAPITestCase):
|
||||
|
||||
async def test_unload_runs_off_the_loop_and_holds_the_gate_until_done(self):
|
||||
started = threading.Event()
|
||||
release = threading.Event()
|
||||
self.addCleanup(release.set)
|
||||
threads = []
|
||||
|
||||
def unload():
|
||||
threads.append(threading.get_ident())
|
||||
started.set()
|
||||
release.wait(5)
|
||||
self.service._loaded_type = None
|
||||
|
||||
self.service._loaded_type = "gguf"
|
||||
self.service.unload_model = unload
|
||||
first = asyncio.ensure_future(
|
||||
self.client.request("POST", "/prompt_manager/autotag/unload")
|
||||
)
|
||||
self.assertTrue(await asyncio.to_thread(started.wait, 5))
|
||||
self.assertNotEqual(threads, [threading.get_ident()])
|
||||
second = await self.client.request("POST", "/prompt_manager/autotag/unload")
|
||||
self.assertEqual(second.status, 409)
|
||||
release.set()
|
||||
data = await (await first).json()
|
||||
self.assertEqual(data["message"], "GGUF model unloaded successfully")
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_nothing_loaded(self):
|
||||
resp = await self.client.request("POST", "/prompt_manager/autotag/unload")
|
||||
data = await resp.json()
|
||||
@@ -1533,7 +1562,8 @@ class TestSavedTaggingDefaults(AutotagAPITestCase):
|
||||
"EXISTING",
|
||||
"GENERAL_THRESHOLD",
|
||||
"CHARACTER_THRESHOLD",
|
||||
"TEMPLATE",
|
||||
"INSTRUCTIONS",
|
||||
"DEFAULT_INSTRUCTIONS",
|
||||
"SUGGESTION_LIMIT",
|
||||
):
|
||||
self.addCleanup(setattr, AutotagConfig, attr, getattr(AutotagConfig, attr))
|
||||
@@ -1554,7 +1584,11 @@ class TestSavedTaggingDefaults(AutotagAPITestCase):
|
||||
self.config.EXISTING = "replace_after_review"
|
||||
self.config.GENERAL_THRESHOLD = 0.4
|
||||
self.config.CHARACTER_THRESHOLD = 0.7
|
||||
self.config.TEMPLATE = "Saved template"
|
||||
# #431: the saved engine's default instructions.
|
||||
self.config.INSTRUCTIONS = (
|
||||
{"id": "0123456789ab", "name": "Saved", "text": "Saved template"},
|
||||
)
|
||||
self.config.DEFAULT_INSTRUCTIONS = {"gguf": "0123456789ab", "hf": "builtin"}
|
||||
self.config.SUGGESTION_LIMIT = 15
|
||||
data = await (await self.client.get("/prompt_manager/autotag/models")).json()
|
||||
self.assertEqual(
|
||||
@@ -1578,6 +1612,27 @@ class TestSavedTaggingDefaults(AutotagAPITestCase):
|
||||
"suggestion_limit": 15,
|
||||
},
|
||||
)
|
||||
# Built-in first with every text, and each engine's default.
|
||||
self.assertEqual(
|
||||
data["instructions"],
|
||||
[
|
||||
{"id": "builtin", "name": "Built-in", "text": DEFAULT_TAG_TEMPLATE},
|
||||
{"id": "0123456789ab", "name": "Saved", "text": "Saved template"},
|
||||
],
|
||||
)
|
||||
self.assertEqual(
|
||||
data["default_instructions"], {"gguf": "0123456789ab", "hf": "builtin"}
|
||||
)
|
||||
# HF reads its own default; a WD14 engine reads GGUF's.
|
||||
for model_type, prompt in (
|
||||
("hf", DEFAULT_TAG_TEMPLATE),
|
||||
("wd14-vit", "Saved template"),
|
||||
):
|
||||
self.config.MODEL_TYPE = model_type
|
||||
data = await (
|
||||
await self.client.get("/prompt_manager/autotag/models")
|
||||
).json()
|
||||
self.assertEqual(data["default_prompt"], prompt, model_type)
|
||||
|
||||
async def test_single_without_thresholds_uses_saved_ones(self):
|
||||
image = self._make_image(self.output_dir)
|
||||
|
||||
+3
-54
@@ -343,7 +343,7 @@ class TestReadJsonBody(AioHTTPTestCase):
|
||||
|
||||
|
||||
class TestLongJobLock(unittest.TestCase):
|
||||
"""_acquire_long_job: one job per family at a time, released on exit."""
|
||||
"""The refusal a busy long job answers with."""
|
||||
|
||||
def setUp(self):
|
||||
database = MagicMock()
|
||||
@@ -353,33 +353,6 @@ class TestLongJobLock(unittest.TestCase):
|
||||
def _run(self, coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
def test_second_acquire_is_refused_until_first_releases(self):
|
||||
async def scenario():
|
||||
first = self.api._acquire_long_job("duplicates")
|
||||
self.assertIsNotNone(first)
|
||||
async with first:
|
||||
self.assertIsNone(self.api._acquire_long_job("duplicates"))
|
||||
# Other families are independent
|
||||
other = self.api._acquire_long_job("thumbnails")
|
||||
self.assertIsNotNone(other)
|
||||
async with other:
|
||||
pass
|
||||
again = self.api._acquire_long_job("duplicates")
|
||||
self.assertIsNotNone(again)
|
||||
async with again:
|
||||
pass
|
||||
|
||||
self._run(scenario())
|
||||
|
||||
def test_lock_is_released_when_the_job_raises(self):
|
||||
async def scenario():
|
||||
with self.assertRaises(RuntimeError):
|
||||
async with self.api._acquire_long_job("duplicates"):
|
||||
raise RuntimeError("boom")
|
||||
self.assertIsNotNone(self.api._acquire_long_job("duplicates"))
|
||||
|
||||
self._run(scenario())
|
||||
|
||||
def test_busy_response_is_409_with_envelope(self):
|
||||
resp = self.api._long_job_busy_response("Duplicate scan")
|
||||
self.assertEqual(resp.status, 409)
|
||||
@@ -796,6 +769,7 @@ class TestRevalidatedCachingBehindMiddlewares(CoreRoutesTestCase):
|
||||
self.assertEqual(resp.status, 200, path)
|
||||
self.assertEqual(resp.headers.get("Content-Encoding"), "gzip", path)
|
||||
self.assertEqual(resp.headers["Cache-Control"], "no-cache", path)
|
||||
await resp.read() # aiohttp before 3.11.11 hangs on an unread body
|
||||
again = await self.client.request(
|
||||
"GET",
|
||||
path,
|
||||
@@ -806,6 +780,7 @@ class TestRevalidatedCachingBehindMiddlewares(CoreRoutesTestCase):
|
||||
)
|
||||
self.assertEqual(again.status, 304, path)
|
||||
self.assertEqual(again.headers["Cache-Control"], "no-cache", path)
|
||||
await again.read()
|
||||
|
||||
|
||||
# ── instance helpers ──────────────────────────────────────────────────
|
||||
@@ -951,32 +926,6 @@ class TestMetadataParsing(unittest.TestCase):
|
||||
f.write(b"nope")
|
||||
self.assertEqual(self.api._extract_comfyui_metadata(garbage), {})
|
||||
|
||||
def test_parse_a1111_parameters(self):
|
||||
meta = {"parameters": "a cat\nNegative prompt: dog\nSteps: 20"}
|
||||
parsed = self.api._parse_comfyui_prompt(meta)
|
||||
self.assertEqual(parsed["positive_prompt"], "a cat")
|
||||
self.assertEqual(parsed["negative_prompt"], "dog")
|
||||
self.assertEqual(parsed["parameters"]["parameters"], meta["parameters"])
|
||||
|
||||
def test_parse_comfyui_fields(self):
|
||||
meta = {
|
||||
"prompt": json.dumps({"1": {"class_type": "X", "inputs": {}}}),
|
||||
"workflow": "not json",
|
||||
"steps": "20",
|
||||
"cfg": "not-json-either",
|
||||
}
|
||||
parsed = self.api._parse_comfyui_prompt(meta)
|
||||
self.assertIsInstance(parsed["prompt"], dict)
|
||||
self.assertEqual(parsed["workflow"], "not json")
|
||||
self.assertEqual(parsed["parameters"]["steps"], 20)
|
||||
self.assertEqual(parsed["parameters"]["cfg"], "not-json-either")
|
||||
|
||||
parsed = self.api._parse_comfyui_prompt(
|
||||
{"prompt": "plain text", "workflow": json.dumps({"nodes": []})}
|
||||
)
|
||||
self.assertEqual(parsed["prompt"], "plain text")
|
||||
self.assertEqual(parsed["workflow"], {"nodes": []})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -452,6 +452,40 @@ class TestReview(DuplicateFilesTestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestReviewProtectedRows(DuplicateFilesTestCase):
|
||||
async def test_cleanup_lists_each_copy_the_review_kept(self):
|
||||
if not TestReview._can_symlink(self.tmpdir): # pragma: no cover
|
||||
self.skipTest("symlinks are not available")
|
||||
self.trio()
|
||||
job, group = await self.found_group()
|
||||
# b.png becomes a link after the find: the review protects it (#586).
|
||||
(self.output_dir / "b.png").unlink()
|
||||
os.symlink(str(self.output_dir / "a.png"), str(self.output_dir / "b.png"))
|
||||
plan = await self.review(job.id, [self.keep_of(group, "a.png")])
|
||||
self.assertEqual(plan["protected"]["count"], 1)
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
self.assertEqual(cleanup.state, "finished", cleanup.log)
|
||||
rows = {row["path"]: row for row in cleanup.outcomes["kept"]}
|
||||
self.assertEqual(
|
||||
rows["b.png"],
|
||||
{
|
||||
"path": "b.png",
|
||||
"root": rows["a.png"]["root"],
|
||||
"root_index": 0,
|
||||
"detail": "Kept: a link, not a regular file",
|
||||
},
|
||||
)
|
||||
self.assertEqual(rows["a.png"]["detail"], "Kept copy")
|
||||
self.assertEqual(cleanup.counters, {"removed": 1, "kept": 2, "errors": 0})
|
||||
report = cleanup.report()["outcomes"]["kept"]
|
||||
self.assertEqual(
|
||||
sorted(row["path"] for row in report["rows"]), ["a.png", "b.png"]
|
||||
)
|
||||
self.assertFalse(report["truncated"])
|
||||
|
||||
|
||||
class TestCleanupJob(DuplicateFilesTestCase):
|
||||
async def test_cleanup_is_a_records_job_with_a_safety_backup(self):
|
||||
self.trio()
|
||||
|
||||
@@ -42,6 +42,37 @@ class TestLibraryAPI(ImageAPITestCase):
|
||||
self.assertEqual(len(media["images"]), 1)
|
||||
self.assertEqual(media["images"][0]["path"], "same.png")
|
||||
|
||||
async def test_used_and_reverse_name_sorts_page_through_every_prompt(self):
|
||||
ids = {text: self._save_prompt(text) for text in ("alpha", "beta", "gamma")}
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
for text, runs, used in (
|
||||
("alpha", 1, "2026-01-01T00:00:00+00:00"),
|
||||
("beta", 9, "2026-02-01T00:00:00+00:00"),
|
||||
("gamma", 0, None),
|
||||
):
|
||||
conn.execute(
|
||||
"UPDATE prompts SET run_count=?, last_used_at=? WHERE id=?",
|
||||
(runs, used, ids[text]),
|
||||
)
|
||||
conn.commit()
|
||||
for sort, expected in (
|
||||
("most-used", ["beta", "alpha", "gamma"]),
|
||||
("last-used", ["beta", "alpha", "gamma"]),
|
||||
("name-desc", ["gamma", "beta", "alpha"]),
|
||||
):
|
||||
with self.subTest(sort=sort):
|
||||
seen, cursor = [], None
|
||||
for _ in expected:
|
||||
params = {"sort": sort, "limit": 1}
|
||||
if cursor:
|
||||
params["cursor"] = cursor
|
||||
response, page = await self.get(**params)
|
||||
self.assertEqual(response.status, 200)
|
||||
seen.extend(item["text"] for item in page["items"])
|
||||
cursor = page["next_cursor"]
|
||||
self.assertEqual(seen, expected)
|
||||
self.assertIsNone(cursor)
|
||||
|
||||
async def test_scoped_cursor_tampering_filters_and_continuation(self):
|
||||
for text in ("one", "two", "three"):
|
||||
self._save_prompt(text)
|
||||
|
||||
@@ -266,6 +266,30 @@ class TestApply(LogCleanupTestCase):
|
||||
report = json.dumps(job.report())
|
||||
self.assertNotIn(self.tmpdir, report)
|
||||
|
||||
async def test_apply_lists_each_kept_file(self):
|
||||
linked = self.seed()
|
||||
plan = await self.preview()
|
||||
_summary, job = await self.run_apply(plan)
|
||||
expected = [
|
||||
("prompt_manager.log", "Kept: the active log"),
|
||||
("prompt_manager.log.9", "Kept: another file"),
|
||||
("prompt_manager.log.bak", "Kept: another file"),
|
||||
]
|
||||
if linked:
|
||||
expected.append(("prompt_manager.log.4", "Kept: a link"))
|
||||
rows = job.outcomes["protected"]
|
||||
self.assertEqual(
|
||||
sorted((row["path"], row["detail"]) for row in rows), sorted(expected)
|
||||
)
|
||||
self.assertEqual(
|
||||
{(row["root"], row["root_index"]) for row in rows}, {("", None)}
|
||||
)
|
||||
report = job.report()["outcomes"]["protected"]
|
||||
self.assertEqual(
|
||||
(report["count"], len(report["rows"]), report["truncated"]),
|
||||
(plan["protected"]["count"], len(expected), False),
|
||||
)
|
||||
|
||||
async def test_apply_rows_name_each_file(self):
|
||||
self.seed()
|
||||
plan = await self.preview()
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
"""Pair category and tags repair through the maintenance routes (#550).
|
||||
|
||||
``pair_metadata`` is reviewed and applied like the other maintenance plans:
|
||||
``options`` must be ``{}``, the job is titled "Pair category and tags", the
|
||||
safety backup follows the ``maintenance.safety_backup`` policy, and the job's
|
||||
downloadable report lists every filled prompt, never cut at 200 rows.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
_mock_server = MagicMock()
|
||||
_mock_server.PromptServer.instance.routes = MagicMock()
|
||||
sys.modules.setdefault("server", _mock_server)
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from py.api.jobs import MAX_OUTCOME_ROWS # noqa: E402
|
||||
from py.api.maintenance import MAINTENANCE_TITLES # noqa: E402
|
||||
from py.config import PromptManagerConfig # noqa: E402
|
||||
|
||||
try:
|
||||
from tests.test_api_maintenance import WAIT, ApplyTestCase # noqa: E402
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_api_maintenance import WAIT, ApplyTestCase # noqa: E402
|
||||
|
||||
OPERATION = "pair_metadata"
|
||||
TITLE = "Pair category and tags"
|
||||
|
||||
|
||||
class PairMetadataAPITestCase(ApplyTestCase):
|
||||
def pair(self, positive, negative="", **metadata):
|
||||
pair_id = self.api.db.save_prompt_pair(positive, negative, **metadata)
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
prompt = conn.execute(
|
||||
"SELECT positive_prompt_id FROM prompt_pairs WHERE id=?", (pair_id,)
|
||||
).fetchone()[0]
|
||||
return pair_id, prompt
|
||||
|
||||
def category(self, prompt_id):
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
return conn.execute(
|
||||
"SELECT category FROM prompts WHERE id=?", (prompt_id,)
|
||||
).fetchone()[0]
|
||||
|
||||
|
||||
class TestPairMetadataRoutes(PairMetadataAPITestCase):
|
||||
async def test_preview_accepts_the_operation_and_no_options(self):
|
||||
self.assertEqual(MAINTENANCE_TITLES[OPERATION], TITLE)
|
||||
self.pair("a cat", category="portrait", tags=["a"])
|
||||
plan = await self.preview(OPERATION)
|
||||
self.assertEqual(plan["operation"], OPERATION)
|
||||
self.assertEqual(plan["scope"]["count"], 1)
|
||||
self.assertEqual(plan["backup"]["policy"], "on")
|
||||
refused = await self.preview(OPERATION, 400, options={"all": True})
|
||||
self.assertEqual(refused["code"], "invalid-request")
|
||||
|
||||
async def test_apply_fills_with_a_backup_and_names_it(self):
|
||||
_pair, prompt = self.pair("a cat", "neg", category="portrait", tags=["a", "b"])
|
||||
plan, summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(summary["title"], TITLE)
|
||||
self.assertEqual(summary["admission"], "records")
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(self.category(prompt), "portrait")
|
||||
[backup] = self.backups()
|
||||
self.assertEqual(
|
||||
job.status_line,
|
||||
"Filled 1 prompt from its saved pairs (1 category, 1 tag set) · "
|
||||
f"Safety backup {backup}",
|
||||
)
|
||||
self.assertEqual(
|
||||
job.outcomes["changed"][0]["detail"],
|
||||
f"Prompt {prompt}: category “portrait”; tags a, b",
|
||||
)
|
||||
self.assertEqual(job.counters["changed"], 1)
|
||||
|
||||
async def test_protected_prompts_are_counted(self):
|
||||
_a, prompt = self.pair("conflict", "n1", category="x")
|
||||
self.pair("conflict", "n2", category="y")
|
||||
self.pair("fill me", category="portrait")
|
||||
plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(plan["protected"]["reasons"], ["1 where pairs disagree"])
|
||||
self.assertEqual(job.counters["protected"], 1)
|
||||
self.assertIsNone(self.category(prompt))
|
||||
# Live test (#550): the count comes with its rows, in the review's wording.
|
||||
self.assertEqual(
|
||||
job.outcomes["protected"],
|
||||
[
|
||||
{
|
||||
"path": "",
|
||||
"root": "",
|
||||
"root_index": None,
|
||||
"detail": (
|
||||
f"Prompt {prompt}: kept, pairs disagree on category: x, y"
|
||||
),
|
||||
}
|
||||
],
|
||||
)
|
||||
report = job.report()["outcomes"]["protected"]
|
||||
self.assertEqual((report["count"], report["truncated"]), (1, False))
|
||||
|
||||
async def test_report_lists_every_kept_prompt(self):
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
count = MAX_OUTCOME_ROWS + 5
|
||||
for n in range(count):
|
||||
_pair, prompt = self.pair(f"kept {n:03d}", category="portrait")
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompts SET updated_at='2099-01-01T00:00:00+00:00' "
|
||||
"WHERE id=?",
|
||||
(prompt,),
|
||||
)
|
||||
_plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(job.counters["protected"], count)
|
||||
self.assertEqual(len(job.outcomes["protected"]), MAX_OUTCOME_ROWS)
|
||||
protected = job.report()["outcomes"]["protected"]
|
||||
self.assertEqual(
|
||||
(len(protected["rows"]), protected["truncated"]), (count, False)
|
||||
)
|
||||
self.assertTrue(
|
||||
protected["rows"][0]["detail"].endswith(
|
||||
": kept, edited after it was saved; its empty category or tags "
|
||||
"may be intentional"
|
||||
)
|
||||
)
|
||||
|
||||
async def test_policy_off_writes_without_a_backup(self):
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
_pair, prompt = self.pair("no backup", category="portrait")
|
||||
plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(plan["backup"]["policy"], "off")
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertIn("No safety backup (the policy is off)", "\n".join(job.log))
|
||||
self.assertTrue(job.status_line.endswith("· No safety backup"))
|
||||
self.assertEqual(self.backups(), [])
|
||||
self.assertEqual(self.category(prompt), "portrait")
|
||||
|
||||
async def test_reviewed_with_the_policy_on_keeps_the_backup(self):
|
||||
self.pair("promise", category="portrait")
|
||||
plan = await self.preview(OPERATION)
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
body = await self.apply(plan["token"])
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
self.assertEqual(len(self.backups()), 1)
|
||||
|
||||
async def test_plan_changed_writes_nothing(self):
|
||||
pair_id, prompt = self.pair("drift", category="portrait")
|
||||
plan = await self.preview(OPERATION)
|
||||
self.api.db.update_prompt_pair_metadata(pair_id, category="landscape")
|
||||
body = await self.apply(plan["token"])
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertEqual(job.last_event["code"], "plan-changed")
|
||||
self.assertIsNone(self.category(prompt))
|
||||
|
||||
async def test_report_lists_every_filled_prompt(self):
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
count = MAX_OUTCOME_ROWS + 5
|
||||
for n in range(count):
|
||||
self.pair(f"bulk {n:03d}", category="portrait")
|
||||
_plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(len(job.outcomes["changed"]), MAX_OUTCOME_ROWS)
|
||||
response = await self.client.request(
|
||||
"GET", f"/prompt_manager/jobs/{job.id}/report"
|
||||
)
|
||||
self.assertEqual(response.status, 200)
|
||||
report = await response.json()
|
||||
changed = report["outcomes"]["changed"]
|
||||
self.assertEqual(changed["count"], count)
|
||||
self.assertEqual(len(changed["rows"]), count)
|
||||
self.assertFalse(changed["truncated"])
|
||||
self.assertTrue(
|
||||
all(
|
||||
row["detail"].endswith("category “portrait”") for row in changed["rows"]
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,140 @@
|
||||
"""Protected maintenance items are listed in Activity, not only counted (#586).
|
||||
|
||||
Each record a maintenance job keeps gets one ``protected`` row with its
|
||||
reason, in the wording of that operation's review, and the job's
|
||||
downloadable report lists every one of them (never cut at 200 rows). A kept
|
||||
image link names its file, so Activity can still open and locate it.
|
||||
"""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
_mock_server = MagicMock()
|
||||
_mock_server.PromptServer.instance.routes = MagicMock()
|
||||
sys.modules.setdefault("server", _mock_server)
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from py.api.jobs import MAX_OUTCOME_ROWS # noqa: E402
|
||||
from py.config import GalleryConfig, PromptManagerConfig # noqa: E402
|
||||
|
||||
try:
|
||||
from tests.test_api_maintenance import ApplyTestCase # noqa: E402
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_api_maintenance import ApplyTestCase # noqa: E402
|
||||
|
||||
|
||||
def _record(detail):
|
||||
return {"path": "", "root": "", "root_index": None, "detail": detail}
|
||||
|
||||
|
||||
class TestProtectedRows(ApplyTestCase):
|
||||
def duplicate(self, text, created_at="2999-01-01"):
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
return conn.execute(
|
||||
"INSERT INTO prompts(text, created_at) VALUES(?, ?)",
|
||||
(text.upper(), created_at),
|
||||
).lastrowid
|
||||
|
||||
def assert_complete(self, job, count):
|
||||
self.assertEqual(job.counters["protected"], count)
|
||||
report = job.report()["outcomes"]["protected"]
|
||||
self.assertEqual(
|
||||
(report["count"], len(report["rows"]), report["truncated"]),
|
||||
(count, count, False),
|
||||
)
|
||||
|
||||
async def test_duplicate_prompts_lists_each_kept_copy(self):
|
||||
paired = self.save("A dog")
|
||||
self.api.db.save_prompt_pair("A dog", "")
|
||||
paired_copy = self.duplicate("A dog")
|
||||
titled = self.save("A bird")
|
||||
titled_copy = self.duplicate("A bird")
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"INSERT INTO prompt_library_state(prompt_id,title) VALUES(?,?)",
|
||||
(titled_copy, "Bird recipe"),
|
||||
)
|
||||
plan, _summary, job = await self.run_apply("duplicate_prompts")
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(
|
||||
sorted(row["detail"] for row in job.outcomes["protected"]),
|
||||
sorted(
|
||||
[
|
||||
f"Prompt {paired}: kept, belongs to a saved pair",
|
||||
f"Prompt {paired_copy}: kept, belongs to a saved pair",
|
||||
f"Prompt {titled}: kept, a copy has its own Library title",
|
||||
f"Prompt {titled_copy}: kept, a copy has its own Library title",
|
||||
]
|
||||
),
|
||||
)
|
||||
self.assertEqual(plan["protected"]["count"], 4)
|
||||
self.assert_complete(job, 4)
|
||||
|
||||
async def test_unused_components_lists_each_kept_prompt_with_its_reasons(self):
|
||||
self.save("Orphan")
|
||||
tagged = self.save("Kept", tags=["__protected__"])
|
||||
self.api.db.save_prompt_pair("Paired", "")
|
||||
paired = self.api.db.search_prompts(text="Paired")[0]["id"]
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"INSERT INTO prompt_tags(prompt_id, tag_id) "
|
||||
"SELECT ?, id FROM tags WHERE name='__protected__'",
|
||||
(paired,),
|
||||
)
|
||||
plan, _summary, job = await self.run_apply("unused_components")
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(
|
||||
sorted(job.outcomes["protected"], key=lambda row: row["detail"]),
|
||||
sorted(
|
||||
[
|
||||
_record(f"Prompt {tagged}: kept, tagged __protected__"),
|
||||
_record(
|
||||
f"Prompt {paired}: kept, in a saved pair; "
|
||||
"tagged __protected__"
|
||||
),
|
||||
],
|
||||
key=lambda row: row["detail"],
|
||||
),
|
||||
)
|
||||
self.assertEqual(plan["protected"]["count"], 2)
|
||||
self.assert_complete(job, 2)
|
||||
|
||||
async def test_report_lists_every_kept_prompt(self):
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
count = MAX_OUTCOME_ROWS + 5
|
||||
for n in range(count):
|
||||
self.save(f"Kept {n:03d}", tags=["__protected__"])
|
||||
_plan, _summary, job = await self.run_apply("unused_components")
|
||||
self.assertEqual(len(job.outcomes["protected"]), MAX_OUTCOME_ROWS)
|
||||
self.assert_complete(job, count)
|
||||
|
||||
async def test_missing_links_lists_each_kept_link_with_its_file(self):
|
||||
unmounted = Path(self.tmpdir) / "unmounted"
|
||||
unmounted.mkdir()
|
||||
GalleryConfig.MONITORING_DIRECTORIES = [str(self.output_dir), str(unmounted)]
|
||||
prompt = self.save("Links")
|
||||
self.image(prompt, "gone.png", exists=False)
|
||||
self.image(prompt, "sub/away.png", folder=unmounted, exists=False)
|
||||
self.image(prompt, "x.png", folder=Path(self.tmpdir) / "elsewhere")
|
||||
shutil.rmtree(unmounted)
|
||||
plan, _summary, job = await self.run_apply("missing_links")
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(plan["protected"]["count"], 2)
|
||||
rows = sorted(job.outcomes["protected"], key=lambda row: row["path"])
|
||||
self.assertEqual(
|
||||
[(row["path"], row["detail"]) for row in rows],
|
||||
[
|
||||
("sub/away.png", "Kept: link under an unavailable source"),
|
||||
("x.png", "Kept: link outside the image sources"),
|
||||
],
|
||||
)
|
||||
# The gone source keeps the path under its label; outside every
|
||||
# source the row names the file only.
|
||||
self.assertEqual((rows[0]["root"], rows[0]["root_index"]), ("unmounted", None))
|
||||
self.assertEqual((rows[1]["root"], rows[1]["root_index"]), ("", None))
|
||||
self.assertNotIn(self.tmpdir, str(job.report()))
|
||||
self.assert_complete(job, 2)
|
||||
@@ -684,6 +684,28 @@ class TestTagMutationRoutes(RouteCoverageCase):
|
||||
|
||||
class TestSaveRoute(RouteCoverageCase):
|
||||
|
||||
async def test_a_malformed_body_is_refused_and_saves_nothing(self):
|
||||
for body in (
|
||||
[],
|
||||
"text",
|
||||
{"text": 123},
|
||||
{"text": "fixture", "category": 123},
|
||||
{"text": "fixture", "notes": 123},
|
||||
{"text": "fixture", "tags": 123},
|
||||
{"text": "fixture", "rating": "high"},
|
||||
):
|
||||
with self.subTest(body=body):
|
||||
status, data = await self._json(
|
||||
"POST", "/prompt_manager/save", json=body
|
||||
)
|
||||
self.assertEqual(status, 400)
|
||||
self.assertFalse(data["success"])
|
||||
status, data = await self._json(
|
||||
"POST", "/prompt_manager/save", data=b"{not json"
|
||||
)
|
||||
self.assertEqual(status, 400)
|
||||
self.assertEqual(self.api.db.search_prompts(text="fixture"), [])
|
||||
|
||||
async def test_invalid_rating_is_400(self):
|
||||
status, data = await self._json(
|
||||
"POST", "/prompt_manager/save", json={"text": "ok", "rating": 99}
|
||||
|
||||
@@ -9,7 +9,6 @@ import json
|
||||
import ntpath
|
||||
import os
|
||||
import posixpath
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
@@ -378,25 +377,6 @@ class TestValidation(unittest.TestCase):
|
||||
self.assertTrue(text.endswith("."))
|
||||
|
||||
|
||||
class TestNoLegacySaveCallers(unittest.TestCase):
|
||||
def test_save_to_file_has_no_callers(self):
|
||||
offenders = []
|
||||
for folder in ("py", "utils", "database"):
|
||||
for path in (REPO / folder).rglob("*.py"):
|
||||
for number, line in enumerate(
|
||||
path.read_text(encoding="utf-8").splitlines(), 1
|
||||
):
|
||||
code = line.split("#", 1)[0]
|
||||
if (
|
||||
re.search(r"\bsave_to_file\(", code)
|
||||
and "def save_to_file" not in code
|
||||
):
|
||||
if "PromptManagerConfig.save_to_file('" in line:
|
||||
continue # its own docstring example
|
||||
offenders.append(f"{path.relative_to(REPO)}:{number}")
|
||||
self.assertEqual(offenders, [])
|
||||
|
||||
|
||||
try:
|
||||
from tests.test_api_images import ImageAPITestCase
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
|
||||
@@ -363,11 +363,11 @@ class TestWorkflowStorageOutcomes(WorkflowStorageTestCase):
|
||||
prompt_id = self.old_library(count=4)
|
||||
conn = self.api.db.model.get_connection()
|
||||
with conn:
|
||||
conn.execute(
|
||||
image_id = conn.execute(
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename, "
|
||||
"file_path, workflow_data) VALUES (?, '/x/b', 'b', '/x/b', x'00')",
|
||||
(prompt_id,),
|
||||
)
|
||||
).lastrowid
|
||||
store_text(conn, "a document no record references")
|
||||
_plan, _summary, job = await self.start()
|
||||
await self.finish(job)
|
||||
@@ -377,6 +377,13 @@ class TestWorkflowStorageOutcomes(WorkflowStorageTestCase):
|
||||
"Rewrote 4 image records; 1 stored value not text, kept as is", log
|
||||
)
|
||||
self.assertIn("Removed 1 unused workflow document", log)
|
||||
# #586: the kept value has its row, in the review's wording.
|
||||
self.assertEqual(
|
||||
[row["detail"] for row in job.outcomes["protected"]],
|
||||
[f"Image record {image_id}: workflow, stored value not text; kept as is"],
|
||||
)
|
||||
report = job.report()["outcomes"]["protected"]
|
||||
self.assertEqual((report["count"], report["truncated"]), (1, False))
|
||||
|
||||
async def reclaim_with(self, **patches):
|
||||
self.old_library(count=4)
|
||||
|
||||
@@ -54,6 +54,8 @@ class PerImageTagger(HookedTagger):
|
||||
names = self.by_bytes.get(self.read_bytes[-1])
|
||||
if names is None:
|
||||
return result
|
||||
skip = kwargs.get("skip") # as the engine does (#431)
|
||||
names = [name for name in names if skip is None or not skip.matches(name)]
|
||||
return [(name, 0.9) for name in names][: kwargs["limit"]]
|
||||
|
||||
|
||||
|
||||
@@ -158,7 +158,30 @@ class TestLegacyBatch(LegacyTestCase):
|
||||
self.image("b.png", (2, 2, 2))
|
||||
await self.relist()
|
||||
await self.legacy("model_type=gguf&prompt=%20")
|
||||
self.assertEqual(self.tagger.calls[-1]["prompt"], AutotagConfig.TEMPLATE)
|
||||
self.assertEqual(
|
||||
self.tagger.calls[-1]["prompt"],
|
||||
AutotagConfig.default_instructions_text("gguf"),
|
||||
)
|
||||
# #431: a blank prompt falls back to the request engine's own default.
|
||||
saved = (AutotagConfig.INSTRUCTIONS, AutotagConfig.DEFAULT_INSTRUCTIONS)
|
||||
self.addCleanup(
|
||||
lambda: (
|
||||
setattr(AutotagConfig, "INSTRUCTIONS", saved[0]),
|
||||
setattr(AutotagConfig, "DEFAULT_INSTRUCTIONS", saved[1]),
|
||||
)
|
||||
)
|
||||
AutotagConfig.INSTRUCTIONS = (
|
||||
{"id": "0123456789ab", "name": "Photo", "text": "Photo tags."},
|
||||
)
|
||||
AutotagConfig.DEFAULT_INSTRUCTIONS = {"gguf": "builtin", "hf": "0123456789ab"}
|
||||
for model_type, expected in (
|
||||
("hf", "Photo tags."),
|
||||
("gguf", AutotagConfig.default_instructions_text("gguf")),
|
||||
):
|
||||
self.image(f"{model_type}.png", (3, 3, len(model_type)))
|
||||
await self.relist()
|
||||
await self.legacy(f"model_type={model_type}&prompt=%20&skip_tagged=false")
|
||||
self.assertEqual(self.tagger.calls[-1]["prompt"], expected, model_type)
|
||||
self.assertEqual(self.tagger.custom_prompt, "legacy prompt")
|
||||
|
||||
async def test_a_missing_backend_package_is_one_error_frame(self):
|
||||
|
||||
@@ -197,7 +197,6 @@ class JobCase(unittest.TestCase):
|
||||
node.logger = logging.getLogger("test.cached_runs")
|
||||
node.prompt_tracker = mock.Mock()
|
||||
self.trackers.append(node.prompt_tracker)
|
||||
node.comfyui_integration = mock.Mock()
|
||||
node._inject_lora_trigger_words = lambda text: text
|
||||
node.pending_first_use = self.pending
|
||||
node.pending_pair_first_use = self.pair_pending
|
||||
|
||||
@@ -1,153 +0,0 @@
|
||||
"""Tests for utils.comfyui_integration."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from utils.comfyui_integration import ComfyUIMetadataIntegration
|
||||
|
||||
|
||||
class TestSaveImageNotPatched(unittest.TestCase):
|
||||
"""Regression for #75: SaveImage must save each node's own prompt inputs.
|
||||
|
||||
The old SaveImage patch wrote the last-run PromptManager prompt into every
|
||||
PromptManager node, so negative prompts were saved as copies of the positive.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
def save_images(
|
||||
self_node,
|
||||
images,
|
||||
filename_prefix="ComfyUI",
|
||||
prompt=None,
|
||||
extra_pnginfo=None,
|
||||
):
|
||||
return {"prompt": prompt}
|
||||
|
||||
self.original_save_images = save_images
|
||||
fake_nodes = types.ModuleType("nodes")
|
||||
fake_nodes.SaveImage = type("SaveImage", (), {"save_images": save_images})
|
||||
self._saved_nodes = sys.modules.get("nodes")
|
||||
sys.modules["nodes"] = fake_nodes
|
||||
self.fake_nodes = fake_nodes
|
||||
ComfyUIMetadataIntegration._instance = None
|
||||
|
||||
def tearDown(self):
|
||||
ComfyUIMetadataIntegration._instance = None
|
||||
if self._saved_nodes is None:
|
||||
sys.modules.pop("nodes", None)
|
||||
else:
|
||||
sys.modules["nodes"] = self._saved_nodes
|
||||
|
||||
def test_save_images_is_left_untouched(self):
|
||||
ComfyUIMetadataIntegration()
|
||||
self.assertIs(self.fake_nodes.SaveImage.save_images, self.original_save_images)
|
||||
|
||||
def test_registered_prompt_does_not_rewrite_saved_metadata(self):
|
||||
integration = ComfyUIMetadataIntegration()
|
||||
integration.register_prompt("pm_1", "positive final text", {})
|
||||
prompt = {
|
||||
"39": {"class_type": "PromptManager", "inputs": {"text": "a cat"}},
|
||||
"40": {"class_type": "PromptManager", "inputs": {"text": "blurry"}},
|
||||
}
|
||||
|
||||
result = self.fake_nodes.SaveImage().save_images([], prompt=prompt)
|
||||
|
||||
self.assertEqual(result["prompt"]["39"]["inputs"]["text"], "a cat")
|
||||
self.assertEqual(result["prompt"]["40"]["inputs"]["text"], "blurry")
|
||||
|
||||
def test_prompt_registry_still_available_to_nodes(self):
|
||||
integration = ComfyUIMetadataIntegration()
|
||||
integration.register_prompt("pm_1", "hello", {})
|
||||
self.assertEqual(integration.get_current_prompt_text("pm_1"), "hello")
|
||||
|
||||
|
||||
import threading # noqa: E402
|
||||
import unittest.mock as mock # noqa: E402
|
||||
|
||||
from utils import comfyui_integration # noqa: E402
|
||||
from utils.comfyui_integration import get_comfyui_integration # noqa: E402
|
||||
|
||||
|
||||
class IntegrationTestCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
ComfyUIMetadataIntegration._instance = None
|
||||
comfyui_integration._integration_instance = None
|
||||
# Patch the module's own ``time`` name rather than ``time.time`` globally:
|
||||
# background threads from other tests (image monitor, watchdog) would
|
||||
# otherwise tick this clock and make fresh registrations look stale.
|
||||
self.clock = mock.patch.object(
|
||||
comfyui_integration, "time", types.SimpleNamespace(time=self._now)
|
||||
)
|
||||
self.clock.start()
|
||||
self.addCleanup(self.clock.stop)
|
||||
self._ticks = 1000.0
|
||||
self.integration = get_comfyui_integration()
|
||||
|
||||
def tearDown(self):
|
||||
ComfyUIMetadataIntegration._instance = None
|
||||
comfyui_integration._integration_instance = None
|
||||
|
||||
def _now(self):
|
||||
self._ticks += 1.0
|
||||
return self._ticks
|
||||
|
||||
|
||||
class TestSingletonAndRegistry(IntegrationTestCase):
|
||||
def test_singleton_keeps_registrations_across_constructions(self):
|
||||
self.assertIs(get_comfyui_integration(), self.integration)
|
||||
self.assertIs(ComfyUIMetadataIntegration(), self.integration)
|
||||
self.integration.register_prompt("n1", "hello", {"category": "x"})
|
||||
ComfyUIMetadataIntegration() # __init__ runs again: must not reset state
|
||||
self.assertEqual(self.integration.get_current_prompt_text("n1"), "hello")
|
||||
|
||||
def test_latest_prompt_of_this_thread_wins_without_a_node_id(self):
|
||||
self.integration.register_prompt("n1", "first", {})
|
||||
self.integration.register_prompt("n2", "second", {})
|
||||
self.assertEqual(self.integration.get_current_prompt_text(), "second")
|
||||
self.assertEqual(self.integration.get_current_prompt_text("missing"), "second")
|
||||
self.assertEqual(self.integration.get_current_prompt_text("n1"), "first")
|
||||
|
||||
def test_nothing_registered_returns_none(self):
|
||||
self.assertIsNone(self.integration.get_current_prompt_text())
|
||||
self.assertIsNone(self.integration.get_current_prompt_text("n1"))
|
||||
|
||||
|
||||
class TestCrossThreadLookup(IntegrationTestCase):
|
||||
def _register_in_thread(self, text):
|
||||
worker = threading.Thread(
|
||||
target=self.integration.register_prompt, args=("w1", text, {})
|
||||
)
|
||||
worker.start()
|
||||
worker.join()
|
||||
|
||||
def test_recent_prompt_from_another_thread_is_used(self):
|
||||
self._register_in_thread("from worker")
|
||||
self.assertEqual(self.integration.get_current_prompt_text(), "from worker")
|
||||
|
||||
def test_stale_prompt_from_another_thread_is_ignored(self):
|
||||
self._register_in_thread("from worker")
|
||||
self._ticks += 400.0
|
||||
self.assertIsNone(self.integration.get_current_prompt_text())
|
||||
|
||||
def test_global_entry_of_this_thread_is_found_without_thread_local(self):
|
||||
self.integration.register_prompt("n1", "mine", {})
|
||||
self.integration._thread_local = threading.local()
|
||||
self.assertEqual(self.integration.get_current_prompt_text(), "mine")
|
||||
|
||||
def test_cleanup_removes_old_registrations_only(self):
|
||||
self._register_in_thread("old")
|
||||
self._ticks += 700.0
|
||||
self.integration.register_prompt("n2", "new", {})
|
||||
self.integration.cleanup_old_prompts(max_age_seconds=600)
|
||||
texts = sorted(v["text"] for v in self.integration._current_prompts.values())
|
||||
self.assertEqual(texts, ["new"])
|
||||
self.integration.cleanup_old_prompts(max_age_seconds=600) # nothing to do
|
||||
self.assertEqual(len(self.integration._current_prompts), 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+44
-75
@@ -213,7 +213,8 @@ class TestPromptManagerConfig(unittest.TestCase):
|
||||
try:
|
||||
# Modify a value
|
||||
PromptManagerConfig.MAX_SEARCH_RESULTS = 42
|
||||
PromptManagerConfig.save_to_file(tmp.name)
|
||||
with open(tmp.name, "w", encoding="utf-8") as handle:
|
||||
json.dump(PromptManagerConfig.get_config(), handle)
|
||||
|
||||
# Reset and reload
|
||||
PromptManagerConfig.MAX_SEARCH_RESULTS = 100
|
||||
@@ -240,55 +241,6 @@ class TestPromptManagerConfig(unittest.TestCase):
|
||||
finally:
|
||||
os.unlink(tmp.name)
|
||||
|
||||
def test_save_creates_directory(self):
|
||||
tmp_dir = tempfile.mkdtemp()
|
||||
nested_path = os.path.join(tmp_dir, "subdir", "config.json")
|
||||
try:
|
||||
PromptManagerConfig.save_to_file(nested_path)
|
||||
self.assertTrue(os.path.exists(nested_path))
|
||||
with open(nested_path) as f:
|
||||
data = json.load(f)
|
||||
self.assertIn("database", data)
|
||||
finally:
|
||||
import shutil
|
||||
|
||||
shutil.rmtree(tmp_dir)
|
||||
|
||||
def test_saved_file_is_private_to_the_user(self):
|
||||
import stat
|
||||
|
||||
tmp_dir = tempfile.mkdtemp()
|
||||
try:
|
||||
path = os.path.join(tmp_dir, "config.json")
|
||||
PromptManagerConfig.save_to_file(path)
|
||||
mode = stat.S_IMODE(os.stat(path).st_mode)
|
||||
if os.name == "posix":
|
||||
self.assertEqual(mode, 0o600)
|
||||
with open(path) as f:
|
||||
self.assertIn("database", json.load(f))
|
||||
# Overwriting an existing, more open file tightens it as well
|
||||
os.chmod(path, 0o644)
|
||||
PromptManagerConfig.save_to_file(path)
|
||||
if os.name == "posix":
|
||||
self.assertEqual(stat.S_IMODE(os.stat(path).st_mode), 0o600)
|
||||
finally:
|
||||
import shutil
|
||||
|
||||
shutil.rmtree(tmp_dir, ignore_errors=True)
|
||||
|
||||
def test_saved_file_is_valid_json(self):
|
||||
tmp = tempfile.NamedTemporaryFile(
|
||||
delete=False, suffix=".json", dir=tempfile.gettempdir()
|
||||
)
|
||||
tmp.close()
|
||||
try:
|
||||
PromptManagerConfig.save_to_file(tmp.name)
|
||||
with open(tmp.name) as f:
|
||||
data = json.load(f)
|
||||
self.assertIsInstance(data, dict)
|
||||
finally:
|
||||
os.unlink(tmp.name)
|
||||
|
||||
|
||||
class TestIntegrationConfig(unittest.TestCase):
|
||||
"""Test IntegrationConfig for LoRA Manager settings."""
|
||||
@@ -404,18 +356,6 @@ class TestConfigPathOverride(unittest.TestCase):
|
||||
os.environ[self.ENV] = custom
|
||||
self.assertEqual(PromptManagerConfig.get_config_path(), custom)
|
||||
|
||||
def test_save_without_path_uses_env_override(self):
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
custom = os.path.join(tmpdir, "nested", "custom.json")
|
||||
os.environ[self.ENV] = custom
|
||||
PromptManagerConfig.RESULT_TIMEOUT = 42
|
||||
|
||||
PromptManagerConfig.save_to_file()
|
||||
|
||||
with open(custom) as f:
|
||||
saved = json.load(f)
|
||||
self.assertEqual(saved["web_ui"]["result_timeout"], 42)
|
||||
|
||||
def test_load_without_path_uses_env_override(self):
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
custom = os.path.join(tmpdir, "custom.json")
|
||||
@@ -442,8 +382,11 @@ class TestAutotagConfig(unittest.TestCase):
|
||||
"EXISTING",
|
||||
"GENERAL_THRESHOLD",
|
||||
"CHARACTER_THRESHOLD",
|
||||
"TEMPLATE",
|
||||
"SUGGESTION_LIMIT",
|
||||
"SKIPPED_TAGS",
|
||||
"INSTRUCTIONS",
|
||||
"DEFAULT_INSTRUCTIONS",
|
||||
"_LEGACY_TEMPLATE",
|
||||
)
|
||||
}
|
||||
|
||||
@@ -487,22 +430,21 @@ class TestAutotagConfig(unittest.TestCase):
|
||||
"general_threshold": 0.35,
|
||||
"character_threshold": 0.85,
|
||||
"suggestion_limit": 10,
|
||||
"skipped_tags": [],
|
||||
"instructions": [],
|
||||
"default_instructions": {"gguf": "builtin", "hf": "builtin"},
|
||||
}
|
||||
},
|
||||
)
|
||||
# The built-in template is not written; a saved one is.
|
||||
self.config_class.TEMPLATE = "Saved."
|
||||
self.assertEqual(
|
||||
PromptManagerConfig.get_config()["autotag"]["defaults"]["template"],
|
||||
"Saved.",
|
||||
)
|
||||
self.config_class.TEMPLATE = config_module.DEFAULT_TAG_TEMPLATE
|
||||
# #431: the old template key is written only as the GGUF default's
|
||||
# mirror (test_saved_instructions covers it).
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
path = os.path.join(tmpdir, "config.json")
|
||||
PromptManagerConfig.update_config(
|
||||
{"autotag": {"defaults": {"suggestion_limit": 25}}}
|
||||
)
|
||||
PromptManagerConfig.save_to_file(path)
|
||||
with open(path, "w", encoding="utf-8") as handle:
|
||||
json.dump(PromptManagerConfig.get_config(), handle)
|
||||
self.config_class.SUGGESTION_LIMIT = 10
|
||||
PromptManagerConfig.load_from_file(path)
|
||||
self.assertEqual(self.config_class.suggestion_limit(), 25)
|
||||
@@ -524,16 +466,16 @@ class TestAutotagConfig(unittest.TestCase):
|
||||
"existing": "merge",
|
||||
"general_threshold": 0.4,
|
||||
"character_threshold": 0.7,
|
||||
"template": "Tags only.",
|
||||
"suggestion_limit": 15,
|
||||
"skipped_tags": ("blurry", "closed eyes"),
|
||||
}
|
||||
fallback = {
|
||||
"model_type": "wd14-swinv2",
|
||||
"existing": "skip",
|
||||
"general_threshold": 0.35,
|
||||
"character_threshold": 0.85,
|
||||
"template": config_module.DEFAULT_TAG_TEMPLATE,
|
||||
"suggestion_limit": 10,
|
||||
"skipped_tags": (),
|
||||
}
|
||||
bad_values = {
|
||||
"model_type": ["wd14-convnext", 3, None],
|
||||
@@ -544,6 +486,8 @@ class TestAutotagConfig(unittest.TestCase):
|
||||
# 4000 characters are at most 16,000 UTF-8 bytes, so the 16 KiB
|
||||
# byte cap can only be met past the character cap.
|
||||
"template": ["", " ", "x" * 4001, "\U0001f600" * 4097, "\ud800", 7],
|
||||
# #431: the whole list falls back; entries are never dropped one by one.
|
||||
"skipped_tags": ["blurry", ["a,b"], ["sky", "Sky"], [""], ["x"] * 201],
|
||||
}
|
||||
for key, values in bad_values.items():
|
||||
for value in values:
|
||||
@@ -552,7 +496,7 @@ class TestAutotagConfig(unittest.TestCase):
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
path = os.path.join(tmpdir, "config.json")
|
||||
with open(path, "w") as f:
|
||||
json.dump({"autotag": {"defaults": saved}}, f)
|
||||
json.dump({"autotag": {"defaults": saved}}, f, default=list)
|
||||
with self.assertLogs(
|
||||
"prompt_manager.config", level="WARNING"
|
||||
) as logs:
|
||||
@@ -563,7 +507,32 @@ class TestAutotagConfig(unittest.TestCase):
|
||||
k: getattr(self.config_class, attr)
|
||||
for k, (attr, _v, _d) in self.config_class._KEYS.items()
|
||||
}
|
||||
self.assertEqual(loaded, dict(good, **{key: fallback[key]}))
|
||||
# The old template is no attribute any more (#431): a bad
|
||||
# one is only warned about and imports nothing.
|
||||
expected = (
|
||||
dict(good, **{key: fallback[key]}) if key in fallback else good
|
||||
)
|
||||
self.assertEqual(loaded, expected)
|
||||
|
||||
def test_skipped_tags_load_trimmed_and_warn_once_when_damaged(self):
|
||||
"""#431: the list is kept as typed (trimmed); a damaged list is
|
||||
replaced by no skipped tags with one warning naming the reason."""
|
||||
PromptManagerConfig.update_config(
|
||||
{"autotag": {"defaults": {"skipped_tags": [" blurry ", "(fate)"]}}}
|
||||
)
|
||||
self.assertEqual(self.config_class.SKIPPED_TAGS, ("blurry", "(fate)"))
|
||||
with self.assertLogs("prompt_manager.config", level="WARNING") as logs:
|
||||
PromptManagerConfig.update_config(
|
||||
{"autotag": {"defaults": {"skipped_tags": ["sky", "a,b"]}}}
|
||||
)
|
||||
self.assertEqual(self.config_class.SKIPPED_TAGS, ())
|
||||
self.assertEqual(
|
||||
[record.getMessage() for record in logs.records],
|
||||
[
|
||||
"autotag.defaults.skipped_tags entry 2 must not contain a comma"
|
||||
" or a line break; using no skipped tags"
|
||||
],
|
||||
)
|
||||
|
||||
def test_tagging_validators_moved_with_their_messages(self):
|
||||
validate_threshold = config_module.validate_threshold
|
||||
|
||||
@@ -15,12 +15,11 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
@patch("prompt_manager_base.get_image_monitor")
|
||||
@patch("prompt_manager_base.get_prompt_tracker")
|
||||
@patch("prompt_manager_base.get_comfyui_integration")
|
||||
@patch("prompt_manager_base.PromptDatabase")
|
||||
class TestPromptManagerTextEmptyInput(unittest.TestCase):
|
||||
"""Test PromptManagerText with empty/blank text inputs (issue #123)."""
|
||||
|
||||
def _make_node(self, mock_db_class, mock_integration, mock_tracker, mock_monitor):
|
||||
def _make_node(self, mock_db_class, mock_tracker, mock_monitor):
|
||||
mock_db = Mock()
|
||||
mock_db_class.return_value = mock_db
|
||||
mock_db.save_prompt.return_value = 1
|
||||
@@ -30,73 +29,64 @@ class TestPromptManagerTextEmptyInput(unittest.TestCase):
|
||||
|
||||
return PromptManagerText()
|
||||
|
||||
def test_empty_string(self, mock_db, mock_integ, mock_tracker, mock_monitor):
|
||||
def test_empty_string(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Empty string should not raise UnboundLocalError."""
|
||||
node = self._make_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(text="")
|
||||
self.assertIsInstance(result, tuple)
|
||||
self.assertEqual(result[0], "")
|
||||
|
||||
def test_whitespace_only(self, mock_db, mock_integ, mock_tracker, mock_monitor):
|
||||
def test_whitespace_only(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Whitespace-only string should not raise UnboundLocalError."""
|
||||
node = self._make_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(text=" ")
|
||||
self.assertIsInstance(result, tuple)
|
||||
|
||||
def test_none_coerced_empty(self, mock_db, mock_integ, mock_tracker, mock_monitor):
|
||||
def test_none_coerced_empty(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Falsy text value should not raise UnboundLocalError."""
|
||||
node = self._make_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_node(mock_db, mock_tracker, mock_monitor)
|
||||
# ComfyUI may pass empty string for unconnected inputs
|
||||
result = node.process_text(text="", category="", tags="")
|
||||
self.assertIsInstance(result, tuple)
|
||||
|
||||
def test_empty_text_with_prepend_append(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_empty_text_with_prepend_append(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Empty main text with prepend/append should produce single space, not
|
||||
double."""
|
||||
node = self._make_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(text="", prepend_text="before", append_text="after")
|
||||
self.assertIsInstance(result, tuple)
|
||||
self.assertEqual(result[0], "before after")
|
||||
self.assertNotIn(" ", result[0])
|
||||
|
||||
def test_valid_text_still_works(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_valid_text_still_works(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Normal text input should continue to work correctly."""
|
||||
node = self._make_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(text="a beautiful sunset")
|
||||
self.assertIsInstance(result, tuple)
|
||||
self.assertEqual(result[0], "a beautiful sunset")
|
||||
|
||||
def test_prepend_text_append_all_present(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_prepend_text_append_all_present(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""All three parts should combine with single spaces."""
|
||||
node = self._make_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(
|
||||
text="main prompt", prepend_text="before", append_text="after"
|
||||
)
|
||||
self.assertEqual(result[0], "before main prompt after")
|
||||
|
||||
def test_only_prepend(self, mock_db, mock_integ, mock_tracker, mock_monitor):
|
||||
def test_only_prepend(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Prepend with empty text should output just prepend."""
|
||||
node = self._make_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(text="", prepend_text="before")
|
||||
self.assertEqual(result[0], "before")
|
||||
|
||||
|
||||
@patch("prompt_manager_base.get_image_monitor")
|
||||
@patch("prompt_manager_base.get_prompt_tracker")
|
||||
@patch("prompt_manager_base.get_comfyui_integration")
|
||||
@patch("prompt_manager_base.PromptDatabase")
|
||||
class TestPromptManagerEmptyInput(unittest.TestCase):
|
||||
"""Test PromptManager (CLIP variant) with empty/blank text inputs (issue #123)."""
|
||||
|
||||
def _make_node_and_clip(
|
||||
self, mock_db_class, mock_integration, mock_tracker, mock_monitor
|
||||
):
|
||||
def _make_node_and_clip(self, mock_db_class, mock_tracker, mock_monitor):
|
||||
mock_db = Mock()
|
||||
mock_db_class.return_value = mock_db
|
||||
mock_db.save_prompt.return_value = 1
|
||||
@@ -110,30 +100,22 @@ class TestPromptManagerEmptyInput(unittest.TestCase):
|
||||
|
||||
return PromptManager(), mock_clip
|
||||
|
||||
def test_empty_string(self, mock_db, mock_integ, mock_tracker, mock_monitor):
|
||||
def test_empty_string(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Empty string should not raise UnboundLocalError."""
|
||||
node, clip = self._make_node_and_clip(
|
||||
mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
)
|
||||
node, clip = self._make_node_and_clip(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.encode_prompt(clip=clip, text="")
|
||||
self.assertIsInstance(result, tuple)
|
||||
|
||||
def test_whitespace_only(self, mock_db, mock_integ, mock_tracker, mock_monitor):
|
||||
def test_whitespace_only(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Whitespace-only string should not raise UnboundLocalError."""
|
||||
node, clip = self._make_node_and_clip(
|
||||
mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
)
|
||||
node, clip = self._make_node_and_clip(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.encode_prompt(clip=clip, text=" ")
|
||||
self.assertIsInstance(result, tuple)
|
||||
|
||||
def test_empty_text_with_prepend_append(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_empty_text_with_prepend_append(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Empty main text with prepend/append should produce single space, not
|
||||
double."""
|
||||
node, clip = self._make_node_and_clip(
|
||||
mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
)
|
||||
node, clip = self._make_node_and_clip(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.encode_prompt(
|
||||
clip=clip, text="", prepend_text="before", append_text="after"
|
||||
)
|
||||
@@ -141,24 +123,16 @@ class TestPromptManagerEmptyInput(unittest.TestCase):
|
||||
self.assertEqual(result[1], "before after")
|
||||
self.assertNotIn(" ", result[1])
|
||||
|
||||
def test_valid_text_still_works(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_valid_text_still_works(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Normal text input should continue to work correctly."""
|
||||
node, clip = self._make_node_and_clip(
|
||||
mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
)
|
||||
node, clip = self._make_node_and_clip(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.encode_prompt(clip=clip, text="a beautiful sunset")
|
||||
self.assertIsInstance(result, tuple)
|
||||
self.assertEqual(result[1], "a beautiful sunset")
|
||||
|
||||
def test_prepend_text_append_all_present(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_prepend_text_append_all_present(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""All three parts should combine with single spaces."""
|
||||
node, clip = self._make_node_and_clip(
|
||||
mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
)
|
||||
node, clip = self._make_node_and_clip(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.encode_prompt(
|
||||
clip=clip, text="main prompt", prepend_text="before", append_text="after"
|
||||
)
|
||||
@@ -167,7 +141,6 @@ class TestPromptManagerEmptyInput(unittest.TestCase):
|
||||
|
||||
@patch("prompt_manager_base.get_image_monitor")
|
||||
@patch("prompt_manager_base.get_prompt_tracker")
|
||||
@patch("prompt_manager_base.get_comfyui_integration")
|
||||
@patch("prompt_manager_base.PromptDatabase")
|
||||
class TestConnectedTextInput(unittest.TestCase):
|
||||
"""Test that externally connected text is preserved in output (issue #123).
|
||||
@@ -177,7 +150,7 @@ class TestConnectedTextInput(unittest.TestCase):
|
||||
value reaches Python; these tests verify Python handles it correctly.
|
||||
"""
|
||||
|
||||
def _make_text_node(self, mock_db_class, mock_integ, mock_tracker, mock_monitor):
|
||||
def _make_text_node(self, mock_db_class, mock_tracker, mock_monitor):
|
||||
mock_db = Mock()
|
||||
mock_db_class.return_value = mock_db
|
||||
mock_db.save_prompt.return_value = 1
|
||||
@@ -186,7 +159,7 @@ class TestConnectedTextInput(unittest.TestCase):
|
||||
|
||||
return PromptManagerText()
|
||||
|
||||
def _make_clip_node(self, mock_db_class, mock_integ, mock_tracker, mock_monitor):
|
||||
def _make_clip_node(self, mock_db_class, mock_tracker, mock_monitor):
|
||||
mock_db = Mock()
|
||||
mock_db_class.return_value = mock_db
|
||||
mock_db.save_prompt.return_value = 1
|
||||
@@ -199,10 +172,10 @@ class TestConnectedTextInput(unittest.TestCase):
|
||||
return PromptManager(), mock_clip
|
||||
|
||||
def test_text_node_connected_text_with_prepend_append(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
self, mock_db, mock_tracker, mock_monitor
|
||||
):
|
||||
"""Simulates screenshot: connected text + prepend + append must all appear."""
|
||||
node = self._make_text_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_text_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(
|
||||
text="This is the prompt",
|
||||
prepend_text="This is prepend text",
|
||||
@@ -213,21 +186,17 @@ class TestConnectedTextInput(unittest.TestCase):
|
||||
"This is prepend text This is the prompt This is append text",
|
||||
)
|
||||
|
||||
def test_text_node_connected_text_only(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_text_node_connected_text_only(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Connected text without prepend/append passes through unchanged."""
|
||||
node = self._make_text_node(mock_db, mock_integ, mock_tracker, mock_monitor)
|
||||
node = self._make_text_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.process_text(text="This is the prompt")
|
||||
self.assertEqual(result[0], "This is the prompt")
|
||||
|
||||
def test_clip_node_connected_text_with_prepend_append(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
self, mock_db, mock_tracker, mock_monitor
|
||||
):
|
||||
"""Same scenario for PromptManager (CLIP variant)."""
|
||||
node, clip = self._make_clip_node(
|
||||
mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
)
|
||||
node, clip = self._make_clip_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.encode_prompt(
|
||||
clip=clip,
|
||||
text="This is the prompt",
|
||||
@@ -239,13 +208,9 @@ class TestConnectedTextInput(unittest.TestCase):
|
||||
"This is prepend text This is the prompt This is append text",
|
||||
)
|
||||
|
||||
def test_clip_node_connected_text_only(
|
||||
self, mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
):
|
||||
def test_clip_node_connected_text_only(self, mock_db, mock_tracker, mock_monitor):
|
||||
"""Connected text without prepend/append passes through unchanged."""
|
||||
node, clip = self._make_clip_node(
|
||||
mock_db, mock_integ, mock_tracker, mock_monitor
|
||||
)
|
||||
node, clip = self._make_clip_node(mock_db, mock_tracker, mock_monitor)
|
||||
result = node.encode_prompt(clip=clip, text="This is the prompt")
|
||||
self.assertEqual(result[1], "This is the prompt")
|
||||
|
||||
|
||||
@@ -324,7 +324,6 @@ class CaptureTests(unittest.TestCase):
|
||||
node.db.save_prompt_pair.side_effect = RuntimeError("offline")
|
||||
node.logger = logging.getLogger("test.capture.nodes")
|
||||
node.prompt_tracker = Mock()
|
||||
node.comfyui_integration = Mock()
|
||||
node._inject_lora_trigger_words = lambda text: text + " TRIGGER"
|
||||
clip = Mock()
|
||||
clip.tokenize.side_effect = lambda text: text
|
||||
|
||||
@@ -24,7 +24,6 @@ class TestFocusedNode(unittest.TestCase):
|
||||
def test_focused_encodes_composition_and_tracks_main_text_and_graph(self):
|
||||
node = PromptManagerFocused.__new__(PromptManagerFocused)
|
||||
node.logger = logging.getLogger("test.focused")
|
||||
node.comfyui_integration = mock.Mock()
|
||||
node._inject_lora_trigger_words = lambda text: text
|
||||
node._track_prompt_execution = mock.Mock(return_value=42)
|
||||
clip = mock.Mock()
|
||||
|
||||
+1
-29
@@ -1,7 +1,7 @@
|
||||
"""
|
||||
Tests for hashing utilities.
|
||||
|
||||
Tests is_duplicate_prompt and generate_prompt_hash, which are
|
||||
Tests and generate_prompt_hash, which are
|
||||
currently untested. generate_prompt_hash is covered in test_basic.py
|
||||
but we add edge cases here.
|
||||
"""
|
||||
@@ -15,7 +15,6 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from tests.background_stops import stop_tracker # noqa: E402
|
||||
from utils.hashing import (
|
||||
generate_prompt_hash,
|
||||
is_duplicate_prompt,
|
||||
STRIPPED_CODE_POINTS,
|
||||
prompt_key,
|
||||
register_prompt_functions,
|
||||
@@ -69,33 +68,6 @@ class TestSingleHashImplementation(unittest.TestCase):
|
||||
self.assertFalse(os.path.exists(os.path.join(ROOT, "restart_gallery.py")))
|
||||
|
||||
|
||||
class TestIsDuplicatePrompt(unittest.TestCase):
|
||||
"""Test is_duplicate_prompt function."""
|
||||
|
||||
def test_identical_texts_are_duplicates(self):
|
||||
self.assertTrue(is_duplicate_prompt("hello", "hello"))
|
||||
|
||||
def test_case_insensitive(self):
|
||||
self.assertTrue(is_duplicate_prompt("Hello World", "hello world"))
|
||||
|
||||
def test_whitespace_insensitive(self):
|
||||
self.assertTrue(is_duplicate_prompt(" hello ", "hello"))
|
||||
|
||||
def test_different_texts_not_duplicates(self):
|
||||
self.assertFalse(is_duplicate_prompt("text A", "text B"))
|
||||
|
||||
def test_threshold_parameter_accepted(self):
|
||||
# threshold is unused but should not cause errors
|
||||
self.assertTrue(is_duplicate_prompt("same", "same", threshold=0.5))
|
||||
|
||||
def test_empty_strings_are_duplicates(self):
|
||||
self.assertTrue(is_duplicate_prompt("", ""))
|
||||
|
||||
def test_unicode_comparison(self):
|
||||
self.assertTrue(is_duplicate_prompt("日本語", "日本語"))
|
||||
self.assertFalse(is_duplicate_prompt("日本語", "中文"))
|
||||
|
||||
|
||||
class TestGeneratePromptHashEdgeCases(unittest.TestCase):
|
||||
"""Additional edge cases for generate_prompt_hash."""
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
# Mock ComfyUI server before importing anything that touches config. Only the
|
||||
# first test module to run installs the stub; nothing else in sys.modules is
|
||||
# replaced, so later modules (test_comfyui_integration) see the real package.
|
||||
# replaced, so later modules see the real package.
|
||||
_mock_server = MagicMock()
|
||||
_mock_server.PromptServer.instance.routes = MagicMock()
|
||||
sys.modules.setdefault("server", _mock_server)
|
||||
@@ -881,17 +881,6 @@ class TestOutputListener(unittest.TestCase):
|
||||
self.assertTrue(monitor.stop_monitoring())
|
||||
|
||||
|
||||
class TestModuleIsolation(unittest.TestCase):
|
||||
"""This module must not leave mocks behind for later test modules."""
|
||||
|
||||
def test_comfyui_integration_is_not_replaced_by_a_mock(self):
|
||||
module = sys.modules.get("utils.comfyui_integration")
|
||||
self.assertFalse(
|
||||
isinstance(module, MagicMock),
|
||||
"utils.comfyui_integration was replaced by a MagicMock in sys.modules",
|
||||
)
|
||||
|
||||
|
||||
class TestMonitorThreadSafety(unittest.TestCase):
|
||||
"""Singleton creation must be thread-safe."""
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user