Compare commits

..
Author SHA1 Message Date
Vito Sansevero d0bfed3614 Merge remote-tracking branch 'origin/main' into fix/430-stale-scope
# Conflicts:
#	web/js/workspace/images.mjs
2026-10-28 15:38:26 -07:00
Vito 56d0a4904c feat(collections): COLLECTIONS sidebar, collection page and saved views (#331, PR 2) (#410)
* feat(saved-views): keep a collection in a saved view

Images saved views gain a collection key (#331 D140): empty or a collection
id of 1 to 9 digits. Writes refuse favorites together with a collection;
older rows read an empty collection, and a stored row with both opens the
collection, as the listing does for a URL with both.

Part of #331

* feat(collections): store collections in the library database

* feat(collections): add, remove and rename collections and keep members through duplicate cleanup

* feat(collections): add collection routes and the Images collection filter

* feat(images): store images' own categories in the library database (#332)

One new table, image_categories, and its category index, created empty
by a forward-only migration that reads and writes only its own objects.
Startup checks the structure; restore also checks the values. The store
sets, clears and reads own categories by file key (the deepest output
source holding the file and the path below it), and copies a removed
duplicate's category to a kept copy that has none.

* feat(images): duplicate cleanup keeps images' own categories (#332)

The kept copy's own category wins; a kept copy without one receives the
removed copy's, keyed by the deepest output source holding each file.
The removed copy's row is never deleted, so a give-back or a failed
delete loses nothing. The job log reports how many were kept.

* feat(images): set, remove and list images' own categories (#332)

PUT /prompt_manager/images/category sets the own category of up to 500
listed files, DELETE removes it (listed or not), and
GET /prompt_manager/images/categories lists the values whose file is
listed. Targets are the Images listing identity of an output source,
checked by media_target (shared with Collections). The routes never
touch prompts, recipes or links; POST /prompt_manager/bulk/category is
unchanged.

* fix(images): key symlinked output folders and keep key work off the event loop (#332)

Review fixes: a source matches by its configured spelling too, so a
detected output folder that is a symlink keys its files (routes and
duplicate cleanup). File keys are computed in the executor. A path no
source holds is a plain 403, a lone surrogate a 400. The install repairs
a table that lacks its category index instead of refusing to open.

* fix(collections): address review findings

* feat(images): set or remove a category on all N matching images (#332)

The category routes also accept the Images listing filter of "Select
all N matching" (#392) instead of explicit items, resolved now and
capped at 500 images like an explicit selection.

* feat(collections): browse a collection from the sidebar

The sidebar gains the concept's COLLECTIONS group (D136): one row per
collection with its folder icon, name and listed-member count, without the
PROPOSED badge; the label shows once a collection exists. A row opens
#/collection/<id>, the Images page filtered to that collection (D137): the
collection's name as h1, Collection as breadcrumb and title, Edit to rename
or delete it through the saved views' Edit dialog with collection nouns, and
a note counting members the sources do not list now. An unknown or deleted
collection shows a page that says so with a link to Images. A saved view on
a deleted collection lists nothing and says so (MC6, D140). The list is read
again after a restore and the counts 300 ms after a change.

Part of #331

* test(collections): compare the COLLECTIONS group with the concept

The concept hides COLLECTIONS at 950 px and below, which would leave a phone
no way to open a collection; production shows the rows there in the wrapped
layout of every other sidebar nav (D136). The shell comparison now compares
the group instead of filtering it out: the concept fixture seeds Landscapes
(12) and Experiments (8), the rows compare by name order, and at phone width
the nav compares against the concept's own sidebar-nav layout. The
collection rows are checked at 1600x1100 and 390x844 in both themes.

Part of #331

* test(vendor): count the #332 host case in the execution matrix

image-attribution.spec.mjs gains the image category route case, so the
execution config lists 92 tests (3 attribution cases per project).

* test(browser): install the slideshow clock before wrapping setTimeout

The slideshow fixture wrapped window.setTimeout before page.clock.install(),
so the viewer's timer ran on real time and clock.runFor() did not drive it;
a late WebKit timer failed 'slideshow exposes exact state labels' on CI.

* fix(collections): address review findings

- A list read superseded by a newer one (the 300 ms count read, a restore)
  no longer shows "could not be loaded": the collection and saved-view
  pages wait for the list to settle.
- A listing that finds the open collection gone shows the deleted page once
  the list no longer holds it; when the list read fails or still holds it,
  the page keeps the listing's error and Retry instead of staying blank, and
  each failed read is reported again.
- The missing-members note follows every list read and goes away at zero.
- A rename that settles after its Edit dialog closed leaves any dialog
  opened since alone.
- The deleted-collection, list-failed and saved-view failure pages focus
  their title after a route change, as every page does.

Part of #331

* test(coverage): keep main's floors for files this PR does not touch

coverage:update raised nine floors of untouched files from one run;
metadata-autocomplete.js branches then measured 84.84 against 85.

* test(legacy): record the image category routes in the legacy contract snapshot (#332)

#332 adds PUT/DELETE /prompt_manager/images/category and
GET /prompt_manager/images/categories; the legacy pages' routes are
unchanged.

* test(duplicates): a removed copy with a collection and a category moves both (#331, #332)
2026-10-28 21:45:09 +00:00
Vito Sansevero f8bc68b5d1 Merge remote-tracking branch 'origin/main' into fix/430-stale-scope 2026-10-28 14:33:14 -07:00
Vito Sansevero 4ff6d6f134 fix(workspace): keep open pages valid across a restart and recover a changed source (#430)
The gallery root scope and the Images listing cursor were keyed with a
per-process random key, so every ComfyUI restart made an open tab's
scopes answer 409 source-changed (blank Tag image preview, a Generate
that did nothing) and its cursors fail Load more.

- utils/workspace_keys.py: one random secret in workspace-scope.key
  beside the database (owner-only, never config.json or the database),
  two derived keys; per-process fallback with a warning. Move library
  copies it.
- Cursor v2 binds the listing scope in its MAC instead of sending an
  unkeyed hash of the root paths; an unverifiable cursor answers 400
  cursor-stale and Images offers Refresh instead of a dead Retry.
- POST /prompt_manager/images/rescope maps a stale target to its
  current root only when the held scope was issued for it.
- Images finds a renumbered file again from the inspector, the viewer
  and the Tag image dialog (the dialog stays open, draft kept); nothing
  else is kept by an old root_index:path key.
- Tag image dialog: a state that blocks Save shows its reason above the
  actions with Refresh, disabled actions look disabled, and an empty
  preview says "Preview unavailable" (D147).
2026-10-28 14:02:26 -07:00
Vito e86c0dcf05 fix(ci): write browser coverage without each script's source (#432) (#433)
Every raw browser coverage file repeated the full source of each mapped
script, about 84% of its bytes, and the frontend coverage merge ran out of
Node's heap at 1,884 files. c8 reads the source from the mapped file on disk,
so the entries keep only the URL, script id and functions.
2026-10-28 20:57:45 +00:00
Vito 39877202b2 feat(ingestion): one prompt decision for scan and monitor; scans link each dataset branch's own prompt (#247, PR 1) (#422)
* feat(inspector): use recorded run history when the graph alone can't name the prompt (#386)

When a file's own graph leaves the prompt open, both inspect routes ask
recorded run history about the file (one run reported it and it was written
during that run). The save node that run reported it from anchors the walk
like a file-name match, so dataset branches that share one saver prefix
show their own prompt, labelled "Recorded run". Still open at one sampler
whose PromptManager text was recorded at execution (a language model's
output), the recorded text shows as "Execution recorded". Results the file
resolves alone never change; a failed lookup judges the file alone.

* fix(inspector): recorded texts follow history's saver; conflicts anchor nothing (#386)

Review follow-ups: the texts PromptManager recorded are looked up at the
sampler history's saver leads to, never at a sampler the unanchored walk
named; a file name naming a different saver than history anchors nothing;
outputs and texts are read in one transaction; "Recorded run" ranks below
every workflow-inferred source; unexpected lookup errors log at error level.

* fix(history): look up recorded runs by history's own spelling of the file (#386)

The history route inspects a case-normalized target on Windows, while
history matches file names exactly, so the occurrence's reference names
the file.

* refactor(inspector): share the inspector's resolve as inspection_resolution

inspect_prompt_evidence now calls inspection_resolution, which returns the
resolution to use, the anchored result and whether the recorded saver chose
it. Ingestion can call the same helper for files the default profile cannot
decide (#247).

* feat(ingestion): one per-file prompt decision for the scan and the monitor

utils/file_prompt.py holds decide_file_prompt, the ladder the output scan
runs today plus the inspector's rule (inspection profile, then the recorded
saver) before the legacy parser, with a SCAN and a MONITOR policy. Junk
text (blank, numbers, LoRA tags only, model file names) is never stored.
graph_prompt_text and parse_comfyui_prompt move here; the parser accepts
parsed metadata (#247).

* feat(ingestion): store a file's prompt in one transaction, one prompt per file

database/scanned_prompts.py links a file to the prompt of one text, creating
it when new, inside one BEGIN IMMEDIATE transaction: a file linked to
another prompt is a conflict, a failed link leaves no prompt behind and two
writers store one prompt. pair_ingestion gains file_link_rows,
link_conflict and Marks/mark_created; ingest_resolved_pair takes optional
marks for the components it creates. recorded_runs and named_prompt in
run_attribution become public (#247).

* feat(scan): decide each scanned file with the shared ladder (#247)

_ingest_scanned_file calls decide_file_prompt (scan policy) and stores the
result: pairs through ingest_resolved_pair and positives through
store_scanned_prompt, both with the scan marks. Files the default profile
leaves undecided now get the inspector's resolve, with the saver recorded
history names, before the legacy parser, so each branch of a shared-prefix
dataset run links its own prompt. Junk text is never stored, a file linked
to another prompt is a conflict instead of a second link, and a failed link
leaves no prompt behind. _BatchHistory reads the stored prompt and the
recorded saver from one recorded_runs pass. _parse_comfyui_prompt calls the
moved parse_comfyui_prompt. TESTING.md documents the decision.

* fix(ingestion): keep the file-name saver anchor out of the ladder (#390)

The #390 ruling queues a file that only its saver's file name resolves for
Needs review, never linking it. inspection_resolution gains name_anchor;
the ladder passes False, so such a file falls to the legacy floor in the
scan (and to Needs review) and is never stored by the monitor, while the
recorded saver still decides. The inspector keeps the name anchor.

* fix(ingestion): no joined PromptManager text from the legacy floor; junk keeps its step's status

The legacy parser returns prepend + text + append, so its answer is refused
when a PromptManager-family node has a prepend or append part (#343). Junk
found at the undecided-graph step keeps that step's unlinked status and
review, so an ambiguous file is still queued for Needs review. The monitor
policy's ambiguous file no longer asks for the recorded run, and the file
size may be read lazily.

* fix(ingestion): keep the legacy answer that is a PromptManager node's own text

The joined-text refusal also refused the #170 parity files whose
PromptManager prepend is a link the browser parser does not follow: its
answer is then the node's literal text, which is what the node stores.
Refuse only an answer that is not such a node's own text.

* fix(scan): one prompt per file for recorded-run links; writes survive a library move

The recorded run's stored prompt now links in one transaction with the file's
link rows, so a file linked to another prompt is a conflict instead of a
second link. The scan's stores run through survive_library_switch. A single
file asks history once (a one-file _BatchHistory), the scan passes
in_output=False and reads the file size only for files with prompt
metadata. A conflict row reads "Kept: already linked to a different
prompt". TESTING.md records the rules.

* fix(ingestion): match a file's link rows under its resolved spelling too

The scan stores realpath spellings and the monitor the watcher's raw path,
so a file reached through a symlinked or junction output folder could get a
second generated_images row. file_link_rows now queries the path and its
realpath (each normalized and raw) and dedupes by row id. TESTING.md notes
the reverse order is matched only when the raw spelling is the realpath.

* fix(ingestion): a switched-off PromptManager node behind a pass-through stores nothing (#90)

The #90 graph walk follows only prompt-like inputs, so a PromptManagerText
with Save to library off reaching the encoder through another input name
(easy showAnything.anything, a custom node's value) was invisible to it,
and the inspector's resolve stored its text. The ladder now checks the
positive every step names: a resolved or inspected positive whose path
passes a node that is off, or a legacy answer that is such a node's own
text, makes the file not_saved.

* fix(ingestion): any switched-off node upstream of a positive stores nothing (#90)

The ladder walks every input upstream of each sampler or guider positive
(not negative, model, clip, latent or image inputs) for PromptManager nodes
whose Save to library is off or wired from an unreadable node, so the
sampler-less step, an off node with a prepend part and a PromptManagerPair
behind a pass-through no longer store its text. The legacy-floor guard reads
a node's resolved text. Every guard reports not_saved for a switch that is
off and not_saved_wired for one it cannot read.

* fix(ingestion): the switch walk skips negative inputs by #90's own rule

The upstream walk treated only names containing "negative" as negative, so
an off node on a neg, neg_text or text_neg input blocked an enabled
positive. prompt_graph gains is_negative_input (the _NEGATIVE_KEY rule, also
used by follows_input) and the walk uses it. The guards accept a node with
null inputs and a side whose provenance is not an object.

* fix(ingestion): the switch walk also skips any input named with "negative"

#90's rule matches only whole underscore-separated words, so negative2,
negativeprompt and NegativePrompt counted as positive inputs and an off
node wired only to one of them blocked the file. The walk skips those too;
follows_input is unchanged.
2026-10-28 20:45:15 +00:00
Vito 41027aa1b1 fix(autotag): report a missing backend package instead of offering the model as ready (#426)
Model status now says, per backend, which Python packages cannot be
imported (importlib.util.find_spec, nothing imported): llama-cpp-python
for GGUF, torch/transformers/accelerate/bitsandbytes for HuggingFace,
onnxruntime for WD14, with the pip install command and a note (the CUDA
CMAKE_ARGS for llama-cpp-python, onnxruntime-gpu for WD14). Nothing is
installed.

load_model refuses such a model before the loader with
MissingBackendPackages (one WARNING line, no traceback); a
ModuleNotFoundError from a loader becomes the same refusal. The suggest
route, the legacy single and batch routes and both tagging jobs answer
409 model-unavailable with that sentence.

System > Auto-tagging reads "Downloaded. Needs llama-cpp-python: pip
install llama-cpp-python. ..." instead of "Downloaded. Not loaded.",
Review explains instead of reviewing, and the bulk tagging dialog keeps
Start disabled with the sentence. "downloaded" keeps its files-only
meaning, so the legacy pages are unchanged.

Closes #424
2026-10-28 19:58:05 +00:00
Vito c7375573ed feat(images): set an image's own category, show its origin and count image categories (#332, PR 3) (#428)
* feat(images): store images' own categories in the library database (#332)

One new table, image_categories, and its category index, created empty
by a forward-only migration that reads and writes only its own objects.
Startup checks the structure; restore also checks the values. The store
sets, clears and reads own categories by file key (the deepest output
source holding the file and the path below it), and copies a removed
duplicate's category to a kept copy that has none.

* feat(images): duplicate cleanup keeps images' own categories (#332)

The kept copy's own category wins; a kept copy without one receives the
removed copy's, keyed by the deepest output source holding each file.
The removed copy's row is never deleted, so a give-back or a failed
delete loses nothing. The job log reports how many were kept.

* feat(images): set, remove and list images' own categories (#332)

PUT /prompt_manager/images/category sets the own category of up to 500
listed files, DELETE removes it (listed or not), and
GET /prompt_manager/images/categories lists the values whose file is
listed. Targets are the Images listing identity of an output source,
checked by media_target (shared with Collections). The routes never
touch prompts, recipes or links; POST /prompt_manager/bulk/category is
unchanged.

* feat(vocabulary): count and rename image categories (#332)

Tags & categories lists every category value with image_count, the
stored image_categories rows holding it. Values only images hold follow
every prompt-backed value (first_pid at SQLite's INTEGER maximum), so
keyset cursors cross the boundary. /library/options keeps its own query
and never lists image-only values.

Manage category counts image rows (impact.images), treats an image-held
target as existing, renames image rows in the same transaction, allows
renaming an image-only value, and hashes the sorted image rowids into
the impact token so a change between preview and apply is refused with
vocabulary-conflict. Every image-category statement reads
idx_image_categories_category.

* feat(images): the category filter matches each image's effective category (#332)

An image's own category wins; without one it inherits its link's: the
recipe's category when set, else the positive prompt's (MC4, one rule
with the inspector chip). Ambiguous and unlinked images inherit nothing,
so an unlinked image with its own category is now listed. The link read
no longer filters by category in SQL; the page filter, select all N
matching and saved views compare the effective value. A failed
own-category read answers the existing listing-failed 500.

* fix(images): key symlinked output folders and keep key work off the event loop (#332)

Review fixes: a source matches by its configured spelling too, so a
detected output folder that is a symlink keys its files (routes and
duplicate cleanup). File keys are computed in the executor. A path no
source holds is a plain 403, a lone surrogate a 400. The install repairs
a table that lacks its category index instead of refusing to open.

* feat(inspector): show an image's own category in the inspector and History (#332)

The inspector's library facts gain category_scope "image": an image's
own category overrides the inherited one for any link status (linked,
ambiguous, none). Images reads it for output sources only; History reads
it once per inspection inside its snapshot, keyed by the same file key
from the symlink-resolved output folder. A failed read hides the chip
rather than show a possibly wrong value. The chip's hidden prefix names
the origin ("Image category: "); the visible origin text comes in PR 3.

* perf(images): keep the category filter's link read narrow; record D141, D142, D145 (#332)

Under a category the link read keeps only links whose raw recipe or
positive category is the value (Python still decides the inherited
one), the links of images whose own category is the value are read by
path, and plain reads skip the inherited-category work. The own-category
map is keyed by normalized path. TESTING.md records D141, D142 and D145
and amends D13, D36 and D39; the filter budget seeds 2,000 own
categories.

* feat(images): set or remove a category on all N matching images (#332)

The category routes also accept the Images listing filter of "Select
all N matching" (#392) instead of explicit items, resolved now and
capped at 500 images like an explicit selection.

* fix(images): Windows path keys for own categories; plan check for the category read (#332)

Review fixes: on Windows the own-category map keys each file with
normalize_image_path (abspath drops trailing dots and spaces), and the
plan test also covers the category's narrowing clause.

* feat(images): Set category sets each image's own category; the chip names its origin (#332)

The selection bar's Set category writes the selected images' own
category (linked or not, up to 500, or all N matching up to 500), with
the toolbar categories as suggestions; Use prompt category removes it
after a confirmation that states the count. It never writes a prompt
(#259's prompt action is retired from Images; the Library editor and
the legacy admin Set Category still change prompt categories). The
toolbar lists the listed images' own categories beside the prompt and
recipe ones. The inspector chip shows its origin: "· this image",
"· prompt" or "· recipe". Tags & categories counts images and opens
Images for a category only images use.

* test(images): compare the category chip's visible origin against the concept (#332)

D146: the inspector comparison measures each chip's span.chip-source and
lays the concept's own chip out with the recorded rule and origin added at
runtime (withChipSource), in the side panel and the viewer's details. The
D39 recipe case asserts the visible " · prompt"; the mutation check
removes the origin.

* test(images): compare Set category and the category options against the concept (#332)

With one image's own category set through the real route, the toolbar's
#category-filter options are compared with D141 mapped (the image category
sorted in with the Library's, D36), and the Set category dialog against the
concept's simpleInput laid out as recorded at runtime: D140 (empty field,
datalist#workspace-input-suggestions of the toolbar's categories, Use prompt
category between Cancel and Apply) and D8 (Apply). Unmapped, the dialog
differs only in those parts.

* test(history): compare the History media inspector's category chip against the concept (#332)

D145: with an own category set on the baseline's newest output, the
Recorded run viewer's chip is compared with the concept viewer's category
chip laid out with the D146 origin and the own category at runtime; D21
maps its hidden "Image category: " prefix. Unmapped, the two differ only
in the text, the prefix, the width and span.chip-source.

* test(images): Set category sets each image's own category in page workflows (#332)

Replace the #259 prompt-category tests with CUJ 1, 2, 3, 8 and 10, the
suggestions datalist, the 500-image limit, the source-changed path, D142
and coded refusals; Images never posts /prompt_manager/bulk/category.

* test(inspector): the category chip names its origin in the panel, viewer and History (#332)

CUJ 4 and 5: "· this image", "· prompt" and "· recipe" chips, an
unlinked file's own category, and History's current-file inspector.

* test(vocabulary): category rename covers image categories; image-only rows browse Images (#332)

CUJ 6 and 7 with own categories set through the API on two output files.

* test(vocabulary): compare image-category rows and the Manage category preview against the concept (#332)

The vocabulary fixture seeds own image categories on request
(image_categories: Landscapes on two images, Portraits on one image only).
mapVocabularyConcept maps D143 ("N positive prompts · M images", image-only
rows before Manage category), a model checked against the concept page with
those rows added at runtime, and mapVocabularyDialog takes the preview
sentence, so the Manage category preview is compared with D9 and D144's
images. The default fixture and mapping are unchanged.

* test(vocabulary): park the pointer before measuring the category rows (#332)

Firefox kept the pointer where the concept's Review was pressed, over the
production Portraits row's Browse, so that button was measured hovered.

* docs(testing): record D140, D143, D144 and D146 and the image category UI (#332)

D140 (Set category writes images' own categories), D146 (the chip's visible
origin) beside D141, D142 and D145; D143 and D144 in the vocabulary
section. D8, D21 (the D20/D21 side panel) and D39 amended; the D8, the
"all N matching" and the Native vocabulary UI paragraphs name the Prompt
library editor and the legacy admin "Set Category" for prompt
categories; the #332 section lists the interface and its comparisons.

* test(vendor): count the #332 host case in the execution matrix

image-attribution.spec.mjs gains the image category route case, so the
execution config lists 92 tests (3 attribution cases per project).

* fix(images): focus after a changed source; Browse is a fresh Images visit (#332)

Review fixes: a 409 source-changed reload moves focus to the heading as a
success does; a selection the listing no longer holds says none could be
found instead of sending nothing; a Tags & categories Browse opens a
plain Images visit filtered by the category (remembered and with the
Appearance defaults) rather than a saved-view visit.

* test(legacy): record the image category routes in the legacy contract snapshot (#332)

#332 adds PUT/DELETE /prompt_manager/images/category and
GET /prompt_manager/images/categories; the legacy pages' routes are
unchanged.

* refactor(images): take the selection bar's categories from its one caller (#332)

The getCategories default was never used (images.mjs always passes the
toolbar's categories) and left images-selection.mjs function coverage at 97.05
(floor 98).
2026-10-28 19:23:41 +00:00
Vito 4f37af209c feat(collections): store collections and add their routes and Images filter (#331, PR 1) (#400)
* feat(collections): store collections in the library database

* feat(collections): add, remove and rename collections and keep members through duplicate cleanup

* feat(collections): add collection routes and the Images collection filter

* feat(images): store images' own categories in the library database (#332)

One new table, image_categories, and its category index, created empty
by a forward-only migration that reads and writes only its own objects.
Startup checks the structure; restore also checks the values. The store
sets, clears and reads own categories by file key (the deepest output
source holding the file and the path below it), and copies a removed
duplicate's category to a kept copy that has none.

* feat(images): duplicate cleanup keeps images' own categories (#332)

The kept copy's own category wins; a kept copy without one receives the
removed copy's, keyed by the deepest output source holding each file.
The removed copy's row is never deleted, so a give-back or a failed
delete loses nothing. The job log reports how many were kept.

* feat(images): set, remove and list images' own categories (#332)

PUT /prompt_manager/images/category sets the own category of up to 500
listed files, DELETE removes it (listed or not), and
GET /prompt_manager/images/categories lists the values whose file is
listed. Targets are the Images listing identity of an output source,
checked by media_target (shared with Collections). The routes never
touch prompts, recipes or links; POST /prompt_manager/bulk/category is
unchanged.

* fix(images): key symlinked output folders and keep key work off the event loop (#332)

Review fixes: a source matches by its configured spelling too, so a
detected output folder that is a symlink keys its files (routes and
duplicate cleanup). File keys are computed in the executor. A path no
source holds is a plain 403, a lone surrogate a 400. The install repairs
a table that lacks its category index instead of refusing to open.

* fix(collections): address review findings

* feat(images): set or remove a category on all N matching images (#332)

The category routes also accept the Images listing filter of "Select
all N matching" (#392) instead of explicit items, resolved now and
capped at 500 images like an explicit selection.

* test(vendor): count the #332 host case in the execution matrix

image-attribution.spec.mjs gains the image category route case, so the
execution config lists 92 tests (3 attribution cases per project).

* test(browser): install the slideshow clock before wrapping setTimeout

The slideshow fixture wrapped window.setTimeout before page.clock.install(),
so the viewer's timer ran on real time and clock.runFor() did not drive it;
a late WebKit timer failed 'slideshow exposes exact state labels' on CI.

* test(legacy): record the image category routes in the legacy contract snapshot (#332)

#332 adds PUT/DELETE /prompt_manager/images/category and
GET /prompt_manager/images/categories; the legacy pages' routes are
unchanged.

* test(duplicates): a removed copy with a collection and a category moves both (#331, #332)
2026-10-28 18:49:06 +00:00
Vito 3ebd5e4166 ci(browser): a hung browser step fails with its running tests named and its traces kept (#425) (#429)
* ci(browser): stop a hung browser run before the job limit and name it

A browser job killed at its 20-minute limit keeps no log and skips its
upload steps (merge-queue run 40723857746, firefox shard 2/2), so nothing
showed which test hung.

- Each browser test step gets timeout-minutes 18, below the job's 20, so
  the step fails and the always() uploads still run.
- Under CI, Playwright's globalTimeout stops the run first:
  PM_TEST_GLOBAL_TIMEOUT_MINUTES 17 for page workflows, 15 for coverage
  workflows, a fixed 3 for the viewer suite. Locally there is no limit
  unless the variable is set. Each test keeps the 30 s default.
- running-tests-reporter.mjs prints the tests still running when the run
  ends; Playwright's own reporters cannot, since those tests never end.
- A blob report goes to tools/vendor/test-results/blob/, inside the
  uploaded traces path.
- ci-workflow.test.mjs pins globalTimeout < step limit < job limit and the
  always() uploads; no existing limit is raised.

Closes #425

* ci(browser): rely on the step limit, report the running tests on SIGTERM

Maintainer ruling on #425: no Playwright globalTimeout. The 18-minute
step limit under the unchanged 20-minute job limit stays, as do the
always() uploads and the CI blob report.

- Drop globalTimeout and PM_TEST_GLOBAL_TIMEOUT_MINUTES with its helper
  and tests.
- A step that reaches its limit gets SIGTERM, which ended Playwright with
  no report and no blob. The running-tests reporter now catches the first
  SIGTERM, prints the tests still running and sends Playwright its own
  SIGINT, so the run ends interrupted and the blob report is written. A
  second SIGTERM kills as usual.
- Test paths are relative to the suite's rootDir instead of the cwd, so
  `npm run --prefix` no longer prints ../../ prefixes.
- The workflow comment and TESTING.md say the guarantee assumes an npm
  cache hit and that coverage shard 1 is tight in its 18-minute step.

* ci(browser): forward SIGTERM and SIGINT from frontend-coverage.mjs to its child

In the coverage job npm signals only frontend-coverage.mjs. It blocked in
spawnSync with no listener, so Playwright never saw a step timeout. It now
runs each step's child with async spawn, in its own process group, and
forwards SIGTERM and SIGINT to the running child. Step order, exit codes
and the "Command failed (N)" message are unchanged.

Locally, in the pinned image, `timeout -s TERM` on test:coverage printed
the still-running tests and wrote both blob reports. TESTING.md records
that a SIGTERM to npm alone stops at npm's `sh -c`, and that delivery
through `docker exec` in CI is not yet observed.

Refs #425

* ci(browser): load the running-tests reporter by absolute path

A config imported from another directory (the workflow-artifacts test copies
it to a temp root) resolved './running-tests-reporter.mjs' against that root
under CI and failed to load it. Resolve it from this module instead.
2026-10-28 18:38:33 +00:00
Vito d9f168c317 feat(nodes): per-node Save to library switch (#90) (#427)
* feat(nodes): per-node Save to library switch (#90)

Every PromptManager node (Classic, Text, Focused, Split, Pair) gets an
optional save_to_library BOOLEAN, default on and declared last. Off, the
node still returns its normal outputs but PromptManager ignores the run:

- the node saves, counts, tracks and records nothing;
- the queue hook reads the switch from the queued graph (literal, or a
  linked PrimitiveBoolean), so a cached node is not counted either; a
  switch wired from an unreadable node is left to the node itself;
- an opted-out node is no run prompt, so no tracker entry, run key,
  embedded-metadata match or history attribution links its images, and
  a run with any opted-out positive node links none of its files;
- the scan reports such files as not_saved (never stored or queued for
  Needs review), and a file whose only opted-out node is its negative is
  stored positive-only, without the pair;
- execution history records no prompt text for an opted-out role.

PromptSearchList only reads the library and gets no switch.

The switch takes the widget index older workflows used for the editor's
own DOM slot (''), so a configure step turns any non-boolean value on:
workflows saved before the switch load with it on and no earlier widget
value moves. The Focused, Split and Pair editors draw it as a "Save to
library" checkbox in their toolbar; Classic and Text show the native
toggle.

* fix(nodes): Save to library review fixes (#90)

- A switch wired from a node the graph cannot read and off at run time
  marks the node's run key, so an earlier run's tracker entry never
  links this run's images.
- The scan reports such files as not_saved_wired ("only the run itself
  can link it") instead of not_saved.
- Workflow-only metadata ignores muted and bypassed PromptManager nodes
  and reads nodes inside subgraph definitions.
- The Focused, Split and Pair footer says when the switch comes from a
  link.

* fix(usage): count cached re-runs of runtime-linked prompt text (#347)

A PromptManager-family node whose text comes from a runtime link
(PreviewAny, an LLM, a list node) counted only the job that executed it;
later jobs served from ComfyUI's output cache counted nothing.

Each execution now records a receipt (digests of its STRING outputs and
hashes of the text it stores) per run key and queue item. On
execution_cached the receipt whose digests equal the cache entry's text is
replayed once for the job, with the node's role and what the queue hook
already saw recomputed from the job's own graph. The executing node and the
replay count through one shared own_counts. Unmatched cache entries count
nothing, as before; a node that executes after its replay in the same job
skips its own count. No schema change, no backfill.

* fix(usage): never replay a node twice in one job; dedupe receipts once complete (#347)

A repeated execution_cached id or message for the same job now counts the
node once. Receipts replace equal older ones only when their job finishes
(or the next job starts), so a list still recording [A] of [A, B] no longer
drops an older [A] receipt. Node-level tests check the receipt each
execution records.

* test(coverage): baseline for the Save to library module (#90)

* test(usage): disabled node then cached jobs counts 0 (#90, #347)

A PromptManager node with Save to library off records no cached-run
receipt and is no run prompt in the replayed job's graph, so ComfyUI
serving it from its cache on later jobs counts nothing. The prompt is
saved first by an enabled run, so a replayed count would show. Covers
PromptManager, Focused, Split, Text and the Pair.

* fix(nodes): hide the editor's native widgets through one helper (#90)

The Save to library switch copied the text fallbacks' zero-height computeSize
arrow, and neither copy was ever called, so focused-node.js function coverage
fell to 87.5 (floor 90). One hideNativeWidget helper now hides both, and the
spec checks that the hidden switch and a text fallback take no node height.
2026-10-28 18:09:16 +00:00
Vito cdb771d722 feat(images): filter, inspect and count by each image's effective category (#332, PR 2) (#406)
* feat(images): store images' own categories in the library database (#332)

One new table, image_categories, and its category index, created empty
by a forward-only migration that reads and writes only its own objects.
Startup checks the structure; restore also checks the values. The store
sets, clears and reads own categories by file key (the deepest output
source holding the file and the path below it), and copies a removed
duplicate's category to a kept copy that has none.

* feat(images): duplicate cleanup keeps images' own categories (#332)

The kept copy's own category wins; a kept copy without one receives the
removed copy's, keyed by the deepest output source holding each file.
The removed copy's row is never deleted, so a give-back or a failed
delete loses nothing. The job log reports how many were kept.

* feat(images): set, remove and list images' own categories (#332)

PUT /prompt_manager/images/category sets the own category of up to 500
listed files, DELETE removes it (listed or not), and
GET /prompt_manager/images/categories lists the values whose file is
listed. Targets are the Images listing identity of an output source,
checked by media_target (shared with Collections). The routes never
touch prompts, recipes or links; POST /prompt_manager/bulk/category is
unchanged.

* feat(vocabulary): count and rename image categories (#332)

Tags & categories lists every category value with image_count, the
stored image_categories rows holding it. Values only images hold follow
every prompt-backed value (first_pid at SQLite's INTEGER maximum), so
keyset cursors cross the boundary. /library/options keeps its own query
and never lists image-only values.

Manage category counts image rows (impact.images), treats an image-held
target as existing, renames image rows in the same transaction, allows
renaming an image-only value, and hashes the sorted image rowids into
the impact token so a change between preview and apply is refused with
vocabulary-conflict. Every image-category statement reads
idx_image_categories_category.

* feat(images): the category filter matches each image's effective category (#332)

An image's own category wins; without one it inherits its link's: the
recipe's category when set, else the positive prompt's (MC4, one rule
with the inspector chip). Ambiguous and unlinked images inherit nothing,
so an unlinked image with its own category is now listed. The link read
no longer filters by category in SQL; the page filter, select all N
matching and saved views compare the effective value. A failed
own-category read answers the existing listing-failed 500.

* fix(images): key symlinked output folders and keep key work off the event loop (#332)

Review fixes: a source matches by its configured spelling too, so a
detected output folder that is a symlink keys its files (routes and
duplicate cleanup). File keys are computed in the executor. A path no
source holds is a plain 403, a lone surrogate a 400. The install repairs
a table that lacks its category index instead of refusing to open.

* feat(inspector): show an image's own category in the inspector and History (#332)

The inspector's library facts gain category_scope "image": an image's
own category overrides the inherited one for any link status (linked,
ambiguous, none). Images reads it for output sources only; History reads
it once per inspection inside its snapshot, keyed by the same file key
from the symlink-resolved output folder. A failed read hides the chip
rather than show a possibly wrong value. The chip's hidden prefix names
the origin ("Image category: "); the visible origin text comes in PR 3.

* perf(images): keep the category filter's link read narrow; record D141, D142, D145 (#332)

Under a category the link read keeps only links whose raw recipe or
positive category is the value (Python still decides the inherited
one), the links of images whose own category is the value are read by
path, and plain reads skip the inherited-category work. The own-category
map is keyed by normalized path. TESTING.md records D141, D142 and D145
and amends D13, D36 and D39; the filter budget seeds 2,000 own
categories.

* feat(images): set or remove a category on all N matching images (#332)

The category routes also accept the Images listing filter of "Select
all N matching" (#392) instead of explicit items, resolved now and
capped at 500 images like an explicit selection.

* fix(images): Windows path keys for own categories; plan check for the category read (#332)

Review fixes: on Windows the own-category map keys each file with
normalize_image_path (abspath drops trailing dots and spaces), and the
plan test also covers the category's narrowing clause.

* test(vendor): count the #332 host case in the execution matrix

image-attribution.spec.mjs gains the image category route case, so the
execution config lists 92 tests (3 attribution cases per project).

* test(legacy): record the image category routes in the legacy contract snapshot (#332)

#332 adds PUT/DELETE /prompt_manager/images/category and
GET /prompt_manager/images/categories; the legacy pages' routes are
unchanged.
2026-10-28 17:19:11 +00:00
Vito 55fa2c4870 fix(runs): count cached PromptManager nodes fed by a runtime text link on every job (#347) (#423)
* fix(usage): count cached re-runs of runtime-linked prompt text (#347)

A PromptManager-family node whose text comes from a runtime link
(PreviewAny, an LLM, a list node) counted only the job that executed it;
later jobs served from ComfyUI's output cache counted nothing.

Each execution now records a receipt (digests of its STRING outputs and
hashes of the text it stores) per run key and queue item. On
execution_cached the receipt whose digests equal the cache entry's text is
replayed once for the job, with the node's role and what the queue hook
already saw recomputed from the job's own graph. The executing node and the
replay count through one shared own_counts. Unmatched cache entries count
nothing, as before; a node that executes after its replay in the same job
skips its own count. No schema change, no backfill.

* fix(usage): never replay a node twice in one job; dedupe receipts once complete (#347)

A repeated execution_cached id or message for the same job now counts the
node once. Receipts replace equal older ones only when their job finishes
(or the next job starts), so a list still recording [A] of [A, B] no longer
drops an older [A] receipt. Node-level tests check the receipt each
execution records.

* test(comfyui): count #347's cached-run cases in the execution suite pin (#347)

The pin listed 19 execution-text-link cases per project; #347 adds five node
kinds and eight one-renderer cases, so the suite lists 32 per project and 118
in total with #332's image-attribution case from main.
2026-10-28 16:37:24 +00:00
Vito 11422d4584 perf(images): read prompt links in one SQLite step (#354) (#418)
* perf(images): read prompt links in one SQLite step (#354)

get_image_link_matches streamed one sqlite3_step per linked row, and
Python's sqlite3 releases and retakes the GIL around every step. While
ComfyUI generates, each retake waits on the busy thread, so the
first-page link read of the workspace Images page stretched from tens of
milliseconds to seconds.

SQLite now returns the unchanged filter SELECT's rows as one JSON value
(json_group_array), decoded with json.loads: a constant two or three GIL
hand-backs instead of one per row. Nothing is cached. Any
OperationalError from the wrapped statement, or a BLOB file_path or
rating, makes that read stream as before with one warning per process; a
missing JSON function stops the attempt for the rest of the process.

tests/test_database_link_matches.py proves the result equals the
streamed read on every filter combination and on NULL ratings, quoted,
backslashed, non-ASCII and multi-line paths, ambiguous and pair links,
an empty database and both fallbacks; the structural budget test also
checks the wrapped query plan.

* fix(images): keep the maintenance link reader, refine the one-step fallback (#354)

The one-step reader was named _link_rows, which shadowed
MaintenanceOperations._link_rows on PromptDatabase and broke the Missing
links preview and apply; it is now _one_step_link_rows, and a test pins
that the maintenance reader is not shadowed.

An interrupted read (Move library retiring the connection) is raised as
before instead of falling back and spending the one warning; a JSON text
over SQLite's length limit (DataError) streams like the other value
failures. The equivalence test compares value types per path rather than
by row order, and a test checks that the read leaves no WAL snapshot
pinned.

* docs(testing): name every one-step link read fallback (#354)
2026-10-28 16:00:15 +00:00
170 changed files with 14675 additions and 1093 deletions
+13
View File
@@ -27,6 +27,15 @@ concurrency:
# coverage shards 7m34s/6m04s/5m22s (tests 6m38s/4m53s/4m04s), merge 54s, firefox 7m57s/5m21s (tests
# 7m08s/4m34s), webkit 8m16s/5m03s (tests 7m26s/3m58s); a cache miss adds up to 5m to npm ci, which
# the 20-minute limits absorb.
# Hangs (#425): a job killed at its limit keeps no log and skips its upload steps, so each browser
# test step has its own timeout-minutes (18) below the job's 20. At the step limit, SIGTERM makes
# tools/vendor/running-tests-reporter.mjs print the tests still running and hand over to Playwright's
# interrupt, which writes the blob report; the always() uploads then run.
# Measured on main (runs 40725289094, 40719375442, 40700439399): setup before the test step <= 52s,
# uploads <= 35s; test steps up to 14m42s (webkit shard 1/2) and 13m05s for coverage (shard 1/3:
# 12m53s, node units and the viewer suite before 11.8m of workflows), so coverage is the tight one
# in its 18 minutes. This holds only when the npm cache hits: a cold cache adds up to 5m to npm ci,
# and then the job limit can still end the job before the step limit does.
jobs:
frontend-coverage:
name: frontend coverage (shard ${{ matrix.shard }}/3)
@@ -79,6 +88,7 @@ jobs:
- if: matrix.shard == 3
run: npm run --prefix tools/vendor test
- name: Page workflows with coverage, shard ${{ matrix.shard }}/3
timeout-minutes: 18
run: npm run --prefix tools/vendor test:coverage:shard -- ${{ matrix.shard }}/3
- name: Upload raw coverage
uses: actions/upload-artifact@v7
@@ -87,6 +97,7 @@ jobs:
path: coverage/js/raw/
if-no-files-found: error
retention-days: 3
# Traces and the blob report (test-results/blob/) of a failed, timed-out or cancelled step.
- name: Upload Chromium and touch traces
if: always()
uses: actions/upload-artifact@v7
@@ -211,7 +222,9 @@ jobs:
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
- run: npm ci --prefix tools/vendor --ignore-scripts
- name: Page workflows in ${{ matrix.browser }}, shard ${{ matrix.shard }}/2
timeout-minutes: 18
run: npm run --prefix tools/vendor test:workflows -- --project ${{ matrix.browser }} --shard ${{ matrix.shard }}/2
# Traces and the blob report (test-results/blob/) of a failed, timed-out or cancelled step.
- name: Upload browser traces
if: always()
uses: actions/upload-artifact@v7
+4
View File
@@ -214,3 +214,7 @@ coverage/
# Old library files a Move library renamed (never deleted; #249 S6b)
*.moved-to-*
# Per-install key behind the workspace scopes and cursors (#430)
workspace-scope.key
.workspace-scope.key.*.tmp
+6
View File
@@ -312,6 +312,12 @@ Both nodes support the same metadata fields:
Both nodes will automatically save prompts to the database and link any generated images to the prompt.
- **Save to library** (`save_to_library`, on by default; "Save to library" in the Focused, Split and Pair editors):
turn it off while iterating and PromptManager ignores that node's runs. The prompt is not saved, the run is not
counted and its images are not linked (a scan skips them too), while the node's output is unchanged. Workflows
saved before this switch existed load with it on. If any PromptManager node feeding a sampler's positive input
has it off, none of that run's images are linked, because the image cannot show which sampler produced it.
### 🖼️ Using the Output-Wide Gallery
The comprehensive gallery system provides complete access to all your ComfyUI output:
+360 -42
View File
@@ -262,6 +262,31 @@ watchdog dump, is a failure like any other: read the dump, find the blocking
call, fix it. Never raise a limit or rerun to get green; a limit changes only
with new measurements written next to it.
Browser jobs (#425): GitHub keeps no log of a job killed at its limit and skips
its upload steps, so the test step in each browser job in `viewerjs-check.yml`
has its own `timeout-minutes` (18, below the job's 20). At the step limit
GitHub sends SIGTERM, which would end Playwright without any report; under `CI`
`tools/vendor/running-tests-reporter.mjs` catches the first one, prints
`Still running when the run ended (SIGTERM): N` and one line per unfinished
test (path relative to the suite's `rootDir`), then sends Playwright its own
SIGINT, so the workers stop, the run ends interrupted and the blob report is
written to `tools/vendor/test-results/blob/`. In the coverage job npm signals
only `frontend-coverage.mjs`, which forwards SIGTERM and SIGINT to the child it
is running (started in its own process group, so a group-wide signal still
arrives once). Delivery is proven locally in the pinned image for a SIGTERM to
the whole process group (`timeout -s TERM`). A SIGTERM to npm alone
(`timeout --foreground`) reaches nothing: npm forwards it to the `sh -c` that
runs the script, and that shell exits without passing it on. Delivery in CI,
where the step runs through `docker exec`, is not yet observed; the next real
step timeout will show it. That directory and the traces
upload with `if: always()`; `npx playwright merge-reports` opens the blob.
Each test keeps the 30 s default and there is no `globalTimeout`. Two limits
of this: it assumes the npm cache hits (a cold cache adds up to 5 minutes to
`npm ci`, and the job limit can then still win), and coverage shard 1 (12m53s
measured) is the tight one in its 18-minute step. Local runs keep the `list`
reporter only; reproduce with `CI=1 timeout -s TERM 40 npm run --prefix
tools/vendor test:workflows -- --project=firefox`.
### After a push
Follow the run for the commit you pushed, never an older one:
@@ -391,6 +416,10 @@ A spec performance budget gets two tests:
`test_filter_budget` (Images filters, spec D12: p95 < 150 ms at 50,000 files
and 20,000 links).
A hot SQLite read that returns many rows aggregates them into one step
(`json_group_array`) instead of stepping once per row, so it does not wait for
the GIL once per row while ComfyUI generates (#354, `get_image_link_matches`).
The `perf budget` job in `.github/workflows/test.yml` is the reference
measurement: `ubuntu-latest`, one Python, no coverage, only the wall-clock tests.
It runs on every push to `main` and on `workflow_dispatch`, never on pull
@@ -522,6 +551,49 @@ hold a walk open with a gated scan instead of sleeping.
time to network idle and the 10 slowest requests, from a fresh browser
context each run.
## Open pages across restarts and source changes (#430)
A page addresses a listed file as `{root_index, root_scope, path}`. The root
scope is an HMAC over the root's index and canonical path, so the server refuses
(409 `source-changed`) an action aimed at a root that was renumbered or replaced
since the page listed it (a same-named `ComfyUI_00001_.png` sits in every output
folder). Its key, and the listing cursors' key, derive from one random secret in
`workspace-scope.key` beside the database (`utils/workspace_keys.py`; owner-only,
64 hex characters; never in config.json or the database, so no backup or export
carries it; Move library copies it, best effort). A ComfyUI restart therefore keeps
every scope and cursor an open page holds: `tests/test_api_workspace_restart.py`
proves it with a second API on its own server and with two fresh interpreters on
the same database. The key file is created with a temp file and `os.link` (of two
first starts one file wins; without hard links `O_EXCL`), re-read after any write,
replaced with a WARNING when corrupt, and a folder that cannot hold it gives that
process a per-process key (one WARNING; open pages then refresh after a restart).
Tests never write it outside their own temp folders.
The cursor is `{"v": 2, "key"}` with the listing scope (it hashes every absolute
root) bound into its MAC only, never sent. Bump `v` whenever the key tuple's shape
changes: a cursor now outlives a restart and an upgrade. A cursor the server
cannot verify (another key, root list or filter set) answers 400
`cursor-stale`; Images then shows "The image collection changed. Refresh image
collection to continue." and its Retry becomes **Refresh image collection**.
When a source really changed, `POST /prompt_manager/images/rescope` (`{"target":
{...}}`, validated by `media_target`, index ≤ 4095) maps the held target to the
first current root whose scope at the old index equals the held scope, and answers
the file's current workspace descriptor; a removed or replaced root, another key
or a foreign scope is 409 `source-changed`, a missing file 404 `file-missing`, a
path leaving the root or a preview 403 (always `no-store`). Only the Images roots
are mapped. Images (`images-source-recovery.mjs`) asks once per held
`(key, scope)` pair, shared by the inspector (an inspection that turns
`refreshRequired` with `code: "source-changed"`), the viewer (a media load error)
and the Tag image dialog. Found under another index, nothing is kept by an old key
(`root_index:path` could name another source's file): the viewer closes, the checks
clear, a Source filter on the old index follows the file (another one falls back to
All sources), only that file stays selected and the collection lists again; a
manual Refresh forgets every pair. Not found, the existing visible Refresh stays.
Browser tests: `workspace-stale-source.spec.mjs` (`/__test__/sources/reverse` and
`/__test__/scope-key/rotate`, both undone by every reset); the manual-Refresh
recovery specs in `workspace-images.spec.mjs` answer every rescope 409.
## Windows portability preflight
Before changing paths, SQLite, streams, restore or cleanup:
@@ -659,7 +731,7 @@ The shared inline and collection inspector reads the current rooted file's suppo
embedded records, exact generation facts and recorded or inferred prompt evidence.
It renders in two concept surfaces (#249 D20–D22, D38). The Images side panel shows
only the concept content: cover, title, filename, the evidence badge with a category
chip, the + and − prompt blocks with copy and send, tag chips (the file's own image
chip naming its origin (D146), the + and − prompt blocks with copy and send, tag chips (the file's own image
tags first, D88, then the linked tags; image tags and the linked positive's own tags
filter Images, D21), the six facts, and a footer with "Send to ComfyUI" and
"Prompt history" only when prompt roles resolved or a link is saved; the sticky panel
@@ -1340,7 +1412,7 @@ How the earlier workspace Images filters moved (old → new). The legacy `/galle
| Folder | "Refine this view" → Folder (D14); shown in the scope chip |
| Media (All media / Images / Videos) | "Refine this view" → Media (D14) |
| Sort: Newest first / Oldest first / Filename A–Z / Filename Z–A | toolbar sort: Recently added / Oldest first / Name A–Z / Name Z–A, plus Highest rated (D13, D14) |
| (none) | toolbar category and "Refine this view" → Tags, through the linked positive (D13, D14) |
| (none) | toolbar category (the effective category, D141) and "Refine this view" → Tags, through the linked positive (D13, D14) |
| Refresh | Import → "Scan an output folder" → Rescan (D15) |
| Grid / List / Masonry, Comfortable / Large / Compact | the same choices in the concept filter row |
| Card image → side panel; preview icon → viewer | image → viewer; Info → side panel (D19) |
@@ -1370,22 +1442,18 @@ Recorded deviations on this page (spec §8):
inspector's own "Download metadata JSON" envelopes in listing order); the button is
`aria-disabled` and ignores a second press while it runs, the selection is kept, and
the toast names the unread images (none read: no download); leaving Images aborts the
reads with no download and no toast. Set category opens the
concept's `simpleInput` dialog ("Categorize selected images", `input#action-input`
labelled Category, Cancel / Apply) and posts the selection's distinct linked positive
ids to `/prompt_manager/bulk/category` (images have no category; the filter matches
the linked positive's). With no linked image checked it posts nothing and opens no
dialog; an empty value keeps the dialog open; the field stops at the server's 100
characters; success reloads the categories and the listing together (clearing the
selection, focus to the heading, and, as any replaced collection does, the inspector
moves to the first listed image); failure keeps the selection. The write carries no
page signal: leaving Images while it runs lets it finish without a toast, and the return
loads the collection afresh instead of the saved visit. Three deviations from the
concept's `simpleInput`: the field starts empty (the concept pre-fills its demo
"Landscapes"), the action reads "Apply" (not "Apply in prototype"), and the
nothing-linked check runs on the Set category press, so no dialog opens. The route
rejects anything but 1-1000 positive integer ids within SQLite's INTEGER range
(deduplicated) and a category that fails `validate_category`. `images selection bar matches
reads with no download and no toast. Set category (amended by D140, #332) sets the
checked images' own categories: it opens the concept's `simpleInput` dialog
("Categorize selected images", `input#action-input` labelled Category) and writes
`PUT`/`DELETE /prompt_manager/images/category`, never a prompt (see D140). Prompt
categories are changed in the Prompt library editor and on the legacy admin page's
"Set Category"; `POST /prompt_manager/bulk/category` is unchanged and serves that
legacy page only. The write carries no page signal: leaving Images while it runs lets
it finish without a toast, and the return loads the collection afresh instead of the
saved visit. Its success reloads the categories and the listing together (clearing
the selection, focus to the heading, and, as any replaced collection does, the
inspector moves to the first listed image); a failure keeps the selection, except a
changed source (409), which reloads the listing as a success does. `images selection bar matches
the concept` compares the bar and the checked card with one card checked; the omitted
actions flow again from the concept's own boxes, checked against the concept page with
them removed at runtime.
@@ -1423,7 +1491,8 @@ Recorded deviations on this page (spec §8):
`overscroll-behavior: contain`.
- **D35**: counts pluralize ("1 positive prompt", "1 image", "1 more image loaded").
- **D36**: the category list is the Library's, sorted alphabetically (case-insensitive)
by the page, whatever order the server sends.
by the page, whatever order the server sends (amended by D141: Images adds its
image categories).
- **D37**: production loads 40 images per page (the concept 12), so the continuation
and toast use 40; after Escape production returns focus to the opening image.
- **D44** (#249 PR E): Filters reads "Filters · N" while N of Source, Folder, Media and
@@ -1451,6 +1520,9 @@ Recorded deviations on this page (spec §8):
filters, while recipe, recorded-run and embedded tags stay plain `span.chip`.
Amended by D88 (#249 PR C): the file's own current image tags lead the same row
and are filters too; a linked tag already shown (case-insensitive) is left out.
Amended by D146 (#332): the category chip also names its origin in visible text
after the category (" · this image", " · prompt", " · recipe"), and an image's own
category shows for an unlinked or ambiguous file too.
- **D22**: an empty positive reads "No positive prompt (explicitly empty)" (the concept
applies its negative wording to both roles), and a confirmed `p.prompt-text` is
focusable (`tabindex=0`, marked focus ring) so long prompts stay keyboard-readable.
@@ -1509,15 +1581,97 @@ Recorded deviations on this page (spec §8):
- **D39** (ruled 2026-10-21: fallback): a recipe-linked image shows its recipe's own
category when set, otherwise the linked positive prompt's, and its recipe's own tags
when it has any, otherwise the positive's; each field falls back on its own and the
two tag sets are never merged. Set category (#259) writes the positive's category and
the Images category and Tags filters match the positive's, so the panel now agrees
with them whenever the recipe adds nothing. The inspection response names each
two tag sets are never merged. The Images Tags filter matches the positive's tags.
The category filter matches the same inherited category as the panel (D142, #332), and
an image's own category overrides both (D141, D146). The inspection response names each
field's source (`library.category_scope`, `library.tags_scope`: `pair`, `positive` or
null); the category chip's visually hidden prefix (D21) says "Recipe category: " or
"Positive prompt category: " from it, and tag chips are Tags filter buttons only when
null, or `image` for an own category, #332); the category chip's visually hidden
prefix (D21) says "Recipe category: ", "Positive prompt category: " or "Image category: "
from it, and its visible origin (D146) " · recipe", " · prompt" or " · this image", and tag chips are Tags filter buttons only when
they are the positive's (#260 D2). History's media inspector uses the same rule. The
concept shows the prompt's category and tags for every image and the concept
fixture's recipes have neither, so the inspector comparison needs no mapping for it.
- **D141** (#332, amends D13 and D36): the Images category filter matches each image's
effective category: its own category (Images Set category, PR 3) when it has one,
else its inherited one (D142), else none. An image whose own category differs never
matches its prompt's category, and an unlinked or ambiguous image with an own
category is listed under it (the members-only gate now applies to the Tags filter
only). "Select all N matching" and saved views use the same value. The caption's
positive-prompt count still counts the linked positives of the images shown. With a
category set the listing reads the own categories once per source (a primary-key
range); a failed read is the existing 500 `listing-failed`, never an unfiltered page.
The link read then keeps only links whose raw recipe or positive category is the
value (a superset of those that inherit it; Python decides), and reads the links of
files whose own category is the value by path, so their rating and positive count.
`test_filter_budget` seeds 2,000 own categories on its 50,000-file fixture.
The option list adds the image categories of listed files
(`GET /prompt_manager/images/categories`, PR 3), merged with the Library's and sorted
(D36); if only that read fails, the prompt categories show and the categories alert is
raised. `images category options and Set category dialog match the concept` sets one
image's own category ("Portraits") through the real route and compares the toolbar
with it sorted into the concept's options; unmapped, only the options differ.
- **D142** (#332 MC4, amends D39's filter note): the inherited category is one rule,
`inherited_category` in `utils/inspection_response.py`: a linked recipe's own
category when it is not blank, else the linked positive's. Before #332 the filter
matched the positive's even when the chip showed the recipe's. The link reads select
the raw recipe and positive categories and never decide inheritance in SQL
(`tests/test_api_workspace_image_categories.py` checks the filter and the chip agree
on blank values stored as spaces, tabs, U+00A0 and BLOB). The concept has no recipe
categories, so this is behavior only.
- **D145** (#332, DC4): History's media inspector shows an image's own category with
the chip rule of the side panel and viewer. Only the inspection request reads it, once,
inside the History snapshot: the file is the symlink-resolved output folder plus the
receipt's subfolder and filename, keyed with `image_category_key` over the configured
output sources, as Images keys it. A temp output or a file no source holds has none;
a failed read hides the chip (DC5). The outputs list and media requests read nothing
new. The chip's hidden prefix reads "Image category: " for an own category; the
visible origin text and its comparison are D146 (PR 3). `history media inspector
category chip matches the concept` (history-concept.spec) sets an own category on the
Quality baseline's newest output, opens its Recorded run and "View current file", and
compares the viewer's chip with the concept viewer's category chip (app.js a402) laid
out at runtime with the D146 origin and that category; its place is not compared (what
precedes it differs), only its size, tokens, text, prefix and origin. Unmapped, the two
differ only in the text, the prefix, the width and `span.chip-source`.
- **D140** (#332, amends D8): Set category sets the checked images' own categories, so
every checked file is a target (no nothing-linked check). More than 500 checked (or
"all N matching" over 500) toasts "Select at most 500 images." and opens nothing.
The dialog is the concept's `simpleInput` with three recorded differences: the field
starts empty (the concept pre-fills its demo "Landscapes") and suggests the toolbar's
current categories (`datalist#workspace-input-suggestions`, `openInputDialog`'s
`suggestions`); "Use prompt category", a plain button between Cancel and Apply
(`secondary`), closes it and opens the "Use prompt category" confirmation ("Remove the
image category from the 1 selected image? / … the N selected images? Each one then
shows its prompt's category, or none if it has no linked prompt. This cannot be
undone.", Cancel and a danger Remove, built like saved views' delete confirmation);
and the action reads "Apply" (D8). Apply sends `PUT /prompt_manager/images/category`
`{items | filter, category}`, Remove `DELETE` `{items | filter}`; Cancel sends
nothing. An empty value keeps the dialog open and the field stops at 100 characters.
Toasts: "Category “X” set on 1 image." / "… on N images." (unchanged files count as
set) plus " N could not be found." for unlisted files; "Removed the image category
from N images." or "None of the selected images had its own category."; 404 "None of
the selected images could be found. Refresh the images and try again."; 409 "The
sources changed. Refresh the images and try again." (the listing reloads); anything
else "Category could not be set. Try again.". `images category options and Set
category dialog match the concept` (images-concept.spec) checks a card, opens the
dialog in both documents and compares it (title, body tree, field, input, suggestions,
actions and every button's box and tokens) with the concept's own dialog changed at
runtime as recorded: field emptied, the datalist added after the field, a clone of
its Cancel reading "Use prompt category" after Cancel, the primary renamed "Apply".
Unmapped, the dialog differs only in the input's value and list, the body's tree, the
suggestions and the actions.
- **D146** (#332, amends D21 and D39): the category chip names its origin in visible
text. It is `span.chip` > `span.ca-visually-hidden` (the D21 prefix) + the category +
`span.chip-source` (" · this image", " · prompt" or " · recipe"; muted
`--ca-muted`, `white-space: nowrap`, so at 390px the origin never splits; the chip may wrap
before it or inside a long category). It shows when the inspection names a non-empty category and either the
scope is `image` (any link status, so an unlinked or ambiguous file with an own
category shows it) or the file is linked. The concept's chip shows the bare name
(app.js a402). The inspector comparison (inspector-concept.spec) measures each chip's
`span.chip-source` (text, font, color, `white-space`, box) and lays the concept's own
chip out with the recorded rule and " · prompt" added at runtime (`withChipSource` in
`concept-inventory.mjs`), in the side panel and the viewer's details; unmapped they
differ only in the chip's text, width and `span.chip-source`. Its mutation check
removes the origin. History's chip (D145) is compared the same way.
- **D42**: the concept never shows a toast and a page alert together. Production keeps
the concept's fixed toast (centred, 22px from the bottom, 12px at 700px and below)
and, when an Import → Rescan toast leaves the Images discovery alert
@@ -1542,7 +1696,8 @@ applies D3 (the `.file-inspector` class of the production-only rules), D20 (the
viewer's plain tag chips, laid out by the concept page itself with its chips made
plain at runtime), the D20/D21 side-panel scroll (`overflow: auto`,
`max-height: calc(100dvh - 40px)` above 700px), D21 (the category's hidden scope
prefix) and D22 (focusable prompt text). `inspector comparison detects mutations`
prefix), D22 (focusable prompt text) and D146 (the chip's visible " · prompt", laid out
by the concept page itself). `inspector comparison detects mutations`
changes `.prompt-heading .role-label` letter spacing and removes `.detail-footer`.
Manual check (quick), after a browser refresh on Images: hover an image (it zooms);
@@ -1585,10 +1740,11 @@ wins), and toasts the mock's copy verbatim, "<n> tags saved for <filename>." eve
for one tag, plus " Prompt association still awaits review." from Needs review.
Tagging one file never changes another file (an equal-byte copy included), a
prompt, a pair or a link. States the mock lacks (generating, busy, conflict,
changed storage or file, expired capture, unconfirmed save, unavailable model,
video) are text in `#tag-generation-status` only; **Retry** (D116) leads the
actions while the controller allows it, where the other workspace dialogs place
their Retry. The Tags
unavailable model, video) are text in `#tag-generation-status` only; a state that
blocks Save (a failed or refused capture, a changed source, file or storage, an
expired capture, an unconfirmed save) shows its reason beside the actions instead
(D147); **Retry** (D116) leads the action buttons while the controller allows it,
where the other workspace dialogs place their Retry. The Tags
page re-reads its lists after every confirmed save, so the Images counts follow.
The image-tagging plan's deviation numbers were renumbered by +23 because the
@@ -1632,8 +1788,8 @@ Inspect plan keeps D60–D66 (plan D60 → D83, D61 → D84, D65 → D88, D68
its "Auto-tag images" always opens the dialog. Production opens it for the image
selected in Images when it was last left; with none it is `aria-disabled` and
pressing it toasts "Select an image in Images first." (no new chrome).
- **D116** (ruling T5): **Retry** leads `.dialog-actions` (before Cancel and the
primary) while a failed observation, a failed Generate, an unconfirmed save or an
- **D116** (ruling T5): **Retry** leads the buttons of `.dialog-actions` (before
Cancel and the primary; D147's notice row sits above them) while a failed observation, a failed Generate, an unconfirmed save or an
answer that needs a fresh observation (changed storage or file, expired capture)
can be retried; the concept has no failure states, so it has no Retry. After a
failed Generate it takes Generate's own path (an invalid field reports its
@@ -1644,6 +1800,25 @@ Inspect plan keeps D60–D66 (plan D60 → D83, D61 → D84, D65 → D88, D68
Browser tests: "D116: Retry leads the actions after an unconfirmed save…", "D116:
Retry after an expired capture observes again and keeps the typed draft (M5)" and
the paired `production-unknown-outcome` capture.
- **D147** (#430, beyond the Global Constraints' "status text only"): a state
that blocks Save shows its reason in `div.tag-action-notice` (its sentence in a
`role="status"` span, written only when it changes), the first, full-width row of
the sticky `.dialog-actions`, above the buttons; `#tag-generation-status` is then
empty, so the sentence shows once. Generate and Save are `aria-disabled` and look
it (`.tagging-modal [aria-disabled="true"]`: opacity .55, `not-allowed`). A failed
or refused capture replaces the empty preview with a same-size
`div.tag-image-preview.tag-preview-empty` "Preview unavailable" (`role="img"`).
A changed source (409 `source-changed` from observe, Generate or Save, phase
`source-changed`, never retried as is) first asks the page where the file is
(`onSourceChanged`, see "Open pages across restarts and source changes"): found,
the dialog stays open and observes it there (`controller.retarget`, the typed
draft kept, the old capture released; refused while a save is in flight); not
found, the notice shows the server's sentence and **Refresh image collection**
(Images and Library-embedded galleries; **Reload page** elsewhere), which closes
the dialog first. Opened, generated and empty states are unchanged, so the concept
comparisons are not. Browser tests: `tests/browser/workspace-stale-source.spec.mjs`;
the file-changed and unconfirmed-save checks in `workspace-tagging.spec.mjs` and
`tagging-concept.spec.mjs` read the notice.
- **D29** (no prototype copy) in the dialog: no "Prototype · suggestions are
simulated…" note, the status reads "Suggestions from <model>. Approve tags to save."
(not "Sample suggestions from …"), and the toast has no " in this tab".
@@ -1696,7 +1871,8 @@ production-versus-concept difference of PR C and its record:
| Image tags first in the inspector and viewer chip row | D88 (ruling T7/C2) |
| Auto-tag images `aria-disabled` without a selected image | D115 |
| Retry leads the actions when the controller allows it | D116 (ruling T5) |
| States the concept lacks (busy, conflict, changed storage or file, unconfirmed save, unavailable model, video) | Global Constraints: `#tag-generation-status` text only |
| States the concept lacks (busy, conflict, unavailable model, video) | Global Constraints: `#tag-generation-status` text only |
| A state that blocks Save (failed capture, changed source, file or storage, unconfirmed save): its reason and Refresh beside the actions, a "Preview unavailable" box | D147 (#430) |
### Bulk tagging from the selection bar (#249 PR D)
@@ -1877,11 +2053,13 @@ ranges include cards that bounded rendering has unmounted.
Over the limit, the client toasts "Narrow the view to at most 10,000 images." and the
server answers 400 `too-many-images`. No match is 400 `no-images`. Auto-tag in this
mode does not offer Review each image, because the review walks images in the
browser. Set category posts `{images_filter, category}` to `/prompt_manager/bulk/category`.
The server sets the category on the distinct linked positives and answers
`prompt_count` and `skipped`. A listing whose `positive_count` is 0 toasts the
nothing-linked sentence and opens no dialog, and over 10,000 matches it toasts the
limit sentence. Export metadata reads every image in
browser. Set category (D140, #332) sends `{filter, category}` to
`PUT /prompt_manager/images/category` (Use prompt category `{filter}` to `DELETE`),
which sets or removes each matching image's own category; over 500 matches the client
toasts "Select at most 500 images." and opens no dialog, and the server answers 400
`too-many-images`. (`POST /prompt_manager/bulk/category` still accepts
`{images_filter, category}` and sets the linked positives' category, but no workspace
control sends it any more.) Export metadata reads every image in
the browser, so it is `aria-disabled` in this mode, and its `title` says why.
Tests: `tests/browser/workspace-images.spec.mjs` ("Shift+click selects a range across
@@ -4624,8 +4802,18 @@ Filters (`mode=workspace`; anything invalid is 400 "Invalid gallery filters"):
rating 5 to 1, then linked but unrated, then unlinked and ambiguous; ties break
newest first. A signed cursor covers every filter and the sort, so reusing it
with a change is 400.
- One link read (`get_image_link_matches`, streamed and bounded to the listed
paths) serves the filters, the rating sort and the count.
- One link read (`get_image_link_matches`) serves the filters, the rating sort
and the count. SQLite returns its rows as one JSON value in one step (#354):
Python's `sqlite3` releases and retakes the GIL around every step, and while
ComfyUI generates each retake can wait on the busy thread, so a read that
stepped once per linked row took seconds. The result keeps only the listed
paths; peak memory grows with the database's matching linked rows (about
530 bytes each, freed on return). When the JSON step fails (a SQLite without
JSON functions, a BLOB value, a JSON text over SQLite's length limit) the read
streams the rows as before and logs one warning per process; a missing JSON
function stops the attempt for the rest of the process. An interrupted read
(Move library) raises as before. `tests/test_database_link_matches.py` proves both reads
equal on every filter and on odd values.
`positive_count` is the number of distinct linked positives over the full matching
set, not the page:
@@ -4739,6 +4927,21 @@ category; without one it shows the category its link supplies.
appears at the same path shows it. The real-host check is the last test in
`tests/comfyui/image-attribution.spec.mjs`: PUT, list, DELETE on a generated
output.
- Interface (PR 3, a browser refresh; no restart beyond PR 1-2's): the Images
selection bar's Set category writes these routes (D140, which amends D8); the toolbar
category options add the listed images' own categories (D141); the inspector's
category chip names its origin, " · this image", " · prompt" or " · recipe" (D146), in
the side panel, the viewer and History's media inspector (D145); Tags & categories
counts images on each category row and lists image-only values (D143), and Manage
category previews and renames image rows too (D144). Prompt categories are changed in
the Prompt library editor and on the legacy admin page's "Set Category". Concept
comparisons, at 1600x1100 and 390x844 in both themes: `images-concept.spec.mjs` (the
Set category dialog and the options), `inspector-concept.spec.mjs` (the chip),
`history-concept.spec.mjs` (History's chip) and `vocabulary-visual.spec.mjs` (rows and
preview, with `vocabulary-canonical-comparison.mjs`). Behavior: `workspace-images`,
`full-inspector`, `workspace-history` and `workspace-vocabulary` specs, and
`tests/js/workspace-images-category.test.js`, `workspace-file-inspector.test.js`,
`workspace-vocabulary-state.test.js`, `workspace-toolbar.test.js`.
Image annotations schema (#249, `database/annotation_schema.py`): the group
`media_annotation_metadata` (one row: schema version, `image_index_revision`),
@@ -5203,6 +5406,75 @@ same as a file the scan could not attribute, so the file stays on disk and in th
gallery as unlinked, and its favorite and tags (kept by file) are untouched. Every
run prints this. Queue-linked batch images can't be proven after the fact and read
`unlink`, so review before applying. Stop ComfyUI first.
### Foreign workflows (#247): the shared per-file decision
The output scan decides each file through `utils/file_prompt.py`
(`decide_file_prompt`, the ladder), which the monitor will share for outputs of
workflows without a PromptManager node. It does no file or database I/O.
- **Save to library (#90).** Before anything else, a file whose PromptManager
node had Save to library off is `not_saved`, and one whose switch is wired
from a node the file cannot show is `not_saved_wired`: nothing is written, no
recorded-run fallback, no Needs review item. A resolved pair whose negative's
node is off stores the positive alone (source `opted-out-negative`). The
#90 graph walk follows only prompt-like inputs, so the ladder also walks
every input upstream of each sampler or guider positive (not negative, model,
clip, latent or image inputs; every PromptManager node of a graph without a
sampler) and checks the text later steps name: a resolved or inspected
positive whose path passes such a node (`side_opted_out`), or a legacy answer
that is such a node's text (resolved, else literal). A switch that is off
makes the file `not_saved`; one wired from a node the graph cannot read makes
it `not_saved_wired`, as #90's direct check does.
- **Order.** The default profile decides first, exactly as before (resolved
pair, A1111, sampler-less). Only a graph it leaves ambiguous or unresolved
reaches the inspector's resolve (`inspection_resolution` in
`utils/prompt_inspector.py`, the inspection profile, then the recorded saver
anchor of #386; `name_anchor=False`: the file-name anchor never decides a
link; the scan's legacy floor may still link the file, as before #247, and a
file nothing links is queued with the anchor's choice per #390), and only that positive's `storage_text` is stored (never
prepend or append text, never `recorded_texts`). Then the #170 browser parser
(`graph_prompt_text`), the legacy floor, then the recorded run's stored
prompt. The legacy answer is refused when a PromptManager-family node has a
prepend or append part and the answer is not that node's own `text` (the
parser joins the parts, so the stored text cannot be told from it). History is asked once per file and only at that step; the scan's
batch lookup stays lazy and makes one `recorded_runs` pass for the stored
prompt and the saver (`_BatchHistory`).
- **Policies.** `SCAN` keeps the legacy floor for every undecided file.
`MONITOR` waits for history inside the output folder (`history`) and stores
nothing for a file the inspector calls ambiguous; an untraceable or
malformed one still gets the legacy floor.
- **Junk.** `junk_text` refuses `blank`, `number` (the #252 `_numeric` rule),
`lora-only` (only `<lora:...>` tags and separators) and `model-file` (one
token ending in a model extension) at whichever step found the text and
nothing later is tried; the file reports `found`, except at the undecided
graph step, which keeps its own status (an ambiguous file is still queued).
- **One prompt per file.** `database/scanned_prompts.py` stores a positive in
one `BEGIN IMMEDIATE` transaction: a file already linked to another prompt
(either spelling, `file_link_rows`/`link_conflict`, the pair ingester's rule)
is `conflict` and nothing is written; a failed link rolls the new prompt back
(`found`); two writers of one text store one prompt. A recorded run's stored
prompt links the same way (`link_stored_prompt_or_raise`). A rescan therefore
never adds a second link to a file an older scan linked to the legacy answer;
the row reads "Kept: already linked to a different prompt". Link rows are
matched under the path and its `realpath` spelling (`file_link_rows`), so a
file the scan stored through the resolved folder is found when it arrives
through a symlinked or junction folder. The reverse order (a row stored under
the watcher's raw spelling, then a scan whose root resolves differently) is
matched only when that raw spelling equals the realpath. Every scan write
runs through `survive_library_switch`, so a library move reruns it.
- **Marks.** Every prompt ingestion creates, pair components included, carries
category `scanned`, tag `auto-scanned` and the note `Auto-scanned from
<file>` (`SCAN_MARKS`); a reused prompt keeps its fields. Run counts are
never touched.
Tests: `tests/test_file_prompt.py` (one test per ladder step and junk class),
`tests/test_scanned_prompts.py` (store, conflict, rollback, concurrency with
`FirstWriterHold`, pair marks), `tests/test_pair_scan.py` `TestScanDecision`,
`tests/test_run_attribution.py` (dataset branches linked by the recorded
saver, a legacy-linked file left as it is, `_BatchHistory.recorded`) and
`tests/test_recorded_saver_anchor.py` `TestInspectionResolution`.
## Linked prompt text (#343)
A link drives `text` on every PromptManager node, and `negative_text` on the Pair,
@@ -5735,6 +6007,20 @@ rotates the epoch, so a pre-restore `expected_epoch` is refused with `storage-ch
Owner rows and their timestamps never change on a tag command; a category apply only
touches rows carrying the exact source.
Image categories (#332 §7, MC9, MC10, DC7): each `GET categories` row also carries
`image_count`, the stored `image_categories` rows holding that exact value (files
missing on disk included). A value prompts and images share appears once, ordered by
its oldest positive; a value only images hold appears once after every prompt-backed
value (`first_pid` 9223372036854775807 in the keyset cursor), ordered by value, with
`prompt_count` 0. `/library/options` keeps its own query and never lists image-only
values. Category preview and apply report `impact.images` (rows holding the source),
the target exists when an image row holds it, an image-only value can be renamed, the
apply renames the image rows in the same transaction (file keys and rowids kept, so a
merge cannot collide), and the impact token hashes the sorted image rowids, so an image
row gaining or losing the value between preview and apply is 409
`vocabulary-conflict` with nothing written. Every image-category statement reads
`idx_image_categories_category` (`test_category_reads_use_the_image_category_index`).
Two legacy routes change additively: `GET /tags` now lists unused registry tags, and
`/tags/suggest` without context appends unused names at count 0. Unused tags become
deletable only through the native Delete tag (vocabulary UI PR), not on the legacy
@@ -5752,13 +6038,18 @@ Deploying this backend needs a ComfyUI restart (Python and route changes).
longer hands off to the legacy tag manager. That page stays reachable from Overview's
"Open existing tag manager" card for tag search and sort, the untagged entry and
multi-tag AND/OR browsing; bulk prompt actions (delete, tag, category) stay on the
legacy admin page. The screen consumes only the `/prompt_manager/vocabulary/*` routes
legacy admin page. A prompt's category is changed in the Prompt library editor or with
the legacy admin page's "Set Category" (`POST /prompt_manager/bulk/category`, unchanged);
the Images selection bar's Set category sets images' own categories only (D140, #332),
and Manage category renames a value on prompts, recipes and images together (D144). The screen consumes only the `/prompt_manager/vocabulary/*` routes
above and adds no route, so PR B needs no ComfyUI restart: a browser refresh is enough.
Screen: a `+ Create tag` heading action, a Tag vocabulary table (columns Tag, Prompts,
Images counting the files that carry the tag (`image_count`, #249 C2), and Actions with Browse and
Rename / merge), and a Categories panel whose category rows end with the Manage
category row (Review). Lists load 100 rows at a time, append when scrolled into view
category row (Review). A category row reads "N positive prompts", "N positive prompts ·
M images" when images hold the value as their own category too, or "M images" for a
value only images hold (D143, #332). Lists load 100 rows at a time, append when scrolled into view
and keep `Load more tags` / `Load more categories` while more remain. Prompts counts
are the Library Browse totals.
@@ -5801,7 +6092,34 @@ Recorded deviations from the concept (maintainer acceptance): bounded lists with
Load more; title "Tags"; Manage categories adds a Category select and Preview before
Apply; "Apply" instead of "Apply in prototype"; Delete tag… inside Rename or merge;
Browse sets the exact filter; singular "1 positive prompt"; the production list appends
fixture-only tags after the concept's sample tags. Paired concept/production captures
fixture-only tags after the concept's sample tags.
- **D143** (#332): category rows count the images whose own category is the value
(`image_count`, the stored rows, missing files included): "N positive prompts · M
images" beside prompts, "M images" for an image-only value, which follows every
prompt-backed row (by value) before Manage category. Browse on an image-only row opens
Images with exactly that Category filter (its Library would be empty), as a fresh
Images visit that becomes the Images return state, the way Browse seeds the Library's; every other
row opens the Library as before. Manage category's select lists image-only values
last, so past 1,000 categories they are the first left out (the dialog says the list is
capped).
- **D144** (#332): Manage category's preview counts the images holding the value ("“X”
is used by N positive prompts and M images. Apply renames it to “Y”."; D9's sentence
with "M images" added when above 0), an image-only value can be renamed, and Apply
renames the image rows in the same transaction; an image row gaining or losing the
value between Preview and Apply is 409 `vocabulary-conflict` with nothing written.
- `category rows and the Manage category preview match the concept with image
categories` (vocabulary-visual.spec, 1600x1100 and 390x844, both themes) seeds the
fixture with `image_categories: true` ("Landscapes" on two images, "Portraits" on one
image only, rows keyed as Images keys them) and compares the page with D143 mapped
(`mapVocabularyConcept(reference, imageCategories)`, whose row model is checked against
the concept page with those rows added at runtime) and the Manage dialog after
Preview with D9 and D144 (`mapVocabularyDialog`'s `preview`). Before the preview is
measured the pointer leaves the pressed Apply and focus returns to ×, the state the
opening comparison in workspace-vocabulary.spec measures. Unmapped, the page differs
in the Landscapes row's text and the added Portraits row, the dialog only in the
sentence. The default fixture seeds no image category, so the other vocabulary
comparisons are unchanged. Paired concept/production captures
for both themes and widths are attached as test artifacts; no PNG baseline is added
until acceptance. Not compared: hover states and the dialog focus ring (the page-level
keyboard ring and link ring offsets are asserted). No coverage floor was lowered;
+33 -7
View File
@@ -9,6 +9,8 @@ IMMEDIATE`` transaction, so no row ever names a deleted file:
already links the kept copy: then its saved-pair links join that record
and it is deleted;
- its favorite;
- its collection memberships (#331), joined onto the kept copy's as their
union under the same keys a favorite moves by;
- its image tags (``media_image_tags`` of its byte record), joined onto the
kept copy's byte record as their union, in order;
- its own image category (#332), copied to the kept copy when that copy has
@@ -54,6 +56,15 @@ INSERT_FAVORITE_SQL = "INSERT OR IGNORE INTO image_favorites VALUES (?, ?, ?)"
DELETE_FAVORITE_SQL = (
"DELETE FROM image_favorites WHERE root_key = ? AND relative_path = ?"
)
COPY_MEMBERSHIPS_SQL = (
"INSERT OR IGNORE INTO image_collection_items "
"(collection_id, root_key, relative_path, added_at) "
"SELECT collection_id, ?, ?, added_at FROM image_collection_items "
"WHERE root_key = ? AND relative_path = ?"
)
DELETE_MEMBERSHIPS_SQL = (
"DELETE FROM image_collection_items WHERE root_key = ? AND relative_path = ?"
)
COPY_BYTE_SQL = (
f"SELECT b.id FROM {FILES_TABLE} f JOIN {BYTES_TABLE} b " # nosec B608
"ON b.file_id=f.id WHERE f.locator=? AND b.sha256=?"
@@ -77,6 +88,8 @@ FILE_LINK_SQL = (
DELETE_LINK_SQL,
FAVORITE_SQL,
DELETE_FAVORITE_SQL,
COPY_MEMBERSHIPS_SQL,
DELETE_MEMBERSHIPS_SQL,
COPY_BYTE_SQL,
COPY_TAG_COUNT_SQL,
PENDING_COUNT_SQL,
@@ -140,14 +153,17 @@ class DuplicateCopyOperations:
by), ``targets`` (``{removed spelling: kept spelling}``, the spelling
a moved record takes; else ``kept_path``), ``favorites`` (``(root key,
removed key, kept root key, kept key)`` per listed root holding the
file), ``removed_locator``,
``kept_locator``, ``sha256`` and the kept copy's ``kept_size`` and
``kept_mtime_ns``; ``categories``: ``(removed file key, kept file
key)`` pairs of the image categories (#332). Returns ``{moved,
merged, favorites, tags, review, categories}`` (``categories``: the
image categories copied to the kept copy).
file; they move its favorite and its collection memberships),
``removed_locator``, ``kept_locator``, ``sha256`` and the kept copy's
``kept_size`` and ``kept_mtime_ns``; ``categories``: ``(removed file
key, kept file key)`` pairs of the image categories (#332). Returns
``{moved, merged, favorites, tags, review, collections, categories}``
(``collections``: the memberships inserted on the kept copy;
``categories``: the image categories copied to the kept copy).
"""
counts = dict.fromkeys(("moved", "merged", "favorites", "tags", "review"), 0)
counts = dict.fromkeys(
("moved", "merged", "favorites", "tags", "review", "collections"), 0
)
kept_keys = [normalize_image_path(path) for path in move["kept"]]
with self._pair_transaction() as conn:
for spelled in dict.fromkeys(move["removed"]):
@@ -180,6 +196,7 @@ class DuplicateCopyOperations:
counts["moved"] += 1
for favorite in move.get("favorites", ()):
counts["favorites"] += _move_favorite(conn, *favorite)
counts["collections"] += _move_memberships(conn, *favorite)
counts["tags"] = self._move_copy_tags(conn, move)
counts["categories"] = sum(
copy_image_category(conn, removed, kept)
@@ -224,3 +241,12 @@ def _move_favorite(conn, root_key, removed_key, kept_root, kept_key):
conn.execute(INSERT_FAVORITE_SQL, (kept_root, kept_key, row[0]))
conn.execute(DELETE_FAVORITE_SQL, (root_key, removed_key))
return 1
def _move_memberships(conn, root_key, removed_key, kept_root, kept_key):
"""Copy a removed copy's memberships to the kept copy; the rows inserted."""
inserted = conn.execute(
COPY_MEMBERSHIPS_SQL, (kept_root, kept_key, root_key, removed_key)
).rowcount
conn.execute(DELETE_MEMBERSHIPS_SQL, (root_key, removed_key))
return inserted
+490
View File
@@ -0,0 +1,490 @@
"""Collections (#331): named, hand-picked groups of image files.
A member names its file exactly as a favorite does (``image_favorites.py``):
the source folder's real, case-normalized path (``root_key``) and the path
below it (``relative_path``), both server-side only. A member whose file is
moved, renamed or deleted outside PromptManager is kept and simply not listed;
nothing follows a file to a new place.
Names are trimmed with each run of whitespace collapsed to one space, 1 to 100
characters without control characters; ``name_key`` (NFC of the case-folded
NFC name) is UNIQUE, so names are unique ignoring case and Unicode form.
The two tables install like ``image_favorites`` and ``saved_views``: optional
in old backups, required once a database is current, created empty with no
backfill. Startup checks only the structure, so a value saved under older
rules never stops the library from opening; restore checks the values too,
but never recomputes ``name_key`` (case tables differ between Python
versions).
"""
import re
import sqlite3
import unicodedata
from typing import Optional
from .image_favorites import MAX_KEY_LENGTH
COLLECTIONS_TABLE = "image_collections"
ITEMS_TABLE = "image_collection_items"
FILE_INDEX = "idx_image_collection_items_file"
TABLES = (COLLECTIONS_TABLE, ITEMS_TABLE)
MAX_NAME_LENGTH = 100
_TIMESTAMP_BYTES = 64
# A name key's byte bound: case folding expands a character to at most
# three, each at most 4 UTF-8 bytes.
_MAX_KEY_BYTES = MAX_NAME_LENGTH * 3 * 4
_WHITESPACE = re.compile(r"\s+")
_TABLE_DDL = {
COLLECTIONS_TABLE: (
f"CREATE TABLE {COLLECTIONS_TABLE} (id INTEGER PRIMARY KEY AUTOINCREMENT, "
"name TEXT NOT NULL, name_key TEXT NOT NULL UNIQUE, "
"created_at TEXT NOT NULL)"
),
ITEMS_TABLE: (
f"CREATE TABLE {ITEMS_TABLE} (collection_id INTEGER NOT NULL REFERENCES "
f"{COLLECTIONS_TABLE}(id) ON DELETE CASCADE, root_key TEXT NOT NULL, "
"relative_path TEXT NOT NULL, added_at TEXT NOT NULL, "
"PRIMARY KEY (collection_id, root_key, relative_path))"
),
}
_INDEX_DDL = f"CREATE INDEX {FILE_INDEX} ON {ITEMS_TABLE}(root_key, relative_path)"
# (name, declared type, not null, default, primary key position)
_COLUMNS = {
COLLECTIONS_TABLE: (
("id", "INTEGER", 0, None, 1),
("name", "TEXT", 1, None, 0),
("name_key", "TEXT", 1, None, 0),
("created_at", "TEXT", 1, None, 0),
),
ITEMS_TABLE: (
("collection_id", "INTEGER", 1, None, 1),
("root_key", "TEXT", 1, None, 2),
("relative_path", "TEXT", 1, None, 3),
("added_at", "TEXT", 1, None, 0),
),
}
# (column, parent table, parent column, on delete)
_FOREIGN_KEYS = {
COLLECTIONS_TABLE: set(),
ITEMS_TABLE: {("collection_id", COLLECTIONS_TABLE, "id", "CASCADE")},
}
class CollectionInvalid(ValueError):
"""A name, id or target breaks the write rules (a 400)."""
def _has_control(text: str) -> bool:
"""Any Unicode control character (category Cc: C0, DEL and C1)."""
return any(unicodedata.category(char) == "Cc" for char in text)
def normalize_collection_name(name: object) -> str:
"""The stored spelling of *name*: trimmed, whitespace collapsed, checked.
Raises ``CollectionInvalid`` unless the result is 1 to 100 characters
without control characters.
"""
if not isinstance(name, str):
raise CollectionInvalid("Name must be a string")
collapsed = _WHITESPACE.sub(" ", name).strip()
if not 1 <= len(collapsed) <= MAX_NAME_LENGTH:
raise CollectionInvalid(f"Name must be 1 to {MAX_NAME_LENGTH} characters")
if _has_control(collapsed):
raise CollectionInvalid("Name contains control characters")
return collapsed
def collection_name_key(name: str) -> str:
"""The uniqueness key of a name: NFC of the case-folded NFC spelling."""
return unicodedata.normalize("NFC", unicodedata.normalize("NFC", name).casefold())
def _compact(sql: str) -> str:
return re.sub(r"\s+", "", sql).lower()
def _expected_catalog() -> set:
objects = {("table", name, _compact(sql)) for name, sql in _TABLE_DDL.items()}
objects.add(("index", FILE_INDEX, _compact(_INDEX_DDL)))
objects |= {("index", f"sqlite_autoindex_{table}_1", "") for table in TABLES}
return objects
def _catalog(conn: sqlite3.Connection) -> list:
# sqlite_sequence (AUTOINCREMENT) is shared with other tables and ignored.
return conn.execute(
"SELECT type,name,sql FROM sqlite_master "
"WHERE name IN (?,?,?) OR tbl_name IN (?,?)",
(*TABLES, FILE_INDEX, *TABLES),
).fetchall()
def _structure_error(conn: sqlite3.Connection) -> Optional[str]:
for table, expected in _COLUMNS.items():
columns = tuple(
(row[1], row[2].upper(), row[3], row[4], row[5])
for row in conn.execute(f"PRAGMA table_info({table})")
)
if columns != expected:
return f"Image collections table {table} has invalid columns"
foreign_keys = {
(row[3], row[2], row[4], row[6])
for row in conn.execute(f"PRAGMA foreign_key_list({table})")
}
if foreign_keys != _FOREIGN_KEYS[table]:
return f"Image collections table {table} has invalid foreign keys"
return None
def _decoded_too_long(conn, sql: str, limit: int) -> bool:
"""Whether a byte-bounded candidate row decodes to over *limit* characters."""
for row in conn.execute(sql):
for raw in row:
if len(bytes(raw).decode("utf-8", "replace")) > limit:
return True
return False
def _value_error(conn: sqlite3.Connection) -> Optional[str]:
# Only this module's constants are interpolated into the SQL below. Bytes
# bound characters from above (UTF-8 is at most 4 bytes each); the few
# candidates over the character bound are measured decoded in Python.
if conn.execute(
f"SELECT 1 FROM {COLLECTIONS_TABLE} WHERE " # nosec B608
"typeof(id) != 'integer' OR typeof(name) != 'text' "
"OR typeof(name_key) != 'text' OR typeof(created_at) != 'text' "
f"OR length(CAST(name AS BLOB)) > {MAX_NAME_LENGTH * 4} "
"OR length(CAST(name_key AS BLOB)) = 0 "
f"OR length(CAST(name_key AS BLOB)) > {_MAX_KEY_BYTES} "
f"OR length(CAST(created_at AS BLOB)) > {_TIMESTAMP_BYTES} LIMIT 1"
).fetchone():
return "Image collections contains invalid values"
# The key is not recomputed: Unicode case tables differ between Python
# versions, and a key written by another version must stay restorable.
# Its UNIQUE constraint keeps names distinct; writes compute it.
for (name,) in conn.execute(f"SELECT name FROM {COLLECTIONS_TABLE}"): # nosec B608
if (
not 1 <= len(name) <= MAX_NAME_LENGTH
or _has_control(name)
or name != name.strip()
):
return "Image collections contains invalid names"
limit = MAX_KEY_LENGTH * 4
if conn.execute(
f"SELECT 1 FROM {ITEMS_TABLE} WHERE " # nosec B608
"typeof(collection_id) != 'integer' OR typeof(root_key) != 'text' "
"OR typeof(relative_path) != 'text' OR typeof(added_at) != 'text' "
f"OR length(CAST(root_key AS BLOB)) > {limit} "
f"OR length(CAST(relative_path AS BLOB)) > {limit} "
f"OR length(CAST(added_at AS BLOB)) > {_TIMESTAMP_BYTES} LIMIT 1"
).fetchone() or _decoded_too_long(
conn,
f"SELECT CAST(root_key AS BLOB), CAST(relative_path AS BLOB) " # nosec B608
f"FROM {ITEMS_TABLE} WHERE length(CAST(root_key AS BLOB)) > {MAX_KEY_LENGTH} "
f"OR length(CAST(relative_path AS BLOB)) > {MAX_KEY_LENGTH}",
MAX_KEY_LENGTH,
):
return "Image collections contains invalid members"
if conn.execute(f"PRAGMA foreign_key_check({ITEMS_TABLE})").fetchone():
return "Image collections contains members of a missing collection"
return None
def image_collections_schema_error(
conn: sqlite3.Connection,
*,
allow_missing: bool = True,
check_values: bool = True,
) -> Optional[str]:
"""Reject partial or unsupported collections schemas and invalid rows.
Reads only: nothing is ever repaired. A database without either table is
acceptable for backups that predate them unless ``allow_missing`` is
false. ``check_values=False`` (live startup) checks the structure only.
"""
objects = _catalog(conn)
if not objects:
return None if allow_missing else "Missing image collections schema"
present = {name for kind, name, _sql in objects if kind == "table"}
if present != set(TABLES):
return "Incomplete image collections schema"
actual = {(kind, name, _compact(sql or "")) for kind, name, sql in objects}
if actual != _expected_catalog():
return "Image collections has an invalid schema definition"
error = _structure_error(conn)
if error or not check_values:
return error
return _value_error(conn)
def migrate_image_collections(conn: sqlite3.Connection) -> None:
"""Install both tables atomically; validate before and after creating them."""
conn.execute("SAVEPOINT image_collections_schema")
try:
error = image_collections_schema_error(conn, check_values=False)
if error:
raise sqlite3.DatabaseError(error)
for sql in _TABLE_DDL.values():
conn.execute(sql.replace("CREATE TABLE", "CREATE TABLE IF NOT EXISTS", 1))
conn.execute(_INDEX_DDL.replace("INDEX", "INDEX IF NOT EXISTS", 1))
error = image_collections_schema_error(
conn, allow_missing=False, check_values=False
)
if error:
raise sqlite3.DatabaseError(error)
conn.execute("RELEASE image_collections_schema")
except BaseException:
conn.execute("ROLLBACK TO image_collections_schema")
conn.execute("RELEASE image_collections_schema")
raise
class CollectionMissing(LookupError):
"""No collection has the id (a 404 ``collection-not-found``)."""
class CollectionNameTaken(ValueError):
"""Another collection already has the name's key (a 409 ``name-taken``)."""
def __init__(self, collection_id: int):
super().__init__("Another collection has this name")
self.collection_id = collection_id
class CollectionFilesMissing(LookupError):
"""A new name with nothing to insert (a 404 ``files-not-found``)."""
LIST_SQL = f"SELECT id, name FROM {COLLECTIONS_TABLE} ORDER BY name_key, id"
GET_SQL = f"SELECT id, name FROM {COLLECTIONS_TABLE} WHERE id = ?"
BY_KEY_SQL = f"SELECT id, name FROM {COLLECTIONS_TABLE} WHERE name_key = ?"
MEMBERS_SQL = (
f"SELECT relative_path FROM {ITEMS_TABLE} "
"WHERE collection_id = ? AND root_key = ?"
)
ROOT_MEMBERS_SQL = (
f"SELECT collection_id, relative_path FROM {ITEMS_TABLE} WHERE root_key = ?"
)
COUNTS_SQL = (
f"SELECT c.id, (SELECT COUNT(*) FROM {ITEMS_TABLE} i "
f"WHERE i.collection_id = c.id) FROM {COLLECTIONS_TABLE} c"
)
FILE_SQL = (
f"SELECT collection_id FROM {ITEMS_TABLE} "
"WHERE root_key = ? AND relative_path = ? ORDER BY collection_id"
)
# Every read, for the EXPLAIN QUERY PLAN check: none scans the items table and
# only the two whole-list reads (LIST_SQL, COUNTS_SQL) scan the collections.
READ_SQL = (
LIST_SQL,
GET_SQL,
BY_KEY_SQL,
MEMBERS_SQL,
ROOT_MEMBERS_SQL,
COUNTS_SQL,
FILE_SQL,
)
_INSERT_ITEM_SQL = f"INSERT OR IGNORE INTO {ITEMS_TABLE} VALUES (?, ?, ?, ?)"
_DELETE_ITEM_SQL = (
f"DELETE FROM {ITEMS_TABLE} "
"WHERE collection_id = ? AND root_key = ? AND relative_path = ?"
)
def _collection_id(value: object) -> int:
if type(value) is not int or not 1 <= value <= 999_999_999:
raise CollectionInvalid("Collection id must be a positive integer")
return value
def _items(items: object) -> list:
"""Distinct ``(root key, relative key)`` pairs, in order."""
if not isinstance(items, (list, tuple)):
raise CollectionInvalid("Items must be a list")
found = []
for item in items:
if (
not isinstance(item, (list, tuple))
or len(item) != 2
or not all(
isinstance(part, str) and 1 <= len(part) <= MAX_KEY_LENGTH
for part in item
)
):
raise CollectionInvalid("Each item is a root key and a relative key")
found.append((item[0], item[1]))
return list(dict.fromkeys(found))
def _row(row) -> dict:
return {"id": row[0], "name": row[1]}
class ImageCollectionOperations:
"""PromptDatabase mixin: collections and their members.
Each write runs in its own ``BEGIN IMMEDIATE`` transaction
(``_pair_transaction``), so two tabs creating one name are serialized.
"""
def list_collections(self) -> list[dict]:
"""Every collection ``{id, name}`` in name-key order, then id."""
rows = self.model.get_connection().execute(LIST_SQL)
return [_row(row) for row in rows]
def get_collection(self, collection_id: int) -> Optional[dict]:
"""``{id, name}`` of one collection, or None."""
row = (
self.model.get_connection()
.execute(GET_SQL, (_collection_id(collection_id),))
.fetchone()
)
return _row(row) if row else None
def collection_exists(self, collection_id: int) -> bool:
return self.get_collection(collection_id) is not None
def collection_members(self, collection_id: int, root_key: str) -> set[str]:
"""The relative keys stored for one collection under one source."""
rows = self.model.get_connection().execute(
MEMBERS_SQL, (_collection_id(collection_id), root_key)
)
return {path for (path,) in rows}
def root_collection_members(self, root_key: str) -> dict[int, set[str]]:
"""``{collection id: relative keys}`` stored under one source."""
found: dict[int, set[str]] = {}
for collection_id, path in self.model.get_connection().execute(
ROOT_MEMBERS_SQL, (root_key,)
):
found.setdefault(collection_id, set()).add(path)
return found
def collection_count_snapshot(self, root_keys: list) -> tuple[dict, dict]:
"""``({root key: {id: relative keys}}, {id: stored rows})`` read in
one read transaction, so a write between the reads cannot skew them.
"""
conn = self.model.get_connection()
if conn.in_transaction:
raise RuntimeError("Collection counts require their own transaction")
conn.execute("BEGIN")
try:
members = {}
for root_key in dict.fromkeys(root_keys):
found: dict[int, set[str]] = {}
for collection_id, path in conn.execute(ROOT_MEMBERS_SQL, (root_key,)):
found.setdefault(collection_id, set()).add(path)
members[root_key] = found
stored = {cid: count for cid, count in conn.execute(COUNTS_SQL)}
finally:
conn.rollback() # it only read
return members, stored
def collection_counts(self) -> dict[int, int]:
"""``{collection id: stored rows}`` for every collection."""
rows = self.model.get_connection().execute(COUNTS_SQL)
return {collection_id: count for collection_id, count in rows}
def file_collections(self, root_key: str, relative_path: str) -> list[int]:
"""Ids of the collections holding one file."""
rows = self.model.get_connection().execute(FILE_SQL, (root_key, relative_path))
return [collection_id for (collection_id,) in rows]
def add_to_collection(
self, *, items: object, collection_id: object = None, name: object = None
) -> dict:
"""Add files to an existing id or a name; create the name when needed.
A name whose key exists uses that collection; otherwise a collection
is created only when at least one item is inserted. Returns
``{collection, created, added, already}``.
"""
from .models import utc_now_iso # models imports this module's schema
if (collection_id is None) == (name is None):
raise CollectionInvalid("Give either a collection id or a name")
pairs = _items(items)
if name is not None:
spelled = normalize_collection_name(name)
else:
collection_id = _collection_id(collection_id)
now = utc_now_iso()
with self._pair_transaction() as conn:
created = False
if name is not None:
row = conn.execute(
BY_KEY_SQL, (collection_name_key(spelled),)
).fetchone()
if row is None:
if not pairs:
raise CollectionFilesMissing("Nothing to add")
cursor = conn.execute(
f"INSERT INTO {COLLECTIONS_TABLE} " # nosec B608
"(name, name_key, created_at) VALUES (?, ?, ?)",
(spelled, collection_name_key(spelled), now),
)
row, created = (cursor.lastrowid, spelled), True
else:
row = conn.execute(GET_SQL, (collection_id,)).fetchone()
if row is None:
raise CollectionMissing("No such collection")
collection = _row(row)
added = sum(
conn.execute(
_INSERT_ITEM_SQL, (collection["id"], root, path, now)
).rowcount
for root, path in pairs
)
return {
"collection": collection,
"created": created,
"added": added,
"already": len(pairs) - added,
}
def remove_from_collection(self, collection_id: object, items: object) -> dict:
"""Remove files from one collection; ``{collection, removed}``."""
collection_id = _collection_id(collection_id)
pairs = _items(items)
with self._pair_transaction() as conn:
row = conn.execute(GET_SQL, (collection_id,)).fetchone()
if row is None:
raise CollectionMissing("No such collection")
removed = sum(
conn.execute(_DELETE_ITEM_SQL, (collection_id, root, path)).rowcount
for root, path in pairs
)
return {"collection": _row(row), "removed": removed}
def rename_collection(self, collection_id: object, name: object) -> dict:
"""The renamed ``{id, name}``; a case variant of its own name is fine."""
collection_id = _collection_id(collection_id)
spelled = normalize_collection_name(name)
key = collection_name_key(spelled)
with self._pair_transaction() as conn:
if conn.execute(GET_SQL, (collection_id,)).fetchone() is None:
raise CollectionMissing("No such collection")
taken = conn.execute(BY_KEY_SQL, (key,)).fetchone()
if taken is not None and taken[0] != collection_id:
raise CollectionNameTaken(taken[0])
conn.execute(
f"UPDATE {COLLECTIONS_TABLE} SET name = ?, name_key = ? " # nosec B608
"WHERE id = ?",
(spelled, key, collection_id),
)
return {"id": collection_id, "name": spelled}
def delete_collection(self, collection_id: object) -> bool:
"""Delete one collection and its members (never a file). True."""
collection_id = _collection_id(collection_id)
with self._pair_transaction() as conn:
cursor = conn.execute(
f"DELETE FROM {COLLECTIONS_TABLE} WHERE id = ?", # nosec B608
(collection_id,),
)
if cursor.rowcount == 0:
raise CollectionMissing("No such collection")
return True
+27 -1
View File
@@ -56,9 +56,11 @@ from .models import BUSY_TIMEOUT_SECONDS, PromptModel
try:
from ..utils import library_location as ll
from ..utils.logging_config import get_logger
from ..utils.workspace_keys import KEY_FILE_NAME
except ImportError:
from utils import library_location as ll
from utils.logging_config import get_logger
from utils.workspace_keys import KEY_FILE_NAME
logger = get_logger("prompt_manager.database.library_move")
@@ -491,7 +493,8 @@ class _Move:
sweep_move_temps(target)
if ll.read_marker(target).get("state") == ll.MARKER_PENDING:
# An earlier move stopped before it switched: its files are ours.
for name in (ll.DATABASE_NAME, ll.CONFIG_NAME) + self.plan.backups:
names = (ll.DATABASE_NAME, ll.CONFIG_NAME, KEY_FILE_NAME)
for name in names + self.plan.backups:
_remove(os.path.join(target, name))
for suffix in ("-wal", "-shm", "-journal"):
_remove(os.path.join(target, ll.DATABASE_NAME + suffix))
@@ -536,6 +539,7 @@ class _Move:
raise MoveFailed("copy-failed") from error
_check_copies([(self.plan.config_path, temp, expected)])
copies = copies + [(self.plan.config_path, temp, expected, ll.CONFIG_NAME)]
copies = copies + self._key_copy()
self.marker = write_marker(
target, ll.MARKER_PENDING, self.plan.db_path, self.plan.config_path
)
@@ -556,6 +560,28 @@ class _Move:
self.checkpoint("marked")
return files
def _key_copy(self) -> list:
"""The workspace key's verified copy (#430), or none: it is re-creatable.
Without it the moved library starts with a new key, so pages left open
across the restart refresh once; that never fails the move.
"""
source = os.path.join(self.plan.source_dir, KEY_FILE_NAME)
if not os.path.isfile(source):
return []
temp = self.temp(".key")
try:
expected = _hash_copy(source, temp)
_check_copies([(source, temp, expected)])
except (OSError, MoveFailed) as error:
_remove(temp)
logger.warning(
f"Library move: the workspace key was not copied ({error}); "
"open workspace pages refresh once after the restart"
)
return []
return [(source, temp, expected, KEY_FILE_NAME)]
def cleanup(self) -> None:
if self.moved:
return
+12 -2
View File
@@ -22,6 +22,10 @@ from .managed_connection import (
from . import restore_stage
from .library_schema import library_schema_error, migrate_library_state
from .image_favorites import image_favorites_schema_error, migrate_image_favorites
from .image_collections import (
image_collections_schema_error,
migrate_image_collections,
)
from .annotation_schema import annotation_schema_error, migrate_annotation_state
from .autotag_schema import (
autotag_schema_error,
@@ -338,6 +342,7 @@ class PromptModel:
migrate_library_state(conn)
migrate_image_favorites(conn)
migrate_saved_views(conn)
migrate_image_collections(conn)
# A review group without its annotation group is refused before the
# annotation install could make it look valid (catalog only; the rows
# are validated once, in migrate_review_state).
@@ -1159,8 +1164,8 @@ class PromptModel:
path: File to inspect
require_library_state: Also require the tables every current
database has (Library state, image favorites, image
categories, saved views, image annotations and the review
queue), as a migrated
categories, saved views, collections, image annotations and
the review queue), as a migrated
restore stage must; old backups may lack them.
documents: Read every workflow document and refuse a damaged
one. A restore checks its upload without it and repairs
@@ -1234,6 +1239,11 @@ class PromptModel:
)
if views_error:
return False, views_error
collections_error = image_collections_schema_error(
conn, allow_missing=not require_library_state
)
if collections_error:
return False, collections_error
annotation_error = annotation_schema_error(
conn, allow_missing=not require_library_state
)
+186 -41
View File
@@ -7,12 +7,18 @@ import json
import math
import datetime
import os
from typing import Optional, List, Dict, Any, Set, Tuple
from typing import Any, Dict, Iterable, List, Optional, Sequence, Set, Tuple
try:
from ..utils.inspection_response import inherited_category
except ImportError:
from utils.inspection_response import inherited_category
from .models import PromptModel, normalize_image_path, utc_now_iso
from .pairs import PairDependencyError, PairOperations # noqa: F401 (re-exported)
from .library import LibraryOperations, titled
from .image_favorites import ImageFavoriteOperations
from .image_collections import ImageCollectionOperations
from .annotations import AnnotationOperations
from .review import ReviewOperations
from .review_decisions import ReviewDecisionOperations
@@ -238,12 +244,17 @@ LEFT JOIN prompt_library_state pls ON pls.prompt_id = pp.positive_prompt_id
WHERE gi.image_path IN ({marks}) OR gi.file_path IN ({marks}){fallback}"""
# One image row describes a card only when no other row claims the same file
# (as in get_image_link_summaries, several rows are ambiguous). The pair scope
# and the linked positive follow _LINK_SUMMARY_SQL; category, rating, tags and
# the searchable text and titles come from that positive (spec D12/D13). Every
# and the linked positive follow _LINK_SUMMARY_SQL; rating, tags and the
# searchable text and titles come from that positive (spec D12/D13). The raw
# recipe and positive categories are read for the inherited category, which
# Python decides (``inherited_category``, #332 MC4); SQL never filters by
# category. Every
# writer since the 3.2.4 migration stores file_path, so it is the path key.
# The prompt-level conditions are uncorrelated IN sets SQLite builds once, and
# the CASE evaluates the per-row ambiguity probe only for admitted rows.
_LINK_FILTER_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id
_LINK_FILTER_SQL = """SELECT gi.file_path AS file_path, p.rating AS rating,
{hit} AS matched, p.id AS positive, pp.id IS NOT NULL AS has_pair,
pp.category AS pair_category, p.category AS category
FROM generated_images gi
LEFT JOIN prompt_pairs pp ON pp.id = (
SELECT CASE WHEN COUNT(DISTINCT ip.pair_id) = 1 THEN MIN(ip.pair_id) END
@@ -252,10 +263,31 @@ JOIN prompts p ON p.id = COALESCE(pp.positive_prompt_id, gi.prompt_id)
WHERE gi.file_path IS NOT NULL AND CASE WHEN {admit} THEN NOT EXISTS (
SELECT 1 FROM generated_images other
WHERE other.file_path = gi.file_path AND other.id <> gi.id) ELSE 0 END"""
# _LINK_FILTER_SQL's rows as one JSON array, built by SQLite in one step (#354).
# Python's sqlite3 releases and retakes the GIL around every step, so a read
# that steps once per linked row waits on a busy ComfyUI thread thousands of
# times. A BLOB value (no writer stores one) makes the value NULL (or
# json_array refuses it), and the read streams instead. The raw categories are
# put in only for a category read (#332), so plain reads stay as small as #354
# measured them.
_LINK_MATCHES_JSON_SQL = """SELECT CASE
WHEN max({blob}) THEN NULL
ELSE json_group_array(json_array({columns})) END
FROM ({select})"""
_LINK_PLAIN_COLUMNS = ("file_path", "rating", "matched", "positive")
_LINK_CATEGORY_COLUMNS = _LINK_PLAIN_COLUMNS + ("has_pair", "pair_category", "category")
# The columns a BLOB can reach: matched, positive and has_pair are computed.
_LINK_STORED_COLUMNS = ("file_path", "rating", "pair_category", "category")
# Set when this SQLite has no JSON functions: every later read streams.
_one_step_unsupported = False
# One warning per process when a read falls back to streaming.
_one_step_warned = False
# The linked positive of each given path (one unambiguous link, pair scope as
# in _LINK_FILTER_SQL) with its category; {hit} as in link_filter_query. Used
# for the few files whose current image tags meet a Tags filter (#249 B3).
_LINK_FACTS_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id, p.category
# in _LINK_FILTER_SQL) with the raw categories of its inherited category; {hit}
# as in link_filter_query. Used for the few files whose current image tags meet
# a Tags filter (#249 B3).
_LINK_FACTS_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id,
pp.id IS NOT NULL, pp.category, p.category
FROM generated_images gi
LEFT JOIN prompt_pairs pp ON pp.id = (
SELECT CASE WHEN COUNT(DISTINCT ip.pair_id) = 1 THEN MIN(ip.pair_id) END
@@ -284,8 +316,24 @@ _LINK_FILTER_TAG = (
)
# One unambiguously linked image's facts for the Images filters, a plain tuple
# (hit, rating, positive id, inherited category): built per linked row, so no
# NamedTuple. The category is the inherited one (#332 MC4: the recipe's when
# set, else the positive's), never the image's own category.
LinkMatch = Tuple[bool, Optional[int], int, Any]
def _link_match(rating, matched, positive, has_pair, pair_category, category):
inherited, _scope = inherited_category(category, pair_category, bool(has_pair))
return (bool(matched), rating, positive, inherited)
def link_filter_query(
q: str, category: Optional[str], tags: List[str], *, every_link: bool = False
q: str,
tags: List[str],
*,
every_link: bool = False,
category: Optional[str] = None,
) -> Tuple[str, Dict[str, Any]]:
"""The workspace filter SELECT and its named parameters (all bound).
@@ -299,7 +347,9 @@ def link_filter_query(
params["pattern"] = f"%{escape_like(q)}%"
admitted = []
if category is not None:
admitted.append("p.category = :category")
# Only narrows: an inherited category is one of the two raw values;
# which one is decided in Python (inherited_category, #332 MC4).
admitted.append("(p.category = :category OR pp.category = :category)")
params["category"] = category
for index, tag in enumerate(tags):
admitted.append(_LINK_FILTER_TAG.format(index=index))
@@ -310,6 +360,50 @@ def link_filter_query(
return sql, params
def link_matches_json_query(sql: str, categories: bool = False) -> str:
"""*sql* from link_filter_query, wrapped to return its rows as one JSON value.
With *categories* each row also carries the raw values the inherited
category is decided from; otherwise only path, rating, hit and positive.
"""
columns = _LINK_CATEGORY_COLUMNS if categories else _LINK_PLAIN_COLUMNS
blob = " OR ".join(
f"typeof({name}) = 'blob'" for name in _LINK_STORED_COLUMNS if name in columns
)
return _LINK_MATCHES_JSON_SQL.format(
blob=blob, columns=", ".join(columns), select=sql
)
def _link_matches_from_rows(
rows: Iterable[Sequence[Any]], categories: bool, wanted: Optional[Set[str]]
) -> Dict[str, LinkMatch]:
"""LinkMatch per wanted path from link rows (one-step or streamed).
Without *categories* the inherited category is None, as the caller then
never compares it.
"""
matches: Dict[str, LinkMatch] = {}
for path, rating, matched, positive, *raw in rows:
if wanted is None or path in wanted:
if categories:
matches[path] = _link_match(rating, matched, positive, *raw[:3])
else:
matches[path] = (bool(matched), rating, positive, None)
return matches
def _streamed_link_matches(
conn: sqlite3.Connection,
sql: str,
params: Dict[str, Any],
wanted: Optional[Set[str]],
categories: bool = False,
) -> Dict[str, LinkMatch]:
"""The link read stepped once per row (the fallback of the one-step read)."""
return _link_matches_from_rows(conn.execute(sql, params), categories, wanted)
_ASCII_FOLD = str.maketrans("ABCDEFGHIJKLMNOPQRSTUVWXYZ", "abcdefghijklmnopqrstuvwxyz")
@@ -360,6 +454,7 @@ class PromptDatabase(
ExecutionOutputOperations,
HistoryReadOperations,
ImageFavoriteOperations,
ImageCollectionOperations,
AnnotationOperations,
ReviewOperations,
ReviewDecisionOperations,
@@ -2543,38 +2638,94 @@ class PromptDatabase(
def get_image_link_matches(
self,
q: str,
category: Optional[str],
tags: List[str],
*,
every_link: bool = False,
wanted: Optional[Set[str]] = None,
) -> Dict[str, Tuple[bool, Optional[int], int]]:
"""Map normalized paths of unambiguously linked images to
(hit, rating, positive id).
category: Optional[str] = None,
) -> Dict[str, LinkMatch]:
"""Map normalized paths of unambiguously linked images to a LinkMatch
(hit, rating, positive id, inherited category).
Rows are limited to images whose linked positive has *category* and
every tag in *tags* (exact names). ``hit`` says whether *q* occurs in
Rows are limited to images whose linked positive has every tag in
*tags* (exact names). The category filter is applied by the caller to
the effective category (#332). ``hit`` says whether *q* occurs in
the positive's text, Library title or a tag name; a pair's own title
names its negative variant and is not searched.
Without category or tags only hits are returned, unless *every_link*
Without tags only hits are returned, unless *every_link*
asks for every linked image (the rating sort and the positive count).
The positive id is the card's linked positive. One SELECT whose rows
are streamed; with *wanted* (the listing's normalized paths) only those
paths are kept, so memory is bounded by the listing, not the database.
The positive id is the card's linked positive. With *category* only
links whose raw recipe or positive category is that value are read (a
superset of the links that inherit it; the caller compares the
inherited value) and the inherited category is decided per row;
otherwise it is None, which keeps the plain reads cheap.
One SELECT, returned as one JSON value (#354): SQLite steps once, so
the read takes the GIL back a constant two or three times instead of
once per linked row while ComfyUI generates. Peak memory therefore
grows with the database's matching linked rows (about 530 bytes each,
more with the raw categories of a category read, freed on return); the
result is bounded by *wanted* (the listing's normalized paths) when
given. Where the JSON step fails (no JSON functions, a BLOB value, a
JSON text over SQLite's length limit) the rows are streamed as before.
"""
sql, params = link_filter_query(q, category, tags, every_link=every_link)
matches: Dict[str, Tuple[bool, Optional[int], int]] = {}
sql, params = link_filter_query(
q, tags, every_link=every_link, category=category
)
categories = category is not None
with self.model.get_connection() as conn:
for path, rating, matched, positive in conn.execute(sql, params):
if wanted is None or path in wanted:
matches[path] = (bool(matched), rating, positive)
return matches
rows = None
if not _one_step_unsupported:
rows = self._one_step_link_rows(conn, sql, params, categories)
if rows is None:
return _streamed_link_matches(conn, sql, params, wanted, categories)
return _link_matches_from_rows(rows, categories, wanted)
def _one_step_link_rows(
self,
conn: sqlite3.Connection,
sql: str,
params: Dict[str, Any],
categories: bool = False,
) -> Optional[List[List[Any]]]:
"""*sql*'s rows from one JSON value, or None when this read must stream.
An interrupted read (Move library retiring the connection) is raised
as before, never retried row by row.
"""
global _one_step_unsupported, _one_step_warned
error: Optional[str] = None
try:
cursor = conn.execute(link_matches_json_query(sql, categories), params)
try:
((document,),) = cursor.fetchall()
finally:
cursor.close() # resets the statement: its snapshot ends now
except (sqlite3.OperationalError, sqlite3.DataError) as failure:
# DataError: the JSON text is longer than SQLITE_LIMIT_LENGTH.
error = str(failure)
if error == "interrupted":
raise
if error.startswith("no such function"):
_one_step_unsupported = True
else:
if document is not None:
return json.loads(document)
error = "a BLOB value"
if not _one_step_warned:
_one_step_warned = True
self.logger.warning(
"Reading image links row by row (slower while ComfyUI "
"generates); the one-step read failed: %s",
error,
)
return None
def get_image_link_facts(
self, paths: List[str], q: str = ""
) -> Dict[str, Tuple[bool, Optional[int], int, Optional[str], Tuple[str, ...]]]:
) -> Dict[str, Tuple[bool, Optional[int], int, Any, Tuple[str, ...]]]:
"""Map each normalized path with one unambiguous link to its positive's
(hit, rating, positive id, category, tag names).
(hit, rating, positive id, inherited category, tag names).
The link is resolved as in get_image_link_matches, and ``hit`` says
whether *q* occurs in the positive's text, Library title or a tag
@@ -2585,7 +2736,7 @@ class PromptDatabase(
if q:
hit = _LINK_FILTER_HIT
params["pattern"] = f"%{escape_like(q)}%"
rows: Dict[str, Tuple[bool, Optional[int], int, Optional[str]]] = {}
rows: Dict[str, LinkMatch] = {}
names: Dict[int, List[str]] = {}
unique = list(dict.fromkeys(paths))
with self.model.get_connection() as conn:
@@ -2594,10 +2745,8 @@ class PromptDatabase(
bound = {f"path{i}": path for i, path in enumerate(chunk)}
marks = ",".join(f":{key}" for key in bound)
sql = _LINK_FACTS_SQL.format(hit=hit, marks=marks)
for path, rating, matched, positive, category in conn.execute(
sql, {**params, **bound}
):
rows[path] = (bool(matched), rating, positive, category)
for path, *row in conn.execute(sql, {**params, **bound}):
rows[path] = _link_match(*row)
positives = sorted({row[2] for row in rows.values()})
for start in range(0, len(positives), _FACTS_CHUNK):
chunk = positives[start : start + _FACTS_CHUNK]
@@ -2610,20 +2759,16 @@ class PromptDatabase(
path: (*row, tuple(names.get(row[2], ()))) for path, row in rows.items()
}
def get_image_paths_matching(
self, q: str, category: Optional[str], tags: List[str]
) -> Set[str]:
"""Normalized paths whose linked positive matches *q*, *category* and *tags*.
def get_image_paths_matching(self, q: str, tags: List[str]) -> Set[str]:
"""Normalized paths whose linked positive matches *q* and *tags*.
See get_image_link_matches; an empty *q* matches every linked image
the category and tags admit.
the tags admit.
"""
return {
path
for path, (hit, _rating, _positive) in self.get_image_link_matches(
q, category, tags
).items()
if hit or not q
for path, match in self.get_image_link_matches(q, tags).items()
if match[0] or not q
}
def get_prompt_id_for_image(self, image_path: str) -> Optional[int]:
+111 -15
View File
@@ -1,7 +1,14 @@
"""Atomic metadata enrichment for a single known generation context."""
"""Atomic metadata enrichment for a single known generation context.
Also the helpers both ingestion store paths share (#247): the file's link
rows and the one-prompt-per-file rule (``file_link_rows``,
``link_conflict``) and the marks a prompt created by ingestion carries
(``Marks``, ``mark_created``).
"""
import os
from typing import Any, Dict
from dataclasses import dataclass
from typing import Any, Dict, List, Optional, Tuple
from .models import normalize_image_path
from .operations import _coerce_file_info
@@ -37,16 +44,85 @@ def blank_positive(resolution: Dict[str, Any]) -> bool:
return not (isinstance(text, str) and text.strip())
@dataclass(frozen=True)
class Marks:
"""What a prompt ingestion creates carries, so it can be found and removed.
``note`` is formatted with ``name``, the file's name.
"""
category: str
tags: Tuple[str, ...]
note: str
SCAN_MARKS = Marks("scanned", ("auto-scanned",), "Auto-scanned from {name}")
RUN_MARKS = Marks("scanned", ("auto-scanned",), "Saved from a workflow run: {name}")
def mark_created(conn: Any, db: Any, prompt_id: int, marks: Marks, name: str) -> None:
"""Apply ``marks`` to a prompt row the caller just created on ``conn``."""
conn.execute(
"UPDATE prompts SET category = ?, notes = ? WHERE id = ?",
(marks.category, marks.note.format(name=name), prompt_id),
)
db._sync_prompt_tags(conn, prompt_id, list(marks.tags))
def file_link_rows(conn: Any, image_path: str) -> List[Any]:
"""The file's link rows, each with its prompt's hash.
Matched under the path and its resolved spelling (an output folder
reached through a symlink or junction), each normalized and raw;
deduplicated by row id.
"""
rows: Dict[Any, Any] = {}
for spelling in dict.fromkeys((image_path, _resolved(image_path))):
for row in conn.execute(
"SELECT gi.*,p.hash AS positive_hash FROM generated_images gi "
"JOIN prompts p ON p.id=gi.prompt_id "
"WHERE gi.file_path=? OR gi.image_path=?",
(normalize_image_path(spelling), spelling),
).fetchall():
rows.setdefault(row["id"], row)
return list(rows.values())
def _resolved(path: str) -> str:
try:
return os.path.realpath(path)
except (OSError, ValueError):
return path
def link_conflict(rows: List[Any], prompt_hash: str) -> bool:
"""One prompt per file: more than one row, or a row of another prompt."""
return len(rows) > 1 or bool(rows and rows[0]["positive_hash"] != prompt_hash)
def ingest_resolved_pair(
db: Any, image_path: str, metadata: Dict[str, Any], resolution: Dict[str, Any]
db: Any,
image_path: str,
metadata: Dict[str, Any],
resolution: Dict[str, Any],
*,
marks: Optional[Marks] = None,
) -> Dict[str, Any]:
"""Store a resolved pair and link the file to it in one transaction.
With ``marks`` (ingestion from files), each pair component this call
creates carries them; without (the PromptManager node path) components
are created bare, as always.
"""
state = resolution.get("state", "unresolved")
if state == "resolved" and blank_positive(resolution):
state = "unresolved"
if state != "resolved":
return {"status": state, "pairs_added": 0, "links_added": 0}
with db._pair_transaction() as conn:
return _ingest_resolved_pair_in(conn, db, image_path, metadata, resolution)
return _ingest_resolved_pair_in(
conn, db, image_path, metadata, resolution, marks=marks
)
def _conflict() -> Dict[str, Any]:
@@ -61,6 +137,7 @@ def _ingest_resolved_pair_in(
resolution: Dict[str, Any],
*,
provenance: Dict[str, Any] | None = None,
marks: Optional[Marks] = None,
) -> Dict[str, Any]:
"""``ingest_resolved_pair`` inside the caller's open ``_pair_transaction``.
@@ -69,7 +146,8 @@ def _ingest_resolved_pair_in(
negative is a ``conflict`` and nothing is written. The caller owns the
transaction, so its own later failure rolls the link back too.
``provenance`` replaces the link's default resolver provenance (Needs
review records the user's selection there).
review records the user's selection there). ``marks`` as in
``ingest_resolved_pair``.
"""
positive, negative = resolution["positive"], resolution["negative"]
positive_text = positive.get("storage_text", positive["text"])
@@ -82,15 +160,8 @@ def _ingest_resolved_pair_in(
"negative": negative,
}
encoded = dumps_strict(provenance, sort_keys=True, separators=(",", ":"))
images = conn.execute(
"SELECT gi.*,p.hash AS positive_hash FROM generated_images gi "
"JOIN prompts p ON p.id=gi.prompt_id "
"WHERE gi.file_path=? OR gi.image_path=?",
(normalized, image_path),
).fetchall()
if len(images) > 1 or (
images and images[0]["positive_hash"] != generate_prompt_hash(positive_text)
):
images = file_link_rows(conn, image_path)
if link_conflict(images, generate_prompt_hash(positive_text)):
return _conflict()
existing = images[0] if images else None
if existing:
@@ -115,7 +186,12 @@ def _ingest_resolved_pair_in(
generate_prompt_hash(negative_text) if negative_text.strip() else None,
),
).fetchone()
pair_id = db._save_pair(conn, positive_text, negative_text)
if marks is None:
pair_id = db._save_pair(conn, positive_text, negative_text)
else:
pair_id = _save_marked_pair(
conn, db, positive_text, negative_text, marks, os.path.basename(image_path)
)
pair = conn.execute("SELECT * FROM prompt_pairs WHERE id=?", (pair_id,)).fetchone()
pairs_added = int(prior_pair is None)
file_size, width, height, image_format = _coerce_file_info(metadata)
@@ -197,6 +273,26 @@ def _ingest_resolved_pair_in(
}
def _save_marked_pair(
conn: Any,
db: Any,
positive_text: str,
negative_text: str,
marks: Marks,
name: str,
) -> int:
"""``_save_pair`` that marks each component it creates."""
texts = [positive_text] + ([negative_text] if negative_text.strip() else [])
ids = []
for text in texts:
prompt_id, created = db._pair_component_outcome(conn, text)
if created:
mark_created(conn, db, prompt_id, marks, name)
ids.append(prompt_id)
negative_id = ids[1] if len(ids) > 1 else None
return db._save_pair_components(conn, ids[0], negative_id)[0]
def _document_form(conn: Any, values: Dict[str, Any]) -> Dict[str, Any]:
"""``values`` with the workflow and prompt graph texts stored once.
+8 -4
View File
@@ -59,6 +59,7 @@ SUMMARY_KEYS = (
"image_links",
"runs",
"saved_views",
"collections",
)
@@ -144,7 +145,8 @@ def library_summary(conn: sqlite3.Connection) -> Dict[str, int]:
``positives`` are the Library's prompts (``library_counts``'s
``positive_prompts``); ``pairs`` the active saved pairs and
``archived_pairs`` the archived ones; ``image_links`` the indexed image
rows; ``runs`` the recorded executions.
rows; ``runs`` the recorded executions; ``collections`` the image
collections (#331).
"""
from .library import _ELIGIBLE
@@ -169,6 +171,7 @@ def library_summary(conn: sqlite3.Connection) -> Dict[str, int]:
"image_links": _count(conn, "generated_images"),
"runs": _count(conn, "execution_runs"),
"saved_views": _count(conn, "saved_views"),
"collections": _count(conn, "image_collections"),
}
@@ -180,6 +183,7 @@ _FINGERPRINT_SQL = (
("prompt_pairs", "SELECT MAX(id), MAX(updated_at) FROM prompt_pairs"),
("execution_runs", "SELECT COUNT(*), MAX(rowid) FROM execution_runs"),
("saved_views", "SELECT MAX(rowid) FROM saved_views"),
("image_collections", "SELECT MAX(rowid) FROM image_collections"),
)
@@ -190,9 +194,9 @@ def live_fingerprint(conn: sqlite3.Connection):
tables a restore replaces, or when a prompt, pair, run or saved view is
the newest edited (counts plus the newest row and edit of each table);
the apply refuses with ``plan-changed`` then. It does not see every
edit (a tag or favorite change on an older row); those are kept in the
restore's safety backup. A read transaction the caller already has is
used and left open; one opened here is ended here.
edit (a tag, favorite or collection membership change on an older row);
those are kept in the restore's safety backup. A read transaction the
caller already has is used and left open; one opened here is ended here.
"""
import json
+7 -6
View File
@@ -164,7 +164,7 @@ def _runs_by_location(db: Any, names: List[str]) -> Optional[Dict[tuple, dict]]:
return found
def _recorded_runs(
def recorded_runs(
db: Any,
image_paths: Iterable[str],
output_root: Optional[str],
@@ -201,7 +201,8 @@ def _recorded_runs(
return attributed
def _named_prompt(db: Any, run_id: str) -> Optional[int]:
def named_prompt(db: Any, run_id: str) -> Optional[int]:
"""The one stored prompt every positive context of ``run_id`` names, else None."""
contexts = db.get_execution_contexts(run_id).get("contexts") or []
named = {
context.get("positive_prompt_id")
@@ -228,11 +229,11 @@ def stored_run_prompts(
paths = list(dict.fromkeys(path for path in image_paths if path))
result: Dict[str, Optional[int]] = dict.fromkeys(paths)
prompts: Dict[str, Optional[int]] = {}
for path, (run_id, _nodes) in _recorded_runs(
for path, (run_id, _nodes) in recorded_runs(
db, paths, output_root, path_style
).items():
if run_id not in prompts:
prompts[run_id] = _named_prompt(db, run_id)
prompts[run_id] = named_prompt(db, run_id)
result[path] = prompts[run_id]
return result
@@ -275,7 +276,7 @@ def recorded_output(
) -> Optional[RecordedOutput]:
"""The recorded run facts for ``image_path``, else None (#386).
The file must be attributed to one run (``_recorded_runs``: one run
The file must be attributed to one run (``recorded_runs``: one run
reported its location and it was written during that run). Read-only; it
never links anything.
"""
@@ -286,7 +287,7 @@ def recorded_output(
if own_transaction: # outputs and texts from one state of history
conn.execute("BEGIN")
try:
found = _recorded_runs(db, [image_path], output_root, path_style)
found = recorded_runs(db, [image_path], output_root, path_style)
if image_path not in found:
return None
run_id, nodes = found[image_path]
+14
View File
@@ -35,6 +35,9 @@ _LIBRARY_SORTS = ("newest", "oldest", "name", "rating")
_MEDIA = ("all", "image", "video")
_CONTROL = re.compile(r"[\x00-\x1f\x7f]")
_SOURCE = re.compile(r"all|0|[1-9][0-9]{0,8}")
# A collection id as the listing's ``collection`` query takes it (#331): none
# ("") or 1 to 9 digits without a leading zero.
_COLLECTION = re.compile(r"|[1-9][0-9]{0,8}")
FALLBACK_NAME = "Saved view"
@@ -55,6 +58,9 @@ _STATE_FIELDS = {
"tags": ("tags", None, ()),
# The Favorites page's filter (#249 D51): a view saved there reopens it.
"favorites": ("bool", None, False),
# A view saved on a collection page (#331 D140) reopens that
# collection; never together with favorites.
"collection": ("collection", None, ""),
},
"library": {
"q": ("text", 500, ""),
@@ -125,6 +131,8 @@ def _validate_value(key: str, kind: str, limit, value: object) -> None:
raise SavedViewInvalid(f"State '{key}' is not a supported value")
if kind == "source" and not _SOURCE.fullmatch(value):
raise SavedViewInvalid(f"State '{key}' must be 'all' or a source index")
if kind == "collection" and not _COLLECTION.fullmatch(value):
raise SavedViewInvalid(f"State '{key}' must be empty or a collection id")
if kind == "text" and (len(value) > limit or _CONTROL.search(value)):
raise SavedViewInvalid(f"State '{key}' is too long or has control characters")
@@ -142,6 +150,8 @@ def encode_state(page: object, state: object) -> str:
)
for key, (kind, limit, _default) in fields.items():
_validate_value(key, kind, limit, state[key])
if state.get("favorites") is True and state.get("collection"):
raise SavedViewInvalid("A view cannot filter by favorites and a collection")
encoded = json.dumps(
{key: state[key] for key in fields}, ensure_ascii=False, separators=(",", ":")
)
@@ -181,6 +191,10 @@ def normalize_state(page: str, raw: object) -> dict:
except SavedViewInvalid:
value = default
state[key] = list(value) if kind == "tags" else value
# Never written by this build; a stored row with both opens the collection,
# as the listing does for a URL with both (#331 D140).
if state.get("favorites") is True and state.get("collection"):
state["favorites"] = False
return state
+87
View File
@@ -0,0 +1,87 @@
"""Link an output file to the prompt of one text, creating it when new (#247).
The store step of ingestion from file metadata when the decision is a
positive text (``utils.file_prompt``). The prompt insert and the link run in
one ``BEGIN IMMEDIATE`` transaction: two writers never store one text twice,
a failed link leaves no new prompt behind, and a file stays linked to one
prompt (a file already linked to another prompt is a ``conflict``).
"""
import logging
import os
from typing import Any, Dict, Optional, Tuple
from .pair_ingestion import Marks, file_link_rows, link_conflict, mark_created
try:
from ..utils.hashing import generate_prompt_hash
except ImportError:
from utils.hashing import generate_prompt_hash
_LOGGER = logging.getLogger("prompt_manager.database.scanned_prompts")
def store_scanned_prompt_or_raise(
db: Any, path: str, canonical: Dict[str, Any], text: str, marks: Marks
) -> Tuple[str, Optional[int]]:
"""``(status, prompt id)`` after linking ``path`` to ``text``'s prompt.
``added`` (prompt created, carrying ``marks``), ``linked`` (stored text
reused), ``unchanged`` (already linked to it) or ``conflict`` (linked to
another prompt; nothing written, id None). Database errors are raised
and roll the whole store back, so a caller can run it again.
"""
with db._pair_transaction() as conn:
rows = file_link_rows(conn, path)
if link_conflict(rows, generate_prompt_hash(text)):
return "conflict", None
if rows:
return "unchanged", rows[0]["prompt_id"]
prompt_id, created = db._pair_component_outcome(conn, text)
if created:
mark_created(conn, db, prompt_id, marks, os.path.basename(path))
db._insert_image_link(conn, prompt_id, path, canonical)
return ("added" if created else "linked"), prompt_id
def store_scanned_prompt(
db: Any, path: str, canonical: Dict[str, Any], text: str, marks: Marks
) -> Tuple[str, Optional[int]]:
"""The scan's form: an error is logged and reported as ``("found", None)``.
A write that failed because Move library switched runs again on the new
library (``survive_library_switch``); the rerun re-checks the link rows.
"""
try:
return db.survive_library_switch(
lambda: store_scanned_prompt_or_raise(db, path, canonical, text, marks)
)
except Exception as error:
_LOGGER.error(
f"Failed to store the prompt of {os.path.basename(path)}: {error}"
)
return "found", None
def link_stored_prompt_or_raise(
db: Any, path: str, canonical: Dict[str, Any], prompt_id: int
) -> Optional[str]:
"""Link ``path`` to the stored prompt ``prompt_id`` (recorded history, #342).
``linked``, ``unchanged`` (already linked to it), ``conflict`` (linked to
another prompt; nothing written) or None when the prompt is gone. One
``BEGIN IMMEDIATE`` transaction; database errors are raised.
"""
with db._pair_transaction() as conn:
rows = file_link_rows(conn, path)
if len(rows) > 1 or (rows and rows[0]["prompt_id"] != prompt_id):
return "conflict"
if rows:
return "unchanged"
exists = conn.execute(
"SELECT 1 FROM prompts WHERE id = ?", (prompt_id,)
).fetchone()
if exists is None:
return None
db._insert_image_link(conn, prompt_id, path, canonical)
return "linked"
+55 -15
View File
@@ -13,7 +13,9 @@ same connection before commit. Owner rows and their timestamps never change.
Category consolidation is previewed in one ``library_snapshot`` and applied in
one ``_pair_transaction`` that recomputes the owner projection and requires the
preview's ``impact_token``; only rows carrying the exact source change.
preview's ``impact_token``; only rows carrying the exact source change. Image
rows holding the source (``image_categories``, #332) are owners too: they are
counted, hashed into the token and renamed with the prompts and pairs.
"""
import sqlite3
@@ -97,16 +99,30 @@ def _validate_tag_id(tag_id: object) -> None:
raise ValueError("Invalid tag ID")
# The /library/options universe: nonblank values on eligible parents or on
# active pairs, grouped exactly (whitespace and case variants stay distinct).
_CATEGORY_GROUPS = (
# Category values in Tags & categories (#332 §7): the prompt-backed values
# (nonblank values on eligible parents or on active pairs, grouped exactly, so
# whitespace and case variants stay distinct; /library/options lists the same
# prompt-backed set with its own query) plus the values only image rows hold.
# A prompt-backed value is ordered by its oldest owning positive; an image-only
# value sorts after all of them (first_pid is SQLite's INTEGER maximum, above
# any prompt id), then by value. ``image_count`` counts stored rows, including
# files missing on disk. No FULL OUTER JOIN, so it runs on SQLite 3.31+.
_IMAGE_ONLY_FIRST_PID = 2**63 - 1
_CATEGORY_ROWS = (
"WITH owners(value,pid) AS ("
"SELECT p.category,p.id FROM prompts p WHERE p.category IS NOT NULL "
"AND TRIM(p.category)!='' AND " + _ELIGIBLE + " "
"UNION SELECT pp.category,pp.positive_prompt_id FROM prompt_pairs pp "
"WHERE pp.archived=0 AND pp.category IS NOT NULL AND TRIM(pp.category)!=''),"
"grouped AS (SELECT value,MIN(pid) AS first_pid,"
"COUNT(DISTINCT pid) AS prompt_count FROM owners GROUP BY value) "
"image_values(value,image_count) AS ("
"SELECT category,COUNT(*) FROM image_categories GROUP BY category),"
"grouped(value,first_pid,prompt_count) AS ("
"SELECT value,MIN(pid),COUNT(DISTINCT pid) FROM owners GROUP BY value "
"UNION ALL SELECT value," + str(_IMAGE_ONLY_FIRST_PID) + ",0 FROM image_values "
"WHERE value NOT IN (SELECT value FROM owners)),"
"counted AS (SELECT g.value,g.first_pid,g.prompt_count,"
"COALESCE(i.image_count,0) AS image_count FROM grouped g "
"LEFT JOIN image_values i ON i.value=g.value) "
)
@@ -117,15 +133,23 @@ _CATEGORY_COMPONENTS = (
"WHERE p.category=? ORDER BY p.id"
)
_CATEGORY_PAIRS = "SELECT id,archived FROM prompt_pairs WHERE category=? ORDER BY id"
# Image rows holding one exact value (#332 DC7), an index search.
_IMAGE_CATEGORY_ROWIDS = (
"SELECT rowid FROM image_categories WHERE category=? ORDER BY rowid"
)
_CATEGORY_EXISTS = (
"SELECT EXISTS(SELECT 1 FROM prompts WHERE category=:v)"
" OR EXISTS(SELECT 1 FROM prompt_pairs WHERE category=:v)"
" OR EXISTS(SELECT 1 FROM image_categories WHERE category=:v)"
)
# Pairs after components, so a failing pair update rolls both back.
# Pairs after components and image rows last, so a failing update rolls every
# earlier one back. An image row is keyed by its file, never by its value, so
# merging into a value other files already hold cannot conflict.
_CATEGORY_UPDATES = (
"UPDATE prompts SET category=?,updated_at=? WHERE category=?",
"UPDATE prompt_pairs SET category=?,updated_at=? WHERE category=?",
"UPDATE image_categories SET category=?,updated_at=? WHERE category=?",
)
@@ -136,8 +160,15 @@ def category_impact_token(
target_exists: bool,
components: list,
pairs: list,
*,
images: list = (),
) -> str:
"""Hash a category projection like ``library_edit_token`` (no secret)."""
"""Hash a category projection like ``library_edit_token`` (no secret).
``images`` are the rowids of the image rows holding ``source`` (#332 DC7).
With none the preimage has no ``images`` key, so a prompt-only token keeps
the value it had before image categories existed.
"""
preimage = {
"epoch": epoch,
"source": source,
@@ -146,6 +177,8 @@ def category_impact_token(
"components": sorted([int(i), bool(flag)] for i, flag in components),
"pairs": sorted([int(i), bool(flag)] for i, flag in pairs),
}
if images:
preimage["images"] = sorted(int(i) for i in images)
return canonical_token(_CATEGORY_DOMAIN, preimage)
@@ -325,7 +358,8 @@ class VocabularyOperations:
pairs = [
[row[0], bool(row[1])] for row in conn.execute(_CATEGORY_PAIRS, (source,))
]
if not components and not pairs:
images = [row[0] for row in conn.execute(_IMAGE_CATEGORY_ROWIDS, (source,))]
if not components and not pairs and not images:
raise LibraryCommandError("vocabulary-conflict", epoch)
target_exists = bool(
conn.execute(_CATEGORY_EXISTS, {"v": target}).fetchone()[0]
@@ -339,9 +373,10 @@ class VocabularyOperations:
"other_prompts": len(components) - eligible,
"negative_variants": len(pairs) - archived,
"archived_negative_variants": archived,
"images": len(images),
},
"impact_token": category_impact_token(
epoch, source, target, target_exists, components, pairs
epoch, source, target, target_exists, components, pairs, images=images
),
}
@@ -365,7 +400,8 @@ class VocabularyOperations:
def apply_vocabulary_category(
self, *, expected_epoch: str, source: str, target: str, impact_token: str
) -> dict:
"""Set the exact ``source`` to ``target`` on every component and pair.
"""Set the exact ``source`` to ``target`` on every component, pair
and image row.
The recomputed projection must still match the preview's token; a
mismatch is ``vocabulary-conflict`` with nothing written.
@@ -422,15 +458,15 @@ class VocabularyOperations:
_limit(limit)
key = _category_after(after)
total = conn.execute(
_CATEGORY_GROUPS + "SELECT COUNT(*) FROM grouped"
_CATEGORY_ROWS + "SELECT COUNT(*) FROM counted"
).fetchone()[0]
where, params = "", []
if key is not None:
where = " WHERE first_pid>? OR (first_pid=? AND value>?)"
params = [key[0], key[0], key[1]]
rows = conn.execute(
_CATEGORY_GROUPS
+ "SELECT value,first_pid,prompt_count FROM grouped"
_CATEGORY_ROWS
+ "SELECT value,first_pid,prompt_count,image_count FROM counted"
+ where
+ " ORDER BY first_pid,value LIMIT ?",
[*params, limit + 1],
@@ -440,7 +476,11 @@ class VocabularyOperations:
return {
"library_epoch": epoch,
"categories": [
{"value": row["value"], "prompt_count": row["prompt_count"]}
{
"value": row["value"],
"prompt_count": row["prompt_count"],
"image_count": row["image_count"],
}
for row in rows
],
"total": total,
+13 -6
View File
@@ -26,13 +26,13 @@ except ImportError:
InputTypeDict = dict
try:
from .prompt_manager_base import PromptManagerBase
from .prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
except ImportError:
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from prompt_manager_base import PromptManagerBase
from prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
class PromptManager(PromptManagerBase, ComfyNodeABC):
@@ -115,6 +115,7 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
)
},
),
"save_to_library": SAVE_TO_LIBRARY_INPUT,
},
"hidden": {
# The queued graph and this node's id: used to tell the positive
@@ -152,6 +153,7 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
append_text: str = "",
prompt: Optional[Dict[str, Any]] = None,
unique_id: Optional[str] = None,
save_to_library: bool = True,
) -> Tuple[Any]:
"""
Encode the text prompt and save it to the database.
@@ -164,6 +166,7 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
search_text: Text to search for in past prompts
prepend_text: Text to prepend to the main prompt
append_text: Text to append to the main prompt
save_to_library: Off: nothing is saved, counted or tracked (#90)
Returns:
Tuple containing the conditioning for the diffusion model and the final
@@ -205,7 +208,9 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
# Save prompt to database and set execution context for gallery tracking
prompt_id = None
extended_tags = []
if storage_text and storage_text.strip():
if not save_to_library:
self._skip_not_saved(prompt, unique_id)
elif storage_text and storage_text.strip():
self.logger.debug(f"Processing prompt text: {storage_text[:100]}...")
# Add prepend/append info to tags if they exist
@@ -249,6 +254,11 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
tokens = clip.tokenize(encoding_text)
conditioning = clip.encode_from_tokens_scheduled(tokens)
self.logger.info(f"CLIP encoding completed, text: {repr(encoding_text)[:80]}")
result = (conditioning, encoding_text)
if not save_to_library:
return result # #90: no metadata registration, no history text
# Register with ComfyUI integration for standard metadata compatibility
node_id = f"promptmanager_{int(time.time() * 1000)}"
self.comfyui_integration.register_prompt(
@@ -262,9 +272,6 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
"append_text": append_text.strip() if append_text else None,
},
)
self.logger.info(f"CLIP encoding completed, text: {repr(encoding_text)[:80]}")
result = (conditioning, encoding_text)
record_runtime_context(
type(self).__name__, text, prepend_text, append_text, encoding_text
)
+38 -2
View File
@@ -23,7 +23,7 @@ try:
from .utils.image_monitor import get_image_monitor
from .utils.prompt_tracker import get_prompt_tracker
from .utils.hashing import generate_prompt_hash
from .utils.prompt_graph import resolve_text, run_prompt_nodes
from .utils.prompt_graph import resolve_text, run_prompt_nodes, save_setting
from .utils.run_keys import executing_run_id, prompt_node_key
from .utils.usage_tracking import PENDING_FIRST_USE
from .utils.cached_runs import (
@@ -42,7 +42,7 @@ except ImportError:
from utils.image_monitor import get_image_monitor
from utils.prompt_tracker import get_prompt_tracker
from utils.hashing import generate_prompt_hash
from utils.prompt_graph import resolve_text, run_prompt_nodes
from utils.prompt_graph import resolve_text, run_prompt_nodes, save_setting
from utils.run_keys import executing_run_id, prompt_node_key
from utils.usage_tracking import PENDING_FIRST_USE
from utils.cached_runs import (
@@ -54,6 +54,22 @@ except ImportError:
)
# The optional input every PromptManager node declares last (#90): declared
# last so no widget value of a workflow saved before it shifts.
SAVE_TO_LIBRARY_INPUT = (
"BOOLEAN",
{
"default": True,
"label_on": "save",
"label_off": "skip",
"tooltip": (
"Save this node's prompt to the library, count its runs and link its "
"images. Off: PromptManager ignores the run; the output is unchanged."
),
},
)
class PromptManagerBase:
"""Mixin providing shared prompt management logic for ComfyUI nodes.
@@ -156,7 +172,9 @@ class PromptManagerBase:
resolved = resolve_text(prompt_graph, node_id)
return {
"is_run_prompt": node_id in run_prompt_nodes(prompt_graph),
# The hook skips a switch it cannot read (#90): the node counts
"seen_at_queue": resolved is not None
and save_setting(prompt_graph, node_id) is True
and generate_prompt_hash(resolved) == generate_prompt_hash(text),
"known": True,
}
@@ -245,6 +263,24 @@ class PromptManagerBase:
"run_id": executing_run_id(),
}
def _skip_not_saved(self, prompt_graph: Any = None, unique_id: Any = None) -> None:
"""``save_to_library`` is off: nothing is saved, counted or tracked (#90).
A literal off makes the node no run prompt (``utils.prompt_graph``), so
no image looks up its run key. A switch wired from a node the graph
cannot read keeps the node a run prompt: its run key is marked like an
empty editor's, so an earlier run's entry of that key never links this
run's images.
"""
message = (
"Save to library is off for this node: its prompt is not saved, the "
"run is not counted and its images are not linked"
)
if isinstance(prompt_graph, dict) and unique_id is not None:
self._mark_untracked(prompt_graph, unique_id, message)
else:
self.logger.info(message)
def _skip_empty_prompt(self, prompt_graph: Any, unique_id: Any) -> None:
"""The ``text`` editor is empty: save and track nothing (#342).
+23 -4
View File
@@ -22,7 +22,11 @@ try:
)
from .utils.hashing import generate_prompt_hash
from .utils.pair_execution import record_pair_execution
from .utils.prompt_graph import intrinsic_pair_nodes, run_prompt_components
from .utils.prompt_graph import (
intrinsic_pair_nodes,
run_prompt_components,
save_setting,
)
from .utils.prompt_pair_resolver import resolve_text_input
from .utils.usage_tracking import (
PENDING_FIRST_USE,
@@ -41,7 +45,11 @@ except ImportError:
)
from utils.hashing import generate_prompt_hash
from utils.pair_execution import record_pair_execution
from utils.prompt_graph import intrinsic_pair_nodes, run_prompt_components
from utils.prompt_graph import (
intrinsic_pair_nodes,
run_prompt_components,
save_setting,
)
from utils.prompt_pair_resolver import resolve_text_input
from utils.usage_tracking import (
PENDING_FIRST_USE,
@@ -90,8 +98,13 @@ class PromptManagerPair(PromptManagerFocused):
prompt: Optional[Dict[str, Any]] = None,
unique_id: Optional[str] = None,
extra_pnginfo: Optional[Dict[str, Any]] = None,
save_to_library: bool = True,
) -> Tuple[Any, Any, str, str]:
"""Encode both sides once; database and tracking failures never block CLIP."""
"""Encode both sides once; database and tracking failures never block CLIP.
With ``save_to_library`` off nothing is saved, counted, tracked or
recorded (#90); the four outputs are the same.
"""
if clip is None:
raise RuntimeError("ERROR: clip input is invalid: None")
if not isinstance(text, str) or not isinstance(negative_text, str):
@@ -109,6 +122,10 @@ class PromptManagerPair(PromptManagerFocused):
negative_conditioning = clip.encode_from_tokens_scheduled(
clip.tokenize(negative_text)
)
result = positive_conditioning, negative_conditioning, positive, negative_text
if not save_to_library:
self._skip_not_saved(prompt, unique_id)
return result
record_pair_execution(
extra_pnginfo,
unique_id,
@@ -169,7 +186,6 @@ class PromptManagerPair(PromptManagerFocused):
)
except Exception as error:
self.logger.warning(f"Could not register paired metadata: {error}")
result = positive_conditioning, negative_conditioning, positive, negative_text
record_runtime_context(
type(self).__name__,
text,
@@ -219,6 +235,9 @@ class PromptManagerPair(PromptManagerFocused):
resolved_negative = (
resolve_text_input(prompt, node_id, "negative_text") if known else None
)
if known and save_setting(prompt, node_id) is not True:
# The hook skips a switch it cannot read (#90): the node counts
resolved_positive = resolved_negative = None
positive_seen = resolved_positive is not None and generate_prompt_hash(
resolved_positive
) == generate_prompt_hash(text)
+13 -6
View File
@@ -26,13 +26,13 @@ except ImportError:
InputTypeDict = dict
try:
from .prompt_manager_base import PromptManagerBase
from .prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
except ImportError:
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from prompt_manager_base import PromptManagerBase
from prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
class PromptManagerText(PromptManagerBase, ComfyNodeABC):
@@ -115,6 +115,7 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
),
},
),
"save_to_library": SAVE_TO_LIBRARY_INPUT,
},
"hidden": {
# The queued graph and this node's id: used to tell the positive
@@ -149,6 +150,7 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
append_text: str = "",
prompt: Optional[Dict[str, Any]] = None,
unique_id: Optional[str] = None,
save_to_library: bool = True,
) -> Tuple[str]:
"""
Process the text prompt and save it to the database.
@@ -160,6 +162,7 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
search_text: Text to search for in past prompts
prepend_text: Text to prepend to the main prompt
append_text: Text to append to the main prompt
save_to_library: Off: nothing is saved, counted or tracked (#90)
Returns:
Tuple containing the final processed text string
@@ -184,7 +187,9 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
# Save prompt to database and set execution context for gallery tracking
prompt_id = None
extended_tags = []
if storage_text and storage_text.strip():
if not save_to_library:
self._skip_not_saved(prompt, unique_id)
elif storage_text and storage_text.strip():
self.logger.debug(f"Processing prompt text: {storage_text[:100]}...")
# Add prepend/append info to tags if they exist
@@ -220,6 +225,11 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
else:
self._skip_empty_prompt(prompt, unique_id)
self.logger.debug(f"Text processing completed: {final_text[:100]}...")
result = (final_text,)
if not save_to_library:
return result # #90: no metadata registration, no history text
# Register with ComfyUI integration for standard metadata compatibility
node_id = f"promptmanagertext_{int(time.time() * 1000)}"
self.comfyui_integration.register_prompt(
@@ -233,9 +243,6 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
"append_text": append_text.strip() if append_text else None,
},
)
self.logger.debug(f"Text processing completed: {final_text[:100]}...")
result = (final_text,)
record_runtime_context(
type(self).__name__, text, prepend_text, append_text, final_text
)
+9 -82
View File
@@ -48,6 +48,7 @@ from .review import ReviewRoutesMixin
from .review_recording import ReviewRecordingMixin
from .review_events import ReviewEventsMixin
from .saved_views import SavedViewRoutesMixin
from .collections import CollectionRoutesMixin
from .history import HistoryRoutesMixin
from .images import ImageRoutesMixin
from .gallery_listing import WALK_STOPPED, RootListing, WalkOutcome
@@ -73,9 +74,11 @@ from .autotag_routes import AutotagRoutesMixin
from .lora_integration import LoraIntegrationMixin
from .lora_import_routes import LoraImportRoutesMixin
from .image_categories import ImageCategoryRoutesMixin
from .source_rescope import SourceRescopeMixin
try:
from ...database.operations import PromptDatabase
from ...utils.file_prompt import parse_comfyui_prompt
from ...utils.logging_config import get_logger
from ...utils.video_metadata import is_video_path, read_video_metadata
except ImportError:
@@ -85,6 +88,7 @@ except ImportError:
0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
)
from database.operations import PromptDatabase
from utils.file_prompt import parse_comfyui_prompt
from utils.logging_config import get_logger
from utils.video_metadata import is_video_path, read_video_metadata
@@ -315,6 +319,7 @@ class PromptManagerAPI(
ReviewRecordingMixin,
ReviewEventsMixin,
SavedViewRoutesMixin,
CollectionRoutesMixin,
HistoryRoutesMixin,
PromptRoutesMixin,
ImageRoutesMixin,
@@ -338,6 +343,7 @@ class PromptManagerAPI(
LoraIntegrationMixin,
LoraImportRoutesMixin,
ImageCategoryRoutesMixin,
SourceRescopeMixin,
):
"""REST API handler for PromptManager operations and web interface.
@@ -686,6 +692,7 @@ class PromptManagerAPI(
self._register_review_routes(routes)
self._register_review_event_routes(routes)
self._register_saved_view_routes(routes)
self._register_collection_routes(routes)
self._register_history_routes(routes)
self._register_prompt_routes(routes)
self._register_image_routes(routes)
@@ -703,6 +710,7 @@ class PromptManagerAPI(
self._register_lora_routes(routes)
self._register_lora_import_routes(routes)
self._register_image_category_routes(routes)
self._register_rescope_routes(routes)
# Register gzip compression middleware (once)
global _gzip_registered
@@ -1028,85 +1036,4 @@ class PromptManagerAPI(
def _parse_comfyui_prompt(self, metadata):
"""Parse ComfyUI and A1111 prompt data from metadata."""
result = {
"prompt": None,
"workflow": None,
"parameters": {},
"positive_prompt": None,
"negative_prompt": None,
}
# Check for A1111/Forge style parameters first.
# The chunk is one block: a possibly multi-line positive prompt, then a
# "Negative prompt:" section, then a trailing "Steps: ..." settings line.
# Taking lines[0] as the positive prompt truncates any multi-line prompt
# to one line, so split on the settings line instead.
if "parameters" in metadata:
import re as _re
params = metadata["parameters"]
body = params
settings = ""
_sm = _re.search(r"\nSteps:\s", body)
if _sm:
settings = body[_sm.start() + 1 :].strip()
body = body[: _sm.start()]
negative = None
_nm = _re.search(r"(?im)^\s*negative prompt:\s*", body)
if _nm:
negative = body[_nm.end() :].strip()
body = body[: _nm.start()]
elif body.lower().lstrip().startswith("negative prompt:"):
negative = body.lower().lstrip()[len("negative prompt:") :].strip()
body = ""
result["positive_prompt"] = body.strip() or None
if negative:
result["negative_prompt"] = negative
# Store raw parameters too
result["parameters"]["parameters"] = params
if settings:
for _f in ("Steps", "Sampler", "Schedule type", "CFG scale", "Seed"):
_fm = _re.search(
r"(?i)(?:^|,\s*)" + _re.escape(_f) + r":\s*([^,]+)", settings
)
if _fm:
result["parameters"][_f.lower().replace(" ", "_")] = _fm.group(
1
).strip()
# If no A1111 format found, proceed with ComfyUI parsing
if result["positive_prompt"] is None:
# Check for direct prompt field
if "prompt" in metadata:
try:
prompt_data = json.loads(metadata["prompt"])
result["prompt"] = prompt_data
except json.JSONDecodeError:
result["prompt"] = metadata["prompt"]
# Check for workflow
if "workflow" in metadata:
try:
workflow_data = json.loads(metadata["workflow"])
result["workflow"] = workflow_data
except json.JSONDecodeError:
result["workflow"] = metadata["workflow"]
# Check for other common ComfyUI fields
common_fields = [
"positive",
"negative",
"steps",
"cfg",
"sampler",
"scheduler",
"seed",
]
for field in common_fields:
if field in metadata:
try:
result["parameters"][field] = json.loads(metadata[field])
except json.JSONDecodeError:
result["parameters"][field] = metadata[field]
return result
return parse_comfyui_prompt(metadata)
+158 -222
View File
@@ -40,43 +40,51 @@ except ImportError:
from utils import exit_signal
try:
from ...database.pair_ingestion import blank_positive, ingest_resolved_pair
from ...database.pair_ingestion import SCAN_MARKS, ingest_resolved_pair
from ...database.run_attribution import (
RecordedOutput,
host_output_directory,
stored_run_prompt,
stored_run_prompts,
named_prompt,
recorded_runs,
)
from ...utils.generation_params import (
PLACEHOLDERS,
GenerationParamsError,
extract_generation_params,
parse_page_json,
from ...database.scanned_prompts import (
link_stored_prompt_or_raise,
store_scanned_prompt,
)
from ...utils.file_prompt import (
NEVER,
NOT_SAVED,
NOT_SAVED_WIRED,
PAIR,
Policy,
decide_file_prompt,
)
from ...utils.prompt_pair_resolver import resolve_prompt_pair, sampler_less_prompt
from ...utils.review_capture import reviewable
except ImportError:
from database.pair_ingestion import blank_positive, ingest_resolved_pair
from database.pair_ingestion import SCAN_MARKS, ingest_resolved_pair
from database.run_attribution import (
RecordedOutput,
host_output_directory,
stored_run_prompt,
stored_run_prompts,
named_prompt,
recorded_runs,
)
from utils.generation_params import (
PLACEHOLDERS,
GenerationParamsError,
extract_generation_params,
parse_page_json,
from database.scanned_prompts import (
link_stored_prompt_or_raise,
store_scanned_prompt,
)
from utils.file_prompt import (
NEVER,
NOT_SAVED,
NOT_SAVED_WIRED,
PAIR,
Policy,
decide_file_prompt,
)
from utils.prompt_pair_resolver import resolve_prompt_pair, sampler_less_prompt
from utils.review_capture import reviewable
try:
from ...database.library import display_title
from ...utils.hashing import generate_prompt_hash
except ImportError:
from database.library import display_title
from utils.hashing import generate_prompt_hash
IMAGE_EXTENSIONS = (".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff")
VIDEO_EXTENSIONS = (".mp4", ".avi", ".mov", ".mkv", ".webm", ".m4v", ".wmv")
@@ -256,10 +264,15 @@ SCAN_DETAILS = {
"skipped": "Skipped · already linked",
"ambiguous": "Needs review: more than one prompt could match",
"unresolved": "Needs review: the prompt could not be traced in the workflow",
"conflict": "Needs review: already linked to a different prompt",
"conflict": "Kept: already linked to a different prompt",
"failed": "Error: the prompt could not be stored",
"error": "Error: the file could not be processed (see the server log)",
"found": "Metadata found, but no prompt text to store",
"not_saved": "Skipped · its PromptManager node does not save to the library",
"not_saved_wired": (
"Skipped · its Save to library switch is wired from another node, so only "
"the run itself can link it"
),
None: "No prompt metadata in the file",
}
@@ -318,22 +331,6 @@ def _file_size(path):
return None
def _with_file_size(canonical, media_file):
"""Record the file's size in ``file_info`` unless the metadata carries one.
Stored as ``generated_images.file_size``; the "new" scan scope skips a known
file only while its size is unchanged.
"""
info = canonical.get("file_info")
info = dict(info) if isinstance(info, dict) else {}
if info.get("size") is None:
size = _file_size(media_file)
if size is not None:
info["size"] = size
if info:
canonical["file_info"] = info
def _scan_roots(output_dirs):
"""(absolute root, public label, listing index), the most specific first.
@@ -589,18 +586,56 @@ class _BatchHistory:
"""A scan batch's recorded-history lookup, made on the first file needing it.
Most files link through their metadata and never ask; the first one that
does triggers one lookup for the whole batch (``_recorded_runs_for``).
does triggers one ``recorded_runs`` pass for the whole batch, from which
both the stored prompt (``get``) and the recorded saver (``recorded``)
of every file are read. A failure is logged and names nothing.
"""
def __init__(self, api, media_files):
self._api = api
self._files = media_files
self._found = None
self._files = [str(path) for path in media_files]
self._runs = None
self._prompts = {}
def _found(self):
if self._runs is None:
try:
self._runs = recorded_runs(
self._api.db, self._files, host_output_directory(), None
)
except Exception:
self._api.logger.warning(
"History attribution failed for a scan batch", exc_info=True
)
self._runs = {}
return self._runs
def get(self, path):
if self._found is None:
self._found = self._api._recorded_runs_for(self._files)
return self._found.get(path)
"""The stored prompt id history attributes to ``path``, else None."""
found = self._found().get(path)
if found is None:
return None
run_id = found[0]
if run_id not in self._prompts:
try:
self._prompts[run_id] = named_prompt(self._api.db, run_id)
except Exception:
self._api.logger.warning(
"History attribution failed for a scan batch", exc_info=True
)
self._prompts[run_id] = None
return self._prompts[run_id]
def recorded(self, path):
"""``RecordedOutput(saver, {})`` for an attributed file, else None.
``saver`` is set only when the run reported the file from one node.
"""
found = self._found().get(path)
if found is None:
return None
nodes = found[1]
return RecordedOutput(next(iter(nodes)) if len(nodes) == 1 else None, {})
class AdminRoutesMixin:
@@ -1665,24 +1700,6 @@ class AdminRoutesMixin:
note(media_file, status)
return counts
@staticmethod
def _graph_prompt_text(metadata):
"""The positive prompt the browser parser traces from the graphs, or None.
Records parse as the legacy pages parsed them (NaN reads as null).
"""
records = {}
for key in ("prompt", "workflow"):
value = metadata.get(key)
records[key] = parse_page_json(value) if isinstance(value, str) else value
try:
positive = extract_generation_params(records)["positivePrompt"]
except GenerationParamsError:
return None
if not isinstance(positive, str) or positive == PLACEHOLDERS["positivePrompt"]:
return None
return positive.strip() or None
def _ingest_scanned_file(
self,
media_file,
@@ -1693,124 +1710,81 @@ class AdminRoutesMixin:
records=None,
history=None,
):
"""Enrich exact pairs; retain legacy positive-only import for absent pairs.
"""Store the prompt the shared per-file decision names (#247).
Returns the file's scan status, or None without prompt metadata. With
``detailed``, a stored or linked file returns a dict instead: its
``status``, the prompt's display ``title`` and, for a resolved pair,
the pair counts. When the resolver finds the file ambiguous (whether
or not the browser parser then links it) and ``ambiguous`` is a list,
``media_file`` is appended to it; only a file that then returns
``"ambiguous"`` (nothing linked it) is a Needs review case. An
ambiguous result with nothing to choose (``reviewable`` is False: fewer
than two candidates carry a prompt) that nothing links returns
``"unresolved"`` instead, and ``media_file`` goes to ``untraced``.
The parsed records of either ambiguous file go to
``records["canonical"]`` when ``records`` is a dict.
``utils.file_prompt.decide_file_prompt`` (scan policy) decides; this
stores the result. Returns the file's scan status, or None without
prompt metadata. With ``detailed``, a stored or linked file returns a
dict instead: its ``status``, the prompt's display ``title`` and, for
a resolved pair, the pair counts. When the resolver finds the file
ambiguous (whether or not a later step then links it) and
``ambiguous`` is a list, ``media_file`` is appended to it; only a file
that then returns ``"ambiguous"`` (nothing linked it) is a Needs
review case. An ambiguous result with nothing to choose
(``reviewable`` is False: fewer than two candidates carry a prompt)
goes to ``untraced`` instead and returns ``"unresolved"`` when nothing
links it. The parsed records of either ambiguous file go to
``records["canonical"]`` when ``records`` is a dict. ``history`` is
the batch's lazy recorded-history lookup.
"""
if not metadata:
# No prompt metadata (--disable-metadata, bare savers): only the
# recorded run can say which prompt made it (#342)
return self._link_recorded_run(media_file, {}, detailed, history)
canonical = dict(metadata)
for key in ("prompt", "workflow"):
if isinstance(canonical.get(key), str):
try:
canonical[key] = json.loads(canonical[key])
except (ValueError, TypeError):
# Preserve malformed metadata for conservative resolver rejection.
pass
if not canonical.get("prompt") and not canonical.get("parameters"):
return self._link_recorded_run(media_file, canonical, detailed, history)
_with_file_size(canonical, media_file)
resolution = resolve_prompt_pair(canonical)
if blank_positive(resolution):
# Only prepend/append text: no stored prompt can own it (#342)
resolution = {**resolution, "state": "unresolved"}
# Ambiguous without a choice: unresolved unless something links it.
unchoosable = resolution["state"] == "ambiguous" and not reviewable(resolution)
if unchoosable:
if untraced is not None:
untraced.append(media_file)
if records is not None:
records["canonical"] = canonical
elif resolution["state"] == "ambiguous" and ambiguous is not None:
ambiguous.append(media_file)
if records is not None:
records["canonical"] = canonical
if resolution["state"] == "resolved":
result = ingest_resolved_pair(
self.db, str(media_file), canonical, resolution
path = str(media_file)
if history is None: # one file: history is still asked once
history = _BatchHistory(self, [media_file])
decision = decide_file_prompt(
metadata,
file_name=Path(path).name,
file_size=functools.partial(_file_size, media_file),
in_output=False, # only the monitor policy reads it
lookup=functools.partial(history.recorded, path),
policy=Policy.SCAN,
)
if decision.kind in (NOT_SAVED, NOT_SAVED_WIRED):
# Save to library off (#90): no write, recorded-run fallback or
# Needs review item
return decision.kind
self._note_review(media_file, decision, ambiguous, untraced, records)
if decision.kind == NEVER:
if decision.try_recorded_run:
linked = self._link_recorded_run(
media_file, decision.canonical, detailed, history
)
if linked is not None:
return linked
return decision.unlinked_status
if decision.kind == PAIR:
# A rerun after a library move re-checks the link rows.
result = self.db.survive_library_switch(
lambda: ingest_resolved_pair(
self.db,
path,
decision.canonical,
decision.resolution,
marks=SCAN_MARKS,
)
)
if not detailed:
return result["status"]
return {
**result,
"title": self._scan_title(media_file, resolution["positive"]["text"]),
}
if canonical.get("prompt") and resolution["state"] in (
"ambiguous",
"unresolved",
):
# The resolver could not decide: link the prompt the inspector and the
# legacy pages show for this file (the browser parser, in Python).
# Graphs without a recognized sampler keep the legacy extractor below.
prompt_text = self._graph_prompt_text(metadata)
if not prompt_text:
linked = self._link_recorded_run(
media_file, canonical, detailed, history
)
if linked is not None:
return linked
return "unresolved" if unchoosable else resolution["state"]
elif resolution["state"] == "ambiguous":
return "unresolved" if unchoosable else "ambiguous"
elif resolution["state"] == "unresolved" and not isinstance(
canonical.get("parameters"), str
):
return resolution["state"]
elif canonical.get("prompt"):
# No node takes a positive or negative input (a guider or wrapper
# graph): the one wired text, never a guess (#252).
found = sampler_less_prompt(canonical["prompt"])
prompt_text = found["text"]
if prompt_text is None and isinstance(canonical.get("parameters"), str):
# A metadata saver also wrote A1111 text: its positive prompt.
parsed = self._parse_comfyui_prompt(metadata)["positive_prompt"]
prompt_text = parsed.strip() if parsed and parsed.strip() else None
if prompt_text is None:
linked = self._link_recorded_run(
media_file, canonical, detailed, history
)
if linked is not None:
return linked
return "unresolved" if found["state"] == "ambiguous" else "found"
else:
prompt_text = self._parse_comfyui_prompt(metadata)["positive_prompt"]
if not (prompt_text and prompt_text.strip()):
return "found"
prompt_text = prompt_text.strip()
status = self._store_scanned_prompt(media_file, canonical, prompt_text)
title_text = decision.resolution["positive"]["text"]
return {**result, "title": self._scan_title(media_file, title_text)}
status, _prompt_id = store_scanned_prompt(
self.db, path, decision.canonical, decision.text, SCAN_MARKS
)
if not detailed or status not in SCAN_TITLED_DETAILS:
return status
return {"status": status, "title": self._scan_title(media_file, prompt_text)}
return {"status": status, "title": self._scan_title(media_file, decision.text)}
def _recorded_runs_for(self, media_files):
"""``{path: prompt id or None}`` from recorded history for a scan batch.
One output lookup per chunk of names (``stored_run_prompts``), so a
batch never queries history once per file. An empty dict on failure.
"""
try:
return stored_run_prompts(
self.db, [str(path) for path in media_files], host_output_directory()
)
except Exception:
self.logger.warning(
"History attribution failed for a scan batch", exc_info=True
)
return {}
@staticmethod
def _note_review(media_file, decision, ambiguous, untraced, records):
"""Hand an ambiguous file to the batch's Needs review lists."""
if decision.review == "untraced":
if untraced is not None:
untraced.append(media_file)
if records is not None:
records["canonical"] = decision.canonical
elif decision.review == "ambiguous" and ambiguous is not None:
ambiguous.append(media_file)
if records is not None:
records["canonical"] = decision.canonical
def _link_recorded_run(self, media_file, canonical, detailed=False, history=None):
"""Link a file recorded history attributes to one stored prompt (#342).
@@ -1819,31 +1793,25 @@ class AdminRoutesMixin:
``{"status", "title"}``), or None when history names no single stored
prompt for the file (``database.run_attribution``). Never creates a
prompt. Works without any prompt metadata in the file. ``history`` is
the batch's precomputed lookup; without it the file is looked up alone.
the batch's lookup (``_BatchHistory``). A file already linked to
another prompt is a ``conflict``: one prompt per file.
"""
if history is not None:
prompt_id = history.get(str(media_file))
else:
try:
prompt_id = stored_run_prompt(
self.db, str(media_file), host_output_directory()
)
except Exception:
self.logger.warning(
f"History attribution failed for {media_file.name}",
exc_info=True,
)
return None
if history is None:
history = _BatchHistory(self, [media_file])
prompt_id = history.get(str(media_file))
if prompt_id is None:
return None
try:
image_id = self.db.link_image_to_prompt(
prompt_id, str(media_file), metadata=canonical
status = self.db.survive_library_switch(
lambda: link_stored_prompt_or_raise(
self.db, str(media_file), canonical, prompt_id
)
)
except Exception as e:
self.logger.error(f"Failed to link {media_file.name} from history: {e}")
return "found"
status = "linked" if image_id else "unchanged"
if status is None:
return None
if not detailed or status not in SCAN_TITLED_DETAILS:
return status
prompt = self.db.get_prompt_by_id(prompt_id) or {}
@@ -1863,35 +1831,3 @@ class AdminRoutesMixin:
except Exception:
self.logger.warning("Scan row title could not be read; using its text")
return display_title((), text)
def _store_scanned_prompt(self, media_file, canonical, prompt_text):
"""Link ``media_file`` to ``prompt_text``'s prompt, saving it when new."""
prompt_hash = generate_prompt_hash(prompt_text)
existing = self.db.get_prompt_by_hash(prompt_hash)
if existing:
try:
image_id = self.db.link_image_to_prompt(
existing["id"], str(media_file), metadata=canonical
)
except Exception as e:
self.logger.error(
f"Failed to link {media_file.name} to existing prompt: {e}"
)
return "found"
return "linked" if image_id else "unchanged"
prompt_id = self.db.save_prompt(
prompt_text,
"scanned",
["auto-scanned"],
None,
f"Auto-scanned from {media_file.name}",
prompt_hash,
)
if not prompt_id:
return "found"
try:
self.db.link_image_to_prompt(prompt_id, str(media_file), metadata=canonical)
except Exception as e:
self.logger.error(f"Failed to link {media_file.name} to new prompt: {e}")
return "added"
+17
View File
@@ -86,6 +86,23 @@ def unavailable_message(model_name: str) -> str:
return f"{model_name} is not downloaded. {UNAVAILABLE_HINT}"
def model_refusal(status: dict, model_type: str, model_name: str):
"""Why *model_type* cannot run, from ``get_models_status()``, else None.
Not downloaded first (``unavailable_message``), then a backend package
that cannot be imported (#424). Both are a 409 ``model-unavailable``.
"""
entry = status.get(model_type) or {}
if not entry.get("downloaded"):
return unavailable_message(model_name)
missing = entry.get("missing_packages") or []
if missing:
from ..autotag import missing_packages_message
return missing_packages_message(model_type, [str(name) for name in missing])
return None
def run_suggestions(service, capture_path: str, model_type: str, options: dict):
"""Blocking: load (or swap to) *model_type*, then suggest for the capture.
+9 -5
View File
@@ -21,9 +21,9 @@ from aiohttp import web
from .annotation_suggest import (
SUGGEST_FIELDS,
OptionError,
model_refusal,
run_suggestions,
suggestion_rows,
unavailable_message,
validate_options,
)
from .autotag_routes import model_busy_response
@@ -595,7 +595,7 @@ class AnnotationRoutesMixin:
it, including a legacy batch), never downloads (``model-unavailable``)
and never writes.
"""
from ..autotag import MODELS, get_autotag_service
from ..autotag import MODELS, MissingBackendPackages, get_autotag_service
from ..config import MAX_SUGGESTION_LIMIT, AutotagConfig
data, error = await self._annotation_body(request, SUGGEST_FIELDS)
@@ -633,9 +633,10 @@ class AnnotationRoutesMixin:
return error
service = get_autotag_service()
status = await self._run_in_executor(service.get_models_status)
if not status.get(model_type, {}).get("downloaded"):
name = MODELS.get(model_type, {}).get("name", model_type)
return _coded("model-unavailable", unavailable_message(name))
name = MODELS.get(model_type, {}).get("name", model_type)
refusal = model_refusal(status, model_type, name)
if refusal is not None:
return _coded("model-unavailable", refusal)
owner = model_gate.new_owner("image-annotations")
if not model_gate.try_acquire(owner):
return model_busy_response()
@@ -654,6 +655,9 @@ class AnnotationRoutesMixin:
options,
on_skipped=functools.partial(store.release_use, token),
)
except MissingBackendPackages as exc:
# A package gone since the status read (load_model logged it).
return _coded("model-unavailable", str(exc))
except Exception:
self.logger.exception("Tag suggestion failed")
return _error(
+20 -4
View File
@@ -102,6 +102,17 @@ def size_label(size: str) -> str:
return f"{match.group(1)} {match.group(2)}" if match else str(size or "")
def package_entry(flags: dict) -> dict:
"""The public package fields of one status entry (#424): the pip names
that cannot be imported, the install command and note (strings or null)."""
missing = flags.get("missing_packages") or []
text = {
key: flags[key] if isinstance(flags.get(key), str) else None
for key in ("install_command", "install_note")
}
return {"missing_packages": [str(name) for name in missing], **text}
def public_models(status: dict, loaded_type: Optional[str]) -> dict:
"""Each engine model's public entry: names and sizes from ``MODELS``, the
flags from ``status``; never a path (``model_path`` is dropped)."""
@@ -122,6 +133,7 @@ def public_models(status: dict, loaded_type: Optional[str]) -> dict:
for key in ("model_exists", "mmproj_exists"):
if key in flags:
entry[key] = bool(flags[key])
entry.update(package_entry(flags))
models[model_type] = entry
return models
@@ -750,6 +762,7 @@ class AutotagRoutesMixin:
"downloaded": bool(
(status.get(model_type) or {}).get("downloaded")
),
**package_entry(status.get(model_type) or {}),
},
"existing": existing,
"review_required": existing == "replace_after_review",
@@ -1073,8 +1086,8 @@ class AutotagRoutesMixin:
return self.db.current_image_tags({locator: fingerprint}).get(locator, [])
async def _suggest_single(self, target, model_type, options, base):
from ..autotag import MODELS, get_autotag_service
from .annotation_suggest import unavailable_message
from ..autotag import MODELS, MissingBackendPackages, get_autotag_service
from .annotation_suggest import model_refusal
from .annotations import _pinned_suggestions, _settled, _submit
try:
@@ -1084,8 +1097,8 @@ class AutotagRoutesMixin:
service = get_autotag_service()
status = await self._run_in_executor(service.get_models_status)
if not (status.get(model_type) or {}).get("downloaded"):
message = unavailable_message(MODELS[model_type]["name"])
message = model_refusal(status, model_type, MODELS[model_type]["name"])
if message is not None:
return self._coded_error(409, "model-unavailable", message)
owner = model_gate.new_owner("legacy-single")
if not model_gate.try_acquire(owner):
@@ -1131,6 +1144,9 @@ class AutotagRoutesMixin:
options,
on_skipped=functools.partial(store.release_use, capture.token),
)
except MissingBackendPackages as exc:
# A package gone since the status read (load_model logged it).
return self._coded_error(409, "model-unavailable", str(exc))
finally:
store.release(capture.token)
finally:
+350
View File
@@ -0,0 +1,350 @@
"""Collections (#331): named, hand-picked groups of images.
Routes under ``/prompt_manager/collections``. A file is named by its listing
identity ``{root_index, root_scope, path}`` (``media_target.py``) and stored
under the favorites' keys (``_favorite_root_key``, ``_favorite_relative_key``),
which never leave the server. A stale scope (the sources were reordered) is a
409 ``source-changed`` before anything is written. Responses are
``{"success": bool, ...}``; an unexpected failure is a logged 500 with a
generic message, never the exception text.
"""
import hmac
import re
from functools import wraps
from typing import Callable
from aiohttp import web
from .images import (
_favorite_relative_key,
_favorite_root_key,
_forbidden,
_json_body_or_error,
collection_not_found,
)
from .media_target import media_target
from .prompts import bad_request
try:
from ...database.image_collections import (
CollectionFilesMissing,
CollectionInvalid,
CollectionMissing,
CollectionNameTaken,
normalize_collection_name,
)
from ...database.image_favorites import MAX_KEY_LENGTH
except ImportError:
from database.image_collections import (
CollectionFilesMissing,
CollectionInvalid,
CollectionMissing,
CollectionNameTaken,
normalize_collection_name,
)
from database.image_favorites import MAX_KEY_LENGTH
BASE = "/prompt_manager/collections"
# The most files one add or remove names (the bulk tagging cap, MC12).
MAX_ITEMS = 500
MAX_ID = 999_999_999
_PATH_ID = re.compile(r"[1-9][0-9]{0,8}")
_ROOT_INDEX = re.compile(r"[0-9]{1,9}")
_ADD_TARGETS = (frozenset({"collection_id", "items"}), frozenset({"name", "items"}))
_REMOVE_FIELDS = frozenset({"collection_id", "items"})
_RENAME_FIELDS = frozenset({"name"})
_MEDIA_MESSAGE = "Only media files can be added to a collection"
READ_FAILED = "Collections could not be read."
CHANGE_FAILED = "The collection could not be changed."
SOURCES_CHANGED = "The sources changed. Refresh the images and try again."
FILES_NOT_FOUND = (
"None of the selected images could be found. Refresh the images and try again."
)
NAME_TAKEN = "Another collection already has this name."
class _Refused(Exception):
"""A request refused before any write; carries its response."""
def __init__(self, response: web.Response):
super().__init__(response.status)
self.response = response
def _coded(status: int, code: str, message: str, **extra) -> web.Response:
return web.json_response(
{"success": False, "code": code, "error": message, **extra},
status=status,
headers={"Cache-Control": "no-store"},
)
def _collection_response(failure: str) -> Callable:
"""Map the database's refusals to their status; anything else to a 500."""
def decorate(handler: Callable) -> Callable:
@wraps(handler)
async def wrapped(self, request: web.Request) -> web.Response:
try:
return await handler(self, request)
except _Refused as refused:
return refused.response
except CollectionInvalid as exc:
return bad_request(str(exc))
except CollectionMissing:
return collection_not_found()
except CollectionFilesMissing:
return _coded(404, "files-not-found", FILES_NOT_FOUND)
except CollectionNameTaken as exc:
return _coded(
409, "name-taken", NAME_TAKEN, collection_id=exc.collection_id
)
except Exception:
self.logger.exception("Collection request failed")
return web.json_response(
{"success": False, "error": failure}, status=500
)
return wrapped
return decorate
def _path_id(request: web.Request) -> int:
raw = request.match_info["collection_id"]
if not raw.isascii() or _PATH_ID.fullmatch(raw) is None:
raise _Refused(bad_request("Invalid collection id"))
return int(raw)
def _body_id(value: object) -> int:
if type(value) is not int or not 1 <= value <= MAX_ID:
raise _Refused(bad_request("Invalid collection id"))
return value
async def _body(request: web.Request, allowed: tuple) -> dict:
data, error = await _json_body_or_error(request)
if error is not None:
raise _Refused(error)
if frozenset(data) not in allowed:
names = " or ".join(", ".join(sorted(fields)) for fields in allowed)
raise _Refused(bad_request(f"Request must contain exactly {names}"))
return data
class CollectionRoutesMixin:
"""Mixin providing the collection routes (#331)."""
def _register_collection_routes(self, routes: web.RouteTableDef) -> None:
# The fixed paths go first: "/items" must never match "/{collection_id}".
routes.get(BASE)(self.list_collections)
routes.post(BASE + "/items")(self.add_collection_items)
routes.delete(BASE + "/items")(self.remove_collection_items)
routes.get(BASE + "/membership")(self.collection_membership)
routes.patch(BASE + "/{collection_id}")(self.rename_collection)
routes.delete(BASE + "/{collection_id}")(self.delete_collection)
# -- reads -------------------------------------------------------------
@_collection_response(READ_FAILED)
async def list_collections(self, request: web.Request) -> web.Response:
"""Every collection with its listed ``count`` and its ``missing`` rows.
``count`` is the members listed under every configured source (one
primary-key range read per source, matched against its cached
listing); ``missing`` the stored rows beyond it. When a scan or a read
fails both are null for every collection and the names still return.
"""
collections = await self._run_in_executor(self.db.list_collections)
counts = None
if collections:
try:
counts = await self._collection_counts()
except Exception:
self.logger.warning(
"Collection counts could not be read", exc_info=True
)
rows = []
for collection in collections:
listed, stored = (None, None) if counts is None else counts
count = None if listed is None else listed.get(collection["id"], 0)
stored_rows = None if stored is None else stored.get(collection["id"], 0)
rows.append(
{
**collection,
"count": count,
# Clamped: never below zero, whatever the snapshot saw.
"missing": None if count is None else max(0, stored_rows - count),
}
)
return web.json_response({"success": True, "collections": rows})
async def _collection_counts(self) -> tuple[dict, dict]:
"""``({id: listed members}, {id: stored rows})`` over every source.
The scans come from the gallery cache; the root keys (realpath), the
listings and one read snapshot of every member and stored count run
in the executor, so a write between two reads cannot skew them.
"""
roots = list(self._get_all_output_dirs())
root_keys = await self._run_in_executor(
lambda: [_favorite_root_key(root) for root in roots]
)
distinct = {} # one folder configured twice counts once
for root, root_key in zip(roots, root_keys):
distinct.setdefault(root_key, root)
scans = {
root_key: await self._get_gallery_files(root)
for root_key, root in distinct.items()
}
return await self._run_in_executor(self._count_members, distinct, scans)
def _count_members(self, roots: dict, scans: dict) -> tuple[dict, dict]:
"""Blocking: match one snapshot's members against each listing."""
members, stored = self.db.collection_count_snapshot(list(roots))
listed: dict[int, int] = {}
for root_key, root in roots.items():
listing = self._favorite_listing(root, scans[root_key])
for collection_id, keys in members.get(root_key, {}).items():
found = sum(1 for key in keys if key in listing)
listed[collection_id] = listed.get(collection_id, 0) + found
return listed, stored
@_collection_response(READ_FAILED)
async def collection_membership(self, request: web.Request) -> web.Response:
"""GET ``?root_index&root_scope&path``: the ids holding that file."""
query = request.query
if len(query) != 3 or set(query) != {"root_index", "root_scope", "path"}:
# len() counts a repeated key (?path=a&path=b) twice.
return bad_request("Query must name root_index, root_scope and path once")
raw_index = query["root_index"]
if not raw_index.isascii() or _ROOT_INDEX.fullmatch(raw_index) is None:
return bad_request("Invalid gallery root")
roots = list(self._get_all_output_dirs())
target = {
"root_index": int(raw_index),
"root_scope": query["root_scope"],
"path": query["path"],
}
root_key, key = await self._run_in_executor(
self._collection_key, target, roots, {}
)
ids = await self._run_in_executor(self.db.file_collections, root_key, key)
return web.json_response({"success": True, "collection_ids": ids})
# -- writes ------------------------------------------------------------
@_collection_response(CHANGE_FAILED)
async def add_collection_items(self, request: web.Request) -> web.Response:
"""POST ``{collection_id | name, items}``: add the listed files.
Only files the source's listing holds are added; the rest count in
``not_found``. A new name with nothing to add creates nothing (404
``files-not-found``).
"""
data = await _body(request, _ADD_TARGETS)
target = {}
if "name" in data:
target["name"] = normalize_collection_name(data["name"])
else:
target["collection_id"] = _body_id(data["collection_id"])
roots = list(self._get_all_output_dirs())
keys = await self._run_in_executor(self._collection_items, data["items"], roots)
found, not_found = [], 0
listings = {}
for index, root_key, key in keys:
if index not in listings:
root = roots[index]
files = await self._get_gallery_files(root)
listings[index] = await self._run_in_executor(
self._favorite_listing, root, files
)
if key in listings[index]:
found.append((root_key, key))
else:
not_found += 1
result = await self._run_in_executor(
lambda: self.db.add_to_collection(items=found, **target)
)
return web.json_response({"success": True, **result, "not_found": not_found})
@_collection_response(CHANGE_FAILED)
async def remove_collection_items(self, request: web.Request) -> web.Response:
"""DELETE ``{collection_id, items}``: listed or not, every named file."""
data = await _body(request, (_REMOVE_FIELDS,))
collection_id = _body_id(data["collection_id"])
roots = list(self._get_all_output_dirs())
keys = await self._run_in_executor(self._collection_items, data["items"], roots)
result = await self._run_in_executor(
self.db.remove_from_collection,
collection_id,
[(root_key, key) for _index, root_key, key in keys],
)
return web.json_response({"success": True, **result})
@_collection_response(CHANGE_FAILED)
async def rename_collection(self, request: web.Request) -> web.Response:
"""PATCH ``{name}``: a case variant of its own name is allowed."""
collection_id = _path_id(request)
data = await _body(request, (_RENAME_FIELDS,))
name = normalize_collection_name(data["name"])
collection = await self._run_in_executor(
self.db.rename_collection, collection_id, name
)
return web.json_response({"success": True, "collection": collection})
@_collection_response(CHANGE_FAILED)
async def delete_collection(self, request: web.Request) -> web.Response:
"""DELETE: the collection and its members; files are never touched."""
collection_id = _path_id(request)
await self._run_in_executor(self.db.delete_collection, collection_id)
return web.json_response({"success": True, "deleted": True})
# -- request steps -----------------------------------------------------
def _collection_items(self, items: object, roots: list) -> list[tuple]:
"""``(index, root key, relative key)`` per distinct file, in order.
Blocking (realpath per root, once per request): run in the executor.
Every item is checked before anything is written: 400 for a malformed
shape, 403 for an unsafe path, 409 for a stale scope. Two spellings
that fold to one key (Windows) count once.
"""
if not isinstance(items, list) or not 1 <= len(items) <= MAX_ITEMS:
raise _Refused(bad_request(f"items must list 1 to {MAX_ITEMS} images"))
keys, sources = {}, {}
for item in items:
index = item.get("root_index") if isinstance(item, dict) else None
root_key, key = self._collection_key(item, roots, sources)
keys.setdefault((root_key, key), index)
return [(index, root_key, key) for (root_key, key), index in keys.items()]
def _collection_key(
self, item: object, roots: list, sources: dict
) -> tuple[str, str]:
"""The stored keys of one target, or raise its refusal (blocking).
*sources* caches ``{index: (scope, root key)}`` for one request, so
each root's realpath runs once however many items name it.
"""
target, error = media_target(item, media_message=_MEDIA_MESSAGE)
if error is not None:
raise _Refused(error)
index, scope, relative = target
if index >= len(roots):
raise _Refused(_coded(409, "source-changed", SOURCES_CHANGED))
if index not in sources:
sources[index] = (
self._gallery_root_scope(roots[index], index),
_favorite_root_key(roots[index]),
)
current, root_key = sources[index]
if not hmac.compare_digest(scope, current):
raise _Refused(_coded(409, "source-changed", SOURCES_CHANGED))
key = _favorite_relative_key(relative)
if max(len(root_key), len(key)) > MAX_KEY_LENGTH:
# A stored key past the schema bound would fail the next restore.
raise _Refused(_forbidden("Path is too long to add to a collection"))
return root_key, key
+8 -1
View File
@@ -588,7 +588,9 @@ class DuplicateFilesMixin:
job.update(total=len(targets), status_line="Removing duplicate files")
spellings = await self._run_in_executor(self._source_spellings)
removed, failure, keepers = 0, None, {}
totals = dict.fromkeys(("moved", "merged", "favorites", "tags", "review"), 0)
totals = dict.fromkeys(
("moved", "merged", "favorites", "tags", "review", "collections"), 0
)
totals["categories"] = 0 # image categories kept on the kept copies (#332)
for done, target in enumerate(targets, start=1):
if failure is not None:
@@ -632,6 +634,11 @@ class DuplicateFilesMixin:
job.log_line(
f"Moved {_plural(totals['favorites'], 'favorite')} to the kept copies"
)
if totals["collections"]:
job.log_line(
f"Moved {_plural(totals['collections'], 'collection membership')} "
"to the kept copies"
)
if totals["tags"]:
job.log_line(
f"Added {_plural(totals['tags'], 'image tag')} to the kept copies"
+2 -1
View File
@@ -104,7 +104,8 @@ def copy_move(target, kept_stat, spellings):
A record found under a folder spelling moves to the kept copy under
the same spelling when that folder holds it too, else to its real
path. Its favorite moves under every listed root holding the file.
path. Its favorite and its collection memberships move under every
listed root holding the file.
"""
removed = spellings.of(target["abs"])
kept = spellings.of(target["keeper_abs"])
+53 -5
View File
@@ -5,6 +5,7 @@ import base64
import binascii
from functools import wraps
import json
import os
import re
import sqlite3
from typing import Callable
@@ -27,6 +28,7 @@ from .history_current import (
snapshot_current_output_scope,
)
from .inspection_service import CurrentFileChanged, inspect_current_file
from .images import image_category_key
try:
from ...database.execution_history import HistoryEpochChanged
@@ -39,7 +41,11 @@ try:
WindowPosition,
)
from ...utils.history_read_fence import HistoryReadFence
from ...utils.inspection_response import encode_history_inspection_response
from ...utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_history_inspection_response,
)
except ImportError:
from database.execution_history import HistoryEpochChanged
from database.history_gallery import read_history_gallery
@@ -51,7 +57,11 @@ except ImportError:
WindowPosition,
)
from utils.history_read_fence import HistoryReadFence
from utils.inspection_response import encode_history_inspection_response
from utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_history_inspection_response,
)
_RUN_MAX_BYTES = 8 * 1024 * 1024
@@ -677,6 +687,32 @@ class HistoryRoutesMixin:
) -> web.Response:
return await self._history_current(request, "inspection")
def _history_own_category(self, reference: dict, scope):
"""The recorded output's own category (#332), inside the snapshot.
The path is the symlink-resolved output folder plus the receipt's
subfolder and filename, keyed by ``image_category_key`` over the
configured output sources as Images keys it. A temp output or a path
no source holds has none. A failed read is OWN_CATEGORY_UNREAD.
"""
if (
reference.get("folder_type") != "output"
or scope.canonical_output is None
or candidate_locator(reference, scope) is None
):
return None
path = os.path.join(
str(scope.canonical_output), reference["subfolder"], reference["filename"]
)
try:
key = image_category_key(self._get_all_output_dirs(), path)
return None if key is None else self.db.image_category(*key)
except Exception:
self.logger.warning(
"History image category could not be read", exc_info=True
)
return OWN_CATEGORY_UNREAD
async def _history_current(
self, request: web.Request, kind: str
) -> web.StreamResponse:
@@ -701,6 +737,7 @@ class HistoryRoutesMixin:
if scope.current_scope != current_scope:
raise _current_stale()
result = db.get_history_occurrence(run_id, node, ordinal)
own = None
if result is not None:
receipt = result["receipt"]
path = candidate_locator(receipt["reference"], scope)
@@ -708,6 +745,10 @@ class HistoryRoutesMixin:
db.model.get_connection(),
(path,) if path is not None else (),
)
if kind == "inspection":
own = self._history_own_category(
receipt["reference"], scope
)
finally:
db.model.close()
if result is None:
@@ -726,9 +767,16 @@ class HistoryRoutesMixin:
resolved.canonical_target,
source_label=resolved.public["source_label"],
saved=saved.get("saved"),
library=saved.get(
"library",
{"status": "unavailable" if saved_unavailable else "none"},
library=apply_own_category(
saved.get(
"library",
{
"status": (
"unavailable" if saved_unavailable else "none"
)
},
),
own,
),
recorded=recorded_output_lookup(
db, close=True, reference=receipt["reference"]
+16 -3
View File
@@ -29,9 +29,11 @@ from .prompts import bad_request
try:
from ...database.image_categories import MAX_CATEGORY_LENGTH, MAX_KEY_LENGTH
from ...database.models import normalize_image_path
from ...utils.validators import validate_category
except ImportError:
from database.image_categories import MAX_CATEGORY_LENGTH, MAX_KEY_LENGTH
from database.models import normalize_image_path
from utils.validators import validate_category
CATEGORY_MEDIA_MESSAGE = "Only media files can have a category"
@@ -143,7 +145,7 @@ class ImageCategoryRoutesMixin:
# -- reading ---------------------------------------------------------------
def _own_category_map(self, sources, listings) -> dict:
"""``{root / listed path: own category}`` for the listed files (blocking).
"""``{normalized listed path: own category}`` for the listed files (blocking).
One primary-key range read per source; each stored row is matched to
the listing of every selected root that holds its source, so the work
@@ -162,19 +164,30 @@ class ImageCategoryRoutesMixin:
for _index, root, files in listings:
listing = self._favorite_listing(root, files)
root_key = _favorite_root_key(root)
# normalize_image_path(root / listed) without a call per file: the
# listed spellings are clean relative POSIX paths. Windows abspath
# also drops a component's trailing dots and spaces, so it keeps
# the call there.
base = os.path.join(normalize_image_path(str(root)), "")
def normalized(listed, base=base, root=root):
if os.name == "nt":
return normalize_image_path(str(root / listed))
return base + os.path.normcase(listed.replace("/", os.sep))
nested = [
item for item in _nested_sources(root, sources) if item[1] != root_key
]
for relative, category in by_source.get(root_key, {}).items():
listed = listing.get(relative)
if listed is not None and _source_prefix(relative, nested) is None:
found[root / listed] = category
found[normalized(listed)] = category
for prefix, source_key in nested:
for relative, category in by_source.get(source_key, {}).items():
full = f"{prefix}/{relative}"
listed = listing.get(full)
if listed is not None and _source_prefix(full, nested) == prefix:
found[root / listed] = category
found[normalized(listed)] = category
return found
async def _source_listings(self, roots) -> list[tuple]:
+16 -9
View File
@@ -42,7 +42,7 @@ from .. import model_gate
from .admin import _plural, _time_left
from .annotation_suggest import (
OptionError,
unavailable_message,
model_refusal,
validate_options,
)
from .annotations import (
@@ -58,7 +58,9 @@ from .annotations import (
from .autotag_routes import BATCH_CANCELLED, MODEL_BUSY_ERROR, model_busy_response
from .images import (
MAX_LISTING_SELECTION,
CollectionMissing,
ListingSourceChanged,
collection_not_found,
_current_fingerprint,
_listing_posix,
_public_path,
@@ -510,10 +512,8 @@ class ImageTagJobsMixin:
)
)
label = MODELS[model_type]["label"]
if not (status.get(model_type) or {}).get("downloaded"):
message = (
f"{label} is not downloaded. Download it in System › Auto-tagging."
)
message = model_refusal(status, model_type, label)
if message is not None:
return None, _refusal(409, "model-unavailable", message)
if backup_on and short is not None:
return None, _refusal(507, "insufficient-space", short_space_message(short))
@@ -689,6 +689,8 @@ class ImageTagJobsMixin:
selection = await self.resolve_listing_filter(spec.filter, cap)
except ListingSourceChanged:
return None, _coded("source-changed")
except CollectionMissing:
return None, collection_not_found()
except ValueError as exc:
return None, bad_request(str(exc))
except Exception:
@@ -788,13 +790,13 @@ class ImageTagJobsMixin:
async def _prepare_tagging_model(self, job, spec: _Spec, owner):
"""The loaded service, or None (logged) when the job cannot run."""
from ..autotag import MODELS, get_autotag_service
from ..autotag import MODELS, MissingBackendPackages, get_autotag_service
service = get_autotag_service()
status = await self._run_in_executor(service.get_models_status)
if not status.get(spec.model_type, {}).get("downloaded"):
name = MODELS.get(spec.model_type, {}).get("name", spec.model_type)
message = unavailable_message(name)
name = MODELS.get(spec.model_type, {}).get("name", spec.model_type)
message = model_refusal(status, spec.model_type, name)
if message is not None:
job.update(status_line=message)
job.log_line(f"! {message}")
return None
@@ -806,6 +808,11 @@ class ImageTagJobsMixin:
spec.model_type,
spec.options["use_gpu"],
)
except MissingBackendPackages as exc:
# A package gone since the status read (load_model logged it).
job.update(status_line=str(exc))
job.log_line(f"! {exc}")
return None
except Exception:
self.logger.exception("The auto-tag job could not load its model")
job.update(status_line=INFERENCE_FAILED)
+272 -76
View File
@@ -13,12 +13,13 @@ import re
import stat
import subprocess # nosec B404 - ffmpeg frames through run_encoder, fixed argv
import sys
import threading
import time as _time
import urllib.parse
from bisect import bisect_left, bisect_right, insort
from collections.abc import Mapping
from pathlib import Path
from typing import Callable, NamedTuple
from typing import Callable, Iterable, NamedTuple
from aiohttp import web
from PIL import Image, UnidentifiedImageError
@@ -58,6 +59,7 @@ from .inspection_service import (
)
try:
from ...database.image_collections import CollectionMissing
from ...database.image_favorites import MAX_KEY_LENGTH
from ...database.library import display_title
from ...database.models import normalize_image_path
@@ -66,6 +68,8 @@ try:
from ...utils.parallel import map_parallel
from ...utils.json_strict import dumps_strict
from ...utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_inspection_response,
library_metadata,
)
@@ -78,7 +82,9 @@ try:
reveal_executor,
reveal_in_file_manager,
)
from ...utils.workspace_keys import gallery_root_scope, workspace_keys_for
except ImportError:
from database.image_collections import CollectionMissing
from database.image_favorites import MAX_KEY_LENGTH
from database.library import display_title
from database.models import normalize_image_path
@@ -87,6 +93,8 @@ except ImportError:
from utils.parallel import map_parallel
from utils.json_strict import dumps_strict
from utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_inspection_response,
library_metadata,
)
@@ -99,6 +107,7 @@ except ImportError:
reveal_executor,
reveal_in_file_manager,
)
from utils.workspace_keys import gallery_root_scope, workspace_keys_for
IMAGE_EXTENSIONS = _IMAGE_EXTENSIONS
@@ -106,6 +115,21 @@ IMAGE_EXTENSIONS = _IMAGE_EXTENSIONS
_CARD_RECORDS = ("prompt", "workflow")
_PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
_LISTING_FAILED = "The image list could not be filtered. Check server logs for details."
_COLLECTION_ID = re.compile(r"[1-9][0-9]{0,8}")
COLLECTION_NOT_FOUND = "That collection no longer exists."
def collection_not_found() -> web.Response:
"""The 404 a write or a listing answers for an unknown collection (#331)."""
return web.json_response(
{
"success": False,
"code": "collection-not-found",
"error": COLLECTION_NOT_FOUND,
},
status=404,
headers={"Cache-Control": "no-store"},
)
def _card_header(path: Path) -> tuple:
@@ -497,7 +521,8 @@ MAX_JSON_BODY_BYTES = 1024 * 1024
_READ_CHUNK = 64 * 1024
_WORKSPACE_CURSOR_SECRET = os.urandom(32)
# Guards the first load of an API's workspace keys (#430, _workspace_keys).
_WORKSPACE_KEYS_LOCK = threading.Lock()
def _workspace_root(query: dict, roots: list[Path]) -> int | None:
@@ -565,7 +590,8 @@ class _ImageTagMatch(NamedTuple):
tags: bool # every filter tag is in its image tags or linked positive tags
query: bool # ``q`` is a case-folded substring of one image tag name
link: tuple | None # (hit, rating, positive id) of its unambiguous link
# (hit, rating, positive id, inherited category) of its unambiguous link
link: tuple | None
def _current_fingerprint(path: Path) -> tuple[int, int] | None:
@@ -596,15 +622,37 @@ def _rated_key(entry: _Listed, rank: int) -> tuple:
return (-rank, *_workspace_key(entry, "rating"))
def _workspace_cursor(key: tuple, scope: str) -> str:
# Bump whenever the cursor's key tuple changes shape: a cursor now outlives a
# restart and an upgrade (#430), so an old shape must be refused, not parsed.
_CURSOR_VERSION = 2
_CURSOR_STALE = "The image collection changed; refresh the gallery"
class CursorStale(ValueError):
"""A well-formed cursor this listing cannot verify (#430).
Another key, another root list or other filters: the page refreshes its
collection instead of retrying the same cursor.
"""
def _cursor_mac(secret: bytes, scope: str, payload: bytes) -> bytes:
# The listing scope (it hashes every absolute root) is bound in the MAC
# only, never sent: the scope is 64 hex, so the "\n" join is unambiguous.
return hmac.new(secret, scope.encode() + b"\n" + payload, hashlib.sha256).digest()
def _workspace_cursor(key: tuple, scope: str, secret: bytes) -> str:
"""The signed keyset cursor of *key*; *secret* is the API's cursor key."""
payload = json.dumps(
{"v": 1, "scope": scope, "key": key}, separators=(",", ":"), ensure_ascii=False
{"v": _CURSOR_VERSION, "key": key}, separators=(",", ":"), ensure_ascii=False
).encode()
signature = hmac.new(_WORKSPACE_CURSOR_SECRET, payload, hashlib.sha256).digest()
signature = _cursor_mac(secret, scope, payload)
return base64.urlsafe_b64encode(signature + payload).decode().rstrip("=")
def _workspace_cursor_key(token: str, scope: str, order: str) -> tuple:
def _workspace_cursor_key(token: str, scope: str, order: str, secret: bytes) -> tuple:
"""The keyset key of *token*; CursorStale when it is not this listing's."""
try:
if len(token) > 4096:
raise ValueError()
@@ -612,13 +660,14 @@ def _workspace_cursor_key(token: str, scope: str, order: str) -> tuple:
token + "=" * (-len(token) % 4), altchars=b"-_", validate=True
)
signature, payload = raw[:32], raw[32:]
expected = hmac.new(_WORKSPACE_CURSOR_SECRET, payload, hashlib.sha256).digest()
if not hmac.compare_digest(signature, expected):
if len(signature) != 32 or not payload:
raise ValueError()
if not hmac.compare_digest(signature, _cursor_mac(secret, scope, payload)):
raise CursorStale(_CURSOR_STALE)
data = json.loads(payload)
key = data["key"]
if data["v"] != 1 or data["scope"] != scope or not isinstance(key, list):
raise ValueError()
if data["v"] != _CURSOR_VERSION or not isinstance(key, list):
raise CursorStale(_CURSOR_STALE)
types = (
(str, str, int, (float, int))
if order in ("name", "name-desc")
@@ -632,6 +681,8 @@ def _workspace_cursor_key(token: str, scope: str, order: str) -> tuple:
):
raise ValueError()
return tuple(key)
except CursorStale:
raise
except (ValueError, KeyError, TypeError, UnicodeError, binascii.Error):
raise ValueError("Invalid gallery cursor; refresh the gallery") from None
@@ -1105,8 +1156,9 @@ def _workspace_filters(query: Mapping) -> dict:
the linked positive's text, title and tag names. ``category`` is exact,
``tags`` is a comma list that must all be present, ``sort=rating`` ranks by
the linked positive, and ``favorites=1`` keeps only favorites (the
Favorites page, #249 D51). Raises ValueError for any unknown or oversized
value.
Favorites page, #249 D51). ``collection=<id>`` (1 to 9 digits) keeps only
that collection's members (#331); it wins over ``favorites`` (a page never
sets both). Raises ValueError for any unknown or oversized value.
"""
text = query.get("q", "").strip()
folded = text.casefold()
@@ -1120,8 +1172,10 @@ def _workspace_filters(query: Mapping) -> dict:
{tag.strip() for tag in query.get("tags", "").split(",") if tag.strip()}
)
favorites = query.get("favorites", "0")
collection = query.get("collection", "")
if (
favorites not in ("0", "1")
or (collection != "" and _COLLECTION_ID.fullmatch(collection) is None)
or len(text) > 500
or len(folded) > 500
or media not in ("all", "image", "video")
@@ -1144,7 +1198,8 @@ def _workspace_filters(query: Mapping) -> dict:
"path": exact.casefold(),
"category": category,
"tags": tags,
"favorites": favorites == "1",
"favorites": favorites == "1" and not collection,
"collection": int(collection) if collection else None,
}
@@ -1162,6 +1217,8 @@ LISTING_FILTER_KEYS = frozenset(
"media",
"sort",
"favorites",
# A collection page's filter (#331): bulk actions act on its members.
"collection",
}
)
MAX_LISTING_FILTER_VALUE = 2048
@@ -1218,22 +1275,29 @@ def _admitted_entries(
entries: list,
filters: dict,
links: dict | None,
favorites: frozenset | None = None,
members: frozenset | None = None,
image_tags: dict | None = None,
own_categories: dict | None = None,
):
"""Yield ``(position, entry, link match)`` for each entry the filters admit.
*links* maps normalized paths to (hit, rating, positive id) for every
unambiguously linked image (see PromptDatabase.get_image_link_matches); a
category or tag filter admits only those, and ``q`` admits a filename match
or a linked-positive hit. *favorites*, when given, admits only those listed
paths (``favorites=1``). *image_tags* maps normalized paths to their
_ImageTagMatch: a tag filter also admits a file whose current image tags
(with its linked positive's tags) carry every tag, and ``q`` also admits an
image tag name hit.
*links* maps normalized paths to (hit, rating, positive id, inherited
category) for every unambiguously linked image (see
PromptDatabase.get_image_link_matches); a tag filter admits only those,
and ``q`` admits a filename match or a linked-positive hit. A category
admits a file whose effective category equals it (#332 MC5): its own
category (*own_categories*, ``{normalized path: category}``) when it has one,
else the inherited one, so an unlinked file with its own category is
admitted. *members*, when given, admits only those listed paths (the
favorites of ``favorites=1`` or a ``collection``'s members). *image_tags*
maps normalized paths to their _ImageTagMatch: a tag filter also admits a
file whose current image tags (with its linked positive's tags) carry every
tag, and ``q`` also admits an image tag name hit.
"""
query, media, folder = filters["query"], filters["media"], filters["folder"]
members_only = filters["category"] is not None or bool(filters["tags"])
category = filters["category"]
members_only = bool(filters["tags"])
own = own_categories or {}
native_folder = folder.replace("/", os.sep)
nested = native_folder + os.sep
exact = filters.get("path", "")
@@ -1246,12 +1310,18 @@ def _admitted_entries(
continue
if media != "all" and (media == "video") != entry.video:
continue
if favorites is not None and entry.path not in favorites:
if members is not None and entry.path not in members:
continue
match = None if links is None else links.get(entry.normalized)
image = None if image_tags is None else image_tags.get(entry.normalized)
if members_only and match is None and not (image and image.tags):
continue
if category is not None:
effective = own.get(entry.normalized) if own else None
if effective is None and match is not None:
effective = match[3]
if effective != category:
continue
if (
query
and query not in entry.folded
@@ -1268,8 +1338,9 @@ def _filtered_page(
size: int,
filters: dict,
links: dict | None,
favorites: frozenset | None = None,
members: frozenset | None = None,
image_tags: dict | None = None,
own_categories: dict | None = None,
) -> tuple[int, list, int | None]:
"""Count the *entries* the filters admit and take *size* from *start* on.
@@ -1279,7 +1350,7 @@ def _filtered_page(
"""
total, page, positives = 0, [], set()
for position, entry, match in _admitted_entries(
entries, filters, links, favorites, image_tags
entries, filters, links, members, image_tags, own_categories
):
total += 1
if match is not None:
@@ -1296,13 +1367,27 @@ def _in_subfolder(rel_dir, subfolder):
class ImageRoutesMixin:
"""Mixin providing image and gallery-related API endpoints."""
def _workspace_keys(self):
"""This API's scope and cursor keys, loaded once (#430).
They derive from the secret persisted beside the database, so a restart
keeps every scope and cursor an open page holds. The first call reads
(or writes) one small file; later calls return the cached keys, which
a live library move or restore does not change before a restart.
"""
keys = self.__dict__.get("_workspace_keys_cache")
if keys is None:
with _WORKSPACE_KEYS_LOCK:
keys = self.__dict__.get("_workspace_keys_cache")
if keys is None:
db = getattr(self, "db", None)
model = getattr(db, "model", None)
keys = workspace_keys_for(getattr(model, "db_path", None))
self.__dict__["_workspace_keys_cache"] = keys
return keys
def _gallery_root_scope(self, root: Path, index: int) -> str:
payload = json.dumps(
["prompt-manager-gallery-root-v1", index, str(_canonical(root))],
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
return hmac.new(_WORKSPACE_CURSOR_SECRET, payload, hashlib.sha256).hexdigest()
return gallery_root_scope(self._workspace_keys(), str(_canonical(root)), index)
def _gallery_scope_error(
self, query: Mapping, roots: list[Path]
@@ -1966,6 +2051,8 @@ class ImageRoutesMixin:
limit, _offset = parse_page_params(request.query, default_limit=100)
except ValueError:
return bad_request("limit and offset must be integers")
# #430: the first use may create the key file; never on the event loop.
cursor_secret = (await self._run_in_executor(self._workspace_keys)).cursor
try:
root_index = _workspace_root(request.query, output_dirs)
filters = _workspace_filters(request.query)
@@ -1983,27 +2070,38 @@ class ImageRoutesMixin:
filters["category"],
filters["tags"],
filters["favorites"],
filters["collection"],
]
).encode()
).hexdigest()
token = request.query.get("cursor")
after = (
_workspace_cursor_key(token, scope, filters["order"]) if token else None
_workspace_cursor_key(token, scope, filters["order"], cursor_secret)
if token
else None
)
except CursorStale as exc:
return web.json_response(
{"success": False, "code": "cursor-stale", "error": str(exc)},
status=400,
headers={"Cache-Control": "no-store"},
)
except ValueError as exc:
return bad_request(str(exc))
listings = [
(index, root, await self._get_gallery_files(root))
for index, root in enumerate(output_dirs)
if root_index is None or index == root_index
]
try:
if filters["collection"] is not None and not await self._run_in_executor(
self.db.collection_exists, filters["collection"]
):
raise CollectionMissing("No such collection")
listings = await self._workspace_listings(output_dirs, root_index)
page, total, has_more, next_cursor, positive_count = (
await self._run_in_executor(
self._select_workspace_page, listings, filters, after, limit, scope
)
)
except CollectionMissing:
return collection_not_found()
except Exception:
# A filter that needs the prompt links or the image tags cannot be
# served without them. Never echo the exception: it can carry SQL
@@ -2219,9 +2317,10 @@ class ImageRoutesMixin:
stat'ed (never every listed file), and a candidate's tags count only
while a byte record is verified at its present ``(size, mtime_ns)``
(spec amendment S1). Under a Tags filter the candidates' linked
positives are read too, so their tags and category combine.
positives are read too, so their tags combine; the category is checked
once, on the effective category, by _admitted_entries (#332).
"""
tags, query, category = filters["tags"], filters["query"], filters["category"]
tags, query = filters["tags"], filters["query"]
wanted = set(tags)
located = self._workspace_locators(listings)
# The tag index answers which files carry a relevant name (#379); no
@@ -2251,11 +2350,9 @@ class ImageRoutesMixin:
link = facts.get(entry.normalized)
carried = set(names).union(link[4] if link else ())
matches[entry.normalized] = _ImageTagMatch(
bool(wanted)
and wanted <= carried
and (category is None or (link is not None and link[3] == category)),
bool(wanted) and wanted <= carried,
bool(query) and any(query in name.casefold() for name in names),
link[:3] if link else None,
tuple(link[:4]) if link else None,
)
return matches
@@ -2264,10 +2361,16 @@ class ImageRoutesMixin:
) -> tuple:
"""The ordered entries and what admits them (blocking; run in executor).
Returns ``(entries, key, links, image_tags, favorites)``: the listing in
the filters' order, its keyset key, and the link, image-tag and favorite
maps _admitted_entries reads. One SELECT (get_image_link_matches) reads
every linked positive when *read_links*. A failed read raises when the
Returns ``(entries, key, links, image_tags, members, own)``: the
listing in the filters' order, its keyset key, and the link, image-tag,
member and own-category maps _admitted_entries reads (members: a
collection's, else the favorites, else None). A collection deleted since
the route checked it raises CollectionMissing. One SELECT
(get_image_link_matches) reads every linked positive when *read_links*;
under a category it keeps only links whose raw recipe or positive
category is that value (the inherited one is decided in Python), and
the links of files whose own category is that value are read by path,
so their rating and positive still count (#332). A failed read raises when the
filters need it (category, tags, the rating sort) or *strict*; else the
listing stays usable without links (a search then matches filenames
only). A failed image-tag read raises under a Tags filter; a search then
@@ -2279,10 +2382,10 @@ class ImageRoutesMixin:
try:
links = self.db.get_image_link_matches(
filters["text"],
category,
tags,
every_link=True,
wanted=self._workspace_wanted(listings),
category=category,
)
except Exception:
if strict or category is not None or tags or order == "rating":
@@ -2296,6 +2399,13 @@ class ImageRoutesMixin:
),
exc_info=True,
)
own = None
if category is not None:
# A failed read raises: an unfiltered page must never pass for the
# category (DC6).
own = self._own_category_map(self._get_all_output_dirs(), listings)
if links is not None:
links = self._own_category_links(links, own, filters)
image_tags = None
if filters["query"] or tags:
try:
@@ -2339,10 +2449,37 @@ class ImageRoutesMixin:
def key(entry):
return _workspace_key(entry, order)
favorites = (
self._workspace_favorite_paths(listings) if filters["favorites"] else None
return (
entries,
key,
links,
image_tags,
self._workspace_members(listings, filters),
own,
)
return entries, key, links, image_tags, favorites
def _own_category_links(self, links: dict, own: dict, filters: dict) -> dict:
"""*links* plus the links of files whose own category is the filter's.
Their inherited category differs, so the category's link read left
them out; read by path (get_image_link_facts), kept only when their
positive has every filter tag, as the link read would.
"""
category, wanted = filters["category"], set(filters["tags"])
missing = [
path
for path, value in own.items()
if value == category and path not in links
]
if not missing:
return links
facts = self.db.get_image_link_facts(missing, filters["text"])
extra = {
path: tuple(fact[:4])
for path, fact in facts.items()
if wanted <= set(fact[4])
}
return {**links, **extra} if extra else links
def _select_workspace_page(
self,
@@ -2371,7 +2508,7 @@ class ImageRoutesMixin:
or bool(filters["tags"])
or order == "rating"
)
entries, key, links, image_tags, favorites = self._workspace_admission(
entries, key, links, image_tags, members, own = self._workspace_admission(
listings, filters, read_links=needed or after is None
)
start = 0
@@ -2382,13 +2519,17 @@ class ImageRoutesMixin:
else:
start = bisect_right(entries, after, key=key)
total, page, positive_count = _filtered_page(
entries, start, limit + 1, filters, links, favorites, image_tags
entries, start, limit + 1, filters, links, members, image_tags, own
)
if not needed and after is not None:
positive_count = _COUNT_OMITTED
has_more = len(page) > limit
page = page[:limit]
next_cursor = _workspace_cursor(key(page[-1]), scope) if has_more else None
next_cursor = (
_workspace_cursor(key(page[-1]), scope, self._workspace_keys().cursor)
if has_more
else None
)
return (
[(entry.path, entry.mtime, entry.root, entry.index) for entry in page],
total,
@@ -2404,12 +2545,12 @@ class ImageRoutesMixin:
The links are read strictly: the positives must be the listing's own.
"""
entries, _key, links, image_tags, favorites = self._workspace_admission(
entries, _key, links, image_tags, members, own = self._workspace_admission(
listings, filters, read_links=True, strict=True
)
targets, positives, unlinked, scopes = [], {}, 0, {}
for _position, entry, match in _admitted_entries(
entries, filters, links, favorites, image_tags
entries, filters, links, members, image_tags, own
):
if len(targets) == cap:
return ListingSelection(targets, list(positives), unlinked, True)
@@ -2431,9 +2572,11 @@ class ImageRoutesMixin:
) -> ListingSelection:
"""The images a parse_listing_filter result admits right now (#392).
Raises ValueError for a root outside the current roots, and
Raises ValueError for a root outside the current roots,
ListingSourceChanged when the filter's ``root_scope`` no longer names
its root (the sources were renumbered since the listing).
its root (the sources were renumbered since the listing), and
CollectionMissing when its ``collection`` no longer exists (checked
before any scan and again after the members are read).
"""
output_dirs = list(self._get_all_output_dirs())
root_index = _workspace_root(params, output_dirs)
@@ -2443,11 +2586,11 @@ class ImageRoutesMixin:
raise ListingSourceChanged()
raise ValueError("Invalid gallery root scope")
filters = _workspace_filters(params)
listings = [
(index, root, await self._get_gallery_files(root))
for index, root in enumerate(output_dirs)
if root_index is None or index == root_index
]
if filters["collection"] is not None and not await self._run_in_executor(
self.db.collection_exists, filters["collection"]
):
raise CollectionMissing("No such collection")
listings = await self._workspace_listings(output_dirs, root_index)
return await self._run_in_executor(
self._select_workspace_matches, listings, filters, cap
)
@@ -3632,25 +3775,58 @@ class ImageRoutesMixin:
self._favorite_listing_cache[cache_key] = (files, listing)
return listing
def _workspace_favorite_paths(self, listings: list[tuple]) -> frozenset:
"""The listed paths of every favorite in the selected roots (blocking).
def _workspace_members(self, listings: list[tuple], filters: dict):
"""The member set the filters ask for, or None (blocking)."""
collection = filters.get("collection")
if collection is not None:
members = self._workspace_member_paths(
listings,
lambda root_key: self.db.collection_members(collection, root_key),
)
# Checked after the reads: a collection deleted before or while
# they ran is a 404, never an empty page passing for it.
if not self.db.collection_exists(collection):
raise CollectionMissing("No such collection")
return members
if filters["favorites"]:
return self._workspace_favorite_paths(listings)
return None
One primary-key read per root (get_root_favorites), matched against the
root's cached ``{relative key: listing spelling}``, so the work follows
the favorites rather than the files, and a key folds case exactly as
the toggle stored it (Windows). A favorite whose file is gone matches
nothing. A failed read raises: an unfiltered page must never pass for
the favorites.
def _workspace_favorite_paths(self, listings: list[tuple]) -> frozenset:
"""The listed paths of every favorite in the selected roots (blocking)."""
return self._workspace_member_paths(listings, self.db.get_root_favorites)
def _workspace_member_paths(
self, listings: list[tuple], keys_for_source: Callable[[str], Iterable[str]]
) -> frozenset:
"""The listed paths of the stored keys of each selected root (blocking).
One read per root (``keys_for_source(root key)``: the favorites' or a
collection's primary-key range), matched against the root's cached
``{relative key: listing spelling}``, so the work follows the members
rather than the files, and a key folds case exactly as it was stored
(Windows). A member whose file is gone matches nothing. A failed read
raises: an unfiltered page must never pass for the members.
"""
paths = set()
for _index, root, files in listings:
listing = self._favorite_listing(root, files)
for key in self.db.get_root_favorites(_favorite_root_key(root)):
for key in keys_for_source(_favorite_root_key(root)):
listed = listing.get(key)
if listed is not None:
paths.add(root / listed)
return frozenset(paths)
async def _workspace_listings(
self, output_dirs: list[Path], root_index: int | None
) -> list[tuple]:
"""``(index, root, scan)`` of every selected root (cached scans)."""
return [
(index, root, await self._get_gallery_files(root))
for index, root in enumerate(output_dirs)
if root_index is None or index == root_index
]
def _workspace_favorites(self, page: list[tuple]) -> set[tuple[int, str]]:
"""``(root index, listing path)`` of the favorites among *page* entries."""
by_root: dict[tuple[int, Path], dict[str, str]] = {}
@@ -3728,13 +3904,20 @@ class ImageRoutesMixin:
except Exception:
# The link stays known; only its title is withheld.
self.logger.warning("Inspection link title could not be read")
# An output file's own category (#332); LoRA roots have none.
own = (
self._read_own_category(outputs, path)
if index < len(outputs)
else None
)
try:
observed = inspect_current_file(
path,
source_label=_public_path(roots[index]),
saved=saved,
library=self._inspection_library(
saved, unavailable=unavailable
library=apply_own_category(
self._inspection_library(saved, unavailable=unavailable),
own,
),
image_tags=self._inspection_image_tags(
roots[index], Path(relative).parts
@@ -3802,6 +3985,19 @@ class ImageRoutesMixin:
return read
def _read_own_category(self, sources, path):
"""One file's own category, None, or OWN_CATEGORY_UNREAD (blocking).
Keyed by ``image_category_key`` over the output sources, as Images
writes it. A failed read is logged and hides the category (DC5).
"""
try:
key = image_category_key(sources, path)
return None if key is None else self.db.image_category(*key)
except Exception:
self.logger.warning("Image category could not be read", exc_info=True)
return OWN_CATEGORY_UNREAD
def _inspection_library(self, saved: dict | None, *, unavailable: bool) -> dict:
"""The inspector's Library facts; the route sets the title afterwards."""
result = {
+4
View File
@@ -1227,7 +1227,9 @@ class PromptRoutesMixin:
"""
from .images import (
MAX_LISTING_SELECTION,
CollectionMissing,
ListingSourceChanged,
collection_not_found,
parse_listing_filter,
)
@@ -1252,6 +1254,8 @@ class PromptRoutesMixin:
},
status=409,
)
except CollectionMissing:
return collection_not_found()
except ValueError as exc:
return refused(str(exc))
if selection.truncated:
+1 -1
View File
@@ -3,7 +3,7 @@
The rule (maintainer test of #284/#288): a file is queued only when its
final outcome is ambiguous and unlinked. The strict resolver must find it
``ambiguous``, the scan's legacy fallback (the browser parser,
``_graph_prompt_text``) must not link it, and no saved row may link it
``graph_prompt_text``) must not link it, and no saved row may link it
already. A file the fallback links, an exact pair the ingester stored and an
already linked file are never queued. Neither is an ambiguous file with
nothing to choose: fewer than two of its candidates carry a prompt
+100
View File
@@ -0,0 +1,100 @@
"""Where a stale Images target is now (#430): ``POST /prompt_manager/images/rescope``.
A page addresses a listed file as ``{root_index, root_scope, path}``. When the
sources are renumbered its scope no longer names the root at that index, and
every per-file action answers 409 ``source-changed``. Before it gives up, the
page asks here. The target is mapped only when its scope is, at its old
index, the scope of one of the current roots: the server issued that scope
for exactly that folder, so the answer (the file's current listing
descriptor) names nothing the page could not already address. A removed or
replaced root, a scope of another key and a foreign scope stay refused.
Only the Images roots (``_get_all_output_dirs``) are mapped; a LoRA-linked
root's target is never re-scoped here.
"""
import hmac
from pathlib import Path
from aiohttp import web
from .images import (
_forbidden,
_is_thumbnail_cache_path,
_json_body_or_error,
_media_access_error,
)
from .media_target import media_target
from .prompts import bad_request
# Far above any real source count; keeps the index out of huge-int territory.
MAX_ROOT_INDEX = 4095
SOURCE_GONE = (
"The source of this image changed or was removed. Refresh the image "
"collection to find it again."
)
def _coded(status: int, code: str, message: str) -> web.Response:
return web.json_response(
{"success": False, "code": code, "error": message}, status=status
)
def _no_store(response: web.Response) -> web.Response:
response.headers["Cache-Control"] = "no-store"
return response
class SourceRescopeMixin:
"""The rescope route; composed into ``PromptManagerAPI``."""
def _register_rescope_routes(self, routes):
@routes.post("/prompt_manager/images/rescope")
async def rescope_route(request):
return _no_store(await self.rescope_image(request))
def _current_root_for(self, scope: str, old_index: int, roots: list) -> int | None:
"""The index of the first current root *scope* was issued for at *old_index*."""
for index, root in enumerate(roots):
if hmac.compare_digest(scope, self._gallery_root_scope(root, old_index)):
return index
return None
def _rescope_blocking(self, scope: str, old_index: int, relative: str):
"""The answer to one rescope (blocking: it resolves paths)."""
roots = list(self._get_all_output_dirs())
index = self._current_root_for(scope, old_index, roots)
if index is None:
return _coded(409, "source-changed", SOURCE_GONE)
root = Path(roots[index])
path = root / relative
denied = _media_access_error(path, [root])
if denied is not None:
return denied
try:
mtime = path.stat().st_mtime if path.is_file() else None
except OSError:
mtime = None
if mtime is None:
return _coded(404, "file-missing", "Image file not found")
[image] = self._format_workspace_page([(path, mtime, root, index)])
return web.json_response({"success": True, "image": image})
async def rescope_image(self, request) -> web.Response:
data, error = await _json_body_or_error(request)
if error is not None:
return error
if set(data) != {"target"}:
return bad_request("Send exactly one target")
target, error = media_target(data["target"])
if error is not None:
return error
old_index, scope, relative = target
if old_index > MAX_ROOT_INDEX:
return bad_request("Invalid gallery root")
if _is_thumbnail_cache_path(relative):
return _forbidden("Previews are not re-scoped; use the original")
return await self._run_in_executor(
self._rescope_blocking, scope, old_index, relative
)
+121 -7
View File
@@ -7,6 +7,7 @@ Refactored from standalone_tagger.py for integration with PromptManager API.
import errno
import gc
import importlib.util
import inspect
import os
import sys
@@ -155,6 +156,91 @@ MODELS = {
},
}
# The Python modules each backend imports when it loads, with their pip names
# (#424). None of them is a PromptManager requirement: they are optional, and
# llama-cpp-python needs a build for the user's GPU. Nothing is ever installed.
BACKEND_PACKAGES: Dict[str, Tuple[Tuple[str, str], ...]] = {
"gguf": (("llama_cpp", "llama-cpp-python"),),
# The default 8-bit load needs bitsandbytes, and device_map needs accelerate.
"hf": (
("torch", "torch"),
("transformers", "transformers"),
("accelerate", "accelerate"),
("bitsandbytes", "bitsandbytes"),
),
"wd14-swinv2": (("onnxruntime", "onnxruntime"),),
"wd14-vit": (("onnxruntime", "onnxruntime"),),
}
_INSTALL_NOTES = {
"gguf": (
'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python'
),
"wd14-swinv2": "Use onnxruntime-gpu for CUDA.",
"wd14-vit": "Use onnxruntime-gpu for CUDA.",
}
_PIP_NAMES = {
module: pip for packages in BACKEND_PACKAGES.values() for module, pip in packages
}
def _importable(module: str) -> bool:
"""Whether *module* (a top-level name) can be imported; imports nothing."""
if sys.modules.get(module) is not None:
return True
try:
return importlib.util.find_spec(module) is not None
except (ImportError, ValueError, OSError):
return False
def missing_packages(model_type: str) -> List[str]:
"""The pip names of the packages *model_type* needs and cannot import."""
return [
pip
for module, pip in BACKEND_PACKAGES.get(model_type, ())
if not _importable(module)
]
def package_fields(model_type: str, missing: List[str]) -> Dict[str, Any]:
"""The status fields of *missing*: pip names, install command and note."""
missing = list(missing)
return {
"missing_packages": missing,
"install_command": "pip install " + " ".join(missing) if missing else None,
"install_note": _INSTALL_NOTES.get(model_type) if missing else None,
}
def _joined(names: List[str]) -> str:
if len(names) <= 1:
return "".join(names)
return ", ".join(names[:-1]) + " and " + names[-1]
def missing_packages_message(model_type: str, missing: List[str]) -> str:
"""The one sentence every refusal of a model with missing packages uses."""
label = MODELS.get(model_type, {}).get("label", model_type)
fields = package_fields(model_type, missing)
pronoun = "it" if len(missing) == 1 else "them"
message = (
f"{label} needs {_joined(missing)}. Install {pronoun} into ComfyUI's "
f"Python: {fields['install_command']}."
)
if fields["install_note"]:
message += " " + fields["install_note"]
return message
class MissingBackendPackages(RuntimeError):
"""A model whose backend packages cannot be imported (#424)."""
def __init__(self, model_type: str, missing: List[str]):
self.model_type = model_type
self.missing = list(missing)
super().__init__(missing_packages_message(model_type, self.missing))
# Default prompts
DEFAULT_PROMPT = DEFAULT_TAG_TEMPLATE
@@ -167,6 +253,10 @@ SYSTEM_PROMPT = (
)
def _none(_model_type: str) -> List[str]:
return []
def _load_rgb(image_path: str) -> Image.Image:
"""Decode the image as RGB and close the file before inference.
@@ -203,6 +293,7 @@ class AutoTagService:
self,
models_dir: Optional[Path] = None,
model_loader: Optional[ModelLoader] = None,
package_probe: Optional[Callable[[str], List[str]]] = None,
):
"""Initialize the AutoTag service.
@@ -213,9 +304,16 @@ class AutoTagService:
``(kind, backend)`` tuple used by ``generate_tags``.
Defaults to ``_load_model``, which imports the ML
backends lazily. Tests inject a fake here.
package_probe: Callable ``(model_type)`` returning the pip names
of the backend packages that cannot be imported.
Defaults to ``missing_packages`` with the default
loader; an injected loader brings its own backend.
"""
self.logger = get_logger("autotag.service")
self._model_loader: ModelLoader = model_loader or self._load_model
if package_probe is None:
package_probe = missing_packages if model_loader is None else _none
self._package_probe = package_probe
# Determine models directory
if models_dir:
@@ -320,6 +418,9 @@ class AutoTagService:
# Get the actual path (local or cache)
model_status["model_path"] = str(self._get_hf_model_path())
model_status.update(
package_fields(model_type, self._package_probe(model_type))
)
status[model_type] = model_status
return status
@@ -455,13 +556,9 @@ class AutoTagService:
def _load_wd14_tagger(self, model_type: str, use_gpu: bool = True):
"""Load WD14 ONNX model and tag list."""
try:
import onnxruntime as ort
except ImportError:
raise RuntimeError(
"onnxruntime is required for WD14 models. "
"Install with: pip install onnxruntime (or onnxruntime-gpu for CUDA)"
)
# A missing onnxruntime is refused by load_model before this runs
# (#424); a ModuleNotFoundError here becomes the same refusal.
import onnxruntime as ort
import csv
config = MODELS[model_type]
@@ -694,6 +791,7 @@ class AutoTagService:
Raises:
ValueError: If model_type is invalid
RuntimeError: If model not downloaded or loading fails
MissingBackendPackages: If its backend packages cannot be imported
"""
if model_type not in MODELS:
raise ValueError(f"Invalid model type: {model_type}")
@@ -705,6 +803,11 @@ class AutoTagService:
status = self.get_models_status()
if not status[model_type]["downloaded"]:
raise RuntimeError(f"Model {model_type} not downloaded")
missing = status[model_type].get("missing_packages") or []
if missing:
refusal = MissingBackendPackages(model_type, missing)
self.logger.warning(str(refusal))
raise refusal
try:
self._tagger = self._model_loader(model_type, use_gpu)
@@ -712,6 +815,17 @@ class AutoTagService:
self.logger.info(f"Model {model_type} loaded successfully")
return True
except ModuleNotFoundError as e:
# A package find_spec found but that does not import (a broken
# install), or a dependency of one: the same refusal, no traceback.
self._tagger = None
self._current_model_type = None
module = (e.name or "").split(".")[0] or str(e)
refusal = MissingBackendPackages(
model_type, [_PIP_NAMES.get(module, module)]
)
self.logger.warning(str(refusal))
raise refusal from None
except Exception as e:
self.logger.error(f"Failed to load model {model_type}: {e}")
self._tagger = None
+9 -1
View File
@@ -19,7 +19,7 @@ test.beforeEach(async ({page, browserName}) => {
await import('/extensions/promptmanager/prompt_manager.js');
class Node {
constructor(){
this.widgets=['text','category','tags','search_text'].map(name=>({name,value:'',inputEl:document.createElement('textarea')}));
this.widgets=[...['text','category','tags','search_text'].map(name=>({name,value:'',inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true}];
this.widgets_values=[]; this.inputs=[];
}
addDOMWidget(name,type,element){document.body.append(element);return {element};}
@@ -36,6 +36,14 @@ test.afterEach(async ({page,browserName}) => {
expect(page.classicErrors).toEqual([]);
});
test('Classic loads an older workflow with Save to library on and keeps a saved off (#90)',async({page})=>{
// A workflow saved before the switch held the search UI's slot ('') at the switch's index
const loaded=await page.evaluate(()=>{const widget=node.widgets.find(w=>w.name==='save_to_library');widget.value='';node.configure({});return widget.value;});
expect(loaded).toBe(true);
const off=await page.evaluate(()=>{const widget=node.widgets.find(w=>w.name==='save_to_library');widget.value=false;node.configure({});return [widget.value,node.serialize().widgets_values[4]];});
expect(off).toEqual([false,false]);
});
test('Classic keeps immediate selection, search query and metadata serialization',async({page})=>{
await page.route('**/prompt_manager/search?**', route=>route.fulfill({json:{success:true,results:[{text:'A new portrait',category:'Portrait',tags:['film grain'],rating:5}]}}));
await page.evaluate(()=>{node.widgets.find(w=>w.name==='search_text').value='tag:"film grain" portrait';});
+31 -1
View File
@@ -187,6 +187,33 @@ export async function mutate(page,{css=null,remove=[],clone=[]}={}) {
},token);
}
/**
* #332 D146: the category chip names its origin after the category, in a muted `span.chip-source` that never splits
* (workspace.css `.chip .chip-source`). The concept's chip shows the bare name (app.js a402), so the recorded rule and
* the span are added to the concept page at runtime and the browser lays the chip out: the first `.tags` row's
* non-badge chip under `root`. `category` also replaces the chip's text (an own category the concept never shows).
* Returns the undo.
*/
export const D146_CHIP_SOURCE='.chip .chip-source { color: var(--ca-muted); white-space: nowrap; }';
export async function withChipSource(page,root,origin,category=null) {
const restoreRule=await mutate(page,{css:D146_CHIP_SOURCE});
const token=await page.evaluate(({root,origin,category})=>{
const chip=document.querySelector(root)?.querySelector('.tags')?.querySelector(':scope > .chip:not(.blue)');
if(!chip)throw new Error(`withChipSource: no category chip under ${root}`);
const text=chip.textContent,source=document.createElement('span');
source.className='chip-source';source.textContent=` · ${origin}`;
if(category!==null)chip.textContent=category;
chip.append(source);
const token=`chip-source-${Math.random().toString(36).slice(2)}`;
(window.__chipSources??=new Map()).set(token,()=>{chip.textContent=text;});
return token;
},{root,origin,category});
return async()=>{
await page.evaluate(token=>{window.__chipSources.get(token)();window.__chipSources.delete(token);},token);
await restoreRule();
};
}
/**
* Inventory sections: {chrome, heading, toolbar, filterRow, bulkbar, caption, grid, cards, continuation, panel, library,
* history, noPageScroll}. A section is null when its element is absent. The Images sections (toolbar … panel, #249 C7)
@@ -615,9 +642,12 @@ export function measureInspector(selector) {
primary:node.classList.contains('ca-primary'),...box(node),...font(node),...edges(node),...layout(node),
minHeight:style.minHeight,minWidth:style.minWidth,icon:iconOf(node)};
}
// #332 D146: a category chip's visible origin (`span.chip-source`), null on every other chip.
function chipOf(node) {
const source=node.querySelector(':scope > .chip-source');
return {tag:node.tagName.toLowerCase(),className:classes(node),text:textOf(node),hidden:hiddenOf(node),...box(node),...font(node),...edges(node),
minHeight:getComputedStyle(node).minHeight};
minHeight:getComputedStyle(node).minHeight,
source:when(source,()=>({text:source.textContent,...font(source),...css(source,'whiteSpace'),...box(source)}))};
}
function rowOf(row) {
return {...layout(row),...edges(row),...box(row),chips:[...row.children].map(chipOf)};
+28 -2
View File
@@ -26,7 +26,7 @@ test.beforeEach(async({page,browserName})=>{
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings:new EventTarget()},graph:{setDirtyCanvas(){}},canvas:{}};
await import('/extensions/promptmanager/prompt_manager.js');
class Node {
constructor(){this.widgets=['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[550,700];}
constructor(){this.widgets=[...['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true,options:{}}];this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}})),{name:'save_to_library',type:'BOOLEAN',link:null,widget:{name:'save_to_library'}}];this.properties={};this.size=[550,700];}
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.height='650px';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
@@ -41,6 +41,32 @@ test.beforeEach(async({page,browserName})=>{
});
test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);expect(page.focusedErrors).toEqual([]);});
test(`${variant} Save to library switch saves its value and loads older workflows on (#90)`, async ({page}) => {
const toggle=page.getByRole('checkbox',{name:'Save to library',exact:true});
await expect(toggle).toBeChecked();
expect(await page.evaluate(()=>node.widgets.map(widget=>widget.name))).toEqual(['text','category','tags','search_text','prepend_text','append_text','save_to_library']);
// The native switch is drawn in the toolbar only: the host keeps it hidden and zero-height, like the text fallbacks
expect(await page.evaluate(()=>{const widget=node.widgets.find(item=>item.name==='save_to_library');return {type:widget.type,hidden:widget.hidden,size:widget.computeSize()};})).toEqual({type:'hidden',hidden:true,size:[0,-4]});
expect(await page.evaluate(()=>node.widgets.find(item=>item.name==='text').computeSize())).toEqual([0,-4]);
await toggle.uncheck();
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Save to library is off: runs of this node are not added to the library.');
const saved=await page.evaluate(()=>node.serialize());
expect(saved.widgets_values[6]).toBe(false);
await page.evaluate(saved=>node.configure(saved),saved);
await expect(toggle).not.toBeChecked();
await page.getByLabel('Main prompt',{exact:true}).fill('Typed while off');
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Save to library is off: runs of this node are not added to the library.');
// A workflow saved before the switch held the editor's own slot ('') at its index
await page.evaluate(()=>node.configure({widgets_values:['Old text','','','','','',''],properties:{}}));
await expect(toggle).toBeChecked();
expect(await page.evaluate(()=>node.widgets.find(widget=>widget.name==='save_to_library').value)).toBe(true);
await toggle.check();
await page.getByLabel('Main prompt',{exact:true}).fill('Typed while on');
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Main prompt is saved when this node runs.');
await page.evaluate(()=>{node.inputs.find(input=>input.name==='save_to_library').link=9;node.onConnectionsChange?.();});
await expect(toggle).toBeDisabled();
});
test(`${variant} library autocompletes filters and searches without changing editor metadata`, async ({page}) => {
const requests=[];
await page.route('**/prompt_manager/categories',route=>route.fulfill({json:{success:true,categories:['Portrait']}}));
@@ -257,7 +283,7 @@ test(`${variant} restores workflow values and disclosures and protects connected
test(`${variant} offers text as a link socket, keeps typed text while linked and through a reload (#343)`,async({page})=>{
const sockets=()=>page.evaluate(()=>Object.fromEntries(node.inputs.map(input=>[input.name,input.widget?.name??null])));
// The hidden text widget no longer owns the input, so the host draws it as a socket; the other widget inputs are unchanged.
expect(await sockets()).toEqual({clip:null,prepend_text:null,append_text:null,text:null,category:'category',tags:'tags',search_text:'search_text'});
expect(await sockets()).toEqual({clip:null,prepend_text:null,append_text:null,text:null,category:'category',tags:'tags',search_text:'search_text',save_to_library:'save_to_library'});
expect(await page.evaluate(()=>node.findInputByType('STRING').slot.name)).toBe('text');
expect(await page.evaluate(()=>node.findInputByType('CLIP').slot.name)).toBe('clip');
const main=page.getByRole('textbox',{name:'Main prompt',exact:true});
+53 -2
View File
@@ -136,7 +136,8 @@ test('panel matches the concept order and copy',async({page,request})=>{
await expect(panel.locator('.detail-inner > p.small')).toHaveText('complete.png');
const badges=panel.locator('.detail-inner > .tags').first();
await expect(badges.locator('span.chip.blue')).toHaveText('Workflow inferred');
await expect(badges.locator('span.chip:not(.blue)')).toHaveText('Recipe category: Inspection');
// #332 MC8: the chip names where its value came from, in visible text after the category.
await expect(badges.locator('span.chip:not(.blue)')).toHaveText('Recipe category: Inspection · recipe');
await expect(badges.locator('span.chip:not(.blue) .ca-visually-hidden')).toHaveText('Recipe category: ');
// Recipe tags are not the positive tags the Images Tags filter matches, so they stay plain chips (D21 ruling).
await expect(panel.locator('.detail-inner > .tags').last().locator('span.chip')).toHaveText(['fixture','geometric']);
@@ -185,6 +186,53 @@ test('panel matches the concept order and copy',async({page,request})=>{
await expect(panel.locator('.detail-inner > button')).toHaveText(['Retry prompt details']);
});
// #332 CUJ 4, CUJ 5 (MC8, D146): every category chip names its origin: the image's own, the prompt's or the recipe's.
const originChip=scope=>scope.locator('span.chip:has(> span.chip-source)');
async function expectOrigin(scope,{prefix,category,origin}) {
await expect(originChip(scope)).toHaveText(`${prefix}${category} · ${origin}`);
await expect(originChip(scope).locator('.ca-visually-hidden')).toHaveText(prefix);
expect(await originChip(scope).locator('.chip-source').evaluate(node=>node.textContent)).toBe(` · ${origin}`);
}
async function putOwnCategory(request,paths,category) {
const {images}=await(await request.get('/prompt_manager/images/output?mode=workspace&subfolder=inspector&limit=40')).json();
const items=paths.map(path=>images.find(image=>image.root_index===0&&image.path===path)).map(({root_index,root_scope,path})=>({root_index,root_scope,path}));
const response=await request.put('/prompt_manager/images/category',{data:{items,category}});await expect(response).toBeOK();
}
test('the category chip names its origin in the side panel and the viewer (#332 CUJ 4, CUJ 5)',async({page,request})=>{
await page.setViewportSize({width:1600,height:1100});
await expect(await request.post('/__test__/reset',{data:{full_inspector:true}})).toBeOK();
// The positive's own category, set through the legacy prompt route; complete.png's recipe keeps "Inspection".
const list=await(await request.get('/prompt_manager/library')).json();const id=list.items.find(item=>item.text.startsWith('Inspector portrait')).id;
await expect(await request.post('/prompt_manager/bulk/category',{data:{prompt_ids:[id],category:'Landscapes'}})).toBeOK();
await page.goto('/prompt_manager/workspace#/images');const panel=inspector(page);
await selectFile(page,'complete.png');await expect(panel.locator('.panel-heading h2')).toHaveText(POSITIVE_TITLE);
await expectOrigin(panel,{prefix:'Recipe category: ',category:'Inspection',origin:'recipe'});
await selectFile(page,'workflow-only.png');await expect(panel.locator('.detail-inner > p.small')).toHaveText('workflow-only.png');
await expectOrigin(panel,{prefix:'Positive prompt category: ',category:'Landscapes',origin:'prompt'});
// An unlinked file shows no category until it has its own.
await selectFile(page,'empty.png');await expect(panel.locator('.panel-heading h2')).toHaveText('empty.png');
await expect(originChip(panel)).toHaveCount(0);
// Own categories: one overrides the prompt's on a linked file, one sits on an unlinked file.
await putOwnCategory(request,['inspector/workflow-only.png'],'Portraits');await putOwnCategory(request,['inspector/empty.png'],'Scans');
await page.reload();
await selectFile(page,'workflow-only.png');await expect(panel.locator('.detail-inner > p.small')).toHaveText('workflow-only.png');
await expectOrigin(panel,{prefix:'Image category: ',category:'Portraits',origin:'this image'});
await selectFile(page,'empty.png');await expect(panel.locator('.panel-heading h2')).toHaveText('empty.png');
await expectOrigin(panel,{prefix:'Image category: ',category:'Scans',origin:'this image'});
await selectFile(page,'complete.png');await expect(panel.locator('.panel-heading h2')).toHaveText(POSITIVE_TITLE);
await expectOrigin(panel,{prefix:'Recipe category: ',category:'Inspection',origin:'recipe'});
// The viewer's full details show the same chips.
let details=await viewerDetails(page,'workflow-only.png');
await expectOrigin(details,{prefix:'Image category: ',category:'Portraits',origin:'this image'});
await page.keyboard.press('Escape');await expect(page.locator('dialog#viewer-dialog[open]')).toHaveCount(0);
details=await viewerDetails(page,'complete.png');
await expectOrigin(details,{prefix:'Recipe category: ',category:'Inspection',origin:'recipe'});
await page.keyboard.press('Escape');await expect(page.locator('dialog#viewer-dialog[open]')).toHaveCount(0);
details=await viewerDetails(page,'empty.png');
await expectOrigin(details,{prefix:'Image category: ',category:'Scans',origin:'this image'});
});
test('tag chip filters Images by that tag',async({page,request})=>{
await expect(await request.post('/__test__/reset',{data:{full_inspector:true}})).toBeOK();
const list=await(await request.get('/prompt_manager/library')).json();const id=list.items.find(item=>item.text.startsWith('Inspector portrait')).id;
@@ -676,6 +724,9 @@ test('collection viewer uses real public transforms and resets on navigation',as
async function slideshowFixture(page,request,preferences={videoAutoplay:false,videoMute:true,videoLoop:false}) {
await expect(await request.post('/__test__/reset',{data:{full_inspector:true,include_video:true}})).toBeOK();
await page.goto('/prompt_manager/workspace#/images');
// Install the fake clock before the fixture wraps setTimeout: the wrapped timer (which the viewer session captures)
// must be the fake one, so page.clock.runFor alone decides when the 3500 ms slideshow deadline fires.
await page.clock.install();
await page.evaluate(async preferences=>{
if(preferences) localStorage.setItem('gallerySettings',JSON.stringify(preferences));
const [{createMediaViewer},{createInspectionController},{createScope},{createPromptSender}]=await Promise.all([
@@ -695,7 +746,7 @@ async function slideshowFixture(page,request,preferences={videoAutoplay:false,vi
},preferences);
await expect(mediaDialog(page).getByRole('button',{name:'Zoom in',exact:true})).toHaveAttribute('aria-disabled','false');
await expect(mediaDialog(page).locator('#viewer-details > div > p.small')).toHaveText('complete.png');
await page.clock.install();await page.clock.pauseAt(new Date(Date.now()+1000));
await page.clock.pauseAt(new Date(Date.now()+1000));
}
const slideshowButton=page=>mediaDialog(page).locator('#viewer-toolbar').getByRole('button',{name:/^(Play|Pause|Stop) slideshow$/});
const timerCount=page=>page.evaluate(()=>window.slideshowTimers.size);
+57 -1
View File
@@ -1,7 +1,7 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact} from './concept-inventory.mjs';
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact,measureInspector,withChipSource} from './concept-inventory.mjs';
// Task F5 of #249: the production Prompt history (#/library/<id>) against the immutable concept's (#prompt/alpine,
// app.js renderHistory a587–622, the dialogs a1282–1304, style.css 1059–1179 and 1546–1567), on the concept-shaped real
@@ -272,6 +272,62 @@ for(const [width,height] of SIZES)for(const theme of THEMES)test(`history geomet
} finally {await host.evaluate(()=>window.conceptHost.dispose());await host.close();}
});
/**
* D145 (#332): the History media inspector (a recorded run's current file, `recorded-run.mjs`) shows the file's own
* category with the chip rule of the Images side panel and viewer (D146). The concept has no recorded-run inspector, so
* its chip is the concept viewer's own category chip (app.js a402, imageDetails(i, true)) with the recorded D146 origin
* and the own category added at runtime (`withChipSource`). The chip's own place depends on what precedes it in either
* inspector, so its box is compared by size only.
*/
const OWN_CATEGORY='Portraits';
const chipOnly=chip=>{const {left,top,source,...rest}=chip;return {...rest,source:source&&(({left,top,...own})=>own)(source)};};
const categoryChipOf=async(page,root)=>(await page.evaluate(measureInspector,root)).badges.chips.find(chip=>chip.source);
async function conceptViewerChip(page,theme) {
await openConcept(page,'images',theme);await page.mouse.move(0,0);
await page.locator('.gallery-body > .image-grid > .media-card').first().locator('.image-open').click();
await expect(page.locator('#viewer-dialog[open] #viewer-details .panel-heading h2')).toHaveText(TITLE);
const restore=await withChipSource(page,'#viewer-details','this image',OWN_CATEGORY);await settle(page);
const chip=await categoryChipOf(page,'#viewer-details');
await restore();
return chip;
}
/** Recorded run for the baseline's newest run, its one reported output opened in the viewer. */
async function productionRunChip(page,theme,positiveId) {
await openProductionHistory(page,theme,'no-preference',positiveId);
await page.getByRole('button',{name:'Show repeat run for Quality baseline',exact:true}).click();
const run=page.getByRole('dialog',{name:'Recorded run',exact:true});
await expect(run).toBeVisible();
const view=run.getByRole('button',{name:'View current file',exact:true});
await expect(view).toBeEnabled();await view.click();
const details=page.locator('#viewer-dialog[open] #viewer-details');
await expect(details.locator('.chip-source')).toHaveText(' · this image');
await page.mouse.move(0,0);await settle(page);
return categoryChipOf(page,'#viewer-dialog[open] #viewer-details');
}
async function setOwnCategory(request,path,category) {
const listing=await (await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json();
const root_scope=listing.images.find(item=>item.root_index===0).root_scope;
const response=await request.put('/prompt_manager/images/category',{data:{items:[{root_index:0,root_scope,path}],category}});
await expect(response).toBeOK();
}
for(const [width,height] of SIZES)for(const theme of THEMES)test(`history media inspector category chip matches the concept ${width}x${height} ${theme} (D145)`,async({page,request},testInfo)=>{
await page.setViewportSize({width,height});
const {manifest}=testInfo;
// The baseline's newest image: the output of the run its dot opens (the fixture records one run per image).
const baseline=manifest.images.find(image=>image.prompt==='alpine'&&image.variant===0);
await setOwnCategory(request,baseline.path,OWN_CATEGORY);
const reference=await conceptViewerChip(page,theme);
const actual=await productionRunChip(page,theme,manifest.prompts[0].positive_id);
await captureCanonicalState(page,testInfo,`production-history-run-chip-${width}-${theme}`,'#viewer-dialog[open]');
expect.soft(reference,'concept chip').toMatchObject({tag:'span',className:'chip',text:`${OWN_CATEGORY} · this image`,hidden:null});
// D21: the hidden prefix names the scope; D146 as above.
expect.soft(diffInventory({...chipOnly(reference),hidden:'Image category: '},chipOnly(actual)),'chip differences after D21, D145, D146').toEqual([]);
});
// The comparison is only worth something if it notices a change (spec §7 mutation check): one CSS mutation (the variant
// dot's width) and one DOM mutation (Compare negatives removed), each applied after an exact match.
test('history comparison detects mutations',async({page},testInfo)=>{
+117 -4
View File
@@ -85,15 +85,17 @@ function listInfo(card,actionsWidth) {
/** Recorded Images deviations only (spec §8); everything else in the concept inventory must equal production. */
const nocase=(a,b)=>{const x=a.toLowerCase(),y=b.toLowerCase();return x<y?-1:x>y?1:a<b?-1:a>b?1:0;};
export function mapImagesConcept(reference,{manifest,now=new Date(manifest.now*1000)}) {
export function mapImagesConcept(reference,{manifest,now=new Date(manifest.now*1000),imageCategories=[]}) {
const paths=new Map(manifest.images.map(image=>[image.filename,image.path]));
// D14: the sort appends production's two sorts. Save view is the concept's own (saved views, #249 D16 retired).
const toolbar={...reference.toolbar};
toolbar.items=toolbar.items.map(item=>item.id==='sort'?{...item,options:[...item.options,'Oldest first','Name Z–A']}:item);
// D36: the category list is the Library's real list, sorted alphabetically (database/library.py `ORDER BY category
// COLLATE NOCASE, category`); the concept lists its samples in prompt order (app.js 348). Only the order is mapped.
// D141 (#332): the list also holds the listed images' own categories (`GET /prompt_manager/images/categories`),
// sorted in with the Library's.
toolbar.items=toolbar.items.map(item=>item.id==='category-filter'
?{...item,options:[item.options[0],...item.options.slice(1).sort(nocase)]}:item);
?{...item,options:[item.options[0],...[...item.options.slice(1),...imageCategories].sort(nocase)]}:item);
// D29: no "Illustrative previews" sample tag.
const filterRow=withoutItems(reference.filterRow,item=>String(item.className).includes('sample-tag'));
filterRow.items=filterRow.items.map(item=>{
@@ -163,9 +165,9 @@ const pick=inventory=>stripExact(inventory);
// The concept's own sample categories, in its first-appearance order (app.js 348): what D36 reorders.
const CONCEPT_CATEGORIES=['All categories','Landscapes','Architecture','Environments','Illustration'];
function compare(reference,actual,manifest,sections=SECTIONS) {
function compare(reference,actual,manifest,sections=SECTIONS,imageCategories=[]) {
const only=inventory=>Object.fromEntries(sections.map(key=>[key,inventory[key]]));
return diffInventory(pick(only(mapImagesConcept(reference,{manifest}))),pick(only(actual)));
return diffInventory(pick(only(mapImagesConcept(reference,{manifest,imageCategories}))),pick(only(actual)));
}
/** The comparison as a gate: throws, naming every differing path, unless production equals the mapped concept. */
@@ -470,6 +472,117 @@ for(const [width,height] of SIZES)for(const theme of THEMES)test(`images selecti
expect(compare(oracle,await inventoryOf(page,'#workspace-main'),manifest,['bulkbar']).map(item=>item.path)).toContain('bulkbar.bar.paddingTop');
});
/** #332: an own category on one listed image, set through the real route (PUT /prompt_manager/images/category). */
const IMAGE_CATEGORY='Portraits';
async function setImageCategory(request,path,category) {
const listing=await (await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json();
const root_scope=listing.images.find(item=>item.root_index===0).root_scope;
const response=await request.put('/prompt_manager/images/category',{data:{items:[{root_index:0,root_scope,path}],category}});
await expect(response).toBeOK();
expect(await response.json()).toMatchObject({success:true,category,updated:1});
}
/**
* Runs in the page: the open Set category dialog (the concept's `simpleInput`, app.js 1384-1390 and 1138): its title,
* the body's tree, the field, the input and its suggestions (`datalist`), and the actions with every button's box,
* all relative to the dialog, with the concept's geometry and tokens.
*/
function measureCategoryDialog() {
const dialog=document.querySelector('dialog#modal[open]');
const css=(node,...keys)=>node?Object.fromEntries(keys.map(key=>[key,getComputedStyle(node)[key]])):null;
const name=node=>[node.localName,...[...node.classList].sort()].join('.');
const text=node=>node?.textContent.trim()??null;
const origin=dialog.getBoundingClientRect();
const box=node=>{const rect=node.getBoundingClientRect();return {left:Math.round(rect.left-origin.left),top:Math.round(rect.top-origin.top),
width:Math.round(rect.width),height:Math.round(rect.height)};};
const body=dialog.querySelector('#modal-body');
const field=body.querySelector(':scope > .field');
const label=field?.querySelector(':scope > label'),input=field?.querySelector(':scope > input');
const list=body.querySelector(':scope > datalist');
const actions=body.querySelector(':scope > .dialog-actions');
return {
title:text(dialog.querySelector('#modal-title')),
tree:[...body.children].map(name),
field:field&&{tree:[...field.children].map(name),...css(field,'display','flexDirection','rowGap','marginBottom'),...box(field)},
label:label&&{text:text(label),for:label.htmlFor,...css(label,'fontSize','fontWeight','color')},
input:input&&{id:input.id,type:input.type,value:input.value,list:input.getAttribute('list'),
...css(input,'height','paddingLeft','fontSize','color','backgroundColor','borderTopColor','borderTopLeftRadius'),...box(input)},
suggestions:list&&{id:list.id,shown:list.getClientRects().length>0,options:[...list.options].map(option=>option.value)},
actions:{tree:[...actions.children].map(name),...css(actions,'display','justifyContent','columnGap','rowGap','flexWrap','marginTop'),...box(actions),
buttons:[...actions.children].map(button=>({text:text(button),className:button.className,
...css(button,'backgroundColor','color','fontSize','paddingTop','paddingLeft','borderTopColor','minHeight'),...box(button)}))},
width:Math.round(origin.width),height:Math.round(origin.height),
};
}
/**
* The concept's own open dialog as production records it, changed at runtime so the browser lays it out: D140 (#332)
* empties the field (the concept pre-fills its demo "Landscapes"), lists the toolbar's categories as the field's
* suggestions (`datalist#workspace-input-suggestions`) and adds "Use prompt category", a plain button of the concept's
* own Cancel, between Cancel and the primary action; D8 names the primary action "Apply" ("Apply in prototype").
*/
async function categoryDialogAsRecorded(page,suggestions) {
await mutate(page,{clone:[{selector:'dialog#modal[open] .dialog-actions > button:first-child',text:'Use prompt category'}]});
await page.evaluate(suggestions=>{
const body=document.querySelector('dialog#modal[open] #modal-body');
const input=body.querySelector('#action-input');input.value='';input.setAttribute('list','workspace-input-suggestions');
const list=document.createElement('datalist');list.id='workspace-input-suggestions';
for(const value of suggestions){const option=document.createElement('option');option.value=value;list.append(option);}
body.querySelector(':scope > .field').after(list);
body.querySelector('.dialog-actions > .ca-primary').textContent='Apply';
},suggestions);
await settle(page);
return page.evaluate(measureCategoryDialog);
}
async function openCategoryDialog(page,scope) {
await page.locator(scope).locator(CARDS).first().locator('.image-overlays > label > input[type="checkbox"]').check();
await expect(page.locator('#bulkbar > .bulkbar > strong')).toHaveText('1 selected');
await page.locator('#bulkbar').getByRole('button',{name:'Set category',exact:true}).click();
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Categorize selected images');
await page.mouse.move(0,0);await settle(page);
return page.evaluate(measureCategoryDialog);
}
// D140 and D141 (#332): with one image's own category set, the toolbar's category options and the Set category dialog
// against the concept's (app.js 348 and 1384-1390). The dialog is compared with the concept page itself changed at
// runtime as recorded (`categoryDialogAsRecorded`); unmapped, it differs only in the recorded parts.
for(const [width,height] of SIZES)for(const theme of THEMES)test(`images category options and Set category dialog match the concept ${width}x${height} ${theme}`,async({page,request},testInfo)=>{
await page.setViewportSize({width,height});
const manifest=testInfo.manifest;
await setImageCategory(request,manifest.images.at(-1).path,IMAGE_CATEGORY);
await openConceptLoaded(page,theme,'no-preference');
const reference=await inventoryOf(page,'#main');
const referenceDialog=await openCategoryDialog(page,'#main');
await captureCanonicalState(page,testInfo,`concept-images-category-${width}-${theme}`,'dialog#modal');
const suggestions=[...CONCEPT_CATEGORIES.slice(1),IMAGE_CATEGORY].sort(nocase);
const oracle=await categoryDialogAsRecorded(page,suggestions);
await page.keyboard.press('Escape');await expect(page.locator('dialog#modal[open]')).toHaveCount(0);
await openProductionLoaded(page,theme,'no-preference');
await expect(page.locator('#category-filter option')).toHaveText(['All categories',...suggestions]);
const actual=await inventoryOf(page,'#workspace-main');
const actualDialog=await openCategoryDialog(page,'#workspace-main');
await captureCanonicalState(page,testInfo,`production-images-category-${width}-${theme}`,'dialog#modal');
const path=testInfo.outputPath('images-category-geometry.json');
await writeFile(path,JSON.stringify({referenceDialog,oracle,actualDialog,reference:reference.toolbar,actual:actual.toolbar},null,2));
await testInfo.attach('images-category-geometry',{path,contentType:'application/json'});
// The concept itself: its sample categories and its pre-filled simpleInput with Cancel / Apply in prototype.
expect.soft(reference.toolbar.items.find(item=>item.id==='category-filter').options,'concept categories').toEqual(CONCEPT_CATEGORIES);
expect.soft(referenceDialog,'concept dialog').toMatchObject({title:'Categorize selected images',label:{text:'Category'},
input:{id:'action-input',value:'Landscapes',list:null},suggestions:null});
expect.soft(referenceDialog.actions.buttons.map(button=>button.text),'concept actions').toEqual(['Cancel','Apply in prototype']);
// D141: the toolbar options add the image's own category, sorted in (D36).
expect.soft(compare(reference,actual,manifest,['toolbar'],[IMAGE_CATEGORY]),'toolbar differences after D14, D36, D141').toEqual([]);
// D140 and D8: the dialog equals the concept page laid out as recorded.
expect.soft(diffInventory(oracle,actualDialog),'dialog differences after D8, D140').toEqual([]);
// Unmapped, the concept differs only in the field's value and suggestions, the body's datalist and the actions.
const recorded=/^(input\.(value|list)|tree\.|suggestions|actions\.)/;
expect.soft(diffInventory(referenceDialog,actualDialog).map(item=>item.path).filter(item=>!recorded.test(item)),'unrecorded dialog differences').toEqual([]);
expect.soft(actualDialog.suggestions,'suggestions').toEqual({id:'workspace-input-suggestions',shown:false,options:suggestions});
});
// The comparison is only worth something if it notices a change (spec §7 mutation check): one CSS mutation (the
// `.media-info` padding) and one DOM mutation (the Inspect buttons removed), each applied after an exact match.
test('images comparison detects mutations',async({page},testInfo)=>{
+27 -6
View File
@@ -1,7 +1,7 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {SIZES,THEMES,seedConcept,openConcept,openProduction,inspectorOf,diffInventory,mutate,countCopy,saveImageTags} from './concept-inventory.mjs';
import {SIZES,THEMES,seedConcept,openConcept,openProduction,inspectorOf,diffInventory,mutate,countCopy,saveImageTags,withChipSource} from './concept-inventory.mjs';
// Task D4 of #249: the Images side panel (app.js imageDetails a400–414) and the viewer's full details (imageDetails(i,
// true)) against the immutable concept, on the concept-shaped real fixture, at 1600x1100 and 390x844 in both themes.
@@ -13,6 +13,8 @@ test.use({locale:'en-US',timezoneId:'UTC'});
const CARDS='.gallery-body > .image-grid > .media-card';
const TITLE='Alpine observatory';
// D146 (#332): alpine_00001.png's recipe has no category, so its chip shows the positive's, " · prompt" (D39).
const ORIGIN='prompt';
const FIXTURE_IMAGES=24;
// The concept's sticky panel never scrolls; production's scrolls inside the window above 700px (D20/D21 side panel).
const PANEL_INSET=40;
@@ -38,14 +40,18 @@ function withFocusablePrompts(inventory) {
return {...inventory,controls,sections:inventory.sections.map(section=>({...section,text:{...section.text,tabIndex:0}}))};
}
/** D21: the category chip names its scope in a visually hidden prefix; here the linked positive's (D39 fallback). */
/**
* D21: the category chip names its scope in a visually hidden prefix; here the linked positive's (D39 fallback). Its
* visible origin (D146, #332: " · prompt") is the concept page's own chip with `span.chip-source` added at runtime
* (`withChipSource`), so `reference.panel` and `viewerPlain` already carry it.
*/
function withCategoryScope(inventory) {
const [badge,category,...rest]=inventory.badges.chips;
return {...inventory,badges:{...inventory.badges,chips:[badge,{...category,hidden:'Positive prompt category: '},...rest]}};
}
/**
* Recorded inspector deviations only (spec §8, TESTING.md D3, D20–D22, D34, D38 and D117); everything else must
* Recorded inspector deviations only (spec §8, TESTING.md D3, D20–D22, D34, D38, D117 and D146); everything else must
* equal production. `reference.viewerPlain` is the concept's own full details
* with its tag chips made plain (D20), D117's summary button added and, under a coarse pointer, D34's targets, all at
* runtime: the browser lays out what follows them, as the Images spec's removed toolbar items do.
@@ -112,6 +118,7 @@ async function openProductionPanel(page,theme) {
*/
async function plainViewerChips(page) {
const restoreTargets=await mutate(page,{css:D34_COARSE_TARGETS});
const restoreSource=await withChipSource(page,'#viewer-details',ORIGIN);
await page.evaluate(()=>{
window.__plainChips=[...document.querySelectorAll('#viewer-details .tags > button.chip')].map(button=>{
const span=document.createElement('span');span.className='chip';span.textContent=button.textContent;
@@ -126,7 +133,7 @@ async function plainViewerChips(page) {
return async()=>{
await page.evaluate(()=>{for(const [span,button] of window.__plainChips)span.replaceWith(button);delete window.__plainChips;
window.__summaryButton.remove();delete window.__summaryButton;});
await restoreTargets();
await restoreSource();await restoreTargets();
};
}
@@ -162,7 +169,11 @@ async function viewerOf(page,concept,mutation=null) {
return {viewerDetails,viewerPlain,fileControls};
}
async function inventory(page,concept=false) {
// D146: the concept panel's category chip with its visible origin, laid out by the concept page itself.
const restoreSource=concept?await withChipSource(page,'#image-inspector',ORIGIN):null;
if(concept)await settle(page);
const {panel}=await inspectorOf(page);
await restoreSource?.();
const {viewerDetails,viewerPlain,fileControls}=await viewerOf(page,concept);
return concept?{panel,viewerDetails,viewerPlain,fileControls}:{panel,viewerDetails,fileControls};
}
@@ -288,7 +299,8 @@ for(const [width,height] of SIZES)for(const theme of THEMES)test(`inspector pane
expect.soft(panel.sections.map(section=>section.label.color)[0],'concept role colors').not.toBe(panel.sections[1].label.color);
expect.soft(panel.facts,'concept facts').toMatchObject({columns:2,columnGap:'12px',rowGap:'12px',fontSize:width>700?'11px':'12px'});
expect.soft(panel.footer,'concept footer').toMatchObject({paddingTop:'13px',paddingRight:'17px',paddingBottom:'13px',paddingLeft:'17px'});
expect.soft(panel.badges.chips.map(chip=>chip.text),'concept badges').toEqual(['Execution recorded','Landscapes']);
// The concept's bare "Landscapes" with the D146 origin added at runtime (`withChipSource`).
expect.soft(panel.badges.chips.map(chip=>chip.text),'concept badges').toEqual(['Execution recorded','Landscapes · prompt']);
expect.soft(reference.viewerDetails.structure.inner.slice(-2),'concept full details').toEqual(['details','details']);
expect.soft(diff,'differences after the recorded deviations').toEqual([]);
@@ -322,8 +334,10 @@ test('a recipe-linked image without recipe metadata shows its positive prompt me
await expect(panelOf(page).locator('.panel-heading h2')).toHaveText('Quiet geometry');
const rows=panelOf(page).locator('.detail-inner > .tags');
await expect(rows).toHaveCount(2);
await expect(rows.first().locator('.chip')).toHaveText(['Execution recorded','Positive prompt category: Architecture']);
await expect(rows.first().locator('.chip')).toHaveText(['Execution recorded','Positive prompt category: Architecture · prompt']);
await expect(rows.first().locator('.chip .ca-visually-hidden')).toHaveText('Positive prompt category: ');
// D146 (#332): the visible origin names where the value came from.
await expect(rows.first().locator('.chip .chip-source')).toHaveText(' · prompt');
await expect(rows.nth(1).locator('button.chip')).toHaveText(['minimal','warm_light','editorial']);
await expect(rows.nth(1).locator('span.chip')).toHaveCount(0);
await expect(panelOf(page).locator('.detail-footer button')).toHaveText(['Send to ComfyUI','Prompt history']);
@@ -400,6 +414,13 @@ test('inspector comparison detects mutations',async({page})=>{
expect(()=>assertInspectorMatch(reference,withoutSummary,viewport)).toThrow(/viewerDetails\.structure\.inner/);
assertInspectorMatch(reference,await viewerMutated(),viewport);
// D146: the visible origin is part of the mapped concept, so losing it is a difference.
const restoreSource=await mutate(page,{remove:['#image-inspector .chip-source']});
const sourceMutated=await measure();
expect(()=>assertInspectorMatch(reference,sourceMutated,viewport)).toThrow(/panel\.badges\.chips\.1\.source/);
await restoreSource();await settle(page);
assertInspectorMatch(reference,await measure(),viewport);
await mutate(page,{remove:['#image-inspector > .detail-footer']});
const domMutated=await measure();
expect(()=>assertInspectorMatch(reference,domMutated,viewport)).toThrow(/panel\.footer/);
+13 -1
View File
@@ -17,7 +17,7 @@ test.beforeEach(async({page,browserName})=>{
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings},graph:{_nodes:[],setDirtyCanvas(){}},canvas:{},graphToPrompt:async()=>({})};
await import('/extensions/promptmanager/prompt_manager.js');
class Node {
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[1100,700];}
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=[...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true,options:{}}];this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}})),{name:'save_to_library',type:'BOOLEAN',link:null,widget:{name:'save_to_library'}}];this.properties={};this.size=[1100,700];}
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.cssText='height:700px;width:1100px;max-width:100%';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
@@ -34,6 +34,18 @@ test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);
async function selectPair(page){await page.getByRole('button',{name:'Recent',exact:true}).click();await page.locator('.pm-editor-result').click();}
async function draft(page,positive='Draft positive',negative='Draft negative'){await page.getByRole('textbox',{name:'Positive prompt',exact:true}).fill(positive);await page.getByRole('textbox',{name:'Negative prompt',exact:true}).fill(negative);}
test('paired editor Save to library switch keeps the widget order and loads older workflows on (#90)',async({page})=>{
await expect(page.locator('.pm-pair')).toBeVisible();
const toggle=page.getByRole('checkbox',{name:'Save to library',exact:true});
await expect(toggle).toBeChecked();
expect(await page.evaluate(()=>node.widgets.map(widget=>widget.name))).toEqual(['text','negative_text','category','tags','search_text','prepend_text','append_text','save_to_library']);
await toggle.uncheck();
expect(await page.evaluate(()=>node.serialize().widgets_values[7])).toBe(false);
await expect(page.locator('.pm-pair > .pm-editor-status[role=status]').last()).toHaveText('Save to library is off: runs of this node are not added to the library.');
await page.evaluate(()=>node.configure({widgets_values:['P','N','','','','','',''],properties:{}}));
await expect(toggle).toBeChecked();
});
test('paired editor previews both texts before explicitly replacing both',async({page})=>{
await expect(page.locator('.pm-pair')).toBeVisible();
await draft(page);
+3
View File
@@ -38,6 +38,9 @@ async function chooseDestination(page) {
test('collection viewer rejects stale source and sends only refreshed API evidence',async({page,request,context})=>{
const host=await fakeHost(context,page);
// #430: the file is never found again here, so the manual Refresh is what this covers (the found-again path is
// workspace-stale-source.spec.mjs).
await page.route('**/prompt_manager/images/rescope',route=>route.fulfill({status:409,json:{success:false,code:'source-changed',error:'The source of this image changed or was removed.'}}));
await page.addInitScript(()=>{window.copied=[];Object.defineProperty(navigator,'clipboard',{configurable:true,value:{writeText:async text=>window.copied.push(text)}});});
try {
const loaded=await(await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json();const stale=loaded.images.find(image=>image.root_index===0&&image.path==='shared/same.png');
+13 -1
View File
@@ -46,7 +46,9 @@ from tests.browser.workspace_library_editor_fixture import (
)
from tests.browser.workspace_vocabulary_fixture import register_vocabulary_commands
from tests.browser.workspace_tagging_fixture import register_tagging_commands
from tests.browser.source_change_fixture import register_source_change_commands
from tests.browser.workspace_review_fixture import register_review_commands
from tests.browser.workspace_collections_fixture import register_collections_commands
from tests.browser.workspace_diagnostics_fixture import (
register_diagnostics_commands,
repoint_logs,
@@ -75,6 +77,8 @@ class TestModel:
loaded = None
# Models the Tag image dialog workflows mark as not downloaded.
unavailable: set = set()
# Pip names per model whose backend packages "cannot be imported" (#424).
missing_packages: dict = {}
# Scored names the concept comparisons answer instead of the fixed list
# (``/__test__/tagging/suggestions``); None keeps the fixed list.
suggestions: list | None = None
@@ -82,10 +86,13 @@ class TestModel:
models_config: dict = {}
def get_models_status(self) -> dict:
from py.autotag import package_fields
return {
name: {
"downloaded": name not in self.unavailable,
"loaded": self.loaded == name,
**package_fields(name, self.missing_packages.get(name, [])),
}
for name in self.models_config
}
@@ -225,8 +232,10 @@ def create_app(root: Path) -> web.Application:
history_commands = register_workspace_history_commands(routes, api, root)
library_editor_commands = register_library_editor_commands(routes, api, root)
register_vocabulary_commands(routes, api, root)
register_tagging_commands(routes, api, root, model)
restore_sources = register_source_change_commands(routes, api)
register_tagging_commands(routes, api, root, model, on_reset=restore_sources)
register_review_commands(routes, api, root)
register_collections_commands(routes, api, root)
reveal_stub = register_reveal_stub(routes, api, root)
register_activity_commands(app, routes, api)
register_system_commands(app, routes, api, root)
@@ -262,6 +271,8 @@ def create_app(root: Path) -> web.Application:
with target.db.model.get_connection() as conn:
for table in (
"saved_views",
"image_collection_items",
"image_collections",
"image_favorites",
"media_review_decisions",
"media_review_items",
@@ -430,6 +441,7 @@ def create_app(root: Path) -> web.Application:
async def reset(request: web.Request) -> web.Response:
nonlocal history_fixture
request_holds.release()
restore_sources()
# No preview queued by the last test may land in the new fixture.
await api._run_in_executor(api.close_auto_thumbnails)
library_editor_commands.reset()
+35 -7
View File
@@ -76,13 +76,41 @@ function savedNav(nav) {
return {...nav,rows:nav.rows.map(row=>row.text==='Needs review'?{...row,count:{...row.count,text:'0'}}:row)};
}
/**
* D136 (#331): the concept's COLLECTIONS rows (app.js a336–339) as production lists them, by name (DC1): the fixture's
* "Landscapes" and "Experiments" hold the concept's 12 and 8 images, so text and counts compare as they are and each
* place keeps the concept's row geometry. At <=950px the concept hides the nav, which would leave a phone no way to
* open a collection: production shows it as the concept lays out every other sidebar nav there (its SAVED VIEWS nav),
* its rows styled as that nav's rows. Text, widths and wrap gaps then follow the rows' own content.
*/
function collectionsNav(nav,navs) {
const byName=[...nav.rows].sort((a,b)=>a.text<b.text?-1:a.text>b.text?1:0);
if(nav.shown)return {...nav,rows:byName.map((row,index)=>({...row,gapAfter:nav.rows[index].gapAfter}))};
const saved=navs.find(item=>item.id==='saved-nav');
const {id,label,rows,...frame}=saved;
return {...nav,...frame,id:nav.id,label:nav.label,rows:byName.map(row=>({...rows[0],text:row.text,
count:row.count&&rows[0].count?{...rows[0].count,text:row.count.text}:rows[0].count,
icon:rows[0].icon&&row.icon?{...rows[0].icon,d:row.icon.d}:row.icon,
width:'(content)',hugs:'(content)',gapAfter:'(content)',current:null}))};
}
/** The production collections nav at <=950px with its content-sized values replaced, as collectionsNav maps them. */
function contentFree(navs) {
return navs.map(nav=>nav.id!=='collections-nav'||!nav.shown?nav:{...nav,
rows:nav.rows.map(row=>({...row,width:'(content)',hugs:'(content)',gapAfter:'(content)'}))});
}
/** D136: where the concept hides COLLECTIONS, the production rows' content-sized values are not compared. */
function comparableNavs(reference,navs) {
return reference.chrome.navs.find(nav=>nav.id==='collections-nav')?.shown?navs:contentFree(navs);
}
/** Recorded shell deviations only (spec §8). Everything else in the concept inventory must equal production. */
function mapShellConcept(reference) {
const {chrome}=reference;
const navs=chrome.navs
// D28: COLLECTIONS is a proposal with no persisted store, so its nav is not rendered.
.filter(nav=>nav.id!=='collections-nav')
.map(nav=>{
if(nav.id==='collections-nav')return collectionsNav(nav,chrome.navs);
if(nav.id==='saved-nav')return savedNav(nav);
if(nav.id!=='system-nav')return nav;
const system=rowNamed(nav,'System');
@@ -99,9 +127,8 @@ function mapShellConcept(reference) {
return {...reference,chrome:{...chrome,
// D29: no prototype suffix in the document title.
title:chrome.title.replace(/ · Prompt Manager concept$/,' · Prompt Manager'),
// D28: no COLLECTIONS label. D49: SAVED VIEWS without its "PROPOSED" badge.
labels:chrome.labels.filter(label=>label.text!=='COLLECTIONS PROPOSED')
.map(label=>label.text==='SAVED VIEWS PROPOSED'?{...label,text:'SAVED VIEWS'}:label),
// D49: SAVED VIEWS without its "PROPOSED" badge. D136: COLLECTIONS without it either.
labels:chrome.labels.map(label=>label.text.endsWith(' PROPOSED')?{...label,text:label.text.replace(/ PROPOSED$/,'')}:label),
navs,
connection:{...chrome.connection,
// D28: truthful copy instead of "Demo connection · Classic / Nodes 2.0".
@@ -134,8 +161,9 @@ function headingFrame(heading) {
function compare(route,reference,actual) {
const expected=mapShellConcept(reference);
const heading=FULL_HEADING.has(route);
const chrome={...actual.chrome,navs:comparableNavs(reference,actual.chrome.navs)};
return [
...diffInventory({chrome:expected.chrome,noPageScroll:expected.noPageScroll},{chrome:actual.chrome,noPageScroll:actual.noPageScroll}),
...diffInventory({chrome:expected.chrome,noPageScroll:expected.noPageScroll},{chrome,noPageScroll:actual.noPageScroll}),
...diffInventory(heading?expected.heading:headingFrame(expected.heading),heading?actual.heading:headingFrame(actual.heading))
.map(item=>({...item,path:`heading.${item.path}`})),
];
@@ -242,7 +270,7 @@ for(const [width,height] of SIZES)test(`a saved view row matches the concept's $
expect(reference.chrome.navs.find(nav=>nav.id==='saved-nav').rows.map(row=>row.text)).toEqual(['Favorites','Needs review','Golden hour']);
await openProduction(page,'images','dark');
const actual=await save('Save','#workspace-main');
expect(diffInventory(mapShellConcept(reference).chrome.navs,actual.chrome.navs)).toEqual([]);
expect(diffInventory(mapShellConcept(reference).chrome.navs,comparableNavs(reference,actual.chrome.navs))).toEqual([]);
const referenceRows=reference.chrome.navs.find(nav=>nav.id==='saved-nav').rows;
expect(actual.chrome.navs.find(nav=>nav.id==='saved-nav').rows.map(row=>[row.text,row.count]))
.toEqual([['Favorites',referenceRows[0].count],['Needs review',{...referenceRows[1].count,text:'0'}],['Golden hour',null]]);
+66
View File
@@ -0,0 +1,66 @@
"""Source changes under an open workspace page (#430), undone by every reset.
- ``POST /__test__/sources/reverse``: the image sources are listed in reverse
order, as System › Sources would renumber them, so every held
``root_scope`` names another index and the page must re-scope its file.
- ``POST /__test__/scope-key/rotate``: the API's scope and cursor keys
change, as when ``workspace-scope.key`` is lost before a restart: a held
scope then cannot be mapped and the page offers Refresh.
"""
from __future__ import annotations
import secrets
from typing import TYPE_CHECKING, Callable
from aiohttp import web
from utils.workspace_keys import WorkspaceKeys
if TYPE_CHECKING:
from py.api import PromptManagerAPI
_CACHE = "_workspace_keys_cache"
def register_source_change_commands(
routes: web.RouteTableDef, api: PromptManagerAPI
) -> Callable[[], None]:
"""Register the commands; returns the restore every reset runs."""
# Imported here: py.config needs the host stubs create_app installs first.
from py.config import GalleryConfig
saved: dict = {}
def restore() -> None:
if "roots" in saved:
GalleryConfig.MONITORING_DIRECTORIES = saved.pop("roots")
api.invalidate_gallery_cache(retire=True)
if "keys" in saved:
keys = saved.pop("keys")
if keys is None:
api.__dict__.pop(_CACHE, None)
else:
api.__dict__[_CACHE] = keys
@routes.post("/__test__/sources/reverse")
async def reverse(request: web.Request) -> web.Response:
if request.query or request.can_read_body:
raise web.HTTPBadRequest()
roots = list(GalleryConfig.MONITORING_DIRECTORIES)
saved.setdefault("roots", roots)
GalleryConfig.MONITORING_DIRECTORIES = roots[::-1]
await api._run_in_executor(api.invalidate_gallery_cache, True)
return web.json_response({"success": True})
@routes.post("/__test__/scope-key/rotate")
async def rotate(request: web.Request) -> web.Response:
if request.query or request.can_read_body:
raise web.HTTPBadRequest()
saved.setdefault("keys", api.__dict__.get(_CACHE))
api.__dict__[_CACHE] = WorkspaceKeys(
secrets.token_bytes(32), secrets.token_bytes(32), False
)
return web.json_response({"success": True})
return restore
+5 -2
View File
@@ -231,6 +231,8 @@ test('paired Tag dialog captures', async ({ page, request }, testInfo) => {
await expect(await request.post('/__test__/tagging/suggestions', { data: { scored: ALPINE } })).toBeOK();
const save = () => tagDialog(page).locator('.dialog-actions .ca-primary').click();
const status = () => tagDialog(page).locator('#tag-generation-status');
// D147: a state that blocks Save shows its reason beside the actions, not in the status line.
const notice = () => tagDialog(page).locator('.dialog-actions > .tag-action-notice');
// Unavailable model.
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: ['wd14-swinv2'] } })).toBeOK();
@@ -269,7 +271,8 @@ test('paired Tag dialog captures', async ({ page, request }, testInfo) => {
await saveImageTags(request, path, CONCEPT_TAGS);
await openTagDialog(page, 'production', 'dark');
await save();
await expect(status()).not.toHaveText('');
await expect(notice()).toHaveText('The file changed since it was previewed. Reopen the Tag dialog.');
await expect(status()).toHaveText('');
await shot('production-file-changed');
await page.unroute(writes);
await page.keyboard.press('Escape');
@@ -278,7 +281,7 @@ test('paired Tag dialog captures', async ({ page, request }, testInfo) => {
await page.route(writes, route => route.fulfill({ status: 504, body: 'Gateway Timeout' }));
await openTagDialog(page, 'production', 'dark');
await save();
await expect(status()).toHaveText('The save could not be confirmed. Retry sends the same tags again.');
await expect(notice()).toHaveText('The save could not be confirmed. Retry sends the same tags again.');
expect((await inventory(page)).actions.buttons.map(button => [button.text, button.primary, button.disabled])).toEqual(
[['Retry', false, false], ['Cancel', false, false], ['Save manual tags', true, true]]);
await shot('production-unknown-outcome');
@@ -1,6 +1,7 @@
// Concept-vs-production inventory for the native Tags & categories screen (concept renderTags, app.js:719-729).
// pageInventoryOf runs in the page; dialogs reuse the editor's inventoryOf. The concept is never rewritten:
// mapVocabularyConcept / mapVocabularyDialog apply only the recorded deviations, each named with its ruling.
import {diffInventory} from './library-editor-canonical-comparison.mjs';
export {inventoryOf, diffInventory} from './library-editor-canonical-comparison.mjs';
export function pageInventoryOf(main) {
@@ -61,23 +62,51 @@ export function pageInventoryOf(main) {
};
}
const countOf=(count,noun)=>`${count} ${noun}${count===1?'':'s'}`;
/**
* D143 (#332): a category row counts the images whose own category it is: "N positive prompts · M images" when
* prompts hold it too, "M images" for a value only images hold. Image-only rows follow every prompt-backed row, before
* the closing Manage category row, each drawn as the concept's last category row with its own text (vocabulary-visual
* checks this against the concept page with those rows added at runtime). `images` is `{value: count}`; `only` lists
* the image-only values in their order.
*/
export function withImageCounts(operations,{images={},only=[]}={}) {
const rows=operations.map(row=>images[row.h3.text]&&!only.includes(row.h3.text)
?{...row,p:{...row.p,text:`${row.p.text} · ${countOf(images[row.h3.text],'image')}`}}:row);
const manage=rows.at(-1),last=rows.at(-2);
const added=only.map(value=>({...last,h3:{...last.h3,text:value},p:{...last.p,text:countOf(images[value],'image')}}));
return [...rows.slice(0,-1),...added,manage];
}
// Recorded deviations only (plan "Mock inventory" + decisions D1-D15 + ledger rulings). Everything else must be equal.
export function mapVocabularyConcept(reference) {
// `imageCategories` (#332 D143) is the fixture's own image categories, none by default.
export function mapVocabularyConcept(reference,imageCategories={}) {
const operations=reference.operations.map(row=>({...row,p:{...row.p,
// Ledger ruling: "1 positive prompts" -> singular, consistent with the shared plural helper (#238/#243).
text:row.p.text.replace(/^1 positive prompts$/,'1 positive prompt')}}));
return {...reference,
table:{...reference.table,rows:reference.table.rows.map(row=>({...row,
// Data, not a deviation: the concept tags each sample image with its prompt's tags; the vocabulary fixture
// tags no image, so every Images count is 0 (#249 C2 retired #245 D3).
images:'0'}))},
operations:reference.operations.map(row=>({...row,p:{...row.p,
// Ledger ruling: "1 positive prompts" -> singular, consistent with the shared plural helper (#238/#243).
text:row.p.text.replace(/^1 positive prompts$/,'1 positive prompt')}})),
operations:withImageCounts(operations,imageCategories),
};
}
/** The page comparison: concept rows are the concept's sample tags; production appends fixture-only tags (D4). */
export function compareVocabularyPage(reference,actual,imageCategories={}) {
const expected=mapVocabularyConcept(reference,imageCategories);
const shared=expected.table.rows.length;
return {diff:diffInventory(expected,{...actual,table:{...actual.table,rows:actual.table.rows.slice(0,shared)}}),
extra:actual.table.rows.slice(shared).map(row=>row.tag)};
}
// Concept dialogs are simpleInput (app.js:1138-1143). donors are concept inventories for controls the concept
// shows elsewhere: a `.field select` and a `p.small` (Filters dialog, app.js:1351) and a danger dialog action
// (Archive… in the editor, app.js:976), so added controls are held to concept tokens, never to production values.
export function mapVocabularyDialog(mode,reference,{filters,editor,categories}) {
// `preview` is the Manage categories sentence once Preview has been pressed (D9; D144, #332, adds its images).
export function mapVocabularyDialog(mode,reference,{filters,editor,categories,preview=null}) {
const fields=reference.fields.map(item=>({...item,
// Inventory rows 27-29: the name is `required` (blank Apply is blocked natively).
required:true}));
@@ -98,9 +127,9 @@ export function mapVocabularyDialog(mode,reference,{filters,editor,categories})
// D9 / Task 8: the proposed name offers a category datalist (no trailing commas: one value).
...fields.map(item=>({...item,listed:true,commaOptions:false}))],
// D9: the preview sentence is a `p.small`, empty and hidden until Preview.
small:[{...filters.small[0],text:''}],
small:[{...filters.small[0],text:preview??''}],
// D9: the primary is `Preview` until a preview is shown, then `Apply`.
actions:{...reference.actions,buttons:buttons.map(item=>item.text==='Apply'?{...item,text:'Preview'}:item)}};
actions:{...reference.actions,buttons:preview===null?buttons.map(item=>item.text==='Apply'?{...item,text:'Preview'}:item):buttons}};
}
return {...reference,fields,actions:{...reference.actions,buttons}};
}
+88 -1
View File
@@ -1,11 +1,14 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {pageInventoryOf,inventoryOf,diffInventory,mapVocabularyConcept,mapVocabularyDialog,compareVocabularyPage} from './vocabulary-canonical-comparison.mjs';
// Artifact-only captures of Tags & categories states that have no concept counterpart: the Manage categories
// preview, loading and 1,000-category cap notes, tag conflict and restore notes, the Delete tag confirmation (D14),
// empty lists (D12), a failed read with Retry, and Load more (D5). The page and the three concept dialogs are captured
// as concept/production pairs by workspace-vocabulary.spec.mjs. There are deliberately no pixel assertions or
// baselines: the maintainer accepts these captures visually before any baseline is added.
// baselines: the maintainer accepts these captures visually before any baseline is added. The last test (#332 D143,
// D144) compares the category rows with image counts and the Manage category preview against the concept.
test.use({locale:'en-US',timezoneId:'UTC',reducedMotion:'reduce'});
const route='/prompt_manager/workspace#/tags';
const main=page=>page.locator('#workspace-main');
@@ -101,3 +104,87 @@ for(const [width,height] of [[1600,1100],[390,844]])for(const theme of ['dark','
await testInfo.attach(`production-load-more-${width}-${theme}`,{path,contentType:'image/png'});
await expect(rows(page)).toHaveCount(100);
});
// D143 and D144 (#332): with own image categories seeded (`image_categories`: "Landscapes" on two images, "Portraits"
// on one image only), the Categories panel and the Manage category preview against the concept (renderTags, app.js
// 719-729; the Manage dialog is its simpleInput, 1138-1143). The page is compared with D143 mapped, a model checked
// against the concept page with those rows added at runtime; the preview with D9's sentence and D144's images.
const concept='/__test__/canonical/workspace';
const IMAGE_CATEGORIES=Object.freeze({images:{Landscapes:2,Portraits:1},only:['Portraits']});
// D9's sentence; D144 adds the images holding the value ("and 2 images").
const PREVIEW='“Landscapes” is used by 3 positive prompts and 2 images. Apply renames it to “Scenery”.';
const pageOf=(page,production)=>page.locator(production?'#workspace-main':'#main').evaluate(pageInventoryOf);
async function openConceptRoute(page,route,theme) {
await page.goto('about:blank');await page.goto(`${concept}#${route}`);
await page.getByRole('combobox',{name:'Theme',exact:true}).selectOption(theme);
}
/** The concept page as D143 records it: image counts after the prompt counts, image-only rows before Manage category. */
function conceptWithImageCounts(page) {
return page.evaluate(({images,only})=>{
const surface=document.querySelectorAll('#main .two-col > section.surface')[1];
const rows=[...surface.querySelectorAll('.operation')];
const count=value=>`${images[value]} image${images[value]===1?'':'s'}`;
for(const row of rows) {
const value=row.querySelector('h3').textContent;
if(images[value]&&!only.includes(value))row.querySelector('p').textContent+=` · ${count(value)}`;
}
const manage=rows.at(-1),last=rows.at(-2);
for(const value of only) {
const row=last.cloneNode(true);row.querySelector('h3').textContent=value;row.querySelector('p').textContent=count(value);manage.before(row);
}
},IMAGE_CATEGORIES);
}
/** Concept controls the Manage dialog adds (as in workspace-vocabulary.spec.mjs): the Filters field select and p.small. */
async function conceptDonors(page,theme,categories) {
await openConceptRoute(page,'images',theme);
await page.locator('#main').getByRole('button',{name:'Filters',exact:true}).click();await expect(page.locator('#modal')).toBeVisible();
const filters=await page.locator('#modal').evaluate(inventoryOf);
await openConceptRoute(page,'prompt/alpine',theme);
await page.getByRole('button',{name:'Edit metadata',exact:true}).click();await expect(page.locator('#modal')).toBeVisible();
const editor=await page.locator('#modal').evaluate(inventoryOf);
return {filters,editor,categories};
}
for(const [width,height] of [[1600,1100],[390,844]])for(const theme of ['dark','light'])test(`category rows and the Manage category preview match the concept with image categories ${width}x${height} ${theme} (D143, D144)`,async({page,request},testInfo)=>{
await page.setViewportSize({width,height});
const manifest=await seed(request,{image_categories:true});
expect(manifest.image_categories).toEqual(IMAGE_CATEGORIES.images);
await openConceptRoute(page,'tags',theme);await expect(page.locator('#main h1')).toHaveText('Tags & categories');
await page.mouse.move(0,0);
const reference=await pageOf(page,false);
await conceptWithImageCounts(page);
const oracle=await pageOf(page,false);
await captureCanonicalState(page,testInfo,`concept-image-categories-${width}-${theme}`,'#main');
const donors=await conceptDonors(page,theme,['Landscapes','Architecture','Environments','Illustration','Portraits']);
await openConceptRoute(page,'tags',theme);
await page.locator('#main').getByRole('button',{name:'Review',exact:true}).click();await expect(page.locator('#modal')).toBeVisible();
const referenceDialog=await page.locator('#modal').evaluate(inventoryOf);
await open(page,theme,manifest.tags.length);
await expect(categoryOperation(page,'Portraits')).toHaveCount(1);
// The pointer still rests where the concept's Review was pressed: park it so no row is hovered.
await page.mouse.move(0,0);
const actual=await pageOf(page,true);
await captureCanonicalState(page,testInfo,`production-image-categories-${width}-${theme}`,'#workspace-main');
const box=await openDialog(page,categoryOperation(page,'Manage category').getByRole('button',{name:'Review',exact:true}));
await expect(box.getByLabel('Category',{exact:true})).toHaveValue('Landscapes');
await box.getByLabel('Proposed new category name',{exact:true}).fill('Scenery');await button(box,'Preview').click();
await expect(box.locator('p.small')).toHaveText(PREVIEW);
// The pressed primary keeps the pointer and focus; the concept is measured as it opens (focus on ×, no hover),
// which workspace-vocabulary.spec.mjs compares, so this state is restored before measuring the preview.
await page.mouse.move(0,0);await box.locator('.dialog-header button').focus();
const actualDialog=await box.evaluate(inventoryOf);
await captureCanonicalState(page,testInfo,`production-image-categories-preview-${width}-${theme}`);
const path=testInfo.outputPath('vocabulary-image-categories.json');
await writeFile(path,JSON.stringify({reference,oracle,actual,referenceDialog,actualDialog},null,2));
await testInfo.attach('vocabulary-image-categories',{path,contentType:'application/json'});
// The D143 model reproduces the concept page with the rows added at runtime.
expect.soft(diffInventory(mapVocabularyConcept(reference,IMAGE_CATEGORIES).operations,mapVocabularyConcept(oracle).operations),'D143 model').toEqual([]);
const {diff,extra}=compareVocabularyPage(reference,actual,IMAGE_CATEGORIES);
expect.soft(actual.operations.map(row=>[row.h3.text,row.p.text]).slice(0,-1),'category rows').toEqual([['Landscapes','3 positive prompts · 2 images'],
['Architecture','1 positive prompt'],['Environments','1 positive prompt'],['Illustration','1 positive prompt'],['Portraits','1 image']]);
expect.soft(diff,'page differences after the recorded deviations and D143').toEqual([]);
expect.soft(extra,'fixture-only rows').toEqual([manifest.pair_only_tag,manifest.archived_only_tag,manifest.negative_only_tag,manifest.unused_tag]);
expect.soft(diffInventory(mapVocabularyDialog('manage',referenceDialog,{...donors,preview:PREVIEW}),actualDialog),'preview differences after D9 and D144').toEqual([]);
});
@@ -0,0 +1,493 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from './fixtures.mjs';
// Collections (#331): the sidebar COLLECTIONS group (D136, concept index.html 28–31, app.js 336–339), the collection
// page `#/collection/<id>` with its Edit (D137, concept render a1186–1189) and saved views on a collection (D140).
const workspace = '/prompt_manager/workspace';
const API = '/prompt_manager/collections';
const main = page => page.locator('#workspace-main');
const heading = page => main(page).getByRole('heading', { level: 1 });
const breadcrumb = page => page.locator('#workspace-breadcrumb');
const description = page => main(page).locator('.heading p.muted');
const missingNote = page => main(page).locator('.heading p.small');
const search = page => page.getByRole('searchbox', { name: 'Search workspace', exact: true });
const imageCards = page => page.locator('article[data-image-key]');
const collectionsNav = page => page.getByRole('navigation', { name: 'Collections', exact: true });
const collectionsLabel = page => page.locator('aside.sidebar .sidebar-label', { hasText: 'COLLECTIONS' });
const rows = page => collectionsNav(page).locator('a[data-route^="collection/"]');
const row = (page, name) => rows(page).filter({ hasText: name });
const savedNav = page => page.getByRole('navigation', { name: 'Saved views', exact: true });
const editButton = page => main(page).locator('.heading-actions > button').first();
const editDialog = page => page.getByRole('dialog', { name: 'Edit collection', exact: true });
const deleteDialog = page => page.getByRole('dialog', { name: 'Delete collection', exact: true });
const nameField = page => editDialog(page).getByRole('textbox', { name: 'Collection name', exact: true });
const toast = page => page.locator('#workspace-toast');
async function seed(request) {
const response = await request.post('/__test__/collections/seed');
await expect(response).toBeOK();
return response.json();
}
async function listed(request) {
const response = await request.get(API);
await expect(response).toBeOK();
return (await response.json()).collections;
}
test.beforeEach(async ({ page, request, browserName }) => {
await startCoverage(page, browserName);
await expect(await request.post('/__test__/reset')).toBeOK();
page.collectionErrors = [];
page.on('pageerror', error => page.collectionErrors.push(error.message));
// D51: deleting asks inside the dialog; no browser dialog (window.confirm) may ever open.
page.on('dialog', dialog => { page.collectionErrors.push(`browser ${dialog.type()}`); dialog.dismiss().catch(() => {}); });
});
test.afterEach(async ({ page, browserName }) => {
await saveCoverage(page, browserName);
expect(page.collectionErrors).toEqual([]);
});
test('a collection row opens its page', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Landscapes;
await page.setViewportSize({ width: 1600, height: 1100 });
await page.goto(`${workspace}#/images`);
// D136: name order, the listed-member count, no "PROPOSED" badge.
await expect(collectionsLabel(page)).toBeVisible();
await expect(collectionsLabel(page)).toHaveText('COLLECTIONS');
await expect(rows(page)).toHaveCount(2);
await expect(rows(page).nth(0)).toContainText('Experiments');
await expect(rows(page).nth(1)).toContainText('Landscapes');
await expect(row(page, 'Experiments').locator('.nav-count')).toHaveText('8');
await expect(row(page, 'Landscapes').locator('.nav-count')).toHaveText('12');
await expect(row(page, 'Landscapes')).toHaveAttribute('href', `#/collection/${id}`);
await expect(row(page, 'Landscapes').locator('svg')).toHaveCount(1);
await expect(page.locator('aside.sidebar .new-label')).toHaveCount(0);
await expect(imageCards(page)).toHaveCount(21);
await row(page, 'Landscapes').click();
await expect(page).toHaveURL(new RegExp(`#/collection/${id}$`));
await expect(heading(page)).toHaveText('Landscapes');
await expect(breadcrumb(page)).toHaveText('Collection');
await expect(page).toHaveTitle('Collection · Prompt Manager');
await expect(row(page, 'Landscapes')).toHaveAttribute('aria-current', 'page');
await expect(description(page)).toHaveText('Your generations, their prompts, and the next idea.');
await expect(imageCards(page)).toHaveCount(12);
for (const path of manifest.files.Landscapes) {
await expect(page.locator(`article[data-image-key$=":${path}"]`)).toHaveCount(1);
}
// MC3: the member whose file is gone is not listed and is counted in the note.
await expect(missingNote(page)).toHaveText('1 image in this collection is not in your current sources.');
// Every Images filter narrows the members.
await search(page).fill('landscape_03');
await expect(imageCards(page)).toHaveCount(1);
await search(page).fill('');
await expect(imageCards(page)).toHaveCount(12);
await row(page, 'Experiments').click();
await expect(heading(page)).toHaveText('Experiments');
await expect(imageCards(page)).toHaveCount(8);
await expect(missingNote(page)).toHaveCount(0);
});
test('without collections the group shows no label and no rows', async ({ page }) => {
await page.goto(`${workspace}#/images`);
await expect(imageCards(page).first()).toBeVisible();
await expect(collectionsNav(page)).toHaveCount(1);
await expect(rows(page)).toHaveCount(0);
await expect(collectionsLabel(page)).toBeHidden();
});
test('a collection page neither restores nor replaces the Images return state', async ({ page, request }) => {
const manifest = await seed(request);
await page.goto(`${workspace}#/images`);
await search(page).fill('experiment');
await expect(imageCards(page)).toHaveCount(8);
await row(page, 'Landscapes').click();
await expect(search(page)).toHaveValue('');
await expect(imageCards(page)).toHaveCount(12);
await search(page).fill('landscape_01');
await expect(imageCards(page)).toHaveCount(1);
await page.goto(`${workspace}#/images`);
await expect(search(page)).toHaveValue('experiment');
expect(manifest.collections.Landscapes.count).toBe(12);
});
test('Edit renames the open collection; a taken name stays in the dialog', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Landscapes;
await page.goto(`${workspace}#/collection/${id}`);
await expect(heading(page)).toHaveText('Landscapes');
await expect(editButton(page)).toHaveText('Edit');
await editButton(page).click();
await expect(nameField(page)).toHaveValue('Landscapes');
await expect(editDialog(page).locator('.dialog-actions button')).toHaveText(['Delete…', 'Cancel', 'Save']);
// CUJ 3: names are unique ignoring case.
await nameField(page).fill('experiments');
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
await expect(editDialog(page).getByRole('alert')).toHaveText('A collection named “experiments” already exists.');
await expect(heading(page)).toHaveText('Landscapes');
await nameField(page).fill(' Landscape grids ');
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
await expect(editDialog(page)).toHaveCount(0);
await expect(toast(page)).toHaveText('Collection renamed.');
await expect(heading(page)).toHaveText('Landscape grids');
await expect(breadcrumb(page)).toHaveText('Collection');
await expect(row(page, 'Landscape grids')).toHaveAttribute('aria-current', 'page');
await expect(rows(page).nth(1)).toContainText('Landscape grids');
expect((await listed(request)).map(item => item.name)).toEqual(['Experiments', 'Landscape grids']);
});
test('a failed rename keeps the dialog; a failed delete changes nothing', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Experiments;
await page.goto(`${workspace}#/collection/${id}`);
await expect(heading(page)).toHaveText('Experiments');
let failing = true;
await page.route(`**${API}/${id}`, route => failing
? route.fulfill({ status: 500, contentType: 'application/json', body: '{"success":false,"error":"The collection could not be changed."}' })
: route.fallback());
await editButton(page).click();
await nameField(page).fill('Tests');
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
await expect(toast(page)).toHaveText('The collection could not be renamed. Try again.');
await expect(editDialog(page)).toBeVisible();
await expect(heading(page)).toHaveText('Experiments');
await editDialog(page).getByRole('button', { name: 'Delete…', exact: true }).click();
await expect(deleteDialog(page).locator('p')).toHaveText(
'Delete “Experiments”? Its 8 images stay where they are; only the collection is removed.');
await deleteDialog(page).getByRole('button', { name: 'Delete', exact: true }).click();
await expect(toast(page)).toHaveText('The collection could not be deleted. Try again.');
await expect(page).toHaveURL(new RegExp(`#/collection/${id}$`));
expect(await listed(request)).toHaveLength(2);
failing = false;
});
test('Delete removes the collection only after confirming, and leaves every file', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Landscapes;
await page.goto(`${workspace}#/collection/${id}`);
await editButton(page).click();
await editDialog(page).getByRole('button', { name: 'Delete…', exact: true }).click();
// The stored count: 12 listed members and the missing one.
await expect(deleteDialog(page).locator('p')).toHaveText(
'Delete “Landscapes”? Its 13 images stay where they are; only the collection is removed.');
await deleteDialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
await expect(deleteDialog(page)).toHaveCount(0);
expect(await listed(request)).toHaveLength(2);
await editButton(page).click();
await editDialog(page).getByRole('button', { name: 'Delete…', exact: true }).click();
await deleteDialog(page).getByRole('button', { name: 'Delete', exact: true }).click();
await expect(toast(page)).toHaveText('Collection deleted.');
await expect(page).toHaveURL(/#\/images$/);
await expect(heading(page)).toHaveText('Images');
await expect(rows(page)).toHaveCount(1);
await expect(imageCards(page)).toHaveCount(21);
expect((await listed(request)).map(item => item.name)).toEqual(['Experiments']);
});
test('an unknown collection says it no longer exists, with the way back to Images', async ({ page, request }) => {
await seed(request);
await page.goto(`${workspace}#/collection/999999`);
await expect(heading(page)).toHaveText('Collection');
await expect(breadcrumb(page)).toHaveText('Collection');
await expect(description(page)).toHaveText('This collection no longer exists.');
await expect(imageCards(page)).toHaveCount(0);
await main(page).getByRole('link', { name: 'Open Images', exact: true }).click();
await expect(page).toHaveURL(/#\/images$/);
await expect(imageCards(page)).toHaveCount(21);
});
test('a failed list says so and Retry reads it again', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Experiments;
let failing = true;
await page.route(`**${API}`, route => failing
? route.fulfill({ status: 500, contentType: 'application/json', body: '{"success":false,"error":"Collections could not be read."}' })
: route.fallback());
await page.goto(`${workspace}#/collection/${id}`);
await expect(description(page)).toHaveText('Collections could not be loaded. Your saved data has not changed.');
await expect(description(page)).toHaveAttribute('role', 'alert');
await expect(rows(page)).toHaveCount(0);
failing = false;
await main(page).getByRole('button', { name: 'Retry', exact: true }).click();
await expect(heading(page)).toHaveText('Experiments');
await expect(imageCards(page)).toHaveCount(8);
await expect(rows(page)).toHaveCount(2);
});
test('a saved view keeps its collection, and says so once the collection is deleted (MC6)', async ({ page, request }) => {
const manifest = await seed(request);
const view = manifest.saved_view;
await page.goto(`${workspace}#/images`);
await savedNav(page).locator(`a[data-route="saved/${view.id}"]`).click();
await expect(heading(page)).toHaveText(view.name);
await expect(breadcrumb(page)).toHaveText(view.name);
await expect(imageCards(page)).toHaveCount(8);
// A saved view is not the collection page: no missing note.
await expect(missingNote(page)).toHaveCount(0);
// Save view on a collection page stores the collection.
await row(page, 'Landscapes').click();
await expect(imageCards(page)).toHaveCount(12);
await search(page).fill('landscape_1');
await expect(imageCards(page)).toHaveCount(3);
await main(page).getByRole('button', { name: 'Save view', exact: true }).click();
const save = page.getByRole('dialog', { name: 'Save this view', exact: true });
await save.getByRole('textbox', { name: 'View name', exact: true }).fill('Grid seed');
await save.getByRole('button', { name: 'Save', exact: true }).click();
await expect(toast(page)).toHaveText('Saved view added to the sidebar.');
const views = (await (await request.get('/prompt_manager/saved-views')).json()).views;
const saved = views.find(item => item.name === 'Grid seed');
expect(saved.state).toMatchObject({ q: 'landscape_1', favorites: false, collection: String(manifest.collections.Landscapes.id) });
await expect(await request.delete(`${API}/${manifest.collections.Experiments.id}`)).toBeOK();
await page.goto(`${workspace}#/saved/${view.id}`);
await expect(heading(page)).toHaveText(view.name);
await expect(main(page).locator('.empty')).toHaveText('The collection this view uses was deleted.');
await expect(main(page).locator('.empty')).toBeVisible();
await expect(imageCards(page)).toHaveCount(0);
await expect(main(page).getByRole('button', { name: 'Retry', exact: true })).toBeHidden();
});
test('a collection deleted in another tab shows the deleted-collection page', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Experiments;
await page.goto(`${workspace}#/images`);
await expect(rows(page)).toHaveCount(2);
await expect(await request.delete(`${API}/${id}`)).toBeOK();
await row(page, 'Experiments').click();
await expect(description(page)).toHaveText('This collection no longer exists.');
await expect(rows(page)).toHaveCount(1);
});
// D136 / CUJ 15: the rows show and open at desktop and phone width in both themes. The concept's own phone rule hides
// every nav count there, so the rows keep their counts only at desktop width.
for (const [width, height] of [[1600, 1100], [390, 844]]) for (const theme of ['dark', 'light']) {
test(`collection rows show and open at ${width}x${height} ${theme}`, async ({ page, request }) => {
const manifest = await seed(request);
await page.setViewportSize({ width, height });
await page.goto(`${workspace}#/images`);
await page.getByRole('combobox', { name: 'Theme', exact: true }).selectOption(theme);
for (const name of ['Experiments', 'Landscapes']) await expect(row(page, name)).toBeVisible();
const count = row(page, 'Landscapes').locator('.nav-count');
if (width > 950) await expect(count).toBeVisible();
else await expect(count).toBeHidden();
// Phone width: the label stays hidden as every sidebar label is; desktop shows it.
if (width > 950) await expect(collectionsLabel(page)).toBeVisible();
else await expect(collectionsLabel(page)).toBeHidden();
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
await row(page, 'Landscapes').click();
await expect(page).toHaveURL(new RegExp(`#/collection/${manifest.collections.Landscapes.id}$`));
await expect(heading(page)).toHaveText('Landscapes');
await expect(imageCards(page)).toHaveCount(12);
});
}
test('a finished restore reads the collections again', async ({ page, request }) => {
await seed(request);
await expect(await request.post('/__test__/system/reset')).toBeOK();
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
try {
const backup = await request.get('/prompt_manager/backup');
await expect(backup).toBeOK();
const bytes = await backup.body();
const [experiments] = await listed(request);
await expect(await request.delete(`${API}/${experiments.id}`)).toBeOK();
await page.goto(`${workspace}#/system/backups`);
await expect(rows(page)).toHaveCount(1);
const chooser = page.waitForEvent('filechooser');
await page.getByRole('button', { name: 'Review restore…', exact: true }).click();
await (await chooser).setFiles({ name: 'prompts_backup_20261027_093000.db', mimeType: 'application/octet-stream', buffer: bytes });
await page.locator('dialog#modal[open]').getByRole('button', { name: 'Restore', exact: true }).click();
await expect(toast(page)).toHaveText(/^Database restore finished · /);
// The list was read again without a reload.
await expect(rows(page)).toHaveCount(2);
await expect(row(page, 'Experiments').locator('.nav-count')).toHaveText('8');
} finally {
await request.post('/__test__/jobs/restart');
await request.post('/__test__/system/reset');
}
});
test('renaming a collection deleted in another tab says so and shows the deleted-collection page', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Experiments;
await page.goto(`${workspace}#/collection/${id}`);
await expect(heading(page)).toHaveText('Experiments');
await editButton(page).click();
await expect(await request.delete(`${API}/${id}`)).toBeOK();
await nameField(page).fill('Tests');
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
await expect(toast(page)).toHaveText('That collection no longer exists.');
await expect(editDialog(page)).toHaveCount(0);
await expect(description(page)).toHaveText('This collection no longer exists.');
await expect(rows(page)).toHaveCount(1);
});
/** A Playwright hold on the collections list read: `held` fills as reads arrive; `release()` lets them all through. */
async function holdListReads(page, answer = route => route.fallback()) {
const held = [];
let open = false;
await page.route(`**${API}`, route => {
if (route.request().method() !== 'GET') return route.fallback();
if (open) return answer(route);
held.push(route);
return undefined;
});
return {
held,
async release() {
open = true;
for (const route of held.splice(0)) await answer(route).catch(() => {});
},
};
}
test('a list read superseded by another read still opens a collection made since (review M1)', async ({ page, request }) => {
const manifest = await seed(request);
const listing = await (await request.get('/prompt_manager/images/output?mode=workspace&limit=1')).json();
const [file] = listing.images;
await page.goto(`${workspace}#/collection/${manifest.collections.Landscapes.id}`);
await expect(heading(page)).toHaveText('Landscapes');
// Another tab makes "Fresh"; this tab does not list it yet.
const added = await request.post(`${API}/items`, { data: { name: 'Fresh',
items: [{ root_index: file.root_index, root_scope: file.root_scope, path: file.path }] } });
await expect(added).toBeOK();
const fresh = (await added.json()).collection.id;
await expect(rows(page)).toHaveCount(2);
const hold = await holdListReads(page);
// A rename schedules the 300 ms count re-read; opening "Fresh" first reads the list once more. The timer's read
// supersedes that one while both are held.
await editButton(page).click();
await nameField(page).fill('Landscape grids');
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
await expect(toast(page)).toHaveText('Collection renamed.');
await page.evaluate(hash => { window.location.hash = hash; }, `#/collection/${fresh}`);
await expect.poll(() => hold.held.length).toBeGreaterThanOrEqual(2);
await expect(main(page)).not.toContainText('Collections could not be loaded.');
await hold.release();
await expect(heading(page)).toHaveText('Fresh');
await expect(imageCards(page)).toHaveCount(1);
});
test('a listing that finds the collection gone while the list still has it offers Retry (review M2)', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Experiments;
let gone = true;
await page.route('**/prompt_manager/images/output?*', route => gone && route.request().url().includes(`collection=${id}`)
? route.fulfill({ status: 404, contentType: 'application/json',
body: '{"success":false,"error":"Collection not found","code":"collection-not-found"}' })
: route.fallback());
await page.goto(`${workspace}#/images`);
await expect(rows(page)).toHaveCount(2);
// The shell's re-read is held, then fails: the page must not stay blank.
const hold = await holdListReads(page, route => route.fulfill({ status: 500, contentType: 'application/json',
body: '{"success":false,"error":"Collections could not be read."}' }));
await row(page, 'Experiments').click();
await expect.poll(() => hold.held.length).toBe(1);
await hold.release();
const retry = main(page).getByRole('button', { name: 'Retry', exact: true });
await expect(retry).toBeVisible();
await expect(main(page).locator('#gallery-continuation p')).toHaveText('Images could not be loaded. Retry or rescan the collection.');
gone = false;
await page.unroute(`**${API}`);
await retry.click();
await expect(imageCards(page)).toHaveCount(8);
// The list read again still lists it: the same Retry, never a blank page.
gone = true;
await page.evaluate(() => { window.location.hash = '#/images'; });
await expect(heading(page)).toHaveText('Images');
await row(page, 'Experiments').click();
await expect(retry).toBeVisible();
gone = false;
await retry.click();
await expect(imageCards(page)).toHaveCount(8);
});
test('the missing-members note follows the list read again (review L2)', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Landscapes;
let missing = null;
await page.route(`**${API}`, async route => {
if (route.request().method() !== 'GET' || missing === null) return route.fallback();
const response = await route.fetch();
const data = await response.json();
data.collections = data.collections.map(item => item.id === id ? { ...item, missing } : item);
return route.fulfill({ response, json: data });
});
await page.goto(`${workspace}#/collection/${id}`);
await expect(missingNote(page)).toHaveText('1 image in this collection is not in your current sources.');
const rename = async name => {
await editButton(page).click();
await nameField(page).fill(name);
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
await expect(toast(page)).toHaveText('Collection renamed.');
};
missing = 3;
await rename('Landscapes A');
await expect(missingNote(page)).toHaveText('3 images in this collection are not in your current sources.');
missing = 0;
await rename('Landscapes B');
await expect(missingNote(page)).toHaveCount(0);
missing = 1;
await rename('Landscapes C');
await expect(missingNote(page)).toHaveText('1 image in this collection is not in your current sources.');
});
test('a rename that settles after its dialog closed leaves the next dialog open (review L1)', async ({ page, request }) => {
const manifest = await seed(request);
const { id } = manifest.collections.Landscapes;
await page.goto(`${workspace}#/collection/${id}`);
await expect(imageCards(page)).toHaveCount(12);
let release;
const released = new Promise(resolve => { release = resolve; });
await page.route(`**${API}/${id}`, async route => {
if (route.request().method() !== 'PATCH') return route.fallback();
await released;
return route.fallback();
});
await editButton(page).click();
await nameField(page).fill('Later');
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
await page.keyboard.press('Escape');
await expect(editDialog(page)).toHaveCount(0);
await main(page).getByRole('button', { name: 'Save view', exact: true }).click();
const save = page.getByRole('dialog', { name: 'Save this view', exact: true });
await expect(save).toBeVisible();
release();
await expect(heading(page)).toHaveText('Later');
await expect(save).toBeVisible();
});
test('a page that says a collection or view could not be opened takes focus after a route change (review L3)', async ({ page, request }) => {
await seed(request);
await page.goto(`${workspace}#/images`);
await expect(heading(page)).toHaveText('Images');
await page.evaluate(() => { window.location.hash = '#/collection/999999'; });
await expect(description(page)).toHaveText('This collection no longer exists.');
await expect(heading(page)).toBeFocused();
await page.route(`**${API}`, route => route.request().method() === 'GET'
? route.fulfill({ status: 500, contentType: 'application/json', body: '{"success":false,"error":"x"}' })
: route.fallback());
await page.evaluate(() => { window.location.hash = '#/images'; });
await expect(heading(page)).toHaveText('Images');
await page.evaluate(() => { window.location.hash = '#/collection/999998'; });
await expect(main(page).getByRole('button', { name: 'Retry', exact: true })).toBeVisible();
await expect(heading(page)).toBeFocused();
await page.route('**/prompt_manager/saved-views', route => route.request().method() === 'GET'
? route.fulfill({ status: 500, contentType: 'application/json', body: '{"success":false,"error":"x"}' })
: route.fallback());
await page.evaluate(() => { window.location.hash = '#/saved/5'; });
await expect(description(page)).toHaveText('Saved views could not be loaded. Your saved data has not changed.');
await expect(heading(page)).toBeFocused();
});
+24
View File
@@ -495,6 +495,30 @@ test('the inline current file names its prompts apart from the frozen run fields
expect(await exact(dialog(page).getByRole('region',{name:'Positive prompt',exact:true}).locator('p.prompt-text'))).toBe(before);
});
// #332 CUJ 5 (MC8, §5.3): History's current-file inspector reads the file's own category with the Images key rule, so
// the inline current file and its viewer show "Portraits · this image"; current.png has two saved links, so it inherits
// nothing and shows no chip before.
test('the current file shows its own category with its origin in the recorded run and its viewer',async({page,request})=>{
const chip=scope=>scope.locator('span.chip:has(> span.chip-source)');
const current=dialog(page).getByRole('region',{name:'Current reported file',exact:true});
const viewCurrent=async()=>{await dialog(page).locator('[data-receipt]').first().getByRole('button',{name:'View current file',exact:true}).click();
const details=page.locator('dialog#viewer-dialog #viewer-details');await expect(details.locator('.panel-heading')).toBeVisible();return details;};
await open(page,page.fixture.detail_id);await marker(page,page.fixture.runs.shared).click();await expect(dialog(page).locator('[data-receipt]')).toHaveCount(5);
let details=await viewCurrent();await expect(details.locator('.panel-heading h2')).toBeVisible();await expect(chip(details)).toHaveCount(0);
await page.keyboard.press('Escape');await expect(current).toBeVisible();await expect(chip(current)).toHaveCount(0);
await dialog(page).getByRole('button',{name:'Close recorded run',exact:true}).click();
const {images}=await(await request.get('/prompt_manager/images/output?mode=workspace&subfolder=history&limit=40')).json();
const file=images.find(image=>image.root_index===0&&image.path==='history/current.png');
const put=await request.put('/prompt_manager/images/category',{data:{items:[{root_index:0,root_scope:file.root_scope,path:file.path}],category:'Portraits'}});
await expect(put).toBeOK();expect((await put.json()).updated).toBe(1);
await open(page,page.fixture.detail_id);await marker(page,page.fixture.runs.shared).click();await expect(dialog(page).locator('[data-receipt]')).toHaveCount(5);
details=await viewCurrent();
await expect(chip(details)).toHaveText('Image category: Portraits · this image');
await page.keyboard.press('Escape');await expect(current).toBeVisible();
await expect(chip(current)).toHaveText('Image category: Portraits · this image');
expect(await chip(current).locator('.chip-source').evaluate(node=>node.textContent)).toBe(' · this image');
});
// LI-4 (D117): the recorded history's current-file viewer is the shared full inspector, so it copies the legacy summary.
test('the current file viewer copies its settings summary',async({page})=>{
await page.addInitScript(()=>{window.__copied=[];Object.defineProperty(navigator,'clipboard',{configurable:true,value:{writeText:async text=>{window.__copied.push(text);}}});});
+288 -72
View File
@@ -569,6 +569,10 @@ test('source and folder discovery failure stays visible and Import Rescan restor
async function staleSourceRecovery(page, request, roots) {
await expect(await request.post('/__test__/reset', { data: { extra_images: 80 } })).toBeOK();
await page.setViewportSize({ width: 1600, height: 1100 });
// #430: the page first asks the server where a stale file is now. These tests cover the manual Refresh, so the
// file is never found; workspace-stale-source.spec.mjs covers a renumbered file that is found again.
await page.route('**/prompt_manager/images/rescope', route => route.fulfill({ status: 409,
json: { success: false, code: 'source-changed', error: 'The source of this image changed or was removed.' } }));
await page.goto(workspace);
const source = refineDialog(page).getByLabel('Source', { exact: true });
const folder = refineDialog(page).getByLabel('Folder', { exact: true });
@@ -1988,70 +1992,286 @@ test('Export metadata downloads nothing when no image can be read', async ({ pag
await expect(bulkbar(page).locator('.bulkbar > strong')).toHaveText('2 selected');
});
// #249 D8 follow-up (image selection, Task 3): Set category sets the linked positives' category (app.js simpleInput).
// #332 (D140, MC6, MC7): Set category writes each selected image's own category (PUT /prompt_manager/images/category);
// "Use prompt category" removes it after a confirmation (DELETE). It never writes a prompt: Images no longer posts
// /prompt_manager/bulk/category (#259's prompt action is retired from Images).
const categoryButton = page => bulkbar(page).getByRole('button', { name: 'Set category', exact: true });
const categoryDialog = page => page.getByRole('dialog', { name: 'Categorize selected images', exact: true });
const categoryRoute = '**/prompt_manager/bulk/category';
/** Unlink the input root's card by deleting its prompt; returns the output card's positive id. */
async function unlinkInputCard(request) {
const { images } = await (await request.get('/prompt_manager/images/output?mode=workspace')).json();
const byKey = Object.fromEntries(images.map(image => [`${image.root_index}:${image.path}`, image]));
const [output, input] = [byKey['0:shared/same.png'], byKey['1:shared/same.png']];
expect([output.linked.status, input.linked.status]).toEqual(['linked', 'linked']);
await expect(await request.delete(`/prompt_manager/delete/${input.linked.positive_id}`)).toBeOK();
return output.linked.positive_id;
const removeDialog = page => page.getByRole('dialog', { name: 'Use prompt category', exact: true });
const categoryFilter = page => page.getByRole('combobox', { name: 'Filter by category', exact: true });
const categoryRoute = '**/prompt_manager/images/category';
const IMAGE_CATEGORY = '/prompt_manager/images/category';
const cardOf = (page, key) => page.locator(`article[data-image-key="${key}"]`);
const keyOf = image => `${image.root_index}:${image.path}`;
const cardKeys = page => cards(page).evaluateAll(nodes => nodes.map(node => node.dataset.imageKey).sort());
// The four outputs of "Cat portrait" (CUJ 1's batch); the input root's same.png is linked to "Dog landscape".
const BATCH = ['0:bulk/0000.png', '0:bulk/0001.png', '0:bulk/0002.png', '0:shared/same.png'];
/** Every category write the page sends, Images' own route and #259's prompt route alike. */
function categoryWrites(page) {
const writes = [];
page.on('request', item => {
const path = new URL(item.url()).pathname;
if (item.method() === 'GET' || !['/prompt_manager/images/category', '/prompt_manager/bulk/category'].includes(path)) return;
writes.push({ method: item.method(), path, body: item.postDataJSON() });
});
return writes;
}
async function listedImages(request, query = '') {
const response = await request.get(`/prompt_manager/images/output?mode=workspace&limit=40${query}`);
await expect(response).toBeOK();
return (await response.json()).images;
}
async function targetsOf(request, keys) {
const images = await listedImages(request);
return keys.map(key => {
const image = images.find(item => keyOf(item) === key);
return { root_index: image.root_index, root_scope: image.root_scope, path: image.path };
});
}
async function putOwnCategory(request, keys, category) {
const response = await request.put(IMAGE_CATEGORY, { data: { items: await targetsOf(request, keys), category } });
await expect(response).toBeOK();
}
const filteredKeys = async (request, category) => (await listedImages(request, `&category=${encodeURIComponent(category)}`)).map(keyOf).sort();
async function positiveOf(request, id) {
const response = await request.get(`/prompt_manager/library/${id}`);
await expect(response).toBeOK();
const { positive } = await response.json();
return { id: positive.id, text: positive.text, category: positive.category };
}
/** CUJ 1's Given: four images linked to the positive "Cat portrait", whose category is "Landscapes". */
async function seedBatch(request) {
await expect(await request.post('/__test__/reset', { data: { extra_images: 3 } })).toBeOK();
const response = await request.post('/prompt_manager/save', { data: { text: 'Cat portrait', category: 'Landscapes' } });
await expect(response).toBeOK();
const id = (await response.json()).prompt_id;
expect(await filteredKeys(request, 'Landscapes')).toEqual(BATCH);
return id;
}
async function openSetCategory(page) {
await categoryButton(page).click();
await expect(categoryDialog(page)).toBeVisible();
return categoryDialog(page);
}
async function applyCategory(page, category) {
const dialog = await openSetCategory(page);
await dialog.getByLabel('Category', { exact: true }).fill(category);
await dialog.getByRole('button', { name: 'Apply', exact: true }).click();
await expect(dialog).toBeHidden();
}
test('Set category applies the category to the linked positives and the category filter then matches', async ({ page, request }) => {
const outputPositive = await unlinkInputCard(request);
test('Set category overrides one image of a batch and never writes its prompt (#332 CUJ 1)', async ({ page, request }) => {
const positiveId = await seedBatch(request);
const before = await positiveOf(request, positiveId);
const writes = categoryWrites(page);
await page.goto(workspace);
await checkBoth(page);
const opener = categoryButton(page);
await opener.click();
const dialog = categoryDialog(page);
await expect(dialog).toBeVisible();
// The concept's simpleInput: one field and Cancel / Apply (primary).
await expect(cards(page)).toHaveCount(5);
await checkbox(cardOf(page, '0:bulk/0001.png'), '0001.png').check();
const dialog = await openSetCategory(page);
// The concept's simpleInput plus "Use prompt category" between Cancel and Apply (primary).
await expect(dialog.locator('#modal-body > .field > label[for="action-input"]')).toHaveText('Category');
await expect(dialog.locator('#modal-body > .field > input#action-input')).toHaveValue('');
await expect(dialog.locator('#modal-body > .dialog-actions > button')).toHaveText(['Cancel', 'Apply']);
await expect(dialog.locator('#modal-body > .dialog-actions > button')).toHaveText(['Cancel', 'Use prompt category', 'Apply']);
await expect(dialog.getByRole('button', { name: 'Apply', exact: true })).toHaveClass(/ca-primary/);
await expect(dialog.getByLabel('Category', { exact: true })).toBeFocused();
await expect(dialog.getByLabel('Category', { exact: true })).toHaveAttribute('maxlength', '100');
// An empty value applies nothing and keeps the dialog open.
const posts = [];
page.on('request', item => { if (item.method() === 'POST' && item.url().endsWith('/prompt_manager/bulk/category')) posts.push(item.postDataJSON()); });
await dialog.getByRole('button', { name: 'Apply', exact: true }).click();
await expect(dialog).toBeVisible();
await dialog.getByLabel('Category', { exact: true }).fill(' Landscapes ');
await dialog.getByLabel('Category', { exact: true }).fill(' Portraits ');
const listings = [];
page.on('request', item => { if (imagePages(new URL(item.url()))) listings.push(item.url()); });
await dialog.getByRole('button', { name: 'Apply', exact: true }).click();
await expect(dialog).toBeHidden();
await expect(toast(page)).toHaveText('Category set on 1 prompt. 1 selected image had no linked prompt and was skipped.');
expect(posts).toEqual([{ prompt_ids: [outputPositive], category: 'Landscapes' }]);
await expect(toast(page)).toHaveText('Category “Portraits” set on 1 image.');
expect(writes).toEqual([{ method: 'PUT', path: IMAGE_CATEGORY,
body: { items: await targetsOf(request, ['0:bulk/0001.png']), category: 'Portraits' } }]);
// The listing and the category options reload; the reload clears the selection.
await expect.poll(() => listings.length).toBeGreaterThan(0);
await expect(bulkbar(page)).toBeEmpty();
await expect(page.locator('#workspace-main h1')).toBeFocused();
await expect(cards(page).getByRole('checkbox', { checked: true })).toHaveCount(0);
const filter = page.getByRole('combobox', { name: 'Filter by category', exact: true });
await expect(filter.locator('option')).toContainText(['Landscapes']);
await filter.selectOption('Landscapes');
await expect(categoryFilter(page).locator('option')).toHaveText(['All categories', 'Landscapes', 'Portraits']);
await categoryFilter(page).selectOption('Portraits');
await expect(cards(page)).toHaveCount(1);
await expect(cards(page)).toHaveAttribute('data-image-key', '0:shared/same.png');
await expect(cards(page)).toHaveAttribute('data-image-key', '0:bulk/0001.png');
await categoryFilter(page).selectOption('Landscapes');
await expect(cards(page)).toHaveCount(3);
expect(await cardKeys(page)).toEqual(BATCH.filter(key => key !== '0:bulk/0001.png'));
// The prompt keeps its category: only the image carries "Portraits".
expect(await positiveOf(request, positiveId)).toEqual(before);
expect(before.category).toBe('Landscapes');
expect(writes.filter(write => write.path === '/prompt_manager/bulk/category')).toEqual([]);
});
test('Set category without a linked image posts nothing and opens no dialog', async ({ page, request }) => {
await unlinkInputCard(request);
test('Set category categorizes unlinked images and lists the value in the toolbar and the suggestions (#332 CUJ 2)', async ({ page, request }) => {
// Unlink both images by deleting their prompts.
for (const image of await listedImages(request)) {
await expect(await request.delete(`/prompt_manager/delete/${image.linked.positive_id}`)).toBeOK();
}
expect((await listedImages(request)).filter(image => image.linked.status === 'linked')).toEqual([]);
const writes = categoryWrites(page);
await page.goto(workspace);
await checkBoth(page);
await applyCategory(page, 'Scans');
await expect(toast(page)).toHaveText('Category “Scans” set on 2 images.');
expect(writes).toEqual([{ method: 'PUT', path: IMAGE_CATEGORY,
body: { items: await targetsOf(request, ['0:shared/same.png', '1:shared/same.png']), category: 'Scans' } }]);
await expect(categoryFilter(page).locator('option')).toHaveText(['All categories', 'Scans']);
await categoryFilter(page).selectOption('Scans');
await expect(cards(page)).toHaveCount(2);
let posted = 0;
await page.route(categoryRoute, route => { posted += 1; return route.continue(); });
await checkbox(cards(page).last(), 'same.png').check();
await categoryButton(page).click();
await expect(toast(page)).toHaveText("None of the selected images is linked to a saved prompt, so there is no prompt category to set.");
expect(await cardKeys(page)).toEqual(['0:shared/same.png', '1:shared/same.png']);
// The field suggests the toolbar's categories through its own datalist.
await checkbox(cards(page).first(), 'same.png').check();
const dialog = await openSetCategory(page);
await expect(dialog.getByLabel('Category', { exact: true })).toHaveAttribute('list', 'workspace-input-suggestions');
expect(await dialog.locator('datalist#workspace-input-suggestions option').evaluateAll(nodes => nodes.map(node => node.value))).toEqual(['Scans']);
await dialog.getByRole('button', { name: 'Cancel', exact: true }).click();
await expect(page.locator('dialog[open]')).toHaveCount(0);
expect(posted).toBe(0);
await expect(bulkbar(page).locator('.bulkbar > strong')).toHaveText('1 selected');
expect(writes).toHaveLength(1);
});
test('Use prompt category removes the own category only after Remove in its confirmation (#332 CUJ 3)', async ({ page, request }) => {
const positiveId = await seedBatch(request);
await putOwnCategory(request, ['0:bulk/0001.png'], 'Portraits');
const writes = categoryWrites(page);
await page.goto(workspace);
await expect(cards(page)).toHaveCount(5);
await expect(categoryFilter(page).locator('option')).toHaveText(['All categories', 'Landscapes', 'Portraits']);
await checkbox(cardOf(page, '0:bulk/0001.png'), '0001.png').check();
// Cancel in the confirmation sends nothing and keeps the selection.
await (await openSetCategory(page)).getByRole('button', { name: 'Use prompt category', exact: true }).click();
await expect(categoryDialog(page)).toHaveCount(0);
const confirm = removeDialog(page);
await expect(confirm).toBeVisible();
await expect(confirm.locator('p').first()).toHaveText("Remove the image category from the 1 selected image? Each one then shows its prompt's category, or none if it has no linked prompt. This cannot be undone.");
await expect(confirm.locator('.dialog-actions > button')).toHaveText(['Cancel', 'Remove']);
await expect(confirm.getByRole('button', { name: 'Remove', exact: true })).toHaveClass(/danger/);
await expect(confirm.getByRole('button', { name: 'Cancel', exact: true })).toBeFocused();
await confirm.getByRole('button', { name: 'Cancel', exact: true }).click();
await expect(page.locator('dialog[open]')).toHaveCount(0);
await expect(selectedCount(page)).toHaveText('1 selected');
expect(writes).toEqual([]);
expect(await filteredKeys(request, 'Portraits')).toEqual(['0:bulk/0001.png']);
// Remove sends exactly one DELETE naming the selected image.
await (await openSetCategory(page)).getByRole('button', { name: 'Use prompt category', exact: true }).click();
await removeDialog(page).getByRole('button', { name: 'Remove', exact: true }).click();
await expect(toast(page)).toHaveText('Removed the image category from 1 image.');
expect(writes).toEqual([{ method: 'DELETE', path: IMAGE_CATEGORY, body: { items: await targetsOf(request, ['0:bulk/0001.png']) } }]);
await expect(bulkbar(page)).toBeEmpty();
// The image shows its prompt's category again: "Landscapes" lists all four, and "Portraits" is gone.
await expect(categoryFilter(page).locator('option')).toHaveText(['All categories', 'Landscapes']);
await categoryFilter(page).selectOption('Landscapes');
await expect(cards(page)).toHaveCount(4);
expect(await cardKeys(page)).toEqual(BATCH);
expect((await positiveOf(request, positiveId)).category).toBe('Landscapes');
});
test('a saved view filtered by a category lists an image categorized after it was saved (#332 CUJ 8)', async ({ page, request }) => {
await seedBatch(request);
await putOwnCategory(request, ['0:bulk/0001.png'], 'Portraits');
await page.goto(workspace);
await categoryFilter(page).selectOption('Portraits');
await expect(cards(page)).toHaveCount(1);
await page.locator('#workspace-main search.toolbar').getByRole('button', { name: 'Save view', exact: true }).click();
const save = page.getByRole('dialog', { name: 'Save this view', exact: true });
await save.getByRole('textbox', { name: 'View name', exact: true }).fill('Portraits');
await save.getByRole('button', { name: 'Save', exact: true }).click();
await expect(toast(page)).toHaveText('Saved view added to the sidebar.');
// Another image gets "Portraits" through Set category.
await categoryFilter(page).selectOption('');
await expect(cards(page)).toHaveCount(5);
await checkbox(cardOf(page, '0:bulk/0002.png'), '0002.png').check();
await applyCategory(page, 'Portraits');
await expect(toast(page)).toHaveText('Category “Portraits” set on 1 image.');
// Reopening the view lists both.
await page.getByRole('navigation', { name: 'Saved views', exact: true }).getByRole('link', { name: 'Portraits', exact: true }).click();
await expect(page.locator('#workspace-main h1')).toHaveText('Portraits');
await expect(categoryFilter(page)).toHaveValue('Portraits');
await expect(cards(page)).toHaveCount(2);
expect(await cardKeys(page)).toEqual(['0:bulk/0001.png', '0:bulk/0002.png']);
});
test('at 390px in light theme the Set category dialog and the category chip fit (#332 CUJ 10)', async ({ page, request }) => {
const LONG = 'Portraits taken on a phone at the end of a long walk through the old harbour at dusk';
await page.addInitScript(() => { try { localStorage.setItem('pm_workspace_theme', 'light'); } catch { /* the default theme */ } });
await page.setViewportSize({ width: 390, height: 844 });
await seedBatch(request);
await putOwnCategory(request, ['0:bulk/0001.png'], LONG);
await page.goto(workspace);
await expect(cards(page)).toHaveCount(5);
await expect(page.locator('html')).toHaveCSS('color-scheme', 'light');
const card = cardOf(page, '0:bulk/0001.png');
// The chip wraps inside the side panel; its origin never splits.
await card.getByRole('button', { name: 'Inspect 0001.png', exact: true }).click();
const chip = details(page).locator('span.chip', { has: page.locator('span.chip-source') });
await expect(chip).toHaveText(`Image category: ${LONG} · this image`);
const fit = await chip.evaluate(node => {
const panel = node.closest('aside, [role="complementary"]').getBoundingClientRect(), box = node.getBoundingClientRect();
const source = node.querySelector('.chip-source');
const line = parseFloat(getComputedStyle(source).lineHeight) || source.getBoundingClientRect().height;
return { inside: box.left >= panel.left - 0.5 && box.right <= panel.right + 0.5, sourceLines: source.getClientRects().length,
wrapped: box.height > line * 1.5, pageScroll: document.documentElement.scrollWidth - window.innerWidth };
});
expect(fit).toEqual({ inside: true, sourceLines: 1, wrapped: true, pageScroll: 0 });
// The dialog fits the width with no horizontal scroll, and its three buttons wrap without overlap.
await checkbox(card, '0001.png').check();
const dialog = await openSetCategory(page);
const geometry = await dialog.evaluate(node => {
const box = node.getBoundingClientRect();
const buttons = [...node.querySelectorAll('.dialog-actions > button')].map(button => button.getBoundingClientRect())
.map(({ left, right, top, bottom }) => ({ left, right, top, bottom }));
const overlaps = buttons.some((a, i) => buttons.some((b, j) => j > i && a.left < b.right && b.left < a.right && a.top < b.bottom && b.top < a.bottom));
const scrollers = [node, ...node.querySelectorAll('*')].filter(item => item.scrollWidth > item.clientWidth + 1 && getComputedStyle(item).overflowX !== 'visible');
return { fits: box.left >= 0 && box.right <= window.innerWidth, scrollers: scrollers.length, count: buttons.length, overlaps,
inside: buttons.every(b => b.left >= box.left && b.right <= box.right), pageScroll: document.documentElement.scrollWidth - window.innerWidth };
});
expect(geometry).toEqual({ fits: true, scrollers: 0, count: 3, overlaps: false, inside: true, pageScroll: 0 });
});
test('Set category of more than 500 images says so before any dialog or request (#332)', async ({ page, request }) => {
await page.route(url => url.pathname === '/prompt_manager/images/output' && url.searchParams.get('limit') !== '1', async route => {
const response = await route.fetch();
const body = await response.json();
await route.fulfill({ response, json: { ...body, total: 501 } });
});
await bulkReset(page, request);
await cardAt(page, 0).getByRole('button', { name: /^Open / }).focus();
await page.keyboard.press('ControlOrMeta+a');
await bulkbar(page).getByRole('button', { name: 'Select all 501 matching', exact: true }).click();
await expect(selectedCount(page)).toHaveText('All 501 matching selected');
const writes = categoryWrites(page);
await categoryButton(page).click();
await expect(toast(page)).toHaveText('Select at most 500 images.');
await expect(page.locator('dialog[open]')).toHaveCount(0);
expect(writes).toEqual([]);
await expect(selectedCount(page)).toHaveText('All 501 matching selected');
});
test('a changed source says so and reloads the listing (#332)', async ({ page }) => {
await page.goto(workspace);
await checkBoth(page);
await page.route(categoryRoute, route => route.fulfill({ status: 409,
json: { success: false, code: 'source-changed', error: 'The output sources changed. Refresh the images and try again.' } }));
const reloaded = page.waitForRequest(item => imagePages(new URL(item.url())));
await applyCategory(page, 'Landscapes');
await expect(toast(page)).toHaveText('The sources changed. Refresh the images and try again.');
await reloaded;
await expect(cards(page)).toHaveCount(2);
// The reload hides the bar with Set category in it: focus moves to the page heading, as after a success.
await expect(page.locator('#workspace-main h1')).toBeFocused();
});
test('a recipe-linked image is listed under its recipe\'s category (#332 D142)', async ({ page, request }) => {
await expect(await request.post('/__test__/reset', { data: { native_library: true } })).toBeOK();
await page.goto(workspace);
await expect(cards(page)).toHaveCount(3);
await expect(categoryFilter(page).locator('option')).toHaveText(['All categories', 'Animals', 'Experiments', 'Portraits']);
// The output same.png is linked to recipe "Portrait variant 1" (Portraits), the input copy to variant 2 (Experiments);
// only the image without a recipe shows its positive's "Animals".
for (const [category, keys] of [['Portraits', ['0:shared/same.png']], ['Experiments', ['1:shared/same.png']], ['Animals', ['0:legacy-unassigned.png']]]) {
await categoryFilter(page).selectOption(category);
await expect(cards(page)).toHaveCount(keys.length);
expect(await cardKeys(page)).toEqual(keys);
}
});
test('Set category reports a failure, keeps the selection and ignores a second press while it runs', async ({ page }) => {
@@ -2059,9 +2279,7 @@ test('Set category reports a failure, keeps the selection and ignores a second p
await checkBoth(page);
const held = [];
await page.route(categoryRoute, route => { held.push(route); });
await categoryButton(page).click();
await categoryDialog(page).getByLabel('Category', { exact: true }).fill('Landscapes');
await categoryDialog(page).getByRole('button', { name: 'Apply', exact: true }).click();
await applyCategory(page, 'Landscapes');
await expect.poll(() => held.length).toBe(1);
await expect(categoryButton(page)).toHaveAttribute('aria-disabled', 'true');
// Playwright waits on an aria-disabled control, so the second press is a dispatched click.
@@ -2098,14 +2316,12 @@ test('leaving Images during Set category finishes the write, and the return relo
await checkBoth(page);
const held = [];
await page.route(categoryRoute, route => { held.push(route); });
await categoryButton(page).click();
await categoryDialog(page).getByLabel('Category', { exact: true }).fill('Landscapes');
await categoryDialog(page).getByRole('button', { name: 'Apply', exact: true }).click();
await applyCategory(page, 'Landscapes');
await expect.poll(() => held.length).toBe(1);
await page.evaluate(() => { window.location.hash = '/overview'; });
await expect(page.locator('#workspace-main h1')).toHaveText('Overview');
// The write is not aborted with the page: the server still receives and answers it.
const answered = page.waitForResponse(response => response.url().endsWith('/prompt_manager/bulk/category'));
const answered = page.waitForResponse(response => new URL(response.url()).pathname === IMAGE_CATEGORY);
await held[0].continue();
expect((await answered).ok()).toBe(true);
await page.evaluate(() => new Promise(resolve => setTimeout(resolve, 150)));
@@ -2116,23 +2332,25 @@ test('leaving Images during Set category finishes the write, and the return relo
await expect(cards(page)).toHaveCount(2);
await expect(bulkbar(page)).toBeEmpty();
await expect(cards(page).getByRole('checkbox', { checked: true })).toHaveCount(0);
await expect(page.getByRole('combobox', { name: 'Filter by category', exact: true }).locator('option')).toContainText(['Landscapes']);
await expect(categoryFilter(page).locator('option')).toContainText(['Landscapes']);
});
test('Set category limits the field to the server length, shows the server count, and a rejection keeps the selection', async ({ page }) => {
test('Set category shows the server counts and coded refusals, and a rejection keeps the selection', async ({ page }) => {
await page.goto(workspace);
await checkBoth(page);
const answers = [
{ status: 400, json: { success: false, error: 'Category cannot exceed 100 characters' } },
{ status: 200, json: { success: true, updated_count: 0 } },
{ status: 404, json: { success: false, code: 'files-not-found', error: 'None of the images could be found.' } },
{ status: 200, json: { success: true, category: 'Landscapes', updated: 1, unchanged: 0, not_found: 1 } },
];
await page.route(categoryRoute, route => route.fulfill(answers.shift()));
for (const expected of ['Category could not be set. Try again.', 'Category set on 0 prompts.']) {
await categoryButton(page).click();
const field = categoryDialog(page).getByLabel('Category', { exact: true });
await expect(field).toHaveAttribute('maxlength', '100');
await field.fill('Landscapes');
await categoryDialog(page).getByRole('button', { name: 'Apply', exact: true }).click();
for (const expected of ['Category could not be set. Try again.',
'None of the selected images could be found. Refresh the images and try again.',
'Category “Landscapes” set on 1 image. 1 could not be found.']) {
const dialog = await openSetCategory(page);
await expect(dialog.getByLabel('Category', { exact: true })).toHaveAttribute('maxlength', '100');
await dialog.getByLabel('Category', { exact: true }).fill('Landscapes');
await dialog.getByRole('button', { name: 'Apply', exact: true }).click();
await expect(toast(page)).toHaveText(expected);
if (answers.length) await expect(bulkbar(page).locator('.bulkbar > strong')).toHaveText('2 selected');
}
@@ -2381,24 +2599,20 @@ test('all matching sets the category through the listing filter and survives a r
// Export metadata waits for loaded images: a press does nothing.
const exportButton = bulkbar(page).getByRole('button', { name: 'Export metadata', exact: true });
await expect(exportButton).toHaveAttribute('title', 'Export metadata reads each image in the browser, so it works on loaded images only.');
// No match linked (a stubbed answer): the nothing-linked sentence, and the selection stays.
await page.route('**/prompt_manager/bulk/category', route => route.fulfill({ json: { success: true, updated_count: 0, prompt_count: 0, skipped: 202 } }), { times: 1 });
const categorize = page.getByRole('dialog', { name: 'Categorize selected images', exact: true });
await bulkbar(page).getByRole('button', { name: 'Set category', exact: true }).click();
await categorize.getByRole('textbox', { name: 'Category', exact: true }).fill('Bulk392');
await categorize.getByRole('button', { name: 'Apply', exact: true }).click();
await expect(toast(page)).toHaveText("None of the selected images is linked to a saved prompt, so there is no prompt category to set.");
// No match listed any more (a stubbed answer): the not-found sentence, and the selection stays.
await page.route(categoryRoute, route => route.fulfill({ status: 404, json: { success: false, code: 'files-not-found', error: 'None of the images could be found.' } }), { times: 1 });
await applyCategory(page, 'Bulk392');
await expect(toast(page)).toHaveText('None of the selected images could be found. Refresh the images and try again.');
await expect(selectedCount(page)).toHaveText('All 202 matching selected');
// The real write: one request with the filter, every matching image's positive changes.
await bulkbar(page).getByRole('button', { name: 'Set category', exact: true }).click();
await categorize.getByRole('textbox', { name: 'Category', exact: true }).fill('Bulk392');
const posted = page.waitForRequest(item => item.method() === 'POST' && new URL(item.url()).pathname === '/prompt_manager/bulk/category');
await categorize.getByRole('button', { name: 'Apply', exact: true }).click();
expect((await posted).postDataJSON()).toEqual({ images_filter: { sort: 'newest' }, category: 'Bulk392' });
await expect(toast(page)).toHaveText(/^Category set on \d+ prompts?\.$/);
// The real write: one PUT with the listing filter sets every matching image's own category; no prompt changes.
const writes = categoryWrites(page);
await applyCategory(page, 'Bulk392');
await expect(toast(page)).toHaveText('Category “Bulk392” set on 202 images.');
expect(writes).toEqual([{ method: 'PUT', path: IMAGE_CATEGORY, body: { filter: { sort: 'newest' }, category: 'Bulk392' } }]);
await expect(bulkbar(page)).toBeEmpty();
const filtered = await (await request.get('/prompt_manager/images/output?mode=workspace&limit=1&category=Bulk392')).json();
expect(filtered.total).toBe(202);
expect((await (await request.get('/prompt_manager/library/options')).json()).categories).not.toContain('Bulk392');
// A return visit keeps "all matching" as the loaded images, and the offer shows again.
await cardAt(page, 0).getByRole('button', { name: /^Open / }).focus();
await page.keyboard.press('ControlOrMeta+a');
@@ -2426,10 +2640,12 @@ test('all matching over 10,000 images says so before any dialog or request', asy
await bulkbar(page).getByRole('button', { name: 'Select all 20,002 matching', exact: true }).click();
await expect(selectedCount(page)).toHaveText('All 20,002 matching selected');
let posted = 0;
page.on('request', item => { if (item.method() === 'POST') posted += 1; });
for (const action of ['Set category', 'Add tags', 'Auto-tag selected']) {
page.on('request', item => { if (item.method() !== 'GET') posted += 1; });
// Set category names at most 500 images (#332); the tagging jobs take up to 10,000.
for (const [action, sentence] of [['Set category', 'Select at most 500 images.'], ['Add tags', 'Narrow the view to at most 10,000 images.'],
['Auto-tag selected', 'Narrow the view to at most 10,000 images.']]) {
await bulkbar(page).getByRole('button', { name: action, exact: true }).click();
await expect(toast(page)).toHaveText('Narrow the view to at most 10,000 images.');
await expect(toast(page)).toHaveText(sentence);
await expect(page.locator('dialog[open]')).toHaveCount(0);
}
expect(posted).toBe(0);
+3 -2
View File
@@ -23,7 +23,7 @@ const heading = page => main(page).getByRole('heading', { level: 1 });
const saveDialog = page => page.getByRole('dialog', { name: 'Save this view', exact: true });
const refineDialog = page => page.getByRole('dialog', { name: 'Refine this view', exact: true });
const toast = page => page.locator('#workspace-toast');
const IMAGE_DEFAULTS = { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false };
const IMAGE_DEFAULTS = { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false, collection: '' };
async function refine(page, values) {
await filtersButton(page).click();
@@ -73,7 +73,8 @@ test('Save view saves the Images toolbar and Refine filters, and its sidebar row
await page.goto(`${workspace}#/images`);
await expect(imageCards(page)).toHaveCount(2);
// The concept's sidebar group: SAVED VIEWS without its PROPOSED badge (D49), empty until a view is saved.
await expect(page.locator('aside.sidebar .sidebar-label')).toHaveText(['WORKSPACE', 'SAVED VIEWS']);
// D136: the COLLECTIONS label stays hidden while no collection is listed.
await expect(page.locator('aside.sidebar .sidebar-label:not([hidden])')).toHaveText(['WORKSPACE', 'SAVED VIEWS']);
await expect(viewRows(page)).toHaveCount(0);
await search(page).fill('dogs');
@@ -0,0 +1,236 @@
// #430: an open workspace page whose sources changed under it. A ComfyUI restart no longer changes any scope (the
// key is persisted, see tests/test_api_workspace_restart.py); what is left is a real source change. Renumbered
// sources (`/__test__/sources/reverse`) are found again: the Tag image dialog stays open on its file and works, and
// the panel keeps the file. A scope that cannot be mapped (`/__test__/scope-key/rotate`, a lost key) blocks Generate
// and Save visibly, says why beside them and offers Refresh. Both commands are undone by every reset.
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from './fixtures.mjs';
const images = '/prompt_manager/workspace#/images';
const FIRST = 'tagging/dunes_00001.png', SECOND = 'tagging/dunes_00002.png';
const SOURCE_CHANGED = 'The source changed. Refresh the image collection before tagging this file.';
const card = (page, path, root = 0) => page.locator(`article[data-image-key="${root}:${path}"]`);
const cards = page => page.locator('.gallery-body article[data-image-key]');
const tagButton = (page, path, root = 0) => card(page, path, root).getByRole('button', { name: `Tag or auto-tag ${path.split('/').pop()}`, exact: true });
const infoButton = (page, path, root = 0) => card(page, path, root).getByRole('button', { name: `Inspect ${path.split('/').pop()}`, exact: true });
const dialog = page => page.getByRole('dialog', { name: 'Tag image', exact: true });
const manual = page => dialog(page).getByRole('textbox', { name: 'Image tags · edit manually', exact: true });
const generate = page => dialog(page).getByRole('button', { name: 'Generate suggestions', exact: true });
const primary = page => dialog(page).locator('.dialog-actions .ca-primary');
const notice = page => dialog(page).locator('.dialog-actions > .tag-action-notice');
const status = page => dialog(page).locator('#tag-generation-status');
const chips = page => dialog(page).locator('#tag-suggestions .chip.tag-suggestion');
const panel = page => page.getByRole('complementary', { name: 'Selected image details', exact: true });
const toast = page => page.locator('#workspace-toast');
const answered = (page, suffix, method = 'POST') => page.waitForResponse(item => new URL(item.url()).pathname.endsWith(suffix) && item.request().method() === method);
test.beforeEach(async ({ page, request, browserName }) => {
await startCoverage(page, browserName);
await expect(await request.post('/__test__/tagging/reset', { data: {} })).toBeOK();
page.staleErrors = [];
page.on('pageerror', error => page.staleErrors.push(error.message));
});
test.afterEach(async ({ page, request, browserName }) => {
await saveCoverage(page, browserName);
expect(page.staleErrors).toEqual([]);
// The next spec file starts on the standard sources and key.
await expect(await request.post('/__test__/tagging/reset', { data: {} })).toBeOK();
});
async function openTag(page, path = FIRST, root = 0) {
const observed = answered(page, '/image-annotations/observe');
await tagButton(page, path, root).click();
await expect(dialog(page)).toBeVisible();
return observed;
}
async function savedTags(request, path, root) {
const listing = await (await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json();
const item = listing.images.find(image => image.path === path && image.root_index === root);
const target = { root_index: root, root_scope: item.root_scope, path };
const observed = await (await request.post('/prompt_manager/image-annotations/observe', { data: { target } })).json();
await request.delete(`/prompt_manager/image-annotations/capture/${observed.capture.token}`);
return observed.tags;
}
test('renumbered sources under an open Tag dialog: it finds the file again, keeps the draft and Generate works', async ({ page, request }) => {
await page.goto(images);
await expect(card(page, FIRST)).toHaveClass(/\bselected\b/);
expect((await openTag(page)).status()).toBe(200);
await manual(page).fill('kept_draft');
await expect(await request.post('/__test__/sources/reverse')).toBeOK();
// Generate is refused (the held scope names another index now); the page asks where the file is and retargets.
const refused = answered(page, '/image-annotations/suggest');
const rescoped = answered(page, '/images/rescope');
const observedAgain = answered(page, '/image-annotations/observe');
await generate(page).click();
expect((await refused).status()).toBe(409);
const mapped = await (await rescoped).json();
expect([mapped.image.root_index, mapped.image.path]).toEqual([1, FIRST]);
const observed = await observedAgain;
expect(observed.status()).toBe(200);
expect(observed.request().postDataJSON().target).toEqual({ root_index: 1, root_scope: mapped.image.root_scope, path: FIRST });
await expect(dialog(page)).toBeVisible();
await expect(notice(page)).toHaveCount(0);
await expect(manual(page)).toHaveValue('kept_draft');
await expect(dialog(page).locator('img.tag-image-preview')).toHaveAttribute('src', /\/image-annotations\/capture\//);
// The page listed again under the new numbering and kept the file selected.
await expect(card(page, FIRST, 1)).toHaveClass(/\bselected\b/);
await expect(card(page, FIRST, 0)).toHaveCount(0);
const suggested = answered(page, '/image-annotations/suggest');
await generate(page).click();
expect((await suggested).status()).toBe(200);
await expect(chips(page).first()).toBeVisible();
const saved = answered(page, '/image-annotations/tags', 'PUT');
await primary(page).click();
expect((await saved).request().postDataJSON().target.root_index).toBe(1);
await expect(toast(page)).toContainText('tags saved for dunes_00001.png.');
expect(await savedTags(request, FIRST, 1)).toContain('kept_draft');
});
test('the inspector finds a renumbered file again; checked files from the old numbering are cleared', async ({ page, request }) => {
await page.goto(images);
await expect(card(page, FIRST)).toHaveClass(/\bselected\b/);
await card(page, FIRST).getByRole('checkbox', { name: /^Select / }).check();
await expect(page.locator('.bulkbar')).toBeVisible();
await expect(await request.post('/__test__/sources/reverse')).toBeOK();
const refused = page.waitForResponse(item => new URL(item.url()).pathname === `/prompt_manager/images/inspect/${SECOND}` &&
new URL(item.url()).searchParams.get('root') === '0');
const reread = page.waitForResponse(item => new URL(item.url()).pathname === `/prompt_manager/images/inspect/${SECOND}` &&
new URL(item.url()).searchParams.get('root') === '1');
await infoButton(page, SECOND).click();
expect((await refused).status()).toBe(409);
expect((await reread).status()).toBe(200);
await expect(card(page, SECOND, 1)).toHaveClass(/\bselected\b/);
await expect(panel(page)).not.toContainText('The source changed');
await expect(panel(page).getByRole('button', { name: 'Refresh image collection', exact: true })).toHaveCount(0);
// A checked key of the old list could name a same-named file in another source: nothing stays checked.
await expect(page.locator('.bulkbar')).toBeHidden();
await expect(cards(page).getByRole('checkbox', { checked: true })).toHaveCount(0);
});
test('a scope that cannot be mapped: Generate and Save are visibly disabled, the reason and Refresh sit beside them', async ({ page }) => {
await page.goto(images);
await expect(card(page, FIRST)).toBeVisible();
await expect(await page.request.post('/__test__/scope-key/rotate')).toBeOK();
const rescoped = answered(page, '/images/rescope');
expect((await openTag(page)).status()).toBe(409);
expect((await rescoped).status()).toBe(409);
await expect(notice(page)).toContainText(SOURCE_CHANGED);
await expect(status(page)).toHaveText('');
const refresh = notice(page).getByRole('button', { name: 'Refresh image collection', exact: true });
await expect(refresh).toBeVisible();
for (const button of [generate(page), primary(page)]) {
await expect(button).toHaveAttribute('aria-disabled', 'true');
await expect(button).toHaveCSS('opacity', '0.55');
}
// The notice is its own first row of the sticky actions, above the buttons.
const rows = await dialog(page).locator('.dialog-actions').evaluate(actions => {
const noticeBox = actions.firstElementChild.getBoundingClientRect();
return { first: actions.firstElementChild.className, below: [...actions.querySelectorAll(':scope > button')].every(button => button.getBoundingClientRect().top >= noticeBox.bottom - 1) };
});
expect(rows).toEqual({ first: 'tag-action-notice', below: true });
const preview = dialog(page).locator('.tag-image-preview');
await expect(preview).toHaveText('Preview unavailable');
await expect(preview).toHaveAttribute('role', 'img');
const suggest = [];
page.on('request', item => { if (item.url().endsWith('/image-annotations/suggest')) suggest.push(item.url()); });
// Pressed anyway (Playwright refuses an aria-disabled target, so the press is dispatched), nothing is sent.
await generate(page).dispatchEvent('click');
await primary(page).dispatchEvent('click');
expect(suggest).toEqual([]);
await page.screenshot({ path: test.info().outputPath('stale-source-tag-dialog.png') });
// Refresh closes the dialog and lists again with the current scopes; Tag then works.
const relisted = page.waitForResponse(item => item.url().includes('/images/output?'));
await refresh.click();
await expect(dialog(page)).toHaveCount(0);
await relisted;
await expect(card(page, FIRST)).toBeVisible();
expect((await openTag(page)).status()).toBe(200);
await expect(notice(page)).toHaveCount(0);
await expect(generate(page)).not.toHaveAttribute('aria-disabled', 'true');
});
test('an unmappable inspector read keeps its visible Refresh, and Refresh recovers the panel', async ({ page }) => {
await page.goto(images);
await expect(card(page, FIRST)).toHaveClass(/\bselected\b/);
await expect(await page.request.post('/__test__/scope-key/rotate')).toBeOK();
const rescoped = answered(page, '/images/rescope');
await infoButton(page, SECOND).click();
expect((await rescoped).status()).toBe(409);
await expect(panel(page)).toContainText('The source changed. Refresh the image collection before inspecting this file.');
const relisted = page.waitForResponse(item => item.url().includes('/images/output?'));
await panel(page).getByRole('button', { name: 'Refresh image collection', exact: true }).click();
await relisted;
await expect(cards(page).first()).toHaveClass(/\bselected\b/);
await expect(panel(page)).not.toContainText('The source changed');
});
test('a Load more cursor the server cannot verify offers Refresh instead of a dead Retry', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/reset', { data: { extra_images: 60 } })).toBeOK();
await page.goto(images);
await expect(cards(page).first()).toBeVisible();
await expect(page.locator('#gallery-continuation')).toContainText('more load as you scroll');
await expect(await request.post('/__test__/scope-key/rotate')).toBeOK();
const stale = page.waitForResponse(item => item.url().includes('/images/output?') && new URL(item.url()).searchParams.has('cursor'));
await page.locator('#gallery-continuation').scrollIntoViewIfNeeded();
const answer = await stale;
expect(answer.status()).toBe(400);
expect((await answer.json()).code).toBe('cursor-stale');
const continuation = page.locator('#gallery-continuation');
await expect(continuation).toContainText('The image collection changed. Refresh image collection to continue.');
const relisted = page.waitForResponse(item => item.url().includes('/images/output?') && !new URL(item.url()).searchParams.has('cursor'));
await continuation.getByRole('button', { name: 'Refresh image collection', exact: true }).click();
expect((await relisted).status()).toBe(200);
await expect(continuation).not.toContainText('The image collection changed');
});
test('a viewer whose media source was renumbered closes onto the file, kept selected', async ({ page, request }) => {
await page.goto(images);
await expect(card(page, FIRST)).toHaveClass(/\bselected\b/);
await expect(await request.post('/__test__/sources/reverse')).toBeOK();
const rescoped = answered(page, '/images/rescope');
await card(page, FIRST).locator('.image-open').click();
const mapped = await (await rescoped).json();
expect([mapped.image.root_index, mapped.image.path]).toEqual([1, FIRST]);
await expect(page.locator('#viewer-dialog')).toHaveCount(0);
await expect(card(page, FIRST, 1)).toHaveClass(/\bselected\b/);
// Opened again, it shows the file under its current source.
await card(page, FIRST, 1).locator('.image-open').click();
await expect(page.locator('#viewer-dialog[open]')).toBeVisible();
await expect(page.locator('#viewer-dialog').getByText('Media could not be loaded. Retry this file.', { exact: true })).toBeHidden();
await expect.poll(() => page.locator('#viewer-dialog img').first().evaluate(image => image.complete && image.naturalWidth > 0)).toBe(true);
});
test('a collection page finds a renumbered member again and stays on its members', async ({ page, request }) => {
// Members are stored by root path, not index (#331), so a renumbering keeps them; the page's scopes go stale.
const listing = await (await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json();
const first = listing.images.find(image => image.path === FIRST && image.root_index === 0);
const name = `Stale ${test.info().workerIndex}-${Date.now()}`;
const added = await request.post('/prompt_manager/collections/items', { data: { name, items: [
{ root_index: 0, root_scope: first.root_scope, path: FIRST }] } });
await expect(added).toBeOK();
const collections = (await (await request.get('/prompt_manager/collections')).json()).collections;
const { id } = collections.find(item => item.name === name);
await page.goto(`/prompt_manager/workspace#/collection/${id}`);
await expect(cards(page)).toHaveCount(1);
await expect(card(page, FIRST)).toHaveClass(/\bselected\b/);
await expect(await request.post('/__test__/sources/reverse')).toBeOK();
const rescoped = answered(page, '/images/rescope');
const relisted = page.waitForResponse(item => item.url().includes('/images/output?') && new URL(item.url()).searchParams.get('collection') === String(id));
await card(page, FIRST).locator('.image-open').click();
expect((await (await rescoped).json()).image.root_index).toBe(1);
expect((await relisted).status()).toBe(200);
await expect(page).toHaveURL(new RegExp(`#/collection/${id}$`));
await expect(cards(page)).toHaveCount(1);
await expect(card(page, FIRST, 1)).toHaveClass(/\bselected\b/);
// Tag image works on the member under its new source.
expect((await openTag(page, FIRST, 1)).status()).toBe(200);
await expect(notice(page)).toHaveCount(0);
});
@@ -123,6 +123,32 @@ test('model options are the four real engines with size and the description show
await expect(actionOf(page, 'Unload')).toBeDisabled();
});
const NEEDS_LLAMA = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
test('a downloaded model whose backend package is missing says so and is not offered as ready (#424)', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: ['wd14-vit'],
missing_packages: { gguf: ['llama-cpp-python'], 'wd14-vit': ['onnxruntime'] } } })).toBeOK();
await openTagging(page);
const line = modelRow(page).locator('p').first();
await page.locator('#tag-model').selectOption('gguf');
await expect(line).toHaveText('Downloaded. Needs llama-cpp-python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python');
// Files and packages are separate: Download (again) stays offered.
await expect(actionOf(page, 'Download again')).not.toHaveAttribute('aria-disabled', 'true');
// Review explains instead of reviewing; nothing is previewed.
await expect(reviewButton(page)).toHaveAttribute('aria-disabled', 'true');
await reviewButton(page).dispatchEvent('click');
await expect(toast(page)).toHaveText(NEEDS_LLAMA);
expect(page.modelCalls.filter(call => call.endsWith('/autotag/batch/preview'))).toEqual([]);
// Not downloaded reads differently and still names what it will need.
await page.locator('#tag-model').selectOption('wd14-vit');
await expect(line).toHaveText('Not downloaded. Download size ~400 MB. Also needs onnxruntime.');
await page.locator('#tag-model').selectOption('wd14-swinv2');
await expect(line).toHaveText('Downloaded. Not loaded.');
await expect(reviewButton(page)).not.toHaveAttribute('aria-disabled', 'true');
});
test('entering the section makes no model call besides status', async ({ page }) => {
await openTagging(page);
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
+1 -1
View File
@@ -472,7 +472,7 @@ async function openImages(page) {
test('defaults apply to a new visit but not to the current view or a saved view', async ({ page, request }) => {
const created = await request.post('/prompt_manager/saved-views', { data: { page: 'images', name: 'Everything',
state: { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false } } });
state: { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false, collection: '' } } });
expect(created.status()).toBe(201);
const view = (await created.json()).view;
try {
@@ -186,6 +186,26 @@ test('a model that is not downloaded keeps Start disabled and says why', async (
expect(posted).toBe(0);
});
const NEEDS_LLAMA = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
test('a model missing its backend package keeps Start disabled and names the install command (#424)', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: [],
missing_packages: { gguf: ['llama-cpp-python'] } } })).toBeOK();
await select(page, FIRST);
await barButton(page, 'Auto-tag selected').click();
const status = modal(page).locator('#tag-generation-status');
const start = actionButtons(page).last();
await modal(page).getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('gguf');
await expect(status).toHaveText(NEEDS_LLAMA);
await expect(start).toHaveAttribute('aria-disabled', 'true');
await modal(page).getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('wd14-vit');
await expect(status).toHaveText('');
await expect(start).not.toHaveAttribute('aria-disabled', 'true');
await actionButtons(page).first().click();
await expect(modal(page)).toHaveCount(0);
});
test('Suggestions field prefills from the setting and caps chips at the entered count', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/defaults', { data: { general: 0.5, character: 0.9, limit: 7 } })).toBeOK();
await select(page, FIRST);
+16 -1
View File
@@ -21,6 +21,8 @@ const model = page => dialog(page).getByRole('combobox', { name: 'Tagging model'
const generate = page => dialog(page).getByRole('button', { name: 'Generate suggestions', exact: true });
const chips = page => dialog(page).locator('#tag-suggestions .chip.tag-suggestion');
const status = page => dialog(page).locator('#tag-generation-status');
// D147: the reason a state blocks Save, on its own row in the sticky actions.
const notice = page => dialog(page).locator('.dialog-actions > .tag-action-notice');
const toast = page => page.locator('#workspace-toast');
const nav = (page, label) => page.locator('aside.sidebar nav').getByRole('link', { name: new RegExp(`^${label}(?:\\s*[\\d,]+)?$`) });
const FIRST = 'tagging/dunes_00001.png', SECOND = 'tagging/dunes_00002.png', COPY = 'copies/dunes_00001.png';
@@ -260,6 +262,18 @@ test('unavailable model shows inline status and never downloads', async ({ page,
await expect(dialog(page).locator('.dialog-actions .ca-primary')).toBeFocused();
});
test('a missing backend package is the server\'s install sentence, no traceback page (#424)', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: [],
missing_packages: { 'wd14-swinv2': ['onnxruntime'] } } })).toBeOK();
await openTag(page);
const answer = await generated(page);
expect(answer.status()).toBe(409);
expect(await answer.json()).toMatchObject({ success: false, code: 'model-unavailable' });
await expect(status(page)).toHaveText('WD14 SwinV2 needs onnxruntime. Install it into ComfyUI\'s Python: '
+ 'pip install onnxruntime. Use onnxruntime-gpu for CUDA.');
await expect(chips(page)).toHaveCount(0);
});
test('video tag is manual only', async ({ page, request }) => {
await openTag(page, VIDEO);
const preview = dialog(page).locator('video.tag-image-preview');
@@ -716,7 +730,8 @@ test('D116: Retry leads the actions after an unconfirmed save and resends the sa
let lost = true;
await page.route('**/prompt_manager/image-annotations/tags', route => (lost ? (lost = false, route.fulfill({ status: 504, body: 'Gateway Timeout' })) : route.continue()));
await dialog(page).getByRole('button', { name: 'Save manual tags', exact: true }).click();
await expect(status(page)).toHaveText('The save could not be confirmed. Retry sends the same tags again.');
await expect(notice(page)).toHaveText('The save could not be confirmed. Retry sends the same tags again.');
await expect(status(page)).toHaveText('');
const inventory = await page.locator('dialog#modal').evaluate(taggingInventory);
expect(inventory.actions.buttons.map(button => [button.text, button.primary, button.disabled])).toEqual(
[['Retry', false, false], ['Cancel', false, false], ['Save manual tags', true, true]]);
@@ -383,6 +383,62 @@ test('manage categories previews real impact before applying',async({page,reques
expect(after.prompts.filter(row=>row[2]==='Scenery')).toHaveLength(3);
});
// #332 MC9, MC10: image categories (their own, set on Images) count beside the prompts. The vocabulary fixture has no
// image files, so these tests add two committed files to the output folder and set their categories through the API.
async function imageCategories(request,category) {
for(const name of ['f02.png','f03.png']) await expect(await request.post('/__test__/add-local-inspect-file',{data:{name}})).toBeOK();
const listing=await(await request.get('/prompt_manager/images/output?mode=workspace&subfolder=local-inspect&limit=40')).json();
const items=listing.images.map(({root_index,root_scope,path})=>({root_index,root_scope,path}));expect(items).toHaveLength(2);
const response=await request.put('/prompt_manager/images/category',{data:{items,category}});await expect(response).toBeOK();
expect((await response.json()).updated).toBe(2);
}
const ownCategories=async request=>(await(await request.get('/prompt_manager/images/categories')).json()).categories;
const IMAGE_KEYS=['0:local-inspect/f02.png','0:local-inspect/f03.png'];
test('manage categories renames the images\' own categories too and the preview counts them (#332 CUJ 6)',async({page,request})=>{
await imageCategories(request,'Landscapes');
const writes=trackWrites(page);await openPage(page);
await expect(categoryRow(page,'Landscapes').locator('p')).toHaveText('3 positive prompts · 2 images');
const box=await openManage(page);
await expect(field(box,'Category')).toHaveValue('Landscapes');
await field(box,'Proposed new category name').fill('Landscape');await button(box,'Preview').click();
await expect(box.locator('p.small')).toHaveText('“Landscapes” is used by 3 positive prompts and 2 images. Apply renames it to “Landscape”.');
await button(box,'Apply').click();
await expect(box).toHaveCount(0);await expect(toast(page)).toHaveText('Category renamed.');
expect(writes.at(-1)).toMatchObject({url:'/prompt_manager/vocabulary/categories/apply',body:{source:'Landscapes',target:'Landscape'}});
await expect(categoryRow(page,'Landscapes')).toHaveCount(0);
await expect(categoryRow(page,'Landscape').locator('p')).toHaveText('3 positive prompts · 2 images');
// The prompts and both images carry the new name; nothing keeps the old one.
expect((await snapshot(request)).prompts.filter(row=>row[2]==='Landscape')).toHaveLength(3);
expect(await ownCategories(request)).toEqual(['Landscape']);
for(const [category,total] of [['Landscape',2],['Landscapes',0]]) {
const listing=await(await request.get(`/prompt_manager/images/output?mode=workspace&limit=40&category=${category}`)).json();
expect(listing.images.map(image=>`${image.root_index}:${image.path}`).sort()).toEqual(total?IMAGE_KEYS:[]);
}
});
test('a category only images hold is listed after the prompt-backed ones and Browse opens Images (#332 CUJ 7)',async({page,request})=>{
await imageCategories(request,'Scans');
await openPage(page);
const operations=surfaces(page).nth(1).locator('.operation');
await expect(operations.locator('h3')).toHaveText([...CATEGORIES.map(([name])=>name),'Scans','Manage category']);
await expect(categoryRow(page,'Scans').locator('p')).toHaveText('2 images');
for(const [name,count] of CATEGORIES) await expect(categoryRow(page,name).locator('p')).toHaveText(`${count} positive prompt${count===1?'':'s'}`);
// Manage category can rename it: its select lists it last.
const box=await openManage(page);
await expect(field(box,'Category').locator('option')).toHaveText([...CATEGORIES.map(([name])=>name),'Scans']);
await button(box,'Cancel').click();await expect(box).toHaveCount(0);
// Browse opens Images filtered by it (its Library would be empty).
await categoryRow(page,'Scans').getByRole('button',{name:'Browse',exact:true}).click();
await expect(page).toHaveURL(/#\/images$/);
await expect(main(page).getByRole('heading',{level:1})).toHaveText('Images');
await expect(page.getByRole('combobox',{name:'Filter by category',exact:true})).toHaveValue('Scans');
const cards=page.locator('article[data-image-key]');await expect(cards).toHaveCount(2);
expect((await cards.evaluateAll(nodes=>nodes.map(node=>node.dataset.imageKey))).sort()).toEqual(IMAGE_KEYS);
await expect(page.getByRole('searchbox',{name:'Search workspace',exact:true})).toHaveValue('');
await expect(page.locator('#workspace-main search.toolbar').getByRole('button',{name:'Filters',exact:true})).toBeVisible();
});
test('an edit during a pending preview needs a new Preview; Apply never sends the old values',async({page,request})=>{
const writes=trackWrites(page);await openPage(page);const before=await snapshot(request);
let release;const held=new Promise(resolve=>{release=resolve;});
+9 -5
View File
@@ -548,12 +548,16 @@ test('sidebar matches the concept groups and shows real counts', async ({ page,
await expect(page.locator('.workspace-shell, .workspace-body, .sidebar-footer, .connection-dot, .workspace-icon')).toHaveCount(0);
const sidebar = page.locator('aside.sidebar');
expect(await childShape(sidebar)).toEqual(['a.brand', 'div.sidebar-label', 'nav#primary-nav', 'div.sidebar-label',
'nav#saved-nav', 'div.sidebar-bottom']);
'nav#saved-nav', 'div.sidebar-label', 'nav#collections-nav', 'div.sidebar-bottom']);
await expect(sidebar.locator('.brand')).toHaveAttribute('href', '#/images');
// D49: SAVED VIEWS without the concept's PROPOSED badge; D28: still no COLLECTIONS.
await expect(sidebar.locator('.sidebar-label')).toHaveText(['WORKSPACE', 'SAVED VIEWS']);
// D49: SAVED VIEWS without the concept's PROPOSED badge. D136: COLLECTIONS without it either, its label hidden
// and its nav empty while no collection exists.
await expect(sidebar.locator('.sidebar-label')).toHaveText(['WORKSPACE', 'SAVED VIEWS', 'COLLECTIONS']);
await expect(sidebar.locator('.sidebar-label').nth(2)).toBeHidden();
await expect(page.locator('nav#saved-nav')).toHaveAttribute('aria-label', 'Saved views');
await expect(page.locator('#collections-nav, .new-label')).toHaveCount(0);
await expect(page.locator('nav#collections-nav')).toHaveAttribute('aria-label', 'Collections');
await expect(page.locator('nav#collections-nav > a')).toHaveCount(0);
await expect(page.locator('.new-label')).toHaveCount(0);
const primary = page.locator('nav#primary-nav');
await expect(primary).toHaveAttribute('aria-label', 'Workspace');
for (const [id] of NAV_TOTALS) await expect(navCount(page, id)).toHaveText(totals[id].toLocaleString('en-US'));
@@ -580,7 +584,7 @@ test('sidebar matches the concept groups and shows real counts', async ({ page,
// At phone width the concept wraps the links and hides counts and the connection (concept 950px block).
await page.setViewportSize({ width: 390, height: 844 });
await expect(navCount(page, 'library')).toBeHidden();
await expect(page.locator('.sidebar-label, .connection')).toHaveCount(3);
await expect(page.locator('.sidebar-label, .connection')).toHaveCount(4);
for (const hidden of await page.locator('.sidebar-label, .connection').all()) await expect(hidden).toBeHidden();
await expect(primary.locator('a[href="#/inspect"]')).toBeVisible();
});
@@ -0,0 +1,131 @@
"""Owned, deterministic collections for the workspace collection workflows (#331).
Twenty-one PNGs under the output source: twelve listed members of
"Landscapes", eight of "Experiments" and one loose file in neither. Landscapes
also holds one member whose file does not exist (MC3: hidden, counted as
missing). One saved view reopens "Experiments" (D140). Members are added
through ``ImageCollectionOperations`` with favorites' keys
(``_favorite_root_key``, ``_favorite_relative_key``), as the routes add them.
"""
from __future__ import annotations
import os
import time
from pathlib import Path
from typing import TYPE_CHECKING
from aiohttp import web
from PIL import Image
from tests.browser.folder_trash import replace_folder
if TYPE_CHECKING:
from py.api import PromptManagerAPI
FOLDER = "collections"
LANDSCAPES = "Landscapes"
EXPERIMENTS = "Experiments"
LANDSCAPE_FILES = tuple(f"landscape_{index:02d}.png" for index in range(1, 13))
EXPERIMENT_FILES = tuple(f"experiment_{index:02d}.png" for index in range(1, 9))
LOOSE_FILE = "loose.png"
MISSING_FILE = "gone.png"
SAVED_VIEW = "Experiments view"
_TABLES = (
"image_collection_items",
"image_collections",
"saved_views",
"image_favorites",
"media_review_decisions",
"media_review_items",
"media_image_tags",
"media_annotation_bytes",
"media_annotation_files",
"image_prompt_pairs",
"prompt_pair_tags",
"prompt_pairs",
"generated_images",
"prompt_tags",
"tags",
"prompt_library_state",
"prompts",
)
_IMAGES_STATE = {
"q": "",
"category": "",
"sort": "newest",
"source": "all",
"folder": "",
"media": "all",
"tags": [],
"favorites": False,
}
def _write_png(path: Path, shade: int, stamp: float) -> None:
with Image.new("RGB", (64, 48), (shade, 120, 255 - shade)) as image:
image.save(path)
os.utime(path, (stamp, stamp))
def seed_collections(api: PromptManagerAPI, root: Path) -> dict:
"""Replace this server's disposable rows and files with the two collections."""
# The API package needs the host stubs the server installs before importing it.
from py.api.images import _favorite_relative_key, _favorite_root_key
db = api.db
with db.model.get_connection() as conn:
for table in _TABLES:
conn.execute(f"DELETE FROM {table}")
for name in ("output", "input"):
replace_folder(root / name)
folder = root / "output" / FOLDER
folder.mkdir()
# Newest first in this order: Landscapes, then Experiments, then the loose file.
now = time.time()
ordered = (*LANDSCAPE_FILES, *EXPERIMENT_FILES, LOOSE_FILE)
for index, name in enumerate(ordered):
_write_png(folder / name, (index * 11) % 256, now - 60 - index * 60)
root_key = _favorite_root_key(list(api._get_all_output_dirs())[0])
members = {
LANDSCAPES: (*LANDSCAPE_FILES, MISSING_FILE),
EXPERIMENTS: EXPERIMENT_FILES,
}
ids = {}
for name, files in members.items():
added = db.add_to_collection(
name=name,
items=[
(root_key, _favorite_relative_key(f"{FOLDER}/{file}")) for file in files
],
)
ids[name] = added["collection"]["id"]
view = db.create_saved_view(
"images", SAVED_VIEW, {**_IMAGES_STATE, "collection": str(ids[EXPERIMENTS])}
)
api.invalidate_gallery_cache()
return {
"collections": {
LANDSCAPES: {"id": ids[LANDSCAPES], "count": 12, "missing": 1},
EXPERIMENTS: {"id": ids[EXPERIMENTS], "count": 8, "missing": 0},
},
"files": {
LANDSCAPES: [f"{FOLDER}/{file}" for file in LANDSCAPE_FILES],
EXPERIMENTS: [f"{FOLDER}/{file}" for file in EXPERIMENT_FILES],
"loose": f"{FOLDER}/{LOOSE_FILE}",
},
"saved_view": {"id": view["id"], "name": SAVED_VIEW},
}
def register_collections_commands(
routes: web.RouteTableDef, api: PromptManagerAPI, root: Path
) -> None:
"""Register the fixed seed command; no path or SQL comes from a request."""
@routes.post("/__test__/collections/seed")
async def seed(request: web.Request) -> web.Response:
data = await request.json() if request.can_read_body else {}
if data != {} or request.query:
raise web.HTTPBadRequest()
manifest = await api._run_in_executor(seed_collections, api, root)
return web.json_response({"success": True, **manifest})
@@ -174,6 +174,8 @@ _TABLES = (
"execution_contexts",
"execution_prompt_snapshots",
"execution_context_texts",
"image_collection_items",
"image_collections",
"saved_views",
"image_favorites",
"media_review_decisions",
@@ -193,6 +195,10 @@ _TABLES = (
)
# The concept's collections and the slice of its images each holds (#331).
CONCEPT_COLLECTIONS = (("Landscapes", slice(0, 12)), ("Experiments", slice(12, 20)))
def concept_starred(prompt_index: int, round_index: int) -> bool:
"""The concept's favorites (app.js ``starred``): each other prompt's first image."""
return round_index == 0 and prompt_index % 2 == 0
@@ -495,6 +501,19 @@ def seed_workspace_concept(
)
images.reverse()
runs.reverse()
# The concept's COLLECTIONS rows (app.js a336–339): "Landscapes" holds the
# first 12 images in the concept's order and "Experiments" the next 8, so
# the rows and counts compare as the concept shows them (#331 D136).
collections = {}
for name, members in CONCEPT_COLLECTIONS:
added = db.add_to_collection(
name=name,
items=[
(favorite_root, _favorite_relative_key(image["path"]))
for image in images[members]
],
)
collections[name] = added["collection"]["id"]
seedless = []
for number in range(seedless_images):
name = f"seedless_{number + 1:05}.png"
@@ -531,4 +550,5 @@ def seed_workspace_concept(
"images": images,
"runs": runs,
"tags": tags,
"collections": collections,
}
@@ -26,6 +26,8 @@ EDITOR_METADATA = {
"notes": "Keep the soft light.",
}
_TABLES = (
"image_collection_items",
"image_collections",
"saved_views",
"image_favorites",
"media_review_decisions",
@@ -78,6 +78,8 @@ LIMITED_SAMPLERS = 17
# More than one queue page (40), so Load more has a second page.
MAX_ITEMS = 45
_TABLES = (
"image_collection_items",
"image_collections",
"saved_views",
"image_favorites",
"media_review_decisions",
+35 -4
View File
@@ -41,9 +41,23 @@ FILES = (
("loose/unlinked.png", "unlinked"),
)
MODELS = ("wd14-swinv2", "wd14-vit", "gguf", "hf")
def _valid_missing(value) -> bool:
"""``{"<model>": ["<pip name>", ...]}`` with known models and string names."""
return isinstance(value, dict) and all(
name in MODELS
and isinstance(names, list)
and all(isinstance(pip, str) and pip for pip in names)
for name, names in value.items()
)
# Unlinked copies enough to virtualize the Images grid (it renders all up to 120).
MAX_EXTRA_IMAGES = 160
_TABLES = (
"image_collection_items",
"image_collections",
"saved_views",
"image_favorites",
"media_review_decisions",
@@ -180,9 +194,16 @@ def tagging_snapshot(api: PromptManagerAPI) -> dict:
def register_tagging_commands(
routes: web.RouteTableDef, api: PromptManagerAPI, root: Path, model
routes: web.RouteTableDef,
api: PromptManagerAPI,
root: Path,
model,
on_reset=lambda: None,
) -> None:
"""Register fixed seed, snapshot and fake-model commands; no paths or SQL input."""
"""Register fixed seed, snapshot and fake-model commands; no paths or SQL input.
``on_reset`` runs first on every tagging reset (the source-change undo).
"""
# The fake download (``/__test__/tagging/download``): waits on ``event``,
# then answers ``result``. Every reset sets the event and puts back the
# model's raising default, so no test can leave a worker hanging.
@@ -216,7 +237,9 @@ def register_tagging_commands(
raise web.HTTPBadRequest()
if not 0 <= extra <= MAX_EXTRA_IMAGES:
raise web.HTTPBadRequest()
on_reset()
model.unavailable = set()
model.missing_packages = {}
model.suggestions = None
model.loaded = None
restore_download()
@@ -235,17 +258,25 @@ def register_tagging_commands(
@routes.post("/__test__/tagging/models")
async def models(request: web.Request) -> web.Response:
"""Mark fake models as not downloaded (``{"unavailable": [...]}``)."""
"""Mark fake models as not downloaded (``{"unavailable": [...]}``) and,
optionally, as missing backend packages (``"missing_packages":
{"<model>": ["<pip name>", ...]}``, #424)."""
data = await request.json() if request.can_read_body else None
if (
request.query
or not isinstance(data, dict)
or set(data) != {"unavailable"}
or "unavailable" not in data
or set(data) - {"unavailable", "missing_packages"}
or not isinstance(data["unavailable"], list)
or any(name not in MODELS for name in data["unavailable"])
or not _valid_missing(data.get("missing_packages", {}))
):
raise web.HTTPBadRequest()
model.unavailable = set(data["unavailable"])
model.missing_packages = {
name: list(names)
for name, names in data.get("missing_packages", {}).items()
}
return web.json_response({"success": True})
@routes.post("/__test__/tagging/loaded")
@@ -112,6 +112,8 @@ def seed_workspace_visual(api: PromptManagerAPI, root: Path) -> None:
"""Replace only the owned browser harness records and media roots."""
with api.db.model.get_connection() as conn:
for table in (
"image_collection_items",
"image_collections",
"saved_views",
"image_favorites",
"media_review_decisions",
+36 -3
View File
@@ -72,6 +72,10 @@ CONCEPT_RECIPES = (
"negative": "oversaturated, flat lighting, watermark, text",
},
)
# #332 D143/D144 (opt-in, ``image_categories``): own image categories, one on
# images only and one shared with prompts. Stored rows count whether or not the
# files exist, so the keys name files the fixture never writes.
IMAGE_CATEGORY_ROWS = (("Landscapes", 2), ("Portraits", 1))
PAIR_ONLY_TAG = "pair_only_detail"
ARCHIVED_ONLY_TAG = "archived_pair_detail"
NEGATIVE_ONLY_TAG = "legacy_negative_detail"
@@ -79,6 +83,8 @@ UNUSED_TAG = "unused_vocabulary"
MAX_EXTRA_TAGS = 300
MAX_EXTRA_CATEGORIES = 200
_TABLES = (
"image_collection_items",
"image_collections",
"saved_views",
"image_favorites",
"media_review_decisions",
@@ -112,8 +118,26 @@ _SNAPSHOT = {
}
def _seed_image_categories(api: PromptManagerAPI) -> dict:
"""Own image category rows keyed as Images keys them; ``{value: count}``."""
from py.api.images import image_category_key
sources = list(api._get_all_output_dirs())
for category, count in IMAGE_CATEGORY_ROWS:
keys = [
image_category_key(sources, str(Path(sources[0]) / f"{category}_{n}.png"))
for n in range(count)
]
api.db.set_image_categories(keys, category)
return dict(IMAGE_CATEGORY_ROWS)
def seed_vocabulary(
api: PromptManagerAPI, root: Path, extra_tags: int = 0, extra_categories: int = 0
api: PromptManagerAPI,
root: Path,
extra_tags: int = 0,
extra_categories: int = 0,
image_categories: bool = False,
) -> dict:
"""Replace this server's disposable rows with the concept vocabulary."""
db = api.db
@@ -167,6 +191,7 @@ def seed_vocabulary(
"category": f"bulk_category_{number:04}",
},
)
images = _seed_image_categories(api) if image_categories else {}
with db.model.get_connection() as conn:
tags = [row[0] for row in conn.execute("SELECT name FROM tags ORDER BY id")]
api.invalidate_gallery_cache()
@@ -180,6 +205,7 @@ def seed_vocabulary(
"archived_only_tag": ARCHIVED_ONLY_TAG,
"negative_only_tag": NEGATIVE_ONLY_TAG,
"unused_tag": UNUSED_TAG,
"image_categories": images,
}
@@ -200,7 +226,14 @@ def register_vocabulary_commands(
@routes.post("/__test__/vocabulary/reset")
async def reset(request: web.Request) -> web.Response:
data = await request.json() if request.can_read_body else {}
if not isinstance(data, dict) or set(data) - {"extra_tags", "extra_categories"}:
if not isinstance(data, dict) or set(data) - {
"extra_tags",
"extra_categories",
"image_categories",
}:
raise web.HTTPBadRequest()
images = data.get("image_categories", False)
if not isinstance(images, bool):
raise web.HTTPBadRequest()
extra = data.get("extra_tags", 0)
categories = data.get("extra_categories", 0)
@@ -215,7 +248,7 @@ def register_vocabulary_commands(
if request.query:
raise web.HTTPBadRequest()
manifest = await api._run_in_executor(
seed_vocabulary, api, root, extra, categories
seed_vocabulary, api, root, extra, categories, images
)
return web.json_response({"success": True, **manifest})
+45 -3
View File
@@ -16,8 +16,8 @@ installHostHooks(test, ['PromptManager', 'PromptManagerText', 'PromptManagerFocu
// `primitive`: a core string node the queue-time resolver follows. `legacy`: the frontend's virtual Primitive node,
// whose value is queued as literal text. `preview`: PrimitiveString -> PreviewAny, the shape of the maintainer's
// TextGenerate -> PreviewAny chain, whose text is only known when the node runs.
function build(page, { kind, positive, negative, prefix }) {
return page.evaluate(async ({ kind, positive, negative, prefix }) => {
function build(page, { kind, positive, negative, prefix, saveToLibrary }) {
return page.evaluate(async ({ kind, positive, negative, prefix, saveToLibrary }) => {
const { app } = await import('/scripts/app.js');
app.graph.clear();
const add = (type, x, y) => { const node = LiteGraph.createNode(type); app.graph.add(node); node.pos = [x, y]; return node; };
@@ -28,6 +28,7 @@ function build(page, { kind, positive, negative, prefix }) {
const set = (name, value) => { const widget = prompt.widgets.find(item => item.name === name); widget.value = value; if (widget.inputEl) widget.inputEl.value = value; };
set('text', positive.typed);
if (negative) set('negative_text', negative.typed);
if (saveToLibrary !== undefined) prompt.widgets.find(item => item.name === 'save_to_library').value = saveToLibrary;
const drive = (name, side, y) => {
if (!side.via) return;
if (side.via === 'legacy') {
@@ -73,7 +74,7 @@ function build(page, { kind, positive, negative, prefix }) {
await prompt._pmTextSerializerReady;
const { output, workflow } = await app.graphToPrompt();
return { output, workflow, promptNode: String(prompt.id) };
}, { kind, positive, negative, prefix });
}, { kind, positive, negative, prefix, saveToLibrary });
}
async function run(execution, page, testCase, built) {
@@ -195,6 +196,47 @@ for (const [label, linked, negativeVia] of [['positive', ['text'], 'primitive'],
});
}
// #90: with Save to library off the run still produces its image, but PromptManager saves, counts and links nothing,
// and the run's history records no prompt text for it.
for (const kind of ['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair']) {
test(`${kind} with Save to library off saves, counts and links nothing (#90)`, async ({ execution, page }) => {
test.setTimeout(90000);
const text = unique(`Not saved ${kind}`);
const negative = kind === 'PromptManagerPair' ? { typed: unique('Not saved negative') } : undefined;
const prefix = `notsaved/${randomUUID()}`;
const { output, workflow, promptNode } = await build(page, { kind, positive: { typed: text }, negative, prefix, saveToLibrary: false });
expect(output[promptNode].inputs.save_to_library).toBe(false);
const id = randomUUID();
const response = await execution.request.post('/prompt', { data: { prompt: output, prompt_id: id, extra_data: { extra_pnginfo: { workflow } } } });
expect(response.ok(), await response.text()).toBe(true);
let item;
await expect.poll(async () => {
item = (await (await execution.request.get(`/history/${id}`)).json())[id];
return item?.status?.status_str ?? null;
}, { timeout: 20000 }).toBe('success');
let result;
await expect.poll(async () => {
result = await readExecutionRuns(execution.databasePath, id);
const run = result.rows[0];
return Boolean(run && result.captures[run.run_id]);
}, { timeout: 10000, message: 'the run reaches SQLite' }).toBe(true);
const run = result.rows[0];
for (const context of result.captures[run.run_id].contexts || []) {
expect(context.positive_prompt_id ?? null).toBeNull();
expect(context.pair_id ?? null).toBeNull();
}
expect((result.usage[run.run_id] || []).every(entry => entry.run_count === 0)).toBe(true);
const files = Object.values(item.outputs).flatMap(value => value.images || []).map(image => image.filename);
expect(files.length).toBeGreaterThan(0);
await scan(execution);
expect(Object.values(await linkedPrompts(execution, files))).toEqual(files.map(() => null));
for (const query of [text, negative?.typed].filter(Boolean)) {
const found = await (await execution.request.get(`/prompt_manager/search?text=${encodeURIComponent(query)}`)).json();
expect(found.results ?? [], query).toEqual([]);
}
});
}
// #347: a cached PromptManager node counts every job, as an executing one does. Jobs after the first change only
// the sampler seed, so ComfyUI serves the PromptManager node (and its PreviewAny source) from its output cache.
const COUNTS = `import json, sqlite3, sys, pathlib
+10 -1
View File
@@ -370,6 +370,7 @@ test('workflows saved before #343 load unchanged and their linked text keeps wor
const nodes = Object.fromEntries(app.graph._nodes.map(node => [node.id, node]));
const reloaded = app.graph.serialize();
return {
saves: Object.fromEntries(Object.entries(output).filter(([, node]) => node.class_type.startsWith('PromptManager')).map(([id, node]) => [id, node.inputs.save_to_library])),
values: Object.fromEntries(reloaded.nodes.map(node => [node.id, node.widgets_values])),
links: Object.fromEntries(Object.entries(output).map(([id, node]) => [id, { text: node.inputs.text, negative_text: node.inputs.negative_text }])),
widgetBound: Object.fromEntries(Object.values(nodes).filter(node => node.type.startsWith('PromptManager')).map(node => [node.id, node.inputs.filter(input => ['text', 'negative_text'].includes(input.name) && input.widget).map(input => input.name)])),
@@ -377,8 +378,16 @@ test('workflows saved before #343 load unchanged and their linked text keeps wor
}, workflow);
for (const node of workflow.nodes) {
// Every saved widget value comes back as it was saved, except the typed text main lost under a link (null).
if (node.widgets_values?.[0] !== null) expect(result.values[node.id], `${node.type} ${node.id}`).toEqual(node.widgets_values);
// #90: the last saved value is the editor's own slot (''), where the Save to library switch is now: it loads on.
if (node.widgets_values?.[0] === null) continue;
if (!node.type.startsWith('PromptManager')) { expect(result.values[node.id], `${node.type} ${node.id}`).toEqual(node.widgets_values); continue; }
const saved = node.widgets_values.length - 1;
expect(result.values[node.id].slice(0, saved), `${node.type} ${node.id}`).toEqual(node.widgets_values.slice(0, saved));
expect(result.values[node.id][saved], `${node.type} ${node.id} save_to_library`).toBe(true);
}
// Every PromptManager node of the old workflow queues with Save to library on.
for (const save of Object.values(result.saves)) expect(save).toBe(true);
expect(Object.keys(result.saves).length).toBeGreaterThan(0);
expect(result.links).toMatchObject({
1: { text: 'Saved PromptManager text' }, 2: { text: 'Saved PromptManagerText text' }, 3: { text: 'Saved PromptManagerFocused text' },
4: { text: 'Saved PromptManagerSplit text' }, 5: { text: 'Saved PromptManagerPair text', negative_text: 'Saved pair negative' },
+7
View File
@@ -1,5 +1,7 @@
{
"routes": [
"DELETE /prompt_manager/collections/items",
"DELETE /prompt_manager/collections/{collection_id}",
"DELETE /prompt_manager/delete/{prompt_id}",
"DELETE /prompt_manager/image-annotations/capture/{token}",
"DELETE /prompt_manager/images/category",
@@ -15,6 +17,8 @@
"GET /prompt_manager/backup",
"GET /prompt_manager/backup/status",
"GET /prompt_manager/categories",
"GET /prompt_manager/collections",
"GET /prompt_manager/collections/membership",
"GET /prompt_manager/diagnostics",
"GET /prompt_manager/export",
"GET /prompt_manager/gallery",
@@ -92,6 +96,7 @@
"GET /prompt_manager/vocabulary/tags/{tag_id}/impact",
"GET /prompt_manager/web",
"GET /prompt_manager/workspace",
"PATCH /prompt_manager/collections/{collection_id}",
"PATCH /prompt_manager/library/{positive_id}",
"PATCH /prompt_manager/pairs/{pair_id}",
"PATCH /prompt_manager/saved-views/{view_id}",
@@ -106,6 +111,7 @@
"POST /prompt_manager/bulk/delete",
"POST /prompt_manager/bulk/tags",
"POST /prompt_manager/cleanup",
"POST /prompt_manager/collections/items",
"POST /prompt_manager/delete_duplicate_images",
"POST /prompt_manager/diagnostics/test-link",
"POST /prompt_manager/gallery/rescan",
@@ -115,6 +121,7 @@
"POST /prompt_manager/images/generate-thumbnails",
"POST /prompt_manager/images/generate-thumbnails/progress",
"POST /prompt_manager/images/link",
"POST /prompt_manager/images/rescope",
"POST /prompt_manager/images/reveal",
"POST /prompt_manager/jobs/autotag",
"POST /prompt_manager/jobs/duplicates",
+68
View File
@@ -0,0 +1,68 @@
// #90: the node's save_to_library switch (web/js/save-to-library.js).
const test = require('node:test');
const assert = require('node:assert/strict');
const source = import('../../web/js/save-to-library.js');
function classicNode(saved) {
const names = ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text', 'save_to_library', 'prompt_manager_search_ui'];
const widgets = names.map(name => ({name, value: name === 'save_to_library' ? true : ''}));
const node = {widgets, widgets_values: saved ? [...saved] : undefined};
// What a host configure does: each saved value goes to the widget at its index.
(saved || []).forEach((value, index) => { if (widgets[index]) widgets[index].value = value; });
return node;
}
test('an older workflow stored the editor slot where the switch is now: it loads on', async () => {
const {normalizeSaveWidget, savesToLibrary} = await source;
const node = classicNode(['a cat', 'c', 't', '', '', '', '']);
assert.equal(node.widgets[6].value, '', 'the host put the old DOM slot value into the switch');
assert.equal(normalizeSaveWidget(node), true);
assert.equal(node.widgets[6].value, true);
assert.equal(node.widgets_values[6], true);
assert.equal(savesToLibrary(node), true);
assert.deepEqual(node.widgets_values.slice(0, 6), ['a cat', 'c', 't', '', '', ''], 'no earlier value moved');
});
test('null, undefined and missing slots load on; saved booleans are kept', async () => {
const {normalizeSaveWidget, savesToLibrary} = await source;
for (const value of [null, undefined, 0, 'false']) {
const node = classicNode(['a', '', '', '', '', '', value]);
normalizeSaveWidget(node);
assert.equal(savesToLibrary(node), true, String(value));
}
const short = classicNode(['a', '', '', '', '', '']);
assert.equal(normalizeSaveWidget(short), false);
assert.equal(savesToLibrary(short), true);
const off = classicNode(['a', '', '', '', '', '', false]);
assert.equal(normalizeSaveWidget(off), false);
assert.equal(savesToLibrary(off), false);
});
test('a node without the widget (an older backend) saves', async () => {
const {normalizeSaveWidget, savesToLibrary} = await source;
assert.equal(savesToLibrary({widgets: [{name: 'text', value: ''}]}), true);
assert.equal(normalizeSaveWidget({widgets: []}), false);
assert.equal(savesToLibrary(undefined), true);
});
test('the switch moves after the editor fallbacks so every frontend has one order', async () => {
const {placeSaveWidgetLast} = await source;
const widgets = ['text', 'category', 'tags', 'search_text', 'save_to_library', 'prepend_text', 'append_text'].map(name => ({name}));
const node = {widgets};
assert.equal(placeSaveWidgetLast(node), true);
assert.deepEqual(node.widgets.map(widget => widget.name), ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text', 'save_to_library']);
assert.equal(placeSaveWidgetLast(node), false, 'already last');
assert.equal(placeSaveWidgetLast({widgets: [{name: 'text'}]}), false);
});
test('setting the switch updates the widget, its saved value and notifies the host', async () => {
const {setSaveToLibrary, savesToLibrary} = await source;
const calls = [];
const node = classicNode(['a', '', '', '', '', '', true]);
node.widgets[6].callback = value => calls.push(value);
setSaveToLibrary(node, false);
assert.equal(savesToLibrary(node), false);
assert.equal(node.widgets_values[6], false);
assert.deepEqual(calls, [false]);
setSaveToLibrary({widgets: []}, false);
});
+313
View File
@@ -0,0 +1,313 @@
// Collections (#331 D136–D140): the client store, the route hash, the names, every user-facing string and the
// mapping of the routes' error codes.
const test = require('node:test');
const assert = require('node:assert/strict');
const source = import('../../web/js/workspace/collections.mjs');
const FILE = Object.freeze({ root_index: 0, root_scope: 'a'.repeat(64), path: 'grids/one.png' });
function apiError(status, code) {
return Object.assign(new Error(code || `HTTP ${status}`), { status, code: code || null });
}
function fakeApi() {
const calls = [];
const pending = [];
const api = {
calls,
get(url, options) {
calls.push(['GET', url, options?.cache]);
if (url.includes('/membership?')) return api.membershipResult ?? Promise.resolve({ success: true, collection_ids: [2] });
return new Promise((resolve, reject) => pending.push({ resolve, reject }));
},
post(url, body) {
calls.push(['POST', url, body]);
return api.postResult ?? Promise.resolve({ success: true, collection: { id: 5, name: 'Grids' }, created: true,
added: body.items.length, already: 0, not_found: 0 });
},
patch(url, body) {
calls.push(['PATCH', url, body]);
return api.patchResult ?? Promise.resolve({ success: true, collection: { id: 2, name: body.name } });
},
del(url, options) {
calls.push(['DELETE', url, options?.body]);
return api.deleteResult ?? Promise.resolve(options?.body
? { success: true, collection: { id: options.body.collection_id, name: 'Portraits' }, removed: options.body.items.length }
: { success: true, deleted: true });
},
};
return { api, pending };
}
const LISTED = { success: true, collections: [
{ id: 4, name: 'Experiments', count: 8, missing: 0 },
{ id: 2, name: 'Landscapes', count: 12, missing: 1 },
] };
/** A store whose first read answered with `data`; the test's own timer seam replaces the 300 ms count timer. */
async function readyStore(data = LISTED) {
const { createCollections } = await source;
const { api, pending } = fakeApi();
const timers = [];
const controller = new AbortController();
const store = createCollections(api, controller.signal, {
setTimer: (run, delay) => { timers.push({ run, delay, cleared: false }); return timers.length; },
clearTimer: id => { if (timers[id - 1]) timers[id - 1].cleared = true; },
});
store.load();
pending.shift().resolve(data);
await store.settled();
return { store, api, pending, timers, controller };
}
test('the route hash, the name rule and the limits', async () => {
const { collectionHash, normalizeCollectionName, MAX_COLLECTION_NAME_LENGTH, MAX_COLLECTION_ITEMS,
COLLECTION_COUNT_DELAY_MS } = await source;
assert.equal(collectionHash(12), '#/collection/12');
assert.throws(() => collectionHash(0), TypeError);
assert.throws(() => collectionHash('3'), TypeError);
assert.equal(MAX_COLLECTION_NAME_LENGTH, 100);
assert.equal(MAX_COLLECTION_ITEMS, 500);
assert.equal(COLLECTION_COUNT_DELAY_MS, 300);
// Trimmed, each run of whitespace one space, 1 to 100 characters, no control characters (design §1).
assert.equal(normalizeCollectionName(' LoRa testing   grids '), 'LoRa testing grids');
assert.equal(normalizeCollectionName('x'.repeat(100)), 'x'.repeat(100));
for (const bad of ['', ' ', 'x'.repeat(101), 'bad\u0007name', 7, null, undefined]) {
assert.equal(normalizeCollectionName(bad), null, String(bad));
}
// A tab or newline is whitespace, collapsed rather than refused.
assert.equal(normalizeCollectionName('a\tb\nc'), 'a b c');
});
test('the listing keeps usable rows in the server order, with counts only when they are counts', async () => {
const { parseCollections, parseCollection } = await source;
assert.deepEqual(parseCollections({ success: true, collections: [
{ id: 3, name: 'b', count: 2, missing: 0 },
{ id: 0, name: 'bad id' },
{ id: 1, name: ' ', count: 1, missing: 0 },
{ id: 7, name: 'A', count: null, missing: null },
{ id: 9, name: 'c', count: -1, missing: 1.5 },
] }), [
{ id: 3, name: 'b', count: 2, missing: 0 },
{ id: 7, name: 'A', count: null, missing: null },
{ id: 9, name: 'c', count: null, missing: null },
]);
assert.equal(parseCollection(null), null);
assert.throws(() => parseCollections({ success: true }), TypeError);
});
test('every user-facing string of the design (§7.4, §8)', async () => {
const c = await source;
assert.equal(c.addedMessage({ collection: { name: 'X' }, added: 1, already: 0, notFound: 0 }), 'Added 1 image to “X”.');
assert.equal(c.addedMessage({ collection: { name: 'X' }, added: 3, already: 0, notFound: 0 }), 'Added 3 images to “X”.');
assert.equal(c.addedMessage({ collection: { name: 'X' }, added: 1, already: 1, notFound: 0 }),
'Added 1 image to “X”. 1 was already in it.');
assert.equal(c.addedMessage({ collection: { name: 'X' }, added: 2, already: 3, notFound: 1 }),
'Added 2 images to “X”. 3 were already in it. 1 could not be found.');
assert.equal(c.addedMessage({ collection: { name: 'X' }, added: 1, already: 0, notFound: 2 }),
'Added 1 image to “X”. 2 could not be found.');
assert.equal(c.addedMessage({ collection: { name: 'X' }, added: 0, already: 4, notFound: 0 }),
'All selected images were already in “X”.');
assert.equal(c.FILES_NOT_FOUND_TOAST, 'None of the selected images could be found. Refresh the images and try again.');
assert.equal(c.SOURCE_CHANGED_TOAST, 'The sources changed. Refresh the images and try again.');
assert.equal(c.ADD_FAILED_TOAST, 'The images could not be added. Try again.');
assert.equal(c.removedMessage('X', 1), 'Removed 1 image from “X”.');
assert.equal(c.removedMessage('X', 4), 'Removed 4 images from “X”.');
assert.equal(c.REMOVE_FAILED_TOAST, 'The images could not be removed. Try again.');
assert.equal(c.COLLECTION_GONE_TOAST, 'That collection no longer exists.');
assert.deepEqual([c.COLLECTION_NOUNS.title, c.COLLECTION_NOUNS.label, c.COLLECTION_NOUNS.deleteTitle],
['Edit collection', 'Collection name', 'Delete collection']);
assert.equal(c.COLLECTION_NOUNS.renamed, 'Collection renamed.');
assert.equal(c.RENAMED_TOAST, 'Collection renamed.');
assert.equal(c.nameTakenMessage('LoRa'), 'A collection named “LoRa” already exists.');
assert.equal(c.RENAME_FAILED_TOAST, 'The collection could not be renamed. Try again.');
// The stored count: listed members plus missing ones.
assert.equal(c.deleteQuestion({ name: 'X', count: 1, missing: 0 }),
'Delete “X”? Its 1 image stays where it is; only the collection is removed.');
assert.equal(c.deleteQuestion({ name: 'X', count: 3, missing: 1 }),
'Delete “X”? Its 4 images stay where they are; only the collection is removed.');
assert.equal(c.deleteQuestion({ name: 'X', count: null, missing: null }),
'Delete “X”? Its images stay where they are; only the collection is removed.');
assert.equal(c.COLLECTION_NOUNS.deleteText({ name: 'X', count: 2, missing: 0 }), c.deleteQuestion({ name: 'X', count: 2, missing: 0 }));
assert.equal(c.DELETED_TOAST, 'Collection deleted.');
assert.equal(c.DELETE_FAILED_TOAST, 'The collection could not be deleted. Try again.');
assert.equal(c.DELETED_PAGE_TEXT, 'This collection no longer exists.');
assert.equal(c.OPEN_IMAGES_LINK, 'Open Images');
assert.equal(c.VIEW_COLLECTION_DELETED, 'The collection this view uses was deleted.');
assert.equal(c.LIST_FAILED_TEXT, 'Collections could not be loaded. Your saved data has not changed.');
assert.equal(c.missingNote(1), '1 image in this collection is not in your current sources.');
assert.equal(c.missingNote(5), '5 images in this collection are not in your current sources.');
for (const none of [0, null, undefined, -1, 1.5]) assert.equal(c.missingNote(none), '');
assert.equal(c.TOO_MANY_TOAST, 'Select at most 500 images.');
assert.deepEqual([c.ADD_DIALOG_TITLE, c.ADD_DIALOG_LABEL, c.ADD_DIALOG_ACTION],
['Add selected images to collection', 'Collection name', 'Add']);
assert.equal(c.COLLECTIONS_UPDATED_TOAST, 'Collections updated.');
assert.equal(c.UPDATE_FAILED_TEXT, 'The collections could not be updated. Try again.');
assert.equal(c.MEMBERSHIP_LOADING_TEXT, 'Loading collections…');
assert.equal(c.MEMBERSHIP_FAILED_TEXT, 'Collections could not be loaded.');
assert.equal(c.NO_COLLECTIONS_TEXT, 'No collections yet. Name one below.');
});
test('a write failure maps the routes\' error codes to their strings', async () => {
const { writeFailure, renameFailure, isCollectionGone, COLLECTION_GONE_TOAST, SOURCE_CHANGED_TOAST,
FILES_NOT_FOUND_TOAST, ADD_FAILED_TOAST, RENAME_FAILED_TOAST } = await source;
assert.equal(writeFailure(apiError(404, 'collection-not-found'), ADD_FAILED_TOAST), COLLECTION_GONE_TOAST);
assert.equal(writeFailure(apiError(409, 'source-changed'), ADD_FAILED_TOAST), SOURCE_CHANGED_TOAST);
assert.equal(writeFailure(apiError(404, 'files-not-found'), ADD_FAILED_TOAST), FILES_NOT_FOUND_TOAST);
for (const other of [apiError(500), apiError(400), apiError(404), new TypeError('x'), null]) {
assert.equal(writeFailure(other, ADD_FAILED_TOAST), ADD_FAILED_TOAST);
}
assert.equal(isCollectionGone(apiError(404, 'collection-not-found')), true);
assert.equal(isCollectionGone(apiError(404)), false);
// The Edit dialog: a taken name stays in the dialog; a gone collection closes it; anything else keeps it open.
assert.deepEqual(renameFailure('LoRa')(apiError(409, 'name-taken')), { field: 'A collection named “LoRa” already exists.' });
assert.deepEqual(renameFailure('LoRa')(apiError(404, 'collection-not-found')), { toast: COLLECTION_GONE_TOAST, close: true });
assert.deepEqual(renameFailure('LoRa')(apiError(500)), { toast: RENAME_FAILED_TOAST });
});
test('the store reports loads, keeps the last list after a failed read, and a newer read supersedes', async () => {
const { createCollections, COLLECTIONS_URL } = await source;
const { api, pending } = fakeApi();
const store = createCollections(api);
assert.equal(store.status, 'loading');
const seen = [];
store.subscribe(next => seen.push([next.status, next.collections.map(item => item.id)]));
store.load();
const settled = store.settled();
pending[0].resolve(LISTED);
assert.equal((await settled).status, 'ready');
assert.deepEqual(api.calls[0], ['GET', COLLECTIONS_URL, 'no-store']);
assert.deepEqual(store.find(2), { id: 2, name: 'Landscapes', count: 12, missing: 1 });
assert.equal(store.find(3), null);
assert.deepEqual(store.collections.map(item => item.name), ['Experiments', 'Landscapes']);
store.load();
pending[1].reject(new Error('offline'));
await store.settled();
assert.equal(store.status, 'error');
assert.deepEqual(store.collections.map(item => item.id), [4, 2]);
store.load();
store.load();
pending[3].resolve({ success: true, collections: [{ id: 8, name: 'New', count: 0, missing: 0 }] });
pending[2].resolve(LISTED);
await store.settled();
assert.deepEqual(store.collections.map(item => item.id), [8]);
assert.deepEqual(seen.at(-1), ['ready', [8]]);
// A malformed listing is a failed read.
store.load();
pending[4].resolve({ success: true });
await store.settled();
assert.equal(store.status, 'error');
});
test('adding by name or id sends only the file identity and re-reads the counts 300 ms after the last change', async () => {
const { store, api, pending, timers } = await readyStore();
const { COLLECTION_ITEMS_URL } = await source;
const result = await store.add({ name: ' Grids ' }, [{ ...FILE, title: 'ignored', favorite: true }]);
assert.deepEqual(api.calls.at(-1), ['POST', COLLECTION_ITEMS_URL, { name: 'Grids', items: [FILE] }]);
assert.deepEqual(result, { collection: { id: 5, name: 'Grids' }, created: true, added: 1, already: 0, notFound: 0 });
// The new collection is listed at once (its count arrives with the re-read), in name order.
assert.deepEqual(store.collections.map(item => [item.id, item.count]), [[4, 8], [5, null], [2, 12]]);
api.postResult = Promise.resolve({ success: true, collection: { id: 2, name: 'Landscapes' }, created: false, added: 0,
already: 1, not_found: 1 });
const again = await store.add({ collectionId: 2 }, [FILE, { ...FILE, path: 'b.png' }]);
assert.deepEqual(api.calls.at(-1)[2], { collection_id: 2, items: [FILE, { ...FILE, path: 'b.png' }] });
assert.deepEqual(again, { collection: { id: 2, name: 'Landscapes' }, created: false, added: 0, already: 1, notFound: 1 });
// One trailing re-read for the run of writes.
assert.deepEqual(timers.map(timer => [timer.delay, timer.cleared]), [[300, true], [300, false]]);
const reads = api.calls.filter(call => call[0] === 'GET').length;
timers.at(-1).run();
assert.equal(api.calls.filter(call => call[0] === 'GET').length, reads + 1);
pending.shift().resolve(LISTED);
await store.settled();
});
test('a write is refused before sending when its target or files are not valid', async () => {
const { store, api } = await readyStore();
const sent = api.calls.length;
const many = Array.from({ length: 501 }, (_, index) => ({ ...FILE, path: `${index}.png` }));
for (const [target, items] of [
[{ name: '' }, [FILE]], [{ name: 'x'.repeat(101) }, [FILE]], [{ collectionId: 0 }, [FILE]], [{}, [FILE]],
[{ name: 'a', collectionId: 2 }, [FILE]], [{ collectionId: 2 }, []], [{ collectionId: 2 }, many],
[{ collectionId: 2 }, [{ ...FILE, root_index: -1 }]], [{ collectionId: 2 }, [{ ...FILE, path: '' }]],
[{ collectionId: 2 }, [{ ...FILE, root_scope: 3 }]], [{ collectionId: 2 }, 'a.png'],
]) {
await assert.rejects(store.add(target, items), TypeError);
}
await assert.rejects(store.remove(0, [FILE]), TypeError);
await assert.rejects(store.remove(2, []), TypeError);
await assert.rejects(store.rename(2, ' '), TypeError);
await assert.rejects(store.delete(-1), TypeError);
await assert.rejects(store.membership({ root_index: 'x', root_scope: 'a', path: 'p' }), TypeError);
assert.equal(api.calls.length, sent);
});
test('a write to a collection that is gone re-reads the list at once and rejects with the error', async () => {
const { store, api, pending } = await readyStore();
const { isCollectionGone } = await source;
api.postResult = Promise.reject(apiError(404, 'collection-not-found'));
const reads = api.calls.filter(call => call[0] === 'GET').length;
await assert.rejects(store.add({ collectionId: 2 }, [FILE]), error => isCollectionGone(error));
assert.equal(api.calls.filter(call => call[0] === 'GET').length, reads + 1);
pending.shift().resolve({ success: true, collections: [LISTED.collections[0]] });
await store.settled();
assert.equal(store.find(2), null);
// Another failure re-reads nothing.
api.deleteResult = Promise.reject(apiError(500));
await assert.rejects(store.remove(4, [FILE]), error => error.status === 500);
assert.equal(api.calls.filter(call => call[0] === 'GET').length, reads + 1);
});
test('remove sends the collection and files; membership reads one file\'s collections', async () => {
const { store, api, timers } = await readyStore();
const { COLLECTION_ITEMS_URL, COLLECTION_MEMBERSHIP_URL } = await source;
assert.deepEqual(await store.remove(2, [FILE]), { collection: { id: 2, name: 'Portraits' }, removed: 1 });
assert.deepEqual(api.calls.at(-1), ['DELETE', COLLECTION_ITEMS_URL, { collection_id: 2, items: [FILE] }]);
assert.equal(timers.length, 1);
assert.deepEqual(await store.membership(FILE), [2]);
const [method, url] = api.calls.at(-1);
assert.equal(method, 'GET');
const query = new URL(url, 'http://local');
assert.equal(query.pathname, COLLECTION_MEMBERSHIP_URL);
assert.deepEqual(Object.fromEntries(query.searchParams), { root_index: '0', root_scope: FILE.root_scope, path: FILE.path });
api.membershipResult = Promise.resolve({ success: true, collection_ids: [1, 'x'] });
await assert.rejects(store.membership(FILE), TypeError);
});
test('rename and delete change the listed row at once and re-read the counts', async () => {
const { store, api, timers } = await readyStore();
const { COLLECTIONS_URL } = await source;
const renamed = await store.rename(2, ' A grids ');
assert.deepEqual(api.calls.at(-1), ['PATCH', `${COLLECTIONS_URL}/2`, { name: 'A grids' }]);
assert.deepEqual(renamed, { id: 2, name: 'A grids' });
// The row keeps its counts and moves to its new place in name order.
assert.deepEqual(store.collections.map(item => [item.id, item.name, item.count]), [[2, 'A grids', 12], [4, 'Experiments', 8]]);
assert.equal(await store.delete(4), 4);
assert.deepEqual(api.calls.at(-1), ['DELETE', `${COLLECTIONS_URL}/4`, undefined]);
assert.deepEqual(store.collections.map(item => item.id), [2]);
// Already gone (another tab): the outcome the user asked for.
api.deleteResult = Promise.reject(apiError(404, 'collection-not-found'));
assert.equal(await store.delete(2), 2);
assert.deepEqual(store.collections, []);
api.patchResult = Promise.reject(apiError(409, 'name-taken'));
await assert.rejects(store.rename(9, 'Taken'), error => error.code === 'name-taken');
assert.equal(timers.filter(timer => !timer.cleared).length, 1);
});
test('a disposed store stops its count timer, drops a read in flight and reads no more', async () => {
const { store, timers, controller, api, pending } = await readyStore();
await store.rename(2, 'Kept');
const seen = [];
store.subscribe(next => seen.push(next));
store.load();
controller.abort();
assert.equal(timers.at(-1).cleared, true);
pending.shift().resolve(LISTED);
await new Promise(resolve => setImmediate(resolve));
assert.equal(seen.filter(next => next.status === 'ready').length, 0);
const sent = api.calls.length;
await store.load();
assert.equal(api.calls.length, sent);
});
+29
View File
@@ -383,3 +383,32 @@ test('the D117 wrap rule targets only the settings summary button', async () =>
assert.deepEqual(rules.map(line => line.trim()), ['.ca-tools #viewer-details .settings-summary { white-space: normal; /* D117 */ }']);
assert.doesNotMatch(css, /#viewer-details > div > button/);
});
test('category chip names its origin visibly and shows an own category for any link status (#332 D146)', async () => {
const base = await localRecord(localPng('a.png'));
const chipTexts = record => {
const {f, done} = renderFull(record);
try {
return f.walk(f.container, n => /(^| )chip( |$)/.test(n.className ?? '')).map(n => n.textContent);
} finally { done(); }
};
const cases = [
[{status:'none', category:'Scans', category_scope:'image'}, 'Image category: Scans · this image'],
[{status:'ambiguous', category:'Scans', category_scope:'image'}, 'Image category: Scans · this image'],
[{status:'linked', category:'Portraits', category_scope:'image'}, 'Image category: Portraits · this image'],
[{status:'linked', category:'Landscapes', category_scope:'positive'}, 'Positive prompt category: Landscapes · prompt'],
[{status:'linked', category:'Recipes', category_scope:'pair'}, 'Recipe category: Recipes · recipe'],
];
for (const [library, text] of cases) {
const chips = chipTexts({...base, library:{...base.library, tags:[], tags_scope:null, ...library}});
assert.ok(chips.includes(text), `${JSON.stringify(library)} -> ${chips}`);
}
for (const library of [
{status:'none', category:'Inherited', category_scope:'positive'},
{status:'linked', category:null, category_scope:null},
{status:'none', category:null, category_scope:null},
]) {
const chips = chipTexts({...base, library:{...base.library, tags:[], tags_scope:null, ...library}});
assert.equal(chips.some(chip => /category: /.test(chip)), false, JSON.stringify(library));
}
});
+66 -3
View File
@@ -253,15 +253,16 @@ test('an unreadable or malformed models read falls back to the documented defaul
test('observe failures and malformed answers end in observe-error and retry observes again', async () => {
const { createImageTaggingController } = await load();
let answer = () => fail(409, 'source-changed', 'The source changed. Refresh the image collection before tagging this file.');
let answer = () => fail(500, 'observe-failed');
const h = harness(defaultRoutes({ [`POST ${OBSERVE}`]: () => answer() }));
const c = createImageTaggingController({ api: h.api, target: TARGET });
await c.open();
assert.equal(c.state.phase, 'observe-error');
assert.equal(c.state.statusText, 'The source changed. Refresh the image collection before tagging this file.');
assert.equal(c.state.statusText, 'The image could not be opened for tagging.');
assert.equal(c.state.canSave, false);
assert.equal(c.state.canGenerate, false);
assert.equal(c.state.canRetry, true);
assert.equal(c.state.blocked, true, 'a failed capture blocks Generate and Save with its reason');
answer = () => response(200, observed({ tag_revision: -1 }));
await c.retry();
assert.equal(c.state.phase, 'observe-error');
@@ -274,9 +275,71 @@ test('observe failures and malformed answers end in observe-error and retry obse
await c.retry();
await settle();
assert.equal(c.state.phase, 'ready');
assert.equal(c.state.blocked, false);
assert.equal(h.of('POST', OBSERVE).length, 4);
});
test('#430: a changed source blocks the dialog until it is retargeted, keeping the draft', async () => {
const { createImageTaggingController } = await load();
const SOURCE = 'The source changed. Refresh the image collection before tagging this file.';
const fresh = { root_index: 1, root_scope: 'e'.repeat(64), path: TARGET.path };
const h = harness(defaultRoutes({
[`POST ${OBSERVE}`]: call => (call.body.target.root_scope === SCOPE ? fail(409, 'source-changed', SOURCE) : response(200, observed())),
}));
const c = createImageTaggingController({ api: h.api, target: TARGET });
await c.open();
assert.equal(c.state.phase, 'source-changed');
assert.equal(c.state.statusText, SOURCE);
assert.deepEqual([c.state.canSave, c.state.canGenerate, c.state.canRetry, c.state.blocked], [false, false, false, true]);
assert.equal(await c.retry(), false, 'the same stale target is never observed again');
assert.equal(h.of('POST', OBSERVE).length, 1);
assert.throws(() => c.retarget({ ...fresh, root_scope: 'bad' }), TypeError);
await c.retarget(fresh);
await settle();
assert.equal(c.state.phase, 'ready');
assert.deepEqual(h.of('POST', OBSERVE).at(-1).body.target, fresh);
assert.deepEqual({ ...c.target }, fresh);
// Generate and Save now carry the new target.
c.setManual('kept draft');
await c.generate();
assert.deepEqual(h.of('POST', SUGGEST)[0].body.target, fresh);
});
test('#430: a source change answered to Generate or Save also blocks; retarget keeps the typed draft', async () => {
for (const route of [SUGGEST, TAGS]) {
const method = route === TAGS ? 'PUT' : 'POST';
let stale = true;
const { c, h } = await opened({
[`${method} ${route}`]: call => (stale ? fail(409, 'source-changed', 'server says source') :
route === TAGS ? response(200, receipt(call.body.tags)) : response(200, suggested())),
});
c.setManual('kept draft');
if (route === TAGS) await c.save(); else await c.generate();
assert.equal(c.state.phase, 'source-changed');
assert.equal(c.state.statusText, 'server says source');
stale = false;
await c.retarget({ root_index: 2, root_scope: 'f'.repeat(64), path: TARGET.path });
await settle();
assert.equal(c.state.phase, 'ready');
assert.equal(c.state.manual, 'kept draft');
assert.equal(h.of('DELETE', CAPTURE).length, 1, 'the old capture is released');
}
});
test('#430: retarget waits for nothing: it refuses while a save is in flight and after close', async () => {
const gate = deferred();
const { c, h } = await opened({ [`PUT ${TAGS}`]: () => gate.promise });
const saving = c.save();
await flush();
assert.equal(c.state.phase, 'saving');
assert.equal(await c.retarget({ ...TARGET, root_index: 3 }), false);
gate.resolve(response(200, receipt(['Minimal'])));
await saving;
c.cancel();
assert.equal(await c.retarget({ ...TARGET, root_index: 3 }), false);
assert.equal(h.of('POST', OBSERVE).length, 1);
});
test('model change clears suggestions count and status but keeps manual', async () => {
const { c } = await opened();
c.setManual('Minimal, dune');
@@ -547,7 +610,7 @@ test('write errors keep the draft and allow another save', async () => {
});
test('storage-changed and file-changed disable save until reopen', async () => {
for (const [code, phase] of [['storage-changed', 'storage-changed'], ['file-changed', 'file-changed'], ['source-changed', 'file-changed'], ['capture-expired', 'capture-expired']]) {
for (const [code, phase] of [['storage-changed', 'storage-changed'], ['file-changed', 'file-changed'], ['source-changed', 'source-changed'], ['capture-expired', 'capture-expired']]) {
const message = `server says ${code}`;
let answer = () => fail(409, code, message);
const { c, h } = await opened({ [`PUT ${TAGS}`]: () => answer() });
+50 -42
View File
@@ -1,53 +1,61 @@
const test = require('node:test');
const assert = require('node:assert/strict');
// #332 D140: Set category writes each selected image's own category; Use prompt category removes it.
const source = import('../../web/js/workspace/images-category.mjs');
const linked = (id, extra = {}) => ({path:`p${id}.png`, linked:{status:'linked', positive_id:id, ...extra}});
test('targets are the distinct linked positives, in selection order, and the rest count as skipped', async () => {
const {categoryTargets} = await source;
const images = [
linked(7), linked(3), linked(7),
{path:'u.png', linked:{status:'unlinked', positive_id:null}},
{path:'a.png', linked:{status:'ambiguous', positive_id:4}},
{path:'v.png', linked:{status:'unavailable', positive_id:null}},
{path:'none.png'},
{path:'bad.png', linked:{status:'linked', positive_id:'9'}},
{path:'zero.png', linked:{status:'linked', positive_id:0}},
{path:'frac.png', linked:{status:'linked', positive_id:1.5}},
null,
];
assert.deepEqual(categoryTargets(images), {promptIds:[7, 3], skipped:8});
assert.deepEqual(categoryTargets([]), {promptIds:[], skipped:0});
assert.deepEqual(categoryTargets([linked(2)]), {promptIds:[2], skipped:0});
});
test('the outcome toast counts the prompts and the skipped images', async () => {
const {categoryOutcome} = await source;
assert.equal(categoryOutcome(3, 0), 'Category set on 3 prompts.');
assert.equal(categoryOutcome(1, 0), 'Category set on 1 prompt.');
assert.equal(categoryOutcome(2, 4), 'Category set on 2 prompts. 4 selected images had no linked prompt and were skipped.');
assert.equal(categoryOutcome(1, 1), 'Category set on 1 prompt. 1 selected image had no linked prompt and was skipped.');
});
test('the dialog and failure copy is the plan\'s', async () => {
const {CATEGORY_DIALOG, CATEGORY_NONE_LINKED, CATEGORY_FAILED} = await source;
assert.deepEqual(CATEGORY_DIALOG, {title:'Categorize selected images', label:'Category', action:'Apply'});
assert.equal(CATEGORY_NONE_LINKED, "None of the selected images is linked to a saved prompt, so there is no prompt category to set.");
test('the dialog keeps the concept\'s simpleInput words and names the routes', async () => {
const {CATEGORY_DIALOG, USE_PROMPT_CATEGORY, REMOVE_DIALOG, CATEGORY_URL, IMAGE_CATEGORIES_URL, CATEGORY_FAILED,
CATEGORY_MAX_LENGTH, MAX_CATEGORY_IMAGES} = await source;
assert.deepEqual({...CATEGORY_DIALOG}, {title:'Categorize selected images', label:'Category', action:'Apply'});
assert.equal(USE_PROMPT_CATEGORY, 'Use prompt category');
assert.deepEqual({...REMOVE_DIALOG}, {title:'Use prompt category', action:'Remove'});
assert.equal(CATEGORY_URL, '/prompt_manager/images/category');
assert.equal(IMAGE_CATEGORIES_URL, '/prompt_manager/images/categories');
assert.equal(CATEGORY_FAILED, 'Category could not be set. Try again.');
assert.ok(Object.isFrozen(CATEGORY_DIALOG));
assert.equal(CATEGORY_MAX_LENGTH, 100);
assert.equal(MAX_CATEGORY_IMAGES, 500);
});
test('the updated count is the server\'s when it is a count, else the number of prompts posted', async () => {
const {updatedCount} = await source;
assert.equal(updatedCount({updated_count:0}, 3), 0);
assert.equal(updatedCount({updated_count:2}, 3), 2);
for (const result of [{}, {updated_count:-1}, {updated_count:1.5}, {updated_count:'2'}, null, undefined]) {
assert.equal(updatedCount(result, 3), 3, JSON.stringify(result));
test('a set counts unchanged files as set and names files that could not be found', async () => {
const {setOutcome} = await source;
assert.equal(setOutcome({updated:1, unchanged:0, not_found:0}, 'Portraits'), 'Category “Portraits” set on 1 image.');
assert.equal(setOutcome({updated:1, unchanged:2, not_found:0}, 'Scans'), 'Category “Scans” set on 3 images.');
assert.equal(setOutcome({updated:2, unchanged:0, not_found:1}, 'Scans'), 'Category “Scans” set on 2 images. 1 could not be found.');
assert.equal(setOutcome({updated:0, unchanged:1, not_found:3}, 'Scans'), 'Category “Scans” set on 1 image. 3 could not be found.');
assert.equal(setOutcome({}, 'Scans'), 'Category “Scans” set on 0 images.');
});
test('a removal says how many lost their own category, or that none had one', async () => {
const {clearOutcome, NONE_HAD_OWN} = await source;
assert.equal(clearOutcome({cleared:1}), 'Removed the image category from 1 image.');
assert.equal(clearOutcome({cleared:4}), 'Removed the image category from 4 images.');
assert.equal(clearOutcome({cleared:0}), NONE_HAD_OWN);
assert.equal(NONE_HAD_OWN, 'None of the selected images had its own category.');
});
test('the confirmation states the count', async () => {
const {removeQuestion} = await source;
assert.equal(removeQuestion(1), "Remove the image category from the 1 selected image? Each one then shows its prompt's " +
'category, or none if it has no linked prompt. This cannot be undone.');
assert.match(removeQuestion(1200), /^Remove the image category from the 1,200 selected images\? /);
});
test('coded failures have their own sentences, anything else the fixed one', async () => {
const {failureMessage, NONE_FOUND, SOURCES_CHANGED, CATEGORY_FAILED} = await source;
assert.equal(failureMessage({code:'files-not-found'}), NONE_FOUND);
assert.equal(NONE_FOUND, 'None of the selected images could be found. Refresh the images and try again.');
assert.equal(failureMessage({code:'source-changed'}), SOURCES_CHANGED);
assert.equal(SOURCES_CHANGED, 'The sources changed. Refresh the images and try again.');
assert.equal(failureMessage({code:'too-many-images', message:'Select at most 500 images.'}), 'Select at most 500 images.');
for (const error of [null, undefined, new Error('x'), {code:'other', message:'secret'}]) {
assert.equal(failureMessage(error), CATEGORY_FAILED);
}
});
test('the category field is limited to the server\'s 100 characters', async () => {
const {CATEGORY_MAX_LENGTH} = await source;
assert.equal(CATEGORY_MAX_LENGTH, 100);
test('the toolbar list merges prompt and image categories once each', async () => {
const {mergeCategories} = await source;
assert.deepEqual(mergeCategories(['Landscapes', 'Portraits'], ['Scans', 'Landscapes']), ['Landscapes', 'Portraits', 'Scans']);
assert.deepEqual(mergeCategories(['A', '', ' ', 7], null), ['A']);
assert.deepEqual(mergeCategories(undefined, ['landscapes', 'Landscapes']), ['landscapes', 'Landscapes']);
});
@@ -0,0 +1,172 @@
// Run with: node --test tests/js/workspace-images-source-recovery.test.js
const test = require('node:test');
const assert = require('node:assert/strict');
const load = () => import('../../web/js/workspace/images-source-recovery.mjs');
const OLD = Object.freeze({ root_index: 0, root_scope: 'a'.repeat(64), path: 'sub/a.png', filename: 'a.png' });
const FRESH = Object.freeze({ root_index: 1, root_scope: 'b'.repeat(64), path: 'sub/a.png', filename: 'a.png', url: '/u' });
function fakeApi(answer) {
const calls = [];
return { calls, post: async (url, body, options) => { calls.push({ url, body, options }); return answer(body); } };
}
const flush = () => new Promise(resolve => setImmediate(resolve));
test('rescopeImage posts only the target and returns the fresh descriptor', async () => {
const { rescopeImage, RESCOPE_URL } = await load();
const api = fakeApi(() => ({ success: true, image: FRESH }));
const signal = new AbortController().signal;
const fresh = await rescopeImage(api, OLD, { signal });
assert.deepEqual(fresh, FRESH);
assert.ok(Object.isFrozen(fresh));
assert.deepEqual(api.calls, [{ url: RESCOPE_URL, body: { target: { root_index: 0, root_scope: OLD.root_scope, path: 'sub/a.png' } }, options: { signal } }]);
});
test('rescopeImage answers null for a refusal, a malformed answer, another path or a bad target', async () => {
const { rescopeImage } = await load();
const refused = { post: async () => { throw Object.assign(new Error('gone'), { status: 409, code: 'source-changed' }); } };
assert.equal(await rescopeImage(refused, OLD), null);
const lost = { post: async () => { throw new TypeError('Failed to fetch'); } };
await assert.rejects(rescopeImage(lost, OLD), TypeError, 'a lost answer is not final');
for (const answer of [null, { success: false }, { success: true, image: { ...FRESH, root_scope: 'x' } },
{ success: true, image: { ...FRESH, path: 'other.png' } }, { success: true, image: { ...FRESH, root_index: -1 } }]) {
assert.equal(await rescopeImage(fakeApi(() => answer), OLD), null);
}
const api = fakeApi(() => ({ success: true, image: FRESH }));
for (const image of [null, { ...OLD, root_scope: 'short' }, { ...OLD, path: '' }, { ...OLD, root_index: 1.5 }]) {
assert.equal(await rescopeImage(api, image), null);
}
assert.equal(api.calls.length, 0, 'a malformed target is never sent');
});
test('one rescope per held pair: callers share it, a tried pair answers its first result, reset forgets', async () => {
const { createSourceRecovery } = await load();
let answer = { success: true, image: FRESH };
const api = fakeApi(() => answer);
const applied = [];
const recovery = createSourceRecovery({ api, generation: () => 7, apply: async (image, fresh, started) => { applied.push([image.path, fresh.root_index, started]); return fresh; } });
const [first, second] = await Promise.all([recovery.recover(OLD), recovery.recover({ ...OLD })]);
assert.deepEqual([first, second], [FRESH, FRESH]);
assert.equal(api.calls.length, 1);
assert.deepEqual(applied, [['sub/a.png', 1, 7]]);
answer = { success: false };
assert.deepEqual(await recovery.recover(OLD), FRESH, 'the pair is not asked again');
recovery.reset();
assert.equal(await recovery.recover(OLD), null, 'after a reset the pair is asked again');
assert.equal(api.calls.length, 2);
assert.equal(await recovery.recover({ ...OLD, root_scope: 'bad' }), null);
assert.equal(api.calls.length, 2);
});
test('a refused pair never loops, an aborted page applies nothing and a failing apply is null', async () => {
const { createSourceRecovery } = await load();
const api = fakeApi(() => ({ success: true, image: FRESH }));
const page = new AbortController();
let applied = 0;
const recovery = createSourceRecovery({ api, signal: page.signal, apply: async () => { applied += 1; return FRESH; } });
page.abort();
assert.equal(await recovery.recover(OLD), null);
assert.equal(applied, 0);
const failing = createSourceRecovery({ api, apply: async () => { throw new Error('page gone'); } });
assert.equal(await failing.recover(OLD), null);
assert.throws(() => createSourceRecovery({ api }), TypeError);
// An api without post (an embedded gallery's page adapter) finds nothing.
assert.equal(await createSourceRecovery({ api: {}, apply: async () => FRESH }).recover(OLD), null);
});
test('the memo forgets its oldest pairs past its limit', async () => {
const { createSourceRecovery } = await load();
const api = fakeApi(body => ({ success: true, image: { ...FRESH, path: body.target.path } }));
const recovery = createSourceRecovery({ api, apply: async (_image, fresh) => fresh });
for (let index = 0; index < 65; index++) await recovery.recover({ ...OLD, path: `f${index}.png` });
assert.equal(api.calls.length, 65);
await recovery.recover({ ...OLD, path: 'f64.png' });
assert.equal(api.calls.length, 65, 'a recent pair is still remembered');
await recovery.recover({ ...OLD, path: 'f0.png' });
assert.equal(api.calls.length, 66, 'the oldest pair was forgotten');
});
function fakePage({ selected = true, root = 'all', current = true } = {}) {
const log = [];
const page = {
filters: { root, subfolder: root === 'all' ? '' : 'sub' },
current: started => { log.push(['current', started]); return current; },
selected: image => selected && image.path === OLD.path,
inspect: image => log.push(['inspect', image.root_index]),
dropSelection: () => log.push(['drop']),
stream: {
refresh: async () => { log.push(['refresh']); throw new Error('read failed'); },
reset: (filters, options) => log.push(['reset', { ...filters }, options]),
},
updateScope: () => log.push(['scope']),
closeViewer: () => log.push(['close-viewer']),
clearChecks: () => log.push(['clear-checks']),
listsNow: () => true,
rediscover: () => log.push(['rediscover']),
};
return { page, log };
}
test('a renumbered source keeps nothing by its old key: only the file stays selected', async () => {
const { recoveredSource } = await load();
const { page, log } = fakePage({ root: '0' });
assert.equal(await recoveredSource(page)(OLD, FRESH, 3), FRESH);
assert.deepEqual(log, [['current', 3], ['close-viewer'], ['clear-checks'], ['inspect', 1], ['scope'],
['reset', { root: '1', subfolder: 'sub' }, { load: true }], ['rediscover']]);
});
test('another Source filter falls back to All sources; an unselected file drops the old selection', async () => {
const { recoveredSource } = await load();
const { page, log } = fakePage({ root: '2', selected: false });
await recoveredSource(page)(OLD, FRESH, 0);
assert.deepEqual(log.find(entry => entry[0] === 'reset')[1], { root: 'all', subfolder: '' });
assert.ok(log.some(entry => entry[0] === 'drop'));
assert.ok(!log.some(entry => entry[0] === 'inspect'));
const all = fakePage({ root: 'all' });
await recoveredSource(all.page)(OLD, FRESH, 0);
assert.deepEqual(all.log.find(entry => entry[0] === 'reset')[1], { root: 'all', subfolder: '' });
});
test('an unchanged numbering re-reads in place; a page that moved on is left alone', async () => {
const { recoveredSource } = await load();
const same = { ...FRESH, root_index: 0 };
const { page, log } = fakePage();
assert.equal(await recoveredSource(page)(OLD, same, 1), same);
assert.deepEqual(log, [['current', 1], ['inspect', 0], ['refresh']], 'a failed re-read is not an error');
const unselected = fakePage({ selected: false });
await recoveredSource(unselected.page)(OLD, same, 1);
assert.deepEqual(unselected.log, [['current', 1], ['refresh']]);
const moved = fakePage({ current: false });
assert.equal(await recoveredSource(moved.page)(OLD, FRESH, 1), FRESH, 'the dialog may still retarget');
assert.deepEqual(moved.log, [['current', 1]]);
await flush();
});
test('a scope that still names its root is not stale: nothing is applied', async () => {
const { recoveredSource } = await load();
const { page, log } = fakePage();
assert.equal(await recoveredSource(page)(OLD, { ...OLD, url: '/u' }, 1), null);
assert.deepEqual(log, []);
});
test('a lost rescope answer is not remembered: the pair is asked again', async () => {
const { createSourceRecovery } = await load();
let down = true;
const api = { calls: 0, post: async () => { api.calls += 1; if (down) throw Object.assign(new Error('bad gateway'), { status: 502 }); return { success: true, image: FRESH }; } };
const recovery = createSourceRecovery({ api, apply: async (_image, fresh) => fresh });
assert.equal(await recovery.recover(OLD), null);
down = false;
assert.deepEqual(await recovery.recover(OLD), FRESH);
assert.equal(api.calls, 2);
});
test('the same index under another scope (a changed key) re-reads in place and keeps the selection', async () => {
const { recoveredSource } = await load();
const rekeyed = { ...OLD, root_scope: 'c'.repeat(64) };
const { page, log } = fakePage();
page.stream.refresh = async () => { log.push(['refresh']); };
assert.equal(await recoveredSource(page)(OLD, rekeyed, 2), rekeyed);
assert.deepEqual(log, [['current', 2], ['inspect', 0], ['refresh']]);
assert.ok(!log.some(entry => ['reset', 'drop', 'clear-checks', 'close-viewer'].includes(entry[0])));
});
+18
View File
@@ -141,6 +141,22 @@ test('the Favorites page asks the listing for favorites only (#249 D51)', async
for(const favorites of ['1',1,'true'])await assert.rejects(stream.reset({favorites}),/Invalid favorites filter/);
});
test('a collection page asks the listing for that collection only (#331 D137)', async () => {
const {createImageStream}=await source;let query;
const stream=createImageStream({api:{get:async url=>{query=new URL(url,'http://local').searchParams;return page([]);}}});
await stream.reset({collection:'12',q:'fog'});
assert.equal(query.get('collection'),'12');assert.equal(query.get('q'),'fog');assert.equal(query.has('favorites'),false);
await stream.reset({collection:7});
assert.equal(query.get('collection'),'7');
for(const collection of ['',null,undefined]) {
await stream.reset({collection});
assert.equal(query.has('collection'),false);
}
for(const collection of ['0','07','x',-1,1.5,'1234567890',true])await assert.rejects(stream.reset({collection}),/Invalid collection filter/);
// Favorites and a collection are never both a page's filter.
await assert.rejects(stream.reset({collection:'3',favorites:true}),/Invalid collection filter/);
});
test('restored browsing state resumes its cursor and filters without an initial request', async () => {
const {createImageStream}=await source;
const calls=[];
@@ -517,5 +533,7 @@ test('listingFilter is the listing query as an object, without paging (#392)', a
assert.deepEqual(listingFilter({q:'cat', root:'1', subfolder:'a/b', category:'Animals', tags:[' x ','y'], media:'video', sort:'oldest', favorites:true}),
{q:'cat', root:'1', subfolder:'a/b', category:'Animals', tags:'x,y', media:'video', sort:'oldest', favorites:'1'});
assert.deepEqual(listingFilter({root:'all', q:'', tags:[], media:'all'}), {sort:'newest'});
// #331: "Select all N matching" on a collection page acts on that collection's listing.
assert.deepEqual(listingFilter({collection:'4', q:'cat'}), {q:'cat', sort:'newest', collection:'4'});
assert.throws(() => listingFilter({sort:'sideways'}), TypeError);
});
@@ -135,6 +135,41 @@ test('a model that is not downloaded is named with the server sentence', () => {
assert.throws(() => modelUnavailable({}, 'nope'), TypeError);
});
test('a downloaded model whose backend package is missing names the package and the command (#424)', () => {
const { modelUnavailable, needsPackages } = state;
const gguf = { downloaded: true, name: 'GGUF (Recommended)', label: 'GGUF JoyCaption', missing_packages: ['llama-cpp-python'],
install_command: 'pip install llama-cpp-python',
install_note: 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python' };
const sentence = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
assert.equal(modelUnavailable({ gguf }, 'gguf'), sentence);
// Not downloaded is named first, in its own sentence.
assert.equal(modelUnavailable({ gguf: { ...gguf, downloaded: false } }, 'gguf'),
'GGUF (Recommended) is not downloaded. Download it in System › Auto-tagging.');
assert.equal(modelUnavailable({ gguf: { ...gguf, missing_packages: [] } }, 'gguf'), '');
assert.equal(needsPackages('HuggingFace JoyCaption', { missing_packages: ['accelerate', 'bitsandbytes'] }),
'HuggingFace JoyCaption needs accelerate and bitsandbytes. Install them into ComfyUI\'s Python: pip install accelerate bitsandbytes.');
assert.equal(needsPackages('X', { missing_packages: ['a', 'b', 'c'], install_command: 'pip install a b c' }),
'X needs a, b and c. Install them into ComfyUI\'s Python: pip install a b c.');
assert.equal(needsPackages('X', null), '');
assert.equal(needsPackages('X', { missing_packages: [1, ''] }), '');
// Without the server's label the option's own name is used.
assert.ok(modelUnavailable({ 'wd14-vit': { downloaded: true, missing_packages: ['onnxruntime'] } }, 'wd14-vit')
.startsWith('WD14 ViT needs onnxruntime.'));
});
test('the bulk dialog keeps Start disabled while the chosen model misses a package (#424)', async () => {
const { createTaggingOptionsController } = state;
const models = { 'wd14-vit': { downloaded: true, label: 'WD14 ViT', missing_packages: ['onnxruntime'],
install_command: 'pip install onnxruntime' } };
const options = createTaggingOptionsController({ api: { get: async () => ({ success: true, models, model_type: 'wd14-vit' }) } });
await options.open();
assert.equal(options.state.model, 'wd14-vit');
assert.equal(options.state.canStart, false);
assert.equal(options.state.unavailable, 'WD14 ViT needs onnxruntime. Install it into ComfyUI\'s Python: pip install onnxruntime.');
options.dispose();
});
test('the bulk options start from the server defaults and wait for them before Start', async () => {
const { createTaggingOptionsController } = state;
let answer;
@@ -278,6 +278,7 @@ test('record rejects root path media and invalid versioned fields with retryable
data => { data.resolution.sampler_id = {}; },
data => { data.library.tags_scope = 'recipe'; },
data => { data.library.category_scope = 'recipe'; },
data => { data.library.tags_scope = 'image'; },
data => { delete data.library.category_scope; },
data => { delete data.library.tags_scope; },
]) {
@@ -942,3 +943,14 @@ test("names the server quotes (parentheses, '!*') still match their original and
assert.equal(record.download.url, entry.url);
assert.deepEqual(record.preview, { url: entry.thumbnail_url });
});
test('record accepts an image\'s own category on any link status (#332)', () => {
const toRecord = contract('toInspectionRecord');
for (const status of ['linked', 'none', 'ambiguous', 'unavailable']) {
const data = envelope();
Object.assign(data.library, { status, category: 'Scans', category_scope: 'image' });
const record = toRecord(data, image(), extractGenerationFacts);
assert.equal(record.library.category, 'Scans');
assert.equal(record.library.category_scope, 'image');
}
});
+20 -5
View File
@@ -4,7 +4,7 @@ const assert = require('node:assert/strict');
const source = import('../../web/js/workspace/saved-views.mjs');
const IMAGE_DEFAULTS = { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false };
const IMAGE_DEFAULTS = { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false, collection: '' };
const LIBRARY_DEFAULTS = { q: '', category: '', sort: 'newest', tags: [] };
test('a saved state always holds every page key and keeps only values its page accepts', async () => {
@@ -12,8 +12,17 @@ test('a saved state always holds every page key and keeps only values its page a
assert.deepEqual(SAVED_VIEW_PAGES, ['images', 'library']);
assert.deepEqual(normalizeViewState('images', {}), IMAGE_DEFAULTS);
assert.deepEqual(normalizeViewState('library', null), LIBRARY_DEFAULTS);
const full = { q: 'castle', category: 'Portraits', sort: 'name-desc', source: '2', folder: 'a/b', media: 'video', tags: ['moody', 'fog'], favorites: true };
const full = { q: 'castle', category: 'Portraits', sort: 'name-desc', source: '2', folder: 'a/b', media: 'video', tags: ['moody', 'fog'], favorites: true, collection: '' };
assert.deepEqual(normalizeViewState('images', full), full);
// #331 D140: a collection id is '' or 1 to 9 digits without a leading zero; an old view without the key reads ''.
for (const collection of ['1', '12', '999999999']) {
assert.equal(normalizeViewState('images', { collection }).collection, collection);
}
for (const collection of ['0', 'x', 12, '012', '1234567890', null]) {
assert.equal(normalizeViewState('images', { collection }).collection, '');
}
// Favorites and a collection are never both a view's filter: the collection wins, as the listing's URL does.
assert.deepEqual(normalizeViewState('images', { ...full, collection: '4' }), { ...full, favorites: false, collection: '4' });
// D51: favorites is a boolean; anything else is the whole listing.
for (const favorites of ['true', 1, null]) assert.equal(normalizeViewState('images', { ...full, favorites }).favorites, false);
// Each rejected value falls back on its own; the rest of the view still applies.
@@ -63,11 +72,17 @@ test('the listing keeps usable views in order and normalizes their state', async
test('page filters and saved states convert both ways', async () => {
const { imagesFilters, imagesViewState, libraryFilters, libraryViewState } = await source;
const filters = { q: 'castle', category: 'Portraits', sort: 'rating', root: '1', subfolder: 'x', media: 'image', tags: ['fog'], favorites: true };
const filters = { q: 'castle', category: 'Portraits', sort: 'rating', root: '1', subfolder: 'x', media: 'image', tags: ['fog'], favorites: true, collection: '' };
const state = imagesViewState(filters);
assert.deepEqual(state, { q: 'castle', category: 'Portraits', sort: 'rating', source: '1', folder: 'x', media: 'image', tags: ['fog'], favorites: true });
assert.deepEqual(state, { q: 'castle', category: 'Portraits', sort: 'rating', source: '1', folder: 'x', media: 'image', tags: ['fog'], favorites: true, collection: '' });
assert.deepEqual(imagesFilters(state), filters);
assert.deepEqual(imagesFilters({}), { q: '', category: '', sort: 'newest', root: 'all', subfolder: '', media: 'all', tags: [], favorites: false });
assert.deepEqual(imagesFilters({}), { q: '', category: '', sort: 'newest', root: 'all', subfolder: '', media: 'all', tags: [], favorites: false, collection: '' });
// #331 D140: the open collection round-trips; a missing one is ''.
const inCollection = { ...filters, favorites: false, collection: '7' };
assert.equal(imagesViewState(inCollection).collection, '7');
assert.deepEqual(imagesFilters(imagesViewState(inCollection)), inCollection);
assert.equal(imagesViewState({ ...filters, collection: undefined }).collection, '');
assert.equal(imagesViewState({ ...filters, collection: 7 }).collection, '7');
assert.equal(imagesViewState({ ...filters, favorites: undefined }).favorites, false);
const shelf = { text: 'neon', category: 'Cities', sort: 'oldest', tags: ['night'] };
assert.deepEqual(libraryViewState(shelf), { q: 'neon', category: 'Cities', sort: 'oldest', tags: ['night'] });
+26
View File
@@ -311,3 +311,29 @@ test('Needs review is a native SAVED VIEWS route titled Review, with its own cou
assert.deepEqual(calls, ['/prompt_manager/review/items?limit=1']);
assert.deepEqual([...counts], [['review', 4]]);
});
// Collections (#331 D137): `#/collection/<id>` opens Images filtered to one collection; its breadcrumb is "Collection".
test('collection links resolve to their id, title Collection, and re-render only when the id changes', async () => {
const {resolveRoute,mountRouter,isHandoff,routeTitle,routes,COLLECTION_ROUTE}=await source;
assert.deepEqual(COLLECTION_ROUTE,{id:'collection',label:'Collection'});
assert.ok(Object.isFrozen(COLLECTION_ROUTE));
const route=resolveRoute('#/collection/7');
assert.deepEqual([route.id,route.collectionId,route.hash,routeTitle(route)],['collection',7,'#/collection/7','Collection']);
assert.equal(isHandoff(route),false);
assert.equal(resolveRoute('#/collection/999999999').collectionId,999999999);
assert.equal(routes.some(item=>item.id==='collection'),false,'a collection is not an Overview card or a WORKSPACE link');
// The listing's and routes' id rule: 1 to 9 digits without a leading zero.
for(const hash of ['#/collection','#/collection/','#/collection/0','#/collection/07','#/collection/-2','#/collection/1.5',
'#/collection/1234567890','#/collection/3/x','#/collection/abc','#/collections/3']) {
assert.equal(resolveRoute(hash).id,'images',hash);
}
const win=new EventTarget();win.location={hash:'#/collection/3'};
win.history={state:null,replaceState(state,unused,hash){win.location.hash=hash;}};
const seen=[];const dispose=mountRouter(win,next=>seen.push([next.id,next.collectionId]));
win.dispatchEvent(new Event('hashchange'));
win.location.hash='#/collection/4';win.dispatchEvent(new Event('hashchange'));
win.location.hash='#/saved/4';win.dispatchEvent(new Event('hashchange'));
win.location.hash='#/collection/4';win.dispatchEvent(new Event('hashchange'));
assert.deepEqual(seen,[['collection',3],['collection',4],['saved',undefined],['collection',4]]);
dispose();
});
+2 -2
View File
@@ -63,9 +63,9 @@ test('status line shows only server-reported backups (D103)', async () => {
const PREVIEW = Object.freeze({
success: true, token: 'r'.repeat(43), expires_at: '2026-10-25T12:15:00+00:00',
summary: { prompts: 1234, positives: 900, pairs: 3, archived_pairs: 0, image_links: 56, runs: 1, saved_views: 2 },
summary: { prompts: 1234, positives: 900, pairs: 3, archived_pairs: 0, image_links: 56, runs: 1, saved_views: 2, collections: 1 },
schema: { migrated: true, library_state: true }, integrity: 'ok', documents: { repaired: 0, cleared: 0, images: 0 },
live: { prompts: 1, positives: 1, pairs: 0, archived_pairs: 0, image_links: 0, runs: 0, saved_views: 0 }, size: 4096,
live: { prompts: 1, positives: 1, pairs: 0, archived_pairs: 0, image_links: 0, runs: 0, saved_views: 0, collections: 0 }, size: 4096,
safety_backup: { destination_label: 'prompts.db.backup_<date and time>' },
});
+54
View File
@@ -482,6 +482,60 @@ test('Tag output images › Review previews the drafts, else says why; a refused
} finally { scope.dispose(); dom.restore(); }
});
// #424: GGUF on disk without llama-cpp-python.
const NEEDS_LLAMA = { missing_packages: ['llama-cpp-python'], install_command: 'pip install llama-cpp-python',
install_note: 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python' };
const NEEDS_SENTENCE = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
test('a model on disk whose backend package is missing never reads as ready (#424)', async () => {
const { modelDescription, readStatus, batchCloseNote, STATUS_COPY } = await tagging;
const status = readStatus({ ...MODELS, models: { ...MODELS.models,
gguf: entry('GGUF JoyCaption', '~4.5 GB', { ...NEEDS_LLAMA }),
'wd14-vit': entry('WD14 ViT', '~400 MB', { downloaded: false, missing_packages: ['onnxruntime'],
install_command: 'pip install onnxruntime', install_note: 'Use onnxruntime-gpu for CUDA.' }),
hf: entry('HuggingFace JoyCaption', '~16 GB', { downloaded: false, model_exists: undefined,
missing_packages: ['accelerate', 'bitsandbytes'], install_command: 'pip install accelerate bitsandbytes' }) } });
const ready = modelDescription(status, 'gguf');
assert.notEqual(ready, STATUS_COPY.downloaded);
assert.equal(ready, 'Downloaded. Needs llama-cpp-python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python');
// Not downloaded reads differently, and still says what it will need.
assert.equal(modelDescription(status, 'wd14-vit'), 'Not downloaded. Download size ~400 MB. Also needs onnxruntime.');
assert.equal(modelDescription(status, 'hf'), 'Not downloaded. Download size ~16 GB. Also needs accelerate and bitsandbytes.');
const partial = readStatus({ ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB',
{ downloaded: false, model_exists: true, mmproj_exists: false, ...NEEDS_LLAMA }) } });
assert.equal(modelDescription(partial, 'gguf'), 'Partly downloaded. Download again to finish. Also needs llama-cpp-python.');
// Untrusted shapes are ignored; a missing command is built from the names.
const odd = readStatus({ ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB',
{ missing_packages: ['llama-cpp-python', 3, ''], install_command: 7, install_note: {} }) } });
assert.equal(modelDescription(odd, 'gguf'), 'Downloaded. Needs llama-cpp-python: pip install llama-cpp-python.');
const junk = readStatus({ ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB',
{ missing_packages: 'llama-cpp-python' }) } });
assert.equal(modelDescription(junk, 'gguf'), STATUS_COPY.downloaded);
// The batch review says why it cannot start, in the server's sentence.
assert.equal(batchCloseNote({ ...PLAN, model: { type: 'gguf', label: 'GGUF JoyCaption', downloaded: true, ...NEEDS_LLAMA } }),
NEEDS_SENTENCE);
assert.equal(batchCloseNote({ ...PLAN, model: { ...PLAN.model, downloaded: false, ...NEEDS_LLAMA } }),
'WD14 SwinV2 is not downloaded. Download it first.');
});
test('Review explains a missing backend package instead of reviewing (#424)', async () => {
const models = { ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB', { ...NEEDS_LLAMA }) } };
const { dom, scope, button, byId, calls } = await mounted({ models });
try {
byId('tag-model').value = 'gguf'; await byId('tag-model').dispatch('change');
const review = button('Review');
assert.equal(review.getAttribute('aria-disabled'), 'true');
assert.equal(review.dataset.reason, NEEDS_SENTENCE);
await review.dispatch('click');
assert.equal(calls.toasts.at(-1), NEEDS_SENTENCE);
assert.deepEqual(calls.post, []);
// Download stays offered: files and packages are separate.
assert.equal(button('Download again').getAttribute('aria-disabled'), null);
} finally { scope.dispose(); dom.restore(); }
});
test('settingRefusal marks only an Auto-tagging save\'s plain 400 as invalid-setting (CUJ 2a)', async () => {
const { settingRefusal } = await import('../../web/js/workspace/system.mjs');
const refusal = (extra = {}) => Object.assign(new Error('General-tag confidence must be between 0 and 1.'),
+37
View File
@@ -567,6 +567,43 @@ test('an input dialog with a fallback applies it for an empty value', async () =
}
});
// #332: Set category's field suggests the categories under its own datalist id, and "Use prompt category" sits
// between Cancel and Apply, closing the dialog before it runs.
test('an input dialog offers suggestions and a secondary action', async () => {
const { openInputDialog, closeWorkspaceDialog, INPUT_SUGGESTIONS_ID } = await dialogsModule;
const dom = fakeDOM();
const scope = (await coreModule).createScope();
try {
const calls = [];
openInputDialog({ scope, title: 'Categorize selected images', label: 'Category', action: 'Apply', maxLength: 100,
suggestions: ['Landscapes', 'Scans'], secondary: { label: 'Use prompt category', onClick: () => calls.push('secondary') },
onApply: value => calls.push(value) });
assert.equal(INPUT_SUGGESTIONS_ID, 'workspace-input-suggestions');
const body = byId(dom, 'modal-body');
const [field, list, actions] = body.children;
const input = field.children[1];
assert.equal(input.getAttribute('list'), 'workspace-input-suggestions');
assert.equal(list.tagName.toLowerCase(), 'datalist');
assert.equal(list.id, 'workspace-input-suggestions');
assert.deepEqual(list.children.map(option => option.value), ['Landscapes', 'Scans']);
assert.deepEqual(actions.children.map(node => node.textContent), ['Cancel', 'Use prompt category', 'Apply']);
actions.children[1].click();
assert.deepEqual(calls, ['secondary']);
// No suggestions: no datalist and no list attribute.
openInputDialog({ scope, title: 'T', label: 'L', action: 'A', onApply() {} });
const [plainField, plainActions] = byId(dom, 'modal-body').children;
assert.equal(plainField.children[1].getAttribute('list'), null);
assert.deepEqual(plainActions.children.map(node => node.textContent), ['Cancel', 'A']);
assert.throws(() => openInputDialog({ scope, title: 'T', label: 'L', action: 'A', suggestions: [1], onApply() {} }), TypeError);
assert.throws(() => openInputDialog({ scope, title: 'T', label: 'L', action: 'A', secondary: { label: '' }, onApply() {} }), TypeError);
assert.throws(() => openInputDialog({ scope, title: 'T', label: 'L', action: 'A', secondary: { label: 'X' }, onApply() {} }), TypeError);
} finally {
closeWorkspaceDialog();
scope.dispose();
dom.restore();
}
});
test('dialogs restore focus to the trigger on Escape', async () => {
const { openRefineDialog, openImportDialog, openCopyFallback, closeWorkspaceDialog } = await dialogsModule;
const dom = fakeDOM();
@@ -553,3 +553,16 @@ test('delete text names the tag, its real links and what stays unchanged', async
assert.equal(deleteTagText('unused', zero),
'“unused” is not linked to any saved prompt or image. Deleting it removes only the name.');
});
test('a category row counts prompts and images, and an image-only row browses Images (#332 D143)', async () => {
const {categoryRowText, categoryBrowseTarget, categoryPreviewText} = await import('../../web/js/workspace/vocabulary-state.mjs');
assert.equal(categoryRowText({value: 'L', prompt_count: 3}), '3 positive prompts');
assert.equal(categoryRowText({value: 'L', prompt_count: 1, image_count: 0}), '1 positive prompt');
assert.equal(categoryRowText({value: 'L', prompt_count: 3, image_count: 2}), '3 positive prompts · 2 images');
assert.equal(categoryRowText({value: 'S', prompt_count: 0, image_count: 1}), '1 image');
assert.equal(categoryBrowseTarget({value: 'L', prompt_count: 3, image_count: 2}), 'library');
assert.equal(categoryBrowseTarget({value: 'S', prompt_count: 0, image_count: 2}), 'images');
const impact = {positive_prompts: 3, other_prompts: 0, negative_variants: 0, archived_negative_variants: 0, images: 2};
assert.equal(categoryPreviewText({source: 'Landscapes', target: 'Landscape', target_exists: false, impact}),
'“Landscapes” is used by 3 positive prompts and 2 images. Apply renames it to “Landscape”.');
});
+15 -9
View File
@@ -2205,24 +2205,30 @@ class TestOutputScanInternals(AdminAPITestCase):
self.assertEqual(self.api._ingest_scanned_file(f, meta), "unchanged")
def test_ingest_link_failures_are_tolerated(self):
"""A failed link rolls the new prompt back: found, nothing left behind."""
f = Path("gen.png")
meta = {"prompt": json.dumps(_wired_prompt("a dog"))}
self.api.db.link_image_to_prompt = _raise
self.assertEqual(self.api._ingest_scanned_file(f, meta), "added")
self.assertEqual(self.api._ingest_scanned_file(f, meta), "found")
self.api.db._insert_image_link = _raise
with self.assertLogs("prompt_manager.database.scanned_prompts", "ERROR"):
self.assertEqual(self.api._ingest_scanned_file(f, meta), "found")
self.assertEqual(self.api._ingest_scanned_file(f, meta), "found")
self.assertEqual(self.api.db.get_recent_prompts()["prompts"], [])
def test_ingest_save_failure_counts_as_found(self):
f = Path("gen.png")
meta = {"prompt": json.dumps(_wired_prompt("a bird"))}
self.api.db.save_prompt = lambda *a, **k: None
self.assertEqual(self.api._ingest_scanned_file(f, meta), "found")
with patch(
"py.api.admin.store_scanned_prompt", return_value=("found", None)
) as store:
self.assertEqual(self.api._ingest_scanned_file(f, meta), "found")
self.assertEqual(store.call_args.args[3], "a bird")
def test_batch_counts_and_per_file_errors(self):
meta = {"prompt": json.dumps(_wired_prompt("a fish"))}
self.api.db.get_prompt_by_hash = _raise
counts = self.api._ingest_scan_batch_sync(
[(Path("a.png"), meta), (Path("b.png"), {})]
)
with patch("py.api.admin.store_scanned_prompt", side_effect=_raise):
counts = self.api._ingest_scan_batch_sync(
[(Path("a.png"), meta), (Path("b.png"), {})]
)
self.assertEqual(
counts,
{
+44 -1
View File
@@ -29,6 +29,7 @@ except ImportError:
model_gate = None
from py import config as config_module
from py.autotag import MissingBackendPackages, package_fields
AutotagConfig = getattr(config_module, "AutotagConfig", None)
@@ -37,6 +38,11 @@ WAIT = 30.0
WD14 = {"general_threshold": 0.35, "character_threshold": 0.85}
LLM = {"instructions": "Generate tags."}
UNAVAILABLE = "Download it in System › Auto-tagging."
NEEDS_LLAMA = (
"GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI's Python: "
'pip install llama-cpp-python. For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" '
"pip install llama-cpp-python"
)
SCORED = [("sandstone", 0.98), ("desert", 0.96)] + [
(f"tag_{i:03d}", round(0.9 - i * 0.005, 4)) for i in range(150)
@@ -59,6 +65,8 @@ class FakeTagger:
"hf": {},
}
self.downloaded = {name: True for name in self.models_config}
# pip names of the backend packages that cannot be imported (#424)
self.missing = {name: [] for name in self.models_config}
self.loaded = None
self.load_calls = []
self.unload_calls = 0
@@ -71,7 +79,11 @@ class FakeTagger:
def get_models_status(self):
return {
name: {"downloaded": self.downloaded[name], "model_path": None}
name: {
"downloaded": self.downloaded[name],
"model_path": None,
**package_fields(name, self.missing[name]),
}
for name in self.models_config
}
@@ -315,6 +327,37 @@ class TestModelAvailability(SuggestTestCase):
self.assertEqual(self.tagger.calls, [])
self.assertIsNone(model_gate.holder())
async def test_missing_backend_package_is_refused_before_the_model(self):
observed = await self.observed()
self.tagger.missing["gguf"] = ["llama-cpp-python"]
status, body = await self.suggest(self.suggest_body(observed, "gguf"))
self.assertEqual((status, body["code"]), (409, "model-unavailable"))
self.assertEqual(body["success"], False)
self.assertEqual(body["error"], NEEDS_LLAMA)
self.assertEqual(self.tagger.load_calls, [])
self.assertIsNone(model_gate.holder())
# Not downloaded still reads as before, whatever the packages.
self.tagger.downloaded["gguf"] = False
status, body = await self.suggest(self.suggest_body(observed, "gguf"))
self.assertIn("is not downloaded", body["error"])
async def test_a_package_lost_after_the_check_is_the_same_409_without_a_traceback(
self,
):
observed = await self.observed()
self.tagger.load_error = MissingBackendPackages("gguf", ["llama-cpp-python"])
with self.assertNoLogs(self.api.logger, level="ERROR"):
status, body = await self.suggest(self.suggest_body(observed, "gguf"))
self.assertEqual((status, body["code"]), (409, "model-unavailable"))
self.assertEqual(body["error"], NEEDS_LLAMA)
self.assertIsNone(model_gate.holder())
entry = self.api._captures._entries[observed["capture"]["token"]]
self.assertEqual(entry.uses, 0)
async def test_load_failure_is_a_generic_error_and_frees_everything(self):
observed = await self.observed()
self.tagger.load_error = RuntimeError(f"cannot load {self.output_dir}")
+24
View File
@@ -73,6 +73,7 @@ class FakeAutotagService:
self.status_calls = 0
self.status_threads = []
self.status_paths = {}
self.status_extra = {}
self.suggest_calls = 0
def get_models_status(self):
@@ -82,6 +83,7 @@ class FakeAutotagService:
name: {
"downloaded": self.downloaded,
"model_path": self.status_paths.get(name),
**self.status_extra.get(name, {}),
}
for name in self.models_config
}
@@ -1111,6 +1113,24 @@ class TestModelStatus(AutotagAPITestCase):
leaked = [text for text in _strings(data) if _absolute(text)]
self.assertEqual(leaked, [])
async def test_models_status_names_missing_backend_packages(self):
fields = {
"missing_packages": ["llama-cpp-python"],
"install_command": "pip install llama-cpp-python",
"install_note": "For a CUDA build: ...",
}
self.service.status_extra = {"gguf": fields}
models = (await self._models())["models"]
self.assertTrue(models["gguf"]["downloaded"])
self.assertEqual(
{key: models["gguf"][key] for key in fields},
fields,
)
self.assertEqual(
{key: models["wd14-vit"][key] for key in fields},
{"missing_packages": [], "install_command": None, "install_note": None},
)
async def test_models_status_adds_label_size_label_and_loaded(self):
self.service._loaded_type = "wd14-vit"
data = await self._models()
@@ -1669,6 +1689,10 @@ class TestTagOutputImagesPreview(BatchJobTestCase):
"label": "WD14 ViT",
"size_label": "~400 MB",
"downloaded": True,
# #424: the backend packages it cannot import, none here.
"missing_packages": [],
"install_command": None,
"install_note": None,
},
)
self.assertEqual(skip["existing"], "skip")
+601
View File
@@ -0,0 +1,601 @@
"""Collections (#331): the routes under /prompt_manager/collections."""
import json
import ntpath
import sqlite3
from unittest.mock import patch
try:
from test_api_images import ImageAPITestCase, make_png
from open_handles import assert_closed
except ImportError:
from tests.test_api_images import ImageAPITestCase, make_png
from tests.open_handles import assert_closed
BASE = "/prompt_manager/collections"
ITEMS = BASE + "/items"
MEMBERSHIP = BASE + "/membership"
SOURCES_CHANGED = "The sources changed. Refresh the images and try again."
class CollectionRoutesTestCase(ImageAPITestCase):
async def tearDownAsync(self):
self.api.db.close_all()
assert_closed(self, self._temp_db.name, self._output_tmp.name)
await super().tearDownAsync()
def make(self, *names, root=None):
for name in names:
make_png((root or self.output_dir) / name)
def scope(self, index=0):
roots = list(self.api._get_all_output_dirs())
return self.api._gallery_root_scope(roots[index], index)
def item(self, path, index=0, scope=None):
return {
"root_index": index,
"root_scope": scope or self.scope(index),
"path": path,
}
def two_sources(self):
second = self.comfy_dir / "second"
second.mkdir()
self.api._get_all_output_dirs = lambda: [self.output_dir, second]
return second
async def call(self, method, url, body=None, status=200, **params):
response = await self.client.request(method, url, json=body, params=params)
text = await response.text()
self.assertEqual(response.status, status, text)
data = json.loads(text)
self.assertNotIn(self.comfy_dir.as_posix(), text)
self.assertNotIn(str(self.comfy_dir).replace("\\", "\\\\"), text)
return data
async def add(self, status=200, **body):
return await self.call("POST", ITEMS, body, status)
async def listed(self):
return (await self.call("GET", BASE))["collections"]
class TestCollectionRoutes(CollectionRoutesTestCase):
async def test_each_route_succeeds(self):
self.make("a.png", "b.png", "c.png")
self.assertEqual(
await self.call("GET", BASE), {"success": True, "collections": []}
)
made = await self.add(
name=" LoRa testing ", items=[self.item("a.png"), self.item("b.png")]
)
grids = made["collection"]
self.assertEqual(grids["name"], "LoRa testing")
self.assertEqual(
made,
{
"success": True,
"collection": grids,
"created": True,
"added": 2,
"already": 0,
"not_found": 0,
},
)
again = await self.add(
collection_id=grids["id"], items=[self.item("b.png"), self.item("c.png")]
)
self.assertEqual(
(again["created"], again["added"], again["already"]), (False, 1, 1)
)
self.assertEqual(
await self.listed(),
[{"id": grids["id"], "name": "LoRa testing", "count": 3, "missing": 0}],
)
membership = await self.call(
"GET", MEMBERSHIP, root_index="0", root_scope=self.scope(), path="a.png"
)
self.assertEqual(membership, {"success": True, "collection_ids": [grids["id"]]})
removed = await self.call(
"DELETE",
ITEMS,
{"collection_id": grids["id"], "items": [self.item("a.png")]},
)
self.assertEqual(removed, {"success": True, "collection": grids, "removed": 1})
renamed = await self.call(
"PATCH", f"{BASE}/{grids['id']}", {"name": "LoRA grids"}
)
self.assertEqual(
renamed,
{"success": True, "collection": {"id": grids["id"], "name": "LoRA grids"}},
)
deleted = await self.call("DELETE", f"{BASE}/{grids['id']}")
self.assertEqual(deleted, {"success": True, "deleted": True})
self.assertEqual(await self.listed(), [])
# Files are never touched.
self.assertTrue((self.output_dir / "a.png").is_file())
async def test_names_are_unique_ignoring_case(self):
self.make("a.png", "b.png")
first = (await self.add(name="LoRa testing", items=[self.item("a.png")]))[
"collection"
]
second = await self.add(name="lora TESTING", items=[self.item("b.png")])
self.assertEqual(second["collection"], first)
self.assertFalse(second["created"])
self.assertEqual(len(await self.listed()), 1)
async def test_bad_requests_are_400(self):
self.make("a.png")
made = await self.add(name="Set", items=[self.item("a.png")])
cid = made["collection"]["id"]
good = [self.item("a.png")]
for body in (
{"items": good},
{"name": "x", "collection_id": cid, "items": good},
{"name": "x"},
{"name": "x", "items": good, "extra": 1},
{"collection_id": "1", "items": good},
{"collection_id": True, "items": good},
{"collection_id": 0, "items": good},
{"collection_id": 10**10, "items": good},
{"name": "", "items": good},
{"name": "x" * 101, "items": good},
{"name": "bell\x07", "items": good},
{"name": 5, "items": good},
{"name": "x", "items": []},
{"name": "x", "items": "a.png"},
{"name": "x", "items": [self.item("a.png")] * 501},
{"name": "x", "items": [{"root_index": 0, "path": "a.png"}]},
{"name": "x", "items": [{**self.item("a.png"), "root_index": "0"}]},
{"name": "x", "items": [{**self.item("a.png"), "root_scope": "xyz"}]},
):
with self.subTest(body=body):
data = await self.add(400, **body)
self.assertFalse(data["success"])
for body in ({"items": good}, {"collection_id": cid}):
with self.subTest(remove=body):
await self.call("DELETE", ITEMS, body, 400)
for raw in ("abc", "0", "1234567890", "-1", "1.0"):
with self.subTest(path_id=raw):
await self.call("PATCH", f"{BASE}/{raw}", {"name": "x"}, 400)
await self.call("DELETE", f"{BASE}/{raw}", None, 400)
await self.call("PATCH", f"{BASE}/{cid}", {"name": ""}, 400)
await self.call("PATCH", f"{BASE}/{cid}", {"label": "x"}, 400)
for params in (
{"root_index": "x", "root_scope": self.scope(), "path": "a.png"},
{"root_index": "0", "path": "a.png"},
{"root_index": "0", "root_scope": "nope", "path": "a.png"},
):
with self.subTest(membership=params):
await self.call("GET", MEMBERSHIP, status=400, **params)
self.assertEqual((await self.listed())[0]["count"], 1)
async def test_unsafe_paths_are_403(self):
self.make("a.png")
for path in (
"../a.png",
"sub/../../a.png",
"C:/a.png",
"a\\b.png",
"/a.png",
"notes.txt",
):
with self.subTest(path=path):
await self.add(403, name="Set", items=[self.item(path)])
await self.call(
"GET",
MEMBERSHIP,
status=403,
root_index="0",
root_scope=self.scope(),
path=path,
)
self.assertEqual(await self.listed(), [])
async def test_a_body_over_the_cap_is_413(self):
response = await self.client.post(
ITEMS, data=b'{"name":"x","items":[' + b" " * (1024 * 1024 + 1) + b"]}"
)
self.assertEqual(response.status, 413)
async def test_missing_collection_and_files_are_404(self):
self.make("a.png")
data = await self.add(404, collection_id=999, items=[self.item("a.png")])
self.assertEqual(data["code"], "collection-not-found")
data = await self.call(
"DELETE", ITEMS, {"collection_id": 999, "items": [self.item("a.png")]}, 404
)
self.assertEqual(data["code"], "collection-not-found")
data = await self.call("PATCH", f"{BASE}/999", {"name": "x"}, 404)
self.assertEqual(data["code"], "collection-not-found")
data = await self.call("DELETE", f"{BASE}/999", None, 404)
self.assertEqual(data["code"], "collection-not-found")
data = await self.add(404, name="New", items=[self.item("never.png")])
self.assertEqual(data["code"], "files-not-found")
self.assertEqual(await self.listed(), [])
async def test_unlisted_files_count_as_not_found(self):
self.make("a.png")
made = await self.add(
name="Set", items=[self.item("a.png"), self.item("later.png")]
)
self.assertEqual((made["added"], made["not_found"]), (1, 1))
again = await self.add(
collection_id=made["collection"]["id"],
items=[self.item("a.png"), self.item("later.png")],
)
self.assertEqual(
(again["added"], again["already"], again["not_found"]), (0, 1, 1)
)
async def test_name_taken_is_409(self):
self.make("a.png")
first = (await self.add(name="First", items=[self.item("a.png")]))["collection"]
second = (await self.add(name="Second", items=[self.item("a.png")]))[
"collection"
]
data = await self.call(
"PATCH", f"{BASE}/{second['id']}", {"name": "FIRST"}, 409
)
self.assertEqual(
(data["code"], data["collection_id"]), ("name-taken", first["id"])
)
renamed = await self.call("PATCH", f"{BASE}/{second['id']}", {"name": "SECOND"})
self.assertEqual(renamed["collection"]["name"], "SECOND")
async def test_stale_scope_is_409_after_reordering_sources(self):
second = self.two_sources()
self.make("a.png")
self.make("b.png", root=second)
stale = self.item("b.png", 1)
made = await self.add(name="Set", items=[self.item("a.png")])
cid = made["collection"]["id"]
self.api._get_all_output_dirs = lambda: [second, self.output_dir]
for method, body in (
("POST", {"collection_id": cid, "items": [self.item("b.png", 0), stale]}),
("DELETE", {"collection_id": cid, "items": [stale]}),
):
with self.subTest(method=method):
data = await self.call(method, ITEMS, body, 409)
self.assertEqual(data["code"], "source-changed")
self.assertEqual(data["error"], SOURCES_CHANGED)
data = await self.call(
"GET",
MEMBERSHIP,
status=409,
root_index="1",
root_scope=stale["root_scope"],
path="b.png",
)
self.assertEqual(data["code"], "source-changed")
# Nothing was written by the refused add; the member follows its folder.
listed = await self.listed()
self.assertEqual((listed[0]["count"], listed[0]["missing"]), (1, 0))
membership = await self.call(
"GET", MEMBERSHIP, root_index="1", root_scope=self.scope(1), path="a.png"
)
self.assertEqual(membership["collection_ids"], [cid])
async def test_counts_exclude_a_missing_member(self):
second = self.two_sources()
self.make("a.png", "b.png")
self.make("c.png", root=second)
made = await self.add(
name="Grids",
items=[self.item("a.png"), self.item("b.png"), self.item("c.png", 1)],
)
other = await self.add(name="Other", items=[self.item("c.png", 1)])
(self.output_dir / "b.png").unlink()
self.api.invalidate_gallery_cache()
listed = await self.listed()
self.assertEqual(
listed,
[
{
"id": made["collection"]["id"],
"name": "Grids",
"count": 2,
"missing": 1,
},
{
"id": other["collection"]["id"],
"name": "Other",
"count": 1,
"missing": 0,
},
],
)
# The member is removable although it is no longer listed.
removed = await self.call(
"DELETE",
ITEMS,
{"collection_id": made["collection"]["id"], "items": [self.item("b.png")]},
)
self.assertEqual(removed["removed"], 1)
async def test_counts_are_null_when_a_source_scan_fails(self):
self.make("a.png")
await self.add(name="Set", items=[self.item("a.png")])
async def broken(_root):
raise OSError("scan failed at /secret/place")
with patch.object(self.api, "_get_gallery_files", side_effect=broken):
data = await self.call("GET", BASE)
self.assertEqual(
data["collections"][0] | {"id": 0},
{"id": 0, "name": "Set", "count": None, "missing": None},
)
self.assertNotIn("secret", json.dumps(data))
async def test_windows_case_spellings_are_one_member(self):
self.make("Sub/A.png")
with patch("py.api.images._path_flavor", ntpath):
made = await self.add(
name="Set", items=[self.item("Sub/A.png"), self.item("SUB/a.PNG")]
)
self.assertEqual((made["added"], made["already"]), (1, 0))
(self.output_dir / "Sub" / "A.png").rename(
self.output_dir / "Sub" / "a.png"
)
(self.output_dir / "Sub").rename(self.output_dir / "sub")
self.api.invalidate_gallery_cache()
listed = await self.listed()
self.assertEqual((listed[0]["count"], listed[0]["missing"]), (1, 0))
membership = await self.call(
"GET",
MEMBERSHIP,
root_index="0",
root_scope=self.scope(),
path="sub/a.png",
)
self.assertEqual(membership["collection_ids"], [made["collection"]["id"]])
response = await self.client.get(
"/prompt_manager/images/output",
params={
"mode": "workspace",
"collection": str(made["collection"]["id"]),
},
)
page = await response.json()
self.assertEqual([i["path"] for i in page["images"]], ["sub/a.png"])
async def test_posix_case_distinct_files_stay_two_members(self):
self.make("A.png", "a.png")
if len(list(self.output_dir.iterdir())) != 2:
self.skipTest("case-insensitive filesystem")
made = await self.add(
name="Set", items=[self.item("A.png"), self.item("a.png")]
)
self.assertEqual(made["added"], 2)
async def test_a_500_never_carries_the_exception_text(self):
self.make("a.png")
made = await self.add(name="Set", items=[self.item("a.png")])
cid = made["collection"]["id"]
def broken(*_args, **_kwargs):
raise sqlite3.OperationalError("disk I/O error at /secret/prompts.db")
cases = (
(
"list_collections",
"GET",
BASE,
None,
{},
"Collections could not be read.",
),
(
"file_collections",
"GET",
MEMBERSHIP,
None,
{"root_index": "0", "root_scope": self.scope(), "path": "a.png"},
"Collections could not be read.",
),
(
"add_to_collection",
"POST",
ITEMS,
{"collection_id": cid, "items": [self.item("a.png")]},
{},
"The collection could not be changed.",
),
(
"remove_from_collection",
"DELETE",
ITEMS,
{"collection_id": cid, "items": [self.item("a.png")]},
{},
"The collection could not be changed.",
),
(
"rename_collection",
"PATCH",
f"{BASE}/{cid}",
{"name": "x"},
{},
"The collection could not be changed.",
),
(
"delete_collection",
"DELETE",
f"{BASE}/{cid}",
None,
{},
"The collection could not be changed.",
),
)
for method_name, method, url, body, params, message in cases:
with self.subTest(method=method_name):
with patch.object(self.api.db, method_name, side_effect=broken):
data = await self.call(method, url, body, 500, **params)
self.assertEqual(data, {"success": False, "error": message})
class TestCollectionListingFilter(CollectionRoutesTestCase):
"""#392's "all N matching" filter carries ``collection`` like the listing."""
async def test_a_filter_with_a_collection_resolves_only_members(self):
second = self.two_sources()
self.make("a.png", "b.png")
self.make("c.png", root=second)
made = await self.add(
name="Set", items=[self.item("a.png"), self.item("c.png", 1)]
)
cid = str(made["collection"]["id"])
selection = await self.api.resolve_listing_filter({"collection": cid})
self.assertEqual(
sorted((index, path) for index, _scope, path in selection.targets),
[(0, "a.png"), (1, "c.png")],
)
selection = await self.api.resolve_listing_filter(
{"collection": cid, "root": "1", "root_scope": self.scope(1)}
)
self.assertEqual([t[2] for t in selection.targets], ["c.png"])
for bad in ("0", "x", "1234567890"):
with self.subTest(bad=bad):
with self.assertRaises(ValueError):
from py.api.images import parse_listing_filter
parse_listing_filter({"collection": bad})
async def test_bulk_category_acts_on_collection_members_only(self):
self.make("a.png", "b.png")
inside = self._save_prompt("Inside", category="Old")
outside = self._save_prompt("Outside", category="Old")
self.api.db.link_image_to_prompt(inside, str(self.output_dir / "a.png"))
self.api.db.link_image_to_prompt(outside, str(self.output_dir / "b.png"))
made = await self.add(name="Set", items=[self.item("a.png")])
cid = str(made["collection"]["id"])
data = await self.call(
"POST",
"/prompt_manager/bulk/category",
{"images_filter": {"collection": cid}, "category": "New"},
)
self.assertEqual((data["updated_count"], data["prompt_count"]), (1, 1))
self.assertEqual(self.api.db.get_prompt_by_id(inside)["category"], "New")
self.assertEqual(self.api.db.get_prompt_by_id(outside)["category"], "Old")
async def test_a_deleted_collection_in_a_bulk_filter_is_a_404(self):
self.make("a.png")
data = await self.call(
"POST",
"/prompt_manager/bulk/category",
{"images_filter": {"collection": "77"}, "category": "New"},
status=404,
)
self.assertEqual(data["code"], "collection-not-found")
with self.assertRaises(Exception) as caught:
await self.api.resolve_listing_filter({"collection": "77"})
self.assertEqual(type(caught.exception).__name__, "CollectionMissing")
class TestReviewFindings(CollectionRoutesTestCase):
"""PR #400 review: loop work, counts snapshot, id and query rules."""
async def test_no_realpath_runs_on_the_event_loop(self):
import os
import threading
names = [f"img{n}.png" for n in range(20)]
self.make(*names)
await self.call("GET", BASE) # warm the listing cache
loop_thread = threading.current_thread()
on_loop = []
real = os.path.realpath
def watched(*args, **kwargs):
if threading.current_thread() is loop_thread:
on_loop.append(args[0])
return real(*args, **kwargs)
items = [self.item(name) for name in names]
cid = None
with patch("os.path.realpath", side_effect=watched):
made = await self.add(name="Set", items=items)
cid = made["collection"]["id"]
await self.call("DELETE", ITEMS, {"collection_id": cid, "items": items[:5]})
await self.call(
"GET",
MEMBERSHIP,
root_index="0",
root_scope=items[0]["root_scope"],
path="img9.png",
)
await self.call("GET", BASE)
self.assertEqual(on_loop, [])
async def test_missing_is_never_negative(self):
self.make("a.png", "b.png")
made = await self.add(
name="Set", items=[self.item("a.png"), self.item("b.png")]
)
cid = made["collection"]["id"]
real = self.api.db.collection_count_snapshot
def raced(root_keys):
members, stored = real(root_keys)
return members, {cid: 1} # a remove landed between two reads
with patch.object(self.api.db, "collection_count_snapshot", side_effect=raced):
listed = await self.listed()
self.assertEqual((listed[0]["count"], listed[0]["missing"]), (2, 0))
async def test_path_ids_follow_the_listing_rule(self):
self.make("a.png")
made = await self.add(name="Set", items=[self.item("a.png")])
cid = made["collection"]["id"]
for raw in ("0", "007", f"0{cid}"):
with self.subTest(raw=raw):
await self.call("PATCH", f"{BASE}/{raw}", {"name": "x"}, 400)
await self.call("DELETE", f"{BASE}/{raw}", None, 400)
self.assertEqual(len(await self.listed()), 1)
async def test_membership_refuses_repeated_query_keys(self):
self.make("a.png", "b.png")
response = await self.client.get(
MEMBERSHIP
+ f"?root_index=0&root_scope={self.scope()}&path=a.png&path=b.png"
)
self.assertEqual(response.status, 400)
response = await self.client.get(
MEMBERSHIP
+ f"?root_index=0&root_index=0&root_scope={self.scope()}&path=a.png"
)
self.assertEqual(response.status, 400)
async def test_deleting_a_collection_leaves_everything_else(self):
from py.api.images import _favorite_root_key
try:
from tests.test_annotation_schema import insert_group_rows, table_rows
except ImportError:
from test_annotation_schema import insert_group_rows, table_rows
self.make("a.png")
db = self.api.db
db.set_image_favorite(_favorite_root_key(self.output_dir), "a.png")
prompt = self._save_prompt("Linked")
db.link_image_to_prompt(prompt, str(self.output_dir / "a.png"))
with db.model.get_connection() as conn:
insert_group_rows(conn, locator=str(self.output_dir / "a.png"))
made = await self.add(name="Set", items=[self.item("a.png")])
tables = (
"image_favorites",
"generated_images",
"prompts",
"tags",
"media_annotation_files",
"media_annotation_bytes",
"media_image_tags",
)
before = table_rows(db.model.get_connection(), tables)
file_bytes = (self.output_dir / "a.png").read_bytes()
await self.call("DELETE", f"{BASE}/{made['collection']['id']}")
self.assertEqual(table_rows(db.model.get_connection(), tables), before)
self.assertEqual((self.output_dir / "a.png").read_bytes(), file_bytes)
self.assertTrue(all(before[t] for t in tables))
+36
View File
@@ -693,6 +693,39 @@ class TestCleanupJob(DuplicateFilesTestCase):
"Moved 2 database links to the kept copies", "\n".join(cleanup.log)
)
async def test_duplicate_removal_moves_collection_memberships(self):
self.trio()
db = self.api.db
root_key = duplicate_files.favorite_root_key(self.output_dir)
grids = db.add_to_collection(name="LoRA grids", items=[(root_key, "b.png")])
grids = grids["collection"]["id"]
other = db.add_to_collection(
name="Other", items=[(root_key, "sub/c.png"), (root_key, "d.png")]
)["collection"]["id"]
db.add_to_collection(collection_id=grids, items=[(root_key, "sub/c.png")])
_plan, cleanup = await self.remove_all_but("a.png")
# The union: a.png joins both; c.png's second grids row was a repeat.
self.assertEqual(db.collection_members(grids, root_key), {"a.png"})
self.assertEqual(db.collection_members(other, root_key), {"a.png", "d.png"})
self.assertEqual(sorted(db.file_collections(root_key, "a.png")), [grids, other])
self.assertEqual(db.file_collections(root_key, "b.png"), [])
self.assertEqual(db.file_collections(root_key, "sub/c.png"), [])
self.assertIn(
"Moved 2 collection memberships to the kept copies", "\n".join(cleanup.log)
)
def test_one_collection_membership_is_singular(self):
job = type(
"Job", (), {"log": [], "log_line": lambda s, line: s.log.append(line)}
)()
totals = dict.fromkeys(
("moved", "merged", "favorites", "tags", "review", "collections"), 0
)
duplicate_files.DuplicateFilesMixin._log_moved_records(
job, {**totals, "collections": 1}
)
self.assertEqual(job.log, ["Moved 1 collection membership to the kept copies"])
async def test_a_failed_removal_leaves_its_links_on_the_kept_copy(self):
self.trio()
prompt = self.save("Stays on c")
@@ -849,8 +882,11 @@ class TestLinkQueries(DuplicateFilesTestCase):
with self.subTest(sql=sql):
self.assertNotIn("SCAN generated_images", plan)
self.assertNotIn("SCAN image_favorites", plan)
self.assertNotIn("SCAN image_collection_items", plan)
finally:
conn.close()
statements = duplicate_files.query_plan_sql()
self.assertTrue(any("image_collection_items" in sql for sql in statements))
class TestReviewHelpers(DuplicateFilesTestCase):
+4 -3
View File
@@ -2556,9 +2556,10 @@ class TestRootScopeServing(ImageAPITestCase):
listing = await self.client.get("/prompt_manager/images/output?mode=workspace")
descriptor = (await listing.json())["images"][0]
self.assertRegex(descriptor.get("root_scope", ""), r"^[0-9a-f]{64}$")
with patch.object(
images_module, "_WORKSPACE_CURSOR_SECRET", b"different-process-key"
):
from utils.workspace_keys import WorkspaceKeys
rotated = WorkspaceKeys(b"n" * 32, b"m" * 32, False)
with patch.object(self.api, "_workspace_keys", return_value=rotated):
await self.assert_no_access("image.png", scope=descriptor["root_scope"])
self.api._get_all_output_dirs = lambda: []
await self.assert_no_access("image.png", scope=descriptor["root_scope"])
+3 -2
View File
@@ -1418,9 +1418,10 @@ class TestPromptInspector(ImageAPITestCase):
async def test_root_scope_rotated_key_and_removed_index_skip_reader_and_db(self):
png(self.output_dir / "image.png", prompt=graph())
descriptor = await self.scoped_descriptor()
import py.api.images as images_module
from utils.workspace_keys import WorkspaceKeys
with patch.object(images_module, "_WORKSPACE_CURSOR_SECRET", b"new-key"):
rotated = WorkspaceKeys(b"n" * 32, b"m" * 32, False)
with patch.object(self.api, "_workspace_keys", return_value=rotated):
await self.assert_scope_skips_access(descriptor.get("root_scope", "0" * 64))
self.api._get_all_output_dirs = lambda: []
await self.assert_scope_skips_access(descriptor.get("root_scope", "0" * 64))
+169
View File
@@ -0,0 +1,169 @@
"""POST /prompt_manager/images/rescope: a stale target's current identity (#430).
A page that holds ``{root_index, root_scope, path}`` after the sources were
renumbered asks where that file is now. The server maps the target only when
its scope was issued, at its old index, for one of the current roots, so the
answer names nothing the page could not already address; a removed or
replaced root, a foreign scope or a changed key stay refused.
"""
import json
import os
import sys
from unittest.mock import patch
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
try:
from test_api_images import ImageAPITestCase, make_png
except ImportError: # imported as tests.test_api_rescope
from tests.test_api_images import ImageAPITestCase, make_png
from utils.workspace_keys import WorkspaceKeys # noqa: E402
RESCOPE = "/prompt_manager/images/rescope"
class TestRescopeRoute(ImageAPITestCase):
async def get_application(self):
app = await super().get_application()
self.other_dir = self.comfy_dir / "other"
self.other_dir.mkdir()
make_png(self.output_dir / "sub" / "a.png")
make_png(self.other_dir / "sub" / "a.png", color=(0, 0, 255))
return app
def target(self, path="sub/a.png", index=0, root=None):
root = root or self.output_dir
return {
"root_index": index,
"root_scope": self.api._gallery_root_scope(root, index),
"path": path,
}
async def post(self, body):
response = await self.client.post(RESCOPE, json=body)
data = await response.json()
self.assertNotIn(str(self.comfy_dir), json.dumps(data))
self.assertEqual(response.headers.get("Cache-Control"), "no-store")
return response.status, data
async def listed(self):
response = await self.client.get(
"/prompt_manager/images/output", params={"mode": "workspace"}
)
return {
(item["root_index"], item["path"]): item
for item in (await response.json())["images"]
}
async def test_an_unchanged_root_answers_the_listing_descriptor(self):
status, data = await self.post({"target": self.target()})
self.assertEqual(status, 200)
self.assertTrue(data["success"])
self.assertEqual(data["image"], (await self.listed())[(0, "sub/a.png")])
async def test_a_renumbered_root_maps_to_its_new_index(self):
held = self.target()
self.api._get_all_output_dirs = lambda: [self.other_dir, self.output_dir]
status, data = await self.post({"target": held})
self.assertEqual(status, 200)
image = data["image"]
self.assertEqual((image["root_index"], image["path"]), (1, "sub/a.png"))
self.assertEqual(
image["root_scope"], self.api._gallery_root_scope(self.output_dir, 1)
)
self.assertEqual(image, (await self.listed())[(1, "sub/a.png")])
async def test_a_removed_replaced_or_foreign_root_is_source_changed(self):
held = self.target()
cases = {
"removed": lambda: [],
"replaced": lambda: [self.other_dir],
}
for name, roots in cases.items():
with self.subTest(name):
self.api._get_all_output_dirs = roots
status, data = await self.post({"target": held})
self.assertEqual((status, data["code"]), (409, "source-changed"))
self.api._get_all_output_dirs = lambda: [self.output_dir]
for scope in ("0" * 64, self.api._gallery_root_scope(self.output_dir, 1)):
with self.subTest(scope=scope[:8]):
status, data = await self.post(
{"target": {**held, "root_scope": scope}}
)
self.assertEqual((status, data["code"]), (409, "source-changed"))
async def test_a_changed_key_cannot_be_mapped(self):
held = self.target()
rotated = WorkspaceKeys(b"n" * 32, b"m" * 32, False)
with patch.object(self.api, "_workspace_keys", return_value=rotated):
status, data = await self.post({"target": held})
self.assertEqual((status, data["code"]), (409, "source-changed"))
async def test_the_first_of_two_spellings_of_one_root_wins(self):
held = self.target()
self.api._get_all_output_dirs = lambda: [
self.other_dir,
self.output_dir,
self.output_dir / "sub" / "..",
]
status, data = await self.post({"target": held})
self.assertEqual((status, data["image"]["root_index"]), (200, 1))
async def test_a_missing_file_is_not_found(self):
status, data = await self.post({"target": self.target("sub/gone.png")})
self.assertEqual((status, data["code"]), (404, "file-missing"))
(self.output_dir / "dir.png").mkdir()
status, data = await self.post({"target": self.target("dir.png")})
self.assertEqual((status, data["code"]), (404, "file-missing"))
async def test_a_path_leaving_the_root_is_refused(self):
outside = make_png(self.comfy_dir / "outside.png")
self._make_symlink(outside, self.output_dir / "escape.png")
status, _data = await self.post({"target": self.target("escape.png")})
self.assertEqual(status, 403)
async def test_previews_are_not_rescoped(self):
make_png(self.output_dir / "thumbnails" / "a_thumb.png")
status, _data = await self.post(
{"target": self.target("thumbnails/a_thumb.png")}
)
self.assertEqual(status, 403)
async def test_malformed_bodies_are_refused_before_any_file_access(self):
good = self.target()
bodies = [
b"{",
[],
{},
{"target": good, "extra": 1},
{"target": {**good, "root_index": -1}},
{"target": {**good, "root_index": True}},
{"target": {**good, "root_index": 4096}},
# Past Python's int-to-text digit limit: never a 500.
b'{"target": {"root_index": 1' + b"0" * 5000 + b"}}",
{"target": {**good, "root_scope": "A" * 64}},
{"target": {**good, "root_scope": "abc"}},
{"target": {**good, "path": ""}},
{"target": {**good, "path": "../a.png"}},
{"target": {**good, "path": "/abs.png"}},
{"target": {**good, "path": "C:/a.png"}},
{"target": {**good, "path": "sub\\a.png"}},
{"target": {**good, "path": "sub//a.png"}},
{"target": {**good, "path": "notes.txt"}},
{"target": {**good, "path": "x" * 2000 + ".png"}},
]
with patch("pathlib.Path.is_file", side_effect=AssertionError("file access")):
for body in bodies:
with self.subTest(body=str(body)[:60]):
if isinstance(body, bytes):
response = await self.client.post(
RESCOPE,
data=body,
headers={"Content-Type": "application/json"},
)
else:
response = await self.client.post(RESCOPE, json=body)
self.assertIn(response.status, (400, 403))
self.assertFalse((await response.json())["success"])
+74 -2
View File
@@ -62,12 +62,16 @@ class TestVocabularyAPI(ImageAPITestCase):
},
)
self.assertEqual(
categories["categories"], [{"value": "Animal", "prompt_count": 1}]
categories["categories"],
[{"value": "Animal", "prompt_count": 1, "image_count": 0}],
)
_, rest = await self.get(
"categories", limit=1, cursor=categories["next_cursor"]
)
self.assertEqual(rest["categories"], [{"value": "Pet", "prompt_count": 1}])
self.assertEqual(
rest["categories"],
[{"value": "Pet", "prompt_count": 1, "image_count": 0}],
)
self.assertFalse(rest["has_more"])
async def test_vocabulary_cursor_is_signed_epoch_scoped(self):
@@ -488,6 +492,7 @@ class TestVocabularyAPI(ImageAPITestCase):
"other_prompts": 0,
"negative_variants": 1,
"archived_negative_variants": 0,
"images": 0,
}
self.assertEqual(
preview,
@@ -528,6 +533,73 @@ class TestVocabularyAPI(ImageAPITestCase):
self.assertEqual(response.status, 409)
self.assertEqual(replay["code"], "vocabulary-conflict")
async def test_category_rows_report_images_and_image_only_values(self):
self._save_prompt("Owner", category="Prompt value")
_, options_before = await self.get_options()
self.api.db.set_image_categories(
[("root", "a.png"), ("root", "b.png")], "Prompt value"
)
self.api.db.set_image_categories([("root", "c.png")], "Image value")
response, first = await self.get("categories", limit=1)
self.assertEqual(response.status, 200)
self.assertEqual(
first["categories"],
[{"value": "Prompt value", "prompt_count": 1, "image_count": 2}],
)
self.assertEqual(first["total"], 2)
_, rest = await self.get("categories", limit=1, cursor=first["next_cursor"])
self.assertEqual(
rest["categories"],
[{"value": "Image value", "prompt_count": 0, "image_count": 1}],
)
self.assertFalse(rest["has_more"])
# The Prompt library's category list does not gain image values.
_, options_after = await self.get_options()
self.assertEqual(options_after, options_before)
self.assertEqual(options_after["categories"], ["Prompt value"])
async def get_options(self):
response = await self.client.get("/prompt_manager/library/options")
return response, await response.json()
async def test_category_routes_rename_image_only_value(self):
epoch = self.api.db.get_execution_epoch()
self._save_prompt("Bystander", category="Kept")
self.api.db.set_image_categories(
[("root", "a.png"), ("second", "b.png")], "Image value"
)
prompts_before = self.category_rows()
body = {"expected_epoch": epoch, "source": "Image value", "target": "Moved"}
response, preview = await self.send("POST", "categories/preview", body)
self.assertEqual(response.status, 200, preview)
impact = {
"positive_prompts": 0,
"other_prompts": 0,
"negative_variants": 0,
"archived_negative_variants": 0,
"images": 2,
}
self.assertEqual(preview["impact"], impact)
stale = preview["impact_token"]
self.api.db.set_image_categories([("root", "late.png")], "Image value")
response, refused = await self.send(
"POST", "categories/apply", {**body, "impact_token": stale}
)
self.assertEqual(response.status, 409)
self.assertEqual(refused["code"], "vocabulary-conflict")
self.assertEqual(self.api.db.image_category("root", "a.png"), "Image value")
_, preview = await self.send("POST", "categories/preview", body)
response, applied = await self.send(
"POST",
"categories/apply",
{**body, "impact_token": preview["impact_token"]},
)
self.assertEqual(response.status, 200, applied)
self.assertEqual(applied["impact"], {**impact, "images": 3})
for key in (("root", "a.png"), ("second", "b.png"), ("root", "late.png")):
self.assertEqual(self.api.db.image_category(*key), "Moved")
self.assertEqual(self.category_rows(), prompts_before)
async def test_preview_validation(self):
epoch = self.api.db.get_execution_epoch()
self._save_prompt("Owner", category="Scenery")
@@ -0,0 +1,262 @@
"""Images filters by the effective category (#332 MC4, MC5, DC6).
An image's effective category is its own category when it has one, else the
category its link supplies (the recipe's when set, else the positive's), else
none. The category filter, "select all N matching" and saved views match it.
"""
import os
import sqlite3
from unittest.mock import patch
try:
import test_api_workspace_images as workspace
from test_image_categories import category_checksum
except ImportError:
from tests import test_api_workspace_images as workspace
from tests.test_image_categories import category_checksum
from py.api.images import image_category_key, parse_listing_filter
class WorkspaceCategoryCase(workspace.ImageAPITestCase):
"""Listing helpers plus own-category writes keyed as the routes key them."""
media = workspace.TestWorkspaceImages.media
get = workspace.TestWorkspaceImages.get
tag = workspace.TestWorkspaceImageTags.tag
names = workspace.TestWorkspaceImageTags.names
async def tearDownAsync(self):
self.api.db.close_all()
workspace.assert_closed(self, self._temp_db.name, self._output_tmp.name)
await super().tearDownAsync()
def own(self, path, value, sources=None):
sources = sources or self.api._get_all_output_dirs()
self.api.db.set_image_categories([image_category_key(sources, path)], value)
async def listed(self, **params):
response, body = await self.get(sort="name", **params)
self.assertEqual(response.status, 200, body)
return self.names(body), body
class TestEffectiveCategoryFilter(WorkspaceCategoryCase):
def batch(self):
"""Four images of "castle at dusk" (Landscapes); one is overridden."""
castle = self._save_prompt("castle at dusk", category="Landscapes", rating=4)
files = [self.media(f"castle_{n}.png", mtime=100 + n) for n in range(4)]
for path in files:
self.api.db.link_image_to_prompt(castle, str(path))
self.own(files[0], "Portraits")
return castle, files
async def test_category_filter_prefers_own_category(self):
castle, _files = self.batch()
conn = self.api.db.model.get_connection()
before = category_checksum(conn)
names, body = await self.listed(category="Portraits")
self.assertEqual(names, ["castle_0.png"])
# The caption still counts the prompt behind the image shown.
self.assertEqual(body["positive_count"], 1)
names, body = await self.listed(category="Landscapes")
self.assertEqual(names, ["castle_1.png", "castle_2.png", "castle_3.png"])
self.assertEqual(body["total"], 3)
names, _ = await self.listed()
self.assertEqual(len(names), 4)
self.assertEqual(category_checksum(conn), before)
self.assertEqual(self.api.db.get_prompt_by_id(castle)["category"], "Landscapes")
async def test_category_filter_lists_unlinked_with_own(self):
scans = [self.media(f"scan_{n}.png", mtime=100 + n) for n in range(2)]
self.media("plain.png", mtime=50)
for path in scans:
self.own(path, "Scans")
names, body = await self.listed(category="Scans")
self.assertEqual(names, ["scan_0.png", "scan_1.png"])
self.assertEqual(body["positive_count"], 0)
async def test_recipe_category_is_inherited(self):
"""MC4: a recipe's own category, not the positive's, is inherited."""
db = self.api.db
self._save_prompt("Pair positive", category="Night")
pair = db.save_prompt_pair("Pair positive", "Noise", category="Recipes")
blank = db.save_prompt_pair("Pair positive", "Other noise", category=" ")
recipe = self.media("recipe.png", mtime=103)
blank_recipe = self.media("blank_recipe.png", mtime=102)
positive = db.get_prompt_pair(pair)["positive_prompt_id"]
for path, pair_id in ((recipe, pair), (blank_recipe, blank)):
image = db.link_image_to_prompt(positive, str(path))
db.link_image_to_pair(pair_id, image, {"source": "fixture"})
self.assertEqual((await self.listed(category="Recipes"))[0], ["recipe.png"])
# A blank recipe category falls back to the positive's.
self.assertEqual((await self.listed(category="Night"))[0], ["blank_recipe.png"])
# An own category still wins over the recipe's.
self.own(recipe, "Mine")
self.assertEqual((await self.listed(category="Recipes"))[0], [])
self.assertEqual((await self.listed(category="Mine"))[0], ["recipe.png"])
async def test_ambiguous_files_inherit_nothing_but_keep_their_own(self):
first = self._save_prompt("Owner one", category="Night")
second = self._save_prompt("Owner two", category="Night")
ambiguous = self.media("ambiguous.png")
for owner in (first, second):
self.api.db.link_image_to_prompt(owner, str(ambiguous))
self.assertEqual((await self.listed(category="Night"))[0], [])
self.own(ambiguous, "Night")
self.assertEqual((await self.listed(category="Night"))[0], ["ambiguous.png"])
async def test_combines_with_tags_search_and_rating(self):
stone = self._save_prompt(
"castle at dusk", category="Landscapes", tags=["stone"], rating=4
)
files = [self.media(f"castle_{n}.png", mtime=100 + n) for n in range(2)]
for path in files:
self.api.db.link_image_to_prompt(stone, str(path))
self.own(files[0], "Portraits")
other = self._save_prompt("harbor at noon", category="Portraits", rating=2)
harbor = self.media("harbor.png", mtime=90)
self.api.db.link_image_to_prompt(other, str(harbor))
names, _ = await self.listed(category="Portraits", tags="stone")
self.assertEqual(names, ["castle_0.png"])
names, _ = await self.listed(category="Portraits", q="harbor")
self.assertEqual(names, ["harbor.png"])
names, _ = await self.listed(category="Portraits", q="castle_0")
self.assertEqual(names, ["castle_0.png"])
response, body = await self.get(category="Portraits", sort="rating")
self.assertEqual(response.status, 200, body)
self.assertEqual(self.names(body), ["castle_0.png", "harbor.png"])
self.assertEqual(body["positive_count"], 2)
async def test_image_tag_admitted_file_keeps_its_inherited_category(self):
night = self._save_prompt("Night owl", category="Night", tags=["a"])
tagged = self.media("tagged.png", mtime=103)
unlinked = self.media("unlinked.png", mtime=102)
self.api.db.link_image_to_prompt(night, str(tagged))
self.tag(tagged, ["b"])
self.tag(unlinked, ["a", "b"])
names, _ = await self.listed(tags="a,b", category="Night")
self.assertEqual(names, ["tagged.png"])
self.own(unlinked, "Night")
names, _ = await self.listed(tags="a,b", category="Night")
self.assertEqual(names, ["tagged.png", "unlinked.png"])
async def test_cursor_pages_walk_only_the_category(self):
files = [self.media(f"img_{n:02}.png", mtime=100 + n) for n in range(7)]
for path in files[::2]:
self.own(path, "Even")
seen, cursor = [], None
while True:
params = {"category": "Even", "limit": 2, "sort": "name"}
if cursor:
params["cursor"] = cursor
response, body = await self.get(**params)
self.assertEqual(response.status, 200, body)
seen += self.names(body)
self.assertEqual(body["total"], 4)
cursor = body["next_cursor"]
if not cursor:
break
self.assertEqual(seen, ["img_00.png", "img_02.png", "img_04.png", "img_06.png"])
async def test_an_outer_source_lists_a_nested_sources_own_category(self):
inner = self.output_dir / "nested"
nested = self.media("nested/a.png")
self.media("b.png")
self.api._get_all_output_dirs = lambda: [self.output_dir, inner]
self.own(nested, "Scans")
names, _ = await self.listed(category="Scans", root="0")
self.assertEqual(names, ["nested/a.png"])
names, _ = await self.listed(category="Scans", root="1")
self.assertEqual(names, ["a.png"])
# The outer source's own key for that path is not the file's key.
from py.api.images import _favorite_root_key
self.api.db.clear_image_categories([image_category_key([inner], nested)])
self.api.db.set_image_categories(
[(_favorite_root_key(self.output_dir), "nested/a.png")], "Stale"
)
self.assertEqual((await self.listed(category="Stale"))[0], [])
async def test_a_gone_files_category_is_not_listed(self):
gone = self.media("gone.png")
self.own(gone, "Scans")
os.unlink(gone)
self.api.invalidate_gallery_cache()
self.assertEqual((await self.listed(category="Scans"))[0], [])
async def test_a_failed_own_category_read_is_listing_failed(self):
self.media("a.png")
def broken(*_args, **_kwargs):
raise sqlite3.OperationalError("secret detail")
with patch.object(self.api.db, "own_categories_of", side_effect=broken):
with self.assertLogs("prompt_manager", level="ERROR"):
response, body = await self.get(category="Scans")
self.assertEqual(response.status, 500)
self.assertNotIn("secret", str(body))
# Without a category nothing new is read.
response, body = await self.get()
self.assertEqual(response.status, 200, body)
async def test_select_all_matching_uses_the_effective_category(self):
_castle, files = self.batch()
resolved = await self.api.resolve_listing_filter(
parse_listing_filter({"category": "Portraits"}), 100
)
self.assertEqual([t[2] for t in resolved.targets], ["castle_0.png"])
resolved = await self.api.resolve_listing_filter(
parse_listing_filter({"category": "Landscapes"}), 100
)
self.assertEqual(len(resolved.targets), 3)
self.assertEqual(len(files), 4)
async def test_saved_view_category_means_the_effective_category(self):
"""CUJ 8: a saved view's category is the listing's query value."""
_castle, files = self.batch()
self.own(files[1], "Portraits")
names, _ = await self.listed(category="Portraits")
self.assertEqual(names, ["castle_0.png", "castle_1.png"])
class TestFilterAndChipAgreeOnBlanks(WorkspaceCategoryCase):
"""The filter and the inspector use one inherited-category rule."""
async def chip(self, path):
response = await self.client.get(
f"/prompt_manager/images/inspect/{path}", params={"root": "0"}
)
body = await response.json()
self.assertEqual(response.status, 200, body)
return body["library"]["category"]
async def test_blank_recipe_values_agree(self):
db = self.api.db
positive = self._save_prompt("Pair positive", category="Night")
blanks = {"spaces": " ", "tab": "\t", "nbsp": " ", "set": "Recipes"}
for index, (label, value) in enumerate(blanks.items()):
pair = db.save_prompt_pair("Pair positive", f"noise {label}")
path = self.media(f"{label}.png", mtime=100 + index)
image = db.link_image_to_prompt(positive, str(path))
db.link_image_to_pair(pair, image, {"source": "fixture"})
with db.model.get_connection() as conn:
conn.execute(
"UPDATE prompt_pairs SET category=? WHERE id=?", (value, pair)
)
blob = db.save_prompt_pair("Pair positive", "noise blob")
path = self.media("blob.png", mtime=90)
image = db.link_image_to_prompt(positive, str(path))
db.link_image_to_pair(blob, image, {"source": "fixture"})
with db.model.get_connection() as conn:
conn.execute(
"UPDATE prompt_pairs SET category=CAST(' ' AS BLOB) WHERE id=?",
(blob,),
)
for name in ("spaces", "tab", "nbsp", "set", "blob"):
with self.subTest(name=name):
shown = await self.chip(f"{name}.png")
for value in ("Night", "Recipes"):
listed = f"{name}.png" in (await self.listed(category=value))[0]
self.assertEqual(listed, shown == value, (name, value, shown))
+279 -15
View File
@@ -1347,7 +1347,7 @@ class TestWorkspaceImages(ImageAPITestCase):
_, data = await self.get(q="harbor")
self.assertEqual(self._names(data), ["night_a.png"])
self.assertEqual(
self.api.db.get_image_paths_matching("zebra", None, []),
self.api.db.get_image_paths_matching("zebra", []),
{
normalize_image_path(str(files[name]))
for name in ("night_ab", "day_ab", "titled")
@@ -1407,8 +1407,10 @@ class TestWorkspaceImages(ImageAPITestCase):
for query in statements
if query.lstrip().upper().startswith(("SELECT", "WITH"))
]
# Plus the one stream of tagged image records q and tags read (B3).
self.assertEqual(len(selects), 4, selects)
# Plus the one stream of tagged image records q and tags read (B3),
# and the category's own-category read of the one source (#332).
self.assertEqual(len(selects), 5, selects)
self.assertEqual(sum("image_categories" in q for q in selects), 1, selects)
self.assertEqual(sum("image_favorites" in q for q in selects), 1, selects)
self.assertEqual(sum("media_image_tags" in q for q in selects), 1, selects)
@@ -1528,15 +1530,16 @@ class TestWorkspaceImages(ImageAPITestCase):
elsewhere = self.api.db.get_prompt_id_for_image("/elsewhere/7.png")
self.assertEqual(
self.api.db.get_image_link_matches(
"", "Night", [], every_link=True, wanted=wanted
"", [], every_link=True, wanted=wanted, category="Night"
),
{
normalize_image_path(str(files["night_ab"])): (
True,
5,
self.api.db.get_prompt_id_for_image(str(files["night_ab"])),
"Night",
),
"/elsewhere/7.png": (True, 5, elsewhere),
"/elsewhere/7.png": (True, 5, elsewhere, "Night"),
},
)
@@ -1588,6 +1591,24 @@ class TestWorkspaceImages(ImageAPITestCase):
for i in range(links)
],
)
# 2,000 images (one in 25 at 50,000 files) carry their own category
# (#332), so a category filter also reads and matches those.
from py.api.images import _favorite_root_key
root_key = _favorite_root_key(self.output_dir)
with self.api.db.model.get_connection() as conn:
conn.executemany(
"INSERT INTO image_categories VALUES(?,?,?,?)",
[
(
root_key,
listed[i][0].relative_to(self.output_dir).as_posix(),
f"cat-{i % 10}",
"2026-10-27T00:00:00+00:00",
)
for i in range(1, files, 25)[:2_000]
],
)
return listed
_BUDGET_REQUESTS = (
@@ -1615,7 +1636,7 @@ class TestWorkspaceImages(ImageAPITestCase):
import math
from py.api import images as images_module
from database.operations import link_filter_query
from database.operations import link_filter_query, link_matches_json_query
listed = await asyncio.get_running_loop().run_in_executor(
None, self._budget_fixture, 5_000, 2_000
@@ -1678,7 +1699,13 @@ class TestWorkspaceImages(ImageAPITestCase):
# q and tags also stream the tagged image records once
# (#249 B3); none are tagged here, so nothing is stat'ed.
streamed = "q" in params or "tags" in params
self.assertEqual(len(found), 2 if streamed else 1, found)
# A category also reads the own categories, one primary
# key range per source (#332).
own = "category" in params
self.assertEqual(len(found), 1 + streamed + own, found)
self.assertEqual(
sum("image_categories" in query for query in found), int(own)
)
self.assertEqual(
sum("media_image_tags" in query for query in found),
int(streamed),
@@ -1738,13 +1765,42 @@ class TestWorkspaceImages(ImageAPITestCase):
self.assertEqual(self.api._workspace_listed_cache, {})
self.assertEqual(self.api._workspace_order_cache, {})
# The per-row ambiguity probe is an index search, never a scan.
sql, params = link_filter_query("lantern", "cat-3", ["tag-1"], every_link=True)
with self.api.db.model.get_connection() as conn:
plan = [row[3] for row in conn.execute("EXPLAIN QUERY PLAN " + sql, params)]
self.assertIn(
"SEARCH other USING COVERING INDEX idx_image_file_path (file_path=?)", plan
# The read runs it wrapped in one JSON aggregate (#354): same plan.
sql, params = link_filter_query("lantern", ["tag-1"], every_link=True)
for query in (sql, link_matches_json_query(sql)):
with self.subTest(wrapped=query is not sql):
with self.api.db.model.get_connection() as conn:
plan = [
row[3]
for row in conn.execute("EXPLAIN QUERY PLAN " + query, params)
]
self.assertIn(
"SEARCH other USING COVERING INDEX idx_image_file_path "
"(file_path=?)",
plan,
)
self.assertFalse(
[step for step in plan if step.startswith("SCAN other")]
)
# The category's narrowing clause (#332) keeps every join an index
# search, plain and wrapped with the raw categories.
sql, params = link_filter_query(
"", ["tag-1"], every_link=True, category="cat-3"
)
self.assertFalse([step for step in plan if step.startswith("SCAN other")])
for query in (sql, link_matches_json_query(sql, categories=True)):
with self.subTest(category_wrapped=query is not sql):
with self.api.db.model.get_connection() as conn:
plan = [
row[3]
for row in conn.execute("EXPLAIN QUERY PLAN " + query, params)
]
self.assertIn(
"SEARCH other USING COVERING INDEX idx_image_file_path "
"(file_path=?)",
plan,
)
scans = [step for step in plan if step.startswith("SCAN")]
self.assertTrue(all(step.startswith("SCAN gi") for step in scans), plan)
@unittest.skipUnless(
PERF_BUDGET and not TRACED,
@@ -1826,9 +1882,13 @@ class TestWorkspaceCursorEncoding(unittest.TestCase):
for order in ("newest", "oldest", "name", "name-desc"):
with self.subTest(sort=order):
key = _workspace_key(entry, order)
token = _workspace_cursor(key, "scope")
token = _workspace_cursor(key, "scope", b"k" * 32)
self.assertLessEqual(len(token), 4096)
self.assertEqual(_workspace_cursor_key(token, "scope", order), key)
self.assertEqual(
_workspace_cursor_key(token, "scope", order, b"k" * 32), key
)
with self.assertRaises(ValueError):
_workspace_cursor_key(token, "scope", order, b"o" * 32)
# The pre-#379 candidate read, kept as the differential reference for the
@@ -2494,3 +2554,207 @@ class TestListingFilterSelection(ImageAPITestCase):
self.assertEqual(response.status, 409)
self.assertEqual((await response.json())["code"], "source-changed")
class TestWorkspaceCollectionFilter(ImageAPITestCase):
"""#331: ``collection=<id>`` keeps only that collection's listed members."""
media = TestWorkspaceImages.media
get = TestWorkspaceImages.get
_names = TestWorkspaceImages._names
async def tearDownAsync(self):
self.api.db.close_all()
assert_closed(self, self._temp_db.name, self._output_tmp.name)
await super().tearDownAsync()
def keys(self, root, relative):
from py.api.images import _favorite_relative_key, _favorite_root_key
return (_favorite_root_key(root), _favorite_relative_key(relative))
def collect(self, name, *members):
"""A collection holding ``(root, relative)`` members; its id."""
result = self.api.db.add_to_collection(
name=name, items=[self.keys(root, rel) for root, rel in members]
)
return result["collection"]["id"]
def two_sources(self):
second = self.comfy_dir / "second"
second.mkdir()
self.api._get_all_output_dirs = lambda: [self.output_dir, second]
return second
async def test_listing_keeps_only_collection_members(self):
second = self.two_sources()
self.media("a.png", 300)
self.media("b.png", 200)
self.media("sub/c.png", 100, root=second)
self.media("d.png", 50, root=second)
grids = self.collect(
"Grids",
(self.output_dir, "a.png"),
(second, "sub/c.png"),
(self.output_dir, "gone.png"), # stored, never listed
)
self.collect("Other", (self.output_dir, "b.png"))
response, data = await self.get(collection=str(grids))
self.assertEqual(response.status, 200, data)
self.assertEqual(self._names(data), ["a.png", "c.png"])
self.assertEqual(data["total"], 2)
_, data = await self.get(collection=str(grids), root="1")
self.assertEqual(self._names(data), ["c.png"])
# Without the filter the listing is unchanged.
_, data = await self.get()
self.assertEqual(data["total"], 4)
async def test_collection_filter_combines_with_the_other_filters(self):
night = self._save_prompt("Neon zebra", category="Night", tags=["a"])
day = self._save_prompt("Morning field", category="Day", tags=["a"])
files = {
name: self.media(name, 100 + index)
for index, name in enumerate(
("night.png", "day.png", "zebra_plain.png", "clip.mp4", "out.png")
)
}
self.api.db.link_image_to_prompt(night, str(files["night.png"]))
self.api.db.link_image_to_prompt(day, str(files["day.png"]))
members = [(self.output_dir, name) for name in files if name != "out.png"]
grids = str(self.collect("Grids", *members))
cases = (
({}, ["clip.mp4", "day.png", "night.png", "zebra_plain.png"]),
({"q": "zebra"}, ["night.png", "zebra_plain.png"]),
({"category": "Night"}, ["night.png"]),
({"tags": "a"}, ["day.png", "night.png"]),
({"media": "video"}, ["clip.mp4"]),
({"media": "image", "q": "day"}, ["day.png"]),
)
for params, expected in cases:
with self.subTest(**params):
response, data = await self.get(collection=grids, sort="name", **params)
self.assertEqual(response.status, 200, data)
self.assertEqual(self._names(data), expected)
self.assertEqual(data["total"], len(expected))
async def test_collection_filter_combines_with_an_images_own_category(self):
"""#331 with #332: a member is admitted by its effective category."""
from py.api.images import image_category_key
night = self._save_prompt("Neon zebra", category="Night")
files = {
name: self.media(name, 100 + index)
for index, name in enumerate(
("linked.png", "overridden.png", "own.png", "outside.png")
)
}
for name in ("linked.png", "overridden.png"):
self.api.db.link_image_to_prompt(night, str(files[name]))
sources = self.api._get_all_output_dirs()
def own(name, value):
key = image_category_key(sources, files[name])
self.api.db.set_image_categories([key], value)
own("overridden.png", "Portraits")
own("own.png", "Night") # unlinked: only its own category
own("outside.png", "Night") # not a member
members = [(self.output_dir, name) for name in files if name != "outside.png"]
grids = str(self.collect("Grids", *members))
for category, expected in (
("Night", ["linked.png", "own.png"]),
("Portraits", ["overridden.png"]),
):
with self.subTest(category=category):
response, data = await self.get(
collection=grids, category=category, sort="name"
)
self.assertEqual(response.status, 200, data)
self.assertEqual(self._names(data), expected)
self.assertEqual(data["total"], len(expected))
async def test_invalid_collection_is_a_400(self):
for value in ("x", "0", "-1", "1.5", "1234567890", " 1"):
with self.subTest(value=value):
response, data = await self.get(collection=value)
self.assertEqual(response.status, 400, data)
async def test_unknown_collection_is_a_404(self):
self.media("a.png")
response, data = await self.get(collection="77")
self.assertEqual(response.status, 404)
self.assertEqual(data["code"], "collection-not-found")
self.assertNotIn("images", data)
self.api._get_all_output_dirs = lambda: []
response, data = await self.get(collection="77")
self.assertEqual((response.status, data["code"]), (404, "collection-not-found"))
async def test_a_collection_deleted_during_the_read_is_a_404(self):
self.media("a.png")
grids = self.collect("Grids", (self.output_dir, "a.png"))
real = self.api.db.collection_members
def deleted_meanwhile(collection_id, root_key):
self.api.db.delete_collection(collection_id)
return real(collection_id, root_key)
with patch.object(
self.api.db, "collection_members", side_effect=deleted_meanwhile
):
response, data = await self.get(collection=str(grids))
self.assertEqual((response.status, data["code"]), (404, "collection-not-found"))
async def test_a_failed_members_read_is_a_500_never_an_unfiltered_page(self):
self.media("a.png")
self.media("b.png")
grids = self.collect("Grids", (self.output_dir, "a.png"))
def broken(*_args, **_kwargs):
raise sqlite3.OperationalError("database is locked")
with patch.object(self.api.db, "collection_members", side_effect=broken):
response, data = await self.get(collection=str(grids))
self.assertEqual(response.status, 500)
self.assertNotIn("images", data)
self.assertNotIn("locked", json.dumps(data))
async def test_reordering_sources_keeps_members(self):
second = self.two_sources()
self.media("a.png", root=second)
self.media("b.png")
grids = str(self.collect("Grids", (second, "a.png")))
_, data = await self.get(collection=grids)
self.assertEqual(
[(i["root_index"], i["path"]) for i in data["images"]], [(1, "a.png")]
)
self.api._get_all_output_dirs = lambda: [second, self.output_dir]
_, data = await self.get(collection=grids)
self.assertEqual(
[(i["root_index"], i["path"]) for i in data["images"]], [(0, "a.png")]
)
async def test_cursor_from_another_collection_is_refused(self):
for index, name in enumerate(("a.png", "b.png", "c.png")):
self.media(name, 100 + index)
everything = [(self.output_dir, name) for name in ("a.png", "b.png", "c.png")]
first = str(self.collect("First", *everything))
second = str(self.collect("Second", *everything))
_, page = await self.get(collection=first, limit=1)
token = page["next_cursor"]
self.assertIsInstance(token, str)
response, _ = await self.get(collection=first, limit=1, cursor=token)
self.assertEqual(response.status, 200)
for params in ({"collection": second}, {}, {"favorites": "1"}):
with self.subTest(**params):
response, _ = await self.get(cursor=token, limit=1, **params)
self.assertEqual(response.status, 400)
async def test_collection_wins_over_favorites(self):
self.media("a.png")
self.media("b.png")
grids = str(self.collect("Grids", (self.output_dir, "a.png")))
from py.api.images import _favorite_root_key
self.api.db.set_image_favorite(_favorite_root_key(self.output_dir), "b.png")
_, data = await self.get(collection=grids, favorites="1")
self.assertEqual(self._names(data), ["a.png"])
+218
View File
@@ -0,0 +1,218 @@
"""A ComfyUI restart keeps an open workspace page's scopes and cursors (#430).
A restart is a new ``PromptManagerAPI`` on the same database in a new
process. Here a second API instance on its own aiohttp server stands in for
it (``tests/test_workspace_keys.py`` also derives the scope in a real child
process); it must accept every ``root_scope`` and listing cursor the first one
handed out, while a changed root list or key still refuses them.
"""
import base64
import json
import os
import sys
from unittest.mock import patch
from urllib.parse import quote
from aiohttp import web
from aiohttp.test_utils import TestClient, TestServer
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
try:
from test_api_images import ImageAPITestCase, make_png
except ImportError: # imported as tests.test_api_workspace_restart
from tests.test_api_images import ImageAPITestCase, make_png
from database.operations import PromptDatabase # noqa: E402
from py.api import PromptManagerAPI # noqa: E402
from utils.workspace_keys import KEY_FILE_NAME, WorkspaceKeys # noqa: E402
LISTING = "/prompt_manager/images/output"
def _payload(token):
raw = base64.urlsafe_b64decode(token + "=" * (-len(token) % 4))
return json.loads(raw[32:])
class TestRestartKeepsTheOpenPage(ImageAPITestCase):
async def get_application(self):
app = await super().get_application()
for index in range(3):
make_png(self.output_dir / f"image_{index}.png")
return app
async def first_page(self, client=None, **params):
response = await (client or self.client).get(
LISTING, params={"mode": "workspace", "limit": "1", **params}
)
self.assertEqual(response.status, 200)
return await response.json()
async def restarted(self):
"""A second API on the same database and roots, on its own server."""
api = PromptManagerAPI(db=PromptDatabase(self._temp_db.name))
api._get_all_output_dirs = lambda: [self.output_dir]
api._find_comfyui_output_dir = lambda: str(self.output_dir)
app = web.Application()
routes = web.RouteTableDef()
api.add_routes(routes)
app.router.add_routes(routes)
client = TestClient(TestServer(app))
await client.start_server()
async def close():
await client.close()
api.close_auto_thumbnails()
api.db.close_all()
self.addAsyncCleanup(close)
return api, client
async def test_a_restart_keeps_every_held_scope_and_cursor_valid(self):
page = await self.first_page()
image, cursor = page["images"][0], page["next_cursor"]
self.assertTrue(
os.path.isfile(
os.path.join(os.path.dirname(self._temp_db.name), KEY_FILE_NAME)
)
)
_api, client = await self.restarted()
more = await client.get(
LISTING, params={"mode": "workspace", "limit": "1", "cursor": cursor}
)
self.assertEqual(more.status, 200, await more.text())
self.assertNotEqual((await more.json())["images"][0]["path"], image["path"])
inspected = await client.get(
f"/prompt_manager/images/inspect/{quote(image['path'])}",
params={"root": "0", "root_scope": image["root_scope"]},
)
self.assertEqual(inspected.status, 200, await inspected.text())
served = await client.get(
f"/prompt_manager/images/serve/{quote(image['path'])}",
params={"root": "0", "root_scope": image["root_scope"]},
)
self.assertEqual(served.status, 200)
relisted = await self.first_page(client)
self.assertEqual(relisted["images"][0]["root_scope"], image["root_scope"])
async def test_a_key_change_refuses_the_cursor_as_stale(self):
cursor = (await self.first_page())["next_cursor"]
rotated = WorkspaceKeys(b"n" * 32, b"m" * 32, False)
with patch.object(self.api, "_workspace_keys", return_value=rotated):
response = await self.client.get(
LISTING, params={"mode": "workspace", "limit": "1", "cursor": cursor}
)
self.assertEqual(response.status, 400)
self.assertEqual((await response.json())["code"], "cursor-stale")
async def test_renumbered_roots_refuse_the_cursor_as_stale(self):
cursor = (await self.first_page())["next_cursor"]
other = self.comfy_dir / "other"
other.mkdir()
self.api._get_all_output_dirs = lambda: [other, self.output_dir]
response = await self.client.get(
LISTING, params={"mode": "workspace", "limit": "1", "cursor": cursor}
)
self.assertEqual(response.status, 400)
self.assertEqual((await response.json())["code"], "cursor-stale")
async def test_another_listing_refuses_the_cursor_as_stale(self):
cursor = (await self.first_page())["next_cursor"]
response = await self.client.get(
LISTING,
params={"mode": "workspace", "limit": "1", "cursor": cursor, "q": "x"},
)
self.assertEqual(response.status, 400)
self.assertEqual((await response.json())["code"], "cursor-stale")
async def test_a_malformed_cursor_is_a_plain_bad_request(self):
# A 32-byte signature over a payload is well formed: a wrong one is stale.
for cursor in ("!!", "a" * 5000, base64.urlsafe_b64encode(b"x" * 20).decode()):
with self.subTest(cursor=cursor[:10]):
response = await self.client.get(
LISTING,
params={"mode": "workspace", "limit": "1", "cursor": cursor},
)
self.assertEqual(response.status, 400)
self.assertNotEqual((await response.json()).get("code"), "cursor-stale")
async def test_the_cursor_carries_no_digest_of_the_root_paths(self):
cursor = (await self.first_page())["next_cursor"]
payload = _payload(cursor)
# v2: the listing scope (it hashes every absolute root) is in the MAC only.
self.assertEqual(set(payload), {"v", "key"})
self.assertEqual(payload["v"], 2)
async def test_another_database_folder_gets_other_scopes(self):
page = await self.first_page()
other_db = self.comfy_dir / "elsewhere" / "prompts.db"
other_db.parent.mkdir()
api = PromptManagerAPI(db=PromptDatabase(str(other_db)))
self.addCleanup(api.db.close_all)
self.addCleanup(api.close_auto_thumbnails)
self.assertNotEqual(
api._gallery_root_scope(self.output_dir, 0), page["images"][0]["root_scope"]
)
# A fresh interpreter: the API on the given database, as after a restart.
_CHILD = """
import json, sys
from pathlib import Path
from unittest.mock import MagicMock
sys.path.insert(0, sys.argv[1])
from tests.host_stubs import install_folder_paths_stub
install_folder_paths_stub()
sys.modules.setdefault("server", MagicMock())
from database.operations import PromptDatabase
from py.api import PromptManagerAPI
from py.api.images import _workspace_cursor_key
api = PromptManagerAPI(db=PromptDatabase(sys.argv[2]))
scope = api._gallery_root_scope(Path(sys.argv[3]), 0)
secret = api._workspace_keys().cursor
key = _workspace_cursor_key(sys.argv[4], "s" * 64, "newest", secret)
api.close_auto_thumbnails()
api.db.close_all()
print(json.dumps({"scope": scope, "key": list(key)}))
"""
class TestRestartInANewProcess(ImageAPITestCase):
"""The real thing: two other interpreters on the same database."""
def child(self, cursor):
import subprocess
repo = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
result = subprocess.run(
[
sys.executable,
"-c",
_CHILD,
repo,
self._temp_db.name,
str(self.output_dir),
cursor,
],
cwd=self.comfy_dir,
capture_output=True,
text=True,
timeout=120,
)
self.assertEqual(result.returncode, 0, result.stderr[-2000:])
return json.loads(result.stdout.strip().splitlines()[-1])
async def test_new_processes_derive_the_same_scope_and_accept_the_cursor(self):
from py.api.images import _workspace_cursor
make_png(self.output_dir / "image.png")
listing = await self.client.get(LISTING, params={"mode": "workspace"})
scope = (await listing.json())["images"][0]["root_scope"]
key = (-12.5, "image.png", "0" * 64, 0)
cursor = _workspace_cursor(key, "s" * 64, self.api._workspace_keys().cursor)
for _restart in range(2):
self.assertEqual(self.child(cursor), {"scope": scope, "key": list(key)})
+43
View File
@@ -553,6 +553,35 @@ class TestStartRefusals(BatchTestCase):
self.assertEqual(self.api._jobs.list(), [])
self.assertIsNone(model_gate.holder())
async def test_a_missing_backend_package_refuses_the_start(self):
self.tagger.missing["gguf"] = ["llama-cpp-python"]
status, data = await self.start_batch(model_type="gguf")
self.assertEqual((status, data["code"]), (409, "model-unavailable"))
self.assertEqual(data["success"], False)
self.assertTrue(
data["error"].startswith("GGUF JoyCaption needs llama-cpp-python."), data
)
self.assertIn("pip install llama-cpp-python", data["error"])
self.assertEqual(self.api._jobs.list(), [])
self.assertEqual(self.tagger.load_calls, [])
self.assertIsNone(model_gate.holder())
async def test_a_package_lost_after_the_start_fails_the_job_without_a_traceback(
self,
):
from py.autotag import MissingBackendPackages
self.image("a.png", (1, 1, 1))
self.tagger.load_error = MissingBackendPackages("gguf", ["llama-cpp-python"])
with self.assertNoLogs(self.api.logger, level="ERROR"):
job = await self.run_batch(model_type="gguf")
self.assertEqual(job.state, "failed")
self.assertTrue(
job.status_line.startswith("GGUF JoyCaption needs"), job.status_line
)
self.assertTrue(any("pip install llama-cpp-python" in line for line in job.log))
self.assertIsNone(model_gate.holder())
async def test_model_busy_when_the_gate_is_held(self):
owner = model_gate.new_owner("image-annotations")
self.assertTrue(model_gate.try_acquire(owner))
@@ -658,6 +687,20 @@ class TestPreview(BatchTestCase):
self.assertIsNone(llm["estimate_device"])
self.assertEqual([s for s in _strings(skip) if _absolute(s)], [])
async def test_preview_names_missing_backend_packages(self):
self.image("a.png", (1, 1, 1))
data = (await self.preview(model_type="gguf"))[1]
self.assertEqual(data["model"]["missing_packages"], [])
self.assertIsNone(data["model"]["install_command"])
self.tagger.missing["gguf"] = ["llama-cpp-python"]
data = (await self.preview(model_type="gguf"))[1]
self.assertTrue(data["model"]["downloaded"])
self.assertEqual(data["model"]["missing_packages"], ["llama-cpp-python"])
self.assertEqual(
data["model"]["install_command"], "pip install llama-cpp-python"
)
self.assertIn("DGGML_CUDA", data["model"]["install_note"])
async def test_preview_uses_the_graphics_card_rate_when_wd14_runs_on_cuda(self):
self.image("a.png", (1, 1, 1))
self.tagger.wd14_device = lambda: "gpu"
+5 -1
View File
@@ -800,8 +800,12 @@ class TestDefaultLoaders(EngineTestCase):
transformers, model, processor, inputs = self._fake_transformers()
pixel = inputs["pixel_values"]
service = AutoTagService(models_dir=self.models_dir)
# The 8-bit load also needs these two (#424); stand-ins satisfy the check.
stubs = {
name: types.ModuleType(name) for name in ("accelerate", "bitsandbytes")
}
with _patch_modules({"torch": torch, "transformers": transformers}):
with _patch_modules({"torch": torch, "transformers": transformers, **stubs}):
service.load_model("hf")
tags = service.generate_tags(str(self.image_path))

Some files were not shown because too many files have changed in this diff Show More