Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8d41f5056c | ||
|
|
a4a7b7129a | ||
|
|
f334004f8f | ||
|
|
0126705045 | ||
|
|
652458e995 | ||
|
|
89d94cdb2b | ||
|
|
b6e0439421 | ||
|
|
b5ca2eda94 | ||
|
|
8870496ea3 | ||
|
|
83199518df | ||
|
|
5e00ee479f | ||
|
|
2c54daf2d8 | ||
|
|
1ce2577733 | ||
|
|
2941a9b748 | ||
|
|
505c16a724 |
@@ -36,6 +36,11 @@ concurrency:
|
||||
# 12m53s, node units and the viewer suite before 11.8m of workflows), so coverage is the tight one
|
||||
# in its 18 minutes. This holds only when the npm cache hits: a cold cache adds up to 5m to npm ci,
|
||||
# and then the job limit can still end the job before the step limit does.
|
||||
# #473: firefox/webkit shard 1/2 reached its 18-minute step limit (queue run 40866252975: webkit 622 tests
|
||||
# passed in 17m56s, 48.5 worker-minutes vs 28.0 in shard 2/2; recent shard 1/2 jobs 13.7m-18.9m, shard 2/2
|
||||
# 7.3m-12.2m), because --shard splits by test count and the slow concept specs sort into shard 1. The page
|
||||
# workflows now run in three shards per browser with the limits unchanged; the per-shard times are
|
||||
# re-measured after three green main runs.
|
||||
jobs:
|
||||
frontend-coverage:
|
||||
name: frontend coverage (shard ${{ matrix.shard }}/3)
|
||||
@@ -175,7 +180,7 @@ jobs:
|
||||
retention-days: 14
|
||||
|
||||
browser-workflows:
|
||||
name: page workflows (${{ matrix.browser }}, shard ${{ matrix.shard }}/2)
|
||||
name: page workflows (${{ matrix.browser }}, shard ${{ matrix.shard }}/3)
|
||||
# Merge queue, main and manual runs only (#420); pull requests run the Chromium/touch coverage shards.
|
||||
if: github.event_name != 'pull_request'
|
||||
runs-on: ubuntu-latest
|
||||
@@ -194,7 +199,7 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
browser: [firefox, webkit]
|
||||
shard: [1, 2]
|
||||
shard: [1, 2, 3]
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
# The image's own Python 3.12 and Node 24: setup-python/setup-node install the runner's toolcache
|
||||
@@ -221,9 +226,9 @@ jobs:
|
||||
path: ~/.npm
|
||||
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
|
||||
- run: npm ci --prefix tools/vendor --ignore-scripts
|
||||
- name: Page workflows in ${{ matrix.browser }}, shard ${{ matrix.shard }}/2
|
||||
- name: Page workflows in ${{ matrix.browser }}, shard ${{ matrix.shard }}/3
|
||||
timeout-minutes: 18
|
||||
run: npm run --prefix tools/vendor test:workflows -- --project ${{ matrix.browser }} --shard ${{ matrix.shard }}/2
|
||||
run: npm run --prefix tools/vendor test:workflows -- --project ${{ matrix.browser }} --shard ${{ matrix.shard }}/3
|
||||
# Traces and the blob report (test-results/blob/) of a failed, timed-out or cancelled step.
|
||||
- name: Upload browser traces
|
||||
if: always()
|
||||
|
||||
+225
-16
@@ -1830,8 +1830,9 @@ Inspect plan keeps D60–D66 (plan D60 → D83, D61 → D84, D65 → D88, D68
|
||||
blocks' context ("Current file image tags" in History). An empty source draws no
|
||||
section. Repeats are removed within a source only, so a linked tag that is also an
|
||||
image tag shows in both sections and each count is true. D21's filter rules are
|
||||
unchanged. History's current-file read carries no `image_tags` facet (D88), so there
|
||||
only the linked section shows. `tests/browser/inspector-concept.spec.mjs` wraps the
|
||||
unchanged. History's current-file read carries the same `image_tags` facet as the Images
|
||||
read (#457), so History's inline current file and its viewer show the image tags section
|
||||
too (`test_current_inspection_carries_the_files_image_tags`). `tests/browser/inspector-concept.spec.mjs` wraps the
|
||||
concept's row in the same sections at runtime (`withTagSections`, the D154 rule
|
||||
injected) and compares the measured sections (frame, heading, label, row); a
|
||||
mutation of the label color, the row margin or the top rule fails it. Browser tests:
|
||||
@@ -2442,10 +2443,38 @@ Recorded deviations on this page:
|
||||
view" filters are active, and plain "Filters" otherwise; the text is its accessible
|
||||
name, so the count is spoken. The concept has no indicator, and without one a Tags
|
||||
filter set by Tags & categories Browse was invisible on the shelf. The Library counts
|
||||
Tags; Images counts Source, Folder, Media and Tags (each once). The toolbar category is
|
||||
Tags, Folder and Tag match (D150); Images counts Source, Folder, Media and Tags (each once). The toolbar category is
|
||||
not a Refine filter and is not counted. The caption's count is a `role="status"`
|
||||
region, written only when it changes and kept shown while the shelf reloads, so
|
||||
Load more and scrolling never re-announce it.
|
||||
- **D150** (#444, cutover #334): "Refine this view" on the Library holds the concept
|
||||
Filters dialog's **Folder** ("All output folders", then each folder below an output
|
||||
root as a tree built by path segment (each folder directly under its own parent,
|
||||
case variants as distinct siblings, siblings case-insensitive then exact), indented
|
||||
by depth, read from `GET /prompt_manager/library/options?include=folders`
|
||||
when the dialog opens; the chosen folder stays listed even when the list lacks it) and
|
||||
**Tags**, plus **Tag match** ("All of these tags" / "Any of these tags" / "Prompts without tags", D149) drawn as the
|
||||
concept's own select; the concept's "Prompt evidence" and prototype note are absent.
|
||||
`GET /prompt_manager/library` takes `folder` (`/`-joined; `\` is a separator only on
|
||||
Windows, on POSIX it is part of a folder's name as the listing shows it; `.`, `..`,
|
||||
empty segments and, on Windows, drives are 400) and `tag_mode` (`all` default, `any`, `untagged` from D149), and echoes
|
||||
both; a client whose non-default value is not echoed (an un-restarted older server)
|
||||
shows the read error, never the unfiltered shelf. A folder keeps a parent with one
|
||||
linked image (`generated_images.prompt_id`) in it or below it under any spelling of
|
||||
any output root (the root as configured or detected and after `realpath`, so a
|
||||
symlinked output folder works): `file_path` (`normcase(normpath(abspath))`) is
|
||||
compared with prefixes built by the same normalization, so case folding is the
|
||||
platform's own, including non-ASCII on Windows. The folder is an outer AND around the
|
||||
parent and pair arms; Any puts one `t.name IN (...)` in each arm, so a card matches
|
||||
when the parent or one active pair carries any tag (text and category still AND
|
||||
inside each arm); with one tag Any equals All. Filters counts Tags, Folder and a Tag
|
||||
match other than All (D44). Saved Library views store `tag_mode` (`all|any|untagged`) and
|
||||
`folder` (text 1024); older rows read the defaults. Tests:
|
||||
`tests/test_library_folders.py` (pure prefixes and names with `posixpath`/`ntpath`,
|
||||
drive, UNC, non-ASCII, symlinked root, the any-tag union), `test_api_library.py`,
|
||||
`test_saved_views.py`, the library-state and saved-views JS units, and
|
||||
`tests/browser/workspace-library-filters.spec.mjs` (behavior, the Refine dialog against
|
||||
the concept Filters dialog, artifact captures). Windows: run `portability.yml` on the PR.
|
||||
|
||||
`tests/browser/library-concept.spec.mjs` compares the shelf (#249 E3) on the
|
||||
concept-shaped fixture, which creates the concept's prompts last to first so Recently
|
||||
@@ -2811,10 +2840,12 @@ mapped differences"):
|
||||
- **D60** (heading action, LI-3): "Open file metadata" (`ca-primary`) replaces the
|
||||
concept's "Open sample metadata". It is `aria-disabled="true"` and
|
||||
`aria-describedby="local-file-note"` (the drop note) until a classified selection is
|
||||
ready, then opens the shared viewer on that file.
|
||||
ready, then opens the shared viewer on that file. Amended by D155: choosing the file
|
||||
already opens it; the action reopens it.
|
||||
- **D61** (copy, LI-3): the surface `p` reads "Read embedded metadata in this browser.
|
||||
The file is not uploaded or added to your library."; the drop note "Your file stays in
|
||||
this browser. Its metadata opens with “Open file metadata”."; the chooser label "Choose
|
||||
this browser. Its metadata opens with “Open file metadata”." (D155: "Your file stays in
|
||||
this browser. Choosing it opens its metadata."); the chooser label "Choose
|
||||
an image or video for a local preview"; the toast "Local preview only. No file
|
||||
uploaded; metadata is available above.". The h2 "Inspect without importing" is the
|
||||
concept's, verbatim.
|
||||
@@ -2868,6 +2899,7 @@ mapped differences"):
|
||||
by the next selection attempt or after the concept's toast interval (6.5 s). The toast
|
||||
"Local preview only…; metadata is available above." shows only for a selection whose
|
||||
status is empty or the AVI/WMV note, never beside an unreadable or failed preview.
|
||||
Amended by D155: only the AVI/WMV note keeps the toast; an empty status opens the viewer.
|
||||
- **D64** (drop, LI-6): `dragover`/`drop` on `.file-drop` accept exactly one File, read
|
||||
only `dataTransfer.files`/`items` (never its strings) and change nothing visually.
|
||||
While the page is mounted, `window` `dragover`/`drop` call `preventDefault`, so a missed
|
||||
@@ -2880,6 +2912,36 @@ mapped differences"):
|
||||
Modified with no Path row and no Show in folder, and ends with "Local file; no saved
|
||||
library association was checked." and Download original. There is no Prompt history
|
||||
action and no category or tag chips.
|
||||
- **D155** (#448, maintainer choice 2026-10-28, beyond the concept): choosing or dropping
|
||||
a file opens the shared viewer on it at once, with `#viewer-details` showing, in one
|
||||
step; the concept previews it and waits for its heading action. The viewer opens once
|
||||
per selection, after the session has selected the record (a microtask after `ready`),
|
||||
only while that selection is current and no other dialog is open. A new selection
|
||||
that starts while the viewer shows the previous file closes it first, so no element
|
||||
holds the old `blob:` URL when the session revokes it (`URLs and media end with their
|
||||
owning session`); the new file's viewer opens once it is ready. A video opens the
|
||||
viewer only once its inline preview has settled (metadata, an error, or D62's 15 s):
|
||||
replacing a video while both the inline and the viewer's video were still starting
|
||||
deadlocked WebKit (`a video replaced while still starting is reset once it settles`
|
||||
hung 2 of 2 runs before, passes 5 of 5 after). It opens for a selection with no status and for one
|
||||
without a preview URL ("The preview could not be displayed. Metadata and the original
|
||||
remain available in the viewer."); an unreadable file stays on the page with its
|
||||
alert, and AVI/WMV keeps its note and the toast (the viewer could not play it and
|
||||
reads no metadata); "Open file metadata" opens either. An image or video that fails to
|
||||
decode after the viewer opened shows the viewer's own media error; the page's alert
|
||||
is there on close. The toast no longer shows for an opened file (the viewer is the
|
||||
confirmation), and the drop note reads "Your file stays in this browser. Choosing it
|
||||
opens its metadata.". On this page the viewer, by choice or by the heading action,
|
||||
opens with the details showing even when "Collapse viewer details by default" is on
|
||||
(D97); the toggle is still drawn then and starts as "Hide details" (`aria-expanded`
|
||||
true). Closing returns focus to the chooser (the heading action when it opened the
|
||||
viewer), with the preview, the chooser naming the file and the heading action in
|
||||
place; the next file chosen or dropped replaces it and opens again. A drop is still
|
||||
ignored while the viewer is open (D64). Nothing else changes: the file is read in this
|
||||
browser only. Browser tests: "D155: choosing a file opens the viewer with its details
|
||||
in one click", "D155: Inspect shows the details even when the viewer collapses them by
|
||||
default (amends D97)", "D155: an unreadable file stays on the page with its alert" and
|
||||
"D155: at 390px the opened details are reachable below the media".
|
||||
- **D117** (Copy settings summary, LI-4): the legacy metadata
|
||||
page's COPY ALL, consolidated into the shared full inspector (the viewer's
|
||||
`#viewer-details`, so Images, the recorded history's current file and Inspect all have
|
||||
@@ -3238,7 +3300,8 @@ until this browser dismissed the 3.2.0 update notice (`pm_update_notice_dismisse
|
||||
the legacy admin page's key); Later dismisses it without scanning, and Review scan
|
||||
dismisses it as the legacy Scan does. "+ Add source" asks validate-source, then adds
|
||||
the folder to the host draft of `gallery_root_paths`; Remove takes one out; "Watch for
|
||||
new outputs" (`monitoring_enabled`) and Worker threads (`worker_threads`) are host
|
||||
new outputs" (`monitoring_enabled`), "Save prompts from other workflows"
|
||||
(`save_foreign_runs`, D158) and Worker threads (`worker_threads`) are host
|
||||
drafts too, and Save writes them with the revision. A save the server refuses because a
|
||||
scan runs toasts "Another job is running: <title>." with a link to that job (D57) and
|
||||
keeps the drafts; a save refused for one source says the server's message as the status
|
||||
@@ -3487,7 +3550,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged
|
||||
`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
|
||||
reader can keep it byte-identical (`test_health_check_never_writes`).
|
||||
|
||||
Recorded deviations (plan D60–D75, D79–D82 renumbered D92–D107, D109–D110 and D111–D114; D121–D134 assigned). D94,
|
||||
Recorded deviations (plan D60–D75, D79–D82 renumbered D92–D107, D109–D110 and D111–D114; D121–D134 and D158 assigned). D94,
|
||||
the per-section hand-off panel, was retired by S12 (#329): every section is native.
|
||||
|
||||
- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
|
||||
@@ -3558,10 +3621,41 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
card's title and file line, Info and Tag stay at the end of the row), "Collapse viewer
|
||||
details by default" (`sidebarCollapsedByDefault`; the viewer opens with
|
||||
`#viewer-details` hidden and a "Show details"/"Hide details" toggle with
|
||||
`aria-expanded`, the stage taking the whole width), and the operation "Reset display
|
||||
`aria-expanded`, the stage taking the whole width; amended by D155: on Inspect a file
|
||||
the viewer opens with the details showing and the toggle reading "Hide details"; amended by D156: the row moved
|
||||
to the "Viewer" surface), and the operation "Reset display
|
||||
defaults" ("Restore this browser’s display defaults, then save. Other settings are
|
||||
kept.", button "Reset"), which drafts the gallery's defaults for the browser rows only.
|
||||
The node search timeout shows the server's real value, not the sample 30. CSS under D97.
|
||||
- **D156** (#450, maintainer request 2026-10-28, beyond the concept): every viewer (the
|
||||
shared `dialog#viewer-dialog` of Images, Library, History, collections and Inspect) has
|
||||
`button#viewer-mode.icon-btn` in `.dialog-header`, right before ✕ (a direct child, so
|
||||
the concept's `.dialog-header > button` and D34's coarse 40px rule cover it; 8px from
|
||||
✕). It names the view it switches to in its accessible name and `title` ("Full screen
|
||||
view" / "Centered view") with a `maximize` / `minimize` outline icon, and keeps focus
|
||||
after a switch. Full screen (`#viewer-dialog.viewer-fullscreen`) fills the browser
|
||||
window edge to edge: `inset: 0`, no width or height cap, no margin, border or radius
|
||||
(`.ca-tools #viewer-dialog.viewer-fullscreen[open]`, beating the concept's box, D34's
|
||||
max-height and the ≤700px height); the layout inside is the concept's. Centered is the
|
||||
concept's framed dialog, unchanged. In full screen the details stay toggleable: the
|
||||
D97 "Show details"/"Hide details" toggle is in the toolbar whenever the view is full
|
||||
screen or "Collapse viewer details by default" is on; going back to Centered without
|
||||
that setting removes the toggle and shows the details. The view is not the browser's
|
||||
Fullscreen API (Esc keeps one meaning, closing the viewer; WebKit has no element
|
||||
fullscreen for a modal dialog; the workspace also runs in ComfyUI popups). The viewer
|
||||
opens in the view System › Appearance › **Viewer** › "Open viewer in" (Centered,
|
||||
Full screen) chooses: a browser display preference, `viewerOpenMode` (`centered` by
|
||||
default, normalized in `gallery-settings.js`) in the shared `gallerySettings`; no
|
||||
server setting. The button changes only the current view (navigation in the viewer
|
||||
keeps it; the next open uses the setting). "Viewer" is its own surface after
|
||||
"Browsing & display" ("How the image viewer opens; the button beside ✕ switches the
|
||||
current view."), holding "Open viewer in" and the moved "Collapse viewer details by
|
||||
default"; "Reset display defaults" still restores both. At ≤700px (the concept's
|
||||
breakpoint, where the viewer is already 96vw × 94dvh) the button is hidden
|
||||
(`display: none !important` against the concept's `.icon-btn` rule), popup windows
|
||||
that narrow included; the stored view still applies there. Browser tests:
|
||||
`tests/browser/workspace-viewer-mode.spec.mjs`; the concept comparisons add the
|
||||
button to the concept's header at runtime and map the Viewer surface.
|
||||
- **D98** (Sources): each configured source's row has a plain Remove after its scan
|
||||
button; nothing changes until Save. A source added this session reads "<path> · not
|
||||
saved yet" with Remove only until Save; a configured source the server cannot use
|
||||
@@ -3976,6 +4070,26 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
Cancel-only review matches the concept review shell` (system-tagging-concept.spec) and
|
||||
the batch tests of workspace-system-tagging.spec; the mutation check renames each grid
|
||||
label in turn.
|
||||
- **D158** (Sources, #247 D-Q4): the concept has no switch for prompts from workflows
|
||||
without a PromptManager node; the maintainer chose a visible switch over a config key.
|
||||
A `.setting-row` in the concept's toggle markup follows "Watch for new outputs":
|
||||
"Save prompts from other workflows" (`#foreign-runs`, host key `save_foreign_runs`,
|
||||
config.json `gallery.monitoring.save_foreign_runs`, default on), "New outputs of
|
||||
workflows without a PromptManager node add their prompt to the library when it is
|
||||
certain.". It is saved with the revision like the other host rows and applied at once
|
||||
(the monitor reads it for every file), so a save names no restart. While "Watch for
|
||||
new outputs" is drafted off, a `p.small` under the description says "Applies while
|
||||
Watch for new outputs is on." (the switch's `aria-describedby`; the value is kept) and
|
||||
the switch is disabled, except while a source still reads "watching until restart"
|
||||
(D128): the running observer still delivers files then, so the value can be set. The
|
||||
server stores no foreign prompt while `monitoring_enabled` is saved off, whatever the
|
||||
observer does.
|
||||
Pinned by `sources matches the concept` (the mapping inserts the row after the
|
||||
monitoring row), `sources comparison detects mutations` (a missing row and a changed
|
||||
label are caught), `D158: Save prompts from other workflows saves save_foreign_runs
|
||||
live; off while monitoring is off (#247)` (keyboard on desktop, a tap on touch) and
|
||||
`D158: saved off while the monitor still watches until a restart, the foreign row can
|
||||
still be set (D128)` (workspace-system-sources.spec) and the system-sources node test.
|
||||
- **D109** (Logs, S12, #329): the concept's Logs surface on the live server. The toolbar
|
||||
keeps "Level", Refresh and the download (renamed "Download log": the active log,
|
||||
`/logs/download/prompt_manager.log`, through the api client), with all six minimum
|
||||
@@ -5903,6 +6017,70 @@ Tests: `tests/test_file_prompt.py` (one test per ladder step and junk class),
|
||||
saver, a legacy-linked file left as it is, `_BatchHistory.recorded`) and
|
||||
`tests/test_recorded_saver_anchor.py` `TestInspectionResolution`.
|
||||
|
||||
**In the monitor.** An output whose workflow has no positive PromptManager node
|
||||
(`_find_prompt_from_metadata` says `foreign`) used to be logged and skipped. It now
|
||||
goes through the same ladder with the `MONITOR` policy (`utils/foreign_runs.py`, the
|
||||
handler's `_ingest_foreign`, `_recheck_foreign`, `_store_foreign`):
|
||||
|
||||
- **At settle.** The ladder runs without history. Certain (`pair` or `positive`): the
|
||||
prompt is stored and the file linked at once. `history` (an undecided graph inside
|
||||
ComfyUI's output folder): the file is registered in its own waiting set, then the
|
||||
ladder runs again at once with the file's recorded output, because the run may have
|
||||
been sealed before the file settled. When that lookup finds a record, its answer is
|
||||
final: certain is stored, anything else is logged "No certain prompt for <file>;
|
||||
left unlinked" and the file stops waiting (no seal is coming). When it finds none,
|
||||
only the inspector's or the recorded saver's answer is stored; anything else (the
|
||||
legacy floor included) keeps waiting for the re-check. Anything else at the first
|
||||
pass is logged "No certain prompt for <file>; left unlinked".
|
||||
- **After the run ends.** When the run's outputs are sealed, each waiting foreign file
|
||||
gets one re-check (`ForeignRecheck(reference)`) with the sealed output reference,
|
||||
so history is asked with the spelling it recorded (a Windows folder whose case
|
||||
differs from the disk still links). Not certain: "No certain prompt for <file>
|
||||
after its run ended; left unlinked". An ambiguous inspector answer is never given
|
||||
the legacy floor in the monitor (D-Q3); an untraceable one still is.
|
||||
- **Waiting sets.** `_awaiting_foreign` and `_foreign_due` sit beside the keyless sets
|
||||
with the same bounds (1000 files, one hour) but are separate, so a large foreign run
|
||||
never evicts a waiting keyless file. An expired entry is logged "No recorded run
|
||||
for <file>; left unlinked", an evicted one (waiting or due) "Too many files wait for
|
||||
their run; <file> left unlinked", both after the lock is released. A due re-check
|
||||
is dropped only when it was scheduled, never a newer reference for the same file. `awaiting_foreign()` lists them;
|
||||
`shutdown()` and `close()` clear them, and a closed handler re-checks nothing.
|
||||
- **Store.** One write through `survive_switch` (rerun after a Move library): a pair
|
||||
through `ingest_resolved_pair(..., marks=RUN_MARKS)`, a positive through
|
||||
`store_scanned_prompt_or_raise(..., RUN_MARKS)`. A created prompt carries category
|
||||
`scanned`, tag `auto-scanned` and the note `Saved from a workflow run: <file>`; a
|
||||
file linked to another prompt is `conflict` and keeps its link. Logged at info
|
||||
"Foreign run: <status> prompt <id> for <file> (<source>)"; a failure is logged and
|
||||
the file stays unlinked.
|
||||
- **Never.** No tracker FIFO entry is read or consumed, nothing is counted (run count
|
||||
stays 0, no `last_used_at`), and nothing is queued for Needs review.
|
||||
- **The setting.** `GalleryConfig.SAVE_FOREIGN_RUNS` (`gallery.monitoring.
|
||||
save_foreign_runs`, default true, only a bool applies) is read for every file, at
|
||||
settle, at the re-check and again just before the store, together with
|
||||
`MONITORING_ENABLED`: either saved off stops foreign writes at once (the running
|
||||
observer watches until a restart, D128); off logs "No PromptManager prompt in
|
||||
workflow, skipping image" as before. System › Sources & scans "Save prompts from other workflows" (D158)
|
||||
saves it through `POST /prompt_manager/settings` with no restart field; the
|
||||
diagnostics report lists it.
|
||||
|
||||
Tests: `tests/test_run_attribution.py` `TestMonitorForeignRuns` (dataset branches
|
||||
linked after the seal, a seal before settle, a name two runs reported, the setting
|
||||
off at settle and at the re-check, a plain workflow linked at settle with no run
|
||||
count and the tracker queue untouched, the sealed reference's spelling, the separate
|
||||
bound, expiry, a seal while the pending set is full, shutdown, a failed store, no
|
||||
record at settle keeps waiting, a record without an answer ends the wait, monitoring
|
||||
saved off, the setting read before the store, a seal between registering and the
|
||||
settle lookup, a newer due reference kept, evictions logged, one link per file),
|
||||
`tests/test_image_linking.py` `TestForeignWorkflows`, `tests/test_image_monitor_singleton.py`
|
||||
(close drops a foreign re-check), `tests/test_config.py`,
|
||||
`tests/test_settings_sources.py` `TestSaveForeignRunsSetting`, and the real-host
|
||||
`tests/comfyui/image-attribution.spec.mjs` case "an output of a workflow without a
|
||||
PromptManager node creates and links its prompt; a second run reuses it" (ComfyUI's
|
||||
own `CLIPTextEncode`, `KSampler` and `SaveImage`), and "each branch of a
|
||||
shared-prefix workflow without a PromptManager node links to its own prompt after the
|
||||
run" (two branches, one save prefix: the recorded saver names each file's branch).
|
||||
The fixture route's GET also lists `foreign`, the waiting foreign files.
|
||||
|
||||
## Linked prompt text (#343)
|
||||
|
||||
A link drives `text` on every PromptManager node, and `negative_text` on the Pair,
|
||||
@@ -6183,7 +6361,7 @@ The versioned public reads are:
|
||||
| `GET /prompt_manager/library/{positive_id}/history/pairs/{pair_id}/newest` | The pair's newest recorded run for the history run dot (#249 D26): `{run: {run_id, anchor_id, context_id}}` or `{run: null}` |
|
||||
| `GET /prompt_manager/history/runs/{run_id}` | Frozen fields, per-field provenance, capture and outcome for a known UUID |
|
||||
| `GET /prompt_manager/history/runs/{run_id}/outputs` | Ordered host-reported occurrences, including repeated locations |
|
||||
| `GET /prompt_manager/history/runs/{run_id}/output/current/{media,thumbnail,inspection}` | Authorized current file at that reported location, using the returned epoch/scope URL |
|
||||
| `GET /prompt_manager/history/runs/{run_id}/output/current/{media,thumbnail,inspection}` | Authorized current file at that reported location, using the returned epoch/scope URL; `inspection` carries the shared inspector envelope including the file's `image_tags` facet (#457) |
|
||||
|
||||
The newest-pair read takes no query. It answers 404 when the positive is not an
|
||||
eligible saved prompt or the pair is not one of its pairs. It walks the positive's exact
|
||||
@@ -6487,15 +6665,16 @@ Deploying this backend needs a ComfyUI restart (Python and route changes).
|
||||
|
||||
`#/tags` ("Tags & categories", page title "Tags") is a native workspace route; it no
|
||||
longer hands off to the legacy tag manager. That page stays reachable from Overview's
|
||||
"Open existing tag manager" card for tag search and sort, the untagged entry and
|
||||
multi-tag AND/OR browsing; bulk prompt delete stays on the legacy admin page (bulk tags:
|
||||
the Prompt library's selection bar, D151). A
|
||||
prompt's category is changed in the Prompt library editor, on many prompts at once with
|
||||
the Prompt library's selection bar (D148, #437), or with the legacy admin page's "Set
|
||||
Category" (both `POST /prompt_manager/bulk/category`);
|
||||
"Open existing tag manager" card for its own page; tag search and sort, the untagged
|
||||
entry (D149) and multi-tag OR (the Library's Tag match, D150) moved to the workspace
|
||||
(#444); bulk prompt delete stays on the legacy admin page (bulk tags: the Prompt
|
||||
library's selection bar, D151). A prompt's category is changed in the Prompt library
|
||||
editor, on many prompts at once with the Prompt library's selection bar (D148, #437), or
|
||||
with the legacy admin page's "Set Category" (both `POST /prompt_manager/bulk/category`);
|
||||
the Images selection bar's Set category sets images' own categories only (D140, #332),
|
||||
and Manage category renames a value on prompts, recipes and images together (D144). The screen consumes only the `/prompt_manager/vocabulary/*` routes
|
||||
above and adds no route, so PR B needs no ComfyUI restart: a browser refresh is enough.
|
||||
above and adds no route, so PR B needed no ComfyUI restart; D149 (#444) adds read
|
||||
parameters to `GET tags` and `GET /prompt_manager/library`, which do.
|
||||
|
||||
Screen: a `+ Create tag` heading action, a Tag vocabulary table (columns Tag, Prompts,
|
||||
Images counting the files that carry the tag (`image_count`, #249 C2), and Actions with Browse and
|
||||
@@ -6547,6 +6726,36 @@ Apply; "Apply" instead of "Apply in prototype"; Delete tag… inside Rename or m
|
||||
Browse sets the exact filter; singular "1 positive prompt"; the production list appends
|
||||
fixture-only tags after the concept's sample tags.
|
||||
|
||||
- **D149** (#444, cutover #334; the concept's renderTags has neither): the Tag vocabulary
|
||||
surface shows, after its paragraph, an "Untagged prompts" `.operation` row ("N positive
|
||||
prompts", Browse) and then the tag search and sort, drawn as the concept page toolbar's
|
||||
`search.toolbar` > `label.search-field` (search icon, `input#tag-search` "Search tags…",
|
||||
maxlength 200) and a `select#tag-sort` (Oldest first, the default and the old order;
|
||||
Name A–Z; Name Z–A; Most prompts; Fewest prompts). Search is a literal, case-insensitive
|
||||
substring on the server (`GET /prompt_manager/vocabulary/tags?search=&sort=`; ASCII-only
|
||||
case folding, like SQLite `LIKE`/`NOCASE`), debounced 250 ms; it resets only the tag list
|
||||
to page one, keeps through command re-reads, reads "No tags match this search." when
|
||||
empty and is announced in a visually hidden status ("N tags"). Every sort breaks ties by
|
||||
ascending id; a cursor belongs to one search and sort (scope-hashed; another is 400 with
|
||||
"refresh"); a count sort's counts can move between pages only through a write, which
|
||||
re-reads from page one. Each tags page reports `untagged_count` (eligible Library parents
|
||||
with no tag and no active pair carrying one, the complement of every tag's Browse; legacy
|
||||
counted every prompt without `prompt_tags` and hid the entry at 0; this row always
|
||||
shows) and echoes `search` and `sort`. Browse opens the Library with `tag_mode=untagged`
|
||||
(`GET /prompt_manager/library`, echoed as `tag_mode`; with a `tag` it is 400): "Filters ·
|
||||
1", and Refine this view adds **Tag match** after Tags ("All of these tags" / "Any of these tags" (D150) / "Prompts
|
||||
without tags", the latter disables Tags and drops them on Apply). A client whose request
|
||||
the server did not echo (an un-restarted older server) shows the read error with Retry,
|
||||
never an unfiltered list. Saved Library views store `tag_mode` (`all|any|untagged`, default
|
||||
`all`); untagged with tags is refused on write and reads back as `all`; an older build
|
||||
opens an untagged view as the whole Library. The concept comparison holds the row to the
|
||||
concept's first category row, its top to the concept table's offset, the table to that
|
||||
offset plus the additions' height (`d149`) and the toolbar to the concept Library
|
||||
toolbar (`toolbarInventoryOf`), with a mutation check. Tests:
|
||||
`tests/test_vocabulary.py` (`TestVocabularyTagSearchAndSort`, `TestLibraryTagMode`),
|
||||
`test_api_vocabulary.py`, `test_api_library.py`, `test_saved_views.py`, the
|
||||
vocabulary/library/saved-views JS units and `tests/browser/workspace-vocabulary-search.spec.mjs`
|
||||
(behavior plus artifact captures).
|
||||
- **D143** (#332): category rows count the images whose own category is the value
|
||||
(`image_count`, the stored rows, missing files included): "N positive prompts · M
|
||||
images" beside prompts, "M images" for an image-only value, which follows every
|
||||
|
||||
@@ -7,7 +7,11 @@ from typing import Callable, ContextManager, Iterator
|
||||
|
||||
try:
|
||||
from ..utils.execution_history_events import RunFinish, encode_command
|
||||
from ..utils.hashing import stored_prompt_text, stored_prompt_text_sql
|
||||
from ..utils.hashing import (
|
||||
generate_prompt_hash,
|
||||
stored_prompt_text,
|
||||
stored_prompt_text_sql,
|
||||
)
|
||||
from ..utils.execution_contexts import (
|
||||
CAPTURE_REASONS,
|
||||
ContextBundle,
|
||||
@@ -16,7 +20,11 @@ try:
|
||||
)
|
||||
except ImportError:
|
||||
from utils.execution_history_events import RunFinish, encode_command
|
||||
from utils.hashing import stored_prompt_text, stored_prompt_text_sql
|
||||
from utils.hashing import (
|
||||
generate_prompt_hash,
|
||||
stored_prompt_text,
|
||||
stored_prompt_text_sql,
|
||||
)
|
||||
from utils.execution_contexts import (
|
||||
CAPTURE_REASONS,
|
||||
ContextBundle,
|
||||
@@ -149,9 +157,12 @@ def _role_hints(role: str) -> str:
|
||||
|
||||
# Set form of the read-time prompt-hint rule in get_execution_contexts():
|
||||
# a stored role hint is live only in the current storage epoch, and only when
|
||||
# exactly one prompt has its raw snapshot text or that text's stored form
|
||||
# (outer whitespace stripped, #438), and that is the hinted one (_prompt_hint).
|
||||
# Evaluated once per statement, never per prompt row.
|
||||
# the hinted prompt is the one match of _prompt_hint: by text (the raw
|
||||
# snapshot text or its stored form, outer whitespace stripped, #438), or, only
|
||||
# when no prompt has either text, by dedup hash (case folded too, #460).
|
||||
# The hash is a fallback, so a hint #438 kept is never made ambiguous.
|
||||
# Needs register_prompt_functions (pm_prompt_hash). The hint rows are the
|
||||
# outer loop (CROSS JOIN), so each prompts lookup is by index.
|
||||
# Defines the CTE ``live_hint_prompts(prompt_id)``; prefix it with WITH.
|
||||
LIVE_HINT_CTES = (
|
||||
"stored_hints(prompt_id, raw, stored) AS ("
|
||||
@@ -160,7 +171,12 @@ LIVE_HINT_CTES = (
|
||||
+ _role_hints("negative")
|
||||
+ "), hint_matches(prompt_id, raw, stored, match_id) AS ("
|
||||
"SELECT h.prompt_id, h.raw, h.stored, p.id FROM stored_hints h "
|
||||
"JOIN prompts p ON p.text COLLATE BINARY IN (h.raw, h.stored)"
|
||||
"CROSS JOIN prompts p ON p.text COLLATE BINARY IN (h.raw, h.stored) "
|
||||
"UNION ALL "
|
||||
"SELECT h.prompt_id, h.raw, h.stored, p.id FROM stored_hints h "
|
||||
"CROSS JOIN prompts p ON p.hash = pm_prompt_hash(h.raw) "
|
||||
"WHERE NOT EXISTS (SELECT 1 FROM prompts q "
|
||||
"WHERE q.text COLLATE BINARY IN (h.raw, h.stored))"
|
||||
"), live_hint_prompts(prompt_id) AS ("
|
||||
"SELECT prompt_id FROM hint_matches GROUP BY prompt_id, raw, stored "
|
||||
"HAVING COUNT(*) = 1 AND MIN(match_id) = prompt_id)"
|
||||
@@ -169,20 +185,32 @@ LIVE_HINT_PROMPT_IDS_SQL = (
|
||||
"WITH " + LIVE_HINT_CTES + " SELECT DISTINCT prompt_id FROM live_hint_prompts"
|
||||
)
|
||||
|
||||
_BY_TEXT = "SELECT id FROM prompts WHERE text COLLATE BINARY IN (?,?) LIMIT 2"
|
||||
|
||||
|
||||
def _text_matches(conn: sqlite3.Connection, raw: str) -> list:
|
||||
return [row[0] for row in conn.execute(_BY_TEXT, (raw, stored_prompt_text(raw)))]
|
||||
|
||||
|
||||
def _prompt_hint(conn: sqlite3.Connection, raw: str | None) -> int | None:
|
||||
"""The one prompt whose text is ``raw`` or the stored form of it (#438).
|
||||
"""The one prompt that stores ``raw`` (#438), else the one it counts toward.
|
||||
|
||||
The Library stores ``text.strip()``; the exact form keeps hints of older
|
||||
rows whose stored text kept outer whitespace.
|
||||
By text first: ``raw`` or ``raw.strip()`` (the exact form keeps hints of
|
||||
older rows that kept outer whitespace). Only when no prompt has either
|
||||
text, by the dedup hash, which also folds case (#460).
|
||||
"""
|
||||
if raw is None:
|
||||
return None
|
||||
rows = conn.execute(
|
||||
"SELECT id FROM prompts WHERE text COLLATE BINARY IN (?,?) LIMIT 2",
|
||||
(raw, stored_prompt_text(raw)),
|
||||
).fetchall()
|
||||
return rows[0][0] if len(rows) == 1 else None
|
||||
matches = _text_matches(conn, raw)
|
||||
if not matches:
|
||||
matches = [
|
||||
row[0]
|
||||
for row in conn.execute(
|
||||
"SELECT id FROM prompts WHERE hash=? LIMIT 2",
|
||||
(generate_prompt_hash(raw),),
|
||||
)
|
||||
]
|
||||
return matches[0] if len(matches) == 1 else None
|
||||
|
||||
|
||||
def _hints(
|
||||
@@ -191,24 +219,16 @@ def _hints(
|
||||
pos_id, neg_id = _prompt_hint(conn, positive), _prompt_hint(conn, negative)
|
||||
stored = None if negative is None else stored_prompt_text(negative)
|
||||
pair_id = None
|
||||
# A known negative that names no prompt because several share its text
|
||||
negative_ambiguous = (
|
||||
stored is not None
|
||||
and neg_id is None
|
||||
and conn.execute(
|
||||
"SELECT 1 FROM prompts WHERE text COLLATE BINARY IN (?,?) LIMIT 1",
|
||||
(negative, stored),
|
||||
).fetchone()
|
||||
is not None
|
||||
stored is not None and neg_id is None and bool(_text_matches(conn, negative))
|
||||
)
|
||||
if pos_id is not None and stored is not None and not negative_ambiguous:
|
||||
rows = conn.execute(
|
||||
"SELECT pp.id FROM prompt_pairs pp LEFT JOIN prompts n ON "
|
||||
"n.id=pp.negative_prompt_id "
|
||||
"SELECT pp.id FROM prompt_pairs pp "
|
||||
"WHERE pp.positive_prompt_id=? AND ((pp.negative_prompt_id IS NULL AND "
|
||||
"?='') "
|
||||
"OR (pp.negative_prompt_id=? AND n.text COLLATE BINARY IN (?,?))) "
|
||||
"LIMIT 2",
|
||||
(pos_id, stored, neg_id, negative, stored),
|
||||
"?='') OR pp.negative_prompt_id=?) LIMIT 2",
|
||||
(pos_id, stored, neg_id),
|
||||
).fetchall()
|
||||
if len(rows) == 1:
|
||||
pair_id = rows[0][0]
|
||||
|
||||
+3
-7
@@ -23,10 +23,10 @@ from pathlib import PurePosixPath, PureWindowsPath
|
||||
from urllib.parse import quote
|
||||
|
||||
try:
|
||||
from ..utils.hashing import generate_prompt_hash
|
||||
from ..utils.hashing import register_prompt_functions
|
||||
from .models import normalize_image_path
|
||||
except ImportError:
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.hashing import register_prompt_functions
|
||||
from database.models import normalize_image_path
|
||||
|
||||
logger = logging.getLogger("prompt_manager.database.health")
|
||||
@@ -125,12 +125,8 @@ def _foreign_keys(conn, _roots):
|
||||
)
|
||||
|
||||
|
||||
def _prompt_hash(text):
|
||||
return generate_prompt_hash(text) if isinstance(text, str) else None
|
||||
|
||||
|
||||
def _canonical_hashes(conn, _roots):
|
||||
conn.create_function("pm_prompt_hash", 1, _prompt_hash, deterministic=True)
|
||||
register_prompt_functions(conn) # pm_prompt_hash, shared with history (#460)
|
||||
return _counted(
|
||||
f"Prompt {row[0]}: the stored hash does not match its text"
|
||||
for row in conn.execute(
|
||||
|
||||
+23
-16
@@ -16,9 +16,9 @@ from .execution_outputs import _capture, _REFERENCE_COLUMNS
|
||||
from .library import _ELIGIBLE
|
||||
|
||||
try:
|
||||
from ..utils.hashing import stored_prompt_text, stored_prompt_text_sql
|
||||
from ..utils.hashing import prompt_key, stored_prompt_text
|
||||
except ImportError:
|
||||
from utils.hashing import stored_prompt_text, stored_prompt_text_sql
|
||||
from utils.hashing import prompt_key, stored_prompt_text
|
||||
|
||||
RunKey = tuple[int, str, int, str]
|
||||
|
||||
@@ -115,12 +115,12 @@ _MATCHING = (
|
||||
)
|
||||
|
||||
|
||||
def _scope_text_ids(conn: sqlite3.Connection, scope: str) -> str:
|
||||
"""Every recorded raw text whose stored form is ``scope``, as a JSON array.
|
||||
def _scope_text_ids(conn: sqlite3.Connection, key: str) -> str:
|
||||
"""Every recorded raw text whose prompt key is ``key``, as a JSON array.
|
||||
|
||||
The Library stores ``text.strip()`` while a run keeps its exact raw text,
|
||||
so a raw text joins the scope of the prompt it saves (#438). The values
|
||||
are compared, never only their digests.
|
||||
A run keeps its exact raw text, while the Library saves and counts it as
|
||||
the prompt with its key (outer whitespace #438, case #460). The values are
|
||||
compared, never only their digests; ``pm_prompt_key`` is Python's key.
|
||||
"""
|
||||
return json.dumps(
|
||||
[
|
||||
@@ -130,16 +130,16 @@ def _scope_text_ids(conn: sqlite3.Connection, scope: str) -> str:
|
||||
"SELECT t.text_id FROM (SELECT DISTINCT raw_text_id AS text_id "
|
||||
"FROM execution_prompt_snapshots) s CROSS JOIN "
|
||||
"execution_context_texts t WHERE t.text_id=s.text_id AND "
|
||||
f"{stored_prompt_text_sql('t.text_value')}=? COLLATE BINARY",
|
||||
(scope,),
|
||||
"pm_prompt_key(t.text_value)=? COLLATE BINARY",
|
||||
(key,),
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def _scope_text_id(raw: str | None) -> str | None:
|
||||
"""The scope digest of a raw text: that of the text the Library stores."""
|
||||
return None if raw is None else content_digest("text", (stored_prompt_text(raw),))
|
||||
"""The scope digest of a raw text: that of its prompt key."""
|
||||
return None if raw is None else content_digest("text", (prompt_key(raw),))
|
||||
|
||||
|
||||
def _limit(limit: int) -> None:
|
||||
@@ -290,7 +290,14 @@ class HistoryReadOperations:
|
||||
return None
|
||||
text = saved["text"]
|
||||
raw_id = _scope_text_id(text)
|
||||
if after is not None and after.raw_text_id != raw_id:
|
||||
# A position from before #460 names the scope by the stripped text
|
||||
# alone; it is the same scope, so a page open across the upgrade resumes.
|
||||
legacy_id = (
|
||||
None
|
||||
if text is None
|
||||
else content_digest("text", (stored_prompt_text(text),))
|
||||
)
|
||||
if after is not None and after.raw_text_id not in (raw_id, legacy_id):
|
||||
raise ValueError("History position text scope changed")
|
||||
group = {
|
||||
"positive_id": positive_id,
|
||||
@@ -309,7 +316,7 @@ class HistoryReadOperations:
|
||||
)
|
||||
if raw_id is None:
|
||||
return result
|
||||
scope = _scope_text_ids(conn, stored_prompt_text(text))
|
||||
scope = _scope_text_ids(conn, prompt_key(text))
|
||||
where = f"r.run_id IN ({_MATCHING})"
|
||||
params = [scope]
|
||||
if after is not None:
|
||||
@@ -465,7 +472,7 @@ class HistoryReadOperations:
|
||||
if saved["text"] is None:
|
||||
return {"run": None}
|
||||
epoch = self.get_execution_epoch()
|
||||
stored = stored_prompt_text(saved["text"])
|
||||
key = prompt_key(saved["text"])
|
||||
candidates = conn.execute(
|
||||
"SELECT r.run_id,rc.anchor_id,rc.context_id,"
|
||||
"c.positive_snapshot_id,c.negative_snapshot_id "
|
||||
@@ -477,7 +484,7 @@ class HistoryReadOperations:
|
||||
"AND rc.reference_epoch=? "
|
||||
f"ORDER BY {_ORDER},rc.anchor_id COLLATE BINARY LIMIT ?",
|
||||
(
|
||||
_scope_text_ids(conn, stored),
|
||||
_scope_text_ids(conn, key),
|
||||
pair_id,
|
||||
epoch,
|
||||
_PAIR_RUN_CANDIDATES,
|
||||
@@ -489,7 +496,7 @@ class HistoryReadOperations:
|
||||
if (
|
||||
positive is None
|
||||
or positive["raw"] is None
|
||||
or stored_prompt_text(positive["raw"]) != stored
|
||||
or prompt_key(positive["raw"]) != key
|
||||
):
|
||||
continue
|
||||
hints = _hints(conn, positive["raw"], negative["raw"] if negative else None)
|
||||
|
||||
+84
-16
@@ -5,6 +5,7 @@ import sqlite3
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
|
||||
from .library_folders import folder_condition, list_library_folders
|
||||
from .library_tokens import library_edit_token
|
||||
|
||||
_ELIGIBLE = """(EXISTS(SELECT 1 FROM prompt_pairs ep WHERE ep.positive_prompt_id=p.id)
|
||||
@@ -121,7 +122,54 @@ def _archive(value: str) -> str:
|
||||
return ARCHIVE_SCOPES[value]
|
||||
|
||||
|
||||
def _filter(text: str, category: str | None, tags: list[str]) -> tuple[str, list]:
|
||||
# How the Tags filter matches (#444): every tag (AND), any of them (OR), or
|
||||
# parents without any tag.
|
||||
TAG_MODES = ("all", "any", "untagged")
|
||||
# A parent that no tag's Browse returns (#444 "Untagged prompts"): no tag of its
|
||||
# own and no active pair carrying one. The vocabulary's untagged count uses it too.
|
||||
UNTAGGED = (
|
||||
"NOT EXISTS(SELECT 1 FROM prompt_tags ut WHERE ut.prompt_id=p.id) "
|
||||
"AND NOT EXISTS(SELECT 1 FROM prompt_pairs up JOIN prompt_pair_tags upt "
|
||||
"ON upt.pair_id=up.id WHERE up.positive_prompt_id=p.id AND up.archived=0)"
|
||||
)
|
||||
|
||||
|
||||
def _tag_mode(tag_mode: object, tags: list[str]) -> str:
|
||||
if not isinstance(tag_mode, str) or tag_mode not in TAG_MODES:
|
||||
raise ValueError("tag_mode must be one of " + ", ".join(TAG_MODES))
|
||||
if tag_mode == "untagged" and tags:
|
||||
raise ValueError("Untagged prompts cannot also filter by tags")
|
||||
return tag_mode
|
||||
|
||||
|
||||
def _tag_conditions(tags: list[str], any_tag: bool) -> tuple[list, list, list]:
|
||||
"""Parent-arm and pair-arm tag conditions and the tags' parameters."""
|
||||
parent_exists = (
|
||||
"EXISTS(SELECT 1 FROM prompt_tags pt JOIN tags t ON t.id=pt.tag_id "
|
||||
"WHERE pt.prompt_id=p.id AND t.name{test})"
|
||||
)
|
||||
pair_exists = (
|
||||
"EXISTS(SELECT 1 FROM prompt_pair_tags pt JOIN tags t ON t.id=pt.tag_id "
|
||||
"WHERE pt.pair_id=pp.id AND t.name{test})"
|
||||
)
|
||||
if any_tag and tags:
|
||||
# One condition per arm: the card, or one active pair, carries any tag.
|
||||
test = " IN (" + ",".join("?" for _ in tags) + ")"
|
||||
return [parent_exists.format(test=test)], [pair_exists.format(test=test)], tags
|
||||
return (
|
||||
[parent_exists.format(test="=?") for _ in tags],
|
||||
[pair_exists.format(test="=?") for _ in tags],
|
||||
tags,
|
||||
)
|
||||
|
||||
|
||||
def _filter(
|
||||
text: str,
|
||||
category: str | None,
|
||||
tags: list[str],
|
||||
tag_mode: str = "all",
|
||||
folder_prefixes: list[str] | None = None,
|
||||
) -> tuple[str, list]:
|
||||
if not isinstance(text, str) or len(text) > 500:
|
||||
raise ValueError("Search text must be a string of at most 500 characters")
|
||||
if category is not None and (not isinstance(category, str) or len(category) > 200):
|
||||
@@ -132,6 +180,8 @@ def _filter(text: str, category: str | None, tags: list[str]) -> tuple[str, list
|
||||
or any(not isinstance(tag, str) or not tag or len(tag) > 200 for tag in tags)
|
||||
):
|
||||
raise ValueError("Invalid library tags")
|
||||
mode = _tag_mode(tag_mode, tags)
|
||||
untagged = mode == "untagged"
|
||||
parent, pair, parent_params, pair_params = [], ["pp.archived=0"], [], []
|
||||
if text:
|
||||
parent.append("(p.text LIKE ? ESCAPE '\\' OR ls.title LIKE ? ESCAPE '\\')")
|
||||
@@ -146,17 +196,11 @@ def _filter(text: str, category: str | None, tags: list[str]) -> tuple[str, list
|
||||
pair.append("pp.category=?")
|
||||
parent_params.append(category)
|
||||
pair_params.append(category)
|
||||
for tag in tags:
|
||||
parent.append(
|
||||
"EXISTS(SELECT 1 FROM prompt_tags pt JOIN tags t ON t.id=pt.tag_id "
|
||||
"WHERE pt.prompt_id=p.id AND t.name=?)"
|
||||
)
|
||||
pair.append(
|
||||
"EXISTS(SELECT 1 FROM prompt_pair_tags pt JOIN tags t ON t.id=pt.tag_id "
|
||||
"WHERE pt.pair_id=pp.id AND t.name=?)"
|
||||
)
|
||||
parent_params.append(tag)
|
||||
pair_params.append(tag)
|
||||
parent_tags, pair_tags, tag_params = _tag_conditions(tags, mode == "any")
|
||||
parent.extend(parent_tags)
|
||||
pair.extend(pair_tags)
|
||||
parent_params.extend(tag_params)
|
||||
pair_params.extend(tag_params)
|
||||
condition = (
|
||||
"("
|
||||
+ (" AND ".join(parent) or "1=1")
|
||||
@@ -168,7 +212,17 @@ def _filter(text: str, category: str | None, tags: list[str]) -> tuple[str, list
|
||||
+ " AND ".join(pair)
|
||||
+ ")"
|
||||
)
|
||||
return f"({_ELIGIBLE}) AND ({condition})", [*parent_params, *pair_params]
|
||||
if untagged:
|
||||
# A whole-card condition: inside the parent-or-pair OR an untagged pair
|
||||
# would let a tagged parent through.
|
||||
condition = f"({condition}) AND {UNTAGGED}"
|
||||
params = [*parent_params, *pair_params]
|
||||
if folder_prefixes:
|
||||
# A whole-card condition around both arms: images link to the parent.
|
||||
folder, folder_params = folder_condition(folder_prefixes)
|
||||
condition = f"({condition}) AND {folder}"
|
||||
params.extend(folder_params)
|
||||
return f"({_ELIGIBLE}) AND ({condition})", params
|
||||
|
||||
|
||||
class LibraryOperations:
|
||||
@@ -251,6 +305,8 @@ class LibraryOperations:
|
||||
text: str = "",
|
||||
category: str | None = None,
|
||||
tags: list[str] | None = None,
|
||||
tag_mode: str = "all",
|
||||
folder_prefixes: list[str] | None = None,
|
||||
sort: str = "newest",
|
||||
limit: int = 40,
|
||||
after: dict | None = None,
|
||||
@@ -258,7 +314,9 @@ class LibraryOperations:
|
||||
_limit(limit)
|
||||
if sort not in _ORDERS:
|
||||
raise ValueError("Unknown library sort order")
|
||||
where, params = _filter(text, category, [] if tags is None else tags)
|
||||
where, params = _filter(
|
||||
text, category, [] if tags is None else tags, tag_mode, folder_prefixes
|
||||
)
|
||||
if after is not None:
|
||||
if (
|
||||
not isinstance(after, dict)
|
||||
@@ -311,6 +369,8 @@ class LibraryOperations:
|
||||
"items": rows,
|
||||
"total": total,
|
||||
"variant_total": variant_total,
|
||||
# Echoed so a client can tell this read applied its tag match.
|
||||
"tag_mode": tag_mode,
|
||||
"has_more": has_more,
|
||||
"next_key": next_key,
|
||||
}
|
||||
@@ -321,6 +381,8 @@ class LibraryOperations:
|
||||
text: str = "",
|
||||
category: str | None = None,
|
||||
tags: list[str] | None = None,
|
||||
tag_mode: str = "all",
|
||||
folder_prefixes: list[str] | None = None,
|
||||
sort: str = "newest",
|
||||
limit: int = 40,
|
||||
after: dict | None = None,
|
||||
@@ -333,6 +395,8 @@ class LibraryOperations:
|
||||
text=text,
|
||||
category=category,
|
||||
tags=tags,
|
||||
tag_mode=tag_mode,
|
||||
folder_prefixes=folder_prefixes,
|
||||
sort=sort,
|
||||
limit=limit,
|
||||
after=after,
|
||||
@@ -463,9 +527,13 @@ class LibraryOperations:
|
||||
"categories": [row["category"] for row in rows],
|
||||
}
|
||||
|
||||
def get_library_options(self) -> dict:
|
||||
def get_library_options(self, folder_roots=None) -> dict:
|
||||
"""Categories; with *folder_roots* (root spellings) also ``folders``."""
|
||||
with self.library_snapshot() as (conn, epoch, _ceiling):
|
||||
return self._get_library_options(conn, epoch)
|
||||
result = self._get_library_options(conn, epoch)
|
||||
if folder_roots is not None:
|
||||
result["folders"] = list_library_folders(conn, folder_roots, _ELIGIBLE)
|
||||
return result
|
||||
|
||||
def get_library_previews(self, positive_ids: list[int]) -> dict[int, list[dict]]:
|
||||
"""Read at most two links per parent in one bounded page query."""
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
"""Library output-folder filter (#444): folder names, root prefixes and listing.
|
||||
|
||||
A folder is a ``/``-joined path relative to an output root, as Images' folder
|
||||
filter and the legacy ``/prompt_manager/subfolders`` name them. A parent
|
||||
matches a folder when one of its linked images (``generated_images.prompt_id``,
|
||||
the link its previews and image count read) lies in that folder or below it,
|
||||
under any spelling of any output root.
|
||||
|
||||
Matching compares ``generated_images.file_path``, which is always
|
||||
``normcase(normpath(abspath(path)))`` (``models.normalize_image_path``), with
|
||||
prefixes built by the same normalization here, so case folding is the
|
||||
platform's own on both sides (Windows folds non-ASCII too, unlike SQLite's
|
||||
``lower()``) and the comparison is an exact ``substr`` with no wildcards.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sqlite3
|
||||
|
||||
_MAX_FOLDER = 1024
|
||||
_MAX_PREFIXES = 64
|
||||
|
||||
|
||||
def normalize_folder(value: object, pathmod=os.path) -> str:
|
||||
"""The folder as ``a/b``, outer slashes dropped; empty means no filter.
|
||||
|
||||
``/`` always separates. ``\\`` separates only on Windows (``ntpath``): on
|
||||
POSIX it is an ordinary character of a folder's name, and the listing
|
||||
(``relative_folder``) names such a folder with it, so both agree.
|
||||
``.``/``..``/empty segments, control characters and (on Windows) a drive
|
||||
are refused: the value only ever names a folder below a root.
|
||||
"""
|
||||
if not isinstance(value, str) or len(value) > _MAX_FOLDER:
|
||||
raise ValueError(f"folder must be text of at most {_MAX_FOLDER} characters")
|
||||
windows = pathmod.sep == "\\"
|
||||
folder = (value.replace("\\", "/") if windows else value).strip("/")
|
||||
if not folder:
|
||||
return ""
|
||||
parts = folder.split("/")
|
||||
if (
|
||||
any(part in ("", ".", "..") for part in parts)
|
||||
or (windows and ":" in parts[0])
|
||||
or any(ord(char) < 32 or ord(char) == 127 for char in folder)
|
||||
):
|
||||
raise ValueError("folder must be a folder below an output root")
|
||||
return folder
|
||||
|
||||
|
||||
def _base(root: object, pathmod) -> str:
|
||||
return pathmod.normcase(pathmod.normpath(str(root)))
|
||||
|
||||
|
||||
def folder_prefixes(roots, folder: str, pathmod=os.path) -> list[str]:
|
||||
"""One ``file_path`` prefix per root spelling for a normalized folder."""
|
||||
prefixes = []
|
||||
for root in roots:
|
||||
joined = pathmod.join(_base(root, pathmod), *folder.split("/"))
|
||||
prefix = pathmod.normcase(pathmod.normpath(joined)) + pathmod.sep
|
||||
if prefix not in prefixes:
|
||||
prefixes.append(prefix)
|
||||
if not prefixes or len(prefixes) > _MAX_PREFIXES:
|
||||
raise ValueError("No output folder to filter by")
|
||||
return prefixes
|
||||
|
||||
|
||||
def folder_condition(prefixes: list[str]) -> tuple[str, list]:
|
||||
"""A condition on alias ``p``: one linked image under any prefix."""
|
||||
tests = " OR ".join("substr(fg.file_path,1,?)=?" for _ in prefixes)
|
||||
params = [value for prefix in prefixes for value in (len(prefix), prefix)]
|
||||
return (
|
||||
"EXISTS(SELECT 1 FROM generated_images fg WHERE fg.prompt_id=p.id "
|
||||
f"AND ({tests}))",
|
||||
params,
|
||||
)
|
||||
|
||||
|
||||
def relative_folder(image_path: str, roots, pathmod=os.path) -> str | None:
|
||||
"""The folder of *image_path* below the first root spelling holding it.
|
||||
|
||||
The spelling of the stored path is kept (its case on Windows); ``None``
|
||||
for an image directly in a root or outside every root.
|
||||
"""
|
||||
directory = pathmod.normpath(pathmod.dirname(image_path))
|
||||
folded = pathmod.normcase(directory)
|
||||
for root in roots:
|
||||
base = _base(root, pathmod)
|
||||
start = base if base.endswith(pathmod.sep) else base + pathmod.sep
|
||||
if not folded.startswith(start):
|
||||
continue
|
||||
# normcase keeps the length for ordinary names; else show the folded form.
|
||||
rest = (directory if len(directory) == len(folded) else folded)[len(start) :]
|
||||
parts = rest.split(pathmod.sep)
|
||||
if any(part in ("", ".", "..") for part in parts):
|
||||
return None
|
||||
return "/".join(parts)
|
||||
return None
|
||||
|
||||
|
||||
def list_library_folders(
|
||||
conn: sqlite3.Connection, roots, eligible: str, pathmod=os.path
|
||||
) -> list[str]:
|
||||
"""Every folder (and ancestor) holding an image of an eligible parent."""
|
||||
folders = set()
|
||||
for (image_path,) in conn.execute(
|
||||
"SELECT DISTINCT g.image_path FROM generated_images g "
|
||||
"JOIN prompts p ON p.id=g.prompt_id WHERE " + eligible
|
||||
):
|
||||
if not isinstance(image_path, str) or not image_path:
|
||||
continue
|
||||
folder = relative_folder(image_path, roots, pathmod)
|
||||
if folder is None:
|
||||
continue
|
||||
parts = folder.split("/")
|
||||
folders.update("/".join(parts[:index]) for index in range(1, len(parts) + 1))
|
||||
return sorted(folders, key=lambda value: (value.casefold(), value))
|
||||
@@ -44,6 +44,11 @@ from .health import BATCH_ROWS, link_states, readable_roots, readonly_uri
|
||||
from .pairs import PairDependencyError
|
||||
from .workflow_documents import documents_of_rows, release_documents
|
||||
|
||||
try:
|
||||
from ..utils.hashing import register_prompt_functions
|
||||
except ImportError:
|
||||
from utils.hashing import register_prompt_functions
|
||||
|
||||
MAINTENANCE_OPERATIONS = (
|
||||
"duplicate_prompts",
|
||||
"optimize",
|
||||
@@ -289,6 +294,7 @@ def _read_snapshot(db_path, read):
|
||||
conn = sqlite3.connect(readonly_uri(db_path), uri=True)
|
||||
try:
|
||||
conn.row_factory = sqlite3.Row
|
||||
register_prompt_functions(conn) # LIVE_HINT_CTES calls them (#460)
|
||||
conn.execute("BEGIN") # one read snapshot for the whole plan
|
||||
return read(conn)
|
||||
finally:
|
||||
|
||||
@@ -38,6 +38,11 @@ from .review_schema import (
|
||||
review_without_annotations_error,
|
||||
)
|
||||
from .saved_views import migrate_saved_views, saved_views_schema_error
|
||||
|
||||
try:
|
||||
from ..utils.hashing import register_prompt_functions
|
||||
except ImportError:
|
||||
from utils.hashing import register_prompt_functions
|
||||
from .image_categories import image_categories_schema_error, migrate_image_categories
|
||||
from .workflow_documents import (
|
||||
DOCUMENT_ID_COLUMNS,
|
||||
@@ -153,12 +158,15 @@ def configure_connection(conn: sqlite3.Connection) -> None:
|
||||
(common on Windows) let other writers' polling busy handlers starve past
|
||||
BUSY_TIMEOUT_SECONDS and fail with "database is locked" (#242). Without
|
||||
WAL (for example an in-memory database) SQLite's FULL default is kept.
|
||||
It also registers the prompt-key SQL functions (register_prompt_functions).
|
||||
"""
|
||||
journal_mode = conn.execute("PRAGMA journal_mode = WAL").fetchone()[0]
|
||||
conn.execute("PRAGMA foreign_keys = ON")
|
||||
conn.execute(f"PRAGMA busy_timeout = {BUSY_TIMEOUT_SECONDS * 1000}")
|
||||
if str(journal_mode).lower() == "wal":
|
||||
conn.execute("PRAGMA synchronous = NORMAL")
|
||||
# History reads and the live-hint rule call Python's prompt key (#460)
|
||||
register_prompt_functions(conn)
|
||||
|
||||
|
||||
class PromptModel:
|
||||
|
||||
@@ -33,6 +33,9 @@ MAX_TAG_LENGTH = 50
|
||||
_IMAGE_SORTS = ("newest", "oldest", "name", "name-desc", "rating")
|
||||
_LIBRARY_SORTS = ("newest", "oldest", "name", "rating")
|
||||
_MEDIA = ("all", "image", "video")
|
||||
# How a Library view's Tags match (#444): every tag, any of them, or parents
|
||||
# without tags ("untagged" never goes with tags; the listing refuses both).
|
||||
_TAG_MODES = ("all", "any", "untagged")
|
||||
_CONTROL = re.compile(r"[\x00-\x1f\x7f]")
|
||||
_SOURCE = re.compile(r"all|0|[1-9][0-9]{0,8}")
|
||||
# A collection id as the listing's ``collection`` query takes it (#331): none
|
||||
@@ -67,6 +70,9 @@ _STATE_FIELDS = {
|
||||
"category": ("text", 100, ""),
|
||||
"sort": ("choice", _LIBRARY_SORTS, "newest"),
|
||||
"tags": ("tags", None, ()),
|
||||
# #444: the Tags match and the output folder (relative, "/"-joined).
|
||||
"tag_mode": ("choice", _TAG_MODES, "all"),
|
||||
"folder": ("text", 1024, ""),
|
||||
},
|
||||
}
|
||||
|
||||
@@ -152,6 +158,8 @@ def encode_state(page: object, state: object) -> str:
|
||||
_validate_value(key, kind, limit, state[key])
|
||||
if state.get("favorites") is True and state.get("collection"):
|
||||
raise SavedViewInvalid("A view cannot filter by favorites and a collection")
|
||||
if state.get("tag_mode") == "untagged" and state.get("tags"):
|
||||
raise SavedViewInvalid("A view cannot list untagged prompts and filter tags")
|
||||
encoded = json.dumps(
|
||||
{key: state[key] for key in fields}, ensure_ascii=False, separators=(",", ":")
|
||||
)
|
||||
@@ -195,6 +203,9 @@ def normalize_state(page: str, raw: object) -> dict:
|
||||
# as the listing does for a URL with both (#331 D140).
|
||||
if state.get("favorites") is True and state.get("collection"):
|
||||
state["favorites"] = False
|
||||
# Never written by this build either: the tags win.
|
||||
if state.get("tag_mode") == "untagged" and state.get("tags"):
|
||||
state["tag_mode"] = "all"
|
||||
return state
|
||||
|
||||
|
||||
|
||||
+133
-10
@@ -20,7 +20,7 @@ counted, hashed into the token and renamed with the prompts and pairs.
|
||||
|
||||
import sqlite3
|
||||
|
||||
from .library import _ELIGIBLE, _limit
|
||||
from .library import _ELIGIBLE, UNTAGGED, _escape, _limit
|
||||
from .library_commands import _TOKEN, LibraryCommandError, _validate_epoch
|
||||
from .library_tokens import canonical_token
|
||||
from .models import utc_now_iso
|
||||
@@ -38,7 +38,7 @@ _IMAGE_FILES = (
|
||||
)
|
||||
|
||||
|
||||
def _tag_rows(page: str) -> str:
|
||||
def _tag_rows(page: str, order: str = "page.id") -> str:
|
||||
"""Count rows for the registry rows selected by ``page`` (id, name)."""
|
||||
# Eligible parent components carrying a tag in the page, plus the
|
||||
# positive parent of every active pair carrying it (always eligible).
|
||||
@@ -57,11 +57,45 @@ def _tag_rows(page: str) -> str:
|
||||
"COALESCE(MAX(images.n),0) AS image_count "
|
||||
"FROM page LEFT JOIN owners ON owners.tag_id=page.id "
|
||||
"LEFT JOIN images ON images.tag_id=page.id "
|
||||
"GROUP BY page.id ORDER BY page.id"
|
||||
"GROUP BY page.id ORDER BY " + order
|
||||
)
|
||||
|
||||
|
||||
_TAG_PAGE = _tag_rows("SELECT id,name FROM tags WHERE id>? ORDER BY id LIMIT ?")
|
||||
# #444 tag sorts: sort -> (key expression on tags/counted, direction). Ties and
|
||||
# "created" follow the id, so every order is total and keyset pages are exact.
|
||||
TAG_SORTS = {
|
||||
"created": (None, "ASC"),
|
||||
"name": ("name COLLATE NOCASE", "ASC"),
|
||||
"name-desc": ("name COLLATE NOCASE", "DESC"),
|
||||
"prompts": ("prompt_count", "DESC"),
|
||||
"prompts-asc": ("prompt_count", "ASC"),
|
||||
}
|
||||
_MAX_SEARCH = 200
|
||||
_TAG_SEARCH = "name LIKE ? ESCAPE '\\'"
|
||||
# Every matching tag ranked by the prompt count its row shows (the owners rule
|
||||
# of ``_tag_rows``), then one keyset page of it with its image counts.
|
||||
_RANKED_TAG_PAGE = (
|
||||
"WITH owners(tag_id,pid) AS ("
|
||||
"SELECT pt.tag_id,p.id FROM prompts p "
|
||||
"JOIN prompt_tags pt ON pt.prompt_id=p.id WHERE " + _ELIGIBLE + " "
|
||||
"UNION SELECT ppt.tag_id,pp.positive_prompt_id FROM prompt_pairs pp "
|
||||
"JOIN prompt_pair_tags ppt ON ppt.pair_id=pp.id WHERE pp.archived=0),"
|
||||
"counted AS (SELECT t.id,t.name,COUNT(DISTINCT o.pid) AS prompt_count "
|
||||
"FROM tags t LEFT JOIN owners o ON o.tag_id=t.id WHERE {search} "
|
||||
"GROUP BY t.id),"
|
||||
"page AS (SELECT id,name,prompt_count FROM counted WHERE {after} "
|
||||
"ORDER BY prompt_count {direction},id LIMIT ?),"
|
||||
"images(tag_id,n) AS (" + _IMAGE_FILES + " "
|
||||
"WHERE mit.tag_id IN (SELECT id FROM page) GROUP BY mit.tag_id) "
|
||||
"SELECT page.id,page.name,page.prompt_count,"
|
||||
"COALESCE(images.n,0) AS image_count FROM page "
|
||||
"LEFT JOIN images ON images.tag_id=page.id "
|
||||
"ORDER BY page.prompt_count {direction},page.id"
|
||||
)
|
||||
_UNTAGGED_COUNT = (
|
||||
"SELECT COUNT(*) FROM prompts p WHERE (" + _ELIGIBLE + ") AND " + UNTAGGED
|
||||
)
|
||||
_TAG_ONE = _tag_rows("SELECT id,name FROM tags WHERE id=?")
|
||||
# Every owner kind a tag can sit on, for the delete confirmation.
|
||||
_TAG_IMPACT = (
|
||||
@@ -210,6 +244,28 @@ def _tag_after(after: object) -> int:
|
||||
return after
|
||||
|
||||
|
||||
def _tag_search(search: object) -> str:
|
||||
if not isinstance(search, str) or len(search) > _MAX_SEARCH:
|
||||
raise ValueError(f"Tag search must be text of at most {_MAX_SEARCH} characters")
|
||||
return search.strip()
|
||||
|
||||
|
||||
def _sorted_tag_after(after: object, sort: str) -> tuple | None:
|
||||
"""The ``[key, id]`` cursor of a name or prompt-count sort."""
|
||||
if after is None:
|
||||
return None
|
||||
key_type = str if sort.startswith("name") else int
|
||||
if (
|
||||
not isinstance(after, list)
|
||||
or len(after) != 2
|
||||
or type(after[0]) is not key_type
|
||||
or type(after[1]) is not int
|
||||
or after[1] <= 0
|
||||
):
|
||||
raise ValueError("Invalid vocabulary cursor")
|
||||
return after[0], after[1]
|
||||
|
||||
|
||||
def _category_after(after: object) -> tuple[int, str] | None:
|
||||
if after is None:
|
||||
return None
|
||||
@@ -431,22 +487,80 @@ class VocabularyOperations:
|
||||
epoch: str,
|
||||
*,
|
||||
limit: int = 100,
|
||||
after: int | None = None,
|
||||
after: object = None,
|
||||
search: str = "",
|
||||
sort: str = "created",
|
||||
) -> dict:
|
||||
"""One page of tags matching ``search`` (a literal, case-insensitive
|
||||
substring) in ``sort`` order, with the untagged parents' count."""
|
||||
_limit(limit)
|
||||
start = _tag_after(after)
|
||||
total = conn.execute("SELECT COUNT(*) FROM tags").fetchone()[0]
|
||||
rows = [dict(row) for row in conn.execute(_TAG_PAGE, (start, limit + 1))]
|
||||
search = _tag_search(search)
|
||||
if not isinstance(sort, str) or sort not in TAG_SORTS:
|
||||
raise ValueError("sort must be one of " + ", ".join(TAG_SORTS))
|
||||
where, params = "1=1", []
|
||||
if search:
|
||||
where, params = _TAG_SEARCH, [f"%{_escape(search)}%"]
|
||||
total, untagged = conn.execute(
|
||||
f"SELECT (SELECT COUNT(*) FROM tags WHERE {where}),({_UNTAGGED_COUNT})",
|
||||
params,
|
||||
).fetchone()
|
||||
rows = [
|
||||
dict(row) for row in self._tag_page(conn, where, params, sort, after, limit)
|
||||
]
|
||||
has_more = len(rows) > limit
|
||||
rows = rows[:limit]
|
||||
key, _direction = TAG_SORTS[sort]
|
||||
next_key = None
|
||||
if has_more:
|
||||
last = rows[-1]
|
||||
next_key = (
|
||||
last["id"]
|
||||
if key is None
|
||||
else [
|
||||
last["prompt_count" if key == "prompt_count" else "name"],
|
||||
last["id"],
|
||||
]
|
||||
)
|
||||
return {
|
||||
"library_epoch": epoch,
|
||||
"tags": rows,
|
||||
"total": total,
|
||||
"untagged_count": untagged,
|
||||
# Echoed so a client can tell this read applied its search and sort.
|
||||
"search": search,
|
||||
"sort": sort,
|
||||
"has_more": has_more,
|
||||
"next_key": rows[-1]["id"] if has_more else None,
|
||||
"next_key": next_key,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _tag_page(conn, where, params, sort, after, limit):
|
||||
key, direction = TAG_SORTS[sort]
|
||||
if key is None:
|
||||
page = (
|
||||
f"SELECT id,name FROM tags WHERE {where} AND id>? ORDER BY id LIMIT ?"
|
||||
)
|
||||
return conn.execute(
|
||||
_tag_rows(page), [*params, _tag_after(after), limit + 1]
|
||||
)
|
||||
cursor = _sorted_tag_after(after, sort)
|
||||
operator = ">" if direction == "ASC" else "<"
|
||||
keyset, keys = "1=1", []
|
||||
if cursor is not None:
|
||||
keyset = f"({key}{operator}? OR ({key}=? AND id>?))"
|
||||
keys = [cursor[0], cursor[0], cursor[1]]
|
||||
if key == "prompt_count":
|
||||
sql = _RANKED_TAG_PAGE.format(
|
||||
search=where, after=keyset, direction=direction
|
||||
)
|
||||
return conn.execute(sql, [*params, *keys, limit + 1])
|
||||
page = (
|
||||
f"SELECT id,name FROM tags WHERE {where} AND {keyset} "
|
||||
f"ORDER BY {key} {direction},id LIMIT ?"
|
||||
)
|
||||
order = f"page.{key} {direction},page.id"
|
||||
return conn.execute(_tag_rows(page, order), [*params, *keys, limit + 1])
|
||||
|
||||
def _vocabulary_categories(
|
||||
self,
|
||||
conn: sqlite3.Connection,
|
||||
@@ -490,9 +604,18 @@ class VocabularyOperations:
|
||||
),
|
||||
}
|
||||
|
||||
def list_vocabulary_tags(self, *, limit: int = 100, after: int | None = None):
|
||||
def list_vocabulary_tags(
|
||||
self,
|
||||
*,
|
||||
limit: int = 100,
|
||||
after: object = None,
|
||||
search: str = "",
|
||||
sort: str = "created",
|
||||
):
|
||||
with self.library_snapshot() as (conn, epoch, _ceiling):
|
||||
return self._vocabulary_tags(conn, epoch, limit=limit, after=after)
|
||||
return self._vocabulary_tags(
|
||||
conn, epoch, limit=limit, after=after, search=search, sort=sort
|
||||
)
|
||||
|
||||
def list_vocabulary_categories(
|
||||
self, *, limit: int = 100, after: list | None = None
|
||||
|
||||
+26
-13
@@ -40,6 +40,7 @@ from aiohttp import web
|
||||
from PIL import Image
|
||||
|
||||
from .prompts import PromptRoutesMixin
|
||||
from .revalidation import revalidated_response
|
||||
from .pairs import PairRoutesMixin
|
||||
from .library import LibraryRoutesMixin
|
||||
from .vocabulary import VocabularyRoutesMixin
|
||||
@@ -653,35 +654,39 @@ class PromptManagerAPI(
|
||||
|
||||
@routes.get("/prompt_manager/workspace")
|
||||
async def serve_workspace_ui(request: web.Request) -> web.Response:
|
||||
return self._serve_html_page("workspace.html", "Workspace")
|
||||
return self._serve_html_page(request, "workspace.html", "Workspace")
|
||||
|
||||
@routes.get("/prompt_manager/web")
|
||||
async def serve_web_ui(request):
|
||||
return self._serve_html_page("index.html", "Web UI", cache=False)
|
||||
return self._serve_html_page(request, "index.html", "Web UI", cache=False)
|
||||
|
||||
@routes.get("/prompt_manager/gallery.html")
|
||||
async def serve_gallery_ui(request):
|
||||
return self._serve_html_page("metadata.html", "Gallery")
|
||||
return self._serve_html_page(request, "metadata.html", "Gallery")
|
||||
|
||||
@routes.get("/prompt_manager/admin")
|
||||
async def serve_admin_ui(request):
|
||||
return self._serve_html_page("admin.html", "Admin UI")
|
||||
return self._serve_html_page(request, "admin.html", "Admin UI")
|
||||
|
||||
@routes.get("/prompt_manager/gallery")
|
||||
async def serve_gallery_admin_ui(request):
|
||||
return self._serve_html_page("gallery.html", "Gallery")
|
||||
return self._serve_html_page(request, "gallery.html", "Gallery")
|
||||
|
||||
# ── Static file serving ───────────────────────────────────────
|
||||
|
||||
@routes.get("/prompt_manager/lib/{filepath:.*}")
|
||||
async def serve_lib_static(request):
|
||||
"""Serve static library files (JS, CSS) from web/lib directory."""
|
||||
return self._serve_static_file("lib", request.match_info.get("filepath"))
|
||||
return self._serve_static_file(
|
||||
request, "lib", request.match_info.get("filepath")
|
||||
)
|
||||
|
||||
@routes.get("/prompt_manager/js/{filepath:.*}")
|
||||
async def serve_js_static(request):
|
||||
"""Serve static JavaScript files from web/js directory."""
|
||||
return self._serve_static_file("js", request.match_info.get("filepath"))
|
||||
return self._serve_static_file(
|
||||
request, "js", request.match_info.get("filepath")
|
||||
)
|
||||
|
||||
# ── Register domain-specific routes from mixins ───────────────
|
||||
|
||||
@@ -726,8 +731,11 @@ class PromptManagerAPI(
|
||||
|
||||
self.logger.info("All routes registered with decorator pattern")
|
||||
|
||||
def _serve_static_file(self, subdir, filepath):
|
||||
"""Serve ``web/<subdir>/<filepath>``; 403 outside it, 404 when missing."""
|
||||
def _serve_static_file(self, request, subdir, filepath):
|
||||
"""Serve ``web/<subdir>/<filepath>``; 403 outside it, 404 when missing.
|
||||
|
||||
A served file is revalidated on every reuse (#478, ``revalidation``).
|
||||
"""
|
||||
root = os.path.join(_get_project_root(), "web", subdir)
|
||||
exact = STATIC_EXACT_FILES.get((subdir, filepath))
|
||||
# An exact file passes the same containment check, its extension only.
|
||||
@@ -742,10 +750,13 @@ class PromptManagerAPI(
|
||||
return web.Response(text="Not Found", status=404)
|
||||
with open(file_path, "rb") as f:
|
||||
content = f.read()
|
||||
return web.Response(body=content, content_type=content_type)
|
||||
return revalidated_response(request, content, content_type)
|
||||
|
||||
def _serve_html_page(self, filename, title, cache=True):
|
||||
"""Serve ``web/<filename>``; 404 when missing, 500 when unreadable."""
|
||||
def _serve_html_page(self, request, filename, title, cache=True):
|
||||
"""Serve ``web/<filename>``; 404 when missing, 500 when unreadable.
|
||||
|
||||
A served page is revalidated on every reuse (#478, ``revalidation``).
|
||||
"""
|
||||
html_path = os.path.join(_get_project_root(), "web", filename)
|
||||
try:
|
||||
if cache and html_path in self._html_cache:
|
||||
@@ -762,7 +773,9 @@ class PromptManagerAPI(
|
||||
content_type="text/html",
|
||||
status=404,
|
||||
)
|
||||
return web.Response(text=content, content_type="text/html", charset="utf-8")
|
||||
return revalidated_response(
|
||||
request, content.encode("utf-8"), "text/html", charset="utf-8"
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception(f"Failed to load {title}")
|
||||
return web.Response(
|
||||
|
||||
@@ -213,6 +213,8 @@ def public_config():
|
||||
"max_image_age_days": GalleryConfig.MAX_IMAGE_AGE_DAYS,
|
||||
"max_concurrent_processing": GalleryConfig.MAX_CONCURRENT_PROCESSING,
|
||||
"metadata_extraction_timeout": GalleryConfig.METADATA_EXTRACTION_TIMEOUT,
|
||||
# config.json gallery.monitoring.save_foreign_runs (#247)
|
||||
"save_foreign_runs": GalleryConfig.SAVE_FOREIGN_RUNS,
|
||||
},
|
||||
"integrations": {
|
||||
"lora_manager_enabled": bool(IntegrationConfig.LORA_MANAGER_ENABLED),
|
||||
|
||||
+13
-1
@@ -27,7 +27,11 @@ from .history_current import (
|
||||
resolve_current_candidate,
|
||||
snapshot_current_output_scope,
|
||||
)
|
||||
from .inspection_service import CurrentFileChanged, inspect_current_file
|
||||
from .inspection_service import (
|
||||
CurrentFileChanged,
|
||||
image_tags_reader,
|
||||
inspect_current_file,
|
||||
)
|
||||
from .images import image_category_key
|
||||
|
||||
try:
|
||||
@@ -778,6 +782,14 @@ class HistoryRoutesMixin:
|
||||
),
|
||||
own,
|
||||
),
|
||||
# The file's own image tags, keyed as the Tag dialog
|
||||
# keys them: the normcased realpath (#457).
|
||||
image_tags=image_tags_reader(
|
||||
db,
|
||||
str(resolved.canonical_target),
|
||||
self.logger,
|
||||
close=True,
|
||||
),
|
||||
recorded=recorded_output_lookup(
|
||||
db, close=True, reference=receipt["reference"]
|
||||
),
|
||||
|
||||
+3
-21
@@ -55,6 +55,7 @@ from .inspection_service import (
|
||||
IMAGE_EXTENSIONS as _IMAGE_EXTENSIONS,
|
||||
SERVABLE_EXTENSIONS,
|
||||
VIDEO_EXTENSIONS,
|
||||
image_tags_reader,
|
||||
inspect_current_file,
|
||||
)
|
||||
|
||||
@@ -3962,28 +3963,9 @@ class ImageRoutesMixin:
|
||||
)
|
||||
|
||||
def _inspection_image_tags(self, root: Path, parts: tuple) -> Callable:
|
||||
"""The inspector's ``image_tags`` reader for one rooted file (S1).
|
||||
|
||||
The returned function takes the inspection's own stable stat and
|
||||
answers ``current`` only for a byte record verified at exactly that
|
||||
``(size, mtime_ns)``; a failed read is ``unavailable``, never an error.
|
||||
"""
|
||||
"""The inspector's ``image_tags`` reader for one rooted file (S1)."""
|
||||
locator = os.path.normcase(os.path.join(os.path.realpath(root), *parts))
|
||||
|
||||
def read(size: int, mtime_ns: int) -> dict:
|
||||
try:
|
||||
found = self.db.current_image_tags({locator: (size, mtime_ns)})
|
||||
except Exception:
|
||||
self.logger.warning(
|
||||
"Inspection image tags could not be read", exc_info=True
|
||||
)
|
||||
return {"state": "unavailable", "tags": []}
|
||||
tags = found.get(locator)
|
||||
if tags is None:
|
||||
return {"state": "none", "tags": []}
|
||||
return {"state": "current", "tags": list(tags)}
|
||||
|
||||
return read
|
||||
return image_tags_reader(self.db, locator, self.logger)
|
||||
|
||||
def _read_own_category(self, sources, path):
|
||||
"""One file's own category, None, or OWN_CATEGORY_UNREAD (blocking).
|
||||
|
||||
@@ -93,6 +93,35 @@ def _in_host_temp(path: Path) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def image_tags_reader(
|
||||
db: Any, locator: str, logger: Any, *, close: bool = False
|
||||
) -> Callable[[int, int], dict]:
|
||||
"""The inspector's ``image_tags`` reader for one file *locator* (S1).
|
||||
|
||||
The returned function takes the inspection's own stable stat and answers
|
||||
``current`` only for a byte record verified at exactly that
|
||||
``(size, mtime_ns)``; a failed read is ``unavailable``, never an error.
|
||||
``close`` closes the calling thread's connection afterwards, for routes
|
||||
that close theirs after reading (History).
|
||||
"""
|
||||
|
||||
def read(size: int, mtime_ns: int) -> dict:
|
||||
try:
|
||||
found = db.current_image_tags({locator: (size, mtime_ns)})
|
||||
except Exception:
|
||||
logger.warning("Inspection image tags could not be read", exc_info=True)
|
||||
return {"state": "unavailable", "tags": []}
|
||||
finally:
|
||||
if close:
|
||||
db.model.close()
|
||||
tags = found.get(locator)
|
||||
if tags is None:
|
||||
return {"state": "none", "tags": []}
|
||||
return {"state": "current", "tags": list(tags)}
|
||||
|
||||
return read
|
||||
|
||||
|
||||
def inspect_current_file(
|
||||
path: Path,
|
||||
*,
|
||||
|
||||
+52
-2
@@ -21,9 +21,11 @@ from .pairs import pair_response
|
||||
try:
|
||||
from ...database.library import ARCHIVE_SCOPES
|
||||
from ...database.library_commands import LibraryCommandError
|
||||
from ...database.library_folders import folder_prefixes, normalize_folder
|
||||
except ImportError:
|
||||
from database.library import ARCHIVE_SCOPES
|
||||
from database.library_commands import LibraryCommandError
|
||||
from database.library_folders import folder_prefixes, normalize_folder
|
||||
|
||||
_SECRET = os.urandom(32)
|
||||
|
||||
@@ -283,36 +285,84 @@ class LibraryRoutesMixin:
|
||||
outputs = list(self._get_all_output_dirs())
|
||||
return self._media_roots(outputs), len(outputs)
|
||||
|
||||
def _library_root_spellings(self) -> list[str]:
|
||||
"""Every spelling of each output root (#444 folder filter): the
|
||||
validated, resolved root and each configured or detected path that
|
||||
resolves to it, so images saved through a symlinked root match."""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
roots = [str(root) for root in self._get_all_output_dirs()]
|
||||
candidates = [
|
||||
*GalleryConfig.MONITORING_DIRECTORIES,
|
||||
self._find_comfyui_output_dir(),
|
||||
*self._comfyui_output_dir_list(),
|
||||
]
|
||||
spellings = []
|
||||
for root in roots:
|
||||
for value in (root, *candidates):
|
||||
if not value or not isinstance(value, (str, os.PathLike)):
|
||||
continue
|
||||
value = str(value)
|
||||
same = value == root or os.path.normcase(
|
||||
os.path.realpath(value)
|
||||
) == os.path.normcase(root)
|
||||
if same and value not in spellings:
|
||||
spellings.append(value)
|
||||
return spellings
|
||||
|
||||
@pair_response
|
||||
async def get_library_options(self, request: web.Request) -> web.Response:
|
||||
result = await self._run_in_executor(self.db.get_library_options)
|
||||
include = request.query.get("include", "")
|
||||
if include not in ("", "folders"):
|
||||
raise ValueError("include must be empty or 'folders'")
|
||||
if include:
|
||||
# Folders are read only when Refine opens (a walk of linked paths).
|
||||
roots = await self._run_in_executor(self._library_root_spellings)
|
||||
result = await self._run_in_executor(self.db.get_library_options, roots)
|
||||
else:
|
||||
result = await self._run_in_executor(self.db.get_library_options)
|
||||
return web.json_response({"success": True, **result})
|
||||
|
||||
@pair_response
|
||||
async def list_library(self, request: web.Request) -> web.Response:
|
||||
folder = normalize_folder(request.query.get("folder", ""))
|
||||
options = {
|
||||
"text": request.query.get("text", ""),
|
||||
"category": request.query.get("category"),
|
||||
"tags": sorted(set(request.query.getall("tag", []))),
|
||||
"tag_mode": request.query.get("tag_mode", "all"),
|
||||
"sort": request.query.get("sort", "newest"),
|
||||
}
|
||||
limit = _page_limit(request)
|
||||
cursor = request.query.get("cursor")
|
||||
roots = (
|
||||
await self._run_in_executor(self._library_root_spellings) if folder else []
|
||||
)
|
||||
prefixes = folder_prefixes(roots, folder) if folder else None
|
||||
|
||||
def read() -> tuple[dict, dict]:
|
||||
with self.db.library_snapshot(include_ceiling=True) as snapshot:
|
||||
conn, epoch, ceiling = snapshot
|
||||
# The folder and the roots it was resolved under join the scope.
|
||||
scope = _scope(
|
||||
{"kind": "parents", "projection": 2, "epoch": epoch, **options}
|
||||
{
|
||||
"kind": "parents",
|
||||
"projection": 2,
|
||||
"epoch": epoch,
|
||||
**options,
|
||||
**({"folder": folder, "roots": roots} if folder else {}),
|
||||
}
|
||||
)
|
||||
result = self.db._list_library(
|
||||
conn,
|
||||
epoch,
|
||||
ceiling,
|
||||
**options,
|
||||
folder_prefixes=prefixes,
|
||||
limit=limit,
|
||||
after=_decode(cursor, scope),
|
||||
)
|
||||
result["folder"] = folder
|
||||
records = self.db.get_library_previews(
|
||||
[parent["id"] for parent in result["items"]]
|
||||
)
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Revalidated responses for the pages and static files PromptManager serves.
|
||||
|
||||
``Cache-Control: no-cache`` lets the browser keep a copy but makes it ask the
|
||||
server before every reuse, so a PromptManager update reaches the browser on a
|
||||
normal reload (#478). The ETag lets an unchanged file answer 304 without a
|
||||
body, which keeps that reload cheap.
|
||||
|
||||
The tag hashes the bytes rather than the file's mtime and size: an update can
|
||||
keep both (an archive extract, a same-length edit within one clock tick), and
|
||||
the handlers read the whole file anyway. It is weak because the gzip
|
||||
middleware may send the same content in a different encoding.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
REVALIDATE = "no-cache"
|
||||
|
||||
|
||||
def entity_tag(body: bytes) -> str:
|
||||
"""Weak ETag for ``body``."""
|
||||
return f'W/"{hashlib.sha256(body).hexdigest()[:32]}"'
|
||||
|
||||
|
||||
def etag_matches(if_none_match, etag: str) -> bool:
|
||||
"""True when an If-None-Match header names ``etag`` (weak comparison)."""
|
||||
if not if_none_match:
|
||||
return False
|
||||
if if_none_match.strip() == "*":
|
||||
return True
|
||||
opaque = etag.removeprefix("W/")
|
||||
return any(
|
||||
candidate.strip().removeprefix("W/") == opaque
|
||||
for candidate in if_none_match.split(",")
|
||||
)
|
||||
|
||||
|
||||
def revalidated_response(request, body: bytes, content_type: str, charset=None):
|
||||
"""200 with ``body``, or 304 when the request already holds this version."""
|
||||
etag = entity_tag(body)
|
||||
headers = {"Cache-Control": REVALIDATE, "ETag": etag}
|
||||
if etag_matches(request.headers.get("If-None-Match"), etag):
|
||||
return web.Response(status=304, headers=headers)
|
||||
return web.Response(
|
||||
body=body, content_type=content_type, charset=charset, headers=headers
|
||||
)
|
||||
@@ -8,7 +8,8 @@ is shared with the LoRA integration's save.
|
||||
GET also reports the image sources and the monitor (``sources.py``). A save
|
||||
that changes the sources runs as a ``source_change`` registry hold, so it is
|
||||
refused while a record-changing job (a scan) runs. ``monitoring_enabled``
|
||||
takes effect at the next restart. ``POST /prompt_manager/settings/validate-source``
|
||||
takes effect at the next restart; ``save_foreign_runs`` (#247) at once, since
|
||||
the monitor reads it for every file. ``POST /prompt_manager/settings/validate-source``
|
||||
checks one path and writes nothing.
|
||||
"""
|
||||
|
||||
@@ -43,6 +44,7 @@ _SETTINGS_FILE_KEYS = {
|
||||
"INFINITE_SCROLL": ("web_ui", "infinite_scroll"),
|
||||
"WORKER_THREADS": ("performance", "worker_threads"),
|
||||
"MONITORING_ENABLED": ("gallery", "monitoring", "enabled"),
|
||||
"SAVE_FOREIGN_RUNS": ("gallery", "monitoring", "save_foreign_runs"),
|
||||
"SAFETY_BACKUP": ("maintenance", "safety_backup"),
|
||||
"THUMBNAIL_FORMAT": ("thumbnails", "format"),
|
||||
"THUMBNAIL_QUALITY": ("thumbnails", "quality"),
|
||||
@@ -58,7 +60,7 @@ _SETTINGS_FILE_KEYS = {
|
||||
# The thumbnail settings a running Convert outdated job fixes (#317).
|
||||
_THUMBNAIL_SETTINGS = ("THUMBNAIL_FORMAT", "THUMBNAIL_QUALITY", "THUMBNAIL_EDGE")
|
||||
# Attributes that live on GalleryConfig; the others on PromptManagerConfig.
|
||||
_GALLERY_SETTINGS = frozenset({"MONITORING_ENABLED"})
|
||||
_GALLERY_SETTINGS = frozenset({"MONITORING_ENABLED", "SAVE_FOREIGN_RUNS"})
|
||||
# Attributes that live on AutotagConfig, named without the AUTOTAG_ prefix.
|
||||
_AUTOTAG_SETTINGS = frozenset(
|
||||
name for name in _SETTINGS_FILE_KEYS if name.startswith("AUTOTAG_")
|
||||
@@ -264,6 +266,7 @@ class SettingsRoutesMixin:
|
||||
"thumbnail_encoders": encoders,
|
||||
"settings": {
|
||||
"monitoring_enabled": GalleryConfig.MONITORING_ENABLED,
|
||||
"save_foreign_runs": GalleryConfig.SAVE_FOREIGN_RUNS,
|
||||
"result_timeout": PromptManagerConfig.RESULT_TIMEOUT,
|
||||
"webui_display_mode": PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||||
"infinite_scroll": PromptManagerConfig.INFINITE_SCROLL,
|
||||
@@ -626,6 +629,10 @@ class SettingsRoutesMixin:
|
||||
if not isinstance(data["monitoring_enabled"], bool):
|
||||
raise ValueError("monitoring_enabled must be true or false")
|
||||
updates["MONITORING_ENABLED"] = data["monitoring_enabled"]
|
||||
if "save_foreign_runs" in data:
|
||||
if not isinstance(data["save_foreign_runs"], bool):
|
||||
raise ValueError("save_foreign_runs must be true or false")
|
||||
updates["SAVE_FOREIGN_RUNS"] = data["save_foreign_runs"]
|
||||
if "safety_backup" in data:
|
||||
if not isinstance(data["safety_backup"], bool):
|
||||
raise ValueError("safety_backup must be true or false")
|
||||
|
||||
+18
-5
@@ -61,18 +61,25 @@ class VocabularyRoutesMixin:
|
||||
)
|
||||
|
||||
async def _vocabulary_page(
|
||||
self, request: web.Request, kind: str, reader_name: str, name: bool
|
||||
self,
|
||||
request: web.Request,
|
||||
kind: str,
|
||||
reader_name: str,
|
||||
name: bool,
|
||||
options: dict | None = None,
|
||||
) -> web.Response:
|
||||
limit = _vocabulary_limit(request)
|
||||
cursor = request.query.get("cursor")
|
||||
options = options or {}
|
||||
|
||||
def read() -> dict:
|
||||
# The snapshot stays open while the cursor is decoded and encoded,
|
||||
# so its epoch scope is the epoch the page was read from.
|
||||
# so its epoch scope is the epoch the page was read from. A tag
|
||||
# search or sort joins the scope: its cursor fits no other order.
|
||||
with self.db.library_snapshot() as (conn, epoch, _ceiling):
|
||||
scope = _scope({"kind": kind, "epoch": epoch})
|
||||
scope = _scope({"kind": kind, "epoch": epoch, **options})
|
||||
result = getattr(self.db, reader_name)(
|
||||
conn, epoch, limit=limit, after=_decode(cursor, scope)
|
||||
conn, epoch, limit=limit, after=_decode(cursor, scope), **options
|
||||
)
|
||||
result["next_cursor"] = _encode(
|
||||
result.pop("next_key"), scope, name=name
|
||||
@@ -84,8 +91,14 @@ class VocabularyRoutesMixin:
|
||||
|
||||
@pair_response
|
||||
async def list_vocabulary_tags(self, request: web.Request) -> web.Response:
|
||||
options = {
|
||||
"search": request.query.get("search", "").strip(),
|
||||
"sort": request.query.get("sort", "created"),
|
||||
}
|
||||
# Name keys are text: kept server side so the cursor stays small.
|
||||
name = options["sort"].startswith("name")
|
||||
return await self._vocabulary_page(
|
||||
request, "vocabulary-tags", "_vocabulary_tags", False
|
||||
request, "vocabulary-tags", "_vocabulary_tags", name, options
|
||||
)
|
||||
|
||||
@pair_response
|
||||
|
||||
@@ -194,6 +194,8 @@ class GalleryConfig:
|
||||
|
||||
Attributes:
|
||||
MONITORING_ENABLED (bool): Enable/disable automatic image monitoring
|
||||
SAVE_FOREIGN_RUNS (bool): The monitor stores the certain prompt of an
|
||||
output whose workflow has no PromptManager node (#247)
|
||||
MONITORING_DIRECTORIES (List[str]): Directories to monitor for new images
|
||||
SUPPORTED_EXTENSIONS (List[str]): Image file extensions to process
|
||||
PROCESSING_DELAY (float): Delay in seconds before processing new files
|
||||
@@ -210,6 +212,8 @@ class GalleryConfig:
|
||||
|
||||
# Image monitoring settings
|
||||
MONITORING_ENABLED = True
|
||||
# Read by the monitor for every file, so a save applies without a restart
|
||||
SAVE_FOREIGN_RUNS = True
|
||||
MONITORING_DIRECTORIES = [] # Auto-detect if empty
|
||||
SUPPORTED_EXTENSIONS = [".png", ".jpg", ".jpeg", ".webp", ".gif"]
|
||||
PROCESSING_DELAY = 2.0 # Seconds to wait before processing new files
|
||||
@@ -251,6 +255,7 @@ class GalleryConfig:
|
||||
return {
|
||||
"monitoring": {
|
||||
"enabled": cls.MONITORING_ENABLED,
|
||||
"save_foreign_runs": cls.SAVE_FOREIGN_RUNS,
|
||||
"directories": cls.MONITORING_DIRECTORIES,
|
||||
"extensions": cls.SUPPORTED_EXTENSIONS,
|
||||
"processing_delay": cls.PROCESSING_DELAY,
|
||||
@@ -303,6 +308,14 @@ class GalleryConfig:
|
||||
config_logger.warning(
|
||||
"Ignoring gallery monitoring enabled: expected true or false"
|
||||
)
|
||||
if "save_foreign_runs" in monitoring:
|
||||
if isinstance(monitoring["save_foreign_runs"], bool):
|
||||
cls.SAVE_FOREIGN_RUNS = monitoring["save_foreign_runs"]
|
||||
else:
|
||||
config_logger.warning(
|
||||
"Ignoring gallery monitoring save_foreign_runs: expected true "
|
||||
"or false"
|
||||
)
|
||||
if "directories" in monitoring:
|
||||
directories = cls._validated_directories(monitoring["directories"])
|
||||
if directories is not None:
|
||||
|
||||
@@ -558,7 +558,8 @@ test.describe('full inspector coarse pointer',()=>{
|
||||
}
|
||||
await openSelected(page);const viewer=page.locator('dialog#viewer-dialog');
|
||||
await expect(viewer.locator('.viewer-canvas img')).toBeVisible();
|
||||
for(const target of await viewer.locator('#viewer-toolbar > :is(button,a),.dialog-header > button').all()) {
|
||||
// #450 D156: the view switch is hidden at 700px and below; a shown one is measured like ×.
|
||||
for(const target of await viewer.locator('#viewer-toolbar > :is(button,a),.dialog-header > button:visible').all()) {
|
||||
const box=await target.evaluate(node=>{const {width,height}=node.getBoundingClientRect();return {width,height,name:node.getAttribute('aria-label')};});
|
||||
expect(box.width,box.name).toBeGreaterThanOrEqual(80);expect(box.height,box.name).toBeGreaterThanOrEqual(80);
|
||||
}
|
||||
@@ -621,6 +622,11 @@ test('viewer matches the concept shell and toolbar order',async({page,request})=
|
||||
const production=await page.evaluate(VIEWER_SHELL);
|
||||
// C8 rebuilds the details content; until then the shared file inspector marks the aside it renders into.
|
||||
production.paths=production.paths.map(path=>path.replace('aside#viewer-details.file-inspector','aside#viewer-details'));
|
||||
// #450 D156: production's view switch sits right before × (hidden at 700px and below); the concept has none.
|
||||
const modePaths=production.paths.filter(path=>path.includes('button#viewer-mode'));
|
||||
expect(modePaths.map(path=>path.split(' > ').slice(2).join(' > '))).toEqual(['button#viewer-mode.icon-btn','button#viewer-mode.icon-btn > svg.icon','button#viewer-mode.icon-btn > svg.icon > path']);
|
||||
expect(production.paths[production.paths.indexOf(modePaths[2])+1]).toBe('dialog#viewer-dialog > div.dialog-header > button#viewer-close');
|
||||
production.paths=production.paths.filter(path=>!modePaths.includes(path));
|
||||
// #249 D34: after Next → production appends its own status lines (position, then the transform and slideshow
|
||||
// reasons), and Download stays the original-file link; every other control keeps the concept's text and order.
|
||||
const appended=production.toolbar.slice(9);
|
||||
|
||||
@@ -9,7 +9,7 @@ import { parityPng } from './local-inspect-files.mjs';
|
||||
// `#metadata` (docs/concepts/workspace/app.js renderMetadata 740–748, selection 1715–1729, style.css 709–722,
|
||||
// 1180–1194, 1543–1545, 1559–1564), at 1600x1100 and 390x844 in both themes, in three states: initial, selected (the
|
||||
// same synthetic PNG set into both `#local-file` inputs) and viewer-open. The concept is never edited: the recorded
|
||||
// differences D60–D66 (TESTING.md) are applied to its measurement, and, where they change copy, to its page before it
|
||||
// differences D60–D66 and D155 (TESTING.md) are applied to its measurement, and, where they change copy, to its page before it
|
||||
// is measured, so the browser lays the concept out with production's words. After that the diff must be empty.
|
||||
// Paired captures are artifacts only; there are no PNG baselines.
|
||||
test.use({ locale: 'en-US', timezoneId: 'UTC', reducedMotion: 'reduce' });
|
||||
@@ -22,7 +22,8 @@ const COPY = {
|
||||
'Read embedded metadata in this browser. The file is not uploaded or added to your library.'],
|
||||
chooser: ['Choose an image for a local preview', 'Choose an image or video for a local preview'],
|
||||
note: ['This mock previews your file locally. Sample metadata is available with “Open sample metadata”.',
|
||||
'Your file stays in this browser. Its metadata opens with “Open file metadata”.'],
|
||||
// D155 (#448): choosing the file opens its metadata.
|
||||
'Your file stays in this browser. Choosing it opens its metadata.'],
|
||||
toast: ['Local preview only. No file uploaded; sample metadata is available above.',
|
||||
'Local preview only. No file uploaded; metadata is available above.'],
|
||||
};
|
||||
@@ -212,23 +213,29 @@ async function productionStates(page, theme, testInfo, tag) {
|
||||
await settle(page);
|
||||
const initial = await page.evaluate(measureInspect, '#workspace-main');
|
||||
await captureCanonicalState(page, testInfo, `production-initial-${tag}`);
|
||||
// D155 (#448): choosing the file opens the viewer with its details at once (the concept waits for its heading
|
||||
// action), so the viewer state is measured first; the selected page is measured after Escape, with the focus the
|
||||
// close returned to the chooser removed (the concept's page has no focused control there).
|
||||
await page.locator('#workspace-main #local-file').setInputFiles(parityPng(FILE));
|
||||
await expect(page.locator('#workspace-main #local-preview img.local-file-preview')).toBeVisible();
|
||||
await page.locator('#workspace-main #local-preview img').evaluate(image => image.decode().catch(() => {}));
|
||||
await expect(page.locator('#workspace-toast')).toHaveText(COPY.toast[1]);
|
||||
await expect(action).toHaveAttribute('aria-disabled', 'false');
|
||||
// D63: a readable PNG shows no status line.
|
||||
await expect(page.locator('#workspace-main #local-preview p.small')).toHaveCount(0);
|
||||
await settle(page);
|
||||
const selected = await page.evaluate(measureInspect, '#workspace-main');
|
||||
await captureCanonicalState(page, testInfo, `production-selected-${tag}`);
|
||||
await action.click();
|
||||
await expect(page.locator('dialog#viewer-dialog[open] #viewer-details .panel-heading')).toBeVisible();
|
||||
await expect(page.locator('dialog#viewer-dialog[open] #viewer-details .meta-grid').first()).toBeVisible();
|
||||
await page.mouse.move(0, 0);
|
||||
await settle(page);
|
||||
const viewer = await page.evaluate(measureViewer);
|
||||
await captureCanonicalState(page, testInfo, `production-viewer-${tag}`);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(page.locator('dialog#viewer-dialog')).toHaveCount(0);
|
||||
await page.evaluate(() => document.activeElement?.blur());
|
||||
await expect(page.locator('#workspace-main #local-preview img.local-file-preview')).toBeVisible();
|
||||
await page.locator('#workspace-main #local-preview img').evaluate(image => image.decode().catch(() => {}));
|
||||
// D155: no toast; the opened viewer was the confirmation.
|
||||
await expect(page.locator('#workspace-toast')).toHaveText('');
|
||||
await expect(action).toHaveAttribute('aria-disabled', 'false');
|
||||
// D63: a readable PNG shows no status line.
|
||||
await expect(page.locator('#workspace-main #local-preview p.small')).toHaveCount(0);
|
||||
await settle(page);
|
||||
const selected = await page.evaluate(measureInspect, '#workspace-main');
|
||||
await captureCanonicalState(page, testInfo, `production-selected-${tag}`);
|
||||
return { initial, selected, viewer };
|
||||
}
|
||||
|
||||
|
||||
@@ -475,9 +475,9 @@ function measureAppearance(mainSelector) {
|
||||
}
|
||||
|
||||
// D97: the rows Appearance adds for the legacy gallery settings, after the concept's own, as production renders them.
|
||||
// D156 (#450) moved "Collapse viewer details by default" to the Viewer surface (workspace-viewer-mode.spec.mjs).
|
||||
const ADDED_APPEARANCE_ROWS = [
|
||||
['card-details', 'Show image details on cards', 'Show each card’s title and file details under the preview.', true],
|
||||
['viewer-details', 'Collapse viewer details by default', 'Open the image viewer with its details panel hidden.', false],
|
||||
];
|
||||
|
||||
/** The concept's Appearance with the recorded deviations applied, given the server's real settings. */
|
||||
@@ -488,10 +488,11 @@ function mapAppearanceConcept(reference, settings) {
|
||||
control: { ...toggleRow.control, id, checked } }));
|
||||
return {
|
||||
...reference,
|
||||
// #317: the server's thumbnail settings follow in their own surface ("Thumbnails", README: retained in Appearance).
|
||||
layout: [reference.layout[0], 'section.surface', ...reference.layout.slice(1)],
|
||||
// D97: two added toggles and the "Reset display defaults" operation close the surface.
|
||||
surface: { ...reference.surface, tree: [...reference.surface.tree, 'div.setting-row', 'div.setting-row', 'div.operation'] },
|
||||
// D156: the Viewer surface, then (#317) the server's thumbnail settings in their own surface ("Thumbnails",
|
||||
// README: retained in Appearance).
|
||||
layout: [reference.layout[0], 'section.surface', 'section.surface', ...reference.layout.slice(1)],
|
||||
// D97: one added toggle (D156 moved the other) and the "Reset display defaults" operation close the surface.
|
||||
surface: { ...reference.surface, tree: [...reference.surface.tree, 'div.setting-row', 'div.operation'] },
|
||||
rows: [...reference.rows.map(row => row.control?.id === 'timeout'
|
||||
// The node search timeout is the server's real `result_timeout`, not the concept's sample 30.
|
||||
? { ...row, control: { ...row.control, value: String(settings.result_timeout) } } : row), ...added],
|
||||
@@ -538,7 +539,7 @@ test('appearance comparison detects mutations', async ({ page, request }) => {
|
||||
// Unmapped, the concept differs only where D97 and the real timeout are recorded.
|
||||
expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 2).join('.'))
|
||||
.filter((path, index, all) => all.indexOf(path) === index).sort())
|
||||
.toEqual(['layout.1', 'layout.2', 'layout.length', 'operations.0', 'operations.length', 'rows.11', 'rows.12', 'rows.13',
|
||||
.toEqual(['layout.1', 'layout.2', 'layout.3', 'layout.length', 'operations.0', 'operations.length', 'rows.11', 'rows.12',
|
||||
'rows.length', 'surface.tree'].sort());
|
||||
const measure = () => page.evaluate(measureAppearance, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
@@ -617,10 +618,21 @@ const WATCHED = new Set(['watching', 'stopping']);
|
||||
const REFRESH_FILES = { title: 'Refresh files', description: 'Read every source again to find files added or removed outside ComfyUI.',
|
||||
label: 'Refresh' };
|
||||
|
||||
// D158 (#247): the row production adds directly below "Watch for new outputs", in that row's markup.
|
||||
const FOREIGN_RUNS = { id: 'foreign-runs', label: 'Save prompts from other workflows',
|
||||
description: 'New outputs of workflows without a PromptManager node add their prompt to the library when it is certain.' };
|
||||
|
||||
/** D158: the concept's "Watch for new outputs" row as the inserted "Save prompts from other workflows" row. */
|
||||
function foreignRunsRow(monitoringRow, settings) {
|
||||
return { ...monitoringRow, text: [['label', FOREIGN_RUNS.label, FOREIGN_RUNS.id], ['p', FOREIGN_RUNS.description, null]],
|
||||
control: { ...monitoringRow.control, id: FOREIGN_RUNS.id, checked: settings.save_foreign_runs } };
|
||||
}
|
||||
|
||||
/** The concept's Sources with the recorded deviations applied, given the server's real settings and sources. */
|
||||
function mapSourcesConcept(reference, { settings, sources }) {
|
||||
const [sample] = reference.sources;
|
||||
const scan = sample.actions[0];
|
||||
const rowAt = reference.surface.tree.indexOf('div.setting-row');
|
||||
return {
|
||||
...reference,
|
||||
// The real sources replace the two samples (without the "sample path" note). D98: each configured one has Remove,
|
||||
@@ -628,13 +640,15 @@ function mapSourcesConcept(reference, { settings, sources }) {
|
||||
sources: sources.effective.map(source => ({ ...sample, title: source.label,
|
||||
description: `${source.path} · ${WATCH_TEXT[source.watch]}`,
|
||||
actions: [{ ...scan, text: WATCHED.has(source.watch) ? 'Scan changes' : 'Scan folder' }, { ...scan, text: 'Remove' }] })),
|
||||
rows: reference.rows.map(row => {
|
||||
if (row.control.id === 'monitoring') return { ...row, control: { ...row.control, checked: settings.monitoring_enabled } };
|
||||
rows: reference.rows.flatMap(row => {
|
||||
if (row.control.id === 'monitoring') return [{ ...row, control: { ...row.control, checked: settings.monitoring_enabled } },
|
||||
foreignRunsRow(row, settings)];
|
||||
// D113: the range ends at the server's core count (the concept's fixed 16) and shows the real worker count.
|
||||
return { ...row, control: { ...row.control, max: String(settings.cpu_count), value: String(settings.worker_threads) } };
|
||||
return [{ ...row, control: { ...row.control, max: String(settings.cpu_count), value: String(settings.worker_threads) } }];
|
||||
}),
|
||||
// D98: "Refresh files" closes the surface.
|
||||
surface: { ...reference.surface, tree: [...reference.surface.tree, 'div.operation'] },
|
||||
// D158: the inserted row after the first setting row; D98: "Refresh files" closes the surface.
|
||||
surface: { ...reference.surface, tree: [...reference.surface.tree.slice(0, rowAt + 1), 'div.setting-row',
|
||||
...reference.surface.tree.slice(rowAt + 1), 'div.operation'] },
|
||||
operations: [...reference.operations, { ...reference.operations[0], title: REFRESH_FILES.title,
|
||||
description: REFRESH_FILES.description, actions: [{ ...reference.operations[0].actions[0], text: REFRESH_FILES.label }] }],
|
||||
};
|
||||
@@ -688,10 +702,12 @@ test('sources comparison detects mutations', async ({ page, request }) => {
|
||||
const expected = mapSourcesConcept(concept, server);
|
||||
const production = await productionSources(page, 'dark', { notice: true });
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the concept differs only in the real sources, the D98 operation and the D113 range.
|
||||
// Unmapped, the concept differs only in the real sources, the D98 operation, the D158 row (which moves the D113
|
||||
// range to the third row) and the D113 range.
|
||||
expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 2).join('.'))
|
||||
.filter((path, index, all) => all.indexOf(path) === index).sort())
|
||||
.toEqual(['operations.2', 'operations.length', 'rows.1', 'sources.0', 'sources.1', 'surface.tree'].sort());
|
||||
.toEqual(['operations.2', 'operations.length', 'rows.1', 'rows.2', 'rows.length', 'sources.0', 'sources.1',
|
||||
'surface.tree'].sort());
|
||||
const measure = () => page.evaluate(measureSources, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
// A removed notice and a removed source row.
|
||||
@@ -705,6 +721,13 @@ test('sources comparison detects mutations', async ({ page, request }) => {
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="monitoring"]').textContent = 'Watch for outputs'; });
|
||||
expect(await paths()).toEqual(['rows.0.text.0.1']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="monitoring"]').textContent = 'Watch for new outputs'; });
|
||||
// D158: the inserted row is compared too: a missing one and a changed label are caught.
|
||||
const undoForeign = await mutate(page, { remove: ['#workspace-main .setting-row:has(#foreign-runs)'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['rows.length', 'surface.tree.length']));
|
||||
await undoForeign();
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="foreign-runs"]').textContent = 'Save other prompts'; });
|
||||
expect(await paths()).toEqual(['rows.1.text.0.1']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="foreign-runs"]').textContent = 'Save prompts from other workflows'; });
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .panel-heading button').textContent = 'Add source'; });
|
||||
expect(await paths()).toEqual(['heading.action.text']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .panel-heading button').textContent = '+ Add source'; });
|
||||
|
||||
@@ -29,6 +29,19 @@ export function pageInventoryOf(main) {
|
||||
const rows=[...table.querySelectorAll('tbody tr')];
|
||||
const firstCell=rows[0]?.querySelector('td');
|
||||
const operations=[...categorySurface.querySelectorAll('.operation')].filter(visible);
|
||||
const operationOf=row=>{const h3=row.querySelector('h3'),p=row.querySelector('p'),rowActions=row.querySelector('.row-actions');return {
|
||||
...css(row,'display','flexDirection','justifyContent','alignItems','columnGap','paddingTop','paddingBottom','borderTopWidth','borderTopStyle','borderTopColor'),
|
||||
height:px(row.getBoundingClientRect().height),h3:{text:h3.textContent,...font(h3)},p:{text:p.textContent,...font(p)},
|
||||
actions:{...css(rowActions,'flexShrink','flexWrap','columnGap'),buttons:[...rowActions.querySelectorAll('button')].map(buttonOf)}};};
|
||||
// D149 (#444): production adds the Untagged prompts row and the tag search and sort above the table; the concept has
|
||||
// neither. The toolbar is compared on its own (toolbarInventoryOf) with the concept's page toolbar as donor.
|
||||
const untaggedRow=tagSurface.querySelector(':scope > .operation');
|
||||
const d149=untaggedRow?{untagged:operationOf(untaggedRow),
|
||||
// It starts where the concept's table starts: right after the paragraph.
|
||||
offset:px(untaggedRow.getBoundingClientRect().top-tagSurface.getBoundingClientRect().top),
|
||||
// The vertical space the two additions take: the table starts this much lower than after the paragraph alone.
|
||||
// (Both offsets rounded first, so offset + shift is exactly the table's rounded offset.)
|
||||
shift:px(wrap.getBoundingClientRect().top-tagSurface.getBoundingClientRect().top)-px(untaggedRow.getBoundingClientRect().top-tagSurface.getBoundingClientRect().top)}:null;
|
||||
return {
|
||||
heading:{...css(heading,'display','flexDirection','justifyContent','alignItems','columnGap','rowGap','marginBottom'),
|
||||
h1:{text:h1.textContent,tabIndex:h1.tabIndex,...font(h1)},
|
||||
@@ -54,14 +67,26 @@ export function pageInventoryOf(main) {
|
||||
rows:rows.map(row=>{const cells=[...row.querySelectorAll('td')];return {cells:cells.length,height:px(row.getBoundingClientRect().height),tagLines:lines(cells[0]),tag:cells[0].textContent,prompts:cells[1].textContent,
|
||||
images:shownText(cells[2]),imagesHidden:cells[2].querySelector('.ca-visually-hidden')?.textContent??null,
|
||||
actions:[...cells[3].querySelectorAll('.row-actions button')].map(node=>node.textContent.trim())};})},
|
||||
operations:operations.map(row=>{const h3=row.querySelector('h3'),p=row.querySelector('p'),rowActions=row.querySelector('.row-actions');return {
|
||||
...css(row,'display','flexDirection','justifyContent','alignItems','columnGap','paddingTop','paddingBottom','borderTopWidth','borderTopStyle','borderTopColor'),
|
||||
height:px(row.getBoundingClientRect().height),h3:{text:h3.textContent,...font(h3)},p:{text:p.textContent,...font(p)},
|
||||
actions:{...css(rowActions,'flexShrink','flexWrap','columnGap'),buttons:[...rowActions.querySelectorAll('button')].map(buttonOf)}};}),
|
||||
operations:operations.map(operationOf),
|
||||
d149,
|
||||
noPageScroll:document.documentElement.scrollWidth<=innerWidth+1,
|
||||
};
|
||||
}
|
||||
|
||||
/** A `search.toolbar`'s tokens (D149): runs in the page on the concept's page toolbar and on the tag search and sort. */
|
||||
export function toolbarInventoryOf(root) {
|
||||
const css=(node,...keys)=>{const style=getComputedStyle(node);return Object.fromEntries(keys.map(key=>[key,style[key]]));};
|
||||
const px=value=>Math.round(value);
|
||||
const field=root.querySelector('.search-field'),input=field.querySelector('input'),select=root.querySelector('select');
|
||||
return {...css(root,'display','columnGap','rowGap','flexWrap','alignItems','marginBottom'),
|
||||
field:{...css(field,'display','alignItems','columnGap','flexGrow','minWidth','paddingLeft','paddingRight','borderTopWidth','borderTopStyle',
|
||||
'borderTopColor','borderTopLeftRadius','backgroundColor'),icon:Boolean(field.querySelector(':scope > svg')),height:px(field.getBoundingClientRect().height)},
|
||||
input:{type:input.type,...css(input,'fontSize','minHeight','backgroundColor','borderTopWidth','color')},
|
||||
select:{...css(select,'fontSize','minHeight','backgroundColor','color','borderTopColor','borderTopLeftRadius','paddingLeft'),
|
||||
height:px(select.getBoundingClientRect().height)}};
|
||||
}
|
||||
|
||||
|
||||
const countOf=(count,noun)=>`${count} ${noun}${count===1?'':'s'}`;
|
||||
|
||||
/**
|
||||
@@ -81,12 +106,17 @@ export function withImageCounts(operations,{images={},only=[]}={}) {
|
||||
|
||||
// Recorded deviations only (plan "Mock inventory" + decisions D1-D15 + ledger rulings). Everything else must be equal.
|
||||
// `imageCategories` (#332 D143) is the fixture's own image categories, none by default.
|
||||
export function mapVocabularyConcept(reference,imageCategories={}) {
|
||||
export function mapVocabularyConcept(reference,imageCategories={},{untaggedText=null,shift=0}={}) {
|
||||
const operations=reference.operations.map(row=>({...row,p:{...row.p,
|
||||
// Ledger ruling: "1 positive prompts" -> singular, consistent with the shared plural helper (#238/#243).
|
||||
text:row.p.text.replace(/^1 positive prompts$/,'1 positive prompt')}}));
|
||||
return {...reference,
|
||||
table:{...reference.table,rows:reference.table.rows.map(row=>({...row,
|
||||
// D149 (#444): the Untagged prompts row is drawn as the concept's first category row with its own text and a Browse
|
||||
// action, and the table moves down by the space it and the tag search and sort take.
|
||||
const first=reference.operations[0];
|
||||
const d149=untaggedText===null?null:{untagged:{...first,h3:{...first.h3,text:'Untagged prompts'},p:{...first.p,text:untaggedText}},
|
||||
offset:reference.table.wrap.offset,shift};
|
||||
return {...reference,d149,
|
||||
table:{...reference.table,wrap:{...reference.table.wrap,offset:reference.table.wrap.offset+shift},rows:reference.table.rows.map(row=>({...row,
|
||||
// Data, not a deviation: the concept tags each sample image with its prompt's tags; the vocabulary fixture
|
||||
// tags no image, so every Images count is 0 (#249 C2 retired #245 D3).
|
||||
images:'0'}))},
|
||||
@@ -94,9 +124,14 @@ export function mapVocabularyConcept(reference,imageCategories={}) {
|
||||
};
|
||||
}
|
||||
|
||||
/** The D149 mapping inputs: the fixture's Untagged prompts count text (none by default) and production's table shift. */
|
||||
export function d149Of(actual,untaggedText='0 positive prompts') {
|
||||
return actual.d149?{untaggedText,shift:actual.d149.shift}:{};
|
||||
}
|
||||
|
||||
/** The page comparison: concept rows are the concept's sample tags; production appends fixture-only tags (D4). */
|
||||
export function compareVocabularyPage(reference,actual,imageCategories={}) {
|
||||
const expected=mapVocabularyConcept(reference,imageCategories);
|
||||
const expected=mapVocabularyConcept(reference,imageCategories,d149Of(actual));
|
||||
const shared=expected.table.rows.length;
|
||||
return {diff:diffInventory(expected,{...actual,table:{...actual.table,rows:actual.table.rows.slice(0,shared)}}),
|
||||
extra:actual.table.rows.slice(shared).map(row=>row.tag)};
|
||||
|
||||
@@ -363,14 +363,26 @@ test('a list read superseded by another read still opens a collection made since
|
||||
const fresh = (await added.json()).collection.id;
|
||||
await expect(rows(page)).toHaveCount(2);
|
||||
const hold = await holdListReads(page);
|
||||
// A rename schedules the 300 ms count re-read; opening "Fresh" first reads the list once more. The timer's read
|
||||
// supersedes that one while both are held.
|
||||
// A confirmed rename schedules the 300 ms count re-read; opening "Fresh" first reads the list once more. The
|
||||
// timer's read supersedes that one while both are held. The rename's answer is held until "Fresh"'s read is, so
|
||||
// the timer starts only then: with the answer let through at once, a slow host could fire the timer before the
|
||||
// hash change, and the page would then wait on the timer's read alone (#459).
|
||||
let releaseRename;
|
||||
const renameHeld = new Promise(resolve => { releaseRename = resolve; });
|
||||
await page.route(`**${API}/${manifest.collections.Landscapes.id}`, async route => {
|
||||
if (route.request().method() !== 'PATCH') return route.fallback();
|
||||
await renameHeld;
|
||||
return route.fallback();
|
||||
});
|
||||
await editButton(page).click();
|
||||
await nameField(page).fill('Landscape grids');
|
||||
await editDialog(page).getByRole('button', { name: 'Save', exact: true }).click();
|
||||
await expect(toast(page)).toHaveText('Collection renamed.');
|
||||
await page.evaluate(hash => { window.location.hash = hash; }, `#/collection/${fresh}`);
|
||||
await expect.poll(() => hold.held.length).toBeGreaterThanOrEqual(2);
|
||||
await expect.poll(() => hold.held.length).toBe(1);
|
||||
releaseRename();
|
||||
// The rename landed (the sidebar row has the new name), so the count read is now due.
|
||||
await expect(row(page, 'Landscape grids')).toHaveCount(1);
|
||||
await expect.poll(() => hold.held.length).toBe(2);
|
||||
await expect(main(page)).not.toContainText('Collections could not be loaded.');
|
||||
await hold.release();
|
||||
await expect(heading(page)).toHaveText('Fresh');
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
|
||||
import {test,expect} from './fixtures.mjs';
|
||||
import {armUnseenRestores,unseenRestore} from './unseen-restore.mjs';
|
||||
import {saveImageTags} from './concept-inventory.mjs';
|
||||
import {readFile,writeFile} from 'node:fs/promises';
|
||||
import {createHash} from 'node:crypto';
|
||||
|
||||
@@ -495,23 +496,38 @@ test('the inline current file names its prompts apart from the frozen run fields
|
||||
expect(await exact(dialog(page).getByRole('region',{name:'Positive prompt',exact:true}).locator('p.prompt-text'))).toBe(before);
|
||||
});
|
||||
|
||||
// #449 (D154): History's inline current file draws tags in the shared labelled sections, named with its context so they
|
||||
// stay apart from the frozen run's fields. The server sends History no image_tags facet today (D88), so the read is
|
||||
// given one here to prove the shared render in History.
|
||||
test('the inline current file labels its image tags after its negative prompt',async({page})=>{
|
||||
// #449 (D154), #457: History's inline current file draws the file's own image tags in the shared labelled section, named
|
||||
// with its context so they stay apart from the frozen run's fields. The tags are saved on the real file through the Tag
|
||||
// routes and come from History's own current-file read (its `image_tags` facet), as Images reads them.
|
||||
test('the inline current file labels its image tags after its negative prompt',async({page,request})=>{
|
||||
const reads=[];
|
||||
await page.route(url=>url.pathname.endsWith('/inspection'),async route=>{const response=await route.fetch();const body=await response.json();
|
||||
reads.push(body.image_tags);
|
||||
body.prompts={positive:{status:'resolved',text:'current file positive',source:'graph-inferred'},negative:{status:'resolved',text:'current file negative',source:'graph-inferred'}};
|
||||
body.image_tags={state:'current',tags:['history_tag']};
|
||||
await route.fulfill({response,json:body});});
|
||||
await open(page,page.fixture.detail_id);await marker(page,page.fixture.runs.shared).click();await expect(dialog(page).locator('[data-receipt]')).toHaveCount(5);
|
||||
await dialog(page).getByRole('button',{name:'Raw',exact:true}).click();
|
||||
await dialog(page).locator('[data-receipt]').first().getByRole('button',{name:'View current file',exact:true}).click();
|
||||
await expect(page.locator('dialog#viewer-dialog')).toBeVisible();await page.keyboard.press('Escape');await expect(dialog(page)).toBeVisible();
|
||||
const current=dialog(page).getByRole('region',{name:'Current reported file',exact:true});
|
||||
const tags=current.getByRole('region',{name:'Current file image tags',exact:true});
|
||||
await expect(tags.locator('.role-label')).toHaveText('Image tags · 1');await expect(tags.locator('.tags > .chip')).toHaveText(['history_tag']);
|
||||
const showCurrent=async()=>{
|
||||
await open(page,page.fixture.detail_id);await marker(page,page.fixture.runs.shared).click();await expect(dialog(page).locator('[data-receipt]')).toHaveCount(5);
|
||||
await dialog(page).getByRole('button',{name:'Raw',exact:true}).click();
|
||||
await dialog(page).locator('[data-receipt]').first().getByRole('button',{name:'View current file',exact:true}).click();
|
||||
await expect(page.locator('dialog#viewer-dialog')).toBeVisible();await page.keyboard.press('Escape');await expect(dialog(page)).toBeVisible();
|
||||
await expect(current.getByRole('region',{name:'Current file negative prompt',exact:true})).toBeVisible();
|
||||
};
|
||||
// An untagged file draws no image tags section.
|
||||
await showCurrent();await expect(tags).toHaveCount(0);expect(reads.at(-1)).toEqual({state:'none',tags:[]});
|
||||
await dialog(page).getByRole('button',{name:'Close recorded run',exact:true}).click();
|
||||
expect(await saveImageTags(request,'history/current.png',['history_tag','dusk'])).toEqual(['history_tag','dusk']);
|
||||
await showCurrent();expect(reads.at(-1)).toEqual({state:'current',tags:['history_tag','dusk']});
|
||||
await expect(tags.locator('.role-label')).toHaveText('Image tags · 2');await expect(tags.locator('.tags > .chip')).toHaveText(['history_tag','dusk']);
|
||||
const negative=current.getByRole('region',{name:'Current file negative prompt',exact:true});
|
||||
expect(await negative.evaluate((node,section)=>({after:Boolean(node.compareDocumentPosition(section)&Node.DOCUMENT_POSITION_FOLLOWING),inside:node.contains(section)}),await tags.elementHandle())).toEqual({after:true,inside:false});
|
||||
// The viewer opened from History shares the inspection: the same section, plain chips (no Tags filter there, D88).
|
||||
await dialog(page).locator('[data-receipt]').first().getByRole('button',{name:'View current file',exact:true}).click();
|
||||
const viewerTags=page.locator('dialog#viewer-dialog #viewer-details section.tag-section.image-tags');
|
||||
await expect(viewerTags.locator('.role-label')).toHaveText('Image tags · 2');
|
||||
await expect(viewerTags.locator('span.chip')).toHaveText(['history_tag','dusk']);await expect(viewerTags.locator('button')).toHaveCount(0);
|
||||
await page.keyboard.press('Escape');await expect(dialog(page)).toBeVisible();
|
||||
});
|
||||
|
||||
// #332 CUJ 5 (MC8, §5.3): History's current-file inspector reads the file's own category with the Images key rule, so
|
||||
|
||||
@@ -8,10 +8,11 @@ import { parityPng, localFixture, tinyWebm, sendablePng, corruptPng, disguised,
|
||||
|
||||
const workspace = '/prompt_manager/workspace';
|
||||
const TOAST = 'Local preview only. No file uploaded; metadata is available above.';
|
||||
const NO_TOAST = '';
|
||||
const ACTION = 'Open file metadata';
|
||||
const COPY = {
|
||||
intro: 'Read embedded metadata in this browser. The file is not uploaded or added to your library.',
|
||||
note: 'Your file stays in this browser. Its metadata opens with “Open file metadata”.',
|
||||
note: 'Your file stays in this browser. Choosing it opens its metadata.',
|
||||
chooser: 'Choose an image or video for a local preview',
|
||||
multiple: 'Drop one file at a time.',
|
||||
directory: 'Folders cannot be inspected. Choose one image or video file.',
|
||||
@@ -53,13 +54,26 @@ async function choose(page, file) {
|
||||
await chooser(page).setInputFiles(file);
|
||||
}
|
||||
|
||||
/** D155: choosing a file opens the viewer on it with its details; waits for that. */
|
||||
async function openViewer(page) {
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
await action(page).click();
|
||||
await expect(viewer(page)).toBeVisible();
|
||||
await expect(rail(page).locator('.panel-heading')).toBeVisible();
|
||||
}
|
||||
|
||||
/** After the viewer was closed, the heading action (D60) opens it again on the current selection. */
|
||||
async function reopenViewer(page) {
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
await action(page).click();
|
||||
await openViewer(page);
|
||||
}
|
||||
|
||||
/** Waits for the viewer a selection opened (D155) and closes it, back on the page. */
|
||||
async function closeOpened(page) {
|
||||
await openViewer(page);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
}
|
||||
|
||||
/** Dispatches dragover + drop on `selector` with Files built in the page (and optional string items). */
|
||||
function drop(page, selector, { files = [], strings = [] } = {}) {
|
||||
return page.evaluate(async ({ selector, files, strings }) => {
|
||||
@@ -152,7 +166,8 @@ test('chooser opens the shared full inspector without a second read', async ({ p
|
||||
const box = await image.evaluate(node => ({ height: node.getBoundingClientRect().height, fit: getComputedStyle(node).objectFit, margin: getComputedStyle(node).marginTop }));
|
||||
expect(box.height).toBeLessThanOrEqual(400);
|
||||
expect(box).toMatchObject({ fit: 'contain', margin: '20px' });
|
||||
await expect(page.locator('#workspace-toast')).toHaveText(TOAST);
|
||||
// D155: the open viewer is the confirmation, so no toast.
|
||||
await expect(page.locator('#workspace-toast')).toHaveText(NO_TOAST);
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
await expect(action(page)).not.toHaveAttribute('aria-describedby', /.*/);
|
||||
|
||||
@@ -195,18 +210,120 @@ test('chooser opens the shared full inspector without a second read', async ({ p
|
||||
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
await expect(action(page)).toBeFocused();
|
||||
// D155: focus returns to the chooser that opened it.
|
||||
await expect(chooser(page)).toBeFocused();
|
||||
// Opening the viewer again shows the same record without reading the File a second time.
|
||||
await page.evaluate(() => {
|
||||
window.__slices = 0;
|
||||
const slice = Blob.prototype.slice;
|
||||
Blob.prototype.slice = function (...args) { if (this.name === 'f02.png') window.__slices++; return slice.apply(this, args); };
|
||||
});
|
||||
await openViewer(page);
|
||||
await reopenViewer(page);
|
||||
expect(await railContent(rail(page))).toEqual(served);
|
||||
expect(await page.evaluate(() => window.__slices)).toBe(0);
|
||||
});
|
||||
|
||||
// #448 (D155): choosing or dropping a file opens the shared viewer with its details showing, in one step. Closing it
|
||||
// returns to the page with the file still chosen and focus on the chooser, ready for the next file.
|
||||
test('D155: choosing a file opens the viewer with its details in one click', async ({ page }) => {
|
||||
await openInspect(page);
|
||||
await choose(page, parityPng('f02.png'));
|
||||
await expect(viewer(page)).toBeVisible();
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText('f02.png');
|
||||
await expect(rail(page)).toBeVisible();
|
||||
await expect(rail(page).locator('.prompt-section.positive p.prompt-text')).toHaveText('a red fox in snow, detailed fur');
|
||||
await expect(viewer(page).getByRole('button', { name: /^(Show|Hide) details$/ })).toHaveCount(0);
|
||||
// The open viewer is the confirmation: no toast says the metadata is "available above".
|
||||
await expect(page.locator('#workspace-toast')).toHaveText('');
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
await expect(chooser(page)).toBeFocused();
|
||||
await expect(preview(page).locator('img.local-file-preview')).toBeVisible();
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
// A dropped second file replaces the first and opens at once.
|
||||
await drop(page, '.file-drop', { files: [sendablePng()] });
|
||||
await expect(viewer(page)).toBeVisible();
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText('send-me.png');
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
await expect(chooser(page)).toBeFocused();
|
||||
// The same file again is a new selection and opens again; the heading action reopens the current one.
|
||||
expect(await chooser(page).evaluate(node => { node.dispatchEvent(new MouseEvent('click')); return node.value; })).toBe('');
|
||||
await choose(page, parityPng('f02.png'));
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText('f02.png');
|
||||
await page.keyboard.press('Escape');
|
||||
await action(page).click();
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText('f02.png');
|
||||
await expect(rail(page)).toBeVisible();
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(action(page)).toBeFocused();
|
||||
});
|
||||
|
||||
test('D155: Inspect shows the details even when the viewer collapses them by default (amends D97)', async ({ page }) => {
|
||||
await page.addInitScript(() => localStorage.setItem('gallerySettings', JSON.stringify({ sidebarCollapsedByDefault: true })));
|
||||
await openInspect(page);
|
||||
await choose(page, parityPng('f02.png'));
|
||||
await expect(viewer(page)).toBeVisible();
|
||||
await expect(rail(page)).toBeVisible();
|
||||
const toggle = viewer(page).getByRole('button', { name: 'Hide details', exact: true });
|
||||
await expect(toggle).toHaveAttribute('aria-expanded', 'true');
|
||||
await toggle.click();
|
||||
await expect(rail(page)).toBeHidden();
|
||||
await page.keyboard.press('Escape');
|
||||
await action(page).click();
|
||||
await expect(rail(page)).toBeVisible();
|
||||
await expect(viewer(page).getByRole('button', { name: 'Hide details', exact: true })).toHaveAttribute('aria-expanded', 'true');
|
||||
});
|
||||
|
||||
test('D155: an unreadable file stays on the page with its alert', async ({ page }) => {
|
||||
await page.addInitScript(() => {
|
||||
const slice = Blob.prototype.slice;
|
||||
Blob.prototype.slice = function (...args) {
|
||||
if (this.name === 'gone.png') return { arrayBuffer: () => Promise.reject(new DOMException('The file is gone', 'NotReadableError')) };
|
||||
return slice.apply(this, args);
|
||||
};
|
||||
});
|
||||
await openInspect(page);
|
||||
await choose(page, parityPng('f02.png', 'gone.png'));
|
||||
await expect(status(page)).toHaveText('Embedded metadata could not be read.');
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('D155: an AVI keeps its note and toast and does not open the viewer', async ({ page }) => {
|
||||
await openInspect(page);
|
||||
// AVI and WMV metadata is never read and the browser cannot play them, so the page keeps the selection's note.
|
||||
const avi = { name: 'clip.avi', mimeType: 'video/x-msvideo', buffer: Buffer.concat([Buffer.from('RIFF'), Buffer.alloc(4), Buffer.from('AVI LIST'), Buffer.alloc(64)]) };
|
||||
await choose(page, avi);
|
||||
await expect(page.locator('#workspace-toast')).toHaveText(TOAST);
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
await reopenViewer(page);
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText('clip.avi');
|
||||
});
|
||||
|
||||
test('D155: a selection that lands while another dialog is open leaves the viewer closed', async ({ page }) => {
|
||||
await openInspect(page);
|
||||
await page.evaluate(() => { const other = document.createElement('dialog'); other.id = 'other-dialog'; other.textContent = 'Other'; document.body.append(other); other.showModal(); });
|
||||
await choose(page, parityPng('f02.png'));
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
await page.evaluate(() => document.querySelector('#other-dialog').close());
|
||||
await reopenViewer(page);
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText('f02.png');
|
||||
});
|
||||
|
||||
test('D155: at 390px the opened details are reachable below the media', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
await openInspect(page);
|
||||
await choose(page, parityPng('f02.png'));
|
||||
await expect(viewer(page)).toBeVisible();
|
||||
const facts = rail(page).locator('.meta-grid').first();
|
||||
await facts.scrollIntoViewIfNeeded();
|
||||
await expect(facts).toBeInViewport();
|
||||
expect(await viewer(page).evaluate(node => node.scrollWidth <= node.clientWidth + 1)).toBe(true);
|
||||
});
|
||||
|
||||
// PR L3 Task 10: local roles are the server inspector's (local-resolver.mjs), so for every parity PNG the rail shows the
|
||||
// prompts, their status lines and the facts Images shows for the same bytes in the output folder.
|
||||
const PARITY_PNGS = ['f01b', 'f02', 'f03', 'f06a', 'f06b', 'f06c', 'f06d', 'f06e', 'f07a', 'f07b', 'f08', 'f09', 'f10', 'f11', 'f12',
|
||||
@@ -260,7 +377,9 @@ test('drop accepts exactly one File and never navigates', async ({ page }) => {
|
||||
await expect(preview(page)).toBeEmpty();
|
||||
expect(await drop(page, '.file-drop', { files: [parityPng('f02.png')] })).toEqual({ over: true, drop: true });
|
||||
await expect(preview(page).locator('img.local-file-preview')).toBeVisible();
|
||||
await expect(page.locator('#workspace-toast')).toHaveText(TOAST);
|
||||
// D155: the drop opened the viewer; drops are ignored while it is open (D64), so close it first.
|
||||
await closeOpened(page);
|
||||
await expect(page.locator('#workspace-toast')).toHaveText(NO_TOAST);
|
||||
// The chooser names the dropped file, never the one it replaced and never "No file chosen".
|
||||
expect(await chooser(page).evaluate(node => [...node.files].map(file => file.name))).toEqual(['f02.png']);
|
||||
const first = await preview(page).locator('img').getAttribute('src');
|
||||
@@ -275,7 +394,7 @@ test('drop accepts exactly one File and never navigates', async ({ page }) => {
|
||||
await expect(preview(page).locator('img')).toHaveAttribute('src', first);
|
||||
}
|
||||
await expect(action(page)).toHaveAttribute('aria-disabled', 'false');
|
||||
await openViewer(page);
|
||||
await reopenViewer(page);
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText('f02.png');
|
||||
await expect(page).toHaveURL(/#\/inspect$/);
|
||||
});
|
||||
@@ -285,6 +404,7 @@ test('cancel and same-file reselect', async ({ page }) => {
|
||||
await openInspect(page);
|
||||
await choose(page, parityPng('f02.png'));
|
||||
await expect(preview(page).locator('img.local-file-preview')).toBeVisible();
|
||||
await closeOpened(page);
|
||||
const first = await preview(page).locator('img').getAttribute('src');
|
||||
// The chooser still names the chosen file, as in the concept; a cancel (no files) keeps the page as it is.
|
||||
expect(await chooser(page).evaluate(node => node.files[0]?.name)).toBe('f02.png');
|
||||
@@ -308,6 +428,7 @@ test('cancel and same-file reselect', async ({ page }) => {
|
||||
const state = await urls(page);
|
||||
expect(state.created).toHaveLength(2);
|
||||
expect(state.revoked).toEqual([first]);
|
||||
// The new selection opened the viewer again (D155).
|
||||
await openViewer(page);
|
||||
await expect(viewer(page).locator('.media-position')).toHaveText('1 of 1 loaded: f02.png');
|
||||
});
|
||||
@@ -337,6 +458,7 @@ test('preview does not imply metadata support', async ({ page }) => {
|
||||
// Text, markup, SVG and a playlist under media names are unsupported and never get a URL.
|
||||
const before = (await urls(page)).created.length;
|
||||
for (const file of disguised()) {
|
||||
// Unsupported: nothing opens (D155).
|
||||
await choose(page, file);
|
||||
await expect(status(page)).toHaveText(COPY.unsupported);
|
||||
await expect(status(page)).toHaveAttribute('role', 'alert');
|
||||
@@ -360,6 +482,7 @@ test('local video plays inline and in the viewer with real tags', async ({ page,
|
||||
await expect(rail(page).locator('.meta-grid').first().locator('dd').first()).toHaveText('7');
|
||||
await expect(viewer(page).locator('#viewer-stage video')).toHaveCount(1);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
await choose(page, tinyWebm());
|
||||
await expect(preview(page).locator('video.local-file-preview')).toBeVisible();
|
||||
await openViewer(page);
|
||||
@@ -386,6 +509,9 @@ test('the toast and a rejected drop never contradict the status line', async ({
|
||||
// concept's toast interval.
|
||||
await choose(page, undecodableMp4());
|
||||
const failed = status(page).filter({ hasText: COPY.previewFailed });
|
||||
// D155: the selection opened the viewer; its media failed there too, and the page's alert waits under it.
|
||||
await expect(failed).toHaveCount(1);
|
||||
await closeOpened(page);
|
||||
await expect(failed).toHaveCount(1);
|
||||
await drop(page, '.file-drop', { files: [parityPng('f02.png'), parityPng('f03.png')] });
|
||||
const rejected = status(page).filter({ hasText: COPY.multiple });
|
||||
@@ -442,7 +568,8 @@ test('URLs and media end with their owning session', async ({ page }) => {
|
||||
let state = await urls(page);
|
||||
expect(state.revoked).toEqual([state.created[0]]);
|
||||
expect(await page.evaluate(() => window.__loads)).toBeGreaterThanOrEqual(1);
|
||||
// Route change.
|
||||
// Route change, from the page (D155: the selection opened the viewer, which makes the sidebar inert).
|
||||
await closeOpened(page);
|
||||
await page.locator('aside.sidebar nav').getByRole('link', { name: 'Overview', exact: true }).click();
|
||||
await expect(main(page).getByRole('heading', { level: 1, name: 'Overview', exact: true })).toBeVisible();
|
||||
state = await urls(page);
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
// #444 (D150): the Library's output-folder filter and multi-tag OR in "Refine this view", against the real server, and
|
||||
// the Refine dialog against the concept's Filters dialog (app.js:1345-1352).
|
||||
import {writeFile} from 'node:fs/promises';
|
||||
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
|
||||
import {test,expect} from './fixtures.mjs';
|
||||
import {captureCanonicalState} from './history-canonical-comparison.mjs';
|
||||
import {inventoryOf,diffInventory} from './library-editor-canonical-comparison.mjs';
|
||||
|
||||
const route='/prompt_manager/workspace#/library';
|
||||
const concept='/__test__/canonical/workspace';
|
||||
const cards=page=>page.locator('article[data-prompt-id]');
|
||||
const card=(page,text)=>cards(page).filter({has:page.getByText(text,{exact:true})});
|
||||
const toolbar=page=>page.locator('#workspace-main search.toolbar');
|
||||
const filtersButton=page=>toolbar(page).getByRole('button',{name:/^Filters(?: · \d)?$/});
|
||||
const refine=page=>page.getByRole('dialog',{name:'Refine this view',exact:true});
|
||||
const folder=page=>refine(page).getByRole('combobox',{name:'Folder',exact:true});
|
||||
const tags=page=>refine(page).getByRole('textbox',{name:'Tags',exact:true});
|
||||
const match=page=>refine(page).getByRole('combobox',{name:'Tag match',exact:true});
|
||||
const apply=page=>refine(page).getByRole('button',{name:'Apply filters',exact:true});
|
||||
const caption=page=>page.locator('#workspace-main .section-caption > span').first();
|
||||
|
||||
test.beforeEach(async({page,request,browserName})=>{
|
||||
await startCoverage(page,browserName);await expect(await request.post('/__test__/reset',{data:{native_library:true}})).toBeOK();
|
||||
page.libraryErrors=[];page.on('pageerror',error=>page.libraryErrors.push(error.message));
|
||||
});
|
||||
test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);expect(page.libraryErrors).toEqual([]);});
|
||||
|
||||
async function open(page) {
|
||||
await page.goto(route);await expect(caption(page)).toHaveText(/^3 positive prompts · /);
|
||||
}
|
||||
/** Open Refine and wait for the folder list it reads on opening. */
|
||||
async function openRefine(page) {
|
||||
const folders=page.waitForResponse(item=>new URL(item.url()).pathname==='/prompt_manager/library/options'&&new URL(item.url()).searchParams.get('include')==='folders');
|
||||
await filtersButton(page).click();await expect(refine(page)).toBeVisible();await folders;
|
||||
}
|
||||
|
||||
test('Refine offers Folder, Tags and Tag match; a folder narrows the shelf to prompts with images there',async({page})=>{
|
||||
await open(page);await openRefine(page);
|
||||
await expect(refine(page).locator('.field label')).toHaveText(['Folder','Tags','Tag match']);
|
||||
// Both roots hold a "shared" folder; the root's own images name no folder.
|
||||
await expect(folder(page).locator('option')).toHaveText(['All output folders','shared']);
|
||||
await expect(match(page).locator('option')).toHaveText(['All of these tags','Any of these tags','Prompts without tags']);
|
||||
await folder(page).selectOption('shared');await apply(page).click();
|
||||
await expect(caption(page)).toHaveText(/^1 positive prompt · /);
|
||||
await expect(cards(page)).toHaveCount(1);await expect(card(page,'Cat portrait')).toHaveCount(1);
|
||||
await expect(filtersButton(page)).toHaveText('Filters · 1');
|
||||
await openRefine(page);await expect(folder(page)).toHaveValue('shared');
|
||||
await refine(page).getByRole('button',{name:'Clear filters',exact:true}).click();
|
||||
await expect(caption(page)).toHaveText(/^3 positive prompts · /);await expect(filtersButton(page)).toHaveText('Filters');
|
||||
});
|
||||
|
||||
test('Tag match Any shows every prompt carrying one of the tags; changing only the match applies it',async({page})=>{
|
||||
await open(page);await openRefine(page);
|
||||
await tags(page).fill('cats, dogs');await apply(page).click();
|
||||
await expect(caption(page)).toHaveText(/^0 positive prompts · /);await expect(filtersButton(page)).toHaveText('Filters · 1');
|
||||
await openRefine(page);await match(page).selectOption('any');await apply(page).click();
|
||||
await expect(caption(page)).toHaveText(/^2 positive prompts · /);
|
||||
await expect(cards(page)).toHaveCount(2);await expect(card(page,'Cat portrait')).toHaveCount(1);await expect(card(page,'Dog landscape')).toHaveCount(1);
|
||||
await expect(filtersButton(page)).toHaveText('Filters · 2');
|
||||
});
|
||||
|
||||
test('a saved view keeps its folder and tag match',async({page})=>{
|
||||
await open(page);await openRefine(page);
|
||||
await folder(page).selectOption('shared');await tags(page).fill('cats, dogs');await match(page).selectOption('any');await apply(page).click();
|
||||
await expect(caption(page)).toHaveText(/^1 positive prompt · /);
|
||||
await toolbar(page).getByRole('button',{name:'Save view',exact:true}).click();
|
||||
const save=page.getByRole('dialog',{name:'Save this view',exact:true});
|
||||
await save.getByRole('textbox',{name:'View name',exact:true}).fill('Shared cats or dogs');
|
||||
const created=page.waitForResponse(item=>new URL(item.url()).pathname==='/prompt_manager/saved-views'&&item.request().method()==='POST');
|
||||
await save.getByRole('button',{name:'Save',exact:true}).click();
|
||||
const body=(await created).request().postDataJSON();
|
||||
expect(body.state).toMatchObject({folder:'shared',tag_mode:'any',tags:['cats','dogs']});
|
||||
await page.goto('about:blank');await open(page);
|
||||
await page.getByRole('navigation',{name:'Saved views',exact:true}).getByRole('link',{name:'Shared cats or dogs',exact:true}).click();
|
||||
await expect(caption(page)).toHaveText(/^1 positive prompt · /);await expect(filtersButton(page)).toHaveText('Filters · 3');
|
||||
});
|
||||
|
||||
test('a chosen folder missing from the folder list stays listed and chosen in Refine',async({page})=>{
|
||||
await open(page);await openRefine(page);await folder(page).selectOption('shared');await apply(page).click();
|
||||
await expect(caption(page)).toHaveText(/^1 positive prompt · /);
|
||||
// The folder has no image any more as far as the list knows (or the read failed): Apply must not drop it.
|
||||
await page.route('**/prompt_manager/library/options?include=folders',route=>route.fulfill({json:{success:true,library_epoch:'x',categories:[],folders:[]}}));
|
||||
await openRefine(page);
|
||||
await expect(folder(page).locator('option')).toHaveText(['All output folders','shared']);await expect(folder(page)).toHaveValue('shared');
|
||||
await apply(page).click();await expect(caption(page)).toHaveText(/^1 positive prompt · /);await expect(filtersButton(page)).toHaveText('Filters · 1');
|
||||
});
|
||||
|
||||
// Live B1: the server's flat list sorts full paths as text ("-" and "_" before "/"); Refine shows each folder directly
|
||||
// under its own parent and keeps case variants apart.
|
||||
test('the Folder list nests each folder under its exact parent',async({page})=>{
|
||||
const folders=['CCC','CCC/mabes9sans-dataset','CCC/mabes9sans-dataset-v1','CCC/mabes9sans-dataset/mabes9sans-dataset_IMG_GEN',
|
||||
'FLUX','FLUX/2025-12-11','flux','flux/images','flux-klein','kiko','kiko-v6','kiko/char'].sort();
|
||||
await page.route('**/prompt_manager/library/options?include=folders',route=>route.fulfill({json:{success:true,library_epoch:'x',categories:[],folders}}));
|
||||
await open(page);await openRefine(page);
|
||||
const I='\u00a0\u00a0';
|
||||
await expect(folder(page).locator('option')).toHaveText(['All output folders','CCC',`${I}mabes9sans-dataset`,`${I}${I}mabes9sans-dataset_IMG_GEN`,
|
||||
`${I}mabes9sans-dataset-v1`,'FLUX',`${I}2025-12-11`,'flux',`${I}images`,'flux-klein','kiko',`${I}char`,'kiko-v6']);
|
||||
expect(await folder(page).locator('option').evaluateAll(nodes=>nodes.map(node=>node.value))).toEqual(['','CCC','CCC/mabes9sans-dataset',
|
||||
'CCC/mabes9sans-dataset/mabes9sans-dataset_IMG_GEN','CCC/mabes9sans-dataset-v1','FLUX','FLUX/2025-12-11','flux','flux/images','flux-klein',
|
||||
'kiko','kiko/char','kiko-v6']);
|
||||
});
|
||||
|
||||
test('an older server that ignores the folder shows an error with Retry, never the whole shelf',async({page})=>{
|
||||
await open(page);
|
||||
await page.route('**/prompt_manager/library?*',async route=>{
|
||||
const response=await route.fetch();const body=await response.json();delete body.folder;delete body.tag_mode;
|
||||
await route.fulfill({response,json:body});
|
||||
});
|
||||
await openRefine(page);await folder(page).selectOption('shared');await apply(page).click();
|
||||
await expect(page.locator('#workspace-main').getByText('Prompt library could not be loaded. Retry.')).toBeVisible();
|
||||
await expect(cards(page)).toHaveCount(0);
|
||||
});
|
||||
|
||||
// D150: the Refine dialog is the concept's Filters dialog: its Folder select and Tags input, plus Tag match drawn as the
|
||||
// concept's own select; the concept's "Prompt evidence" (no data yet) and its prototype note are not shown.
|
||||
function mapFiltersDialog(reference,{folders,matchOptions}) {
|
||||
const [conceptFolder,,conceptTags]=reference.fields;
|
||||
return {...reference,
|
||||
fields:[{...conceptFolder,options:folders},{...conceptTags},{...conceptFolder,label:'Tag match',options:matchOptions}],
|
||||
small:[],
|
||||
// Production's Apply reads "Apply filters" (Images and Library share it, #249).
|
||||
actions:{...reference.actions,buttons:reference.actions.buttons.map(item=>item.text==='Apply tag filter'?{...item,text:'Apply filters'}:item)}};
|
||||
}
|
||||
for(const [width,height] of [[1600,1100],[390,844]])for(const theme of ['dark','light'])test(`Refine dialog matches the concept Filters dialog ${width} ${theme} (D150)`,async({page},testInfo)=>{
|
||||
await page.setViewportSize({width,height});
|
||||
await page.goto('about:blank');await page.goto(`${concept}#library`);await page.getByRole('combobox',{name:'Theme',exact:true}).selectOption(theme);
|
||||
await page.locator('#main').getByRole('button',{name:'Filters',exact:true}).click();await expect(page.locator('#modal')).toBeVisible();
|
||||
const reference=await page.locator('#modal').evaluate(inventoryOf);
|
||||
await captureCanonicalState(page,testInfo,`concept-filters-${width}-${theme}`,'#modal');
|
||||
await open(page);await page.getByRole('combobox',{name:'Theme',exact:true}).selectOption(theme);await openRefine(page);
|
||||
await expect(folder(page).locator('option')).toHaveCount(2);
|
||||
const actual=await refine(page).evaluate(inventoryOf);
|
||||
await captureCanonicalState(page,testInfo,`production-refine-${width}-${theme}`,'dialog[open]');
|
||||
const expected=mapFiltersDialog(reference,{folders:[['','All output folders'],['shared','shared']],matchOptions:[['all','All of these tags'],['any','Any of these tags'],['untagged','Prompts without tags']]});
|
||||
const path=testInfo.outputPath('refine-inventory.json');await writeFile(path,JSON.stringify({reference,actual},null,2));
|
||||
await testInfo.attach('refine-inventory',{path,contentType:'application/json'});
|
||||
expect(reference.fields.map(item=>item.label)).toEqual(['Folder','Prompt evidence','Tags']);
|
||||
expect(diffInventory(expected,actual)).toEqual([]);
|
||||
await refine(page).getByRole('button',{name:'Close dialog',exact:true}).click();
|
||||
await openRefine(page);await folder(page).selectOption('shared');await tags(page).fill('cats, dogs');await match(page).selectOption('any');await apply(page).click();
|
||||
await expect(caption(page)).toHaveText(/^1 positive prompt · /);await page.mouse.move(0,0);
|
||||
await captureCanonicalState(page,testInfo,`production-library-folder-any-${width}-${theme}`,'#workspace-main');
|
||||
});
|
||||
@@ -57,6 +57,26 @@ test('library groups reusable negatives under one positive without exposing nega
|
||||
for(const item of rows.items) await expect(page.locator(`article[data-prompt-id="${item.id}"] .card-meta > span`).nth(1)).toHaveText(`${item.run_count} run${item.run_count===1?'':'s'}`);
|
||||
});
|
||||
|
||||
// #464: a starved renderer (headless WebKit under CPU load, a background tab) can hold animation frames for seconds;
|
||||
// a loaded page of prompts is shown without waiting for one.
|
||||
test('a held animation frame does not hide a loaded page of prompts, the first or a later one',async({page,request})=>{
|
||||
await expect(await request.post('/__test__/reset',{data:{native_library:true,extra_prompts:45}})).toBeOK();
|
||||
const {total}=await(await request.get('/prompt_manager/library?limit=100')).json();
|
||||
await page.addInitScript(()=>{
|
||||
const request=window.requestAnimationFrame.bind(window),cancel=window.cancelAnimationFrame.bind(window);
|
||||
const held=new Map(),real=new Map();let holding=true,next=0;
|
||||
window.requestAnimationFrame=callback=>{const id=--next;if(holding)held.set(id,callback);else real.set(id,request(callback));return id;};
|
||||
window.cancelAnimationFrame=id=>{held.delete(id);if(real.has(id))cancel(real.get(id));real.delete(id);};
|
||||
window.__releaseAnimationFrames=()=>{holding=false;for(const [id,callback] of held)real.set(id,request(callback));held.clear();};
|
||||
});
|
||||
await page.goto(route);
|
||||
const more=page.getByRole('button',{name:'Load more prompts',exact:true});
|
||||
await expect(more).toHaveAttribute('aria-disabled','false');await expect(cards(page)).toHaveCount(40);
|
||||
await more.evaluate(node=>node.click());await expect(page.locator('.library-status')).toHaveText('You’re all caught up.');
|
||||
await expect(cards(page)).toHaveCount(total);
|
||||
await page.evaluate(()=>window.__releaseAnimationFrames());await nextFrame(page);
|
||||
await expect(cards(page)).toHaveCount(total);await expect(page.locator('.library-row-spacer')).toHaveCount(0);
|
||||
});
|
||||
test('history displays the selected full negative beneath its positive and filters one shared physical gallery',async({page,request})=>{
|
||||
const rows=await(await request.get('/prompt_manager/library')).json();const cat=rows.items.find(item=>item.text==='Cat portrait');
|
||||
const data=await(await request.get(`/prompt_manager/library/${cat.id}`)).json();
|
||||
@@ -135,7 +155,8 @@ test('Retry re-reads a failed category list with the shelf',async({page})=>{
|
||||
await page.locator('#workspace-main search.toolbar').getByRole('button',{name:'Filters',exact:true}).click();
|
||||
await refine(page).getByRole('button',{name:'Apply filters',exact:true}).click();await expect(refine(page)).toHaveCount(0);
|
||||
const search=page.getByRole('searchbox',{name:'Search workspace',exact:true});await search.fill(' ');
|
||||
await page.waitForTimeout(400);expect(reads).toEqual([]);await expect(cards(page)).toHaveCount(3);
|
||||
// Only the folder list Refine reads on opening (D150), never the shelf.
|
||||
await page.waitForTimeout(400);expect(reads).toEqual(['/prompt_manager/library/options']);await expect(cards(page)).toHaveCount(3);
|
||||
});
|
||||
|
||||
test('failed library reads retry while late search responses cannot overwrite newer results',async({page})=>{
|
||||
@@ -180,7 +201,7 @@ test('library toolbar matches the concept and tags live in Refine this view',asy
|
||||
// D44: the active Tags filter shows on the button; the category is the toolbar's own and is not counted.
|
||||
await expect(toolbar.getByRole('button')).toHaveText(['Save view','Filters · 1']);
|
||||
await toolbar.getByRole('button',{name:'Filters · 1',exact:true}).click();
|
||||
await expect(refine(page).locator('.field label')).toHaveText(['Tags']);await expect(refine(page).getByRole('textbox',{name:'Tags',exact:true})).toHaveValue('studio');
|
||||
await expect(refine(page).locator('.field label')).toHaveText(['Folder','Tags','Tag match']);await expect(refine(page).getByRole('textbox',{name:'Tags',exact:true})).toHaveValue('studio');
|
||||
await refine(page).getByRole('button',{name:'Clear filters',exact:true}).click();await expect(refine(page)).toHaveCount(0);
|
||||
await expect(toolbar.getByRole('button',{name:'Filters',exact:true})).toBeFocused();
|
||||
await expect(page.getByRole('combobox',{name:'Filter by category',exact:true})).toHaveValue('');await expect(cards(page)).toHaveCount(3);
|
||||
|
||||
@@ -208,7 +208,7 @@ test('a view stored under older rules still lists and opens with its usable valu
|
||||
await raw({ page: 'library', name: '', state: JSON.stringify({ q: 'dog', sort: 'name-desc' }) });
|
||||
expect(await listed(request)).toEqual([
|
||||
{ id: imagesId, page: 'images', name: 'Old cats', state: { ...IMAGE_DEFAULTS, q: 'cat' } },
|
||||
expect.objectContaining({ page: 'library', name: 'Saved view', state: { q: 'dog', category: '', sort: 'newest', tags: [] } }),
|
||||
expect.objectContaining({ page: 'library', name: 'Saved view', state: { q: 'dog', category: '', sort: 'newest', tags: [], tag_mode: 'all', folder: '' } }),
|
||||
]);
|
||||
// A state that is no JSON object is never listed, and its link behaves like a missing view.
|
||||
const corruptId = await raw({ page: 'images', name: 'Corrupt', state: '{not json' });
|
||||
@@ -233,7 +233,7 @@ test('a view stored under older rules still lists and opens with its usable valu
|
||||
// D153 (#447): the workspace no longer offers "Highest rated"; the API still stores such a view as it was saved.
|
||||
test('a view saved under Highest rated opens on Recently added on Images and the Library', async ({ page, request }) => {
|
||||
const images = await create(request, { page: 'images', name: 'Rated cats', state: { ...IMAGE_DEFAULTS, q: 'cat', sort: 'rating' } });
|
||||
const library = await create(request, { page: 'library', name: 'Rated dogs', state: { q: 'dog', category: '', sort: 'rating', tags: [] } });
|
||||
const library = await create(request, { page: 'library', name: 'Rated dogs', state: { q: 'dog', category: '', sort: 'rating', tags: [], tag_mode: 'all', folder: '' } });
|
||||
expect((await listed(request)).map(view => view.state.sort)).toEqual(['rating', 'rating']);
|
||||
const listings = [];
|
||||
page.on('request', item => {
|
||||
@@ -257,7 +257,7 @@ test('a view saved under Highest rated opens on Recently added on Images and the
|
||||
|
||||
test('leaving a saved view returns Images and the Library to their own last state (#249 M2)', async ({ page, request }) => {
|
||||
const dogs = await create(request, { page: 'images', name: 'Dogs', state: { ...IMAGE_DEFAULTS, q: 'dogs' } });
|
||||
const cats = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [] } });
|
||||
const cats = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [], tag_mode: 'all', folder: '' } });
|
||||
await page.goto(`${workspace}#/images`);
|
||||
await search(page).fill('cat');
|
||||
await expect(imageCards(page)).toHaveCount(1);
|
||||
@@ -300,7 +300,7 @@ test('a toolbar change while sources load keeps the saved Source and Folder (#24
|
||||
});
|
||||
|
||||
test('a saved view left while its list or rename is pending leaves the next page alone', async ({ page, request }) => {
|
||||
const view = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [] } });
|
||||
const view = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [], tag_mode: 'all', folder: '' } });
|
||||
let releaseList = () => {};
|
||||
const holdList = async () => {
|
||||
const held = new Promise(resolve => { releaseList = resolve; });
|
||||
@@ -348,7 +348,7 @@ test('an unknown saved view opens Images, and a failed listing offers Retry', as
|
||||
await expect(page).toHaveURL(/#\/images$/);
|
||||
await expect(main(page).getByRole('heading', { level: 1 })).toHaveText('Images');
|
||||
|
||||
const view = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [] } });
|
||||
const view = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [], tag_mode: 'all', folder: '' } });
|
||||
let failing = true;
|
||||
await page.route(`**${API}`, route => failing && route.request().method() === 'GET'
|
||||
? route.fulfill({ status: 500, contentType: 'application/json', body: '{"success":false,"error":"down"}' })
|
||||
@@ -539,7 +539,7 @@ test('Edit renames an open view, and deletes it only after confirming inside the
|
||||
});
|
||||
|
||||
test('a deleted Library view returns to the Library, and failed edits say so', async ({ page, request }) => {
|
||||
const view = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [] } });
|
||||
const view = await create(request, { page: 'library', name: 'Cats', state: { q: 'cat', category: '', sort: 'newest', tags: [], tag_mode: 'all', folder: '' } });
|
||||
await page.goto(`${workspace}#/saved/${view.id}`);
|
||||
await expect(heading(page)).toHaveText('Cats');
|
||||
const edit = main(page).locator('.heading-actions > button').first();
|
||||
|
||||
@@ -268,6 +268,71 @@ test('Watch for new outputs saves monitoring_enabled; a restart applies it', asy
|
||||
await expect(sourceList(page).locator('.operation p')).toHaveText(['output · watching', 'input · watching']);
|
||||
});
|
||||
|
||||
test('D158: Save prompts from other workflows saves save_foreign_runs live; off while monitoring is off (#247)', async ({ page, request }, testInfo) => {
|
||||
try {
|
||||
const layout = await openSources(page);
|
||||
const rows = layout.locator('section.surface > .setting-row');
|
||||
await expect(rows.locator(':scope > div > label')).toHaveText(['Watch for new outputs', 'Save prompts from other workflows', 'Worker threads']);
|
||||
const foreign = layout.getByRole('switch', { name: 'Save prompts from other workflows', exact: true });
|
||||
const foreignRow = rows.filter({ has: page.locator('#foreign-runs') });
|
||||
await expect(foreignRow.locator(':scope > div > p')).toHaveText('New outputs of workflows without a PromptManager node add their prompt to the library when it is certain.');
|
||||
await expect(foreign).toBeChecked();
|
||||
await expect(foreign).toBeEnabled();
|
||||
// Keyboard on desktop, a tap on the touch project.
|
||||
if (testInfo.project.name === 'touch') await foreign.tap();
|
||||
else {
|
||||
await foreign.focus();
|
||||
await page.keyboard.press('Space');
|
||||
}
|
||||
await expect(foreign).not.toBeChecked();
|
||||
await expect(statusOf(page)).toHaveText('Unsaved changes.');
|
||||
const saved = page.waitForResponse(response => response.request().method() === 'POST' && response.url().endsWith(SETTINGS_URL));
|
||||
await saveButton(page).click();
|
||||
const response = await saved;
|
||||
expect(response.request().postDataJSON()).toEqual(expect.objectContaining({ save_foreign_runs: false }));
|
||||
expect((await response.json()).restart_required_fields).toEqual([]);
|
||||
// Applied live: the monitor reads it for every file, so no restart is named.
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).settings.save_foreign_runs).toBe(false);
|
||||
// With Watch for new outputs drafted off nothing is watched: the row is disabled and its note says why.
|
||||
const monitoring = layout.getByRole('switch', { name: 'Watch for new outputs', exact: true });
|
||||
await monitoring.uncheck();
|
||||
await expect(foreign).toBeDisabled();
|
||||
const note = foreignRow.locator('p.small');
|
||||
await expect(note).toHaveText('Applies while Watch for new outputs is on.');
|
||||
await expect(foreign).toHaveAttribute('aria-describedby', await note.getAttribute('id'));
|
||||
await monitoring.check();
|
||||
await expect(foreign).toBeEnabled();
|
||||
await expect(note).toHaveCount(0);
|
||||
await expect(foreign).not.toHaveAttribute('aria-describedby', /./);
|
||||
await expect(statusOf(page)).toHaveText('No unsaved changes.');
|
||||
} finally {
|
||||
// The server is shared by the spec's tests: put the default back whatever happened.
|
||||
await request.post(SETTINGS_URL, { data: { save_foreign_runs: true } });
|
||||
}
|
||||
});
|
||||
|
||||
test('D158: saved off while the monitor still watches until a restart, the foreign row can still be set (D128)', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/monitor', { data: { watching: ['output', 'input'] } })).toBeOK();
|
||||
try {
|
||||
const layout = await openSources(page);
|
||||
await layout.getByRole('switch', { name: 'Watch for new outputs', exact: true }).uncheck();
|
||||
await saveButton(page).click();
|
||||
await expect(sourceList(page).locator('.operation p').first()).toHaveText('output · watching until restart');
|
||||
const foreign = layout.getByRole('switch', { name: 'Save prompts from other workflows', exact: true });
|
||||
await expect(foreign).toBeEnabled();
|
||||
await expect(layout.locator('.setting-row').filter({ has: page.locator('#foreign-runs') }).locator('p.small'))
|
||||
.toHaveText('Applies while Watch for new outputs is on.');
|
||||
await foreign.uncheck();
|
||||
const saved = page.waitForRequest(item => item.method() === 'POST' && item.url().endsWith(SETTINGS_URL));
|
||||
await saveButton(page).click();
|
||||
expect((await saved).postDataJSON()).toEqual({ save_foreign_runs: false, expected_revision: expect.any(String) });
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
} finally {
|
||||
await request.post(SETTINGS_URL, { data: { save_foreign_runs: true } });
|
||||
}
|
||||
});
|
||||
|
||||
test('worker range max is cpu_count', async ({ page, request }) => {
|
||||
const { settings } = await serverJson(request, SETTINGS_URL);
|
||||
const layout = await openSources(page);
|
||||
|
||||
@@ -287,7 +287,8 @@ test('Appearance › Thumbnails saves the server thumbnail settings and never co
|
||||
await expect(await request.post('/__test__/system/jxl-encoder', { data: { available: false } })).toBeOK();
|
||||
await expect(await request.post('/__test__/system/legacy-previews', { data: { count: 1 } })).toBeOK();
|
||||
await page.goto(`${workspace}#/system/appearance`);
|
||||
const surface = page.locator('#workspace-main .settings-layout > section.surface').nth(1);
|
||||
// #450 D156: the Viewer surface sits between Browsing & display and Thumbnails.
|
||||
const surface = page.locator('#workspace-main .settings-layout > section.surface').nth(2);
|
||||
await expect(surface.locator(':scope > h2')).toHaveText('Thumbnails');
|
||||
await expect(surface.locator('.setting-row label')).toHaveText(['Thumbnail format', 'Thumbnail quality', 'Thumbnail size limit']);
|
||||
const format = surface.locator('#thumbnail-format');
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
// #450 (TESTING.md D156): every viewer has a full-screen toggle beside ✕. Full screen fills the browser window edge to
|
||||
// edge, Centered is the concept's framed dialog; the button names the view it switches to. System › Appearance ›
|
||||
// Viewer › "Open viewer in" chooses how the viewer opens; the button changes the current view only.
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { seedConcept, openProduction } from './concept-inventory.mjs';
|
||||
import { parityPng } from './local-inspect-files.mjs';
|
||||
|
||||
test.use({ locale: 'en-US', timezoneId: 'UTC', reducedMotion: 'reduce' });
|
||||
|
||||
const CARDS = '.gallery-body > .image-grid > .media-card';
|
||||
const viewer = page => page.locator('dialog#viewer-dialog[open]');
|
||||
const modeButton = page => viewer(page).locator('.dialog-header > button#viewer-mode');
|
||||
const details = page => viewer(page).locator('#viewer-details');
|
||||
|
||||
test.beforeEach(async ({ request }, testInfo) => { testInfo.manifest = await seedConcept(request); });
|
||||
|
||||
async function openImages(page, { width = 1600, height = 1100, theme = 'dark' } = {}) {
|
||||
await page.setViewportSize({ width, height });
|
||||
await openProduction(page, 'images', theme);
|
||||
await page.locator(CARDS).first().locator('.image-open').click();
|
||||
await expect(viewer(page).locator('#viewer-details .panel-heading')).toBeVisible();
|
||||
}
|
||||
|
||||
/** The dialog's box and edges against the window. */
|
||||
const frame = page => viewer(page).evaluate(node => {
|
||||
const box = node.getBoundingClientRect(), style = getComputedStyle(node);
|
||||
return { left: Math.round(box.left), top: Math.round(box.top), right: Math.round(innerWidth - box.right),
|
||||
bottom: Math.round(document.documentElement.clientHeight - box.bottom), radius: style.borderTopLeftRadius,
|
||||
border: style.borderTopWidth, fills: Math.abs(box.width - document.documentElement.clientWidth) <= 1 };
|
||||
});
|
||||
|
||||
test('the button beside ✕ switches between Centered and Full screen and names the view it switches to', async ({ page }) => {
|
||||
await openImages(page);
|
||||
const header = await viewer(page).locator('.dialog-header').evaluate(node => [...node.children].map(child => child.id || child.localName));
|
||||
expect(header).toEqual(['div', 'viewer-mode', 'viewer-close']);
|
||||
await expect(modeButton(page)).toHaveAccessibleName('Full screen view');
|
||||
await expect(modeButton(page)).toHaveAttribute('title', 'Full screen view');
|
||||
await expect(modeButton(page).locator('svg.icon')).toHaveCount(1);
|
||||
const centered = await frame(page);
|
||||
expect(centered.left).toBeGreaterThan(0);
|
||||
expect(centered.fills).toBe(false);
|
||||
// Keyboard: the button is reachable and keeps focus after a switch.
|
||||
await viewer(page).locator('#viewer-close').focus();
|
||||
await page.keyboard.press('Shift+Tab');
|
||||
await expect(modeButton(page)).toBeFocused();
|
||||
await page.keyboard.press('Enter');
|
||||
await expect(viewer(page)).toHaveClass(/\bviewer-fullscreen\b/);
|
||||
await expect(modeButton(page)).toBeFocused();
|
||||
await expect(modeButton(page)).toHaveAccessibleName('Centered view');
|
||||
expect(await frame(page)).toEqual({ left: 0, top: 0, right: 0, bottom: 0, radius: '0px', border: '0px', fills: true });
|
||||
// The details stay toggleable in full screen.
|
||||
const toggle = viewer(page).getByRole('button', { name: 'Hide details', exact: true });
|
||||
await expect(toggle).toHaveAttribute('aria-expanded', 'true');
|
||||
await toggle.click();
|
||||
await expect(details(page)).toBeHidden();
|
||||
await viewer(page).getByRole('button', { name: 'Show details', exact: true }).click();
|
||||
await expect(details(page)).toBeVisible();
|
||||
// Back to Centered: today's dialog, with no toggle while the collapse setting is off.
|
||||
await modeButton(page).click();
|
||||
await expect(viewer(page)).not.toHaveClass(/\bviewer-fullscreen\b/);
|
||||
expect(await frame(page)).toEqual(centered);
|
||||
await expect(viewer(page).getByRole('button', { name: /^(Show|Hide) details$/ })).toHaveCount(0);
|
||||
// Navigation keeps the current view; Esc closes from full screen; the next open is Centered again.
|
||||
await modeButton(page).click();
|
||||
await viewer(page).getByRole('button', { name: 'Next media', exact: true }).click();
|
||||
await expect(viewer(page)).toHaveClass(/\bviewer-fullscreen\b/);
|
||||
await viewer(page).getByRole('button', { name: 'Hide details', exact: true }).focus();
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
await page.locator(CARDS).first().locator('.image-open').click();
|
||||
await expect(viewer(page)).not.toHaveClass(/\bviewer-fullscreen\b/);
|
||||
});
|
||||
|
||||
test('switching back to Centered keeps the collapse setting\'s toggle and hidden details (D97)', async ({ page }) => {
|
||||
await page.addInitScript(() => localStorage.setItem('gallerySettings', JSON.stringify({ sidebarCollapsedByDefault: true })));
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
await openProduction(page, 'images', 'dark');
|
||||
await page.locator(CARDS).first().locator('.image-open').click();
|
||||
await expect(viewer(page)).toBeVisible();
|
||||
await expect(details(page)).toBeHidden();
|
||||
await modeButton(page).click();
|
||||
await expect(viewer(page).getByRole('button', { name: 'Show details', exact: true })).toBeVisible();
|
||||
await modeButton(page).click();
|
||||
await expect(details(page)).toBeHidden();
|
||||
await expect(viewer(page).getByRole('button', { name: 'Show details', exact: true })).toHaveAttribute('aria-expanded', 'false');
|
||||
});
|
||||
|
||||
test('System › Appearance › Viewer chooses how the viewer opens; the button changes only the current view', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
await openProduction(page, 'system/appearance', 'dark');
|
||||
const surface = page.locator('#workspace-main section.surface').filter({ has: page.locator('h2', { hasText: /^Viewer$/ }) });
|
||||
await expect(surface.locator('.setting-row > div > :is(label, h3)')).toHaveText(['Open viewer in', 'Collapse viewer details by default']);
|
||||
const select = surface.getByRole('combobox', { name: 'Open viewer in', exact: true });
|
||||
await expect(select).toHaveValue('centered');
|
||||
await select.selectOption('fullscreen');
|
||||
await page.getByRole('button', { name: 'Save preferences', exact: true }).click();
|
||||
await expect.poll(() => page.evaluate(() => JSON.parse(localStorage.getItem('gallerySettings') || '{}').viewerOpenMode)).toBe('fullscreen');
|
||||
await page.evaluate(() => { location.hash = '#/images'; });
|
||||
await page.locator(CARDS).first().locator('.image-open').click();
|
||||
await expect(viewer(page)).toHaveClass(/\bviewer-fullscreen\b/);
|
||||
await expect(modeButton(page)).toHaveAccessibleName('Centered view');
|
||||
await modeButton(page).click();
|
||||
await expect(viewer(page)).not.toHaveClass(/\bviewer-fullscreen\b/);
|
||||
expect(await page.evaluate(() => JSON.parse(localStorage.getItem('gallerySettings')).viewerOpenMode)).toBe('fullscreen');
|
||||
});
|
||||
|
||||
test('Inspect opens full screen with its details when that is the default (D155)', async ({ page }) => {
|
||||
await page.addInitScript(() => localStorage.setItem('gallerySettings', JSON.stringify({ viewerOpenMode: 'fullscreen', sidebarCollapsedByDefault: true })));
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
await openProduction(page, 'inspect', 'dark');
|
||||
await page.locator('#local-file').setInputFiles(parityPng('f02.png'));
|
||||
await expect(viewer(page)).toHaveClass(/\bviewer-fullscreen\b/);
|
||||
await expect(details(page)).toBeVisible();
|
||||
await expect(viewer(page).getByRole('button', { name: 'Hide details', exact: true })).toBeVisible();
|
||||
});
|
||||
|
||||
test('the Library gallery viewer has the button too', async ({ page }, testInfo) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
await openProduction(page, `library/${testInfo.manifest.prompts[0].positive_id}`, 'dark');
|
||||
await page.locator('.media-card .image-open').first().click();
|
||||
await expect(modeButton(page)).toHaveAccessibleName('Full screen view');
|
||||
});
|
||||
|
||||
test('at 390px the viewer is already full-bleed and the button hides', async ({ page }) => {
|
||||
await openImages(page, { width: 390, height: 844 });
|
||||
await expect(viewer(page).locator('#viewer-mode')).toBeHidden();
|
||||
await expect(viewer(page).locator('#viewer-close')).toBeVisible();
|
||||
});
|
||||
|
||||
test('full screen holds at 200% zoom and coarse pointers get a 40px target', async ({ page, browserName }) => {
|
||||
await openImages(page);
|
||||
await modeButton(page).click();
|
||||
await page.evaluate(() => { document.documentElement.style.zoom = '2'; });
|
||||
const zoomed = await frame(page);
|
||||
expect(zoomed).toMatchObject({ left: 0, top: 0, right: 0, bottom: 0 });
|
||||
await page.evaluate(() => { document.documentElement.style.zoom = '1'; });
|
||||
if (await page.evaluate(() => matchMedia('(pointer: coarse)').matches)) {
|
||||
const size = await modeButton(page).evaluate(node => [node.getBoundingClientRect().width, node.getBoundingClientRect().height]);
|
||||
expect(Math.min(...size)).toBeGreaterThanOrEqual(40);
|
||||
}
|
||||
test.info().annotations.push({ type: 'engine', description: browserName });
|
||||
});
|
||||
@@ -0,0 +1,167 @@
|
||||
// #444 (D149): Tags & categories tag search, tag sort and the Untagged prompts row, against the real server.
|
||||
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
|
||||
import {test,expect} from './fixtures.mjs';
|
||||
import {armUnseenRestores} from './unseen-restore.mjs';
|
||||
import {captureCanonicalState} from './history-canonical-comparison.mjs';
|
||||
|
||||
const route='/prompt_manager/workspace#/tags';
|
||||
const main=page=>page.locator('#workspace-main');
|
||||
const tagSurface=page=>main(page).locator('.two-col > section.surface').first();
|
||||
const table=page=>main(page).getByRole('table',{name:'Reusable tags',exact:true});
|
||||
const rows=page=>table(page).locator('tbody tr');
|
||||
const names=page=>rows(page).locator('td:first-child');
|
||||
const tagRow=(page,name)=>rows(page).filter({has:page.locator('td:first-child').getByText(name,{exact:true})});
|
||||
const search=page=>page.getByRole('searchbox',{name:'Search tags',exact:true});
|
||||
const sort=page=>page.getByRole('combobox',{name:'Sort tags',exact:true});
|
||||
const untagged=page=>tagSurface(page).locator('.operation').filter({has:page.getByRole('heading',{level:3,name:'Untagged prompts',exact:true})});
|
||||
const status=page=>tagSurface(page).locator('search.vocabulary-tools [role="status"]');
|
||||
const shelf=page=>page.locator('article[data-prompt-id]');
|
||||
const filtersButton=(page,name)=>page.locator('#workspace-main search.toolbar').getByRole('button',{name,exact:true});
|
||||
const refine=page=>page.getByRole('dialog',{name:'Refine this view',exact:true});
|
||||
|
||||
async function seed(request,data={}) {
|
||||
const response=await request.post('/__test__/vocabulary/reset',{data});await expect(response).toBeOK();return response.json();
|
||||
}
|
||||
let manifest;
|
||||
test.beforeEach(async({page,request,browserName})=>{
|
||||
await startCoverage(page,browserName);await armUnseenRestores(page);
|
||||
// Two minimal recipes without tags (the concept recipes all carry tags).
|
||||
manifest=await seed(request,{extra_categories:2});
|
||||
page.vocabularyErrors=[];page.on('pageerror',error=>page.vocabularyErrors.push(error.message));
|
||||
});
|
||||
test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);expect(page.vocabularyErrors).toEqual([]);});
|
||||
|
||||
async function open(page) {
|
||||
await page.goto(route);await expect(rows(page)).toHaveCount(manifest.tags.length);
|
||||
}
|
||||
|
||||
test('the tag vocabulary surface orders its paragraph, Untagged prompts, the search and sort, then the table',async({page})=>{
|
||||
await open(page);
|
||||
const order=await tagSurface(page).evaluate(node=>[...node.children].map(child=>child.matches('.operation')?'operation':
|
||||
child.matches('search.toolbar')?'toolbar':child.matches('.table-wrap')?'table':child.tagName.toLowerCase()).slice(0,5));
|
||||
expect(order).toEqual(['h2','p','operation','toolbar','table']);
|
||||
await expect(untagged(page).locator('p')).toHaveText('2 positive prompts');
|
||||
await expect(untagged(page).getByRole('button')).toHaveText(['Browse']);
|
||||
await expect(search(page)).toHaveAttribute('placeholder','Search tags…');
|
||||
await expect(search(page)).toHaveAttribute('maxlength','200');
|
||||
await expect(sort(page).locator('option')).toHaveText(['Oldest first','Name A–Z','Name Z–A','Most prompts','Fewest prompts']);
|
||||
await expect(sort(page)).toHaveValue('created');
|
||||
await expect(tagSurface(page).locator('label.search-field svg, label.search-field .ca-icon').first()).toBeVisible();
|
||||
});
|
||||
|
||||
test('search narrows the tags on the server, says when nothing matches and keeps the categories',async({page})=>{
|
||||
await open(page);
|
||||
const categories=await main(page).locator('.two-col > section.surface').nth(1).locator('.operation h3').allTextContents();
|
||||
const reads=[];page.on('request',request=>{if(request.url().includes('/vocabulary/tags?'))reads.push(new URL(request.url()).searchParams);});
|
||||
await search(page).fill('LIGHT');
|
||||
await expect(names(page)).toHaveText(['warm_light','soft_light']);
|
||||
await expect(status(page)).toHaveText('2 tags');
|
||||
expect(reads.at(-1).get('search')).toBe('LIGHT');
|
||||
// Literal: an underscore is not a wildcard.
|
||||
await search(page).fill('a_c');await expect(rows(page)).toHaveCount(0);
|
||||
await expect(tagSurface(page).locator('p.empty')).toHaveText('No tags match this search.');
|
||||
await expect(status(page)).toHaveText('No tags match this search.');
|
||||
await expect(main(page).locator('.two-col > section.surface').nth(1).locator('.operation h3')).toHaveText(categories);
|
||||
await search(page).fill('');await expect(rows(page)).toHaveCount(manifest.tags.length);
|
||||
await expect(status(page)).toHaveText(`${manifest.tags.length} tags`);
|
||||
});
|
||||
|
||||
test('sort orders every tag by name or by prompts, ties in creation order',async({page})=>{
|
||||
await open(page);
|
||||
const created=await names(page).allTextContents();
|
||||
await sort(page).selectOption('name');
|
||||
const byName=[...created].sort((a,b)=>a.toLowerCase()<b.toLowerCase()?-1:a.toLowerCase()>b.toLowerCase()?1:0);
|
||||
await expect(names(page)).toHaveText(byName);
|
||||
await sort(page).selectOption('name-desc');await expect(names(page)).toHaveText([...byName].reverse());
|
||||
await sort(page).selectOption('prompts');
|
||||
await expect(rows(page).first().locator('td:first-child')).toHaveText('cinematic');
|
||||
const counts=(await rows(page).locator('td:nth-child(2)').allTextContents()).map(Number);
|
||||
expect(counts).toEqual([...counts].sort((a,b)=>b-a));
|
||||
await sort(page).selectOption('prompts-asc');
|
||||
const ascending=(await rows(page).locator('td:nth-child(2)').allTextContents()).map(Number);
|
||||
expect(ascending).toEqual([...ascending].sort((a,b)=>a-b));
|
||||
await sort(page).selectOption('created');await expect(names(page)).toHaveText(created);
|
||||
});
|
||||
|
||||
test('a rename while searching re-reads the same search and sort',async({page})=>{
|
||||
await open(page);
|
||||
await sort(page).selectOption('name');await search(page).fill('light');
|
||||
await expect(names(page)).toHaveText(['soft_light','warm_light']);
|
||||
await tagRow(page,'warm_light').getByRole('button',{name:'Rename / merge',exact:true}).click();
|
||||
const box=page.locator('dialog.vocabulary-dialog');
|
||||
await box.getByLabel('New name; existing names merge',{exact:true}).fill('amber_light');
|
||||
await box.getByRole('button',{name:'Apply',exact:true}).click();await expect(box).toHaveCount(0);
|
||||
await expect(names(page)).toHaveText(['amber_light','soft_light']);
|
||||
await expect(tagRow(page,'amber_light').getByRole('button',{name:'Rename / merge',exact:true})).toBeFocused();
|
||||
await expect(search(page)).toHaveValue('light');await expect(sort(page)).toHaveValue('name');
|
||||
});
|
||||
|
||||
test('Untagged prompts Browse opens the Library on exactly the prompts without tags',async({page})=>{
|
||||
await open(page);
|
||||
await untagged(page).getByRole('button',{name:'Browse',exact:true}).click();
|
||||
await expect(page).toHaveURL(/#\/library$/);
|
||||
await expect(main(page).getByRole('heading',{level:1})).toBeFocused();
|
||||
await expect(shelf(page)).toHaveCount(2);
|
||||
await expect(page.locator('#workspace-main .section-caption > span').first()).toHaveText(/^2 positive prompts · /);
|
||||
for(const card of await shelf(page).all())await expect(card.locator('.tags .chip')).toHaveCount(0);
|
||||
await filtersButton(page,'Filters · 1').click();
|
||||
await expect(refine(page).getByRole('combobox',{name:'Tag match',exact:true})).toHaveValue('untagged');
|
||||
await expect(refine(page).getByRole('textbox',{name:'Tags',exact:true})).toBeDisabled();
|
||||
// Changing only the Tag match applies it: the whole shelf returns.
|
||||
await refine(page).getByRole('combobox',{name:'Tag match',exact:true}).selectOption('all');
|
||||
await expect(refine(page).getByRole('textbox',{name:'Tags',exact:true})).toBeEnabled();
|
||||
await refine(page).getByRole('button',{name:'Apply filters',exact:true}).click();
|
||||
await expect(filtersButton(page,'Filters')).toBeVisible();
|
||||
await expect.poll(()=>shelf(page).count()).toBeGreaterThan(2);
|
||||
});
|
||||
|
||||
test('Refine › Prompts without tags on the shelf drops typed tags and Clear restores All',async({page})=>{
|
||||
await page.goto('/prompt_manager/workspace#/library');await expect(shelf(page).first()).toBeVisible();
|
||||
await filtersButton(page,'Filters').click();
|
||||
await refine(page).getByRole('textbox',{name:'Tags',exact:true}).fill('cinematic');
|
||||
await refine(page).getByRole('combobox',{name:'Tag match',exact:true}).selectOption('untagged');
|
||||
await refine(page).getByRole('button',{name:'Apply filters',exact:true}).click();
|
||||
await expect(shelf(page)).toHaveCount(2);
|
||||
await filtersButton(page,'Filters · 1').click();
|
||||
await expect(refine(page).getByRole('textbox',{name:'Tags',exact:true})).toHaveValue('');
|
||||
await refine(page).getByRole('button',{name:'Clear filters',exact:true}).click();
|
||||
await expect(filtersButton(page,'Filters')).toBeVisible();await expect.poll(()=>shelf(page).count()).toBeGreaterThan(2);
|
||||
});
|
||||
|
||||
test('an older server that ignores the search shows an error with Retry, never an unfiltered list',async({page})=>{
|
||||
await open(page);
|
||||
// An un-restarted server answers without the echo.
|
||||
await page.route('**/prompt_manager/vocabulary/tags?*',async route=>{
|
||||
const response=await route.fetch();const body=await response.json();delete body.search;delete body.sort;
|
||||
await route.fulfill({response,json:body});
|
||||
});
|
||||
await search(page).fill('light');
|
||||
await expect(tagSurface(page).locator('.vocabulary-error')).toBeVisible();
|
||||
await expect(rows(page)).toHaveCount(0);
|
||||
await expect(tagSurface(page).getByRole('button',{name:'Retry',exact:true})).toBeVisible();
|
||||
});
|
||||
|
||||
test('the search and sort fit 390px without a page scroll',async({page})=>{
|
||||
await page.setViewportSize({width:390,height:844});await open(page);
|
||||
const layout=await tagSurface(page).evaluate(node=>{
|
||||
const tools=node.querySelector('search.vocabulary-tools'),field=tools.querySelector('.search-field'),select=tools.querySelector('select');
|
||||
const box=node.getBoundingClientRect(),fieldBox=field.getBoundingClientRect(),selectBox=select.getBoundingClientRect();
|
||||
return {inside:fieldBox.left>=box.left&&selectBox.right<=box.right+0.5,fullField:Math.round(fieldBox.width)===Math.round(tools.getBoundingClientRect().width)};
|
||||
});
|
||||
expect(layout).toEqual({inside:true,fullField:true});
|
||||
expect(await page.evaluate(()=>document.documentElement.scrollWidth-document.documentElement.clientWidth)).toBeLessThanOrEqual(1);
|
||||
});
|
||||
|
||||
// Artifact-only captures for the maintainer's quick check (no pixel assertions): a search with a sort, the Library on
|
||||
// Untagged prompts and its Refine dialog.
|
||||
for(const [width,height] of [[1600,1100],[390,844]])for(const theme of ['dark','light'])test(`D149 captures ${width} ${theme}`,async({page},testInfo)=>{
|
||||
await page.setViewportSize({width,height});await open(page);
|
||||
await page.getByRole('combobox',{name:'Theme',exact:true}).selectOption(theme);
|
||||
await sort(page).selectOption('prompts');await search(page).fill('i');await expect(status(page)).toHaveText(/^\d+ tags$/);
|
||||
await page.mouse.move(0,0);
|
||||
await captureCanonicalState(page,testInfo,`production-tags-search-${width}-${theme}`,'#workspace-main');
|
||||
await untagged(page).getByRole('button',{name:'Browse',exact:true}).click();await expect(shelf(page)).toHaveCount(2);
|
||||
await captureCanonicalState(page,testInfo,`production-library-untagged-${width}-${theme}`,'#workspace-main');
|
||||
await filtersButton(page,'Filters · 1').click();await expect(refine(page)).toBeVisible();
|
||||
await captureCanonicalState(page,testInfo,`production-refine-untagged-${width}-${theme}`,'dialog[open]');
|
||||
});
|
||||
@@ -3,7 +3,7 @@ import {writeFile} from 'node:fs/promises';
|
||||
import {test,expect} from './fixtures.mjs';
|
||||
import {armUnseenRestores,unseenRestore,followRestore,SEEN_RESTORE_PILL,SEEN_RESTORE_TOAST} from './unseen-restore.mjs';
|
||||
import {captureCanonicalState} from './history-canonical-comparison.mjs';
|
||||
import {pageInventoryOf,inventoryOf,diffInventory,mapVocabularyConcept,mapVocabularyDialog} from './vocabulary-canonical-comparison.mjs';
|
||||
import {pageInventoryOf,inventoryOf,diffInventory,mapVocabularyConcept,mapVocabularyDialog,d149Of,toolbarInventoryOf} from './vocabulary-canonical-comparison.mjs';
|
||||
|
||||
const route='/prompt_manager/workspace#/tags';
|
||||
// Prompts per concept tag (app.js:71-165): first appearance order, Library Browse counts.
|
||||
@@ -786,6 +786,15 @@ async function openProductionTags(page,theme,motion) {
|
||||
await expect(rows(page)).toHaveCount(manifest.tags.length);await expect(categoryRow(page,'Landscapes')).toHaveCount(1);
|
||||
}
|
||||
const pageOf=(page,production)=>page.locator(production?'#workspace-main':'#main').evaluate(pageInventoryOf);
|
||||
/** D149 donor: the concept's own page toolbar (Prompt library), whose search field and select the tag search reuses. */
|
||||
async function conceptToolbar(page,theme) {
|
||||
await page.goto('about:blank');await page.goto(`${concept}#library`);
|
||||
await page.getByRole('combobox',{name:'Theme',exact:true}).selectOption(theme);
|
||||
const inventory=await page.locator('#main search.toolbar').evaluate(toolbarInventoryOf);
|
||||
await page.goto('about:blank');await page.goto(`${concept}#tags`);await page.getByRole('combobox',{name:'Theme',exact:true}).selectOption(theme);
|
||||
return inventory;
|
||||
}
|
||||
const tagToolbar=page=>page.locator('#workspace-main .two-col > section.surface:first-child > search.toolbar').evaluate(toolbarInventoryOf);
|
||||
/** Breadcrumb, document title (D7: "Tags", only the concept suffix differs) and the keyboard focus ring on a row action. */
|
||||
async function chromeOf(page,production) {
|
||||
const scope=page.locator(production?'#workspace-main':'#main');
|
||||
@@ -805,7 +814,7 @@ async function chromeOf(page,production) {
|
||||
}
|
||||
/** Concept rows are the concept's own sample tags; production appends the fixture-only tags after them (D4 creation order). */
|
||||
function comparePage(reference,actual) {
|
||||
const expected=mapVocabularyConcept(reference);
|
||||
const expected=mapVocabularyConcept(reference,{},d149Of(actual));
|
||||
const shared=expected.table.rows.length;
|
||||
return {diff:diffInventory(expected,{...actual,table:{...actual.table,rows:actual.table.rows.slice(0,shared)}}),
|
||||
extra:actual.table.rows.slice(shared).map(row=>row.tag)};
|
||||
@@ -818,13 +827,18 @@ for(const [width,height] of SIZES)for(const theme of ['dark','light'])test(`tags
|
||||
await openConceptTags(page,theme,motion);const reference=await pageOf(page,false);
|
||||
if(motion==='no-preference')await captureCanonicalState(page,testInfo,`concept-tags-${width}-${theme}`,'#main');
|
||||
const referenceChrome=await chromeOf(page,false);
|
||||
const referenceTools=await conceptToolbar(page,theme);
|
||||
await openProductionTags(page,theme,motion);const actual=await pageOf(page,true);
|
||||
const actualTools=await tagToolbar(page);
|
||||
if(motion==='no-preference')await captureCanonicalState(page,testInfo,`production-tags-${width}-${theme}`,'#workspace-main');
|
||||
result[motion]={reference,actual,...comparePage(reference,actual),referenceChrome,actualChrome:await chromeOf(page,true)};
|
||||
result[motion]={reference,actual,...comparePage(reference,actual),referenceChrome,actualChrome:await chromeOf(page,true),
|
||||
toolbarDiff:diffInventory(referenceTools,actualTools)};
|
||||
}
|
||||
const path=testInfo.outputPath('tags-geometry.json');await writeFile(path,JSON.stringify(result,null,2));
|
||||
await testInfo.attach('tags-geometry',{path,contentType:'application/json'});
|
||||
for(const [motion,{reference,diff,extra,referenceChrome,actualChrome}] of Object.entries(result)) {
|
||||
for(const [motion,{reference,diff,extra,referenceChrome,actualChrome,toolbarDiff}] of Object.entries(result)) {
|
||||
// D149 (#444): the tag search and sort carry the concept page toolbar's tokens.
|
||||
expect.soft(toolbarDiff,`${motion} tag search and sort against the concept toolbar`).toEqual([]);
|
||||
expect.soft(referenceChrome,`${motion} concept chrome`).toMatchObject({breadcrumb:'Tags',title:'Tags',ring:{visible:true},linkRing:{outlineOffset:'3px'}});
|
||||
expect.soft(actualChrome,`${motion} breadcrumb, title and focus ring`).toEqual(referenceChrome);
|
||||
expect.soft(reference.heading.actions.buttons.map(item=>item.text),`${motion} concept heading actions`).toEqual(['+ Create tag','Auto-tag images']);
|
||||
@@ -894,6 +908,13 @@ test('mutation check: the comparison reports a changed surface padding and a cha
|
||||
expect(paths).toContain('surfaces.1.paddingLeft');
|
||||
expect(paths).toContain('table.rows.0.actions.0');
|
||||
expect(paths).toContain('table.rowActions.buttons.0.text');
|
||||
// D149: a moved Untagged prompts row and a restyled tag search are reported too.
|
||||
await page.addStyleTag({content:'.vocabulary-main .surface > .operation{margin-top:6px!important}.vocabulary-tools .search-field{border-radius:9px!important}'});
|
||||
const moved=comparePage(reference,await pageOf(page,true)).diff.map(item=>item.path);
|
||||
expect(moved).toContain('d149.offset');
|
||||
const tools=await conceptToolbar(page,'dark');await openProductionTags(page,'dark','no-preference');
|
||||
await page.addStyleTag({content:'.vocabulary-tools .search-field{border-radius:9px!important}'});
|
||||
expect(diffInventory(tools,await tagToolbar(page)).map(item=>item.path)).toContain('field.borderTopLeftRadius');
|
||||
// The dialog comparison is not vacuous either: a removed Cancel and a recolored field are reported.
|
||||
const donors=await conceptDonors(page,'dark');
|
||||
await openConceptTags(page,'dark','no-preference');await conceptDialogs.create(page);
|
||||
|
||||
@@ -472,7 +472,7 @@ test('System is native: the concept shell, section navigation, sections and the
|
||||
if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible();
|
||||
if (hash === 'sources') await expect(layout.locator('#source-list .operation')).toHaveCount(2);
|
||||
expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`)))
|
||||
.toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'section.surface', 'div.save-row'],
|
||||
.toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'section.surface', 'section.surface', 'div.save-row'] /* D156: Browsing & display, Viewer, Thumbnails */,
|
||||
sources: ['div.notice', 'section.surface', 'div.save-row'],
|
||||
integrations: ['section.surface', 'section.surface', 'div.save-row'],
|
||||
maintenance: ['div.notice', 'section.surface', 'section.surface', 'div.save-row'],
|
||||
|
||||
@@ -314,6 +314,16 @@ test('fresh registered PM variants preserve exact raw whitespace and known empty
|
||||
const runId = (await readExecutionRuns(execution.databasePath, id)).rows[0].run_id;
|
||||
const response = await execution.request.get(`/prompt_manager/library/${value.positive_prompt_id}/history`);expect(response.ok()).toBe(true);
|
||||
expect((await response.json()).items.map(item => item.run_id)).toContain(runId);
|
||||
if (kind !== 'PromptManager') continue;
|
||||
// #460: an upper-case re-queue counts toward the same prompt and joins its history
|
||||
const upper = await submit(execution.request, recipeGraph({ kind, text: `${kind.toUpperCase()} E\u0301` }));
|
||||
await history(execution.request, upper);
|
||||
const variant = (await capture(execution, upper)).contexts[0];
|
||||
expect(variant.positive.raw).toBe(`${kind.toUpperCase()} E\u0301`);
|
||||
expect(variant.positive_prompt_id, 'the case variant names the same prompt').toBe(value.positive_prompt_id);
|
||||
const variantRun = (await readExecutionRuns(execution.databasePath, upper)).rows[0].run_id;
|
||||
const scoped = await execution.request.get(`/prompt_manager/library/${value.positive_prompt_id}/history`);
|
||||
expect((await scoped.json()).items.map(item => item.run_id)).toContain(variantRun);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
@@ -334,8 +334,15 @@ async def image_monitor(request: web.Request) -> web.Response:
|
||||
|
||||
@PromptServer.instance.routes.get("/pm_execution_fixture/monitor")
|
||||
async def image_monitor_state(request: web.Request) -> web.Response:
|
||||
"""File names of keyless images waiting for their run's recorded outputs."""
|
||||
"""File names of images waiting for their run's recorded outputs: keyless
|
||||
ones (``awaiting``) and foreign ones (``foreign``, #247)."""
|
||||
monitor = _plugin_module("utils.image_monitor")._monitor_instance
|
||||
handler = getattr(monitor, "handler", None)
|
||||
waiting = handler.awaiting_history() if handler is not None else []
|
||||
return web.json_response({"awaiting": [os.path.basename(p) for p in waiting]})
|
||||
foreign = handler.awaiting_foreign() if handler is not None else []
|
||||
return web.json_response(
|
||||
{
|
||||
"awaiting": [os.path.basename(p) for p in waiting],
|
||||
"foreign": [os.path.basename(p) for p in foreign],
|
||||
}
|
||||
)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
// #342 on the real pinned host: the image monitor links an output only to a prompt
|
||||
// that provably produced it. An unknown node (PMHistoryCached) stands in for the
|
||||
// maintainer's LLM; KSampler is the fixture's deterministic stand-in.
|
||||
// maintainer's LLM; KSampler is the fixture's deterministic stand-in. #247: an output
|
||||
// of a workflow without a PromptManager node gets its own prompt.
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { execFile } from 'node:child_process';
|
||||
import { promisify } from 'node:util';
|
||||
@@ -141,6 +142,68 @@ test('an output saved without metadata links once its recorded run names a store
|
||||
}
|
||||
});
|
||||
|
||||
// #247: no PromptManager node at all, only ComfyUI's own text encoders, sampler and
|
||||
// saver (the fixture's inputs and decoder stand in for the model files).
|
||||
function foreignGraph({ text, seed = 1, prefix }) {
|
||||
return {
|
||||
'10': { class_type: 'PMHistoryInputs', inputs: {} },
|
||||
'20': { class_type: 'CLIPTextEncode', inputs: { text, clip: ['10', 0] } },
|
||||
'30': { class_type: 'CLIPTextEncode', inputs: { text: '', clip: ['10', 0] } },
|
||||
'40': { class_type: 'KSampler', inputs: { model: ['10', 1], latent_image: ['10', 2], positive: ['20', 0],
|
||||
negative: ['30', 0], seed, steps: 1, cfg: 1, sampler_name: 'euler', scheduler: 'normal', denoise: 1 } },
|
||||
'60': { class_type: 'PMHistoryImages', inputs: { value: 64, batch: 1, latent: ['40', 0] } },
|
||||
'70': { class_type: 'SaveImage', inputs: { images: ['60', 0], filename_prefix: prefix } },
|
||||
};
|
||||
}
|
||||
|
||||
test('an output of a workflow without a PromptManager node creates and links its prompt; a second run reuses it', async ({ execution }, testInfo) => {
|
||||
const text = `foreign lighthouse at dusk ${testInfo.project.name} ${randomUUID()}`;
|
||||
const first = await saved(execution.request, foreignGraph({ text, prefix: 'foreign/run' }));
|
||||
await linkedTo(execution, first, text);
|
||||
// Only the seed changes: a new file, the same text, so the same stored prompt.
|
||||
const second = await saved(execution.request, foreignGraph({ text, seed: 2, prefix: 'foreign/run' }));
|
||||
await linkedTo(execution, second, text);
|
||||
const state = await readLinks(execution.databasePath, [first, second], [text]);
|
||||
expect(state).toEqual({ links: { [first]: text, [second]: text }, stored: { [text]: true } });
|
||||
// Decided from the graph at settle: nothing waits for the run's recorded outputs.
|
||||
expect((await (await execution.request.get('/pm_execution_fixture/monitor')).json()).foreign).toEqual([]);
|
||||
});
|
||||
|
||||
// CUJ 1: two branches saved under one prefix, so neither the graph nor the file name
|
||||
// names a file's branch; the run's recorded saver does, once its outputs are sealed.
|
||||
function sharedPrefixGraph(texts) {
|
||||
const nodes = {
|
||||
'10': { class_type: 'PMHistoryInputs', inputs: {} },
|
||||
'30': { class_type: 'CLIPTextEncode', inputs: { text: '', clip: ['10', 0] } },
|
||||
};
|
||||
texts.forEach((text, index) => {
|
||||
const base = (index + 1) * 100;
|
||||
nodes[String(base)] = { class_type: 'CLIPTextEncode', inputs: { text, clip: ['10', 0] } };
|
||||
nodes[String(base + 1)] = { class_type: 'KSampler', inputs: { model: ['10', 1], latent_image: ['10', 2],
|
||||
positive: [String(base), 0], negative: ['30', 0], seed: index + 1, steps: 1, cfg: 1, sampler_name: 'euler',
|
||||
scheduler: 'normal', denoise: 1 } };
|
||||
nodes[String(base + 2)] = { class_type: 'PMHistoryImages', inputs: { value: 64, batch: 1, latent: [String(base + 1), 0] } };
|
||||
nodes[String(base + 3)] = { class_type: 'SaveImage', inputs: { images: [String(base + 2), 0], filename_prefix: 'foreign/dataset' } };
|
||||
});
|
||||
return nodes;
|
||||
}
|
||||
|
||||
test('each branch of a shared-prefix workflow without a PromptManager node links to its own prompt after the run', async ({ execution }, testInfo) => {
|
||||
const tag = `${testInfo.project.name} ${randomUUID()}`;
|
||||
const texts = [`dataset branch one harbor ${tag}`, `dataset branch two meadow ${tag}`];
|
||||
const id = randomUUID();
|
||||
const response = await execution.request.post('/prompt', { data: { prompt: sharedPrefixGraph(texts), prompt_id: id } });
|
||||
expect(response.ok()).toBe(true);
|
||||
let item;
|
||||
await expect.poll(async () => {
|
||||
item = (await (await execution.request.get(`/history/${id}`)).json())[id];
|
||||
return item?.status?.messages?.some(([kind]) => kind === 'execution_success') || false;
|
||||
}, { timeout: 15000 }).toBe(true);
|
||||
const files = ['103', '203'].map(node => item.outputs[node].images[0].filename);
|
||||
for (const [index, file] of files.entries()) await linkedTo(execution, file, texts[index]);
|
||||
expect((await (await execution.request.get('/pm_execution_fixture/monitor')).json()).foreign).toEqual([]);
|
||||
});
|
||||
|
||||
// #332 on the real host: the migration ran at startup and the image category routes
|
||||
// set, list and remove an own category on a generated output.
|
||||
test('an output gets its own image category and loses it again', async ({ execution }, testInfo) => {
|
||||
|
||||
@@ -36,11 +36,11 @@ async function watchHost(page,id,reset=true) {
|
||||
}
|
||||
const restore=page=>page.evaluate(async()=>{const {app}=await import('/scripts/app.js');Object.assign(app,window.inspectOriginals);});
|
||||
|
||||
/** Chooses `file` on the Inspect page and opens the shared viewer on its local record. */
|
||||
/** Chooses `file` on the Inspect page, which opens the shared viewer on its local record (D155, #448). */
|
||||
async function inspect(workspace,{file,positive,shown}) {
|
||||
await workspace.locator('#local-file').setInputFiles(file());
|
||||
const open=workspace.locator('#workspace-main .heading-actions').getByRole('button',{name:'Open file metadata',exact:true});
|
||||
await expect(open).toHaveAttribute('aria-disabled','false');await open.click();
|
||||
await expect(open).toHaveAttribute('aria-disabled','false');
|
||||
const rail=workspace.locator('dialog#viewer-dialog[open] #viewer-details');
|
||||
await expect(rail.locator('.prompt-section.positive p.prompt-text')).toHaveText(positive);
|
||||
await expect(rail.locator('.prompt-section.negative p.prompt-text')).toHaveText(shown);
|
||||
|
||||
@@ -192,6 +192,7 @@
|
||||
"settings.monitoring_enabled",
|
||||
"settings.result_timeout",
|
||||
"settings.safety_backup",
|
||||
"settings.save_foreign_runs",
|
||||
"settings.thumbnail_edge",
|
||||
"settings.thumbnail_format",
|
||||
"settings.thumbnail_quality",
|
||||
|
||||
@@ -79,6 +79,15 @@ test("the workspace keys have their own defaults and validation", () => {
|
||||
assert.equal(normalizeGallerySettings({ defaultViewMode: "masonry" }).defaultViewMode, "grid");
|
||||
});
|
||||
|
||||
// #450 (D156): the viewer's default view, Centered unless Full screen was chosen.
|
||||
test("the viewer open mode defaults to centered and accepts only its two values", () => {
|
||||
const { VIEWER_OPEN_MODES } = require("../../web/js/gallery-settings.js");
|
||||
assert.deepEqual(VIEWER_OPEN_MODES, ["centered", "fullscreen"]);
|
||||
assert.equal(DEFAULT_SETTINGS.viewerOpenMode, "centered");
|
||||
assert.equal(normalizeGallerySettings({ viewerOpenMode: "fullscreen" }).viewerOpenMode, "fullscreen");
|
||||
for (const value of ["full", true, 1, null]) assert.equal(normalizeGallerySettings({ viewerOpenMode: value }).viewerOpenMode, "centered");
|
||||
});
|
||||
|
||||
// #330 S13: the legacy gallery's Page options edit only the page-only keys (PAGE_OPTION_KEYS); System › Appearance owns the rest, so a Save
|
||||
// or a Reset from the legacy page keeps every other stored key.
|
||||
function legacyGallery(stored, controls, extra = {}) {
|
||||
@@ -112,7 +121,8 @@ const PAGE_OPTION_CONTROLS = () => ({
|
||||
|
||||
test("a legacy gallery Save keeps the workspace keys and every other stored key", () => {
|
||||
const { manager, saved } = legacyGallery({ workspaceLayout: "masonry", workspaceDensity: "compact", someOtherTool: 1,
|
||||
imageQuality: "high", showImageInfo: false, videoMute: false, thumbnailPromptDismissed: "sig", defaultViewMode: "list" },
|
||||
imageQuality: "high", showImageInfo: false, videoMute: false, thumbnailPromptDismissed: "sig", defaultViewMode: "list",
|
||||
viewerOpenMode: "fullscreen" },
|
||||
PAGE_OPTION_CONTROLS());
|
||||
manager.saveSettings();
|
||||
const result = saved();
|
||||
@@ -123,6 +133,7 @@ test("a legacy gallery Save keeps the workspace keys and every other stored key"
|
||||
assert.equal(result.showImageInfo, false);
|
||||
assert.equal(result.videoMute, false);
|
||||
assert.equal(result.thumbnailPromptDismissed, "sig");
|
||||
assert.equal(result.viewerOpenMode, "fullscreen");
|
||||
// Page options no longer edit the default view (S13 G5), so a stored one stays.
|
||||
assert.equal(result.defaultViewMode, "list");
|
||||
// The page-only keys come from the dialog.
|
||||
|
||||
@@ -10,6 +10,7 @@ export function mountedDom() {
|
||||
// As in the DOM, a string argument becomes a text node.
|
||||
static wrap(node){if(typeof node!=='string')return node;const text=new Node('#text');text.textContent=node;return text;}
|
||||
append(...nodes){for(const node of nodes.map(Node.wrap)){node.parent=this;this.children.push(node);}}
|
||||
insertBefore(node,before){node.parent=this;const index=this.children.indexOf(before);this.children.splice(index<0?this.children.length:index,0,node);return node;}
|
||||
prepend(...nodes){nodes=nodes.map(Node.wrap);for(const node of nodes)node.parent=this;this.children.unshift(...nodes);}
|
||||
replaceChildren(...nodes){for(const child of this.children)child.parent=null;this.children=[];this._text='';this.append(...nodes);}
|
||||
remove(){if(this.parent)this.parent.children=this.parent.children.filter(c=>c!==this);this.parent=null;}
|
||||
|
||||
@@ -104,6 +104,31 @@ test('archived variant reads use their own scope and cursor without changing the
|
||||
const active=[];const plain=createLibraryDetail({api:{get:async url=>{active.push(url);return {positive:item(1),variants:[]};}},onChange:()=>{}});await plain.load(1);assert.equal(new URL(active[0],'https://x').searchParams.has('archive'),false);
|
||||
assert.throws(()=>createLibraryDetail({api:{},onChange:()=>{},archive:'parents'}));
|
||||
});
|
||||
test('tag mode any and a folder are sent, echoed and validated (#444)',async()=>{
|
||||
const {createLibraryStream,LIBRARY_TAG_MODES}=await source;assert.deepEqual(LIBRARY_TAG_MODES,['all','any','untagged']);
|
||||
const calls=[];const pending=[{...page([item(1)]),tag_mode:'any',folder:'2026/Aug'},{...page([item(2)]),tag_mode:'all',folder:''},page([item(3)]),{...page([item(4)]),tag_mode:'all'}];
|
||||
const s=createLibraryStream({api:{get:async(url)=>{calls.push(new URL(url,'https://x').searchParams);return pending.shift();}}});
|
||||
assert.equal(await s.reset({tags:['a','b'],tagMode:'any',folder:'2026/Aug'}),1);
|
||||
assert.equal(calls[0].get('tag_mode'),'any');assert.equal(calls[0].get('folder'),'2026/Aug');assert.deepEqual(calls[0].getAll('tag'),['a','b']);
|
||||
// An older server that ignored them answers without the echo: an error, never the unfiltered shelf.
|
||||
assert.equal(await s.reset({tags:['a','b'],tagMode:'any',folder:'2026/Aug'}),0);assert.ok(s.state.error instanceof TypeError);
|
||||
// The defaults send neither, so they need no echo.
|
||||
assert.equal(await s.reset({tagMode:'all',folder:''}),1);assert.equal(calls[2].has('tag_mode'),false);assert.equal(calls[2].has('folder'),false);
|
||||
assert.equal(await s.reset({folder:'x'}),0);assert.ok(s.state.error instanceof TypeError,'a folder without its echo is refused');
|
||||
await assert.rejects(s.reset({tagMode:'or'}),TypeError);await assert.rejects(s.reset({folder:3}),TypeError);
|
||||
assert.equal(calls.length,4);s.dispose();
|
||||
});
|
||||
test('tag mode untagged is sent, echoed and never combined with tags (#444)',async()=>{
|
||||
const {createLibraryStream}=await source;const calls=[];const pending=[{...page([item(1)]),tag_mode:'untagged'},page([item(2)]),page([item(3)])];
|
||||
const s=createLibraryStream({api:{get:async(url)=>{calls.push(url);return pending.shift();}}});
|
||||
assert.equal(await s.reset({tagMode:'untagged'}),1);assert.equal(new URL(calls[0],'https://x').searchParams.get('tag_mode'),'untagged');
|
||||
// An older server that ignored tag_mode answers without the echo: an error, never the whole Library.
|
||||
assert.equal(await s.reset({tagMode:'untagged'}),0);assert.ok(s.state.error instanceof TypeError);assert.deepEqual(s.state.items,[]);
|
||||
// The default sends no tag_mode, so it needs no echo.
|
||||
assert.equal(await s.reset({tagMode:'all'}),1);assert.equal(new URL(calls[2],'https://x').searchParams.has('tag_mode'),false);
|
||||
await assert.rejects(s.reset({tagMode:'untagged',tags:['a']}),TypeError);await assert.rejects(s.reset({tagMode:'or'}),TypeError);
|
||||
assert.equal(calls.length,3);s.dispose();
|
||||
});
|
||||
test('the stream keeps the first page\'s storage epoch and refuses a later page from another one (#437)',async()=>{
|
||||
const {createLibraryStream}=await source;const A='11111111-1111-4111-8111-111111111111',B='22222222-2222-4222-8222-222222222222';
|
||||
const pending=[{...page([item(1)],'next'),library_epoch:A},{...page([item(2)],'more'),library_epoch:A},{...page([item(3)]),library_epoch:B}];
|
||||
@@ -120,3 +145,28 @@ test('the shelf no longer sorts by rating (D153)',async()=>{
|
||||
assert.throws(()=>createLibraryStream({api:{get:async()=>page([])},restore:{filters:{sort:'rating'},state:{}}}),/Invalid library sort/);
|
||||
for(const sort of ['newest','oldest','name']) assert.equal(await s.reset({sort}),0);s.dispose();
|
||||
});
|
||||
test('Refine › Folder nests each folder under its exact parent, by path segment (#444 live B1)',async()=>{
|
||||
const {libraryFolderOptions}=await source;assert.equal(typeof libraryFolderOptions,'function');
|
||||
const I=' ';
|
||||
// The server's flat list sorts full paths as text, so "-" and "_" land before "/" (live report shapes).
|
||||
const folders=['CCC','CCC/mabes9sans-dataset','CCC/mabes9sans-dataset-v1','CCC/mabes9sans-dataset/mabes9sans-dataset_IMG_GEN',
|
||||
'FLUX','FLUX/2025-12-11','flux','flux/images','flux-klein','illustrious','illustrious-base','illustrious/2026-10-26','illustrious/training',
|
||||
'illustrious/training/a','kiko','kiko-v6','kiko/char','kiko/XXX','QWEN','QWEN/x','Qwen','Qwen/y','Qwen-Training','Video','Video/v','video'].sort();
|
||||
const options=libraryFolderOptions(folders);
|
||||
assert.deepEqual(options[0],['','All output folders']);
|
||||
assert.deepEqual(options.slice(1),[
|
||||
['CCC','CCC'],['CCC/mabes9sans-dataset',`${I}mabes9sans-dataset`],['CCC/mabes9sans-dataset/mabes9sans-dataset_IMG_GEN',`${I}${I}mabes9sans-dataset_IMG_GEN`],
|
||||
['CCC/mabes9sans-dataset-v1',`${I}mabes9sans-dataset-v1`],
|
||||
['FLUX','FLUX'],['FLUX/2025-12-11',`${I}2025-12-11`],['flux','flux'],['flux/images',`${I}images`],['flux-klein','flux-klein'],
|
||||
['illustrious','illustrious'],['illustrious/2026-10-26',`${I}2026-10-26`],['illustrious/training',`${I}training`],['illustrious/training/a',`${I}${I}a`],
|
||||
['illustrious-base','illustrious-base'],
|
||||
['kiko','kiko'],['kiko/char',`${I}char`],['kiko/XXX',`${I}XXX`],['kiko-v6','kiko-v6'],
|
||||
['QWEN','QWEN'],['QWEN/x',`${I}x`],['Qwen','Qwen'],['Qwen/y',`${I}y`],['Qwen-Training','Qwen-Training'],
|
||||
['Video','Video'],['Video/v',`${I}v`],['video','video'],
|
||||
]);
|
||||
// Every parent precedes its children directly; a chosen folder missing from the list is kept, with its parents.
|
||||
assert.deepEqual(libraryFolderOptions(['a'],'a/b/c').slice(1),[['a','a'],['a/b',`${I}b`],['a/b/c',`${I}${I}c`]]);
|
||||
assert.deepEqual(libraryFolderOptions([],''),[['','All output folders']]);
|
||||
// A POSIX folder whose name holds a backslash is one segment.
|
||||
assert.deepEqual(libraryFolderOptions(['Metadata\\Klein_4b Wildcard Run']).slice(1),[['Metadata\\Klein_4b Wildcard Run','Metadata\\Klein_4b Wildcard Run']]);
|
||||
});
|
||||
|
||||
@@ -188,6 +188,7 @@ test('an inline Viewer still loading its image is destroyed only after it has vi
|
||||
} finally { view.dispose(); parentScope.dispose(); f.restore(); Object.assign(globalThis, globals); }
|
||||
});
|
||||
|
||||
const left0 = dialog => dialog.children[1].children[0];
|
||||
test('viewer renders the concept shell and toolbar order', async () => {
|
||||
const {reportedLocationAdapter:adapter} = await import('../../web/js/workspace/history-media.mjs');
|
||||
const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c'),occurrenceMedia('2',0,'d')];
|
||||
@@ -199,7 +200,14 @@ test('viewer renders the concept shell and toolbar order', async () => {
|
||||
assert.equal(dialog.attributes['aria-label'],undefined,'the title names the dialog');
|
||||
const [header,layout] = dialog.children;
|
||||
assert.equal(header.className,'dialog-header');
|
||||
const [heading,close] = header.children, [title,subtitle] = heading.children;
|
||||
const [heading,mode,close] = header.children, [title,subtitle] = heading.children;
|
||||
// #450 D156: the view switch before ✕ names the view it switches to (Centered is the stored default).
|
||||
assert.deepEqual([mode.tagName,mode.id,mode.className,mode.attributes['aria-label'],mode.title],['button','viewer-mode','icon-btn','Full screen view','Full screen view']);
|
||||
mode.fire('click');
|
||||
assert.equal(mode.attributes['aria-label'],'Centered view');
|
||||
assert.ok(f.walk(left0(dialog),n => n.textContent === 'Hide details').length === 1,'full screen adds the details toggle');
|
||||
mode.fire('click');
|
||||
assert.equal(mode.attributes['aria-label'],'Full screen view');
|
||||
assert.deepEqual([heading.tagName,title.tagName,title.id,title.textContent],['div','h2','viewer-title','a.png']);
|
||||
assert.deepEqual([subtitle.tagName,subtitle.className,subtitle.textContent],['p','muted small','One viewer · image, prompts, generation data, workflow']);
|
||||
assert.deepEqual([close.tagName,close.id,close.textContent,close.attributes['aria-label']],['button','viewer-close','×','Close image viewer']);
|
||||
|
||||
@@ -5,7 +5,7 @@ const assert = require('node:assert/strict');
|
||||
const source = import('../../web/js/workspace/saved-views.mjs');
|
||||
|
||||
const IMAGE_DEFAULTS = { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false, collection: '' };
|
||||
const LIBRARY_DEFAULTS = { q: '', category: '', sort: 'newest', tags: [] };
|
||||
const LIBRARY_DEFAULTS = { q: '', category: '', sort: 'newest', tags: [], tag_mode: 'all', folder: '' };
|
||||
|
||||
test('a saved state always holds every page key and keeps only values its page accepts', async () => {
|
||||
const { normalizeViewState, SAVED_VIEW_PAGES } = await source;
|
||||
@@ -31,8 +31,8 @@ test('a saved state always holds every page key and keeps only values its page a
|
||||
assert.deepEqual(normalizeViewState('images', { ...full, q: 'x'.repeat(501), folder: 7, tags: Array(21).fill('t') }),
|
||||
{ ...full, q: '', folder: '', tags: [] });
|
||||
assert.deepEqual(normalizeViewState('images', { ...full, category: 'bad\u0007', tags: [' '] }), { ...full, category: '', tags: [] });
|
||||
// Library has no source, folder or media, and no Name Z–A.
|
||||
assert.deepEqual(normalizeViewState('library', { ...full, sort: 'name-desc' }), { q: 'castle', category: 'Portraits', sort: 'newest', tags: ['moody', 'fog'] });
|
||||
// Library has no source or media, and no Name Z–A; its folder (#444) is the Library's own.
|
||||
assert.deepEqual(normalizeViewState('library', { ...full, sort: 'name-desc' }), { q: 'castle', category: 'Portraits', sort: 'newest', tags: ['moody', 'fog'], tag_mode: 'all', folder: 'a/b' });
|
||||
assert.deepEqual(normalizeViewState('library', { sort: 'oldest' }), { ...LIBRARY_DEFAULTS, sort: 'oldest' });
|
||||
// The returned tags are a copy.
|
||||
const tags = ['one'];
|
||||
@@ -62,7 +62,7 @@ test('the listing keeps usable views in order and normalizes their state', async
|
||||
{ id: 1, page: 'images', name: 'Videos', state: { media: 'video', source: '9' } },
|
||||
] });
|
||||
assert.deepEqual(views, [
|
||||
{ id: 3, page: 'library', name: 'Neon', state: { q: 'neon', category: '', sort: 'name', tags: [] } },
|
||||
{ id: 3, page: 'library', name: 'Neon', state: { q: 'neon', category: '', sort: 'name', tags: [], tag_mode: 'all', folder: '' } },
|
||||
{ id: 1, page: 'images', name: 'Videos', state: { ...IMAGE_DEFAULTS, media: 'video', source: '9' } },
|
||||
]);
|
||||
assert.equal(parseSavedView(null), null);
|
||||
@@ -84,8 +84,8 @@ test('page filters and saved states convert both ways', async () => {
|
||||
assert.equal(imagesViewState({ ...filters, collection: undefined }).collection, '');
|
||||
assert.equal(imagesViewState({ ...filters, collection: 7 }).collection, '7');
|
||||
assert.equal(imagesViewState({ ...filters, favorites: undefined }).favorites, false);
|
||||
const shelf = { text: 'neon', category: 'Cities', sort: 'oldest', tags: ['night'] };
|
||||
assert.deepEqual(libraryViewState(shelf), { q: 'neon', category: 'Cities', sort: 'oldest', tags: ['night'] });
|
||||
const shelf = { text: 'neon', category: 'Cities', sort: 'oldest', tags: ['night'], tagMode: 'any', folder: '2026' };
|
||||
assert.deepEqual(libraryViewState(shelf), { q: 'neon', category: 'Cities', sort: 'oldest', tags: ['night'], tag_mode: 'any', folder: '2026' });
|
||||
assert.deepEqual(libraryFilters(libraryViewState(shelf)), shelf);
|
||||
});
|
||||
|
||||
@@ -216,6 +216,27 @@ test('a state over the API 4 KB cap is refused before anything is sent (#249 M5)
|
||||
assert.deepEqual(api.calls, []);
|
||||
});
|
||||
|
||||
test('a Library view keeps its tag match and folder; an old view reads the defaults (#444)', async () => {
|
||||
const { normalizeViewState, libraryFilters } = await source;
|
||||
assert.deepEqual(normalizeViewState('library', { tag_mode: 'any', folder: '2026/Aug' }), { ...LIBRARY_DEFAULTS, tag_mode: 'any', folder: '2026/Aug' });
|
||||
assert.deepEqual(normalizeViewState('library', { tag_mode: 'or', folder: 'x'.repeat(1025) }), LIBRARY_DEFAULTS);
|
||||
assert.deepEqual(libraryFilters({ q: 'a' }), { text: 'a', category: '', sort: 'newest', tags: [], tagMode: 'all', folder: '' });
|
||||
});
|
||||
|
||||
test('a Library view keeps untagged, and untagged never travels with tags (#444)', async () => {
|
||||
const { normalizeViewState, libraryFilters, libraryViewState } = await source;
|
||||
assert.equal(normalizeViewState('library', { tag_mode: 'untagged' }).tag_mode, 'untagged');
|
||||
assert.deepEqual(normalizeViewState('library', { tags: ['a'], tag_mode: 'untagged' }), { ...LIBRARY_DEFAULTS, tags: ['a'] });
|
||||
const shelf = { text: '', category: '', sort: 'newest', tags: [], tagMode: 'untagged', folder: '' };
|
||||
assert.equal(libraryViewState(shelf).tag_mode, 'untagged');
|
||||
assert.deepEqual(libraryFilters(libraryViewState(shelf)), shelf);
|
||||
});
|
||||
|
||||
test('a shelf without the #444 filters saves their defaults', async () => {
|
||||
const { libraryViewState } = await source;
|
||||
assert.deepEqual(libraryViewState({ text: '', category: '', sort: 'newest', tags: [] }), LIBRARY_DEFAULTS);
|
||||
});
|
||||
|
||||
test('a view saved under the retired Highest rated sort opens on Recently added (D153)', async () => {
|
||||
const { normalizeViewState, imagesFilters, libraryFilters } = await source;
|
||||
assert.equal(normalizeViewState('images', { sort: 'rating' }).sort, 'newest');
|
||||
|
||||
@@ -65,7 +65,7 @@ function fakeDOM() {
|
||||
|
||||
const BROWSER = { workspaceLayout: 'grid', workspaceDensity: 'comfortable', autoLoadMetadata: true, cacheMetadata: true,
|
||||
imageQuality: 'medium', videoAutoplay: false, videoMute: true, videoLoop: true, showFilePaths: true, showImageInfo: true,
|
||||
sidebarCollapsedByDefault: false };
|
||||
sidebarCollapsedByDefault: false, viewerOpenMode: 'centered' };
|
||||
|
||||
async function mounted({ host = { webui_display_mode: 'newtab', result_timeout: 5 }, browser = BROWSER, hostFails = false } = {}) {
|
||||
const { mountAppearance } = await appearanceModule;
|
||||
@@ -80,9 +80,9 @@ async function mounted({ host = { webui_display_mode: 'newtab', result_timeout:
|
||||
});
|
||||
await drafts.load('browser');
|
||||
const scope = createScope();
|
||||
const [surface] = mountAppearance(null, { scope, drafts });
|
||||
const byId = id => dom.find(surface, node => node.dataset?.setting === id);
|
||||
return { dom, drafts, scope, surface, byId, loadHost: async () => { hostValues = host; await drafts.load('host'); },
|
||||
const [surface, viewer] = mountAppearance(null, { scope, drafts });
|
||||
const byId = id => dom.find(surface, node => node.dataset?.setting === id) ?? dom.find(viewer, node => node.dataset?.setting === id);
|
||||
return { dom, drafts, scope, surface, viewer, byId, loadHost: async () => { hostValues = host; await drafts.load('host'); },
|
||||
fixHost: () => { hostFails = false; } };
|
||||
}
|
||||
|
||||
@@ -98,13 +98,26 @@ test('the surface is the concept section with the concept rows in order, then th
|
||||
assert.deepEqual(titles, ['Default layout', 'Thumbnail size', 'Continuous browsing', 'Load metadata on selection',
|
||||
'Cache metadata in this browser', 'Image loading quality', 'Autoplay videos', 'Mute videos by default', 'Loop videos',
|
||||
'Show file paths', 'Open from ComfyUI', 'Node search auto-hide (seconds)', 'Show image details on cards',
|
||||
'Collapse viewer details by default', 'Reset display defaults']);
|
||||
'Reset display defaults']);
|
||||
const continuous = rows[2];
|
||||
assert.deepEqual(dom.tags(continuous.children[0]), ['h3', 'p']);
|
||||
assert.equal(continuous.children[1].className, 'chip blue');
|
||||
assert.equal(continuous.children[1].textContent, 'Infinite scroll');
|
||||
assert.equal(rows.at(-1).className, 'operation');
|
||||
assert.equal(APPEARANCE_ROWS.filter(Array.isArray).length, 13);
|
||||
assert.equal(APPEARANCE_ROWS.filter(Array.isArray).length, 12);
|
||||
} finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); }
|
||||
});
|
||||
|
||||
// #450 (D156, amends D97): a "Viewer" surface after "Browsing & display" holds the viewer's two defaults.
|
||||
test('the Viewer surface holds Open viewer in and the moved collapse row', async () => {
|
||||
const { VIEWER_ROWS } = await appearanceModule;
|
||||
const { dom, viewer, scope } = await mounted();
|
||||
try {
|
||||
assert.equal(viewer.className, 'surface');
|
||||
assert.equal(viewer.children[0].textContent, 'Viewer');
|
||||
assert.equal(viewer.children[1].textContent, 'How the image viewer opens; the button beside ✕ switches the current view.');
|
||||
assert.deepEqual(viewer.children.slice(2).map(row => row.children[0].children[0].textContent), ['Open viewer in', 'Collapse viewer details by default']);
|
||||
assert.deepEqual(VIEWER_ROWS.map(row => row[0]), ['viewer-mode', 'viewer-details']);
|
||||
} finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); }
|
||||
});
|
||||
|
||||
@@ -127,9 +140,11 @@ test('added rows reach their consumers: each control edits its stored key with t
|
||||
byId(id).checked = false; byId(id).dispatch('change');
|
||||
}
|
||||
for (const id of ['video-auto', 'viewer-details']) { byId(id).checked = true; byId(id).dispatch('change'); }
|
||||
assert.deepEqual(byId('viewer-mode').children.map(option => [option.value, option.textContent]), [['centered', 'Centered'], ['fullscreen', 'Full screen']]);
|
||||
byId('viewer-mode').value = 'fullscreen'; byId('viewer-mode').dispatch('change');
|
||||
assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'masonry', workspaceDensity: 'large', imageQuality: 'low',
|
||||
autoLoadMetadata: false, cacheMetadata: false, videoMute: false, videoLoop: false, showFilePaths: false, showImageInfo: false,
|
||||
videoAutoplay: true, sidebarCollapsedByDefault: true });
|
||||
videoAutoplay: true, sidebarCollapsedByDefault: true, viewerOpenMode: 'fullscreen' });
|
||||
assert.deepEqual(drafts.changes('host'), { webui_display_mode: 'popup' });
|
||||
} finally { scope.dispose(); dom.restore(); }
|
||||
});
|
||||
@@ -154,7 +169,7 @@ test('the auto-hide timeout takes whole seconds from 0 to 300 only', async () =>
|
||||
|
||||
test('reset restores defaults without touching other keys', async () => {
|
||||
const { DISPLAY_KEYS } = await appearanceModule;
|
||||
const changed = { ...BROWSER, workspaceLayout: 'list', videoMute: false, showImageInfo: false, sidebarCollapsedByDefault: true };
|
||||
const changed = { ...BROWSER, workspaceLayout: 'list', videoMute: false, showImageInfo: false, sidebarCollapsedByDefault: true, viewerOpenMode: 'fullscreen' };
|
||||
const { dom, drafts, scope, surface, loadHost } = await mounted({ browser: changed });
|
||||
try {
|
||||
await loadHost();
|
||||
@@ -163,7 +178,7 @@ test('reset restores defaults without touching other keys', async () => {
|
||||
assert.equal(reset.textContent, 'Reset');
|
||||
reset.click();
|
||||
// #451: the default layout is Masonry.
|
||||
assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'masonry', videoMute: true, showImageInfo: true, sidebarCollapsedByDefault: false });
|
||||
assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'masonry', videoMute: true, showImageInfo: true, sidebarCollapsedByDefault: false, viewerOpenMode: 'centered' });
|
||||
// The host fields and every key outside the display defaults are left as they are.
|
||||
assert.deepEqual(drafts.changes('host'), { result_timeout: 9 });
|
||||
assert.deepEqual([...DISPLAY_KEYS].sort(), Object.keys(BROWSER).sort());
|
||||
|
||||
@@ -157,3 +157,20 @@ test('a valid configured source missing from the effective list is another spell
|
||||
assert.deepEqual(sourceRows({ sources, saved, current: saved }).rows.at(-1), { key: 'output/.', title: 'output',
|
||||
description: 'output/. · same folder as another source', scan: null, remove: 'output/.' });
|
||||
});
|
||||
|
||||
test('D158: "Save prompts from other workflows" drafts save_foreign_runs, disabled while monitoring is off (#247)', async () => {
|
||||
const { FOREIGN_RUNS, foreignRunsState } = await sourcesModule;
|
||||
assert.deepEqual(FOREIGN_RUNS, { id: 'foreign-runs', key: 'save_foreign_runs', label: 'Save prompts from other workflows',
|
||||
description: 'New outputs of workflows without a PromptManager node add their prompt to the library when it is certain.',
|
||||
off: 'Applies while Watch for new outputs is on.' });
|
||||
assert.deepEqual(foreignRunsState({ value: true, monitoring: true }), { disabled: false, checked: true, note: null });
|
||||
assert.deepEqual(foreignRunsState({ value: false, monitoring: true }), { disabled: false, checked: false, note: null });
|
||||
// Monitoring drafted or saved off: nothing is watched, so the row says why it is disabled and keeps its value.
|
||||
assert.deepEqual(foreignRunsState({ value: true, monitoring: false }), { disabled: true, checked: true, note: FOREIGN_RUNS.off });
|
||||
// Not read yet (or the host read failed): disabled without a value, the host rows' own note applies.
|
||||
assert.deepEqual(foreignRunsState({ value: undefined, monitoring: undefined }), { disabled: true, checked: null, note: null });
|
||||
assert.deepEqual(foreignRunsState({ value: undefined, monitoring: false }), { disabled: true, checked: null, note: null });
|
||||
// Saved off while the running monitor still watches until the restart (D128 `stopping`): it can still be set.
|
||||
assert.deepEqual(foreignRunsState({ value: true, monitoring: false, stopping: true }),
|
||||
{ disabled: false, checked: true, note: FOREIGN_RUNS.off });
|
||||
});
|
||||
|
||||
@@ -461,6 +461,57 @@ test('after dispose an unconfirmed write invalidates and a failed write does not
|
||||
}
|
||||
});
|
||||
|
||||
test('a tag stream sends its search and sort, replaces them on reset and needs the server to echo them (#444)', async () => {
|
||||
const {createVocabularyStream, TAG_SORTS} = await source;
|
||||
assert.deepEqual(TAG_SORTS.map(([value]) => value), ['created', 'name', 'name-desc', 'prompts', 'prompts-asc']);
|
||||
assert.deepEqual(TAG_SORTS.map(([, label]) => label), ['Oldest first', 'Name A–Z', 'Name Z–A', 'Most prompts', 'Fewest prompts']);
|
||||
const echo = (page, search, sort) => ({...page, search, sort, untagged_count: 4});
|
||||
const {calls, api} = queued([
|
||||
echo(tagPage([tag(1)], 'next'), 'cat', 'name'), echo(tagPage([tag(2)]), 'cat', 'name'),
|
||||
echo(tagPage([tag(3)]), '', 'created'), tagPage([tag(4)]), echo(tagPage([tag(5)]), 'dog', 'created'),
|
||||
]);
|
||||
const stream = createVocabularyStream({api, kind: 'tags', query: {search: ' cat ', sort: 'name'}});
|
||||
await stream.loadMore();
|
||||
assert.equal(params(calls[0].url).get('search'), 'cat');
|
||||
assert.equal(params(calls[0].url).get('sort'), 'name');
|
||||
assert.equal(stream.state.untaggedCount, 4);
|
||||
await stream.loadMore();
|
||||
assert.equal(params(calls[1].url).get('search'), 'cat');
|
||||
assert.equal(params(calls[1].url).get('cursor'), 'next');
|
||||
assert.deepEqual(stream.state.query, {search: 'cat', sort: 'name'});
|
||||
// The default query sends neither, so an older server answers it unchanged.
|
||||
await stream.reset({search: '', sort: 'created'});
|
||||
assert.equal(params(calls[2].url).has('search'), false);
|
||||
assert.equal(params(calls[2].url).has('sort'), false);
|
||||
assert.deepEqual(stream.state.items.map(row => row.id), [3]);
|
||||
// A server that ignored the search (no echo) is an error, never a wrong list.
|
||||
await stream.reset({search: 'dog', sort: 'created'});
|
||||
assert.deepEqual(stream.state.items, []);
|
||||
assert.ok(stream.state.error instanceof TypeError);
|
||||
assert.equal(stream.state.untaggedCount, 4, 'the last known untagged count survives a reset');
|
||||
await stream.loadMore();
|
||||
assert.deepEqual(stream.state.items.map(row => row.id), [5]);
|
||||
assert.throws(() => stream.reset({search: 'x', sort: 'count'}), TypeError);
|
||||
assert.throws(() => createVocabularyStream({api, kind: 'categories', query: {search: 'x'}}), TypeError);
|
||||
assert.throws(() => createVocabularyStream({api, kind: 'tags', query: {search: 3}}), TypeError);
|
||||
stream.dispose();
|
||||
});
|
||||
|
||||
test('an invalid untagged_count rejects the page', async () => {
|
||||
const {createVocabularyStream} = await source;
|
||||
for (const untagged_count of [-1, 1.5, '2']) {
|
||||
const stream = createVocabularyStream({api: queued([{...tagPage([tag(1)]), untagged_count}]).api, kind: 'tags'});
|
||||
await stream.loadMore();
|
||||
assert.ok(stream.state.error instanceof TypeError, String(untagged_count));
|
||||
assert.equal(stream.state.untaggedCount, null);
|
||||
}
|
||||
});
|
||||
|
||||
test('untaggedBrowseFilters opens the Library on prompts without tags (#444)', async () => {
|
||||
const {untaggedBrowseFilters} = await source;
|
||||
assert.deepEqual(untaggedBrowseFilters(), {text: '', category: '', tags: [], sort: 'newest', tagMode: 'untagged'});
|
||||
});
|
||||
|
||||
test('browseFilters hands Library one exact constraint', async () => {
|
||||
const {browseFilters} = await source;
|
||||
assert.deepEqual(browseFilters('tags', 'a'), {text: '', category: '', tags: ['a'], sort: 'newest'});
|
||||
|
||||
@@ -684,6 +684,132 @@ class TestStaticRoutes(CoreRoutesTestCase):
|
||||
self.assertEqual(resp.status, 403)
|
||||
|
||||
|
||||
class TestRevalidatedCaching(CoreRoutesTestCase):
|
||||
"""Every served page and static file is revalidated on reload (#478).
|
||||
|
||||
``Cache-Control: no-cache`` makes the browser ask before it reuses a copy,
|
||||
and the ETag lets an unchanged file answer 304 without a body, so an update
|
||||
reaches the browser on a normal reload and an unchanged reload stays cheap.
|
||||
"""
|
||||
|
||||
STATIC_PATHS = (
|
||||
"/prompt_manager/js/workspace/index.mjs",
|
||||
"/prompt_manager/js/admin.js",
|
||||
"/prompt_manager/js/inspection-semantics.json",
|
||||
"/prompt_manager/lib/viewerjs/viewer.min.css",
|
||||
)
|
||||
|
||||
async def _assert_revalidated(self, path):
|
||||
resp = await self.client.request("GET", path)
|
||||
self.assertEqual(resp.status, 200, path)
|
||||
self.assertEqual(resp.headers.get("Cache-Control"), "no-cache", path)
|
||||
etag = resp.headers.get("ETag")
|
||||
self.assertTrue(etag, path)
|
||||
body = await resp.read()
|
||||
self.assertTrue(body, path)
|
||||
|
||||
again = await self.client.request("GET", path, headers={"If-None-Match": etag})
|
||||
self.assertEqual(again.status, 304, path)
|
||||
self.assertEqual(again.headers.get("ETag"), etag, path)
|
||||
self.assertEqual(again.headers.get("Cache-Control"), "no-cache", path)
|
||||
self.assertEqual(await again.read(), b"", path)
|
||||
|
||||
stale = await self.client.request(
|
||||
"GET", path, headers={"If-None-Match": 'W/"stale"'}
|
||||
)
|
||||
self.assertEqual(stale.status, 200, path)
|
||||
self.assertEqual(await stale.read(), body, path)
|
||||
return etag
|
||||
|
||||
async def test_static_files_are_revalidated(self):
|
||||
for path in self.STATIC_PATHS:
|
||||
await self._assert_revalidated(path)
|
||||
|
||||
async def test_pages_are_revalidated(self):
|
||||
for path in UI_PATHS:
|
||||
await self._assert_revalidated(path)
|
||||
|
||||
async def test_etag_matches_in_a_list_and_as_wildcard(self):
|
||||
path = "/prompt_manager/js/workspace/index.mjs"
|
||||
etag = (await self.client.request("GET", path)).headers["ETag"]
|
||||
for header in (f'W/"other", {etag}', etag.removeprefix("W/"), "*"):
|
||||
resp = await self.client.request(
|
||||
"GET", path, headers={"If-None-Match": header}
|
||||
)
|
||||
self.assertEqual(resp.status, 304, header)
|
||||
|
||||
async def test_changed_file_gets_a_new_etag_and_its_new_body(self):
|
||||
web_dir = self._fake_root()
|
||||
module = web_dir / "js" / "m.mjs"
|
||||
module.write_text("export const v = 1;")
|
||||
path = "/prompt_manager/js/m.mjs"
|
||||
first = await self.client.request("GET", path)
|
||||
etag = first.headers["ETag"]
|
||||
# Same size and, on coarse clocks, the same mtime: only the bytes differ.
|
||||
module.write_text("export const v = 2;")
|
||||
resp = await self.client.request("GET", path, headers={"If-None-Match": etag})
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertNotEqual(resp.headers["ETag"], etag)
|
||||
self.assertEqual(await resp.text(), "export const v = 2;")
|
||||
|
||||
async def test_gzipped_and_plain_bodies_share_a_weak_etag(self):
|
||||
"""The gzip middleware changes the bytes, so the tag must be weak."""
|
||||
resp = await self.client.request("GET", "/prompt_manager/js/admin.js")
|
||||
self.assertTrue(resp.headers["ETag"].startswith('W/"'))
|
||||
|
||||
async def test_errors_carry_no_validator(self):
|
||||
resp = await self.client.request("GET", "/prompt_manager/js/nope.js")
|
||||
self.assertEqual(resp.status, 404)
|
||||
self.assertNotIn("ETag", resp.headers)
|
||||
|
||||
|
||||
@web.middleware
|
||||
async def _comfyui_cache_control(request, handler):
|
||||
"""ComfyUI's own middleware (middleware/cache_middleware.py), JS/CSS branch.
|
||||
|
||||
It only fills a missing Cache-Control, and ``.mjs`` is not ``.js``: that is
|
||||
why the workspace modules went out with no header at all before #478.
|
||||
"""
|
||||
response = await handler(request)
|
||||
if request.path.endswith(".js") or request.path.endswith(".css"):
|
||||
response.headers.setdefault("Cache-Control", "no-store")
|
||||
return response
|
||||
|
||||
|
||||
class TestRevalidatedCachingBehindMiddlewares(CoreRoutesTestCase):
|
||||
"""The host's middlewares keep the header and the 304 intact."""
|
||||
|
||||
async def get_application(self):
|
||||
app = await super().get_application()
|
||||
app.middlewares.append(_comfyui_cache_control)
|
||||
app.middlewares.append(api_module._gzip_middleware)
|
||||
return app
|
||||
|
||||
async def test_gzipped_files_keep_no_cache_and_answer_304(self):
|
||||
for path in (
|
||||
"/prompt_manager/js/workspace/index.mjs",
|
||||
"/prompt_manager/js/admin.js",
|
||||
"/prompt_manager/lib/viewerjs/viewer.min.css",
|
||||
"/prompt_manager/workspace",
|
||||
):
|
||||
resp = await self.client.request(
|
||||
"GET", path, headers={"Accept-Encoding": "gzip"}
|
||||
)
|
||||
self.assertEqual(resp.status, 200, path)
|
||||
self.assertEqual(resp.headers.get("Content-Encoding"), "gzip", path)
|
||||
self.assertEqual(resp.headers["Cache-Control"], "no-cache", path)
|
||||
again = await self.client.request(
|
||||
"GET",
|
||||
path,
|
||||
headers={
|
||||
"Accept-Encoding": "gzip",
|
||||
"If-None-Match": resp.headers["ETag"],
|
||||
},
|
||||
)
|
||||
self.assertEqual(again.status, 304, path)
|
||||
self.assertEqual(again.headers["Cache-Control"], "no-cache", path)
|
||||
|
||||
|
||||
# ── instance helpers ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -85,6 +85,7 @@ CONFIG = {
|
||||
"max_image_age_days",
|
||||
"max_concurrent_processing",
|
||||
"metadata_extraction_timeout",
|
||||
"save_foreign_runs",
|
||||
},
|
||||
"integrations": {"lora_manager_enabled", "has_civitai_api_key"},
|
||||
}
|
||||
|
||||
@@ -18,6 +18,8 @@ from unittest.mock import patch
|
||||
|
||||
from aiohttp import FormData, web
|
||||
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
try:
|
||||
from test_api_images import ImageAPITestCase
|
||||
except ImportError:
|
||||
@@ -191,7 +193,11 @@ class HistoryAPITests(ImageAPITestCase):
|
||||
context["evidence"]["positive"]["raw"]["provenance"], "recorded"
|
||||
)
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
conn.execute("UPDATE prompts SET text='edited' WHERE id=?", (positive,))
|
||||
# An edit changes the text and its dedup hash together (#460)
|
||||
conn.execute(
|
||||
"UPDATE prompts SET text='edited',hash=? WHERE id=?",
|
||||
(generate_prompt_hash("edited"), positive),
|
||||
)
|
||||
_, changed = await self.detail(start.run_id)
|
||||
self.assertEqual(changed["contexts"][0]["positive"], context["positive"])
|
||||
self.assertFalse(changed["contexts"][0]["navigation"]["positive_available"])
|
||||
|
||||
@@ -1015,6 +1015,122 @@ class HistoryOutputAPITests(ImageAPITestCase):
|
||||
response = await self.current(descriptor["inspection_url"])
|
||||
self.assert_error(response, await response.json(), 422, "response-limit")
|
||||
|
||||
async def tag_image(self, relative, tags):
|
||||
"""Save *tags* on Images root 0's *relative* file through the Tag routes.
|
||||
|
||||
The real observe and save routes store the locator the Tag dialog
|
||||
stores (``media_capture._resolve``), so History must find it (#457).
|
||||
"""
|
||||
self.addCleanup(self.api.close_annotation_captures)
|
||||
roots = self.api._get_all_output_dirs()
|
||||
target = {
|
||||
"root_index": 0,
|
||||
"root_scope": self.api._gallery_root_scope(roots[0], 0),
|
||||
"path": relative,
|
||||
}
|
||||
response = await self.client.post(
|
||||
"/prompt_manager/image-annotations/observe", json={"target": target}
|
||||
)
|
||||
observed = await response.json()
|
||||
self.assertEqual(response.status, 200, observed)
|
||||
token = observed["capture"]["token"]
|
||||
try:
|
||||
response = await self.client.put(
|
||||
"/prompt_manager/image-annotations/tags",
|
||||
json={
|
||||
"target": target,
|
||||
"capture_token": token,
|
||||
"sha256": observed["capture"]["sha256"],
|
||||
"expected_epoch": observed["library_epoch"],
|
||||
"expected_tag_revision": observed["tag_revision"],
|
||||
"tags": list(tags),
|
||||
},
|
||||
)
|
||||
saved = await response.json()
|
||||
self.assertEqual(response.status, 200, saved)
|
||||
finally:
|
||||
await self.client.delete(
|
||||
f"/prompt_manager/image-annotations/capture/{token}"
|
||||
)
|
||||
return saved["tags"]
|
||||
|
||||
async def test_current_inspection_carries_the_files_image_tags(self):
|
||||
"""#457: History reads the file's own image tags, as Images does (S1)."""
|
||||
sub = self.output_dir / "batch"
|
||||
sub.mkdir()
|
||||
tagged = png(sub / "tagged.png", prompt=graph("Tagged file"))
|
||||
png(self.output_dir / "plain.png", prompt=graph("Plain file"))
|
||||
self.assertEqual(
|
||||
await self.tag_image("batch/tagged.png", ["dunes", "warm_light"]),
|
||||
["dunes", "warm_light"],
|
||||
)
|
||||
start = self.capture(
|
||||
[
|
||||
self.ref("1", subfolder="batch", filename="tagged.png"),
|
||||
self.ref("2", filename="plain.png"),
|
||||
]
|
||||
)
|
||||
_, page = await self.outputs(start)
|
||||
urls = [
|
||||
item["current_candidate"]["media"]["inspection_url"]
|
||||
for item in page["items"]
|
||||
]
|
||||
bodies = []
|
||||
for url in urls:
|
||||
response = await self.current(url)
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
bodies.append(body)
|
||||
self.assertEqual(
|
||||
bodies[0].get("image_tags"),
|
||||
{"state": "current", "tags": ["dunes", "warm_light"]},
|
||||
)
|
||||
# Image tags are the file's own search metadata, never prompt tags.
|
||||
self.assertEqual(bodies[0]["prompts"]["positive"]["text"], "Tagged file")
|
||||
self.assertEqual(bodies[1].get("image_tags"), {"state": "none", "tags": []})
|
||||
# New bytes at the same path: the tags belong to the old byte record.
|
||||
stat = tagged.stat()
|
||||
os.utime(tagged, ns=(stat.st_atime_ns, stat.st_mtime_ns + 10**9))
|
||||
response = await self.current(urls[0])
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
self.assertEqual(body["image_tags"], {"state": "none", "tags": []})
|
||||
|
||||
async def test_current_inspection_finds_image_tags_under_a_symlinked_output(self):
|
||||
"""An output folder reached through a link (another drive) still matches."""
|
||||
link = self.comfy_dir / "linked_output"
|
||||
self._make_symlink(self.output_dir, link, target_is_directory=True)
|
||||
self.api._get_all_output_dirs = lambda: [link]
|
||||
self.host_output = str(link)
|
||||
png(self.output_dir / "image.png", prompt=graph("Linked folder"))
|
||||
self.assertEqual(await self.tag_image("image.png", ["dunes"]), ["dunes"])
|
||||
start = self.capture([self.ref()])
|
||||
descriptor = await self.media(start)
|
||||
response = await self.current(descriptor["inspection_url"])
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
self.assertEqual(body["image_tags"], {"state": "current", "tags": ["dunes"]})
|
||||
|
||||
async def test_current_inspection_image_tag_failure_is_unavailable(self):
|
||||
png(self.output_dir / "image.png", prompt=graph("File evidence"))
|
||||
await self.tag_image("image.png", ["dunes"])
|
||||
start = self.capture([self.ref()])
|
||||
descriptor = await self.media(start)
|
||||
with (
|
||||
self.assertLogs(self.api.logger, "WARNING") as logs,
|
||||
patch.object(
|
||||
type(self.api.db),
|
||||
"current_image_tags",
|
||||
side_effect=sqlite3.OperationalError("locked"),
|
||||
),
|
||||
):
|
||||
response = await self.current(descriptor["inspection_url"])
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
self.assertEqual(body["image_tags"], {"state": "unavailable", "tags": []})
|
||||
self.assertEqual(body["prompts"]["positive"]["text"], "File evidence")
|
||||
self.assertNotIn(str(self.output_dir), "\n".join(logs.output))
|
||||
|
||||
async def test_optional_candidate_failure_preserves_every_occurrence_without_urls(
|
||||
self,
|
||||
):
|
||||
|
||||
@@ -35,6 +35,7 @@ from aiohttp import web # noqa: E402
|
||||
from aiohttp.test_utils import AioHTTPTestCase # noqa: E402
|
||||
from PIL import Image # noqa: E402
|
||||
|
||||
from database.models import PromptModel # noqa: E402
|
||||
from database.operations import PromptDatabase # noqa: E402
|
||||
import py.api.images as images_module
|
||||
from py.api import gallery_listing # noqa: E402
|
||||
@@ -92,14 +93,41 @@ class ImageAPITestCase(AioHTTPTestCase):
|
||||
self._anchor_patch.stop()
|
||||
# Windows refuses to unlink a database that still has open handles.
|
||||
self.api.db.close_all()
|
||||
for path in (
|
||||
self._temp_db.name,
|
||||
self._temp_db.name + "-wal",
|
||||
self._temp_db.name + "-shm",
|
||||
):
|
||||
if os.path.exists(path):
|
||||
os.unlink(path)
|
||||
self._output_tmp.cleanup()
|
||||
try:
|
||||
leaked = self._close_leaked_connections()
|
||||
finally:
|
||||
for path in (
|
||||
self._temp_db.name,
|
||||
self._temp_db.name + "-wal",
|
||||
self._temp_db.name + "-shm",
|
||||
):
|
||||
if os.path.exists(path):
|
||||
os.unlink(path)
|
||||
self._output_tmp.cleanup()
|
||||
if leaked:
|
||||
raise AssertionError(
|
||||
f"{leaked} connection(s) to the test database were still open at "
|
||||
"teardown; Windows cannot unlink it (WinError 32). Close every "
|
||||
"extra PromptDatabase before tearDownAsync, not in a cleanup."
|
||||
)
|
||||
|
||||
def _close_leaked_connections(self):
|
||||
"""Close and count connections other models still hold on the temp DB.
|
||||
|
||||
Linux unlinks an open SQLite file, Windows does not; counting here makes
|
||||
a handle left open past teardown fail on every OS (#475).
|
||||
"""
|
||||
key = os.path.normcase(os.path.realpath(self._temp_db.name))
|
||||
with PromptModel._instances_lock:
|
||||
models = list(PromptModel._instances)
|
||||
leaked = 0
|
||||
for model in models:
|
||||
if model._schema_key() != key:
|
||||
continue
|
||||
with model._conn_lock:
|
||||
leaked += len(model._connections)
|
||||
model.close_all()
|
||||
return leaked
|
||||
|
||||
# -- helpers ---------------------------------------------------------
|
||||
|
||||
|
||||
@@ -60,6 +60,32 @@ class TestLibraryAPI(ImageAPITestCase):
|
||||
self.assertEqual(response.status, 400)
|
||||
self.assertNotIn(str(self.comfy_dir), json.dumps(result))
|
||||
|
||||
async def test_untagged_tag_mode_lists_parents_without_tags(self):
|
||||
self._save_prompt("plain one")
|
||||
self._save_prompt("plain two")
|
||||
self._save_prompt("tagged", tags=["t"])
|
||||
self.api.db.save_prompt_pair("pair tagged", "noise", tags=["p"])
|
||||
_, first = await self.get(limit=1, tag_mode="untagged")
|
||||
self.assertEqual(first["total"], 2)
|
||||
self.assertEqual(first["tag_mode"], "untagged")
|
||||
_, second = await self.get(
|
||||
limit=1, tag_mode="untagged", cursor=first["next_cursor"]
|
||||
)
|
||||
self.assertEqual(
|
||||
sorted([first["items"][0]["text"], second["items"][0]["text"]]),
|
||||
["plain one", "plain two"],
|
||||
)
|
||||
_, default = await self.get(tag_mode="all")
|
||||
self.assertEqual(default["total"], 4)
|
||||
for params in (
|
||||
{"tag_mode": "untagged", "tag": "t"},
|
||||
{"tag_mode": "sql"},
|
||||
{"cursor": first["next_cursor"]},
|
||||
):
|
||||
response, result = await self.get(limit=1, **params)
|
||||
self.assertEqual(response.status, 400, params)
|
||||
self.assertEqual(result["success"], False)
|
||||
|
||||
async def test_outside_missing_and_unsupported_media_do_not_leak_or_scan(self):
|
||||
positive = self._save_prompt("Cat")
|
||||
outside = make_png(self.comfy_dir / "outside.png")
|
||||
@@ -97,6 +123,72 @@ class TestLibraryAPI(ImageAPITestCase):
|
||||
},
|
||||
)
|
||||
|
||||
async def test_folder_filter_lists_and_matches_prompts_by_output_folder(self):
|
||||
deep = self._save_prompt("deep")
|
||||
top = self._save_prompt("top")
|
||||
other = self._save_prompt("other")
|
||||
self._save_prompt("no images")
|
||||
for prompt, parts in ((deep, ("a", "b")), (top, ("a",)), (other, ("ab",))):
|
||||
path = make_png(self.output_dir.joinpath(*parts, f"{prompt}.png"))
|
||||
self.api.db.link_image_to_prompt(prompt, str(path))
|
||||
response, options = await self.get("/options", include="folders")
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(options["folders"], ["a", "a/b", "ab"])
|
||||
_, plain = await self.get("/options")
|
||||
self.assertNotIn("folders", plain)
|
||||
_, first = await self.get(folder="a", limit=1)
|
||||
self.assertEqual((first["total"], first["folder"]), (2, "a"))
|
||||
# Outer slashes are dropped (a backslash separates only on Windows).
|
||||
outer = "\\a\\" if os.name == "nt" else "/a/"
|
||||
_, second = await self.get(folder=outer, limit=1, cursor=first["next_cursor"])
|
||||
self.assertEqual(
|
||||
sorted([first["items"][0]["text"], second["items"][0]["text"]]),
|
||||
["deep", "top"],
|
||||
)
|
||||
_, all_rows = await self.get()
|
||||
self.assertEqual((all_rows["total"], all_rows["folder"]), (4, ""))
|
||||
for params in (
|
||||
{"folder": "a/../ab"},
|
||||
*([{"folder": "C:/x"}] if os.name == "nt" else []),
|
||||
{"folder": "b", "cursor": first["next_cursor"]},
|
||||
{"cursor": first["next_cursor"]},
|
||||
):
|
||||
response, result = await self.get(limit=1, **params)
|
||||
self.assertEqual(response.status, 400, params)
|
||||
self.assertEqual(result["success"], False)
|
||||
response, _ = await self.get("/options", include="tags")
|
||||
self.assertEqual(response.status, 400)
|
||||
|
||||
@unittest.skipIf(os.name == "nt", "a backslash is a separator on Windows")
|
||||
async def test_a_posix_folder_name_with_a_backslash_filters_its_prompts(self):
|
||||
prompt = self._save_prompt("klein")
|
||||
name = "Metadata\\Klein_4b Wildcard Run"
|
||||
path = make_png(self.output_dir / name / "x.png")
|
||||
self.api.db.link_image_to_prompt(prompt, str(path))
|
||||
_, options = await self.get("/options", include="folders")
|
||||
self.assertIn(name, options["folders"])
|
||||
_, result = await self.get(folder=name)
|
||||
self.assertEqual((result["total"], result["folder"]), (1, name))
|
||||
|
||||
async def test_any_tag_mode_is_sent_through_and_echoed(self):
|
||||
self._save_prompt("a only", tags=["a"])
|
||||
self._save_prompt("b only", tags=["b"])
|
||||
self._save_prompt("neither")
|
||||
_, result = await self.get(tag=["a", "b"], tag_mode="any")
|
||||
self.assertEqual(result["tag_mode"], "any")
|
||||
self.assertEqual(
|
||||
sorted(i["text"] for i in result["items"]), ["a only", "b only"]
|
||||
)
|
||||
_, both = await self.get(tag=["a", "b"])
|
||||
self.assertEqual((both["total"], both["tag_mode"]), (0, "all"))
|
||||
_, first = await self.get(tag=["a", "b"], tag_mode="any", limit=1)
|
||||
response, _ = await self.get(
|
||||
tag=["a", "b"], limit=1, cursor=first["next_cursor"]
|
||||
)
|
||||
self.assertEqual(response.status, 400)
|
||||
response, _ = await self.get(tag_mode="or")
|
||||
self.assertEqual(response.status, 400)
|
||||
|
||||
async def test_variant_cursor_is_positive_scoped_and_required_limits_are_bounded(
|
||||
self,
|
||||
):
|
||||
|
||||
@@ -35,7 +35,17 @@ class TestVocabularyAPI(ImageAPITestCase):
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(
|
||||
set(first),
|
||||
{"success", "library_epoch", "tags", "total", "has_more", "next_cursor"},
|
||||
{
|
||||
"success",
|
||||
"library_epoch",
|
||||
"tags",
|
||||
"total",
|
||||
"untagged_count",
|
||||
"search",
|
||||
"sort",
|
||||
"has_more",
|
||||
"next_cursor",
|
||||
},
|
||||
)
|
||||
self.assertEqual(first["library_epoch"], epoch)
|
||||
self.assertEqual(first["total"], 3)
|
||||
@@ -111,6 +121,41 @@ class TestVocabularyAPI(ImageAPITestCase):
|
||||
response, _ = await self.get("tags", limit=1, cursor=fresh["next_cursor"])
|
||||
self.assertEqual(response.status, 200)
|
||||
|
||||
async def test_tag_search_and_sort_page_with_scoped_cursors(self):
|
||||
self._save_prompt("One", tags=["beta", "alpha"])
|
||||
self._save_prompt("Two", tags=["beta"])
|
||||
self._save_prompt("Plain")
|
||||
self.add_unused("gamma")
|
||||
_, first = await self.get("tags", limit=1, sort="name", search="A")
|
||||
self.assertEqual([row["name"] for row in first["tags"]], ["alpha"])
|
||||
self.assertEqual(first["total"], 3)
|
||||
self.assertEqual(first["untagged_count"], 1)
|
||||
self.assertEqual((first["search"], first["sort"]), ("A", "name"))
|
||||
_, second = await self.get(
|
||||
"tags", limit=1, sort="name", search="A", cursor=first["next_cursor"]
|
||||
)
|
||||
self.assertEqual([row["name"] for row in second["tags"]], ["beta"])
|
||||
_, ranked = await self.get("tags", sort="prompts")
|
||||
self.assertEqual(
|
||||
[(row["name"], row["prompt_count"]) for row in ranked["tags"]],
|
||||
[("beta", 2), ("alpha", 1), ("gamma", 0)],
|
||||
)
|
||||
# A cursor belongs to one search and one sort.
|
||||
for params in (
|
||||
{"sort": "name-desc", "search": "A"},
|
||||
{"sort": "name", "search": "B"},
|
||||
{"sort": "created", "search": "A"},
|
||||
):
|
||||
response, result = await self.get(
|
||||
"tags", limit=1, cursor=first["next_cursor"], **params
|
||||
)
|
||||
self.assertEqual(response.status, 400, params)
|
||||
self.assertIn("refresh", result["error"])
|
||||
for params in ({"sort": "count"}, {"search": "x" * 201}):
|
||||
response, result = await self.get("tags", **params)
|
||||
self.assertEqual(response.status, 400, params)
|
||||
self.assertEqual(result["success"], False)
|
||||
|
||||
async def test_read_failures_keep_the_read_envelope(self):
|
||||
with patch.object(
|
||||
self.api.db,
|
||||
|
||||
@@ -38,11 +38,25 @@ def _payload(token):
|
||||
|
||||
class TestRestartKeepsTheOpenPage(ImageAPITestCase):
|
||||
async def get_application(self):
|
||||
self._restarts = []
|
||||
app = await super().get_application()
|
||||
for index in range(3):
|
||||
make_png(self.output_dir / f"image_{index}.png")
|
||||
return app
|
||||
|
||||
async def tearDownAsync(self):
|
||||
# Before the base teardown unlinks the database: cleanups run after
|
||||
# tearDownAsync, and Windows cannot unlink a file still open (#475).
|
||||
errors = []
|
||||
while self._restarts:
|
||||
try:
|
||||
await self._restarts.pop()()
|
||||
except Exception as error: # every closer runs; the first error wins
|
||||
errors.append(error)
|
||||
await super().tearDownAsync()
|
||||
if errors:
|
||||
raise errors[0]
|
||||
|
||||
async def first_page(self, client=None, **params):
|
||||
response = await (client or self.client).get(
|
||||
LISTING, params={"mode": "workspace", "limit": "1", **params}
|
||||
@@ -67,7 +81,7 @@ class TestRestartKeepsTheOpenPage(ImageAPITestCase):
|
||||
api.close_auto_thumbnails()
|
||||
api.db.close_all()
|
||||
|
||||
self.addAsyncCleanup(close)
|
||||
self._restarts.append(close)
|
||||
return api, client
|
||||
|
||||
async def test_a_restart_keeps_every_held_scope_and_cursor_valid(self):
|
||||
@@ -152,8 +166,13 @@ class TestRestartKeepsTheOpenPage(ImageAPITestCase):
|
||||
other_db = self.comfy_dir / "elsewhere" / "prompts.db"
|
||||
other_db.parent.mkdir()
|
||||
api = PromptManagerAPI(db=PromptDatabase(str(other_db)))
|
||||
self.addCleanup(api.db.close_all)
|
||||
self.addCleanup(api.close_auto_thumbnails)
|
||||
|
||||
async def close():
|
||||
api.close_auto_thumbnails()
|
||||
api.db.close_all()
|
||||
|
||||
# Closed before teardown removes the folder that holds its database.
|
||||
self._restarts.append(close)
|
||||
self.assertNotEqual(
|
||||
api._gallery_root_scope(self.output_dir, 0), page["images"][0]["root_scope"]
|
||||
)
|
||||
|
||||
@@ -59,6 +59,28 @@ class TestGalleryConfig(unittest.TestCase):
|
||||
self.assertFalse(GalleryConfig.MONITORING_ENABLED)
|
||||
self.assertEqual(GalleryConfig.PROCESSING_DELAY, 5.0)
|
||||
|
||||
def test_save_foreign_runs_defaults_on_and_only_booleans_apply(self):
|
||||
"""#247: gallery.monitoring.save_foreign_runs, default true."""
|
||||
self.assertIs(
|
||||
GalleryConfig.get_config()["monitoring"]["save_foreign_runs"], True
|
||||
)
|
||||
GalleryConfig.update_config({"monitoring": {"save_foreign_runs": False}})
|
||||
self.assertIs(GalleryConfig.SAVE_FOREIGN_RUNS, False)
|
||||
for junk in ("true", 1, None):
|
||||
with self.subTest(value=junk):
|
||||
with self.assertLogs("prompt_manager.config", "WARNING") as logs:
|
||||
GalleryConfig.update_config(
|
||||
{"monitoring": {"save_foreign_runs": junk}}
|
||||
)
|
||||
self.assertIn(
|
||||
"Ignoring gallery monitoring save_foreign_runs: expected true "
|
||||
"or false",
|
||||
"\n".join(logs.output),
|
||||
)
|
||||
self.assertIs(GalleryConfig.SAVE_FOREIGN_RUNS, False)
|
||||
GalleryConfig.update_config({"monitoring": {"save_foreign_runs": True}})
|
||||
self.assertIs(GalleryConfig.SAVE_FOREIGN_RUNS, True)
|
||||
|
||||
def test_update_tracking(self):
|
||||
GalleryConfig.update_config({"tracking": {"prompt_timeout": 300}})
|
||||
self.assertEqual(GalleryConfig.PROMPT_TIMEOUT, 300)
|
||||
|
||||
@@ -15,6 +15,7 @@ from database import execution_contexts
|
||||
from database.execution_history import HistoryEpochChanged
|
||||
from database.execution_context_schema import context_schema_error
|
||||
from database.execution_contexts import LIVE_HINT_PROMPT_IDS_SQL
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from database.execution_history_schema import rotate_history_epoch
|
||||
from utils.execution_contexts import (
|
||||
ContextBundle,
|
||||
@@ -466,7 +467,11 @@ class ContextDatabaseTests(unittest.TestCase):
|
||||
with conn:
|
||||
conn.execute("DELETE FROM prompts WHERE id=3")
|
||||
self.assertEqual(assert_same(), {1, 2, 4})
|
||||
# A case-only edit keeps the prompt's hash, so the run still counts
|
||||
# toward it (#460); another text drops the hint.
|
||||
db.update_prompt_text(4, "Other")
|
||||
self.assertEqual(assert_same(), {1, 2, 4})
|
||||
db.update_prompt_text(4, "another")
|
||||
self.assertEqual(assert_same(), {1, 2})
|
||||
with conn:
|
||||
conn.execute("INSERT INTO prompts(id,text) VALUES(5,'none')")
|
||||
@@ -522,6 +527,83 @@ class ContextDatabaseTests(unittest.TestCase):
|
||||
live = {row[0] for row in conn.execute(LIVE_HINT_PROMPT_IDS_SQL)}
|
||||
self.assertEqual(live, {1, 2})
|
||||
|
||||
def test_hints_follow_the_hash_for_case_variant_raw_text(self):
|
||||
# #460: the run names the prompt the Library counted it toward.
|
||||
db = self.database()
|
||||
first = db.save_prompt(
|
||||
"\u00e9t\u00e9 positive",
|
||||
prompt_hash=generate_prompt_hash("\u00e9t\u00e9 positive"),
|
||||
)
|
||||
second = db.save_prompt(
|
||||
"negative \u03c3", prompt_hash=generate_prompt_hash("negative \u03c3")
|
||||
)
|
||||
with db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"INSERT INTO prompt_pairs(id,positive_prompt_id,negative_prompt_id) "
|
||||
"VALUES(7,?,?)",
|
||||
(first, second),
|
||||
)
|
||||
start = self.capture(
|
||||
db, 0, bundle("NEGATIVE \u03a3", positive="\u00c9T\u00c9 Positive")
|
||||
)
|
||||
context = db.get_execution_contexts(start.run_id)["contexts"][0]
|
||||
self.assertEqual(
|
||||
(
|
||||
context["positive_prompt_id"],
|
||||
context["negative_prompt_id"],
|
||||
context["pair_id"],
|
||||
),
|
||||
(first, second, 7),
|
||||
)
|
||||
conn = db.model.get_connection()
|
||||
live = {row[0] for row in conn.execute(LIVE_HINT_PROMPT_IDS_SQL)}
|
||||
self.assertEqual(live, {first, second})
|
||||
# A prompt that only shares SQLite's ASCII lower() is not a match
|
||||
other = db.save_prompt("other", prompt_hash=generate_prompt_hash("other"))
|
||||
start = self.capture(db, 1, bundle("x", positive="\u00d6THER"))
|
||||
context = db.get_execution_contexts(start.run_id)["contexts"][0]
|
||||
self.assertIsNone(context["positive_prompt_id"])
|
||||
self.assertNotIn(
|
||||
other, {row[0] for row in conn.execute(LIVE_HINT_PROMPT_IDS_SQL)}
|
||||
)
|
||||
|
||||
def test_hash_is_only_a_fallback_so_a_text_hint_stays_unambiguous(self):
|
||||
# #460 review: a hash match must never make a #438 text hint ambiguous,
|
||||
# or its prompt would lose the recorded-run prune protection.
|
||||
db = self.database()
|
||||
conn = db.model.get_connection()
|
||||
with conn:
|
||||
# A legacy row without a hash, and a case variant with the hash
|
||||
conn.execute("INSERT INTO prompts(id,text) VALUES(1,'a lighthouse')")
|
||||
conn.execute(
|
||||
"INSERT INTO prompts(id,text,hash) VALUES(2,'A Lighthouse',?)",
|
||||
(generate_prompt_hash("a lighthouse"),),
|
||||
)
|
||||
exact = self.capture(db, 0, bundle("x", positive="a lighthouse"))
|
||||
variant = self.capture(db, 1, bundle("x", positive="A LIGHTHOUSE"))
|
||||
hints = [
|
||||
db.get_execution_contexts(start.run_id)["contexts"][0]["positive_prompt_id"]
|
||||
for start in (exact, variant)
|
||||
]
|
||||
self.assertEqual(hints, [1, 2])
|
||||
live = {row[0] for row in conn.execute(LIVE_HINT_PROMPT_IDS_SQL)}
|
||||
self.assertEqual(live, {1, 2})
|
||||
self.assertEqual(db.prune_orphaned_prompts(), 0)
|
||||
|
||||
def test_live_hint_prompt_lookups_use_indexes(self):
|
||||
db = self.database()
|
||||
conn = db.model.get_connection()
|
||||
plan = [
|
||||
row[3]
|
||||
for row in conn.execute("EXPLAIN QUERY PLAN " + LIVE_HINT_PROMPT_IDS_SQL)
|
||||
]
|
||||
prompt_steps = [
|
||||
step for step in plan if " p " in f" {step} " or "TABLE p" in step
|
||||
]
|
||||
self.assertTrue(prompt_steps, plan)
|
||||
self.assertEqual([step for step in plan if step.startswith("SCAN p")], [])
|
||||
self.assertEqual([step for step in plan if step.startswith("SCAN q")], [])
|
||||
|
||||
def test_hints_require_exact_unique_raw_text_and_never_refresh(self):
|
||||
db = self.database()
|
||||
conn = db.model.get_connection()
|
||||
|
||||
@@ -19,6 +19,7 @@ from unittest.mock import patch
|
||||
from tests.history_waits import drain_writer
|
||||
from database import PromptDatabase
|
||||
from utils import usage_tracking
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.execution_history_writer import HistoryWriter
|
||||
|
||||
try:
|
||||
@@ -918,6 +919,30 @@ class CachedRuntimeHistoryTests(HooksCase):
|
||||
self.assertIsNone(named_prompt(self.db, self.rows()[1]["run_id"]))
|
||||
|
||||
|
||||
class CachedRuntimeCaseHistoryTests(CachedRuntimeHistoryTests):
|
||||
"""Raw text that differs only in case joins the saved prompt's history (#460).
|
||||
|
||||
The hash folds case, so the run counts toward the lower-case prompt saved
|
||||
first; the run keeps its exact raw spelling.
|
||||
"""
|
||||
|
||||
TEXT = "\u00c9 LIGHTHOUSE the LLM wrote"
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.db.delete_prompt(self.prompt_id)
|
||||
lower = self.TEXT.lower()
|
||||
self.prompt_id = self.db.save_prompt(
|
||||
text=lower, prompt_hash=generate_prompt_hash(lower)
|
||||
)
|
||||
|
||||
def test_the_library_counts_the_case_variant_as_this_prompt(self):
|
||||
self.assertEqual(
|
||||
self.db.get_prompt_by_hash(generate_prompt_hash(self.TEXT))["id"],
|
||||
self.prompt_id,
|
||||
)
|
||||
|
||||
|
||||
class CachedRuntimeWhitespaceHistoryTests(CachedRuntimeHistoryTests):
|
||||
"""Raw text with outer whitespace joins the history of the prompt it saves (#438).
|
||||
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
"""#247 PR 2 x #90: the monitor's foreign path never stores switched-off text.
|
||||
|
||||
A file whose run's prompt has Save to library off is the opted-out path's,
|
||||
not the foreign path's; a foreign graph whose ladder decision is
|
||||
``not_saved`` (a switched-off node the #90 walk does not reach) stores
|
||||
nothing at settle, at the re-check after the seal, or from recorded history,
|
||||
and leaves no wait behind.
|
||||
"""
|
||||
|
||||
import os
|
||||
import unittest
|
||||
|
||||
from tests import saver_anchor_graphs as shapes
|
||||
from tests import test_run_attribution as attribution
|
||||
from tests import test_save_to_library as save_tests
|
||||
from tests.test_pair_scan import sampled_graph
|
||||
from utils.foreign_runs import ForeignRecheck
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from py.config import GalleryConfig # after tests: it stubs the host
|
||||
|
||||
LOGGER = "prompt_manager.image_monitor"
|
||||
OFF_TEXT = "SECRET OFF TEXT"
|
||||
SKIPPED = "Save to library is off for this run's prompt, skipping image"
|
||||
|
||||
# The class dict, not the class: a TestCase bound here would run again
|
||||
Fixture = vars(attribution.TestMonitorForeignRuns)
|
||||
|
||||
|
||||
def off_behind(middle="MyNode", key="value"):
|
||||
"""A switched-off PromptManagerText reaching a core CLIPTextEncode through
|
||||
``middle``'s ``key`` input: the #90 walk does not follow ``key``."""
|
||||
graph = save_tests.run_graph()
|
||||
graph["1"] = {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": OFF_TEXT, "save_to_library": False},
|
||||
}
|
||||
graph["7"] = {"class_type": middle, "inputs": {key: ["1", 0]}}
|
||||
graph["8"] = {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": ["7", 0], "clip": ["4", 1]},
|
||||
}
|
||||
graph["3"]["inputs"]["positive"] = ["8", 0]
|
||||
return graph
|
||||
|
||||
|
||||
def off_text_unwired():
|
||||
"""Only history can decide; the legacy answer is a dangling switched-off
|
||||
node's own text (the ladder says ``not_saved`` once it may ask history)."""
|
||||
graph = sampled_graph("unused", "blurry")
|
||||
graph["1"]["inputs"]["text"] = ["30", 0]
|
||||
graph["30"] = {"class_type": "VendorPromptMaker", "inputs": {"prompt": OFF_TEXT}}
|
||||
graph["40"] = {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": OFF_TEXT, "save_to_library": False},
|
||||
}
|
||||
return graph
|
||||
|
||||
|
||||
class TestForeignPathKeepsSaveToLibrary(attribution.RecordedRuns, unittest.TestCase):
|
||||
setUp = Fixture["setUp"]
|
||||
close_databases = Fixture["close_databases"]
|
||||
wait_for = Fixture["wait_for"]
|
||||
image = Fixture["image"]
|
||||
linked_text = Fixture["linked_text"]
|
||||
key = Fixture["key"]
|
||||
prompt_count = Fixture["prompt_count"]
|
||||
settled = Fixture["settled"]
|
||||
|
||||
def reference(self, path):
|
||||
return {
|
||||
"folder_type": "output",
|
||||
"subfolder": "dataset",
|
||||
"filename": os.path.basename(path),
|
||||
}
|
||||
|
||||
def seal(self, path, positives=(), nodes=("9",)):
|
||||
"""Record one run reporting ``path`` (a new run id for every call)."""
|
||||
self.runs = getattr(self, "runs", -1) + 1
|
||||
self.record_run(
|
||||
self.db,
|
||||
self.output,
|
||||
[("dataset", os.path.basename(path))],
|
||||
list(positives),
|
||||
item=self.runs,
|
||||
nodes=list(nodes),
|
||||
)
|
||||
|
||||
def recheck(self, path):
|
||||
"""The re-check the seal schedules, run through the monitor's entry."""
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(path, ForeignRecheck(self.reference(path)))
|
||||
return "\n".join(logs.output)
|
||||
|
||||
def assert_nothing_stored(self, path):
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
self.assertEqual(self.prompt_count(), 0)
|
||||
self.assertIsNone(self.db.get_prompt_by_hash(generate_prompt_hash(OFF_TEXT)))
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
self.assertEqual(list(self.handler._foreign_due), [])
|
||||
|
||||
def test_q1_the_only_prompt_manager_node_off_is_opted_out_not_foreign(self):
|
||||
graph = save_tests.run_graph(save=False)
|
||||
# The switched-off positive is the only PromptManager node
|
||||
graph["2"] = {"class_type": "CLIPTextEncode", "inputs": {"text": "blurry"}}
|
||||
sampler_less = {
|
||||
"1": {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": OFF_TEXT, "save_to_library": False},
|
||||
}
|
||||
}
|
||||
for name, graph in (("sampled", graph), ("sampler-less", sampler_less)):
|
||||
with self.subTest(name=name):
|
||||
self.metadata = {"prompt": graph}
|
||||
state, _ = self.handler._find_prompt_from_metadata(self.metadata)
|
||||
self.assertEqual(state, "opted_out")
|
||||
path = self.image(f"IMG_GEN_{name}_00001_.png")
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIn(SKIPPED, "\n".join(logs.output))
|
||||
self.seal(path)
|
||||
self.assertTrue(self.settled())
|
||||
self.assertIn(SKIPPED, self.recheck(path))
|
||||
self.assert_nothing_stored(path)
|
||||
|
||||
def test_q2_an_off_text_node_behind_a_pass_through_stores_nothing(self):
|
||||
for middle, key in (("MyNode", "value"), ("easy showAnything", "anything")):
|
||||
with self.subTest(middle=middle):
|
||||
self.metadata = {"prompt": off_behind(middle, key)}
|
||||
state, _ = self.handler._find_prompt_from_metadata(self.metadata)
|
||||
self.assertEqual(state, "foreign")
|
||||
path = self.image(f"IMG_GEN_{key}_00001_.png")
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIn(SKIPPED, "\n".join(logs.output))
|
||||
self.assert_nothing_stored(path)
|
||||
self.seal(path)
|
||||
self.assertTrue(self.settled())
|
||||
self.assertIn(SKIPPED, self.recheck(path))
|
||||
self.assert_nothing_stored(path)
|
||||
|
||||
def test_q2_a_wired_switch_behind_a_pass_through_stores_nothing(self):
|
||||
graph = off_behind()
|
||||
graph["1"]["inputs"]["save_to_library"] = ["20", 0]
|
||||
graph["20"] = {"class_type": "SomeBoolNode", "inputs": {}}
|
||||
self.metadata = {"prompt": graph}
|
||||
path = self.image("IMG_GEN_wired_00001_.png")
|
||||
wired = "Save to library is wired from a node the image cannot show"
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIn(wired, "\n".join(logs.output))
|
||||
self.seal(path)
|
||||
self.assertTrue(self.settled())
|
||||
self.assertIn(wired, self.recheck(path))
|
||||
self.assert_nothing_stored(path)
|
||||
|
||||
def test_q3_an_undecided_not_saved_waits_for_the_record_then_skips(self):
|
||||
"""No record yet: a not_saved of the undecided step is not final (a
|
||||
recorded saver may resolve another branch), so the file waits for its
|
||||
run; the decision with the record stores nothing and logs the skip."""
|
||||
wired = off_text_unwired()
|
||||
wired["40"]["inputs"]["save_to_library"] = ["20", 0]
|
||||
wired["20"] = {"class_type": "SomeBoolNode", "inputs": {}}
|
||||
cases = (
|
||||
("off", off_text_unwired(), SKIPPED),
|
||||
("wired", wired, "Save to library is wired from a node"),
|
||||
)
|
||||
for name, graph, line in cases:
|
||||
with self.subTest(name=name):
|
||||
self.metadata = {"prompt": graph}
|
||||
path = self.image(f"vendor_{name}_00001_.png")
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [self.key(path)])
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.seal(path)
|
||||
self.assertTrue(
|
||||
self.wait_for(lambda: not self.handler.awaiting_foreign())
|
||||
)
|
||||
self.assertTrue(self.settled())
|
||||
self.assertIn(line, "\n".join(logs.output))
|
||||
self.assert_nothing_stored(path)
|
||||
|
||||
def test_q3_an_early_not_saved_never_waits_even_with_a_saver_branch(self):
|
||||
"""Review: a switched-off node feeding one branch's positive is the
|
||||
ladder's early, final not_saved (PR 1's #90 rule covers every
|
||||
sampler's positive), so the file never waits and nothing is stored,
|
||||
not even the plain branch the recorded saver names."""
|
||||
graph = shapes.shared_prefix()
|
||||
graph["20"]["inputs"]["text"] = ["50", 0]
|
||||
graph["50"] = {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": OFF_TEXT, "save_to_library": False},
|
||||
}
|
||||
self.metadata = {"prompt": graph}
|
||||
path = self.image("IMG_GEN_00001_.png")
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIn(SKIPPED, "\n".join(logs.output))
|
||||
self.assert_nothing_stored(path)
|
||||
self.seal(path, nodes=("13",))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertIn(SKIPPED, self.recheck(path))
|
||||
self.assert_nothing_stored(path)
|
||||
self.assertIsNone(
|
||||
self.db.get_prompt_by_hash(generate_prompt_hash(shapes.BRANCH_TEXTS[0]))
|
||||
)
|
||||
|
||||
def test_q3_an_off_node_in_the_recorded_saver_branch_is_never_foreign(self):
|
||||
"""PR 3's parity case: a switched-off node feeds the very branch the
|
||||
recorded saver names (saver "23" is branch 2). A literal switch is the
|
||||
opted-out path's; a wired one is the node's own run's (#90), never the
|
||||
foreign path's. Nothing is stored at settle or after the run, and no
|
||||
file waits for a foreign re-check."""
|
||||
cases = (
|
||||
("off", False, "opted_out", SKIPPED),
|
||||
("wired", ["52", 0], "unknown", "No certain prompt for"),
|
||||
)
|
||||
for name, save, state, line in cases:
|
||||
with self.subTest(name=name):
|
||||
graph = shapes.shared_prefix()
|
||||
graph["50"] = {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": OFF_TEXT, "save_to_library": save},
|
||||
}
|
||||
graph["20"]["inputs"]["text"] = ["50", 0]
|
||||
if isinstance(save, list):
|
||||
graph["52"] = {"class_type": "SomeBoolNode", "inputs": {}}
|
||||
self.metadata = {"prompt": graph}
|
||||
found, _ = self.handler._find_prompt_from_metadata(self.metadata)
|
||||
self.assertEqual(found, state)
|
||||
path = self.image(f"IMG_GEN_{name}_00001_.png")
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIn(line, "\n".join(logs.output))
|
||||
self.assert_nothing_stored(path)
|
||||
self.seal(path, positives=[OFF_TEXT], nodes=("23",))
|
||||
self.assertTrue(self.settled())
|
||||
self.handler.process_new_image(path)
|
||||
self.assert_nothing_stored(path)
|
||||
self.assertIsNone(
|
||||
self.db.get_prompt_by_hash(
|
||||
generate_prompt_hash(shapes.BRANCH_TEXTS[1])
|
||||
)
|
||||
)
|
||||
|
||||
def test_q4_an_off_node_on_the_negative_only_still_links_the_positive(self):
|
||||
positive = "a lighthouse on a cliff at dusk"
|
||||
plain = sampled_graph(positive, "unused")
|
||||
plain["2"] = {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {"text": OFF_TEXT, "save_to_library": False},
|
||||
}
|
||||
behind = sampled_graph(positive, "unused")
|
||||
behind["2"]["inputs"]["text"] = ["7", 0]
|
||||
behind["7"] = {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": OFF_TEXT, "save_to_library": False},
|
||||
}
|
||||
for name, graph in (("direct", plain), ("behind an encoder", behind)):
|
||||
with self.subTest(name=name):
|
||||
self.metadata = {"prompt": graph}
|
||||
state, _ = self.handler._find_prompt_from_metadata(self.metadata)
|
||||
self.assertEqual(state, "foreign")
|
||||
path = self.image(f"plain_{len(name)}_00001_.png")
|
||||
self.handler.process_new_image(path)
|
||||
self.assertEqual(self.linked_text(path), positive)
|
||||
self.assertIsNone(
|
||||
self.db.get_prompt_by_hash(generate_prompt_hash(OFF_TEXT))
|
||||
)
|
||||
pairs = self.db.list_prompt_pairs()["pairs"]
|
||||
self.assertEqual(pairs, [])
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
|
||||
def test_q5_recorded_history_never_brings_back_switched_off_text(self):
|
||||
"""Even a recorded run whose contexts hold the off text (an older
|
||||
history row) cannot name it, at settle or at the re-check."""
|
||||
cases = {"pass-through": off_behind(), "dangling": off_text_unwired()}
|
||||
for name, graph in cases.items():
|
||||
with self.subTest(name=name):
|
||||
self.metadata = {"prompt": graph}
|
||||
path = self.image(f"IMG_GEN_{len(name)}_00001_.png")
|
||||
self.seal(path, positives=[OFF_TEXT], nodes=("9",))
|
||||
self.handler.process_new_image(path)
|
||||
self.assert_nothing_stored(path)
|
||||
self.assertIn(SKIPPED, self.recheck(path))
|
||||
self.assert_nothing_stored(path)
|
||||
|
||||
def test_q6_with_foreign_saving_off_save_to_library_is_unchanged(self):
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS = False
|
||||
opted_out = save_tests.run_graph(save=False)
|
||||
self.metadata = {"prompt": opted_out}
|
||||
skipped = self.image("IMG_GEN_off_00001_.png")
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(skipped)
|
||||
self.assertIn(SKIPPED, "\n".join(logs.output))
|
||||
self.metadata = {"prompt": off_behind()}
|
||||
foreign = self.image("IMG_GEN_foreign_00001_.png")
|
||||
with self.assertLogs(LOGGER, "INFO") as logs:
|
||||
self.handler.process_new_image(foreign)
|
||||
self.assertIn("No PromptManager prompt in workflow", "\n".join(logs.output))
|
||||
for path in (skipped, foreign):
|
||||
self.assert_nothing_stored(path)
|
||||
# A switched-on PromptManager positive still links by its metadata
|
||||
stored = self.db.save_prompt(
|
||||
text=save_tests.POSITIVE,
|
||||
prompt_hash=generate_prompt_hash(save_tests.POSITIVE),
|
||||
)
|
||||
self.metadata = {"prompt": save_tests.run_graph(save=True)}
|
||||
on = self.image("IMG_GEN_on_00001_.png")
|
||||
self.handler.process_new_image(on)
|
||||
self.assertEqual(self.linked_text(on), save_tests.POSITIVE)
|
||||
self.assertEqual(
|
||||
self.db.get_prompt_by_hash(generate_prompt_hash(save_tests.POSITIVE))["id"],
|
||||
stored,
|
||||
)
|
||||
|
||||
def test_the_shared_prefix_dataset_still_links_after_the_seal(self):
|
||||
"""Regression guard (passes before the fix): the wait still works."""
|
||||
self.metadata = {"prompt": shapes.shared_prefix()}
|
||||
path = self.image("IMG_GEN_00001_.png")
|
||||
self.handler.process_new_image(path)
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [self.key(path)])
|
||||
self.seal(path, nodes=("13",))
|
||||
self.assertTrue(self.wait_for(lambda: self.linked_text(path)))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertEqual(self.linked_text(path), shapes.BRANCH_TEXTS[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -17,6 +17,8 @@ from utils.hashing import (
|
||||
generate_prompt_hash,
|
||||
is_duplicate_prompt,
|
||||
STRIPPED_CODE_POINTS,
|
||||
prompt_key,
|
||||
register_prompt_functions,
|
||||
stored_prompt_text,
|
||||
stored_prompt_text_sql,
|
||||
)
|
||||
@@ -158,5 +160,56 @@ class TestStoredPromptText(unittest.TestCase):
|
||||
self.assertEqual(self.sql_strip(value), stored_prompt_text(value))
|
||||
|
||||
|
||||
class TestPromptKey(unittest.TestCase):
|
||||
"""History compares the hash's own key; SQL calls Python for it (#460)."""
|
||||
|
||||
CASES = (
|
||||
"A Lighthouse",
|
||||
" \u00c9t\u00c9\n",
|
||||
"\u03a3\u03a3",
|
||||
"\u0130stanbul",
|
||||
"\u1e9e",
|
||||
"\u212a",
|
||||
"ok",
|
||||
)
|
||||
|
||||
def test_key_is_the_hash_normalization(self):
|
||||
import hashlib
|
||||
|
||||
for value in self.CASES:
|
||||
with self.subTest(value=repr(value)):
|
||||
self.assertEqual(
|
||||
generate_prompt_hash(value),
|
||||
hashlib.sha256(prompt_key(value).encode("utf-8")).hexdigest(),
|
||||
)
|
||||
self.assertEqual(prompt_key(" A Lighthouse\n"), "a lighthouse")
|
||||
|
||||
def test_sql_functions_give_pythons_result_where_sql_lower_does_not(self):
|
||||
import sqlite3
|
||||
|
||||
conn = sqlite3.connect(":memory:")
|
||||
try:
|
||||
register_prompt_functions(conn)
|
||||
for value in self.CASES:
|
||||
with self.subTest(value=repr(value)):
|
||||
key, digest = conn.execute(
|
||||
"SELECT pm_prompt_key(?), pm_prompt_hash(?)", (value, value)
|
||||
).fetchone()
|
||||
self.assertEqual(key, prompt_key(value))
|
||||
self.assertEqual(digest, generate_prompt_hash(value))
|
||||
self.assertEqual(
|
||||
conn.execute(
|
||||
"SELECT pm_prompt_key(1), pm_prompt_hash(NULL)"
|
||||
).fetchone(),
|
||||
(None, None),
|
||||
)
|
||||
# SQLite's own lower() folds ASCII only
|
||||
self.assertNotEqual(
|
||||
conn.execute("SELECT lower(?)", ("\u00c9",)).fetchone()[0], "\u00e9"
|
||||
)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -17,6 +17,7 @@ from utils.execution_contexts import (
|
||||
)
|
||||
from utils.execution_outputs import OutputBundle
|
||||
from tests.test_execution_context_database import bundle, snapshot
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from tests.test_execution_output_database import bundle as outputs
|
||||
from tests import test_execution_history_database as history_tests
|
||||
|
||||
@@ -33,6 +34,10 @@ class HistoryReadTests(unittest.TestCase):
|
||||
conn.execute("INSERT INTO prompts(id,text) VALUES(?,?)", (identity, text))
|
||||
return identity
|
||||
|
||||
def hashed(self, db, text):
|
||||
"""A prompt saved as the nodes save it: with its dedup hash."""
|
||||
return db.save_prompt(text, prompt_hash=generate_prompt_hash(text))
|
||||
|
||||
def capture(self, db, item=0, context=None, output=None, time=100):
|
||||
start = self.start(db, item, time)
|
||||
db.write_execution_start(start)
|
||||
@@ -151,9 +156,9 @@ class HistoryReadTests(unittest.TestCase):
|
||||
)
|
||||
self.require_reads(db)
|
||||
with db.recorded_history_snapshot():
|
||||
# Outer whitespace joins the stored form's scope (#438); case and
|
||||
# Unicode normalization stay distinct.
|
||||
expected = (4, 1, 4, 4, 1, 1, 2, 2)
|
||||
# Outer whitespace (#438) and case (#460) join one scope;
|
||||
# Unicode normalization stays distinct.
|
||||
expected = (5, 5, 5, 5, 1, 1, 2, 2)
|
||||
for i, count in enumerate(expected, 1):
|
||||
self.assertEqual(db.list_history_window(i)["recorded_run_count"], count)
|
||||
self.assertIsNone(db.list_history_window(100))
|
||||
@@ -1047,3 +1052,70 @@ class HistoryReadTests(unittest.TestCase):
|
||||
with db.recorded_history_snapshot():
|
||||
found = db.get_newest_pair_run(positive, pair)
|
||||
self.assertEqual(found["run"]["run_id"], newest.run_id)
|
||||
|
||||
def test_case_variant_raw_text_joins_the_saved_prompt_scope(self):
|
||||
# #460: the hash folds case, so these runs count toward this prompt.
|
||||
db = self.database()
|
||||
positive = self.hashed(db, "\u00e9t\u00e9 lighthouse \u03c3")
|
||||
raws = (
|
||||
"\u00c9T\u00c9 Lighthouse \u03a3\n",
|
||||
"\u00e9t\u00e9 lighthouse \u03c3",
|
||||
" \u00c9t\u00e9 LIGHTHOUSE \u03a3",
|
||||
)
|
||||
starts = [
|
||||
self.capture(db, i, bundle(positive=raw), time=100 + i)
|
||||
for i, raw in enumerate(raws)
|
||||
]
|
||||
# Unicode normalization and inner text stay distinct
|
||||
self.capture(db, 7, bundle(positive="e\u0301te\u0301 lighthouse \u03c3"))
|
||||
self.capture(db, 8, bundle(positive="\u00e9t\u00e9 lighthouse \u03c3"))
|
||||
dotted = self.hashed(db, "\u0130stanbul")
|
||||
dotted_run = self.capture(db, 9, bundle(positive="\u0130STANBUL"), time=300)
|
||||
with db.recorded_history_snapshot():
|
||||
page = db.list_history_window(positive)
|
||||
details = [db.get_history_run(start.run_id) for start in starts]
|
||||
turkish = db.list_history_window(dotted)
|
||||
self.assertEqual(page["recorded_run_count"], 3)
|
||||
self.assertEqual(
|
||||
[r["run_id"] for r in page["items"]], [s.run_id for s in starts]
|
||||
)
|
||||
for raw, detail in zip(raws, details):
|
||||
context = detail["contexts"][0]
|
||||
self.assertEqual(context["positive"]["raw"], raw)
|
||||
self.assertEqual(
|
||||
context["positive_raw_text_id"], page["group"]["raw_text_id"]
|
||||
)
|
||||
self.assertEqual(context["positive_prompt_id"], positive)
|
||||
self.assertEqual([r["run_id"] for r in turkish["items"]], [dotted_run.run_id])
|
||||
|
||||
def test_newest_pair_run_follows_case_variant_raw_text(self):
|
||||
db = self.database()
|
||||
pair = db.save_prompt_pair("a lighthouse", "fog")
|
||||
positive = db.get_prompt_pair(pair)["positive_prompt_id"]
|
||||
newest = self.capture(db, 0, bundle("FOG", positive="A Lighthouse"), time=100)
|
||||
with db.recorded_history_snapshot():
|
||||
found = db.get_newest_pair_run(positive, pair)
|
||||
self.assertEqual(found["run"]["run_id"], newest.run_id)
|
||||
|
||||
def test_a_position_from_before_460_resumes_the_same_scope(self):
|
||||
db = self.database()
|
||||
positive = self.hashed(db, "A Lighthouse")
|
||||
for i in range(3):
|
||||
self.capture(db, i, bundle(positive="a lighthouse"), time=100 + i)
|
||||
with db.recorded_history_snapshot():
|
||||
first = db.list_history_window(positive, limit=2)
|
||||
position = first["next_position"]
|
||||
legacy = replace(
|
||||
position, raw_text_id=content_digest("text", ("A Lighthouse",))
|
||||
)
|
||||
rest = db.list_history_window(positive, limit=2, after=legacy)
|
||||
with self.assertRaises(ValueError):
|
||||
db.list_history_window(
|
||||
positive,
|
||||
after=replace(position, raw_text_id=content_digest("text", ("x",))),
|
||||
)
|
||||
self.assertEqual(first["recorded_run_count"], 3)
|
||||
self.assertEqual(len(rest["items"]), 1)
|
||||
self.assertEqual(
|
||||
rest["group"]["raw_text_id"], content_digest("text", ("a lighthouse",))
|
||||
)
|
||||
|
||||
@@ -172,17 +172,40 @@ class TestForeignWorkflows(LinkingTestCase):
|
||||
self.tracker.set_current_prompt(text, {"prompt_id": prompt_id})
|
||||
return prompt_id
|
||||
|
||||
def _linked_text(self):
|
||||
row = (
|
||||
self.db.model.get_connection()
|
||||
.execute(
|
||||
"SELECT p.text FROM generated_images g "
|
||||
"JOIN prompts p ON p.id = g.prompt_id WHERE g.image_path = ?",
|
||||
(self.image,),
|
||||
)
|
||||
.fetchone()
|
||||
)
|
||||
return row[0] if row else None
|
||||
|
||||
def test_foreign_image_leaves_the_batch_queue_aligned(self):
|
||||
"""#247: it links to its own prompt and never takes a queue entry."""
|
||||
first = self._queue_batch_item("item one")
|
||||
self.assertIsNone(self._process({"prompt": foreign_graph()}))
|
||||
self.assertEqual(self.linked, [])
|
||||
self.assertEqual(self.tracker.pop_next_prompt()["id"], first)
|
||||
self.assertIsNone(self.tracker.pop_next_prompt())
|
||||
self.assertEqual(self._linked_text(), "a cat")
|
||||
prompt = self.db.get_prompt_by_hash(generate_prompt_hash("a cat"))
|
||||
self.assertEqual(prompt["run_count"] or 0, 0)
|
||||
|
||||
def test_foreign_image_is_not_linked_by_the_later_fallbacks_either(self):
|
||||
def test_a_foreign_image_is_linked_only_by_the_ladder(self):
|
||||
self._queue_batch_item("item one") # live tracker and recent prompt exist
|
||||
untraceable = foreign_graph()
|
||||
untraceable["9"]["inputs"]["text"] = ["30", 0]
|
||||
untraceable["30"] = {"class_type": "VendorPromptMaker", "inputs": {}}
|
||||
self.assertIsNone(self._process({"prompt": untraceable}))
|
||||
self.assertEqual(self.linked, [])
|
||||
self.assertIsNone(self._linked_text())
|
||||
self.assertIsNone(self._process({"prompt": foreign_graph()}))
|
||||
self.assertEqual(self.linked, [])
|
||||
self.assertEqual(self._linked_text(), "a cat")
|
||||
|
||||
def test_negative_only_workflow_is_foreign(self):
|
||||
self._save("blurry")
|
||||
|
||||
@@ -597,6 +597,11 @@ class TestHandlerScheduling(unittest.TestCase):
|
||||
self.handler._schedule_recheck(path)
|
||||
self.assertIsNone(self.handler._worker)
|
||||
self.assertEqual(list(self.handler._history_due), [])
|
||||
# #247: a foreign file's sealed re-check is dropped the same way
|
||||
self.handler._await_foreign(path)
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
self.handler._schedule_recheck(path, {"filename": "late.png"})
|
||||
self.assertEqual(list(self.handler._foreign_due), [])
|
||||
self.assertEqual(self._live_workers() - before, set())
|
||||
|
||||
def test_a_sealed_run_starts_a_worker_on_an_open_handler(self):
|
||||
|
||||
@@ -887,6 +887,36 @@ class TestImageTagsFacet(unittest.TestCase):
|
||||
self.assertNotIn("image_tags", json.loads(encode(inspection({}))))
|
||||
self.assertNotIn("image_tags", self.encoded(None))
|
||||
|
||||
def history_encoded(self, facet):
|
||||
from utils.inspection_response import encode_history_inspection_response
|
||||
|
||||
body = encode_history_inspection_response(
|
||||
current_output={"history_epoch": "epoch"},
|
||||
file={"observation": "current-file", "size": 100},
|
||||
prompts=PROMPTS,
|
||||
resolution={"state": "resolved", "sampler_id": "3", "reason": ""},
|
||||
inspection=inspection({"prompt": graph_text()}),
|
||||
library={"status": "none"},
|
||||
image_tags=facet,
|
||||
)
|
||||
return json.loads(body)
|
||||
|
||||
def test_history_envelope_validates_image_tags_like_the_rooted_one(self):
|
||||
"""#457: History's current-file read carries the same facet."""
|
||||
for facet in (
|
||||
{"state": "current", "tags": ["dunes", "warm_light"]},
|
||||
{"state": "none", "tags": []},
|
||||
{"state": "unavailable", "tags": []},
|
||||
{"state": "current", "tags": ["a,b"]},
|
||||
{"state": "current", "tags": [], "locator": "/abs/path.png"},
|
||||
):
|
||||
with self.subTest(facet=facet):
|
||||
self.assertEqual(
|
||||
self.history_encoded(facet)["image_tags"],
|
||||
self.encoded(facet)["image_tags"],
|
||||
)
|
||||
self.assertNotIn("image_tags", self.history_encoded(None))
|
||||
|
||||
|
||||
class TestVideoProbeBoundsMatchPngBudgets(unittest.TestCase):
|
||||
"""ffprobe escapes tag text; the stdout bounds must never gate a record
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
"""#444: the Library's output-folder filter and multi-tag OR."""
|
||||
|
||||
import ntpath
|
||||
import os
|
||||
import posixpath
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from database.library_folders import (
|
||||
folder_prefixes,
|
||||
normalize_folder,
|
||||
relative_folder,
|
||||
)
|
||||
from database.operations import PromptDatabase
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
|
||||
class TestFolderNames(unittest.TestCase):
|
||||
def test_normalize_strips_outer_slashes_on_both_flavors(self):
|
||||
for flavor in (posixpath, ntpath):
|
||||
with self.subTest(flavor=flavor.__name__):
|
||||
self.assertEqual(normalize_folder("", flavor), "")
|
||||
self.assertEqual(normalize_folder("/", flavor), "")
|
||||
self.assertEqual(normalize_folder("/a/b/", flavor), "a/b")
|
||||
self.assertEqual(normalize_folder("..foo/b", flavor), "..foo/b")
|
||||
|
||||
def test_a_backslash_separates_only_on_windows(self):
|
||||
# Live B1: on POSIX a backslash is part of a folder name, as listed.
|
||||
name = "Metadata\\Klein_4b Wildcard Run"
|
||||
self.assertEqual(normalize_folder(name, posixpath), name)
|
||||
self.assertEqual(
|
||||
normalize_folder("2026\\08-Aug\\", posixpath), "2026\\08-Aug\\"
|
||||
)
|
||||
self.assertEqual(
|
||||
normalize_folder(name, ntpath), "Metadata/Klein_4b Wildcard Run"
|
||||
)
|
||||
self.assertEqual(normalize_folder("2026\\08-Aug\\", ntpath), "2026/08-Aug")
|
||||
# A colon names a drive only on Windows.
|
||||
self.assertEqual(normalize_folder("a:b/c", posixpath), "a:b/c")
|
||||
|
||||
def test_normalize_refuses_anything_but_a_folder_below_a_root(self):
|
||||
for flavor, value in (
|
||||
*(
|
||||
(flavor, value)
|
||||
for flavor in (posixpath, ntpath)
|
||||
for value in ("a/../b", "./a", "a//b", "a\x00b", 3, "x" * 1025)
|
||||
),
|
||||
(ntpath, "C:/x"),
|
||||
(ntpath, "c:"),
|
||||
(ntpath, "a\\..\\b"),
|
||||
):
|
||||
with self.subTest(flavor=flavor.__name__, value=value):
|
||||
with self.assertRaises(ValueError):
|
||||
normalize_folder(value, flavor)
|
||||
|
||||
def test_the_default_flavor_is_the_platform_path_module(self):
|
||||
self.assertEqual(normalize_folder("a\\b"), normalize_folder("a\\b", os.path))
|
||||
|
||||
def test_posix_prefixes_are_exact_and_end_in_a_separator(self):
|
||||
self.assertEqual(
|
||||
folder_prefixes(["/out", "/data/out/"], "a/b", posixpath),
|
||||
["/out/a/b/", "/data/out/a/b/"],
|
||||
)
|
||||
self.assertEqual(folder_prefixes(["/"], "a", posixpath), ["/a/"])
|
||||
self.assertEqual(
|
||||
folder_prefixes(["/out", "/out/"], "a", posixpath), ["/out/a/"]
|
||||
)
|
||||
with self.assertRaises(ValueError):
|
||||
folder_prefixes([], "a", posixpath)
|
||||
|
||||
def test_windows_prefixes_fold_case_including_non_ascii(self):
|
||||
self.assertEqual(
|
||||
folder_prefixes(["C:\\ComfyUI\\Output"], "Ä/Sub", ntpath),
|
||||
["c:\\comfyui\\output\\ä\\sub\\"],
|
||||
)
|
||||
self.assertEqual(folder_prefixes(["D:\\"], "x", ntpath), ["d:\\x\\"])
|
||||
self.assertEqual(
|
||||
folder_prefixes(["\\\\Server\\Share\\Out"], "a", ntpath),
|
||||
["\\\\server\\share\\out\\a\\"],
|
||||
)
|
||||
# The stored file_path is normcase(normpath(abspath)), the same fold.
|
||||
stored = ntpath.normcase(ntpath.normpath("C:/ComfyUI/output/Ä/sub/x.png"))
|
||||
self.assertTrue(
|
||||
stored.startswith(
|
||||
folder_prefixes(["C:/ComfyUI/Output"], "ä/SUB", ntpath)[0]
|
||||
)
|
||||
)
|
||||
|
||||
def test_relative_folder_keeps_the_stored_spelling(self):
|
||||
self.assertEqual(
|
||||
relative_folder("/out/2026/Aug/x.png", ["/out"], posixpath), "2026/Aug"
|
||||
)
|
||||
self.assertIsNone(relative_folder("/out/x.png", ["/out"], posixpath))
|
||||
self.assertIsNone(relative_folder("/outside/x.png", ["/out"], posixpath))
|
||||
self.assertIsNone(relative_folder("/output2/a/x.png", ["/out"], posixpath))
|
||||
self.assertEqual(
|
||||
relative_folder("/out/..foo/x.png", ["/out"], posixpath), "..foo"
|
||||
)
|
||||
self.assertEqual(
|
||||
relative_folder("C:\\Out\\Ä\\Sub\\x.png", ["c:\\out"], ntpath), "Ä/Sub"
|
||||
)
|
||||
self.assertEqual(relative_folder("D:\\a\\x.png", ["D:\\"], ntpath), "a")
|
||||
|
||||
|
||||
class TestLibraryFolderAndAnyTag(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
base = Path(os.path.realpath(self.temp.name))
|
||||
self.root = base / "output"
|
||||
self.root.mkdir()
|
||||
self.db = PromptDatabase(str(base / "folders.db"))
|
||||
|
||||
def tearDown(self):
|
||||
try:
|
||||
self.db.close_all()
|
||||
finally:
|
||||
self.temp.cleanup()
|
||||
|
||||
def save(self, text, **metadata):
|
||||
return self.db.save_prompt(
|
||||
text, prompt_hash=generate_prompt_hash(text), **metadata
|
||||
)
|
||||
|
||||
def link(self, prompt_id, *parts):
|
||||
path = self.root.joinpath(*parts)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(b"x")
|
||||
self.db.link_image_to_prompt(prompt_id, str(path))
|
||||
|
||||
def texts(self, **options):
|
||||
return sorted(item["text"] for item in self.db.list_library(**options)["items"])
|
||||
|
||||
def in_folder(self, folder, roots=None):
|
||||
return self.texts(
|
||||
folder_prefixes=folder_prefixes(roots or [str(self.root)], folder)
|
||||
)
|
||||
|
||||
def test_folder_matches_its_descendants_only(self):
|
||||
self.link(self.save("deep"), "a", "b", "x.png")
|
||||
self.link(self.save("top"), "a", "y.png")
|
||||
self.link(self.save("sibling"), "ab", "z.png")
|
||||
self.link(self.save("literal"), "100%_off", "w.png")
|
||||
self.save("no images")
|
||||
self.assertEqual(self.in_folder("a"), ["deep", "top"])
|
||||
self.assertEqual(self.in_folder("a/b"), ["deep"])
|
||||
self.assertEqual(self.in_folder("ab"), ["sibling"])
|
||||
self.assertEqual(self.in_folder("100%_off"), ["literal"])
|
||||
self.assertEqual(self.in_folder("1"), [])
|
||||
cut = self.db.list_library(
|
||||
folder_prefixes=folder_prefixes([str(self.root)], "a"), limit=1
|
||||
)
|
||||
self.assertEqual(cut["total"], 2)
|
||||
|
||||
def test_folder_combines_with_text_category_and_tags(self):
|
||||
self.link(self.save("red cat", category="Art", tags=["cat"]), "a", "1.png")
|
||||
self.link(self.save("red dog", category="Art", tags=["dog"]), "a", "2.png")
|
||||
self.link(self.save("red bird", category="Art", tags=["cat"]), "b", "3.png")
|
||||
prefixes = folder_prefixes([str(self.root)], "a")
|
||||
self.assertEqual(
|
||||
self.texts(folder_prefixes=prefixes, tags=["cat"]), ["red cat"]
|
||||
)
|
||||
self.assertEqual(
|
||||
self.texts(folder_prefixes=prefixes, text="dog", category="Art"),
|
||||
["red dog"],
|
||||
)
|
||||
|
||||
@unittest.skipIf(os.name == "nt", "symlinks need privileges on Windows")
|
||||
def test_folder_listing_and_match_through_a_symlinked_root(self):
|
||||
link = self.root.parent / "linked"
|
||||
os.symlink(self.root, link)
|
||||
prompt = self.save("via link")
|
||||
(self.root / "a").mkdir()
|
||||
(self.root / "a" / "x.png").write_bytes(b"x")
|
||||
self.db.link_image_to_prompt(prompt, str(link / "a" / "x.png"))
|
||||
spellings = [str(self.root), str(link)]
|
||||
self.assertEqual(self.in_folder("a", spellings), ["via link"])
|
||||
self.assertEqual(self.in_folder("a", [str(self.root)]), [])
|
||||
self.assertEqual(self.db.get_library_options(spellings)["folders"], ["a"])
|
||||
|
||||
@unittest.skipIf(os.name == "nt", "a backslash is a separator on Windows")
|
||||
def test_a_posix_folder_with_a_backslash_lists_and_filters_alike(self):
|
||||
name = "Metadata\\Klein_4b Wildcard Run"
|
||||
self.link(self.save("klein"), name, "x.png")
|
||||
self.link(self.save("other"), "Metadata", "y.png")
|
||||
listed = self.db.get_library_options([str(self.root)])["folders"]
|
||||
self.assertEqual(listed, ["Metadata", name])
|
||||
self.assertEqual(self.in_folder(normalize_folder(name)), ["klein"])
|
||||
self.assertEqual(self.in_folder(normalize_folder("Metadata")), ["other"])
|
||||
|
||||
def test_folder_listing_offers_ancestors_and_skips_outside_images(self):
|
||||
self.link(self.save("deep"), "2026", "08-Aug", "x.png")
|
||||
self.link(self.save("root image"), "y.png")
|
||||
outside = Path(self.temp.name) / "elsewhere" / "z.png"
|
||||
outside.parent.mkdir()
|
||||
outside.write_bytes(b"x")
|
||||
self.db.link_image_to_prompt(self.save("outside"), str(outside))
|
||||
options = self.db.get_library_options([str(self.root)])
|
||||
self.assertEqual(options["folders"], ["2026", "2026/08-Aug"])
|
||||
self.assertNotIn("folders", self.db.get_library_options())
|
||||
|
||||
def test_folder_combines_with_untagged(self):
|
||||
self.link(self.save("plain in a"), "a", "1.png")
|
||||
self.link(self.save("tagged in a", tags=["t"]), "a", "2.png")
|
||||
self.link(self.save("plain in b"), "b", "3.png")
|
||||
prefixes = folder_prefixes([str(self.root)], "a")
|
||||
self.assertEqual(
|
||||
self.texts(folder_prefixes=prefixes, tag_mode="untagged"), ["plain in a"]
|
||||
)
|
||||
|
||||
def test_any_tag_mode_is_the_union_across_parent_and_pair(self):
|
||||
self.save("parent a", tags=["a"])
|
||||
self.save("parent b", tags=["b"])
|
||||
self.save("both", tags=["a", "b"])
|
||||
self.save("neither", tags=["c"])
|
||||
self.db.save_prompt_pair("pair b", "noise", tags=["b"])
|
||||
split = self.save("split", tags=["a"])
|
||||
self.db.save_prompt_pair("split", "split noise", tags=["b"])
|
||||
archived = self.db.save_prompt_pair("archived b", "old", tags=["b"])
|
||||
self.db.archive_prompt_pair(archived)
|
||||
self.assertIsInstance(split, int)
|
||||
self.assertEqual(
|
||||
self.texts(tags=["a", "b"], tag_mode="any"),
|
||||
["both", "pair b", "parent a", "parent b", "split"],
|
||||
)
|
||||
self.assertEqual(self.texts(tags=["a", "b"], tag_mode="all"), ["both"])
|
||||
self.assertEqual(self.texts(tags=["a", "b"]), ["both"])
|
||||
for tags in (["a"], ["b"], ["c"]):
|
||||
self.assertEqual(
|
||||
self.texts(tags=tags, tag_mode="any"), self.texts(tags=tags)
|
||||
)
|
||||
self.assertEqual(
|
||||
self.texts(tags=["a", "b"], tag_mode="any", text="parent"),
|
||||
["parent a", "parent b"],
|
||||
)
|
||||
self.assertEqual(self.texts(tag_mode="any"), self.texts())
|
||||
for mode in ("none", None, "ANY"):
|
||||
with self.subTest(mode=mode), self.assertRaises(ValueError):
|
||||
self.db.list_library(tag_mode=mode)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,951 @@
|
||||
"""tools/repair_foreign_links.py: the reviewed repair of the #247 dataset links.
|
||||
|
||||
The fixture is a library whose foreign (no PromptManager node) dataset run
|
||||
wrote shared-prefix files that the old scan linked to the wrong branch text.
|
||||
Recorded history names each file's saver, so the ladder (``recorded-run``)
|
||||
names the right branch.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sqlite3
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
from PIL.PngImagePlugin import PngInfo
|
||||
|
||||
try:
|
||||
from tests.open_handles import assert_closed
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from open_handles import assert_closed
|
||||
|
||||
from database.models import PromptModel
|
||||
from database.operations import PromptDatabase
|
||||
from tests import saver_anchor_graphs as shapes
|
||||
from tests.test_run_attribution import RecordedRuns
|
||||
from tools import repair_foreign_links as tool
|
||||
from utils.file_prompt import Decision
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
TEXTS = shapes.BRANCH_TEXTS
|
||||
NAME = "IMG_GEN_{:05}_.png"
|
||||
|
||||
|
||||
def png(path, graph):
|
||||
info = PngInfo()
|
||||
info.add_text("prompt", json.dumps(graph))
|
||||
Image.new("RGB", (4, 4)).save(path, pnginfo=info)
|
||||
|
||||
|
||||
def sha(text):
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def file_sha(path):
|
||||
return hashlib.sha256(Path(path).read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def dump(path):
|
||||
"""The database's logical content (independent of WAL checkpoints)."""
|
||||
conn = sqlite3.connect(path)
|
||||
try:
|
||||
return list(conn.iterdump())
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def not_running(_comfyui_dir, _port):
|
||||
return []
|
||||
|
||||
|
||||
def no_handles(_database):
|
||||
return []
|
||||
|
||||
|
||||
class RepairFixture(RecordedRuns, unittest.TestCase):
|
||||
"""Rows: A relink to a new prompt, B relink to a stored prompt (with a
|
||||
pair link), C keep, D file gone, E two savers (skip), F the UNIQUE case
|
||||
(a wrong row beside a correct row), H relink from a rated prompt, G a
|
||||
PromptManager run (not a candidate)."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self.addCleanup(shutil.rmtree, self.tmp, True)
|
||||
self.addCleanup(assert_closed, self, self.tmp)
|
||||
self.addCleanup(PromptModel.reset_schema_cache)
|
||||
self.output = os.path.join(self.tmp, "output")
|
||||
os.makedirs(os.path.join(self.output, "dataset"))
|
||||
self.db_path = os.path.join(self.tmp, "prompts.db")
|
||||
self.plan_path = os.path.join(self.tmp, "plan.json")
|
||||
self.backups = os.path.join(self.tmp, "backups")
|
||||
os.makedirs(self.backups)
|
||||
self.comfy = os.path.join(self.tmp, "comfy")
|
||||
os.makedirs(self.comfy)
|
||||
Path(self.comfy, "main.py").write_text("# ComfyUI\n", encoding="utf-8")
|
||||
self.files = {}
|
||||
for key, number in zip("ABCDEFHG", range(1, 9)):
|
||||
path = os.path.join(self.output, "dataset", NAME.format(number))
|
||||
png(path, shapes.shared_prefix())
|
||||
self.files[key] = path
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
self.build(db)
|
||||
finally:
|
||||
db.close_all()
|
||||
os.remove(self.files["D"])
|
||||
|
||||
def build(self, db):
|
||||
save = lambda text, **kw: db.save_prompt( # noqa: E731
|
||||
text=text, prompt_hash=generate_prompt_hash(text), **kw
|
||||
)
|
||||
self.p0 = save(TEXTS[0], category="scanned", tags=["auto-scanned"])
|
||||
self.p2 = save(TEXTS[2], category="scanned", tags=["auto-scanned"])
|
||||
self.orphan = save(
|
||||
"an old guess",
|
||||
category="scanned",
|
||||
tags=["auto-scanned"],
|
||||
notes="Auto-scanned from IMG_GEN_00001_.png",
|
||||
)
|
||||
self.rated = save("a rated guess", rating=5)
|
||||
link = lambda prompt, key: db.link_image_to_prompt( # noqa: E731
|
||||
prompt, self.files[key], metadata={}
|
||||
)
|
||||
self.rows = {
|
||||
"A": link(self.orphan, "A"),
|
||||
"B": link(self.p0, "B"),
|
||||
"C": link(self.p0, "C"),
|
||||
"D": link(self.p0, "D"),
|
||||
"E": link(self.p0, "E"),
|
||||
"F": link(self.p0, "F"),
|
||||
"F_ok": link(self.p2, "F"),
|
||||
"H": link(self.rated, "H"),
|
||||
"G": link(self.p0, "G"),
|
||||
}
|
||||
conn = db.model.get_connection()
|
||||
with conn:
|
||||
pair = conn.execute(
|
||||
"INSERT INTO prompt_pairs(positive_prompt_id) VALUES(?)", (self.p0,)
|
||||
).lastrowid
|
||||
conn.execute(
|
||||
"INSERT INTO image_prompt_pairs(image_id, pair_id, provenance) "
|
||||
"VALUES(?,?,'{}')",
|
||||
(self.rows["B"], pair),
|
||||
)
|
||||
name = lambda key: ("dataset", os.path.basename(self.files[key])) # noqa
|
||||
# Savers of the shared-prefix graph: "13", "23", "33", "43" (branches 1-4)
|
||||
self.record_run(
|
||||
db,
|
||||
self.output,
|
||||
[name(k) for k in "CDEAEBFH"],
|
||||
[],
|
||||
nodes=["13", "13", "13", "23", "23", "33", "33", "43"],
|
||||
)
|
||||
self.record_run(db, self.output, [name("G")], [TEXTS[0]], item=1, nodes=["23"])
|
||||
|
||||
def dry_run(self, *args):
|
||||
out = io.StringIO()
|
||||
code = tool.main(
|
||||
[
|
||||
self.db_path,
|
||||
"--output-dir",
|
||||
self.output,
|
||||
"--plan",
|
||||
self.plan_path,
|
||||
*args,
|
||||
],
|
||||
out=out,
|
||||
)
|
||||
return code, out.getvalue()
|
||||
|
||||
def apply(self, *args, probe=not_running, handles=no_handles):
|
||||
out = io.StringIO()
|
||||
code = tool.main(
|
||||
[
|
||||
self.db_path,
|
||||
"--apply",
|
||||
self.plan_path,
|
||||
"--comfyui-dir",
|
||||
os.path.join(self.tmp, "comfy"),
|
||||
"--backup-dir",
|
||||
self.backups,
|
||||
*args,
|
||||
],
|
||||
out=out,
|
||||
probe=probe,
|
||||
handles=handles,
|
||||
)
|
||||
return code, out.getvalue()
|
||||
|
||||
def plan(self):
|
||||
with open(self.plan_path, encoding="utf-8") as stream:
|
||||
return json.load(stream)
|
||||
|
||||
def write_plan(self, plan):
|
||||
with open(self.plan_path, "w", encoding="utf-8") as stream:
|
||||
json.dump(plan, stream)
|
||||
|
||||
def query(self, sql, args=()):
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
try:
|
||||
return conn.execute(sql, args).fetchall()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def query_write(self, sql, args):
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
try:
|
||||
with conn:
|
||||
conn.execute(sql, args)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def links(self):
|
||||
return dict(self.query("SELECT id, prompt_id FROM generated_images"))
|
||||
|
||||
|
||||
class DryRunTests(RepairFixture):
|
||||
def test_plan_names_relink_keep_and_skip_rows(self):
|
||||
code, out = self.dry_run()
|
||||
self.assertEqual(code, 0, out)
|
||||
items = {item["row_id"]: item for item in self.plan()["rows"]}
|
||||
rows = self.rows
|
||||
self.assertNotIn(rows["G"], items, "a PromptManager run is not foreign")
|
||||
actions = {key: items[rows[key]]["action"] for key in rows if key != "G"}
|
||||
self.assertEqual(
|
||||
actions,
|
||||
{
|
||||
"A": "relink",
|
||||
"B": "relink",
|
||||
"C": "keep",
|
||||
"D": "skip",
|
||||
"E": "skip",
|
||||
"F": "relink",
|
||||
"F_ok": "keep",
|
||||
"H": "relink",
|
||||
},
|
||||
)
|
||||
a = items[rows["A"]]
|
||||
self.assertEqual(a["current_prompt_id"], self.orphan)
|
||||
self.assertEqual(a["current_text_sha256"], sha("an old guess"))
|
||||
self.assertEqual(a["target_text"], TEXTS[1])
|
||||
self.assertEqual(a["target_hash"], generate_prompt_hash(TEXTS[1]))
|
||||
self.assertEqual(a["target_prompt_id"], "new")
|
||||
self.assertEqual(a["file"], self.files["A"])
|
||||
self.assertEqual(items[rows["B"]]["target_prompt_id"], self.p2)
|
||||
self.assertEqual(items[rows["H"]]["target_prompt_id"], "new")
|
||||
self.assertEqual(items[rows["D"]]["reason"], "file gone")
|
||||
self.assertEqual(
|
||||
items[rows["E"]]["reason"], "not decided by the recorded saver"
|
||||
)
|
||||
self.assertIn("relink: 4", out)
|
||||
self.assertIn("keep: 2", out)
|
||||
self.assertIn("skip: 2", out)
|
||||
|
||||
def test_orphan_audit_lists_the_prompts_losing_every_link(self):
|
||||
self.dry_run()
|
||||
plan = self.plan()
|
||||
orphans = {o["id"]: o for o in plan["orphans"]}
|
||||
self.assertEqual(set(orphans), {self.orphan, self.rated})
|
||||
self.assertEqual(orphans[self.orphan]["blockers"], [])
|
||||
self.assertTrue(orphans[self.orphan]["safe"])
|
||||
self.assertEqual(orphans[self.rated]["blockers"], ["usage or rating"])
|
||||
self.assertFalse(orphans[self.rated]["safe"])
|
||||
self.assertFalse(plan["delete_orphans"], "off by default")
|
||||
self.dry_run("--delete-orphans")
|
||||
self.assertTrue(self.plan()["delete_orphans"])
|
||||
|
||||
def test_dry_run_writes_nothing_and_writes_the_markdown_list(self):
|
||||
before = file_sha(self.db_path)
|
||||
listing = sorted(os.listdir(self.tmp))
|
||||
code, out = self.dry_run()
|
||||
self.assertEqual(code, 0)
|
||||
self.assertEqual(file_sha(self.db_path), before)
|
||||
self.assertIn("Dry run: nothing was changed.", out)
|
||||
# A read-only reader of a WAL library may leave its empty side files
|
||||
sidecars = {"prompts.db-shm", "prompts.db-wal"}
|
||||
self.assertEqual(
|
||||
sorted(set(os.listdir(self.tmp)) - sidecars),
|
||||
sorted(set(listing + ["plan.json", "plan.list.md"]) - sidecars),
|
||||
)
|
||||
wal = Path(self.tmp, "prompts.db-wal")
|
||||
self.assertEqual(wal.stat().st_size if wal.exists() else 0, 0)
|
||||
markdown = Path(self.tmp, "plan.list.md").read_text(encoding="utf-8")
|
||||
self.assertIn(f"| {self.rows['A']} |", markdown)
|
||||
long_path = os.path.join("dataset", NAME.format(1))
|
||||
self.assertIn(f"| {long_path} |", markdown, "file paths are never cut")
|
||||
self.assertIn(generate_prompt_hash(TEXTS[1])[:12], markdown)
|
||||
self.assertIn("new", markdown)
|
||||
self.assertIn("would lose every link", markdown)
|
||||
|
||||
|
||||
class DryRunReviewTests(RepairFixture):
|
||||
def test_plan_records_its_library_and_fingerprint(self):
|
||||
self.dry_run()
|
||||
plan = self.plan()
|
||||
self.assertEqual(
|
||||
plan["source_db"], os.path.normcase(os.path.realpath(self.db_path))
|
||||
)
|
||||
self.assertEqual(plan["max_row_id"], max(self.links()))
|
||||
|
||||
def test_a_markdown_plan_path_is_refused(self):
|
||||
out = io.StringIO()
|
||||
with (
|
||||
self.assertRaises(SystemExit),
|
||||
contextlib.redirect_stderr(io.StringIO()),
|
||||
):
|
||||
tool.main(
|
||||
[
|
||||
self.db_path,
|
||||
"--output-dir",
|
||||
self.output,
|
||||
"--plan",
|
||||
os.path.join(self.tmp, "plan.md"),
|
||||
],
|
||||
out=out,
|
||||
)
|
||||
self.assertFalse(os.path.exists(os.path.join(self.tmp, "plan.md")))
|
||||
|
||||
def test_keep_compares_the_linked_text_not_the_stored_hash(self):
|
||||
self.query_write("UPDATE prompts SET hash='stale' WHERE id=?", (self.p0,))
|
||||
self.dry_run()
|
||||
items = {item["row_id"]: item for item in self.plan()["rows"]}
|
||||
self.assertEqual(items[self.rows["C"]]["action"], "keep")
|
||||
|
||||
def test_delete_orphans_is_rehearsed_with_the_deletes(self):
|
||||
code, out = self.dry_run("--delete-orphans")
|
||||
self.assertEqual(code, 0, out)
|
||||
simulated = self.plan()["simulated"]
|
||||
self.assertEqual(simulated["result"]["orphans_deleted"], 1)
|
||||
self.assertEqual(
|
||||
simulated["after"]["prompts"], simulated["before"]["prompts"] + 2 - 1
|
||||
)
|
||||
|
||||
def test_a_file_on_another_drive_is_listed_by_its_full_path(self):
|
||||
self.dry_run()
|
||||
plan = self.plan()
|
||||
with unittest.mock.patch.object(
|
||||
tool.os.path, "relpath", side_effect=ValueError("other drive")
|
||||
):
|
||||
listing = tool.markdown(plan, {})
|
||||
self.assertIn(self.files["A"], listing)
|
||||
|
||||
|
||||
class ApplyTests(RepairFixture):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
code, out = self.dry_run()
|
||||
self.assertEqual(code, 0, out)
|
||||
|
||||
def test_apply_relinks_creates_targets_and_drops_pair_links(self):
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 0, out)
|
||||
links = self.links()
|
||||
new = dict(self.query("SELECT hash, id FROM prompts"))
|
||||
target_a = new[generate_prompt_hash(TEXTS[1])]
|
||||
target_h = new[generate_prompt_hash(TEXTS[3])]
|
||||
self.assertEqual(links[self.rows["A"]], target_a)
|
||||
self.assertEqual(links[self.rows["B"]], self.p2)
|
||||
self.assertEqual(links[self.rows["H"]], target_h)
|
||||
self.assertEqual(links[self.rows["C"]], self.p0, "keep")
|
||||
self.assertEqual(links[self.rows["E"]], self.p0, "skip")
|
||||
self.assertEqual(links[self.rows["G"]], self.p0, "not foreign")
|
||||
self.assertEqual(
|
||||
self.query(
|
||||
"SELECT COUNT(*) FROM image_prompt_pairs WHERE image_id=?",
|
||||
(self.rows["B"],),
|
||||
),
|
||||
[(0,)],
|
||||
)
|
||||
category, notes = self.query(
|
||||
"SELECT category, notes FROM prompts WHERE id=?", (target_a,)
|
||||
)[0]
|
||||
self.assertEqual(category, "scanned")
|
||||
self.assertEqual(
|
||||
notes, f"Auto-scanned from {os.path.basename(self.files['A'])}"
|
||||
)
|
||||
tags = self.query(
|
||||
"SELECT t.name FROM prompt_tags pt JOIN tags t ON t.id=pt.tag_id "
|
||||
"WHERE pt.prompt_id=?",
|
||||
(target_a,),
|
||||
)
|
||||
self.assertEqual(tags, [("auto-scanned",)])
|
||||
self.assertIn("integrity_check", out)
|
||||
self.assertIn("foreign_key_check", out)
|
||||
self.assertRegex(out, r"integrity_check\s+ok\s+ok")
|
||||
# Orphans are kept unless the plan chose to delete them
|
||||
self.assertTrue(self.query("SELECT 1 FROM prompts WHERE id=?", (self.orphan,)))
|
||||
|
||||
def test_unique_case_deletes_the_wrong_row(self):
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 0, out)
|
||||
links = self.links()
|
||||
self.assertNotIn(self.rows["F"], links)
|
||||
self.assertEqual(links[self.rows["F_ok"]], self.p2)
|
||||
rows = self.query(
|
||||
"SELECT prompt_id FROM generated_images WHERE image_path=?",
|
||||
(self.files["F"],),
|
||||
)
|
||||
self.assertEqual(rows, [(self.p2,)])
|
||||
|
||||
def test_backup_is_taken_before_writing(self):
|
||||
before = self.links()
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 0, out)
|
||||
names = os.listdir(self.backups)
|
||||
self.assertEqual(len(names), 1)
|
||||
self.assertRegex(names[0], r"^prompts\.db\.pre-247-repair_\d{8}_\d{6}$")
|
||||
backup = os.path.join(self.backups, names[0])
|
||||
conn = sqlite3.connect(backup)
|
||||
try:
|
||||
saved = dict(conn.execute("SELECT id, prompt_id FROM generated_images"))
|
||||
finally:
|
||||
conn.close()
|
||||
self.assertEqual(saved, before)
|
||||
self.assertIn(f"Backup: {backup}", out)
|
||||
|
||||
def test_backup_beside_the_database_by_default(self):
|
||||
out = io.StringIO()
|
||||
code = tool.main(
|
||||
[
|
||||
self.db_path,
|
||||
"--apply",
|
||||
self.plan_path,
|
||||
"--comfyui-dir",
|
||||
os.path.join(self.tmp, "comfy"),
|
||||
],
|
||||
out=out,
|
||||
probe=not_running,
|
||||
handles=no_handles,
|
||||
)
|
||||
self.assertEqual(code, 0, out.getvalue())
|
||||
self.assertTrue(
|
||||
any(
|
||||
n.startswith("prompts.db.pre-247-repair_") for n in os.listdir(self.tmp)
|
||||
)
|
||||
)
|
||||
|
||||
def test_refuses_while_comfyui_runs(self):
|
||||
before = file_sha(self.db_path)
|
||||
code, out = self.apply(probe=lambda _d, _p: ["process 42 (main.py)"])
|
||||
self.assertEqual(code, 2)
|
||||
self.assertIn("REFUSED", out)
|
||||
self.assertIn("process 42", out)
|
||||
self.assertEqual(os.listdir(self.backups), [], "no backup either")
|
||||
self.assertEqual(file_sha(self.db_path), before)
|
||||
|
||||
def test_rehearsal_skips_the_running_check_on_a_copy_only(self):
|
||||
code, out = self.apply("--rehearsal", probe=lambda _d, _p: ["port 8188"])
|
||||
self.assertEqual(code, 0, out)
|
||||
self.assertIn("rehearsal", out)
|
||||
# The live library of that ComfyUI folder: --rehearsal is refused
|
||||
live_dir = os.path.join(self.tmp, "comfy", "user", "__promptmanager")
|
||||
os.makedirs(live_dir)
|
||||
live = os.path.join(live_dir, "prompts.db")
|
||||
shutil.copy(self.db_path, live)
|
||||
out = io.StringIO()
|
||||
code = tool.main(
|
||||
[
|
||||
live,
|
||||
"--apply",
|
||||
self.plan_path,
|
||||
"--comfyui-dir",
|
||||
os.path.join(self.tmp, "comfy"),
|
||||
"--rehearsal",
|
||||
],
|
||||
out=out,
|
||||
probe=lambda _d, _p: [],
|
||||
handles=no_handles,
|
||||
)
|
||||
self.assertEqual(code, 2)
|
||||
self.assertIn("--rehearsal", out.getvalue())
|
||||
|
||||
def test_drift_refuses_and_changes_nothing(self):
|
||||
cases = {
|
||||
"row moved": lambda: self.query_write(
|
||||
"UPDATE generated_images SET prompt_id=? WHERE id=?",
|
||||
(self.p2, self.rows["H"]),
|
||||
),
|
||||
"text changed": lambda: self.query_write(
|
||||
"UPDATE prompts SET text='an older guess' WHERE id=?", (self.orphan,)
|
||||
),
|
||||
"file changed": lambda: png(self.files["B"], shapes.branch_bench()),
|
||||
"target stored since": lambda: self.query_write(
|
||||
"INSERT INTO prompts(text, hash) VALUES(?,?)",
|
||||
(TEXTS[1], generate_prompt_hash(TEXTS[1])),
|
||||
),
|
||||
}
|
||||
pristine = os.path.join(self.tmp, "pristine.db")
|
||||
shutil.copy(self.db_path, pristine)
|
||||
image = Path(self.files["B"]).read_bytes()
|
||||
for name, drift in cases.items():
|
||||
with self.subTest(name):
|
||||
shutil.copy(pristine, self.db_path)
|
||||
Path(self.files["B"]).write_bytes(image)
|
||||
shutil.rmtree(self.backups)
|
||||
os.makedirs(self.backups)
|
||||
drift()
|
||||
before = dump(self.db_path)
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 2, out)
|
||||
self.assertIn("REFUSED (rolled back, nothing changed)", out)
|
||||
self.assertEqual(dump(self.db_path), before)
|
||||
|
||||
def test_late_failure_rolls_back_every_write(self):
|
||||
plan = self.plan()
|
||||
plan["delete_orphans"] = True
|
||||
self.write_plan(plan)
|
||||
# The safe orphan gained a user tag after the dry run
|
||||
self.query_write("INSERT INTO tags(name) VALUES('mine')", ())
|
||||
self.query_write(
|
||||
"INSERT INTO prompt_tags(prompt_id, tag_id) "
|
||||
"SELECT ?, id FROM tags WHERE name='mine'",
|
||||
(self.orphan,),
|
||||
)
|
||||
before = self.links()
|
||||
prompts = self.query("SELECT COUNT(*) FROM prompts")
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 2, out)
|
||||
self.assertIn(f"prompt {self.orphan}", out)
|
||||
self.assertIn("user tags", out)
|
||||
self.assertEqual(self.links(), before, "the relinks were rolled back")
|
||||
self.assertEqual(self.query("SELECT COUNT(*) FROM prompts"), prompts)
|
||||
|
||||
def test_orphans_are_deleted_only_when_chosen_and_unreferenced(self):
|
||||
plan = self.plan()
|
||||
plan["delete_orphans"] = True
|
||||
self.write_plan(plan)
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 0, out)
|
||||
self.assertFalse(self.query("SELECT 1 FROM prompts WHERE id=?", (self.orphan,)))
|
||||
self.assertTrue(
|
||||
self.query("SELECT 1 FROM prompts WHERE id=?", (self.rated,)),
|
||||
"a prompt with a rating is never deleted",
|
||||
)
|
||||
self.assertIn("orphans_deleted", out)
|
||||
|
||||
def test_an_applied_plan_refuses_a_second_time(self):
|
||||
self.assertEqual(self.apply()[0], 0)
|
||||
before = dump(self.db_path)
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 2)
|
||||
self.assertIn("REFUSED", out)
|
||||
self.assertEqual(dump(self.db_path), before)
|
||||
|
||||
|
||||
class ApplySafetyTests(RepairFixture):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
code, out = self.dry_run()
|
||||
self.assertEqual(code, 0, out)
|
||||
|
||||
def assert_unchanged(self, before, code, out):
|
||||
self.assertEqual(code, 2, out)
|
||||
self.assertEqual(dump(self.db_path), before)
|
||||
|
||||
def test_refuses_when_another_process_has_the_library_open(self):
|
||||
before = dump(self.db_path)
|
||||
held = lambda _db: ["process 7 has prompts.db-wal open"] # noqa: E731
|
||||
for args in ((), ("--rehearsal",)):
|
||||
with self.subTest(args):
|
||||
code, out = self.apply(*args, handles=held)
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("process 7", out)
|
||||
self.assertEqual(os.listdir(self.backups), [])
|
||||
|
||||
def test_the_running_check_runs_again_just_before_writing(self):
|
||||
calls = []
|
||||
|
||||
def late(_d, _p):
|
||||
calls.append(1)
|
||||
return [] if len(calls) == 1 else ["something listens on port 8188"]
|
||||
|
||||
before = dump(self.db_path)
|
||||
code, out = self.apply(probe=late)
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertEqual(len(calls), 2)
|
||||
self.assertIn("rolled back", out)
|
||||
self.assertIn("Backup:", out)
|
||||
|
||||
def test_without_proc_only_an_explicit_stop_lets_it_through(self):
|
||||
blind = lambda _db: [tool.NO_PROC] # noqa: E731
|
||||
before = dump(self.db_path)
|
||||
code, out = self.apply(handles=blind)
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("--i-stopped-comfyui", out)
|
||||
code, out = self.apply("--i-stopped-comfyui", handles=blind)
|
||||
self.assertEqual(code, 0, out)
|
||||
|
||||
def test_a_folder_without_main_py_is_refused(self):
|
||||
os.remove(os.path.join(self.comfy, "main.py"))
|
||||
before = dump(self.db_path)
|
||||
code, out = self.apply()
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("main.py", out)
|
||||
|
||||
def test_a_plan_for_another_library_is_refused(self):
|
||||
other = os.path.join(self.tmp, "other.db")
|
||||
shutil.copy(self.db_path, other)
|
||||
out = io.StringIO()
|
||||
code = tool.main(
|
||||
[other, "--apply", self.plan_path, "--comfyui-dir", self.comfy],
|
||||
out=out,
|
||||
probe=not_running,
|
||||
handles=no_handles,
|
||||
)
|
||||
self.assertEqual(code, 2)
|
||||
self.assertIn("made for", out.getvalue())
|
||||
|
||||
def test_opening_the_library_must_not_change_its_schema(self):
|
||||
self.query_write("DROP INDEX idx_prompt_pairs_positive_only", ())
|
||||
PromptModel.reset_schema_cache()
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 2, out)
|
||||
self.assertIn("schema", out)
|
||||
self.assertIn("live code version", out)
|
||||
|
||||
def test_an_existing_backup_file_is_refused(self):
|
||||
stamp = "20261028_000000"
|
||||
Path(self.backups, f"prompts.db.pre-247-repair_{stamp}").write_bytes(b"x")
|
||||
before = dump(self.db_path)
|
||||
with unittest.mock.patch.object(tool, "_stamp", return_value=stamp):
|
||||
code, out = self.apply()
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("already exists", out)
|
||||
|
||||
def test_a_backup_that_does_not_match_is_refused(self):
|
||||
real = tool.backup_database
|
||||
|
||||
def short(source, target):
|
||||
real(source, target)
|
||||
conn = sqlite3.connect(target)
|
||||
try:
|
||||
with conn:
|
||||
conn.execute("DELETE FROM image_prompt_pairs")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
before = dump(self.db_path)
|
||||
with unittest.mock.patch.object(tool, "backup_database", short):
|
||||
code, out = self.apply()
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("backup", out)
|
||||
|
||||
def test_a_changed_target_text_is_refused(self):
|
||||
plan = self.plan()
|
||||
for item in plan["rows"]:
|
||||
if item["row_id"] == self.rows["A"]:
|
||||
item["target_text"] = "something else"
|
||||
self.write_plan(plan)
|
||||
before = dump(self.db_path)
|
||||
code, out = self.apply()
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("target_text", out)
|
||||
|
||||
def test_an_orphan_gone_or_changed_at_apply_is_refused(self):
|
||||
plan = self.plan()
|
||||
plan["delete_orphans"] = True
|
||||
plan["orphans"][0]["text_sha256"] = "0" * 64
|
||||
self.write_plan(plan)
|
||||
before = dump(self.db_path)
|
||||
code, out = self.apply()
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("gone or text changed", out)
|
||||
|
||||
def test_any_error_rolls_back_and_names_the_backup(self):
|
||||
before = dump(self.db_path)
|
||||
with unittest.mock.patch.object(
|
||||
tool, "_relink", side_effect=RuntimeError("disk on fire")
|
||||
):
|
||||
code, out = self.apply()
|
||||
self.assert_unchanged(before, code, out)
|
||||
self.assertIn("rolled back", out)
|
||||
self.assertIn("disk on fire", out)
|
||||
self.assertIn(self.backups, out)
|
||||
|
||||
def test_a_failed_integrity_check_after_writing_exits_1(self):
|
||||
real = tool.counts
|
||||
seen = []
|
||||
|
||||
def counts(path, readonly=True):
|
||||
found = real(path, readonly)
|
||||
if path == self.db_path:
|
||||
seen.append(path)
|
||||
if len(seen) > 1:
|
||||
return {**found, "integrity_check": "corrupt"}
|
||||
return found
|
||||
|
||||
with unittest.mock.patch.object(tool, "counts", counts):
|
||||
code, out = self.apply()
|
||||
self.assertEqual(code, 1, out)
|
||||
self.assertIn("restore", out)
|
||||
|
||||
|
||||
class BlockerTests(RepairFixture):
|
||||
"""Each reference that keeps a prompt from being deleted."""
|
||||
|
||||
def test_every_blocker(self):
|
||||
pid = self.orphan
|
||||
cases = {
|
||||
"linked images": None,
|
||||
"saved pairs": "INSERT INTO prompt_pairs(positive_prompt_id) VALUES(:id)",
|
||||
"recorded runs": "UPDATE execution_run_contexts "
|
||||
"SET positive_prompt_id=:id",
|
||||
"library state": "INSERT INTO prompt_library_state(prompt_id) "
|
||||
"VALUES(:id)",
|
||||
"user tags": "INSERT INTO prompt_tags(prompt_id, tag_id) "
|
||||
"SELECT :id, id FROM tags WHERE name='mine'",
|
||||
"usage or rating": "UPDATE prompts SET run_count=3 WHERE id=:id",
|
||||
"category": "UPDATE prompts SET category='portraits' WHERE id=:id",
|
||||
"notes": "UPDATE prompts SET notes='mine' WHERE id=:id",
|
||||
}
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
try:
|
||||
conn.execute("INSERT INTO tags(name) VALUES('mine')")
|
||||
conn.commit()
|
||||
for name, sql in cases.items():
|
||||
with self.subTest(name):
|
||||
conn.execute("SAVEPOINT blocker")
|
||||
if sql is None:
|
||||
unused = {pid}
|
||||
else:
|
||||
conn.execute(
|
||||
"DELETE FROM generated_images WHERE prompt_id=?", (pid,)
|
||||
)
|
||||
conn.execute(sql, {"id": pid})
|
||||
unused = {pid}
|
||||
found = tool.blockers(conn, pid, unused)
|
||||
self.assertTrue(any(item.startswith(name) for item in found), found)
|
||||
conn.execute("ROLLBACK TO blocker")
|
||||
conn.execute("RELEASE blocker")
|
||||
conn.execute("DELETE FROM generated_images WHERE prompt_id=?", (pid,))
|
||||
self.assertEqual(tool.blockers(conn, pid, {pid}), [])
|
||||
self.assertEqual(
|
||||
tool.blockers(conn, pid, set()), ["not unused by the app's rule"]
|
||||
)
|
||||
conn.rollback()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def _off_text_node(text, save=False):
|
||||
return {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": text, "save_to_library": save},
|
||||
}
|
||||
|
||||
|
||||
def _encode_from(source):
|
||||
return {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": source, "clip": ["1", 1]},
|
||||
}
|
||||
|
||||
|
||||
def off_behind_pass_through(branch=2):
|
||||
"""The dataset graph with ``branch``'s text coming from a switched-off
|
||||
PromptManagerText through a pass-through the #90 walk does not follow."""
|
||||
graph = shapes.shared_prefix()
|
||||
graph["50"] = _off_text_node(TEXTS[branch - 1])
|
||||
graph["51"] = {"class_type": "MyNode", "inputs": {"value": ["50", 0]}}
|
||||
graph[str(branch * 10)] = _encode_from(["51", 0])
|
||||
return graph
|
||||
|
||||
|
||||
def off_target(branch=2, save=False):
|
||||
"""``branch``'s text straight from a PromptManagerText whose switch is
|
||||
``save`` (False, or a link to a node the graph cannot read)."""
|
||||
graph = shapes.shared_prefix()
|
||||
graph["50"] = _off_text_node(TEXTS[branch - 1], save)
|
||||
graph[str(branch * 10)] = _encode_from(["50", 0])
|
||||
if isinstance(save, list):
|
||||
graph["52"] = {"class_type": "SomeBoolNode", "inputs": {}}
|
||||
return graph
|
||||
|
||||
|
||||
def off_negative_only():
|
||||
"""Every branch's negative from a switched-off PromptManagerText."""
|
||||
graph = shapes.shared_prefix()
|
||||
graph["50"] = _off_text_node(shapes.NEGATIVE)
|
||||
graph["3"] = _encode_from(["50", 0])
|
||||
return graph
|
||||
|
||||
|
||||
class SaveToLibraryTests(RecordedRuns, unittest.TestCase):
|
||||
"""#90: the repair never relinks a file to a switched-off node's text.
|
||||
|
||||
The plan decides each file through ``decide_file_prompt``; every answer
|
||||
but a positive from the recorded saver or the inspector is a skip, so a
|
||||
``not_saved`` or ``not_saved_wired`` file is never written. Each file is
|
||||
reported by a run without contexts (as a cached or unrecorded run is),
|
||||
so only the ladder can stop it. Saver "23" is branch 2.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self.addCleanup(shutil.rmtree, self.tmp, True)
|
||||
self.addCleanup(assert_closed, self, self.tmp)
|
||||
self.addCleanup(PromptModel.reset_schema_cache)
|
||||
self.output = os.path.join(self.tmp, "output")
|
||||
os.makedirs(os.path.join(self.output, "dataset"))
|
||||
self.db_path = os.path.join(self.tmp, "prompts.db")
|
||||
|
||||
def plan_for(self, graph):
|
||||
path = os.path.join(self.output, "dataset", NAME.format(1))
|
||||
png(path, graph)
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
wrong = db.save_prompt(
|
||||
text=TEXTS[0], prompt_hash=generate_prompt_hash(TEXTS[0])
|
||||
)
|
||||
db.link_image_to_prompt(wrong, path, metadata={})
|
||||
self.record_run(
|
||||
db, self.output, [("dataset", NAME.format(1))], [], nodes=["23"]
|
||||
)
|
||||
plan = tool.build_plan(db, self.output, self.db_path)
|
||||
finally:
|
||||
db.close_all()
|
||||
(item,) = plan["rows"]
|
||||
return item
|
||||
|
||||
def test_an_off_node_behind_a_pass_through_is_skipped(self):
|
||||
item = self.plan_for(off_behind_pass_through())
|
||||
self.assertEqual(item["action"], "skip")
|
||||
self.assertEqual(item["decision"], "not_saved/none")
|
||||
self.assertNotIn("target_text", item)
|
||||
|
||||
def test_an_off_node_in_the_recorded_run_target_is_skipped(self):
|
||||
cases = {
|
||||
"off": (False, "not_saved/none"),
|
||||
"wired": (["52", 0], "not_saved_wired/none"),
|
||||
}
|
||||
for name, (save, decision) in cases.items():
|
||||
with self.subTest(name):
|
||||
if os.path.exists(self.db_path):
|
||||
os.remove(self.db_path)
|
||||
item = self.plan_for(off_target(save=save))
|
||||
self.assertEqual(item["action"], "skip")
|
||||
self.assertEqual(item["decision"], decision)
|
||||
|
||||
def test_an_off_node_on_another_branch_still_skips_the_file(self):
|
||||
"""The ladder judges the file's whole graph: any switched-off text
|
||||
upstream of a positive makes the file not saved."""
|
||||
item = self.plan_for(off_target(branch=3))
|
||||
self.assertEqual(item["action"], "skip")
|
||||
self.assertEqual(item["decision"], "not_saved/none")
|
||||
|
||||
def test_an_off_node_wired_only_to_the_negative_still_repairs(self):
|
||||
item = self.plan_for(off_negative_only())
|
||||
self.assertEqual(item["action"], "relink")
|
||||
self.assertEqual(item["source"], "recorded-run")
|
||||
self.assertEqual(item["target_text"], TEXTS[1])
|
||||
|
||||
def test_a_switched_on_node_in_the_target_still_repairs(self):
|
||||
"""Regression guard: a switch that is on is no reason to skip."""
|
||||
item = self.plan_for(off_target(save=True))
|
||||
self.assertEqual(item["action"], "relink")
|
||||
self.assertEqual(item["target_text"], TEXTS[1])
|
||||
|
||||
|
||||
class ActionTests(unittest.TestCase):
|
||||
"""Only the recorded saver relinks; a certain step naming the linked
|
||||
prompt keeps it; the legacy floor and the rest are skipped."""
|
||||
|
||||
def test_action_by_decision(self):
|
||||
linked = generate_prompt_hash("linked")
|
||||
cases = [
|
||||
(Decision("positive", "other", source="recorded-run"), "relink"),
|
||||
(Decision("positive", "linked", source="recorded-run"), "keep"),
|
||||
(Decision("positive", "linked", source="inspector"), "keep"),
|
||||
(Decision("positive", "other", source="inspector"), "skip"),
|
||||
(Decision("positive", "linked", source="legacy-parser"), "skip"),
|
||||
(Decision("pair", "linked", source="resolver"), "skip"),
|
||||
(Decision("never"), "skip"),
|
||||
(Decision("not_saved"), "skip"),
|
||||
(Decision("not_saved_wired"), "skip"),
|
||||
(Decision("positive", "other", source="opted-out-negative"), "skip"),
|
||||
]
|
||||
for decision, action in cases:
|
||||
with self.subTest(decision.source, text=decision.text):
|
||||
self.assertEqual(tool.action_of(decision, linked), action)
|
||||
|
||||
|
||||
class CommandLineTests(unittest.TestCase):
|
||||
def test_running_probe_reports_a_listening_port(self):
|
||||
import socket
|
||||
|
||||
with socket.socket() as server:
|
||||
server.bind(("127.0.0.1", 0))
|
||||
server.listen(1)
|
||||
port = server.getsockname()[1]
|
||||
found = tool.comfyui_running("/nonexistent/comfy", port)
|
||||
self.assertTrue(any(str(port) in item for item in found))
|
||||
|
||||
def test_running_probe_is_quiet_when_nothing_runs(self):
|
||||
import socket
|
||||
|
||||
with socket.socket() as probe:
|
||||
probe.bind(("127.0.0.1", 0))
|
||||
port = probe.getsockname()[1]
|
||||
self.assertEqual(tool.comfyui_running("/nonexistent/comfy", port), [])
|
||||
|
||||
def fake_proc(self, tmp):
|
||||
proc = Path(tmp, "proc")
|
||||
db = Path(tmp, "prompts.db")
|
||||
db.write_bytes(b"")
|
||||
Path(tmp, "prompts.db-wal").write_bytes(b"")
|
||||
comfy = Path(tmp, "comfy")
|
||||
comfy.mkdir()
|
||||
holder = proc / "125" / "fd"
|
||||
holder.mkdir(parents=True)
|
||||
os.symlink(Path(tmp, "prompts.db-wal"), holder / "3")
|
||||
(proc / "125" / "cmdline").write_bytes(b"python\0main.py\0")
|
||||
os.symlink(comfy, proc / "125" / "cwd")
|
||||
(proc / "123").mkdir() # unreadable: no fd, cmdline or cwd
|
||||
(proc / "self").mkdir() # not a pid
|
||||
return proc, db, comfy
|
||||
|
||||
@unittest.skipIf(os.name == "nt", "symlinks need privileges on Windows")
|
||||
def test_proc_scan_finds_holders_and_skips_unreadable_entries(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
proc, db, comfy = self.fake_proc(tmp)
|
||||
found = tool.open_by_others(str(db), proc=str(proc), own_pid=1)
|
||||
self.assertEqual(len(found), 1)
|
||||
self.assertIn("125", found[0])
|
||||
self.assertEqual(
|
||||
tool.open_by_others(str(db), proc=str(proc), own_pid=125), []
|
||||
)
|
||||
running = tool.comfyui_running(str(comfy), 1, proc=str(proc))
|
||||
self.assertTrue(any("125" in item for item in running), running)
|
||||
|
||||
def test_without_proc_the_scan_says_it_cannot_tell(self):
|
||||
self.assertEqual(
|
||||
tool.open_by_others("/x/prompts.db", proc="/nonexistent/proc"),
|
||||
[tool.NO_PROC],
|
||||
)
|
||||
|
||||
def test_missing_database_is_an_argument_error(self):
|
||||
with self.assertRaises(SystemExit), contextlib.redirect_stderr(io.StringIO()):
|
||||
tool.main(
|
||||
["/nonexistent/prompts.db", "--output-dir", "/tmp"], out=io.StringIO()
|
||||
)
|
||||
|
||||
def test_apply_needs_the_comfyui_folder(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
db = os.path.join(tmp, "prompts.db")
|
||||
Path(db).write_bytes(b"")
|
||||
with (
|
||||
self.assertRaises(SystemExit),
|
||||
contextlib.redirect_stderr(io.StringIO()),
|
||||
):
|
||||
tool.main([db, "--apply", "plan.json"], out=io.StringIO())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -33,6 +33,7 @@ from utils.execution_outputs import (
|
||||
snapshot_output_roots,
|
||||
)
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from py.config import GalleryConfig
|
||||
from utils.image_monitor import ImageGenerationHandler
|
||||
|
||||
STYLE = "windows" if os.name == "nt" else "posix"
|
||||
@@ -771,6 +772,487 @@ class TestMonitorHistoryRecheck(RecordedRuns, unittest.TestCase):
|
||||
run_attribution.notify_outputs_sealed("run", []) # no dead callback raises
|
||||
|
||||
|
||||
def folder_is_case_sensitive(folder):
|
||||
"""True when ``folder``'s volume tells names apart by case: the existing
|
||||
folder spelled with swapped case is not found (#476)."""
|
||||
head, name = os.path.split(folder)
|
||||
return not os.path.exists(os.path.join(head, name.swapcase()))
|
||||
|
||||
|
||||
class TestMonitorForeignRuns(RecordedRuns, unittest.TestCase):
|
||||
"""#247: outputs of workflows without a PromptManager node, in the monitor."""
|
||||
|
||||
def setUp(self):
|
||||
history_tests.HistoryDatabaseTests.setUp(self)
|
||||
self.db = history_tests.HistoryDatabaseTests.database(self)
|
||||
self.output = str(Path(self.temp.name) / "output")
|
||||
os.makedirs(os.path.join(self.output, "dataset"))
|
||||
self.tracker = make_tracker(self.db)
|
||||
self.handler = ImageGenerationHandler(self.db, self.tracker)
|
||||
self.handler.processing_delay = 0
|
||||
self.handler.settle_interval = 0.01
|
||||
self.addCleanup(close_handler, self, self.handler)
|
||||
self.metadata = {"prompt": shapes.shared_prefix()}
|
||||
saved = GalleryConfig.SAVE_FOREIGN_RUNS
|
||||
self.addCleanup(setattr, GalleryConfig, "SAVE_FOREIGN_RUNS", saved)
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS = True
|
||||
patches = (
|
||||
mock.patch(
|
||||
"utils.image_monitor.host_output_directory", return_value=self.output
|
||||
),
|
||||
mock.patch(
|
||||
"database.run_attribution.host_output_directory",
|
||||
return_value=self.output,
|
||||
),
|
||||
mock.patch.object(
|
||||
self.handler.metadata_extractor,
|
||||
"extract_metadata",
|
||||
side_effect=lambda path: self.metadata,
|
||||
),
|
||||
)
|
||||
for patch in patches:
|
||||
patch.start()
|
||||
self.addCleanup(patch.stop)
|
||||
|
||||
close_databases = history_tests.HistoryDatabaseTests.close_databases
|
||||
wait_for = TestMonitorHistoryRecheck.wait_for
|
||||
|
||||
def image(self, name, subfolder="dataset"):
|
||||
path = os.path.join(self.output, subfolder, name)
|
||||
Path(path).write_bytes(b"png")
|
||||
return path
|
||||
|
||||
def linked_text(self, path):
|
||||
row = (
|
||||
self.db.model.get_connection()
|
||||
.execute(
|
||||
"SELECT p.text FROM generated_images g "
|
||||
"JOIN prompts p ON p.id = g.prompt_id WHERE g.image_path = ?",
|
||||
(str(path),),
|
||||
)
|
||||
.fetchone()
|
||||
)
|
||||
return row[0] if row else None
|
||||
|
||||
def key(self, path):
|
||||
return self.handler._awaiting_key(path)
|
||||
|
||||
def prompt_count(self):
|
||||
connection = self.db.model.get_connection()
|
||||
return connection.execute("SELECT COUNT(*) FROM prompts").fetchone()[0]
|
||||
|
||||
def settled(self):
|
||||
return self.wait_for(lambda: self.handler.pending_count() == 0)
|
||||
|
||||
def test_each_dataset_file_links_to_its_branch_once_the_run_ends(self):
|
||||
"""CUJ 1: unlinked while the run is open, its own branch after the seal."""
|
||||
paths = [self.image(f"IMG_GEN_0000{n}_.png") for n in (1, 2)]
|
||||
for path in paths:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertEqual([self.linked_text(path) for path in paths], [None, None])
|
||||
self.assertEqual(
|
||||
self.handler.awaiting_foreign(), [self.key(path) for path in paths]
|
||||
)
|
||||
self.assertEqual(self.handler.awaiting_history(), [])
|
||||
self.record_run(
|
||||
self.db,
|
||||
self.output,
|
||||
[("dataset", os.path.basename(path)) for path in paths],
|
||||
[],
|
||||
nodes=["13", "23"],
|
||||
)
|
||||
self.assertTrue(self.wait_for(lambda: all(map(self.linked_text, paths))))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertEqual(
|
||||
[self.linked_text(path) for path in paths], list(shapes.BRANCH_TEXTS[:2])
|
||||
)
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
prompt = self.db.get_prompt_by_hash(
|
||||
generate_prompt_hash(shapes.BRANCH_TEXTS[0])
|
||||
)
|
||||
self.assertEqual(prompt["category"], "scanned")
|
||||
self.assertIn("Saved from a workflow run: IMG_GEN_00001_.png", prompt["notes"])
|
||||
|
||||
def test_a_run_sealed_before_the_file_settles_links_it_at_once(self):
|
||||
path = self.image("IMG_GEN_00003_.png")
|
||||
self.record_run(
|
||||
self.db, self.output, [("dataset", "IMG_GEN_00003_.png")], [], nodes=["33"]
|
||||
)
|
||||
self.handler.process_new_image(path)
|
||||
self.assertEqual(self.linked_text(path), shapes.BRANCH_TEXTS[2])
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
|
||||
def test_a_name_two_runs_reported_stays_unlinked(self):
|
||||
"""CUJ 3: the file was overwritten; no run proves its saver."""
|
||||
path = self.image("IMG_GEN_00004_.png")
|
||||
reported = [("dataset", "IMG_GEN_00004_.png")]
|
||||
self.record_run(self.db, self.output, reported, [], nodes=["13"])
|
||||
self.record_run(self.db, self.output, reported, [], item=1, nodes=["23"])
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
reference = {"folder_type": "output", "subfolder": "dataset"}
|
||||
reference["filename"] = "IMG_GEN_00004_.png"
|
||||
with self.assertLogs("prompt_manager.image_monitor", "INFO") as logs:
|
||||
self.handler._recheck_foreign(path, reference)
|
||||
self.assertIn(
|
||||
"No certain prompt for IMG_GEN_00004_.png after its run ended; "
|
||||
"left unlinked",
|
||||
"\n".join(logs.output),
|
||||
)
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
self.assertEqual(self.prompt_count(), 0)
|
||||
|
||||
def test_the_setting_off_stores_nothing_at_settle_or_after_the_run(self):
|
||||
"""CUJ 4: read for every file, so it applies without a restart."""
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS = False
|
||||
skipped = self.image("IMG_GEN_00005_.png")
|
||||
with self.assertLogs("prompt_manager.image_monitor", "INFO") as logs:
|
||||
self.handler.process_new_image(skipped)
|
||||
self.assertIn(
|
||||
"No PromptManager prompt in workflow, skipping image",
|
||||
"\n".join(logs.output),
|
||||
)
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS = True
|
||||
waiting = self.image("IMG_GEN_00006_.png")
|
||||
self.handler.process_new_image(waiting)
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [self.key(waiting)])
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS = False
|
||||
self.record_run(
|
||||
self.db, self.output, [("dataset", "IMG_GEN_00006_.png")], [], nodes=["13"]
|
||||
)
|
||||
self.assertTrue(self.wait_for(lambda: not self.handler.awaiting_foreign()))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertEqual(
|
||||
[self.linked_text(p) for p in (skipped, waiting)], [None, None]
|
||||
)
|
||||
self.assertEqual(self.prompt_count(), 0)
|
||||
|
||||
def test_a_plain_foreign_workflow_links_at_settle_and_counts_no_run(self):
|
||||
"""CUJ 2: the default profile decides; the tracker queue is untouched."""
|
||||
text = "a lighthouse on a cliff at dusk"
|
||||
self.metadata = {"prompt": sampled_graph(text, "blurry")}
|
||||
queued = self.db.save_prompt(
|
||||
text="queued item", prompt_hash=generate_prompt_hash("queued item")
|
||||
)
|
||||
self.tracker.set_current_prompt("queued item", {"prompt_id": queued})
|
||||
first, second = self.image("plain_00001_.png"), self.image("plain_00002_.png")
|
||||
self.handler.process_new_image(first)
|
||||
self.handler.process_new_image(second)
|
||||
self.assertEqual(
|
||||
[self.linked_text(first), self.linked_text(second)], [text, text]
|
||||
)
|
||||
prompt = self.db.get_prompt_by_hash(generate_prompt_hash(text))
|
||||
self.assertEqual(prompt["run_count"] or 0, 0)
|
||||
self.assertIsNone(prompt.get("last_used_at"))
|
||||
self.assertEqual(self.tracker.pop_next_prompt()["id"], queued)
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
|
||||
def test_the_sealed_reference_spells_the_file_as_history_recorded_it(self):
|
||||
"""Windows: the watcher's spelling differs in case from the run's report.
|
||||
|
||||
On a case-sensitive folder the two spellings are two files and the
|
||||
folded key joins them; on a case-insensitive one (Windows, default
|
||||
macOS) they already name one file, so only history's spelling and
|
||||
the watcher's differ (#476)."""
|
||||
seen = os.path.join(self.output, "dataset", "img_gen_00007_.png")
|
||||
Path(seen).write_bytes(b"png")
|
||||
recorded = os.path.join(self.output, "Dataset", "IMG_GEN_00007_.png")
|
||||
self.assertNotEqual(os.path.basename(recorded), os.path.basename(seen))
|
||||
if folder_is_case_sensitive(os.path.join(self.output, "dataset")):
|
||||
Path(self.output, "Dataset").mkdir()
|
||||
self.image("IMG_GEN_00007_.png", subfolder="Dataset")
|
||||
else:
|
||||
self.assertTrue(os.path.exists(recorded))
|
||||
folded = staticmethod(lambda path: os.path.realpath(path).lower())
|
||||
with mock.patch.object(ImageGenerationHandler, "_awaiting_key", folded):
|
||||
self.handler.process_new_image(seen)
|
||||
self.assertIsNone(self.linked_text(seen))
|
||||
self.record_run(
|
||||
self.db,
|
||||
self.output,
|
||||
[("Dataset", os.path.basename(recorded))],
|
||||
[],
|
||||
nodes=["43"],
|
||||
)
|
||||
self.assertTrue(self.wait_for(lambda: self.linked_text(seen)))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertEqual(self.linked_text(seen), shapes.BRANCH_TEXTS[3])
|
||||
|
||||
def test_a_full_foreign_set_never_evicts_a_waiting_keyless_file(self):
|
||||
from utils import image_monitor
|
||||
|
||||
keyless = self.image("bare_00001_.png", subfolder="")
|
||||
with mock.patch.object(
|
||||
self.handler.metadata_extractor, "extract_metadata", return_value=None
|
||||
):
|
||||
self.handler.process_new_image(keyless)
|
||||
paths = [self.image(f"IMG_GEN_1000{n}_.png") for n in range(3)]
|
||||
with (
|
||||
mock.patch.object(image_monitor, "MAX_AWAITING_HISTORY", 2),
|
||||
self.assertLogs("prompt_manager.image_monitor", "INFO") as logs,
|
||||
):
|
||||
for path in paths:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertEqual(
|
||||
self.handler.awaiting_foreign(), [self.key(p) for p in paths[1:]]
|
||||
)
|
||||
self.assertEqual(self.handler.awaiting_history(), [self.key(keyless)])
|
||||
self.assertIn(
|
||||
"Too many files wait for their run; IMG_GEN_10000_.png left unlinked",
|
||||
"\n".join(logs.output),
|
||||
)
|
||||
|
||||
def test_an_expired_foreign_entry_is_logged_and_left_unlinked(self):
|
||||
from utils import image_monitor
|
||||
|
||||
path = self.image("IMG_GEN_00008_.png")
|
||||
self.handler.process_new_image(path)
|
||||
later = time.monotonic() + image_monitor.AWAIT_HISTORY_SECONDS + 1
|
||||
with (
|
||||
mock.patch.object(image_monitor.time, "monotonic", return_value=later),
|
||||
self.assertLogs("prompt_manager.image_monitor", "INFO") as logs,
|
||||
):
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
self.assertIn(
|
||||
"No recorded run for IMG_GEN_00008_.png; left unlinked",
|
||||
"\n".join(logs.output),
|
||||
)
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
|
||||
def test_a_seal_while_the_pending_set_is_full_is_rechecked_later(self):
|
||||
from utils import image_monitor
|
||||
|
||||
path = self.image("IMG_GEN_00009_.png")
|
||||
self.handler.process_new_image(path)
|
||||
blocker = self.image("blocker.png", subfolder="")
|
||||
release = threading.Event()
|
||||
original = self.handler.process_new_image
|
||||
|
||||
def slow(image_path, prompt_snapshot=None):
|
||||
if image_path == blocker:
|
||||
release.wait(5)
|
||||
return None
|
||||
return original(image_path, prompt_snapshot=prompt_snapshot)
|
||||
|
||||
self.handler.process_new_image = slow
|
||||
with mock.patch.object(image_monitor, "MAX_PENDING_IMAGES", 1):
|
||||
self.assertTrue(self.handler.schedule(blocker))
|
||||
self.assertTrue(self.wait_for(lambda: self.handler.pending_count() == 1))
|
||||
self.record_run(
|
||||
self.db,
|
||||
self.output,
|
||||
[("dataset", "IMG_GEN_00009_.png")],
|
||||
[],
|
||||
nodes=["13"],
|
||||
)
|
||||
self.assertEqual(list(self.handler._foreign_due), [path])
|
||||
release.set()
|
||||
self.assertTrue(self.wait_for(lambda: self.linked_text(path)))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertEqual(self.linked_text(path), shapes.BRANCH_TEXTS[0])
|
||||
|
||||
def test_shutdown_forgets_waiting_and_due_foreign_files(self):
|
||||
path = self.image("IMG_GEN_00010_.png")
|
||||
self.handler.process_new_image(path)
|
||||
self.handler._foreign_due[path] = {"filename": "IMG_GEN_00010_.png"}
|
||||
self.handler.shutdown()
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
self.assertEqual(list(self.handler._foreign_due), [])
|
||||
|
||||
def link_count(self, path):
|
||||
connection = self.db.model.get_connection()
|
||||
return connection.execute(
|
||||
"SELECT COUNT(*) FROM generated_images WHERE image_path = ?", (str(path),)
|
||||
).fetchone()[0]
|
||||
|
||||
def test_a_file_linked_to_another_prompt_never_gets_a_second_link(self):
|
||||
"""One prompt per file: the pair path and the positive path both refuse."""
|
||||
other = self.db.save_prompt(
|
||||
"an earlier prompt", generate_prompt_hash("an earlier prompt")
|
||||
)
|
||||
plain = self.image("plain_00004_.png")
|
||||
self.db.link_image_to_prompt(other, plain)
|
||||
self.metadata = {"prompt": sampled_graph("a quiet harbor", "blurry")}
|
||||
with self.assertLogs("prompt_manager.image_monitor", "INFO") as logs:
|
||||
self.handler.process_new_image(plain)
|
||||
self.assertIn("Foreign run: conflict", "\n".join(logs.output))
|
||||
dataset = self.image("IMG_GEN_00011_.png")
|
||||
self.db.link_image_to_prompt(other, dataset)
|
||||
self.metadata = {"prompt": shapes.shared_prefix()}
|
||||
self.record_run(
|
||||
self.db, self.output, [("dataset", "IMG_GEN_00011_.png")], [], nodes=["13"]
|
||||
)
|
||||
self.handler.process_new_image(dataset)
|
||||
for path in (plain, dataset):
|
||||
self.assertEqual(self.linked_text(path), "an earlier prompt")
|
||||
self.assertEqual(self.link_count(path), 1)
|
||||
for text in ("a quiet harbor", shapes.BRANCH_TEXTS[0]):
|
||||
self.assertIsNone(self.db.get_prompt_by_hash(generate_prompt_hash(text)))
|
||||
|
||||
def test_the_file_size_is_read_only_when_the_ladder_needs_it(self):
|
||||
"""PR 1's lazy size: a foreign file decided without a store reads none."""
|
||||
# Foreign (a core encoder only) and no API graph: never, at step 3
|
||||
encoder = {"type": "CLIPTextEncode", "inputs": [], "widgets_values": ["x"]}
|
||||
self.metadata = {"text_encoder_nodes": [encoder]}
|
||||
path = self.image("bare_00002_.png", subfolder="")
|
||||
with mock.patch("utils.foreign_runs.os.path.getsize") as size:
|
||||
self.handler.process_new_image(path)
|
||||
size.assert_not_called()
|
||||
|
||||
def untraceable(self):
|
||||
"""Unresolved and only the legacy parser names a text (an unknown node)."""
|
||||
graph = sampled_graph("unused", "blurry")
|
||||
graph["1"]["inputs"]["text"] = ["30", 0]
|
||||
graph["30"] = {
|
||||
"class_type": "VendorPromptMaker",
|
||||
"inputs": {"prompt": "a misty valley at dawn"},
|
||||
}
|
||||
return {"prompt": graph}
|
||||
|
||||
def test_no_record_at_settle_keeps_the_file_waiting_for_the_legacy_floor(self):
|
||||
"""Review: a lookup that found no record is not the run's answer yet."""
|
||||
self.metadata = self.untraceable()
|
||||
path = self.image("vendor_00001_.png")
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [self.key(path)])
|
||||
self.record_run(self.db, self.output, [("dataset", "vendor_00001_.png")], [])
|
||||
self.assertTrue(self.wait_for(lambda: self.linked_text(path)))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertEqual(self.linked_text(path), "a misty valley at dawn")
|
||||
|
||||
def test_a_record_found_at_settle_without_a_certain_prompt_ends_the_wait(self):
|
||||
"""Review: sealed before settle, still ambiguous: no seal is coming."""
|
||||
path = self.image("IMG_GEN_00012_.png")
|
||||
# One run reported it from two savers: a record, but no saver
|
||||
reported = [("dataset", "IMG_GEN_00012_.png")] * 2
|
||||
self.record_run(self.db, self.output, reported, [], nodes=["13", "23"])
|
||||
with self.assertLogs("prompt_manager.image_monitor", "INFO") as logs:
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIn(
|
||||
"No certain prompt for IMG_GEN_00012_.png; left unlinked",
|
||||
"\n".join(logs.output),
|
||||
)
|
||||
self.assertEqual(self.handler.awaiting_foreign(), [])
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
|
||||
def test_monitoring_saved_off_stops_foreign_writes_at_settle_and_recheck(self):
|
||||
"""Review: the observer runs until a restart; nothing is written."""
|
||||
enabled = GalleryConfig.MONITORING_ENABLED
|
||||
self.addCleanup(setattr, GalleryConfig, "MONITORING_ENABLED", enabled)
|
||||
GalleryConfig.MONITORING_ENABLED = False
|
||||
self.metadata = {"prompt": sampled_graph("a quiet harbor", "blurry")}
|
||||
plain = self.image("plain_00005_.png")
|
||||
self.handler.process_new_image(plain)
|
||||
self.assertIsNone(self.linked_text(plain))
|
||||
GalleryConfig.MONITORING_ENABLED = True
|
||||
self.metadata = {"prompt": shapes.shared_prefix()}
|
||||
waiting = self.image("IMG_GEN_00013_.png")
|
||||
self.handler.process_new_image(waiting)
|
||||
GalleryConfig.MONITORING_ENABLED = False
|
||||
reference = {"folder_type": "output", "subfolder": "dataset"}
|
||||
reference["filename"] = "IMG_GEN_00013_.png"
|
||||
self.record_run(
|
||||
self.db, self.output, [("dataset", reference["filename"])], [], nodes=["13"]
|
||||
)
|
||||
self.assertTrue(self.wait_for(lambda: not self.handler.awaiting_foreign()))
|
||||
self.assertTrue(self.settled())
|
||||
self.handler._recheck_foreign(waiting, reference)
|
||||
self.assertIsNone(self.linked_text(waiting))
|
||||
self.assertEqual(self.prompt_count(), 0)
|
||||
|
||||
def test_the_setting_is_read_again_just_before_the_store(self):
|
||||
self.metadata = {"prompt": sampled_graph("a quiet harbor", "blurry")}
|
||||
path = self.image("plain_00006_.png")
|
||||
real = __import__("utils.foreign_runs", fromlist=["x"]).decide_foreign
|
||||
|
||||
def decide_then_switch_off(*args, **kwargs):
|
||||
decision = real(*args, **kwargs)
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS = False
|
||||
return decision
|
||||
|
||||
with mock.patch(
|
||||
"utils.image_monitor.decide_foreign", side_effect=decide_then_switch_off
|
||||
):
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
self.assertEqual(self.prompt_count(), 0)
|
||||
|
||||
def test_a_seal_between_registering_and_the_settle_lookup_links_once(self):
|
||||
"""Regression guard (passes before the review fixes): one link only."""
|
||||
path = self.image("IMG_GEN_00014_.png")
|
||||
real = __import__("utils.foreign_runs", fromlist=["x"]).history_lookup
|
||||
|
||||
def sealed_first(db, image_path, reference=None):
|
||||
lookup = real(db, image_path, reference)
|
||||
|
||||
def run_then_look():
|
||||
if reference is None: # the settle lookup: the run seals now
|
||||
self.record_run(
|
||||
self.db,
|
||||
self.output,
|
||||
[("dataset", "IMG_GEN_00014_.png")],
|
||||
[],
|
||||
nodes=["13"],
|
||||
)
|
||||
return lookup()
|
||||
|
||||
return run_then_look
|
||||
|
||||
with mock.patch("utils.image_monitor.history_lookup", side_effect=sealed_first):
|
||||
self.assertTrue(self.handler.schedule(path))
|
||||
self.assertTrue(self.wait_for(lambda: self.linked_text(path)))
|
||||
self.assertTrue(self.settled())
|
||||
self.assertEqual(self.linked_text(path), shapes.BRANCH_TEXTS[0])
|
||||
self.assertEqual(self.link_count(path), 1)
|
||||
self.assertEqual(list(self.handler._foreign_due), [])
|
||||
|
||||
def test_a_drained_recheck_never_drops_a_newer_reference(self):
|
||||
path = self.image("IMG_GEN_00015_.png")
|
||||
older, newer = {"filename": "old"}, {"filename": "new"}
|
||||
self.handler._foreign_due[path] = older
|
||||
real = self.handler.schedule
|
||||
|
||||
def schedule(image_path, snapshot=None):
|
||||
# The run is sealed again while the older one is scheduled
|
||||
self.handler._foreign_due[image_path] = newer
|
||||
return real(image_path, snapshot)
|
||||
|
||||
self.handler.schedule = schedule
|
||||
with mock.patch.object(self.handler, "process_new_image"):
|
||||
self.handler._drain_history_due()
|
||||
self.assertIs(self.handler._foreign_due[path], newer)
|
||||
|
||||
def test_an_evicted_due_recheck_is_logged(self):
|
||||
from utils import image_monitor
|
||||
|
||||
self.handler.schedule = lambda *args, **kwargs: False
|
||||
with (
|
||||
mock.patch.object(image_monitor, "MAX_AWAITING_HISTORY", 1),
|
||||
self.assertLogs("prompt_manager.image_monitor", "INFO") as logs,
|
||||
):
|
||||
for name in ("a_00001_.png", "b_00001_.png"):
|
||||
self.handler._schedule_recheck(self.image(name), {"filename": name})
|
||||
self.assertIn(
|
||||
"Too many files wait for their run; a_00001_.png left unlinked",
|
||||
"\n".join(logs.output),
|
||||
)
|
||||
|
||||
def test_a_failed_store_is_logged_and_leaves_the_file_unlinked(self):
|
||||
self.metadata = {"prompt": sampled_graph("a quiet harbor", "blurry")}
|
||||
path = self.image("plain_00003_.png")
|
||||
with (
|
||||
mock.patch(
|
||||
"utils.foreign_runs.ingest_resolved_pair",
|
||||
side_effect=sqlite3.OperationalError("locked"),
|
||||
),
|
||||
self.assertLogs("prompt_manager.image_monitor", "ERROR"),
|
||||
):
|
||||
self.handler.process_new_image(path)
|
||||
self.assertIsNone(self.linked_text(path))
|
||||
|
||||
|
||||
class TestBoundedLookup(RecordedRuns, unittest.TestCase):
|
||||
"""M5: history lookups are batched, and the history schema is unchanged."""
|
||||
|
||||
|
||||
@@ -42,7 +42,14 @@ IMAGES_STATE = {
|
||||
"favorites": False,
|
||||
"collection": "",
|
||||
}
|
||||
LIBRARY_STATE = {"q": "", "category": "", "sort": "rating", "tags": []}
|
||||
LIBRARY_STATE = {
|
||||
"q": "",
|
||||
"category": "",
|
||||
"sort": "rating",
|
||||
"tags": [],
|
||||
"tag_mode": "untagged",
|
||||
"folder": "2026/Aug",
|
||||
}
|
||||
IMAGES_DEFAULTS = {
|
||||
"q": "",
|
||||
"category": "",
|
||||
@@ -54,7 +61,14 @@ IMAGES_DEFAULTS = {
|
||||
"favorites": False,
|
||||
"collection": "",
|
||||
}
|
||||
LIBRARY_DEFAULTS = {"q": "", "category": "", "sort": "newest", "tags": []}
|
||||
LIBRARY_DEFAULTS = {
|
||||
"q": "",
|
||||
"category": "",
|
||||
"sort": "newest",
|
||||
"tags": [],
|
||||
"tag_mode": "all",
|
||||
"folder": "",
|
||||
}
|
||||
# A row saved by an older build: no favorites key, a retired key, a sort
|
||||
# that is no longer offered and an over-long folder.
|
||||
OLD_IMAGES_STATE = json.dumps(
|
||||
@@ -228,9 +242,17 @@ class TestSavedViewsApi(ImageAPITestCase):
|
||||
favorite = await self.create(state={"favorites": True})
|
||||
self.assertIs(favorite["state"]["favorites"], True)
|
||||
for sort in ("newest", "oldest", "name", "rating"):
|
||||
await self.create(page="library", state={"sort": sort, "tags": ["a b"]})
|
||||
await self.create(
|
||||
page="library",
|
||||
state={"sort": sort, "tags": ["a b"], "tag_mode": "all", "folder": ""},
|
||||
)
|
||||
for mode in ("untagged", "any"):
|
||||
view = await self.create(
|
||||
page="library", state={"tags": [], "tag_mode": mode}
|
||||
)
|
||||
self.assertEqual(view["state"]["tag_mode"], mode)
|
||||
_, listed = await self.send("GET")
|
||||
self.assertEqual(len(listed["views"]), 20)
|
||||
self.assertEqual(len(listed["views"]), 22)
|
||||
|
||||
async def test_write_requires_the_collection_key(self):
|
||||
# #331 D140: a view saved on a collection page keeps the collection id.
|
||||
@@ -675,6 +697,32 @@ class TestSavedViewsSchema(unittest.TestCase):
|
||||
)
|
||||
self.assertEqual(normalize_state("history", {}), {})
|
||||
|
||||
def test_library_tag_mode_and_folder_are_kept_with_defaults_for_old_rows(self):
|
||||
# A row from before #444 has neither key.
|
||||
old = {"q": "", "category": "", "sort": "rating", "tags": ["a"]}
|
||||
self.assertEqual(normalize_state("library", old), {**LIBRARY_DEFAULTS, **old})
|
||||
self.assertEqual(
|
||||
normalize_state("library", {"tag_mode": "or", "folder": 3}),
|
||||
LIBRARY_DEFAULTS,
|
||||
)
|
||||
for mode in ("all", "any", "untagged"):
|
||||
self.assertEqual(
|
||||
normalize_state("library", {"tag_mode": mode})["tag_mode"], mode
|
||||
)
|
||||
with self.assertRaises(SavedViewInvalid):
|
||||
encode_state("library", {**LIBRARY_STATE, "tag_mode": "or"})
|
||||
with self.assertRaises(SavedViewInvalid):
|
||||
encode_state("library", {**LIBRARY_STATE, "folder": "x" * 1025})
|
||||
|
||||
def test_library_untagged_is_never_paired_with_tags(self):
|
||||
# Never written by this build: the tags win, as the listing refuses both.
|
||||
self.assertEqual(
|
||||
normalize_state("library", {"tags": ["a"], "tag_mode": "untagged"}),
|
||||
{**LIBRARY_DEFAULTS, "tags": ["a"]},
|
||||
)
|
||||
with self.assertRaises(SavedViewInvalid):
|
||||
encode_state("library", {**LIBRARY_STATE, "tags": ["a"]})
|
||||
|
||||
def test_validation_errors_are_saved_view_invalid(self):
|
||||
for call in (
|
||||
lambda: validate_name(""),
|
||||
|
||||
@@ -323,6 +323,43 @@ class TestMonitoringSetting(SourcesTestCase):
|
||||
self.assertIs(GalleryConfig.MONITORING_ENABLED, False)
|
||||
|
||||
|
||||
class TestSaveForeignRunsSetting(SourcesTestCase):
|
||||
"""#247 D-Q4: "Save prompts from other workflows", applied live."""
|
||||
|
||||
async def asyncSetUp(self):
|
||||
await super().asyncSetUp()
|
||||
saved = GalleryConfig.SAVE_FOREIGN_RUNS
|
||||
self.addCleanup(setattr, GalleryConfig, "SAVE_FOREIGN_RUNS", saved)
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS = True
|
||||
|
||||
async def test_read_and_write_published_live_without_a_restart(self):
|
||||
self.assertIs((await self._get())["settings"]["save_foreign_runs"], True)
|
||||
self._install_monitor(FakeMonitor([self.output_dir]))
|
||||
status, data = await self._post(
|
||||
"/prompt_manager/settings", {"save_foreign_runs": False}
|
||||
)
|
||||
self.assertEqual(status, 200, data)
|
||||
self.assertTrue(data["persisted"])
|
||||
self.assertEqual(data["restart_required_fields"], [])
|
||||
saved = self._read_config()
|
||||
self.assertIs(saved["gallery"]["monitoring"]["save_foreign_runs"], False)
|
||||
self.assertIs(GalleryConfig.SAVE_FOREIGN_RUNS, False)
|
||||
self.assertIs((await self._get())["settings"]["save_foreign_runs"], False)
|
||||
|
||||
async def test_only_true_or_false_is_accepted(self):
|
||||
self._write_config({"custom": {"keep": True}})
|
||||
before = self._config_bytes()
|
||||
for value in ("false", 0, None, []):
|
||||
with self.subTest(value=value):
|
||||
status, data = await self._post(
|
||||
"/prompt_manager/settings", {"save_foreign_runs": value}
|
||||
)
|
||||
self.assertEqual(status, 400, data)
|
||||
self.assertIn("save_foreign_runs must be true or false", data["error"])
|
||||
self.assertEqual(self._config_bytes(), before)
|
||||
self.assertIs(GalleryConfig.SAVE_FOREIGN_RUNS, True)
|
||||
|
||||
|
||||
class TestSourceChangeHold(SourcesTestCase):
|
||||
async def test_root_save_refused_while_scan_job_runs(self):
|
||||
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.a)]
|
||||
|
||||
@@ -364,6 +364,162 @@ class TestVocabularyReads(unittest.TestCase):
|
||||
self.assertTrue(conn.in_transaction)
|
||||
|
||||
|
||||
class _ReadFixture(unittest.TestCase):
|
||||
"""TestVocabularyReads' database and helpers, without re-running its tests."""
|
||||
|
||||
setUp = TestVocabularyReads.setUp
|
||||
tearDown = TestVocabularyReads.tearDown
|
||||
save = TestVocabularyReads.save
|
||||
read = TestVocabularyReads.read
|
||||
add_unused = TestVocabularyReads.add_unused
|
||||
all_tags = TestVocabularyReads.all_tags
|
||||
|
||||
|
||||
class TestVocabularyTagSearchAndSort(_ReadFixture):
|
||||
"""#444: tag search, tag sort and the untagged prompts count."""
|
||||
|
||||
def names(self, **options):
|
||||
rows, after = [], None
|
||||
while True:
|
||||
page = self.read("list_vocabulary_tags", limit=2, after=after, **options)
|
||||
rows.extend(row["name"] for row in page["tags"])
|
||||
if not page["has_more"]:
|
||||
return rows
|
||||
after = page["next_key"]
|
||||
|
||||
def seed_counts(self):
|
||||
self.save("One", tags=["beta", "Alpha"])
|
||||
self.save("Two", tags=["beta", "gamma"])
|
||||
self.save("Three", tags=["beta"])
|
||||
self.add_unused("delta", "100%_off", "100x-off")
|
||||
|
||||
def test_search_is_a_case_insensitive_literal_substring(self):
|
||||
self.seed_counts()
|
||||
self.assertEqual(self.names(search="ALP"), ["Alpha"])
|
||||
self.assertEqual(self.names(search="%"), ["100%_off"])
|
||||
self.assertEqual(self.names(search="_"), ["100%_off"])
|
||||
self.assertEqual(
|
||||
self.names(search=" a "), ["beta", "Alpha", "gamma", "delta"]
|
||||
)
|
||||
page = self.read("list_vocabulary_tags", search="a")
|
||||
self.assertEqual(page["total"], 4)
|
||||
self.assertEqual(self.read("list_vocabulary_tags", search="")["total"], 6)
|
||||
|
||||
def test_each_sort_orders_rows_and_continues_without_gaps(self):
|
||||
self.seed_counts()
|
||||
self.assertEqual(
|
||||
self.names(), ["beta", "Alpha", "gamma", "delta", "100%_off", "100x-off"]
|
||||
)
|
||||
self.assertEqual(self.names(sort="created"), self.names())
|
||||
self.assertEqual(
|
||||
self.names(sort="name"),
|
||||
["100%_off", "100x-off", "Alpha", "beta", "delta", "gamma"],
|
||||
)
|
||||
self.assertEqual(
|
||||
self.names(sort="name-desc"),
|
||||
["gamma", "delta", "beta", "Alpha", "100x-off", "100%_off"],
|
||||
)
|
||||
# Ties keep creation order: Alpha and gamma (1), then the unused tags (0).
|
||||
self.assertEqual(
|
||||
self.names(sort="prompts"),
|
||||
["beta", "Alpha", "gamma", "delta", "100%_off", "100x-off"],
|
||||
)
|
||||
self.assertEqual(
|
||||
self.names(sort="prompts-asc"),
|
||||
["delta", "100%_off", "100x-off", "Alpha", "gamma", "beta"],
|
||||
)
|
||||
self.assertEqual(
|
||||
self.names(sort="name", search="a"), ["Alpha", "beta", "delta", "gamma"]
|
||||
)
|
||||
|
||||
def test_count_sort_uses_the_row_counts(self):
|
||||
self.save("Parent", tags=["direct"])
|
||||
self.db.save_prompt_pair("Owner", "Noise", tags=["pair"])
|
||||
self.db.save_prompt_pair("Owner two", "Noise two", tags=["pair"])
|
||||
archived = self.db.save_prompt_pair("Gone", "Gone noise", tags=["archived"])
|
||||
self.db.archive_prompt_pair(archived)
|
||||
page = self.read("list_vocabulary_tags", sort="prompts")
|
||||
self.assertEqual(
|
||||
[(row["name"], row["prompt_count"]) for row in page["tags"]],
|
||||
[("pair", 2), ("direct", 1), ("archived", 0)],
|
||||
)
|
||||
|
||||
def test_invalid_search_sort_and_keys_are_refused(self):
|
||||
self.seed_counts()
|
||||
for options in (
|
||||
{"sort": "count"},
|
||||
{"sort": None},
|
||||
{"search": "x" * 201},
|
||||
{"search": 3},
|
||||
{"sort": "name", "after": 3},
|
||||
{"sort": "name", "after": ["a"]},
|
||||
{"sort": "prompts", "after": ["a", 1]},
|
||||
{"sort": "created", "after": ["a", 1]},
|
||||
):
|
||||
with self.assertRaises(ValueError, msg=options):
|
||||
self.read("list_vocabulary_tags", **options)
|
||||
|
||||
def test_untagged_count_is_every_parent_no_tag_browse_returns(self):
|
||||
self.save("Plain")
|
||||
self.save("Tagged", tags=["t"])
|
||||
self.db.save_prompt_pair("Pair tagged", "Noise", tags=["p"])
|
||||
self.db.save_prompt_pair("Pair plain", "Noise plain")
|
||||
archived = self.db.save_prompt_pair("Archived tag", "Old", tags=["old"])
|
||||
self.db.archive_prompt_pair(archived)
|
||||
self.add_unused("unused")
|
||||
page = self.read("list_vocabulary_tags", search="zzz")
|
||||
untagged = self.db.list_library(tag_mode="untagged")
|
||||
self.assertEqual(
|
||||
sorted(item["text"] for item in untagged["items"]),
|
||||
["Archived tag", "Pair plain", "Plain"],
|
||||
)
|
||||
self.assertEqual(page["untagged_count"], untagged["total"])
|
||||
self.assertEqual(page["untagged_count"], 3)
|
||||
tagged = set()
|
||||
for row in self.all_tags():
|
||||
tagged |= {
|
||||
item["id"] for item in self.db.list_library(tags=[row["name"]])["items"]
|
||||
}
|
||||
everyone = {item["id"] for item in self.db.list_library()["items"]}
|
||||
self.assertEqual({item["id"] for item in untagged["items"]}, everyone - tagged)
|
||||
|
||||
|
||||
class TestLibraryTagMode(_ReadFixture):
|
||||
def test_untagged_mode_combines_with_text_and_category(self):
|
||||
self.save("Red plain", category="Colors")
|
||||
self.save("Blue plain")
|
||||
self.save("Red tagged", category="Colors", tags=["red"])
|
||||
self.assertEqual(
|
||||
[
|
||||
i["text"]
|
||||
for i in self.db.list_library(text="red", tag_mode="untagged")["items"]
|
||||
],
|
||||
["Red plain"],
|
||||
)
|
||||
self.assertEqual(
|
||||
[
|
||||
i["text"]
|
||||
for i in self.db.list_library(category="Colors", tag_mode="untagged")[
|
||||
"items"
|
||||
]
|
||||
],
|
||||
["Red plain"],
|
||||
)
|
||||
self.assertEqual(
|
||||
self.db.list_library(tag_mode="all")["total"],
|
||||
self.db.list_library()["total"],
|
||||
)
|
||||
|
||||
def test_untagged_mode_refuses_tags_and_unknown_modes(self):
|
||||
for options in (
|
||||
{"tag_mode": "untagged", "tags": ["x"]},
|
||||
{"tag_mode": "none"},
|
||||
{"tag_mode": None},
|
||||
):
|
||||
with self.assertRaises(ValueError, msg=options):
|
||||
self.db.list_library(**options)
|
||||
|
||||
|
||||
_TAG_TABLES = ('"tags"', '"prompt_tags"', '"prompt_pair_tags"', '"sqlite_sequence"')
|
||||
|
||||
|
||||
|
||||
Vendored
+7
-3
@@ -212,11 +212,15 @@ test('every browser job sets PM_TEST_WORKERS and runs in the pinned image', () =
|
||||
assert.match(mergeJob(), PINNED_IMAGE, 'the merge sees the same workspace paths as the shards');
|
||||
});
|
||||
|
||||
test('page workflows run each browser in two shards', () => {
|
||||
// #473: two shards split by test count put the slow concept specs in shard 1, which reached the 18-minute step limit.
|
||||
test('page workflows run each browser in three shards', () => {
|
||||
const workflows = allJobs.get('browser-workflows');
|
||||
assert.deepEqual(matrixList(workflows, 'browser'), ['firefox', 'webkit']);
|
||||
assert.deepEqual(matrixList(workflows, 'shard'), ['1', '2']);
|
||||
assert.match(workflows, /test:workflows -- --project \$\{\{ matrix\.browser \}\} --shard \$\{\{ matrix\.shard \}\}\/2/);
|
||||
assert.deepEqual(matrixList(workflows, 'shard'), ['1', '2', '3']);
|
||||
assert.match(workflows, /test:workflows -- --project \$\{\{ matrix\.browser \}\} --shard \$\{\{ matrix\.shard \}\}\/3/);
|
||||
assert.match(workflows, /^ {4}name: page workflows \(\$\{\{ matrix\.browser \}\}, shard \$\{\{ matrix\.shard \}\}\/3\)$/m);
|
||||
assert.match(workflows, /name: Page workflows in \$\{\{ matrix\.browser \}\}, shard \$\{\{ matrix\.shard \}\}\/3$/m);
|
||||
assert.doesNotMatch(workflows, /\/2\b/, 'no step still names two shards');
|
||||
assert.match(workflows, /^ {6}fail-fast: false$/m);
|
||||
assert.match(workflows, /name: browser-traces-\$\{\{ matrix\.browser \}\}-shard-\$\{\{ matrix\.shard \}\}/);
|
||||
});
|
||||
|
||||
Vendored
+3
-3
@@ -36,14 +36,14 @@ test('execution script discovers only the new matrix and preserves all 78 origin
|
||||
// + the fresh-host library location case, + #342 image attribution, + #343 linked text,
|
||||
// + #332 image category routes on a generated output (image-attribution.spec.mjs, 2 projects),
|
||||
// + #347 cached runtime-linked runs: five node kinds and eight one-renderer cases (execution-text-link.spec.mjs, 2 projects),
|
||||
// + #90 Save to library off (five node types)
|
||||
assert.match(executions, /Total: 128 tests in 5 files/);
|
||||
// + #90 Save to library off (five node types), + #247 foreign workflows (image-attribution.spec.mjs, 2 tests x 2 projects)
|
||||
assert.match(executions, /Total: 132 tests in 5 files/);
|
||||
for (const project of ['classic-start', 'nodes2-start']) {
|
||||
const cases = executions.split('\n').filter(line => line.includes(`[${project}]`));
|
||||
assert.equal(cases.filter(line => line.includes('execution-history.spec.mjs')).length, 15);
|
||||
assert.equal(cases.filter(line => line.includes('execution-outputs.spec.mjs')).length, 8);
|
||||
assert.equal(cases.filter(line => line.includes('library-location.spec.mjs')).length, 1);
|
||||
assert.equal(cases.filter(line => line.includes('image-attribution.spec.mjs')).length, 3);
|
||||
assert.equal(cases.filter(line => line.includes('image-attribution.spec.mjs')).length, 5);
|
||||
assert.equal(cases.filter(line => line.includes('execution-text-link.spec.mjs')).length, 37);
|
||||
}
|
||||
assert.doesNotMatch(executions, /prompt-manager.spec/);
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Is ComfyUI stopped? The checks a library repair runs before writing (#247).
|
||||
|
||||
``comfyui_running`` looks for ``main.py`` processes of one ComfyUI folder
|
||||
and a listener on its port. ``open_by_others`` lists the processes that hold
|
||||
the library file, its ``-wal`` or its ``-shm`` open. Both read ``/proc``
|
||||
(Linux); without it ``open_by_others`` answers ``[NO_PROC]``: it cannot
|
||||
tell, and the caller refuses unless the user says ComfyUI is stopped.
|
||||
Entries that cannot be read (another user's process, one that just exited)
|
||||
are skipped.
|
||||
"""
|
||||
|
||||
import os
|
||||
import socket
|
||||
|
||||
NO_PROC = (
|
||||
"cannot list the processes that hold the library open (no /proc): stop "
|
||||
"ComfyUI and pass --i-stopped-comfyui"
|
||||
)
|
||||
|
||||
|
||||
def _same(path):
|
||||
return os.path.normcase(os.path.realpath(path))
|
||||
|
||||
|
||||
def _pids(proc, own_pid=None):
|
||||
try:
|
||||
names = os.listdir(proc)
|
||||
except OSError:
|
||||
return []
|
||||
return [name for name in names if name.isdigit() and name != str(own_pid)]
|
||||
|
||||
|
||||
def comfyui_running(comfyui_dir, port, proc="/proc"):
|
||||
"""``main.py`` processes running from ``comfyui_dir`` and a listener on
|
||||
``port`` (127.0.0.1), each as one line; empty when none is found."""
|
||||
found = []
|
||||
folder = _same(comfyui_dir)
|
||||
for pid in _pids(proc):
|
||||
try:
|
||||
with open(os.path.join(proc, pid, "cmdline"), "rb") as stream:
|
||||
parts = stream.read().split(b"\0")
|
||||
cwd = _same(os.readlink(os.path.join(proc, pid, "cwd")))
|
||||
except OSError:
|
||||
continue
|
||||
if any(part.endswith(b"main.py") for part in parts) and (
|
||||
cwd == folder or any(folder.encode() in part for part in parts)
|
||||
):
|
||||
found.append(f"process {pid} (main.py in {cwd})")
|
||||
with socket.socket() as probe:
|
||||
probe.settimeout(1)
|
||||
if probe.connect_ex(("127.0.0.1", port)) == 0:
|
||||
found.append(f"something listens on port {port}")
|
||||
return found
|
||||
|
||||
|
||||
def open_by_others(database, proc="/proc", own_pid=None):
|
||||
"""Processes other than ``own_pid`` (default: this one) holding
|
||||
``database``, its ``-wal`` or ``-shm`` open; ``[NO_PROC]`` without /proc."""
|
||||
if not os.path.isdir(proc):
|
||||
return [NO_PROC]
|
||||
own_pid = os.getpid() if own_pid is None else own_pid
|
||||
targets = {_same(database + suffix): suffix for suffix in ("", "-wal", "-shm")}
|
||||
found = []
|
||||
for pid in _pids(proc, own_pid):
|
||||
fd_dir = os.path.join(proc, pid, "fd")
|
||||
try:
|
||||
fds = os.listdir(fd_dir)
|
||||
except OSError:
|
||||
continue
|
||||
held = set()
|
||||
for fd in fds:
|
||||
try:
|
||||
target = os.readlink(os.path.join(fd_dir, fd))
|
||||
except OSError:
|
||||
continue
|
||||
target = target.replace(" (deleted)", "")
|
||||
if os.path.isabs(target) and _same(target) in targets:
|
||||
held.add(os.path.basename(database) + targets[_same(target)])
|
||||
if held:
|
||||
found.append(f"process {pid} has {', '.join(sorted(held))} open")
|
||||
return found
|
||||
@@ -0,0 +1,776 @@
|
||||
"""Repair the links the old scan gave foreign dataset files (#247), on review.
|
||||
|
||||
Before #247 the output scan linked each file of a foreign run (a workflow
|
||||
with no PromptManager node) through the legacy parser, so every file of a
|
||||
shared-prefix dataset run got the first branch's text. Recorded history
|
||||
knows which save node wrote each file; the shared ladder
|
||||
(``utils.file_prompt.decide_file_prompt``, scan policy) uses it
|
||||
(``recorded-run``) to name the right branch. This tool judges every
|
||||
``generated_images`` row of a file whose recording runs all have no
|
||||
PromptManager context:
|
||||
|
||||
``relink`` the ladder names a positive by the recorded saver, and its text
|
||||
is not the linked prompt's
|
||||
``keep`` the recorded saver or the inspector names the linked prompt
|
||||
``skip`` anything else (file gone, no single recorded saver, ambiguous,
|
||||
the legacy floor or another step decided)
|
||||
|
||||
Dry run (the default) judges a copy of the database (SQLite backup from a
|
||||
read-only connection) and writes a JSON plan and a Markdown list beside it
|
||||
(``<plan stem>.list.md``). It also rehearses the apply on a second copy, to
|
||||
list the prompts that would lose every link with their remaining
|
||||
references. Deleting those is OFF unless the dry run is given
|
||||
``--delete-orphans``; the choice is recorded in the plan and rehearsed on a
|
||||
third copy. The library is never written.
|
||||
|
||||
``--apply <plan.json>`` (ComfyUI stopped), only on the library the plan was
|
||||
made from (its real path, recorded in the plan):
|
||||
|
||||
1. refuses unless ``--comfyui-dir`` holds ``main.py``; refuses while a
|
||||
``main.py`` runs from it, anything listens on ``--port``, or any other
|
||||
process holds the library, its ``-wal`` or ``-shm`` open
|
||||
(``tools/comfyui_stopped.py``; without /proc only with
|
||||
``--i-stopped-comfyui``). ``--rehearsal`` skips the process and port
|
||||
check, never the open-file check, and never for that folder's library;
|
||||
2. saves ``<db>.pre-247-repair_<stamp>`` (or into ``--backup-dir``), never
|
||||
over an existing file, and refuses unless the backup passes
|
||||
``integrity_check`` with the library's counts. Opening the library must
|
||||
not change its schema (a newer checkout's migration): run the tool from a
|
||||
checkout at the live code version. The running check runs once more just
|
||||
before writing;
|
||||
3. in ONE ``BEGIN IMMEDIATE`` transaction: re-judges every planned row and
|
||||
refuses on any drift (row moved, prompt text changed, file changed, target
|
||||
stored since), creates the target prompts that do not exist (category
|
||||
``scanned``, tag ``auto-scanned``, note ``Auto-scanned from <first file>``),
|
||||
points each row at its target and drops its pair links (they named the old
|
||||
positive). A file that already has a row for the target keeps that row and
|
||||
loses the wrong one (UNIQUE(prompt_id, file_path)). Orphans are deleted only
|
||||
when the plan chose it and each is still unreferenced.
|
||||
|
||||
Any failed check rolls everything back and exits 2. Before/after table
|
||||
counts, ``integrity_check`` and ``foreign_key_check`` are printed.
|
||||
|
||||
python3 tools/repair_foreign_links.py <db> --output-dir <ComfyUI output> \\
|
||||
--plan repair_plan.json
|
||||
python3 tools/repair_foreign_links.py <db> --apply repair_plan.json \\
|
||||
--comfyui-dir <ComfyUI folder>
|
||||
|
||||
Run from a checkout at the live code version (after the merge: main) with
|
||||
its venv.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_ROOT))
|
||||
|
||||
from database.maintenance import ORPHANED_PROMPTS_SQL, _delete_ids # noqa: E402
|
||||
from database.operations import PromptDatabase # noqa: E402
|
||||
from database.pair_ingestion import ( # noqa: E402
|
||||
SCAN_MARKS,
|
||||
file_link_rows,
|
||||
mark_created,
|
||||
)
|
||||
from database.run_attribution import recorded_output # noqa: E402
|
||||
from py.api import PromptManagerAPI # noqa: E402
|
||||
from tools.comfyui_stopped import ( # noqa: E402
|
||||
NO_PROC,
|
||||
comfyui_running,
|
||||
open_by_others,
|
||||
)
|
||||
from tools.review_image_links import backup_database # noqa: E402
|
||||
from utils.execution_outputs import snapshot_output_roots # noqa: E402
|
||||
from utils.file_prompt import POSITIVE, Policy, decide_file_prompt # noqa: E402
|
||||
from utils.hashing import generate_prompt_hash # noqa: E402
|
||||
|
||||
PLAN_VERSION = 1
|
||||
COMFY_PORT = 8188
|
||||
LIVE_LIBRARY = ("user", "__promptmanager", "prompts.db")
|
||||
EXCERPT = 80
|
||||
COUNTED = (
|
||||
"prompts",
|
||||
"generated_images",
|
||||
"prompt_tags",
|
||||
"image_prompt_pairs",
|
||||
"workflow_documents",
|
||||
"prompt_pairs",
|
||||
"prompt_library_state",
|
||||
"execution_run_contexts",
|
||||
"tags",
|
||||
)
|
||||
# The ladder steps whose answer is certain enough to keep a link.
|
||||
KEEP_SOURCES = ("recorded-run", "inspector")
|
||||
# What must be unchanged since the dry run for a row to be written.
|
||||
DRIFT_KEYS = (
|
||||
"action",
|
||||
"file",
|
||||
"current_prompt_id",
|
||||
"current_text_sha256",
|
||||
"file_size",
|
||||
"file_mtime_ns",
|
||||
"target_text",
|
||||
"target_hash",
|
||||
"target_prompt_id",
|
||||
)
|
||||
# Locations reported only by runs without any PromptManager context.
|
||||
_FOREIGN_LOCATIONS = (
|
||||
"SELECT o.subfolder, o.filename FROM execution_run_outputs o "
|
||||
"WHERE o.folder_type = 'output' AND o.root_key = ? AND o.path_style = ? "
|
||||
"GROUP BY o.subfolder, o.filename "
|
||||
"HAVING SUM(EXISTS (SELECT 1 FROM execution_run_contexts c "
|
||||
"WHERE c.run_id = o.run_id)) = 0 "
|
||||
"ORDER BY o.subfolder, o.filename"
|
||||
)
|
||||
|
||||
|
||||
class Refused(Exception):
|
||||
"""A check failed: nothing is written."""
|
||||
|
||||
|
||||
def sha256(text):
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def metadata_reader():
|
||||
"""The scan's metadata extraction (PNG text chunks, ffprobe for videos)."""
|
||||
api = SimpleNamespace(logger=logging.getLogger("prompt_manager.repair_247"))
|
||||
return lambda path: PromptManagerAPI._extract_comfyui_metadata(api, path) or {}
|
||||
|
||||
|
||||
def foreign_rows(db, output_dir):
|
||||
"""The link rows of every file only foreign runs reported, by id."""
|
||||
style = "windows" if os.name == "nt" else "posix"
|
||||
root_key = snapshot_output_roots(style, output_dir, output_dir).output_key
|
||||
conn = db.model.get_connection()
|
||||
rows = {}
|
||||
for subfolder, filename in conn.execute(_FOREIGN_LOCATIONS, (root_key, style)):
|
||||
path = os.path.join(output_dir, subfolder, filename)
|
||||
rows.update((row["id"], row) for row in file_link_rows(conn, path))
|
||||
return [rows[key] for key in sorted(rows)]
|
||||
|
||||
|
||||
class Judge:
|
||||
"""One file's decision per path, for all of its rows."""
|
||||
|
||||
def __init__(self, db, output_dir, read=None):
|
||||
self.db = db
|
||||
self.output_dir = output_dir
|
||||
self.read = read or metadata_reader()
|
||||
self.decisions = {}
|
||||
|
||||
def decide(self, path, size):
|
||||
if path not in self.decisions:
|
||||
self.decisions[path] = decide_file_prompt(
|
||||
self.read(path),
|
||||
file_name=os.path.basename(path),
|
||||
file_size=size,
|
||||
in_output=True,
|
||||
lookup=lambda: recorded_output(self.db, path, self.output_dir),
|
||||
policy=Policy.SCAN,
|
||||
)
|
||||
return self.decisions[path]
|
||||
|
||||
def row(self, row):
|
||||
"""The plan entry of one ``generated_images`` row."""
|
||||
conn = self.db.model.get_connection()
|
||||
path = row["image_path"]
|
||||
(text,) = conn.execute(
|
||||
"SELECT text FROM prompts WHERE id = ?", (row["prompt_id"],)
|
||||
).fetchone()
|
||||
prompt_hash = generate_prompt_hash(text)
|
||||
item = {
|
||||
"row_id": row["id"],
|
||||
"file": path,
|
||||
"current_prompt_id": row["prompt_id"],
|
||||
"current_text_sha256": sha256(text),
|
||||
}
|
||||
try:
|
||||
stat = os.stat(path)
|
||||
except OSError:
|
||||
return {**item, "action": "skip", "reason": "file gone"}
|
||||
item.update(file_size=stat.st_size, file_mtime_ns=stat.st_mtime_ns)
|
||||
decision = self.decide(path, stat.st_size)
|
||||
action = action_of(decision, prompt_hash)
|
||||
if action == "skip":
|
||||
return {
|
||||
**item,
|
||||
"action": "skip",
|
||||
"reason": "not decided by the recorded saver",
|
||||
"decision": f"{decision.kind}/{decision.source}",
|
||||
}
|
||||
target_hash = generate_prompt_hash(decision.text)
|
||||
stored = conn.execute(
|
||||
"SELECT id FROM prompts WHERE hash = ?", (target_hash,)
|
||||
).fetchone()
|
||||
item.update(
|
||||
target_text=decision.text,
|
||||
target_hash=target_hash,
|
||||
target_prompt_id=stored[0] if stored else "new",
|
||||
action=action,
|
||||
source=decision.source,
|
||||
)
|
||||
return item
|
||||
|
||||
|
||||
def action_of(decision, linked_hash):
|
||||
"""``relink``, ``keep`` or ``skip`` for a row linked to ``linked_hash``.
|
||||
|
||||
Only the recorded saver (``recorded-run``) names a new target. A certain
|
||||
step (the recorded saver or the inspector) naming the linked prompt keeps
|
||||
it; the legacy floor and every other answer are skipped.
|
||||
"""
|
||||
if decision.kind != POSITIVE or decision.source not in KEEP_SOURCES:
|
||||
return "skip"
|
||||
if generate_prompt_hash(decision.text) == linked_hash:
|
||||
return "keep"
|
||||
return "relink" if decision.source == "recorded-run" else "skip"
|
||||
|
||||
|
||||
def library_identity(path):
|
||||
"""How a plan names its library: the case-normalized real path."""
|
||||
return os.path.normcase(os.path.realpath(path))
|
||||
|
||||
|
||||
def build_plan(db, output_dir, source_db, read=None):
|
||||
"""The plan for ``db`` (a copy of ``source_db``)."""
|
||||
judge = Judge(db, output_dir, read)
|
||||
conn = db.model.get_connection()
|
||||
(max_row_id,) = conn.execute("SELECT MAX(id) FROM generated_images").fetchone()
|
||||
return {
|
||||
"version": PLAN_VERSION,
|
||||
"created": datetime.datetime.now().isoformat(timespec="seconds"),
|
||||
"source_db": library_identity(source_db),
|
||||
"max_row_id": max_row_id,
|
||||
"output_dir": output_dir,
|
||||
"rows": [judge.row(row) for row in foreign_rows(db, output_dir)],
|
||||
"orphans": [],
|
||||
"delete_orphans": False,
|
||||
}
|
||||
|
||||
|
||||
def blockers(conn, prompt_id, unused):
|
||||
"""Why ``prompt_id`` must not be deleted (empty: nothing references it)."""
|
||||
found = []
|
||||
values = {"id": prompt_id, "tag": SCAN_MARKS.tags[0]}
|
||||
checks = {
|
||||
"linked images": "SELECT COUNT(*) FROM generated_images WHERE prompt_id=:id",
|
||||
"saved pairs": "SELECT COUNT(*) FROM prompt_pairs "
|
||||
"WHERE positive_prompt_id=:id OR negative_prompt_id=:id",
|
||||
"recorded runs": "SELECT COUNT(*) FROM execution_run_contexts "
|
||||
"WHERE positive_prompt_id=:id OR negative_prompt_id=:id",
|
||||
"library state": "SELECT COUNT(*) FROM prompt_library_state "
|
||||
"WHERE prompt_id=:id",
|
||||
"user tags": "SELECT COUNT(*) FROM prompt_tags pt JOIN tags t "
|
||||
"ON t.id=pt.tag_id WHERE pt.prompt_id=:id AND t.name != :tag",
|
||||
}
|
||||
for name, sql in checks.items():
|
||||
count = conn.execute(sql, values).fetchone()[0]
|
||||
if count:
|
||||
found.append(f"{name}: {count}")
|
||||
run_count, last_used, rating, category, notes = conn.execute(
|
||||
"SELECT run_count, last_used_at, rating, category, notes FROM prompts "
|
||||
"WHERE id=?",
|
||||
(prompt_id,),
|
||||
).fetchone()
|
||||
if run_count or last_used or rating:
|
||||
found.append("usage or rating")
|
||||
if category not in (None, SCAN_MARKS.category):
|
||||
found.append(f"category {category}")
|
||||
if notes and not notes.startswith("Auto-scanned from "):
|
||||
found.append("notes")
|
||||
if prompt_id not in unused:
|
||||
found.append("not unused by the app's rule")
|
||||
return found
|
||||
|
||||
|
||||
def _rejudge(conn, judge, plan):
|
||||
for item in plan["rows"]:
|
||||
if item["action"] == "skip":
|
||||
continue
|
||||
row = conn.execute(
|
||||
"SELECT * FROM generated_images WHERE id = ?", (item["row_id"],)
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise Refused(f"row {item['row_id']}: gone")
|
||||
if generate_prompt_hash(item.get("target_text") or "") != item.get(
|
||||
"target_hash"
|
||||
):
|
||||
raise Refused(f"row {item['row_id']}: target_text does not match its hash")
|
||||
fresh = judge.row(row)
|
||||
for key in DRIFT_KEYS:
|
||||
if fresh.get(key) != item.get(key):
|
||||
raise Refused(
|
||||
f"row {item['row_id']}: {key} changed "
|
||||
f"({item.get(key)!r} -> {fresh.get(key)!r})"
|
||||
)
|
||||
|
||||
|
||||
def _targets(conn, db, relinks):
|
||||
"""``{target hash: prompt id}``, creating each "new" target once."""
|
||||
targets = {}
|
||||
for item in relinks:
|
||||
digest = item["target_hash"]
|
||||
if digest in targets:
|
||||
continue
|
||||
if item["target_prompt_id"] != "new":
|
||||
targets[digest] = item["target_prompt_id"]
|
||||
continue
|
||||
prompt_id, created = db._pair_component_outcome(conn, item["target_text"])
|
||||
if not created:
|
||||
raise Refused(f"row {item['row_id']}: target prompt already stored")
|
||||
mark_created(conn, db, prompt_id, SCAN_MARKS, os.path.basename(item["file"]))
|
||||
targets[digest] = prompt_id
|
||||
return targets
|
||||
|
||||
|
||||
def _relink(conn, item, target):
|
||||
"""Point the row at ``target``; True when the wrong row was deleted instead."""
|
||||
row_id = item["row_id"]
|
||||
correct = [
|
||||
row
|
||||
for row in file_link_rows(conn, item["file"])
|
||||
if row["prompt_id"] == target and row["id"] != row_id
|
||||
]
|
||||
if correct: # UNIQUE(prompt_id, file_path): the correct row stays alone
|
||||
if _delete_ids(conn, "generated_images", [row_id]) != 1:
|
||||
raise Refused(f"row {row_id}: delete touched no row")
|
||||
return True
|
||||
# A pair link records the old positive: it goes with the old link
|
||||
conn.execute("DELETE FROM image_prompt_pairs WHERE image_id = ?", (row_id,))
|
||||
cursor = conn.execute(
|
||||
"UPDATE generated_images SET prompt_id = ? WHERE id = ? AND prompt_id = ?",
|
||||
(target, row_id, item["current_prompt_id"]),
|
||||
)
|
||||
if cursor.rowcount != 1:
|
||||
raise Refused(f"row {row_id}: relink touched {cursor.rowcount} rows")
|
||||
return False
|
||||
|
||||
|
||||
def _audit(conn, relinks):
|
||||
"""The prompts the relinks leave without any link, with their blockers."""
|
||||
sources = {}
|
||||
for item in relinks:
|
||||
sources.setdefault(item["current_prompt_id"], []).append(item)
|
||||
unused = {row[0] for row in conn.execute(ORPHANED_PROMPTS_SQL)}
|
||||
audit = []
|
||||
for prompt_id in sorted(sources):
|
||||
if conn.execute(
|
||||
"SELECT 1 FROM generated_images WHERE prompt_id = ?", (prompt_id,)
|
||||
).fetchone():
|
||||
continue
|
||||
(text,) = conn.execute(
|
||||
"SELECT text FROM prompts WHERE id = ?", (prompt_id,)
|
||||
).fetchone()
|
||||
found = blockers(conn, prompt_id, unused)
|
||||
audit.append(
|
||||
{
|
||||
"id": prompt_id,
|
||||
"text_excerpt": " ".join(text.split())[:EXCERPT],
|
||||
"text_sha256": sha256(text),
|
||||
"images_before": len(sources[prompt_id]),
|
||||
"blockers": found,
|
||||
"safe": not found,
|
||||
}
|
||||
)
|
||||
return audit
|
||||
|
||||
|
||||
def _delete_orphans(conn, plan):
|
||||
chosen = [orphan for orphan in plan["orphans"] if orphan["safe"]]
|
||||
unused = {row[0] for row in conn.execute(ORPHANED_PROMPTS_SQL)}
|
||||
for orphan in chosen:
|
||||
row = conn.execute(
|
||||
"SELECT text FROM prompts WHERE id = ?", (orphan["id"],)
|
||||
).fetchone()
|
||||
if row is None or sha256(row[0]) != orphan["text_sha256"]:
|
||||
raise Refused(f"prompt {orphan['id']}: gone or text changed")
|
||||
found = blockers(conn, orphan["id"], unused)
|
||||
if found:
|
||||
raise Refused(f"prompt {orphan['id']}: {', '.join(found)}")
|
||||
ids = [orphan["id"] for orphan in chosen]
|
||||
if _delete_ids(conn, "prompts", ids) != len(ids):
|
||||
raise Refused("an orphan prompt was not deleted")
|
||||
return len(ids)
|
||||
|
||||
|
||||
def apply_plan(db, plan, read=None, before_write=None):
|
||||
"""Write ``plan`` in one transaction; returns ``(result, orphan audit)``.
|
||||
|
||||
``before_write()`` runs just before ``BEGIN IMMEDIATE`` (the running
|
||||
check). Raises ``Refused`` (and rolls back) on any drift or failed check.
|
||||
"""
|
||||
judge = Judge(db, plan["output_dir"], read)
|
||||
relinks = [item for item in plan["rows"] if item["action"] == "relink"]
|
||||
if before_write is not None:
|
||||
before_write()
|
||||
with db._pair_transaction() as conn:
|
||||
_rejudge(conn, judge, plan)
|
||||
targets = _targets(conn, db, relinks)
|
||||
dropped = sum(
|
||||
_relink(conn, item, targets[item["target_hash"]]) for item in relinks
|
||||
)
|
||||
audit = _audit(conn, relinks)
|
||||
deleted = _delete_orphans(conn, plan) if plan.get("delete_orphans") else 0
|
||||
created = sum(1 for item in relinks if item["target_prompt_id"] == "new")
|
||||
result = {
|
||||
"relinked": len(relinks) - dropped,
|
||||
"duplicates_dropped": dropped,
|
||||
"prompts_created": len(
|
||||
{i["target_hash"] for i in relinks if i["target_prompt_id"] == "new"}
|
||||
),
|
||||
"rows_to_new_prompts": created,
|
||||
"kept": sum(1 for item in plan["rows"] if item["action"] == "keep"),
|
||||
"orphans_deleted": deleted,
|
||||
}
|
||||
return result, audit
|
||||
|
||||
|
||||
def counts(path, readonly=True):
|
||||
"""Table counts plus the integrity and foreign-key checks.
|
||||
|
||||
Read-only for a library; the tool's own backup is opened read-write so
|
||||
a WAL file leaves no side files behind once it closes.
|
||||
"""
|
||||
uri = Path(os.path.abspath(path)).as_uri() + ("?mode=ro" if readonly else "")
|
||||
conn = sqlite3.connect(uri, uri=True)
|
||||
try:
|
||||
found = {
|
||||
table: conn.execute(f'SELECT COUNT(*) FROM "{table}"').fetchone()[0]
|
||||
for table in COUNTED # nosec B608 - fixed table names
|
||||
}
|
||||
found["integrity_check"] = conn.execute("PRAGMA integrity_check").fetchone()[0]
|
||||
found["foreign_key_check"] = len(
|
||||
conn.execute("PRAGMA foreign_key_check").fetchall()
|
||||
)
|
||||
return found
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def _print_counts(before, after, out):
|
||||
out.write(f"{'':24}{'before':>12}{'after':>12}\n")
|
||||
for key in before:
|
||||
out.write(f"{key:24}{before[key]!s:>12}{after[key]!s:>12}\n")
|
||||
|
||||
|
||||
def _summary(plan):
|
||||
actions = [item["action"] for item in plan["rows"]]
|
||||
return " ".join(
|
||||
f"{name}: {actions.count(name)}" for name in ("relink", "keep", "skip")
|
||||
)
|
||||
|
||||
|
||||
def _cell(text, limit=EXCERPT):
|
||||
"""One table cell: whitespace folded, cut to ``limit`` (None: whole)."""
|
||||
folded = " ".join(str(text).split())
|
||||
return (folded if limit is None else folded[:limit]).replace("|", "\\|")
|
||||
|
||||
|
||||
def _relative(path, root):
|
||||
try:
|
||||
return os.path.relpath(path, root)
|
||||
except ValueError: # another drive on Windows
|
||||
return path
|
||||
|
||||
|
||||
def markdown(plan, prompt_texts):
|
||||
"""The maintainer's list of the plan."""
|
||||
root = plan["output_dir"]
|
||||
rel = lambda path: _relative(path, root) # noqa: E731
|
||||
rows = plan["rows"]
|
||||
lines = [
|
||||
"# #247 repair plan (dry run)",
|
||||
"",
|
||||
f"Created {plan['created']}; output folder `{root}`. {_summary(plan)}.",
|
||||
"",
|
||||
f"## Relink ({sum(i['action'] == 'relink' for i in rows)})",
|
||||
"",
|
||||
"| row | file | linked prompt | linked text | target | target hash "
|
||||
"| target text |",
|
||||
"|---:|---|---:|---|---|---|---|",
|
||||
]
|
||||
for item in (i for i in rows if i["action"] == "relink"):
|
||||
lines.append(
|
||||
f"| {item['row_id']} | {_cell(rel(item['file']), None)} | "
|
||||
f"{item['current_prompt_id']} | "
|
||||
f"{_cell(prompt_texts.get(item['current_prompt_id'], ''))} | "
|
||||
f"{item['target_prompt_id']} | `{item['target_hash'][:12]}` | "
|
||||
f"{_cell(item['target_text'])} |"
|
||||
)
|
||||
lines += ["", f"## Keep ({sum(i['action'] == 'keep' for i in rows)})", ""]
|
||||
lines += ["| row | file | prompt |", "|---:|---|---:|"]
|
||||
for item in (i for i in rows if i["action"] == "keep"):
|
||||
lines.append(
|
||||
f"| {item['row_id']} | {_cell(rel(item['file']), None)} | "
|
||||
f"{item['current_prompt_id']} |"
|
||||
)
|
||||
lines += ["", f"## Skip ({sum(i['action'] == 'skip' for i in rows)})", ""]
|
||||
lines += ["| row | file | prompt | reason |", "|---:|---|---:|---|"]
|
||||
for item in (i for i in rows if i["action"] == "skip"):
|
||||
reason = item["reason"] + (
|
||||
f" ({item['decision']})" if item.get("decision") else ""
|
||||
)
|
||||
lines.append(
|
||||
f"| {item['row_id']} | {_cell(rel(item['file']), None)} | "
|
||||
f"{item['current_prompt_id']} | {reason} |"
|
||||
)
|
||||
choice = "ON (--delete-orphans)" if plan["delete_orphans"] else "OFF"
|
||||
lines += [
|
||||
"",
|
||||
f"## Prompts that would lose every link ({len(plan['orphans'])})",
|
||||
"",
|
||||
f"Deleting the safe ones: {choice}.",
|
||||
"",
|
||||
"| id | text | images before | blockers | safe to delete |",
|
||||
"|---:|---|---:|---|---|",
|
||||
]
|
||||
for orphan in plan["orphans"]:
|
||||
lines.append(
|
||||
f"| {orphan['id']} | {_cell(orphan['text_excerpt'])} | "
|
||||
f"{orphan['images_before']} | {', '.join(orphan['blockers']) or 'none'} | "
|
||||
f"{'yes' if orphan['safe'] else 'no'} |"
|
||||
)
|
||||
simulated = plan.get("simulated")
|
||||
if simulated:
|
||||
lines += ["", "## Rehearsed on the dry run's copy", ""]
|
||||
lines += ["| table | before | after |", "|---|---:|---:|"]
|
||||
for key, value in simulated["before"].items():
|
||||
lines.append(f"| {key} | {value} | {simulated['after'][key]} |")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def _texts(db):
|
||||
return dict(
|
||||
db.model.get_connection().execute(
|
||||
"SELECT id, text FROM prompts WHERE id IN (SELECT prompt_id "
|
||||
"FROM generated_images)"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _rehearse(source, scratch, name, plan):
|
||||
"""``apply_plan`` on a fresh backup of ``source``: (counts, result, audit)."""
|
||||
copy = os.path.join(scratch, name)
|
||||
backup_database(source, copy)
|
||||
db = PromptDatabase(copy)
|
||||
try:
|
||||
before = counts(copy)
|
||||
result, audit = apply_plan(db, plan)
|
||||
finally:
|
||||
db.close_all()
|
||||
return {"before": before, "after": counts(copy), "result": result}, audit
|
||||
|
||||
|
||||
def dry_run(args, out):
|
||||
with tempfile.TemporaryDirectory() as scratch:
|
||||
copy = os.path.join(scratch, "judge.db")
|
||||
backup_database(args.database, copy)
|
||||
db = PromptDatabase(copy)
|
||||
try:
|
||||
plan = build_plan(db, os.path.abspath(args.output_dir), args.database)
|
||||
texts = _texts(db)
|
||||
finally:
|
||||
db.close_all()
|
||||
simulated, audit = _rehearse(args.database, scratch, "relinks.db", plan)
|
||||
plan.update(orphans=audit, delete_orphans=bool(args.delete_orphans))
|
||||
if plan["delete_orphans"]:
|
||||
# The plan as the apply will write it, deletes included
|
||||
simulated, _audit = _rehearse(args.database, scratch, "deletes.db", plan)
|
||||
plan["simulated"] = simulated
|
||||
plan_path = Path(args.plan)
|
||||
plan_path.write_text(
|
||||
json.dumps(plan, indent=1, ensure_ascii=False) + "\n", encoding="utf-8"
|
||||
)
|
||||
list_path = plan_path.with_name(plan_path.stem + ".list.md")
|
||||
list_path.write_text(markdown(plan, texts), encoding="utf-8")
|
||||
out.write(f"{_summary(plan)}\n")
|
||||
out.write(
|
||||
f"Prompts that would lose every link: {len(audit)} "
|
||||
f"({sum(o['safe'] for o in audit)} safe to delete; deleting "
|
||||
f"{'ON' if plan['delete_orphans'] else 'OFF'})\n"
|
||||
)
|
||||
out.write(f"Plan: {plan_path}\nList: {list_path}\n")
|
||||
out.write("Dry run: nothing was changed.\n")
|
||||
return 0
|
||||
|
||||
|
||||
def _is_live(database, comfyui_dir):
|
||||
live = os.path.join(comfyui_dir, *LIVE_LIBRARY)
|
||||
return library_identity(database) == library_identity(live)
|
||||
|
||||
|
||||
def _stamp():
|
||||
return datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
|
||||
|
||||
def schema_of(path):
|
||||
"""``sqlite_master`` and ``user_version`` from a read-only connection."""
|
||||
uri = Path(os.path.abspath(path)).as_uri() + "?mode=ro"
|
||||
conn = sqlite3.connect(uri, uri=True)
|
||||
try:
|
||||
master = conn.execute(
|
||||
"SELECT type, name, tbl_name, sql FROM sqlite_master ORDER BY type, name"
|
||||
).fetchall()
|
||||
return master, conn.execute("PRAGMA user_version").fetchone()[0]
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def running_reasons(args, probe, handles):
|
||||
"""Why writing must not start now (empty: go ahead).
|
||||
|
||||
Another process holding the library open always counts, also in a
|
||||
rehearsal; ComfyUI's processes and port count unless rehearsing.
|
||||
"""
|
||||
held = [
|
||||
reason
|
||||
for reason in handles(args.database)
|
||||
if not (reason == NO_PROC and args.i_stopped_comfyui)
|
||||
]
|
||||
return held + ([] if args.rehearsal else probe(args.comfyui_dir, args.port))
|
||||
|
||||
|
||||
def _preflight(args, plan, probe, handles):
|
||||
"""The refusals before any backup or write, as a message (None: go on)."""
|
||||
if plan.get("version") != PLAN_VERSION:
|
||||
return "not a plan of this tool"
|
||||
if args.rehearsal and _is_live(args.database, args.comfyui_dir):
|
||||
return f"--rehearsal is for a copy; this is the library of {args.comfyui_dir}"
|
||||
if plan.get("source_db") != library_identity(args.database):
|
||||
return f"this plan was made for {plan.get('source_db')}"
|
||||
if not os.path.isfile(os.path.join(args.comfyui_dir, "main.py")):
|
||||
return f"no main.py in {args.comfyui_dir}: not a ComfyUI folder"
|
||||
reasons = running_reasons(args, probe, handles)
|
||||
if reasons:
|
||||
return "ComfyUI may be running: " + "; ".join(reasons)
|
||||
return None
|
||||
|
||||
|
||||
def _backup(args, before):
|
||||
"""The verified backup's path; raises ``Refused``."""
|
||||
folder = args.backup_dir or os.path.dirname(os.path.abspath(args.database))
|
||||
name = f"{os.path.basename(args.database)}.pre-247-repair_{_stamp()}"
|
||||
backup = os.path.join(folder, name)
|
||||
if os.path.exists(backup):
|
||||
raise Refused(f"the backup {backup} already exists")
|
||||
backup_database(args.database, backup)
|
||||
saved = counts(backup, readonly=False)
|
||||
if saved != before or saved["integrity_check"] != "ok":
|
||||
raise Refused(f"the backup {backup} does not match the library")
|
||||
return backup
|
||||
|
||||
|
||||
def _open_unchanged(path):
|
||||
"""``PromptDatabase(path)``; raises ``Refused`` when opening it changed
|
||||
its schema (a migration of a newer checkout, committed on open)."""
|
||||
schema = schema_of(path)
|
||||
db = PromptDatabase(path)
|
||||
db.model.get_connection()
|
||||
if schema_of(path) != schema:
|
||||
db.close_all()
|
||||
raise Refused(
|
||||
"opening the library with this checkout changed its schema (that "
|
||||
"change is committed); run the tool from a checkout at the live "
|
||||
"code version"
|
||||
)
|
||||
return db
|
||||
|
||||
|
||||
def apply(args, out, probe, handles):
|
||||
with open(args.apply, encoding="utf-8") as stream:
|
||||
plan = json.load(stream)
|
||||
refusal = _preflight(args, plan, probe, handles)
|
||||
if refusal:
|
||||
out.write(f"REFUSED: {refusal}\n")
|
||||
return 2
|
||||
if args.rehearsal:
|
||||
out.write("(rehearsal on a copy: ComfyUI's process and port not checked)\n")
|
||||
before = counts(args.database)
|
||||
try:
|
||||
backup = _backup(args, before)
|
||||
except Refused as error:
|
||||
out.write(f"REFUSED: {error}\n")
|
||||
return 2
|
||||
out.write(f"Backup: {backup}\n")
|
||||
|
||||
def still_stopped():
|
||||
reasons = running_reasons(args, probe, handles)
|
||||
if reasons:
|
||||
raise Refused("ComfyUI may be running: " + "; ".join(reasons))
|
||||
|
||||
try:
|
||||
db = _open_unchanged(args.database)
|
||||
except Refused as error:
|
||||
out.write(f"REFUSED: {error}. Backup: {backup}\n")
|
||||
return 2
|
||||
try:
|
||||
result, _audit = apply_plan(db, plan, before_write=still_stopped)
|
||||
except Exception as error:
|
||||
out.write(
|
||||
f"REFUSED (rolled back, nothing changed): {error}. Backup: {backup}\n"
|
||||
)
|
||||
return 2
|
||||
finally:
|
||||
db.close_all()
|
||||
after = counts(args.database)
|
||||
_print_counts(before, after, out)
|
||||
out.write(f"Applied: {json.dumps(result, sort_keys=True)}\n")
|
||||
if after["integrity_check"] != "ok" or after["foreign_key_check"]:
|
||||
out.write(f"WARNING: integrity or foreign-key check failed; restore {backup}\n")
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv=None, out=sys.stdout, probe=comfyui_running, handles=open_by_others):
|
||||
parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
|
||||
parser.add_argument("database", help="the library database (prompts.db)")
|
||||
parser.add_argument("--output-dir", help="ComfyUI's output folder (dry run)")
|
||||
parser.add_argument(
|
||||
"--plan", default="repair_foreign_links_plan.json", help="plan to write"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--delete-orphans",
|
||||
action="store_true",
|
||||
help="record in the plan that the safe orphan prompts are deleted",
|
||||
)
|
||||
parser.add_argument("--apply", metavar="PLAN", help="write this reviewed plan")
|
||||
parser.add_argument("--comfyui-dir", help="the ComfyUI folder (apply)")
|
||||
parser.add_argument("--backup-dir", help="where the backup goes (apply)")
|
||||
parser.add_argument("--port", type=int, default=COMFY_PORT)
|
||||
parser.add_argument(
|
||||
"--rehearsal",
|
||||
action="store_true",
|
||||
help="apply to a copy while ComfyUI runs (never its own library)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--i-stopped-comfyui",
|
||||
action="store_true",
|
||||
help="without /proc: say ComfyUI is stopped (the port is still checked)",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
if not os.path.isfile(args.database):
|
||||
parser.error(f"no database at {args.database}")
|
||||
if args.apply:
|
||||
if not args.comfyui_dir:
|
||||
parser.error("--apply needs --comfyui-dir (the running check)")
|
||||
return apply(args, out, probe, handles)
|
||||
if not args.output_dir:
|
||||
parser.error("the dry run needs --output-dir")
|
||||
if Path(args.plan).suffix.lower() == ".md":
|
||||
parser.error("--plan is the JSON plan; the list is written beside it")
|
||||
return dry_run(args, out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,142 @@
|
||||
"""The monitor's path for outputs of workflows without a PromptManager node (#247).
|
||||
|
||||
A foreign output (no positive PromptManager node in its graph) is decided by
|
||||
the ladder the output scan uses (``utils.file_prompt.decide_file_prompt``)
|
||||
with the ``MONITOR`` policy: certain files are stored at once, files only
|
||||
recorded history can decide wait for their run's outputs to be sealed, and
|
||||
everything else stays unlinked. This module holds the parts that do not
|
||||
need the monitor's waiting sets: the setting, the decision call and the
|
||||
store. The waiting and the re-check stay on ``ImageGenerationHandler``.
|
||||
"""
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Callable, Mapping, Optional, Tuple
|
||||
|
||||
from .file_prompt import (
|
||||
NOT_SAVED,
|
||||
NOT_SAVED_WIRED,
|
||||
PAIR,
|
||||
POSITIVE,
|
||||
Decision,
|
||||
Policy,
|
||||
decide_file_prompt,
|
||||
)
|
||||
from .hashing import generate_prompt_hash
|
||||
from .usage_tracking import survive_switch
|
||||
|
||||
try:
|
||||
from ..database.pair_ingestion import RUN_MARKS, ingest_resolved_pair
|
||||
from ..database.run_attribution import recorded_output_lookup
|
||||
from ..database.scanned_prompts import store_scanned_prompt_or_raise
|
||||
except ImportError:
|
||||
from database.pair_ingestion import RUN_MARKS, ingest_resolved_pair
|
||||
from database.run_attribution import recorded_output_lookup
|
||||
from database.scanned_prompts import store_scanned_prompt_or_raise
|
||||
|
||||
# Decisions that name a prompt to store
|
||||
CERTAIN = (PAIR, POSITIVE)
|
||||
# Decisions of a run whose prompt has Save to library off, or wired from a
|
||||
# node the file cannot show (#90): final, nothing is stored, nothing waits
|
||||
NOT_SAVED_KINDS = (NOT_SAVED, NOT_SAVED_WIRED)
|
||||
# Sources certain without a recorded run: the graph alone, or its recorded saver
|
||||
ANCHORED_SOURCES = ("inspector", "recorded-run")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ForeignRecheck:
|
||||
"""``schedule`` snapshot: the one re-check of a foreign output after its
|
||||
run's outputs were sealed. ``reference`` is the sealed output reference,
|
||||
the spelling history recorded (``recorded_output_lookup(reference=)``)."""
|
||||
|
||||
reference: Mapping[str, Any]
|
||||
|
||||
|
||||
def save_foreign_runs() -> bool:
|
||||
"""Whether a foreign output may be stored now, read for every file.
|
||||
|
||||
``GalleryConfig.SAVE_FOREIGN_RUNS`` and ``MONITORING_ENABLED``: a
|
||||
monitoring switch saved off stops foreign writes at once, although the
|
||||
running observer watches until a restart. True when the configuration
|
||||
cannot be imported (the defaults).
|
||||
"""
|
||||
try:
|
||||
from ..py.config import GalleryConfig
|
||||
except ImportError:
|
||||
try:
|
||||
from py.config import GalleryConfig
|
||||
except ImportError:
|
||||
return True
|
||||
return (
|
||||
GalleryConfig.SAVE_FOREIGN_RUNS is not False
|
||||
and GalleryConfig.MONITORING_ENABLED is not False
|
||||
)
|
||||
|
||||
|
||||
def history_lookup(
|
||||
db: Any, image_path: str, reference: Optional[Mapping[str, Any]] = None
|
||||
) -> Callable[[], Any]:
|
||||
"""The ladder's ``lookup`` for one file: its ``RecordedOutput`` or None.
|
||||
|
||||
With ``reference`` the file is spelled as history recorded it. Failures
|
||||
are logged by ``recorded_output_lookup`` and name nothing.
|
||||
"""
|
||||
lookup = recorded_output_lookup(
|
||||
db, reference=dict(reference) if reference is not None else None
|
||||
)
|
||||
return lambda: lookup(image_path)
|
||||
|
||||
|
||||
def decide_foreign(
|
||||
image_path: str,
|
||||
metadata: Any,
|
||||
*,
|
||||
in_output: bool,
|
||||
lookup: Optional[Callable[[], Any]],
|
||||
) -> Decision:
|
||||
"""The ``MONITOR`` ladder for one foreign output (the size is read only
|
||||
when the ladder needs it)."""
|
||||
|
||||
def file_size() -> Optional[int]:
|
||||
try:
|
||||
return os.path.getsize(image_path)
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
return decide_file_prompt(
|
||||
metadata,
|
||||
file_name=os.path.basename(image_path),
|
||||
file_size=file_size,
|
||||
in_output=in_output,
|
||||
lookup=lookup,
|
||||
policy=Policy.MONITOR,
|
||||
)
|
||||
|
||||
|
||||
def store_foreign(
|
||||
db: Any, image_path: str, decision: Decision
|
||||
) -> Tuple[str, Optional[int]]:
|
||||
"""Store a certain decision: ``(status, prompt id)``.
|
||||
|
||||
One write through ``survive_switch``, so it runs again on the moved
|
||||
library when a Move library switched while it waited (#249 S6b). Created
|
||||
prompts carry ``RUN_MARKS``. A file linked to another prompt is a
|
||||
``conflict`` and nothing is written. Database errors are raised.
|
||||
"""
|
||||
|
||||
def store():
|
||||
if decision.kind == PAIR:
|
||||
result = ingest_resolved_pair(
|
||||
db, image_path, decision.canonical, decision.resolution, marks=RUN_MARKS
|
||||
)
|
||||
return result["status"], result.get("image_id")
|
||||
return store_scanned_prompt_or_raise(
|
||||
db, image_path, decision.canonical, decision.text, RUN_MARKS
|
||||
)
|
||||
|
||||
status, linked = survive_switch(db, store)
|
||||
if decision.kind != PAIR or linked is None:
|
||||
return status, linked
|
||||
# The pair ingester reports the image row; its prompt is the positive's.
|
||||
prompt = db.get_prompt_by_hash(generate_prompt_hash(decision.text))
|
||||
return status, prompt["id"] if prompt else None
|
||||
+29
-5
@@ -58,11 +58,7 @@ def generate_prompt_hash(text: str) -> str:
|
||||
if not isinstance(text, str):
|
||||
raise TypeError("Text must be a string")
|
||||
|
||||
# Normalize the text by stripping whitespace and converting to lowercase
|
||||
# for consistent hashing regardless of minor formatting differences
|
||||
normalized_text = text.strip().lower()
|
||||
|
||||
return hashlib.sha256(normalized_text.encode("utf-8")).hexdigest()
|
||||
return hashlib.sha256(prompt_key(text).encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def stored_prompt_text(text: str) -> str:
|
||||
@@ -93,6 +89,34 @@ STRIPPED_CODE_POINTS = (
|
||||
_STRIP_CODES = ",".join(map(str, STRIPPED_CODE_POINTS))
|
||||
|
||||
|
||||
def prompt_key(text: str) -> str:
|
||||
"""The dedup key: the stored text with case folded by ``str.lower()``.
|
||||
|
||||
Two texts with one key are one prompt (the hash is sha256 of the key), so
|
||||
a run belongs to the history of the prompt with its key (#460).
|
||||
"""
|
||||
return stored_prompt_text(text).lower()
|
||||
|
||||
|
||||
def _sql_key(value):
|
||||
return prompt_key(value) if isinstance(value, str) else None
|
||||
|
||||
|
||||
def _sql_hash(value):
|
||||
return generate_prompt_hash(value) if isinstance(value, str) else None
|
||||
|
||||
|
||||
def register_prompt_functions(conn) -> None:
|
||||
"""Give ``conn`` ``pm_prompt_key(x)`` and ``pm_prompt_hash(x)`` (#460).
|
||||
|
||||
SQLite's lower() folds ASCII only; these call Python, so SQL gets exactly
|
||||
``prompt_key`` / ``generate_prompt_hash`` (NULL for a non-text value).
|
||||
Never used in schema objects, so the file stays readable without them.
|
||||
"""
|
||||
conn.create_function("pm_prompt_key", 1, _sql_key, deterministic=True)
|
||||
conn.create_function("pm_prompt_hash", 1, _sql_hash, deterministic=True)
|
||||
|
||||
|
||||
def stored_prompt_text_sql(expression: str) -> str:
|
||||
"""``stored_prompt_text`` as an SQLite expression over ``expression``.
|
||||
|
||||
|
||||
+201
-41
@@ -33,6 +33,17 @@ from typing import Callable, Optional, Dict, Any, List, Tuple
|
||||
from watchdog.observers import Observer
|
||||
from watchdog.events import FileSystemEventHandler
|
||||
|
||||
from .file_prompt import HISTORY, NOT_SAVED, NOT_SAVED_WIRED
|
||||
from .foreign_runs import (
|
||||
ANCHORED_SOURCES,
|
||||
CERTAIN,
|
||||
NOT_SAVED_KINDS,
|
||||
ForeignRecheck,
|
||||
decide_foreign,
|
||||
history_lookup,
|
||||
save_foreign_runs,
|
||||
store_foreign,
|
||||
)
|
||||
from .hashing import generate_prompt_hash
|
||||
from .metadata_extractor import ComfyUIMetadataExtractor
|
||||
from .parallel import note_refusal, refused_at_shutdown
|
||||
@@ -153,6 +164,11 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
# Sealed re-checks ``schedule`` refused (the path still pending, or the
|
||||
# pending set full): retried each time a pending image finishes.
|
||||
self._history_due: "OrderedDict[str, None]" = OrderedDict()
|
||||
# The same two sets for foreign outputs (#247), apart so a large
|
||||
# foreign run never evicts a waiting keyless file; the due set keeps
|
||||
# each file's sealed output reference.
|
||||
self._awaiting_foreign: "OrderedDict[str, Tuple[str, float]]" = OrderedDict()
|
||||
self._foreign_due: "OrderedDict[str, dict]" = OrderedDict()
|
||||
self._awaiting_lock = threading.Lock()
|
||||
self._sealed_subscribed = False
|
||||
# Set by ``close``: no new work, no new worker, ever (#302).
|
||||
@@ -253,6 +269,8 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
with self._awaiting_lock:
|
||||
self._awaiting.clear()
|
||||
self._history_due.clear()
|
||||
self._awaiting_foreign.clear()
|
||||
self._foreign_due.clear()
|
||||
with self._pending_lock:
|
||||
self._pending.clear()
|
||||
worker, self._worker = self._worker, None
|
||||
@@ -438,6 +456,9 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
if prompt_snapshot is HISTORY_ONLY:
|
||||
self._recheck_history(image_path)
|
||||
return
|
||||
if isinstance(prompt_snapshot, ForeignRecheck):
|
||||
self._recheck_foreign(image_path, prompt_snapshot.reference)
|
||||
return
|
||||
self.logger.info(f"Processing image: {image_path}")
|
||||
|
||||
if not os.path.exists(image_path):
|
||||
@@ -457,17 +478,12 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
# negative-prompt PromptManager nodes in the workflow.
|
||||
state, current_prompt = self._find_prompt_from_metadata(metadata)
|
||||
if state == "opted_out":
|
||||
self.logger.info(
|
||||
"Save to library is off for this run's prompt, skipping image: "
|
||||
f"{image_path}"
|
||||
)
|
||||
self._log_not_saved(image_path)
|
||||
return
|
||||
if state == "foreign":
|
||||
# Another workflow saved this image: it belongs to no prompt of
|
||||
# ours, and must not consume a batch queue entry.
|
||||
self.logger.info(
|
||||
f"No PromptManager prompt in workflow, skipping image: {image_path}"
|
||||
)
|
||||
# Another workflow saved this image: never a batch queue entry
|
||||
# of ours; its own graph may name its prompt (#247).
|
||||
self._ingest_foreign(image_path, metadata)
|
||||
return
|
||||
if current_prompt:
|
||||
self.logger.info(
|
||||
@@ -589,36 +605,55 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
resolved = os.path.abspath(image_path)
|
||||
return os.path.normcase(resolved)
|
||||
|
||||
def _prune_awaiting(self) -> None:
|
||||
"""Drop expired and surplus entries (``_awaiting_lock`` held)."""
|
||||
def _prune_awaiting(self, store=None) -> List[Tuple[str, bool]]:
|
||||
"""Drop expired and surplus entries of ``store`` (the keyless set by
|
||||
default; ``_awaiting_lock`` held). Returns the dropped foreign files
|
||||
as ``(path, evicted)`` for ``_log_dropped`` after the lock."""
|
||||
store = self._awaiting if store is None else store
|
||||
now = time.monotonic()
|
||||
for key in [k for k, (_p, due) in self._awaiting.items() if due <= now]:
|
||||
del self._awaiting[key]
|
||||
while len(self._awaiting) > MAX_AWAITING_HISTORY:
|
||||
self._awaiting.popitem(last=False)
|
||||
expired = [k for k, (_p, due) in store.items() if due <= now]
|
||||
dropped = [(store.pop(key)[0], False) for key in expired]
|
||||
while len(store) > MAX_AWAITING_HISTORY:
|
||||
dropped.append((store.popitem(last=False)[1][0], True))
|
||||
return dropped if store is self._awaiting_foreign else []
|
||||
|
||||
def _await_history(self, image_path: str) -> None:
|
||||
def _log_dropped(self, dropped: List[Tuple[str, bool]]) -> None:
|
||||
"""Log foreign files that stopped waiting (outside ``_awaiting_lock``)."""
|
||||
for image_path, evicted in dropped:
|
||||
name = os.path.basename(image_path)
|
||||
if evicted:
|
||||
self.logger.info(
|
||||
f"Too many files wait for their run; {name} left unlinked"
|
||||
)
|
||||
else:
|
||||
self.logger.info(f"No recorded run for {name}; left unlinked")
|
||||
|
||||
def _await_history(self, image_path: str, store=None) -> None:
|
||||
"""Re-check ``image_path`` once, history only, when its run is sealed.
|
||||
|
||||
A closed handler re-checks nothing.
|
||||
``store`` is the waiting set (``_awaiting_foreign`` for a foreign
|
||||
output). A closed handler re-checks nothing.
|
||||
"""
|
||||
if self._closed:
|
||||
return
|
||||
if not self._sealed_subscribed:
|
||||
subscribe_outputs_sealed(self._on_outputs_sealed)
|
||||
self._sealed_subscribed = True
|
||||
store = self._awaiting if store is None else store
|
||||
key = self._awaiting_key(image_path)
|
||||
with self._awaiting_lock:
|
||||
self._awaiting.pop(key, None)
|
||||
self._awaiting[key] = (
|
||||
image_path,
|
||||
time.monotonic() + AWAIT_HISTORY_SECONDS,
|
||||
)
|
||||
self._prune_awaiting()
|
||||
store.pop(key, None)
|
||||
store[key] = (image_path, time.monotonic() + AWAIT_HISTORY_SECONDS)
|
||||
dropped = self._prune_awaiting(store)
|
||||
self._log_dropped(dropped)
|
||||
|
||||
def _stop_awaiting(self, image_path: str) -> None:
|
||||
def _await_foreign(self, image_path: str) -> None:
|
||||
self._await_history(image_path, self._awaiting_foreign)
|
||||
|
||||
def _stop_awaiting(self, image_path: str, store=None) -> None:
|
||||
store = self._awaiting if store is None else store
|
||||
with self._awaiting_lock:
|
||||
self._awaiting.pop(self._awaiting_key(image_path), None)
|
||||
store.pop(self._awaiting_key(image_path), None)
|
||||
|
||||
def awaiting_history(self) -> List[str]:
|
||||
"""Normalized paths still waiting for their run's recorded outputs."""
|
||||
@@ -626,6 +661,14 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
self._prune_awaiting()
|
||||
return list(self._awaiting)
|
||||
|
||||
def awaiting_foreign(self) -> List[str]:
|
||||
"""Normalized paths of foreign outputs waiting for their run (#247)."""
|
||||
with self._awaiting_lock:
|
||||
dropped = self._prune_awaiting(self._awaiting_foreign)
|
||||
waiting = list(self._awaiting_foreign)
|
||||
self._log_dropped(dropped)
|
||||
return waiting
|
||||
|
||||
def _on_outputs_sealed(self, run_id: str, references: List[dict]) -> None:
|
||||
"""A run's outputs were recorded: re-check its waiting images once.
|
||||
|
||||
@@ -637,6 +680,7 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
due = []
|
||||
with self._awaiting_lock:
|
||||
self._prune_awaiting()
|
||||
dropped = self._prune_awaiting(self._awaiting_foreign)
|
||||
for reference in references:
|
||||
if reference.get("folder_type") != "output":
|
||||
continue
|
||||
@@ -647,39 +691,53 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
)
|
||||
waiting = self._awaiting.pop(key, None)
|
||||
if waiting is not None:
|
||||
due.append(waiting[0])
|
||||
for image_path in due:
|
||||
self._schedule_recheck(image_path)
|
||||
due.append((waiting[0], None))
|
||||
waiting = self._awaiting_foreign.pop(key, None)
|
||||
if waiting is not None:
|
||||
due.append((waiting[0], dict(reference)))
|
||||
self._log_dropped(dropped)
|
||||
for image_path, reference in due:
|
||||
self._schedule_recheck(image_path, reference)
|
||||
|
||||
def _schedule_recheck(self, image_path: str) -> None:
|
||||
"""Schedule the history-only re-check, or keep it due for later.
|
||||
@staticmethod
|
||||
def _recheck_snapshot(reference):
|
||||
return HISTORY_ONLY if reference is None else ForeignRecheck(reference)
|
||||
|
||||
def _schedule_recheck(self, image_path: str, reference=None) -> None:
|
||||
"""Schedule the history-only re-check (with the sealed ``reference``:
|
||||
a foreign output's), or keep it due for later.
|
||||
|
||||
A closed handler drops it.
|
||||
"""
|
||||
if self._closed:
|
||||
return
|
||||
if self.schedule(image_path, HISTORY_ONLY):
|
||||
if self.schedule(image_path, self._recheck_snapshot(reference)):
|
||||
return
|
||||
with self._awaiting_lock:
|
||||
# ``close`` sets the flag before ``shutdown`` clears this set, so
|
||||
# a close that raced the refusal above is seen here.
|
||||
if self._closed:
|
||||
return
|
||||
self._history_due.pop(image_path, None)
|
||||
self._history_due[image_path] = None
|
||||
while len(self._history_due) > MAX_AWAITING_HISTORY:
|
||||
self._history_due.popitem(last=False)
|
||||
due = self._history_due if reference is None else self._foreign_due
|
||||
due.pop(image_path, None)
|
||||
due[image_path] = reference
|
||||
evicted = []
|
||||
while len(due) > MAX_AWAITING_HISTORY:
|
||||
evicted.append(due.popitem(last=False)[0])
|
||||
if reference is not None:
|
||||
self._log_dropped([(path, True) for path in evicted])
|
||||
|
||||
def _drain_history_due(self) -> None:
|
||||
"""Schedule due re-checks now that a pending slot or path is free."""
|
||||
with self._awaiting_lock:
|
||||
if not self._history_due:
|
||||
return
|
||||
due = list(self._history_due)
|
||||
for image_path in due:
|
||||
if self.schedule(image_path, HISTORY_ONLY):
|
||||
due = [(self._history_due, p, None) for p in self._history_due]
|
||||
due += [(self._foreign_due, *item) for item in self._foreign_due.items()]
|
||||
for store, image_path, reference in due:
|
||||
if self.schedule(image_path, self._recheck_snapshot(reference)):
|
||||
with self._awaiting_lock:
|
||||
self._history_due.pop(image_path, None)
|
||||
# A newer seal may have replaced it meanwhile: keep that one
|
||||
if image_path in store and store[image_path] is reference:
|
||||
del store[image_path]
|
||||
|
||||
def _recheck_history(self, image_path: str) -> None:
|
||||
"""The one history-only re-check of a keyless image (#342)."""
|
||||
@@ -703,6 +761,108 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
metadata or {"file_info": self.get_basic_file_info(image_path)},
|
||||
)
|
||||
|
||||
def _ingest_foreign(self, image_path: str, metadata) -> None:
|
||||
"""A foreign output at settle: store its certain prompt, or wait for
|
||||
its run's recorded outputs when only history can decide (#247)."""
|
||||
name = os.path.basename(image_path)
|
||||
if not save_foreign_runs():
|
||||
self.logger.info(
|
||||
f"No PromptManager prompt in workflow, skipping image: {image_path}"
|
||||
)
|
||||
return
|
||||
in_output = in_output_folder(image_path, host_output_directory())
|
||||
decision = decide_foreign(
|
||||
image_path, metadata, in_output=in_output, lookup=None
|
||||
)
|
||||
if decision.kind == HISTORY:
|
||||
decision = self._settle_lookup(image_path, metadata)
|
||||
if decision is None:
|
||||
self.logger.debug(f"{name} waits for its run to end")
|
||||
return
|
||||
self._stop_awaiting(image_path, self._awaiting_foreign)
|
||||
if decision.kind in NOT_SAVED_KINDS:
|
||||
self._log_not_saved(image_path, decision.kind)
|
||||
elif decision.kind in CERTAIN:
|
||||
self._store_foreign(image_path, decision)
|
||||
else:
|
||||
self.logger.info(f"No certain prompt for {name}; left unlinked")
|
||||
|
||||
def _log_not_saved(self, image_path: str, kind: str = NOT_SAVED) -> None:
|
||||
"""The log line of a file whose run's prompt does not save (#90)."""
|
||||
if kind == NOT_SAVED_WIRED:
|
||||
reason = "is wired from a node the image cannot show"
|
||||
else:
|
||||
reason = "is off"
|
||||
self.logger.info(
|
||||
f"Save to library {reason} for this run's prompt, skipping image: "
|
||||
f"{image_path}"
|
||||
)
|
||||
|
||||
def _settle_lookup(self, image_path: str, metadata):
|
||||
"""The settle-time history lookup of a file only history can decide.
|
||||
|
||||
Registered first, so a run sealed before this lookup is not missed.
|
||||
Returns None to keep waiting: no record yet, and the answer is not
|
||||
the inspector's or the recorded saver's (the legacy floor waits for
|
||||
the re-check). With a record, any decision ends the wait. A
|
||||
``not_saved`` here is the undecided step's (an early one never asks
|
||||
history, so it never reaches this lookup): a recorded saver may still
|
||||
resolve another branch, so without a record it waits too (#90).
|
||||
"""
|
||||
self._await_foreign(image_path)
|
||||
found = []
|
||||
lookup = history_lookup(self.db_manager, image_path)
|
||||
|
||||
def recorded():
|
||||
found.append(lookup())
|
||||
return found[-1]
|
||||
|
||||
decision = decide_foreign(image_path, metadata, in_output=True, lookup=recorded)
|
||||
if any(record is not None for record in found):
|
||||
return decision
|
||||
if decision.kind in CERTAIN and decision.source in ANCHORED_SOURCES:
|
||||
return decision
|
||||
return None
|
||||
|
||||
def _recheck_foreign(self, image_path: str, reference) -> None:
|
||||
"""The one re-check of a foreign output after its run ended (#247)."""
|
||||
name = os.path.basename(image_path)
|
||||
if not os.path.exists(image_path):
|
||||
return
|
||||
if not save_foreign_runs():
|
||||
self.logger.info(f"Saving prompts from other workflows is off: {name}")
|
||||
return
|
||||
try:
|
||||
metadata = self.metadata_extractor.extract_metadata(image_path)
|
||||
except Exception as error:
|
||||
self.logger.warning(f"Metadata of {name} could not be read: {error}")
|
||||
return
|
||||
lookup = history_lookup(self.db_manager, image_path, reference)
|
||||
decision = decide_foreign(image_path, metadata, in_output=True, lookup=lookup)
|
||||
if decision.kind in NOT_SAVED_KINDS:
|
||||
self._log_not_saved(image_path, decision.kind)
|
||||
return
|
||||
if decision.kind in CERTAIN:
|
||||
self._store_foreign(image_path, decision)
|
||||
return
|
||||
self.logger.info(
|
||||
f"No certain prompt for {name} after its run ended; left unlinked"
|
||||
)
|
||||
|
||||
def _store_foreign(self, image_path: str, decision) -> None:
|
||||
name = os.path.basename(image_path)
|
||||
if not save_foreign_runs(): # switched off while this file was decided
|
||||
self.logger.info(f"Saving prompts from other workflows is off: {name}")
|
||||
return
|
||||
try:
|
||||
status, prompt_id = store_foreign(self.db_manager, image_path, decision)
|
||||
except Exception as error:
|
||||
self.logger.error(f"Failed to store the prompt of {name}: {error}")
|
||||
return
|
||||
self.logger.info(
|
||||
f"Foreign run: {status} prompt {prompt_id} for {name} ({decision.source})"
|
||||
)
|
||||
|
||||
def _history_prompt(self, image_path: str) -> Optional[int]:
|
||||
"""The stored prompt recorded history attributes to this file, if one."""
|
||||
try:
|
||||
|
||||
@@ -548,8 +548,12 @@ def encode_history_inspection_response(
|
||||
resolution: dict,
|
||||
inspection: "InspectionRead | None",
|
||||
library: dict,
|
||||
image_tags: dict | None = None,
|
||||
) -> bytes:
|
||||
"""Use occurrence identity with the same current-file evidence projection."""
|
||||
"""Use occurrence identity with the same current-file evidence projection.
|
||||
|
||||
*image_tags*, when given, adds the validated ``image_tags`` facet (#457).
|
||||
"""
|
||||
return _encode_inspection(
|
||||
identity={
|
||||
"history_version": 1,
|
||||
@@ -561,4 +565,5 @@ def encode_history_inspection_response(
|
||||
resolution=resolution,
|
||||
inspection=inspection,
|
||||
library=library,
|
||||
image_tags=image_tags,
|
||||
)
|
||||
|
||||
@@ -17,6 +17,7 @@
|
||||
// The workspace Images defaults; the legacy gallery's defaultViewMode has no masonry.
|
||||
const WORKSPACE_LAYOUTS = ["grid", "list", "masonry"];
|
||||
const WORKSPACE_DENSITIES = ["comfortable", "large", "compact"];
|
||||
const VIEWER_OPEN_MODES = ["centered", "fullscreen"];
|
||||
const LIMIT_OPTIONS = [50, 100, 200, 500];
|
||||
const MIN_COLUMNS = 2;
|
||||
const MAX_COLUMNS = 12;
|
||||
@@ -43,6 +44,8 @@
|
||||
// #451: Images opens in Masonry unless this browser chose another layout.
|
||||
workspaceLayout: "masonry",
|
||||
workspaceDensity: "comfortable",
|
||||
// #450 (D156): how the workspace viewer opens.
|
||||
viewerOpenMode: "centered",
|
||||
});
|
||||
|
||||
function toInt(value) {
|
||||
@@ -83,6 +86,7 @@
|
||||
videoLoop: bool(merged.videoLoop, DEFAULT_SETTINGS.videoLoop),
|
||||
workspaceLayout: WORKSPACE_LAYOUTS.includes(merged.workspaceLayout) ? merged.workspaceLayout : DEFAULT_SETTINGS.workspaceLayout,
|
||||
workspaceDensity: WORKSPACE_DENSITIES.includes(merged.workspaceDensity) ? merged.workspaceDensity : DEFAULT_SETTINGS.workspaceDensity,
|
||||
viewerOpenMode: VIEWER_OPEN_MODES.includes(merged.viewerOpenMode) ? merged.viewerOpenMode : DEFAULT_SETTINGS.viewerOpenMode,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -92,7 +96,7 @@
|
||||
return `repeat(${n}, minmax(0, 1fr))`;
|
||||
}
|
||||
|
||||
const api = { DEFAULT_SETTINGS, LIMIT_OPTIONS, VIEW_MODES, WORKSPACE_LAYOUTS, WORKSPACE_DENSITIES, normalizeGallerySettings, gridColumnsStyle };
|
||||
const api = { DEFAULT_SETTINGS, LIMIT_OPTIONS, VIEW_MODES, WORKSPACE_LAYOUTS, WORKSPACE_DENSITIES, VIEWER_OPEN_MODES, normalizeGallerySettings, gridColumnsStyle };
|
||||
if (typeof module !== "undefined" && module.exports) module.exports = api;
|
||||
else root.GallerySettings = api;
|
||||
})(typeof window !== "undefined" ? window : globalThis);
|
||||
|
||||
@@ -21,7 +21,10 @@ const paths = Object.freeze({
|
||||
"tagImage": "M3 4h8l9 9-7 7L3 10V4ZM7 8h.01M18 2v5M15.5 4.5h5",
|
||||
"folder": "M3 5h7l2 3h9v12H3z",
|
||||
"history": "M3 11a9 9 0 1 1 2 7M3 4v7h7M12 7v6l4 2",
|
||||
"review": "M4 3h16v18H4zM8 7h8M8 12h8M8 17h5"
|
||||
"review": "M4 3h16v18H4zM8 7h8M8 12h8M8 17h5",
|
||||
// #450 (D156): the viewer's Full screen / Centered switch.
|
||||
"maximize": "M4 9V4h5M20 9V4h-5M4 15v5h5M20 15v5h-5",
|
||||
"minimize": "M9 4v5H4M15 4v5h5M9 20v-5H4M15 20v-5h5"
|
||||
});
|
||||
|
||||
/** Create an approved decorative outline icon without injecting markup. */
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// The concept's "Inspect a file" page (docs/concepts/workspace/app.js renderMetadata 740–748, selection 1715–1729):
|
||||
// one local file, chosen or dropped, previews inline; the heading's one action opens the shared viewer with the full
|
||||
// inspector on a record read entirely in this browser. Nothing is uploaded, imported or stored. The selection, its
|
||||
// one local file, chosen or dropped, previews inline and (D155, #448) opens the shared viewer with its details showing
|
||||
// at once; the heading's one action opens it again. The record is read entirely in this browser. Nothing is uploaded, imported or stored. The selection, its
|
||||
// blob: URL and its record belong to one local session (local-session.mjs) that ends with this page.
|
||||
import { createButton, createScope } from '../shared/toolkit/core.js';
|
||||
import { pageHeading } from './heading.mjs';
|
||||
@@ -17,7 +17,8 @@ export const INSPECT_COPY = Object.freeze({
|
||||
heading: 'Inspect without importing',
|
||||
intro: 'Read embedded metadata in this browser. The file is not uploaded or added to your library.',
|
||||
chooser: 'Choose an image or video for a local preview',
|
||||
note: 'Your file stays in this browser. Its metadata opens with “Open file metadata”.',
|
||||
// D155 (#448): choosing a file opens its metadata (D61's note named the heading action).
|
||||
note: 'Your file stays in this browser. Choosing it opens its metadata.',
|
||||
toast: 'Local preview only. No file uploaded; metadata is available above.',
|
||||
});
|
||||
// D62: the concept's image list plus the videos the local readers support (AVI/WMV preview only).
|
||||
@@ -38,8 +39,11 @@ function element(tag, className, text) {
|
||||
return node;
|
||||
}
|
||||
|
||||
// The status shown for the selection; AVI/WMV's note and an empty status keep the toast's "metadata is available".
|
||||
const TOASTABLE = new Set(['', LOCAL_SESSION_MESSAGES.containerOnly]);
|
||||
// D155 (#448): a selection with no status, or one without a preview URL (its copy points at the viewer), opens the viewer
|
||||
// at once. AVI/WMV's note keeps the toast instead (the viewer could not play it and reads no metadata), and an
|
||||
// unreadable file stays on the page with its alert; the heading action opens either.
|
||||
const AUTO_OPEN = new Set(['', LOCAL_SESSION_MESSAGES.previewFailed]);
|
||||
const TOASTABLE = new Set([LOCAL_SESSION_MESSAGES.containerOnly]);
|
||||
// How long a replaced video still starting may stay in place before it is released anyway.
|
||||
const STARTING_VIDEO_MS = 15000;
|
||||
|
||||
@@ -131,9 +135,10 @@ export function mountInspect(main, { parentScope, api, sender }) {
|
||||
};
|
||||
const session = createLocalSession({ parentScope: sessionScope, pageTarget: null, revokeObjectURL });
|
||||
const viewer = createMediaViewer({ api, container: main, parentScope: scope, inspectionController: session.controller, sender,
|
||||
adapter: localFileAdapter, onRestoreFocus: () => { if (open.isConnected) open.focus({ preventScroll: true }); } });
|
||||
adapter: localFileAdapter, onRestoreFocus: () => { if (open.isConnected) open.focus({ preventScroll: true }); },
|
||||
onClose: () => { viewerKey = null; } });
|
||||
|
||||
let media = null, mediaUrl = null, statusLine = null, rejectionLine = null, rejectionTimer = 0, announced = null, chosen = null;
|
||||
let viewerKey = null, media = null, mediaUrl = null, statusLine = null, rejectionLine = null, rejectionTimer = 0, announced = null, chosen = null;
|
||||
function statusElement(message) {
|
||||
// /* D63 */ one status line for the states the mock never shows.
|
||||
const line = element('p', 'small spaced-16', message);
|
||||
@@ -202,14 +207,25 @@ export function mountInspect(main, { parentScope, api, sender }) {
|
||||
else preview.prepend(node);
|
||||
}
|
||||
scope.cleanup(session.subscribe(state => {
|
||||
// D155: a new selection can start while the viewer shows the previous one; it closes first, so no element holds
|
||||
// the old URL when the session revokes it (the new file's viewer opens once it is ready).
|
||||
if (viewerKey !== null && (state.phase !== 'ready' || session.entry?.key !== viewerKey)) { viewer.close(); viewerKey = null; }
|
||||
showMedia(state);
|
||||
showStatus(state.message);
|
||||
const entry = session.entry;
|
||||
setAvailable(state.phase === 'ready' && !!entry);
|
||||
if (state.phase === 'ready' && entry && announced !== entry.key) {
|
||||
announced = entry.key;
|
||||
// The session selects the record right after it publishes `ready`, so the viewer opens after that, and only
|
||||
// while this selection is still the current one and no other dialog is open (the viewer is replaced). D62:
|
||||
// a video opens it once the inline preview has settled, so a replacement while it starts never tears down a
|
||||
// second starting video in the viewer (that deadlocked WebKit).
|
||||
if (AUTO_OPEN.has(state.message)) whenSettled(media, () => {
|
||||
if (scope.signal.aborted || session.entry !== entry || session.state.phase !== 'ready' || document.querySelector('dialog[open]:not(#viewer-dialog)')) return;
|
||||
openViewer(input);
|
||||
});
|
||||
// An unreadable or failed preview says so in its alert; the toast never claims metadata is available.
|
||||
if (TOASTABLE.has(state.message)) showWorkspaceToast(INSPECT_COPY.toast);
|
||||
else if (TOASTABLE.has(state.message)) showWorkspaceToast(INSPECT_COPY.toast);
|
||||
}
|
||||
}));
|
||||
// A replacement or page leave revokes the URL after this subscriber dropped the media; a dispose closes the
|
||||
@@ -261,10 +277,25 @@ export function mountInspect(main, { parentScope, api, sender }) {
|
||||
});
|
||||
// A drop that misses the area never navigates away from the page while it is mounted.
|
||||
for (const type of ['dragover', 'drop']) scope.listen(window, type, event => event.preventDefault());
|
||||
scope.listen(open, 'click', () => {
|
||||
/** Runs `run` after this task, or once `preview`'s video stops starting (metadata, an error, or the D62 limit). */
|
||||
function whenSettled(preview, run) {
|
||||
if (!preview || !isStarting(preview.node)) { queueMicrotask(run); return; }
|
||||
// `run` checks that the page and the selection are still current, so the page's end may call `done` too.
|
||||
const settled = new AbortController();
|
||||
const done = () => { settled.abort(); clearTimeout(timer); run(); };
|
||||
const timer = setTimeout(done, STARTING_VIDEO_MS);
|
||||
for (const type of ['loadedmetadata', 'error']) preview.node.addEventListener(type, done, { once: true, signal: settled.signal });
|
||||
scope.cleanup(done);
|
||||
}
|
||||
/** The viewer on the current selection, details showing (D155 amends D97); focus returns to `trigger` on close. */
|
||||
function openViewer(trigger) {
|
||||
const entry = session.entry;
|
||||
if (open.getAttribute('aria-disabled') === 'true' || !entry || session.state.phase !== 'ready') return;
|
||||
viewer.open({ items: [entry], selectedKey: entry.key, trigger: open });
|
||||
if (!entry || session.state.phase !== 'ready') return;
|
||||
viewer.open({ items: [entry], selectedKey: entry.key, trigger, details: 'shown' });
|
||||
viewerKey = entry.key;
|
||||
}
|
||||
scope.listen(open, 'click', () => {
|
||||
if (open.getAttribute('aria-disabled') !== 'true') openViewer(open);
|
||||
});
|
||||
/** Settles when no video on the page is still starting; `null` when none is. */
|
||||
function leaveGuard() {
|
||||
|
||||
@@ -12,10 +12,22 @@ function parameters(filters,limit) {
|
||||
for(const tag of filters.tags || []) if(tag.trim()) query.append('tag',tag.trim());
|
||||
const sort=filters.sort || 'newest';
|
||||
if(!['newest','oldest','name'].includes(sort)) throw new TypeError('Invalid library sort');
|
||||
// #444: how Tags match (every tag, any of them, or parents no tag's Browse returns, which never goes with tags) and
|
||||
// the output folder below a root.
|
||||
const tagMode=filters.tagMode ?? 'all',folder=filters.folder ?? '';
|
||||
if(!LIBRARY_TAG_MODES.includes(tagMode) || (tagMode==='untagged' && query.has('tag'))) throw new TypeError('Invalid library tag mode');
|
||||
if(typeof folder!=='string' || folder.length>1024) throw new TypeError('Invalid library folder');
|
||||
if(tagMode!=='all') query.set('tag_mode',tagMode);
|
||||
if(folder) query.set('folder',folder);
|
||||
query.set('sort',sort);return query;
|
||||
}
|
||||
/** The Library's tag match values (#444); the first is the default, sent as no parameter. */
|
||||
export const LIBRARY_TAG_MODES=Object.freeze(['all','any','untagged']);
|
||||
// `variant_total` is the filtered parents' active negative variants (the concept caption's "M negative variants").
|
||||
function validatePage(page,cursor) {
|
||||
function validatePage(page,cursor,query) {
|
||||
// A tag match or folder the server did not echo was not applied (an older server): never show the whole shelf as filtered.
|
||||
if(query?.has('tag_mode') && page?.tag_mode!==query.get('tag_mode')) throw new TypeError('Unconfirmed library tag mode');
|
||||
if(query?.has('folder') && page?.folder!==query.get('folder')) throw new TypeError('Unconfirmed library folder');
|
||||
if(!page || !Array.isArray(page.items) || page.items.some(item=>!parent(item)) || !Number.isSafeInteger(page.total) || page.total<0 || !Number.isSafeInteger(page.variant_total) || page.variant_total<0 || typeof page.has_more!=='boolean' || !(page.next_cursor===null || (typeof page.next_cursor==='string' && page.next_cursor.length)) || page.has_more!==(page.next_cursor!==null) || (page.has_more && cursor===page.next_cursor)) throw new TypeError('Invalid library page');
|
||||
}
|
||||
export function createLibraryStream({api,onChange=()=>{},restore}) {
|
||||
@@ -37,7 +49,7 @@ export function createLibraryStream({api,onChange=()=>{},restore}) {
|
||||
try {
|
||||
const page=await api.get(`/prompt_manager/library?${params}`,{signal:current.signal});
|
||||
if(disposed || current!==request) return 0;
|
||||
validatePage(page,cursor);
|
||||
validatePage(page,cursor,query);
|
||||
// #437: the storage epoch the shelf's items were read under, which a bulk write sends as its fence; a later
|
||||
// page from a restored storage is refused (its items would belong to another database).
|
||||
const epoch=typeof page.library_epoch==='string'?page.library_epoch:null;
|
||||
@@ -113,3 +125,28 @@ export function libraryWindow(count,columns,top,height,focused=-1) {
|
||||
if(focused>=0 && focused<count) keep.add(Math.floor(focused/columns));
|
||||
return [...keep].sort((a,b)=>a-b).flatMap(row=>Array.from({length:columns},(_,column)=>row*columns+column).filter(index=>index<count));
|
||||
}
|
||||
|
||||
const byFolderName=(a,b)=>{const x=a.toLowerCase(),y=b.toLowerCase();return x<y?-1:x>y?1:a<b?-1:a>b?1:0;};
|
||||
/**
|
||||
* Refine › Folder options (#444 D150): the concept's "All output folders" first, then the folders as a tree built by
|
||||
* path segment, so each folder sits directly under its own parent (never under a longer sibling such as `kiko-v6`) and
|
||||
* folders differing only in case stay distinct siblings. Siblings sort case-insensitively, then by exact spelling. A
|
||||
* label is the last segment indented by depth (the legacy admin select). The chosen folder is kept even when the list
|
||||
* lacks it, with its parents.
|
||||
*/
|
||||
export function libraryFolderOptions(folders,current='') {
|
||||
const root=new Map();
|
||||
for(const folder of current?[...folders,current]:folders) {
|
||||
let level=root;
|
||||
for(const segment of folder.split('/')) {if(!level.has(segment)) level.set(segment,new Map());level=level.get(segment);}
|
||||
}
|
||||
const options=[['','All output folders']];
|
||||
(function walk(level,prefix,depth) {
|
||||
for(const segment of [...level.keys()].sort(byFolderName)) {
|
||||
const path=prefix?`${prefix}/${segment}`:segment;
|
||||
options.push([path,`${'\u00a0\u00a0'.repeat(depth)}${segment}`]);
|
||||
walk(level.get(segment),path,depth+1);
|
||||
}
|
||||
})(root,'',0);
|
||||
return options;
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ import {createScope,createButton} from '../shared/toolkit/core.js';
|
||||
import {createPromptSender} from './prompt-sender.mjs';
|
||||
import {mountPromptBlocks} from './prompt-inspector.mjs';
|
||||
import {mountImages} from './images.mjs';
|
||||
import {createLibraryStream,createLibraryDetail,libraryPromptRoles} from './library-state.mjs';
|
||||
import {createLibraryStream,createLibraryDetail,libraryPromptRoles,LIBRARY_TAG_MODES,libraryFolderOptions} from './library-state.mjs';
|
||||
import {promptRoles} from './prompt-blocks.mjs';
|
||||
import {createLibraryEditor,showWorkspaceToast} from './library-editor.mjs';
|
||||
import {createLibraryEditorController} from './library-editor-state.mjs';
|
||||
@@ -31,6 +31,8 @@ const metadataDrafts=new Map();
|
||||
const archivedVisible=new Map();
|
||||
// The concept's sorts first, then the Library's appended Oldest first (D14); D153 (#447) drops "Highest rated".
|
||||
const LIBRARY_SORTS=Object.freeze([['newest','Recently added'],['name','Name A–Z'],['oldest','Oldest first']]);
|
||||
// Refine › Tag match (#444 D149, D150), in LIBRARY_TAG_MODES order.
|
||||
const TAG_MATCH_OPTIONS=Object.freeze([['all','All of these tags'],['any','Any of these tags'],['untagged','Prompts without tags']]);
|
||||
// A card's focusable controls in order: the D148 selection checkbox, then its buttons (#437).
|
||||
const CARD_CONTROLS='button, input[type="checkbox"]';
|
||||
const CATEGORIES_FAILED='Categories could not be loaded. Open the Prompt library again to retry.';
|
||||
@@ -83,9 +85,9 @@ export function onLibraryWrite(listener) {
|
||||
* with the existing Library filters, and the heading takes focus as after a save.
|
||||
*/
|
||||
export function browseLibrary(filters) {
|
||||
const {text='',category='',tags=[],sort='newest'}=filters || {};
|
||||
if(typeof text!=='string' || typeof category!=='string' || !Array.isArray(tags) || tags.some(tag=>typeof tag!=='string') || typeof sort!=='string') throw new TypeError('Invalid Library filters');
|
||||
shelfVisit={filters:{text,category,tags:[...tags],sort},anchor:null};
|
||||
const {text='',category='',tags=[],sort='newest',tagMode='all',folder=''}=filters || {};
|
||||
if(typeof text!=='string' || typeof category!=='string' || !Array.isArray(tags) || tags.some(tag=>typeof tag!=='string') || typeof sort!=='string' || !LIBRARY_TAG_MODES.includes(tagMode) || typeof folder!=='string') throw new TypeError('Invalid Library filters');
|
||||
shelfVisit={filters:{text,category,tags:[...tags],sort,tagMode,folder},anchor:null};
|
||||
focusHeadingOnMount=true;window.location.hash='#/library';
|
||||
}
|
||||
/** Remember the selection a reconciled save names, so the destination history opens on it once. */
|
||||
@@ -158,15 +160,16 @@ export function mountLibrary(main,api,parentScope,{positiveId,savedState=null,on
|
||||
const visit=saved?null:shelfVisit, mountedGeneration=writeGeneration;
|
||||
// An invalidated visit keeps only filters and anchor: the stream restarts and heights are measured again.
|
||||
const restorable=visit?.state?visit:null;
|
||||
const filters={text:'',category:'',tags:[],sort:'newest',...visit?.filters,...saved};
|
||||
const filters={text:'',category:'',tags:[],sort:'newest',tagMode:'all',folder:'',...visit?.filters,...saved};
|
||||
// D153: a sort the shelf no longer offers (an older "Highest rated" visit) reads as Recently added.
|
||||
if(!LIBRARY_SORTS.some(([value])=>value===filters.sort)) filters.sort='newest';
|
||||
const section=el('section',undefined,'library-shelf');section.setAttribute('aria-label','Prompt library');
|
||||
// The shared concept toolbar (app.js toolbar(), a347–360): Tags live in "Refine this view" (D14), no Refresh (D15).
|
||||
// Its own scope, disposed after the visit is saved: removing it first would shorten the page and clamp scrollY.
|
||||
const toolbarScope=createScope();
|
||||
// D44: Filters reads "Filters · 1" while a Tags filter (from Refine or a Browse) narrows the shelf.
|
||||
const refinements=()=>filters.tags.length?1:0;
|
||||
// D44: Filters reads "Filters · N" while Refine filters (from Refine or a Browse) narrow the shelf: Tags, a Folder
|
||||
// and a Tag match other than All (#444 D149, D150) each count once.
|
||||
const refinements=()=>(filters.tags.length?1:0)+(filters.folder?1:0)+(filters.tagMode!=='all'?1:0);
|
||||
const toolbar=mountToolbar(section,{scope:toolbarScope,categories:filters.category?[filters.category]:[],sorts:LIBRARY_SORTS,
|
||||
value:{q:filters.text,category:filters.category,sort:filters.sort},activeFilters:refinements(),onChange:applyToolbar,onFilters:openFilters,
|
||||
onSaveView:trigger=>{const value=toolbar.value();onSaveView(trigger,libraryViewState({...filters,text:value.q.trim(),category:value.category,sort:value.sort}));}});
|
||||
@@ -213,7 +216,7 @@ export function mountLibrary(main,api,parentScope,{positiveId,savedState=null,on
|
||||
// Nothing follows the grid in the concept; the legacy Library stays one click away on Overview (spec §5).
|
||||
continuation.append(message,retry,more);section.append(optionsStatus,exportStatus,selection.element,caption,grid,continuation);main.append(section);
|
||||
const cards=new Map(), rowNodes=new Map(), spacers=new Map(), heights=new Map(), pendingMeasurements=new Map();
|
||||
let state,frame=0,loadFrame=0,observer,optionsRequest,focusOverride=-1;
|
||||
let state,frame=0,loadFrame=0,observer,optionsRequest,folderRequest,focusOverride=-1;
|
||||
const columns=()=>window.innerWidth>=1700?3:window.innerWidth<=700?1:2;
|
||||
const readInlineSize=()=>Math.round(grid.getBoundingClientRect().width*64)/64;
|
||||
let inlineSize=readInlineSize(), columnCount=columns(), layout={rows:[],height:0}, dirty=true, items, indexesById=new Map();
|
||||
@@ -394,7 +397,10 @@ export function mountLibrary(main,api,parentScope,{positiveId,savedState=null,on
|
||||
const hidden=hiddenSave!==null && !next.loading && !next.error && !next.hasMore && !next.items.some(item=>item.id===hiddenSave)?' The saved prompt is hidden by the current filters.':'';
|
||||
message.textContent=(next.error?'Prompt library could not be loaded. Retry.':next.loading?'Loading prompt library…':!next.items.length?'No matching prompts. Try changing the filters.':!next.hasMore?'You’re all caught up.':'')+hidden;
|
||||
message.setAttribute('role',next.error?'alert':'status');retry.hidden=!next.error;more.hidden=!next.hasMore || !!next.error;more.setAttribute('aria-disabled',String(next.loading));
|
||||
schedule();scheduleLoad();
|
||||
// #464: a loaded page is laid out now, not on the next frame: a starved renderer (a background tab, headless
|
||||
// WebKit under load) can hold frames for seconds, which left a loaded shelf empty. Measurements still flush per frame.
|
||||
if(batch && grid.isConnected) {cancelAnimationFrame(frame);flush();} else schedule();
|
||||
scheduleLoad();
|
||||
}
|
||||
const stream=createLibraryStream({api,onChange:render,restore:restorable?{filters,state:restorable.state}:undefined});
|
||||
function maybeLoad() {
|
||||
@@ -417,15 +423,26 @@ export function mountLibrary(main,api,parentScope,{positiveId,savedState=null,on
|
||||
if(Object.entries(next).every(([key,item])=>filters[key]===item)) return;
|
||||
Object.assign(filters,next);reset();
|
||||
}
|
||||
/** "Refine this view" with the Library's one extra filter, Tags (D14); Clear also clears the toolbar category. */
|
||||
/**
|
||||
* "Refine this view" with the Library's filters: the concept's Folder (#444 D150, its options read when the dialog
|
||||
* opens), Tags (D14) and Tag match (D149, D150); Clear also clears the toolbar category.
|
||||
*/
|
||||
function openFilters(trigger) {
|
||||
openRefineDialog({scope,trigger,fields:[{key:'tags'}],values:{tags:filters.tags.join(', ')},onApply:applyRefinement,
|
||||
onClear:cleared=>{toolbar.setValue({category:cleared.category});applyRefinement(cleared);}});
|
||||
const refine=openRefineDialog({scope,trigger,fields:[{key:'folder',options:libraryFolderOptions([],filters.folder)},{key:'tags'},
|
||||
{key:'tagMode',options:TAG_MATCH_OPTIONS}],values:{folder:filters.folder,tags:filters.tags.join(', '),tagMode:filters.tagMode},
|
||||
onApply:applyRefinement,onClear:cleared=>{toolbar.setValue({category:cleared.category});applyRefinement(cleared);}});
|
||||
folderRequest?.abort();const request=folderRequest=new AbortController();
|
||||
// A failed read leaves "All output folders" and the current folder; nothing else depends on it.
|
||||
api.get('/prompt_manager/library/options?include=folders',{signal:request.signal}).then(data=>{
|
||||
if(scope.signal.aborted || folderRequest!==request || !Array.isArray(data?.folders) || data.folders.some(folder=>typeof folder!=='string')) return;
|
||||
refine.setOptions('folder',libraryFolderOptions(data.folders,filters.folder));
|
||||
},()=>{});
|
||||
}
|
||||
function applyRefinement({tags,category=filters.category}) {
|
||||
const next=[...new Set(tags.split(',').map(tag=>tag.trim()).filter(Boolean))];
|
||||
if(category===filters.category && next.length===filters.tags.length && next.every((tag,index)=>tag===filters.tags[index])) return;
|
||||
filters.category=category;filters.tags=next;reset();
|
||||
function applyRefinement({tags,tagMode='all',folder='',category=filters.category}) {
|
||||
// "Prompts without tags" takes no Tags (the dialog disables them).
|
||||
const next=tagMode==='untagged'?[]:[...new Set(tags.split(',').map(tag=>tag.trim()).filter(Boolean))];
|
||||
if(category===filters.category && tagMode===filters.tagMode && folder===filters.folder && next.length===filters.tags.length && next.every((tag,index)=>tag===filters.tags[index])) return;
|
||||
filters.category=category;filters.tags=next;filters.tagMode=tagMode;filters.folder=folder;reset();
|
||||
}
|
||||
scope.listen(more,'click',()=>stream.loadMore());
|
||||
// Retry also re-reads a category list that failed (D15: Refresh is gone).
|
||||
@@ -438,7 +455,7 @@ export function mountLibrary(main,api,parentScope,{positiveId,savedState=null,on
|
||||
// A write while mounted makes this stream stale: keep only what still means something.
|
||||
// A saved view is not the plain shelf, so it never replaces the Library return state (#249 M2, as Favorites).
|
||||
if(!saved) shelfVisit=mountedGeneration!==writeGeneration?{filters:savedFilters,anchor:savedAnchor}:state?{filters:savedFilters,state,scrollY:window.scrollY,layout:{inlineSize,columns:columnCount,heights:[...heights],anchor:savedAnchor}}:null;
|
||||
toolbarScope.dispose();stream.dispose();optionsRequest?.abort();observer?.disconnect();measurementObserver?.disconnect();cancelAnimationFrame(frame);cancelAnimationFrame(loadFrame);for(const card of cards.values())removeCard(card);cards.clear();});
|
||||
toolbarScope.dispose();stream.dispose();optionsRequest?.abort();folderRequest?.abort();observer?.disconnect();measurementObserver?.disconnect();cancelAnimationFrame(frame);cancelAnimationFrame(loadFrame);for(const card of cards.values())removeCard(card);cards.clear();});
|
||||
loadCategories();if(restorable) {render(stream.state);if(restorable.state.loading || restorable.state.error)stream.loadMore();} else reset();
|
||||
}
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ import {createViewerSession} from './viewer-state.mjs';
|
||||
import {rootedFileAdapter} from './inspection-state.mjs';
|
||||
import {mountFileInspector} from './file-inspector.mjs';
|
||||
import {displayPreferences, readPreferences} from './preferences.mjs';
|
||||
import {createWorkspaceIcon} from './icons.mjs';
|
||||
|
||||
let viewerId = 0;
|
||||
// Upper bound on waiting for a closed Viewer's in-flight image view before destroying it.
|
||||
@@ -39,7 +40,7 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle
|
||||
function open(options) {
|
||||
if (scope.signal.aborted) return;
|
||||
active?.close(false);
|
||||
const {items,selectedKey,trigger = document.activeElement} = options;
|
||||
const {items,selectedKey,trigger = document.activeElement,details:detailsMode} = options;
|
||||
let mounted = false, initial, entry, currentKey, viewer = null, withoutViewer = false, mediaScope = null, closed = false;
|
||||
let originalDownload = null, inspectionState = inspectionController.state, collectionChanges = null;
|
||||
const session = createViewerSession({adapter,items,initialKey:selectedKey,onChange:(state,selected) => {
|
||||
@@ -56,7 +57,10 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle
|
||||
const heading = element('div'), title = element('h2');title.id = 'viewer-title';
|
||||
heading.append(title,element('p','muted small','One viewer · image, prompts, generation data, workflow'));
|
||||
const close = createButton('×');close.id = 'viewer-close';close.setAttribute('aria-label','Close image viewer');close.autofocus = true;
|
||||
header.append(heading,close);
|
||||
// #450 (D156): the view switch before ✕ names the view it switches to; the stored default picks the first view.
|
||||
let fullscreen = displayPreferences()?.viewerOpenMode === 'fullscreen';
|
||||
const mode = createButton('','icon-btn');mode.id = 'viewer-mode';
|
||||
header.append(heading,mode,close);
|
||||
const body = element('div','viewer-layout'), left = element('div','viewer-left');
|
||||
const stage = element('div');stage.id = 'viewer-stage';stage.tabIndex = 0;
|
||||
stage.setAttribute('role','region');stage.setAttribute('aria-label','Media preview');
|
||||
@@ -79,27 +83,37 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle
|
||||
local.listen(download,'click',event => {session.stop();if (download.getAttribute('aria-disabled') === 'true') event.preventDefault();});
|
||||
controls.append(download);
|
||||
const next = button('Next media','Next →',() => session.navigate(1));
|
||||
// D97: "Collapse viewer details by default" opens the viewer with its details hidden behind one toggle.
|
||||
const collapsed = displayPreferences()?.sidebarCollapsedByDefault === true;
|
||||
let detailsToggle = null;
|
||||
if (collapsed) {
|
||||
detailsToggle = createButton('Show details');detailsToggle.setAttribute('aria-expanded','false');
|
||||
detailsToggle.setAttribute('aria-controls','viewer-details');controls.append(detailsToggle);
|
||||
}
|
||||
// D97: "Collapse viewer details by default" opens the viewer with its details hidden behind one toggle. D155
|
||||
// (#448): `details: 'shown'` (Inspect a file) keeps the toggle but opens with the details showing. D156 (#450):
|
||||
// the toggle is also there while the view is full screen.
|
||||
const collapsible = displayPreferences()?.sidebarCollapsedByDefault === true, collapsed = collapsible && detailsMode !== 'shown';
|
||||
const detailsToggle = createButton('');detailsToggle.setAttribute('aria-controls','viewer-details');
|
||||
const position = element('p','media-position muted');position.id = `media-viewer-position-${++viewerId}`;
|
||||
position.setAttribute('role','status');
|
||||
const reason = element('p','media-reason muted');reason.id = `media-viewer-reason-${viewerId}`;
|
||||
const slideshowReason = element('p','media-slideshow-reason muted');slideshowReason.id = `media-viewer-slideshow-${viewerId}`;
|
||||
controls.append(position,reason,slideshowReason);left.append(stage,controls);
|
||||
const details = element('aside');details.id = 'viewer-details';details.setAttribute('aria-label','Image metadata');
|
||||
if (detailsToggle) {
|
||||
details.hidden = true;
|
||||
local.listen(detailsToggle,'click',() => {
|
||||
details.hidden = !details.hidden;
|
||||
detailsToggle.textContent = details.hidden ? 'Show details' : 'Hide details';
|
||||
detailsToggle.setAttribute('aria-expanded',String(!details.hidden));
|
||||
});
|
||||
details.hidden = collapsed;
|
||||
function paintToggle() {
|
||||
detailsToggle.textContent = details.hidden ? 'Show details' : 'Hide details';
|
||||
detailsToggle.setAttribute('aria-expanded',String(!details.hidden));
|
||||
}
|
||||
/** The toggle is in the toolbar only in full screen or with the collapse setting; without both the details show. */
|
||||
function syncToggle() {
|
||||
if (collapsible || fullscreen) { if (!detailsToggle.isConnected) controls.insertBefore(detailsToggle,position); }
|
||||
else { detailsToggle.remove();details.hidden = false; }
|
||||
paintToggle();
|
||||
}
|
||||
function paintMode() {
|
||||
if (fullscreen) dialog.classList.add('viewer-fullscreen'); else dialog.classList.remove('viewer-fullscreen');
|
||||
const label = fullscreen ? 'Centered view' : 'Full screen view';
|
||||
mode.setAttribute('aria-label',label);mode.title = label;
|
||||
mode.replaceChildren(createWorkspaceIcon(fullscreen ? 'minimize' : 'maximize'));
|
||||
}
|
||||
local.listen(detailsToggle,'click',() => {details.hidden = !details.hidden;paintToggle();});
|
||||
local.listen(mode,'click',() => {fullscreen = !fullscreen;paintMode();syncToggle();});
|
||||
paintMode();syncToggle();
|
||||
body.append(left,details);dialog.append(header,body);container.append(dialog);
|
||||
const localSender = {open(roles,trigger) {session.stop();sender.open(roles,trigger);}};
|
||||
const inspector = mountFileInspector(details,{api,sender:localSender,parentScope:local,variant:'full',onInteraction:() => session.stop(),onRetry:() => inspectionController.retry(),
|
||||
|
||||
@@ -21,7 +21,7 @@
|
||||
*/
|
||||
export const PREFERENCES_KEY = 'gallerySettings';
|
||||
export const WORKSPACE_PREFERENCE_KEYS = Object.freeze(['workspaceLayout', 'workspaceDensity', 'autoLoadMetadata', 'cacheMetadata',
|
||||
'imageQuality', 'videoAutoplay', 'videoMute', 'videoLoop', 'showFilePaths', 'showImageInfo', 'sidebarCollapsedByDefault']);
|
||||
'imageQuality', 'videoAutoplay', 'videoMute', 'videoLoop', 'showFilePaths', 'showImageInfo', 'sidebarCollapsedByDefault', 'viewerOpenMode']);
|
||||
|
||||
// The changes this tab could not store, overlaid on every read until storage accepts them.
|
||||
const tabOnly = new Map();
|
||||
|
||||
@@ -47,6 +47,9 @@ const FIELDS = Object.freeze({
|
||||
}),
|
||||
library: Object.freeze({
|
||||
q: text(500), category: text(100), sort: choice(['newest', 'oldest', 'name'], 'newest'), tags,
|
||||
// #444: how the Tags match ('untagged' never goes with tags; the listing refuses both) and the output folder
|
||||
// ('/'-joined, below a root).
|
||||
tag_mode: choice(['all', 'any', 'untagged'], 'all'), folder: text(1024),
|
||||
}),
|
||||
});
|
||||
|
||||
@@ -63,6 +66,8 @@ export function normalizeViewState(page, state) {
|
||||
}));
|
||||
// Favorites and a collection are never both a view's filter: the collection wins, as the listing does (D140).
|
||||
if (normalized.favorites === true && normalized.collection) normalized.favorites = false;
|
||||
// Untagged and tags are never both a view's filter either: the tags win, as the server's normalizer does.
|
||||
if (normalized.tag_mode === 'untagged' && normalized.tags.length) normalized.tag_mode = 'all';
|
||||
return normalized;
|
||||
}
|
||||
|
||||
@@ -113,10 +118,12 @@ export function imagesViewState(filters) {
|
||||
/** Library shelf filters (library.mjs `filters`) from a saved state, and the state of the current filters. */
|
||||
export function libraryFilters(state) {
|
||||
const view = normalizeViewState('library', state);
|
||||
return { text: view.q, category: view.category, sort: view.sort, tags: view.tags };
|
||||
return { text: view.q, category: view.category, sort: view.sort, tags: view.tags, tagMode: view.tag_mode, folder: view.folder };
|
||||
}
|
||||
export function libraryViewState(filters) {
|
||||
return normalizeViewState('library', { q: filters.text, category: filters.category, sort: filters.sort, tags: filters.tags });
|
||||
return normalizeViewState('library', { q: filters.text, category: filters.category, sort: filters.sort, tags: filters.tags,
|
||||
// A filter the shelf lacks (undefined) takes its key's default in normalizeViewState.
|
||||
tag_mode: filters.tagMode, folder: filters.folder });
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -13,7 +13,10 @@
|
||||
* (`showImageInfo`), "Collapse viewer details by default" (`sidebarCollapsedByDefault`) and the operation "Reset
|
||||
* display defaults", which drafts the gallery's defaults for these browser rows and touches no other key.
|
||||
*
|
||||
* - #317: a second surface, "Thumbnails", holds the server's thumbnail format, quality and size limit
|
||||
* - #450 (D156, amends D97): a "Viewer" surface after it holds "Open viewer in" (Centered→`centered`, Full
|
||||
* screen→`fullscreen`, `viewerOpenMode`) and the moved "Collapse viewer details by default"; "Reset display defaults"
|
||||
* covers both surfaces' browser rows.
|
||||
* - #317: a further surface, "Thumbnails", holds the server's thumbnail format, quality and size limit
|
||||
* (system-thumbnails.mjs).
|
||||
*
|
||||
* Controls follow the store: a value read later (the host settings, another tab's change) repaints them. A host row
|
||||
@@ -50,14 +53,20 @@ export const APPEARANCE_ROWS = Object.freeze([
|
||||
{ type: 'select', options: [['newtab', 'New tab'], ['popup', 'Popup window']] }],
|
||||
['timeout', 'Node search auto-hide (seconds)', 'Node search behavior only. Use 0 to keep results visible.', 'host', 'result_timeout',
|
||||
{ type: 'number' }],
|
||||
// D97: the legacy gallery's card and viewer settings.
|
||||
// D97: the legacy gallery's card setting (its viewer setting moved to VIEWER_ROWS, D156).
|
||||
['card-details', 'Show image details on cards', 'Show each card’s title and file details under the preview.', 'browser', 'showImageInfo', BOOLEAN],
|
||||
]);
|
||||
|
||||
/** #450 (D156): the "Viewer" surface's rows, in the same shape. */
|
||||
export const VIEWER_ROWS = Object.freeze([
|
||||
['viewer-mode', 'Open viewer in', 'Full screen fills the browser window; Centered is the framed viewer.', 'browser', 'viewerOpenMode',
|
||||
{ type: 'select', options: [['centered', 'Centered'], ['fullscreen', 'Full screen']] }],
|
||||
['viewer-details', 'Collapse viewer details by default', 'Open the image viewer with its details panel hidden.', 'browser',
|
||||
'sidebarCollapsedByDefault', BOOLEAN],
|
||||
]);
|
||||
|
||||
/** The browser keys "Reset display defaults" restores (D97). */
|
||||
export const DISPLAY_KEYS = Object.freeze(APPEARANCE_ROWS.filter(row => Array.isArray(row) && row[3] === 'browser').map(row => row[4]));
|
||||
/** The browser keys "Reset display defaults" restores (D97), on both surfaces. */
|
||||
export const DISPLAY_KEYS = Object.freeze([...APPEARANCE_ROWS, ...VIEWER_ROWS].filter(row => Array.isArray(row) && row[3] === 'browser').map(row => row[4]));
|
||||
|
||||
/** A whole number of seconds from 0 to 300 (the server's `validate_result_timeout`), else null. */
|
||||
export function resultTimeout(text) {
|
||||
@@ -104,29 +113,36 @@ function paint(input, control, value) {
|
||||
|
||||
/**
|
||||
* Mount Appearance in `container` (System's `.settings-layout`); returns the section's nodes: "Browsing & display",
|
||||
* then the server's thumbnail settings (#317, system-thumbnails.mjs).
|
||||
* "Viewer" (D156), then the server's thumbnail settings (#317, system-thumbnails.mjs).
|
||||
*/
|
||||
export function mountAppearance(container, { scope, drafts, api, activity }) {
|
||||
const surface = el('section', undefined, 'surface');
|
||||
surface.append(el('h2', 'Browsing & display'), el('p', 'One set of defaults; toolbar changes apply to the current view.'));
|
||||
const controls = [];
|
||||
for (const row of APPEARANCE_ROWS) {
|
||||
if (row === 'continuous') { surface.append(continuousRow()); continue; }
|
||||
const [id, label, description, domain, key, control] = row;
|
||||
const onChange = value => {
|
||||
if (key === 'result_timeout') {
|
||||
const seconds = resultTimeout(input.value);
|
||||
if (seconds === null) { drafts.reject(domain, key); return; }
|
||||
value = seconds;
|
||||
}
|
||||
drafts.set(domain, key, value);
|
||||
};
|
||||
const node = settingRow({ id, label, description, control, scope, onChange });
|
||||
surface.append(node);
|
||||
const input = controlOf(node, id);
|
||||
input.dataset.key = key;
|
||||
controls.push({ input, control, domain, key, text: node.children[0], note: null });
|
||||
function addRows(target, rows) {
|
||||
for (const row of rows) {
|
||||
if (row === 'continuous') { target.append(continuousRow()); continue; }
|
||||
const [id, label, description, domain, key, control] = row;
|
||||
const onChange = value => {
|
||||
if (key === 'result_timeout') {
|
||||
const seconds = resultTimeout(input.value);
|
||||
if (seconds === null) { drafts.reject(domain, key); return; }
|
||||
value = seconds;
|
||||
}
|
||||
drafts.set(domain, key, value);
|
||||
};
|
||||
const node = settingRow({ id, label, description, control, scope, onChange });
|
||||
target.append(node);
|
||||
const input = controlOf(node, id);
|
||||
input.dataset.key = key;
|
||||
controls.push({ input, control, domain, key, text: node.children[0], note: null });
|
||||
}
|
||||
}
|
||||
addRows(surface, APPEARANCE_ROWS);
|
||||
// D156: the viewer's defaults in their own surface.
|
||||
const viewer = el('section', undefined, 'surface');
|
||||
viewer.append(el('h2', 'Viewer'), el('p', 'How the image viewer opens; the button beside ✕ switches the current view.'));
|
||||
addRows(viewer, VIEWER_ROWS);
|
||||
surface.append(operationRow({ title: 'Reset display defaults',
|
||||
description: 'Restore this browser’s display defaults, then save. Other settings are kept.', label: 'Reset', scope,
|
||||
onClick: () => {
|
||||
@@ -146,5 +162,5 @@ export function mountAppearance(container, { scope, drafts, api, activity }) {
|
||||
};
|
||||
scope.cleanup(drafts.subscribe(repaint));
|
||||
repaint();
|
||||
return [surface, mountThumbnailSettings(container, { scope, drafts, api, activity })];
|
||||
return [surface, viewer, mountThumbnailSettings(container, { scope, drafts, api, activity })];
|
||||
}
|
||||
|
||||
@@ -14,6 +14,9 @@
|
||||
* it to the host draft of `gallery_root_paths`; Remove takes one out. Both wait for the one Save row.
|
||||
* - "Watch for new outputs" (`monitoring_enabled`) and "Worker threads" (`worker_threads`, a range up to the server's
|
||||
* `cpu_count`, D113) are host drafts too. "Enrich existing metadata" reviews a scan of every source.
|
||||
* - D158 (#247): "Save prompts from other workflows" (`save_foreign_runs`), a host draft below "Watch for new outputs",
|
||||
* applied by the server at once (the monitor reads it for every file); disabled, saying why, while monitoring is off,
|
||||
* unless the running monitor still watches until a restart (D128 `stopping`): then it can still be set.
|
||||
* - "Thumbnail generation → Review options" opens "Thumbnail maintenance" (thumbnail-review.mjs, S5 Task 14), which
|
||||
* starts the thumbnail Activity job.
|
||||
*
|
||||
@@ -63,6 +66,21 @@ export const LOAD_FAILED_TEXT = 'Sources could not be loaded. Your saved data ha
|
||||
export const DUPLICATE_TEXT = 'This folder is already a source.';
|
||||
export const VALIDATE_FAILED_TEXT = 'The folder could not be checked. Try again.';
|
||||
export const REFRESH_FAILED_TEXT = 'Files could not be refreshed. Try again.';
|
||||
/** D158 (#247): the row the concept does not draw, below "Watch for new outputs". */
|
||||
export const FOREIGN_RUNS = Object.freeze({ id: 'foreign-runs', key: 'save_foreign_runs', label: 'Save prompts from other workflows',
|
||||
description: 'New outputs of workflows without a PromptManager node add their prompt to the library when it is certain.',
|
||||
off: 'Applies while Watch for new outputs is on.' });
|
||||
|
||||
/**
|
||||
* D158: the row for the drafted `save_foreign_runs` (`value`) and `monitoring_enabled` (`monitoring`): disabled until
|
||||
* read, and while monitoring is off, when `note` says why (the value is kept and saved as it is). `stopping`: a source
|
||||
* is still watched until the restart (D128), so the value can still be set; the note stays.
|
||||
*/
|
||||
export function foreignRunsState({ value, monitoring, stopping = false }) {
|
||||
if (value === undefined) return { disabled: true, checked: null, note: null };
|
||||
const off = monitoring === false;
|
||||
return { disabled: off && !stopping, checked: value === true, note: off ? FOREIGN_RUNS.off : null };
|
||||
}
|
||||
|
||||
// Labels of the sources added in this page session (validate-source answers them), for the rows not saved yet.
|
||||
const addedLabels = new Map();
|
||||
@@ -267,13 +285,16 @@ export function mountSources(container, { api, activity, scope, drafts, notify =
|
||||
const monitoring = settingRow({ id: 'monitoring', label: 'Watch for new outputs',
|
||||
description: 'New files enter the same library. Existing scans do not count as new runs.', control: { type: 'toggle' },
|
||||
scope, onChange: value => drafts.set('host', 'monitoring_enabled', value) });
|
||||
const foreign = settingRow({ id: FOREIGN_RUNS.id, label: FOREIGN_RUNS.label, description: FOREIGN_RUNS.description,
|
||||
control: { type: 'toggle' }, scope, onChange: value => drafts.set('host', FOREIGN_RUNS.key, value) });
|
||||
const workers = settingRow({ id: 'workers', label: 'Worker threads',
|
||||
description: 'Balance background scans and thumbnail work against generation.', control: { type: 'range', value: 4 },
|
||||
scope, onChange: value => drafts.set('host', 'worker_threads', value) });
|
||||
const hostRows = [[monitoring, 'monitoring', 'monitoring_enabled'], [workers, 'workers', 'worker_threads']]
|
||||
const hostRows = [[monitoring, 'monitoring', 'monitoring_enabled'], [foreign, FOREIGN_RUNS.id, FOREIGN_RUNS.key],
|
||||
[workers, 'workers', 'worker_threads']]
|
||||
.map(([row, id, key]) => ({ input: controlOf(row, id), key, text: row.children[0], note: null }));
|
||||
for (const row of hostRows) row.input.dataset.key = row.key;
|
||||
surface.append(monitoring, workers,
|
||||
surface.append(monitoring, foreign, workers,
|
||||
operationRow({ scope, title: 'Enrich existing metadata', description: 'Resolve both prompts, deduplicate recipes, preserve uncertain links.',
|
||||
label: 'Review scan', onClick: scanAll }),
|
||||
operationRow({ scope, title: 'Thumbnail generation', description: 'Generate missing previews; rebuilding all is a separate explicit option.',
|
||||
@@ -348,6 +369,10 @@ export function mountSources(container, { api, activity, scope, drafts, notify =
|
||||
for (const row of hostRows) {
|
||||
const value = drafts.get('host', row.key);
|
||||
row.input.disabled = value === undefined;
|
||||
if (row.key === FOREIGN_RUNS.key) {
|
||||
paintForeign(row, value, unavailable);
|
||||
continue;
|
||||
}
|
||||
if (row.key === 'worker_threads') {
|
||||
// D113: the range ends at the server's core count (the concept's fixed 16).
|
||||
const cpus = drafts.get('host', 'cpu_count');
|
||||
@@ -360,6 +385,24 @@ export function mountSources(container, { api, activity, scope, drafts, notify =
|
||||
}
|
||||
}
|
||||
|
||||
// D158: disabled while monitoring is drafted off, with the reason under the description.
|
||||
function paintForeign(row, value, unavailable) {
|
||||
const stopping = sources?.effective?.some(source => source.watch === 'stopping') === true;
|
||||
const state = foreignRunsState({ value, monitoring: drafts.get('host', 'monitoring_enabled'), stopping });
|
||||
row.input.disabled = state.disabled;
|
||||
if (state.checked !== null) row.input.checked = state.checked;
|
||||
const note = unavailable && value === undefined ? HOST_UNAVAILABLE : state.note;
|
||||
if (note && row.note?.textContent !== note) {
|
||||
row.note?.remove();
|
||||
row.note = el('p', note, 'small');
|
||||
row.note.id = `${FOREIGN_RUNS.id}-note`;
|
||||
row.text.append(row.note);
|
||||
} else if (!note && row.note) { row.note.remove(); row.note = null; }
|
||||
// The note is the switch's description, so a screen reader says why it is disabled.
|
||||
if (row.note) row.input.setAttribute('aria-describedby', row.note.id);
|
||||
else row.input.removeAttribute('aria-describedby');
|
||||
}
|
||||
|
||||
// The sources changed since this page read them (a refused per-source scan): read them and the saved values again.
|
||||
function reread() {
|
||||
load();
|
||||
@@ -378,6 +421,7 @@ export function mountSources(container, { api, activity, scope, drafts, notify =
|
||||
if (answer === STALE || scope.signal.aborted) return;
|
||||
sources = answer?.sources ?? null;
|
||||
paintList();
|
||||
paintControls();
|
||||
}
|
||||
|
||||
// A save (or a reread after a conflict) changed the saved sources or monitoring: read their state again. The first
|
||||
|
||||
@@ -26,6 +26,12 @@ const UNCONFIRMED_STATUS = new Set([502, 503, 504]);
|
||||
// Phases that may send a command; done, storage-changed and unknown-outcome need reset().
|
||||
const READY = new Set(['idle', 'previewed', 'write-error', 'conflict']);
|
||||
const IMPACT_FIELDS = ['positive_prompts', 'other_prompts', 'negative_variants', 'archived_negative_variants'];
|
||||
/** #444 tag sorts as [value, label]: `created` (today's order) first, then the legacy tag manager's four. */
|
||||
export const TAG_SORTS = Object.freeze([['created', 'Oldest first'], ['name', 'Name A–Z'], ['name-desc', 'Name Z–A'],
|
||||
['prompts', 'Most prompts'], ['prompts-asc', 'Fewest prompts']].map(pair => Object.freeze(pair)));
|
||||
// The route's cap on a tag search (database/vocabulary.py _MAX_SEARCH).
|
||||
export const TAG_SEARCH_MAX = 200;
|
||||
const DEFAULT_QUERY = Object.freeze({search: '', sort: 'created'});
|
||||
|
||||
const object = value => value !== null && typeof value === 'object' && !Array.isArray(value);
|
||||
const id = value => Number.isSafeInteger(value) && value > 0;
|
||||
@@ -45,9 +51,23 @@ function storageChanged() {
|
||||
// Duck-typed by name: this ES module cannot import the UMD api-client.js.
|
||||
function isApiError(error) {return error instanceof Error && error.name === 'ApiError' && Number.isInteger(error.status);}
|
||||
|
||||
function validatePage(page, kind, cursor) {
|
||||
/** A tags query, trimmed; categories take none. Throws on anything the route would refuse. */
|
||||
function tagQuery(kind, query) {
|
||||
if (query === undefined) return DEFAULT_QUERY;
|
||||
requireValue(kind === 'tags' && object(query), 'Only tags take a search and sort');
|
||||
const {search = '', sort = 'created'} = query;
|
||||
requireValue(typeof search === 'string' && search.trim().length <= TAG_SEARCH_MAX, 'Invalid tag search');
|
||||
requireValue(TAG_SORTS.some(([value]) => value === sort), 'Invalid tag sort');
|
||||
return Object.freeze({search: search.trim(), sort});
|
||||
}
|
||||
|
||||
function validatePage(page, kind, cursor, query) {
|
||||
const rows = page?.[kind];
|
||||
const {key, valid} = KINDS[kind];
|
||||
// A search or sort the server did not echo was not applied (an older server): never show that list as filtered.
|
||||
const plain = query.search === '' && query.sort === 'created' && page?.search === undefined && page?.sort === undefined;
|
||||
const applied = plain || (page?.search === query.search && page?.sort === query.sort);
|
||||
requireValue(applied && optionalCount(page?.untagged_count), `Unconfirmed ${kind} search`);
|
||||
requireValue(object(page) && page.success === true && epochOf(page.library_epoch) && Array.isArray(rows) &&
|
||||
rows.every(row => object(row) && valid(row) && count(row.prompt_count)) &&
|
||||
new Set(rows.map(key)).size === rows.length && count(page.total) && typeof page.has_more === 'boolean' &&
|
||||
@@ -57,12 +77,18 @@ function validatePage(page, kind, cursor) {
|
||||
}
|
||||
|
||||
/** One bounded, append-only list of tags or categories. */
|
||||
export function createVocabularyStream({api, kind, onChange = () => {}}) {
|
||||
/**
|
||||
* One bounded, append-only list of tags or categories. A tags stream takes `query` (`{search, sort}`, #444) and
|
||||
* `reset(query)` replaces it; `state.untaggedCount` is the last count a tags page reported (kept across resets so the
|
||||
* Untagged prompts row does not flicker), `null` before any.
|
||||
*/
|
||||
export function createVocabularyStream({api, kind, onChange = () => {}, query: initialQuery}) {
|
||||
requireValue(Object.hasOwn(KINDS, kind), 'Unknown vocabulary kind');
|
||||
const {key} = KINDS[kind];
|
||||
const initial = () => ({items: Object.freeze([]), total: 0, cursor: null, hasMore: true, loading: false, error: null,
|
||||
epoch: null, resetRequired: false});
|
||||
let state = Object.freeze(initial());
|
||||
let query = tagQuery(kind, initialQuery);
|
||||
const initial = untaggedCount => ({items: Object.freeze([]), total: 0, cursor: null, hasMore: true, loading: false, error: null,
|
||||
epoch: null, resetRequired: false, query, untaggedCount});
|
||||
let state = Object.freeze(initial(null));
|
||||
let request = null, disposed = false, seen = new Set();
|
||||
const emit = patch => {state = Object.freeze({...state, ...patch}); onChange(state);};
|
||||
|
||||
@@ -70,13 +96,16 @@ export function createVocabularyStream({api, kind, onChange = () => {}}) {
|
||||
if (disposed || state.loading || !state.hasMore || state.resetRequired) return 0;
|
||||
const current = request = new AbortController();
|
||||
const cursor = state.cursor;
|
||||
const query = new URLSearchParams({limit: String(LIMIT)});
|
||||
if (cursor !== null) query.set('cursor', cursor);
|
||||
const asked = query;
|
||||
const params = new URLSearchParams({limit: String(LIMIT)});
|
||||
if (asked.search) params.set('search', asked.search);
|
||||
if (asked.sort !== 'created') params.set('sort', asked.sort);
|
||||
if (cursor !== null) params.set('cursor', cursor);
|
||||
emit({loading: true, error: null});
|
||||
try {
|
||||
const page = await api.get(`${BASE}/${kind}?${query}`, {signal: current.signal, cache: 'no-store'});
|
||||
const page = await api.get(`${BASE}/${kind}?${params}`, {signal: current.signal, cache: 'no-store'});
|
||||
if (disposed || current !== request) return 0;
|
||||
validatePage(page, kind, cursor);
|
||||
validatePage(page, kind, cursor, asked);
|
||||
request = null;
|
||||
if (state.epoch !== null && page.library_epoch !== state.epoch) {
|
||||
emit({loading: false, error: storageChanged(), resetRequired: true});
|
||||
@@ -85,7 +114,8 @@ export function createVocabularyStream({api, kind, onChange = () => {}}) {
|
||||
const incoming = page[kind].filter(row => !seen.has(key(row))).map(row => Object.freeze({...row}));
|
||||
incoming.forEach(row => seen.add(key(row)));
|
||||
emit({items: incoming.length ? Object.freeze([...state.items, ...incoming]) : state.items, total: page.total,
|
||||
cursor: page.next_cursor, hasMore: page.has_more, loading: false, error: null, epoch: page.library_epoch});
|
||||
cursor: page.next_cursor, hasMore: page.has_more, loading: false, error: null, epoch: page.library_epoch,
|
||||
untaggedCount: page.untagged_count ?? state.untaggedCount});
|
||||
return incoming.length;
|
||||
} catch (error) {
|
||||
if (!disposed && current === request) {request = null; emit({loading: false, error});}
|
||||
@@ -96,13 +126,14 @@ export function createVocabularyStream({api, kind, onChange = () => {}}) {
|
||||
return {
|
||||
get state() {return state;},
|
||||
loadMore,
|
||||
/** Drop every loaded row and read again from the first page. */
|
||||
reset() {
|
||||
/** Drop every loaded row and read again from the first page, with `next` as the new query when given. */
|
||||
reset(next) {
|
||||
if (next !== undefined) query = tagQuery(kind, next);
|
||||
if (disposed) return Promise.resolve(0);
|
||||
request?.abort();
|
||||
request = null;
|
||||
seen = new Set();
|
||||
emit(initial());
|
||||
emit(initial(state.untaggedCount));
|
||||
return loadMore();
|
||||
},
|
||||
dispose() {disposed = true; request?.abort(); request = null;},
|
||||
@@ -254,6 +285,11 @@ export function browseFilters(kind, value) {
|
||||
: {text: '', category: value, tags: [], sort: 'newest'};
|
||||
}
|
||||
|
||||
/** Library filters for the Untagged prompts row's Browse (#444): parents that no tag's Browse returns. */
|
||||
export function untaggedBrowseFilters() {
|
||||
return {text: '', category: '', tags: [], sort: 'newest', tagMode: 'untagged'};
|
||||
}
|
||||
|
||||
/**
|
||||
* A category row's count (#332 D143): "N positive prompts" when no image holds it as its own category, "N positive
|
||||
* prompts · M images" when both do, "M images" when only images do (stored rows, files on disk or not).
|
||||
|
||||
@@ -4,12 +4,18 @@
|
||||
* Composition, labels and order follow the approved concept: Auto-tag images
|
||||
* opens the Tag image dialog for the image selected in Images (aria-disabled
|
||||
* without one, D115) and the Images cells count the files carrying each tag
|
||||
* (#249 C2). The recorded deviation left is bounded lists with Load more. Rows
|
||||
* are real vocabulary pages from vocabulary-state.mjs; Browse hands the exact
|
||||
* constraint to the existing Library filters.
|
||||
* (#249 C2). The recorded deviations are bounded lists with Load more and (D149,
|
||||
* #444) the Tag vocabulary's Untagged prompts row and its tag search and sort,
|
||||
* drawn with the concept's `.operation` and toolbar tokens. Rows are real
|
||||
* vocabulary pages from vocabulary-state.mjs; Browse hands the exact constraint
|
||||
* to the existing Library filters.
|
||||
*/
|
||||
import { createButton } from '../shared/toolkit/core.js';
|
||||
import { browseFilters, categoryBrowseTarget, categoryRowText, createVocabularyCommand, createVocabularyStream } from './vocabulary-state.mjs';
|
||||
import { TAG_SEARCH_MAX, TAG_SORTS, browseFilters, categoryBrowseTarget, categoryRowText, createVocabularyCommand, createVocabularyStream,
|
||||
untaggedBrowseFilters } from './vocabulary-state.mjs';
|
||||
import { SEARCH_DELAY } from './toolbar.mjs';
|
||||
import { createWorkspaceIcon } from './icons.mjs';
|
||||
import { countOf } from './plural.mjs';
|
||||
import { browseLibrary, invalidateLibraryAfterWrite } from './library.mjs';
|
||||
import { createVocabularyDialog } from './vocabulary-dialog.mjs';
|
||||
import { pageHeading } from './heading.mjs';
|
||||
@@ -20,6 +26,9 @@ import { showWorkspaceToast } from './toast.mjs';
|
||||
/** D115: why Auto-tag images (and System › Auto-tagging's Preview tagging review) only explains without a selection. */
|
||||
export const NO_SELECTION = 'Select an image in Images first.';
|
||||
|
||||
const NO_MATCH = 'No tags match this search.';
|
||||
// The Untagged prompts row before the first tags page reports its count.
|
||||
const UNTAGGED_PENDING = 'Positive prompts without a tag';
|
||||
const COPY = {
|
||||
tags: {empty: 'No tags yet.', error: 'Tags could not be loaded. Nothing was changed.', more: 'Load more tags'},
|
||||
categories: {empty: 'No categories yet.', error: 'Categories could not be loaded. Nothing was changed.', more: 'Load more categories'},
|
||||
@@ -75,7 +84,7 @@ function categoryRow(category) {
|
||||
* in view (stacked layout below 700px), pages do not chain on their own. Before any row arrives (loading, failed) the list shows no rows, so a
|
||||
* failed read never looks like zero counts.
|
||||
*/
|
||||
function mountList(scope, api, kind, {content, rowsHost, renderRow, after}) {
|
||||
function mountList(scope, api, kind, {content, rowsHost, renderRow, after, query}) {
|
||||
const copy = COPY[kind];
|
||||
const empty = el('p', copy.empty, 'empty');
|
||||
const continuation = el('div', undefined, 'vocabulary-continuation');
|
||||
@@ -115,6 +124,7 @@ function mountList(scope, api, kind, {content, rowsHost, renderRow, after}) {
|
||||
const failed = Boolean(state.error);
|
||||
content.hidden = !rendered;
|
||||
empty.hidden = Boolean(rendered || state.loading || failed || state.hasMore);
|
||||
empty.textContent = state.query?.search ? NO_MATCH : copy.empty;
|
||||
status.textContent = state.loading ? 'Loading…' : '';
|
||||
status.hidden = !state.loading;
|
||||
alert.hidden = !failed;
|
||||
@@ -129,7 +139,7 @@ function mountList(scope, api, kind, {content, rowsHost, renderRow, after}) {
|
||||
continuation.hidden = !(state.loading || failed || state.hasMore);
|
||||
for (const listener of listeners) listener(state);
|
||||
}
|
||||
const stream = createVocabularyStream({api, kind, onChange: render});
|
||||
const stream = createVocabularyStream({api, kind, onChange: render, query});
|
||||
scope.listen(more, 'click', () => stream.loadMore());
|
||||
scope.listen(retry, 'click', () => (stream.state.resetRequired ? stream.reset() : stream.loadMore()));
|
||||
let observer = null;
|
||||
@@ -156,6 +166,38 @@ function mountList(scope, api, kind, {content, rowsHost, renderRow, after}) {
|
||||
return {stream, itemOf: row => items.get(row), rowFor, subscribe};
|
||||
}
|
||||
|
||||
/**
|
||||
* The tag search and sort (D149): the concept toolbar's `search.toolbar` > `label.search-field` (search icon and a
|
||||
* search input) and a `select`, built here because the shared toolbar owns the page-level ids. A visually hidden
|
||||
* status announces the result of a changed search or sort.
|
||||
*/
|
||||
function tagTools() {
|
||||
const root = document.createElement('search');
|
||||
root.className = 'toolbar vocabulary-tools';
|
||||
const label = el('label', undefined, 'search-field');
|
||||
label.htmlFor = 'tag-search';
|
||||
const search = document.createElement('input');
|
||||
search.id = 'tag-search';
|
||||
search.type = 'search';
|
||||
search.placeholder = 'Search tags…';
|
||||
search.maxLength = TAG_SEARCH_MAX;
|
||||
search.setAttribute('aria-label', 'Search tags');
|
||||
label.append(createWorkspaceIcon('search'), search);
|
||||
const sort = document.createElement('select');
|
||||
sort.id = 'tag-sort';
|
||||
sort.setAttribute('aria-label', 'Sort tags');
|
||||
for (const [value, text] of TAG_SORTS) {
|
||||
const option = el('option', text);
|
||||
option.value = value;
|
||||
sort.append(option);
|
||||
}
|
||||
const status = el('span', '', 'ca-visually-hidden');
|
||||
status.setAttribute('role', 'status');
|
||||
root.append(label, sort, status);
|
||||
return {root, search, sort, read: () => ({search: search.value, sort: sort.value}),
|
||||
announce: text => { if (status.textContent !== text) status.textContent = text; }};
|
||||
}
|
||||
|
||||
/** Mount the screen into `main` below the shell's heading; `scope` owns every request, listener and observer. */
|
||||
export function mountVocabulary(main, api, scope) {
|
||||
main.classList.add('vocabulary-main');
|
||||
@@ -188,8 +230,11 @@ export function mountVocabulary(main, api, scope) {
|
||||
const body = el('tbody');
|
||||
table.append(el('caption', 'Reusable tags'), head, body);
|
||||
wrap.append(table);
|
||||
// D149 (#444): the parents no tag's Browse returns, as a concept `.operation` row, then the tag search and sort.
|
||||
const untagged = operation('Untagged prompts', UNTAGGED_PENDING, 'Browse');
|
||||
const tools = tagTools();
|
||||
tagSurface.append(el('h2', 'Tag vocabulary'),
|
||||
el('p', 'Category/tag autocomplete uses this same vocabulary across nodes and dialogs.'), wrap);
|
||||
el('p', 'Category/tag autocomplete uses this same vocabulary across nodes and dialogs.'), untagged.row, tools.root, wrap);
|
||||
|
||||
const categorySurface = el('section', undefined, 'surface');
|
||||
const categoryRows = el('div', undefined, 'vocabulary-categories');
|
||||
@@ -200,7 +245,17 @@ export function mountVocabulary(main, api, scope) {
|
||||
grid.append(tagSurface, categorySurface);
|
||||
main.append(grid);
|
||||
|
||||
const tags = mountList(scope, api, 'tags', {content: wrap, rowsHost: body, renderRow: tagRow, after: wrap});
|
||||
const tags = mountList(scope, api, 'tags', {content: wrap, rowsHost: body, renderRow: tagRow, after: wrap, query: tools.read()});
|
||||
let announcing = false;
|
||||
tags.subscribe(state => {
|
||||
if (state.untaggedCount !== null) untagged.row.querySelector('p').textContent = countOf(state.untaggedCount, 'positive prompt');
|
||||
// A search or sort the person changed is announced once its first page lands (or fails, which the alert says).
|
||||
// A reset first emits its empty initial state (no epoch yet), then the read: wait for the page or the failure.
|
||||
if (!announcing || state.loading || (state.epoch === null && !state.error)) return;
|
||||
announcing = false;
|
||||
if (!state.error) tools.announce(state.total ? countOf(state.total, 'tag') : state.query.search ? NO_MATCH : COPY.tags.empty);
|
||||
});
|
||||
scope.listen(untagged.button, 'click', () => browseLibrary(untaggedBrowseFilters()));
|
||||
const categories = mountList(scope, api, 'categories', {content: categoryRows, rowsHost: categoryRows, renderRow: categoryRow, after: categoryRows});
|
||||
scope.listen(body, 'click', event => {
|
||||
const button = event.target.closest('button[data-action="browse"]');
|
||||
@@ -217,11 +272,20 @@ export function mountVocabulary(main, api, scope) {
|
||||
});
|
||||
|
||||
let settled = Promise.resolve();
|
||||
/** Re-read both lists from page one: after any command, rows can move between pages. */
|
||||
/** Re-read both lists from page one: after any command, rows can move between pages. The tag search and sort stay. */
|
||||
function reload() {
|
||||
settled = Promise.all([tags.stream.reset(), categories.stream.reset()]).then(() => undefined);
|
||||
return settled;
|
||||
}
|
||||
/** A changed tag search or sort reads the tags from page one; the categories are not affected. */
|
||||
function searchTags() {
|
||||
const next = tools.read(), current = tags.stream.state.query;
|
||||
if (next.search.trim() === current.search && next.sort === current.sort) return;
|
||||
announcing = true;
|
||||
settled = tags.stream.reset(next).then(() => undefined);
|
||||
}
|
||||
scope.listen(tools.search, 'input', scope.debounce(searchTags, SEARCH_DELAY));
|
||||
scope.listen(tools.sort, 'change', searchTags);
|
||||
/** A guarded command for one dialog; any outcome that may have changed storage re-reads the lists and the Library shelf. */
|
||||
function createCommand(onChange) {
|
||||
return createVocabularyCommand({api, onChange, onInvalidate: event => {
|
||||
|
||||
@@ -24,7 +24,8 @@
|
||||
import { createButton, createSwitch } from '../shared/toolkit/core.js';
|
||||
import { noun } from './plural.mjs';
|
||||
|
||||
const FIELD_LABELS = Object.freeze({ source: 'Source', folder: 'Folder', media: 'Media', tags: 'Tags' });
|
||||
// `tagMode` (#444, D149, D150): the Library's Tag match select after Tags; its `untagged` value disables Tags.
|
||||
const FIELD_LABELS = Object.freeze({ source: 'Source', folder: 'Folder', media: 'Media', tags: 'Tags', tagMode: 'Tag match' });
|
||||
const FIELD_ORDER = Object.keys(FIELD_LABELS);
|
||||
|
||||
let shell = null;
|
||||
@@ -179,7 +180,7 @@ export function openRefineDialog({ scope, fields, values = {}, onApply, onClear,
|
||||
requireScope(scope);
|
||||
requireCallbacks({ onApply, onClear });
|
||||
if (!Array.isArray(fields) || fields.some(item => !FIELD_ORDER.includes(item?.key))) {
|
||||
throw new TypeError('Refine fields are source, folder, media and tags');
|
||||
throw new TypeError('Refine fields are source, folder, media, tags and tagMode');
|
||||
}
|
||||
const controls = FIELD_ORDER.flatMap(key => {
|
||||
const item = fields.find(candidate => candidate.key === key);
|
||||
@@ -199,6 +200,12 @@ export function openRefineDialog({ scope, fields, values = {}, onApply, onClear,
|
||||
for (const { key, control } of controls) {
|
||||
if (key !== 'tags') control.addEventListener('change', () => wanted.set(key, control.value));
|
||||
}
|
||||
// "Prompts without tags" takes no Tags: the field is disabled (and Apply reports it empty) while it is chosen.
|
||||
const tagsField = controls.find(item => item.key === 'tags');
|
||||
const tagMode = controls.find(item => item.key === 'tagMode');
|
||||
const followTagMode = () => { if (tagsField && tagMode) tagsField.control.disabled = tagMode.control.value === 'untagged'; };
|
||||
tagMode?.control.addEventListener('change', followTagMode);
|
||||
followTagMode();
|
||||
const source = controls.find(item => item.key === 'source');
|
||||
const folder = controls.find(item => item.key === 'folder' && typeof item.optionsFor === 'function');
|
||||
const followSource = (chosen = folder.control.value) => {
|
||||
@@ -211,7 +218,7 @@ export function openRefineDialog({ scope, fields, values = {}, onApply, onClear,
|
||||
source.control.addEventListener('change', () => { followSource(); wanted.set('folder', folder.control.value); });
|
||||
}
|
||||
const read = () => Object.fromEntries(controls.map(({ key, control }) =>
|
||||
[key, key === 'tags' ? control.value.trim() : control.value]));
|
||||
[key, key === 'tags' ? (control.disabled ? '' : control.value.trim()) : control.value]));
|
||||
|
||||
const clear = createButton('Clear filters');
|
||||
const apply = createButton('Apply filters', 'ca-primary');
|
||||
@@ -219,6 +226,7 @@ export function openRefineDialog({ scope, fields, values = {}, onApply, onClear,
|
||||
// The cleared Source's folders first, so the cleared Folder value is one of them.
|
||||
if (source && folder) { source.control.value = source.cleared; followSource(); }
|
||||
for (const { control, cleared } of controls) control.value = cleared;
|
||||
followTagMode();
|
||||
const reset = read();
|
||||
finish();
|
||||
onClear({ ...reset, category: '' });
|
||||
|
||||
@@ -187,6 +187,13 @@ main:focus-visible { outline: 2px solid var(--ca-focus); outline-offset: 4px; }
|
||||
/* #249 D34: under 200% page zoom the concept's 94dvh outgrows the viewport; the dialog keeps within it. */
|
||||
.ca-tools #viewer-dialog { max-height: min(94dvh, 94%); /* D34 */ }
|
||||
#viewer-dialog .dialog-header > div { min-width: 0; }
|
||||
/* #450 D156: the view switch sits before ✕ (8px apart; the header's gap is the concept's 18px), and Full screen fills the
|
||||
browser window edge to edge (no outer margin), beating the concept's box, D34's max-height and the ≤700px height.
|
||||
At ≤700px the viewer is already full-bleed, so the switch hides there. */
|
||||
#viewer-dialog .dialog-header > #viewer-mode { margin-inline-start: auto; margin-inline-end: -10px; }
|
||||
.ca-tools #viewer-dialog.viewer-fullscreen[open] { inset: 0; width: auto; height: auto; max-width: none; max-height: none; margin: 0; border: 0; border-radius: 0; }
|
||||
/* `!important` only to beat the concept's `.icon-btn { display: inline-grid !important }`. */
|
||||
@media (max-width: 700px) { #viewer-dialog .dialog-header > #viewer-mode { display: none !important; } }
|
||||
#viewer-title { overflow-wrap: anywhere; /* D34 */ }
|
||||
#viewer-dialog, #viewer-dialog::backdrop { overscroll-behavior: contain; }
|
||||
#viewer-details { overscroll-behavior: contain; /* D34 */ }
|
||||
|
||||
Reference in New Issue
Block a user