Compare commits

..
Author SHA1 Message Date
Vito Sansevero d2ff73d299 test(system): match the scan scope keys beside the review counters 2026-10-23 13:59:23 -07:00
Vito Sansevero 40cecbced2 test(scan): pin only the per-source scope keys beside the review counters 2026-10-23 13:41:08 -07:00
Vito Sansevero f9a781c30a Merge remote-tracking branch 'origin/main' into feat/249-needs-review-backend
# Conflicts:
#	tests/test_api_admin.py
2026-10-23 13:34:28 -07:00
Vito Sansevero 18d81c9d96 fix(review): never queue an ambiguous file with nothing to choose
The maintainer's live queue held 39 items whose candidates all read
"unresolved" ("No positive / No negative"). An ambiguous result is now
reviewable only when at least two candidates are resolved with non-blank
positive text. With none the file could not be traced; with exactly one,
the untraced branches may still have made the file, so that text is not
certain either. Both are reported unresolved, never linked and never
queued (also the A1111 two-delimiter case, which has no candidates).

A rescan removes the pending items of such files whose own evidence
offers no choice, with the same safe delete as the linked settle
(pending, revision 0, no decision row). A limited item (over 16 candidates) that holds at least two
candidates is kept. The job scope gains review_untraced and the log
"= N images left Needs review · nothing to choose".
2026-10-23 13:03:10 -07:00
Vito Sansevero 9e125ee4a3 Merge remote-tracking branch 'origin/main' into feat/249-needs-review-backend 2026-10-23 11:09:28 -07:00
Vito Sansevero 95f88d3966 fix(review): queue only ambiguous files that nothing links, and heal the queue
The maintainer's live scan (19,908 files, scope "new") reported 4,677 files
under Needs review, but the queue held 10,232 items: every file the strict
resolver found ambiguous was queued, including the ones the legacy
browser-parser fallback then linked (all sampled "added" rows) and the ones
already linked to a known prompt (135 of 200 sampled "known" rows).

A file is now queued only when its final outcome is ambiguous and unlinked:
the resolver finds it ambiguous, the fallback links nothing and no saved row
links it already. An ambiguous file that a saved row links is reported as
unchanged ("Already linked to its prompt"), not as Needs review.

The queue heals itself. After each batch, the scan removes the pending items
of every listed file (read, or skipped as known) that a saved row now links.
The monitor does the same for each ambiguous output it links, and never
queues one. Only an undecided item is removed (pending, revision 0, no
decision row), so a decision is never deleted or changed and the schema
needs no new state.

The scan job's scope.review_items now counts this scan's files that are in
the pending queue after it, new or already there, so it agrees with the
review group's ambiguous rows and with the queue total. review_added and
review_settled are new, and the log gains "already in Needs review" and
"left Needs review · now linked" lines. A duplicate whose item is decided is
reported as "decided" and is not counted.
2026-10-23 09:41:27 -07:00
Vito Sansevero 9bfb5992b3 Merge remote-tracking branch 'origin/main' into feat/249-needs-review-backend
# Conflicts:
#	py/api/system.py
#	tests/test_system_api.py
2026-10-23 09:24:07 -07:00
Vito Sansevero 519ab59d71 fix(system): answer the Overview and Statistics only after every read ended
asyncio.gather answered at the first failed read while its sibling
executor reads were still inside SQLite. In the suite the test teardown
then deleted the database under the running read, and py3.10 on Ubuntu
segfaulted in get_statistics (test_statistics_failure_is_an_error_envelope).
Both routes now wait for all their reads before answering.
2026-10-23 07:45:57 -07:00
Vito Sansevero 369f38be9a fix(review): show the pending count on the System Overview review row
The Overview's review row now reads "N ambiguous image links" with the
concept's line, from needs_review.pending. Its Review metadata action
stays aria-disabled until the Needs review page ships. The system specs
and their concept comparison follow the real count.
2026-10-23 07:11:56 -07:00
Vito Sansevero e6ed2914e1 fix(review): store non-finite evidence as null and count pending items
canonical_evidence now follows main's non-finite policy (#303): NaN and
±Infinity are written as null instead of raising, so a NaN-stamped graph
never turns a review item limited. The System Overview reports the review
queue's pending total in needs_review.pending, read in the same snapshot
as the Library counts.
2026-10-23 06:55:31 -07:00
Vito Sansevero 878612c22d Merge remote-tracking branch 'origin/main' into feat/249-needs-review-backend
# Conflicts:
#	database/pair_ingestion.py
#	py/api/__init__.py
2026-10-23 06:55:29 -07:00
Vito Sansevero 4e5c810f96 test(backup): wait for the restore, not a 5 s timer, in the thread test
test_database_is_usable_from_every_thread_after_restore let its worker
wait 5 s for the restore. On a stalled Windows runner (py3.11, where the
previous restore test also took 25 s instead of 0.3 s) the restore took
longer, the wait timed out, and the worker read the old database before
the restore ran. The worker now waits for the event under a 120 s bound
that only a hung test reaches, reports a timeout as an error, and the
main thread releases it even when the restore raises. With a restore
delayed by 6 s the old test fails and the new one passes.
2026-10-23 04:05:44 -07:00
Vito Sansevero fcf6445de0 fix(review): refuse monitor recording beside record-changing jobs
System S1 (#289) gives every record change a registry admission class. The
scan already records Needs review items under its own records job. The
monitor's recording lasts one file and is not a job, so it now checks the
records admission: while a records or guarded job or hold runs it records
nothing, logs that the file waits for the next scan, and answers deferred.
The fallback link stands and the next "new" scan reads the ambiguous file
again, because a fallback link is not a resolved one.

JobRegistry.blocker and active iterate a snapshot of the jobs, since the
monitor's worker thread now asks them too.
2026-10-23 03:43:21 -07:00
Vito Sansevero 0f3559bedf Merge remote-tracking branch 'origin/main' into feat/249-needs-review-backend
# Conflicts:
#	py/api/__init__.py
#	tests/test_api_core.py
2026-10-23 03:34:00 -07:00
Vito Sansevero 7f1b5b64bd fix(review): record the review fix round and the skip's residual risks in TESTING.md
The Needs review paragraphs now state the rescan skip's residual risks (an
in-place rewrite that keeps size and mtime_ns is never re-queued; a deleted
or moved file leaves its pending item counted), the physical identity rule
for Apply, the candidate-only recheck and shared resolver version, the
spelling refresh, background capture admission, the final shutdown close,
the stale-cursor answer and the measured startup validation cost.
2026-10-22 17:42:01 -07:00
Vito Sansevero f00933ee16 refactor(admin): move Needs review recording into its own mixin
_record_scanned_review, _record_review_sync, record_review_file and
_log_review_items move unchanged from py/api/admin.py to
py/api/review_recording.py (ReviewRecordingMixin), keeping admin.py from
growing further past the size ceiling. No behaviour change.
2026-10-22 17:41:15 -07:00
Vito Sansevero beaba9ae84 fix(review): make only the shutdown cleanup final
The previous commit made every close of the capture store final, which
also refused the explicit closes that swap in a fresh store. Only the host's
on_cleanup hook is final now; an observe after it answers a coded capture
error instead of an unhandled 500.
2026-10-22 17:41:14 -07:00
Vito Sansevero ab47d7529b fix(review): final capture store close and recoverable stale cursors
Closing the capture store at shutdown now runs under the store lock and is
final: a monitor recording or scan batch still in flight gets a capture
error instead of creating a new temp directory and restore subscription
after the cleanup.

A well-formed queue cursor this process cannot verify (opened before a
ComfyUI restart, whose cursor secret is per process) now answers 409
review-conflict with the current total, like any other stale page, instead
of an uncoded 400. Malformed cursors stay 400.
2026-10-22 17:38:09 -07:00
Vito Sansevero b84e6ea111 fix(review): refresh the recorded spelling when a file is seen under a new one
A case-only rename on Windows or macOS keeps size and mtime_ns, so the
rescan skipped the file and the item kept the old spelling. Its target was
then formatted from a name the directory no longer lists, the capture
refused it, and the item stayed pending with no way to apply it. The skip
path and a duplicate insert now record the current spelling for every item
at the locator and move the review index revision; item revisions do not
change, so an open decision stays valid. Tested under case-folding
semantics on every platform.
2026-10-22 17:37:00 -07:00
Vito Sansevero fe5a0017a9 fix(review): keep one capture worker for interactive requests
Needs review recording from the scan and the monitor shared the capture
store's two workers with the Tag dialog and Apply, so a busy scan could
answer a user's preview or decision with "Too many previews are open".
Background captures now never take the last worker or capture slot, and
wait up to two seconds for room before counting the file as not added.
Interactive captures are admitted or refused at once, as before.
2026-10-22 17:35:06 -07:00
Vito Sansevero d24f3f989f fix(review): validate stored review evidence once per startup
Startup used to run the full review validation twice (the early guard and
the install) and the annotation group's row checks inside each, so every
evidence document was parsed and re-encoded twice. The early guard is now
catalog-only (a review group without annotation tables), and the install
validates the group once, trusting the annotation group it has just
validated. The evidence predicate no longer pattern-scans every document in
SQL, because the streamed canonical pass proves it, and the codec is built
once.

Measured on a database with 40,000 items of 4 KiB evidence: startup open
went from 7.7 s to 2.5 s. A test pins one evidence pass per reopen.
Restore and verify_database_file keep the full validation.
2026-10-22 17:33:46 -07:00
Vito Sansevero 32cc071d07 fix(review): Apply sees the file under every spelling and rechecks only its choice
Apply now finds the legacy rows of the same physical file however its root
is reached. The roots are real paths while the monitor links under the
directory it watches, so a symlinked or junctioned root, subst or a mapped
drive used to hide the monitor's row and Apply added a second, differently
linked row. Same-name rows are resolved to their physical identity
(normcase(realpath), else samefile); the dependency fingerprint covers all
of them, two rows refuse the choice, one row is enriched under the
ingester's compatible-only rules, and a same-name row the request did not
classify is a changed association.

Apply re-verifies only what the decision depends on: the bytes are still
ambiguous and still offer the chosen candidate. A resolver change that
rewords the reason or reorders candidates leaves pending items applicable.
The resolver version is one constant in prompt_pair_resolver, shared by
ingestion provenance and Needs review.

The transaction's byte check receives the re-hashed digest (or the body's
for Leave unresolved), every Apply refusal branch is tested with zero
writes, and the root listing and realpath calls run on a worker thread.
2026-10-22 17:29:15 -07:00
Vito Sansevero c4ac90e9ec Merge remote-tracking branch 'origin/main' into feat/249-needs-review-backend 2026-10-22 17:22:15 -07:00
Vito Sansevero ed392e77e2 fix(review): build the spelling validation test from native paths
On Windows the locator is case-folded and uses backslashes, so the fixed
POSIX pair /out/a.png no longer folded to itself and the new spelling
check refused the good case. The test now spells a mixed-case native path
and folds it with os.path.normcase, and also refuses a locator of another
file.
2026-10-22 16:50:46 -07:00
Vito Sansevero 3e20c533fa feat(review): explicit association decisions with compatible linking
Needs review gains its queue reads and decisions:

- GET /prompt_manager/review/items lists pending items oldest first with
  the exact pending total and an HMAC keyset cursor bound to the storage
  epoch and the review index revision.
- GET /prompt_manager/review/items/{id} adds the candidates (role texts,
  label, selectability), the dependency fingerprint and the digest.
- POST /prompt_manager/review/items/{id}/decision applies a candidate only
  for a capture of the item's bytes that the current file still hashes to
  and whose evidence is unchanged, through the ingester's compatible-only
  rules in the same transaction as the decision row; a conflict writes
  nothing. Leave unresolved records the decision and links nothing.

ingest_resolved_pair keeps its contract; its body moves into
_ingest_resolved_pair_in, which runs in the caller's transaction.

Items keep the file's recorded spelling so targets keep their case on
Windows, and a rescan skips the copy and hash of a file whose size and
mtime match the bytes an item already cites.
2026-10-22 16:27:49 -07:00
Vito Sansevero d25c2cb4f2 Merge remote-tracking branch 'origin/main' into feat/249-needs-review-backend 2026-10-22 16:09:06 -07:00
Vito Sansevero f3366a13f8 feat(review): persist ambiguous sampler evidence from scans and new outputs
The output scan and the live monitor now queue every file the ingestion
resolver finds ambiguous for Needs review. The file is captured, its metadata
is read again from the capture and resolved with the default profile, and only
an ambiguous result is stored, bound to the capture's annotation byte record
and fenced by its storage epoch. Evidence holds one hashed projection per
selectable candidate; none, more than 16 or over 8 MiB is a limited item.
Recording is idempotent per bytes and evidence and never touches a decision.

Scan statuses, counters, outcome rows and legacy SSE events are unchanged; the
job scope gains review_items/review_errors and the log a Needs review line.
The monitor records after its fallback link, best effort.
2026-10-22 15:56:58 -07:00
Vito Sansevero de6af76064 feat(review): add review queue schema with restore validation
Add the Needs review schema group (media_review_metadata,
media_review_items, media_review_decisions) beside the image annotation
group it cites. review_schema_error validates the catalog, columns,
foreign keys, row values and canonical 8 MiB evidence read-only;
migrate_review_state installs it in a SAVEPOINT after the annotation
group, and verify_database_file requires it under require_library_state.
A review group without a valid annotation group is refused. Decisions
null their pair and image navigation through indexes when those are
deleted. Old backups and annotation-era backups restore and gain the
group; browser fixtures clear the new tables on reset.
2026-10-22 15:33:26 -07:00
55 changed files with 554 additions and 7228 deletions
+22 -320
View File
@@ -627,9 +627,8 @@ Recorded shell deviations:
`a saved view row matches the concept's` saves one view in both documents and
compares the rows; `tests/browser/workspace-saved-views.spec.mjs` covers saving,
reopening, the defaults that replace rejected values, and failed reads and saves.
- **D50** (retired by #249 PR F, ruling R5): SAVED VIEWS now lists the concept's
"Needs review" row after Favorites, with its count; the comparison maps nothing out
for it (only the concept's sample count "3" reads as the fixture's "0").
- **D50**: no "Needs review" row in SAVED VIEWS until the tag-review slice (maintainer
ruling, 2026-10-21). The comparison maps only that row out.
- **D51** (maintainer ruling, 2026-10-21, beyond the concept): SAVED VIEWS starts with
the concept's Favorites row (star icon, name, count). Its count is the Favorites
listing's own `limit=1` total (`/images/output?mode=workspace&favorites=1`), follows
@@ -1353,8 +1352,8 @@ targets of the longest legal path (1024 code points, even escaped as `\uXXXX`) f
the selection bar's tagging shows is the concept's or the plan's (D85, D90, D91).
- **D89** (plan D66, Activity for the `image-tags` job): the job's Activity item and its
view are the scan's (D54, D56, D57, D59) with the job kind's own counters, from one
per-kind table (`activity-kinds.mjs` since S5, D111; a kind without one gets no
tiles, "Job log" and "Cancel job"). The titles are the server's: "Add
per-kind table (`JOB_VIEWS` in `activity-store.mjs`; a kind without one keeps the
scan's tiles under "Job log" and "Cancel job"). The titles are the server's: "Add
tags to images" and "Auto-tag images".
- The pill's verb is "Tagging" ("Tagging · 41% · 8,240 / 19,622").
- The item line and the view's heading line read "Started HH:MM · N images · closing
@@ -1387,7 +1386,7 @@ targets of the longest legal path (1024 code points, even escaped as `\uXXXX`) f
finished · N tagged · M errors", "<title> cancelled · N tagged" or "<title> failed ·
N tagged · M errors".
- Tests: `tests/js/workspace-activity-store.test.js` ("image-tags jobs get their own
tiles outcomes and pill verb", "unknown kinds get the generic view without the scan tiles");
tiles outcomes and pill verb", "unknown kinds keep the generic view");
`tests/browser/workspace-activity.spec.mjs` ("an image tagging job shows Tagged
Unchanged Skipped Errors and opens tagged rows", "cancel tagging stops at a file
boundary", "the report download names the kind", on fake `image-tags` jobs from
@@ -1881,187 +1880,6 @@ npm run --prefix tools/vendor test:workflows -- inspect-concept.spec.mjs workspa
npm run --prefix tools/vendor test:comfyui -- inspect-sender.spec.mjs
```
### Needs review page and recorded deviations (#249 PR F)
`#/review` is the concept's Metadata review page (app.js `renderReview` a749–760) over
the review routes (`web/js/workspace/review.mjs`, `review-state.mjs`): breadcrumb and
document title "Review" (the concept's), heading "Metadata review" with its line and
**Open activity** (`#/activity`), the `.notice` "N images need a decision" with the
concept's sentence, then `.gallery-layout`: the queue's concept media cards
(`image-card.mjs`, shared with Images; the image opens the shared viewer, Info selects
the item, Tag opens the shared Tag image dialog with the review scope note) and
`section.surface` "Resolve the association" (filename, Tag image, Sampler context,
the chosen path's prompt blocks, Leave for later and primary Apply association). The
first item is selected on load, as the concept's first image is. Sampler context lists
"Sampler <id> · <positive> / <negative>" per candidate (the server's label), disables
unusable ones, preselects the first workable one and ends with "Leave unresolved"
(rulings R2/R3); "Leave unresolved" shows the ambiguous status in both blocks. Apply
takes a fresh capture (`observe`) and posts the decision; Leave unresolved on a file
under no current root posts without a capture. Both actions return to Images (R7); a
recorded decision re-reads the row's count, as does every metadata scan that ends. A
refused decision keeps the selection and shows the server's sentence (the
association conflict's explanation included) as `p.warning-note` in the panel; a
restored storage or a changed queue reads the queue again; a request with no answer
keeps its capture and body, and the next Apply of the same choice resends it once.
The SAVED VIEWS row "Needs review" (review icon) follows Favorites with the exact
pending count (`GET /prompt_manager/review/items?limit=1`, its `total`).
- **D86** (plan D63, ruling R6): an empty queue shows only the notice, "No images need a
decision" and "Ambiguous images are added when a metadata scan or a new generation
finds more than one sampler path."
- **D87** (plan D64): an ended metadata scan whose `scope.review_items` is above 0
carries the concept's **Review metadata** (app.js a738 `go-review`): in its Activity
item after the item's line, and in its job view's heading after Download report. The
item's chip still names its state (D54), where the concept's says "Needs attention".
- **D119**: the queue cards have no selection checkbox and
no star (the page has no selection bar and the queue carries no favorite state); a
file under no current root has an empty, `aria-disabled` image button and Tag; more
than 40 items add the Images page's `.infinite-continuation` ("40 of 45 images",
Load more images) after the layout; loading and a failed read are the page's status
line (with Retry); a limited or unavailable file and the decision states are text in
the panel's `p.warning-note`; the toasts "Association accepted." and "Left
unresolved." replace the concept's prototype toast. The concept's "Sample review
queue" chip and "sample" wording are prototype copy (D29). When a re-read (below)
finds the open item gone, the next one opens with "<file> was decided elsewhere, so
the next image is open."; a failed re-read keeps the queue and says "The review queue
could not be read again. Reload the page to see the latest images." A queue card's
title is its file name and its line the extension only (an undecided item names no
prompt, and D11 omits the size and seed a queue item does not carry); its image's alt
is the file name; the panel's Copy and Send buttons carry no prompt title in their
names; the Tag image dialog opened from the queue heads with the file name. An
unavailable file's empty media button keeps the concept's 1.33 media area, so its
card lines up with the others. A long file name in the panel's `p.small` wraps inside
the panel (`overflow-wrap: anywhere`), where the concept's short sample name never
overflows. A queue card shows the Images listing's thumbnail when the server knows one
(`thumbnail_url`, the same URL the Images page uses), else the original; a re-read
rebuilds a card whose media, target or availability changed (a root came or went), so
Open and Tag follow it. When an unanswered decision's item leaves the queue in a
re-read, the decision was recorded: the page finishes as a recorded decision does (its
toast, back to Images), not with "decided elsewhere".
- **D120** (the concept has no live-update state): while the
change stream is down, the heading carries the quiet status line "Live updates are
paused: changes made elsewhere appear once the connection returns." (`p.small
.review-live`, `role="status"`); the stream's next `ready` hides it.
**Live queue (F2).** An open Needs review page holds one server-sent event stream,
`GET /prompt_manager/review/events` (`py/api/review_events.py`): `event: ready` when it
opens, then `event: change` after each committed queue change (an item the scan or the
monitor recorded, a spelling refresh, a decision from any tab; `database/review_changes.py`
notifies after the commit, from whichever thread wrote it) and after a restore. `change`
carries no data. `ready` carries the queue's version `{library_epoch, review_revision}`,
read after the stream subscribed, and every list read answers the same pair
(`review_revision` is the review index revision, which every queue change moves; a
restore moves the epoch). **One read per visit (F3):** the page re-reads on `ready` only
when the queue it shows is older than that version (an unknown version, `{}` when it
cannot be read, re-reads); a `ready` heard while the first read runs is compared once that
read answers. The test server counts a stream as open only once it has sent `ready`, so a
browser test that changes the queue after that count is a `change`, never part of the
version. The page re-reads on every `change`, keeping the open item, its choice and an
unanswered decision; re-reads asked for while one runs collapse into one more. A change
heard while a decision is in flight is owed: a refused or unanswered decision reads it
once the panel is idle again (a recorded one leaves the page). Load more asked for during
a re-read waits for it and continues after the queue it read; a re-read started during
Load more takes that click over and reads one page deeper, and a failed re-read leaves
Load more clickable. Apply fences on the epoch of the detail it decides: an observe under
another storage releases its capture and reads the queue and the item again, and any
re-read under another epoch reads the open item again. **Every re-read keeps the loaded depth (F3):** it
reads page after page (40 each, the route's limit) until it holds as many items as Load
more had shown, so the cards stay and the scroll position does not move; a restore met
after the first page reads once more from the first page. The row's count follows the
total the page shows (no count read per change). Nothing polls: a `: keepalive` comment
every 25 s only lets the server drop a closed tab's subscription. The server sends the
first change at once and collapses the notices of the next 0.4 s into one trailing
`change` (`COALESCE_SECONDS`), so a busy scan costs an open page one re-read per 0.4 s at
most. Shutdown (`on_shutdown`) ends every stream and answers a later one 503. A
reconnect the browser gives up on (a 503, or a proxy's error while ComfyUI restarts) is
opened again by the page after 3 s, doubling to 30 s, with D120's line shown while
the stream is down; its `ready` re-syncs the queue. The stream lives only while the tab is
visible: a hidden tab closes it and a visible one opens it again, and that `ready` reads
only when the queue changed meanwhile. A duplicate recording and a refused decision change
nothing and send nothing. Residual: each visible Needs review page holds one HTTP/1.1
connection (browsers allow six per origin, shared with ComfyUI's editor), so several
side-by-side visible windows on the page, during a metadata scan's job stream, can still
queue the origin's other requests; a re-read of a deep queue costs one request per 40
items shown.
Pinned by `tests/test_api_review_events.py`, `tests/js/workspace-review-state.test.js`
("refresh …", "a refresh keeps the loaded depth …", "ready naming …") and
`tests/browser/workspace-review.spec.mjs` ("an open page shows …", "a visit reads its
queue once …", "a live re-read keeps the loaded depth and the scroll position", "a stream
the browser gives up on …", "a hidden tab closes its stream …", "a re-read refreshes a
card …"); the controller's owed, Load more and epoch cases are "I1: …", "I2: …", "M3:
…", "M5: …" and "M7: …" in `workspace-review-state.test.js`, and the coalescing is "a
burst of changes …" in `test_api_review_events.py`.
Leaving the page mid-decision releases that decision's capture (the server may never see
the request), and leaving during the observe releases the capture it then answers
("leaving during the observe …").
**Concept comparison (F3).** `tests/browser/review-concept.spec.mjs` measures the whole
page (`reviewInventory` in `tests/browser/tagging-canonical-comparison.mjs`: every shown
box below `main` with its place, fonts, edges and layout, a leaf's text, a button's name,
a select's options) in the concept at `#review` and in production on the fixture's
concept-shaped queue (`POST /__test__/review/reset {items: 3, concept: true}`: the
concept's first three image names and sizes, and the concept's first prompt on the open
item), and `mapConceptReview` applies only: D29 (no "Sample review queue" chip, no
"sample" in the panel heading and Apply), D119 (above), D3 (the grid's
`position: relative`), production's hooks and accessible names (`review-queue`,
`review-panel`, the list roles, the region names) and sample data (the candidate options
except Leave unresolved; widths and heights that follow from the changed copy, moved by
exactly that much). It then matches at 1600×1100 and 390×844 in both themes on all four
projects; the gallery layout follows the concept at 1600, 1200, 900 and 390px (310px,
275px with a 15px gap, 280px, one column); the SAVED VIEWS rows match with Needs review
after Favorites; the queue's Tag image dialog matches the concept's needs-review dialog
(`needs-review-tagging.png`) under the C4 mapping; and the comparison is shown to notice
a dropped notice border and reordered actions. The paired captures (populated queue in
both themes and widths, conflict, limited item, unavailable file, empty queue) are test
artifacts; no screenshot baseline is added. The comparison found one gap, the collapsed
media area of an unavailable file's card, fixed above (`workspace-review.spec.mjs` "a
limited item and an unavailable file …" pins it).
- **D3** and **D20** apply to the queue grid (its cards stay in the grid's flow; it is
not virtualized) and to the panel's prompt blocks (their mount point is transparent).
Tests: `tests/browser/review-concept.spec.mjs` (concept comparison),
`tests/js/workspace-review-state.test.js` (controller),
`tests/js/workspace-state.test.js` (route and count),
`tests/browser/workspace-review.spec.mjs` (queue, count, apply, leave, Tag, empty
queue, Load more, Activity's Review metadata; items recorded by the real recorder in
`tests/browser/workspace_review_fixture.py`), and the SAVED VIEWS rows in
`shell-concept.spec.mjs` and `workspace-saved-views.spec.mjs`.
### Native image tagging and Needs review (#249 PRs A–F)
One place for the whole image-tagging slice; the details live in the sections named.
- **Routes.** Tagging: `POST /prompt_manager/image-annotations/observe`, `GET`/`DELETE
/prompt_manager/image-annotations/capture/<token>`, `POST
/prompt_manager/image-annotations/suggest`, `PUT /prompt_manager/image-annotations/tags`
("Tag image dialog and recorded deviations"). Needs review: `GET
/prompt_manager/review/items?limit=1..40&cursor=` (answers `library_epoch`,
`review_revision`, `total`), `GET /prompt_manager/review/items/<id>`, `POST
/prompt_manager/review/items/<id>/decision` and the change stream `GET
/prompt_manager/review/events` ("Needs review decisions and queue reads", "Needs review
page and recorded deviations"). Every legacy route keeps its payload; the only legacy
change is `409 model-busy` from the legacy Auto-tag routes while another model
operation holds the shared gate.
- **S1 (fingerprint-current reads, ruled 2026-10-22).** The Images Tags filter, search and
the inspector/viewer chips use a byte record only while the file's `(st_size,
st_mtime_ns)` equals its verified fingerprint; writes hash the bytes. Residual risk
under "Image annotations schema".
- **Capture limits.** 256 MiB per file, 8 completed captures, 512 MiB reserved, 2
capture/verify workers, 900 s idle, 1800 s hard; the browser fixtures release every
capture on reset, and leaving the Needs review page mid-decision releases its capture.
- **The record-changing job.** Bulk Add tags and Auto-tag selected run as the
`image-tags` Activity job (PR D); like the metadata scan it is a record-changing job, so
a start while another runs is refused (`busy`). Needs review items come from that scan
and from the live monitor, and a decision changes links only when compatible
(`association-conflict` otherwise, with zero writes).
- **No downloads.** No automated test downloads or loads a model: the browser server
injects `TestModel` (its `download_model` raises), native routes and jobs never
download, and every tagging spec asserts no `/autotag/download/` request.
- **Restart.** PRs A, B, D and E and PR F (the review change stream, the queue version)
change Python: the maintainer restarts ComfyUI once after merging. HTML, CSS and JS
changes need only a browser refresh.
### Native System (#249 System)
`#/system/<section>` is a native workspace route rendered by `web/js/workspace/system.mjs`
@@ -2157,79 +1975,10 @@ keeps the drafts; a save refused for one source says the server's message as the
and marks that row. After a save the source list is read again; a monitoring change
waits for a restart (the running observer keeps watching until then, and the status
says "Restart ComfyUI to apply: Watch for new outputs."). Thumbnail generation's
Review options opens the concept's "Thumbnail maintenance" dialog (D99, below).
Review options opens the legacy gallery in a new tab (D94) until its review dialog (S5).
A list that cannot be read shows "Sources could not be loaded. Your saved data has not
changed." and Retry, never an empty list.
Thumbnail generation is an Activity job (S5, `py/api/thumbnail_jobs.py`). `GET
/prompt_manager/images/thumbnails/plan?mode=missing|rebuild&size=128..2048&quality=low|medium|high`
counts the work (`targets`, `fresh`, `remaining` past the 10,000-file cap, the source
labels) and writes nothing; a value out of range is a 400 `invalid-request` that starts
nothing. `POST /prompt_manager/jobs/thumbnails {mode, size, quality}` starts the
`thumbnails` job ("Thumbnail generation"): admission `guarded` (never beside a scan or
another record-changing job, one at a time, a legacy clear refused while it runs),
cancellable at a file boundary (a file not yet started is left alone; written previews
stay), outcome groups Generated / Skipped (already fresh, listed without a log line
each) / Errors. `size` is the longest edge and `quality` the encoder (low: JPEG 70, no
optimize; medium: 85, optimized, what the legacy routes always wrote; high: 95,
optimized). Video previews take the same choice: the OpenCV frame and the placeholder
are saved with those JPEG settings, and the ffmpeg fallback (no OpenCV) maps them to
`-q:v` 8 / 4 / 2 and fits the frame inside the edge keeping its aspect ratio (`-vf
scale=E:E:force_original_aspect_ratio=decrease`, no longer a stretched square).
`rebuild` writes fresh previews again, each through the same temp-file
rename, so nothing is cleared first and a failed file keeps its old preview (a video
whose frame cannot be decoded keeps its preview instead of getting the placeholder);
originals are only read (`tests/test_thumbnail_jobs.py`). Cancel is also checked per
file during the planning walk: a job cancelled there stops the walk at the next file,
publishes no `start` frame and ends "Cancelled before any preview was written". Every
exit sets a final status line: "N previews written · M errors" (plus "· R more on the
next run" past the cap), "No previews to generate · N already up to date" (or "No
images or videos to preview"), "Cancelled · N previews written", or the failure; a job
with nothing to do never reads "Processing file 0 of 0" (`test_every_exit_sets_a_truthful_final_status_line`,
and "a finished thumbnail job with nothing to do says so" in `workspace-activity.spec.mjs`).
Windows: replacing a preview another process holds open (the gallery serving it, an
image viewer) fails with WinError 5 or 32; that file counts as an error reading
"Preview in use by another program; the old one was kept" (never the raw "[WinError
5]"), and its old preview stays. Any other PermissionError reads "Permission denied
writing the preview" (`test_a_locked_preview_is_a_readable_coded_error`).
Thumbnail work (the job and the gallery's generate and clear holds) is `guarded`, but it
writes only preview files and no records, so the image monitor's Needs review
recording runs beside it instead of being deferred for a whole rebuild
(`PREVIEW_ONLY_KINDS` in `py/api/review_recording.py`, `blocker(..., compatible=)`;
`test_a_running_thumbnail_job_does_not_defer_monitor_recording`). Other `guarded` and
all `records` jobs still defer it.
Deliberate change: the legacy gallery stream `POST /images/generate-thumbnails/progress`
now starts this job (missing previews, its quality preset as the size, medium encoding)
or attaches to the running one, and streams the job's unchanged `status`/`start`/
`progress`/`file_error`/`complete` frames after a first `job {id, started}` frame that
names the job and whether this stream started it. The stream reads the registry after
its headers are sent, so a job that starts in that gap is attached to, not refused. A
client that goes away no longer stops the work (`test_thumbnail_job_survives_client_disconnect`:
one worker, 9 files in three batches; the stream sees its client go after the first
batch and all 9 previews are written, where a stream that stops on disconnect leaves 1).
The gallery's Cancel (its settings panel and the startup dialog) cancels by that id,
and only a job its own stream started: a Cancel before the `job` frame keeps the stream
open until it arrives, a double Cancel posts once, and a job another tab or System
started keeps running while the gallery stops following it (`tests/js/gallery-thumbnails.test.js`).
A job cancelled from Activity ends the stream with the error frame "Thumbnail
generation was cancelled". The blocking
`POST /images/generate-thumbnails` and the clear stay legacy holds.
The workspace starts it from "Thumbnail maintenance" (`thumbnail-review.mjs`, the
concept's `thumbnail-review`), opened by Sources' Thumbnail generation → Review options
and Overview's Review thumbnails. Scope is Missing thumbnails only (`missing`, the
default) or Rebuild all (explicit opt-in) (`rebuild`); Quality is Medium · optimized /
High · original / Low · fastest (`medium` / `high` / `low`), preselected from
Appearance's Image loading quality (`gallerySettings.imageQuality`, the same values,
Q3); Preview size limit is the longest edge, 512 by default, a whole number from 128 to
2048. A size outside that range marks the field `aria-invalid`, the plan line says
"Enter a size from 128 to 2048 pixels." and nothing is requested; Start reports the
field's validity and keeps the dialog open. Start closes the dialog and starts the job
through the Activity store, which toasts "Thumbnail generation started."; the job opens
in Activity. A start refused because another job holds the slot toasts the D57 busy
link toast and stays. The legacy gallery's thumbnail tools are unchanged.
Overview (`system-overview.mjs`) reads `GET /prompt_manager/system/overview` once per
visit through `latestOnly()` on the section scope; a late answer after the section left
paints nothing. Until it answers, the section shows "Loading the overview…"; a failure
@@ -2257,7 +2006,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged
`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
reader can keep it byte-identical (`test_health_check_never_writes`).
Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D113; D121–D128 assigned):
Recorded deviations (plan D60–D66 and D81 renumbered D92–D98 and D113; D121–D128 assigned):
- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server
@@ -2271,14 +2020,14 @@ Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D
integrity" is "Not checked" until a check ran this session, then "N issues" (checks
that found issues, even beside a check that could not run), "Passed" or "Incomplete".
The review row is "N ambiguous image links" (the Needs review queue's pending count,
`needs_review.pending`) with the concept's line; its Review metadata action opens
`#/review` (the concept's `go-review`). A count
`needs_review.pending`) with the concept's line; its Review metadata action is
`aria-disabled` (CSS under D93) until the Needs review page ships (#288). A count
that is not an integer reads "Ambiguous image links" / "Not available yet." (Q6).
"Backup status" reads "No retained backup record." until verified backups exist (S8).
The thumbnail row is "Checking thumbnails…" until its own read answers, then "No
missing thumbnails", "N missing thumbnails" (the pending count of
`/images/thumbnails/status`) or "Thumbnail status unavailable"; Review thumbnails
opens the thumbnail dialog (D99). "One display configuration" says "Grid
opens Sources until the thumbnail dialog (S5). "One display configuration" says "Grid
and density settings are shared across views." (no paging). The sources row is
"N output sources" with "Monitoring and enrichment share one source list.", prefixed
by "M of N configured sources available." or "None configured is available; ComfyUI's
@@ -2339,43 +2088,6 @@ Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D
legacy gallery's `/gallery/rescan` and toasts "Files refreshed. N files in M output
folders." or "Files could not be refreshed. Try again.". The concept's "sample path"
note is prototype copy and is dropped.
- **D99** (Sources/Overview, S5): the concept's `thumbnail-review` dialog ("Thumbnail
maintenance") with production copy. A `p.small[role=status]` line before the actions
says what the server's read-only plan (`GET /images/thumbnails/plan`) counts for the
chosen scope: "N previews to generate · M already up to date." or, for Rebuild all,
"Every preview is regenerated: N files.", with "· R more on the next run" past the
per-run cap; "Counting previews…" while it is read and "The previews could not be
counted." when the read fails (Start still starts). Each scope is read once per
dialog, since size and quality change no count. The primary is "Start" (the
concept's "Simulate thumbnail job"). Overview's Review thumbnails opens the same
dialog (the concept's row starts a sample "Thumbnail check" job directly). Maintenance's
Thumbnail cache row takes the dialog with S7. `thumbnail dialog matches the concept
from sources|overview` (system-concept.spec, both themes and sizes) maps the line and
"Start"; `thumbnail dialog comparison detects mutations` proves they are the only
difference and catches a renamed option, swapped options, swapped actions and a
changed style.
- **D111** (Activity, S5): each job kind's pill verb, toasts, line, tiles, outcomes,
openable rows and labels come from one descriptor table (`activity-kinds.mjs`;
`activity-store.mjs` re-exports it). The scan's (D56) and image tagging's (D89) are
unchanged. A thumbnail job reads "Generating thumbnails · N% · a / b" in the pill,
counts Generated (green), Skipped and Errors (red), lists Generated, Errors, Skipped
(Generated and Skipped rows open the viewer), says "Thumbnails finished · N generated ·
M errors" ("cancelled · N generated", "failed · …") and keeps "Job log" and "Cancel
job". Every other kind (the health check, a kind this page does not know) gets the
generic descriptor: title-based toasts and no tiles or Results, so it never shows the
scan's zero tiles (S1 review M6, S2 review M3). A running job without a known total
has an indeterminate `<progress>` (no `value`) and a pill without a percentage, whose
mini bar (`aria-hidden`; the pill's label carries the state) is marked `indeterminate`
and sweeps instead of showing an empty fill (a dimmed full bar under reduced motion). Only a
running job the server can cancel shows Cancel. Tests:
`tests/js/workspace-activity-kinds.test.js`, and in `workspace-activity.spec.mjs` "a
thumbnail job shows Generated Skipped Errors, its pill verb and its toasts" and "a held
legacy job and the health check show no tiles, no Results and no made-up percentage".
- **D112** (Activity, S5): a legacy-page operation held on the registry (not
cancellable, no outcome groups) reads as its server title ("Database restore (admin
page)", "Thumbnail clear (gallery page)"), with no tiles, no Results, no Cancel, an
indeterminate bar while it runs and "<title> finished." when it ends, whatever its
kind (the gallery's clear is a `thumbnails` hold).
- **D113** (Sources): the Worker threads range ends at the server's `cpu_count` (the
concept's fixed `max=16`) and shows the server's real `worker_threads` (the concept's
sample 4).
@@ -2464,13 +2176,10 @@ and a source row, renames a label and the Add source button and swaps the `--lin
token. Paired captures (`{concept,production}-system-{dark,light}-{1600,390}.png`,
`{concept,production}-{statistics,health}-{dark,light}.png`,
`{concept,production}-appearance-{dark,light}-{1600,390}.png`,
`{concept,production}-sources-{with,without}-notice-{dark,light}-{1600,390}.png`,
`concept-thumbnail-review-{dark,light}-{1600,390}.png`,
`production-thumbnail-review-{sources,overview}-{dark,light}-{1600,390}.png`) are test artifacts; there
`{concept,production}-sources-{with,without}-notice-{dark,light}-{1600,390}.png`) are test artifacts; there
are no PNG baselines. `tests/browser/workspace-system.spec.mjs` covers the real counts,
the health check and metric 4, a failed check, the thumbnail row against the status
endpoint (painted after the rest when its read is held, unavailable when it fails), its Review thumbnails
opening the thumbnail dialog and starting the job, the review row (Q6), the section links, the Statistics dialog against the
endpoint (painted after the rest when its read is held, unavailable when it fails), the review row (Q6), the section links, the Statistics dialog against the
route, a load failure with Retry and late answers after leaving, and Appearance: new
visits versus the current view and a saved view, the embedded gallery keeping Grid,
metadata off and on request, the cache (counted up to a request barrier), hidden paths,
@@ -2480,9 +2189,7 @@ the D97 rows and reset. `tests/browser/workspace-system-sources.spec.mjs` covers
notice's Later and Review scan, a scoped scan from one source into Activity, Add source
with its server check and Save, a refused folder, Remove down to auto-detect, an absent
configured source, the three watch states, the monitoring and worker rows, Refresh
files, the thumbnail dialog (its plan line per scope with the preselected quality, an
invalid size that requests nothing, Start into Activity, Rebuild all, a failed plan and
a refused start), a save refused by a running scan and a failed read. Its hooks are
files, the thumbnail hand-off, a save refused by a running scan and a failed read. Its hooks are
`tests/browser/workspace_system_fixture.py` (`/__test__/system/*`: reset, injected
failures, an absent source, a deleted linked file, unthumbnailed files, a fake image
monitor) on the
@@ -2497,7 +2204,7 @@ node --test tests/js/workspace-system-state.test.js tests/js/workspace-system-co
tests/js/workspace-system-overview.test.js tests/js/workspace-state.test.js \
tests/js/workspace-system-appearance.test.js tests/js/workspace-preferences.test.js \
tests/js/workspace-inspection-state.test.js tests/js/workspace-images-state.test.js \
tests/js/workspace-system-sources.test.js tests/js/workspace-thumbnail-review.test.js
tests/js/workspace-system-sources.test.js
python3 -m unittest tests.test_settings_sources tests.test_scan_scope -v
npm run --prefix tools/vendor test:workflows -- system-concept.spec.mjs workspace-system.spec.mjs \
workspace-system-sources.spec.mjs \
@@ -2766,7 +2473,7 @@ pass runs after its own link, best effort: a failure is logged and the link
stays. Recording follows the registry's `records`
admission (#289): the scan's recording runs under its own `records` job, and
the monitor's (one file, not a job) is refused while a `records` or `guarded`
job or hold runs, except thumbnail work, which writes no records (S5 M7). It then records nothing, logs that the file waits for the
job or hold runs. It then records nothing, logs that the file waits for the
next scan, and answers `deferred`; the next "new" scan reads the ambiguous file
again (`test_monitor_recording_is_refused_beside_record_changing_jobs`). A new item moves `review_index_revision`. A
rescan does not copy or hash an ambiguous file whose current size and
@@ -2785,18 +2492,13 @@ Needs review decisions and queue reads (#249, `py/api/review.py`,
`database/review.py`, `database/review_decisions.py`,
`tests/test_api_review.py`): `GET /prompt_manager/review/items?limit=1..40&cursor=`
answers `{success, library_epoch, total, items, next_cursor}` with pending items
oldest first (each item adds the card's `url`, the Images listing's
`/prompt_manager/images/serve/<path>?root=<i>&root_scope=<scope>` formatted without a
stat, null when the file is under no current root, and `is_video` from its extension); `total` is the exact pending count. The cursor is an HMAC keyset
on `(created_at, id)` bound to the storage epoch only (ruling M3, so "load more"
keeps working while the monitor adds an item per generation): a malformed one
is 400, a well-formed one this process cannot verify (tampered, or opened
before a ComfyUI restart: the secret is per process) is 409 `review-conflict`
with the current `total`, and one from before a restore is 409
`storage-changed`. A decision or a new item leaves an open cursor valid: a
decided item drops out of later pages and new items (stamped `utc_now_iso()`,
microseconds) append after it. Residual risk: an item recorded after a wall
clock step backwards sorts before an open cursor and shows only on reload.
oldest first; `total` is the exact pending count. The cursor is an HMAC keyset
on `(created_at, id)` bound to the storage epoch and `review_index_revision`: a
malformed one is 400, a well-formed one this process cannot verify (tampered,
or opened before a ComfyUI restart: the secret is per process) is 409
`review-conflict` with the current `total`, one from before a restore is 409
`storage-changed`, one from before any queue change (a decision, a new item, a
refreshed spelling) is 409 `review-conflict`.
Targets are formatted from `spelled_path` against the current roots (the
roots' real paths are the only file access); a file under no current root is
`unavailable` with a null target. Every read and decision lists the roots and
-3
View File
@@ -91,9 +91,6 @@ try:
if host_app is not None and hasattr(host_app, "on_cleanup"):
host_app.on_cleanup.append(api.cleanup_history_reads)
host_app.on_cleanup.append(api.cleanup_annotation_captures)
if host_app is not None and hasattr(host_app, "on_shutdown"):
# Open Needs review change streams end before the server waits on them.
host_app.on_shutdown.append(api.close_review_events)
except Exception as e:
# Log to internal logging system without console spam
+2 -22
View File
@@ -31,7 +31,6 @@ from .annotation_schema import (
)
from .annotations import _new_id, _validate_locator, _validate_target
from .models import normalize_image_path, utc_now_iso
from .review_changes import notify_review_change
from .review_schema import (
DECISIONS_TABLE,
ITEMS_TABLE,
@@ -316,9 +315,7 @@ class ReviewOperations:
refreshed = _refresh_spelling(conn, locator, spelled_path)
if inserted or refreshed:
conn.execute(_BUMP_REVIEW_INDEX)
if inserted or refreshed:
notify_review_change(self.model.db_path)
return inserted
return inserted
def refresh_review_spelling(self, locator: str, spelled_path: str) -> bool:
"""Record ``spelled_path`` as the spelling of every item at ``locator``.
@@ -335,9 +332,7 @@ class ReviewOperations:
refreshed = _refresh_spelling(conn, locator, spelled_path)
if refreshed:
conn.execute(_BUMP_REVIEW_INDEX)
if refreshed:
notify_review_change(self.model.db_path)
return refreshed
return refreshed
def review_index_revision(self) -> int:
"""The queue's revision (moved by every queue change)."""
@@ -346,19 +341,6 @@ class ReviewOperations:
raise sqlite3.DatabaseError("Missing review queue metadata")
return row[0]
def review_queue_version(self) -> dict:
"""``{library_epoch, review_revision}`` of the queue, from one snapshot.
The pair a list read answers: equal pairs mean the same queue (a
restore moves the epoch, every queue change moves the revision).
"""
with self._annotation_read() as conn:
epoch = conn.execute(_EPOCH).fetchone()
revision = conn.execute(_REVIEW_INDEX).fetchone()
if epoch is None or revision is None:
raise sqlite3.DatabaseError("Missing review queue metadata")
return {"library_epoch": epoch[0], "review_revision": revision[0]}
def review_items_for(self, locator: str) -> list:
"""Every item of the file at ``locator``, oldest first, evidence parsed."""
_validate_locator(locator)
@@ -441,8 +423,6 @@ class ReviewOperations:
removed += conn.execute(_SETTLE_ITEM, (item_id,)).rowcount
if removed:
conn.execute(_BUMP_REVIEW_INDEX)
if removed:
notify_review_change(self.model.db_path)
return removed
def settle_untraced_review_items(self, locators) -> int:
-64
View File
@@ -1,64 +0,0 @@
"""Needs review queue change notices (#249 PR F2).
Every committed change to the pending queue (an item recorded by the scan or
the monitor, a spelling refresh, a decision) calls ``notify_review_change``
after its transaction, from whichever thread wrote it. The review event
stream (``py/api/review_events.py``) subscribes, so an open Needs review page
re-reads its queue when it changes instead of polling. Observers are keyed by
the database file, like ``subscribe_history_restore``; an observer that fails
is logged and never fails the write.
"""
import os
import threading
from typing import Callable
try:
from ..utils.logging_config import get_logger
except ImportError:
from utils.logging_config import get_logger
_logger = get_logger("prompt_manager.database.review_changes")
_lock = threading.Lock()
_observers: dict[str, dict[object, Callable[[], None]]] = {}
def _identity(db_path: str) -> str:
return os.path.normcase(os.path.realpath(db_path))
def subscribe_review_changes(
db_path: str, callback: Callable[[], None]
) -> Callable[[], None]:
"""Call ``callback()`` after each committed queue change; return the unsubscribe."""
identity = _identity(db_path)
token = object()
with _lock:
_observers.setdefault(identity, {})[token] = callback
def unsubscribe() -> None:
with _lock:
observers = _observers.get(identity)
if observers is not None:
observers.pop(token, None)
if not observers:
_observers.pop(identity, None)
return unsubscribe
def observer_count(db_path: str) -> int:
"""How many observers watch ``db_path`` (tests and diagnostics)."""
with _lock:
return len(_observers.get(_identity(db_path), {}))
def notify_review_change(db_path: str) -> None:
"""Tell every observer of ``db_path``; call it after the commit."""
with _lock:
callbacks = tuple(_observers.get(_identity(db_path), {}).values())
for callback in callbacks:
try:
callback()
except Exception:
_logger.warning("Needs review change observer failed", exc_info=True)
+1 -5
View File
@@ -29,7 +29,6 @@ from .review import (
dependent_images,
same_name_rows,
)
from .review_changes import notify_review_change
from .review_schema import DECISIONS_TABLE, ITEMS_TABLE, canonical_evidence
DECISION_KIND = "user-selection"
@@ -120,7 +119,7 @@ class ReviewDecisionOperations:
raise ValueError("A selected decision needs a candidate and a file")
try:
with self._pair_transaction() as conn:
decided = self._decide_in(
return self._decide_in(
conn,
item_id=item_id,
expected_epoch=expected_epoch,
@@ -136,9 +135,6 @@ class ReviewDecisionOperations:
)
except _Rollback as rollback:
raise rollback.error from None
# After the commit: open Needs review pages re-read their queue.
notify_review_change(self.model.db_path)
return decided
def _decide_in(self, conn: sqlite3.Connection, **command) -> dict:
epoch = self._library_epoch(conn, command["expected_epoch"])
-7
View File
@@ -8,10 +8,8 @@ Split into domain-specific mixins for maintainability:
- AutotagRoutesMixin: auto-tagging model management and tagging
- JobRoutesMixin: the session job registry behind Activity (py/api/jobs.py)
- ImageTagJobsMixin: bulk Add tags / Auto-tag as image-tags jobs
- ThumbnailJobsMixin: thumbnail plan, thumbnail job and the gallery stream on it
- ReviewRoutesMixin: the Needs review queue, item detail and decisions
- ReviewRecordingMixin: Needs review recording for the scan and the monitor
- ReviewEventsMixin: the Needs review change stream an open page re-reads on
- SystemRoutesMixin: System health check, Overview and usage statistics
"""
@@ -36,7 +34,6 @@ from .vocabulary import VocabularyRoutesMixin
from .annotations import AnnotationRoutesMixin
from .review import ReviewRoutesMixin
from .review_recording import ReviewRecordingMixin
from .review_events import ReviewEventsMixin
from .saved_views import SavedViewRoutesMixin
from .history import HistoryRoutesMixin
from .images import ImageRoutesMixin
@@ -44,7 +41,6 @@ from .admin import AdminRoutesMixin
from .settings_routes import SettingsRoutesMixin
from .job_routes import JobRoutesMixin
from .image_tag_jobs import ImageTagJobsMixin
from .thumbnail_jobs import ThumbnailJobsMixin
from .system import SystemRoutesMixin
from .jobs import JobRegistry
from .logging_routes import LoggingRoutesMixin
@@ -274,7 +270,6 @@ class PromptManagerAPI(
AnnotationRoutesMixin,
ReviewRoutesMixin,
ReviewRecordingMixin,
ReviewEventsMixin,
SavedViewRoutesMixin,
HistoryRoutesMixin,
PromptRoutesMixin,
@@ -283,7 +278,6 @@ class PromptManagerAPI(
SettingsRoutesMixin,
JobRoutesMixin,
ImageTagJobsMixin,
ThumbnailJobsMixin,
SystemRoutesMixin,
LoggingRoutesMixin,
AutotagRoutesMixin,
@@ -499,7 +493,6 @@ class PromptManagerAPI(
self._register_vocabulary_routes(routes)
self._register_annotation_routes(routes)
self._register_review_routes(routes)
self._register_review_event_routes(routes)
self._register_saved_view_routes(routes)
self._register_history_routes(routes)
self._register_prompt_routes(routes)
+183 -192
View File
@@ -20,7 +20,7 @@ from typing import Callable, NamedTuple
from aiohttp import web
from PIL import Image, UnidentifiedImageError
from .jobs import MAX_OUTCOME_ROWS, THUMBNAILS_JOB_KIND
from .jobs import THUMBNAILS_JOB_KIND
from .prompts import bad_request, parse_page_params, publish_image_paths
from .prompts import thumbnail_url_for
from .prompts import safe_error_message as _safe_error
@@ -235,8 +235,6 @@ MEDIA_EXTENSIONS = GALLERY_IMAGE_EXTENSIONS | VIDEO_EXTENSIONS
# Caps on request-triggered filesystem work.
MAX_SCAN_DEPTH = 12
MAX_THUMBNAIL_FILES = 10_000
# Already-fresh files a thumbnail job lists under Skipped (its counter stays exact).
MAX_FRESH_LISTED = MAX_OUTCOME_ROWS
def _worker_threads():
@@ -273,52 +271,15 @@ def _atomic_save(img, dst, fmt=None, **save_kwargs):
tmp.unlink()
# Encoder settings per thumbnail quality (the System thumbnail dialog's
# "Medium · optimized", "High · original", "Low · fastest"). The legacy
# routes always encode as "medium", which is what they always wrote.
THUMBNAIL_ENCODINGS = {
"low": {"quality": 70, "optimize": False},
"medium": {"quality": 85, "optimize": True},
"high": {"quality": 95, "optimize": True},
}
DEFAULT_THUMBNAIL_ENCODING = THUMBNAIL_ENCODINGS["medium"]
# ffmpeg's JPEG scale (-q:v, 2 best to 31 worst) for each encoding's quality.
_FFMPEG_QSCALE = {95: 2, 85: 4, 70: 8}
# A preview that could not be replaced (#249 S5 M6). Windows refuses to rename
# over a file another process has open (WinError 5 or 32), such as a preview
# the gallery is serving; the old preview stays and the file counts as an error.
PREVIEW_IN_USE = "Preview in use by another program; the old one was kept"
PREVIEW_NOT_WRITABLE = "Permission denied writing the preview"
_WINDOWS_LOCKED = (5, 32) # ERROR_ACCESS_DENIED, ERROR_SHARING_VIOLATION
def _preview_write_error(exc):
"""The readable error for a PermissionError while writing a preview."""
if getattr(exc, "winerror", None) in _WINDOWS_LOCKED:
return PREVIEW_IN_USE
return PREVIEW_NOT_WRITABLE
def _ffmpeg_qscale(encoding):
"""ffmpeg's ``-q:v`` for a THUMBNAIL_ENCODINGS entry."""
return str(_FFMPEG_QSCALE.get(encoding["quality"], 4))
def _write_image_thumbnail(src, dst, thumbnail_size, encoding=None):
"""Resize *src* into *dst* (blocking PIL work), atomically.
``encoding`` is one of THUMBNAIL_ENCODINGS (medium when None); a PNG
preview keeps only its ``optimize`` flag.
"""
encoding = encoding or DEFAULT_THUMBNAIL_ENCODING
def _write_image_thumbnail(src, dst, thumbnail_size):
"""Resize *src* into *dst* (blocking PIL work), atomically."""
with Image.open(src) as img:
if img.mode in ("RGBA", "LA", "P"):
img = img.convert("RGB")
img.thumbnail(thumbnail_size, Image.Resampling.LANCZOS)
save_kwargs = dict(encoding)
save_kwargs = {"quality": 85, "optimize": True}
if dst.suffix.lower() == ".png":
save_kwargs = {"optimize": encoding["optimize"]}
save_kwargs = {"optimize": True}
_atomic_save(img, dst, **save_kwargs)
@@ -710,25 +671,6 @@ def _iter_media_files(output_path, max_files):
yield root_path / name
def media_thumbnail_url(output_path, rel_path, root_index, *, root_scope=None):
"""The Images listing's thumbnail URL for *rel_path* under *output_path*,
or None when no thumbnail exists (one blocking stat). The Needs review
queue cards use the same URL (#249 F, M10)."""
rel_path = Path(rel_path)
extension = rel_path.suffix.lower()
thumb_rel = (
f"thumbnails/{rel_path.with_suffix('').as_posix()}_thumb"
f"{'.jpg' if extension in VIDEO_EXTENSIONS else extension}"
)
thumbnail_url = thumbnail_url_for(thumb_rel, Path(output_path) / thumb_rel)
if thumbnail_url is None:
return None
thumbnail_url += f"&root={root_index}"
if root_scope is not None:
thumbnail_url += f"&root_scope={root_scope}"
return thumbnail_url
def _output_image_entry(
media_path, output_path, root_index, public_root=None, *, root_scope=None
):
@@ -744,13 +686,19 @@ def _output_image_entry(
extension = media_path.suffix.lower()
is_video = extension in VIDEO_EXTENSIONS
thumbnail_url = media_thumbnail_url(
output_path, rel_path, root_index, root_scope=root_scope
thumb_rel = (
f"thumbnails/{rel_path.with_suffix('').as_posix()}_thumb"
f"{'.jpg' if is_video else extension}"
)
thumbnail_url = thumbnail_url_for(thumb_rel, output_path / thumb_rel)
if thumbnail_url is not None:
thumbnail_url += f"&root={root_index}"
encoded_path = urllib.parse.quote(rel_path.as_posix(), safe="/")
url = f"/prompt_manager/images/serve/{encoded_path}?root={root_index}"
if root_scope is not None:
url += f"&root_scope={root_scope}"
if thumbnail_url is not None:
thumbnail_url += f"&root_scope={root_scope}"
entry = {
"id": hashlib.sha1(
@@ -984,10 +932,6 @@ class ImageRoutesMixin:
async def thumbnail_status_route(request):
return await self.get_thumbnail_status(request)
@routes.get("/prompt_manager/images/thumbnails/plan")
async def thumbnail_plan_route(request):
return await self.get_thumbnail_plan(request)
@routes.post("/prompt_manager/images/generate-thumbnails")
async def generate_thumbnails_route(request):
return await self.generate_thumbnails(request)
@@ -1860,35 +1804,15 @@ class ImageRoutesMixin:
self.logger.info(payload["message"])
return payload
def _plan_gallery_thumbnails(
self, output_dirs: list[Path], rebuild: bool = False, stop=None
) -> dict:
"""Plan every gallery root with one shared per-run generation cap.
See ``_plan_thumbnails``; ``fresh_files`` keeps at most
MAX_FRESH_LISTED files across the roots. ``stop()`` true ends the walk
at the next file (a cancelled job); the partial plan is returned.
"""
combined = {
"targets": [],
"skipped": 0,
"remaining": 0,
"fresh": 0,
"fresh_files": [],
}
def _plan_gallery_thumbnails(self, output_dirs: list[Path]) -> dict:
"""Plan every gallery root with one shared per-run generation cap."""
combined = {"targets": [], "skipped": 0, "remaining": 0}
for root in output_dirs:
if stop is not None and stop():
break
root = Path(root)
plan = self._plan_thumbnails(
root, root / "thumbnails", rebuild=rebuild, stop=stop
)
plan = self._plan_thumbnails(root, root / "thumbnails")
available = max(0, MAX_THUMBNAIL_FILES - len(combined["targets"]))
combined["targets"].extend(plan["targets"][:available])
combined["skipped"] += plan["skipped"]
combined["fresh"] += plan["fresh"]
room = max(0, MAX_FRESH_LISTED - len(combined["fresh_files"]))
combined["fresh_files"].extend(plan["fresh_files"][:room])
combined["remaining"] += plan["remaining"] + max(
0, len(plan["targets"]) - available
)
@@ -1921,30 +1845,22 @@ class ImageRoutesMixin:
{"success": False, "error": _safe_error(exc)}, status=500
)
def _plan_thumbnails(self, output_path, thumbnails_dir, rebuild=False, stop=None):
def _plan_thumbnails(self, output_path, thumbnails_dir):
"""Decide which media files need a thumbnail (blocking walk).
Returns ``{"targets": [(src, dst, is_video)], "skipped": n,
"remaining": n, "fresh": n, "fresh_files": [src]}``. Files whose
thumbnail is already newer than the source are ``fresh``. Without
``rebuild`` they are counted as skipped and never queued, so a large
library that is mostly done only pays for the files that changed;
``fresh_files`` lists the first MAX_FRESH_LISTED of them. With
``rebuild`` every file is a target (nothing is skipped). The walk is
bounded like the gallery listing (MAX_SCAN_DEPTH, MAX_GALLERY_FILES,
no symlinks) and at most MAX_THUMBNAIL_FILES targets are queued per
run; ``remaining`` counts the targets beyond that cap. ``stop()`` true
ends the walk before the next file.
Returns ``{"targets": [(src, dst, is_video)], "skipped": n, "remaining": n}``.
Files whose thumbnail is already newer than the source are counted as
skipped and never queued, so a large library that is mostly done only
pays for the files that changed. The walk is bounded like the gallery
listing (MAX_SCAN_DEPTH, MAX_GALLERY_FILES, no symlinks) and at most
MAX_THUMBNAIL_FILES pending files are queued per run; ``remaining``
counts the pending files beyond that cap.
"""
output_path = Path(output_path)
targets = []
skipped = 0
remaining = 0
fresh = 0
fresh_files = []
for src in _iter_media_files(output_path, MAX_GALLERY_FILES):
if stop is not None and stop():
break
suffix = src.suffix.lower()
is_video = suffix in VIDEO_EXTENSIONS
rel_no_ext = src.relative_to(output_path).with_suffix("")
@@ -1954,12 +1870,8 @@ class ImageRoutesMixin:
self.logger.warning(f"Skipping thumbnail outside safe dir: {src.name}")
continue
if _thumbnail_is_fresh(src, dst):
fresh += 1
if not rebuild:
skipped += 1
if len(fresh_files) < MAX_FRESH_LISTED:
fresh_files.append(src)
continue
skipped += 1
continue
if len(targets) >= MAX_THUMBNAIL_FILES:
remaining += 1
continue
@@ -1969,81 +1881,190 @@ class ImageRoutesMixin:
f"Thumbnail run capped at {MAX_THUMBNAIL_FILES} new files; "
f"{remaining} more will be generated on the next run"
)
return {
"targets": targets,
"skipped": skipped,
"remaining": remaining,
"fresh": fresh,
"fresh_files": fresh_files,
}
return {"targets": targets, "skipped": skipped, "remaining": remaining}
def _thumbnail_targets(self, output_path, thumbnails_dir):
"""Pending (src, dst, is_video) triples; see _plan_thumbnails."""
return self._plan_thumbnails(output_path, thumbnails_dir)["targets"]
def _generate_batch(self, targets, thumbnail_size, **options):
def _generate_batch(self, targets, thumbnail_size):
"""Create thumbnails for ``targets`` on the configured worker threads.
Blocking; returns one result dict per target, in order. ``options``
go to ``_generate_one`` (``encoding``, ``rebuild``, ``stop``).
Blocking; returns one result dict per target, in order.
"""
def one(target):
src, dst, is_video = target
return self._generate_one(src, dst, thumbnail_size, is_video, **options)
return self._generate_one(src, dst, thumbnail_size, is_video)
return map_parallel(one, targets, _worker_threads())
def _generate_one(
self,
src,
dst,
thumbnail_size,
is_video,
encoding=None,
rebuild=False,
stop=None,
):
"""Create one thumbnail (blocking). Never raises; returns a result dict.
``stop()`` true before the file starts gives ``action: "cancelled"``
and touches nothing (a job's file boundary). ``rebuild`` writes a
fresh preview again and never replaces an existing video preview with
the placeholder. Every write is atomic, so a failure keeps the old
preview.
"""
def _generate_one(self, src, dst, thumbnail_size, is_video):
"""Create one thumbnail (blocking). Never raises; returns a result dict."""
result = {
"file": src.name,
"dir": src.parent.name,
"type": "video" if is_video else "image",
}
try:
if stop is not None and stop():
return {**result, "action": "cancelled"}
if not src.is_file():
return {**result, "action": "error", "error": "File no longer exists"}
if not rebuild and _thumbnail_is_fresh(src, dst):
if _thumbnail_is_fresh(src, dst):
return {**result, "action": "skipped"}
dst.parent.mkdir(parents=True, exist_ok=True)
if is_video:
made = self._generate_video_thumbnail(
src, dst, thumbnail_size, encoding, keep_existing=rebuild
)
if not made:
if not self._generate_video_thumbnail(src, dst, thumbnail_size):
return {**result, "action": "error", "error": "Video decode failed"}
else:
_write_image_thumbnail(src, dst, thumbnail_size, encoding)
_write_image_thumbnail(src, dst, thumbnail_size)
return {**result, "action": "generated"}
except UnidentifiedImageError:
return {**result, "action": "error", "error": "Not a valid image file"}
except PermissionError as e:
message = _preview_write_error(e)
self.logger.warning(f"Thumbnail failed for {src.name}: {message}")
return {**result, "action": "error", "error": message}
except Exception as e:
self.logger.warning(f"Thumbnail failed for {src.name}", exc_info=True)
return {**result, "action": "error", "error": _safe_error(e)}
async def generate_thumbnails_with_progress(self, request):
"""Generate thumbnails with Server-Sent Events progress updates (POST).
``quality`` is read from the query string so a POST-capable SSE
client can keep using the same URL shape. Every blocking step (scan,
PIL/ffmpeg) runs in the executor; the event loop only writes SSE
frames between files.
"""
thumbnail_size = self._thumbnail_size(request.query.get("quality", "medium"))
response = web.StreamResponse(
status=200,
headers={
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
},
)
busy = self._jobs.blocker(THUMBNAILS_JOB_KIND, admission="guarded")
if busy is not None:
# The gallery reads this stream: refuse with its error event.
await response.prepare(request)
message = f"{busy.title} already running"
await self._emit_sse(
response, "error", {"error": message, "message": message}
)
return response
async with self._jobs.hold(
THUMBNAILS_JOB_KIND, THUMBNAIL_GENERATION_TITLE, admission="guarded"
) as job:
await response.prepare(request)
if not await self._thumbnail_progress_stream(response, thumbnail_size):
job.finish("failed") # an error event ended the stream
return response
async def _thumbnail_progress_stream(self, response, thumbnail_size) -> bool:
"""Stream one generation run; False when it ended with an error event."""
try:
output_dirs = self._get_all_output_dirs()
if not output_dirs:
await self._emit_sse(
response, "error", {"error": "ComfyUI output directory not found"}
)
return False
await self._emit_sse(
response,
"status",
{"phase": "scanning", "message": "Scanning for images and videos..."},
)
plan = await self._run_in_executor(
self._plan_gallery_thumbnails, output_dirs
)
await self._stream_thumbnails(
response,
plan["targets"],
thumbnail_size,
skipped=plan["skipped"],
remaining=plan["remaining"],
)
self.invalidate_gallery_cache()
return True
except Exception:
self.logger.exception("Thumbnail generation failed")
await self._emit_sse(
response,
"error",
{
"error": "An internal error occurred",
"message": "Thumbnail generation failed. Check server logs.",
},
)
return False
async def _stream_thumbnails(
self, response, targets, thumbnail_size, skipped=0, remaining=0
):
"""Process pending *targets* in worker-thread batches, emitting SSE per file.
``skipped`` files already had fresh thumbnails and ``remaining`` were
left for the next run; both only feed the start/complete payloads.
"""
total = len(targets)
video_count = sum(1 for _, _, is_video in targets if is_video)
message = (
f"Found {total - video_count} images and {video_count} videos to process"
)
if skipped:
message += f" ({skipped} already have thumbnails)"
connected = await self._emit_sse(
response,
"start",
{
"total_images": total,
"skipped": skipped,
"remaining": remaining,
"phase": "processing",
"image_count": total - video_count,
"video_count": video_count,
"message": message,
},
)
if not connected:
return
stats = _new_thumbnail_stats()
stats["skipped"] = skipped
start = _time.monotonic()
batch_size = _thumbnail_batch_size(_worker_threads())
index = 0
for batch_start in range(0, total, batch_size):
batch = targets[batch_start : batch_start + batch_size]
results = await self._run_in_executor(
self._generate_batch, batch, thumbnail_size
)
for result in results:
index += 1
_record_thumbnail_result(stats, result)
if result["action"] == "error":
await self._emit_sse(
response,
"file_error",
{"file": result["file"], "error": result["error"]},
)
connected = await self._emit_sse(
response,
"progress",
_thumbnail_progress_payload(index, total, stats, result, start),
)
if not connected:
self.logger.info("Thumbnail client disconnected; stopping early")
return
await asyncio.sleep(0) # let other requests run between batches
payload = _thumbnail_complete_payload(
total + skipped + remaining,
stats,
start,
remaining=remaining,
processed=total,
)
await self._emit_sse(response, "complete", payload)
self.logger.info(payload["message"])
async def _emit_sse(self, response, event, data) -> bool:
"""Write one SSE frame. Returns False once the client has gone away."""
try:
@@ -2054,25 +2075,8 @@ class ImageRoutesMixin:
self.logger.warning(f"Failed to send SSE message: {e}")
return False
def _generate_video_thumbnail(
self,
video_path,
thumbnail_path,
thumbnail_size,
encoding=None,
keep_existing=False,
):
"""Generate thumbnail from video file. Returns True if successful.
``encoding`` is one of THUMBNAIL_ENCODINGS (medium when None); it
encodes the OpenCV frame, ffmpeg's frame (as ``-q:v``) and the
placeholder alike. Both decoders fit the frame inside
``thumbnail_size`` keeping its aspect ratio. With ``keep_existing`` an
existing preview is kept, and the run fails, when no frame could be
decoded, rather than replaced by the placeholder. A PermissionError
while replacing the preview propagates (``_generate_one`` names it).
"""
encoding = encoding or DEFAULT_THUMBNAIL_ENCODING
def _generate_video_thumbnail(self, video_path, thumbnail_path, thumbnail_size):
"""Generate thumbnail from video file. Returns True if successful."""
try:
# Try using OpenCV first (most reliable)
try:
@@ -2096,7 +2100,7 @@ class ImageRoutesMixin:
img = Image.fromarray(frame_rgb)
img.thumbnail(thumbnail_size, Image.Resampling.LANCZOS)
_atomic_save(img, thumbnail_path, "JPEG", **encoding)
_atomic_save(img, thumbnail_path, "JPEG", quality=85, optimize=True)
self.logger.debug(
f"Generated video thumbnail using OpenCV: {thumbnail_path}"
)
@@ -2118,11 +2122,8 @@ class ImageRoutesMixin:
"00:00:01",
"-vframes",
"1",
"-vf",
f"scale={thumbnail_size[0]}:{thumbnail_size[1]}"
":force_original_aspect_ratio=decrease",
"-q:v",
_ffmpeg_qscale(encoding),
"-s",
f"{thumbnail_size[0]}x{thumbnail_size[1]}",
"-y",
str(tmp_path),
]
@@ -2147,12 +2148,6 @@ class ImageRoutesMixin:
except (NameError, OSError):
pass
if keep_existing and thumbnail_path.exists():
self.logger.warning(
f"No frame decoded for {video_path.name}; keeping its preview"
)
return False
# Last resort: create a placeholder thumbnail
try:
from PIL import ImageDraw, ImageFont
@@ -2187,22 +2182,18 @@ class ImageRoutesMixin:
except (OSError, AttributeError):
pass
_atomic_save(img, thumbnail_path, "JPEG", **encoding)
_atomic_save(img, thumbnail_path, "JPEG", quality=85)
self.logger.debug(
f"Generated placeholder video thumbnail: {thumbnail_path}"
)
return True
except PermissionError:
raise
except Exception as e:
self.logger.warning(
f"Failed to create placeholder thumbnail for {video_path}: {e}"
)
return False
except PermissionError:
raise
except Exception as e:
self.logger.error(
f"Video thumbnail generation failed for {video_path}: {e}"
+1 -6
View File
@@ -1,8 +1,7 @@
"""Session job routes (Activity): list, inspect, follow and cancel jobs.
The registry itself is ``py/api/jobs.py``. The scan job's runner is in
``py/api/admin.py``, the image tagging job's in ``py/api/image_tag_jobs.py``
and the thumbnail job's in ``py/api/thumbnail_jobs.py``.
``py/api/admin.py``, the image tagging job's in ``py/api/image_tag_jobs.py``.
Every response leaves out absolute server paths: job rows already carry
root-relative paths and public root labels.
"""
@@ -130,10 +129,6 @@ class JobRoutesMixin:
async def start_image_tag_job_route(request):
return await self.start_image_tag_job(request)
@routes.post("/prompt_manager/jobs/thumbnails")
async def start_thumbnail_job_route(request):
return await self.start_thumbnail_job(request)
@routes.get("/prompt_manager/jobs/{job_id}")
async def job_detail_route(request):
return await self.job_detail(request)
+7 -45
View File
@@ -72,9 +72,6 @@ SCAN_GROUPS = {
IMAGE_TAG_JOB_KIND = "image-tags"
IMAGE_TAG_GROUP_NAMES = ("tagged", "unchanged", "skipped", "errors")
# Thumbnail generation (#249 System S5): previews written, already fresh, failed.
THUMBNAIL_GROUP_NAMES = ("generated", "skipped", "errors")
ADMISSIONS = ("read", "guarded", "records")
# What each admission class is refused beside (besides its own kind).
_REFUSED_BESIDE = {
@@ -250,32 +247,6 @@ class Job:
)
self.log_line(log_line)
def record_counted(self, group, count, rows=()):
"""Count ``count`` outcomes in ``group`` at once, listing ``rows``.
For outcomes that need no log line each (a thumbnail job's already
fresh previews): the counter is exact, ``rows`` (``record``'s row
shape) are listed up to MAX_OUTCOME_ROWS, and nothing is logged.
"""
if group not in self.counters:
raise ValueError(f"Unknown outcome group {group!r} for {self.kind}")
self.counters[group] += max(0, int(count))
listed = self.outcomes[group]
for row in rows:
if len(listed) >= MAX_OUTCOME_ROWS:
break
index = row.get("root_index")
if isinstance(index, bool) or not isinstance(index, int) or index < 0:
index = None
listed.append(
{
"path": _relative_path(row.get("path")),
"root": str(row.get("root") or ""),
"root_index": index,
"detail": str(row.get("detail") or ""),
}
)
def finish(self, state="finished"):
"""End the job in ``state`` and release every subscriber.
@@ -292,24 +263,18 @@ class Job:
for queue in list(self._subscribers):
queue.put_nowait(None)
async def events(self, lead=None):
async def events(self):
"""Yield the latest event, then each later one, until the job finishes.
``lead``, when given, is yielded first, after the subscription is in
place, so nothing published while the consumer sends it is missed.
Consume it under ``contextlib.aclosing`` so a consumer that stops early
(a closed socket) drops its queue at once instead of at garbage collection.
"""
queue = asyncio.Queue()
self._subscribers.add(queue)
# As of the subscription: a later event or the end reaches the queue.
snapshot, ended = self.last_event, self.finished
try:
if lead is not None:
yield lead
if snapshot is not None:
yield snapshot
if ended:
if self.last_event is not None:
yield self.last_event
if self.finished:
return
while True:
payload = await queue.get()
@@ -460,14 +425,11 @@ class JobRegistry:
self._jobs[job.id] = job
return job
def blocker(self, kind, *, admission=None, record_changing=None, compatible=()):
def blocker(self, kind, *, admission=None, record_changing=None):
"""The running job a new ``kind`` job would be refused for, or None.
The same kind refuses first; then the oldest running job whose class
the new job's admission class may not run beside. Running jobs of a
kind in ``compatible`` never refuse it: the caller knows that work
cannot conflict with its own (the monitor's Needs review recording
beside thumbnail work, which writes previews and no records).
the new job's admission class may not run beside.
"""
same_kind = self.active(kind)
if same_kind is not None:
@@ -477,7 +439,7 @@ class JobRegistry:
running = [job for job in tuple(self._jobs.values()) if job.running]
running.sort(key=lambda job: job._started_seq)
for job in running:
if job.admission in refused and job.kind not in compatible:
if job.admission in refused:
return job
return None
+14 -31
View File
@@ -1,9 +1,8 @@
"""Needs review routes (#249 E3): the pending queue, item detail, decisions.
``GET /prompt_manager/review/items`` lists pending items oldest first behind
an HMAC keyset cursor bound to the storage epoch only (a restore makes an
old cursor stale; a decision or a new item does not, because the
``(created_at, id)`` keyset stays ordered when items leave or are appended);
an HMAC keyset cursor bound to the storage epoch and the review index
revision (any decision, new item or restore makes an old cursor stale);
``total`` is the exact pending count. ``GET .../items/{id}`` adds the
candidates and the dependency fingerprint. ``POST .../items/{id}/decision``
records one explicit decision: Apply links the chosen sampler's pair only
@@ -24,13 +23,10 @@ import hmac
import json
import os
import re
import urllib.parse
from aiohttp import web
from .annotations import _HEX64, _TOKEN, _UUID, _capture_error, _error
from .images import media_thumbnail_url
from .inspection_service import VIDEO_EXTENSIONS
from .prompts import bad_request
try:
@@ -157,9 +153,9 @@ class _StaleCursor(ValueError):
"""A well-formed cursor this process cannot verify (e.g. after a restart)."""
def _encode_cursor(epoch: str, key: tuple) -> str:
def _encode_cursor(epoch: str, revision: int, key: tuple) -> str:
payload = json.dumps(
{"v": 2, "e": epoch, "k": list(key)},
{"v": 1, "e": epoch, "r": revision, "k": list(key)},
separators=(",", ":"),
ensure_ascii=True,
).encode()
@@ -168,7 +164,7 @@ def _encode_cursor(epoch: str, key: tuple) -> str:
def _decode_cursor(token: str) -> tuple:
"""``(epoch, (created_at, id))`` of a cursor.
"""``(epoch, revision, (created_at, id))`` of a cursor.
``ValueError`` when malformed; :class:`_StaleCursor` when well-formed but
signed by another secret (the per-process secret changes on every start,
@@ -189,14 +185,15 @@ def _decode_cursor(token: str) -> tuple:
data = json.loads(payload)
key = data["k"]
if (
data["v"] != 2
data["v"] != 1
or not isinstance(data["e"], str)
or type(data["r"]) is not int
or not isinstance(key, list)
or len(key) != 2
or not all(isinstance(part, str) for part in key)
):
raise ValueError()
return data["e"], tuple(key)
return data["e"], data["r"], tuple(key)
except _StaleCursor:
raise
except (ValueError, KeyError, TypeError, UnicodeError, binascii.Error):
@@ -256,7 +253,7 @@ class ReviewRoutesMixin:
def _review_view(self, row: dict, roots, real_roots) -> dict:
"""The public ReviewItem of one stored row (no path, no locator)."""
found = format_review_target(row["spelled_path"], row["locator"], real_roots)
target = url = thumbnail_url = None
target = None
if found is not None:
index, relative = found
target = {
@@ -264,30 +261,16 @@ class ReviewRoutesMixin:
"root_scope": self._gallery_root_scope(roots[index], index),
"path": relative,
}
# The Images listing's media URL for the same file (formatting only).
url = (
"/prompt_manager/images/serve/"
f"{urllib.parse.quote(relative, safe='/')}?root={index}"
f"&root_scope={target['root_scope']}"
)
# The Images listing's thumbnail, when one exists (one stat).
thumbnail_url = media_thumbnail_url(
roots[index], relative, index, root_scope=target["root_scope"]
)
filename = os.path.basename(row["spelled_path"])
return {
"id": row["id"],
"state": row["state"],
"revision": row["revision"],
"created_at": row["created_at"],
"filename": filename,
"filename": os.path.basename(row["spelled_path"]),
"target": target,
"unavailable": target is None,
"candidate_count": row["candidate_count"],
"limited": row["limited"],
"url": url,
"thumbnail_url": thumbnail_url,
"is_video": os.path.splitext(filename)[1].lower() in VIDEO_EXTENSIONS,
}
@staticmethod
@@ -316,26 +299,26 @@ class ReviewRoutesMixin:
return bad_request(str(exc))
try:
page, views = await self._run_in_executor(
self._review_page, limit, bound[1] if bound else None
self._review_page, limit, bound[2] if bound else None
)
except Exception:
self.logger.exception("Review queue read failed")
return _failed()
if bound is not None and bound[0] != page["epoch"]:
return _coded("storage-changed", library_epoch=page["epoch"])
if bound is not None and bound[1] != page["revision"]:
return _coded("review-conflict", total=page["total"])
items = page["items"]
next_cursor = None
if page["more"] and items:
last = items[-1]
next_cursor = _encode_cursor(
page["epoch"], (last["created_at"], last["id"])
page["epoch"], page["revision"], (last["created_at"], last["id"])
)
return web.json_response(
{
"success": True,
"library_epoch": page["epoch"],
# F3: the queue version the change stream's ``ready`` names.
"review_revision": page["revision"],
"total": page["total"],
"items": views,
"next_cursor": next_cursor,
-171
View File
@@ -1,171 +0,0 @@
"""Needs review change stream (#249 PR F2).
``GET /prompt_manager/review/events`` is a server-sent event stream for an
open Needs review page: ``event: ready`` when it opens, then
``event: change`` after each committed queue change (an item the scan or the
monitor recorded, a decision from any tab, a spelling refresh) and after a
restore. The first notice is sent at once; notices in the next
``COALESCE_SECONDS`` collapse into one trailing ``change`` (M9: a busy scan
records file after file, and each event costs every open page a re-read).
The page re-reads its queue on each ``change``; nothing polls. ``change``
carries no data (``{}``). ``ready`` carries the queue's version
``{library_epoch, review_revision}``, read after the stream subscribed, the
same pair a list read answers: a page whose first read already answered that
version skips the re-read (F3, one read per visit), and any later change is a
``change`` event. Nothing else about the queue is pushed.
A ``: keepalive`` comment every ``HEARTBEAT_SECONDS`` lets the server notice
a closed tab (its write fails) and drop the subscription; clients ignore
comments. Shutdown (``close_review_events``, an ``on_shutdown`` hook) ends
every stream, and a stream asked for afterwards is a 503.
"""
import asyncio
import json
from aiohttp import web
try:
from ...database.execution_history import subscribe_history_restore
from ...database.review_changes import subscribe_review_changes
except ImportError:
from database.execution_history import subscribe_history_restore
from database.review_changes import subscribe_review_changes
EVENTS_URL = "/prompt_manager/review/events"
HEARTBEAT_SECONDS = 25
# After a ``change``, the next one waits this long, collapsing the notices in between.
COALESCE_SECONDS = 0.4
_CHANGE = b"event: change\ndata: {}\n\n"
_KEEPALIVE = b": keepalive\n\n"
def _ready(version: dict) -> bytes:
data = json.dumps(version, separators=(",", ":"), sort_keys=True)
return f"event: ready\ndata: {data}\n\n".encode("utf-8")
class _Stream:
"""One open stream: a change flag set from any thread, and a close flag."""
def __init__(self, loop: asyncio.AbstractEventLoop) -> None:
self._loop = loop
self.changed = asyncio.Event()
self.closed = asyncio.Event()
self.ready = False
def wake(self) -> None:
"""Thread-safe: mark a change (the loop may already be gone)."""
try:
self._loop.call_soon_threadsafe(self.changed.set)
except RuntimeError: # the loop closed
pass
async def next(self, timeout: float) -> str:
"""``change``, ``closed`` or ``idle`` (nothing within ``timeout``)."""
waits = {
asyncio.ensure_future(self.changed.wait()): "change",
asyncio.ensure_future(self.closed.wait()): "closed",
}
try:
done, _ = await asyncio.wait(
waits, timeout=timeout, return_when=asyncio.FIRST_COMPLETED
)
finally:
for waiter in waits:
waiter.cancel()
if self.closed.is_set():
return "closed"
if done:
self.changed.clear()
return "change"
return "idle"
async def closed_within(self, timeout: float) -> bool:
"""Wait up to ``timeout``; true when the stream closed meanwhile."""
try:
await asyncio.wait_for(self.closed.wait(), timeout)
except asyncio.TimeoutError:
return False
return True
class ReviewEventsMixin:
"""The Needs review change stream."""
def _register_review_event_routes(self, routes: web.RouteTableDef) -> None:
routes.get(EVENTS_URL)(self.review_events)
def _review_streams(self) -> set:
streams = getattr(self, "_open_review_streams", None)
if streams is None:
streams = self._open_review_streams = set()
return streams
def review_stream_count(self) -> int:
"""How many change streams have sent ``ready`` and are open (tests and
diagnostics): a queue change after this count moved is a ``change``
event on them, never part of the version ``ready`` named."""
return sum(1 for stream in self._review_streams() if stream.ready)
async def close_review_events(self, app) -> None:
"""End every open stream; later requests get 503 (shutdown hook)."""
self._review_events_closed = True
for stream in list(self._review_streams()):
stream.closed.set()
async def _queue_version(self) -> dict:
"""The queue version for ``ready``, read after subscribing (so a change
after this read is an event); ``{}`` when it cannot be read, and the
page then re-reads."""
try:
return await self._run_in_executor(self.db.review_queue_version)
except Exception:
self.logger.warning("Needs review version read failed", exc_info=True)
return {}
async def review_events(self, request: web.Request) -> web.StreamResponse:
if getattr(self, "_review_events_closed", False):
return web.json_response(
{"success": False, "error": "The server is shutting down."},
status=503,
)
stream = _Stream(asyncio.get_running_loop())
db_path = self.db.model.db_path
unsubscribers = (
subscribe_review_changes(db_path, stream.wake),
subscribe_history_restore(db_path, stream.wake),
)
streams = self._review_streams()
streams.add(stream)
response = web.StreamResponse(
status=200,
headers={
"Content-Type": "text/event-stream",
"Cache-Control": "no-store",
"X-Accel-Buffering": "no",
},
)
try:
version = await self._queue_version()
await response.prepare(request)
await response.write(_ready(version))
stream.ready = True
while True:
outcome = await stream.next(HEARTBEAT_SECONDS)
if outcome == "closed":
break
if outcome == "idle":
await response.write(_KEEPALIVE)
continue
await response.write(_CHANGE)
if await stream.closed_within(COALESCE_SECONDS):
break
await response.write_eof()
except (ConnectionError, RuntimeError) as exc: # the tab went away
self.logger.debug(f"Needs review stream closed ({exc})")
finally:
streams.discard(stream)
for unsubscribe in unsubscribers:
unsubscribe()
return response
+4 -14
View File
@@ -28,23 +28,16 @@ Recording changes records, so it follows the registry's ``records``
admission (#289). The scan records under its own ``records`` job. The
monitor's pass lasts one file and is not a job: it is refused while a
``records`` or ``guarded`` job runs (``deferred``), and the next scan reads
the file again. Thumbnail work (the job and the gallery's generate and clear
holds, kind ``thumbnails``) is ``guarded`` but only writes and removes preview
files under ``thumbnails/`` and never touches the database, so it does not
defer the recording (#249 S5 M7): a long rebuild no longer holds Needs review
back for its whole run.
the file again.
"""
import os
from pathlib import Path
from .admin import SCAN_LOG_PREFIXES, _plural, _scan_location, _scan_roots
from .jobs import THUMBNAILS_JOB_KIND
# The kind the monitor's recording checks admission under (never registered).
REVIEW_RECORDING_KIND = "review_recording"
# Guarded kinds that change no records, so the recording may run beside them.
PREVIEW_ONLY_KINDS = (THUMBNAILS_JOB_KIND,)
# record_ambiguous_file outcomes that leave the file in the pending queue.
QUEUED_OUTCOMES = ("recorded", "limited", "duplicate")
NEW_OUTCOMES = ("recorded", "limited")
@@ -192,14 +185,11 @@ class ReviewRecordingMixin:
links it); any other file is recorded when it is ambiguous with a
choice, and is ``untraced`` (its pending items without a choice
removed) when it is ambiguous without one. While a
job runs that a ``records`` job may not run beside, other than
thumbnail work (``PREVIEW_ONLY_KINDS``), nothing is written and the
answer is ``deferred``. Errors propagate: the monitor logs
job runs that a ``records`` job may not run beside, nothing is written
and the answer is ``deferred``. Errors propagate: the monitor logs
them and keeps its link.
"""
busy = self._jobs.blocker(
REVIEW_RECORDING_KIND, admission="records", compatible=PREVIEW_ONLY_KINDS
)
busy = self._jobs.blocker(REVIEW_RECORDING_KIND, admission="records")
if busy is not None:
self.logger.info(
f"Needs review was not updated for "
-494
View File
@@ -1,494 +0,0 @@
"""Thumbnail generation as an Activity job (#249 System S5, Task 12).
- ``GET /prompt_manager/images/thumbnails/plan?mode=&size=&quality=`` counts
what a run would do over every gallery root and writes nothing:
``{success, targets, fresh, remaining, roots, mode, size, quality}``.
- ``POST /prompt_manager/jobs/thumbnails {mode, size, quality}`` starts the
``thumbnails`` job (202 ``{job}``). Its admission class is ``guarded``: it
never runs beside a record-changing job, and one thumbnail job runs at a
time. It stops at a file boundary when cancelled, keeping the previews it
wrote. Its outcome groups are ``generated``, ``skipped`` (already fresh)
and ``errors``.
``mode`` is ``missing`` (previews that are absent or older than their file)
or ``rebuild`` (every preview again, fresh ones too). ``size`` is the
preview's longest edge, 128 to 2048 pixels. ``quality`` picks the encoder
settings (``THUMBNAIL_ENCODINGS``). Every preview is written next to the old
one and renamed over it, so nothing is cleared first and a failure keeps the
old preview; originals are only read.
The legacy gallery stream ``POST /images/generate-thumbnails/progress``
starts this job (its quality preset still sizes the previews) or attaches to
the running one. Its first frame, ``job {id, started}``, names that job and
whether this stream started it (the gallery cancels by that id, and only a
job it started); then come the job's legacy ``status``/``start``/
``progress``/``file_error``/``complete`` frames. A client that goes away no
longer stops the work: the job outlives the request, as a scan does.
"""
import asyncio
import functools
import json
import time
from dataclasses import dataclass
from aiohttp import web
from .admin import _plural, _scan_location, _scan_roots, _scan_status_line
from .images import (
THUMBNAIL_ENCODINGS,
_BodyTooLarge,
_new_thumbnail_stats,
_read_json_body,
_record_thumbnail_result,
_thumbnail_batch_size,
_thumbnail_complete_payload,
_thumbnail_progress_payload,
_worker_threads,
)
from .job_routes import busy_response
from .jobs import THUMBNAIL_GROUP_NAMES, THUMBNAILS_JOB_KIND, JobBusy
THUMBNAIL_JOB_TITLE = "Thumbnail generation"
THUMBNAIL_MODES = ("missing", "rebuild")
THUMBNAIL_QUALITIES = tuple(THUMBNAIL_ENCODINGS)
MIN_THUMBNAIL_EDGE = 128
MAX_THUMBNAIL_EDGE = 2048
# The concept's "Preview size limit" default.
DEFAULT_THUMBNAIL_EDGE = 512
DEFAULT_THUMBNAIL_OPTIONS = {
"mode": "missing",
"size": DEFAULT_THUMBNAIL_EDGE,
"quality": "medium",
}
_OPTIONS_ERROR = (
'mode must be "missing" or "rebuild", quality "low", "medium" or "high", '
f"and size a whole number of pixels from {MIN_THUMBNAIL_EDGE} "
f"to {MAX_THUMBNAIL_EDGE}"
)
CANCELLED_ERROR = "Thumbnail generation was cancelled"
CANCELLED_BEFORE_WRITING = "Cancelled before any preview was written"
NOTHING_TO_PREVIEW = "No images or videos to preview"
_MODE_LOG = {"missing": "missing previews", "rebuild": "every preview"}
# Outcome rows: what happened to each file's preview.
_DETAILS = {
"generated": "Preview written",
"skipped": "Preview already up to date",
}
_LOG_PREFIXES = {"generated": "+", "errors": "!"}
@dataclass(frozen=True)
class ThumbnailOptions:
"""One run's choices: the mode, the preview's longest edge and quality."""
mode: str
edge: int
quality: str
@property
def size(self):
return (self.edge, self.edge)
@property
def encoding(self):
return THUMBNAIL_ENCODINGS[self.quality]
@property
def rebuild(self):
return self.mode == "rebuild"
def scope(self):
return {"mode": self.mode, "size": self.edge, "quality": self.quality}
def _edge(value, *, from_query):
"""The preview edge as an int, or None when it is not one in range."""
if from_query and isinstance(value, str):
if not value.isascii() or not value.isdigit() or len(value) > 5:
return None
value = int(value)
if isinstance(value, bool) or not isinstance(value, int):
return None
return value if MIN_THUMBNAIL_EDGE <= value <= MAX_THUMBNAIL_EDGE else None
def thumbnail_options(raw, *, from_query=False):
"""``(ThumbnailOptions, None)`` from a body or query, or ``(None, message)``.
Absent fields take ``DEFAULT_THUMBNAIL_OPTIONS``. Query values are
strings, so a query ``size`` may be decimal digits; a JSON ``size`` must
be an int.
"""
if not isinstance(raw, dict) and not hasattr(raw, "get"):
return None, _OPTIONS_ERROR
mode = raw.get("mode", DEFAULT_THUMBNAIL_OPTIONS["mode"])
quality = raw.get("quality", DEFAULT_THUMBNAIL_OPTIONS["quality"])
edge = _edge(raw.get("size", DEFAULT_THUMBNAIL_EDGE), from_query=from_query)
if mode not in THUMBNAIL_MODES or quality not in THUMBNAIL_QUALITIES:
return None, _OPTIONS_ERROR
if edge is None:
return None, _OPTIONS_ERROR
return ThumbnailOptions(mode, edge, quality), None
def _invalid(message):
return web.json_response(
{"success": False, "code": "invalid-request", "error": message}, status=400
)
def _frame(event, data):
"""A legacy stream frame as the job publishes it."""
return {"event": event, "data": data}
def _finished_line(plan, stats):
"""A finished job's status line: what it wrote, or that it had nothing to do."""
if not plan["targets"]:
if plan["skipped"]:
return f"No previews to generate · {plan['skipped']:,} already up to date"
return NOTHING_TO_PREVIEW
parts = [f"{_plural(stats['generated'], 'preview')} written"]
if stats["errors"]:
parts.append(_plural(len(stats["errors"]), "error"))
if plan["remaining"]:
parts.append(f"{plan['remaining']:,} more on the next run")
return " · ".join(parts)
class ThumbnailJobsMixin:
"""The thumbnail plan, the thumbnail job and the legacy stream on it."""
async def get_thumbnail_plan(self, request):
"""What a thumbnail run would do (read-only; capped like a run)."""
options, error = thumbnail_options(request.query, from_query=True)
if error:
return _invalid(error)
try:
output_dirs = await self._run_in_executor(self._get_all_output_dirs)
plan = await self._run_in_executor(
self._plan_gallery_thumbnails, output_dirs, rebuild=options.rebuild
)
except Exception:
self.logger.exception("Thumbnail plan failed")
return web.json_response(
{
"success": False,
"code": None,
"error": "The thumbnail plan could not be read",
},
status=500,
)
return web.json_response(
{
"success": True,
"targets": len(plan["targets"]),
"fresh": plan["fresh"],
"remaining": plan["remaining"],
"roots": [self._public_path(root) for root in output_dirs],
**options.scope(),
},
headers={"Cache-Control": "no-store"},
)
async def start_thumbnail_job(self, request):
"""Start the thumbnail job (202), or 409 naming the job in the way."""
body = {}
if request.can_read_body:
try:
body = await _read_json_body(request)
except _BodyTooLarge:
return web.json_response(
{"success": False, "code": "too-large", "error": "Body too large"},
status=413,
)
except (ValueError, UnicodeDecodeError):
return _invalid(_OPTIONS_ERROR)
options, error = thumbnail_options(body)
if error:
return _invalid(error)
try:
job = self._start_thumbnail_job(options)
except JobBusy as busy:
return busy_response(busy)
return web.json_response({"success": True, "job": job.summary()}, status=202)
def _start_thumbnail_job(self, options):
"""Register the thumbnail job synchronously; raises JobBusy."""
return self._jobs.start(
THUMBNAILS_JOB_KIND,
THUMBNAIL_JOB_TITLE,
admission="guarded",
cancellable=True,
scope=options.scope(),
runner=functools.partial(self._run_thumbnail_job, options=options),
groups=THUMBNAIL_GROUP_NAMES,
)
# ── the job ───────────────────────────────────────────────────────
async def _run_thumbnail_job(self, job, options):
try:
state = await self._thumbnail_job_body(job, options)
except Exception:
self.logger.exception("Thumbnail generation failed")
job.update(status_line="Thumbnail generation stopped on an internal error.")
job.log_line("Thumbnails failed · internal error")
job.publish(
_frame(
"error",
{
"error": "An internal error occurred",
"message": "Thumbnail generation failed. Check server logs.",
},
)
)
state = "failed"
job.finish(state)
async def _thumbnail_job_body(self, job, options):
"""Plan, then generate in worker batches; returns the final state.
Every exit sets the job's final status line (I3): what was written,
that there was nothing to do, or what a cancel kept.
"""
output_dirs = await self._run_in_executor(self._get_all_output_dirs)
if not output_dirs:
message = "ComfyUI output directory not found"
job.update(status_line=message)
job.log_line(f"Thumbnails failed · {message}")
job.publish(_frame("error", {"error": message}))
return "failed"
folders = _plural(len(output_dirs), "output folder")
job.scope["output_folders"] = len(output_dirs)
job.update(status_line=f"Looking for media files in {folders}")
job.log_line(
f"Thumbnails started · {folders} · {_MODE_LOG[options.mode]} · "
f"{options.edge} px · {options.quality}"
)
job.publish(
_frame(
"status",
{"phase": "scanning", "message": "Scanning for images and videos..."},
)
)
plan = await self._run_in_executor(
self._plan_gallery_thumbnails,
output_dirs,
rebuild=options.rebuild,
stop=lambda: job.cancelled,
)
if job.cancelled: # M2: stopped during the walk, before any start frame
return self._end_cancelled(job, 0, CANCELLED_BEFORE_WRITING)
roots = _scan_roots(output_dirs)
self._record_fresh(job, plan, roots)
stats, start = await self._generate_thumbnail_targets(job, options, plan, roots)
self.invalidate_gallery_cache()
generated = _plural(stats["generated"], "preview")
if job.cancelled:
return self._end_cancelled(
job, stats["generated"], f"Cancelled · {generated} written"
)
payload = _thumbnail_complete_payload(
len(plan["targets"]) + plan["skipped"] + plan["remaining"],
stats,
start,
remaining=plan["remaining"],
processed=len(plan["targets"]),
)
job.update(progress=100, status_line=_finished_line(plan, stats))
if plan["remaining"]:
job.log_line(
f"{plan['remaining']:,} more previews are left for the next run"
)
errors = _plural(len(stats["errors"]), "error")
job.log_line(f"Thumbnails finished · {generated} written · {errors}")
self.logger.info(payload["message"])
job.publish(_frame("complete", payload))
return "finished"
@staticmethod
def _end_cancelled(job, generated, status_line):
"""Close a cancelled job: its status line, log line and error frame."""
job.update(status_line=status_line)
job.log_line(f"Thumbnails cancelled · {_plural(generated, 'preview')} written")
job.publish(
_frame("error", {"error": CANCELLED_ERROR, "message": CANCELLED_ERROR})
)
return "cancelled"
def _record_fresh(self, job, plan, roots):
"""List the already fresh previews under Skipped (no log line each)."""
rows = []
for src in plan["fresh_files"]:
path, label, index = _scan_location(src, roots)
rows.append(
{
"path": path,
"root": label,
"root_index": index,
"detail": _DETAILS["skipped"],
}
)
job.record_counted("skipped", plan["skipped"], rows)
if plan["skipped"]:
job.log_line(f"= {plan['skipped']:,} previews already up to date")
async def _generate_thumbnail_targets(self, job, options, plan, roots):
"""Generate ``plan``'s targets in worker batches, publishing each file."""
targets = plan["targets"]
total = len(targets)
video_count = sum(1 for _, _, is_video in targets if is_video)
message = (
f"Found {total - video_count} images and {video_count} videos to process"
)
if plan["skipped"]:
message += f" ({plan['skipped']} already have thumbnails)"
job.update(total=total)
if total: # never "Processing file 0 of 0" (I3)
job.update(status_line=_scan_status_line(0, total, 0, 0))
job.publish(
_frame(
"start",
{
"total_images": total,
"skipped": plan["skipped"],
"remaining": plan["remaining"],
"phase": "processing",
"image_count": total - video_count,
"video_count": video_count,
"message": message,
},
)
)
stats = _new_thumbnail_stats()
stats["skipped"] = plan["skipped"]
start = time.monotonic()
batch_size = _thumbnail_batch_size(_worker_threads())
generate = {
"encoding": options.encoding,
"rebuild": options.rebuild,
"stop": lambda: job.cancelled,
}
index = 0
batches = 0
for batch_start in range(0, total, batch_size):
if job.cancelled:
break
batch = targets[batch_start : batch_start + batch_size]
results = await self._run_in_executor(
self._generate_batch, batch, options.size, **generate
)
for (src, _dst, _video), result in zip(batch, results):
if result["action"] == "cancelled":
continue
index += 1
_record_thumbnail_result(stats, result)
self._record_thumbnail_outcome(job, src, result, roots)
job.update(processed=index, progress=index * 100 // total)
if result["action"] == "error":
job.publish(
_frame(
"file_error",
{"file": result["file"], "error": result["error"]},
)
)
job.publish(
_frame(
"progress",
_thumbnail_progress_payload(index, total, stats, result, start),
)
)
batches += 1
job.update(
status_line=_scan_status_line(
index, total, batches, time.monotonic() - start
)
)
await asyncio.sleep(0) # let other requests run between batches
return stats, start
@staticmethod
def _record_thumbnail_outcome(job, src, result, roots):
path, label, index = _scan_location(src, roots)
where = f"{label}/{path}" if label else path
if result["action"] == "error":
group, detail = "errors", result.get("error") or "failed"
elif result["action"] == "skipped":
group, detail = "skipped", _DETAILS["skipped"]
else:
group, detail = "generated", _DETAILS["generated"]
row = {"path": path, "root": label, "root_index": index, "detail": detail}
if group == "skipped":
job.record_counted(group, 1, [row])
return
job.record(group, row, f"{_LOG_PREFIXES[group]} {where} · {detail}")
# ── the legacy gallery stream ─────────────────────────────────────
async def generate_thumbnails_with_progress(self, request):
"""The gallery's thumbnail stream (POST, server-sent events).
Starts the thumbnail job with the gallery's quality preset as the
preview size (missing previews, medium encoding, as before), or
attaches to the running thumbnail job, and streams its frames. A
legacy hold (the blocking generate or clear) or a record-changing
job refuses it with the stream's error frame. The job goes on when
the client goes away.
"""
thumbnail_size = self._thumbnail_size(request.query.get("quality", "medium"))
response = web.StreamResponse(
status=200,
headers={
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
},
)
await response.prepare(request)
# M3: read the registry after the last await, so a job that started
# while the headers went out is attached to, not refused.
job, started, busy = self._legacy_stream_job(thumbnail_size)
if busy is not None:
message = f"{busy.title} already running"
await self._emit_sse(
response, "error", {"error": message, "message": message}
)
return response
# No await between the start and the subscription: the lead frame is
# sent after it, so the stream misses none of the runner's frames.
events = job.events(lead=_frame("job", {"id": job.id, "started": started}))
try:
async for payload in events:
frame = (
f"event: {payload['event']}\ndata: {json.dumps(payload['data'])}"
)
await response.write(f"{frame}\n\n".encode("utf-8"))
except Exception as exc: # a write failed: the client went away
self.logger.info(
f"Thumbnail progress client disconnected ({exc}); "
"the thumbnails continue in the background"
)
finally:
# Also when aiohttp cancels the handler on disconnect: the job is
# its own task and goes on.
await events.aclose()
return response
def _legacy_stream_job(self, thumbnail_size):
"""``(job, started, busy)`` for the legacy stream, synchronously.
Attaches to the running cancellable thumbnail job, or starts one with
the gallery's preset as the size (missing previews, medium encoding,
as the gallery always wrote), or names the job in the way.
"""
running = self._jobs.active(THUMBNAILS_JOB_KIND)
if running is not None and running.cancellable:
return running, False, None
busy = self._jobs.blocker(THUMBNAILS_JOB_KIND, admission="guarded")
if busy is not None:
return None, False, busy
# blocker() said free and nothing awaited since, so the start is admitted.
options = ThumbnailOptions("missing", thumbnail_size[0], "medium")
return self._start_thumbnail_job(options), True, None
-2
View File
@@ -127,8 +127,6 @@ export async function openProduction(page,route,theme,motion='no-preference') {
await expect(page.locator('#primary-nav .nav-count')).toHaveCount(3);
// SAVED VIEWS' Favorites count (D51), the concept's starred images in the concept fixture.
await expect(page.locator('#saved-nav a[data-route="favorites"] .nav-count')).toHaveCount(1);
// SAVED VIEWS' Needs review count (ruling R5): the exact pending count.
await expect(page.locator('#saved-nav a[data-route="review"] .nav-count')).toHaveCount(1);
}
/** Computed outline of the focused element, after keyboard focus so `:focus-visible` applies. */
-212
View File
@@ -1,212 +0,0 @@
// Needs review against the approved concept (#249 PR F, plan Task F3, Mock inventory M4: app.js renderReview
// a749–760, style.css s407–415, s796–814 and their breakpoints, screenshot needs-review-tagging.png). Production runs
// on the review fixture's concept-shaped queue (workspace_review_fixture.py `concept: true`: the concept's first three
// image names and sizes, and the concept's first prompt on the open item), so every remaining difference is a recorded
// deviation applied by mapConceptReview (D3, D29, D119) or a test failure. Paired captures of every state are attached
// as artifacts; no screenshot baseline is added.
import { test, expect } from './fixtures.mjs';
import { openConcept, openProduction, inventoryOf, mutate, saveImageTags, diffInventory, SIZES, THEMES } from './concept-inventory.mjs';
import { reviewInventory, mapConceptReview, taggingInventory, mapConceptTagging, diffTaggingInventory } from './tagging-canonical-comparison.mjs';
const CONCEPT_FILE = 'alpine_00001.png';
const CONCEPT_TAGS = ['cinematic', 'blue_hour', 'architecture'];
const SURFACE = ['h2', 'p.small', 'div.row-actions.spaced-15', 'div.field', 'section.positive.prompt-section',
'section.negative.prompt-section', 'div.dialog-actions'];
// The fake model's server default instructions (tests/browser/server.py TestModel.default_prompt), D84.
const INSTRUCTIONS = 'Describe the image';
const panel = page => page.locator('#workspace-main section.review-panel');
const tagDialog = page => page.getByRole('dialog', { name: 'Tag image', exact: true });
async function seed(request, data = {}) {
await expect(await request.post('/__test__/review/reset', { data: { items: 3, concept: true, ...data } })).toBeOK();
}
/** Production's Needs review page with its first item's sampler paths loaded. */
async function openReview(page, theme) {
await openProduction(page, 'review', theme);
await expect(panel(page).getByRole('combobox', { name: 'Sampler context', exact: true })).toBeEnabled();
await settle(page);
}
async function settle(page) {
await page.mouse.move(0, 0);
await page.evaluate(async () => {
await document.fonts.ready;
await Promise.all([...document.images].map(image => image.decode().catch(() => {})));
});
}
const conceptInventory = page => page.evaluate(reviewInventory, '#main');
const productionInventory = page => page.evaluate(reviewInventory, '#workspace-main');
const node = (tree, prefix) => tree.tag?.startsWith(prefix) ? tree : tree.children?.map(child => node(child, prefix)).find(Boolean) ?? null;
/** Computed values of CSS tokens, read through a probe element in the open page. */
const tokens = (page, declarations) => page.evaluate(text => {
const probe = document.createElement('i');
probe.style.cssText = text;
document.body.append(probe);
const style = getComputedStyle(probe), values = [style.borderTopColor, style.backgroundColor];
probe.remove();
return values;
}, declarations);
async function compareAt(page, request, width, height, theme) {
await seed(request);
await page.setViewportSize({ width, height });
await openConcept(page, 'review', theme);
await settle(page);
const concept = await conceptInventory(page);
await openReview(page, theme);
const actual = await productionInventory(page);
return { concept, actual, reference: mapConceptReview(concept, actual) };
}
test.beforeEach(async ({ page }) => {
page.errors = [];
page.on('pageerror', error => page.errors.push(error.message));
});
test.afterEach(async ({ page, request }) => {
expect(page.errors).toEqual([]);
await request.post('/__test__/jobs/restart');
});
for (const [width, height] of SIZES) for (const theme of THEMES) {
test(`concept and production Metadata review match at ${width}x${height} ${theme}`, async ({ page, request }) => {
const { actual, reference } = await compareAt(page, request, width, height, theme);
expect(diffInventory(reference, actual), `${width}x${height} ${theme}`).toEqual([]);
// Pinned concept values (M4), so a mapping mistake on both sides cannot pass unseen.
const main = actual.main;
expect(main.children.map(child => child.tag)).toEqual(['div.heading', 'div.notice', 'div.gallery-layout']);
const notice = node(main, 'div.notice');
const [accent, soft] = await tokens(page, 'border-top:1px solid var(--ca-accent);background:var(--soft-blue)');
expect([notice.borderTopColor, notice.borderLeftColor, notice.backgroundColor]).toEqual([accent, accent, soft]);
expect([notice.borderTopWidth, notice.borderTopLeftRadius, notice.paddingTop, notice.paddingLeft, notice.marginBottom])
.toEqual(['1px', '6px', '15px', '18px', '22px']);
expect(notice.flexDirection).toBe(width <= 700 ? 'column' : 'row');
expect(notice.children.map(child => child.children.map(item => [item.tag, item.text])))
.toEqual([[['h2', '3 images need a decision'], ['p', 'Multiple sampler paths were found. The existing image links were preserved. Choose the correct context or leave them unresolved.']]]);
const layout = node(main, 'div.gallery-layout');
expect(layout.gridTemplateColumns).toBe(width <= 700 ? `${layout.width}px` : `${layout.width - 22 - 310}px 310px`);
const surface = node(main, 'section.review-panel');
expect(surface.children.map(child => child.tag)).toEqual(SURFACE);
expect(surface.children[0].text).toBe('Resolve the association');
expect(surface.children[1].text).toBe(CONCEPT_FILE);
const actions = surface.children.at(-1);
expect(actions.children.map(button => [button.text, button.primary])).toEqual([['Leave for later', false], ['Apply association', true]]);
expect(node(surface, 'select').options.at(-1)).toEqual(['Leave unresolved', false]);
expect(node(main, 'div.image-grid').children.map(card => node(card, 'h3').text))
.toEqual(['alpine_00001.png', 'dunes_00001.png', 'forest_00001.png']);
expect(actual.noPageScroll).toBe(true);
});
}
test('the gallery layout follows the concept at every breakpoint', async ({ page, request }) => {
await seed(request);
// 310px beside the queue, 275px and a 15px gap at ≤1250px, 280px at ≤950px, one column at ≤700px.
for (const [width, columns, gap] of [[1600, '310px', '22px'], [1200, '275px', '15px'], [900, '280px', '15px'], [390, null, '15px']]) {
await page.setViewportSize({ width, height: 1000 });
await openConcept(page, 'review', 'dark');
const concept = node((await conceptInventory(page)).main, 'div.gallery-layout');
await openReview(page, 'dark');
const actual = node((await productionInventory(page)).main, 'div.gallery-layout');
expect([actual.gridTemplateColumns, actual.columnGap], `${width}px`).toEqual([concept.gridTemplateColumns, concept.columnGap]);
expect([actual.gridTemplateColumns.split(' ').at(-1), actual.columnGap], `${width}px`).toEqual([columns ?? `${actual.width}px`, gap]);
expect(actual.gridTemplateColumns.split(' ').length, `${width}px`).toBe(width <= 700 ? 1 : 2);
}
});
test('SAVED VIEWS rows match the concept with Needs review after Favorites', async ({ page, request }) => {
await seed(request);
await page.setViewportSize({ width: 1600, height: 1100 });
await openConcept(page, 'review', 'dark');
const concept = (await inventoryOf(page, '#main')).chrome.navs.find(nav => nav.id === 'saved-nav');
await openReview(page, 'dark');
const actual = (await inventoryOf(page, '#workspace-main')).chrome.navs.find(nav => nav.id === 'saved-nav');
// Sample data: the concept stars three images; this queue's fixture stars none. Needs review counts the queue (3).
const reference = { ...concept, rows: concept.rows.map(row => row.text === 'Favorites' ? { ...row, count: { ...row.count, text: '0' } } : row) };
expect(diffInventory(reference, actual)).toEqual([]);
expect(actual.rows.map(row => [row.text, row.count?.text, row.current])).toEqual([['Favorites', '0', null], ['Needs review', '3', 'page']]);
// The count's auto start margin follows the row's own text; every other count token is shared.
const { text: favoriteCount, marginInlineStart: favoriteMargin, ...favoriteStyle } = actual.rows[0].count;
const { text: reviewCount, marginInlineStart: reviewMargin, ...reviewStyle } = actual.rows[1].count;
expect(reviewStyle).toEqual(favoriteStyle);
expect(actual.rows[1].icon.d).toBe('M4 3h16v18H4zM8 7h8M8 12h8M8 17h5');
});
test('review Tag dialog matches the needs-review-tagging reference', async ({ page, request }) => {
await seed(request);
await saveImageTags(request, `review/${CONCEPT_FILE}`, CONCEPT_TAGS);
await page.setViewportSize({ width: 1600, height: 1100 });
await openConcept(page, 'review', 'dark');
await page.locator('#main section.surface').getByRole('button', { name: 'Tag image', exact: true }).click();
await expect(tagDialog(page)).toBeVisible();
await settle(page);
const opened = await page.locator('dialog#modal').evaluate(taggingInventory);
await page.keyboard.press('Escape');
await openReview(page, 'dark');
await panel(page).getByRole('button', { name: 'Tag image', exact: true }).click();
await expect(tagDialog(page).locator('img.tag-image-preview')).toHaveAttribute('src', /\/image-annotations\/capture\//);
await expect(tagDialog(page).locator('#manual-image-tags')).toHaveValue(CONCEPT_TAGS.join(', '));
await settle(page);
const actual = await page.locator('dialog#modal').evaluate(taggingInventory);
const numberField = opened.modelPanel.fields.find(field => field.control.id === 'image-general-threshold');
const mapped = mapConceptTagging(opened, { instructions: INSTRUCTIONS, chip: opened.heading.size, numberField });
// D119: an undecided queue item names no prompt, so the heading and the preview's alt are the file name.
const reference = { ...mapped, heading: { ...mapped.heading, h3: { ...mapped.heading.h3, text: CONCEPT_FILE },
preview: { ...mapped.heading.preview, alt: CONCEPT_FILE } } };
expect(actual.scopeNote.text).toBe('Tags belong to this image. Resolve its prompt association separately in Needs review.');
expect(diffTaggingInventory(reference, actual)).toEqual([]);
await page.keyboard.press('Escape');
await expect(tagDialog(page)).toHaveCount(0);
});
test('the comparison notices a dropped notice border and reordered actions', async ({ page, request }) => {
const { actual, reference } = await compareAt(page, request, 1600, 1100, 'dark');
expect(diffInventory(reference, actual)).toEqual([]);
const undoBorder = await mutate(page, { css: '#workspace-main .notice { border: 0 !important; }' });
const borderless = await productionInventory(page);
expect(diffInventory(reference, borderless).map(item => item.path)).toEqual(expect.arrayContaining([
'main.children.1.borderTopWidth', 'main.children.1.borderTopStyle']));
await undoBorder();
await panel(page).locator('.dialog-actions').evaluate(row => row.prepend(row.lastElementChild));
const reordered = await productionInventory(page);
expect(diffInventory(reference, reordered).map(item => item.path)).toEqual(expect.arrayContaining([
'main.children.2.children.1.children.6.children.0.text', 'main.children.2.children.1.children.6.children.0.primary']));
await panel(page).locator('.dialog-actions').evaluate(row => row.prepend(row.lastElementChild));
expect(diffInventory(reference, await productionInventory(page))).toEqual([]);
});
// Paired captures for the PR (no baselines): the populated queue beside the concept in both themes and widths, then
// the states the concept lacks (conflict, limited item, unavailable file, empty queue) as production captures.
test('paired Needs review captures', async ({ page, request }, testInfo) => {
test.setTimeout(120_000);
const shot = async name => {
await settle(page);
const file = testInfo.outputPath(`${name}.png`);
await page.screenshot({ path: file, fullPage: true });
await testInfo.attach(name, { path: file, contentType: 'image/png' });
};
await seed(request);
for (const [width, height] of SIZES) for (const theme of THEMES) {
await page.setViewportSize({ width, height });
await openConcept(page, 'review', theme);
await shot(`concept-review-${width}-${theme}`);
await openReview(page, theme);
await shot(`production-review-${width}-${theme}`);
}
await page.setViewportSize({ width: 1600, height: 1100 });
await seed(request, { items: 0, conflict: true, limited: true, unavailable: true });
await openReview(page, 'dark');
await panel(page).getByRole('combobox', { name: 'Sampler context', exact: true }).selectOption({ index: 0 });
const decided = page.waitForResponse(item => /\/decision$/.test(item.url()));
await panel(page).getByRole('button', { name: 'Apply association', exact: true }).click();
expect((await decided).status()).toBe(409);
await expect(panel(page).locator('.warning-note')).toBeVisible();
await shot('production-review-conflict');
for (const [index, name] of [[1, 'limited'], [2, 'unavailable']]) {
await page.locator('#workspace-main .review-queue article').nth(index).getByRole('button', { name: /^Inspect / }).click();
await expect(panel(page).locator('.warning-note')).toBeVisible();
await shot(`production-review-${name}`);
}
await seed(request, { items: 0 });
await openProduction(page, 'review', 'dark');
await expect(page.locator('#workspace-main .notice h2')).toHaveText('No images need a decision');
await shot('production-review-empty');
});
-4
View File
@@ -40,7 +40,6 @@ from tests.browser.workspace_library_editor_fixture import (
)
from tests.browser.workspace_vocabulary_fixture import register_vocabulary_commands
from tests.browser.workspace_tagging_fixture import register_tagging_commands
from tests.browser.workspace_review_fixture import register_review_commands
CANONICAL_ROOT = Path(__file__).resolve().parents[2]
CANONICAL_ASSETS = {
@@ -198,7 +197,6 @@ def create_app(root: Path) -> web.Application:
library_editor_commands = register_library_editor_commands(routes, api, root)
register_vocabulary_commands(routes, api, root)
register_tagging_commands(routes, api, root, model)
register_review_commands(routes, api, root)
reveal_stub = register_reveal_stub(routes, api, root)
register_activity_commands(app, routes, api)
register_system_commands(app, routes, api, root)
@@ -600,8 +598,6 @@ def create_app(root: Path) -> web.Application:
api.db.close_all()
seed()
# Open Needs review change streams end first, as on the real host.
app.on_shutdown.append(api.close_review_events)
app.on_cleanup.append(cleanup)
app.add_routes(routes)
return app
+5 -6
View File
@@ -68,12 +68,12 @@ function remainingFooter(footer,spans) {
const OVERVIEW_GLYPH='M3 11 12 4l9 7M5 10v10h14V10M10 20v-6h4v6';
/**
* The concept's SAVED VIEWS rows (app.js a331–335) as production lists them: Favorites with its count (D51), Needs
* review with the exact pending count (ruling R5), then the saved views. The concept's "3" is sample data: the concept
* fixture records no review items, so production's count reads "0".
* The concept's SAVED VIEWS rows (app.js a331–335) as production lists them: Favorites with its count (D51), then the
* saved views. D50: no "Needs review" row until the tag-review slice (maintainer ruling 2026-10-21).
*/
function savedNav(nav) {
return {...nav,rows:nav.rows.map(row=>row.text==='Needs review'?{...row,count:{...row.count,text:'0'}}:row)};
const rows=nav.rows.filter(row=>row.text!=='Needs review');
return {...nav,rows:rows.map((row,index)=>index===rows.length-1?{...row,gapAfter:null}:row)};
}
/** Recorded shell deviations only (spec §8). Everything else in the concept inventory must equal production. */
@@ -243,9 +243,8 @@ for(const [width,height] of SIZES)test(`a saved view row matches the concept's $
await openProduction(page,'images','dark');
const actual=await save('Save','#workspace-main');
expect(diffInventory(mapShellConcept(reference).chrome.navs,actual.chrome.navs)).toEqual([]);
const referenceRows=reference.chrome.navs.find(nav=>nav.id==='saved-nav').rows;
expect(actual.chrome.navs.find(nav=>nav.id==='saved-nav').rows.map(row=>[row.text,row.count]))
.toEqual([['Favorites',referenceRows[0].count],['Needs review',{...referenceRows[1].count,text:'0'}],['Golden hour',null]]);
.toEqual([['Favorites',reference.chrome.navs.find(nav=>nav.id==='saved-nav').rows[0].count],['Golden hour',null]]);
});
test('the inventory reports a missing shell element by its path instead of failing',async({page})=>{
+5 -131
View File
@@ -169,9 +169,10 @@ function mapOverviewConcept(reference, overview) {
const relabel = { 'Database integrity': 'Not checked', 'Positive prompts': number(overview.library.positive_prompts),
'Negative variants': number(overview.library.negative_variants), Images: number(overview.images.indexed) };
const rows = {
// D93: the real pending count; Review metadata opens Needs review (the concept's go-review).
// D93: the real pending count; the action is unavailable (dimmed, as the Save row's D92) until the review page.
'3 ambiguous image links': row => ({ ...row,
title: `${number(overview.needs_review.pending)} ambiguous image link${overview.needs_review.pending === 1 ? '' : 's'}` }),
title: `${number(overview.needs_review.pending)} ambiguous image link${overview.needs_review.pending === 1 ? '' : 's'}`,
action: { ...row.action, opacity: '0.55', cursor: 'not-allowed' } }),
// D93: no backup has been recorded (verified backups arrive with S8).
'Backup status': row => ({ ...row, description: 'No retained backup record.' }),
// D93: there is no paging, so the shared settings are grid and density.
@@ -230,10 +231,11 @@ for (const theme of THEMES) for (const [width, height] of SIZES) {
});
}
// The recorded differences, by path: D93 the review row (its real count), the backup row, the
// The recorded differences, by path: D93 the review row (its real count, dimmed action), the backup row, the
// preferences row and the sources row, and metric 4; D96 the statistics row; D92 the chip, the status and the dimmed
// Save; D122 the nav label. (The fixture's first three metrics are the concept's own counts.)
const UNMAPPED_PATHS = [
'columns.surfaces.0.rows.0.action.cursor', 'columns.surfaces.0.rows.0.action.opacity',
'columns.surfaces.0.rows.0.title', 'columns.surfaces.0.rows.1.description',
'columns.surfaces.1.rows.0.description', 'columns.surfaces.1.rows.1.title', 'columns.surfaces.1.rows.2.description',
'heading.actions.0', 'heading.actions.length', 'metrics.items.3.value', 'nav.label',
@@ -775,131 +777,3 @@ test('add source comparison detects mutations', async ({ page, request }) => {
await undoStyle();
expect(diffInventory(expected, await page.evaluate(measureAddSource))).toEqual([]);
});
/**
* Runs in the page: the open "Thumbnail maintenance" dialog (the concept's `thumbnail-review`, app.js 1433–1437): its
* title, the body's tree, each `.field` (label, control, options, selection, limits), the `p.small` line and the
* actions, with the concept's geometry and tokens. The concept has no `p.small`, so a probe one stands in for its style.
*/
function measureThumbnailReview() {
const dialog = document.querySelector('dialog#modal[open]');
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
const text = node => node?.textContent.trim() ?? null;
const body = dialog.querySelector('#modal-body');
const fields = [...body.querySelectorAll(':scope > .field')].map(field => {
const label = field.querySelector(':scope > label');
const control = field.querySelector(':scope > select, :scope > input');
return {
tree: [...field.children].map(name),
...css(field, 'display', 'flexDirection', 'rowGap', 'marginBottom'),
label: { text: text(label), for: label.htmlFor, ...css(label, 'fontSize', 'fontWeight', 'color') },
control: { id: control.id, tag: control.localName, type: control.type,
options: control.options ? [...control.options].map(option => option.textContent) : null,
selected: control.options ? control.selectedIndex : null,
value: control.localName === 'input' ? control.value : null,
min: control.getAttribute('min'), max: control.getAttribute('max'),
...css(control, 'height', 'paddingLeft', 'fontSize', 'color', 'backgroundColor', 'borderTopColor', 'borderTopLeftRadius') },
};
});
let line = body.querySelector(':scope > p.small');
const probe = !line;
if (probe) { line = document.createElement('p'); line.className = 'small'; body.append(line); }
const plan = { text: probe ? null : text(line), role: probe ? null : line.getAttribute('role'),
...css(line, 'fontSize', 'color', 'marginTop', 'marginBottom') };
if (probe) line.remove();
const actions = body.querySelector(':scope > .dialog-actions');
const rect = dialog.getBoundingClientRect();
return {
title: text(dialog.querySelector('#modal-title')),
tree: [...body.children].map(name),
fields,
plan,
actions: { tree: [...actions.children].map(name), ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
buttons: [...actions.children].map(button => ({ text: text(button), className: button.className,
...css(button, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') })) },
width: Math.round(rect.width),
};
}
/**
* D99: the planned counts line (`p.small[role=status]`) before the actions, from the server's plan for the chosen
* options, and the primary "Start" (the concept's "Simulate thumbnail job").
*/
function mapThumbnailConcept(reference, plan) {
const [cancel, start] = reference.actions.buttons;
const tree = [...reference.tree];
tree.splice(tree.indexOf('div.dialog-actions'), 0, 'p.small');
return { ...reference, tree,
plan: { ...reference.plan, role: 'status',
text: `${number(plan.targets)} ${plan.targets === 1 ? 'preview' : 'previews'} to generate · ${number(plan.fresh)} already up to date.` },
actions: { ...reference.actions, buttons: [cancel, { ...start, text: 'Start' }] } };
}
async function conceptThumbnailReview(page, theme) {
await conceptSources(page, theme, { notice: false });
const row = page.locator('#main .operation').filter({ has: page.getByRole('heading', { name: 'Thumbnail generation', exact: true }) });
await row.getByRole('button', { name: 'Review options', exact: true }).click();
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Thumbnail maintenance');
await page.mouse.move(0, 0);
return page.evaluate(measureThumbnailReview);
}
/** Production's dialog from Sources' Review options or Overview's Review thumbnails, once its plan line answered. */
async function productionThumbnailReview(page, theme, entry) {
if (entry === 'sources') {
await productionSources(page, theme, { notice: false });
const row = page.locator('#workspace-main .operation').filter({ has: page.getByRole('heading', { name: 'Thumbnail generation', exact: true }) });
await row.getByRole('button', { name: 'Review options', exact: true }).click();
} else {
await openProduction(page, 'system/overview', theme);
await page.locator('#workspace-main').getByRole('button', { name: 'Review thumbnails', exact: true }).click();
}
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Thumbnail maintenance');
await expect(page.locator('dialog#modal[open] #modal-body > p.small')).toHaveText(/ already up to date\.$/);
await page.mouse.move(0, 0);
return page.evaluate(measureThumbnailReview);
}
const thumbnailPlan = request => serverJson(request, '/prompt_manager/images/thumbnails/plan?mode=missing&size=512&quality=medium');
for (const entry of ['sources', 'overview']) for (const theme of THEMES) for (const [width, height] of SIZES) {
test(`thumbnail dialog matches the concept from ${entry} ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
await page.setViewportSize({ width, height });
await watchedSources(request);
const plan = await thumbnailPlan(request);
const concept = await conceptThumbnailReview(page, theme);
await page.screenshot({ path: testInfo.outputPath(`concept-thumbnail-review-${theme}-${width}.png`) });
const production = await productionThumbnailReview(page, theme, entry);
await page.screenshot({ path: testInfo.outputPath(`production-thumbnail-review-${entry}-${theme}-${width}.png`) });
expect(diffInventory(mapThumbnailConcept(concept, plan), production)).toEqual([]);
});
}
test('thumbnail dialog comparison detects mutations', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await watchedSources(request);
const plan = await thumbnailPlan(request);
const concept = await conceptThumbnailReview(page, 'dark');
const expected = mapThumbnailConcept(concept, plan);
const production = await productionThumbnailReview(page, 'dark', 'sources');
expect(diffInventory(expected, production)).toEqual([]);
// Unmapped, the concept differs only where D99 is recorded.
expect(diffInventory(concept, production).map(item => item.path).sort())
.toEqual(['actions.buttons.1.text', 'plan.role', 'plan.text', 'tree.3', 'tree.4', 'tree.length']);
const paths = async () => diffInventory(expected, await page.evaluate(measureThumbnailReview)).map(item => item.path);
// A changed option label, a swapped option order, a swapped action order and a changed style.
await page.evaluate(() => { document.querySelector('#thumb-quality option').textContent = 'Medium'; });
expect(await paths()).toEqual(['fields.1.control.options.0']);
await page.evaluate(() => { document.querySelector('#thumb-quality option').textContent = 'Medium · optimized'; });
await page.evaluate(() => { const select = document.querySelector('#thumb-mode'); select.prepend(select.lastElementChild); });
expect(await paths()).toEqual(expect.arrayContaining(['fields.0.control.options.0', 'fields.0.control.options.1']));
await page.evaluate(() => { const select = document.querySelector('#thumb-mode'); select.prepend(select.lastElementChild); select.selectedIndex = 0; });
await page.evaluate(() => { const actions = document.querySelector('dialog#modal[open] .dialog-actions'); actions.prepend(actions.lastElementChild); });
expect(await paths()).toEqual(expect.arrayContaining(['actions.buttons.0.text', 'actions.buttons.1.text']));
await page.evaluate(() => { const actions = document.querySelector('dialog#modal[open] .dialog-actions'); actions.prepend(actions.lastElementChild); });
const undoStyle = await mutate(page, { css: 'dialog#modal #thumb-limit { border-color: rgb(255, 0, 0); } dialog#modal #modal-body > p.small { color: rgb(255, 0, 0); }' });
expect(await paths()).toEqual(expect.arrayContaining(['fields.2.control.borderTopColor', 'plan.color']));
await undoStyle();
expect(diffInventory(expected, await page.evaluate(measureThumbnailReview))).toEqual([]);
});
@@ -140,152 +140,3 @@ export function mapConceptTagging(reference,{instructions,limit=10,chip,numberFi
export function withoutChipGeometry(inventory) {
return {...inventory,suggestions:{...inventory.suggestions,rows:inventory.suggestions.rows.map(({paddingTop,paddingLeft,alignItems,...row})=>row)}};
}
/**
* Needs review (plan Task F3, Mock inventory M4: app.js renderReview a749–760, style.css s407–415, s796–814 and their
* breakpoints): the page's main element as a tree, runs in the page. Every shown box below `main` to `depth` levels is
* recorded with its tag and classes, its box relative to its parent, its fonts, edges and layout, a leaf's text, a
* button's label and icon, a select's options; a `display: contents` node is replaced by its children (D20). A flex
* row of actions also records the gaps between its children and its right inset, so a row whose button copy differs
* (D29) still compares by order, spacing and alignment.
*/
export function reviewInventory(selector) {
const main=document.querySelector(selector);
// Half pixels: the concept's 13.5px line heights add up exactly, so a box that loses one line moves by 13.5px.
const px=value=>Math.round(value*2)/2;
const css=(node,...keys)=>{const style=getComputedStyle(node);return Object.fromEntries(keys.map(key=>[key,style[key]]));};
const rect=node=>node.getBoundingClientRect();
const shown=node=>node.getClientRects().length>0&&getComputedStyle(node).visibility!=='hidden';
const relative=(node,origin)=>{const a=rect(node),b=rect(origin);return {left:px(a.left-b.left),top:px(a.top-b.top),width:px(a.width),height:px(a.height)};};
const font=node=>css(node,'fontFamily','fontSize','fontWeight','lineHeight','letterSpacing','color','textTransform','textAlign');
const edges=node=>css(node,'paddingTop','paddingRight','paddingBottom','paddingLeft','marginTop','marginRight','marginBottom','marginLeft',
'borderTopWidth','borderTopStyle','borderTopColor','borderRightWidth','borderRightStyle','borderRightColor','borderBottomWidth',
'borderBottomStyle','borderBottomColor','borderLeftWidth','borderLeftStyle','borderLeftColor','borderTopLeftRadius','backgroundColor',
'boxShadow','outlineStyle');
const layout=node=>css(node,'display','position','flexDirection','flexWrap','justifyContent','alignItems','columnGap','rowGap',
'gridTemplateColumns','overflowX','overflowY','objectFit','minHeight','minWidth');
const tagOf=node=>`${node.localName}${[...node.classList].sort().map(name=>`.${name}`).join('')}`;
const textOf=node=>node.textContent.replace(/\s+/g,' ').trim();
const boxed=node=>[...node.children].flatMap(child=>getComputedStyle(child).display==='contents'?boxed(child):[child]);
const LEAVES=new Set(['button','select','input','img','svg','textarea','progress']);
function nodeOf(node,origin,depth) {
const base={tag:tagOf(node),...relative(node,origin),...font(node),...edges(node),...layout(node)};
const name=node.localName;
if(name==='svg')return {tag:base.tag,width:base.width,height:base.height,d:node.querySelector('path')?.getAttribute('d')??null,
...css(node,'strokeWidth','color','verticalAlign')};
if(name==='button')return {...base,text:textOf(node),label:node.getAttribute('aria-label'),pressed:node.getAttribute('aria-pressed'),
disabled:node.getAttribute('aria-disabled')==='true'||node.disabled,primary:node.classList.contains('ca-primary'),
children:boxed(node).filter(shown).map(child=>nodeOf(child,node,0))};
if(name==='select')return {...base,id:node.id,options:[...node.options].map(option=>[option.textContent,option.disabled])};
if(name==='input')return {...base,id:node.id,type:node.type,label:node.getAttribute('aria-label')};
if(name==='img')return {...base,alt:node.getAttribute('alt')};
const children=boxed(node).filter(shown);
const item={...base,id:node.id||null,role:node.getAttribute('role'),label:node.getAttribute('aria-label'),
text:children.length?null:textOf(node),htmlFor:node.htmlFor||null};
if(/(^|\s)(dialog-actions|row-actions|image-card-actions)(\s|$)/.test(node.className)&&children.length) {
const boxes=children.map(rect);
item.gaps=boxes.slice(1).map((box,index)=>px(box.left-boxes[index].right));
item.rightInset=px(rect(node).right-boxes.at(-1).right);
}
if(depth>0&&!LEAVES.has(name))item.children=children.map(child=>nodeOf(child,node,depth-1));
return item;
}
return {main:nodeOf(main,main,8),noPageScroll:document.documentElement.scrollWidth<=innerWidth+1};
}
/**
* `tree` with `edit(node)` applied to every node `match` picks (a tag prefix or a predicate), depth first, new objects
* only.
*/
function editNodes(tree,match,edit) {
const children=tree.children?.map(child=>editNodes(child,match,edit));
const node=children?{...tree,children}:tree;
const picked=typeof match==='function'?match(node):node.tag?.startsWith(match);
return picked?edit(node):node;
}
const findNode=(tree,prefix)=>tree.tag?.startsWith(prefix)?tree:tree.children?.map(child=>findNode(child,prefix)).find(Boolean)??null;
/**
* The concept's Metadata review inventory (reviewInventory) with the recorded deviations applied, the remaining sample
* data taken from production (`actual`) only where the concept's own copy cannot say it:
* - D29: no "Sample review queue" chip in the notice; "Resolve the association" and "Apply association" without the
* prototype's "sample";
* - D119: queue cards carry no selection checkbox and no star (the overlay row stays, empty); a card's title is its
* file name (an undecided item names no prompt) and its line is the extension only (D11 omits the unknown size and
* seed, which a queue item does not carry); the image's alt is the file name; the prompt blocks' Copy and Send names
* carry no prompt title;
* - D3: the grid keeps the Images grid's `position: relative` (its cards stay in its flow);
* - production's hooks and names: the `review-queue` / `review-panel` classes, the queue's list roles and name, the
* panel heading's id (the region's name) and the prompt sections' region names;
* - sample data: the Sampler context's candidate options (the file's own evidence; the last option, Leave unresolved,
* stays compared) and the widths that follow from copy the deviations above change (a card's text column, the
* panel's action buttons; their gaps and right inset stay compared).
* Where that copy changes the height of a box, the boxes after it move by exactly that much: the stacked notice (390)
* loses the chip and its row gap; a card whose concept line wraps (390: "1536 × 1152 · PNG · Seed 42001") takes
* production's one-line height, its row of the grid shrinks by the most any of its cards lost, and in one column the
* panel below the grid moves up by the grid's loss. The actions keep production's centred place in the card's info.
*/
export function mapConceptReview(reference,actual) {
const named=(node,id)=>({...node,id});
const actualCards=findNode(actual.main,'div.image-grid')?.children??[];
const actualActions=findNode(actual.main,'div.dialog-actions');
let cardIndex=0;
let main={...reference.main,id:actual.main.id};
let noticeLoss=0;
main=editNodes(main,'div.notice',notice=>{
const chip=notice.children.find(child=>child.tag==='span.chip');
if(chip&&notice.flexDirection==='column')noticeLoss=chip.height+parseFloat(notice.rowGap);
return {...notice,height:notice.height-noticeLoss,children:notice.children.filter(child=>child!==chip)};
});
const noticeIndex=main.children.findIndex(child=>child.tag==='div.notice');
main={...main,height:main.height-noticeLoss,children:main.children.map((child,index)=>index>noticeIndex?{...child,top:child.top-noticeLoss}:child)};
const losses=[];
main=editNodes(main,'div.image-grid',grid=>({...grid,tag:'div.image-grid.review-queue',position:'relative',role:'list',label:'Review queue'}));
main=editNodes(main,'article.media-card',card=>{
const twin=actualCards[cardIndex++];
const filename=findNode(card,'button.image-open').label.replace(/^Open /,'');
const art=editNodes(card.children[0],'div.image-overlays',overlays=>({...overlays,height:0,text:'',children:[]}));
const withAlt=editNodes(art,'img',img=>({...img,alt:filename}));
const info=card.children[1];
const [copy,...rest]=info.children;
const twinCopy=twin?.children[1]?.children[0];
const [h3,p]=copy.children;
const twinLine=twinCopy?.children[1];
const loss=twinLine?p.height-twinLine.height:0;
losses.push({top:card.top,loss});
const mappedCopy={...copy,width:twinCopy?.width,height:copy.height-loss,children:[{...h3,text:filename,width:twinCopy?.children[0]?.width},
{...p,text:'PNG',width:twinLine?.width,height:p.height-loss}]};
const [actions,...after]=rest;
const twinActions=twin?.children[1]?.children[1];
const mappedActions=actions&&loss?{...actions,top:twinActions?.top}:actions;
return {...card,role:'listitem',height:card.height-loss,
children:[withAlt,{...info,height:info.height-loss,children:[mappedCopy,...(actions?[mappedActions]:[]),...after]},...card.children.slice(2)]};
});
// Each grid row loses what its most shortened card lost; later rows move up by the rows before them.
const rowLoss=new Map();
for(const {top,loss} of losses)rowLoss.set(top,Math.max(rowLoss.get(top)??0,loss));
const lossBefore=top=>[...rowLoss].filter(([rowTop])=>rowTop<top).reduce((sum,[,loss])=>sum+loss,0);
const gridLoss=[...rowLoss.values()].reduce((sum,loss)=>sum+loss,0);
main=editNodes(main,'div.image-grid',grid=>({...grid,height:grid.height-gridLoss,
children:grid.children.map(card=>({...card,top:card.top-lossBefore(card.top)}))}));
main=editNodes(main,'div.gallery-layout',layout=>{
if(!gridLoss||layout.gridTemplateColumns.trim().split(/\s+/).length>1)return layout;
return {...layout,height:layout.height-gridLoss,children:layout.children.map((child,index)=>index?{...child,top:child.top-gridLoss}:child)};
});
if(gridLoss&&findNode(main,'div.gallery-layout').gridTemplateColumns.trim().split(/\s+/).length===1)main={...main,height:main.height-gridLoss};
main=editNodes(main,'section.surface',surface=>{
const [h2,...rest]=surface.children;
return {...surface,tag:'section.review-panel.surface',children:[named({...h2,text:h2.text.replace('Resolve the sample association','Resolve the association')},'review-panel-title'),...rest]};
});
main=editNodes(main,'select',select=>{
const twin=findNode(actual.main,'select');
return select.id==='review-context'?{...select,options:[...twin.options.slice(0,-1),select.options.at(-1)]}:select;
});
main=editNodes(main,node=>/^section\..*\bprompt-section\b/.test(node.tag),section=>({...section,label:section.tag.includes('positive')?'Positive prompt':'Negative prompt'}));
main=editNodes(main,node=>node.tag?.startsWith('button')&&/^(Copy|Send) (positive|negative) prompt/.test(node.label??''),
button=>({...button,label:button.label.replace(/: .+$/,'')}));
main=editNodes(main,'div.dialog-actions',actions=>({...actions,children:actions.children.map((button,index)=>({...button,
text:button.text.replace('Apply sample association','Apply association'),
left:actualActions?.children[index]?.left,width:actualActions?.children[index]?.width}))}));
return {...reference,main};
}
+2 -95
View File
@@ -54,8 +54,8 @@ async function step(request, id, data) {
await expect(await request.post(`/__test__/jobs/${id}/step`, { data })).toBeOK();
}
async function finish(request, id, state = 'finished', extra = {}) {
const response = await request.post(`/__test__/jobs/${id}/finish`, { data: { state, ...extra } });
async function finish(request, id, state = 'finished') {
const response = await request.post(`/__test__/jobs/${id}/finish`, { data: { state } });
await expect(response).toBeOK();
return (await response.json()).job;
}
@@ -1008,96 +1008,3 @@ test('the report download names the kind', async ({ page, request }) => {
expect(report.outcomes.tagged.rows.map(row => [row.path, row.detail])).toEqual([['sub/a.png', 'Added 1 tag']]);
await expect(toast(page)).toHaveText('Job report downloaded.');
});
// D111 (System S5): each kind's own descriptor. A thumbnail job counts Generated (green), Skipped and Errors (red),
// its pill says "Generating thumbnails" and its toasts "Thumbnails finished · N generated · M errors".
const THUMB_JOB = { kind: 'thumbnails', total: 10, output_folders: 2 };
test('a thumbnail job shows Generated Skipped Errors, its pill verb and its toasts', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request, THUMB_JOB);
await step(request, job.id, { processed: 4, records: [
{ group: 'generated', path: 'sub/a.png', root: 'output', detail: 'Preview written' },
{ group: 'skipped', path: 'b.png', root: 'input', detail: 'Preview already up to date' },
{ group: 'errors', path: 'broken.png', root: 'output', detail: 'Not a valid image file' },
] });
await page.goto(`${workspace}#/images`);
await expect(pill(page)).toHaveText('Generating thumbnails · 40% · 4 / 10');
await page.goto(`${workspace}#/activity`);
await expect(items(page).first().locator('p')).toHaveText(/^Started \d\d:\d\d · 2 output folders · closing this view does not cancel it$/);
await expect(items(page).first().locator('progress')).toHaveAttribute('value', '40');
await openJob(page, job.id);
await expect(heading(page).locator('h1')).toHaveText('Thumbnail generation');
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report', 'Cancel job']);
expect(await tileValues(page)).toEqual([['stat added', 'Generated', '1'], ['stat', 'Skipped', '1'], ['stat bad', 'Errors', '1']]);
await expect(outcomeButtons(page)).toHaveText(['Generated1', 'Errors1', 'Skipped1']);
await expect(fileRows(page).first().locator('.path')).toHaveText('sub/a.png');
expect(await fileRows(page).first().evaluate(node => node.localName)).toBe('button');
await outcomeButtons(page).filter({ hasText: 'Errors' }).click();
expect(await fileRows(page).first().evaluate(node => node.localName)).toBe('div');
await expect(scanLog(page).locator('summary')).toHaveText('● Job log — raw detail, click to show');
await finish(request, job.id);
await expect(toast(page)).toHaveText('Thumbnails finished · 1 generated · 1 error');
await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d · 1 generated · 1 error$/);
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
});
test('a held legacy job and the health check show no tiles, no Results and no made-up percentage', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const response = await request.post('/__test__/jobs/hold',
{ data: { kind: 'thumbnails', title: 'Thumbnail clear (gallery page)', admission: 'guarded' } });
await expect(response).toBeOK();
const held = (await response.json()).job;
await page.goto(`${workspace}#/images`);
// D112: the pill names the held operation and shows no percentage; its bar is indeterminate, not an empty fill
// (S5 review M5), and stays hidden from assistive tech, which reads the label.
await expect(pill(page)).toHaveText('Thumbnail clear (gallery page)');
await expect(pill(page)).toHaveAttribute('aria-label', 'Thumbnail clear (gallery page)');
await expect(pill(page).locator('.activity-pill-bar')).toHaveClass('activity-pill-bar indeterminate');
await expect(pill(page).locator('.activity-pill-bar')).toHaveAttribute('aria-hidden', 'true');
expect(await pill(page).locator('.activity-pill-bar i').evaluate(node => node.style.width)).toBe('');
await page.goto(`${workspace}#/activity`);
const item = items(page).filter({ has: page.getByRole('link', { name: /Thumbnail clear \(gallery page\)$/ }) });
await expect(item.locator('p')).toHaveText(/^Started \d\d:\d\d · closing this view does not cancel it$/);
await expect(item.locator('progress')).toHaveCount(1);
await expect(item.locator('progress')).not.toHaveAttribute('value');
await expect(item.getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(0);
await item.getByRole('link').click();
await expect(heading(page).locator('h1')).toHaveText('Thumbnail clear (gallery page)');
await expect(progressSection(page).locator('progress')).not.toHaveAttribute('value');
await expect(progressSection(page).locator('dl.stat-row')).toHaveCount(0);
await expect(resultsSection(page)).toHaveCount(0);
await expect(scanLog(page)).toHaveCount(1);
await finish(request, held.id);
await expect(toast(page)).toHaveText('Thumbnail clear (gallery page) finished.');
await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d$/);
await expect(progressSection(page).locator('progress')).toHaveAttribute('value', '100');
// The health check (a read job of its own kind): its status line, no tiles, no Cancel.
const health = await startJob(request, { kind: 'health', total: 0 });
await openJob(page, health.id);
await expect(heading(page).locator('h1')).toHaveText('Health check');
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Checking the database and image links');
await expect(progressSection(page).locator('progress')).not.toHaveAttribute('value');
await expect(progressSection(page).locator('dl.stat-row')).toHaveCount(0);
await expect(resultsSection(page)).toHaveCount(0);
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
await expect(pill(page)).toHaveText('Health check');
await finish(request, health.id);
await expect(toast(page)).toHaveText('Health check finished.');
});
// S5 review I3: a thumbnail job with nothing to generate ends on the server's final line ("No previews to generate · N
// already up to date"), never on a stale "Processing file 0 of 0" under the Completed chip.
test('a finished thumbnail job with nothing to do says so', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request, { ...THUMB_JOB, total: 0 });
await step(request, job.id, { total: 0, processed: 0 }); // the stale line an older server left
await openJob(page, job.id);
const line = progressSection(page).locator('.panel-heading > span.small');
await finish(request, job.id, 'finished', { status_line: 'No previews to generate · 24 already up to date' });
await expect(line).toHaveText('No previews to generate · 24 already up to date');
await expect(page.getByText('Processing file 0 of 0')).toHaveCount(0);
await expect(progressSection(page).locator('progress')).toHaveAttribute('value', '100');
});
-556
View File
@@ -1,556 +0,0 @@
// Needs review (#249 PR F, plan Tasks F1 and F2, Mock inventory M4): the SAVED VIEWS row and its count, the Metadata
// review page over the real review routes, and Activity's Review metadata. Items are recorded by the real recorder
// (tests/browser/workspace_review_fixture.py) through the monitor's entry point; the conflict item's file was linked
// to another prompt after it was queued. An open page re-reads its queue from the review change stream (F2).
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from './fixtures.mjs';
const review = '/prompt_manager/workspace#/review';
const main = page => page.locator('#workspace-main');
const savedNav = page => page.getByRole('navigation', { name: 'Saved views', exact: true });
const reviewRow = page => savedNav(page).locator('a[data-route="review"]');
const panel = page => main(page).getByRole('region', { name: 'Resolve the association', exact: true });
const context = page => panel(page).getByRole('combobox', { name: 'Sampler context', exact: true });
const cards = page => main(page).locator('.review-queue article.media-card');
const toast = page => page.locator('#workspace-toast');
const NOTICE = 'Multiple sampler paths were found. The existing image links were preserved. Choose the correct context or leave them unresolved.';
const EMPTY = 'Ambiguous images are added when a metadata scan or a new generation finds more than one sampler path.';
const OPTIONS = ['Sampler 11 · Golden dunes at dusk / Soft negative, blur', 'Sampler 13 · Harbor lights at night / Soft negative, blur',
'Leave unresolved'];
async function seed(request, items = 3, special = {}) {
const response = await request.post('/__test__/review/reset', { data: { items, ...special } });
await expect(response).toBeOK();
return response.json();
}
async function pending(request) {
return (await (await request.get('/prompt_manager/review/items?limit=1')).json()).total;
}
const links = async request => (await (await request.get('/__test__/review/links')).json()).links;
const streams = async request => (await (await request.get('/__test__/review/streams')).json());
const apply = page => panel(page).getByRole('button', { name: 'Apply association', exact: true });
const warning = page => panel(page).locator('.warning-note');
/** The page with its first item's sampler paths loaded. */
async function open(page) {
await page.goto(review);
await expect(context(page)).toBeEnabled();
}
test.beforeEach(async ({ page, browserName }) => {
await startCoverage(page, browserName);
page.reviewErrors = [];
page.on('pageerror', error => page.reviewErrors.push(error.message));
});
test.afterEach(async ({ page, browserName, request }) => {
await saveCoverage(page, browserName);
expect(page.reviewErrors).toEqual([]);
await request.post('/__test__/jobs/restart');
});
test('queue shows persisted items and the nav count', async ({ page, request }) => {
await seed(request, 3);
await open(page);
// SAVED VIEWS: Favorites, then Needs review with the exact pending count (ruling R5).
await expect(savedNav(page).locator('a').nth(1)).toHaveAttribute('data-route', 'review');
await expect(reviewRow(page)).toHaveText('Needs review3');
await expect(reviewRow(page)).toHaveAttribute('aria-current', 'page');
await expect(page).toHaveTitle('Review · Prompt Manager');
await expect(page.locator('#workspace-breadcrumb')).toHaveText('Review');
await expect(main(page).getByRole('heading', { level: 1 })).toHaveText('Metadata review');
await expect(main(page).locator('.heading p.muted')).toHaveText('Uncertain evidence stays visible until you decide.');
await expect(main(page).locator('.notice h2')).toHaveText('3 images need a decision');
await expect(main(page).locator('.notice p')).toHaveText(NOTICE);
await expect(cards(page)).toHaveCount(3);
await expect(cards(page).first()).toHaveClass(/selected/);
await expect(panel(page).locator('p.small')).toHaveText('review_0000.png');
await expect(context(page).locator('option')).toHaveText(OPTIONS);
await expect(context(page)).toHaveValue(/^[0-9a-f]{64}$/);
await expect(panel(page).locator('.prompt-section.positive .prompt-text')).toHaveText('Golden dunes at dusk');
await expect(panel(page).locator('.prompt-section.negative .prompt-text')).toHaveText('Soft negative, blur');
await context(page).selectOption({ label: OPTIONS[1] });
await expect(panel(page).locator('.prompt-section.positive .prompt-text')).toHaveText('Harbor lights at night');
await context(page).selectOption({ label: 'Leave unresolved' });
await expect(panel(page).locator('.prompt-section.positive .prompt-text')).toHaveText('Multiple possible prompts. Source could not be determined.');
// Info selects another item.
await cards(page).nth(1).getByRole('button', { name: 'Inspect review_0001.png', exact: true }).click();
await expect(panel(page).locator('p.small')).toHaveText('review_0001.png');
await expect(cards(page).nth(1)).toHaveClass(/selected/);
await expect(context(page)).toBeEnabled();
// The card's image opens the shared viewer.
await cards(page).nth(1).getByRole('button', { name: 'Open review_0001.png', exact: true }).click();
await expect(page.locator('#viewer-dialog')).toBeVisible();
await page.keyboard.press('Escape');
await expect(page.locator('#viewer-dialog')).toHaveCount(0);
// Open activity.
await main(page).getByRole('button', { name: 'Open activity', exact: true }).click();
await expect(page).toHaveURL(/#\/activity$/);
});
test('apply candidate links the pair and the item leaves the queue and the count drops', async ({ page, request }) => {
await seed(request, 2);
await open(page);
await expect(reviewRow(page)).toHaveText('Needs review2');
const decided = page.waitForResponse(item => /\/review\/items\/[^/]+\/decision$/.test(item.url()));
await panel(page).getByRole('button', { name: 'Apply association', exact: true }).click();
expect((await decided).status()).toBe(200);
await expect(toast(page)).toHaveText('Association accepted.');
await expect(page).toHaveURL(/#\/images$/);
await expect(reviewRow(page)).toHaveText('Needs review1');
expect(await pending(request)).toBe(1);
// The first sampler path's pair is linked to that file, and only to it.
expect(await links(request)).toEqual([{ filename: 'review_0000.png', prompt: 'Golden dunes at dusk',
pair_id: expect.any(Number), positive: 'Golden dunes at dusk', negative: 'Soft negative, blur' }]);
});
test('conflict leaves links unchanged and explains', async ({ page, request }) => {
// Queued first, then linked to another prompt (before a scan or monitor pass could settle it).
await seed(request, 0, { conflict: true });
const before = await links(request);
expect(before).toEqual([{ filename: 'conflict.png', prompt: 'Misty forest path', pair_id: null, positive: null,
negative: null }]);
await open(page);
await expect(panel(page).locator('p.small')).toHaveText('conflict.png');
await expect(context(page).locator('option')).toHaveText(OPTIONS);
const decided = page.waitForResponse(item => /\/decision$/.test(item.url()));
await apply(page).click();
const answer = await decided;
expect(answer.status()).toBe(409);
expect((await answer.json()).code).toBe('association-conflict');
await expect(warning(page)).toHaveText('This image is already linked to another prompt or negative, so this sampler path cannot be attached safely. The existing links were preserved.');
await expect(page).toHaveURL(/#\/review$/);
await expect(context(page)).toBeEnabled();
await expect(context(page)).toHaveValue(/^[0-9a-f]{64}$/);
expect(await links(request)).toEqual(before);
expect(await pending(request)).toBe(1);
await expect(reviewRow(page)).toHaveText('Needs review1');
// The file can still be left unresolved, which links nothing either.
await context(page).selectOption({ label: 'Leave unresolved' });
await apply(page).click();
await expect(toast(page)).toHaveText('Left unresolved.');
expect(await links(request)).toEqual(before);
expect(await pending(request)).toBe(0);
});
test('a limited item and an unavailable file can only be left unresolved', async ({ page, request }) => {
await seed(request, 0, { limited: true, unavailable: true });
await open(page);
await expect(cards(page)).toHaveCount(2);
await expect(panel(page).locator('p.small')).toHaveText('limited.png');
await expect(context(page).locator('option')).toHaveText(['Leave unresolved']);
await expect(warning(page)).toHaveText('This file has too many sampler paths to list. You can leave it unresolved.');
// The unavailable file: no current root lists it, so its media and Tag are inert.
const unavailable = cards(page).nth(1);
await expect(unavailable.locator('.image-open')).toHaveAttribute('aria-disabled', 'true');
await expect(unavailable.locator('.image-card-actions button').last()).toHaveAttribute('aria-disabled', 'true');
// F3: its empty media button keeps the concept's 1.33 media area, so the card lines up with its neighbours.
await expect(cards(page).first().locator('.image-open img')).toBeVisible();
const sizes = await cards(page).evaluateAll(nodes => nodes.map(node => {
const art = node.querySelector('.image-open').getBoundingClientRect();
return [Math.round(art.width), Math.round(art.height), Math.round(node.getBoundingClientRect().height)];
}));
expect(sizes[1]).toEqual(sizes[0]);
await unavailable.getByRole('button', { name: 'Inspect unavailable.png', exact: true }).click();
await expect(panel(page).locator('p.small')).toHaveText('unavailable.png');
await expect(warning(page)).toHaveText('This file is not under a current image source, so it cannot be opened. You can leave it unresolved.');
await expect(panel(page).getByRole('button', { name: 'Tag image', exact: true })).toHaveAttribute('aria-disabled', 'true');
const observes = [];
page.on('request', item => { if (item.url().endsWith('/image-annotations/observe')) observes.push(item.url()); });
await apply(page).click();
await expect(toast(page)).toHaveText('Left unresolved.');
expect(observes).toEqual([]);
expect(await pending(request)).toBe(1);
});
test('leave unresolved removes the item from pending', async ({ page, request }) => {
await seed(request, 2);
await open(page);
await context(page).selectOption({ label: 'Leave unresolved' });
await panel(page).getByRole('button', { name: 'Apply association', exact: true }).click();
await expect(toast(page)).toHaveText('Left unresolved.');
await expect(page).toHaveURL(/#\/images$/);
expect(await pending(request)).toBe(1);
});
test('leave for later writes nothing and goes to Images', async ({ page, request }) => {
await seed(request, 2);
await open(page);
const writes = [];
page.on('request', item => { if (item.method() !== 'GET') writes.push(item.url()); });
await panel(page).getByRole('button', { name: 'Leave for later', exact: true }).click();
await expect(page).toHaveURL(/#\/images$/);
expect(writes).toEqual([]);
expect(await pending(request)).toBe(2);
});
test('tag image from review saves tags and the item remains pending', async ({ page, request }) => {
await seed(request, 1);
await open(page);
const before = await links(request);
await panel(page).getByRole('button', { name: 'Tag image', exact: true }).click();
const dialog = page.getByRole('dialog', { name: 'Tag image', exact: true });
await expect(dialog).toBeVisible();
await expect(dialog.locator('p.tag-scope-note')).toHaveText('Tags belong to this image. Resolve its prompt association separately in Needs review.');
await dialog.getByRole('textbox', { name: 'Image tags · edit manually', exact: true }).fill('dunes');
await dialog.getByRole('button', { name: 'Save manual tags', exact: true }).click();
await expect(toast(page)).toHaveText('1 tags saved for review_0000.png. Prompt association still awaits review.');
await expect(dialog).toHaveCount(0);
expect(await pending(request)).toBe(1);
await expect(page).toHaveURL(/#\/review$/);
// The item is untouched: still listed, still open, nothing linked.
await expect(cards(page)).toHaveCount(1);
await expect(context(page)).toBeEnabled();
expect(await links(request)).toEqual(before);
const [item] = (await (await request.get('/prompt_manager/review/items')).json()).items;
expect([item.state, item.revision]).toEqual(['pending', 0]);
// The card's Tag opens the same dialog.
await cards(page).first().getByRole('button', { name: 'Tag or auto-tag review_0000.png', exact: true }).click();
await expect(dialog).toBeVisible();
await expect(dialog.locator('p.tag-scope-note')).toBeVisible();
await page.keyboard.press('Escape');
await expect(dialog).toHaveCount(0);
});
test('an empty queue shows the empty notice only', async ({ page, request }) => {
await seed(request, 0);
await page.goto(review);
await expect(main(page).locator('.notice h2')).toHaveText('No images need a decision');
await expect(main(page).locator('.notice p')).toHaveText(EMPTY);
await expect(main(page).locator('.gallery-layout')).toHaveCount(0);
await expect(reviewRow(page)).toHaveText('Needs review0');
});
test('load more appends the next page', async ({ page, request }) => {
await seed(request, 45);
await open(page);
await expect(cards(page)).toHaveCount(40);
const continuation = main(page).locator('.infinite-continuation');
await expect(continuation.locator('p')).toHaveText('40 of 45 images');
await continuation.getByRole('button', { name: 'Load more images', exact: true }).click();
await expect(cards(page)).toHaveCount(45);
await expect(continuation).toHaveCount(0);
});
test('a scan job with review items offers Review metadata in Activity', async ({ page, request }) => {
await seed(request, 1);
const started = await (await request.post('/__test__/jobs/start', { data: { total: 10, review_items: 2 } })).json();
await expect(await request.post(`/__test__/jobs/${started.job.id}/finish`, { data: {} })).toBeOK();
const plain = await (await request.post('/__test__/jobs/start', { data: { total: 10 } })).json();
await expect(await request.post(`/__test__/jobs/${plain.job.id}/finish`, { data: {} })).toBeOK();
await page.goto('/prompt_manager/workspace#/activity');
const item = main(page).locator(`.activity-item[data-job-id="${started.job.id}"]`);
await expect(item.getByRole('button', { name: 'Review metadata', exact: true })).toBeVisible();
await expect(main(page).locator(`.activity-item[data-job-id="${plain.job.id}"] button`)).toHaveCount(0);
await page.goto(`/prompt_manager/workspace#/activity/${started.job.id}`);
const actions = main(page).locator('.heading-actions button');
await expect(actions).toHaveText(['Download report', 'Review metadata']);
await actions.nth(1).click();
await expect(page).toHaveURL(/#\/review$/);
await expect(main(page).getByRole('heading', { level: 1 })).toHaveText('Metadata review');
});
test('a failed item read says so and Retry reads it again', async ({ page, request }) => {
await seed(request, 1);
await page.route(url => /\/prompt_manager\/review\/items\/[0-9a-f-]{36}$/.test(url.pathname),
route => route.fulfill({ status: 500, contentType: 'application/json', body: JSON.stringify({ success: false, code: 'review-failed', error: 'x' }) }),
{ times: 1 });
await page.goto(review);
await expect(panel(page).locator('.warning-note')).toHaveText('This item could not be read. Select it again to retry.');
await expect(panel(page).getByText('Could not read prompts from this file.', { exact: true })).toBeVisible();
await expect(panel(page).getByRole('button', { name: 'Apply association', exact: true })).toHaveAttribute('aria-disabled', 'true');
await panel(page).getByRole('button', { name: 'Retry prompt details', exact: true }).click();
await expect(context(page)).toBeEnabled();
await expect(context(page).locator('option')).toHaveText(OPTIONS);
await expect(panel(page).locator('.warning-note')).toBeHidden();
});
test('a failed queue read offers Retry', async ({ page, request }) => {
await seed(request, 1);
await page.route(url => url.pathname.endsWith('/prompt_manager/review/items') && url.searchParams.get('limit') === '40',
route => route.fulfill({ status: 500, contentType: 'application/json', body: JSON.stringify({ success: false, code: 'review-failed', error: 'x' }) }),
{ times: 1 });
await page.goto(review);
await expect(main(page).getByRole('alert')).toHaveText('The review queue could not be read.');
await main(page).getByRole('button', { name: 'Retry', exact: true }).click();
await expect(context(page)).toBeEnabled();
await expect(main(page).locator('.notice h2')).toHaveText('1 image needs a decision');
});
test('keyboard focus labels and touch targets', async ({ page, request }) => {
await seed(request, 2);
await open(page);
const first = cards(page).first();
for (const [name, role] of [['Open review_0000.png', 'button'], ['Inspect review_0000.png', 'button'],
['Tag or auto-tag review_0000.png', 'button']]) await expect(first.getByRole(role, { name, exact: true })).toBeVisible();
await expect(main(page).getByRole('list', { name: 'Review queue', exact: true })).toBeVisible();
await expect(panel(page)).toBeVisible();
// Keyboard: Info on the second card opens it, and the panel's controls take focus in their order.
const info = cards(page).nth(1).getByRole('button', { name: 'Inspect review_0001.png', exact: true });
await info.focus();
await page.keyboard.press('Enter');
await expect(panel(page).locator('p.small')).toHaveText('review_0001.png');
await expect(context(page)).toBeEnabled();
await context(page).focus();
await expect(context(page)).toBeFocused();
const order = await panel(page).evaluate(node => [...node.querySelectorAll('button, select')]
.filter(control => control.offsetParent !== null).map(control => control.getAttribute('aria-label') || control.textContent.trim()));
expect(order[0]).toBe('Tag image');
expect(order.slice(-2)).toEqual(['Leave for later', 'Apply association']);
// Targets: the panel's controls are 40px on a coarse pointer (the concept's coarse buttons) and 24px otherwise (the
// WCAG 2.2 minimum); the card's buttons keep the concept's compact size, never under 24px (its ≤700px rule).
const coarse = await page.evaluate(() => matchMedia('(pointer: coarse)').matches);
const sized = [
[panel(page).getByRole('button', { name: 'Tag image', exact: true }), coarse ? 40 : 24],
[context(page), coarse ? 40 : 24],
[panel(page).getByRole('button', { name: 'Leave for later', exact: true }), coarse ? 40 : 24],
[apply(page), coarse ? 40 : 24],
[first.getByRole('button', { name: 'Inspect review_0000.png', exact: true }), 24],
[first.getByRole('button', { name: 'Tag or auto-tag review_0000.png', exact: true }), 24],
];
for (const [control, minimum] of sized) {
const box = await control.boundingBox();
const label = await control.evaluate(node => node.getAttribute('aria-label') || node.textContent.trim().slice(0, 40));
expect(box.height, label).toBeGreaterThanOrEqual(minimum);
expect(box.width, label).toBeGreaterThanOrEqual(24);
}
// Enter on Leave for later returns to Images.
await panel(page).getByRole('button', { name: 'Leave for later', exact: true }).focus();
await page.keyboard.press('Enter');
await expect(page).toHaveURL(/#\/images$/);
});
test('an open page shows items the monitor adds and keeps the open choice', async ({ page, request }) => {
await seed(request, 1);
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
await context(page).selectOption({ label: OPTIONS[1] });
await expect(reviewRow(page)).toHaveText('Needs review1');
await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
await expect(cards(page)).toHaveCount(2);
await expect(main(page).locator('.notice h2')).toHaveText('2 images need a decision');
await expect(reviewRow(page)).toHaveText('Needs review2');
await expect(cards(page).first()).toHaveClass(/selected/);
await expect(panel(page).locator('p.small')).toHaveText('review_0000.png');
await expect(context(page).locator('option:checked')).toHaveText(OPTIONS[1]);
await expect(warning(page)).toBeHidden();
});
test('an open page shows items a metadata scan adds', async ({ page, request }) => {
await seed(request, 1);
await expect(await request.post('/__test__/review/add', { data: { kind: 'file' } })).toBeOK();
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
await expect(cards(page)).toHaveCount(1);
const started = await request.post('/prompt_manager/jobs/scan', { data: { scope: 'new' } });
expect(started.status()).toBe(202);
await expect(cards(page)).toHaveCount(2);
await expect(cards(page).nth(1).getByRole('button', { name: 'Inspect added_0000.png', exact: true })).toBeVisible();
await expect(reviewRow(page)).toHaveText('Needs review2');
});
test('a decision in another tab leaves the open page and opens the next image', async ({ page, request, context: browserContext }) => {
await seed(request, 2);
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
const other = await browserContext.newPage();
await open(other);
await context(other).selectOption({ label: 'Leave unresolved' });
await apply(other).click();
await expect(other).toHaveURL(/#\/images$/);
await other.close();
await expect(cards(page)).toHaveCount(1);
await expect(panel(page).locator('p.small')).toHaveText('review_0001.png');
await expect(warning(page)).toHaveText('review_0000.png was decided elsewhere, so the next image is open.');
await expect(main(page).locator('.notice h2')).toHaveText('1 image needs a decision');
await expect(reviewRow(page)).toHaveText('Needs review1');
await expect(context(page)).toBeEnabled();
});
/** The queue reads (`?limit=40`, cursors) and count reads (`?limit=1`) the page sends from now on. */
function reviewReads(page) {
const reads = [];
page.on('request', item => {
const url = new URL(item.url());
if (url.pathname === '/prompt_manager/review/items') reads.push(url.searchParams.get('cursor') ? 'more' : url.search);
});
return reads;
}
test('a visit reads its queue once, and a change reads it once more', async ({ page, request }) => {
// F3 ruling: the stream's ready must not read a queue the first read already holds.
await seed(request, 1);
const reads = reviewReads(page);
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
await expect(cards(page)).toHaveCount(2);
await expect(reviewRow(page)).toHaveText('Needs review2');
// The page's first read and the change's re-read; the shell's count read at start, and the row follows the page.
expect(reads.filter(search => search === '?limit=40')).toHaveLength(2);
expect(reads.filter(search => search === '?limit=1')).toHaveLength(1);
});
test('a ready whose version cannot be read re-reads the queue once', async ({ page, request }) => {
// The stream's ready names no readable version (an older server, a proxy that rewrote it): the page reads again.
await seed(request, 1);
const reads = reviewReads(page);
// M1: `retry` keeps the browser from reconnecting the ended stream during the test, so exactly one re-read follows.
await page.route(url => url.pathname === '/prompt_manager/review/events', route => route.fulfill({
status: 200, headers: { 'Content-Type': 'text/event-stream', 'Cache-Control': 'no-store' },
body: 'retry: 3600000\nevent: ready\ndata: not json\n\n' }));
let reread = 0;
const second = page.waitForResponse(item => item.url().endsWith('/prompt_manager/review/items?limit=40') && ++reread === 2);
await open(page);
await second;
await expect(cards(page)).toHaveCount(1);
expect(reads.filter(search => search === '?limit=40')).toHaveLength(2);
});
const live = page => main(page).locator('.heading .review-live');
const NOT_LIVE = 'Live updates are paused: changes made elsewhere appear once the connection returns.';
test('a stream the browser gives up on is opened again, and the page says it is not live meanwhile', async ({ page, request }) => {
// I3: a 503 (shutdown, or a proxy while the backend restarts) fails an EventSource for good.
await seed(request, 1);
let attempts = 0;
await page.route(url => url.pathname === '/prompt_manager/review/events', route => {
attempts += 1;
if (attempts === 1) return route.fulfill({ status: 503, contentType: 'application/json', body: '{"success":false}' });
return route.fallback();
});
await open(page);
await expect(live(page)).toHaveText(NOT_LIVE);
await expect(live(page)).toBeVisible();
// The page opens the stream again (after 3 s) and is live once it sends ready.
await expect.poll(async () => (await streams(request)).streams, { timeout: 15000 }).toBe(1);
await expect(live(page)).toBeHidden();
await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
await expect(cards(page)).toHaveCount(2);
expect(attempts).toBe(2);
});
/** Make the page hidden or visible as a tab switch does. */
async function setVisible(page, visible) {
await page.evaluate(shown => {
Object.defineProperty(document, 'hidden', { configurable: true, get: () => !shown });
Object.defineProperty(document, 'visibilityState', { configurable: true, get: () => (shown ? 'visible' : 'hidden') });
document.dispatchEvent(new Event('visibilitychange'));
}, visible);
}
test('a hidden tab closes its stream and a visible one reads only what changed meanwhile', async ({ page, request }) => {
// I4: background tabs must not hold one of the origin's six connections.
await seed(request, 1);
const reads = reviewReads(page);
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
await setVisible(page, false);
// The server drops a closed stream at its next write: this monitor addition is one.
await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
await expect.poll(async () => (await streams(request)).streams).toBe(0);
await expect(cards(page)).toHaveCount(1);
expect(reads.filter(search => search === '?limit=40')).toHaveLength(1);
// Visible again: the reopened stream's ready names a newer version, so the page reads once.
const reread = page.waitForResponse(item => item.url().endsWith('/prompt_manager/review/items?limit=40'));
await setVisible(page, true);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
await reread;
await expect(cards(page)).toHaveCount(2);
await expect(live(page)).toBeHidden();
expect(reads.filter(search => search === '?limit=40')).toHaveLength(2);
});
test('a re-read refreshes a card whose media or target changed', async ({ page, request }) => {
// M6 and M10: a root came or went between reads; the card follows, and a known thumbnail is its media.
await seed(request, 2);
let rewrite = false;
await page.route(url => url.pathname === '/prompt_manager/review/items' && url.search === '?limit=40', async route => {
if (!rewrite) return route.fallback();
const response = await route.fetch();
const body = await response.json();
body.items[0] = { ...body.items[0], thumbnail_url: `${body.items[0].url}&thumb=1` };
body.items[1] = { ...body.items[1], url: null, thumbnail_url: null, target: null, unavailable: true };
return route.fulfill({ response, json: body });
});
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
const second = cards(page).nth(1);
await expect(second.getByRole('button', { name: 'Open review_0001.png', exact: true })).not.toHaveAttribute('aria-disabled', 'true');
rewrite = true;
await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
await expect(cards(page)).toHaveCount(3);
await expect(cards(page).first().locator('img')).toHaveAttribute('src', /&thumb=1$/);
await expect(second.getByRole('button', { name: 'Open review_0001.png', exact: true })).toHaveAttribute('aria-disabled', 'true');
await expect(second.locator('img')).toHaveCount(0);
await expect(second.locator('.image-card-actions button').last()).toHaveAttribute('aria-disabled', 'true');
await expect(cards(page).first()).toHaveClass(/selected/);
});
test('leaving during the observe releases the capture it answers', async ({ page, request }) => {
// M4: the server made the capture; the page left before the answer came.
await seed(request, 1);
await open(page);
let held;
const sent = new Promise(resolve => { held = resolve; });
await page.route(url => url.pathname === '/prompt_manager/image-annotations/observe', async route => {
const response = await route.fetch();
held(() => route.fulfill({ response }));
});
await apply(page).click();
const answer = await sent;
expect((await streams(request)).captures).toBe(1);
await page.evaluate(() => { window.location.hash = '#/tags'; });
await expect(main(page).getByRole('heading', { level: 1 })).toHaveText('Tags & categories');
await answer();
await expect.poll(async () => (await streams(request)).captures).toBe(0);
});
test('a live re-read keeps the loaded depth and the scroll position', async ({ page, request }) => {
// F3 ruling: a queue change re-reads as many items as Load more had shown, and the page does not jump.
await seed(request, 45);
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
await main(page).getByRole('button', { name: 'Load more images', exact: true }).click();
await expect(cards(page)).toHaveCount(45);
await cards(page).last().scrollIntoViewIfNeeded();
const scrollTop = () => cards(page).first().evaluate(node => {
for (let el = node.parentElement; el; el = el.parentElement) {
const style = getComputedStyle(el);
if (/(auto|scroll)/.test(style.overflowY) && el.scrollHeight > el.clientHeight) return el.scrollTop;
}
return document.scrollingElement.scrollTop;
});
const before = await scrollTop();
expect(before).toBeGreaterThan(0);
const reads = reviewReads(page);
await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
await expect(main(page).locator('.notice h2')).toHaveText('46 images need a decision');
await expect(cards(page)).toHaveCount(46);
expect(await scrollTop()).toBe(before);
expect(reads).toEqual(['?limit=40', 'more']);
await expect(main(page).locator('.infinite-continuation')).toHaveCount(0);
});
test('teardown releases captures listeners and requests', async ({ page, request }) => {
await seed(request, 1);
await open(page);
await expect.poll(async () => (await streams(request)).streams).toBe(1);
const inFlight = new Set();
page.on('request', item => { if (item.url().includes('/prompt_manager/review/')) inFlight.add(item); });
page.on('requestfinished', item => inFlight.delete(item));
page.on('requestfailed', item => inFlight.delete(item));
// A decision whose answer never comes: leaving the page must still release its capture.
let held;
const sent = new Promise(resolve => { held = resolve; });
await page.route(url => /\/decision$/.test(url.pathname), route => held(route));
await apply(page).click();
const route = await sent;
expect((await streams(request)).captures).toBe(1);
await page.evaluate(() => { window.location.hash = '#/tags'; });
await expect(main(page).getByRole('heading', { level: 1 })).toHaveText('Tags & categories');
await expect.poll(async () => (await streams(request)).captures).toBe(0);
await route.abort().catch(() => {});
// The server drops a closed stream at its next write: a monitor addition is one.
await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
await expect.poll(async () => (await streams(request)).streams).toBe(0);
await expect.poll(() => inFlight.size).toBe(0);
});
+1 -1
View File
@@ -372,7 +372,7 @@ test('Favorites is the first SAVED VIEWS row, with a real count that follows eve
expect(await favoritesRow(page).evaluate(link => [...link.childNodes].map(node => node.nodeType === Node.TEXT_NODE
? `text:${node.textContent}` : `${node.localName}.${node.getAttribute('class')}`)))
.toEqual(['svg.icon', 'text:Favorites', 'span.nav-count']);
expect(await savedNav(page).locator('a').evaluateAll(links => links.map(link => link.dataset.route))).toEqual(['favorites', 'review']);
expect(await savedNav(page).locator('a').evaluateAll(links => links.map(link => link.dataset.route))).toEqual(['favorites']);
await star(page, '1:shared/same.png');
await expect(toast(page)).toHaveText('Added to favorites.');
+6 -141
View File
@@ -304,151 +304,16 @@ test('Refresh files calls /gallery/rescan and toasts its counts', async ({ page
await expect(toast(page)).toHaveText('Files could not be refreshed. Try again.');
});
const THUMBNAIL_PLAN_URL = '/prompt_manager/images/thumbnails/plan';
const THUMBNAIL_JOB_URL = '/prompt_manager/jobs/thumbnails';
const planCount = value => value.toLocaleString('en-US');
/** Open "Thumbnail maintenance" from Sources' Thumbnail generation → Review options. */
async function openThumbnailReview(page) {
test('Thumbnail generation hands off to the existing gallery until its review dialog', async ({ page }) => {
const layout = await openSources(page);
const thumbnails = layout.locator('.operation').filter({ has: page.getByRole('heading', { name: 'Thumbnail generation', exact: true }) });
await expect(thumbnails.locator('p')).toHaveText('Generate missing previews; rebuilding all is a separate explicit option.');
const popup = page.waitForEvent('popup');
await thumbnails.getByRole('button', { name: 'Review options', exact: true }).click();
await expect(dialog(page).locator('#modal-title')).toHaveText('Thumbnail maintenance');
return dialog(page);
}
/** The thumbnail requests this page sends: plan reads (their query) and job starts (their body). */
function thumbnailRequests(page) {
const seen = { plans: [], starts: [] };
page.on('request', item => {
const url = new URL(item.url());
if (url.pathname === THUMBNAIL_PLAN_URL) seen.plans.push(Object.fromEntries(url.searchParams));
if (item.method() === 'POST' && url.pathname === THUMBNAIL_JOB_URL) seen.starts.push(item.postDataJSON());
});
return seen;
}
const missingLine = plan => `${planCount(plan.targets)} ${plan.targets === 1 ? 'preview' : 'previews'} to generate · ` +
`${planCount(plan.fresh)} already up to date.`;
test('Review options shows the plan counts for the chosen scope/quality/size before Start', async ({ page, request }) => {
await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 3 } })).toBeOK();
// Image loading quality (Appearance, Q3) preselects the dialog's quality.
await page.addInitScript(() => { if (location.protocol.startsWith('http')) localStorage.setItem('gallerySettings', JSON.stringify({ imageQuality: 'high' })); });
const seen = thumbnailRequests(page);
const plan = await serverJson(request, `${THUMBNAIL_PLAN_URL}?mode=missing&size=512&quality=high`);
expect(plan.targets).toBe(3);
const review = await openThumbnailReview(page);
await expect(review.locator('#modal-body > *')).toHaveCount(5);
await expect(review.locator('.field > label')).toHaveText(['Scope', 'Quality', 'Preview size limit']);
await expect(review.locator('#thumb-mode option')).toHaveText(['Missing thumbnails only', 'Rebuild all (explicit opt-in)']);
await expect(review.locator('#thumb-quality option')).toHaveText(['Medium · optimized', 'High · original', 'Low · fastest']);
await expect(review.locator('#thumb-mode')).toHaveValue('missing');
await expect(review.locator('#thumb-quality')).toHaveValue('high');
await expect(review.locator('#thumb-limit')).toHaveValue('512');
await expect(review.locator('#thumb-limit')).toHaveAttribute('min', '128');
await expect(review.locator('#thumb-limit')).toHaveAttribute('max', '2048');
const line = review.locator('#modal-body > p.small');
await expect(line).toHaveAttribute('role', 'status');
await expect(line).toHaveText(missingLine(plan));
expect(seen.plans).toEqual([{ mode: 'missing', size: '512', quality: 'high' }]);
await expect(review.locator('.dialog-actions > button')).toHaveText(['Cancel', 'Start']);
// Reviewing starts nothing; Cancel closes.
await review.getByRole('button', { name: 'Cancel', exact: true }).click();
await expect(dialog(page)).toHaveCount(0);
expect(seen.starts).toEqual([]);
expect((await serverJson(request, '/prompt_manager/jobs')).jobs).toEqual([]);
});
test('size outside 128–2048 reports validity and requests nothing', async ({ page, request }) => {
const seen = thumbnailRequests(page);
const review = await openThumbnailReview(page);
const line = review.locator('#modal-body > p.small');
await expect(line).toHaveText(/ already up to date\.$/);
const size = review.locator('#thumb-limit');
for (const value of ['4096', '64', '']) {
await size.fill(value);
await expect(size).toHaveAttribute('aria-invalid', 'true');
await expect(line).toHaveText('Enter a size from 128 to 2048 pixels.');
expect(await size.evaluate(input => input.validity.valid)).toBe(false);
// Start keeps the dialog open and sends nothing; neither does a scope change.
await review.getByRole('button', { name: 'Start', exact: true }).click();
await expect(dialog(page).locator('#modal-title')).toHaveText('Thumbnail maintenance');
await review.locator('#thumb-mode').selectOption('rebuild');
await expect(line).toHaveText('Enter a size from 128 to 2048 pixels.');
await review.locator('#thumb-mode').selectOption('missing');
}
expect(seen.starts).toEqual([]);
expect(seen.plans).toEqual([{ mode: 'missing', size: '512', quality: 'medium' }]);
await size.fill('1024');
await expect(size).not.toHaveAttribute('aria-invalid');
await expect(line).toHaveText(/ already up to date\.$/);
expect((await serverJson(request, '/prompt_manager/jobs')).jobs).toEqual([]);
});
test('Start closes the dialog, toasts, and opens the job in Activity', async ({ page, request }) => {
await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 2 } })).toBeOK();
const seen = thumbnailRequests(page);
const review = await openThumbnailReview(page);
await review.locator('#thumb-quality').selectOption('low');
await review.locator('#thumb-limit').fill('256');
await review.getByRole('button', { name: 'Start', exact: true }).click();
await expect(dialog(page)).toHaveCount(0);
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
// The toast says it started (or, for a run this quick, that it already finished); Activity shows its progress.
await expect(toast(page)).toHaveText(/^(Thumbnail generation started\.|Thumbnails finished · 2 generated · 0 errors)$/);
expect(seen.starts).toEqual([{ mode: 'missing', size: 256, quality: 'low' }]);
const id = page.url().split('/').pop();
const { job } = await serverJson(request, `/prompt_manager/jobs/${id}`);
expect(job).toMatchObject({ kind: 'thumbnails', title: 'Thumbnail generation', scope: { mode: 'missing', size: 256, quality: 'low' } });
await expect(page.locator('#workspace-main h1')).toHaveText('Thumbnail generation');
await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).toBe('finished');
});
test('Rebuild all is explicit and the plan line says every preview is regenerated', async ({ page, request }) => {
const seen = thumbnailRequests(page);
const rebuild = await serverJson(request, `${THUMBNAIL_PLAN_URL}?mode=rebuild&size=512&quality=medium`);
expect(rebuild.targets).toBeGreaterThan(0);
const review = await openThumbnailReview(page);
// Missing thumbnails only is the default; rebuilding is the explicit second option.
await expect(review.locator('#thumb-mode')).toHaveValue('missing');
const line = review.locator('#modal-body > p.small');
await expect(line).toHaveText(/ already up to date\.$/);
await review.locator('#thumb-mode').selectOption('rebuild');
await expect(line).toHaveText(`Every preview is regenerated: ${planCount(rebuild.targets)} files.`);
expect(seen.plans).toEqual([{ mode: 'missing', size: '512', quality: 'medium' }, { mode: 'rebuild', size: '512', quality: 'medium' }]);
// Back and forth reads each scope once.
await review.locator('#thumb-mode').selectOption('missing');
await expect(line).toHaveText(/ already up to date\.$/);
await review.locator('#thumb-mode').selectOption('rebuild');
await expect(line).toHaveText(`Every preview is regenerated: ${planCount(rebuild.targets)} files.`);
expect(seen.plans).toHaveLength(2);
await review.getByRole('button', { name: 'Start', exact: true }).click();
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
expect(seen.starts).toEqual([{ mode: 'rebuild', size: 512, quality: 'medium' }]);
const id = page.url().split('/').pop();
await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).toBe('finished');
});
test('a plan that cannot be read says so; Start still starts', async ({ page }) => {
await page.route(`**${THUMBNAIL_PLAN_URL}?*`, route => route.fulfill({ status: 500, json: { success: false, error: 'disk' } }));
const review = await openThumbnailReview(page);
await expect(review.locator('#modal-body > p.small')).toHaveText('The previews could not be counted.');
await expect(review.getByRole('button', { name: 'Start', exact: true })).not.toHaveAttribute('aria-disabled');
await review.getByRole('button', { name: 'Cancel', exact: true }).click();
});
test('a refused start shows the busy link toast', async ({ page, request }) => {
const running = await (await request.post('/__test__/jobs/start', { data: { title: 'Scan for metadata' } })).json();
const review = await openThumbnailReview(page);
await review.getByRole('button', { name: 'Start', exact: true }).click();
await expect(dialog(page)).toHaveCount(0);
await expect(toast(page)).toHaveText('Another job is running: Scan for metadata.');
await expect(toast(page).getByRole('link', { name: 'Scan for metadata', exact: true })).toHaveAttribute('href', `#/activity/${running.job.id}`);
await expect(page).toHaveURL(`${workspace}#/system/sources`);
expect((await serverJson(request, '/prompt_manager/jobs')).jobs.map(job => job.kind)).toEqual(['scan']);
await request.post(`/__test__/jobs/${running.job.id}/finish`, { data: { state: 'finished' } });
const gallery = await popup;
expect(new URL(gallery.url()).pathname).toBe('/prompt_manager/gallery');
await gallery.close();
await expect(statusOf(page)).toHaveText('No unsaved changes.');
});
test('a save refused by a running scan shows the busy link toast', async ({ page, request }) => {
+6 -31
View File
@@ -140,32 +140,9 @@ test('thumbnail row reflects the status endpoint', async ({ page, request }) =>
const layout = await openOverview(page);
const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) });
await expect(row.locator('h3')).toHaveText('3 missing thumbnails');
});
test('Overview Review thumbnails opens the same dialog', async ({ page, request }) => {
await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 2 } })).toBeOK();
const plan = await serverJson(request, '/prompt_manager/images/thumbnails/plan?mode=missing&size=512&quality=medium');
const starts = [];
page.on('request', item => { if (item.method() === 'POST' && item.url().endsWith('/prompt_manager/jobs/thumbnails')) starts.push(item.postDataJSON()); });
const layout = await openOverview(page);
const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) });
await expect(row.locator('h3')).toHaveText('2 missing thumbnails');
// Until the thumbnail review dialog (S5), Review thumbnails opens Sources.
await row.getByRole('button', { name: 'Review thumbnails', exact: true }).click();
const review = page.locator('dialog#modal[open]');
await expect(review.locator('#modal-title')).toHaveText('Thumbnail maintenance');
await expect(page).toHaveURL(`${workspace}#/system/overview`);
await expect(review.locator('.field > label')).toHaveText(['Scope', 'Quality', 'Preview size limit']);
await expect(review.locator('#modal-body > p.small')).toHaveText(`2 previews to generate · ${number(plan.fresh)} already up to date.`);
// Escape closes it and focus returns to the row's button.
await page.keyboard.press('Escape');
await expect(review).toHaveCount(0);
await expect(row.getByRole('button', { name: 'Review thumbnails', exact: true })).toBeFocused();
await row.getByRole('button', { name: 'Review thumbnails', exact: true }).click();
await review.getByRole('button', { name: 'Start', exact: true }).click();
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
expect(starts).toEqual([{ mode: 'missing', size: 512, quality: 'medium' }]);
const id = page.url().split('/').pop();
await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).toBe('finished');
await expect(page).toHaveURL(`${workspace}#/system/sources`);
});
test('the thumbnail row paints when its own status read answers', async ({ page, request }) => {
@@ -191,17 +168,15 @@ test('a failed thumbnail read is unavailable, never zero', async ({ page }) => {
await expect(layout.locator('.metric').nth(2).locator('strong')).toHaveText('24');
});
test('ambiguous row reads the pending count and Review metadata opens Needs review', async ({ page }) => {
test('ambiguous row reads the pending count; its action waits for the review page', async ({ page }) => {
const layout = await openOverview(page);
const row = layout.locator('.operation').filter({ hasText: 'ambiguous image links' });
await expect(row.locator('h3')).toHaveText('0 ambiguous image links');
await expect(row.locator('p')).toHaveText('Review metadata before assigning a positive / negative pair.');
const review = row.getByRole('button', { name: 'Review metadata', exact: true });
await expect(review).not.toHaveAttribute('aria-disabled');
await review.click();
// The concept's go-review: the Needs review page.
await expect(page).toHaveURL(`${workspace}#/review`);
await expect(page.getByRole('heading', { name: 'Metadata review', exact: true })).toBeVisible();
await expect(review).toHaveAttribute('aria-disabled', 'true');
await review.click({ force: true });
await expect(page).toHaveURL(`${workspace}#/system/overview`);
});
test('View backups / Appearance / Manage sources switch sections', async ({ page }) => {
+5 -7
View File
@@ -22,8 +22,6 @@ const routes = [
['system', 'System', null],
// #249 D51: SAVED VIEWS' Favorites, Images filtered to favorites.
['favorites', 'Favorites', null],
// #249 PR F (ruling R5): SAVED VIEWS' Needs review, the concept's Metadata review page (its own heading).
['review', 'Needs review', null, 'Metadata review'],
// #249 Activity-1: the concept's Activity page in the tools-and-system group.
['activity', 'Activity', null],
];
@@ -79,10 +77,10 @@ test('workspace starts with real persisted statistics without mutating data', as
});
test('all shell destinations deep-link, reload and expose working existing applications', async ({ page, request }) => {
for (const [route, label, legacy, title = label] of routes) {
for (const [route, label, legacy] of routes) {
await page.goto(`${workspace}#/${route}`);
const main = page.locator('#workspace-main');
await expect(main.getByRole('heading', { level: 1, name: title, exact: true })).toBeVisible();
await expect(main.getByRole('heading', { level: 1, name: label, exact: true })).toBeVisible();
await expect(sidebarLink(page, label)).toHaveAttribute('aria-current', 'page');
if (route === 'tags' || route === 'inspect') await expect(main.locator('.handoff')).toHaveCount(0);
if (route === 'inspect') await expect(main.locator('#local-file')).toBeAttached();
@@ -91,7 +89,7 @@ test('all shell destinations deep-link, reload and expose working existing appli
await expect(await request.get(legacy.split('#')[0])).toBeOK();
}
await page.reload();
await expect(main.getByRole('heading', { level: 1, name: title, exact: true })).toBeVisible();
await expect(main.getByRole('heading', { level: 1, name: label, exact: true })).toBeVisible();
}
// D29 and #249 E3: the legacy gallery and Library are reached from Overview's tool cards, not from their pages.
await page.goto(`${workspace}#/overview`);
@@ -311,9 +309,9 @@ test('overview reports loading, offers retry after failure and renders real empt
test('small screens keep navigation and the theme accessible without horizontal page overflow', async ({ page }) => {
await page.setViewportSize({ width: 375, height: 812 });
await page.goto(`${workspace}#/overview`);
for (const [, label, , title = label] of routes) {
for (const [, label] of routes) {
await sidebarLink(page, label).click();
await expect(page.locator('#workspace-main h1')).toHaveText(title);
await expect(page.locator('#workspace-main h1')).toHaveText(label);
}
await expect(page.getByRole('combobox', { name: 'Theme', exact: true })).toBeVisible();
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
+10 -47
View File
@@ -5,23 +5,18 @@ reading files, so a test sets its progress and counters exactly:
- ``POST /__test__/jobs/start`` starts one (``kind``, ``title``, ``total``,
``output_folders``; for ``kind: "image-tags"`` the image tagging job's
groups and its ``{mode, images}`` scope instead; for ``kind:
"thumbnails"`` the thumbnail job's groups, ``guarded`` admission and its
``{mode, size, quality, output_folders}`` scope; for ``kind: "health"`` the
health check's: no groups, ``read``, not cancellable, with its status
line) and returns its summary, or 409 while one runs;
groups and its ``{mode, images}`` scope instead) and returns its summary,
or 409 while one runs;
- ``POST /__test__/jobs/{id}/step`` records ``records`` (``{group, path,
root, root_index, detail}`` file outcomes, counted, listed and logged as the scan
records them) and ``bulk`` (``{group, count, root}``: ``count`` numbered
rows), sets ``processed`` (and ``total``) and ``counters``, then publishes
the progress to every subscriber;
- ``POST /__test__/jobs/{id}/finish`` ends it as ``finished``, ``cancelled``
or ``failed``, with the runner's final ``status_line`` when given; a cancel
through the real route ends it ``cancelled``;
- ``POST /__test__/jobs/hold`` opens a legacy-page hold (``kind``, ``title``,
``admission``, default ``records``): a running, non-cancellable job with no
outcome groups, as a legacy restore, maintenance or thumbnail clear request
takes, until ``finish`` ends it;
or ``failed``; a cancel through the real route ends it ``cancelled``;
- ``POST /__test__/jobs/hold`` opens a legacy-page hold (``kind``, ``title``):
a running, non-cancellable ``records`` job, as a legacy restore or
maintenance request takes, until ``finish`` ends it;
- ``POST /__test__/jobs/restart`` acts out a ComfyUI restart: the open job
streams drop without a final event and the registry starts empty.
"""
@@ -40,9 +35,6 @@ PREFIXES = {"added": "+", "known": "=", "review": "?", "errors": "!"}
PREFIXES["no_metadata"] = "-"
# The image tagging job's groups (#249 PR D), logged with the runner's prefixes.
PREFIXES.update({"tagged": "+", "unchanged": "=", "skipped": "-"})
# The thumbnail job's (#249 System S5).
PREFIXES["generated"] = "+"
HEALTH_STATUS_LINE = "Checking the database and image links"
ROOT_INDEXES = {"output": 0, "input": 1}
@@ -88,12 +80,9 @@ def outcome_records(data):
def register_activity_commands(app, routes, api):
from py.api.jobs import (
HEALTH_JOB_KIND,
IMAGE_TAG_GROUP_NAMES,
IMAGE_TAG_JOB_KIND,
SCAN_GROUP_NAMES,
THUMBNAIL_GROUP_NAMES,
THUMBNAILS_JOB_KIND,
JobBusy,
JobRegistry,
)
@@ -149,21 +138,7 @@ def register_activity_commands(app, routes, api):
data = await body(request)
ending = asyncio.get_running_loop().create_future()
kind = str(data.get("kind", "scan"))
admission, cancellable = "records", True
if kind == THUMBNAILS_JOB_KIND:
title = "Thumbnail generation"
scope = {
"mode": str(data.get("mode", "missing")),
"size": count(data.get("size"), 512),
"quality": str(data.get("quality", "medium")),
"output_folders": count(data.get("output_folders"), 2),
}
groups = THUMBNAIL_GROUP_NAMES
admission = "guarded"
elif kind == HEALTH_JOB_KIND:
title, scope, groups = "Health check", {}, ()
admission, cancellable = "read", False
elif kind == IMAGE_TAG_JOB_KIND:
if kind == IMAGE_TAG_JOB_KIND:
title = "Add tags to images"
scope = {
"mode": str(data.get("mode", "add")),
@@ -175,20 +150,13 @@ def register_activity_commands(app, routes, api):
scope = {
"files": str(data.get("files", "new")),
"output_folders": count(data.get("output_folders"), 2),
# A real scan's count of its files in the Needs review queue.
**(
{"review_items": count(data["review_items"])}
if "review_items" in data
else {}
),
}
groups = SCAN_GROUP_NAMES
try:
job = api._jobs.start(
kind,
str(data.get("title", title)),
admission=admission,
cancellable=cancellable,
record_changing=True,
scope=scope,
runner=fake_runner(ending),
groups=groups,
@@ -196,8 +164,6 @@ def register_activity_commands(app, routes, api):
except JobBusy:
raise web.HTTPConflict()
job.update(total=count(data.get("total")))
if kind == HEALTH_JOB_KIND:
job.update(status_line=HEALTH_STATUS_LINE)
endings[job.id] = ending
return web.json_response({"success": True, "job": job.summary()})
@@ -212,7 +178,7 @@ def register_activity_commands(app, routes, api):
async with api._jobs.hold(
str(data.get("kind", "restore")),
str(data.get("title", "Database restore (admin page)")),
admission=str(data.get("admission", "records")),
admission="records",
) as job:
entered.set_result(job)
state = await ending
@@ -260,12 +226,9 @@ def register_activity_commands(app, routes, api):
@routes.post("/__test__/jobs/{job_id}/finish")
async def finish_job(request):
job = job_or_404(request)
data = await body(request)
state = data.get("state", "finished")
state = (await body(request)).get("state", "finished")
if state not in FINAL_STATES:
raise web.HTTPBadRequest()
if isinstance(data.get("status_line"), str): # the runner's final line
job.update(status_line=data["status_line"])
ending = endings[job.id]
if not ending.done():
ending.set_result(state)
-368
View File
@@ -1,368 +0,0 @@
"""Owned, deterministic Needs review items for the workspace page (#249 PR F).
Each file is a PNG whose workflow has two samplers with their own positive
encoders and one shared negative, so the resolver finds it ``ambiguous``. Its
loader node carries ComfyUI's ``Infinity`` stamp, which the legacy pages'
parser (the scan's fallback) cannot read, so nothing links the file and it is
a Needs review case (#284: only ambiguous, unlinked files are queued). It is
recorded through the producers' entry point (``record_review_file``). No path
or SQL comes from a request.
``POST /__test__/review/reset`` also takes the special items F2 needs, queued
after the plain ones in this order:
- ``conflict``: an item whose file was linked to another prompt ("Misty
forest path") after it was queued (the next scan or monitor pass would
remove the item); applying a sampler path then conflicts with that link.
- ``limited``: a graph of 17 samplers, each with its own prompt, which the
recorder stores ``limited`` (more than 16 candidates, so none are stored).
An A1111 block with two negative lines is not queued any more: it offers
no candidate to choose (#284).
- ``unavailable``: a file recorded under the ``temp`` directory while it was a
gallery root; the root is then removed again, so no current root lists it.
``concept: true`` (F3, the concept comparison) names and sizes the plain
files as the concept's first three images (app.js ``images``) and gives
their first sampler the concept's first prompt (``alpine``: its positive and
its first negative, the one the concept's panel shows), so the panel's text
wraps as the concept's does.
``POST /__test__/review/add {kind}`` adds one file while a page is open:
``monitor`` records a new ambiguous, unlinked output from a worker thread
through ``record_review_file`` (the monitor's own entry point; the monitor
never queues an output it linked itself), ``file`` only writes one (a
metadata scan records it). ``GET
/__test__/review/links`` answers each review file's prompt and pair links, and
``GET /__test__/review/streams`` the open change streams and captures.
"""
from __future__ import annotations
import json
import os
import threading
from pathlib import Path
import shutil
import time
from typing import TYPE_CHECKING
from aiohttp import web
from PIL import Image
from PIL.PngImagePlugin import PngInfo
if TYPE_CHECKING:
from py.api import PromptManagerAPI
POSITIVES = ("Golden dunes at dusk", "Harbor lights at night")
NEGATIVE = "Soft negative, blur"
# The prompt the conflict item's file is linked to after it was queued.
FALLBACK = "Misty forest path"
SPECIAL = ("conflict", "limited", "unavailable")
# The concept's first three images (concept app.js `images`) and first prompt.
CONCEPT_IMAGES = (
("alpine_00001.png", (1536, 1152)),
("dunes_00001.png", (1024, 1536)),
("forest_00001.png", (1280, 1280)),
)
CONCEPT_POSITIVE = (
"A remote alpine observatory at blue hour, warm light through glass, dramatic "
"mountain peaks, cinematic composition, atmospheric mist, fine detail"
)
CONCEPT_NEGATIVE = "blurry, low quality, low resolution"
# More candidates than an item stores (16): the ``limited`` item.
LIMITED_SAMPLERS = 17
# More than one queue page (40), so Load more has a second page.
MAX_ITEMS = 45
_TABLES = (
"saved_views",
"image_favorites",
"media_review_decisions",
"media_review_items",
"media_image_tags",
"media_annotation_bytes",
"media_annotation_files",
"image_prompt_pairs",
"prompt_pair_tags",
"prompt_pairs",
"generated_images",
"prompt_tags",
"tags",
"prompt_library_state",
"prompts",
)
def _graph(
encoder_class: str = "CLIPTextEncode",
positives: tuple = POSITIVES,
negative: str = NEGATIVE,
) -> dict:
graph = {
# ``Infinity`` (as ComfyUI writes it): the legacy parser cannot read the
# graph, so the scan's fallback links nothing and the file is queued.
"1": {
"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "m"},
"is_changed": [float("inf")],
},
"2": {"class_type": "CLIPTextEncode", "inputs": {"text": negative}},
}
for index, text in enumerate(positives):
encoder, sampler = str(10 + 2 * index), str(11 + 2 * index)
graph[encoder] = {"class_type": encoder_class, "inputs": {"text": text}}
graph[sampler] = {
"class_type": "KSampler",
"inputs": {
"model": ["1", 0],
"positive": [encoder, 0],
"negative": ["2", 0],
},
}
return graph
def _png(
path: Path,
colour: tuple[int, int, int],
*,
graph: dict | None = None,
parameters: str | None = None,
size: tuple[int, int] = (96, 64),
) -> None:
info = PngInfo()
if parameters is not None:
info.add_text("parameters", parameters)
else:
info.add_text("prompt", json.dumps(graph or _graph()))
path.parent.mkdir(parents=True, exist_ok=True)
with Image.new("RGB", size, colour) as image:
image.save(path, pnginfo=info)
def _record(api: PromptManagerAPI, path: Path, expected: str) -> None:
outcome = api.record_review_file(str(path))
if outcome != expected:
raise RuntimeError(f"{path.name} was {outcome}, not {expected}")
def _stamp(path: Path, age: float) -> None:
stamp = time.time() - age
os.utime(path, (stamp, stamp))
class _Monitor:
"""The monitor's recorder over this server's database (one per server)."""
def __init__(self, api: PromptManagerAPI) -> None:
self.api = api
self.lock = threading.Lock()
def new_output(self, path: Path) -> None:
"""The monitor's Needs review pass for a new output, from a worker thread."""
outcome = []
with self.lock:
worker = threading.Thread(
target=lambda: outcome.append(self.api.record_review_file(str(path)))
)
worker.start()
worker.join()
if outcome != ["recorded"]:
raise RuntimeError(f"{path.name} was {outcome}, not recorded")
def _monitored_output(api: PromptManagerAPI, monitor: _Monitor, path: Path) -> None:
_png(path, (150, 60, 40))
monitor.new_output(path)
def _linked_after_queueing(api: PromptManagerAPI, path: Path) -> None:
from utils.hashing import generate_prompt_hash
_png(path, (150, 60, 40))
_record(api, path, "recorded")
prompt = api.db.get_prompt_by_hash(generate_prompt_hash(FALLBACK))
prompt_id = (
prompt["id"]
if prompt is not None
else api.db.save_prompt(FALLBACK, prompt_hash=generate_prompt_hash(FALLBACK))
)
api.db.link_image_to_prompt(prompt_id, str(path))
def _unavailable(api: PromptManagerAPI, root: Path, path: Path) -> None:
from py.config import GalleryConfig
roots = list(GalleryConfig.MONITORING_DIRECTORIES)
GalleryConfig.MONITORING_DIRECTORIES = [*roots, str(root / "temp")]
try:
_record(api, path, "recorded")
finally:
GalleryConfig.MONITORING_DIRECTORIES = roots
def seed_review(
api: PromptManagerAPI,
root: Path,
items: int,
special: tuple = (),
monitor: _Monitor | None = None,
concept: bool = False,
) -> dict:
"""Replace this server's disposable rows and files with ``items`` queued files."""
with api.db.model.get_connection() as conn:
for table in _TABLES:
conn.execute(f"DELETE FROM {table}")
api.close_annotation_captures()
for name in ("output", "input"):
shutil.rmtree(root / name)
(root / name).mkdir()
shutil.rmtree(root / "temp" / "review", ignore_errors=True)
output = root / "output"
files = []
for number in range(items):
relative = f"review/review_{number:04}.png"
options = {}
if concept:
name, size = CONCEPT_IMAGES[number % len(CONCEPT_IMAGES)]
relative = f"review/{name}" if number < len(CONCEPT_IMAGES) else relative
graph = _graph(
positives=(CONCEPT_POSITIVE, POSITIVES[1]), negative=CONCEPT_NEGATIVE
)
options = {"graph": graph, "size": size}
_png(output / relative, (40 + number * 4, 90, 140), **options)
_stamp(output / relative, 60 + number)
_record(api, output / relative, "recorded")
files.append({"path": relative, "filename": Path(relative).name})
for kind in SPECIAL:
if kind not in special:
continue
relative = f"review/{kind}.png"
path = (root / "temp" if kind == "unavailable" else output) / relative
if kind == "conflict":
_linked_after_queueing(api, path)
elif kind == "limited":
many = tuple(f"{POSITIVES[0]} {n}" for n in range(LIMITED_SAMPLERS))
_png(path, (90, 90, 90), graph=_graph(positives=many))
_record(api, path, "limited")
else:
_png(path, (20, 140, 90))
_unavailable(api, root, path)
files.append({"path": relative, "filename": path.name, "kind": kind})
api.invalidate_gallery_cache()
return {
"files": files,
"positives": list(POSITIVES),
"negative": NEGATIVE,
"fallback": FALLBACK,
}
def add_review_file(
api: PromptManagerAPI, root: Path, kind: str, monitor: _Monitor
) -> dict:
"""One more ambiguous output: through the monitor, or only written."""
output = root / "output"
number = (
len(list((output / "added").glob("*.png")))
if (output / "added").exists()
else 0
)
relative = f"added/added_{number:04}.png"
if kind == "monitor":
_monitored_output(api, monitor, output / relative)
else:
_png(output / relative, (200, 200, 40))
api.invalidate_gallery_cache()
return {"path": relative, "filename": Path(relative).name}
_LINKS = (
"SELECT gi.filename, p.text AS prompt, pp.id AS pair_id, pos.text AS positive, "
"neg.text AS negative FROM generated_images gi "
"JOIN prompts p ON p.id = gi.prompt_id "
"LEFT JOIN image_prompt_pairs ipp ON ipp.image_id = gi.id "
"LEFT JOIN prompt_pairs pp ON pp.id = ipp.pair_id "
"LEFT JOIN prompts pos ON pos.id = pp.positive_prompt_id "
"LEFT JOIN prompts neg ON neg.id = pp.negative_prompt_id "
"ORDER BY gi.filename, gi.id, pp.id"
)
def review_links(api: PromptManagerAPI) -> list:
"""Every file's prompt link and pair links, as plain rows."""
conn = api.db.model.get_connection()
return [dict(row) for row in conn.execute(_LINKS)]
def register_review_commands(
routes: web.RouteTableDef, api: PromptManagerAPI, root: Path
) -> None:
"""Register the review fixture's commands; no path or SQL comes from a request."""
monitors = []
def monitor() -> _Monitor:
if not monitors:
monitors.append(_Monitor(api))
return monitors[0]
@routes.post("/__test__/review/reset")
async def reset(request: web.Request) -> web.Response:
data = await request.json() if request.can_read_body else {}
if (
request.query
or not isinstance(data, dict)
or set(data) - {"items", "concept", *SPECIAL}
):
raise web.HTTPBadRequest()
items = data.get("items", 3)
if isinstance(items, bool) or not isinstance(items, int):
raise web.HTTPBadRequest()
if not 0 <= items <= MAX_ITEMS:
raise web.HTTPBadRequest()
if any(
not isinstance(data.get(kind, False), bool)
for kind in (*SPECIAL, "concept")
):
raise web.HTTPBadRequest()
special = tuple(kind for kind in SPECIAL if data.get(kind))
manifest = await api._run_in_executor(
seed_review,
api,
root,
items,
special,
monitor(),
data.get("concept", False),
)
return web.json_response({"success": True, **manifest})
@routes.post("/__test__/review/add")
async def add(request: web.Request) -> web.Response:
data = await request.json() if request.can_read_body else None
if request.query or not isinstance(data, dict) or set(data) != {"kind"}:
raise web.HTTPBadRequest()
if data["kind"] not in ("monitor", "file"):
raise web.HTTPBadRequest()
added = await api._run_in_executor(
add_review_file, api, root, data["kind"], monitor()
)
return web.json_response({"success": True, **added})
@routes.get("/__test__/review/links")
async def links(request: web.Request) -> web.Response:
if request.query:
raise web.HTTPBadRequest()
return web.json_response(
{"links": await api._run_in_executor(review_links, api)}
)
@routes.get("/__test__/review/streams")
async def streams(request: web.Request) -> web.Response:
"""Open review change streams and held captures (teardown checks)."""
if request.query:
raise web.HTTPBadRequest()
store = api._captures
captures = store.stats()["captures"] if store is not None else 0
return web.json_response(
{"streams": api.review_stream_count(), "captures": captures}
)
-120
View File
@@ -110,123 +110,3 @@ test('saving gallery settings retains the remembered thumbnail dismissal', () =>
f.manager.saveSettings();
assert.equal(JSON.parse(f.storage.get('gallerySettings')).thumbnailPromptDismissed, 'remembered-state');
});
// #249 System S5: the stream's work is now an Activity job that outlives the page's connection, so the gallery's
// Cancel asks the server to stop the job. The stream's first frame, `job {id, started}`, names the job; the gallery
// cancels by that id, only a job this stream started, and a Cancel before the id waits for it (review I1).
const JOB = 'b'.repeat(32);
function streamFixture({ modal = false, postFails = false } = {}) {
const posts = [];
const warnings = [];
const gets = [];
const listeners = {};
const source = { closed: 0, readyState: 1, CLOSED: 2, url: null,
addEventListener(type, handler) { (listeners[type] ||= []).push(handler); },
close() { this.closed += 1; } };
const element = () => ({ classList: { add() {}, remove() {} }, style: {}, textContent: '', handlers: {},
addEventListener(type, handler) { this.handlers[type] = handler; },
removeEventListener(type, handler) { if (this.handlers[type] === handler) delete this.handlers[type]; },
click() { this.handlers.click?.(); } });
const elements = {};
const document = { addEventListener() {}, getElementById: id => elements[id] ||= element(),
createElement: () => element(), body: { appendChild() {}, removeChild() {} } };
const context = vm.createContext({ document, window: {},
console: { debug() {}, warn: (...args) => warnings.push(args), error() {} },
setTimeout() {}, localStorage: { setItem() {} },
SseStream: { connect: (url, options) => { source.url = url; source.options = options; return source; } } });
const file = path.join(__dirname, '../../web/js/gallery.js');
const Manager = vm.runInContext(fs.readFileSync(file, 'utf8') + '\nGalleryManager;', context, { filename: file });
const manager = Object.create(Manager.prototype);
Object.assign(manager, { getSettings: () => ({}),
api: { get: async url => { gets.push(url); return { success: true, jobs: [] }; },
post: async (url, body) => {
posts.push([url, body]);
if (postFails) throw new Error('offline');
return { success: true };
} } });
const emit = (type, data) => (listeners[type] || []).forEach(handler => handler({ data: JSON.stringify(data) }));
const ui = { addStatusMessage() {}, progressText: element(), progressPercent: element(), progressBar: element(),
progressDetails: element(), progressETA: element(), isCancelled: () => false,
onCancel: handler => { ui.cancel = handler; } };
const result = modal ? manager.generateThumbnailsWithProgressInModal(ui)
: manager.generateThumbnailsWithProgress({ quality: 'medium' });
const outcome = result.then(value => ({ value }), error => ({ error: error.message }));
const cancel = () => (modal ? ui.cancel() : elements.cancelThumbnailsBtn.click());
const settle = () => new Promise(resolve => setImmediate(resolve));
return { emit, source, posts, gets, warnings, outcome, cancel, settle };
}
const cancelUrl = id => `/prompt_manager/jobs/${id}/cancel`;
test('Cancel before the job id arrives cancels that job once it does', async () => {
const f = streamFixture();
f.cancel();
await f.settle();
assert.deepEqual(f.posts, []);
assert.equal(f.source.closed, 0); // kept open until the stream names its job
assert.deepEqual(await f.outcome, { error: 'Thumbnail generation cancelled by user' });
f.emit('status', { phase: 'scanning', message: 'Scanning…' });
f.emit('job', { id: JOB, started: true });
await f.settle();
assert.equal(JSON.stringify(f.posts), JSON.stringify([[cancelUrl(JOB), {}]]));
assert.equal(f.source.closed, 1);
f.emit('job', { id: 'c'.repeat(32), started: true }); // a stray second frame changes nothing
await f.settle();
assert.equal(f.posts.length, 1);
assert.deepEqual(f.gets, []); // never looks a job up by kind
});
test('Cancel after the id cancels it at once, and a double Cancel posts once', async () => {
const f = streamFixture();
f.emit('job', { id: JOB, started: true });
f.cancel();
f.cancel();
await f.settle();
assert.equal(JSON.stringify(f.posts), JSON.stringify([[cancelUrl(JOB), {}]]));
assert.equal(f.source.closed, 1);
assert.deepEqual(await f.outcome, { error: 'Thumbnail generation cancelled by user' });
assert.deepEqual(f.gets, []);
});
test("Cancel leaves another tab's or System's job running and only stops following it", async () => {
const f = streamFixture();
f.emit('job', { id: JOB, started: false });
f.cancel();
await f.settle();
assert.deepEqual(f.posts, []);
assert.deepEqual(f.gets, []);
assert.equal(f.source.closed, 1);
assert.match((await f.outcome).error, /keeps running/);
});
test('a Cancel the stream never names a job for posts nothing and closes when it ends', async () => {
const f = streamFixture();
f.cancel();
f.emit('error', { error: 'Scan for metadata already running' });
await f.settle();
assert.deepEqual(f.posts, []);
assert.equal(f.source.closed, 1);
});
test('a cancel post that fails is reported, not thrown', async () => {
const f = streamFixture({ postFails: true });
f.emit('job', { id: JOB, started: true });
f.cancel();
await f.settle();
assert.equal(f.posts.length, 1);
assert.equal(f.warnings.length, 1);
assert.deepEqual(await f.outcome, { error: 'Thumbnail generation cancelled by user' });
});
test('the startup dialog cancels its own job by id too', async () => {
const f = streamFixture({ modal: true });
f.cancel();
await f.settle();
assert.deepEqual(f.posts, []);
f.emit('job', { id: JOB, started: true });
await f.settle();
assert.equal(JSON.stringify(f.posts), JSON.stringify([[cancelUrl(JOB), {}]]));
assert.equal(f.source.closed, 1);
assert.match((await f.outcome).error, /cancelled/);
});
-148
View File
@@ -1,148 +0,0 @@
// Per-kind Activity descriptors (#249 System S5 Task 13, D111): each job kind's pill verb, toasts, line, tiles,
// outcomes, openable rows and labels, from one table (web/js/workspace/activity-kinds.mjs).
const test = require('node:test');
const assert = require('node:assert/strict');
const source = import('../../web/js/workspace/activity-kinds.mjs');
const store = import('../../web/js/workspace/activity-store.mjs');
const ID = 'a'.repeat(32);
function job(overrides = {}) {
return { id: ID, kind: 'scan', title: 'Scan for metadata', state: 'running', started_at: 1, finished_at: null,
progress: 42, status_line: '', processed: 8240, total: 19622,
counters: { added: 1, known: 0, review: 0, no_metadata: 0, errors: 1 }, scope: {}, listed: {}, log_length: 0,
cancellable: true, ...overrides };
}
test('scan descriptors are unchanged', async () => {
const { kindOf } = await source;
const scan = kindOf('scan');
assert.equal(scan.pillVerb, 'Scanning');
assert.equal(scan.startToast(job()), 'Scan started.');
assert.deepEqual(scan.tiles, [['added', 'Added to Library', 'added'], ['known', 'Already known', ''],
['review', 'Needs review', 'warn'], ['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad']]);
assert.deepEqual(scan.outcomes, [['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
['no_metadata', 'No usable metadata'], ['known', 'Already known']]);
assert.deepEqual([...scan.openable].sort(), ['added', 'known']);
assert.deepEqual([scan.log, scan.cancelLabel], ['Scan log', 'Cancel scan']);
assert.equal(scan.finishToast(job({ state: 'finished' })), 'Scan finished · 1 added · 1 error');
assert.equal(scan.finishToast(job({ state: 'cancelled' })), 'Scan cancelled · 1 added');
assert.equal(scan.finishToast(job({ state: 'failed' })), 'Scan failed · 1 added · 1 error');
assert.equal(scan.line(job()), '1 added · 1 error');
assert.equal(scan.kept(job()), '1 added');
assert.equal(scan.scope(job({ scope: { output_folders: 2 } })), '2 output folders');
// The store keeps answering with the same table.
const { jobView, finishToast, startToast, pillParts, outcomeLine } = await store;
assert.equal(jobView('scan'), scan);
assert.equal(finishToast(job({ state: 'finished' })), 'Scan finished · 1 added · 1 error');
assert.equal(startToast(job()), 'Scan started.');
assert.equal(pillParts(job()).text, 'Scanning · 42% · 8,240 / 19,622');
assert.equal(outcomeLine(job()), '1 added · 1 error');
});
test('thumbnail job shows its own tiles and toasts', async () => {
const { kindOf, viewOf } = await source;
const { finishToast, startToast, pillParts, outcomeLine } = await store;
const thumbs = job({ kind: 'thumbnails', title: 'Thumbnail generation', processed: 4, total: 10, progress: 40,
counters: { generated: 1420, skipped: 3, errors: 1 }, scope: { mode: 'missing', size: 512, quality: 'medium',
output_folders: 2 } });
const view = kindOf('thumbnails');
assert.equal(viewOf(thumbs), view);
assert.deepEqual(view.tiles, [['generated', 'Generated', 'added'], ['skipped', 'Skipped', ''], ['errors', 'Errors', 'bad']]);
assert.deepEqual(view.outcomes, [['generated', 'Generated'], ['errors', 'Errors'], ['skipped', 'Skipped']]);
assert.deepEqual([...view.openable].sort(), ['generated', 'skipped']);
assert.deepEqual([view.log, view.cancelLabel], ['Job log', 'Cancel job']);
assert.equal(pillParts(thumbs).text, 'Generating thumbnails · 40% · 4 / 10');
assert.equal(startToast(thumbs), 'Thumbnail generation started.');
assert.equal(finishToast({ ...thumbs, state: 'finished' }), 'Thumbnails finished · 1,420 generated · 1 error');
assert.equal(finishToast({ ...thumbs, state: 'failed' }), 'Thumbnails failed · 1,420 generated · 1 error');
assert.equal(finishToast({ ...thumbs, state: 'cancelled' }), 'Thumbnails cancelled · 1,420 generated');
assert.equal(finishToast({ ...thumbs, counters: { generated: 0, skipped: 0, errors: 2 }, state: 'finished' }),
'Thumbnails finished · 0 generated · 2 errors');
assert.equal(outcomeLine(thumbs), '1,420 generated · 1 error');
assert.equal(view.kept(thumbs), '1,420 generated');
assert.equal(view.scope(thumbs), '2 output folders');
});
test('non-cancellable jobs show no Cancel job', async () => {
const { canCancel } = await source;
assert.equal(canCancel(job()), true);
assert.equal(canCancel(job({ cancellable: false })), false);
for (const state of ['finished', 'cancelled', 'failed', 'lost']) assert.equal(canCancel(job({ state })), false);
// The health check is a read job the server cannot stop.
assert.equal(canCancel(job({ kind: 'health', title: 'Health check', cancellable: false, counters: {} })), false);
});
test('indeterminate progress has no value attribute', async () => {
const { indeterminate, showProgress } = await source;
const { pillParts } = await store;
function node() {
const attributes = new Map();
return { attributes, textContent: '', set value(v) { attributes.set('value', String(v)); },
get value() { return Number(attributes.get('value') ?? 0); },
setAttribute(key, v) { attributes.set(key, String(v)); }, removeAttribute(key) { attributes.delete(key); } };
}
// Running with no total yet: the server has not said how much there is, so there is no percentage.
const health = job({ kind: 'health', title: 'Health check', progress: 0, processed: 0, total: 0, counters: {},
cancellable: false, status_line: 'Checking the database and image links' });
assert.equal(indeterminate(health), true);
const bar = node();
bar.value = 37;
showProgress(bar, health);
assert.equal(bar.attributes.has('value'), false);
assert.equal(bar.textContent, '');
assert.equal(pillParts(health).text, 'Health check');
assert.deepEqual(pillParts(health), { label: 'Health check', files: '', text: 'Health check' });
// A known total is determinate, and so is any ended job.
const scan = job();
assert.equal(indeterminate(scan), false);
showProgress(bar, scan);
assert.equal(bar.attributes.get('value'), '42');
assert.equal(bar.textContent, '42%');
assert.equal(indeterminate({ ...health, state: 'finished', progress: 100 }), false);
// A scan before its file count is known is indeterminate too, never "0%".
assert.equal(pillParts(job({ total: 0, processed: 0, progress: 0 })).text, 'Scanning');
});
test('held legacy jobs read "(admin page)" with no outcomes', async () => {
const { viewOf } = await source;
const { jobDetail } = await import('../../web/js/workspace/activity.mjs');
const { finishToast, outcomeLine } = await store;
// A hold has no outcome groups (counters {}), whatever its kind: thumbnails held by the gallery's clear included.
for (const [kind, title] of [['restore', 'Database restore (admin page)'],
['thumbnails', 'Thumbnail clear (gallery page)'], ['maintenance', 'Database maintenance (admin page)']]) {
const held = job({ kind, title, counters: {}, cancellable: false, progress: 0, processed: 0, total: 0 });
const view = viewOf(held);
assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []], kind);
assert.equal(outcomeLine(held), '');
assert.match(jobDetail(held), /^Started \d\d:\d\d · closing this view does not cancel it$/);
const ended = { ...held, state: 'finished', finished_at: 2 };
assert.match(jobDetail(ended), /^Finished \d\d:\d\d$/);
assert.equal(finishToast(ended), `${title} finished.`);
assert.ok(title.endsWith('page)'));
}
});
test('unknown and generic kinds show no tiles of another kind', async () => {
const { kindOf, viewOf } = await source;
for (const kind of ['maintenance', '', 'constructor', '__proto__', 'health']) {
const view = kindOf(kind);
assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []], kind);
assert.deepEqual([view.log, view.cancelLabel], ['Job log', 'Cancel job']);
assert.equal(view.pillVerb, null);
assert.equal(view.line(job({ kind })), '');
assert.equal(view.kept(job({ kind })), '');
}
const backup = job({ kind: 'backup', title: 'Backup' });
assert.equal(kindOf('backup').startToast(backup), 'Backup started.');
assert.equal(kindOf('backup').finishToast({ ...backup, state: 'finished' }), 'Backup finished.');
assert.equal(kindOf('backup').finishToast({ ...backup, state: 'cancelled' }), 'Backup was cancelled.');
assert.equal(kindOf('backup').finishToast({ ...backup, state: 'failed' }), 'Backup failed.');
// A kind's tiles are only the groups its job counts: no zero tile for a group the server never reported.
const partial = viewOf(job({ counters: { added: 3, errors: 0 } }));
assert.deepEqual(partial.tiles, [['added', 'Added to Library', 'added'], ['errors', 'Errors', 'bad']]);
assert.deepEqual(partial.outcomes, [['added', 'Added to Library'], ['errors', 'Errors']]);
assert.deepEqual([...partial.openable], ['added']);
assert.equal(partial.log, 'Scan log');
});
+4 -7
View File
@@ -289,10 +289,8 @@ test('summaries are validated and the copy follows the spec', async () => {
const scan = parseJob(summary({ progress: 42, processed: 8240, total: 19622, counters: { added: 1, errors: 1 } }));
assert.deepEqual(pillParts(scan), { label: 'Scanning · 42%', files: ' · 8,240 / 19,622', text: 'Scanning · 42% · 8,240 / 19,622' });
// D111: without a total the progress is indeterminate, so the pill shows no percentage.
assert.equal(pillParts({ ...scan, total: 0 }).text, 'Scanning');
// A kind without a pill verb shows its title (D111: thumbnails have their own, see workspace-activity-kinds.test.js).
assert.equal(pillParts({ ...scan, kind: 'backup', title: 'Backup' }).text, 'Backup · 42% · 8,240 / 19,622');
assert.equal(pillParts({ ...scan, total: 0 }).text, 'Scanning · 42%');
assert.equal(pillParts({ ...scan, kind: 'thumbnails', title: 'Thumbnails' }).text, 'Thumbnails · 42% · 8,240 / 19,622');
assert.equal(finishToast({ ...scan, state: 'finished' }), 'Scan finished · 1 added · 1 error');
assert.equal(finishToast({ ...scan, state: 'cancelled' }), 'Scan cancelled · 1 added');
assert.equal(finishToast({ ...scan, state: 'failed' }), 'Scan failed · 1 added · 1 error');
@@ -334,8 +332,7 @@ test('image-tags jobs get their own tiles outcomes and pill verb', async () => {
'Add tags to images finished · 0 tagged · 0 errors');
});
// D111 (S5): an unknown kind no longer borrows the scan's tiles (S1 review M6, S2 review M3); it has none.
test('unknown kinds get the generic view without the scan tiles', async () => {
test('unknown kinds keep the generic view', async () => {
const { jobView } = await source;
const scan = jobView('scan');
assert.deepEqual(scan.tiles, [['added', 'Added to Library', 'added'], ['known', 'Already known', ''],
@@ -346,7 +343,7 @@ test('unknown kinds get the generic view without the scan tiles', async () => {
assert.deepEqual([scan.log, scan.cancel], ['Scan log', 'Cancel scan']);
for (const kind of ['maintenance', '', 'constructor', '__proto__']) {
const view = jobView(kind);
assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []]);
assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [scan.tiles, scan.outcomes, [...scan.openable]]);
assert.deepEqual([view.log, view.cancel], ['Job log', 'Cancel job']);
}
});
-788
View File
@@ -1,788 +0,0 @@
// Run with: node --test tests/js/workspace-review-state.test.js
const test = require('node:test');
const assert = require('node:assert/strict');
const { createApiClient } = require('../../web/js/api-client.js');
// Only an absent module is tolerated, so RED is an assertion failure, not a crash.
const source = import('../../web/js/workspace/review-state.mjs').catch(error => {
if (error.code !== 'ERR_MODULE_NOT_FOUND') throw error;
return {};
});
async function load() {
const module = await source;
assert.equal(typeof module.createReviewController, 'function', 'review-state.mjs is not implemented');
return module;
}
const EPOCH = '11111111-1111-4111-8111-111111111111';
const SCOPE = 'c'.repeat(64);
const SHA = 'd'.repeat(64);
const PRINT = 'e'.repeat(64);
const HASH_A = 'a'.repeat(64);
const HASH_B = 'b'.repeat(64);
const HASH_C = 'f'.repeat(64);
const ITEMS = '/prompt_manager/review/items';
const OBSERVE = '/prompt_manager/image-annotations/observe';
const CAPTURE = '/prompt_manager/image-annotations/capture/';
const flush = () => new Promise(resolve => setImmediate(resolve));
async function settle(times = 8) { for (let i = 0; i < times; i++) await flush(); }
function response(status, body) {
return { ok: status >= 200 && status < 300, status, statusText: `Status ${status}`, json: async () => body };
}
const fail = (status, code, error = `failure ${code}`, extra = {}) => response(status, { success: false, code, error, ...extra });
function item(id, overrides = {}) {
const path = `dunes/${id}.png`;
return {
id, state: 'pending', revision: 0, created_at: `2026-10-22T10:00:0${id.length}+00:00`, filename: `${id}.png`,
target: { root_index: 0, root_scope: SCOPE, path }, unavailable: false, candidate_count: 3, limited: false,
url: `/prompt_manager/images/serve/${path}?root=0&root_scope=${SCOPE}`, is_video: false, ...overrides,
};
}
const candidate = (hash, sampler, label, positive, negative, state = 'resolved') => ({
hash, sampler_id: sampler, state, reason: '', label,
positive: positive === null ? null : { text: positive }, negative: negative === null ? null : { text: negative },
});
function detailOf(entry, overrides = {}) {
return {
success: true, library_epoch: EPOCH, ...entry, total: 2, sha256: SHA, dependency_fingerprint: PRINT,
candidates: [
candidate(HASH_C, '9', 'No positive / Soft negative', null, 'blur', 'unresolved'),
candidate(HASH_A, '12', 'Dunes / Soft negative', 'golden dunes', 'blur'),
candidate(HASH_B, '28', 'Dunes / Harsh negative', 'golden dunes', 'noise'),
],
...overrides,
};
}
const page = (items, overrides = {}) => ({ success: true, library_epoch: EPOCH, total: items.length, items, next_cursor: null, ...overrides });
const observed = (token = 'capture-1') => ({
success: true, library_epoch: EPOCH, target: item('one').target, filename: 'one.png', title: 'one.png', tags: [], tag_revision: 0,
capture: { token, sha256: SHA, preview_url: CAPTURE + token, size: 10, width: 8, height: 8, media_type: 'image',
expires_at: '2026-10-22T12:00:00+00:00', generate: { supported: true, reason: null } },
});
/** Real createApiClient over a fetch whose answers come from routes['METHOD path'](call). */
function harness(routes = {}) {
const calls = [];
const fetch = (url, init = {}) => {
const call = { url, method: init.method, body: init.body === undefined ? undefined : JSON.parse(init.body) };
calls.push(call);
const path = url.startsWith(CAPTURE) ? CAPTURE : url.split('?')[0];
const route = routes[`${call.method} ${path}`];
return new Promise((resolve, reject) => {
init.signal?.addEventListener('abort', () => reject(new DOMException('Aborted', 'AbortError')));
Promise.resolve().then(() => (route ? route(call) : response(200, { success: true }))).then(resolve, reject);
});
};
return {
api: createApiClient(fetch), calls,
of: (method, prefix) => calls.filter(call => call.method === method && call.url.startsWith(prefix)),
};
}
const QUEUE = [item('one'), item('two')];
function routes(extra = {}) {
return {
[`GET ${ITEMS}`]: () => response(200, page(QUEUE)),
[`GET ${ITEMS}/one`]: () => response(200, detailOf(QUEUE[0])),
[`GET ${ITEMS}/two`]: () => response(200, detailOf(QUEUE[1], { candidates: [], limited: true, candidate_count: 0 })),
[`POST ${OBSERVE}`]: () => response(200, observed()),
[`POST ${ITEMS}/one/decision`]: call => response(200, { success: true, library_epoch: EPOCH, total: 1,
item: { ...QUEUE[0], state: call.body.choice === 'unresolved' ? 'unresolved' : 'selected', revision: 1 } }),
...extra,
};
}
async function loaded(extra) {
const { createReviewController } = await load();
const h = harness(routes(extra));
const c = createReviewController({ api: h.api });
await c.load();
await settle();
return { c, h };
}
test('notice titles for 0, 1 and 3', async () => {
const { noticeTitle } = await load();
assert.equal(noticeTitle(0), 'No images need a decision');
assert.equal(noticeTitle(1), '1 image needs a decision');
assert.equal(noticeTitle(3), '3 images need a decision');
assert.equal(noticeTitle(1234), '1,234 images need a decision');
});
test('loads pending items and the exact count', async () => {
const { c, h } = await loaded();
assert.equal(c.state.status, 'ready');
assert.equal(c.state.total, 2);
assert.deepEqual(c.state.items.map(entry => entry.id), ['one', 'two']);
const [list] = h.of('GET', ITEMS);
assert.equal(list.url, `${ITEMS}?limit=40`);
// The first item is selected, as the concept selects its first queue image.
assert.equal(c.state.selectedId, 'one');
assert.ok(Object.isFrozen(c.state));
});
test('selecting a card loads detail and option labels, first workable preselected, unusable disabled', async () => {
const { c } = await loaded();
assert.equal(c.state.detailStatus, 'ready');
assert.deepEqual(c.state.options.map(option => [option.label, option.disabled]), [
['Sampler 9 · No positive / Soft negative', true],
['Sampler 12 · Dunes / Soft negative', false],
['Sampler 28 · Dunes / Harsh negative', false],
['Leave unresolved', false],
]);
assert.equal(c.state.options.at(-1).value, 'unresolved');
assert.equal(c.state.choice, HASH_A);
assert.deepEqual(c.state.prompts, { positive: { status: 'resolved', text: 'golden dunes' }, negative: { status: 'resolved', text: 'blur' } });
c.choose(HASH_C);
assert.equal(c.state.choice, HASH_A, 'a disabled option cannot be chosen');
c.choose(HASH_B);
assert.equal(c.state.prompts.negative.text, 'noise');
c.choose('unresolved');
assert.equal(c.state.prompts.positive.status, 'ambiguous');
await c.select('two');
await settle();
assert.equal(c.state.selectedId, 'two');
assert.deepEqual(c.state.options.map(option => option.label), ['Leave unresolved']);
assert.equal(c.state.choice, 'unresolved');
assert.match(c.state.warning, /too many sampler paths/i);
});
test('apply sends the selected candidate hash or unresolved with a fresh capture', async () => {
const { c, h } = await loaded();
c.choose(HASH_B);
const done = await c.apply();
assert.equal(done, true);
const [observe] = h.of('POST', OBSERVE);
assert.deepEqual(observe.body, { target: QUEUE[0].target });
const [decision] = h.of('POST', `${ITEMS}/one/decision`);
assert.deepEqual(decision.body, { expected_epoch: EPOCH, expected_revision: 0, capture_token: 'capture-1', sha256: SHA,
dependency_fingerprint: PRINT, choice: HASH_B });
assert.equal(c.state.phase, 'done');
assert.equal(c.state.total, 1);
assert.equal(c.state.toast, 'Association accepted.');
// The server releases a used capture itself.
assert.equal(h.of('DELETE', CAPTURE).length, 0);
const again = await loaded();
again.c.choose('unresolved');
assert.equal(await again.c.apply(), true);
const [unresolved] = again.h.of('POST', `${ITEMS}/one/decision`);
assert.equal(unresolved.body.choice, 'unresolved');
assert.equal(unresolved.body.capture_token, 'capture-1');
assert.equal(again.c.state.toast, 'Left unresolved.');
});
test('an unavailable file can only be left unresolved, without a capture', async () => {
const gone = item('one', { target: null, unavailable: true, url: null });
const { c, h } = await loaded({
[`GET ${ITEMS}`]: () => response(200, page([gone])),
[`GET ${ITEMS}/one`]: () => response(200, detailOf(gone)),
});
assert.ok(c.state.options.slice(0, -1).every(option => option.disabled));
assert.equal(c.state.choice, 'unresolved');
assert.match(c.state.warning, /not under a current image source/i);
assert.equal(await c.apply(), true);
assert.equal(h.of('POST', OBSERVE).length, 0);
const [decision] = h.of('POST', `${ITEMS}/one/decision`);
assert.equal(decision.body.capture_token, null);
assert.equal(decision.body.sha256, SHA);
});
test('association conflict keeps the selection and shows the inline warning', async () => {
const message = 'This image is already linked to another prompt or negative, so this sampler path cannot be attached safely.';
const { c, h } = await loaded({ [`POST ${ITEMS}/one/decision`]: () => fail(409, 'association-conflict', message) });
assert.equal(await c.apply(), false);
assert.equal(c.state.selectedId, 'one');
assert.equal(c.state.choice, HASH_A);
assert.equal(c.state.phase, 'idle');
assert.equal(c.state.warning, message);
// The refused decision's capture is released.
assert.equal(h.of('DELETE', CAPTURE).length, 1);
assert.equal(h.of('GET', `${ITEMS}?`).length, 1, 'no reload');
});
test('storage-changed requires reload', async () => {
let lists = 0;
const { c, h } = await loaded({
[`GET ${ITEMS}`]: () => { lists += 1; return response(200, page(QUEUE)); },
[`POST ${ITEMS}/one/decision`]: () => fail(409, 'storage-changed', 'Prompt storage was restored; reload the review queue.'),
});
assert.equal(await c.apply(), false);
await settle();
assert.equal(lists, 2, 'the queue is read again under the new epoch');
assert.equal(c.state.warning, 'Prompt storage was restored; reload the review queue.');
assert.equal(c.state.canRetry, false);
assert.equal(h.of('DELETE', CAPTURE).length, 1);
});
test('lost response keeps the selection and retry resends once', async () => {
let attempts = 0;
const { c, h } = await loaded({
[`POST ${ITEMS}/one/decision`]: call => {
attempts += 1;
if (attempts === 1) return Promise.reject(new TypeError('Failed to fetch'));
return response(200, { success: true, library_epoch: EPOCH, total: 1, item: { ...QUEUE[0], state: 'selected', revision: 1 } });
},
});
assert.equal(await c.apply(), false);
assert.equal(c.state.selectedId, 'one');
assert.equal(c.state.canRetry, true);
assert.match(c.state.warning, /may not have been saved/i);
assert.equal(h.of('DELETE', CAPTURE).length, 0, 'the capture is kept for the retry');
const [first] = h.of('POST', `${ITEMS}/one/decision`);
assert.equal(await c.retry(), true);
const sent = h.of('POST', `${ITEMS}/one/decision`);
assert.equal(sent.length, 2);
assert.deepEqual(sent[1].body, first.body, 'the identical decision is resent');
assert.equal(h.of('POST', OBSERVE).length, 1, 'no second capture');
assert.equal(await c.retry(), false, 'nothing left to resend');
assert.equal(h.of('POST', `${ITEMS}/one/decision`).length, 2);
});
test('a retry answered by review-conflict reloads the queue', async () => {
let attempts = 0, lists = 0;
const { c } = await loaded({
[`GET ${ITEMS}`]: () => { lists += 1; return response(200, page(lists === 1 ? QUEUE : [QUEUE[1]])); },
[`POST ${ITEMS}/one/decision`]: () => {
attempts += 1;
if (attempts === 1) return Promise.reject(new TypeError('Failed to fetch'));
return fail(409, 'review-conflict', 'This item was decided or changed since it was opened. Reload the queue.');
},
});
await c.apply();
assert.equal(await c.retry(), false);
await settle();
assert.equal(lists, 2);
assert.equal(c.state.selectedId, 'two');
});
test('a review-conflict on a still pending item reads its detail again', async () => {
let details = 0, decisions = 0;
const { c, h } = await loaded({
[`GET ${ITEMS}/one`]: () => { details += 1; return response(200, detailOf({ ...QUEUE[0], revision: details - 1 })); },
[`POST ${ITEMS}/one/decision`]: () => {
decisions += 1;
return decisions === 1 ? fail(409, 'review-conflict', 'This item was decided or changed since it was opened. Reload the queue.')
: response(200, { success: true, library_epoch: EPOCH, total: 1, item: { ...QUEUE[0], state: 'selected', revision: 2 } });
},
});
assert.equal(await c.apply(), false);
await settle();
assert.equal(details, 2, 'the item is read again');
assert.equal(c.state.detail.revision, 1);
assert.equal(c.state.warning, 'This item was decided or changed since it was opened. Reload the queue.');
assert.equal(await c.apply(), true);
assert.equal(h.of('POST', `${ITEMS}/one/decision`).at(-1).body.expected_revision, 1);
});
test('leave for later writes nothing', async () => {
const { c, h } = await loaded();
c.leave();
assert.equal(c.state.phase, 'done');
assert.equal(c.state.toast, null);
assert.equal(h.of('POST', '/').length, 0);
assert.equal(h.of('DELETE', '/').length, 0);
});
test('load more pages with the cursor and keeps the selection', async () => {
const more = [item('three')];
const { c, h } = await loaded({
[`GET ${ITEMS}`]: call => response(200, call.url.includes('cursor=')
? page(more, { total: 3 }) : page(QUEUE, { total: 3, next_cursor: 'abc_-1' })),
});
assert.equal(c.state.nextCursor, 'abc_-1');
await c.loadMore();
assert.equal(h.of('GET', ITEMS).at(-1).url, `${ITEMS}?limit=40&cursor=abc_-1`);
assert.deepEqual(c.state.items.map(entry => entry.id), ['one', 'two', 'three']);
assert.equal(c.state.nextCursor, null);
assert.equal(c.state.selectedId, 'one');
});
test('a failed queue read is an error state and load retries', async () => {
let lists = 0;
const { c } = await loaded({ [`GET ${ITEMS}`]: () => { lists += 1; return lists === 1 ? fail(500, 'review-failed') : response(200, page(QUEUE)); } });
assert.equal(c.state.status, 'error');
await c.load();
await settle();
assert.equal(c.state.status, 'ready');
});
test('tag save toast carries the review suffix and the item stays pending', async () => {
const { c, h } = await loaded({
['GET /prompt_manager/autotag/models']: () => response(200, { success: true, models: {}, default_prompt: 'x',
wd14_general_threshold: 0.35, wd14_character_threshold: 0.85, suggestion_limit: 10 }),
['PUT /prompt_manager/image-annotations/tags']: call => response(200, { success: true, library_epoch: EPOCH, sha256: SHA,
tags: call.body.tags, tag_revision: 1, replayed: false }),
});
const { createImageTaggingController } = await import('../../web/js/workspace/image-tagging-state.mjs');
const before = h.calls.filter(call => call.url.startsWith(ITEMS)).length;
// The Tag dialog opened from Needs review is the shared controller with the review source.
const tagging = createImageTaggingController({ api: h.api, target: QUEUE[0].target, source: 'review' });
assert.equal(tagging.state.showScopeNote, true);
await tagging.open();
await settle();
tagging.setManual('dunes');
await tagging.save();
await settle();
assert.equal(tagging.state.toast, '1 tags saved for one.png. Prompt association still awaits review.');
tagging.dispose();
assert.equal(h.calls.filter(call => call.url.startsWith(ITEMS)).length, before, 'a tag save neither decides nor re-reads the queue');
assert.deepEqual(c.state.items.map(entry => [entry.id, entry.state]), [['one', 'pending'], ['two', 'pending']]);
assert.equal(c.state.phase, 'idle');
});
test('dispose releases a kept capture and stops reads', async () => {
const { c, h } = await loaded({ [`POST ${ITEMS}/one/decision`]: () => Promise.reject(new TypeError('Failed to fetch')) });
await c.apply();
c.dispose();
await settle();
const released = h.of('DELETE', CAPTURE);
assert.equal(released.length, 1);
assert.equal(released[0].url, CAPTURE + 'capture-1');
assert.equal(await c.retry(), false);
});
// F2: an open page re-reads its queue when the server says it changed (the review change stream).
function refreshOf(c) {
assert.equal(typeof c.refresh, 'function', 'refresh is not implemented');
return c.refresh();
}
test('refresh re-reads the queue quietly and keeps the selection and choice', async () => {
let queue = QUEUE;
const { c, h } = await loaded({ [`GET ${ITEMS}`]: () => response(200, page(queue)) });
c.choose(HASH_B);
const statuses = [];
c.subscribe(state => statuses.push(state.status));
queue = [...QUEUE, item('three')];
await refreshOf(c);
await settle();
assert.deepEqual(c.state.items.map(entry => entry.id), ['one', 'two', 'three']);
assert.equal(c.state.total, 3);
assert.equal(c.state.selectedId, 'one');
assert.equal(c.state.choice, HASH_B);
assert.equal(c.state.warning, '');
assert.ok(statuses.every(status => status === 'ready'), `no loading state: ${statuses}`);
assert.equal(h.of('GET', `${ITEMS}/one`).length, 1, 'the open item is not read again');
});
test('refreshes asked for during a refresh collapse into one more read', async () => {
let release;
const held = new Promise(resolve => { release = resolve; });
let lists = 0;
const { c, h } = await loaded({ [`GET ${ITEMS}`]: async () => {
lists += 1;
if (lists === 2) await held;
return response(200, page(QUEUE));
} });
const first = refreshOf(c);
await settle();
refreshOf(c);
refreshOf(c);
release();
await first;
await settle();
assert.equal(h.of('GET', `${ITEMS}?`).length, 3, 'the load, the refresh and one collapsed re-read');
assert.equal(c.state.status, 'ready');
});
test('a refresh after the open item was decided elsewhere opens the next one and says so', async () => {
let queue = QUEUE;
const { c, h } = await loaded({
[`GET ${ITEMS}`]: () => response(200, page(queue)),
[`GET ${ITEMS}/three`]: () => response(200, detailOf(item('three'))),
});
queue = [item('three')];
await refreshOf(c);
await settle();
assert.equal(c.state.selectedId, 'three');
assert.equal(c.state.detailStatus, 'ready');
assert.equal(c.state.warning, 'one.png was decided elsewhere, so the next image is open.');
assert.equal(h.of('GET', `${ITEMS}/three`).length, 1);
});
test('a refresh to an empty queue clears the selection', async () => {
let queue = QUEUE;
const { c } = await loaded({ [`GET ${ITEMS}`]: () => response(200, page(queue)) });
queue = [];
await refreshOf(c);
await settle();
assert.equal(c.state.total, 0);
assert.deepEqual(c.state.items, []);
assert.equal(c.state.selectedId, null);
assert.equal(c.state.status, 'ready');
});
test('a failed refresh keeps the queue and says so', async () => {
let lists = 0;
const { c } = await loaded({ [`GET ${ITEMS}`]: () => { lists += 1; return lists === 1 ? response(200, page(QUEUE)) : fail(500, 'review-failed'); } });
await refreshOf(c);
await settle();
assert.equal(c.state.status, 'ready');
assert.deepEqual(c.state.items.map(entry => entry.id), ['one', 'two']);
assert.equal(c.state.warning, 'The review queue could not be read again. Reload the page to see the latest images.');
});
test('dispose during a decision releases its capture', async () => {
const { c, h } = await loaded({ [`POST ${ITEMS}/one/decision`]: () => new Promise(() => {}) });
const applying = c.apply();
await settle();
assert.equal(c.state.phase, 'applying');
c.dispose();
assert.equal(await applying, false);
await settle();
assert.deepEqual(h.of('DELETE', CAPTURE).map(call => call.url), [CAPTURE + 'capture-1']);
await refreshOf(c);
assert.equal(h.of('GET', `${ITEMS}?`).length, 1, 'a disposed controller reads nothing');
});
test('a change heard during the first read re-reads once after it, and a failed first read stays an error', async () => {
const { createReviewController } = await load();
let release;
let lists = 0;
const h = harness(routes({ [`GET ${ITEMS}`]: async () => {
lists += 1;
if (lists === 1) await new Promise(resolve => { release = resolve; });
return response(200, page(QUEUE));
} }));
const c = createReviewController({ api: h.api });
const first = c.load();
await settle();
refreshOf(c);
assert.equal(lists, 1, 'the first read is not interrupted');
release();
await first;
await settle();
assert.equal(lists, 2, 'one re-read after the first read');
assert.equal(c.state.status, 'ready');
const failing = harness(routes({ [`GET ${ITEMS}`]: () => fail(500, 'review-failed') }));
const broken = createReviewController({ api: failing.api });
const reading = broken.load();
refreshOf(broken);
await reading;
await settle();
assert.equal(broken.state.status, 'error');
assert.equal(failing.of('GET', `${ITEMS}?`).length, 1, 'Retry is the way back, not a re-read');
});
// F3 rulings: a live re-read keeps the loaded depth, and the stream's `ready` reads nothing when the first read is current.
/** A keyset pager over `source()`: pages of 40 with an index cursor, each answering `version()`. */
function pager(source, version = () => 0, { onPage } = {}) {
return call => {
const query = new URL(call.url, 'http://x').searchParams;
const cursor = query.get('cursor');
const all = source();
const from = cursor ? Number(cursor.slice(1)) : 0;
const answer = onPage?.(cursor);
if (answer) return answer;
const items = all.slice(from, from + 40);
const next = from + 40 < all.length ? `c${from + 40}` : null;
return response(200, page(items, { total: all.length, next_cursor: next, review_revision: version() }));
};
}
const many = (count, prefix = 'i') => Array.from({ length: count }, (_, index) => item(`${prefix}${String(index).padStart(3, '0')}`));
function syncOf(c, version) {
assert.equal(typeof c.sync, 'function', 'sync is not implemented');
return c.sync(version);
}
const version = revision => ({ library_epoch: EPOCH, review_revision: revision });
test('a refresh keeps the loaded depth: it re-reads as many items as were loaded', async () => {
let queue = many(100);
const { c, h } = await loaded({ [`GET ${ITEMS}`]: pager(() => queue) });
await c.loadMore();
assert.equal(c.state.items.length, 80);
const before = h.of('GET', `${ITEMS}?`).length;
queue = [...queue, item('late')];
await refreshOf(c);
await settle();
assert.equal(c.state.items.length, 80, 'the Load more page is kept');
assert.deepEqual(c.state.items.map(entry => entry.id), queue.slice(0, 80).map(entry => entry.id));
assert.equal(c.state.total, 101);
assert.equal(c.state.nextCursor, 'c80', 'Load more continues after the kept depth');
assert.deepEqual(h.of('GET', `${ITEMS}?`).slice(before).map(call => call.url),
[`${ITEMS}?limit=40`, `${ITEMS}?limit=40&cursor=c40`]);
assert.equal(c.state.selectedId, 'i000');
});
test('a refresh after decisions elsewhere fills the loaded depth from what is left', async () => {
let queue = many(85);
const { c } = await loaded({ [`GET ${ITEMS}`]: pager(() => queue) });
await c.loadMore();
await c.loadMore();
assert.equal(c.state.items.length, 85);
queue = queue.filter((_, index) => index % 10 !== 5);
await refreshOf(c);
await settle();
assert.equal(c.state.items.length, queue.length);
assert.equal(c.state.nextCursor, null);
});
test('a deep refresh whose next page meets a restore reads again from the first page', async () => {
let restored = false;
let queue = many(60);
const { c, h } = await loaded({ [`GET ${ITEMS}`]: pager(() => queue, () => 0, {
onPage: cursor => {
if (cursor && restored) { restored = false; return fail(409, 'storage-changed', 'The library was restored.'); }
return null;
},
}) });
await c.loadMore();
const before = h.of('GET', `${ITEMS}?`).length;
restored = true;
queue = many(60, 'r');
await refreshOf(c);
await settle();
assert.deepEqual(h.of('GET', `${ITEMS}?`).slice(before).map(call => call.url),
[`${ITEMS}?limit=40`, `${ITEMS}?limit=40&cursor=c40`, `${ITEMS}?limit=40`, `${ITEMS}?limit=40&cursor=c40`]);
assert.equal(c.state.items.length, 60);
assert.equal(c.state.items[0].id, 'r000');
assert.notEqual(c.state.warning, 'The review queue could not be read again. Reload the page to see the latest images.',
'a restore met mid-read is not a failed re-read');
assert.equal(c.state.status, 'ready');
});
test('ready naming the version the first read answered reads nothing more', async () => {
const { c, h } = await loaded({ [`GET ${ITEMS}`]: pager(() => QUEUE, () => 4) });
await syncOf(c, version(4));
await settle();
assert.equal(h.of('GET', `${ITEMS}?`).length, 1, 'one list read per visit');
assert.equal(h.of('GET', `${ITEMS}/one`).length, 1);
});
test('ready naming a newer version, another epoch or no version re-reads once', async () => {
const { c, h } = await loaded({ [`GET ${ITEMS}`]: pager(() => QUEUE, () => 4) });
await syncOf(c, version(5));
await settle();
assert.equal(h.of('GET', `${ITEMS}?`).length, 2);
await syncOf(c, { library_epoch: '22222222-2222-4222-8222-222222222222', review_revision: 4 });
await settle();
assert.equal(h.of('GET', `${ITEMS}?`).length, 3);
await syncOf(c, {});
await settle();
assert.equal(h.of('GET', `${ITEMS}?`).length, 4, 'an unknown version is read, as before versions existed');
});
test('ready heard during the first read re-reads only when that read is older', async () => {
const { createReviewController } = await load();
for (const [readRevision, readyRevision, reads] of [[4, 4, 1], [5, 4, 1], [4, 5, 2]]) {
let release;
let lists = 0;
const answer = pager(() => QUEUE, () => (lists === 1 ? readRevision : readyRevision));
const h = harness(routes({ [`GET ${ITEMS}`]: async call => {
lists += 1;
if (lists === 1) await new Promise(resolve => { release = resolve; });
return answer(call);
} }));
const c = createReviewController({ api: h.api });
const first = c.load();
await settle();
syncOf(c, version(readyRevision));
release();
await first;
await settle();
assert.equal(lists, reads, `first read r${readRevision}, ready r${readyRevision}`);
c.dispose();
}
});
test('a change heard during the first read still re-reads after a current ready', async () => {
const { createReviewController } = await load();
let release;
let lists = 0;
const h = harness(routes({ [`GET ${ITEMS}`]: async call => {
lists += 1;
if (lists === 1) await new Promise(resolve => { release = resolve; });
return pager(() => QUEUE, () => 4)(call);
} }));
const c = createReviewController({ api: h.api });
const first = c.load();
await settle();
syncOf(c, version(4));
refreshOf(c);
release();
await first;
await settle();
assert.equal(lists, 2);
});
// Final review round (I1, I2, M3–M5, M7).
test('I1: a change heard during a refused Apply is re-read once the phase is idle again', async () => {
let queue = [item('one')], releaseDecision;
const { c, h } = await loaded({
[`GET ${ITEMS}`]: () => response(200, page(queue, { review_revision: queue.length })),
[`POST ${ITEMS}/one/decision`]: () => new Promise(resolve => {
releaseDecision = () => resolve(fail(409, 'association-conflict', 'conflict'));
}),
});
const applying = c.apply();
await settle();
assert.equal(c.state.phase, 'applying');
queue = [item('one'), item('two')]; // the monitor records an item while the decision is in flight
c.refresh();
await settle();
assert.equal(h.of('GET', `${ITEMS}?`).length, 1, 'no read while the decision is in flight');
releaseDecision();
await applying;
await settle();
assert.equal(c.state.phase, 'idle');
assert.deepEqual(c.state.items.map(entry => entry.id), ['one', 'two']);
assert.equal(c.state.total, 2);
assert.equal(c.state.warning, 'conflict', 'the refusal stays the warning');
assert.equal(h.of('GET', `${ITEMS}?`).length, 2, 'exactly one owed re-read');
});
test('I1: a change heard during an Apply whose observe is refused is re-read too', async () => {
let queue = [item('one')], releaseObserve;
const { c } = await loaded({
[`GET ${ITEMS}`]: () => response(200, page(queue)),
[`POST ${OBSERVE}`]: () => new Promise(resolve => {
releaseObserve = () => resolve(fail(409, 'capture-capacity', 'busy'));
}),
});
const applying = c.apply();
await settle();
queue = [item('one'), item('two')];
c.refresh();
releaseObserve();
await applying;
await settle();
assert.deepEqual(c.state.items.map(entry => entry.id), ['one', 'two']);
});
const forty = Array.from({ length: 41 }, (_, i) => item(`i${String(i).padStart(2, '0')}`));
function deepRoutes(state) {
return {
[`GET ${ITEMS}`]: call => {
const cursor = new URL(call.url, 'http://x').searchParams.get('cursor');
if (cursor) {
if (state.moreGate) return state.moreGate.then(() => state.more ?? response(200, page([forty[40]], { total: 41 })));
return state.more ?? response(200, page([forty[40]], { total: 41 }));
}
const body = response(200, page(state.first, { total: state.first.length + 1, review_revision: state.revision ?? 1, next_cursor: 'cur' }));
return state.gate ? state.gate.then(() => state.fail ?? body) : body;
},
[`GET ${ITEMS}/i00`]: () => response(200, detailOf(forty[0])),
};
}
test('I2: Load more during a re-read waits for it and continues from the re-read queue', async () => {
const state = { first: forty.slice(0, 40) };
const { c } = await loaded(deepRoutes(state));
// Another tab decides i05; the re-read is slow and the user clicks Load more meanwhile.
let open;
state.first = forty.slice(0, 40).filter(entry => entry.id !== 'i05');
state.gate = new Promise(resolve => { open = resolve; });
const refreshing = c.refresh();
await settle();
const more = c.loadMore();
await settle();
assert.equal(c.state.more, 'loading', 'the click shows as loading while the re-read runs');
open();
await Promise.all([refreshing, more]);
await settle();
const ids = c.state.items.map(entry => entry.id);
assert.ok(!ids.includes('i05'), 'the re-read was applied');
assert.equal(ids.at(-1), 'i40', 'Load more appended the next page after the re-read');
assert.equal(c.state.more, 'idle');
});
test('I2: a re-read started during Load more takes over the click and reads one page deeper', async () => {
const state = { first: forty.slice(0, 40) };
const { c, h } = await loaded(deepRoutes(state));
let openMore;
state.moreGate = new Promise(resolve => { openMore = resolve; });
const more = c.loadMore();
await settle();
state.moreGate = null;
state.first = forty.slice(0, 40).filter(entry => entry.id !== 'i05');
await c.refresh();
openMore();
await more;
await settle();
const ids = c.state.items.map(entry => entry.id);
assert.ok(!ids.includes('i05'));
assert.equal(ids.at(-1), 'i40', 'the aborted Load more page is part of the re-read');
assert.equal(c.state.more, 'idle');
assert.equal(h.of('GET', `${ITEMS}?`).length, 4, 'load, the aborted more, and the re-read of two pages');
});
test('M7: a failed re-read that took over Load more clears its loading state', async () => {
const state = { first: forty.slice(0, 40) };
const { c } = await loaded(deepRoutes(state));
state.moreGate = new Promise(() => {});
c.loadMore();
await settle();
assert.equal(c.state.more, 'loading');
state.moreGate = null;
state.gate = Promise.resolve();
state.fail = fail(500, 'review-failed');
await c.refresh();
await settle();
assert.equal(c.state.more, 'idle', 'Load more can be clicked again');
assert.equal(c.state.warning, COPY_REFRESH_FAILED);
});
const COPY_REFRESH_FAILED = 'The review queue could not be read again. Reload the page to see the latest images.';
test('M3: Apply fences on the epoch of the detail it decides', async () => {
const OTHER = '22222222-2222-4222-8222-222222222222';
const { c, h } = await loaded({ [`POST ${OBSERVE}`]: () => response(200, { ...observed(), library_epoch: OTHER }) });
assert.equal(await c.apply(), false);
await settle();
assert.equal(h.of('POST', `${ITEMS}/one/decision`).length, 0, 'a restored storage sends no decision');
assert.deepEqual(h.of('DELETE', CAPTURE).map(call => call.url), [CAPTURE + 'capture-1']);
assert.equal(h.of('GET', `${ITEMS}?`).length, 2, 'the queue is read again');
assert.equal(h.of('GET', `${ITEMS}/one`).length, 2, 'the open item is read again under the new storage');
assert.equal(c.state.phase, 'idle');
});
test('M3: the decision carries the detail epoch', async () => {
const { c, h } = await loaded();
assert.equal(await c.apply(), true);
assert.equal(h.of('POST', `${ITEMS}/one/decision`)[0].body.expected_epoch, EPOCH);
});
test('M3: a quiet re-read under another epoch reads the open item again', async () => {
const OTHER = '22222222-2222-4222-8222-222222222222';
let epoch = EPOCH;
const { c, h } = await loaded({
[`GET ${ITEMS}`]: () => response(200, page(QUEUE, { library_epoch: epoch })),
[`GET ${ITEMS}/one`]: () => response(200, detailOf(QUEUE[0], { library_epoch: epoch })),
});
epoch = OTHER;
await c.refresh();
await settle();
assert.equal(h.of('GET', `${ITEMS}/one`).length, 2);
assert.equal(c.state.detail.library_epoch, OTHER);
});
test('M4: leaving during the observe releases the capture it then answers', async () => {
let answer;
const { c, h } = await loaded({ [`POST ${OBSERVE}`]: () => new Promise(resolve => { answer = resolve; }) });
const applying = c.apply();
await settle();
c.dispose();
answer(response(200, observed('late')));
assert.equal(await applying, false);
await settle();
assert.deepEqual(h.of('DELETE', CAPTURE).map(call => call.url), [CAPTURE + 'late']);
assert.equal(h.of('POST', `${ITEMS}/one/decision`).length, 0);
});
test('M5: an unanswered decision that landed finishes as recorded, not as decided elsewhere', async () => {
let queue = QUEUE;
const { c, h } = await loaded({
[`GET ${ITEMS}`]: () => response(200, page(queue)),
[`POST ${ITEMS}/one/decision`]: () => Promise.reject(new TypeError('Failed to fetch')),
});
await c.apply();
assert.equal(c.state.canRetry, true);
queue = [QUEUE[1]]; // the server did record it: its change arrives
await c.refresh();
await settle();
assert.equal(c.state.phase, 'done');
assert.equal(c.state.toast, 'Association accepted.');
assert.doesNotMatch(c.state.warning, /decided elsewhere/);
assert.equal(h.of('DELETE', CAPTURE).length, 0, 'the server used the capture');
});
+10 -33
View File
@@ -4,7 +4,7 @@ const source = import('../../web/js/workspace/state.mjs');
test('workspace routes resolve only supported deep links', async () => {
const { resolveRoute } = await source;
for (const route of ['overview', 'images', 'library', 'tags', 'inspect', 'system', 'activity', 'review']) {
for (const route of ['overview', 'images', 'library', 'tags', 'inspect', 'system', 'activity']) {
assert.equal(resolveRoute(`#/${route}`).id, route);
}
for (const hash of ['', '#', '#/unknown', '#/images/other', '#/<script>', '#/activity/abc', `#/activity/${'A'.repeat(32)}`]) {
@@ -24,7 +24,7 @@ test('tags is a native route titled Tags; only unbuilt routes hand off to legacy
}
assert.equal(routeTitle(resolveRoute('#/library/7')), 'Prompt library');
// System is native from S2 (#249 System): its undelivered sections hand off per section (D94), not the route.
for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity', 'review', 'system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id);
for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity', 'system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id);
});
test('system resolves its sections; #/overview stays the production hub', async () => {
@@ -157,10 +157,9 @@ test('positive library deep links preserve the selected identity without duplica
test('nav counts reject negative, non-integer and missing totals', async () => {
const { summarizeNavCounts } = await source;
const counts = summarizeNavCounts({images: {total: 24}, favorites: {total: 3}, library: {total: 0}, tags: {total: 7},
review: {total: 3}});
const counts = summarizeNavCounts({images: {total: 24}, favorites: {total: 3}, library: {total: 0}, tags: {total: 7}});
assert.ok(counts instanceof Map);
assert.deepEqual([...counts], [['images', 24], ['favorites', 3], ['library', 0], ['tags', 7], ['review', 3]]);
assert.deepEqual([...counts], [['images', 24], ['favorites', 3], ['library', 0], ['tags', 7]]);
assert.equal(counts.has('inspect'), false, 'Inspect a file has no count');
for (const invalid of [undefined, null, {}, {total: -1}, {total: 1.5}, {total: '3'}, {total: NaN},
{total: Infinity}, {total: Number.MAX_SAFE_INTEGER + 1}, {total: null}, [], 'x']) {
@@ -170,8 +169,7 @@ test('nav counts reject negative, non-integer and missing totals', async () => {
assert.equal(result.get('library'), 2);
assert.equal(result.get('tags'), null);
}
assert.deepEqual([...summarizeNavCounts()], [['images', null], ['favorites', null], ['library', null], ['tags', null],
['review', null]]);
assert.deepEqual([...summarizeNavCounts()], [['images', null], ['favorites', null], ['library', null], ['tags', null]]);
});
test('nav counts settle independently so a slow images read never delays the others', async () => {
@@ -187,14 +185,14 @@ test('nav counts settle independently so a slow images read never delays the oth
const settled = [];
const done = loadNavCounts(api, new AbortController().signal, (id, count) => settled.push([id, count]));
await new Promise(resolve => setTimeout(resolve, 0));
assert.deepEqual(settled.sort(), [['favorites', 2], ['library', 3], ['review', null], ['tags', null]]);
assert.deepEqual(settled.sort(), [['favorites', 2], ['library', 3], ['tags', null]]);
releaseImages({success: true, total: 40});
const counts = await done;
assert.deepEqual(settled.at(-1), ['images', 40]);
assert.deepEqual([...counts], [['images', 40], ['favorites', 2], ['library', 3], ['tags', null], ['review', null]]);
assert.deepEqual([...counts], [['images', 40], ['favorites', 2], ['library', 3], ['tags', null]]);
});
test('nav counts issue five bounded requests and drop only the failed count', async () => {
test('nav counts issue four bounded requests and drop only the failed count', async () => {
const { loadNavCounts } = await source;
const calls = [];
const signal = new AbortController().signal;
@@ -202,15 +200,13 @@ test('nav counts issue five bounded requests and drop only the failed count', as
calls.push([url, options.signal]);
if (url.startsWith('/prompt_manager/library')) throw new Error('Unavailable');
if (url.startsWith('/prompt_manager/vocabulary/tags')) return {success: true, total: 5};
if (url.startsWith('/prompt_manager/review/items')) return {success: true, total: 2};
return {success: true, total: 12};
}};
const counts = await loadNavCounts(api, signal);
assert.deepEqual([...counts], [['images', 12], ['favorites', 12], ['library', null], ['tags', 5], ['review', 2]]);
assert.deepEqual([...counts], [['images', 12], ['favorites', 12], ['library', null], ['tags', 5]]);
const urls = calls.map(([url]) => new URL(url, 'http://host'));
assert.deepEqual(urls.map(url => url.pathname),
['/prompt_manager/images/output', '/prompt_manager/images/output', '/prompt_manager/library', '/prompt_manager/vocabulary/tags',
'/prompt_manager/review/items']);
['/prompt_manager/images/output', '/prompt_manager/images/output', '/prompt_manager/library', '/prompt_manager/vocabulary/tags']);
for (const url of urls) assert.equal(url.searchParams.get('limit'), '1');
assert.equal(urls[0].searchParams.get('mode'), 'workspace');
assert.equal(urls[0].searchParams.has('favorites'), false);
@@ -292,22 +288,3 @@ test('saved-view links resolve to their id and re-render only when the id change
assert.deepEqual(seen,[['saved',3],['saved',4],['images',undefined]]);
dispose();
});
test('Needs review is a native SAVED VIEWS route titled Review, with its own count', async () => {
const { resolveRoute, isHandoff, routeTitle, routes, loadNavCounts, REVIEW_WRITE_COUNTS, NAV_COUNT_ROUTES } = await source;
const route = resolveRoute('#/review');
assert.equal(route.id, 'review');
assert.equal(route.label, 'Needs review');
assert.equal(routeTitle(route), 'Review');
assert.equal(isHandoff(route), false);
// No href: Overview lists no legacy card for it.
assert.equal(routes.find(item => item.id === 'review').href, undefined);
assert.ok(NAV_COUNT_ROUTES.includes('review'));
assert.deepEqual([...REVIEW_WRITE_COUNTS], ['review']);
assert.ok(Object.isFrozen(REVIEW_WRITE_COUNTS));
const calls = [];
const api = {get: async url => { calls.push(url); return {success: true, total: 4}; }};
const counts = await loadNavCounts(api, new AbortController().signal, () => {}, REVIEW_WRITE_COUNTS);
assert.deepEqual(calls, ['/prompt_manager/review/items?limit=1']);
assert.deepEqual([...counts], [['review', 4]]);
});
@@ -1,93 +0,0 @@
// System › thumbnail review (#249 System S5 Task 14): the concept's "Thumbnail maintenance" dialog (app.js 1433–1437,
// `thumbnail-review`) on the thumbnail job (S5 Task 12). Pure parts: the option values the job takes, the preview size
// check, the Image loading quality preselection (Q3), the D99 plan line and the start outcome.
const test = require('node:test');
const assert = require('node:assert/strict');
const reviewModule = import('../../web/js/workspace/thumbnail-review.mjs');
test('the concept options map to the job values in the concept order', async () => {
const { SCOPE_OPTIONS, QUALITY_OPTIONS, SIZE_LIMIT } = await reviewModule;
assert.deepEqual(SCOPE_OPTIONS, [['missing', 'Missing thumbnails only'], ['rebuild', 'Rebuild all (explicit opt-in)']]);
assert.deepEqual(QUALITY_OPTIONS, [['medium', 'Medium · optimized'], ['high', 'High · original'], ['low', 'Low · fastest']]);
assert.deepEqual(SIZE_LIMIT, { min: 128, max: 2048, value: 512 });
});
test('size outside 128–2048 reports validity: only whole pixels in range are a size', async () => {
const { previewSize } = await reviewModule;
assert.equal(previewSize('512'), 512);
assert.equal(previewSize('128'), 128);
assert.equal(previewSize('2048'), 2048);
for (const text of ['', ' ', '127', '2049', '4096', '512.5', '-512', 'abc', '1e3', '0x200']) {
assert.equal(previewSize(text), null, text);
}
});
test('Image loading quality preselects the dialog quality (Q3); anything else is Medium', async () => {
const { preselectedQuality } = await reviewModule;
assert.equal(preselectedQuality('low'), 'low');
assert.equal(preselectedQuality('high'), 'high');
assert.equal(preselectedQuality('medium'), 'medium');
assert.equal(preselectedQuality(undefined), 'medium');
assert.equal(preselectedQuality('original'), 'medium');
});
test('the plan line counts what a missing-only run generates and what is already up to date (D99)', async () => {
const { planText } = await reviewModule;
assert.equal(planText({ mode: 'missing', targets: 12, fresh: 30, remaining: 0 }), '12 previews to generate · 30 already up to date.');
assert.equal(planText({ mode: 'missing', targets: 1, fresh: 1, remaining: 0 }), '1 preview to generate · 1 already up to date.');
assert.equal(planText({ mode: 'missing', targets: 0, fresh: 0, remaining: 0 }), '0 previews to generate · 0 already up to date.');
assert.equal(planText({ mode: 'missing', targets: 10000, fresh: 5, remaining: 2500 }),
'10,000 previews to generate · 5 already up to date · 2,500 more on the next run.');
});
test('Rebuild all says every preview is regenerated (D99)', async () => {
const { planText } = await reviewModule;
assert.equal(planText({ mode: 'rebuild', targets: 42, fresh: 30, remaining: 0 }), 'Every preview is regenerated: 42 files.');
assert.equal(planText({ mode: 'rebuild', targets: 1, fresh: 1, remaining: 0 }), 'Every preview is regenerated: 1 file.');
assert.equal(planText({ mode: 'rebuild', targets: 10000, fresh: 9000, remaining: 3 }),
'Every preview is regenerated: 10,000 files · 3 more on the next run.');
});
test('an answer without counts is a failed plan, never zeros', async () => {
const { planText, PLAN_FAILED } = await reviewModule;
assert.equal(planText(null), PLAN_FAILED);
assert.equal(planText({ mode: 'missing', targets: 2 }), PLAN_FAILED);
assert.equal(planText({ mode: 'missing', targets: -1, fresh: 0, remaining: 0 }), PLAN_FAILED);
assert.equal(planText({ mode: 'other', targets: 1, fresh: 0, remaining: 0 }), PLAN_FAILED);
});
const scopeOf = () => ({ signal: new AbortController().signal });
test('Start starts the thumbnail job with the chosen options and opens it in Activity', async () => {
const { startThumbnails } = await reviewModule;
const calls = [];
const navigated = [];
const store = { start: async (kind, body) => { calls.push([kind, body]); return { job: { id: 'a'.repeat(32) } }; } };
await startThumbnails({ store, scope: scopeOf(), options: { mode: 'rebuild', size: 1024, quality: 'high' },
navigate: hash => navigated.push(hash), notify: () => assert.fail('no toast of its own') });
assert.deepEqual(calls, [['thumbnails', { mode: 'rebuild', size: 1024, quality: 'high' }]]);
assert.deepEqual(navigated, [`#/activity/${'a'.repeat(32)}`]);
});
test('a refused start shows the busy link toast and stays', async () => {
const { startThumbnails } = await reviewModule;
const toasts = [];
const store = { start: async () => ({ busy: { id: 'b'.repeat(32), title: 'Scan for metadata' } }) };
await startThumbnails({ store, scope: scopeOf(), options: { mode: 'missing', size: 512, quality: 'medium' },
navigate: () => assert.fail('stays'), notify: message => toasts.push(message) });
assert.deepEqual(toasts, [['Another job is running: ', { text: 'Scan for metadata', href: `#/activity/${'b'.repeat(32)}` }, '.']]);
});
test('a failed start says so; a start after the section left says nothing', async () => {
const { startThumbnails, START_FAILED } = await reviewModule;
const toasts = [];
const store = { start: async () => { throw new Error('offline'); } };
await startThumbnails({ store, scope: scopeOf(), options: { mode: 'missing', size: 512, quality: 'medium' },
navigate: () => assert.fail('stays'), notify: message => toasts.push(message) });
assert.deepEqual(toasts, [START_FAILED]);
const gone = new AbortController();
gone.abort();
await startThumbnails({ store, scope: { signal: gone.signal }, options: { mode: 'missing', size: 512, quality: 'medium' },
navigate: () => assert.fail('stays'), notify: () => assert.fail('silent') });
});
+25 -7
View File
@@ -413,11 +413,11 @@ class TestThumbnailProgressStream(ThumbnailTestCase):
lock = threading.Lock()
original = images_module._write_image_thumbnail
def recording(src, dst, size, *args):
def recording(src, dst, size):
with lock:
seen.add(threading.current_thread().name)
time.sleep(0.05)
return original(src, dst, size, *args)
return original(src, dst, size)
images_module._write_image_thumbnail = recording
self.addCleanup(setattr, images_module, "_write_image_thumbnail", original)
@@ -1939,11 +1939,7 @@ class TestGalleryRescan(ImageAPITestCase):
class TestOutputScanHelpers(ImageAPITestCase):
"""Stable ids for filesystem entries and no symlink reads.
The thumbnail stream no longer stops when its client goes away: see
``tests/test_thumbnail_jobs.py`` (the work is a job that outlives it).
"""
"""Stable ids for filesystem entries, no symlink reads, stop on disconnect."""
def test_output_entry_id_is_a_stable_digest_of_the_path(self):
import hashlib
@@ -1968,6 +1964,28 @@ class TestOutputScanHelpers(ImageAPITestCase):
names = sorted(p.name for p in images_module._iter_media_files(output, 100))
self.assertEqual(names, ["real.png"])
async def test_thumbnail_stream_stops_when_the_client_disconnects(self):
generated = []
def fake_generate(src, dst, size, is_video):
generated.append(src)
return {
"action": "generated",
"file": src.name,
"dir": src.parent.name,
"type": "image",
}
self.api._generate_one = fake_generate
class GoneResponse:
async def write(self, data):
raise ConnectionResetError("client went away")
targets = [(Path(f"/x/{i}.png"), Path(f"/x/t{i}.jpg"), False) for i in range(3)]
await self.api._stream_thumbnails(GoneResponse(), targets, (256, 256))
self.assertEqual(generated, [])
class TestThumbnailStartupStatus(ThumbnailTestCase):
"""Startup counts and generation cover the same configured collection."""
+11 -72
View File
@@ -1,11 +1,12 @@
"""Needs review decisions and queue reads (#249 E3).
The queue lists pending items oldest first behind an HMAC cursor bound to the
storage epoch; ``total`` is the exact pending count. A decision is explicit:
Apply links the chosen sampler's pair only under the ingester's compatible
rules and only for the current, re-hashed bytes of the reviewed file, in one
transaction with its decision row; Leave unresolved records the decision and
links nothing. Responses never carry a server path.
storage epoch and the review index revision; ``total`` is the exact pending
count. A decision is explicit: Apply links the chosen sampler's pair only
under the ingester's compatible rules and only for the current, re-hashed
bytes of the reviewed file, in one transaction with its decision row; Leave
unresolved records the decision and links nothing. Responses never carry a
server path.
"""
import json
@@ -198,22 +199,12 @@ class TestQueueReads(ReviewRoutesTestCase):
"unavailable",
"candidate_count",
"limited",
"url",
"thumbnail_url",
"is_video",
},
)
self.assertEqual(
(item["state"], item["revision"], item["filename"]),
("pending", 0, "f0.png"),
)
# The card's media URL, formatted like the Images listing's (no stat).
scope = item["target"]["root_scope"]
self.assertEqual(
item["url"],
f"/prompt_manager/images/serve/f0.png?root=0&root_scope={scope}",
)
self.assertIs(item["is_video"], False)
self.assertEqual(item["candidate_count"], 2)
self.assertIs(item["unavailable"], False)
self.assertEqual(item["target"]["path"], "f0.png")
@@ -251,7 +242,7 @@ class TestQueueReads(ReviewRoutesTestCase):
status, data = await self._json("GET", ITEMS_URL + "?cursor=" + "x" * 5000)
self.assertEqual(status, 400)
async def test_cursor_survives_queue_changes_and_is_invalidated_by_restore(self):
async def test_cursor_invalidated_by_decision_and_restore(self):
self._require_routes()
for index in range(3):
self._record(self._ambiguous(f"f{index}.png", color=(index, 0, 0)))
@@ -273,43 +264,16 @@ class TestQueueReads(ReviewRoutesTestCase):
status, data = await self._json("GET", f"{ITEMS_URL}?cursor={cursor}")
self.assertEqual((status, data["code"]), (409, "review-conflict"), data)
self.assertEqual(data["library_epoch"], first["library_epoch"])
# The cursor is bound to the storage epoch only (M3): a decision and a
# new item (the monitor records one per generation) leave "load more"
# working, because the (created_at, id) keyset is stable under both.
# A decision moves the review index revision: the old cursor is stale.
item, detail, observed = await self._open()
self.assertEqual(item["id"], first["items"][0]["id"])
status, data = await self._decide(
item["id"], {**self._body(detail, observed, "unresolved")}
)
self.assertEqual(status, 200, data)
self._record(self._ambiguous("late.png", color=(9, 9, 9)))
rows = self._items()
status, data = await self._json("GET", f"{ITEMS_URL}?limit=1&cursor={cursor}")
self.assertEqual(status, 200, data)
self.assertEqual(data["total"], 3)
self.assertEqual([i["filename"] for i in data["items"]], ["f1.png"])
seen = ["f1.png"]
while data["next_cursor"]:
data = await self._list(f"?limit=1&cursor={data['next_cursor']}")
seen.extend(i["filename"] for i in data["items"])
self.assertEqual(seen, ["f1.png", "f2.png", "late.png"])
self.assertEqual(len(rows), 4)
# A decision on an item after the cursor drops it from later pages.
second = await self._list("?limit=1")
self.assertEqual([i["filename"] for i in second["items"]], ["f1.png"])
conn = self.api.db.model.get_connection()
with conn:
conn.execute(
"UPDATE media_review_items SET state='selected', revision=1 "
"WHERE id=(SELECT i.id FROM media_review_items i "
"WHERE i.spelled_path LIKE ?)",
("%f2.png",),
)
data = await self._list(f"?limit=1&cursor={second['next_cursor']}")
self.assertEqual(
([i["filename"] for i in data["items"]], data["total"]),
(["late.png"], 2),
)
self.assertEqual(status, 409, data)
self.assertEqual(data["code"], "review-conflict")
self.assertEqual(data["total"], 2)
fresh = (await self._list("?limit=1"))["next_cursor"]
conn = self.api.db.model.get_connection()
with conn: # a restore replaces the storage epoch
@@ -390,35 +354,10 @@ class TestQueueReads(ReviewRoutesTestCase):
(item["unavailable"], item["target"], item["filename"]),
(True, None, "two.png"),
)
self.assertEqual((item["url"], item["is_video"]), (None, False))
self.assertIsNone(item["thumbnail_url"])
detail = await self._detail(item["id"])
self.assertIs(detail["unavailable"], True)
self._no_paths()
async def test_queue_cards_use_the_images_listing_thumbnail(self):
# M10: a known thumbnail is the card's media, as on the Images page;
# without one the card falls back to the original.
self._require_routes()
from py.api.images import _output_image_entry
path = self._ambiguous("sub/f0.png")
self._record(path)
item = (await self._list())["items"][0]
self.assertIsNone(item["thumbnail_url"])
thumb = self.output_dir / "thumbnails" / "sub" / "f0_thumb.png"
thumb.parent.mkdir(parents=True)
write_png(thumb, prompt=ambiguous_graph("cat", "dog"), color=(1, 2, 3))
item = (await self._list())["items"][0]
scope = item["target"]["root_scope"]
listed = _output_image_entry(path, self.output_dir, 0, root_scope=scope)
self.assertIsNotNone(listed["thumbnail_url"])
self.assertEqual(item["thumbnail_url"], listed["thumbnail_url"])
self.assertEqual(item["url"], listed["url"])
detail = await self._detail(item["id"])
self.assertEqual(detail["thumbnail_url"], listed["thumbnail_url"])
self._no_paths()
def test_target_formatting_keeps_the_recorded_spelling_on_windows(self):
self._require_routes()
spelled = "C:\\Out\\Sub\\Image_0001.PNG"
-364
View File
@@ -1,364 +0,0 @@
"""Needs review change notices (#249 PR F2).
``GET /prompt_manager/review/events`` is a server-sent event stream: one
``ready`` event when it opens, then one ``change`` event after every
committed queue change (an item recorded by the scan or the monitor, a
spelling refresh, a decision from any tab) and after a restore. An open
Needs review page re-reads its queue on ``change``; nothing polls. A refused
decision and a duplicate recording change nothing and send nothing.
"""
import asyncio
import json
import sqlite3
import threading
try:
from tests.test_api_review import ReviewRoutesTestCase
from tests.test_review import unlinkable_graph, write_png
except ImportError: # discovered with tests/ as the top-level directory
from test_api_review import ReviewRoutesTestCase
from test_review import unlinkable_graph, write_png
try: # guarded so a missing module fails assertions, not test loading
from database import review_changes
except ImportError:
review_changes = None
try:
from py.api import review_events
except ImportError:
review_events = None
EVENTS_URL = "/prompt_manager/review/events"
# A safety bound for one read; every wait below ends on an event first.
READ_SECONDS = 10
class ReviewEventsTestCase(ReviewRoutesTestCase):
def _require_events(self):
self.assertIsNotNone(review_changes, "database.review_changes is missing")
self.assertIsNotNone(review_events, "py.api.review_events is missing")
async def _stream(self):
resp = await self.client.get(EVENTS_URL)
self.assertEqual(resp.status, 200)
self.assertEqual(resp.headers["Content-Type"], "text/event-stream")
self.assertEqual(resp.headers["Cache-Control"], "no-store")
self.addAsyncCleanup(self._close, resp)
self.assertEqual(await self._event(resp), "ready")
return resp
@staticmethod
async def _close(resp):
resp.close()
async def _event(self, resp):
"""The next event's name (keepalive comments are skipped)."""
return (await self._event_with_data(resp))[0]
async def _event_with_data(self, resp):
"""The next event's name and its parsed ``data`` line."""
name, data = None, None
while True:
line = await asyncio.wait_for(resp.content.readline(), READ_SECONDS)
self.assertTrue(line, "the stream ended")
text = line.decode("utf-8").rstrip("\n")
if text.startswith("event: "):
name = text[len("event: ") :]
elif text.startswith("data: "):
data = json.loads(text[len("data: ") :])
elif text == "" and name is not None:
return name, data
async def _ready_version(self):
resp = await self.client.get(EVENTS_URL)
self.assertEqual(resp.status, 200)
self.addAsyncCleanup(self._close, resp)
name, data = await self._event_with_data(resp)
self.assertEqual(name, "ready")
return data
def _queueable(self, name="two.png", color=(255, 0, 0)):
"""A file the producers queue: ambiguous, and nothing links it."""
path = self.output_dir / name
path.parent.mkdir(parents=True, exist_ok=True)
write_png(path, prompt=unlinkable_graph("cat", "dog"), color=color)
return path
async def _record_in_thread(self, path):
"""Record as the monitor does: from a worker thread, not the loop."""
self._roots()
outcomes = []
thread = threading.Thread(
target=lambda: outcomes.append(self.api.record_review_file(str(path)))
)
thread.start()
await asyncio.get_running_loop().run_in_executor(None, thread.join)
return outcomes[0]
class TestReviewEvents(ReviewEventsTestCase):
async def test_monitor_recording_sends_one_change(self):
self._require_events()
stream = await self._stream()
self.assertEqual(await self._record_in_thread(self._queueable()), "recorded")
self.assertEqual(await self._event(stream), "change")
async def test_duplicate_recording_and_refused_decision_send_nothing(self):
# A spy on the notice itself: a wrong notice would otherwise pass as
# the next real change's event.
self._require_events()
from unittest.mock import patch
from database import review as review_db
from database import review_decisions
calls = []
original = review_changes.notify_review_change
def spy(db_path):
calls.append(db_path)
original(db_path)
path = self._queueable("two.png")
self._record(path)
with (
patch.object(review_db, "notify_review_change", spy),
patch.object(review_decisions, "notify_review_change", spy),
):
# The same bytes again: nothing changes.
self.assertEqual(await self._record_in_thread(path), "duplicate")
self.assertEqual(calls, [], "a duplicate recording sent a notice")
item, detail, observed = await self._open()
# A stale revision: refused with zero writes.
body = {
**self._body(detail, observed, "unresolved"),
"expected_revision": 9,
}
status, data = await self._decide(item["id"], body)
self.assertEqual(status, 409, data)
self.assertEqual(calls, [], "a refused decision sent a notice")
self._record(self._ambiguous("other.png", color=(0, 0, 9)))
self.assertEqual(len(calls), 1, "a real recording sends one notice")
self.assertEqual(len(self._items()), 2)
async def test_a_burst_of_changes_coalesces_into_one_more_event(self):
# M9: a busy scan records file after file; an open page hears the
# first at once and the rest as one trailing ``change``.
self._require_events()
from unittest.mock import patch
path = self.api.db.model.db_path
with (
patch.object(review_events, "COALESCE_SECONDS", 0.3),
patch.object(review_events, "HEARTBEAT_SECONDS", 0.6),
):
stream = await self._stream()
review_changes.notify_review_change(path)
self.assertEqual(await self._event(stream), "change")
for _ in range(5):
review_changes.notify_review_change(path)
await asyncio.sleep(0.02)
# Every change event up to the next keepalive: exactly one.
changes = 0
while True:
line = await asyncio.wait_for(stream.content.readline(), READ_SECONDS)
self.assertTrue(line, "the stream ended")
if line == b": keepalive\n":
break
changes += line == b"event: change\n"
self.assertEqual(changes, 1)
async def test_decision_from_any_tab_sends_a_change(self):
self._require_events()
self._record(self._ambiguous())
stream = await self._stream()
item, detail, observed = await self._open()
status, data = await self._decide(
item["id"], self._body(detail, observed, "unresolved")
)
self.assertEqual(status, 200, data)
self.assertEqual(await self._event(stream), "change")
async def test_restore_sends_a_change(self):
self._require_events()
stream = await self._stream()
from database.execution_history import notify_history_restore
notify_history_restore(self.api.db.model.db_path)
self.assertEqual(await self._event(stream), "change")
async def test_scan_recording_sends_a_change(self):
self._require_events()
self._roots()
self._queueable("sub/two.png")
stream = await self._stream()
job = await self._scan("all")
self.assertEqual(job["scope"]["review_items"], 1)
self.assertEqual(await self._event(stream), "change")
async def test_a_linked_file_leaving_the_queue_sends_a_change(self):
"""The queue heals (#284 maintainer test): a pending item whose file is
linked now leaves the queue, and the open page hears about it."""
self._require_events()
self._roots()
path = self._ambiguous("two.png")
self._record(path) # as the pre-fix scan queued fallback-linked files
stream = await self._stream()
job = await self._scan("new")
self.assertEqual(job["scope"]["review_settled"], 1)
self.assertEqual(self._items(), [])
self.assertEqual(await self._event(stream), "change")
async def test_ready_carries_the_version_a_list_read_answers(self):
# F3 (no double read per visit): the page skips the re-read on
# ``ready`` when its first read already answered this version.
self._require_events()
self._record(self._ambiguous())
page = await self._list()
self.assertIsInstance(page.get("review_revision"), int, page)
version = await self._ready_version()
self.assertEqual(
version,
{
"library_epoch": page["library_epoch"],
"review_revision": page["review_revision"],
},
)
async def test_a_queue_change_moves_the_version_ready_carries(self):
self._require_events()
before = await self._list()
self._record(self._ambiguous())
after = await self._list()
self.assertIn("review_revision", after)
self.assertGreater(after["review_revision"], before["review_revision"])
version = await self._ready_version()
self.assertEqual(version["review_revision"], after["review_revision"])
self.assertNotEqual(version["review_revision"], before["review_revision"])
async def test_ready_names_no_version_when_it_cannot_be_read(self):
# The page then re-reads, as it did before versions existed.
self._require_events()
def broken():
raise sqlite3.DatabaseError("locked")
self.api.db.review_queue_version = broken
self.assertEqual(await self._ready_version(), {})
async def test_a_stream_counts_as_open_once_it_has_sent_ready(self):
# The browser fixture waits on this count before it changes the
# queue, so a change after it is a ``change`` event and never part of
# the version ``ready`` named.
self._require_events()
reading, release = asyncio.Event(), asyncio.Event()
original = self.api._queue_version
async def held():
reading.set()
await release.wait()
return await original()
self.api._queue_version = held
opening = asyncio.ensure_future(self._ready_version())
await asyncio.wait_for(reading.wait(), READ_SECONDS)
self.assertEqual(self.api.review_stream_count(), 0)
release.set()
await asyncio.wait_for(opening, READ_SECONDS)
self.assertEqual(self.api.review_stream_count(), 1)
async def test_change_events_carry_no_data(self):
self._require_events()
stream = await self._stream()
self._record(self._ambiguous())
self.assertEqual(await self._event_with_data(stream), ("change", {}))
async def test_closing_the_stream_unsubscribes(self):
self._require_events()
path = self.api.db.model.db_path
stream = await self._stream()
self.assertEqual(review_changes.observer_count(path), 1)
self.assertEqual(self.api.review_stream_count(), 1)
stream.close()
# The server notices the closed socket at its next write: a change.
self._record(self._ambiguous())
for _ in range(100):
if not self.api.review_stream_count():
break
await asyncio.sleep(0.02)
self.assertEqual(self.api.review_stream_count(), 0)
self.assertEqual(review_changes.observer_count(path), 0)
async def test_shutdown_ends_every_stream(self):
self._require_events()
stream = await self._stream()
await self.api.close_review_events(None)
line = await asyncio.wait_for(stream.content.read(), READ_SECONDS)
self.assertEqual(line, b"")
self.assertEqual(self.api.review_stream_count(), 0)
# A stream opened after shutdown ends at once.
resp = await self.client.get(EVENTS_URL)
self.assertEqual(resp.status, 503)
resp.close()
async def test_keepalive_comment_is_skipped_by_clients(self):
self._require_events()
with self.patch_heartbeat(0.05):
stream = await self._stream()
line = await asyncio.wait_for(stream.content.readline(), READ_SECONDS)
self.assertEqual(line, b": keepalive\n")
def patch_heartbeat(self, seconds):
from unittest.mock import patch
return patch.object(review_events, "HEARTBEAT_SECONDS", seconds)
class TestReviewChangeRegistry(ReviewEventsTestCase):
def test_a_failing_observer_never_fails_a_write(self):
self._require_events()
path = self.api.db.model.db_path
seen = []
def broken():
raise RuntimeError("observer failed")
unsubscribe_broken = review_changes.subscribe_review_changes(path, broken)
unsubscribe = review_changes.subscribe_review_changes(
path, lambda: seen.append(1)
)
try:
with self.assertLogs(
"prompt_manager.database.review_changes", level="WARNING"
):
self._record(self._ambiguous())
self.assertEqual(seen, [1])
finally:
unsubscribe_broken()
unsubscribe()
self.assertEqual(review_changes.observer_count(path), 0)
def test_notice_comes_after_the_commit(self):
self._require_events()
path = self.api.db.model.db_path
seen = []
def committed():
# A separate connection sees only committed rows.
other = sqlite3.connect(path)
try:
seen.append(
other.execute("SELECT COUNT(*) FROM media_review_items").fetchone()[
0
]
)
finally:
other.close()
unsubscribe = review_changes.subscribe_review_changes(path, committed)
try:
self._record(self._ambiguous())
finally:
unsubscribe()
self.assertEqual(seen, [1])
-75
View File
@@ -101,35 +101,6 @@ class TestJobFanOut(unittest.IsolatedAsyncioTestCase):
self.assertEqual(await _collect(job), [{"type": "complete"}])
async def test_lead_comes_first_and_nothing_published_meanwhile_is_lost(self):
"""S5 I1: the ``job`` frame leads, with the subscription already in place."""
job = Job("thumbnails", "Thumbnail generation")
job.publish({"type": "status"})
lead = {"type": "job"}
seen = []
async with contextlib.aclosing(job.events(lead=lead)) as events:
async for payload in events:
seen.append(payload)
if payload is lead:
# Published while the consumer is still sending the lead.
job.publish({"type": "progress"})
job.publish({"type": "complete"})
job.finish()
self.assertEqual(
seen,
[lead, {"type": "status"}, {"type": "progress"}, {"type": "complete"}],
)
async def test_lead_on_a_finished_job_then_its_last_event(self):
job = Job("thumbnails", "Thumbnail generation")
job.publish({"type": "complete"})
job.finish()
seen = []
async with contextlib.aclosing(job.events(lead={"type": "job"})) as events:
async for payload in events:
seen.append(payload)
self.assertEqual(seen, [{"type": "job"}, {"type": "complete"}])
async def test_departed_subscriber_is_forgotten(self):
job = Job("scan", "Scan")
job.publish({"type": "progress", "progress": 1})
@@ -180,30 +151,6 @@ class TestJobFanOut(unittest.IsolatedAsyncioTestCase):
class TestJobRegistry(unittest.IsolatedAsyncioTestCase):
async def test_a_compatible_kind_never_blocks(self):
"""S5 M7: a caller may name running kinds that cannot conflict with it."""
registry = JobRegistry()
gate = asyncio.Event()
thumbs = registry.start(
"thumbnails",
"Thumbnail generation",
admission="guarded",
runner=_gated_runner(gate),
)
self.assertIs(registry.blocker("probe", admission="records"), thumbs)
self.assertIsNone(
registry.blocker("probe", admission="records", compatible=("thumbnails",))
)
backup = registry.start(
"backup", "Backup", admission="guarded", runner=_gated_runner(gate)
)
self.assertIs(
registry.blocker("probe", admission="records", compatible=("thumbnails",)),
backup,
)
gate.set()
await asyncio.wait_for(asyncio.gather(thumbs.task, backup.task), 2)
async def test_start_lists_and_finishes_a_job(self):
registry = JobRegistry()
gate = asyncio.Event()
@@ -474,28 +421,6 @@ class TestJobOutcomesAndLog(unittest.TestCase):
SCAN_GROUP_NAMES, ("added", "known", "review", "no_metadata", "errors")
)
def test_record_counted_counts_exactly_lists_capped_and_logs_nothing(self):
job = Job("thumbnails", "Thumbnail generation", groups=("skipped",))
rows = [
{"path": os.path.abspath(f"out{os.sep}f{i}.png"), "root": "output"}
for i in range(MAX_OUTCOME_ROWS + 5)
]
job.record_counted("skipped", 10_000, rows)
job.record_counted("skipped", 1, [{"path": "late.png", "root_index": True}])
self.assertEqual(job.counters["skipped"], 10_001)
self.assertEqual(len(job.outcomes["skipped"]), MAX_OUTCOME_ROWS)
self.assertEqual(job.outcomes["skipped"][0]["path"], "f0.png")
self.assertIsNone(job.outcomes["skipped"][0]["root_index"])
self.assertEqual(job.log, [])
with self.assertRaises(ValueError):
job.record_counted("bogus", 1)
def test_thumbnail_job_groups(self):
self.assertEqual(
jobs_module.THUMBNAIL_GROUP_NAMES, ("generated", "skipped", "errors")
)
self.assertEqual(jobs_module.THUMBNAILS_JOB_KIND, "thumbnails")
def test_image_tag_job_kind_and_groups(self):
# #249 D1: bulk Add tags / Auto-tag, counter-tile order.
self.assertEqual(getattr(jobs_module, "IMAGE_TAG_JOB_KIND", None), "image-tags")
+2 -40
View File
@@ -859,9 +859,7 @@ class TestMonitorRecording(ReviewTestCase):
While a ``records`` or ``guarded`` job runs, the monitor's recording
records nothing and answers ``deferred`` (its fallback link stands and
a later scan reads the ambiguous file again); a ``read`` job does not
refuse it, and it records once the blocking job has ended. Thumbnail
work (the job and the gallery's holds) is ``guarded`` but writes only
previews, never records, so it does not defer the recording (S5 M7).
refuse it, and it records once the blocking job has ended.
"""
self._require_module()
from py.api.jobs import RESTORE_JOB_KIND, THUMBNAILS_JOB_KIND
@@ -872,7 +870,7 @@ class TestMonitorRecording(ReviewTestCase):
for kind, admission in (
(RESTORE_JOB_KIND, "records"),
("backup", "guarded"),
(THUMBNAILS_JOB_KIND, "guarded"),
):
with self.subTest(admission=admission):
async with self.api._jobs.hold(
@@ -890,43 +888,7 @@ class TestMonitorRecording(ReviewTestCase):
)
async with self.api._jobs.hold("probe-read", "Read probe", admission="read"):
async with self.api._jobs.hold(
THUMBNAILS_JOB_KIND,
"Thumbnail clear (gallery page)",
admission="guarded",
):
outcome = await asyncio.to_thread(
self.api.record_review_file, str(path)
)
self.assertEqual(outcome, "recorded")
self.assertEqual(len(self._items()), 1)
async def test_a_running_thumbnail_job_does_not_defer_monitor_recording(self):
"""S5 M7: a long rebuild never holds Needs review back (no records)."""
self._require_module()
from py.api.jobs import THUMBNAILS_JOB_KIND
self._roots()
path = self.output_dir / "beside.png"
write_png(path, prompt=unlinkable_graph("cat", "dog"))
gate = asyncio.Event()
async def runner(job):
await gate.wait()
job = self.api._jobs.start(
THUMBNAILS_JOB_KIND,
"Thumbnail generation",
admission="guarded",
cancellable=True,
scope={},
runner=runner,
)
try:
outcome = await asyncio.to_thread(self.api.record_review_file, str(path))
finally:
gate.set()
await asyncio.wait_for(job.task, 5)
self.assertEqual(outcome, "recorded")
self.assertEqual(len(self._items()), 1)
-846
View File
@@ -1,846 +0,0 @@
"""Thumbnail generation as an Activity job (#249 System S5, Task 12).
``GET /prompt_manager/images/thumbnails/plan`` counts what a run would do and
writes nothing. ``POST /prompt_manager/jobs/thumbnails`` starts the
``thumbnails`` job (admission ``guarded``, cancellable at a file boundary).
The legacy gallery stream ``POST /images/generate-thumbnails/progress`` starts
or attaches to that job and streams its unchanged events from it, so a client
that goes away no longer stops the work.
Ordering is driven by threading/asyncio events, never by timers.
"""
import asyncio
import hashlib
import os
import sys
import threading
import unittest
from pathlib import Path
from unittest.mock import patch
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
try:
from tests.test_api_images import ThumbnailTestCase, make_png, parse_sse
except ImportError: # discovered with tests/ as the top-level directory
from test_api_images import ThumbnailTestCase, make_png, parse_sse
from PIL import Image # noqa: E402
import py.api.images as images_module # noqa: E402
from py.api.thumbnail_jobs import thumbnail_options # noqa: E402
from py.config import PromptManagerConfig # noqa: E402
PLAN_URL = "/prompt_manager/images/thumbnails/plan"
JOB_URL = "/prompt_manager/jobs/thumbnails"
def _tree_state(root):
"""{relative path: (size, mtime_ns, sha256)} of every file under ``root``."""
state = {}
for folder, _dirs, files in os.walk(root):
for name in files:
path = Path(folder) / name
data = path.read_bytes()
stat = path.stat()
state[path.relative_to(root).as_posix()] = (
stat.st_size,
stat.st_mtime_ns,
hashlib.sha256(data).hexdigest(),
)
return state
class ThumbnailJobCase(ThumbnailTestCase):
"""Helpers: start, gate and wait for thumbnail jobs."""
async def _start(self, **body):
resp = await self.client.post(JOB_URL, json=body)
return resp.status, await resp.json()
async def _finished(self, job_id):
job = self.api._jobs.get(job_id)
await asyncio.wait_for(job.task, 10)
return job
def _gate_generation(self):
"""Block every file in the worker until released; returns (entered, release)."""
entered = threading.Event()
release = threading.Event()
original = self.api._generate_one
def gated(*args, **kwargs):
entered.set()
release.wait(10)
return original(*args, **kwargs)
self.api._generate_one = gated
self.addCleanup(release.set)
return entered, release
async def _wait(self, event):
loop = asyncio.get_running_loop()
self.assertTrue(await loop.run_in_executor(None, event.wait, 10))
@staticmethod
def _watch_subscribers(job):
"""(subscribed, left): threading events for a stream joining or leaving."""
subscribed, left = threading.Event(), threading.Event()
class Watched(set):
def add(self, item):
super().add(item)
subscribed.set()
def discard(self, item):
super().discard(item)
left.set()
job._subscribers = Watched(job._subscribers)
return subscribed, left
def _status_lines(self):
"""Every status line any job is given, in order."""
from py.api.jobs import Job
lines = []
original = Job.update
def update(job, **kwargs):
if kwargs.get("status_line") is not None:
lines.append(kwargs["status_line"])
return original(job, **kwargs)
patcher = patch.object(Job, "update", update)
patcher.start()
self.addCleanup(patcher.stop)
return lines
def _one_worker(self):
original = PromptManagerConfig.WORKER_THREADS
PromptManagerConfig.WORKER_THREADS = 1 # batches of 4
self.addCleanup(setattr, PromptManagerConfig, "WORKER_THREADS", original)
def _run_scan(self):
gate = asyncio.Event()
async def runner(job):
await gate.wait()
job = self.api._jobs.start(
"scan", "Scan for metadata", admission="records", scope={}, runner=runner
)
async def release():
gate.set()
await asyncio.wait_for(job.task, 5)
self.addAsyncCleanup(release)
return job
class TestThumbnailPlan(ThumbnailJobCase):
async def test_plan_is_read_only_and_counts_cap(self):
for name in ("a", "b", "c"):
make_png(self.output_dir / f"{name}.png", size=(64, 64))
fresh = make_png(self._thumb_path("a"))
source_time = (self.output_dir / "a.png").stat().st_mtime
os.utime(fresh, (source_time + 10, source_time + 10))
before = _tree_state(self.comfy_dir)
with patch.object(images_module, "MAX_THUMBNAIL_FILES", 1):
missing = await self.client.get(
PLAN_URL, params={"mode": "missing", "size": "512", "quality": "high"}
)
rebuild = await self.client.get(
PLAN_URL, params={"mode": "rebuild", "size": "256", "quality": "low"}
)
self.assertEqual(missing.status, 200)
self.assertEqual(
await missing.json(),
{
"success": True,
"targets": 1,
"fresh": 1,
"remaining": 1,
"roots": ["output"],
"mode": "missing",
"size": 512,
"quality": "high",
},
)
data = await rebuild.json()
# Rebuild regenerates fresh previews too: all three are work, one fits the cap.
self.assertEqual((data["targets"], data["fresh"], data["remaining"]), (1, 1, 2))
self.assertEqual(
(data["mode"], data["size"], data["quality"]), ("rebuild", 256, "low")
)
self.assertEqual(_tree_state(self.comfy_dir), before)
self.assertEqual(self.api._jobs.list(), [])
async def test_plan_defaults_to_missing_previews_at_the_concept_size(self):
make_png(self.output_dir / "a.png")
data = await (await self.client.get(PLAN_URL)).json()
self.assertEqual(
(data["mode"], data["size"], data["quality"], data["targets"]),
("missing", 512, "medium", 1),
)
async def test_size_bounds_validated(self):
make_png(self.output_dir / "a.png")
for params in (
{"size": "127"},
{"size": "2049"},
{"size": "512.5"},
{"size": "big"},
{"mode": "all"},
{"quality": "ultra"},
):
with self.subTest(params=params):
resp = await self.client.get(PLAN_URL, params=params)
body = await resp.json()
self.assertEqual(resp.status, 400)
self.assertEqual(
(body["success"], body["code"]), (False, "invalid-request")
)
for body in (
{"size": 127},
{"size": 2049},
{"size": "512"},
{"size": 512.0},
{"size": True},
{"mode": "rebuild-all"},
{"quality": None},
[],
):
with self.subTest(body=body):
resp = await self.client.post(JOB_URL, json=body)
self.assertEqual(resp.status, 400)
self.assertEqual((await resp.json())["code"], "invalid-request")
for edge in (128, 2048):
with self.subTest(edge=edge):
resp = await self.client.get(PLAN_URL, params={"size": str(edge)})
self.assertEqual(resp.status, 200)
self.assertEqual(self.api._jobs.list(), [])
self.assertFalse(self._thumb_path("a").exists())
class TestThumbnailJob(ThumbnailJobCase):
async def test_job_generates_missing_previews_at_the_chosen_edge(self):
make_png(self.output_dir / "a.png", size=(800, 400))
make_png(self.output_dir / "sub" / "b.png", size=(64, 64))
(self.output_dir / "bad.png").write_bytes(b"\x89PNG not really")
status, body = await self._start(mode="missing", size=200, quality="medium")
self.assertEqual(status, 202)
summary = body["job"]
self.assertEqual(
(summary["kind"], summary["title"], summary["admission"]),
("thumbnails", "Thumbnail generation", "guarded"),
)
self.assertTrue(summary["cancellable"])
self.assertEqual(set(summary["counters"]), {"generated", "skipped", "errors"})
self.assertEqual(
summary["scope"], {"mode": "missing", "size": 200, "quality": "medium"}
)
job = await self._finished(summary["id"])
self.assertEqual(job.state, "finished")
self.assertEqual(job.counters, {"generated": 2, "skipped": 0, "errors": 1})
self.assertEqual((job.processed, job.total, job.progress), (3, 3, 100))
self.assertEqual(job.scope["output_folders"], 1)
with Image.open(self._thumb_path("a")) as thumb:
self.assertEqual(thumb.size, (200, 100))
rows = {row["path"]: row for row in job.outcomes["generated"]}
self.assertEqual(set(rows), {"a.png", "sub/b.png"})
self.assertEqual(
(rows["a.png"]["root"], rows["a.png"]["root_index"]), ("output", 0)
)
self.assertEqual(job.outcomes["errors"][0]["path"], "bad.png")
self.assertTrue(any("Thumbnails finished" in line for line in job.log))
# Rows and log lines never carry the server's absolute layout.
self.assertNotIn(str(self.comfy_dir), repr(job.detail()))
async def test_fresh_previews_are_counted_as_skipped(self):
make_png(self.output_dir / "a.png")
make_png(self.output_dir / "b.png")
fresh = make_png(self._thumb_path("a"))
source_time = (self.output_dir / "a.png").stat().st_mtime
os.utime(fresh, (source_time + 10, source_time + 10))
_, body = await self._start()
job = await self._finished(body["job"]["id"])
self.assertEqual(job.counters, {"generated": 1, "skipped": 1, "errors": 0})
self.assertEqual([r["path"] for r in job.outcomes["skipped"]], ["a.png"])
async def test_rebuild_never_touches_originals(self):
for name in ("a", "b"):
make_png(self.output_dir / f"{name}.png", size=(400, 400))
_, body = await self._start(size=128)
await self._finished(body["job"]["id"])
originals = {
name: hashlib.sha256(
(self.output_dir / f"{name}.png").read_bytes()
).hexdigest()
for name in ("a", "b")
}
_, body = await self._start(mode="rebuild", size=256, quality="high")
job = await self._finished(body["job"]["id"])
self.assertEqual(job.counters, {"generated": 2, "skipped": 0, "errors": 0})
for name in ("a", "b"):
digest = hashlib.sha256(
(self.output_dir / f"{name}.png").read_bytes()
).hexdigest()
self.assertEqual(digest, originals[name], name)
with Image.open(self._thumb_path(name)) as thumb:
self.assertEqual(thumb.size, (256, 256))
async def test_rebuild_keeps_old_preview_when_generation_fails(self):
source = make_png(self.output_dir / "a.png", size=(64, 64))
_, body = await self._start()
await self._finished(body["job"]["id"])
preview = self._thumb_path("a").read_bytes()
source.write_bytes(b"\x89PNG broken now")
later = self._thumb_path("a").stat().st_mtime + 10
os.utime(self._thumb_path("a"), (later, later)) # still fresh
_, body = await self._start(mode="rebuild")
job = await self._finished(body["job"]["id"])
self.assertEqual(job.counters["errors"], 1)
self.assertEqual(self._thumb_path("a").read_bytes(), preview)
leftovers = [p.name for p in self._thumb_path("a").parent.iterdir()]
self.assertEqual(leftovers, ["a_thumb.png"]) # no temp file left
async def test_rebuild_keeps_an_old_video_preview_instead_of_a_placeholder(self):
clip = self.output_dir / "clip.mp4"
clip.write_bytes(b"\x00")
preview = make_png(self.output_dir / "thumbnails" / "clip_thumb.jpg")
old = preview.read_bytes()
with (
patch.dict(sys.modules, {"cv2": None}),
patch("subprocess.run", side_effect=FileNotFoundError("no ffmpeg")),
):
_, body = await self._start(mode="rebuild")
job = await self._finished(body["job"]["id"])
self.assertEqual(job.counters["errors"], 1)
self.assertEqual(preview.read_bytes(), old)
async def test_quality_sets_the_encoder_not_the_edge(self):
noise = bytes(
(i * 7919 + (i >> 3) * 104729) % 251 for i in range(600 * 600 * 3)
)
with Image.frombytes("RGB", (600, 600), noise) as img:
img.save(self.output_dir / "photo.jpg", "JPEG", quality=95)
sizes = {}
for quality in ("low", "high"):
_, body = await self._start(mode="rebuild", size=300, quality=quality)
await self._finished(body["job"]["id"])
thumb = self._thumb_path("photo", ".jpg")
with Image.open(thumb) as image:
self.assertEqual(image.size, (300, 300))
sizes[quality] = thumb.stat().st_size
self.assertLess(sizes["low"], sizes["high"])
async def test_no_output_folder_fails_the_job(self):
self.api._get_all_output_dirs = lambda: []
_, body = await self._start()
job = await self._finished(body["job"]["id"])
self.assertEqual(job.state, "failed")
self.assertEqual(job.status_line, "ComfyUI output directory not found")
async def test_cancel_stops_at_file_boundary_keeping_written_previews(self):
for i in range(6):
make_png(self.output_dir / f"p{i}.png")
original = PromptManagerConfig.WORKER_THREADS
PromptManagerConfig.WORKER_THREADS = 1
self.addCleanup(setattr, PromptManagerConfig, "WORKER_THREADS", original)
written = []
real_write = images_module._write_image_thumbnail
job_ids = []
def counting(src, dst, *args, **kwargs):
real_write(src, dst, *args, **kwargs)
written.append(dst)
if len(written) == 2:
self.assertTrue(self.api._jobs.cancel(job_ids[0]))
with patch.object(images_module, "_write_image_thumbnail", counting):
_, body = await self._start()
job_ids.append(body["job"]["id"])
job = await self._finished(body["job"]["id"])
self.assertEqual(job.state, "cancelled")
self.assertEqual(job.counters["generated"], 2)
self.assertEqual(len(written), 2)
for dst in written:
self.assertTrue(dst.is_file())
made = sorted((self.output_dir / "thumbnails").glob("*_thumb.png"))
self.assertEqual(made, sorted(written))
self.assertTrue(any("Thumbnails cancelled" in line for line in job.log))
async def test_a_second_thumbnail_job_is_refused_naming_the_first(self):
make_png(self.output_dir / "a.png")
entered, release = self._gate_generation()
_, first = await self._start()
await self._wait(entered)
status, body = await self._start(mode="rebuild")
self.assertEqual((status, body["code"]), (409, "busy"))
self.assertEqual(body["running"]["id"], first["job"]["id"])
release.set()
await self._finished(first["job"]["id"])
async def test_thumbnail_job_refused_while_scan_runs(self):
make_png(self.output_dir / "a.png")
self._run_scan()
status, body = await self._start()
self.assertEqual((status, body["code"]), (409, "busy"))
self.assertEqual(body["running"]["title"], "Scan for metadata")
self.assertFalse(self._thumb_path("a").exists())
async def test_scan_refused_while_thumbnail_job_runs(self):
make_png(self.output_dir / "a.png")
entered, release = self._gate_generation()
_, started = await self._start()
await self._wait(entered)
resp = await self.client.post("/prompt_manager/jobs/scan")
body = await resp.json()
self.assertEqual((resp.status, body["code"]), (409, "busy"))
self.assertEqual(body["running"]["title"], "Thumbnail generation")
release.set()
await self._finished(started["job"]["id"])
async def test_clear_refused_while_thumbnail_job_runs(self):
make_png(self.output_dir / "a.png")
kept = make_png(self.output_dir / "thumbnails" / "old_thumb.png")
entered, release = self._gate_generation()
_, started = await self._start()
await self._wait(entered)
resp = await self.client.post("/prompt_manager/images/clear-thumbnails")
self.assertEqual(resp.status, 409)
self.assertEqual(
await resp.json(),
{"success": False, "error": "Thumbnail generation already running"},
)
self.assertTrue(kept.exists())
release.set()
await self._finished(started["job"]["id"])
async def test_every_exit_sets_a_truthful_final_status_line(self):
"""I3: finished, nothing to do, cancelled and failed each say what happened."""
lines = self._status_lines()
make_png(self.output_dir / "a.png")
(self.output_dir / "bad.png").write_bytes(b"\x89PNG nope")
_, body = await self._start()
job = await self._finished(body["job"]["id"])
self.assertEqual(job.status_line, "1 preview written · 1 error")
# Every preview fresh: the job finishes with nothing to do.
source_time = (self.output_dir / "a.png").stat().st_mtime
bad_thumb = make_png(self._thumb_path("bad"))
for thumb in (self._thumb_path("a"), bad_thumb):
os.utime(thumb, (source_time + 10, source_time + 10))
os.utime(self.output_dir / "bad.png", (source_time, source_time))
_, body = await self._start()
job = await self._finished(body["job"]["id"])
self.assertEqual((job.state, job.total), ("finished", 0))
self.assertEqual(
job.status_line, "No previews to generate · 2 already up to date"
)
self.assertNotIn("Processing file 0 of 0", lines)
async def test_an_empty_library_says_there_is_nothing_to_preview(self):
lines = self._status_lines()
_, body = await self._start()
job = await self._finished(body["job"]["id"])
self.assertEqual(
(job.state, job.status_line), ("finished", "No images or videos to preview")
)
self.assertNotIn("Processing file 0 of 0", lines)
async def test_a_cancelled_job_says_what_it_kept(self):
for i in range(6):
make_png(self.output_dir / f"p{i}.png")
self._one_worker()
real_write = images_module._write_image_thumbnail
job_ids = []
written = []
def counting(src, dst, *args, **kwargs):
real_write(src, dst, *args, **kwargs)
written.append(dst)
if len(written) == 1:
self.api._jobs.cancel(job_ids[0])
with patch.object(images_module, "_write_image_thumbnail", counting):
_, body = await self._start()
job_ids.append(body["job"]["id"])
job = await self._finished(body["job"]["id"])
self.assertEqual(job.state, "cancelled")
self.assertEqual(job.status_line, "Cancelled · 1 preview written")
async def test_an_internal_error_says_the_job_failed(self):
def broken(*_args, **_kwargs):
raise RuntimeError("boom")
self.api._plan_gallery_thumbnails = broken
_, body = await self._start()
job = await self._finished(body["job"]["id"])
self.assertEqual(
(job.state, job.status_line),
("failed", "Thumbnail generation stopped on an internal error."),
)
async def test_cancel_during_planning_stops_the_walk_and_starts_nothing(self):
"""M2: the walk checks Cancel per file; no ``start`` frame is published."""
for i in range(10):
make_png(self.output_dir / f"p{i}.png")
entered, release = threading.Event(), threading.Event()
self.addCleanup(release.set)
checked = []
real_fresh = images_module._thumbnail_is_fresh
def gated_fresh(src, dst):
checked.append(src.name)
if len(checked) == 1:
entered.set()
release.wait(10)
return real_fresh(src, dst)
with patch.object(images_module, "_thumbnail_is_fresh", gated_fresh):
stream = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
await self._wait(entered)
job = self.api._jobs.active("thumbnails")
self.assertTrue(self.api._jobs.cancel(job.id))
release.set()
events = parse_sse(await (await stream).text())
await asyncio.wait_for(job.task, 10)
self.assertEqual(checked, ["p0.png"]) # the walk stopped at the next file
self.assertEqual([name for name, _ in events], ["job", "status", "error"])
self.assertEqual(job.state, "cancelled")
self.assertEqual(job.status_line, "Cancelled before any preview was written")
self.assertEqual(job.total, 0)
self.assertFalse((self.output_dir / "thumbnails").exists())
async def test_a_locked_preview_is_a_readable_coded_error(self):
"""M6: Windows refuses to replace a preview another process has open."""
make_png(self.output_dir / "a.png")
locked = PermissionError(13, "Access is denied")
locked.winerror = 5
def refuse(*_args, **_kwargs):
raise locked
with patch.object(images_module, "_write_image_thumbnail", refuse):
_, body = await self._start(mode="rebuild")
job = await self._finished(body["job"]["id"])
self.assertEqual(job.counters["errors"], 1)
[row] = job.outcomes["errors"]
self.assertEqual(row["detail"], images_module.PREVIEW_IN_USE)
self.assertNotIn("WinError", repr(job.detail()))
self.assertTrue(any(images_module.PREVIEW_IN_USE in line for line in job.log))
async def test_a_locked_video_preview_is_the_same_coded_error(self):
clip = self.output_dir / "clip.mp4"
clip.write_bytes(b"\x00")
locked = PermissionError(13, "Access is denied")
locked.winerror = 32
def refuse(*_args, **_kwargs):
raise locked
with (
patch.dict(sys.modules, {"cv2": None}),
patch("subprocess.run", side_effect=FileNotFoundError("no ffmpeg")),
patch.object(images_module, "_atomic_save", refuse),
):
_, body = await self._start()
job = await self._finished(body["job"]["id"])
[row] = job.outcomes["errors"]
self.assertEqual(row["detail"], images_module.PREVIEW_IN_USE)
async def test_a_permission_error_off_windows_names_the_permission(self):
make_png(self.output_dir / "a.png")
def refuse(*_args, **_kwargs):
raise PermissionError(13, "Permission denied")
with patch.object(images_module, "_write_image_thumbnail", refuse):
_, body = await self._start()
job = await self._finished(body["job"]["id"])
[row] = job.outcomes["errors"]
self.assertEqual(row["detail"], images_module.PREVIEW_NOT_WRITABLE)
async def test_ffmpeg_fallback_keeps_quality_and_aspect_ratio(self):
"""M1: without OpenCV, ffmpeg keeps the chosen quality and the aspect ratio."""
(self.output_dir / "clip.mp4").write_bytes(b"\x00")
commands = []
def fake_ffmpeg(cmd, **_kwargs):
commands.append(cmd)
Path(cmd[-1]).write_bytes(b"jpeg")
return type("Done", (), {"returncode": 0, "stderr": ""})()
for quality, qscale in (("high", "2"), ("medium", "4"), ("low", "8")):
with (
patch.dict(sys.modules, {"cv2": None}),
patch("subprocess.run", side_effect=fake_ffmpeg),
):
_, body = await self._start(mode="rebuild", size=640, quality=quality)
job = await self._finished(body["job"]["id"])
self.assertEqual(job.counters["generated"], 1, quality)
cmd = commands[-1]
self.assertNotIn("-s", cmd)
self.assertEqual(
cmd[cmd.index("-vf") + 1],
"scale=640:640:force_original_aspect_ratio=decrease",
)
self.assertEqual(cmd[cmd.index("-q:v") + 1], qscale, quality)
async def test_the_placeholder_uses_the_chosen_quality(self):
(self.output_dir / "clip.mp4").write_bytes(b"\x00")
saved = []
real_save = images_module._atomic_save
def recording(img, dst, fmt=None, **kwargs):
saved.append(kwargs)
return real_save(img, dst, fmt, **kwargs)
with (
patch.dict(sys.modules, {"cv2": None}),
patch("subprocess.run", side_effect=FileNotFoundError("no ffmpeg")),
patch.object(images_module, "_atomic_save", recording),
):
_, body = await self._start(quality="low")
await self._finished(body["job"]["id"])
self.assertEqual(saved, [images_module.THUMBNAIL_ENCODINGS["low"]])
# The legacy stream's frames and their keys, as the gallery page reads them.
LEGACY_KEYS = {
"job": {"id", "started"},
"status": {"phase", "message"},
"start": {
"total_images",
"skipped",
"remaining",
"phase",
"image_count",
"video_count",
"message",
},
"progress": {
"processed",
"total_images",
"generated",
"skipped",
"error_count",
"percentage",
"rate",
"eta",
"elapsed",
"current_file",
"file_type",
"action",
},
"complete": {
"count",
"skipped",
"total_images",
"remaining",
"errors",
"error_count",
"elapsed_time",
"processing_rate",
"message",
},
}
class TestLegacyStreamOnTheJob(ThumbnailJobCase):
async def test_legacy_progress_events_unchanged_on_success(self):
make_png(self.output_dir / "a.png", size=(400, 400))
make_png(self.output_dir / "b.png")
resp = await self.client.post(f"{self.PROGRESS_URL}?quality=low")
events = parse_sse(await resp.text())
self.assertEqual(
[name for name, _ in events],
["job", "status", "start", "progress", "progress", "complete"],
)
for name, data in events:
self.assertEqual(set(data), LEGACY_KEYS[name], name)
complete = events[-1][1]
self.assertEqual((complete["count"], complete["total_images"]), (2, 2))
self.assertEqual(
events[1][1],
{"phase": "scanning", "message": "Scanning for images and videos..."},
)
with Image.open(self._thumb_path("a")) as thumb:
self.assertEqual(max(thumb.size), 150) # the legacy preset still sizes
[job] = self.api._jobs.list()
self.assertEqual(
(job.kind, job.title, job.state, job.cancellable),
("thumbnails", "Thumbnail generation", "finished", True),
)
self.assertEqual(job.counters["generated"], 2)
# I1: the first frame names the job this stream started, for its Cancel.
self.assertEqual(events[0][1], {"id": job.id, "started": True})
async def test_legacy_file_errors_still_stream(self):
make_png(self.output_dir / "good.png")
(self.output_dir / "bad.png").write_bytes(b"\x89PNG nope")
resp = await self.client.post(self.PROGRESS_URL)
events = parse_sse(await resp.text())
self.assertEqual(
[d for e, d in events if e == "file_error"],
[{"file": "bad.png", "error": "Not a valid image file"}],
)
async def test_thumbnail_job_survives_client_disconnect(self):
"""I2: the stream sees its client go after batch 1; every batch still runs.
One worker thread makes batches of 4, so 9 files take three batches. The
first file waits until the client is gone; the second batch waits until
the stream has seen it go (its subscription dropped) and records whether
the job was still running then. A stream that stopped the work on
disconnect would leave 4 previews; the job writes all 9.
"""
for i in range(9):
make_png(self.output_dir / f"p{i}.png")
self._one_worker()
first, closed = threading.Event(), threading.Event()
state = {"calls": 0, "left": None, "running_when_left": None}
original = self.api._generate_one
def gated(*args, **kwargs):
state["calls"] += 1
if state["calls"] == 1:
first.set()
closed.wait(10)
elif state["calls"] == 5: # the second batch
state["running_when_left"] = state["left"].wait(10)
return original(*args, **kwargs)
self.api._generate_one = gated
self.addCleanup(closed.set)
resp = await self.client.post(self.PROGRESS_URL)
self.assertEqual(resp.status, 200)
await self._wait(first)
job = self.api._jobs.active("thumbnails")
_subscribed, state["left"] = self._watch_subscribers(job)
resp.close() # the gallery tab went away
closed.set()
await self._wait(state["left"]) # the server saw the client go
self.assertEqual(job.subscriber_count, 0)
await asyncio.wait_for(job.task, 10)
self.assertEqual(job.counters["generated"], 9)
self.assertEqual(job.state, "finished")
self.assertTrue(state["running_when_left"])
for i in range(9):
self.assertTrue(self._thumb_path(f"p{i}").is_file())
async def test_legacy_stream_attaches_to_a_running_job(self):
make_png(self.output_dir / "a.png")
entered, release = self._gate_generation()
_, started = await self._start(mode="missing", size=256)
await self._wait(entered)
job = self.api._jobs.get(started["job"]["id"])
subscribed, _left = self._watch_subscribers(job)
attached = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
await self._wait(subscribed) # M4: an event with a ceiling, never a poll
release.set()
events = parse_sse(await (await attached).text())
self.assertEqual(events[-1][0], "complete")
# I1: the stream names the job it did not start: the gallery never cancels it.
self.assertEqual(events[0], ("job", {"id": job.id, "started": False}))
# One job: the stream followed the running one with its own options.
[listed] = self.api._jobs.list()
self.assertIs(listed, job)
self.assertEqual(job.scope["size"], 256)
async def test_cancel_from_activity_ends_the_legacy_stream_with_an_error(self):
for i in range(2):
make_png(self.output_dir / f"p{i}.png")
entered, release = self._gate_generation()
stream = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
await self._wait(entered)
job = self.api._jobs.active("thumbnails")
self.assertTrue(self.api._jobs.cancel(job.id))
release.set()
events = parse_sse(await (await stream).text())
self.assertEqual(events[-1][0], "error")
self.assertEqual(events[-1][1]["error"], "Thumbnail generation was cancelled")
await asyncio.wait_for(job.task, 10)
self.assertEqual(job.state, "cancelled")
async def test_a_job_started_while_the_stream_prepares_is_attached(self):
"""M3: a job that starts during ``prepare`` is followed, not refused."""
from aiohttp import web
make_png(self.output_dir / "a.png")
entered, release = self._gate_generation()
real_prepare = web.StreamResponse.prepare
raced = []
async def prepare(response, request):
if not raced and type(response) is web.StreamResponse:
options, _ = thumbnail_options({"size": 256})
raced.append(self.api._start_thumbnail_job(options))
return await real_prepare(response, request)
with patch.object(web.StreamResponse, "prepare", prepare):
stream = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
await self._wait(entered)
release.set()
events = parse_sse(await (await stream).text())
[job] = raced
self.assertEqual(events[0], ("job", {"id": job.id, "started": False}))
self.assertEqual(events[-1][0], "complete")
self.assertEqual(self.api._jobs.list(), [job])
async def test_a_refused_stream_names_no_job(self):
make_png(self.output_dir / "a.png")
self._run_scan()
events = parse_sse(await (await self.client.post(self.PROGRESS_URL)).text())
self.assertEqual(
events,
[
(
"error",
{
"error": "Scan for metadata already running",
"message": "Scan for metadata already running",
},
)
],
)
if __name__ == "__main__":
unittest.main()
+5 -35
View File
@@ -42,10 +42,10 @@
"statements": 98
},
"gallery.js": {
"lines": 45,
"branches": 64,
"functions": 48,
"statements": 45
"lines": 41,
"branches": 63,
"functions": 44,
"statements": 41
},
"html-escape.js": {
"lines": 98,
@@ -161,12 +161,6 @@
"functions": 98,
"statements": 97
},
"workspace/activity-kinds.mjs": {
"lines": 98,
"branches": 95,
"functions": 98,
"statements": 98
},
"workspace/activity-scan.mjs": {
"lines": 96,
"branches": 78,
@@ -181,7 +175,7 @@
},
"workspace/activity.mjs": {
"lines": 96,
"branches": 91,
"branches": 87,
"functions": 98,
"statements": 96
},
@@ -221,12 +215,6 @@
"functions": 98,
"statements": 98
},
"workspace/image-card.mjs": {
"lines": 98,
"branches": 98,
"functions": 98,
"statements": 98
},
"workspace/image-layout.mjs": {
"lines": 98,
"branches": 98,
@@ -449,18 +437,6 @@
"functions": 80,
"statements": 95
},
"workspace/review-state.mjs": {
"lines": 96,
"branches": 84,
"functions": 98,
"statements": 96
},
"workspace/review.mjs": {
"lines": 98,
"branches": 89,
"functions": 93,
"statements": 98
},
"workspace/saved-views.mjs": {
"lines": 98,
"branches": 95,
@@ -515,12 +491,6 @@
"functions": 98,
"statements": 98
},
"workspace/thumbnail-review.mjs": {
"lines": 96,
"branches": 93,
"functions": 98,
"statements": 96
},
"workspace/toast.mjs": {
"lines": 98,
"branches": 98,
+13 -87
View File
@@ -1,7 +1,4 @@
class GalleryManager {
// A Cancel on a thumbnail job another tab or System started (#249 System S5, review I1).
static THUMBNAILS_STOPPED_FOLLOWING = 'Stopped following thumbnail generation started elsewhere; it keeps running in Activity';
constructor() {
this.images = [];
this.currentPage = 1;
@@ -1914,54 +1911,6 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
}
}
/** Ask the server to stop thumbnail job `jobId` at its next file boundary (#249 System S5). */
async cancelThumbnailJob(jobId) {
try {
await this.api.post(`/prompt_manager/jobs/${encodeURIComponent(jobId)}/cancel`, {});
return true;
} catch (error) {
console.warn('The thumbnail job could not be cancelled:', error);
return false;
}
}
/**
* Cancel for one thumbnail stream (#249 System S5, review I1). The stream's first frame, `job {id, started}`,
* names the job it follows and whether it started it. `request()` stops that job by its id, and only when this
* stream started it: a job another tab or System started keeps running, and the stream just stops following.
* A request before the `job` frame keeps the stream open until the frame arrives; `ended()` (the stream closed
* or refused without naming a job) leaves nothing to cancel.
*/
thumbnailCanceller(eventSource) {
let job = null;
let wanted = false;
let done = false;
const apply = () => {
if (done || !wanted || !job) return;
done = true;
eventSource.close();
if (job.started) this.cancelThumbnailJob(job.id);
};
return {
identify(raw) {
if (job) return;
let data = null;
try { data = JSON.parse(raw); } catch (error) { return; }
if (typeof data?.id !== 'string' || !data.id) return;
job = { id: data.id, started: data.started === true };
apply();
},
request() { wanted = true; apply(); },
/** True once the stream named a job this page did not start. */
get following() { return job !== null && !job.started; },
ended() {
if (done || !wanted || job) return;
done = true;
eventSource.close();
},
};
}
async generateThumbnailsWithProgress(options) {
const progressBar = document.getElementById('progressBar');
const progressText = document.getElementById('progressText');
@@ -1983,19 +1932,13 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
let cancelled = false;
let resultData = null;
const canceller = this.thumbnailCanceller(eventSource);
eventSource.addEventListener('job', event => canceller.identify(event.data));
// Handle cancellation. The work is an Activity job that outlives this stream, so closing it no
// longer stops anything: ask the server to stop this stream's job by its id (review I1).
// Handle cancellation
const cancelHandler = () => {
if (cancelled) return;
cancelled = true;
cancelBtn.removeEventListener('click', cancelHandler);
eventSource.close();
cancelBtn.classList.add('hidden');
const following = canceller.following;
canceller.request();
reject(new Error(following ? GalleryManager.THUMBNAILS_STOPPED_FOLLOWING : 'Thumbnail generation cancelled by user'));
reject(new Error('Thumbnail generation cancelled by user'));
};
cancelBtn.addEventListener('click', cancelHandler);
@@ -2070,7 +2013,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
// Handle completion
eventSource.addEventListener('complete', (event) => {
if (cancelled) { canceller.ended(); return; }
if (cancelled) return;
resultData = JSON.parse(event.data);
// Update final progress
@@ -2118,7 +2061,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
// Handle errors
eventSource.addEventListener('error', (event) => {
if (cancelled) { canceller.ended(); return; }
if (cancelled) return;
const data = JSON.parse(event.data);
// Clean up
@@ -2131,7 +2074,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
// Handle EventSource errors
eventSource.onerror = (error) => {
if (cancelled) { canceller.ended(); return; }
if (cancelled) return;
console.error('EventSource error:', error);
console.error('EventSource readyState:', eventSource.readyState);
@@ -2154,7 +2097,6 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
// Stream ended without a complete/error frame (close() does not trigger this)
eventSource.onclose = () => {
if (cancelled) canceller.ended();
if (cancelled || resultData) return;
cancelBtn.removeEventListener('click', cancelHandler);
cancelBtn.classList.add('hidden');
@@ -2750,15 +2692,12 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
statusMessages.scrollTop = statusMessages.scrollHeight;
};
// Handle cancel: the stream's own job stops at its next file boundary (#249 System S5, review I1)
let cancelStream = null;
// Handle cancel
cancelBtn.addEventListener('click', () => {
if (isCancelled) return;
isCancelled = true;
addStatusMessage('Cancellation requested...', 'warning');
cancelBtn.disabled = true;
cancelBtn.textContent = 'Cancelling...';
cancelStream?.();
});
// Handle close (only appears when done)
@@ -2782,8 +2721,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
progressDetails,
progressETA,
addStatusMessage,
isCancelled: () => isCancelled,
onCancel: handler => { cancelStream = handler; }
isCancelled: () => isCancelled
});
// Show completion
@@ -2803,14 +2741,10 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
}
} catch (error) {
if (isCancelled) {
addStatusMessage(error.message, 'warning');
} else {
console.error('Thumbnail generation failed:', error);
addStatusMessage(`Generation failed: ${error.message}`, 'error');
progressBar.classList.add('bg-pm-error');
progressBar.classList.remove('bg-pm-accent');
}
console.error('Thumbnail generation failed:', error);
addStatusMessage(`Generation failed: ${error.message}`, 'error');
progressBar.classList.add('bg-pm-error');
progressBar.classList.remove('bg-pm-accent');
} finally {
// Show close button and hide cancel
cancelBtn.classList.add('hidden');
@@ -2828,7 +2762,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
}, 100);
}
} async generateThumbnailsWithProgressInModal(uiElements) {
const { progressText, progressPercent, progressBar, progressDetails, progressETA, addStatusMessage, isCancelled, onCancel } = uiElements;
const { progressText, progressPercent, progressBar, progressDetails, progressETA, addStatusMessage, isCancelled } = uiElements;
return new Promise((resolve, reject) => {
addStatusMessage('Starting thumbnail generation...');
@@ -2842,14 +2776,6 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
const eventSource = SseStream.connect(`/prompt_manager/images/generate-thumbnails/progress?quality=${encodeURIComponent(quality)}`, { method: 'POST' });
let resultData = null;
const canceller = this.thumbnailCanceller(eventSource);
eventSource.addEventListener('job', event => canceller.identify(event.data));
// Cancel stops this stream's job by id, keeps what it wrote, and settles the dialog.
onCancel?.(() => {
const following = canceller.following;
canceller.request();
reject(new Error(following ? GalleryManager.THUMBNAILS_STOPPED_FOLLOWING : 'Thumbnail generation cancelled; previews already written are kept'));
});
// Handle scanning/status events
eventSource.addEventListener('status', (event) => {
+14 -29
View File
@@ -3,8 +3,7 @@
* concept's classes.
*
* - Header: the page heading names the job; its line is the Activity item's (D54), and its actions are Download report
* (the job's JSON report, D59) and, while the job runs, "Cancel scan" (danger). D87: an ended scan that recorded
* Needs review items adds the concept's **Review metadata** after Download report.
* (the job's JSON report, D59) and, while the job runs, "Cancel scan" (danger).
* - Progress (`section.surface`): `<progress>`, the server's "Processing file N of M · about X min left", and the
* outcome counters as `dl.stat-row` tiles: Added to Library (green), Already known, Needs review (warning), No usable
* metadata, Errors (red).
@@ -14,11 +13,7 @@
* - Log: `details.term` "Scan log", collapsed, holding the raw timestamped lines in `pre[tabindex=0]`.
* - D89: the tiles, the outcome list, the openable rows and the log and cancel labels are the job kind's (`jobView`):
* an image tagging job shows Tagged, Unchanged, Skipped and Errors, opens Tagged and Unchanged rows, and reads
* "Tagging log" and "Cancel tagging".
* - D111: the kind's descriptor (activity-kinds.mjs `viewOf`) decides them for every kind: a thumbnail job counts
* Generated, Skipped and Errors; a kind without groups (the health check, a held legacy-page operation) shows no
* tiles and no Results, never another kind's zeros. A job without a known total has an indeterminate `<progress>`;
* only a job the server can stop shows Cancel.
* "Tagging log" and "Cancel tagging"; another kind keeps the scan's tiles as "Job log" and "Cancel job".
*
* The summary follows the Activity store (one SSE stream per running job); the listed rows and the log are read from
* `GET /prompt_manager/jobs/<id>` at most once a second while the job runs, and once more when it ends.
@@ -27,9 +22,8 @@ import { createButton } from '../shared/toolkit/core.js';
import { pageHeading } from './heading.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { noun } from './plural.mjs';
import { parseJob, JOB_VIEWS, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
import { viewOf, canCancel, showProgress } from './activity-kinds.mjs';
import { jobDetail, CANCEL_FAILED_TOAST, createReviewMetadataButton, hasReviewItems } from './activity.mjs';
import { parseJob, jobView, JOB_VIEWS, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
import { jobDetail, CANCEL_FAILED_TOAST } from './activity.mjs';
import { downloadJobReport } from './activity-scan.mjs';
import { createMediaViewer } from './media-viewer.mjs';
import { createInspectionController } from './inspection-state.mjs';
@@ -76,10 +70,7 @@ export function parseDetail(raw) {
/** The progress line: the server's while running, then what the job got through. */
export function progressLine(job) {
const files = `${number(job.processed)} of ${number(job.total)} ${noun(job.total, 'file')}`;
// D111: a job that never counted files (a held legacy job, the health check) says only what the server says.
const counted = job.total > 0;
if (job.state === 'running') return job.status_line || (counted ? `${job.progress}%` : '');
if (!counted && (job.state === 'finished' || job.state === 'cancelled')) return job.status_line;
if (job.state === 'running') return job.status_line || `${job.progress}%`;
if (job.state === 'finished') return `${files} processed`;
if (job.state === 'cancelled') return `Cancelled after ${files}`;
if (job.state === 'lost') return LOST_DETAIL;
@@ -105,7 +96,7 @@ export function filterRows(rows, filter) {
/** The first outcome with files, so a view opens on what happened; the first outcome when nothing has yet. */
export function defaultOutcome(counters, outcomes = JOB_VIEWS.scan.outcomes) {
return (outcomes.find(([group]) => (counters[group] ?? 0) > 0) ?? outcomes[0])?.[0] ?? null;
return (outcomes.find(([group]) => (counters[group] ?? 0) > 0) ?? outcomes[0])[0];
}
function section(title, id) {
@@ -124,7 +115,6 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
const parts = pageHeading(main);
const download = createButton('Download report');
const cancel = createButton('Cancel scan', 'danger');
const review = createReviewMetadataButton(scope); // D87
const progressPart = section('Progress', 'job-progress-title');
const status = element('span', '', 'small');
@@ -187,10 +177,9 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
const viewer = createMediaViewer({ api, container: main, parentScope: scope, inspectionController: inspection,
sender: createPromptSender({ scope }), onSelect: image => inspection.select(image) });
/** The tiles and the outcome list of the job's kind, built once (D89, D111: only the groups it counts). */
function layout(job) {
view = viewOf(job);
if (!view.tiles.length) stats.remove();
/** The tiles and the outcome list of the job's kind, built once. */
function layout(kind) {
view = jobView(kind); // D89
for (const [group, label, tone] of view.tiles) {
const tile = element('div', undefined, tone ? `stat ${tone}` : 'stat');
const value = element('dd', '0');
@@ -212,11 +201,10 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
function showBody(job) {
if (mounted) return;
mounted = true;
if (!view) layout(job);
if (!view) layout(job.kind);
missing.remove();
parts.actions.append(download, cancel);
// A kind without outcome groups has no Results to browse (D111).
main.append(...(view.outcomes.length ? [progressPart.node, resultsPart.node, log] : [progressPart.node, log]));
main.append(progressPart.node, resultsPart.node, log);
selected ??= defaultOutcome(job.counters, view.outcomes);
}
@@ -252,7 +240,6 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
}
function renderRows(job) {
if (!view.outcomes.length) return;
const rows = detail?.outcomes[selected] ?? [];
const openable = view.openable.has(selected);
const wanted = filterRows(rows.map((row, index) => ({ ...row, index })), input.value);
@@ -287,10 +274,8 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
const lost = job.state === 'lost';
if (lost && document.activeElement === download) parts.h1.focus();
download.disabled = lost;
if (hasReviewItems(job)) { if (!review.isConnected) download.after(review); } // D87
else review.remove();
// Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel.
if (canCancel(job)) {
if (job.state === 'running' && job.cancellable) {
if (!cancel.isConnected) parts.actions.append(cancel);
} else if (cancel.isConnected) {
// A focused Cancel that goes away hands its focus to the job's heading, never to the page body.
@@ -298,8 +283,8 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
cancel.remove();
}
status.textContent = progressLine(job);
// D111: no value while the total is unknown; a job that ended with nothing counted shows how it ended.
showProgress(bar, job.total > 0 || job.state !== 'finished' ? job : { ...job, progress: 100 });
bar.value = job.progress;
bar.textContent = `${job.progress}%`;
bar.setAttribute('aria-label', scan ? 'Scan progress' : `${job.title} progress`);
for (const [group, value] of tileValues) value.textContent = number(job.counters[group] ?? 0);
for (const [group, button] of buttons) {
-189
View File
@@ -1,189 +0,0 @@
/**
* Per-kind Activity descriptors (#249 System S5, D111): one table that says, for each job kind, what Activity shows.
*
* `kindOf(kind)` → `{pillVerb, startToast(job), finishToast(job), line(job), kept(job), scope(job), tiles, outcomes,
* openable, log, cancelLabel}`:
* - `pillVerb`: the pill's verb ("Scanning"); null shows the job's title.
* - `startToast` / `finishToast`: what a toast says when this tab starts the job and when it ends.
* - `line`: the counters as one line ("N added · M errors"); `kept`: what a cancelled job kept ("N added"); `scope`:
* what the job covers ("2 output folders").
* - `tiles` (`[group, label, tone]`), `outcomes` (`[group, label]`) and `openable` (rows that open the viewer, D57).
* - `log` and `cancelLabel` (`cancel` is the same label under the job view's older name).
*
* `scan` is exactly the D56 view and `image-tags` the D89 one. `thumbnails` counts Generated (green), Skipped and
* Errors (red). Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
* the generic descriptor: title-based copy and no tiles or outcomes, so it never shows another kind's zeros (S1 review
* M6, S2 review M3). `viewOf(job)` keeps only the groups the job itself counts, so a hold (no groups) has none either.
*
* Progress is indeterminate while a job runs without a known total (`indeterminate`): the `<progress>` has no value
* and the pill shows no percentage, never a made-up 0%. A job shows Cancel only while it runs and the server can stop
* it (`canCancel`).
*/
import { noun, countOf } from './plural.mjs';
// Grouped as the server writes its status lines ("8,240 of 19,622"), whatever the browser's locale.
const number = value => value.toLocaleString('en-US');
const counted = (job, group, word) => `${number(job.counters?.[group] ?? 0)} ${word}`;
const errorsOf = job => `${number(job.counters?.errors ?? 0)} ${noun(job.counters?.errors ?? 0, 'error')}`;
const whole = value => Number.isSafeInteger(value) && value >= 0;
const tiles = rows => Object.freeze(rows.map(Object.freeze));
const groups = (...names) => Object.freeze(new Set(names));
/** "1,420 added". */
export const addedOf = job => counted(job, 'added', 'added');
/** "12 tagged" (D89). */
export const taggedOf = job => counted(job, 'tagged', 'tagged');
/** "3 generated" (D111). */
export const generatedOf = job => counted(job, 'generated', 'generated');
/** "2 output folders": what a scan or a thumbnail job walks. */
function foldersOf(job) {
const folders = job.scope?.output_folders;
return whole(folders) ? countOf(folders, 'output folder') : '';
}
/** "4 images" (D89), grouped as the server writes its lines. */
function imagesOf(job) {
const images = job.scope?.images;
return whole(images) ? `${number(images)} ${noun(images, 'image')}` : '';
}
const titleStarted = job => `${job.title} started.`;
function titleFinished(job) {
if (job.state === 'cancelled') return `${job.title} was cancelled.`;
if (job.state === 'failed') return `${job.title} failed.`;
return `${job.title} finished.`;
}
/** "<Noun> finished · <line>", "<Noun> cancelled · <kept>", "<Noun> failed · <line>". */
const namedFinish = (name, kept, line) => job => {
if (job.state === 'cancelled') return `${name(job)} cancelled · ${kept(job)}`;
return `${name(job)} ${job.state === 'failed' ? 'failed' : 'finished'} · ${line(job)}`;
};
const nothing = () => '';
const GENERIC = Object.freeze({
pillVerb: null, startToast: titleStarted, finishToast: titleFinished, line: nothing, kept: nothing,
scope: foldersOf, tiles: tiles([]), outcomes: tiles([]), openable: groups(), log: 'Job log', cancelLabel: 'Cancel job',
cancel: 'Cancel job',
});
const scanLine = job => `${addedOf(job)} · ${errorsOf(job)}`;
const tagLine = job => `${taggedOf(job)} · ${errorsOf(job)}`;
const thumbnailLine = job => `${generatedOf(job)} · ${errorsOf(job)}`;
// D85: an image tagging job says which of its two modes started.
const IMAGE_TAG_START_TOASTS = Object.freeze({ add: 'Tagging started.', auto: 'Auto-tagging started.' });
/** Every kind with a descriptor of its own. */
export const KINDS = Object.freeze({
__proto__: null,
// D56: the scan's tiles and outcome list in the mockup's orders; what got added and what went wrong first.
scan: Object.freeze({
...GENERIC,
pillVerb: 'Scanning',
startToast: () => 'Scan started.',
finishToast: namedFinish(() => 'Scan', addedOf, scanLine),
line: scanLine,
kept: addedOf,
tiles: tiles([['added', 'Added to Library', 'added'], ['known', 'Already known', ''],
['review', 'Needs review', 'warn'], ['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad']]),
outcomes: tiles([['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
['no_metadata', 'No usable metadata'], ['known', 'Already known']]),
openable: groups('added', 'known'),
log: 'Scan log',
cancelLabel: 'Cancel scan',
cancel: 'Cancel scan',
}),
// D89: Tagged (green), Unchanged, Skipped (warning) and Errors (red); Tagged and Unchanged rows open the viewer.
'image-tags': Object.freeze({
...GENERIC,
pillVerb: 'Tagging',
startToast: job => IMAGE_TAG_START_TOASTS[job.scope?.mode] ?? titleStarted(job),
finishToast: namedFinish(job => job.title, taggedOf, tagLine),
line: tagLine,
kept: taggedOf,
scope: imagesOf,
tiles: tiles([['tagged', 'Tagged', 'added'], ['unchanged', 'Unchanged', ''], ['skipped', 'Skipped', 'warn'],
['errors', 'Errors', 'bad']]),
outcomes: tiles([['tagged', 'Tagged'], ['errors', 'Errors'], ['skipped', 'Skipped'], ['unchanged', 'Unchanged']]),
openable: groups('tagged', 'unchanged'),
log: 'Tagging log',
cancelLabel: 'Cancel tagging',
cancel: 'Cancel tagging',
}),
// D111: Generated (green), Skipped (already fresh) and Errors (red); a row with a preview opens the viewer.
thumbnails: Object.freeze({
...GENERIC,
pillVerb: 'Generating thumbnails',
finishToast: namedFinish(() => 'Thumbnails', generatedOf, thumbnailLine),
line: thumbnailLine,
kept: generatedOf,
tiles: tiles([['generated', 'Generated', 'added'], ['skipped', 'Skipped', ''], ['errors', 'Errors', 'bad']]),
outcomes: tiles([['generated', 'Generated'], ['errors', 'Errors'], ['skipped', 'Skipped']]),
openable: groups('generated', 'skipped'),
}),
});
/** The descriptor of a job `kind`; any kind without one gets the generic descriptor. */
export function kindOf(kind) {
return (typeof kind === 'string' && KINDS[kind]) || GENERIC;
}
const countsOf = job => job.counters && typeof job.counters === 'object' ? job.counters : {};
/**
* A legacy-page operation held on the registry (D112): not cancellable and counting no outcome groups. Whatever its
* kind (the gallery's thumbnail clear is a `thumbnails` hold), it reads as the generic job under its own title.
*/
export const held = job => job.cancellable === false && Object.keys(countsOf(job)).length === 0;
/**
* The descriptor for one job: its kind's, keeping only the outcome groups the job counts; a held job's is the generic
* one.
*/
export function viewOf(job) {
if (held(job)) return GENERIC;
const view = kindOf(job.kind);
const counts = countsOf(job);
const has = ([group]) => Object.hasOwn(counts, group);
if (view.tiles.every(has) && view.outcomes.every(has)) return view;
return Object.freeze({ ...view, tiles: tiles(view.tiles.filter(has)), outcomes: tiles(view.outcomes.filter(has)),
openable: groups(...[...view.openable].filter(group => Object.hasOwn(counts, group))) });
}
/** A running job whose total the server has not reported (yet) has no percentage. */
export function indeterminate(job) {
return job.state === 'running' && !(job.total > 0);
}
/** Set a `<progress>` to the job's: its value and "N%", or no value at all while indeterminate. */
export function showProgress(node, job) {
if (indeterminate(job)) {
node.removeAttribute('value');
node.textContent = '';
return;
}
node.value = job.progress;
node.textContent = `${job.progress}%`;
}
/** Truth rule: Cancel only while the job runs and the server can stop it (never on a held legacy job). */
export function canCancel(job) {
return job.state === 'running' && job.cancellable !== false;
}
/** The pill's text: "Scanning · 42% · 8,240 / 19,622"; the file counts once the total is known, no "%" before. */
export function pillParts(job) {
const verb = viewOf(job).pillVerb ?? job.title;
const label = indeterminate(job) ? verb : `${verb} · ${job.progress}%`;
const files = job.total > 0 ? ` · ${number(job.processed)} / ${number(job.total)}` : '';
return { label, files, text: label + files };
}
/** The toast after this tab starts a job. */
export const startToast = job => viewOf(job).startToast(job);
/** The toast when a job ends. */
export const finishToast = job => viewOf(job).finishToast(job);
/** The counters as one line; '' for a kind without one. */
export const outcomeLine = job => viewOf(job).line(job);
+94 -7
View File
@@ -12,9 +12,7 @@
* the finish toast follows any job this store saw running when it ends.
*/
import { showWorkspaceToast } from './toast.mjs';
import { KINDS, kindOf, finishToast, startToast } from './activity-kinds.mjs';
export { addedOf, taggedOf, finishToast, startToast, pillParts, outcomeLine } from './activity-kinds.mjs';
import { noun } from './plural.mjs';
export const JOBS_URL = '/prompt_manager/jobs';
export const BACKOFF_FIRST_MS = 1000;
@@ -25,13 +23,51 @@ export const LOST_DETAIL = 'This job ended when ComfyUI restarted.';
const SERVER_STATES = new Set(['running', 'finished', 'cancelled', 'failed']);
const FINAL_STATES = new Set(['finished', 'cancelled', 'failed']);
const JOB_ID = /^[0-9a-f]{32}$/;
// The pill's verb for each job kind; a kind without one shows its title (D89: image tagging jobs read "Tagging").
const PILL_VERBS = Object.freeze({ scan: 'Scanning', 'image-tags': 'Tagging' });
const START_TOASTS = Object.freeze({ scan: 'Scan started.' });
/** The scan's job view (D56): its counter tiles `[group, label, tone]` and outcome list in the mockup's orders. */
const SCAN_VIEW = Object.freeze({
tiles: Object.freeze([
['added', 'Added to Library', 'added'], ['known', 'Already known', ''], ['review', 'Needs review', 'warn'],
['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad'],
].map(Object.freeze)),
// What got added and what went wrong first.
outcomes: Object.freeze([
['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
['no_metadata', 'No usable metadata'], ['known', 'Already known'],
].map(Object.freeze)),
// Rows that open the file in the viewer (D57); the others are plain text.
openable: Object.freeze(new Set(['added', 'known'])),
log: 'Scan log',
cancel: 'Cancel scan',
});
/**
* Each job kind's copy and view live in one table (activity-kinds.mjs, D111): the scan's (D56), image tagging's (D89),
* thumbnails' and the generic one. The names below are that table's, kept for this store's importers.
* Each job kind's view: its tiles, outcome list, openable rows, log label and cancel label. D89: an image tagging job
* (`image-tags`) counts Tagged (green), Unchanged, Skipped (warning) and Errors (red), lists what got tagged and what
* went wrong first, and opens Tagged and Unchanged rows in the viewer like the scan's (D57).
*/
export const JOB_VIEWS = KINDS;
export const JOB_VIEWS = Object.freeze({
scan: SCAN_VIEW,
'image-tags': Object.freeze({
tiles: Object.freeze([['tagged', 'Tagged', 'added'], ['unchanged', 'Unchanged', ''], ['skipped', 'Skipped', 'warn'],
['errors', 'Errors', 'bad']].map(Object.freeze)),
outcomes: Object.freeze([['tagged', 'Tagged'], ['errors', 'Errors'], ['skipped', 'Skipped'],
['unchanged', 'Unchanged']].map(Object.freeze)),
openable: Object.freeze(new Set(['tagged', 'unchanged'])),
log: 'Tagging log',
cancel: 'Cancel tagging',
}),
});
// Another kind keeps the scan's tiles and outcomes under the generic labels.
const GENERIC_VIEW = Object.freeze({ ...SCAN_VIEW, log: 'Job log', cancel: 'Cancel job' });
/** The view of a job `kind`; an unknown kind gets the generic one. */
export const jobView = kindOf;
export function jobView(kind) {
return Object.hasOwn(JOB_VIEWS, kind) ? JOB_VIEWS[kind] : GENERIC_VIEW;
}
/** The wait before the `attempt`-th re-read (0-based): 1 s, 2 s, 4 s … never more than 30 s. */
export function backoffDelay(attempt) {
@@ -72,6 +108,57 @@ export function parseJob(raw) {
};
}
// Grouped as the server writes its status lines ("8,240 of 19,622"), whatever the browser's locale.
const number = value => value.toLocaleString('en-US');
const errorsOf = job => `${number(job.counters.errors ?? 0)} ${noun(job.counters.errors ?? 0, 'error')}`;
/** "1,420 added". */
export const addedOf = job => `${number(job.counters.added ?? 0)} added`;
/**
* The toast when a job ends: "Scan finished · N added · M errors", "Scan cancelled · N added"; an image tagging job
* the same way with its own title and count (D89): "Auto-tag images finished · N tagged · M errors".
*/
export function finishToast(job) {
if (job.kind === 'scan') {
if (job.state === 'cancelled') return `Scan cancelled · ${addedOf(job)}`;
if (job.state === 'failed') return `Scan failed · ${addedOf(job)} · ${errorsOf(job)}`;
return `Scan finished · ${addedOf(job)} · ${errorsOf(job)}`;
}
if (job.kind === 'image-tags') {
if (job.state === 'cancelled') return `${job.title} cancelled · ${taggedOf(job)}`;
return `${job.title} ${job.state === 'failed' ? 'failed' : 'finished'} · ${outcomeLine(job)}`;
}
if (job.state === 'cancelled') return `${job.title} was cancelled.`;
if (job.state === 'failed') return `${job.title} failed.`;
return `${job.title} finished.`;
}
// D85: an image tagging job says which of its two modes started.
const IMAGE_TAG_START_TOASTS = Object.freeze({ add: 'Tagging started.', auto: 'Auto-tagging started.' });
/** The toast after this tab starts a job. */
export function startToast(job) {
if (job.kind === 'image-tags' && IMAGE_TAG_START_TOASTS[job.scope.mode]) return IMAGE_TAG_START_TOASTS[job.scope.mode];
return START_TOASTS[job.kind] ?? `${job.title} started.`;
}
/** The pill's text: "Scanning · 42% · 8,240 / 19,622"; the file counts only once the total is known. */
export function pillParts(job) {
const label = `${PILL_VERBS[job.kind] ?? job.title} · ${job.progress}%`;
const files = job.total > 0 ? ` · ${number(job.processed)} / ${number(job.total)}` : '';
return { label, files, text: label + files };
}
/** "12 tagged" (D89). */
export const taggedOf = job => `${number(job.counters.tagged ?? 0)} tagged`;
/** The counters as one line: "N added · M errors" for a scan, "N tagged · M errors" for image tagging (D89). */
export function outcomeLine(job) {
if (job.kind === 'scan') return `${addedOf(job)} · ${errorsOf(job)}`;
if (job.kind === 'image-tags') return `${taggedOf(job)} · ${errorsOf(job)}`;
return '';
}
function defaultConnect(url) {
const resolve = globalThis.ApiClient?.resolveUrl ?? (value => value);
return new EventSource(resolve(url));
+20 -31
View File
@@ -11,36 +11,19 @@
* - D54: each `.activity-item` is a real job. Its chip names the state (green "Completed", or blue "Running",
* "Cancelled", "Failed", "Ended"), its line says when and what, and a running job shows its `<progress>` and
* "Cancel job" (no "Simulate completion").
* - D87 (plan D64): an ended scan that recorded Needs review items carries the concept's **Review metadata**
* (`go-review`, app.js a738) after its line, which opens `#/review`; its chip still names its state (D54).
* - D89: an image tagging job's line names its images and, once ended, "N tagged · M errors".
* - D111: what a line says (scope, counters, what a cancel kept) is the job kind's descriptor (activity-kinds.mjs); a
* job without a known total shows an indeterminate `<progress>`, and only a job the server can stop shows Cancel.
*/
import { createButton } from '../shared/toolkit/core.js';
import { navigateTo } from './state.mjs';
import { pageHeading } from './heading.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { pillParts, outcomeLine, LOST_DETAIL } from './activity-store.mjs';
import { viewOf, canCancel, showProgress, indeterminate } from './activity-kinds.mjs';
import { countOf, noun } from './plural.mjs';
import { pillParts, outcomeLine, addedOf, taggedOf, LOST_DETAIL } from './activity-store.mjs';
export const ACTIVITY_DESCRIPTION = 'Scans, tagging, downloads and maintenance share one history.';
export const EMPTY_ACTIVITY = 'No jobs have run since ComfyUI started. Scans and other long operations appear here.';
export const LOAD_FAILED = 'Activity could not be read. It is tried again shortly.';
export const CANCEL_FAILED_TOAST = 'The job could not be cancelled. Try again.';
/** Whether an ended job recorded items for Needs review (a metadata scan's `scope.review_items`). */
export function hasReviewItems(job) {
const count = job?.scope?.review_items;
return job?.state !== 'running' && Number.isSafeInteger(count) && count > 0;
}
/** The concept's **Review metadata** (D87): opens Needs review. */
export function createReviewMetadataButton(scope) {
const button = createButton('Review metadata');
scope.listen(button, 'click', () => navigateTo('#/review'));
return button;
}
const CHIPS = Object.freeze({
running: ['blue', 'Running'], finished: ['green', 'Completed'], cancelled: ['blue', 'Cancelled'],
failed: ['blue', 'Failed'], lost: ['blue', 'Ended'],
@@ -62,9 +45,20 @@ export function clockTime(seconds) {
const join = parts => parts.filter(Boolean).join(' · ');
/** What a job covers ("2 output folders", "4 images") and what a cancelled one kept ("N added"), by kind (D111). */
const scopeLine = job => viewOf(job).scope(job);
const keptOf = job => viewOf(job).kept(job);
const whole = value => Number.isSafeInteger(value) && value >= 0;
/** What a job covers: "2 output folders" for a scan, "4 images" for image tagging (D89). */
function scopeLine(job) {
// D89: grouped as the server writes its lines ("19,622 images").
const images = job.scope?.images;
if (job.kind === 'image-tags') return whole(images) ? `${images.toLocaleString('en-US')} ${noun(images, 'image')}` : '';
const folders = job.scope?.output_folders;
return whole(folders) ? countOf(folders, 'output folder') : '';
}
/** What a cancelled job kept: "N added" for a scan, "N tagged" for image tagging (D89). */
const KEPT = Object.freeze({ __proto__: null, scan: addedOf, 'image-tags': taggedOf }); // D89
const keptOf = job => KEPT[job.kind]?.(job) ?? '';
/** The item's line under its title, by state. */
export function jobDetail(job) {
@@ -110,10 +104,7 @@ export function mountActivityTopbar({ store, scope, doc = document }) {
const parts = pillParts(job);
label.textContent = parts.label;
files.textContent = parts.files;
// D111 (S5 review M5): no total, no fill: the bar sweeps instead of showing an empty determinate 0%.
const unknown = indeterminate(job);
bar.classList.toggle('indeterminate', unknown);
fill.style.width = unknown ? '' : `${job.progress}%`;
fill.style.width = `${job.progress}%`;
pill.href = `#/activity/${job.id}`;
pill.setAttribute('aria-label', parts.text);
};
@@ -138,7 +129,6 @@ function itemFor(job, scope) {
const actions = element('div', undefined, 'row-actions');
const cancel = createButton('Cancel job');
actions.append(cancel);
const review = createReviewMetadataButton(scope); // D87
item.append(heading, detail, progress, actions);
scope.listen(cancel, 'click', async () => {
if (cancel.getAttribute('aria-disabled') === 'true') return;
@@ -161,22 +151,21 @@ function itemFor(job, scope) {
// parentNode, not isConnected: on the first render the item is not in the page yet, so a job that had already
// ended when Activity opened kept its Cancel job (and a running one's progress moved after its actions).
if (running) {
showProgress(progress, next); // D111: no value while the total is unknown
progress.value = next.progress;
progress.textContent = `${next.progress}%`;
progress.setAttribute('aria-label', `${next.title} progress`);
if (!progress.parentNode) item.insertBefore(progress, actions.parentNode ? actions : null);
} else {
progress.remove();
}
// Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel job.
if (canCancel(next)) {
if (running && next.cancellable) {
if (!actions.parentNode) item.append(actions);
} else if (actions.parentNode) {
// A focused Cancel job that goes away hands its focus to the item's title link, never to the page body.
if (actions.contains(document.activeElement)) link.focus();
actions.remove();
}
if (hasReviewItems(next)) { if (!review.isConnected) detail.after(review); } // D87
else review.remove();
};
return item;
}
-81
View File
@@ -1,81 +0,0 @@
/**
* The concept media card (docs/concepts/workspace/app.js `mediaCard` a396–399), shared by Images, every
* Library-embedded gallery and the Needs review queue: `article.media-card` > `.media-art` (the `.image-open` button
* holding the media, then `.image-overlays`) and `.media-info` (title, "W × H · EXT · Seed N", then
* `.image-card-actions` with Info and Tag). #249 D7–D11, D19.
*/
import { createButton } from '../shared/toolkit/core.js';
import { cardTitle, cardSubtitle, imageKey, mediaRatio } from './images-state.mjs';
import { createWorkspaceIcon } from './icons.mjs';
import { createTagButton } from './image-tagging.mjs';
import { paintStar } from './images-favorites.mjs';
function el(tag, text, className) {
const node = document.createElement(tag);
if (text !== undefined) node.textContent = text;
if (className) node.className = className;
return node;
}
/** The card's media: a video's first frame when it has no thumbnail, else the thumbnail or the image. */
export function cardMedia(image, lazy = false) {
const { resolveUrl } = window.ApiClient;
const video = image.is_video && !image.thumbnail_url;
const node = el(video ? 'video' : 'img');
node.src = resolveUrl(image.thumbnail_url || image.url);
if (video) { node.muted = true; node.playsInline = true; node.preload = 'metadata'; }
else {
// The concept card image: alt is the card title, with the listing's intrinsic size when known.
node.alt = cardTitle(image); node.decoding = 'async'; if (lazy) node.loading = 'lazy';
if ([image.width, image.height].every(value => Number.isFinite(value) && value > 0)) {
node.width = image.width; node.height = image.height;
}
}
return node;
}
/**
* One card for `image`. `checked` (a boolean) adds the D8 selection checkbox, `favoriteBusy` paints a pending star
* (D7, shown wherever the image reports `favorite`), `media: false` leaves the open button empty for a file that
* cannot be served, and `showInfo: false` hides the title and file line (D97).
*/
export function createImageCard(image, { checked = null, favoriteBusy = false, media = true, showInfo = true } = {}) {
const card = el('article', undefined, 'media-card');
card.dataset.imageKey = imageKey(image); card.setAttribute('role', 'listitem');
card.style.setProperty('--media-ratio', mediaRatio(image));
const art = el('div', undefined, 'media-art');
const open = createButton('', 'image-open');
open.setAttribute('aria-label', `Open ${image.filename}`);
if (media) open.append(cardMedia(image, true));
// D8: the selection checkbox (Images only: Library-embedded media has no selection bar), then D7: the star,
// wherever the listing reports favorites (Images and Library-embedded media).
const overlays = el('div', undefined, 'image-overlays');
if (typeof checked === 'boolean') {
const label = el('label'), box = el('input');
box.type = 'checkbox'; box.checked = checked;
box.setAttribute('aria-label', `Select ${image.filename}`);
label.append(box);
overlays.append(label);
}
if (typeof image.favorite === 'boolean') {
const star = createButton('', 'star-button');
star.setAttribute('aria-label', `Favorite ${image.filename}`);
paintStar(star, image.favorite, favoriteBusy);
overlays.append(star);
}
art.append(open, overlays);
const info = el('div', undefined, 'media-info');
const text = el('div');
const title = el('h3', cardTitle(image)); if (image.path) title.title = image.path;
text.append(title, el('p', cardSubtitle(image)));
// D97: `showInfo` false (System › Appearance) hides the title and file line; Info and Tag stay.
if (!showInfo) text.hidden = true;
const actions = el('div', undefined, 'image-card-actions');
const inspectButton = createButton('', 'icon-btn');
inspectButton.append(createWorkspaceIcon('info'));
inspectButton.setAttribute('aria-label', `Inspect ${image.filename}`);
actions.append(inspectButton, createTagButton(image));
info.append(text, actions);
card.append(art, info);
return card;
}
+55 -7
View File
@@ -1,6 +1,5 @@
import { createScope, createButton } from '../shared/toolkit/core.js';
import { createImageStream, withItem, imageKey, countLine, captionDate, scopeLabel, activeRefinements, initialImagesView } from './images-state.mjs';
import { createImageCard } from './image-card.mjs';
import { createImageStream, withItem, imageKey, countLine, captionDate, scopeLabel, activeRefinements, cardTitle, cardSubtitle, mediaRatio, initialImagesView } from './images-state.mjs';
import { layoutImages, visibleImages, gridMetrics, stacksCardInfo } from './image-layout.mjs';
import { mountFileInspector } from './file-inspector.mjs';
import { createInspectionController } from './inspection-state.mjs';
@@ -17,7 +16,7 @@ import { openMetadataScan } from './activity-scan.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { imagesFilters, imagesViewState } from './saved-views.mjs';
import { displayPreferences } from './preferences.mjs';
import { onImageTagsSaved } from './image-tagging.mjs';
import { createTagButton, onImageTagsSaved } from './image-tagging.mjs';
import { followImageTagSaves, openCardTagging } from './images-tagging.mjs';
import { favoriteBusy, paintStar, receiveFavorites, setFavoriteVisitTarget, toggleFavorite } from './images-favorites.mjs';
export { onFavoriteChange } from './images-favorites.mjs';
@@ -110,6 +109,7 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
parentScope.signal.addEventListener('abort', dispose, {once:true});
scope.cleanup(() => parentScope.signal.removeEventListener('abort', dispose));
if (parentScope.signal.aborted) { scope.dispose(); return {dispose}; }
const { resolveUrl } = window.ApiClient;
const sender = createPromptSender({scope});
let detailInspector;
// System › Appearance: the display defaults this visit starts from (preferences.mjs); null without gallery-settings.js.
@@ -271,6 +271,20 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
}});
scope.cleanup(() => mediaViewer.dispose());
function mediaElement(image, lazy = false) {
const video = image.is_video && !image.thumbnail_url;
const node = el(video ? 'video' : 'img');
node.src = resolveUrl(image.thumbnail_url || image.url);
if (video) { node.muted = true; node.playsInline = true; node.preload = 'metadata'; }
else {
// The concept card image: alt is the card title, with the listing's intrinsic size when known.
node.alt = cardTitle(image); node.decoding = 'async'; if (lazy) node.loading = 'lazy';
if ([image.width, image.height].every(value => Number.isFinite(value) && value > 0)) {
node.width = image.width; node.height = image.height;
}
}
return node;
}
function releaseVideo(video) {
video.pause(); video.removeAttribute('src'); video.load();
}
@@ -292,12 +306,46 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
if (!image) return;
mediaViewer.open({items:state.items,selectedKey:imageKey(image),trigger:document.activeElement});
}
/** The concept media card (image-card.mjs): the image opens the viewer, Info inspects (D7–D11, D19). */
/** The concept media card (app.js mediaCard): the image opens the viewer, Info inspects (D7–D11, D19). */
function cardFor(image) {
const key = imageKey(image);
const card = createImageCard(image, {checked:bar ? bar.isChecked(key) : null, favoriteBusy:favoriteBusy(key),
// D97: "Show image details on cards" off hides the title and file line; Info and Tag stay.
showInfo:preferences?.showImageInfo !== false});
const card = el('article', undefined, 'media-card');
card.dataset.imageKey = key; card.setAttribute('role', 'listitem');
card.style.setProperty('--media-ratio', mediaRatio(image));
const art = el('div', undefined, 'media-art');
const open = createButton('', 'image-open');
open.setAttribute('aria-label', `Open ${image.filename}`);
open.append(mediaElement(image, true));
// D8: the selection checkbox (Images only: Library-embedded media has no selection bar), then D7: the star,
// wherever the listing reports favorites (Images and Library-embedded media).
const overlays = el('div', undefined, 'image-overlays');
if (bar) {
const label = el('label'), box = el('input');
box.type = 'checkbox'; box.checked = bar.isChecked(key);
box.setAttribute('aria-label', `Select ${image.filename}`);
label.append(box);
overlays.append(label);
}
if (typeof image.favorite === 'boolean') {
const star = createButton('', 'star-button');
star.setAttribute('aria-label', `Favorite ${image.filename}`);
paintStar(star, image.favorite, favoriteBusy(key));
overlays.append(star);
}
art.append(open, overlays);
const info = el('div', undefined, 'media-info');
const text = el('div');
const title = el('h3', cardTitle(image)); title.title = image.path;
text.append(title, el('p', cardSubtitle(image)));
// D97: "Show image details on cards" off hides the title and file line; Info and Tag stay.
if (preferences?.showImageInfo === false) text.hidden = true;
const actions = el('div', undefined, 'image-card-actions');
const inspectButton = createButton('', 'icon-btn');
inspectButton.append(createWorkspaceIcon('info'));
inspectButton.setAttribute('aria-label', `Inspect ${image.filename}`);
actions.append(inspectButton, createTagButton(image));
info.append(text, actions);
card.append(art, info);
markSelected(card, key === selection);
return card;
}
+10 -38
View File
@@ -1,7 +1,7 @@
import '../api-client.js';
import { createButton, createScope } from '../shared/toolkit/core.js';
import { routes, mountRouter, summarizeStats, routeTitle, loadNavCounts,
LIBRARY_WRITE_COUNTS, NAV_COUNT_ROUTES, FAVORITE_WRITE_COUNTS, FAVORITE_COUNT_DELAY_MS, REVIEW_WRITE_COUNTS } from './state.mjs';
LIBRARY_WRITE_COUNTS, NAV_COUNT_ROUTES, FAVORITE_WRITE_COUNTS, FAVORITE_COUNT_DELAY_MS } from './state.mjs';
import { mountWorkspaceTheme } from './theme.mjs';
import { mountImages, onFavoriteChange } from './images.mjs';
import { mountLibrary, mountNewPromptAction, onLibraryWrite } from './library.mjs';
@@ -15,7 +15,6 @@ import { createActivityStore } from './activity-store.mjs';
import { mountActivityPage, mountActivityTopbar } from './activity.mjs';
import { mountActivityJobView } from './activity-job-view.mjs';
import { mountInspect } from './inspect.mjs';
import { mountReview } from './review.mjs';
import { createPromptSender } from './prompt-sender.mjs';
import { mountSystem } from './system.mjs';
@@ -86,14 +85,13 @@ function mountWorkspace() {
}
};
// SAVED VIEWS (#249 D16): the concept's saved-nav rows (search icon, name, no count), loaded with the shell and
// re-read after every change. /* D49 */ no "PROPOSED" badge. The first rows are the concept's Favorites (star icon,
// count, D51) and Needs review (review icon, the exact pending count, ruling R5), kept across every re-read so
// their counts stay.
const [favoritesLink, reviewLink] = navLinks([['favorites', 'star'], ['review', 'review']]);
savedNav.replaceChildren(favoritesLink, reviewLink);
// re-read after every change. /* D49 */ no "PROPOSED" badge; /* D50 */ no "Needs review" row. The first row is
// the concept's Favorites (star icon, count), kept across every re-read so its count stays (D51).
const [favoritesLink] = navLinks([['favorites', 'star']]);
savedNav.replaceChildren(favoritesLink);
const savedViews = createSavedViews(api, scope.signal);
scope.cleanup(savedViews.subscribe(({ views }) => {
savedNav.replaceChildren(favoritesLink, reviewLink, ...views.map(view => {
savedNav.replaceChildren(favoritesLink, ...views.map(view => {
const link = element('a');
link.append(createWorkspaceIcon('search'), document.createTextNode(view.name));
link.href = savedViewHash(view.id);
@@ -115,21 +113,13 @@ function mountWorkspace() {
countRequests.set(id, request);
// Each badge lands as its own read settles; a count that failed to refresh is removed, never left stale.
loadNavCounts(api, request.signal, (_, count) => {
if (!request.signal.aborted) showCount(id, count);
if (request.signal.aborted) return;
const link = sidebar.querySelector(`nav [data-route="${id}"]`);
link?.querySelector('.nav-count')?.remove();
if (link && Number.isSafeInteger(count)) link.append(element('span', count.toLocaleString(), 'nav-count'));
}, [id]);
}
}
function showCount(id, count) {
const link = sidebar.querySelector(`nav [data-route="${id}"]`);
link?.querySelector('.nav-count')?.remove();
if (link && Number.isSafeInteger(count)) link.append(element('span', count.toLocaleString(), 'nav-count'));
}
/** A count a page has just read itself: it replaces the badge, and an older read in flight no longer lands. */
function setCount(id, count) {
countRequests.get(id)?.abort();
countRequests.delete(id);
showCount(id, count);
}
scope.cleanup(() => { for (const request of countRequests.values()) request.abort(); });
// A confirmed Library or vocabulary write cannot change the Images count, so only Library and Tags are re-read.
scope.cleanup(onLibraryWrite(() => refreshCounts(LIBRARY_WRITE_COUNTS)));
@@ -141,17 +131,6 @@ function mountWorkspace() {
favoriteCountTimer = setTimeout(() => refreshCounts(FAVORITE_WRITE_COUNTS), FAVORITE_COUNT_DELAY_MS);
}));
scope.cleanup(() => clearTimeout(favoriteCountTimer));
// A metadata scan that ends may have added items to Needs review: re-read its count once per ended scan.
const runningScans = new Set();
scope.cleanup(activity.subscribe(() => {
let ended = false;
for (const job of activity.jobs()) {
if (job.kind !== 'scan') continue;
if (job.state === 'running') runningScans.add(job.id);
else if (runningScans.delete(job.id)) ended = true;
}
if (ended) refreshCounts(REVIEW_WRITE_COUNTS);
}));
refreshCounts(NAV_COUNT_ROUTES);
let viewScope;
let firstRender = true;
@@ -282,13 +261,6 @@ function mountWorkspace() {
else mountActivityPage(main, { store: activity, scope: view });
return;
}
if (route.id === 'review') {
// Needs review (ruling R5): a recorded decision re-reads the row's count, and the open page's total is the
// count (F3: a change the page hears of costs no count read).
mountReview(main, { api, scope: view, onDecision: () => refreshCounts(REVIEW_WRITE_COUNTS),
onTotal: total => setCount('review', total) });
return;
}
if (route.id === 'system') {
// The concept's System (app.js renderSystem): its heading description, section nav and Save row.
description.textContent = route.description;
-449
View File
@@ -1,449 +0,0 @@
/**
* Needs review state (#249 PR F): the pending queue, the selected item's sampler paths and one explicit decision, over
* the review routes (`py/api/review.py`).
*
* - The queue is read 40 at a time, oldest first; `total` is the exact pending count. The cursor is bound to the
* storage epoch only, so Load more keeps working while new items arrive.
* - Selecting an item reads its detail: one "Sampler <id> · <positive> / <negative>" option per candidate, unusable
* ones disabled, the first workable one chosen, then "Leave unresolved" (rulings R2/R3).
* - Apply takes a fresh capture of the file (`observe`), then posts the decision; the server releases a capture it
* used. A refused decision releases the capture and keeps the selection with the server's sentence as the warning;
* a restored storage or a changed queue reads the queue again. A request that got no answer keeps its capture and
* its exact body, and the next Apply of the same choice (`retry()`) resends that body once.
* - Leave for later writes nothing. Tagging the image (the shared Tag dialog) never changes the item.
* - `refresh()` re-reads the queue quietly when the server says the queue changed (F2: the review change stream:
* a scan or the monitor added items, another tab decided one). The open item, its choice and an unanswered decision
* stay; when the open item has left the queue the next one opens with a note. Refreshes asked for while one runs
* collapse into one more read. Every re-read keeps the loaded depth (F3): it reads page after page until it holds as
* many items as were shown, so Load more pages stay and the page does not jump.
* - `sync(version)` is the stream's `ready`: it re-reads only when the queue read last is older than the version the
* stream names (`{library_epoch, review_revision}`, the pair every list read answers), so a visit reads its queue
* once (F3). An unknown version re-reads.
* - A change heard while a decision is in flight is owed: it is read once the decision is refused or unanswered (a
* recorded one leaves the page). Load more asked for during a re-read waits for it and continues from the queue it
* read; a re-read started during Load more takes the click over and reads that page too.
* - Apply fences on the epoch of the detail it decides: an observe under another storage releases its capture and
* reads the queue and the item again. An unanswered decision that a re-read shows landed finishes as recorded.
*/
import { noun } from './plural.mjs';
export const REVIEW_URL = '/prompt_manager/review/items';
const OBSERVE_URL = '/prompt_manager/image-annotations/observe';
const CAPTURE_URL = '/prompt_manager/image-annotations/capture/';
export const PAGE_LIMIT = 40;
export const UNRESOLVED = 'unresolved';
export const COPY = Object.freeze({
title: 'Metadata review',
description: 'Uncertain evidence stays visible until you decide.',
notice: 'Multiple sampler paths were found. The existing image links were preserved. Choose the correct context or leave them unresolved.',
// D86 (plan D63, ruling R6): the empty queue's sentence.
empty: 'Ambiguous images are added when a metadata scan or a new generation finds more than one sampler path.',
panelTitle: 'Resolve the association',
leaveUnresolved: 'Leave unresolved',
// The states the concept lacks are text only (Global Constraints); D119.
loading: 'Loading the review queue…',
loadFailed: 'The review queue could not be read.',
detailLoading: 'Loading sampler paths…',
detailFailed: 'This item could not be read. Select it again to retry.',
moreFailed: 'More items could not be loaded. Try again.',
limited: 'This file has too many sampler paths to list. You can leave it unresolved.',
unavailable: 'This file is not under a current image source, so it cannot be opened. You can leave it unresolved.',
lost: 'The decision may not have been saved. Apply again to send the same decision once more.',
storageChanged: 'Prompt storage was restored; reload the review queue.',
// D119 (F2): the open item left the queue in a re-read, and a failed re-read.
decidedElsewhere: name => `${name} was decided elsewhere, so the next image is open.`,
refreshFailed: 'The review queue could not be read again. Reload the page to see the latest images.',
failed: 'The decision could not be saved. Try again.',
applied: 'Association accepted.',
leftUnresolved: 'Left unresolved.',
});
// Answers after which the queue itself is out of date: read it again.
const RELOAD = new Set(['storage-changed', 'review-conflict', 'review-missing']);
/** The notice heading: "No images need a decision", "1 image needs a decision", "3 images need a decision". */
export function noticeTitle(count) {
if (!count) return 'No images need a decision';
return `${count.toLocaleString('en-US')} ${noun(count, 'image')} ${count === 1 ? 'needs' : 'need'} a decision`;
}
/** One Sampler context option's text. */
export function optionLabel(candidate) {
return `Sampler ${candidate.sampler_id} · ${candidate.label}`;
}
const workable = candidate => candidate?.state === 'resolved' && typeof candidate.positive?.text === 'string' &&
typeof candidate.negative?.text === 'string';
/** The Sampler context options of a detail: its candidates (unusable ones disabled), then Leave unresolved. */
export function contextOptions(detail) {
const usable = detail.target !== null && !detail.unavailable;
const options = (Array.isArray(detail.candidates) ? detail.candidates : []).map(candidate => Object.freeze({
value: candidate.hash, label: optionLabel(candidate), disabled: !usable || !workable(candidate), candidate,
}));
options.push(Object.freeze({ value: UNRESOLVED, label: COPY.leaveUnresolved, disabled: false, candidate: null }));
return Object.freeze(options);
}
/** The prompt blocks of the chosen option: its two role texts, or the ambiguous status for Leave unresolved. */
function promptsOf(option) {
if (!option?.candidate) return { positive: { status: 'ambiguous' }, negative: { status: 'ambiguous' } };
const { positive, negative } = option.candidate;
return { positive: { status: 'resolved', text: positive.text }, negative: { status: 'resolved', text: negative.text } };
}
function warningOf(detail) {
if (detail.unavailable || detail.target === null) return COPY.unavailable;
if (detail.limited) return COPY.limited;
return '';
}
const isAbort = error => error?.name === 'AbortError';
// No answer at all (the network, or a body that was not the server's JSON): the decision may have been saved.
const isLost = error => !isAbort(error) && !error?.code && !(error?.status >= 400 && error?.status < 500);
const initial = () => ({
status: 'idle', epoch: null, total: 0, items: Object.freeze([]), nextCursor: null, more: 'idle',
selectedId: null, detail: null, detailStatus: 'idle', options: Object.freeze([]), choice: null, prompts: null,
phase: 'idle', warning: '', canRetry: false, toast: null,
});
/**
* @param {{api: {get: Function, post: Function, del: Function}}} options
* @returns the controller: `state`, `subscribe(listener)`, `load()`, `refresh()`, `loadMore()`, `select(id)`, `choose(value)`,
* `sync(version)`, `apply()`, `retry()`, `leave()`, `dispose()`.
*/
export function createReviewController({ api }) {
let state = Object.freeze(initial());
const listeners = new Set();
const lifetime = new AbortController();
let listRead = null, detailRead = null, pending = null, disposed = false;
let refreshing = null, refreshAgain = false;
// A change heard while a decision was in flight (I1); read when the phase is idle again.
let owedRefresh = false;
// Load more's own read, the one waiting for a re-read, and a click a re-read took over (I2).
let moreRead = null, moreTask = null, moreOwed = false, listing = null;
// What the first read owes once it answers: nothing, a re-read ('always'), or a re-read unless it is this version.
let afterLoad = null;
// The version of the queue read last: {epoch, revision}.
let shown = null;
function set(update) {
if (disposed) return;
state = Object.freeze({ ...state, ...update });
for (const listener of [...listeners]) listener(state);
}
const signalOf = controller => AbortSignal.any ? AbortSignal.any([lifetime.signal, controller.signal]) : controller.signal;
function release(token) {
if (!token) return;
api.del(`${CAPTURE_URL}${encodeURIComponent(token)}`, { keepalive: true }).catch(() => {});
}
function dropPending() {
release(pending?.body.capture_token);
pending = null;
}
const isVersion = version => typeof version?.library_epoch === 'string' && Number.isSafeInteger(version?.review_revision);
const current = version => isVersion(version) && shown !== null && shown.epoch === version.library_epoch &&
shown.revision >= version.review_revision;
/** One list page; `cursor` continues after the last item already read. */
function readPage(read, cursor) {
const query = new URLSearchParams({ limit: String(PAGE_LIMIT) });
if (cursor) query.set('cursor', cursor);
return api.get(`${REVIEW_URL}?${query}`, { signal: signalOf(read), cache: 'no-store' });
}
/**
* The queue down to `depth` items: the first page, then the next ones while fewer than `depth` are held. A restore
* met after the first page (its cursor names the old storage) reads once more from the first page.
*/
async function readQueue(read, depth) {
for (let attempt = 0; ; attempt += 1) {
const first = await readPage(read, null);
const items = [...(first.items ?? [])];
let { total } = first;
let next = first.next_cursor ?? null;
try {
while (next && items.length < depth) {
const page = await readPage(read, next);
const known = new Set(items.map(entry => entry.id));
items.push(...(page.items ?? []).filter(entry => !known.has(entry.id)));
total = page.total;
next = page.next_cursor ?? null;
}
} catch (error) {
if (attempt === 0 && RELOAD.has(error?.code)) continue;
throw error;
}
return { first, items, total, next };
}
}
/**
* Read the queue as deep as it is shown; `stale` re-reads the selected item's detail too (its revision or epoch
* moved). `quiet` (a refresh) keeps the page as it is while reading and after a failed read.
*/
function load(options) {
const run = listing = loadQueue(options);
run.finally(() => { if (listing === run) listing = null; });
return run;
}
async function loadQueue({ keepWarning = false, stale = false, quiet = false } = {}) {
listRead?.abort();
if (moreRead) { moreRead.abort(); moreRead = null; moreOwed = true; }
const read = listRead = new AbortController();
if (!quiet) set({ status: state.items.length ? state.status : 'loading', ...(keepWarning ? {} : { warning: '' }) });
let readOk = false;
try {
// One page, or as many as Load more had shown (and one page more for a click this read took over).
const depth = state.items.length + (moreOwed ? PAGE_LIMIT : 0);
const queue = await readQueue(read, depth > PAGE_LIMIT ? depth : 0);
if (read !== listRead || disposed) return;
moreOwed = false;
// M3: another storage (a restore) makes the open item's detail stale too.
if (state.epoch !== null && queue.first.library_epoch !== state.epoch) stale = true;
const items = Object.freeze(queue.items.map(entry => Object.freeze({ ...entry })));
const left = state.items.find(entry => entry.id === state.selectedId);
shown = Number.isSafeInteger(queue.first.review_revision)
? { epoch: queue.first.library_epoch, revision: queue.first.review_revision } : null;
set({ status: 'ready', epoch: queue.first.library_epoch, total: queue.total, items, nextCursor: queue.next,
more: 'idle' });
readOk = true;
const keep = items.find(entry => entry.id === state.selectedId);
// A re-read keeps the open item as it is (even a failed read of it, which has its own Retry).
if (keep && !stale && (quiet || state.detail)) return;
if (quiet && left && !keep && pending?.id === left.id) {
// M5: the decision that got no answer was recorded (the server used its capture).
const choice = pending.body.choice;
pending = null;
set({ phase: 'done', warning: '', canRetry: false,
toast: choice === UNRESOLVED ? COPY.leftUnresolved : COPY.applied });
} else if (quiet && left && !keep && items.length) {
set({ warning: COPY.decidedElsewhere(left.filename) });
await select(items[0].id, { keepWarning: true });
} else if (items.length) await select((keep ?? items[0]).id, { keepWarning });
else set({ selectedId: null, detail: null, detailStatus: 'idle', options: Object.freeze([]), choice: null, prompts: null });
} catch (error) {
if (read !== listRead || disposed || isAbort(error)) return;
// M7: a click this read took over can be made again.
moreOwed = false;
if (quiet && state.status === 'ready') set({ warning: COPY.refreshFailed, more: 'idle' });
else set({ status: 'error', more: 'idle' });
} finally {
// A change heard while the first read ran: that read may predate it, so read once more (unless the
// stream's ready named a version this read already holds).
const owed = afterLoad;
if (readOk && owed && !disposed) {
afterLoad = null;
if (owed === 'always' || !current(owed)) refresh();
}
}
}
/** The stream's `ready`: re-read only when the queue shown is older than `version`. */
function sync(version) {
if (disposed) return Promise.resolve();
if (state.status === 'loading') {
// A second version, or a change, while the first read runs: no longer one version to compare.
afterLoad = afterLoad === null && isVersion(version) ? version : 'always';
return Promise.resolve();
}
if (state.status === 'ready' && current(version)) return Promise.resolve();
return refresh();
}
/** Re-read the queue after a change elsewhere; calls made while one runs collapse into one more read. */
function refresh() {
if (disposed) return Promise.resolve();
// Before the queue has been read there is nothing to keep: the first read (or Retry) reads the latest.
if (state.status !== 'ready') {
if (state.status === 'loading') afterLoad = 'always';
return Promise.resolve();
}
if (refreshing) {
refreshAgain = true;
return refreshing;
}
refreshing = (async () => {
do {
refreshAgain = false;
// A decision in flight finishes first: the page leaves when it is recorded, and a refused or
// unanswered one reads the owed change (I1).
if (state.phase === 'applying') { owedRefresh = true; return; }
await load({ keepWarning: true, quiet: true });
} while (refreshAgain && !disposed);
})().finally(() => { refreshing = null; });
return refreshing;
}
function loadMore() {
if (moreTask) return moreTask;
if (!state.nextCursor || state.more === 'loading') return Promise.resolve();
set({ more: 'loading' });
moreTask = readMore().finally(() => { moreTask = null; });
return moreTask;
}
async function readMore() {
// I2: a re-read in flight finishes first; the next page continues after the queue it read.
while ((refreshing || listing) && !disposed) await (refreshing ?? listing);
if (disposed || !state.nextCursor || state.status !== 'ready') {
if (state.more === 'loading') set({ more: 'idle' });
return;
}
if (state.more !== 'loading') set({ more: 'loading' });
const read = moreRead = new AbortController();
try {
const query = new URLSearchParams({ limit: String(PAGE_LIMIT), cursor: state.nextCursor });
const page = await api.get(`${REVIEW_URL}?${query}`, { signal: signalOf(read), cache: 'no-store' });
if (read !== moreRead || disposed) return;
moreRead = null;
const known = new Set(state.items.map(entry => entry.id));
const added = (page.items ?? []).filter(entry => !known.has(entry.id)).map(entry => Object.freeze({ ...entry }));
set({ items: Object.freeze([...state.items, ...added]), total: page.total, nextCursor: page.next_cursor ?? null,
more: 'idle' });
} catch (error) {
if (read !== moreRead || disposed || isAbort(error)) return;
moreRead = null;
if (RELOAD.has(error?.code)) { set({ more: 'idle' }); await load({ stale: true }); return; }
set({ more: 'error' });
}
}
async function select(id, { keepWarning = false } = {}) {
if (!state.items.some(entry => entry.id === id) || state.phase === 'applying') return;
detailRead?.abort();
dropPending();
const read = detailRead = new AbortController();
set({ selectedId: id, detail: null, detailStatus: 'loading', options: Object.freeze([]), choice: null, prompts: null,
canRetry: false, ...(keepWarning ? {} : { warning: '' }) });
try {
const detail = await api.get(`${REVIEW_URL}/${encodeURIComponent(id)}`, { signal: signalOf(read), cache: 'no-store' });
if (read !== detailRead || disposed) return;
const options = contextOptions(detail);
const chosen = options.find(option => !option.disabled);
set({ detail: Object.freeze(detail), detailStatus: 'ready', options, choice: chosen.value, prompts: promptsOf(chosen),
warning: (keepWarning && state.warning) || warningOf(detail) });
} catch (error) {
if (read !== detailRead || disposed || isAbort(error)) return;
if (RELOAD.has(error?.code)) { set({ warning: error.message }); await load({ keepWarning: true, stale: true }); return; }
set({ detailStatus: 'error', warning: COPY.detailFailed });
}
}
function choose(value) {
const option = state.options.find(entry => entry.value === value);
if (!option || option.disabled || state.phase === 'applying') return;
// Another choice is another decision: an unanswered one is no longer resent.
if (pending && pending.body.choice !== value) { dropPending(); set({ canRetry: false }); }
set({ choice: value, prompts: promptsOf(option) });
}
/** The phase is idle again after a refused or unanswered decision: read a change heard meanwhile (I1). */
function idleAgain(reloading) {
const owed = owedRefresh;
owedRefresh = false;
if (owed && !reloading && !disposed) refresh();
}
/** Send `body`; true once the decision is recorded. */
async function send(body) {
const id = state.selectedId;
const choice = body.choice;
set({ phase: 'applying', canRetry: false });
try {
const result = await api.post(`${REVIEW_URL}/${encodeURIComponent(id)}/decision`, body, { signal: lifetime.signal });
pending = null;
set({ phase: 'done', total: result.total, warning: '',
toast: choice === UNRESOLVED ? COPY.leftUnresolved : COPY.applied });
return true;
} catch (error) {
// Leaving the page mid-request: the server may never see it, so the capture is released here (a release
// of a capture the server already used is a harmless 404).
if (disposed || isAbort(error)) { release(body.capture_token); return false; }
if (isLost(error)) {
pending = { id, body };
set({ phase: 'idle', warning: COPY.lost, canRetry: true });
idleAgain(false);
return false;
}
pending = null;
release(body.capture_token);
set({ phase: 'idle', warning: error?.message || COPY.failed });
idleAgain(RELOAD.has(error?.code));
if (RELOAD.has(error?.code)) await load({ keepWarning: true, stale: true });
return false;
}
}
async function apply() {
const { detail, choice } = state;
if (state.phase !== 'idle' || state.detailStatus !== 'ready' || !detail || !choice) return false;
// After an unanswered request, Apply resends that same decision once (its capture and body).
if (pending?.id === state.selectedId && pending.body.choice === choice) return retry();
dropPending();
set({ phase: 'applying', warning: '', canRetry: false });
let capture = null;
if (detail.target) {
let observed;
try {
// M4: not aborted on leave, so the capture it answers can be released.
observed = await api.post(OBSERVE_URL, { target: detail.target });
} catch (error) {
if (disposed || isAbort(error)) return false;
set({ phase: 'idle', warning: error?.message || COPY.failed });
idleAgain(RELOAD.has(error?.code));
if (RELOAD.has(error?.code)) await load({ keepWarning: true, stale: true });
return false;
}
capture = observed.capture;
if (disposed) { release(capture?.token); return false; }
// M3: the file was observed under another storage than the detail was read from.
if (observed.library_epoch !== detail.library_epoch) {
release(capture?.token);
set({ phase: 'idle', warning: COPY.storageChanged });
idleAgain(true);
await load({ keepWarning: true, stale: true });
return false;
}
}
return send({
expected_epoch: detail.library_epoch, expected_revision: detail.revision, capture_token: capture?.token ?? null,
sha256: capture?.sha256 ?? detail.sha256, dependency_fingerprint: detail.dependency_fingerprint, choice,
});
}
/** Resend the unanswered decision once, with its own capture; false when there is nothing to resend. */
async function retry() {
if (!pending || state.phase !== 'idle' || pending.id !== state.selectedId) return false;
const { body } = pending;
pending = null;
return send(body);
}
function leave() {
if (state.phase === 'applying') return;
dropPending();
set({ phase: 'done', toast: null, canRetry: false });
}
return {
get state() { return state; },
subscribe(listener) {
listeners.add(listener);
return () => listeners.delete(listener);
},
load: () => load(), refresh, sync, loadMore, select: id => select(id), choose, apply, retry, leave,
dispose() {
if (disposed) return;
dropPending();
disposed = true;
lifetime.abort();
listeners.clear();
},
};
}
-347
View File
@@ -1,347 +0,0 @@
/**
* Needs review (#249 PR F): the concept's Metadata review page (docs/concepts/workspace/app.js `renderReview`
* a749–760; style.css s407–415, s796–814 and their breakpoints) over the review controller (review-state.mjs).
*
* - Heading "Metadata review", its line, and **Open activity** (`#/activity`).
* - `.notice`: "N images need a decision" and the concept's sentence; with an empty queue, D86 (plan D63, ruling R6)
* "No images need a decision" and the scan/new-generation sentence, and nothing else.
* - `.gallery-layout`: the queue's concept media cards (image-card.mjs; the image opens the shared viewer, Info selects
* the item, Tag opens the shared Tag image dialog with the review scope note), then `section.surface` "Resolve the
* association": filename, Tag image, Sampler context, the chosen path's prompt blocks and the actions Leave for later
* and Apply association. Both return to Images (ruling R7).
* - States the concept lacks (loading, a failed read, a refused or unanswered decision, a limited or unavailable file)
* are text: `p.warning-note` in the panel, or the page's status line (D119).
* - The open page re-reads its queue on each `change` of the review change stream (`/prompt_manager/review/events`,
* F2): items a scan or the monitor added, and decisions made in other tabs. Each `ready` (the first one, and one
* after the browser reconnects a dropped stream) names the queue's version, and the page re-reads only when the
* queue it shows is older (F3: one read per visit). A re-read keeps the depth Load more had reached. Nothing polls.
* - The stream lives only while the tab is visible: a hidden tab closes it and a visible one opens it again, so
* background tabs hold no connection (I4); the reopened stream's `ready` reads only when the queue changed meanwhile.
* A stream the browser gave up on (a 503 or a proxy error on its reconnect) is opened again after 3 s, doubling to
* 30 s; while the stream is down the page says so in a quiet status line (D120, I3).
* - A re-read refreshes each card whose media, target or availability changed (M6). Cards show the Images listing's
* thumbnail when the server knows one (M10).
* - The SAVED VIEWS row's count follows the total the page shows (`onTotal`), so a change costs no count read.
* - D119: the queue cards carry no selection checkbox and no star (this page has no selection bar, and the queue
* carries no favorite state); more than 40 items add the Images page's `.infinite-continuation` Load more after the
* layout; the toasts "Association accepted." and "Left unresolved." stand in for the concept's prototype toast.
*/
import { createButton } from '../shared/toolkit/core.js';
import { createReviewController, COPY, noticeTitle } from './review-state.mjs';
import { createImageCard } from './image-card.mjs';
import { isTagButton, openImageTagging } from './image-tagging.mjs';
import { mountPromptBlocks } from './prompt-blocks.mjs';
import { createPromptSender } from './prompt-sender.mjs';
import { createMediaViewer } from './media-viewer.mjs';
import { createInspectionController } from './inspection-state.mjs';
import { createWorkspaceIcon } from './icons.mjs';
import { field } from './workspace-dialogs.mjs';
import { pageHeading } from './heading.mjs';
import { navigateTo } from './state.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { imageKey } from './images-state.mjs';
export const REVIEW_EVENTS_URL = '/prompt_manager/review/events';
// I3: reopening a stream the browser closed for good waits this long, doubling up to the maximum.
export const REOPEN_FIRST_MS = 3000;
export const REOPEN_MAX_MS = 30000;
// D120: the page's line while the change stream is down (the concept has no such state).
export const NOT_LIVE = 'Live updates are paused: changes made elsewhere appear once the connection returns.';
const openEvents = url => new EventSource(window.ApiClient.resolveUrl(url));
function el(tag, text, className) {
const node = document.createElement(tag);
if (text !== undefined) node.textContent = text;
if (className) node.className = className;
return node;
}
function setText(node, text) {
if (node.textContent !== text) node.textContent = text;
}
function setDisabled(button, disabled) {
if (disabled) button.setAttribute('aria-disabled', 'true');
else button.removeAttribute('aria-disabled');
}
const isDisabled = button => button.getAttribute('aria-disabled') === 'true';
/** A queue item as the card, the viewer and the Tag dialog read a listed image. */
export function imageOf(item) {
const target = item.target ?? {};
return { filename: item.filename, root_index: target.root_index, root_scope: target.root_scope, path: target.path,
url: item.url, thumbnail_url: item.thumbnail_url ?? null, is_video: item.is_video === true,
media_type: item.is_video ? 'video' : 'image' };
}
/** True when a re-read changed what a card shows or can open (M6). */
const cardChanged = (before, after) => before.url !== after.url || before.thumbnail_url !== after.thumbnail_url ||
before.unavailable !== after.unavailable || JSON.stringify(before.target) !== JSON.stringify(after.target);
/** The queue version a `ready` event names, or null (the page then re-reads). */
export function readyVersion(event) {
try {
const version = JSON.parse(event?.data ?? 'null');
return version && typeof version === 'object' ? version : null;
} catch {
return null;
}
}
/**
* Mount the page under the shell's heading. `onDecision()` runs after a recorded decision and `onTotal(total)` each
* time the pending total the page shows changes (the shell's Needs review count); `controller` and `connect` (the
* change stream's EventSource) are injectable for tests.
*/
export function mountReview(main, { api, scope, onDecision = () => {}, onTotal,
connect = openEvents, controller = createReviewController({ api }) }) {
scope.cleanup(() => controller.dispose());
const heading = pageHeading(main);
heading.h1.textContent = COPY.title;
heading.description.textContent = COPY.description;
const openActivity = createButton('Open activity');
heading.actions.append(openActivity);
scope.listen(openActivity, 'click', () => navigateTo('#/activity'));
const status = el('p', COPY.loading, 'empty-state');
status.setAttribute('role', 'status');
const retryLoad = createButton('Retry');
const notice = el('div', undefined, 'notice');
const noticeCopy = el('div');
const noticeTitleNode = el('h2');
const noticeText = el('p');
noticeCopy.append(noticeTitleNode, noticeText);
notice.append(noticeCopy);
const layout = el('div', undefined, 'gallery-layout');
const grid = el('div', undefined, 'image-grid review-queue');
grid.setAttribute('role', 'list');
grid.setAttribute('aria-label', 'Review queue');
const panel = el('section', undefined, 'surface review-panel');
const panelTitle = el('h2', COPY.panelTitle);
panelTitle.id = 'review-panel-title';
panel.setAttribute('aria-labelledby', panelTitle.id);
const filename = el('p', '', 'small');
const tagRow = el('div', undefined, 'row-actions spaced-15');
const tagButton = createButton('');
tagButton.append(createWorkspaceIcon('tagImage'), ' Tag image');
tagRow.append(tagButton);
const select = el('select');
const contextField = field('Sampler context', select, 'review-context');
const warning = el('p', '', 'warning-note'); // D119: the states the concept lacks
warning.setAttribute('role', 'status');
const blocks = el('div');
const actions = el('div', undefined, 'dialog-actions');
const later = createButton('Leave for later');
const apply = createButton('Apply association', 'ca-primary');
actions.append(later, apply);
panel.append(panelTitle, filename, tagRow, contextField, warning, blocks, actions);
layout.append(grid, panel);
const live = el('p', NOT_LIVE, 'small review-live'); // D120: the change stream is down
live.setAttribute('role', 'status');
live.hidden = true;
heading.description.after(live);
const continuation = el('div', undefined, 'infinite-continuation'); // D119: Load more past 40 items
const moreText = el('p', '', 'small');
const more = createButton('Load more images');
continuation.append(moreText, more);
const sender = createPromptSender({ scope });
const prompts = mountPromptBlocks(blocks, { prompts: null, sender, parentScope: scope, includePairAction: false,
onRetry: () => { if (controller.state.selectedId) controller.select(controller.state.selectedId); } });
const inspection = createInspectionController({ api });
scope.cleanup(() => inspection.dispose());
const viewer = createMediaViewer({ api, container: main, parentScope: scope, inspectionController: inspection,
sender, onSelect: image => inspection.select(image) });
const cards = new Map();
let shownItems = null, shownSelected, shownOptions = null, shownPrompts, shownDetail;
function createCard(item) {
// D119: no selection checkbox and no star on a queue card.
const card = createImageCard(imageOf(item), { media: Boolean(item.url) });
card.dataset.reviewId = item.id;
card.item = item;
if (!item.url) setDisabled(card.querySelector('.image-open'), true);
if (!item.target) setDisabled(card.querySelector('.image-card-actions button:last-child'), true);
cards.set(item.id, card);
return card;
}
function renderCards(state) {
if (state.items !== shownItems) {
shownItems = state.items;
const ids = new Set(state.items.map(item => item.id));
for (const [id, card] of cards) if (!ids.has(id)) { card.remove(); cards.delete(id); }
state.items.forEach((item, index) => {
let card = cards.get(item.id);
if (card && cardChanged(card.item, item)) {
// M6: a root came or went: the card's media, Open and Tag follow the re-read.
const fresh = createCard(item);
card.replaceWith(fresh);
card = fresh;
} else if (card) card.item = item;
if (!card) card = createCard(item);
if (grid.children[index] !== card) grid.insertBefore(card, grid.children[index] ?? null);
});
shownSelected = undefined;
}
if (state.selectedId !== shownSelected) {
shownSelected = state.selectedId;
for (const [id, card] of cards) card.classList.toggle('selected', id === state.selectedId);
}
}
function renderPanel(state) {
const item = state.items.find(entry => entry.id === state.selectedId);
setText(filename, item?.filename ?? '');
setDisabled(tagButton, !item?.target || state.phase === 'applying');
if (state.options !== shownOptions) {
shownOptions = state.options;
select.replaceChildren(...state.options.map(option => {
const node = el('option', option.label);
node.value = option.value;
node.disabled = option.disabled;
return node;
}));
}
if (state.choice !== null && select.value !== state.choice) select.value = state.choice;
select.disabled = state.detailStatus !== 'ready' || state.phase === 'applying';
setText(warning, state.warning);
warning.hidden = !state.warning;
if (state.prompts !== shownPrompts || state.detailStatus !== shownDetail) {
shownPrompts = state.prompts;
shownDetail = state.detailStatus;
if (state.detailStatus === 'ready') prompts.setState({ prompts: state.prompts });
else prompts.setState({ loading: state.detailStatus === 'loading', error: state.detailStatus === 'error' });
}
setDisabled(apply, state.detailStatus !== 'ready' || state.phase !== 'idle');
setDisabled(later, state.phase === 'applying');
}
function render(state) {
if (scope.signal.aborted) return;
if (state.phase === 'done') return finish(state);
const ready = state.status === 'ready';
if (!ready) {
setText(status, state.status === 'error' ? COPY.loadFailed : COPY.loading);
status.setAttribute('role', state.status === 'error' ? 'alert' : 'status');
if (!status.isConnected) main.append(status);
if (state.status === 'error' && !retryLoad.isConnected) main.append(retryLoad);
if (state.status !== 'error') retryLoad.remove();
if (state.status === 'error' || !state.items.length) { notice.remove(); layout.remove(); continuation.remove(); }
return;
}
status.remove();
retryLoad.remove();
setText(noticeTitleNode, noticeTitle(state.total));
setText(noticeText, state.total ? COPY.notice : COPY.empty); // D86
if (!notice.isConnected) main.append(notice);
if (!state.items.length) { layout.remove(); continuation.remove(); return; }
if (!layout.isConnected) notice.after(layout);
renderCards(state);
renderPanel(state);
if (state.nextCursor) {
setText(moreText, `${state.items.length.toLocaleString('en-US')} of ${state.total.toLocaleString('en-US')} images`);
setDisabled(more, state.more === 'loading');
if (state.more === 'error') setText(moreText, COPY.moreFailed);
if (!continuation.isConnected) layout.after(continuation);
} else continuation.remove();
}
let finished = false;
function finish(state) {
if (finished) return;
finished = true;
if (state.toast) {
showWorkspaceToast(state.toast);
onDecision();
}
// Ruling R7: Apply association and Leave for later both return to Images.
navigateTo('#/images');
}
scope.listen(retryLoad, 'click', () => controller.load());
scope.listen(more, 'click', () => { if (!isDisabled(more)) controller.loadMore(); });
scope.listen(select, 'change', () => controller.choose(select.value));
scope.listen(apply, 'click', () => { if (!isDisabled(apply)) controller.apply(); });
scope.listen(later, 'click', () => { if (!isDisabled(later)) controller.leave(); });
scope.listen(tagButton, 'click', () => {
const item = controller.state.items.find(entry => entry.id === controller.state.selectedId);
if (isDisabled(tagButton) || !item?.target) return;
openImageTagging({ api, scope, image: imageOf(item), source: 'review', trigger: tagButton });
});
scope.listen(grid, 'click', event => {
const card = event.target.closest('article[data-review-id]');
if (!card) return;
const { item } = card;
const button = event.target.closest('button');
if (!button || isDisabled(button)) return;
if (isTagButton(button)) {
openImageTagging({ api, scope, image: imageOf(item), source: 'review', trigger: button });
return;
}
if (button.matches('.image-open')) {
const items = controller.state.items.filter(entry => entry.url).map(imageOf);
viewer.open({ items, selectedKey: imageKey(imageOf(item)), trigger: button });
return;
}
// Info selects the item, as the concept's `inspect` does on this page (app.js a1244–1248).
controller.select(item.id);
});
let shownTotal = null;
const reportTotal = state => {
if (state.status !== 'ready' || state.phase === 'done' || state.total === shownTotal) return;
shownTotal = state.total;
onTotal?.(state.total);
};
scope.cleanup(controller.subscribe(render));
scope.cleanup(controller.subscribe(reportTotal));
render(controller.state);
controller.load();
connectEvents({ scope, connect, controller, live });
return { controller };
}
/**
* The change stream while the page is mounted: open while the tab is visible (I4), reopened with a bounded backoff
* after the browser gives up on it (I3), with `live` shown while it is down.
*/
function connectEvents({ scope, connect, controller, live }) {
let events = null, timer = null, delay = REOPEN_FIRST_MS;
const setLive = up => { live.hidden = up; };
function close() {
clearTimeout(timer);
timer = null;
events?.close();
events = null;
}
function open() {
if (events || scope.signal.aborted || document.hidden) return;
const source = events = connect(REVIEW_EVENTS_URL);
source.addEventListener('ready', event => {
if (source !== events || scope.signal.aborted) return;
delay = REOPEN_FIRST_MS;
setLive(true);
controller.sync(readyVersion(event));
});
source.addEventListener('change', () => { if (source === events && !scope.signal.aborted) controller.refresh(); });
source.addEventListener('error', () => {
if (source !== events || scope.signal.aborted) return;
setLive(false);
// The browser retries a dropped connection itself; one it closed for good is reopened here.
if (source.readyState !== 2) return; // EventSource.CLOSED
events = null;
source.close();
timer = setTimeout(() => { timer = null; open(); }, delay);
delay = Math.min(delay * 2, REOPEN_MAX_MS);
});
}
scope.listen(document, 'visibilitychange', () => {
if (document.hidden) close();
else open();
});
scope.cleanup(close);
open();
}
+2 -10
View File
@@ -12,10 +12,6 @@ export const routes = [
{ id: 'overview', label: 'Overview' },
// SAVED VIEWS' first row (#249 D51): Images filtered to favorites. No href, so Overview lists no card for it.
{ id: 'favorites', label: 'Favorites' },
// SAVED VIEWS' Needs review row (ruling R5): the concept's Metadata review page (app.js renderReview 749–760; its
// breadcrumb is "Review", app.js render 1175–1188),
// the queue of ambiguous sampler evidence. No href: no Overview card.
{ id: 'review', label: 'Needs review', title: 'Review' },
// The concept's Activity (app.js renderActivity 732–738): this server session's jobs. No href: no Overview card.
{ id: 'activity', label: 'Activity' },
// Native System (#249 System S2): `#/system/<section>`. Q12: Overview's card opens its Overview section (`native`:
@@ -25,7 +21,7 @@ export const routes = [
description: 'Configuration, maintenance and recovery—with browsing kept in the workspace.', action: 'Open System' },
];
const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'inspect', 'overview', 'saved', 'favorites', 'activity', 'review', 'system']);
const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'inspect', 'overview', 'saved', 'favorites', 'activity', 'system']);
/**
* System's sections in the concept's order (app.js `systemSections` 635–644); the concept's `general` is production's
@@ -92,8 +88,6 @@ const NAV_COUNT_SOURCES = Object.freeze([
['favorites', '/prompt_manager/images/output?mode=workspace&favorites=1&limit=1'],
['library', '/prompt_manager/library?limit=1'],
['tags', '/prompt_manager/vocabulary/tags?limit=1'],
// Needs review (ruling R5): the queue's `total` is the exact pending count.
['review', '/prompt_manager/review/items?limit=1'],
]);
/** Route id -> count for the sidebar links; null when a page is missing or its total is not a count. */
@@ -108,7 +102,7 @@ export function summarizeNavCounts(pages = {}) {
* Read the counts; a failed read yields null for its route only. `onCount(id, count)` runs as each read settles, so a
* slow listing (the Images output scan) never holds back the cheap Library and Tags counts.
*/
/** The sidebar routes that carry a count (WORKSPACE, plus SAVED VIEWS' Favorites and Needs review). */
/** The sidebar routes that carry a count (WORKSPACE, plus SAVED VIEWS' Favorites). */
export const NAV_COUNT_ROUTES = Object.freeze(NAV_COUNT_SOURCES.map(([id]) => id));
/** The counts a confirmed Library or vocabulary write can change; the Images listing cannot. */
@@ -116,8 +110,6 @@ export const LIBRARY_WRITE_COUNTS = Object.freeze(['library', 'tags']);
/** The count a confirmed favorite change can change. */
export const FAVORITE_WRITE_COUNTS = Object.freeze(['favorites']);
/** The count a review decision or a finished metadata scan can change. */
export const REVIEW_WRITE_COUNTS = Object.freeze(['review']);
// Trailing delay that coalesces quick star toggles into one Favorites count read (#249 M1).
export const FAVORITE_COUNT_DELAY_MS = 300;
+8 -9
View File
@@ -7,7 +7,6 @@
* N issues / Incomplete); the review, backup, thumbnail and sources rows say what the server reports, and nothing
* reads 0 before the read answers: a load shows a status line, a failure an alert and Retry. The thumbnail row is
* its own read of `/images/thumbnails/status` (a walk of every source): "Checking thumbnails…" until it answers.
* D99: its Review thumbnails opens the same "Thumbnail maintenance" dialog as Sources (thumbnail-review.mjs).
* - D96: the Statistics dialog lists the real runs and average rating per category, then "Library totals".
* - D95: the "Health report" lists every check the server ran, Missing files per image source, with a chip per real
* result and when it was checked. The check runs from Maintenance's notice ("Run health checks", D123) as a
@@ -21,7 +20,6 @@ import { closeWorkspaceDialog, openContentDialog } from './workspace-dialogs.mjs
import { showWorkspaceToast } from './toast.mjs';
import { noun } from './plural.mjs';
import { operationRow } from './system-controls.mjs';
import { openThumbnailReview } from './thumbnail-review.mjs';
export const OVERVIEW_URL = '/prompt_manager/system/overview';
export const STATISTICS_URL = '/prompt_manager/system/statistics';
@@ -319,20 +317,22 @@ function surface(heading, rows) {
}
/** The Overview's nodes, and its thumbnail row (painted by its own read). */
function overviewNodes(overview, { scope, navigate, openStatistics, openThumbnails }) {
function overviewNodes(overview, { scope, navigate, openStatistics }) {
const go = section => () => navigate(`#/system/${section}`);
const review = operationRow({ scope, title: ambiguousTitle(overview.needs_review),
description: Number.isSafeInteger(overview.needs_review?.pending)
? 'Review metadata before assigning a positive / negative pair.' : NOT_AVAILABLE,
label: 'Review metadata', onClick: () => navigate('#/review') });
label: 'Review metadata', onClick: () => {} });
// D93: the row reads the queue's pending count; its action waits for the Needs review page (#288).
review.querySelector('button').setAttribute('aria-disabled', 'true');
const [sourcesTitle, sourcesDescription] = sourcesCopy(overview.sources);
const thumbnails = operationRow({ scope, title: THUMBNAILS_LOADING, description: 'Generate only missing previews when necessary.',
label: 'Review thumbnails', onClick: openThumbnails });
label: 'Review thumbnails', onClick: go('sources') });
const health = surface('Library health', [
review,
operationRow({ scope, title: 'Backup status', description: backupText(overview.backup), label: 'View backups',
onClick: go('backups') }),
// D99: Review thumbnails opens the thumbnail dialog (the concept's row starts a sample job directly).
// Until the thumbnail review dialog (S5), Review thumbnails opens Sources, where thumbnail generation lives.
thumbnails,
]);
const preferences = surface('Workspace preferences', [
@@ -352,7 +352,7 @@ function overviewNodes(overview, { scope, navigate, openStatistics, openThumbnai
* answer replaces in place (the concept's metrics and columns sit directly in `.settings-layout`). A read that a newer
* one superseded, or one answering after the section left, paints nothing.
*/
export function mountOverview(container, { api, activity, scope, navigate = navigateTo, notify = showWorkspaceToast,
export function mountOverview(container, { api, scope, navigate = navigateTo, notify = showWorkspaceToast,
latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) {
const overviewRead = latestOnly({ lifetime: scope.signal });
const statisticsRead = latestOnly({ lifetime: scope.signal });
@@ -394,8 +394,7 @@ export function mountOverview(container, { api, activity, scope, navigate = navi
try {
const overview = await overviewRead(signal => api.get(OVERVIEW_URL, { signal, cache: 'no-store' }));
if (overview === STALE || scope.signal.aborted) return;
const { nodes, thumbnails } = overviewNodes(overview, { scope, navigate, openStatistics,
openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate }) });
const { nodes, thumbnails } = overviewNodes(overview, { scope, navigate, openStatistics });
thumbnailRow = thumbnails;
sources = overview.sources;
swap(nodes);
+7 -5
View File
@@ -14,8 +14,8 @@
* it to the host draft of `gallery_root_paths`; Remove takes one out. Both wait for the one Save row.
* - "Watch for new outputs" (`monitoring_enabled`) and "Worker threads" (`worker_threads`, a range up to the server's
* `cpu_count`, D113) are host drafts too. "Enrich existing metadata" reviews a scan of every source.
* - "Thumbnail generation → Review options" opens "Thumbnail maintenance" (thumbnail-review.mjs, S5 Task 14), which
* starts the thumbnail Activity job.
* - "Thumbnail generation → Review options" opens the existing gallery's thumbnail tool until its review dialog (S5),
* the D94 hand-off for the one capability not native yet.
*
* D98: per-source Remove; an added source reads "not saved yet" until Save; the line under the list says when
* ComfyUI's output folder is used instead; "restart pending" for a source the running monitor does not watch yet; the
@@ -28,7 +28,6 @@
import { createButton } from '../shared/toolkit/core.js';
import { closeWorkspaceDialog, field, openContentDialog } from './workspace-dialogs.mjs';
import { openMetadataScan } from './activity-scan.mjs';
import { openThumbnailReview } from './thumbnail-review.mjs';
import { settingRow, operationRow } from './system-controls.mjs';
import { HOST_UNAVAILABLE } from './system-appearance.mjs';
import { showWorkspaceToast } from './toast.mjs';
@@ -37,6 +36,7 @@ import { noun } from './plural.mjs';
const SETTINGS_URL = '/prompt_manager/settings';
const VALIDATE_URL = '/prompt_manager/settings/validate-source';
const RESCAN_URL = '/prompt_manager/gallery/rescan';
const GALLERY_URL = '/prompt_manager/gallery';
const ROOTS_KEY = 'gallery_root_paths';
export const NOTICE_KEY = 'pm_update_notice_dismissed';
@@ -222,7 +222,8 @@ const controlOf = (row, id) => row.querySelector(`[data-setting="${id}"]`);
* Activity store (scans start through it), `drafts` the System drafts.
*/
export function mountSources(container, { api, activity, scope, drafts, notify = showWorkspaceToast, storage = defaultStorage,
latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) {
openWindow = (url) => window.open(url, '_blank', 'noopener'), latestOnly = window.ApiClient.latestOnly,
STALE = window.ApiClient.STALE }) {
const statusRead = latestOnly({ lifetime: scope.signal });
// undefined while read, null when the read failed, else GET /settings' `sources`.
let sources;
@@ -269,8 +270,9 @@ export function mountSources(container, { api, activity, scope, drafts, notify =
surface.append(monitoring, workers,
operationRow({ scope, title: 'Enrich existing metadata', description: 'Resolve both prompts, deduplicate recipes, preserve uncertain links.',
label: 'Review scan', onClick: scanAll }),
// D94: the gallery keeps thumbnail generation until its review dialog (S5).
operationRow({ scope, title: 'Thumbnail generation', description: 'Generate missing previews; rebuilding all is a separate explicit option.',
label: 'Review options', onClick: trigger => openThumbnailReview({ api, store: activity, scope, trigger }) }),
label: 'Review options', onClick: () => openWindow(window.ApiClient.resolveUrl(GALLERY_URL)) }),
// D98: the legacy gallery's Refresh, which rereads every source's file list.
operationRow({ scope, title: 'Refresh files', description: 'Read every source again to find files added or removed outside ComfyUI.',
label: 'Refresh', onClick: refresh }));
-172
View File
@@ -1,172 +0,0 @@
/**
* Thumbnail review (#249 System S5 Task 14): the concept's "Thumbnail maintenance" dialog (`thumbnail-review`, app.js
* 1433–1437), opened by Sources' "Thumbnail generation → Review options" and Overview's "Review thumbnails". It starts
* the `thumbnails` Activity job (S5 Task 12) with the chosen options.
*
* - Scope: "Missing thumbnails only" (`missing`, the default) or "Rebuild all (explicit opt-in)" (`rebuild`).
* - Quality: "Medium · optimized" / "High · original" / "Low · fastest" (`medium` / `high` / `low`), preselected from
* Appearance's Image loading quality (`gallerySettings.imageQuality`, the same three values; Q3).
* - Preview size limit: the longest edge, a whole number from 128 to 2048 (concept default 512). A size outside that
* marks the field `aria-invalid`, says so in the plan line and sends nothing; Start reports the field's validity.
* - D99: a `p.small[role=status]` line before the actions says what the server's plan counts for the chosen scope
* (`GET /images/thumbnails/plan`, read-only): "N previews to generate · M already up to date." or, for a rebuild,
* "Every preview is regenerated: N files.", plus "· R more on the next run" past the per-run cap. Each scope is read
* once per dialog (the size and quality change no count). The primary is "Start" (the concept's "Simulate thumbnail
* job"): it closes the dialog and starts the job through the Activity store, which toasts that it started; the job
* opens in Activity, where its progress lives. A start refused because another job holds the slot toasts the D57
* busy link toast and stays.
*/
import { createButton } from '../shared/toolkit/core.js';
import { closeWorkspaceDialog, field, openContentDialog } from './workspace-dialogs.mjs';
import { busyToast } from './activity-scan.mjs';
import { readPreferences } from './preferences.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { navigateTo } from './state.mjs';
import { noun } from './plural.mjs';
export const THUMBNAIL_PLAN_URL = '/prompt_manager/images/thumbnails/plan';
export const THUMBNAILS_KIND = 'thumbnails';
export const SCOPE_OPTIONS = Object.freeze([['missing', 'Missing thumbnails only'], ['rebuild', 'Rebuild all (explicit opt-in)']]);
export const QUALITY_OPTIONS = Object.freeze([['medium', 'Medium · optimized'], ['high', 'High · original'], ['low', 'Low · fastest']]);
export const SIZE_LIMIT = Object.freeze({ min: 128, max: 2048, value: 512 });
export const PLAN_LOADING = 'Counting previews…';
export const PLAN_FAILED = 'The previews could not be counted.';
export const SIZE_INVALID = `Enter a size from ${SIZE_LIMIT.min} to ${SIZE_LIMIT.max} pixels.`;
export const START_FAILED = 'Thumbnail generation could not start. Try again.';
const QUALITIES = new Set(QUALITY_OPTIONS.map(([value]) => value));
const count = value => value.toLocaleString('en-US');
const isCount = value => Number.isSafeInteger(value) && value >= 0;
/** The preview size for the field's text: a whole number of pixels in range, else null. */
export function previewSize(text) {
if (typeof text !== 'string' || !/^\d{1,5}$/.test(text)) return null;
const size = Number(text);
return size >= SIZE_LIMIT.min && size <= SIZE_LIMIT.max ? size : null;
}
/** Q3: Image loading quality preselects the dialog's quality; anything it does not know is Medium. */
export function preselectedQuality(imageQuality) {
return QUALITIES.has(imageQuality) ? imageQuality : 'medium';
}
/** D99: the plan line for the server's `{mode, targets, fresh, remaining}`; an answer without counts failed. */
export function planText(plan) {
if (!plan || !['missing', 'rebuild'].includes(plan.mode) || ![plan.targets, plan.fresh, plan.remaining].every(isCount)) {
return PLAN_FAILED;
}
const more = plan.remaining ? ` · ${count(plan.remaining)} more on the next run` : '';
if (plan.mode === 'rebuild') return `Every preview is regenerated: ${count(plan.targets)} ${noun(plan.targets, 'file')}${more}.`;
return `${count(plan.targets)} ${noun(plan.targets, 'preview')} to generate · ${count(plan.fresh)} already up to date${more}.`;
}
/**
* Start the thumbnail job with `options` (`{mode, size, quality}`) and open its view; a refused or failed start says
* why and stays. Nothing is said once the section that started it is gone.
*/
export async function startThumbnails({ store, scope, options, notify = showWorkspaceToast, navigate = navigateTo }) {
try {
const result = await store.start(THUMBNAILS_KIND, options);
if (scope.signal.aborted) return;
if (result.busy !== undefined) {
notify(busyToast(result.busy));
return;
}
navigate(`#/activity/${result.job.id}`);
} catch {
if (!scope.signal.aborted) notify(START_FAILED);
}
}
function el(tag, text, className) {
const node = document.createElement(tag);
if (text !== undefined) node.textContent = text;
if (className) node.className = className;
return node;
}
function select(options, value) {
const node = el('select');
node.append(...options.map(([optionValue, label]) => {
const option = el('option', label);
option.value = optionValue;
return option;
}));
node.value = value;
return node;
}
const defaultQuality = () => {
try {
return readPreferences().values.imageQuality;
} catch {
return undefined;
}
};
/**
* Open "Thumbnail maintenance" on the section's `scope`. `store` is the page's Activity store; `trigger` gets focus
* back when the dialog closes.
*/
export function openThumbnailReview({ api, store, scope, trigger = null, quality = defaultQuality(), notify = showWorkspaceToast,
navigate = navigateTo }) {
const mode = select(SCOPE_OPTIONS, 'missing');
const encoder = select(QUALITY_OPTIONS, preselectedQuality(quality));
const size = el('input');
size.type = 'number';
size.value = String(SIZE_LIMIT.value);
size.min = String(SIZE_LIMIT.min);
size.max = String(SIZE_LIMIT.max);
const line = el('p', PLAN_LOADING, 'small');
line.setAttribute('role', 'status');
const cancel = createButton('Cancel');
const start = createButton('Start', 'ca-primary');
const actions = el('div', undefined, 'dialog-actions');
actions.append(cancel, start);
// The plan per scope: a promise of its answer (null when it failed); the size and quality change no count.
const plans = new Map();
let open = true;
const chosenSize = () => previewSize(size.value);
const validate = () => {
const valid = chosenSize() !== null;
size.setCustomValidity(valid ? '' : SIZE_INVALID);
if (valid) size.removeAttribute('aria-invalid');
else size.setAttribute('aria-invalid', 'true');
return valid;
};
const readPlan = chosen => {
const query = new URLSearchParams({ mode: chosen, size: String(chosenSize()), quality: encoder.value });
return api.get(`${THUMBNAIL_PLAN_URL}?${query}`, { signal: scope.signal, cache: 'no-store' }).catch(() => null);
};
async function paint() {
if (!validate()) { line.textContent = SIZE_INVALID; return; }
const chosen = mode.value;
if (!plans.has(chosen)) plans.set(chosen, readPlan(chosen));
const pending = plans.get(chosen);
line.textContent = PLAN_LOADING;
const plan = await pending;
// A later choice painted its own line meanwhile, or the dialog closed.
if (!open || scope.signal.aborted || mode.value !== chosen || !validate()) return;
line.textContent = planText(plan);
}
scope.listen(mode, 'change', paint);
scope.listen(size, 'input', paint);
scope.listen(cancel, 'click', () => closeWorkspaceDialog());
scope.listen(start, 'click', () => {
if (!validate()) {
size.reportValidity();
size.focus();
return;
}
const options = { mode: mode.value, size: chosenSize(), quality: encoder.value };
closeWorkspaceDialog();
startThumbnails({ store, scope, options, notify, navigate });
});
openContentDialog({ scope, title: 'Thumbnail maintenance', trigger,
nodes: [field('Scope', mode, 'thumb-mode'), field('Quality', encoder, 'thumb-quality'),
field('Preview size limit', size, 'thumb-limit'), line, actions],
onClose: () => { open = false; } });
paint();
}
-15
View File
@@ -24,10 +24,6 @@ main:focus-visible { outline: 2px solid var(--ca-focus); outline-offset: 4px; }
.activity-pill:hover { border-color: var(--ca-accent); /* D53 */ }
.activity-pill-bar { display: block; inline-size: 70px; block-size: 5px; border-radius: 3px; background: var(--line); overflow: hidden; /* D53 */ }
.activity-pill-bar i { display: block; block-size: 100%; inline-size: 0; background: var(--ca-accent); /* D53 */ }
/* D111 (S5): a running job without a known total sweeps a short segment; reduced motion shows a dimmed full bar. */
.activity-pill-bar.indeterminate i { inline-size: 40%; animation: activity-pill-sweep 1.4s ease-in-out infinite; /* D111 */ }
@keyframes activity-pill-sweep { from { transform: translateX(-100%); } to { transform: translateX(250%); } }
@media (prefers-reduced-motion: reduce) { .activity-pill-bar.indeterminate i { inline-size: 100%; opacity: .45; animation: none; /* D111 */ } }
@media (pointer: coarse) { .activity-pill:not([hidden]) { min-block-size: 40px; /* D53 */ } }
@media (max-width: 700px) {
/* D52: with the Activity button the full-width row can be wider than a zoomed phone, so its items wrap. */
@@ -398,17 +394,6 @@ pre.prompt-text { font: inherit; font-size: 12px; line-height: 1.75; margin-bloc
.ca-tools #tag-suggestions .chip.tag-suggestion > button { min-block-size: 40px; min-inline-size: 40px; }
}
/* Needs review (#249 PR F): the concept's Metadata review page (app.js a749–760) in its own classes. */
/* #249 D3: the review queue is not virtualized, so its cards stay in the concept grid's flow. */
.image-grid.review-queue > .media-card { position: static; /* D3 */ }
/* #249 D20: the prompt blocks' mount point is transparent, so the chosen path's sections sit in the panel as the
concept's do. */
.review-panel > .prompt-inspector { display: contents; /* D20 */ }
/* D119: a file that cannot be served (no current root) has an inert, empty media button that keeps the concept's
1.33 media area (app.js `.image-open img`), so its card lines up with the queue's other cards. */
.review-queue .image-open[aria-disabled="true"] { cursor: not-allowed; aspect-ratio: 1.33; /* D119 */ }
/* D119: a long file name in the panel wraps inside the panel, where the concept's sample name never overflows. */
.review-panel > p.small { min-inline-size: 0; overflow-wrap: anywhere; /* D119 */ }
/* Auto-tag selected images (#249 PR D, D85/D90): the Tag image dialog's model panel, then How to apply, a radio group
in the concept's `.field` row whose radios keep their own size (the concept's `.field input` is a full-width text
field) beside their label, as the concept's own radio rows (s1160 `.compare-item label`). */