fix(thumbnails): answer shared flights with the outcome only and restart a failed worker

The history and serve routes share a flight key, so a flight now answers
only the ensure outcome whichever route started it; the serve route
derives the preview path and, for a fallback, looks its source up again
on the view pool. A view joining a history flight no longer gets a 500
or an image answered with the video placeholder.

The capture worker clears its running state however its loop ends, and a
worker that cannot start leaves the file queued for the next enqueue.

README.md

@@ -970,6 +970,11 @@ MIT License - see LICENSE file for details.
 
 ## Changelog
 
+### Unreleased (Workflows stored once)
+
+- **🗜️ Smaller database**: each distinct workflow and prompt graph is stored once, compressed, instead of again for every image. Images recorded by this version use it from the first start; the maintenance job "Store each workflow once" rewrites the existing records and then reclaims the space. A 20,000-image test library went from 1.4 GB to 60 MB. The job always takes a verified backup first: the first time, `prompts.db.pre-workflow-storage_<date time>` beside the database, kept until you delete it; on later runs, a normal pooled safety backup. No image or video file is opened or changed.
+- **⚠️ Downgrading**: images recorded by this version (and, after the job, all images) are stored in the new form. An older Prompt Manager version still opens and even restores such a database, but shows those images without workflows or prompt graphs until you update again. To go back to an older version, stop ComfyUI, delete `prompts.db-wal` and `prompts.db-shm`, and copy the **earliest** `prompts.db.pre-workflow-storage_<date time>` over `prompts.db`, or, if the job never ran, a backup taken before this update (records added since that backup are lost). Otherwise stay on this version.
+
 ### v3.3.1 (Proxy Routing Fix)
 
 - **🌐 Proxy support**: the dashboard, gallery, tag page and metadata viewer now work when ComfyUI is reached through a proxy mount such as SwarmUI's `/ComfyBackendDirect/`. Scripts, images, API calls, progress streams and gallery folder filters stay under the mount, and the canvas node's buttons use ComfyUI's `fileURL()`. Direct ComfyUI URLs are unchanged. Thanks to @ghosttigerllc-bit ([#164](https://github.com/ComfyAssets/ComfyUI_PromptManager/pull/164))

TESTING.md

@@ -1353,8 +1353,8 @@ targets of the longest legal path (1024 code points, even escaped as `\uXXXX`) f
   the selection bar's tagging shows is the concept's or the plan's (D85, D90, D91).
 - **D89** (plan D66, Activity for the `image-tags` job): the job's Activity item and its
   view are the scan's (D54, D56, D57, D59) with the job kind's own counters, from one
-  per-kind table (`JOB_VIEWS` in `activity-store.mjs`; a kind without one keeps the
-  scan's tiles under "Job log" and "Cancel job"). The titles are the server's: "Add
+  per-kind table (`activity-kinds.mjs` since S5, D111; a kind without one gets no
+  tiles, "Job log" and "Cancel job"). The titles are the server's: "Add
   tags to images" and "Auto-tag images".
   - The pill's verb is "Tagging" ("Tagging · 41% · 8,240 / 19,622").
   - The item line and the view's heading line read "Started HH:MM · N images · closing
@@ -1387,7 +1387,7 @@ targets of the longest legal path (1024 code points, even escaped as `\uXXXX`) f
     finished · N tagged · M errors", "<title> cancelled · N tagged" or "<title> failed ·
     N tagged · M errors".
   - Tests: `tests/js/workspace-activity-store.test.js` ("image-tags jobs get their own
-    tiles outcomes and pill verb", "unknown kinds keep the generic view");
+    tiles outcomes and pill verb", "unknown kinds get the generic view without the scan tiles");
     `tests/browser/workspace-activity.spec.mjs` ("an image tagging job shows Tagged
     Unchanged Skipped Errors and opens tagged rows", "cancel tagging stops at a file
     boundary", "the report download names the kind", on fake `image-tags` jobs from
@@ -1932,8 +1932,8 @@ pending count (`GET /prompt_manager/review/items?limit=1`, its `total`).
   unavailable file's empty media button keeps the concept's 1.33 media area, so its
   card lines up with the others. A long file name in the panel's `p.small` wraps inside
   the panel (`overflow-wrap: anywhere`), where the concept's short sample name never
-  overflows. A queue card shows the Images listing's thumbnail when the server knows one
-  (`thumbnail_url`, the same URL the Images page uses), else the original; a re-read
+  overflows. A queue card shows the Images listing's thumbnail (`thumbnail_url`, the
+  same URL the Images page uses; a preview not made yet is made on first view); a re-read
   rebuilds a card whose media, target or availability changed (a root came or went), so
   Open and Tag follow it. When an unanswered decision's item leaves the queue in a
   re-read, the decision was recorded: the page finishes as a recorded decision does (its
@@ -2157,10 +2157,132 @@ keeps the drafts; a save refused for one source says the server's message as the
 and marks that row. After a save the source list is read again; a monitoring change
 waits for a restart (the running observer keeps watching until then, and the status
 says "Restart ComfyUI to apply: Watch for new outputs."). Thumbnail generation's
-Review options opens the legacy gallery in a new tab (D94) until its review dialog (S5).
+Review options opens the concept's "Thumbnail maintenance" dialog (D99, below).
 A list that cannot be read shows "Sources could not be loaded. Your saved data has not
 changed." and Retry, never an empty list.
 
+Thumbnail generation is an Activity job (S5, `py/api/thumbnail_jobs.py`). `GET
+/prompt_manager/images/thumbnails/plan?mode=missing|rebuild&size=128..2048&quality=low|medium|high`
+counts the work (`targets`, `fresh`, `remaining` past the 10,000-file cap, the source
+labels) and writes nothing; a value out of range is a 400 `invalid-request` that starts
+nothing. `POST /prompt_manager/jobs/thumbnails {mode, size, quality}` starts the
+`thumbnails` job ("Thumbnail generation"): admission `guarded` (never beside a scan or
+another record-changing job, one at a time, a legacy clear refused while it runs),
+cancellable at a file boundary (a file not yet started is left alone; written previews
+stay), outcome groups Generated / Skipped (already fresh, listed without a log line
+each) / Errors. `size` is the longest edge and `quality` the encoder (low: JPEG 70, no
+optimize; medium: 85, optimized, what the legacy routes always wrote; high: 95,
+optimized). Video previews take the same choice: the OpenCV frame and the placeholder
+are saved with those JPEG settings, and the ffmpeg fallback (no OpenCV) maps them to
+`-q:v` 8 / 4 / 2 and fits the frame inside the edge keeping its aspect ratio (`-vf
+scale=E:E:force_original_aspect_ratio=decrease`, no longer a stretched square).
+`rebuild` writes fresh previews again, each through the same temp-file
+rename, so nothing is cleared first and a failed file keeps its old preview (a video
+whose frame cannot be decoded keeps its preview instead of getting the placeholder);
+originals are only read (`tests/test_thumbnail_jobs.py`). Cancel is also checked per
+file during the planning walk: a job cancelled there stops the walk at the next file,
+publishes no `start` frame and ends "Cancelled before any preview was written". Every
+exit sets a final status line: "N previews written · M errors" (plus "· R more on the
+next run" past the cap), "No previews to generate · N already up to date" (or "No
+images or videos to preview"), "Cancelled · N previews written", or the failure; a job
+with nothing to do never reads "Processing file 0 of 0" (`test_every_exit_sets_a_truthful_final_status_line`,
+and "a finished thumbnail job with nothing to do says so" in `workspace-activity.spec.mjs`).
+Windows: replacing a preview another process holds open (the gallery serving it, an
+image viewer) fails with WinError 5 or 32; that file counts as an error reading
+"Preview in use by another program; the old one was kept" (never the raw "[WinError
+5]"), and its old preview stays. Any other PermissionError reads "Permission denied
+writing the preview" (`test_a_locked_preview_is_a_readable_coded_error`).
+Thumbnail work (the job and the gallery's generate and clear holds) is `guarded`, but it
+writes only preview files and no records, so the image monitor's Needs review
+recording runs beside it instead of being deferred for a whole rebuild
+(`PREVIEW_ONLY_KINDS` in `py/api/review_recording.py`, `blocker(..., compatible=)`;
+`test_a_running_thumbnail_job_does_not_defer_monitor_recording`). Other `guarded` and
+all `records` jobs still defer it.
+
+Deliberate change: the legacy gallery stream `POST /images/generate-thumbnails/progress`
+now starts this job (missing previews, its quality preset as the size, medium encoding)
+or attaches to the running one, and streams the job's unchanged `status`/`start`/
+`progress`/`file_error`/`complete` frames after a first `job {id, started}` frame that
+names the job and whether this stream started it. The stream reads the registry after
+its headers are sent, so a job that starts in that gap is attached to, not refused. A
+client that goes away no longer stops the work (`test_thumbnail_job_survives_client_disconnect`:
+one worker, 9 files in three batches; the stream sees its client go after the first
+batch and all 9 previews are written, where a stream that stops on disconnect leaves 1).
+The gallery's Cancel (its settings panel and the startup dialog) cancels by that id,
+and only a job its own stream started: a Cancel before the `job` frame keeps the stream
+open until it arrives, a double Cancel posts once, and a job another tab or System
+started keeps running while the gallery stops following it (`tests/js/gallery-thumbnails.test.js`).
+A job cancelled from Activity ends the stream with the error frame "Thumbnail
+generation was cancelled". The blocking
+`POST /images/generate-thumbnails` and the clear stay legacy holds.
+
+The workspace starts it from "Thumbnail maintenance" (`thumbnail-review.mjs`, the
+concept's `thumbnail-review`), opened by Sources' Thumbnail generation → Review options
+and Overview's Review thumbnails. Scope is Missing thumbnails only (`missing`, the
+default) or Rebuild all (explicit opt-in) (`rebuild`); Quality is Medium · optimized /
+High · original / Low · fastest (`medium` / `high` / `low`), preselected from
+Appearance's Image loading quality (`gallerySettings.imageQuality`, the same values,
+Q3); Preview size limit is the longest edge, 512 by default, a whole number from 128 to
+2048. A size outside that range marks the field `aria-invalid`, the plan line says
+"Enter a size from 128 to 2048 pixels." and nothing is requested; Start reports the
+field's validity and keeps the dialog open. Start closes the dialog and starts the job
+through the Activity store, which toasts "Thumbnail generation started."; the job opens
+in Activity. A start refused because another job holds the slot toasts the D57 busy
+link toast and stays. The legacy gallery's thumbnail tools are unchanged.
+
+Automatic previews (#249, `py/api/thumbnail_auto.py`). The approved mock has no
+repeated global thumbnail prompt (`docs/concepts/workspace/README.md`), so the
+workspace never asks; previews are a cache the server keeps:
+
+- Every listing gives an image or video a thumbnail URL (`media_thumbnail_url`: Images,
+  saved views and favorites, Library cards and galleries, Needs review, duplicate
+  groups). An existing preview's URL keeps its `v=<mtime>`; a missing one is listed
+  without `v=`.
+- `GET /images/serve/thumbnails/<rel>_thumb<ext>` for a missing preview is refused as
+  text first (traversal, a backslash, a drive or stream colon, empty or dot segments, so
+  `D:/x`, `\\server\share` and a leading `/` never reach a joined path), then admitted
+  on the event loop (`_PreviewGate`): 3 previews are written at once on the API's own
+  `pm-preview` pool (never the default executor), one flight runs per file (the history
+  route shares the flight key and the admission), and at most 64 requests wait; one
+  more gets its fallback at once without listing a folder. Inside the admission the
+  preview maps back to its media file under the same root (exact names, then one folder
+  listing for a mixed-case extension; a regular file inside the root, never through a
+  link) and is made with the job's defaults (512 px, medium). When it cannot be made an
+  image's original is served and a video's placeholder frame (a video is never sent to
+  an `<img>`), both `Cache-Control: no-store`; cards and the inspector cover also swap a
+  video back to its own element when its preview image fails. Only a file that cannot
+  be decoded is remembered for the session (path, size, mtime; retried once it changes)
+  and counted as `failed` in `/images/thumbnails/status`, which Overview's row adds
+  ("N missing thumbnails · M failed"); I/O errors and a preview another program holds
+  open (Windows) are not. A preview folder (or any link segment of it) resolving out of
+  the root is never written. Media inside a nested `thumbnails` folder is listed with no
+  thumbnail and keeps its original. The listing's `thumbnail_ready` says whether the
+  preview file exists (the legacy gallery's dot and "Fast").
+- The video decoder falls through OpenCV, ffmpeg (a tracked process, killed on
+  shutdown) and the placeholder frame: an OpenCV that cannot open or read a clip (AV1)
+  no longer ends the run.
+- The image monitor hands every settled output to `queue_capture_thumbnail`, which
+  queues it for one background worker: each wake takes at most Worker threads files and
+  writes them on that many threads; a queued file is not queued twice; the queue holds
+  10,000 and a file left out is made on view.
+- A finished scan that added files counts the missing previews of the sources it
+  walked: 200 or more start the thumbnail job (missing only, visible and cancellable in
+  Activity, never silent bulk work); fewer go through the worker.
+- Holds: automatic previews write only `thumbnails/` files and no records, so they run
+  beside a scan, image tagging, the thumbnail job and the Needs review recording (S5
+  M7). They wait (a view gets the original, a queued file is left for its view) while a
+  restore, a duplicate removal, a sources change, the gallery's thumbnail clear or an
+  `exclusive` job runs (`_auto_thumbnail_blocker`), checked when a write starts (after
+  its slot) and again just before its rename.
+- Shutdown is bounded and logged: the worker stops at a file boundary, ffmpeg is
+  killed, an idle worker leaves (the next output starts a fresh one), the view pool
+  shuts down, and after the app's cleanup a late monitor event
+  starts nothing. A package reload stops the workers the previous import left.
+- Cards and strips load previews only: Images in every layout and size, Library cards,
+  Needs review cards and the inspector's cover (a video's cover is its preview frame).
+  The media viewer's main media and downloads load the original
+  (`workspace-thumbnails.spec.mjs` records every media request; `tests/test_thumbnail_auto.py`).
+
 Overview (`system-overview.mjs`) reads `GET /prompt_manager/system/overview` once per
 visit through `latestOnly()` on the section scope; a late answer after the section left
 paints nothing. Until it answers, the section shows "Loading the overview…"; a failure
@@ -2188,7 +2310,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged
 `-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
 reader can keep it byte-identical (`test_health_check_never_writes`).
 
-Recorded deviations (plan D60–D66 and D81 renumbered D92–D98 and D113; D121–D128 assigned):
+Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D113; D121–D128 assigned):
 
 - **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
   unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server
@@ -2208,8 +2330,9 @@ Recorded deviations (plan D60–D66 and D81 renumbered D92–D98 and D113; D121
   "Backup status" reads "No retained backup record." until verified backups exist (S8).
   The thumbnail row is "Checking thumbnails…" until its own read answers, then "No
   missing thumbnails", "N missing thumbnails" (the pending count of
-  `/images/thumbnails/status`) or "Thumbnail status unavailable"; Review thumbnails
-  opens Sources until the thumbnail dialog (S5). "One display configuration" says "Grid
+  `/images/thumbnails/status`, followed by " · M failed" when automatic previews
+  failed this session) or "Thumbnail status unavailable"; Review thumbnails
+  opens the thumbnail dialog (D99). "One display configuration" says "Grid
   and density settings are shared across views." (no paging). The sources row is
   "N output sources" with "Monitoring and enrichment share one source list.", prefixed
   by "M of N configured sources available." or "None configured is available; ComfyUI's
@@ -2270,6 +2393,43 @@ Recorded deviations (plan D60–D66 and D81 renumbered D92–D98 and D113; D121
   legacy gallery's `/gallery/rescan` and toasts "Files refreshed. N files in M output
   folders." or "Files could not be refreshed. Try again.". The concept's "sample path"
   note is prototype copy and is dropped.
+- **D99** (Sources/Overview, S5): the concept's `thumbnail-review` dialog ("Thumbnail
+  maintenance") with production copy. A `p.small[role=status]` line before the actions
+  says what the server's read-only plan (`GET /images/thumbnails/plan`) counts for the
+  chosen scope: "N previews to generate · M already up to date." or, for Rebuild all,
+  "Every preview is regenerated: N files.", with "· R more on the next run" past the
+  per-run cap; "Counting previews…" while it is read and "The previews could not be
+  counted." when the read fails (Start still starts). Each scope is read once per
+  dialog, since size and quality change no count. The primary is "Start" (the
+  concept's "Simulate thumbnail job"). Overview's Review thumbnails opens the same
+  dialog (the concept's row starts a sample "Thumbnail check" job directly). Maintenance's
+  Thumbnail cache row takes the dialog with S7. `thumbnail dialog matches the concept
+  from sources|overview` (system-concept.spec, both themes and sizes) maps the line and
+  "Start"; `thumbnail dialog comparison detects mutations` proves they are the only
+  difference and catches a renamed option, swapped options, swapped actions and a
+  changed style.
+- **D111** (Activity, S5): each job kind's pill verb, toasts, line, tiles, outcomes,
+  openable rows and labels come from one descriptor table (`activity-kinds.mjs`;
+  `activity-store.mjs` re-exports it). The scan's (D56) and image tagging's (D89) are
+  unchanged. A thumbnail job reads "Generating thumbnails · N% · a / b" in the pill,
+  counts Generated (green), Skipped and Errors (red), lists Generated, Errors, Skipped
+  (Generated and Skipped rows open the viewer), says "Thumbnails finished · N generated ·
+  M errors" ("cancelled · N generated", "failed · …") and keeps "Job log" and "Cancel
+  job". Every other kind (the health check, a kind this page does not know) gets the
+  generic descriptor: title-based toasts and no tiles or Results, so it never shows the
+  scan's zero tiles (S1 review M6, S2 review M3). A running job without a known total
+  has an indeterminate `<progress>` (no `value`) and a pill without a percentage, whose
+  mini bar (`aria-hidden`; the pill's label carries the state) is marked `indeterminate`
+  and sweeps instead of showing an empty fill (a dimmed full bar under reduced motion). Only a
+  running job the server can cancel shows Cancel. Tests:
+  `tests/js/workspace-activity-kinds.test.js`, and in `workspace-activity.spec.mjs` "a
+  thumbnail job shows Generated Skipped Errors, its pill verb and its toasts" and "a held
+  legacy job and the health check show no tiles, no Results and no made-up percentage".
+- **D112** (Activity, S5): a legacy-page operation held on the registry (not
+  cancellable, no outcome groups) reads as its server title ("Database restore (admin
+  page)", "Thumbnail clear (gallery page)"), with no tiles, no Results, no Cancel, an
+  indeterminate bar while it runs and "<title> finished." when it ends, whatever its
+  kind (the gallery's clear is a `thumbnails` hold).
 - **D113** (Sources): the Worker threads range ends at the server's `cpu_count` (the
   concept's fixed `max=16`) and shows the server's real `worker_threads` (the concept's
   sample 4).
@@ -2358,10 +2518,13 @@ and a source row, renames a label and the Add source button and swaps the `--lin
 token. Paired captures (`{concept,production}-system-{dark,light}-{1600,390}.png`,
 `{concept,production}-{statistics,health}-{dark,light}.png`,
 `{concept,production}-appearance-{dark,light}-{1600,390}.png`,
-`{concept,production}-sources-{with,without}-notice-{dark,light}-{1600,390}.png`) are test artifacts; there
+`{concept,production}-sources-{with,without}-notice-{dark,light}-{1600,390}.png`,
+`concept-thumbnail-review-{dark,light}-{1600,390}.png`,
+`production-thumbnail-review-{sources,overview}-{dark,light}-{1600,390}.png`) are test artifacts; there
 are no PNG baselines. `tests/browser/workspace-system.spec.mjs` covers the real counts,
 the health check and metric 4, a failed check, the thumbnail row against the status
-endpoint (painted after the rest when its read is held, unavailable when it fails), the review row (Q6), the section links, the Statistics dialog against the
+endpoint (painted after the rest when its read is held, unavailable when it fails), its Review thumbnails
+opening the thumbnail dialog and starting the job, the review row (Q6), the section links, the Statistics dialog against the
 route, a load failure with Retry and late answers after leaving, and Appearance: new
 visits versus the current view and a saved view, the embedded gallery keeping Grid,
 metadata off and on request, the cache (counted up to a request barrier), hidden paths,
@@ -2371,7 +2534,9 @@ the D97 rows and reset. `tests/browser/workspace-system-sources.spec.mjs` covers
 notice's Later and Review scan, a scoped scan from one source into Activity, Add source
 with its server check and Save, a refused folder, Remove down to auto-detect, an absent
 configured source, the three watch states, the monitoring and worker rows, Refresh
-files, the thumbnail hand-off, a save refused by a running scan and a failed read. Its hooks are
+files, the thumbnail dialog (its plan line per scope with the preselected quality, an
+invalid size that requests nothing, Start into Activity, Rebuild all, a failed plan and
+a refused start), a save refused by a running scan and a failed read. Its hooks are
 `tests/browser/workspace_system_fixture.py` (`/__test__/system/*`: reset, injected
 failures, an absent source, a deleted linked file, unthumbnailed files, a fake image
 monitor) on the
@@ -2386,13 +2551,284 @@ node --test tests/js/workspace-system-state.test.js tests/js/workspace-system-co
   tests/js/workspace-system-overview.test.js tests/js/workspace-state.test.js \
   tests/js/workspace-system-appearance.test.js tests/js/workspace-preferences.test.js \
   tests/js/workspace-inspection-state.test.js tests/js/workspace-images-state.test.js \
-  tests/js/workspace-system-sources.test.js
+  tests/js/workspace-system-sources.test.js tests/js/workspace-thumbnail-review.test.js
 python3 -m unittest tests.test_settings_sources tests.test_scan_scope -v
+python3 -m unittest tests.test_maintenance_plans tests.test_api_maintenance -v
 npm run --prefix tools/vendor test:workflows -- system-concept.spec.mjs workspace-system.spec.mjs \
   workspace-system-sources.spec.mjs \
   --project chromium --project touch --project firefox --project webkit
 ```
 
+### Maintenance contract (#249 System S6)
+
+`POST /prompt_manager/maintenance/preview {operation, options:{}}` reviews one
+operation and answers a `Plan` (`py/api/maintenance.py`, `database/maintenance.py`):
+`duplicate_prompts` (legacy `cleanup_duplicates`: equal text ignoring case and outer
+spaces, the oldest prompt of each group kept; a group with a member in a saved pair is
+kept whole), `optimize` (VACUUM, with the database size, its free pages and the free
+space it needs), `unused_components` (legacy `prune_orphaned_prompts`; the #239
+protections are the predicates `ORPHANED_PROMPTS_SQL` is built from, so the plan's
+protected reasons are exactly the prune's rules) and `missing_links` (legacy
+`cleanup_orphaned_images`, judged as the health check's `missing_files` through
+`health.link_states`: only a link under a source that can be read now counts; one under
+an unavailable source or outside every source is protected). The operation enum is
+strict, `options` must be `{}` and the body is capped at 4 KiB (512 KiB for
+`duplicate_files`: 1 MiB, below). A preview meets the
+read-only standard above on its own `mode=ro` snapshot (`test_preview_writes_nothing`,
+per operation, plus the media bytes) and stats files after the snapshot ended. The
+plan lists at most 50 detail rows and no server path. Its fingerprint (the ids, text
+hashes, metadata, tags, Library state and pair refs of every prompt involved, or each
+link's row and source), its targets and the sources it judged stay in the server's
+`PlanStore` under a single-use `secrets.token_urlsafe(32)` token: 15 minutes, at most 8
+plans and 16 MiB of payload. A plan over that budget is refused with `too-large`, never
+truncated; a new plan drops the oldest ones, which then read as expired; a restart
+forgets them all. A plan is bound to the database it read: a restore during the preview
+answers `storage-changed`. Errors are coded (`invalid-request`, `too-large`,
+`storage-changed`, or a null code with "Failed to review maintenance") and carry no
+path or exception text.
+
+`POST /prompt_manager/maintenance/apply {token}` starts the reviewed plan as an
+Activity job (202 `{job}`): kind `maintenance`, titled after the concept's row
+("Duplicate prompt text", "Optimize database", "Unused prompt components", "Missing-file
+links"), admission `records` (it never runs beside a scan, restore, LoRA import or other
+record-changing job, legacy-page holds included: `test_apply_refused_while_restore_runs`),
+not cancellable, with the groups `changed`, `protected` and `errors`. The token is consumed
+on the loop turn the job starts, so a second apply answers 409 `plan-used` with that job;
+a refused start (`busy`) keeps it. Other errors: `invalid-request`, `plan-expired` (410:
+unknown, expired, or forgotten after a restart) and `storage-changed` (409: a restore
+replaced the database since the review; the plan is dropped). The job:
+
+1. takes a safety backup when the plan was reviewed with the policy on or the policy is
+   on now (`maintenance.safety_backup` in `config.json`, default `true`; GET/POST
+   `/prompt_manager/settings` field `safety_backup` with the revision). The copy is made
+   with SQLite's backup API into a temp file beside the database, switched to a
+   rollback journal (so a later read-only open leaves no `-wal`/`-shm` beside it),
+   verified with `verify_database_file`, fsynced, made `0o600` on POSIX and only then
+   renamed with `os.replace` onto `<db>.backup_<YYYYmmdd_HHMMSS_ffffff>`, reserved with
+   `O_EXCL` (`_1`, `_2`… when a coarse clock repeats a stamp:
+   `test_backup_names_unique_within_one_second`). A backup file on disk is therefore
+   always a verified one. Any failure removes what it wrote and ends the job
+   `backup-failed` before anything is changed (`test_backup_failure_aborts_before_write`);
+2. applies the plan: inside `BEGIN IMMEDIATE` the fingerprint is computed again on the
+   write connection and a difference ends the job `plan-changed` with nothing written
+   (`test_apply_rejects_changed_target`; a missing link whose file is back or whose source
+   went away is never removed). The targets are deleted in batches of 400 ids in that one
+   transaction. Missing-file links remove database rows only; every media file's bytes
+   are unchanged (`test_missing_links_never_deletes_files`). VACUUM runs outside a
+   transaction and binds to its database only;
+3. counts the changed and protected records (outcome rows hold root-relative paths or
+   prompt ids only), logs the backup by its file name, and prunes safety backups to the
+   newest 10. With no backup the log and the status line say "No safety backup".
+
+The legacy restore (`POST /prompt_manager/restore`) takes a backup of any size the disks
+have room for, so a real library's safety backup restores: the upload, its staged copy
+and the swap (the upload twice in the temporary folder, once more plus the current
+database beside the database, added up on one disk, each with the 16 MiB margin) must
+fit, under a 64 GiB ceiling (`restore_max_bytes`). A request whose Content-Length does
+not fit is refused before anything is written, one without is refused as soon as its
+running size does not, both with 507 `insufficient-space` and "Not enough free disk
+space to restore a 1.6 GB backup; there is room for about …. Nothing was changed."
+(no path); past the ceiling it is the legacy 400 "File too large" with code `too-large`.
+The upload streams with `read_chunk`, which aiohttp's `client_max_size` (ComfyUI's
+`--max-upload-size`, 100 MB by default) does not cap, and its temp file is closed and
+removed on every path (`TestLargeRestores`, a 150 MB backup under a 100 MB app limit).
+
+Safety backups and restore backups are one pool: every `<db>.backup_*` file beside the
+database counts toward the same `MAX_SAFETY_BACKUPS` (10), whichever wrote it, and the
+pool is pruned after every backup a job took, also when the apply then failed
+(`test_safety_backups_share_the_restore_pool`,
+`test_the_pool_is_pruned_when_the_apply_fails_after_the_backup`). The legacy
+`GET /backup` download writes no file beside the database, so it adds nothing to the
+pool. Each plan reports `backup.expected_bytes` (the backup's size, `page_count *
+page_size`, or null with the policy off) and `disk: {free_bytes, required_bytes,
+sufficient}` for the database folder: the backup needs its size there, and Optimize
+needs the database's size twice (the WAL beside it and SQLite's temporary copy, added
+up when the temporary folder is on the same disk), each plus a 16 MiB margin. The
+temporary folder is the one SQLite really uses (`sqlite_temp_dir`): on Unix
+`SQLITE_TMPDIR`, `TMPDIR`, then the first writable of `/var/tmp`, `/usr/tmp` and
+`/tmp` (not Python's `tempfile` order, which picks a RAM-backed `/tmp` on many Linux
+systems); on Windows `TMP`, `TEMP`, then `USERPROFILE`. Whether a backup is taken is
+decided once, when Apply measured the disk, and handed to the job. Apply looks at the
+token again with no await before it starts the job, so a token used or expired during
+the measurement answers its code, never a 500. A backup is put in place with
+`os.link` (atomic, refuses an existing name; reserve-and-replace where a filesystem
+has no hard links), and startup removes the `.pm-safety-*.tmp` copies an interrupted
+backup left. Optimize runs alone: its job has the `exclusive` admission class (refused
+beside any job, and every job is refused while it runs), and its plan says the database
+is locked while VACUUM runs and that run counts or captures waiting longer than the
+30-second busy timeout are not recorded. The duplicate prompt merge is strict inside
+the apply: a group whose copies link one image twice, or whose removed copy has its own
+Library title, is protected in the plan, and any merge error rolls the transaction back
+(`MergeConflict`); the legacy cleanup keeps its lenient behaviour. Apply
+measures again before it starts a job and refuses a short disk up front with 507
+`insufficient-space` and a truthful message ("Not enough free disk space: the safety
+backup needs about … in the database folder and … is free. Nothing was changed."); no
+job starts and the token stays usable
+(`test_apply_refuses_up_front_when_the_backup_does_not_fit`,
+`test_apply_refuses_up_front_when_vacuum_does_not_fit`).
+
+**Duplicate image files** (`py/api/duplicate_files.py`, `py/api/duplicate_review.py`;
+`tests/test_api_duplicate_files.py`). `POST /prompt_manager/jobs/duplicates` starts the
+find: kind `duplicates`, "Find duplicate image files", admission `read`, cancellable,
+group `groups_found`, over every effective source (the legacy scan's
+`_hash_media_groups`: no symlinks, no `thumbnails`, bounded, each file once). The
+find lists a physical file once, by `(st_dev, st_ino)`: a source listed twice under two
+spellings, a symlinked source inside another one and a hardlink are one file, never a
+copy of themselves (`test_a_folder_listed_twice_never_loses_its_only_copy`,
+`test_a_symlinked_source_inside_another_is_one_file`,
+`test_a_hardlink_is_never_a_copy_to_remove`). One file reached through two mounts is
+protected, never guessed at: a removal on another `st_dev` than its kept copy, with the
+same size and mtimes within 2 s, is kept as "may be the same file reached through another
+mount or share" when its `st_ino` is equal or its path within its source is the same, or
+one ends with the other on a whole folder name (sources listed at different depths:
+`test_a_deeper_source_on_another_mount_is_protected`)
+(NFS and SMB, a loopback mount, a FUSE pool: `TestAnotherMount`). Some true copies on
+another disk are therefore never offered. A volume that reports no file id protects
+every pair ("Can't confirm this is a separate file on this drive"). The job checks both
+before each file's records move, holds the kept copy open, and right before the unlink
+checks again that the held kept copy is still the file at its path (`fstat` of the held
+handle against `os.stat` of the path) and that the guard still holds; otherwise the file
+is kept, its records move back to it, and it counts as not removed
+(`test_a_kept_copy_deleted_during_the_move_keeps_the_file`). On Windows the hold also
+makes deleting the kept copy itself through another path fail. The server holds the groups (content
+hash, root-relative paths, source index, size, mtime and inode) for the last three
+finds; `GET /prompt_manager/maintenance/duplicates/{job_id}?offset&limit` pages
+them ten at a time (`not-found` 404, `job-running` or `no-groups` 409). The preview
+`{operation:"duplicate_files", options:{job_id, keep:[{group_id, root_index, path}]}}`
+(body cap 1 MiB, at most 2,000 keep entries; `test_the_keep_limit_fits_the_body_cap`) removes, in each group named, every copy
+not kept; a keeper that is not in its group is `invalid-request`, and a source no longer
+listed at its index is `sources-changed`. Every keeper and removal is checked with
+`file_problem`: containment as text first, with no filesystem access (inside its
+source, not under `thumbnails`, a media extension), then `lstat` (a link or any
+non-regular file is refused), the resolved path still inside the resolved source, not
+the kept copy itself (`os.path.samestat`), and the size, mtime and inode the find saw.
+The preview reads no file; the removal job hashes each file and its kept copy again
+right before the unlink (`test_preview_reads_no_file_and_spots_a_touched_one`). A removal that fails, or whose kept copies all fail, is
+protected with its reason. The plan names each removal, its kept copy and how many
+database links move, says the safety backup covers the database and not the files,
+and writes nothing. Apply runs `duplicate_cleanup` (records, not cancellable, groups
+`removed`, `kept`, `errors`): sources unchanged, the promised safety backup, then per
+file the kept copy and the file are checked again (hashes included, and never the same
+file), the file's records move to the kept copy in one short `BEGIN IMMEDIATE`
+transaction, and only then is the file removed (every hash handle closed first:
+`test_handles_closed_before_unlink`), with its thumbnail (a video's `.jpg` too). A crash
+or a failed unlink therefore leaves the records on the identical kept copy, and a failed
+move leaves the file and its records alone: no row ever names a deleted file
+(`test_a_failure_between_two_files_leaves_no_row_on_a_deleted_file`). The status line
+counts the files removed and not removed. A changed file is kept (`test_changed_content_skips_file`); the
+first failed unlink stops the job, a locked or read-only file with "The file is open in
+another program or read-only; it was not deleted, and its links now name the kept copy."
+and never the OS text, and the
+files after it are listed as not attempted (`test_partial_failure_reports_each_path`).
+Records move for the reviewed files only (`database/duplicate_copies.py`): each image
+record, looked up under every spelling of the file by containment
+(`py/api/duplicate_spellings.py`: each configured, monitored, ComfyUI output or listed
+folder whose `os.stat` identity is a folder above the file, plus the path below it, so a
+parent spelling of a nested source and a case variant match:
+`test_links_under_a_symlinked_source_spelling_move`,
+`test_a_parent_spelling_of_a_nested_source`), moves to the kept copy under the same
+spelling, or, when its prompt already links the kept copy, hands its saved-pair
+links to that record and is deleted; a favorite under any listed root that holds the
+file moves with it (`test_nested_sources_subfolder_first_with_a_parent_favorite`)
+(`test_db_links_reconciled_only_for_removed_files`); the copy's image tags join the kept
+copy's as their union; and its pending Needs review items (still `pending` at revision
+0 with no decision row) are settled (`test_tags_move_as_a_union_and_pending_items_settle`).
+The plan states each of these. The find refuses while a removal
+runs and the removal while a find runs.
+
+The legacy gallery routes keep their shapes on the same kinds: `/scan_duplicates`
+holds `duplicates` (read, "Find duplicate image files (gallery page)"),
+`/scan_duplicates/stream` starts the find over the gallery's output folder or attaches
+to a running one that covers it (409 otherwise), `/scan_duplicates/status` reports the
+latest find, and `/delete_duplicate_images` holds `duplicate_cleanup` (records) and is
+refused while a find runs (`test_legacy_delete_duplicates_unchanged`). The link
+statements are index searches (`test_link_queries_use_the_path_indexes`; the fixture
+check is in the PR). Windows: the hash reads close before `os.remove`; a sharing
+violation is a `PermissionError` answered with the coded message (native on the
+Windows runners, `test_locked_file_gives_a_coded_message`).
+
+A failure the job cannot name ends it "Maintenance failed; nothing was changed." with a
+null code and no exception text. Windows: every connection the backup and the previews
+open is closed before the temp copy is renamed or removed, and the suites' teardown
+asserts no handle under the temp root is still open.
+
+The legacy `POST /prompt_manager/maintenance` keeps its shape. Its `vacuum` result is
+`success:false` with the error when VACUUM fails (`vacuum_database` now raises after
+logging), and `cleanup_orphaned_images` removes only links under a source that can be
+read now (the health check's sources), never every link of an unmounted drive.
+`cleanup_missing_images()` without sources keeps the old rule for its other callers.
+Query plans on a fixture of 5,100 prompts, 8,000 images, 300 pairs and 200 recorded
+runs: every per-prompt protection and fingerprint read is an index search, the
+duplicate groups and the link rows are one scan each, and a delete batch (400 ids)
+searches by rowid with its cascades by index.
+
+**Workflow storage** (`database/workflow_documents.py`, `database/workflow_storage.py`,
+`py/api/workflow_storage.py`; `tests/test_workflow_documents.py`,
+`tests/test_workflow_migration.py`, `tests/test_api_workflow_storage.py`). Each distinct
+workflow and prompt graph text is one `workflow_documents` row: `sha256` of its exact
+UTF-8 bytes (unique), `byte_length` and `data` = `zlib.compress` of those bytes. Image rows
+reference it through `workflow_document_id` / `prompt_document_id` (foreign keys, one
+index each). The text is stored and hashed exactly as written, never parsed and
+re-serialized. Two forms are readable forever: old (text inline, no id) and new (inline
+NULL, id set); inline text wins when both are present. Every reader goes through
+`read_stored_text` / `resolve_image_row` (image lists, image by id, prompt info for the
+inspector and metadata page, diagnostics, pair ingestion's comparison), which also drops
+the id keys so payloads keep their shape; a document that fails its length or hash check
+reads as unreadable (None), never as other text. Round trips are proven byte for byte
+over edge texts (spacing, key order, duplicate keys, number spellings, `NaN`/`Infinity`
+tokens, NUL, BOM, astral and combining characters, 500-deep nesting, a 1.2 MB+
+workflow, `""` and NULL) and seeded random text and workflows
+(`TestRoundTrip`). New writes (`link_image_to_prompt`, pair ingestion for scan, monitor
+and Needs review) store the new form. No trigger is installed (restore validators refuse
+them): a document goes in the transaction that drops its last reference (image and
+prompt deletes, missing-link cleanup, duplicate merges, duplicate-file link merges,
+ingestion updates: `TestGarbage`, `TestMoreGarbagePaths`), and startup, the migration's
+end and Reclaim space sweep anything a path missed. `verify_database_file` checks the
+table definition, the id columns, their foreign keys and indexes, the stored values and
+dangling ids (`TestSchemaErrors`); a pre-dedupe file is accepted and migrated on open
+and on restore. The `workflow_storage` maintenance plan counts the old-form records
+(one pass over record headers) and its apply starts "Store each workflow once"
+(`records`, cancellable): first, always and whatever `maintenance.safety_backup` says,
+a verified pre-migration backup named `<db>.pre-workflow-storage_<YYYYmmdd_HHMMSS_ffffff>`,
+outside the pruned `<db>.backup_*` pool and kept until the user removes it (the preview
+reports `backup.policy: "required"`; `test_the_backup_is_taken_even_with_the_policy_off`,
+`test_the_backup_is_outside_the_pool_never_pruned_and_restores`); then batches of at most
+200 rows, 16 MB of text or 0.5 s, each one `BEGIN IMMEDIATE` transaction found by a rowid
+range search and read one row at a time, followed by 0.15 s with the write lock free so
+a writer on another connection gets in (`test_a_writer_on_another_connection_gets_in_between_batches`).
+Each text is read back through the accessor before its row changes. Each row has its own
+savepoint: a row that cannot be rewritten (text SQLite cannot decode, a text that would
+not come back) stays inline, is listed in `errors` and named in the job's last line, and
+the rest go on, Reclaim space included (`TestRowErrors`); any other failure (a full disk,
+an I/O error, a malformed database, a bug) rolls the batch back whole and fails the job
+(`TestSystemErrorsFailTheBatch`, `TestSystemErrorsInTheJob`). With nothing to rewrite the
+plan says "Nothing to rewrite" and Apply answers 409 `nothing-to-do` with no backup; only
+the first apply keeps a `pre-workflow-storage` backup, and a later one (after a Cancel, a
+rerun) takes a pooled safety backup (`TestOneKeptBackup`). The job loop's pause is tested
+with a second-connection writer (`TestLiveLoopLetsWritersIn`). A reused document is decompressed and checked first and repaired from the text
+when damaged, and `verify_database_file` (restore and backup verification) reads every
+document and refuses a damaged one (`TestDamagedDocuments`). The legacy table rebuilds
+refuse a file that already has the document id columns (`TestRebuildGuards`).
+`tools/workflow_text_digests.py <db> <out>` writes the sha256 of every record's stored
+texts, so a `diff` of its output before and after the rewrite proves byte identity. It
+loads only `database/workflow_documents.py` (no package start-up, no log handler) and
+opens the database `mode=ro`, with `immutable=1` when no `-wal`/`-shm` exists, so it
+creates none (`TestDigestToolSideEffects`). A damaged document's verify reason names it
+and says what to do, and a backup that fails on it says so in the job line. A crash or Cancel leaves
+committed batches new form and the rest old form, and the next apply continues
+(`test_a_crash_mid_migration_leaves_every_row_readable_and_resumes`,
+`test_cancel_stops_after_the_current_batch_and_a_rerun_continues`). Values that are not
+text stay inline (protected). "Reclaim space" follows when every row was looked at
+(`exclusive`, not cancellable): a disk check for VACUUM's copies of the live pages,
+then sweep, VACUUM and `wal_checkpoint(TRUNCATE)`; when another job holds the registry it
+does not start and the finished migration names Optimize database. No image or video
+file is opened (`test_the_jobs_never_open_a_media_file` tracks every Python open under
+the output folder and compares bytes and mtimes). Memory stays bounded by one row plus
+the batch's distinct documents (`test_memory_stays_bounded_by_one_row`, tracemalloc).
+Restoring a deduped backup on a version before this one is not supported: that
+version's restore accepts the file and then reads rewritten images without their
+workflow (README changelog).
+
 ## Workspace data fields for the concept pages (#249)
 
 These fields are additive. The legacy `/prompt_manager/images/output` listing (no
@@ -2614,15 +3050,52 @@ still have made the file, so that text is not certain either. Both report
 `unresolved` ("could not be traced"), are never linked, and their pending,
 undecided items whose own evidence offers no choice (a limited item counts as
 a choice when it holds at least two candidates) leave the queue on the next
-scan or monitor pass (`settle_untraced_review_items`). The maintainer's
-second live check found 39 items whose 11 candidates were all "unresolved". The queue heals itself: after each batch, the scan removes the pending
+scan (`settle_untraced_review_items`). The maintainer's
+second live check found 39 items whose 11 candidates were all "unresolved".
+Inspection-profile choices (`tests/test_review_inspection.py`, ruling a): when
+the default profile is ambiguous without a choice and nothing links the file,
+the scan (never the monitor) resolves it again with `INSPECTION_PROFILE`, which
+knows `TextEncodeQwenImageEdit`, `StringConstantMultiline`, `JoinStringMulti`
+and the rest of `web/js/inspection-semantics.json`. With at least two choosable
+candidates there, the file is `ambiguous` and is queued from that profile's
+evidence, which adds `profile: "inspection"` and `semantics`: the SHA-256 of
+`class_projection`, the table restricted to the graph's node classes plus its
+class-independent `schema` and `stage_inputs` (the resolver looks the table up
+only by a graph node's class). The item's `resolver_version` is
+`inspection_review_version(metadata)`, 2^40 plus 40 bits of SHA-256 over the
+profile with its budgets, `RESOLVER_VERSION`, `INSPECTION_RESOLVER_VERSION`
+(bump it when the inspection profile would resolve the same metadata otherwise)
+and that fingerprint. A table edit about the
+graph's classes behaves like a resolver change (I2); an edit about other
+classes changes nothing. One file never holds two pending items for the same
+bytes: a new version's item supersedes the byte record's other items when all
+are pending and undecided (revision 0, no decision row, the self-heal's safe
+delete); beside a decided or advanced item nothing new is stored, and the
+rescan's stat shortcut then skips those bytes under any version. A pending item
+stays applicable while the bytes still offer its chosen candidate.
+Apply re-resolves the capture with the profile the item recorded and re-checks
+the chosen hash; the link goes through the same compatible-only ingestion. The
+inspection profile never links on its own, and ingestion keeps the default
+profile (D1). The heavier profile runs only for files already ambiguous without
+a choice, and once per file state: the capture reuses the scan's result when
+it holds the same records; a rescan skips files already queued or decided at
+their current size and `mtime_ns` under their version, and files this process
+already found untraced at the same state (an in-memory map of at most 65,536
+files, oldest forgotten first, keyed by 16-byte digests: 8.6 MiB measured at
+the cap). A file whose inspection raises is logged, counted in
+`review_errors` and reported `unresolved`; the scan goes on. An older version's
+pending, undecided item of bytes now untraced leaves the queue
+(`settle_stale_review_items`), so an item never stays stuck at 409. The 16-candidate limited
+rule and the 8 MiB bound apply unchanged. The queue heals itself: after each batch, the scan removes the pending
 items of every listed file (read or skipped as known) that a saved row now
 links, and the monitor does the same for each ambiguous output it links
 (`settle_linked_review_items`; the review index revision moves). Only an
 undecided item goes (`pending`, revision 0, no decision row): a decision is
 never deleted or changed, so no schema state was added. The monitor never
-queues an output it linked; an unlinked file passed to `record_review_file`
-follows the scan's rule (a file the fallback would link is `skipped`). The
+records an item (only scans add items, maintainer ruling 2026-10-23): an
+unlinked file passed to `record_review_file` is `skipped` with no capture,
+and only a linked one's pending items are settled
+(`test_monitor_never_records_an_unlinked_ambiguous_output`). The
 maintainer's live check of #284/#288 found the queue at 10,232 against a review
 group of 4,677: every fallback-linked ambiguous file had been queued. It captures
 the file (scope `needs-review`), reads its metadata again from the capture,
@@ -2654,9 +3127,10 @@ Apply, and waits up to 2 s for room before counting the file as not added
 pass runs after its own link, best effort: a failure is logged and the link
 stays. Recording follows the registry's `records`
 admission (#289): the scan's recording runs under its own `records` job, and
-the monitor's (one file, not a job) is refused while a `records` or `guarded`
-job or hold runs. It then records nothing, logs that the file waits for the
-next scan, and answers `deferred`; the next "new" scan reads the ambiguous file
+the monitor's pass (one file, not a job, which only settles) is refused
+while a `records` or `guarded` job or hold runs, except thumbnail work, which
+writes no records (S5 M7). It then writes nothing, logs that the file waits
+for the next scan, and answers `deferred`; the next "new" scan reads the ambiguous file
 again (`test_monitor_recording_is_refused_beside_record_changing_jobs`). A new item moves `review_index_revision`. A
 rescan does not copy or hash an ambiguous file whose current size and
 `mtime_ns` equal the verified fingerprint of bytes an item already cites (a

__init__.py

@@ -132,11 +132,18 @@ try:
     _review_api = globals().get("api")
     if _review_api is not None:
         _global_image_monitor.set_review_recorder(_review_api.record_review_file)
+        # Every settled output's preview is queued for the API's background
+        # worker (#249): the workspace never asks to generate thumbnails.
+        _global_image_monitor.set_thumbnail_queue(_review_api.queue_capture_thumbnail)
         _review_host = getattr(config.server_instance, "app", None)
         if _review_host is not None and hasattr(_review_host, "on_cleanup"):
 
-            async def _stop_review_recording(app, monitor=_global_image_monitor):
+            async def _stop_review_recording(
+                app, monitor=_global_image_monitor, api=_review_api
+            ):
                 monitor.set_review_recorder(None)
+                monitor.set_thumbnail_queue(None)
+                api.close_auto_thumbnails(final=True)
 
             _review_host.on_cleanup.insert(0, _stop_review_recording)
 

database/duplicate_copies.py

@@ -0,0 +1,215 @@
+"""Records of duplicate image files: count them and move them (#249 S6, T17).
+
+A removed copy's records move to its verified kept copy before the copy is
+unlinked (``py/api/duplicate_files.py``), one file per short ``BEGIN
+IMMEDIATE`` transaction, so no row ever names a deleted file:
+
+- its image records (``generated_images``), found under every spelling the
+  file is linked by. A record moves to the kept copy, unless its prompt
+  already links the kept copy: then its saved-pair links join that record
+  and it is deleted;
+- its favorite;
+- its image tags (``media_image_tags`` of its byte record), joined onto the
+  kept copy's byte record as their union, in order;
+- its pending Needs review items, settled: only an item still ``pending`` at
+  revision 0 with no decision row is removed (``review._SETTLE_ITEM``), so a
+  decided item is never touched.
+
+Every statement is an index search (see ``FILE_LINK_SQL``).
+"""
+
+import os
+
+from .annotation_schema import BYTES_TABLE, FILES_TABLE
+from .annotations import _BUMP_INDEX
+from .models import normalize_image_path, utc_now_iso
+from .review import _BUMP_REVIEW_INDEX, _PENDING_OF_LOCATOR, _SETTLE_ITEM
+from .review_changes import notify_review_change
+from .workflow_documents import documents_of_rows, release_documents
+
+LINKS_BY_FILE_SQL = "SELECT id, prompt_id FROM generated_images WHERE file_path = ?"
+LEGACY_LINKS_SQL = (
+    "SELECT id, prompt_id FROM generated_images "
+    "WHERE image_path = ? AND file_path IS NULL"
+)
+KEEPER_LINK_SQL = (
+    "SELECT id FROM generated_images WHERE prompt_id = ? AND file_path = ?"
+)
+MOVE_PAIR_LINKS_SQL = (
+    "UPDATE OR IGNORE image_prompt_pairs SET image_id = ? WHERE image_id = ?"
+)
+MOVE_LINK_SQL = (
+    "UPDATE generated_images SET image_path = ?, filename = ?, file_path = ? "
+    "WHERE id = ?"
+)
+DELETE_LINK_SQL = "DELETE FROM generated_images WHERE id = ?"
+FAVORITE_SQL = (
+    "SELECT created_at FROM image_favorites WHERE root_key = ? AND relative_path = ?"
+)
+INSERT_FAVORITE_SQL = "INSERT OR IGNORE INTO image_favorites VALUES (?, ?, ?)"
+DELETE_FAVORITE_SQL = (
+    "DELETE FROM image_favorites WHERE root_key = ? AND relative_path = ?"
+)
+COPY_BYTE_SQL = (
+    f"SELECT b.id FROM {FILES_TABLE} f JOIN {BYTES_TABLE} b "  # nosec B608
+    "ON b.file_id=f.id WHERE f.locator=? AND b.sha256=?"
+)
+COPY_TAG_COUNT_SQL = (
+    "SELECT COUNT(*) FROM media_image_tags WHERE byte_id=?"  # nosec B608
+)
+PENDING_COUNT_SQL = (
+    f"SELECT COUNT(*) FROM {FILES_TABLE} f "  # nosec B608
+    f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
+    "JOIN media_review_items i ON i.byte_id=b.id "
+    "WHERE f.locator=? AND +i.state='pending' AND i.revision=0 "
+    "AND NOT EXISTS (SELECT 1 FROM media_review_decisions d WHERE d.item_id=i.id)"
+)
+FILE_LINK_SQL = (
+    LINKS_BY_FILE_SQL,
+    LEGACY_LINKS_SQL,
+    KEEPER_LINK_SQL,
+    MOVE_PAIR_LINKS_SQL,
+    MOVE_LINK_SQL,
+    DELETE_LINK_SQL,
+    FAVORITE_SQL,
+    DELETE_FAVORITE_SQL,
+    COPY_BYTE_SQL,
+    COPY_TAG_COUNT_SQL,
+    PENDING_COUNT_SQL,
+)
+
+
+def _file_link_rows(conn, spelled):
+    """``(id, prompt_id)`` of the image records linking one spelling."""
+    rows = list(conn.execute(LINKS_BY_FILE_SQL, (normalize_image_path(spelled),)))
+    rows += list(conn.execute(LEGACY_LINKS_SQL, (spelled,)))
+    return [tuple(row) for row in rows]
+
+
+def _copy_byte(conn, locator, sha256):
+    row = conn.execute(COPY_BYTE_SQL, (locator, sha256)).fetchone()
+    return row[0] if row else None
+
+
+class DuplicateCopyOperations:
+    """PromptDatabase mixin: the records of duplicate copies."""
+
+    def count_copy_records(self, copies) -> dict:
+        """``{path: {links, tags, review}}`` for each copy (one read snapshot).
+
+        ``copies`` maps each path to ``{spellings, locator, sha256}``:
+        ``links`` counts the image records under any spelling, ``tags`` the
+        image tags of its bytes and ``review`` its pending, undecided Needs
+        review items.
+        """
+        from .maintenance import _read_snapshot
+
+        def read(conn):
+            found = {}
+            for path, copy in copies.items():
+                ids = {
+                    row[0]
+                    for spelled in copy["spellings"]
+                    for row in _file_link_rows(conn, spelled)
+                }
+                byte_id = _copy_byte(conn, copy["locator"], copy["sha256"])
+                tags = (
+                    conn.execute(COPY_TAG_COUNT_SQL, (byte_id,)).fetchone()[0]
+                    if byte_id
+                    else 0
+                )
+                review = conn.execute(PENDING_COUNT_SQL, (copy["locator"],))
+                found[path] = {
+                    "links": len(ids),
+                    "tags": tags,
+                    "review": review.fetchone()[0],
+                }
+            return found
+
+        return _read_snapshot(self.model.db_path, read)
+
+    def move_copy_records(self, move) -> dict:
+        """Move one removed copy's records to its kept copy (one transaction).
+
+        ``move``: ``removed`` and ``kept`` (the spellings each file is linked
+        by), ``targets`` (``{removed spelling: kept spelling}``, the spelling
+        a moved record takes; else ``kept_path``), ``favorites`` (``(root key,
+        removed key, kept root key, kept key)`` per listed root holding the
+        file), ``removed_locator``,
+        ``kept_locator``, ``sha256`` and the kept copy's ``kept_size`` and
+        ``kept_mtime_ns``. Returns ``{moved, merged, favorites, tags,
+        review}``.
+        """
+        counts = dict.fromkeys(("moved", "merged", "favorites", "tags", "review"), 0)
+        kept_keys = [normalize_image_path(path) for path in move["kept"]]
+        with self._pair_transaction() as conn:
+            for spelled in dict.fromkeys(move["removed"]):
+                target = move["targets"].get(spelled, move["kept_path"])
+                keys = list(dict.fromkeys(kept_keys + [normalize_image_path(target)]))
+                for image_id, prompt_id in _file_link_rows(conn, spelled):
+                    existing = None
+                    for key in keys:
+                        existing = conn.execute(
+                            KEEPER_LINK_SQL, (prompt_id, key)
+                        ).fetchone()
+                        if existing is not None:
+                            break
+                    if existing is not None:
+                        conn.execute(MOVE_PAIR_LINKS_SQL, (existing[0], image_id))
+                        documents = documents_of_rows(conn, "id", [image_id])
+                        conn.execute(DELETE_LINK_SQL, (image_id,))
+                        release_documents(conn, documents)
+                        counts["merged"] += 1
+                        continue
+                    conn.execute(
+                        MOVE_LINK_SQL,
+                        (
+                            target,
+                            os.path.basename(target),
+                            normalize_image_path(target),
+                            image_id,
+                        ),
+                    )
+                    counts["moved"] += 1
+            for favorite in move.get("favorites", ()):
+                counts["favorites"] += _move_favorite(conn, *favorite)
+            counts["tags"] = self._move_copy_tags(conn, move)
+            for (item_id,) in conn.execute(
+                _PENDING_OF_LOCATOR, (move["removed_locator"],)
+            ).fetchall():
+                counts["review"] += conn.execute(_SETTLE_ITEM, (item_id,)).rowcount
+            if counts["review"]:
+                conn.execute(_BUMP_REVIEW_INDEX)
+        if counts["review"]:
+            notify_review_change(self.model.db_path)
+        return counts
+
+    def _move_copy_tags(self, conn, move):
+        """Join the removed copy's image tags onto the kept copy's; the count added."""
+        removed = _copy_byte(conn, move["removed_locator"], move["sha256"])
+        if removed is None:
+            return 0
+        tags = self._byte_tags(conn, removed)
+        if not tags:
+            return 0
+        now = utc_now_iso()
+        file_id = self._annotation_owner(conn, move["kept_locator"], now)
+        byte_id, _revision, created, moved = self._annotation_byte(
+            conn, file_id, move["sha256"], move["kept_size"], move["kept_mtime_ns"], now
+        )
+        current = self._byte_tags(conn, byte_id)
+        union = current + [tag for tag in tags if tag not in current]
+        if union != current:
+            self._write_image_tags(conn, byte_id, union)
+        elif created or moved:
+            conn.execute(_BUMP_INDEX)
+        return len(union) - len(current)
+
+
+def _move_favorite(conn, root_key, removed_key, kept_root, kept_key):
+    row = conn.execute(FAVORITE_SQL, (root_key, removed_key)).fetchone()
+    if row is None:
+        return 0
+    conn.execute(INSERT_FAVORITE_SQL, (kept_root, kept_key, row[0]))
+    conn.execute(DELETE_FAVORITE_SQL, (root_key, removed_key))
+    return 1

database/health.py

@@ -189,6 +189,33 @@ def _image_owner(image_path, file_path, prefixes):
     return owner
 
 
+def readable_roots(roots):
+    """Whether each source can be read now (an unmounted drive cannot)."""
+    return [_readable(root) for root in roots]
+
+
+def link_states(rows, roots, readable):
+    """``(row, owner, state)`` for each image row ``(id, image_path, filename,
+    file_path)``: ``owner`` is the index of the source it lies under (None
+    outside every source) and ``state`` is ``missing``, ``present``,
+    ``unavailable`` (its source cannot be read, so the file is not looked at)
+    or ``outside``. ``readable`` is ``readable_roots(roots)``. The health
+    check's ``missing_files`` and the maintenance missing-link plan both
+    judge links this way, so their counts agree.
+    """
+    prefixes = [_root_prefix(root) for root in roots]
+    for row in rows:
+        owner = _image_owner(row[1], row[3], prefixes)
+        if owner is None:
+            yield row, None, "outside"
+        elif not readable[owner]:
+            yield row, owner, "unavailable"
+        elif os.path.exists(row[1]):
+            yield row, owner, "present"
+        else:
+            yield row, owner, "missing"
+
+
 def _unavailable_line(unavailable, *, partial):
     noun = "source is" if unavailable == 1 else "sources are"
     effect = "its files were" if unavailable == 1 else "their files were"
@@ -224,7 +251,7 @@ def _missing_files(conn, roots):
 
 def _check_files(rows, roots):
     """The ``missing_files`` result of the image *rows*, from the file system."""
-    readable = [_readable(root) for root in roots]
+    readable = readable_roots(roots)
     sources = [
         {
             "name": _source_name(root, index),
@@ -239,13 +266,12 @@ def _check_files(rows, roots):
             None, [_unavailable_line(unavailable, partial=False)], status="unavailable"
         )
         return {**result, "partial": False, "sources": sources}
-    prefixes = [_root_prefix(root) for root in roots]
     outside, missing = 0, []
-    for image_id, image_path, filename, file_path in rows:
-        owner = _image_owner(image_path, file_path, prefixes)
-        if owner is None:
+    for row, owner, state in link_states(rows, roots, readable):
+        if state == "outside":
             outside += 1
-        elif readable[owner] and not os.path.exists(image_path):
+        elif state == "missing":
+            image_id, image_path, filename, _file_path = row
             sources[owner]["count"] += 1
             if len(missing) < MAX_DETAIL:
                 name = filename or os.path.basename(image_path)

database/maintenance.py

@@ -0,0 +1,740 @@
+"""Reviewed maintenance plans: read-only previews and their exact apply (#249 S6).
+
+Four operations, each the native owner of a legacy ``/maintenance`` one and
+built on the same queries, so a preview counts what the legacy cleanup would
+change:
+
+- ``duplicate_prompts``: prompts whose text is equal ignoring case and outer
+  spaces (``DUPLICATE_PROMPTS_SQL``, legacy ``cleanup_duplicates``); each
+  group keeps its oldest prompt, which takes over the others' images and
+  metadata. A group with a member in a saved pair is kept whole.
+- ``optimize``: VACUUM (legacy ``vacuum``).
+- ``unused_components``: prompts nothing references (``ORPHANED_PROMPTS_SQL``,
+  legacy ``prune_orphaned_prompts``, with the #239 protections).
+- ``missing_links``: image rows whose file is gone (legacy
+  ``cleanup_orphaned_images``), counted only under a source that can be read
+  now, as the health check's ``missing_files`` judges them
+  (``health.link_states``). It deletes database rows only, never a file.
+
+``preview_maintenance`` reads one snapshot on its own ``mode=ro`` connection
+(``health.readonly_uri``) and closes it in ``finally``; files are looked at
+after the snapshot ended. It returns a ``MaintenanceDraft``: the public plan
+fields, the targets an apply needs and the fingerprint of exactly what was
+reviewed (ids, text hashes, metadata, tags, Library state and pair refs of
+every prompt involved, or each link's row and source). The fingerprint never
+leaves the server.
+
+``apply_maintenance`` recomputes that fingerprint on the write connection
+inside ``BEGIN IMMEDIATE`` and raises ``PlanChanged`` before writing anything
+when it differs; otherwise it applies the reviewed targets in batches of
+``ID_BATCH`` in that one transaction. VACUUM cannot run in a transaction, so
+``optimize`` binds to its database only (the caller checks that).
+"""
+
+import hashlib
+import json
+import os
+import shutil
+import sqlite3
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from .execution_contexts import LIVE_HINT_CTES
+from .health import BATCH_ROWS, link_states, readable_roots, readonly_uri
+from .pairs import PairDependencyError
+from .workflow_documents import documents_of_rows, release_documents
+
+MAINTENANCE_OPERATIONS = (
+    "duplicate_prompts",
+    "optimize",
+    "unused_components",
+    "missing_links",
+    # Applied as its own batched job (database/workflow_storage.py).
+    "workflow_storage",
+)
+MAX_DETAILS = 50  # detail rows a plan lists
+MAX_RESULT_ROWS = 200  # changed records an apply describes (the job lists 200)
+EXCERPT_CHARS = 80
+# Ids bound per statement: the pair-reference read binds each id twice, which
+# stays under SQLite's historical 999-variable limit.
+ID_BATCH = 400
+
+# Why a prompt no image links to is kept (#239): predicates on ``p``, in the
+# order the plan names them. ORPHANED_PROMPTS_SQL is built from them, so the
+# reasons a plan lists are exactly the rules the prune applies.
+PROTECTIONS = (
+    (
+        "EXISTS (SELECT 1 FROM prompt_pairs pp "
+        "WHERE pp.positive_prompt_id = p.id OR pp.negative_prompt_id = p.id)",
+        "in saved pairs",
+        "in a saved pair",
+    ),
+    (
+        "EXISTS (SELECT 1 FROM prompt_tags pt JOIN tags t ON pt.tag_id = t.id "
+        "WHERE pt.prompt_id = p.id AND t.name = '__protected__')",
+        "tagged __protected__",
+        "tagged __protected__",
+    ),
+    (
+        "EXISTS (SELECT 1 FROM prompt_library_state ls WHERE ls.prompt_id = p.id)",
+        "with Library state (a title or an explicit parent)",
+        "with Library state (a title or an explicit parent)",
+    ),
+    (
+        "p.id IN (SELECT prompt_id FROM live_hint_prompts)",
+        "used by recorded runs",
+        "used by a recorded run",
+    ),
+)
+
+# Exempt references: images, pairs, the __protected__ tag, Library parent
+# state, and live recorded-run hints (computed once, see LIVE_HINT_CTES).
+ORPHANED_PROMPTS_SQL = (
+    "WITH "
+    + LIVE_HINT_CTES
+    + " SELECT p.id FROM prompts p"
+    + " LEFT JOIN generated_images gi ON p.id = gi.prompt_id"
+    + " WHERE gi.prompt_id IS NULL"
+    + "".join(f" AND NOT ({predicate})" for predicate, _many, _one in PROTECTIONS)
+)
+# The prompts no image links to, with which protections keep each one.
+_UNLINKED_PROTECTIONS_SQL = (
+    "WITH "
+    + LIVE_HINT_CTES
+    + " SELECT "
+    + ", ".join(
+        f"COALESCE(SUM({predicate}), 0)" for predicate, _many, _one in PROTECTIONS
+    )
+    + ", COALESCE(SUM("
+    + " OR ".join(f"({predicate})" for predicate, _many, _one in PROTECTIONS)
+    + "), 0) FROM prompts p WHERE NOT EXISTS "
+    "(SELECT 1 FROM generated_images gi WHERE gi.prompt_id = p.id)"
+)
+
+# Duplicate prompt groups, as the legacy cleanup finds them (case-insensitive
+# text; GROUP_CONCAT order is sorted by created_at afterwards).
+DUPLICATE_PROMPTS_SQL = """
+    SELECT LOWER(TRIM(text)) as normalized_text, COUNT(*) as count,
+           GROUP_CONCAT(id) as ids,
+           GROUP_CONCAT(created_at) as created_dates
+    FROM prompts
+    GROUP BY LOWER(TRIM(text))
+    HAVING COUNT(*) > 1
+"""
+
+_LINK_ROWS_SQL = (
+    "SELECT id, image_path, filename, file_path FROM generated_images ORDER BY id"
+)
+
+# What a prompt's fingerprint covers, per batch of ids.
+_PROMPT_STATE_SQL = (
+    "SELECT id, hash, text, category, rating, notes, created_at, updated_at "
+    "FROM prompts WHERE id IN ({marks}) ORDER BY id",
+    "SELECT prompt_id, tag_id FROM prompt_tags WHERE prompt_id IN ({marks}) "
+    "ORDER BY prompt_id, tag_id",
+    "SELECT * FROM prompt_library_state WHERE prompt_id IN ({marks}) "
+    "ORDER BY prompt_id",
+    "SELECT id, positive_prompt_id, negative_prompt_id FROM prompt_pairs "
+    "WHERE positive_prompt_id IN ({marks}) OR negative_prompt_id IN ({marks}) "
+    "ORDER BY id",
+)
+
+NO_FILES_DELETED = "No image file is deleted."
+LINKS_ONLY = "Database links only; no image file is deleted."
+UNUSED_DETAIL = "No image, saved pair, Library state or recorded run uses it"
+
+
+# Why a duplicate group is kept whole (``_duplicate_conflict``).
+CONFLICT_REASONS = {
+    "collision": ("group links one image twice", "groups link one image twice"),
+    "title": (
+        "group has its own Library title",
+        "groups have their own Library titles",
+    ),
+}
+CONFLICT_DETAILS = {
+    "collision": "Kept: two copies link the same image",
+    "title": "Kept: a copy has its own Library title",
+}
+OPTIMIZE_LOCK_NOTE = (
+    "Runs alone: the database is locked while it runs, and on a large library "
+    "this takes a while. Prompt runs counted and images captured meanwhile "
+    "wait up to 30 seconds each; anything that waits longer is not recorded."
+)
+
+
+class PlanChanged(Exception):
+    """What a plan was reviewed on changed before it was applied."""
+
+
+class MergeConflict(Exception):
+    """A strict duplicate merge would drop an image link or a Library title."""
+
+
+@dataclass
+class MaintenanceDraft:
+    """A preview: the public plan fields plus what only the server keeps."""
+
+    operation: str
+    fingerprint: str
+    targets: list
+    scope: dict
+    protected: dict
+    media: dict
+    details: list = field(default_factory=list)
+
+    def public(self):
+        """The plan fields a response may carry (no fingerprint, no targets)."""
+        return {
+            "operation": self.operation,
+            "scope": dict(self.scope),
+            "protected": {
+                "count": self.protected["count"],
+                "reasons": list(self.protected["reasons"]),
+            },
+            "media": dict(self.media),
+            "details": [dict(row) for row in self.details],
+        }
+
+
+def _plural(count, one, many=None):
+    return f"{count:,} {one if count == 1 else (many or one + 's')}"
+
+
+def _excerpt(text):
+    text = " ".join(str(text or "").split())
+    if len(text) <= EXCERPT_CHARS:
+        return text
+    return text[: EXCERPT_CHARS - 1].rstrip() + "…"
+
+
+def _megabytes(size):
+    return f"{size / 2**20:,.1f} MB"
+
+
+class _Fingerprint:
+    """A streaming sha256 over JSON-encoded parts."""
+
+    def __init__(self, operation):
+        self._hash = hashlib.sha256()
+        self.add(operation)
+
+    def add(self, part):
+        encoded = json.dumps(part, separators=(",", ":"), default=str)
+        self._hash.update(encoded.encode("utf-8") + b"\n")
+
+    def hexdigest(self):
+        return "sha256:" + self._hash.hexdigest()
+
+
+def _batches(ids):
+    ids = list(ids)
+    for start in range(0, len(ids), ID_BATCH):
+        yield ids[start : start + ID_BATCH]
+
+
+def _marks(batch):
+    return ",".join("?" for _ in batch)
+
+
+def _add_prompt_state(fingerprint, conn, ids):
+    """Fold every reviewed fact about the prompts ``ids`` into ``fingerprint``."""
+    for batch in _batches(sorted(ids)):
+        marks = _marks(batch)
+        for sql in _PROMPT_STATE_SQL:
+            params = batch * sql.count("{marks}")
+            for row in conn.execute(sql.format(marks=marks), params):
+                fingerprint.add(list(row))
+
+
+def _texts(conn, ids, limit=MAX_DETAILS):
+    """``{id: text}`` of the first ``limit`` prompts, for detail rows."""
+    ids = list(ids)[:limit]
+    if not ids:
+        return {}
+    rows = conn.execute(
+        f"SELECT id, text FROM prompts WHERE id IN ({_marks(ids)})", ids
+    )
+    return {row[0]: row[1] for row in rows}
+
+
+def _orphaned_ids(conn):
+    return sorted(row[0] for row in conn.execute(ORPHANED_PROMPTS_SQL))
+
+
+def _delete_ids(conn, table, ids):
+    """Delete ``table`` rows by id in batches; returns the rows removed.
+
+    The workflow documents only those rows referenced go with them (a
+    prompt's images go by cascade), in the same transaction.
+    """
+    if table not in ("prompts", "generated_images"):
+        raise ValueError(f"Unsupported table {table!r}")
+    removed = 0
+    for batch in _batches(ids):
+        documents = documents_of_rows(
+            conn, "id" if table == "generated_images" else "prompt_id", batch
+        )
+        cursor = conn.execute(
+            f"DELETE FROM {table} WHERE id IN ({_marks(batch)})",  # nosec B608
+            batch,
+        )
+        removed += cursor.rowcount
+        release_documents(conn, documents)
+    return removed
+
+
+def _read_snapshot(db_path, read):
+    """``read(conn)`` on one ``mode=ro`` snapshot; the connection always closes."""
+    conn = sqlite3.connect(readonly_uri(db_path), uri=True)
+    try:
+        conn.row_factory = sqlite3.Row
+        conn.execute("BEGIN")  # one read snapshot for the whole plan
+        return read(conn)
+    finally:
+        conn.close()
+
+
+def _relative(image_path, root):
+    try:
+        return Path(os.path.relpath(image_path, root)).as_posix()
+    except ValueError:  # another drive (Windows)
+        return os.path.basename(image_path)
+
+
+def _source_name(root, index):
+    return os.path.basename(str(root).rstrip("\\/")) or f"Source {index + 1}"
+
+
+class MaintenanceOperations:
+    """Mixin for PromptDatabase: maintenance previews and their apply."""
+
+    # ── shared with the legacy cleanup ───────────────────────────────
+
+    def _duplicate_groups(self, conn):
+        """``(groups, paired)``: duplicate id groups, oldest first.
+
+        ``paired`` are the groups with a member in a saved pair, which the
+        cleanup keeps whole.
+        """
+        groups, paired = [], []
+        for duplicate in conn.execute(DUPLICATE_PROMPTS_SQL).fetchall():
+            ids = duplicate["ids"].split(",")
+            created_dates = duplicate["created_dates"].split(",")
+            # Sort IDs by created_at date to keep the oldest
+            id_date_pairs = list(zip(ids, created_dates))
+            id_date_pairs.sort(key=lambda x: x[1])
+            sorted_ids = [int(pair[0]) for pair in id_date_pairs]
+            try:
+                self._ensure_prompts_unpaired(conn, sorted_ids)
+            except PairDependencyError:
+                paired.append(sorted_ids)
+                continue
+            groups.append(sorted_ids)
+        return groups, paired
+
+    @staticmethod
+    def _duplicate_conflict(conn, sorted_ids):
+        """Why merging the group would lose something, or None.
+
+        ``collision``: two members link the same file, so moving the images
+        to the oldest prompt would hit ``UNIQUE(prompt_id, file_path)``.
+        ``title``: a member that would be deleted has its own Library title.
+        """
+        marks = _marks(sorted_ids)
+        collision = conn.execute(
+            "SELECT 1 FROM generated_images WHERE prompt_id IN "  # nosec B608
+            f"({marks}) AND file_path IS NOT NULL GROUP BY file_path "
+            "HAVING COUNT(DISTINCT prompt_id) > 1 LIMIT 1",
+            sorted_ids,
+        ).fetchone()
+        if collision:
+            return "collision"
+        duplicates = sorted_ids[1:]
+        titled = conn.execute(
+            "SELECT 1 FROM prompt_library_state WHERE prompt_id IN "  # nosec B608
+            f"({_marks(duplicates)}) AND trim(coalesce(title, '')) != '' LIMIT 1",
+            duplicates,
+        ).fetchone()
+        return "title" if titled else None
+
+    def _merge_duplicate_group(self, conn, sorted_ids, strict=False):
+        """Keep the oldest prompt of a group; returns (removed, images moved).
+
+        ``strict`` (the reviewed apply): a merge that would drop an image
+        link or a Library title raises MergeConflict, and every helper error
+        propagates, so the transaction rolls back. The legacy cleanup keeps
+        its old, lenient behaviour.
+        """
+        primary_id, duplicate_ids = sorted_ids[0], sorted_ids[1:]
+        if strict:
+            conflict = self._duplicate_conflict(conn, sorted_ids)
+            if conflict:
+                raise MergeConflict(conflict)
+        self.logger.debug(
+            f"Merging duplicates: keeping {primary_id}, removing {duplicate_ids}"
+        )
+        primary = conn.execute(
+            "SELECT * FROM prompts WHERE id = ?", (primary_id,)
+        ).fetchone()
+        if not primary:
+            return 0, 0
+        merged_metadata = self._merge_duplicate_metadata(
+            conn, primary_id, duplicate_ids, strict=strict
+        )
+        moved = self._transfer_images_to_primary(
+            conn, primary_id, duplicate_ids, strict=strict
+        )
+        if merged_metadata:
+            self._update_primary_with_merged_metadata(
+                conn, primary_id, merged_metadata, strict=strict
+            )
+        removed = 0
+        # Images the lenient merge could not move go with their prompt.
+        documents = documents_of_rows(conn, "prompt_id", duplicate_ids)
+        for duplicate_id in duplicate_ids:
+            conn.execute("DELETE FROM prompts WHERE id = ?", (duplicate_id,))
+            removed += 1
+            self.logger.debug(f"Removed duplicate prompt {duplicate_id}")
+        release_documents(conn, documents)
+        return removed, moved
+
+    @staticmethod
+    def _missing_link_ids(rows, roots):
+        """Ids of the ``rows`` whose file is gone under a readable source."""
+        readable = readable_roots(roots)
+        return [
+            row[0]
+            for row, _owner, state in link_states(rows, roots, readable)
+            if state == "missing"
+        ]
+
+    # ── previews ──────────────────────────────────────────────────────
+
+    def preview_maintenance(self, operation, roots=()) -> MaintenanceDraft:
+        """The reviewed plan of ``operation``; writes nothing (blocking).
+
+        ``roots`` are the sources missing links are judged under (the health
+        check's: the effective sources plus configured ones that are gone).
+        """
+        if operation not in MAINTENANCE_OPERATIONS:
+            raise ValueError(f"Unknown maintenance operation {operation!r}")
+        roots = [str(root) for root in roots]
+        if operation == "optimize":
+            return self._plan_optimize()
+        if operation == "workflow_storage":
+            return self._plan_workflow_storage()
+        if operation == "missing_links":
+            rows = _read_snapshot(self.model.db_path, self._link_rows)
+            return self._plan_missing_links(rows, roots)
+        reader = (
+            self._plan_duplicate_prompts
+            if operation == "duplicate_prompts"
+            else self._plan_unused_components
+        )
+        return _read_snapshot(self.model.db_path, reader)
+
+    @staticmethod
+    def _link_rows(conn):
+        rows = []
+        cursor = conn.execute(_LINK_ROWS_SQL)
+        while batch := cursor.fetchmany(BATCH_ROWS):
+            rows.extend(tuple(row) for row in batch)
+        return rows
+
+    def _duplicates_fingerprint(self, conn):
+        """``(groups, paired, conflicted, fingerprint)`` of the duplicate merge.
+
+        ``conflicted`` holds ``(ids, reason)`` of the groups a merge would
+        lose an image link or a Library title in (``_duplicate_conflict``);
+        the plan protects them.
+        """
+        candidates, paired = self._duplicate_groups(conn)
+        groups, conflicted = [], []
+        for ids in candidates:
+            reason = self._duplicate_conflict(conn, ids)
+            if reason:
+                conflicted.append((ids, reason))
+            else:
+                groups.append(ids)
+        fingerprint = _Fingerprint("duplicate_prompts")
+        fingerprint.add({"groups": groups, "paired": paired, "conflicted": conflicted})
+        _add_prompt_state(fingerprint, conn, [i for group in groups for i in group])
+        return groups, paired, conflicted, fingerprint.hexdigest()
+
+    def _plan_duplicate_prompts(self, conn):
+        groups, paired, conflicted, fingerprint = self._duplicates_fingerprint(conn)
+        removed = sum(len(group) - 1 for group in groups)
+        held = [ids for ids, _reason in conflicted]
+        texts = _texts(conn, [group[0] for group in (groups + paired + held)])
+        details = (
+            [
+                {
+                    "label": _excerpt(texts.get(group[0])),
+                    "detail": f"Keeps the oldest of {len(group)} copies",
+                }
+                for group in groups
+            ]
+            + [
+                {
+                    "label": _excerpt(texts.get(group[0])),
+                    "detail": "Kept: belongs to a saved pair",
+                }
+                for group in paired
+            ]
+            + [
+                {
+                    "label": _excerpt(texts.get(ids[0])),
+                    "detail": CONFLICT_DETAILS[reason],
+                }
+                for ids, reason in conflicted
+            ]
+        )
+        if groups:
+            label = (
+                f"{_plural(removed, 'duplicate prompt')} in "
+                f"{_plural(len(groups), 'group')}"
+            )
+        else:
+            label = "No duplicate prompt text"
+        reasons = []
+        if len(paired) == 1:
+            reasons.append("1 group belongs to a saved pair")
+        elif paired:
+            reasons.append(f"{len(paired):,} groups belong to saved pairs")
+        for reason, (one, many) in CONFLICT_REASONS.items():
+            count = sum(1 for _ids, why in conflicted if why == reason)
+            if count:
+                reasons.append(f"{count:,} {one if count == 1 else many}")
+        return MaintenanceDraft(
+            operation="duplicate_prompts",
+            fingerprint=fingerprint,
+            targets=groups,
+            scope={"count": removed, "label": label},
+            protected={
+                "count": sum(len(group) for group in paired + held),
+                "reasons": reasons,
+            },
+            media={"deletes_files": False, "label": NO_FILES_DELETED},
+            details=details[:MAX_DETAILS],
+        )
+
+    def _unused_fingerprint(self, conn):
+        ids = _orphaned_ids(conn)
+        fingerprint = _Fingerprint("unused_components")
+        fingerprint.add(ids)
+        _add_prompt_state(fingerprint, conn, ids)
+        return ids, fingerprint.hexdigest()
+
+    def _plan_unused_components(self, conn):
+        ids, fingerprint = self._unused_fingerprint(conn)
+        *by_reason, kept = conn.execute(_UNLINKED_PROTECTIONS_SQL).fetchone()
+        reasons = [
+            f"{count:,} {many if count != 1 else one}"
+            for count, (_predicate, many, one) in zip(by_reason, PROTECTIONS)
+            if count
+        ]
+        texts = _texts(conn, ids)
+        count = len(ids)
+        return MaintenanceDraft(
+            operation="unused_components",
+            fingerprint=fingerprint,
+            targets=ids,
+            scope={
+                "count": count,
+                "label": (
+                    _plural(count, "unused prompt component")
+                    if count
+                    else "No unused prompt components"
+                ),
+            },
+            protected={"count": kept, "reasons": reasons},
+            media={"deletes_files": False, "label": NO_FILES_DELETED},
+            details=[
+                {"label": _excerpt(texts.get(prompt_id)), "detail": UNUSED_DETAIL}
+                for prompt_id in ids[:MAX_DETAILS]
+            ],
+        )
+
+    @staticmethod
+    def _links_fingerprint(candidates):
+        fingerprint = _Fingerprint("missing_links")
+        for row, owner in candidates:
+            fingerprint.add([row[0], row[1], row[3], owner])
+        return fingerprint.hexdigest()
+
+    def _plan_missing_links(self, rows, roots):
+        readable = readable_roots(roots)
+        candidates, unavailable, outside = [], 0, 0
+        for row, owner, state in link_states(rows, roots, readable):
+            if state == "missing":
+                candidates.append((row, owner))
+            elif state == "unavailable":
+                unavailable += 1
+            elif state == "outside":
+                outside += 1
+        reasons = []
+        if unavailable:
+            reasons.append(
+                _plural(
+                    unavailable,
+                    "link under an unavailable source",
+                    "links under unavailable sources",
+                )
+            )
+        if outside:
+            reasons.append(
+                _plural(
+                    outside,
+                    "link outside the image sources",
+                    "links outside the image sources",
+                )
+            )
+        count = len(candidates)
+        return MaintenanceDraft(
+            operation="missing_links",
+            fingerprint=self._links_fingerprint(candidates),
+            targets=[row[0] for row, _owner in candidates],
+            scope={
+                "count": count,
+                "label": (
+                    _plural(count, "link to a missing file", "links to missing files")
+                    if count
+                    else "No links to missing files"
+                ),
+            },
+            protected={"count": unavailable + outside, "reasons": reasons},
+            media={"deletes_files": False, "label": LINKS_ONLY},
+            details=[
+                {
+                    "label": _relative(row[1], roots[owner]),
+                    "detail": (
+                        f"{_source_name(roots[owner], owner)} · the file is missing"
+                    ),
+                }
+                for row, owner in candidates[:MAX_DETAILS]
+            ],
+        )
+
+    def _plan_optimize(self):
+        def read(conn):
+            return [
+                conn.execute(f"PRAGMA {name}").fetchone()[0]
+                for name in ("page_size", "page_count", "freelist_count")
+            ]
+
+        page_size, page_count, free_pages = _read_snapshot(self.model.db_path, read)
+        db_path = self.model.db_path
+        wal = db_path + "-wal"
+        size = page_size * page_count
+        wal_size = os.path.getsize(wal) if os.path.exists(wal) else 0
+        free = shutil.disk_usage(os.path.dirname(os.path.abspath(db_path))).free
+        # VACUUM writes the rebuilt database through the WAL beside it.
+        return MaintenanceDraft(
+            operation="optimize",
+            fingerprint=_Fingerprint("optimize").hexdigest(),
+            targets=[],
+            scope={"count": 1, "label": "The whole database"},
+            protected={"count": 0, "reasons": []},
+            media={"deletes_files": False, "label": NO_FILES_DELETED},
+            details=[
+                {
+                    "label": "Database file",
+                    "detail": f"{_megabytes(size)}, plus {_megabytes(wal_size)} "
+                    "not yet checkpointed",
+                },
+                {
+                    "label": "Free pages",
+                    "detail": f"About {_megabytes(free_pages * page_size)} "
+                    "can be reclaimed",
+                },
+                {
+                    "label": "Lock",
+                    "detail": OPTIMIZE_LOCK_NOTE,
+                },
+                {
+                    "label": "Free space needed",
+                    "detail": f"About {_megabytes(size)} beside the database and "
+                    f"{_megabytes(size)} in SQLite's temporary folder; "
+                    f"{_megabytes(free)} available beside the database",
+                },
+            ],
+        )
+
+    # ── apply ─────────────────────────────────────────────────────────
+
+    def apply_maintenance(self, operation, fingerprint, targets, roots=()) -> dict:
+        """Apply a reviewed plan exactly, or raise PlanChanged (blocking).
+
+        Inside ``BEGIN IMMEDIATE`` the fingerprint is computed again on the
+        write connection; when it differs nothing is written. ``targets`` and
+        ``roots`` are the plan's. Returns ``{changed, rows}``: how many
+        records changed and up to ``MAX_RESULT_ROWS`` descriptions of them
+        (``{prompt_id[, kept_id]}`` or ``{image_path}``).
+        """
+        if operation not in MAINTENANCE_OPERATIONS or operation == "workflow_storage":
+            raise ValueError(f"Unknown maintenance operation {operation!r}")
+        if operation == "optimize":
+            # VACUUM cannot run in a transaction; the plan binds the database.
+            if fingerprint != _Fingerprint("optimize").hexdigest():
+                raise PlanChanged("optimize")
+            self.model.vacuum_database()
+            return {"changed": 1, "rows": []}
+        appliers = {
+            "duplicate_prompts": self._apply_duplicate_prompts,
+            "unused_components": self._apply_unused_components,
+            "missing_links": self._apply_missing_links,
+        }
+        with self._pair_transaction() as conn:
+            return appliers[operation](conn, fingerprint, targets, roots)
+
+    def _apply_duplicate_prompts(self, conn, fingerprint, targets, _roots):
+        groups, _paired, _conflicted, current = self._duplicates_fingerprint(conn)
+        if current != fingerprint or groups != targets:
+            raise PlanChanged("duplicate_prompts")
+        changed, rows = 0, []
+        for sorted_ids in groups:
+            removed, _moved = self._merge_duplicate_group(conn, sorted_ids, strict=True)
+            changed += removed
+            rows.extend(
+                {"prompt_id": duplicate_id, "kept_id": sorted_ids[0]}
+                for duplicate_id in sorted_ids[1:]
+            )
+        return {"changed": changed, "rows": rows[:MAX_RESULT_ROWS]}
+
+    def _apply_unused_components(self, conn, fingerprint, targets, _roots):
+        ids, current = self._unused_fingerprint(conn)
+        if current != fingerprint or ids != targets:
+            raise PlanChanged("unused_components")
+        texts = _texts(conn, ids, limit=MAX_RESULT_ROWS)
+        changed = _delete_ids(conn, "prompts", ids)
+        rows = [
+            {"prompt_id": prompt_id, "text": _excerpt(texts.get(prompt_id))}
+            for prompt_id in ids[:MAX_RESULT_ROWS]
+        ]
+        return {"changed": changed, "rows": rows}
+
+    def _apply_missing_links(self, conn, fingerprint, targets, roots):
+        rows = []
+        for batch in _batches(sorted(targets)):
+            rows.extend(
+                tuple(row)
+                for row in conn.execute(
+                    "SELECT id, image_path, filename, file_path FROM generated_images "
+                    f"WHERE id IN ({_marks(batch)}) ORDER BY id",  # nosec B608
+                    batch,
+                )
+            )
+        readable = readable_roots(roots)
+        candidates = []
+        for row, owner, state in link_states(rows, roots, readable):
+            if state != "missing":  # back, or its source went away
+                raise PlanChanged("missing_links")
+            candidates.append((row, owner))
+        if self._links_fingerprint(candidates) != fingerprint:
+            raise PlanChanged("missing_links")
+        changed = _delete_ids(conn, "generated_images", [row[0] for row in rows])
+        return {
+            "changed": changed,
+            "rows": [{"image_path": row[1]} for row in rows[:MAX_RESULT_ROWS]],
+        }

database/models.py

@@ -24,6 +24,13 @@ from .review_schema import (
     review_without_annotations_error,
 )
 from .saved_views import migrate_saved_views, saved_views_schema_error
+from .workflow_documents import (
+    DOCUMENT_ID_COLUMNS,
+    migrate_workflow_documents,
+    workflow_documents_schema_error,
+)
+
+
 from .execution_history_schema import (
     history_schema_error,
     migrate_execution_history,
@@ -50,6 +57,19 @@ MAX_PAIR_PROVENANCE_LENGTH = 4_000_000
 MAX_PAIR_PROVENANCE_DEPTH = 64
 
 
+def _refuse_rebuild_with_documents(columns) -> None:
+    """The legacy generated_images rebuilds copy a fixed column list.
+
+    A file that needs one (``prompt_id`` TEXT, no ``file_path``) but already
+    has the workflow document ids would lose them, so it is refused.
+    """
+    if any(column in columns for column in DOCUMENT_ID_COLUMNS):
+        raise sqlite3.DatabaseError(
+            "This database needs an old image table rebuild but already has "
+            "workflow document references; the rebuild would drop them"
+        )
+
+
 def pair_provenance_error(encoded: object) -> Optional[str]:
     """Validate the same bounded JSON object for live writes and restores."""
     if not isinstance(encoded, str):
@@ -316,6 +336,8 @@ class PromptModel:
             raise sqlite3.DatabaseError(review_error)
         migrate_annotation_state(conn)
         migrate_review_state(conn)
+        # Last: the older generated_images rebuilds run only before file_path.
+        migrate_workflow_documents(conn)
 
     def _migrate_prompt_pairs(self, conn: sqlite3.Connection) -> None:
         conn.execute("SAVEPOINT prompt_pair_schema")
@@ -711,6 +733,7 @@ class PromptModel:
             columns = {column[1]: column[2] for column in cursor.fetchall()}
             if columns.get("prompt_id") != "TEXT":
                 return
+            _refuse_rebuild_with_documents(columns)
 
             self.logger.info("Migrating foreign key types: prompt_id TEXT -> INTEGER")
             with self._table_rebuild(conn):
@@ -876,6 +899,7 @@ class PromptModel:
             }
             if "file_path" in columns:
                 return
+            _refuse_rebuild_with_documents(columns)
 
             self.logger.info("Migrating database: image uniqueness by full path")
             with self._table_rebuild(conn):
@@ -941,6 +965,10 @@ class PromptModel:
 
         Reclaims unused space and defragments the database file,
         improving query performance and reducing file size.
+
+        Raises:
+            Exception: whatever stopped VACUUM (logged first); the database is
+            then unchanged.
         """
         try:
             with closing(sqlite3.connect(self.db_path)) as conn:
@@ -948,6 +976,7 @@ class PromptModel:
                 conn.commit()
         except Exception as e:
             self.logger.error(f"Error vacuuming database: {e}")
+            raise
 
     def get_database_info(self) -> dict:
         """
@@ -1112,6 +1141,11 @@ class PromptModel:
             )
             if review_error:
                 return False, review_error
+            documents_error = workflow_documents_schema_error(
+                conn, allow_missing=not require_library_state, documents=True
+            )
+            if documents_error:
+                return False, documents_error
             return True, "ok"
         except sqlite3.Error as e:
             return False, f"Invalid SQLite database: {e}"

database/operations.py

@@ -10,7 +10,7 @@ import os
 from typing import Optional, List, Dict, Any, Set, Tuple
 
 from .models import PromptModel, normalize_image_path, utc_now_iso
-from .pairs import PairDependencyError, PairOperations
+from .pairs import PairDependencyError, PairOperations  # noqa: F401 (re-exported)
 from .library import LibraryOperations, titled
 from .image_favorites import ImageFavoriteOperations
 from .annotations import AnnotationOperations
@@ -20,10 +20,26 @@ from .saved_views import SavedViewOperations
 from .library_commands import LibraryCommandOperations
 from .vocabulary import VocabularyOperations
 from .execution_history import ExecutionHistoryOperations
-from .execution_contexts import ExecutionContextOperations, LIVE_HINT_CTES
+from .execution_contexts import ExecutionContextOperations
 from .execution_outputs import ExecutionOutputOperations
 from .history_reads import HistoryReadOperations
 from .health import HealthOperations, iter_consistency_issues
+from .duplicate_copies import DuplicateCopyOperations
+from .workflow_storage import WorkflowStorageOperations
+from .workflow_documents import (
+    DocumentReader,
+    documents_of_rows,
+    release_documents,
+    resolve_image_row,
+    stored_values,
+)
+from .maintenance import (  # noqa: F401  (ORPHANED_PROMPTS_SQL is re-exported)
+    ORPHANED_PROMPTS_SQL,
+    _LINK_ROWS_SQL,
+    MaintenanceOperations,
+    _delete_ids,
+    _orphaned_ids,
+)
 
 # Import logging system
 try:
@@ -166,28 +182,6 @@ def _coerce_file_info(metadata: Any) -> tuple:
     )
 
 
-# Exempt references: images, pairs, the __protected__ tag, Library parent
-# state, and live recorded-run hints (computed once, see LIVE_HINT_CTES).
-ORPHANED_PROMPTS_SQL = "WITH " + LIVE_HINT_CTES + """
-SELECT p.id FROM prompts p
-LEFT JOIN generated_images gi ON p.id = gi.prompt_id
-WHERE gi.prompt_id IS NULL
-AND NOT EXISTS (
-    SELECT 1 FROM prompt_pairs pp
-    WHERE pp.positive_prompt_id = p.id OR pp.negative_prompt_id = p.id
-)
-AND NOT EXISTS (
-    SELECT 1 FROM prompt_tags pt
-    JOIN tags t ON pt.tag_id = t.id
-    WHERE pt.prompt_id = p.id AND t.name = '__protected__'
-)
-AND NOT EXISTS (
-    SELECT 1 FROM prompt_library_state ls WHERE ls.prompt_id = p.id
-)
-AND p.id NOT IN (SELECT prompt_id FROM live_hint_prompts)
-"""
-
-
 # The scan's known set (#249 Activity-2): every image row, with its stored file
 # size and whether its pair link is resolved: exactly one distinct pair, the rule
 # _LINK_SUMMARY_SQL and get_image_prompt_info use. A file is known and resolved
@@ -351,6 +345,9 @@ class PromptDatabase(
     ReviewDecisionOperations,
     SavedViewOperations,
     HealthOperations,
+    MaintenanceOperations,
+    DuplicateCopyOperations,
+    WorkflowStorageOperations,
 ):
     """Database operations class for managing prompts."""
 
@@ -788,10 +785,12 @@ class PromptDatabase(
         """
         with self._pair_transaction() as conn:
             self._ensure_prompts_unpaired(conn, [prompt_id])
+            documents = documents_of_rows(conn, "prompt_id", [prompt_id])
             # First delete related images to avoid foreign key constraint
             conn.execute(
                 "DELETE FROM generated_images WHERE prompt_id = ?", (prompt_id,)
             )
+            release_documents(conn, documents)
             # Then delete the prompt
             cursor = conn.execute("DELETE FROM prompts WHERE id = ?", (prompt_id,))
             conn.commit()
@@ -815,10 +814,12 @@ class PromptDatabase(
                 return 0
             self._ensure_prompts_unpaired(conn, ids)
             placeholders = ",".join("?" * len(ids))
+            documents = documents_of_rows(conn, "prompt_id", ids)
             conn.execute(
                 f"DELETE FROM generated_images WHERE prompt_id IN ({placeholders})",
                 ids,
             )
+            release_documents(conn, documents)
             conn.execute(
                 f"DELETE FROM prompt_tags WHERE prompt_id IN ({placeholders})",
                 ids,
@@ -1541,80 +1542,14 @@ class PromptDatabase(
         self.logger.info("Starting duplicate cleanup process with image preservation")
         try:
             with self._pair_transaction() as conn:
-                # Find duplicates by text content (case-insensitive)
-                # Note: Removed ORDER BY from GROUP_CONCAT for SQLite compatibility
-                # We'll sort the IDs manually after fetching
-                cursor = conn.execute("""
-                    SELECT LOWER(TRIM(text)) as normalized_text, COUNT(*) as count,
-                           GROUP_CONCAT(id) as ids,
-                           GROUP_CONCAT(created_at) as created_dates
-                    FROM prompts
-                    GROUP BY LOWER(TRIM(text))
-                    HAVING COUNT(*) > 1
-                """)
-
-                duplicates = cursor.fetchall()
-                self.logger.debug(
-                    f"Found {len(duplicates)} groups of duplicate prompts"
-                )
+                groups, _paired = self._duplicate_groups(conn)
+                self.logger.debug(f"Found {len(groups)} groups of duplicate prompts")
                 total_removed = 0
                 total_images_transferred = 0
-
-                for duplicate in duplicates:
-                    ids = duplicate["ids"].split(",")
-                    created_dates = duplicate["created_dates"].split(",")
-
-                    # Sort IDs by created_at date to keep the oldest
-                    id_date_pairs = list(zip(ids, created_dates))
-                    id_date_pairs.sort(key=lambda x: x[1])  # Sort by date
-                    sorted_ids = [int(pair[0]) for pair in id_date_pairs]
-
-                    try:
-                        self._ensure_prompts_unpaired(conn, sorted_ids)
-                    except PairDependencyError:
-                        continue
-
-                    # Keep the oldest one (first), merge and delete the rest
-                    primary_id = sorted_ids[0]  # Keep the oldest
-                    duplicate_ids = sorted_ids[1:]
-
-                    self.logger.debug(
-                        f"Merging duplicates: keeping {primary_id}, removing "
-                        f"{duplicate_ids}"
-                    )
-
-                    # Get primary prompt details
-                    cursor = conn.execute(
-                        "SELECT * FROM prompts WHERE id = ?", (primary_id,)
-                    )
-                    primary_prompt = cursor.fetchone()
-                    if not primary_prompt:
-                        continue
-
-                    # Collect and merge metadata from all duplicates
-                    merged_metadata = self._merge_duplicate_metadata(
-                        conn, primary_id, duplicate_ids
-                    )
-
-                    # Transfer all images from duplicates to primary prompt
-                    images_transferred = self._transfer_images_to_primary(
-                        conn, primary_id, duplicate_ids
-                    )
-                    total_images_transferred += images_transferred
-
-                    # Update primary prompt with merged metadata
-                    if merged_metadata:
-                        self._update_primary_with_merged_metadata(
-                            conn, primary_id, merged_metadata
-                        )
-
-                    # Delete duplicate prompts (images already transferred)
-                    for duplicate_id in duplicate_ids:
-                        conn.execute(
-                            "DELETE FROM prompts WHERE id = ?", (duplicate_id,)
-                        )
-                        total_removed += 1
-                        self.logger.debug(f"Removed duplicate prompt {duplicate_id}")
+                for sorted_ids in groups:
+                    removed, moved = self._merge_duplicate_group(conn, sorted_ids)
+                    total_removed += removed
+                    total_images_transferred += moved
 
                 conn.commit()
 
@@ -1685,6 +1620,14 @@ class PromptDatabase(
                 filename = os.path.basename(image_path)
                 file_path = normalize_image_path(image_path)
                 file_size, width, height, image_format = _coerce_file_info(metadata)
+                # Each workflow and prompt graph is stored once (new form).
+                documents = stored_values(
+                    conn,
+                    {
+                        "workflow_data": dumps_strict(metadata.get("workflow", {})),
+                        "prompt_metadata": dumps_strict(metadata.get("prompt", {})),
+                    },
+                )
 
                 # INSERT OR IGNORE skips a file already linked to this prompt
                 # (same prompt_id + normalised full path)
@@ -1692,8 +1635,9 @@ class PromptDatabase(
                     """
                     INSERT OR IGNORE INTO generated_images
                     (prompt_id, image_path, filename, file_path, file_size, width,
-                     height, format, workflow_data, prompt_metadata, parameters)
-                    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+                     height, format, workflow_data, prompt_metadata, parameters,
+                     workflow_document_id, prompt_document_id)
+                    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                     """,
                     (
                         prompt_id_int,
@@ -1704,11 +1648,22 @@ class PromptDatabase(
                         width,
                         height,
                         image_format,
-                        dumps_strict(metadata.get("workflow", {})),
-                        dumps_strict(metadata.get("prompt", {})),
+                        documents["workflow_data"],
+                        documents["prompt_metadata"],
                         dumps_strict(metadata.get("parameters", {})),
+                        documents["workflow_document_id"],
+                        documents["prompt_document_id"],
                     ),
                 )
+                if cursor.rowcount == 0:
+                    # Already linked: a document made only for this row goes.
+                    release_documents(
+                        conn,
+                        (
+                            documents["workflow_document_id"],
+                            documents["prompt_document_id"],
+                        ),
+                    )
                 conn.commit()
 
                 if cursor.rowcount == 0:
@@ -1742,7 +1697,8 @@ class PromptDatabase(
                 """,
                 (prompt_id,),
             )
-            return [self._image_row_to_dict(row) for row in cursor.fetchall()]
+            reader = DocumentReader(conn)
+            return [self._image_row_to_dict(row, reader) for row in cursor]
 
     def get_recent_images(
         self, limit: int = 50, offset: int = 0
@@ -1768,7 +1724,8 @@ class PromptDatabase(
                 """,
                 (int(limit), max(0, int(offset))),
             )
-            return [self._image_row_to_dict(row) for row in cursor.fetchall()]
+            reader = DocumentReader(conn)
+            return [self._image_row_to_dict(row, reader) for row in cursor]
 
     def get_all_images(
         self,
@@ -1802,10 +1759,10 @@ class PromptDatabase(
 
         with self.model.get_connection() as conn:
             cursor = conn.execute(sql, params)
-            rows = cursor.fetchall()
+            reader = DocumentReader(conn)
             result = []
-            for row in rows:
-                data = self._image_row_to_dict(row)
+            for row in cursor:
+                data = self._image_row_to_dict(row, reader)
                 if row["_prompt_tags_list"]:
                     data["prompt_tags"] = row["_prompt_tags_list"].split("|||")
                 else:
@@ -1843,7 +1800,8 @@ class PromptDatabase(
                     max(0, int(offset)),
                 ),
             )
-            return [self._image_row_to_dict(row) for row in cursor.fetchall()]
+            reader = DocumentReader(conn)
+            return [self._image_row_to_dict(row, reader) for row in cursor]
 
     def get_image_by_id(self, image_id: int) -> Optional[Dict[str, Any]]:
         """
@@ -1860,7 +1818,7 @@ class PromptDatabase(
                 "SELECT * FROM generated_images WHERE id = ?", (image_id,)
             )
             row = cursor.fetchone()
-            return self._image_row_to_dict(row) if row else None
+            return self._image_row_to_dict(row, DocumentReader(conn)) if row else None
 
     def delete_image(self, image_id: int) -> bool:
         """
@@ -1873,31 +1831,54 @@ class PromptDatabase(
             bool: True if deletion was successful
         """
         with self.model.get_connection() as conn:
+            documents = documents_of_rows(conn, "id", [image_id])
             cursor = conn.execute(
                 "DELETE FROM generated_images WHERE id = ?", (image_id,)
             )
+            release_documents(conn, documents)
             conn.commit()
             return cursor.rowcount > 0
 
-    def cleanup_missing_images(self) -> int:
+    def cleanup_missing_images(self, roots=None) -> int:
         """
         Remove image records where the actual file no longer exists.
 
+        Args:
+            roots: The image sources links are judged under (the health
+                check's). When given, only a link under a source that can be
+                read now is removed: a link under an unavailable source (an
+                unmounted drive) or outside every source is kept. None keeps
+                the old rule: every link whose file does not exist.
+
         Returns:
             int: Number of orphaned records removed
         """
+        if roots is not None:
+            with self.model.get_connection() as conn:
+                rows = [tuple(row) for row in conn.execute(_LINK_ROWS_SQL)]
+            # Files are looked at before the write transaction starts.
+            ids = self._missing_link_ids(rows, [str(root) for root in roots])
+            if not ids:
+                return 0
+            with self._pair_transaction() as conn:
+                return _delete_ids(conn, "generated_images", ids)
+
         removed_count = 0
 
         with self.model.get_connection() as conn:
             cursor = conn.execute("SELECT id, image_path FROM generated_images")
             images = cursor.fetchall()
 
-            for image in images:
-                if not os.path.exists(image["image_path"]):
-                    conn.execute(
-                        "DELETE FROM generated_images WHERE id = ?", (image["id"],)
-                    )
-                    removed_count += 1
+            removed = [
+                image["id"]
+                for image in images
+                if not os.path.exists(image["image_path"])
+            ]
+            documents = documents_of_rows(conn, "id", removed)
+            for image_id in removed:
+                conn.execute("DELETE FROM generated_images WHERE id = ?", (image_id,))
+                removed_count += 1
+            release_documents(conn, documents)
 
             conn.commit()
 
@@ -1923,7 +1904,11 @@ class PromptDatabase(
             return obj
 
     def _merge_duplicate_metadata(
-        self, conn: sqlite3.Connection, primary_id: int, duplicate_ids: List[int]
+        self,
+        conn: sqlite3.Connection,
+        primary_id: int,
+        duplicate_ids: List[int],
+        strict: bool = False,
     ) -> Dict[str, Any]:
         """
         Merge metadata from duplicate prompts, prioritizing non-empty values.
@@ -1983,11 +1968,17 @@ class PromptDatabase(
             return merged
 
         except Exception as e:
+            if strict:  # a reviewed apply rolls back instead
+                raise
             self.logger.error(f"Error merging metadata: {e}", exc_info=True)
             return {}
 
     def _transfer_images_to_primary(
-        self, conn: sqlite3.Connection, primary_id: int, duplicate_ids: List[int]
+        self,
+        conn: sqlite3.Connection,
+        primary_id: int,
+        duplicate_ids: List[int],
+        strict: bool = False,
     ) -> int:
         """
         Transfer all images from duplicate prompts to the primary prompt.
@@ -2019,11 +2010,17 @@ class PromptDatabase(
             return transferred_count
 
         except Exception as e:
+            if strict:  # a reviewed apply rolls back instead
+                raise
             self.logger.error(f"Error transferring images: {e}", exc_info=True)
             return 0
 
     def _update_primary_with_merged_metadata(
-        self, conn: sqlite3.Connection, primary_id: int, merged_metadata: Dict[str, Any]
+        self,
+        conn: sqlite3.Connection,
+        primary_id: int,
+        merged_metadata: Dict[str, Any],
+        strict: bool = False,
     ) -> None:
         """
         Update the primary prompt with merged metadata.
@@ -2054,6 +2051,8 @@ class PromptDatabase(
             )
 
         except Exception as e:
+            if strict:  # a reviewed apply rolls back instead
+                raise
             self.logger.error(
                 f"Error updating primary prompt metadata: {e}", exc_info=True
             )
@@ -2170,11 +2169,13 @@ class PromptDatabase(
         count = 0
         with self._pair_transaction() as conn:
             self._ensure_prompts_unpaired(conn, prompt_ids)
+            documents = documents_of_rows(conn, "prompt_id", prompt_ids)
             for pid in prompt_ids:
                 conn.execute("DELETE FROM generated_images WHERE prompt_id = ?", (pid,))
                 cursor = conn.execute("DELETE FROM prompts WHERE id = ?", (pid,))
                 if cursor.rowcount > 0:
                     count += 1
+            release_documents(conn, documents)
             conn.commit()
         return count
 
@@ -2248,7 +2249,8 @@ class PromptDatabase(
             cursor = conn.execute(
                 """SELECT gi.prompt_id, p.text, p.category, p.rating, p.notes,
                           gi.workflow_data, gi.prompt_metadata, gi.generation_time,
-                          gi.parameters, gi.id, gi.image_path, gi.file_path
+                          gi.parameters, gi.id, gi.image_path, gi.file_path,
+                          gi.workflow_document_id, gi.prompt_document_id
                    FROM generated_images gi
                    JOIN prompts p ON gi.prompt_id = p.id
                    WHERE gi.image_path = ? OR gi.file_path = ?
@@ -2328,12 +2330,21 @@ class PromptDatabase(
                         "text_source": source,
                     }
             tags = self._get_prompt_tags(conn, row[0])
+            texts = resolve_image_row(DocumentReader(conn), row)
             try:
-                workflow = json.loads(row[5]) if row[5] else None
+                workflow = (
+                    json.loads(texts["workflow_data"])
+                    if texts["workflow_data"]
+                    else None
+                )
             except (json.JSONDecodeError, TypeError):
                 workflow = None
             try:
-                prompt_meta = json.loads(row[6]) if row[6] else None
+                prompt_meta = (
+                    json.loads(texts["prompt_metadata"])
+                    if texts["prompt_metadata"]
+                    else None
+                )
             except (json.JSONDecodeError, TypeError):
                 prompt_meta = None
             return {
@@ -2567,16 +2578,12 @@ class PromptDatabase(
         Returns number of prompts removed.
         """
         with self._pair_transaction() as conn:
-            cursor = conn.execute(ORPHANED_PROMPTS_SQL)
-            orphaned = [row["id"] for row in cursor.fetchall()]
+            orphaned = _orphaned_ids(conn)
             if not orphaned:
                 return 0
-            placeholders = ",".join(["?"] * len(orphaned))
-            cursor = conn.execute(
-                f"DELETE FROM prompts WHERE id IN ({placeholders})", orphaned
-            )
+            removed = _delete_ids(conn, "prompts", orphaned)
             conn.commit()
-            return cursor.rowcount
+            return removed
 
     def check_consistency(self) -> List[str]:
         """Run consistency checks and return a list of issue descriptions.
@@ -2587,17 +2594,20 @@ class PromptDatabase(
         with self.model.get_connection() as conn:
             return list(iter_consistency_issues(conn))
 
-    def _image_row_to_dict(self, row: sqlite3.Row) -> Dict[str, Any]:
+    def _image_row_to_dict(
+        self, row: sqlite3.Row, reader: DocumentReader
+    ) -> Dict[str, Any]:
         """
         Convert an image database row to a dictionary with parsed JSON fields.
 
         Args:
             row: SQLite row object
+            reader: The workflow document reader of the row's connection
 
         Returns:
             Dictionary representation of the row
         """
-        data = dict(row)
+        data = resolve_image_row(reader, row)
 
         # Parse JSON fields
         for field in ["workflow_data", "prompt_metadata", "parameters"]:

database/pair_ingestion.py

@@ -5,6 +5,13 @@ from typing import Any, Dict
 
 from .models import normalize_image_path
 from .operations import _coerce_file_info
+from .workflow_documents import (
+    TEXT_COLUMNS,
+    DocumentReader,
+    release_documents,
+    resolve_image_row,
+    stored_values,
+)
 
 try:
     from ..utils.hashing import generate_prompt_hash
@@ -116,25 +123,35 @@ def _ingest_resolved_pair_in(
     changed = False
     if existing:
         image_id = existing["id"]
+        # Compared as stored text, whichever form the row is in.
+        current = resolve_image_row(DocumentReader(conn), existing)
         updates = {
-            key: value for key, value in snapshots.items() if existing[key] != value
+            key: value for key, value in snapshots.items() if current[key] != value
         }
         if updates:
+            replaced = [
+                existing[reference]
+                for column, reference in TEXT_COLUMNS
+                if column in updates
+            ]
+            updates = _document_form(conn, updates)
             conn.execute(
                 "UPDATE generated_images SET "
                 + ",".join(f"{key}=?" for key in updates)
                 + " WHERE id=?",
                 [*updates.values(), image_id],
             )
+            release_documents(conn, replaced)
             changed = True
     else:
-        columns = ["prompt_id", "image_path", "filename", "file_path", *snapshots]
+        stored = _document_form(conn, snapshots)
+        columns = ["prompt_id", "image_path", "filename", "file_path", *stored]
         values = [
             pair["positive_prompt_id"],
             image_path,
             os.path.basename(image_path),
             normalized,
-            *snapshots.values(),
+            *stored.values(),
         ]
         image_id = conn.execute(
             "INSERT INTO generated_images("
@@ -162,3 +179,18 @@ def _ingest_resolved_pair_in(
         "pair_id": pair_id,
         "image_id": image_id,
     }
+
+
+def _document_form(conn: Any, values: Dict[str, Any]) -> Dict[str, Any]:
+    """``values`` with the workflow and prompt graph texts stored once.
+
+    Each of those columns becomes NULL beside its document id (new form);
+    every other column is unchanged.
+    """
+    texts = {
+        column: values[column]
+        for column, _reference in TEXT_COLUMNS
+        if column in values
+    }
+    rest = {key: value for key, value in values.items() if key not in texts}
+    return {**rest, **stored_values(conn, texts)}

database/review.py

@@ -90,6 +90,30 @@ _PENDING_AT = (
     "WHERE f.locator=? AND b.size=? AND b.mtime_ns=? AND i.resolver_version=? "
     "AND i.state='pending' LIMIT 1"
 )
+# Bytes verified at this stat whose item carries a decision (any version).
+_DECIDED_AT = (
+    f"SELECT 1 FROM {FILES_TABLE} f "  # nosec B608
+    f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
+    f"JOIN {ITEMS_TABLE} i ON i.byte_id=b.id "
+    "WHERE f.locator=? AND b.size=? AND b.mtime_ns=? "
+    "AND (i.state<>'pending' OR i.revision<>0 OR EXISTS (SELECT 1 FROM "
+    f"{DECISIONS_TABLE} d WHERE d.item_id=i.id)) LIMIT 1"
+)
+# The other items of one byte record (another version or evidence), with
+# whether each may be superseded: pending, revision 0, no decision row.
+_OTHER_ITEMS_OF_BYTES = (
+    "SELECT i.id, (i.state='pending' AND i.revision=0 AND NOT EXISTS "  # nosec B608
+    f"(SELECT 1 FROM {DECISIONS_TABLE} d WHERE d.item_id=i.id)) "
+    f"FROM {ITEMS_TABLE} i WHERE i.byte_id=? "
+    "AND NOT (i.resolver_version=? AND i.evidence_hash=?)"
+)
+# Pending items of the bytes verified at one stat, with their versions.
+_PENDING_AT_STAT = (
+    f"SELECT i.id,i.resolver_version FROM {FILES_TABLE} f "  # nosec B608
+    f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
+    f"JOIN {ITEMS_TABLE} i ON i.byte_id=b.id "
+    "WHERE f.locator=? AND b.size=? AND b.mtime_ns=? AND i.state='pending'"
+)
 # Pending items of one file: the candidates a link settles.
 _PENDING_OF_LOCATOR = (
     f"SELECT i.id FROM {FILES_TABLE} f "  # nosec B608
@@ -274,6 +298,12 @@ class ReviewOperations:
         ``LibraryCommandError("storage-changed")`` when the storage epoch is
         no longer ``epoch`` (a restore since the capture) and ``ValueError``
         for invalid facts or evidence.
+
+        One file never holds two pending items for the same bytes: the new
+        item supersedes the byte record's other items (an older resolver or
+        semantics version) when every one is pending and undecided (the safe
+        delete of the self-heal); when one carries a decision or a later
+        revision, it is kept and nothing new is stored (False).
         """
         _validate_target(locator, sha256, size, mtime_ns)
         _validate_spelling(spelled_path, locator)
@@ -297,6 +327,18 @@ class ReviewOperations:
             )
             if created or moved:
                 conn.execute(_BUMP_IMAGE_INDEX)
+            # The byte record is created or refreshed even when the item is
+            # refused below (an observe would do the same); that commits
+            # with the image index bump, and the review queue is unchanged.
+            others = conn.execute(
+                _OTHER_ITEMS_OF_BYTES, (byte_id, resolver_version, evidence_hash)
+            ).fetchall()
+            if any(not removable for _id, removable in others):
+                return False
+            superseded = sum(
+                conn.execute(_SETTLE_ITEM, (item_id,)).rowcount
+                for item_id, _removable in others
+            )
             cursor = conn.execute(
                 _INSERT_ITEM,
                 (
@@ -314,9 +356,9 @@ class ReviewOperations:
             )
             inserted = cursor.rowcount > 0
             refreshed = _refresh_spelling(conn, locator, spelled_path)
-            if inserted or refreshed:
+            if inserted or refreshed or superseded:
                 conn.execute(_BUMP_REVIEW_INDEX)
-        if inserted or refreshed:
+        if inserted or refreshed or superseded:
             notify_review_change(self.model.db_path)
         return inserted
 
@@ -389,6 +431,19 @@ class ReviewOperations:
         )
         return row is not None
 
+    def review_decided_at(self, locator: str, size: int, mtime_ns: int) -> bool:
+        """True when bytes verified at this stat carry a decided item.
+
+        Any version: a newer resolver or table never adds an item beside a
+        decision, so the rescan need not copy those bytes again.
+        """
+        row = (
+            self.model.get_connection()
+            .execute(_DECIDED_AT, (locator, size, mtime_ns))
+            .fetchone()
+        )
+        return row is not None
+
     def review_pending_at(
         self, locator: str, size: int, mtime_ns: int, resolver_version: int
     ) -> bool:
@@ -445,6 +500,38 @@ class ReviewOperations:
             notify_review_change(self.model.db_path)
         return removed
 
+    def settle_stale_review_items(
+        self, locator: str, size: int, mtime_ns: int, versions
+    ) -> int:
+        """Remove pending, undecided items of the bytes verified at this stat
+        whose ``resolver_version`` is not one of ``versions``.
+
+        The caller found these bytes untraced under the current versions, so
+        an older version's item could only refuse Apply. The same safe delete
+        as the other settles; one indexed read when nothing is pending.
+        """
+        _validate_locator(locator)
+        current = set(versions)
+        conn = self.model.get_connection()
+        stale = [
+            item_id
+            for item_id, version in conn.execute(
+                _PENDING_AT_STAT, (locator, size, mtime_ns)
+            )
+            if version not in current
+        ]
+        if not stale:
+            return 0
+        removed = 0
+        with self._pair_transaction() as conn:
+            for item_id in stale:
+                removed += conn.execute(_SETTLE_ITEM, (item_id,)).rowcount
+            if removed:
+                conn.execute(_BUMP_REVIEW_INDEX)
+        if removed:
+            notify_review_change(self.model.db_path)
+        return removed
+
     def settle_untraced_review_items(self, locators) -> int:
         """Remove the pending, undecided items that offer nothing to choose.
 

database/workflow_documents.py

@@ -0,0 +1,477 @@
+"""Each workflow and prompt graph stored once, compressed (#249 S6 carry-over).
+
+``generated_images.workflow_data`` (the UI workflow) and ``prompt_metadata``
+(the API prompt graph) used to hold the same large JSON text again for every
+image of a run. Now each distinct text is one ``workflow_documents`` row:
+
+- ``sha256``: the SHA-256 of the text's exact UTF-8 bytes (unique key);
+- ``byte_length``: the length of those bytes;
+- ``data``: ``zlib.compress`` of those bytes.
+
+An image row references its texts through ``workflow_document_id`` and
+``prompt_document_id``. Two forms coexist until the migration has rewritten
+every row, and both stay readable forever:
+
+- old form: the text inline in ``workflow_data`` / ``prompt_metadata``, no id;
+- new form: the inline column NULL and the document id set.
+
+The text is stored and hashed exactly as written, never parsed and
+re-serialized, because a JSON round trip can change bytes (key order,
+spacing, number spelling, duplicate keys, ComfyUI's ``NaN`` tokens).
+
+``read_stored_text`` is the one decoder. Inline text, when present, wins (a
+row an older app version rewrote inline stays right); otherwise the document
+is read by its rowid, decompressed within its recorded length and checked
+against its length and hash. A document that fails a check reads as
+unreadable (None, logged), never as other text. ``resolve_image_row`` puts
+the texts back into a ``generated_images`` row and drops the id keys, so
+every payload keeps its shape.
+
+No trigger: every restore validator refuses triggers in a restored file. A
+document is removed in code: ``release_documents`` in the same transaction
+that drops a row's last reference, and ``sweep_documents`` (at startup, after
+the migration and before Reclaim space) for anything a path missed.
+"""
+
+import hashlib
+import logging
+import re
+import sqlite3
+import zlib
+from collections import OrderedDict
+from typing import Any, Dict, Iterable, Optional, Set
+
+logger = logging.getLogger("prompt_manager.database.workflow_documents")
+
+TABLE = "workflow_documents"
+# (inline text column, document id column) on generated_images.
+TEXT_COLUMNS = (
+    ("workflow_data", "workflow_document_id"),
+    ("prompt_metadata", "prompt_document_id"),
+)
+DOCUMENT_ID_COLUMNS = tuple(reference for _column, reference in TEXT_COLUMNS)
+INDEXES = {
+    "workflow_document_id": "idx_image_workflow_document",
+    "prompt_document_id": "idx_image_prompt_document",
+}
+COMPRESSION_LEVEL = 6
+# A text larger than this stays inline: a document is decompressed in one
+# piece, and its recorded length bounds that (a restored file cannot make a
+# small row expand without limit).
+MAX_DOCUMENT_BYTES = 256 * 2**20
+ID_BATCH = 400  # ids bound per statement (under SQLite's old 999 limit)
+
+_DDL = (
+    f"CREATE TABLE {TABLE} (id INTEGER PRIMARY KEY, sha256 BLOB NOT NULL UNIQUE, "
+    "byte_length INTEGER NOT NULL, data BLOB NOT NULL)"
+)
+_AUTOINDEX = f"sqlite_autoindex_{TABLE}_1"
+_REFERENCE_TYPE = f"INTEGER REFERENCES {TABLE}(id)"
+
+_UNREFERENCED = (
+    "NOT EXISTS (SELECT 1 FROM generated_images g "
+    "WHERE g.workflow_document_id=workflow_documents.id) "
+    "AND NOT EXISTS (SELECT 1 FROM generated_images g "
+    "WHERE g.prompt_document_id=workflow_documents.id)"
+)
+LOOKUP_SQL = (
+    "SELECT id, sha256, byte_length, data FROM workflow_documents WHERE sha256=?"
+)
+REPAIR_SQL = "UPDATE workflow_documents SET byte_length=?, data=? WHERE id=?"
+ALL_DOCUMENTS_SQL = "SELECT sha256, byte_length, data, id FROM workflow_documents"
+READ_SQL = "SELECT sha256, byte_length, data FROM workflow_documents WHERE id=?"
+INSERT_SQL = (
+    "INSERT INTO workflow_documents (sha256, byte_length, data) VALUES (?, ?, ?)"
+)
+# Built from constants only; the ids are bound as "?" marks.
+RELEASE_SQL = "DELETE FROM workflow_documents WHERE id IN ({marks}) AND "
+RELEASE_SQL += _UNREFERENCED
+# The sweep walks the ids through the small unique index, not the blobs.
+SWEEP_SQL = (
+    "DELETE FROM workflow_documents WHERE id IN (SELECT id FROM workflow_documents "
+    "INDEXED BY sqlite_autoindex_workflow_documents_1 WHERE "
+)
+SWEEP_SQL += _UNREFERENCED + ")"
+IMAGE_DOCUMENTS_SQL = (
+    "SELECT workflow_document_id, prompt_document_id FROM generated_images WHERE "
+)
+
+# Statements whose query plans tests/test_workflow_documents.py records.
+QUERY_PLAN_STATEMENTS = {
+    "lookup by hash": (LOOKUP_SQL, (b"\0" * 32,)),
+    "read by id": (READ_SQL, (1,)),
+    "release": (RELEASE_SQL.format(marks="?,?"), (1, 2)),
+    "sweep": (SWEEP_SQL, ()),
+    "documents of images": (IMAGE_DOCUMENTS_SQL + "id IN (?,?)", (1, 2)),
+    "documents of prompts": (IMAGE_DOCUMENTS_SQL + "prompt_id IN (?,?)", (1, 2)),
+}
+
+
+class DocumentMismatch(Exception):
+    """A document would not give back exactly the text it was made from."""
+
+
+def _compact(sql: str) -> str:
+    return re.sub(r"\s+", "", sql or "").lower()
+
+
+# ── schema ────────────────────────────────────────────────────────────
+
+
+def _reference_columns(conn: sqlite3.Connection) -> Dict[str, tuple]:
+    return {
+        row[1]: tuple(row)
+        for row in conn.execute("PRAGMA table_info(generated_images)")
+        if row[1] in DOCUMENT_ID_COLUMNS
+    }
+
+
+def _reference_error(conn: sqlite3.Connection, columns: Dict[str, tuple]):
+    for name, info in columns.items():
+        # (cid, name, type, notnull, default, pk)
+        if (str(info[2]).upper(), info[3], info[4], info[5]) != ("INTEGER", 0, None, 0):
+            return "Workflow document references have an invalid column"
+    keys = {
+        (row[3], row[2], row[4])
+        for row in conn.execute("PRAGMA foreign_key_list(generated_images)")
+        if row[3] in DOCUMENT_ID_COLUMNS
+    }
+    if keys != {(name, TABLE, "id") for name in DOCUMENT_ID_COLUMNS}:
+        return "Workflow document references have invalid foreign keys"
+    for column, index in INDEXES.items():
+        info = conn.execute(f'PRAGMA index_info("{index}")').fetchall()
+        owner = conn.execute(
+            "SELECT tbl_name FROM sqlite_master WHERE type='index' AND name=?",
+            (index,),
+        ).fetchone()
+        if [row[2] for row in info] != [column] or not owner:
+            return "Workflow document references are missing their index"
+        if owner[0] != "generated_images":
+            return "Workflow document references are missing their index"
+    return None
+
+
+def _value_error(conn: sqlite3.Connection):
+    if conn.execute(
+        "SELECT 1 FROM workflow_documents WHERE typeof(sha256)!='blob' "
+        "OR length(sha256)!=32 OR typeof(byte_length)!='integer' OR byte_length<0 "
+        "OR byte_length>? OR typeof(data)!='blob' LIMIT 1",
+        (MAX_DOCUMENT_BYTES,),
+    ).fetchone():
+        return "Workflow documents contain invalid values"
+    for column in DOCUMENT_ID_COLUMNS:
+        # typeof() reads no overflow page; a dangling id is a rowid probe.
+        # ``column`` is one of DOCUMENT_ID_COLUMNS, never caller data.
+        sql = (
+            f"SELECT 1 FROM generated_images g WHERE typeof(g.{column}) "  # nosec B608
+            f"NOT IN ('null','integer') OR (g.{column} IS NOT NULL AND NOT EXISTS "
+            f"(SELECT 1 FROM workflow_documents d WHERE d.id=g.{column})) LIMIT 1"
+        )
+        if conn.execute(sql).fetchone():
+            return "Workflow document references point at missing documents"
+    return None
+
+
+def damaged_message(document_id: int) -> str:
+    """The verify reason for a damaged document: which one, and what to do."""
+    return (
+        f"Workflow document {document_id} is damaged: its text no longer matches "
+        "its checksum, and the images that use it show no workflow. To fix it, "
+        "restore a backup from before the damage, or store a new image with the "
+        "same workflow (that repairs it)."
+    )
+
+
+def _damaged_document(conn: sqlite3.Connection) -> Optional[int]:
+    """The first document failing its length or hash check, or None."""
+    for row in conn.execute(ALL_DOCUMENTS_SQL):
+        if _decode(row[:3], row[3], quiet=True) is None:
+            return row[3]
+    return None
+
+
+def workflow_documents_schema_error(
+    conn: sqlite3.Connection, *, allow_missing: bool = True, documents: bool = False
+) -> Optional[str]:
+    """Reject an unsupported document schema or invalid stored values (reads only).
+
+    A database from before the dedupe has neither the table nor the id
+    columns; that is acceptable unless ``allow_missing`` is false. Half of
+    the schema never is.
+    """
+    objects = conn.execute(
+        "SELECT type,name,sql FROM sqlite_master WHERE name=? "
+        "OR (type IN ('index','trigger') AND tbl_name=?)",
+        (TABLE, TABLE),
+    ).fetchall()
+    has_images = conn.execute(
+        "SELECT 1 FROM sqlite_master WHERE type='table' AND name='generated_images'"
+    ).fetchone()
+    columns = _reference_columns(conn) if has_images else {}
+    if not objects and not columns:
+        if allow_missing or not has_images:
+            return None
+        return "Missing workflow document schema"
+    if not objects or set(columns) != set(DOCUMENT_ID_COLUMNS):
+        return "Incomplete workflow document schema"
+    tables = [sql for kind, _name, sql in objects if kind == "table"]
+    others = {(kind, name, sql) for kind, name, sql in objects if kind != "table"}
+    if len(tables) != 1 or others != {("index", _AUTOINDEX, None)}:
+        return "Workflow documents name is reserved by an unsupported schema object"
+    # The exact definition fixes the columns, their types and constraints.
+    if _compact(tables[0]) != _compact(_DDL):
+        return "Workflow documents have an invalid table definition"
+    error = _reference_error(conn, columns) or _value_error(conn)
+    if error is None and documents:
+        # Restore and backup verification only: every document is read.
+        damaged = _damaged_document(conn)
+        if damaged is not None:
+            return damaged_message(damaged)
+    return error
+
+
+def migrate_workflow_documents(conn: sqlite3.Connection) -> None:
+    """Install the table, the id columns and their indexes atomically.
+
+    Runs after every older migration (the table rebuilds only run on
+    databases without ``file_path``, before this). Validates before and
+    after, then sweeps documents no row references.
+    """
+    conn.execute("SAVEPOINT workflow_documents_schema")
+    try:
+        error = workflow_documents_schema_error(conn)
+        if error:
+            raise sqlite3.DatabaseError(error)
+        conn.execute(_DDL.replace("CREATE TABLE", "CREATE TABLE IF NOT EXISTS", 1))
+        present = _reference_columns(conn)
+        for column in DOCUMENT_ID_COLUMNS:
+            if column not in present:
+                conn.execute(
+                    f"ALTER TABLE generated_images ADD COLUMN {column} "
+                    f"{_REFERENCE_TYPE}"
+                )
+        for column, index in INDEXES.items():
+            conn.execute(
+                f"CREATE INDEX IF NOT EXISTS {index} ON generated_images({column})"
+            )
+        error = workflow_documents_schema_error(conn, allow_missing=False)
+        if error:
+            raise sqlite3.DatabaseError(error)
+        sweep_documents(conn)
+        conn.execute("RELEASE workflow_documents_schema")
+    except BaseException:
+        conn.execute("ROLLBACK TO workflow_documents_schema")
+        conn.execute("RELEASE workflow_documents_schema")
+        raise
+
+
+# ── write ─────────────────────────────────────────────────────────────
+
+
+def storable(text: Any) -> bool:
+    """Whether ``text`` can be a document (text within MAX_DOCUMENT_BYTES)."""
+    if not isinstance(text, str):
+        return False
+    # UTF-8 takes at most 4 bytes a character: most texts need no encoding.
+    if len(text) * 4 <= MAX_DOCUMENT_BYTES:
+        return True
+    return len(text.encode("utf-8")) <= MAX_DOCUMENT_BYTES
+
+
+def store_text(
+    conn: sqlite3.Connection, text: Optional[str], verified: Optional[set] = None
+) -> Optional[int]:
+    """The id of the document holding exactly ``text``; None for None.
+
+    Reuses the document with the same hash, or inserts a new one whose
+    compressed bytes were checked to decompress to ``text``'s bytes. A reused
+    document is decompressed and checked first; a damaged one (a restored
+    file's corruption under a valid hash) is repaired from ``text``, whose
+    hash is its key, so every row that references it reads right again.
+    ``verified``, when given, holds the ids already checked (a migration
+    batch), which are then not read again. Opens a write transaction first
+    when none is open, so a concurrent ``release_documents`` cannot remove a
+    reused document before the caller references it. Raises TypeError for a
+    non-text value and ValueError for a text over MAX_DOCUMENT_BYTES.
+    """
+    if text is None:
+        return None
+    if not isinstance(text, str):
+        raise TypeError("Only text is stored as a workflow document")
+    raw = text.encode("utf-8")
+    if len(raw) > MAX_DOCUMENT_BYTES:
+        raise ValueError("Text is too large for a workflow document")
+    digest = hashlib.sha256(raw).digest()
+    if not conn.in_transaction:
+        conn.execute("BEGIN IMMEDIATE")
+    row = conn.execute(LOOKUP_SQL, (digest,)).fetchone()
+    if row is not None and verified is not None and row[0] in verified:
+        return row[0]
+    data = zlib.compress(raw, COMPRESSION_LEVEL)
+    if zlib.decompress(data) != raw:  # pragma: no cover - zlib is exact
+        raise DocumentMismatch("Compression did not round-trip")
+    if row is None:
+        document_id = conn.execute(INSERT_SQL, (digest, len(raw), data)).lastrowid
+    else:
+        document_id = row[0]
+        if _decode(row[1:], document_id, quiet=True) != text:
+            logger.warning(f"Workflow document {document_id} was damaged; repaired")
+            conn.execute(REPAIR_SQL, (len(raw), data, document_id))
+    if verified is not None:
+        verified.add(document_id)
+    return document_id
+
+
+def stored_values(
+    conn: sqlite3.Connection, texts: Dict[str, Optional[str]]
+) -> Dict[str, Any]:
+    """New-form column values for ``{text column: text}`` (writers).
+
+    Each text becomes ``{column: None, id column: document id}``; None
+    stays ``{column: None, id column: None}``; a value that cannot be a
+    document (not text, or too large) stays inline with no id.
+    """
+    references = dict(TEXT_COLUMNS)
+    values: Dict[str, Any] = {}
+    for column, text in texts.items():
+        reference = references[column]
+        if text is None or not storable(text):
+            values[column], values[reference] = text, None
+        else:
+            values[column], values[reference] = None, store_text(conn, text)
+    return values
+
+
+# ── read ──────────────────────────────────────────────────────────────
+
+
+def _decode(row, document_id, quiet=False) -> Optional[str]:
+    """``(sha256, byte_length, data)`` as checked text, or None (logged)."""
+    if row is None:
+        logger.warning(f"Workflow document {document_id} is missing")
+        return None
+    digest, length, data = bytes(row[0]), row[1], row[2]
+    try:
+        if not isinstance(length, int) or not 0 <= length <= MAX_DOCUMENT_BYTES:
+            raise ValueError("invalid length")
+        inflater = zlib.decompressobj()
+        raw = inflater.decompress(bytes(data), length + 1)
+        if len(raw) != length or inflater.unconsumed_tail or not inflater.eof:
+            raise ValueError("length differs")
+        if hashlib.sha256(raw).digest() != digest:
+            raise ValueError("hash differs")
+        return raw.decode("utf-8")
+    except (zlib.error, ValueError, TypeError) as error:
+        if not quiet:
+            logger.warning(f"Workflow document {document_id} is unreadable: {error}")
+        return None
+
+
+def read_stored_text(
+    conn: sqlite3.Connection, inline: Any, document_id: Optional[int]
+) -> Any:
+    """The stored text of one column: inline when present, else the document.
+
+    Returns None when there is neither, or when the document is missing or
+    fails its checks (logged). An inline value is returned unchanged.
+    """
+    if inline is not None:
+        return inline
+    if document_id is None:
+        return None
+    return _decode(conn.execute(READ_SQL, (document_id,)).fetchone(), document_id)
+
+
+class DocumentReader:
+    """``read_stored_text`` with a small memo for list reads on one connection."""
+
+    MEMO = 32
+
+    def __init__(self, conn: sqlite3.Connection):
+        self.conn = conn
+        self._memo: "OrderedDict[int, Optional[str]]" = OrderedDict()
+
+    def text(self, inline: Any, document_id: Optional[int]) -> Any:
+        if inline is not None or document_id is None:
+            return read_stored_text(self.conn, inline, document_id)
+        if document_id in self._memo:
+            self._memo.move_to_end(document_id)
+            return self._memo[document_id]
+        text = read_stored_text(self.conn, None, document_id)
+        self._memo[document_id] = text
+        if len(self._memo) > self.MEMO:
+            self._memo.popitem(last=False)
+        return text
+
+
+def resolve_image_row(reader: DocumentReader, row) -> Dict[str, Any]:
+    """A ``generated_images`` row as a dict with its texts read back.
+
+    The id columns are dropped, so a payload built from it has the shape it
+    had before the dedupe. Columns the row did not select stay absent.
+    """
+    data = dict(row)
+    for column, reference in TEXT_COLUMNS:
+        has_reference = reference in data
+        document_id = data.pop(reference, None)
+        if column in data or has_reference:
+            data[column] = reader.text(data.get(column), document_id)
+    return data
+
+
+# ── garbage ───────────────────────────────────────────────────────────
+
+
+def _chunks(values):
+    values = list(values)
+    for start in range(0, len(values), ID_BATCH):
+        yield values[start : start + ID_BATCH]
+
+
+def image_documents(conn: sqlite3.Connection, where: str, params) -> Set[int]:
+    """The document ids the image rows matching ``where`` reference.
+
+    ``where`` is a fixed condition written in this package (``id IN (...)``,
+    ``prompt_id IN (...)``), never caller data; values go in ``params``.
+    """
+    ids: Set[int] = set()
+    sql = IMAGE_DOCUMENTS_SQL + where
+    for row in conn.execute(sql, list(params)):  # nosec B608
+        ids.update(value for value in row if value is not None)
+    return ids
+
+
+def documents_of_rows(
+    conn: sqlite3.Connection, column: str, values: Iterable[Any]
+) -> Set[int]:
+    """Document ids of the image rows whose ``column`` is in ``values``.
+
+    ``column`` is ``id`` or ``prompt_id`` (the rows a prompt delete cascades to).
+    """
+    if column not in ("id", "prompt_id"):
+        raise ValueError(f"Unsupported column {column!r}")
+    ids: Set[int] = set()
+    for chunk in _chunks(values):
+        marks = ",".join("?" for _ in chunk)
+        ids |= image_documents(conn, f"{column} IN ({marks})", chunk)
+    return ids
+
+
+def release_documents(conn: sqlite3.Connection, ids: Iterable[Optional[int]]) -> int:
+    """Delete the documents in ``ids`` that no image row references now.
+
+    Runs in the caller's transaction, right after the delete or update that
+    dropped the references. Returns how many documents were removed.
+    """
+    removed = 0
+    for chunk in _chunks(sorted({i for i in ids if i is not None})):
+        marks = ",".join("?" for _ in chunk)
+        # Only "?" marks are interpolated.
+        sql = RELEASE_SQL.format(marks=marks)
+        removed += conn.execute(sql, chunk).rowcount  # nosec B608
+    return removed
+
+
+def sweep_documents(conn: sqlite3.Connection) -> int:
+    """Delete every document no image row references; returns the count."""
+    return conn.execute(SWEEP_SQL).rowcount

database/workflow_storage.py

@@ -0,0 +1,455 @@
+"""The workflow storage migration and Reclaim space (#249 S6 carry-over).
+
+``migrate_workflow_storage`` rewrites old-form image rows (workflow and
+prompt graph text inline) to the new form (a ``workflow_documents`` id,
+``database/workflow_documents.py``) in batches:
+
+- each batch is one ``BEGIN IMMEDIATE`` transaction over at most
+  ``BATCH_ROWS`` rows or about ``BATCH_BYTES`` of text, found by a rowid
+  range search (``NEXT_IDS_SQL``) and read one row at a time, so memory is
+  bounded by one row plus the batch's distinct documents;
+- each text is stored (or its document reused) and read back through the
+  one accessor before its row changes; a text that would not come back
+  identical fails the batch (``DocumentMismatch``) and rolls it back whole;
+- a value that is not text (or is over ``MAX_DOCUMENT_BYTES``) stays inline
+  and is counted as protected;
+- each row runs in its own savepoint: one that cannot be rewritten (text
+  SQLite cannot decode, a text that would not come back) stays inline and
+  is listed, and the rest of the batch goes on;
+- a batch ends after ``BATCH_SECONDS``, and the write lock is then left
+  free for ``BATCH_PAUSE``, so other writers are never starved.
+
+A crash or a stop between batches leaves committed batches in the new form
+and the rest in the old form; both read the same, and a rerun continues
+from the first old-form row. Only database rows change: no image or video
+file is opened.
+
+``reclaim_space`` is the VACUUM afterwards: it sweeps unreferenced
+documents, rebuilds the file and checkpoints the WAL with TRUNCATE, so the
+freed pages go back to the disk.
+"""
+
+import logging
+import os
+import sqlite3
+import time
+from contextlib import closing
+
+from .workflow_documents import (
+    TEXT_COLUMNS,
+    DocumentMismatch,
+    read_stored_text,
+    release_documents,
+    storable,
+    store_text,
+    sweep_documents,
+)
+
+OPERATION = "workflow_storage"
+# The first apply's backup: ``<db>.pre-workflow-storage_<stamp>``, outside the
+# pruned ``<db>.backup_*`` pool. Only one is ever made; later applies take a
+# pooled safety backup.
+PRE_MIGRATION_LABEL = "pre-workflow-storage"
+BATCH_ROWS = 200
+BATCH_BYTES = 16 * 2**20  # characters of text rewritten per transaction
+# A batch holds the write lock: it ends after this long, and the loop then
+# leaves the lock free for BATCH_PAUSE, longer than SQLite's longest busy
+# poll (100 ms), so the image monitor and the run counting get in between.
+BATCH_SECONDS = 0.5
+BATCH_PAUSE = 0.15
+
+
+def _row_error(error) -> bool:
+    """Whether ``error`` is one row's own cause, which keeps only that row inline.
+
+    Text SQLite cannot decode for Python, and a text that would not come
+    back identical. Anything else (a full disk, an I/O error, a malformed
+    database, a bug) fails the whole batch. The decode error is matched by
+    its message: ``sqlite_errorcode`` needs Python 3.11.
+    """
+    if isinstance(error, (DocumentMismatch, UnicodeError)):
+        return True
+    return isinstance(error, sqlite3.OperationalError) and str(error).startswith(
+        "Could not decode"
+    )
+
+
+logger = logging.getLogger("prompt_manager.database.workflow_storage")
+_clock = time.monotonic
+
+# typeof() reads a column's type from the record header: no overflow page.
+NEXT_IDS_SQL = (
+    "SELECT id FROM generated_images WHERE id>? AND "
+    "(typeof(workflow_data)!='null' OR typeof(prompt_metadata)!='null') "
+    "ORDER BY id LIMIT ?"
+)
+ROW_SQL = (
+    "SELECT workflow_data, prompt_metadata, workflow_document_id, "
+    "prompt_document_id FROM generated_images WHERE id=?"
+)
+UPDATE_SQL = (
+    "UPDATE generated_images SET {assignments} WHERE id=?"  # columns are constants
+)
+# One pass over the record headers: rows with inline text, and inline
+# values that are not text (they stay inline).
+INLINE_COUNTS_SQL = (
+    "SELECT "
+    "SUM(typeof(workflow_data)='text' OR typeof(prompt_metadata)='text'), "
+    "SUM(typeof(workflow_data) NOT IN ('null','text')) "
+    "+ SUM(typeof(prompt_metadata) NOT IN ('null','text')) "
+    "FROM generated_images"
+)
+# octet_length() (SQLite 3.43+) reads no overflow page either.
+INLINE_BYTES_SQL = (
+    "SELECT SUM(octet_length(workflow_data)), SUM(octet_length(prompt_metadata)) "
+    "FROM generated_images"
+)
+
+QUERY_PLAN_STATEMENTS = {
+    "next old-form rows": (NEXT_IDS_SQL, (0, BATCH_ROWS)),
+    "one row": (ROW_SQL, (1,)),
+}
+
+
+def _rewrite_row(conn, image_id, row, verified, checked):
+    """Store the row's inline texts as documents; returns (changed, protected, size).
+
+    ``verified`` maps document ids already read back in this batch to their
+    text, and ``checked`` holds the reused ids ``store_text`` already
+    checked, so a document shared by many rows is decompressed once.
+    """
+    changes, replaced = {}, []
+    protected = size = 0
+    for index, (column, reference) in enumerate(TEXT_COLUMNS):
+        value, current = row[index], row[index + len(TEXT_COLUMNS)]
+        if value is None:
+            continue
+        if not storable(value):
+            protected += 1
+            continue
+        document_id = store_text(conn, value, checked)
+        if document_id not in verified:
+            verified[document_id] = read_stored_text(conn, None, document_id)
+        if verified[document_id] != value:
+            raise DocumentMismatch(f"Image {image_id}: {column} would change")
+        changes[column], changes[reference] = None, document_id
+        if current is not None and current != document_id:
+            replaced.append(current)
+        size += len(value)
+    if changes:
+        assignments = ",".join(f"{key}=?" for key in changes)
+        conn.execute(
+            UPDATE_SQL.format(assignments=assignments),
+            [*changes.values(), image_id],
+        )
+    release_documents(conn, replaced)
+    return bool(changes), protected, size
+
+
+def migrate_batch(
+    conn,
+    after_id,
+    max_rows=BATCH_ROWS,
+    max_bytes=BATCH_BYTES,
+    max_seconds=BATCH_SECONDS,
+):
+    """Rewrite the next batch after ``after_id`` in one transaction.
+
+    The batch ends after ``max_rows`` rows, about ``max_bytes`` of text or
+    ``max_seconds``, whichever comes first. Each row runs in its own
+    savepoint: a row that cannot be rewritten (``_row_error``) is rolled
+    back alone, stays inline and is listed in ``errors``; anything else rolls
+    the whole batch back and propagates. Returns ``{last_id, rows,
+    protected, errors, done}``: the last row looked at, rows rewritten,
+    inline values kept (not text), the ids kept by an error and whether
+    there was nothing left to look at.
+    """
+    if conn.in_transaction:
+        raise RuntimeError("The migration needs its own transaction")
+    started = _clock()
+    conn.execute("BEGIN IMMEDIATE")
+    try:
+        ids = [row[0] for row in conn.execute(NEXT_IDS_SQL, (after_id, max_rows))]
+        result = {
+            "last_id": after_id,
+            "rows": 0,
+            "protected": 0,
+            "errors": [],
+            "done": not ids,
+        }
+        verified, checked, size = {}, set(), 0
+        for image_id in ids:
+            conn.execute("SAVEPOINT workflow_row")
+            try:
+                row = conn.execute(ROW_SQL, (image_id,)).fetchone()
+                changed, protected, row_size = _rewrite_row(
+                    conn, image_id, row, verified, checked
+                )
+            except Exception as error:
+                if not _row_error(error):
+                    raise
+                conn.execute("ROLLBACK TO workflow_row")
+                conn.execute("RELEASE workflow_row")
+                verified.clear()  # the row's documents were rolled back too
+                checked.clear()
+                logger.warning(f"Image record {image_id} kept inline: {error}")
+                result["errors"].append(image_id)
+                changed, protected, row_size = False, 0, 0
+            else:
+                conn.execute("RELEASE workflow_row")
+            result["rows"] += int(changed)
+            result["protected"] += protected
+            result["last_id"] = image_id
+            size += row_size
+            if size >= max_bytes or _clock() - started >= max_seconds:
+                break
+        conn.commit()
+        return result
+    except BaseException:
+        conn.rollback()
+        raise
+
+
+def pre_migration_backups(db_path):
+    """The kept pre-migration backups beside ``db_path``, earliest first."""
+    db_path = os.path.abspath(db_path)
+    prefix = os.path.basename(db_path) + "." + PRE_MIGRATION_LABEL + "_"
+    folder = os.path.dirname(db_path)
+    try:
+        names = os.listdir(folder)
+    except OSError:
+        return []
+    return sorted(
+        name
+        for name in names
+        if name.startswith(prefix) and os.path.isfile(os.path.join(folder, name))
+    )
+
+
+def _file_bytes(db_path):
+    return sum(
+        os.path.getsize(path)
+        for path in (db_path, db_path + "-wal")
+        if os.path.exists(path)
+    )
+
+
+class WorkflowStorageOperations:
+    """PromptDatabase mixin: the workflow storage migration and Reclaim space."""
+
+    def migrate_workflow_storage(
+        self,
+        *,
+        on_batch=None,
+        should_stop=None,
+        max_rows=BATCH_ROWS,
+        max_bytes=BATCH_BYTES,
+    ) -> dict:
+        """Rewrite every old-form row, batch by batch (blocking).
+
+        ``on_batch(progress)`` runs after each committed batch with the
+        running totals ``{rows, protected, batches, last_id}``;
+        ``should_stop()`` is asked before each batch. Each committed batch is
+        followed by ``BATCH_PAUSE`` with the write lock free. Returns
+        ``{rows, protected, batches, stopped, errors, documents_removed}``
+        (``errors``: the ids of rows kept inline by an error). Unreferenced
+        documents are swept once every row was looked at.
+        """
+        totals = {"rows": 0, "protected": 0, "batches": 0, "last_id": 0}
+        errors = []
+        stopped = False
+        while True:
+            if should_stop is not None and should_stop():
+                stopped = True
+                break
+            batch = self.migrate_workflow_batch(totals["last_id"], max_rows, max_bytes)
+            if batch["done"]:
+                break
+            time.sleep(BATCH_PAUSE)  # leave the write lock to other writers
+            errors += batch["errors"]
+            totals["rows"] += batch["rows"]
+            totals["protected"] += batch["protected"]
+            totals["batches"] += 1
+            totals["last_id"] = batch["last_id"]
+            if on_batch is not None:
+                on_batch(dict(totals))
+        removed = 0 if stopped else self.sweep_workflow_documents()
+        return {
+            "rows": totals["rows"],
+            "protected": totals["protected"],
+            "batches": totals["batches"],
+            "stopped": stopped,
+            "errors": errors,
+            "documents_removed": removed,
+        }
+
+    def migrate_workflow_batch(
+        self, after_id, max_rows=BATCH_ROWS, max_bytes=BATCH_BYTES
+    ) -> dict:
+        """One batch on the calling thread's connection (``migrate_batch``)."""
+        return migrate_batch(self.model.get_connection(), after_id, max_rows, max_bytes)
+
+    def sweep_workflow_documents(self) -> int:
+        """Remove every document no image row references (one transaction)."""
+        conn = self.model.get_connection()
+        with conn:
+            return sweep_documents(conn)
+
+    def workflow_storage_status(self) -> dict:
+        """``workflow_storage_counts`` on one read-only snapshot (blocking)."""
+        from .maintenance import _read_snapshot
+
+        return _read_snapshot(self.model.db_path, self.workflow_storage_counts)
+
+    def workflow_storage_counts(self, conn) -> dict:
+        """``{rows, protected, documents, inline_bytes}`` on ``conn`` (reads only).
+
+        ``inline_bytes`` is None where SQLite has no ``octet_length``.
+        """
+        rows, protected = conn.execute(INLINE_COUNTS_SQL).fetchone()
+        documents = conn.execute("SELECT COUNT(*) FROM workflow_documents").fetchone()
+        inline_bytes = None
+        if sqlite3.sqlite_version_info >= (3, 43, 0):
+            row = conn.execute(INLINE_BYTES_SQL).fetchone()
+            inline_bytes = sum(value or 0 for value in row)
+        return {
+            "rows": rows or 0,
+            "protected": protected or 0,
+            "documents": documents[0],
+            "inline_bytes": inline_bytes,
+        }
+
+    def reclaim_space(self) -> dict:
+        """Sweep, VACUUM and truncate the WAL (blocking); file bytes before/after.
+
+        The caller holds the registry's ``exclusive`` slot. Raises whatever
+        stopped VACUUM; the database is then unchanged.
+        """
+        db_path = self.model.db_path
+        removed = self.sweep_workflow_documents()
+        before = _file_bytes(db_path)
+        self.model.vacuum_database()
+        with closing(sqlite3.connect(db_path)) as checkpoint:
+            busy = checkpoint.execute("PRAGMA wal_checkpoint(TRUNCATE)").fetchone()[0]
+        return {
+            "before_bytes": before,
+            "after_bytes": _file_bytes(db_path),
+            "documents_removed": removed,
+            "checkpoint_busy": bool(busy),
+        }
+
+    # ── preview ───────────────────────────────────────────────────────
+
+    def _plan_workflow_storage(self):
+        """The reviewed plan of the migration (one read-only snapshot)."""
+        from .maintenance import (
+            MaintenanceDraft,
+            _Fingerprint,
+            _megabytes,
+            _plural,
+            _read_snapshot,
+        )
+
+        def read(conn):
+            sizes = [
+                conn.execute(f"PRAGMA {name}").fetchone()[0]
+                for name in ("page_size", "page_count", "freelist_count")
+            ]
+            return sizes, self.workflow_storage_counts(conn)
+
+        (page_size, page_count, free_pages), counts = _read_snapshot(
+            self.model.db_path, read
+        )
+        size = page_size * page_count
+        details = [
+            {
+                "label": "Database file",
+                "detail": f"{_megabytes(size)}, of which "
+                f"{_megabytes(free_pages * page_size)} is free pages",
+            },
+            {
+                "label": "Image records",
+                "detail": f"{_plural(counts['rows'], 'record')} keep their workflow "
+                "or prompt graph inline; each distinct text is stored once, "
+                "compressed",
+            },
+        ]
+        if not counts["rows"]:
+            details[1]["detail"] = NOTHING_TO_DO
+        if counts["inline_bytes"] is not None:
+            details.append(
+                {
+                    "label": "Inline text",
+                    "detail": f"{_megabytes(counts['inline_bytes'])} now",
+                }
+            )
+        if counts["documents"]:
+            details.append(
+                {
+                    "label": "Already stored once",
+                    "detail": _plural(counts["documents"], "workflow document"),
+                }
+            )
+        name = os.path.basename(self.model.db_path)
+        kept = pre_migration_backups(self.model.db_path)
+        backup = (
+            POOLED_BACKUP_NOTE.format(
+                name=f"{name}.backup_<date and time>", kept=kept[0]
+            )
+            if kept
+            else BACKUP_NOTE.format(
+                name=f"{name}.{PRE_MIGRATION_LABEL}_<date and time>"
+            )
+        )
+        details += [
+            {"label": "Backup", "detail": backup},
+            {"label": "Reclaim space", "detail": RECLAIM_NOTE},
+            {"label": "Older versions", "detail": OLDER_VERSIONS_NOTE},
+        ]
+        protected = counts["protected"]
+        return MaintenanceDraft(
+            operation=OPERATION,
+            fingerprint=_Fingerprint(OPERATION).hexdigest(),
+            targets=[],
+            scope={
+                "count": counts["rows"],
+                "label": (
+                    _plural(counts["rows"], "image record") + " to rewrite"
+                    if counts["rows"]
+                    else "Nothing to rewrite"
+                ),
+            },
+            protected={
+                "count": protected,
+                "reasons": (
+                    [f"{_plural(protected, 'stored value')} not text; kept as is"]
+                    if protected
+                    else []
+                ),
+            },
+            media={"deletes_files": False, "label": MEDIA_NOTE},
+            details=details,
+        )
+
+
+MEDIA_NOTE = "No image or video file is opened or changed; only the database."
+RECLAIM_NOTE = (
+    "When every record is rewritten, Reclaim space runs alone to give the freed "
+    "space back to the disk (VACUUM). The database is locked while it runs."
+)
+BACKUP_NOTE = (
+    "Always taken first, whatever the safety backup setting: {name} beside "
+    "the database, verified, and kept until you remove it (it is never pruned)."
+)
+POOLED_BACKUP_NOTE = (
+    "A safety backup is taken first, whatever the safety backup setting: {name} "
+    "among the pooled backups; the pre-migration backup {kept} is already kept."
+)
+NOTHING_TO_DO = "Every image record already stores its workflow once."
+OLDER_VERSIONS_NOTE = (
+    "This version stores the workflows of images it records in the new form, "
+    "whether or not this job runs, and this job rewrites the rest. An older "
+    "Prompt Manager version shows those images without their workflow until you "
+    "update again. The way back to an older version is the earliest "
+    "pre-workflow-storage backup."
+)

py/api/__init__.py

@@ -8,10 +8,15 @@ Split into domain-specific mixins for maintainability:
 - AutotagRoutesMixin: auto-tagging model management and tagging
 - JobRoutesMixin: the session job registry behind Activity (py/api/jobs.py)
 - ImageTagJobsMixin: bulk Add tags / Auto-tag as image-tags jobs
+- ThumbnailJobsMixin: thumbnail plan, thumbnail job and the gallery stream on it
+- AutoThumbnailMixin: previews made on first view, on capture and after a scan
 - ReviewRoutesMixin: the Needs review queue, item detail and decisions
 - ReviewRecordingMixin: Needs review recording for the scan and the monitor
 - ReviewEventsMixin: the Needs review change stream an open page re-reads on
 - SystemRoutesMixin: System health check, Overview and usage statistics
+- MaintenanceRoutesMixin: reviewed maintenance plans (preview, apply)
+- WorkflowStorageRoutesMixin: store each workflow once, then Reclaim space
+- DuplicateFilesMixin: duplicate image files as find and removal jobs
 """
 
 import asyncio
@@ -43,7 +48,13 @@ from .admin import AdminRoutesMixin
 from .settings_routes import SettingsRoutesMixin
 from .job_routes import JobRoutesMixin
 from .image_tag_jobs import ImageTagJobsMixin
+from .thumbnail_jobs import ThumbnailJobsMixin
+from .thumbnail_auto import AutoThumbnailMixin
 from .system import SystemRoutesMixin
+from .maintenance import MaintenanceRoutesMixin
+from .workflow_storage import WorkflowStorageRoutesMixin
+from .duplicate_files import DuplicateFilesMixin
+from .maintenance_plans import PlanStore
 from .jobs import JobRegistry
 from .logging_routes import LoggingRoutesMixin
 from .autotag_routes import AutotagRoutesMixin
@@ -281,7 +292,12 @@ class PromptManagerAPI(
     SettingsRoutesMixin,
     JobRoutesMixin,
     ImageTagJobsMixin,
+    ThumbnailJobsMixin,
+    AutoThumbnailMixin,
     SystemRoutesMixin,
+    MaintenanceRoutesMixin,
+    WorkflowStorageRoutesMixin,
+    DuplicateFilesMixin,
     LoggingRoutesMixin,
     AutotagRoutesMixin,
     LoraIntegrationMixin,
@@ -338,13 +354,27 @@ class PromptManagerAPI(
         self._jobs = JobRegistry(self.logger)
         # The latest metadata scan job (also in self._jobs), if one was started.
         self._scan_job = None
-        # The current duplicate scan (a py/api/jobs.Job outside the registry).
-        self._duplicate_job = None
+        # Duplicate-file finds by job id, with the groups each one holds for
+        # review (py/api/duplicate_files.py); the last few only.
+        self._duplicate_finds = {}
         # The last System health report this session and the database
         # generation it describes, bumped when a restore replaces self.db
         # (py/api/system.py).
         self._health_report = None
         self._db_generation = 0
+        # Reviewed maintenance plans under single-use tokens; a restart
+        # forgets them (py/api/maintenance.py).
+        self._maintenance_plans = PlanStore()
+
+        # A safety backup interrupted by a crash leaves its temp copy behind.
+        try:
+            from .maintenance import sweep_safety_temps
+
+            swept = sweep_safety_temps(self.db.model.db_path)
+            if swept:
+                self.logger.info(f"Removed {len(swept)} interrupted backup copies")
+        except Exception as e:
+            self.logger.warning(f"Could not sweep interrupted backup copies: {e}")
 
         # Run cleanup on initialization to remove any existing duplicates
         try:
@@ -505,6 +535,8 @@ class PromptManagerAPI(
         self._register_settings_routes(routes)
         self._register_job_routes(routes)
         self._register_system_routes(routes)
+        self._register_maintenance_routes(routes)
+        self._register_duplicate_routes(routes)
         self._register_logging_routes(routes)
         self._register_autotag_routes(routes)
         self._register_lora_routes(routes)

py/api/admin.py

@@ -24,9 +24,9 @@ from .jobs import (
     SCAN_GROUPS,
     SCAN_SCOPES,
     DUPLICATE_CLEANUP_JOB_KIND,
+    DUPLICATES_JOB_KIND,
     MAINTENANCE_JOB_KIND,
     RESTORE_JOB_KIND,
-    Job,
     JobBusy,
 )
 
@@ -69,7 +69,9 @@ except ImportError:
 IMAGE_EXTENSIONS = (".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff")
 VIDEO_EXTENSIONS = (".mp4", ".avi", ".mov", ".mkv", ".webm", ".m4v", ".wmv")
 SCAN_BATCH_SIZE = 50  # Files per executor call during the output scan
-MAX_SAFETY_BACKUPS = 10  # '<db>.backup_*' files kept next to the live database
+# '<db>.backup_*' files kept next to the live database: one pool for the
+# restore backups and the maintenance safety backups (py/api/maintenance.py).
+MAX_SAFETY_BACKUPS = 10
 MAX_SCAN_FILES = 50_000  # Upper bound on media files one scan will look at
 SCAN_COUNT_KEYS = (
     "processed",
@@ -90,6 +92,7 @@ MAX_SCAN_DEPTH = 12  # Directory levels below an output root that are scanned
 # the common single-f TIFF spelling. Anything else inside the output
 # directory (databases, scripts, notes) is never a "duplicate image".
 DELETABLE_MEDIA_EXTENSIONS = frozenset(IMAGE_EXTENSIONS + VIDEO_EXTENSIONS + (".tif",))
+DUPLICATE_FIND_LEGACY_TITLE = "Find duplicate image files (gallery page)"
 
 
 def _collect_media_files(output_dirs, extensions):
@@ -132,11 +135,52 @@ def _collect_media_files(output_dirs, extensions):
     return found
 
 
+def _physical_file_key(path, stat):
+    """What tells two physical files apart: ``(st_dev, st_ino)``.
+
+    A filesystem that reports no inode (``st_ino`` 0, some network or FAT
+    volumes on Windows) falls back to the resolved, case-folded path.
+    """
+    if stat.st_ino:
+        return ("inode", stat.st_dev, stat.st_ino)
+    return ("path", os.path.normcase(os.path.realpath(str(path))))
+
+
 def _thumbnail_rel_path(rel_path, thumbnail_ext):
     """Relative path of the thumbnail the gallery would generate for ``rel_path``."""
     return f"thumbnails/{rel_path.with_suffix('').as_posix()}_thumb{thumbnail_ext}"
 
 
+def _media_thumbnail_rel_path(rel_path):
+    """The thumbnail of a media file: a video's preview is a ``.jpg``."""
+    extension = rel_path.suffix.lower()
+    return _thumbnail_rel_path(
+        rel_path, ".jpg" if extension in VIDEO_EXTENSIONS else rel_path.suffix
+    )
+
+
+def _remove_thumbnail(output_path, rel_path, logger=None):
+    """Remove the thumbnail of ``rel_path`` under ``output_path``, if any.
+
+    Shared by the legacy duplicate delete and the reviewed removal job. A
+    thumbnail that is a link, or one that cannot be removed, is left alone
+    (a stale preview never fails a removal). Returns whether one was removed.
+    """
+    output_path = Path(output_path)
+    thumbnail_path = output_path / _media_thumbnail_rel_path(Path(rel_path))
+    try:
+        if thumbnail_path.is_symlink() or not thumbnail_path.is_file():
+            return False
+        os.remove(thumbnail_path)
+    except OSError as e:
+        if logger is not None:
+            logger.warning(f"Could not delete thumbnail for {rel_path}: {e}")
+        return False
+    if logger is not None:
+        logger.debug(f"Deleted associated thumbnail: {thumbnail_path}")
+    return True
+
+
 async def _read_json(request):
     """Parse a JSON object body through the package's capped reader.
 
@@ -561,12 +605,21 @@ class AdminRoutesMixin:
     LONG_JOB_DUPLICATES = "duplicates"
 
     async def scan_duplicates_endpoint(self, request):
-        """Scan for duplicate images without removing them (single flight)."""
-        job = self._acquire_long_job(self.LONG_JOB_DUPLICATES)
-        if job is None:
-            return self._long_job_busy_response("Duplicate scan")
-        async with job:
-            return await self._scan_duplicates(request)
+        """Scan for duplicate images without removing them.
+
+        Holds the ``duplicates`` kind on the registry (a ``read`` hold, so it
+        is listed in Activity), so it is single flight with every duplicate
+        find, and it is refused while a duplicate removal runs.
+        """
+        cleanup = self._jobs.active(DUPLICATE_CLEANUP_JOB_KIND)
+        if cleanup is not None:
+            return self._long_job_busy_response(cleanup.title)
+        return await self._run_held(
+            DUPLICATES_JOB_KIND,
+            DUPLICATE_FIND_LEGACY_TITLE,
+            "read",
+            functools.partial(self._scan_duplicates, request),
+        )
 
     async def _scan_duplicates(self, request):
         try:
@@ -588,24 +641,29 @@ class AdminRoutesMixin:
             )
 
     async def scan_duplicates_stream(self, request):
-        """Run the duplicate scan as a background job and stream its progress.
+        """Stream a duplicate find to the legacy gallery (its event shapes).
 
-        A request while a scan is running attaches to it. The job holds the
-        duplicates lock, so a delete is refused (409) until it finishes.
+        Starts the registry's ``duplicates`` job over the gallery's output
+        folder, or attaches to the running one when it covers that folder
+        (one started in the workspace, for instance). A held legacy scan, a
+        find that does not cover the folder or a running removal answers the
+        legacy 409. The job outlives the request.
         """
-        job = self._duplicate_job
-        if job is None or job.finished:
-            guard = self._acquire_long_job(self.LONG_JOB_DUPLICATES)
-            if guard is None:
-                return self._long_job_busy_response("Duplicate scan or delete")
-            job = Job("duplicates", "Duplicate scan")
-            self._duplicate_job = job
-            job.task = asyncio.ensure_future(self._run_duplicate_job(job, guard))
+        job = self._jobs.active(DUPLICATES_JOB_KIND)
+        if job is None:
+            try:
+                job = self._start_duplicate_find(origin="gallery")
+            except JobBusy as busy:
+                return self._long_job_busy_response(busy.running.title)
+        else:
+            legacy_dir = await self._run_in_executor(self._find_comfyui_output_dir)
+            if not self._find_covers(job, legacy_dir):
+                return self._long_job_busy_response(job.title)
         return await self._stream_scan_job(request, job)
 
     async def scan_duplicates_status(self, request):
-        """Whether a duplicate scan is running, and its latest event."""
-        job = self._duplicate_job
+        """Whether a duplicate find is running, and its latest legacy event."""
+        job = self._latest_duplicate_find()
         return web.json_response(
             {
                 "success": True,
@@ -614,62 +672,6 @@ class AdminRoutesMixin:
             }
         )
 
-    async def _run_duplicate_job(self, job, guard):
-        loop = asyncio.get_running_loop()
-        counts = {"processed": 0}
-
-        def report(done, total):
-            # Called from worker threads; hand the event to the loop.
-            counts["processed"] = done
-            pct = int(done / total * 100) if total else 100
-            loop.call_soon_threadsafe(
-                job.publish,
-                {
-                    "type": "progress",
-                    "progress": pct,
-                    "status": f"Hashing file {done}/{total}...",
-                    "processed": done,
-                    "found": 0,
-                },
-            )
-
-        try:
-            async with guard:
-                job.publish(
-                    {
-                        "type": "progress",
-                        "progress": 0,
-                        "status": "Collecting media files...",
-                        "processed": 0,
-                        "found": 0,
-                    }
-                )
-                output_dir = await self._run_in_executor(self._find_comfyui_output_dir)
-                groups = await self._run_in_executor(
-                    self._find_duplicate_images_sync, output_dir, report
-                )
-                job.publish(
-                    {
-                        "type": "complete",
-                        "processed": counts["processed"],
-                        "found": len(groups),
-                        "duplicates": groups,
-                        "message": f"Found {len(groups)} groups of duplicate images",
-                    }
-                )
-        except Exception:
-            self.logger.exception("Duplicate scan error")
-            job.publish(
-                {
-                    "type": "error",
-                    "message": (
-                        "An internal error occurred. Check server logs for details."
-                    ),
-                }
-            )
-        finally:
-            job.finish()
-
     async def cleanup_duplicates_endpoint(self, request):
         """Cleanup duplicate prompts endpoint (holds the registry: records)."""
         return await self._run_held(
@@ -706,31 +708,109 @@ class AdminRoutesMixin:
     def _find_duplicate_images_sync(self, output_dir, progress=None):
         """Group media files under ``output_dir`` by content hash (blocking).
 
-        Files are hashed on ``PromptManagerConfig.WORKER_THREADS`` threads.
-        ``progress(done, total)`` is called from those threads about every
-        percent and once at the end.
+        The legacy response shape over ``_hash_media_groups``, which the
+        workspace's duplicate find shares.
         """
-        from ..config import PromptManagerConfig
-
         if not output_dir or not os.path.isdir(output_dir):
             self.logger.warning("ComfyUI output directory not found")
             return []
+        groups = self._hash_media_groups([os.path.abspath(output_dir)], progress)
+        duplicates = self._legacy_duplicate_groups(groups, output_dir)
+        self.logger.info(f"Found {len(duplicates)} groups of duplicate images")
+        return duplicates
+
+    def _legacy_duplicate_groups(self, groups, output_dir):
+        """``_hash_media_groups`` groups as the legacy gallery reads them.
+
+        Only the files under ``output_dir`` count (paths are relative to it);
+        a group needs two of them. A file that vanished since it was hashed
+        is left out.
+        """
+        output_path = Path(os.path.abspath(output_dir))
+        base = os.path.normcase(str(output_path)).rstrip("\\/") + os.sep
+        duplicates = []
+        for group in groups:
+            images = []
+            for record in group["files"]:
+                if not os.path.normcase(record["abs"]).startswith(base):
+                    continue
+                try:
+                    images.append(
+                        self._describe_media_file(
+                            Path(record["abs"]), output_path, file_hash=group["hash"]
+                        )
+                    )
+                except (OSError, ValueError) as e:
+                    self.logger.error(
+                        f"Error describing {Path(record['abs']).name}: {e}"
+                    )
+            if len(images) > 1:
+                duplicates.append(
+                    {"hash": group["hash"], "images": images, "count": len(images)}
+                )
+        return duplicates
+
+    def _hash_media_groups(self, roots, progress=None, stop=None):
+        """Content-identical media files under ``roots`` (blocking).
+
+        Each root is walked as ``_collect_media_files`` walks (no symlinks,
+        no ``thumbnails``, bounded). A physical file is listed once, under
+        the first root that holds it: files are told apart by ``(st_dev,
+        st_ino)``, so a source listed twice under two spellings (a symlink, a
+        bind mount, a case variant), a source inside another one or a
+        hardlink is one file, never a copy of itself. At most MAX_SCAN_FILES
+        files are read. Files
+        are hashed on ``PromptManagerConfig.WORKER_THREADS`` threads;
+        ``progress(done, total)`` is called from those threads about every
+        percent and once at the end, and ``stop()`` true skips the files not
+        yet read. Returns ``[{hash, size, files:[{slot, abs, rel, size,
+        mtime, mtime_ns, inode}]}]`` for every hash shared by two or more
+        files (``inode`` is ``[st_dev, st_ino]``), ``slot``
+        being the root's place in ``roots`` and the files oldest first.
+        """
+        from ..config import PromptManagerConfig
 
-        output_path = Path(output_dir)
         extensions = IMAGE_EXTENSIONS + VIDEO_EXTENSIONS
-        media_files = _collect_media_files([output_path], extensions)
+        media_files = []
+        seen = set()
+        for slot, root in enumerate(roots):
+            for path in _collect_media_files([root], extensions):
+                try:
+                    # os.stat, not Path.stat: one stat call for every check
+                    # (Python 3.10's Path.stat bypasses a patched os.stat).
+                    stat = os.stat(path)
+                except OSError:
+                    continue
+                key = _physical_file_key(path, stat)
+                if key not in seen:
+                    seen.add(key)
+                    media_files.append((slot, root, path, stat))
+                if len(media_files) >= MAX_SCAN_FILES:
+                    break
+        media_files = media_files[:MAX_SCAN_FILES]
         total = len(media_files)
         self.logger.info(f"Found {total} media files to analyze")
         step = max(1, total // 100)
         counter = {"done": 0}
         lock = threading.Lock()
 
-        def describe(media_path):
-            try:
-                info = self._describe_media_file(media_path, output_path)
-            except Exception as e:
-                self.logger.error(f"Error processing file {media_path.name}: {e}")
-                info = None
+        def describe(item):
+            slot, root, media_path, stat = item
+            info = None
+            if stop is None or not stop():
+                try:
+                    info = {
+                        "slot": slot,
+                        "abs": str(media_path),
+                        "rel": Path(os.path.relpath(str(media_path), root)).as_posix(),
+                        "size": stat.st_size,
+                        "mtime": stat.st_mtime,
+                        "mtime_ns": stat.st_mtime_ns,
+                        "inode": [stat.st_dev, stat.st_ino],
+                        "hash": self._calculate_file_hash(media_path),
+                    }
+                except Exception as e:
+                    self.logger.error(f"Error processing file {media_path.name}: {e}")
             with lock:
                 counter["done"] += 1
                 done = counter["done"]
@@ -744,24 +824,22 @@ class AdminRoutesMixin:
 
         infos = map_parallel(describe, media_files, PromptManagerConfig.WORKER_THREADS)
 
-        file_hashes = {}
+        by_hash = {}
         for info in infos:
             if info is not None:
-                file_hashes.setdefault(info["hash"], []).append(info)
-
-        duplicates = []
-        for file_hash, images in file_hashes.items():
-            if len(images) > 1:
-                images.sort(key=lambda x: x["modified_time"])
-                duplicates.append(
-                    {"hash": file_hash, "images": images, "count": len(images)}
+                by_hash.setdefault(info.pop("hash"), []).append(info)
+        groups = []
+        for file_hash, files in by_hash.items():
+            if len(files) > 1:
+                files.sort(key=lambda record: (record["mtime"], record["abs"]))
+                groups.append(
+                    {"hash": file_hash, "size": files[0]["size"], "files": files}
                 )
+        return groups
 
-        self.logger.info(f"Found {len(duplicates)} groups of duplicate images")
-        return duplicates
-
-    def _describe_media_file(self, media_path, output_path):
-        """Hash one media file and describe it for the duplicates response."""
+    def _describe_media_file(self, media_path, output_path, file_hash=None):
+        """Describe one media file for the duplicates response (hashing it
+        unless ``file_hash`` is given)."""
         stat = media_path.stat()
         rel_path = media_path.relative_to(output_path)
         extension = media_path.suffix.lower()
@@ -782,7 +860,7 @@ class AdminRoutesMixin:
             "extension": extension,
             "media_type": "video" if is_video else "image",
             "is_video": is_video,
-            "hash": self._calculate_file_hash(media_path),
+            "hash": file_hash or self._calculate_file_hash(media_path),
         }
 
     def _calculate_file_hash(self, file_path):
@@ -796,9 +874,9 @@ class AdminRoutesMixin:
     async def delete_duplicate_images_endpoint(self, request):
         """Delete duplicate image files from disk.
 
-        Single flight with the duplicate scan, and a records hold on the
-        registry, so no scan, restore or tagging job runs meanwhile. A body
-        without paths is refused before the hold (it is no job).
+        Refused while a duplicate find runs, and a records hold on the
+        registry, so no scan, restore, removal or tagging job runs meanwhile.
+        A body without paths is refused before the hold (it is no job).
         """
         data, error_response = await _read_json(request)
         if error_response is not None:
@@ -809,20 +887,14 @@ class AdminRoutesMixin:
                 {"success": False, "error": "No image paths provided"},
                 status=400,
             )
-        guard = self._acquire_long_job(self.LONG_JOB_DUPLICATES)
-        if guard is None:
-            return self._long_job_busy_response("Duplicate scan or delete")
-
-        async def delete():
-            # No await since the check above: the lock is still free.
-            async with guard:
-                return await self._delete_duplicate_images(image_paths)
-
+        find = self._jobs.active(DUPLICATES_JOB_KIND)
+        if find is not None:
+            return self._long_job_busy_response(find.title)
         return await self._run_held(
             DUPLICATE_CLEANUP_JOB_KIND,
             "Duplicate image delete (gallery page)",
             "records",
-            delete,
+            functools.partial(self._delete_duplicate_images, image_paths),
         )
 
     async def _delete_duplicate_images(self, image_paths):
@@ -902,14 +974,7 @@ class AdminRoutesMixin:
 
         os.remove(file_path)
         self.logger.info(f"Deleted duplicate image: {image_path}")
-
-        thumbnail_path = output_path / _thumbnail_rel_path(rel_path, file_path.suffix)
-        try:
-            if thumbnail_path.exists():
-                os.remove(thumbnail_path)
-                self.logger.debug(f"Deleted associated thumbnail: {thumbnail_path}")
-        except OSError as e:
-            self.logger.warning(f"Could not delete thumbnail for {image_path}: {e}")
+        _remove_thumbnail(output_path.resolve(), rel_path, self.logger)
         return None
 
     async def get_statistics(self, request):
@@ -1112,8 +1177,10 @@ class AdminRoutesMixin:
                 db.cleanup_duplicates(), "duplicate prompts"
             ),
             "vacuum": vacuum,
+            # Only under sources that can be read now (never an unmounted drive).
             "cleanup_orphaned_images": lambda: count_result(
-                db.cleanup_missing_images(), "orphaned image records"
+                db.cleanup_missing_images(self._health_roots()),
+                "orphaned image records",
             ),
             "check_hash_duplicates": lambda: self._hash_duplicates_result(
                 db.check_hash_duplicates()
@@ -1207,7 +1274,11 @@ class AdminRoutesMixin:
     async def restore_database(self, request):
         """Restore the prompts database from an uploaded SQLite file.
 
-        The upload is streamed to a temp file (capped at restore_max_bytes),
+        The upload is streamed to a temp file. Free disk space decides how
+        large it may be (``maintenance.restore_room``), under a 64 GiB ceiling
+        (``restore_max_bytes``): a request whose Content-Length does not fit
+        is refused before anything is written, and one without is refused as
+        soon as its running size does not. The file is
         verified with PromptModel.verify_database_file and only then copied
         over the live database, which is backed up first. The whole request
         holds the registry (records): it is refused while a scan or another
@@ -1220,10 +1291,44 @@ class AdminRoutesMixin:
             lambda: self._restore_database(request),
         )
 
+    def _restore_refusal(self, size, ceiling, room, *, known):
+        """``400 too-large`` past the ceiling, else ``507 insufficient-space``."""
+        from .maintenance import restore_space_message
+
+        if size > ceiling:
+            return web.json_response(
+                {
+                    "success": False,
+                    "code": "too-large",
+                    "error": (
+                        "File too large. Maximum size is "
+                        f"{ceiling // (1024 * 1024)}MB"
+                    ),
+                },
+                status=400,
+            )
+        return web.json_response(
+            {
+                "success": False,
+                "code": "insufficient-space",
+                "error": restore_space_message(size, room, known=known),
+            },
+            status=507,
+        )
+
     async def _restore_database(self, request):
-        max_bytes = getattr(self, "restore_max_bytes", 100 * 1024 * 1024)
+        from .maintenance import RESTORE_MAX_BYTES, restore_room
+
+        ceiling = getattr(self, "restore_max_bytes", RESTORE_MAX_BYTES)
         temp_path = None
         try:
+            # Free disk space decides how large a backup may be (#313): the
+            # upload, its staged copy and the swap must all fit.
+            room = await self._run_in_executor(restore_room, self.db.model.db_path)
+            max_bytes = min(ceiling, room)
+            length = request.content_length
+            if length is not None and length > max_bytes:
+                return self._restore_refusal(length, ceiling, room, known=True)
             reader = await request.multipart()
             field = await reader.next()
 
@@ -1249,16 +1354,7 @@ class AdminRoutesMixin:
                         break
                     size += len(chunk)
                     if size > max_bytes:
-                        return web.json_response(
-                            {
-                                "success": False,
-                                "error": (
-                                    "File too large. Maximum size is "
-                                    f"{max_bytes // (1024 * 1024)}MB"
-                                ),
-                            },
-                            status=400,
-                        )
+                        return self._restore_refusal(size, ceiling, room, known=False)
                     out.write(chunk)
 
             if size == 0:
@@ -1408,6 +1504,9 @@ class AdminRoutesMixin:
             verb = "cancelled" if state == "cancelled" else "finished"
             job.log_line(f"Scan {verb} · {_plural(job.processed, 'file')} processed")
         job.finish(state)
+        if state == "finished":
+            # After the scan, so the guarded thumbnail job is admitted (#249).
+            await self._hand_off_scan_thumbnails(job, output_dirs)
 
     async def _stream_scan_job(self, request, job):
         """Stream ``job`` as SSE; stop quietly if the client goes away."""
@@ -1646,7 +1745,7 @@ class AdminRoutesMixin:
             if self._scan_job and self._scan_job.cancelled:
                 break
             counts["processed"] += 1
-            ambiguous, untraced = [], []
+            ambiguous, untraced, records = [], [], {}
             try:
                 outcome = self._ingest_scanned_file(
                     media_file,
@@ -1654,6 +1753,7 @@ class AdminRoutesMixin:
                     detailed=True,
                     ambiguous=ambiguous,
                     untraced=untraced,
+                    records=records,
                 )
             except Exception as e:
                 self.logger.error(f"Error processing {media_file.name}: {e}")
@@ -1666,7 +1766,11 @@ class AdminRoutesMixin:
                 else:
                     self._record_scanned_review(media_file, roots, review)
             elif untraced and outcome == "unresolved":
-                self._settle_untraced_reviews(untraced, roots, review)
+                # Untraced by the default profile: the inspection profile may
+                # still offer a choice (queued for review, never linked).
+                outcome = self._untraced_scan_outcome(
+                    media_file, roots, records.get("canonical"), review
+                )
             if outcome is None:
                 note(media_file, None)
                 continue
@@ -1701,7 +1805,13 @@ class AdminRoutesMixin:
         return positive.strip() or None
 
     def _ingest_scanned_file(
-        self, media_file, metadata, detailed=False, ambiguous=None, untraced=None
+        self,
+        media_file,
+        metadata,
+        detailed=False,
+        ambiguous=None,
+        untraced=None,
+        records=None,
     ):
         """Enrich exact pairs; retain legacy positive-only import for absent pairs.
 
@@ -1714,7 +1824,9 @@ class AdminRoutesMixin:
         ``"ambiguous"`` (nothing linked it) is a Needs review case. An
         ambiguous result with nothing to choose (``reviewable`` is False: fewer
         than two candidates carry a prompt) that nothing links returns
-        ``"unresolved"`` instead, and ``media_file`` goes to ``untraced``.
+        ``"unresolved"`` instead, and ``media_file`` goes to ``untraced``;
+        its parsed records go to ``records["canonical"]`` when ``records`` is
+        a dict.
         """
         if not metadata:
             return None
@@ -1735,6 +1847,8 @@ class AdminRoutesMixin:
         if unchoosable:
             if untraced is not None:
                 untraced.append(media_file)
+            if records is not None:
+                records["canonical"] = canonical
         elif resolution["state"] == "ambiguous" and ambiguous is not None:
             ambiguous.append(media_file)
         if resolution["state"] == "resolved":

py/api/duplicate_files.py

@@ -0,0 +1,779 @@
+"""Duplicate image files on the job registry (#249 System S6, Task 17).
+
+- ``POST /prompt_manager/jobs/duplicates`` starts the find (202 ``{job}``):
+  kind ``duplicates``, "Find duplicate image files", admission ``read``,
+  cancellable, outcome group ``groups_found``. It hashes the media files of
+  every effective source (``_hash_media_groups``, shared with the legacy
+  scan) and the server keeps the groups it found under the job id: content
+  hash, root-relative paths and each file's source index. A find is refused
+  while a duplicate removal runs, and one find runs at a time.
+- ``GET /prompt_manager/maintenance/duplicates/{job_id}?offset&limit`` pages
+  those groups, at most ``MAX_GROUP_PAGE`` at a time: ``{success, job:{id,
+  state}, total, offset, limit, groups:[{group_id, size, count, files}]}``.
+  Each file names its root-relative ``path``, its source's public label and
+  index and the Images serve URLs; never an absolute path.
+- ``POST /prompt_manager/maintenance/preview {operation:"duplicate_files",
+  options:{job_id, keep:[{group_id, root_index, path}]}}`` reviews a removal:
+  in each group named, the files kept are the ones listed and every other
+  copy is removed. Each keeper and each removal is checked against the held
+  group, with containment decided as text before any filesystem access
+  (``file_problem``): inside its source, a media file outside
+  ``thumbnails``, then a regular file (not a link) whose resolved path is
+  still inside the source, with the same size and content hash. A removal
+  that fails, or whose kept copy fails, is protected and listed with the
+  reason. The plan carries the files' database links, which move to the
+  kept copy.
+- ``POST /prompt_manager/maintenance/apply {token}`` runs the reviewed plan
+  as the ``duplicate_cleanup`` job: admission ``records``, not cancellable,
+  groups ``removed``, ``kept`` and ``errors``. It checks the sources are
+  unchanged, takes the safety backup the policy promises (the database only;
+  removed files are not in it), then for each removal rechecks its kept copy
+  and the file itself (``file_problem``) and removes it with every handle
+  closed (``_unlink``), together with its thumbnail. A file that changed is
+  kept; the first unlink that fails stops the job, a locked file with the
+  coded ``LOCKED_FILE`` message (never the OS text), and the files after it
+  are not attempted. The database links of the files actually removed then
+  move to their kept copy (``PromptDatabase.move_file_links``).
+
+The legacy gallery routes (``/scan_duplicates``, its stream and status, and
+``/delete_duplicate_images``) keep their shapes on the same registry kinds;
+see ``py/api/admin.py``.
+"""
+
+import asyncio
+import os
+from pathlib import Path
+
+from aiohttp import web
+
+from .admin import DUPLICATE_FIND_LEGACY_TITLE, _remove_thumbnail
+from .duplicate_review import (  # noqa: F401 (CHANGED, OUTSIDE_SOURCE: public)
+    CHANGED,
+    DELETE_FAILED,
+    FIND_CANCELLED,
+    FIND_ERROR_EVENT,
+    FIND_FAILED,
+    GONE,
+    KEEPER,
+    KEEPER_CHANGED,
+    KEEPER_GONE,
+    KEEPER_WENT,
+    KEPT_COPY,
+    LOCKED_FILE,
+    MAYBE_SAME_FILE,
+    NO_FILE_ID_ROW,
+    NOT_ATTEMPTED,
+    OUTSIDE_SOURCE,
+    RECORDS_FAILED,
+    ROW_DETAILS,
+    SAME_FILE,
+    SOURCES_CHANGED,
+    SOURCES_CHANGED_REVIEW,
+    HeldFind,
+    SourcesChanged,
+    _fingerprint,
+    _group_order,
+    _identity,
+    _parse_options,
+    _plural,
+    _public_plan,
+    _select,
+    file_problem,
+    human_bytes,
+    still_kept,
+)
+from .duplicate_spellings import Spellings, copy_move
+from .images import _favorite_root_key, _output_image_entry
+from .job_routes import busy_response
+from .jobs import (
+    DUPLICATE_CLEANUP_GROUP_NAMES,
+    DUPLICATE_CLEANUP_JOB_KIND,
+    DUPLICATE_GROUP_NAMES,
+    DUPLICATES_JOB_KIND,
+    JobBusy,
+)
+from .maintenance import DUPLICATE_FILES, _fail_job, _failure, _invalid, backup_note
+
+try:
+    from ...database.duplicate_copies import FILE_LINK_SQL
+except ImportError:
+    from database.duplicate_copies import FILE_LINK_SQL
+
+DUPLICATE_FIND_TITLE = "Find duplicate image files"
+DUPLICATE_CLEANUP_TITLE = "Remove duplicate image files"
+MAX_GROUP_PAGE = 10  # groups a page lists (D69: ten at a time)
+MAX_HELD_FINDS = 3  # finds whose groups the server keeps for review
+favorite_root_key = _favorite_root_key
+
+
+def query_plan_sql():
+    """The link reconciliation's statements (EXPLAIN QUERY PLAN checks)."""
+    return list(FILE_LINK_SQL)
+
+
+def _unlink(path):
+    """Remove one file (the caller closed every handle on it)."""
+    os.remove(path)
+
+
+class DuplicateFilesMixin:
+    """Mixin providing the duplicate-file find, its groups and its removal."""
+
+    def _register_duplicate_routes(self, routes):
+        @routes.post("/prompt_manager/jobs/duplicates")
+        async def start_duplicate_find_route(request):
+            return await self.start_duplicate_find(request)
+
+        @routes.get("/prompt_manager/maintenance/duplicates/{job_id}")
+        async def duplicate_groups_route(request):
+            return await self.duplicate_groups(request)
+
+    # ── find ──────────────────────────────────────────────────────────
+
+    async def start_duplicate_find(self, request):
+        """Start the duplicate find (202), or 409 naming the job in the way."""
+        try:
+            job = self._start_duplicate_find(origin="workspace")
+        except JobBusy as busy:
+            return busy_response(busy)
+        return web.json_response({"success": True, "job": job.summary()}, status=202)
+
+    def _start_duplicate_find(self, origin):
+        """Register a find job (on the loop); raises JobBusy.
+
+        A workspace find walks every effective source; a gallery one the
+        gallery's output folder, as the legacy scan did. Both resolve their
+        folders when the job begins, so a source change later cannot move it.
+        """
+        cleanup = self._jobs.active(DUPLICATE_CLEANUP_JOB_KIND)
+        if cleanup is not None:
+            raise JobBusy(cleanup)
+        held = HeldFind(origin)
+        if origin == "workspace":
+            title = DUPLICATE_FIND_TITLE
+        else:
+            title = DUPLICATE_FIND_LEGACY_TITLE
+        job = self._jobs.start(
+            DUPLICATES_JOB_KIND,
+            title,
+            admission="read",
+            cancellable=True,
+            groups=DUPLICATE_GROUP_NAMES,
+            runner=lambda job: self._run_duplicate_find(job, held),
+        )
+        held.job_id = job.id
+        self._remember_find(held)
+        return job
+
+    def _remember_find(self, held):
+        finds = self._duplicate_finds
+        finds[held.job_id] = held
+        for job_id in list(finds):
+            if self._jobs.get(job_id) is None:
+                del finds[job_id]
+        while len(finds) > MAX_HELD_FINDS:
+            del finds[next(iter(finds))]
+
+    def _find_covers(self, job, legacy_dir):
+        held = self._duplicate_finds.get(job.id)
+        return held is not None and held.covers(legacy_dir)
+
+    def _latest_duplicate_find(self):
+        """The running find, else the newest finished one this session."""
+        for job in self._jobs.list():
+            if job.kind == DUPLICATES_JOB_KIND and job.id in self._duplicate_finds:
+                return job
+        return None
+
+    def _resolve_find_roots(self, held):
+        """Fill in where a find looks (blocking): sources and the gallery folder."""
+        listed = [os.path.abspath(str(d)) for d in self._get_all_output_dirs()]
+        held.legacy_dir = self._find_comfyui_output_dir()
+        if held.origin == "workspace":
+            held.roots = list(enumerate(listed))
+            return
+        if not held.legacy_dir or not os.path.isdir(held.legacy_dir):
+            self.logger.warning("ComfyUI output directory not found")
+            held.roots = []
+            return
+        legacy = os.path.abspath(held.legacy_dir)
+        index = next(
+            (i for i, d in enumerate(listed) if _identity(d) == _identity(legacy)),
+            None,
+        )
+        held.roots = [(index, legacy)]
+
+    def _find_progress(self, job, done, total):
+        if done < job.processed:
+            return  # hashing threads report out of order; keep the count rising
+        pct = int(done / total * 100) if total else 100
+        job.update(
+            processed=done,
+            total=total,
+            status_line=f"Hashing file {done:,} of {total:,}",
+        )
+        job.publish(
+            {
+                "type": "progress",
+                "progress": pct,
+                "status": f"Hashing file {done}/{total}...",
+                "processed": done,
+                "found": 0,
+            }
+        )
+
+    async def _run_duplicate_find(self, job, held):
+        loop = asyncio.get_running_loop()
+
+        def report(done, total):  # from the hashing threads
+            loop.call_soon_threadsafe(self._find_progress, job, done, total)
+
+        try:
+            job.update(status_line="Looking for media files")
+            job.publish(
+                {
+                    "type": "progress",
+                    "progress": 0,
+                    "status": "Collecting media files...",
+                    "processed": 0,
+                    "found": 0,
+                }
+            )
+            await self._run_in_executor(self._resolve_find_roots, held)
+            job.scope["output_folders"] = len(held.roots)
+            job.log_line(f"Searching {_plural(len(held.roots), 'source')}")
+            groups = await self._run_in_executor(
+                self._hash_media_groups,
+                [root for _index, root in held.roots],
+                report,
+                lambda: job.cancelled,
+            )
+            if job.cancelled:
+                job.update(status_line="Cancelled; nothing was changed.")
+                job.publish({"type": "error", "message": FIND_CANCELLED})
+                job.finish("cancelled")
+                return
+            await self._finish_find(job, held, groups)
+        except Exception:
+            self.logger.exception("Duplicate search failed")
+            job.update(status_line=FIND_FAILED)
+            job.publish({"type": "error", "message": FIND_ERROR_EVENT})
+            job.finish("failed")
+
+    async def _finish_find(self, job, held, groups):
+        for group in groups:
+            group["group_id"] = group["hash"]
+            for record in group["files"]:
+                index, root = held.roots[record.pop("slot")]
+                record["root_index"] = index
+                record["root"] = root
+        groups.sort(key=_group_order)
+        held.groups = groups
+        labels = {root: self._public_path(root) for _index, root in held.roots}
+        rows = [
+            {
+                "path": group["files"][0]["rel"],
+                "root": labels[group["files"][0]["root"]],
+                "root_index": group["files"][0]["root_index"],
+                "detail": f"{len(group['files'])} identical files · "
+                f"{human_bytes(group['size'])} each",
+            }
+            for group in groups
+        ]
+        job.record_counted("groups_found", len(groups), rows)
+        line = f"Found {_plural(len(groups), 'group')} of duplicate files"
+        job.log_line(line)
+        legacy = []
+        if held.covers(held.legacy_dir) and held.legacy_dir:
+            legacy = await self._run_in_executor(
+                self._legacy_duplicate_groups, groups, held.legacy_dir
+            )
+        # Every file was read by now (the last reports may still be queued).
+        job.update(progress=100, status_line=line, processed=job.total)
+        job.publish(
+            {
+                "type": "complete",
+                "processed": job.processed,
+                "found": len(legacy),
+                "duplicates": legacy,
+                "message": f"Found {len(legacy)} groups of duplicate images",
+            }
+        )
+
+    # ── groups ────────────────────────────────────────────────────────
+
+    def _held_result(self, job_id):
+        """``(held, None)`` for a finished find's groups, else an error reply."""
+        held = self._duplicate_finds.get(job_id)
+        job = self._jobs.get(job_id)
+        if held is None or job is None or job.kind != DUPLICATES_JOB_KIND:
+            return None, _failure(
+                "No duplicate search with this id. Search again.", 404, "not-found"
+            )
+        if job.running:
+            return None, _failure(
+                "The duplicate search is still running.", 409, "job-running"
+            )
+        if held.groups is None:
+            return None, _failure(
+                "This search did not finish. Search again.", 409, "no-groups"
+            )
+        return held, None
+
+    async def duplicate_groups(self, request):
+        """One page of a finished find's groups (at most MAX_GROUP_PAGE)."""
+        try:
+            offset = int(request.query.get("offset", "0"))
+            limit = int(request.query.get("limit", str(MAX_GROUP_PAGE)))
+        except ValueError:
+            return _invalid("offset and limit must be whole numbers")
+        if offset < 0 or not 0 < limit <= MAX_GROUP_PAGE:
+            return _invalid(f"limit must be 1 to {MAX_GROUP_PAGE}; offset 0 or more")
+        job_id = request.match_info["job_id"]
+        held, error = self._held_result(job_id)
+        if error is not None:
+            return error
+        page = held.groups[offset : offset + limit]
+        groups = await self._run_in_executor(self._describe_groups, page)
+        job = self._jobs.get(job_id)
+        return web.json_response(
+            {
+                "success": True,
+                "job": {"id": job.id, "state": job.state},
+                "total": len(held.groups),
+                "offset": offset,
+                "limit": limit,
+                "groups": groups,
+            },
+            headers={"Cache-Control": "no-store"},
+        )
+
+    def _describe_groups(self, groups):
+        """Public group rows (blocking: one stat per file)."""
+        return [
+            {
+                "group_id": group["group_id"],
+                "size": group["size"],
+                "count": len(group["files"]),
+                "files": [self._describe_copy(record) for record in group["files"]],
+            }
+            for group in groups
+        ]
+
+    def _describe_copy(self, record):
+        label = self._public_path(record["root"])
+        base = {
+            "filename": os.path.basename(record["rel"]),
+            "path": record["rel"],
+            "root": label,
+            "root_index": record["root_index"],
+            "url": None,
+            "thumbnail_url": None,
+            "size": record["size"],
+            "modified_time": record["mtime"],
+            "media_type": "image",
+            "missing": False,
+        }
+        if record["root_index"] is None:
+            return base
+        try:
+            entry = _output_image_entry(
+                Path(record["abs"]), Path(record["root"]), record["root_index"], label
+            )
+        except (OSError, ValueError):
+            return {**base, "missing": True}
+        return {
+            **base,
+            "url": entry["url"],
+            "thumbnail_url": entry["thumbnail_url"],
+            "size": entry["size"],
+            "modified_time": entry["modified_time"],
+            "media_type": entry["media_type"],
+        }
+
+    # ── review ────────────────────────────────────────────────────────
+
+    def _listed_sources(self):
+        """``{identity: index}`` of the effective sources now (blocking)."""
+        return {
+            _identity(d): index for index, d in enumerate(self._get_all_output_dirs())
+        }
+
+    @staticmethod
+    def _source_listed(listed, index, root):
+        return index is not None and listed.get(_identity(root)) == index
+
+    async def preview_duplicate_files(self, options):
+        """The ``duplicate_files`` plan for one finished find and its keepers."""
+        job_id, keep, message = _parse_options(options)
+        if message:
+            return _invalid(message)
+        held, error = self._held_result(job_id)
+        if error is not None:
+            return error
+        selection, message = _select(held, keep)
+        if message:
+            return _invalid(message)
+        db = self.db
+        identity = self._database_identity(db)
+        try:
+            review = await self._run_in_executor(
+                self._review_duplicate_files, db, selection
+            )
+        except SourcesChanged:
+            return _failure(SOURCES_CHANGED_REVIEW, 409, "sources-changed")
+        except Exception:
+            self.logger.exception("Duplicate file review failed")
+            return _failure("Failed to review maintenance", 500)
+        return await self._plan_response(
+            db,
+            identity,
+            DUPLICATE_FILES,
+            review["fingerprint"],
+            review["public"],
+            {
+                "targets": review["targets"],
+                "kept": review["kept"],
+                "protected": review["public"]["protected"]["count"],
+            },
+        )
+
+    def _review_duplicate_files(self, db, selection):
+        """Check every keeper and removal now (blocking); the plan's parts.
+
+        No file is read: each copy is bound to what the search saw by its
+        size, mtime and inode, and never to its kept copy itself. The removal
+        job hashes each file again right before it unlinks it.
+        """
+        listed = self._listed_sources()
+        targets, kept, protected = [], [], {}
+        for group, keepers in selection.values():
+            removals = [r for r in group["files"] if r not in keepers]
+            for record in removals:
+                if record["root_index"] is not None and not self._source_listed(
+                    listed, record["root_index"], record["root"]
+                ):
+                    raise SourcesChanged(record["root"])
+            keeper = next(
+                (k for k in keepers if _bound_problem(k, group) is None), None
+            )
+            reviewed = []
+            for record in removals:
+                if keeper is None:
+                    problem = KEEPER
+                elif record["root_index"] is None:
+                    problem = OUTSIDE_SOURCE
+                else:
+                    problem = _bound_problem(record, group, keeper)
+                if problem:
+                    protected[problem] = protected.get(problem, 0) + 1
+                    continue
+                reviewed.append(self._target(group, record, keeper))
+            if reviewed:
+                targets.extend(reviewed)
+                kept.append(self._copy_row(keeper, KEPT_COPY))
+        records = {}
+        if targets:
+            spellings = self._source_spellings()
+            records = db.count_copy_records(
+                {
+                    t["abs"]: {
+                        "spellings": list(spellings.of(t["abs"]).values()),
+                        "locator": t["locator"],
+                        "sha256": t["hash"],
+                    }
+                    for t in targets
+                }
+            )
+        return {
+            "targets": targets,
+            "kept": kept,
+            "fingerprint": _fingerprint(targets),
+            "public": _public_plan(targets, protected, records),
+        }
+
+    def _source_spellings(self):
+        """The folder spellings records may name files by (blocking).
+
+        The search walks each source's real path, while a record names a file
+        as it was seen when written: under the folder the image monitor
+        watches, the configured spelling, ComfyUI's output folder, a symlink
+        to it, or a parent of a source added later. ``Spellings`` matches
+        them to each file by containment (``py/api/duplicate_spellings.py``).
+        """
+        from ..config import GalleryConfig
+
+        listed = [os.path.abspath(str(d)) for d in self._get_all_output_dirs()]
+        candidates = list(self._monitored_directories())
+        candidates += list(GalleryConfig.MONITORING_DIRECTORIES)
+        try:
+            import folder_paths
+
+            candidates.append(folder_paths.get_output_directory())
+        except Exception:  # nosec B110 - outside ComfyUI there is none
+            pass
+        candidates.append(self._find_comfyui_output_dir())
+        folders = [
+            os.path.abspath(str(c)) for c in candidates if c and os.path.isabs(str(c))
+        ]
+        return Spellings(folders + listed, listed)
+
+    def _target(self, group, record, keeper):
+        return {
+            "group_id": group["group_id"],
+            "hash": group["hash"],
+            "size": record["size"],
+            "root_index": record["root_index"],
+            "root": record["root"],
+            "root_label": self._public_path(record["root"]),
+            "abs": record["abs"],
+            "rel": record["rel"],
+            "keeper_abs": keeper["abs"],
+            "keeper_rel": keeper["rel"],
+            "keeper_root": keeper["root"],
+            "keeper_size": keeper["size"],
+            "locator": os.path.normcase(record["abs"]),
+            "keeper_locator": os.path.normcase(keeper["abs"]),
+        }
+
+    def _copy_row(self, record, detail):
+        return {
+            "path": record["rel"],
+            "root": self._public_path(record["root"]),
+            "root_index": record["root_index"],
+            "detail": detail,
+        }
+
+    # ── removal ───────────────────────────────────────────────────────
+
+    def _start_duplicate_cleanup(self, db, plan, backup_on):
+        """Register the removal job for a reviewed plan (on the loop)."""
+        find = self._jobs.active(DUPLICATES_JOB_KIND)
+        if find is not None:
+            raise JobBusy(find)
+        return self._jobs.start(
+            DUPLICATE_CLEANUP_JOB_KIND,
+            DUPLICATE_CLEANUP_TITLE,
+            admission="records",
+            cancellable=False,
+            scope={"operation": DUPLICATE_FILES},
+            groups=DUPLICATE_CLEANUP_GROUP_NAMES,
+            runner=lambda job: self._run_duplicate_cleanup(job, db, plan, backup_on),
+        )
+
+    def _sources_unchanged(self, targets):
+        listed = self._listed_sources()
+        return all(
+            self._source_listed(listed, t["root_index"], t["root"]) for t in targets
+        )
+
+    async def _run_duplicate_cleanup(self, job, db, plan, backup_on):
+        targets = plan.payload["targets"]
+        unchanged = await self._run_in_executor(self._sources_unchanged, targets)
+        if not unchanged:
+            return _fail_job(job, "sources-changed", SOURCES_CHANGED)
+        ok, backup = await self._safety_backup_step(job, db, backup_on)
+        if not ok:
+            return
+        try:
+            await self._remove_duplicates(job, db, plan, backup)
+        finally:
+            if backup:
+                await self._prune_backup_pool(job, db)
+
+    async def _remove_duplicates(self, job, db, plan, backup):
+        targets = plan.payload["targets"]
+        for row in plan.payload["kept"]:
+            job.record("kept", row, f"= {row['root']}/{row['path']} · {KEPT_COPY}")
+        job.update(total=len(targets), status_line="Removing duplicate files")
+        spellings = await self._run_in_executor(self._source_spellings)
+        removed, failure, keepers = 0, None, {}
+        totals = dict.fromkeys(("moved", "merged", "favorites", "tags", "review"), 0)
+        for done, target in enumerate(targets, start=1):
+            if failure is not None:
+                self._record_target(job, "kept", target, NOT_ATTEMPTED)
+                continue
+            outcome, detail, counts = await self._run_in_executor(
+                self._remove_one_duplicate, db, target, keepers, spellings
+            )
+            for key, value in (counts or {}).items():
+                totals[key] += value
+            if outcome == "removed":
+                removed += 1
+                text = f"A copy of {target['keeper_rel']}"
+                self._record_target(job, "removed", target, text, "-")
+            elif outcome == "kept":
+                self._record_target(job, "kept", target, ROW_DETAILS[detail])
+            else:
+                failure = detail
+                self._record_target(job, "errors", target, detail, "!")
+            job.update(processed=done, progress=int(done / len(targets) * 100))
+        if removed:
+            self.invalidate_gallery_cache()
+        self._log_moved_records(job, totals)
+        line = f"Removed {_plural(removed, 'duplicate file')}"
+        if removed < len(targets):
+            line += f", {len(targets) - removed:,} not removed"
+        job.log_line(line)
+        if failure is not None:
+            line = f"Stopped after an error · {line}"
+        job.update(progress=100, status_line=f"{line} · {backup_note(backup)}")
+        if failure is not None:
+            job.publish({"type": "error", "code": "delete-failed", "message": line})
+            job.finish("failed")
+
+    @staticmethod
+    def _log_moved_records(job, totals):
+        links = totals["moved"] + totals["merged"]
+        if links:
+            job.log_line(f"Moved {_plural(links, 'database link')} to the kept copies")
+        if totals["favorites"]:
+            job.log_line(
+                f"Moved {_plural(totals['favorites'], 'favorite')} to the kept copies"
+            )
+        if totals["tags"]:
+            job.log_line(
+                f"Added {_plural(totals['tags'], 'image tag')} to the kept copies"
+            )
+        if totals["review"]:
+            job.log_line(
+                f"Settled {_plural(totals['review'], 'Needs review item')} "
+                "of removed copies"
+            )
+
+    @staticmethod
+    def _record_target(job, group, target, detail, mark="="):
+        row = {
+            "path": target["rel"],
+            "root": target["root_label"],
+            "root_index": target["root_index"],
+            "detail": detail,
+        }
+        job.record(
+            group, row, f"{mark} {target['root_label']}/{target['rel']} · {detail}"
+        )
+
+    def _remove_one_duplicate(self, db, target, keepers, spellings):
+        """``(outcome, detail, counts)`` for one reviewed removal (blocking).
+
+        The kept copy (once per group; cached in ``keepers``) and the file
+        are checked again, hashes included, and the file must not be its kept
+        copy. Then the file's records move to the kept copy in one short
+        transaction, and only then is the file unlinked: a crash or a failed
+        unlink leaves its records on the identical kept copy, never on a
+        deleted file. A failed move leaves the file and its records alone.
+        """
+        hasher = self._calculate_file_hash
+        group_id = target["group_id"]
+        if group_id not in keepers:
+            keepers[group_id] = _kept_copy_state(target, hasher)
+        if keepers[group_id] is None:
+            return "kept", KEEPER, None
+        problem = file_problem(
+            target["abs"],
+            target["root"],
+            target["hash"],
+            target["size"],
+            hasher,
+            keeper=target["keeper_abs"],
+            keeper_root=target["keeper_root"],
+        )
+        if problem:
+            return "kept", problem, None
+        try:
+            # Held open until the unlink: on Windows, deleting the kept copy
+            # itself through another path then fails instead of removing it.
+            hold = open(target["keeper_abs"], "rb")
+        except OSError:
+            return "kept", KEEPER, None
+        with hold:
+            return self._move_then_unlink(
+                db, target, keepers[group_id], spellings, hold
+            )
+
+    def _move_then_unlink(self, db, target, kept_stat, spellings, hold):
+        try:
+            counts = db.move_copy_records(copy_move(target, kept_stat, spellings))
+        except Exception:
+            self.logger.exception(f"Could not move the records of {target['rel']}")
+            return "error", RECORDS_FAILED, None
+        if not still_kept(target, hold):
+            # The kept copy went away or changed while the records moved:
+            # keep this file and give its records back to it.
+            return "kept", KEEPER_GONE, self._move_back(db, target, spellings)
+        try:
+            _unlink(target["abs"])
+        except FileNotFoundError:
+            return "removed", None, counts
+        except PermissionError:
+            self.logger.warning(
+                f"Duplicate file is locked: {target['rel']}; "
+                "its records now name the kept copy"
+            )
+            return "error", LOCKED_FILE, counts
+        except OSError:
+            self.logger.warning(
+                f"Could not delete duplicate file {target['rel']}; "
+                "its records now name the kept copy",
+                exc_info=True,
+            )
+            return "error", DELETE_FAILED, counts
+        self.logger.info(f"Deleted duplicate file: {target['rel']}")
+        _remove_thumbnail(Path(target["root"]), Path(target["rel"]), self.logger)
+        return "removed", None, counts
+
+    def _move_back(self, db, target, spellings):
+        """Give the kept copy's records to the file that stays (blocking)."""
+        swapped = {
+            **target,
+            "abs": target["keeper_abs"],
+            "root": target["keeper_root"],
+            "rel": target["keeper_rel"],
+            "locator": target["keeper_locator"],
+            "keeper_abs": target["abs"],
+            "keeper_root": target["root"],
+            "keeper_rel": target["rel"],
+            "keeper_locator": target["locator"],
+        }
+        try:
+            return db.move_copy_records(
+                copy_move(swapped, os.stat(target["abs"]), spellings)
+            )
+        except Exception:
+            self.logger.exception(f"Could not give the records back to {target['rel']}")
+            return None
+
+
+def _bound_problem(record, group, keeper=None):
+    """``file_problem`` without reading the file: size, mtime and inode.
+
+    ``keeper`` is the kept copy's record, never offered as a copy of itself.
+    """
+    return file_problem(
+        record["abs"],
+        record["root"],
+        group["hash"],
+        record["size"],
+        hash_check=False,
+        mtime_ns=record.get("mtime_ns"),
+        inode=record.get("inode"),
+        keeper=keeper["abs"] if keeper else None,
+        keeper_root=keeper["root"] if keeper else None,
+    )
+
+
+def _kept_copy_state(target, hasher):
+    """The kept copy's ``os.stat`` when it is still the reviewed bytes, else None."""
+    problem = file_problem(
+        target["keeper_abs"],
+        target["keeper_root"],
+        target["hash"],
+        target["keeper_size"],
+        hasher,
+    )
+    if problem:
+        return None
+    try:
+        return os.stat(target["keeper_abs"])
+    except OSError:
+        return None

py/api/duplicate_review.py

@@ -0,0 +1,451 @@
+"""Reviewing duplicate image files: the checks and the plan (#249 S6, T17).
+
+Pure helpers of ``py/api/duplicate_files.py``: the keep-list parsing, the
+per-file check ``file_problem`` (containment decided as text before any
+filesystem access, then a regular file of the found size and hash) and the
+public ``duplicate_files`` plan.
+"""
+
+import hashlib
+import json
+import os
+import stat as stat_module
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from .admin import DELETABLE_MEDIA_EXTENSIONS
+from .maintenance import DUPLICATE_FILES
+
+# Keep entries one review takes; with paths of a few hundred characters the
+# list stays inside the preview body cap (DUPLICATE_PREVIEW_BODY_MAX, 1 MiB).
+MAX_KEEP = 2000
+MAX_DETAILS = 50
+MAX_JOB_ID = 64
+HASH_CHUNK = 1024 * 1024
+
+# What ``file_problem`` finds, and how a plan counts it as protected.
+OUTSIDE_SOURCE = "outside"
+LINK = "link"
+CHANGED_CONTENT = "changed"
+GONE = "gone"
+UNREADABLE = "unreadable"
+KEEPER = "keeper"
+SAME = "same"
+MAYBE_SAME = "maybe-same"
+NO_FILE_ID = "no-file-id"
+# Two mounts of one share may round mtimes differently (SMB keeps 100 ns,
+# FAT 2 s): within this, equal sizes on another device may be one file.
+MOUNT_MTIME_SLACK_NS = 2_000_000_000
+PROTECTED_REASONS = {
+    MAYBE_SAME: (
+        "file may be the same file reached through another mount or share",
+        "files may be the same file reached through another mount or share",
+    ),
+    NO_FILE_ID: (
+        "file: can't confirm this is a separate file on this drive",
+        "files: can't confirm these are separate files on this drive",
+    ),
+    SAME: (
+        "file is the same file as its kept copy",
+        "files are the same file as their kept copy",
+    ),
+    OUTSIDE_SOURCE: (
+        "file is not inside a configured source",
+        "files are not inside a configured source",
+    ),
+    LINK: ("file is a link, not a regular file", "files are links, not regular files"),
+    CHANGED_CONTENT: (
+        "file changed since the search",
+        "files changed since the search",
+    ),
+    GONE: ("file no longer exists", "files no longer exist"),
+    UNREADABLE: ("file could not be read", "files could not be read"),
+    KEEPER: (
+        "file kept: its kept copy changed or is gone",
+        "files kept: their kept copy changed or is gone",
+    ),
+}
+# Job outcome rows (never an OS error text).
+CHANGED = "The file changed since the review; it was kept."
+KEEPER_CHANGED = "The kept copy changed or is gone; this file was kept."
+SAME_FILE = "This is the same file as its kept copy; it was kept."
+MAYBE_SAME_FILE = (
+    "May be the same file reached through another mount or share; it was kept."
+)
+NO_FILE_ID_ROW = "Can't confirm this is a separate file on this drive; it was kept."
+KEEPER_GONE = "keeper-gone"
+KEEPER_WENT = (
+    "The kept copy changed or went away during the removal; this file was kept "
+    "and its links stay with it."
+)
+ROW_DETAILS = {
+    KEEPER_GONE: None,  # filled below
+    MAYBE_SAME: MAYBE_SAME_FILE,
+    NO_FILE_ID: NO_FILE_ID_ROW,
+    SAME: SAME_FILE,
+    OUTSIDE_SOURCE: "The file is no longer inside its source; it was kept.",
+    LINK: "The file is now a link, not a regular file; it was kept.",
+    CHANGED_CONTENT: CHANGED,
+    GONE: "The file was already gone.",
+    UNREADABLE: "The file could not be read; it was kept.",
+    KEEPER: KEEPER_CHANGED,
+}
+# An unlink fails only after the file's records moved to its kept copy.
+LOCKED_FILE = (
+    "The file is open in another program or read-only; it was not deleted, "
+    "and its links now name the kept copy."
+)
+DELETE_FAILED = "The file could not be deleted; its links now name the kept copy."
+NOT_ATTEMPTED = "Not attempted: the job stopped after an error."
+KEPT_COPY = "Kept copy"
+SOURCES_CHANGED = (
+    "The sources changed since the review; nothing was deleted. Review again."
+)
+RECORDS_FAILED = (
+    "The database could not be updated, so the file was not deleted. "
+    "Its links and tags are unchanged."
+)
+FIND_FAILED = "The duplicate search failed."
+FIND_ERROR_EVENT = "An internal error occurred. Check server logs for details."
+FIND_CANCELLED = "Duplicate scan cancelled"
+
+
+class SourcesChanged(Exception):
+    """A source a review or removal names is no longer listed at its index."""
+
+
+ROW_DETAILS[KEEPER_GONE] = KEEPER_WENT
+
+
+def still_kept(target, hold):
+    """Right before an unlink: the held kept copy is still the file at its
+    path, and the file is still not the kept copy itself (``keeper_problem``).
+    """
+    try:
+        if not os.path.samestat(os.fstat(hold.fileno()), os.stat(target["keeper_abs"])):
+            return False
+        info = os.lstat(target["abs"])
+        problem = keeper_problem(
+            target["abs"],
+            target["root"],
+            info,
+            target["keeper_abs"],
+            target["keeper_root"],
+        )
+        return problem is None
+    except OSError:
+        return False
+
+
+def _plural(count, one, many=None):
+    return f"{count:,} {one if count == 1 else (many or one + 's')}"
+
+
+def human_bytes(size):
+    """``10 B``, ``1.5 KB``, ``2.0 MB``…"""
+    if size < 1024:
+        return f"{size:,} B"
+    for unit in ("KB", "MB", "GB", "TB"):
+        size /= 1024
+        if size < 1024 or unit == "TB":
+            return f"{size:,.1f} {unit}"
+    return f"{size:,.1f} TB"  # pragma: no cover - unreachable
+
+
+def _hash_file(path):
+    digest = hashlib.sha256()
+    with open(path, "rb") as handle:
+        for chunk in iter(lambda: handle.read(HASH_CHUNK), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def file_problem(
+    path,
+    root,
+    digest,
+    size,
+    hasher=None,
+    *,
+    hash_check=True,
+    mtime_ns=None,
+    inode=None,
+    keeper=None,
+    keeper_root=None,
+):
+    """Why ``path`` may not be treated as a copy in ``root``, or None (blocking).
+
+    Containment is decided as text first, with no filesystem access: the
+    normalized path must lie below the normalized root, outside
+    ``thumbnails``, with a media extension. Only then is the file looked at:
+    a regular file, not a link, whose resolved path is still inside the
+    resolved root and that is not ``keeper`` itself (``keeper_problem``: a
+    hardlink, one file reached through two spellings or two mounts, or a
+    volume that cannot tell files apart), of ``size`` bytes.
+    ``mtime_ns`` and ``inode`` (``[st_dev, st_ino]``), when given, bind it to
+    what the search saw; ``hash_check`` compares its content hash with
+    ``digest`` (the removal job, right before an unlink). Every handle is
+    closed when this returns.
+    """
+    # py.config imports ComfyUI's server module: only when it is in use.
+    from ..config import _canonical_path, _lexical_path, _lexically_inside
+
+    lexical = _lexical_path(str(path))
+    base = _lexical_path(str(root))
+    if lexical == base or not _lexically_inside(lexical, [base]):
+        return OUTSIDE_SOURCE
+    parts = Path(os.path.relpath(lexical, base)).parts
+    if "thumbnails" in parts or ".." in parts:
+        return OUTSIDE_SOURCE
+    if os.path.splitext(lexical)[1].lower() not in DELETABLE_MEDIA_EXTENSIONS:
+        return OUTSIDE_SOURCE
+    try:
+        info = os.lstat(path)
+    except FileNotFoundError:
+        return GONE
+    except OSError:
+        return UNREADABLE
+    if stat_module.S_ISLNK(info.st_mode):
+        return LINK
+    if not stat_module.S_ISREG(info.st_mode):
+        return LINK
+    try:
+        if not _lexically_inside(_canonical_path(path), [_canonical_path(root)]):
+            return OUTSIDE_SOURCE  # a linked folder on the way
+        if keeper is not None:
+            same = keeper_problem(path, root, info, keeper, keeper_root)
+            if same:
+                return same
+        if info.st_size != size:
+            return CHANGED_CONTENT
+        if mtime_ns is not None and info.st_mtime_ns != mtime_ns:
+            return CHANGED_CONTENT
+        if inode is not None and [info.st_dev, info.st_ino] != list(inode):
+            return CHANGED_CONTENT
+        if hash_check and (hasher or _hash_file)(path) != digest:
+            return CHANGED_CONTENT
+    except FileNotFoundError:
+        return GONE
+    except OSError:
+        return UNREADABLE
+    return None
+
+
+def keeper_problem(path, root, info, keeper, keeper_root=None):
+    """Why ``path`` (its ``lstat`` is ``info``) may be ``keeper`` itself, or None.
+
+    - ``NO_FILE_ID``: a volume reports no file id (``st_ino`` 0), so the two
+      cannot be told apart;
+    - ``SAME``: ``os.path.samestat`` (a hardlink, or one file reached through
+      two spellings of one mount);
+    - ``MAYBE_SAME``: another device, the same size, mtimes within
+      ``MOUNT_MTIME_SLACK_NS``, and the same ``st_ino`` (NFS and SMB report
+      the server's file id) or the same path within their sources, or one
+      ending with the other on a folder name (one share mounted twice, a
+      FUSE pool over a listed disk, sources listed at different depths).
+      Protect, never guess: a true copy on another disk at the same relative
+      path is kept too.
+
+    Raises OSError when the kept copy cannot be read (the caller's GONE or
+    UNREADABLE).
+    """
+    kept = os.stat(keeper)
+    if not info.st_ino or not kept.st_ino:
+        return NO_FILE_ID
+    if os.path.samestat(info, kept):
+        return SAME
+    if info.st_dev == kept.st_dev or info.st_size != kept.st_size:
+        return None
+    if abs(info.st_mtime_ns - kept.st_mtime_ns) > MOUNT_MTIME_SLACK_NS:
+        return None
+    if info.st_ino == kept.st_ino:
+        return MAYBE_SAME
+    if keeper_root is not None and _same_tail(
+        _relative_key(path, root), _relative_key(keeper, keeper_root)
+    ):
+        return MAYBE_SAME
+    return None
+
+
+def _same_tail(one, other):
+    """Whether one relative path is the other, or ends with it on a folder name.
+
+    Two mounts of one tree may be listed at different depths (``2024/x.png``
+    in one source, ``x.png`` in the other); ``ax.png`` never matches ``x.png``.
+    """
+    if one is None or other is None:
+        return False
+    return one == other or one.endswith("/" + other) or other.endswith("/" + one)
+
+
+def _relative_key(path, root):
+    try:
+        relative = os.path.relpath(str(path), str(root))
+    except ValueError:  # another drive (Windows)
+        return None
+    return os.path.normcase(relative).replace("\\", "/")
+
+
+@dataclass
+class HeldFind:
+    """One find: where it looked and, once it finished, what it found."""
+
+    origin: str  # "workspace" or "gallery"
+    job_id: str = ""
+    roots: list = field(default_factory=list)  # [(source index or None, root)]
+    legacy_dir: str = None
+    groups: list = None  # [{group_id, hash, size, files:[record]}]
+
+    def covers(self, legacy_dir):
+        """Whether its events answer the legacy gallery for ``legacy_dir``."""
+        if self.origin == "gallery":
+            return True
+        if not legacy_dir:
+            return False
+        wanted = _identity(legacy_dir)
+        return any(_identity(root) == wanted for _index, root in self.roots)
+
+
+def _identity(path):
+    return os.path.normcase(os.path.realpath(str(path)))
+
+
+def _group_order(group):
+    return (-(len(group["files"]) - 1) * group["size"], group["hash"])
+
+
+def _keep_entry(entry):
+    """``((group_id, root_index, path), None)`` or ``(None, message)``."""
+    if not isinstance(entry, dict) or set(entry) != {"group_id", "root_index", "path"}:
+        return None, "Each keep entry is {group_id, root_index, path}"
+    group_id, index, path = entry["group_id"], entry["root_index"], entry["path"]
+    if not isinstance(group_id, str) or not 0 < len(group_id) <= 128:
+        return None, "group_id must be a group's id"
+    if index is not None and (
+        isinstance(index, bool) or not isinstance(index, int) or index < 0
+    ):
+        return None, "root_index must be a source index"
+    if not isinstance(path, str) or not 0 < len(path) <= 4096:
+        return None, "path must be a file's path in its group"
+    return (group_id, index, path), None
+
+
+def _parse_options(options):
+    """``(job_id, keep, None)`` for duplicate_files options, else a message."""
+    if not isinstance(options, dict) or set(options) != {"job_id", "keep"}:
+        return None, None, "duplicate_files takes options {job_id, keep}"
+    job_id, keep = options["job_id"], options["keep"]
+    if not isinstance(job_id, str) or not 0 < len(job_id) <= MAX_JOB_ID:
+        return None, None, "job_id must be a duplicate search's job id"
+    if not isinstance(keep, list) or not 0 < len(keep) <= MAX_KEEP:
+        return None, None, f"keep must list 1 to {MAX_KEEP:,} files to keep"
+    parsed = []
+    for entry in keep:
+        key, message = _keep_entry(entry)
+        if message:
+            return None, None, message
+        parsed.append(key)
+    return job_id, parsed, None
+
+
+def _select(held, keep):
+    """``({group_id: (group, [keeper records])}, None)`` or ``(None, message)``."""
+    groups = {group["group_id"]: group for group in held.groups}
+    selection = {}
+    for group_id, index, path in keep:
+        group = groups.get(group_id)
+        if group is None:
+            return None, "A kept file names a group this search did not find"
+        match = [
+            record
+            for record in group["files"]
+            if record["root_index"] == index and record["rel"] == path
+        ]
+        if not match:
+            return None, "A kept file is not in its group"
+        _group, keepers = selection.setdefault(group_id, (group, []))
+        if match[0] not in keepers:
+            keepers.append(match[0])
+    return selection, None
+
+
+SOURCES_CHANGED_REVIEW = (
+    "The sources changed since the search. Search for duplicates again."
+)
+
+
+def _fingerprint(targets):
+    digest = hashlib.sha256()
+    for target in sorted(targets, key=lambda t: _identity(t["abs"])):
+        part = [
+            os.path.normcase(target["abs"]),
+            target["hash"],
+            target["size"],
+            os.path.normcase(target["keeper_abs"]),
+        ]
+        digest.update(json.dumps(part).encode("utf-8") + b"\n")
+    return "sha256:" + digest.hexdigest()
+
+
+def _record_parts(records, target):
+    """What moves with one removal: its links, tags and Needs review items."""
+    found = records.get(target["abs"], {})
+    links, tags, review = (found.get(key, 0) for key in ("links", "tags", "review"))
+    parts = [
+        (
+            (
+                f"{_plural(links, 'database link')} "
+                f"{'moves' if links == 1 else 'move'} to the kept copy"
+            )
+            if links
+            else "no database links"
+        )
+    ]
+    if tags:
+        parts.append("its image tags join the kept copy's")
+    if review:
+        item = "Needs review item is" if review == 1 else "Needs review items are"
+        parts.append(f"{review:,} {item} settled")
+    return parts
+
+
+def _public_plan(targets, protected, records):
+    count = len(targets)
+    groups = len({t["group_id"] for t in targets})
+    size = sum(t["size"] for t in targets)
+    reasons = [
+        f"{n:,} {PROTECTED_REASONS[code][0 if n == 1 else 1]}"
+        for code, n in sorted(protected.items())
+    ]
+    details = [
+        {
+            "label": target["rel"],
+            "detail": f"{target['root_label']} · a copy of {target['keeper_rel']}"
+            + "".join(f" · {part}" for part in _record_parts(records, target)),
+        }
+        for target in targets[:MAX_DETAILS]
+    ]
+    label = f"{_plural(count, 'duplicate file')} in {_plural(groups, 'group')}"
+    media = (
+        f"Deletes {_plural(count, 'file')} ({human_bytes(size)}) from disk; the "
+        "kept copies stay. The safety backup covers the database, not these files."
+    )
+    tagged = sum(1 for t in targets if records.get(t["abs"], {}).get("tags"))
+    review = sum(records.get(t["abs"], {}).get("review", 0) for t in targets)
+    if tagged:
+        media += (
+            f" The image tags of {_plural(tagged, 'removed copy', 'removed copies')}"
+            " join their kept copy's."
+        )
+    if review:
+        media += (
+            f" {_plural(review, 'pending Needs review item')} of the removed "
+            f"{'copy is' if review == 1 else 'copies are'} settled."
+        )
+    return {
+        "operation": DUPLICATE_FILES,
+        "scope": {"count": count, "label": label},
+        "protected": {"count": sum(protected.values()), "reasons": reasons},
+        "media": {"deletes_files": True, "label": media},
+        "details": details,
+    }

py/api/duplicate_spellings.py

@@ -0,0 +1,133 @@
+"""Every path a duplicate copy is recorded under (#249 System S6, Task 17).
+
+The search walks each source's real path, but a record names a file the way
+it was seen when it was written: under the folder the image monitor watches,
+the configured spelling, ComfyUI's output folder, a symlink to it, or a
+parent of a source listed later. ``Spellings`` finds them by containment:
+for each known folder spelling ``S`` (resolving to the folder ``R``) and a
+file ``P`` inside ``R``, the file is also recorded as ``S + relpath(P, R)``.
+
+Folders are compared by ``os.stat`` identity (``st_dev``, ``st_ino``), not
+by text, so a case variant on a case-insensitive filesystem (macOS, Windows)
+or a symlinked spelling matches. A volume that reports no file id falls
+back to ``normcase(realpath)``. Favorites are keyed by a listed root and the
+path below it, so ``favorites`` lists every listed root that holds a file.
+Blocking: each lookup stats the file's folders once (cached).
+"""
+
+import os
+from pathlib import Path
+
+from .images import _favorite_relative_key, _favorite_root_key
+
+
+def _identity(path):
+    try:
+        info = os.stat(path)
+    except OSError:
+        return None
+    if info.st_ino:
+        return ("inode", info.st_dev, info.st_ino)
+    return ("path", os.path.normcase(os.path.realpath(str(path))))
+
+
+class Spellings:
+    """Folder spellings and listed roots, matched to files by containment."""
+
+    def __init__(self, folders, listed_roots):
+        self._cache = {}
+        self._folders = []
+        for spelled in dict.fromkeys(str(f) for f in folders if f):
+            identity = self._identity(spelled)
+            if identity is not None:
+                self._folders.append((spelled, identity))
+        self._listed = []
+        for root in dict.fromkeys(str(r) for r in listed_roots if r):
+            identity = self._identity(root)
+            if identity is not None:
+                self._listed.append((root, identity))
+
+    def _identity(self, path):
+        if path not in self._cache:
+            self._cache[path] = _identity(path)
+        return self._cache[path]
+
+    def _ancestors(self, path):
+        """``{identity: folder}`` of every folder above ``path``, nearest first."""
+        found = {}
+        folder = os.path.dirname(os.path.abspath(str(path)))
+        while True:
+            identity = self._identity(folder)
+            if identity is not None:
+                found.setdefault(identity, folder)
+            parent = os.path.dirname(folder)
+            if parent == folder:
+                return found
+            folder = parent
+
+    def of(self, path):
+        """``{folder spelling: path spelled under it}``; ``""`` is ``path`` itself."""
+        path = os.path.abspath(str(path))
+        spelled = {"": path}
+        ancestors = self._ancestors(path)
+        for folder, identity in self._folders:
+            inside = ancestors.get(identity)
+            if inside is not None:
+                spelled.setdefault(
+                    folder, os.path.join(folder, os.path.relpath(path, inside))
+                )
+        return spelled
+
+    def favorites(self, path):
+        """``{listed root: path below it (POSIX)}`` for each listed root holding it."""
+        path = os.path.abspath(str(path))
+        ancestors = self._ancestors(path)
+        held = {}
+        for root, identity in self._listed:
+            inside = ancestors.get(identity)
+            if inside is not None:
+                held[root] = Path(os.path.relpath(path, inside)).as_posix()
+        return held
+
+
+def copy_move(target, kept_stat, spellings):
+    """What ``move_copy_records`` moves for one removal.
+
+    A record found under a folder spelling moves to the kept copy under
+    the same spelling when that folder holds it too, else to its real
+    path. Its favorite moves under every listed root holding the file.
+    """
+    removed = spellings.of(target["abs"])
+    kept = spellings.of(target["keeper_abs"])
+    targets = {
+        spelled: kept.get(folder, target["keeper_abs"])
+        for folder, spelled in removed.items()
+    }
+    kept_favorites = spellings.favorites(target["keeper_abs"])
+    favorites = []
+    for root, relative in spellings.favorites(target["abs"]).items():
+        kept_root, kept_rel = (
+            (root, kept_favorites[root])
+            if root in kept_favorites
+            else (target["keeper_root"], target["keeper_rel"])
+        )
+        favorites.append(
+            (
+                _favorite_root_key(root),
+                _favorite_relative_key(relative),
+                _favorite_root_key(kept_root),
+                _favorite_relative_key(kept_rel),
+            )
+        )
+    return {
+        "removed": list(removed.values()),
+        "kept": list(kept.values()),
+        "targets": targets,
+        "kept_path": target["keeper_abs"],
+        "favorites": favorites,
+        "removed_locator": target["locator"],
+        "kept_locator": target["keeper_locator"],
+        "sha256": target["hash"],
+        "kept_size": kept_stat.st_size,
+        "kept_mtime_ns": kept_stat.st_mtime_ns,
+    }

py/api/history.py

@@ -734,7 +734,10 @@ class HistoryRoutesMixin:
                         raise HistoryReadError(422, "response-limit") from None
                 else:
                     prepared = _prepare_current_media(
-                        resolved, scope, thumbnail=kind == "thumbnail"
+                        resolved,
+                        scope,
+                        thumbnail=kind == "thumbnail",
+                        **media_options,
                     )
                 _check_current(scope, self, fence, epoch, generation)
                 return prepared
@@ -761,25 +764,44 @@ class HistoryRoutesMixin:
                     _check_current(scope, self, fence, epoch, generation)
                 raise
 
-        pending = asyncio.create_task(self._run_in_executor(project))
-        try:
-            prepared = await asyncio.shield(pending)
-        except asyncio.CancelledError:
-            pending.add_done_callback(_close_prepared_result)
-            raise
-        except (
-            HistoryReadError,
-            HistoryEpochChanged,
-            sqlite3.Error,
-            OSError,
-            ValueError,
-        ):
-            raise
-        except Exception as error:
-            self.logger.warning(
-                "Current history media unavailable (%s)", type(error).__name__
-            )
-            raise _current_unavailable("temporary_unavailable") from None
+        async def execute():
+            pending = asyncio.create_task(self._run_in_executor(project))
+            try:
+                return await asyncio.shield(pending)
+            except asyncio.CancelledError:
+                pending.add_done_callback(_close_prepared_result)
+                raise
+            except (
+                HistoryReadError,
+                HistoryEpochChanged,
+                sqlite3.Error,
+                OSError,
+                ValueError,
+            ):
+                raise
+            except Exception as error:
+                self.logger.warning(
+                    "Current history media unavailable (%s)", type(error).__name__
+                )
+                raise _current_unavailable("temporary_unavailable") from None
+
+        # A missing preview (#249) is noted on the first read, made through
+        # the serve route's admission and single flight, then read again. A
+        # video whose preview cannot be made is the placeholder frame.
+        needed = []
+        media_options = {}
+        if kind == "thumbnail":
+            media_options = {
+                "ensure_preview": lambda *preview: needed.append(preview),
+                "placeholder": self._video_placeholder_bytes(),
+            }
+        prepared = await execute()
+        if needed:
+            if not isinstance(prepared, bytes):
+                prepared.close()
+            await self._admit_preview(*needed[0])
+            media_options["ensure_preview"] = None
+            prepared = await execute()
         if isinstance(prepared, bytes):
             return _success_response(prepared)
         return await _current_media_response(request, prepared)

py/api/history_current.py

@@ -5,6 +5,7 @@ from dataclasses import dataclass
 import errno
 import hashlib
 import hmac
+import io
 import json
 import mimetypes
 import os
@@ -309,17 +310,35 @@ class _OpenedCurrentMedia:
         self.handle.close()
 
 
+def _preview_writable(root: Path, preview: Path) -> bool:
+    """A preview may be written: ``thumbnails`` and every link segment of the
+    preview's folder resolve inside *root* (the serve route's rule, #249)."""
+    folder = root / "thumbnails"
+    return _canonical(folder).is_relative_to(_canonical(root)) and _canonical(
+        preview
+    ).is_relative_to(_canonical(folder))
+
+
 def _prepare_current_media(
-    resolved: ResolvedCurrentCandidate, scope: CurrentOutputScope, *, thumbnail: bool
+    resolved: ResolvedCurrentCandidate,
+    scope: CurrentOutputScope,
+    *,
+    thumbnail: bool,
+    ensure_preview=None,
+    placeholder: bytes | None = None,
 ) -> _OpenedCurrentMedia:
+    """Open the current file, or its preview for ``thumbnail``.
+
+    A missing preview that may be written is handed to
+    ``ensure_preview(src, dst, is_video)`` (the route makes it and reads
+    again, #249). Without one an image's original is opened and a video is
+    the ``placeholder`` JPEG, never the video inside an ``<img>``.
+    """
     target = resolved.canonical_target
     if thumbnail:
         relative = target.relative_to(resolved.gallery_root)
-        extension = (
-            ".jpg"
-            if target.suffix.lower() in VIDEO_EXTENSIONS
-            else target.suffix.lower()
-        )
+        is_video = target.suffix.lower() in VIDEO_EXTENSIONS
+        extension = ".jpg" if is_video else target.suffix.lower()
         preview = (
             resolved.gallery_root
             / "thumbnails"
@@ -328,7 +347,14 @@ def _prepare_current_media(
         try:
             preview.stat()
         except FileNotFoundError:
-            pass
+            if ensure_preview is not None and _preview_writable(
+                resolved.gallery_root, preview
+            ):
+                ensure_preview(target, preview, is_video)
+            if is_video and placeholder is not None:
+                return _OpenedCurrentMedia(
+                    io.BytesIO(placeholder), len(placeholder), "image/jpeg"
+                )
         else:
             target = _canonical(preview)
     handle = _open_current_target(target, resolved.namespace, scope)

py/api/images.py

@@ -20,7 +20,7 @@ from typing import Callable, NamedTuple
 from aiohttp import web
 from PIL import Image, UnidentifiedImageError
 
-from .jobs import THUMBNAILS_JOB_KIND
+from .jobs import MAX_OUTCOME_ROWS, THUMBNAILS_JOB_KIND
 from .prompts import bad_request, parse_page_params, publish_image_paths
 from .prompts import thumbnail_url_for
 from .prompts import safe_error_message as _safe_error
@@ -110,6 +110,8 @@ def _card_header(path: Path) -> tuple:
 
 # The legacy gallery page's thumbnail generation, as Activity lists it (D80).
 THUMBNAIL_GENERATION_TITLE = "Thumbnail generation (gallery page)"
+# Its clear: automatic previews are not written while it runs.
+THUMBNAIL_CLEAR_TITLE = "Thumbnail clear (gallery page)"
 
 
 class _CardFacts(NamedTuple):
@@ -235,6 +237,8 @@ MEDIA_EXTENSIONS = GALLERY_IMAGE_EXTENSIONS | VIDEO_EXTENSIONS
 # Caps on request-triggered filesystem work.
 MAX_SCAN_DEPTH = 12
 MAX_THUMBNAIL_FILES = 10_000
+# Already-fresh files a thumbnail job lists under Skipped (its counter stays exact).
+MAX_FRESH_LISTED = MAX_OUTCOME_ROWS
 
 
 def _worker_threads():
@@ -259,28 +263,127 @@ def _temp_sibling(dst):
     return dst.with_name(f".{dst.stem}.part-{token}{dst.suffix}")
 
 
-def _atomic_save(img, dst, fmt=None, **save_kwargs):
+class PreviewWriteHeld(Exception):
+    """A preview write stopped before its rename: ``guard()`` said no (#249)."""
+
+
+# The result error of a write held before its rename (never remembered).
+PREVIEW_HELD = "Preview writes are held while another job runs"
+
+
+def _replace_unless_held(tmp, dst, guard=None):
+    """Rename *tmp* over *dst*, unless ``guard()`` is false (PreviewWriteHeld)."""
+    if guard is not None and not guard():
+        raise PreviewWriteHeld(PREVIEW_HELD)
+    os.replace(tmp, dst)
+
+
+def _atomic_save(img, dst, fmt=None, *, guard=None, **save_kwargs):
     """Save a PIL image to *dst* without a partially written file ever being
-    visible at that path: write a temp sibling, then rename over it."""
+    visible at that path: write a temp sibling, then rename over it.
+
+    ``guard()`` (automatic previews) is asked again just before the rename;
+    false leaves *dst* untouched and raises PreviewWriteHeld.
+    """
     tmp = _temp_sibling(dst)
     try:
         img.save(tmp, fmt, **save_kwargs)
-        os.replace(tmp, dst)
+        _replace_unless_held(tmp, dst, guard)
     finally:
         if tmp.exists():
             tmp.unlink()
 
 
-def _write_image_thumbnail(src, dst, thumbnail_size):
-    """Resize *src* into *dst* (blocking PIL work), atomically."""
+# Encoder settings per thumbnail quality (the System thumbnail dialog's
+# "Medium · optimized", "High · original", "Low · fastest"). The legacy
+# routes always encode as "medium", which is what they always wrote.
+THUMBNAIL_ENCODINGS = {
+    "low": {"quality": 70, "optimize": False},
+    "medium": {"quality": 85, "optimize": True},
+    "high": {"quality": 95, "optimize": True},
+}
+DEFAULT_THUMBNAIL_ENCODING = THUMBNAIL_ENCODINGS["medium"]
+# ffmpeg's JPEG scale (-q:v, 2 best to 31 worst) for each encoding's quality.
+_FFMPEG_QSCALE = {95: 2, 85: 4, 70: 8}
+
+# A preview that could not be replaced (#249 S5 M6). Windows refuses to rename
+# over a file another process has open (WinError 5 or 32), such as a preview
+# the gallery is serving; the old preview stays and the file counts as an error.
+PREVIEW_IN_USE = "Preview in use by another program; the old one was kept"
+PREVIEW_NOT_WRITABLE = "Permission denied writing the preview"
+_WINDOWS_LOCKED = (5, 32)  # ERROR_ACCESS_DENIED, ERROR_SHARING_VIOLATION
+
+
+def _preview_write_error(exc):
+    """The readable error for a PermissionError while writing a preview."""
+    if getattr(exc, "winerror", None) in _WINDOWS_LOCKED:
+        return PREVIEW_IN_USE
+    return PREVIEW_NOT_WRITABLE
+
+
+def _ffmpeg_qscale(encoding):
+    """ffmpeg's ``-q:v`` for a THUMBNAIL_ENCODINGS entry."""
+    return str(_FFMPEG_QSCALE.get(encoding["quality"], 4))
+
+
+def _write_image_thumbnail(src, dst, thumbnail_size, encoding=None, guard=None):
+    """Resize *src* into *dst* (blocking PIL work), atomically.
+
+    ``encoding`` is one of THUMBNAIL_ENCODINGS (medium when None); a PNG
+    preview keeps only its ``optimize`` flag. ``guard`` see _atomic_save.
+    """
+    encoding = encoding or DEFAULT_THUMBNAIL_ENCODING
     with Image.open(src) as img:
         if img.mode in ("RGBA", "LA", "P"):
             img = img.convert("RGB")
         img.thumbnail(thumbnail_size, Image.Resampling.LANCZOS)
-        save_kwargs = {"quality": 85, "optimize": True}
+        save_kwargs = dict(encoding)
         if dst.suffix.lower() == ".png":
-            save_kwargs = {"optimize": True}
-        _atomic_save(img, dst, **save_kwargs)
+            save_kwargs = {"optimize": encoding["optimize"]}
+        _atomic_save(img, dst, guard=guard, **save_kwargs)
+
+
+def _video_placeholder_image(thumbnail_size):
+    """The placeholder frame for a video no decoder could read: a play mark."""
+    from PIL import ImageDraw, ImageFont
+
+    img = Image.new("RGB", thumbnail_size, color=(50, 50, 50))
+    draw = ImageDraw.Draw(img)
+    center_x, center_y = thumbnail_size[0] // 2, thumbnail_size[1] // 2
+    triangle_size = min(thumbnail_size) // 4
+    points = [
+        (center_x - triangle_size // 2, center_y - triangle_size // 2),
+        (center_x - triangle_size // 2, center_y + triangle_size // 2),
+        (center_x + triangle_size // 2, center_y),
+    ]
+    draw.polygon(points, fill=(255, 255, 255))
+    try:
+        font = ImageFont.load_default()
+        text = "VIDEO"
+        bbox = draw.textbbox((0, 0), text, font=font)
+        text_width = bbox[2] - bbox[0]
+        draw.text(
+            (center_x - text_width // 2, center_y + triangle_size // 2 + 10),
+            text,
+            fill=(255, 255, 255),
+            font=font,
+        )
+    except (OSError, AttributeError):
+        pass
+    return img
+
+
+def _is_decode_error(exc):
+    """True when *exc* says the file itself cannot be decoded (#249 M4).
+
+    Pillow's decoder errors are OSErrors without an errno; a real I/O error
+    (disk full, a share gone) carries one and is worth another try.
+    """
+    if isinstance(exc, (UnidentifiedImageError, Image.DecompressionBombError)):
+        return True
+    if isinstance(exc, SyntaxError):
+        return True
+    return isinstance(exc, OSError) and exc.errno is None
 
 
 def _new_thumbnail_stats():
@@ -671,23 +774,51 @@ def _iter_media_files(output_path, max_files):
             yield root_path / name
 
 
-def media_thumbnail_url(output_path, rel_path, root_index, *, root_scope=None):
-    """The Images listing's thumbnail URL for *rel_path* under *output_path*,
-    or None when no thumbnail exists (one blocking stat). The Needs review
-    queue cards use the same URL (#249 F, M10)."""
+def media_thumbnail_rel(rel_path):
+    """``thumbnails/<rel>_thumb<ext>`` (POSIX) for the media file *rel_path*.
+
+    The extension is lower case, and a video's preview is a ``.jpg``: the
+    name every thumbnail writer (the plan, the job, automatic previews) uses.
+    """
     rel_path = Path(rel_path)
     extension = rel_path.suffix.lower()
-    thumb_rel = (
+    return (
         f"thumbnails/{rel_path.with_suffix('').as_posix()}_thumb"
         f"{'.jpg' if extension in VIDEO_EXTENSIONS else extension}"
     )
+
+
+def media_thumbnail(output_path, rel_path, root_index, *, root_scope=None):
+    """``(thumbnail_url, ready)`` for *rel_path* under *output_path* (one
+    blocking stat). The Images listing, the Needs review queue cards and the
+    Library use the same URL (#249 F, M10).
+
+    An existing preview's URL carries its mtime as ``v=`` and ``ready`` is
+    True. A missing one is still listed (without ``v=``): the serve route
+    makes it on first view and falls back when it cannot
+    (py/api/thumbnail_auto.py), so no card loads the full file because its
+    preview was not made yet. Media inside a nested ``thumbnails`` folder is
+    never previewed: ``(None, False)``, and its card shows the original.
+    """
+    rel_path = Path(rel_path)
+    if "thumbnails" in rel_path.parts[:-1]:
+        return None, False
+    thumb_rel = media_thumbnail_rel(rel_path)
     thumbnail_url = thumbnail_url_for(thumb_rel, Path(output_path) / thumb_rel)
+    ready = thumbnail_url is not None
     if thumbnail_url is None:
-        return None
-    thumbnail_url += f"&root={root_index}"
+        quoted = urllib.parse.quote(thumb_rel, safe="/")
+        thumbnail_url = f"/prompt_manager/images/serve/{quoted}?root={root_index}"
+    else:
+        thumbnail_url += f"&root={root_index}"
     if root_scope is not None:
         thumbnail_url += f"&root_scope={root_scope}"
-    return thumbnail_url
+    return thumbnail_url, ready
+
+
+def media_thumbnail_url(output_path, rel_path, root_index, *, root_scope=None):
+    """The thumbnail URL of ``media_thumbnail`` (None for nested previews)."""
+    return media_thumbnail(output_path, rel_path, root_index, root_scope=root_scope)[0]
 
 
 def _output_image_entry(
@@ -705,7 +836,7 @@ def _output_image_entry(
     extension = media_path.suffix.lower()
     is_video = extension in VIDEO_EXTENSIONS
 
-    thumbnail_url = media_thumbnail_url(
+    thumbnail_url, thumbnail_ready = media_thumbnail(
         output_path, rel_path, root_index, root_scope=root_scope
     )
     encoded_path = urllib.parse.quote(rel_path.as_posix(), safe="/")
@@ -724,6 +855,8 @@ def _output_image_entry(
         "root_index": root_index,
         "url": url,
         "thumbnail_url": thumbnail_url,
+        # Whether the preview file exists yet (the legacy gallery's dot, #249).
+        "thumbnail_ready": thumbnail_ready,
         "size": stat.st_size,
         "modified_time": stat.st_mtime,
         "extension": extension,
@@ -945,6 +1078,10 @@ class ImageRoutesMixin:
         async def thumbnail_status_route(request):
             return await self.get_thumbnail_status(request)
 
+        @routes.get("/prompt_manager/images/thumbnails/plan")
+        async def thumbnail_plan_route(request):
+            return await self.get_thumbnail_plan(request)
+
         @routes.post("/prompt_manager/images/generate-thumbnails")
         async def generate_thumbnails_route(request):
             return await self.generate_thumbnails(request)
@@ -1639,6 +1776,11 @@ class ImageRoutesMixin:
                 if image_path.is_file():
                     return self._file_response(image_path)
 
+            # A listed preview that was not made yet is made now (#249).
+            preview = await self._serve_missing_thumbnail(output_dirs, filepath)
+            if preview is not None:
+                return preview
+
             # ComfyUI's cache middleware would otherwise let browsers cache this
             # miss for an hour, hiding a thumbnail generated a moment later.
             return web.json_response(
@@ -1817,15 +1959,35 @@ class ImageRoutesMixin:
         self.logger.info(payload["message"])
         return payload
 
-    def _plan_gallery_thumbnails(self, output_dirs: list[Path]) -> dict:
-        """Plan every gallery root with one shared per-run generation cap."""
-        combined = {"targets": [], "skipped": 0, "remaining": 0}
+    def _plan_gallery_thumbnails(
+        self, output_dirs: list[Path], rebuild: bool = False, stop=None
+    ) -> dict:
+        """Plan every gallery root with one shared per-run generation cap.
+
+        See ``_plan_thumbnails``; ``fresh_files`` keeps at most
+        MAX_FRESH_LISTED files across the roots. ``stop()`` true ends the walk
+        at the next file (a cancelled job); the partial plan is returned.
+        """
+        combined = {
+            "targets": [],
+            "skipped": 0,
+            "remaining": 0,
+            "fresh": 0,
+            "fresh_files": [],
+        }
         for root in output_dirs:
+            if stop is not None and stop():
+                break
             root = Path(root)
-            plan = self._plan_thumbnails(root, root / "thumbnails")
+            plan = self._plan_thumbnails(
+                root, root / "thumbnails", rebuild=rebuild, stop=stop
+            )
             available = max(0, MAX_THUMBNAIL_FILES - len(combined["targets"]))
             combined["targets"].extend(plan["targets"][:available])
             combined["skipped"] += plan["skipped"]
+            combined["fresh"] += plan["fresh"]
+            room = max(0, MAX_FRESH_LISTED - len(combined["fresh_files"]))
+            combined["fresh_files"].extend(plan["fresh_files"][:room])
             combined["remaining"] += plan["remaining"] + max(
                 0, len(plan["targets"]) - available
             )
@@ -1846,6 +2008,8 @@ class ImageRoutesMixin:
                     "total_images": pending + plan["skipped"],
                     "pending": pending,
                     "ready": plan["skipped"],
+                    # Files whose automatic preview failed this session (#249).
+                    "failed": self._auto_thumbnail_failures(),
                     "collection_id": hashlib.sha256(collection.encode()).hexdigest()[
                         :16
                     ],
@@ -1858,22 +2022,30 @@ class ImageRoutesMixin:
                 {"success": False, "error": _safe_error(exc)}, status=500
             )
 
-    def _plan_thumbnails(self, output_path, thumbnails_dir):
+    def _plan_thumbnails(self, output_path, thumbnails_dir, rebuild=False, stop=None):
         """Decide which media files need a thumbnail (blocking walk).
 
-        Returns ``{"targets": [(src, dst, is_video)], "skipped": n, "remaining": n}``.
-        Files whose thumbnail is already newer than the source are counted as
-        skipped and never queued, so a large library that is mostly done only
-        pays for the files that changed. The walk is bounded like the gallery
-        listing (MAX_SCAN_DEPTH, MAX_GALLERY_FILES, no symlinks) and at most
-        MAX_THUMBNAIL_FILES pending files are queued per run; ``remaining``
-        counts the pending files beyond that cap.
+        Returns ``{"targets": [(src, dst, is_video)], "skipped": n,
+        "remaining": n, "fresh": n, "fresh_files": [src]}``. Files whose
+        thumbnail is already newer than the source are ``fresh``. Without
+        ``rebuild`` they are counted as skipped and never queued, so a large
+        library that is mostly done only pays for the files that changed;
+        ``fresh_files`` lists the first MAX_FRESH_LISTED of them. With
+        ``rebuild`` every file is a target (nothing is skipped). The walk is
+        bounded like the gallery listing (MAX_SCAN_DEPTH, MAX_GALLERY_FILES,
+        no symlinks) and at most MAX_THUMBNAIL_FILES targets are queued per
+        run; ``remaining`` counts the targets beyond that cap. ``stop()`` true
+        ends the walk before the next file.
         """
         output_path = Path(output_path)
         targets = []
         skipped = 0
         remaining = 0
+        fresh = 0
+        fresh_files = []
         for src in _iter_media_files(output_path, MAX_GALLERY_FILES):
+            if stop is not None and stop():
+                break
             suffix = src.suffix.lower()
             is_video = suffix in VIDEO_EXTENSIONS
             rel_no_ext = src.relative_to(output_path).with_suffix("")
@@ -1883,8 +2055,12 @@ class ImageRoutesMixin:
                 self.logger.warning(f"Skipping thumbnail outside safe dir: {src.name}")
                 continue
             if _thumbnail_is_fresh(src, dst):
-                skipped += 1
-                continue
+                fresh += 1
+                if not rebuild:
+                    skipped += 1
+                    if len(fresh_files) < MAX_FRESH_LISTED:
+                        fresh_files.append(src)
+                    continue
             if len(targets) >= MAX_THUMBNAIL_FILES:
                 remaining += 1
                 continue
@@ -1894,189 +2070,107 @@ class ImageRoutesMixin:
                 f"Thumbnail run capped at {MAX_THUMBNAIL_FILES} new files; "
                 f"{remaining} more will be generated on the next run"
             )
-        return {"targets": targets, "skipped": skipped, "remaining": remaining}
+        return {
+            "targets": targets,
+            "skipped": skipped,
+            "remaining": remaining,
+            "fresh": fresh,
+            "fresh_files": fresh_files,
+        }
 
     def _thumbnail_targets(self, output_path, thumbnails_dir):
         """Pending (src, dst, is_video) triples; see _plan_thumbnails."""
         return self._plan_thumbnails(output_path, thumbnails_dir)["targets"]
 
-    def _generate_batch(self, targets, thumbnail_size):
+    def _generate_batch(self, targets, thumbnail_size, **options):
         """Create thumbnails for ``targets`` on the configured worker threads.
 
-        Blocking; returns one result dict per target, in order.
+        Blocking; returns one result dict per target, in order. ``options``
+        go to ``_generate_one`` (``encoding``, ``rebuild``, ``stop``).
         """
 
         def one(target):
             src, dst, is_video = target
-            return self._generate_one(src, dst, thumbnail_size, is_video)
+            return self._generate_one(src, dst, thumbnail_size, is_video, **options)
 
         return map_parallel(one, targets, _worker_threads())
 
-    def _generate_one(self, src, dst, thumbnail_size, is_video):
-        """Create one thumbnail (blocking). Never raises; returns a result dict."""
+    def _generate_one(
+        self,
+        src,
+        dst,
+        thumbnail_size,
+        is_video,
+        encoding=None,
+        rebuild=False,
+        stop=None,
+        guard=None,
+        track=None,
+    ):
+        """Create one thumbnail (blocking). Never raises; returns a result dict.
+
+        ``stop()`` true before the file starts gives ``action: "cancelled"``
+        and touches nothing (a job's file boundary). ``rebuild`` writes a
+        fresh preview again and never replaces an existing video preview with
+        the placeholder. Every write is atomic, so a failure keeps the old
+        preview. ``guard()`` false just before the rename gives ``action:
+        "error"`` with ``held: True`` and writes nothing; ``track(process)``
+        is handed the ffmpeg process (both for automatic previews, #249). An
+        error result carries ``decode: True`` when the file itself cannot be
+        decoded (an I/O error does not).
+        """
         result = {
             "file": src.name,
             "dir": src.parent.name,
             "type": "video" if is_video else "image",
         }
         try:
+            if stop is not None and stop():
+                return {**result, "action": "cancelled"}
             if not src.is_file():
                 return {**result, "action": "error", "error": "File no longer exists"}
-            if _thumbnail_is_fresh(src, dst):
+            if not rebuild and _thumbnail_is_fresh(src, dst):
                 return {**result, "action": "skipped"}
             dst.parent.mkdir(parents=True, exist_ok=True)
             if is_video:
-                if not self._generate_video_thumbnail(src, dst, thumbnail_size):
-                    return {**result, "action": "error", "error": "Video decode failed"}
+                made = self._generate_video_thumbnail(
+                    src,
+                    dst,
+                    thumbnail_size,
+                    encoding,
+                    keep_existing=rebuild,
+                    guard=guard,
+                    track=track,
+                )
+                if not made:
+                    return {
+                        **result,
+                        "action": "error",
+                        "error": "Video decode failed",
+                        "decode": True,
+                    }
             else:
-                _write_image_thumbnail(src, dst, thumbnail_size)
+                _write_image_thumbnail(src, dst, thumbnail_size, encoding, guard)
             return {**result, "action": "generated"}
+        except PreviewWriteHeld:
+            return {**result, "action": "error", "error": PREVIEW_HELD, "held": True}
         except UnidentifiedImageError:
-            return {**result, "action": "error", "error": "Not a valid image file"}
+            return {
+                **result,
+                "action": "error",
+                "error": "Not a valid image file",
+                "decode": True,
+            }
+        except PermissionError as e:
+            message = _preview_write_error(e)
+            self.logger.warning(f"Thumbnail failed for {src.name}: {message}")
+            return {**result, "action": "error", "error": message}
         except Exception as e:
             self.logger.warning(f"Thumbnail failed for {src.name}", exc_info=True)
-            return {**result, "action": "error", "error": _safe_error(e)}
-
-    async def generate_thumbnails_with_progress(self, request):
-        """Generate thumbnails with Server-Sent Events progress updates (POST).
-
-        ``quality`` is read from the query string so a POST-capable SSE
-        client can keep using the same URL shape. Every blocking step (scan,
-        PIL/ffmpeg) runs in the executor; the event loop only writes SSE
-        frames between files.
-        """
-        thumbnail_size = self._thumbnail_size(request.query.get("quality", "medium"))
-        response = web.StreamResponse(
-            status=200,
-            headers={
-                "Content-Type": "text/event-stream",
-                "Cache-Control": "no-cache",
-                "Connection": "keep-alive",
-            },
-        )
-        busy = self._jobs.blocker(THUMBNAILS_JOB_KIND, admission="guarded")
-        if busy is not None:
-            # The gallery reads this stream: refuse with its error event.
-            await response.prepare(request)
-            message = f"{busy.title} already running"
-            await self._emit_sse(
-                response, "error", {"error": message, "message": message}
-            )
-            return response
-        async with self._jobs.hold(
-            THUMBNAILS_JOB_KIND, THUMBNAIL_GENERATION_TITLE, admission="guarded"
-        ) as job:
-            await response.prepare(request)
-            if not await self._thumbnail_progress_stream(response, thumbnail_size):
-                job.finish("failed")  # an error event ended the stream
-        return response
-
-    async def _thumbnail_progress_stream(self, response, thumbnail_size) -> bool:
-        """Stream one generation run; False when it ended with an error event."""
-        try:
-            output_dirs = self._get_all_output_dirs()
-            if not output_dirs:
-                await self._emit_sse(
-                    response, "error", {"error": "ComfyUI output directory not found"}
-                )
-                return False
-
-            await self._emit_sse(
-                response,
-                "status",
-                {"phase": "scanning", "message": "Scanning for images and videos..."},
-            )
-            plan = await self._run_in_executor(
-                self._plan_gallery_thumbnails, output_dirs
-            )
-            await self._stream_thumbnails(
-                response,
-                plan["targets"],
-                thumbnail_size,
-                skipped=plan["skipped"],
-                remaining=plan["remaining"],
-            )
-            self.invalidate_gallery_cache()
-            return True
-        except Exception:
-            self.logger.exception("Thumbnail generation failed")
-            await self._emit_sse(
-                response,
-                "error",
-                {
-                    "error": "An internal error occurred",
-                    "message": "Thumbnail generation failed. Check server logs.",
-                },
-            )
-            return False
-
-    async def _stream_thumbnails(
-        self, response, targets, thumbnail_size, skipped=0, remaining=0
-    ):
-        """Process pending *targets* in worker-thread batches, emitting SSE per file.
-
-        ``skipped`` files already had fresh thumbnails and ``remaining`` were
-        left for the next run; both only feed the start/complete payloads.
-        """
-        total = len(targets)
-        video_count = sum(1 for _, _, is_video in targets if is_video)
-        message = (
-            f"Found {total - video_count} images and {video_count} videos to process"
-        )
-        if skipped:
-            message += f" ({skipped} already have thumbnails)"
-        connected = await self._emit_sse(
-            response,
-            "start",
-            {
-                "total_images": total,
-                "skipped": skipped,
-                "remaining": remaining,
-                "phase": "processing",
-                "image_count": total - video_count,
-                "video_count": video_count,
-                "message": message,
-            },
-        )
-        if not connected:
-            return
-        stats = _new_thumbnail_stats()
-        stats["skipped"] = skipped
-        start = _time.monotonic()
-        batch_size = _thumbnail_batch_size(_worker_threads())
-        index = 0
-        for batch_start in range(0, total, batch_size):
-            batch = targets[batch_start : batch_start + batch_size]
-            results = await self._run_in_executor(
-                self._generate_batch, batch, thumbnail_size
-            )
-            for result in results:
-                index += 1
-                _record_thumbnail_result(stats, result)
-                if result["action"] == "error":
-                    await self._emit_sse(
-                        response,
-                        "file_error",
-                        {"file": result["file"], "error": result["error"]},
-                    )
-                connected = await self._emit_sse(
-                    response,
-                    "progress",
-                    _thumbnail_progress_payload(index, total, stats, result, start),
-                )
-                if not connected:
-                    self.logger.info("Thumbnail client disconnected; stopping early")
-                    return
-            await asyncio.sleep(0)  # let other requests run between batches
-        payload = _thumbnail_complete_payload(
-            total + skipped + remaining,
-            stats,
-            start,
-            remaining=remaining,
-            processed=total,
-        )
-        await self._emit_sse(response, "complete", payload)
-        self.logger.info(payload["message"])
+            failed = {**result, "action": "error", "error": _safe_error(e)}
+            if _is_decode_error(e):
+                failed["decode"] = True
+            return failed
 
     async def _emit_sse(self, response, event, data) -> bool:
         """Write one SSE frame. Returns False once the client has gone away."""
@@ -2088,131 +2182,155 @@ class ImageRoutesMixin:
             self.logger.warning(f"Failed to send SSE message: {e}")
             return False
 
-    def _generate_video_thumbnail(self, video_path, thumbnail_path, thumbnail_size):
-        """Generate thumbnail from video file. Returns True if successful."""
+    def _generate_video_thumbnail(
+        self,
+        video_path,
+        thumbnail_path,
+        thumbnail_size,
+        encoding=None,
+        keep_existing=False,
+        *,
+        guard=None,
+        track=None,
+    ):
+        """Generate thumbnail from video file. Returns True if successful.
+
+        Tries OpenCV, then ffmpeg, then writes the placeholder frame: a
+        decoder that cannot open or read the clip (an AV1 file OpenCV was
+        built without) falls through to the next one (#249 I1). ``encoding``
+        is one of THUMBNAIL_ENCODINGS (medium when None); it encodes the
+        OpenCV frame, ffmpeg's frame (as ``-q:v``) and the placeholder alike.
+        Both decoders fit the frame inside ``thumbnail_size`` keeping its
+        aspect ratio. With ``keep_existing`` an existing preview is kept, and
+        the run fails, when no frame could be decoded, rather than replaced
+        by the placeholder. A PermissionError while replacing the preview
+        propagates (``_generate_one`` names it), as does PreviewWriteHeld
+        when ``guard()`` refuses the rename. ``track(process)`` is handed the
+        ffmpeg process so a shutdown can kill it.
+        """
+        encoding = encoding or DEFAULT_THUMBNAIL_ENCODING
         try:
-            # Try using OpenCV first (most reliable)
-            try:
-                import cv2
-
-                cap = cv2.VideoCapture(str(video_path))
-                if not cap.isOpened():
-                    return False
-
-                frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
-                target_frame = max(1, int(frame_count * 0.1))
-                cap.set(cv2.CAP_PROP_POS_FRAMES, target_frame)
-
-                ret, frame = cap.read()
-                cap.release()
-
-                if not ret or frame is None:
-                    return False
-
-                frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
-
-                img = Image.fromarray(frame_rgb)
-                img.thumbnail(thumbnail_size, Image.Resampling.LANCZOS)
-                _atomic_save(img, thumbnail_path, "JPEG", quality=85, optimize=True)
+            frame = self._opencv_frame(video_path, thumbnail_size)
+            if frame is not None:
+                _atomic_save(frame, thumbnail_path, "JPEG", guard=guard, **encoding)
                 self.logger.debug(
                     f"Generated video thumbnail using OpenCV: {thumbnail_path}"
                 )
                 return True
-
-            except ImportError:
-                pass
-
-            # Fallback to ffmpeg
+            if self._ffmpeg_frame(
+                video_path, thumbnail_path, thumbnail_size, encoding, guard, track
+            ):
+                return True
+            if keep_existing and thumbnail_path.exists():
+                self.logger.warning(
+                    f"No frame decoded for {video_path.name}; keeping its preview"
+                )
+                return False
+            # Last resort: the placeholder frame.
             try:
-                import subprocess
-
-                tmp_path = _temp_sibling(thumbnail_path)
-                cmd = [
-                    "ffmpeg",
-                    "-i",
-                    str(video_path),
-                    "-ss",
-                    "00:00:01",
-                    "-vframes",
-                    "1",
-                    "-s",
-                    f"{thumbnail_size[0]}x{thumbnail_size[1]}",
-                    "-y",
-                    str(tmp_path),
-                ]
-
-                result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
-                if result.returncode == 0 and tmp_path.exists():
-                    os.replace(tmp_path, thumbnail_path)
-                    self.logger.debug(
-                        f"Generated video thumbnail using ffmpeg: {thumbnail_path}"
-                    )
-                    return True
-                else:
-                    self.logger.warning(
-                        f"ffmpeg failed for {video_path}: {result.stderr}"
-                    )
-
-            except (ImportError, subprocess.TimeoutExpired, FileNotFoundError):
-                pass
-            finally:
-                try:
-                    tmp_path.unlink()
-                except (NameError, OSError):
-                    pass
-
-            # Last resort: create a placeholder thumbnail
-            try:
-                from PIL import ImageDraw, ImageFont
-
-                img = Image.new("RGB", thumbnail_size, color=(50, 50, 50))
-                draw = ImageDraw.Draw(img)
-
-                center_x, center_y = thumbnail_size[0] // 2, thumbnail_size[1] // 2
-                triangle_size = min(thumbnail_size) // 4
-
-                points = [
-                    (center_x - triangle_size // 2, center_y - triangle_size // 2),
-                    (center_x - triangle_size // 2, center_y + triangle_size // 2),
-                    (center_x + triangle_size // 2, center_y),
-                ]
-                draw.polygon(points, fill=(255, 255, 255))
-
-                try:
-                    font = ImageFont.load_default()
-                    text = "VIDEO"
-                    bbox = draw.textbbox((0, 0), text, font=font)
-                    text_width = bbox[2] - bbox[0]
-                    draw.text(
-                        (
-                            center_x - text_width // 2,
-                            center_y + triangle_size // 2 + 10,
-                        ),
-                        text,
-                        fill=(255, 255, 255),
-                        font=font,
-                    )
-                except (OSError, AttributeError):
-                    pass
-
-                _atomic_save(img, thumbnail_path, "JPEG", quality=85)
+                img = _video_placeholder_image(thumbnail_size)
+                _atomic_save(img, thumbnail_path, "JPEG", guard=guard, **encoding)
                 self.logger.debug(
                     f"Generated placeholder video thumbnail: {thumbnail_path}"
                 )
                 return True
-
+            except (PermissionError, PreviewWriteHeld):
+                raise
             except Exception as e:
                 self.logger.warning(
                     f"Failed to create placeholder thumbnail for {video_path}: {e}"
                 )
                 return False
-
+        except (PermissionError, PreviewWriteHeld):
+            raise
         except Exception as e:
             self.logger.error(
                 f"Video thumbnail generation failed for {video_path}: {e}"
             )
             return False
 
+    def _opencv_frame(self, video_path, thumbnail_size):
+        """A frame about 10% in, fitted to *thumbnail_size*, or None (no
+        OpenCV, or it could not open or read the clip)."""
+        try:
+            import cv2
+        except ImportError:
+            return None
+        try:
+            cap = cv2.VideoCapture(str(video_path))
+            try:
+                if not cap.isOpened():
+                    return None
+                frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+                cap.set(cv2.CAP_PROP_POS_FRAMES, max(1, int(frame_count * 0.1)))
+                ret, frame = cap.read()
+            finally:
+                cap.release()
+            if not ret or frame is None:
+                return None
+            img = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
+            img.thumbnail(thumbnail_size, Image.Resampling.LANCZOS)
+            return img
+        except Exception as e:
+            self.logger.debug(f"OpenCV could not read {video_path}: {e}")
+            return None
+
+    def _ffmpeg_frame(
+        self, video_path, thumbnail_path, thumbnail_size, encoding, guard, track
+    ):
+        """Write ffmpeg's frame at 1 s into *thumbnail_path*; True on success."""
+        import subprocess  # nosec B404 - fixed argv, no shell
+
+        tmp_path = _temp_sibling(thumbnail_path)
+        cmd = [
+            "ffmpeg",
+            "-i",
+            str(video_path),
+            "-ss",
+            "00:00:01",
+            "-vframes",
+            "1",
+            "-vf",
+            f"scale={thumbnail_size[0]}:{thumbnail_size[1]}"
+            ":force_original_aspect_ratio=decrease",
+            "-q:v",
+            _ffmpeg_qscale(encoding),
+            "-y",
+            str(tmp_path),
+        ]
+        try:
+            process = subprocess.Popen(  # nosec B603 B607 - fixed argv
+                cmd,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+                text=True,
+            )
+            if track is not None:
+                track(process)
+            try:
+                _out, stderr = process.communicate(timeout=30)
+            except subprocess.TimeoutExpired:
+                process.kill()
+                process.communicate()
+                return False
+            if process.returncode == 0 and tmp_path.exists():
+                _replace_unless_held(tmp_path, thumbnail_path, guard)
+                self.logger.debug(
+                    f"Generated video thumbnail using ffmpeg: {thumbnail_path}"
+                )
+                return True
+            self.logger.warning(f"ffmpeg failed for {video_path}: {stderr}")
+            return False
+        except (FileNotFoundError, OSError) as e:
+            if isinstance(e, PermissionError) and tmp_path.exists():
+                raise
+            return False
+        finally:
+            try:
+                tmp_path.unlink()
+            except OSError:
+                pass
+
     async def clear_thumbnails(self, request: web.Request) -> web.Response:
         """Clear generated thumbnails across gallery roots, preserving originals.
 
@@ -2222,7 +2340,7 @@ class ImageRoutesMixin:
         """
         return await self._run_held(
             THUMBNAILS_JOB_KIND,
-            "Thumbnail clear (gallery page)",
+            THUMBNAIL_CLEAR_TITLE,
             "guarded",
             self._clear_thumbnails,
         )

py/api/job_routes.py

@@ -1,7 +1,8 @@
 """Session job routes (Activity): list, inspect, follow and cancel jobs.
 
 The registry itself is ``py/api/jobs.py``. The scan job's runner is in
-``py/api/admin.py``, the image tagging job's in ``py/api/image_tag_jobs.py``.
+``py/api/admin.py``, the image tagging job's in ``py/api/image_tag_jobs.py``
+and the thumbnail job's in ``py/api/thumbnail_jobs.py``.
 Every response leaves out absolute server paths: job rows already carry
 root-relative paths and public root labels.
 """
@@ -129,6 +130,10 @@ class JobRoutesMixin:
         async def start_image_tag_job_route(request):
             return await self.start_image_tag_job(request)
 
+        @routes.post("/prompt_manager/jobs/thumbnails")
+        async def start_thumbnail_job_route(request):
+            return await self.start_thumbnail_job(request)
+
         @routes.get("/prompt_manager/jobs/{job_id}")
         async def job_detail_route(request):
             return await self.job_detail(request)

py/api/jobs.py

@@ -19,7 +19,11 @@ ones. One job of a kind runs at a time, and each job has an admission class
 - ``guarded`` must not overlap record changes but may overlap other guarded
   work (thumbnail generation and clearing, later backup). It is refused while
   a ``records`` job runs.
-- ``read`` only reads; it is refused only by a running job of its own kind.
+- ``read`` only reads; it is refused only by a running job of its own kind
+  or an ``exclusive`` one.
+- ``exclusive`` runs alone (Optimize's VACUUM, which rewrites the whole
+  database under the write lock): it is refused while any job runs, and every
+  job is refused while it runs.
 
 A short legacy-page operation takes a ``hold``: a running, non-cancellable
 job that lasts as long as the request, so the Activity page lists it and a
@@ -72,12 +76,17 @@ SCAN_GROUPS = {
 IMAGE_TAG_JOB_KIND = "image-tags"
 IMAGE_TAG_GROUP_NAMES = ("tagged", "unchanged", "skipped", "errors")
 
-ADMISSIONS = ("read", "guarded", "records")
+# Thumbnail generation (#249 System S5): previews written, already fresh, failed.
+THUMBNAIL_GROUP_NAMES = ("generated", "skipped", "errors")
+
+ADMISSIONS = ("read", "guarded", "records", "exclusive")
 # What each admission class is refused beside (besides its own kind).
+# ``exclusive`` (Optimize's VACUUM) runs alone: beside no job of any class.
 _REFUSED_BESIDE = {
-    "read": frozenset(),
-    "guarded": frozenset({"records"}),
-    "records": frozenset({"records", "guarded"}),
+    "read": frozenset({"exclusive"}),
+    "guarded": frozenset({"records", "exclusive"}),
+    "records": frozenset({"records", "guarded", "exclusive"}),
+    "exclusive": frozenset(ADMISSIONS),
 }
 
 # Kinds of the legacy-page operations that hold the registry (D80; titles in
@@ -88,6 +97,11 @@ DUPLICATE_CLEANUP_JOB_KIND = "duplicate_cleanup"
 THUMBNAILS_JOB_KIND = "thumbnails"
 LORA_IMPORT_JOB_KIND = "lora_import"
 AUTOTAG_BATCH_JOB_KIND = "autotag"
+# Duplicate image files (#249 System S6, py/api/duplicate_files.py): the
+# find is a cancellable ``read`` job; the reviewed removal is ``records``.
+DUPLICATES_JOB_KIND = "duplicates"
+DUPLICATE_GROUP_NAMES = ("groups_found",)
+DUPLICATE_CLEANUP_GROUP_NAMES = ("removed", "kept", "errors")
 # The System health check (py/api/system.py): a read-only ``read`` job with
 # no file outcomes, so it may run beside a scan.
 HEALTH_JOB_KIND = "health"
@@ -247,6 +261,32 @@ class Job:
             )
         self.log_line(log_line)
 
+    def record_counted(self, group, count, rows=()):
+        """Count ``count`` outcomes in ``group`` at once, listing ``rows``.
+
+        For outcomes that need no log line each (a thumbnail job's already
+        fresh previews): the counter is exact, ``rows`` (``record``'s row
+        shape) are listed up to MAX_OUTCOME_ROWS, and nothing is logged.
+        """
+        if group not in self.counters:
+            raise ValueError(f"Unknown outcome group {group!r} for {self.kind}")
+        self.counters[group] += max(0, int(count))
+        listed = self.outcomes[group]
+        for row in rows:
+            if len(listed) >= MAX_OUTCOME_ROWS:
+                break
+            index = row.get("root_index")
+            if isinstance(index, bool) or not isinstance(index, int) or index < 0:
+                index = None
+            listed.append(
+                {
+                    "path": _relative_path(row.get("path")),
+                    "root": str(row.get("root") or ""),
+                    "root_index": index,
+                    "detail": str(row.get("detail") or ""),
+                }
+            )
+
     def finish(self, state="finished"):
         """End the job in ``state`` and release every subscriber.
 
@@ -263,18 +303,24 @@ class Job:
         for queue in list(self._subscribers):
             queue.put_nowait(None)
 
-    async def events(self):
+    async def events(self, lead=None):
         """Yield the latest event, then each later one, until the job finishes.
 
+        ``lead``, when given, is yielded first, after the subscription is in
+        place, so nothing published while the consumer sends it is missed.
         Consume it under ``contextlib.aclosing`` so a consumer that stops early
         (a closed socket) drops its queue at once instead of at garbage collection.
         """
         queue = asyncio.Queue()
         self._subscribers.add(queue)
+        # As of the subscription: a later event or the end reaches the queue.
+        snapshot, ended = self.last_event, self.finished
         try:
-            if self.last_event is not None:
-                yield self.last_event
-            if self.finished:
+            if lead is not None:
+                yield lead
+            if snapshot is not None:
+                yield snapshot
+            if ended:
                 return
             while True:
                 payload = await queue.get()
@@ -425,11 +471,14 @@ class JobRegistry:
         self._jobs[job.id] = job
         return job
 
-    def blocker(self, kind, *, admission=None, record_changing=None):
+    def blocker(self, kind, *, admission=None, record_changing=None, compatible=()):
         """The running job a new ``kind`` job would be refused for, or None.
 
         The same kind refuses first; then the oldest running job whose class
-        the new job's admission class may not run beside.
+        the new job's admission class may not run beside. Running jobs of a
+        kind in ``compatible`` never refuse it: the caller knows that work
+        cannot conflict with its own (the monitor's Needs review recording
+        beside thumbnail work, which writes previews and no records).
         """
         same_kind = self.active(kind)
         if same_kind is not None:
@@ -439,7 +488,7 @@ class JobRegistry:
         running = [job for job in tuple(self._jobs.values()) if job.running]
         running.sort(key=lambda job: job._started_seq)
         for job in running:
-            if job.admission in refused:
+            if job.admission in refused and job.kind not in compatible:
                 return job
         return None
 
@@ -466,6 +515,12 @@ class JobRegistry:
     def get(self, job_id):
         return self._jobs.get(job_id)
 
+    def running(self):
+        """The running jobs, oldest first (a snapshot, safe off the event loop)."""
+        jobs = [job for job in tuple(self._jobs.values()) if job.running]
+        jobs.sort(key=lambda job: job._started_seq)
+        return jobs
+
     def active(self, kind):
         """The running job of ``kind``, or None (safe off the event loop)."""
         for job in tuple(self._jobs.values()):

py/api/maintenance.py

@@ -0,0 +1,815 @@
+"""Maintenance routes: reviewed, read-only plans (#249 System S6).
+
+``POST /prompt_manager/maintenance/preview {operation, options:{}}`` reviews
+one operation (``duplicate_prompts``, ``optimize``, ``unused_components`` or
+``missing_links``; ``database/maintenance.py``) off the event loop and
+answers a ``Plan``: ``{success, token, operation, expires_at, scope:{count,
+label}, protected:{count, reasons}, backup:{policy, destination_label},
+media:{deletes_files, label}, details:[≤50 {label, detail}]}``. The preview
+writes nothing. The plan's fingerprint, its targets and the sources it judged
+stay in the server's ``PlanStore`` under the token, bound to the database the
+preview read (a restore that replaces it invalidates the plan).
+
+The operation enum is strict, ``options`` must be an empty object and the
+body is capped at ``PREVIEW_BODY_MAX`` bytes. Errors are ``{success:false,
+code, error}`` with ``invalid-request`` (400), ``too-large`` (413: the body,
+or a plan over the store's budget), ``storage-changed`` (409: the database
+was replaced while the preview read it) or a null code (500); none carries a
+server path or exception text.
+
+``POST /prompt_manager/maintenance/apply {token}`` applies a reviewed plan as
+an Activity job (202 ``{job}``): kind ``maintenance``, titled after the
+operation, admission ``records`` (refused while a scan, restore, LoRA import
+or other record-changing job runs, and refusing them meanwhile), not
+cancellable, with outcome groups ``changed``, ``protected`` and ``errors``.
+The token is consumed when the job starts (on the same loop turn, before its
+runner runs); a refused start keeps it. The runner:
+
+1. takes a safety backup when the plan was reviewed with the policy on or the
+   policy (``maintenance.safety_backup``, default on) is on now: SQLite's
+   backup API into a temp file beside the database, ``verify_database_file``,
+   fsync, ``0o600`` on POSIX, then ``os.replace`` onto a unique
+   ``<db>.backup_<YYYYmmdd_HHMMSS_ffffff>[_n]`` name. Any failure removes
+   what it wrote and ends the job ``backup-failed`` with nothing changed;
+2. applies the plan, which first rechecks the reviewed fingerprint inside
+   ``BEGIN IMMEDIATE`` (``plan-changed`` writes nothing);
+3. counts the changed and protected records, and prunes old safety backups.
+
+The job log names the backup by its file name only, and the job says "No
+safety backup" when none was taken. Apply errors: ``invalid-request`` (400),
+``plan-expired`` (410: unknown, expired or forgotten after a restart),
+``plan-used`` (409, with the job that applied it), ``storage-changed`` (409:
+a restore replaced the database since the review) and ``busy`` (409).
+"""
+
+import datetime
+import json
+import os
+import shutil
+import sqlite3
+import tempfile
+from contextlib import closing
+
+from aiohttp import web
+
+from .admin import _prune_safety_backups, _scan_location, _scan_roots
+from .job_routes import busy_response
+from .jobs import MAINTENANCE_JOB_KIND, JobBusy
+from .maintenance_plans import PlanExpired, PlanUsed, ScopeTooLarge
+
+try:
+    from ...database.workflow_storage import PRE_MIGRATION_LABEL, pre_migration_backups
+    from ...database.health import readonly_uri
+    from ...database.maintenance import MAINTENANCE_OPERATIONS, PlanChanged
+except ImportError:
+    from database.workflow_storage import PRE_MIGRATION_LABEL, pre_migration_backups
+    from database.health import readonly_uri
+    from database.maintenance import MAINTENANCE_OPERATIONS, PlanChanged
+
+PREVIEW_BODY_MAX = 4096
+# Duplicate-file previews carry the reviewed keepers (py/api/duplicate_files.py).
+DUPLICATE_PREVIEW_BODY_MAX = 1024 * 1024
+DUPLICATE_FILES = "duplicate_files"
+WORKFLOW_STORAGE = "workflow_storage"  # applied by py/api/workflow_storage.py
+PREVIEW_OPERATIONS = MAINTENANCE_OPERATIONS + (DUPLICATE_FILES,)
+# Room left over beyond what a step writes, so a disk is never filled to 0.
+DISK_MARGIN_BYTES = 16 * 2**20
+# The concept's Maintenance row titles, which name each operation's job.
+MAINTENANCE_TITLES = {
+    "duplicate_prompts": "Duplicate prompt text",
+    "optimize": "Optimize database",
+    "unused_components": "Unused prompt components",
+    "missing_links": "Missing-file links",
+    "workflow_storage": "Store each workflow once",
+}
+_OPERATION_ERROR = "operation must be one of: " + ", ".join(PREVIEW_OPERATIONS)
+MAINTENANCE_GROUPS = ("changed", "protected", "errors")
+MAX_TOKEN_LENGTH = 128
+BACKUP_FAILED = (
+    "The safety backup could not be taken and verified; nothing was changed."
+)
+PLAN_CHANGED = (
+    "The library changed since the review; nothing was changed. "
+    "Review the plan again."
+)
+APPLY_FAILED = "Maintenance failed; nothing was changed."
+NOTHING_TO_DO = "Every image record already stores its workflow once; nothing to do."
+NO_BACKUP = "No safety backup"
+_NAME_ATTEMPTS = 1000
+
+
+class SafetyBackupFailed(Exception):
+    """The safety backup could not be written, verified or put in place.
+
+    ``reason`` is the verify reason when the copy did not verify.
+    """
+
+    def __init__(self, message, reason=None):
+        super().__init__(message)
+        self.reason = reason
+
+
+def backup_failed_message(error):
+    """The job line for a failed backup, naming a damaged workflow document."""
+    reason = getattr(error, "reason", None) or ""
+    if reason.startswith("Workflow document "):
+        return f"{BACKUP_FAILED} {reason}"
+    return BACKUP_FAILED
+
+
+def _now():
+    return datetime.datetime.now()
+
+
+def _free_bytes(path):
+    return shutil.disk_usage(path).free
+
+
+def _device(path):
+    try:
+        return os.stat(path).st_dev
+    except OSError:
+        return path
+
+
+def sqlite_temp_dir(environ=None, *, windows=None, usable=None):
+    """The folder SQLite builds its temporary files in (VACUUM's copy).
+
+    Mirrors SQLite's own search, which differs from ``tempfile``'s. On Unix
+    (``unixTempFileDir``): ``SQLITE_TMPDIR``, ``TMPDIR``, then ``/var/tmp``,
+    ``/usr/tmp`` and ``/tmp``, the first that is a writable directory, else
+    the current directory. On Windows (``GetTempPathW``): ``TMP``, ``TEMP``,
+    then ``USERPROFILE``, else the current directory.
+    """
+    environ = os.environ if environ is None else environ
+    if windows is None:
+        windows = os.name == "nt"
+    if windows:
+        for name in ("TMP", "TEMP", "USERPROFILE"):
+            if environ.get(name):
+                return environ[name]
+        return os.path.abspath(".")
+    if usable is None:
+
+        def usable(path):
+            return os.path.isdir(path) and os.access(path, os.W_OK | os.X_OK)
+
+    candidates = [environ.get("SQLITE_TMPDIR"), environ.get("TMPDIR")]
+    candidates += ["/var/tmp", "/usr/tmp", "/tmp"]
+    for candidate in candidates:
+        if candidate and usable(candidate):
+            return candidate
+    return os.path.abspath(".")
+
+
+def database_bytes(db_path):
+    """The size a safety backup of the database will be (blocking).
+
+    SQLite's backup API copies every page, committed WAL content included, so
+    the copy is ``page_count * page_size`` read on one read-only snapshot.
+    """
+    conn = sqlite3.connect(readonly_uri(db_path), uri=True)
+    try:
+        page_size = conn.execute("PRAGMA page_size").fetchone()[0]
+        return page_size * conn.execute("PRAGMA page_count").fetchone()[0]
+    finally:
+        conn.close()
+
+
+def _megabytes(size):
+    return f"{size / 2**20:,.1f} MB"
+
+
+def disk_plan(db_path, *, backup, vacuum, vacuum_bytes=None):
+    """Whether the disk has room for an apply's writes (blocking).
+
+    ``backup``: the safety backup, one database-sized file beside the
+    database. ``vacuum``: VACUUM writes the rebuilt database through the WAL
+    beside it and builds it in a temporary copy in SQLite's temporary folder
+    (``sqlite_temp_dir``; SQLite's documentation: up to twice the database
+    in free space). Needs
+    on one device add up. ``vacuum_bytes``, when given, is what VACUUM
+    copies (the live pages, for Reclaim space) instead of the whole file.
+    Returns ``(public, short)``: ``public`` is
+    ``{free_bytes, required_bytes, sufficient}`` for the database folder,
+    ``short`` None or ``(place, what, required, free)`` of the first place
+    that is short, and ``size`` the database's size (0 when nothing is
+    written).
+    """
+    db_path = os.path.abspath(db_path)
+    folder = os.path.dirname(db_path)
+    size = database_bytes(db_path) if backup or vacuum else 0
+    needs = []  # (place, folder, bytes, what)
+    if backup:
+        needs.append(("database folder", folder, size, "the safety backup"))
+    if vacuum:
+        copy = size if vacuum_bytes is None else vacuum_bytes
+        what = "Optimize" if vacuum_bytes is None else "Reclaim space"
+        needs.append(("database folder", folder, copy, what))
+        needs.append(("temporary folder", sqlite_temp_dir(), copy, what))
+    devices = {}
+    for place, path, needed, what in needs:
+        entry = devices.setdefault(_device(path), {"place": place, "path": path})
+        entry["bytes"] = entry.get("bytes", 0) + needed
+        entry.setdefault("what", [])
+        if what not in entry["what"]:
+            entry["what"].append(what)
+    short = None
+    public = {"free_bytes": _free_bytes(folder), "required_bytes": 0}
+    for device, entry in devices.items():
+        required = entry["bytes"] + DISK_MARGIN_BYTES
+        free = (
+            public["free_bytes"]
+            if entry["path"] == folder
+            else _free_bytes(entry["path"])
+        )
+        if device == _device(folder):
+            public["required_bytes"] = required
+        if free < required and short is None:
+            short = (entry["place"], " and ".join(entry["what"]), required, free)
+    public["sufficient"] = short is None
+    return public, short, size
+
+
+# A restore upload's hard ceiling; below it, free disk space decides.
+RESTORE_MAX_BYTES = 64 * 2**30
+
+
+def size_label(size):
+    """``1.6 GB`` or ``120.0 MB``: a backup's size for a message."""
+    if size >= 2**30:
+        return f"{size / 2**30:,.1f} GB"
+    return _megabytes(size)
+
+
+def restore_room(db_path):
+    """The largest restore upload the disks have room for (blocking).
+
+    The upload is written to a temp file and staged once more in the
+    temporary folder (2x), then copied over the live database beside it,
+    whose current contents are first kept as a safety backup (1x the upload
+    for the WAL, plus the current database). Needs on one device add up,
+    each with ``DISK_MARGIN_BYTES``.
+    """
+    db_path = os.path.abspath(db_path)
+    folder = os.path.dirname(db_path)
+    current = database_bytes(db_path) if os.path.exists(db_path) else 0
+    devices = {}
+    for path, per_upload, fixed in (
+        (tempfile.gettempdir(), 2, 0),
+        (folder, 1, current),
+    ):
+        entry = devices.setdefault(_device(path), {"path": path, "per": 0, "fixed": 0})
+        entry["per"] += per_upload
+        entry["fixed"] += fixed
+    room = min(
+        (_free_bytes(e["path"]) - e["fixed"] - DISK_MARGIN_BYTES) // e["per"]
+        for e in devices.values()
+    )
+    return max(0, room)
+
+
+def restore_space_message(size, room, *, known):
+    """The truthful refusal of a restore the disks have no room for."""
+    if known:
+        return (
+            f"Not enough free disk space to restore a {size_label(size)} backup; "
+            f"there is room for about {size_label(room)}. Nothing was changed."
+        )
+    return (
+        "Not enough free disk space to restore this backup: it is larger than "
+        f"the {size_label(room)} there is room for. Nothing was changed."
+    )
+
+
+def short_space_message(short):
+    """The truthful refusal for ``disk_plan``'s ``short``."""
+    place, what, required, free = short
+    verb = "need" if " and " in what else "needs"
+    return (
+        f"Not enough free disk space: {what} {verb} about {_megabytes(required)} "
+        f"in the {place} and {_megabytes(free)} is free. Nothing was changed."
+    )
+
+
+def _changed_line(operation, count):
+    if operation == "optimize":
+        return "Optimized the database"
+    if operation == "duplicate_prompts":
+        noun = "duplicate prompt" if count == 1 else "duplicate prompts"
+        return f"Merged {count:,} {noun}"
+    if operation == "unused_components":
+        noun = "unused prompt component" if count == 1 else "unused prompt components"
+        return f"Removed {count:,} {noun}"
+    noun = "link to a missing file" if count == 1 else "links to missing files"
+    return f"Removed {count:,} {noun}"
+
+
+def _remove_quietly(*paths):
+    for path in paths:
+        try:
+            if path and os.path.exists(path):
+                os.remove(path)
+        except OSError:
+            pass
+
+
+def _reserve_backup_name(db_path, label="backup"):
+    """Create and return an unused ``<db>.<label>_<stamp>[_n]`` (O_EXCL)."""
+    for candidate in _backup_names(db_path, label):
+        try:
+            fd = os.open(candidate, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600)
+        except FileExistsError:
+            continue
+        os.close(fd)
+        return candidate
+    raise SafetyBackupFailed("No free backup name")
+
+
+def _backup_names(db_path, label="backup"):
+    base = f"{db_path}.{label}_{_now().strftime('%Y%m%d_%H%M%S_%f')}"
+    for attempt in range(_NAME_ATTEMPTS):
+        yield base if attempt == 0 else f"{base}_{attempt}"
+
+
+def _place_backup(temp, db_path, label="backup"):
+    """Give the verified ``temp`` its unique backup name; return that path.
+
+    ``os.link`` is atomic and refuses an existing name, so no empty or half
+    file ever carries a backup name, even across a crash. A filesystem
+    without hard links falls back to reserving the name (``O_EXCL``) and
+    ``os.replace``. The temp file is removed by the caller.
+    """
+    for candidate in _backup_names(db_path, label):
+        try:
+            os.link(temp, candidate)
+        except FileExistsError:
+            continue
+        except (OSError, NotImplementedError):
+            break  # no hard links here
+        else:
+            return candidate
+    else:
+        raise SafetyBackupFailed("No free backup name")
+    final = _reserve_backup_name(db_path, label)
+    try:
+        os.replace(temp, final)
+    except OSError:
+        _remove_quietly(final)
+        raise
+    return final
+
+
+SAFETY_TEMP_PREFIX = ".pm-safety-"
+
+
+def sweep_safety_temps(db_path):
+    """Remove the temp copies an interrupted safety backup left (blocking).
+
+    Called once at startup, before any job can take a backup. Returns the
+    names removed.
+    """
+    folder = os.path.dirname(os.path.abspath(db_path))
+    removed = []
+    try:
+        names = os.listdir(folder)
+    except OSError:
+        return removed
+    for name in names:
+        if not (name.startswith(SAFETY_TEMP_PREFIX) and ".tmp" in name):
+            continue
+        try:
+            os.remove(os.path.join(folder, name))
+            removed.append(name)
+        except OSError:
+            continue
+    return removed
+
+
+def _standalone(path):
+    """Make the copy one self-contained file (rollback journal, not WAL).
+
+    The backup API copies the live database's WAL flag; a WAL file opened
+    read-only later (a verify, a restore preview) would leave ``-wal`` and
+    ``-shm`` sidecars beside it.
+    """
+    try:
+        with closing(sqlite3.connect(path)) as conn:
+            conn.execute("PRAGMA journal_mode=DELETE").fetchone()
+    except sqlite3.Error as exc:
+        raise SafetyBackupFailed("The backup could not be finished") from exc
+
+
+def _fsync(path):
+    with open(path, "rb+") as handle:
+        os.fsync(handle.fileno())
+
+
+def take_safety_backup(model, label="backup"):
+    """Write and verify a safety backup; return its file name (blocking).
+
+    The copy is made in a temp file beside the database and only renamed
+    onto its final, unique name ``<db>.<label>_<stamp>`` once it verified,
+    so a backup file on disk is always a verified one. ``label`` "backup"
+    joins the pruned pool; another label (the workflow storage migration's
+    ``pre-workflow-storage``) is kept until the user removes it. Raises
+    SafetyBackupFailed, leaving no file.
+    """
+    db_path = os.path.abspath(model.db_path)
+    fd, temp = tempfile.mkstemp(
+        dir=os.path.dirname(db_path), prefix=SAFETY_TEMP_PREFIX, suffix=".tmp"
+    )
+    os.close(fd)
+    final = None
+    try:
+        if not model.backup_database(temp):
+            raise SafetyBackupFailed("The backup could not be written")
+        _standalone(temp)
+        ok, reason = model.verify_database_file(temp)
+        if not ok:
+            raise SafetyBackupFailed("The backup did not verify", reason)
+        _fsync(temp)
+        if os.name != "nt":
+            os.chmod(temp, 0o600)
+        final = _place_backup(temp, db_path, label)
+        return os.path.basename(final)
+    except OSError as exc:
+        _remove_quietly(final)
+        raise SafetyBackupFailed("The backup could not be put in place") from exc
+    except SafetyBackupFailed:
+        _remove_quietly(final)
+        raise
+    finally:
+        _remove_quietly(temp, temp + "-journal", temp + "-wal", temp + "-shm")
+
+
+def _failure(message, status, code=None, **extra):
+    return web.json_response(
+        {"success": False, "code": code, "error": message, **extra}, status=status
+    )
+
+
+def _invalid(message):
+    return _failure(message, 400, "invalid-request")
+
+
+def _iso(timestamp):
+    return datetime.datetime.fromtimestamp(timestamp, datetime.timezone.utc).isoformat()
+
+
+class MaintenanceRoutesMixin:
+    """Mixin providing the /prompt_manager/maintenance plan endpoints."""
+
+    def _register_maintenance_routes(self, routes):
+        @routes.post("/prompt_manager/maintenance/preview")
+        async def maintenance_preview_route(request):
+            return await self.preview_maintenance(request)
+
+        @routes.post("/prompt_manager/maintenance/apply")
+        async def maintenance_apply_route(request):
+            return await self.apply_maintenance_plan(request)
+
+    def _database_identity(self, db):
+        """What binds a plan to one database: the swap generation and its file."""
+        path = os.path.normcase(os.path.realpath(db.model.db_path))
+        return (self._db_generation, path)
+
+    @staticmethod
+    def _safety_backup_policy():
+        """``on`` or ``off``: whether an apply takes a safety backup first."""
+        from ..config import PromptManagerConfig
+
+        return "on" if PromptManagerConfig.SAFETY_BACKUP else "off"
+
+    async def _read_maintenance_body(self, request, allowed, cap=None):
+        """``(data, None)`` for a capped JSON object with only ``allowed`` keys.
+
+        ``cap(data)``, when given, is the body's byte limit for what it asks
+        (read up to ``DUPLICATE_PREVIEW_BODY_MAX``); else PREVIEW_BODY_MAX.
+        """
+        limit = PREVIEW_BODY_MAX if cap is None else DUPLICATE_PREVIEW_BODY_MAX
+        data, error = await self._read_json_body(request, max_bytes=limit)
+        if error is not None:
+            if error.status == 413:
+                return None, _failure("Request body too large", 413, "too-large")
+            return None, _invalid("Request body must be a JSON object")
+        if cap is not None:
+            size = len(json.dumps(data, separators=(",", ":")).encode("utf-8"))
+            if size > cap(data):
+                return None, _failure("Request body too large", 413, "too-large")
+        unknown = sorted(set(data) - set(allowed))
+        if unknown:
+            return None, _invalid("Unknown field: " + ", ".join(unknown)[:200])
+        return data, None
+
+    async def preview_maintenance(self, request):
+        data, error = await self._read_maintenance_body(
+            request,
+            ("operation", "options"),
+            cap=lambda body: (
+                DUPLICATE_PREVIEW_BODY_MAX
+                if body.get("operation") == DUPLICATE_FILES
+                else PREVIEW_BODY_MAX
+            ),
+        )
+        if error is not None:
+            return error
+        operation = data.get("operation")
+        options = data.get("options", {})
+        if not isinstance(operation, str) or operation not in PREVIEW_OPERATIONS:
+            return _invalid(_OPERATION_ERROR)
+        if operation == DUPLICATE_FILES:
+            return await self.preview_duplicate_files(options)
+        if not isinstance(options, dict) or options:
+            return _invalid(f"{operation} takes no options")
+        db = self.db
+        identity = self._database_identity(db)
+        try:
+            roots = (
+                await self._run_in_executor(self._health_roots)
+                if operation == "missing_links"
+                else []
+            )
+            draft = await self._run_in_executor(
+                db.preview_maintenance, operation, roots
+            )
+        except Exception:
+            self.logger.exception(f"Maintenance preview of {operation} failed")
+            return _failure("Failed to review maintenance", 500)
+        return await self._plan_response(
+            db,
+            identity,
+            draft.operation,
+            draft.fingerprint,
+            draft.public(),
+            {
+                "targets": draft.targets,
+                "roots": roots,
+                "protected": draft.protected["count"],
+                "rows": draft.scope["count"],
+            },
+        )
+
+    async def _plan_response(
+        self, db, identity, operation, fingerprint, public, payload
+    ):
+        """Issue the reviewed plan's token and answer the ``Plan``.
+
+        Refused with ``storage-changed`` when the database was replaced while
+        it was reviewed. The plan reports the safety backup's expected size
+        and the disk space the apply needs (``disk_plan``).
+        """
+        if identity != self._database_identity(self.db):
+            return _failure(
+                "The database was replaced while it was reviewed. Review again.",
+                409,
+                "storage-changed",
+            )
+        policy = self._safety_backup_policy()
+        destination = "backup"
+        if operation == WORKFLOW_STORAGE:
+            # Always backed up first, whatever the policy (the way back);
+            # one kept pre-migration backup ever, pooled ones after it.
+            policy = "required"
+            kept = await self._run_in_executor(pre_migration_backups, db.model.db_path)
+            destination = "backup" if kept else PRE_MIGRATION_LABEL
+        try:
+            disk, _short, size = await self._run_in_executor(
+                disk_plan,
+                db.model.db_path,
+                backup=policy != "off",
+                vacuum=operation == "optimize",
+            )
+        except (OSError, sqlite3.Error):
+            self.logger.exception("Could not measure the disk space for a plan")
+            return _failure("Failed to review maintenance", 500)
+        store = self._maintenance_plans
+        try:
+            token = store.issue(
+                operation, fingerprint, {**payload, "backup": policy}, identity
+            )
+        except ScopeTooLarge:
+            return _failure(
+                "This plan is too large to review at once.", 413, "too-large"
+            )
+        plan = store.peek(token, operation)
+        name = os.path.basename(db.model.db_path)
+        return web.json_response(
+            {
+                "success": True,
+                "token": token,
+                "expires_at": _iso(plan.expires_at),
+                **public,
+                "backup": {
+                    "policy": policy,
+                    "destination_label": f"{name}.{destination}_<date and time>",
+                    "expected_bytes": size if policy != "off" else None,
+                },
+                "disk": disk,
+            },
+            headers={"Cache-Control": "no-store"},
+        )
+
+    # ── apply ─────────────────────────────────────────────────────────
+
+    async def apply_maintenance_plan(self, request):
+        data, error = await self._read_maintenance_body(request, ("token",))
+        if error is not None:
+            return error
+        token = data.get("token")
+        if not isinstance(token, str) or not 0 < len(token) <= MAX_TOKEN_LENGTH:
+            return _invalid("token must be the plan's token")
+        plan, refused = self._peek_plan(token)
+        if refused is not None:
+            return refused
+        store = self._maintenance_plans
+        db = self.db
+        if plan.db_identity != self._database_identity(db):
+            store.discard(token)
+            return _failure(
+                "The database was replaced since the review. Review again.",
+                409,
+                "storage-changed",
+            )
+        if plan.operation == WORKFLOW_STORAGE and not plan.payload.get("rows"):
+            store.discard(token)  # no job, no backup: nothing to rewrite
+            return _failure(NOTHING_TO_DO, 409, "nothing-to-do")
+        backup = self._backup_promised(plan)
+        try:
+            _disk, short, _size = await self._run_in_executor(
+                disk_plan,
+                db.model.db_path,
+                backup=backup,
+                vacuum=plan.operation == "optimize",
+            )
+        except (OSError, sqlite3.Error):
+            self.logger.exception("Could not measure the disk space for an apply")
+            return _failure(APPLY_FAILED, 500)
+        if short is not None:
+            # Refused before any job starts: nothing fails halfway for space.
+            return _failure(short_space_message(short), 507, "insufficient-space")
+        # The token may have been used, expired or dropped while the disk was
+        # measured: look again with no await before the job starts.
+        plan, refused = self._peek_plan(token)
+        if refused is not None:
+            return refused
+        try:
+            if plan.operation == DUPLICATE_FILES:
+                job = self._start_duplicate_cleanup(db, plan, backup)
+            elif plan.operation == WORKFLOW_STORAGE:
+                job = self._start_workflow_storage(db, backup)
+            else:
+                job = self._jobs.start(
+                    MAINTENANCE_JOB_KIND,
+                    MAINTENANCE_TITLES[plan.operation],
+                    # VACUUM rewrites the whole file under the write lock: it
+                    # runs alone, beside no other job of any class.
+                    admission=(
+                        "exclusive" if plan.operation == "optimize" else "records"
+                    ),
+                    cancellable=False,
+                    scope={"operation": plan.operation},
+                    groups=MAINTENANCE_GROUPS,
+                    runner=lambda job: self._run_maintenance_plan(
+                        job, db, plan, backup
+                    ),
+                )
+        except JobBusy as busy:
+            return busy_response(busy)  # the token stays usable
+        # Same loop turn as the peek and the start: the runner has not run.
+        store.consume(token, plan.operation, job_id=job.id)
+        return web.json_response({"success": True, "job": job.summary()}, status=202)
+
+    def _peek_plan(self, token):
+        """``(plan, None)`` for a usable token, else ``(None, refusal)``."""
+        try:
+            return self._maintenance_plans.peek(token), None
+        except PlanUsed as used:
+            job = self._jobs.get(used.job_id)
+            return None, _failure(
+                "This plan was already applied.",
+                409,
+                "plan-used",
+                job={"id": used.job_id, "title": job.title if job else None},
+            )
+        except PlanExpired:
+            return None, _failure(
+                "This review expired. Review the plan again.", 410, "plan-expired"
+            )
+
+    def _backup_promised(self, plan):
+        """A plan reviewed with the policy on keeps that promise; so does on now.
+
+        Decided once, when the apply starts (the disk check measured it), and
+        handed to the job.
+        """
+        if plan.operation == WORKFLOW_STORAGE:
+            return True  # never without its pre-migration backup
+        return plan.payload["backup"] == "on" or self._safety_backup_policy() == "on"
+
+    async def _safety_backup_step(self, job, db, backup_on):
+        """``(ok, name)``: take the promised safety backup, or report none.
+
+        ``name`` is the backup's file name, or None when no backup was
+        promised. A failed backup ends the job ``backup-failed`` (``ok``
+        False).
+        """
+        if not backup_on:
+            job.log_line("No safety backup (the policy is off)")
+            return True, None
+        job.update(status_line="Taking a safety backup")
+        job.log_line("Taking a safety backup of the database")
+        try:
+            backup = await self._run_in_executor(take_safety_backup, db.model)
+        except SafetyBackupFailed as error:
+            self.logger.error("Safety backup failed", exc_info=True)
+            _fail_job(job, "backup-failed", backup_failed_message(error))
+            return False, None
+        job.log_line(f"Safety backup {backup} verified")
+        return True, backup
+
+    async def _prune_backup_pool(self, job, db):
+        """Keep the one ``<db>.backup_*`` pool (restore's too) at its cap."""
+        removed = await self._run_in_executor(_prune_safety_backups, db.model.db_path)
+        if removed:
+            job.log_line(f"Removed {len(removed)} older safety backups")
+
+    async def _run_maintenance_plan(self, job, db, plan, backup_on):
+        ok, backup = await self._safety_backup_step(job, db, backup_on)
+        if not ok:
+            return
+        try:
+            await self._apply_maintenance_plan(job, db, plan, backup)
+        finally:
+            if backup:
+                await self._prune_backup_pool(job, db)
+
+    async def _apply_maintenance_plan(self, job, db, plan, backup):
+        payload = plan.payload
+        job.update(status_line=f"Applying: {job.title}")
+        try:
+            result = await self._run_in_executor(
+                db.apply_maintenance,
+                plan.operation,
+                plan.fingerprint,
+                payload["targets"],
+                payload["roots"],
+            )
+        except PlanChanged:
+            return _fail_job(job, "plan-changed", PLAN_CHANGED)
+        except Exception:
+            self.logger.exception(f"Maintenance {plan.operation} failed")
+            return _fail_job(job, None, APPLY_FAILED)
+        rows = await self._run_in_executor(
+            self._maintenance_rows, plan.operation, result["rows"]
+        )
+        job.record_counted("changed", result["changed"], rows)
+        job.record_counted("protected", payload["protected"])
+        line = _changed_line(plan.operation, result["changed"])
+        job.log_line(line)
+        job.update(progress=100, status_line=f"{line} · {backup_note(backup)}")
+
+    def _maintenance_rows(self, operation, described):
+        """Outcome rows for the changed records (root-relative paths only)."""
+        if operation == "missing_links":
+            roots = _scan_roots(self._get_all_output_dirs())
+            rows = []
+            for item in described:
+                path, label, index = _scan_location(item["image_path"], roots)
+                rows.append(
+                    {
+                        "path": path,
+                        "root": label,
+                        "root_index": index,
+                        "detail": "Link removed; the file was already missing",
+                    }
+                )
+            return rows
+        if operation == "duplicate_prompts":
+            return [
+                {
+                    "path": "",
+                    "detail": (
+                        f"Prompt {item['prompt_id']} merged into "
+                        f"prompt {item['kept_id']}"
+                    ),
+                }
+                for item in described
+            ]
+        return [
+            {"path": "", "detail": f"Removed: {item.get('text') or item['prompt_id']}"}
+            for item in described
+        ]
+
+
+def backup_note(backup):
+    """The status line's backup part: the file name, or that there is none."""
+    return f"Safety backup {backup}" if backup else NO_BACKUP
+
+
+def _fail_job(job, code, message):
+    """End ``job`` failed with ``message`` (one error row) and its code."""
+    job.record("errors", {"path": "", "detail": message}, f"! {message}")
+    job.update(status_line=message)
+    job.publish({"type": "error", "code": code, "message": message})
+    job.finish("failed")

py/api/maintenance_plans.py

@@ -0,0 +1,138 @@
+"""The reviewed-plan store behind maintenance previews (#249 System S6).
+
+A preview issues a plan: its operation, the fingerprint of exactly what was
+reviewed, the payload an apply needs (target ids, the sources it judged) and
+the identity of the database it describes. Everything is held here, in
+server memory, under a single-use token (``secrets.token_urlsafe(32)``); the
+client only ever sees the token and the public plan. Plans expire after
+``ttl`` seconds and are gone after a restart.
+
+The store keeps at most ``max_plans`` plans and ``max_bytes`` of payload
+(measured as compact JSON). A plan larger than ``max_bytes`` is refused with
+``ScopeTooLarge``: it is never cut down to fit, since an apply must do
+exactly what was reviewed. A new plan that fits makes room by dropping the
+oldest ones, which then read as expired.
+
+A consumed token is remembered (the last ``MAX_USED``) with the job it
+started, so a second apply of it is told which job ran it.
+"""
+
+import json
+import secrets
+import time
+from collections import OrderedDict
+from dataclasses import dataclass, field
+
+
+class PlanExpired(Exception):
+    """No live plan has the token: unknown, expired, dropped or another operation's."""
+
+
+class PlanUsed(Exception):
+    """The token was already applied; ``job_id`` is the job that applied it."""
+
+    def __init__(self, job_id):
+        super().__init__("This plan was already applied")
+        self.job_id = job_id
+
+
+class ScopeTooLarge(Exception):
+    """The plan's payload exceeds the store's byte budget."""
+
+
+@dataclass
+class StoredPlan:
+    """One reviewed plan, as only the server sees it."""
+
+    operation: str
+    fingerprint: str
+    payload: dict
+    db_identity: tuple
+    expires_at: float  # wall-clock seconds since the epoch, for the response
+    deadline: float = field(repr=False)  # on the store's monotonic clock
+    size: int = field(repr=False)
+
+
+def _payload_size(payload):
+    return len(json.dumps(payload, separators=(",", ":"), default=str))
+
+
+class PlanStore:
+    """Reviewed maintenance plans under single-use tokens."""
+
+    MAX_USED = 64  # consumed tokens remembered for plan-used answers
+
+    def __init__(
+        self,
+        ttl=900,
+        max_plans=8,
+        max_bytes=16 * 2**20,
+        *,
+        clock=time.monotonic,
+        wall_clock=time.time,
+    ):
+        self.ttl = ttl
+        self.max_plans = max_plans
+        self.max_bytes = max_bytes
+        self._clock = clock
+        self._wall_clock = wall_clock
+        self._plans = OrderedDict()  # token -> StoredPlan, oldest first
+        self._used = OrderedDict()  # token -> job id, oldest first
+
+    def issue(self, operation, fingerprint, payload, db_identity) -> str:
+        """Hold a plan and return its token; ScopeTooLarge when it cannot fit."""
+        size = _payload_size(payload)
+        if size > self.max_bytes:
+            raise ScopeTooLarge(f"The plan needs {size} bytes")
+        self._drop_expired()
+        while self._plans and (
+            len(self._plans) >= self.max_plans
+            or self._held_bytes() + size > self.max_bytes
+        ):
+            self._plans.popitem(last=False)
+        token = secrets.token_urlsafe(32)
+        self._plans[token] = StoredPlan(
+            operation=operation,
+            fingerprint=fingerprint,
+            payload=payload,
+            db_identity=tuple(db_identity),
+            expires_at=self._wall_clock() + self.ttl,
+            deadline=self._clock() + self.ttl,
+            size=size,
+        )
+        return token
+
+    def peek(self, token, operation=None) -> StoredPlan:
+        """The live plan of ``token``, left in place; raises PlanUsed or PlanExpired."""
+        if not isinstance(token, str):
+            raise PlanExpired("No such plan")
+        if token in self._used:
+            raise PlanUsed(self._used[token])
+        plan = self._plans.get(token)
+        if plan is None or (operation is not None and plan.operation != operation):
+            raise PlanExpired("No such plan")
+        if self._clock() > plan.deadline:
+            del self._plans[token]
+            raise PlanExpired("The plan expired")
+        return plan
+
+    def consume(self, token, operation=None, *, job_id=None) -> StoredPlan:
+        """Take the plan of ``token`` once; later calls raise PlanUsed(job_id)."""
+        plan = self.peek(token, operation)
+        del self._plans[token]
+        self._used[token] = job_id
+        while len(self._used) > self.MAX_USED:
+            self._used.popitem(last=False)
+        return plan
+
+    def discard(self, token):
+        """Forget a plan that can no longer apply (its database was replaced)."""
+        self._plans.pop(token, None)
+
+    def _held_bytes(self):
+        return sum(plan.size for plan in self._plans.values())
+
+    def _drop_expired(self):
+        now = self._clock()
+        for token in [t for t, plan in self._plans.items() if now > plan.deadline]:
+            del self._plans[token]

py/api/review.py

@@ -38,13 +38,17 @@ try:
     from ...database.library_commands import LibraryCommandError
     from ...database.review_decisions import ReviewCommandError
     from ...utils.media_capture import CaptureError
-    from ...utils.review_capture import captured_review, review_evidence
+    from ...utils.review_capture import (
+        captured_review,
+        item_profile,
+        review_evidence,
+    )
 except ImportError:
     from database.library import display_title
     from database.library_commands import LibraryCommandError
     from database.review_decisions import ReviewCommandError
     from utils.media_capture import CaptureError
-    from utils.review_capture import captured_review, review_evidence
+    from utils.review_capture import captured_review, item_profile, review_evidence
 
 ITEMS_URL = "/prompt_manager/review/items"
 DEFAULT_LIMIT = 40
@@ -641,7 +645,8 @@ def _verify_and_read(store, capture, item, root, need_metadata: bool):
     """Worker thread: ``(relative, facts, captured)``, or None off the root.
 
     Formats the item's target under the capture's root (resolving the root's
-    real path), re-hashes the current file, then reads the capture.
+    real path), re-hashes the current file, then reads the capture and
+    resolves it with the profile the item's evidence was recorded with.
     """
     found = format_review_target(
         item["spelled_path"],
@@ -652,7 +657,9 @@ def _verify_and_read(store, capture, item, root, need_metadata: bool):
         return None
     relative = found[1]
     facts = store.verify(root, relative)
-    captured = captured_review(store, capture) if need_metadata else None
+    captured = None
+    if need_metadata:
+        captured = captured_review(store, capture, item_profile(item["evidence"]))
     return relative, facts, captured
 
 

py/api/review_recording.py

@@ -8,45 +8,76 @@ already. A file the fallback links, an exact pair the ingester stored and an
 already linked file are never queued. Neither is an ambiguous file with
 nothing to choose: fewer than two of its candidates carry a prompt
 (``utils.review_capture.reviewable``); the scan reports it unresolved ("could
-not be traced").
+not be traced") unless the inspection profile finds a choice in it.
+
+Only scans add items (maintainer ruling, 2026-10-23). The monitor's pass
+records nothing: it only removes the pending items of an output it linked.
 
 The queue heals itself: a pending item whose file is now linked (by the
 fallback, a later resolution or the monitor) is removed by the next scan
-that lists the file or by the monitor's next pass over it, and so is a
-pending item without a choice whose file is still ambiguous without one
-(``settle_untraced_review_items``). Only an undecided item goes
+that lists the file or by the monitor's next pass over it. The next scan
+also removes a pending item without a choice whose file is still ambiguous
+without one (``settle_untraced_review_items``), and an older version's
+item of bytes now untraced (``settle_stale_review_items``). Only an undecided item goes
 (``pending``, revision 0, no decision row); a decision is never deleted or
 changed (``settle_linked_review_items``).
 
 The scan records through ``_record_scanned_review`` (best effort: a failure
-is logged and counted); the monitor calls ``record_review_file`` from its
-worker thread after its own link. Both go through
-``utils.review_capture.record_ambiguous_file`` on the API's capture store.
-Nothing here links a file.
+is logged and counted), which goes through
+``utils.review_capture.record_ambiguous_file`` on the API's capture store;
+the monitor calls ``record_review_file`` from its worker thread after its
+own link. Nothing here links a file.
 
 Recording changes records, so it follows the registry's ``records``
 admission (#289). The scan records under its own ``records`` job. The
 monitor's pass lasts one file and is not a job: it is refused while a
 ``records`` or ``guarded`` job runs (``deferred``), and the next scan reads
-the file again.
+the file again. Thumbnail work (the job and the gallery's generate and clear
+holds, kind ``thumbnails``) is ``guarded`` but only writes and removes preview
+files under ``thumbnails/`` and never touches the database, so it does not
+defer the recording (#249 S5 M7): a long rebuild no longer holds Needs review
+back for its whole run.
 """
 
+import hashlib
 import os
 from pathlib import Path
 
 from .admin import SCAN_LOG_PREFIXES, _plural, _scan_location, _scan_roots
+from .jobs import THUMBNAILS_JOB_KIND
 
 # The kind the monitor's recording checks admission under (never registered).
 REVIEW_RECORDING_KIND = "review_recording"
+# Guarded kinds that change no records, so the recording may run beside them.
+PREVIEW_ONLY_KINDS = (THUMBNAILS_JOB_KIND,)
 # record_ambiguous_file outcomes that leave the file in the pending queue.
 QUEUED_OUTCOMES = ("recorded", "limited", "duplicate")
 NEW_OUTCOMES = ("recorded", "limited")
 
 try:
+    from ...utils import review_capture
     from ...utils.review_capture import record_ambiguous_file
 except ImportError:
+    from utils import review_capture
     from utils.review_capture import record_ambiguous_file
 
+# Untraced files remembered per process so a rescan of an unchanged file does
+# not run the inspection profile again (oldest forgotten first). A plain dict
+# of 16-byte digests (the locator, and its size, mtime_ns and item version):
+# measured at 8.6 MiB with 65,536 entries (tracemalloc, CPython 3.14).
+MAX_UNTRACED_MEMORY = 65_536
+
+
+def _digest(text: str) -> bytes:
+    data = text.encode("utf-8", "surrogatepass")
+    return hashlib.blake2b(data, digest_size=16).digest()
+
+
+def _memory_key(locator: str) -> bytes:
+    """A 16-byte digest of ``locator`` (the untraced memory's key)."""
+    return _digest(locator)
+
+
 # The scan log's reason for items that leave the queue with nothing to choose.
 NOTHING_TO_CHOOSE = "nothing to choose"
 
@@ -54,10 +85,15 @@ NOTHING_TO_CHOOSE = "nothing to choose"
 class ReviewRecordingMixin:
     """Record the scan's and the monitor's ambiguous files for Needs review."""
 
-    def _record_scanned_review(self, media_file, roots, review=None):
-        """Record one ambiguous, unlinked scanned file (logged and counted)."""
+    def _record_scanned_review(self, media_file, roots, review=None, scanned=None):
+        """Record one ambiguous, unlinked scanned file (logged and counted).
+
+        ``scanned`` is the scan's ``(metadata, inspection_resolution)``.
+        """
         try:
-            outcome = self._record_review_sync(media_file, roots, review)
+            outcome = self._record_review_sync(
+                media_file, roots, review, inspection=True, scanned=scanned
+            )
         except Exception:
             self.logger.warning(
                 f"{media_file.name} could not be added to Needs review",
@@ -73,9 +109,13 @@ class ReviewRecordingMixin:
         if outcome in QUEUED_OUTCOMES:
             review["queued"] += 1
 
-    def _record_review_sync(self, media_file, roots, review=None):
+    def _record_review_sync(
+        self, media_file, roots, review=None, inspection=False, scanned=None
+    ):
         """``record_ambiguous_file`` for a file under one of ``roots`` (blocking).
 
+        ``inspection`` (the scan only) lets a file without a choice under the
+        default profile be recorded from the inspection profile's choices.
         An ``untraced`` file (its captured bytes offer nothing to choose) also
         loses its pending items that offer nothing either.
         """
@@ -84,12 +124,134 @@ class ReviewRecordingMixin:
             return "skipped"
         root, index, relative, _spelled = located
         outcome = record_ambiguous_file(
-            self.db, self._capture_store(), Path(root), index, relative
+            self.db,
+            self._capture_store(),
+            Path(root),
+            index,
+            relative,
+            inspection=inspection,
+            scanned=scanned,
         )
         if outcome == "untraced":
             self._settle_untraced_reviews([media_file], roots, review)
         return outcome
 
+    def _untraced_scan_outcome(self, media_file, roots, canonical, review=None):
+        """The scan status of an unlinked file the default profile left
+        ambiguous without a choice (blocking).
+
+        ``ambiguous`` when the inspection profile offers a choice: the file is
+        recorded for review from that profile (never linked), or ``unchanged``
+        when a saved row links it already. Otherwise ``unresolved``, and its
+        pending items without a choice leave the queue. The profile runs at
+        most once per file state: not for bytes already queued (or decided)
+        at the file's current size and ``mtime_ns`` under the version that
+        its records depend on, and not again for a state it already found
+        untraced in this process.
+        """
+        try:
+            choice = self._inspection_choice(media_file, roots, canonical)
+        except Exception:
+            # One bad file never stops the scan: it stays "could not be traced".
+            self.logger.warning(
+                f"{media_file.name} could not be inspected for Needs review",
+                exc_info=True,
+            )
+            if review is not None:
+                review["errors"] += 1
+            return "unresolved"
+        if choice is None:
+            self._settle_untraced_reviews([media_file], roots, review)
+            self._settle_stale_reviews(media_file, roots, canonical, review)
+            return "unresolved"
+        if self._review_file_linked(media_file, roots):
+            return "unchanged"
+        self._record_scanned_review(media_file, roots, review, scanned=choice)
+        return "ambiguous"
+
+    def _inspection_choice(self, media_file, roots, canonical):
+        """``(canonical, resolution_or_None)`` when the file offers a choice.
+
+        The resolution is None when the file's current state is already in
+        the queue (so the profile did not run). None when it offers none.
+        """
+        if canonical is None:
+            return None
+        version = review_capture.inspection_review_version(canonical)
+        located = self._review_location(media_file, roots)
+        if version is None or located is None:
+            return None
+        locator = os.path.normcase(located[3])
+        try:
+            facts = os.lstat(located[3])
+        except OSError:
+            facts = None
+        state = None
+        if facts is not None:
+            state = _digest(f"{facts.st_size}:{facts.st_mtime_ns}:{version}")
+            if self.db.review_decided_at(
+                locator, facts.st_size, facts.st_mtime_ns
+            ) or self.db.review_fingerprint_queued(
+                locator, facts.st_size, facts.st_mtime_ns, version
+            ):
+                return canonical, None
+        memory = self._untraced_memory()
+        key = _memory_key(locator)
+        if state is not None and memory.get(key) == state:
+            return None
+        resolution = review_capture.inspection_choice(canonical)
+        if resolution is not None:
+            memory.pop(key, None)
+            return canonical, resolution
+        if state is not None:
+            memory.pop(key, None)  # re-inserted last: the newest state
+            memory[key] = state
+            while len(memory) > MAX_UNTRACED_MEMORY:
+                del memory[next(iter(memory))]
+        return None
+
+    def _settle_stale_reviews(self, media_file, roots, canonical, review=None):
+        """Remove the pending, undecided items of an untraced file's current
+        bytes recorded under another version (best effort, counted in
+        ``review["untraced"]``).
+
+        The current resolver and table offer nothing to choose in these
+        bytes, so an item of an older version would only refuse Apply.
+        """
+        located = self._review_location(media_file, roots)
+        if located is None or canonical is None:
+            return 0
+        # An unusable semantics table says nothing about the current
+        # versions; never let it retire items a working table recorded.
+        if review_capture.inspection_review_version(canonical) is None:
+            return 0
+        try:
+            facts = os.lstat(located[3])
+            removed = self.db.settle_stale_review_items(
+                os.path.normcase(located[3]),
+                facts.st_size,
+                facts.st_mtime_ns,
+                review_capture.item_versions(canonical),
+            )
+        except Exception:
+            self.logger.warning(
+                f"Stale Needs review items of {media_file.name} could not be "
+                "removed",
+                exc_info=True,
+            )
+            if review is not None:
+                review["errors"] += 1
+            return 0
+        if review is not None:
+            review["untraced"] += removed
+        return removed
+
+    def _untraced_memory(self):
+        memory = getattr(self, "_untraced_inspections", None)
+        if memory is None:
+            memory = self._untraced_inspections = {}
+        return memory
+
     @staticmethod
     def _review_location(media_file, roots, real_roots=None):
         """``(root, root_index, relative, spelled)`` of a listed file, or None.
@@ -171,25 +333,23 @@ class ReviewRecordingMixin:
             review["untraced"] += removed
         return removed
 
-    def _fallback_links(self, metadata):
-        """True when the scan's legacy fallback would link these records."""
-        return bool(metadata.get("prompt")) and bool(self._graph_prompt_text(metadata))
-
     def record_review_file(self, image_path):
         """Apply the Needs review rule to one monitor output (its worker thread).
 
-        Maps ``image_path`` to the gallery root that lists it; a file under no
-        current root is ``skipped``. A linked file is never queued: its
-        pending items are removed and the answer is ``linked``. An unlinked
-        file the scan's fallback would link is ``skipped`` (the next scan
-        links it); any other file is recorded when it is ambiguous with a
-        choice, and is ``untraced`` (its pending items without a choice
-        removed) when it is ambiguous without one. While a
-        job runs that a ``records`` job may not run beside, nothing is written
-        and the answer is ``deferred``. Errors propagate: the monitor logs
-        them and keeps its link.
+        Only scans add items (maintainer ruling, 2026-10-23): the monitor
+        only clears the items of outputs it links. Maps ``image_path`` to the
+        gallery root that lists it; a file under no current root is
+        ``skipped``. A linked file's pending items are removed and the answer
+        is ``linked``. Any other file is ``skipped``: nothing is captured or
+        recorded, and the next scan reads it. While a job runs that a
+        ``records`` job may not run beside, other than thumbnail work
+        (``PREVIEW_ONLY_KINDS``), nothing is written and the answer is
+        ``deferred``. Errors propagate: the monitor logs them and keeps its
+        link.
         """
-        busy = self._jobs.blocker(REVIEW_RECORDING_KIND, admission="records")
+        busy = self._jobs.blocker(
+            REVIEW_RECORDING_KIND, admission="records", compatible=PREVIEW_ONLY_KINDS
+        )
         if busy is not None:
             self.logger.info(
                 f"Needs review was not updated for "
@@ -205,9 +365,7 @@ class ReviewRecordingMixin:
         if self._review_file_linked(located, roots):
             self._settle_scanned_reviews([located], roots)
             return "linked"
-        if self._fallback_links(self._extract_comfyui_metadata(str(located))):
-            return "skipped"
-        return self._record_review_sync(located, roots)
+        return "skipped"
 
     @staticmethod
     def _log_review_items(job, review):

py/api/settings_routes.py

@@ -40,6 +40,7 @@ _SETTINGS_FILE_KEYS = {
     "INFINITE_SCROLL": ("web_ui", "infinite_scroll"),
     "WORKER_THREADS": ("performance", "worker_threads"),
     "MONITORING_ENABLED": ("gallery", "monitoring", "enabled"),
+    "SAFETY_BACKUP": ("maintenance", "safety_backup"),
 }
 # Attributes that live on GalleryConfig; the others on PromptManagerConfig.
 _GALLERY_SETTINGS = frozenset({"MONITORING_ENABLED"})
@@ -187,6 +188,7 @@ class SettingsRoutesMixin:
                         "webui_display_mode": PromptManagerConfig.WEBUI_DISPLAY_MODE,
                         "infinite_scroll": PromptManagerConfig.INFINITE_SCROLL,
                         "worker_threads": PromptManagerConfig.WORKER_THREADS,
+                        "safety_backup": PromptManagerConfig.SAFETY_BACKUP,
                         "cpu_count": PromptManagerConfig.max_worker_threads(),
                         "gallery_root_paths": root_paths,
                         "gallery_root_path": root_paths[0] if root_paths else "",
@@ -481,6 +483,10 @@ class SettingsRoutesMixin:
             if not isinstance(data["monitoring_enabled"], bool):
                 raise ValueError("monitoring_enabled must be true or false")
             updates["MONITORING_ENABLED"] = data["monitoring_enabled"]
+        if "safety_backup" in data:
+            if not isinstance(data["safety_backup"], bool):
+                raise ValueError("safety_backup must be true or false")
+            updates["SAFETY_BACKUP"] = data["safety_backup"]
         if "worker_threads" in data:
             updates["WORKER_THREADS"] = validate_worker_threads(
                 data["worker_threads"], PromptManagerConfig.max_worker_threads()

py/api/thumbnail_auto.py

@@ -0,0 +1,843 @@
+"""Automatic thumbnails (#249): previews are a cache the server keeps itself.
+
+The approved mock has no repeated global thumbnail prompt
+(``docs/concepts/workspace/README.md``), so the workspace never asks. Every
+listing gives an image or video a thumbnail URL (``media_thumbnail``), and the
+previews behind those URLs are made here, in three ways:
+
+- **On first view.** ``GET /prompt_manager/images/serve/thumbnails/<rel>_thumb
+  <ext>`` for a preview that does not exist yet is admitted on the event loop
+  (``_PreviewGate``): at most ``ON_DEMAND_LIMIT`` previews are written at once,
+  on this API's own small pool (never the default executor), one flight runs
+  per file, and the requests waiting are capped (``MAX_WAITING``); past the
+  cap a request gets its fallback at once. Inside the admission the preview is
+  mapped back to its media file under the same root (the request text is
+  checked first: no traversal, drive, UNC or empty segment; then a regular
+  file inside the root, never through a link), made and served. When it
+  cannot be made an image's original is served and a video's placeholder
+  frame, both ``Cache-Control: no-store``: a video is never sent to an
+  ``<img>``. A file that cannot be decoded is remembered for this session
+  (keyed by its path, size and mtime) and counted for the System Overview
+  (``failed`` in ``/images/thumbnails/status``); I/O errors and a preview
+  another program holds open (Windows) are not remembered. The recorded
+  history's thumbnail route goes through the same admission and flight key.
+- **On capture.** The image monitor hands every new output it settles to
+  ``queue_capture_thumbnail``, which queues its preview for one background
+  worker thread (``AutoThumbnailer``). Each wake takes at most Worker threads
+  (``WORKER_THREADS``) files and writes them on that many threads; enqueuing
+  is O(1), so the monitor and ComfyUI's queue never wait. A file already
+  queued is not queued again, the queue is bounded (``MAX_QUEUED``) and a
+  preview left out is made on its first view instead.
+- **After a scan.** A scan that added files checks the missing previews; from
+  ``SCAN_THUMBNAIL_JOB_THRESHOLD`` up it starts the thumbnail Activity job
+  (missing only, visible and cancellable) instead of writing them silently,
+  and fewer go through the worker.
+
+Every preview is written by ``_generate_one`` with the thumbnail job's default
+size and quality (``DEFAULT_THUMBNAIL_EDGE``, medium), into a temp sibling
+renamed over the old one, so a reader never sees a partial file.
+
+Holds: automatic previews only write files under ``thumbnails/`` and never
+touch the database, like the thumbnail job, so they run beside a scan, image
+tagging and the Needs review recording (S5 M7). They are not written while a
+job runs that replaces the database or removes media and previews: a restore,
+a duplicate removal, a sources change, the gallery's thumbnail clear or an
+``exclusive`` job (``_auto_thumbnail_blocker``). The holds are checked when a
+write starts (after its slot) and again just before its rename.
+
+Shutdown: ``close_auto_thumbnails`` stops the worker at a file boundary,
+kills a running ffmpeg, waits a bounded time (and logs when the wait ran
+out) and shuts the view pool down; after the app's cleanup
+(``final=True``) nothing starts a worker again. Importing this module again
+(a package reload) stops the workers the previous import left.
+"""
+
+import asyncio
+import collections
+import functools
+import io
+import os
+import threading
+from concurrent.futures import ThreadPoolExecutor
+from pathlib import Path
+
+from aiohttp import web
+
+from .images import (
+    GALLERY_IMAGE_EXTENSIONS,
+    MEDIA_EXTENSIONS,
+    THUMBNAIL_CLEAR_TITLE,
+    THUMBNAIL_ENCODINGS,
+    VIDEO_EXTENSIONS,
+    _has_traversal,
+    _is_within,
+    _thumbnail_is_fresh,
+    _video_placeholder_image,
+    _worker_threads,
+    media_thumbnail_rel,
+)
+from .jobs import (
+    DUPLICATE_CLEANUP_JOB_KIND,
+    RESTORE_JOB_KIND,
+    THUMBNAILS_JOB_KIND,
+    JobBusy,
+)
+from .settings_routes import SOURCE_CHANGE_JOB_KIND
+from .thumbnail_jobs import DEFAULT_THUMBNAIL_EDGE, ThumbnailOptions
+
+try:
+    from ...utils.parallel import map_parallel
+except ImportError:  # tests import the package top-level
+    from utils.parallel import map_parallel
+
+AUTO_THUMBNAIL_EDGE = DEFAULT_THUMBNAIL_EDGE
+AUTO_THUMBNAIL_SIZE = (AUTO_THUMBNAIL_EDGE, AUTO_THUMBNAIL_EDGE)
+AUTO_THUMBNAIL_ENCODING = THUMBNAIL_ENCODINGS["medium"]
+# Previews views may write at once (the size of the view pool).
+ON_DEMAND_LIMIT = 3
+# View requests that may wait on the gate; one more gets its fallback at once.
+MAX_WAITING = 64
+# How long a pool thread waits for the capture worker's flight of the same file.
+FLIGHT_WAIT_SECONDS = 20
+# How long close() waits for the worker's batch in hand (ffmpeg's own limit).
+CLOSE_WAIT_SECONDS = 30
+# Files the capture worker holds; more are made on their first view.
+MAX_QUEUED = 10_000
+# Undecodable files remembered per session (oldest forgotten first).
+MAX_FAILURES_REMEMBERED = 4096
+# Missing previews after a scan from which the visible thumbnail job starts:
+# a fresh install or a newly added source. Fewer go through the worker.
+SCAN_THUMBNAIL_JOB_THRESHOLD = 200
+# Jobs beside which no automatic preview is written (see the module docstring).
+AUTO_REFUSED_KINDS = frozenset(
+    {RESTORE_JOB_KIND, DUPLICATE_CLEANUP_JOB_KIND, SOURCE_CHANGE_JOB_KIND}
+)
+WORKER_NAME = "PromptManagerThumbnailWorker"
+POOL_PREFIX = "pm-preview"
+# Identifies this import's workers: a reload stops the ones it did not start.
+_MODULE_TOKEN = object()
+
+READY, FAILED, BUSY, DEFERRED = "ready", "failed", "busy", "deferred"
+# What the gate answers a request past MAX_WAITING.
+OVER = object()
+
+
+def _fingerprint(src):
+    """``(path, size, mtime_ns)`` of *src*, or None when it cannot be read."""
+    try:
+        stat = os.stat(src)
+    except OSError:
+        return None
+    return (os.path.normcase(str(src)), stat.st_size, stat.st_mtime_ns)
+
+
+def flight_key(root, rel_parts):
+    """The single-flight key of ``root/rel_parts``: its folder resolved, case
+    folded. The serve route, the history route and the worker share it."""
+    path = Path(root).joinpath(*rel_parts)
+    return os.path.normcase(os.path.join(os.path.realpath(path.parent), path.name))
+
+
+def _dst_key(dst):
+    return flight_key(Path(dst).parent, (Path(dst).name,))
+
+
+@functools.lru_cache(maxsize=1)
+def video_placeholder_jpeg():
+    """The placeholder frame as JPEG bytes (a video whose preview is missing)."""
+    buffer = io.BytesIO()
+    _video_placeholder_image(AUTO_THUMBNAIL_SIZE).save(
+        buffer, "JPEG", **AUTO_THUMBNAIL_ENCODING
+    )
+    return buffer.getvalue()
+
+
+def _is_preview_request(filepath):
+    """Text check only: ``thumbnails/<...>/<name>_thumb<lower-case ext>``.
+
+    Refuses, as text and before any path is joined, what a joined path could
+    take as an anchor: a backslash, a drive or stream colon, and empty, ``.``
+    or ``..`` segments (so a leading ``/`` and ``//server/share`` too).
+    """
+    if "\\" in filepath or ":" in filepath:
+        return False
+    parts = filepath.split("/")
+    if len(parts) < 2 or parts[0] != "thumbnails":
+        return False
+    if any(part in ("", ".", "..") for part in parts) or "thumbnails" in parts[1:-1]:
+        return False
+    stem, extension = os.path.splitext(parts[-1])
+    return (
+        stem.endswith("_thumb")
+        and len(stem) > len("_thumb")
+        and extension == extension.lower()
+        and extension in GALLERY_IMAGE_EXTENSIONS
+    )
+
+
+def _regular_file(path):
+    """True for a regular file that is not a link (one lstat)."""
+    try:
+        return not path.is_symlink() and path.is_file()
+    except OSError:
+        return False
+
+
+def _candidates(parts):
+    """``(folder parts, base, extensions)`` of a checked preview request."""
+    stem, extension = os.path.splitext(parts[-1])
+    wanted = [extension] + (sorted(VIDEO_EXTENSIONS) if extension == ".jpg" else [])
+    return parts[1:-1], stem[: -len("_thumb")], wanted
+
+
+def _quick_source(root, parts):
+    """``(src, is_video)`` by exact name only (no folder listing), or None."""
+    root = Path(root)
+    folder_parts, base, wanted = _candidates(parts)
+    folder = root.joinpath(*folder_parts)
+    for extension in wanted:
+        for name in (base + extension, base + extension.upper()):
+            path = folder / name
+            if _regular_file(path) and _is_within(path, root):
+                return path, extension in VIDEO_EXTENSIONS
+    return None
+
+
+def _thumbnail_source(root, parts):
+    """``(src, dst, is_video)`` for the preview *parts* under *root*, or None.
+
+    *parts* passed ``_is_preview_request``. The source has the preview's name
+    without ``_thumb`` and an extension that previews as the requested one
+    (any case: exact names first, then one listing of the folder); a ``.jpg``
+    may also be a video's. It must be a regular file inside *root*. ``dst``
+    is None when the preview's folder resolves outside ``root/thumbnails`` (a
+    link): nothing is written there.
+    """
+    root = Path(root)
+    folder_parts, base, wanted = _candidates(parts)
+    quick = _quick_source(root, parts)
+    if quick is not None:
+        found = quick[0]
+    else:
+        found = _scan_folder(root.joinpath(*folder_parts), base, set(wanted))
+        if found is None or not _is_within(found, root):
+            return None
+    rel = Path(*folder_parts, found.name)
+    dst = root / media_thumbnail_rel(rel)
+    if dst != root.joinpath(*parts):
+        return None
+    if not _is_within(root / "thumbnails", root) or not _is_within(
+        dst, root / "thumbnails"
+    ):
+        dst = None
+    return found, dst, found.suffix.lower() in VIDEO_EXTENSIONS
+
+
+def _scan_folder(folder, base, extensions):
+    """A mixed-case extension (``.Png``): one listing of *folder*, or None."""
+    try:
+        with os.scandir(folder) as entries:
+            for entry in entries:
+                stem, extension = os.path.splitext(entry.name)
+                if stem == base and extension.lower() in extensions:
+                    path = Path(entry.path)
+                    if _regular_file(path):
+                        return path
+    except OSError:
+        return None
+    return None
+
+
+def _stop_stale_workers():
+    """Stop capture workers a previous import of this module started."""
+    for thread in threading.enumerate():
+        token = getattr(thread, "pm_preview_token", None)
+        close = getattr(thread, "pm_preview_close", None)
+        if token is not None and token is not _MODULE_TOKEN and close is not None:
+            close()
+
+
+class AutoThumbnailer:
+    """Single-flight preview writes and the capture worker (thread-safe).
+
+    ``generate(src, dst, is_video, *, stop, guard, track)`` writes one
+    preview and returns ``_generate_one``'s result dict; ``may_write()``
+    says whether automatic previews may be written now. Views call
+    ``ensure`` on the view pool, the monitor calls ``enqueue`` from its
+    thread.
+    """
+
+    def __init__(
+        self,
+        generate,
+        may_write,
+        logger,
+        *,
+        workers=_worker_threads,
+        max_queued=MAX_QUEUED,
+        start_worker=True,
+    ):
+        self._generate = generate
+        self._may_write = may_write
+        self._logger = logger
+        self._workers = workers
+        self._max_queued = max_queued
+        self._start_worker = start_worker
+        self._changed = threading.Condition()
+        self._inflight = {}
+        self._failures = collections.OrderedDict()
+        self._queue = collections.deque()
+        self._queued = set()
+        self._processes = []
+        self._worker = None
+        # True from a worker's start until it leaves (under the lock), so an
+        # idle worker can exit and the next enqueue starts a fresh one.
+        self._worker_running = False
+        self._closed = False
+        self._busy = False
+        self._batch = 0
+
+    @property
+    def failure_count(self):
+        with self._changed:
+            return len(self._failures)
+
+    @property
+    def closed(self):
+        return self._closed
+
+    def _allowed(self):
+        return not self._closed and self._may_write()
+
+    def _track(self, process):
+        """Keep an ffmpeg process so close() can kill it."""
+        with self._changed:
+            self._processes = [p for p in self._processes if p.poll() is None]
+            self._processes.append(process)
+            closed = self._closed
+        if closed:
+            process.kill()
+
+    # ── one preview ───────────────────────────────────────────────────
+
+    def ensure(self, src, dst, is_video):
+        """Make *dst* for *src* unless it is fresh; returns the outcome.
+
+        ``ready`` (the preview is there), ``failed`` (the file cannot be
+        decoded, remembered), ``busy`` (an I/O error, the file is gone, or
+        another flight did not finish) or ``deferred`` (a hold refuses
+        automatic writes, checked when the write starts and before its
+        rename). Blocking: call it off the event loop.
+        """
+        if _thumbnail_is_fresh(src, dst):
+            return READY
+        fingerprint = _fingerprint(src)
+        if fingerprint is None:
+            return BUSY
+        key = _dst_key(dst)
+        with self._changed:
+            if fingerprint in self._failures:
+                return FAILED
+            flight = self._inflight.get(key)
+            if flight is None:
+                self._inflight[key] = done = threading.Event()
+        if flight is not None:
+            flight.wait(FLIGHT_WAIT_SECONDS)
+            if _thumbnail_is_fresh(src, dst):
+                return READY
+            with self._changed:
+                return FAILED if fingerprint in self._failures else BUSY
+        try:
+            return self._lead(src, dst, is_video, fingerprint)
+        finally:
+            with self._changed:
+                self._inflight.pop(key, None)
+                self._changed.notify_all()
+            done.set()
+
+    def _lead(self, src, dst, is_video, fingerprint):
+        if not self._allowed():
+            return DEFERRED
+        if _thumbnail_is_fresh(src, dst):
+            return READY
+        result = self._generate(
+            src,
+            dst,
+            is_video,
+            stop=lambda: self._closed,
+            guard=self._allowed,
+            track=self._track,
+        )
+        if result.get("action") in ("generated", "skipped"):
+            return READY
+        if result.get("held"):
+            return DEFERRED
+        if not result.get("decode"):
+            return BUSY  # an I/O error says nothing about the file
+        error = result.get("error") or "failed"
+        with self._changed:
+            self._failures[fingerprint] = error
+            while len(self._failures) > MAX_FAILURES_REMEMBERED:
+                self._failures.popitem(last=False)
+        self._logger.info(
+            f"No preview for {os.path.basename(str(src))} ({error}); "
+            "the original is shown"
+        )
+        return FAILED
+
+    # ── the capture worker ────────────────────────────────────────────
+
+    def enqueue(self, src, dst, is_video):
+        """Queue *dst*'s preview for the worker; False when already queued,
+        full or closed. Never blocks on a write."""
+        key = _dst_key(dst)
+        with self._changed:
+            if self._closed or key in self._queued:
+                return False
+            if len(self._queue) >= self._max_queued:
+                return False
+            self._queued.add(key)
+            self._queue.append((key, Path(src), Path(dst), bool(is_video)))
+            self._busy = True
+            if self._start_worker and not self._worker_running:
+                self._start_worker_locked()
+            self._changed.notify_all()
+        return True
+
+    def _start_worker_locked(self):
+        """Start the worker (lock held). A start that fails leaves the file
+        queued and the next enqueue tries again."""
+        worker = threading.Thread(target=self._run, name=WORKER_NAME, daemon=True)
+        worker.pm_preview_token = _MODULE_TOKEN
+        worker.pm_preview_close = self.close
+        self._worker_running = True
+        try:
+            worker.start()
+        except RuntimeError:
+            self._worker_running = False
+            self._busy = False
+            self._logger.warning(
+                "The automatic preview worker could not start; the next output "
+                "tries again",
+                exc_info=True,
+            )
+            return
+        self._worker = worker
+
+    def _run(self):
+        try:
+            self._drain()
+        except Exception:
+            self._logger.warning("The automatic preview worker stopped", exc_info=True)
+        finally:
+            # Whatever ended the loop, the next enqueue starts a fresh worker.
+            with self._changed:
+                self._worker_running = False
+                self._busy = False
+                self._batch = 0
+                self._changed.notify_all()
+
+    def _drain(self):
+        while True:
+            with self._changed:
+                if self._closed or not self._queue:
+                    # Idle or closed: leave, so no thread outlives its work.
+                    self._busy = False
+                    self._worker_running = False
+                    self._changed.notify_all()
+                    return
+                take = min(len(self._queue), max(1, int(self._workers() or 1)))
+                batch = [self._queue.popleft() for _ in range(take)]
+                self._batch = len(batch)
+                self._changed.notify_all()
+            try:
+                map_parallel(self._write_queued, batch, len(batch))
+            finally:
+                with self._changed:
+                    self._batch = 0
+                    for key, *_rest in batch:
+                        self._queued.discard(key)
+
+    def _write_queued(self, item):
+        _key, src, dst, is_video = item
+        try:
+            self.ensure(src, dst, is_video)
+        except Exception:  # one file never stops the worker
+            self._logger.warning(
+                f"Automatic preview failed for {src.name}", exc_info=True
+            )
+
+    def close(self):
+        """Stop the worker at its next file boundary (queued files are
+        dropped), kill a running ffmpeg and wait up to CLOSE_WAIT_SECONDS;
+        a worker still running after that is logged."""
+        with self._changed:
+            self._closed = True
+            self._queue.clear()
+            self._queued.clear()
+            processes, self._processes = self._processes, []
+            self._changed.notify_all()
+            worker = self._worker
+        for process in processes:
+            try:
+                process.kill()
+            except OSError:
+                pass
+        if worker is None or worker is threading.current_thread():
+            return
+        worker.join(CLOSE_WAIT_SECONDS)
+        if worker.is_alive():
+            self._logger.warning(
+                "The automatic preview worker is still running after "
+                f"{CLOSE_WAIT_SECONDS} s; it stops at its next file"
+            )
+
+    # ── waits (tests and shutdown; never timers) ──────────────────────
+
+    def _wait_for(self, predicate, timeout):
+        with self._changed:
+            return self._changed.wait_for(predicate, timeout)
+
+    def wait_idle(self, timeout=None):
+        """True once the worker has nothing queued or in hand."""
+        return self._wait_for(lambda: not self._busy, timeout)
+
+    def wait_batch(self, size, timeout=None):
+        """True once the worker holds a batch of *size* files."""
+        return self._wait_for(lambda: self._batch == size, timeout)
+
+
+class _PreviewGate:
+    """Admission of view requests on the event loop (#249 I2).
+
+    Waiters are coroutines: at most ``limit`` previews are written at once
+    on the view pool, one flight runs per key, and past ``MAX_WAITING``
+    waiting requests ``run`` answers ``OVER`` at once.
+    """
+
+    def __init__(self, limit, logger):
+        self.loop = asyncio.get_running_loop()
+        self._slots = asyncio.Semaphore(limit)
+        self._flights = {}
+        self._tasks = set()
+        self._logger = logger
+        self._tick = asyncio.Event()
+        self.waiting = 0
+
+    def _changed(self):
+        tick, self._tick = self._tick, asyncio.Event()
+        tick.set()
+
+    async def settled(self, predicate):
+        """Return once ``predicate()`` holds (re-checked at each change)."""
+        while not predicate():
+            await self._tick.wait()
+
+    async def run(self, key, work, pool):
+        """``work()``'s answer for *key* (shared by every request of the
+        flight), None when it raised, or OVER past the cap."""
+        if self.waiting >= MAX_WAITING:
+            return OVER
+        flight = self._flights.get(key)
+        if flight is None:
+            flight = self.loop.create_future()
+            self._flights[key] = flight
+            task = self.loop.create_task(self._lead(key, flight, work, pool))
+            self._tasks.add(task)
+            task.add_done_callback(self._tasks.discard)
+        self.waiting += 1
+        self._changed()
+        try:
+            return await asyncio.shield(flight)
+        finally:
+            self.waiting -= 1
+            self._changed()
+
+    async def _lead(self, key, flight, work, pool):
+        result = None
+        try:
+            async with self._slots:
+                result = await self.loop.run_in_executor(pool, work)
+        except Exception:
+            self._logger.warning("Automatic preview failed", exc_info=True)
+        finally:
+            self._flights.pop(key, None)
+            if not flight.done():
+                flight.set_result(result)
+
+
+class AutoThumbnailMixin:
+    """Automatic previews on the API: views, the monitor's queue, scans."""
+
+    _auto_thumbnailer = None
+    _auto_thumbnails_shut = False
+    _preview_gate_obj = None
+    _preview_pool_obj = None
+    _auto_thumbnailer_lock = threading.Lock()
+
+    def _auto_thumbnails(self):
+        """This API's AutoThumbnailer, created on first use (no worker after
+        the app's cleanup)."""
+        with AutoThumbnailMixin._auto_thumbnailer_lock:
+            if self._auto_thumbnailer is None:
+                self._auto_thumbnailer = AutoThumbnailer(
+                    self._write_auto_thumbnail,
+                    lambda: self._auto_thumbnail_blocker() is None,
+                    self.logger,
+                    start_worker=not self._auto_thumbnails_shut,
+                )
+            return self._auto_thumbnailer
+
+    def close_auto_thumbnails(self, final=False):
+        """Stop the capture worker and the view pool, and forget them.
+
+        Bounded (see AutoThumbnailer.close). ``final`` (the app's cleanup)
+        also keeps a late monitor event from starting a worker again; a
+        test reset passes False and the next use starts fresh ones.
+        """
+        with AutoThumbnailMixin._auto_thumbnailer_lock:
+            if final:
+                self._auto_thumbnails_shut = True
+            thumbnailer, self._auto_thumbnailer = self._auto_thumbnailer, None
+            pool, self._preview_pool_obj = self._preview_pool_obj, None
+            self._preview_gate_obj = None
+        if thumbnailer is not None:
+            thumbnailer.close()
+        if pool is not None:
+            pool.shutdown(wait=False, cancel_futures=True)
+
+    def _auto_thumbnail_failures(self):
+        thumbnailer = self._auto_thumbnailer
+        return 0 if thumbnailer is None else thumbnailer.failure_count
+
+    def _write_auto_thumbnail(self, src, dst, is_video, *, stop, guard, track):
+        return self._generate_one(
+            src,
+            dst,
+            AUTO_THUMBNAIL_SIZE,
+            is_video,
+            encoding=AUTO_THUMBNAIL_ENCODING,
+            stop=stop,
+            guard=guard,
+            track=track,
+        )
+
+    def _auto_thumbnail_blocker(self):
+        """The running job automatic previews wait for, or None (any thread)."""
+        for job in self._jobs.running():
+            if job.admission == "exclusive" or job.kind in AUTO_REFUSED_KINDS:
+                return job
+            if job.kind == THUMBNAILS_JOB_KIND and job.title == THUMBNAIL_CLEAR_TITLE:
+                return job
+        return None
+
+    # ── the view admission ────────────────────────────────────────────
+
+    def _preview_gate(self):
+        """The gate of the running loop (a new loop gets a new one)."""
+        gate = self._preview_gate_obj
+        if gate is None or gate.loop is not asyncio.get_running_loop():
+            gate = self._preview_gate_obj = _PreviewGate(ON_DEMAND_LIMIT, self.logger)
+        return gate
+
+    def _preview_pool(self):
+        with AutoThumbnailMixin._auto_thumbnailer_lock:
+            if self._preview_pool_obj is None:
+                self._preview_pool_obj = ThreadPoolExecutor(
+                    ON_DEMAND_LIMIT, thread_name_prefix=POOL_PREFIX
+                )
+            return self._preview_pool_obj
+
+    def _preview_pool_threads(self):
+        pool = self._preview_pool_obj
+        return [] if pool is None else list(getattr(pool, "_threads", ()))
+
+    def _video_placeholder_bytes(self):
+        return video_placeholder_jpeg()
+
+    async def _admit_preview(self, src, dst, is_video):
+        """Make one preview through the view admission (the history route).
+
+        Shares the serve route's flight key; every flight answers the
+        ``ensure`` outcome (or None, or OVER), whichever route started it.
+        """
+        return await self._preview_gate().run(
+            flight_key(Path(dst).parent, (Path(dst).name,)),
+            functools.partial(self._ensure_view, src, dst, is_video),
+            self._preview_pool(),
+        )
+
+    def _ensure_view(self, src, dst, is_video):
+        """``ensure``, reporting a preview removed under us as busy."""
+        outcome = self._auto_thumbnails().ensure(src, dst, is_video)
+        if outcome == READY and not Path(dst).is_file():
+            return BUSY  # removed under us: the fallback this once
+        return outcome
+
+    # ── on first view ─────────────────────────────────────────────────
+
+    def _preview_on_demand(self, root, parts):
+        """The ``ensure`` outcome for the preview *parts* under *root*
+        (``deferred`` when its folder is linked out of the root), or None
+        when no media file previews as *parts*."""
+        located = _thumbnail_source(root, parts)
+        if located is None:
+            return None
+        src, dst, is_video = located
+        if dst is None:
+            return DEFERRED
+        return self._ensure_view(src, dst, is_video)
+
+    def _preview_response(self, outcome, src, dst, is_video):
+        if outcome == READY and dst is not None:
+            return self._file_response(dst)
+        if is_video:
+            return web.Response(
+                body=video_placeholder_jpeg(),
+                content_type="image/jpeg",
+                headers={"Cache-Control": "no-store"},
+            )
+        response = web.FileResponse(src)
+        response.headers["Cache-Control"] = "no-store"
+        return response
+
+    async def _serve_missing_thumbnail(self, output_dirs, filepath):
+        """The made preview, or its fallback; None when *filepath* names no
+        preview of a media file in *output_dirs*.
+
+        The flight answers only the outcome (a history flight may be the one
+        joined); the file it previews is derived here: the preview path is
+        the request's own, and a fallback looks its source up again on the
+        view pool (bounded by it, never the default executor).
+        """
+        if _has_traversal(filepath) or not _is_preview_request(filepath):
+            return None
+        parts = tuple(filepath.split("/"))
+        gate = self._preview_gate()
+        pool = self._preview_pool()
+        for root in output_dirs:
+            root = Path(root)
+            outcome = await gate.run(
+                flight_key(root, parts),
+                functools.partial(self._preview_on_demand, root, parts),
+                pool,
+            )
+            if outcome is None:
+                continue
+            if outcome is OVER:
+                # Past the cap: exact names only, no listing, no write.
+                quick = _quick_source(root, parts)
+                if quick is None:
+                    continue
+                return self._preview_response(BUSY, quick[0], None, quick[1])
+            dst = root.joinpath(*parts)
+            if outcome == READY:
+                return self._file_response(dst)
+            located = await gate.loop.run_in_executor(
+                pool, _thumbnail_source, root, parts
+            )
+            if located is None:
+                continue
+            src, _dst, is_video = located
+            return self._preview_response(outcome, src, None, is_video)
+        return None
+
+    # ── on capture ────────────────────────────────────────────────────
+
+    def queue_capture_thumbnail(self, image_path):
+        """Queue the preview of a new output (the monitor's thread).
+
+        Returns ``queued``, ``fresh`` (its preview is current) or ``skipped``
+        (not a media file under a current root, a link, a preview itself,
+        already queued, or after the app's cleanup). Never writes on the
+        caller's thread.
+        """
+        if self._auto_thumbnails_shut:
+            return "skipped"
+        located = self._capture_target(image_path)
+        if located is None:
+            return "skipped"
+        src, dst, is_video = located
+        if _thumbnail_is_fresh(src, dst):
+            return "fresh"
+        if self._auto_thumbnails().enqueue(src, dst, is_video):
+            return "queued"
+        return "skipped"
+
+    def _capture_target(self, image_path):
+        path = Path(os.path.abspath(str(image_path)))
+        if path.suffix.lower() not in MEDIA_EXTENSIONS or not _regular_file(path):
+            return None
+        folded = os.path.normcase(str(path))
+        for root in self._get_all_output_dirs():
+            root = Path(os.path.abspath(str(root)))
+            base = os.path.normcase(str(root)).rstrip("\\/") + os.sep
+            if not folded.startswith(base) or not _is_within(path, root):
+                continue
+            rel = Path(os.path.relpath(path, root))
+            if "thumbnails" in rel.parts[:-1]:
+                return None
+            dst = root / media_thumbnail_rel(rel)
+            if not _is_within(root / "thumbnails", root) or not _is_within(
+                dst, root / "thumbnails"
+            ):
+                return None
+            return path, dst, path.suffix.lower() in VIDEO_EXTENSIONS
+        return None
+
+    # ── after a scan ──────────────────────────────────────────────────
+
+    async def _thumbnails_after_scan(self, job, output_dirs=None):
+        """Hand a finished scan's missing previews on; returns what it did.
+
+        None (the scan added nothing), ``job`` (the thumbnail job started for
+        ``SCAN_THUMBNAIL_JOB_THRESHOLD`` or more missing previews), ``busy``
+        (a job refused it; the previews are made on view), ``queued`` (fewer,
+        through the worker) or ``none`` (nothing missing). It counts the
+        sources the scan walked (``output_dirs``, narrowed to the job's
+        ``roots``; every source when None). The server log says which.
+        """
+        if job.counters.get("added", 0) <= 0:
+            return None
+        if output_dirs is None:
+            output_dirs = await self._run_in_executor(self._get_all_output_dirs)
+        roots = job.scope.get("roots")
+        if roots:
+            output_dirs = [output_dirs[i] for i in roots if 0 <= i < len(output_dirs)]
+        plan = await self._run_in_executor(self._plan_gallery_thumbnails, output_dirs)
+        missing = len(plan["targets"]) + plan["remaining"]
+        if missing >= SCAN_THUMBNAIL_JOB_THRESHOLD:
+            options = ThumbnailOptions("missing", AUTO_THUMBNAIL_EDGE, "medium")
+            try:
+                self._start_thumbnail_job(options)
+            except JobBusy as busy:
+                self.logger.info(
+                    f"{missing:,} previews missing after the scan; the thumbnail "
+                    f"job did not start while {busy.running.title} runs"
+                )
+                return "busy"
+            self.logger.info(
+                f"{missing:,} previews missing after the scan; thumbnail job started"
+            )
+            return "job"
+        if not missing:
+            return "none"
+        thumbnailer = self._auto_thumbnails()
+        for src, dst, is_video in plan["targets"]:
+            thumbnailer.enqueue(src, dst, is_video)
+        self.logger.info(f"{missing:,} previews missing after the scan; queued")
+        return "queued"
+
+    async def _hand_off_scan_thumbnails(self, job, output_dirs=None):
+        """``_thumbnails_after_scan`` for the scan runner: never raises."""
+        try:
+            return await self._thumbnails_after_scan(job, output_dirs)
+        except asyncio.CancelledError:
+            raise
+        except Exception:
+            self.logger.warning("Previews after the scan failed", exc_info=True)
+            return None
+
+
+# A package reload re-runs this module: stop the workers the old one left.
+_stop_stale_workers()

py/api/thumbnail_jobs.py

@@ -0,0 +1,494 @@
+"""Thumbnail generation as an Activity job (#249 System S5, Task 12).
+
+- ``GET /prompt_manager/images/thumbnails/plan?mode=&size=&quality=`` counts
+  what a run would do over every gallery root and writes nothing:
+  ``{success, targets, fresh, remaining, roots, mode, size, quality}``.
+- ``POST /prompt_manager/jobs/thumbnails {mode, size, quality}`` starts the
+  ``thumbnails`` job (202 ``{job}``). Its admission class is ``guarded``: it
+  never runs beside a record-changing job, and one thumbnail job runs at a
+  time. It stops at a file boundary when cancelled, keeping the previews it
+  wrote. Its outcome groups are ``generated``, ``skipped`` (already fresh)
+  and ``errors``.
+
+``mode`` is ``missing`` (previews that are absent or older than their file)
+or ``rebuild`` (every preview again, fresh ones too). ``size`` is the
+preview's longest edge, 128 to 2048 pixels. ``quality`` picks the encoder
+settings (``THUMBNAIL_ENCODINGS``). Every preview is written next to the old
+one and renamed over it, so nothing is cleared first and a failure keeps the
+old preview; originals are only read.
+
+The legacy gallery stream ``POST /images/generate-thumbnails/progress``
+starts this job (its quality preset still sizes the previews) or attaches to
+the running one. Its first frame, ``job {id, started}``, names that job and
+whether this stream started it (the gallery cancels by that id, and only a
+job it started); then come the job's legacy ``status``/``start``/
+``progress``/``file_error``/``complete`` frames. A client that goes away no
+longer stops the work: the job outlives the request, as a scan does.
+"""
+
+import asyncio
+import functools
+import json
+import time
+from dataclasses import dataclass
+
+from aiohttp import web
+
+from .admin import _plural, _scan_location, _scan_roots, _scan_status_line
+from .images import (
+    THUMBNAIL_ENCODINGS,
+    _BodyTooLarge,
+    _new_thumbnail_stats,
+    _read_json_body,
+    _record_thumbnail_result,
+    _thumbnail_batch_size,
+    _thumbnail_complete_payload,
+    _thumbnail_progress_payload,
+    _worker_threads,
+)
+from .job_routes import busy_response
+from .jobs import THUMBNAIL_GROUP_NAMES, THUMBNAILS_JOB_KIND, JobBusy
+
+THUMBNAIL_JOB_TITLE = "Thumbnail generation"
+THUMBNAIL_MODES = ("missing", "rebuild")
+THUMBNAIL_QUALITIES = tuple(THUMBNAIL_ENCODINGS)
+MIN_THUMBNAIL_EDGE = 128
+MAX_THUMBNAIL_EDGE = 2048
+# The concept's "Preview size limit" default.
+DEFAULT_THUMBNAIL_EDGE = 512
+DEFAULT_THUMBNAIL_OPTIONS = {
+    "mode": "missing",
+    "size": DEFAULT_THUMBNAIL_EDGE,
+    "quality": "medium",
+}
+_OPTIONS_ERROR = (
+    'mode must be "missing" or "rebuild", quality "low", "medium" or "high", '
+    f"and size a whole number of pixels from {MIN_THUMBNAIL_EDGE} "
+    f"to {MAX_THUMBNAIL_EDGE}"
+)
+CANCELLED_ERROR = "Thumbnail generation was cancelled"
+CANCELLED_BEFORE_WRITING = "Cancelled before any preview was written"
+NOTHING_TO_PREVIEW = "No images or videos to preview"
+_MODE_LOG = {"missing": "missing previews", "rebuild": "every preview"}
+# Outcome rows: what happened to each file's preview.
+_DETAILS = {
+    "generated": "Preview written",
+    "skipped": "Preview already up to date",
+}
+_LOG_PREFIXES = {"generated": "+", "errors": "!"}
+
+
+@dataclass(frozen=True)
+class ThumbnailOptions:
+    """One run's choices: the mode, the preview's longest edge and quality."""
+
+    mode: str
+    edge: int
+    quality: str
+
+    @property
+    def size(self):
+        return (self.edge, self.edge)
+
+    @property
+    def encoding(self):
+        return THUMBNAIL_ENCODINGS[self.quality]
+
+    @property
+    def rebuild(self):
+        return self.mode == "rebuild"
+
+    def scope(self):
+        return {"mode": self.mode, "size": self.edge, "quality": self.quality}
+
+
+def _edge(value, *, from_query):
+    """The preview edge as an int, or None when it is not one in range."""
+    if from_query and isinstance(value, str):
+        if not value.isascii() or not value.isdigit() or len(value) > 5:
+            return None
+        value = int(value)
+    if isinstance(value, bool) or not isinstance(value, int):
+        return None
+    return value if MIN_THUMBNAIL_EDGE <= value <= MAX_THUMBNAIL_EDGE else None
+
+
+def thumbnail_options(raw, *, from_query=False):
+    """``(ThumbnailOptions, None)`` from a body or query, or ``(None, message)``.
+
+    Absent fields take ``DEFAULT_THUMBNAIL_OPTIONS``. Query values are
+    strings, so a query ``size`` may be decimal digits; a JSON ``size`` must
+    be an int.
+    """
+    if not isinstance(raw, dict) and not hasattr(raw, "get"):
+        return None, _OPTIONS_ERROR
+    mode = raw.get("mode", DEFAULT_THUMBNAIL_OPTIONS["mode"])
+    quality = raw.get("quality", DEFAULT_THUMBNAIL_OPTIONS["quality"])
+    edge = _edge(raw.get("size", DEFAULT_THUMBNAIL_EDGE), from_query=from_query)
+    if mode not in THUMBNAIL_MODES or quality not in THUMBNAIL_QUALITIES:
+        return None, _OPTIONS_ERROR
+    if edge is None:
+        return None, _OPTIONS_ERROR
+    return ThumbnailOptions(mode, edge, quality), None
+
+
+def _invalid(message):
+    return web.json_response(
+        {"success": False, "code": "invalid-request", "error": message}, status=400
+    )
+
+
+def _frame(event, data):
+    """A legacy stream frame as the job publishes it."""
+    return {"event": event, "data": data}
+
+
+def _finished_line(plan, stats):
+    """A finished job's status line: what it wrote, or that it had nothing to do."""
+    if not plan["targets"]:
+        if plan["skipped"]:
+            return f"No previews to generate · {plan['skipped']:,} already up to date"
+        return NOTHING_TO_PREVIEW
+    parts = [f"{_plural(stats['generated'], 'preview')} written"]
+    if stats["errors"]:
+        parts.append(_plural(len(stats["errors"]), "error"))
+    if plan["remaining"]:
+        parts.append(f"{plan['remaining']:,} more on the next run")
+    return " · ".join(parts)
+
+
+class ThumbnailJobsMixin:
+    """The thumbnail plan, the thumbnail job and the legacy stream on it."""
+
+    async def get_thumbnail_plan(self, request):
+        """What a thumbnail run would do (read-only; capped like a run)."""
+        options, error = thumbnail_options(request.query, from_query=True)
+        if error:
+            return _invalid(error)
+        try:
+            output_dirs = await self._run_in_executor(self._get_all_output_dirs)
+            plan = await self._run_in_executor(
+                self._plan_gallery_thumbnails, output_dirs, rebuild=options.rebuild
+            )
+        except Exception:
+            self.logger.exception("Thumbnail plan failed")
+            return web.json_response(
+                {
+                    "success": False,
+                    "code": None,
+                    "error": "The thumbnail plan could not be read",
+                },
+                status=500,
+            )
+        return web.json_response(
+            {
+                "success": True,
+                "targets": len(plan["targets"]),
+                "fresh": plan["fresh"],
+                "remaining": plan["remaining"],
+                "roots": [self._public_path(root) for root in output_dirs],
+                **options.scope(),
+            },
+            headers={"Cache-Control": "no-store"},
+        )
+
+    async def start_thumbnail_job(self, request):
+        """Start the thumbnail job (202), or 409 naming the job in the way."""
+        body = {}
+        if request.can_read_body:
+            try:
+                body = await _read_json_body(request)
+            except _BodyTooLarge:
+                return web.json_response(
+                    {"success": False, "code": "too-large", "error": "Body too large"},
+                    status=413,
+                )
+            except (ValueError, UnicodeDecodeError):
+                return _invalid(_OPTIONS_ERROR)
+        options, error = thumbnail_options(body)
+        if error:
+            return _invalid(error)
+        try:
+            job = self._start_thumbnail_job(options)
+        except JobBusy as busy:
+            return busy_response(busy)
+        return web.json_response({"success": True, "job": job.summary()}, status=202)
+
+    def _start_thumbnail_job(self, options):
+        """Register the thumbnail job synchronously; raises JobBusy."""
+        return self._jobs.start(
+            THUMBNAILS_JOB_KIND,
+            THUMBNAIL_JOB_TITLE,
+            admission="guarded",
+            cancellable=True,
+            scope=options.scope(),
+            runner=functools.partial(self._run_thumbnail_job, options=options),
+            groups=THUMBNAIL_GROUP_NAMES,
+        )
+
+    # ── the job ───────────────────────────────────────────────────────
+
+    async def _run_thumbnail_job(self, job, options):
+        try:
+            state = await self._thumbnail_job_body(job, options)
+        except Exception:
+            self.logger.exception("Thumbnail generation failed")
+            job.update(status_line="Thumbnail generation stopped on an internal error.")
+            job.log_line("Thumbnails failed · internal error")
+            job.publish(
+                _frame(
+                    "error",
+                    {
+                        "error": "An internal error occurred",
+                        "message": "Thumbnail generation failed. Check server logs.",
+                    },
+                )
+            )
+            state = "failed"
+        job.finish(state)
+
+    async def _thumbnail_job_body(self, job, options):
+        """Plan, then generate in worker batches; returns the final state.
+
+        Every exit sets the job's final status line (I3): what was written,
+        that there was nothing to do, or what a cancel kept.
+        """
+        output_dirs = await self._run_in_executor(self._get_all_output_dirs)
+        if not output_dirs:
+            message = "ComfyUI output directory not found"
+            job.update(status_line=message)
+            job.log_line(f"Thumbnails failed · {message}")
+            job.publish(_frame("error", {"error": message}))
+            return "failed"
+        folders = _plural(len(output_dirs), "output folder")
+        job.scope["output_folders"] = len(output_dirs)
+        job.update(status_line=f"Looking for media files in {folders}")
+        job.log_line(
+            f"Thumbnails started · {folders} · {_MODE_LOG[options.mode]} · "
+            f"{options.edge} px · {options.quality}"
+        )
+        job.publish(
+            _frame(
+                "status",
+                {"phase": "scanning", "message": "Scanning for images and videos..."},
+            )
+        )
+        plan = await self._run_in_executor(
+            self._plan_gallery_thumbnails,
+            output_dirs,
+            rebuild=options.rebuild,
+            stop=lambda: job.cancelled,
+        )
+        if job.cancelled:  # M2: stopped during the walk, before any start frame
+            return self._end_cancelled(job, 0, CANCELLED_BEFORE_WRITING)
+        roots = _scan_roots(output_dirs)
+        self._record_fresh(job, plan, roots)
+        stats, start = await self._generate_thumbnail_targets(job, options, plan, roots)
+        self.invalidate_gallery_cache()
+        generated = _plural(stats["generated"], "preview")
+        if job.cancelled:
+            return self._end_cancelled(
+                job, stats["generated"], f"Cancelled · {generated} written"
+            )
+        payload = _thumbnail_complete_payload(
+            len(plan["targets"]) + plan["skipped"] + plan["remaining"],
+            stats,
+            start,
+            remaining=plan["remaining"],
+            processed=len(plan["targets"]),
+        )
+        job.update(progress=100, status_line=_finished_line(plan, stats))
+        if plan["remaining"]:
+            job.log_line(
+                f"{plan['remaining']:,} more previews are left for the next run"
+            )
+        errors = _plural(len(stats["errors"]), "error")
+        job.log_line(f"Thumbnails finished · {generated} written · {errors}")
+        self.logger.info(payload["message"])
+        job.publish(_frame("complete", payload))
+        return "finished"
+
+    @staticmethod
+    def _end_cancelled(job, generated, status_line):
+        """Close a cancelled job: its status line, log line and error frame."""
+        job.update(status_line=status_line)
+        job.log_line(f"Thumbnails cancelled · {_plural(generated, 'preview')} written")
+        job.publish(
+            _frame("error", {"error": CANCELLED_ERROR, "message": CANCELLED_ERROR})
+        )
+        return "cancelled"
+
+    def _record_fresh(self, job, plan, roots):
+        """List the already fresh previews under Skipped (no log line each)."""
+        rows = []
+        for src in plan["fresh_files"]:
+            path, label, index = _scan_location(src, roots)
+            rows.append(
+                {
+                    "path": path,
+                    "root": label,
+                    "root_index": index,
+                    "detail": _DETAILS["skipped"],
+                }
+            )
+        job.record_counted("skipped", plan["skipped"], rows)
+        if plan["skipped"]:
+            job.log_line(f"= {plan['skipped']:,} previews already up to date")
+
+    async def _generate_thumbnail_targets(self, job, options, plan, roots):
+        """Generate ``plan``'s targets in worker batches, publishing each file."""
+        targets = plan["targets"]
+        total = len(targets)
+        video_count = sum(1 for _, _, is_video in targets if is_video)
+        message = (
+            f"Found {total - video_count} images and {video_count} videos to process"
+        )
+        if plan["skipped"]:
+            message += f" ({plan['skipped']} already have thumbnails)"
+        job.update(total=total)
+        if total:  # never "Processing file 0 of 0" (I3)
+            job.update(status_line=_scan_status_line(0, total, 0, 0))
+        job.publish(
+            _frame(
+                "start",
+                {
+                    "total_images": total,
+                    "skipped": plan["skipped"],
+                    "remaining": plan["remaining"],
+                    "phase": "processing",
+                    "image_count": total - video_count,
+                    "video_count": video_count,
+                    "message": message,
+                },
+            )
+        )
+        stats = _new_thumbnail_stats()
+        stats["skipped"] = plan["skipped"]
+        start = time.monotonic()
+        batch_size = _thumbnail_batch_size(_worker_threads())
+        generate = {
+            "encoding": options.encoding,
+            "rebuild": options.rebuild,
+            "stop": lambda: job.cancelled,
+        }
+        index = 0
+        batches = 0
+        for batch_start in range(0, total, batch_size):
+            if job.cancelled:
+                break
+            batch = targets[batch_start : batch_start + batch_size]
+            results = await self._run_in_executor(
+                self._generate_batch, batch, options.size, **generate
+            )
+            for (src, _dst, _video), result in zip(batch, results):
+                if result["action"] == "cancelled":
+                    continue
+                index += 1
+                _record_thumbnail_result(stats, result)
+                self._record_thumbnail_outcome(job, src, result, roots)
+                job.update(processed=index, progress=index * 100 // total)
+                if result["action"] == "error":
+                    job.publish(
+                        _frame(
+                            "file_error",
+                            {"file": result["file"], "error": result["error"]},
+                        )
+                    )
+                job.publish(
+                    _frame(
+                        "progress",
+                        _thumbnail_progress_payload(index, total, stats, result, start),
+                    )
+                )
+            batches += 1
+            job.update(
+                status_line=_scan_status_line(
+                    index, total, batches, time.monotonic() - start
+                )
+            )
+            await asyncio.sleep(0)  # let other requests run between batches
+        return stats, start
+
+    @staticmethod
+    def _record_thumbnail_outcome(job, src, result, roots):
+        path, label, index = _scan_location(src, roots)
+        where = f"{label}/{path}" if label else path
+        if result["action"] == "error":
+            group, detail = "errors", result.get("error") or "failed"
+        elif result["action"] == "skipped":
+            group, detail = "skipped", _DETAILS["skipped"]
+        else:
+            group, detail = "generated", _DETAILS["generated"]
+        row = {"path": path, "root": label, "root_index": index, "detail": detail}
+        if group == "skipped":
+            job.record_counted(group, 1, [row])
+            return
+        job.record(group, row, f"{_LOG_PREFIXES[group]} {where} · {detail}")
+
+    # ── the legacy gallery stream ─────────────────────────────────────
+
+    async def generate_thumbnails_with_progress(self, request):
+        """The gallery's thumbnail stream (POST, server-sent events).
+
+        Starts the thumbnail job with the gallery's quality preset as the
+        preview size (missing previews, medium encoding, as before), or
+        attaches to the running thumbnail job, and streams its frames. A
+        legacy hold (the blocking generate or clear) or a record-changing
+        job refuses it with the stream's error frame. The job goes on when
+        the client goes away.
+        """
+        thumbnail_size = self._thumbnail_size(request.query.get("quality", "medium"))
+        response = web.StreamResponse(
+            status=200,
+            headers={
+                "Content-Type": "text/event-stream",
+                "Cache-Control": "no-cache",
+                "Connection": "keep-alive",
+            },
+        )
+        await response.prepare(request)
+        # M3: read the registry after the last await, so a job that started
+        # while the headers went out is attached to, not refused.
+        job, started, busy = self._legacy_stream_job(thumbnail_size)
+        if busy is not None:
+            message = f"{busy.title} already running"
+            await self._emit_sse(
+                response, "error", {"error": message, "message": message}
+            )
+            return response
+        # No await between the start and the subscription: the lead frame is
+        # sent after it, so the stream misses none of the runner's frames.
+        events = job.events(lead=_frame("job", {"id": job.id, "started": started}))
+        try:
+            async for payload in events:
+                frame = (
+                    f"event: {payload['event']}\ndata: {json.dumps(payload['data'])}"
+                )
+                await response.write(f"{frame}\n\n".encode("utf-8"))
+        except Exception as exc:  # a write failed: the client went away
+            self.logger.info(
+                f"Thumbnail progress client disconnected ({exc}); "
+                "the thumbnails continue in the background"
+            )
+        finally:
+            # Also when aiohttp cancels the handler on disconnect: the job is
+            # its own task and goes on.
+            await events.aclose()
+        return response
+
+    def _legacy_stream_job(self, thumbnail_size):
+        """``(job, started, busy)`` for the legacy stream, synchronously.
+
+        Attaches to the running cancellable thumbnail job, or starts one with
+        the gallery's preset as the size (missing previews, medium encoding,
+        as the gallery always wrote), or names the job in the way.
+        """
+        running = self._jobs.active(THUMBNAILS_JOB_KIND)
+        if running is not None and running.cancellable:
+            return running, False, None
+        busy = self._jobs.blocker(THUMBNAILS_JOB_KIND, admission="guarded")
+        if busy is not None:
+            return None, False, busy
+        # blocker() said free and nothing awaited since, so the start is admitted.
+        options = ThumbnailOptions("missing", thumbnail_size[0], "medium")
+        return self._start_thumbnail_job(options), True, None

py/api/workflow_storage.py

@@ -0,0 +1,277 @@
+"""Store each workflow once: the migration and Reclaim space jobs (#249 S6).
+
+Reviewed like every maintenance operation (``POST /prompt_manager/
+maintenance/preview {operation:"workflow_storage"}``, a ``Plan`` with the
+records to rewrite, the backup and the disk check) and applied with its token
+(``POST /prompt_manager/maintenance/apply``). The apply starts:
+
+1. **Store each workflow once**: kind ``maintenance``, admission ``records``,
+   cancellable, groups ``changed``/``protected``/``errors``. It always takes
+   a verified pre-migration backup, whatever the safety backup policy, named
+   ``<db>.pre-workflow-storage_<stamp>`` outside the pruned ``<db>.backup_*``
+   pool and kept until the user removes it (``backup-failed`` ends the job
+   with nothing rewritten). It then rewrites old-form rows batch by batch
+   (``database/workflow_storage.py``): one transaction per batch of at most
+   ``BATCH_SECONDS``, then ``BATCH_PAUSE`` with the write lock free, progress
+   after each, Cancel honoured between batches. A row that cannot be
+   rewritten stays inline and is listed in ``errors``; any other failure
+   rolls its batch back whole and fails the job. Every record still reads as
+   before, and the next apply continues from the first old-form row.
+2. **Reclaim space**, once every record was looked at: kind ``maintenance``,
+   admission ``exclusive`` (it runs alone; VACUUM holds the write lock), not
+   cancellable. It checks the disk for VACUUM's copies of the live pages
+   (``disk_plan``), then sweeps, runs VACUUM and truncates the WAL. When
+   another job holds the registry, it does not start and the finished
+   migration says so and names Optimize database.
+
+Only the database changes: no image or video file is opened.
+"""
+
+import asyncio
+import sqlite3
+
+from .jobs import MAINTENANCE_JOB_KIND, JobBusy
+from .maintenance import (
+    MAINTENANCE_GROUPS,
+    PRE_MIGRATION_LABEL,
+    SafetyBackupFailed,
+    backup_failed_message,
+    _fail_job,
+    _megabytes,
+    disk_plan,
+    short_space_message,
+    take_safety_backup,
+)
+
+try:
+    from ...database import workflow_storage
+    from ...database.health import readonly_uri
+except ImportError:
+    from database import workflow_storage
+    from database.health import readonly_uri
+
+OPERATION = workflow_storage.OPERATION
+TITLE = "Store each workflow once"
+RECLAIM_TITLE = "Reclaim space"
+REWRITE_FAILED = (
+    "Storing workflows once stopped on an error. The batch in progress was "
+    "rolled back; every record reads as before. Run it again to continue."
+)
+RECLAIM_FAILED = "Reclaim space failed; the database is unchanged."
+ROW_KEPT = "could not be rewritten; it keeps its inline text and reads as before"
+
+
+NOTHING_LEFT = "Every image record already stores its workflow once"
+
+
+def _plural(count, one, many=None):
+    return f"{count:,} {one if count == 1 else (many or one + 's')}"
+
+
+def live_bytes(db_path):
+    """The pages VACUUM copies: page_count minus the free list (blocking)."""
+    conn = sqlite3.connect(readonly_uri(db_path), uri=True)
+    try:
+        page_size = conn.execute("PRAGMA page_size").fetchone()[0]
+        pages = conn.execute("PRAGMA page_count").fetchone()[0]
+        free = conn.execute("PRAGMA freelist_count").fetchone()[0]
+        return page_size * (pages - free)
+    finally:
+        conn.close()
+
+
+class WorkflowStorageRoutesMixin:
+    """Mixin for the maintenance routes: the workflow storage jobs."""
+
+    def _start_workflow_storage(self, db, backup):
+        """Start the migration job (raises JobBusy); the token is the caller's."""
+        return self._jobs.start(
+            MAINTENANCE_JOB_KIND,
+            TITLE,
+            admission="records",
+            cancellable=True,
+            scope={"operation": OPERATION},
+            groups=MAINTENANCE_GROUPS,
+            runner=lambda job: self._run_workflow_storage(job, db, backup),
+        )
+
+    async def _run_workflow_storage(self, job, db, _backup_on):
+        # Always backed up first, whatever the policy: the way back. The first
+        # apply keeps a pre-migration backup; later ones (after a Cancel, a
+        # rerun) take a pooled safety backup, so only one copy is never pruned.
+        counts = await self._run_in_executor(db.workflow_storage_status)
+        if not counts["rows"]:
+            job.log_line(NOTHING_LEFT)
+            job.update(progress=100, status_line=NOTHING_LEFT)
+            return
+        kept = await self._run_in_executor(
+            workflow_storage.pre_migration_backups, db.model.db_path
+        )
+        label = "backup" if kept else PRE_MIGRATION_LABEL
+        backup = await self._backup_before_rewrite(job, db, label, kept)
+        if backup is None:
+            return
+        try:
+            done = await self._rewrite_workflows(job, db, backup)
+        finally:
+            if kept:
+                await self._prune_backup_pool(job, db)
+        if done:
+            job.finish("finished")
+            self._start_reclaim_space(job, db)
+
+    async def _backup_before_rewrite(self, job, db, label, kept):
+        """The verified backup's status note, or None (the job failed).
+
+        ``label`` ``pre-workflow-storage``: kept until the user removes it.
+        ``backup``: a pooled safety backup, while the earliest kept
+        pre-migration backup (``kept[0]``) stays. A failure ends the job
+        ``backup-failed``, nothing changed.
+        """
+        noun = "safety backup" if kept else "pre-migration backup"
+        job.update(status_line=f"Taking the {noun}")
+        job.log_line(f"Taking the {noun} of the database")
+        try:
+            name = await self._run_in_executor(take_safety_backup, db.model, label)
+        except SafetyBackupFailed as error:
+            self.logger.error("Backup before the rewrite failed", exc_info=True)
+            _fail_job(job, "backup-failed", backup_failed_message(error))
+            return None
+        if kept:
+            job.log_line(
+                f"Safety backup {name} verified; the pre-migration backup "
+                f"{kept[0]} is kept"
+            )
+            return f"Safety backup {name}"
+        job.log_line(
+            f"Pre-migration backup {name} verified; it is kept until you remove it"
+        )
+        return f"Pre-migration backup {name}"
+
+    async def _rewrite_workflows(self, job, db, backup):
+        """Rewrite every old-form row; True when every row was looked at."""
+        counts = await self._run_in_executor(db.workflow_storage_status)
+        job.update(total=counts["rows"], processed=0)
+        job.update(status_line="Rewriting image records")
+        job.log_line(f"{_plural(counts['rows'], 'image record')} to rewrite")
+        rows = protected = errors = 0
+        last_id = 0
+        while True:
+            if job.cancelled:
+                return self._stopped(job, rows, backup)
+            try:
+                batch = await self._run_in_executor(
+                    db.migrate_workflow_batch,
+                    last_id,
+                    workflow_storage.BATCH_ROWS,
+                    workflow_storage.BATCH_BYTES,
+                )
+            except Exception:
+                self.logger.exception("Storing workflows once failed")
+                _fail_job(job, None, REWRITE_FAILED)
+                return False
+            if batch["done"]:
+                break
+            last_id = batch["last_id"]
+            rows += batch["rows"]
+            protected += batch["protected"]
+            errors += len(batch["errors"])
+            job.record_counted("changed", batch["rows"])
+            job.record_counted("protected", batch["protected"])
+            for image_id in batch["errors"]:
+                job.record(
+                    "errors",
+                    {"path": "", "detail": f"Image record {image_id}: {ROW_KEPT}"},
+                    f"! Image record {image_id}: {ROW_KEPT}",
+                )
+            total = max(job.total, rows)
+            job.update(
+                processed=rows,
+                total=total,
+                progress=100 * rows // total if total else 100,
+                status_line=f"Rewrote {_plural(rows, 'image record')}",
+            )
+            job.publish({"type": "progress", "processed": rows, "total": total})
+            # Leave the write lock free between batches (monitor, run counts).
+            await asyncio.sleep(workflow_storage.BATCH_PAUSE)
+        removed = await self._run_in_executor(db.sweep_workflow_documents)
+        line = f"Rewrote {_plural(rows, 'image record')}"
+        if protected:
+            line += f"; {_plural(protected, 'stored value')} not text, kept as is"
+        if errors:
+            kept = "keeps its" if errors == 1 else "keep their"
+            line += (
+                f"; {_plural(errors, 'image record')} could not be rewritten and "
+                f"{kept} inline text"
+            )
+        job.log_line(line)
+        if removed:
+            job.log_line(f"Removed {_plural(removed, 'unused workflow document')}")
+        job.update(progress=100, status_line=f"{line} · {backup}")
+        return True
+
+    def _stopped(self, job, rows, backup):
+        line = (
+            f"Stopped after {_plural(rows, 'image record')}; "
+            "run it again to continue"
+        )
+        job.log_line(line)
+        job.update(status_line=f"{line} · {backup}")
+        job.finish("cancelled")
+        return False
+
+    def _start_reclaim_space(self, finished, db):
+        """Start Reclaim space after the migration ``finished``, or say why not."""
+        try:
+            self._jobs.start(
+                MAINTENANCE_JOB_KIND,
+                RECLAIM_TITLE,
+                admission="exclusive",
+                cancellable=False,
+                scope={"operation": OPERATION},
+                groups=MAINTENANCE_GROUPS,
+                runner=lambda job: self._run_reclaim_space(job, db),
+            )
+        except JobBusy as busy:
+            line = (
+                f"Reclaim space did not start: {busy.running.title} is running. "
+                "Run Optimize database to give the space back."
+            )
+            finished.log_line(line)
+            finished.update(status_line=f"{finished.status_line} · {line}")
+            return
+        finished.log_line("Reclaim space started")
+
+    async def _run_reclaim_space(self, job, db):
+        job.update(status_line="Checking free disk space")
+        try:
+            live = await self._run_in_executor(live_bytes, db.model.db_path)
+            _disk, short, _size = await self._run_in_executor(
+                disk_plan,
+                db.model.db_path,
+                backup=False,
+                vacuum=True,
+                vacuum_bytes=live,
+            )
+        except (OSError, sqlite3.Error):
+            self.logger.exception("Could not measure the disk space for Reclaim space")
+            return _fail_job(job, None, RECLAIM_FAILED)
+        if short is not None:
+            return _fail_job(job, "insufficient-space", short_space_message(short))
+        job.update(status_line="Reclaiming space (the database is locked)")
+        job.log_line("Rebuilding the database file (VACUUM)")
+        try:
+            result = await self._run_in_executor(db.reclaim_space)
+        except Exception:
+            self.logger.exception("Reclaim space failed")
+            return _fail_job(job, None, RECLAIM_FAILED)
+        saved = max(0, result["before_bytes"] - result["after_bytes"])
+        line = (
+            f"Reclaimed {_megabytes(saved)} · the database is now "
+            f"{_megabytes(result['after_bytes'])}"
+        )
+        if result["checkpoint_busy"]:
+            line += " (the rest after the next restart)"
+        job.record_counted("changed", 1)
+        job.log_line(line)
+        job.update(progress=100, status_line=line)

py/config.py

@@ -659,6 +659,10 @@ class PromptManagerConfig:
     DEFAULT_WORKER_THREADS = max(1, (os.cpu_count() or 1) // 2)
     WORKER_THREADS = DEFAULT_WORKER_THREADS
 
+    # Maintenance: a reviewed apply takes a verified safety backup of the
+    # database first (System › Backup & export › Recovery policy, #249 S6).
+    SAFETY_BACKUP = True
+
     @classmethod
     def max_worker_threads(cls) -> int:
         """Largest accepted worker-thread count: the detected core count."""
@@ -713,6 +717,7 @@ class PromptManagerConfig:
                 "auto_backup_interval": cls.AUTO_BACKUP_INTERVAL,
                 "worker_threads": cls.WORKER_THREADS,
             },
+            "maintenance": {"safety_backup": cls.SAFETY_BACKUP},
             "gallery": GalleryConfig.get_config(),
             "integrations": IntegrationConfig.get_config(),
             "autotag": AutotagConfig.get_config(),
@@ -848,6 +853,12 @@ class PromptManagerConfig:
                 performance["worker_threads"], cls.WORKER_THREADS
             )
 
+        maintenance = new_config.get("maintenance", {})
+        if isinstance(maintenance, dict) and isinstance(
+            maintenance.get("safety_backup"), bool
+        ):
+            cls.SAFETY_BACKUP = maintenance["safety_backup"]
+
         # Update gallery config
         if "gallery" in new_config:
             GalleryConfig.update_config(new_config["gallery"])

tests/browser/server.py

@@ -386,6 +386,8 @@ def create_app(root: Path) -> web.Application:
     async def reset(request: web.Request) -> web.Response:
         nonlocal history_fixture
         request_holds.release()
+        # No preview queued by the last test may land in the new fixture.
+        await api._run_in_executor(api.close_auto_thumbnails)
         library_editor_commands.reset()
         data = await request.json() if request.can_read_body else {}
         if not isinstance(data, dict):

tests/browser/system-concept.spec.mjs

@@ -775,3 +775,131 @@ test('add source comparison detects mutations', async ({ page, request }) => {
     await undoStyle();
     expect(diffInventory(expected, await page.evaluate(measureAddSource))).toEqual([]);
 });
+
+/**
+ * Runs in the page: the open "Thumbnail maintenance" dialog (the concept's `thumbnail-review`, app.js 1433–1437): its
+ * title, the body's tree, each `.field` (label, control, options, selection, limits), the `p.small` line and the
+ * actions, with the concept's geometry and tokens. The concept has no `p.small`, so a probe one stands in for its style.
+ */
+function measureThumbnailReview() {
+    const dialog = document.querySelector('dialog#modal[open]');
+    const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
+    const name = node => [node.localName, ...[...node.classList].sort()].join('.');
+    const text = node => node?.textContent.trim() ?? null;
+    const body = dialog.querySelector('#modal-body');
+    const fields = [...body.querySelectorAll(':scope > .field')].map(field => {
+        const label = field.querySelector(':scope > label');
+        const control = field.querySelector(':scope > select, :scope > input');
+        return {
+            tree: [...field.children].map(name),
+            ...css(field, 'display', 'flexDirection', 'rowGap', 'marginBottom'),
+            label: { text: text(label), for: label.htmlFor, ...css(label, 'fontSize', 'fontWeight', 'color') },
+            control: { id: control.id, tag: control.localName, type: control.type,
+                options: control.options ? [...control.options].map(option => option.textContent) : null,
+                selected: control.options ? control.selectedIndex : null,
+                value: control.localName === 'input' ? control.value : null,
+                min: control.getAttribute('min'), max: control.getAttribute('max'),
+                ...css(control, 'height', 'paddingLeft', 'fontSize', 'color', 'backgroundColor', 'borderTopColor', 'borderTopLeftRadius') },
+        };
+    });
+    let line = body.querySelector(':scope > p.small');
+    const probe = !line;
+    if (probe) { line = document.createElement('p'); line.className = 'small'; body.append(line); }
+    const plan = { text: probe ? null : text(line), role: probe ? null : line.getAttribute('role'),
+        ...css(line, 'fontSize', 'color', 'marginTop', 'marginBottom') };
+    if (probe) line.remove();
+    const actions = body.querySelector(':scope > .dialog-actions');
+    const rect = dialog.getBoundingClientRect();
+    return {
+        title: text(dialog.querySelector('#modal-title')),
+        tree: [...body.children].map(name),
+        fields,
+        plan,
+        actions: { tree: [...actions.children].map(name), ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
+            buttons: [...actions.children].map(button => ({ text: text(button), className: button.className,
+                ...css(button, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') })) },
+        width: Math.round(rect.width),
+    };
+}
+
+/**
+ * D99: the planned counts line (`p.small[role=status]`) before the actions, from the server's plan for the chosen
+ * options, and the primary "Start" (the concept's "Simulate thumbnail job").
+ */
+function mapThumbnailConcept(reference, plan) {
+    const [cancel, start] = reference.actions.buttons;
+    const tree = [...reference.tree];
+    tree.splice(tree.indexOf('div.dialog-actions'), 0, 'p.small');
+    return { ...reference, tree,
+        plan: { ...reference.plan, role: 'status',
+            text: `${number(plan.targets)} ${plan.targets === 1 ? 'preview' : 'previews'} to generate · ${number(plan.fresh)} already up to date.` },
+        actions: { ...reference.actions, buttons: [cancel, { ...start, text: 'Start' }] } };
+}
+
+async function conceptThumbnailReview(page, theme) {
+    await conceptSources(page, theme, { notice: false });
+    const row = page.locator('#main .operation').filter({ has: page.getByRole('heading', { name: 'Thumbnail generation', exact: true }) });
+    await row.getByRole('button', { name: 'Review options', exact: true }).click();
+    await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Thumbnail maintenance');
+    await page.mouse.move(0, 0);
+    return page.evaluate(measureThumbnailReview);
+}
+
+/** Production's dialog from Sources' Review options or Overview's Review thumbnails, once its plan line answered. */
+async function productionThumbnailReview(page, theme, entry) {
+    if (entry === 'sources') {
+        await productionSources(page, theme, { notice: false });
+        const row = page.locator('#workspace-main .operation').filter({ has: page.getByRole('heading', { name: 'Thumbnail generation', exact: true }) });
+        await row.getByRole('button', { name: 'Review options', exact: true }).click();
+    } else {
+        await openProduction(page, 'system/overview', theme);
+        await page.locator('#workspace-main').getByRole('button', { name: 'Review thumbnails', exact: true }).click();
+    }
+    await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Thumbnail maintenance');
+    await expect(page.locator('dialog#modal[open] #modal-body > p.small')).toHaveText(/ already up to date\.$/);
+    await page.mouse.move(0, 0);
+    return page.evaluate(measureThumbnailReview);
+}
+
+const thumbnailPlan = request => serverJson(request, '/prompt_manager/images/thumbnails/plan?mode=missing&size=512&quality=medium');
+
+for (const entry of ['sources', 'overview']) for (const theme of THEMES) for (const [width, height] of SIZES) {
+    test(`thumbnail dialog matches the concept from ${entry} ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
+        await page.setViewportSize({ width, height });
+        await watchedSources(request);
+        const plan = await thumbnailPlan(request);
+        const concept = await conceptThumbnailReview(page, theme);
+        await page.screenshot({ path: testInfo.outputPath(`concept-thumbnail-review-${theme}-${width}.png`) });
+        const production = await productionThumbnailReview(page, theme, entry);
+        await page.screenshot({ path: testInfo.outputPath(`production-thumbnail-review-${entry}-${theme}-${width}.png`) });
+        expect(diffInventory(mapThumbnailConcept(concept, plan), production)).toEqual([]);
+    });
+}
+
+test('thumbnail dialog comparison detects mutations', async ({ page, request }) => {
+    await page.setViewportSize({ width: 1600, height: 1100 });
+    await watchedSources(request);
+    const plan = await thumbnailPlan(request);
+    const concept = await conceptThumbnailReview(page, 'dark');
+    const expected = mapThumbnailConcept(concept, plan);
+    const production = await productionThumbnailReview(page, 'dark', 'sources');
+    expect(diffInventory(expected, production)).toEqual([]);
+    // Unmapped, the concept differs only where D99 is recorded.
+    expect(diffInventory(concept, production).map(item => item.path).sort())
+        .toEqual(['actions.buttons.1.text', 'plan.role', 'plan.text', 'tree.3', 'tree.4', 'tree.length']);
+    const paths = async () => diffInventory(expected, await page.evaluate(measureThumbnailReview)).map(item => item.path);
+    // A changed option label, a swapped option order, a swapped action order and a changed style.
+    await page.evaluate(() => { document.querySelector('#thumb-quality option').textContent = 'Medium'; });
+    expect(await paths()).toEqual(['fields.1.control.options.0']);
+    await page.evaluate(() => { document.querySelector('#thumb-quality option').textContent = 'Medium · optimized'; });
+    await page.evaluate(() => { const select = document.querySelector('#thumb-mode'); select.prepend(select.lastElementChild); });
+    expect(await paths()).toEqual(expect.arrayContaining(['fields.0.control.options.0', 'fields.0.control.options.1']));
+    await page.evaluate(() => { const select = document.querySelector('#thumb-mode'); select.prepend(select.lastElementChild); select.selectedIndex = 0; });
+    await page.evaluate(() => { const actions = document.querySelector('dialog#modal[open] .dialog-actions'); actions.prepend(actions.lastElementChild); });
+    expect(await paths()).toEqual(expect.arrayContaining(['actions.buttons.0.text', 'actions.buttons.1.text']));
+    await page.evaluate(() => { const actions = document.querySelector('dialog#modal[open] .dialog-actions'); actions.prepend(actions.lastElementChild); });
+    const undoStyle = await mutate(page, { css: 'dialog#modal #thumb-limit { border-color: rgb(255, 0, 0); } dialog#modal #modal-body > p.small { color: rgb(255, 0, 0); }' });
+    expect(await paths()).toEqual(expect.arrayContaining(['fields.2.control.borderTopColor', 'plan.color']));
+    await undoStyle();
+    expect(diffInventory(expected, await page.evaluate(measureThumbnailReview))).toEqual([]);
+});

tests/browser/workspace-activity.spec.mjs

@@ -54,8 +54,8 @@ async function step(request, id, data) {
     await expect(await request.post(`/__test__/jobs/${id}/step`, { data })).toBeOK();
 }
 
-async function finish(request, id, state = 'finished') {
-    const response = await request.post(`/__test__/jobs/${id}/finish`, { data: { state } });
+async function finish(request, id, state = 'finished', extra = {}) {
+    const response = await request.post(`/__test__/jobs/${id}/finish`, { data: { state, ...extra } });
     await expect(response).toBeOK();
     return (await response.json()).job;
 }
@@ -1008,3 +1008,96 @@ test('the report download names the kind', async ({ page, request }) => {
     expect(report.outcomes.tagged.rows.map(row => [row.path, row.detail])).toEqual([['sub/a.png', 'Added 1 tag']]);
     await expect(toast(page)).toHaveText('Job report downloaded.');
 });
+
+// D111 (System S5): each kind's own descriptor. A thumbnail job counts Generated (green), Skipped and Errors (red),
+// its pill says "Generating thumbnails" and its toasts "Thumbnails finished · N generated · M errors".
+const THUMB_JOB = { kind: 'thumbnails', total: 10, output_folders: 2 };
+
+test('a thumbnail job shows Generated Skipped Errors, its pill verb and its toasts', async ({ page, request }) => {
+    await page.setViewportSize({ width: 1600, height: 1100 });
+    const job = await startJob(request, THUMB_JOB);
+    await step(request, job.id, { processed: 4, records: [
+        { group: 'generated', path: 'sub/a.png', root: 'output', detail: 'Preview written' },
+        { group: 'skipped', path: 'b.png', root: 'input', detail: 'Preview already up to date' },
+        { group: 'errors', path: 'broken.png', root: 'output', detail: 'Not a valid image file' },
+    ] });
+    await page.goto(`${workspace}#/images`);
+    await expect(pill(page)).toHaveText('Generating thumbnails · 40% · 4 / 10');
+    await page.goto(`${workspace}#/activity`);
+    await expect(items(page).first().locator('p')).toHaveText(/^Started \d\d:\d\d · 2 output folders · closing this view does not cancel it$/);
+    await expect(items(page).first().locator('progress')).toHaveAttribute('value', '40');
+    await openJob(page, job.id);
+    await expect(heading(page).locator('h1')).toHaveText('Thumbnail generation');
+    await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report', 'Cancel job']);
+    expect(await tileValues(page)).toEqual([['stat added', 'Generated', '1'], ['stat', 'Skipped', '1'], ['stat bad', 'Errors', '1']]);
+    await expect(outcomeButtons(page)).toHaveText(['Generated1', 'Errors1', 'Skipped1']);
+    await expect(fileRows(page).first().locator('.path')).toHaveText('sub/a.png');
+    expect(await fileRows(page).first().evaluate(node => node.localName)).toBe('button');
+    await outcomeButtons(page).filter({ hasText: 'Errors' }).click();
+    expect(await fileRows(page).first().evaluate(node => node.localName)).toBe('div');
+    await expect(scanLog(page).locator('summary')).toHaveText('● Job log — raw detail, click to show');
+
+    await finish(request, job.id);
+    await expect(toast(page)).toHaveText('Thumbnails finished · 1 generated · 1 error');
+    await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d · 1 generated · 1 error$/);
+    await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
+});
+
+test('a held legacy job and the health check show no tiles, no Results and no made-up percentage', async ({ page, request }) => {
+    await page.setViewportSize({ width: 1600, height: 1100 });
+    const response = await request.post('/__test__/jobs/hold',
+        { data: { kind: 'thumbnails', title: 'Thumbnail clear (gallery page)', admission: 'guarded' } });
+    await expect(response).toBeOK();
+    const held = (await response.json()).job;
+    await page.goto(`${workspace}#/images`);
+    // D112: the pill names the held operation and shows no percentage; its bar is indeterminate, not an empty fill
+    // (S5 review M5), and stays hidden from assistive tech, which reads the label.
+    await expect(pill(page)).toHaveText('Thumbnail clear (gallery page)');
+    await expect(pill(page)).toHaveAttribute('aria-label', 'Thumbnail clear (gallery page)');
+    await expect(pill(page).locator('.activity-pill-bar')).toHaveClass('activity-pill-bar indeterminate');
+    await expect(pill(page).locator('.activity-pill-bar')).toHaveAttribute('aria-hidden', 'true');
+    expect(await pill(page).locator('.activity-pill-bar i').evaluate(node => node.style.width)).toBe('');
+    await page.goto(`${workspace}#/activity`);
+    const item = items(page).filter({ has: page.getByRole('link', { name: /Thumbnail clear \(gallery page\)$/ }) });
+    await expect(item.locator('p')).toHaveText(/^Started \d\d:\d\d · closing this view does not cancel it$/);
+    await expect(item.locator('progress')).toHaveCount(1);
+    await expect(item.locator('progress')).not.toHaveAttribute('value');
+    await expect(item.getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(0);
+    await item.getByRole('link').click();
+    await expect(heading(page).locator('h1')).toHaveText('Thumbnail clear (gallery page)');
+    await expect(progressSection(page).locator('progress')).not.toHaveAttribute('value');
+    await expect(progressSection(page).locator('dl.stat-row')).toHaveCount(0);
+    await expect(resultsSection(page)).toHaveCount(0);
+    await expect(scanLog(page)).toHaveCount(1);
+    await finish(request, held.id);
+    await expect(toast(page)).toHaveText('Thumbnail clear (gallery page) finished.');
+    await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d$/);
+    await expect(progressSection(page).locator('progress')).toHaveAttribute('value', '100');
+
+    // The health check (a read job of its own kind): its status line, no tiles, no Cancel.
+    const health = await startJob(request, { kind: 'health', total: 0 });
+    await openJob(page, health.id);
+    await expect(heading(page).locator('h1')).toHaveText('Health check');
+    await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Checking the database and image links');
+    await expect(progressSection(page).locator('progress')).not.toHaveAttribute('value');
+    await expect(progressSection(page).locator('dl.stat-row')).toHaveCount(0);
+    await expect(resultsSection(page)).toHaveCount(0);
+    await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
+    await expect(pill(page)).toHaveText('Health check');
+    await finish(request, health.id);
+    await expect(toast(page)).toHaveText('Health check finished.');
+});
+
+// S5 review I3: a thumbnail job with nothing to generate ends on the server's final line ("No previews to generate · N
+// already up to date"), never on a stale "Processing file 0 of 0" under the Completed chip.
+test('a finished thumbnail job with nothing to do says so', async ({ page, request }) => {
+    await page.setViewportSize({ width: 1600, height: 1100 });
+    const job = await startJob(request, { ...THUMB_JOB, total: 0 });
+    await step(request, job.id, { total: 0, processed: 0 }); // the stale line an older server left
+    await openJob(page, job.id);
+    const line = progressSection(page).locator('.panel-heading > span.small');
+    await finish(request, job.id, 'finished', { status_line: 'No previews to generate · 24 already up to date' });
+    await expect(line).toHaveText('No previews to generate · 24 already up to date');
+    await expect(page.getByText('Processing file 0 of 0')).toHaveCount(0);
+    await expect(progressSection(page).locator('progress')).toHaveAttribute('value', '100');
+});

tests/browser/workspace-images.spec.mjs

@@ -398,18 +398,20 @@ test('preview uses actual Viewer.js and cleans up when closing or leaving Images
 test('real video media has a usable card and native playback cleans up on close and navigation', async ({ page, request }) => {
     await expect(await request.post('/__test__/reset', { data: { include_video: true } })).toBeOK();
     const media = await (await request.get('/prompt_manager/images/output?limit=40')).json();
-    expect(media.images.find(image => image.filename === 'clip.webm').thumbnail_url).toBeNull();
+    // #249: a video with no preview yet still lists its preview URL; the server makes the frame on first view.
+    expect(media.images.find(image => image.filename === 'clip.webm').thumbnail_url).toMatch(/\/thumbnails\/clip_thumb\.jpg\?root=0/);
     await page.goto(workspace);
     await refine(page, { Media: 'video' });
     await expect(cards(page)).toHaveCount(1);
     const card = cards(page).first();
     await expect(card).toHaveAttribute('data-image-key', '0:clip.webm');
-    await expect(card.locator('img')).toHaveCount(0);
     await expect(card.getByRole('button', { name: 'Open clip.webm', exact: true })).toBeVisible();
-    // Legacy video art inside the concept card; D11 names the video's own extension.
-    await expect(card.locator('.media-art > .image-open > video')).toHaveCount(1);
+    // The card shows the preview frame, never the video itself; D11 names the video's own extension.
+    await expect(card.locator('video')).toHaveCount(0);
+    await expect(card.locator('.media-art > .image-open > img')).toHaveCount(1);
     await expect(card.locator('.media-info p')).toHaveText(/(?:^|· )WEBM$/);
-    expect(await card.locator('video').evaluate(node => node.getBoundingClientRect().height)).toBeGreaterThan(20);
+    await expect.poll(() => card.locator('img').evaluate(node => node.complete && node.naturalHeight > 0)).toBe(true);
+    expect(await card.locator('img').evaluate(node => node.getBoundingClientRect().height)).toBeGreaterThan(20);
     await card.getByRole('button', { name: 'Open clip.webm', exact: true }).click();
     const video = page.locator('dialog[open] video');
     await expect(video).toBeVisible();
@@ -713,16 +715,45 @@ test('selected layout stays blue with readable text on hover in either theme', a
     }
 });
 
-test('refresh releases the previous inspector video source', async ({ page, request }) => {
+test('refresh releases the previous inspector cover source', async ({ page, request }) => {
     await expect(await request.post('/__test__/reset', { data: { include_video: true } })).toBeOK();
     await page.goto(workspace);
     const card = cards(page).filter({ hasText: 'clip.webm' });
     await card.getByRole('button', { name: 'Inspect clip.webm', exact: true }).click();
+    // #249: a video's cover is its preview frame, not the video.
+    const cover = details(page).locator('img.detail-cover');
+    await expect(cover).toHaveAttribute('src', /\/thumbnails\/clip_thumb\.jpg\?/);
+    await expect(details(page).locator('video')).toHaveCount(0);
+    await cover.evaluate(element => { window.__oldInspectorCover = element; });
+    await rescan(page);
+    await expect.poll(() => page.evaluate(() => !window.__oldInspectorCover.isConnected)).toBe(true);
+    expect(await page.evaluate(() => window.__oldInspectorCover.getAttribute('src'))).toBeNull();
+});
+
+/** A listing that names no preview for its videos (the fallback a card and the cover keep for one). */
+async function videosWithoutPreview(page) {
+    await page.route(url => url.pathname === '/prompt_manager/images/output', async route => {
+        const response = await route.fetch();
+        const body = await response.json();
+        if (Array.isArray(body.images)) body.images = body.images.map(image => image.is_video ? { ...image, thumbnail_url: null } : image);
+        await route.fulfill({ response, json: body });
+    });
+}
+
+test('a video listed without a preview keeps its own element, released on refresh', async ({ page, request }) => {
+    await expect(await request.post('/__test__/reset', { data: { include_video: true } })).toBeOK();
+    await videosWithoutPreview(page);
+    await page.goto(workspace);
+    const card = cards(page).filter({ hasText: 'clip.webm' });
+    await expect(card.locator('.media-art > .image-open > video')).toHaveCount(1);
+    await expect(card.locator('img')).toHaveCount(0);
+    await card.getByRole('button', { name: 'Inspect clip.webm', exact: true }).click();
     await expect(details(page).locator('video')).toBeVisible();
     await details(page).locator('video').evaluate(element => { window.__oldInspectorVideo = element; });
+    await card.locator('video').evaluate(element => { window.__oldCardVideo = element; });
     await rescan(page);
-    await expect.poll(() => page.evaluate(() => !window.__oldInspectorVideo.isConnected)).toBe(true);
-    expect(await page.evaluate(() => window.__oldInspectorVideo.getAttribute('src'))).toBeNull();
+    await expect.poll(() => page.evaluate(() => !window.__oldInspectorVideo.isConnected && !window.__oldCardVideo.isConnected)).toBe(true);
+    expect(await page.evaluate(() => [window.__oldInspectorVideo.getAttribute('src'), window.__oldCardVideo.getAttribute('src')])).toEqual([null, null]);
 });
 
 test('Import Rescan rescans cached files and preserves the collection when rescan fails', async ({ page, request }) => {

tests/browser/workspace-review.spec.mjs

@@ -1,6 +1,6 @@
 // Needs review (#249 PR F, plan Tasks F1 and F2, Mock inventory M4): the SAVED VIEWS row and its count, the Metadata
 // review page over the real review routes, and Activity's Review metadata. Items are recorded by the real recorder
-// (tests/browser/workspace_review_fixture.py) through the monitor's entry point; the conflict item's file was linked
+// (tests/browser/workspace_review_fixture.py) through the scan's recorder (only scans add items); the conflict item's file was linked
 // to another prompt after it was queued. An open page re-reads its queue from the review change stream (F2).
 import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
 import { test, expect } from './fixtures.mjs';
@@ -28,6 +28,17 @@ async function pending(request) {
 }
 const links = async request => (await (await request.get('/__test__/review/links')).json()).links;
 const streams = async request => (await (await request.get('/__test__/review/streams')).json());
+/**
+ * Wait until the server has dropped every stream a page closed. It notices a closed stream only when a write to it
+ * fails, and the close can reach it after any one write (WebKit under load): each poll sends one more change notice,
+ * which changes nothing in the queue, until the count is 0.
+ */
+async function closedStreamsDropped(request) {
+    await expect.poll(async () => {
+        await expect(await request.post('/__test__/review/notify')).toBeOK();
+        return (await streams(request)).streams;
+    }).toBe(0);
+}
 const apply = page => panel(page).getByRole('button', { name: 'Apply association', exact: true });
 const warning = page => panel(page).locator('.warning-note');
 /** The page with its first item's sampler paths loaded. */
@@ -318,13 +329,13 @@ test('keyboard focus labels and touch targets', async ({ page, request }) => {
     await expect(page).toHaveURL(/#\/images$/);
 });
 
-test('an open page shows items the monitor adds and keeps the open choice', async ({ page, request }) => {
+test('an open page shows items a scan adds and keeps the open choice', async ({ page, request }) => {
     await seed(request, 1);
     await open(page);
     await expect.poll(async () => (await streams(request)).streams).toBe(1);
     await context(page).selectOption({ label: OPTIONS[1] });
     await expect(reviewRow(page)).toHaveText('Needs review1');
-    await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
+    await expect(await request.post('/__test__/review/add', { data: { kind: 'scan' } })).toBeOK();
     await expect(cards(page)).toHaveCount(2);
     await expect(main(page).locator('.notice h2')).toHaveText('2 images need a decision');
     await expect(reviewRow(page)).toHaveText('Needs review2');
@@ -381,7 +392,7 @@ test('a visit reads its queue once, and a change reads it once more', async ({ p
     const reads = reviewReads(page);
     await open(page);
     await expect.poll(async () => (await streams(request)).streams).toBe(1);
-    await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
+    await expect(await request.post('/__test__/review/add', { data: { kind: 'scan' } })).toBeOK();
     await expect(cards(page)).toHaveCount(2);
     await expect(reviewRow(page)).toHaveText('Needs review2');
     // The page's first read and the change's re-read; the shell's count read at start, and the row follows the page.
@@ -423,7 +434,7 @@ test('a stream the browser gives up on is opened again, and the page says it is
     // The page opens the stream again (after 3 s) and is live once it sends ready.
     await expect.poll(async () => (await streams(request)).streams, { timeout: 15000 }).toBe(1);
     await expect(live(page)).toBeHidden();
-    await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
+    await expect(await request.post('/__test__/review/add', { data: { kind: 'scan' } })).toBeOK();
     await expect(cards(page)).toHaveCount(2);
     expect(attempts).toBe(2);
 });
@@ -444,9 +455,8 @@ test('a hidden tab closes its stream and a visible one reads only what changed m
     await open(page);
     await expect.poll(async () => (await streams(request)).streams).toBe(1);
     await setVisible(page, false);
-    // The server drops a closed stream at its next write: this monitor addition is one.
-    await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
-    await expect.poll(async () => (await streams(request)).streams).toBe(0);
+    await expect(await request.post('/__test__/review/add', { data: { kind: 'scan' } })).toBeOK();
+    await closedStreamsDropped(request);
     await expect(cards(page)).toHaveCount(1);
     expect(reads.filter(search => search === '?limit=40')).toHaveLength(1);
     // Visible again: the reopened stream's ready names a newer version, so the page reads once.
@@ -459,6 +469,30 @@ test('a hidden tab closes its stream and a visible one reads only what changed m
     expect(reads.filter(search => search === '?limit=40')).toHaveLength(2);
 });
 
+test('a stream whose close reaches the server after a change is dropped at a later write', async ({ page, request }) => {
+    // The CI ordering made certain: the hidden page's close arrives only after the scan addition's change was
+    // written to its stream, so that write succeeds and a later one finds the stream closed.
+    await seed(request, 1);
+    await open(page);
+    await expect.poll(async () => (await streams(request)).streams).toBe(1);
+    await page.evaluate(() => {
+        const close = EventSource.prototype.close;
+        window.lateChanges = 0;
+        window.heldCloses = [];
+        EventSource.prototype.close = function () {
+            this.addEventListener('change', () => { window.lateChanges += 1; });
+            window.heldCloses.push(() => close.call(this));
+        };
+    });
+    await setVisible(page, false);
+    await expect(await request.post('/__test__/review/add', { data: { kind: 'scan' } })).toBeOK();
+    await expect.poll(() => page.evaluate(() => window.lateChanges)).toBe(1);
+    await page.evaluate(() => window.heldCloses.splice(0).forEach(close => close()));
+    expect((await streams(request)).streams).toBe(1);
+    await closedStreamsDropped(request);
+    await expect(cards(page)).toHaveCount(1);
+});
+
 test('a re-read refreshes a card whose media or target changed', async ({ page, request }) => {
     // M6 and M10: a root came or went between reads; the card follows, and a known thumbnail is its media.
     await seed(request, 2);
@@ -476,7 +510,7 @@ test('a re-read refreshes a card whose media or target changed', async ({ page,
     const second = cards(page).nth(1);
     await expect(second.getByRole('button', { name: 'Open review_0001.png', exact: true })).not.toHaveAttribute('aria-disabled', 'true');
     rewrite = true;
-    await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
+    await expect(await request.post('/__test__/review/add', { data: { kind: 'scan' } })).toBeOK();
     await expect(cards(page)).toHaveCount(3);
     await expect(cards(page).first().locator('img')).toHaveAttribute('src', /&thumb=1$/);
     await expect(second.getByRole('button', { name: 'Open review_0001.png', exact: true })).toHaveAttribute('aria-disabled', 'true');
@@ -522,7 +556,7 @@ test('a live re-read keeps the loaded depth and the scroll position', async ({ p
     const before = await scrollTop();
     expect(before).toBeGreaterThan(0);
     const reads = reviewReads(page);
-    await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
+    await expect(await request.post('/__test__/review/add', { data: { kind: 'scan' } })).toBeOK();
     await expect(main(page).locator('.notice h2')).toHaveText('46 images need a decision');
     await expect(cards(page)).toHaveCount(46);
     expect(await scrollTop()).toBe(before);
@@ -549,8 +583,6 @@ test('teardown releases captures listeners and requests', async ({ page, request
     await expect(main(page).getByRole('heading', { level: 1 })).toHaveText('Tags & categories');
     await expect.poll(async () => (await streams(request)).captures).toBe(0);
     await route.abort().catch(() => {});
-    // The server drops a closed stream at its next write: a monitor addition is one.
-    await expect(await request.post('/__test__/review/add', { data: { kind: 'monitor' } })).toBeOK();
-    await expect.poll(async () => (await streams(request)).streams).toBe(0);
+    await closedStreamsDropped(request);
     await expect.poll(() => inFlight.size).toBe(0);
 });

tests/browser/workspace-system-sources.spec.mjs

@@ -304,16 +304,151 @@ test('Refresh files calls /gallery/rescan and toasts its counts', async ({ page
     await expect(toast(page)).toHaveText('Files could not be refreshed. Try again.');
 });
 
-test('Thumbnail generation hands off to the existing gallery until its review dialog', async ({ page }) => {
+const THUMBNAIL_PLAN_URL = '/prompt_manager/images/thumbnails/plan';
+const THUMBNAIL_JOB_URL = '/prompt_manager/jobs/thumbnails';
+const planCount = value => value.toLocaleString('en-US');
+
+/** Open "Thumbnail maintenance" from Sources' Thumbnail generation → Review options. */
+async function openThumbnailReview(page) {
     const layout = await openSources(page);
     const thumbnails = layout.locator('.operation').filter({ has: page.getByRole('heading', { name: 'Thumbnail generation', exact: true }) });
     await expect(thumbnails.locator('p')).toHaveText('Generate missing previews; rebuilding all is a separate explicit option.');
-    const popup = page.waitForEvent('popup');
     await thumbnails.getByRole('button', { name: 'Review options', exact: true }).click();
-    const gallery = await popup;
-    expect(new URL(gallery.url()).pathname).toBe('/prompt_manager/gallery');
-    await gallery.close();
-    await expect(statusOf(page)).toHaveText('No unsaved changes.');
+    await expect(dialog(page).locator('#modal-title')).toHaveText('Thumbnail maintenance');
+    return dialog(page);
+}
+
+/** The thumbnail requests this page sends: plan reads (their query) and job starts (their body). */
+function thumbnailRequests(page) {
+    const seen = { plans: [], starts: [] };
+    page.on('request', item => {
+        const url = new URL(item.url());
+        if (url.pathname === THUMBNAIL_PLAN_URL) seen.plans.push(Object.fromEntries(url.searchParams));
+        if (item.method() === 'POST' && url.pathname === THUMBNAIL_JOB_URL) seen.starts.push(item.postDataJSON());
+    });
+    return seen;
+}
+
+const missingLine = plan => `${planCount(plan.targets)} ${plan.targets === 1 ? 'preview' : 'previews'} to generate · ` +
+    `${planCount(plan.fresh)} already up to date.`;
+
+test('Review options shows the plan counts for the chosen scope/quality/size before Start', async ({ page, request }) => {
+    await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 3 } })).toBeOK();
+    // Image loading quality (Appearance, Q3) preselects the dialog's quality.
+    await page.addInitScript(() => { if (location.protocol.startsWith('http')) localStorage.setItem('gallerySettings', JSON.stringify({ imageQuality: 'high' })); });
+    const seen = thumbnailRequests(page);
+    const plan = await serverJson(request, `${THUMBNAIL_PLAN_URL}?mode=missing&size=512&quality=high`);
+    expect(plan.targets).toBe(3);
+    const review = await openThumbnailReview(page);
+    await expect(review.locator('#modal-body > *')).toHaveCount(5);
+    await expect(review.locator('.field > label')).toHaveText(['Scope', 'Quality', 'Preview size limit']);
+    await expect(review.locator('#thumb-mode option')).toHaveText(['Missing thumbnails only', 'Rebuild all (explicit opt-in)']);
+    await expect(review.locator('#thumb-quality option')).toHaveText(['Medium · optimized', 'High · original', 'Low · fastest']);
+    await expect(review.locator('#thumb-mode')).toHaveValue('missing');
+    await expect(review.locator('#thumb-quality')).toHaveValue('high');
+    await expect(review.locator('#thumb-limit')).toHaveValue('512');
+    await expect(review.locator('#thumb-limit')).toHaveAttribute('min', '128');
+    await expect(review.locator('#thumb-limit')).toHaveAttribute('max', '2048');
+    const line = review.locator('#modal-body > p.small');
+    await expect(line).toHaveAttribute('role', 'status');
+    await expect(line).toHaveText(missingLine(plan));
+    expect(seen.plans).toEqual([{ mode: 'missing', size: '512', quality: 'high' }]);
+    await expect(review.locator('.dialog-actions > button')).toHaveText(['Cancel', 'Start']);
+    // Reviewing starts nothing; Cancel closes.
+    await review.getByRole('button', { name: 'Cancel', exact: true }).click();
+    await expect(dialog(page)).toHaveCount(0);
+    expect(seen.starts).toEqual([]);
+    expect((await serverJson(request, '/prompt_manager/jobs')).jobs).toEqual([]);
+});
+
+test('size outside 128–2048 reports validity and requests nothing', async ({ page, request }) => {
+    const seen = thumbnailRequests(page);
+    const review = await openThumbnailReview(page);
+    const line = review.locator('#modal-body > p.small');
+    await expect(line).toHaveText(/ already up to date\.$/);
+    const size = review.locator('#thumb-limit');
+    for (const value of ['4096', '64', '']) {
+        await size.fill(value);
+        await expect(size).toHaveAttribute('aria-invalid', 'true');
+        await expect(line).toHaveText('Enter a size from 128 to 2048 pixels.');
+        expect(await size.evaluate(input => input.validity.valid)).toBe(false);
+        // Start keeps the dialog open and sends nothing; neither does a scope change.
+        await review.getByRole('button', { name: 'Start', exact: true }).click();
+        await expect(dialog(page).locator('#modal-title')).toHaveText('Thumbnail maintenance');
+        await review.locator('#thumb-mode').selectOption('rebuild');
+        await expect(line).toHaveText('Enter a size from 128 to 2048 pixels.');
+        await review.locator('#thumb-mode').selectOption('missing');
+    }
+    expect(seen.starts).toEqual([]);
+    expect(seen.plans).toEqual([{ mode: 'missing', size: '512', quality: 'medium' }]);
+    await size.fill('1024');
+    await expect(size).not.toHaveAttribute('aria-invalid');
+    await expect(line).toHaveText(/ already up to date\.$/);
+    expect((await serverJson(request, '/prompt_manager/jobs')).jobs).toEqual([]);
+});
+
+test('Start closes the dialog, toasts, and opens the job in Activity', async ({ page, request }) => {
+    await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 2 } })).toBeOK();
+    const seen = thumbnailRequests(page);
+    const review = await openThumbnailReview(page);
+    await review.locator('#thumb-quality').selectOption('low');
+    await review.locator('#thumb-limit').fill('256');
+    await review.getByRole('button', { name: 'Start', exact: true }).click();
+    await expect(dialog(page)).toHaveCount(0);
+    await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
+    // The toast says it started (or, for a run this quick, that it already finished); Activity shows its progress.
+    await expect(toast(page)).toHaveText(/^(Thumbnail generation started\.|Thumbnails finished · 2 generated · 0 errors)$/);
+    expect(seen.starts).toEqual([{ mode: 'missing', size: 256, quality: 'low' }]);
+    const id = page.url().split('/').pop();
+    const { job } = await serverJson(request, `/prompt_manager/jobs/${id}`);
+    expect(job).toMatchObject({ kind: 'thumbnails', title: 'Thumbnail generation', scope: { mode: 'missing', size: 256, quality: 'low' } });
+    await expect(page.locator('#workspace-main h1')).toHaveText('Thumbnail generation');
+    await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).toBe('finished');
+});
+
+test('Rebuild all is explicit and the plan line says every preview is regenerated', async ({ page, request }) => {
+    const seen = thumbnailRequests(page);
+    const rebuild = await serverJson(request, `${THUMBNAIL_PLAN_URL}?mode=rebuild&size=512&quality=medium`);
+    expect(rebuild.targets).toBeGreaterThan(0);
+    const review = await openThumbnailReview(page);
+    // Missing thumbnails only is the default; rebuilding is the explicit second option.
+    await expect(review.locator('#thumb-mode')).toHaveValue('missing');
+    const line = review.locator('#modal-body > p.small');
+    await expect(line).toHaveText(/ already up to date\.$/);
+    await review.locator('#thumb-mode').selectOption('rebuild');
+    await expect(line).toHaveText(`Every preview is regenerated: ${planCount(rebuild.targets)} files.`);
+    expect(seen.plans).toEqual([{ mode: 'missing', size: '512', quality: 'medium' }, { mode: 'rebuild', size: '512', quality: 'medium' }]);
+    // Back and forth reads each scope once.
+    await review.locator('#thumb-mode').selectOption('missing');
+    await expect(line).toHaveText(/ already up to date\.$/);
+    await review.locator('#thumb-mode').selectOption('rebuild');
+    await expect(line).toHaveText(`Every preview is regenerated: ${planCount(rebuild.targets)} files.`);
+    expect(seen.plans).toHaveLength(2);
+    await review.getByRole('button', { name: 'Start', exact: true }).click();
+    await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
+    expect(seen.starts).toEqual([{ mode: 'rebuild', size: 512, quality: 'medium' }]);
+    const id = page.url().split('/').pop();
+    await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).toBe('finished');
+});
+
+test('a plan that cannot be read says so; Start still starts', async ({ page }) => {
+    await page.route(`**${THUMBNAIL_PLAN_URL}?*`, route => route.fulfill({ status: 500, json: { success: false, error: 'disk' } }));
+    const review = await openThumbnailReview(page);
+    await expect(review.locator('#modal-body > p.small')).toHaveText('The previews could not be counted.');
+    await expect(review.getByRole('button', { name: 'Start', exact: true })).not.toHaveAttribute('aria-disabled');
+    await review.getByRole('button', { name: 'Cancel', exact: true }).click();
+});
+
+test('a refused start shows the busy link toast', async ({ page, request }) => {
+    const running = await (await request.post('/__test__/jobs/start', { data: { title: 'Scan for metadata' } })).json();
+    const review = await openThumbnailReview(page);
+    await review.getByRole('button', { name: 'Start', exact: true }).click();
+    await expect(dialog(page)).toHaveCount(0);
+    await expect(toast(page)).toHaveText('Another job is running: Scan for metadata.');
+    await expect(toast(page).getByRole('link', { name: 'Scan for metadata', exact: true })).toHaveAttribute('href', `#/activity/${running.job.id}`);
+    await expect(page).toHaveURL(`${workspace}#/system/sources`);
+    expect((await serverJson(request, '/prompt_manager/jobs')).jobs.map(job => job.kind)).toEqual(['scan']);
+    await request.post(`/__test__/jobs/${running.job.id}/finish`, { data: { state: 'finished' } });
 });
 
 test('a save refused by a running scan shows the busy link toast', async ({ page, request }) => {

tests/browser/workspace-system.spec.mjs

@@ -140,9 +140,32 @@ test('thumbnail row reflects the status endpoint', async ({ page, request }) =>
     const layout = await openOverview(page);
     const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) });
     await expect(row.locator('h3')).toHaveText('3 missing thumbnails');
-    // Until the thumbnail review dialog (S5), Review thumbnails opens Sources.
+});
+
+test('Overview Review thumbnails opens the same dialog', async ({ page, request }) => {
+    await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 2 } })).toBeOK();
+    const plan = await serverJson(request, '/prompt_manager/images/thumbnails/plan?mode=missing&size=512&quality=medium');
+    const starts = [];
+    page.on('request', item => { if (item.method() === 'POST' && item.url().endsWith('/prompt_manager/jobs/thumbnails')) starts.push(item.postDataJSON()); });
+    const layout = await openOverview(page);
+    const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) });
+    await expect(row.locator('h3')).toHaveText('2 missing thumbnails');
     await row.getByRole('button', { name: 'Review thumbnails', exact: true }).click();
-    await expect(page).toHaveURL(`${workspace}#/system/sources`);
+    const review = page.locator('dialog#modal[open]');
+    await expect(review.locator('#modal-title')).toHaveText('Thumbnail maintenance');
+    await expect(page).toHaveURL(`${workspace}#/system/overview`);
+    await expect(review.locator('.field > label')).toHaveText(['Scope', 'Quality', 'Preview size limit']);
+    await expect(review.locator('#modal-body > p.small')).toHaveText(`2 previews to generate · ${number(plan.fresh)} already up to date.`);
+    // Escape closes it and focus returns to the row's button.
+    await page.keyboard.press('Escape');
+    await expect(review).toHaveCount(0);
+    await expect(row.getByRole('button', { name: 'Review thumbnails', exact: true })).toBeFocused();
+    await row.getByRole('button', { name: 'Review thumbnails', exact: true }).click();
+    await review.getByRole('button', { name: 'Start', exact: true }).click();
+    await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
+    expect(starts).toEqual([{ mode: 'missing', size: 512, quality: 'medium' }]);
+    const id = page.url().split('/').pop();
+    await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).toBe('finished');
 });
 
 test('the thumbnail row paints when its own status read answers', async ({ page, request }) => {

tests/browser/workspace-thumbnails.spec.mjs

@@ -0,0 +1,135 @@
+import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
+import { test, expect } from './fixtures.mjs';
+
+/**
+ * #249: cards and strips load previews, never the full file. The approved mock has no repeated thumbnail prompt
+ * (docs/concepts/workspace/README.md), so a preview that was not made yet is made by the server on first view:
+ * the listing still names the `_thumb` URL. Only the media viewer's main image (and downloads) load the original.
+ */
+const images = '/prompt_manager/workspace#/images';
+const library = '/prompt_manager/workspace#/library';
+const cards = page => page.locator('article[data-image-key]');
+const details = page => page.getByRole('complementary', { name: 'Selected image details', exact: true });
+const SERVE = '/prompt_manager/images/serve/';
+
+/** Every media file the page asks the server for, as `{path, preview}` (path relative to its root). */
+function recordMedia(page) {
+    const seen = [];
+    page.on('request', request => {
+        const url = new URL(request.url());
+        if (!url.pathname.startsWith(SERVE)) return;
+        const path = decodeURIComponent(url.pathname.slice(SERVE.length));
+        seen.push({ path, preview: /^thumbnails\/.+_thumb\.[a-z0-9]+$/.test(path) });
+    });
+    return seen;
+}
+/** Every card image has loaded (or failed): what it asked for is all in `seen`. */
+const settled = page => page.evaluate(() => Promise.all([...document.querySelectorAll('article img')].map(image => image.decode().catch(() => {}))));
+const originals = seen => seen.filter(item => !item.preview).map(item => item.path);
+
+test.beforeEach(async ({ page, browserName }) => {
+    await startCoverage(page, browserName);
+    page.thumbErrors = [];
+    page.on('pageerror', error => page.thumbErrors.push(error.message));
+});
+test.afterEach(async ({ page, browserName }) => {
+    await saveCoverage(page, browserName);
+    expect(page.thumbErrors).toEqual([]);
+});
+
+test('a page of the Images grid requests previews, never originals, in every layout and size', async ({ page, request }) => {
+    await expect(await request.post('/__test__/reset')).toBeOK();
+    // Newest outputs with no preview yet: the live finding (844 such files listed with a null thumbnail).
+    await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 6 } })).toBeOK();
+    const listing = await (await request.get('/prompt_manager/images/output?limit=40')).json();
+    const fresh = listing.images.filter(image => image.path.startsWith('unthumbnailed/'));
+    expect(fresh).toHaveLength(6);
+    for (const image of fresh) expect(image.thumbnail_url).toMatch(/\/thumbnails\/unthumbnailed\/plain_\d{3}_thumb\.png\?root=0/);
+    const seen = recordMedia(page);
+    await page.goto(images);
+    await expect(cards(page)).toHaveCount(8);
+    await settled(page);
+    expect(originals(seen)).toEqual([]);
+    expect(seen.filter(item => item.path.startsWith('thumbnails/unthumbnailed/'))).toHaveLength(6);
+    // The server made them on that first view.
+    await expect.poll(async () => (await (await request.get('/prompt_manager/images/thumbnails/status')).json()).pending).toBe(0);
+    for (const layout of ['List', 'Masonry', 'Grid']) {
+        await page.getByRole('button', { name: layout, exact: true }).click();
+        await expect(page.getByRole('button', { name: layout, exact: true })).toHaveAttribute('aria-pressed', 'true');
+        for (const density of ['large', 'compact', 'comfortable']) {
+            await page.getByRole('combobox', { name: 'Thumbnail size', exact: true }).selectOption(density);
+            await expect(cards(page)).toHaveCount(8);
+            await settled(page);
+        }
+    }
+    expect(originals(seen)).toEqual([]);
+});
+
+test('the inspector cover is the preview and only the media viewer loads the original', async ({ page, request }) => {
+    await expect(await request.post('/__test__/reset')).toBeOK();
+    await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 1 } })).toBeOK();
+    const seen = recordMedia(page);
+    await page.goto(images);
+    const card = cards(page).filter({ hasText: 'plain_001.png' });
+    await card.getByRole('button', { name: 'Inspect plain_001.png', exact: true }).click();
+    const cover = details(page).locator('img.detail-cover');
+    await expect(cover).toHaveAttribute('src', /\/thumbnails\/unthumbnailed\/plain_001_thumb\.png\?/);
+    await settled(page);
+    await cover.evaluate(image => image.decode().catch(() => {}));
+    expect(originals(seen)).toEqual([]);
+    await card.getByRole('button', { name: 'Open plain_001.png', exact: true }).click();
+    await expect(page.locator('dialog#viewer-dialog')).toBeVisible();
+    await expect.poll(() => originals(seen)).toEqual(['unthumbnailed/plain_001.png']);
+    await page.getByRole('button', { name: 'Close image viewer', exact: true }).click();
+});
+
+test('a video card shows its preview frame instead of loading the video', async ({ page, request }) => {
+    await expect(await request.post('/__test__/reset', { data: { include_video: true } })).toBeOK();
+    const seen = recordMedia(page);
+    await page.goto(images);
+    const card = cards(page).filter({ hasText: 'clip.webm' });
+    await expect(card.locator('.media-art > .image-open > img')).toHaveAttribute('src', /\/thumbnails\/clip_thumb\.jpg\?/);
+    await expect(card.locator('video')).toHaveCount(0);
+    await settled(page);
+    expect(originals(seen)).toEqual([]);
+});
+
+test('Library cards and the shared gallery request previews only', async ({ page, request }) => {
+    await expect(await request.post('/__test__/reset', { data: { native_library: true } })).toBeOK();
+    const seen = recordMedia(page);
+    await page.goto(library);
+    const cat = page.locator('article[data-prompt-id]').filter({ has: page.getByText('Cat portrait', { exact: true }) });
+    await expect(cat.locator('.card-art img')).not.toHaveCount(0);
+    await settled(page);
+    expect(seen.length).toBeGreaterThan(0);
+    expect(originals(seen)).toEqual([]);
+});
+
+test('a video whose preview cannot be loaded swaps back to its own element', async ({ page, request }) => {
+    await expect(await request.post('/__test__/reset', { data: { include_video: true } })).toBeOK();
+    await page.route(url => url.pathname.endsWith('/thumbnails/clip_thumb.jpg'), route => route.fulfill({ status: 404, body: '' }));
+    await page.goto(images);
+    const card = cards(page).filter({ hasText: 'clip.webm' });
+    await expect(card.locator('.media-art > .image-open > video')).toHaveCount(1);
+    await expect(card.locator('.media-art > .image-open > img')).toHaveCount(0);
+    await card.getByRole('button', { name: 'Inspect clip.webm', exact: true }).click();
+    await expect(details(page).locator('video.detail-cover')).toHaveCount(1);
+    await expect(details(page).locator('img.detail-cover')).toHaveCount(0);
+});
+
+test('an image listed without a preview (a nested thumbnails folder) shows its original', async ({ page, request }) => {
+    await expect(await request.post('/__test__/reset')).toBeOK();
+    // M10: such files are listed with a null thumbnail_url; the card falls back to the original as before.
+    await page.route(url => url.pathname === '/prompt_manager/images/output', async route => {
+        const response = await route.fetch();
+        const body = await response.json();
+        if (Array.isArray(body.images)) body.images = body.images.map(image => ({ ...image, thumbnail_url: null }));
+        await route.fulfill({ response, json: body });
+    });
+    const seen = recordMedia(page);
+    await page.goto(images);
+    await expect(cards(page)).toHaveCount(2);
+    await settled(page);
+    expect(seen.length).toBeGreaterThan(0);
+    expect(seen.every(item => !item.preview)).toBe(true);
+});

tests/browser/workspace_activity_fixture.py

@@ -5,18 +5,23 @@ reading files, so a test sets its progress and counters exactly:
 
 - ``POST /__test__/jobs/start`` starts one (``kind``, ``title``, ``total``,
   ``output_folders``; for ``kind: "image-tags"`` the image tagging job's
-  groups and its ``{mode, images}`` scope instead) and returns its summary,
-  or 409 while one runs;
+  groups and its ``{mode, images}`` scope instead; for ``kind:
+  "thumbnails"`` the thumbnail job's groups, ``guarded`` admission and its
+  ``{mode, size, quality, output_folders}`` scope; for ``kind: "health"`` the
+  health check's: no groups, ``read``, not cancellable, with its status
+  line) and returns its summary, or 409 while one runs;
 - ``POST /__test__/jobs/{id}/step`` records ``records`` (``{group, path,
   root, root_index, detail}`` file outcomes, counted, listed and logged as the scan
   records them) and ``bulk`` (``{group, count, root}``: ``count`` numbered
   rows), sets ``processed`` (and ``total``) and ``counters``, then publishes
   the progress to every subscriber;
 - ``POST /__test__/jobs/{id}/finish`` ends it as ``finished``, ``cancelled``
-  or ``failed``; a cancel through the real route ends it ``cancelled``;
-- ``POST /__test__/jobs/hold`` opens a legacy-page hold (``kind``, ``title``):
-  a running, non-cancellable ``records`` job, as a legacy restore or
-  maintenance request takes, until ``finish`` ends it;
+  or ``failed``, with the runner's final ``status_line`` when given; a cancel
+  through the real route ends it ``cancelled``;
+- ``POST /__test__/jobs/hold`` opens a legacy-page hold (``kind``, ``title``,
+  ``admission``, default ``records``): a running, non-cancellable job with no
+  outcome groups, as a legacy restore, maintenance or thumbnail clear request
+  takes, until ``finish`` ends it;
 - ``POST /__test__/jobs/restart`` acts out a ComfyUI restart: the open job
   streams drop without a final event and the registry starts empty.
 """
@@ -35,6 +40,9 @@ PREFIXES = {"added": "+", "known": "=", "review": "?", "errors": "!"}
 PREFIXES["no_metadata"] = "-"
 # The image tagging job's groups (#249 PR D), logged with the runner's prefixes.
 PREFIXES.update({"tagged": "+", "unchanged": "=", "skipped": "-"})
+# The thumbnail job's (#249 System S5).
+PREFIXES["generated"] = "+"
+HEALTH_STATUS_LINE = "Checking the database and image links"
 ROOT_INDEXES = {"output": 0, "input": 1}
 
 
@@ -80,9 +88,12 @@ def outcome_records(data):
 
 def register_activity_commands(app, routes, api):
     from py.api.jobs import (
+        HEALTH_JOB_KIND,
         IMAGE_TAG_GROUP_NAMES,
         IMAGE_TAG_JOB_KIND,
         SCAN_GROUP_NAMES,
+        THUMBNAIL_GROUP_NAMES,
+        THUMBNAILS_JOB_KIND,
         JobBusy,
         JobRegistry,
     )
@@ -138,7 +149,21 @@ def register_activity_commands(app, routes, api):
         data = await body(request)
         ending = asyncio.get_running_loop().create_future()
         kind = str(data.get("kind", "scan"))
-        if kind == IMAGE_TAG_JOB_KIND:
+        admission, cancellable = "records", True
+        if kind == THUMBNAILS_JOB_KIND:
+            title = "Thumbnail generation"
+            scope = {
+                "mode": str(data.get("mode", "missing")),
+                "size": count(data.get("size"), 512),
+                "quality": str(data.get("quality", "medium")),
+                "output_folders": count(data.get("output_folders"), 2),
+            }
+            groups = THUMBNAIL_GROUP_NAMES
+            admission = "guarded"
+        elif kind == HEALTH_JOB_KIND:
+            title, scope, groups = "Health check", {}, ()
+            admission, cancellable = "read", False
+        elif kind == IMAGE_TAG_JOB_KIND:
             title = "Add tags to images"
             scope = {
                 "mode": str(data.get("mode", "add")),
@@ -162,7 +187,8 @@ def register_activity_commands(app, routes, api):
             job = api._jobs.start(
                 kind,
                 str(data.get("title", title)),
-                record_changing=True,
+                admission=admission,
+                cancellable=cancellable,
                 scope=scope,
                 runner=fake_runner(ending),
                 groups=groups,
@@ -170,6 +196,8 @@ def register_activity_commands(app, routes, api):
         except JobBusy:
             raise web.HTTPConflict()
         job.update(total=count(data.get("total")))
+        if kind == HEALTH_JOB_KIND:
+            job.update(status_line=HEALTH_STATUS_LINE)
         endings[job.id] = ending
         return web.json_response({"success": True, "job": job.summary()})
 
@@ -184,7 +212,7 @@ def register_activity_commands(app, routes, api):
             async with api._jobs.hold(
                 str(data.get("kind", "restore")),
                 str(data.get("title", "Database restore (admin page)")),
-                admission="records",
+                admission=str(data.get("admission", "records")),
             ) as job:
                 entered.set_result(job)
                 state = await ending
@@ -232,9 +260,12 @@ def register_activity_commands(app, routes, api):
     @routes.post("/__test__/jobs/{job_id}/finish")
     async def finish_job(request):
         job = job_or_404(request)
-        state = (await body(request)).get("state", "finished")
+        data = await body(request)
+        state = data.get("state", "finished")
         if state not in FINAL_STATES:
             raise web.HTTPBadRequest()
+        if isinstance(data.get("status_line"), str):  # the runner's final line
+            job.update(status_line=data["status_line"])
         ending = endings[job.id]
         if not ending.done():
             ending.set_result(state)

tests/browser/workspace_review_fixture.py

@@ -5,8 +5,9 @@ encoders and one shared negative, so the resolver finds it ``ambiguous``. Its
 loader node carries ComfyUI's ``Infinity`` stamp, which the legacy pages'
 parser (the scan's fallback) cannot read, so nothing links the file and it is
 a Needs review case (#284: only ambiguous, unlinked files are queued). It is
-recorded through the producers' entry point (``record_review_file``). No path
-or SQL comes from a request.
+recorded through the scan's recorder (``_record_review_sync`` with
+``inspection=True``; only scans add items). No path or SQL comes from a
+request.
 
 ``POST /__test__/review/reset`` also takes the special items F2 needs, queued
 after the plain ones in this order:
@@ -28,12 +29,14 @@ its first negative, the one the concept's panel shows), so the panel's text
 wraps as the concept's does.
 
 ``POST /__test__/review/add {kind}`` adds one file while a page is open:
-``monitor`` records a new ambiguous, unlinked output from a worker thread
-through ``record_review_file`` (the monitor's own entry point; the monitor
-never queues an output it linked itself), ``file`` only writes one (a
-metadata scan records it). ``GET
+``scan`` records a new ambiguous, unlinked output from a worker thread
+through the scan's recorder (as a running scan's batch does; the monitor
+never adds items), ``file`` only writes one (a metadata scan records it). ``GET
 /__test__/review/links`` answers each review file's prompt and pair links, and
 ``GET /__test__/review/streams`` the open change streams and captures.
+``POST /__test__/review/notify`` sends one queue change notice without
+changing the queue: each open stream writes a ``change``, and a stream whose
+page closed is dropped at such a write.
 """
 
 from __future__ import annotations
@@ -139,8 +142,16 @@ def _png(
         image.save(path, pnginfo=info)
 
 
+def _scan_record(api: PromptManagerAPI, path: Path) -> str:
+    """Record ``path`` as the scan's recorder does."""
+    from py.api.admin import _scan_roots
+
+    roots = _scan_roots(api._get_all_output_dirs())
+    return api._record_review_sync(Path(str(path)), roots, inspection=True)
+
+
 def _record(api: PromptManagerAPI, path: Path, expected: str) -> None:
-    outcome = api.record_review_file(str(path))
+    outcome = _scan_record(api, path)
     if outcome != expected:
         raise RuntimeError(f"{path.name} was {outcome}, not {expected}")
 
@@ -150,19 +161,19 @@ def _stamp(path: Path, age: float) -> None:
     os.utime(path, (stamp, stamp))
 
 
-class _Monitor:
-    """The monitor's recorder over this server's database (one per server)."""
+class _ScanRecorder:
+    """The scan's recorder over this server's database (one per server)."""
 
     def __init__(self, api: PromptManagerAPI) -> None:
         self.api = api
         self.lock = threading.Lock()
 
     def new_output(self, path: Path) -> None:
-        """The monitor's Needs review pass for a new output, from a worker thread."""
+        """The scan's Needs review recording of a new output, from a worker thread."""
         outcome = []
         with self.lock:
             worker = threading.Thread(
-                target=lambda: outcome.append(self.api.record_review_file(str(path)))
+                target=lambda: outcome.append(_scan_record(self.api, path))
             )
             worker.start()
             worker.join()
@@ -170,9 +181,9 @@ class _Monitor:
             raise RuntimeError(f"{path.name} was {outcome}, not recorded")
 
 
-def _monitored_output(api: PromptManagerAPI, monitor: _Monitor, path: Path) -> None:
+def _scanned_output(api: PromptManagerAPI, recorder: _ScanRecorder, path: Path) -> None:
     _png(path, (150, 60, 40))
-    monitor.new_output(path)
+    recorder.new_output(path)
 
 
 def _linked_after_queueing(api: PromptManagerAPI, path: Path) -> None:
@@ -205,7 +216,7 @@ def seed_review(
     root: Path,
     items: int,
     special: tuple = (),
-    monitor: _Monitor | None = None,
+    recorder: _ScanRecorder | None = None,
     concept: bool = False,
 ) -> dict:
     """Replace this server's disposable rows and files with ``items`` queued files."""
@@ -258,9 +269,9 @@ def seed_review(
 
 
 def add_review_file(
-    api: PromptManagerAPI, root: Path, kind: str, monitor: _Monitor
+    api: PromptManagerAPI, root: Path, kind: str, recorder: _ScanRecorder
 ) -> dict:
-    """One more ambiguous output: through the monitor, or only written."""
+    """One more ambiguous output: recorded as a scan does, or only written."""
     output = root / "output"
     number = (
         len(list((output / "added").glob("*.png")))
@@ -268,8 +279,8 @@ def add_review_file(
         else 0
     )
     relative = f"added/added_{number:04}.png"
-    if kind == "monitor":
-        _monitored_output(api, monitor, output / relative)
+    if kind == "scan":
+        _scanned_output(api, recorder, output / relative)
     else:
         _png(output / relative, (200, 200, 40))
     api.invalidate_gallery_cache()
@@ -298,12 +309,12 @@ def register_review_commands(
     routes: web.RouteTableDef, api: PromptManagerAPI, root: Path
 ) -> None:
     """Register the review fixture's commands; no path or SQL comes from a request."""
-    monitors = []
+    recorders = []
 
-    def monitor() -> _Monitor:
-        if not monitors:
-            monitors.append(_Monitor(api))
-        return monitors[0]
+    def recorder() -> _ScanRecorder:
+        if not recorders:
+            recorders.append(_ScanRecorder(api))
+        return recorders[0]
 
     @routes.post("/__test__/review/reset")
     async def reset(request: web.Request) -> web.Response:
@@ -331,7 +342,7 @@ def register_review_commands(
             root,
             items,
             special,
-            monitor(),
+            recorder(),
             data.get("concept", False),
         )
         return web.json_response({"success": True, **manifest})
@@ -341,13 +352,22 @@ def register_review_commands(
         data = await request.json() if request.can_read_body else None
         if request.query or not isinstance(data, dict) or set(data) != {"kind"}:
             raise web.HTTPBadRequest()
-        if data["kind"] not in ("monitor", "file"):
+        if data["kind"] not in ("scan", "file"):
             raise web.HTTPBadRequest()
         added = await api._run_in_executor(
-            add_review_file, api, root, data["kind"], monitor()
+            add_review_file, api, root, data["kind"], recorder()
         )
         return web.json_response({"success": True, **added})
 
+    @routes.post("/__test__/review/notify")
+    async def notify(request: web.Request) -> web.Response:
+        if request.query or request.can_read_body:
+            raise web.HTTPBadRequest()
+        from database.review_changes import notify_review_change
+
+        notify_review_change(api.db.model.db_path)
+        return web.json_response({"success": True})
+
     @routes.get("/__test__/review/links")
     async def links(request: web.Request) -> web.Response:
         if request.query:

tests/js/gallery-thumbnails.test.js

@@ -110,3 +110,123 @@ test('saving gallery settings retains the remembered thumbnail dismissal', () =>
     f.manager.saveSettings();
     assert.equal(JSON.parse(f.storage.get('gallerySettings')).thumbnailPromptDismissed, 'remembered-state');
 });
+
+// #249 System S5: the stream's work is now an Activity job that outlives the page's connection, so the gallery's
+// Cancel asks the server to stop the job. The stream's first frame, `job {id, started}`, names the job; the gallery
+// cancels by that id, only a job this stream started, and a Cancel before the id waits for it (review I1).
+const JOB = 'b'.repeat(32);
+
+function streamFixture({ modal = false, postFails = false } = {}) {
+    const posts = [];
+    const warnings = [];
+    const gets = [];
+    const listeners = {};
+    const source = { closed: 0, readyState: 1, CLOSED: 2, url: null,
+        addEventListener(type, handler) { (listeners[type] ||= []).push(handler); },
+        close() { this.closed += 1; } };
+    const element = () => ({ classList: { add() {}, remove() {} }, style: {}, textContent: '', handlers: {},
+        addEventListener(type, handler) { this.handlers[type] = handler; },
+        removeEventListener(type, handler) { if (this.handlers[type] === handler) delete this.handlers[type]; },
+        click() { this.handlers.click?.(); } });
+    const elements = {};
+    const document = { addEventListener() {}, getElementById: id => elements[id] ||= element(),
+        createElement: () => element(), body: { appendChild() {}, removeChild() {} } };
+    const context = vm.createContext({ document, window: {},
+        console: { debug() {}, warn: (...args) => warnings.push(args), error() {} },
+        setTimeout() {}, localStorage: { setItem() {} },
+        SseStream: { connect: (url, options) => { source.url = url; source.options = options; return source; } } });
+    const file = path.join(__dirname, '../../web/js/gallery.js');
+    const Manager = vm.runInContext(fs.readFileSync(file, 'utf8') + '\nGalleryManager;', context, { filename: file });
+    const manager = Object.create(Manager.prototype);
+    Object.assign(manager, { getSettings: () => ({}),
+        api: { get: async url => { gets.push(url); return { success: true, jobs: [] }; },
+            post: async (url, body) => {
+                posts.push([url, body]);
+                if (postFails) throw new Error('offline');
+                return { success: true };
+            } } });
+    const emit = (type, data) => (listeners[type] || []).forEach(handler => handler({ data: JSON.stringify(data) }));
+    const ui = { addStatusMessage() {}, progressText: element(), progressPercent: element(), progressBar: element(),
+        progressDetails: element(), progressETA: element(), isCancelled: () => false,
+        onCancel: handler => { ui.cancel = handler; } };
+    const result = modal ? manager.generateThumbnailsWithProgressInModal(ui)
+        : manager.generateThumbnailsWithProgress({ quality: 'medium' });
+    const outcome = result.then(value => ({ value }), error => ({ error: error.message }));
+    const cancel = () => (modal ? ui.cancel() : elements.cancelThumbnailsBtn.click());
+    const settle = () => new Promise(resolve => setImmediate(resolve));
+    return { emit, source, posts, gets, warnings, outcome, cancel, settle };
+}
+
+const cancelUrl = id => `/prompt_manager/jobs/${id}/cancel`;
+
+test('Cancel before the job id arrives cancels that job once it does', async () => {
+    const f = streamFixture();
+    f.cancel();
+    await f.settle();
+    assert.deepEqual(f.posts, []);
+    assert.equal(f.source.closed, 0); // kept open until the stream names its job
+    assert.deepEqual(await f.outcome, { error: 'Thumbnail generation cancelled by user' });
+    f.emit('status', { phase: 'scanning', message: 'Scanning…' });
+    f.emit('job', { id: JOB, started: true });
+    await f.settle();
+    assert.equal(JSON.stringify(f.posts), JSON.stringify([[cancelUrl(JOB), {}]]));
+    assert.equal(f.source.closed, 1);
+    f.emit('job', { id: 'c'.repeat(32), started: true }); // a stray second frame changes nothing
+    await f.settle();
+    assert.equal(f.posts.length, 1);
+    assert.deepEqual(f.gets, []); // never looks a job up by kind
+});
+
+test('Cancel after the id cancels it at once, and a double Cancel posts once', async () => {
+    const f = streamFixture();
+    f.emit('job', { id: JOB, started: true });
+    f.cancel();
+    f.cancel();
+    await f.settle();
+    assert.equal(JSON.stringify(f.posts), JSON.stringify([[cancelUrl(JOB), {}]]));
+    assert.equal(f.source.closed, 1);
+    assert.deepEqual(await f.outcome, { error: 'Thumbnail generation cancelled by user' });
+    assert.deepEqual(f.gets, []);
+});
+
+test("Cancel leaves another tab's or System's job running and only stops following it", async () => {
+    const f = streamFixture();
+    f.emit('job', { id: JOB, started: false });
+    f.cancel();
+    await f.settle();
+    assert.deepEqual(f.posts, []);
+    assert.deepEqual(f.gets, []);
+    assert.equal(f.source.closed, 1);
+    assert.match((await f.outcome).error, /keeps running/);
+});
+
+test('a Cancel the stream never names a job for posts nothing and closes when it ends', async () => {
+    const f = streamFixture();
+    f.cancel();
+    f.emit('error', { error: 'Scan for metadata already running' });
+    await f.settle();
+    assert.deepEqual(f.posts, []);
+    assert.equal(f.source.closed, 1);
+});
+
+test('a cancel post that fails is reported, not thrown', async () => {
+    const f = streamFixture({ postFails: true });
+    f.emit('job', { id: JOB, started: true });
+    f.cancel();
+    await f.settle();
+    assert.equal(f.posts.length, 1);
+    assert.equal(f.warnings.length, 1);
+    assert.deepEqual(await f.outcome, { error: 'Thumbnail generation cancelled by user' });
+});
+
+test('the startup dialog cancels its own job by id too', async () => {
+    const f = streamFixture({ modal: true });
+    f.cancel();
+    await f.settle();
+    assert.deepEqual(f.posts, []);
+    f.emit('job', { id: JOB, started: true });
+    await f.settle();
+    assert.equal(JSON.stringify(f.posts), JSON.stringify([[cancelUrl(JOB), {}]]));
+    assert.equal(f.source.closed, 1);
+    assert.match((await f.outcome).error, /cancelled/);
+});

tests/js/workspace-activity-kinds.test.js

@@ -0,0 +1,148 @@
+// Per-kind Activity descriptors (#249 System S5 Task 13, D111): each job kind's pill verb, toasts, line, tiles,
+// outcomes, openable rows and labels, from one table (web/js/workspace/activity-kinds.mjs).
+const test = require('node:test');
+const assert = require('node:assert/strict');
+
+const source = import('../../web/js/workspace/activity-kinds.mjs');
+const store = import('../../web/js/workspace/activity-store.mjs');
+
+const ID = 'a'.repeat(32);
+
+function job(overrides = {}) {
+    return { id: ID, kind: 'scan', title: 'Scan for metadata', state: 'running', started_at: 1, finished_at: null,
+        progress: 42, status_line: '', processed: 8240, total: 19622,
+        counters: { added: 1, known: 0, review: 0, no_metadata: 0, errors: 1 }, scope: {}, listed: {}, log_length: 0,
+        cancellable: true, ...overrides };
+}
+
+test('scan descriptors are unchanged', async () => {
+    const { kindOf } = await source;
+    const scan = kindOf('scan');
+    assert.equal(scan.pillVerb, 'Scanning');
+    assert.equal(scan.startToast(job()), 'Scan started.');
+    assert.deepEqual(scan.tiles, [['added', 'Added to Library', 'added'], ['known', 'Already known', ''],
+        ['review', 'Needs review', 'warn'], ['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad']]);
+    assert.deepEqual(scan.outcomes, [['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
+        ['no_metadata', 'No usable metadata'], ['known', 'Already known']]);
+    assert.deepEqual([...scan.openable].sort(), ['added', 'known']);
+    assert.deepEqual([scan.log, scan.cancelLabel], ['Scan log', 'Cancel scan']);
+    assert.equal(scan.finishToast(job({ state: 'finished' })), 'Scan finished · 1 added · 1 error');
+    assert.equal(scan.finishToast(job({ state: 'cancelled' })), 'Scan cancelled · 1 added');
+    assert.equal(scan.finishToast(job({ state: 'failed' })), 'Scan failed · 1 added · 1 error');
+    assert.equal(scan.line(job()), '1 added · 1 error');
+    assert.equal(scan.kept(job()), '1 added');
+    assert.equal(scan.scope(job({ scope: { output_folders: 2 } })), '2 output folders');
+    // The store keeps answering with the same table.
+    const { jobView, finishToast, startToast, pillParts, outcomeLine } = await store;
+    assert.equal(jobView('scan'), scan);
+    assert.equal(finishToast(job({ state: 'finished' })), 'Scan finished · 1 added · 1 error');
+    assert.equal(startToast(job()), 'Scan started.');
+    assert.equal(pillParts(job()).text, 'Scanning · 42% · 8,240 / 19,622');
+    assert.equal(outcomeLine(job()), '1 added · 1 error');
+});
+
+test('thumbnail job shows its own tiles and toasts', async () => {
+    const { kindOf, viewOf } = await source;
+    const { finishToast, startToast, pillParts, outcomeLine } = await store;
+    const thumbs = job({ kind: 'thumbnails', title: 'Thumbnail generation', processed: 4, total: 10, progress: 40,
+        counters: { generated: 1420, skipped: 3, errors: 1 }, scope: { mode: 'missing', size: 512, quality: 'medium',
+            output_folders: 2 } });
+    const view = kindOf('thumbnails');
+    assert.equal(viewOf(thumbs), view);
+    assert.deepEqual(view.tiles, [['generated', 'Generated', 'added'], ['skipped', 'Skipped', ''], ['errors', 'Errors', 'bad']]);
+    assert.deepEqual(view.outcomes, [['generated', 'Generated'], ['errors', 'Errors'], ['skipped', 'Skipped']]);
+    assert.deepEqual([...view.openable].sort(), ['generated', 'skipped']);
+    assert.deepEqual([view.log, view.cancelLabel], ['Job log', 'Cancel job']);
+    assert.equal(pillParts(thumbs).text, 'Generating thumbnails · 40% · 4 / 10');
+    assert.equal(startToast(thumbs), 'Thumbnail generation started.');
+    assert.equal(finishToast({ ...thumbs, state: 'finished' }), 'Thumbnails finished · 1,420 generated · 1 error');
+    assert.equal(finishToast({ ...thumbs, state: 'failed' }), 'Thumbnails failed · 1,420 generated · 1 error');
+    assert.equal(finishToast({ ...thumbs, state: 'cancelled' }), 'Thumbnails cancelled · 1,420 generated');
+    assert.equal(finishToast({ ...thumbs, counters: { generated: 0, skipped: 0, errors: 2 }, state: 'finished' }),
+        'Thumbnails finished · 0 generated · 2 errors');
+    assert.equal(outcomeLine(thumbs), '1,420 generated · 1 error');
+    assert.equal(view.kept(thumbs), '1,420 generated');
+    assert.equal(view.scope(thumbs), '2 output folders');
+});
+
+test('non-cancellable jobs show no Cancel job', async () => {
+    const { canCancel } = await source;
+    assert.equal(canCancel(job()), true);
+    assert.equal(canCancel(job({ cancellable: false })), false);
+    for (const state of ['finished', 'cancelled', 'failed', 'lost']) assert.equal(canCancel(job({ state })), false);
+    // The health check is a read job the server cannot stop.
+    assert.equal(canCancel(job({ kind: 'health', title: 'Health check', cancellable: false, counters: {} })), false);
+});
+
+test('indeterminate progress has no value attribute', async () => {
+    const { indeterminate, showProgress } = await source;
+    const { pillParts } = await store;
+    function node() {
+        const attributes = new Map();
+        return { attributes, textContent: '', set value(v) { attributes.set('value', String(v)); },
+            get value() { return Number(attributes.get('value') ?? 0); },
+            setAttribute(key, v) { attributes.set(key, String(v)); }, removeAttribute(key) { attributes.delete(key); } };
+    }
+    // Running with no total yet: the server has not said how much there is, so there is no percentage.
+    const health = job({ kind: 'health', title: 'Health check', progress: 0, processed: 0, total: 0, counters: {},
+        cancellable: false, status_line: 'Checking the database and image links' });
+    assert.equal(indeterminate(health), true);
+    const bar = node();
+    bar.value = 37;
+    showProgress(bar, health);
+    assert.equal(bar.attributes.has('value'), false);
+    assert.equal(bar.textContent, '');
+    assert.equal(pillParts(health).text, 'Health check');
+    assert.deepEqual(pillParts(health), { label: 'Health check', files: '', text: 'Health check' });
+    // A known total is determinate, and so is any ended job.
+    const scan = job();
+    assert.equal(indeterminate(scan), false);
+    showProgress(bar, scan);
+    assert.equal(bar.attributes.get('value'), '42');
+    assert.equal(bar.textContent, '42%');
+    assert.equal(indeterminate({ ...health, state: 'finished', progress: 100 }), false);
+    // A scan before its file count is known is indeterminate too, never "0%".
+    assert.equal(pillParts(job({ total: 0, processed: 0, progress: 0 })).text, 'Scanning');
+});
+
+test('held legacy jobs read "(admin page)" with no outcomes', async () => {
+    const { viewOf } = await source;
+    const { jobDetail } = await import('../../web/js/workspace/activity.mjs');
+    const { finishToast, outcomeLine } = await store;
+    // A hold has no outcome groups (counters {}), whatever its kind: thumbnails held by the gallery's clear included.
+    for (const [kind, title] of [['restore', 'Database restore (admin page)'],
+        ['thumbnails', 'Thumbnail clear (gallery page)'], ['maintenance', 'Database maintenance (admin page)']]) {
+        const held = job({ kind, title, counters: {}, cancellable: false, progress: 0, processed: 0, total: 0 });
+        const view = viewOf(held);
+        assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []], kind);
+        assert.equal(outcomeLine(held), '');
+        assert.match(jobDetail(held), /^Started \d\d:\d\d · closing this view does not cancel it$/);
+        const ended = { ...held, state: 'finished', finished_at: 2 };
+        assert.match(jobDetail(ended), /^Finished \d\d:\d\d$/);
+        assert.equal(finishToast(ended), `${title} finished.`);
+        assert.ok(title.endsWith('page)'));
+    }
+});
+
+test('unknown and generic kinds show no tiles of another kind', async () => {
+    const { kindOf, viewOf } = await source;
+    for (const kind of ['maintenance', '', 'constructor', '__proto__', 'health']) {
+        const view = kindOf(kind);
+        assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []], kind);
+        assert.deepEqual([view.log, view.cancelLabel], ['Job log', 'Cancel job']);
+        assert.equal(view.pillVerb, null);
+        assert.equal(view.line(job({ kind })), '');
+        assert.equal(view.kept(job({ kind })), '');
+    }
+    const backup = job({ kind: 'backup', title: 'Backup' });
+    assert.equal(kindOf('backup').startToast(backup), 'Backup started.');
+    assert.equal(kindOf('backup').finishToast({ ...backup, state: 'finished' }), 'Backup finished.');
+    assert.equal(kindOf('backup').finishToast({ ...backup, state: 'cancelled' }), 'Backup was cancelled.');
+    assert.equal(kindOf('backup').finishToast({ ...backup, state: 'failed' }), 'Backup failed.');
+    // A kind's tiles are only the groups its job counts: no zero tile for a group the server never reported.
+    const partial = viewOf(job({ counters: { added: 3, errors: 0 } }));
+    assert.deepEqual(partial.tiles, [['added', 'Added to Library', 'added'], ['errors', 'Errors', 'bad']]);
+    assert.deepEqual(partial.outcomes, [['added', 'Added to Library'], ['errors', 'Errors']]);
+    assert.deepEqual([...partial.openable], ['added']);
+    assert.equal(partial.log, 'Scan log');
+});

tests/js/workspace-activity-store.test.js

@@ -289,8 +289,10 @@ test('summaries are validated and the copy follows the spec', async () => {
 
     const scan = parseJob(summary({ progress: 42, processed: 8240, total: 19622, counters: { added: 1, errors: 1 } }));
     assert.deepEqual(pillParts(scan), { label: 'Scanning · 42%', files: ' · 8,240 / 19,622', text: 'Scanning · 42% · 8,240 / 19,622' });
-    assert.equal(pillParts({ ...scan, total: 0 }).text, 'Scanning · 42%');
-    assert.equal(pillParts({ ...scan, kind: 'thumbnails', title: 'Thumbnails' }).text, 'Thumbnails · 42% · 8,240 / 19,622');
+    // D111: without a total the progress is indeterminate, so the pill shows no percentage.
+    assert.equal(pillParts({ ...scan, total: 0 }).text, 'Scanning');
+    // A kind without a pill verb shows its title (D111: thumbnails have their own, see workspace-activity-kinds.test.js).
+    assert.equal(pillParts({ ...scan, kind: 'backup', title: 'Backup' }).text, 'Backup · 42% · 8,240 / 19,622');
     assert.equal(finishToast({ ...scan, state: 'finished' }), 'Scan finished · 1 added · 1 error');
     assert.equal(finishToast({ ...scan, state: 'cancelled' }), 'Scan cancelled · 1 added');
     assert.equal(finishToast({ ...scan, state: 'failed' }), 'Scan failed · 1 added · 1 error');
@@ -332,7 +334,8 @@ test('image-tags jobs get their own tiles outcomes and pill verb', async () => {
         'Add tags to images finished · 0 tagged · 0 errors');
 });
 
-test('unknown kinds keep the generic view', async () => {
+// D111 (S5): an unknown kind no longer borrows the scan's tiles (S1 review M6, S2 review M3); it has none.
+test('unknown kinds get the generic view without the scan tiles', async () => {
     const { jobView } = await source;
     const scan = jobView('scan');
     assert.deepEqual(scan.tiles, [['added', 'Added to Library', 'added'], ['known', 'Already known', ''],
@@ -343,7 +346,7 @@ test('unknown kinds keep the generic view', async () => {
     assert.deepEqual([scan.log, scan.cancel], ['Scan log', 'Cancel scan']);
     for (const kind of ['maintenance', '', 'constructor', '__proto__']) {
         const view = jobView(kind);
-        assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [scan.tiles, scan.outcomes, [...scan.openable]]);
+        assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []]);
         assert.deepEqual([view.log, view.cancel], ['Job log', 'Cancel job']);
     }
 });

tests/js/workspace-history-media.test.js

@@ -39,3 +39,4 @@ test('stale inspection echoes use the canonical recovery vocabulary and identity
         });
     }
 });
+test('the recorded output inspector cover is its preview, not the original (#249)',()=>{const a=adapter(),target=media('2',1);const record=a.toRecord(envelope(target),target,extractGenerationFacts);assert.deepEqual(record.preview,{url:target.thumbnail_url});assert.notEqual(record.preview.url,record.download.url);});

tests/js/workspace-inspection-state.test.js

@@ -895,3 +895,30 @@ test('cache off never rereads a selection whose source changed, and refresh neve
     assert.equal(deferred.state.deferred, true);
     deferred.dispose();
 });
+
+test('record carries the listing thumbnail as its preview, never another file (#249)', () => {
+    const toRecord = contract('toInspectionRecord'), entry = image();
+    const thumb = `/prompt_manager/images/serve/thumbnails/folder/a%20%23%25_thumb.png?root=0&root_scope=${scopeA}`;
+    assert.deepEqual(toRecord(envelope(entry), { ...entry, thumbnail_url: thumb }, extractGenerationFacts).preview, { url: thumb });
+    const versioned = `/prompt_manager/images/serve/thumbnails/folder/a%20%23%25_thumb.png?v=12&root=0&root_scope=${scopeA}`;
+    assert.deepEqual(toRecord(envelope(entry), { ...entry, thumbnail_url: versioned }, extractGenerationFacts).preview, { url: versioned });
+    // No thumbnail, or one naming another file, root or origin: no preview (the cover keeps the original).
+    for (const thumbnail_url of [undefined, null, '', thumb.replace('a%20%23%25_thumb', 'b_thumb'), thumb.replace('root=0', 'root=1'),
+        thumb.replace(scopeA, scopeB), `https://other.test${thumb}`, thumb.replace('thumbnails/', ''), `${thumb}#x`]) {
+        assert.equal(toRecord(envelope(entry), { ...entry, thumbnail_url }, extractGenerationFacts).preview, null, String(thumbnail_url));
+    }
+    const video = { root_index: 0, path: 'clip.MP4', filename: 'clip.MP4', url: `/prompt_manager/images/serve/clip.MP4?root=0&root_scope=${scopeA}`, root_scope: scopeA,
+        thumbnail_url: `/prompt_manager/images/serve/thumbnails/clip_thumb.jpg?root=0&root_scope=${scopeA}` };
+    const data = envelope(video); data.file = { ...data.file, media_type: 'video', extension: '.mp4', format: 'MP4' };
+    assert.deepEqual(toRecord(data, video, extractGenerationFacts).preview, { url: video.thumbnail_url });
+});
+
+test("names the server quotes (parentheses, '!*') still match their original and preview (#249 M11)", () => {
+    const toRecord = contract('toInspectionRecord');
+    const entry = { root_index: 0, path: "folder/a (1)!'*.png", filename: "a (1)!'*.png", root_scope: scopeA,
+        url: `/prompt_manager/images/serve/folder/a%20%281%29%21%27%2A.png?root=0&root_scope=${scopeA}`,
+        thumbnail_url: `/prompt_manager/images/serve/thumbnails/folder/a%20%281%29%21%27%2A_thumb.png?root=0&root_scope=${scopeA}` };
+    const record = toRecord(envelope(entry), entry, extractGenerationFacts);
+    assert.equal(record.download.url, entry.url);
+    assert.deepEqual(record.preview, { url: entry.thumbnail_url });
+});

tests/js/workspace-system-overview.test.js

@@ -253,3 +253,11 @@ test('waiting stops when its signal aborts', async () => {
     gone.abort(new Error('already'));
     await assert.rejects(waitForJob(fakeActivity(), JOB, { signal: gone.signal }), { message: 'already' });
 });
+
+test('the thumbnail row counts previews that could not be made (#249)', async () => {
+    const { thumbnailTitle } = await overviewModule;
+    assert.equal(thumbnailTitle({ pending: 12, failed: 2 }), '12 missing thumbnails · 2 failed');
+    assert.equal(thumbnailTitle({ pending: 1, failed: 1 }), '1 missing thumbnail · 1 failed');
+    assert.equal(thumbnailTitle({ pending: 0, failed: 0 }), 'No missing thumbnails');
+    assert.equal(thumbnailTitle({ pending: 3, failed: 'x' }), '3 missing thumbnails');
+});

tests/js/workspace-thumbnail-review.test.js

@@ -0,0 +1,93 @@
+// System › thumbnail review (#249 System S5 Task 14): the concept's "Thumbnail maintenance" dialog (app.js 1433–1437,
+// `thumbnail-review`) on the thumbnail job (S5 Task 12). Pure parts: the option values the job takes, the preview size
+// check, the Image loading quality preselection (Q3), the D99 plan line and the start outcome.
+const test = require('node:test');
+const assert = require('node:assert/strict');
+
+const reviewModule = import('../../web/js/workspace/thumbnail-review.mjs');
+
+test('the concept options map to the job values in the concept order', async () => {
+    const { SCOPE_OPTIONS, QUALITY_OPTIONS, SIZE_LIMIT } = await reviewModule;
+    assert.deepEqual(SCOPE_OPTIONS, [['missing', 'Missing thumbnails only'], ['rebuild', 'Rebuild all (explicit opt-in)']]);
+    assert.deepEqual(QUALITY_OPTIONS, [['medium', 'Medium · optimized'], ['high', 'High · original'], ['low', 'Low · fastest']]);
+    assert.deepEqual(SIZE_LIMIT, { min: 128, max: 2048, value: 512 });
+});
+
+test('size outside 128–2048 reports validity: only whole pixels in range are a size', async () => {
+    const { previewSize } = await reviewModule;
+    assert.equal(previewSize('512'), 512);
+    assert.equal(previewSize('128'), 128);
+    assert.equal(previewSize('2048'), 2048);
+    for (const text of ['', ' ', '127', '2049', '4096', '512.5', '-512', 'abc', '1e3', '0x200']) {
+        assert.equal(previewSize(text), null, text);
+    }
+});
+
+test('Image loading quality preselects the dialog quality (Q3); anything else is Medium', async () => {
+    const { preselectedQuality } = await reviewModule;
+    assert.equal(preselectedQuality('low'), 'low');
+    assert.equal(preselectedQuality('high'), 'high');
+    assert.equal(preselectedQuality('medium'), 'medium');
+    assert.equal(preselectedQuality(undefined), 'medium');
+    assert.equal(preselectedQuality('original'), 'medium');
+});
+
+test('the plan line counts what a missing-only run generates and what is already up to date (D99)', async () => {
+    const { planText } = await reviewModule;
+    assert.equal(planText({ mode: 'missing', targets: 12, fresh: 30, remaining: 0 }), '12 previews to generate · 30 already up to date.');
+    assert.equal(planText({ mode: 'missing', targets: 1, fresh: 1, remaining: 0 }), '1 preview to generate · 1 already up to date.');
+    assert.equal(planText({ mode: 'missing', targets: 0, fresh: 0, remaining: 0 }), '0 previews to generate · 0 already up to date.');
+    assert.equal(planText({ mode: 'missing', targets: 10000, fresh: 5, remaining: 2500 }),
+        '10,000 previews to generate · 5 already up to date · 2,500 more on the next run.');
+});
+
+test('Rebuild all says every preview is regenerated (D99)', async () => {
+    const { planText } = await reviewModule;
+    assert.equal(planText({ mode: 'rebuild', targets: 42, fresh: 30, remaining: 0 }), 'Every preview is regenerated: 42 files.');
+    assert.equal(planText({ mode: 'rebuild', targets: 1, fresh: 1, remaining: 0 }), 'Every preview is regenerated: 1 file.');
+    assert.equal(planText({ mode: 'rebuild', targets: 10000, fresh: 9000, remaining: 3 }),
+        'Every preview is regenerated: 10,000 files · 3 more on the next run.');
+});
+
+test('an answer without counts is a failed plan, never zeros', async () => {
+    const { planText, PLAN_FAILED } = await reviewModule;
+    assert.equal(planText(null), PLAN_FAILED);
+    assert.equal(planText({ mode: 'missing', targets: 2 }), PLAN_FAILED);
+    assert.equal(planText({ mode: 'missing', targets: -1, fresh: 0, remaining: 0 }), PLAN_FAILED);
+    assert.equal(planText({ mode: 'other', targets: 1, fresh: 0, remaining: 0 }), PLAN_FAILED);
+});
+
+const scopeOf = () => ({ signal: new AbortController().signal });
+
+test('Start starts the thumbnail job with the chosen options and opens it in Activity', async () => {
+    const { startThumbnails } = await reviewModule;
+    const calls = [];
+    const navigated = [];
+    const store = { start: async (kind, body) => { calls.push([kind, body]); return { job: { id: 'a'.repeat(32) } }; } };
+    await startThumbnails({ store, scope: scopeOf(), options: { mode: 'rebuild', size: 1024, quality: 'high' },
+        navigate: hash => navigated.push(hash), notify: () => assert.fail('no toast of its own') });
+    assert.deepEqual(calls, [['thumbnails', { mode: 'rebuild', size: 1024, quality: 'high' }]]);
+    assert.deepEqual(navigated, [`#/activity/${'a'.repeat(32)}`]);
+});
+
+test('a refused start shows the busy link toast and stays', async () => {
+    const { startThumbnails } = await reviewModule;
+    const toasts = [];
+    const store = { start: async () => ({ busy: { id: 'b'.repeat(32), title: 'Scan for metadata' } }) };
+    await startThumbnails({ store, scope: scopeOf(), options: { mode: 'missing', size: 512, quality: 'medium' },
+        navigate: () => assert.fail('stays'), notify: message => toasts.push(message) });
+    assert.deepEqual(toasts, [['Another job is running: ', { text: 'Scan for metadata', href: `#/activity/${'b'.repeat(32)}` }, '.']]);
+});
+
+test('a failed start says so; a start after the section left says nothing', async () => {
+    const { startThumbnails, START_FAILED } = await reviewModule;
+    const toasts = [];
+    const store = { start: async () => { throw new Error('offline'); } };
+    await startThumbnails({ store, scope: scopeOf(), options: { mode: 'missing', size: 512, quality: 'medium' },
+        navigate: () => assert.fail('stays'), notify: message => toasts.push(message) });
+    assert.deepEqual(toasts, [START_FAILED]);
+    const gone = new AbortController();
+    gone.abort();
+    await startThumbnails({ store, scope: { signal: gone.signal }, options: { mode: 'missing', size: 512, quality: 'medium' },
+        navigate: () => assert.fail('stays'), notify: () => assert.fail('silent') });
+});

tests/test_api_admin.py

@@ -79,6 +79,8 @@ class AdminAPITestCase(AioHTTPTestCase):
 
     async def tearDownAsync(self):
         await super().tearDownAsync()  # closes the aiohttp test client
+        # The automatic preview worker writes into the temp tree: stop it first.
+        self.api.close_auto_thumbnails()
         GalleryConfig.MONITORING_DIRECTORIES = self._orig_gallery_dirs
         PromptManagerConfig.update_config(self._orig_pm_config)
         for key, value in self._orig_env.items():
@@ -2243,7 +2245,12 @@ class TestActivityJobs(AdminAPITestCase):
             (self.output_dir / name).write_bytes(b"not an image")
 
         def fake_ingest(
-            media_file, metadata, detailed=False, ambiguous=None, untraced=None
+            media_file,
+            metadata,
+            detailed=False,
+            ambiguous=None,
+            untraced=None,
+            records=None,
         ):
             key = Path(media_file).relative_to(self.output_dir).as_posix()
             outcome = names[key]

tests/test_api_autotag.py

@@ -131,6 +131,8 @@ class AutotagAPITestCase(AioHTTPTestCase):
 
     async def tearDownAsync(self):
         await super().tearDownAsync()  # closes the aiohttp test client
+        # The automatic preview worker writes into the temp tree: stop it first.
+        self.api.close_auto_thumbnails()
         self._service_patch.stop()
         if model_gate is not None:
             leaked = model_gate.holder()

tests/test_api_backup_restore.py

@@ -47,11 +47,13 @@ def _texts(path):
 
 
 class BackupRestoreTestCase(AioHTTPTestCase):
+    app_options = {}
+
     async def get_application(self):
         self.tmpdir = tempfile.mkdtemp()
         self.db_path = os.path.join(self.tmpdir, "prompts.db")
 
-        app = web.Application()
+        app = web.Application(**self.app_options)
         routes = web.RouteTableDef()
         self.api = PromptManagerAPI(db=PromptDatabase(self.db_path))
         self.api.add_routes(routes)
@@ -60,6 +62,8 @@ class BackupRestoreTestCase(AioHTTPTestCase):
 
     async def tearDownAsync(self):
         await super().tearDownAsync()  # closes the aiohttp test client
+        # The automatic preview worker writes into the temp tree: stop it first.
+        self.api.close_auto_thumbnails()
         PromptModel.close_all_instances()  # restore may have swapped api.db
         assert_closed(self, self.tmpdir)
         for name in os.listdir(self.tmpdir):
@@ -327,6 +331,15 @@ class TestRestoreRoute(BackupRestoreTestCase):
         self.assertIn("too large", data["error"].lower())
         self.assertEqual([p["text"] for p in self.api.db.search_prompts()], ["keep me"])
 
+    async def test_restore_cap_keeps_the_legacy_shape(self):
+        self.api.restore_max_bytes = 1024
+        upload = self._write("big.db", b"SQLite format 3\x00" + b"\x00" * 8192)
+        resp = await self._restore(upload)
+        data = await resp.json()
+        self.assertEqual(resp.status, 400)
+        self.assertEqual(data["code"], "too-large")
+        self.assertNotIn(self.tmpdir, data["error"])
+
     async def test_non_multipart_request_is_rejected(self):
         resp = await self.client.post("/prompt_manager/restore", data=b"raw bytes")
         self.assertIn(resp.status, (400, 500))
@@ -336,3 +349,132 @@ class TestRestoreRoute(BackupRestoreTestCase):
 
 if __name__ == "__main__":
     unittest.main()
+
+
+MIB = 2**20
+
+
+class TestLargeRestores(BackupRestoreTestCase):
+    """A real library's backup restores: room on disk decides, not 100 MB."""
+
+    # As ComfyUI builds its app: client_max_size is --max-upload-size (100
+    # MB by default). The restore streams with read_chunk, which it does not
+    # cap.
+    app_options = {"client_max_size": 100 * 2**20}
+
+    def _large_db_file(self, texts, pad_bytes):
+        path = self._make_db_file(texts, name="large.db")
+        conn = sqlite3.connect(path)
+        try:
+            conn.execute("CREATE TABLE padding(b BLOB)")
+            conn.execute("INSERT INTO padding VALUES (zeroblob(?))", (pad_bytes,))
+            conn.commit()
+        finally:
+            conn.close()
+        return path
+
+    async def _stream_restore(self, path, chunked=False):
+        with open(path, "rb") as fh:
+            data = fh.read()
+        form = aiohttp.FormData()
+        form.add_field(
+            "database_file",
+            data,
+            filename=os.path.basename(path),
+            content_type="application/octet-stream",
+        )
+        # chunked=False must not be passed: aiohttp 3.14 then frames the body
+        # as chunked while also sending a Content-Length.
+        options = {"chunked": True} if chunked else {}
+        return await self.client.post("/prompt_manager/restore", data=form, **options)
+
+    def _spy_uploads(self):
+        """Patch the upload's mkstemp; the list collects each temp path made."""
+        created = []
+        real_mkstemp = tempfile.mkstemp
+
+        def spy(*args, **kwargs):
+            result = real_mkstemp(*args, **kwargs)
+            created.append(result[1])
+            return result
+
+        return patch("py.api.admin.tempfile.mkstemp", spy), created
+
+    def _assert_removed(self, created):
+        for path in created:
+            for suffix in ("", "-wal", "-shm", "-journal"):
+                self.assertFalse(os.path.exists(path + suffix), path + suffix)
+
+    async def test_a_150_mb_backup_restores(self):
+        self._save("before")
+        upload = self._large_db_file(["restored"], 150 * MIB)
+        self.assertGreater(os.path.getsize(upload), 150 * MIB)
+        spy, created = self._spy_uploads()
+        with spy:
+            resp = await self._stream_restore(upload)
+        data = await resp.json()
+        self.assertEqual(resp.status, 200, data)
+        self.assertEqual(
+            [p["text"] for p in self.api.db.search_prompts()], ["restored"]
+        )
+        self.assertEqual(len(created), 1)
+        self._assert_removed(created)
+
+    async def test_a_short_disk_is_refused_up_front(self):
+        from py.api import maintenance
+
+        self._save("keep me")
+        upload = self._make_db_file(["restored"])
+        spy, created = self._spy_uploads()
+        with spy, patch.object(maintenance, "_free_bytes", return_value=MIB):
+            resp = await self._restore(upload)
+        data = await resp.json()
+        self.assertEqual(resp.status, 507)
+        self.assertEqual(data["code"], "insufficient-space")
+        self.assertTrue(
+            data["error"].startswith("Not enough free disk space to restore a "),
+            data["error"],
+        )
+        self.assertTrue(data["error"].endswith("Nothing was changed."))
+        self.assertNotIn(self.tmpdir, data["error"])
+        self.assertNotIn(tempfile.gettempdir(), data["error"])
+        self.assertEqual([p["text"] for p in self.api.db.search_prompts()], ["keep me"])
+        self.assertEqual(created, [])  # refused before anything was written
+
+    async def test_a_short_disk_is_refused_while_streaming_without_a_length(self):
+        from py.api import maintenance
+
+        self._save("keep me")
+        upload = self._large_db_file(["restored"], 4 * MIB)
+        spy, created = self._spy_uploads()
+        # Room for 1 MiB only, and no Content-Length to refuse up front.
+        with spy, patch.object(maintenance, "restore_room", return_value=MIB):
+            resp = await self._stream_restore(upload, chunked=True)
+        data = await resp.json()
+        self.assertEqual(resp.status, 507)
+        self.assertEqual(data["code"], "insufficient-space")
+        self.assertTrue(data["error"].endswith("Nothing was changed."))
+        self.assertEqual([p["text"] for p in self.api.db.search_prompts()], ["keep me"])
+        self.assertEqual(len(created), 1)
+        self._assert_removed(created)
+
+
+class TestRestoreRoom(unittest.TestCase):
+    def test_room_counts_the_upload_its_staged_copy_and_the_swap(self):
+        from py.api import maintenance
+
+        tmp = tempfile.mkdtemp()
+        try:
+            db_path = os.path.join(tmp, "prompts.db")
+            free = 10 * 2**30
+            with (
+                patch.object(maintenance, "_free_bytes", return_value=free),
+                patch.object(maintenance, "_device", return_value="one disk"),
+            ):
+                room = maintenance.restore_room(db_path)
+            # Upload + staged copy + WAL growth on one disk, no database yet.
+            self.assertEqual(room, (free - maintenance.DISK_MARGIN_BYTES) // 3)
+            self.assertEqual(maintenance.size_label(int(1.6 * 2**30)), "1.6 GB")
+            self.assertEqual(maintenance.size_label(120 * 2**20), "120.0 MB")
+        finally:
+            os.rmdir(tmp)

tests/test_api_core.py

@@ -384,6 +384,8 @@ class CoreRoutesTestCase(AioHTTPTestCase):
 
     async def tearDownAsync(self):
         await super().tearDownAsync()  # closes the aiohttp test client
+        # The automatic preview worker writes into the temp tree: stop it first.
+        self.api.close_auto_thumbnails()
         self.api.db.close_all()
         assert_closed(self, self.tmpdir)
         shutil.rmtree(self.tmpdir, ignore_errors=True)

tests/test_api_duplicate_files.py

@@ -0,0 +1,995 @@
+"""Duplicate image files on the job registry (#249 System S6, Task 17).
+
+``POST /prompt_manager/jobs/duplicates`` finds content-identical media files
+as a ``read`` Activity job; ``GET /prompt_manager/maintenance/duplicates/{id}``
+pages the groups it holds; ``duplicate_files`` previews a removal bound to the
+reviewed keepers, and its apply job removes only the reviewed, revalidated
+non-keepers, moving their database links to the kept copy. The legacy gallery
+routes keep their shapes on the same registry kinds.
+"""
+
+import asyncio
+import hashlib
+import json
+import os
+import sqlite3
+import threading
+from pathlib import Path
+from unittest.mock import patch
+
+try:
+    from tests.open_handles import open_handles_under
+    from tests.test_api_maintenance import ApplyTestCase, _digest
+except ImportError:  # discovered with tests/ as the top-level directory
+    from open_handles import open_handles_under
+    from test_api_maintenance import ApplyTestCase, _digest
+
+from py.api import duplicate_files  # noqa: E402
+from py.config import GalleryConfig, PromptManagerConfig  # noqa: E402
+
+WAIT = 10
+FIND = "/prompt_manager/jobs/duplicates"
+GROUPS = "/prompt_manager/maintenance/duplicates/"
+PREVIEW = "/prompt_manager/maintenance/preview"
+STREAM = "/prompt_manager/scan_duplicates/stream"
+STATUS = "/prompt_manager/scan_duplicates/status"
+LEGACY_DELETE = "/prompt_manager/delete_duplicate_images"
+
+
+def _sse_payloads(body):
+    return [
+        json.loads(line[len("data: ") :])
+        for line in body.splitlines()
+        if line.startswith("data: ")
+    ]
+
+
+def _sha(path):
+    with open(path, "rb") as handle:
+        return hashlib.sha256(handle.read()).hexdigest()
+
+
+class DuplicateFilesTestCase(ApplyTestCase):
+    async def get_application(self):
+        app = await super().get_application()
+        GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
+        return app
+
+    def write(self, name, data, *, folder=None, mtime=None):
+        path = Path(folder or self.output_dir) / name
+        path.parent.mkdir(parents=True, exist_ok=True)
+        path.write_bytes(data)
+        if mtime is not None:
+            os.utime(path, (mtime, mtime))
+        return path
+
+    def second_source(self):
+        """A second valid source outside ComfyUI (an extra gallery root)."""
+        second = self.outside_dir / "renders"
+        second.mkdir(parents=True, exist_ok=True)
+        os.environ["PROMPT_MANAGER_EXTRA_GALLERY_ROOTS"] = str(second)
+        return second
+
+    def trio(self):
+        """a.png (oldest), b.png and c.png share bytes; d.png is unique."""
+        self.write("a.png", b"same bytes", mtime=1_000_000)
+        self.write("b.png", b"same bytes", mtime=2_000_000)
+        self.write("sub/c.png", b"same bytes", mtime=3_000_000)
+        self.write("d.png", b"unique", mtime=4_000_000)
+
+    async def find(self):
+        response = await self.client.request("POST", FIND)
+        text = await response.text()
+        self.assertEqual(response.status, 202, text)
+        self.assertNotIn(self.tmpdir, text)
+        job = self.api._jobs.get(json.loads(text)["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(job.task), WAIT)
+        return job
+
+    async def groups(self, job_id, status=200, **query):
+        params = "&".join(f"{key}={value}" for key, value in query.items())
+        response = await self.client.request("GET", f"{GROUPS}{job_id}?{params}")
+        text = await response.text()
+        self.assertEqual(response.status, status, text)
+        self.assertNotIn(self.tmpdir, text)
+        return json.loads(text)
+
+    async def review(self, job_id, keep, status=200):
+        return await self.post(
+            PREVIEW,
+            {
+                "operation": "duplicate_files",
+                "options": {"job_id": job_id, "keep": keep},
+            },
+            status,
+        )
+
+    async def found_group(self):
+        job = await self.find()
+        [group] = (await self.groups(job.id))["groups"]
+        return job, group
+
+    @staticmethod
+    def keep_of(group, path, root_index=None):
+        [entry] = [
+            f
+            for f in group["files"]
+            if f["path"] == path and root_index in (None, f["root_index"])
+        ]
+        return {
+            "group_id": group["group_id"],
+            "root_index": entry["root_index"],
+            "path": entry["path"],
+        }
+
+    async def remove_all_but(self, path):
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, path)])
+        body = await self.apply(plan["token"])
+        cleanup = self.api._jobs.get(body["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        return plan, cleanup
+
+    async def until(self, predicate):
+        """Wait for a server-side state (a frame at a time), never a fixed timer."""
+        deadline = asyncio.get_running_loop().time() + WAIT
+        while not predicate():
+            if asyncio.get_running_loop().time() > deadline:
+                self.fail("the expected server state never appeared")
+            await asyncio.sleep(0)
+
+    def gate_hashing(self):
+        gate = threading.Event()
+        calls = []
+        original = self.api._calculate_file_hash
+
+        def gated(path):
+            calls.append(path)
+            gate.wait(WAIT)
+            return original(path)
+
+        self.api._calculate_file_hash = gated
+        self.addCleanup(gate.set)
+        return gate, calls
+
+
+class TestFindJob(DuplicateFilesTestCase):
+    async def test_find_is_listed_in_activity_and_legacy_stream_attaches(self):
+        self.trio()
+        gate, calls = self.gate_hashing()
+        response = await self.client.request("POST", FIND)
+        self.assertEqual(response.status, 202)
+        summary = (await response.json())["job"]
+        listed = (
+            await (await self.client.request("GET", "/prompt_manager/jobs")).json()
+        )["jobs"]
+        self.assertEqual([j["id"] for j in listed], [summary["id"]])
+        self.assertEqual(summary["kind"], "duplicates")
+        self.assertEqual(summary["title"], "Find duplicate image files")
+        self.assertEqual(summary["admission"], "read")
+        self.assertTrue(summary["cancellable"])
+        self.assertEqual(summary["counters"], {"groups_found": 0})
+        # The legacy gallery attaches to the same job, and its status says so.
+        legacy = asyncio.ensure_future(self.client.request("POST", STREAM))
+        job = self.api._jobs.get(summary["id"])
+        await self.until(lambda: job.subscriber_count > 0)
+        status = await (await self.client.request("GET", STATUS)).json()
+        self.assertTrue(status["running"])
+        gate.set()
+        body = await (await legacy).text()
+        job = self.api._jobs.get(summary["id"])
+        await asyncio.wait_for(asyncio.shield(job.task), WAIT)
+        events = _sse_payloads(body)
+        done = events[-1]
+        self.assertEqual(done["type"], "complete")
+        self.assertEqual(done["found"], 1)
+        self.assertEqual(
+            sorted(i["filename"] for i in done["duplicates"][0]["images"]),
+            ["a.png", "b.png", "c.png"],
+        )
+        self.assertEqual(len(calls), 4)  # one pass: each file hashed once
+        self.assertEqual(job.state, "finished")
+        self.assertEqual(job.counters, {"groups_found": 1})
+        [row] = job.outcomes["groups_found"]
+        self.assertEqual(row["path"], "a.png")
+        self.assertEqual(row["root_index"], 0)
+        self.assertEqual(row["detail"], "3 identical files · 10 B each")
+        self.assertEqual(job.status_line, "Found 1 group of duplicate files")
+        finished = await (await self.client.request("GET", STATUS)).json()
+        self.assertFalse(finished["running"])
+        self.assertEqual(finished["last_event"]["found"], 1)
+
+    async def test_legacy_stream_starts_a_registry_job(self):
+        self.trio()
+        body = await (await self.client.request("POST", STREAM)).text()
+        self.assertEqual(_sse_payloads(body)[-1]["type"], "complete")
+        [job] = self.api._jobs.list()
+        self.assertEqual(job.kind, "duplicates")
+        self.assertEqual(job.title, "Find duplicate image files (gallery page)")
+        self.assertNotIn(self.tmpdir, body)
+        # Its groups are reviewable in the workspace too.
+        page = await self.groups(job.id)
+        self.assertEqual(page["total"], 1)
+
+    async def test_find_covers_every_source_with_its_index(self):
+        second = self.second_source()
+        self.write("x.png", b"twin", folder=second)
+        GalleryConfig.MONITORING_DIRECTORIES = [
+            self._root(self.output_dir),
+            self._root(second),
+        ]
+        self.write("x.png", b"twin")
+        job = await self.find()
+        [group] = (await self.groups(job.id))["groups"]
+        self.assertEqual(
+            sorted((f["root_index"], f["path"]) for f in group["files"]),
+            [(0, "x.png"), (1, "x.png")],
+        )
+        self.assertEqual({f["root"] for f in group["files"]}, {"output", "renders"})
+
+    async def test_find_can_be_cancelled(self):
+        self.trio()
+        gate, _calls = self.gate_hashing()
+        response = await self.client.request("POST", FIND)
+        job_id = (await response.json())["job"]["id"]
+        stream = asyncio.ensure_future(self.client.request("POST", STREAM))
+        found = self.api._jobs.get(job_id)
+        await self.until(lambda: found.subscriber_count > 0)
+        cancel = await self.client.request(
+            "POST", f"/prompt_manager/jobs/{job_id}/cancel"
+        )
+        self.assertTrue((await cancel.json())["cancellation_requested"])
+        gate.set()
+        job = self.api._jobs.get(job_id)
+        await asyncio.wait_for(asyncio.shield(job.task), WAIT)
+        self.assertEqual(job.state, "cancelled")
+        last = _sse_payloads(await (await stream).text())[-1]
+        self.assertEqual(last, {"type": "error", "message": "Duplicate scan cancelled"})
+        await self.groups(job_id, 409)
+
+    async def test_second_find_is_busy_and_names_the_running_job(self):
+        self.trio()
+        gate, _calls = self.gate_hashing()
+        first = (await (await self.client.request("POST", FIND)).json())["job"]
+        second = await self.client.request("POST", FIND)
+        self.assertEqual(second.status, 409)
+        body = await second.json()
+        self.assertEqual(body["code"], "busy")
+        self.assertEqual(body["running"]["id"], first["id"])
+        gate.set()
+
+    async def test_failed_find_is_coded_and_releases_the_kind(self):
+        self.write("a.png", b"x")
+        with patch.object(
+            self.api, "_hash_media_groups", side_effect=OSError(f"boom {self.tmpdir}")
+        ):
+            with self.assertLogs("prompt_manager", "ERROR"):
+                job = await self.find()
+        self.assertEqual(job.state, "failed")
+        self.assertEqual(job.status_line, "The duplicate search failed.")
+        self.assertNotIn("boom", json.dumps(job.report()))
+        await self.groups(job.id, 409)
+        self.assertEqual((await self.find()).state, "finished")
+
+
+class TestGroupPages(DuplicateFilesTestCase):
+    async def test_groups_are_paged_ten_at_a_time(self):
+        for index in range(12):
+            for copy in range(2):
+                self.write(f"g{index:02d}_{copy}.png", f"group {index}".encode())
+        job = await self.find()
+        first = await self.groups(job.id)
+        self.assertEqual((first["total"], first["offset"], first["limit"]), (12, 0, 10))
+        self.assertEqual(len(first["groups"]), 10)
+        rest = await self.groups(job.id, offset=10, limit=10)
+        self.assertEqual(len(rest["groups"]), 2)
+        ids = [g["group_id"] for g in first["groups"] + rest["groups"]]
+        self.assertEqual(len(set(ids)), 12)
+        entry = first["groups"][0]["files"][0]
+        for key in ("path", "root", "root_index", "url", "size", "media_type"):
+            self.assertIn(key, entry)
+        self.assertTrue(entry["url"].endswith("?root=0"))
+        for query in ({"limit": 11}, {"limit": 0}, {"offset": -1}, {"offset": "x"}):
+            with self.subTest(query=query):
+                reply = await self.groups(job.id, 400, **query)
+                self.assertEqual(reply["code"], "invalid-request")
+
+    async def test_unknown_or_running_job_has_no_groups(self):
+        reply = await self.groups("f" * 32, 404)
+        self.assertEqual(reply["code"], "not-found")
+        self.trio()
+        gate, _calls = self.gate_hashing()
+        job_id = (await (await self.client.request("POST", FIND)).json())["job"]["id"]
+        reply = await self.groups(job_id, 409)
+        self.assertEqual(reply["code"], "job-running")
+        gate.set()
+
+
+class TestReview(DuplicateFilesTestCase):
+    async def test_preview_lists_only_non_keepers_and_writes_nothing(self):
+        self.trio()
+        prompt = self.save("Linked")
+        self.api.db.link_image_to_prompt(prompt, str(self.output_dir / "b.png"))
+        job, group = await self.found_group()
+        db_path = self.api.db.model.db_path
+        files = [db_path, db_path + "-wal"]
+        before = [_digest(path) for path in files]
+        media = {p: _digest(p) for p in self.output_dir.rglob("*") if p.is_file()}
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.assertEqual([_digest(path) for path in files], before)
+        self.assertEqual(
+            {p: _digest(p) for p in self.output_dir.rglob("*") if p.is_file()},
+            media,
+        )
+        self.assertEqual(plan["operation"], "duplicate_files")
+        self.assertEqual(
+            plan["scope"], {"count": 2, "label": "2 duplicate files in 1 group"}
+        )
+        self.assertTrue(plan["media"]["deletes_files"])
+        self.assertEqual(
+            plan["media"]["label"],
+            "Deletes 2 files (20 B) from disk; the kept copies stay. "
+            "The safety backup covers the database, not these files.",
+        )
+        self.assertEqual(
+            [row["label"] for row in plan["details"]], ["b.png", "sub/c.png"]
+        )
+        self.assertEqual(
+            plan["details"][0]["detail"],
+            "output · a copy of a.png · 1 database link moves to the kept copy",
+        )
+        self.assertEqual(plan["protected"], {"count": 0, "reasons": []})
+
+    async def test_cannot_delete_keeper_or_outside_group(self):
+        self.trio()
+        self.write("e.png", b"other pair")
+        self.write("f.png", b"other pair")
+        job = await self.find()
+        groups = (await self.groups(job.id))["groups"]
+        trio = next(g for g in groups if len(g["files"]) == 3)
+        pair = next(g for g in groups if len(g["files"]) == 2)
+        bad = [
+            # a file of another group named as this group's keeper
+            [{**self.keep_of(trio, "a.png"), "path": "e.png"}],
+            # a file that is in no group
+            [{**self.keep_of(trio, "a.png"), "path": "d.png"}],
+            # the right path under another source index
+            [{**self.keep_of(trio, "a.png"), "root_index": 1}],
+            [{**self.keep_of(trio, "a.png"), "group_id": "0" * 64}],
+            [{**self.keep_of(trio, "a.png"), "extra": 1}],
+            [],
+            "a.png",
+        ]
+        for keep in bad:
+            with self.subTest(keep=keep):
+                reply = await self.review(job.id, keep, 400)
+                self.assertEqual(reply["code"], "invalid-request")
+        # Two keepers of one group: only the third copy goes.
+        plan = await self.review(
+            job.id,
+            [self.keep_of(trio, "a.png"), self.keep_of(trio, "sub/c.png")],
+        )
+        self.assertEqual([row["label"] for row in plan["details"]], ["b.png"])
+        body = await self.apply(plan["token"])
+        cleanup = self.api._jobs.get(body["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.counters["removed"], 1)
+        self.assertTrue((self.output_dir / "a.png").is_file())
+        self.assertTrue((self.output_dir / "sub" / "c.png").is_file())
+        self.assertFalse((self.output_dir / "b.png").exists())
+        # The untouched group is untouched.
+        self.assertTrue((self.output_dir / "e.png").is_file())
+        self.assertTrue((self.output_dir / "f.png").is_file())
+        self.assertEqual(len(pair["files"]), 2)
+
+    async def test_a_source_removed_since_the_find_is_refused(self):
+        self.trio()
+        job, group = await self.found_group()
+        GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.second_source())]
+        reply = await self.review(job.id, [self.keep_of(group, "a.png")], 409)
+        self.assertEqual(reply["code"], "sources-changed")
+
+    def test_containment_is_decided_as_text_before_the_filesystem(self):
+        root = str(self.output_dir)
+        outside = os.path.join(str(self.outside_dir), "x.png")
+        touched = []
+        real_lstat = os.lstat
+
+        def lstat(path, *args, **kwargs):
+            touched.append(str(path))
+            return real_lstat(path, *args, **kwargs)
+
+        with patch.object(duplicate_files.os, "lstat", side_effect=lstat):
+            for candidate in (
+                outside,
+                os.path.join(root, "..", "outside", "x.png"),
+                os.path.join(root, "thumbnails", "a_thumb.png"),
+                os.path.join(root, "notes.txt"),
+            ):
+                with self.subTest(candidate=candidate):
+                    reason = duplicate_files.file_problem(candidate, root, "0" * 64, 1)
+                    self.assertEqual(reason, duplicate_files.OUTSIDE_SOURCE)
+        self.assertEqual(touched, [])
+
+    @staticmethod
+    def _can_symlink(directory):
+        probe = Path(directory) / "probe-link"
+        try:
+            os.symlink(str(Path(directory) / "x"), str(probe))
+        except (OSError, NotImplementedError):
+            return False
+        probe.unlink()
+        return True
+
+    async def test_symlink_rejected(self):
+        if not self._can_symlink(self.tmpdir):  # pragma: no cover - Windows w/o rights
+            self.skipTest("symlinks are not available")
+        self.trio()
+        job, group = await self.found_group()
+        # b.png becomes a link to the keeper after the find: never removed.
+        (self.output_dir / "b.png").unlink()
+        os.symlink(str(self.output_dir / "a.png"), str(self.output_dir / "b.png"))
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.assertEqual([row["label"] for row in plan["details"]], ["sub/c.png"])
+        self.assertEqual(
+            plan["protected"],
+            {"count": 1, "reasons": ["1 file is a link, not a regular file"]},
+        )
+        # A keeper that became a link protects its whole group.
+        (self.output_dir / "a.png").rename(self.output_dir / "real.png")
+        os.symlink(str(self.output_dir / "real.png"), str(self.output_dir / "a.png"))
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.assertEqual(plan["scope"]["count"], 0)
+        self.assertEqual(
+            plan["protected"]["reasons"],
+            ["2 files kept: their kept copy changed or is gone"],
+        )
+
+
+class TestCleanupJob(DuplicateFilesTestCase):
+    async def test_cleanup_is_a_records_job_with_a_safety_backup(self):
+        self.trio()
+        plan, cleanup = await self.remove_all_but("a.png")
+        summary = cleanup.summary()
+        self.assertEqual(summary["kind"], "duplicate_cleanup")
+        self.assertEqual(summary["title"], "Remove duplicate image files")
+        self.assertEqual(summary["admission"], "records")
+        self.assertFalse(summary["cancellable"])
+        self.assertEqual(summary["scope"], {"operation": "duplicate_files"})
+        self.assertEqual(cleanup.state, "finished", cleanup.log)
+        self.assertEqual(cleanup.counters, {"removed": 2, "kept": 1, "errors": 0})
+        self.assertEqual(
+            sorted(row["path"] for row in cleanup.outcomes["removed"]),
+            ["b.png", "sub/c.png"],
+        )
+        self.assertEqual(cleanup.outcomes["kept"][0]["path"], "a.png")
+        [backup] = self.backups()
+        self.assertEqual(
+            cleanup.status_line,
+            f"Removed 2 duplicate files · Safety backup {backup}",
+        )
+        self.assertEqual(
+            _sha(self.output_dir / "a.png"), hashlib.sha256(b"same bytes").hexdigest()
+        )
+        self.assertFalse((self.output_dir / "b.png").exists())
+        self.assertFalse((self.output_dir / "sub" / "c.png").exists())
+        self.assertNotIn(self.tmpdir, json.dumps(cleanup.report()))
+        # Single use, like every plan.
+        again = await self.apply(plan["token"], 409)
+        self.assertEqual(again["code"], "plan-used")
+
+    async def test_thumbnails_of_removed_files_are_removed(self):
+        self.trio()
+        self.write("clip.mp4", b"video")
+        self.write("copy.mp4", b"video")
+        kept = self.write("thumbnails/a_thumb.png", b"t")
+        gone = self.write("thumbnails/b_thumb.png", b"t")
+        video_thumb = self.write("thumbnails/copy_thumb.jpg", b"t")
+        job = await self.find()
+        groups = (await self.groups(job.id))["groups"]
+        keep = []
+        for group in groups:
+            paths = sorted(f["path"] for f in group["files"])
+            keep.append(
+                self.keep_of(group, "a.png" if "a.png" in paths else "clip.mp4")
+            )
+        plan = await self.review(job.id, keep)
+        body = await self.apply(plan["token"])
+        cleanup = self.api._jobs.get(body["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertTrue(kept.is_file())
+        self.assertFalse(gone.exists())
+        self.assertFalse(video_thumb.exists())
+
+    async def test_changed_content_skips_file(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.write("b.png", b"new bytes")  # changed since the review
+        body = await self.apply(plan["token"])
+        cleanup = self.api._jobs.get(body["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.counters, {"removed": 1, "kept": 2, "errors": 0})
+        self.assertEqual((self.output_dir / "b.png").read_bytes(), b"new bytes")
+        self.assertFalse((self.output_dir / "sub" / "c.png").exists())
+        rows = {row["path"]: row["detail"] for row in cleanup.outcomes["kept"]}
+        self.assertEqual(rows["b.png"], duplicate_files.CHANGED)
+
+    async def test_a_changed_keeper_keeps_its_whole_group(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.write("a.png", b"edited keeper")
+        body = await self.apply(plan["token"])
+        cleanup = self.api._jobs.get(body["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.counters["removed"], 0)
+        self.assertTrue((self.output_dir / "b.png").is_file())
+        self.assertTrue((self.output_dir / "sub" / "c.png").is_file())
+        self.assertTrue(
+            cleanup.status_line.startswith(
+                "Removed 0 duplicate files, 2 not removed · "
+            )
+        )
+        rows = {row["path"]: row["detail"] for row in cleanup.outcomes["kept"]}
+        self.assertEqual(rows["b.png"], duplicate_files.KEEPER_CHANGED)
+
+    async def test_partial_failure_reports_each_path(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        real = duplicate_files._unlink
+
+        def unlink(path):
+            if os.path.basename(path) == "c.png":
+                raise PermissionError(13, f"Permission denied: {path}")
+            real(path)
+
+        self.write("z.png", b"same bytes")  # never reviewed: not touched
+        with patch.object(duplicate_files, "_unlink", side_effect=unlink):
+            body = await self.apply(plan["token"])
+            cleanup = self.api._jobs.get(body["job"]["id"])
+            await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.state, "failed")
+        self.assertEqual(cleanup.counters, {"removed": 1, "kept": 1, "errors": 1})
+        self.assertEqual(
+            [row["path"] for row in cleanup.outcomes["removed"]], ["b.png"]
+        )
+        [error] = cleanup.outcomes["errors"]
+        self.assertEqual(
+            (error["path"], error["detail"]),
+            ("sub/c.png", duplicate_files.LOCKED_FILE),
+        )
+        self.assertEqual(
+            cleanup.status_line,
+            "Stopped after an error · Removed 1 duplicate file, 1 not removed · "
+            "Safety backup " + self.backups()[0],
+        )
+        report = json.dumps(cleanup.report())
+        self.assertNotIn("Permission denied", report)
+        self.assertNotIn("Errno", report)
+        self.assertTrue((self.output_dir / "z.png").is_file())
+
+    async def test_stop_on_failure_leaves_later_files_untouched(self):
+        self.write("a.png", b"same", mtime=1)
+        for name in ("b.png", "c.png", "e.png"):
+            self.write(name, b"same", mtime=2)
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        with (
+            patch.object(
+                duplicate_files, "_unlink", side_effect=OSError(5, "I/O error")
+            ),
+            self.assertLogs("prompt_manager", "WARNING"),
+        ):
+            body = await self.apply(plan["token"])
+            cleanup = self.api._jobs.get(body["job"]["id"])
+            await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.counters, {"removed": 0, "kept": 3, "errors": 1})
+        self.assertEqual(
+            cleanup.outcomes["errors"][0]["detail"], duplicate_files.DELETE_FAILED
+        )
+        not_attempted = [
+            row
+            for row in cleanup.outcomes["kept"]
+            if row["detail"] == duplicate_files.NOT_ATTEMPTED
+        ]
+        self.assertEqual(len(not_attempted), 2)
+        for name in ("b.png", "c.png", "e.png"):
+            self.assertTrue((self.output_dir / name).is_file())
+
+    async def test_locked_file_gives_a_coded_message(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        target = self.output_dir / "b.png"
+        handle = None
+        if os.name == "nt":  # pragma: no cover - the native sharing violation
+            handle = open(target, "rb")
+            context = patch.object(duplicate_files, "_unlink", duplicate_files._unlink)
+        else:
+            context = patch.object(
+                duplicate_files,
+                "_unlink",
+                side_effect=PermissionError(
+                    32, "The process cannot access the file", str(target)
+                ),
+            )
+        try:
+            with context:
+                body = await self.apply(plan["token"])
+                cleanup = self.api._jobs.get(body["job"]["id"])
+                await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        finally:
+            if handle is not None:  # pragma: no cover - Windows
+                handle.close()
+        [error] = cleanup.outcomes["errors"]
+        self.assertEqual(error["detail"], duplicate_files.LOCKED_FILE)
+        self.assertNotIn("WinError", json.dumps(cleanup.report()))
+        self.assertTrue(target.is_file())
+
+    async def test_handles_closed_before_unlink(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        real = duplicate_files._unlink
+        open_at_unlink = []
+
+        def unlink(path):
+            open_at_unlink.extend(open_handles_under(path))
+            real(path)
+
+        with patch.object(duplicate_files, "_unlink", side_effect=unlink):
+            body = await self.apply(plan["token"])
+            cleanup = self.api._jobs.get(body["job"]["id"])
+            await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.counters["removed"], 2)
+        self.assertEqual(open_at_unlink, [])
+
+    async def test_db_links_reconciled_only_for_removed_files(self):
+        self.trio()
+        db = self.api.db
+        moved = self.save("Moves to the keeper")
+        merged = self.save("Already on the keeper")
+        db.link_image_to_prompt(moved, str(self.output_dir / "b.png"))
+        db.link_image_to_prompt(moved, str(self.output_dir / "d.png"))
+        db.link_image_to_prompt(merged, str(self.output_dir / "a.png"))
+        db.link_image_to_prompt(
+            merged,
+            str(self.output_dir / "sub" / "c.png"),
+            {"workflow": {"only": "c"}, "prompt": {"only": "c"}},
+        )
+        db.set_image_favorite(
+            duplicate_files.favorite_root_key(self.output_dir), "b.png"
+        )
+        plan, cleanup = await self.remove_all_but("a.png")
+        # The merged link's own workflow document went with it (#249 S6 dedupe).
+        documents = db.model.get_connection().execute(
+            "SELECT COUNT(*) FROM workflow_documents"
+        )
+        self.assertEqual(documents.fetchone()[0], 1)  # the shared "{}"
+        self.assertEqual(
+            plan["details"][0]["detail"],
+            "output · a copy of a.png · 1 database link moves to the kept copy",
+        )
+
+        def files_of(prompt_id):
+            return sorted(
+                os.path.relpath(image["image_path"], self.output_dir)
+                for image in db.get_prompt_images(prompt_id)
+            )
+
+        self.assertEqual(files_of(moved), ["a.png", "d.png"])
+        self.assertEqual(files_of(merged), ["a.png"])
+        root_key = duplicate_files.favorite_root_key(self.output_dir)
+        self.assertEqual(db.get_root_favorites(root_key), {"a.png"})
+        self.assertIn(
+            "Moved 2 database links to the kept copies", "\n".join(cleanup.log)
+        )
+
+    async def test_a_failed_removal_leaves_its_links_on_the_kept_copy(self):
+        self.trio()
+        prompt = self.save("Stays on c")
+        self.api.db.link_image_to_prompt(prompt, str(self.output_dir / "sub" / "c.png"))
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        real = duplicate_files._unlink
+
+        def unlink(path):
+            if os.path.basename(path) == "c.png":
+                raise PermissionError(13, "denied")
+            real(path)
+
+        with patch.object(duplicate_files, "_unlink", side_effect=unlink):
+            body = await self.apply(plan["token"])
+            cleanup = self.api._jobs.get(body["job"]["id"])
+            await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        # Its records moved to the identical kept copy before the unlink failed.
+        [image] = self.api.db.get_prompt_images(prompt)
+        self.assertTrue(image["image_path"].endswith("a.png"))
+        self.assertTrue((self.output_dir / "sub" / "c.png").is_file())
+
+    async def test_cleanup_and_find_exclude_each_other(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        gate, _calls = self.gate_hashing()
+        find = (await (await self.client.request("POST", FIND)).json())["job"]
+        busy = await self.apply(plan["token"], 409)
+        self.assertEqual((busy["code"], busy["running"]["id"]), ("busy", find["id"]))
+        gate.set()
+        await asyncio.wait_for(
+            asyncio.shield(self.api._jobs.get(find["id"]).task), WAIT
+        )
+        # The token survived the refusal; the cleanup holds off a new find.
+        release = asyncio.Event()
+        real = self.api._run_duplicate_cleanup
+
+        async def held(*args):
+            await release.wait()
+            return await real(*args)
+
+        with patch.object(self.api, "_run_duplicate_cleanup", side_effect=held):
+            body = await self.apply(plan["token"])
+            refused = await self.client.request("POST", FIND)
+            self.assertEqual(refused.status, 409)
+            legacy = await self.client.request("POST", STREAM)
+            self.assertEqual(legacy.status, 409)
+            release.set()
+            cleanup = self.api._jobs.get(body["job"]["id"])
+            await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+
+    async def test_cleanup_refused_up_front_when_the_backup_does_not_fit(self):
+        from py.api import maintenance
+
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        with patch.object(maintenance, "_free_bytes", return_value=0):
+            reply = await self.apply(plan["token"], 507)
+        self.assertEqual(reply["code"], "insufficient-space")
+        self.assertTrue((self.output_dir / "b.png").is_file())
+
+    async def test_policy_off_removes_without_a_backup(self):
+        PromptManagerConfig.SAFETY_BACKUP = False
+        self.trio()
+        _plan, cleanup = await self.remove_all_but("a.png")
+        self.assertEqual(self.backups(), [])
+        self.assertEqual(
+            cleanup.status_line, "Removed 2 duplicate files · No safety backup"
+        )
+
+
+class TestLegacyDuplicateRoutes(DuplicateFilesTestCase):
+    async def test_legacy_delete_duplicates_unchanged(self):
+        self.trio()
+        response = await self.client.request(
+            "POST", LEGACY_DELETE, json={"image_paths": ["b.png", "nope.png"]}
+        )
+        body = await response.json()
+        self.assertEqual(response.status, 200)
+        self.assertEqual(
+            set(body),
+            {"success", "deleted_count", "failed_count", "message", "failed_files"},
+        )
+        self.assertEqual((body["deleted_count"], body["failed_count"]), (1, 1))
+        self.assertEqual(body["failed_files"], ["nope.png (file not found)"])
+        [hold] = self.api._jobs.list()
+        self.assertEqual(
+            (hold.kind, hold.title, hold.admission),
+            ("duplicate_cleanup", "Duplicate image delete (gallery page)", "records"),
+        )
+
+    async def test_legacy_delete_refused_while_a_find_runs(self):
+        self.trio()
+        gate, _calls = self.gate_hashing()
+        await self.client.request("POST", FIND)
+        response = await self.client.request(
+            "POST", LEGACY_DELETE, json={"image_paths": ["b.png"]}
+        )
+        self.assertEqual(response.status, 409)
+        self.assertEqual(
+            await response.json(),
+            {"success": False, "error": "Find duplicate image files already running"},
+        )
+        self.assertTrue((self.output_dir / "b.png").is_file())
+        gate.set()
+
+    async def test_legacy_scan_holds_the_duplicates_kind(self):
+        self.trio()
+        gate, _calls = self.gate_hashing()
+        legacy = asyncio.ensure_future(
+            self.client.request("GET", "/prompt_manager/scan_duplicates")
+        )
+        await self.until(lambda: self.api._jobs.active("duplicates") is not None)
+        [hold] = self.api._jobs.list()
+        self.assertEqual(
+            (hold.kind, hold.title, hold.admission, hold.cancellable),
+            ("duplicates", "Find duplicate image files (gallery page)", "read", False),
+        )
+        refused = await self.client.request("POST", FIND)
+        self.assertEqual(refused.status, 409)
+        # A held legacy scan publishes no events: the stream does not attach.
+        stream = await self.client.request("POST", STREAM)
+        self.assertEqual(stream.status, 409)
+        gate.set()
+        body = await (await legacy).json()
+        self.assertEqual(set(body), {"success", "duplicates", "message"})
+        self.assertEqual(body["duplicates"][0]["count"], 3)
+
+    async def test_legacy_stream_does_not_attach_to_a_find_missing_its_folder(self):
+        other = self.second_source()
+        self.write("x.png", b"x", folder=other)
+        GalleryConfig.MONITORING_DIRECTORIES = [self._root(other)]
+        self.api._find_comfyui_output_dir = lambda: str(self.output_dir)
+        gate, _calls = self.gate_hashing()
+        await self.client.request("POST", FIND)
+        stream = await self.client.request("POST", STREAM)
+        self.assertEqual(stream.status, 409)
+        gate.set()
+
+
+class TestLinkQueries(DuplicateFilesTestCase):
+    def test_link_queries_use_the_path_indexes(self):
+        conn = sqlite3.connect(self.api.db.model.db_path)
+        try:
+            for sql in duplicate_files.query_plan_sql():
+                plan = " ".join(
+                    row[3]
+                    for row in conn.execute(
+                        "EXPLAIN QUERY PLAN " + sql, [None] * sql.count("?")
+                    )
+                )
+                with self.subTest(sql=sql):
+                    self.assertNotIn("SCAN generated_images", plan)
+                    self.assertNotIn("SCAN image_favorites", plan)
+        finally:
+            conn.close()
+
+
+class TestReviewHelpers(DuplicateFilesTestCase):
+    """The pure checks of py/api/duplicate_review.py."""
+
+    def test_file_problem_on_disk(self):
+        from py.api import duplicate_review as review
+
+        root = str(self.output_dir)
+        good = self.write("ok.png", b"bytes")
+        digest = hashlib.sha256(b"bytes").hexdigest()
+        self.assertIsNone(review.file_problem(str(good), root, digest, 5))
+        self.assertEqual(
+            review.file_problem(str(good), root, digest, 6), review.CHANGED_CONTENT
+        )
+        self.assertEqual(
+            review.file_problem(str(good), root, "0" * 64, 5), review.CHANGED_CONTENT
+        )
+        self.assertEqual(
+            review.file_problem(os.path.join(root, "gone.png"), root, digest, 5),
+            review.GONE,
+        )
+        folder = self.output_dir / "folder.png"
+        folder.mkdir()
+        self.assertEqual(review.file_problem(str(folder), root, digest, 5), review.LINK)
+        with patch.object(review.os, "lstat", side_effect=OSError("io")):
+            self.assertEqual(
+                review.file_problem(str(good), root, digest, 5), review.UNREADABLE
+            )
+
+        def unreadable(_path):
+            raise OSError("io")
+
+        self.assertEqual(
+            review.file_problem(str(good), root, digest, 5, unreadable),
+            review.UNREADABLE,
+        )
+
+        def vanished(_path):
+            raise FileNotFoundError
+
+        self.assertEqual(
+            review.file_problem(str(good), root, digest, 5, vanished), review.GONE
+        )
+
+    def test_a_linked_folder_is_outside_its_source(self):
+        from py.api import duplicate_review as review
+
+        if not TestReview._can_symlink(self.tmpdir):  # pragma: no cover
+            self.skipTest("symlinks are not available")
+        target = self.write("x.png", b"bytes", folder=self.outside_dir)
+        os.symlink(str(self.outside_dir), str(self.output_dir / "linked"))
+        digest = hashlib.sha256(b"bytes").hexdigest()
+        path = os.path.join(str(self.output_dir), "linked", "x.png")
+        self.assertTrue(os.path.isfile(path))
+        self.assertEqual(
+            review.file_problem(path, str(self.output_dir), digest, 5),
+            review.OUTSIDE_SOURCE,
+        )
+        self.assertTrue(target.is_file())
+
+    def test_options_are_validated(self):
+        from py.api import duplicate_review as review
+
+        keep = {"group_id": "g", "root_index": 0, "path": "a.png"}
+        cases = [
+            None,
+            {"job_id": "x"},
+            {"job_id": "", "keep": [keep]},
+            {"job_id": "x" * 65, "keep": [keep]},
+            {"job_id": "x", "keep": {}},
+            {"job_id": "x", "keep": [keep] * (review.MAX_KEEP + 1)},
+            {"job_id": "x", "keep": [{**keep, "group_id": 1}]},
+            {"job_id": "x", "keep": [{**keep, "root_index": True}]},
+            {"job_id": "x", "keep": [{**keep, "root_index": -1}]},
+            {"job_id": "x", "keep": [{**keep, "path": ""}]},
+            {"job_id": "x", "keep": [["g", 0, "a.png"]]},
+        ]
+        for options in cases:
+            with self.subTest(options=options):
+                self.assertIsNotNone(review._parse_options(options)[2])
+        job_id, parsed, message = review._parse_options(
+            {"job_id": "x", "keep": [{**keep, "root_index": None}]}
+        )
+        self.assertEqual((job_id, parsed, message), ("x", [("g", None, "a.png")], None))
+
+    def test_sizes_read_naturally(self):
+        from py.api.duplicate_review import human_bytes
+
+        self.assertEqual(human_bytes(10), "10 B")
+        self.assertEqual(human_bytes(1536), "1.5 KB")
+        self.assertEqual(human_bytes(3 * 2**20), "3.0 MB")
+        self.assertEqual(human_bytes(2**40 * 5000), "5,000.0 TB")
+
+    async def test_a_kept_copy_listed_twice_counts_once(self):
+        self.trio()
+        job, group = await self.found_group()
+        keep = self.keep_of(group, "a.png")
+        plan = await self.review(job.id, [keep, keep])
+        self.assertEqual(plan["scope"]["count"], 2)
+
+    async def test_preview_failure_is_coded(self):
+        self.trio()
+        job, group = await self.found_group()
+        with (
+            patch.object(
+                self.api, "_review_duplicate_files", side_effect=OSError("boom")
+            ),
+            self.assertLogs("prompt_manager", "ERROR"),
+        ):
+            reply = await self.review(job.id, [self.keep_of(group, "a.png")], 500)
+        self.assertEqual(reply["error"], "Failed to review maintenance")
+
+    async def test_a_source_removed_before_the_apply_deletes_nothing(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.second_source())]
+        body = await self.apply(plan["token"])
+        cleanup = self.api._jobs.get(body["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.state, "failed")
+        self.assertEqual(cleanup.last_event["code"], "sources-changed")
+        self.assertEqual(self.backups(), [])
+        self.assertTrue((self.output_dir / "b.png").is_file())
+
+    async def test_a_records_move_failure_deletes_nothing(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        with (
+            patch.object(
+                self.api.db,
+                "move_copy_records",
+                side_effect=sqlite3.OperationalError("x"),
+            ),
+            self.assertLogs("prompt_manager", "ERROR"),
+        ):
+            body = await self.apply(plan["token"])
+            cleanup = self.api._jobs.get(body["job"]["id"])
+            await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        self.assertEqual(cleanup.state, "failed")
+        self.assertEqual(
+            cleanup.outcomes["errors"][-1]["detail"], duplicate_files.RECORDS_FAILED
+        )
+        self.assertEqual(cleanup.counters["removed"], 0)
+        self.assertTrue((self.output_dir / "b.png").is_file())
+        self.assertTrue((self.output_dir / "sub" / "c.png").is_file())

tests/test_api_duplicate_safety.py

@@ -0,0 +1,558 @@
+"""Duplicate-file removal safety (#249 System S6 final review: C1, I1, I2, I5, M9).
+
+- C1: one physical file reached through two listed sources, a symlinked
+  source or a hardlink is never offered or removed as a copy of itself.
+- I1: each file's records move to its verified kept copy before that file
+  is unlinked, so no database row ever names a deleted file.
+- I2: links stored under another spelling of the source move too.
+- I5: image tags move to the kept copy (their union) and the removed copy's
+  pending Needs review items are settled.
+- M9: the preview binds files by size, mtime and inode without hashing.
+"""
+
+import asyncio
+import hashlib
+import os
+import sqlite3
+import types
+from pathlib import Path
+from unittest.mock import patch
+
+try:
+    from tests.open_handles import open_handles_under
+    from tests import test_api_duplicate_files as dup_tests
+except ImportError:  # discovered with tests/ as the top-level directory
+    import test_api_duplicate_files as dup_tests
+    from open_handles import open_handles_under
+
+from py.api import duplicate_files  # noqa: E402
+from py.config import GalleryConfig  # noqa: E402
+
+WAIT = 10
+
+
+def _locator(path):
+    return os.path.normcase(os.path.realpath(str(path)))
+
+
+class SafetyTestCase(dup_tests.DuplicateFilesTestCase):
+    def need_symlinks(self):
+        if not dup_tests.TestReview._can_symlink(self.tmpdir):  # pragma: no cover
+            self.skipTest("symlinks are not available")
+
+    def linked_files(self):
+        """Every image record's path that names a file that does not exist."""
+        with self.api.db.model.get_connection() as conn:
+            rows = conn.execute("SELECT image_path FROM generated_images").fetchall()
+        return [row[0] for row in rows if not os.path.exists(row[0])]
+
+    async def apply_plan(self, plan):
+        body = await self.apply(plan["token"])
+        cleanup = self.api._jobs.get(body["job"]["id"])
+        await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
+        return cleanup
+
+
+class TestOnePhysicalFile(SafetyTestCase):
+    """C1: the kept copy survives every alias of itself."""
+
+    async def test_a_folder_listed_twice_never_loses_its_only_copy(self):
+        self.need_symlinks()
+        only = self.write("a.png", b"the only copy")
+        alias = Path(self.tmpdir) / "Output-alias"
+        os.symlink(str(self.output_dir), str(alias))
+        with patch.object(
+            self.api, "_get_all_output_dirs", return_value=[self.output_dir, alias]
+        ):
+            job = await self.find()
+            page = await self.groups(job.id)
+        self.assertEqual(page["total"], 0)
+        self.assertEqual(only.read_bytes(), b"the only copy")
+
+    async def test_a_symlinked_source_inside_another_is_one_file(self):
+        self.need_symlinks()
+        only = self.write("sub/c.png", b"one file")
+        linked = self.output_dir / "linked"  # walked by neither as a link
+        os.symlink(str(self.output_dir / "sub"), str(linked))
+        GalleryConfig.MONITORING_DIRECTORIES = [
+            self._root(self.output_dir),
+            str(linked),
+        ]
+        self.assertEqual(len(self.api._get_all_output_dirs()), 2)
+        job = await self.find()
+        self.assertEqual((await self.groups(job.id))["total"], 0)
+        self.assertTrue(only.is_file())
+
+    async def test_a_hardlink_is_never_a_copy_to_remove(self):
+        first = self.write("a.png", b"same bytes", mtime=1)
+        os.link(str(first), str(self.output_dir / "b.png"))
+        self.write("c.png", b"same bytes", mtime=3)
+        job, group = await self.found_group()
+        self.assertEqual(len(group["files"]), 2)  # a.png and b.png are one file
+        # c.png becomes a hardlink of the keeper after the find: protected.
+        (self.output_dir / "c.png").unlink()
+        os.link(str(first), str(self.output_dir / "c.png"))
+        kept = group["files"][0]["path"]
+        plan = await self.review(job.id, [self.keep_of(group, kept)])
+        self.assertEqual(plan["scope"]["count"], 0)
+        self.assertEqual(
+            plan["protected"]["reasons"], ["1 file is the same file as its kept copy"]
+        )
+
+    async def test_the_job_refuses_a_file_that_became_its_kept_copy(self):
+        first = self.write("a.png", b"same bytes", mtime=1)
+        self.write("b.png", b"same bytes", mtime=2)
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        (self.output_dir / "b.png").unlink()
+        os.link(str(first), str(self.output_dir / "b.png"))
+        cleanup = await self.apply_plan(plan)
+        self.assertEqual(cleanup.counters["removed"], 0)
+        rows = {row["path"]: row["detail"] for row in cleanup.outcomes["kept"]}
+        self.assertEqual(rows["b.png"], duplicate_files.SAME_FILE)
+        self.assertTrue((self.output_dir / "b.png").is_file())
+
+
+class TestRecordsMoveBeforeTheUnlink(SafetyTestCase):
+    """I1: a deleted file is never named by a database row."""
+
+    async def test_a_failure_between_two_files_leaves_no_row_on_a_deleted_file(self):
+        self.write("a.png", b"same", mtime=1)
+        self.write("b.png", b"same", mtime=2)
+        self.write("c.png", b"same", mtime=3)
+        prompt = self.save("Linked to every copy")
+        for name in ("b.png", "c.png"):
+            self.api.db.link_image_to_prompt(prompt, str(self.output_dir / name))
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        real = self.api.db.move_copy_records
+        calls = []
+
+        def move(*args, **kwargs):
+            calls.append(args)
+            if len(calls) == 2:
+                raise sqlite3.OperationalError("database is locked")
+            return real(*args, **kwargs)
+
+        with patch.object(self.api.db, "move_copy_records", side_effect=move):
+            with self.assertLogs("prompt_manager", "ERROR"):
+                cleanup = await self.apply_plan(plan)
+        self.assertEqual(self.linked_files(), [])
+        self.assertEqual(cleanup.counters["removed"], 1)
+        self.assertEqual(cleanup.state, "failed")
+        [error] = cleanup.outcomes["errors"]
+        self.assertEqual(error["detail"], duplicate_files.RECORDS_FAILED)
+        self.assertNotIn("Missing-file", cleanup.status_line)
+        self.assertEqual(
+            cleanup.status_line.split(" · ")[:2],
+            ["Stopped after an error", "Removed 1 duplicate file, 1 not removed"],
+        )
+
+    async def test_a_failed_unlink_leaves_its_links_on_the_kept_copy(self):
+        self.trio()
+        prompt = self.save("On c")
+        self.api.db.link_image_to_prompt(prompt, str(self.output_dir / "sub" / "c.png"))
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        real = duplicate_files._unlink
+
+        def unlink(path):
+            if os.path.basename(path) == "c.png":
+                raise PermissionError(13, "denied")
+            real(path)
+
+        with patch.object(duplicate_files, "_unlink", side_effect=unlink):
+            with self.assertLogs("prompt_manager", "WARNING"):
+                cleanup = await self.apply_plan(plan)
+        [image] = self.api.db.get_prompt_images(prompt)
+        self.assertEqual(os.path.basename(image["image_path"]), "a.png")
+        self.assertTrue((self.output_dir / "sub" / "c.png").is_file())
+        self.assertEqual(self.linked_files(), [])
+        self.assertEqual(
+            cleanup.outcomes["errors"][0]["detail"], duplicate_files.LOCKED_FILE
+        )
+
+
+class TestEverySpelling(SafetyTestCase):
+    """I2: links under the monitor's spelling of a source move too."""
+
+    async def test_links_under_a_symlinked_source_spelling_move(self):
+        self.need_symlinks()
+        self.trio()
+        spelled = Path(self.comfy_dir) / "out-link"
+        os.symlink(str(self.output_dir), str(spelled))
+        GalleryConfig.MONITORING_DIRECTORIES = [str(spelled)]
+        prompt = self.save("Linked through the symlink")
+        self.api.db.link_image_to_prompt(prompt, str(spelled / "b.png"))
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.assertIn("1 database link moves", plan["details"][0]["detail"])
+        await self.apply_plan(plan)
+        [image] = self.api.db.get_prompt_images(prompt)
+        self.assertEqual(
+            _locator(image["image_path"]), _locator(self.output_dir / "a.png")
+        )
+        self.assertEqual(self.linked_files(), [])
+
+
+class TestTagsAndReview(SafetyTestCase):
+    """I5: tags join the kept copy; pending review items of the copy settle."""
+
+    def tag(self, path, tags):
+        data = Path(path).read_bytes()
+        info = os.stat(path)
+        return self.api.db.replace_image_tags(
+            locator=_locator(path),
+            sha256=hashlib.sha256(data).hexdigest(),
+            size=info.st_size,
+            mtime_ns=info.st_mtime_ns,
+            expected_epoch=self.api.db.get_execution_epoch(),
+            expected_tag_revision=0,
+            tags=tags,
+        )
+
+    def tags_of(self, path):
+        data = Path(path).read_bytes()
+        return self.api.db.read_image_tags(
+            _locator(path), hashlib.sha256(data).hexdigest()
+        )["tags"]
+
+    def queue(self, path):
+        data = Path(path).read_bytes()
+        info = os.stat(path)
+        return self.api.db.record_review_item(
+            locator=_locator(path),
+            spelled_path=os.path.realpath(str(path)),
+            sha256=hashlib.sha256(data).hexdigest(),
+            size=info.st_size,
+            mtime_ns=info.st_mtime_ns,
+            epoch=self.api.db.get_execution_epoch(),
+            resolver_version=1,
+            evidence={"candidates": []},
+            candidate_count=0,
+            limited=True,
+        )
+
+    def pending(self):
+        return self.api.db.review_queue_page(limit=50)["items"]
+
+    async def test_tags_move_as_a_union_and_pending_items_settle(self):
+        self.write("a.png", b"same", mtime=1)
+        self.write("b.png", b"same", mtime=2)
+        keeper, removed = self.output_dir / "a.png", self.output_dir / "b.png"
+        self.tag(keeper, ["kept"])
+        self.tag(removed, ["moved", "kept"])
+        self.assertTrue(self.queue(removed))
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.assertIn("image tags join the kept copy", plan["details"][0]["detail"])
+        self.assertIn("1 Needs review item is settled", plan["details"][0]["detail"])
+        await self.apply_plan(plan)
+        self.assertEqual(self.tags_of(keeper), ["kept", "moved"])
+        self.assertEqual(self.pending(), [])
+
+
+class TestPreviewDoesNotHash(SafetyTestCase):
+    """M9: the preview binds by size, mtime and inode; the job hashes."""
+
+    async def test_preview_reads_no_file_and_spots_a_touched_one(self):
+        self.trio()
+        job, group = await self.found_group()
+        calls = []
+        real = self.api._calculate_file_hash
+
+        def spy(path):
+            calls.append(path)
+            return real(path)
+
+        self.api._calculate_file_hash = spy
+        os.utime(self.output_dir / "b.png", (5_000_000, 5_000_000))
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.assertEqual(calls, [])
+        self.assertEqual(plan["scope"]["count"], 1)
+        self.assertEqual(
+            plan["protected"]["reasons"], ["1 file changed since the search"]
+        )
+        cleanup = await self.apply_plan(plan)
+        self.assertEqual(cleanup.counters["removed"], 1)
+        self.assertGreaterEqual(len(calls), 2)  # the job hashes keeper and file
+
+
+def _fake_stat(real, **changes):
+    """A stat result with some fields changed (st_dev, st_ino, st_mtime_ns...)."""
+    fields = {name: getattr(real, name) for name in dir(real) if name.startswith("st_")}
+    fields.update(changes)
+    return types.SimpleNamespace(**fields)
+
+
+class TestAnotherMount(SafetyTestCase):
+    """Round 2, R1: one file reached through two mounts is never removed.
+
+    The second source stands for the same folder seen through another
+    filesystem: each test reports its file on another ``st_dev``.
+    """
+
+    MAYBE = "1 file may be the same file reached through another mount or share"
+
+    def setUp(self):
+        super().setUp()
+        self.real_lstat, self.real_stat = os.lstat, os.stat
+
+    def mount(self, rel, keeper_rel, *, ino="same", mtime_shift=0):
+        """Copies at ``output/keeper_rel`` and ``renders/rel``; the patch."""
+        second = self.second_source()
+        GalleryConfig.MONITORING_DIRECTORIES = [
+            self._root(self.output_dir),
+            self._root(second),
+        ]
+        stamp = 1_700_000_000_123_456_789
+        keeper = self.write(keeper_rel, b"one file")
+        other = self.write(rel, b"one file", folder=second)
+        os.utime(keeper, ns=(stamp, stamp))
+        os.utime(other, ns=(stamp, stamp))
+        target = os.path.normcase(str(other))
+        keeper_ino = self.real_stat(keeper).st_ino
+
+        def changed(real):
+            return _fake_stat(
+                real,
+                st_dev=real.st_dev + 7,
+                st_ino=keeper_ino if ino == "same" else real.st_ino + 1000,
+                st_mtime_ns=real.st_mtime_ns + mtime_shift,
+            )
+
+        def lstat(path, *args, **kwargs):
+            real = self.real_lstat(path, *args, **kwargs)
+            return changed(real) if os.path.normcase(str(path)) == target else real
+
+        def stat(path, *args, **kwargs):
+            real = self.real_stat(path, *args, **kwargs)
+            return changed(real) if os.path.normcase(str(path)) == target else real
+
+        patches = (
+            patch("os.lstat", side_effect=lstat),
+            patch("os.stat", side_effect=stat),
+        )
+        return keeper, other, patches
+
+    async def review_with(self, patches, keeper_rel):
+        # The mount reports its device and file id to the find too.
+        with patches[0], patches[1]:
+            job, group = await self.found_group()
+            plan = await self.review(job.id, [self.keep_of(group, keeper_rel, 0)])
+        return job, group, plan
+
+    async def test_nfs_and_smb_spellings_of_one_share_are_protected(self):
+        # NFS and SMB report their own device and file id; SMB rounds mtime.
+        keeper, other, patches = self.mount(
+            "x.png", "x.png", ino="other", mtime_shift=-89
+        )
+        _job, _group, plan = await self.review_with(patches, "x.png")
+        self.assertEqual(plan["scope"]["count"], 0)
+        self.assertEqual(plan["protected"]["reasons"], [self.MAYBE])
+        self.assertTrue(other.is_file() and keeper.is_file())
+
+    async def test_a_loopback_mount_with_the_same_inode_is_protected(self):
+        keeper, other, patches = self.mount("y.png", "sub/y.png", ino="same")
+        _job, _group, plan = await self.review_with(patches, "sub/y.png")
+        self.assertEqual(plan["protected"]["reasons"], [self.MAYBE])
+
+    async def test_a_fuse_pool_path_is_protected(self):
+        # mergerfs, rclone or sshfs synthesize both the device and the inode.
+        keeper, other, patches = self.mount("x.png", "x.png", ino="other")
+        _job, _group, plan = await self.review_with(patches, "x.png")
+        self.assertEqual(plan["protected"]["reasons"], [self.MAYBE])
+
+    async def test_a_true_copy_on_another_disk_is_still_offered(self):
+        keeper, other, patches = self.mount("elsewhere.png", "x.png", ino="other")
+        _job, _group, plan = await self.review_with(patches, "x.png")
+        self.assertEqual(plan["scope"]["count"], 1)
+
+    async def test_the_job_checks_again_before_the_unlink(self):
+        keeper, other, patches = self.mount("x.png", "x.png", ino="other")
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "x.png", 0)])
+        self.assertEqual(plan["scope"]["count"], 1)  # looked separate then
+        with patches[0], patches[1]:
+            cleanup = await self.apply_plan(plan)
+        self.assertEqual(cleanup.counters["removed"], 0)
+        rows = {row["path"]: row["detail"] for row in cleanup.outcomes["kept"]}
+        self.assertEqual(rows["x.png"], duplicate_files.MAYBE_SAME_FILE)
+        self.assertTrue(other.is_file() and keeper.is_file())
+
+    async def test_a_volume_without_file_ids_is_named_truthfully(self):
+        keeper, other, patches = self.mount("x.png", "x.png")
+
+        def zero(path, *args, **kwargs):
+            return _fake_stat(self.real_stat(path), st_dev=0, st_ino=0)
+
+        with patch("os.lstat", side_effect=zero), patch("os.stat", side_effect=zero):
+            job, group = await self.found_group()
+            plan = await self.review(job.id, [self.keep_of(group, "x.png", 0)])
+        self.assertEqual(
+            plan["protected"]["reasons"],
+            ["1 file: can't confirm this is a separate file on this drive"],
+        )
+
+    async def test_the_kept_copy_is_held_open_during_the_unlink(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        real = duplicate_files._unlink
+        held = []
+
+        def unlink(path):
+            held.append(bool(open_handles_under(self.output_dir / "a.png")))
+            real(path)
+
+        if not os.path.isdir("/proc/self/fd"):  # pragma: no cover - no /proc
+            self.skipTest("open handles are read from /proc")
+        with patch.object(duplicate_files, "_unlink", side_effect=unlink):
+            await self.apply_plan(plan)
+        self.assertEqual(held, [True, True])
+
+
+class TestNestedSpellings(SafetyTestCase):
+    """Round 2, R2: spellings and favorites by containment, not equality."""
+
+    async def test_a_parent_spelling_of_a_nested_source(self):
+        # The reviewer's probe: sources [output/sub, out-link -> output].
+        self.need_symlinks()
+        self.write("sub/a.png", b"same", mtime=1)
+        self.write("sub/b.png", b"same", mtime=2)
+        out_link = Path(self.comfy_dir) / "out-link"
+        os.symlink(str(self.output_dir), str(out_link))
+        GalleryConfig.MONITORING_DIRECTORIES = [
+            self._root(self.output_dir / "sub"),
+            str(out_link),
+        ]
+        prompt = self.save("Linked before sub was a source")
+        self.api.db.link_image_to_prompt(prompt, str(out_link / "sub" / "b.png"))
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        self.assertIn("1 database link moves", plan["details"][0]["detail"])
+        await self.apply_plan(plan)
+        [image] = self.api.db.get_prompt_images(prompt)
+        self.assertEqual(
+            _locator(image["image_path"]), _locator(self.output_dir / "sub" / "a.png")
+        )
+        self.assertEqual(self.linked_files(), [])
+
+    async def test_nested_sources_subfolder_first_with_a_parent_favorite(self):
+        self.write("sub/a.png", b"same", mtime=1)
+        self.write("sub/b.png", b"same", mtime=2)
+        GalleryConfig.MONITORING_DIRECTORIES = [
+            self._root(self.output_dir / "sub"),
+            self._root(self.output_dir),
+        ]
+        parent_key = duplicate_files.favorite_root_key(self.output_dir)
+        self.api.db.set_image_favorite(parent_key, "sub/b.png")
+        prompt = self.save("Linked under the parent")
+        self.api.db.link_image_to_prompt(prompt, str(self.output_dir / "sub" / "b.png"))
+        job, group = await self.found_group()
+        self.assertEqual({f["root_index"] for f in group["files"]}, {0})
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        await self.apply_plan(plan)
+        self.assertEqual(self.api.db.get_root_favorites(parent_key), {"sub/a.png"})
+        self.assertEqual(self.linked_files(), [])
+
+
+class TestLockedRowSaysWhereLinksAre(SafetyTestCase):
+    """Round 2, m1."""
+
+    async def test_the_locked_row_names_the_kept_copy(self):
+        self.trio()
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        with (
+            patch.object(
+                duplicate_files, "_unlink", side_effect=PermissionError(13, "denied")
+            ),
+            self.assertLogs("prompt_manager", "WARNING"),
+        ):
+            cleanup = await self.apply_plan(plan)
+        self.assertIn(
+            "its links now name the kept copy", cleanup.outcomes["errors"][0]["detail"]
+        )
+
+
+class TestRound3(SafetyTestCase):
+    """Re-review 2: n1 (sources at different depths) and n2 (the kept copy
+    must still be the held file right before the unlink)."""
+
+    MAYBE = TestAnotherMount.MAYBE
+
+    def setUp(self):
+        super().setUp()
+        self.real_lstat, self.real_stat = os.lstat, os.stat
+
+    mount = TestAnotherMount.mount
+    review_with = TestAnotherMount.review_with
+
+    async def test_a_deeper_source_on_another_mount_is_protected(self):
+        # The reviewer's probe: ComfyUI/output -> /mnt/pool/output and
+        # /mnt/disk1/output/2024 as an extra root, FUSE-style inodes.
+        _keeper, other, patches = self.mount("x.png", "2024/x.png", ino="other")
+        _job, _group, plan = await self.review_with(patches, "2024/x.png")
+        self.assertEqual(plan["protected"]["reasons"], [self.MAYBE])
+        self.assertTrue(other.is_file())
+
+    async def test_a_shallower_source_on_another_mount_is_protected(self):
+        _keeper, _other, patches = self.mount("2024/x.png", "x.png", ino="other")
+        _job, _group, plan = await self.review_with(patches, "x.png")
+        self.assertEqual(plan["protected"]["reasons"], [self.MAYBE])
+
+    async def test_only_whole_folder_names_match(self):
+        _keeper, _other, patches = self.mount("photo-x.png", "x.png", ino="other")
+        _job, _group, plan = await self.review_with(patches, "x.png")
+        self.assertEqual(plan["scope"]["count"], 1)
+
+    async def removal_while_the_keeper_goes(self, change):
+        self.write("a.png", b"same", mtime=1)
+        self.write("b.png", b"same", mtime=2)
+        prompt = self.save("On b")
+        self.api.db.link_image_to_prompt(prompt, str(self.output_dir / "b.png"))
+        job, group = await self.found_group()
+        plan = await self.review(job.id, [self.keep_of(group, "a.png")])
+        real = self.api.db.move_copy_records
+        refused = []
+
+        def move(*args, **kwargs):
+            counts = real(*args, **kwargs)
+            if args[0]["kept_path"].endswith("a.png"):
+                try:
+                    change(self.output_dir / "a.png")
+                except PermissionError:
+                    # Windows: the job holds the kept copy open (no
+                    # FILE_SHARE_DELETE), so it cannot be deleted or replaced.
+                    if os.name != "nt":
+                        raise
+                    refused.append(True)
+            return counts
+
+        with patch.object(self.api.db, "move_copy_records", side_effect=move):
+            cleanup = await self.apply_plan(plan)
+        if refused:  # pragma: no cover - native Windows
+            self.assertEqual((self.output_dir / "a.png").read_bytes(), b"same")
+            self.assertEqual(cleanup.counters["removed"], 1)
+            self.assertEqual(self.linked_files(), [])
+            return
+        self.assertTrue((self.output_dir / "b.png").is_file())
+        self.assertEqual(cleanup.counters["removed"], 0)
+        rows = {row["path"]: row["detail"] for row in cleanup.outcomes["kept"]}
+        self.assertEqual(rows["b.png"], duplicate_files.KEEPER_WENT)
+        self.assertEqual(self.linked_files(), [])
+        [image] = self.api.db.get_prompt_images(prompt)
+        self.assertEqual(os.path.basename(image["image_path"]), "b.png")
+        self.assertIn("1 not removed", cleanup.status_line)
+
+    async def test_a_kept_copy_deleted_during_the_move_keeps_the_file(self):
+        await self.removal_while_the_keeper_goes(lambda path: path.unlink())
+
+    async def test_a_kept_copy_replaced_during_the_move_keeps_the_file(self):
+        def replace(path):
+            spare = path.with_name("spare.tmp")
+            spare.write_bytes(b"other bytes")
+            os.replace(spare, path)
+
+        await self.removal_while_the_keeper_goes(replace)

tests/test_api_history_outputs.py

@@ -401,7 +401,7 @@ class HistoryOutputAPITests(ImageAPITestCase):
         self.assertEqual(response.status, 400)
         await response.read()
 
-    async def test_media_ranges_head_and_existing_thumbnail_without_generation(self):
+    async def test_media_ranges_head_and_a_preview_made_on_first_view(self):
         path = self.output_dir / "clip.mp4"
         payload = bytes(range(256)) * 600
         path.write_bytes(payload)
@@ -430,9 +430,14 @@ class HistoryOutputAPITests(ImageAPITestCase):
         self.assertEqual(response.status, 200)
         self.assertEqual(int(response.headers["Content-Length"]), len(payload))
         self.assertEqual(await response.read(), b"")
+        # A missing preview is made on first view (#249): an undecodable video
+        # gets the placeholder frame, never the whole file.
+        preview = self.output_dir / "thumbnails" / "clip_thumb.jpg"
         response = await self.current(descriptor["thumbnail_url"])
-        self.assertEqual(await response.read(), payload)
-        preview = make_png(self.output_dir / "thumbnails" / "clip_thumb.jpg")
+        self.assertEqual(response.headers["Content-Type"], "image/jpeg")
+        self.assertEqual(await response.read(), preview.read_bytes())
+        self.assertLess(preview.stat().st_size, len(payload))
+        make_png(preview)
         response = await self.current(descriptor["thumbnail_url"])
         self.assertEqual(await response.read(), preview.read_bytes())
 
@@ -1111,6 +1116,73 @@ class HistoryOutputAPITests(ImageAPITestCase):
         response = await self.current(descriptor["thumbnail_url"])
         self.assert_error(response, await response.json(), 403, "not-authorized")
 
+    async def test_missing_preview_is_never_made_through_a_linked_out_folder(self):
+        original = make_png(self.output_dir / "image.png", size=(64, 64))
+        elsewhere = self.temp_dir / "elsewhere"
+        elsewhere.mkdir()
+        (self.output_dir / "thumbnails").symlink_to(elsewhere, target_is_directory=True)
+        start = self.capture([self.ref()])
+        descriptor = await self.media(start)
+        response = await self.current(descriptor["thumbnail_url"])
+        self.assertEqual(response.status, 200)
+        self.assertEqual(await response.read(), original.read_bytes())
+        self.assertEqual(os.listdir(elsewhere), [])
+
+    async def test_missing_preview_is_never_made_through_a_linked_sub_folder(self):
+        # M6: every link segment of the preview's folder is checked.
+        original = make_png(self.output_dir / "sub" / "image.png", size=(64, 64))
+        elsewhere = self.temp_dir / "elsewhere"
+        elsewhere.mkdir()
+        (self.output_dir / "thumbnails").mkdir()
+        (self.output_dir / "thumbnails" / "sub").symlink_to(
+            elsewhere, target_is_directory=True
+        )
+        start = self.capture([self.ref(subfolder="sub")])
+        descriptor = await self.media(start)
+        response = await self.current(descriptor["thumbnail_url"])
+        self.assertEqual(response.status, 200)
+        self.assertEqual(await response.read(), original.read_bytes())
+        self.assertEqual(os.listdir(elsewhere), [])
+
+    async def test_a_video_preview_that_cannot_be_made_is_the_placeholder(self):
+        # I1: never the video bytes inside an <img>.
+        from py.api.jobs import RESTORE_JOB_KIND
+
+        (self.output_dir / "clip.webm").write_bytes(b"AV1 stream no decoder here")
+        start = self.capture([self.ref(filename="clip.webm")])
+        descriptor = await self.media(start)
+        gate = asyncio.Event()
+
+        async def runner(job):
+            await gate.wait()
+
+        job = self.api._jobs.start(
+            RESTORE_JOB_KIND,
+            "restore",
+            admission="records",
+            cancellable=False,
+            runner=runner,
+        )
+        try:
+            response = await self.current(descriptor["thumbnail_url"])
+            self.assertEqual(response.status, 200)
+            self.assertEqual(response.headers["Content-Type"], "image/jpeg")
+            self.assertTrue((await response.read()).startswith(b"\xff\xd8"))
+        finally:
+            gate.set()
+            await asyncio.wait_for(job.task, 5)
+        self.assertFalse((self.output_dir / "thumbnails").exists())
+
+    async def test_a_preview_that_cannot_be_made_serves_the_original(self):
+        broken = self.output_dir / "image.png"
+        broken.write_bytes(b"\x89PNG\r\n\x1a\n not an image")
+        start = self.capture([self.ref()])
+        descriptor = await self.media(start)
+        response = await self.current(descriptor["thumbnail_url"])
+        self.assertEqual(response.status, 200)
+        self.assertEqual(await response.read(), broken.read_bytes())
+        self.assertFalse((self.output_dir / "thumbnails" / "image_thumb.png").exists())
+
     async def test_disappearance_before_open_is_missing_and_closes_handle(
         self,
     ):

tests/test_api_images.py

@@ -80,6 +80,8 @@ class ImageAPITestCase(AioHTTPTestCase):
 
     async def tearDownAsync(self):
         await super().tearDownAsync()  # closes the aiohttp test client
+        # The automatic preview worker writes into the temp tree: stop it first.
+        self.api.close_auto_thumbnails()
         self._anchor_patch.stop()
         # Windows refuses to unlink a database that still has open handles.
         self.api.db.close_all()
@@ -413,11 +415,11 @@ class TestThumbnailProgressStream(ThumbnailTestCase):
         lock = threading.Lock()
         original = images_module._write_image_thumbnail
 
-        def recording(src, dst, size):
+        def recording(src, dst, size, *args):
             with lock:
                 seen.add(threading.current_thread().name)
             time.sleep(0.05)
-            return original(src, dst, size)
+            return original(src, dst, size, *args)
 
         images_module._write_image_thumbnail = recording
         self.addCleanup(setattr, images_module, "_write_image_thumbnail", original)
@@ -1565,39 +1567,65 @@ class TestVideoThumbnailFallbacks(ImageRouteCoverageCase):
         with Image.open(self.thumb) as img:
             self.assertEqual(img.format, "JPEG")
 
-    def test_cv2_cannot_open_or_read(self):
-        with patch.dict(sys.modules, {"cv2": self._fake_cv2(opened=False)}):
-            closed = self.api._generate_video_thumbnail(
-                self.video, self.thumb, self.SIZE
-            )
-        with patch.dict(sys.modules, {"cv2": self._fake_cv2(read_ok=False)}):
-            unread = self.api._generate_video_thumbnail(
-                self.video, self.thumb, self.SIZE
-            )
-        self.assertEqual((closed, unread), (False, False))
+    def test_cv2_cannot_open_or_read_falls_through_to_ffmpeg_then_placeholder(self):
+        # #249 I1: an OpenCV that cannot decode the clip (AV1) no longer ends
+        # the run; ffmpeg and then the placeholder are tried.
+        for fake in (self._fake_cv2(opened=False), self._fake_cv2(read_ok=False)):
+            self.thumb.unlink(missing_ok=True)
+            with (
+                patch.dict(sys.modules, {"cv2": fake}),
+                patch("subprocess.Popen", side_effect=FileNotFoundError("ffmpeg")),
+            ):
+                ok = self.api._generate_video_thumbnail(
+                    self.video, self.thumb, self.SIZE
+                )
+            self.assertTrue(ok)
+            with Image.open(self.thumb) as img:
+                self.assertEqual((img.format, img.size), ("JPEG", self.SIZE))
 
     def test_cv2_crash_is_contained(self):
         cv2 = MagicMock()
         cv2.VideoCapture.side_effect = RuntimeError("driver crash")
-        with patch.dict(sys.modules, {"cv2": cv2}):
+        with (
+            patch.dict(sys.modules, {"cv2": cv2}),
+            patch("subprocess.Popen", side_effect=FileNotFoundError("ffmpeg")),
+        ):
             ok = self.api._generate_video_thumbnail(self.video, self.thumb, self.SIZE)
+        self.assertTrue(ok, "the placeholder is written")
+
+    def test_cv2_failure_keeps_an_existing_preview_on_rebuild(self):
+        make_png(self.thumb)
+        before = self.thumb.read_bytes()
+        with (
+            patch.dict(sys.modules, {"cv2": self._fake_cv2(opened=False)}),
+            patch("subprocess.Popen", side_effect=FileNotFoundError("ffmpeg")),
+        ):
+            ok = self.api._generate_video_thumbnail(
+                self.video, self.thumb, self.SIZE, keep_existing=True
+            )
         self.assertFalse(ok)
+        self.assertEqual(self.thumb.read_bytes(), before)
 
     def test_ffmpeg_absent_yields_placeholder(self):
         with (
             patch.dict(sys.modules, {"cv2": None}),
-            patch("subprocess.run", side_effect=FileNotFoundError("ffmpeg")),
+            patch("subprocess.Popen", side_effect=FileNotFoundError("ffmpeg")),
         ):
             ok = self.api._generate_video_thumbnail(self.video, self.thumb, self.SIZE)
         self.assertTrue(ok)
         with Image.open(self.thumb) as img:
             self.assertEqual(img.size, self.SIZE)
 
+    @staticmethod
+    def _process(returncode, stderr=""):
+        process = MagicMock(returncode=returncode)
+        process.communicate.return_value = ("", stderr)
+        return process
+
     def test_ffmpeg_failure_yields_placeholder(self):
-        failed = MagicMock(returncode=1, stderr="no stream")
         with (
             patch.dict(sys.modules, {"cv2": None}),
-            patch("subprocess.run", return_value=failed),
+            patch("subprocess.Popen", return_value=self._process(1, "no stream")),
         ):
             ok = self.api._generate_video_thumbnail(self.video, self.thumb, self.SIZE)
         self.assertTrue(ok)
@@ -1606,16 +1634,41 @@ class TestVideoThumbnailFallbacks(ImageRouteCoverageCase):
     def test_ffmpeg_success(self):
         with (
             patch.dict(sys.modules, {"cv2": None}),
-            patch("subprocess.run", return_value=MagicMock(returncode=0)) as run,
+            patch("subprocess.Popen", return_value=self._process(0)) as run,
         ):
             ok = self.api._generate_video_thumbnail(self.video, self.thumb, self.SIZE)
         self.assertTrue(ok)
         self.assertEqual(run.call_args.args[0][0], "ffmpeg")
 
+    def test_ffmpeg_is_tracked_and_a_hold_before_the_rename_writes_nothing(self):
+        tracked = []
+
+        def writing(cmd, **kwargs):
+            Path(cmd[-1]).write_bytes(b"\xff\xd8 frame")
+            return self._process(0)
+
+        with (
+            patch.dict(sys.modules, {"cv2": None}),
+            patch("subprocess.Popen", side_effect=writing),
+        ):
+            with self.assertRaises(images_module.PreviewWriteHeld):
+                self.api._generate_video_thumbnail(
+                    self.video,
+                    self.thumb,
+                    self.SIZE,
+                    guard=lambda: False,
+                    track=tracked.append,
+                )
+        self.assertEqual(len(tracked), 1)
+        self.assertFalse(self.thumb.exists())
+        self.assertEqual(
+            [p.name for p in self.thumb.parent.iterdir() if "part" in p.name], []
+        )
+
     def test_placeholder_failure_returns_false(self):
         with (
             patch.dict(sys.modules, {"cv2": None}),
-            patch("subprocess.run", side_effect=FileNotFoundError),
+            patch("subprocess.Popen", side_effect=FileNotFoundError),
             patch.object(
                 images_module.Image, "new", side_effect=RuntimeError("no PIL")
             ),
@@ -1939,7 +1992,11 @@ class TestGalleryRescan(ImageAPITestCase):
 
 
 class TestOutputScanHelpers(ImageAPITestCase):
-    """Stable ids for filesystem entries, no symlink reads, stop on disconnect."""
+    """Stable ids for filesystem entries and no symlink reads.
+
+    The thumbnail stream no longer stops when its client goes away: see
+    ``tests/test_thumbnail_jobs.py`` (the work is a job that outlives it).
+    """
 
     def test_output_entry_id_is_a_stable_digest_of_the_path(self):
         import hashlib
@@ -1964,28 +2021,6 @@ class TestOutputScanHelpers(ImageAPITestCase):
             names = sorted(p.name for p in images_module._iter_media_files(output, 100))
         self.assertEqual(names, ["real.png"])
 
-    async def test_thumbnail_stream_stops_when_the_client_disconnects(self):
-        generated = []
-
-        def fake_generate(src, dst, size, is_video):
-            generated.append(src)
-            return {
-                "action": "generated",
-                "file": src.name,
-                "dir": src.parent.name,
-                "type": "image",
-            }
-
-        self.api._generate_one = fake_generate
-
-        class GoneResponse:
-            async def write(self, data):
-                raise ConnectionResetError("client went away")
-
-        targets = [(Path(f"/x/{i}.png"), Path(f"/x/t{i}.jpg"), False) for i in range(3)]
-        await self.api._stream_thumbnails(GoneResponse(), targets, (256, 256))
-        self.assertEqual(generated, [])
-
 
 class TestThumbnailStartupStatus(ThumbnailTestCase):
     """Startup counts and generation cover the same configured collection."""

tests/test_api_review.py

@@ -397,15 +397,20 @@ class TestQueueReads(ReviewRoutesTestCase):
         self._no_paths()
 
     async def test_queue_cards_use_the_images_listing_thumbnail(self):
-        # M10: a known thumbnail is the card's media, as on the Images page;
-        # without one the card falls back to the original.
+        # M10: the card's media is the Images listing's thumbnail. Without a
+        # preview yet it is still the thumbnail URL (made on first view, #249).
         self._require_routes()
         from py.api.images import _output_image_entry
 
         path = self._ambiguous("sub/f0.png")
         self._record(path)
         item = (await self._list())["items"][0]
-        self.assertIsNone(item["thumbnail_url"])
+        scope = item["target"]["root_scope"]
+        self.assertEqual(
+            item["thumbnail_url"],
+            "/prompt_manager/images/serve/thumbnails/sub/f0_thumb.png"
+            f"?root=0&root_scope={scope}",
+        )
         thumb = self.output_dir / "thumbnails" / "sub" / "f0_thumb.png"
         thumb.parent.mkdir(parents=True)
         write_png(thumb, prompt=ambiguous_graph("cat", "dog"), color=(1, 2, 3))

tests/test_api_review_events.py

@@ -12,6 +12,7 @@ import asyncio
 import json
 import sqlite3
 import threading
+from pathlib import Path
 
 try:
     from tests.test_api_review import ReviewRoutesTestCase
@@ -86,11 +87,16 @@ class ReviewEventsTestCase(ReviewRoutesTestCase):
         return path
 
     async def _record_in_thread(self, path):
-        """Record as the monitor does: from a worker thread, not the loop."""
+        """Record as the scan does: from a worker thread, not the loop."""
+        from py.api.admin import _scan_roots
+
         self._roots()
+        roots = _scan_roots(self.api._get_all_output_dirs())
         outcomes = []
         thread = threading.Thread(
-            target=lambda: outcomes.append(self.api.record_review_file(str(path)))
+            target=lambda: outcomes.append(
+                self.api._record_review_sync(Path(str(path)), roots, inspection=True)
+            )
         )
         thread.start()
         await asyncio.get_running_loop().run_in_executor(None, thread.join)
@@ -98,7 +104,7 @@ class ReviewEventsTestCase(ReviewRoutesTestCase):
 
 
 class TestReviewEvents(ReviewEventsTestCase):
-    async def test_monitor_recording_sends_one_change(self):
+    async def test_scan_recording_sends_one_change(self):
         self._require_events()
         stream = await self._stream()
         self.assertEqual(await self._record_in_thread(self._queueable()), "recorded")

tests/test_api_workflow_storage.py

@@ -0,0 +1,424 @@
+"""Store each workflow once, as a maintenance job (py/api/workflow_storage.py).
+
+``POST /prompt_manager/maintenance/preview {operation:"workflow_storage"}``
+reviews the rewrite; applying it starts a ``records`` job that takes the
+verified safety backup first, rewrites old-form rows batch by batch (Cancel
+stops after the current batch) and then starts "Reclaim space", an
+``exclusive`` job with its own disk check. No image file is ever opened.
+"""
+
+import asyncio
+import builtins
+import contextlib
+import hashlib
+import json
+import os
+import sqlite3
+from unittest.mock import patch
+
+from PIL import Image, PngImagePlugin
+
+from database import workflow_storage as ws
+from database.workflow_documents import DocumentReader, resolve_image_row, store_text
+from py.api import maintenance
+from py.api.jobs import MAINTENANCE_JOB_KIND
+
+try:
+    from tests.test_api_maintenance import ApplyTestCase
+    from tests.test_workflow_documents import sample_prompt_graph, sample_workflow
+except ImportError:  # discovered with tests/ as the top-level directory
+    from test_api_maintenance import ApplyTestCase
+    from test_workflow_documents import sample_prompt_graph, sample_workflow
+
+WAIT = 20
+OPERATION = "workflow_storage"
+
+
+def _digest(path):
+    with open(path, "rb") as handle:
+        return hashlib.sha256(handle.read()).hexdigest()
+
+
+class WorkflowStorageTestCase(ApplyTestCase):
+    def old_library(self, count=12, media=False):
+        """Old-form rows (text inline), four images per run."""
+        prompt_id = self.save("a cat")
+        conn = self.api.db.model.get_connection()
+        for index in range(count):
+            run = index // 4
+            path = self.output_dir / f"{index}.png"
+            workflow = json.dumps(sample_workflow(run, nodes=120))
+            if media:
+                info = PngImagePlugin.PngInfo()
+                info.add_text("workflow", workflow)
+                with Image.new("RGB", (4, 4)) as image:
+                    image.save(path, pnginfo=info)
+            with conn:
+                conn.execute(
+                    "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                    "file_path, workflow_data, prompt_metadata, parameters) "
+                    "VALUES (?, ?, ?, ?, ?, ?, '{}')",
+                    (
+                        prompt_id,
+                        str(path),
+                        path.name,
+                        str(path),
+                        workflow,
+                        json.dumps(sample_prompt_graph(run)),
+                    ),
+                )
+        return prompt_id
+
+    def texts(self):
+        conn = self.api.db.model.get_connection()
+        reader = DocumentReader(conn)
+        return {
+            row["id"]: resolve_image_row(reader, row)
+            for row in conn.execute(
+                "SELECT id, workflow_data, prompt_metadata, workflow_document_id, "
+                "prompt_document_id FROM generated_images ORDER BY id"
+            )
+        }
+
+    def inline_count(self):
+        conn = self.api.db.model.get_connection()
+        return conn.execute(
+            "SELECT COUNT(*) FROM generated_images WHERE workflow_data IS NOT NULL"
+        ).fetchone()[0]
+
+    def reclaim_job(self):
+        jobs = [job for job in self.api._jobs.list() if job.title == "Reclaim space"]
+        return jobs[0] if jobs else None
+
+    async def finish(self, job):
+        await asyncio.wait_for(asyncio.shield(job.task), WAIT)
+        reclaim = self.reclaim_job()
+        if reclaim is not None and reclaim.task is not None:
+            await asyncio.wait_for(asyncio.shield(reclaim.task), WAIT)
+        return reclaim
+
+    async def start(self):
+        plan = await self.preview(OPERATION)
+        body = await self.apply(plan["token"])
+        return plan, body["job"], self.api._jobs.get(body["job"]["id"])
+
+
+class TestWorkflowStorageJob(WorkflowStorageTestCase):
+    async def test_preview_reviews_the_rewrite_and_writes_nothing(self):
+        self.old_library()
+        db_path = self.api.db.model.db_path
+        before = _digest(db_path), _digest(db_path + "-wal")
+        plan = await self.preview(OPERATION)
+        self.assertEqual((_digest(db_path), _digest(db_path + "-wal")), before)
+        self.assertEqual(plan["operation"], OPERATION)
+        self.assertEqual(plan["scope"]["count"], 12)
+        self.assertEqual(plan["scope"]["label"], "12 image records to rewrite")
+        self.assertFalse(plan["media"]["deletes_files"])
+        self.assertEqual(
+            plan["media"]["label"],
+            "No image or video file is opened or changed; only the database.",
+        )
+        self.assertEqual(plan["backup"]["policy"], "required")
+        self.assertEqual(
+            plan["backup"]["destination_label"],
+            "prompts.db.pre-workflow-storage_<date and time>",
+        )
+        self.assertGreater(plan["backup"]["expected_bytes"], 0)
+        self.assertTrue(plan["disk"]["sufficient"])
+
+    async def test_apply_backs_up_rewrites_then_reclaims_space(self):
+        prompt_id = self.old_library(count=40)
+        before = self.texts()
+        listed = await self.client.request(
+            "GET", f"/prompt_manager/prompts/{prompt_id}/images"
+        )
+        listed_before = await listed.json()
+        size_before = self.database_bytes()
+        plan, summary, job = await self.start()
+        self.assertEqual(summary["kind"], MAINTENANCE_JOB_KIND)
+        self.assertEqual(summary["title"], "Store each workflow once")
+        self.assertEqual(summary["admission"], "records")
+        self.assertTrue(summary["cancellable"])
+        self.assertEqual(summary["scope"], {"operation": OPERATION})
+        reclaim = await self.finish(job)
+        self.assertEqual(job.state, "finished", job.log)
+        self.assertEqual(job.counters, {"changed": 40, "protected": 0, "errors": 0})
+        self.assertEqual(job.total, 40)
+        self.assertEqual(job.processed, 40)
+        self.assertEqual(self.inline_count(), 0)
+        self.assertEqual(self.texts(), before)
+        self.assertEqual(self.backups(), [])  # never in the pruned pool
+        [backup] = [
+            name
+            for name in os.listdir(self.tmpdir)
+            if name.startswith("prompts.db.pre-workflow-storage_")
+        ]
+        self.assertIn(f"Pre-migration backup {backup} verified", "\n".join(job.log))
+        self.assertEqual(
+            job.status_line,
+            f"Rewrote 40 image records · Pre-migration backup {backup}",
+        )
+        # The safety backup holds the old form, verified.
+        conn = sqlite3.connect(os.path.join(self.tmpdir, backup))
+        try:
+            inline = conn.execute(
+                "SELECT COUNT(*) FROM generated_images WHERE workflow_data IS NOT NULL"
+            ).fetchone()[0]
+        finally:
+            conn.close()
+        self.assertEqual(inline, 40)
+        # Reclaim space ran alone afterwards and gave the space back.
+        self.assertIsNotNone(reclaim)
+        self.assertEqual(reclaim.admission, "exclusive")
+        self.assertFalse(reclaim.cancellable)
+        self.assertEqual(reclaim.state, "finished", reclaim.log)
+        self.assertIn("Reclaim space started", "\n".join(job.log))
+        self.assertLess(self.database_bytes(), size_before / 3)
+        wal = self.api.db.model.db_path + "-wal"
+        self.assertEqual(os.path.getsize(wal) if os.path.exists(wal) else 0, 0)
+        self.assertRegex(reclaim.status_line, r"^Reclaimed [\d.,]+ MB · ")
+        listed = await self.client.request(
+            "GET", f"/prompt_manager/prompts/{prompt_id}/images"
+        )
+        self.assertEqual(await listed.json(), listed_before)
+        self.assertEqual(self.strays(), [backup])  # only the kept backup
+        self.assertNotIn(self.tmpdir, json.dumps(job.report()))
+
+    async def test_a_backup_failure_rewrites_nothing(self):
+        self.old_library()
+        from py.api import workflow_storage as api_storage
+
+        with patch.object(
+            api_storage,
+            "take_safety_backup",
+            side_effect=maintenance.SafetyBackupFailed("no"),
+        ):
+            _plan, _summary, job = await self.start()
+            await self.finish(job)
+        self.assertEqual(job.state, "failed")
+        self.assertEqual(job.last_event["code"], "backup-failed")
+        self.assertEqual(self.inline_count(), 12)
+        self.assertIsNone(self.reclaim_job())
+
+    async def test_cancel_stops_after_the_current_batch_and_a_rerun_continues(self):
+        self.old_library()
+        before = self.texts()
+        real = self.api.db.migrate_workflow_batch
+        holder = {}
+
+        def first_then_cancel(*args, **kwargs):
+            result = real(*args, **kwargs)
+            holder["job"].cancelled = True  # Cancel pressed during batch one
+            return result
+
+        with (
+            patch.object(ws, "BATCH_ROWS", 5),
+            patch.object(
+                self.api.db, "migrate_workflow_batch", side_effect=first_then_cancel
+            ),
+        ):
+            plan = await self.preview(OPERATION)
+            body = await self.apply(plan["token"])
+            holder["job"] = job = self.api._jobs.get(body["job"]["id"])
+            await self.finish(job)
+        self.assertEqual(job.state, "cancelled")
+        self.assertEqual(job.counters["changed"], 5)
+        self.assertEqual(self.inline_count(), 7)
+        self.assertIn("run it again to continue", job.status_line)
+        self.assertIsNone(self.reclaim_job())
+        self.assertEqual(self.texts(), before)
+        _plan, _summary, again = await self.start()
+        await self.finish(again)
+        self.assertEqual(again.state, "finished", again.log)
+        self.assertEqual(again.counters["changed"], 7)
+        self.assertEqual(self.inline_count(), 0)
+        self.assertEqual(self.texts(), before)
+
+    async def test_a_failed_batch_fails_the_job_and_every_record_still_reads(self):
+        self.old_library()
+        before = self.texts()
+        real = self.api.db.migrate_workflow_batch
+        calls = []
+
+        def failing(*args, **kwargs):
+            calls.append(1)
+            if len(calls) == 2:
+                raise sqlite3.OperationalError("disk I/O error")
+            return real(*args, **kwargs)
+
+        with (
+            patch.object(ws, "BATCH_ROWS", 5),
+            patch.object(self.api.db, "migrate_workflow_batch", side_effect=failing),
+        ):
+            _plan, _summary, job = await self.start()
+            await self.finish(job)
+        self.assertEqual(job.state, "failed")
+        self.assertIn(
+            "The batch in progress was rolled back; every record reads as before.",
+            job.status_line,
+        )
+        self.assertNotIn("disk I/O", job.status_line)
+        self.assertEqual(job.counters["changed"], 5)
+        self.assertEqual(job.counters["errors"], 1)
+        self.assertEqual(self.texts(), before)
+        self.assertIsNone(self.reclaim_job())
+
+    async def test_reclaim_space_that_cannot_start_says_why(self):
+        self.old_library()
+        release = asyncio.Event()
+
+        async def held(job):
+            await release.wait()
+
+        reader = self.api._jobs.start(
+            "health", "Health check", admission="read", runner=held
+        )
+        try:
+            _plan, _summary, job = await self.start()
+            await self.finish(job)
+        finally:
+            release.set()
+            await asyncio.wait_for(reader.task, WAIT)
+        self.assertEqual(job.state, "finished", job.log)
+        self.assertIsNone(self.reclaim_job())
+        self.assertIn(
+            "Reclaim space did not start: Health check is running. "
+            "Run Optimize database to give the space back.",
+            "\n".join(job.log),
+        )
+        self.assertIn("Reclaim space did not start", job.status_line)
+
+    async def test_reclaim_space_checks_the_disk_first(self):
+        self.old_library()
+        before = self.texts()
+        _plan, _summary, job = await self.start()
+        free = {"bytes": 10**12}
+
+        def free_bytes(_path):
+            return free["bytes"]
+
+        with patch.object(maintenance, "_free_bytes", side_effect=free_bytes):
+            await asyncio.wait_for(asyncio.shield(job.task), WAIT)
+            reclaim = self.reclaim_job()
+            free["bytes"] = 1024
+            # The reclaim job checks when it runs, after the rewrite.
+            await asyncio.wait_for(asyncio.shield(reclaim.task), WAIT)
+        self.assertEqual(reclaim.state, "failed")
+        self.assertEqual(reclaim.last_event["code"], "insufficient-space")
+        self.assertIn("Not enough free disk space", reclaim.status_line)
+        self.assertEqual(self.texts(), before)
+
+    async def test_apply_refuses_up_front_when_the_backup_does_not_fit(self):
+        self.old_library()
+        plan = await self.preview(OPERATION)
+        with patch.object(maintenance, "_free_bytes", return_value=1024):
+            body = await self.apply(plan["token"], status=507)
+        self.assertEqual(body["code"], "insufficient-space")
+        self.assertEqual(self.inline_count(), 12)
+
+    async def test_the_jobs_never_open_a_media_file(self):
+        self.old_library(media=True)
+        media = str(self.output_dir)
+        before = {
+            name: (
+                _digest(os.path.join(media, name)),
+                os.stat(os.path.join(media, name)),
+            )
+            for name in os.listdir(media)
+        }
+        opened = []
+        real_open, real_os_open = builtins.open, os.open
+
+        def tracking_open(file, *args, **kwargs):
+            opened.append(str(file))
+            return real_open(file, *args, **kwargs)
+
+        def tracking_os_open(path, *args, **kwargs):
+            opened.append(str(path))
+            return real_os_open(path, *args, **kwargs)
+
+        with patch("builtins.open", tracking_open), patch("os.open", tracking_os_open):
+            with patch.object(Image, "open", side_effect=AssertionError("opened")):
+                _plan, _summary, job = await self.start()
+                reclaim = await self.finish(job)
+        self.assertEqual(job.state, "finished", job.log)
+        self.assertEqual(reclaim.state, "finished", reclaim.log)
+        self.assertEqual([path for path in opened if path.startswith(media)], [])
+        for name, (digest, stat) in before.items():
+            path = os.path.join(media, name)
+            self.assertEqual(_digest(path), digest)
+            self.assertEqual(os.stat(path).st_mtime_ns, stat.st_mtime_ns)
+
+    async def test_a_second_apply_of_the_token_is_plan_used(self):
+        self.old_library(count=4)
+        plan = await self.preview(OPERATION)
+        body = await self.apply(plan["token"])
+        await self.finish(self.api._jobs.get(body["job"]["id"]))
+        again = await self.apply(plan["token"], status=409)
+        self.assertEqual(again["code"], "plan-used")
+
+
+class TestWorkflowStorageOutcomes(WorkflowStorageTestCase):
+    async def test_kept_values_and_swept_documents_are_logged(self):
+        prompt_id = self.old_library(count=4)
+        conn = self.api.db.model.get_connection()
+        with conn:
+            conn.execute(
+                "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                "file_path, workflow_data) VALUES (?, '/x/b', 'b', '/x/b', x'00')",
+                (prompt_id,),
+            )
+            store_text(conn, "a document no record references")
+        _plan, _summary, job = await self.start()
+        await self.finish(job)
+        log = "\n".join(job.log)
+        self.assertEqual(job.counters["protected"], 1)
+        self.assertIn(
+            "Rewrote 4 image records; 1 stored value not text, kept as is", log
+        )
+        self.assertIn("Removed 1 unused workflow document", log)
+
+    async def reclaim_with(self, **patches):
+        self.old_library(count=4)
+        before = self.texts()
+        with contextlib.ExitStack() as stack:
+            for name, value in patches.items():
+                stack.enter_context(patch.object(self.api.db, name, value))
+            _plan, _summary, job = await self.start()
+            reclaim = await self.finish(job)
+        self.assertEqual(self.texts(), before)
+        return reclaim
+
+    async def test_a_failed_reclaim_says_the_database_is_unchanged(self):
+        reclaim = await self.reclaim_with(
+            reclaim_space=lambda: (_ for _ in ()).throw(
+                sqlite3.OperationalError("database is locked")
+            )
+        )
+        self.assertEqual(reclaim.state, "failed")
+        self.assertEqual(
+            reclaim.status_line, "Reclaim space failed; the database is unchanged."
+        )
+
+    async def test_a_reclaim_whose_disk_cannot_be_measured_fails_cleanly(self):
+        from py.api import workflow_storage as api_storage
+
+        with patch.object(api_storage, "live_bytes", side_effect=OSError("gone")):
+            reclaim = await self.reclaim_with()
+        self.assertEqual(reclaim.state, "failed")
+        self.assertNotIn("gone", reclaim.status_line)
+
+    async def test_a_busy_checkpoint_says_the_rest_follows_a_restart(self):
+        reclaim = await self.reclaim_with(
+            reclaim_space=lambda: {
+                "before_bytes": 3 * 2**20,
+                "after_bytes": 2**20,
+                "documents_removed": 0,
+                "checkpoint_busy": True,
+            }
+        )
+        self.assertEqual(
+            reclaim.status_line,
+            "Reclaimed 2.0 MB · the database is now 1.0 MB "
+            "(the rest after the next restart)",
+        )

tests/test_database.py

@@ -1374,13 +1374,17 @@ class TestModelHousekeeping(DatabaseTestCase):
         ):
             self.assertEqual(self.db.model.get_database_info(), {})
 
-    def test_vacuum_logs_errors_instead_of_raising(self):
+    def test_vacuum_logs_and_raises_its_failure(self):
+        # S6: a failed VACUUM is reported (legacy /maintenance success:false),
+        # never logged as done.
         self._save("x")
         self.db.model.vacuum_database()
         with patch(
             "database.models.sqlite3.connect", side_effect=sqlite3.OperationalError
         ):
-            self.db.model.vacuum_database()  # logged, not raised
+            with self.assertLogs("prompt_manager", "ERROR"):
+                with self.assertRaises(sqlite3.OperationalError):
+                    self.db.model.vacuum_database()
         self.assertEqual(len(self.db.search_prompts()), 1)
 
     def test_close_tolerates_a_connection_that_fails_to_close(self):

tests/test_job_admission.py

@@ -63,7 +63,7 @@ class TestAdmissionClasses(unittest.IsolatedAsyncioTestCase):
         )
 
     async def test_admission_names_and_explicit_read(self):
-        self.assertEqual(ADMISSIONS, ("read", "guarded", "records"))
+        self.assertEqual(ADMISSIONS, ("read", "guarded", "records", "exclusive"))
         registry = JobRegistry()
         job = registry.start(
             "report", "Report", admission="read", scope={}, runner=_noop
@@ -72,7 +72,7 @@ class TestAdmissionClasses(unittest.IsolatedAsyncioTestCase):
         self.assertEqual(job.admission, "read")
         self.assertFalse(job.record_changing)
         with self.assertRaises(ValueError):
-            registry.start("x", "X", admission="exclusive", scope={}, runner=_noop)
+            registry.start("x", "X", admission="bogus", scope={}, runner=_noop)
         with self.assertRaises(ValueError):
             registry.start(
                 "x",

tests/test_jobs.py

@@ -101,6 +101,35 @@ class TestJobFanOut(unittest.IsolatedAsyncioTestCase):
 
         self.assertEqual(await _collect(job), [{"type": "complete"}])
 
+    async def test_lead_comes_first_and_nothing_published_meanwhile_is_lost(self):
+        """S5 I1: the ``job`` frame leads, with the subscription already in place."""
+        job = Job("thumbnails", "Thumbnail generation")
+        job.publish({"type": "status"})
+        lead = {"type": "job"}
+        seen = []
+        async with contextlib.aclosing(job.events(lead=lead)) as events:
+            async for payload in events:
+                seen.append(payload)
+                if payload is lead:
+                    # Published while the consumer is still sending the lead.
+                    job.publish({"type": "progress"})
+                    job.publish({"type": "complete"})
+                    job.finish()
+        self.assertEqual(
+            seen,
+            [lead, {"type": "status"}, {"type": "progress"}, {"type": "complete"}],
+        )
+
+    async def test_lead_on_a_finished_job_then_its_last_event(self):
+        job = Job("thumbnails", "Thumbnail generation")
+        job.publish({"type": "complete"})
+        job.finish()
+        seen = []
+        async with contextlib.aclosing(job.events(lead={"type": "job"})) as events:
+            async for payload in events:
+                seen.append(payload)
+        self.assertEqual(seen, [{"type": "job"}, {"type": "complete"}])
+
     async def test_departed_subscriber_is_forgotten(self):
         job = Job("scan", "Scan")
         job.publish({"type": "progress", "progress": 1})
@@ -151,6 +180,30 @@ class TestJobFanOut(unittest.IsolatedAsyncioTestCase):
 
 class TestJobRegistry(unittest.IsolatedAsyncioTestCase):
 
+    async def test_a_compatible_kind_never_blocks(self):
+        """S5 M7: a caller may name running kinds that cannot conflict with it."""
+        registry = JobRegistry()
+        gate = asyncio.Event()
+        thumbs = registry.start(
+            "thumbnails",
+            "Thumbnail generation",
+            admission="guarded",
+            runner=_gated_runner(gate),
+        )
+        self.assertIs(registry.blocker("probe", admission="records"), thumbs)
+        self.assertIsNone(
+            registry.blocker("probe", admission="records", compatible=("thumbnails",))
+        )
+        backup = registry.start(
+            "backup", "Backup", admission="guarded", runner=_gated_runner(gate)
+        )
+        self.assertIs(
+            registry.blocker("probe", admission="records", compatible=("thumbnails",)),
+            backup,
+        )
+        gate.set()
+        await asyncio.wait_for(asyncio.gather(thumbs.task, backup.task), 2)
+
     async def test_start_lists_and_finishes_a_job(self):
         registry = JobRegistry()
         gate = asyncio.Event()
@@ -421,6 +474,28 @@ class TestJobOutcomesAndLog(unittest.TestCase):
             SCAN_GROUP_NAMES, ("added", "known", "review", "no_metadata", "errors")
         )
 
+    def test_record_counted_counts_exactly_lists_capped_and_logs_nothing(self):
+        job = Job("thumbnails", "Thumbnail generation", groups=("skipped",))
+        rows = [
+            {"path": os.path.abspath(f"out{os.sep}f{i}.png"), "root": "output"}
+            for i in range(MAX_OUTCOME_ROWS + 5)
+        ]
+        job.record_counted("skipped", 10_000, rows)
+        job.record_counted("skipped", 1, [{"path": "late.png", "root_index": True}])
+        self.assertEqual(job.counters["skipped"], 10_001)
+        self.assertEqual(len(job.outcomes["skipped"]), MAX_OUTCOME_ROWS)
+        self.assertEqual(job.outcomes["skipped"][0]["path"], "f0.png")
+        self.assertIsNone(job.outcomes["skipped"][0]["root_index"])
+        self.assertEqual(job.log, [])
+        with self.assertRaises(ValueError):
+            job.record_counted("bogus", 1)
+
+    def test_thumbnail_job_groups(self):
+        self.assertEqual(
+            jobs_module.THUMBNAIL_GROUP_NAMES, ("generated", "skipped", "errors")
+        )
+        self.assertEqual(jobs_module.THUMBNAILS_JOB_KIND, "thumbnails")
+
     def test_image_tag_job_kind_and_groups(self):
         # #249 D1: bulk Add tags / Auto-tag, counter-tile order.
         self.assertEqual(getattr(jobs_module, "IMAGE_TAG_JOB_KIND", None), "image-tags")

tests/test_maintenance_plans.py

@@ -0,0 +1,546 @@
+"""Maintenance plans (#249 System S6): the plan store and the read-only previews.
+
+``PlanStore`` holds each reviewed plan server-side under a single-use token
+for ``ttl`` seconds, within a count and byte budget it never truncates to fit.
+``PromptDatabase.preview_maintenance`` builds a plan on its own ``mode=ro``
+connection: it writes nothing (the TESTING.md read-only standard) and reuses
+the legacy maintenance queries, so a preview counts what the legacy cleanup
+would change, except that a missing-file link is only counted under a source
+that can be read now.
+"""
+
+import hashlib
+import os
+import sqlite3
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from database import maintenance  # noqa: E402
+from database.operations import PromptDatabase  # noqa: E402
+from py.api.maintenance_plans import (  # noqa: E402
+    PlanExpired,
+    PlanStore,
+    PlanUsed,
+    ScopeTooLarge,
+)
+from utils.hashing import generate_prompt_hash  # noqa: E402
+
+try:
+    from tests import test_execution_history_database as history_tests  # noqa: E402
+    from tests.open_handles import assert_closed  # noqa: E402
+    from tests.test_execution_context_database import bundle  # noqa: E402
+except ImportError:  # discovered with tests/ as the top-level directory
+    import test_execution_history_database as history_tests  # noqa: E402
+    from open_handles import assert_closed  # noqa: E402
+    from test_execution_context_database import bundle  # noqa: E402
+
+OPERATIONS = ("duplicate_prompts", "optimize", "unused_components", "missing_links")
+
+
+def _digest(path):
+    if not os.path.exists(path):
+        return None
+    with open(path, "rb") as handle:
+        return hashlib.sha256(handle.read()).hexdigest()
+
+
+class FakeClock:
+    def __init__(self):
+        self.now = 1000.0
+
+    def __call__(self):
+        return self.now
+
+
+class TestPlanStore(unittest.TestCase):
+    def test_plan_expiry_budget_and_single_use(self):
+        clock = FakeClock()
+        store = PlanStore(ttl=900, max_plans=2, max_bytes=200, clock=clock)
+        first = store.issue("optimize", "sha256:a", {"targets": []}, ("db", 0))
+        self.assertGreaterEqual(len(first), 40)  # token_urlsafe(32)
+        plan = store.peek(first, "optimize")
+        self.assertEqual(
+            (plan.operation, plan.fingerprint, plan.db_identity),
+            ("optimize", "sha256:a", ("db", 0)),
+        )
+        # Single use: the second consume names the job the first one started.
+        self.assertIs(store.consume(first, "optimize", job_id="job-1"), plan)
+        with self.assertRaises(PlanUsed) as used:
+            store.consume(first, "optimize", job_id="job-2")
+        self.assertEqual(used.exception.job_id, "job-1")
+        with self.assertRaises(PlanUsed):
+            store.peek(first, "optimize")
+        # Another operation's token, an unknown token and a non-string are expired.
+        second = store.issue("missing_links", "sha256:b", {"targets": [1]}, ("db", 0))
+        for token, operation in (
+            (second, "optimize"),
+            ("nope", "missing_links"),
+            (None, "missing_links"),
+        ):
+            with self.subTest(token=token), self.assertRaises(PlanExpired):
+                store.peek(token, operation)
+        # Expiry: valid up to ttl, gone after it.
+        clock.now += 900
+        self.assertEqual(store.peek(second, "missing_links").fingerprint, "sha256:b")
+        clock.now += 0.001
+        with self.assertRaises(PlanExpired):
+            store.consume(second, "missing_links", job_id="job-3")
+        # Budget: a plan over max_bytes is refused whole, never truncated.
+        with self.assertRaises(ScopeTooLarge):
+            store.issue("unused_components", "x", {"targets": list(range(100))}, ())
+        # Count budget: the oldest plan makes room for a new one and is expired.
+        a = store.issue("optimize", "a", {}, ())
+        b = store.issue("optimize", "b", {}, ())
+        c = store.issue("optimize", "c", {}, ())
+        with self.assertRaises(PlanExpired):
+            store.peek(a, "optimize")
+        self.assertEqual(
+            [store.peek(t, "optimize").fingerprint for t in (b, c)], ["b", "c"]
+        )
+        # Byte budget across plans: older plans are dropped until the new one fits.
+        sized = PlanStore(max_plans=8, max_bytes=100, clock=clock)
+        pad = {"pad": "x" * 40}  # 50 bytes as compact JSON
+        p1, p2, p3 = (sized.issue("optimize", str(n), pad, ()) for n in range(3))
+        with self.assertRaises(PlanExpired):
+            sized.peek(p1, "optimize")
+        self.assertEqual([sized.peek(t).fingerprint for t in (p2, p3)], ["1", "2"])
+
+    def test_expires_at_is_wall_clock(self):
+        store = PlanStore(ttl=60, clock=FakeClock(), wall_clock=lambda: 5000.0)
+        token = store.issue("optimize", "f", {}, ())
+        self.assertEqual(store.peek(token, "optimize").expires_at, 5060.0)
+
+    def test_discard_forgets_a_plan(self):
+        store = PlanStore()
+        token = store.issue("optimize", "f", {}, ())
+        store.discard(token)
+        store.discard("unknown")
+        with self.assertRaises(PlanExpired):
+            store.peek(token, "optimize")
+
+    def test_used_tokens_are_remembered_within_a_bound(self):
+        store = PlanStore(max_plans=2)
+        tokens = []
+        for index in range(store.MAX_USED + 1):
+            token = store.issue("optimize", str(index), {}, ())
+            store.consume(token, "optimize", job_id=f"job-{index}")
+            tokens.append(token)
+        with self.assertRaises(PlanExpired):  # the oldest record was dropped
+            store.peek(tokens[0], "optimize")
+        with self.assertRaises(PlanUsed):
+            store.peek(tokens[-1], "optimize")
+
+
+class MaintenanceDatabaseCase(unittest.TestCase):
+    def setUp(self):
+        self.temp = tempfile.TemporaryDirectory()
+        self.root = Path(os.path.realpath(self.temp.name))
+        self.output = self.root / "output"
+        self.output.mkdir()
+        self.db_path = str(self.root / "prompts.db")
+        self.db = PromptDatabase(self.db_path)
+
+    def tearDown(self):
+        self.db.close_all()
+        assert_closed(self, self.root)
+        self.temp.cleanup()
+
+    def save(self, text, **metadata):
+        return self.db.save_prompt(
+            text, prompt_hash=generate_prompt_hash(text), **metadata
+        )
+
+    def image(self, prompt_id, name, *, exists=True, folder=None):
+        path = Path(folder or self.output) / name
+        path.parent.mkdir(parents=True, exist_ok=True)
+        if exists:
+            path.write_bytes(b"png")
+        return self.db.link_image_to_prompt(prompt_id, str(path))
+
+    def duplicate(self, text):
+        """A second prompt row whose text differs only in case (legacy duplicate)."""
+        with self.db.model.get_connection() as conn:
+            cursor = conn.execute(
+                "INSERT INTO prompts(text, created_at) VALUES(?, '2999-01-01')",
+                (text,),
+            )
+        return cursor.lastrowid
+
+    def preview(self, operation, roots=None):
+        roots = [str(self.output)] if roots is None else roots
+        return self.db.preview_maintenance(operation, roots)
+
+    def ids(self, sql, *args):
+        with self.db.model.get_connection() as conn:
+            return [row[0] for row in conn.execute(sql, args)]
+
+
+class TestPreviews(MaintenanceDatabaseCase):
+    def populate(self):
+        kept = self.save("Kept prompt")
+        self.image(kept, "kept.png")
+        self.image(kept, "gone.png", exists=False)
+        self.save("Unused prompt")
+        self.duplicate("KEPT PROMPT")
+        return kept
+
+    def test_preview_writes_nothing(self):
+        self.populate()
+        # The app's own connection stays open, as in the running server.
+        self.db.get_statistics()
+        files = [self.db_path, self.db_path + "-wal"]
+        real_connect = sqlite3.connect
+
+        class Recording(sqlite3.Connection):
+            changes_at_close = []
+
+            def close(self):
+                Recording.changes_at_close.append(self.total_changes)
+                super().close()
+
+        opened = []
+
+        def connect(*args, **kwargs):
+            conn = real_connect(*args, factory=Recording, **kwargs)
+            opened.append((args, kwargs, conn))
+            return conn
+
+        for operation in OPERATIONS:
+            with self.subTest(operation=operation):
+                before = [_digest(path) for path in files]
+                shm_size = os.path.getsize(self.db_path + "-shm")
+                media = {
+                    name: _digest(self.output / name)
+                    for name in os.listdir(self.output)
+                }
+                prompts = self.ids("SELECT id FROM prompts ORDER BY id")
+                images = self.ids("SELECT id FROM generated_images ORDER BY id")
+                opened.clear()
+                Recording.changes_at_close.clear()
+                with patch.object(maintenance.sqlite3, "connect", side_effect=connect):
+                    draft = self.preview(operation)
+                self.assertTrue(draft.fingerprint.startswith("sha256:"))
+                self.assertEqual(len(opened), 1)
+                args, kwargs, conn = opened[0]
+                self.assertTrue(str(args[0]).endswith("?mode=ro"), args)
+                self.assertTrue(kwargs.get("uri"))
+                self.assertEqual(Recording.changes_at_close, [0])
+                with self.assertRaises(sqlite3.ProgrammingError):  # closed
+                    conn.execute("SELECT 1")
+                self.assertEqual([_digest(path) for path in files], before)
+                self.assertEqual(os.path.getsize(self.db_path + "-shm"), shm_size)
+                self.assertEqual(
+                    {
+                        name: _digest(self.output / name)
+                        for name in os.listdir(self.output)
+                    },
+                    media,
+                )
+                self.assertEqual(
+                    self.ids("SELECT id FROM prompts ORDER BY id"), prompts
+                )
+                self.assertEqual(
+                    self.ids("SELECT id FROM generated_images ORDER BY id"), images
+                )
+
+    def test_unknown_operation_raises(self):
+        with self.assertRaises(ValueError):
+            self.db.preview_maintenance("reboot", [])
+
+    def test_same_library_gives_the_same_fingerprint(self):
+        self.populate()
+        for operation in OPERATIONS:
+            with self.subTest(operation=operation):
+                self.assertEqual(
+                    self.preview(operation).fingerprint,
+                    self.preview(operation).fingerprint,
+                )
+
+    def test_preview_details_are_capped_and_carry_no_server_path(self):
+        prompt = self.save("Many")
+        for index in range(60):
+            self.image(prompt, f"sub/gone-{index}.png", exists=False)
+        draft = self.preview("missing_links")
+        self.assertEqual(draft.scope["count"], 60)
+        self.assertEqual(len(draft.details), maintenance.MAX_DETAILS)
+        self.assertEqual(draft.details[0]["label"], "sub/gone-0.png")
+        self.assertEqual(draft.details[0]["detail"], "output · the file is missing")
+        self.assertNotIn(str(self.root), str(draft.public()))
+        self.assertEqual(len(draft.targets), 60)
+
+    def test_duplicate_prompts_plan_counts_the_legacy_groups(self):
+        kept = self.save("A cat")
+        self.duplicate("A CAT")
+        self.duplicate("a cat")
+        paired = self.db.get_prompt_pair(self.db.save_prompt_pair("a dog", "blur"))
+        self.duplicate("A DOG")
+        draft = self.preview("duplicate_prompts")
+        self.assertEqual(draft.scope["count"], 2)
+        self.assertEqual(draft.scope["label"], "2 duplicate prompts in 1 group")
+        self.assertEqual(draft.protected["count"], 2)
+        self.assertEqual(
+            draft.protected["reasons"], ["1 group belongs to a saved pair"]
+        )
+        self.assertEqual(draft.targets[0][0], kept)
+        self.assertEqual(len(draft.targets), 1)
+        self.assertEqual(
+            [row["detail"] for row in draft.details],
+            ["Keeps the oldest of 3 copies", "Kept: belongs to a saved pair"],
+        )
+        self.assertEqual(draft.details[0]["label"], "A cat")
+        self.assertIsNotNone(paired)
+
+    def test_unused_components_plan_protects_pairs_history_and_titled_parents(self):
+        orphan = self.save("Orphan")
+        pair = self.db.get_prompt_pair(self.db.save_prompt_pair("paired", "neg"))
+        titled = self.save("Titled parent")
+        untitled = self.save("Explicit parent")
+        protected = self.save("Protected", tags=["__protected__"])
+        with self.db.model.get_connection() as conn:
+            conn.execute(
+                "INSERT INTO prompt_library_state(prompt_id,title) "
+                "VALUES(?,?),(?,NULL)",
+                (titled, "My recipe", untitled),
+            )
+        hinted = self.save("Recorded run")
+        draft_before_hint = self.preview("unused_components")
+        self.assertEqual(sorted(draft_before_hint.targets), sorted([orphan, hinted]))
+        self.record_run_hint(hinted)
+        draft = self.preview("unused_components")
+        self.assertEqual(draft.targets, [orphan])
+        self.assertEqual(draft.scope["count"], 1)
+        self.assertEqual(draft.scope["label"], "1 unused prompt component")
+        self.assertEqual(draft.protected["count"], 6)
+        self.assertEqual(
+            draft.protected["reasons"],
+            [
+                "2 in saved pairs",
+                "1 tagged __protected__",
+                "2 with Library state (a title or an explicit parent)",
+                "1 used by a recorded run",
+            ],
+        )
+        self.assertEqual(
+            draft.details,
+            [
+                {
+                    "label": "Orphan",
+                    "detail": maintenance.UNUSED_DETAIL,
+                }
+            ],
+        )
+        # The plan counts exactly what the legacy prune removes.
+        self.assertIsNotNone(pair)
+        self.assertEqual(protected > 0, True)
+        self.assertEqual(self.db.prune_orphaned_prompts(), 1)
+        self.assertEqual(self.preview("unused_components").targets, [])
+
+    def record_run_hint(self, prompt_id, run=0):
+        """Make ``prompt_id`` the live positive hint of a recorded run."""
+        text = self.db.get_prompt_by_id(prompt_id)["text"]
+        history = history_tests.HistoryDatabaseTests
+        start = history.start(self, self.db, run)
+        self.db.write_execution_start(start)
+        self.db.write_execution_finish(history.finish(self, start))
+        self.assertTrue(
+            self.db.write_execution_contexts(
+                start.run_id, start.storage_epoch, bundle(positive=text, negative="")
+            )
+        )
+
+    def test_missing_links_plan_skips_unavailable_roots(self):
+        prompt = self.save("Links")
+        self.image(prompt, "here.png")
+        gone = self.image(prompt, "gone.png", exists=False)
+        unmounted = self.root / "unmounted"
+        self.image(prompt, "away.png", folder=unmounted)
+        os.remove(unmounted / "away.png")
+        os.rmdir(unmounted)  # the drive went away: its links look missing
+        self.image(
+            prompt, "elsewhere.png", folder=self.root / "elsewhere", exists=False
+        )
+        draft = self.preview("missing_links", [str(self.output), str(unmounted)])
+        self.assertEqual(draft.targets, [gone])
+        self.assertEqual(draft.scope["count"], 1)
+        self.assertEqual(draft.scope["label"], "1 link to a missing file")
+        self.assertEqual(draft.protected["count"], 2)
+        self.assertEqual(
+            draft.protected["reasons"],
+            [
+                "1 link under an unavailable source",
+                "1 link outside the image sources",
+            ],
+        )
+        self.assertFalse(draft.media["deletes_files"])
+        # No readable source: nothing is counted, every link is protected.
+        none = self.preview("missing_links", [str(unmounted)])
+        self.assertEqual((none.targets, none.scope["count"]), ([], 0))
+        self.assertEqual(none.protected["count"], 4)
+
+    def test_missing_links_plan_matches_the_health_check(self):
+        prompt = self.save("Links")
+        self.image(prompt, "a.png", exists=False)
+        self.image(prompt, "b.png", exists=False)
+        self.image(prompt, "c.png")
+        report = self.db.run_health_checks([str(self.output)])
+        missing = next(c for c in report["checks"] if c["id"] == "missing_files")
+        self.assertEqual(self.preview("missing_links").scope["count"], missing["count"])
+
+    def test_optimize_plan_reports_sizes_without_paths(self):
+        self.populate()
+        draft = self.preview("optimize")
+        self.assertEqual(draft.targets, [])
+        self.assertEqual(draft.scope, {"count": 1, "label": "The whole database"})
+        self.assertEqual(
+            [row["label"] for row in draft.details],
+            ["Database file", "Free pages", "Lock", "Free space needed"],
+        )
+        self.assertNotIn(str(self.root), str(draft.public()))
+
+    def test_legacy_cleanup_missing_images_skips_unavailable_roots(self):
+        prompt = self.save("Links")
+        gone = self.image(prompt, "gone.png", exists=False)
+        unmounted = self.root / "unmounted"
+        away = self.image(prompt, "away.png", folder=unmounted, exists=False)
+        outside = self.image(prompt, "x.png", folder=self.root / "other", exists=False)
+        unmounted.rmdir()
+        self.assertEqual(
+            self.db.cleanup_missing_images(roots=[str(self.output), str(unmounted)]), 1
+        )
+        remaining = self.ids("SELECT id FROM generated_images ORDER BY id")
+        self.assertEqual(remaining, sorted([away, outside]))
+        self.assertNotIn(gone, remaining)
+        # Without roots it keeps the legacy rule: every link whose file is gone.
+        self.assertEqual(self.db.cleanup_missing_images(), 2)
+
+    def test_reader_closes_its_connection_when_a_read_raises(self):
+        self.populate()
+        with patch.object(
+            maintenance, "_orphaned_ids", side_effect=sqlite3.OperationalError("boom")
+        ):
+            with self.assertRaises(sqlite3.OperationalError):
+                self.preview("unused_components")
+        # The teardown's assert_closed proves the read-only handle was closed.
+
+
+class TestApply(MaintenanceDatabaseCase):
+    """``apply_maintenance`` rechecks the fingerprint inside BEGIN IMMEDIATE."""
+
+    def apply(self, draft, roots=None):
+        roots = [str(self.output)] if roots is None else roots
+        return self.db.apply_maintenance(
+            draft.operation, draft.fingerprint, draft.targets, roots
+        )
+
+    def test_unused_components_apply_removes_exactly_the_reviewed_prompts(self):
+        first, second = self.save("One"), self.save("Two")
+        draft = self.preview("unused_components")
+        with patch.object(maintenance, "ID_BATCH", 1):  # one id per statement
+            result = self.apply(draft)
+        self.assertEqual(result["changed"], 2)
+        self.assertEqual(
+            [row["prompt_id"] for row in result["rows"]], sorted([first, second])
+        )
+        self.assertIsNone(self.db.get_prompt_by_id(first))
+        self.assertIsNone(self.db.get_prompt_by_id(second))
+
+    def test_changed_targets_raise_before_anything_is_written(self):
+        orphan, protected = self.save("Orphan"), self.save("Soon protected")
+        prompt = self.save("Links")
+        gone = self.image(prompt, "gone.png", exists=False)
+        kept = self.save("Kept")
+        self.duplicate("KEPT")
+        cases = {
+            "unused_components": lambda: self.db.model.get_connection().execute(
+                "INSERT INTO prompt_library_state(prompt_id,title) VALUES(?,?)",
+                (protected, "Now a recipe"),
+            ),
+            "missing_links": lambda: (self.output / "gone.png").write_bytes(b"back"),
+            "duplicate_prompts": lambda: self.db.update_prompt_metadata(kept, rating=4),
+        }
+        for operation, change in cases.items():
+            with self.subTest(operation=operation):
+                draft = self.preview(operation)
+                self.assertTrue(draft.targets)
+                change()
+                self.db.model.get_connection().commit()
+                with self.assertRaises(maintenance.PlanChanged):
+                    self.apply(draft)
+        self.assertIsNotNone(self.db.get_prompt_by_id(orphan))
+        self.assertIsNotNone(self.db.get_image_by_id(gone))
+        self.assertEqual(len(self.db.search_prompts(text="kept")), 2)
+
+    def test_a_link_whose_source_went_away_is_never_removed(self):
+        folder = self.root / "drive"
+        prompt = self.save("Links")
+        link = self.image(prompt, "gone.png", folder=folder, exists=False)
+        roots = [str(folder)]
+        draft = self.preview("missing_links", roots)
+        self.assertEqual(draft.targets, [link])
+        folder.rmdir()  # unmounted between the review and the apply
+        with self.assertRaises(maintenance.PlanChanged):
+            self.apply(draft, roots)
+        self.assertIsNotNone(self.db.get_image_by_id(link))
+
+    def test_missing_links_apply_removes_rows_only(self):
+        prompt = self.save("Links")
+        kept = self.image(prompt, "kept.png")
+        gone = [self.image(prompt, f"gone-{n}.png", exists=False) for n in range(3)]
+        media = {p.name: _digest(p) for p in self.output.iterdir()}
+        draft = self.preview("missing_links")
+        with patch.object(maintenance, "ID_BATCH", 2):
+            result = self.apply(draft)
+        self.assertEqual(result["changed"], 3)
+        self.assertEqual(
+            [os.path.basename(row["image_path"]) for row in result["rows"]],
+            [f"gone-{n}.png" for n in range(3)],
+        )
+        self.assertEqual({p.name: _digest(p) for p in self.output.iterdir()}, media)
+        self.assertIsNotNone(self.db.get_image_by_id(kept))
+        for image_id in gone:
+            self.assertIsNone(self.db.get_image_by_id(image_id))
+
+    def test_duplicate_prompts_apply_merges_into_the_oldest(self):
+        kept = self.save("A cat")
+        copy = self.duplicate("A CAT")
+        image = self.image(copy, "cat.png")
+        draft = self.preview("duplicate_prompts")
+        result = self.apply(draft)
+        self.assertEqual(result["changed"], 1)
+        self.assertEqual(result["rows"], [{"prompt_id": copy, "kept_id": kept}])
+        self.assertIsNone(self.db.get_prompt_by_id(copy))
+        self.assertEqual(self.db.get_image_by_id(image)["prompt_id"], kept)
+
+    def test_optimize_apply_vacuums(self):
+        self.save("Some")
+        draft = self.preview("optimize")
+        with patch.object(self.db.model, "vacuum_database") as vacuum:
+            self.assertEqual(self.apply(draft), {"changed": 1, "rows": []})
+        vacuum.assert_called_once_with()
+        self.assertEqual(self.apply(self.preview("optimize"))["changed"], 1)
+        with self.assertRaises(maintenance.PlanChanged):
+            self.db.apply_maintenance("optimize", "sha256:other", [], [])
+
+    def test_apply_of_an_unknown_operation_raises(self):
+        with self.assertRaises(ValueError):
+            self.db.apply_maintenance("reboot", "x", [], [])
+
+
+class TestVacuum(MaintenanceDatabaseCase):
+    def test_vacuum_failure_raises(self):
+        with patch(
+            "database.models.sqlite3.connect",
+            side_effect=sqlite3.OperationalError("disk I/O error"),
+        ):
+            with self.assertLogs("prompt_manager", "ERROR"):
+                with self.assertRaises(sqlite3.OperationalError):
+                    self.db.model.vacuum_database()
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_nonfinite_metadata.py

@@ -99,11 +99,16 @@ class StrictRows:
     """Assertions over the rows a write left behind."""
 
     def image_rows(self, db):
+        """The stored texts, read through the one workflow document accessor."""
+        from database.workflow_documents import DocumentReader, resolve_image_row
+
         with db.model.get_connection() as conn:
+            reader = DocumentReader(conn)
             return [
-                dict(row)
+                resolve_image_row(reader, row)
                 for row in conn.execute(
-                    "SELECT prompt_id, workflow_data, prompt_metadata, parameters "
+                    "SELECT prompt_id, workflow_data, prompt_metadata, parameters, "
+                    "workflow_document_id, prompt_document_id "
                     "FROM generated_images ORDER BY id"
                 )
             ]

tests/test_review.py

@@ -171,6 +171,13 @@ class ReviewTestCase(AdminAPITestCase):
     def _require_module(self):
         self.assertIsNotNone(review_capture, "utils.review_capture is not implemented")
 
+    def _scan_record(self, path):
+        """Record ``path`` as the scan's recorder does (only scans add items)."""
+        from py.api.admin import _scan_roots
+
+        roots = _scan_roots(self.api._get_all_output_dirs())
+        return self.api._record_review_sync(Path(str(path)), roots, inspection=True)
+
 
 class TestScanRecording(ReviewTestCase):
     async def test_scan_persists_only_ambiguous_files_with_capture_digest(self):
@@ -502,13 +509,13 @@ class TestNonFiniteGraphs(ReviewTestCase):
             ["cat NaN", "dog Infinity"],
         )
 
-    async def test_record_review_file_records_a_nan_stamped_graph(self):
+    async def test_scan_recorder_records_a_nan_stamped_graph(self):
         self._require_module()
         self._roots()
         path = self.output_dir / "nan.png"
         write_png(path, prompt=self._nan_stamped("cat", "dog"))
 
-        self.assertEqual(self.api.record_review_file(str(path)), "recorded")
+        self.assertEqual(self._scan_record(path), "recorded")
         (item,) = self._items()
         self.assertEqual(item["candidate_count"], 2)
 
@@ -856,10 +863,13 @@ class TestMonitorRecording(ReviewTestCase):
     async def test_monitor_recording_is_refused_beside_record_changing_jobs(self):
         """S1 admission: a ``records`` recording waits for no job; it is refused.
 
-        While a ``records`` or ``guarded`` job runs, the monitor's recording
-        records nothing and answers ``deferred`` (its fallback link stands and
+        While a ``records`` or ``guarded`` job runs, the monitor's pass
+        writes nothing and answers ``deferred`` (its fallback link stands and
         a later scan reads the ambiguous file again); a ``read`` job does not
-        refuse it, and it records once the blocking job has ended.
+        refuse it. Thumbnail work (the job and the gallery's holds) is
+        ``guarded`` but writes only previews, never records, so it does not
+        defer the pass either (S5 M7). Not deferred, the pass still records
+        nothing: only scans add items, so an unlinked output is ``skipped``.
         """
         self._require_module()
         from py.api.jobs import RESTORE_JOB_KIND, THUMBNAILS_JOB_KIND
@@ -870,7 +880,7 @@ class TestMonitorRecording(ReviewTestCase):
 
         for kind, admission in (
             (RESTORE_JOB_KIND, "records"),
-            (THUMBNAILS_JOB_KIND, "guarded"),
+            ("backup", "guarded"),
         ):
             with self.subTest(admission=admission):
                 async with self.api._jobs.hold(
@@ -888,9 +898,46 @@ class TestMonitorRecording(ReviewTestCase):
                 )
 
         async with self.api._jobs.hold("probe-read", "Read probe", admission="read"):
+            async with self.api._jobs.hold(
+                THUMBNAILS_JOB_KIND,
+                "Thumbnail clear (gallery page)",
+                admission="guarded",
+            ):
+                outcome = await asyncio.to_thread(
+                    self.api.record_review_file, str(path)
+                )
+        self.assertEqual(outcome, "skipped")
+        self.assertEqual(self._items(), [])
+
+    async def test_a_running_thumbnail_job_does_not_defer_monitor_recording(self):
+        """S5 M7: a long rebuild never defers the monitor's pass (no records)."""
+        self._require_module()
+        from py.api.jobs import THUMBNAILS_JOB_KIND
+
+        self._roots()
+        path = self.output_dir / "beside.png"
+        write_png(path, prompt=unlinkable_graph("cat", "dog"))
+        gate = asyncio.Event()
+
+        async def runner(job):
+            await gate.wait()
+
+        job = self.api._jobs.start(
+            THUMBNAILS_JOB_KIND,
+            "Thumbnail generation",
+            admission="guarded",
+            cancellable=True,
+            scope={},
+            runner=runner,
+        )
+        try:
             outcome = await asyncio.to_thread(self.api.record_review_file, str(path))
-        self.assertEqual(outcome, "recorded")
-        self.assertEqual(len(self._items()), 1)
+        finally:
+            gate.set()
+            await asyncio.wait_for(job.task, 5)
+        # Not deferred (thumbnail work writes no records); only scans add items.
+        self.assertEqual(outcome, "skipped")
+        self.assertEqual(self._items(), [])
 
     async def test_monitor_wiring_reaches_new_and_existing_handlers(self):
         self._require_module()
@@ -958,12 +1005,12 @@ class TestRecordedSpelling(ReviewTestCase):
         upper = self.output_dir / "Two.PNG"
         write_png(upper, prompt=unlinkable_graph("cat", "dog"))
         with _case_folding():
-            self.assertEqual(self.api.record_review_file(str(upper)), "recorded")
+            self.assertEqual(self._scan_record(upper), "recorded")
             (before,) = self._items()
             revision = self._review_revision()
             lower = self.output_dir / "two.png"
             os.rename(upper, lower)  # keeps size and mtime_ns
-            self.assertEqual(self.api.record_review_file(str(lower)), "duplicate")
+            self.assertEqual(self._scan_record(lower), "duplicate")
             (after,) = self._items()
             self.assertEqual(self._spelled(), [str(lower)])
             self.assertEqual(self._review_revision(), revision + 1)
@@ -991,7 +1038,7 @@ class TestRecordedSpelling(ReviewTestCase):
             self.assertEqual(capture.locator, after["locator"])
             self.api._captures.release(capture.token)
             # Seeing the same spelling again writes nothing.
-            self.assertEqual(self.api.record_review_file(str(lower)), "duplicate")
+            self.assertEqual(self._scan_record(lower), "duplicate")
             self.assertEqual(self._review_revision(), revision + 1)
 
     async def test_duplicate_insert_refreshes_the_spelling(self):
@@ -1032,7 +1079,8 @@ class TestRecordedSpelling(ReviewTestCase):
 
 class TestCaptureStoreShutdown(ReviewTestCase):
     async def test_recording_after_close_creates_no_store(self):
-        """A monitor recording still in flight at shutdown leaks nothing (M2)."""
+        """A scan recording still in flight at shutdown leaks nothing (M2),
+        and the monitor's pass after it captures nothing at all."""
         self._require_module()
         self._roots()
         path = self.output_dir / "two.png"
@@ -1041,11 +1089,12 @@ class TestCaptureStoreShutdown(ReviewTestCase):
         self.assertIsNone(self.api._captures)
         with patch("utils.media_capture.tempfile.mkdtemp") as mkdtemp:
             with self.assertRaises(CaptureError):
-                self.api.record_review_file(str(path))
-            # The monitor logs the failure and keeps its link.
+                self._scan_record(path)
+            # The monitor's pass records nothing, so it needs no store.
             handler = ImageGenerationHandler(self.api.db, PromptTracker(self.api.db))
             handler.review_recorder = self.api.record_review_file
             handler._record_review(str(path))
+            self.assertEqual(self.api.record_review_file(str(path)), "skipped")
         mkdtemp.assert_not_called()
         self.assertIsNone(self.api._captures)
         self.assertIsNone(self.api._captures_unsubscribe)

tests/test_review_choosable.py

@@ -202,7 +202,7 @@ class TestNothingToChooseIsNotQueued(ChoosableTestCase):
         self._roots()
         path = self._write("none.png", opaque_graph("cat", "dog"))
 
-        self.assertEqual(self.api.record_review_file(str(path)), "untraced")
+        self.assertEqual(self.api.record_review_file(str(path)), "skipped")
         self.assertEqual(self._items(), [])
 
 
@@ -320,14 +320,18 @@ class TestUntracedItemsLeaveTheQueue(ChoosableTestCase):
         await self._scan("all")
         self.assertEqual(self._pending(), 0)
 
-    def test_monitor_removes_the_pending_item_of_an_untraced_output(self):
+    async def test_the_scan_not_the_monitor_removes_an_untraced_outputs_item(self):
+        """The monitor only clears items of outputs it links; the next scan
+        removes the item of an output that offers nothing to choose."""
         self._roots()
         path = self._write("none.png", opaque_graph("cat", "dog"))
         self._record_pre_fix("none.png")
         # A pre-fix item of other bytes keeps the stat shortcut out of the way.
         self._write("none.png", opaque_graph("a cow", "dog"))
 
-        self.assertEqual(self.api.record_review_file(str(path)), "untraced")
+        self.assertEqual(self.api.record_review_file(str(path)), "skipped")
+        self.assertEqual(len(self._items()), 1)
+        await self._scan("all")
         self.assertEqual(self._items(), [])
 
 

tests/test_review_inspection.py

@@ -0,0 +1,971 @@
+"""Needs review choices from the inspection profile (follow-up to #284/#288).
+
+A Qwen image-edit batch runs several KSamplers, each with its own
+``TextEncodeQwenImageEdit(prompt=...)``, and every SaveImage shares one
+prefix: the file cannot say which prompt made it. The scan's default
+(ingestion) profile does not know the encoder or the string nodes, so every
+candidate is unresolved and the file used to leave the queue as "could not
+be traced". The inspection profile resolves every text, so these are real
+choices. Ruling (option a): when the default profile is ambiguous with fewer
+than two candidates that carry text, the scan asks the inspection profile;
+with at least two choosable candidates it records the item from that
+profile, and Apply re-resolves with the same profile. Nothing links
+automatically from the inspection profile, and the live monitor never queues.
+"""
+
+import contextlib
+import json
+import os
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+try:
+    from tests.test_api_review import ReviewRoutesTestCase
+    from tests.test_review import unlinkable_graph, write_png
+except ImportError:  # discovered with tests/ as the top-level directory
+    from test_api_review import ReviewRoutesTestCase
+    from test_review import unlinkable_graph, write_png
+
+from utils import inspection_semantics, review_capture
+from utils.hashing import generate_prompt_hash
+from utils.image_monitor import ImageGenerationHandler
+from utils.prompt_tracker import PromptTracker
+
+NAME = "kittybear_00001_.png"
+PROMPTS = ("make it night", "add a red hat", "turn the cat into a bear")
+
+
+@contextlib.contextmanager
+def _counted(module, name):
+    """Count the calls of ``module.name`` while calling through."""
+    real = getattr(module, name)
+    calls = []
+
+    def spy(*args, **kwargs):
+        calls.append(args)
+        return real(*args, **kwargs)
+
+    with patch.object(module, name, side_effect=spy):
+        yield calls
+
+
+def qwen_edit_graph(*prompts, joined=()):
+    """One Qwen image-edit sampler per prompt, all saved under one prefix.
+
+    Each sampler has its own ``TextEncodeQwenImageEdit``, used for both
+    positive and negative as the real batches do. ``joined`` prompts are
+    ``(first, second)`` pairs joined by ``JoinStringMulti`` over two
+    ``StringConstantMultiline`` nodes. Every SaveImage takes its prefix from
+    one linked string node.
+    """
+    graph = {
+        "1": {"class_type": "UNETLoader", "inputs": {"unet_name": "q.safetensors"}},
+        "2": {"class_type": "CLIPLoader", "inputs": {"clip_name": "c.safetensors"}},
+        "3": {"class_type": "VAELoader", "inputs": {"vae_name": "v.safetensors"}},
+        "4": {"class_type": "LoadImage", "inputs": {"image": "source.png"}},
+        "5": {"class_type": "PrimitiveString", "inputs": {"value": "kittybear"}},
+        "6": {
+            "class_type": "EmptyLatentImage",
+            "inputs": {"width": 512, "height": 512, "batch_size": 1},
+        },
+    }
+    sides = [("literal", text) for text in prompts]
+    sides += [("joined", parts) for parts in joined]
+    for index, (kind, text) in enumerate(sides):
+        base = 100 + 10 * index
+        prompt = text
+        if kind == "joined":
+            for offset, part in enumerate(text, start=1):
+                graph[str(base + offset)] = {
+                    "class_type": "StringConstantMultiline",
+                    "inputs": {"string": part, "strip_newlines": True},
+                }
+            graph[str(base + 3)] = {
+                "class_type": "JoinStringMulti",
+                "inputs": {
+                    "inputcount": 2,
+                    "delimiter": ", ",
+                    "return_list": False,
+                    "string_1": [str(base + 1), 0],
+                    "string_2": [str(base + 2), 0],
+                },
+            }
+            prompt = [str(base + 3), 0]
+        encoder, sampler = str(base), str(base + 5)
+        graph[encoder] = {
+            "class_type": "TextEncodeQwenImageEdit",
+            "inputs": {
+                "clip": ["2", 0],
+                "prompt": prompt,
+                "vae": ["3", 0],
+                "image": ["4", 0],
+            },
+        }
+        graph[sampler] = {
+            "class_type": "KSampler",
+            "inputs": {
+                "model": ["1", 0],
+                "positive": [encoder, 0],
+                "negative": [encoder, 0],
+                "latent_image": ["6", 0],
+                "seed": index,
+            },
+        }
+        graph[str(base + 6)] = {
+            "class_type": "VAEDecode",
+            "inputs": {"samples": [sampler, 0], "vae": ["3", 0]},
+        }
+        graph[str(base + 7)] = {
+            "class_type": "SaveImage",
+            "inputs": {"images": [str(base + 6), 0], "filename_prefix": ["5", 0]},
+        }
+    return graph
+
+
+def _prefixed(graph, prefix):
+    """``graph`` with every node id (and link to one) prefixed."""
+    renamed = {}
+    for node_id, node in graph.items():
+        inputs = {
+            key: (
+                [prefix + value[0], value[1]]
+                if isinstance(value, list) and len(value) == 2
+                else value
+            )
+            for key, value in node["inputs"].items()
+        }
+        renamed[prefix + node_id] = {**node, "inputs": inputs}
+    return renamed
+
+
+def _version(graph=None):
+    """The item version of a Qwen-edit graph under the current table."""
+    graph = graph if graph is not None else qwen_edit_graph(*PROMPTS)
+    return review_capture.inspection_review_version({"prompt": graph})
+
+
+def _retarget_unet_loader(doc):
+    """A table edit for a class the Qwen-edit graph uses (no prompt effect)."""
+    doc["facts"]["model_outputs"]["UNETLoader"] = "unet"
+
+
+def _unrelated_encoder(doc):
+    """A table edit for a class the Qwen-edit graph does not use."""
+    doc["encoders"].append(
+        {
+            "class": "UnrelatedTextEncode",
+            "slot": 0,
+            "strategy": "single",
+            "inputs": ["text"],
+        }
+    )
+
+
+def _texts(evidence):
+    return sorted(
+        c["positive"]["text"]
+        for c in evidence["candidates"]
+        if isinstance(c.get("positive"), dict)
+    )
+
+
+class InspectionTestCase(ReviewRoutesTestCase):
+    def _write(self, name=NAME, graph=None, color=(255, 0, 0)):
+        path = self.output_dir / name
+        write_png(
+            path,
+            prompt=graph if graph is not None else qwen_edit_graph(*PROMPTS),
+            color=color,
+        )
+        return path
+
+    def _pending(self):
+        return self.api.db.library_counts()["review_pending"]
+
+    def _evidence(self, item):
+        return json.loads(item["evidence"])
+
+    def _table(self, edit):
+        """Point the inspection profile at an edited copy of the table."""
+        doc = json.loads(inspection_semantics.SEMANTICS_PATH.read_text("utf-8"))
+        edit(doc)
+        folder = tempfile.mkdtemp(dir=self.tmpdir)
+        table = Path(folder) / "inspection-semantics.json"
+        table.write_text(json.dumps(doc), encoding="utf-8")
+        patcher = patch.object(inspection_semantics, "SEMANTICS_PATH", table)
+        patcher.start()
+        inspection_semantics.get_semantics.cache_clear()
+        self.addCleanup(inspection_semantics.get_semantics.cache_clear)
+        self.addCleanup(patcher.stop)
+
+    async def _apply(self, positive):
+        item, detail, observed = await self._open()
+        candidate = self._candidate(detail, positive)
+        return item, await self._decide(
+            item["id"], self._body(detail, observed, candidate["hash"])
+        )
+
+    def _linked_text(self, path):
+        row = (
+            self.api.db.model.get_connection()
+            .execute(
+                "SELECT p.text FROM generated_images gi "
+                "JOIN prompts p ON p.id=gi.prompt_id WHERE gi.image_path=?",
+                (str(path),),
+            )
+            .fetchone()
+        )
+        return row[0] if row else None
+
+
+class TestQwenEditBatchIsQueued(InspectionTestCase):
+    async def test_qwen_edit_batch_is_queued_with_its_texts(self):
+        self._roots()
+        graph = qwen_edit_graph(*PROMPTS, joined=[("a red scarf", "snow")])
+        self._write(graph=graph)
+
+        job = await self._scan()
+
+        (item,) = self._items()
+        evidence = self._evidence(item)
+        self.assertEqual(_texts(evidence), sorted(PROMPTS + ("a red scarf, snow",)))
+        self.assertEqual(item["candidate_count"], 4)
+        self.assertEqual(evidence["profile"], "inspection")
+        self.assertEqual(
+            evidence["semantics"], review_capture.semantics_version({"prompt": graph})
+        )
+        self.assertEqual(
+            item["resolver_version"],
+            review_capture.inspection_review_version({"prompt": graph}),
+        )
+        self.assertNotEqual(item["resolver_version"], review_capture.RESOLVER_VERSION)
+        self.assertEqual(job["scope"]["review_items"], 1)
+        (row,) = job["outcomes"]["review"]
+        self.assertEqual(row["path"], NAME)
+        self.assertIn("more than one prompt could match", row["detail"])
+        # Review choices only: nothing linked the file.
+        self.assertIsNone(self._linked_text(self.output_dir / NAME))
+        self.assertEqual(self._capture_files(), [])
+
+    async def test_rescan_keeps_one_item_and_skips_the_copy(self):
+        self._roots()
+        self._write()
+        await self._scan()
+        (item,) = self._items()
+
+        with _counted(self.api._captures, "capture") as copies:
+            job = await self._scan("all")
+
+        self.assertEqual(copies, [])
+        self.assertEqual([i["id"] for i in self._items()], [item["id"]])
+        self.assertEqual(job["scope"]["review_items"], 1)
+        self.assertEqual(job["scope"]["review_untraced"], 0)
+
+    async def test_more_than_sixteen_qwen_samplers_make_a_limited_item(self):
+        self._roots()
+        self._write(graph=qwen_edit_graph(*(f"edit {n}" for n in range(17))))
+
+        await self._scan()
+
+        (item,) = self._items()
+        evidence = self._evidence(item)
+        self.assertTrue(item["limited"])
+        self.assertEqual(item["candidate_count"], 0)
+        self.assertEqual(evidence["candidate_total"], 17)
+        self.assertEqual(evidence["profile"], "inspection")
+
+    async def test_evidence_over_the_cap_makes_a_limited_item(self):
+        self._roots()
+        self._write()
+        real = review_capture.canonical_evidence
+
+        def capped(value):
+            if value.get("candidates"):
+                raise ValueError("evidence exceeds the bound")
+            return real(value)
+
+        with patch.object(review_capture, "canonical_evidence", side_effect=capped):
+            await self._scan()
+
+        (item,) = self._items()
+        self.assertTrue(item["limited"])
+        self.assertEqual(self._evidence(item)["candidate_total"], 3)
+        self.assertEqual(self._evidence(item)["profile"], "inspection")
+
+
+class TestDefaultProfileStillDecides(InspectionTestCase):
+    async def test_ambiguous_file_with_texts_keeps_the_default_profile(self):
+        """A default-ambiguous file with a choice never runs the inspection
+        profile, even when that profile would offer more candidates."""
+        self._roots()
+        graph = {
+            **_prefixed(qwen_edit_graph("make it night"), "q"),
+            **unlinkable_graph("cat", "dog"),
+        }
+        self._write(graph=graph)
+
+        with patch.object(
+            review_capture,
+            "resolve_inspection",
+            side_effect=AssertionError("inspection profile ran"),
+        ):
+            await self._scan()
+
+        (item,) = self._items()
+        evidence = self._evidence(item)
+        self.assertNotIn("profile", evidence)
+        self.assertNotIn("semantics", evidence)
+        self.assertEqual(item["resolver_version"], review_capture.RESOLVER_VERSION)
+        self.assertEqual(_texts(evidence), ["cat", "dog"])
+
+    async def test_apply_of_a_default_item_never_runs_the_inspection_profile(self):
+        self._roots()
+        write_png(self.output_dir / "two.png", prompt=unlinkable_graph("cat", "dog"))
+        await self._scan()
+        with patch.object(
+            review_capture,
+            "resolve_inspection",
+            side_effect=AssertionError("inspection profile ran"),
+        ):
+            _item, (status, data) = await self._apply("cat")
+        self.assertEqual(status, 200, data)
+        self.assertEqual(self._linked_text(self.output_dir / "two.png"), "cat")
+
+
+class TestApplyInspectionCandidate(InspectionTestCase):
+    async def test_apply_links_the_chosen_qwen_edit_text(self):
+        self._require_routes()
+        self._roots()
+        path = self._write()
+        await self._scan()
+
+        item, (status, data) = await self._apply("add a red hat")
+
+        self.assertEqual(status, 200, data)
+        self.assertEqual(data["item"]["state"], "selected")
+        self.assertEqual(data["total"], 0)
+        self.assertEqual(self._linked_text(path), "add a red hat")
+        links = self.api.db.get_image_prompt_pairs(data["pair"]["image_id"])
+        (link,) = links
+        provenance = link["provenance"]
+        self.assertEqual(provenance["kind"], "user-selection")
+        self.assertEqual(provenance["resolver_version"], _version())
+        self.assertEqual(provenance["positive"]["text"], "add a red hat")
+        (decision,) = self._decisions()
+        self.assertEqual(decision["item_id"], item["id"])
+        self._no_paths()
+
+    async def test_apply_rechecks_with_the_recorded_profile(self):
+        """The default profile would not offer the candidate: Apply must use
+        the profile the item recorded, and refuses when that profile no
+        longer offers it."""
+        self._require_routes()
+        self._roots()
+        self._write()
+        await self._scan()
+        calls = []
+        real = review_capture.resolve_inspection
+
+        def spy(metadata):
+            calls.append(metadata)
+            return real(metadata)
+
+        with patch.object(review_capture, "resolve_inspection", side_effect=spy):
+            _item, (status, data) = await self._apply("make it night")
+
+        self.assertEqual(status, 200, data)
+        self.assertEqual(len(calls), 1)
+
+    async def test_table_change_keeps_the_item_applicable(self):
+        """A table change for a class the graph uses moves the item's
+        version; the pending item stays applicable while its candidate is
+        still offered."""
+        self._require_routes()
+        self._roots()
+        path = self._write()
+        await self._scan()
+        (old,) = self._items()
+        version = _version()
+        item, detail, observed = await self._open()
+
+        self._table(_retarget_unet_loader)
+        self.assertNotEqual(_version(), version)
+
+        candidate = self._candidate(detail, "add a red hat")
+        status, data = await self._decide(
+            item["id"], self._body(detail, observed, candidate["hash"])
+        )
+        self.assertEqual(status, 200, data)
+        self.assertEqual(item["id"], old["id"])
+        self.assertEqual(self._linked_text(path), "add a red hat")
+
+    async def test_table_change_that_drops_the_encoder_is_a_review_conflict(self):
+        self._require_routes()
+        self._roots()
+        self._write()
+        await self._scan()
+        item, detail, observed = await self._open()
+        candidate = self._candidate(detail, "add a red hat")
+
+        def drop_encoder(doc):
+            doc["encoders"] = [
+                entry
+                for entry in doc["encoders"]
+                if entry["class"] != "TextEncodeQwenImageEdit"
+            ]
+
+        version = _version()
+        self._table(drop_encoder)
+        self.assertNotEqual(_version(), version)
+        before = self._snapshot()
+        status, data = await self._decide(
+            item["id"], self._body(detail, observed, candidate["hash"])
+        )
+
+        self.assertEqual(status, 409, data)
+        self.assertEqual(data["code"], "review-conflict")
+        self.assertEqual(self._snapshot(), before)
+        # The rescan finds nothing to choose under the new version: the
+        # stale, undecided item leaves the queue instead of staying stuck.
+        job = await self._scan("all")
+        self.assertEqual(self._items(), [])
+        self.assertEqual(job["scope"]["review_untraced"], 1)
+
+
+class TestMonitorNeverQueues(InspectionTestCase):
+    def _handler(self):
+        handler = ImageGenerationHandler(self.api.db, PromptTracker(self.api.db))
+        handler.review_recorder = self.api.record_review_file
+        return handler
+
+    async def test_monitor_pass_never_queues_but_the_next_scan_does(self):
+        self._roots()
+        path = self._write()
+
+        self.assertEqual(self.api.record_review_file(str(path)), "skipped")
+        self.assertEqual(self._items(), [])
+        self.assertEqual(self._capture_files(), [])
+
+        await self._scan()
+        (item,) = self._items()
+        self.assertEqual(self._evidence(item)["profile"], "inspection")
+
+    async def test_monitor_keeps_the_inspection_item_of_an_unlinked_output(self):
+        """The monitor's untraced settle judges each item by its own
+        evidence: an inspection item offers a choice and stays."""
+        self._roots()
+        path = self._write()
+        await self._scan()
+        (item,) = self._items()
+        # Other bytes, so the stat shortcut does not answer for the file.
+        self._write(color=(0, 0, 255))
+
+        self.assertEqual(self.api.record_review_file(str(path)), "skipped")
+        self.assertEqual([i["id"] for i in self._items()], [item["id"]])
+
+    async def test_monitor_link_removes_the_inspection_item(self):
+        self._roots()
+        path = self._write()
+        await self._scan()
+        self.assertEqual(len(self._items()), 1)
+        prompt_id = self.api.db.save_prompt(
+            text="latest", prompt_hash=generate_prompt_hash("latest")
+        )
+
+        self._handler().link_image_to_prompt(
+            str(path),
+            {"id": prompt_id, "text": "latest", "fallback": True},
+            {"prompt": qwen_edit_graph(*PROMPTS)},
+        )
+
+        self.assertEqual(self._items(), [])
+        self.assertEqual(os.path.basename(str(path)), NAME)
+
+
+class TestTableVersion(InspectionTestCase):
+    def test_table_version_follows_content_not_layout(self):
+        doc = {"schema": 1, "encoders": [{"class": "A", "slot": 0}]}
+        same = json.loads(json.dumps(doc, indent=4))
+        other = {"schema": 1, "encoders": [{"class": "B", "slot": 0}]}
+
+        version = inspection_semantics.table_version(doc)
+
+        self.assertRegex(version, "^[0-9a-f]{64}$")
+        self.assertEqual(inspection_semantics.table_version(same), version)
+        self.assertNotEqual(inspection_semantics.table_version(other), version)
+        self.assertEqual(
+            inspection_semantics.load_semantics().version,
+            inspection_semantics.table_version(
+                json.loads(inspection_semantics.SEMANTICS_PATH.read_text("utf-8"))
+            ),
+        )
+
+    async def test_an_unusable_table_queues_nothing(self):
+        self._roots()
+        self._write()
+        unusable = inspection_semantics.SemanticsError("bad table")
+        with (
+            patch.object(inspection_semantics, "get_semantics", side_effect=unusable),
+            patch.object(review_capture, "get_semantics", side_effect=unusable),
+        ):
+            self.assertIsNone(review_capture.inspection_review_version())
+            self.assertIsNone(_version())
+            job = await self._scan()
+
+        self.assertEqual(self._items(), [])
+        (row,) = job["outcomes"]["review"]
+        self.assertIn("could not be traced", row["detail"])
+
+    async def test_an_unusable_table_keeps_existing_items_on_rescan(self):
+        self._require_routes()
+        self._roots()
+        self._write()
+        await self._scan()
+        (item,) = self._items()
+
+        unusable = inspection_semantics.SemanticsError("bad table")
+        with (
+            patch.object(inspection_semantics, "get_semantics", side_effect=unusable),
+            patch.object(review_capture, "get_semantics", side_effect=unusable),
+        ):
+            await self._scan("all")
+
+        self.assertEqual([i["id"] for i in self._items()], [item["id"]])
+
+
+class TestVersionChanges(InspectionTestCase):
+    """A version change never leaves two pending items for one file."""
+
+    async def test_table_change_for_a_graph_class_leaves_one_item(self):
+        self._require_routes()
+        self._roots()
+        path = self._write()
+        await self._scan()
+        (old,) = self._items()
+
+        self._table(_retarget_unet_loader)
+        job = await self._scan("all")
+
+        (item,) = self._items()
+        self.assertNotEqual(item["id"], old["id"])
+        self.assertEqual(item["resolver_version"], _version())
+        self.assertNotEqual(item["resolver_version"], old["resolver_version"])
+        self.assertEqual(job["scope"]["review_items"], 1)
+        self.assertEqual(self._pending(), 1)
+        _item, (status, data) = await self._apply("make it night")
+        self.assertEqual(status, 200, data)
+        self.assertEqual(self._linked_text(path), "make it night")
+
+    async def test_unrelated_table_edit_keeps_the_item_and_does_no_work(self):
+        self._roots()
+        self._write()
+        await self._scan()
+        (old,) = self._items()
+        version = _version()
+
+        self._table(_unrelated_encoder)
+        self.assertEqual(_version(), version)
+        self.assertNotEqual(
+            inspection_semantics.get_semantics().version,
+            json.loads(old["evidence"])["semantics"],
+        )
+        with (
+            _counted(self.api._captures, "capture") as copies,
+            _counted(review_capture, "resolve_inspection") as calls,
+        ):
+            await self._scan("all")
+
+        self.assertEqual(copies, [])
+        self.assertEqual([i["id"] for i in self._items()], [old["id"]])
+        self.assertEqual(calls, [])
+
+    async def test_resolver_change_leaves_one_default_item(self):
+        self._roots()
+        write_png(self.output_dir / "two.png", prompt=unlinkable_graph("cat", "dog"))
+        await self._scan()
+        (old,) = self._items()
+        self.assertEqual(old["resolver_version"], 1)
+
+        with patch.object(review_capture, "RESOLVER_VERSION", 2):
+            await self._scan("all")
+
+        (item,) = self._items()
+        self.assertNotEqual(item["id"], old["id"])
+        self.assertEqual(item["resolver_version"], 2)
+        self.assertEqual(self._pending(), 1)
+
+    async def test_decided_old_item_is_kept_and_no_duplicate_is_made(self):
+        self._require_routes()
+        self._roots()
+        self._write()
+        await self._scan()
+        item, detail, observed = await self._open()
+        status, data = await self._decide(
+            item["id"], self._body(detail, observed, "unresolved")
+        )
+        self.assertEqual(status, 200, data)
+        before = (self._items(), self._decisions())
+
+        self._table(_retarget_unet_loader)
+        await self._scan("all")
+        self.assertEqual((self._items(), self._decisions()), before)
+        # The decided bytes are not copied again on the next rescan.
+        with _counted(self.api._captures, "capture") as copies:
+            await self._scan("all")
+
+        self.assertEqual(copies, [])
+        self.assertEqual((self._items(), self._decisions()), before)
+        self.assertEqual(self._pending(), 0)
+
+    async def test_advanced_pending_item_is_kept_and_no_duplicate_is_made(self):
+        self._roots()
+        self._write()
+        await self._scan()
+        (old,) = self._items()
+        conn = self.api.db.model.get_connection()
+        with conn:
+            conn.execute(
+                "UPDATE media_review_items SET revision=1 WHERE id=?", (old["id"],)
+            )
+        before = self._items()
+
+        self._table(_retarget_unet_loader)
+        await self._scan("all")
+
+        self.assertEqual(self._items(), before)
+
+
+class TestInspectionRuns(InspectionTestCase):
+    """The heavier profile runs once per new file state, never again."""
+
+    async def test_new_file_is_inspected_once_and_not_on_rescan(self):
+        self._roots()
+        self._write()
+        with _counted(review_capture, "resolve_inspection") as first:
+            await self._scan()
+        self.assertEqual(len(self._items()), 1)
+        self.assertEqual(len(first), 1)
+
+        with _counted(review_capture, "resolve_inspection") as again:
+            await self._scan("all")
+
+        self.assertEqual(again, [])
+        self.assertEqual(len(self._items()), 1)
+
+    async def test_untraced_file_is_not_inspected_again_until_it_changes(self):
+        self._roots()
+        graph = qwen_edit_graph(*PROMPTS)
+        for node in graph.values():
+            if node["class_type"] == "TextEncodeQwenImageEdit":
+                node["class_type"] = "OpaqueTextEncode"
+        self._write(graph=graph)
+        with _counted(review_capture, "resolve_inspection") as first:
+            job = await self._scan()
+        self.assertEqual(len(first), 1)
+        (row,) = job["outcomes"]["review"]
+        self.assertIn("could not be traced", row["detail"])
+
+        with _counted(review_capture, "resolve_inspection") as again:
+            await self._scan("all")
+        self.assertEqual(again, [])
+
+        self._write(graph=graph, color=(0, 255, 0))
+        with _counted(review_capture, "resolve_inspection") as changed:
+            await self._scan("all")
+        self.assertEqual(len(changed), 1)
+        self.assertEqual(self._items(), [])
+
+    async def test_capture_with_other_records_is_inspected_again(self):
+        """The scan's result is reused only for identical records."""
+        self._roots()
+        self._write()
+        real = review_capture.captured_review
+
+        def edited(store, capture, profile=review_capture.DEFAULT):
+            metadata, resolution = real(store, capture, profile)
+            graph = json.loads(json.dumps(metadata["prompt"]))
+            graph["100"]["inputs"]["prompt"] = "a captured edit"
+            return {**metadata, "prompt": graph}, resolution
+
+        with (
+            patch.object(review_capture, "captured_review", side_effect=edited),
+            _counted(review_capture, "resolve_inspection") as calls,
+        ):
+            await self._scan()
+
+        self.assertEqual(len(calls), 2)
+        (item,) = self._items()
+        self.assertIn("a captured edit", _texts(self._evidence(item)))
+
+
+class TestClassProjection(unittest.TestCase):
+    def test_projection_over_every_named_class_is_the_whole_table(self):
+        doc = json.loads(inspection_semantics.SEMANTICS_PATH.read_text("utf-8"))
+        names = set(inspection_semantics.load_semantics().all_classes)
+        names |= set(doc["facts"]["configuration_outputs"])
+        names |= set(doc["facts"]["model_outputs"])
+        names |= set(doc["facts"]["model_passthroughs"])
+        names |= set(doc["facts"].get("scalar_outputs", {}))
+        names |= {entry["class"] for entry in doc["lineage"]}
+
+        self.assertEqual(inspection_semantics.class_projection(doc, names), doc)
+
+    def test_class_version_moves_only_for_the_classes_it_covers(self):
+        doc = json.loads(inspection_semantics.SEMANTICS_PATH.read_text("utf-8"))
+        classes = frozenset({"KSampler", "UNETLoader"})
+        version = inspection_semantics.class_version(doc, classes)
+        _unrelated_encoder(doc)
+        self.assertEqual(inspection_semantics.class_version(doc, classes), version)
+        _retarget_unet_loader(doc)
+        self.assertNotEqual(inspection_semantics.class_version(doc, classes), version)
+
+
+class TestRecordingEdges(InspectionTestCase):
+    async def test_store_refuses_a_new_version_beside_a_decided_item(self):
+        self._roots()
+        self._write()
+        await self._scan()
+        (old,) = self._items()
+        conn = self.api.db.model.get_connection()
+        with conn:
+            conn.execute(
+                "UPDATE media_review_items SET state='unresolved', revision=1 "
+                "WHERE id=?",
+                (old["id"],),
+            )
+        evidence = {**self._evidence(old), "resolver_version": 7}
+
+        inserted = self.api.db.record_review_item(
+            locator=old["locator"],
+            spelled_path=old["spelled_path"],
+            sha256=old["sha256"],
+            size=old["size"],
+            mtime_ns=old["mtime_ns"],
+            epoch=self.api.db.get_execution_epoch(),
+            resolver_version=7,
+            evidence=evidence,
+            candidate_count=old["candidate_count"],
+            limited=bool(old["limited"]),
+        )
+
+        self.assertFalse(inserted)
+        self.assertEqual([i["id"] for i in self._items()], [old["id"]])
+
+    async def test_already_linked_file_with_an_inspection_choice_is_unchanged(self):
+        self._roots()
+        path = self._write()
+        prompt_id = self.api.db.save_prompt(
+            text="latest", prompt_hash=generate_prompt_hash("latest")
+        )
+        handler = ImageGenerationHandler(self.api.db, PromptTracker(self.api.db))
+        handler.link_image_to_prompt(
+            str(path),
+            {"id": prompt_id, "text": "latest", "fallback": True},
+            {"prompt": qwen_edit_graph(*PROMPTS)},
+        )
+
+        job = await self._scan("all")
+
+        self.assertEqual(self._items(), [])
+        self.assertEqual(job["outcomes"]["review"], [])
+        (row,) = job["outcomes"]["known"]
+        self.assertIn("Already linked", row["detail"])
+        self.assertEqual(self._linked_text(path), "latest")
+
+    async def test_untraced_memory_forgets_the_oldest_state(self):
+        from py.api import review_recording
+
+        self._roots()
+        graph = qwen_edit_graph(*PROMPTS)
+        for node in graph.values():
+            if node["class_type"] == "TextEncodeQwenImageEdit":
+                node["class_type"] = "OpaqueTextEncode"
+        self._write("a.png", graph=graph)
+        self._write("b.png", graph=graph)
+        with patch.object(review_recording, "MAX_UNTRACED_MEMORY", 1):
+            await self._scan()
+            with _counted(review_capture, "resolve_inspection") as calls:
+                await self._scan("all")
+
+        # One remembered state: each file evicts the other, so both run.
+        self.assertEqual(len(calls), 2)
+        self.assertEqual(len(self.api._untraced_inspections), 1)
+
+    def test_graph_classes(self):
+        self.assertEqual(review_capture.graph_classes({"parameters": "x"}), set())
+        self.assertEqual(
+            review_capture.graph_classes({"prompt": {"1": {"class_type": "A"}}}),
+            {"A"},
+        )
+        many = {
+            str(n): {"class_type": f"C{n}"}
+            for n in range(review_capture.MAX_FINGERPRINT_CLASSES + 1)
+        }
+        self.assertIsNone(review_capture.graph_classes({"prompt": many}))
+        self.assertEqual(
+            review_capture.semantics_version({"prompt": many}),
+            inspection_semantics.get_semantics().version,
+        )
+
+
+def _move_qwen_encoder_slot(doc):
+    """An edit of the Qwen-edit encoder's own table entry."""
+    for entry in doc["encoders"]:
+        if entry["class"] == "TextEncodeQwenImageEdit":
+            entry["slot"] = 1
+
+
+def _unused_sampler(doc):
+    """A new dict-keyed sampler class the Qwen-edit graph does not use."""
+    doc["samplers"]["UnusedSampler"] = dict(doc["samplers"]["KSampler"])
+
+
+class TestFingerprintCoverage(InspectionTestCase):
+    def test_editing_the_encoders_entry_moves_the_version(self):
+        version = _version()
+        self._table(_move_qwen_encoder_slot)
+        self.assertIsNotNone(_version())
+        self.assertNotEqual(_version(), version)
+
+    def test_an_unused_dict_keyed_class_leaves_the_version(self):
+        version = _version()
+        table = inspection_semantics.get_semantics().version
+        self._table(_unused_sampler)
+        self.assertNotEqual(inspection_semantics.get_semantics().version, table)
+        self.assertEqual(_version(), version)
+
+
+class TestInspectionResolverVersion(InspectionTestCase):
+    async def test_inspection_resolver_change_supersedes_the_item(self):
+        """An inspection-only resolver change moves the item version, so the
+        rescan re-inspects and the new item is applicable (never a stuck
+        409)."""
+        self._require_routes()
+        self._roots()
+        path = self._write()
+        await self._scan()
+        (old,) = self._items()
+        version = _version()
+        real = review_capture.resolve_inspection
+
+        def newer(metadata):
+            resolution = real(metadata)
+            return {
+                **resolution,
+                "candidates": [
+                    {**c, "reason": "a newer inspection resolver"}
+                    for c in resolution["candidates"]
+                ],
+            }
+
+        with (
+            patch.object(review_capture, "INSPECTION_RESOLVER_VERSION", 2),
+            patch.object(review_capture, "resolve_inspection", side_effect=newer),
+        ):
+            self.assertNotEqual(_version(), version)
+            await self._scan("all")
+            (item,) = self._items()
+            self.assertNotEqual(item["id"], old["id"])
+            _item, (status, data) = await self._apply("add a red hat")
+
+        self.assertEqual(status, 200, data)
+        self.assertEqual(self._linked_text(path), "add a red hat")
+
+    def test_profile_budgets_are_folded_into_the_version(self):
+        from dataclasses import replace
+
+        version = _version()
+        wider = replace(
+            review_capture.INSPECTION_PROFILE,
+            total_visits=review_capture.INSPECTION_PROFILE.total_visits + 1,
+        )
+        with patch.object(review_capture, "INSPECTION_PROFILE", wider):
+            self.assertNotEqual(_version(), version)
+
+
+class TestScanContainment(InspectionTestCase):
+    async def test_an_inspection_failure_is_counted_and_the_scan_continues(self):
+        self._roots()
+        self._write("a.png")
+        self._write("b.png", graph=qwen_edit_graph("snow", "rain"))
+        real = review_capture.resolve_inspection
+
+        def fails_for_a(metadata):
+            if "make it night" in json.dumps(metadata):
+                raise RuntimeError("inspection blew up")
+            return real(metadata)
+
+        with (
+            patch.object(review_capture, "resolve_inspection", side_effect=fails_for_a),
+            self.assertLogs("prompt_manager.api", "WARNING") as logs,
+        ):
+            job = await self._scan()
+
+        self.assertEqual(job["state"], "finished")
+        (item,) = self._items()
+        self.assertEqual(os.path.basename(item["locator"]), "b.png")
+        self.assertEqual(job["scope"]["review_errors"], 1)
+        rows = {row["path"]: row for row in job["outcomes"]["review"]}
+        self.assertIn("could not be traced", rows["a.png"]["detail"])
+        self.assertIn("more than one prompt could match", rows["b.png"]["detail"])
+        self.assertTrue(any("a.png" in line for line in logs.output))
+
+
+class TestUntracedMemoryKeys(InspectionTestCase):
+    async def test_untraced_memory_is_keyed_by_a_16_byte_digest(self):
+        self._roots()
+        graph = qwen_edit_graph(*PROMPTS)
+        for node in graph.values():
+            if node["class_type"] == "TextEncodeQwenImageEdit":
+                node["class_type"] = "OpaqueTextEncode"
+        self._write(graph=graph)
+
+        await self._scan()
+
+        (key,) = self.api._untraced_inspections
+        self.assertIsInstance(key, bytes)
+        self.assertEqual(len(key), 16)
+
+
+class TestMonitorNeverRecords(InspectionTestCase):
+    async def test_monitor_never_records_an_unlinked_ambiguous_output(self):
+        """Only scans add items (maintainer ruling); the monitor only clears
+        the items of outputs it links."""
+        self._roots()
+        path = self.output_dir / "two.png"
+        write_png(path, prompt=unlinkable_graph("cat", "dog"))
+
+        with _counted(self.api._captures, "capture") as copies:
+            self.assertEqual(self.api.record_review_file(str(path)), "skipped")
+
+        self.assertEqual(copies, [])
+        self.assertEqual(self._items(), [])
+        await self._scan()
+        self.assertEqual(len(self._items()), 1)
+
+
+class TestStaleSettleContainment(InspectionTestCase):
+    async def test_a_failed_stale_settle_is_logged_and_counted(self):
+        import sqlite3
+
+        self._roots()
+        graph = qwen_edit_graph(*PROMPTS)
+        for node in graph.values():
+            if node["class_type"] == "TextEncodeQwenImageEdit":
+                node["class_type"] = "OpaqueTextEncode"
+        self._write(graph=graph)
+        with (
+            patch.object(
+                self.api.db,
+                "settle_stale_review_items",
+                side_effect=sqlite3.OperationalError("database is locked"),
+            ),
+            self.assertLogs("prompt_manager.api", "WARNING") as logs,
+        ):
+            job = await self._scan()
+
+        self.assertEqual(job["state"], "finished")
+        self.assertEqual(job["scope"]["review_errors"], 1)
+        self.assertTrue(any("Stale Needs review" in line for line in logs.output))

tests/test_review_outcome.py

@@ -355,7 +355,9 @@ class TestMonitorRule(OutcomeTestCase):
         self.assertEqual((self._items(), self._decisions()), before)
         self.assertEqual(self._pending(), 0)
 
-    async def test_record_review_file_applies_the_scan_rule(self):
+    async def test_record_review_file_only_clears_what_it_links(self):
+        """Only scans add items: the monitor's pass skips every unlinked file
+        and settles the items of a linked one."""
         self._roots()
         write_png(self.output_dir / "fallback.png", prompt=ambiguous_graph("a", "b"))
         write_png(self.output_dir / "two.png", prompt=unlinkable_graph("cat", "dog"))
@@ -366,8 +368,8 @@ class TestMonitorRule(OutcomeTestCase):
         record = self.api.record_review_file
         self.assertEqual(record(str(self.output_dir / "fallback.png")), "skipped")
         self.assertEqual(record(str(linked)), "linked")
-        self.assertEqual(record(str(self.output_dir / "two.png")), "recorded")
-        self.assertEqual(len(self._items()), 1)
+        self.assertEqual(record(str(self.output_dir / "two.png")), "skipped")
+        self.assertEqual(self._items(), [])
 
 
 if __name__ == "__main__":

tests/test_thumbnail_jobs.py

@@ -0,0 +1,849 @@
+"""Thumbnail generation as an Activity job (#249 System S5, Task 12).
+
+``GET /prompt_manager/images/thumbnails/plan`` counts what a run would do and
+writes nothing. ``POST /prompt_manager/jobs/thumbnails`` starts the
+``thumbnails`` job (admission ``guarded``, cancellable at a file boundary).
+The legacy gallery stream ``POST /images/generate-thumbnails/progress`` starts
+or attaches to that job and streams its unchanged events from it, so a client
+that goes away no longer stops the work.
+
+Ordering is driven by threading/asyncio events, never by timers.
+"""
+
+import asyncio
+import hashlib
+import os
+import sys
+import threading
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+try:
+    from tests.test_api_images import ThumbnailTestCase, make_png, parse_sse
+except ImportError:  # discovered with tests/ as the top-level directory
+    from test_api_images import ThumbnailTestCase, make_png, parse_sse
+
+from PIL import Image  # noqa: E402
+
+import py.api.images as images_module  # noqa: E402
+from py.api.thumbnail_jobs import thumbnail_options  # noqa: E402
+from py.config import PromptManagerConfig  # noqa: E402
+
+PLAN_URL = "/prompt_manager/images/thumbnails/plan"
+JOB_URL = "/prompt_manager/jobs/thumbnails"
+
+
+def _tree_state(root):
+    """{relative path: (size, mtime_ns, sha256)} of every file under ``root``."""
+    state = {}
+    for folder, _dirs, files in os.walk(root):
+        for name in files:
+            path = Path(folder) / name
+            data = path.read_bytes()
+            stat = path.stat()
+            state[path.relative_to(root).as_posix()] = (
+                stat.st_size,
+                stat.st_mtime_ns,
+                hashlib.sha256(data).hexdigest(),
+            )
+    return state
+
+
+class ThumbnailJobCase(ThumbnailTestCase):
+    """Helpers: start, gate and wait for thumbnail jobs."""
+
+    async def _start(self, **body):
+        resp = await self.client.post(JOB_URL, json=body)
+        return resp.status, await resp.json()
+
+    async def _finished(self, job_id):
+        job = self.api._jobs.get(job_id)
+        await asyncio.wait_for(job.task, 10)
+        return job
+
+    def _gate_generation(self):
+        """Block every file in the worker until released; returns (entered, release)."""
+        entered = threading.Event()
+        release = threading.Event()
+        original = self.api._generate_one
+
+        def gated(*args, **kwargs):
+            entered.set()
+            release.wait(10)
+            return original(*args, **kwargs)
+
+        self.api._generate_one = gated
+        self.addCleanup(release.set)
+        return entered, release
+
+    async def _wait(self, event):
+        loop = asyncio.get_running_loop()
+        self.assertTrue(await loop.run_in_executor(None, event.wait, 10))
+
+    @staticmethod
+    def _watch_subscribers(job):
+        """(subscribed, left): threading events for a stream joining or leaving."""
+        subscribed, left = threading.Event(), threading.Event()
+
+        class Watched(set):
+            def add(self, item):
+                super().add(item)
+                subscribed.set()
+
+            def discard(self, item):
+                super().discard(item)
+                left.set()
+
+        job._subscribers = Watched(job._subscribers)
+        return subscribed, left
+
+    def _status_lines(self):
+        """Every status line any job is given, in order."""
+        from py.api.jobs import Job
+
+        lines = []
+        original = Job.update
+
+        def update(job, **kwargs):
+            if kwargs.get("status_line") is not None:
+                lines.append(kwargs["status_line"])
+            return original(job, **kwargs)
+
+        patcher = patch.object(Job, "update", update)
+        patcher.start()
+        self.addCleanup(patcher.stop)
+        return lines
+
+    def _one_worker(self):
+        original = PromptManagerConfig.WORKER_THREADS
+        PromptManagerConfig.WORKER_THREADS = 1  # batches of 4
+        self.addCleanup(setattr, PromptManagerConfig, "WORKER_THREADS", original)
+
+    def _run_scan(self):
+        gate = asyncio.Event()
+
+        async def runner(job):
+            await gate.wait()
+
+        job = self.api._jobs.start(
+            "scan", "Scan for metadata", admission="records", scope={}, runner=runner
+        )
+
+        async def release():
+            gate.set()
+            await asyncio.wait_for(job.task, 5)
+
+        self.addAsyncCleanup(release)
+        return job
+
+
+class TestThumbnailPlan(ThumbnailJobCase):
+    async def test_plan_is_read_only_and_counts_cap(self):
+        for name in ("a", "b", "c"):
+            make_png(self.output_dir / f"{name}.png", size=(64, 64))
+        fresh = make_png(self._thumb_path("a"))
+        source_time = (self.output_dir / "a.png").stat().st_mtime
+        os.utime(fresh, (source_time + 10, source_time + 10))
+        before = _tree_state(self.comfy_dir)
+
+        with patch.object(images_module, "MAX_THUMBNAIL_FILES", 1):
+            missing = await self.client.get(
+                PLAN_URL, params={"mode": "missing", "size": "512", "quality": "high"}
+            )
+            rebuild = await self.client.get(
+                PLAN_URL, params={"mode": "rebuild", "size": "256", "quality": "low"}
+            )
+
+        self.assertEqual(missing.status, 200)
+        self.assertEqual(
+            await missing.json(),
+            {
+                "success": True,
+                "targets": 1,
+                "fresh": 1,
+                "remaining": 1,
+                "roots": ["output"],
+                "mode": "missing",
+                "size": 512,
+                "quality": "high",
+            },
+        )
+        data = await rebuild.json()
+        # Rebuild regenerates fresh previews too: all three are work, one fits the cap.
+        self.assertEqual((data["targets"], data["fresh"], data["remaining"]), (1, 1, 2))
+        self.assertEqual(
+            (data["mode"], data["size"], data["quality"]), ("rebuild", 256, "low")
+        )
+        self.assertEqual(_tree_state(self.comfy_dir), before)
+        self.assertEqual(self.api._jobs.list(), [])
+
+    async def test_plan_defaults_to_missing_previews_at_the_concept_size(self):
+        make_png(self.output_dir / "a.png")
+        data = await (await self.client.get(PLAN_URL)).json()
+        self.assertEqual(
+            (data["mode"], data["size"], data["quality"], data["targets"]),
+            ("missing", 512, "medium", 1),
+        )
+
+    async def test_size_bounds_validated(self):
+        make_png(self.output_dir / "a.png")
+        for params in (
+            {"size": "127"},
+            {"size": "2049"},
+            {"size": "512.5"},
+            {"size": "big"},
+            {"mode": "all"},
+            {"quality": "ultra"},
+        ):
+            with self.subTest(params=params):
+                resp = await self.client.get(PLAN_URL, params=params)
+                body = await resp.json()
+                self.assertEqual(resp.status, 400)
+                self.assertEqual(
+                    (body["success"], body["code"]), (False, "invalid-request")
+                )
+        for body in (
+            {"size": 127},
+            {"size": 2049},
+            {"size": "512"},
+            {"size": 512.0},
+            {"size": True},
+            {"mode": "rebuild-all"},
+            {"quality": None},
+            [],
+        ):
+            with self.subTest(body=body):
+                resp = await self.client.post(JOB_URL, json=body)
+                self.assertEqual(resp.status, 400)
+                self.assertEqual((await resp.json())["code"], "invalid-request")
+        for edge in (128, 2048):
+            with self.subTest(edge=edge):
+                resp = await self.client.get(PLAN_URL, params={"size": str(edge)})
+                self.assertEqual(resp.status, 200)
+        self.assertEqual(self.api._jobs.list(), [])
+        self.assertFalse(self._thumb_path("a").exists())
+
+
+class TestThumbnailJob(ThumbnailJobCase):
+    async def test_job_generates_missing_previews_at_the_chosen_edge(self):
+        make_png(self.output_dir / "a.png", size=(800, 400))
+        make_png(self.output_dir / "sub" / "b.png", size=(64, 64))
+        (self.output_dir / "bad.png").write_bytes(b"\x89PNG not really")
+
+        status, body = await self._start(mode="missing", size=200, quality="medium")
+
+        self.assertEqual(status, 202)
+        summary = body["job"]
+        self.assertEqual(
+            (summary["kind"], summary["title"], summary["admission"]),
+            ("thumbnails", "Thumbnail generation", "guarded"),
+        )
+        self.assertTrue(summary["cancellable"])
+        self.assertEqual(set(summary["counters"]), {"generated", "skipped", "errors"})
+        self.assertEqual(
+            summary["scope"], {"mode": "missing", "size": 200, "quality": "medium"}
+        )
+        job = await self._finished(summary["id"])
+        self.assertEqual(job.state, "finished")
+        self.assertEqual(job.counters, {"generated": 2, "skipped": 0, "errors": 1})
+        self.assertEqual((job.processed, job.total, job.progress), (3, 3, 100))
+        self.assertEqual(job.scope["output_folders"], 1)
+        with Image.open(self._thumb_path("a")) as thumb:
+            self.assertEqual(thumb.size, (200, 100))
+        rows = {row["path"]: row for row in job.outcomes["generated"]}
+        self.assertEqual(set(rows), {"a.png", "sub/b.png"})
+        self.assertEqual(
+            (rows["a.png"]["root"], rows["a.png"]["root_index"]), ("output", 0)
+        )
+        self.assertEqual(job.outcomes["errors"][0]["path"], "bad.png")
+        self.assertTrue(any("Thumbnails finished" in line for line in job.log))
+        # Rows and log lines never carry the server's absolute layout.
+        self.assertNotIn(str(self.comfy_dir), repr(job.detail()))
+
+    async def test_fresh_previews_are_counted_as_skipped(self):
+        make_png(self.output_dir / "a.png")
+        make_png(self.output_dir / "b.png")
+        fresh = make_png(self._thumb_path("a"))
+        source_time = (self.output_dir / "a.png").stat().st_mtime
+        os.utime(fresh, (source_time + 10, source_time + 10))
+
+        _, body = await self._start()
+        job = await self._finished(body["job"]["id"])
+
+        self.assertEqual(job.counters, {"generated": 1, "skipped": 1, "errors": 0})
+        self.assertEqual([r["path"] for r in job.outcomes["skipped"]], ["a.png"])
+
+    async def test_rebuild_never_touches_originals(self):
+        for name in ("a", "b"):
+            make_png(self.output_dir / f"{name}.png", size=(400, 400))
+        _, body = await self._start(size=128)
+        await self._finished(body["job"]["id"])
+        originals = {
+            name: hashlib.sha256(
+                (self.output_dir / f"{name}.png").read_bytes()
+            ).hexdigest()
+            for name in ("a", "b")
+        }
+
+        _, body = await self._start(mode="rebuild", size=256, quality="high")
+        job = await self._finished(body["job"]["id"])
+
+        self.assertEqual(job.counters, {"generated": 2, "skipped": 0, "errors": 0})
+        for name in ("a", "b"):
+            digest = hashlib.sha256(
+                (self.output_dir / f"{name}.png").read_bytes()
+            ).hexdigest()
+            self.assertEqual(digest, originals[name], name)
+            with Image.open(self._thumb_path(name)) as thumb:
+                self.assertEqual(thumb.size, (256, 256))
+
+    async def test_rebuild_keeps_old_preview_when_generation_fails(self):
+        source = make_png(self.output_dir / "a.png", size=(64, 64))
+        _, body = await self._start()
+        await self._finished(body["job"]["id"])
+        preview = self._thumb_path("a").read_bytes()
+        source.write_bytes(b"\x89PNG broken now")
+        later = self._thumb_path("a").stat().st_mtime + 10
+        os.utime(self._thumb_path("a"), (later, later))  # still fresh
+
+        _, body = await self._start(mode="rebuild")
+        job = await self._finished(body["job"]["id"])
+
+        self.assertEqual(job.counters["errors"], 1)
+        self.assertEqual(self._thumb_path("a").read_bytes(), preview)
+        leftovers = [p.name for p in self._thumb_path("a").parent.iterdir()]
+        self.assertEqual(leftovers, ["a_thumb.png"])  # no temp file left
+
+    async def test_rebuild_keeps_an_old_video_preview_instead_of_a_placeholder(self):
+        clip = self.output_dir / "clip.mp4"
+        clip.write_bytes(b"\x00")
+        preview = make_png(self.output_dir / "thumbnails" / "clip_thumb.jpg")
+        old = preview.read_bytes()
+        with (
+            patch.dict(sys.modules, {"cv2": None}),
+            patch("subprocess.Popen", side_effect=FileNotFoundError("no ffmpeg")),
+        ):
+            _, body = await self._start(mode="rebuild")
+            job = await self._finished(body["job"]["id"])
+        self.assertEqual(job.counters["errors"], 1)
+        self.assertEqual(preview.read_bytes(), old)
+
+    async def test_quality_sets_the_encoder_not_the_edge(self):
+        noise = bytes(
+            (i * 7919 + (i >> 3) * 104729) % 251 for i in range(600 * 600 * 3)
+        )
+        with Image.frombytes("RGB", (600, 600), noise) as img:
+            img.save(self.output_dir / "photo.jpg", "JPEG", quality=95)
+        sizes = {}
+        for quality in ("low", "high"):
+            _, body = await self._start(mode="rebuild", size=300, quality=quality)
+            await self._finished(body["job"]["id"])
+            thumb = self._thumb_path("photo", ".jpg")
+            with Image.open(thumb) as image:
+                self.assertEqual(image.size, (300, 300))
+            sizes[quality] = thumb.stat().st_size
+        self.assertLess(sizes["low"], sizes["high"])
+
+    async def test_no_output_folder_fails_the_job(self):
+        self.api._get_all_output_dirs = lambda: []
+        _, body = await self._start()
+        job = await self._finished(body["job"]["id"])
+        self.assertEqual(job.state, "failed")
+        self.assertEqual(job.status_line, "ComfyUI output directory not found")
+
+    async def test_cancel_stops_at_file_boundary_keeping_written_previews(self):
+        for i in range(6):
+            make_png(self.output_dir / f"p{i}.png")
+        original = PromptManagerConfig.WORKER_THREADS
+        PromptManagerConfig.WORKER_THREADS = 1
+        self.addCleanup(setattr, PromptManagerConfig, "WORKER_THREADS", original)
+        written = []
+        real_write = images_module._write_image_thumbnail
+        job_ids = []
+
+        def counting(src, dst, *args, **kwargs):
+            real_write(src, dst, *args, **kwargs)
+            written.append(dst)
+            if len(written) == 2:
+                self.assertTrue(self.api._jobs.cancel(job_ids[0]))
+
+        with patch.object(images_module, "_write_image_thumbnail", counting):
+            _, body = await self._start()
+            job_ids.append(body["job"]["id"])
+            job = await self._finished(body["job"]["id"])
+
+        self.assertEqual(job.state, "cancelled")
+        self.assertEqual(job.counters["generated"], 2)
+        self.assertEqual(len(written), 2)
+        for dst in written:
+            self.assertTrue(dst.is_file())
+        made = sorted((self.output_dir / "thumbnails").glob("*_thumb.png"))
+        self.assertEqual(made, sorted(written))
+        self.assertTrue(any("Thumbnails cancelled" in line for line in job.log))
+
+    async def test_a_second_thumbnail_job_is_refused_naming_the_first(self):
+        make_png(self.output_dir / "a.png")
+        entered, release = self._gate_generation()
+        _, first = await self._start()
+        await self._wait(entered)
+
+        status, body = await self._start(mode="rebuild")
+
+        self.assertEqual((status, body["code"]), (409, "busy"))
+        self.assertEqual(body["running"]["id"], first["job"]["id"])
+        release.set()
+        await self._finished(first["job"]["id"])
+
+    async def test_thumbnail_job_refused_while_scan_runs(self):
+        make_png(self.output_dir / "a.png")
+        self._run_scan()
+
+        status, body = await self._start()
+
+        self.assertEqual((status, body["code"]), (409, "busy"))
+        self.assertEqual(body["running"]["title"], "Scan for metadata")
+        self.assertFalse(self._thumb_path("a").exists())
+
+    async def test_scan_refused_while_thumbnail_job_runs(self):
+        make_png(self.output_dir / "a.png")
+        entered, release = self._gate_generation()
+        _, started = await self._start()
+        await self._wait(entered)
+
+        resp = await self.client.post("/prompt_manager/jobs/scan")
+        body = await resp.json()
+
+        self.assertEqual((resp.status, body["code"]), (409, "busy"))
+        self.assertEqual(body["running"]["title"], "Thumbnail generation")
+        release.set()
+        await self._finished(started["job"]["id"])
+
+    async def test_clear_refused_while_thumbnail_job_runs(self):
+        make_png(self.output_dir / "a.png")
+        kept = make_png(self.output_dir / "thumbnails" / "old_thumb.png")
+        entered, release = self._gate_generation()
+        _, started = await self._start()
+        await self._wait(entered)
+
+        resp = await self.client.post("/prompt_manager/images/clear-thumbnails")
+
+        self.assertEqual(resp.status, 409)
+        self.assertEqual(
+            await resp.json(),
+            {"success": False, "error": "Thumbnail generation already running"},
+        )
+        self.assertTrue(kept.exists())
+        release.set()
+        await self._finished(started["job"]["id"])
+
+    async def test_every_exit_sets_a_truthful_final_status_line(self):
+        """I3: finished, nothing to do, cancelled and failed each say what happened."""
+        lines = self._status_lines()
+        make_png(self.output_dir / "a.png")
+        (self.output_dir / "bad.png").write_bytes(b"\x89PNG nope")
+        _, body = await self._start()
+        job = await self._finished(body["job"]["id"])
+        self.assertEqual(job.status_line, "1 preview written · 1 error")
+
+        # Every preview fresh: the job finishes with nothing to do.
+        source_time = (self.output_dir / "a.png").stat().st_mtime
+        bad_thumb = make_png(self._thumb_path("bad"))
+        for thumb in (self._thumb_path("a"), bad_thumb):
+            os.utime(thumb, (source_time + 10, source_time + 10))
+        os.utime(self.output_dir / "bad.png", (source_time, source_time))
+        _, body = await self._start()
+        job = await self._finished(body["job"]["id"])
+        self.assertEqual((job.state, job.total), ("finished", 0))
+        self.assertEqual(
+            job.status_line, "No previews to generate · 2 already up to date"
+        )
+        self.assertNotIn("Processing file 0 of 0", lines)
+
+    async def test_an_empty_library_says_there_is_nothing_to_preview(self):
+        lines = self._status_lines()
+        _, body = await self._start()
+        job = await self._finished(body["job"]["id"])
+        self.assertEqual(
+            (job.state, job.status_line), ("finished", "No images or videos to preview")
+        )
+        self.assertNotIn("Processing file 0 of 0", lines)
+
+    async def test_a_cancelled_job_says_what_it_kept(self):
+        for i in range(6):
+            make_png(self.output_dir / f"p{i}.png")
+        self._one_worker()
+        real_write = images_module._write_image_thumbnail
+        job_ids = []
+        written = []
+
+        def counting(src, dst, *args, **kwargs):
+            real_write(src, dst, *args, **kwargs)
+            written.append(dst)
+            if len(written) == 1:
+                self.api._jobs.cancel(job_ids[0])
+
+        with patch.object(images_module, "_write_image_thumbnail", counting):
+            _, body = await self._start()
+            job_ids.append(body["job"]["id"])
+            job = await self._finished(body["job"]["id"])
+        self.assertEqual(job.state, "cancelled")
+        self.assertEqual(job.status_line, "Cancelled · 1 preview written")
+
+    async def test_an_internal_error_says_the_job_failed(self):
+        def broken(*_args, **_kwargs):
+            raise RuntimeError("boom")
+
+        self.api._plan_gallery_thumbnails = broken
+        _, body = await self._start()
+        job = await self._finished(body["job"]["id"])
+        self.assertEqual(
+            (job.state, job.status_line),
+            ("failed", "Thumbnail generation stopped on an internal error."),
+        )
+
+    async def test_cancel_during_planning_stops_the_walk_and_starts_nothing(self):
+        """M2: the walk checks Cancel per file; no ``start`` frame is published."""
+        for i in range(10):
+            make_png(self.output_dir / f"p{i}.png")
+        entered, release = threading.Event(), threading.Event()
+        self.addCleanup(release.set)
+        checked = []
+        real_fresh = images_module._thumbnail_is_fresh
+
+        def gated_fresh(src, dst):
+            checked.append(src.name)
+            if len(checked) == 1:
+                entered.set()
+                release.wait(10)
+            return real_fresh(src, dst)
+
+        with patch.object(images_module, "_thumbnail_is_fresh", gated_fresh):
+            stream = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
+            await self._wait(entered)
+            job = self.api._jobs.active("thumbnails")
+            self.assertTrue(self.api._jobs.cancel(job.id))
+            release.set()
+            events = parse_sse(await (await stream).text())
+            await asyncio.wait_for(job.task, 10)
+
+        self.assertEqual(checked, ["p0.png"])  # the walk stopped at the next file
+        self.assertEqual([name for name, _ in events], ["job", "status", "error"])
+        self.assertEqual(job.state, "cancelled")
+        self.assertEqual(job.status_line, "Cancelled before any preview was written")
+        self.assertEqual(job.total, 0)
+        self.assertFalse((self.output_dir / "thumbnails").exists())
+
+    async def test_a_locked_preview_is_a_readable_coded_error(self):
+        """M6: Windows refuses to replace a preview another process has open."""
+        make_png(self.output_dir / "a.png")
+        locked = PermissionError(13, "Access is denied")
+        locked.winerror = 5
+
+        def refuse(*_args, **_kwargs):
+            raise locked
+
+        with patch.object(images_module, "_write_image_thumbnail", refuse):
+            _, body = await self._start(mode="rebuild")
+            job = await self._finished(body["job"]["id"])
+
+        self.assertEqual(job.counters["errors"], 1)
+        [row] = job.outcomes["errors"]
+        self.assertEqual(row["detail"], images_module.PREVIEW_IN_USE)
+        self.assertNotIn("WinError", repr(job.detail()))
+        self.assertTrue(any(images_module.PREVIEW_IN_USE in line for line in job.log))
+
+    async def test_a_locked_video_preview_is_the_same_coded_error(self):
+        clip = self.output_dir / "clip.mp4"
+        clip.write_bytes(b"\x00")
+        locked = PermissionError(13, "Access is denied")
+        locked.winerror = 32
+
+        def refuse(*_args, **_kwargs):
+            raise locked
+
+        with (
+            patch.dict(sys.modules, {"cv2": None}),
+            patch("subprocess.Popen", side_effect=FileNotFoundError("no ffmpeg")),
+            patch.object(images_module, "_atomic_save", refuse),
+        ):
+            _, body = await self._start()
+            job = await self._finished(body["job"]["id"])
+        [row] = job.outcomes["errors"]
+        self.assertEqual(row["detail"], images_module.PREVIEW_IN_USE)
+
+    async def test_a_permission_error_off_windows_names_the_permission(self):
+        make_png(self.output_dir / "a.png")
+
+        def refuse(*_args, **_kwargs):
+            raise PermissionError(13, "Permission denied")
+
+        with patch.object(images_module, "_write_image_thumbnail", refuse):
+            _, body = await self._start()
+            job = await self._finished(body["job"]["id"])
+        [row] = job.outcomes["errors"]
+        self.assertEqual(row["detail"], images_module.PREVIEW_NOT_WRITABLE)
+
+    async def test_ffmpeg_fallback_keeps_quality_and_aspect_ratio(self):
+        """M1: without OpenCV, ffmpeg keeps the chosen quality and the aspect ratio."""
+        (self.output_dir / "clip.mp4").write_bytes(b"\x00")
+        commands = []
+
+        def fake_ffmpeg(cmd, **_kwargs):
+            commands.append(cmd)
+            Path(cmd[-1]).write_bytes(b"jpeg")
+            done = type("Done", (), {"returncode": 0})()
+            done.communicate = lambda timeout=None: ("", "")
+            return done
+
+        for quality, qscale in (("high", "2"), ("medium", "4"), ("low", "8")):
+            with (
+                patch.dict(sys.modules, {"cv2": None}),
+                patch("subprocess.Popen", side_effect=fake_ffmpeg),
+            ):
+                _, body = await self._start(mode="rebuild", size=640, quality=quality)
+                job = await self._finished(body["job"]["id"])
+            self.assertEqual(job.counters["generated"], 1, quality)
+            cmd = commands[-1]
+            self.assertNotIn("-s", cmd)
+            self.assertEqual(
+                cmd[cmd.index("-vf") + 1],
+                "scale=640:640:force_original_aspect_ratio=decrease",
+            )
+            self.assertEqual(cmd[cmd.index("-q:v") + 1], qscale, quality)
+
+    async def test_the_placeholder_uses_the_chosen_quality(self):
+        (self.output_dir / "clip.mp4").write_bytes(b"\x00")
+        saved = []
+        real_save = images_module._atomic_save
+
+        def recording(img, dst, fmt=None, **kwargs):
+            # The encoder settings (guard is the automatic previews' hold check).
+            saved.append({k: v for k, v in kwargs.items() if k != "guard"})
+            return real_save(img, dst, fmt, **kwargs)
+
+        with (
+            patch.dict(sys.modules, {"cv2": None}),
+            patch("subprocess.Popen", side_effect=FileNotFoundError("no ffmpeg")),
+            patch.object(images_module, "_atomic_save", recording),
+        ):
+            _, body = await self._start(quality="low")
+            await self._finished(body["job"]["id"])
+        self.assertEqual(saved, [images_module.THUMBNAIL_ENCODINGS["low"]])
+
+
+# The legacy stream's frames and their keys, as the gallery page reads them.
+LEGACY_KEYS = {
+    "job": {"id", "started"},
+    "status": {"phase", "message"},
+    "start": {
+        "total_images",
+        "skipped",
+        "remaining",
+        "phase",
+        "image_count",
+        "video_count",
+        "message",
+    },
+    "progress": {
+        "processed",
+        "total_images",
+        "generated",
+        "skipped",
+        "error_count",
+        "percentage",
+        "rate",
+        "eta",
+        "elapsed",
+        "current_file",
+        "file_type",
+        "action",
+    },
+    "complete": {
+        "count",
+        "skipped",
+        "total_images",
+        "remaining",
+        "errors",
+        "error_count",
+        "elapsed_time",
+        "processing_rate",
+        "message",
+    },
+}
+
+
+class TestLegacyStreamOnTheJob(ThumbnailJobCase):
+    async def test_legacy_progress_events_unchanged_on_success(self):
+        make_png(self.output_dir / "a.png", size=(400, 400))
+        make_png(self.output_dir / "b.png")
+
+        resp = await self.client.post(f"{self.PROGRESS_URL}?quality=low")
+        events = parse_sse(await resp.text())
+
+        self.assertEqual(
+            [name for name, _ in events],
+            ["job", "status", "start", "progress", "progress", "complete"],
+        )
+        for name, data in events:
+            self.assertEqual(set(data), LEGACY_KEYS[name], name)
+        complete = events[-1][1]
+        self.assertEqual((complete["count"], complete["total_images"]), (2, 2))
+        self.assertEqual(
+            events[1][1],
+            {"phase": "scanning", "message": "Scanning for images and videos..."},
+        )
+        with Image.open(self._thumb_path("a")) as thumb:
+            self.assertEqual(max(thumb.size), 150)  # the legacy preset still sizes
+        [job] = self.api._jobs.list()
+        self.assertEqual(
+            (job.kind, job.title, job.state, job.cancellable),
+            ("thumbnails", "Thumbnail generation", "finished", True),
+        )
+        self.assertEqual(job.counters["generated"], 2)
+        # I1: the first frame names the job this stream started, for its Cancel.
+        self.assertEqual(events[0][1], {"id": job.id, "started": True})
+
+    async def test_legacy_file_errors_still_stream(self):
+        make_png(self.output_dir / "good.png")
+        (self.output_dir / "bad.png").write_bytes(b"\x89PNG nope")
+
+        resp = await self.client.post(self.PROGRESS_URL)
+        events = parse_sse(await resp.text())
+
+        self.assertEqual(
+            [d for e, d in events if e == "file_error"],
+            [{"file": "bad.png", "error": "Not a valid image file"}],
+        )
+
+    async def test_thumbnail_job_survives_client_disconnect(self):
+        """I2: the stream sees its client go after batch 1; every batch still runs.
+
+        One worker thread makes batches of 4, so 9 files take three batches. The
+        first file waits until the client is gone; the second batch waits until
+        the stream has seen it go (its subscription dropped) and records whether
+        the job was still running then. A stream that stopped the work on
+        disconnect would leave 4 previews; the job writes all 9.
+        """
+        for i in range(9):
+            make_png(self.output_dir / f"p{i}.png")
+        self._one_worker()
+        first, closed = threading.Event(), threading.Event()
+        state = {"calls": 0, "left": None, "running_when_left": None}
+        original = self.api._generate_one
+
+        def gated(*args, **kwargs):
+            state["calls"] += 1
+            if state["calls"] == 1:
+                first.set()
+                closed.wait(10)
+            elif state["calls"] == 5:  # the second batch
+                state["running_when_left"] = state["left"].wait(10)
+            return original(*args, **kwargs)
+
+        self.api._generate_one = gated
+        self.addCleanup(closed.set)
+
+        resp = await self.client.post(self.PROGRESS_URL)
+        self.assertEqual(resp.status, 200)
+        await self._wait(first)
+        job = self.api._jobs.active("thumbnails")
+        _subscribed, state["left"] = self._watch_subscribers(job)
+        resp.close()  # the gallery tab went away
+        closed.set()
+        await self._wait(state["left"])  # the server saw the client go
+        self.assertEqual(job.subscriber_count, 0)
+        await asyncio.wait_for(job.task, 10)
+
+        self.assertEqual(job.counters["generated"], 9)
+        self.assertEqual(job.state, "finished")
+        self.assertTrue(state["running_when_left"])
+        for i in range(9):
+            self.assertTrue(self._thumb_path(f"p{i}").is_file())
+
+    async def test_legacy_stream_attaches_to_a_running_job(self):
+        make_png(self.output_dir / "a.png")
+        entered, release = self._gate_generation()
+        _, started = await self._start(mode="missing", size=256)
+        await self._wait(entered)
+
+        job = self.api._jobs.get(started["job"]["id"])
+        subscribed, _left = self._watch_subscribers(job)
+        attached = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
+        await self._wait(subscribed)  # M4: an event with a ceiling, never a poll
+        release.set()
+        events = parse_sse(await (await attached).text())
+
+        self.assertEqual(events[-1][0], "complete")
+        # I1: the stream names the job it did not start: the gallery never cancels it.
+        self.assertEqual(events[0], ("job", {"id": job.id, "started": False}))
+        # One job: the stream followed the running one with its own options.
+        [listed] = self.api._jobs.list()
+        self.assertIs(listed, job)
+        self.assertEqual(job.scope["size"], 256)
+
+    async def test_cancel_from_activity_ends_the_legacy_stream_with_an_error(self):
+        for i in range(2):
+            make_png(self.output_dir / f"p{i}.png")
+        entered, release = self._gate_generation()
+        stream = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
+        await self._wait(entered)
+        job = self.api._jobs.active("thumbnails")
+        self.assertTrue(self.api._jobs.cancel(job.id))
+        release.set()
+
+        events = parse_sse(await (await stream).text())
+
+        self.assertEqual(events[-1][0], "error")
+        self.assertEqual(events[-1][1]["error"], "Thumbnail generation was cancelled")
+        await asyncio.wait_for(job.task, 10)
+        self.assertEqual(job.state, "cancelled")
+
+    async def test_a_job_started_while_the_stream_prepares_is_attached(self):
+        """M3: a job that starts during ``prepare`` is followed, not refused."""
+        from aiohttp import web
+
+        make_png(self.output_dir / "a.png")
+        entered, release = self._gate_generation()
+        real_prepare = web.StreamResponse.prepare
+        raced = []
+
+        async def prepare(response, request):
+            if not raced and type(response) is web.StreamResponse:
+                options, _ = thumbnail_options({"size": 256})
+                raced.append(self.api._start_thumbnail_job(options))
+            return await real_prepare(response, request)
+
+        with patch.object(web.StreamResponse, "prepare", prepare):
+            stream = asyncio.ensure_future(self.client.post(self.PROGRESS_URL))
+            await self._wait(entered)
+            release.set()
+            events = parse_sse(await (await stream).text())
+
+        [job] = raced
+        self.assertEqual(events[0], ("job", {"id": job.id, "started": False}))
+        self.assertEqual(events[-1][0], "complete")
+        self.assertEqual(self.api._jobs.list(), [job])
+
+    async def test_a_refused_stream_names_no_job(self):
+        make_png(self.output_dir / "a.png")
+        self._run_scan()
+        events = parse_sse(await (await self.client.post(self.PROGRESS_URL)).text())
+        self.assertEqual(
+            events,
+            [
+                (
+                    "error",
+                    {
+                        "error": "Scan for metadata already running",
+                        "message": "Scan for metadata already running",
+                    },
+                )
+            ],
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_workflow_documents.py

@@ -0,0 +1,817 @@
+"""Each workflow and prompt graph stored once (database/workflow_documents.py).
+
+``generated_images.workflow_data`` and ``prompt_metadata`` held the same large
+JSON text again for every image of a run. The text now lives once in
+``workflow_documents`` (keyed by the sha256 of its exact UTF-8 bytes, zlib
+compressed) and image rows reference it. Old rows keep their inline text until
+the migration rewrites them; every reader goes through one accessor, which
+reads both forms, and gives back the exact text that was stored.
+"""
+
+import json
+import os
+import random
+import shutil
+import sqlite3
+import sys
+import tempfile
+import unittest
+import zlib
+from unittest.mock import patch
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from database import workflow_documents as wd  # noqa: E402
+from database.models import PromptModel  # noqa: E402
+from database.operations import PromptDatabase  # noqa: E402
+from utils.hashing import generate_prompt_hash  # noqa: E402
+from utils.json_strict import dumps_strict  # noqa: E402
+
+try:
+    from tests.open_handles import assert_closed
+except ImportError:  # discovered with tests/ as the top-level directory
+    from open_handles import assert_closed
+
+NAN = float("nan")
+INF = float("inf")
+
+
+def sample_workflow(seed=0, nodes=40):
+    """A ComfyUI-shaped UI workflow: nodes, links, widgets and groups."""
+    rng = random.Random(seed)
+    return {
+        "last_node_id": nodes,
+        "last_link_id": nodes * 2,
+        "nodes": [
+            {
+                "id": index,
+                "type": rng.choice(
+                    ["KSampler", "CLIPTextEncode", "VAEDecode", "LoraLoader"]
+                ),
+                "pos": [rng.uniform(-2000, 2000), rng.uniform(-2000, 2000)],
+                "size": {"0": 315, "1": 262},
+                "flags": {},
+                "order": index,
+                "mode": 0,
+                "inputs": [{"name": "model", "type": "MODEL", "link": index}],
+                "outputs": [{"name": "LATENT", "type": "LATENT", "links": [index]}],
+                "properties": {"Node name for S&R": "KSampler"},
+                "widgets_values": [seed * 1000 + index, "fixed", 20, 7.5, "euler"],
+            }
+            for index in range(nodes)
+        ],
+        "links": [[i, i, 0, i + 1, 0, "MODEL"] for i in range(nodes)],
+        "groups": [{"title": "Sampler ✨ 采样", "bounding": [0, 0, 100, 100]}],
+        "config": {},
+        "extra": {"ds": {"scale": 0.5, "offset": [0, 0]}},
+        "version": 0.4,
+    }
+
+
+def sample_prompt_graph(seed=0):
+    return {
+        "3": {
+            "class_type": "KSampler",
+            "inputs": {"seed": seed, "steps": 20, "cfg": 7.0, "model": ["4", 0]},
+        },
+        "6": {"class_type": "CLIPTextEncode", "inputs": {"text": f"a cat {seed}"}},
+    }
+
+
+def edge_texts():
+    """Texts whose bytes a parse-and-serialize round trip would change."""
+    nested = "[" * 500 + "]" * 500
+    return [
+        "",
+        "{}",
+        "null",
+        " ",
+        "\n",
+        '{"a": 1}',
+        '{"a":1}',
+        '{"a": 1, "a": 2}',  # duplicate key: a parse would keep one
+        '{"b": 1, "a": 2}',  # key order
+        '{"v": 1.0}',
+        '{"v": 1e3}',  # number spelling
+        '{"v": 1.10}',
+        '{"v": NaN, "w": Infinity, "x": -Infinity}',  # ComfyUI's tokens
+        '{"text": "NaN"}',
+        '{"u": "\\u00e9\\ud83d\\ude00"}',  # escaped
+        '{"u": "é 😀 中文 עברית é"}',  # raw unicode, astral, RTL, combining
+        "trailing space ",
+        '{"nul": "a\u0000b"}',
+        "a\u0000b",
+        "{}",  # byte-order mark
+        "

",
+        "not json at all",
+        nested,
+        json.dumps(sample_workflow(1)),
+        json.dumps(sample_workflow(1), indent=2),
+        json.dumps(sample_workflow(1), separators=(",", ":")),
+        json.dumps({"v": NAN, "w": [INF, -INF]}),  # Python's lenient output
+        dumps_strict({"v": NAN, "w": [INF, -INF]}),
+    ]
+
+
+def random_text(rng):
+    """Arbitrary text of valid code points (no surrogates), ASCII to astral."""
+    length = rng.choice([0, 1, 2, 7, 64, 1000, 20000])
+    ranges = [(0x00, 0x7F), (0x80, 0x7FF), (0x800, 0xD7FF), (0xE000, 0xFFFF)]
+    ranges.append((0x10000, 0x10FFFF))
+    chars = []
+    for _ in range(length):
+        low, high = rng.choice(ranges)
+        chars.append(chr(rng.randint(low, high)))
+    return "".join(chars)
+
+
+class DocumentsTestCase(unittest.TestCase):
+    def setUp(self):
+        self.tmpdir = os.path.realpath(tempfile.mkdtemp())
+        self.db_path = os.path.join(self.tmpdir, "prompts.db")
+        self.db = PromptDatabase(self.db_path)
+        self.addCleanup(self._cleanup)
+
+    def _cleanup(self):
+        PromptModel.close_all_instances()
+        PromptModel.reset_schema_cache()
+        assert_closed(self, self.tmpdir)
+        shutil.rmtree(self.tmpdir)
+
+    def conn(self):
+        return self.db.model.get_connection()
+
+    def prompt(self, text="a cat"):
+        return self.db.save_prompt(text, prompt_hash=generate_prompt_hash(text))
+
+    def old_form_image(self, prompt_id, name, workflow, prompt_graph):
+        """A row as every version before the dedupe wrote it: text inline."""
+        conn = self.conn()
+        with conn:
+            path = os.path.join(self.tmpdir, name)
+            cursor = conn.execute(
+                "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                "file_path, workflow_data, prompt_metadata, parameters) "
+                "VALUES (?, ?, ?, ?, ?, ?, ?)",
+                (prompt_id, path, name, path, workflow, prompt_graph, "{}"),
+            )
+        return cursor.lastrowid
+
+    def raw(self, image_id):
+        return dict(
+            self.conn()
+            .execute(
+                "SELECT workflow_data, prompt_metadata, workflow_document_id, "
+                "prompt_document_id FROM generated_images WHERE id=?",
+                (image_id,),
+            )
+            .fetchone()
+        )
+
+    def document_count(self):
+        return (
+            self.conn().execute("SELECT COUNT(*) FROM workflow_documents").fetchone()[0]
+        )
+
+    def linked(self, prompt_id, name, seed):
+        return self.db.link_image_to_prompt(
+            prompt_id,
+            os.path.join(self.tmpdir, name),
+            {"workflow": sample_workflow(seed), "prompt": sample_prompt_graph(seed)},
+        )
+
+    def store(self, text):
+        conn = self.conn()
+        with conn:
+            return wd.store_text(conn, text)
+
+
+class TestRoundTrip(DocumentsTestCase):
+    def assert_round_trip(self, text):
+        document_id = self.store(text)
+        self.assertIsInstance(document_id, int)
+        back = wd.read_stored_text(self.conn(), None, document_id)
+        self.assertIsInstance(back, str)
+        self.assertEqual(back, text)
+        self.assertEqual(back.encode("utf-8"), text.encode("utf-8"))
+        return document_id
+
+    def test_edge_texts_come_back_byte_identical(self):
+        for text in edge_texts():
+            with self.subTest(text=text[:40]):
+                self.assert_round_trip(text)
+
+    def test_texts_that_parse_alike_stay_distinct(self):
+        spaced = self.assert_round_trip('{"a": 1}')
+        tight = self.assert_round_trip('{"a":1}')
+        self.assertNotEqual(spaced, tight)
+
+    def test_random_texts_come_back_byte_identical(self):
+        rng = random.Random(20261024)
+        for case in range(400):
+            text = random_text(rng)
+            with self.subTest(case=case):
+                self.assert_round_trip(text)
+
+    def test_random_workflows_come_back_byte_identical(self):
+        rng = random.Random(7)
+        for case in range(150):
+            value = sample_workflow(rng.randint(0, 10**9), nodes=rng.randint(0, 60))
+            if rng.random() < 0.3:
+                value["nodes"].append({"widgets_values": [NAN, INF, -INF]})
+            for text in (json.dumps(value), dumps_strict(value)):
+                with self.subTest(case=case):
+                    self.assert_round_trip(text)
+
+    def test_huge_workflow_round_trips(self):
+        text = json.dumps(sample_workflow(3, nodes=4000))
+        self.assertGreater(len(text), 1_200_000)
+        self.assert_round_trip(text)
+
+    def test_identical_text_is_stored_once(self):
+        text = json.dumps(sample_workflow(5))
+        first = self.store(text)
+        second = self.store(text)
+        self.assertEqual(first, second)
+        self.assertEqual(self.document_count(), 1)
+
+    def test_null_stays_null(self):
+        self.assertIsNone(self.store(None))
+        self.assertIsNone(wd.read_stored_text(self.conn(), None, None))
+        self.assertEqual(self.document_count(), 0)
+
+    def test_document_is_compressed_utf8_keyed_by_sha256(self):
+        text = "é" + json.dumps(sample_workflow(9))
+        document_id = self.store(text)
+        row = (
+            self.conn()
+            .execute(
+                "SELECT sha256, byte_length, data FROM workflow_documents WHERE id=?",
+                (document_id,),
+            )
+            .fetchone()
+        )
+        raw = text.encode("utf-8")
+        import hashlib
+
+        self.assertEqual(bytes(row["sha256"]), hashlib.sha256(raw).digest())
+        self.assertEqual(row["byte_length"], len(raw))
+        self.assertEqual(zlib.decompress(row["data"]), raw)
+        self.assertLess(len(row["data"]), len(raw) / 3)
+
+    def test_inline_text_wins_over_a_document(self):
+        document_id = self.store("from the document")
+        self.assertEqual(
+            wd.read_stored_text(self.conn(), "inline", document_id), "inline"
+        )
+        self.assertEqual(wd.read_stored_text(self.conn(), "", document_id), "")
+
+    def test_a_tampered_document_reads_as_unreadable_never_as_other_text(self):
+        document_id = self.store('{"a": 1}')
+        conn = self.conn()
+        for data in (zlib.compress(b'{"a": 2}'), b"not zlib"):
+            with conn:
+                conn.execute(
+                    "UPDATE workflow_documents SET data=? WHERE id=?",
+                    (data, document_id),
+                )
+            with self.subTest(data=data), self.assertLogs(wd.logger, level="WARNING"):
+                self.assertIsNone(wd.read_stored_text(conn, None, document_id))
+
+    def test_a_missing_document_reads_as_unreadable(self):
+        with self.assertLogs(wd.logger, level="WARNING"):
+            self.assertIsNone(wd.read_stored_text(self.conn(), None, 424242))
+
+    def test_reader_memo_returns_the_same_text(self):
+        text = json.dumps(sample_workflow(2))
+        document_id = self.store(text)
+        reader = wd.DocumentReader(self.conn())
+        self.assertEqual(reader.text(None, document_id), text)
+        self.assertEqual(reader.text(None, document_id), text)
+        self.assertEqual(reader.text("inline", document_id), "inline")
+
+
+class TestSchema(DocumentsTestCase):
+    def test_fresh_database_has_the_table_columns_and_indexes(self):
+        conn = self.conn()
+        self.assertIsNone(wd.workflow_documents_schema_error(conn, allow_missing=False))
+        columns = {
+            row[1] for row in conn.execute("PRAGMA table_info(generated_images)")
+        }
+        self.assertTrue(
+            {"workflow_document_id", "prompt_document_id"}.issubset(columns)
+        )
+        indexes = {
+            row[1] for row in conn.execute("PRAGMA index_list(generated_images)")
+        }
+        self.assertIn("idx_image_workflow_document", indexes)
+        self.assertIn("idx_image_prompt_document", indexes)
+
+    def test_no_trigger_is_installed(self):
+        # Every restore validator refuses triggers; GC is done in code.
+        row = (
+            self.conn()
+            .execute("SELECT name FROM sqlite_master WHERE type='trigger'")
+            .fetchone()
+        )
+        self.assertIsNone(row)
+
+    def test_current_database_verifies_and_restores(self):
+        image = self.db.link_image_to_prompt(
+            self.prompt(), os.path.join(self.tmpdir, "a.png"), {"workflow": {"n": 1}}
+        )
+        self.assertTrue(image)
+        ok, reason = self.db.model.verify_database_file(
+            self.db_path, require_library_state=True
+        )
+        self.assertTrue(ok, reason)
+
+    def test_an_old_database_opens_migrates_and_keeps_its_inline_rows(self):
+        PromptModel.close_all_instances()
+        PromptModel.reset_schema_cache()
+        old_path = os.path.join(self.tmpdir, "old.db")
+        conn = sqlite3.connect(old_path)
+        try:
+            conn.executescript(
+                "CREATE TABLE prompts (id INTEGER PRIMARY KEY AUTOINCREMENT, "
+                "text TEXT NOT NULL, created_at TIMESTAMP, updated_at TIMESTAMP, "
+                "category TEXT, tags TEXT, rating INTEGER, notes TEXT, "
+                "hash TEXT UNIQUE);"
+                "CREATE TABLE generated_images (id INTEGER PRIMARY KEY "
+                "AUTOINCREMENT, prompt_id INTEGER NOT NULL, image_path TEXT NOT "
+                "NULL, filename TEXT NOT NULL, generation_time TIMESTAMP, "
+                "file_size INTEGER, width INTEGER, height INTEGER, format TEXT, "
+                "workflow_data TEXT, prompt_metadata TEXT, parameters TEXT, "
+                "file_path TEXT, FOREIGN KEY (prompt_id) REFERENCES prompts(id) "
+                "ON DELETE CASCADE, UNIQUE(prompt_id, file_path));"
+                "INSERT INTO prompts (id, text, hash) VALUES (1, 'a cat', 'h');"
+                "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                "file_path, workflow_data, prompt_metadata) VALUES (1, '/x/a.png', "
+                "'a.png', '/x/a.png', '{\"n\": NaN}', '{\"3\": {}}');"
+            )
+            conn.commit()
+        finally:
+            conn.close()
+        ok, reason = PromptModel.verify_database_file(old_path)
+        self.assertTrue(ok, reason)
+        old = PromptDatabase(old_path)
+        [image] = old.get_prompt_images(1)
+        self.assertEqual(image["workflow_data"], {"n": None})
+        self.assertEqual(image["prompt_metadata"], {"3": {}})
+        row = (
+            old.model.get_connection()
+            .execute("SELECT workflow_data, workflow_document_id FROM generated_images")
+            .fetchone()
+        )
+        self.assertEqual(tuple(row), ('{"n": NaN}', None))
+        old.close_all()
+
+    def test_verify_refuses_a_malformed_documents_table(self):
+        bad = os.path.join(self.tmpdir, "bad.db")
+        PromptModel.close_all_instances()
+        conn = sqlite3.connect(self.db_path)
+        try:
+            with sqlite3.connect(bad) as target:
+                conn.backup(target)
+            target.close()
+        finally:
+            conn.close()
+        cases = {
+            "table": "DROP TABLE workflow_documents; CREATE TABLE "
+            "workflow_documents (id INTEGER PRIMARY KEY, sha256 TEXT, data TEXT)",
+            "values": "INSERT INTO workflow_documents (sha256, byte_length, data) "
+            "VALUES (zeroblob(32), -1, x'00')",
+            "reference": "PRAGMA foreign_keys=OFF; INSERT INTO prompts (text, hash) "
+            "VALUES ('p', 'h2'); INSERT INTO generated_images (prompt_id, image_path,"
+            " filename, workflow_document_id) VALUES (last_insert_rowid(), '/a', 'a',"
+            " 99)",
+            "index": "DROP INDEX idx_image_workflow_document",
+        }
+        for name, script in cases.items():
+            copy = os.path.join(self.tmpdir, f"bad-{name}.db")
+            with open(bad, "rb") as source, open(copy, "wb") as target:
+                target.write(source.read())
+            conn = sqlite3.connect(copy)
+            try:
+                conn.executescript(script)
+                conn.commit()
+            finally:
+                conn.close()
+            with self.subTest(name=name):
+                ok, reason = PromptModel.verify_database_file(copy)
+                self.assertFalse(ok)
+                self.assertIn("orkflow", reason)
+
+
+class TestSchemaErrors(unittest.TestCase):
+    """Each way a restored file's document schema can be wrong is named."""
+
+    IMAGES = (
+        "CREATE TABLE generated_images (id INTEGER PRIMARY KEY, "
+        "workflow_data TEXT, prompt_metadata TEXT{extra})"
+    )
+    TABLE = (
+        "CREATE TABLE workflow_documents (id INTEGER PRIMARY KEY, sha256 BLOB NOT "
+        "NULL UNIQUE, byte_length INTEGER NOT NULL, data BLOB NOT NULL{extra})"
+    )
+    REFS = (
+        ", workflow_document_id {kind}, prompt_document_id "
+        "INTEGER REFERENCES workflow_documents(id)"
+    )
+    GOOD_REF = "INTEGER REFERENCES workflow_documents(id)"
+
+    def schema(self, images_extra="", table_extra="", indexes=True, table=True):
+        conn = sqlite3.connect(":memory:")
+        self.addCleanup(conn.close)
+        conn.execute(self.IMAGES.format(extra=images_extra))
+        if table:
+            conn.execute(self.TABLE.format(extra=table_extra))
+        if indexes and images_extra:
+            for column, index in wd.INDEXES.items():
+                conn.execute(f"CREATE INDEX {index} ON generated_images({column})")
+        return conn
+
+    def test_each_error_is_named(self):
+        refs = self.REFS.format(kind=self.GOOD_REF)
+        cases = {
+            "Missing workflow document schema": self.schema(table=False),
+            "Incomplete workflow document schema": self.schema(),
+            "Workflow documents have an invalid table definition": self.schema(
+                refs, table_extra=", extra TEXT"
+            ),
+            "Workflow document references have an invalid column": self.schema(
+                self.REFS.format(kind="TEXT REFERENCES workflow_documents(id)")
+            ),
+            "Workflow document references have invalid foreign keys": self.schema(
+                self.REFS.format(kind="INTEGER")
+            ),
+            "Workflow document references are missing their index": self.schema(
+                refs, indexes=False
+            ),
+        }
+        for message, conn in cases.items():
+            with self.subTest(message=message):
+                self.assertEqual(
+                    wd.workflow_documents_schema_error(conn, allow_missing=False),
+                    message,
+                )
+        self.assertIsNone(wd.workflow_documents_schema_error(self.schema(table=False)))
+        self.assertIsNone(
+            wd.workflow_documents_schema_error(self.schema(refs), allow_missing=False)
+        )
+
+    def test_an_index_on_another_table_is_not_the_index(self):
+        conn = self.schema(self.REFS.format(kind=self.GOOD_REF), indexes=False)
+        conn.execute("CREATE TABLE other (workflow_document_id INTEGER)")
+        conn.execute(
+            "CREATE INDEX idx_image_workflow_document ON other(workflow_document_id)"
+        )
+        self.assertEqual(
+            wd.workflow_documents_schema_error(conn),
+            "Workflow document references are missing their index",
+        )
+
+    def test_migrate_refuses_a_bad_schema_and_leaves_it_alone(self):
+        conn = self.schema(table_extra="")  # the table without the columns
+        with self.assertRaisesRegex(sqlite3.DatabaseError, "Incomplete"):
+            wd.migrate_workflow_documents(conn)
+        columns = {
+            row[1] for row in conn.execute("PRAGMA table_info(generated_images)")
+        }
+        self.assertNotIn("workflow_document_id", columns)
+
+    def test_migrate_validates_what_it_installed(self):
+        conn = self.schema(table=False)
+        with patch.object(
+            wd,
+            "workflow_documents_schema_error",
+            side_effect=[None, "Workflow documents broke"],
+        ):
+            with self.assertRaisesRegex(sqlite3.DatabaseError, "broke"):
+                wd.migrate_workflow_documents(conn)
+        tables = {row[0] for row in conn.execute("SELECT name FROM sqlite_master")}
+        self.assertNotIn("workflow_documents", tables)
+
+
+class TestEdgeErrors(DocumentsTestCase):
+    def test_store_refuses_values_that_cannot_be_documents(self):
+        conn = self.conn()
+        with self.assertRaises(TypeError):
+            wd.store_text(conn, b"bytes")
+        with patch.object(wd, "MAX_DOCUMENT_BYTES", 8):
+            with self.assertRaises(ValueError):
+                wd.store_text(conn, "123456789")
+            self.assertFalse(wd.storable("12345678é"))
+            self.assertTrue(wd.storable("1234567"))
+            self.assertEqual(
+                wd.stored_values(conn, {"workflow_data": "123456789"}),
+                {"workflow_data": "123456789", "workflow_document_id": None},
+            )
+        conn.rollback()
+
+    def test_a_hash_match_with_another_length_is_a_mismatch(self):
+        document_id = self.store("abc")
+        conn = self.conn()
+        with conn:
+            conn.execute(
+                "UPDATE workflow_documents SET byte_length=5 WHERE id=?",
+                (document_id,),
+            )
+        with conn, self.assertLogs(wd.logger, "WARNING"):  # repaired from the text
+            self.assertEqual(wd.store_text(conn, "abc"), document_id)
+        self.assertEqual(wd.read_stored_text(conn, None, document_id), "abc")
+        for length in (-1, 2, 4):
+            with conn:
+                conn.execute(
+                    "UPDATE workflow_documents SET byte_length=? WHERE id=?",
+                    (length, document_id),
+                )
+            with self.subTest(length=length), self.assertLogs(wd.logger, "WARNING"):
+                self.assertIsNone(wd.read_stored_text(conn, None, document_id))
+
+    def test_the_reader_memo_is_bounded(self):
+        ids = [self.store(f"text {index}") for index in range(40)]
+        reader = wd.DocumentReader(self.conn())
+        for index, document_id in enumerate(ids):
+            self.assertEqual(reader.text(None, document_id), f"text {index}")
+        self.assertEqual(len(reader._memo), wd.DocumentReader.MEMO)
+
+    def test_a_row_without_text_columns_is_left_alone(self):
+        reader = wd.DocumentReader(self.conn())
+        self.assertEqual(wd.resolve_image_row(reader, {"id": 1}), {"id": 1})
+        self.assertEqual(
+            wd.resolve_image_row(reader, {"workflow_document_id": None}),
+            {"workflow_data": None},
+        )
+
+    def test_documents_of_rows_takes_only_known_columns(self):
+        with self.assertRaises(ValueError):
+            wd.documents_of_rows(self.conn(), "image_path", ["x"])
+
+
+class TestWritersStoreNewForm(DocumentsTestCase):
+    def test_link_image_stores_documents_not_inline_text(self):
+        workflow = sample_workflow(1)
+        graph = sample_prompt_graph(1)
+        prompt_id = self.prompt()
+        first = self.db.link_image_to_prompt(
+            prompt_id,
+            os.path.join(self.tmpdir, "a.png"),
+            {"workflow": workflow, "prompt": graph},
+        )
+        second = self.db.link_image_to_prompt(
+            prompt_id,
+            os.path.join(self.tmpdir, "b.png"),
+            {"workflow": workflow, "prompt": graph},
+        )
+        a, b = self.raw(first), self.raw(second)
+        self.assertIsNone(a["workflow_data"])
+        self.assertIsNone(a["prompt_metadata"])
+        self.assertIsNotNone(a["workflow_document_id"])
+        self.assertEqual(a["workflow_document_id"], b["workflow_document_id"])
+        self.assertEqual(a["prompt_document_id"], b["prompt_document_id"])
+        self.assertEqual(self.document_count(), 2)
+        conn = self.conn()
+        self.assertEqual(
+            wd.read_stored_text(conn, None, a["workflow_document_id"]),
+            dumps_strict(workflow),
+        )
+
+    def test_readers_answer_the_same_for_both_forms(self):
+        workflow, graph = sample_workflow(4), sample_prompt_graph(4)
+        workflow["nodes"][0]["widgets_values"] = [NAN, INF]
+        prompt_id = self.prompt()
+        new_id = self.db.link_image_to_prompt(
+            prompt_id,
+            os.path.join(self.tmpdir, "new.png"),
+            {"workflow": workflow, "prompt": graph},
+        )
+        old_id = self.old_form_image(
+            prompt_id, "old.png", dumps_strict(workflow), dumps_strict(graph)
+        )
+        new, old = self.db.get_image_by_id(new_id), self.db.get_image_by_id(old_id)
+        for key in ("workflow_data", "prompt_metadata", "parameters"):
+            self.assertEqual(new[key], old[key])
+        self.assertEqual(new["workflow_data"]["nodes"][0]["widgets_values"], [None] * 2)
+        self.assertNotIn("workflow_document_id", new)
+        self.assertNotIn("prompt_document_id", old)
+        listed = {
+            image["id"]: image
+            for reader in (
+                lambda: self.db.get_prompt_images(prompt_id),
+                lambda: self.db.get_recent_images(10),
+                lambda: self.db.get_all_images(),
+                lambda: self.db.search_images_by_prompt("cat"),
+            )
+            for image in reader()
+        }
+        self.assertEqual(listed[new_id]["workflow_data"], new["workflow_data"])
+        self.assertEqual(listed[old_id]["prompt_metadata"], graph)
+        info_new = self.db.get_image_prompt_info(os.path.join(self.tmpdir, "new.png"))
+        info_old = self.db.get_image_prompt_info(os.path.join(self.tmpdir, "old.png"))
+        self.assertEqual(info_new["workflow_data"], info_old["workflow_data"])
+        self.assertEqual(info_new["prompt_metadata"], graph)
+
+    def test_diagnostics_recent_images_resolve_documents(self):
+        prompt_id = self.prompt()
+        self.db.link_image_to_prompt(
+            prompt_id,
+            os.path.join(self.tmpdir, "d.png"),
+            {"workflow": {"n": 1}, "prompt": {"3": {}}},
+        )
+        from utils import diagnostics
+
+        tool = diagnostics.GalleryDiagnostics(self.db_path)
+        result = tool.check_images_table()
+        tool_images = result["recent_images"]
+        self.assertEqual(tool_images[0]["workflow_data"], '{"n": 1}')
+        self.assertNotIn("workflow_document_id", tool_images[0])
+
+
+class TestGarbage(DocumentsTestCase):
+    """A document no row references is removed with its last reference."""
+
+    def test_delete_image_releases_only_unshared_documents(self):
+        prompt_id = self.prompt()
+        shared_a = self.linked(prompt_id, "a.png", 1)
+        self.linked(prompt_id, "b.png", 1)
+        alone = self.linked(prompt_id, "c.png", 2)
+        self.assertEqual(self.document_count(), 4)
+        self.db.delete_image(shared_a)
+        self.assertEqual(self.document_count(), 4)
+        self.db.delete_image(alone)
+        self.assertEqual(self.document_count(), 2)
+
+    def test_deleting_prompts_releases_their_images_documents(self):
+        first, second = self.prompt("one"), self.prompt("two")
+        self.linked(first, "a.png", 1)
+        self.linked(second, "b.png", 2)
+        self.db.delete_prompt(first)
+        self.assertEqual(self.document_count(), 2)
+        self.db.update_prompt_metadata(second, category="gone")
+        self.assertEqual(self.db.delete_prompts_by_category("gone"), 1)
+        self.assertEqual(self.document_count(), 0)
+
+    def test_cleanup_missing_images_releases_documents(self):
+        prompt_id = self.prompt()
+        self.linked(prompt_id, "gone.png", 1)
+        self.db.cleanup_missing_images()
+        self.assertEqual(self.document_count(), 0)
+        self.linked(prompt_id, "gone-too.png", 2)
+        self.assertEqual(self.db.cleanup_missing_images(roots=[self.tmpdir]), 1)
+        self.assertEqual(self.document_count(), 0)
+
+    def test_sweep_removes_what_a_path_without_release_left(self):
+        prompt_id = self.prompt()
+        image = self.linked(prompt_id, "a.png", 1)
+        conn = self.conn()
+        with conn:
+            conn.execute("DELETE FROM generated_images WHERE id=?", (image,))
+        self.assertEqual(self.document_count(), 2)
+        with conn:
+            self.assertEqual(wd.sweep_documents(conn), 2)
+        self.assertEqual(self.document_count(), 0)
+
+    def test_startup_sweeps_unreferenced_documents(self):
+        prompt_id = self.prompt()
+        image = self.linked(prompt_id, "a.png", 1)
+        conn = self.conn()
+        with conn:
+            conn.execute("DELETE FROM generated_images WHERE id=?", (image,))
+        PromptModel.close_all_instances()
+        PromptModel.reset_schema_cache()
+        self.db = PromptDatabase(self.db_path)
+        self.assertEqual(self.document_count(), 0)
+
+    def test_release_keeps_a_document_the_other_column_references(self):
+        prompt_id = self.prompt()
+        text = '{"same": true}'
+        first = self.db.link_image_to_prompt(
+            prompt_id, os.path.join(self.tmpdir, "a.png"), {"workflow": {"same": True}}
+        )
+        second = self.db.link_image_to_prompt(
+            prompt_id, os.path.join(self.tmpdir, "b.png"), {"prompt": {"same": True}}
+        )
+        self.assertEqual(
+            self.raw(first)["workflow_document_id"],
+            self.raw(second)["prompt_document_id"],
+        )
+        self.db.delete_image(first)
+        self.assertEqual(
+            wd.read_stored_text(
+                self.conn(), None, self.raw(second)["prompt_document_id"]
+            ),
+            text,
+        )
+
+
+class TestMoreGarbagePaths(DocumentsTestCase):
+    def test_bulk_delete_prompts_releases_documents(self):
+        first, second = self.prompt("one"), self.prompt("two")
+        self.linked(first, "a.png", 1)
+        self.linked(second, "b.png", 2)
+        self.assertEqual(self.db.bulk_delete_prompts([first, second]), 2)
+        self.assertEqual(self.document_count(), 0)
+
+    def test_duplicate_prompt_merge_releases_images_it_could_not_move(self):
+        primary = self.prompt("A cat")
+        copy = self.db.save_prompt("a cat ", prompt_hash="other-hash")
+        self.linked(primary, "same.png", 1)
+        self.linked(copy, "same.png", 2)  # collides with the primary's link
+        self.assertEqual(self.document_count(), 4)
+        self.db.cleanup_duplicates()
+        self.assertEqual(self.document_count(), 2)
+
+    def test_maintenance_delete_ids_releases_documents(self):
+        from database.maintenance import _delete_ids
+
+        prompt_id = self.prompt()
+        image = self.linked(prompt_id, "a.png", 1)
+        other = self.prompt("other")
+        self.linked(other, "b.png", 2)
+        conn = self.conn()
+        with conn:
+            _delete_ids(conn, "generated_images", [image])
+        self.assertEqual(self.document_count(), 2)
+        with conn:
+            _delete_ids(conn, "prompts", [other])
+        self.assertEqual(self.document_count(), 0)
+
+    def test_failed_insert_of_a_known_link_leaves_no_new_document(self):
+        prompt_id = self.prompt()
+        path = os.path.join(self.tmpdir, "a.png")
+        self.db.link_image_to_prompt(prompt_id, path, {"workflow": {"v": 1}})
+        before = self.document_count()
+        self.assertEqual(
+            self.db.link_image_to_prompt(prompt_id, path, {"workflow": {"v": 2}}), 0
+        )
+        self.assertEqual(self.document_count(), before)
+
+
+class TestPairIngestionDocuments(DocumentsTestCase):
+    def resolution(self):
+        return {
+            "state": "resolved",
+            "positive": {"text": "cat", "provenance": {"source": "api"}},
+            "negative": {"text": "blur", "provenance": {"source": "api"}},
+        }
+
+    def test_ingestion_stores_new_form_and_replaces_documents_it_updates(self):
+        from database.pair_ingestion import ingest_resolved_pair
+
+        path = os.path.join(self.tmpdir, "scan.png")
+        first = {"workflow": sample_workflow(1), "prompt": sample_prompt_graph(1)}
+        added = ingest_resolved_pair(self.db, path, first, self.resolution())
+        self.assertEqual(added["status"], "added")
+        row = self.raw(added["image_id"])
+        self.assertIsNone(row["workflow_data"])
+        self.assertIsNotNone(row["workflow_document_id"])
+        again = ingest_resolved_pair(self.db, path, first, self.resolution())
+        self.assertEqual(again["status"], "unchanged")
+        second = {"workflow": sample_workflow(2), "prompt": sample_prompt_graph(1)}
+        updated = ingest_resolved_pair(self.db, path, second, self.resolution())
+        self.assertEqual(updated["status"], "enriched")
+        self.assertEqual(self.document_count(), 2)  # the first workflow went
+        image = self.db.get_image_by_id(added["image_id"])
+        self.assertEqual(image["workflow_data"], sample_workflow(2))
+
+    def test_an_old_form_row_with_the_same_text_is_unchanged(self):
+        from database.pair_ingestion import ingest_resolved_pair
+
+        path = os.path.join(self.tmpdir, "old.png")
+        metadata = {"workflow": sample_workflow(3), "prompt": sample_prompt_graph(3)}
+        added = ingest_resolved_pair(self.db, path, metadata, self.resolution())
+        conn = self.conn()
+        with conn:  # rewrite it as an older version stored it
+            conn.execute(
+                "UPDATE generated_images SET workflow_data=?, prompt_metadata=?, "
+                "workflow_document_id=NULL, prompt_document_id=NULL WHERE id=?",
+                (
+                    dumps_strict(metadata["workflow"], sort_keys=True),
+                    dumps_strict(metadata["prompt"], sort_keys=True),
+                    added["image_id"],
+                ),
+            )
+            wd.sweep_documents(conn)
+        again = ingest_resolved_pair(self.db, path, metadata, self.resolution())
+        self.assertEqual(again["status"], "unchanged")
+        self.assertEqual(self.document_count(), 0)
+
+
+class TestQueryPlans(DocumentsTestCase):
+    """Every statement the store and the migration run is an index search."""
+
+    def plan(self, sql, params=()):
+        rows = self.conn().execute("EXPLAIN QUERY PLAN " + sql, params).fetchall()
+        return " | ".join(row[3] for row in rows)
+
+    def test_statements_use_indexes(self):
+        for name, (sql, params) in wd.QUERY_PLAN_STATEMENTS.items():
+            with self.subTest(name=name):
+                plan = self.plan(sql, params)
+                self.assertNotRegex(plan, r"\bSCAN (generated_images|gi)\b(?! USING)")
+                self.assertTrue("USING" in plan or "SEARCH" in plan, f"{name}: {plan}")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_workflow_migration.py

@@ -0,0 +1,469 @@
+"""The workflow storage migration (database/workflow_storage.py).
+
+Rewrites old-form rows (text inline) to the new form (a document id) in
+batches. Each batch is one transaction, so a crash leaves every row in one
+form or the other and readable either way; a rerun continues where the last
+committed batch stopped. Rows are read one at a time, the stored text comes
+back byte for byte, and no image or video file is ever opened.
+"""
+
+import builtins
+import hashlib
+import json
+import os
+import random
+import sqlite3
+import tracemalloc
+import unittest
+from unittest.mock import patch
+
+from PIL import Image, PngImagePlugin
+
+from database import workflow_documents as wd
+from database import workflow_storage as ws
+from database.models import PromptModel
+
+try:
+    from tests.test_workflow_documents import (
+        DocumentsTestCase,
+        edge_texts,
+        random_text,
+        sample_prompt_graph,
+        sample_workflow,
+    )
+except ImportError:  # discovered with tests/ as the top-level directory
+    from test_workflow_documents import (
+        DocumentsTestCase,
+        edge_texts,
+        random_text,
+        sample_prompt_graph,
+        sample_workflow,
+    )
+
+
+def _digest(path):
+    if not os.path.exists(path):
+        return None
+    with open(path, "rb") as handle:
+        return hashlib.sha256(handle.read()).hexdigest()
+
+
+class MigrationTestCase(DocumentsTestCase):
+    def library(self, count=40, seed=11):
+        """Old-form rows: duplicated runs, edge texts and random text."""
+        rng = random.Random(seed)
+        prompt_id = self.prompt()
+        texts = edge_texts()
+        expected = {}
+        for index in range(count):
+            run = index // 4  # four images per run share their texts
+            if index % 10 == 9:
+                workflow = texts[index % len(texts)]
+                graph = random_text(rng)
+            else:
+                workflow = json.dumps(sample_workflow(run, nodes=20))
+                graph = json.dumps(sample_prompt_graph(run))
+            if index % 13 == 5:
+                graph = None
+            image_id = self.old_form_image(prompt_id, f"{index}.png", workflow, graph)
+            expected[image_id] = (workflow, graph)
+        return prompt_id, expected
+
+    def texts(self):
+        conn = self.conn()
+        reader = wd.DocumentReader(conn)
+        rows = conn.execute(
+            "SELECT id, workflow_data, prompt_metadata, workflow_document_id, "
+            "prompt_document_id FROM generated_images ORDER BY id"
+        )
+        result = {}
+        for row in rows:
+            data = wd.resolve_image_row(reader, row)
+            result[row["id"]] = (data["workflow_data"], data["prompt_metadata"])
+        return result
+
+    def inline_ids(self):
+        return {
+            row[0]
+            for row in self.conn().execute(
+                "SELECT id FROM generated_images WHERE workflow_data IS NOT NULL "
+                "OR prompt_metadata IS NOT NULL"
+            )
+        }
+
+    def payloads(self, prompt_id):
+        images = sorted(self.db.get_prompt_images(prompt_id), key=lambda i: i["id"])
+        for image in images:
+            image.pop("generation_time", None)
+        return images
+
+    def migrate(self, **kwargs):
+        kwargs.setdefault("max_rows", 7)
+        return self.db.migrate_workflow_storage(**kwargs)
+
+
+class TestMigration(MigrationTestCase):
+    def test_every_row_is_rewritten_and_reads_back_identical(self):
+        prompt_id, expected = self.library()
+        before = self.payloads(prompt_id)
+        result = self.migrate()
+        self.assertEqual(self.texts(), expected)
+        self.assertEqual(self.inline_ids(), set())
+        self.assertEqual(self.payloads(prompt_id), before)
+        distinct = {t for pair in expected.values() for t in pair if t is not None}
+        self.assertEqual(self.document_count(), len(distinct))
+        self.assertEqual(result["rows"], len(expected))
+        self.assertGreater(result["batches"], 1)
+        self.assertFalse(result["stopped"])
+
+    def test_a_second_run_changes_nothing(self):
+        _prompt_id, expected = self.library(count=12)
+        self.migrate()
+        conn = self.conn()
+        documents = list(conn.execute("SELECT * FROM workflow_documents ORDER BY id"))
+        result = self.migrate()
+        self.assertEqual(result["rows"], 0)
+        self.assertEqual(
+            [tuple(r) for r in documents],
+            [tuple(r) for r in conn.execute("SELECT * FROM workflow_documents")],
+        )
+        self.assertEqual(self.texts(), expected)
+
+    def test_a_crash_mid_migration_leaves_every_row_readable_and_resumes(self):
+        prompt_id, expected = self.library()
+        before = self.payloads(prompt_id)
+        real = wd.store_text
+        calls = []
+
+        def crashing(conn, text, *args):
+            calls.append(text)
+            if len(calls) == 23:
+                raise RuntimeError("power cut")
+            return real(conn, text, *args)
+
+        with patch.object(ws, "store_text", side_effect=crashing):
+            with self.assertRaises(RuntimeError):
+                self.migrate()
+        self.assertFalse(self.conn().in_transaction)
+        self.assertEqual(self.texts(), expected)
+        self.assertEqual(self.payloads(prompt_id), before)
+        remaining = self.inline_ids()
+        self.assertTrue(0 < len(remaining) < len(expected))
+        # The failed batch rolled back whole: what is left is a suffix by id.
+        self.assertEqual(remaining, {i for i in expected if i >= min(remaining)})
+        self.migrate()
+        self.assertEqual(self.inline_ids(), set())
+        self.assertEqual(self.texts(), expected)
+
+    def test_a_crash_inside_a_batch_writes_nothing_of_that_batch(self):
+        _prompt_id, expected = self.library(count=10)
+        with patch.object(ws, "store_text", side_effect=RuntimeError("full")):
+            with self.assertRaises(RuntimeError):
+                self.migrate(max_rows=50)
+        self.assertEqual(self.inline_ids(), set(expected))
+        self.assertEqual(self.document_count(), 0)
+
+    def test_a_document_that_reads_back_differently_keeps_its_row(self):
+        _prompt_id, expected = self.library(count=6)
+        with patch.object(ws, "read_stored_text", return_value="something else"):
+            result = self.migrate(max_rows=50)
+        self.assertEqual(result["errors"], sorted(expected))
+        self.assertEqual(self.inline_ids(), set(expected))
+        self.assertEqual(self.texts(), expected)
+        self.assertEqual(self.document_count(), 0)
+
+    def test_stop_between_batches_then_continue(self):
+        _prompt_id, expected = self.library(count=30)
+        seen = []
+        result = self.migrate(on_batch=seen.append, should_stop=lambda: len(seen) >= 2)
+        self.assertTrue(result["stopped"])
+        self.assertEqual(result["rows"], 14)
+        self.assertEqual(len(self.inline_ids()), len(expected) - 14)
+        self.assertEqual(seen[-1]["rows"], 14)
+        self.migrate()
+        self.assertEqual(self.inline_ids(), set())
+        self.assertEqual(self.texts(), expected)
+
+    def test_values_that_are_not_text_stay_inline(self):
+        # A TEXT column turns numbers into text; only a blob stays other.
+        prompt_id = self.prompt()
+        blob = self.old_form_image(prompt_id, "blob.png", b"\x00raw", "{}")
+        other = self.old_form_image(prompt_id, "other.png", "42", b"\x01")
+        result = self.migrate()
+        self.assertEqual(result["protected"], 2)
+        conn = self.conn()
+        rows = {
+            row[0]: tuple(row[1:])
+            for row in conn.execute(
+                "SELECT id, workflow_data, prompt_metadata, workflow_document_id, "
+                "prompt_document_id FROM generated_images"
+            )
+        }
+        self.assertEqual(rows[blob][:2], (b"\x00raw", None))
+        self.assertIsNone(rows[blob][2])
+        self.assertIsNotNone(rows[blob][3])
+        self.assertEqual(rows[other][:2], (None, b"\x01"))
+        self.assertEqual(self.texts()[other], ("42", b"\x01"))
+        self.assertEqual(self.migrate()["protected"], 2)  # still there, still kept
+
+    def test_a_row_an_older_version_rewrote_inline_takes_the_inline_text(self):
+        prompt_id = self.prompt()
+        image = self.db.link_image_to_prompt(
+            prompt_id, os.path.join(self.tmpdir, "a.png"), {"workflow": {"v": 1}}
+        )
+        conn = self.conn()
+        with conn:  # an older version updates the inline column only
+            conn.execute(
+                "UPDATE generated_images SET workflow_data=? WHERE id=?",
+                ('{"v": 2}', image),
+            )
+        self.assertEqual(self.texts()[image][0], '{"v": 2}')
+        self.migrate()
+        self.assertEqual(self.texts()[image][0], '{"v": 2}')
+        self.assertEqual(self.inline_ids(), set())
+        stored = {row[0] for row in conn.execute("SELECT data FROM workflow_documents")}
+        self.assertNotIn(b'{"v": 1}', {bytes(wd_data) for wd_data in stored})
+        texts = [
+            wd.read_stored_text(conn, None, row[0])
+            for row in conn.execute("SELECT id FROM workflow_documents")
+        ]
+        self.assertNotIn('{"v": 1}', texts)
+
+    def test_new_writes_during_the_migration_are_kept(self):
+        prompt_id, expected = self.library(count=20)
+        added = []
+
+        def between(_progress):
+            if not added:
+                added.append(
+                    self.db.link_image_to_prompt(
+                        prompt_id,
+                        os.path.join(self.tmpdir, "new.png"),
+                        {"workflow": {"new": True}},
+                    )
+                )
+                added.append(
+                    self.old_form_image(prompt_id, "late.png", '{"late": 1}', None)
+                )
+
+        self.migrate(on_batch=between)
+        texts = self.texts()
+        self.assertEqual(texts[added[0]][0], '{"new": true}')
+        self.assertEqual(texts[added[1]], ('{"late": 1}', None))
+        self.assertEqual(self.inline_ids(), set())
+        for image_id, pair in expected.items():
+            self.assertEqual(texts[image_id], pair)
+
+    def test_memory_stays_bounded_by_one_row(self):
+        prompt_id = self.prompt()
+        big = json.dumps(sample_workflow(1, nodes=3000))  # about 0.9 MB
+        for index in range(40):
+            self.old_form_image(
+                prompt_id, f"{index}.png", big[:-1] + f', "n": {index}}}', None
+            )
+        tracemalloc.start()
+        try:
+            self.migrate(max_rows=200, max_bytes=4 * 2**20)
+            _current, peak = tracemalloc.get_traced_memory()
+        finally:
+            tracemalloc.stop()
+        self.assertLess(peak, 12 * 2**20, f"peak {peak / 2**20:.1f} MB")
+        self.assertEqual(self.inline_ids(), set())
+
+    def test_batches_are_bounded_by_bytes(self):
+        prompt_id = self.prompt()
+        text = "x" * 100_000
+        for index in range(10):
+            self.old_form_image(prompt_id, f"{index}.png", text + str(index), None)
+        seen = []
+        self.migrate(max_rows=200, max_bytes=250_000, on_batch=seen.append)
+        self.assertEqual([batch["rows"] for batch in seen], [3, 6, 9, 10])
+
+    def test_unreferenced_documents_are_swept_at_the_end(self):
+        prompt_id = self.prompt()
+        image = self.db.link_image_to_prompt(
+            prompt_id, os.path.join(self.tmpdir, "a.png"), {"workflow": {"v": 1}}
+        )
+        conn = self.conn()
+        with conn:
+            conn.execute("DELETE FROM generated_images WHERE id=?", (image,))
+        result = self.migrate()
+        self.assertEqual(result["documents_removed"], 2)
+        self.assertEqual(self.document_count(), 0)
+
+    def test_no_media_file_is_opened_or_changed(self):
+        media = os.path.join(self.tmpdir, "output")
+        os.makedirs(media)
+        prompt_id = self.prompt()
+        for index in range(5):
+            path = os.path.join(media, f"{index}.png")
+            info = PngImagePlugin.PngInfo()
+            info.add_text("workflow", json.dumps(sample_workflow(index)))
+            with Image.new("RGB", (4, 4)) as image:
+                image.save(path, pnginfo=info)
+            conn = self.conn()
+            with conn:
+                conn.execute(
+                    "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                    "file_path, workflow_data) VALUES (?, ?, ?, ?, ?)",
+                    (prompt_id, path, f"{index}.png", path, '{"n": 1}'),
+                )
+        before = {
+            name: (
+                _digest(os.path.join(media, name)),
+                os.stat(os.path.join(media, name)),
+            )
+            for name in os.listdir(media)
+        }
+        opened = []
+        real_open, real_os_open = builtins.open, os.open
+
+        def tracking_open(file, *args, **kwargs):
+            opened.append(str(file))
+            return real_open(file, *args, **kwargs)
+
+        def tracking_os_open(path, *args, **kwargs):
+            opened.append(str(path))
+            return real_os_open(path, *args, **kwargs)
+
+        with patch("builtins.open", tracking_open), patch("os.open", tracking_os_open):
+            with patch.object(Image, "open", side_effect=AssertionError("opened")):
+                self.migrate()
+        self.assertEqual([p for p in opened if p.startswith(media)], [])
+        for name, (digest, stat) in before.items():
+            now = os.stat(os.path.join(media, name))
+            self.assertEqual(_digest(os.path.join(media, name)), digest)
+            self.assertEqual(now.st_mtime_ns, stat.st_mtime_ns)
+        self.assertEqual(self.inline_ids(), set())
+
+
+class TestPreview(MigrationTestCase):
+    def test_preview_counts_and_writes_nothing(self):
+        prompt_id, expected = self.library(count=20)
+        self.old_form_image(prompt_id, "blob.png", b"raw", None)
+        files = [self.db_path + suffix for suffix in ("", "-wal")]
+        before = [_digest(path) for path in files]
+        draft = self.db.preview_maintenance("workflow_storage")
+        self.assertEqual([_digest(path) for path in files], before)
+        self.assertEqual(draft.operation, "workflow_storage")
+        self.assertEqual(draft.scope["count"], len(expected))
+        self.assertEqual(draft.protected["count"], 1)
+        self.assertFalse(draft.media["deletes_files"])
+        self.assertIn("No image or video file", draft.media["label"])
+        labels = [row["label"] for row in draft.details]
+        self.assertIn("Database file", labels)
+        self.assertIn("Older versions", labels)
+        self.assertIn("Reclaim space", labels)
+
+    def test_preview_without_octet_length_omits_the_inline_bytes(self):
+        # SQLite before 3.43 (older Windows Python builds) has no octet_length.
+        self.library(count=4)
+        with patch.object(ws.sqlite3, "sqlite_version_info", (3, 37, 2)):
+            draft = self.db.preview_maintenance("workflow_storage")
+        labels = [row["label"] for row in draft.details]
+        self.assertNotIn("Inline text", labels)
+        self.assertEqual(draft.scope["count"], 4)
+
+    def test_preview_of_a_migrated_library_has_nothing_to_do(self):
+        self.library(count=5)
+        self.migrate()
+        draft = self.db.preview_maintenance("workflow_storage")
+        self.assertEqual(draft.scope["count"], 0)
+
+    def test_the_batch_reads_are_index_searches(self):
+        conn = self.conn()
+        for name, (sql, params) in ws.QUERY_PLAN_STATEMENTS.items():
+            plan = " | ".join(
+                row[3] for row in conn.execute("EXPLAIN QUERY PLAN " + sql, params)
+            )
+            with self.subTest(name=name):
+                self.assertIn("USING INTEGER PRIMARY KEY", plan, plan)
+
+
+class TestReclaimSpace(MigrationTestCase):
+    def test_reclaim_space_gives_the_freed_pages_back(self):
+        prompt_id = self.prompt()
+        big = json.dumps(sample_workflow(1, nodes=600))
+        for index in range(60):
+            self.old_form_image(prompt_id, f"{index}.png", big, big)
+        expected = self.texts()
+        self.migrate()
+        conn = self.conn()
+        with conn:  # a document a path forgot to release
+            conn.execute(
+                "INSERT INTO workflow_documents (sha256, byte_length, data) "
+                "VALUES (zeroblob(32), 0, x'789c030000000001')"
+            )
+        result = self.db.reclaim_space()
+        self.assertEqual(result["documents_removed"], 1)
+        self.assertFalse(result["checkpoint_busy"])
+        self.assertLess(result["after_bytes"], result["before_bytes"] / 4)
+        wal = self.db_path + "-wal"
+        self.assertEqual(os.path.getsize(wal) if os.path.exists(wal) else 0, 0)
+        self.assertEqual(self.texts(), expected)
+
+
+class TestRestore(MigrationTestCase):
+    def test_restoring_a_pre_dedupe_backup_reads_and_migrates(self):
+        prompt_id, expected = self.library(count=8)
+        backup = os.path.join(self.tmpdir, "old-backup.db")
+        conn = sqlite3.connect(backup)
+        try:  # as a version before the dedupe wrote it: no table, no columns
+            conn.executescript(
+                "CREATE TABLE prompts (id INTEGER PRIMARY KEY AUTOINCREMENT, "
+                "text TEXT NOT NULL, created_at TIMESTAMP, updated_at TIMESTAMP, "
+                "category TEXT, tags TEXT, rating INTEGER, notes TEXT, "
+                "hash TEXT UNIQUE);"
+                "CREATE TABLE generated_images (id INTEGER PRIMARY KEY "
+                "AUTOINCREMENT, prompt_id INTEGER NOT NULL, image_path TEXT NOT "
+                "NULL, filename TEXT NOT NULL, generation_time TIMESTAMP, "
+                "file_size INTEGER, width INTEGER, height INTEGER, format TEXT, "
+                "workflow_data TEXT, prompt_metadata TEXT, parameters TEXT, "
+                "file_path TEXT, FOREIGN KEY (prompt_id) REFERENCES prompts(id) "
+                "ON DELETE CASCADE, UNIQUE(prompt_id, file_path));"
+            )
+            conn.execute(
+                "INSERT INTO prompts (id, text, hash) VALUES (?, 'a cat', 'h')",
+                (prompt_id,),
+            )
+            for image_id, (workflow, graph) in expected.items():
+                conn.execute(
+                    "INSERT INTO generated_images (id, prompt_id, image_path, "
+                    "filename, file_path, workflow_data, prompt_metadata) "
+                    "VALUES (?, ?, ?, ?, ?, ?, ?)",
+                    (
+                        image_id,
+                        prompt_id,
+                        f"/x/{image_id}",
+                        "x",
+                        f"/x/{image_id}",
+                        workflow,
+                        graph,
+                    ),
+                )
+            conn.commit()
+        finally:
+            conn.close()
+        self.migrate()  # the live library is new form now
+        self.db.model.restore_from_file(backup)
+        self.assertEqual(self.texts(), expected)
+        self.assertEqual(self.inline_ids(), set(expected))
+        self.migrate()
+        self.assertEqual(self.texts(), expected)
+        self.assertEqual(self.inline_ids(), set())
+
+    def test_restoring_a_deduped_backup_reads_the_same(self):
+        prompt_id, expected = self.library(count=8)
+        self.migrate()
+        backup = os.path.join(self.tmpdir, "new-backup.db")
+        self.assertTrue(self.db.model.backup_database(backup))
+        ok, reason = PromptModel.verify_database_file(backup)
+        self.assertTrue(ok, reason)
+        self.db.delete_prompt(prompt_id)
+        self.assertEqual(self.document_count(), 0)
+        self.db.model.restore_from_file(backup)
+        self.assertEqual(self.texts(), expected)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/test_workflow_storage_rereview.py

@@ -0,0 +1,267 @@
+"""PR #315 scoped re-review fixes: R1, R3, R4, R5 and R6 (R2 is wording).
+
+- R1: only a row's own cause (undecodable text, a text that would not come
+  back) keeps that row inline; disk-full, I/O and corruption fail the batch.
+- R3: nothing to rewrite is refused up front; one kept pre-migration backup
+  ever, later applies take a pooled safety backup instead.
+- R4: the live job loop lets a writer on another connection in.
+- R5: the digest tool loads no package, adds no log handler and leaves no
+  ``-wal``/``-shm`` beside a stopped database.
+- R6: a damaged document is named, with what to do, in the verify reason and
+  in the job's backup failure.
+"""
+
+import asyncio
+import os
+import sqlite3
+import subprocess  # nosec B404
+import sys
+import threading
+from unittest.mock import patch
+
+from database import workflow_documents as wd
+from database import workflow_storage as ws
+from database.models import PromptModel
+
+try:
+    from tests.test_api_workflow_storage import OPERATION, WorkflowStorageTestCase
+    from tests.test_workflow_migration import MigrationTestCase
+except ImportError:  # discovered with tests/ as the top-level directory
+    from test_api_workflow_storage import OPERATION, WorkflowStorageTestCase
+    from test_workflow_migration import MigrationTestCase
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+TOOL = os.path.join(ROOT, "tools", "workflow_text_digests.py")
+KEPT = "prompts.db.pre-workflow-storage_"
+WAIT = 20
+
+
+def disk_full(*_args, **_kwargs):
+    raise sqlite3.OperationalError("database or disk is full")
+
+
+class TestSystemErrorsFailTheBatch(MigrationTestCase):
+    def test_a_disk_error_fails_the_batch_instead_of_listing_rows(self):
+        _prompt_id, expected = self.library(count=6)
+        for error in (
+            sqlite3.OperationalError("database or disk is full"),
+            sqlite3.OperationalError("disk I/O error"),
+            sqlite3.DatabaseError("database disk image is malformed"),
+            TypeError("a bug"),
+        ):
+            with self.subTest(error=str(error)):
+                with patch.object(ws, "store_text", side_effect=error):
+                    with self.assertRaises(type(error)):
+                        self.migrate(max_rows=50)
+                self.assertFalse(self.conn().in_transaction)
+                self.assertEqual(self.inline_ids(), set(expected))
+                self.assertEqual(self.texts(), expected)
+
+
+class TestSystemErrorsInTheJob(WorkflowStorageTestCase):
+    async def test_a_disk_error_ends_the_job_failed_with_a_truthful_line(self):
+        self.old_library(count=4)
+        before = self.texts()
+        with patch.object(ws, "store_text", side_effect=disk_full):
+            _plan, _summary, job = await self.start()
+            await self.finish(job)
+        self.assertEqual(job.state, "failed")
+        self.assertIn("The batch in progress was rolled back", job.status_line)
+        self.assertEqual(job.counters["errors"], 1)  # the job's one error row
+        self.assertNotIn("could not be rewritten", job.status_line)
+        self.assertIsNone(self.reclaim_job())
+        self.assertEqual(self.texts(), before)
+
+
+class TestOneKeptBackup(WorkflowStorageTestCase):
+    def kept(self):
+        return sorted(n for n in os.listdir(self.tmpdir) if n.startswith(KEPT))
+
+    async def test_nothing_to_rewrite_is_said_and_refused_without_a_backup(self):
+        self.save("no images")
+        plan = await self.preview(OPERATION)
+        self.assertEqual(plan["scope"]["count"], 0)
+        self.assertEqual(plan["scope"]["label"], "Nothing to rewrite")
+        reply = await self.apply(plan["token"], status=409)
+        self.assertEqual(reply["code"], "nothing-to-do")
+        self.assertEqual(
+            reply["error"],
+            "Every image record already stores its workflow once; nothing to do.",
+        )
+        self.assertEqual(self.kept(), [])
+        self.assertEqual(self.backups(), [])
+        self.assertEqual(self.api._jobs.list(), [])
+
+    async def test_a_cancel_and_rerun_keep_one_kept_backup_and_pool_the_next(self):
+        self.old_library(count=12)
+        real = self.api.db.migrate_workflow_batch
+        holder = {}
+
+        def first_then_cancel(*args, **kwargs):
+            result = real(*args, **kwargs)
+            holder["job"].cancelled = True
+            return result
+
+        with (
+            patch.object(ws, "BATCH_ROWS", 5),
+            patch.object(
+                self.api.db, "migrate_workflow_batch", side_effect=first_then_cancel
+            ),
+        ):
+            plan = await self.preview(OPERATION)
+            body = await self.apply(plan["token"])
+            holder["job"] = job = self.api._jobs.get(body["job"]["id"])
+            await self.finish(job)
+        self.assertEqual(job.state, "cancelled")
+        [kept] = self.kept()
+        plan = await self.preview(OPERATION)
+        self.assertEqual(plan["backup"]["policy"], "required")
+        self.assertEqual(
+            plan["backup"]["destination_label"], "prompts.db.backup_<date and time>"
+        )
+        self.assertIn(
+            f"the pre-migration backup {kept} is already kept",
+            " ".join(row["detail"] for row in plan["details"]),
+        )
+        body = await self.apply(plan["token"])
+        again = self.api._jobs.get(body["job"]["id"])
+        await self.finish(again)
+        self.assertEqual(again.state, "finished", again.log)
+        self.assertEqual(self.kept(), [kept])  # still one never-pruned copy
+        [pooled] = self.backups()
+        self.assertIn(f"Safety backup {pooled} verified", "\n".join(again.log))
+        self.assertIn(f"pre-migration backup {kept} is kept", "\n".join(again.log))
+
+
+class TestLiveLoopLetsWritersIn(WorkflowStorageTestCase):
+    async def test_a_writer_on_another_connection_gets_in_during_the_job(self):
+        prompt_id = self.save("writer")
+        conn = self.api.db.model.get_connection()
+        with conn:
+            for index in range(200):
+                conn.execute(
+                    "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                    "file_path, workflow_data) VALUES (?, ?, 'x', ?, ?)",
+                    (prompt_id, f"/x/{index}", f"/x/{index}", f'{{"n": {index}}}'),
+                )
+        started, done = threading.Event(), threading.Event()
+        errors, written = [], []
+        real = ws.store_text
+
+        def slow(conn, text, *args, **kwargs):
+            started.set()
+            done.wait(0.01)  # a slow disk: each row takes a while
+            return real(conn, text, *args, **kwargs)
+
+        def writer():
+            started.wait(WAIT)
+            own = self.api.db.model.get_connection()  # this thread's connection
+            own.execute("PRAGMA busy_timeout = 2000")
+            try:
+                for index in range(5):
+                    written.append(
+                        self.api.db.link_image_to_prompt(
+                            prompt_id,
+                            os.path.join(self.tmpdir, f"w{index}.png"),
+                            {"workflow": {"w": index}},
+                        )
+                    )
+            except sqlite3.OperationalError as error:  # pragma: no cover - RED
+                errors.append(str(error))
+            finally:
+                self.api.db.model.close()
+
+        thread = threading.Thread(target=writer)
+        thread.start()
+        try:
+            with patch.object(ws, "store_text", side_effect=slow):
+                _plan, _summary, job = await self.start()
+                await asyncio.wait_for(asyncio.shield(job.task), 120)
+            await self.finish(job)  # and Reclaim space after it
+        finally:
+            done.set()
+            await asyncio.get_running_loop().run_in_executor(None, thread.join, WAIT)
+        self.assertEqual(job.state, "finished", job.log)
+        self.assertEqual(errors, [])
+        self.assertEqual(len([image for image in written if image]), 5)
+
+
+class TestDigestToolSideEffects(MigrationTestCase):
+    def run_tool(self, path, cwd):
+        out = os.path.join(cwd, "out.sha")
+        code = (
+            "import logging, runpy, sys\n"
+            "sys.argv = ['tool', sys.argv[1], sys.argv[2]]\n"
+            "try:\n"
+            f"    runpy.run_path({TOOL!r}, run_name='__main__')\n"
+            "except SystemExit:\n"
+            "    pass\n"
+            "loaded = sorted(m for m in sys.modules\n"
+            "                if m.split('.')[0] in ('database', 'utils', 'py'))\n"
+            "handlers = [h for n in ('', 'prompt_manager')\n"
+            "            for h in logging.getLogger(n).handlers]\n"
+            "print('LOADED', loaded, 'HANDLERS', len(handlers))\n"
+        )
+        result = subprocess.run(  # nosec B603
+            [sys.executable, "-c", code, path, out],
+            capture_output=True,
+            text=True,
+            cwd=cwd,
+            check=False,
+        )
+        self.assertEqual(result.returncode, 0, result.stderr)
+        return result.stdout
+
+    def test_a_stopped_database_gets_no_sidecars_and_no_log(self):
+        self.library(count=3)
+        self.migrate()
+        PromptModel.close_all_instances()
+        for suffix in ("-wal", "-shm"):
+            self.assertFalse(os.path.exists(self.db_path + suffix))
+        cwd = os.path.join(self.tmpdir, "run")
+        os.mkdir(cwd)
+        before = sorted(os.listdir(self.tmpdir))
+        printed = self.run_tool(self.db_path, cwd)
+        self.assertEqual(sorted(os.listdir(self.tmpdir)), before)
+        self.assertEqual(sorted(os.listdir(cwd)), ["out.sha"])  # no logs/ folder
+        self.assertIn("LOADED [] HANDLERS 0", printed)
+        with open(os.path.join(cwd, "out.sha"), encoding="utf-8") as handle:
+            self.assertEqual(len(handle.read().splitlines()), 3)
+
+    def test_a_live_database_is_read_with_its_wal(self):
+        prompt_id = self.prompt()
+        self.old_form_image(prompt_id, "a.png", '{"in": "wal"}', None)
+        # The connection stays open: the row may still be only in the WAL.
+        cwd = os.path.join(self.tmpdir, "run")
+        os.mkdir(cwd)
+        self.run_tool(self.db_path, cwd)
+        with open(os.path.join(cwd, "out.sha"), encoding="utf-8") as handle:
+            self.assertEqual(len(handle.read().splitlines()), 1)
+
+
+class TestDamagedDocumentIsNamed(WorkflowStorageTestCase):
+    def damage(self):
+        conn = self.api.db.model.get_connection()
+        with conn:
+            document_id = wd.store_text(conn, '{"damaged": true}')
+            conn.execute(
+                "UPDATE workflow_documents SET data=x'00' WHERE id=?", (document_id,)
+            )
+        return document_id
+
+    async def test_verify_names_the_document_and_what_to_do(self):
+        document_id = self.damage()
+        ok, reason = PromptModel.verify_database_file(self.api.db.model.db_path)
+        self.assertFalse(ok)
+        self.assertEqual(reason, wd.damaged_message(document_id))
+        self.assertIn(f"Workflow document {document_id} is damaged", reason)
+        self.assertIn("restore a backup", reason)
+
+    async def test_the_backup_failure_says_which_document(self):
+        self.old_library(count=2)
+        document_id = self.damage()
+        _plan, _summary, job = await self.start()
+        await self.finish(job)
+        self.assertEqual(job.state, "failed")
+        self.assertEqual(job.last_event["code"], "backup-failed")
+        self.assertIn(f"Workflow document {document_id} is damaged", job.status_line)

tests/test_workflow_storage_review.py

@@ -0,0 +1,374 @@
+"""PR #315 final review fixes: I1, I2, I3, M1, M2, M3 and M5.
+
+- I1: the migration always takes a verified backup, whatever the policy.
+- I2: that backup has its own name outside the pruned pool.
+- I3: ``tools/workflow_text_digests.py`` proves byte identity across the rewrite.
+- M1: a writer on another connection is never starved by the batches.
+- M2: a row that cannot be rewritten stays inline, is counted, and the rest go on.
+- M3: a damaged document is refused on restore and repaired before reuse.
+- M5: the legacy table rebuilds refuse a file with document id columns.
+"""
+
+import hashlib
+import os
+import re
+import sqlite3
+import subprocess  # nosec B404
+import sys
+import threading
+import zlib
+from unittest.mock import patch
+
+from database import workflow_documents as wd
+from database import workflow_storage as ws
+from database.models import PromptModel
+
+try:
+    from tests.test_api_workflow_storage import OPERATION, WorkflowStorageTestCase
+    from tests.test_workflow_migration import MigrationTestCase
+except ImportError:  # discovered with tests/ as the top-level directory
+    from test_api_workflow_storage import OPERATION, WorkflowStorageTestCase
+    from test_workflow_migration import MigrationTestCase
+
+# After the API test modules, which stand in ComfyUI's ``server`` module.
+from py.api.admin import _prune_safety_backups  # noqa: E402
+from py.config import PromptManagerConfig  # noqa: E402
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+PRE_MIGRATION = re.compile(
+    r"^prompts\.db\.pre-workflow-storage_\d{8}_\d{6}_\d{6}(_\d+)?$"
+)
+WAIT = 20
+
+
+class TestMigrationBackup(WorkflowStorageTestCase):
+    def pre_migration(self):
+        return sorted(
+            name for name in os.listdir(self.tmpdir) if PRE_MIGRATION.match(name)
+        )
+
+    async def test_the_backup_is_taken_even_with_the_policy_off(self):
+        PromptManagerConfig.SAFETY_BACKUP = False
+        self.old_library(count=4)
+        plan = await self.preview(OPERATION)
+        self.assertEqual(plan["backup"]["policy"], "required")
+        self.assertGreater(plan["backup"]["expected_bytes"], 0)
+        self.assertGreater(plan["disk"]["required_bytes"], 0)
+        body = await self.apply(plan["token"])
+        job = self.api._jobs.get(body["job"]["id"])
+        await self.finish(job)
+        self.assertEqual(job.state, "finished", job.log)
+        [backup] = self.pre_migration()
+        self.assertNotIn("No safety backup", "\n".join(job.log))
+        self.assertIn(f"Pre-migration backup {backup} verified", "\n".join(job.log))
+
+    async def test_the_backup_is_outside_the_pool_never_pruned_and_restores(self):
+        self.old_library(count=4)
+        before = self.texts()
+        plan = await self.preview(OPERATION)
+        self.assertIn(
+            "prompts.db.pre-workflow-storage_<date and time>",
+            " ".join(row["detail"] for row in plan["details"]),
+        )
+        _plan, _summary, job = await self.start()
+        await self.finish(job)
+        [backup] = self.pre_migration()
+        self.assertEqual(self.backups(), [])  # not in the pruned pool
+        self.assertIn(f"Pre-migration backup {backup}", job.status_line)
+        path = os.path.join(self.tmpdir, backup)
+        ok, reason = PromptModel.verify_database_file(path)
+        self.assertTrue(ok, reason)
+        _prune_safety_backups(self.api.db.model.db_path, keep=0)
+        self.assertEqual(self.pre_migration(), [backup])
+        # The way back: the legacy restore accepts it, old form included.
+        self.api.db.model.restore_from_file(path)
+        self.assertEqual(self.texts(), before)
+        self.assertEqual(self.inline_count(), 4)
+
+
+class TestTextDigestsTool(MigrationTestCase):
+    def digests(self, path, out):
+        result = subprocess.run(  # nosec B603
+            [sys.executable, os.path.join(ROOT, "tools", "workflow_text_digests.py")]
+            + [path, out],
+            capture_output=True,
+            text=True,
+            cwd=self.tmpdir,
+            check=False,
+        )
+        self.assertEqual(result.returncode, 0, result.stderr)
+        with open(out, encoding="utf-8") as handle:
+            return handle.read()
+
+    def test_digests_match_across_the_rewrite_and_read_only(self):
+        _prompt_id, expected = self.library(count=12)
+        PromptModel.close_all_instances()
+        before = self.digests(self.db_path, os.path.join(self.tmpdir, "a.sha"))
+        self.assertEqual(len(before.splitlines()), len(expected))
+        self.migrate()
+        PromptModel.close_all_instances()
+        with open(self.db_path, "rb") as handle:
+            frozen = handle.read()
+        after = self.digests(self.db_path, os.path.join(self.tmpdir, "b.sha"))
+        with open(self.db_path, "rb") as handle:
+            self.assertEqual(handle.read(), frozen)
+        self.assertEqual(after, before)
+
+    def test_a_changed_text_shows_in_the_diff(self):
+        prompt_id = self.prompt()
+        self.old_form_image(prompt_id, "a.png", '{"a": 1}', None)
+        PromptModel.close_all_instances()
+        before = self.digests(self.db_path, os.path.join(self.tmpdir, "a.sha"))
+        conn = self.conn()
+        with conn:
+            conn.execute("UPDATE generated_images SET workflow_data='{\"a\":1}'")
+        PromptModel.close_all_instances()
+        after = self.digests(self.db_path, os.path.join(self.tmpdir, "b.sha"))
+        self.assertNotEqual(after, before)
+
+    def test_main_in_process_hashes_undecodable_text_too(self):
+        import contextlib
+        import importlib.util
+        import io
+
+        spec = importlib.util.spec_from_file_location(
+            "workflow_text_digests",
+            os.path.join(ROOT, "tools", "workflow_text_digests.py"),
+        )
+        tool = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(tool)
+        prompt_id = self.prompt()
+        conn = self.conn()
+        with conn:
+            conn.execute(
+                "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                "file_path, workflow_data) VALUES (?, '/x', 'x', '/x', "
+                "CAST(x'ff' AS TEXT))",
+                (prompt_id,),
+            )
+        out = os.path.join(self.tmpdir, "c.sha")
+        printed = io.StringIO()
+        with contextlib.redirect_stdout(printed):
+            self.assertEqual(tool.main([self.db_path, out]), 0)
+        self.assertEqual(printed.getvalue(), "1 image records\n")
+        with open(out, encoding="utf-8") as handle:
+            line = handle.read().split()
+        self.assertEqual(line[1], hashlib.sha256(b"\xff").hexdigest())
+        self.assertEqual(line[2], "-")
+
+
+class TestWriterIsNotStarved(MigrationTestCase):
+    def test_a_writer_on_another_connection_gets_in_between_batches(self):
+        prompt_id = self.prompt()
+        for index in range(200):
+            self.old_form_image(prompt_id, f"{index}.png", f'{{"n": {index}}}', None)
+        started, done = threading.Event(), threading.Event()
+        errors, written = [], []
+        real = ws.store_text
+
+        def slow(conn, text, *args, **kwargs):
+            started.set()
+            done.wait(0.01)  # a slow disk: each row takes a while
+            return real(conn, text, *args, **kwargs)
+
+        def writer():
+            started.wait(WAIT)
+            conn = self.db.model.get_connection()  # this thread's own connection
+            conn.execute("PRAGMA busy_timeout = 2000")
+            try:
+                for index in range(5):
+                    written.append(
+                        self.db.link_image_to_prompt(
+                            prompt_id,
+                            os.path.join(self.tmpdir, f"w{index}.png"),
+                            {"workflow": {"w": index}},
+                        )
+                    )
+            except sqlite3.OperationalError as error:  # pragma: no cover - RED
+                errors.append(str(error))
+            finally:
+                self.db.model.close()
+
+        thread = threading.Thread(target=writer)
+        thread.start()
+        with patch.object(ws, "store_text", side_effect=slow):
+            self.db.migrate_workflow_storage()
+        done.set()
+        thread.join(WAIT)
+        self.assertEqual(errors, [])
+        self.assertEqual(len([image for image in written if image]), 5)
+
+    def test_a_batch_has_a_time_budget(self):
+        prompt_id = self.prompt()
+        for index in range(30):
+            self.old_form_image(prompt_id, f"{index}.png", f'{{"n": {index}}}', None)
+        ticks = iter(range(10**6))
+        with patch.object(ws, "_clock", side_effect=lambda: next(ticks)):
+            batch = ws.migrate_batch(self.conn(), 0, max_seconds=5)
+        self.assertLess(batch["rows"], 30)
+        self.assertGreater(batch["rows"], 0)
+
+
+class TestRowErrors(MigrationTestCase):
+    def test_an_unreadable_row_stays_inline_and_the_rest_are_rewritten(self):
+        prompt_id, expected = self.library(count=10)
+        conn = self.conn()
+        with conn:
+            bad = conn.execute(
+                "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                "file_path, workflow_data) VALUES (?, '/x/bad', 'bad', '/x/bad', "
+                "CAST(x'7b22ff227d' AS TEXT))",
+                (prompt_id,),
+            ).lastrowid
+        result = self.migrate()
+        self.assertEqual(result["errors"], [bad])
+        self.assertEqual(result["rows"], len(expected))
+        self.assertEqual(self.inline_ids_without(bad), set())
+        raw = conn.execute(
+            "SELECT CAST(workflow_data AS BLOB), workflow_document_id "
+            "FROM generated_images WHERE id=?",
+            (bad,),
+        ).fetchone()
+        self.assertEqual((bytes(raw[0]), raw[1]), (b'{"\xff"}', None))
+        self.assertEqual(result["documents_removed"], 0)
+
+    def inline_ids_without(self, image_id):
+        return {
+            row[0]
+            for row in self.conn().execute(
+                "SELECT id FROM generated_images WHERE id != ? AND "
+                "(workflow_data IS NOT NULL OR prompt_metadata IS NOT NULL)",
+                (image_id,),
+            )
+        }
+
+    def test_a_row_whose_text_will_not_come_back_is_kept_and_counted(self):
+        prompt_id = self.prompt()
+        good = self.old_form_image(prompt_id, "good.png", '{"good": 1}', None)
+        bad = self.old_form_image(prompt_id, "bad.png", '{"bad": 1}', None)
+        real = ws.read_stored_text
+
+        def lying(conn, inline, document_id):
+            text = real(conn, inline, document_id)
+            return "other" if text == '{"bad": 1}' else text
+
+        with patch.object(ws, "read_stored_text", side_effect=lying):
+            result = self.migrate()
+        self.assertEqual(result["errors"], [bad])
+        self.assertEqual(self.inline_ids(), {bad})
+        self.assertEqual(self.texts()[good][0], '{"good": 1}')
+        self.assertEqual(self.texts()[bad][0], '{"bad": 1}')
+        self.assertEqual(self.document_count(), 1)  # the bad row's went
+
+
+class TestRowErrorsInTheJob(WorkflowStorageTestCase):
+    async def test_the_job_reports_kept_rows_and_reclaim_space_still_runs(self):
+        prompt_id = self.old_library(count=4)
+        conn = self.api.db.model.get_connection()
+        with conn:
+            conn.execute(
+                "INSERT INTO generated_images (prompt_id, image_path, filename, "
+                "file_path, workflow_data) VALUES (?, '/x/bad', 'bad', '/x/bad', "
+                "CAST(x'7b22ff227d' AS TEXT))",
+                (prompt_id,),
+            )
+        _plan, _summary, job = await self.start()
+        reclaim = await self.finish(job)
+        self.assertEqual(job.state, "finished", job.log)
+        self.assertEqual(job.counters["changed"], 4)
+        self.assertEqual(job.counters["errors"], 1)
+        self.assertIn(
+            "1 image record could not be rewritten and keeps its inline text",
+            job.status_line,
+        )
+        self.assertIsNotNone(reclaim)
+        self.assertEqual(reclaim.state, "finished", reclaim.log)
+
+
+class TestDamagedDocuments(MigrationTestCase):
+    def damage(self, document_id, data=b"not zlib"):
+        conn = self.conn()
+        with conn:
+            conn.execute(
+                "UPDATE workflow_documents SET data=? WHERE id=?", (data, document_id)
+            )
+
+    def test_verify_refuses_a_damaged_document(self):
+        document_id = self.store('{"a": 1}')
+        ok, _reason = PromptModel.verify_database_file(self.db_path)
+        self.assertTrue(ok)
+        for data in (b"not zlib", zlib.compress(b'{"a": 2}')):
+            self.damage(document_id, data)
+            with self.subTest(data=data):
+                ok, reason = PromptModel.verify_database_file(self.db_path)
+                self.assertFalse(ok)
+                self.assertEqual(reason, wd.damaged_message(document_id))
+
+    def test_a_damaged_document_is_repaired_before_it_is_reused(self):
+        prompt_id = self.prompt()
+        first = self.db.link_image_to_prompt(
+            prompt_id, os.path.join(self.tmpdir, "a.png"), {"workflow": {"v": 1}}
+        )
+        document_id = self.raw(first)["workflow_document_id"]
+        self.damage(document_id)
+        with self.assertLogs(wd.logger, "WARNING"):
+            second = self.db.link_image_to_prompt(
+                prompt_id, os.path.join(self.tmpdir, "b.png"), {"workflow": {"v": 1}}
+            )
+        self.assertEqual(self.raw(second)["workflow_document_id"], document_id)
+        for image_id in (first, second):
+            self.assertEqual(self.texts()[image_id][0], '{"v": 1}')
+
+    def test_store_verifies_a_reused_document_once_per_given_set(self):
+        document_id = self.store("abc")
+        verified = set()
+        conn = self.conn()
+        with conn:
+            self.assertEqual(wd.store_text(conn, "abc", verified), document_id)
+        self.assertEqual(verified, {document_id})
+        self.damage(document_id)
+        with conn:  # already verified in this set: not read again
+            self.assertEqual(wd.store_text(conn, "abc", verified), document_id)
+
+
+class TestRebuildGuards(MigrationTestCase):
+    def test_the_legacy_rebuilds_refuse_document_columns(self):
+        for column, kind in (("prompt_id", "TEXT"), ("file_path", None)):
+            path = os.path.join(self.tmpdir, f"guard-{column}.db")
+            conn = sqlite3.connect(path)
+            try:
+                columns = [
+                    "id INTEGER PRIMARY KEY AUTOINCREMENT",
+                    f"prompt_id {kind or 'INTEGER'} NOT NULL",
+                    "image_path TEXT NOT NULL",
+                    "filename TEXT NOT NULL",
+                    "generation_time TIMESTAMP",
+                    "workflow_data TEXT",
+                    "prompt_metadata TEXT",
+                    "parameters TEXT",
+                    "workflow_document_id INTEGER",
+                    "prompt_document_id INTEGER",
+                ]
+                if kind is not None:
+                    columns.append("file_path TEXT")
+                conn.executescript(
+                    "CREATE TABLE prompts (id INTEGER PRIMARY KEY AUTOINCREMENT, "
+                    "text TEXT NOT NULL, created_at TIMESTAMP, updated_at TIMESTAMP, "
+                    "category TEXT, tags TEXT, rating INTEGER, notes TEXT, "
+                    "hash TEXT UNIQUE);"
+                    f"CREATE TABLE generated_images ({', '.join(columns)});"
+                )
+                conn.commit()
+            finally:
+                conn.close()
+            with self.subTest(column=column):
+                with self.assertRaisesRegex(sqlite3.DatabaseError, "workflow document"):
+                    PromptModel(path)
+                PromptModel.close_all_instances()
+                PromptModel.reset_schema_cache()
+
+
+if __name__ == "__main__":  # pragma: no cover
+    import unittest
+
+    unittest.main()

tools/vendor/frontend-coverage-baseline.json

@@ -42,10 +42,10 @@
     "statements": 98
   },
   "gallery.js": {
-    "lines": 41,
-    "branches": 63,
-    "functions": 44,
-    "statements": 41
+    "lines": 45,
+    "branches": 64,
+    "functions": 48,
+    "statements": 45
   },
   "html-escape.js": {
     "lines": 98,
@@ -161,6 +161,12 @@
     "functions": 98,
     "statements": 97
   },
+  "workspace/activity-kinds.mjs": {
+    "lines": 98,
+    "branches": 95,
+    "functions": 98,
+    "statements": 98
+  },
   "workspace/activity-scan.mjs": {
     "lines": 96,
     "branches": 78,
@@ -175,7 +181,7 @@
   },
   "workspace/activity.mjs": {
     "lines": 96,
-    "branches": 88,
+    "branches": 91,
     "functions": 98,
     "statements": 96
   },
@@ -509,6 +515,12 @@
     "functions": 98,
     "statements": 98
   },
+  "workspace/thumbnail-review.mjs": {
+    "lines": 96,
+    "branches": 93,
+    "functions": 98,
+    "statements": 96
+  },
   "workspace/toast.mjs": {
     "lines": 98,
     "branches": 98,

tools/workflow_text_digests.py

@@ -0,0 +1,94 @@
+"""Digests of every image record's stored workflow and prompt graph text.
+
+Proves the workflow storage rewrite (#249 S6) kept every text byte for byte.
+Run it from this repository on the database before the rewrite (or on its
+``prompts.db.pre-workflow-storage_*`` backup) and again after, then compare::
+
+    python3 tools/workflow_text_digests.py prompts.db.pre-workflow-storage_<stamp> \\
+        before.sha
+    python3 tools/workflow_text_digests.py prompts.db after.sha
+    diff before.sha after.sha        # prints nothing when every text is identical
+
+Each line is ``<image id> <sha256 of workflow_data> <sha256 of prompt_metadata>``
+(``-`` for none), hashing the exact stored bytes: the inline column before the
+rewrite, the document read through the one accessor after it.
+
+Read-only, with no side effects: the database is opened with ``mode=ro``,
+and with ``immutable=1`` when no ``-wal``/``-shm`` file exists beside it (a
+stopped database or a backup), so SQLite creates none. A live database
+already has them and is read through its WAL. Only
+``database/workflow_documents.py`` is loaded (by path; it needs only the
+standard library), so no package start-up runs and no log file or handler is
+added. Run it as the user that runs ComfyUI.
+"""
+
+import argparse
+import hashlib
+import importlib.util
+import os
+import sqlite3
+import sys
+from pathlib import Path
+
+_ROOT = Path(__file__).resolve().parents[1]
+
+
+def _load_documents():
+    """``database/workflow_documents.py`` alone, without the ``database`` package."""
+    path = _ROOT / "database" / "workflow_documents.py"
+    spec = importlib.util.spec_from_file_location("_pm_workflow_documents", path)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+def database_uri(db_path):
+    """``mode=ro``, plus ``immutable=1`` when nothing has the database open."""
+    path = os.path.abspath(db_path)
+    uri = Path(path).as_uri() + "?mode=ro"
+    if not any(os.path.exists(path + suffix) for suffix in ("-wal", "-shm")):
+        uri += "&immutable=1"
+    return uri
+
+
+def _digest(value):
+    if value is None:
+        return "-"
+    raw = value if isinstance(value, bytes) else value.encode("utf-8")
+    return hashlib.sha256(raw).hexdigest()
+
+
+def write_digests(db_path, out):
+    """Write one digest line per image record of ``db_path`` to ``out``."""
+    documents = _load_documents()
+    conn = sqlite3.connect(database_uri(db_path), uri=True)
+    try:
+        conn.row_factory = sqlite3.Row
+        conn.text_factory = bytes  # inline text exactly as stored, even undecodable
+        reader = documents.DocumentReader(conn)
+        count = 0
+        with open(out, "w", encoding="utf-8") as handle:
+            for row in conn.execute("SELECT * FROM generated_images ORDER BY id"):
+                texts = documents.resolve_image_row(reader, row)
+                handle.write(
+                    f"{row['id']} {_digest(texts.get('workflow_data'))} "
+                    f"{_digest(texts.get('prompt_metadata'))}\n"
+                )
+                count += 1
+        return count
+    finally:
+        conn.close()
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    parser.add_argument("database", help="prompts.db or one of its backups")
+    parser.add_argument("out", help="file to write the digests to")
+    args = parser.parse_args(argv)
+    count = write_digests(args.database, args.out)
+    print(f"{count} image records")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

utils/diagnostics.py

@@ -38,6 +38,11 @@ from typing import Dict, Any
 
 from .logging_config import get_logger
 
+try:
+    from ..database.workflow_documents import DocumentReader, resolve_image_row
+except ImportError:  # loaded as a top-level ``utils`` package
+    from database.workflow_documents import DocumentReader, resolve_image_row
+
 
 class GalleryDiagnostics:
     """Diagnostics for the gallery system.
@@ -221,7 +226,8 @@ class GalleryDiagnostics:
                     ORDER BY gi.generation_time DESC
                     LIMIT 5
                 """)
-                recent_images = [dict(row) for row in cursor.fetchall()]
+                reader = DocumentReader(conn)
+                recent_images = [resolve_image_row(reader, row) for row in cursor]
 
                 self.logger.info(f"   [TIME] Recent images: {len(recent_images)}")
                 for img in recent_images:

utils/image_monitor.py

@@ -79,6 +79,10 @@ class ImageGenerationHandler(FileSystemEventHandler):
         # ``record_review_file``, wired at start-up): a linked output is
         # never queued and its pending items leave the queue. None: no pass.
         self.review_recorder: Optional[Callable[[str], str]] = None
+        # Queues a settled output's preview (the API's
+        # ``queue_capture_thumbnail``, wired at start-up; #249). It only
+        # queues: the preview is written on its own worker. None: no queue.
+        self.thumbnail_queue: Optional[Callable[[str], str]] = None
 
         # Read from GalleryConfig if available, otherwise use defaults
         try:
@@ -220,10 +224,30 @@ class ImageGenerationHandler(FileSystemEventHandler):
                     )
                 return
             self.process_new_image(image_path, prompt_snapshot=prompt_snapshot)
+            self._queue_thumbnail(image_path)
         finally:
             with self._pending_lock:
                 self._pending.pop(image_path, None)
 
+    def _queue_thumbnail(self, image_path: str) -> None:
+        """Queue a settled output's preview, best effort (linked or not).
+
+        A failure is logged, never raised: the preview is then made on its
+        first view.
+        """
+        queue_preview = getattr(self, "thumbnail_queue", None)
+        if queue_preview is None:
+            return
+        try:
+            outcome = queue_preview(image_path)
+        except Exception:
+            self.logger.warning(
+                f"Preview of {os.path.basename(image_path)} could not be queued",
+                exc_info=True,
+            )
+            return
+        self.logger.debug(f"Preview {outcome}: {os.path.basename(image_path)}")
+
     def wait_until_settled(self, image_path: str, stop=None) -> bool:
         """True once the file's size is non-zero and unchanged across two reads.
 
@@ -636,6 +660,16 @@ class ImageMonitor:
         self.monitored_directories = []
         self.logger = get_logger("prompt_manager.image_monitor")
         self.review_recorder: Optional[Callable[[str], str]] = None
+        self.thumbnail_queue: Optional[Callable[[str], str]] = None
+
+    def set_thumbnail_queue(
+        self, queue_preview: Optional[Callable[[str], str]]
+    ) -> None:
+        """Queue every settled output's preview through ``queue_preview``
+        (None stops it), on the running handler and every later one."""
+        self.thumbnail_queue = queue_preview
+        if self.handler is not None:
+            self.handler.thumbnail_queue = queue_preview
 
     def set_review_recorder(self, recorder: Optional[Callable[[str], str]]) -> None:
         """Record ambiguous outputs through ``recorder`` (None stops recording).
@@ -703,6 +737,7 @@ class ImageMonitor:
         # leaves the monitor stuck "already running" with nothing to stop.
         handler = ImageGenerationHandler(self.db_manager, self.prompt_tracker)
         handler.review_recorder = self.review_recorder
+        handler.thumbnail_queue = self.thumbnail_queue
         observer = Observer()
         scheduled = []
         for output_dir in valid_directories:

utils/inspection_semantics.py

@@ -16,10 +16,21 @@ nesting before it is validated. Every class appears in exactly one prompt
 category; classes that are not listed stay ``unsupported-node``. ``lineage``
 (image and latent pass-through outputs for multi-stage chains) describes no
 prompt, so its classes are not claimed by a category.
+
+``Semantics.version`` is the SHA-256 of the table's canonical JSON (sorted
+keys, no whitespace): any change of content, never of layout, moves it.
+``class_version(doc, classes)`` is the narrower fingerprint Needs review
+folds into the version of the items it records from the inspection profile
+(``utils.review_capture``): the SHA-256 of ``class_projection``, the table
+restricted to what it says about ``classes`` (the node classes of one graph)
+plus the class-independent ``schema`` and ``stage_inputs``. The resolver looks
+the table up only by the class of a graph node, so an edit about other
+classes cannot change how that graph resolves and leaves the version alone.
 """
 
+import hashlib
 import json
-from dataclasses import dataclass
+from dataclasses import dataclass, replace
 from functools import lru_cache
 from pathlib import Path
 from types import MappingProxyType
@@ -118,6 +129,10 @@ class Semantics:
     stage_inputs: Mapping[str, Tuple[str, ...]]
     facts: Mapping[str, Any]
     all_classes: FrozenSet[str]
+    # SHA-256 of the canonical table JSON (set by ``load_semantics``).
+    version: str = ""
+    # The canonical table JSON itself, for ``class_version``.
+    source: str = ""
 
 
 def _freeze(value: Any) -> Frozen:
@@ -521,7 +536,71 @@ def load_semantics(path: Optional[Union[str, Path]] = None) -> Semantics:
     except (OSError, UnicodeDecodeError, ValueError) as error:
         raise SemanticsError(f"cannot read semantics table {source.name}: {error}")
     _check_nesting(doc)
-    return _validate(doc)
+    return replace(
+        _validate(doc), version=table_version(doc), source=_canonical_json(doc)
+    )
+
+
+# Table parts that hold no class names: they apply to every graph.
+_CLASS_INDEPENDENT = frozenset({"schema", "stage_inputs"})
+
+
+def _canonical_json(doc: Any) -> str:
+    return json.dumps(doc, sort_keys=True, separators=(",", ":"), ensure_ascii=True)
+
+
+def table_version(doc: Any) -> str:
+    """SHA-256 of ``doc`` as canonical JSON: content, not layout."""
+    return hashlib.sha256(_canonical_json(doc).encode("utf-8")).hexdigest()
+
+
+def _about(value: Any, classes: FrozenSet[str]) -> Any:
+    """The part of one category that names a class in ``classes``.
+
+    Categories are keyed by class (objects), list classes (strings) or list
+    entries that carry a ``class``.
+    """
+    if isinstance(value, dict):
+        return {key: item for key, item in value.items() if key in classes}
+    if isinstance(value, list):
+        return [
+            entry
+            for entry in value
+            if (isinstance(entry, str) and entry in classes)
+            or (isinstance(entry, dict) and entry.get("class") in classes)
+        ]
+    return value
+
+
+def class_projection(doc: Mapping[str, Any], classes) -> dict:
+    """What ``doc`` says about ``classes``, plus its class-independent parts."""
+    classes = frozenset(classes)
+    projected = {}
+    for category, value in doc.items():
+        if category in _CLASS_INDEPENDENT:
+            projected[category] = value
+        elif category == "facts" and isinstance(value, dict):
+            projected[category] = {
+                table: _about(entries, classes) for table, entries in value.items()
+            }
+        else:
+            projected[category] = _about(value, classes)
+    return projected
+
+
+def class_version(doc: Mapping[str, Any], classes) -> str:
+    """SHA-256 of ``class_projection(doc, classes)`` as canonical JSON."""
+    return table_version(class_projection(doc, classes))
+
+
+@lru_cache(maxsize=256)
+def _cached_class_version(source: str, classes: FrozenSet[str]) -> str:
+    return class_version(json.loads(source), classes)
+
+
+def semantics_class_version(semantics: Semantics, classes) -> str:
+    """``class_version`` of a loaded table (cached per table and class set)."""
+    return _cached_class_version(semantics.source, frozenset(classes))
 
 
 @lru_cache(maxsize=1)

utils/prompt_pair_resolver.py

@@ -44,6 +44,11 @@ _LOGGER = logging.getLogger("prompt_manager.prompt_pair_resolver")
 # otherwise; a pending item stays applicable while its chosen candidate is
 # still offered by the captured bytes.
 RESOLVER_VERSION = 1
+# The inspection profile's own version, folded into the version of Needs
+# review items recorded from that profile (with the profile's budgets and
+# the semantics table). Bump it when a change would make the inspection
+# profile resolve the same metadata otherwise, even if ingestion is unchanged.
+INSPECTION_RESOLVER_VERSION = 1
 
 
 @dataclass(frozen=True)

utils/review_capture.py

@@ -1,7 +1,8 @@
 """Capture-bound Needs review items for ambiguous files (#249 E2).
 
-The output scan and the live monitor hand every file the ingestion resolver
-finds ``ambiguous`` to :func:`record_ambiguous_file`. It captures the file
+The output scan hands every unlinked file the ingestion resolver finds
+``ambiguous`` to :func:`record_ambiguous_file` (only scans add items; the
+live monitor records nothing). It captures the file
 (the bounded private copy of ``utils.media_capture``, hashed while copied),
 reads the metadata again from the capture, resolves it with the default
 (ingestion) profile and stores only an ``ambiguous`` result, bound to the
@@ -21,6 +22,24 @@ candidate's ``hash`` is the SHA-256 of its canonical JSON without the hash.
 More than ``MAX_CANDIDATES`` candidates or evidence over the 8 MiB bound make
 a ``limited`` item without candidates (a result without any candidate is not
 ``reviewable`` and is never stored).
+
+Inspection-profile choices (ruling a, after the maintainer's Qwen image-edit
+batches): when the default profile is ambiguous without a choice, the scan
+(``inspection=True``) asks ``INSPECTION_PROFILE``,
+which knows more encoder and string nodes. With at least two choosable
+candidates there, the item is recorded from that profile: its evidence names
+``profile: "inspection"`` and ``semantics``, the fingerprint of what the table
+says about the graph's node classes, and its ``resolver_version`` is
+``inspection_review_version(metadata)``, which folds the profile,
+``RESOLVER_VERSION`` and that fingerprint together. A table edit about those
+classes moves the version the way a resolver change does (I2); an edit about
+other classes moves nothing. A pending item stays applicable while the bytes
+still offer its chosen candidate, and a rescan's new item supersedes it only
+while it is pending and undecided (never beside a decision, see
+``PromptDatabase.record_review_item``). Apply re-resolves with the
+profile the item recorded (``captured_review(..., item_profile(evidence))``).
+The inspection profile only ever produces review choices; ingestion's own
+links stay on the default profile, which never loads the table (D1).
 """
 
 from __future__ import annotations
@@ -29,6 +48,7 @@ import hashlib
 import json
 import logging
 import os
+from dataclasses import astuple
 from pathlib import Path
 from typing import Any
 
@@ -40,9 +60,19 @@ try:
         canonical_evidence,
         choosable,
     )
+    from .inspection_semantics import (
+        SemanticsError,
+        get_semantics,
+        semantics_class_version,
+    )
     from .media_capture import CaptureCapacity, CaptureError
     from .prompt_inspector import MetadataReadError, read_inspection_file
-    from .prompt_pair_resolver import RESOLVER_VERSION, resolve_prompt_pair
+    from .prompt_pair_resolver import (
+        INSPECTION_PROFILE,
+        INSPECTION_RESOLVER_VERSION,
+        RESOLVER_VERSION,
+        resolve_prompt_pair,
+    )
 except ImportError:  # imported with the repo root on sys.path
     from database.library_commands import LibraryCommandError
     from database.review_schema import (
@@ -51,9 +81,19 @@ except ImportError:  # imported with the repo root on sys.path
         canonical_evidence,
         choosable,
     )
+    from utils.inspection_semantics import (
+        SemanticsError,
+        get_semantics,
+        semantics_class_version,
+    )
     from utils.media_capture import CaptureCapacity, CaptureError
     from utils.prompt_inspector import MetadataReadError, read_inspection_file
-    from utils.prompt_pair_resolver import RESOLVER_VERSION, resolve_prompt_pair
+    from utils.prompt_pair_resolver import (
+        INSPECTION_PROFILE,
+        INSPECTION_RESOLVER_VERSION,
+        RESOLVER_VERSION,
+        resolve_prompt_pair,
+    )
 
 logger = logging.getLogger("prompt_manager.review_capture")
 
@@ -61,11 +101,136 @@ logger = logging.getLogger("prompt_manager.review_capture")
 REVIEW_SCOPE = "needs-review"
 # Nested ambiguous stages are flattened at most this deep.
 MAX_CANDIDATE_DEPTH = 8
+# The resolver profile an item's evidence was built with (``profile``).
+DEFAULT = "default"
+INSPECTION = "inspection"
+# Inspection item versions: above every plausible ``RESOLVER_VERSION`` and,
+# with 40 bits of the folded digest, below 2**53 (exact as a JSON number).
+INSPECTION_VERSION_BASE = 1 << 40
+_INSPECTION_VERSION_HEX = 10
 
 
-def record_ambiguous_file(db, store, root, root_index: int, relative: str) -> str:
+# Above this many distinct node classes the whole table is the fingerprint.
+MAX_FINGERPRINT_CLASSES = 1024
+
+
+def graph_classes(metadata: Any) -> frozenset | None:
+    """The node classes of ``metadata``'s prompt graph (empty without one).
+
+    None when the graph names more than ``MAX_FINGERPRINT_CLASSES`` classes.
+    """
+    graph = metadata.get("prompt") if isinstance(metadata, dict) else None
+    if not isinstance(graph, dict):
+        return frozenset()
+    classes = set()
+    for node in graph.values():
+        name = node.get("class_type") if isinstance(node, dict) else None
+        if isinstance(name, str):
+            classes.add(name)
+            if len(classes) > MAX_FINGERPRINT_CLASSES:
+                return None
+    return frozenset(classes)
+
+
+def semantics_version(metadata: Any = None) -> str | None:
+    """The semantics table fingerprint that ``metadata``'s graph depends on.
+
+    The table restricted to the graph's node classes
+    (``inspection_semantics.class_version``), so an edit about other classes
+    leaves it alone; the whole table's version without ``metadata`` or for a
+    graph of very many classes. None when the table is unusable.
+    """
+    try:
+        semantics = get_semantics()
+    except SemanticsError:
+        return None
+    classes = graph_classes(metadata) if metadata is not None else None
+    if classes is None:
+        return semantics.version
+    return semantics_class_version(semantics, classes)
+
+
+def inspection_review_version(metadata: Any = None) -> int | None:
+    """The ``resolver_version`` of an item recorded from the inspection profile.
+
+    Folds the profile with its budgets, ``RESOLVER_VERSION``,
+    ``INSPECTION_RESOLVER_VERSION`` and ``semantics_version(metadata)``, so
+    a change of either resolver, of the profile's bounds or of the table
+    about the graph's classes moves it. None when the table cannot be loaded.
+    """
+    table = semantics_version(metadata)
+    if table is None:
+        return None
+    profile = repr(astuple(INSPECTION_PROFILE))
+    seed = f"{profile}:{RESOLVER_VERSION}:{INSPECTION_RESOLVER_VERSION}:{table}"
+    folded = hashlib.sha256(seed.encode("utf-8")).hexdigest()
+    return INSPECTION_VERSION_BASE + int(folded[:_INSPECTION_VERSION_HEX], 16)
+
+
+def resolve_inspection(metadata: dict) -> dict:
+    """``metadata`` resolved with the inspection profile (bounded work)."""
+    return resolve_prompt_pair(metadata, profile=INSPECTION_PROFILE)
+
+
+def inspection_choice(metadata: dict) -> dict | None:
+    """The inspection profile's ambiguous result when it offers a choice.
+
+    The scan asks only for a file the default profile found ambiguous
+    without a choice that nothing links.
+    """
+    if inspection_review_version(metadata) is None:
+        return None
+    resolution = resolve_inspection(metadata)
+    if resolution.get("state") != "ambiguous" or not reviewable(resolution):
+        return None
+    return resolution
+
+
+def item_profile(evidence: Any) -> str:
+    """The profile an item's stored evidence was built with."""
+    if isinstance(evidence, dict) and evidence.get("profile") == INSPECTION:
+        return INSPECTION
+    return DEFAULT
+
+
+def item_versions(metadata: Any = None) -> tuple:
+    """The item versions a file with ``metadata`` may already be queued under.
+
+    Without ``metadata``, only the ingestion version.
+    """
+    version = inspection_review_version(metadata) if metadata is not None else None
+    return (RESOLVER_VERSION,) if version is None else (RESOLVER_VERSION, version)
+
+
+def _same_records(captured: dict, scanned: dict) -> bool:
+    """True when the capture holds the records the scan resolved."""
+    keys = (set(captured) | set(scanned)) - {"file_info"}
+    return all(captured.get(key) == scanned.get(key) for key in keys)
+
+
+def record_ambiguous_file(
+    db,
+    store,
+    root,
+    root_index: int,
+    relative: str,
+    *,
+    inspection: bool = False,
+    scanned: tuple | None = None,
+) -> str:
     """Persist the ambiguous evidence of one rooted file.
 
+    With ``inspection`` (the scan), a default result without a choice is
+    resolved again with the inspection profile, and recorded from it when
+    that offers a choice. ``scanned`` is the
+    scan's ``(metadata, inspection_resolution_or_None)`` for the file: its
+    metadata names the item versions that count, and when the capture holds
+    the same records its resolution is reused instead of resolving again.
+
+    A new item supersedes the file's other items of the same bytes (an older
+    resolver or table version) when every one is still pending and
+    undecided; when one carries a decision, nothing new is stored.
+
     Returns ``recorded`` (new item with candidates), ``limited`` (new item
     without candidates), ``duplicate`` (these bytes already carry this
     evidence, or the file's current stat is the verified fingerprint of
@@ -79,9 +244,10 @@ def record_ambiguous_file(db, store, root, root_index: int, relative: str) -> st
     ``CaptureCapacity`` and database errors propagate to the caller, which
     logs and counts them.
     """
-    spelled_path, queued = _queued_spelling(db, root, relative)
+    versions = item_versions(scanned[0] if scanned is not None else None)
+    spelled_path, queued = _queued_spelling(db, root, relative, versions)
     if queued is not None:
-        return _already_queued(db, *queued)
+        return _already_queued(db, *queued, versions)
     epoch = db.get_execution_epoch()
     try:
         # Background: the last worker stays free for an interactive request.
@@ -105,9 +271,26 @@ def record_ambiguous_file(db, store, root, root_index: int, relative: str) -> st
     resolution = captured[1] if captured is not None else None
     if resolution is None or resolution.get("state") != "ambiguous":
         return "skipped"
+    profile, version = DEFAULT, RESOLVER_VERSION
     if not reviewable(resolution):
-        return "untraced"
-    evidence, count, limited = review_evidence(resolution)
+        version = inspection_review_version(captured[0]) if inspection else None
+        if version is None:
+            return "untraced"
+        profile = INSPECTION
+        if (
+            scanned is not None
+            and scanned[1] is not None
+            and _same_records(captured[0], scanned[0])
+        ):
+            resolution = scanned[1]  # the same records: no second run
+        else:
+            # Heavier, so only for a file ambiguous with nothing to choose.
+            resolution = resolve_inspection(captured[0])
+        if resolution.get("state") != "ambiguous" or not reviewable(resolution):
+            return "untraced"
+    evidence, count, limited = review_evidence(
+        resolution, profile, version, captured[0]
+    )
     try:
         inserted = db.record_review_item(
             locator=capture.locator,
@@ -116,7 +299,7 @@ def record_ambiguous_file(db, store, root, root_index: int, relative: str) -> st
             size=capture.size,
             mtime_ns=capture.mtime_ns,
             epoch=capture.epoch,
-            resolver_version=RESOLVER_VERSION,
+            resolver_version=version,
             evidence=evidence,
             candidate_count=count,
             limited=limited,
@@ -124,18 +307,21 @@ def record_ambiguous_file(db, store, root, root_index: int, relative: str) -> st
     except LibraryCommandError:
         return "skipped"
     if not inserted:
-        return _already_queued(db, capture.locator, capture.size, capture.mtime_ns)
+        return _already_queued(
+            db, capture.locator, capture.size, capture.mtime_ns, versions
+        )
     return "limited" if limited else "recorded"
 
 
-def _already_queued(db, locator: str, size: int, mtime_ns: int) -> str:
-    """``duplicate`` while the existing item is pending, else ``decided``."""
-    if db.review_pending_at(locator, size, mtime_ns, RESOLVER_VERSION):
-        return "duplicate"
+def _already_queued(db, locator: str, size: int, mtime_ns: int, versions) -> str:
+    """``duplicate`` while an existing item is pending, else ``decided``."""
+    for version in versions:
+        if db.review_pending_at(locator, size, mtime_ns, version):
+            return "duplicate"
     return "decided"
 
 
-def _queued_spelling(db, root, relative: str) -> tuple[str, tuple | None]:
+def _queued_spelling(db, root, relative: str, versions) -> tuple[str, tuple | None]:
     """The file's spelling, and ``(locator, size, mtime_ns)`` when queued.
 
     The second value is None unless the file's current stat is already
@@ -147,7 +333,9 @@ def _queued_spelling(db, root, relative: str) -> tuple[str, tuple | None]:
     an item already cites is not copied or hashed again (maintainer ruling:
     the rescan stays cheap and the file stays in the default scope); its
     items take the current spelling. A stat failure leaves the decision to
-    the capture.
+    the capture. ``versions`` are the item versions that count; bytes whose
+    item carries a decision count under any version (a new version never
+    adds an item beside a decision).
     """
     spelled = os.path.join(os.path.realpath(os.fspath(root)), *relative.split("/"))
     try:
@@ -155,8 +343,9 @@ def _queued_spelling(db, root, relative: str) -> tuple[str, tuple | None]:
     except OSError:
         return spelled, None
     locator = os.path.normcase(spelled)
-    if db.review_fingerprint_queued(
-        locator, facts.st_size, facts.st_mtime_ns, RESOLVER_VERSION
+    if db.review_decided_at(locator, facts.st_size, facts.st_mtime_ns) or any(
+        db.review_fingerprint_queued(locator, facts.st_size, facts.st_mtime_ns, v)
+        for v in versions
     ):
         # Not copied again, but a case-only rename keeps the stat: record
         # the current spelling so the item's target still opens.
@@ -165,12 +354,14 @@ def _queued_spelling(db, root, relative: str) -> tuple[str, tuple | None]:
     return spelled, None
 
 
-def captured_review(store, capture) -> tuple[dict, dict] | None:
+def captured_review(store, capture, profile: str = DEFAULT) -> tuple[dict, dict] | None:
     """``(metadata, resolution)`` of the capture's own bytes, else None.
 
     ``metadata`` is what the scan hands the ingester: prompt and workflow
     JSON-decoded and ``file_info`` (size, dimensions, format) from the
-    capture. None when the capture is gone or carries no prompt metadata.
+    capture. ``resolution`` comes from ``profile`` (``DEFAULT`` or
+    ``INSPECTION``). None when the capture is gone or carries no prompt
+    metadata.
     """
     if store.acquire(capture.token) is None:
         return None
@@ -197,6 +388,8 @@ def captured_review(store, capture) -> tuple[dict, dict] | None:
     if read.format is not None:
         info["format"] = read.format
     canonical["file_info"] = info
+    if profile == INSPECTION:
+        return canonical, resolve_inspection(canonical)
     return canonical, resolve_prompt_pair(canonical)
 
 
@@ -218,12 +411,25 @@ def reviewable(resolution: Any) -> bool:
     return False
 
 
-def review_evidence(resolution: dict) -> tuple[dict, int, bool]:
-    """``(evidence, candidate_count, limited)`` for one ambiguous result."""
+def review_evidence(
+    resolution: dict,
+    profile: str = DEFAULT,
+    version: int | None = None,
+    metadata: Any = None,
+) -> tuple[dict, int, bool]:
+    """``(evidence, candidate_count, limited)`` for one ambiguous result.
+
+    Inspection evidence also names its ``profile`` and ``semantics``, the
+    table fingerprint of ``metadata``'s graph it was resolved with;
+    ``version`` is the item's version.
+    """
     base = {
-        "resolver_version": RESOLVER_VERSION,
+        "resolver_version": RESOLVER_VERSION if version is None else version,
         "reason": _text(resolution.get("reason")),
     }
+    if profile == INSPECTION:
+        base["profile"] = INSPECTION
+        base["semantics"] = semantics_version(metadata)
     candidates: list[dict] = []
     seen: set[str] = set()
     total = 0

web/js/gallery.js

@@ -1,4 +1,7 @@
         class GalleryManager {
+            // A Cancel on a thumbnail job another tab or System started (#249 System S5, review I1).
+            static THUMBNAILS_STOPPED_FOLLOWING = 'Stopped following thumbnail generation started elsewhere; it keeps running in Activity';
+
             constructor() {
                 this.images = [];
                 this.currentPage = 1;
@@ -485,7 +488,8 @@
                     const index = from + offset;
                     // Use thumbnail for display if available, fallback to original
                     const displayUrl = image.thumbnail_url || image.url;
-                    const hasThumb = !!image.thumbnail_url;
+                    // The listing names a preview URL for every file (made on view, #249); the dot means the file exists.
+                    const hasThumb = typeof image.thumbnail_ready === 'boolean' ? image.thumbnail_ready : !!image.thumbnail_url;
                     const isVideo = image.is_video || false;
                     const mediaType = image.media_type || 'image';
                     
@@ -548,7 +552,8 @@
                     const index = from + offset;
                     // Use thumbnail for display if available, fallback to original
                     const displayUrl = image.thumbnail_url || image.url;
-                    const hasThumb = !!image.thumbnail_url;
+                    // The listing names a preview URL for every file (made on view, #249); the dot means the file exists.
+                    const hasThumb = typeof image.thumbnail_ready === 'boolean' ? image.thumbnail_ready : !!image.thumbnail_url;
                     const isVideo = image.is_video || false;
                     const mediaType = image.media_type || 'image';
                     
@@ -1911,6 +1916,54 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                 }
             }
 
+            /** Ask the server to stop thumbnail job `jobId` at its next file boundary (#249 System S5). */
+            async cancelThumbnailJob(jobId) {
+                try {
+                    await this.api.post(`/prompt_manager/jobs/${encodeURIComponent(jobId)}/cancel`, {});
+                    return true;
+                } catch (error) {
+                    console.warn('The thumbnail job could not be cancelled:', error);
+                    return false;
+                }
+            }
+
+            /**
+             * Cancel for one thumbnail stream (#249 System S5, review I1). The stream's first frame, `job {id, started}`,
+             * names the job it follows and whether it started it. `request()` stops that job by its id, and only when this
+             * stream started it: a job another tab or System started keeps running, and the stream just stops following.
+             * A request before the `job` frame keeps the stream open until the frame arrives; `ended()` (the stream closed
+             * or refused without naming a job) leaves nothing to cancel.
+             */
+            thumbnailCanceller(eventSource) {
+                let job = null;
+                let wanted = false;
+                let done = false;
+                const apply = () => {
+                    if (done || !wanted || !job) return;
+                    done = true;
+                    eventSource.close();
+                    if (job.started) this.cancelThumbnailJob(job.id);
+                };
+                return {
+                    identify(raw) {
+                        if (job) return;
+                        let data = null;
+                        try { data = JSON.parse(raw); } catch (error) { return; }
+                        if (typeof data?.id !== 'string' || !data.id) return;
+                        job = { id: data.id, started: data.started === true };
+                        apply();
+                    },
+                    request() { wanted = true; apply(); },
+                    /** True once the stream named a job this page did not start. */
+                    get following() { return job !== null && !job.started; },
+                    ended() {
+                        if (done || !wanted || job) return;
+                        done = true;
+                        eventSource.close();
+                    },
+                };
+            }
+
             async generateThumbnailsWithProgress(options) {
                 const progressBar = document.getElementById('progressBar');
                 const progressText = document.getElementById('progressText');
@@ -1932,13 +1985,19 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                     
                     let cancelled = false;
                     let resultData = null;
+                    const canceller = this.thumbnailCanceller(eventSource);
+                    eventSource.addEventListener('job', event => canceller.identify(event.data));
                     
-                    // Handle cancellation
+                    // Handle cancellation. The work is an Activity job that outlives this stream, so closing it no
+                    // longer stops anything: ask the server to stop this stream's job by its id (review I1).
                     const cancelHandler = () => {
+                        if (cancelled) return;
                         cancelled = true;
-                        eventSource.close();
+                        cancelBtn.removeEventListener('click', cancelHandler);
                         cancelBtn.classList.add('hidden');
-                        reject(new Error('Thumbnail generation cancelled by user'));
+                        const following = canceller.following;
+                        canceller.request();
+                        reject(new Error(following ? GalleryManager.THUMBNAILS_STOPPED_FOLLOWING : 'Thumbnail generation cancelled by user'));
                     };
                     cancelBtn.addEventListener('click', cancelHandler);
                     
@@ -2013,7 +2072,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                     
                     // Handle completion
                     eventSource.addEventListener('complete', (event) => {
-                        if (cancelled) return;
+                        if (cancelled) { canceller.ended(); return; }
                         resultData = JSON.parse(event.data);
                         
                         // Update final progress
@@ -2061,7 +2120,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
 
                     // Handle errors
                     eventSource.addEventListener('error', (event) => {
-                        if (cancelled) return;
+                        if (cancelled) { canceller.ended(); return; }
                         const data = JSON.parse(event.data);
                         
                         // Clean up
@@ -2074,7 +2133,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                     
                     // Handle EventSource errors
                     eventSource.onerror = (error) => {
-                        if (cancelled) return;
+                        if (cancelled) { canceller.ended(); return; }
                         console.error('EventSource error:', error);
                         console.error('EventSource readyState:', eventSource.readyState);
                         
@@ -2097,6 +2156,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
 
                     // Stream ended without a complete/error frame (close() does not trigger this)
                     eventSource.onclose = () => {
+                        if (cancelled) canceller.ended();
                         if (cancelled || resultData) return;
                         cancelBtn.removeEventListener('click', cancelHandler);
                         cancelBtn.classList.add('hidden');
@@ -2692,12 +2752,15 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                     statusMessages.scrollTop = statusMessages.scrollHeight;
                 };
                 
-                // Handle cancel
+                // Handle cancel: the stream's own job stops at its next file boundary (#249 System S5, review I1)
+                let cancelStream = null;
                 cancelBtn.addEventListener('click', () => {
+                    if (isCancelled) return;
                     isCancelled = true;
                     addStatusMessage('Cancellation requested...', 'warning');
                     cancelBtn.disabled = true;
                     cancelBtn.textContent = 'Cancelling...';
+                    cancelStream?.();
                 });
                 
                 // Handle close (only appears when done)
@@ -2721,7 +2784,8 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                         progressDetails,
                         progressETA,
                         addStatusMessage,
-                        isCancelled: () => isCancelled
+                        isCancelled: () => isCancelled,
+                        onCancel: handler => { cancelStream = handler; }
                     });
                     
                     // Show completion
@@ -2741,10 +2805,14 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                     }
                     
                 } catch (error) {
-                    console.error('Thumbnail generation failed:', error);
-                    addStatusMessage(`Generation failed: ${error.message}`, 'error');
-                    progressBar.classList.add('bg-pm-error');
-                    progressBar.classList.remove('bg-pm-accent');
+                    if (isCancelled) {
+                        addStatusMessage(error.message, 'warning');
+                    } else {
+                        console.error('Thumbnail generation failed:', error);
+                        addStatusMessage(`Generation failed: ${error.message}`, 'error');
+                        progressBar.classList.add('bg-pm-error');
+                        progressBar.classList.remove('bg-pm-accent');
+                    }
                 } finally {
                     // Show close button and hide cancel
                     cancelBtn.classList.add('hidden');
@@ -2762,7 +2830,7 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                     }, 100);
                 }
             }            async generateThumbnailsWithProgressInModal(uiElements) {
-                const { progressText, progressPercent, progressBar, progressDetails, progressETA, addStatusMessage, isCancelled } = uiElements;
+                const { progressText, progressPercent, progressBar, progressDetails, progressETA, addStatusMessage, isCancelled, onCancel } = uiElements;
                 
                 return new Promise((resolve, reject) => {
                     addStatusMessage('Starting thumbnail generation...');
@@ -2776,6 +2844,14 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
                     const eventSource = SseStream.connect(`/prompt_manager/images/generate-thumbnails/progress?quality=${encodeURIComponent(quality)}`, { method: 'POST' });
                     
                     let resultData = null;
+                    const canceller = this.thumbnailCanceller(eventSource);
+                    eventSource.addEventListener('job', event => canceller.identify(event.data));
+                    // Cancel stops this stream's job by id, keeps what it wrote, and settles the dialog.
+                    onCancel?.(() => {
+                        const following = canceller.following;
+                        canceller.request();
+                        reject(new Error(following ? GalleryManager.THUMBNAILS_STOPPED_FOLLOWING : 'Thumbnail generation cancelled; previews already written are kept'));
+                    });
                     
                     // Handle scanning/status events
                     eventSource.addEventListener('status', (event) => {

web/js/workspace/activity-job-view.mjs

@@ -14,7 +14,11 @@
  * - Log: `details.term` "Scan log", collapsed, holding the raw timestamped lines in `pre[tabindex=0]`.
  * - D89: the tiles, the outcome list, the openable rows and the log and cancel labels are the job kind's (`jobView`):
  *   an image tagging job shows Tagged, Unchanged, Skipped and Errors, opens Tagged and Unchanged rows, and reads
- *   "Tagging log" and "Cancel tagging"; another kind keeps the scan's tiles as "Job log" and "Cancel job".
+ *   "Tagging log" and "Cancel tagging".
+ * - D111: the kind's descriptor (activity-kinds.mjs `viewOf`) decides them for every kind: a thumbnail job counts
+ *   Generated, Skipped and Errors; a kind without groups (the health check, a held legacy-page operation) shows no
+ *   tiles and no Results, never another kind's zeros. A job without a known total has an indeterminate `<progress>`;
+ *   only a job the server can stop shows Cancel.
  *
  * The summary follows the Activity store (one SSE stream per running job); the listed rows and the log are read from
  * `GET /prompt_manager/jobs/<id>` at most once a second while the job runs, and once more when it ends.
@@ -23,7 +27,8 @@ import { createButton } from '../shared/toolkit/core.js';
 import { pageHeading } from './heading.mjs';
 import { showWorkspaceToast } from './toast.mjs';
 import { noun } from './plural.mjs';
-import { parseJob, jobView, JOB_VIEWS, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
+import { parseJob, JOB_VIEWS, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
+import { viewOf, canCancel, showProgress } from './activity-kinds.mjs';
 import { jobDetail, CANCEL_FAILED_TOAST, createReviewMetadataButton, hasReviewItems } from './activity.mjs';
 import { downloadJobReport } from './activity-scan.mjs';
 import { createMediaViewer } from './media-viewer.mjs';
@@ -71,7 +76,10 @@ export function parseDetail(raw) {
 /** The progress line: the server's while running, then what the job got through. */
 export function progressLine(job) {
     const files = `${number(job.processed)} of ${number(job.total)} ${noun(job.total, 'file')}`;
-    if (job.state === 'running') return job.status_line || `${job.progress}%`;
+    // D111: a job that never counted files (a held legacy job, the health check) says only what the server says.
+    const counted = job.total > 0;
+    if (job.state === 'running') return job.status_line || (counted ? `${job.progress}%` : '');
+    if (!counted && (job.state === 'finished' || job.state === 'cancelled')) return job.status_line;
     if (job.state === 'finished') return `${files} processed`;
     if (job.state === 'cancelled') return `Cancelled after ${files}`;
     if (job.state === 'lost') return LOST_DETAIL;
@@ -97,7 +105,7 @@ export function filterRows(rows, filter) {
 
 /** The first outcome with files, so a view opens on what happened; the first outcome when nothing has yet. */
 export function defaultOutcome(counters, outcomes = JOB_VIEWS.scan.outcomes) {
-    return (outcomes.find(([group]) => (counters[group] ?? 0) > 0) ?? outcomes[0])[0];
+    return (outcomes.find(([group]) => (counters[group] ?? 0) > 0) ?? outcomes[0])?.[0] ?? null;
 }
 
 function section(title, id) {
@@ -179,9 +187,10 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
     const viewer = createMediaViewer({ api, container: main, parentScope: scope, inspectionController: inspection,
         sender: createPromptSender({ scope }), onSelect: image => inspection.select(image) });
 
-    /** The tiles and the outcome list of the job's kind, built once. */
-    function layout(kind) {
-        view = jobView(kind); // D89
+    /** The tiles and the outcome list of the job's kind, built once (D89, D111: only the groups it counts). */
+    function layout(job) {
+        view = viewOf(job);
+        if (!view.tiles.length) stats.remove();
         for (const [group, label, tone] of view.tiles) {
             const tile = element('div', undefined, tone ? `stat ${tone}` : 'stat');
             const value = element('dd', '0');
@@ -203,10 +212,11 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
     function showBody(job) {
         if (mounted) return;
         mounted = true;
-        if (!view) layout(job.kind);
+        if (!view) layout(job);
         missing.remove();
         parts.actions.append(download, cancel);
-        main.append(progressPart.node, resultsPart.node, log);
+        // A kind without outcome groups has no Results to browse (D111).
+        main.append(...(view.outcomes.length ? [progressPart.node, resultsPart.node, log] : [progressPart.node, log]));
         selected ??= defaultOutcome(job.counters, view.outcomes);
     }
 
@@ -242,6 +252,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
     }
 
     function renderRows(job) {
+        if (!view.outcomes.length) return;
         const rows = detail?.outcomes[selected] ?? [];
         const openable = view.openable.has(selected);
         const wanted = filterRows(rows.map((row, index) => ({ ...row, index })), input.value);
@@ -279,7 +290,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
         if (hasReviewItems(job)) { if (!review.isConnected) download.after(review); } // D87
         else review.remove();
         // Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel.
-        if (job.state === 'running' && job.cancellable) {
+        if (canCancel(job)) {
             if (!cancel.isConnected) parts.actions.append(cancel);
         } else if (cancel.isConnected) {
             // A focused Cancel that goes away hands its focus to the job's heading, never to the page body.
@@ -287,8 +298,8 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
             cancel.remove();
         }
         status.textContent = progressLine(job);
-        bar.value = job.progress;
-        bar.textContent = `${job.progress}%`;
+        // D111: no value while the total is unknown; a job that ended with nothing counted shows how it ended.
+        showProgress(bar, job.total > 0 || job.state !== 'finished' ? job : { ...job, progress: 100 });
         bar.setAttribute('aria-label', scan ? 'Scan progress' : `${job.title} progress`);
         for (const [group, value] of tileValues) value.textContent = number(job.counters[group] ?? 0);
         for (const [group, button] of buttons) {

web/js/workspace/activity-kinds.mjs

@@ -0,0 +1,189 @@
+/**
+ * Per-kind Activity descriptors (#249 System S5, D111): one table that says, for each job kind, what Activity shows.
+ *
+ * `kindOf(kind)` → `{pillVerb, startToast(job), finishToast(job), line(job), kept(job), scope(job), tiles, outcomes,
+ * openable, log, cancelLabel}`:
+ * - `pillVerb`: the pill's verb ("Scanning"); null shows the job's title.
+ * - `startToast` / `finishToast`: what a toast says when this tab starts the job and when it ends.
+ * - `line`: the counters as one line ("N added · M errors"); `kept`: what a cancelled job kept ("N added"); `scope`:
+ *   what the job covers ("2 output folders").
+ * - `tiles` (`[group, label, tone]`), `outcomes` (`[group, label]`) and `openable` (rows that open the viewer, D57).
+ * - `log` and `cancelLabel` (`cancel` is the same label under the job view's older name).
+ *
+ * `scan` is exactly the D56 view and `image-tags` the D89 one. `thumbnails` counts Generated (green), Skipped and
+ * Errors (red). Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
+ * the generic descriptor: title-based copy and no tiles or outcomes, so it never shows another kind's zeros (S1 review
+ * M6, S2 review M3). `viewOf(job)` keeps only the groups the job itself counts, so a hold (no groups) has none either.
+ *
+ * Progress is indeterminate while a job runs without a known total (`indeterminate`): the `<progress>` has no value
+ * and the pill shows no percentage, never a made-up 0%. A job shows Cancel only while it runs and the server can stop
+ * it (`canCancel`).
+ */
+import { noun, countOf } from './plural.mjs';
+
+// Grouped as the server writes its status lines ("8,240 of 19,622"), whatever the browser's locale.
+const number = value => value.toLocaleString('en-US');
+const counted = (job, group, word) => `${number(job.counters?.[group] ?? 0)} ${word}`;
+const errorsOf = job => `${number(job.counters?.errors ?? 0)} ${noun(job.counters?.errors ?? 0, 'error')}`;
+const whole = value => Number.isSafeInteger(value) && value >= 0;
+const tiles = rows => Object.freeze(rows.map(Object.freeze));
+const groups = (...names) => Object.freeze(new Set(names));
+
+/** "1,420 added". */
+export const addedOf = job => counted(job, 'added', 'added');
+/** "12 tagged" (D89). */
+export const taggedOf = job => counted(job, 'tagged', 'tagged');
+/** "3 generated" (D111). */
+export const generatedOf = job => counted(job, 'generated', 'generated');
+
+/** "2 output folders": what a scan or a thumbnail job walks. */
+function foldersOf(job) {
+    const folders = job.scope?.output_folders;
+    return whole(folders) ? countOf(folders, 'output folder') : '';
+}
+
+/** "4 images" (D89), grouped as the server writes its lines. */
+function imagesOf(job) {
+    const images = job.scope?.images;
+    return whole(images) ? `${number(images)} ${noun(images, 'image')}` : '';
+}
+
+const titleStarted = job => `${job.title} started.`;
+
+function titleFinished(job) {
+    if (job.state === 'cancelled') return `${job.title} was cancelled.`;
+    if (job.state === 'failed') return `${job.title} failed.`;
+    return `${job.title} finished.`;
+}
+
+/** "<Noun> finished · <line>", "<Noun> cancelled · <kept>", "<Noun> failed · <line>". */
+const namedFinish = (name, kept, line) => job => {
+    if (job.state === 'cancelled') return `${name(job)} cancelled · ${kept(job)}`;
+    return `${name(job)} ${job.state === 'failed' ? 'failed' : 'finished'} · ${line(job)}`;
+};
+
+const nothing = () => '';
+const GENERIC = Object.freeze({
+    pillVerb: null, startToast: titleStarted, finishToast: titleFinished, line: nothing, kept: nothing,
+    scope: foldersOf, tiles: tiles([]), outcomes: tiles([]), openable: groups(), log: 'Job log', cancelLabel: 'Cancel job',
+    cancel: 'Cancel job',
+});
+
+const scanLine = job => `${addedOf(job)} · ${errorsOf(job)}`;
+const tagLine = job => `${taggedOf(job)} · ${errorsOf(job)}`;
+const thumbnailLine = job => `${generatedOf(job)} · ${errorsOf(job)}`;
+// D85: an image tagging job says which of its two modes started.
+const IMAGE_TAG_START_TOASTS = Object.freeze({ add: 'Tagging started.', auto: 'Auto-tagging started.' });
+
+/** Every kind with a descriptor of its own. */
+export const KINDS = Object.freeze({
+    __proto__: null,
+    // D56: the scan's tiles and outcome list in the mockup's orders; what got added and what went wrong first.
+    scan: Object.freeze({
+        ...GENERIC,
+        pillVerb: 'Scanning',
+        startToast: () => 'Scan started.',
+        finishToast: namedFinish(() => 'Scan', addedOf, scanLine),
+        line: scanLine,
+        kept: addedOf,
+        tiles: tiles([['added', 'Added to Library', 'added'], ['known', 'Already known', ''],
+            ['review', 'Needs review', 'warn'], ['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad']]),
+        outcomes: tiles([['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
+            ['no_metadata', 'No usable metadata'], ['known', 'Already known']]),
+        openable: groups('added', 'known'),
+        log: 'Scan log',
+        cancelLabel: 'Cancel scan',
+        cancel: 'Cancel scan',
+    }),
+    // D89: Tagged (green), Unchanged, Skipped (warning) and Errors (red); Tagged and Unchanged rows open the viewer.
+    'image-tags': Object.freeze({
+        ...GENERIC,
+        pillVerb: 'Tagging',
+        startToast: job => IMAGE_TAG_START_TOASTS[job.scope?.mode] ?? titleStarted(job),
+        finishToast: namedFinish(job => job.title, taggedOf, tagLine),
+        line: tagLine,
+        kept: taggedOf,
+        scope: imagesOf,
+        tiles: tiles([['tagged', 'Tagged', 'added'], ['unchanged', 'Unchanged', ''], ['skipped', 'Skipped', 'warn'],
+            ['errors', 'Errors', 'bad']]),
+        outcomes: tiles([['tagged', 'Tagged'], ['errors', 'Errors'], ['skipped', 'Skipped'], ['unchanged', 'Unchanged']]),
+        openable: groups('tagged', 'unchanged'),
+        log: 'Tagging log',
+        cancelLabel: 'Cancel tagging',
+        cancel: 'Cancel tagging',
+    }),
+    // D111: Generated (green), Skipped (already fresh) and Errors (red); a row with a preview opens the viewer.
+    thumbnails: Object.freeze({
+        ...GENERIC,
+        pillVerb: 'Generating thumbnails',
+        finishToast: namedFinish(() => 'Thumbnails', generatedOf, thumbnailLine),
+        line: thumbnailLine,
+        kept: generatedOf,
+        tiles: tiles([['generated', 'Generated', 'added'], ['skipped', 'Skipped', ''], ['errors', 'Errors', 'bad']]),
+        outcomes: tiles([['generated', 'Generated'], ['errors', 'Errors'], ['skipped', 'Skipped']]),
+        openable: groups('generated', 'skipped'),
+    }),
+});
+
+/** The descriptor of a job `kind`; any kind without one gets the generic descriptor. */
+export function kindOf(kind) {
+    return (typeof kind === 'string' && KINDS[kind]) || GENERIC;
+}
+
+const countsOf = job => job.counters && typeof job.counters === 'object' ? job.counters : {};
+
+/**
+ * A legacy-page operation held on the registry (D112): not cancellable and counting no outcome groups. Whatever its
+ * kind (the gallery's thumbnail clear is a `thumbnails` hold), it reads as the generic job under its own title.
+ */
+export const held = job => job.cancellable === false && Object.keys(countsOf(job)).length === 0;
+
+/**
+ * The descriptor for one job: its kind's, keeping only the outcome groups the job counts; a held job's is the generic
+ * one.
+ */
+export function viewOf(job) {
+    if (held(job)) return GENERIC;
+    const view = kindOf(job.kind);
+    const counts = countsOf(job);
+    const has = ([group]) => Object.hasOwn(counts, group);
+    if (view.tiles.every(has) && view.outcomes.every(has)) return view;
+    return Object.freeze({ ...view, tiles: tiles(view.tiles.filter(has)), outcomes: tiles(view.outcomes.filter(has)),
+        openable: groups(...[...view.openable].filter(group => Object.hasOwn(counts, group))) });
+}
+
+/** A running job whose total the server has not reported (yet) has no percentage. */
+export function indeterminate(job) {
+    return job.state === 'running' && !(job.total > 0);
+}
+
+/** Set a `<progress>` to the job's: its value and "N%", or no value at all while indeterminate. */
+export function showProgress(node, job) {
+    if (indeterminate(job)) {
+        node.removeAttribute('value');
+        node.textContent = '';
+        return;
+    }
+    node.value = job.progress;
+    node.textContent = `${job.progress}%`;
+}
+
+/** Truth rule: Cancel only while the job runs and the server can stop it (never on a held legacy job). */
+export function canCancel(job) {
+    return job.state === 'running' && job.cancellable !== false;
+}
+
+/** The pill's text: "Scanning · 42% · 8,240 / 19,622"; the file counts once the total is known, no "%" before. */
+export function pillParts(job) {
+    const verb = viewOf(job).pillVerb ?? job.title;
+    const label = indeterminate(job) ? verb : `${verb} · ${job.progress}%`;
+    const files = job.total > 0 ? ` · ${number(job.processed)} / ${number(job.total)}` : '';
+    return { label, files, text: label + files };
+}
+
+/** The toast after this tab starts a job. */
+export const startToast = job => viewOf(job).startToast(job);
+/** The toast when a job ends. */
+export const finishToast = job => viewOf(job).finishToast(job);
+/** The counters as one line; '' for a kind without one. */
+export const outcomeLine = job => viewOf(job).line(job);

web/js/workspace/activity-store.mjs

@@ -12,7 +12,9 @@
  *   the finish toast follows any job this store saw running when it ends.
  */
 import { showWorkspaceToast } from './toast.mjs';
-import { noun } from './plural.mjs';
+import { KINDS, kindOf, finishToast, startToast } from './activity-kinds.mjs';
+
+export { addedOf, taggedOf, finishToast, startToast, pillParts, outcomeLine } from './activity-kinds.mjs';
 
 export const JOBS_URL = '/prompt_manager/jobs';
 export const BACKOFF_FIRST_MS = 1000;
@@ -23,51 +25,13 @@ export const LOST_DETAIL = 'This job ended when ComfyUI restarted.';
 const SERVER_STATES = new Set(['running', 'finished', 'cancelled', 'failed']);
 const FINAL_STATES = new Set(['finished', 'cancelled', 'failed']);
 const JOB_ID = /^[0-9a-f]{32}$/;
-// The pill's verb for each job kind; a kind without one shows its title (D89: image tagging jobs read "Tagging").
-const PILL_VERBS = Object.freeze({ scan: 'Scanning', 'image-tags': 'Tagging' });
-const START_TOASTS = Object.freeze({ scan: 'Scan started.' });
-
-/** The scan's job view (D56): its counter tiles `[group, label, tone]` and outcome list in the mockup's orders. */
-const SCAN_VIEW = Object.freeze({
-    tiles: Object.freeze([
-        ['added', 'Added to Library', 'added'], ['known', 'Already known', ''], ['review', 'Needs review', 'warn'],
-        ['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad'],
-    ].map(Object.freeze)),
-    // What got added and what went wrong first.
-    outcomes: Object.freeze([
-        ['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
-        ['no_metadata', 'No usable metadata'], ['known', 'Already known'],
-    ].map(Object.freeze)),
-    // Rows that open the file in the viewer (D57); the others are plain text.
-    openable: Object.freeze(new Set(['added', 'known'])),
-    log: 'Scan log',
-    cancel: 'Cancel scan',
-});
-
 /**
- * Each job kind's view: its tiles, outcome list, openable rows, log label and cancel label. D89: an image tagging job
- * (`image-tags`) counts Tagged (green), Unchanged, Skipped (warning) and Errors (red), lists what got tagged and what
- * went wrong first, and opens Tagged and Unchanged rows in the viewer like the scan's (D57).
+ * Each job kind's copy and view live in one table (activity-kinds.mjs, D111): the scan's (D56), image tagging's (D89),
+ * thumbnails' and the generic one. The names below are that table's, kept for this store's importers.
  */
-export const JOB_VIEWS = Object.freeze({
-    scan: SCAN_VIEW,
-    'image-tags': Object.freeze({
-        tiles: Object.freeze([['tagged', 'Tagged', 'added'], ['unchanged', 'Unchanged', ''], ['skipped', 'Skipped', 'warn'],
-            ['errors', 'Errors', 'bad']].map(Object.freeze)),
-        outcomes: Object.freeze([['tagged', 'Tagged'], ['errors', 'Errors'], ['skipped', 'Skipped'],
-            ['unchanged', 'Unchanged']].map(Object.freeze)),
-        openable: Object.freeze(new Set(['tagged', 'unchanged'])),
-        log: 'Tagging log',
-        cancel: 'Cancel tagging',
-    }),
-});
-// Another kind keeps the scan's tiles and outcomes under the generic labels.
-const GENERIC_VIEW = Object.freeze({ ...SCAN_VIEW, log: 'Job log', cancel: 'Cancel job' });
-
+export const JOB_VIEWS = KINDS;
 /** The view of a job `kind`; an unknown kind gets the generic one. */
-export function jobView(kind) {
-    return Object.hasOwn(JOB_VIEWS, kind) ? JOB_VIEWS[kind] : GENERIC_VIEW;
-}
+export const jobView = kindOf;
 
 /** The wait before the `attempt`-th re-read (0-based): 1 s, 2 s, 4 s … never more than 30 s. */
 export function backoffDelay(attempt) {
@@ -108,57 +72,6 @@ export function parseJob(raw) {
     };
 }
 
-// Grouped as the server writes its status lines ("8,240 of 19,622"), whatever the browser's locale.
-const number = value => value.toLocaleString('en-US');
-const errorsOf = job => `${number(job.counters.errors ?? 0)} ${noun(job.counters.errors ?? 0, 'error')}`;
-/** "1,420 added". */
-export const addedOf = job => `${number(job.counters.added ?? 0)} added`;
-
-/**
- * The toast when a job ends: "Scan finished · N added · M errors", "Scan cancelled · N added"; an image tagging job
- * the same way with its own title and count (D89): "Auto-tag images finished · N tagged · M errors".
- */
-export function finishToast(job) {
-    if (job.kind === 'scan') {
-        if (job.state === 'cancelled') return `Scan cancelled · ${addedOf(job)}`;
-        if (job.state === 'failed') return `Scan failed · ${addedOf(job)} · ${errorsOf(job)}`;
-        return `Scan finished · ${addedOf(job)} · ${errorsOf(job)}`;
-    }
-    if (job.kind === 'image-tags') {
-        if (job.state === 'cancelled') return `${job.title} cancelled · ${taggedOf(job)}`;
-        return `${job.title} ${job.state === 'failed' ? 'failed' : 'finished'} · ${outcomeLine(job)}`;
-    }
-    if (job.state === 'cancelled') return `${job.title} was cancelled.`;
-    if (job.state === 'failed') return `${job.title} failed.`;
-    return `${job.title} finished.`;
-}
-
-// D85: an image tagging job says which of its two modes started.
-const IMAGE_TAG_START_TOASTS = Object.freeze({ add: 'Tagging started.', auto: 'Auto-tagging started.' });
-
-/** The toast after this tab starts a job. */
-export function startToast(job) {
-    if (job.kind === 'image-tags' && IMAGE_TAG_START_TOASTS[job.scope.mode]) return IMAGE_TAG_START_TOASTS[job.scope.mode];
-    return START_TOASTS[job.kind] ?? `${job.title} started.`;
-}
-
-/** The pill's text: "Scanning · 42% · 8,240 / 19,622"; the file counts only once the total is known. */
-export function pillParts(job) {
-    const label = `${PILL_VERBS[job.kind] ?? job.title} · ${job.progress}%`;
-    const files = job.total > 0 ? ` · ${number(job.processed)} / ${number(job.total)}` : '';
-    return { label, files, text: label + files };
-}
-
-/** "12 tagged" (D89). */
-export const taggedOf = job => `${number(job.counters.tagged ?? 0)} tagged`;
-
-/** The counters as one line: "N added · M errors" for a scan, "N tagged · M errors" for image tagging (D89). */
-export function outcomeLine(job) {
-    if (job.kind === 'scan') return `${addedOf(job)} · ${errorsOf(job)}`;
-    if (job.kind === 'image-tags') return `${taggedOf(job)} · ${errorsOf(job)}`;
-    return '';
-}
-
 function defaultConnect(url) {
     const resolve = globalThis.ApiClient?.resolveUrl ?? (value => value);
     return new EventSource(resolve(url));

web/js/workspace/activity.mjs

@@ -14,13 +14,15 @@
  * - D87 (plan D64): an ended scan that recorded Needs review items carries the concept's **Review metadata**
  *   (`go-review`, app.js a738) after its line, which opens `#/review`; its chip still names its state (D54).
  * - D89: an image tagging job's line names its images and, once ended, "N tagged · M errors".
+ * - D111: what a line says (scope, counters, what a cancel kept) is the job kind's descriptor (activity-kinds.mjs); a
+ *   job without a known total shows an indeterminate `<progress>`, and only a job the server can stop shows Cancel.
  */
 import { createButton } from '../shared/toolkit/core.js';
 import { navigateTo } from './state.mjs';
 import { pageHeading } from './heading.mjs';
 import { showWorkspaceToast } from './toast.mjs';
-import { countOf, noun } from './plural.mjs';
-import { pillParts, outcomeLine, addedOf, taggedOf, LOST_DETAIL } from './activity-store.mjs';
+import { pillParts, outcomeLine, LOST_DETAIL } from './activity-store.mjs';
+import { viewOf, canCancel, showProgress, indeterminate } from './activity-kinds.mjs';
 
 export const ACTIVITY_DESCRIPTION = 'Scans, tagging, downloads and maintenance share one history.';
 export const EMPTY_ACTIVITY = 'No jobs have run since ComfyUI started. Scans and other long operations appear here.';
@@ -60,20 +62,9 @@ export function clockTime(seconds) {
 
 const join = parts => parts.filter(Boolean).join(' · ');
 
-const whole = value => Number.isSafeInteger(value) && value >= 0;
-
-/** What a job covers: "2 output folders" for a scan, "4 images" for image tagging (D89). */
-function scopeLine(job) {
-    // D89: grouped as the server writes its lines ("19,622 images").
-    const images = job.scope?.images;
-    if (job.kind === 'image-tags') return whole(images) ? `${images.toLocaleString('en-US')} ${noun(images, 'image')}` : '';
-    const folders = job.scope?.output_folders;
-    return whole(folders) ? countOf(folders, 'output folder') : '';
-}
-
-/** What a cancelled job kept: "N added" for a scan, "N tagged" for image tagging (D89). */
-const KEPT = Object.freeze({ __proto__: null, scan: addedOf, 'image-tags': taggedOf }); // D89
-const keptOf = job => KEPT[job.kind]?.(job) ?? '';
+/** What a job covers ("2 output folders", "4 images") and what a cancelled one kept ("N added"), by kind (D111). */
+const scopeLine = job => viewOf(job).scope(job);
+const keptOf = job => viewOf(job).kept(job);
 
 /** The item's line under its title, by state. */
 export function jobDetail(job) {
@@ -119,7 +110,10 @@ export function mountActivityTopbar({ store, scope, doc = document }) {
         const parts = pillParts(job);
         label.textContent = parts.label;
         files.textContent = parts.files;
-        fill.style.width = `${job.progress}%`;
+        // D111 (S5 review M5): no total, no fill: the bar sweeps instead of showing an empty determinate 0%.
+        const unknown = indeterminate(job);
+        bar.classList.toggle('indeterminate', unknown);
+        fill.style.width = unknown ? '' : `${job.progress}%`;
         pill.href = `#/activity/${job.id}`;
         pill.setAttribute('aria-label', parts.text);
     };
@@ -167,15 +161,14 @@ function itemFor(job, scope) {
         // parentNode, not isConnected: on the first render the item is not in the page yet, so a job that had already
         // ended when Activity opened kept its Cancel job (and a running one's progress moved after its actions).
         if (running) {
-            progress.value = next.progress;
-            progress.textContent = `${next.progress}%`;
+            showProgress(progress, next); // D111: no value while the total is unknown
             progress.setAttribute('aria-label', `${next.title} progress`);
             if (!progress.parentNode) item.insertBefore(progress, actions.parentNode ? actions : null);
         } else {
             progress.remove();
         }
         // Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel job.
-        if (running && next.cancellable) {
+        if (canCancel(next)) {
             if (!actions.parentNode) item.append(actions);
         } else if (actions.parentNode) {
             // A focused Cancel job that goes away hands its focus to the item's title link, never to the page body.

web/js/workspace/file-inspector.mjs

@@ -318,18 +318,25 @@ export function mountFileInspector(container, {api, sender, parentScope, variant
         });
         return button;
     }
-    /** The concept's plain cover; the card image opens the viewer (D19). Video keeps its own element. */
+    /**
+     * The concept's plain cover; the card image opens the viewer (D19). #249: it is the file's preview (a video's
+     * is a JPEG frame) whenever the listing gave one; without one the original, and a video keeps its own element.
+     */
     function cover(record) {
-        const {file} = record, video = file?.media_type === 'video';
-        const node = el(video ? 'video' : 'img', undefined, 'detail-cover'); node.src = record.download.url;
-        if (video) {
-            node.muted = true; node.playsInline = true; node.preload = 'metadata';
-            contentScope.cleanup(() => {node.pause(); node.removeAttribute('src'); node.load();});
-        } else {
-            node.alt = ''; node.decoding = 'async';
-            if (file?.width && file?.height) {node.width = file.width; node.height = file.height;}
-            contentScope.cleanup(() => node.removeAttribute('src'));
-        }
+        const {file} = record, preview = record.preview?.url, isVideo = file?.media_type === 'video';
+        if (isVideo && !preview) return videoCover(record);
+        const node = el('img', undefined, 'detail-cover'); node.src = preview || record.download.url;
+        node.alt = ''; node.decoding = 'async';
+        if (file?.width && file?.height) {node.width = file.width; node.height = file.height;}
+        contentScope.cleanup(() => node.removeAttribute('src'));
+        // #249: a video whose preview cannot be loaded swaps back to its own element.
+        if (isVideo) contentScope.listen(node, 'error', () => { if (node.isConnected) node.replaceWith(videoCover(record)); }, {once:true});
+        return node;
+    }
+    function videoCover(record) {
+        const node = el('video', undefined, 'detail-cover'); node.src = record.download.url;
+        node.muted = true; node.playsInline = true; node.preload = 'metadata';
+        contentScope.cleanup(() => {node.pause(); node.removeAttribute('src'); node.load();});
         return node;
     }
     /** "File & provenance": the file facts, the D21 prompt evidence, the occurrence and the saved association (D66 for a local file). */

web/js/workspace/history-media.mjs

@@ -64,7 +64,10 @@ export function toHistoryInspectionRecord(result, target, extractFacts) {
     const path = [target.reference.subfolder, target.reference.filename].filter(Boolean).join('/');
     const identity = {kind:'reported-location', key:target.location_key, path:target.reference.path_style === 'windows' ? path.replaceAll('\\', '/') : path,
         occurrence:{...target.occurrence}, history_epoch:target.history_epoch, current_scope:target.current_scope, reference:{...target.reference}};
-    return projectInspectionPayload(result, identity, {url:urlFor(target, 'url', 'media'), filename:target.filename}, extractFacts);
+    const record = projectInspectionPayload(result, identity, {url:urlFor(target, 'url', 'media'), filename:target.filename}, extractFacts);
+    // #249: the inspector's cover is the recorded output's preview (made by the server on first view).
+    record.preview = {url:urlFor(target, 'thumbnail_url', 'thumbnail')};
+    return record;
 }
 export const reportedLocationAdapter = Object.freeze({
     validate,

web/js/workspace/image-card.mjs

@@ -17,19 +17,29 @@ function el(tag, text, className) {
     return node;
 }
 
-/** The card's media: a video's first frame when it has no thumbnail, else the thumbnail or the image. */
+/** A video's own element (its first frame), for a video without a usable preview. */
+function videoMedia(image) {
+    const node = el('video');
+    node.src = window.ApiClient.resolveUrl(image.url);
+    node.muted = true; node.playsInline = true; node.preload = 'metadata';
+    return node;
+}
+
+/**
+ * The card's media: the thumbnail (a video's is its preview frame), else the image, else a video's own element.
+ * #249: a video whose preview cannot be loaded swaps back to its own element.
+ */
 export function cardMedia(image, lazy = false) {
     const { resolveUrl } = window.ApiClient;
-    const video = image.is_video && !image.thumbnail_url;
-    const node = el(video ? 'video' : 'img');
+    if (image.is_video && !image.thumbnail_url) return videoMedia(image);
+    const node = el('img');
     node.src = resolveUrl(image.thumbnail_url || image.url);
-    if (video) { node.muted = true; node.playsInline = true; node.preload = 'metadata'; }
-    else {
-        // The concept card image: alt is the card title, with the listing's intrinsic size when known.
-        node.alt = cardTitle(image); node.decoding = 'async'; if (lazy) node.loading = 'lazy';
-        if ([image.width, image.height].every(value => Number.isFinite(value) && value > 0)) {
-            node.width = image.width; node.height = image.height;
-        }
+    // replaceWith does nothing for a card no longer in the page.
+    if (image.is_video) node.addEventListener('error', () => node.replaceWith(videoMedia(image)), { once: true });
+    // The concept card image: alt is the card title, with the listing's intrinsic size when known.
+    node.alt = cardTitle(image); node.decoding = 'async'; if (lazy) node.loading = 'lazy';
+    if ([image.width, image.height].every(value => Number.isFinite(value) && value > 0)) {
+        node.width = image.width; node.height = image.height;
     }
     return node;
 }

web/js/workspace/inspection-state.mjs

@@ -44,6 +44,9 @@ export function inspectPath(image) {
     return `/prompt_manager/images/inspect/${image.path.split('/').map(encodeURIComponent).join('/')}?${query}`;
 }
 
+/** One path segment as the server quotes it (Python `quote(..., safe="/")` also escapes `!'()*`, #249 M11). */
+const quoteSegment = segment => encodeURIComponent(segment).replace(/[!'()*]/g, c => `%${c.charCodeAt(0).toString(16).toUpperCase()}`);
+
 function downloadFor(image, required) {
     try {
         if (typeof image.url !== 'string' || !image.url) throw new TypeError('Missing original media URL');
@@ -55,7 +58,7 @@ function downloadFor(image, required) {
         const sameOrigin = 'Origin' in globalThis ?
             globalThis.Origin.from(url).isSameOrigin(globalThis.Origin.from(reference)) :
             url.origin !== 'null' && url.origin === reference.origin;
-        const expected = new URL(resolveUrl(`/prompt_manager/images/serve/${image.path.split('/').map(encodeURIComponent).join('/')}`), reference);
+        const expected = new URL(resolveUrl(`/prompt_manager/images/serve/${image.path.split('/').map(quoteSegment).join('/')}`), reference);
         const scope = descriptorScope(image);
         if (!sameOrigin || !['http:', 'https:'].includes(url.protocol) || url.username || url.password || url.hash ||
             url.pathname !== expected.pathname || url.searchParams.getAll('root').length !== 1 || url.searchParams.get('root') !== String(image.root_index) ||
@@ -68,6 +71,39 @@ function downloadFor(image, required) {
     }
 }
 
+// The server's video extensions (py/api/inspection_service.py VIDEO_EXTENSIONS): their preview is a JPEG.
+const VIDEO_EXTENSIONS = new Set(['.mp4', '.webm', '.avi', '.mov', '.mkv', '.m4v', '.wmv']);
+
+/** `thumbnails/<path>_thumb<ext>` for a listed file, as the server names its preview (images.media_thumbnail_rel). */
+function previewPath(path) {
+    const slash = path.lastIndexOf('/'), dot = path.lastIndexOf('.');
+    const cut = dot > slash ? dot : path.length, extension = path.slice(cut).toLowerCase();
+    return `thumbnails/${path.slice(0, cut)}_thumb${VIDEO_EXTENSIONS.has(extension) ? '.jpg' : extension}`;
+}
+
+/**
+ * #249: the listing's thumbnail of `image` as `{url}` for the inspector's cover, or null. Only the same-origin
+ * serve URL of this file's own preview under the same root and source scope is used; anything else keeps the
+ * original (a missing preview is made by the server on first view).
+ */
+function previewFor(image) {
+    try {
+        if (typeof image.thumbnail_url !== 'string' || !image.thumbnail_url) return null;
+        const location = globalThis.location || globalThis.window?.location;
+        const reference = new URL(location?.href || 'http://localhost/');
+        const resolveUrl = globalThis.window?.ApiClient?.resolveUrl || globalThis.ApiClient?.resolveUrl || (url => url);
+        const value = resolveUrl(image.thumbnail_url), url = new URL(value, reference);
+        const expected = new URL(resolveUrl(`/prompt_manager/images/serve/${previewPath(image.path).split('/').map(quoteSegment).join('/')}`), reference);
+        const scope = descriptorScope(image);
+        if (url.origin !== reference.origin || url.username || url.password || url.hash || url.pathname !== expected.pathname ||
+            url.searchParams.getAll('root').length !== 1 || url.searchParams.get('root') !== String(image.root_index) ||
+            url.searchParams.getAll('root_scope').length > 1 || (scope !== null && url.searchParams.get('root_scope') !== scope)) return null;
+        return { url: value };
+    } catch {
+        return null;
+    }
+}
+
 export function originalMediaDownload(image) {
     validateIdentity(image);
     return downloadFor(image, true);
@@ -284,16 +320,18 @@ export function toInspectionRecord(result, image, extractFacts = globalThis.Comf
     if (!legacy && result.inspection_version !== 1) throw new TypeError('Unsupported inspection version');
     if (!legacy && result.success !== true) throw new TypeError('Invalid inspection success status');
     if ((!legacy || result.image !== undefined) && (!object(result.image) || result.image.root_index !== image.root_index || result.image.path !== image.path || result.image.filename !== image.path.split('/').at(-1))) throw new TypeError('Mismatched inspection identity');
-    return projectInspectionPayload(result,
+    const record = projectInspectionPayload(result,
         { kind: 'rooted-file', key: imageKey(image), root_index: image.root_index, path: image.path, root_scope: scope },
         downloadFor(image, !legacy || scope !== null), extractFacts, legacy, parser);
+    record.preview = record.download ? previewFor(image) : null;
+    return record;
 }
 
 /** Shared bounded payload projection; adapters validate identity before entering here. */
 export function projectInspectionPayload(result, identity, download, extractFacts = globalThis.ComfyMetadata?.extractGenerationFacts || globalThis.window?.ComfyMetadata?.extractGenerationFacts, legacy = false, parser = comfyMetadata()) {
     const record = {
         identity, file: null, prompts: promptsFor(result.prompts), library: null, metadata: null,
-        generation: unavailableFacts(), historyTarget: null, download, imageTags: null,
+        generation: unavailableFacts(), historyTarget: null, download, imageTags: null, preview: null,
     };
     if (legacy) return record;
     record.imageTags = imageTagsFor(result.image_tags);

web/js/workspace/system-overview.mjs

@@ -7,6 +7,7 @@
  *   N issues / Incomplete); the review, backup, thumbnail and sources rows say what the server reports, and nothing
  *   reads 0 before the read answers: a load shows a status line, a failure an alert and Retry. The thumbnail row is
  *   its own read of `/images/thumbnails/status` (a walk of every source): "Checking thumbnails…" until it answers.
+ *   D99: its Review thumbnails opens the same "Thumbnail maintenance" dialog as Sources (thumbnail-review.mjs).
  * - D96: the Statistics dialog lists the real runs and average rating per category, then "Library totals".
  * - D95: the "Health report" lists every check the server ran, Missing files per image source, with a chip per real
  *   result and when it was checked. The check runs from Maintenance's notice ("Run health checks", D123) as a
@@ -20,6 +21,7 @@ import { closeWorkspaceDialog, openContentDialog } from './workspace-dialogs.mjs
 import { showWorkspaceToast } from './toast.mjs';
 import { noun } from './plural.mjs';
 import { operationRow } from './system-controls.mjs';
+import { openThumbnailReview } from './thumbnail-review.mjs';
 
 export const OVERVIEW_URL = '/prompt_manager/system/overview';
 export const STATISTICS_URL = '/prompt_manager/system/statistics';
@@ -86,7 +88,9 @@ export function thumbnailTitle(status, sources) {
     const missing = status?.pending;
     if (!Number.isSafeInteger(missing) || sources?.effective === 0) return 'Thumbnail status unavailable';
     if (missing === 0) return 'No missing thumbnails';
-    return `${number(missing)} missing ${noun(missing, 'thumbnail')}`;
+    // #249: files whose automatic preview could not be made this session (they show the original).
+    const failed = Number.isSafeInteger(status.failed) && status.failed > 0 ? ` · ${number(status.failed)} failed` : '';
+    return `${number(missing)} missing ${noun(missing, 'thumbnail')}${failed}`;
 }
 
 const SOURCES_SHARED = 'Monitoring and enrichment share one source list.';
@@ -317,7 +321,7 @@ function surface(heading, rows) {
 }
 
 /** The Overview's nodes, and its thumbnail row (painted by its own read). */
-function overviewNodes(overview, { scope, navigate, openStatistics }) {
+function overviewNodes(overview, { scope, navigate, openStatistics, openThumbnails }) {
     const go = section => () => navigate(`#/system/${section}`);
     const review = operationRow({ scope, title: ambiguousTitle(overview.needs_review),
         description: Number.isSafeInteger(overview.needs_review?.pending)
@@ -325,12 +329,12 @@ function overviewNodes(overview, { scope, navigate, openStatistics }) {
         label: 'Review metadata', onClick: () => navigate('#/review') });
     const [sourcesTitle, sourcesDescription] = sourcesCopy(overview.sources);
     const thumbnails = operationRow({ scope, title: THUMBNAILS_LOADING, description: 'Generate only missing previews when necessary.',
-        label: 'Review thumbnails', onClick: go('sources') });
+        label: 'Review thumbnails', onClick: openThumbnails });
     const health = surface('Library health', [
         review,
         operationRow({ scope, title: 'Backup status', description: backupText(overview.backup), label: 'View backups',
             onClick: go('backups') }),
-        // Until the thumbnail review dialog (S5), Review thumbnails opens Sources, where thumbnail generation lives.
+        // D99: Review thumbnails opens the thumbnail dialog (the concept's row starts a sample job directly).
         thumbnails,
     ]);
     const preferences = surface('Workspace preferences', [
@@ -350,7 +354,7 @@ function overviewNodes(overview, { scope, navigate, openStatistics }) {
  * answer replaces in place (the concept's metrics and columns sit directly in `.settings-layout`). A read that a newer
  * one superseded, or one answering after the section left, paints nothing.
  */
-export function mountOverview(container, { api, scope, navigate = navigateTo, notify = showWorkspaceToast,
+export function mountOverview(container, { api, activity, scope, navigate = navigateTo, notify = showWorkspaceToast,
     latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) {
     const overviewRead = latestOnly({ lifetime: scope.signal });
     const statisticsRead = latestOnly({ lifetime: scope.signal });
@@ -392,7 +396,8 @@ export function mountOverview(container, { api, scope, navigate = navigateTo, no
         try {
             const overview = await overviewRead(signal => api.get(OVERVIEW_URL, { signal, cache: 'no-store' }));
             if (overview === STALE || scope.signal.aborted) return;
-            const { nodes, thumbnails } = overviewNodes(overview, { scope, navigate, openStatistics });
+            const { nodes, thumbnails } = overviewNodes(overview, { scope, navigate, openStatistics,
+                openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate }) });
             thumbnailRow = thumbnails;
             sources = overview.sources;
             swap(nodes);

web/js/workspace/system-sources.mjs

@@ -14,8 +14,8 @@
  *   it to the host draft of `gallery_root_paths`; Remove takes one out. Both wait for the one Save row.
  * - "Watch for new outputs" (`monitoring_enabled`) and "Worker threads" (`worker_threads`, a range up to the server's
  *   `cpu_count`, D113) are host drafts too. "Enrich existing metadata" reviews a scan of every source.
- * - "Thumbnail generation → Review options" opens the existing gallery's thumbnail tool until its review dialog (S5),
- *   the D94 hand-off for the one capability not native yet.
+ * - "Thumbnail generation → Review options" opens "Thumbnail maintenance" (thumbnail-review.mjs, S5 Task 14), which
+ *   starts the thumbnail Activity job.
  *
  * D98: per-source Remove; an added source reads "not saved yet" until Save; the line under the list says when
  * ComfyUI's output folder is used instead; "restart pending" for a source the running monitor does not watch yet; the
@@ -28,6 +28,7 @@
 import { createButton } from '../shared/toolkit/core.js';
 import { closeWorkspaceDialog, field, openContentDialog } from './workspace-dialogs.mjs';
 import { openMetadataScan } from './activity-scan.mjs';
+import { openThumbnailReview } from './thumbnail-review.mjs';
 import { settingRow, operationRow } from './system-controls.mjs';
 import { HOST_UNAVAILABLE } from './system-appearance.mjs';
 import { showWorkspaceToast } from './toast.mjs';
@@ -36,7 +37,6 @@ import { noun } from './plural.mjs';
 const SETTINGS_URL = '/prompt_manager/settings';
 const VALIDATE_URL = '/prompt_manager/settings/validate-source';
 const RESCAN_URL = '/prompt_manager/gallery/rescan';
-const GALLERY_URL = '/prompt_manager/gallery';
 const ROOTS_KEY = 'gallery_root_paths';
 
 export const NOTICE_KEY = 'pm_update_notice_dismissed';
@@ -222,8 +222,7 @@ const controlOf = (row, id) => row.querySelector(`[data-setting="${id}"]`);
  * Activity store (scans start through it), `drafts` the System drafts.
  */
 export function mountSources(container, { api, activity, scope, drafts, notify = showWorkspaceToast, storage = defaultStorage,
-    openWindow = (url) => window.open(url, '_blank', 'noopener'), latestOnly = window.ApiClient.latestOnly,
-    STALE = window.ApiClient.STALE }) {
+    latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) {
     const statusRead = latestOnly({ lifetime: scope.signal });
     // undefined while read, null when the read failed, else GET /settings' `sources`.
     let sources;
@@ -270,9 +269,8 @@ export function mountSources(container, { api, activity, scope, drafts, notify =
     surface.append(monitoring, workers,
         operationRow({ scope, title: 'Enrich existing metadata', description: 'Resolve both prompts, deduplicate recipes, preserve uncertain links.',
             label: 'Review scan', onClick: scanAll }),
-        // D94: the gallery keeps thumbnail generation until its review dialog (S5).
         operationRow({ scope, title: 'Thumbnail generation', description: 'Generate missing previews; rebuilding all is a separate explicit option.',
-            label: 'Review options', onClick: () => openWindow(window.ApiClient.resolveUrl(GALLERY_URL)) }),
+            label: 'Review options', onClick: trigger => openThumbnailReview({ api, store: activity, scope, trigger }) }),
         // D98: the legacy gallery's Refresh, which rereads every source's file list.
         operationRow({ scope, title: 'Refresh files', description: 'Read every source again to find files added or removed outside ComfyUI.',
             label: 'Refresh', onClick: refresh }));

web/js/workspace/thumbnail-review.mjs

@@ -0,0 +1,172 @@
+/**
+ * Thumbnail review (#249 System S5 Task 14): the concept's "Thumbnail maintenance" dialog (`thumbnail-review`, app.js
+ * 1433–1437), opened by Sources' "Thumbnail generation → Review options" and Overview's "Review thumbnails". It starts
+ * the `thumbnails` Activity job (S5 Task 12) with the chosen options.
+ *
+ * - Scope: "Missing thumbnails only" (`missing`, the default) or "Rebuild all (explicit opt-in)" (`rebuild`).
+ * - Quality: "Medium · optimized" / "High · original" / "Low · fastest" (`medium` / `high` / `low`), preselected from
+ *   Appearance's Image loading quality (`gallerySettings.imageQuality`, the same three values; Q3).
+ * - Preview size limit: the longest edge, a whole number from 128 to 2048 (concept default 512). A size outside that
+ *   marks the field `aria-invalid`, says so in the plan line and sends nothing; Start reports the field's validity.
+ * - D99: a `p.small[role=status]` line before the actions says what the server's plan counts for the chosen scope
+ *   (`GET /images/thumbnails/plan`, read-only): "N previews to generate · M already up to date." or, for a rebuild,
+ *   "Every preview is regenerated: N files.", plus "· R more on the next run" past the per-run cap. Each scope is read
+ *   once per dialog (the size and quality change no count). The primary is "Start" (the concept's "Simulate thumbnail
+ *   job"): it closes the dialog and starts the job through the Activity store, which toasts that it started; the job
+ *   opens in Activity, where its progress lives. A start refused because another job holds the slot toasts the D57
+ *   busy link toast and stays.
+ */
+import { createButton } from '../shared/toolkit/core.js';
+import { closeWorkspaceDialog, field, openContentDialog } from './workspace-dialogs.mjs';
+import { busyToast } from './activity-scan.mjs';
+import { readPreferences } from './preferences.mjs';
+import { showWorkspaceToast } from './toast.mjs';
+import { navigateTo } from './state.mjs';
+import { noun } from './plural.mjs';
+
+export const THUMBNAIL_PLAN_URL = '/prompt_manager/images/thumbnails/plan';
+export const THUMBNAILS_KIND = 'thumbnails';
+export const SCOPE_OPTIONS = Object.freeze([['missing', 'Missing thumbnails only'], ['rebuild', 'Rebuild all (explicit opt-in)']]);
+export const QUALITY_OPTIONS = Object.freeze([['medium', 'Medium · optimized'], ['high', 'High · original'], ['low', 'Low · fastest']]);
+export const SIZE_LIMIT = Object.freeze({ min: 128, max: 2048, value: 512 });
+export const PLAN_LOADING = 'Counting previews…';
+export const PLAN_FAILED = 'The previews could not be counted.';
+export const SIZE_INVALID = `Enter a size from ${SIZE_LIMIT.min} to ${SIZE_LIMIT.max} pixels.`;
+export const START_FAILED = 'Thumbnail generation could not start. Try again.';
+
+const QUALITIES = new Set(QUALITY_OPTIONS.map(([value]) => value));
+const count = value => value.toLocaleString('en-US');
+const isCount = value => Number.isSafeInteger(value) && value >= 0;
+
+/** The preview size for the field's text: a whole number of pixels in range, else null. */
+export function previewSize(text) {
+    if (typeof text !== 'string' || !/^\d{1,5}$/.test(text)) return null;
+    const size = Number(text);
+    return size >= SIZE_LIMIT.min && size <= SIZE_LIMIT.max ? size : null;
+}
+
+/** Q3: Image loading quality preselects the dialog's quality; anything it does not know is Medium. */
+export function preselectedQuality(imageQuality) {
+    return QUALITIES.has(imageQuality) ? imageQuality : 'medium';
+}
+
+/** D99: the plan line for the server's `{mode, targets, fresh, remaining}`; an answer without counts failed. */
+export function planText(plan) {
+    if (!plan || !['missing', 'rebuild'].includes(plan.mode) || ![plan.targets, plan.fresh, plan.remaining].every(isCount)) {
+        return PLAN_FAILED;
+    }
+    const more = plan.remaining ? ` · ${count(plan.remaining)} more on the next run` : '';
+    if (plan.mode === 'rebuild') return `Every preview is regenerated: ${count(plan.targets)} ${noun(plan.targets, 'file')}${more}.`;
+    return `${count(plan.targets)} ${noun(plan.targets, 'preview')} to generate · ${count(plan.fresh)} already up to date${more}.`;
+}
+
+/**
+ * Start the thumbnail job with `options` (`{mode, size, quality}`) and open its view; a refused or failed start says
+ * why and stays. Nothing is said once the section that started it is gone.
+ */
+export async function startThumbnails({ store, scope, options, notify = showWorkspaceToast, navigate = navigateTo }) {
+    try {
+        const result = await store.start(THUMBNAILS_KIND, options);
+        if (scope.signal.aborted) return;
+        if (result.busy !== undefined) {
+            notify(busyToast(result.busy));
+            return;
+        }
+        navigate(`#/activity/${result.job.id}`);
+    } catch {
+        if (!scope.signal.aborted) notify(START_FAILED);
+    }
+}
+
+function el(tag, text, className) {
+    const node = document.createElement(tag);
+    if (text !== undefined) node.textContent = text;
+    if (className) node.className = className;
+    return node;
+}
+
+function select(options, value) {
+    const node = el('select');
+    node.append(...options.map(([optionValue, label]) => {
+        const option = el('option', label);
+        option.value = optionValue;
+        return option;
+    }));
+    node.value = value;
+    return node;
+}
+
+const defaultQuality = () => {
+    try {
+        return readPreferences().values.imageQuality;
+    } catch {
+        return undefined;
+    }
+};
+
+/**
+ * Open "Thumbnail maintenance" on the section's `scope`. `store` is the page's Activity store; `trigger` gets focus
+ * back when the dialog closes.
+ */
+export function openThumbnailReview({ api, store, scope, trigger = null, quality = defaultQuality(), notify = showWorkspaceToast,
+    navigate = navigateTo }) {
+    const mode = select(SCOPE_OPTIONS, 'missing');
+    const encoder = select(QUALITY_OPTIONS, preselectedQuality(quality));
+    const size = el('input');
+    size.type = 'number';
+    size.value = String(SIZE_LIMIT.value);
+    size.min = String(SIZE_LIMIT.min);
+    size.max = String(SIZE_LIMIT.max);
+    const line = el('p', PLAN_LOADING, 'small');
+    line.setAttribute('role', 'status');
+    const cancel = createButton('Cancel');
+    const start = createButton('Start', 'ca-primary');
+    const actions = el('div', undefined, 'dialog-actions');
+    actions.append(cancel, start);
+
+    // The plan per scope: a promise of its answer (null when it failed); the size and quality change no count.
+    const plans = new Map();
+    let open = true;
+    const chosenSize = () => previewSize(size.value);
+    const validate = () => {
+        const valid = chosenSize() !== null;
+        size.setCustomValidity(valid ? '' : SIZE_INVALID);
+        if (valid) size.removeAttribute('aria-invalid');
+        else size.setAttribute('aria-invalid', 'true');
+        return valid;
+    };
+    const readPlan = chosen => {
+        const query = new URLSearchParams({ mode: chosen, size: String(chosenSize()), quality: encoder.value });
+        return api.get(`${THUMBNAIL_PLAN_URL}?${query}`, { signal: scope.signal, cache: 'no-store' }).catch(() => null);
+    };
+    async function paint() {
+        if (!validate()) { line.textContent = SIZE_INVALID; return; }
+        const chosen = mode.value;
+        if (!plans.has(chosen)) plans.set(chosen, readPlan(chosen));
+        const pending = plans.get(chosen);
+        line.textContent = PLAN_LOADING;
+        const plan = await pending;
+        // A later choice painted its own line meanwhile, or the dialog closed.
+        if (!open || scope.signal.aborted || mode.value !== chosen || !validate()) return;
+        line.textContent = planText(plan);
+    }
+
+    scope.listen(mode, 'change', paint);
+    scope.listen(size, 'input', paint);
+    scope.listen(cancel, 'click', () => closeWorkspaceDialog());
+    scope.listen(start, 'click', () => {
+        if (!validate()) {
+            size.reportValidity();
+            size.focus();
+            return;
+        }
+        const options = { mode: mode.value, size: chosenSize(), quality: encoder.value };
+        closeWorkspaceDialog();
+        startThumbnails({ store, scope, options, notify, navigate });
+    });
+    openContentDialog({ scope, title: 'Thumbnail maintenance', trigger,
+        nodes: [field('Scope', mode, 'thumb-mode'), field('Quality', encoder, 'thumb-quality'),
+            field('Preview size limit', size, 'thumb-limit'), line, actions],
+        onClose: () => { open = false; } });
+    paint();
+}

web/js/workspace/workspace.css

@@ -24,6 +24,10 @@ main:focus-visible { outline: 2px solid var(--ca-focus); outline-offset: 4px; }
 .activity-pill:hover { border-color: var(--ca-accent); /* D53 */ }
 .activity-pill-bar { display: block; inline-size: 70px; block-size: 5px; border-radius: 3px; background: var(--line); overflow: hidden; /* D53 */ }
 .activity-pill-bar i { display: block; block-size: 100%; inline-size: 0; background: var(--ca-accent); /* D53 */ }
+/* D111 (S5): a running job without a known total sweeps a short segment; reduced motion shows a dimmed full bar. */
+.activity-pill-bar.indeterminate i { inline-size: 40%; animation: activity-pill-sweep 1.4s ease-in-out infinite; /* D111 */ }
+@keyframes activity-pill-sweep { from { transform: translateX(-100%); } to { transform: translateX(250%); } }
+@media (prefers-reduced-motion: reduce) { .activity-pill-bar.indeterminate i { inline-size: 100%; opacity: .45; animation: none; /* D111 */ } }
 @media (pointer: coarse) { .activity-pill:not([hidden]) { min-block-size: 40px; /* D53 */ } }
 @media (max-width: 700px) {
     /* D52: with the Activity button the full-width row can be wider than a zoomed phone, so its items wrap. */