Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d2ff73d299 | ||
|
|
40cecbced2 | ||
|
|
f9a781c30a | ||
|
|
69de20a427 | ||
|
|
18d81c9d96 | ||
|
|
9e125ee4a3 | ||
|
|
cbc53ee9f1 | ||
|
|
95f88d3966 | ||
|
|
7c25b351f7 | ||
|
|
9bfb5992b3 | ||
|
|
785b744554 | ||
|
|
519ab59d71 | ||
|
|
369f38be9a | ||
|
|
e6ed2914e1 | ||
|
|
878612c22d | ||
|
|
966a25d129 | ||
|
|
7511a0fd38 | ||
|
|
add035d940 | ||
|
|
88e5b78207 | ||
|
|
b0a025a326 | ||
|
|
c86739178d | ||
|
|
4e5c810f96 | ||
|
|
b50bf862f0 | ||
|
|
fcf6445de0 | ||
|
|
0f3559bedf | ||
|
|
4f1c6e236b | ||
|
|
5b2c21bdaf | ||
|
|
25e7ae158b | ||
|
|
08151ca93d | ||
|
|
b2cc489fd4 | ||
|
|
347de6a023 | ||
|
|
fd747a0ffc | ||
|
|
7a4b6eebb2 | ||
|
|
7f1b5b64bd | ||
|
|
f00933ee16 | ||
|
|
beaba9ae84 | ||
|
|
ab47d7529b | ||
|
|
b84e6ea111 | ||
|
|
fe5a0017a9 | ||
|
|
d24f3f989f | ||
|
|
32cc071d07 | ||
|
|
c4ac90e9ec | ||
|
|
ed392e77e2 | ||
|
|
3e20c533fa | ||
|
|
d25c2cb4f2 | ||
|
|
f3366a13f8 | ||
|
|
de6af76064 |
+723
-29
@@ -50,6 +50,13 @@ stdin open and the server (`--exit-on-stdin-eof`) shuts down cleanly, removing i
|
||||
temp root, when that pipe closes, so a worker killed without teardown leaves no
|
||||
server behind. A test that fails gets the stderr its server wrote during the test
|
||||
attached to the report as `server-stderr` (an aiohttp traceback behind a 500).
|
||||
A `/__test__/.../reset` still running after 2 s writes a report to the job log,
|
||||
passing test or not: every thread's stack taken while it waits, the default
|
||||
executor's threads and queue, the open connections and WAL size, then the total
|
||||
and how long each executor call queued and ran. Its lines start with
|
||||
`pm-slow-reset| ` (`tests/browser/slow_reset_watch.py`). The report never reads a
|
||||
connection another thread is using: one that is busy inside SQLite would block
|
||||
that read with the GIL held and freeze the whole server.
|
||||
|
||||
The coverage run can be split the way CI splits it, and each shard keeps the raw
|
||||
V8 files it writes under `coverage/js/raw`:
|
||||
@@ -114,6 +121,13 @@ npm run --prefix tools/vendor test:comfyui
|
||||
CI performs this setup automatically. `PM_COMFY_URL` can select another
|
||||
`http://127.0.0.1:PORT/` address (default port 8189). This is an owned managed
|
||||
server: a running host is never reused. Do not point it at your live checkout.
|
||||
One port serves one run. The launcher refuses a port that is already in use and
|
||||
reports ready only when `/system_stats` names its own base directory, so two runs
|
||||
started together on 8189 fail at startup ("already in use" or "answered by a
|
||||
ComfyUI host this run did not start") instead of one run testing the other's
|
||||
host and losing it mid-test (`ECONNREFUSED`, `ERR_CONNECTION_REFUSED`, Network
|
||||
Error). Every test checks the answering host again. Agents sharing a machine
|
||||
give each run its own `PM_COMFY_URL` port.
|
||||
The launcher verifies the upstream SHA and clean tracked files, snapshots only
|
||||
tracked PromptManager files, and creates temporary user/input/output/database/
|
||||
configuration/log directories. Personal untracked files are excluded. It runs
|
||||
@@ -378,7 +392,8 @@ with source `zeroed-conditioning`. `easy ifElse` with a literal boolean follows
|
||||
branch only; with a linked boolean both branches are traced, equal texts resolve and
|
||||
different texts make the sampler `ambiguous` with one candidate per choice (at most
|
||||
32, else `work-limit`). ConditioningCombine/Concat/Average resolve only when every
|
||||
part has the same text, otherwise `unresolved` with `composite` naming the node.
|
||||
part has the same text, otherwise `unresolved` with `composite` naming the node
|
||||
(but see the added-conditioning floor below for a later stage).
|
||||
Results that depend on a forced choice are never memoized. A `FaceDetailer`
|
||||
(`detailer-stage`) is a sampler stage: the samplers reached through its `image`
|
||||
ancestry are compared with it. Equal texts resolve with the detailer as the sampler;
|
||||
@@ -389,15 +404,78 @@ A sampler's latent ancestry is walked through every link input of non-sampler
|
||||
nodes within the same visit and depth budgets: with no earlier sampler it is a
|
||||
single stage whatever the origin (VAEEncode, EmptyLatentBatch, EmptyFlux2LatentImage,
|
||||
EmptyHunyuanLatentVideo, SetLatentNoiseMask, InpaintModelConditioning or
|
||||
WanImageToVideo slot 2). An earlier sampler (img2img from a decode, hires fix) stays
|
||||
`unresolved` "Multistage sampler ancestry is ambiguous" until stage texts are
|
||||
compared; an unlisted node with sampler-shaped inputs (positive, negative, guider,
|
||||
WanImageToVideo slot 2). Pass-through lineage (#304 follow-up, maintainer option B,
|
||||
2026-10-23): the table's `lineage` section lists the outputs whose image or latent
|
||||
continues one input's (`VAEDecode`/`VAEDecodeTiled` `samples`, `VAEEncode`/
|
||||
`VAEEncodeTiled` `pixels`, `ImageUpscaleWithModel`, `ImageScale`, `ImageScaleBy`,
|
||||
`ImageScaleToTotalPixels` `image`, `LatentUpscale`/`LatentUpscaleBy` `samples`,
|
||||
cg-image-filter `Image Filter` slot 0 `images` and slot 1 `latents`; a selection of
|
||||
the same batch), plus Reroutes. Lineage describes no prompt, so its classes are not
|
||||
claimed by a prompt category. When the latent walk finds exactly one earlier sampler
|
||||
and the latent descends from that sampler's output slot 0 only through lineage
|
||||
nodes (a hires-fix or refiner chain, img2img from an in-graph decode, a direct
|
||||
latent link), the two stages are compared like a detailer and its upstream stage:
|
||||
equal role texts resolve with the later sampler named, different texts are
|
||||
`ambiguous` `multistage-differs` with one candidate per stage, and an unreadable
|
||||
earlier stage makes the chain `unresolved` with that stage's reason. Unanchored, the
|
||||
earlier sampler is judged inside the chain when its output reaches no other file
|
||||
output. A size or resolution node reading the same image beside the lineage (a
|
||||
`ResolutionCalculator` feeding `ImageScale` width and height) is allowed, since the
|
||||
walk then still finds only that sampler. Any other earlier sampler, or one reached
|
||||
only through an unlisted image node, stays `unresolved` "Multistage sampler ancestry
|
||||
is ambiguous". Added-conditioning floor: a later stage whose own positive fails only
|
||||
as `composite` keeps its earlier stages' shared pair ("A later stage adds
|
||||
conditioning to the base prompt", naming the earlier stage's sampler) when that
|
||||
positive reaches, through Reroutes and conditioning pass-throughs, a composite the
|
||||
table marks `additive` (ConditioningCombine, ConditioningConcat; never
|
||||
ConditioningAverage, which can weight a part away), one of its parts is exactly the
|
||||
earlier stages' positive text, and its negative text equals theirs. This is the
|
||||
three-stage Flux run whose UltimateSDUpscale combines the PromptManager prompt with
|
||||
a style-trigger encoder: legacy linked the PromptManager text and every stage is
|
||||
conditioned on it. The positive is always labelled (maintainer ruling): its source is
|
||||
`added-conditioning` (D126; with a D125 omission too,
|
||||
`runtime-parts-omitted-and-added-conditioning`) and its provenance names the
|
||||
composite (`added_by`), whatever the added text is; no length heuristic. A combine without the base text, a different negative, or
|
||||
different earlier stages stay as they were (`composite` or `multistage-differs`).
|
||||
An image-edit batch (several samplers with distinct prompts, one shared saver
|
||||
prefix) stays `ambiguous`, and text generated at run time stays untraceable
|
||||
(`tests/test_resolver_pass_through.py` on synthetic shapes from
|
||||
`tests/passthrough_graphs.py`; the browser port is compared on the named
|
||||
`select-*` chain cases and `passthrough_fuzz_cases`). Ingestion keeps the default
|
||||
profile's historical latent rule (D1), so the admin scan still reaches these files
|
||||
through its legacy-parser fallback. Run-time PromptManager parts (maintainer option
|
||||
A, 2026-10-23): LoraManager's `TriggerWord Toggle` is a `runtime` class (its output is
|
||||
the LoRA's trigger words as filtered at run time). A PromptManager (or
|
||||
PromptManagerPair) `prepend_text` or `append_text` linked directly to a `runtime`
|
||||
node is left out: the text is the node's own, combined with any other literal or
|
||||
traced part exactly as before, the side's source is `runtime-parts-omitted`, and a
|
||||
resolved pair's reason is "Text added at run time, such as trigger words, is not
|
||||
included". The prompt block shows that line under the text (D125; the evidence
|
||||
badge stays "Workflow inferred"). The label follows the text through encoders,
|
||||
concatenations and chained PromptManagers: a side is labelled when any node on its
|
||||
traced path left a part out, and its provenance lists those PromptManager nodes
|
||||
(`omitted_parts`). Comparisons include what a text leaves out, so a prompt that left
|
||||
out a part never equals a complete one (a branch with the toggle and one without stay
|
||||
ambiguous; such stages are `multistage-differs`). Only classes in the table's
|
||||
`runtime` list are left out, never another `runtime-value` source (a JoinStringMulti
|
||||
with a linked list flag stays `runtime-value`). The toggle's output is never computed. A run-time node further upstream (inside a
|
||||
concatenation), a run-time main `text`, or an unknown feeder keep their outcome
|
||||
(`runtime-value` or `unsupported-node`). This is what legacy linked for the June
|
||||
hires-fix runs (`TestRunTimePromptManagerParts`, the `select-runtime-*` parity
|
||||
cases); an unlisted node with sampler-shaped inputs (positive, negative, guider,
|
||||
latent_image, a pipe, video embeds, custom-sampler parts, or a `model` link together
|
||||
with conditioning, neg_conditioning, samples or latent) is `unsupported-node`
|
||||
naming it, here and in a detailer's image ancestry; a latent that is not a link or
|
||||
reaches its own sampler is `malformed`; running out of budget is `work-limit` or
|
||||
`depth-limit`, never single stage. Each sampler's walk runs once per resolution;
|
||||
selector branch runs share it and other candidates are charged its visit cost.
|
||||
A UI workflow node whose `type` is a list or an object (read only for the overwritten
|
||||
PromptManager check) is skipped like a non-object node, since it can be neither a
|
||||
subgraph instance nor a PromptManager node, so a prompt that resolves still resolves
|
||||
and the other nodes still feed the check; a list or object `class_type` is never a
|
||||
preview output. Before #286 both raised and the inspect route answered 500
|
||||
(`test_unhashable_*` in `tests/test_api_prompt_inspector.py`, the `workflow-type-*`
|
||||
golden cases on both sides).
|
||||
Text encoders: `single` encoders (the Qwen image encoders, `TextEncodeQwenImage21`
|
||||
slot 0 `prompt` and slot 1 `negative_prompt`, `KikoTextEncode`) read one string
|
||||
input unchanged. `parts-equal` encoders (CLIPTextEncodeSDXL, Flux, SD3) resolve when
|
||||
@@ -1581,7 +1659,43 @@ AVI/WMV preview only), chosen or dropped, previews inline. The heading's one act
|
||||
the shared viewer with the full inspector on a record read entirely in this browser
|
||||
(`web/js/workspace/inspect.mjs`, `local-session.mjs`, `local-inspection.mjs` and the L1
|
||||
readers `png-metadata.js`, `local-video.mjs`, `local-projection.mjs`). Nothing is
|
||||
uploaded, imported or stored. Only HTML and JS change, so no ComfyUI restart is needed.
|
||||
uploaded, imported or stored. PRs L1 and L2 change only HTML and JS, so they need no
|
||||
ComfyUI restart; PR L3 does (below).
|
||||
|
||||
Local roles (#249 PR L3, decision LI-1): the prompts and resolution of a local record are
|
||||
the server inspector's for the same records. `web/js/workspace/local-resolver.mjs` (with
|
||||
`local-resolver-support.mjs`, `local-resolver-stages.mjs`, `local-parameters.mjs` and
|
||||
`py-json.mjs`) ports `inspect_prompt_evidence` with `INSPECTION_PROFILE`: execution
|
||||
evidence, graph tracing, every bound, several sampler candidates with the
|
||||
identical-branch rule (#270), preview-only stages, detailer stages (multistage) and A1111
|
||||
`parameters`. It reads the server's own node table, `web/js/inspection-semantics.json`,
|
||||
which the page fetches once through the API client as it mounts, before any selection
|
||||
(`local-semantics.mjs`; `/prompt_manager/js/inspection-semantics.json` is the one JSON
|
||||
file the static route serves, by exact path). A table that cannot be fetched answers
|
||||
"The inspector's node table is unavailable." as the server does. The legacy fill in
|
||||
`projectInspectionPayload` then applies to a local record exactly where it applies to the
|
||||
Images answer for the same bytes. Python is the oracle:
|
||||
`tests/test_local_inspect_parity.py` `LocalResolverParityTests` compares the local
|
||||
evidence byte for byte with `inspect_prompt_evidence` for every parity PNG and inspection
|
||||
vector (prompts, resolution and facts), the golden and selection record sets
|
||||
(`tests/local_resolver_cases.py`, `tests/local_resolver_selection_cases.py`), every work
|
||||
bound and 1,200 seeded fuzz cases; `tests/fixtures/local-inspect/resolver-vectors.json`
|
||||
holds the committed answers the Node unit test reads. The browser spec `chooser shows the
|
||||
Images roles and resolution for every parity PNG` compares the rail with Images for all
|
||||
25 committed parity PNGs. The static route change and the resolver fixes (Python) need a
|
||||
ComfyUI restart.
|
||||
|
||||
Main thread (M4): the resolver runs on the page's thread, not a worker. On a desktop
|
||||
(Node 26) `resolveLocalEvidence` on maximal 2 MiB graphs (a 19,990-sampler chain, 6,600
|
||||
independent samplers, a consumer-dense scan, a 100,000-node PromptManager workflow) took
|
||||
130–455 ms; the server took 80–905 ms on the same inputs. `localFileAdapter.read` runs in
|
||||
bounded steps (read, table, parse and resolve, project, fit) with a turn of the event
|
||||
loop between them, every wait ends with the selection's signal (a stalled table request
|
||||
included), and the inspector carries `aria-busy="true"` with "Loading file details…"
|
||||
until the answer lands. `tests/js/workspace-local-inspection.test.js` pins a 2,500 ms
|
||||
budget per step on maximal independent-sampler and stage-chain graphs, checks that the
|
||||
stepped read answers byte for byte as one synchronous build, and that an abort at any
|
||||
step ends the read.
|
||||
|
||||
Bounds (the server's own budgets, ported by PR L1; see "Follow-up: local Inspect parity"
|
||||
under "Chronological Library history" for the known divergences):
|
||||
@@ -1600,7 +1714,8 @@ under "Chronological Library history" for the known divergences):
|
||||
|
||||
Privacy, asserted by `local inspection makes no content request or storage write` and
|
||||
`local roles send exactly` in `tests/browser/workspace-inspect.spec.mjs`, and by the
|
||||
managed-host spec below: no request carries the file's bytes, name or text (fetch, XHR,
|
||||
managed-host spec below: the only static file Inspect reads is its node table, once per
|
||||
page and before any selection; no request carries the file's bytes, name or text (fetch, XHR,
|
||||
beacon, WebSocket, EventSource); Inspect writes nothing to `localStorage`,
|
||||
`sessionStorage` or IndexedDB; only an explicit Send crosses the host bridge, and its
|
||||
payload carries neither the `blob:` URL nor the filename; no console message names the
|
||||
@@ -1627,19 +1742,40 @@ mapped differences"):
|
||||
`image/png,image/jpeg,image/webp` plus `video/*,.mp4,.mov,.m4v,.webm,.mkv,.avi,.wmv`. A
|
||||
video previews in the same `#local-preview` slot as `video.local-file-preview` with
|
||||
native `controls`, `playsInline`, `preload="metadata"` and `muted`, never autoplays
|
||||
inline, and is paused, has its `src` removed and is removed from the page on replacement
|
||||
or dispose, before its URL is revoked. Each preview owns its listeners, so the page never
|
||||
pins a released node. A settled video (metadata read, or failed) is then reset with
|
||||
`load()` at once; one still starting is reset only once it reports metadata or an error,
|
||||
because WebKit (GStreamer) blocked the page when such a pipeline was reset (`replacing a
|
||||
video that just started loading never blocks the page`, `a video replaced while still
|
||||
starting is reset once it settles`). A WebKit load cut off by the revoked URL usually
|
||||
stalls and never settles; a fallback then only drops the waiting listeners. The viewer's
|
||||
own video teardown was checked the same way (`closing the viewer on a video that just
|
||||
started never blocks the page`, 15 fresh pages on WebKit) and needed no change.
|
||||
Follow-up: on WebKit, a video released exactly at its `loadstart` sometimes blocks the
|
||||
page, with or without any reset and also before the deferred reset (2 of 24 runs in the
|
||||
pinned image), so the `loadstart` spec is skipped on WebKit until that is investigated.
|
||||
inline. On replacement or dispose it is paused, has its `src` removed, is reset with
|
||||
`load()` (which frees its decoder) and is removed from the page, before its URL is
|
||||
revoked. Each preview owns its listeners, so the page never pins a released node.
|
||||
A video replaced while still starting (a source, but neither metadata nor an error yet)
|
||||
stays where it is, inert and `aria-hidden`, with the new preview inserted after it, and
|
||||
the session's revoke of its URL waits; once it reports metadata or an error (or after
|
||||
15 s) it is released as above and only then is its URL revoked. For that moment (the
|
||||
time to metadata, or at most 15 s for a video that never settles) the old video and the
|
||||
new preview both show, the new one below it: a known, brief overlap. Stacking them in one
|
||||
slot instead hung again, because it changes the video's own box: the old video
|
||||
absolutely positioned in its own box hung 5 of 120 pages (8 of 120 with `opacity: 0`),
|
||||
and a one-cell grid 4 of 120. Only `opacity: 0` on the video, left in the flow, did not
|
||||
hang (0 of 120), and alone it would leave an empty hole above the new preview. A route
|
||||
change within the workspace waits the same way: the shell renders the next route once a
|
||||
video still starting on Inspect settles (`leaveGuard`; the latest route wins). Route
|
||||
changes at `loadstart` + 0–7 ms hung 4–5 of 120 pages before and 0 of 120 after.
|
||||
A real page leave cannot wait. With no page code releasing anything, navigating
|
||||
away hung 8 of 120 pages and reload 22 of 120 (3 and 8 of 120 with the page's own
|
||||
release), so that WebKit teardown stays a known limitation; dispose still releases at
|
||||
once. Cause, found with native stacks of the hung WPE web process in the
|
||||
pinned image: WebKit (GStreamer) deadlocks when a video leaves the page or the viewport
|
||||
before its pipeline has started. The main thread changes the pipeline state and waits
|
||||
for the source's streaming thread, which is itself waiting on the main thread in a
|
||||
synchronous bus handler. Removing the video, hiding it, or only pushing it below the
|
||||
fold with a taller preview inserted before it all hung (7 to 21 of 120 fresh pages each,
|
||||
replaced at `loadstart` + 0–7 ms), and none of the earlier steps (`pause()`, removing
|
||||
`src`, `load()`, revoking the URL) was the cause on its own; pushing it out after
|
||||
`loadedmetadata` hung 0 of 180. Specs: `replacing a video that just started loading
|
||||
never blocks the page`, and `a video replaced while still starting is reset once it
|
||||
settles`, which now runs on WebKit too (15 fresh pages, replaced at `loadstart` + 0–4
|
||||
ms), and `leaving for another route while a video starts waits for it to settle`. The
|
||||
viewer's own video teardown was checked the same way (`closing the viewer on a video
|
||||
that just started never blocks the page`, 15 fresh pages on WebKit) and needed no
|
||||
change.
|
||||
- **D63** (status line): one `p.small.spaced-16` in `#local-preview`, for states the
|
||||
concept never shows: "Drop one file at a time.", "Folders cannot be inspected. Choose
|
||||
one image or video file.", "This file type cannot be previewed or inspected here.
|
||||
@@ -1664,6 +1800,29 @@ mapped differences"):
|
||||
Modified with no Path row and no Show in folder, and ends with "Local file; no saved
|
||||
library association was checked." and Download original. There is no Prompt history
|
||||
action and no category or tag chips.
|
||||
- **D117** (Copy settings summary, LI-4): the legacy metadata
|
||||
page's COPY ALL, consolidated into the shared full inspector (the viewer's
|
||||
`#viewer-details`, so Images, the recorded history's current file and Inspect all have
|
||||
it). The concept has no such action. One plain button "Copy settings summary" follows
|
||||
the six facts (before "File & provenance"); the Images side panel keeps the concept's
|
||||
footer and has none. Its text is the legacy `copyAllData` template, line for line:
|
||||
`Checkpoint`, `Positive Prompt`, `Negative Prompt`, `Steps`, `CFG Scale`, `Sampler`,
|
||||
`Seed`, with the legacy placeholders ("Unknown", "No prompt found", "No negative prompt
|
||||
found"). The values are the ones the inspector shows: a confirmed prompt and a known
|
||||
fact verbatim, anything else (an explicitly empty prompt included, as the legacy parse
|
||||
treats '') the placeholder. Against the legacy page, for the same file, only these
|
||||
differ: numbers keep the server's spelling ("8.0" where the legacy page wrote 8); a fact
|
||||
the legacy parse could not read can be known; and a negative the #249 ZeroOut guard
|
||||
zeroed reads "No negative prompt found" where the legacy parse read the positive's text
|
||||
through the zero-out. `workspace-file-inspector.test.js` runs the template taken from
|
||||
`web/metadata.html` itself on the legacy parse of every parity PNG: 22 of 25 are
|
||||
byte-equal, f01b and f23 differ by the ZeroOut guard and f17 by facts the legacy parse
|
||||
missed. Copying works as the concept's prompt copy: the toast "Settings summary
|
||||
copied.", or, without a clipboard or when it is refused, the select-to-copy dialog
|
||||
titled "Copy settings summary". `.ca-tools #viewer-details > div > button` wraps
|
||||
(`white-space: normal`) in the narrow rail at 200% zoom, as D20's buttons do. The concept
|
||||
comparisons add the button to the concept's viewer at runtime (a copy of its own plain
|
||||
button) and to the Inspect rail's section order.
|
||||
|
||||
The chooser keeps naming the chosen file, as the concept's does: it is cleared as the
|
||||
picker opens (so choosing the same file again is a new selection), not after `change`,
|
||||
@@ -1677,9 +1836,8 @@ Legacy floor (`web/metadata.html`), dispositions:
|
||||
- Full text (the old "Show more"): the prompt blocks show the whole text.
|
||||
- Raw workflow view and Download workflow JSON: kept, in "Workflow & raw metadata".
|
||||
- Copy all summary and filename copy (LI-4): the filename is selectable text in the rail
|
||||
and Download original saves the file under its name. The ruled "Copy settings summary"
|
||||
button after the six facts of the shared full inspector is not part of PR L2; until it
|
||||
lands, the legacy page keeps COPY ALL and stays reachable from Overview.
|
||||
and Download original saves the file under its name. COPY ALL is the shared full
|
||||
inspector's "Copy settings summary" (D117), with the legacy text.
|
||||
- Fully browser-local reading: kept and asserted (privacy above).
|
||||
- Case-variant keys `Workflow|comfy|ComfyUI|Prompt|Parameters` (LI-5): a separate
|
||||
backend issue; Inspect follows the server's readers, so the two never disagree.
|
||||
@@ -1722,6 +1880,337 @@ npm run --prefix tools/vendor test:workflows -- inspect-concept.spec.mjs workspa
|
||||
npm run --prefix tools/vendor test:comfyui -- inspect-sender.spec.mjs
|
||||
```
|
||||
|
||||
### Native System (#249 System)
|
||||
|
||||
`#/system/<section>` is a native workspace route rendered by `web/js/workspace/system.mjs`
|
||||
on the concept's `renderSystem` markup (app.js a698–706; `concept.css` 709–895). `#/system`
|
||||
is Overview under the sidebar's own hash; `#/system/overview|appearance|sources|
|
||||
integrations|tagging|maintenance|backups|diagnostics` select a section, and an unknown
|
||||
child (the concept's `#/system/general`) is replaced by `#/system/overview`. The
|
||||
production hub `#/overview` is unchanged (D28); its System card opens
|
||||
`#/system/overview` (Q12). Switching sections keeps the shell, the drafts and focus on
|
||||
the pressed section button and disposes the previous section's scope, so its reads,
|
||||
listeners and dialogs never outlive it.
|
||||
|
||||
Drafts (`system-state.mjs`) live for the page session (Q10). Save writes the dirty
|
||||
browser fields, then the host fields (`POST /prompt_manager/settings` with
|
||||
`expected_revision`, #220), then the runtime fields. A failed domain keeps its drafts and
|
||||
highlights their controls; `settings-changed` is a conflict and keeps the draft. System
|
||||
reads the host values on mount and again after a conflict; until a read returned a
|
||||
revision, a host save fails without a request, so it never sends a blind
|
||||
`expected_revision: null` (which S1 merges into the file on disk). Save stays disabled
|
||||
until a section has an unsaved change.
|
||||
|
||||
Appearance (`system-appearance.mjs`, S3) is the concept's "Browsing & display" surface
|
||||
on real values. Its browser rows are this browser's display defaults in the legacy
|
||||
gallery's own `gallerySettings` object (`web/js/workspace/preferences.mjs`, Q3): Default
|
||||
layout and Thumbnail size (`workspaceLayout` grid/list/masonry, `workspaceDensity`
|
||||
comfortable/large/compact, new keys), Load metadata on selection (`autoLoadMetadata`),
|
||||
Cache metadata in this browser (`cacheMetadata`), Image loading quality (`imageQuality`:
|
||||
Optimized→`medium`, Original→`high`, Fast→`low`; the thumbnail dialog's preselected quality
|
||||
once S5 lands), the three video rows (`videoAutoplay`, `videoMute`, `videoLoop`) and Show
|
||||
file paths (`showFilePaths`). A write re-reads the stored object and writes back only the
|
||||
changed keys, every other key (legacy paging, thumbnail state, unknown keys) as stored;
|
||||
the legacy gallery's Save also keeps the keys it does not edit, so the two pages stay in
|
||||
sync. The workspace never reads the legacy paging keys (`defaultLimit`, `gridColumns`,
|
||||
`infiniteScroll`; README continuous-browsing ruling). A browser that refuses storage
|
||||
(denied, or a full quota that leaves the stored bytes as they were) keeps the change for
|
||||
this tab only and the status says "Saved for this tab only; browser storage is
|
||||
unavailable.", never "Saved in this browser.". Another tab's change (a `storage` event)
|
||||
repaints the clean controls; a draft it overtook stays, highlighted, with the conflict
|
||||
status. Open from ComfyUI (New tab→`newtab`, Popup window→`popup`) and Node search
|
||||
auto-hide (seconds) (a whole number 0–300; anything else is highlighted and never sent)
|
||||
are the server's `webui_display_mode` and `result_timeout`, saved through
|
||||
`POST /prompt_manager/settings` with the revision. The consumers read the defaults on
|
||||
their next visit, never the current view: a new Images visit starts from the layout and
|
||||
size (the last visit keeps its own, a saved view opens at Grid and Comfortable as
|
||||
before); with metadata off a selection reads nothing until "Load metadata"; with the
|
||||
cache off selecting the shown file again re-reads it (on, the one current record is
|
||||
reused); paths off leaves out only the Path row of File & provenance; the viewer reads
|
||||
the video defaults.
|
||||
|
||||
Sources & scans (`system-sources.mjs`, S4) is the concept's "sources" body on the
|
||||
server's sources. `GET /prompt_manager/settings` reports them (public paths only):
|
||||
`sources.configured` (each `gallery_root_paths` entry with `valid`, `code` and the
|
||||
server's `reason`), `sources.effective` (what is scanned and served, with its
|
||||
`root_index` and `watch`: `watching` only when the image monitor's observer is alive,
|
||||
scheduled that folder and monitoring is on, `stopping` when it still watches it but
|
||||
monitoring was switched off, `manual` when monitoring is off or no monitor runs, else
|
||||
`restart_pending`), `sources.auto_detected`, `monitor` and `settings.monitoring_enabled`.
|
||||
`POST /prompt_manager/settings/validate-source {path}` checks one folder and writes
|
||||
nothing. Every caller (validate-source, the settings save, config.json at load) refuses
|
||||
a source through the same checks (`sources.source_problem`): its form first, then
|
||||
containment decided as text (`normcase(normpath)` against the allowed folders' own
|
||||
spellings), before anything resolves, stats or opens the path, so a UNC share, the
|
||||
`\\?\` and `\\.\` namespaces, a device or any folder outside the allowed ones never
|
||||
reaches the filesystem or the network (no SMB request, no existence or timing oracle).
|
||||
Only then is the path resolved and contained again, which catches a symlink out. The
|
||||
filesystem root and the home directory answer `outside-allowed` with the same reason as
|
||||
any outside path. ComfyUI folders outside its base directory (`--output-directory`)
|
||||
are path anchors too, so their sources keep a relative public path
|
||||
(`renders/portraits`) that round-trips and never collides by folder name. A save whose
|
||||
list holds a source the server cannot use is a 400 `invalid-source` naming its public
|
||||
path (`source`, as text: a refused path is never resolved to name it), its `index` and
|
||||
`reason`, and saves nothing. A save that changes the sources (compared as folders, not
|
||||
spellings) holds the registry as `records` ("Source change"), so it is refused with
|
||||
409 `busy` while a scan runs. `POST /prompt_manager/jobs/scan {scope, roots, paths}`
|
||||
scans only the given effective `root_index`es (strict ints, at most 64), each with the
|
||||
public path the page read for it; an index past the list or one that now names another
|
||||
folder is 409 `sources-changed` and starts nothing (the page toasts the server's
|
||||
reason and reads the sources again); the legacy `/scan` is unchanged. The section
|
||||
lists one row per source ("<path> · watching | watching until restart | manual scan |
|
||||
restart pending" with "Scan changes" for a watched source, else "Scan folder"), which
|
||||
opens the shared "Review metadata scan" for that one source ("1 output folder") and
|
||||
starts the session job over it; the job opens in Activity, and the toast only says it
|
||||
started. Enrich
|
||||
existing metadata and the notice's Review scan review every source. The notice shows
|
||||
until this browser dismissed the 3.2.0 update notice (`pm_update_notice_dismissed`,
|
||||
the legacy admin page's key); Later dismisses it without scanning, and Review scan
|
||||
dismisses it as the legacy Scan does. "+ Add source" asks validate-source, then adds
|
||||
the folder to the host draft of `gallery_root_paths`; Remove takes one out; "Watch for
|
||||
new outputs" (`monitoring_enabled`) and Worker threads (`worker_threads`) are host
|
||||
drafts too, and Save writes them with the revision. A save the server refuses because a
|
||||
scan runs toasts "Another job is running: <title>." with a link to that job (D57) and
|
||||
keeps the drafts; a save refused for one source says the server's message as the status
|
||||
and marks that row. After a save the source list is read again; a monitoring change
|
||||
waits for a restart (the running observer keeps watching until then, and the status
|
||||
says "Restart ComfyUI to apply: Watch for new outputs."). Thumbnail generation's
|
||||
Review options opens the legacy gallery in a new tab (D94) until its review dialog (S5).
|
||||
A list that cannot be read shows "Sources could not be loaded. Your saved data has not
|
||||
changed." and Retry, never an empty list.
|
||||
|
||||
Overview (`system-overview.mjs`) reads `GET /prompt_manager/system/overview` once per
|
||||
visit through `latestOnly()` on the section scope; a late answer after the section left
|
||||
paints nothing. Until it answers, the section shows "Loading the overview…"; a failure
|
||||
shows "The overview could not be loaded. Your saved data has not changed." and Retry,
|
||||
never zeros. The thumbnail row is a separate read of `/images/thumbnails/status` (a
|
||||
walk of every source), so the other values never wait for it: "Checking thumbnails…"
|
||||
until it answers. The health check (`POST /prompt_manager/system/health`) is a `read`
|
||||
registry job of kind `health`: one at a time, beside a scan, not cancellable and with
|
||||
no file outcomes. The page tracks the started job in the Activity store (its event
|
||||
stream; the store marks it lost if the server stops listing it), never by polling. A press while one runs (another tab) waits
|
||||
for that one. The last report lives for this server session (`GET
|
||||
/prompt_manager/system/health`) and describes the database it was taken from: a
|
||||
restore that replaces the database forgets it, and a check that started before the
|
||||
swap is not kept. The check's own connection opens a `mode=ro` URI that keeps a Windows
|
||||
UNC path (`file:////server/share/prompts.db`; pathlib's `file://server/share/...` is a
|
||||
remote authority SQLite refuses). A check that raises fails only itself, and the
|
||||
missing-files check reads its rows inside the snapshot but stats the files after it
|
||||
ended, so a slow source never pins the WAL. Statistics
|
||||
read `GET /prompt_manager/system/statistics` (`by_category`, the legacy `/stats` totals
|
||||
and `rated_images`).
|
||||
|
||||
Read-only standard for a check or preview (S2's health check; S6 and S8 previews use
|
||||
the same): the database and `-wal` bytes are unchanged, the `-shm` size is unchanged,
|
||||
the connection's `total_changes` is 0 at close and the data reads back unchanged. The
|
||||
`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
|
||||
reader can keep it byte-identical (`test_health_check_never_writes`).
|
||||
|
||||
Recorded deviations (plan D60–D66 and D81 renumbered D92–D98 and D113; D121–D128 assigned):
|
||||
|
||||
- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
|
||||
unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server
|
||||
configuration." / "Applied until the server restarts.", "Some changes were not saved.
|
||||
Review the highlighted settings.", "Saved for this tab only; browser storage is
|
||||
unavailable." and "Settings changed elsewhere. Reload this section to see them."
|
||||
"Save preferences" is `aria-disabled` (opacity .55, `not-allowed`) while clean or
|
||||
saving; a setting the last save could not save gets a red outline. CSS under D92.
|
||||
- **D93** (Overview): real values. The metrics are the Library's eligible positives
|
||||
and active saved negatives and the indexed `generated_images` rows (Q2). "Database
|
||||
integrity" is "Not checked" until a check ran this session, then "N issues" (checks
|
||||
that found issues, even beside a check that could not run), "Passed" or "Incomplete".
|
||||
The review row is "N ambiguous image links" (the Needs review queue's pending count,
|
||||
`needs_review.pending`) with the concept's line; its Review metadata action is
|
||||
`aria-disabled` (CSS under D93) until the Needs review page ships (#288). A count
|
||||
that is not an integer reads "Ambiguous image links" / "Not available yet." (Q6).
|
||||
"Backup status" reads "No retained backup record." until verified backups exist (S8).
|
||||
The thumbnail row is "Checking thumbnails…" until its own read answers, then "No
|
||||
missing thumbnails", "N missing thumbnails" (the pending count of
|
||||
`/images/thumbnails/status`) or "Thumbnail status unavailable"; Review thumbnails
|
||||
opens Sources until the thumbnail dialog (S5). "One display configuration" says "Grid
|
||||
and density settings are shared across views." (no paging). The sources row is
|
||||
"N output sources" with "Monitoring and enrichment share one source list.", prefixed
|
||||
by "M of N configured sources available." or "None configured is available; ComfyUI's
|
||||
output folder is used." when that is the case.
|
||||
- **D94** (all sections): a section not delivered yet shows the hand-off panel "Your
|
||||
current tools are ready" with a link to each legacy tool that still owns it (the
|
||||
plan's legacy coverage table); retired section by section, gone after S12.
|
||||
- **D95** (health report): the concept's `health-check` dialog titled "Health report",
|
||||
with no sample note. It lists every check in the server's order: Integrity check,
|
||||
Foreign keys & pair references, Canonical hashes, Pair uniqueness (added), Missing
|
||||
files, Ambiguous metadata, Consistency (added). Missing files has one row per image
|
||||
source when there are several ("Missing files · output"). Each chip is the real
|
||||
result: "Passed", a count with what was found ("2 broken references", "1 link needs
|
||||
review", the concept's "N links need review" and "N images need review"; "100+
|
||||
problems found" when SQLite's integrity check stopped at its limit), "Not
|
||||
available" (an unavailable source), "Not available yet" (Ambiguous metadata, Q6) or
|
||||
"Could not check". A `p.small` after the list says when it was checked and whether
|
||||
every check passed, some could not run, or issues need review. Close / Review missing
|
||||
links (to Maintenance, its hand-off until S7). Nothing is changed by a check.
|
||||
- **D96** (Statistics): "Usage statistics" without the sample note: the concept's
|
||||
"Runs by category" table (Category / Runs / Avg. rating) with the real runs and
|
||||
average rating of eligible parents per category ("Uncategorized", "Not rated"),
|
||||
then a second table "Library totals" (Total / Count: Prompts, Tags, Images from the
|
||||
legacy statistics, and Rated images: indexed images whose linked prompt is rated).
|
||||
The Overview row says "Runs and ratings by category, with library totals." because
|
||||
that is what the dialog shows (the concept's own dialog shows only "Runs by
|
||||
category"); per-prompt run counts, prompt tags and dated runs are recorded, but no
|
||||
table of them exists yet. A failed read toasts "Statistics could not
|
||||
be loaded. Your saved data has not changed."
|
||||
- **D121** (production hub): the Overview hub's System card reads the concept's System
|
||||
heading and "Open System"; the concept has no hub card.
|
||||
- **D122** (shell): the section nav's `aria-label` is "System sections"; the concept's
|
||||
literal label is its prototype action name "system-tab sections".
|
||||
- **D123** (Maintenance): the concept's Maintenance notice
|
||||
("Check first. Review changes. Then apply." + Run health checks) is delivered ahead of
|
||||
the rest of Maintenance, above its D94 hand-off, as the native place that runs the
|
||||
health check. A press shows "Checking…" (`aria-disabled`) and opens the report; a
|
||||
failed check toasts "The health check could not finish. Nothing was changed."
|
||||
- **D97** (Appearance): added rows for the legacy gallery settings the concept leaves
|
||||
out, after its own rows: "Show image details on cards" (`showImageInfo`; off hides a
|
||||
card's title and file line, Info and Tag stay at the end of the row), "Collapse viewer
|
||||
details by default" (`sidebarCollapsedByDefault`; the viewer opens with
|
||||
`#viewer-details` hidden and a "Show details"/"Hide details" toggle with
|
||||
`aria-expanded`, the stage taking the whole width), and the operation "Reset display
|
||||
defaults" ("Restore this browser’s display defaults, then save. Other settings are
|
||||
kept.", button "Reset"), which drafts the gallery's defaults for the browser rows only.
|
||||
The node search timeout shows the server's real value, not the sample 30. CSS under D97.
|
||||
- **D98** (Sources): each configured source's row has a plain Remove after its scan
|
||||
button; nothing changes until Save. A source added this session reads "<path> · not
|
||||
saved yet" with Remove only until Save; a configured source the server cannot use
|
||||
reads "<path> · <server reason>" with Remove only. A line under the list says when the
|
||||
list is not the whole story: "No source is configured. After saving, ComfyUI’s output
|
||||
folder is used." (the last one removed), "No source is configured; ComfyUI’s output
|
||||
folder is used." and "No configured source is available; ComfyUI’s output folder is
|
||||
used." (auto-detected rows, listed without Remove). "restart pending" is the third
|
||||
watch state. The operation "Refresh files" ("Read every source again to find files
|
||||
added or removed outside ComfyUI.", button "Refresh") closes the surface: it calls the
|
||||
legacy gallery's `/gallery/rescan` and toasts "Files refreshed. N files in M output
|
||||
folders." or "Files could not be refreshed. Try again.". The concept's "sample path"
|
||||
note is prototype copy and is dropped.
|
||||
- **D113** (Sources): the Worker threads range ends at the server's `cpu_count` (the
|
||||
concept's fixed `max=16`) and shows the server's real `worker_threads` (the concept's
|
||||
sample 4).
|
||||
- **D127** (Sources): the concept's `add-source` `simpleInput` with production
|
||||
copy: the field is "Folder path" (the concept's "(sample only)" dropped), it starts
|
||||
empty (not the demo "/images/new-collection"), and the action is "Add" (not "Apply in
|
||||
prototype"); Enter in the field adds too. `add source dialog matches the concept`
|
||||
(system-concept.spec) compares the open dialog with these three mapped, and `add
|
||||
source comparison detects mutations` proves the mapping is the only difference. A
|
||||
folder the server refuses keeps the dialog open with the field
|
||||
`aria-invalid` and the server's reason in a `p.small[role=alert]` above the actions;
|
||||
a folder the server names as one already listed says "This folder is already a
|
||||
source.", and a check that cannot finish says "The folder could not be checked. Try
|
||||
again."; neither adds anything.
|
||||
The concept dialog does no check, so `invalid source shows the server reason`
|
||||
(workspace-system-sources.spec) pins it.
|
||||
- **D128** (Sources): states the concept's sample rows never show. A source the
|
||||
running monitor still watches after "Watch for new outputs" was saved off reads
|
||||
"<path> · watching until restart" (still "Scan changes"), and the save's status adds
|
||||
"Restart ComfyUI to apply: Watch for new outputs." from `restart_required_fields`. A
|
||||
configured source the effective list holds under another spelling reads "<path> ·
|
||||
same folder as another source" with only Remove. A save refused for one source
|
||||
(`invalid-source`) marks that row (`aria-invalid`, outlined like a D92 setting) and
|
||||
the status is the server's "<path> can't be used: <reason>. Remove it or reconnect it,
|
||||
then save." ("Remove it, then save." when reconnecting cannot help). Pinned by
|
||||
`Watch for new outputs saves monitoring_enabled; a restart applies it` and `a save
|
||||
refused for one unusable source names it and marks its row`
|
||||
(workspace-system-sources.spec) and the system-state and system-sources node tests.
|
||||
- **D124** (inspector): with "Load metadata on selection" off, the Images panel and
|
||||
the viewer's details show "Metadata loads when you ask for it." and a "Load metadata"
|
||||
button instead of reading the file. The concept has no such state, so no concept
|
||||
comparison renders it: `metadata off shows an explicit load and loads once on request`
|
||||
(workspace-system.spec) pins it instead.
|
||||
- **D125** (prompt blocks, maintainer option A): a resolved prompt whose source is
|
||||
`runtime-parts-omitted` (a PromptManager `prepend_text` or `append_text` fed directly
|
||||
by a listed run-time node, such as LoraManager's TriggerWord Toggle, was left out)
|
||||
shows the muted line "Text added at run time, such as trigger words, is not
|
||||
included." under its text. The evidence badge stays "Workflow inferred"; the line is
|
||||
never sent or copied. The concept has no such state (`a prompt that left out run-time
|
||||
trigger words says so under its text (D125)` in `tests/js/prompt-blocks.test.js`).
|
||||
- **D126** (prompt blocks, maintainer ruling on the added-conditioning floor): a
|
||||
resolved positive whose source is `added-conditioning` (a later stage combined more
|
||||
conditioning with the base prompt, as UltimateSDUpscale's ConditioningCombine of the
|
||||
PromptManager and a style trigger) shows "Conditioning added at a later stage is not
|
||||
included." the same way. A prompt with both shows both lines, D125 first (source
|
||||
`runtime-parts-omitted-and-added-conditioning`). Always labelled, with no length or
|
||||
"short trigger" heuristic (`a prompt a later stage added conditioning to says so, and
|
||||
both lines show together (D126)`).
|
||||
|
||||
Appearance scope and states. The Default layout and Thumbnail size defaults start the
|
||||
Images page only (a new visit, not the current view or a saved view); the embedded
|
||||
galleries (a prompt history's Shared gallery and its All images / Repeat run dialogs)
|
||||
have no toolbar, so they keep Grid/Comfortable, while the metadata, cache and card
|
||||
defaults apply there too. A host row whose `/settings` read failed stays disabled with
|
||||
"Server settings could not be read."; a rejected timeout says "Some changes were not
|
||||
saved. Review the highlighted settings." A display default another page saved under one
|
||||
of this tab's drafts flags that draft with the conflict status, whether it arrived while
|
||||
System was mounted (a `storage` event) or while it was not (checked again on mount, and
|
||||
before a write); the draft is never written over the other page's value without that
|
||||
status first. A change kept for this tab only is stored by the next write storage accepts.
|
||||
|
||||
Open question (saved-views follow-up, M8): a saved view stores no layout or size and
|
||||
always opens at Grid/Comfortable, even when the Default layout is Masonry. Should saved
|
||||
views store a layout, or follow the default?
|
||||
|
||||
`tests/browser/system-concept.spec.mjs` compares the shell and Overview with the concept
|
||||
(dark and light, 1600×1100 and 390×844), mapping only the deviations above; unmapped,
|
||||
the difference is exactly the D92, D93, D96 and D122 paths. Each surface's `top` is
|
||||
measured, which pins the concept's `.surface + .surface` 18px offset that concept.css
|
||||
also applies inside `.two-col` (copied as is). `statistics and health dialogs match
|
||||
the concept` maps D95 from the concept's own five rows (labels verbatim): it inserts
|
||||
Pair uniqueness after Canonical hashes and Consistency last, splits Missing files per
|
||||
source when there are several, replaces only the chips with the fixture's expected
|
||||
results, and asserts the full unmapped path list; it maps D96 and compares the
|
||||
Maintenance notice unchanged. `system comparison detects mutations` removes a row,
|
||||
renames a metric label, swaps the `--line` token, recolours a chip, renames a check,
|
||||
removes the totals table and renames a caption. `appearance matches the concept` maps
|
||||
only D97 and the real timeout (unmapped, exactly those paths differ), and `appearance
|
||||
comparison detects mutations` removes a row, renames a label and an option and swaps
|
||||
the `--line` token. `sources matches the concept with|without the notice` (both
|
||||
themes and sizes; the fixture's monitor watches the first source, as the concept's
|
||||
first sample is watched, and the concept's Later removes its notice) maps the real
|
||||
sources for the two samples, D98's Remove and Refresh files, and D113; unmapped,
|
||||
exactly those paths differ. `sources comparison detects mutations` removes the notice
|
||||
and a source row, renames a label and the Add source button and swaps the `--line`
|
||||
token. Paired captures (`{concept,production}-system-{dark,light}-{1600,390}.png`,
|
||||
`{concept,production}-{statistics,health}-{dark,light}.png`,
|
||||
`{concept,production}-appearance-{dark,light}-{1600,390}.png`,
|
||||
`{concept,production}-sources-{with,without}-notice-{dark,light}-{1600,390}.png`) are test artifacts; there
|
||||
are no PNG baselines. `tests/browser/workspace-system.spec.mjs` covers the real counts,
|
||||
the health check and metric 4, a failed check, the thumbnail row against the status
|
||||
endpoint (painted after the rest when its read is held, unavailable when it fails), the review row (Q6), the section links, the Statistics dialog against the
|
||||
route, a load failure with Retry and late answers after leaving, and Appearance: new
|
||||
visits versus the current view and a saved view, the embedded gallery keeping Grid,
|
||||
metadata off and on request, the cache (counted up to a request barrier), hidden paths,
|
||||
the video defaults, the host fields with their revision, a stale one and a failed read,
|
||||
reload and a second tab, a second tab's save while System was closed, storage refused,
|
||||
the D97 rows and reset. `tests/browser/workspace-system-sources.spec.mjs` covers the
|
||||
notice's Later and Review scan, a scoped scan from one source into Activity, Add source
|
||||
with its server check and Save, a refused folder, Remove down to auto-detect, an absent
|
||||
configured source, the three watch states, the monitoring and worker rows, Refresh
|
||||
files, the thumbnail hand-off, a save refused by a running scan and a failed read. Its hooks are
|
||||
`tests/browser/workspace_system_fixture.py` (`/__test__/system/*`: reset, injected
|
||||
failures, an absent source, a deleted linked file, unthumbnailed files, a fake image
|
||||
monitor) on the
|
||||
concept-shaped fixture, with delays through the shared request holds; reset also restores
|
||||
the server's host settings and its `config.json`, which must be the one inside the server's
|
||||
temp root (`tests/test_workspace_system_fixture.py`).
|
||||
|
||||
```sh
|
||||
python3 -m unittest tests.test_database_health tests.test_system_api tests.test_library \
|
||||
tests.test_workspace_system_fixture -v
|
||||
node --test tests/js/workspace-system-state.test.js tests/js/workspace-system-controls.test.js \
|
||||
tests/js/workspace-system-overview.test.js tests/js/workspace-state.test.js \
|
||||
tests/js/workspace-system-appearance.test.js tests/js/workspace-preferences.test.js \
|
||||
tests/js/workspace-inspection-state.test.js tests/js/workspace-images-state.test.js \
|
||||
tests/js/workspace-system-sources.test.js
|
||||
python3 -m unittest tests.test_settings_sources tests.test_scan_scope -v
|
||||
npm run --prefix tools/vendor test:workflows -- system-concept.spec.mjs workspace-system.spec.mjs \
|
||||
workspace-system-sources.spec.mjs \
|
||||
--project chromium --project touch --project firefox --project webkit
|
||||
```
|
||||
|
||||
## Workspace data fields for the concept pages (#249)
|
||||
|
||||
These fields are additive. The legacy `/prompt_manager/images/output` listing (no
|
||||
@@ -1763,8 +2252,24 @@ and skip the parse.
|
||||
integers past 2**53 as their exact digits, so the card and the inspector read
|
||||
the same digits. The parse runs after the file handle closes. A record over
|
||||
2 MiB (UTF-8), a parser error, a recursion error, a placeholder, or a
|
||||
non-integral, boolean or non-finite seed gives null. A card never shows a seed
|
||||
that differs from the inspector's
|
||||
non-integral, boolean or non-finite seed gives null. When the prompt graph's
|
||||
sampler stages read different seed inputs and the inspector's resolution names
|
||||
one sampler for a resolved pair (a detailer or pass-through chain, whose facts
|
||||
are the named stage's, D19), the card compares that stage's seed (read by the
|
||||
same parser, the other samplers hidden from it) with its own and shows no seed
|
||||
only when they differ. A seed that agrees, or one the named stage does not make
|
||||
known, is kept, as on main (a saved sampler beside a preview-only one keeps its
|
||||
seed; an ambiguous chain, whose facts merge and fill from the parser, keeps it).
|
||||
Only such graphs are resolved for the card, with `INSPECTION_PROFILE`
|
||||
(`test_a_chain_the_inspector_names_one_stage_of_shows_no_differing_seed`,
|
||||
`test_a_seed_that_agrees_with_the_named_stage_is_kept`,
|
||||
`test_a_named_ambiguous_chain_keeps_the_parser_seed`; a literal named seed needs no
|
||||
second parse, and stages sharing one linked seed skip the check. Measured on a
|
||||
200-card page of about 54 KB graphs, half multi-stage with per-stage seeds: 618 ms
|
||||
cold against main's 387 ms, about 1.2 ms more per such card, paid once per file
|
||||
through the header-facts cache below;
|
||||
before this rule a KSampler then FaceDetailer chain with two seeds broke the
|
||||
contract below). A card never shows a seed that differs from the inspector's
|
||||
(`test_card_seed_equals_inspector_seed_when_both_present`, which `JSON.parse`s
|
||||
the listing in Node and compares `String(seed)` with the inspector's seed,
|
||||
including one past 2**53). The card shows no seed for A1111 `parameters` files
|
||||
@@ -1880,6 +2385,162 @@ inspection's own stable stat (`tests/test_api_workspace_images.py`
|
||||
browser fixtures clear `media_image_tags`, `media_annotation_bytes` and
|
||||
`media_annotation_files` on reset and never the metadata row.
|
||||
|
||||
Review queue schema (#249, `database/review_schema.py`,
|
||||
`tests/test_review_schema.py`): the group `media_review_metadata` (one row:
|
||||
schema version, `review_index_revision`), `media_review_items` (one ambiguous
|
||||
resolver result per `(byte_id, resolver_version, evidence_hash)`, citing an
|
||||
annotation byte record `ON DELETE RESTRICT`, with the file's recorded
|
||||
`spelled_path` (the locator is case-folded on Windows, a listing target is
|
||||
not), 0..16 candidates, `limited`,
|
||||
`state` pending/selected/unresolved and a revision) and `media_review_decisions`
|
||||
(one row per item decision revision; `selected` exactly when a
|
||||
`candidate_hash` is present; `pair_id`/`image_id` navigation is set to NULL,
|
||||
never cascaded, when the pair or image is deleted, through
|
||||
`idx_media_review_decisions_pair`/`_image`). Evidence is canonical JSON
|
||||
(`canonical_evidence`: one object, sorted keys, no whitespace, ASCII only, no
|
||||
NaN/infinity, at most 8 MiB) and `evidence_hash` is its SHA-256. The group is
|
||||
valid only beside a valid annotation group (a review group without annotation
|
||||
tables is refused, catalog only, before the annotation install), installs like
|
||||
the annotation group (optional in old backups, required once current under
|
||||
`require_library_state`, SAVEPOINT install validated before and after), and
|
||||
validation re-canonicalizes every evidence document in one streamed pass. A
|
||||
startup runs that pass once (`test_startup_validates_stored_evidence_once`; the
|
||||
install trusts the annotation group it has just validated, and the SQL row
|
||||
predicates check type and size only, because the streamed pass proves the
|
||||
rest): 2.5 s to open a database of 40,000 items with 4 KiB of evidence each,
|
||||
was 7.7 s with two passes. Restore and `verify_database_file` keep the full
|
||||
validation, annotation group included. The browser fixtures clear
|
||||
`media_review_decisions` and `media_review_items` before the annotation tables,
|
||||
never the metadata row.
|
||||
|
||||
Needs review items (#249, `utils/review_capture.py`, `database/review.py`,
|
||||
`tests/test_review.py`, `tests/test_review_outcome.py`,
|
||||
`tests/test_review_choosable.py`): a file is queued only when its final outcome
|
||||
is ambiguous and unlinked and there is something to choose. Only scans produce
|
||||
items; the monitor links new outputs and clears pending items it resolves
|
||||
(maintainer rulings, 2026-10-23). The output scan (Activity's
|
||||
Scan for metadata and the legacy `/scan`) hands `record_ambiguous_file` a file
|
||||
the ingestion resolver finds `ambiguous` only when the browser-parser fallback
|
||||
then links nothing (status `ambiguous`) and no saved row links the file
|
||||
already; an ambiguous file a saved row links is reported `unchanged` ("Already
|
||||
linked to its prompt"), not Needs review. A file the fallback links, an exact
|
||||
pair the ingester stores, and `conflict` and `unresolved` files are not
|
||||
queued. Neither is an ambiguous file with nothing to choose: fewer than two of
|
||||
its candidates are `resolved` with non-blank positive text (`reviewable`). With
|
||||
none the file could not be traced; with exactly one, the untraced branches may
|
||||
still have made the file, so that text is not certain either. Both report
|
||||
`unresolved` ("could not be traced"), are never linked, and their pending,
|
||||
undecided items whose own evidence offers no choice (a limited item counts as
|
||||
a choice when it holds at least two candidates) leave the queue on the next
|
||||
scan or monitor pass (`settle_untraced_review_items`). The maintainer's
|
||||
second live check found 39 items whose 11 candidates were all "unresolved". The queue heals itself: after each batch, the scan removes the pending
|
||||
items of every listed file (read or skipped as known) that a saved row now
|
||||
links, and the monitor does the same for each ambiguous output it links
|
||||
(`settle_linked_review_items`; the review index revision moves). Only an
|
||||
undecided item goes (`pending`, revision 0, no decision row): a decision is
|
||||
never deleted or changed, so no schema state was added. The monitor never
|
||||
queues an output it linked; an unlinked file passed to `record_review_file`
|
||||
follows the scan's rule (a file the fallback would link is `skipped`). The
|
||||
maintainer's live check of #284/#288 found the queue at 10,232 against a review
|
||||
group of 4,677: every fallback-linked ambiguous file had been queued. It captures
|
||||
the file (scope `needs-review`), reads its metadata again from the capture,
|
||||
resolves it with the default profile and stores only an ambiguous result,
|
||||
bound to the capture's byte record (created or refreshed like an observe) and
|
||||
fenced by the capture's storage epoch. Evidence is the resolver reason plus one
|
||||
`{sampler_id, state, reason, positive, negative, hash}` per selectable
|
||||
candidate (nested ambiguous stages flattened, duplicates dropped; each side
|
||||
`{text, storage_text, provenance}`; `hash` = SHA-256 of the canonical candidate
|
||||
without it). More than 16 candidates, or evidence over 8 MiB, is a `limited`
|
||||
item without candidates. Recording is idempotent per `(byte_id,
|
||||
resolver_version, evidence_hash)` and never touches an existing item or its
|
||||
decisions; changed bytes add a new pending item and keep the old one. The
|
||||
capture is released before the write; no model is loaded and no pixel is
|
||||
decoded. Scan statuses, counters, outcome rows and the legacy SSE events are
|
||||
unchanged apart from the already-linked and nothing-to-choose cases above; the job's `scope` gains
|
||||
`review_items` (this scan's files that are in the pending queue after it, new
|
||||
or already there: the review group's ambiguous rows, less files a capture could
|
||||
not read or a decision already settled), `review_added` (new items),
|
||||
`review_settled` (items removed because their file is linked now),
|
||||
`review_untraced` (items removed because they offer nothing to choose) and
|
||||
`review_errors`. The log ends with `? N images added to Needs review`,
|
||||
`? N images already in Needs review`, `= N images left Needs review · now
|
||||
linked`, `= N images left Needs review · nothing to choose` and `! N images could not be added to Needs review` (failures, e.g.
|
||||
capture capacity), each only when N > 0. Recording captures in the background: it never takes the store's
|
||||
last worker or capture slot, which stay free for an interactive Tag dialog or
|
||||
Apply, and waits up to 2 s for room before counting the file as not added
|
||||
(`test_background_recording_leaves_a_worker_for_the_tag_dialog`). The monitor's
|
||||
pass runs after its own link, best effort: a failure is logged and the link
|
||||
stays. Recording follows the registry's `records`
|
||||
admission (#289): the scan's recording runs under its own `records` job, and
|
||||
the monitor's (one file, not a job) is refused while a `records` or `guarded`
|
||||
job or hold runs. It then records nothing, logs that the file waits for the
|
||||
next scan, and answers `deferred`; the next "new" scan reads the ambiguous file
|
||||
again (`test_monitor_recording_is_refused_beside_record_changing_jobs`). A new item moves `review_index_revision`. A
|
||||
rescan does not copy or hash an ambiguous file whose current size and
|
||||
`mtime_ns` equal the verified fingerprint of bytes an item already cites (a
|
||||
stat compare); the file still stays in the default "new" scope, because it is
|
||||
not a known image. The skip and a duplicate insert record the current spelling
|
||||
of the locator on its items (`refresh_review_spelling`, the review index
|
||||
revision moves, item revisions do not), so a case-only rename on Windows or
|
||||
macOS keeps its item applicable. Residual risks of the skip, like S1's: an
|
||||
in-place rewrite that keeps both size and `mtime_ns` is never re-queued (Apply
|
||||
still re-hashes, so it cannot link those bytes), and a file that is deleted or
|
||||
moved leaves its pending item counted in `total` (its target then formats as
|
||||
`unavailable` or fails to open, and Leave unresolved clears it).
|
||||
|
||||
Needs review decisions and queue reads (#249, `py/api/review.py`,
|
||||
`database/review.py`, `database/review_decisions.py`,
|
||||
`tests/test_api_review.py`): `GET /prompt_manager/review/items?limit=1..40&cursor=`
|
||||
answers `{success, library_epoch, total, items, next_cursor}` with pending items
|
||||
oldest first; `total` is the exact pending count. The cursor is an HMAC keyset
|
||||
on `(created_at, id)` bound to the storage epoch and `review_index_revision`: a
|
||||
malformed one is 400, a well-formed one this process cannot verify (tampered,
|
||||
or opened before a ComfyUI restart: the secret is per process) is 409
|
||||
`review-conflict` with the current `total`, one from before a restore is 409
|
||||
`storage-changed`, one from before any queue change (a decision, a new item, a
|
||||
refreshed spelling) is 409 `review-conflict`.
|
||||
Targets are formatted from `spelled_path` against the current roots (the
|
||||
roots' real paths are the only file access); a file under no current root is
|
||||
`unavailable` with a null target. Every read and decision lists the roots and
|
||||
resolves real paths on a worker thread, never on the event loop. `GET .../items/{id}` adds `candidates`
|
||||
(`hash`, `sampler_id`, `state` resolved/unresolved, `reason`, `label` "<positive
|
||||
title> / <negative title>" from the saved prompt's Library title else the
|
||||
first line, and the role texts), `dependency_fingerprint` (SHA-256 of the
|
||||
file's legacy rows under every spelling of the same physical file, positive
|
||||
hashes and pair links with their negative hashes, as the ingester reads them) and `sha256`; a detail over 8 MiB, or a
|
||||
limited item, has no candidates. `POST .../items/{id}/decision
|
||||
{expected_epoch, expected_revision, capture_token, sha256, dependency_fingerprint,
|
||||
choice}` with a candidate hash (Apply) requires a capture of the item's file
|
||||
whose bytes are the item's, re-hashes the current file (`file-changed`
|
||||
otherwise, also for a fresh capture of new bytes), re-reads the capture and
|
||||
requires that its bytes are still ambiguous and still offer the chosen
|
||||
candidate (`review-conflict` otherwise; a resolver change that rewords the
|
||||
reason or reorders candidates leaves the item applicable, and
|
||||
`RESOLVER_VERSION` in `utils/prompt_pair_resolver.py` is the one version
|
||||
ingestion provenance and review share), then in one `_pair_transaction`
|
||||
checks the epoch, the item revision and state (`review-conflict` with the
|
||||
current item for a repeat), the re-hashed digest, the dependency fingerprint
|
||||
and the public ingester's compatible-only rules through
|
||||
`_ingest_resolved_pair_in` (any refusal is 409 `association-conflict` with
|
||||
zero writes, explained inline; Apply never reparents a link). Physical
|
||||
identity: the roots are real paths while the monitor links under the
|
||||
directory it watches, so a symlinked or junctioned root, `subst` or a mapped
|
||||
drive spells the same file otherwise. Every legacy row whose spelling ends with
|
||||
the file's name (the link summaries' unindexed suffix lookup) is resolved by
|
||||
`normcase(realpath)` against the locator, else `samefile`; two rows of the
|
||||
file refuse the choice, one is enriched under the spelling it was linked
|
||||
with, and a same-name row the request did not classify is a changed
|
||||
association (`TestPhysicalIdentity`, symlinked root and a portable fake
|
||||
junction), and writes
|
||||
the link with `kind:"user-selection"` provenance (decision id, reviewed digest,
|
||||
candidate hash, resolver version, role provenance), the decision row, the
|
||||
item's new state and revision and `review_index_revision`. `"unresolved"`
|
||||
records the decision only and accepts `capture_token: null`. A successful
|
||||
decision releases its capture. No decision changes run or usage counts or
|
||||
calls an execution writer; a Tag save never changes a review item. Errors carry
|
||||
fixed sentences: `review-failed` / `decision-failed` 500s never echo exception
|
||||
text or paths.
|
||||
|
||||
Capture store (#249, `utils/media_capture.py`, `tests/test_media_capture.py`):
|
||||
one explicit action captures one rooted media file into an owner-only
|
||||
`prompt-manager-captures-*` temp directory, streaming it in 1 MiB chunks while
|
||||
@@ -1896,8 +2557,12 @@ directories are `CaptureDenied`; the locator is
|
||||
`normcase(join(realpath(root), relative))`, so a symlinked root shares the
|
||||
listing's identity. Fixed limits: 256 MiB per file (stat and stream), 8
|
||||
completed captures, 512 MiB reserved, 2 capture/verify workers, 900 s idle,
|
||||
1800 s hard; admission is nonblocking (`CaptureCapacity`) and lifetimes are
|
||||
tested with an injected clock. An acquired capture outlives expiry and release
|
||||
1800 s hard; interactive admission is nonblocking (`CaptureCapacity`), a
|
||||
`background=True` capture (Needs review recording) never takes the last worker
|
||||
or capture slot and waits up to `BACKGROUND_WAIT_SECONDS` (2 s) for room, and
|
||||
lifetimes are tested with an injected clock. The host's on_cleanup close is
|
||||
final: a recording still in flight gets a capture error instead of creating a
|
||||
new temp directory after it. An acquired capture outlives expiry and release
|
||||
until its last use ends. Header-only dimensions (4096 px per side,
|
||||
16,777,216 px) and video mark Generate unsupported; `verify` re-hashes the
|
||||
current file without a copy. Every handle closes before unlink (a patched
|
||||
@@ -2272,12 +2937,41 @@ skip without ffmpeg). The remaining known divergences, checked against ffprobe 9
|
||||
`unreadable`); ffprobe reads such files.
|
||||
- An EBML DocType other than `matroska`/`webm` is an invalid container locally; ffmpeg
|
||||
only warns and reads it.
|
||||
- Envelope JSON deeper than 1,000 levels is refused locally as Python's default
|
||||
recursion limit refuses it; the server's real limit depends on the interpreter and
|
||||
call stack. A duplicate key in an object enclosing a nest deeper than the running
|
||||
- Deep JSON (local roles, PR L3, and video envelopes): both browser CPython-JSON
|
||||
parsers (`py-json.mjs`, `local-video.mjs`) share one cap, `JSON_MAX_DEPTH` = 1,000
|
||||
nested arrays or objects (each counts, an empty one too); past it a record is invalid
|
||||
JSON locally. The server's `json.loads` raises RecursionError at a limit that depends
|
||||
on the Python, measured on Linux x86-64: 993 levels from a bare script on 3.10 and
|
||||
3.11 (the 1,000 recursion limit minus the caller's frames, so about 990 inside the
|
||||
server), 9,997 on 3.12 and 9,998 on 3.13 (the C recursion counter; lower on Windows
|
||||
builds), and about 52,150 on 3.14 (stack-based, varies with the thread's stack).
|
||||
The record kinds answer: a deep `prompt` graph (or `comfyassets_prompt_pairs`
|
||||
envelope) is malformed; a deep `workflow` is ignored, so the overwritten PromptManager
|
||||
check (R1) does not run and a prompt can resolve; a deep video `comment` or
|
||||
`description` envelope makes the file unreadable on both sides (the server's
|
||||
RecursionError reaches `read_inspection_file`). That gives two directions:
|
||||
- 3.10/3.11, between the server's limit and 1,000: the server refuses and the local
|
||||
page reads, so a deep prompt resolves locally while the server says malformed, and
|
||||
a deep workflow can make R1 refuse locally where the server resolves.
|
||||
- 3.12 and newer, from 1,001 up to the server's limit: the local page refuses and
|
||||
the server reads, so a deep prompt is malformed locally while the server resolves,
|
||||
and a deep workflow is ignored locally while the server can refuse it under R1.
|
||||
Past both limits they agree. `test_local_json_depth_cap_at_its_boundary` checks the
|
||||
local answer at 999, 1,000 and 1,001 for both kinds and that any server difference
|
||||
is only one of the two directions above.
|
||||
- Local roles (PR L3): `\d` in the A1111 settings footer is any Unicode decimal
|
||||
digit on both sides, but the browser's Unicode version can be newer than the server
|
||||
Python's, so a footer whose step count uses a digit added after Unicode 15 can differ.
|
||||
`str.lower()` and `toLowerCase()` differ only for characters no node class name uses.
|
||||
- A duplicate key in an object enclosing a nest deeper than the running
|
||||
Python's json limit is `malformed` locally and `limited` on that Python.
|
||||
- Decimal exponent limits assume CPython's C `_decimal`; on a `_pydecimal` interpreter
|
||||
the server would read the over-limit exponents the local projection refuses.
|
||||
- Follow-up (two CPython-JSON emulations, review M6): `local-video.mjs`
|
||||
(`pythonJsonLoads`/`pythonFloatRepr`, numbers kept as lexemes) and `py-json.mjs`
|
||||
(`pyJsonLoads`/`pyFloatRepr`, Python value kinds) both emulate `json.loads`; only the
|
||||
depth cap is shared so far. A fix to one parser can miss the other, so the video
|
||||
reader should move onto `py-json.mjs` in a later PR.
|
||||
|
||||
Preserve raw logs, failed traces, paired PNGs, geometry, runtime versions and
|
||||
source hashes before later managed commands clear their output directories.
|
||||
|
||||
+15
@@ -122,6 +122,21 @@ try:
|
||||
_global_prompt_tracker = get_prompt_tracker(_global_db)
|
||||
_global_image_monitor = get_image_monitor(_global_db, _global_prompt_tracker)
|
||||
|
||||
# The Needs review pass for ambiguous new outputs runs through the API's
|
||||
# capture store (the one the scan and the Tag dialog use): a linked output
|
||||
# is never queued and its pending items leave the queue. It stops before
|
||||
# that store is closed at shutdown.
|
||||
_review_api = globals().get("api")
|
||||
if _review_api is not None:
|
||||
_global_image_monitor.set_review_recorder(_review_api.record_review_file)
|
||||
_review_host = getattr(config.server_instance, "app", None)
|
||||
if _review_host is not None and hasattr(_review_host, "on_cleanup"):
|
||||
|
||||
async def _stop_review_recording(app, monitor=_global_image_monitor):
|
||||
monitor.set_review_recorder(None)
|
||||
|
||||
_review_host.on_cleanup.insert(0, _stop_review_recording)
|
||||
|
||||
# Start monitoring
|
||||
_global_image_monitor.start_monitoring()
|
||||
_init_logger.info("Global image monitoring system started successfully")
|
||||
|
||||
@@ -0,0 +1,354 @@
|
||||
"""The read-only health check behind System › Overview and Maintenance.
|
||||
|
||||
``run_health_checks`` opens its own ``mode=ro`` connection, reads one
|
||||
snapshot and closes the connection in ``finally``; it never writes. Each
|
||||
check reports ``passed``, ``issues`` (with an exact ``count``; the
|
||||
integrity check stops at ``INTEGRITY_LIMIT`` problems and then says
|
||||
``capped``),
|
||||
``unavailable`` (it could not be checked here, so ``count`` is None, or
|
||||
``missing_files`` checked only some sources and found nothing) or ``failed``
|
||||
(it raised; the other checks still run). ``missing_files`` also reports each
|
||||
source and whether the check was ``partial``; it reads the image rows inside
|
||||
the snapshot and checks their files after the snapshot ended, so a slow source
|
||||
never holds the WAL. At most ``MAX_DETAIL`` detail strings are returned per
|
||||
check, and none of them carries a server path.
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import ntpath
|
||||
import os
|
||||
import sqlite3
|
||||
from pathlib import PurePosixPath, PureWindowsPath
|
||||
from urllib.parse import quote
|
||||
|
||||
try:
|
||||
from ..utils.hashing import generate_prompt_hash
|
||||
from .models import normalize_image_path
|
||||
except ImportError:
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from database.models import normalize_image_path
|
||||
|
||||
logger = logging.getLogger("prompt_manager.database.health")
|
||||
|
||||
MAX_DETAIL = 10 # detail strings returned per check
|
||||
INTEGRITY_LIMIT = 100 # PRAGMA integrity_check(N): problems SQLite reports
|
||||
BATCH_ROWS = 100 # image rows read per fetch
|
||||
REVIEW_UNAVAILABLE = "Needs review is not available yet."
|
||||
CHECK_FAILED = "The check could not finish."
|
||||
# Check ids and the labels of the concept's health report, in its order;
|
||||
# pair uniqueness and consistency are the added rows (D63, now D95).
|
||||
CHECKS = (
|
||||
("integrity", "Integrity check"),
|
||||
("foreign_keys", "Foreign keys & pair references"),
|
||||
("canonical_hashes", "Canonical hashes"),
|
||||
("pair_uniqueness", "Pair uniqueness"),
|
||||
("missing_files", "Missing files"),
|
||||
("ambiguous_metadata", "Ambiguous metadata"),
|
||||
("consistency", "Consistency"),
|
||||
)
|
||||
_DATABASE_CHECKS = (
|
||||
"integrity",
|
||||
"foreign_keys",
|
||||
"canonical_hashes",
|
||||
"pair_uniqueness",
|
||||
"missing_files",
|
||||
"consistency",
|
||||
)
|
||||
|
||||
|
||||
def iter_consistency_issues(conn):
|
||||
"""The legacy consistency issues: tag and image rows of a missing prompt."""
|
||||
for ref in conn.execute(
|
||||
"SELECT pt.prompt_id FROM prompt_tags pt "
|
||||
"LEFT JOIN prompts p ON pt.prompt_id = p.id WHERE p.id IS NULL"
|
||||
):
|
||||
yield f"prompt_tags entry references non-existent prompt {ref[0]}"
|
||||
for ref in conn.execute(
|
||||
"SELECT gi.id, gi.prompt_id FROM generated_images gi "
|
||||
"LEFT JOIN prompts p ON gi.prompt_id = p.id WHERE p.id IS NULL"
|
||||
):
|
||||
yield f"Image {ref[0]} references non-existent prompt {ref[1]}"
|
||||
|
||||
|
||||
def _result(count, detail, *, status=None):
|
||||
return {
|
||||
"status": status or ("issues" if count else "passed"),
|
||||
"count": count,
|
||||
"detail": detail[:MAX_DETAIL],
|
||||
}
|
||||
|
||||
|
||||
def _counted(lines):
|
||||
"""(count, first MAX_DETAIL lines) of an iterable, consumed once."""
|
||||
count, detail = 0, []
|
||||
for line in lines:
|
||||
count += 1
|
||||
if len(detail) < MAX_DETAIL:
|
||||
detail.append(line)
|
||||
return _result(count, detail)
|
||||
|
||||
|
||||
def readonly_uri(db_path, *, windows=None):
|
||||
"""The ``mode=ro`` SQLite URI of *db_path*, on this platform or the given one.
|
||||
|
||||
pathlib spells a UNC path ``\\\\server\\share\\x.db`` as
|
||||
``file://server/share/x.db``, a remote authority SQLite refuses to open.
|
||||
With an empty authority (``file:////server/share/x.db``) SQLite hands
|
||||
Windows the ``//server/share`` path, which it opens as the UNC path.
|
||||
"""
|
||||
if windows is None:
|
||||
windows = os.name == "nt"
|
||||
db_path = os.path.abspath(db_path)
|
||||
if not windows:
|
||||
return PurePosixPath(db_path).as_uri() + "?mode=ro"
|
||||
path = PureWindowsPath(db_path)
|
||||
if ntpath.splitdrive(str(path))[0].startswith("\\\\"):
|
||||
return "file:" + quote("//" + path.as_posix()) + "?mode=ro"
|
||||
return path.as_uri() + "?mode=ro"
|
||||
|
||||
|
||||
def _integrity(conn, _roots):
|
||||
rows = [
|
||||
row[0] for row in conn.execute(f"PRAGMA integrity_check({INTEGRITY_LIMIT})")
|
||||
]
|
||||
if rows == ["ok"]:
|
||||
return {**_result(0, []), "capped": False}
|
||||
# SQLite stops listing at the limit: there may be more problems than that.
|
||||
return {**_result(len(rows), rows), "capped": len(rows) >= INTEGRITY_LIMIT}
|
||||
|
||||
|
||||
def _foreign_keys(conn, _roots):
|
||||
return _counted(
|
||||
f"{table} row {rowid} references a missing {parent} row"
|
||||
for table, rowid, parent, _fkid in conn.execute("PRAGMA foreign_key_check")
|
||||
)
|
||||
|
||||
|
||||
def _prompt_hash(text):
|
||||
return generate_prompt_hash(text) if isinstance(text, str) else None
|
||||
|
||||
|
||||
def _canonical_hashes(conn, _roots):
|
||||
conn.create_function("pm_prompt_hash", 1, _prompt_hash, deterministic=True)
|
||||
return _counted(
|
||||
f"Prompt {row[0]}: the stored hash does not match its text"
|
||||
for row in conn.execute(
|
||||
"SELECT id FROM prompts WHERE hash IS NOT pm_prompt_hash(text) ORDER BY id"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _pair_uniqueness(conn, _roots):
|
||||
groups = conn.execute(
|
||||
"SELECT positive_prompt_id,negative_prompt_id,COUNT(*) FROM prompt_pairs "
|
||||
"GROUP BY positive_prompt_id,negative_prompt_id HAVING COUNT(*)>1 "
|
||||
"ORDER BY positive_prompt_id,negative_prompt_id"
|
||||
).fetchall()
|
||||
detail = [
|
||||
f"Positive prompt {positive} with "
|
||||
+ ("no negative" if negative is None else f"negative {negative}")
|
||||
+ f": {rows} pair rows"
|
||||
for positive, negative, rows in groups
|
||||
]
|
||||
# The surplus rows: what keeping one row per pair would leave out.
|
||||
return _result(sum(rows - 1 for _p, _n, rows in groups), detail)
|
||||
|
||||
|
||||
def _consistency(conn, _roots):
|
||||
return _counted(iter_consistency_issues(conn))
|
||||
|
||||
|
||||
def _readable(root):
|
||||
return os.path.isdir(root) and os.access(root, os.R_OK | os.X_OK)
|
||||
|
||||
|
||||
def _root_prefix(root):
|
||||
return normalize_image_path(root).rstrip("\\/") + os.sep
|
||||
|
||||
|
||||
def _source_name(root, index):
|
||||
"""A source's folder name for the report (never its server path)."""
|
||||
return os.path.basename(str(root).rstrip("\\/")) or f"Source {index + 1}"
|
||||
|
||||
|
||||
def _owner(path, prefixes):
|
||||
"""Index of the longest source prefix *path* lies under, or None."""
|
||||
owner, longest = None, -1
|
||||
for index, prefix in enumerate(prefixes):
|
||||
if path.startswith(prefix) and len(prefix) > longest:
|
||||
owner, longest = index, len(prefix)
|
||||
return owner
|
||||
|
||||
|
||||
def _image_owner(image_path, file_path, prefixes):
|
||||
spelled = file_path or normalize_image_path(image_path)
|
||||
owner = _owner(spelled, prefixes)
|
||||
if owner is None:
|
||||
owner = _owner(normalize_image_path(os.path.realpath(image_path)), prefixes)
|
||||
return owner
|
||||
|
||||
|
||||
def _unavailable_line(unavailable, *, partial):
|
||||
noun = "source is" if unavailable == 1 else "sources are"
|
||||
effect = "its files were" if unavailable == 1 else "their files were"
|
||||
if not partial:
|
||||
effect = "missing files were"
|
||||
return f"{unavailable} image {noun} unavailable, so {effect} not counted."
|
||||
|
||||
|
||||
def _missing_files(conn, roots):
|
||||
"""Links whose file is gone, counted per source (only readable ones).
|
||||
|
||||
Each source reports ``checked`` with its own count, or ``unavailable``
|
||||
(an unmounted drive) with no count: its links are never turned into a
|
||||
count, while the readable sources still are. The check's ``count`` is the
|
||||
readable sources' total (None when none is readable) and ``partial`` says
|
||||
some sources were not checked. Links outside every source are named in the
|
||||
detail, not counted.
|
||||
|
||||
Inside the snapshot this only reads the image rows; it returns the step
|
||||
that checks their files, which runs after the snapshot ended.
|
||||
"""
|
||||
if not roots:
|
||||
result = _result(None, ["No image source is available."], status="unavailable")
|
||||
return {**result, "partial": False, "sources": []}
|
||||
rows = []
|
||||
cursor = conn.execute(
|
||||
"SELECT id,image_path,filename,file_path FROM generated_images ORDER BY id"
|
||||
)
|
||||
while batch := cursor.fetchmany(BATCH_ROWS):
|
||||
rows.extend(batch)
|
||||
return lambda: _check_files(rows, roots)
|
||||
|
||||
|
||||
def _check_files(rows, roots):
|
||||
"""The ``missing_files`` result of the image *rows*, from the file system."""
|
||||
readable = [_readable(root) for root in roots]
|
||||
sources = [
|
||||
{
|
||||
"name": _source_name(root, index),
|
||||
"status": "checked" if ok else "unavailable",
|
||||
"count": 0 if ok else None,
|
||||
}
|
||||
for index, (root, ok) in enumerate(zip(roots, readable))
|
||||
]
|
||||
unavailable = readable.count(False)
|
||||
if not any(readable):
|
||||
result = _result(
|
||||
None, [_unavailable_line(unavailable, partial=False)], status="unavailable"
|
||||
)
|
||||
return {**result, "partial": False, "sources": sources}
|
||||
prefixes = [_root_prefix(root) for root in roots]
|
||||
outside, missing = 0, []
|
||||
for image_id, image_path, filename, file_path in rows:
|
||||
owner = _image_owner(image_path, file_path, prefixes)
|
||||
if owner is None:
|
||||
outside += 1
|
||||
elif readable[owner] and not os.path.exists(image_path):
|
||||
sources[owner]["count"] += 1
|
||||
if len(missing) < MAX_DETAIL:
|
||||
name = filename or os.path.basename(image_path)
|
||||
missing.append(f"Image {image_id}: {name} is missing")
|
||||
count = sum(source["count"] or 0 for source in sources)
|
||||
detail = [_unavailable_line(unavailable, partial=True)] if unavailable else []
|
||||
detail += missing
|
||||
if outside:
|
||||
links = "link" if outside == 1 else "links"
|
||||
verb = "was" if outside == 1 else "were"
|
||||
detail.append(
|
||||
f"{outside} {links} outside the image sources {verb} not checked."
|
||||
)
|
||||
status = None if count or not unavailable else "unavailable"
|
||||
result = _result(count, detail, status=status)
|
||||
return {**result, "partial": bool(unavailable), "sources": sources}
|
||||
|
||||
|
||||
def _ambiguous(review_count):
|
||||
if review_count is None:
|
||||
return _result(None, [REVIEW_UNAVAILABLE], status="unavailable")
|
||||
return _result(review_count, [])
|
||||
|
||||
|
||||
def overall_status(checks) -> str:
|
||||
"""``passed`` only when every check passed; any check not run is incomplete."""
|
||||
statuses = {check["status"] for check in checks}
|
||||
if statuses & {"unavailable", "failed"} or any(
|
||||
check.get("partial") for check in checks
|
||||
):
|
||||
return "incomplete"
|
||||
return "issues" if "issues" in statuses else "passed"
|
||||
|
||||
|
||||
def _failed():
|
||||
return _result(None, [CHECK_FAILED], status="failed")
|
||||
|
||||
|
||||
def _run_check(check_id, step, *args):
|
||||
"""One check's result, or None (reported as failed) when it raised."""
|
||||
try:
|
||||
return step(*args)
|
||||
except Exception: # any error fails this check only; the others still run
|
||||
logger.exception(f"Health check {check_id} failed")
|
||||
return None
|
||||
|
||||
|
||||
def _run_database_checks(db_path, roots):
|
||||
"""Each database check's result (None when it raised), from one snapshot.
|
||||
|
||||
A check may return a step instead of its result: the step runs after the
|
||||
snapshot ended and the connection closed (``missing_files``' file checks).
|
||||
"""
|
||||
results = dict.fromkeys(_DATABASE_CHECKS)
|
||||
try:
|
||||
conn = sqlite3.connect(readonly_uri(db_path), uri=True)
|
||||
except sqlite3.Error:
|
||||
logger.exception("Health check could not open the database")
|
||||
return results
|
||||
try:
|
||||
conn.execute("BEGIN") # one read snapshot for every check
|
||||
for check_id in _DATABASE_CHECKS:
|
||||
runner = globals()["_" + check_id] # looked up per run (tests patch it)
|
||||
results[check_id] = _run_check(check_id, runner, conn, roots)
|
||||
except sqlite3.Error:
|
||||
logger.exception("Health check could not read the database")
|
||||
finally:
|
||||
conn.close()
|
||||
for check_id, result in results.items():
|
||||
if callable(result):
|
||||
results[check_id] = _run_check(check_id, result)
|
||||
return results
|
||||
|
||||
|
||||
class HealthOperations:
|
||||
"""Mixin for PromptDatabase: the read-only System health check."""
|
||||
|
||||
def run_health_checks(self, roots, *, review_count=None) -> dict:
|
||||
"""Run every check and return a ``HealthReport``.
|
||||
|
||||
Args:
|
||||
roots: The image source directories to judge missing files under:
|
||||
the effective sources plus any configured source that is not
|
||||
available (reported as an unavailable source of ``missing_files``).
|
||||
review_count: The Needs-review pending count, or None while no
|
||||
review queue exists (``ambiguous_metadata`` is then unavailable).
|
||||
"""
|
||||
if review_count is not None and (
|
||||
type(review_count) is not int or review_count < 0
|
||||
):
|
||||
raise ValueError("review_count must be a non-negative integer or None")
|
||||
roots = [str(root) for root in roots]
|
||||
database = _run_database_checks(self.model.db_path, roots)
|
||||
checks = []
|
||||
for check_id, label in CHECKS:
|
||||
if check_id == "ambiguous_metadata":
|
||||
result = _ambiguous(review_count)
|
||||
else:
|
||||
result = database[check_id] or _failed()
|
||||
checks.append({"id": check_id, "label": label, **result})
|
||||
return {
|
||||
"success": True,
|
||||
"checked_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
"status": overall_status(checks),
|
||||
"checks": checks,
|
||||
}
|
||||
@@ -628,3 +628,57 @@ class LibraryOperations:
|
||||
"kind": "prompt-manager-library",
|
||||
"positives": [owners[key] for key in order],
|
||||
}
|
||||
|
||||
def library_counts(self) -> dict:
|
||||
"""System › Overview counts from one snapshot, writing nothing.
|
||||
|
||||
``positive_prompts`` and ``negative_variants`` are the unfiltered
|
||||
Library's ``total`` and ``variant_total``: eligible parents and active
|
||||
pairs (every pair's positive is eligible). ``images`` counts indexed
|
||||
``generated_images`` rows, not files on disk. An image has no rating
|
||||
of its own (it shows its linked prompt's), so ``rated_images`` counts
|
||||
the indexed images whose linked prompt is rated. ``review_pending`` is
|
||||
the Needs review queue's pending total.
|
||||
"""
|
||||
with self.library_snapshot() as (conn, _epoch, _ceiling):
|
||||
row = conn.execute(
|
||||
"SELECT (SELECT COUNT(*) FROM prompts p WHERE " + _ELIGIBLE + "),"
|
||||
"(SELECT COUNT(*) FROM prompt_pairs WHERE archived=0),"
|
||||
"(SELECT COUNT(*) FROM generated_images),"
|
||||
"(SELECT COUNT(*) FROM generated_images gi JOIN prompts r "
|
||||
"ON r.id=gi.prompt_id WHERE r.rating IS NOT NULL),"
|
||||
"(SELECT COUNT(*) FROM media_review_items WHERE state='pending')"
|
||||
).fetchone()
|
||||
return {
|
||||
"positive_prompts": row[0],
|
||||
"negative_variants": row[1],
|
||||
"images": row[2],
|
||||
"rated_images": row[3],
|
||||
"review_pending": row[4],
|
||||
}
|
||||
|
||||
def usage_by_category(self) -> list[dict]:
|
||||
"""Runs and average rating of the Library's parents, per category.
|
||||
|
||||
Only eligible parents count, so a negative-only prompt's runs (which
|
||||
repeat its pairs' runs) are never added twice. A blank category groups
|
||||
with no category as ``None``. Unrated parents are left out of the
|
||||
average, which is ``None`` when no parent is rated. Most runs first.
|
||||
"""
|
||||
with self.library_snapshot() as (conn, _epoch, _ceiling):
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM (SELECT NULLIF(TRIM(p.category),'') AS category,"
|
||||
"COALESCE(SUM(p.run_count),0) AS runs,AVG(p.rating) AS rating "
|
||||
"FROM prompts p WHERE " + _ELIGIBLE + " GROUP BY 1) "
|
||||
"ORDER BY runs DESC,category IS NULL,category"
|
||||
).fetchall()
|
||||
return [
|
||||
{
|
||||
"category": row["category"],
|
||||
"runs": row["runs"],
|
||||
"average_rating": (
|
||||
None if row["rating"] is None else round(row["rating"], 2)
|
||||
),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
+20
-3
@@ -18,6 +18,11 @@ from typing import Optional, Set, Tuple
|
||||
from .library_schema import library_schema_error, migrate_library_state
|
||||
from .image_favorites import image_favorites_schema_error, migrate_image_favorites
|
||||
from .annotation_schema import annotation_schema_error, migrate_annotation_state
|
||||
from .review_schema import (
|
||||
migrate_review_state,
|
||||
review_schema_error,
|
||||
review_without_annotations_error,
|
||||
)
|
||||
from .saved_views import migrate_saved_views, saved_views_schema_error
|
||||
from .execution_history_schema import (
|
||||
history_schema_error,
|
||||
@@ -303,7 +308,14 @@ class PromptModel:
|
||||
migrate_library_state(conn)
|
||||
migrate_image_favorites(conn)
|
||||
migrate_saved_views(conn)
|
||||
# A review group without its annotation group is refused before the
|
||||
# annotation install could make it look valid (catalog only; the rows
|
||||
# are validated once, in migrate_review_state).
|
||||
review_error = review_without_annotations_error(conn)
|
||||
if review_error:
|
||||
raise sqlite3.DatabaseError(review_error)
|
||||
migrate_annotation_state(conn)
|
||||
migrate_review_state(conn)
|
||||
|
||||
def _migrate_prompt_pairs(self, conn: sqlite3.Connection) -> None:
|
||||
conn.execute("SAVEPOINT prompt_pair_schema")
|
||||
@@ -1023,9 +1035,9 @@ class PromptModel:
|
||||
Args:
|
||||
path: File to inspect
|
||||
require_library_state: Also require the tables every current
|
||||
database has (Library state, image favorites, saved views
|
||||
and image annotations), as a migrated restore stage must;
|
||||
old backups may lack them.
|
||||
database has (Library state, image favorites, saved views,
|
||||
image annotations and the review queue), as a migrated
|
||||
restore stage must; old backups may lack them.
|
||||
|
||||
Returns:
|
||||
(ok, reason): reason is "ok" when the file is acceptable, otherwise
|
||||
@@ -1095,6 +1107,11 @@ class PromptModel:
|
||||
)
|
||||
if annotation_error:
|
||||
return False, annotation_error
|
||||
review_error = review_schema_error(
|
||||
conn, allow_missing=not require_library_state
|
||||
)
|
||||
if review_error:
|
||||
return False, review_error
|
||||
return True, "ok"
|
||||
except sqlite3.Error as e:
|
||||
return False, f"Invalid SQLite database: {e}"
|
||||
|
||||
+20
-31
@@ -4,6 +4,7 @@ Database operations for KikoTextEncode prompt storage and retrieval.
|
||||
|
||||
import sqlite3
|
||||
import json
|
||||
import math
|
||||
import datetime
|
||||
import os
|
||||
from typing import Optional, List, Dict, Any, Set, Tuple
|
||||
@@ -13,6 +14,8 @@ from .pairs import PairDependencyError, PairOperations
|
||||
from .library import LibraryOperations, titled
|
||||
from .image_favorites import ImageFavoriteOperations
|
||||
from .annotations import AnnotationOperations
|
||||
from .review import ReviewOperations
|
||||
from .review_decisions import ReviewDecisionOperations
|
||||
from .saved_views import SavedViewOperations
|
||||
from .library_commands import LibraryCommandOperations
|
||||
from .vocabulary import VocabularyOperations
|
||||
@@ -20,11 +23,13 @@ from .execution_history import ExecutionHistoryOperations
|
||||
from .execution_contexts import ExecutionContextOperations, LIVE_HINT_CTES
|
||||
from .execution_outputs import ExecutionOutputOperations
|
||||
from .history_reads import HistoryReadOperations
|
||||
from .health import HealthOperations, iter_consistency_issues
|
||||
|
||||
# Import logging system
|
||||
try:
|
||||
from ..utils.logging_config import get_logger
|
||||
from ..utils.hashing import generate_prompt_hash
|
||||
from ..utils.json_strict import dumps_strict
|
||||
except ImportError:
|
||||
import sys
|
||||
|
||||
@@ -32,6 +37,7 @@ except ImportError:
|
||||
sys.path.insert(0, current_dir)
|
||||
from utils.logging_config import get_logger
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.json_strict import dumps_strict
|
||||
|
||||
|
||||
# Subquery to fetch tags from junction table, embedded in SELECT statements
|
||||
@@ -341,7 +347,10 @@ class PromptDatabase(
|
||||
HistoryReadOperations,
|
||||
ImageFavoriteOperations,
|
||||
AnnotationOperations,
|
||||
ReviewOperations,
|
||||
ReviewDecisionOperations,
|
||||
SavedViewOperations,
|
||||
HealthOperations,
|
||||
):
|
||||
"""Database operations class for managing prompts."""
|
||||
|
||||
@@ -1695,9 +1704,9 @@ class PromptDatabase(
|
||||
width,
|
||||
height,
|
||||
image_format,
|
||||
json.dumps(metadata.get("workflow", {})),
|
||||
json.dumps(metadata.get("prompt", {})),
|
||||
json.dumps(metadata.get("parameters", {})),
|
||||
dumps_strict(metadata.get("workflow", {})),
|
||||
dumps_strict(metadata.get("prompt", {})),
|
||||
dumps_strict(metadata.get("parameters", {})),
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
@@ -1902,13 +1911,13 @@ class PromptDatabase(
|
||||
obj: The object to clean (dict, list, or scalar)
|
||||
|
||||
Returns:
|
||||
Cleaned object with NaN values replaced by None
|
||||
Cleaned object with NaN and Infinity values replaced by None
|
||||
"""
|
||||
if isinstance(obj, dict):
|
||||
return {key: self._clean_nan_values(value) for key, value in obj.items()}
|
||||
elif isinstance(obj, list):
|
||||
return [self._clean_nan_values(item) for item in obj]
|
||||
elif isinstance(obj, float) and str(obj) == "nan":
|
||||
elif isinstance(obj, float) and not math.isfinite(obj):
|
||||
return None
|
||||
else:
|
||||
return obj
|
||||
@@ -2570,33 +2579,13 @@ class PromptDatabase(
|
||||
return cursor.rowcount
|
||||
|
||||
def check_consistency(self) -> List[str]:
|
||||
"""Run consistency checks and return a list of issue descriptions."""
|
||||
issues: List[str] = []
|
||||
with self.model.get_connection() as conn:
|
||||
# Check for orphaned prompt_tags entries
|
||||
cursor = conn.execute("""
|
||||
SELECT pt.prompt_id, pt.tag_id FROM prompt_tags pt
|
||||
LEFT JOIN prompts p ON pt.prompt_id = p.id
|
||||
WHERE p.id IS NULL
|
||||
""")
|
||||
for ref in cursor.fetchall():
|
||||
issues.append(
|
||||
"prompt_tags entry references non-existent prompt "
|
||||
f"{ref['prompt_id']}"
|
||||
)
|
||||
"""Run consistency checks and return a list of issue descriptions.
|
||||
|
||||
# Check for orphaned image entries
|
||||
cursor = conn.execute("""
|
||||
SELECT gi.id, gi.prompt_id FROM generated_images gi
|
||||
LEFT JOIN prompts p ON gi.prompt_id = p.id
|
||||
WHERE p.id IS NULL
|
||||
""")
|
||||
for ref in cursor.fetchall():
|
||||
issues.append(
|
||||
f"Image {ref['id']} references non-existent prompt "
|
||||
f"{ref['prompt_id']}"
|
||||
)
|
||||
return issues
|
||||
The same queries back the System health check's Consistency row
|
||||
(database/health.py).
|
||||
"""
|
||||
with self.model.get_connection() as conn:
|
||||
return list(iter_consistency_issues(conn))
|
||||
|
||||
def _image_row_to_dict(self, row: sqlite3.Row) -> Dict[str, Any]:
|
||||
"""
|
||||
|
||||
+136
-108
@@ -1,6 +1,5 @@
|
||||
"""Atomic metadata enrichment for a single known generation context."""
|
||||
|
||||
import json
|
||||
import os
|
||||
from typing import Any, Dict
|
||||
|
||||
@@ -9,8 +8,12 @@ from .operations import _coerce_file_info
|
||||
|
||||
try:
|
||||
from ..utils.hashing import generate_prompt_hash
|
||||
from ..utils.json_strict import dumps_strict
|
||||
from ..utils.prompt_pair_resolver import RESOLVER_VERSION
|
||||
except ImportError:
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.json_strict import dumps_strict
|
||||
from utils.prompt_pair_resolver import RESOLVER_VERSION
|
||||
|
||||
|
||||
def ingest_resolved_pair(
|
||||
@@ -19,118 +22,143 @@ def ingest_resolved_pair(
|
||||
state = resolution.get("state", "unresolved")
|
||||
if state != "resolved":
|
||||
return {"status": state, "pairs_added": 0, "links_added": 0}
|
||||
with db._pair_transaction() as conn:
|
||||
return _ingest_resolved_pair_in(conn, db, image_path, metadata, resolution)
|
||||
|
||||
|
||||
def _conflict() -> Dict[str, Any]:
|
||||
return {"status": "conflict", "pairs_added": 0, "links_added": 0}
|
||||
|
||||
|
||||
def _ingest_resolved_pair_in(
|
||||
conn: Any,
|
||||
db: Any,
|
||||
image_path: str,
|
||||
metadata: Dict[str, Any],
|
||||
resolution: Dict[str, Any],
|
||||
*,
|
||||
provenance: Dict[str, Any] | None = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""``ingest_resolved_pair`` inside the caller's open ``_pair_transaction``.
|
||||
|
||||
The compatible-only rules are the public ingester's: an image with more
|
||||
than one exact row, another positive, or a prior pair with another
|
||||
negative is a ``conflict`` and nothing is written. The caller owns the
|
||||
transaction, so its own later failure rolls the link back too.
|
||||
``provenance`` replaces the link's default resolver provenance (Needs
|
||||
review records the user's selection there).
|
||||
"""
|
||||
positive, negative = resolution["positive"], resolution["negative"]
|
||||
positive_text = positive.get("storage_text", positive["text"])
|
||||
negative_text = negative.get("storage_text", negative["text"])
|
||||
normalized = normalize_image_path(image_path)
|
||||
provenance = {"resolver_version": 1, "positive": positive, "negative": negative}
|
||||
encoded = json.dumps(
|
||||
provenance, sort_keys=True, separators=(",", ":"), allow_nan=False
|
||||
)
|
||||
with db._pair_transaction() as conn:
|
||||
images = conn.execute(
|
||||
"SELECT gi.*,p.hash AS positive_hash FROM generated_images gi "
|
||||
"JOIN prompts p ON p.id=gi.prompt_id "
|
||||
"WHERE gi.file_path=? OR gi.image_path=?",
|
||||
(normalized, image_path),
|
||||
if provenance is None:
|
||||
provenance = {
|
||||
"resolver_version": RESOLVER_VERSION,
|
||||
"positive": positive,
|
||||
"negative": negative,
|
||||
}
|
||||
encoded = dumps_strict(provenance, sort_keys=True, separators=(",", ":"))
|
||||
images = conn.execute(
|
||||
"SELECT gi.*,p.hash AS positive_hash FROM generated_images gi "
|
||||
"JOIN prompts p ON p.id=gi.prompt_id "
|
||||
"WHERE gi.file_path=? OR gi.image_path=?",
|
||||
(normalized, image_path),
|
||||
).fetchall()
|
||||
if len(images) > 1 or (
|
||||
images and images[0]["positive_hash"] != generate_prompt_hash(positive_text)
|
||||
):
|
||||
return _conflict()
|
||||
existing = images[0] if images else None
|
||||
if existing:
|
||||
prior = conn.execute(
|
||||
"SELECT n.hash AS negative_hash FROM image_prompt_pairs ip "
|
||||
"JOIN prompt_pairs pp ON pp.id=ip.pair_id "
|
||||
"LEFT JOIN prompts n ON n.id=pp.negative_prompt_id WHERE ip.image_id=?",
|
||||
(existing["id"],),
|
||||
).fetchall()
|
||||
if len(images) > 1 or (
|
||||
images and images[0]["positive_hash"] != generate_prompt_hash(positive_text)
|
||||
):
|
||||
return {"status": "conflict", "pairs_added": 0, "links_added": 0}
|
||||
existing = images[0] if images else None
|
||||
if existing:
|
||||
prior = conn.execute(
|
||||
"SELECT n.hash AS negative_hash FROM image_prompt_pairs ip "
|
||||
"JOIN prompt_pairs pp ON pp.id=ip.pair_id "
|
||||
"LEFT JOIN prompts n ON n.id=pp.negative_prompt_id WHERE ip.image_id=?",
|
||||
(existing["id"],),
|
||||
).fetchall()
|
||||
expected = (
|
||||
generate_prompt_hash(negative_text) if negative_text.strip() else None
|
||||
)
|
||||
if any(row["negative_hash"] != expected for row in prior):
|
||||
return {"status": "conflict", "pairs_added": 0, "links_added": 0}
|
||||
prior_pair = conn.execute(
|
||||
"SELECT pp.id FROM prompt_pairs pp "
|
||||
"JOIN prompts p ON p.id=pp.positive_prompt_id "
|
||||
"LEFT JOIN prompts n ON n.id=pp.negative_prompt_id "
|
||||
"WHERE p.hash=? AND n.hash IS ?",
|
||||
(
|
||||
generate_prompt_hash(positive_text),
|
||||
generate_prompt_hash(negative_text) if negative_text.strip() else None,
|
||||
),
|
||||
).fetchone()
|
||||
pair_id = db._save_pair(conn, positive_text, negative_text)
|
||||
pair = conn.execute(
|
||||
"SELECT * FROM prompt_pairs WHERE id=?", (pair_id,)
|
||||
).fetchone()
|
||||
pairs_added = int(prior_pair is None)
|
||||
file_size, width, height, image_format = _coerce_file_info(metadata)
|
||||
snapshots = {
|
||||
column: json.dumps(metadata[key], sort_keys=True, allow_nan=False)
|
||||
for column, key in (
|
||||
("workflow_data", "workflow"),
|
||||
("prompt_metadata", "prompt"),
|
||||
("parameters", "parameters"),
|
||||
)
|
||||
if key in metadata
|
||||
}
|
||||
info = {
|
||||
"file_size": file_size,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"format": image_format,
|
||||
}
|
||||
snapshots.update(
|
||||
{key: value for key, value in info.items() if value is not None}
|
||||
expected = (
|
||||
generate_prompt_hash(negative_text) if negative_text.strip() else None
|
||||
)
|
||||
changed = False
|
||||
if existing:
|
||||
image_id = existing["id"]
|
||||
updates = {
|
||||
key: value for key, value in snapshots.items() if existing[key] != value
|
||||
}
|
||||
if updates:
|
||||
conn.execute(
|
||||
"UPDATE generated_images SET "
|
||||
+ ",".join(f"{key}=?" for key in updates)
|
||||
+ " WHERE id=?",
|
||||
[*updates.values(), image_id],
|
||||
)
|
||||
changed = True
|
||||
else:
|
||||
columns = ["prompt_id", "image_path", "filename", "file_path", *snapshots]
|
||||
values = [
|
||||
pair["positive_prompt_id"],
|
||||
image_path,
|
||||
os.path.basename(image_path),
|
||||
normalized,
|
||||
*snapshots.values(),
|
||||
]
|
||||
image_id = conn.execute(
|
||||
"INSERT INTO generated_images("
|
||||
+ ",".join(columns)
|
||||
+ ") VALUES("
|
||||
+ ",".join("?" for _ in columns)
|
||||
+ ")",
|
||||
values,
|
||||
).lastrowid
|
||||
linked = conn.execute(
|
||||
"SELECT 1 FROM image_prompt_pairs WHERE pair_id=? "
|
||||
"AND image_id=? AND provenance=?",
|
||||
(pair_id, image_id, encoded),
|
||||
).fetchone()
|
||||
db._link_image_to_pair(conn, pair_id, image_id, provenance)
|
||||
status = (
|
||||
"added"
|
||||
if existing is None
|
||||
else "enriched" if changed or not linked else "unchanged"
|
||||
if any(row["negative_hash"] != expected for row in prior):
|
||||
return _conflict()
|
||||
prior_pair = conn.execute(
|
||||
"SELECT pp.id FROM prompt_pairs pp "
|
||||
"JOIN prompts p ON p.id=pp.positive_prompt_id "
|
||||
"LEFT JOIN prompts n ON n.id=pp.negative_prompt_id "
|
||||
"WHERE p.hash=? AND n.hash IS ?",
|
||||
(
|
||||
generate_prompt_hash(positive_text),
|
||||
generate_prompt_hash(negative_text) if negative_text.strip() else None,
|
||||
),
|
||||
).fetchone()
|
||||
pair_id = db._save_pair(conn, positive_text, negative_text)
|
||||
pair = conn.execute("SELECT * FROM prompt_pairs WHERE id=?", (pair_id,)).fetchone()
|
||||
pairs_added = int(prior_pair is None)
|
||||
file_size, width, height, image_format = _coerce_file_info(metadata)
|
||||
# ComfyUI's NaN/Infinity are stored as null: the snapshot is strict JSON.
|
||||
snapshots = {
|
||||
column: dumps_strict(metadata[key], sort_keys=True)
|
||||
for column, key in (
|
||||
("workflow_data", "workflow"),
|
||||
("prompt_metadata", "prompt"),
|
||||
("parameters", "parameters"),
|
||||
)
|
||||
return {
|
||||
"status": status,
|
||||
"pairs_added": pairs_added,
|
||||
"links_added": int(not linked),
|
||||
"pair_id": pair_id,
|
||||
"image_id": image_id,
|
||||
if key in metadata
|
||||
}
|
||||
info = {
|
||||
"file_size": file_size,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"format": image_format,
|
||||
}
|
||||
snapshots.update({key: value for key, value in info.items() if value is not None})
|
||||
changed = False
|
||||
if existing:
|
||||
image_id = existing["id"]
|
||||
updates = {
|
||||
key: value for key, value in snapshots.items() if existing[key] != value
|
||||
}
|
||||
if updates:
|
||||
conn.execute(
|
||||
"UPDATE generated_images SET "
|
||||
+ ",".join(f"{key}=?" for key in updates)
|
||||
+ " WHERE id=?",
|
||||
[*updates.values(), image_id],
|
||||
)
|
||||
changed = True
|
||||
else:
|
||||
columns = ["prompt_id", "image_path", "filename", "file_path", *snapshots]
|
||||
values = [
|
||||
pair["positive_prompt_id"],
|
||||
image_path,
|
||||
os.path.basename(image_path),
|
||||
normalized,
|
||||
*snapshots.values(),
|
||||
]
|
||||
image_id = conn.execute(
|
||||
"INSERT INTO generated_images("
|
||||
+ ",".join(columns)
|
||||
+ ") VALUES("
|
||||
+ ",".join("?" for _ in columns)
|
||||
+ ")",
|
||||
values,
|
||||
).lastrowid
|
||||
linked = conn.execute(
|
||||
"SELECT 1 FROM image_prompt_pairs WHERE pair_id=? "
|
||||
"AND image_id=? AND provenance=?",
|
||||
(pair_id, image_id, encoded),
|
||||
).fetchone()
|
||||
db._link_image_to_pair(conn, pair_id, image_id, provenance)
|
||||
status = (
|
||||
"added"
|
||||
if existing is None
|
||||
else "enriched" if changed or not linked else "unchanged"
|
||||
)
|
||||
return {
|
||||
"status": status,
|
||||
"pairs_added": pairs_added,
|
||||
"links_added": int(not linked),
|
||||
"pair_id": pair_id,
|
||||
"image_id": image_id,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,541 @@
|
||||
"""Needs review items (#249): persist and read capture-bound ambiguous evidence.
|
||||
|
||||
An item is one ambiguous resolver result for one annotation byte record (the
|
||||
whole-file SHA-256 of one physical file at one locator), keyed by
|
||||
``(byte_id, resolver_version, evidence_hash)``. Recording is idempotent: the
|
||||
same evidence for the same bytes is a duplicate and never touches an existing
|
||||
item, its state, revision or decisions. Changed bytes are a new byte record
|
||||
and therefore a new pending item; the old one stays.
|
||||
|
||||
Callers pass verified facts (the capture's digest, size and ``mtime_ns``);
|
||||
nothing here reads the filesystem. Writes run in one ``_pair_transaction``
|
||||
fenced by the storage epoch the capture was taken under.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
from typing import Iterator
|
||||
|
||||
try:
|
||||
from ..utils.hashing import generate_prompt_hash
|
||||
except ImportError:
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
from .annotation_schema import (
|
||||
BYTES_TABLE,
|
||||
FILES_TABLE,
|
||||
METADATA_TABLE as ANNOTATION_METADATA_TABLE,
|
||||
)
|
||||
from .annotations import _new_id, _validate_locator, _validate_target
|
||||
from .models import normalize_image_path, utc_now_iso
|
||||
from .review_schema import (
|
||||
DECISIONS_TABLE,
|
||||
ITEMS_TABLE,
|
||||
MAX_CANDIDATES,
|
||||
MAX_SPELLED_PATH_BYTES,
|
||||
METADATA_TABLE,
|
||||
canonical_evidence,
|
||||
evidence_digest,
|
||||
evidence_offers_choice,
|
||||
)
|
||||
|
||||
# Only this module's table constants are interpolated into the SQL below.
|
||||
_INSERT_ITEM = (
|
||||
f"INSERT OR IGNORE INTO {ITEMS_TABLE}(id,byte_id,spelled_path," # nosec B608
|
||||
"resolver_version,evidence_hash,evidence,candidate_count,limited,state,"
|
||||
"revision,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?,'pending',0,?,?)"
|
||||
)
|
||||
_BUMP_REVIEW_INDEX = (
|
||||
f"UPDATE {METADATA_TABLE} SET " # nosec B608
|
||||
"review_index_revision=review_index_revision+1 WHERE singleton_id=1"
|
||||
)
|
||||
# A later spelling of the same locator (a case-only rename on a case-folding
|
||||
# filesystem) replaces the recorded one, so targets keep formatting.
|
||||
_REFRESH_SPELLING = (
|
||||
f"UPDATE {ITEMS_TABLE} SET spelled_path=? " # nosec B608
|
||||
f"WHERE spelled_path<>? AND byte_id IN (SELECT b.id FROM {BYTES_TABLE} b "
|
||||
f"JOIN {FILES_TABLE} f ON f.id=b.file_id WHERE f.locator=?)"
|
||||
)
|
||||
_REVIEW_INDEX = (
|
||||
f"SELECT review_index_revision FROM {METADATA_TABLE} " # nosec B608
|
||||
"WHERE singleton_id=1"
|
||||
)
|
||||
_BUMP_IMAGE_INDEX = (
|
||||
f"UPDATE {ANNOTATION_METADATA_TABLE} SET " # nosec B608
|
||||
"image_index_revision=image_index_revision+1 WHERE singleton_id=1"
|
||||
)
|
||||
_ITEMS_OF_LOCATOR = (
|
||||
"SELECT i.id,i.byte_id,i.resolver_version,i.evidence_hash," # nosec B608
|
||||
"i.evidence,i.candidate_count,i.limited,i.state,i.revision,i.created_at,"
|
||||
f"i.updated_at,b.sha256 FROM {FILES_TABLE} f "
|
||||
f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
|
||||
f"JOIN {ITEMS_TABLE} i ON i.byte_id=b.id "
|
||||
"WHERE f.locator=? ORDER BY i.created_at,i.id"
|
||||
)
|
||||
_QUEUED = (
|
||||
f"SELECT 1 FROM {FILES_TABLE} f " # nosec B608
|
||||
f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
|
||||
f"JOIN {ITEMS_TABLE} i ON i.byte_id=b.id "
|
||||
"WHERE f.locator=? AND b.size=? AND b.mtime_ns=? AND i.resolver_version=? "
|
||||
"LIMIT 1"
|
||||
)
|
||||
_PENDING_AT = (
|
||||
f"SELECT 1 FROM {FILES_TABLE} f " # nosec B608
|
||||
f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
|
||||
f"JOIN {ITEMS_TABLE} i ON i.byte_id=b.id "
|
||||
"WHERE f.locator=? AND b.size=? AND b.mtime_ns=? AND i.resolver_version=? "
|
||||
"AND i.state='pending' LIMIT 1"
|
||||
)
|
||||
# Pending items of one file: the candidates a link settles.
|
||||
_PENDING_OF_LOCATOR = (
|
||||
f"SELECT i.id FROM {FILES_TABLE} f " # nosec B608
|
||||
f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
|
||||
f"JOIN {ITEMS_TABLE} i ON i.byte_id=b.id "
|
||||
"WHERE f.locator=? AND i.state='pending'"
|
||||
)
|
||||
# Pending items of one file with their evidence: an untraced file's settle.
|
||||
_PENDING_EVIDENCE_OF_LOCATOR = (
|
||||
f"SELECT i.id,i.evidence FROM {FILES_TABLE} f " # nosec B608
|
||||
f"JOIN {BYTES_TABLE} b ON b.file_id=f.id "
|
||||
f"JOIN {ITEMS_TABLE} i ON i.byte_id=b.id "
|
||||
"WHERE f.locator=? AND i.state='pending'"
|
||||
)
|
||||
# Only an undecided item goes: a decision row (RESTRICT) is never deleted.
|
||||
_SETTLE_ITEM = (
|
||||
f"DELETE FROM {ITEMS_TABLE} WHERE id=? AND state='pending' " # nosec B608
|
||||
f"AND revision=0 AND NOT EXISTS (SELECT 1 FROM {DECISIONS_TABLE} d "
|
||||
f"WHERE d.item_id={ITEMS_TABLE}.id)"
|
||||
)
|
||||
_EPOCH = "SELECT storage_epoch FROM execution_history_metadata WHERE singleton_id=1"
|
||||
_PENDING_COUNT = (
|
||||
f"SELECT COUNT(*) FROM {ITEMS_TABLE} WHERE state='pending'" # nosec B608
|
||||
)
|
||||
_QUEUE_COLUMNS = (
|
||||
"id",
|
||||
"state",
|
||||
"revision",
|
||||
"created_at",
|
||||
"candidate_count",
|
||||
"limited",
|
||||
"spelled_path",
|
||||
"locator",
|
||||
)
|
||||
_QUEUE_SELECT = (
|
||||
"SELECT i.id,i.state,i.revision,i.created_at,i.candidate_count," # nosec B608
|
||||
f"i.limited,i.spelled_path,f.locator FROM {ITEMS_TABLE} i "
|
||||
f"JOIN {BYTES_TABLE} b ON b.id=i.byte_id "
|
||||
f"JOIN {FILES_TABLE} f ON f.id=b.file_id "
|
||||
)
|
||||
_QUEUE_FIRST = _QUEUE_SELECT + (
|
||||
"WHERE i.state='pending' ORDER BY i.created_at,i.id LIMIT ?"
|
||||
)
|
||||
_QUEUE_AFTER = _QUEUE_SELECT + (
|
||||
"WHERE i.state='pending' AND (i.created_at,i.id)>(?,?) "
|
||||
"ORDER BY i.created_at,i.id LIMIT ?"
|
||||
)
|
||||
_DETAIL_COLUMNS = _QUEUE_COLUMNS + (
|
||||
"byte_id",
|
||||
"resolver_version",
|
||||
"evidence_hash",
|
||||
"evidence",
|
||||
"updated_at",
|
||||
"sha256",
|
||||
)
|
||||
_DETAIL = (
|
||||
"SELECT i.id,i.state,i.revision,i.created_at,i.candidate_count," # nosec B608
|
||||
"i.limited,i.spelled_path,f.locator,i.byte_id,i.resolver_version,"
|
||||
"i.evidence_hash,i.evidence,i.updated_at,b.sha256 "
|
||||
f"FROM {ITEMS_TABLE} i JOIN {BYTES_TABLE} b ON b.id=i.byte_id "
|
||||
f"JOIN {FILES_TABLE} f ON f.id=b.file_id WHERE i.id=?"
|
||||
)
|
||||
# The legacy rows and pair links the ingester's compatibility rules read.
|
||||
_DEPENDENT_IMAGES = (
|
||||
"SELECT gi.id,p.hash,gi.image_path FROM generated_images gi "
|
||||
"JOIN prompts p ON p.id=gi.prompt_id "
|
||||
"WHERE gi.file_path=? OR gi.image_path=? ORDER BY gi.id"
|
||||
)
|
||||
_DEPENDENT_LINKS = (
|
||||
"SELECT DISTINCT ip.pair_id,n.hash FROM image_prompt_pairs ip "
|
||||
"JOIN prompt_pairs pp ON pp.id=ip.pair_id "
|
||||
"LEFT JOIN prompts n ON n.id=pp.negative_prompt_id "
|
||||
"WHERE ip.image_id=? ORDER BY ip.pair_id"
|
||||
)
|
||||
# Rows whose spelling ends with the file's name: the candidates for another
|
||||
# spelling of the same physical file (the link summaries' filename fallback).
|
||||
_SAME_NAME_ROWS = (
|
||||
"SELECT id,image_path FROM generated_images "
|
||||
"WHERE image_path LIKE ? ESCAPE '\\' ORDER BY id"
|
||||
)
|
||||
_TITLES = (
|
||||
"SELECT p.hash,s.title FROM prompts p "
|
||||
"JOIN prompt_library_state s ON s.prompt_id=p.id "
|
||||
"WHERE p.hash IN ({marks}) AND s.title IS NOT NULL ORDER BY p.id"
|
||||
)
|
||||
_ITEM_ID = re.compile(r"[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}")
|
||||
_ITEM_COLUMNS = (
|
||||
"id",
|
||||
"byte_id",
|
||||
"resolver_version",
|
||||
"evidence_hash",
|
||||
"evidence",
|
||||
"candidate_count",
|
||||
"limited",
|
||||
"state",
|
||||
"revision",
|
||||
"created_at",
|
||||
"updated_at",
|
||||
"sha256",
|
||||
)
|
||||
|
||||
|
||||
def _validate_spelling(spelled_path: object, locator: str) -> None:
|
||||
if (
|
||||
not isinstance(spelled_path, str)
|
||||
or not spelled_path
|
||||
or len(spelled_path.encode("utf-8", "surrogatepass")) > MAX_SPELLED_PATH_BYTES
|
||||
or os.path.normcase(spelled_path) != locator
|
||||
):
|
||||
raise ValueError("spelled_path must be the file's spelling of its locator")
|
||||
|
||||
|
||||
def dependent_images(conn: sqlite3.Connection, spellings) -> list:
|
||||
"""``(id, image_path, positive_hash)`` of every legacy row of the file.
|
||||
|
||||
``spellings`` are the paths the file is linked under (the Apply path and
|
||||
every alias of the same physical file); each is matched as the ingester
|
||||
matches one path. Rows are distinct and ordered by id.
|
||||
"""
|
||||
found = {}
|
||||
for spelled in dict.fromkeys(spellings):
|
||||
for row in conn.execute(
|
||||
_DEPENDENT_IMAGES, (normalize_image_path(spelled), spelled)
|
||||
):
|
||||
found[row[0]] = (row[0], row[2], row[1])
|
||||
return [found[key] for key in sorted(found)]
|
||||
|
||||
|
||||
def same_name_rows(conn: sqlite3.Connection, name: str) -> list:
|
||||
"""``(id, image_path)`` of rows whose spelling may end with ``name``."""
|
||||
# Imported here: operations composes this module's mixin.
|
||||
from .operations import escape_like
|
||||
|
||||
return [
|
||||
(row[0], row[1])
|
||||
for row in conn.execute(_SAME_NAME_ROWS, (f"%{escape_like(name)}",))
|
||||
]
|
||||
|
||||
|
||||
def dependency_fingerprint(
|
||||
conn: sqlite3.Connection, image_path: str, aliases=()
|
||||
) -> str:
|
||||
"""SHA-256 of what Apply's compatibility depends on for ``image_path``.
|
||||
|
||||
The exact legacy rows of the file under ``image_path`` and every alias
|
||||
spelling of it, with their positive hashes, and each row's pair links
|
||||
with the negative hash, read as the ingester reads them. Unrelated prompt
|
||||
changes (tags, titles) do not move it.
|
||||
"""
|
||||
rows = []
|
||||
for image_id, _spelled, positive_hash in dependent_images(
|
||||
conn, (image_path, *aliases)
|
||||
):
|
||||
links = conn.execute(_DEPENDENT_LINKS, (image_id,)).fetchall()
|
||||
rows.append([image_id, positive_hash, [list(link) for link in links]])
|
||||
text = canonical_evidence({"v": 1, "images": rows})
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class ReviewOperations:
|
||||
"""PromptDatabase mixin: insert and read Needs review items."""
|
||||
|
||||
def record_review_item(
|
||||
self,
|
||||
*,
|
||||
locator: str,
|
||||
spelled_path: str,
|
||||
sha256: str,
|
||||
size: int,
|
||||
mtime_ns: int,
|
||||
epoch: str,
|
||||
resolver_version: int,
|
||||
evidence: dict,
|
||||
candidate_count: int,
|
||||
limited: bool,
|
||||
) -> bool:
|
||||
"""Store one ambiguous result for these exact bytes; True when new.
|
||||
|
||||
The byte record is created (or its verified fingerprint refreshed)
|
||||
like an observe. ``spelled_path`` is the file's own spelling (it must
|
||||
fold to ``locator``); targets are formatted from it. Raises
|
||||
``LibraryCommandError("storage-changed")`` when the storage epoch is
|
||||
no longer ``epoch`` (a restore since the capture) and ``ValueError``
|
||||
for invalid facts or evidence.
|
||||
"""
|
||||
_validate_target(locator, sha256, size, mtime_ns)
|
||||
_validate_spelling(spelled_path, locator)
|
||||
if type(resolver_version) is not int or resolver_version < 1:
|
||||
raise ValueError("resolver_version must be a positive integer")
|
||||
if (
|
||||
type(candidate_count) is not int
|
||||
or not 0 <= candidate_count <= MAX_CANDIDATES
|
||||
):
|
||||
raise ValueError(f"candidate_count must be between 0 and {MAX_CANDIDATES}")
|
||||
if not isinstance(limited, bool) or (limited and candidate_count):
|
||||
raise ValueError("A limited item carries no candidates")
|
||||
text = canonical_evidence(evidence)
|
||||
evidence_hash = evidence_digest(text)
|
||||
with self._pair_transaction() as conn:
|
||||
self._library_epoch(conn, epoch)
|
||||
now = utc_now_iso()
|
||||
file_id = self._annotation_owner(conn, locator, now)
|
||||
byte_id, _revision, created, moved = self._annotation_byte(
|
||||
conn, file_id, sha256, size, mtime_ns, now
|
||||
)
|
||||
if created or moved:
|
||||
conn.execute(_BUMP_IMAGE_INDEX)
|
||||
cursor = conn.execute(
|
||||
_INSERT_ITEM,
|
||||
(
|
||||
_new_id(),
|
||||
byte_id,
|
||||
spelled_path,
|
||||
resolver_version,
|
||||
evidence_hash,
|
||||
text,
|
||||
candidate_count,
|
||||
int(limited),
|
||||
now,
|
||||
now,
|
||||
),
|
||||
)
|
||||
inserted = cursor.rowcount > 0
|
||||
refreshed = _refresh_spelling(conn, locator, spelled_path)
|
||||
if inserted or refreshed:
|
||||
conn.execute(_BUMP_REVIEW_INDEX)
|
||||
return inserted
|
||||
|
||||
def refresh_review_spelling(self, locator: str, spelled_path: str) -> bool:
|
||||
"""Record ``spelled_path`` as the spelling of every item at ``locator``.
|
||||
|
||||
The rescan's skip calls it for a file it does not copy again: a
|
||||
case-only rename keeps size and ``mtime_ns``, and a target formatted
|
||||
from the old spelling would be refused. True when a spelling changed
|
||||
(the review index revision moves; item revisions do not, so an open
|
||||
decision stays valid).
|
||||
"""
|
||||
_validate_locator(locator)
|
||||
_validate_spelling(spelled_path, locator)
|
||||
with self._pair_transaction() as conn:
|
||||
refreshed = _refresh_spelling(conn, locator, spelled_path)
|
||||
if refreshed:
|
||||
conn.execute(_BUMP_REVIEW_INDEX)
|
||||
return refreshed
|
||||
|
||||
def review_index_revision(self) -> int:
|
||||
"""The queue's revision (moved by every queue change)."""
|
||||
row = self.model.get_connection().execute(_REVIEW_INDEX).fetchone()
|
||||
if row is None:
|
||||
raise sqlite3.DatabaseError("Missing review queue metadata")
|
||||
return row[0]
|
||||
|
||||
def review_items_for(self, locator: str) -> list:
|
||||
"""Every item of the file at ``locator``, oldest first, evidence parsed."""
|
||||
_validate_locator(locator)
|
||||
return list(self._iter_review_items(locator))
|
||||
|
||||
def _iter_review_items(self, locator: str) -> Iterator[dict]:
|
||||
conn = self.model.get_connection()
|
||||
for row in conn.execute(_ITEMS_OF_LOCATOR, (locator,)):
|
||||
item = dict(zip(_ITEM_COLUMNS, tuple(row)))
|
||||
item["evidence"] = json.loads(item["evidence"])
|
||||
item["limited"] = bool(item["limited"])
|
||||
yield item
|
||||
|
||||
# -- queue reads (#249 E3) -------------------------------------------------
|
||||
|
||||
def review_fingerprint_queued(
|
||||
self, locator: str, size: int, mtime_ns: int, resolver_version: int
|
||||
) -> bool:
|
||||
"""True when an item cites a byte record verified at this exact stat.
|
||||
|
||||
A rescan uses it to skip the copy and hash of a file it already
|
||||
queued: the same ``(size, mtime_ns)`` the bytes were verified with.
|
||||
"""
|
||||
row = (
|
||||
self.model.get_connection()
|
||||
.execute(_QUEUED, (locator, size, mtime_ns, resolver_version))
|
||||
.fetchone()
|
||||
)
|
||||
return row is not None
|
||||
|
||||
def review_pending_at(
|
||||
self, locator: str, size: int, mtime_ns: int, resolver_version: int
|
||||
) -> bool:
|
||||
"""True when a *pending* item cites bytes verified at this exact stat.
|
||||
|
||||
``review_fingerprint_queued`` also matches decided items; a file whose
|
||||
only item is decided is not in the queue.
|
||||
"""
|
||||
row = (
|
||||
self.model.get_connection()
|
||||
.execute(_PENDING_AT, (locator, size, mtime_ns, resolver_version))
|
||||
.fetchone()
|
||||
)
|
||||
return row is not None
|
||||
|
||||
def review_file_linked(self, spellings) -> bool:
|
||||
"""True when a saved row links the file under any of ``spellings``."""
|
||||
with self._annotation_read() as conn:
|
||||
return bool(dependent_images(conn, spellings))
|
||||
|
||||
def settle_linked_review_items(self, files) -> int:
|
||||
"""Remove the pending, undecided items of files that are now linked.
|
||||
|
||||
``files`` holds ``(locator, spellings)`` per file. An item leaves the
|
||||
queue only when it is still ``pending`` at revision 0 with no decision
|
||||
row and a saved row links its file under one of ``spellings``; a
|
||||
decided item is never touched. Files without a pending item cost one
|
||||
indexed read and open no write transaction. Returns how many items
|
||||
were removed (the review index revision moves when any was).
|
||||
"""
|
||||
files = [(locator, tuple(spellings)) for locator, spellings in files]
|
||||
for locator, _spellings in files:
|
||||
_validate_locator(locator)
|
||||
conn = self.model.get_connection()
|
||||
candidates = [
|
||||
(locator, spellings)
|
||||
for locator, spellings in files
|
||||
if conn.execute(_PENDING_OF_LOCATOR, (locator,)).fetchone() is not None
|
||||
]
|
||||
if not candidates:
|
||||
return 0
|
||||
removed = 0
|
||||
with self._pair_transaction() as conn:
|
||||
for locator, spellings in candidates:
|
||||
if not dependent_images(conn, spellings):
|
||||
continue
|
||||
for (item_id,) in conn.execute(
|
||||
_PENDING_OF_LOCATOR, (locator,)
|
||||
).fetchall():
|
||||
removed += conn.execute(_SETTLE_ITEM, (item_id,)).rowcount
|
||||
if removed:
|
||||
conn.execute(_BUMP_REVIEW_INDEX)
|
||||
return removed
|
||||
|
||||
def settle_untraced_review_items(self, locators) -> int:
|
||||
"""Remove the pending, undecided items that offer nothing to choose.
|
||||
|
||||
``locators`` are files the caller found ambiguous without a choice
|
||||
(fewer than two candidates carry a prompt). Of their items, one leaves
|
||||
the queue only when its own stored evidence offers no choice either
|
||||
(``evidence_offers_choice``) and it is still ``pending`` at revision 0
|
||||
with no decision row, the same safe delete as
|
||||
``settle_linked_review_items``. Files without a pending item cost one
|
||||
indexed read and open no write transaction. Returns how many items
|
||||
were removed (the review index revision moves when any was).
|
||||
"""
|
||||
locators = list(locators)
|
||||
for locator in locators:
|
||||
_validate_locator(locator)
|
||||
conn = self.model.get_connection()
|
||||
pending = [
|
||||
locator
|
||||
for locator in locators
|
||||
if conn.execute(_PENDING_OF_LOCATOR, (locator,)).fetchone() is not None
|
||||
]
|
||||
if not pending:
|
||||
return 0
|
||||
removed = 0
|
||||
with self._pair_transaction() as conn:
|
||||
for locator in pending:
|
||||
rows = conn.execute(_PENDING_EVIDENCE_OF_LOCATOR, (locator,)).fetchall()
|
||||
for item_id, evidence in rows:
|
||||
if evidence_offers_choice(json.loads(evidence)):
|
||||
continue
|
||||
removed += conn.execute(_SETTLE_ITEM, (item_id,)).rowcount
|
||||
if removed:
|
||||
conn.execute(_BUMP_REVIEW_INDEX)
|
||||
return removed
|
||||
|
||||
def review_queue_page(self, *, limit: int, after: tuple | None = None) -> dict:
|
||||
"""One page of pending items, oldest first, from one snapshot.
|
||||
|
||||
``after`` is the ``(created_at, id)`` keyset of the last item already
|
||||
shown. Returns ``{epoch, revision, total, items, more}``; ``total`` is
|
||||
the exact pending count and ``items`` carry their ``spelled_path`` and
|
||||
``locator`` (server-side only).
|
||||
"""
|
||||
if type(limit) is not int or limit < 1:
|
||||
raise ValueError("limit must be a positive integer")
|
||||
with self._annotation_read() as conn:
|
||||
epoch = conn.execute(_EPOCH).fetchone()
|
||||
revision = conn.execute(_REVIEW_INDEX).fetchone()
|
||||
if epoch is None or revision is None:
|
||||
raise sqlite3.DatabaseError("Missing review queue metadata")
|
||||
total = conn.execute(_PENDING_COUNT).fetchone()[0]
|
||||
if after is None:
|
||||
rows = conn.execute(_QUEUE_FIRST, (limit + 1,)).fetchall()
|
||||
else:
|
||||
rows = conn.execute(_QUEUE_AFTER, (*after, limit + 1)).fetchall()
|
||||
items = [_queue_row(row, _QUEUE_COLUMNS) for row in rows[:limit]]
|
||||
return {
|
||||
"epoch": epoch[0],
|
||||
"revision": revision[0],
|
||||
"total": total,
|
||||
"items": items,
|
||||
"more": len(rows) > limit,
|
||||
}
|
||||
|
||||
def review_item(self, item_id: str) -> dict | None:
|
||||
"""One item with its parsed evidence and the pending total, or None."""
|
||||
if not isinstance(item_id, str) or _ITEM_ID.fullmatch(item_id) is None:
|
||||
return None
|
||||
with self._annotation_read() as conn:
|
||||
epoch = conn.execute(_EPOCH).fetchone()
|
||||
if epoch is None:
|
||||
raise sqlite3.DatabaseError("Missing execution history metadata")
|
||||
row = conn.execute(_DETAIL, (item_id,)).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
item = _queue_row(row, _DETAIL_COLUMNS)
|
||||
item["evidence"] = json.loads(item["evidence"])
|
||||
item["epoch"] = epoch[0]
|
||||
item["total"] = conn.execute(_PENDING_COUNT).fetchone()[0]
|
||||
return item
|
||||
|
||||
def review_dependency_fingerprint(self, image_path: str, aliases=()) -> str:
|
||||
with self._annotation_read() as conn:
|
||||
return dependency_fingerprint(conn, image_path, aliases)
|
||||
|
||||
def review_same_name_rows(self, name: str) -> list:
|
||||
"""``(id, image_path)`` of legacy rows that may spell a file ``name``."""
|
||||
with self._annotation_read() as conn:
|
||||
return same_name_rows(conn, name)
|
||||
|
||||
def stored_prompt_titles(self, texts) -> dict:
|
||||
"""Library titles of saved prompts by text (dedup hash), when set."""
|
||||
hashes = {generate_prompt_hash(text): text for text in texts if text.strip()}
|
||||
if not hashes:
|
||||
return {}
|
||||
marks = ",".join("?" * len(hashes))
|
||||
found: dict[str, tuple] = {}
|
||||
conn = self.model.get_connection()
|
||||
for prompt_hash, title in conn.execute(
|
||||
_TITLES.format(marks=marks), list(hashes)
|
||||
):
|
||||
text = hashes[prompt_hash]
|
||||
found[text] = found.get(text, ()) + (title,)
|
||||
return found
|
||||
|
||||
|
||||
def _refresh_spelling(conn: sqlite3.Connection, locator: str, spelled: str) -> bool:
|
||||
return conn.execute(_REFRESH_SPELLING, (spelled, spelled, locator)).rowcount > 0
|
||||
|
||||
|
||||
def _queue_row(row, columns) -> dict:
|
||||
item = dict(zip(columns, tuple(row)))
|
||||
item["limited"] = bool(item["limited"])
|
||||
return item
|
||||
@@ -0,0 +1,287 @@
|
||||
"""Needs review decisions (#249 E3): explicit, auditable and compatible-only.
|
||||
|
||||
A decision is one row of ``media_review_decisions`` for one item revision.
|
||||
``selected`` links the chosen candidate's pair to the file through the public
|
||||
ingester's compatible-only rules (``_ingest_resolved_pair_in``) in the same
|
||||
``_pair_transaction`` as the decision row, the item's new state and revision
|
||||
and the review index revision; any conflict or later failure rolls all of it
|
||||
back. ``unresolved`` records the decision and links nothing. No decision
|
||||
counts a run, touches usage or calls an execution writer.
|
||||
|
||||
Callers verify the file first (the capture must be the item's bytes and the
|
||||
current file must still hash to them); nothing here reads the filesystem.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sqlite3
|
||||
|
||||
from .annotation_schema import BYTES_TABLE
|
||||
from .annotations import _new_id
|
||||
from .library_commands import LibraryCommandError
|
||||
from .models import utc_now_iso
|
||||
from .review import (
|
||||
_BUMP_REVIEW_INDEX,
|
||||
_DETAIL,
|
||||
_DETAIL_COLUMNS,
|
||||
_PENDING_COUNT,
|
||||
_queue_row,
|
||||
dependency_fingerprint,
|
||||
dependent_images,
|
||||
same_name_rows,
|
||||
)
|
||||
from .review_schema import DECISIONS_TABLE, ITEMS_TABLE, canonical_evidence
|
||||
|
||||
DECISION_KIND = "user-selection"
|
||||
_INSERT_DECISION = (
|
||||
f"INSERT INTO {DECISIONS_TABLE}(id,item_id,decision_revision," # nosec B608
|
||||
"created_at,outcome,candidate_hash,evidence,pair_id,image_id) "
|
||||
"VALUES(?,?,?,?,?,?,?,?,?)"
|
||||
)
|
||||
_ADVANCE_ITEM = (
|
||||
f"UPDATE {ITEMS_TABLE} SET state=?,revision=revision+1," # nosec B608
|
||||
"updated_at=? WHERE id=? AND revision=?"
|
||||
)
|
||||
_ITEM_BYTES = (
|
||||
f"SELECT b.sha256 FROM {ITEMS_TABLE} i " # nosec B608
|
||||
f"JOIN {BYTES_TABLE} b ON b.id=i.byte_id WHERE i.id=?"
|
||||
)
|
||||
|
||||
|
||||
class ReviewCommandError(LibraryCommandError):
|
||||
"""A decision lost a race or would rewrite an association.
|
||||
|
||||
``code`` is ``review-conflict`` (the item was decided or changed since it
|
||||
was read), ``association-conflict`` (the choice is not compatible with
|
||||
the file's saved links, or those links changed), ``file-changed`` (the
|
||||
item is not about these bytes) or ``review-missing``. ``item`` is the
|
||||
current item read inside the transaction, ``total`` the pending count and
|
||||
``fingerprint`` the current dependency fingerprint, when known;
|
||||
``changed`` marks an association conflict caused by links that changed
|
||||
since the reviewer read them (rather than an incompatible choice).
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, code, epoch, *, item=None, total=None, fingerprint=None, changed=False
|
||||
):
|
||||
super().__init__(code, epoch)
|
||||
self.item = item
|
||||
self.total = total
|
||||
self.fingerprint = fingerprint
|
||||
self.changed = changed
|
||||
# The public view of ``item``, set by the API when it formats one.
|
||||
self.view = None
|
||||
|
||||
|
||||
class _Rollback(Exception):
|
||||
"""Raised inside the transaction so the context manager rolls back."""
|
||||
|
||||
def __init__(self, error: ReviewCommandError):
|
||||
super().__init__(error.code)
|
||||
self.error = error
|
||||
|
||||
|
||||
class ReviewDecisionOperations:
|
||||
"""PromptDatabase mixin: record one decision on one Needs review item."""
|
||||
|
||||
def decide_review_item(
|
||||
self,
|
||||
*,
|
||||
item_id: str,
|
||||
expected_epoch: str,
|
||||
expected_revision: int,
|
||||
sha256: str,
|
||||
outcome: str,
|
||||
candidate: dict | None = None,
|
||||
image_path: str | None = None,
|
||||
metadata: dict | None = None,
|
||||
expected_fingerprint: str | None = None,
|
||||
aliases: tuple = (),
|
||||
known_rows: frozenset | None = None,
|
||||
) -> dict:
|
||||
"""Record the decision; return ``{item, total, pair}``.
|
||||
|
||||
``selected`` needs the evidence ``candidate`` (with its ``hash``), the
|
||||
``image_path`` the scan and the monitor link under, the captured
|
||||
``metadata`` and the ``expected_fingerprint`` the reviewer saw.
|
||||
``aliases`` are the other spellings legacy rows of the same physical
|
||||
file use (a symlinked, junctioned or mapped root) and ``known_rows``
|
||||
the ids of every same-name row the caller classified; a same-name row
|
||||
it did not see is a changed association.
|
||||
Raises ``LibraryCommandError("storage-changed")`` after a restore and
|
||||
:class:`ReviewCommandError` for every refused decision, with zero
|
||||
writes either way.
|
||||
"""
|
||||
if outcome not in ("selected", "unresolved"):
|
||||
raise ValueError("outcome must be selected or unresolved")
|
||||
if outcome == "selected" and (
|
||||
not isinstance(candidate, dict) or not image_path
|
||||
):
|
||||
raise ValueError("A selected decision needs a candidate and a file")
|
||||
try:
|
||||
with self._pair_transaction() as conn:
|
||||
return self._decide_in(
|
||||
conn,
|
||||
item_id=item_id,
|
||||
expected_epoch=expected_epoch,
|
||||
expected_revision=expected_revision,
|
||||
sha256=sha256,
|
||||
outcome=outcome,
|
||||
candidate=candidate,
|
||||
image_path=image_path,
|
||||
metadata=metadata or {},
|
||||
expected_fingerprint=expected_fingerprint,
|
||||
aliases=tuple(aliases),
|
||||
known_rows=known_rows,
|
||||
)
|
||||
except _Rollback as rollback:
|
||||
raise rollback.error from None
|
||||
|
||||
def _decide_in(self, conn: sqlite3.Connection, **command) -> dict:
|
||||
epoch = self._library_epoch(conn, command["expected_epoch"])
|
||||
item_id = command["item_id"]
|
||||
current = _current_item(conn, item_id)
|
||||
if current is None:
|
||||
raise _Rollback(ReviewCommandError("review-missing", epoch))
|
||||
total = conn.execute(_PENDING_COUNT).fetchone()[0]
|
||||
if (
|
||||
current["state"] != "pending"
|
||||
or current["revision"] != command["expected_revision"]
|
||||
):
|
||||
raise _Rollback(
|
||||
ReviewCommandError("review-conflict", epoch, item=current, total=total)
|
||||
)
|
||||
if conn.execute(_ITEM_BYTES, (item_id,)).fetchone()[0] != command["sha256"]:
|
||||
raise _Rollback(ReviewCommandError("file-changed", epoch))
|
||||
decision_id = _new_id()
|
||||
pair = None
|
||||
candidate = command["candidate"] if command["outcome"] == "selected" else None
|
||||
if candidate is not None:
|
||||
pair = self._apply_candidate(
|
||||
conn, epoch, current, decision_id, total, **command
|
||||
)
|
||||
evidence = {
|
||||
"kind": DECISION_KIND,
|
||||
"decision_id": decision_id,
|
||||
"outcome": command["outcome"],
|
||||
"reviewed_digest": command["sha256"],
|
||||
"item_evidence_hash": current["evidence_hash"],
|
||||
"resolver_version": current["resolver_version"],
|
||||
"candidate": candidate,
|
||||
"dependency_fingerprint": command["expected_fingerprint"],
|
||||
"pair": pair,
|
||||
}
|
||||
now = utc_now_iso()
|
||||
self._insert_review_decision(
|
||||
conn,
|
||||
(
|
||||
decision_id,
|
||||
item_id,
|
||||
current["revision"] + 1,
|
||||
now,
|
||||
command["outcome"],
|
||||
candidate["hash"] if candidate is not None else None,
|
||||
canonical_evidence(evidence),
|
||||
pair["id"] if pair else None,
|
||||
pair["image_id"] if pair else None,
|
||||
),
|
||||
)
|
||||
conn.execute(
|
||||
_ADVANCE_ITEM, (command["outcome"], now, item_id, current["revision"])
|
||||
)
|
||||
conn.execute(_BUMP_REVIEW_INDEX)
|
||||
return {
|
||||
"item": _current_item(conn, item_id),
|
||||
"total": conn.execute(_PENDING_COUNT).fetchone()[0],
|
||||
"pair": pair,
|
||||
"decision_id": decision_id,
|
||||
}
|
||||
|
||||
def _apply_candidate(
|
||||
self, conn, epoch, current, decision_id, total, **command
|
||||
) -> dict:
|
||||
"""Link the candidate's pair, compatible-only; raise on any conflict."""
|
||||
# Imported here: pair_ingestion imports operations, which composes this.
|
||||
from .pair_ingestion import _ingest_resolved_pair_in
|
||||
|
||||
image_path = command["image_path"]
|
||||
aliases = command["aliases"]
|
||||
fingerprint = dependency_fingerprint(conn, image_path, aliases)
|
||||
known = command["known_rows"]
|
||||
unseen = known is not None and any(
|
||||
row_id not in known
|
||||
for row_id, _path in same_name_rows(conn, os.path.basename(image_path))
|
||||
)
|
||||
if unseen or fingerprint != command["expected_fingerprint"]:
|
||||
raise _Rollback(
|
||||
ReviewCommandError(
|
||||
"association-conflict",
|
||||
epoch,
|
||||
item=current,
|
||||
total=total,
|
||||
fingerprint=fingerprint,
|
||||
changed=True,
|
||||
)
|
||||
)
|
||||
rows = dependent_images(conn, (image_path, *aliases))
|
||||
if len(rows) > 1:
|
||||
# One file linked under several spellings: no single row to enrich.
|
||||
raise _Rollback(
|
||||
ReviewCommandError(
|
||||
"association-conflict",
|
||||
epoch,
|
||||
item=current,
|
||||
total=total,
|
||||
fingerprint=fingerprint,
|
||||
)
|
||||
)
|
||||
if rows:
|
||||
# Enrich the one row of this file under the spelling it was
|
||||
# linked with, so the ingester's compatibility rules see it.
|
||||
image_path = rows[0][1]
|
||||
candidate = command["candidate"]
|
||||
positive, negative = candidate["positive"], candidate["negative"]
|
||||
provenance = {
|
||||
"kind": DECISION_KIND,
|
||||
"decision_id": decision_id,
|
||||
"reviewed_digest": command["sha256"],
|
||||
"candidate_hash": candidate["hash"],
|
||||
"resolver_version": current["resolver_version"],
|
||||
"positive": positive,
|
||||
"negative": negative,
|
||||
}
|
||||
result = _ingest_resolved_pair_in(
|
||||
conn,
|
||||
self,
|
||||
image_path,
|
||||
command["metadata"],
|
||||
{"state": "resolved", "positive": positive, "negative": negative},
|
||||
provenance=provenance,
|
||||
)
|
||||
if result["status"] == "conflict":
|
||||
raise _Rollback(
|
||||
ReviewCommandError(
|
||||
"association-conflict",
|
||||
epoch,
|
||||
item=current,
|
||||
total=total,
|
||||
fingerprint=fingerprint,
|
||||
)
|
||||
)
|
||||
return {
|
||||
"id": result["pair_id"],
|
||||
"image_id": result["image_id"],
|
||||
"status": result["status"],
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _insert_review_decision(conn: sqlite3.Connection, values: tuple) -> None:
|
||||
conn.execute(_INSERT_DECISION, values)
|
||||
|
||||
|
||||
def _current_item(conn: sqlite3.Connection, item_id: str) -> dict | None:
|
||||
row = conn.execute(_DETAIL, (item_id,)).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
item = _queue_row(row, _DETAIL_COLUMNS)
|
||||
item.pop("evidence")
|
||||
return item
|
||||
@@ -0,0 +1,507 @@
|
||||
"""Needs review schema group (#249): capture-bound ambiguous items and decisions.
|
||||
|
||||
The group installs like ``image_annotations``: optional in old backups,
|
||||
required once a database is current, and valid only beside a valid image
|
||||
annotation group, because every item cites one annotation byte record. It owns
|
||||
|
||||
- ``media_review_metadata``: one row, the group's schema version and the
|
||||
review index revision (moved by every queue change);
|
||||
- ``media_review_items``: one ambiguous resolver result for one byte record,
|
||||
keyed by ``(byte_id, resolver_version, evidence_hash)``, with the file's
|
||||
spelling when it was recorded (the locator is case-folded on Windows, a
|
||||
target is not), its canonical evidence, candidate count, ``limited`` flag,
|
||||
state and revision;
|
||||
- ``media_review_decisions``: the auditable decision history of an item, one
|
||||
row per decision revision, with optional navigation to the pair and image
|
||||
the decision linked (nulled, never cascaded, when those are deleted).
|
||||
|
||||
Evidence is canonical JSON (:func:`canonical_evidence`): one object, sorted
|
||||
keys, no whitespace, ASCII only (so characters, bytes and the 8 MiB bound
|
||||
agree and lone surrogates survive), no NaN or infinity. ``evidence_hash`` is
|
||||
the SHA-256 of the item's evidence text.
|
||||
|
||||
Installation never backfills anything and never touches the filesystem.
|
||||
Validation reads only: catalog objects compared after whitespace compaction,
|
||||
columns and foreign keys through the PRAGMAs, persisted values in single
|
||||
bounded SQL passes, then one streamed pass that re-canonicalizes each evidence
|
||||
document (one row in memory at a time).
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
from typing import Any, Optional
|
||||
|
||||
from .annotation_schema import BYTES_TABLE
|
||||
from .annotation_schema import TABLES as ANNOTATION_TABLES
|
||||
from .annotation_schema import _catalog as _annotation_catalog
|
||||
from .annotation_schema import annotation_schema_error
|
||||
|
||||
try:
|
||||
from ..utils.json_strict import dumps_strict
|
||||
except ImportError:
|
||||
from utils.json_strict import dumps_strict
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
METADATA_TABLE = "media_review_metadata"
|
||||
ITEMS_TABLE = "media_review_items"
|
||||
DECISIONS_TABLE = "media_review_decisions"
|
||||
TABLES = (METADATA_TABLE, ITEMS_TABLE, DECISIONS_TABLE)
|
||||
MAX_EVIDENCE_BYTES = 8 * 1024 * 1024
|
||||
MAX_CANDIDATES = 16
|
||||
# The file's own spelling when it was recorded (bytes of UTF-8 text). Locators
|
||||
# are case-folded on Windows, so a target is formatted from this instead.
|
||||
MAX_SPELLED_PATH_BYTES = 16384
|
||||
ITEM_STATES = ("pending", "selected", "unresolved")
|
||||
DECISION_OUTCOMES = ("selected", "unresolved")
|
||||
_TIMESTAMP_BYTES = 64
|
||||
_HEX64 = "length({c})=64 AND {c} NOT GLOB '*[^0-9a-f]*'"
|
||||
_EVIDENCE_CHECK = f"CHECK(length(CAST(evidence AS BLOB)) <= {MAX_EVIDENCE_BYTES})"
|
||||
|
||||
_TABLE_DDL = {
|
||||
METADATA_TABLE: (
|
||||
f"CREATE TABLE {METADATA_TABLE} (singleton_id INTEGER PRIMARY KEY "
|
||||
"CHECK(singleton_id=1), schema_version INTEGER NOT NULL "
|
||||
f"CHECK(schema_version={SCHEMA_VERSION}), "
|
||||
"review_index_revision INTEGER NOT NULL DEFAULT 0 "
|
||||
"CHECK(review_index_revision>=0))"
|
||||
),
|
||||
ITEMS_TABLE: (
|
||||
f"CREATE TABLE {ITEMS_TABLE} (id TEXT PRIMARY KEY CHECK(length(id)=36), "
|
||||
f"byte_id TEXT NOT NULL REFERENCES {BYTES_TABLE}(id) ON DELETE RESTRICT, "
|
||||
"spelled_path TEXT NOT NULL CHECK(length(CAST(spelled_path AS BLOB)) "
|
||||
f"BETWEEN 1 AND {MAX_SPELLED_PATH_BYTES}), "
|
||||
"resolver_version INTEGER NOT NULL, evidence_hash TEXT NOT NULL "
|
||||
f"CHECK({_HEX64.format(c='evidence_hash')}), "
|
||||
f"evidence TEXT NOT NULL {_EVIDENCE_CHECK}, "
|
||||
"candidate_count INTEGER NOT NULL "
|
||||
f"CHECK(candidate_count BETWEEN 0 AND {MAX_CANDIDATES}), "
|
||||
"limited INTEGER NOT NULL CHECK(limited IN (0,1)), "
|
||||
"state TEXT NOT NULL CHECK(state IN ('pending','selected','unresolved')), "
|
||||
"revision INTEGER NOT NULL CHECK(revision>=0), created_at TEXT NOT NULL, "
|
||||
"updated_at TEXT NOT NULL, UNIQUE(byte_id, resolver_version, evidence_hash))"
|
||||
),
|
||||
DECISIONS_TABLE: (
|
||||
f"CREATE TABLE {DECISIONS_TABLE} (id TEXT PRIMARY KEY "
|
||||
"CHECK(length(id)=36), item_id TEXT NOT NULL "
|
||||
f"REFERENCES {ITEMS_TABLE}(id) ON DELETE RESTRICT, "
|
||||
"decision_revision INTEGER NOT NULL, created_at TEXT NOT NULL, "
|
||||
"outcome TEXT NOT NULL CHECK(outcome IN ('selected','unresolved')), "
|
||||
"candidate_hash TEXT CHECK(candidate_hash IS NULL OR "
|
||||
f"({_HEX64.format(c='candidate_hash')})), "
|
||||
f"evidence TEXT NOT NULL {_EVIDENCE_CHECK}, "
|
||||
"pair_id INTEGER REFERENCES prompt_pairs(id) ON DELETE SET NULL, "
|
||||
"image_id INTEGER REFERENCES generated_images(id) ON DELETE SET NULL, "
|
||||
"UNIQUE(item_id, decision_revision), "
|
||||
"CHECK((outcome='selected') = (candidate_hash IS NOT NULL)))"
|
||||
),
|
||||
}
|
||||
_INDEX_DDL = {
|
||||
# The queue reads pending items oldest first with a stable tiebreak.
|
||||
"idx_media_review_items_queue": (
|
||||
"CREATE INDEX idx_media_review_items_queue ON "
|
||||
f"{ITEMS_TABLE}(state, created_at, id)"
|
||||
),
|
||||
# Deleting a pair or an image nulls decision navigation by index, not scan.
|
||||
"idx_media_review_decisions_pair": (
|
||||
"CREATE INDEX idx_media_review_decisions_pair ON " f"{DECISIONS_TABLE}(pair_id)"
|
||||
),
|
||||
"idx_media_review_decisions_image": (
|
||||
"CREATE INDEX idx_media_review_decisions_image ON "
|
||||
f"{DECISIONS_TABLE}(image_id)"
|
||||
),
|
||||
}
|
||||
# ``sqlite_autoindex_<table>_<n>``: the TEXT primary key and the UNIQUE key.
|
||||
_AUTOINDEXES = {ITEMS_TABLE: 2, DECISIONS_TABLE: 2}
|
||||
# (name, declared type, not null, default, primary key position)
|
||||
_COLUMNS = {
|
||||
METADATA_TABLE: (
|
||||
("singleton_id", "INTEGER", 0, None, 1),
|
||||
("schema_version", "INTEGER", 1, None, 0),
|
||||
("review_index_revision", "INTEGER", 1, "0", 0),
|
||||
),
|
||||
ITEMS_TABLE: (
|
||||
("id", "TEXT", 0, None, 1),
|
||||
("byte_id", "TEXT", 1, None, 0),
|
||||
("spelled_path", "TEXT", 1, None, 0),
|
||||
("resolver_version", "INTEGER", 1, None, 0),
|
||||
("evidence_hash", "TEXT", 1, None, 0),
|
||||
("evidence", "TEXT", 1, None, 0),
|
||||
("candidate_count", "INTEGER", 1, None, 0),
|
||||
("limited", "INTEGER", 1, None, 0),
|
||||
("state", "TEXT", 1, None, 0),
|
||||
("revision", "INTEGER", 1, None, 0),
|
||||
("created_at", "TEXT", 1, None, 0),
|
||||
("updated_at", "TEXT", 1, None, 0),
|
||||
),
|
||||
DECISIONS_TABLE: (
|
||||
("id", "TEXT", 0, None, 1),
|
||||
("item_id", "TEXT", 1, None, 0),
|
||||
("decision_revision", "INTEGER", 1, None, 0),
|
||||
("created_at", "TEXT", 1, None, 0),
|
||||
("outcome", "TEXT", 1, None, 0),
|
||||
("candidate_hash", "TEXT", 0, None, 0),
|
||||
("evidence", "TEXT", 1, None, 0),
|
||||
("pair_id", "INTEGER", 0, None, 0),
|
||||
("image_id", "INTEGER", 0, None, 0),
|
||||
),
|
||||
}
|
||||
# (column, parent table, parent column, on delete)
|
||||
_FOREIGN_KEYS = {
|
||||
METADATA_TABLE: set(),
|
||||
ITEMS_TABLE: {("byte_id", BYTES_TABLE, "id", "RESTRICT")},
|
||||
DECISIONS_TABLE: {
|
||||
("item_id", ITEMS_TABLE, "id", "RESTRICT"),
|
||||
("pair_id", "prompt_pairs", "id", "SET NULL"),
|
||||
("image_id", "generated_images", "id", "SET NULL"),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class _NonCanonical(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def _reject_constant(name: str) -> Any:
|
||||
raise _NonCanonical(f"JSON constant {name} is not allowed")
|
||||
|
||||
|
||||
# Built once: validation re-encodes every stored document at startup.
|
||||
_ENCODER = json.JSONEncoder(
|
||||
sort_keys=True, separators=(",", ":"), ensure_ascii=True, allow_nan=False
|
||||
)
|
||||
_DECODER = json.JSONDecoder(parse_constant=_reject_constant)
|
||||
|
||||
|
||||
def canonical_evidence(value: Any) -> str:
|
||||
"""Serialize one evidence object canonically; ``ValueError`` when invalid.
|
||||
|
||||
The text is a JSON object with sorted keys, no whitespace, ASCII only and
|
||||
no NaN/infinity, at most ``MAX_EVIDENCE_BYTES`` bytes. A non-finite number
|
||||
(ComfyUI stamps ``NaN`` into graphs) is written as ``null``, the store's
|
||||
policy everywhere (``utils.json_strict.dumps_strict``).
|
||||
"""
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("Review evidence must be a JSON object")
|
||||
try:
|
||||
try:
|
||||
text = _ENCODER.encode(value)
|
||||
except ValueError as error:
|
||||
if "Out of range float" not in str(error):
|
||||
raise
|
||||
text = dumps_strict(
|
||||
value, sort_keys=True, separators=(",", ":"), ensure_ascii=True
|
||||
)
|
||||
except (TypeError, RecursionError) as error:
|
||||
raise ValueError(f"Review evidence is not serializable: {error}") from None
|
||||
if len(text) > MAX_EVIDENCE_BYTES:
|
||||
raise ValueError("Review evidence is larger than 8 MiB")
|
||||
return text
|
||||
|
||||
|
||||
def choosable(candidate: Any) -> bool:
|
||||
"""True when a candidate offers a prompt to choose: it is ``resolved`` and
|
||||
its positive side carries non-blank text. The page disables any other
|
||||
candidate, so it is never something to choose.
|
||||
"""
|
||||
if not isinstance(candidate, dict) or candidate.get("state") != "resolved":
|
||||
return False
|
||||
positive = candidate.get("positive")
|
||||
if not isinstance(positive, dict):
|
||||
return False
|
||||
text = positive.get("text")
|
||||
return isinstance(text, str) and bool(text.strip())
|
||||
|
||||
|
||||
# An item is worth a decision only when at least this many candidates offer
|
||||
# a prompt; with fewer there is nothing to choose between (maintainer test of
|
||||
# #284/#288: 39 items whose candidates were all "unresolved").
|
||||
MIN_CHOOSABLE = 2
|
||||
|
||||
|
||||
def evidence_offers_choice(evidence: Any) -> bool:
|
||||
"""False when stored evidence provably offers nothing to choose.
|
||||
|
||||
Unlimited evidence must list ``MIN_CHOOSABLE`` choosable candidates. A
|
||||
limited item stores none, only ``candidate_total``; it may offer a choice
|
||||
(more than ``MAX_CANDIDATES`` prompts) unless that total is too small.
|
||||
"""
|
||||
if not isinstance(evidence, dict):
|
||||
return False
|
||||
if evidence.get("limited"):
|
||||
total = evidence.get("candidate_total")
|
||||
return type(total) is int and total >= MIN_CHOOSABLE
|
||||
candidates = evidence.get("candidates")
|
||||
if not isinstance(candidates, list):
|
||||
return False
|
||||
return sum(1 for c in candidates if choosable(c)) >= MIN_CHOOSABLE
|
||||
|
||||
|
||||
def evidence_digest(text: str) -> str:
|
||||
"""SHA-256 of canonical evidence text (ASCII, hence UTF-8 identical)."""
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _is_canonical(text: Any) -> bool:
|
||||
if not isinstance(text, str):
|
||||
return False
|
||||
try:
|
||||
value = _DECODER.decode(text)
|
||||
return canonical_evidence(value) == text
|
||||
except (ValueError, RecursionError):
|
||||
return False
|
||||
|
||||
|
||||
def _compact(sql: str) -> str:
|
||||
return re.sub(r"\s+", "", sql).lower()
|
||||
|
||||
|
||||
def _uuid(column: str) -> str:
|
||||
"""SQL: *column* is a lowercase hyphenated 36-character UUID string."""
|
||||
return (
|
||||
f"typeof({column}) = 'text' AND length({column}) = 36 "
|
||||
f"AND substr({column}, 9, 1) = '-' AND substr({column}, 14, 1) = '-' "
|
||||
f"AND substr({column}, 19, 1) = '-' AND substr({column}, 24, 1) = '-' "
|
||||
f"AND length(replace({column}, '-', '')) = 32 "
|
||||
f"AND replace({column}, '-', '') NOT GLOB '*[^0-9a-f]*'"
|
||||
)
|
||||
|
||||
|
||||
def _timestamp(column: str) -> str:
|
||||
return (
|
||||
f"typeof({column}) = 'text' "
|
||||
f"AND length(CAST({column} AS BLOB)) <= {_TIMESTAMP_BYTES}"
|
||||
)
|
||||
|
||||
|
||||
def _non_negative(column: str) -> str:
|
||||
return f"typeof({column}) = 'integer' AND {column} >= 0"
|
||||
|
||||
|
||||
def _hex64(column: str) -> str:
|
||||
return f"typeof({column}) = 'text' AND " + _HEX64.format(c=column)
|
||||
|
||||
|
||||
def _evidence(column: str) -> str:
|
||||
# Type and size only: the streamed pass below proves the text canonical
|
||||
# (which implies printable ASCII), so a pattern scan here would read every
|
||||
# document a second time.
|
||||
return (
|
||||
f"typeof({column}) = 'text' "
|
||||
f"AND length(CAST({column} AS BLOB)) BETWEEN 2 AND {MAX_EVIDENCE_BYTES}"
|
||||
)
|
||||
|
||||
|
||||
def _nullable_integer(column: str) -> str:
|
||||
return f"({column} IS NULL OR typeof({column}) = 'integer')"
|
||||
|
||||
|
||||
def _in(column: str, values: tuple) -> str:
|
||||
return f"typeof({column}) = 'text' AND {column} IN ({repr(values)[1:-1]})"
|
||||
|
||||
|
||||
# Every row of a table must satisfy its predicate; one pass each, LIMIT 1.
|
||||
_ROW_PREDICATES = {
|
||||
ITEMS_TABLE: " AND ".join(
|
||||
(
|
||||
_uuid("id"),
|
||||
_uuid("byte_id"),
|
||||
"typeof(spelled_path) = 'text' AND length(CAST(spelled_path AS BLOB)) "
|
||||
f"BETWEEN 1 AND {MAX_SPELLED_PATH_BYTES}",
|
||||
"typeof(resolver_version) = 'integer'",
|
||||
_hex64("evidence_hash"),
|
||||
_evidence("evidence"),
|
||||
_non_negative("candidate_count"),
|
||||
f"candidate_count <= {MAX_CANDIDATES}",
|
||||
"typeof(limited) = 'integer' AND limited IN (0, 1)",
|
||||
_in("state", ITEM_STATES),
|
||||
_non_negative("revision"),
|
||||
_timestamp("created_at"),
|
||||
_timestamp("updated_at"),
|
||||
)
|
||||
),
|
||||
DECISIONS_TABLE: " AND ".join(
|
||||
(
|
||||
_uuid("id"),
|
||||
_uuid("item_id"),
|
||||
_non_negative("decision_revision"),
|
||||
_timestamp("created_at"),
|
||||
_in("outcome", DECISION_OUTCOMES),
|
||||
"(CASE WHEN outcome = 'selected' THEN "
|
||||
f"{_hex64('candidate_hash')} ELSE candidate_hash IS NULL END)",
|
||||
_evidence("evidence"),
|
||||
_nullable_integer("pair_id"),
|
||||
_nullable_integer("image_id"),
|
||||
)
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _expected_catalog() -> set:
|
||||
objects = {("table", name, _compact(sql)) for name, sql in _TABLE_DDL.items()}
|
||||
objects |= {("index", name, _compact(sql)) for name, sql in _INDEX_DDL.items()}
|
||||
objects |= {
|
||||
("index", f"sqlite_autoindex_{table}_{n}", "")
|
||||
for table, count in _AUTOINDEXES.items()
|
||||
for n in range(1, count + 1)
|
||||
}
|
||||
return objects
|
||||
|
||||
|
||||
def _catalog(conn: sqlite3.Connection) -> list:
|
||||
names = TABLES + tuple(_INDEX_DDL)
|
||||
marks = ",".join("?" * len(names))
|
||||
table_marks = ",".join("?" * len(TABLES))
|
||||
return conn.execute( # only "?" placeholders are interpolated
|
||||
"SELECT type,name,sql FROM sqlite_master "
|
||||
f"WHERE name IN ({marks}) OR tbl_name IN ({table_marks})", # nosec B608
|
||||
names + TABLES,
|
||||
).fetchall()
|
||||
|
||||
|
||||
def _structure_error(conn: sqlite3.Connection) -> Optional[str]:
|
||||
for table, expected in _COLUMNS.items():
|
||||
columns = tuple(
|
||||
(row[1], row[2].upper(), row[3], row[4], row[5])
|
||||
for row in conn.execute(f"PRAGMA table_info({table})")
|
||||
)
|
||||
if columns != expected:
|
||||
return f"Review queue table {table} has invalid columns"
|
||||
foreign_keys = {
|
||||
(row[3], row[2], row[4], row[6])
|
||||
for row in conn.execute(f"PRAGMA foreign_key_list({table})")
|
||||
}
|
||||
if foreign_keys != _FOREIGN_KEYS[table]:
|
||||
return f"Review queue table {table} has invalid foreign keys"
|
||||
if conn.execute(f"PRAGMA foreign_key_check({table})").fetchone():
|
||||
return f"Review queue table {table} contains foreign key violations"
|
||||
return None
|
||||
|
||||
|
||||
def _evidence_error(conn: sqlite3.Connection) -> Optional[str]:
|
||||
"""Stream every evidence document once; one row is held at a time."""
|
||||
for evidence_hash, text in conn.execute(
|
||||
f"SELECT evidence_hash, evidence FROM {ITEMS_TABLE}" # nosec B608
|
||||
):
|
||||
if not _is_canonical(text) or evidence_digest(text) != evidence_hash:
|
||||
return f"Review queue table {ITEMS_TABLE} contains invalid evidence"
|
||||
for (text,) in conn.execute(
|
||||
f"SELECT evidence FROM {DECISIONS_TABLE}" # nosec B608
|
||||
):
|
||||
if not _is_canonical(text):
|
||||
return f"Review queue table {DECISIONS_TABLE} contains invalid evidence"
|
||||
return None
|
||||
|
||||
|
||||
def _value_error(conn: sqlite3.Connection) -> Optional[str]:
|
||||
# Only this module's constants are interpolated into the SQL below.
|
||||
rows = conn.execute(
|
||||
"SELECT COUNT(*), MIN(typeof(singleton_id) = 'integer' " # nosec B608
|
||||
"AND singleton_id = 1 AND typeof(schema_version) = 'integer' "
|
||||
f"AND schema_version = {SCHEMA_VERSION} "
|
||||
f"AND {_non_negative('review_index_revision')}) FROM {METADATA_TABLE}"
|
||||
).fetchone()
|
||||
if rows[0] != 1 or rows[1] != 1:
|
||||
return "Review queue metadata is invalid"
|
||||
for table, predicate in _ROW_PREDICATES.items():
|
||||
if conn.execute(
|
||||
f"SELECT 1 FROM {table} WHERE NOT ({predicate}) LIMIT 1" # nosec B608
|
||||
).fetchone():
|
||||
return f"Review queue table {table} contains invalid values"
|
||||
return _evidence_error(conn)
|
||||
|
||||
|
||||
def review_without_annotations_error(conn: sqlite3.Connection) -> Optional[str]:
|
||||
"""Catalog only: a review group present without its annotation tables.
|
||||
|
||||
Startup asks this before the annotation group installs, which would make
|
||||
a review group copied without it look valid. It reads no rows; the full
|
||||
validation runs once in :func:`migrate_review_state`.
|
||||
"""
|
||||
if not _catalog(conn):
|
||||
return None
|
||||
present = {
|
||||
name for kind, name, _sql in _annotation_catalog(conn) if kind == "table"
|
||||
}
|
||||
if present != set(ANNOTATION_TABLES):
|
||||
return (
|
||||
"Review queue schema requires a valid image annotation schema: "
|
||||
"Missing image annotation schema"
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def review_schema_error(
|
||||
conn: sqlite3.Connection,
|
||||
*,
|
||||
allow_missing: bool = True,
|
||||
annotations_valid: bool = False,
|
||||
) -> Optional[str]:
|
||||
"""Reject partial or unsupported review schemas and invalid rows.
|
||||
|
||||
Reads only: nothing is ever repaired. A database without any of the
|
||||
group's objects is acceptable for backups that predate it unless
|
||||
``allow_missing`` is false. A present group requires a present, valid
|
||||
image annotation group; ``annotations_valid`` says the caller has just
|
||||
validated it (startup's annotation install), so only its presence is
|
||||
checked again.
|
||||
"""
|
||||
objects = _catalog(conn)
|
||||
if not objects:
|
||||
return None if allow_missing else "Missing review queue schema"
|
||||
if annotations_valid:
|
||||
annotation_error = review_without_annotations_error(conn)
|
||||
else:
|
||||
annotation_error = annotation_schema_error(conn, allow_missing=False)
|
||||
if annotation_error:
|
||||
annotation_error = (
|
||||
"Review queue schema requires a valid image annotation schema: "
|
||||
+ annotation_error
|
||||
)
|
||||
if annotation_error:
|
||||
return annotation_error
|
||||
present = {name for kind, name, _sql in objects if kind == "table"}
|
||||
if present != set(TABLES):
|
||||
return "Incomplete review queue schema"
|
||||
actual = {(kind, name, _compact(sql or "")) for kind, name, sql in objects}
|
||||
if actual != _expected_catalog():
|
||||
return "Review queue has an invalid schema definition"
|
||||
return _structure_error(conn) or _value_error(conn)
|
||||
|
||||
|
||||
def migrate_review_state(conn: sqlite3.Connection) -> None:
|
||||
"""Install the group atomically; validate before and after creating it.
|
||||
|
||||
Runs after ``migrate_annotation_state`` (which has just validated the
|
||||
annotation group): the group is only valid beside the annotation group it
|
||||
cites. A present group is validated once, evidence included.
|
||||
"""
|
||||
conn.execute("SAVEPOINT media_review_schema")
|
||||
try:
|
||||
installed = bool(_catalog(conn))
|
||||
error = review_schema_error(conn, annotations_valid=True)
|
||||
if error:
|
||||
raise sqlite3.DatabaseError(error)
|
||||
if installed:
|
||||
# Already complete and valid: nothing to create, and the evidence
|
||||
# pass is the expensive part of validation, so it runs once.
|
||||
conn.execute("RELEASE media_review_schema")
|
||||
return
|
||||
for sql in _TABLE_DDL.values():
|
||||
conn.execute(sql.replace("CREATE TABLE", "CREATE TABLE IF NOT EXISTS", 1))
|
||||
for sql in _INDEX_DDL.values():
|
||||
conn.execute(sql.replace("INDEX", "INDEX IF NOT EXISTS", 1))
|
||||
conn.execute(
|
||||
f"INSERT OR IGNORE INTO {METADATA_TABLE} "
|
||||
"(singleton_id, schema_version, review_index_revision) VALUES (1, ?, 0)",
|
||||
(SCHEMA_VERSION,),
|
||||
)
|
||||
error = review_schema_error(conn, allow_missing=False, annotations_valid=True)
|
||||
if error:
|
||||
raise sqlite3.DatabaseError(error)
|
||||
conn.execute("RELEASE media_review_schema")
|
||||
except BaseException:
|
||||
conn.execute("ROLLBACK TO media_review_schema")
|
||||
conn.execute("RELEASE media_review_schema")
|
||||
raise
|
||||
+49
-13
@@ -8,6 +8,9 @@ Split into domain-specific mixins for maintainability:
|
||||
- AutotagRoutesMixin: auto-tagging model management and tagging
|
||||
- JobRoutesMixin: the session job registry behind Activity (py/api/jobs.py)
|
||||
- ImageTagJobsMixin: bulk Add tags / Auto-tag as image-tags jobs
|
||||
- ReviewRoutesMixin: the Needs review queue, item detail and decisions
|
||||
- ReviewRecordingMixin: Needs review recording for the scan and the monitor
|
||||
- SystemRoutesMixin: System health check, Overview and usage statistics
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
@@ -15,6 +18,7 @@ import datetime
|
||||
import functools
|
||||
import gzip as gzip_module
|
||||
import json
|
||||
import math
|
||||
import ntpath
|
||||
import os
|
||||
import re
|
||||
@@ -28,6 +32,8 @@ from .pairs import PairRoutesMixin
|
||||
from .library import LibraryRoutesMixin
|
||||
from .vocabulary import VocabularyRoutesMixin
|
||||
from .annotations import AnnotationRoutesMixin
|
||||
from .review import ReviewRoutesMixin
|
||||
from .review_recording import ReviewRecordingMixin
|
||||
from .saved_views import SavedViewRoutesMixin
|
||||
from .history import HistoryRoutesMixin
|
||||
from .images import ImageRoutesMixin
|
||||
@@ -35,6 +41,7 @@ from .admin import AdminRoutesMixin
|
||||
from .settings_routes import SettingsRoutesMixin
|
||||
from .job_routes import JobRoutesMixin
|
||||
from .image_tag_jobs import ImageTagJobsMixin
|
||||
from .system import SystemRoutesMixin
|
||||
from .jobs import JobRegistry
|
||||
from .logging_routes import LoggingRoutesMixin
|
||||
from .autotag_routes import AutotagRoutesMixin
|
||||
@@ -108,16 +115,20 @@ STATIC_MIME_TYPES = {
|
||||
".html": "text/html",
|
||||
}
|
||||
_STATIC_SEPARATORS = re.compile(r"[\\/]+")
|
||||
# Served by exact request path beside the extension allow-list: the browser's
|
||||
# copy of the inspection node table (local Inspect resolves with it, #249).
|
||||
STATIC_EXACT_FILES = {("js", "inspection-semantics.json"): "application/json"}
|
||||
|
||||
|
||||
def _resolve_static_file(root, filepath):
|
||||
def _resolve_static_file(root, filepath, allowed=STATIC_MIME_TYPES):
|
||||
"""Real path of ``filepath`` under ``root``, or None when it must not be served.
|
||||
|
||||
The request path is untrusted. It is rejected when it is empty, carries a
|
||||
``..`` segment, is absolute on any platform (a leading slash or backslash,
|
||||
a drive letter such as ``C:`` or a UNC ``\\\\server\\share`` prefix all
|
||||
re-anchor ``os.path.join`` on Windows), has an extension outside
|
||||
STATIC_MIME_TYPES, or resolves (symlinks included) outside ``root``.
|
||||
``allowed`` (STATIC_MIME_TYPES), or resolves (symlinks included) outside
|
||||
``root``.
|
||||
"""
|
||||
if not isinstance(filepath, str) or not filepath:
|
||||
return None
|
||||
@@ -130,7 +141,7 @@ def _resolve_static_file(root, filepath):
|
||||
segments = _STATIC_SEPARATORS.split(filepath)
|
||||
if any(segment in ("", ".", "..") for segment in segments):
|
||||
return None
|
||||
if os.path.splitext(filepath)[1].lower() not in STATIC_MIME_TYPES:
|
||||
if os.path.splitext(filepath)[1].lower() not in allowed:
|
||||
return None
|
||||
try:
|
||||
root_real = Path(os.path.normcase(os.path.realpath(root)))
|
||||
@@ -257,6 +268,8 @@ class PromptManagerAPI(
|
||||
LibraryRoutesMixin,
|
||||
VocabularyRoutesMixin,
|
||||
AnnotationRoutesMixin,
|
||||
ReviewRoutesMixin,
|
||||
ReviewRecordingMixin,
|
||||
SavedViewRoutesMixin,
|
||||
HistoryRoutesMixin,
|
||||
PromptRoutesMixin,
|
||||
@@ -265,6 +278,7 @@ class PromptManagerAPI(
|
||||
SettingsRoutesMixin,
|
||||
JobRoutesMixin,
|
||||
ImageTagJobsMixin,
|
||||
SystemRoutesMixin,
|
||||
LoggingRoutesMixin,
|
||||
AutotagRoutesMixin,
|
||||
LoraIntegrationMixin,
|
||||
@@ -296,6 +310,8 @@ class PromptManagerAPI(
|
||||
# created on first use with their restore subscription.
|
||||
self._captures = None
|
||||
self._captures_unsubscribe = None
|
||||
# Set by close_annotation_captures (shutdown): no store is recreated.
|
||||
self._captures_closed = False
|
||||
self._cached_output_dir = None # Lazy-cached by _find_comfyui_output_dir()
|
||||
self._html_cache = {} # Cached HTML file contents keyed by path
|
||||
self._gallery_cache = {} # dict: path_str -> (files, timestamp)
|
||||
@@ -321,6 +337,11 @@ class PromptManagerAPI(
|
||||
self._scan_job = None
|
||||
# The current duplicate scan (a py/api/jobs.Job outside the registry).
|
||||
self._duplicate_job = None
|
||||
# The last System health report this session and the database
|
||||
# generation it describes, bumped when a restore replaces self.db
|
||||
# (py/api/system.py).
|
||||
self._health_report = None
|
||||
self._db_generation = 0
|
||||
|
||||
# Run cleanup on initialization to remove any existing duplicates
|
||||
try:
|
||||
@@ -471,6 +492,7 @@ class PromptManagerAPI(
|
||||
self._register_library_routes(routes)
|
||||
self._register_vocabulary_routes(routes)
|
||||
self._register_annotation_routes(routes)
|
||||
self._register_review_routes(routes)
|
||||
self._register_saved_view_routes(routes)
|
||||
self._register_history_routes(routes)
|
||||
self._register_prompt_routes(routes)
|
||||
@@ -478,6 +500,7 @@ class PromptManagerAPI(
|
||||
self._register_admin_routes(routes)
|
||||
self._register_settings_routes(routes)
|
||||
self._register_job_routes(routes)
|
||||
self._register_system_routes(routes)
|
||||
self._register_logging_routes(routes)
|
||||
self._register_autotag_routes(routes)
|
||||
self._register_lora_routes(routes)
|
||||
@@ -499,15 +522,20 @@ class PromptManagerAPI(
|
||||
def _serve_static_file(self, subdir, filepath):
|
||||
"""Serve ``web/<subdir>/<filepath>``; 403 outside it, 404 when missing."""
|
||||
root = os.path.join(_get_project_root(), "web", subdir)
|
||||
file_path = _resolve_static_file(root, filepath)
|
||||
exact = STATIC_EXACT_FILES.get((subdir, filepath))
|
||||
# An exact file passes the same containment check, its extension only.
|
||||
allowed = STATIC_MIME_TYPES if exact is None else {".json"}
|
||||
file_path = _resolve_static_file(root, filepath, allowed)
|
||||
if file_path is None:
|
||||
return web.Response(text="Forbidden", status=403)
|
||||
content_type = (
|
||||
exact or STATIC_MIME_TYPES[os.path.splitext(file_path)[1].lower()]
|
||||
)
|
||||
if not os.path.isfile(file_path):
|
||||
return web.Response(text="Not Found", status=404)
|
||||
ext = os.path.splitext(file_path)[1].lower()
|
||||
with open(file_path, "rb") as f:
|
||||
content = f.read()
|
||||
return web.Response(body=content, content_type=STATIC_MIME_TYPES[ext])
|
||||
return web.Response(body=content, content_type=content_type)
|
||||
|
||||
def _serve_html_page(self, filename, title, cache=True):
|
||||
"""Serve ``web/<filename>``; 404 when missing, 500 when unreadable."""
|
||||
@@ -591,12 +619,12 @@ class PromptManagerAPI(
|
||||
return prompts
|
||||
|
||||
def _clean_nan_recursive(self, obj):
|
||||
"""Recursively clean NaN values from nested data structures."""
|
||||
"""Recursively replace NaN and Infinity in nested data with None."""
|
||||
if isinstance(obj, dict):
|
||||
return {key: self._clean_nan_recursive(value) for key, value in obj.items()}
|
||||
elif isinstance(obj, list):
|
||||
return [self._clean_nan_recursive(item) for item in obj]
|
||||
elif isinstance(obj, float) and str(obj) == "nan":
|
||||
elif isinstance(obj, float) and not math.isfinite(obj):
|
||||
return None
|
||||
else:
|
||||
return obj
|
||||
@@ -719,6 +747,13 @@ class PromptManagerAPI(
|
||||
Returns:
|
||||
List[Path]: Validated directory paths, possibly empty.
|
||||
"""
|
||||
return self._output_dirs_with_origin()[0]
|
||||
|
||||
def _output_dirs_with_origin(self):
|
||||
"""``(dirs, auto_detected)``: the ``_get_all_output_dirs`` list, and
|
||||
whether it comes from detection or ComfyUI's output rather than from
|
||||
a valid configured root (False when there is no directory at all).
|
||||
"""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
configured = list(GalleryConfig.MONITORING_DIRECTORIES)
|
||||
@@ -737,15 +772,16 @@ class PromptManagerAPI(
|
||||
output_dirs.append(Path(real))
|
||||
|
||||
if output_dirs:
|
||||
return output_dirs
|
||||
return output_dirs, False
|
||||
|
||||
if configured:
|
||||
# Every configured root was rejected: never let auto-detection
|
||||
# resurrect it, serve ComfyUI's own output directory instead.
|
||||
return self._comfyui_output_dir_list()
|
||||
|
||||
fallback = self._find_comfyui_output_dir()
|
||||
return [Path(fallback)] if fallback else []
|
||||
fallback_dirs = self._comfyui_output_dir_list()
|
||||
else:
|
||||
fallback = self._find_comfyui_output_dir()
|
||||
fallback_dirs = [Path(fallback)] if fallback else []
|
||||
return fallback_dirs, bool(fallback_dirs)
|
||||
|
||||
def _comfyui_output_dir_list(self):
|
||||
"""ComfyUI's output directory as a one-element list, or [] if unknown."""
|
||||
|
||||
+132
-32
@@ -4,6 +4,7 @@ import asyncio
|
||||
import contextlib
|
||||
from contextlib import closing
|
||||
import datetime
|
||||
import functools
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
@@ -41,9 +42,10 @@ try:
|
||||
PLACEHOLDERS,
|
||||
GenerationParamsError,
|
||||
extract_generation_params,
|
||||
parse_js_json,
|
||||
parse_page_json,
|
||||
)
|
||||
from ...utils.prompt_pair_resolver import resolve_prompt_pair
|
||||
from ...utils.review_capture import reviewable
|
||||
except ImportError:
|
||||
from database.operations import PromptDatabase
|
||||
from database.pair_ingestion import ingest_resolved_pair
|
||||
@@ -51,9 +53,10 @@ except ImportError:
|
||||
PLACEHOLDERS,
|
||||
GenerationParamsError,
|
||||
extract_generation_params,
|
||||
parse_js_json,
|
||||
parse_page_json,
|
||||
)
|
||||
from utils.prompt_pair_resolver import resolve_prompt_pair
|
||||
from utils.review_capture import reviewable
|
||||
|
||||
|
||||
try:
|
||||
@@ -211,6 +214,30 @@ def _time_left(done, total, batches, elapsed):
|
||||
return f" · about {minutes} min left"
|
||||
|
||||
|
||||
def new_review_counts():
|
||||
"""The scan's Needs review tally: ``added`` new items, ``queued`` files
|
||||
in the queue after this scan (new or already there), ``settled`` items
|
||||
removed because their file is linked now, ``untraced`` items removed
|
||||
because they offer nothing to choose, and ``errors``."""
|
||||
return {"added": 0, "queued": 0, "settled": 0, "untraced": 0, "errors": 0}
|
||||
|
||||
|
||||
def _review_scope(review):
|
||||
"""The scan job's ``scope`` keys for its Needs review tally.
|
||||
|
||||
``review_items`` is the number of this scan's files that are in the
|
||||
pending queue after it: the review group's ambiguous files, less those a
|
||||
capture could not read or a decision already settled.
|
||||
"""
|
||||
return {
|
||||
"review_items": review["queued"],
|
||||
"review_added": review["added"],
|
||||
"review_settled": review["settled"],
|
||||
"review_untraced": review["untraced"],
|
||||
"review_errors": review["errors"],
|
||||
}
|
||||
|
||||
|
||||
def _scan_status_line(done, total, batches, elapsed):
|
||||
""" "Processing file N of M", with an estimate once enough batches ran."""
|
||||
line = f"Processing file {done:,} of {total:,}"
|
||||
@@ -804,6 +831,9 @@ class AdminRoutesMixin:
|
||||
result = await self._run_in_executor(
|
||||
self._delete_duplicate_images_sync, image_paths, output_dir
|
||||
)
|
||||
if result["deleted_count"]:
|
||||
# The cached listing would keep (and count) the deleted files.
|
||||
self.invalidate_gallery_cache()
|
||||
|
||||
response_data = {
|
||||
"success": True,
|
||||
@@ -1271,6 +1301,7 @@ class AdminRoutesMixin:
|
||||
self.logger.info(f"Removed {len(removed)} old safety backups")
|
||||
previous = self.db
|
||||
self.db = PromptDatabase(model.db_path)
|
||||
self._database_replaced() # a health report described the old one
|
||||
# Its connections would otherwise linger (a file lock on Windows)
|
||||
previous.close_all()
|
||||
prompt_count = self.db.model.get_database_info().get("total_prompts", 0)
|
||||
@@ -1340,28 +1371,34 @@ class AdminRoutesMixin:
|
||||
}
|
||||
)
|
||||
|
||||
def _start_scan_job(self, files=DEFAULT_SCAN_SCOPE):
|
||||
def _start_scan_job(self, files=DEFAULT_SCAN_SCOPE, roots=None, output_dirs=None):
|
||||
"""Register the scan job synchronously; its runner starts on the loop.
|
||||
|
||||
``files`` is the scope (``SCAN_SCOPES``). Raises JobBusy while another
|
||||
record-changing job runs.
|
||||
``files`` is the scope (``SCAN_SCOPES``). ``roots`` selects sources by
|
||||
their index in ``output_dirs`` (the effective sources the caller
|
||||
resolved); None scans every source, resolved when the job starts.
|
||||
Raises JobBusy while another record-changing job runs.
|
||||
"""
|
||||
if files not in SCAN_SCOPES:
|
||||
raise ValueError(f"Unknown scan scope: {files!r}")
|
||||
if roots is not None and (
|
||||
output_dirs is None or not roots or max(roots) >= len(output_dirs)
|
||||
):
|
||||
raise ValueError("A scoped scan needs the sources its roots index")
|
||||
job = self._jobs.start(
|
||||
SCAN_JOB_KIND,
|
||||
SCAN_JOB_TITLE,
|
||||
record_changing=True,
|
||||
scope={"files": files},
|
||||
runner=self._run_scan_job,
|
||||
scope={"files": files, "roots": None if roots is None else list(roots)},
|
||||
runner=functools.partial(self._run_scan_job, output_dirs=output_dirs),
|
||||
groups=SCAN_GROUP_NAMES,
|
||||
)
|
||||
self._scan_job = job
|
||||
return job
|
||||
|
||||
async def _run_scan_job(self, job):
|
||||
async def _run_scan_job(self, job, output_dirs=None):
|
||||
final = None
|
||||
async for payload in self._scan_images_events(job):
|
||||
async for payload in self._scan_images_events(job, output_dirs):
|
||||
job.publish(payload)
|
||||
final = payload.get("type")
|
||||
state = SCAN_FINAL_STATES.get(final, "finished")
|
||||
@@ -1396,16 +1433,25 @@ class AdminRoutesMixin:
|
||||
)
|
||||
return response
|
||||
|
||||
async def _scan_images_events(self, job):
|
||||
"""Yield progress payloads while scanning every configured output directory.
|
||||
async def _scan_images_events(self, job, output_dirs=None):
|
||||
"""Yield progress payloads while scanning the output directories.
|
||||
|
||||
The payloads are the legacy SSE events, unchanged; ``job`` also gets
|
||||
its status line, totals and one outcome per processed file.
|
||||
Every effective source is walked, or only those ``job.scope["roots"]``
|
||||
selects (indexes into ``output_dirs``). The payloads are the legacy SSE
|
||||
events, unchanged; ``job`` also gets its status line, totals and one
|
||||
outcome per processed file. Rows keep each source's listing index.
|
||||
"""
|
||||
try:
|
||||
self.logger.info("Starting image scan operation")
|
||||
output_dirs = self._get_all_output_dirs()
|
||||
if not output_dirs:
|
||||
if output_dirs is None:
|
||||
output_dirs = self._get_all_output_dirs()
|
||||
selected = job.scope.get("roots")
|
||||
walked = (
|
||||
output_dirs
|
||||
if selected is None
|
||||
else [output_dirs[index] for index in selected]
|
||||
)
|
||||
if not walked:
|
||||
self.logger.error("No output directories found")
|
||||
message = (
|
||||
"No output directories found. "
|
||||
@@ -1415,18 +1461,18 @@ class AdminRoutesMixin:
|
||||
yield {"type": "error", "message": message}
|
||||
return
|
||||
|
||||
dir_names = [self._public_path(d) for d in output_dirs]
|
||||
folders = _plural(len(output_dirs), "output folder")
|
||||
job.scope["output_folders"] = len(output_dirs)
|
||||
dir_names = [self._public_path(d) for d in walked]
|
||||
folders = _plural(len(walked), "output folder")
|
||||
job.scope["output_folders"] = len(walked)
|
||||
scope = SCAN_SCOPE_LOG[job.scope.get("files", "all")]
|
||||
job.log_line(f"Scan started · {folders} · {scope}")
|
||||
job.update(status_line=f"Looking for media files in {folders}")
|
||||
yield self._scan_progress(
|
||||
0, f"Scanning {len(output_dirs)} directory(ies) for media files..."
|
||||
0, f"Scanning {len(walked)} directory(ies) for media files..."
|
||||
)
|
||||
|
||||
media_files = await self._run_in_executor(
|
||||
self._collect_output_media_sync, output_dirs
|
||||
self._collect_output_media_sync, walked
|
||||
)
|
||||
counts = dict.fromkeys(SCAN_COUNT_KEYS, 0)
|
||||
job.update(
|
||||
@@ -1441,10 +1487,13 @@ class AdminRoutesMixin:
|
||||
known = set()
|
||||
if media_files and job.scope.get("files") != "all":
|
||||
known = await self._known_scan_files(job, media_files)
|
||||
review = new_review_counts()
|
||||
job.scope.update(_review_scope(review))
|
||||
async for event in self._scan_batches(
|
||||
media_files, counts, job, roots, known
|
||||
media_files, counts, job, roots, known, review
|
||||
):
|
||||
yield event
|
||||
self._log_review_items(job, review)
|
||||
|
||||
self.logger.info(
|
||||
f"Scan completed: processed={counts['processed']}, "
|
||||
@@ -1481,12 +1530,16 @@ class AdminRoutesMixin:
|
||||
sizes = {str(media_file): _file_size(media_file) for media_file in media_files}
|
||||
return self.db.get_known_resolved_paths(sizes)
|
||||
|
||||
async def _scan_batches(self, media_files, counts, job, roots=(), known=()):
|
||||
async def _scan_batches(
|
||||
self, media_files, counts, job, roots=(), known=(), review=None
|
||||
):
|
||||
"""Process ``media_files`` in executor batches, updating ``counts`` in place.
|
||||
|
||||
Records each file's outcome in ``job`` and yields one progress event
|
||||
per batch. Files in ``known`` (path strings) are not read: they count
|
||||
as processed and unchanged and are recorded as skipped.
|
||||
as processed and unchanged and are recorded as skipped. ``review``
|
||||
counts the Needs review items the batches record (``job.scope``
|
||||
``review_items`` / ``review_errors`` follow it after every batch).
|
||||
"""
|
||||
total = len(media_files)
|
||||
started = time.monotonic()
|
||||
@@ -1509,9 +1562,16 @@ class AdminRoutesMixin:
|
||||
batch_results,
|
||||
roots=roots,
|
||||
outcomes=outcomes,
|
||||
review=review,
|
||||
)
|
||||
for key in counts:
|
||||
counts[key] += batch_counts[key]
|
||||
if review is not None:
|
||||
# After the batch's links: a file linked now leaves the queue.
|
||||
await self._run_in_executor(
|
||||
self._settle_scanned_reviews, batch, roots, review
|
||||
)
|
||||
job.scope.update(_review_scope(review))
|
||||
for group, row, log_line in outcomes:
|
||||
job.record(group, row, log_line)
|
||||
|
||||
@@ -1561,12 +1621,20 @@ class AdminRoutesMixin:
|
||||
|
||||
return map_parallel(extract, file_batch, PromptManagerConfig.WORKER_THREADS)
|
||||
|
||||
def _ingest_scan_batch_sync(self, batch_results, roots=(), outcomes=None):
|
||||
def _ingest_scan_batch_sync(
|
||||
self, batch_results, roots=(), outcomes=None, review=None
|
||||
):
|
||||
"""Store the prompts found in a metadata batch (blocking DB work).
|
||||
|
||||
Returns the legacy counts. When ``outcomes`` is a list, every processed
|
||||
file appends its ``(group, row, log_line)`` for the scan job, with the
|
||||
path relative to its root in ``roots`` (see ``_scan_roots``).
|
||||
path relative to its root in ``roots`` (see ``_scan_roots``). A file
|
||||
whose final outcome is ambiguous and unlinked (the resolver found it
|
||||
ambiguous and the fallback did not link it) is recorded for Needs
|
||||
review (best effort, counted in ``review`` when given); one a saved
|
||||
row already links is reported ``unchanged`` and never queued. An
|
||||
ambiguous file with nothing to choose is reported ``unresolved`` and
|
||||
its pending items without a choice leave the queue.
|
||||
"""
|
||||
counts = dict.fromkeys(SCAN_COUNT_KEYS, 0)
|
||||
|
||||
@@ -1578,13 +1646,27 @@ class AdminRoutesMixin:
|
||||
if self._scan_job and self._scan_job.cancelled:
|
||||
break
|
||||
counts["processed"] += 1
|
||||
ambiguous, untraced = [], []
|
||||
try:
|
||||
outcome = self._ingest_scanned_file(media_file, metadata, detailed=True)
|
||||
outcome = self._ingest_scanned_file(
|
||||
media_file,
|
||||
metadata,
|
||||
detailed=True,
|
||||
ambiguous=ambiguous,
|
||||
untraced=untraced,
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error processing {media_file.name}: {e}")
|
||||
counts["failed"] += 1
|
||||
note(media_file, "error")
|
||||
continue
|
||||
if ambiguous and outcome == "ambiguous":
|
||||
if self._review_file_linked(media_file, roots):
|
||||
outcome = "unchanged" # already linked: not a review case
|
||||
else:
|
||||
self._record_scanned_review(media_file, roots, review)
|
||||
elif untraced and outcome == "unresolved":
|
||||
self._settle_untraced_reviews(untraced, roots, review)
|
||||
if outcome is None:
|
||||
note(media_file, None)
|
||||
continue
|
||||
@@ -1602,11 +1684,14 @@ class AdminRoutesMixin:
|
||||
|
||||
@staticmethod
|
||||
def _graph_prompt_text(metadata):
|
||||
"""The positive prompt the browser parser traces from the graphs, or None."""
|
||||
"""The positive prompt the browser parser traces from the graphs, or None.
|
||||
|
||||
Records parse as the legacy pages parsed them (NaN reads as null).
|
||||
"""
|
||||
records = {}
|
||||
for key in ("prompt", "workflow"):
|
||||
value = metadata.get(key)
|
||||
records[key] = parse_js_json(value) if isinstance(value, str) else value
|
||||
records[key] = parse_page_json(value) if isinstance(value, str) else value
|
||||
try:
|
||||
positive = extract_generation_params(records)["positivePrompt"]
|
||||
except GenerationParamsError:
|
||||
@@ -1615,13 +1700,21 @@ class AdminRoutesMixin:
|
||||
return None
|
||||
return positive.strip() or None
|
||||
|
||||
def _ingest_scanned_file(self, media_file, metadata, detailed=False):
|
||||
def _ingest_scanned_file(
|
||||
self, media_file, metadata, detailed=False, ambiguous=None, untraced=None
|
||||
):
|
||||
"""Enrich exact pairs; retain legacy positive-only import for absent pairs.
|
||||
|
||||
Returns the file's scan status, or None without prompt metadata. With
|
||||
``detailed``, a stored or linked file returns a dict instead: its
|
||||
``status``, the prompt's display ``title`` and, for a resolved pair,
|
||||
the pair counts.
|
||||
the pair counts. When the resolver finds the file ambiguous (whether
|
||||
or not the browser parser then links it) and ``ambiguous`` is a list,
|
||||
``media_file`` is appended to it; only a file that then returns
|
||||
``"ambiguous"`` (nothing linked it) is a Needs review case. An
|
||||
ambiguous result with nothing to choose (``reviewable`` is False: fewer
|
||||
than two candidates carry a prompt) that nothing links returns
|
||||
``"unresolved"`` instead, and ``media_file`` goes to ``untraced``.
|
||||
"""
|
||||
if not metadata:
|
||||
return None
|
||||
@@ -1637,6 +1730,13 @@ class AdminRoutesMixin:
|
||||
return None
|
||||
_with_file_size(canonical, media_file)
|
||||
resolution = resolve_prompt_pair(canonical)
|
||||
# Ambiguous without a choice: unresolved unless something links it.
|
||||
unchoosable = resolution["state"] == "ambiguous" and not reviewable(resolution)
|
||||
if unchoosable:
|
||||
if untraced is not None:
|
||||
untraced.append(media_file)
|
||||
elif resolution["state"] == "ambiguous" and ambiguous is not None:
|
||||
ambiguous.append(media_file)
|
||||
if resolution["state"] == "resolved":
|
||||
result = ingest_resolved_pair(
|
||||
self.db, str(media_file), canonical, resolution
|
||||
@@ -1656,9 +1756,9 @@ class AdminRoutesMixin:
|
||||
# Graphs without a recognized sampler keep the legacy extractor below.
|
||||
prompt_text = self._graph_prompt_text(metadata)
|
||||
if not prompt_text:
|
||||
return resolution["state"]
|
||||
return "unresolved" if unchoosable else resolution["state"]
|
||||
elif resolution["state"] == "ambiguous":
|
||||
return "ambiguous"
|
||||
return "unresolved" if unchoosable else "ambiguous"
|
||||
elif resolution["state"] == "unresolved" and not isinstance(
|
||||
canonical.get("parameters"), str
|
||||
):
|
||||
|
||||
+35
-13
@@ -13,6 +13,7 @@ import datetime
|
||||
import functools
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
from aiohttp import web
|
||||
@@ -73,6 +74,9 @@ _HEX64 = re.compile(r"[0-9a-f]{64}")
|
||||
_UUID = re.compile(r"[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}")
|
||||
# secrets.token_urlsafe(32) is 43 characters; the bound only rejects junk.
|
||||
_TOKEN = re.compile(r"[A-Za-z0-9_-]{1,128}")
|
||||
# The store is created on first use by the event loop, the scan's executor or
|
||||
# the monitor's worker (Needs review), so its creation is serialized.
|
||||
_CAPTURE_STORE_LOCK = threading.Lock()
|
||||
|
||||
_PREVIEW_TYPES = {
|
||||
".png": "image/png",
|
||||
@@ -259,31 +263,46 @@ class AnnotationRoutesMixin:
|
||||
Created lazily so an API object that never observes a file (most tests,
|
||||
a server that never opens the Tag dialog) owns no temp directory. A
|
||||
restore of the database dooms every capture (``invalidate_all``).
|
||||
Thread-safe: Needs review recording calls it off the event loop.
|
||||
Raises ``CaptureError`` after ``close_annotation_captures``.
|
||||
"""
|
||||
if self._captures is None:
|
||||
self._captures = CaptureStore()
|
||||
if self._captures_unsubscribe is None:
|
||||
self._captures_unsubscribe = subscribe_history_restore(
|
||||
self.db.model.db_path, self._invalidate_captures
|
||||
)
|
||||
return self._captures
|
||||
with _CAPTURE_STORE_LOCK:
|
||||
if getattr(self, "_captures_closed", False):
|
||||
# A recording still in flight at shutdown must not create a
|
||||
# new temp directory after the cleanup removed the old one.
|
||||
raise CaptureError("The capture store is closed")
|
||||
if self._captures is None:
|
||||
self._captures = CaptureStore()
|
||||
if self._captures_unsubscribe is None:
|
||||
self._captures_unsubscribe = subscribe_history_restore(
|
||||
self.db.model.db_path, self._invalidate_captures
|
||||
)
|
||||
return self._captures
|
||||
|
||||
def _invalidate_captures(self) -> None:
|
||||
store = self._captures
|
||||
if store is not None:
|
||||
store.invalidate_all()
|
||||
|
||||
def close_annotation_captures(self) -> None:
|
||||
"""Remove every capture file and the owned directory. Idempotent."""
|
||||
unsubscribe, self._captures_unsubscribe = self._captures_unsubscribe, None
|
||||
def close_annotation_captures(self, *, final: bool = False) -> None:
|
||||
"""Remove every capture file and the owned directory. Idempotent.
|
||||
|
||||
Taken under the store lock. ``final`` (the shutdown cleanup) also makes
|
||||
``_capture_store`` refuse to create another store, so a recording
|
||||
still in flight cannot leak a new temp directory after it.
|
||||
"""
|
||||
with _CAPTURE_STORE_LOCK:
|
||||
if final:
|
||||
self._captures_closed = True
|
||||
unsubscribe, self._captures_unsubscribe = self._captures_unsubscribe, None
|
||||
store, self._captures = self._captures, None
|
||||
if unsubscribe is not None:
|
||||
unsubscribe()
|
||||
store, self._captures = self._captures, None
|
||||
if store is not None:
|
||||
store.close()
|
||||
|
||||
async def cleanup_annotation_captures(self, app: web.Application) -> None:
|
||||
self.close_annotation_captures()
|
||||
self.close_annotation_captures(final=True)
|
||||
|
||||
# -- shared request steps --------------------------------------------------
|
||||
|
||||
@@ -329,7 +348,10 @@ class AnnotationRoutesMixin:
|
||||
root, error = self._scoped_root(index, scope)
|
||||
if error is not None:
|
||||
return error
|
||||
store = self._capture_store()
|
||||
try:
|
||||
store = self._capture_store()
|
||||
except CaptureError as exc: # the server is shutting down
|
||||
return _capture_error(exc)
|
||||
try:
|
||||
epoch = await self._run_in_executor(self.db.get_execution_epoch)
|
||||
capture = await _settled(
|
||||
|
||||
+9
-21
@@ -3,6 +3,7 @@
|
||||
import asyncio
|
||||
import base64
|
||||
import binascii
|
||||
import functools
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
@@ -37,6 +38,7 @@ try:
|
||||
from ...database.models import normalize_image_path
|
||||
from ...utils.card_seed import card_seed, header_params, parsed_positive
|
||||
from ...utils.parallel import map_parallel
|
||||
from ...utils.json_strict import dumps_strict
|
||||
from ...utils.inspection_response import (
|
||||
encode_inspection_response,
|
||||
library_metadata,
|
||||
@@ -56,6 +58,7 @@ except ImportError:
|
||||
from database.models import normalize_image_path
|
||||
from utils.card_seed import card_seed, header_params, parsed_positive
|
||||
from utils.parallel import map_parallel
|
||||
from utils.json_strict import dumps_strict
|
||||
from utils.inspection_response import (
|
||||
encode_inspection_response,
|
||||
library_metadata,
|
||||
@@ -982,27 +985,12 @@ class ImageRoutesMixin:
|
||||
# Add url and thumbnail_url so the frontend can serve them
|
||||
cleaned_images = self._present_images(cleaned_images)
|
||||
|
||||
# Additional fallback: convert to JSON string and clean NaN values manually
|
||||
try:
|
||||
response_data = {"success": True, "images": cleaned_images}
|
||||
# Convert to JSON string
|
||||
json_str = json.dumps(response_data, default=str)
|
||||
|
||||
# Clean any remaining NaN values with regex
|
||||
json_str = re.sub(r":\s*NaN", ": null", json_str)
|
||||
json_str = re.sub(r"\[\s*NaN\s*\]", "[null]", json_str)
|
||||
json_str = re.sub(r",\s*NaN\s*,", ", null,", json_str)
|
||||
json_str = re.sub(r",\s*NaN\s*\]", ", null]", json_str)
|
||||
json_str = re.sub(r"\[\s*NaN\s*,", "[null,", json_str)
|
||||
|
||||
# Parse back to verify it's valid JSON
|
||||
cleaned_data = json.loads(json_str)
|
||||
|
||||
return web.json_response(cleaned_data)
|
||||
except Exception as json_error:
|
||||
self.logger.error(f"JSON cleaning error: {json_error}")
|
||||
# Fallback to original response
|
||||
return web.json_response({"success": True, "images": cleaned_images})
|
||||
# Rows an older version stored may still hold NaN/Infinity: answer
|
||||
# strict JSON, and leave prompt text that spells "NaN" untouched.
|
||||
return web.json_response(
|
||||
{"success": True, "images": cleaned_images},
|
||||
dumps=functools.partial(dumps_strict, default=str),
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Get prompt images error: {e}")
|
||||
|
||||
+93
-17
@@ -18,29 +18,74 @@ def _not_found():
|
||||
return web.json_response({"success": False, "error": "Job not found"}, status=404)
|
||||
|
||||
|
||||
def _bad_request(message):
|
||||
return web.json_response({"success": False, "error": message}, status=400)
|
||||
MAX_SCAN_ROOTS = 64
|
||||
_SCOPE_ERROR = 'The scan scope must be "new" or "all"'
|
||||
_ROOTS_ERROR = (
|
||||
f"roots must be a non-empty list of up to {MAX_SCAN_ROOTS} source indexes, "
|
||||
"with paths naming the public path of each"
|
||||
)
|
||||
SOURCES_CHANGED_ERROR = "The sources changed since they were loaded. Reload them."
|
||||
|
||||
|
||||
async def _scan_scope(request):
|
||||
"""The requested scan scope ("new" or "all"), or None when the body is invalid.
|
||||
def _bad_request(message, code=None):
|
||||
body = {"success": False, "error": message}
|
||||
if code is not None:
|
||||
body["code"] = code
|
||||
return web.json_response(body, status=400)
|
||||
|
||||
An empty body (also a chunked one with no bytes), or one without
|
||||
``scope``, asks for the default.
|
||||
|
||||
def _scan_roots(body):
|
||||
"""``(roots, error)``: ``{index: public path}`` of the selected sources, or None.
|
||||
|
||||
Absent ``roots`` means every source. When present it is a non-empty list
|
||||
of at most MAX_SCAN_ROOTS non-negative ints (never bools or floats), and
|
||||
``paths`` the same-length list of the public path the caller read for
|
||||
each (GET /settings), so a stale index never scans another folder.
|
||||
"""
|
||||
if "roots" not in body:
|
||||
return None, None
|
||||
raw = body["roots"]
|
||||
paths = body.get("paths")
|
||||
if not isinstance(raw, list) or not raw or len(raw) > MAX_SCAN_ROOTS:
|
||||
return None, _ROOTS_ERROR
|
||||
if not isinstance(paths, list) or len(paths) != len(raw):
|
||||
return None, _ROOTS_ERROR
|
||||
selected = {}
|
||||
for value, path in zip(raw, paths):
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
|
||||
return None, _ROOTS_ERROR
|
||||
if not isinstance(path, str) or selected.get(value, path) != path:
|
||||
return None, _ROOTS_ERROR
|
||||
selected[value] = path
|
||||
return dict(sorted(selected.items())), None
|
||||
|
||||
|
||||
async def _scan_request(request):
|
||||
"""``({files, roots}, None)`` for a scan start, or ``(None, message)``.
|
||||
|
||||
``files`` is the scope ("new" or "all"); ``roots`` the selected source
|
||||
indexes or None for every source. An empty body (also a chunked one with
|
||||
no bytes), or one without ``scope``, asks for the default scope.
|
||||
"""
|
||||
default = {"files": DEFAULT_SCAN_SCOPE, "roots": None}
|
||||
if not request.can_read_body:
|
||||
return DEFAULT_SCAN_SCOPE
|
||||
return default, None
|
||||
text = await request.text()
|
||||
if not text.strip(): # e.g. a chunked POST that sent no bytes
|
||||
return DEFAULT_SCAN_SCOPE
|
||||
return default, None
|
||||
try:
|
||||
body = json.loads(text)
|
||||
except ValueError:
|
||||
return None
|
||||
return None, _SCOPE_ERROR
|
||||
if not isinstance(body, dict):
|
||||
return None
|
||||
scope = body.get("scope", DEFAULT_SCAN_SCOPE)
|
||||
return scope if scope in SCAN_SCOPES else None
|
||||
return None, _SCOPE_ERROR
|
||||
files = body.get("scope", DEFAULT_SCAN_SCOPE)
|
||||
if files not in SCAN_SCOPES:
|
||||
return None, _SCOPE_ERROR
|
||||
roots, error = _scan_roots(body)
|
||||
if error:
|
||||
return None, error
|
||||
return {"files": files, "roots": roots}, None
|
||||
|
||||
|
||||
def busy_response(busy: JobBusy):
|
||||
@@ -64,6 +109,13 @@ def _summary_event(job):
|
||||
class JobRoutesMixin:
|
||||
"""Mixin providing the /prompt_manager/jobs endpoints."""
|
||||
|
||||
def _roots_still_listed(self, selected, output_dirs):
|
||||
"""Whether each ``{index: public path}`` still names that source."""
|
||||
return all(
|
||||
index < len(output_dirs) and self._public_path(output_dirs[index]) == path
|
||||
for index, path in selected.items()
|
||||
)
|
||||
|
||||
def _register_job_routes(self, routes):
|
||||
@routes.get("/prompt_manager/jobs")
|
||||
async def list_jobs_route(request):
|
||||
@@ -132,13 +184,37 @@ class JobRoutesMixin:
|
||||
"""Start the metadata scan as a job (202), or 409 naming the busy job.
|
||||
|
||||
The body's ``scope`` is "new" (the default: new and unresolved files)
|
||||
or "all"; anything else is a 400.
|
||||
or "all". ``roots``, when given, selects sources by their index in
|
||||
the effective sources (``_get_all_output_dirs``), and ``paths`` names
|
||||
the public path read for each. An index past them, or one whose
|
||||
source now has another public path (the list changed since it was
|
||||
read), is a 409 ``sources-changed`` and starts nothing. The job walks
|
||||
the sources resolved here, so a later source change cannot move it.
|
||||
"""
|
||||
files = await _scan_scope(request)
|
||||
if files is None:
|
||||
return _bad_request('The scan scope must be "new" or "all"')
|
||||
scan, error = await _scan_request(request)
|
||||
if error:
|
||||
return _bad_request(error)
|
||||
output_dirs = None
|
||||
roots = None
|
||||
if scan["roots"] is not None:
|
||||
output_dirs = await self._run_in_executor(self._get_all_output_dirs)
|
||||
listed = await self._run_in_executor(
|
||||
self._roots_still_listed, scan["roots"], output_dirs
|
||||
)
|
||||
if not listed:
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"code": "sources-changed",
|
||||
"error": SOURCES_CHANGED_ERROR,
|
||||
},
|
||||
status=409,
|
||||
)
|
||||
roots = list(scan["roots"])
|
||||
try:
|
||||
job = self._start_scan_job(files)
|
||||
job = self._start_scan_job(
|
||||
scan["files"], roots=roots, output_dirs=output_dirs
|
||||
)
|
||||
except JobBusy as busy:
|
||||
return busy_response(busy)
|
||||
return web.json_response({"success": True, "job": job.summary()}, status=202)
|
||||
|
||||
+7
-3
@@ -88,6 +88,9 @@ DUPLICATE_CLEANUP_JOB_KIND = "duplicate_cleanup"
|
||||
THUMBNAILS_JOB_KIND = "thumbnails"
|
||||
LORA_IMPORT_JOB_KIND = "lora_import"
|
||||
AUTOTAG_BATCH_JOB_KIND = "autotag"
|
||||
# The System health check (py/api/system.py): a read-only ``read`` job with
|
||||
# no file outcomes, so it may run beside a scan.
|
||||
HEALTH_JOB_KIND = "health"
|
||||
|
||||
|
||||
def resolve_admission(admission=None, record_changing=None) -> str:
|
||||
@@ -432,7 +435,8 @@ class JobRegistry:
|
||||
if same_kind is not None:
|
||||
return same_kind
|
||||
refused = _REFUSED_BESIDE[resolve_admission(admission, record_changing)]
|
||||
running = [job for job in self._jobs.values() if job.running]
|
||||
# A snapshot: the monitor's worker thread asks too (Needs review).
|
||||
running = [job for job in tuple(self._jobs.values()) if job.running]
|
||||
running.sort(key=lambda job: job._started_seq)
|
||||
for job in running:
|
||||
if job.admission in refused:
|
||||
@@ -463,8 +467,8 @@ class JobRegistry:
|
||||
return self._jobs.get(job_id)
|
||||
|
||||
def active(self, kind):
|
||||
"""The running job of ``kind``, or None."""
|
||||
for job in self._jobs.values():
|
||||
"""The running job of ``kind``, or None (safe off the event loop)."""
|
||||
for job in tuple(self._jobs.values()):
|
||||
if job.kind == kind and job.running:
|
||||
return job
|
||||
return None
|
||||
|
||||
@@ -0,0 +1,689 @@
|
||||
"""Needs review routes (#249 E3): the pending queue, item detail, decisions.
|
||||
|
||||
``GET /prompt_manager/review/items`` lists pending items oldest first behind
|
||||
an HMAC keyset cursor bound to the storage epoch and the review index
|
||||
revision (any decision, new item or restore makes an old cursor stale);
|
||||
``total`` is the exact pending count. ``GET .../items/{id}`` adds the
|
||||
candidates and the dependency fingerprint. ``POST .../items/{id}/decision``
|
||||
records one explicit decision: Apply links the chosen sampler's pair only
|
||||
when the capture is the item's bytes, the current file still hashes to them,
|
||||
the evidence they yield is unchanged and the ingester's compatible-only rules
|
||||
allow it; Leave unresolved links nothing.
|
||||
|
||||
Targets are formatted from the recorded spelling against the current roots
|
||||
(no file access beyond the roots' real paths); a file under no current root
|
||||
is ``unavailable``. Responses carry no server path, no locator and no
|
||||
exception text.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from .annotations import _HEX64, _TOKEN, _UUID, _capture_error, _error
|
||||
from .prompts import bad_request
|
||||
|
||||
try:
|
||||
from ...database.library import display_title
|
||||
from ...database.library_commands import LibraryCommandError
|
||||
from ...database.review_decisions import ReviewCommandError
|
||||
from ...utils.media_capture import CaptureError
|
||||
from ...utils.review_capture import captured_review, review_evidence
|
||||
except ImportError:
|
||||
from database.library import display_title
|
||||
from database.library_commands import LibraryCommandError
|
||||
from database.review_decisions import ReviewCommandError
|
||||
from utils.media_capture import CaptureError
|
||||
from utils.review_capture import captured_review, review_evidence
|
||||
|
||||
ITEMS_URL = "/prompt_manager/review/items"
|
||||
DEFAULT_LIMIT = 40
|
||||
MAX_LIMIT = 40
|
||||
MAX_CURSOR = 4096
|
||||
MAX_BODY_BYTES = 64 * 1024
|
||||
# The serialized detail stays under this; past it the detail is ``limited``.
|
||||
MAX_DETAIL_BYTES = 8 * 1024 * 1024
|
||||
UNRESOLVED = "unresolved"
|
||||
_DECISION_FIELDS = frozenset(
|
||||
{
|
||||
"expected_epoch",
|
||||
"expected_revision",
|
||||
"capture_token",
|
||||
"sha256",
|
||||
"dependency_fingerprint",
|
||||
"choice",
|
||||
}
|
||||
)
|
||||
_CURSOR_SECRET = os.urandom(32)
|
||||
# The real path of a spelling; module-level so a test can fake a junction.
|
||||
_realpath = os.path.realpath
|
||||
_LIMIT = re.compile(r"[0-9]{1,3}")
|
||||
_NO_STORE = {"Cache-Control": "no-store"}
|
||||
|
||||
_STATUS = {
|
||||
"review-missing": 404,
|
||||
"review-conflict": 409,
|
||||
"association-conflict": 409,
|
||||
"file-changed": 409,
|
||||
"storage-changed": 409,
|
||||
"source-changed": 409,
|
||||
"capture-expired": 409,
|
||||
}
|
||||
_MESSAGES = {
|
||||
"review-missing": "This review item no longer exists. Reload the queue.",
|
||||
"review-conflict": (
|
||||
"This item was decided or changed since it was opened. Reload the queue."
|
||||
),
|
||||
"association-conflict": (
|
||||
"This image is already linked to another prompt or negative, so this "
|
||||
"sampler path cannot be attached safely. The existing links were "
|
||||
"preserved."
|
||||
),
|
||||
"dependencies-changed": (
|
||||
"The image's saved links changed since this item was opened. Reopen it "
|
||||
"to see the current association."
|
||||
),
|
||||
"file-changed": (
|
||||
"The file changed since it was reviewed. A metadata scan adds its new "
|
||||
"bytes as a new item."
|
||||
),
|
||||
"storage-changed": "Prompt storage was restored; reload the review queue.",
|
||||
"source-changed": (
|
||||
"The source changed. Refresh the image collection before reviewing this "
|
||||
"file."
|
||||
),
|
||||
"capture-expired": "The preview expired. Open the item again.",
|
||||
"other-file": "The preview belongs to another file. Open the item again.",
|
||||
"evidence-changed": (
|
||||
"This file's metadata no longer gives the evidence that was reviewed. "
|
||||
"Reload the queue."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
_DEFAULT_MESSAGE = "The decision could not be saved. Reload the queue."
|
||||
|
||||
|
||||
def _coded(code: str, message_key: str | None = None, **extra) -> web.Response:
|
||||
message = _MESSAGES.get(message_key or code, _DEFAULT_MESSAGE)
|
||||
return _error(_STATUS.get(code, 409), code, message, **extra)
|
||||
|
||||
|
||||
def _failed() -> web.Response:
|
||||
return _error(500, "review-failed", "The review queue could not be read.")
|
||||
|
||||
|
||||
def format_review_target(spelled_path, locator, real_roots, *, pathmod=os.path):
|
||||
"""``(root_index, relative)`` of the first root that lists the file, or None.
|
||||
|
||||
``spelled_path`` is the file's recorded spelling, ``locator`` its folded
|
||||
identity and ``real_roots`` ``(index, realpath)`` pairs. The relative
|
||||
path keeps the recorded spelling (a Windows locator is case-folded, a
|
||||
listing target is not) and must fold back to the locator.
|
||||
"""
|
||||
sep = pathmod.sep
|
||||
for index, real_root in real_roots:
|
||||
head = spelled_path[: len(real_root)]
|
||||
if pathmod.normcase(head) != pathmod.normcase(real_root):
|
||||
continue
|
||||
rest = spelled_path[len(real_root) :]
|
||||
if not real_root.endswith(sep):
|
||||
if not rest.startswith(sep):
|
||||
continue
|
||||
rest = rest[1:]
|
||||
parts = rest.split(sep)
|
||||
if any(
|
||||
part in ("", ".", "..") or "/" in part or "\\" in part or ":" in part
|
||||
for part in parts
|
||||
):
|
||||
continue
|
||||
if pathmod.normcase(pathmod.join(real_root, *parts)) != locator:
|
||||
continue
|
||||
return index, "/".join(parts)
|
||||
return None
|
||||
|
||||
|
||||
class _StaleCursor(ValueError):
|
||||
"""A well-formed cursor this process cannot verify (e.g. after a restart)."""
|
||||
|
||||
|
||||
def _encode_cursor(epoch: str, revision: int, key: tuple) -> str:
|
||||
payload = json.dumps(
|
||||
{"v": 1, "e": epoch, "r": revision, "k": list(key)},
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=True,
|
||||
).encode()
|
||||
signature = hmac.new(_CURSOR_SECRET, payload, hashlib.sha256).digest()
|
||||
return base64.urlsafe_b64encode(signature + payload).decode().rstrip("=")
|
||||
|
||||
|
||||
def _decode_cursor(token: str) -> tuple:
|
||||
"""``(epoch, revision, (created_at, id))`` of a cursor.
|
||||
|
||||
``ValueError`` when malformed; :class:`_StaleCursor` when well-formed but
|
||||
signed by another secret (the per-process secret changes on every start,
|
||||
so an open page's cursor is stale after a restart, not forged).
|
||||
"""
|
||||
try:
|
||||
if len(token) > MAX_CURSOR:
|
||||
raise ValueError()
|
||||
raw = base64.b64decode(
|
||||
token + "=" * (-len(token) % 4), altchars=b"-_", validate=True
|
||||
)
|
||||
signature, payload = raw[:32], raw[32:]
|
||||
if len(signature) != 32 or not payload:
|
||||
raise ValueError()
|
||||
expected = hmac.new(_CURSOR_SECRET, payload, hashlib.sha256).digest()
|
||||
if not hmac.compare_digest(signature, expected):
|
||||
raise _StaleCursor("This page of the queue is out of date")
|
||||
data = json.loads(payload)
|
||||
key = data["k"]
|
||||
if (
|
||||
data["v"] != 1
|
||||
or not isinstance(data["e"], str)
|
||||
or type(data["r"]) is not int
|
||||
or not isinstance(key, list)
|
||||
or len(key) != 2
|
||||
or not all(isinstance(part, str) for part in key)
|
||||
):
|
||||
raise ValueError()
|
||||
return data["e"], data["r"], tuple(key)
|
||||
except _StaleCursor:
|
||||
raise
|
||||
except (ValueError, KeyError, TypeError, UnicodeError, binascii.Error):
|
||||
raise ValueError("Invalid review cursor; reload the queue") from None
|
||||
|
||||
|
||||
def _limit(value):
|
||||
if value is None:
|
||||
return DEFAULT_LIMIT
|
||||
if _LIMIT.fullmatch(value) is None or not 1 <= int(value) <= MAX_LIMIT:
|
||||
return None
|
||||
return int(value)
|
||||
|
||||
|
||||
def _side_text(side) -> str | None:
|
||||
if isinstance(side, dict) and isinstance(side.get("text"), str):
|
||||
return side["text"]
|
||||
return None
|
||||
|
||||
|
||||
def _workable(candidate: dict) -> bool:
|
||||
return (
|
||||
candidate.get("state") == "resolved"
|
||||
and _side_text(candidate.get("positive")) is not None
|
||||
and _side_text(candidate.get("negative")) is not None
|
||||
)
|
||||
|
||||
|
||||
def _side_label(titles: dict, side, empty: str) -> str:
|
||||
"""The Library title of the side's saved prompt, else its first line."""
|
||||
text = _side_text(side)
|
||||
if text is None:
|
||||
return empty
|
||||
stored = side.get("storage_text")
|
||||
stored = stored if isinstance(stored, str) else text
|
||||
return display_title(titles.get(stored, ()), text) or empty
|
||||
|
||||
|
||||
class ReviewRoutesMixin:
|
||||
"""The Needs review queue over E1's schema and E2's recorded items."""
|
||||
|
||||
def _register_review_routes(self, routes: web.RouteTableDef) -> None:
|
||||
routes.get(ITEMS_URL)(self.list_review_items)
|
||||
routes.get(ITEMS_URL + "/{item_id}")(self.get_review_item)
|
||||
routes.post(ITEMS_URL + "/{item_id}/decision")(self.decide_review_item)
|
||||
|
||||
# -- shared ----------------------------------------------------------------
|
||||
|
||||
def _review_roots(self):
|
||||
"""``(roots, real_roots)``: the current gallery roots and their realpaths.
|
||||
|
||||
Blocking (it lists the roots and resolves them): call it on a worker.
|
||||
"""
|
||||
roots = list(self._get_all_output_dirs())
|
||||
return roots, [(i, _realpath(str(root))) for i, root in enumerate(roots)]
|
||||
|
||||
def _review_view(self, row: dict, roots, real_roots) -> dict:
|
||||
"""The public ReviewItem of one stored row (no path, no locator)."""
|
||||
found = format_review_target(row["spelled_path"], row["locator"], real_roots)
|
||||
target = None
|
||||
if found is not None:
|
||||
index, relative = found
|
||||
target = {
|
||||
"root_index": index,
|
||||
"root_scope": self._gallery_root_scope(roots[index], index),
|
||||
"path": relative,
|
||||
}
|
||||
return {
|
||||
"id": row["id"],
|
||||
"state": row["state"],
|
||||
"revision": row["revision"],
|
||||
"created_at": row["created_at"],
|
||||
"filename": os.path.basename(row["spelled_path"]),
|
||||
"target": target,
|
||||
"unavailable": target is None,
|
||||
"candidate_count": row["candidate_count"],
|
||||
"limited": row["limited"],
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _review_image_path(row: dict, roots, target) -> str:
|
||||
"""The path the scan and the monitor link this file under."""
|
||||
if target is None:
|
||||
return row["spelled_path"]
|
||||
root = roots[target["root_index"]]
|
||||
return os.path.join(str(root), *target["path"].split("/"))
|
||||
|
||||
# -- queue -----------------------------------------------------------------
|
||||
|
||||
async def list_review_items(self, request: web.Request) -> web.Response:
|
||||
"""GET ``?limit=1..40&cursor=``: one page of pending items."""
|
||||
limit = _limit(request.query.get("limit"))
|
||||
if limit is None:
|
||||
return bad_request(f"limit must be an integer from 1 to {MAX_LIMIT}")
|
||||
bound = None
|
||||
cursor = request.query.get("cursor")
|
||||
if cursor:
|
||||
try:
|
||||
bound = _decode_cursor(cursor)
|
||||
except _StaleCursor:
|
||||
return await self._stale_cursor()
|
||||
except ValueError as exc:
|
||||
return bad_request(str(exc))
|
||||
try:
|
||||
page, views = await self._run_in_executor(
|
||||
self._review_page, limit, bound[2] if bound else None
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("Review queue read failed")
|
||||
return _failed()
|
||||
if bound is not None and bound[0] != page["epoch"]:
|
||||
return _coded("storage-changed", library_epoch=page["epoch"])
|
||||
if bound is not None and bound[1] != page["revision"]:
|
||||
return _coded("review-conflict", total=page["total"])
|
||||
items = page["items"]
|
||||
next_cursor = None
|
||||
if page["more"] and items:
|
||||
last = items[-1]
|
||||
next_cursor = _encode_cursor(
|
||||
page["epoch"], page["revision"], (last["created_at"], last["id"])
|
||||
)
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
"library_epoch": page["epoch"],
|
||||
"total": page["total"],
|
||||
"items": views,
|
||||
"next_cursor": next_cursor,
|
||||
},
|
||||
headers=_NO_STORE,
|
||||
)
|
||||
|
||||
async def _stale_cursor(self) -> web.Response:
|
||||
"""409 ``review-conflict`` with the current total: reload the queue."""
|
||||
try:
|
||||
page = await self._run_in_executor(self.db.review_queue_page, limit=1)
|
||||
except Exception:
|
||||
self.logger.exception("Review queue read failed")
|
||||
return _failed()
|
||||
return _coded(
|
||||
"review-conflict", total=page["total"], library_epoch=page["epoch"]
|
||||
)
|
||||
|
||||
def _review_page(self, limit: int, after):
|
||||
"""Worker thread: one queue page and the public views of its items."""
|
||||
page = self.db.review_queue_page(limit=limit, after=after)
|
||||
roots, real_roots = self._review_roots()
|
||||
return page, [
|
||||
self._review_view(row, roots, real_roots) for row in page["items"]
|
||||
]
|
||||
|
||||
# -- detail ----------------------------------------------------------------
|
||||
|
||||
async def get_review_item(self, request: web.Request) -> web.Response:
|
||||
"""GET one item with its candidates and dependency fingerprint."""
|
||||
try:
|
||||
read = await self._run_in_executor(
|
||||
self._review_detail, request.match_info["item_id"]
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("Review item read failed")
|
||||
return _failed()
|
||||
if read is None:
|
||||
return _coded("review-missing")
|
||||
item, view, fingerprint, candidates = read
|
||||
detail = {
|
||||
"success": True,
|
||||
"library_epoch": item["epoch"],
|
||||
**view,
|
||||
"candidates": candidates,
|
||||
"dependency_fingerprint": fingerprint,
|
||||
"sha256": item["sha256"],
|
||||
"total": item["total"],
|
||||
}
|
||||
if len(json.dumps(detail).encode("utf-8")) > MAX_DETAIL_BYTES:
|
||||
detail = {**detail, "candidates": [], "limited": True}
|
||||
return web.json_response(detail, headers=_NO_STORE)
|
||||
|
||||
def _review_detail(self, item_id: str):
|
||||
"""Worker thread: ``(item, view, fingerprint, candidates)`` or None."""
|
||||
item = self.db.review_item(item_id)
|
||||
if item is None:
|
||||
return None
|
||||
roots, real_roots = self._review_roots()
|
||||
view = self._review_view(item, roots, real_roots)
|
||||
image_path = self._review_image_path(item, roots, view["target"])
|
||||
aliases, _known = physical_spellings(self.db, image_path, item["locator"])
|
||||
fingerprint = self.db.review_dependency_fingerprint(image_path, aliases)
|
||||
return item, view, fingerprint, self._review_candidates(item)
|
||||
|
||||
def _review_candidates(self, item: dict) -> list:
|
||||
"""The detail's candidates: role texts, a label and selectability."""
|
||||
if item["limited"]:
|
||||
return []
|
||||
candidates = [
|
||||
c for c in item["evidence"].get("candidates", []) if isinstance(c, dict)
|
||||
]
|
||||
texts = set()
|
||||
for candidate in candidates:
|
||||
for role in ("positive", "negative"):
|
||||
side = candidate.get(role)
|
||||
if isinstance(side, dict):
|
||||
for key in ("text", "storage_text"):
|
||||
if isinstance(side.get(key), str):
|
||||
texts.add(side[key])
|
||||
titles = self.db.stored_prompt_titles(texts)
|
||||
rows = []
|
||||
for candidate in candidates:
|
||||
positive = _side_text(candidate.get("positive"))
|
||||
negative = _side_text(candidate.get("negative"))
|
||||
label = (
|
||||
_side_label(titles, candidate.get("positive"), "No positive")
|
||||
+ " / "
|
||||
+ _side_label(titles, candidate.get("negative"), "No negative")
|
||||
)
|
||||
rows.append(
|
||||
{
|
||||
"hash": candidate.get("hash"),
|
||||
"sampler_id": candidate.get("sampler_id"),
|
||||
"state": "resolved" if _workable(candidate) else "unresolved",
|
||||
"reason": candidate.get("reason") or "",
|
||||
"label": label,
|
||||
"positive": {"text": positive} if positive is not None else None,
|
||||
"negative": {"text": negative} if negative is not None else None,
|
||||
}
|
||||
)
|
||||
return rows
|
||||
|
||||
# -- decision --------------------------------------------------------------
|
||||
|
||||
async def decide_review_item(self, request: web.Request) -> web.Response:
|
||||
"""POST one explicit decision on one pending item."""
|
||||
data, error = await self._read_json_body(request, max_bytes=MAX_BODY_BYTES)
|
||||
if error is not None:
|
||||
return error
|
||||
guards, error = _decision_guards(data)
|
||||
if error is not None:
|
||||
return error
|
||||
try:
|
||||
item = await self._run_in_executor(
|
||||
self.db.review_item, request.match_info["item_id"]
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("Review item read failed")
|
||||
return _failed()
|
||||
if item is None:
|
||||
return _coded("review-missing")
|
||||
choice = guards["choice"]
|
||||
candidate = None
|
||||
if choice != UNRESOLVED:
|
||||
candidate = next(
|
||||
(
|
||||
c
|
||||
for c in item["evidence"].get("candidates", [])
|
||||
if isinstance(c, dict) and c.get("hash") == choice
|
||||
),
|
||||
None,
|
||||
)
|
||||
if candidate is None or item["limited"]:
|
||||
return bad_request("choice is not one of this item's candidates")
|
||||
if not _workable(candidate):
|
||||
return bad_request("This sampler path cannot be applied")
|
||||
if guards["capture_token"] is None:
|
||||
return bad_request("Apply needs a capture of the file")
|
||||
verified = None
|
||||
if guards["capture_token"] is not None:
|
||||
verified, error = await self._verified_review_capture(
|
||||
item, guards, need_metadata=candidate is not None
|
||||
)
|
||||
if error is not None:
|
||||
return error
|
||||
elif guards["sha256"] != item["sha256"]:
|
||||
return bad_request("sha256 does not match the item")
|
||||
return await self._record_decision(item, guards, candidate, verified)
|
||||
|
||||
async def _verified_review_capture(self, item, guards, *, need_metadata):
|
||||
"""``((capture, facts, image_path, metadata), None)`` or ``(None, error)``.
|
||||
|
||||
The capture must be the item's file and bytes, under the expected
|
||||
epoch and a current root; the current file must re-hash to the same
|
||||
bytes; for Apply, the capture's metadata must still yield exactly the
|
||||
reviewed evidence.
|
||||
"""
|
||||
store = self._captures
|
||||
capture = store.get(guards["capture_token"]) if store is not None else None
|
||||
if capture is None:
|
||||
return None, _coded("capture-expired")
|
||||
if capture.locator != item["locator"]:
|
||||
return None, _coded("capture-expired", "other-file")
|
||||
if guards["sha256"] != capture.sha256:
|
||||
return None, bad_request("sha256 does not match the preview")
|
||||
if capture.sha256 != item["sha256"]:
|
||||
return None, _coded("file-changed")
|
||||
if guards["expected_epoch"] != capture.epoch:
|
||||
current = await self._run_in_executor(self.db.get_execution_epoch)
|
||||
return None, _coded("storage-changed", library_epoch=current)
|
||||
try:
|
||||
root, error = await self._run_in_executor(
|
||||
self._scoped_root, capture.root_index, capture.scope
|
||||
)
|
||||
if error is not None:
|
||||
return None, error
|
||||
read = await self._run_in_executor(
|
||||
_verify_and_read, store, capture, item, root, need_metadata
|
||||
)
|
||||
except CaptureError as exc:
|
||||
return None, _capture_error(exc)
|
||||
if read is None:
|
||||
return None, _coded("source-changed")
|
||||
relative, facts, captured = read
|
||||
if facts.locator != item["locator"]:
|
||||
return None, _coded("capture-expired", "other-file")
|
||||
if facts.sha256 != item["sha256"]:
|
||||
return None, _coded("file-changed")
|
||||
metadata = None
|
||||
if need_metadata:
|
||||
if captured is None or not _candidate_still_offered(
|
||||
captured[1], guards["choice"]
|
||||
):
|
||||
return None, _coded("review-conflict", "evidence-changed")
|
||||
metadata = captured[0]
|
||||
image_path = os.path.join(str(root), *relative.split("/"))
|
||||
return (capture, facts, image_path, metadata), None
|
||||
|
||||
def _decide_sync(self, item, guards, candidate, verified):
|
||||
"""Worker thread: resolve the file's spellings, decide, build the view.
|
||||
|
||||
Returns ``(result, view)``; a refusal raises with ``exc.view`` set.
|
||||
"""
|
||||
_capture, facts, image_path, metadata = verified or (None, None, None, None)
|
||||
aliases, known = (), None
|
||||
if candidate is not None:
|
||||
aliases, known = physical_spellings(self.db, image_path, item["locator"])
|
||||
try:
|
||||
result = self.db.decide_review_item(
|
||||
item_id=item["id"],
|
||||
expected_epoch=guards["expected_epoch"],
|
||||
expected_revision=guards["expected_revision"],
|
||||
# The bytes the re-hash proved, or the body's for Leave unresolved.
|
||||
sha256=facts.sha256 if facts is not None else guards["sha256"],
|
||||
outcome="selected" if candidate is not None else UNRESOLVED,
|
||||
candidate=candidate,
|
||||
image_path=image_path,
|
||||
metadata=metadata,
|
||||
expected_fingerprint=guards["dependency_fingerprint"],
|
||||
aliases=aliases,
|
||||
known_rows=known,
|
||||
)
|
||||
except ReviewCommandError as exc:
|
||||
if exc.item is not None:
|
||||
roots, real_roots = self._review_roots()
|
||||
exc.view = self._review_view(exc.item, roots, real_roots)
|
||||
raise
|
||||
roots, real_roots = self._review_roots()
|
||||
return result, self._review_view(result["item"], roots, real_roots)
|
||||
|
||||
async def _record_decision(self, item, guards, candidate, verified):
|
||||
capture = verified[0] if verified else None
|
||||
try:
|
||||
result, view = await self._run_in_executor(
|
||||
self._decide_sync, item, guards, candidate, verified
|
||||
)
|
||||
except ReviewCommandError as exc:
|
||||
return self._decision_refused(exc)
|
||||
except LibraryCommandError as exc:
|
||||
return _coded(exc.code, library_epoch=exc.epoch)
|
||||
except Exception:
|
||||
self.logger.exception("Review decision failed")
|
||||
return _error(500, "decision-failed", "The decision could not be saved.")
|
||||
if capture is not None:
|
||||
# The decision used the capture; the page returns to Images.
|
||||
self._captures.release(capture.token)
|
||||
body = {
|
||||
"success": True,
|
||||
"library_epoch": guards["expected_epoch"],
|
||||
"item": view,
|
||||
"total": result["total"],
|
||||
}
|
||||
if result["pair"] is not None:
|
||||
body["pair"] = result["pair"]
|
||||
return web.json_response(body, headers=_NO_STORE)
|
||||
|
||||
def _decision_refused(self, exc: ReviewCommandError) -> web.Response:
|
||||
extra = {"library_epoch": exc.epoch}
|
||||
if exc.total is not None:
|
||||
extra["total"] = exc.total
|
||||
if exc.view is not None:
|
||||
extra["item"] = exc.view
|
||||
message = None
|
||||
if exc.fingerprint is not None:
|
||||
extra["dependency_fingerprint"] = exc.fingerprint
|
||||
if exc.code == "association-conflict" and exc.changed:
|
||||
message = "dependencies-changed"
|
||||
return _coded(exc.code, message, **extra)
|
||||
|
||||
|
||||
def _decision_guards(data: dict):
|
||||
"""The validated decision fields, or ``(None, error)``."""
|
||||
if set(data) != _DECISION_FIELDS:
|
||||
return None, bad_request(
|
||||
"Request must contain exactly " + ", ".join(sorted(_DECISION_FIELDS))
|
||||
)
|
||||
epoch, revision = data["expected_epoch"], data["expected_revision"]
|
||||
if not isinstance(epoch, str) or _UUID.fullmatch(epoch) is None:
|
||||
return None, bad_request("expected_epoch must be a canonical UUID")
|
||||
if type(revision) is not int or revision < 0:
|
||||
return None, bad_request("expected_revision must be a non-negative integer")
|
||||
token = data["capture_token"]
|
||||
if token is not None and (
|
||||
not isinstance(token, str) or _TOKEN.fullmatch(token) is None
|
||||
):
|
||||
return None, bad_request("Invalid capture token")
|
||||
for name in ("sha256", "dependency_fingerprint"):
|
||||
value = data[name]
|
||||
if not isinstance(value, str) or _HEX64.fullmatch(value) is None:
|
||||
return None, bad_request(
|
||||
f"{name} must be 64 lowercase hexadecimal characters"
|
||||
)
|
||||
choice = data["choice"]
|
||||
if not isinstance(choice, str) or (
|
||||
choice != UNRESOLVED and _HEX64.fullmatch(choice) is None
|
||||
):
|
||||
return None, bad_request("choice must be a candidate hash or unresolved")
|
||||
return {**data}, None
|
||||
|
||||
|
||||
def _verify_and_read(store, capture, item, root, need_metadata: bool):
|
||||
"""Worker thread: ``(relative, facts, captured)``, or None off the root.
|
||||
|
||||
Formats the item's target under the capture's root (resolving the root's
|
||||
real path), re-hashes the current file, then reads the capture.
|
||||
"""
|
||||
found = format_review_target(
|
||||
item["spelled_path"],
|
||||
item["locator"],
|
||||
[(capture.root_index, _realpath(str(root)))],
|
||||
)
|
||||
if found is None:
|
||||
return None
|
||||
relative = found[1]
|
||||
facts = store.verify(root, relative)
|
||||
captured = captured_review(store, capture) if need_metadata else None
|
||||
return relative, facts, captured
|
||||
|
||||
|
||||
def _same_file(spelled: str, image_path: str, locator: str) -> bool:
|
||||
"""True when ``spelled`` reaches the physical file at ``locator``."""
|
||||
try:
|
||||
if os.path.normcase(_realpath(spelled)) == locator:
|
||||
return True
|
||||
return os.path.samefile(spelled, image_path)
|
||||
except (OSError, ValueError):
|
||||
return False
|
||||
|
||||
|
||||
def physical_spellings(db, image_path: str, locator: str):
|
||||
"""``(aliases, known_rows)`` for the legacy rows of one physical file.
|
||||
|
||||
The roots are real paths, while the monitor links a file under the
|
||||
directory it watches, so a symlinked or junctioned root, a ``subst`` or a
|
||||
mapped drive spells the same file otherwise. Every row whose spelling
|
||||
ends with the file's name (the link summaries' fallback lookup) is a
|
||||
candidate; ``aliases`` are the candidates that resolve to ``locator``
|
||||
(``normcase(realpath)``, as the capture forms it) or are the same file,
|
||||
and ``known_rows`` every candidate id, so a decision can refuse a
|
||||
same-name row it did not classify. Blocking: call it on a worker.
|
||||
"""
|
||||
rows = db.review_same_name_rows(os.path.basename(image_path))
|
||||
aliases = tuple(
|
||||
spelled
|
||||
for _row_id, spelled in rows
|
||||
if isinstance(spelled, str)
|
||||
and spelled != image_path
|
||||
and _same_file(spelled, image_path, locator)
|
||||
)
|
||||
return tuple(dict.fromkeys(aliases)), frozenset(row_id for row_id, _ in rows)
|
||||
|
||||
|
||||
def _candidate_still_offered(resolution: dict, choice: str) -> bool:
|
||||
"""True when the captured bytes still offer the chosen candidate.
|
||||
|
||||
Only what the decision depends on is re-checked: the bytes are still
|
||||
``ambiguous`` and the chosen candidate (its hash covers its sampler,
|
||||
sides and provenance) is still among their candidates. A resolver change
|
||||
that rewords the top-level reason or reorders candidates leaves a
|
||||
pending item applicable.
|
||||
"""
|
||||
if not isinstance(resolution, dict) or resolution.get("state") != "ambiguous":
|
||||
return False
|
||||
evidence, _count, limited = review_evidence(resolution)
|
||||
if limited:
|
||||
return False
|
||||
return any(c.get("hash") == choice for c in evidence.get("candidates", ()))
|
||||
@@ -0,0 +1,241 @@
|
||||
"""Needs review recording for the output scan and the live monitor (#249).
|
||||
|
||||
The rule (maintainer test of #284/#288): a file is queued only when its
|
||||
final outcome is ambiguous and unlinked. The strict resolver must find it
|
||||
``ambiguous``, the scan's legacy fallback (the browser parser,
|
||||
``_graph_prompt_text``) must not link it, and no saved row may link it
|
||||
already. A file the fallback links, an exact pair the ingester stored and an
|
||||
already linked file are never queued. Neither is an ambiguous file with
|
||||
nothing to choose: fewer than two of its candidates carry a prompt
|
||||
(``utils.review_capture.reviewable``); the scan reports it unresolved ("could
|
||||
not be traced").
|
||||
|
||||
The queue heals itself: a pending item whose file is now linked (by the
|
||||
fallback, a later resolution or the monitor) is removed by the next scan
|
||||
that lists the file or by the monitor's next pass over it, and so is a
|
||||
pending item without a choice whose file is still ambiguous without one
|
||||
(``settle_untraced_review_items``). Only an undecided item goes
|
||||
(``pending``, revision 0, no decision row); a decision is never deleted or
|
||||
changed (``settle_linked_review_items``).
|
||||
|
||||
The scan records through ``_record_scanned_review`` (best effort: a failure
|
||||
is logged and counted); the monitor calls ``record_review_file`` from its
|
||||
worker thread after its own link. Both go through
|
||||
``utils.review_capture.record_ambiguous_file`` on the API's capture store.
|
||||
Nothing here links a file.
|
||||
|
||||
Recording changes records, so it follows the registry's ``records``
|
||||
admission (#289). The scan records under its own ``records`` job. The
|
||||
monitor's pass lasts one file and is not a job: it is refused while a
|
||||
``records`` or ``guarded`` job runs (``deferred``), and the next scan reads
|
||||
the file again.
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from .admin import SCAN_LOG_PREFIXES, _plural, _scan_location, _scan_roots
|
||||
|
||||
# The kind the monitor's recording checks admission under (never registered).
|
||||
REVIEW_RECORDING_KIND = "review_recording"
|
||||
# record_ambiguous_file outcomes that leave the file in the pending queue.
|
||||
QUEUED_OUTCOMES = ("recorded", "limited", "duplicate")
|
||||
NEW_OUTCOMES = ("recorded", "limited")
|
||||
|
||||
try:
|
||||
from ...utils.review_capture import record_ambiguous_file
|
||||
except ImportError:
|
||||
from utils.review_capture import record_ambiguous_file
|
||||
|
||||
# The scan log's reason for items that leave the queue with nothing to choose.
|
||||
NOTHING_TO_CHOOSE = "nothing to choose"
|
||||
|
||||
|
||||
class ReviewRecordingMixin:
|
||||
"""Record the scan's and the monitor's ambiguous files for Needs review."""
|
||||
|
||||
def _record_scanned_review(self, media_file, roots, review=None):
|
||||
"""Record one ambiguous, unlinked scanned file (logged and counted)."""
|
||||
try:
|
||||
outcome = self._record_review_sync(media_file, roots, review)
|
||||
except Exception:
|
||||
self.logger.warning(
|
||||
f"{media_file.name} could not be added to Needs review",
|
||||
exc_info=True,
|
||||
)
|
||||
if review is not None:
|
||||
review["errors"] += 1
|
||||
return
|
||||
if review is None:
|
||||
return
|
||||
if outcome in NEW_OUTCOMES:
|
||||
review["added"] += 1
|
||||
if outcome in QUEUED_OUTCOMES:
|
||||
review["queued"] += 1
|
||||
|
||||
def _record_review_sync(self, media_file, roots, review=None):
|
||||
"""``record_ambiguous_file`` for a file under one of ``roots`` (blocking).
|
||||
|
||||
An ``untraced`` file (its captured bytes offer nothing to choose) also
|
||||
loses its pending items that offer nothing either.
|
||||
"""
|
||||
located = self._review_location(media_file, roots)
|
||||
if located is None:
|
||||
return "skipped"
|
||||
root, index, relative, _spelled = located
|
||||
outcome = record_ambiguous_file(
|
||||
self.db, self._capture_store(), Path(root), index, relative
|
||||
)
|
||||
if outcome == "untraced":
|
||||
self._settle_untraced_reviews([media_file], roots, review)
|
||||
return outcome
|
||||
|
||||
@staticmethod
|
||||
def _review_location(media_file, roots, real_roots=None):
|
||||
"""``(root, root_index, relative, spelled)`` of a listed file, or None.
|
||||
|
||||
``spelled`` is the capture's spelling of the file (the real root joined
|
||||
with the relative path); ``os.path.normcase`` of it is the locator
|
||||
review items are stored under. ``real_roots`` caches each root's
|
||||
``realpath`` across a batch.
|
||||
"""
|
||||
relative, _label, index = _scan_location(media_file, roots)
|
||||
root = next((r for r, _l, i in roots if i == index), None)
|
||||
if index is None or root is None:
|
||||
return None
|
||||
if real_roots is None:
|
||||
real_roots = {}
|
||||
real = real_roots.get(index)
|
||||
if real is None:
|
||||
real = real_roots[index] = os.path.realpath(os.fspath(root))
|
||||
spelled = os.path.join(real, *relative.split("/"))
|
||||
return root, index, relative, spelled
|
||||
|
||||
def _review_file_linked(self, media_file, roots):
|
||||
"""True when a saved row links the scanned file (any spelling)."""
|
||||
located = self._review_location(media_file, roots)
|
||||
spellings = (str(media_file),) + ((located[3],) if located else ())
|
||||
return self.db.review_file_linked(spellings)
|
||||
|
||||
def _settle_scanned_reviews(self, media_files, roots, review=None):
|
||||
"""Remove the pending items of ``media_files`` that are now linked.
|
||||
|
||||
Blocking; best effort like recording (a failure is logged and
|
||||
counted in ``review["errors"]``). Returns how many items left.
|
||||
"""
|
||||
real_roots = {}
|
||||
files = []
|
||||
for media_file in media_files:
|
||||
located = self._review_location(media_file, roots, real_roots)
|
||||
if located is not None:
|
||||
spelled = located[3]
|
||||
files.append((os.path.normcase(spelled), (str(media_file), spelled)))
|
||||
try:
|
||||
settled = self.db.settle_linked_review_items(files) if files else 0
|
||||
except Exception:
|
||||
self.logger.warning(
|
||||
"Needs review items of linked files could not be removed",
|
||||
exc_info=True,
|
||||
)
|
||||
if review is not None:
|
||||
review["errors"] += 1
|
||||
return 0
|
||||
if review is not None:
|
||||
review["settled"] += settled
|
||||
return settled
|
||||
|
||||
def _settle_untraced_reviews(self, media_files, roots, review=None):
|
||||
"""Remove the pending items without a choice of untraced ``media_files``.
|
||||
|
||||
``media_files`` are files found ambiguous with fewer than two
|
||||
candidates that carry a prompt. Blocking; best effort like the linked
|
||||
settle (a failure is logged and counted in ``review["errors"]``).
|
||||
Returns how many items left.
|
||||
"""
|
||||
locators = []
|
||||
for media_file in media_files:
|
||||
located = self._review_location(media_file, roots)
|
||||
if located is not None:
|
||||
locators.append(os.path.normcase(located[3]))
|
||||
try:
|
||||
removed = self.db.settle_untraced_review_items(locators) if locators else 0
|
||||
except Exception:
|
||||
self.logger.warning(
|
||||
"Needs review items without a choice could not be removed",
|
||||
exc_info=True,
|
||||
)
|
||||
if review is not None:
|
||||
review["errors"] += 1
|
||||
return 0
|
||||
if review is not None:
|
||||
review["untraced"] += removed
|
||||
return removed
|
||||
|
||||
def _fallback_links(self, metadata):
|
||||
"""True when the scan's legacy fallback would link these records."""
|
||||
return bool(metadata.get("prompt")) and bool(self._graph_prompt_text(metadata))
|
||||
|
||||
def record_review_file(self, image_path):
|
||||
"""Apply the Needs review rule to one monitor output (its worker thread).
|
||||
|
||||
Maps ``image_path`` to the gallery root that lists it; a file under no
|
||||
current root is ``skipped``. A linked file is never queued: its
|
||||
pending items are removed and the answer is ``linked``. An unlinked
|
||||
file the scan's fallback would link is ``skipped`` (the next scan
|
||||
links it); any other file is recorded when it is ambiguous with a
|
||||
choice, and is ``untraced`` (its pending items without a choice
|
||||
removed) when it is ambiguous without one. While a
|
||||
job runs that a ``records`` job may not run beside, nothing is written
|
||||
and the answer is ``deferred``. Errors propagate: the monitor logs
|
||||
them and keeps its link.
|
||||
"""
|
||||
busy = self._jobs.blocker(REVIEW_RECORDING_KIND, admission="records")
|
||||
if busy is not None:
|
||||
self.logger.info(
|
||||
f"Needs review was not updated for "
|
||||
f"{os.path.basename(str(image_path))} while {busy.title} runs; "
|
||||
"the next scan reads it again"
|
||||
)
|
||||
return "deferred"
|
||||
full = os.path.abspath(str(image_path))
|
||||
located = Path(os.path.realpath(os.path.dirname(full))) / os.path.basename(full)
|
||||
roots = _scan_roots(self._get_all_output_dirs())
|
||||
if self._review_location(located, roots) is None:
|
||||
return "skipped"
|
||||
if self._review_file_linked(located, roots):
|
||||
self._settle_scanned_reviews([located], roots)
|
||||
return "linked"
|
||||
if self._fallback_links(self._extract_comfyui_metadata(str(located))):
|
||||
return "skipped"
|
||||
return self._record_review_sync(located, roots)
|
||||
|
||||
@staticmethod
|
||||
def _log_review_items(job, review):
|
||||
"""The scan log's Needs review summary lines (none when nothing moved)."""
|
||||
prefix = SCAN_LOG_PREFIXES["review"]
|
||||
if review["added"]:
|
||||
job.log_line(
|
||||
f"{prefix} {_plural(review['added'], 'image')} added to Needs review"
|
||||
)
|
||||
already = review["queued"] - review["added"]
|
||||
if already:
|
||||
job.log_line(
|
||||
f"{prefix} {_plural(already, 'image')} already in Needs review"
|
||||
)
|
||||
if review["settled"]:
|
||||
job.log_line(
|
||||
f"{SCAN_LOG_PREFIXES['known']} "
|
||||
f"{_plural(review['settled'], 'image')} left Needs review · now linked"
|
||||
)
|
||||
if review["untraced"]:
|
||||
job.log_line(
|
||||
f"{SCAN_LOG_PREFIXES['known']} "
|
||||
f"{_plural(review['untraced'], 'image')} left Needs review · "
|
||||
f"{NOTHING_TO_CHOOSE}"
|
||||
)
|
||||
if review["errors"]:
|
||||
job.log_line(
|
||||
f"{SCAN_LOG_PREFIXES['errors']} "
|
||||
f"{_plural(review['errors'], 'image')} could not be added to "
|
||||
"Needs review"
|
||||
)
|
||||
+246
-31
@@ -4,8 +4,15 @@ A save validates the payload, commits it to config.json through the config
|
||||
store and publishes it at runtime inside that commit; a save that cannot be
|
||||
written changes nothing at runtime and says so. ``_config_commit_failure``
|
||||
is shared with the LoRA integration's save.
|
||||
|
||||
GET also reports the image sources and the monitor (``sources.py``). A save
|
||||
that changes the sources runs as a ``source_change`` registry hold, so it is
|
||||
refused while a record-changing job (a scan) runs. ``monitoring_enabled``
|
||||
takes effect at the next restart. ``POST /prompt_manager/settings/validate-source``
|
||||
checks one path and writes nothing.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
|
||||
from aiohttp import web
|
||||
@@ -15,7 +22,10 @@ from ..config_store import (
|
||||
RevisionConflict,
|
||||
get_config_store,
|
||||
)
|
||||
from . import sources
|
||||
from .admin import _current_image_monitor
|
||||
from .job_routes import busy_response
|
||||
from .jobs import JobBusy
|
||||
|
||||
try:
|
||||
from ...utils.validators import validate_result_timeout, validate_worker_threads
|
||||
@@ -23,13 +33,75 @@ except ImportError:
|
||||
from utils.validators import validate_result_timeout, validate_worker_threads
|
||||
|
||||
|
||||
# Settings owned by POST /settings: config attribute -> (section, key).
|
||||
# Settings owned by POST /settings: config attribute -> its keys in config.json.
|
||||
_SETTINGS_FILE_KEYS = {
|
||||
"RESULT_TIMEOUT": ("web_ui", "result_timeout"),
|
||||
"WEBUI_DISPLAY_MODE": ("web_ui", "webui_display_mode"),
|
||||
"INFINITE_SCROLL": ("web_ui", "infinite_scroll"),
|
||||
"WORKER_THREADS": ("performance", "worker_threads"),
|
||||
"MONITORING_ENABLED": ("gallery", "monitoring", "enabled"),
|
||||
}
|
||||
# Attributes that live on GalleryConfig; the others on PromptManagerConfig.
|
||||
_GALLERY_SETTINGS = frozenset({"MONITORING_ENABLED"})
|
||||
SOURCE_CHANGE_JOB_KIND = "source_change"
|
||||
SOURCE_CHANGE_JOB_TITLE = "Source change"
|
||||
|
||||
|
||||
def _source_identity(path):
|
||||
"""The case-folded spelling two roots are the same directory by (Windows)."""
|
||||
return os.path.normcase(path)
|
||||
|
||||
|
||||
def _stat_identity(path):
|
||||
"""``(st_dev, st_ino)`` of an existing directory, or None.
|
||||
|
||||
Catches case variants on a case-insensitive filesystem whose spelling
|
||||
normcase keeps (macOS).
|
||||
"""
|
||||
try:
|
||||
stat = os.stat(path)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return (stat.st_dev, stat.st_ino) if stat.st_ino else None
|
||||
|
||||
|
||||
# Refusals that a reconnected drive or folder can lift.
|
||||
_RECONNECTABLE = frozenset({"not-found", "not-directory"})
|
||||
|
||||
|
||||
def _root_key(root):
|
||||
"""The directory a configured root names (resolved), else its spelling."""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
try:
|
||||
return GalleryConfig.resolve_gallery_root(root)
|
||||
except (OSError, TypeError, ValueError, AttributeError):
|
||||
return root
|
||||
|
||||
|
||||
def _request_error(message):
|
||||
return {"code": "invalid-request", "error": message}
|
||||
|
||||
|
||||
def _invalid_source(index, entry, code, reason):
|
||||
"""The 400 body for the source at ``index`` that cannot be used.
|
||||
|
||||
Names it by its public spelling (computed as text: a refused path is
|
||||
never resolved), so the page can mark that row and say why.
|
||||
"""
|
||||
source = sources.public_spelling(entry)
|
||||
advice = (
|
||||
"Remove it or reconnect it, then save."
|
||||
if code in _RECONNECTABLE
|
||||
else "Remove it, then save."
|
||||
)
|
||||
return {
|
||||
"code": "invalid-source",
|
||||
"source": source,
|
||||
"index": index,
|
||||
"reason": reason,
|
||||
"error": f"{source} can't be used: {reason}. {advice}",
|
||||
}
|
||||
|
||||
|
||||
def _section(config, name):
|
||||
@@ -47,10 +119,8 @@ def _settings_mutation(updates, new_roots):
|
||||
|
||||
def mutate(config):
|
||||
for name, value in updates.items():
|
||||
section, key = _SETTINGS_FILE_KEYS[name]
|
||||
merged = _section(config, section)
|
||||
merged[key] = value
|
||||
config[section] = merged
|
||||
*sections, key = _SETTINGS_FILE_KEYS[name]
|
||||
_set_nested(config, sections, key, value)
|
||||
if new_roots is not None:
|
||||
gallery = _section(config, "gallery")
|
||||
monitoring = _section(gallery, "monitoring")
|
||||
@@ -62,6 +132,16 @@ def _settings_mutation(updates, new_roots):
|
||||
return mutate
|
||||
|
||||
|
||||
def _set_nested(config, sections, key, value):
|
||||
"""``config[s1][s2]...[key] = value``, each section a fresh dict."""
|
||||
if not sections:
|
||||
config[key] = value
|
||||
return
|
||||
merged = _section(config, sections[0])
|
||||
_set_nested(merged, sections[1:], key, value)
|
||||
config[sections[0]] = merged
|
||||
|
||||
|
||||
async def _read_json(request):
|
||||
"""Parse a JSON object body through the package's capped reader."""
|
||||
from . import _read_json_body
|
||||
@@ -81,6 +161,10 @@ class SettingsRoutesMixin:
|
||||
async def save_settings_route(request):
|
||||
return await self.save_settings(request)
|
||||
|
||||
@routes.post("/prompt_manager/settings/validate-source")
|
||||
async def validate_source_route(request):
|
||||
return await self.validate_source(request)
|
||||
|
||||
async def get_settings(self, request):
|
||||
"""Get current settings."""
|
||||
try:
|
||||
@@ -90,11 +174,15 @@ class SettingsRoutesMixin:
|
||||
self._public_path(d) for d in GalleryConfig.MONITORING_DIRECTORIES
|
||||
]
|
||||
revision = await self._run_in_executor(self._config_revision)
|
||||
source_status, monitor = await self._run_in_executor(self._source_status)
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
**revision,
|
||||
"sources": source_status,
|
||||
"monitor": monitor,
|
||||
"settings": {
|
||||
"monitoring_enabled": GalleryConfig.MONITORING_ENABLED,
|
||||
"result_timeout": PromptManagerConfig.RESULT_TIMEOUT,
|
||||
"webui_display_mode": PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||||
"infinite_scroll": PromptManagerConfig.INFINITE_SCROLL,
|
||||
@@ -115,6 +203,61 @@ class SettingsRoutesMixin:
|
||||
status=500,
|
||||
)
|
||||
|
||||
def _source_status(self):
|
||||
"""``(sources, monitor)`` of GET /settings (blocking)."""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
output_dirs, auto_detected = self._output_dirs_with_origin()
|
||||
return sources.describe_sources(
|
||||
configured=list(GalleryConfig.MONITORING_DIRECTORIES),
|
||||
output_dirs=output_dirs,
|
||||
auto_detected=auto_detected,
|
||||
monitor=_current_image_monitor(),
|
||||
enabled=GalleryConfig.MONITORING_ENABLED,
|
||||
)
|
||||
|
||||
async def validate_source(self, request):
|
||||
"""Check one source path ``{path}``; writes nothing.
|
||||
|
||||
200 ``{success, valid, label, path, code, reason}``: a valid source
|
||||
answers its label and public path, an invalid one its code and reason
|
||||
only. A body without a string ``path`` is a 400 ``invalid-request``.
|
||||
"""
|
||||
data, error_response = await _read_json(request)
|
||||
if error_response is not None:
|
||||
too_large = error_response.status == 413
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"code": "too-large" if too_large else "invalid-request",
|
||||
"error": (
|
||||
"Request body too large"
|
||||
if too_large
|
||||
else "Request body must be a JSON object"
|
||||
),
|
||||
},
|
||||
status=error_response.status,
|
||||
)
|
||||
path = data.get("path")
|
||||
if not isinstance(path, str):
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"code": "invalid-request",
|
||||
"error": "path must be a string",
|
||||
},
|
||||
status=400,
|
||||
)
|
||||
try:
|
||||
result = await self._run_in_executor(sources.validate_source, path)
|
||||
except Exception:
|
||||
self.logger.exception("Source validation failed")
|
||||
return web.json_response(
|
||||
{"success": False, "code": None, "error": "Failed to check the source"},
|
||||
status=500,
|
||||
)
|
||||
return web.json_response({"success": True, **result})
|
||||
|
||||
def _monitored_directories(self):
|
||||
"""Directories the running image monitor watches, else the configured roots."""
|
||||
from ..config import GalleryConfig
|
||||
@@ -151,9 +294,15 @@ class SettingsRoutesMixin:
|
||||
{"success": False, "error": str(ve)}, status=400
|
||||
)
|
||||
|
||||
new_roots, error = self._parse_gallery_roots(data)
|
||||
# Off the loop: an accepted root is resolved (filesystem access).
|
||||
new_roots, error = await self._run_in_executor(
|
||||
self._parse_gallery_roots, data
|
||||
)
|
||||
if error:
|
||||
return web.json_response({"success": False, "error": error}, status=400)
|
||||
return web.json_response({"success": False, **error}, status=400)
|
||||
roots_change = new_roots is not None and await self._run_in_executor(
|
||||
self._roots_change, new_roots
|
||||
)
|
||||
|
||||
published = {}
|
||||
|
||||
@@ -162,15 +311,30 @@ class SettingsRoutesMixin:
|
||||
# they were written, so the runtime ends on the last one.
|
||||
published.update(self._publish_settings(updates, new_roots))
|
||||
|
||||
try:
|
||||
_saved, revision = await self._run_in_executor(
|
||||
get_config_store().commit,
|
||||
_settings_mutation(updates, new_roots),
|
||||
expected_revision=expected_revision,
|
||||
publish=publish,
|
||||
)
|
||||
except (ConfigUnreadable, RevisionConflict, OSError) as exc:
|
||||
return self._config_commit_failure(exc)
|
||||
async with contextlib.AsyncExitStack() as stack:
|
||||
hold = None
|
||||
if roots_change:
|
||||
try:
|
||||
hold = await stack.enter_async_context(
|
||||
self._jobs.hold(
|
||||
SOURCE_CHANGE_JOB_KIND,
|
||||
SOURCE_CHANGE_JOB_TITLE,
|
||||
admission="records",
|
||||
)
|
||||
)
|
||||
except JobBusy as busy:
|
||||
return busy_response(busy)
|
||||
try:
|
||||
_saved, revision = await self._run_in_executor(
|
||||
get_config_store().commit,
|
||||
_settings_mutation(updates, new_roots),
|
||||
expected_revision=expected_revision,
|
||||
publish=publish,
|
||||
)
|
||||
except (ConfigUnreadable, RevisionConflict, OSError) as exc:
|
||||
if hold is not None:
|
||||
hold.finish("failed")
|
||||
return self._config_commit_failure(exc)
|
||||
|
||||
return web.json_response(
|
||||
{
|
||||
@@ -196,10 +360,11 @@ class SettingsRoutesMixin:
|
||||
as ``runtime_applied: false`` with the fields that need a restart;
|
||||
nothing is retried or rolled back. Never raises.
|
||||
"""
|
||||
from ..config import PromptManagerConfig
|
||||
from ..config import GalleryConfig, PromptManagerConfig
|
||||
|
||||
for name, value in updates.items():
|
||||
setattr(PromptManagerConfig, name, value)
|
||||
owner = GalleryConfig if name in _GALLERY_SETTINGS else PromptManagerConfig
|
||||
setattr(owner, name, value)
|
||||
restart_fields = []
|
||||
runtime_applied = True
|
||||
if new_roots is not None:
|
||||
@@ -212,11 +377,40 @@ class SettingsRoutesMixin:
|
||||
)
|
||||
runtime_applied = False
|
||||
restart_fields.append("gallery_root_paths")
|
||||
if "MONITORING_ENABLED" in updates:
|
||||
# The monitor starts with the server: a restart applies the setting
|
||||
# unless the observer already matches it.
|
||||
if updates["MONITORING_ENABLED"] != self._observer_alive():
|
||||
restart_fields.append("monitoring_enabled")
|
||||
return {
|
||||
"runtime_applied": runtime_applied,
|
||||
"restart_required_fields": restart_fields,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _roots_change(new_roots):
|
||||
"""Whether a save changes the sources (blocking: it resolves them).
|
||||
|
||||
Compared as directories, not spellings: the legacy page resends the
|
||||
roots resolved, while config.json may hold them as hand-written
|
||||
(relative, another case), and an unchanged resend is no change.
|
||||
"""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
if new_roots is None:
|
||||
return False
|
||||
return [_root_key(root) for root in new_roots] != [
|
||||
_root_key(root) for root in GalleryConfig.MONITORING_DIRECTORIES
|
||||
]
|
||||
|
||||
@staticmethod
|
||||
def _observer_alive():
|
||||
"""Whether the image monitor's observer is running now. Never raises."""
|
||||
try:
|
||||
return sources.monitor_state(_current_image_monitor())[1]
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def _config_commit_failure(self, exc):
|
||||
"""The response for a config commit that wrote nothing."""
|
||||
not_saved = {"success": False, "persisted": False, "runtime_applied": False}
|
||||
@@ -283,6 +477,10 @@ class SettingsRoutesMixin:
|
||||
if not isinstance(data["infinite_scroll"], bool):
|
||||
raise ValueError("infinite_scroll must be true or false")
|
||||
updates["INFINITE_SCROLL"] = data["infinite_scroll"]
|
||||
if "monitoring_enabled" in data:
|
||||
if not isinstance(data["monitoring_enabled"], bool):
|
||||
raise ValueError("monitoring_enabled must be true or false")
|
||||
updates["MONITORING_ENABLED"] = data["monitoring_enabled"]
|
||||
if "worker_threads" in data:
|
||||
updates["WORKER_THREADS"] = validate_worker_threads(
|
||||
data["worker_threads"], PromptManagerConfig.max_worker_threads()
|
||||
@@ -295,15 +493,21 @@ class SettingsRoutesMixin:
|
||||
Accepts ``gallery_root_paths`` (list, preferred) or the legacy single
|
||||
``gallery_root_path`` string. Blank entries are dropped.
|
||||
|
||||
Each entry passes the same checks as validate-source
|
||||
(``sources.source_problem``): the form, then containment as text,
|
||||
before anything is resolved. Blocking (an accepted root is resolved).
|
||||
|
||||
Returns:
|
||||
(roots, error): ``roots`` is the validated list, or ``None`` when
|
||||
the payload carries no gallery root key; ``error`` is a message
|
||||
when validation failed (and ``roots`` is then ``None``).
|
||||
the payload carries no gallery root key; ``error`` is the coded
|
||||
body of the 400 when validation failed (``roots`` is then None).
|
||||
An unusable source is ``invalid-source`` naming its public path
|
||||
and its position in the list.
|
||||
"""
|
||||
if "gallery_root_paths" in data:
|
||||
raw = data["gallery_root_paths"]
|
||||
if not isinstance(raw, list):
|
||||
return None, "gallery_root_paths must be a list"
|
||||
return None, _request_error("gallery_root_paths must be a list")
|
||||
elif "gallery_root_path" in data:
|
||||
raw = [data["gallery_root_path"]]
|
||||
else:
|
||||
@@ -312,27 +516,38 @@ class SettingsRoutesMixin:
|
||||
from ..config import GalleryConfig
|
||||
|
||||
roots = []
|
||||
for entry in raw:
|
||||
seen = set()
|
||||
for index, entry in enumerate(raw):
|
||||
if not isinstance(entry, str):
|
||||
return None, "Gallery root paths must be strings"
|
||||
return None, _request_error("Gallery root paths must be strings")
|
||||
entry = entry.strip()
|
||||
if not entry:
|
||||
continue
|
||||
ok, reason = GalleryConfig.validate_gallery_root(entry)
|
||||
if not ok:
|
||||
return (
|
||||
None,
|
||||
f"Invalid gallery root '{os.path.basename(entry)}': {reason}",
|
||||
)
|
||||
roots.append(GalleryConfig.resolve_gallery_root(entry))
|
||||
code, reason = sources.source_problem(entry)
|
||||
if code is not None:
|
||||
return None, _invalid_source(index, entry, code, reason)
|
||||
root = GalleryConfig.resolve_gallery_root(entry)
|
||||
keys = {("path", _source_identity(root)), ("file", _stat_identity(root))}
|
||||
keys.discard(("file", None))
|
||||
if keys & seen:
|
||||
continue # another spelling of a root already listed
|
||||
seen |= keys
|
||||
roots.append(root)
|
||||
return roots, None
|
||||
|
||||
def _apply_gallery_roots(self, roots):
|
||||
"""Install validated gallery roots; returns True when they changed."""
|
||||
"""Install validated gallery roots; returns True when they changed.
|
||||
|
||||
Another spelling of the same directories installs the resolved
|
||||
spelling and is no change (nothing to restart for).
|
||||
"""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
if roots == list(GalleryConfig.MONITORING_DIRECTORIES):
|
||||
return False
|
||||
if not self._roots_change(roots):
|
||||
GalleryConfig.MONITORING_DIRECTORIES = roots
|
||||
return False
|
||||
GalleryConfig.MONITORING_DIRECTORIES = roots
|
||||
self._cached_output_dir = None
|
||||
self.reset_gallery_roots_cache()
|
||||
|
||||
@@ -0,0 +1,280 @@
|
||||
"""Image sources as System › Sources & scans shows them (#249 S4).
|
||||
|
||||
- ``source_form_error`` is the lexical check a submitted source path passes
|
||||
before any filesystem access: no control characters, no ``..`` part, and
|
||||
only the native path forms (on Windows a drive or a UNC share, never the
|
||||
``\\\\?\\`` and ``\\\\.\\`` namespaces, a drive-relative path, an alternate
|
||||
data stream, a device name or a part Windows would silently trim; on POSIX
|
||||
no Windows drive or backslash form).
|
||||
- ``source_problem`` is the one refusal every caller answers (validate-source
|
||||
and the settings save): the form check, then
|
||||
``GalleryConfig.gallery_root_problem``, which decides containment as text
|
||||
before it touches the filesystem.
|
||||
- ``describe_sources`` (blocking: it resolves paths) reports the configured
|
||||
sources with their validity, the effective sources with their listing
|
||||
index and watch state, and the image monitor's state. Every path in it is
|
||||
a public path (``_public_path``), never an absolute one.
|
||||
|
||||
A source's watch state is ``watching`` when the monitor's observer is alive,
|
||||
scheduled that directory and monitoring is on; ``stopping`` when it still
|
||||
watches it but monitoring was switched off (it stops at the next restart);
|
||||
``manual`` when monitoring is off or this process has no monitor; else
|
||||
``restart_pending`` (a source added since the monitor started, monitoring
|
||||
switched on, or an observer that stopped).
|
||||
"""
|
||||
|
||||
import logging
|
||||
import ntpath
|
||||
import os
|
||||
import re
|
||||
|
||||
logger = logging.getLogger("prompt_manager.api")
|
||||
|
||||
MAX_SOURCE_PATH = 4096 # characters
|
||||
FORM_REASONS = {
|
||||
"empty": "Gallery root must be a non-empty path",
|
||||
"invalid-form": "Gallery root is not a folder path this server can use",
|
||||
"traversal": "Gallery root must not contain '..'",
|
||||
}
|
||||
_DRIVE = re.compile(r"[A-Za-z]:")
|
||||
_POSIX_FOREIGN = re.compile(r"^[A-Za-z]:")
|
||||
# Reserved device names: Windows opens the device for any folder of that name.
|
||||
_DEVICE_NAME = re.compile(r"^(con|prn|aux|nul|com[0-9¹²³]|lpt[0-9¹²³])(\..*)?$", re.I)
|
||||
# folder_paths getters and the label of the directory each one reports.
|
||||
_COMFYUI_LABELS = (
|
||||
("get_output_directory", "ComfyUI output"),
|
||||
("get_input_directory", "ComfyUI input"),
|
||||
("get_temp_directory", "ComfyUI temp"),
|
||||
("get_user_directory", "ComfyUI user"),
|
||||
)
|
||||
WATCH_STATES = ("watching", "stopping", "manual", "restart_pending")
|
||||
|
||||
|
||||
def source_form_error(text, *, windows=None):
|
||||
"""The form code that refuses ``text`` as a source path, or None.
|
||||
|
||||
``windows`` selects the path rules (default: this OS). No filesystem
|
||||
access happens here.
|
||||
"""
|
||||
if windows is None:
|
||||
windows = os.name == "nt"
|
||||
if not isinstance(text, str) or len(text) > MAX_SOURCE_PATH:
|
||||
return "invalid-form"
|
||||
if any(ord(char) < 32 or ord(char) == 127 for char in text):
|
||||
return "invalid-form"
|
||||
text = text.strip()
|
||||
if not text:
|
||||
return "empty"
|
||||
return _windows_form_error(text) if windows else _posix_form_error(text)
|
||||
|
||||
|
||||
def _posix_form_error(text):
|
||||
if "\\" in text or _POSIX_FOREIGN.match(text):
|
||||
return "invalid-form" # a Windows path names nothing on this server
|
||||
return "traversal" if ".." in text.split("/") else None
|
||||
|
||||
|
||||
def _windows_form_error(text):
|
||||
portable = text.replace("/", "\\")
|
||||
if portable.startswith(("\\\\?\\", "\\\\.\\")):
|
||||
return "invalid-form" # verbatim and device namespaces
|
||||
if portable.startswith("\\\\"):
|
||||
drive, rest = ntpath.splitdrive(portable)
|
||||
share = drive[2:].split("\\")
|
||||
if len(share) != 2 or any(part in ("", ".", "..") for part in share):
|
||||
return "invalid-form"
|
||||
else:
|
||||
drive, rest = ntpath.splitdrive(portable)
|
||||
if drive and (not _DRIVE.fullmatch(drive) or not rest.startswith("\\")):
|
||||
return "invalid-form" # "C:folder" is relative to that drive's cwd
|
||||
if ":" in rest:
|
||||
return "invalid-form" # an alternate data stream
|
||||
parts = rest.split("\\")
|
||||
if ".." in parts:
|
||||
return "traversal"
|
||||
for part in parts:
|
||||
if part in ("", "."):
|
||||
continue
|
||||
if part[-1] in ". " or _DEVICE_NAME.match(part):
|
||||
return "invalid-form"
|
||||
return None
|
||||
|
||||
|
||||
def source_problem(text):
|
||||
"""``(code, reason)`` that refuses ``text`` as a source, else ``(None, "")``.
|
||||
|
||||
The form check runs first and touches nothing; then containment is
|
||||
decided as text before any filesystem access (see
|
||||
``GalleryConfig.gallery_root_problem``). Every caller answers alike.
|
||||
"""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
code = source_form_error(text)
|
||||
if code is not None:
|
||||
return code, FORM_REASONS[code]
|
||||
return GalleryConfig.gallery_root_problem(text)
|
||||
|
||||
|
||||
def public_spelling(text):
|
||||
"""How a refused source is named in a response, computed as text only.
|
||||
|
||||
Relative to the path anchor it lies under (its public path), else its
|
||||
last part: the caller sent the path, so naming it reveals nothing, and a
|
||||
refused path is never resolved to name it.
|
||||
"""
|
||||
from ..config import GalleryConfig, _lexical_path, _lexically_inside
|
||||
|
||||
text = str(text).strip()
|
||||
try:
|
||||
lexical = os.path.normpath(text)
|
||||
key = _lexical_path(text) # the same length as ``lexical``
|
||||
if not os.path.isabs(text):
|
||||
return lexical.replace("\\", "/")
|
||||
for anchor in GalleryConfig.path_anchors():
|
||||
if _lexically_inside(key, [anchor]):
|
||||
rest = lexical[len(anchor.rstrip(os.sep)) :].lstrip(os.sep)
|
||||
return rest.replace(os.sep, "/") or "."
|
||||
return os.path.basename(lexical) or lexical
|
||||
except (TypeError, ValueError):
|
||||
return ""
|
||||
|
||||
|
||||
def identity(path):
|
||||
"""The comparison key of a directory: resolved and case-folded (Windows)."""
|
||||
return os.path.normcase(os.path.realpath(str(path)))
|
||||
|
||||
|
||||
def _comfyui_labels():
|
||||
"""``{identity: label}`` of the directories ComfyUI reports."""
|
||||
try:
|
||||
import folder_paths
|
||||
except ImportError:
|
||||
return {}
|
||||
labels = {}
|
||||
for getter_name, label in _COMFYUI_LABELS:
|
||||
getter = getattr(folder_paths, getter_name, None)
|
||||
if getter is None:
|
||||
continue
|
||||
try:
|
||||
directory = getter()
|
||||
except Exception:
|
||||
continue
|
||||
if isinstance(directory, str) and directory:
|
||||
labels.setdefault(identity(directory), label)
|
||||
return labels
|
||||
|
||||
|
||||
def source_label(path, labels=None):
|
||||
""" "ComfyUI output" for ComfyUI's own directories, else the folder name."""
|
||||
if labels is None:
|
||||
labels = _comfyui_labels()
|
||||
label = labels.get(identity(path))
|
||||
if label:
|
||||
return label
|
||||
return os.path.basename(os.path.normpath(str(path))) or str(path)
|
||||
|
||||
|
||||
def _configured(entry, labels, public_path):
|
||||
from ..config import GalleryConfig
|
||||
|
||||
code, reason = GalleryConfig.gallery_root_problem(entry)
|
||||
try:
|
||||
resolved = GalleryConfig.resolve_gallery_root(entry)
|
||||
except (OSError, TypeError, ValueError):
|
||||
resolved = str(entry)
|
||||
return {
|
||||
"label": source_label(resolved, labels),
|
||||
"path": public_path(resolved),
|
||||
"valid": code is None,
|
||||
"code": code,
|
||||
"reason": reason or None,
|
||||
}
|
||||
|
||||
|
||||
def monitor_state(monitor):
|
||||
"""``(present, observer_alive, scheduled directories)`` of the image monitor."""
|
||||
if monitor is None:
|
||||
return False, False, []
|
||||
status = monitor.get_status()
|
||||
scheduled = [str(d) for d in status.get("monitored_directories") or []]
|
||||
return True, bool(status.get("observer_alive")), scheduled
|
||||
|
||||
|
||||
def watch_state(path, *, present, alive, scheduled_keys, enabled):
|
||||
"""One of ``WATCH_STATES`` (see the module docstring)."""
|
||||
if alive and identity(path) in scheduled_keys:
|
||||
return "watching" if enabled else "stopping"
|
||||
if not present or not enabled:
|
||||
return "manual"
|
||||
return "restart_pending"
|
||||
|
||||
|
||||
def describe_sources(*, configured, output_dirs, auto_detected, monitor, enabled):
|
||||
"""``(sources, monitor)`` for GET /settings (blocking; public paths only).
|
||||
|
||||
``configured`` is the configured roots, ``output_dirs`` and
|
||||
``auto_detected`` what ``_output_dirs_with_origin`` returned, ``monitor``
|
||||
the running ImageMonitor or None, ``enabled`` the monitoring setting.
|
||||
"""
|
||||
from . import _public_path
|
||||
|
||||
labels = _comfyui_labels()
|
||||
present, alive, scheduled = monitor_state(monitor)
|
||||
scheduled_keys = {identity(path) for path in scheduled}
|
||||
effective = [
|
||||
{
|
||||
"label": source_label(path, labels),
|
||||
"path": _public_path(path),
|
||||
"root_index": index,
|
||||
"watch": watch_state(
|
||||
path,
|
||||
present=present,
|
||||
alive=alive,
|
||||
scheduled_keys=scheduled_keys,
|
||||
enabled=enabled,
|
||||
),
|
||||
}
|
||||
for index, path in enumerate(output_dirs)
|
||||
]
|
||||
return (
|
||||
{
|
||||
"configured": [
|
||||
_configured(entry, labels, _public_path) for entry in configured
|
||||
],
|
||||
"effective": effective,
|
||||
"auto_detected": bool(auto_detected),
|
||||
},
|
||||
{
|
||||
"present": present,
|
||||
"observer_alive": alive,
|
||||
"scheduled": [_public_path(path) for path in scheduled],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def validate_source(text):
|
||||
"""``{valid, label, path, code, reason}`` for one submitted path (blocking).
|
||||
|
||||
Writes nothing. An invalid path answers its code and reason with no
|
||||
label or path, so a response never echoes what was typed.
|
||||
"""
|
||||
from ..config import GalleryConfig
|
||||
from . import _public_path
|
||||
|
||||
code, reason = source_problem(text)
|
||||
if code is not None:
|
||||
return {
|
||||
"valid": False,
|
||||
"label": None,
|
||||
"path": None,
|
||||
"code": code,
|
||||
"reason": reason,
|
||||
}
|
||||
resolved = GalleryConfig.resolve_gallery_root(text)
|
||||
return {
|
||||
"valid": True,
|
||||
"label": source_label(resolved),
|
||||
"path": _public_path(resolved),
|
||||
"code": None,
|
||||
"reason": None,
|
||||
}
|
||||
@@ -0,0 +1,221 @@
|
||||
"""System routes: the read-only health check, the Overview and usage statistics.
|
||||
|
||||
- ``POST /prompt_manager/system/health`` starts the health check
|
||||
(``database/health.py``) off-loop as a ``read`` job of kind ``health``: one
|
||||
at a time, beside a scan, not cancellable and with no file outcomes. The
|
||||
API keeps the last report for this session; ``GET`` returns it or null.
|
||||
A report describes the database it was taken from: replacing the database
|
||||
(a restore) forgets it, and a check that started before the swap is not
|
||||
kept.
|
||||
- ``GET /prompt_manager/system/overview`` is the System › Overview aggregate.
|
||||
Every value is real or null: there is no backup status until backups report
|
||||
one and no health result until a check ran; ``needs_review.pending`` is the
|
||||
Needs review queue's pending total. The missing-thumbnail count is not part
|
||||
of it: the page reads ``/images/thumbnails/status``, which walks every
|
||||
source.
|
||||
- ``GET /prompt_manager/system/statistics`` lists runs and ratings per
|
||||
category, with the legacy ``/stats`` totals and the Overview's count of
|
||||
rated images (images whose linked prompt is rated).
|
||||
|
||||
New routes answer errors with ``{success:false, code, error}``.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from collections import namedtuple
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from .job_routes import busy_response
|
||||
from .jobs import HEALTH_JOB_KIND, JobBusy
|
||||
|
||||
HEALTH_JOB_TITLE = "Health check"
|
||||
_HEALTH_STATUS_LINES = {
|
||||
"passed": "Health check passed",
|
||||
"issues": "Health check found issues",
|
||||
"incomplete": "Health check incomplete",
|
||||
}
|
||||
|
||||
|
||||
# The last health report and the database generation it was taken from.
|
||||
_StoredReport = namedtuple("_StoredReport", "generation report")
|
||||
|
||||
|
||||
async def _gather_settled(*reads):
|
||||
"""Await every read, then raise the first failure.
|
||||
|
||||
Unlike a plain gather, a failure never answers while the other executor
|
||||
reads still run against the database.
|
||||
"""
|
||||
results = await asyncio.gather(*reads, return_exceptions=True)
|
||||
for result in results:
|
||||
if isinstance(result, BaseException):
|
||||
raise result
|
||||
return results
|
||||
|
||||
|
||||
def _failure(message, status=500, code=None):
|
||||
return web.json_response(
|
||||
{"success": False, "code": code, "error": message}, status=status
|
||||
)
|
||||
|
||||
|
||||
def health_summary(report):
|
||||
"""What the Overview shows of a report: when, the outcome, checks with issues."""
|
||||
if report is None:
|
||||
return None
|
||||
return {
|
||||
"checked_at": report["checked_at"],
|
||||
"status": report["status"],
|
||||
"issues": sum(1 for check in report["checks"] if check["status"] == "issues"),
|
||||
}
|
||||
|
||||
|
||||
class SystemRoutesMixin:
|
||||
"""Mixin providing the /prompt_manager/system endpoints."""
|
||||
|
||||
def _register_system_routes(self, routes):
|
||||
@routes.post("/prompt_manager/system/health")
|
||||
async def start_health_check_route(request):
|
||||
return await self.start_health_check(request)
|
||||
|
||||
@routes.get("/prompt_manager/system/health")
|
||||
async def health_report_route(request):
|
||||
return await self.get_health_report(request)
|
||||
|
||||
@routes.get("/prompt_manager/system/overview")
|
||||
async def system_overview_route(request):
|
||||
return await self.get_system_overview(request)
|
||||
|
||||
@routes.get("/prompt_manager/system/statistics")
|
||||
async def system_statistics_route(request):
|
||||
return await self.get_system_statistics(request)
|
||||
|
||||
# ── health ────────────────────────────────────────────────────────
|
||||
|
||||
def _health_roots(self):
|
||||
"""The sources missing files are judged under.
|
||||
|
||||
The effective sources, plus every configured source that is not an
|
||||
existing directory (an unmounted drive): the check then reports
|
||||
missing files as unavailable instead of counting that source's links.
|
||||
"""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
roots = [str(path) for path in self._get_all_output_dirs()]
|
||||
for configured in GalleryConfig.MONITORING_DIRECTORIES:
|
||||
try:
|
||||
resolved = GalleryConfig.resolve_gallery_root(configured)
|
||||
except (OSError, TypeError, ValueError):
|
||||
continue
|
||||
if not os.path.isdir(resolved):
|
||||
roots.append(resolved)
|
||||
return roots
|
||||
|
||||
def _database_replaced(self):
|
||||
"""Forget what described the previous database (call after swapping ``db``)."""
|
||||
self._db_generation += 1
|
||||
self._health_report = None
|
||||
|
||||
def _current_health_report(self):
|
||||
"""The last report, or None when none was kept for the current database."""
|
||||
stored = self._health_report
|
||||
generation = self._db_generation
|
||||
if isinstance(stored, _StoredReport) and stored.generation == generation:
|
||||
return stored.report
|
||||
return None
|
||||
|
||||
async def start_health_check(self, request):
|
||||
"""Start the health check job (202), or 409 naming the running check."""
|
||||
generation = self._db_generation
|
||||
db = self.db
|
||||
try:
|
||||
roots = await self._run_in_executor(self._health_roots)
|
||||
except Exception:
|
||||
self.logger.exception("Health check could not resolve the image sources")
|
||||
return _failure("Failed to start the health check")
|
||||
|
||||
async def run(job):
|
||||
job.update(status_line="Checking the database and image links")
|
||||
report = await self._run_in_executor(db.run_health_checks, roots)
|
||||
if generation == self._db_generation:
|
||||
self._health_report = _StoredReport(generation, report)
|
||||
else:
|
||||
self.logger.info("Health report dropped: the database was replaced")
|
||||
job.update(progress=100, status_line=_HEALTH_STATUS_LINES[report["status"]])
|
||||
|
||||
try:
|
||||
job = self._jobs.start(
|
||||
HEALTH_JOB_KIND,
|
||||
HEALTH_JOB_TITLE,
|
||||
admission="read",
|
||||
cancellable=False,
|
||||
scope={},
|
||||
runner=run,
|
||||
)
|
||||
except JobBusy as busy:
|
||||
return busy_response(busy)
|
||||
return web.json_response({"success": True, "job": job.summary()}, status=202)
|
||||
|
||||
async def get_health_report(self, request):
|
||||
return web.json_response(
|
||||
{"success": True, "report": self._current_health_report()}
|
||||
)
|
||||
|
||||
# ── overview ──────────────────────────────────────────────────────
|
||||
|
||||
async def get_system_overview(self, request):
|
||||
from ..config import GalleryConfig
|
||||
|
||||
try:
|
||||
(output_dirs, auto_detected), counts = await _gather_settled(
|
||||
self._run_in_executor(self._output_dirs_with_origin),
|
||||
self._run_in_executor(self.db.library_counts),
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("System overview failed")
|
||||
return _failure("Failed to read the overview")
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
"library": {
|
||||
"positive_prompts": counts["positive_prompts"],
|
||||
"negative_variants": counts["negative_variants"],
|
||||
},
|
||||
"images": {
|
||||
"indexed": counts["images"],
|
||||
"rated": counts["rated_images"],
|
||||
},
|
||||
"health": health_summary(self._current_health_report()),
|
||||
"backup": None, # no verified backup status exists yet (S8)
|
||||
"needs_review": {"pending": counts["review_pending"]},
|
||||
"sources": {
|
||||
"configured": len(GalleryConfig.MONITORING_DIRECTORIES),
|
||||
"effective": len(output_dirs),
|
||||
"auto_detected": auto_detected,
|
||||
},
|
||||
},
|
||||
headers={"Cache-Control": "no-store"},
|
||||
)
|
||||
|
||||
# ── statistics ────────────────────────────────────────────────────
|
||||
|
||||
async def get_system_statistics(self, request):
|
||||
try:
|
||||
by_category, totals, counts = await _gather_settled(
|
||||
self._run_in_executor(self.db.usage_by_category),
|
||||
self._run_in_executor(self.db.get_statistics),
|
||||
self._run_in_executor(self.db.library_counts),
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("System statistics failed")
|
||||
return _failure("Failed to read statistics")
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
"by_category": by_category,
|
||||
"totals": totals,
|
||||
"rated_images": counts["rated_images"],
|
||||
},
|
||||
headers={"Cache-Control": "no-store"},
|
||||
)
|
||||
+121
-25
@@ -31,7 +31,6 @@ routes = server_instance.routes
|
||||
extension_uri = None # Will be set in __init__.py
|
||||
|
||||
import os # noqa: E402
|
||||
from pathlib import Path # noqa: E402
|
||||
from typing import Any, Dict, List, Optional, Tuple # noqa: E402
|
||||
|
||||
# Import logging system
|
||||
@@ -51,6 +50,14 @@ config_logger = get_logger("prompt_manager.config")
|
||||
# may be used as gallery roots in addition to ComfyUI's own directories.
|
||||
EXTRA_GALLERY_ROOTS_ENV = "PROMPT_MANAGER_EXTRA_GALLERY_ROOTS"
|
||||
|
||||
# The one answer for every path outside the allowed folders. The filesystem
|
||||
# root and the home directory answer it too, so no caller learns the server's
|
||||
# layout (home path, user name) from a refusal (#249 S4).
|
||||
OUTSIDE_ALLOWED_REASON = (
|
||||
"Gallery root must be inside a ComfyUI directory (output, input, temp, user) "
|
||||
f"or one listed in {EXTRA_GALLERY_ROOTS_ENV}"
|
||||
)
|
||||
|
||||
# Environment variable overriding where config.json is read from and saved to.
|
||||
CONFIG_PATH_ENV = "PROMPT_MANAGER_CONFIG_PATH"
|
||||
|
||||
@@ -89,6 +96,29 @@ def write_private_json(path: str, data: Any) -> None:
|
||||
os.chmod(path, 0o600)
|
||||
|
||||
|
||||
def _lexical_path(path: str) -> str:
|
||||
"""``path`` normalised for comparison without touching the filesystem.
|
||||
|
||||
normpath collapses ``.`` and ``..`` as text and normcase folds case on
|
||||
Windows; neither opens, stats nor resolves anything (no SMB request for a
|
||||
UNC path). A symlink is only seen by :func:`_canonical_path` later.
|
||||
"""
|
||||
return os.path.normcase(os.path.normpath(path))
|
||||
|
||||
|
||||
def _lexically_inside(path: str, parents: List[str]) -> bool:
|
||||
"""Whether the normalised ``path`` is one of ``parents`` or below one.
|
||||
|
||||
Pure text on normalised spellings (:func:`_lexical_path` or
|
||||
:func:`_canonical_path`): no filesystem access.
|
||||
"""
|
||||
sep = os.path.sep
|
||||
return any(
|
||||
path == parent or path.startswith(parent.rstrip(sep) + sep)
|
||||
for parent in parents
|
||||
)
|
||||
|
||||
|
||||
def _is_filesystem_root(path: str) -> bool:
|
||||
"""True for '/' on POSIX and drive roots such as 'C:\\' on Windows."""
|
||||
return os.path.dirname(path) == path
|
||||
@@ -268,7 +298,12 @@ class GalleryConfig:
|
||||
"""
|
||||
monitoring = new_config.get("monitoring", {})
|
||||
if "enabled" in monitoring:
|
||||
cls.MONITORING_ENABLED = monitoring["enabled"]
|
||||
if isinstance(monitoring["enabled"], bool):
|
||||
cls.MONITORING_ENABLED = monitoring["enabled"]
|
||||
else: # a hand-edited "false" would otherwise switch it on
|
||||
config_logger.warning(
|
||||
"Ignoring gallery monitoring enabled: expected true or false"
|
||||
)
|
||||
if "directories" in monitoring:
|
||||
directories = cls._validated_directories(monitoring["directories"])
|
||||
if directories is not None:
|
||||
@@ -337,31 +372,56 @@ class GalleryConfig:
|
||||
Filesystem roots are never allowed as parents, even when listed in
|
||||
the environment, because that would re-open the whole disk.
|
||||
"""
|
||||
parents = []
|
||||
return [canonical for canonical, _spelling in cls._allowed_parent_pairs()]
|
||||
|
||||
@classmethod
|
||||
def _allowed_parent_pairs(cls) -> List[Tuple[str, str]]:
|
||||
"""``(canonical, lexical spelling)`` of each allowed parent.
|
||||
|
||||
Only trusted directories (ComfyUI's own and the environment's) are
|
||||
resolved here; the spelling is how ComfyUI or the environment wrote
|
||||
the directory, so a submitted path can be compared without resolving it.
|
||||
"""
|
||||
pairs = []
|
||||
seen = set()
|
||||
for candidate in _comfyui_directories() + _extra_gallery_roots():
|
||||
try:
|
||||
canonical = _canonical_path(candidate)
|
||||
spelling = _lexical_path(os.path.abspath(candidate))
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
if _is_filesystem_root(canonical) or not os.path.isdir(canonical):
|
||||
continue
|
||||
if canonical not in parents:
|
||||
parents.append(canonical)
|
||||
return parents
|
||||
if canonical not in seen:
|
||||
seen.add(canonical)
|
||||
pairs.append((canonical, spelling))
|
||||
return pairs
|
||||
|
||||
@classmethod
|
||||
def path_anchors(cls) -> List[str]:
|
||||
"""Canonical directories that relative gallery paths are resolved against.
|
||||
|
||||
The ComfyUI base directory comes first, followed by the parent of each
|
||||
directory listed in PROMPT_MANAGER_EXTRA_GALLERY_ROOTS, so an extra
|
||||
root is addressed by its own name (``gallery/sub``) rather than by an
|
||||
absolute path. The same anchors drive the public (relative) form of
|
||||
paths in API responses.
|
||||
ComfyUI directory outside it (``--output-directory D:\\Renders``) and of
|
||||
each directory listed in PROMPT_MANAGER_EXTRA_GALLERY_ROOTS, so such a
|
||||
directory is addressed by its own name (``Renders/sub``) rather than by
|
||||
an absolute path, and two folders of one name stay apart. The same
|
||||
anchors drive the public (relative) form of paths in API responses.
|
||||
"""
|
||||
anchors = []
|
||||
base = _comfyui_base_directory()
|
||||
candidates = [base] if base else []
|
||||
try:
|
||||
base_key = _canonical_path(base) if base else None
|
||||
except (OSError, ValueError):
|
||||
base_key = None
|
||||
for directory in _comfyui_directories():
|
||||
try:
|
||||
canonical = _canonical_path(directory)
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
if base_key is None or not _lexically_inside(canonical, [base_key]):
|
||||
candidates.append(os.path.dirname(canonical))
|
||||
candidates.extend(os.path.dirname(root) for root in _extra_gallery_roots())
|
||||
for candidate in candidates:
|
||||
try:
|
||||
@@ -399,33 +459,69 @@ class GalleryConfig:
|
||||
(True, "") when the directory lies inside one of
|
||||
:meth:`allowed_gallery_parents`; otherwise (False, reason).
|
||||
"""
|
||||
code, reason = cls.gallery_root_problem(path)
|
||||
return code is None, reason
|
||||
|
||||
@classmethod
|
||||
def gallery_root_problem(cls, path: Any) -> Tuple[Optional[str], str]:
|
||||
"""``(code, reason)`` why ``path`` may not be a gallery root, else (None, "").
|
||||
|
||||
Codes: ``empty``, ``unresolvable``, ``outside-allowed``, ``not-found``
|
||||
and ``not-directory``. No reason names the path.
|
||||
|
||||
Containment is decided twice. First lexically, with no filesystem
|
||||
access at all: a path (UNC, ``\\\\?\\``, a device, anything) whose
|
||||
normalised spelling lies outside every allowed parent answers
|
||||
``outside-allowed`` before it is resolved, so the server never opens
|
||||
or stats it (no SMB connection, no existence or timing oracle). Only
|
||||
then is it resolved and checked again, which catches a symlink out.
|
||||
The filesystem root and the home directory answer ``outside-allowed``
|
||||
too: one answer for every caller (validate-source, the settings save,
|
||||
config.json), and the detail goes to the log only.
|
||||
"""
|
||||
if not isinstance(path, str) or not path.strip():
|
||||
return False, "Gallery root must be a non-empty path"
|
||||
return "empty", "Gallery root must be a non-empty path"
|
||||
|
||||
pairs = cls._allowed_parent_pairs()
|
||||
spellings = [spelling for pair in pairs for spelling in pair]
|
||||
try:
|
||||
lexical = cls._lexical_candidates(path)
|
||||
except (TypeError, ValueError):
|
||||
return "unresolvable", "Gallery root could not be resolved"
|
||||
if not any(_lexically_inside(candidate, spellings) for candidate in lexical):
|
||||
return "outside-allowed", OUTSIDE_ALLOWED_REASON
|
||||
|
||||
try:
|
||||
canonical = cls.resolve_gallery_root(path)
|
||||
except (OSError, ValueError):
|
||||
return False, "Gallery root could not be resolved"
|
||||
return "unresolvable", "Gallery root could not be resolved"
|
||||
|
||||
if _is_filesystem_root(canonical):
|
||||
return False, "Gallery root cannot be a filesystem root"
|
||||
config_logger.info("Gallery root refused: a filesystem root")
|
||||
return "outside-allowed", OUTSIDE_ALLOWED_REASON
|
||||
if canonical == _canonical_path(os.path.expanduser("~")):
|
||||
return False, "Gallery root cannot be the home directory"
|
||||
config_logger.info("Gallery root refused: the home directory")
|
||||
return "outside-allowed", OUTSIDE_ALLOWED_REASON
|
||||
if not _lexically_inside(canonical, [canonical for canonical, _ in pairs]):
|
||||
return "outside-allowed", OUTSIDE_ALLOWED_REASON
|
||||
if not os.path.exists(canonical):
|
||||
return False, "Gallery root does not exist"
|
||||
return "not-found", "Gallery root does not exist"
|
||||
if not os.path.isdir(canonical):
|
||||
return False, "Gallery root is not a directory"
|
||||
return "not-directory", "Gallery root is not a directory"
|
||||
return None, ""
|
||||
|
||||
candidate = Path(canonical)
|
||||
for parent in cls.allowed_gallery_parents():
|
||||
if candidate.is_relative_to(Path(parent)):
|
||||
return True, ""
|
||||
@classmethod
|
||||
def _lexical_candidates(cls, path: str) -> List[str]:
|
||||
"""Every spelling ``path`` may resolve to, normalised as text (no IO).
|
||||
|
||||
return (
|
||||
False,
|
||||
"Gallery root must be inside a ComfyUI directory "
|
||||
f"(output, input, temp, user) or one listed in {EXTRA_GALLERY_ROOTS_ENV}",
|
||||
)
|
||||
An absolute path is itself; a relative one is joined to each
|
||||
:meth:`path_anchors` entry, as :meth:`resolve_gallery_root` tries them.
|
||||
"""
|
||||
path = path.strip()
|
||||
if os.path.isabs(path):
|
||||
return [_lexical_path(path)]
|
||||
anchors = cls.path_anchors()
|
||||
return [_lexical_path(os.path.join(anchor, path)) for anchor in anchors]
|
||||
|
||||
|
||||
class IntegrationConfig:
|
||||
|
||||
@@ -78,6 +78,17 @@ export async function seedConcept(request,data={}) {
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/**
|
||||
* Seed the concept fixture and fix the browser's clock at the fixture's own instant (`manifest.now`), so the page and
|
||||
* the fixture read one "now": a prompt used now reads "Used today" and the caption names the fixture's day even when
|
||||
* the run crosses midnight between seeding and rendering. Timers keep running; only `Date` stands still.
|
||||
*/
|
||||
export async function seedConceptAt(page,request,data={}) {
|
||||
const manifest=await seedConcept(request,data);
|
||||
await page.clock.setFixedTime(new Date(manifest.now*1000));
|
||||
return manifest;
|
||||
}
|
||||
|
||||
/**
|
||||
* Save `tags` as the current image tag set of the listed file at `path` (root 0) through the real observe and tag
|
||||
* routes (#249 PR C), releasing the capture afterwards; returns the receipt's saved names.
|
||||
|
||||
@@ -237,9 +237,12 @@ const viewerStructure=details=>details.evaluate(node=>{
|
||||
test('viewer details hold provenance, raw records and downloads',async({page,request})=>{
|
||||
await page.setViewportSize({width:1600,height:1100});const reads=await patchInspection(page);
|
||||
await fullFixture(page,request);let details=await viewerDetails(page,'complete.png');
|
||||
// The panel content without the cover, then the two disclosures, then the pair send and history buttons (D20).
|
||||
// The panel content without the cover, D117's Copy settings summary after the facts, then the two disclosures,
|
||||
// then the pair send and history buttons (D20).
|
||||
expect(await viewerStructure(details)).toEqual({outer:['div','button.send','button'],
|
||||
inner:['div.panel-heading','p.small','div.tags','section.prompt-section.positive','section.prompt-section.negative','div.tags','dl.meta-grid','details','details']});
|
||||
inner:['div.panel-heading','p.small','div.tags','section.prompt-section.positive','section.prompt-section.negative','div.tags','dl.meta-grid','button.settings-summary','details','details']});
|
||||
await expect(details.locator(':scope > div > button')).toHaveText('Copy settings summary');
|
||||
await expect(details.locator(':scope > div > button.settings-summary')).toHaveCSS('white-space','normal');
|
||||
await expect(details.locator('.detail-cover,.detail-inner,.detail-footer')).toHaveCount(0);
|
||||
await expect(details.locator('.panel-heading h2')).toHaveText(POSITIVE_TITLE);await expect(details.locator('.panel-heading .chip')).toHaveText('PNG');
|
||||
await expect(details.locator(':scope > div > p.small')).toHaveText('complete.png');
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import {writeFile} from 'node:fs/promises';
|
||||
import {test,expect} from './fixtures.mjs';
|
||||
import {captureCanonicalState} from './history-canonical-comparison.mjs';
|
||||
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact,countCopy,reflow,withoutItems} from './concept-inventory.mjs';
|
||||
import {SIZES,THEMES,MOTIONS,seedConceptAt,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact,countCopy,reflow,withoutItems} from './concept-inventory.mjs';
|
||||
|
||||
// Task C7 of #249: the production Images page against the immutable concept, on the concept-shaped real fixture (24
|
||||
// images of the concept's six prompts, its seeds, sizes and favorites). Heading, toolbar, filter row, caption, grid,
|
||||
@@ -85,7 +85,7 @@ function listInfo(card,actionsWidth) {
|
||||
/** Recorded Images deviations only (spec §8); everything else in the concept inventory must equal production. */
|
||||
const nocase=(a,b)=>{const x=a.toLowerCase(),y=b.toLowerCase();return x<y?-1:x>y?1:a<b?-1:a>b?1:0;};
|
||||
|
||||
export function mapImagesConcept(reference,{manifest,now=new Date()}) {
|
||||
export function mapImagesConcept(reference,{manifest,now=new Date(manifest.now*1000)}) {
|
||||
const paths=new Map(manifest.images.map(image=>[image.filename,image.path]));
|
||||
// D14: the sort appends production's two sorts. Save view is the concept's own (saved views, #249 D16 retired).
|
||||
const toolbar={...reference.toolbar};
|
||||
@@ -318,7 +318,7 @@ async function behaviorsOf(page,scope) {
|
||||
return {hover,opened,viewerTitle,focusAfterViewer,selected,layouts,search,single};
|
||||
}
|
||||
|
||||
test.beforeEach(async({request},testInfo)=>{testInfo.manifest=await seedConcept(request);});
|
||||
test.beforeEach(async({page,request},testInfo)=>{testInfo.manifest=await seedConceptAt(page,request);});
|
||||
|
||||
for(const [width,height] of SIZES)for(const theme of THEMES)test(`images page geometry, tokens, controls and behaviors match the concept ${width}x${height} ${theme}`,async({page,request},testInfo)=>{
|
||||
test.setTimeout(240_000);
|
||||
@@ -348,7 +348,7 @@ for(const [width,height] of SIZES)for(const theme of THEMES)test(`images page ge
|
||||
importDiff:diffInventory(mapImportConcept(referenceImport),actualImport),referenceBehaviors,actualBehaviors};
|
||||
}
|
||||
// Load more in production: a fixture past one listing page appends one batch and shows the concept's toast.
|
||||
testInfo.manifest=await seedConcept(request,{extra_images:LOAD_MORE_EXTRA});
|
||||
testInfo.manifest=await seedConceptAt(page,request,{extra_images:LOAD_MORE_EXTRA});
|
||||
for(const motion of MOTIONS) {
|
||||
const total=FIXTURE_IMAGES+LOAD_MORE_EXTRA;
|
||||
await openProductionLoaded(page,theme,motion,total);
|
||||
@@ -514,7 +514,7 @@ test('one appended image reads in the singular',async({page,request})=>{
|
||||
test.setTimeout(120_000);
|
||||
await page.setViewportSize({width:1600,height:1100});
|
||||
const total=PRODUCTION_BATCH+1;
|
||||
await seedConcept(request,{extra_images:total-FIXTURE_IMAGES});
|
||||
await seedConceptAt(page,request,{extra_images:total-FIXTURE_IMAGES});
|
||||
await openProductionLoaded(page,'dark','no-preference',total);
|
||||
await expect(page.locator('.infinite-continuation > p').first()).toHaveText(countCopy(`${PRODUCTION_BATCH} of ${total} images · more load as you scroll`));
|
||||
// A dispatched press: scrolling the button into view lets the page's own observer load the one image first.
|
||||
@@ -530,7 +530,7 @@ test('one image appended by scrolling reads in the singular',async({page,request
|
||||
test.setTimeout(120_000);
|
||||
await page.setViewportSize({width:1600,height:1100});
|
||||
const total=PRODUCTION_BATCH+1;
|
||||
await seedConcept(request,{extra_images:total-FIXTURE_IMAGES});
|
||||
await seedConceptAt(page,request,{extra_images:total-FIXTURE_IMAGES});
|
||||
await openProductionLoaded(page,'dark','no-preference',total);
|
||||
await expect(page.locator('#result-count')).toHaveText(new RegExp(`^${PRODUCTION_BATCH} of ${total} images`));
|
||||
await page.locator('.infinite-continuation').scrollIntoViewIfNeeded();
|
||||
@@ -551,7 +551,7 @@ test('one footer height for the collection (D3)',async({page,request})=>{
|
||||
const reference=await inventoryOf(page,'#main');
|
||||
expect(reference.grid.columns).toBe(2);
|
||||
expect(reference.cards.every(card=>card.info.p.height>parseFloat(card.info.p.lineHeight))).toBe(true);
|
||||
const manifest=await seedConcept(request,{seedless_images:2,short_seedless:true});
|
||||
const manifest=await seedConceptAt(page,request,{seedless_images:2,short_seedless:true});
|
||||
await openProductionLoaded(page,'dark','no-preference',FIXTURE_IMAGES+2);
|
||||
const actual=await inventoryOf(page,'#workspace-main');
|
||||
const box=card=>({left:card.left,top:card.top,width:card.width,height:card.height});
|
||||
|
||||
@@ -117,7 +117,7 @@ function measureViewer() {
|
||||
['dl.meta-grid', node => node.closest('details') ? null : 'facts'],
|
||||
['details > summary', node => text(node)],
|
||||
['button.send', node => node.closest('.prompt-section') ? `send ${node.closest('.prompt-section').matches('.positive') ? 'positive' : 'negative'}` : 'send pair'],
|
||||
['button', node => /Prompt history/.test(node.textContent) ? 'Prompt history' : null],
|
||||
['button', node => /Prompt history/.test(node.textContent) ? 'Prompt history' : text(node) === 'Copy settings summary' ? 'settings summary' : null],
|
||||
];
|
||||
const sections = [...rail.querySelectorAll('*')].map(node => {
|
||||
for (const [selector, label] of SECTIONS) if (node.matches(selector)) return label(node);
|
||||
@@ -158,13 +158,18 @@ async function withProductionCopy(page) {
|
||||
}, { COPY, ACCEPT });
|
||||
}
|
||||
|
||||
/** D66: the viewer of a local file is titled by its filename and has no tag chips and no Prompt history. */
|
||||
/**
|
||||
* D66: the viewer of a local file is titled by its filename and has no tag chips and no Prompt history. D117 (LI-4):
|
||||
* "Copy settings summary" follows the six facts.
|
||||
*/
|
||||
function mapViewerConcept(reference) {
|
||||
const sections = reference.sections.filter(section => section !== 'tag chips' && section !== 'Prompt history');
|
||||
const facts = sections.indexOf('facts');
|
||||
return {
|
||||
...reference,
|
||||
title: FILE,
|
||||
railTitle: FILE,
|
||||
sections: reference.sections.filter(section => section !== 'tag chips' && section !== 'Prompt history'),
|
||||
sections: [...sections.slice(0, facts + 1), 'settings summary', ...sections.slice(facts + 1)],
|
||||
};
|
||||
}
|
||||
|
||||
@@ -269,9 +274,9 @@ for (const [width, height] of SIZES) for (const theme of THEMES) {
|
||||
// The chooser names the chosen file after a real setInputFiles, as the concept's does.
|
||||
expect(production.initial.chosen).toEqual({ name: '', files: [] });
|
||||
expect(production.selected.chosen).toEqual({ name: FILE, files: [FILE] });
|
||||
// D66 in the rail: no tag chips and no Prompt history; the title is the filename.
|
||||
// D66 in the rail: no tag chips and no Prompt history; the title is the filename. D117 follows the facts.
|
||||
expect(production.viewer.sections).toEqual(['heading', 'filename', 'badges', 'positive', 'send positive', 'negative',
|
||||
'send negative', 'facts', 'File & provenance', 'Workflow & raw metadata', 'send pair']);
|
||||
'send negative', 'facts', 'settings summary', 'File & provenance', 'Workflow & raw metadata', 'send pair']);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -45,10 +45,10 @@ function withCategoryScope(inventory) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Recorded inspector deviations only (spec §8, TESTING.md D3, D20–D22, D34 and D38); everything else must equal
|
||||
* production. `reference.viewerPlain` is the concept's own full details
|
||||
* with its tag chips made plain (D20) and, under a coarse pointer, D34's targets, both at runtime: the browser lays out
|
||||
* what follows them, as the Images spec's removed toolbar items do.
|
||||
* Recorded inspector deviations only (spec §8, TESTING.md D3, D20–D22, D34, D38 and D117); everything else must
|
||||
* equal production. `reference.viewerPlain` is the concept's own full details
|
||||
* with its tag chips made plain (D20), D117's summary button added and, under a coarse pointer, D34's targets, all at
|
||||
* runtime: the browser lays out what follows them, as the Images spec's removed toolbar items do.
|
||||
*/
|
||||
export function mapInspectorConcept(reference,{viewport}) {
|
||||
const {panel,viewerPlain,fileControls}=reference;
|
||||
@@ -105,7 +105,11 @@ async function openProductionPanel(page,theme) {
|
||||
await settle(page);
|
||||
}
|
||||
|
||||
/** The concept's tag chips in the open viewer as plain `span.chip`s (D20) and D34's coarse targets; returns an undo. */
|
||||
/**
|
||||
* The concept's tag chips in the open viewer as plain `span.chip`s (D20), D34's coarse targets and D117's "Copy
|
||||
* settings summary" after the six facts (LI-4), drawn as the concept's own plain button (its "Download sample JSON",
|
||||
* without its action); returns an undo.
|
||||
*/
|
||||
async function plainViewerChips(page) {
|
||||
const restoreTargets=await mutate(page,{css:D34_COARSE_TARGETS});
|
||||
await page.evaluate(()=>{
|
||||
@@ -113,10 +117,15 @@ async function plainViewerChips(page) {
|
||||
const span=document.createElement('span');span.className='chip';span.textContent=button.textContent;
|
||||
button.replaceWith(span);return [span,button];
|
||||
});
|
||||
const summary=document.querySelector('#viewer-details [data-action="download-workflow"]').cloneNode(false);
|
||||
for(const key of Object.keys(summary.dataset))delete summary.dataset[key];
|
||||
summary.textContent='Copy settings summary';summary.classList.add('settings-summary');
|
||||
document.querySelector('#viewer-details > div > dl.meta-grid').after(summary);window.__summaryButton=summary;
|
||||
});
|
||||
await settle(page);
|
||||
return async()=>{
|
||||
await page.evaluate(()=>{for(const [span,button] of window.__plainChips)span.replaceWith(button);delete window.__plainChips;});
|
||||
await page.evaluate(()=>{for(const [span,button] of window.__plainChips)span.replaceWith(button);delete window.__plainChips;
|
||||
window.__summaryButton.remove();delete window.__summaryButton;});
|
||||
await restoreTargets();
|
||||
};
|
||||
}
|
||||
@@ -342,7 +351,7 @@ for(const [width,height] of SIZES)test(`image tags saved on the concept image ke
|
||||
|
||||
// The comparison is only worth something if it notices a change (spec §7 mutation check): on the panel one CSS
|
||||
// mutation (the role label's letter spacing) and one DOM mutation (the footer removed); in the viewer (final review m5)
|
||||
// its pair actions removed and D38's Show in folder removed; each applied after an exact match.
|
||||
// its pair actions removed, D38's Show in folder removed and D117's summary removed; each applied after an exact match.
|
||||
test('inspector comparison detects mutations',async({page})=>{
|
||||
test.setTimeout(120_000);
|
||||
const viewport={width:1600,height:1100};
|
||||
@@ -366,6 +375,9 @@ test('inspector comparison detects mutations',async({page})=>{
|
||||
expect(()=>assertInspectorMatch(reference,withoutActions,viewport)).toThrow(/viewerDetails\.actions/);
|
||||
const withoutReveal=await viewerMutated({remove:['#viewer-details button[aria-label$=" in folder"]']});
|
||||
expect(()=>assertInspectorMatch(reference,withoutReveal,viewport)).toThrow(/fileControls/);
|
||||
// D117: the summary action after the facts is part of the mapped concept, so losing it is a difference.
|
||||
const withoutSummary=await viewerMutated({remove:['#viewer-details > div > button']});
|
||||
expect(()=>assertInspectorMatch(reference,withoutSummary,viewport)).toThrow(/viewerDetails\.structure\.inner/);
|
||||
assertInspectorMatch(reference,await viewerMutated(),viewport);
|
||||
|
||||
await mutate(page,{remove:['#image-inspector > .detail-footer']});
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import {writeFile} from 'node:fs/promises';
|
||||
import {test,expect} from './fixtures.mjs';
|
||||
import {captureCanonicalState} from './history-canonical-comparison.mjs';
|
||||
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact,countCopy,reflow} from './concept-inventory.mjs';
|
||||
import {SIZES,THEMES,MOTIONS,seedConceptAt,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact,countCopy,reflow} from './concept-inventory.mjs';
|
||||
|
||||
// Task E3 of #249: the production Prompt library shelf against the immutable concept (app.js renderLibrary a572–586,
|
||||
// style.css s648–708), on the concept-shaped real fixture (the concept's six prompts with their negatives, categories,
|
||||
@@ -37,7 +37,8 @@ export function mapLibraryConcept(reference) {
|
||||
// D43: tag chips come in the store's alphabetical order; the row flows again from the concept's own chips.
|
||||
tags:sortedTags(card.tags),
|
||||
// D23: the stars are aria-hidden, with a visually hidden "Rated N out of 5" spoken in their place. The fixture's
|
||||
// run_count (4) and last_used_at (now) give the concept's own "4 runs" and "Used today". The two-line clamp is
|
||||
// run_count (4) and last_used_at (its own "now", where the page's clock stands) give the concept's own "4 runs"
|
||||
// and "Used today". The two-line clamp is
|
||||
// applied below through `clampedCards`, measured on the concept with `CLAMP_CSS` (its samples wrap at 390px).
|
||||
footer:{...card.footer,facts:{...card.footer.facts,spoken:card.footer.facts.spoken.replace(/^(★+)/,stars=>`Rated ${stars.length} out of 5`)}}}));
|
||||
return {...reference,library:{...library,toolbar,
|
||||
@@ -238,7 +239,8 @@ async function openHost(context) {
|
||||
}
|
||||
|
||||
test.beforeEach(async({page,request},testInfo)=>{
|
||||
testInfo.manifest=await seedConcept(request);
|
||||
// The page's clock stands at the fixture's own "now", so its last use reads "Used today" on either side of midnight.
|
||||
testInfo.manifest=await seedConceptAt(page,request);
|
||||
// Both documents copy through the clipboard; the stub records what they wrote.
|
||||
await page.addInitScript(()=>{window.copied=[];Object.defineProperty(navigator,'clipboard',{configurable:true,value:{writeText:async text=>{window.copied.push(text);}}});});
|
||||
});
|
||||
@@ -322,6 +324,27 @@ for(const theme of THEMES)test(`library grid and cards match the concept 1800x11
|
||||
expect(compare(reference,actual,{sections:['library'],library:['grid','cards','trailing']})).toEqual([]);
|
||||
});
|
||||
|
||||
// PR #284's CI run: the fixture recorded the last use at 23:59:5x UTC and the shelf rendered at 00:00:05, so production
|
||||
// read "Used yesterday" against the concept's "Used today". Here the browser's wall clock starts at 23:59:58 UTC, the
|
||||
// fixture records its use at that instant, and midnight passes before either document opens.
|
||||
const BEFORE_MIDNIGHT=Date.UTC(2026,9,22,23,59,58);
|
||||
const PAST_MIDNIGHT_MS=5_000;
|
||||
test('a run that crosses midnight still reads the concept fixture\'s day',async({page,request})=>{
|
||||
await page.setViewportSize({width:1800,height:1100});
|
||||
await page.clock.install({time:BEFORE_MIDNIGHT});
|
||||
const manifest=await seedConceptAt(page,request,{now:BEFORE_MIDNIGHT/1000});
|
||||
expect(manifest.now*1000,'the fixture\'s now').toBe(BEFORE_MIDNIGHT);
|
||||
const used=(await (await request.get('/prompt_manager/library')).json()).items.map(item=>Date.parse(item.last_used_at));
|
||||
expect(used,'every last use at the fixture\'s now').toEqual(Array(CONCEPT_PROMPTS).fill(BEFORE_MIDNIGHT));
|
||||
await page.clock.fastForward(PAST_MIDNIGHT_MS);
|
||||
await openConceptLibrary(page,'dark','no-preference');
|
||||
const reference=await conceptInventory(page);
|
||||
await openProductionLibrary(page,'dark','no-preference');
|
||||
const actual=await inventoryOf(page,'#workspace-main');
|
||||
expect(actual.library.cards.map(card=>card.footer.facts.text.replace(/^★+ · /,''))).toEqual(Array(CONCEPT_PROMPTS).fill('Used today'));
|
||||
expect(compare(reference,actual,{sections:['library'],library:['grid','cards','trailing']})).toEqual([]);
|
||||
});
|
||||
|
||||
// The comparison is only worth something if it notices a change (spec §7 mutation check): one CSS mutation (the footer
|
||||
// padding) and one DOM mutation (the negative-variants chip removed), each applied after an exact match.
|
||||
test('library comparison detects mutations',async({page})=>{
|
||||
|
||||
+20
-1
@@ -19,10 +19,13 @@ from PIL import Image
|
||||
from PIL.PngImagePlugin import PngInfo
|
||||
|
||||
from tests.browser.request_hold import register_request_holds
|
||||
from tests.browser import slow_reset_watch
|
||||
from tests.browser.workspace_activity_fixture import register_activity_commands
|
||||
from tests.browser.workspace_system_fixture import register_system_commands
|
||||
from tests.browser.workspace_concept_fixture import (
|
||||
MAX_EXTRA_IMAGES as CONCEPT_MAX_EXTRA_IMAGES,
|
||||
MAX_SEEDLESS_IMAGES as CONCEPT_MAX_SEEDLESS_IMAGES,
|
||||
MAX_EPOCH_SECONDS as CONCEPT_MAX_EPOCH_SECONDS,
|
||||
seed_workspace_concept,
|
||||
)
|
||||
from tests.browser.workspace_visual_fixture import seed_workspace_visual
|
||||
@@ -196,7 +199,9 @@ def create_app(root: Path) -> web.Application:
|
||||
register_tagging_commands(routes, api, root, model)
|
||||
reveal_stub = register_reveal_stub(routes, api, root)
|
||||
register_activity_commands(app, routes, api)
|
||||
register_system_commands(app, routes, api, root)
|
||||
request_holds = register_request_holds(app, routes)
|
||||
slow_reset_watch.install(app, api)
|
||||
|
||||
async def canonical_asset(request: web.Request) -> web.Response:
|
||||
filename, content_type = CANONICAL_ASSETS[request.path]
|
||||
@@ -221,6 +226,8 @@ def create_app(root: Path) -> web.Application:
|
||||
for table in (
|
||||
"saved_views",
|
||||
"image_favorites",
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
@@ -489,8 +496,17 @@ def create_app(root: Path) -> web.Application:
|
||||
if (
|
||||
request.query
|
||||
or not isinstance(data, dict)
|
||||
or set(data) - set(limits) - {"short_seedless"}
|
||||
or set(data) - set(limits) - {"short_seedless", "now"}
|
||||
or not isinstance(data.get("short_seedless", False), bool)
|
||||
# A chosen "now" (epoch seconds) pins every time the fixture writes.
|
||||
or (
|
||||
"now" in data
|
||||
and (
|
||||
isinstance(data["now"], bool)
|
||||
or not isinstance(data["now"], (int, float))
|
||||
or not 0 < data["now"] < CONCEPT_MAX_EPOCH_SECONDS
|
||||
)
|
||||
)
|
||||
or any(
|
||||
isinstance(data.get(key, 0), bool)
|
||||
or not isinstance(data.get(key, 0), int)
|
||||
@@ -510,6 +526,7 @@ def create_app(root: Path) -> web.Application:
|
||||
extra_images=data.get("extra_images", 0),
|
||||
seedless_images=data.get("seedless_images", 0),
|
||||
short_seedless=data.get("short_seedless", False),
|
||||
now=data.get("now"),
|
||||
)
|
||||
)
|
||||
return web.json_response({"success": True, **manifest})
|
||||
@@ -576,6 +593,8 @@ def create_app(root: Path) -> web.Application:
|
||||
return web.json_response("dark")
|
||||
|
||||
async def cleanup(application: web.Application) -> None:
|
||||
# A scan of an ambiguous fixture file captures it for Needs review.
|
||||
api.close_annotation_captures()
|
||||
api.db.close_all()
|
||||
|
||||
seed()
|
||||
|
||||
@@ -0,0 +1,237 @@
|
||||
"""Report a slow test reset with what the server was doing while it waited.
|
||||
|
||||
A reset normally answers within about a second. Once one runs past
|
||||
SLOW_RESET_SECONDS, this writes to stderr, while the reset is still waiting:
|
||||
|
||||
- every thread's stack, so a reset waiting on a lock, a full executor or a
|
||||
slow disk shows where it waits and who holds it;
|
||||
- the default executor's threads and queued work;
|
||||
- how many database connections are open, and the WAL size;
|
||||
- on Linux, each thread's scheduler state and the kernel function it sleeps
|
||||
in (``D`` with a journal or writeback wait is a slow disk flush, a nanosleep
|
||||
is SQLite's busy handler, a futex is a Python lock or an idle worker).
|
||||
|
||||
When the reset ends it writes how long it took and, for each executor call it
|
||||
made, how long the call queued and how long it ran.
|
||||
|
||||
Every report line starts with REPORT_PREFIX. The worker fixture
|
||||
(tools/vendor/start-browser-server.mjs) copies those lines to the job log,
|
||||
so a slow reset is visible even in a test that still passes.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import contextvars
|
||||
import faulthandler
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from typing import Callable, TextIO
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
SLOW_RESET_SECONDS = 2.0
|
||||
REPORT_PREFIX = "pm-slow-reset| "
|
||||
|
||||
_current: contextvars.ContextVar["ResetWatch | None"] = contextvars.ContextVar(
|
||||
"pm_reset_watch", default=None
|
||||
)
|
||||
|
||||
|
||||
def is_reset_path(path: str) -> bool:
|
||||
return path.startswith("/__test__/") and path.endswith("/reset")
|
||||
|
||||
|
||||
class ResetWatch:
|
||||
"""Time one reset; dump the server's state if it passes the threshold."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
label: str,
|
||||
describe: Callable[[], list[str]],
|
||||
*,
|
||||
threshold: float = SLOW_RESET_SECONDS,
|
||||
stream: TextIO | None = None,
|
||||
) -> None:
|
||||
self.label = label
|
||||
self.threshold = threshold
|
||||
self._describe = describe
|
||||
self._stream = stream
|
||||
self._start = time.monotonic()
|
||||
self._calls: list[str] = []
|
||||
self._calls_lock = threading.Lock()
|
||||
self._stalled = threading.Event()
|
||||
self._timer = threading.Timer(threshold, self._report_stall)
|
||||
self._timer.daemon = True
|
||||
self._timer.name = "pm-slow-reset-watch"
|
||||
|
||||
def start(self) -> "ResetWatch":
|
||||
self._timer.start()
|
||||
return self
|
||||
|
||||
def elapsed(self) -> float:
|
||||
return time.monotonic() - self._start
|
||||
|
||||
def record_call(self, name: str, queued: float, ran: float) -> None:
|
||||
with self._calls_lock:
|
||||
self._calls.append(
|
||||
f"executor call {name}: queued {queued:.3f}s, ran {ran:.3f}s"
|
||||
)
|
||||
|
||||
def finish(self, outcome: str) -> None:
|
||||
self._timer.cancel()
|
||||
elapsed = self.elapsed()
|
||||
# A coarse clock (Windows) can fire the timer just short of the
|
||||
# threshold; a reset that was reported always gets its summary.
|
||||
if elapsed < self.threshold and not self._stalled.is_set():
|
||||
return
|
||||
with self._calls_lock:
|
||||
calls = list(self._calls)
|
||||
self._write([f"{self.label} {outcome} after {elapsed:.3f}s", *calls])
|
||||
|
||||
def _report_stall(self) -> None:
|
||||
self._stalled.set()
|
||||
lines = [f"{self.label} still running after {self.elapsed():.3f}s"]
|
||||
try:
|
||||
lines.extend(self._describe())
|
||||
except Exception as error: # a diagnostic never breaks the reset
|
||||
lines.append(f"state unavailable: {error!r}")
|
||||
lines.extend(thread_stacks())
|
||||
self._write(lines)
|
||||
|
||||
def _write(self, lines: list[str]) -> None:
|
||||
stream = self._stream or sys.stderr
|
||||
text = "".join(
|
||||
f"{REPORT_PREFIX}{part}\n" for line in lines for part in line.splitlines()
|
||||
)
|
||||
try:
|
||||
stream.write(text)
|
||||
stream.flush()
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def thread_stacks() -> list[str]:
|
||||
"""Every thread's stack, from faulthandler.
|
||||
|
||||
Formatting sys._current_frames() showed a thread blocked in SQLite as idle
|
||||
(Python 3.14); faulthandler reports where each thread really is.
|
||||
"""
|
||||
# Before Python 3.14 faulthandler prints ids only; name each id here.
|
||||
names = [
|
||||
f"thread 0x{thread.ident:016x} is {thread.name}"
|
||||
for thread in threading.enumerate()
|
||||
if thread.ident is not None
|
||||
]
|
||||
with tempfile.TemporaryFile() as dump:
|
||||
faulthandler.dump_traceback(file=dump, all_threads=True)
|
||||
dump.seek(0)
|
||||
return [*names, *dump.read().decode("utf-8", "replace").splitlines()]
|
||||
|
||||
|
||||
def executor_state(loop: asyncio.AbstractEventLoop) -> list[str]:
|
||||
executor = getattr(loop, "_default_executor", None)
|
||||
if executor is None:
|
||||
return ["default executor: not started"]
|
||||
threads = getattr(executor, "_threads", ())
|
||||
queue = getattr(executor, "_work_queue", None)
|
||||
queued = queue.qsize() if queue is not None else "?"
|
||||
return [
|
||||
f"default executor: {len(threads)} of "
|
||||
f"{getattr(executor, '_max_workers', '?')} threads, {queued} queued"
|
||||
]
|
||||
|
||||
|
||||
def database_state(model) -> list[str]:
|
||||
"""How many connections are open, and the WAL size.
|
||||
|
||||
Never touches another thread's connection: a connection busy in SQLite
|
||||
holds its mutex, and reading even ``in_transaction`` from here would block
|
||||
on it while holding the GIL, freezing every thread until that query ends.
|
||||
"""
|
||||
with model._conn_lock:
|
||||
count = len(model._connections)
|
||||
wal = model.db_path + "-wal"
|
||||
size = os.path.getsize(wal) if os.path.exists(wal) else 0
|
||||
return [f"database: {count} connections, WAL {size} bytes"]
|
||||
|
||||
|
||||
def kernel_wait_state() -> list[str]:
|
||||
"""Each thread's state and kernel wait channel, where /proc has them."""
|
||||
lines = []
|
||||
for thread in threading.enumerate():
|
||||
task = f"/proc/self/task/{thread.native_id}"
|
||||
try:
|
||||
with open(f"{task}/stat", encoding="ascii") as stat:
|
||||
state = stat.read().rsplit(")", 1)[1].split()[0]
|
||||
with open(f"{task}/wchan", encoding="ascii") as wchan:
|
||||
channel = wchan.read().strip() or "0"
|
||||
except (OSError, IndexError):
|
||||
continue
|
||||
# The id matches the "Thread 0x..." headers of the stacks below.
|
||||
lines.append(
|
||||
f"kernel: {thread.name} (0x{thread.ident:016x}) state {state}, "
|
||||
f"waiting in {channel}"
|
||||
)
|
||||
return lines
|
||||
|
||||
|
||||
def install(app: web.Application, api) -> None:
|
||||
"""Watch every reset route, and time the executor calls each one makes."""
|
||||
run_in_executor = api._run_in_executor
|
||||
|
||||
async def timed_run_in_executor(func, *args, **kwargs):
|
||||
watch = _current.get()
|
||||
if watch is None:
|
||||
return await run_in_executor(func, *args, **kwargs)
|
||||
queued_at = time.monotonic()
|
||||
started: list[float] = []
|
||||
|
||||
def timed(*call_args, **call_kwargs):
|
||||
started.append(time.monotonic())
|
||||
return func(*call_args, **call_kwargs)
|
||||
|
||||
try:
|
||||
return await run_in_executor(timed, *args, **kwargs)
|
||||
finally:
|
||||
end = time.monotonic()
|
||||
begin = started[0] if started else end
|
||||
watch.record_call(
|
||||
getattr(func, "__qualname__", repr(func)),
|
||||
begin - queued_at,
|
||||
end - begin,
|
||||
)
|
||||
|
||||
api._run_in_executor = timed_run_in_executor
|
||||
|
||||
@web.middleware
|
||||
async def watch_resets(request: web.Request, handler):
|
||||
if not is_reset_path(request.path):
|
||||
return await handler(request)
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
def describe() -> list[str]:
|
||||
return [
|
||||
*executor_state(loop),
|
||||
*database_state(api.db.model),
|
||||
*kernel_wait_state(),
|
||||
]
|
||||
|
||||
watch = ResetWatch(
|
||||
f"{request.method} {request.path}", describe, threshold=SLOW_RESET_SECONDS
|
||||
).start()
|
||||
token = _current.set(watch)
|
||||
outcome = "failed"
|
||||
try:
|
||||
response = await handler(request)
|
||||
outcome = f"answered {response.status}"
|
||||
return response
|
||||
except web.HTTPException as error:
|
||||
outcome = f"answered {error.status}"
|
||||
raise
|
||||
finally:
|
||||
_current.reset(token)
|
||||
watch.finish(outcome)
|
||||
|
||||
app.middlewares.append(watch_resets)
|
||||
@@ -0,0 +1,779 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { SIZES, THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate } from './concept-inventory.mjs';
|
||||
|
||||
// #249 System S2 Task 6: System's shell and its Overview section (app.js renderSystem 658–706), the Statistics dialog
|
||||
// ("Usage statistics", a1600–1604) and the health report (a1439–1456) against the immutable concept, on the
|
||||
// concept-shaped real fixture. Each concept measurement is mapped only by the recorded deviations (D92–D96, D121,
|
||||
// D122, D123), each commented where it applies; after that the difference must be empty, and a mutation of a
|
||||
// row, a label or a token must be noticed.
|
||||
test.use({ locale: 'en-US', timezoneId: 'UTC' });
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await seedConcept(request);
|
||||
await expect(await request.post('/__test__/system/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
page.conceptErrors = [];
|
||||
page.on('pageerror', error => page.conceptErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await request.post('/__test__/system/reset');
|
||||
expect(page.conceptErrors).toEqual([]);
|
||||
});
|
||||
|
||||
/** Runs in the page; self-contained. System's heading, section nav, Overview body and Save row under `mainSelector`. */
|
||||
function measureSystem(mainSelector) {
|
||||
const main = document.querySelector(mainSelector);
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const box = (node, origin = main) => {
|
||||
const rect = node.getBoundingClientRect(), base = origin.getBoundingClientRect();
|
||||
return { left: Math.round(rect.left - base.left), width: Math.round(rect.width) };
|
||||
};
|
||||
// A surface's top: from `.two-col` beside its neighbour, else from the bottom of the surface above it. This pins the
|
||||
// concept's `.surface + .surface` 18px offset, which concept.css applies inside `.two-col` too (copied, not ruled).
|
||||
const top = node => {
|
||||
const rect = node.getBoundingClientRect(), previous = node.previousElementSibling?.getBoundingClientRect();
|
||||
if (previous && rect.top >= previous.bottom) return Math.round(rect.top - previous.bottom);
|
||||
return Math.round(rect.top - node.parentElement.getBoundingClientRect().top);
|
||||
};
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const button = node => ({ text: text(node), className: node.className,
|
||||
...css(node, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopWidth', 'borderTopColor',
|
||||
'borderTopLeftRadius', 'opacity', 'cursor') });
|
||||
const heading = main.querySelector(':scope > .heading');
|
||||
const nav = main.querySelector(':scope > .section-nav');
|
||||
const layout = main.querySelector(':scope > .settings-layout');
|
||||
const metrics = layout.querySelector(':scope > .summary-metrics');
|
||||
const columns = layout.querySelector(':scope > .two-col');
|
||||
const save = layout.querySelector(':scope > .save-row');
|
||||
const firstMetric = metrics.querySelector('.metric');
|
||||
const firstSurface = columns.querySelector('.surface');
|
||||
const firstOperation = firstSurface.querySelector('.operation');
|
||||
return {
|
||||
tree: [...main.children].map(name),
|
||||
heading: {
|
||||
title: text(heading.querySelector('h1')), description: text(heading.querySelector('p.muted')),
|
||||
actions: [...(heading.querySelector('.heading-actions')?.children ?? [])].map(name),
|
||||
},
|
||||
nav: {
|
||||
role: nav.getAttribute('role'), label: nav.getAttribute('aria-label'),
|
||||
...css(nav, 'display', 'flexWrap', 'columnGap', 'rowGap', 'paddingBottom', 'marginBottom', 'borderBottomWidth', 'borderBottomColor'),
|
||||
box: box(nav),
|
||||
buttons: [...nav.children].map(item => ({ pressed: item.getAttribute('aria-pressed'), ...button(item) })),
|
||||
},
|
||||
layout: { tree: [...layout.children].map(name), ...css(layout, 'maxWidth'), box: box(layout) },
|
||||
metrics: {
|
||||
...css(metrics, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap', 'marginBottom'), box: box(metrics),
|
||||
items: [...metrics.children].map(item => ({ tree: [...item.children].map(name), value: text(item.querySelector('strong')),
|
||||
label: text(item.querySelector('span')), box: box(item) })),
|
||||
metric: css(firstMetric, 'paddingTop', 'paddingLeft', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius', 'backgroundColor'),
|
||||
strong: css(firstMetric.querySelector('strong'), 'display', 'fontSize', 'fontWeight', 'color'),
|
||||
span: css(firstMetric.querySelector('span'), 'fontSize', 'color', 'marginTop'),
|
||||
},
|
||||
columns: {
|
||||
...css(columns, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap'), box: box(columns),
|
||||
surfaces: [...columns.children].map(surface => ({ tree: [...surface.children].map(name), box: box(surface), top: top(surface),
|
||||
heading: text(surface.querySelector(':scope > h2')),
|
||||
rows: [...surface.querySelectorAll(':scope > .operation')].map(row => ({ tree: [...row.children].map(name),
|
||||
title: text(row.querySelector('h3')), description: text(row.querySelector('p')),
|
||||
action: button(row.querySelector('.row-actions > button')) })) })),
|
||||
surface: css(firstSurface, 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius'),
|
||||
h2: css(firstSurface.querySelector('h2'), 'fontSize', 'fontWeight', 'color', 'marginBottom'),
|
||||
operation: css(firstOperation, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom',
|
||||
'borderTopWidth', 'borderTopColor'),
|
||||
h3: css(firstOperation.querySelector('h3'), 'fontSize', 'fontWeight', 'color'),
|
||||
p: css(firstOperation.querySelector('p'), 'fontSize', 'color', 'marginTop'),
|
||||
actions: css(firstOperation.querySelector('.row-actions'), 'display', 'columnGap', 'flexShrink'),
|
||||
},
|
||||
save: {
|
||||
tree: [...save.children].map(name), ...css(save, 'display', 'alignItems', 'justifyContent', 'columnGap', 'marginTop', 'paddingTop'),
|
||||
status: { text: text(save.querySelector('#settings-status')), ...css(save.querySelector('#settings-status'), 'fontSize', 'color') },
|
||||
button: button(save.querySelector('button')),
|
||||
},
|
||||
pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth,
|
||||
};
|
||||
}
|
||||
|
||||
/** Runs in the page: the open dialog's title, body sequence, tables, operation rows and actions. */
|
||||
function measureDialog() {
|
||||
const dialog = document.querySelector('dialog#modal[open]');
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const body = dialog.querySelector('#modal-body');
|
||||
const firstCell = selector => body.querySelector(selector);
|
||||
const list = body.querySelector(':scope > .operation-list');
|
||||
const actions = body.querySelector(':scope > .dialog-actions');
|
||||
return {
|
||||
title: dialog.querySelector('#modal-title').textContent,
|
||||
body: [...body.children].map(name),
|
||||
tables: [...body.querySelectorAll(':scope > table')].map(table => ({
|
||||
caption: table.caption?.textContent ?? null,
|
||||
head: [...table.tHead.rows[0].cells].map(cell => [cell.localName, cell.scope, cell.textContent]),
|
||||
rows: [...table.tBodies[0].rows].map(row => [...row.cells].map(cell => cell.textContent)),
|
||||
})),
|
||||
tableStyle: firstCell(':scope > table') && {
|
||||
table: css(firstCell(':scope > table'), 'width', 'borderCollapse', 'fontSize', 'marginTop'),
|
||||
caption: css(firstCell(':scope > table caption'), 'textAlign', 'fontSize', 'fontWeight', 'color', 'paddingBottom'),
|
||||
th: css(firstCell(':scope > table thead th'), 'textAlign', 'fontSize', 'fontWeight', 'color', 'paddingTop', 'paddingLeft',
|
||||
'borderBottomWidth', 'borderBottomColor'),
|
||||
td: css(firstCell(':scope > table tbody td'), 'textAlign', 'fontSize', 'color', 'paddingTop', 'paddingLeft',
|
||||
'borderBottomWidth', 'borderBottomColor'),
|
||||
},
|
||||
operations: list && {
|
||||
...css(list, 'display', 'rowGap', 'marginTop'),
|
||||
rows: [...list.children].map(row => ({ tree: [...row.children].map(name), title: row.querySelector('h3').textContent,
|
||||
chip: row.querySelector('.chip').textContent })),
|
||||
operation: css(list.querySelector('.operation'), 'display', 'alignItems', 'justifyContent', 'paddingTop', 'borderTopWidth'),
|
||||
h3: css(list.querySelector('.operation h3'), 'fontSize', 'fontWeight', 'color'),
|
||||
chip: css(list.querySelector('.operation .chip'), 'fontSize', 'color', 'backgroundColor', 'borderTopLeftRadius', 'paddingLeft'),
|
||||
},
|
||||
actions: actions && {
|
||||
...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
|
||||
buttons: [...actions.children].map(item => ({ text: item.textContent, className: item.className,
|
||||
...css(item, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft') })),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Runs in the page: a `.notice` (Maintenance's "Check first…") with its copy. */
|
||||
function measureNotice(mainSelector) {
|
||||
const notice = document.querySelector(`${mainSelector} .settings-layout > .notice`);
|
||||
const css = (node, ...keys) => Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]]));
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const run = notice.querySelector(':scope > button');
|
||||
return {
|
||||
tree: [...notice.children].map(name), text: [...notice.querySelectorAll(':scope > div > *')].map(node => [node.localName, node.textContent]),
|
||||
frame: css(notice, 'display', 'alignItems', 'justifyContent', 'columnGap', 'borderTopWidth', 'borderTopColor', 'backgroundColor',
|
||||
'borderTopLeftRadius', 'paddingTop', 'paddingLeft', 'marginBottom'),
|
||||
h2: css(notice.querySelector('h2'), 'fontSize', 'fontWeight', 'color'),
|
||||
p: css(notice.querySelector('p'), 'fontSize', 'color', 'marginTop'),
|
||||
run: { text: run.textContent, className: run.className, ...css(run, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft') },
|
||||
};
|
||||
}
|
||||
|
||||
async function serverJson(request, url) {
|
||||
const response = await request.get(url);
|
||||
await expect(response).toBeOK();
|
||||
return response.json();
|
||||
}
|
||||
|
||||
const number = value => value.toLocaleString('en-US');
|
||||
const ratingText = value => value === null ? 'Not rated' : `${number(value)} / 5`;
|
||||
|
||||
/** The concept's shell and Overview with the recorded deviations applied, given the server's real overview. */
|
||||
function mapOverviewConcept(reference, overview) {
|
||||
const relabel = { 'Database integrity': 'Not checked', 'Positive prompts': number(overview.library.positive_prompts),
|
||||
'Negative variants': number(overview.library.negative_variants), Images: number(overview.images.indexed) };
|
||||
const rows = {
|
||||
// D93: the real pending count; the action is unavailable (dimmed, as the Save row's D92) until the review page.
|
||||
'3 ambiguous image links': row => ({ ...row,
|
||||
title: `${number(overview.needs_review.pending)} ambiguous image link${overview.needs_review.pending === 1 ? '' : 's'}`,
|
||||
action: { ...row.action, opacity: '0.55', cursor: 'not-allowed' } }),
|
||||
// D93: no backup has been recorded (verified backups arrive with S8).
|
||||
'Backup status': row => ({ ...row, description: 'No retained backup record.' }),
|
||||
// D93: there is no paging, so the shared settings are grid and density.
|
||||
'One display configuration': row => ({ ...row, description: 'Grid and density settings are shared across views.' }),
|
||||
// D93: the real sources, not samples.
|
||||
'2 sample output sources': row => ({ ...row, title: `${number(overview.sources.effective)} output sources` }),
|
||||
// D96: the dialog shows runs and ratings by category and library totals, not prompt, tag or date usage.
|
||||
Statistics: row => ({ ...row, description: 'Runs and ratings by category, with library totals.' }),
|
||||
};
|
||||
return {
|
||||
...reference,
|
||||
// D92: no "Sample configuration" chip.
|
||||
heading: { ...reference.heading, actions: [] },
|
||||
// D122: the concept's literal label is its prototype action name ("system-tab sections").
|
||||
nav: { ...reference.nav, label: 'System sections' },
|
||||
metrics: { ...reference.metrics, items: reference.metrics.items.map(item => ({ ...item, value: relabel[item.label] })) },
|
||||
columns: { ...reference.columns, surfaces: reference.columns.surfaces.map(surface => ({ ...surface,
|
||||
rows: surface.rows.map(row => (rows[row.title] ?? (value => value))(row)) })) },
|
||||
save: { ...reference.save,
|
||||
// D92: truthful status copy, and "Save preferences" is aria-disabled (dimmed) while nothing is unsaved.
|
||||
status: { ...reference.save.status, text: 'No unsaved changes.' },
|
||||
button: { ...reference.save.button, opacity: '0.55', cursor: 'not-allowed' } },
|
||||
};
|
||||
}
|
||||
|
||||
async function conceptOverview(page, theme) {
|
||||
await openConcept(page, 'system', theme);
|
||||
await expect(page.locator('#main .summary-metrics')).toBeVisible();
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureSystem, '#main');
|
||||
}
|
||||
|
||||
async function productionOverview(page, theme) {
|
||||
await openProduction(page, 'system/overview', theme);
|
||||
await expect(page.locator('#workspace-main .summary-metrics')).toBeVisible();
|
||||
// The thumbnail row is its own read (I2): compare once it answered.
|
||||
await expect(page.locator('#workspace-main .operation h3').filter({ hasText: 'thumbnails' })).toHaveText('No missing thumbnails');
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureSystem, '#workspace-main');
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`system shell and overview match the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
const overview = await serverJson(request, '/prompt_manager/system/overview');
|
||||
// The fixture's thumbnails all exist, so the concept's "No missing thumbnails" is also the real title.
|
||||
expect((await serverJson(request, '/prompt_manager/images/thumbnails/status')).pending).toBe(0);
|
||||
const concept = await conceptOverview(page, theme);
|
||||
expect(concept.heading.actions).toEqual(['span.chip']);
|
||||
// Paired captures are artifacts only; there are no PNG baselines.
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-system-${theme}-${width}.png`), fullPage: true });
|
||||
const production = await productionOverview(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-system-${theme}-${width}.png`), fullPage: true });
|
||||
expect(diffInventory(mapOverviewConcept(concept, overview), production)).toEqual([]);
|
||||
expect(production.pageScroll).toBe(false);
|
||||
});
|
||||
}
|
||||
|
||||
// The recorded differences, by path: D93 the review row (its real count, dimmed action), the backup row, the
|
||||
// preferences row and the sources row, and metric 4; D96 the statistics row; D92 the chip, the status and the dimmed
|
||||
// Save; D122 the nav label. (The fixture's first three metrics are the concept's own counts.)
|
||||
const UNMAPPED_PATHS = [
|
||||
'columns.surfaces.0.rows.0.action.cursor', 'columns.surfaces.0.rows.0.action.opacity',
|
||||
'columns.surfaces.0.rows.0.title', 'columns.surfaces.0.rows.1.description',
|
||||
'columns.surfaces.1.rows.0.description', 'columns.surfaces.1.rows.1.title', 'columns.surfaces.1.rows.2.description',
|
||||
'heading.actions.0', 'heading.actions.length', 'metrics.items.3.value', 'nav.label',
|
||||
'save.button.cursor', 'save.button.opacity', 'save.status.text',
|
||||
];
|
||||
|
||||
test('system comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const overview = await serverJson(request, '/prompt_manager/system/overview');
|
||||
const concept = await conceptOverview(page, 'dark');
|
||||
const expected = mapOverviewConcept(concept, overview);
|
||||
const production = await productionOverview(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the concept differs exactly where a deviation is recorded, and nowhere else.
|
||||
expect(diffInventory(concept, production).map(item => item.path).sort()).toEqual(UNMAPPED_PATHS);
|
||||
const measure = () => page.evaluate(measureSystem, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
|
||||
// A removed row.
|
||||
const undoRow = await mutate(page, { remove: ['#workspace-main .two-col > .surface:first-child > .operation:nth-of-type(2)'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['columns.surfaces.0.rows.length', 'columns.surfaces.0.tree.length']));
|
||||
await undoRow();
|
||||
// A changed label.
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .metric span').textContent = 'Positive prompt'; });
|
||||
expect(await paths()).toEqual(['metrics.items.0.label']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .metric span').textContent = 'Positive prompts'; });
|
||||
// A swapped token.
|
||||
const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['metrics.metric.borderTopColor', 'columns.surface.borderTopColor']));
|
||||
await undoToken();
|
||||
expect(diffInventory(expected, await measure())).toEqual([]);
|
||||
});
|
||||
|
||||
/** D96: the concept's table, its rows the real ones; then "Library totals" in the same table markup. */
|
||||
function mapStatisticsConcept(reference, statistics) {
|
||||
const [runs] = reference.tables;
|
||||
return {
|
||||
...reference,
|
||||
// No sample note (prototype copy); a second table follows the first.
|
||||
body: ['table', 'table'],
|
||||
tables: [
|
||||
{ ...runs, rows: statistics.by_category.map(row => [row.category ?? 'Uncategorized', number(row.runs), ratingText(row.average_rating)]) },
|
||||
{ caption: 'Library totals', head: [['th', 'col', 'Total'], ['th', 'col', 'Count']],
|
||||
rows: [['Prompts', number(statistics.totals.total_prompts)], ['Tags', number(statistics.totals.total_tags)],
|
||||
['Images', number(statistics.totals.total_images)], ['Rated images', number(statistics.rated_images)]] },
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// The fixture's image sources and each check's real chip on it: nothing is damaged, Needs review does not exist yet (Q6).
|
||||
const FIXTURE_SOURCES = ['output', 'input'];
|
||||
const FIXTURE_CHIPS = Object.freeze({ 'Integrity check': 'Passed', 'Foreign keys & pair references': 'Passed',
|
||||
'Canonical hashes': 'Passed', 'Pair uniqueness': 'Passed', 'Missing files': 'Passed', 'Ambiguous metadata': 'Not available yet',
|
||||
Consistency: 'Passed' });
|
||||
const chipOf = title => {
|
||||
if (!(title in FIXTURE_CHIPS)) throw new Error(`No expected chip for ${title}`);
|
||||
return FIXTURE_CHIPS[title];
|
||||
};
|
||||
|
||||
/**
|
||||
* D95: the concept's health report from its own five rows, labels verbatim, titled "Health report". Only the D95 rows
|
||||
* are inserted where they are recorded (Pair uniqueness after Canonical hashes, Consistency last); Missing files has a
|
||||
* row per image source when there are several; each chip is the fixture's expected result.
|
||||
*/
|
||||
function mapHealthConcept(reference, sources = FIXTURE_SOURCES) {
|
||||
const [first] = reference.operations.rows;
|
||||
const added = title => ({ ...first, title, chip: chipOf(title) });
|
||||
const rows = reference.operations.rows.flatMap(row => {
|
||||
const mapped = row.title === 'Missing files' && sources.length > 1
|
||||
? sources.map(source => ({ ...row, title: `Missing files · ${source}`, chip: chipOf(row.title) }))
|
||||
: [{ ...row, chip: chipOf(row.title) }];
|
||||
return row.title === 'Canonical hashes' ? [...mapped, added('Pair uniqueness')] : mapped;
|
||||
});
|
||||
return {
|
||||
...reference,
|
||||
title: 'Health report',
|
||||
// No sample note (prototype copy); D95 adds the checked line after the list.
|
||||
body: ['div.operation-list', 'p.small', 'div.dialog-actions'],
|
||||
operations: { ...reference.operations, rows: [...rows, added('Consistency')] },
|
||||
};
|
||||
}
|
||||
|
||||
// Unmapped, the health report differs exactly where D95 is recorded: the title, the body sequence (no sample note, the
|
||||
// checked line), the rows after Canonical hashes (Pair uniqueness inserted, Missing files per source) and the chips of
|
||||
// the concept's sample findings.
|
||||
const UNMAPPED_HEALTH_PATHS = [
|
||||
'body.0', 'body.1',
|
||||
...[3, 4].flatMap(row => [`operations.rows.${row}.chip`, `operations.rows.${row}.title`]),
|
||||
// The concept's five rows end at row 4; rows 5–7 exist only with the D95 rows and the second source.
|
||||
...[5, 6, 7].flatMap(row => ['chip', 'title', 'tree.0', 'tree.1', 'tree.length'].map(key => `operations.rows.${row}.${key}`)),
|
||||
'operations.rows.length', 'title',
|
||||
];
|
||||
|
||||
async function conceptDialogs(page, theme, testInfo) {
|
||||
await openConcept(page, 'system', theme);
|
||||
await page.locator('#main').getByRole('button', { name: 'Explore', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics');
|
||||
await page.mouse.move(0, 0);
|
||||
const statistics = await page.evaluate(measureDialog);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-statistics-${theme}.png`) });
|
||||
await page.keyboard.press('Escape');
|
||||
await page.locator('#main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click();
|
||||
const notice = await page.evaluate(measureNotice, '#main');
|
||||
await page.locator('#main .notice').getByRole('button', { name: 'Run health checks', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Sample health report');
|
||||
await page.mouse.move(0, 0);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-health-${theme}.png`) });
|
||||
return { statistics, notice, health: await page.evaluate(measureDialog) };
|
||||
}
|
||||
|
||||
async function productionDialogs(page, theme, testInfo) {
|
||||
await openProduction(page, 'system/overview', theme);
|
||||
await page.locator('#workspace-main').getByRole('button', { name: 'Explore', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics');
|
||||
await page.mouse.move(0, 0);
|
||||
const statistics = await page.evaluate(measureDialog);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-statistics-${theme}.png`) });
|
||||
await page.keyboard.press('Escape');
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click();
|
||||
await expect(page.locator('#workspace-main .notice')).toBeVisible();
|
||||
const notice = await page.evaluate(measureNotice, '#workspace-main');
|
||||
await page.locator('#workspace-main .notice').getByRole('button', { name: 'Run health checks', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Health report');
|
||||
await page.mouse.move(0, 0);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-health-${theme}.png`) });
|
||||
return { statistics, notice, health: await page.evaluate(measureDialog) };
|
||||
}
|
||||
|
||||
for (const theme of THEMES) test(`statistics and health dialogs match the concept ${theme}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const concept = await conceptDialogs(page, theme, testInfo);
|
||||
expect(concept.statistics.body).toEqual(['p.demo-note', 'table']);
|
||||
expect(concept.health.body).toEqual(['p.demo-note', 'div.operation-list', 'div.dialog-actions']);
|
||||
const production = await productionDialogs(page, theme, testInfo);
|
||||
const statistics = await serverJson(request, '/prompt_manager/system/statistics');
|
||||
const { report } = await serverJson(request, '/prompt_manager/system/health');
|
||||
// The fixture the expected chips describe: its two sources, and the concept's checks plus the D95 ones.
|
||||
const missing = report.checks.find(check => check.id === 'missing_files');
|
||||
expect(missing.sources.map(source => source.name)).toEqual(FIXTURE_SOURCES);
|
||||
expect(diffInventory(mapStatisticsConcept(concept.statistics, statistics), production.statistics)).toEqual([]);
|
||||
expect(diffInventory(mapHealthConcept(concept.health), production.health)).toEqual([]);
|
||||
// D123: Maintenance's notice is the concept's own, delivered ahead of the rest of the section.
|
||||
expect(diffInventory(concept.notice, production.notice)).toEqual([]);
|
||||
|
||||
// Unmapped, the dialogs differ only where D95 and D96 are recorded.
|
||||
expect(diffInventory(concept.health, production.health).map(item => item.path).sort()).toEqual(UNMAPPED_HEALTH_PATHS);
|
||||
expect(diffInventory(concept.statistics, production.statistics).map(item => item.path.split('.')[0])
|
||||
.filter((part, index, all) => all.indexOf(part) === index).sort()).toEqual(['body', 'tables']);
|
||||
|
||||
// Mutation check: a changed chip token in the health report is noticed, then undone to an exact match again.
|
||||
const undo = await mutate(page, { css: '#modal .operation .chip { background: rgb(255, 0, 0); }' });
|
||||
expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog)).map(item => item.path))
|
||||
.toEqual(['operations.chip.backgroundColor']);
|
||||
await undo();
|
||||
expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog))).toEqual([]);
|
||||
// A renamed check (database/health.py's label) is noticed against the concept's own label.
|
||||
await page.evaluate(() => { document.querySelector('#modal .operation h3').textContent = 'Integrity checks'; });
|
||||
expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog)).map(item => item.path))
|
||||
.toEqual(['operations.rows.0.title']);
|
||||
await page.evaluate(() => { document.querySelector('#modal .operation h3').textContent = 'Integrity check'; });
|
||||
// And a removed "Library totals" table and a changed caption in the statistics dialog.
|
||||
await page.keyboard.press('Escape');
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await page.locator('#workspace-main').getByRole('button', { name: 'Explore', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics');
|
||||
const expectedStatistics = mapStatisticsConcept(concept.statistics, statistics);
|
||||
const undoTable = await mutate(page, { remove: ['#modal-body > table:nth-of-type(2)'] });
|
||||
expect(diffInventory(expectedStatistics, await page.evaluate(measureDialog)).map(item => item.path))
|
||||
.toEqual(expect.arrayContaining(['body.length', 'tables.length']));
|
||||
await undoTable();
|
||||
await page.evaluate(() => { document.querySelector('#modal-body caption').textContent = 'Runs per category'; });
|
||||
expect(diffInventory(expectedStatistics, await page.evaluate(measureDialog)).map(item => item.path)).toEqual(['tables.0.caption']);
|
||||
});
|
||||
|
||||
/** Runs in the page: Appearance's surface, every row's text and control, and the concept's row geometry and tokens. */
|
||||
function measureAppearance(mainSelector) {
|
||||
const layout = document.querySelector(`${mainSelector} .settings-layout`);
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const surface = layout.querySelector(':scope > section.surface');
|
||||
const control = row => {
|
||||
const select = row.querySelector(':scope > select');
|
||||
if (select) return { kind: 'select', id: select.id, options: [...select.options].map(option => option.textContent),
|
||||
selected: select.selectedOptions[0]?.textContent ?? null };
|
||||
const input = row.querySelector(':scope > input');
|
||||
if (input) return { kind: `input.${input.type}`, id: input.id, value: input.value };
|
||||
const toggle = row.querySelector(':scope > * input[type="checkbox"]');
|
||||
if (toggle) return { kind: 'switch', id: toggle.id, wrapper: name(toggle.parentElement), checked: toggle.checked,
|
||||
...css(toggle.parentElement, 'width', 'height') };
|
||||
const chip = row.querySelector(':scope > .chip');
|
||||
return chip ? { kind: name(chip), text: text(chip), ...css(chip, 'color', 'backgroundColor', 'fontSize') } : null;
|
||||
};
|
||||
const rows = [...surface.querySelectorAll(':scope > .setting-row')];
|
||||
const operations = [...surface.querySelectorAll(':scope > .operation')];
|
||||
const firstRow = rows[0];
|
||||
return {
|
||||
layout: [...layout.children].map(name),
|
||||
surface: { tree: [...surface.children].map(name), heading: text(surface.querySelector(':scope > h2')),
|
||||
intro: text(surface.querySelector(':scope > p')),
|
||||
...css(surface, 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius') },
|
||||
rows: rows.map(row => ({ tree: [...row.children].map(name), text: [...row.firstElementChild.children].map(node => [node.localName, text(node),
|
||||
node.localName === 'label' ? node.htmlFor : null]), control: control(row) })),
|
||||
operations: operations.map(row => ({ tree: [...row.children].map(name), title: text(row.querySelector('h3')),
|
||||
description: text(row.querySelector('p')), action: text(row.querySelector('.row-actions > button')) })),
|
||||
row: css(firstRow, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', 'borderTopWidth',
|
||||
'borderTopColor', 'marginTop'),
|
||||
label: css(firstRow.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
|
||||
description: css(firstRow.querySelector('p'), 'fontSize', 'color', 'marginTop', 'maxWidth'),
|
||||
select: css(firstRow.querySelector('select'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor', 'color', 'borderTopColor'),
|
||||
heading3: css(surface.querySelector('.setting-row h3'), 'fontSize', 'fontWeight', 'color'),
|
||||
number: css(surface.querySelector('.setting-row input[type="number"]'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor'),
|
||||
pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth,
|
||||
};
|
||||
}
|
||||
|
||||
// D97: the rows Appearance adds for the legacy gallery settings, after the concept's own, as production renders them.
|
||||
const ADDED_APPEARANCE_ROWS = [
|
||||
['card-details', 'Show image details on cards', 'Show each card’s title and file details under the preview.', true],
|
||||
['viewer-details', 'Collapse viewer details by default', 'Open the image viewer with its details panel hidden.', false],
|
||||
];
|
||||
|
||||
/** The concept's Appearance with the recorded deviations applied, given the server's real settings. */
|
||||
function mapAppearanceConcept(reference, settings) {
|
||||
const toggleRow = reference.rows.find(row => row.control?.kind === 'switch');
|
||||
const added = ADDED_APPEARANCE_ROWS.map(([id, label, description, checked]) => ({
|
||||
tree: toggleRow.tree, text: [['label', label, id], ['p', description, null]],
|
||||
control: { ...toggleRow.control, id, checked } }));
|
||||
return {
|
||||
...reference,
|
||||
// D97: two added toggles and the "Reset display defaults" operation close the surface.
|
||||
surface: { ...reference.surface, tree: [...reference.surface.tree, 'div.setting-row', 'div.setting-row', 'div.operation'] },
|
||||
rows: [...reference.rows.map(row => row.control?.id === 'timeout'
|
||||
// The node search timeout is the server's real `result_timeout`, not the concept's sample 30.
|
||||
? { ...row, control: { ...row.control, value: String(settings.result_timeout) } } : row), ...added],
|
||||
operations: [{ tree: ['div', 'div.row-actions'], title: 'Reset display defaults',
|
||||
description: 'Restore this browser’s display defaults, then save. Other settings are kept.', action: 'Reset' }],
|
||||
};
|
||||
}
|
||||
|
||||
async function conceptAppearance(page, theme) {
|
||||
await openConcept(page, 'system', theme);
|
||||
await page.locator('#main .section-nav').getByRole('button', { name: 'Appearance', exact: true }).click();
|
||||
await expect(page.locator('#main .settings-layout > section.surface > h2')).toHaveText('Browsing & display');
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureAppearance, '#main');
|
||||
}
|
||||
|
||||
async function productionAppearance(page, theme) {
|
||||
await openProduction(page, 'system/appearance', theme);
|
||||
await expect(page.locator('#workspace-main #timeout')).toBeEnabled();
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureAppearance, '#workspace-main');
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`appearance matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
const { settings } = await serverJson(request, '/prompt_manager/settings');
|
||||
const concept = await conceptAppearance(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-appearance-${theme}-${width}.png`), fullPage: true });
|
||||
const production = await productionAppearance(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-appearance-${theme}-${width}.png`), fullPage: true });
|
||||
expect(diffInventory(mapAppearanceConcept(concept, settings), production)).toEqual([]);
|
||||
expect(production.pageScroll).toBe(false);
|
||||
});
|
||||
}
|
||||
|
||||
test('appearance comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const { settings } = await serverJson(request, '/prompt_manager/settings');
|
||||
const concept = await conceptAppearance(page, 'dark');
|
||||
const expected = mapAppearanceConcept(concept, settings);
|
||||
const production = await productionAppearance(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the concept differs only where D97 and the real timeout are recorded.
|
||||
expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 2).join('.'))
|
||||
.filter((path, index, all) => all.indexOf(path) === index).sort())
|
||||
.toEqual(['operations.0', 'operations.length', 'rows.11', 'rows.12', 'rows.13', 'rows.length', 'surface.tree'].sort());
|
||||
const measure = () => page.evaluate(measureAppearance, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
// A removed row.
|
||||
const undoRow = await mutate(page, { remove: ['#workspace-main .setting-row:nth-of-type(3)'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['rows.length', 'surface.tree.length']));
|
||||
await undoRow();
|
||||
// A changed label and a changed option.
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="density"]').textContent = 'Thumbnail sizes'; });
|
||||
expect(await paths()).toEqual(['rows.1.text.0.1']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="density"]').textContent = 'Thumbnail size'; });
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main #image-quality option').textContent = 'Medium'; });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['rows.5.control.options.0']));
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main #image-quality option').textContent = 'Optimized'; });
|
||||
// A swapped token.
|
||||
const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['row.borderTopColor']));
|
||||
await undoToken();
|
||||
expect(diffInventory(expected, await measure())).toEqual([]);
|
||||
});
|
||||
|
||||
/**
|
||||
* Runs in the page: Sources & scans (app.js 679–681): the optional notice, then the "Image sources" surface with its
|
||||
* panel heading, the source list, the two setting rows and the operations, with the concept's geometry and tokens.
|
||||
*/
|
||||
function measureSources(mainSelector) {
|
||||
const layout = document.querySelector(`${mainSelector} .settings-layout`);
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const button = node => ({ text: text(node), className: node.className,
|
||||
...css(node, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') });
|
||||
const operation = row => ({ tree: [...row.children].map(name), title: text(row.querySelector('h3')),
|
||||
description: text(row.querySelector('p')), actions: [...row.querySelectorAll('.row-actions > button')].map(button) });
|
||||
const notice = layout.querySelector(':scope > .notice');
|
||||
const surface = layout.querySelector(':scope > section.surface');
|
||||
const heading = surface.querySelector(':scope > .panel-heading');
|
||||
const list = surface.querySelector(':scope > #source-list');
|
||||
const control = row => {
|
||||
const toggle = row.querySelector(':scope > * input[type="checkbox"]');
|
||||
if (toggle) return { kind: 'switch', id: toggle.id, wrapper: name(toggle.parentElement), checked: toggle.checked,
|
||||
...css(toggle.parentElement, 'width', 'height') };
|
||||
const input = row.querySelector(':scope > input');
|
||||
return { kind: `input.${input.type}`, id: input.id, min: input.min, max: input.max, value: input.value,
|
||||
...css(input, 'width', 'accentColor') };
|
||||
};
|
||||
const rows = [...surface.querySelectorAll(':scope > .setting-row')];
|
||||
return {
|
||||
layout: [...layout.children].map(name),
|
||||
notice: notice && { tree: [...notice.children].map(name),
|
||||
text: [...notice.querySelectorAll(':scope > div > *')].map(node => [node.localName, text(node)]),
|
||||
actions: [...notice.querySelectorAll(':scope > .row-actions > button')].map(button),
|
||||
...css(notice, 'display', 'alignItems', 'columnGap', 'borderTopColor', 'backgroundColor', 'paddingTop', 'marginBottom') },
|
||||
surface: { tree: [...surface.children].map(name), intro: text(surface.querySelector(':scope > p')),
|
||||
...css(surface, 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopColor', 'borderTopLeftRadius') },
|
||||
heading: { tree: [...heading.children].map(name), title: text(heading.querySelector('h2')),
|
||||
action: button(heading.querySelector('button')), ...css(heading, 'display', 'alignItems', 'justifyContent', 'marginBottom'),
|
||||
h2: css(heading.querySelector('h2'), 'fontSize', 'fontWeight', 'color') },
|
||||
sources: [...list.children].map(operation),
|
||||
rows: rows.map(row => ({ tree: [...row.children].map(name), text: [...row.firstElementChild.children].map(node => [node.localName, text(node),
|
||||
node.localName === 'label' ? node.htmlFor : null]), control: control(row) })),
|
||||
operations: [...surface.querySelectorAll(':scope > .operation')].map(operation),
|
||||
operation: css(list.querySelector('.operation'), 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop',
|
||||
'paddingBottom', 'borderTopWidth', 'borderTopColor'),
|
||||
h3: css(list.querySelector('.operation h3'), 'fontSize', 'fontWeight', 'color'),
|
||||
p: css(list.querySelector('.operation p'), 'fontSize', 'color', 'marginTop'),
|
||||
row: css(rows[0], 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', 'borderTopColor'),
|
||||
pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth,
|
||||
};
|
||||
}
|
||||
|
||||
// D128: "watching until restart" after monitoring was switched off while the observer still runs.
|
||||
const WATCH_TEXT = { watching: 'watching', stopping: 'watching until restart', manual: 'manual scan', restart_pending: 'restart pending' };
|
||||
const WATCHED = new Set(['watching', 'stopping']);
|
||||
// D98: the operation Sources adds after the concept's own, as production renders it.
|
||||
const REFRESH_FILES = { title: 'Refresh files', description: 'Read every source again to find files added or removed outside ComfyUI.',
|
||||
label: 'Refresh' };
|
||||
|
||||
/** The concept's Sources with the recorded deviations applied, given the server's real settings and sources. */
|
||||
function mapSourcesConcept(reference, { settings, sources }) {
|
||||
const [sample] = reference.sources;
|
||||
const scan = sample.actions[0];
|
||||
return {
|
||||
...reference,
|
||||
// The real sources replace the two samples (without the "sample path" note). D98: each configured one has Remove,
|
||||
// a plain button after its scan.
|
||||
sources: sources.effective.map(source => ({ ...sample, title: source.label,
|
||||
description: `${source.path} · ${WATCH_TEXT[source.watch]}`,
|
||||
actions: [{ ...scan, text: WATCHED.has(source.watch) ? 'Scan changes' : 'Scan folder' }, { ...scan, text: 'Remove' }] })),
|
||||
rows: reference.rows.map(row => {
|
||||
if (row.control.id === 'monitoring') return { ...row, control: { ...row.control, checked: settings.monitoring_enabled } };
|
||||
// D113: the range ends at the server's core count (the concept's fixed 16) and shows the real worker count.
|
||||
return { ...row, control: { ...row.control, max: String(settings.cpu_count), value: String(settings.worker_threads) } };
|
||||
}),
|
||||
// D98: "Refresh files" closes the surface.
|
||||
surface: { ...reference.surface, tree: [...reference.surface.tree, 'div.operation'] },
|
||||
operations: [...reference.operations, { ...reference.operations[0], title: REFRESH_FILES.title,
|
||||
description: REFRESH_FILES.description, actions: [{ ...reference.operations[0].actions[0], text: REFRESH_FILES.label }] }],
|
||||
};
|
||||
}
|
||||
|
||||
async function conceptSources(page, theme, { notice }) {
|
||||
await openConcept(page, 'system', theme);
|
||||
await page.locator('#main .section-nav').getByRole('button', { name: 'Sources & scans', exact: true }).click();
|
||||
await expect(page.locator('#main .panel-heading > h2')).toHaveText('Image sources');
|
||||
if (!notice) await page.locator('#main .notice').getByRole('button', { name: 'Later', exact: true }).click();
|
||||
await expect(page.locator('#main .settings-layout > .notice')).toHaveCount(notice ? 1 : 0);
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureSources, '#main');
|
||||
}
|
||||
|
||||
async function productionSources(page, theme, { notice }) {
|
||||
// about:blank has no storage; the workspace page reads the dismissal.
|
||||
if (!notice) await page.addInitScript(() => { if (location.protocol.startsWith('http')) localStorage.setItem('pm_update_notice_dismissed', '3.2.0'); });
|
||||
await openProduction(page, 'system/sources', theme);
|
||||
await expect(page.locator('#workspace-main #source-list .operation')).toHaveCount(2);
|
||||
await expect(page.locator('#workspace-main #workers')).toBeEnabled();
|
||||
await expect(page.locator('#workspace-main .settings-layout > .notice')).toHaveCount(notice ? 1 : 0);
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureSources, '#workspace-main');
|
||||
}
|
||||
|
||||
/** The fixture's sources as the concept shows its samples: the first one watched, the second not. */
|
||||
async function watchedSources(request) {
|
||||
await expect(await request.post('/__test__/system/monitor', { data: { watching: ['output'] } })).toBeOK();
|
||||
return serverJson(request, '/prompt_manager/settings');
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) for (const notice of [true, false]) {
|
||||
const variant = notice ? 'with' : 'without';
|
||||
test(`sources matches the concept ${variant} the notice ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
const server = await watchedSources(request);
|
||||
const concept = await conceptSources(page, theme, { notice });
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-sources-${variant}-notice-${theme}-${width}.png`), fullPage: true });
|
||||
const production = await productionSources(page, theme, { notice });
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-sources-${variant}-notice-${theme}-${width}.png`), fullPage: true });
|
||||
expect(diffInventory(mapSourcesConcept(concept, server), production)).toEqual([]);
|
||||
expect(production.pageScroll).toBe(false);
|
||||
});
|
||||
}
|
||||
|
||||
test('sources comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const server = await watchedSources(request);
|
||||
const concept = await conceptSources(page, 'dark', { notice: true });
|
||||
const expected = mapSourcesConcept(concept, server);
|
||||
const production = await productionSources(page, 'dark', { notice: true });
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the concept differs only in the real sources, the D98 operation and the D113 range.
|
||||
expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 2).join('.'))
|
||||
.filter((path, index, all) => all.indexOf(path) === index).sort())
|
||||
.toEqual(['operations.2', 'operations.length', 'rows.1', 'sources.0', 'sources.1', 'surface.tree'].sort());
|
||||
const measure = () => page.evaluate(measureSources, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
// A removed notice and a removed source row.
|
||||
const undoNotice = await mutate(page, { remove: ['#workspace-main .settings-layout > .notice'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['layout.length', 'notice.tree.length']));
|
||||
await undoNotice();
|
||||
const undoRow = await mutate(page, { remove: ['#workspace-main #source-list > .operation:nth-child(2)'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['sources.length']));
|
||||
await undoRow();
|
||||
// A changed label and a changed button.
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="monitoring"]').textContent = 'Watch for outputs'; });
|
||||
expect(await paths()).toEqual(['rows.0.text.0.1']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="monitoring"]').textContent = 'Watch for new outputs'; });
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .panel-heading button').textContent = 'Add source'; });
|
||||
expect(await paths()).toEqual(['heading.action.text']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .panel-heading button').textContent = '+ Add source'; });
|
||||
// A swapped token.
|
||||
const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['operation.borderTopColor']));
|
||||
await undoToken();
|
||||
expect(diffInventory(expected, await measure())).toEqual([]);
|
||||
});
|
||||
|
||||
/**
|
||||
* Runs in the page: the open "Add image source" dialog (the concept's `simpleInput`, app.js 1138 and 1497): its title,
|
||||
* the body's tree, the field's label and input, and the actions, with the concept's geometry and tokens.
|
||||
*/
|
||||
function measureAddSource() {
|
||||
const dialog = document.querySelector('dialog#modal[open]');
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const body = dialog.querySelector('#modal-body');
|
||||
const field = body.querySelector(':scope > .field');
|
||||
const label = field?.querySelector(':scope > label');
|
||||
const input = field?.querySelector(':scope > input');
|
||||
const actions = body.querySelector(':scope > .dialog-actions');
|
||||
const rect = dialog.getBoundingClientRect();
|
||||
return {
|
||||
title: text(dialog.querySelector('#modal-title')),
|
||||
tree: [...body.children].map(name),
|
||||
field: field && { tree: [...field.children].map(name), ...css(field, 'display', 'flexDirection', 'rowGap', 'marginBottom') },
|
||||
label: label && { text: text(label), for: label.htmlFor, ...css(label, 'fontSize', 'fontWeight', 'color') },
|
||||
input: input && { id: input.id, type: input.type, value: input.value,
|
||||
...css(input, 'height', 'paddingLeft', 'fontSize', 'color', 'backgroundColor', 'borderTopColor', 'borderTopLeftRadius') },
|
||||
actions: { tree: [...actions.children].map(name), ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
|
||||
buttons: [...actions.children].map(button => ({ text: text(button), className: button.className,
|
||||
...css(button, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') })) },
|
||||
width: Math.round(rect.width),
|
||||
};
|
||||
}
|
||||
|
||||
/** D127: the field is "Folder path", starts empty (no sample folder) and its action is "Add". */
|
||||
function mapAddSourceConcept(reference) {
|
||||
const [cancel, apply] = reference.actions.buttons;
|
||||
return { ...reference, label: { ...reference.label, text: 'Folder path' }, input: { ...reference.input, value: '' },
|
||||
actions: { ...reference.actions, buttons: [cancel, { ...apply, text: 'Add' }] } };
|
||||
}
|
||||
|
||||
async function conceptAddSource(page, theme) {
|
||||
await conceptSources(page, theme, { notice: false });
|
||||
await page.locator('#main .panel-heading').getByRole('button', { name: '+ Add source', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Add image source');
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureAddSource);
|
||||
}
|
||||
|
||||
async function productionAddSource(page, theme) {
|
||||
await productionSources(page, theme, { notice: false });
|
||||
await page.locator('#workspace-main .panel-heading').getByRole('button', { name: '+ Add source', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Add image source');
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureAddSource);
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`add source dialog matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
await watchedSources(request);
|
||||
const concept = await conceptAddSource(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-add-source-${theme}-${width}.png`) });
|
||||
const production = await productionAddSource(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-add-source-${theme}-${width}.png`) });
|
||||
expect(diffInventory(mapAddSourceConcept(concept), production)).toEqual([]);
|
||||
});
|
||||
}
|
||||
|
||||
test('add source comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
await watchedSources(request);
|
||||
const concept = await conceptAddSource(page, 'dark');
|
||||
const expected = mapAddSourceConcept(concept);
|
||||
const production = await productionAddSource(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the concept differs only where D127 is recorded.
|
||||
expect(diffInventory(concept, production).map(item => item.path).sort())
|
||||
.toEqual(['actions.buttons.1.text', 'input.value', 'label.text']);
|
||||
const paths = async () => diffInventory(expected, await page.evaluate(measureAddSource)).map(item => item.path);
|
||||
// A changed label, a swapped action order and a changed style.
|
||||
await page.evaluate(() => { document.querySelector('dialog#modal[open] label[for="action-input"]').textContent = 'Folder'; });
|
||||
expect(await paths()).toEqual(['label.text']);
|
||||
await page.evaluate(() => { document.querySelector('dialog#modal[open] label[for="action-input"]').textContent = 'Folder path'; });
|
||||
await page.evaluate(() => { const actions = document.querySelector('dialog#modal[open] .dialog-actions'); actions.prepend(actions.lastElementChild); });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['actions.buttons.0.text', 'actions.buttons.1.text']));
|
||||
await page.evaluate(() => { const actions = document.querySelector('dialog#modal[open] .dialog-actions'); actions.prepend(actions.lastElementChild); });
|
||||
const undoStyle = await mutate(page, { css: 'dialog#modal #action-input { border-color: rgb(255, 0, 0); } dialog#modal .dialog-actions { gap: 31px; }' });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['input.borderTopColor', 'actions.columnGap']));
|
||||
await undoStyle();
|
||||
expect(diffInventory(expected, await page.evaluate(measureAddSource))).toEqual([]);
|
||||
});
|
||||
@@ -354,39 +354,48 @@ test('prompt deletion requires confirmation and can retry after a network failur
|
||||
}
|
||||
});
|
||||
|
||||
test('duplicate cleanup cancels safely and deletes only the selected file after a retry', async ({ page, request }) => {
|
||||
// The duplicate scan reads the output root only, but the fixture's input root holds a different
|
||||
// shared/same.png. A bare /images/serve/<path> URL tries every gallery root in turn, so once the
|
||||
// output copy is deleted it answers 200 with the input copy: check each file in its own root.
|
||||
const serveIn = (root, path) => `/prompt_manager/images/serve/${path.split(/[\\/]/).map(encodeURIComponent).join('/')}?root=${root}`;
|
||||
const rootOf = url => new URL(url, 'http://gallery.test').searchParams.get('root');
|
||||
|
||||
// Which copy the scan keeps depends on file times: the fixture stamps shared/same.png with the
|
||||
// wall clock, and the kernel can stamp its bulk copies a fraction of a millisecond earlier.
|
||||
// listOrder rewrites the finished scan's groups so a test can pick the order it needs.
|
||||
async function openDuplicateScan(page, request, listOrder = null) {
|
||||
await expect(await request.post('/__test__/reset', { data: { extra_images: 2 } })).toBeOK();
|
||||
if (listOrder) {
|
||||
await page.route('**/prompt_manager/scan_duplicates/stream', async route => {
|
||||
const response = await route.fetch();
|
||||
const body = (await response.text()).replace(/^data: (.*)$/gm, (line, json) => {
|
||||
const event = JSON.parse(json);
|
||||
if (event.type !== 'complete') return line;
|
||||
for (const group of event.duplicates) group.images = listOrder(group.images);
|
||||
return `data: ${JSON.stringify(event)}`;
|
||||
});
|
||||
await route.fulfill({ response, body });
|
||||
});
|
||||
}
|
||||
await page.goto('/prompt_manager/gallery');
|
||||
await expect(page.locator('#galleryGrid .image-item')).toHaveCount(4);
|
||||
const originals = await page.locator('#galleryGrid img[data-original]').evaluateAll(images => images.map(image => image.dataset.original));
|
||||
const output = rootOf(originals.find(url => url.includes('/bulk/')));
|
||||
const input = rootOf(originals.find(url => url.includes('/shared/same.png') && rootOf(url) !== output));
|
||||
expect(output).not.toBeNull();
|
||||
expect(input).not.toBeNull();
|
||||
await page.locator('#settingsBtn').click();
|
||||
await page.locator('#scanDuplicatesBtn').click();
|
||||
await expect(page.locator('#duplicatesCount')).toHaveText('1');
|
||||
const candidates = page.locator('.duplicate-checkbox');
|
||||
await expect(candidates).toHaveCount(2);
|
||||
const selected = await candidates.first().getAttribute('data-image-path');
|
||||
const retained = await candidates.last().getAttribute('data-image-path');
|
||||
const serve = path => `/prompt_manager/images/serve/${path.split(/[\\/]/).map(encodeURIComponent).join('/')}`;
|
||||
await candidates.first().check();
|
||||
let deletes = 0;
|
||||
page.on('request', request => { if (request.url().endsWith('/delete_duplicate_images')) deletes++; });
|
||||
confirmNextDialog(page, false, 'permanently delete 1 duplicate image files');
|
||||
await page.locator('#removeDuplicatesBtn').click();
|
||||
expect(deletes).toBe(0);
|
||||
await expect(await request.get(serve(selected))).toBeOK();
|
||||
await page.route('**/prompt_manager/delete_duplicate_images', route => route.abort('failed'), { times: 1 });
|
||||
confirmNextDialog(page, true, 'permanently delete 1 duplicate image files');
|
||||
await page.locator('#removeDuplicatesBtn').click();
|
||||
await expect(page.getByText('Failed to delete duplicate images', { exact: true })).toBeVisible();
|
||||
await expect(page.locator('#removeDuplicatesBtn')).toBeEnabled();
|
||||
await expect(await request.get(serve(selected))).toBeOK();
|
||||
confirmNextDialog(page, true, 'permanently delete 1 duplicate image files');
|
||||
const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images'));
|
||||
await page.locator('#removeDuplicatesBtn').click();
|
||||
expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 });
|
||||
await expect(page.locator('#duplicatesModal')).toBeHidden();
|
||||
expect((await request.get(serve(selected))).status()).toBe(404);
|
||||
await expect(await request.get(serve(retained))).toBeOK();
|
||||
return { originals, output, input, candidates };
|
||||
}
|
||||
|
||||
async function expectOnlySelectedGone(page, request, { originals, output, input }, selected, retained) {
|
||||
expect((await request.get(serveIn(output, selected))).status()).toBe(404);
|
||||
await expect(await request.get(serveIn(output, retained))).toBeOK();
|
||||
await expect(await request.get(serveIn(input, 'shared/same.png'))).toBeOK();
|
||||
await page.reload();
|
||||
await expect(page.locator('#galleryGrid .image-item')).toHaveCount(3);
|
||||
const remaining = await page.locator('#galleryGrid img[data-original]').evaluateAll(images => images.map(image => image.dataset.original));
|
||||
@@ -395,4 +404,47 @@ test('duplicate cleanup cancels safely and deletes only the selected file after
|
||||
expect(originals).toContain(url);
|
||||
await expect(await request.get(url)).toBeOK();
|
||||
}
|
||||
}
|
||||
|
||||
test('duplicate cleanup cancels safely and deletes only the selected file after a retry', async ({ page, request }) => {
|
||||
const scan = await openDuplicateScan(page, request);
|
||||
const { output, candidates } = scan;
|
||||
const selected = await candidates.first().getAttribute('data-image-path');
|
||||
const retained = await candidates.last().getAttribute('data-image-path');
|
||||
await candidates.first().check();
|
||||
let deletes = 0;
|
||||
page.on('request', request => { if (request.url().endsWith('/delete_duplicate_images')) deletes++; });
|
||||
confirmNextDialog(page, false, 'permanently delete 1 duplicate image files');
|
||||
await page.locator('#removeDuplicatesBtn').click();
|
||||
expect(deletes).toBe(0);
|
||||
await expect(await request.get(serveIn(output, selected))).toBeOK();
|
||||
await page.route('**/prompt_manager/delete_duplicate_images', route => route.abort('failed'), { times: 1 });
|
||||
confirmNextDialog(page, true, 'permanently delete 1 duplicate image files');
|
||||
await page.locator('#removeDuplicatesBtn').click();
|
||||
await expect(page.getByText('Failed to delete duplicate images', { exact: true })).toBeVisible();
|
||||
await expect(page.locator('#removeDuplicatesBtn')).toBeEnabled();
|
||||
await expect(await request.get(serveIn(output, selected))).toBeOK();
|
||||
confirmNextDialog(page, true, 'permanently delete 1 duplicate image files');
|
||||
const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images'));
|
||||
await page.locator('#removeDuplicatesBtn').click();
|
||||
expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 });
|
||||
await expect(page.locator('#duplicatesModal')).toBeHidden();
|
||||
await expectOnlySelectedGone(page, request, scan, selected, retained);
|
||||
});
|
||||
|
||||
test('deleting a duplicate whose path another gallery root also holds removes only the scanned copy', async ({ page, request }) => {
|
||||
// The order CI saw: a bulk copy stamped first, so shared/same.png is the first Delete candidate.
|
||||
const ciOrder = ['bulk/0000.png', 'shared/same.png', 'bulk/0001.png'];
|
||||
const rank = image => ciOrder.indexOf(image.path.replace(/\\/g, '/'));
|
||||
const scan = await openDuplicateScan(page, request, images => [...images].sort((a, b) => rank(a) - rank(b)));
|
||||
const selected = await scan.candidates.first().getAttribute('data-image-path');
|
||||
const retained = await scan.candidates.last().getAttribute('data-image-path');
|
||||
expect(selected.replace(/\\/g, '/')).toBe('shared/same.png');
|
||||
await scan.candidates.first().check();
|
||||
confirmNextDialog(page, true, 'permanently delete 1 duplicate image files');
|
||||
const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images'));
|
||||
await page.locator('#removeDuplicatesBtn').click();
|
||||
expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 });
|
||||
await expect(page.locator('#duplicatesModal')).toBeHidden();
|
||||
await expectOnlySelectedGone(page, request, scan, selected, retained);
|
||||
});
|
||||
|
||||
@@ -2,6 +2,13 @@ import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { seedConcept, openConcept, openProduction } from './concept-inventory.mjs';
|
||||
|
||||
// The Activity store's periodic list read (activity-store.mjs POLL_MS) and the detail view's read throttle
|
||||
// (activity-job-view.mjs DETAIL_INTERVAL_MS).
|
||||
const POLL_MS = 15_000;
|
||||
const DETAIL_INTERVAL_MS = 1_000;
|
||||
// A request is reported to the test a little after the browser sends it; that delay is not a shorter wait.
|
||||
const REPORT_SLACK_MS = 100;
|
||||
|
||||
// Activity (#249 Activity-1): the concept's Activity page (app.js renderActivity 732–738), its top-bar button
|
||||
// (index.html 51–52) and sidebar row (app.js ~341), with the recorded D52 (counts of running jobs), D53 (the progress
|
||||
// pill) and D54 (real job items). Fake jobs come from tests/browser/workspace_activity_fixture.py.
|
||||
@@ -277,7 +284,22 @@ test('reload during a scan reattaches to the running job', async ({ page, reques
|
||||
await expect(pill(page)).toBeHidden();
|
||||
});
|
||||
|
||||
test('an empty Activity says no job has run, and a list that cannot be read says so', async ({ page }) => {
|
||||
// The empty and failed states were only reached in passing by other tests, so their coverage depended on timing.
|
||||
const empty = main(page).locator('section.surface > p.empty-state');
|
||||
await page.goto(`${workspace}#/activity`);
|
||||
await expect(empty).toHaveText('No jobs have run since ComfyUI started. Scans and other long operations appear here.');
|
||||
await expect(empty).not.toHaveAttribute('role');
|
||||
await expect(items(page)).toHaveCount(0);
|
||||
await page.route(url => url.pathname === '/prompt_manager/jobs', route => route.abort());
|
||||
await page.reload();
|
||||
await expect(empty).toHaveText('Activity could not be read. It is tried again shortly.');
|
||||
await expect(empty).toHaveAttribute('role', 'alert');
|
||||
});
|
||||
|
||||
test('lost server clears the pill and marks jobs ended', async ({ page, request }) => {
|
||||
// A poll interval after the restart, then failed re-reads backing off for 1 + 2 + 4 s.
|
||||
test.setTimeout(90_000);
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const job = await startJob(request);
|
||||
await step(request, job.id, { processed: 4000 });
|
||||
@@ -294,9 +316,12 @@ test('lost server clears the pill and marks jobs ended', async ({ page, request
|
||||
await expect(items(page).first().locator('p')).toHaveText(LOST);
|
||||
await expect(items(page).first().locator('progress, .row-actions')).toHaveCount(0);
|
||||
await expect(navRow(page).locator('.nav-count')).toHaveCount(0);
|
||||
// One re-read found the restart; there is no retry storm and no finish toast for a job that only vanished.
|
||||
await page.waitForTimeout(2500);
|
||||
expect(reads.length).toBeLessThanOrEqual(2);
|
||||
// One re-read found the restart; there is no retry storm and no finish toast for a job that only vanished. The
|
||||
// read after it is the periodic one, a whole poll interval later: any extra read lands in that gap and fails it.
|
||||
const found = reads.length;
|
||||
expect(found, 'reads that found the restart').toBeGreaterThanOrEqual(1);
|
||||
await expect.poll(() => reads.length, { timeout: POLL_MS + 10_000 }).toBeGreaterThan(found);
|
||||
expect(reads[found] - reads[found - 1], 'the next read waits for the poll').toBeGreaterThanOrEqual(POLL_MS - REPORT_SLACK_MS);
|
||||
await expect(toast(page)).toHaveText('');
|
||||
|
||||
// A server that cannot be read at all also ends the running job, then is read again with a growing backoff.
|
||||
@@ -308,9 +333,12 @@ test('lost server clears the pill and marks jobs ended', async ({ page, request
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
await expect(pill(page)).toBeHidden();
|
||||
await expect(items(page).filter({ hasText: LOST })).toHaveCount(1);
|
||||
await page.waitForTimeout(3500);
|
||||
// 1 s, then 2 s: at most two re-reads in 3.5 s.
|
||||
expect(failed.length).toBeLessThanOrEqual(2);
|
||||
// 1 s, then 2 s, then 4 s: each failed re-read waits twice as long as the one before. The gaps between the
|
||||
// reads themselves are the evidence, so a slow runner noticing the ended job late cannot add a read to a window.
|
||||
await expect.poll(() => failed.length, { timeout: 20_000 }).toBeGreaterThanOrEqual(3);
|
||||
const gaps = failed.slice(1, 3).map((at, index) => at - failed[index]);
|
||||
expect(gaps[0]).toBeGreaterThanOrEqual(2000 - REPORT_SLACK_MS);
|
||||
expect(gaps[1]).toBeGreaterThanOrEqual(4000 - REPORT_SLACK_MS);
|
||||
await page.unroute('**/prompt_manager/jobs**');
|
||||
expect(second.id).not.toBe(job.id);
|
||||
});
|
||||
@@ -607,6 +635,8 @@ test('filter narrows the selected outcome', async ({ page, request }) => {
|
||||
|
||||
test('job view renders 200-row cap note', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
// The page's timers are Playwright's, running at real speed, so the M12 check below can run a read throttle out.
|
||||
await page.clock.install();
|
||||
const job = await startJob(request);
|
||||
await step(request, job.id, { processed: 260, bulk: { group: 'added', count: 250 }, records: [
|
||||
...Array.from({ length: 10 }, (_, index) => ({ group: 'no_metadata', path: `misc/plain_${index}.png`, detail: 'No prompt metadata in the file' })),
|
||||
@@ -646,11 +676,15 @@ test('job view renders 200-row cap note', async ({ page, request }) => {
|
||||
page.on('request', item => { if (new URL(item.url()).pathname === `/prompt_manager/jobs/${job.id}`) reads.push(Date.now()); });
|
||||
await step(request, job.id, { processed: 300, bulk: { group: 'added', count: 40 } });
|
||||
await expect(tiles(page).first().locator('dd')).toHaveText('290');
|
||||
await page.waitForTimeout(1500);
|
||||
// Every read the counter update could have scheduled is due within one throttle interval: run it out, so any such
|
||||
// read is sent before the log opens. Requests reach the test in the order they were sent, so once the log's read
|
||||
// has answered, a read sent before it would already be listed.
|
||||
await page.clock.runFor(DETAIL_INTERVAL_MS);
|
||||
expect(reads).toEqual([]);
|
||||
await scanLog(page).locator('summary').click();
|
||||
await expect.poll(() => reads.length).toBe(1);
|
||||
await expect(scanLog(page).locator('pre')).toContainText('bulk/file_00039.png');
|
||||
expect(reads, 'only the log\'s own read').toHaveLength(1);
|
||||
});
|
||||
|
||||
test('cancel scan ends cancelled with partial counts', async ({ page, request }) => {
|
||||
@@ -703,16 +737,55 @@ test('a held legacy job shows no Cancel in the list or its view', async ({ page,
|
||||
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
|
||||
});
|
||||
|
||||
// Since #289 an item dropped its Cancel job only when it was already in the page, so the first render of jobs the
|
||||
// store already knew (Activity opened from another page) kept a Cancel job on a job that had ended, and moved a
|
||||
// running job's progress after its actions. The read Activity starts on opening is held, so that first render is
|
||||
// what the page shows (the read's own render would otherwise repair it a moment later).
|
||||
test('jobs that ended before Activity opened show no Cancel job, a running one keeps its progress first', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const finished = await startJob(request, { title: 'Finished scan' });
|
||||
await finish(request, finished.id);
|
||||
const cancelled = await startJob(request, { title: 'Cancelled scan' });
|
||||
await finish(request, cancelled.id, 'cancelled');
|
||||
await startJob(request, { title: 'Running scan' });
|
||||
// The page's first list read has all three jobs once the running one shows in the pill.
|
||||
await page.goto(`${workspace}#/images`);
|
||||
await expect(pill(page)).toBeVisible();
|
||||
let hold;
|
||||
const held = new Promise(resolve => { hold = resolve; });
|
||||
// Only the first read is held; any later one goes through.
|
||||
await page.route(url => url.pathname === '/prompt_manager/jobs', route => {
|
||||
if (!hold) return route.continue();
|
||||
hold(route);
|
||||
hold = null;
|
||||
});
|
||||
await activityButton(page).click();
|
||||
const route = await held;
|
||||
await expect(items(page)).toHaveCount(3);
|
||||
const item = title => items(page).filter({ has: page.getByRole('link', { name: new RegExp(`${title}$`) }) });
|
||||
for (const ended of ['Finished scan', 'Cancelled scan']) {
|
||||
await expect(item(ended).locator('progress, .row-actions')).toHaveCount(0);
|
||||
await expect(item(ended).getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(0);
|
||||
}
|
||||
await expect(item('Running scan').getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(1);
|
||||
expect(await item('Running scan').evaluate(node => [...node.children].map(child => child.localName + (child.className ? `.${child.className}` : ''))))
|
||||
.toEqual(['div.panel-heading', 'p', 'progress', 'div.row-actions']);
|
||||
await route.continue();
|
||||
});
|
||||
|
||||
test('busy start names the running job', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
// The page's timers are Playwright's, so the toast's lifetime below is run out rather than waited for.
|
||||
await page.clock.install();
|
||||
const running = await startJob(request, { title: 'Database maintenance', kind: 'maintenance' });
|
||||
await startFromImport(page);
|
||||
await expect(toast(page)).toHaveText('Another job is running: Database maintenance.');
|
||||
await expect(toast(page).getByRole('link', { name: 'Database maintenance', exact: true })).toHaveAttribute('href', `#/activity/${running.id}`);
|
||||
await expect(page).toHaveURL(/#\/images$/);
|
||||
expect((await listed(request)).map(item => item.id)).toEqual([running.id]);
|
||||
// Review M14: a toast with a link outlasts the plain 6.5 s, so the link stays reachable.
|
||||
await page.waitForTimeout(7000);
|
||||
// Review M14: a toast with a link outlasts the plain 6.5 s, so the link stays reachable. Running the page's timers
|
||||
// 7 s on fires a 6.5 s clear for certain, however slowly this runner reached this line.
|
||||
await page.clock.runFor(7000);
|
||||
await expect(toast(page).getByRole('link', { name: 'Database maintenance', exact: true })).toBeVisible();
|
||||
await toast(page).getByRole('link', { name: 'Database maintenance', exact: true }).click();
|
||||
await expect(heading(page).locator('h1')).toHaveText('Database maintenance');
|
||||
|
||||
@@ -494,6 +494,23 @@ test('the inline current file names its prompts apart from the frozen run fields
|
||||
expect(await exact(dialog(page).getByRole('region',{name:'Positive prompt',exact:true}).locator('p.prompt-text'))).toBe(before);
|
||||
});
|
||||
|
||||
// LI-4 (D117): the recorded history's current-file viewer is the shared full inspector, so it copies the legacy summary.
|
||||
test('the current file viewer copies its settings summary',async({page})=>{
|
||||
await page.addInitScript(()=>{window.__copied=[];Object.defineProperty(navigator,'clipboard',{configurable:true,value:{writeText:async text=>{window.__copied.push(text);}}});});
|
||||
await page.route(url=>url.pathname.endsWith('/inspection'),async route=>{const response=await route.fetch();const body=await response.json();
|
||||
body.prompts={positive:{status:'resolved',text:'current file positive',source:'graph-inferred'},negative:{status:'resolved',text:'current file negative',source:'graph-inferred'}};
|
||||
await route.fulfill({response,json:body});});
|
||||
await open(page,page.fixture.detail_id);await marker(page,page.fixture.runs.shared).click();await expect(dialog(page).locator('[data-receipt]')).toHaveCount(5);
|
||||
await dialog(page).locator('[data-receipt]').first().getByRole('button',{name:'View current file',exact:true}).click();
|
||||
const details=page.locator('dialog#viewer-dialog #viewer-details');await expect(details.locator('.panel-heading')).toBeVisible();
|
||||
const facts=await details.locator(':scope > div > dl.meta-grid dd').allTextContents();
|
||||
await details.getByRole('button',{name:'Copy settings summary',exact:true}).click();
|
||||
await expect(page.locator('#workspace-toast')).toHaveText('Settings summary copied.');
|
||||
const [seed,steps,cfg,sampler,,model]=facts;
|
||||
expect(await page.evaluate(()=>window.__copied)).toEqual([[`Checkpoint: ${model}`,'Positive Prompt: current file positive','Negative Prompt: current file negative',
|
||||
`Steps: ${steps}`,`CFG Scale: ${cfg}`,`Sampler: ${sampler}`,`Seed: ${seed}`].join('\n').replace(/: (Not recorded|Unresolved|Ambiguous|Unavailable within inspection limits)(?=\n|$)/g,': Unknown')]);
|
||||
});
|
||||
|
||||
test('the recorded run dialog with a resolved current file has no duplicate button or region names',async({page})=>{
|
||||
await page.route(url=>url.pathname.endsWith('/inspection'),async route=>{const response=await route.fetch();const body=await response.json();
|
||||
body.prompts={positive:{status:'resolved',text:'current file positive',source:'graph-inferred'},negative:{status:'resolved',text:'current file negative',source:'graph-inferred'}};
|
||||
|
||||
@@ -463,6 +463,50 @@ test('collection viewer ignores delayed media callbacks and keeps inspection ret
|
||||
await page.evaluate(()=>{location.hash='/overview';});await expect(page.locator('.viewer-container,#viewer-dialog')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('collection viewer leaving an image Viewer.js has not built yet releases it at once',async({page})=>{
|
||||
await page.goto(workspace);await expect(cards(page)).toHaveCount(2);
|
||||
let release,held;const waiting=new Promise(resolve=>{held=resolve;});const pending=new Promise(resolve=>{release=resolve;});
|
||||
await page.route('**/images/serve/shared/same.png?*',async route=>{
|
||||
const url=new URL(route.request().url());
|
||||
if(url.searchParams.get('root')==='0'&&url.searchParams.has('_pm_viewer')) {held();await pending;}
|
||||
await route.continue().catch(()=>{});
|
||||
});
|
||||
await cards(page).first().getByRole('button',{name:'Open same.png',exact:true}).click();await waiting;
|
||||
const preview=page.locator('dialog#viewer-dialog');
|
||||
await page.evaluate(()=>{window.unbuiltImage=document.querySelector('#viewer-stage > img');});
|
||||
await preview.getByRole('button',{name:'Next media',exact:true}).click();
|
||||
await expect(preview.getByRole('button',{name:'Zoom in',exact:true})).toHaveAttribute('aria-disabled','false');
|
||||
// The left image is off the stage. Viewer.js has not built it, so the viewer is destroyed and the src released at
|
||||
// once: its late load must never build Viewer.js into an image without a parent (getComputedStyle(null)).
|
||||
release();
|
||||
await expect.poll(()=>page.evaluate(()=>[window.unbuiltImage.isConnected,window.unbuiltImage.hasAttribute('src'),!!window.unbuiltImage.viewer])).toEqual([false,false,false]);
|
||||
await page.evaluate(()=>{location.hash='/overview';});await expect(page.locator('.viewer-container,#viewer-dialog')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('collection viewer closed while its image finishes loading never builds Viewer.js afterwards',async({page})=>{
|
||||
await page.goto(workspace);await expect(cards(page)).toHaveCount(2);
|
||||
let held;const waiting=new Promise(resolve=>{held=resolve;});
|
||||
// The image is held for the whole test: its late load is delivered below, at the moment Viewer.js cannot cancel.
|
||||
await page.route('**/images/serve/shared/same.png?*',async route=>{
|
||||
const url=new URL(route.request().url());
|
||||
if(url.searchParams.get('root')==='0'&&url.searchParams.has('_pm_viewer')) {held();return;}
|
||||
await route.continue().catch(()=>{});
|
||||
});
|
||||
await cards(page).first().getByRole('button',{name:'Open same.png',exact:true}).click();await waiting;
|
||||
const preview=page.locator('dialog#viewer-dialog');
|
||||
// Viewer.js's own cancel removes its image listeners only while `image.complete` is false. An image that has
|
||||
// finished downloading but whose load event is still queued is already complete: close in that state, then deliver it.
|
||||
await page.evaluate(()=>{window.finishingImage=document.querySelector('#viewer-stage > img');Object.defineProperty(window.finishingImage,'complete',{get:()=>true});});
|
||||
await page.keyboard.press('Escape');await expect(preview).toHaveCount(0);
|
||||
const after=await page.evaluate(async()=>{
|
||||
const image=window.finishingImage;image.dispatchEvent(new Event('load'));
|
||||
// Viewer.js builds an inline viewer in a zero-delay timeout after its image loads; this runs after it.
|
||||
await new Promise(resolve=>setTimeout(resolve));
|
||||
return {src:image.hasAttribute('src'),built:!!image.parentNode?.querySelector('.viewer-container')};
|
||||
});
|
||||
expect(after).toEqual({src:false,built:false});
|
||||
});
|
||||
|
||||
test('collection viewer describes video transforms and releases native playback on navigation',async({page,request})=>{
|
||||
await expect(await request.post('/__test__/reset',{data:{include_video:true}})).toBeOK();await page.goto(workspace);
|
||||
await page.locator('article[data-image-key="0:clip.webm"] .image-open').click();
|
||||
@@ -907,6 +951,10 @@ test('browser Back closes the open Images dialog and leaves the next page usable
|
||||
|
||||
test('count line shows images and positive prompts', async ({ page, request }) => {
|
||||
await page.addInitScript(() => { window.IntersectionObserver = undefined; });
|
||||
// D17's "Today" is the newest file's day: the page's clock stands at that file's mtime, so the fixture's day and
|
||||
// the page's agree on either side of midnight.
|
||||
const { images } = await (await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json();
|
||||
await page.clock.setFixedTime(new Date(Math.max(...images.map(image => image.modified_time)) * 1000));
|
||||
await page.goto(workspace);
|
||||
await expect(cards(page)).toHaveCount(2);
|
||||
await expect(caption(page).locator('#result-count')).toHaveText('2 of 2 images · 2 positive prompts');
|
||||
|
||||
@@ -207,6 +207,40 @@ test('chooser opens the shared full inspector without a second read', async ({ p
|
||||
expect(await page.evaluate(() => window.__slices)).toBe(0);
|
||||
});
|
||||
|
||||
// PR L3 Task 10: local roles are the server inspector's (local-resolver.mjs), so for every parity PNG the rail shows the
|
||||
// prompts, their status lines and the facts Images shows for the same bytes in the output folder.
|
||||
const PARITY_PNGS = ['f01b', 'f02', 'f03', 'f06a', 'f06b', 'f06c', 'f06d', 'f06e', 'f07a', 'f07b', 'f08', 'f09', 'f10', 'f11', 'f12',
|
||||
'f14', 'f15', 'f16', 'f17', 'f19', 'f20', 'f21', 'f22', 'f23', 'f24'];
|
||||
const roleSections = locator => locator.evaluate(node => [...node.querySelectorAll('.prompt-section')].map(item => item.textContent));
|
||||
|
||||
test('chooser shows the Images roles and resolution for every parity PNG', async ({ page, request }) => {
|
||||
test.slow();
|
||||
for (const name of PARITY_PNGS) {
|
||||
await expect(await request.post('/__test__/add-local-inspect-file', { data: { name: `${name}.png` } })).toBeOK();
|
||||
}
|
||||
const served = {};
|
||||
await page.goto(`${workspace}#/images`);
|
||||
for (const name of PARITY_PNGS) {
|
||||
await page.locator(`article[data-image-key="0:local-inspect/${name}.png"] .image-open`).click();
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText(`${name}.png`);
|
||||
await expect(rail(page).locator('.panel-heading')).toBeVisible();
|
||||
await expect(rail(page).locator('.prompt-section')).not.toHaveCount(0);
|
||||
served[name] = { content: await railContent(rail(page)), roles: await roleSections(rail(page)) };
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
}
|
||||
await openInspect(page);
|
||||
for (const name of PARITY_PNGS) {
|
||||
await choose(page, parityPng(`${name}.png`));
|
||||
await openViewer(page);
|
||||
await expect(viewer(page).locator('#viewer-title')).toHaveText(`${name}.png`);
|
||||
await expect(rail(page).locator('.prompt-section')).not.toHaveCount(0);
|
||||
expect({ content: await railContent(rail(page)), roles: await roleSections(rail(page)) }, name).toEqual(served[name]);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
}
|
||||
});
|
||||
|
||||
test('Import → Inspect a local image lands here', async ({ page }) => {
|
||||
await page.goto(`${workspace}#/images`);
|
||||
await page.getByRole('button', { name: 'Import', exact: true }).click();
|
||||
@@ -481,15 +515,13 @@ test('replacing a video that just started loading never blocks the page', async
|
||||
});
|
||||
|
||||
test('a video replaced while still starting is reset once it settles', async ({ context, browserName }) => {
|
||||
// A WebKit (GStreamer) video released exactly at its `loadstart` sometimes blocks the page, with or without any
|
||||
// reset and also with the code before the deferred reset (2 of 24 runs); tracked in TESTING.md (D62). WebKit's
|
||||
// no-hang guard is `replacing a video that just started loading never blocks the page`.
|
||||
test.skip(browserName === 'webkit', 'WebKit release at loadstart is a tracked follow-up (D62)');
|
||||
test.setTimeout(120000);
|
||||
// The video is replaced from its own `loadstart` (readyState HAVE_NOTHING). It leaves the page with no src before
|
||||
// its URL is revoked, and is reset (load() with no src, which frees its decoder) only once it reports metadata or
|
||||
// an error, so the page never blocks. Fresh pages share one browser process.
|
||||
for (let attempt = 0; attempt < 5; attempt++) {
|
||||
test.setTimeout(180000);
|
||||
// The video is replaced from its own `loadstart` (readyState HAVE_NOTHING), or a few milliseconds after it. On
|
||||
// WebKit (GStreamer) a video that left the page or the viewport before its pipeline had started deadlocked the
|
||||
// page (TESTING.md D62), so it stays in place, inert and still playing its URL, with the new preview after it.
|
||||
// Once it reports metadata or an error it is paused, loses its src, is reset (load() with no src, which frees its
|
||||
// decoder) and leaves the page, and only then is its URL revoked. Fresh pages share one browser process.
|
||||
for (let attempt = 0; attempt < 15; attempt++) {
|
||||
const page = await context.newPage();
|
||||
try {
|
||||
if (attempt === 0) await startCoverage(page, browserName);
|
||||
@@ -510,10 +542,12 @@ test('a video replaced while still starting is reset once it settles', async ({
|
||||
const video = slot.querySelector('video');
|
||||
if (!video) return;
|
||||
observer.disconnect();
|
||||
video.addEventListener('loadstart', () => {
|
||||
const replace = () => {
|
||||
window.__starting = { node: video, url: video.getAttribute('src'), readyState: video.readyState };
|
||||
input.files = window.__pngFiles; input.dispatchEvent(new Event('change', { bubbles: true }));
|
||||
}, { once: true });
|
||||
window.__starting.replaced = { connected: video.isConnected, src: video.hasAttribute('src'), inert: video.inert, resets: video.__resets ?? 0 };
|
||||
};
|
||||
video.addEventListener('loadstart', () => { if (window.__delay) setTimeout(replace, window.__delay); else replace(); }, { once: true });
|
||||
}).observe(slot, { childList: true });
|
||||
});
|
||||
const png = parityPng('f02.png');
|
||||
@@ -522,13 +556,17 @@ test('a video replaced while still starting is reset once it settles', async ({
|
||||
transfer.items.add(new File([new Uint8Array(bytes)], 'f02.png', { type: 'image/png' }));
|
||||
window.__pngFiles = transfer.files;
|
||||
}, [...png.buffer]);
|
||||
await page.evaluate(delay => { window.__delay = delay; }, attempt % 5);
|
||||
await choose(page, localFixture('core-tags.webm', 'video/webm'));
|
||||
await expect(preview(page).locator('img.local-file-preview')).toBeVisible({ timeout: 8000 });
|
||||
await expect(preview(page).locator('video')).toHaveCount(0);
|
||||
// Released while starting, and never reset synchronously: the URL went once the node had no src.
|
||||
expect(await page.evaluate(() => ({ readyState: window.__starting.readyState, revoked: window.__starting.revoked })))
|
||||
.toEqual({ readyState: 0, revoked: { src: false, connected: false, resets: 0 } });
|
||||
await expect.poll(() => page.evaluate(() => window.__starting.node.__resets ?? 0), { timeout: 15000 }).toBe(1);
|
||||
// Replaced while starting (always so at `loadstart` itself): kept in place and untouched. A video that had
|
||||
// settled is released at once. Either way it is reset once, and its URL goes last.
|
||||
await expect.poll(() => page.evaluate(() => window.__starting.revoked ?? null)).toEqual({ src: false, connected: false, resets: 1 });
|
||||
const { readyState, replaced } = await page.evaluate(() => ({ readyState: window.__starting.readyState, replaced: window.__starting.replaced }));
|
||||
if (attempt % 5 === 0) expect(readyState).toBe(0);
|
||||
expect(replaced).toEqual(readyState === 0 ? { connected: true, src: true, inert: true, resets: 0 } : { connected: false, src: false, inert: false, resets: 1 });
|
||||
expect(await page.evaluate(() => window.__starting.node.__resets)).toBe(1);
|
||||
// The page still answers.
|
||||
await chooser(page).setInputFiles(parityPng('f02.png', 'after.png'), { timeout: 8000 });
|
||||
await expect(preview(page).locator('img.local-file-preview')).toBeVisible({ timeout: 8000 });
|
||||
@@ -539,6 +577,54 @@ test('a video replaced while still starting is reset once it settles', async ({
|
||||
}
|
||||
});
|
||||
|
||||
test('leaving for another route while a video starts waits for it to settle', async ({ context, browserName }) => {
|
||||
test.setTimeout(180000);
|
||||
// A WebKit (GStreamer) video detached before its pipeline had started deadlocked the page (TESTING.md D62); a
|
||||
// route change detaches the whole page, so the shell renders the next route once the video reports metadata or an
|
||||
// error. The route changes from the video's own `loadstart`, or a few milliseconds after it.
|
||||
for (let attempt = 0; attempt < 15; attempt++) {
|
||||
const page = await context.newPage();
|
||||
try {
|
||||
if (attempt === 0) await startCoverage(page, browserName);
|
||||
await page.addInitScript(() => {
|
||||
const load = HTMLMediaElement.prototype.load;
|
||||
HTMLMediaElement.prototype.load = function () {
|
||||
if (!this.hasAttribute('src') && window.__starting?.node === this) window.__starting.reset = { readyState: this.readyState, failed: this.error !== null, connected: this.isConnected };
|
||||
return load.call(this);
|
||||
};
|
||||
});
|
||||
await openInspect(page);
|
||||
await page.evaluate(delay => {
|
||||
const slot = document.querySelector('#local-preview');
|
||||
new MutationObserver((records, observer) => {
|
||||
const video = slot.querySelector('video');
|
||||
if (!video) return;
|
||||
observer.disconnect();
|
||||
const leave = () => {
|
||||
window.__starting = { node: video, readyState: video.readyState };
|
||||
location.hash = '#/overview';
|
||||
};
|
||||
video.addEventListener('loadstart', () => { if (delay) setTimeout(leave, delay); else leave(); }, { once: true });
|
||||
}).observe(slot, { childList: true });
|
||||
}, attempt % 5);
|
||||
await choose(page, localFixture('core-tags.webm', 'video/webm'));
|
||||
await expect(main(page).getByRole('heading', { level: 1, name: 'Overview', exact: true })).toBeVisible({ timeout: 8000 });
|
||||
const { readyState, reset } = await page.evaluate(() => ({ readyState: window.__starting.readyState, reset: window.__starting.reset }));
|
||||
if (attempt % 5 === 0) expect(readyState).toBe(0);
|
||||
// The video was reset and left the page only once it had settled.
|
||||
expect(reset.readyState >= 1 || reset.failed).toBe(true);
|
||||
expect(reset.connected).toBe(true);
|
||||
// The page still answers.
|
||||
await page.locator('aside.sidebar nav').getByRole('link', { name: 'Inspect a file', exact: true }).click();
|
||||
await chooser(page).setInputFiles(parityPng('f02.png'), { timeout: 8000 });
|
||||
await expect(preview(page).locator('img.local-file-preview')).toBeVisible({ timeout: 8000 });
|
||||
if (attempt === 0) await saveCoverage(page, browserName);
|
||||
} finally {
|
||||
await page.close();
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('closing the viewer on a video that just started never blocks the page', async ({ context, browserName }) => {
|
||||
test.setTimeout(180000);
|
||||
// The viewer's video teardown resets its element; on WebKit a reset of a still-starting pipeline blocked the page.
|
||||
@@ -652,13 +738,19 @@ test('local roles send exactly', async ({ page, context }) => {
|
||||
|
||||
test('local inspection makes no content request or storage write', async ({ page }) => {
|
||||
await instrument(page);
|
||||
await openInspect(page);
|
||||
// The shell's own reads (counts, Activity) settle before the baseline.
|
||||
await expect(page.locator('aside.sidebar nav [data-route="images"] .nav-count')).toBeAttached();
|
||||
const baseline = (await urls(page)).writes.length;
|
||||
const requests = [];
|
||||
page.on('request', request => requests.push({ url: request.url(), method: request.method(), body: request.postData() }));
|
||||
page.on('websocket', socket => requests.push({ url: socket.url(), method: 'WS', body: null }));
|
||||
const table = /\/prompt_manager\/js\/inspection-semantics\.json$/;
|
||||
const tableRead = page.waitForResponse(response => table.test(response.url()));
|
||||
await openInspect(page);
|
||||
// The one static file Inspect reads, its node table, is fetched as the page mounts: before any selection.
|
||||
expect((await tableRead).ok()).toBe(true);
|
||||
// The shell's own reads (counts, Activity) settle before the baseline.
|
||||
await expect(page.locator('aside.sidebar nav [data-route="images"] .nav-count')).toBeAttached();
|
||||
const baseline = (await urls(page)).writes.length;
|
||||
const before = requests.length;
|
||||
expect(requests.filter(entry => table.test(entry.url))).toHaveLength(1);
|
||||
for (const file of [sendablePng('private-name.png'), localFixture('core-tags.webm', 'video/webm', 'private-clip.webm')]) {
|
||||
await choose(page, file);
|
||||
await openViewer(page);
|
||||
@@ -666,7 +758,8 @@ test('local inspection makes no content request or storage write', async ({ page
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
}
|
||||
const network = requests.filter(entry => !entry.url.startsWith('blob:') && !entry.url.startsWith('data:'));
|
||||
const network = requests.slice(before).filter(entry => !entry.url.startsWith('blob:') && !entry.url.startsWith('data:'));
|
||||
expect(requests.filter(entry => table.test(entry.url)), 'the table is read once per page').toHaveLength(1);
|
||||
expect(network.filter(entry => /private|lighthouse|fox/.test(`${entry.url} ${entry.body ?? ''}`))).toEqual([]);
|
||||
expect(network.filter(entry => entry.method !== 'GET')).toEqual([]);
|
||||
expect(network.filter(entry => !/\/prompt_manager\/(activity|images\/output|library|vocabulary)/.test(entry.url))).toEqual([]);
|
||||
@@ -685,3 +778,80 @@ test('touch layout at 390 has no horizontal overflow at 200% zoom', async ({ pag
|
||||
expect(area).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
// LI-4 (D117): "Copy settings summary" after the six facts is the legacy metadata page's COPY ALL, in the shared full
|
||||
// inspector, so a file served in Images and the same bytes chosen here copy the same summary of what the rail shows.
|
||||
/** Records clipboard writes in the page, or (`clipboard: false`) leaves the page without a clipboard. */
|
||||
async function stubClipboard(page, { clipboard = true } = {}) {
|
||||
await page.addInitScript(available => {
|
||||
window.__copied = [];
|
||||
Object.defineProperty(navigator, 'clipboard', { configurable: true,
|
||||
value: available ? { writeText: async text => { window.__copied.push(text); } } : undefined });
|
||||
}, clipboard);
|
||||
}
|
||||
/** The summary the rail's own prompts and facts make, in the legacy COPY ALL lines and placeholders. */
|
||||
const expectedSummary = locator => locator.evaluate(node => {
|
||||
const facts = Object.fromEntries([...node.querySelectorAll(':scope .meta-grid:not(details .meta-grid) > div')]
|
||||
.map(item => [item.querySelector('dt').textContent, item.querySelector('dd').textContent]));
|
||||
// A confirmed prompt has its Copy action; an explicitly empty one takes the legacy placeholder too.
|
||||
const prompt = role => {
|
||||
const text = node.querySelector(`.prompt-section.${role} [data-copy-role]`) ? node.querySelector(`.prompt-section.${role} p.prompt-text`).textContent : null;
|
||||
return text === `No ${role} prompt (explicitly empty)` ? null : text;
|
||||
};
|
||||
return [`Checkpoint: ${facts.Model}`, `Positive Prompt: ${prompt('positive') ?? 'No prompt found'}`,
|
||||
`Negative Prompt: ${prompt('negative') ?? 'No negative prompt found'}`, `Steps: ${facts.Steps}`, `CFG Scale: ${facts.CFG}`,
|
||||
`Sampler: ${facts.Sampler}`, `Seed: ${facts.Seed}`].join('\n');
|
||||
});
|
||||
async function copySummary(page) {
|
||||
const button = rail(page).getByRole('button', { name: 'Copy settings summary', exact: true });
|
||||
// It follows the six facts, before "File & provenance".
|
||||
expect(await button.evaluate(node => [node.previousElementSibling?.localName, node.nextElementSibling?.localName])).toEqual(['dl', 'details']);
|
||||
await button.click();
|
||||
}
|
||||
|
||||
test('Copy settings summary copies the same legacy summary in Images and Inspect', async ({ page, request }) => {
|
||||
await stubClipboard(page);
|
||||
await expect(await request.post('/__test__/add-local-inspect-file', { data: { name: 'f02.png' } })).toBeOK();
|
||||
await page.goto(`${workspace}#/images`);
|
||||
await page.locator('article[data-image-key="0:local-inspect/f02.png"] .image-open').click();
|
||||
await expect(rail(page).locator('.prompt-section.positive p.prompt-text')).toBeVisible();
|
||||
const served = await expectedSummary(rail(page));
|
||||
expect(served).toMatch(/^Checkpoint: .+\nPositive Prompt: [\s\S]+\nNegative Prompt: [\s\S]+\nSteps: .+\nCFG Scale: .+\nSampler: .+\nSeed: .+$/);
|
||||
expect(served).not.toContain('Unknown');
|
||||
await copySummary(page);
|
||||
await expect(page.locator('#workspace-toast')).toHaveText('Settings summary copied.');
|
||||
expect(await page.evaluate(() => window.__copied)).toEqual([served]);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer(page)).toHaveCount(0);
|
||||
|
||||
await openInspect(page);
|
||||
await choose(page, parityPng('f02.png'));
|
||||
await openViewer(page);
|
||||
expect(await expectedSummary(rail(page))).toBe(served);
|
||||
await copySummary(page);
|
||||
await expect.poll(() => page.evaluate(() => window.__copied)).toEqual([served, served]);
|
||||
// The panel variant (the Images side panel) keeps the concept's footer: no summary action there.
|
||||
await page.keyboard.press('Escape');
|
||||
await page.goto(`${workspace}#/images`);
|
||||
await expect(page.locator('#image-inspector .panel-heading')).toBeVisible();
|
||||
await expect(page.locator('#image-inspector').getByRole('button', { name: 'Copy settings summary', exact: true })).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('Copy settings summary without a clipboard opens the select-to-copy dialog', async ({ page }) => {
|
||||
await stubClipboard(page, { clipboard: false });
|
||||
await openInspect(page);
|
||||
await choose(page, sendablePng());
|
||||
await openViewer(page);
|
||||
const summary = await expectedSummary(rail(page));
|
||||
expect(summary).toContain(`Positive Prompt: ${POSITIVE_LINES}`);
|
||||
await copySummary(page);
|
||||
const fallback = page.getByRole('dialog', { name: 'Copy settings summary', exact: true });
|
||||
await expect(fallback).toBeVisible();
|
||||
await expect(fallback.getByRole('textbox', { name: 'Select and copy this text', exact: true })).toHaveValue(summary);
|
||||
await expect(fallback.getByRole('textbox', { name: 'Select and copy this text', exact: true })).toBeFocused();
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(fallback).toHaveCount(0);
|
||||
// Escape closes only the copy dialog; focus returns to the action in the still-open viewer.
|
||||
await expect(viewer(page)).toBeVisible();
|
||||
await expect(rail(page).getByRole('button', { name: 'Copy settings summary', exact: true })).toBeFocused();
|
||||
});
|
||||
|
||||
@@ -610,7 +610,9 @@ test('card matches the concept and never wraps the category mid-word',async({pag
|
||||
await expect(await request.post('/prompt_manager/bulk/category',{data:{prompt_ids:[dual.id],category:''}})).toBeOK();
|
||||
// A long saved title fills the title row, so only the chip's own min-content keeps "Uncategorized" and "asian" whole.
|
||||
const wide='A remarkably long saved positive prompt title that fills the whole card title row';
|
||||
await patchShelf(page,item=>item.id===cat.id?{run_count:4,last_used_at:new Date().toISOString(),rating:5}:item.id===dual.id||item.id===dog.id?{title:wide,run_count:1,last_used_at:null,rating:null}:{});
|
||||
// One "now" for the patched last use and the page's clock, so it reads "Used today" on either side of midnight.
|
||||
const usedAt=new Date();await page.clock.setFixedTime(usedAt);
|
||||
await patchShelf(page,item=>item.id===cat.id?{run_count:4,last_used_at:usedAt.toISOString(),rating:5}:item.id===dual.id||item.id===dog.id?{title:wide,run_count:1,last_used_at:null,rating:null}:{});
|
||||
for(const [width,height] of [[1600,1100],[390,844]]) {
|
||||
await page.setViewportSize({width,height});await page.goto(route);await expect(cards(page)).toHaveCount(3);
|
||||
const card=page.locator(`article[data-prompt-id="${cat.id}"]`);
|
||||
|
||||
@@ -0,0 +1,401 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { seedConcept } from './concept-inventory.mjs';
|
||||
|
||||
// System › Sources & scans (#249 System S4 Task 11) on the concept-shaped real fixture, whose two sources are this
|
||||
// server's `output` and `input` folders: the source list, monitoring and worker rows read GET /settings (S4 Task 9),
|
||||
// Add/Remove edit the host draft behind the one Save row, and every scan starts the session job through the shared
|
||||
// "Review metadata scan" (D55), scoped to one source from its row (S4 Task 10). The hooks are workspace_system_fixture.py's.
|
||||
const workspace = '/prompt_manager/workspace';
|
||||
const SETTINGS_URL = '/prompt_manager/settings';
|
||||
const NOTICE_KEY = 'pm_update_notice_dismissed';
|
||||
const OUTSIDE_REASON = 'Gallery root must be inside a ComfyUI directory (output, input, temp, user) or one listed in PROMPT_MANAGER_EXTRA_GALLERY_ROOTS';
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await seedConcept(request);
|
||||
await expect(await request.post('/__test__/system/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
page.systemErrors = [];
|
||||
page.on('pageerror', error => page.systemErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await request.post('/__test__/release');
|
||||
await request.post('/__test__/jobs/restart');
|
||||
await request.post('/__test__/system/reset');
|
||||
expect(page.systemErrors).toEqual([]);
|
||||
});
|
||||
|
||||
const toast = page => page.locator('#workspace-toast');
|
||||
const layoutOf = page => page.locator('#workspace-main .settings-layout');
|
||||
const sourceList = page => layoutOf(page).locator('#source-list');
|
||||
const statusOf = page => layoutOf(page).locator('#settings-status');
|
||||
const saveButton = page => layoutOf(page).getByRole('button', { name: 'Save preferences', exact: true });
|
||||
const dialog = page => page.locator('dialog#modal[open]');
|
||||
|
||||
async function serverJson(request, url) {
|
||||
const response = await request.get(url);
|
||||
await expect(response).toBeOK();
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/** Open Sources with the update notice dismissed (or shown, `notice: true`) once the source list has loaded. */
|
||||
async function openSources(page, { notice = false } = {}) {
|
||||
if (!notice) await page.addInitScript(key => { if (location.protocol.startsWith('http')) localStorage.setItem(key, '3.2.0'); }, NOTICE_KEY);
|
||||
await page.goto(`${workspace}#/system/sources`);
|
||||
await expect(sourceList(page).locator('.operation').first()).toBeVisible();
|
||||
await expect(layoutOf(page).locator('#workers')).toBeEnabled();
|
||||
return layoutOf(page);
|
||||
}
|
||||
|
||||
/** The source list's rows: `[h3, p, [buttons]]`, then its note line when there is one. */
|
||||
function rowsOf(page) {
|
||||
return sourceList(page).evaluate(list => [...list.children].map(row => row.matches('.operation')
|
||||
? [row.querySelector('h3').textContent, row.querySelector('p').textContent,
|
||||
[...row.querySelectorAll('.row-actions > button')].map(button => button.textContent)]
|
||||
: [row.localName, row.className, row.textContent]));
|
||||
}
|
||||
|
||||
const sourceRow = (page, title) => sourceList(page).locator('.operation').filter({ has: page.getByRole('heading', { name: title, exact: true }) });
|
||||
|
||||
test('notice Later persists the 3.2.0 dismissal (pm_update_notice_dismissed) without scanning', async ({ page, request }) => {
|
||||
const writes = [];
|
||||
page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) writes.push(item.url()); });
|
||||
const layout = await openSources(page, { notice: true });
|
||||
const notice = layout.locator(':scope > .notice');
|
||||
await expect(notice.locator('h2')).toHaveText('Bring older images into your prompt history');
|
||||
await expect(notice.locator('p')).toHaveText('The schema upgrades automatically. An optional metadata scan links existing positives and negatives without regenerating all thumbnails.');
|
||||
await expect(notice.locator('.row-actions > button')).toHaveText(['Later', 'Review scan']);
|
||||
await notice.getByRole('button', { name: 'Later', exact: true }).click();
|
||||
await expect(layout.locator(':scope > .notice')).toHaveCount(0);
|
||||
expect(await page.evaluate(key => localStorage.getItem(key), NOTICE_KEY)).toBe('3.2.0');
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
// The dismissal is the legacy admin page's own, so neither page shows the notice again.
|
||||
await page.reload();
|
||||
await expect(sourceList(page).locator('.operation').first()).toBeVisible();
|
||||
await expect(layout.locator(':scope > .notice')).toHaveCount(0);
|
||||
expect(writes).toEqual([]);
|
||||
expect((await serverJson(request, '/prompt_manager/jobs')).jobs).toEqual([]);
|
||||
});
|
||||
|
||||
test('Review scan opens the shared Review metadata scan with all sources', async ({ page, request }) => {
|
||||
const layout = await openSources(page, { notice: true });
|
||||
// The notice's Review scan dismisses it, as the legacy page's does, and reviews every source.
|
||||
await layout.locator(':scope > .notice').getByRole('button', { name: 'Review scan', exact: true }).click();
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Review metadata scan');
|
||||
await expect(dialog(page).locator('dl.meta-grid > div > dd')).toHaveText(['2 output folders', 'New and unresolved files']);
|
||||
expect(await page.evaluate(key => localStorage.getItem(key), NOTICE_KEY)).toBe('3.2.0');
|
||||
await dialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await expect(layout.locator(':scope > .notice')).toHaveCount(0);
|
||||
// Enrich existing metadata is the same review.
|
||||
const enrich = layout.locator('.operation').filter({ hasText: 'Enrich existing metadata' });
|
||||
await expect(enrich.locator('p')).toHaveText('Resolve both prompts, deduplicate recipes, preserve uncertain links.');
|
||||
await enrich.getByRole('button', { name: 'Review scan', exact: true }).click();
|
||||
await expect(dialog(page).locator('dl.meta-grid > div > dd')).toHaveText(['2 output folders', 'New and unresolved files']);
|
||||
await dialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
expect((await serverJson(request, '/prompt_manager/jobs')).jobs).toEqual([]);
|
||||
});
|
||||
|
||||
test('Scan changes opens it with that one source ("1 output folder") and starts a scoped job that opens in Activity', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/monitor', { data: { watching: ['input'] } })).toBeOK();
|
||||
const bodies = [];
|
||||
page.on('request', item => { if (item.method() === 'POST' && item.url().endsWith('/prompt_manager/jobs/scan')) bodies.push(item.postDataJSON()); });
|
||||
await openSources(page);
|
||||
const input = sourceRow(page, 'ComfyUI input');
|
||||
await expect(input.locator('p')).toHaveText('input · watching');
|
||||
await input.getByRole('button', { name: 'Scan changes', exact: true }).click();
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Review metadata scan');
|
||||
await expect(dialog(page).locator('dl.meta-grid > div > dd')).toHaveText(['1 output folder', 'New and unresolved files']);
|
||||
await dialog(page).getByRole('button', { name: 'Start scan', exact: true }).click();
|
||||
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
|
||||
// The toast says it started (or, for a scan this quick, that it already finished); Activity shows its progress.
|
||||
await expect(toast(page)).toHaveText(/^Scan (started\.|finished · )/);
|
||||
expect(bodies).toEqual([{ scope: 'new', roots: [1], paths: ['input'] }]);
|
||||
const id = page.url().split('/').pop();
|
||||
// The job walks only that source (the count is recorded once the walk has listed it).
|
||||
await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.scope)
|
||||
.toMatchObject({ files: 'new', roots: [1], output_folders: 1 });
|
||||
await expect(page.locator('#workspace-main h1')).toHaveText('Scan for metadata');
|
||||
});
|
||||
|
||||
test('Add source validates on the server, then marks the host draft dirty; Save persists', async ({ page, request }) => {
|
||||
// A folder inside ComfyUI's output folder that is not a source yet.
|
||||
await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 1 } })).toBeOK();
|
||||
const layout = await openSources(page);
|
||||
expect(await rowsOf(page)).toEqual([
|
||||
['ComfyUI output', 'output · manual scan', ['Scan folder', 'Remove']],
|
||||
['ComfyUI input', 'input · manual scan', ['Scan folder', 'Remove']],
|
||||
]);
|
||||
await expect(layout.locator('.panel-heading > h2')).toHaveText('Image sources');
|
||||
const add = layout.locator('.panel-heading').getByRole('button', { name: '+ Add source', exact: true });
|
||||
await add.click();
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Add image source');
|
||||
await expect(dialog(page).locator('label[for="action-input"]')).toHaveText('Folder path');
|
||||
await expect(dialog(page).locator('#action-input')).toHaveValue('');
|
||||
await expect(dialog(page).locator('.dialog-actions > button')).toHaveText(['Cancel', 'Add']);
|
||||
await dialog(page).locator('#action-input').fill('output/unthumbnailed');
|
||||
const validated = page.waitForResponse(response => response.url().endsWith('/prompt_manager/settings/validate-source'));
|
||||
await dialog(page).getByRole('button', { name: 'Add', exact: true }).click();
|
||||
expect(await (await validated).json()).toEqual({ success: true, valid: true, label: 'unthumbnailed', path: 'output/unthumbnailed', code: null, reason: null });
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(add).toBeFocused();
|
||||
// Validation wrote nothing: the draft is dirty and the server still has two sources.
|
||||
expect(await rowsOf(page)).toEqual([
|
||||
['ComfyUI output', 'output · manual scan', ['Scan folder', 'Remove']],
|
||||
['ComfyUI input', 'input · manual scan', ['Scan folder', 'Remove']],
|
||||
['unthumbnailed', 'output/unthumbnailed · not saved yet', ['Remove']],
|
||||
]);
|
||||
await expect(statusOf(page)).toHaveText('Unsaved changes.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).sources.effective).toHaveLength(2);
|
||||
const saved = page.waitForRequest(item => item.method() === 'POST' && item.url().endsWith(SETTINGS_URL));
|
||||
await saveButton(page).click();
|
||||
const body = (await saved).postDataJSON();
|
||||
expect(body.gallery_root_paths).toEqual(['output', 'input', 'output/unthumbnailed']);
|
||||
expect(typeof body.expected_revision).toBe('string');
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
// The list is the server's again: three effective sources, each scannable.
|
||||
await expect(sourceRow(page, 'unthumbnailed').locator('p')).toHaveText('output/unthumbnailed · manual scan');
|
||||
await expect(sourceRow(page, 'unthumbnailed').getByRole('button', { name: 'Scan folder', exact: true })).toBeVisible();
|
||||
expect((await serverJson(request, SETTINGS_URL)).sources.effective.map(source => source.path)).toEqual(['output', 'input', 'output/unthumbnailed']);
|
||||
});
|
||||
|
||||
test('invalid source shows the server reason', async ({ page, request }) => {
|
||||
const layout = await openSources(page);
|
||||
await layout.getByRole('button', { name: '+ Add source', exact: true }).click();
|
||||
const field = dialog(page).locator('#action-input');
|
||||
for (const [path, reason, submit] of [
|
||||
['output/not-there', 'Gallery root does not exist', 'click'],
|
||||
// Enter in the field adds too (M6).
|
||||
['/etc', OUTSIDE_REASON, 'Enter'],
|
||||
]) {
|
||||
await field.fill(path);
|
||||
if (submit === 'Enter') await field.press('Enter');
|
||||
else await dialog(page).getByRole('button', { name: 'Add', exact: true }).click();
|
||||
await expect(dialog(page).locator('p.small[role="alert"]')).toHaveText(reason);
|
||||
await expect(field).toHaveAttribute('aria-invalid', 'true');
|
||||
await expect(field).toHaveValue(path);
|
||||
}
|
||||
// A folder already listed (the server's answer names it) is not added twice.
|
||||
await field.fill('input');
|
||||
await dialog(page).getByRole('button', { name: 'Add', exact: true }).click();
|
||||
await expect(dialog(page).locator('p.small[role="alert"]')).toHaveText('This folder is already a source.');
|
||||
// A check that cannot finish says so and adds nothing.
|
||||
await page.route('**/prompt_manager/settings/validate-source', route => route.fulfill({ status: 500,
|
||||
json: { success: false, code: null, error: 'Failed to check the source' } }));
|
||||
await field.fill('output/not-checked');
|
||||
await dialog(page).getByRole('button', { name: 'Add', exact: true }).click();
|
||||
await expect(dialog(page).locator('p.small[role="alert"]')).toHaveText('The folder could not be checked. Try again.');
|
||||
await page.unroute('**/prompt_manager/settings/validate-source');
|
||||
// Typing again clears the reason.
|
||||
await field.fill('output/unthumbnailed');
|
||||
await expect(dialog(page).locator('p.small[role="alert"]')).toHaveCount(0);
|
||||
await expect(field).not.toHaveAttribute('aria-invalid', 'true');
|
||||
await dialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await expect(statusOf(page)).toHaveText('No unsaved changes.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).sources.configured).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('Remove marks the draft; removing the last entry states auto-detect', async ({ page, request }) => {
|
||||
await openSources(page);
|
||||
await sourceRow(page, 'ComfyUI input').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
expect(await rowsOf(page)).toEqual([['ComfyUI output', 'output · manual scan', ['Scan folder', 'Remove']]]);
|
||||
await expect(statusOf(page)).toHaveText('Unsaved changes.');
|
||||
await sourceRow(page, 'ComfyUI output').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
expect(await rowsOf(page)).toEqual([['p', 'small', 'No source is configured. After saving, ComfyUI’s output folder is used.']]);
|
||||
await saveButton(page).click();
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
// Saved: ComfyUI's output folder is detected and listed without Remove.
|
||||
await expect.poll(rowsOf.bind(null, page)).toEqual([
|
||||
['ComfyUI output', 'output · manual scan', ['Scan folder']],
|
||||
['p', 'small', 'No source is configured; ComfyUI’s output folder is used.'],
|
||||
]);
|
||||
const { sources } = await serverJson(request, SETTINGS_URL);
|
||||
expect([sources.configured, sources.auto_detected]).toEqual([[], true]);
|
||||
});
|
||||
|
||||
test('a configured source that does not exist says why and can be removed', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/missing-source')).toBeOK();
|
||||
await openSources(page);
|
||||
// The fixture's unmounted drive is not inside a ComfyUI folder, so the server says that (it never probes outside).
|
||||
expect((await rowsOf(page)).at(-1)).toEqual(['unmounted-drive', `unmounted-drive · ${OUTSIDE_REASON}`, ['Remove']]);
|
||||
await sourceRow(page, 'unmounted-drive').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
await expect(sourceList(page).locator('.operation')).toHaveCount(2);
|
||||
await saveButton(page).click();
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).sources.configured.map(source => source.path)).toEqual(['output', 'input']);
|
||||
});
|
||||
|
||||
test('watching only with a live observer; restart pending otherwise', async ({ page, request }) => {
|
||||
// No image monitor in this process: every source is scanned by hand.
|
||||
await openSources(page);
|
||||
await expect(sourceList(page).locator('.operation p')).toHaveText(['output · manual scan', 'input · manual scan']);
|
||||
// A live observer that scheduled output: output is watched; input was added since it started.
|
||||
await expect(await request.post('/__test__/system/monitor', { data: { watching: ['output'] } })).toBeOK();
|
||||
await page.reload();
|
||||
await expect(sourceList(page).locator('.operation p')).toHaveText(['output · watching', 'input · restart pending']);
|
||||
await expect(sourceList(page).locator('.operation .row-actions > button:first-child')).toHaveText(['Scan changes', 'Scan folder']);
|
||||
// An observer that died watches nothing.
|
||||
await expect(await request.post('/__test__/system/monitor', { data: { watching: ['output', 'input'], alive: false } })).toBeOK();
|
||||
await page.reload();
|
||||
await expect(sourceList(page).locator('.operation p')).toHaveText(['output · restart pending', 'input · restart pending']);
|
||||
});
|
||||
|
||||
test('Watch for new outputs saves monitoring_enabled; a restart applies it', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/monitor', { data: { watching: ['output', 'input'] } })).toBeOK();
|
||||
const layout = await openSources(page);
|
||||
const monitoring = layout.getByRole('switch', { name: 'Watch for new outputs', exact: true });
|
||||
await expect(monitoring).toBeChecked();
|
||||
await expect(layout.locator('.setting-row').filter({ has: page.locator('#monitoring') }).locator('p'))
|
||||
.toHaveText('New files enter the same library. Existing scans do not count as new runs.');
|
||||
await monitoring.uncheck();
|
||||
await expect(statusOf(page)).toHaveText('Unsaved changes.');
|
||||
const saved = page.waitForResponse(response => response.request().method() === 'POST' && response.url().endsWith(SETTINGS_URL));
|
||||
await saveButton(page).click();
|
||||
const answer = await (await saved).json();
|
||||
expect(answer.restart_required_fields).toEqual(['monitoring_enabled']);
|
||||
// D128: saved for the next start, and the status says a restart applies it.
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration. Restart ComfyUI to apply: Watch for new outputs.');
|
||||
// The running observer keeps watching until ComfyUI restarts, and the list says so.
|
||||
const { settings, monitor } = await serverJson(request, SETTINGS_URL);
|
||||
expect([settings.monitoring_enabled, monitor.observer_alive]).toEqual([false, true]);
|
||||
await expect(sourceList(page).locator('.operation p')).toHaveText(['output · watching until restart', 'input · watching until restart']);
|
||||
await expect(sourceList(page).locator('.operation .row-actions > button:first-child')).toHaveText(['Scan changes', 'Scan changes']);
|
||||
// Switched back on before the restart: the observer still runs, so nothing waits for one.
|
||||
await monitoring.check();
|
||||
await saveButton(page).click();
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
await expect(sourceList(page).locator('.operation p')).toHaveText(['output · watching', 'input · watching']);
|
||||
});
|
||||
|
||||
test('worker range max is cpu_count', async ({ page, request }) => {
|
||||
const { settings } = await serverJson(request, SETTINGS_URL);
|
||||
const layout = await openSources(page);
|
||||
const workers = layout.locator('#workers');
|
||||
await expect(layout.locator('label[for="workers"]')).toHaveText('Worker threads');
|
||||
await expect(workers).toHaveAttribute('type', 'range');
|
||||
await expect(workers).toHaveAttribute('min', '1');
|
||||
await expect(workers).toHaveAttribute('max', String(settings.cpu_count));
|
||||
await expect(workers).toHaveValue(String(settings.worker_threads));
|
||||
const next = settings.worker_threads === 1 ? settings.cpu_count : 1;
|
||||
await workers.fill(String(next));
|
||||
await expect(statusOf(page)).toHaveText(next === settings.worker_threads ? 'No unsaved changes.' : 'Unsaved changes.');
|
||||
test.skip(next === settings.worker_threads, 'One core: the range has a single value');
|
||||
const saved = page.waitForRequest(item => item.method() === 'POST' && item.url().endsWith(SETTINGS_URL));
|
||||
await saveButton(page).click();
|
||||
expect((await saved).postDataJSON().worker_threads).toBe(next);
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).settings.worker_threads).toBe(next);
|
||||
});
|
||||
|
||||
test('Refresh files calls /gallery/rescan and toasts its counts', async ({ page }) => {
|
||||
const layout = await openSources(page);
|
||||
const refresh = layout.locator('.operation').filter({ has: page.getByRole('heading', { name: 'Refresh files', exact: true }) });
|
||||
await expect(refresh.locator('p')).toHaveText('Read every source again to find files added or removed outside ComfyUI.');
|
||||
const answered = page.waitForResponse(response => response.url().endsWith('/prompt_manager/gallery/rescan'));
|
||||
await refresh.getByRole('button', { name: 'Refresh', exact: true }).click();
|
||||
const { total, roots } = await (await answered).json();
|
||||
expect(roots).toBe(2);
|
||||
await expect(toast(page)).toHaveText(`Files refreshed. ${total.toLocaleString('en-US')} files in 2 output folders.`);
|
||||
// A failed refresh says so.
|
||||
await page.route('**/prompt_manager/gallery/rescan', route => route.fulfill({ status: 500, json: { success: false, error: 'busy disk' } }));
|
||||
await refresh.getByRole('button', { name: 'Refresh', exact: true }).click();
|
||||
await expect(toast(page)).toHaveText('Files could not be refreshed. Try again.');
|
||||
});
|
||||
|
||||
test('Thumbnail generation hands off to the existing gallery until its review dialog', async ({ page }) => {
|
||||
const layout = await openSources(page);
|
||||
const thumbnails = layout.locator('.operation').filter({ has: page.getByRole('heading', { name: 'Thumbnail generation', exact: true }) });
|
||||
await expect(thumbnails.locator('p')).toHaveText('Generate missing previews; rebuilding all is a separate explicit option.');
|
||||
const popup = page.waitForEvent('popup');
|
||||
await thumbnails.getByRole('button', { name: 'Review options', exact: true }).click();
|
||||
const gallery = await popup;
|
||||
expect(new URL(gallery.url()).pathname).toBe('/prompt_manager/gallery');
|
||||
await gallery.close();
|
||||
await expect(statusOf(page)).toHaveText('No unsaved changes.');
|
||||
});
|
||||
|
||||
test('a save refused by a running scan shows the busy link toast', async ({ page, request }) => {
|
||||
const running = await (await request.post('/__test__/jobs/start', { data: { title: 'Scan for metadata' } })).json();
|
||||
await openSources(page);
|
||||
await sourceRow(page, 'ComfyUI input').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
await saveButton(page).click();
|
||||
await expect(toast(page)).toHaveText('Another job is running: Scan for metadata.');
|
||||
await expect(toast(page).getByRole('link', { name: 'Scan for metadata', exact: true })).toHaveAttribute('href', `#/activity/${running.job.id}`);
|
||||
await expect(statusOf(page)).toHaveText('Some changes were not saved. Review the highlighted settings.');
|
||||
// The draft is kept for the next Save, and the server still has both sources.
|
||||
await expect(sourceList(page).locator('.operation')).toHaveCount(1);
|
||||
expect((await serverJson(request, SETTINGS_URL)).sources.configured).toHaveLength(2);
|
||||
// An answer that does not name its job (an older server): the toast names the running one from the job list.
|
||||
await page.route(`**${SETTINGS_URL}`, route => (route.request().method() === 'POST'
|
||||
? route.fulfill({ status: 409, json: { success: false, code: 'busy', error: 'Scan for metadata already running' } })
|
||||
: route.continue()));
|
||||
await page.locator('#workspace-toast').evaluate(node => { node.textContent = ''; });
|
||||
await saveButton(page).click();
|
||||
await expect(toast(page)).toHaveText('Another job is running: Scan for metadata.');
|
||||
await expect(toast(page).getByRole('link', { name: 'Scan for metadata', exact: true })).toHaveAttribute('href', `#/activity/${running.job.id}`);
|
||||
await page.unroute(`**${SETTINGS_URL}`);
|
||||
await request.post(`/__test__/jobs/${running.job.id}/finish`, { data: { state: 'finished' } });
|
||||
});
|
||||
|
||||
test('a source list that cannot be read says so and retries', async ({ page }) => {
|
||||
await page.addInitScript(key => { if (location.protocol.startsWith('http')) localStorage.setItem(key, '3.2.0'); }, NOTICE_KEY);
|
||||
let failing = true;
|
||||
await page.route(`**${SETTINGS_URL}`, route => (failing && route.request().method() === 'GET'
|
||||
? route.fulfill({ status: 500, json: { success: false, error: 'Injected failure' } }) : route.continue()));
|
||||
await page.goto(`${workspace}#/system/sources`);
|
||||
await expect(sourceList(page).getByRole('alert')).toHaveText('Sources could not be loaded. Your saved data has not changed.');
|
||||
await expect(sourceList(page).locator('.operation')).toHaveCount(0);
|
||||
// The server settings rows say so too, and stay disabled.
|
||||
await expect(layoutOf(page).locator('#workers')).toBeDisabled();
|
||||
failing = false;
|
||||
await sourceList(page).getByRole('button', { name: 'Retry', exact: true }).click();
|
||||
await expect(sourceList(page).locator('.operation')).toHaveCount(2);
|
||||
});
|
||||
|
||||
test('a scan from a list that changed since it was read starts nothing, says why and reads the list again', async ({ page, request }) => {
|
||||
await openSources(page);
|
||||
// The legacy page (or another tab) removes ComfyUI output: the page's "input" is index 1, now past the end.
|
||||
await expect(await request.post(SETTINGS_URL, { data: { gallery_root_paths: ['input'] } })).toBeOK();
|
||||
const refused = page.waitForResponse(response => response.url().endsWith('/prompt_manager/jobs/scan'));
|
||||
await sourceRow(page, 'ComfyUI input').getByRole('button', { name: 'Scan folder', exact: true }).click();
|
||||
await dialog(page).getByRole('button', { name: 'Start scan', exact: true }).click();
|
||||
const answer = await refused;
|
||||
expect([answer.status(), (await answer.json()).code]).toEqual([409, 'sources-changed']);
|
||||
await expect(toast(page)).toHaveText('The sources changed since they were loaded. Reload them.');
|
||||
// No scan started (the other save ran as its own source-change hold).
|
||||
expect((await serverJson(request, '/prompt_manager/jobs')).jobs.map(job => job.kind)).toEqual(['source_change']);
|
||||
// Read again: the one source is index 0 now, and its scan starts.
|
||||
await expect.poll(rowsOf.bind(null, page)).toEqual([['ComfyUI input', 'input · manual scan', ['Scan folder', 'Remove']]]);
|
||||
await expect(statusOf(page)).toHaveText('No unsaved changes.');
|
||||
const started = page.waitForRequest(item => item.method() === 'POST' && item.url().endsWith('/prompt_manager/jobs/scan'));
|
||||
await sourceRow(page, 'ComfyUI input').getByRole('button', { name: 'Scan folder', exact: true }).click();
|
||||
await dialog(page).getByRole('button', { name: 'Start scan', exact: true }).click();
|
||||
expect((await started).postDataJSON()).toEqual({ scope: 'new', roots: [0], paths: ['input'] });
|
||||
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
|
||||
});
|
||||
|
||||
test('a save refused for one unusable source names it and marks its row', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/missing-source', { data: { inside: true } })).toBeOK();
|
||||
await openSources(page);
|
||||
await expect(sourceRow(page, 'unplugged').locator('p')).toHaveText('output/unplugged · manual scan');
|
||||
// The drive is unplugged after the page read the list; removing another source resends the whole list.
|
||||
await expect(await request.post('/__test__/system/unplug')).toBeOK();
|
||||
await sourceRow(page, 'ComfyUI input').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
const saved = page.waitForResponse(response => response.request().method() === 'POST' && response.url().endsWith(SETTINGS_URL));
|
||||
await saveButton(page).click();
|
||||
const answer = await saved;
|
||||
expect(answer.status()).toBe(400);
|
||||
expect(await answer.json()).toMatchObject({ code: 'invalid-source', source: 'output/unplugged', index: 1 });
|
||||
await expect(statusOf(page)).toHaveText("output/unplugged can't be used: Gallery root does not exist. Remove it or reconnect it, then save.");
|
||||
await expect(sourceRow(page, 'unplugged')).toHaveAttribute('aria-invalid', 'true');
|
||||
await expect(sourceList(page).locator('.operation[aria-invalid="true"]')).toHaveCount(1);
|
||||
expect((await serverJson(request, SETTINGS_URL)).sources.configured).toHaveLength(3);
|
||||
// Removed, the list saves.
|
||||
await sourceRow(page, 'unplugged').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
await expect(sourceList(page).locator('.operation[aria-invalid="true"]')).toHaveCount(0);
|
||||
await saveButton(page).click();
|
||||
await expect(statusOf(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).sources.configured.map(source => source.path)).toEqual(['output']);
|
||||
});
|
||||
@@ -0,0 +1,621 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { seedConcept } from './concept-inventory.mjs';
|
||||
|
||||
// System › Overview (#249 System S2 Task 6) on the concept-shaped real fixture: the metrics, rows, Statistics dialog and
|
||||
// Health report read the server, never sample values (D93, D95, D96). The hooks are workspace_system_fixture.py's.
|
||||
const workspace = '/prompt_manager/workspace';
|
||||
const OVERVIEW_URL = '/prompt_manager/system/overview';
|
||||
const THUMBNAILS_URL = '/prompt_manager/images/thumbnails/status';
|
||||
const number = value => value.toLocaleString('en-US');
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
page.manifest = await seedConcept(request);
|
||||
await expect(await request.post('/__test__/system/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
page.systemErrors = [];
|
||||
page.on('pageerror', error => page.systemErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await request.post('/__test__/release');
|
||||
await request.post('/__test__/system/reset');
|
||||
expect(page.systemErrors).toEqual([]);
|
||||
});
|
||||
|
||||
async function serverJson(request, url) {
|
||||
const response = await request.get(url);
|
||||
await expect(response).toBeOK();
|
||||
return response.json();
|
||||
}
|
||||
|
||||
const layoutOf = page => page.locator('#workspace-main .settings-layout');
|
||||
|
||||
async function openOverview(page) {
|
||||
await page.goto(`${workspace}#/system/overview`);
|
||||
await expect(layoutOf(page).locator(':scope > .summary-metrics')).toBeVisible();
|
||||
return layoutOf(page);
|
||||
}
|
||||
|
||||
/** The rows of one Overview surface: `[h3, p, button]`. */
|
||||
function rowsOf(layout, heading) {
|
||||
return layout.locator('.two-col > section.surface').filter({ has: layout.page().getByRole('heading', { name: heading, exact: true }) })
|
||||
.locator('.operation').evaluateAll(rows => rows.map(row => [row.querySelector('h3').textContent,
|
||||
row.querySelector('p').textContent, row.querySelector('.row-actions button').textContent]));
|
||||
}
|
||||
|
||||
test('overview shows real counts and Not checked', async ({ page, request }) => {
|
||||
const writes = [];
|
||||
page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) writes.push(item.url()); });
|
||||
const overview = await serverJson(request, OVERVIEW_URL);
|
||||
const thumbnails = await serverJson(request, THUMBNAILS_URL);
|
||||
// The concept fixture: six positives, their saved negatives and 24 linked images, all with thumbnails.
|
||||
expect([overview.library.positive_prompts, overview.images.indexed, thumbnails.pending]).toEqual([6, 24, 0]);
|
||||
// I2: the Overview never walks the sources for thumbnails; the row reads the status route itself.
|
||||
expect(overview).not.toHaveProperty('thumbnails');
|
||||
// The fixture records no ambiguous file: the Needs review queue is empty.
|
||||
expect(overview.needs_review).toEqual({ pending: 0 });
|
||||
const layout = await openOverview(page);
|
||||
expect(await layout.locator('.summary-metrics > .metric').evaluateAll(items => items.map(item =>
|
||||
[item.querySelector('strong').textContent, item.querySelector('span').textContent]))).toEqual([
|
||||
[number(overview.library.positive_prompts), 'Positive prompts'],
|
||||
[number(overview.library.negative_variants), 'Negative variants'],
|
||||
[number(overview.images.indexed), 'Images'],
|
||||
['Not checked', 'Database integrity'],
|
||||
]);
|
||||
expect(await rowsOf(layout, 'Library health')).toEqual([
|
||||
['0 ambiguous image links', 'Review metadata before assigning a positive / negative pair.', 'Review metadata'],
|
||||
['Backup status', 'No retained backup record.', 'View backups'],
|
||||
['No missing thumbnails', 'Generate only missing previews when necessary.', 'Review thumbnails'],
|
||||
]);
|
||||
expect(await rowsOf(layout, 'Workspace preferences')).toEqual([
|
||||
['One display configuration', 'Grid and density settings are shared across views.', 'Appearance'],
|
||||
['2 output sources', 'Monitoring and enrichment share one source list.', 'Manage sources'],
|
||||
['Statistics', 'Runs and ratings by category, with library totals.', 'Explore'],
|
||||
]);
|
||||
expect(writes).toEqual([]);
|
||||
});
|
||||
|
||||
test('Run health checks opens Health report and updates metric 4', async ({ page, request }) => {
|
||||
await page.goto(`${workspace}#/system/maintenance`);
|
||||
const layout = layoutOf(page);
|
||||
await expect(layout.locator(':scope > .notice h2')).toHaveText('Check first. Review changes. Then apply.');
|
||||
// The rest of Maintenance is still the existing tools' (D94).
|
||||
await expect(layout.locator(':scope > section.handoff h2')).toHaveText('Your current tools are ready');
|
||||
const run = layout.getByRole('button', { name: 'Run health checks', exact: true });
|
||||
await run.click();
|
||||
const dialog = page.locator('dialog#modal[open]');
|
||||
await expect(dialog.locator('#modal-title')).toHaveText('Health report');
|
||||
const { report } = await serverJson(request, '/prompt_manager/system/health');
|
||||
expect(report.status).toBe('incomplete');
|
||||
expect(await dialog.locator('.operation-list > .operation').evaluateAll(rows => rows.map(row =>
|
||||
[row.querySelector('h3').textContent, row.querySelector('span.chip').textContent]))).toEqual([
|
||||
['Integrity check', 'Passed'], ['Foreign keys & pair references', 'Passed'], ['Canonical hashes', 'Passed'],
|
||||
['Pair uniqueness', 'Passed'], ['Missing files · output', 'Passed'], ['Missing files · input', 'Passed'],
|
||||
['Ambiguous metadata', 'Not available yet'], ['Consistency', 'Passed'],
|
||||
]);
|
||||
await expect(dialog.locator('#modal-body > p.small')).toHaveText(/^Checked .+\. Some checks could not run, so this report is incomplete\.$/);
|
||||
await expect(dialog.locator('.dialog-actions button')).toHaveText(['Close', 'Review missing links']);
|
||||
await dialog.getByRole('button', { name: 'Close', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open]')).toHaveCount(0);
|
||||
await expect(run).toBeFocused();
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await expect(layout.locator('.metric').nth(3).locator('strong')).toHaveText('Incomplete');
|
||||
|
||||
// A missing file and an unmounted source: the report counts the readable sources and names the other one.
|
||||
await expect(await request.post('/__test__/system/missing-file')).toBeOK();
|
||||
await expect(await request.post('/__test__/system/missing-source')).toBeOK();
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click();
|
||||
await layout.getByRole('button', { name: 'Run health checks', exact: true }).click();
|
||||
await expect(dialog.locator('.operation').filter({ hasText: 'Missing files' }).locator('.chip'))
|
||||
.toHaveText(['1 link needs review', 'Passed', 'Not available']);
|
||||
await expect(dialog.locator('.operation h3').filter({ hasText: 'Missing files' }))
|
||||
.toHaveText(['Missing files · output', 'Missing files · input', 'Missing files · unmounted-drive']);
|
||||
// Review missing links goes to Maintenance, where the cleanup's existing tool is linked.
|
||||
await dialog.getByRole('button', { name: 'Review missing links', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open]')).toHaveCount(0);
|
||||
await expect(page).toHaveURL(`${workspace}#/system/maintenance`);
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await expect(layout.locator('.metric').nth(3).locator('strong')).toHaveText('1 issue');
|
||||
});
|
||||
|
||||
test('a health check that fails says so and keeps the last result', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/fail', { data: { path: '/prompt_manager/system/health' } })).toBeOK();
|
||||
await page.goto(`${workspace}#/system/maintenance`);
|
||||
const run = layoutOf(page).getByRole('button', { name: 'Run health checks', exact: true });
|
||||
await run.click();
|
||||
await expect(page.locator('#workspace-toast')).toHaveText('The health check could not finish. Nothing was changed.');
|
||||
await expect(page.locator('dialog#modal[open]')).toHaveCount(0);
|
||||
await expect(run).not.toHaveAttribute('aria-disabled', 'true');
|
||||
await openOverview(page);
|
||||
await expect(layoutOf(page).locator('.metric').nth(3).locator('strong')).toHaveText('Not checked');
|
||||
});
|
||||
|
||||
test('thumbnail row reflects the status endpoint', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 3 } })).toBeOK();
|
||||
const status = await serverJson(request, '/prompt_manager/images/thumbnails/status');
|
||||
expect(status.pending).toBe(3);
|
||||
const layout = await openOverview(page);
|
||||
const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) });
|
||||
await expect(row.locator('h3')).toHaveText('3 missing thumbnails');
|
||||
// Until the thumbnail review dialog (S5), Review thumbnails opens Sources.
|
||||
await row.getByRole('button', { name: 'Review thumbnails', exact: true }).click();
|
||||
await expect(page).toHaveURL(`${workspace}#/system/sources`);
|
||||
});
|
||||
|
||||
test('the thumbnail row paints when its own status read answers', async ({ page, request }) => {
|
||||
const held = async () => (await serverJson(request, '/__test__/hold')).held[THUMBNAILS_URL] ?? 0;
|
||||
await expect(await request.post('/__test__/hold', { data: { path: THUMBNAILS_URL } })).toBeOK();
|
||||
const layout = await openOverview(page);
|
||||
const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) });
|
||||
// The other values do not wait for the walk of every source.
|
||||
await expect(row.locator('h3')).toHaveText('Checking thumbnails…');
|
||||
await expect(layout.locator('.summary-metrics > .metric')).toHaveCount(4);
|
||||
await expect.poll(held).toBe(1);
|
||||
await expect(await request.post('/__test__/release')).toBeOK();
|
||||
await expect(row.locator('h3')).toHaveText('No missing thumbnails');
|
||||
});
|
||||
|
||||
test('a failed thumbnail read is unavailable, never zero', async ({ page }) => {
|
||||
await page.route(`**${THUMBNAILS_URL}`, route => route.fulfill({ status: 500, contentType: 'application/json',
|
||||
body: JSON.stringify({ success: false, error: 'Injected failure' }) }));
|
||||
const layout = await openOverview(page);
|
||||
const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) });
|
||||
await expect(row.locator('h3')).toHaveText('Thumbnail status unavailable');
|
||||
await expect(layout.getByRole('alert')).toHaveCount(0);
|
||||
await expect(layout.locator('.metric').nth(2).locator('strong')).toHaveText('24');
|
||||
});
|
||||
|
||||
test('ambiguous row reads the pending count; its action waits for the review page', async ({ page }) => {
|
||||
const layout = await openOverview(page);
|
||||
const row = layout.locator('.operation').filter({ hasText: 'ambiguous image links' });
|
||||
await expect(row.locator('h3')).toHaveText('0 ambiguous image links');
|
||||
await expect(row.locator('p')).toHaveText('Review metadata before assigning a positive / negative pair.');
|
||||
const review = row.getByRole('button', { name: 'Review metadata', exact: true });
|
||||
await expect(review).toHaveAttribute('aria-disabled', 'true');
|
||||
await review.click({ force: true });
|
||||
await expect(page).toHaveURL(`${workspace}#/system/overview`);
|
||||
});
|
||||
|
||||
test('View backups / Appearance / Manage sources switch sections', async ({ page }) => {
|
||||
for (const [label, section, pressed] of [['View backups', 'backups', 'Backup & export'], ['Appearance', 'appearance', 'Appearance'],
|
||||
['Manage sources', 'sources', 'Sources & scans']]) {
|
||||
const layout = await openOverview(page);
|
||||
await layout.getByRole('button', { name: label, exact: true }).click();
|
||||
await expect(page).toHaveURL(`${workspace}#/system/${section}`);
|
||||
await expect(page.locator('#workspace-main .section-nav [aria-pressed="true"]')).toHaveText(pressed);
|
||||
await expect(layout.locator(':scope > .summary-metrics')).toHaveCount(0);
|
||||
}
|
||||
});
|
||||
|
||||
test('statistics dialog lists runs by category and library totals', async ({ page, request }) => {
|
||||
const statistics = await serverJson(request, '/prompt_manager/system/statistics');
|
||||
expect(statistics.by_category.length).toBeGreaterThan(1);
|
||||
const layout = await openOverview(page);
|
||||
const explore = layout.getByRole('button', { name: 'Explore', exact: true });
|
||||
await explore.click();
|
||||
const dialog = page.locator('dialog#modal[open]');
|
||||
await expect(dialog.locator('#modal-title')).toHaveText('Usage statistics');
|
||||
const tables = dialog.locator('#modal-body > table');
|
||||
await expect(tables.locator('caption')).toHaveText(['Runs by category', 'Library totals']);
|
||||
await expect(tables.first().locator('thead th')).toHaveText(['Category', 'Runs', 'Avg. rating']);
|
||||
const cells = table => table.locator('tbody tr').evaluateAll(rows => rows.map(row => [...row.cells].map(cell => cell.textContent)));
|
||||
expect(await cells(tables.first())).toEqual(statistics.by_category.map(row => [row.category ?? 'Uncategorized',
|
||||
number(row.runs), row.average_rating === null ? 'Not rated' : `${number(row.average_rating)} / 5`]));
|
||||
expect(await cells(tables.nth(1))).toEqual([['Prompts', number(statistics.totals.total_prompts)],
|
||||
['Tags', number(statistics.totals.total_tags)], ['Images', number(statistics.totals.total_images)],
|
||||
['Rated images', number(statistics.rated_images)]]);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(explore).toBeFocused();
|
||||
// A statistics read that fails says so; the Overview stays.
|
||||
await expect(await request.post('/__test__/system/fail', { data: { path: '/prompt_manager/system/statistics' } })).toBeOK();
|
||||
await explore.click();
|
||||
await expect(page.locator('#workspace-toast')).toHaveText('Statistics could not be loaded. Your saved data has not changed.');
|
||||
await expect(page.locator('dialog#modal[open]')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('load failure shows Retry and no zeros', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/fail', { data: { path: OVERVIEW_URL } })).toBeOK();
|
||||
await page.goto(`${workspace}#/system/overview`);
|
||||
const layout = layoutOf(page);
|
||||
await expect(layout.getByRole('alert')).toHaveText('The overview could not be loaded. Your saved data has not changed.');
|
||||
await expect(layout.locator('.metric')).toHaveCount(0);
|
||||
await expect(layout).not.toContainText('0');
|
||||
await layout.getByRole('button', { name: 'Retry', exact: true }).click();
|
||||
await expect(layout.locator('.summary-metrics > .metric')).toHaveCount(4);
|
||||
await expect(layout.getByRole('alert')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('out-of-order overview responses never repaint a newer section', async ({ page, request }) => {
|
||||
const held = async () => (await serverJson(request, '/__test__/hold')).held[OVERVIEW_URL] ?? 0;
|
||||
await expect(await request.post('/__test__/hold', { data: { path: OVERVIEW_URL } })).toBeOK();
|
||||
await page.goto(`${workspace}#/system/overview`);
|
||||
const layout = layoutOf(page);
|
||||
await expect(layout.getByRole('status')).toHaveText('Loading the overview…');
|
||||
await expect.poll(held).toBe(1);
|
||||
// Leave for Appearance, come back (a second read), then leave again before either answers.
|
||||
const nav = page.locator('#workspace-main .section-nav');
|
||||
await nav.getByRole('button', { name: 'Appearance', exact: true }).click();
|
||||
await nav.getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await expect.poll(held).toBe(2);
|
||||
// Integrations still hands off (S10), so its body is one known panel.
|
||||
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
await expect(await request.post('/__test__/release')).toBeOK();
|
||||
await expect.poll(held).toBe(0);
|
||||
// Both late answers reached a section that had left: Integrations keeps its own body.
|
||||
await expect(layout.locator(':scope > section.handoff')).toBeVisible();
|
||||
await page.evaluate(() => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve))));
|
||||
await expect(layout.locator('.summary-metrics')).toHaveCount(0);
|
||||
expect(await layout.evaluate(node => [...node.children].map(child => child.className))).toEqual(['surface handoff', 'save-row']);
|
||||
// Back on Overview, one fresh read paints one set of metrics.
|
||||
await nav.getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await expect(layout.locator('.summary-metrics')).toHaveCount(1);
|
||||
await expect(layout.locator('.two-col')).toHaveCount(1);
|
||||
});
|
||||
|
||||
// System › Appearance (#249 System S3 Task 8): the browser display defaults (`gallerySettings`, preferences.mjs) and the
|
||||
// two host fields behind the one Save row, and the consumers that read them on their next visit.
|
||||
const SETTINGS_URL = '/prompt_manager/settings';
|
||||
const INSPECT_PATH = '/prompt_manager/images/inspect/';
|
||||
const statusOf = page => page.locator('#workspace-main #settings-status');
|
||||
const saveOf = page => page.locator('#workspace-main .save-row > button');
|
||||
const storedSettings = page => page.evaluate(() => JSON.parse(localStorage.getItem('gallerySettings') || 'null'));
|
||||
const imageCards = page => page.locator('#workspace-main article.media-card');
|
||||
const panel = page => page.getByRole('complementary', { name: 'Selected image details', exact: true });
|
||||
|
||||
async function openAppearance(page) {
|
||||
await page.goto(`${workspace}#/system/appearance`);
|
||||
await expect(page.locator('#workspace-main .settings-layout > section.surface > h2')).toHaveText('Browsing & display');
|
||||
// The host rows are enabled once the server settings answered.
|
||||
await expect(page.locator('#timeout')).toBeEnabled();
|
||||
}
|
||||
|
||||
async function saveAppearance(page, text) {
|
||||
await expect(saveOf(page)).not.toHaveAttribute('aria-disabled', 'true');
|
||||
await saveOf(page).click();
|
||||
await expect(statusOf(page)).toHaveText(text);
|
||||
}
|
||||
|
||||
/** Set toggles (by row id) through their Toolkit switches. */
|
||||
async function toggle(page, values) {
|
||||
for (const [id, checked] of Object.entries(values)) await page.locator(`#${id}`).setChecked(checked, { force: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* A request barrier: one request issued and answered after everything the page has asked for so far, so a read the
|
||||
* last action started has been seen (Playwright reports a page's requests in order).
|
||||
*/
|
||||
async function requestBarrier(page) {
|
||||
expect(await page.evaluate(url => fetch(url, { cache: 'no-store' }).then(response => response.ok), SETTINGS_URL)).toBe(true);
|
||||
}
|
||||
|
||||
async function openImages(page) {
|
||||
await page.goto(`${workspace}#/images`);
|
||||
await expect(imageCards(page).first()).toBeVisible();
|
||||
}
|
||||
|
||||
test('defaults apply to a new visit but not to the current view or a saved view', async ({ page, request }) => {
|
||||
const created = await request.post('/prompt_manager/saved-views', { data: { page: 'images', name: 'Everything',
|
||||
state: { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false } } });
|
||||
expect(created.status()).toBe(201);
|
||||
const view = (await created.json()).view;
|
||||
try {
|
||||
await openImages(page);
|
||||
// The current view: the toolbar sets List.
|
||||
await page.getByRole('group', { name: 'Image layout' }).getByRole('button', { name: 'List', exact: true }).click();
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list');
|
||||
await openAppearance(page);
|
||||
await expect(page.locator('#default-view')).toHaveValue('grid');
|
||||
await page.locator('#default-view').selectOption('masonry');
|
||||
await page.locator('#density').selectOption('compact');
|
||||
await expect(statusOf(page)).toHaveText('Unsaved changes.');
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
const stored = await storedSettings(page);
|
||||
expect([stored.workspaceLayout, stored.workspaceDensity]).toEqual(['masonry', 'compact']);
|
||||
// Back in Images (the page session's last visit): the current view keeps its own layout and size.
|
||||
await page.locator('#primary-nav a[data-route="images"]').click();
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list');
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'comfortable');
|
||||
// The saved view opens as before.
|
||||
await page.goto(`${workspace}#/saved/${view.id}`);
|
||||
await expect(imageCards(page).first()).toBeVisible();
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'grid');
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'comfortable');
|
||||
// A new visit starts from the defaults.
|
||||
await openImages(page);
|
||||
await page.reload();
|
||||
await expect(imageCards(page).first()).toBeVisible();
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'masonry');
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'compact');
|
||||
await expect(page.getByRole('group', { name: 'Image layout' }).getByRole('button', { name: 'Masonry', exact: true }))
|
||||
.toHaveAttribute('aria-pressed', 'true');
|
||||
} finally {
|
||||
await request.delete(`/prompt_manager/saved-views/${view.id}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('the defaults reach the Images page only; embedded galleries keep their own presentation', async ({ page }) => {
|
||||
await openAppearance(page);
|
||||
await page.locator('#default-view').selectOption('list');
|
||||
await page.locator('#density').selectOption('compact');
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
// The prompt history's shared gallery has no toolbar to change them, so it opens as it always did.
|
||||
await page.goto(`${workspace}#/library/${page.manifest.prompts[0].positive_id}`);
|
||||
await expect(imageCards(page).first()).toBeVisible();
|
||||
await expect(page.locator('#workspace-main .image-grid')).toHaveAttribute('data-layout', 'grid');
|
||||
await expect(page.locator('#workspace-main .image-grid')).toHaveAttribute('data-density', 'comfortable');
|
||||
// The Images page starts from them.
|
||||
await page.locator('#primary-nav a[data-route="images"]').click();
|
||||
await expect(imageCards(page).first()).toBeVisible();
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list');
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'compact');
|
||||
});
|
||||
|
||||
test('metadata off shows an explicit load and loads once on request', async ({ page }) => {
|
||||
await openAppearance(page);
|
||||
await toggle(page, { 'metadata-auto': false });
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
const reads = [];
|
||||
page.on('request', item => { if (new URL(item.url()).pathname.startsWith(INSPECT_PATH)) reads.push(item.url()); });
|
||||
await openImages(page);
|
||||
const first = imageCards(page).first();
|
||||
await first.locator('.image-card-actions button').first().click();
|
||||
await expect(panel(page).getByRole('status')).toHaveText('Metadata loads when you ask for it.');
|
||||
const load = panel(page).getByRole('button', { name: 'Load metadata', exact: true });
|
||||
await expect(load).toBeVisible();
|
||||
expect(reads).toEqual([]);
|
||||
await load.click();
|
||||
await expect(panel(page).locator('.panel-heading h2')).toBeVisible();
|
||||
expect(reads).toHaveLength(1);
|
||||
// The same file again reads nothing more (the cache is on).
|
||||
await first.locator('.image-card-actions button').first().click();
|
||||
await expect(panel(page).locator('.panel-heading h2')).toBeVisible();
|
||||
await requestBarrier(page);
|
||||
expect(reads).toHaveLength(1);
|
||||
// The viewer's details use the same inspector: another file waits there too, and loads on request.
|
||||
await imageCards(page).nth(1).locator('.image-open').click();
|
||||
const viewerDetails = page.locator('#viewer-details');
|
||||
await expect(viewerDetails.getByRole('status')).toHaveText('Metadata loads when you ask for it.');
|
||||
expect(reads).toHaveLength(1);
|
||||
await viewerDetails.getByRole('button', { name: 'Load metadata', exact: true }).click();
|
||||
await expect(viewerDetails.locator('.panel-heading h2')).toBeVisible();
|
||||
expect(reads).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('cache off rereads the current selection; on reuses the one current record', async ({ page }) => {
|
||||
const reads = [];
|
||||
page.on('request', item => { if (new URL(item.url()).pathname.startsWith(INSPECT_PATH)) reads.push(item.url()); });
|
||||
for (const cache of [true, false]) {
|
||||
if (!cache) {
|
||||
await openAppearance(page);
|
||||
await toggle(page, { 'metadata-cache': false });
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
await page.reload();
|
||||
}
|
||||
await openImages(page);
|
||||
const info = imageCards(page).first().locator('.image-card-actions button').first();
|
||||
await info.click();
|
||||
await expect(panel(page).locator('.panel-heading h2')).toBeVisible();
|
||||
await expect.poll(() => reads.length).toBeGreaterThan(0);
|
||||
await requestBarrier(page);
|
||||
reads.length = 0;
|
||||
// Inspect the selected file twice more.
|
||||
for (const _ of [1, 2]) {
|
||||
const answered = cache ? null : page.waitForResponse(response => new URL(response.url()).pathname.startsWith(INSPECT_PATH));
|
||||
await info.click();
|
||||
if (answered) await answered;
|
||||
await expect(panel(page).locator('.panel-heading h2')).toBeVisible();
|
||||
}
|
||||
await requestBarrier(page);
|
||||
expect(reads, `cache ${cache}`).toHaveLength(cache ? 0 : 2);
|
||||
}
|
||||
});
|
||||
|
||||
test('paths hidden only changes display', async ({ page }) => {
|
||||
const rows = async () => page.locator('#viewer-details details').filter({ hasText: 'File & provenance' }).locator('dt')
|
||||
.evaluateAll(terms => terms.map(term => term.textContent));
|
||||
const openViewer = async () => {
|
||||
await openImages(page);
|
||||
await imageCards(page).first().locator('.image-open').click();
|
||||
const provenance = page.locator('#viewer-details details').filter({ hasText: 'File & provenance' });
|
||||
await provenance.locator('summary').click();
|
||||
await expect(provenance.locator('dl')).toBeVisible();
|
||||
return provenance;
|
||||
};
|
||||
let provenance = await openViewer();
|
||||
expect(await rows()).toContain('Path');
|
||||
const before = { show: await provenance.getByRole('button', { name: /^Show .* in folder$/ }).count(),
|
||||
download: await provenance.getByRole('link', { name: 'Download original', exact: true }).getAttribute('href') };
|
||||
await page.keyboard.press('Escape');
|
||||
await openAppearance(page);
|
||||
await toggle(page, { paths: false });
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
provenance = await openViewer();
|
||||
const shown = await rows();
|
||||
expect(shown).not.toContain('Path');
|
||||
expect(shown).toEqual(expect.arrayContaining(['Source', 'Size']));
|
||||
// Everything else is as it was: the folder action and the original download stay.
|
||||
expect(await provenance.getByRole('button', { name: /^Show .* in folder$/ }).count()).toBe(before.show);
|
||||
await expect(provenance.getByRole('link', { name: 'Download original', exact: true })).toHaveAttribute('href', before.download);
|
||||
});
|
||||
|
||||
test('video defaults reach the viewer', async ({ page }) => {
|
||||
await openAppearance(page);
|
||||
await toggle(page, { 'video-auto': true, 'video-muted': false, 'video-loop': false });
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
expect(await page.evaluate(async () => (await import('/prompt_manager/js/workspace/media-viewer.mjs')).readViewerPreferences()))
|
||||
.toEqual({ videoAutoplay: true, videoMute: false, videoLoop: false });
|
||||
const stored = await storedSettings(page);
|
||||
expect([stored.videoAutoplay, stored.videoMute, stored.videoLoop]).toEqual([true, false, false]);
|
||||
});
|
||||
|
||||
test('host fields round-trip through /settings with revision', async ({ page, request }) => {
|
||||
const posts = [];
|
||||
page.on('request', item => { if (item.method() === 'POST' && new URL(item.url()).pathname === SETTINGS_URL) posts.push(item.postDataJSON()); });
|
||||
const before = await serverJson(request, SETTINGS_URL);
|
||||
await openAppearance(page);
|
||||
await expect(page.locator('#open-mode')).toHaveValue(before.settings.webui_display_mode);
|
||||
await expect(page.locator('#timeout')).toHaveValue(String(before.settings.result_timeout));
|
||||
// Only whole seconds from 0 to 300: anything else is highlighted and never sent.
|
||||
for (const bad of ['301', '-1', '2.5']) {
|
||||
await page.locator('#timeout').fill(bad);
|
||||
await page.locator('#timeout').press('Tab');
|
||||
await expect(page.locator('#timeout')).toHaveAttribute('aria-invalid', 'true');
|
||||
await expect(statusOf(page)).toHaveText('Some changes were not saved. Review the highlighted settings.');
|
||||
await expect(saveOf(page)).toHaveAttribute('aria-disabled', 'true');
|
||||
}
|
||||
await page.locator('#timeout').fill('0');
|
||||
await page.locator('#timeout').press('Tab');
|
||||
await expect(page.locator('#timeout')).not.toHaveAttribute('aria-invalid');
|
||||
await page.locator('#open-mode').selectOption('popup');
|
||||
await saveAppearance(page, 'Saved to the server configuration.');
|
||||
expect(posts).toEqual([{ webui_display_mode: 'popup', result_timeout: 0, expected_revision: before.revision }]);
|
||||
const after = await serverJson(request, SETTINGS_URL);
|
||||
expect([after.settings.webui_display_mode, after.settings.result_timeout]).toEqual(['popup', 0]);
|
||||
expect(after.revision).not.toBe(before.revision);
|
||||
// No browser key was written for a host field.
|
||||
expect(await storedSettings(page)).toBeNull();
|
||||
});
|
||||
|
||||
test('server settings that cannot be read say so in their rows', async ({ page }) => {
|
||||
await page.route(url => url.pathname === SETTINGS_URL, route => (route.request().method() === 'GET'
|
||||
? route.fulfill({ status: 500, contentType: 'application/json', body: JSON.stringify({ success: false, error: 'Injected failure' }) })
|
||||
: route.continue()));
|
||||
await page.goto(`${workspace}#/system/appearance`);
|
||||
for (const id of ['open-mode', 'timeout']) {
|
||||
const row = page.locator('#workspace-main .setting-row').filter({ has: page.locator(`#${id}`) });
|
||||
await expect(row.locator('p.small')).toHaveText('Server settings could not be read.');
|
||||
await expect(page.locator(`#${id}`)).toBeDisabled();
|
||||
}
|
||||
// The browser rows are this browser's, and work as before.
|
||||
await expect(page.locator('#default-view')).toBeEnabled();
|
||||
await expect(page.locator('#workspace-main .setting-row p.small')).toHaveCount(2);
|
||||
});
|
||||
|
||||
test('stale revision shows the conflict status', async ({ page, request }) => {
|
||||
await openAppearance(page);
|
||||
// Another page saves after this one read the settings.
|
||||
const current = await serverJson(request, SETTINGS_URL);
|
||||
await expect(await request.post(SETTINGS_URL, { data: { result_timeout: 42, expected_revision: current.revision } })).toBeOK();
|
||||
await page.locator('#open-mode').selectOption('popup');
|
||||
await saveAppearance(page, 'Settings changed elsewhere. Reload this section to see them.');
|
||||
await expect(page.locator('#open-mode')).toHaveAttribute('aria-invalid', 'true');
|
||||
// The draft is kept and the section now shows the other page's value.
|
||||
await expect(page.locator('#open-mode')).toHaveValue('popup');
|
||||
await expect(page.locator('#timeout')).toHaveValue('42');
|
||||
expect((await serverJson(request, SETTINGS_URL)).settings.webui_display_mode).toBe('newtab');
|
||||
// With the reloaded revision, Save succeeds.
|
||||
await saveAppearance(page, 'Saved to the server configuration.');
|
||||
expect((await serverJson(request, SETTINGS_URL)).settings.webui_display_mode).toBe('popup');
|
||||
});
|
||||
|
||||
test('reload keeps saved values; a new tab reads them', async ({ page, context }) => {
|
||||
await openAppearance(page);
|
||||
await page.locator('#image-quality').selectOption('high');
|
||||
await toggle(page, { 'card-details': false });
|
||||
await page.locator('#open-mode').selectOption('popup');
|
||||
await saveAppearance(page, 'Saved in this browser. Saved to the server configuration.');
|
||||
await page.reload();
|
||||
await expect(page.locator('#timeout')).toBeEnabled();
|
||||
await expect(page.locator('#image-quality')).toHaveValue('high');
|
||||
await expect(page.locator('#card-details')).not.toBeChecked();
|
||||
await expect(page.locator('#open-mode')).toHaveValue('popup');
|
||||
const other = await context.newPage();
|
||||
await openAppearance(other);
|
||||
await expect(other.locator('#image-quality')).toHaveValue('high');
|
||||
await expect(other.locator('#card-details')).not.toBeChecked();
|
||||
// A change saved in one tab reaches the other's clean controls; a draft it overtakes is flagged and kept.
|
||||
await other.locator('#density').selectOption('large');
|
||||
await page.locator('#density').selectOption('compact');
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
await expect(other.locator('#density')).toHaveValue('large');
|
||||
await expect(other.locator('#settings-status')).toHaveText('Settings changed elsewhere. Reload this section to see them.');
|
||||
await expect(other.locator('#density')).toHaveAttribute('aria-invalid', 'true');
|
||||
await page.locator('#image-quality').selectOption('low');
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
await expect(other.locator('#image-quality')).toHaveValue('low');
|
||||
await other.close();
|
||||
});
|
||||
|
||||
test('a change another tab saved while System was closed is a conflict on return, never saved over silently', async ({ page, context }) => {
|
||||
await openAppearance(page);
|
||||
await page.locator('#density').selectOption('compact');
|
||||
await expect(statusOf(page)).toHaveText('Unsaved changes.');
|
||||
// Leave System for Images: the draft stays for the page session, but System no longer listens.
|
||||
await page.locator('#primary-nav a[data-route="images"]').click();
|
||||
await expect(imageCards(page).first()).toBeVisible();
|
||||
const other = await context.newPage();
|
||||
await openAppearance(other);
|
||||
await other.locator('#density').selectOption('large');
|
||||
await saveAppearance(other, 'Saved in this browser.');
|
||||
await other.close();
|
||||
// Back in System: the draft is kept and flagged, and nothing was written over the other tab's value.
|
||||
await page.evaluate(() => { location.hash = '#/system/appearance'; });
|
||||
await expect(page.locator('#workspace-main .settings-layout > section.surface > h2')).toHaveText('Browsing & display');
|
||||
await expect(statusOf(page)).toHaveText('Settings changed elsewhere. Reload this section to see them.');
|
||||
await expect(page.locator('#density')).toHaveAttribute('aria-invalid', 'true');
|
||||
await expect(page.locator('#density')).toHaveValue('compact');
|
||||
expect((await storedSettings(page)).workspaceDensity).toBe('large');
|
||||
// Saving after the conflict was shown is an explicit choice: it wins.
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
expect((await storedSettings(page)).workspaceDensity).toBe('compact');
|
||||
await expect(page.locator('#density')).not.toHaveAttribute('aria-invalid');
|
||||
});
|
||||
|
||||
test('a browser that refuses storage is saved for this tab only, never "Saved"', async ({ page }) => {
|
||||
await openAppearance(page);
|
||||
await page.evaluate(() => {
|
||||
Storage.prototype.setItem = function setItem() { throw new DOMException('Quota exceeded', 'QuotaExceededError'); };
|
||||
});
|
||||
await page.locator('#default-view').selectOption('list');
|
||||
await saveAppearance(page, 'Saved for this tab only; browser storage is unavailable.');
|
||||
expect(await storedSettings(page)).toBeNull();
|
||||
// This tab's next Images visit still uses it.
|
||||
await page.locator('#primary-nav a[data-route="images"]').click();
|
||||
await expect(imageCards(page).first()).toBeVisible();
|
||||
await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list');
|
||||
});
|
||||
|
||||
test('added rows reach their consumers', async ({ page }) => {
|
||||
await openImages(page);
|
||||
await expect(imageCards(page).first().locator('.media-info h3')).toBeVisible();
|
||||
await openAppearance(page);
|
||||
await toggle(page, { 'card-details': false, 'viewer-details': true });
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
await page.reload();
|
||||
await openImages(page);
|
||||
// Show image details on cards off: no title or file line; Info and Tag stay.
|
||||
const card = imageCards(page).first();
|
||||
await expect(card.locator('.media-info h3')).toBeHidden();
|
||||
await expect(card.locator('.image-card-actions button')).toHaveCount(2);
|
||||
// Collapse viewer details by default: the viewer opens with its details hidden behind one toggle.
|
||||
await card.locator('.image-open').click();
|
||||
const details = page.locator('#viewer-details');
|
||||
await expect(page.locator('#viewer-dialog')).toBeVisible();
|
||||
await expect(details).toBeHidden();
|
||||
const show = page.locator('#viewer-toolbar').getByRole('button', { name: 'Show details', exact: true });
|
||||
await expect(show).toHaveAttribute('aria-expanded', 'false');
|
||||
await show.click();
|
||||
await expect(details).toBeVisible();
|
||||
await expect(page.locator('#viewer-toolbar').getByRole('button', { name: 'Hide details', exact: true })).toHaveAttribute('aria-expanded', 'true');
|
||||
});
|
||||
|
||||
test('reset restores defaults without touching other keys', async ({ page }) => {
|
||||
await page.goto(`${workspace}#/system/appearance`);
|
||||
await page.evaluate(() => localStorage.setItem('gallerySettings', JSON.stringify({ gridColumns: 5, defaultLimit: 200,
|
||||
thumbnailPromptDismissed: 'kept', workspaceLayout: 'list', videoMute: false, showImageInfo: false })));
|
||||
await page.reload();
|
||||
await expect(page.locator('#timeout')).toBeEnabled();
|
||||
await expect(page.locator('#default-view')).toHaveValue('list');
|
||||
const reset = page.locator('#workspace-main .operation').filter({ hasText: 'Reset display defaults' }).getByRole('button', { name: 'Reset', exact: true });
|
||||
await reset.click();
|
||||
await expect(page.locator('#default-view')).toHaveValue('grid');
|
||||
await expect(page.locator('#video-muted')).toBeChecked();
|
||||
await expect(statusOf(page)).toHaveText('Unsaved changes.');
|
||||
await saveAppearance(page, 'Saved in this browser.');
|
||||
const stored = await storedSettings(page);
|
||||
expect(stored).toEqual({ gridColumns: 5, defaultLimit: 200, thumbnailPromptDismissed: 'kept', workspaceLayout: 'grid',
|
||||
videoMute: true, showImageInfo: true });
|
||||
});
|
||||
@@ -340,6 +340,9 @@ test('Tag opens the dialog without selecting the card', async ({ page }) => {
|
||||
const panel = page.getByRole('complementary', { name: 'Selected image details', exact: true });
|
||||
await page.goto(images);
|
||||
await expect(card(page, FIRST)).toHaveClass(/\bselected\b/);
|
||||
// Capture the panel once it shows the first card's details, not while it is still reading them.
|
||||
await expect(panel.getByRole('heading', { level: 2 })).toBeVisible();
|
||||
await expect(panel).not.toContainText('Loading file details');
|
||||
const before = await panel.innerText();
|
||||
await openTag(page, SECOND, { goto: false });
|
||||
await page.keyboard.press('Escape');
|
||||
|
||||
@@ -18,7 +18,8 @@ const routes = [
|
||||
['tags', 'Tags & categories', null],
|
||||
// #249 local Inspect: the native Inspect a file page; Overview's tool card still links the legacy metadata page.
|
||||
['inspect', 'Inspect a file', null],
|
||||
['system', 'System', '/prompt_manager/admin'],
|
||||
// #249 System S2: native System; its undelivered sections hand off per section (D94, asserted below).
|
||||
['system', 'System', null],
|
||||
// #249 D51: SAVED VIEWS' Favorites, Images filtered to favorites.
|
||||
['favorites', 'Favorites', null],
|
||||
// #249 Activity-1: the concept's Activity page in the tools-and-system group.
|
||||
@@ -409,6 +410,8 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept
|
||||
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
|
||||
|
||||
await sidebarLink(page, 'System').click();
|
||||
// System's Overview is native (S2 Task 6); a section not delivered yet still hands off (D94).
|
||||
await main.locator('.section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
const handoff = main.locator('section.handoff');
|
||||
await expect(handoff).toHaveClass(/\bsurface\b/);
|
||||
expect(await handoff.evaluate(surfaceStyle)).toMatchObject(SURFACE);
|
||||
@@ -417,6 +420,100 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept
|
||||
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
|
||||
});
|
||||
|
||||
// #249 System S2 Task 5: the concept's renderSystem shell (app.js 698–706) at `#/system/<section>`, every section a D94
|
||||
// hand-off until its PR, and the Save row's truthful status (D92).
|
||||
const SYSTEM_SECTION_LABELS = ['Overview', 'Appearance', 'Sources & scans', 'Integrations', 'Auto-tagging', 'Maintenance',
|
||||
'Backup & export', 'Diagnostics & logs'];
|
||||
const SYSTEM_HASHES = ['overview', 'appearance', 'sources', 'integrations', 'tagging', 'maintenance', 'backups', 'diagnostics'];
|
||||
const SYSTEM_HANDOFFS = { maintenance: ['/prompt_manager/admin', '/prompt_manager/gallery'] };
|
||||
|
||||
test('System is native: the concept shell, section navigation, sections, hand-offs and the Save row', async ({ page }) => {
|
||||
const mutations = [];
|
||||
page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) mutations.push(item.url()); });
|
||||
await page.goto(`${workspace}#/system`);
|
||||
const main = page.locator('#workspace-main');
|
||||
await expect(main.locator('h1')).toHaveText('System');
|
||||
await expect(main.locator('.heading p.muted')).toHaveText('Configuration, maintenance and recovery—with browsing kept in the workspace.');
|
||||
// D92: no "Sample configuration" chip.
|
||||
await expect(main.locator('.heading .chip')).toHaveCount(0);
|
||||
expect(await main.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`)))
|
||||
.toEqual(['div.heading', 'div.section-nav', 'div.settings-layout']);
|
||||
const nav = main.getByRole('group', { name: 'System sections', exact: true });
|
||||
await expect(nav.getByRole('button')).toHaveText(SYSTEM_SECTION_LABELS);
|
||||
await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true');
|
||||
await expect(nav.locator('[aria-pressed="true"]')).toHaveCount(1);
|
||||
await expect(page.locator('aside.sidebar a[href="#/system"]')).toHaveAttribute('aria-current', 'page');
|
||||
const layout = main.locator('.settings-layout');
|
||||
await expect(layout.locator(':scope > .save-row > span.small#settings-status')).toHaveText('No unsaved changes.');
|
||||
const save = layout.locator(':scope > .save-row > button.ca-primary');
|
||||
await expect(save).toHaveText('Save preferences');
|
||||
await expect(save).toHaveAttribute('aria-disabled', 'true');
|
||||
// Playwright waits on an aria-disabled control, so the ignored press is forced.
|
||||
await save.click({ force: true });
|
||||
await expect(layout.locator('#settings-status')).toHaveText('No unsaved changes.');
|
||||
|
||||
for (const [index, label] of SYSTEM_SECTION_LABELS.entries()) {
|
||||
const hash = SYSTEM_HASHES[index];
|
||||
const button = nav.getByRole('button', { name: label, exact: true });
|
||||
await button.click();
|
||||
await expect(page).toHaveURL(`${workspace}#/system/${hash}`);
|
||||
await expect(button).toHaveAttribute('aria-pressed', 'true');
|
||||
await expect(nav.locator('[aria-pressed="true"]')).toHaveCount(1);
|
||||
// The section changes in place: focus stays on the pressed section, not the page title.
|
||||
await expect(button).toBeFocused();
|
||||
// S2 Task 6: Overview is native (workspace-system.spec.mjs), and so are Appearance (S3 Task 8) and Sources & scans
|
||||
// (S4 Task 11, workspace-system-sources.spec.mjs, its update notice shown in a fresh browser); Maintenance shows
|
||||
// its health-check notice above the hand-off (D123); every other section hands off.
|
||||
if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible();
|
||||
if (hash === 'sources') await expect(layout.locator('#source-list .operation')).toHaveCount(2);
|
||||
expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`)))
|
||||
.toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'div.save-row'],
|
||||
sources: ['div.notice', 'section.surface', 'div.save-row'],
|
||||
maintenance: ['div.notice', 'section.surface handoff', 'div.save-row'] }[hash]
|
||||
?? ['section.surface handoff', 'div.save-row']);
|
||||
if (['overview', 'appearance', 'sources'].includes(hash)) continue;
|
||||
const handoff = layout.locator('section.handoff');
|
||||
await expect(handoff.locator('h2')).toHaveText('Your current tools are ready');
|
||||
expect(await handoff.locator('a.workspace-link').evaluateAll(links => links.map(link => link.getAttribute('href'))))
|
||||
.toEqual(SYSTEM_HANDOFFS[hash] ?? ['/prompt_manager/admin']);
|
||||
await expect(layout.locator('#settings-status')).toHaveText('No unsaved changes.');
|
||||
await expect(page.locator('aside.sidebar a[href="#/system"]')).toHaveAttribute('aria-current', 'page');
|
||||
}
|
||||
// Deep links, reload and history keep the section; an unknown section is Overview under its own hash.
|
||||
await page.reload();
|
||||
await expect(nav.getByRole('button', { name: 'Diagnostics & logs', exact: true })).toHaveAttribute('aria-pressed', 'true');
|
||||
await page.goBack();
|
||||
await expect(page).toHaveURL(`${workspace}#/system/backups`);
|
||||
await expect(nav.getByRole('button', { name: 'Backup & export', exact: true })).toHaveAttribute('aria-pressed', 'true');
|
||||
await page.goto(`${workspace}#/system/general`);
|
||||
await expect(page).toHaveURL(`${workspace}#/system/overview`);
|
||||
await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true');
|
||||
// The sidebar's System returns to the Overview section.
|
||||
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
await sidebarLink(page, 'System').click();
|
||||
await expect(page).toHaveURL(`${workspace}#/system`);
|
||||
await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true');
|
||||
// Leaving System and coming back renders it once, with a fresh section.
|
||||
await sidebarLink(page, 'Images').click();
|
||||
await expect(main.locator('h1')).toHaveText('Images');
|
||||
await expect(main.locator('.section-nav')).toHaveCount(0);
|
||||
await page.goBack();
|
||||
await expect(main.locator('.section-nav')).toHaveCount(1);
|
||||
await expect(main.locator('.settings-layout')).toHaveCount(1);
|
||||
expect(mutations).toEqual([]);
|
||||
});
|
||||
|
||||
test('Overview opens native System from its System card (Q12)', async ({ page }) => {
|
||||
await page.goto(`${workspace}#/overview`);
|
||||
const card = page.locator('#workspace-main .tool-card').filter({ has: page.getByRole('heading', { name: 'System', exact: true }) });
|
||||
await expect(card.locator('p.small')).toHaveText('Configuration, maintenance and recovery—with browsing kept in the workspace.');
|
||||
const link = card.getByRole('link', { name: 'Open System', exact: true });
|
||||
await expect(link).toHaveAttribute('href', '#/system/overview');
|
||||
await link.click();
|
||||
await expect(page.locator('#workspace-main h1')).toHaveText('System');
|
||||
await expect(page.locator('#workspace-main .section-nav [aria-pressed="true"]')).toHaveText('Overview');
|
||||
});
|
||||
|
||||
// Task A3: the concept sidebar (index.html h16-41, app.js nav()) rendered from real data (D27, D28).
|
||||
const NAV_TOTALS = [
|
||||
['images', 'Images', '/prompt_manager/images/output?mode=workspace&limit=1'],
|
||||
|
||||
@@ -11,6 +11,7 @@ the same sidebar counts (24 images, 6 prompts, 15 tags). Nothing is mocked.
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
import datetime
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
@@ -157,6 +158,8 @@ CONCEPT_MODEL = "illustrative-model.safetensors"
|
||||
MAX_EXTRA_IMAGES = 120
|
||||
# Unlinked, seedless images (D11 "W × H · PNG") newer than the concept's, for D3 checks.
|
||||
MAX_SEEDLESS_IMAGES = 2
|
||||
# A test's chosen "now" (epoch seconds) stays before the year 3000.
|
||||
MAX_EPOCH_SECONDS = 32_503_680_000
|
||||
SEEDLESS_SIZE = (1536, 1152)
|
||||
# The same 4:3 shape with a short "96 × 72 · PNG" subtitle that stays on one line at
|
||||
# 390px beside the card's Info and Tag (the D3 footer check).
|
||||
@@ -173,6 +176,8 @@ _TABLES = (
|
||||
"execution_context_texts",
|
||||
"saved_views",
|
||||
"image_favorites",
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
@@ -342,6 +347,17 @@ def _bounded(name: str, value: int, maximum: int) -> None:
|
||||
raise ValueError(f"{name} is out of range")
|
||||
|
||||
|
||||
def _instant(now: float | None) -> float:
|
||||
"""The fixture's one "now": ``now`` itself, or the wall clock when omitted."""
|
||||
if now is None:
|
||||
return time.time()
|
||||
if isinstance(now, bool) or not isinstance(now, (int, float)):
|
||||
raise TypeError("now must be a number of epoch seconds")
|
||||
if not 0 < now < MAX_EPOCH_SECONDS:
|
||||
raise ValueError("now is out of range")
|
||||
return float(now)
|
||||
|
||||
|
||||
def seed_workspace_concept(
|
||||
api: PromptManagerAPI,
|
||||
root: Path,
|
||||
@@ -349,9 +365,16 @@ def seed_workspace_concept(
|
||||
extra_images: int = 0,
|
||||
seedless_images: int = 0,
|
||||
short_seedless: bool = False,
|
||||
now: float | None = None,
|
||||
) -> dict:
|
||||
"""Replace this server's disposable rows and files with the concept's data.
|
||||
|
||||
Every relative time the fixture writes (the prompts' last use, the files'
|
||||
mtimes, the runs' observation times) is taken from one instant, ``now``
|
||||
(epoch seconds, the wall clock when omitted), returned as ``manifest["now"]``.
|
||||
A browser whose clock is fixed at that instant reads the concept's "Used
|
||||
today" and caption day however close to midnight the run starts.
|
||||
|
||||
``extra_images`` continues the concept's rounds (same prompts, dimensions and
|
||||
seed rule) after its 24 images, for bounded-rendering and Load more checks.
|
||||
``seedless_images`` adds unlinked PNGs without metadata, newest of all;
|
||||
@@ -361,6 +384,10 @@ def seed_workspace_concept(
|
||||
raise TypeError("short_seedless must be a boolean")
|
||||
_bounded("extra_images", extra_images, MAX_EXTRA_IMAGES)
|
||||
_bounded("seedless_images", seedless_images, MAX_SEEDLESS_IMAGES)
|
||||
now = _instant(now)
|
||||
used_at = datetime.datetime.fromtimestamp(now, datetime.timezone.utc).isoformat(
|
||||
timespec="microseconds"
|
||||
)
|
||||
_clear(api, root)
|
||||
db = api.db
|
||||
epoch = db.get_execution_epoch()
|
||||
@@ -385,8 +412,15 @@ def seed_workspace_concept(
|
||||
db.update_prompt_pair_metadata(pair_ids[0], title=first_name)
|
||||
for name, text in prompt.negatives[1:]:
|
||||
pair_ids.append(db.save_prompt_pair(prompt.positive, text, title=name))
|
||||
# Real queue-hook counting (D23): four runs, the last one now.
|
||||
# Real queue-hook counting (D23): four runs, the last one now, at the
|
||||
# fixture's own instant rather than the database's later wall clock.
|
||||
db.record_prompt_use(positive_id, times=CONCEPT_RUN_COUNT)
|
||||
with db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompts SET last_used_at = ? WHERE id = ?",
|
||||
(used_at, positive_id),
|
||||
)
|
||||
conn.commit()
|
||||
prompts.insert(
|
||||
0, {"id": prompt.id, "positive_id": positive_id, "pair_ids": pair_ids}
|
||||
)
|
||||
@@ -400,7 +434,6 @@ def seed_workspace_concept(
|
||||
favorite_root = _favorite_root_key(list(api._get_all_output_dirs())[0])
|
||||
total = len(CONCEPT_PROMPTS) * CONCEPT_IMAGE_ROUNDS + extra_images
|
||||
rounds = math.ceil(total / len(CONCEPT_PROMPTS))
|
||||
now = time.time()
|
||||
images, runs = [], []
|
||||
slots = [
|
||||
(round_index, prompt_index, prompt)
|
||||
@@ -484,6 +517,8 @@ def seed_workspace_concept(
|
||||
api.invalidate_gallery_cache()
|
||||
return {
|
||||
"epoch": epoch,
|
||||
# The fixture's "now" (epoch seconds), for the browser's fixed clock.
|
||||
"now": now,
|
||||
# The newest file's mtime (epoch seconds), for the D17 caption date.
|
||||
"newest_modified": now - 10 if seedless_images else now - 60,
|
||||
"seedless": seedless,
|
||||
|
||||
@@ -28,6 +28,8 @@ EDITOR_METADATA = {
|
||||
_TABLES = (
|
||||
"saved_views",
|
||||
"image_favorites",
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
"""System test hooks for the browser tests (#249 System S2 Task 6).
|
||||
|
||||
The data is the concept-shaped real fixture (``POST /__test__/concept/reset``,
|
||||
``workspace_concept_fixture.py``): its parents, saved pairs, images, ratings
|
||||
and runs are what the Overview, the Statistics dialog and the health check
|
||||
read. These per-server hooks only change what a System test needs on top:
|
||||
|
||||
- ``POST /__test__/system/reset`` restores the configured sources, forgets the
|
||||
session's health report and ends every injected failure;
|
||||
- ``POST /__test__/system/fail`` ``{path, times}`` answers the next ``times``
|
||||
requests for ``path`` (a ``/prompt_manager/system/`` route) with the routes'
|
||||
500 error envelope;
|
||||
- ``POST /__test__/system/missing-source`` configures one more source that
|
||||
does not exist (an unmounted drive); with ``{"inside": true}`` it is an
|
||||
existing folder inside the output folder (``output/unplugged``), which
|
||||
``POST /__test__/system/unplug`` then removes (a drive unplugged while the
|
||||
page shows it);
|
||||
- ``POST /__test__/system/missing-file`` deletes the newest linked image's
|
||||
file from disk, leaving its database row (a missing-file link);
|
||||
- ``POST /__test__/system/unthumbnailed`` ``{count}`` writes ``count`` plain
|
||||
PNGs with no thumbnail into the output folder;
|
||||
- ``POST /__test__/system/monitor`` ``{watching, alive}`` stands in a fake
|
||||
image monitor whose observer is ``alive`` (default true) and schedules the
|
||||
named folders of this server's root (``["output"]``), so Sources reads
|
||||
watching, manual scan and restart pending states (S4).
|
||||
|
||||
Delays use the shared request holds (``request_hold.py``).
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from aiohttp import web
|
||||
from PIL import Image
|
||||
|
||||
FAILABLE_PREFIX = "/prompt_manager/system/"
|
||||
MAX_UNTHUMBNAILED = 50
|
||||
|
||||
|
||||
def register_system_commands(app, routes, api, root: Path):
|
||||
# System › Appearance saves the host fields to this server's config.json (S3);
|
||||
# reset restores both. The file is this server's own, inside its temp root:
|
||||
# reset rewrites or deletes it, so a server whose configuration points
|
||||
# anywhere else (a real config.json) refuses to register these hooks.
|
||||
config_path = root / "config.json"
|
||||
configured_path = os.environ.get("PROMPT_MANAGER_CONFIG_PATH")
|
||||
if (
|
||||
configured_path is None
|
||||
or Path(configured_path).resolve() != config_path.resolve()
|
||||
):
|
||||
raise RuntimeError(
|
||||
f"PROMPT_MANAGER_CONFIG_PATH must be {config_path}, not {configured_path!r}"
|
||||
)
|
||||
from py.config import GalleryConfig, PromptManagerConfig
|
||||
from utils import image_monitor
|
||||
|
||||
configured = list(GalleryConfig.MONITORING_DIRECTORIES)
|
||||
config_bytes = config_path.read_bytes() if config_path.exists() else None
|
||||
host_values = (
|
||||
PromptManagerConfig.RESULT_TIMEOUT,
|
||||
PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||||
PromptManagerConfig.WORKER_THREADS,
|
||||
GalleryConfig.MONITORING_ENABLED,
|
||||
)
|
||||
monitor = getattr(image_monitor, "_monitor_instance", None)
|
||||
failures: dict[str, int] = {}
|
||||
|
||||
@web.middleware
|
||||
async def inject_failures(request, handler):
|
||||
left = failures.get(request.path, 0)
|
||||
if left <= 0:
|
||||
return await handler(request)
|
||||
failures[request.path] = left - 1
|
||||
return web.json_response(
|
||||
{"success": False, "code": None, "error": "Injected failure"}, status=500
|
||||
)
|
||||
|
||||
app.middlewares.append(inject_failures)
|
||||
|
||||
unplugged = root / "output" / "unplugged"
|
||||
|
||||
def reset():
|
||||
failures.clear()
|
||||
if unplugged.is_dir():
|
||||
unplugged.rmdir()
|
||||
GalleryConfig.MONITORING_DIRECTORIES = list(configured)
|
||||
api._health_report = None
|
||||
(
|
||||
PromptManagerConfig.RESULT_TIMEOUT,
|
||||
PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||||
PromptManagerConfig.WORKER_THREADS,
|
||||
GalleryConfig.MONITORING_ENABLED,
|
||||
) = host_values
|
||||
image_monitor._monitor_instance = monitor
|
||||
if config_bytes is None:
|
||||
config_path.unlink(missing_ok=True)
|
||||
else:
|
||||
# Replace in one step, as the server writes it (Windows preflight).
|
||||
staged = config_path.with_name(config_path.name + ".reset")
|
||||
staged.write_bytes(config_bytes)
|
||||
os.replace(staged, config_path)
|
||||
|
||||
@routes.post("/__test__/system/reset")
|
||||
async def system_reset(request):
|
||||
reset()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/fail")
|
||||
async def system_fail(request):
|
||||
data = await request.json()
|
||||
path, times = data.get("path"), data.get("times", 1)
|
||||
if (
|
||||
not isinstance(path, str)
|
||||
or not path.startswith(FAILABLE_PREFIX)
|
||||
or isinstance(times, bool)
|
||||
or not isinstance(times, int)
|
||||
or not 1 <= times <= 10
|
||||
):
|
||||
raise web.HTTPBadRequest()
|
||||
failures[path] = times
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/missing-source")
|
||||
async def missing_source(request):
|
||||
data = await request.json() if request.can_read_body else {}
|
||||
if data.get("inside") is True:
|
||||
unplugged.mkdir(exist_ok=True)
|
||||
absent = str(unplugged)
|
||||
else:
|
||||
absent = str(root / "unmounted-drive")
|
||||
if os.path.exists(absent):
|
||||
raise web.HTTPConflict()
|
||||
GalleryConfig.MONITORING_DIRECTORIES = [*configured, absent]
|
||||
return web.json_response(
|
||||
{"success": True, "configured": len(GalleryConfig.MONITORING_DIRECTORIES)}
|
||||
)
|
||||
|
||||
@routes.post("/__test__/system/unplug")
|
||||
async def unplug(request):
|
||||
if not unplugged.is_dir():
|
||||
raise web.HTTPConflict()
|
||||
unplugged.rmdir()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/missing-file")
|
||||
async def missing_file(request):
|
||||
with api.db.model.get_connection() as conn:
|
||||
row = conn.execute(
|
||||
"SELECT file_path FROM generated_images ORDER BY id DESC LIMIT 1"
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise web.HTTPNotFound()
|
||||
Path(row[0]).unlink()
|
||||
api.invalidate_gallery_cache()
|
||||
return web.json_response({"success": True, "filename": Path(row[0]).name})
|
||||
|
||||
@routes.post("/__test__/system/unthumbnailed")
|
||||
async def unthumbnailed(request):
|
||||
data = await request.json()
|
||||
count = data.get("count")
|
||||
if (
|
||||
isinstance(count, bool)
|
||||
or not isinstance(count, int)
|
||||
or not 1 <= count <= MAX_UNTHUMBNAILED
|
||||
):
|
||||
raise web.HTTPBadRequest()
|
||||
folder = root / "output" / "unthumbnailed"
|
||||
folder.mkdir(exist_ok=True)
|
||||
for number in range(count):
|
||||
with Image.new("RGB", (64, 64), "#2b3442") as image:
|
||||
image.save(folder / f"plain_{number + 1:03}.png")
|
||||
api.invalidate_gallery_cache()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/monitor")
|
||||
async def fake_monitor(request):
|
||||
data = await request.json()
|
||||
watching, alive = data.get("watching", []), data.get("alive", True)
|
||||
if (
|
||||
not isinstance(watching, list)
|
||||
or not all(name in ("output", "input") for name in watching)
|
||||
or not isinstance(alive, bool)
|
||||
):
|
||||
raise web.HTTPBadRequest()
|
||||
image_monitor._monitor_instance = _FakeMonitor(
|
||||
[str(root / name) for name in watching], alive
|
||||
)
|
||||
return web.json_response({"success": True})
|
||||
|
||||
return reset
|
||||
|
||||
|
||||
class _FakeMonitor:
|
||||
"""The image monitor's status surface (``get_status``) with no observer thread."""
|
||||
|
||||
def __init__(self, directories, alive):
|
||||
self.monitored_directories = directories
|
||||
self._alive = alive
|
||||
|
||||
def get_status(self):
|
||||
return {
|
||||
"running": True,
|
||||
"monitored_directories": list(self.monitored_directories),
|
||||
"handler_active": True,
|
||||
"observer_alive": self._alive,
|
||||
}
|
||||
@@ -44,6 +44,8 @@ MAX_EXTRA_IMAGES = 160
|
||||
_TABLES = (
|
||||
"saved_views",
|
||||
"image_favorites",
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
|
||||
@@ -114,6 +114,8 @@ def seed_workspace_visual(api: PromptManagerAPI, root: Path) -> None:
|
||||
for table in (
|
||||
"saved_views",
|
||||
"image_favorites",
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
|
||||
@@ -81,6 +81,8 @@ MAX_EXTRA_CATEGORIES = 200
|
||||
_TABLES = (
|
||||
"saved_views",
|
||||
"image_favorites",
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
|
||||
@@ -1,11 +1,20 @@
|
||||
import { expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
|
||||
import runtime from '../../tools/vendor/comfyui-runtime.json' with { type: 'json' };
|
||||
import { hostBaseDirectory } from '../../tools/vendor/comfyui-host.mjs';
|
||||
|
||||
export function installHostHooks(test, expectedTypes = ['PromptManager', 'PromptManagerText', 'PromptSearchList']) {
|
||||
test.beforeEach(async ({ page, context, request }, testInfo) => {
|
||||
const response = await request.get('/system_stats');
|
||||
expect(response.ok()).toBe(true);
|
||||
const info = (await response.json()).system;
|
||||
const stats = await response.json();
|
||||
// comfyui.config.mjs's launcher reports the host it owns. The execution suite has no
|
||||
// web server: it starts its own host per worker behind a port refusal.
|
||||
if (testInfo.config.webServer) {
|
||||
const ownedBase = process.env.PM_COMFY_HOST_BASE;
|
||||
expect(ownedBase, 'the launcher reported the host it started').toBeTruthy();
|
||||
expect(hostBaseDirectory(stats), 'the host answering the port is the one this run started').toBe(ownedBase);
|
||||
}
|
||||
const info = stats.system;
|
||||
expect(info.comfyui_version).toBe(runtime.serverVersion);
|
||||
expect(info.pytorch_version).toBe(`${runtime.torchVersion}+cpu`);
|
||||
expect(info.required_frontend_version).toBe(runtime.frontendVersion);
|
||||
|
||||
+4
-4
@@ -594,12 +594,12 @@
|
||||
"anchor": null,
|
||||
"expect": {
|
||||
"metadata_status": "available",
|
||||
"state": "ambiguous",
|
||||
"reason": "Multiple sampler branches",
|
||||
"state": "resolved",
|
||||
"reason": "",
|
||||
"reason_code": null,
|
||||
"selection": null,
|
||||
"positive": null,
|
||||
"negative": null,
|
||||
"positive": "a red fox in snow, detailed fur",
|
||||
"negative": "blurry, lowres",
|
||||
"candidate_count": 0,
|
||||
"facts": {
|
||||
"seed": {
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -355,3 +355,20 @@ test("keepalive is forwarded only when asked, so a request can outlive its page"
|
||||
for (const call of fetch.calls) call.resolve(jsonResponse(200, { success: true }));
|
||||
await Promise.all(writes);
|
||||
});
|
||||
|
||||
test("ApiError keeps a validated running job on busy and a validated source on invalid-source", () => {
|
||||
const body = { running: { id: "a".repeat(32), title: "Scan for metadata" }, source: "output/gone" };
|
||||
const busy = new ApiError(409, "busy", "busy", null, body);
|
||||
assert.deepEqual([busy.running, busy.source], [{ id: "a".repeat(32), title: "Scan for metadata" }, null]);
|
||||
assert.notEqual(busy.running, body.running, "server objects must be copied");
|
||||
const invalid = new ApiError(400, "refused", "invalid-source", null, body);
|
||||
assert.deepEqual([invalid.running, invalid.source], [null, "output/gone"]);
|
||||
for (const running of [null, "x", { id: 1, title: "t" }, { id: "a b", title: "t" }, { id: "a", title: "" },
|
||||
{ id: "a", title: "x".repeat(201) }, { id: "a".repeat(65), title: "t" }]) {
|
||||
assert.equal(new ApiError(409, "busy", "busy", null, { running }).running, null);
|
||||
}
|
||||
for (const source of [null, 7, "", "x".repeat(4097)]) {
|
||||
assert.equal(new ApiError(400, "refused", "invalid-source", null, { source }).source, null);
|
||||
}
|
||||
assert.deepEqual([new ApiError(409, "old caller").running, new ApiError(409, "old caller").source], [null, null]);
|
||||
});
|
||||
|
||||
@@ -61,3 +61,50 @@ test("infinite scrolling is off by default and must be a boolean", () => {
|
||||
assert.equal(normalizeGallerySettings({ infiniteScroll: true }).infiniteScroll, true);
|
||||
assert.equal(normalizeGallerySettings({ infiniteScroll: "yes" }).infiniteScroll, false);
|
||||
});
|
||||
|
||||
test("the workspace keys have their own defaults and validation", () => {
|
||||
const { WORKSPACE_LAYOUTS, WORKSPACE_DENSITIES } = require("../../web/js/gallery-settings.js");
|
||||
assert.equal(DEFAULT_SETTINGS.workspaceLayout, "grid");
|
||||
assert.equal(DEFAULT_SETTINGS.workspaceDensity, "comfortable");
|
||||
assert.deepEqual(WORKSPACE_LAYOUTS, ["grid", "list", "masonry"]);
|
||||
assert.deepEqual(WORKSPACE_DENSITIES, ["comfortable", "large", "compact"]);
|
||||
assert.equal(normalizeGallerySettings({ workspaceLayout: "masonry" }).workspaceLayout, "masonry");
|
||||
assert.equal(normalizeGallerySettings({ workspaceLayout: "mosaic" }).workspaceLayout, "grid");
|
||||
assert.equal(normalizeGallerySettings({ workspaceDensity: "large" }).workspaceDensity, "large");
|
||||
assert.equal(normalizeGallerySettings({ workspaceDensity: 3 }).workspaceDensity, "comfortable");
|
||||
// The legacy gallery's own view mode is separate: it has no masonry.
|
||||
assert.equal(normalizeGallerySettings({ defaultViewMode: "masonry" }).defaultViewMode, "grid");
|
||||
});
|
||||
|
||||
test("a legacy gallery Save keeps the workspace keys and every other stored key", () => {
|
||||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
const vm = require("node:vm");
|
||||
const settings = require("../../web/js/gallery-settings.js");
|
||||
const storage = new Map([["gallerySettings", JSON.stringify({ workspaceLayout: "masonry", workspaceDensity: "compact", someOtherTool: 1 })]]);
|
||||
const controls = {
|
||||
lazyLoadingToggle: { checked: true }, imageQualitySelect: { value: "high" }, defaultViewModeSelect: { value: "list" },
|
||||
defaultLimitSelect: { value: "200" }, gridColumnsSelect: { value: "6" }, showImageInfoToggle: { checked: false },
|
||||
autoLoadMetadataToggle: { checked: true }, cacheMetadataToggle: { checked: true }, showFilePathsToggle: { checked: false },
|
||||
sidebarCollapsedByDefaultToggle: { checked: true }, infiniteScrollToggle: { checked: false }, debugModeToggle: { checked: false },
|
||||
apiTimeoutInput: { value: "30" }, checkThumbnailsAtStartup: { checked: true }, videoAutoplayToggle: { checked: true },
|
||||
videoMuteToggle: { checked: false }, videoLoopToggle: { checked: true },
|
||||
};
|
||||
const context = vm.createContext({
|
||||
document: { addEventListener() {}, getElementById: id => controls[id] },
|
||||
window: {}, console: { debug() {}, warn() {}, error() {} }, GallerySettings: settings,
|
||||
localStorage: { getItem: key => storage.get(key) ?? null, setItem: (key, value) => storage.set(key, value) },
|
||||
});
|
||||
const file = path.join(__dirname, "../../web/js/gallery.js");
|
||||
const Manager = vm.runInContext(`${fs.readFileSync(file, "utf8")}\nGalleryManager;`, context, { filename: file });
|
||||
const manager = Object.create(Manager.prototype);
|
||||
Object.assign(manager, { showNotification() {}, hideSettings() {}, applySettings() {} });
|
||||
manager.saveSettings();
|
||||
const saved = JSON.parse(storage.get("gallerySettings"));
|
||||
assert.equal(saved.workspaceLayout, "masonry");
|
||||
assert.equal(saved.workspaceDensity, "compact");
|
||||
assert.equal(saved.someOtherTool, 1);
|
||||
assert.equal(saved.defaultViewMode, "list");
|
||||
assert.equal(saved.showImageInfo, false);
|
||||
assert.equal(saved.videoMute, false);
|
||||
});
|
||||
|
||||
@@ -82,6 +82,32 @@ test('a zeroed conditioning negative shows as an explicit empty negative without
|
||||
}finally{scope.dispose();f.restore();}
|
||||
});
|
||||
|
||||
test('a prompt that left out run-time trigger words says so under its text (D125)',()=>{
|
||||
const f=dom(),scope=createScope(),container=new f.Element('div'),sent=[];
|
||||
const note='Text added at run time, such as trigger words, is not included.';
|
||||
try{
|
||||
module.mountPromptBlocks(container,{parentScope:scope,sender:{open:roles=>sent.push(roles)},prompts:{positive:{status:'resolved',text:'own text',source:'runtime-parts-omitted'},negative:{status:'resolved',text:'blur',source:'graph-inferred'}}});
|
||||
const notes=f.walk(container,node=>node.textContent===note);
|
||||
assert.equal(notes.length,1,'one note, under the positive only');
|
||||
assert.equal(notes[0].parent.attributes['aria-label'],'Positive prompt');
|
||||
container.click(f.walk(container,node=>node.dataset.sendRole==='pair')[0]);
|
||||
assert.deepEqual(sent,[{positive:'own text',negative:'blur'}],'the note is never sent');
|
||||
}finally{scope.dispose();f.restore();}
|
||||
});
|
||||
|
||||
test('a prompt a later stage added conditioning to says so, and both lines show together (D126)',()=>{
|
||||
const f=dom(),scope=createScope(),container=new f.Element('div');
|
||||
const added='Conditioning added at a later stage is not included.', runtime='Text added at run time, such as trigger words, is not included.';
|
||||
try{
|
||||
const view=module.mountPromptBlocks(container,{parentScope:scope,prompts:{positive:{status:'resolved',text:'base',source:'added-conditioning'}}});
|
||||
assert.deepEqual(f.walk(container,node=>node.className==='small muted').map(node=>node.textContent),[added]);
|
||||
view.update({positive:{status:'resolved',text:'base',source:'runtime-parts-omitted-and-added-conditioning'}});
|
||||
assert.deepEqual(f.walk(container,node=>node.className==='small muted').map(node=>node.textContent),[runtime,added]);
|
||||
view.update({positive:{status:'resolved',text:'base',source:'graph-inferred'}});
|
||||
assert.equal(f.walk(container,node=>node.className==='small muted').length,0);
|
||||
}finally{scope.dispose();f.restore();}
|
||||
});
|
||||
|
||||
const unresolved=message=>({status:'unresolved',text:null,source:null,...(message===undefined?{}:{message})});
|
||||
const statusLines=(f,container)=>f.walk(container,node=>node.tagName==='p'&&node.className==='prompt-text muted').map(node=>node.textContent);
|
||||
test('unsupported node status names the class',()=>{
|
||||
|
||||
@@ -14,6 +14,11 @@
|
||||
// the carrier rule alone, as the server selects from ffprobe's tag map.
|
||||
// A {records} entry (records null = an unreadable file) answers
|
||||
// {classified: null, result: null, metadata} for the projection alone.
|
||||
// A {mode: 'resolve'} entry (PR L3) answers {evidence}: resolveLocalEvidence of its {records} texts
|
||||
// with the repository semantics table. With a {path} it reads the file locally and also answers
|
||||
// {facts} (and {unreadable: true} with the server's unreadable evidence when nothing could be read):
|
||||
// extractGenerationFacts on the local payload through projectInspectionPayload, exactly as
|
||||
// tests/js/support/inspection-parity-facts.mjs reads the server's response for vectors.json.
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { createRequire } from 'node:module';
|
||||
@@ -28,6 +33,22 @@ const web = path.join(here, '..', '..', '..', 'web', 'js', 'workspace');
|
||||
const { selectVideoCarrier } = await import(moduleUrl(path.join(web, 'local-video.mjs')));
|
||||
const { projectMetadata } = await import(moduleUrl(path.join(web, 'local-projection.mjs')));
|
||||
const { readLocalFile, toInspectionRead } = await import(moduleUrl(path.join(web, 'local-inspection.mjs')));
|
||||
const { resolveLocalEvidence, compileSemantics, unreadableEvidence } = await import(moduleUrl(path.join(web, 'local-resolver.mjs')));
|
||||
const { buildLocalPayload } = await import(moduleUrl(path.join(web, 'local-inspection.mjs')));
|
||||
const { projectInspectionPayload } = await import(moduleUrl(path.join(web, 'inspection-state.mjs')));
|
||||
const semantics = compileSemantics(JSON.parse(fs.readFileSync(path.join(here, '..', '..', '..', 'web', 'js', 'inspection-semantics.json'), 'utf8')));
|
||||
const { extractGenerationFacts } = require(path.join(here, '..', '..', '..', 'web', 'js', 'comfy-metadata.js'));
|
||||
|
||||
/** The local evidence for one set of record texts. */
|
||||
const resolved = records => ({ evidence: resolveLocalEvidence(records, semantics) });
|
||||
|
||||
/** The local wire's evidence and six facts for one read (no legacy fill, as in the vectors' own harness). */
|
||||
function localPayload(read, file, classification) {
|
||||
const payload = buildLocalPayload({ read, file, classification, semantics });
|
||||
const record = projectInspectionPayload(payload, { kind: 'local-file', key: 'local:1', path: file.name }, null, extractGenerationFacts);
|
||||
const facts = Object.fromEntries(Object.entries(record.generation).map(([field, fact]) => [field, { status: fact.status, value: fact.value }]));
|
||||
return { evidence: { prompts: payload.prompts, resolution: payload.resolution }, facts };
|
||||
}
|
||||
|
||||
const selectionRead = (selection) => selection.status === 'complete' ?
|
||||
{ records: selection.records, record_formats: selection.record_formats, carrierText: selection.carrierText, limited: selection.limited } : null;
|
||||
@@ -37,6 +58,19 @@ for await (const chunk of process.stdin) chunks.push(chunk);
|
||||
const output = {};
|
||||
for (const entry of JSON.parse(Buffer.concat(chunks).toString('utf8'))) {
|
||||
const { name } = entry;
|
||||
if (entry.mode === 'resolve') {
|
||||
if (entry.records !== undefined) { output[name] = resolved(entry.records); continue; }
|
||||
const blob = await fs.openAsBlob(entry.path);
|
||||
const { classification, result } = await readLocalFile(blob, { pngMetadata });
|
||||
const read = classification === null ? null : toInspectionRead(classification, result).read;
|
||||
if (classification === null) { output[name] = { unreadable: true }; continue; }
|
||||
// The evidence is the local wire's own (buildLocalPayload), and it equals resolveLocalEvidence of the records.
|
||||
const local = localPayload(result, { name: path.basename(entry.path), size: blob.size }, classification);
|
||||
const direct = read === null ? unreadableEvidence() : resolved(read.records).evidence;
|
||||
if (JSON.stringify(direct) !== JSON.stringify(local.evidence)) throw new Error(`${name}: the wire's roles are not the resolver's`);
|
||||
output[name] = read === null ? { unreadable: true, ...local } : local;
|
||||
continue;
|
||||
}
|
||||
if (entry.records !== undefined) {
|
||||
const read = entry.records === null ? null : {
|
||||
records: entry.records,
|
||||
|
||||
@@ -182,6 +182,24 @@ test('start returns busy with the running job', async () => {
|
||||
world.dispose();
|
||||
});
|
||||
|
||||
test('track follows a job another route started, with no read of its own', async () => {
|
||||
const { createActivityStore } = await source;
|
||||
const { world, ...deps } = harness();
|
||||
const store = createActivityStore(deps);
|
||||
const seen = [];
|
||||
store.subscribe(() => seen.push(store.find(ID_A)?.state));
|
||||
const job = store.track(summary({ kind: 'health', title: 'Health check' }));
|
||||
assert.equal(job.id, ID_A);
|
||||
assert.equal(world.gets, 0, 'the started job is known without a list read');
|
||||
assert.deepEqual(world.open().map(stream => stream.url), [`/prompt_manager/jobs/${ID_A}/events`]);
|
||||
assert.deepEqual(world.toasts, [], 'the route that started it says so itself');
|
||||
send(world.open()[0], summary({ kind: 'health', title: 'Health check', state: 'finished', progress: 100 }));
|
||||
assert.deepEqual(seen, ['running', 'finished']);
|
||||
assert.equal(world.open().length, 0, 'an ended job is no longer followed');
|
||||
assert.throws(() => store.track({ id: 'nope' }), /Invalid job/);
|
||||
world.dispose();
|
||||
});
|
||||
|
||||
test('a list that is not a job list counts as a failed read', async () => {
|
||||
const { createActivityStore } = await source;
|
||||
const { world, api, ...deps } = harness();
|
||||
|
||||
@@ -215,3 +215,151 @@ test('none and unavailable render no image chips', async () => {
|
||||
assert.deepEqual(inspectorTagChips(tagged({state, tags:[]}, {status:'none', tags:[], tags_scope:null}), {filterable:true}), []);
|
||||
}
|
||||
});
|
||||
|
||||
// LI-4 (D117): "Copy settings summary" after the six facts of the full inspector consolidates the legacy metadata
|
||||
// page's COPY ALL (web/metadata.html copyAllData). The legacy template is read from that page itself and run on the
|
||||
// legacy parse, so the summary's text is the legacy page's, line for line, wherever the two agree on the values.
|
||||
const METADATA_PAGE = fs.readFileSync(new URL('../../web/metadata.html', import.meta.url), 'utf8');
|
||||
function legacyCopyAll(params) {
|
||||
const template = METADATA_PAGE.match(/const allData = `([\s\S]*?)`;/)[1];
|
||||
return new Function('currentWorkflowData', `return \`${template}\`;`)(params);
|
||||
}
|
||||
const legacyParams = record => ComfyMetadata.extractGenerationParams({prompt:record.metadata.prompt_graph.status === 'parsed' ? record.metadata.prompt_graph.value : null,
|
||||
workflow:record.metadata.workflow_graph.status === 'parsed' ? record.metadata.workflow_graph.value : null});
|
||||
|
||||
/**
|
||||
* Each summary line against the legacy page's, allowing only the recorded differences (D117): a number keeps the
|
||||
* inspector's (the server's) spelling ("8.0" where the legacy page wrote 8); a fact the legacy parse left "Unknown" may
|
||||
* be known; and a negative the #249 ZeroOut guard zeroed reads "No negative prompt found" where the legacy parse read
|
||||
* the positive's text through the zero-out.
|
||||
*/
|
||||
function assertLegacySummary(summary, legacy, record) {
|
||||
const lines = summary.split('\n'), expected = legacy.split('\n');
|
||||
assert.deepEqual(lines.map(line => line.split(':')[0]), expected.map(line => line.split(':')[0]), 'the legacy labels, in order');
|
||||
lines.forEach((line, index) => {
|
||||
if (line === expected[index]) return;
|
||||
const [label, value] = [line.slice(0, line.indexOf(': ')), line.slice(line.indexOf(': ') + 2)];
|
||||
const old = expected[index].slice(label.length + 2);
|
||||
if (label === 'Negative Prompt' && record.prompts.negative?.source === 'zeroed-conditioning' && value === 'No negative prompt found') return;
|
||||
if (old === 'Unknown' && !/Prompt$/.test(label)) return;
|
||||
assert.ok(value.trim() !== '' && Number(value) === Number(old), `${label}: "${value}" differs from the legacy "${old}"`);
|
||||
});
|
||||
}
|
||||
|
||||
test('settings summary is the legacy COPY ALL text for a local file', async () => {
|
||||
const {settingsSummary} = await load();
|
||||
assert.equal(typeof settingsSummary, 'function', 'settingsSummary is exported');
|
||||
const record = await localRecord(localPng('fox.png'));
|
||||
assertLegacySummary(settingsSummary(record), legacyCopyAll(legacyParams(record)), record);
|
||||
assert.equal(settingsSummary(record), ['Checkpoint: synthetic-v1.safetensors', 'Positive Prompt: a red fox in snow, detailed fur',
|
||||
'Negative Prompt: blurry, lowres', 'Steps: 20', 'CFG Scale: 8.0', 'Sampler: euler', 'Seed: 7'].join('\n'));
|
||||
// Every parity PNG read in this browser: the legacy page's text but for the recorded differences, and exactly the
|
||||
// legacy text for most of them (f09 among them: the server is ambiguous, so the record shows the legacy fill).
|
||||
const exact = [];
|
||||
for (const name of fs.readdirSync(PARITY).filter(item => item.endsWith('.png')).sort()) {
|
||||
const parity = await localRecord(new File([fs.readFileSync(new URL(name, PARITY))], name));
|
||||
const summary = settingsSummary(parity), legacy = legacyCopyAll(legacyParams(parity));
|
||||
assertLegacySummary(summary, legacy, parity);
|
||||
if (summary === legacy) exact.push(name);
|
||||
}
|
||||
assert.ok(exact.includes('f09.png'));
|
||||
assert.deepEqual(fs.readdirSync(PARITY).filter(item => item.endsWith('.png') && !exact.includes(item)).sort(), ['f01b.png', 'f17.png', 'f23.png']);
|
||||
});
|
||||
|
||||
test('settings summary uses the legacy placeholders for what the record does not know', async () => {
|
||||
const {settingsSummary} = await load();
|
||||
const unknown = {...record(), prompts:{positive:{status:'unresolved'}, negative:{status:'ambiguous'}}, generation:{}};
|
||||
assert.equal(settingsSummary(unknown), legacyCopyAll({}).replace('Positive Prompt: undefined', 'Positive Prompt: No prompt found')
|
||||
.replace('Negative Prompt: undefined', 'Negative Prompt: No negative prompt found'));
|
||||
assert.equal(settingsSummary(unknown), ['Checkpoint: Unknown', 'Positive Prompt: No prompt found', 'Negative Prompt: No negative prompt found',
|
||||
'Steps: Unknown', 'CFG Scale: Unknown', 'Sampler: Unknown', 'Seed: Unknown'].join('\n'));
|
||||
// A server answer is shown as the inspector shows it: known facts verbatim, every other state the legacy "Unknown".
|
||||
const server = {...record(), prompts:{positive:{status:'resolved', text:'a\n猫'}, negative:{status:'resolved', text:''}}, generation:{
|
||||
model:{status:'known', value:'m.safetensors'}, steps:{status:'not-recorded', value:null}, cfg:{status:'known', value:'6.50'},
|
||||
sampler:{status:'ambiguous', value:null}, seed:{status:'known', value:'9007199254740993'}, scheduler:{status:'known', value:'karras'}}};
|
||||
assert.equal(settingsSummary(server), ['Checkpoint: m.safetensors', 'Positive Prompt: a\n猫', 'Negative Prompt: No negative prompt found',
|
||||
'Steps: Unknown', 'CFG Scale: 6.50', 'Sampler: Unknown', 'Seed: 9007199254740993'].join('\n'));
|
||||
});
|
||||
|
||||
/** The full inspector with a toast node and `clipboard` as `navigator.clipboard`; returns the summary button. */
|
||||
function renderSummary(record, {variant = 'full', clipboard} = {}) {
|
||||
const f = mountedDom(), scope = createScope(), fallbacks = [], toast = new f.Node('div');
|
||||
f.doc.getElementById = id => id === 'workspace-toast' ? toast : null;
|
||||
const previous = Object.getOwnPropertyDescriptor(globalThis, 'navigator');
|
||||
Object.defineProperty(globalThis, 'navigator', {configurable:true, value:clipboard === undefined ? {} : {clipboard}});
|
||||
const view = mountFileInspector(f.container, {sender:{open() {}}, parentScope:scope, variant, copyFallback:(text, options) => fallbacks.push({text, ...options})});
|
||||
view.setState({loading:false, error:false, record});
|
||||
const button = f.walk(f.container, n => n.tagName === 'button' && n.textContent === 'Copy settings summary')[0] ?? null;
|
||||
return {f, button, toast, fallbacks, scope, done() {
|
||||
view.dispose(); scope.dispose(); f.restore();
|
||||
if (previous) Object.defineProperty(globalThis, 'navigator', previous); else delete globalThis.navigator;
|
||||
}};
|
||||
}
|
||||
|
||||
test('Copy settings summary follows the facts in the full inspector only', async () => {
|
||||
const full = renderSummary(record());
|
||||
try {
|
||||
assert.ok(full.button, 'the full inspector offers Copy settings summary');
|
||||
const inner = full.button.parent, index = inner.children.indexOf(full.button);
|
||||
assert.equal(inner.children[index - 1].tagName, 'dl', 'it follows the six facts');
|
||||
assert.equal(inner.children[index + 1].tagName, 'details', 'File & provenance follows it');
|
||||
// M7: the D117 wrap rule targets this button by its class, never any button of a details block.
|
||||
assert.equal(full.button.className, 'settings-summary');
|
||||
} finally { full.done(); }
|
||||
const panel = renderSummary(record(), {variant:'panel'});
|
||||
try { assert.equal(panel.button, null, 'the Images side panel keeps the concept footer'); } finally { panel.done(); }
|
||||
});
|
||||
|
||||
test('Copy settings summary copies the summary and shows the shared toast', async () => {
|
||||
const {settingsSummary} = await load(), copied = [];
|
||||
const view = renderSummary(record(), {clipboard:{writeText:async text => {copied.push(text);}}});
|
||||
try {
|
||||
await view.button.listeners.get('click')[0].callback({target:view.button});
|
||||
assert.deepEqual(copied, [settingsSummary(record())]);
|
||||
assert.equal(view.toast.textContent, 'Settings summary copied.');
|
||||
assert.equal(view.fallbacks.length, 0);
|
||||
} finally { view.done(); }
|
||||
});
|
||||
|
||||
test('Copy settings summary without a clipboard opens the select-to-copy fallback', async () => {
|
||||
const {settingsSummary} = await load();
|
||||
for (const clipboard of [undefined, {writeText:async () => {throw new Error('Denied');}}]) {
|
||||
const view = renderSummary(record(), {clipboard});
|
||||
try {
|
||||
await view.button.listeners.get('click')[0].callback({target:view.button});
|
||||
assert.equal(view.fallbacks.length, 1);
|
||||
assert.equal(view.fallbacks[0].text, settingsSummary(record()));
|
||||
assert.equal(view.fallbacks[0].title, 'Copy settings summary');
|
||||
assert.equal(view.fallbacks[0].trigger, view.button);
|
||||
assert.notEqual(view.toast.textContent, 'Settings summary copied.');
|
||||
} finally { view.done(); }
|
||||
}
|
||||
});
|
||||
|
||||
test('the inspector is busy while a read is loading, then not', async () => {
|
||||
const f = mountedDom(), scope = createScope();
|
||||
const view = mountFileInspector(f.container, {sender:{open() {}}, parentScope:scope, variant:'full'});
|
||||
try {
|
||||
view.setState({loading:true, error:false, record:null});
|
||||
assert.equal(f.container.getAttribute('aria-busy'), 'true');
|
||||
assert.ok(f.walk(f.container, n => n.textContent === 'Loading file details…').length);
|
||||
view.setState({loading:false, error:false, record:record()});
|
||||
assert.equal(f.container.getAttribute('aria-busy'), null);
|
||||
view.setState({loading:true, error:false, record:null});
|
||||
view.setState({loading:false, error:true, record:null});
|
||||
assert.equal(f.container.getAttribute('aria-busy'), null);
|
||||
view.setState({loading:true, error:false, record:null});
|
||||
} finally {
|
||||
view.dispose();
|
||||
assert.equal(f.container.getAttribute('aria-busy'), null, 'dispose clears the busy state');
|
||||
scope.dispose(); f.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('the D117 wrap rule targets only the settings summary button', async () => {
|
||||
const fs = await import('node:fs');
|
||||
const css = fs.readFileSync(new URL('../../web/js/workspace/workspace.css', import.meta.url), 'utf8');
|
||||
const rules = css.split('\n').filter(line => line.includes('D117'));
|
||||
assert.deepEqual(rules.map(line => line.trim()), ['.ca-tools #viewer-details .settings-summary { white-space: normal; /* D117 */ }']);
|
||||
assert.doesNotMatch(css, /#viewer-details > div > button/);
|
||||
});
|
||||
|
||||
@@ -478,3 +478,19 @@ test('a refresh deferred behind a page load is dropped when the stream is dispos
|
||||
await new Promise(resolve=>setImmediate(resolve));
|
||||
assert.equal(requests.length,2,'no refresh runs after dispose');
|
||||
});
|
||||
|
||||
test('defaults apply to a new visit but not to the current view or a saved view', async () => {
|
||||
const { initialImagesView } = await source;
|
||||
assert.equal(typeof initialImagesView, 'function', 'initialImagesView is exported');
|
||||
const preferences = { workspaceLayout: 'masonry', workspaceDensity: 'compact' };
|
||||
// A new visit starts from System › Appearance.
|
||||
assert.deepEqual(initialImagesView({ preferences }), { layout: 'masonry', density: 'compact' });
|
||||
// The current view (the last visit) keeps its own toolbar choices.
|
||||
assert.deepEqual(initialImagesView({ prior: { layout: 'list', density: 'large' }, preferences }), { layout: 'list', density: 'large' });
|
||||
assert.deepEqual(initialImagesView({ prior: {}, preferences }), { layout: 'grid', density: 'comfortable' });
|
||||
// A saved view opens as it always did.
|
||||
assert.deepEqual(initialImagesView({ saved: { q: '' }, preferences }), { layout: 'grid', density: 'comfortable' });
|
||||
// Without readable preferences nothing changes.
|
||||
assert.deepEqual(initialImagesView({}), { layout: 'grid', density: 'comfortable' });
|
||||
assert.deepEqual(initialImagesView({ preferences: {} }), { layout: 'grid', density: 'comfortable' });
|
||||
});
|
||||
|
||||
@@ -825,3 +825,73 @@ test('refresh re-reads the selected file and keeps the shown record until the ne
|
||||
assert.equal(controller.state.record.identity.path, 'folder/b.png');
|
||||
controller.dispose(); assert.equal(await controller.refresh(), undefined);
|
||||
});
|
||||
|
||||
test('metadata off shows an explicit load and loads once on request', async () => {
|
||||
const { api, requests } = pendingApi(), updates = [];
|
||||
const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, autoLoad: false });
|
||||
controller.subscribe(state => updates.push(state));
|
||||
// Selecting reads nothing: the state says the metadata waits for a request.
|
||||
assert.equal(controller.select(image()), undefined);
|
||||
assert.equal(requests.length, 0);
|
||||
assert.equal(controller.state.deferred, true);
|
||||
assert.equal(controller.state.loading, false);
|
||||
assert.equal(controller.state.record, null);
|
||||
// Selecting the same file again still reads nothing.
|
||||
controller.select({ ...image() });
|
||||
assert.equal(requests.length, 0);
|
||||
const load = controller.load();
|
||||
assert.equal(requests.length, 1);
|
||||
assert.equal(controller.state.loading, true);
|
||||
assert.equal(controller.state.deferred, false);
|
||||
// A second press while it reads, and the same file selected again once it loaded, read nothing more.
|
||||
controller.load();
|
||||
assert.equal(requests.length, 1);
|
||||
requests[0].resolve(envelope()); await load;
|
||||
assert.equal(controller.state.record.generation.seed.value, '9007199254740993');
|
||||
controller.load(); controller.select(image());
|
||||
assert.equal(requests.length, 1);
|
||||
// Another file waits for its own request again.
|
||||
controller.select({ ...image(), path: 'folder/b.png', filename: 'b.png', url: image().url.replace('a%20%23%25', 'b') });
|
||||
assert.equal(controller.state.deferred, true);
|
||||
assert.equal(requests.length, 1);
|
||||
// The default controller loads on selection, as before.
|
||||
const eager = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts });
|
||||
eager.select(image());
|
||||
assert.equal(requests.length, 2);
|
||||
assert.equal(eager.state.deferred, false);
|
||||
eager.dispose(); controller.dispose();
|
||||
assert.equal(controller.load(), undefined);
|
||||
});
|
||||
|
||||
test('cache off rereads the current selection; on reuses the one current record', async () => {
|
||||
for (const cache of [true, false]) {
|
||||
const { api, requests } = pendingApi();
|
||||
const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, cache });
|
||||
const first = controller.select(image());
|
||||
// A read in flight is shared either way.
|
||||
controller.select(image());
|
||||
assert.equal(requests.length, 1);
|
||||
requests[0].resolve(envelope()); await first;
|
||||
const again = controller.select(image());
|
||||
assert.equal(requests.length, cache ? 1 : 2, `cache ${cache}`);
|
||||
if (!cache) { requests[1].resolve(envelope()); await again; assert.equal(controller.state.record.identity.path, 'folder/a #%.png'); }
|
||||
controller.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
test('cache off never rereads a selection whose source changed, and refresh never loads a deferred one', async () => {
|
||||
const { api, requests } = pendingApi();
|
||||
const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, cache: false });
|
||||
const first = controller.select(image());
|
||||
requests[0].reject(new ApiError(409, refreshMessage, 'source-changed')); await first;
|
||||
assert.equal(controller.state.refreshRequired, true);
|
||||
controller.select(image());
|
||||
assert.equal(requests.length, 1);
|
||||
controller.dispose();
|
||||
const deferred = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, autoLoad: false });
|
||||
deferred.select(image());
|
||||
assert.equal(deferred.refresh(), undefined);
|
||||
assert.equal(requests.length, 1);
|
||||
assert.equal(deferred.state.deferred, true);
|
||||
deferred.dispose();
|
||||
});
|
||||
|
||||
@@ -20,6 +20,16 @@ const { extractGenerationFacts } = ComfyMetadata;
|
||||
// The page loads both as scripts; the legacy fill and the PNG reader find them on the global.
|
||||
globalThis.PngMetadata = PngMetadata;
|
||||
globalThis.ComfyMetadata = ComfyMetadata;
|
||||
// The page's API client and the node table it serves (local-semantics.mjs fetches it once).
|
||||
const SEMANTICS = fs.readFileSync(path.join(here, "..", "..", "web", "js", "inspection-semantics.json"), "utf8");
|
||||
globalThis.window = { ApiClient: { resolveUrl: (value) => value } };
|
||||
// A test can hold the table request (`tableGate`) to stand for a slow or stalled proxy.
|
||||
let tableGate = null;
|
||||
globalThis.fetch = async (url) => {
|
||||
if (url !== "/prompt_manager/js/inspection-semantics.json") throw new Error(`unexpected request ${url}`);
|
||||
if (tableGate) await tableGate;
|
||||
return new Response(SEMANTICS, { status: 200 });
|
||||
};
|
||||
const m = await import("../../web/js/workspace/local-inspection.mjs").catch((error) => {
|
||||
if (error.code !== "ERR_MODULE_NOT_FOUND") throw error;
|
||||
return {};
|
||||
@@ -59,6 +69,37 @@ test("contract", () => {
|
||||
assert.ok(Object.isFrozen(m.localFileAdapter));
|
||||
});
|
||||
|
||||
// M3: the first read waits for the page's one table request; an aborted, replaced or cleared selection must not.
|
||||
test("a read waiting for the node table ends with its selection", async () => {
|
||||
let release;
|
||||
tableGate = new Promise((resolve) => { release = resolve; });
|
||||
try {
|
||||
const file = pngFile("slow.png", [tEXt("prompt", PROMPT)]);
|
||||
const aborted = new AbortController();
|
||||
const reading = m.localFileAdapter.read(await handleFor(file), { signal: aborted.signal });
|
||||
const settled = (promise) => Promise.race([promise.then(() => "resolved", (error) => error.name), new Promise((resolve) => setTimeout(() => resolve("pending"), 200))]);
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
aborted.abort();
|
||||
assert.equal(await settled(reading), "AbortError");
|
||||
// Through the controller: clearing the selection ends its read; nothing is published for it.
|
||||
const controller = createInspectionController({ api: null, adapter: m.localFileAdapter, extractFacts: extractGenerationFacts });
|
||||
const states = [];
|
||||
controller.subscribe((state) => states.push(state));
|
||||
const selected = controller.select(await handleFor(file));
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
controller.clear();
|
||||
assert.equal(await settled(selected), "resolved");
|
||||
assert.ok(states.every((state) => state.record === null && state.error === false), "no record or error for a cleared selection");
|
||||
controller.dispose();
|
||||
} finally {
|
||||
tableGate = null;
|
||||
release();
|
||||
}
|
||||
// The shared request still lands for the next selection.
|
||||
const { wire } = await recordFor(pngFile("next.png", [tEXt("prompt", PROMPT)]));
|
||||
assert.equal(wire.resolution.state, "resolved");
|
||||
});
|
||||
|
||||
test("a PNG File becomes the server-shaped wire with local provenance", async () => {
|
||||
const file = pngFile("Fox Render.PNG", [tEXt("prompt", PROMPT), tEXt("workflow", WORKFLOW)]);
|
||||
const { wire, record, handle } = await recordFor(file);
|
||||
@@ -66,8 +107,12 @@ test("a PNG File becomes the server-shaped wire with local provenance", async ()
|
||||
size: file.size, width: 640, height: 480, modified_time: 1_700_000_000.123 });
|
||||
assert.equal(wire.success, true);
|
||||
assert.equal(wire.inspection_version, 1);
|
||||
assert.deepEqual(wire.prompts, { positive: { status: "unresolved" }, negative: { status: "unresolved" } });
|
||||
assert.deepEqual(wire.resolution, { state: "unresolved", sampler_id: null, reason: "Resolved in this browser from the workflow.", reason_code: null, node: null });
|
||||
// The roles the server's inspector resolves for the same records (local-resolver.mjs).
|
||||
assert.deepEqual(wire.prompts, {
|
||||
positive: { status: "resolved", text: "a red fox in snow, detailed fur", source: "graph-inferred" },
|
||||
negative: { status: "resolved", text: "blurry, lowres", source: "graph-inferred" },
|
||||
});
|
||||
assert.deepEqual(wire.resolution, { state: "resolved", sampler_id: "3", reason: "", reason_code: null, node: null });
|
||||
assert.deepEqual(wire.library, { status: "unavailable", scope: "saved-library", tags: [], category_scope: null, tags_scope: null });
|
||||
assert.equal(wire.metadata.status, "available");
|
||||
assert.deepEqual(wire.metadata.raw, { prompt: PROMPT, workflow: WORKFLOW });
|
||||
@@ -96,6 +141,33 @@ test("a PNG File becomes the server-shaped wire with local provenance", async ()
|
||||
assert.equal(exported.file.source_label, "Selected local file");
|
||||
});
|
||||
|
||||
test("roles answer as the server's inspector: unreadable, unavailable table, ambiguous", async () => {
|
||||
const png = { media: "image", kind: "png" };
|
||||
const unreadable = m.buildLocalPayload({ read: null, file: { name: "a.png" }, classification: png });
|
||||
assert.deepEqual(unreadable.resolution, { state: "unresolved", sampler_id: null, reason: "Media metadata could not be read", reason_code: null, node: null });
|
||||
assert.equal(unreadable.prompts.positive.message, "Metadata in this file could not be read.");
|
||||
const records = { prompt: PROMPT };
|
||||
const read = { status: "complete", records, limited: [], width: 2, height: 3 };
|
||||
// Without a table (it could not be fetched) the roles stay open, with the server's own line.
|
||||
const missing = m.buildLocalPayload({ read, file: { name: "a.png" }, classification: png });
|
||||
assert.equal(missing.resolution.reason, "Inspection node semantics are unavailable");
|
||||
assert.equal(missing.prompts.positive.message, "The inspector's node table is unavailable.");
|
||||
const semantics = await (await import("../../web/js/workspace/local-semantics.mjs")).loadInspectionSemantics();
|
||||
const graph = JSON.parse(PROMPT);
|
||||
graph["9"] = { ...graph["3"], inputs: { ...graph["3"].inputs, positive: ["8", 0] } };
|
||||
graph["8"] = { class_type: "CLIPTextEncode", inputs: { text: "a cat", clip: ["4", 1] } };
|
||||
const several = m.buildLocalPayload({ read: { ...read, records: { prompt: JSON.stringify(graph) } }, file: { name: "a.png" }, classification: png, semantics });
|
||||
assert.equal(several.resolution.reason, "Multiple sampler branches");
|
||||
assert.deepEqual(several.prompts.positive, { status: "ambiguous", text: null, source: null });
|
||||
// The legacy fill then applies exactly as it does to the server's answer for the same bytes in Images.
|
||||
const handle = await handleFor(pngFile("grid.png", [tEXt("prompt", JSON.stringify(graph))]));
|
||||
const local = m.localFileAdapter.toRecord(several, handle, extractGenerationFacts);
|
||||
const asServed = { ...several, file: { ...several.file, observation: "current-file", source_label: "output" } };
|
||||
const served = projectInspectionPayload(asServed, { kind: "rooted-file", key: "0:grid.png", root_index: 0, path: "grid.png", root_scope: null }, null, extractGenerationFacts);
|
||||
assert.deepEqual(local.prompts, served.prompts);
|
||||
assert.deepEqual(local.generation, served.generation);
|
||||
});
|
||||
|
||||
test("file facts follow the server: extension like Path.suffix, null when unknown", async () => {
|
||||
const cases = [["a.b.PnG", ".png"], [".hidden", ""], ["noext", ""], ["dot.", ""], ["x.tar.gz", ".gz"]];
|
||||
for (const [name, extension] of cases) {
|
||||
@@ -307,3 +379,89 @@ test("the parity corpus PNGs read into renderable records", async () => {
|
||||
assert.ok(["parsed", "limited", "malformed"].includes(wire.metadata.prompt_graph.status));
|
||||
}
|
||||
});
|
||||
|
||||
// M4: resolution runs on the page's main thread, so a read yields between its bounded steps (the busy state paints
|
||||
// and input is handled) and no step may exceed a budget on the largest record the readers keep (2 MiB of roles).
|
||||
const ROLE_BYTES = 2 * 1024 * 1024;
|
||||
// Generous for a shared CI runner: a desktop resolves these in 130-455 ms.
|
||||
const STEP_BUDGET_MS = 2500;
|
||||
const ENCODE = (text) => ({ class_type: "CLIPTextEncode", inputs: { text, clip: ["4", 1] } });
|
||||
const KSAMPLER = (positive, negative, latent) => ({ class_type: "KSampler", inputs: { seed: 1, steps: 20, cfg: 8.0, sampler_name: "euler", scheduler: "normal",
|
||||
denoise: 1.0, model: ["4", 0], positive, negative, latent_image: latent } });
|
||||
const LOADERS = { "4": { class_type: "CheckpointLoaderSimple", inputs: { ckpt_name: "m.safetensors" } }, "5": { class_type: "EmptyLatentImage", inputs: { width: 64, height: 64, batch_size: 1 } } };
|
||||
/** The largest graph of this shape whose JSON fits the role budget. */
|
||||
function maximal(build, high) {
|
||||
let low = 1;
|
||||
while (low < high) {
|
||||
const middle = (low + high + 1) >> 1;
|
||||
if (Buffer.byteLength(JSON.stringify(build(middle)), "utf8") <= ROLE_BYTES) low = middle;
|
||||
else high = middle - 1;
|
||||
}
|
||||
return JSON.stringify(build(low));
|
||||
}
|
||||
const MAXIMAL_GRAPHS = {
|
||||
// Many candidates: every sampler is a final stage with its own pair.
|
||||
independent: maximal((count) => {
|
||||
const graph = { ...LOADERS };
|
||||
for (let index = 0; index < count; index++) {
|
||||
graph[`s${index}`] = KSAMPLER([`p${index}`, 0], [`n${index}`, 0], ["5", 0]);
|
||||
graph[`p${index}`] = ENCODE(`fox ${index}`);
|
||||
graph[`n${index}`] = ENCODE(`blur ${index}`);
|
||||
}
|
||||
return graph;
|
||||
}, 6666),
|
||||
// One long stage chain through the latent: the deepest stage walk.
|
||||
chain: maximal((count) => {
|
||||
const graph = { ...LOADERS, "6": ENCODE("fox"), "7": ENCODE("blur") };
|
||||
for (let index = 0; index < count; index++) graph[`s${index}`] = KSAMPLER(["6", 0], ["7", 0], index ? [`s${index - 1}`, 0] : ["5", 0]);
|
||||
return graph;
|
||||
}, 19996),
|
||||
};
|
||||
|
||||
test("resolution of a maximal 2 MiB graph stays within the step budget", async () => {
|
||||
const { resolveLocalEvidence } = await import("../../web/js/workspace/local-resolver.mjs");
|
||||
const semantics = await (await import("../../web/js/workspace/local-semantics.mjs")).loadInspectionSemantics();
|
||||
for (const [name, prompt] of Object.entries(MAXIMAL_GRAPHS)) {
|
||||
assert.ok(Buffer.byteLength(prompt, "utf8") > ROLE_BYTES - 1024, `${name} fills the role budget`);
|
||||
const started = performance.now();
|
||||
const evidence = resolveLocalEvidence({ prompt }, semantics);
|
||||
const elapsed = performance.now() - started;
|
||||
assert.ok(elapsed < STEP_BUDGET_MS, `${name}: ${Math.round(elapsed)} ms`);
|
||||
assert.notEqual(evidence.resolution.reason_code, "malformed", name);
|
||||
}
|
||||
});
|
||||
|
||||
test("a read yields between bounded steps and answers byte for byte as one synchronous build", async () => {
|
||||
const semantics = await (await import("../../web/js/workspace/local-semantics.mjs")).loadInspectionSemantics();
|
||||
const file = pngFile("maximal.png", [tEXt("prompt", MAXIMAL_GRAPHS.independent)]);
|
||||
const handle = await handleFor(file);
|
||||
const marks = [];
|
||||
const yieldControl = async (signal) => {
|
||||
signal?.throwIfAborted();
|
||||
marks.push(performance.now());
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
};
|
||||
const started = performance.now();
|
||||
const wire = await m.localFileAdapter.read(handle, { signal: new AbortController().signal, yieldControl });
|
||||
marks.push(performance.now());
|
||||
// Resolution and the projection each run in a turn of their own after the table is in.
|
||||
assert.ok(marks.length >= 3, `yields: ${marks.length - 1}`);
|
||||
const gaps = marks.map((mark, index) => mark - (index ? marks[index - 1] : started));
|
||||
assert.ok(Math.max(...gaps) < STEP_BUDGET_MS, `longest step ${Math.round(Math.max(...gaps))} ms`);
|
||||
const { result } = await m.readLocalFile(file, { classification: handle.classification });
|
||||
const direct = await m.fitLocalPayload(m.buildLocalPayload({ read: result, file: { name: handle.filename, size: file.size, lastModified: file.lastModified },
|
||||
classification: handle.classification, semantics }));
|
||||
assert.equal(JSON.stringify(wire), JSON.stringify(direct));
|
||||
assert.equal(wire.resolution.state, "ambiguous");
|
||||
// An abort at any yield ends the read there.
|
||||
for (let stop = 0; stop < marks.length - 1; stop++) {
|
||||
const controller = new AbortController();
|
||||
let seen = 0;
|
||||
const stopping = async (signal) => {
|
||||
if (seen++ === stop) controller.abort();
|
||||
signal.throwIfAborted();
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
};
|
||||
await assert.rejects(m.localFileAdapter.read(handle, { signal: controller.signal, yieldControl: stopping }), { name: "AbortError" }, `yield ${stop}`);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
// PR L3 Tasks 8-9: the server's inspection resolver in this browser. The values below are what CPython answers
|
||||
// (json.loads, len, str.strip, repr, utils/prompt_inspector.inspect_prompt_evidence); the full comparison with the
|
||||
// server runs in tests/test_local_inspect_parity.py through tests/js/support/local-inspect-oracle.mjs.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
|
||||
const py = await import('../../web/js/workspace/py-json.mjs');
|
||||
const resolver = await import('../../web/js/workspace/local-resolver.mjs');
|
||||
const semantics = resolver.compileSemantics(JSON.parse(fs.readFileSync(new URL('../../web/js/inspection-semantics.json', import.meta.url), 'utf8')));
|
||||
|
||||
test('json.loads keeps Python value kinds and dict order', () => {
|
||||
const value = py.pyJsonLoads('{"9": 1, "10": 1.0, "a": NaN, "9": [true, -0, 9007199254740993, -Infinity]}');
|
||||
assert.ok(value instanceof Map);
|
||||
// A repeated key keeps its first position and takes its last value; integer-like keys keep text order.
|
||||
assert.deepEqual([...value.keys()], ['9', '10', 'a']);
|
||||
const [flag, zero, big, negative] = value.get('9');
|
||||
assert.equal(flag, true); assert.equal(zero, 0); assert.equal(big, 9007199254740993n);
|
||||
assert.ok(negative instanceof py.PyFloat); assert.equal(negative.value, -Infinity);
|
||||
assert.ok(value.get('10') instanceof py.PyFloat, '1.0 is a float, not an int');
|
||||
assert.ok(Number.isNaN(value.get('a').value));
|
||||
assert.equal(py.isInt(value.get('10')), false);
|
||||
assert.equal(py.pyJsonLoads('"\\ud83d\\ude00\\ud800"'), '\u{1f600}\ud800');
|
||||
for (const bad of ['{}', '{"a":1,}', '[01]', '{"a" 1}', '"\u0001"', '"\\x"', '1 2', '-', `${'9'.repeat(4301)}`, '"\\ud800\\uzzzz"']) {
|
||||
assert.throws(() => py.pyJsonLoads(bad), py.PyValueError, bad);
|
||||
}
|
||||
assert.equal(py.pyJsonLoads(` ${'9'.repeat(4300)}\n`), BigInt('9'.repeat(4300)));
|
||||
});
|
||||
|
||||
// One depth cap for both browser CPython-JSON parsers (py-json.mjs here, local-video.mjs for video envelopes): every
|
||||
// array or object counts, an empty one too, as CPython's scanner enters recursion for each; past the cap it refuses.
|
||||
test('json.loads refuses nesting past the shared depth cap, counting every container', () => {
|
||||
const video = import('../../web/js/workspace/local-video.mjs');
|
||||
assert.equal(py.JSON_MAX_DEPTH, 1000);
|
||||
const nest = (depth, inner = '') => '['.repeat(depth) + inner + ']'.repeat(depth);
|
||||
const objects = (depth, inner) => '{"a":'.repeat(depth) + inner + '}'.repeat(depth);
|
||||
for (const text of [nest(1000), nest(1000, '1'), nest(999, '{}'), objects(1000, '1'), objects(999, '[]')]) assert.doesNotThrow(() => py.pyJsonLoads(text));
|
||||
for (const text of [nest(1001), nest(1001, '1'), nest(1000, '{}'), objects(1001, '1'), objects(1000, '[]')]) {
|
||||
assert.throws(() => py.pyJsonLoads(text), py.PyValueError);
|
||||
}
|
||||
return video.then(({ pythonJsonLoads, PythonJsonError }) => {
|
||||
for (const text of [nest(1000), nest(1000, '1'), objects(999, '[]')]) assert.doesNotThrow(() => pythonJsonLoads(text));
|
||||
for (const text of [nest(1001), nest(1001, '1'), objects(1000, '[]')]) {
|
||||
assert.throws(() => pythonJsonLoads(text), error => error instanceof PythonJsonError && error.recursion === true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
test('str helpers measure, slice and strip as CPython', () => {
|
||||
assert.equal(py.pyLen('a\u{1f600}\ud800'), 3);
|
||||
assert.equal(py.pySlice('\u{1f600}\u{1f600}x', 2), '\u{1f600}\u{1f600}');
|
||||
// str.isspace() includes \x1c-\x1f and \x85 and excludes , unlike String.prototype.trim.
|
||||
assert.equal(py.pyStrip('\x1c\x85 a b \x1f'), 'a b');
|
||||
assert.equal(py.pyStrip('a'), 'a');
|
||||
assert.deepEqual([1.0, 0.0001, 1e-5, 1e16, 123456789012345.6, -2.5, 1e22].map(py.pyFloatRepr),
|
||||
['1.0', '0.0001', '1e-05', '1e+16', '123456789012345.6', '-2.5', '1e+22']);
|
||||
assert.deepEqual([null, true, 0, 1n, new py.PyFloat(2), '', [], new Map()].map(py.pyTruthy), [false, true, false, true, true, false, false, false]);
|
||||
assert.deepEqual([1, 1n, true, new py.PyFloat(1), '1', 2].map(py.pyEqualsOne), [true, true, true, true, false, false]);
|
||||
});
|
||||
|
||||
const encode = text => ({class_type: 'CLIPTextEncode', inputs: {text, clip: ['4', 1]}});
|
||||
const sampler = (extra = {}) => ({class_type: 'KSampler', inputs: {seed: 1, positive: ['6', 0], negative: ['7', 0], latent_image: ['5', 0], model: ['4', 0], ...extra}});
|
||||
const base = (nodes = {}) => ({'3': sampler(), '4': {class_type: 'CheckpointLoaderSimple', inputs: {ckpt_name: 'm.safetensors'}},
|
||||
'5': {class_type: 'EmptyLatentImage', inputs: {width: 64, height: 64, batch_size: 1}}, '6': encode('a fox'), '7': encode('blurry'), ...nodes});
|
||||
|
||||
test('a single chain resolves both roles with their sources', () => {
|
||||
const evidence = resolver.resolveLocalEvidence({prompt: JSON.stringify(base())}, semantics);
|
||||
assert.deepEqual(evidence, {
|
||||
prompts: {positive: {status: 'resolved', text: 'a fox', source: 'graph-inferred'}, negative: {status: 'resolved', text: 'blurry', source: 'graph-inferred'}},
|
||||
resolution: {state: 'resolved', sampler_id: '3', reason: '', reason_code: null, node: null},
|
||||
});
|
||||
});
|
||||
|
||||
test('records without roles, malformed text and a missing sampler answer as the server', () => {
|
||||
assert.deepEqual(resolver.resolveLocalEvidence({workflow: '{}'}, semantics).resolution,
|
||||
{state: 'absent', sampler_id: null, reason: 'No parsed prompt metadata', reason_code: null, node: null});
|
||||
const malformed = resolver.resolveLocalEvidence({prompt: '{broken'}, semantics);
|
||||
assert.equal(malformed.resolution.reason, 'Prompt evidence is malformed');
|
||||
assert.equal(malformed.prompts.positive.message, 'Prompt metadata in this file is malformed.');
|
||||
const none = resolver.resolveLocalEvidence({prompt: JSON.stringify({'1': encode('x'), '2': {class_type: 'MySampler', inputs: {positive: ['1', 0]}}})}, semantics);
|
||||
assert.deepEqual(none.resolution, {state: 'unresolved', sampler_id: null, reason: 'No supported sampler pair found (saw MySampler)', reason_code: 'no-sampler', node: null});
|
||||
assert.equal(none.prompts.negative.message, 'No supported sampler was found.');
|
||||
assert.equal(resolver.resolveLocalEvidence({prompt: JSON.stringify(base()), parameters: 'x'}, null).resolution.reason, 'Inspection node semantics are unavailable');
|
||||
});
|
||||
|
||||
test('a float output slot is not a link, and a runtime node is named', () => {
|
||||
const floatSlot = resolver.resolveLocalEvidence({prompt: JSON.stringify(base()).replace('"positive":["6",0]', '"positive":["6",0.0]')}, semantics);
|
||||
assert.deepEqual(floatSlot.resolution, {state: 'unresolved', sampler_id: '3', reason: 'Expected a typed graph link or text input', reason_code: 'malformed', node: null});
|
||||
const runtime = resolver.resolveLocalEvidence({prompt: JSON.stringify(base({'6': {class_type: 'PromptSearchList', inputs: {}}}))}, semantics);
|
||||
assert.deepEqual(runtime.resolution.node, {id: '6', class_type: 'PromptSearchList'});
|
||||
assert.equal(runtime.prompts.positive.message, 'Prompt is chosen at run time (PromptSearchList).');
|
||||
});
|
||||
|
||||
test('execution evidence replaces a PromptManagerPair text when it matches the node inputs', () => {
|
||||
const pair = {class_type: 'PromptManagerPair', inputs: {text: 'fox', negative_text: 'blur', prepend_text: '', append_text: ''}};
|
||||
const record = {text: 'fox', negative_text: 'blur', positive_text: 'fox, executed', negative_effective_text: 'blur', prepend_text: '', append_text: ''};
|
||||
const graph = base({'3': sampler({positive: ['8', 0], negative: ['8', 1]}), '8': pair});
|
||||
const evidence = resolver.resolveLocalEvidence({prompt: JSON.stringify(graph), comfyassets_prompt_pairs: JSON.stringify({version: 1, nodes: {'8': {records: [record]}}})}, semantics);
|
||||
assert.deepEqual(evidence.prompts.positive, {status: 'resolved', text: 'fox, executed', source: 'execution'});
|
||||
const stale = resolver.resolveLocalEvidence({prompt: JSON.stringify(graph), comfyassets_prompt_pairs: JSON.stringify({version: 1, nodes: {'8': {records: [{...record, text: 'other'}]}}})}, semantics);
|
||||
assert.equal(stale.resolution.reason, 'Execution evidence does not match recorded node inputs');
|
||||
});
|
||||
|
||||
test('identical sampler branches resolve to their shared pair, differing ones stay ambiguous (#270)', () => {
|
||||
const shared = resolver.resolveLocalEvidence({prompt: JSON.stringify(base({'9': sampler()}))}, semantics);
|
||||
assert.deepEqual(shared.resolution, {state: 'resolved', sampler_id: null, reason: 'Every sampler branch uses the same prompt pair', reason_code: null, node: null});
|
||||
assert.equal(shared.prompts.positive.text, 'a fox');
|
||||
const differ = resolver.resolveLocalEvidence({prompt: JSON.stringify(base({'9': sampler({positive: ['8', 0]}), '8': encode('a cat')}))}, semantics);
|
||||
assert.deepEqual(differ.resolution, {state: 'ambiguous', sampler_id: null, reason: 'Multiple sampler branches', reason_code: null, node: null});
|
||||
assert.equal(differ.prompts.positive.status, 'ambiguous');
|
||||
});
|
||||
|
||||
test('a detailer stage is compared with the sampler in its image ancestry', () => {
|
||||
const graph = base({'10': {class_type: 'VAEDecode', inputs: {samples: ['3', 0]}}, '8': encode('a cat'), '12': {class_type: 'SaveImage', inputs: {images: ['11', 0]}},
|
||||
'11': {class_type: 'FaceDetailer', inputs: {image: ['10', 0], model: ['4', 0], positive: ['8', 0], negative: ['7', 0]}}});
|
||||
const evidence = resolver.resolveLocalEvidence({prompt: JSON.stringify(graph)}, semantics);
|
||||
assert.deepEqual(evidence.resolution, {state: 'ambiguous', sampler_id: '11', reason: 'Sampler stages have different prompts', reason_code: 'multistage-differs', node: null});
|
||||
assert.equal(evidence.prompts.positive.message, undefined);
|
||||
});
|
||||
|
||||
test('A1111 parameters split on the unique settings footer as Python does', () => {
|
||||
const read = text => resolver.resolveLocalEvidence({parameters: text}, semantics);
|
||||
assert.deepEqual(read('a fox\nNegative prompt: blur\nSteps: 20, Sampler: euler').prompts, {
|
||||
positive: {status: 'resolved', text: 'a fox', source: 'embedded-parameters'}, negative: {status: 'resolved', text: 'blur', source: 'embedded-parameters'},
|
||||
});
|
||||
// Python's multiline ^ and $ break only at \n: a U+2028 or a lone \r never starts the footer line.
|
||||
for (const text of ['a fox\u2028Steps: 20, x', 'a fox\rSteps: 20, x']) assert.equal(read(text).resolution.reason, 'A1111 settings footer is not unique');
|
||||
// \d is any Unicode decimal digit and \s spans line breaks, as in Python's re.
|
||||
assert.equal(read('a fox\nSteps: \u0662\u0660, x').resolution.reason, 'Negative prompt delimiter is absent');
|
||||
assert.equal(read('a fox\nSteps:\n 20 \n, x').resolution.state, 'absent');
|
||||
assert.equal(read('a fox\nSteps: 1, x\nNegative Template: n\nTemplate: t').resolution.reason_code, 'malformed');
|
||||
assert.equal(read('\u{1f600}'.repeat(60000) + '\nSteps: 1, x').resolution.state, 'absent', 'the length limit counts code points');
|
||||
});
|
||||
|
||||
test('an unusable semantics table is refused', () => {
|
||||
for (const doc of [null, {schema: 2}, {schema: 1, samplers: []}]) assert.throws(() => resolver.compileSemantics(doc), resolver.SemanticsError);
|
||||
});
|
||||
|
||||
test('the lineage section is compiled, and a bad one refuses the table', () => {
|
||||
const doc = JSON.parse(fs.readFileSync(new URL('../../web/js/inspection-semantics.json', import.meta.url), 'utf8'));
|
||||
assert.deepEqual(semantics.lineage.get('Image Filter\u00001'), { strategy: 'continue', input: 'latents' });
|
||||
// Lineage describes no prompt: its classes are not table classes for the stage walk.
|
||||
assert.equal(semantics.allClasses.has('VAEEncode'), false);
|
||||
const row = { class: 'VAEEncode', slot: 0, strategy: 'continue', input: 'pixels' };
|
||||
for (const lineage of [undefined, {}, [null], [{ ...row, slot: -1 }], [{ ...row, strategy: 'passthrough' }], [{ ...row, input: '' }], [row, row]]) {
|
||||
assert.throws(() => resolver.compileSemantics({ ...doc, lineage }), resolver.SemanticsError, JSON.stringify(lineage));
|
||||
}
|
||||
});
|
||||
|
||||
// The committed server answers (tests/fixtures/local-inspect/resolver-vectors.json, written by
|
||||
// `python3 -m tests.local_resolver_cases --write` and checked current by the Python suite).
|
||||
test('committed server answers equal the local evidence', () => {
|
||||
const {vectors} = JSON.parse(fs.readFileSync(new URL('../fixtures/local-inspect/resolver-vectors.json', import.meta.url), 'utf8'));
|
||||
const canonical = value => Array.isArray(value) ? value.map(canonical) :
|
||||
value !== null && typeof value === 'object' ? Object.fromEntries(Object.keys(value).sort().map(key => [key, canonical(value[key])])) : value;
|
||||
let compared = 0;
|
||||
for (const {name, records, evidence} of vectors) {
|
||||
assert.deepEqual(canonical(resolver.resolveLocalEvidence(records, semantics)), canonical(evidence), name);
|
||||
compared++;
|
||||
}
|
||||
assert.ok(compared > 280, `compared ${compared}`);
|
||||
});
|
||||
|
||||
test('a list or object workflow node type is skipped, as on the server (#286)', () => {
|
||||
const graph = base({'6': {class_type: 'PromptManager', inputs: {text: 'a fox', clip: ['4', 1]}}});
|
||||
for (const type of [['PromptManager'], {name: 'PromptManager'}]) {
|
||||
const evidence = resolver.resolveLocalEvidence({prompt: JSON.stringify(graph), workflow: JSON.stringify({nodes: [{id: 6, type, widgets_values: ['x']}]})}, semantics);
|
||||
assert.equal(evidence.resolution.state, 'resolved');
|
||||
assert.equal(evidence.prompts.positive.status, 'resolved');
|
||||
assert.equal(evidence.prompts.positive.text, 'a fox');
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,43 @@
|
||||
// web/js/workspace/local-semantics.mjs (#249 local Inspect, PR L3 Task 10): the node table local Inspect resolves
|
||||
// with is the server's own web/js/inspection-semantics.json, fetched once through the API client before a file is
|
||||
// chosen. A table that cannot be fetched or used answers as the server does when its table is unusable.
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
|
||||
const TABLE = fs.readFileSync(new URL("../../web/js/inspection-semantics.json", import.meta.url), "utf8");
|
||||
let instance = 0;
|
||||
/** A fresh module instance (its cache is per page) with `fetch` and the API client stubbed. */
|
||||
async function load(respond) {
|
||||
const calls = [];
|
||||
globalThis.window = { ApiClient: { resolveUrl: (path) => `/proxy${path}` } };
|
||||
globalThis.fetch = async (url, init) => { calls.push([url, init]); return respond(); };
|
||||
const module = await import(`../../web/js/workspace/local-semantics.mjs?instance=${++instance}`);
|
||||
return { module, calls };
|
||||
}
|
||||
test.after(() => { delete globalThis.window; delete globalThis.fetch; });
|
||||
|
||||
test("the table is fetched once through the API client and compiled", async () => {
|
||||
const { module, calls } = await load(() => new Response(TABLE, { status: 200 }));
|
||||
const [first, second] = await Promise.all([module.loadInspectionSemantics(), module.loadInspectionSemantics()]);
|
||||
assert.equal(first, second);
|
||||
assert.ok(first.samplers.has("KSampler"));
|
||||
assert.equal(await module.loadInspectionSemantics(), first);
|
||||
assert.deepEqual(calls, [["/proxy/prompt_manager/js/inspection-semantics.json", { credentials: "same-origin" }]]);
|
||||
});
|
||||
|
||||
test("a failed or unusable table is null, and a later selection asks again", async () => {
|
||||
for (const respond of [() => new Response("nope", { status: 404 }), () => new Response("{\"schema\": 2}", { status: 200 }), () => { throw new TypeError("offline"); }]) {
|
||||
const { module, calls } = await load(respond);
|
||||
assert.equal(await module.loadInspectionSemantics(), null);
|
||||
assert.equal(await module.loadInspectionSemantics(), null);
|
||||
assert.equal(calls.length, 2);
|
||||
}
|
||||
});
|
||||
|
||||
test("without the API client nothing is fetched", async () => {
|
||||
const { module, calls } = await load(() => new Response(TABLE));
|
||||
delete globalThis.window;
|
||||
assert.equal(await module.loadInspectionSemantics(), null);
|
||||
assert.deepEqual(calls, []);
|
||||
});
|
||||
@@ -193,8 +193,8 @@ test("unclassified input creates no object URL", async () => {
|
||||
// declares a supported type it opens the inspector's unreadable state, with no preview and no URL (ledger ruling).
|
||||
const gone = pngFile("gone.png", "fox", { gate: () => Promise.reject(new DOMException("The file was removed", "NotReadableError")) });
|
||||
assert.equal(value.select([gone]), "accepted");
|
||||
await settle();
|
||||
const unreadable = value.controller.state;
|
||||
// The read yields between its steps (local-inspection.mjs): wait for the answer, not a fixed number of turns.
|
||||
const unreadable = await recorded(value.controller);
|
||||
assert.ok(unreadable.record, "the unreadable record is published");
|
||||
assert.deepEqual([value.state.phase, value.state.message, value.state.preview], ["ready", m.LOCAL_SESSION_MESSAGES.unreadable, null]);
|
||||
assert.equal(m.LOCAL_SESSION_MESSAGES.unreadable, "Embedded metadata could not be read.");
|
||||
|
||||
@@ -124,18 +124,51 @@ test('real inspection navigation preserves slideshow across two advances and def
|
||||
} finally {view.dispose();controller.dispose();parentScope.dispose();globalThis.setTimeout=oldSetTimeout;globalThis.clearTimeout=oldClearTimeout;f.restore();}
|
||||
});
|
||||
|
||||
test('an inline Viewer still loading its image is destroyed only after it has viewed it', async () => {
|
||||
test('Viewer.js is built only on a loaded image and an unbuilt viewer is released at once', async () => {
|
||||
const {reportedLocationAdapter:adapter} = await import('../../web/js/workspace/history-media.mjs');
|
||||
const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c')];
|
||||
const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c'),occurrenceMedia('2',0,'d')];
|
||||
const instances = [];
|
||||
// Viewer.js's inline destroy() leaves an in-flight view running; its callback then needs a live instance.
|
||||
f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.viewed = false; instances.push(this); }
|
||||
// On a loaded image Viewer.js schedules its inline build; until it runs `ready` is false and destroy() cancels it.
|
||||
f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.ready = false; this.viewed = false; instances.push(this); }
|
||||
destroy() { this.destroyed++; } resize() {} };
|
||||
const globals = {ResizeObserver:globalThis.ResizeObserver, requestAnimationFrame:globalThis.requestAnimationFrame, cancelAnimationFrame:globalThis.cancelAnimationFrame};
|
||||
Object.assign(globalThis, {ResizeObserver:class { observe() {} disconnect() {} }, requestAnimationFrame:() => 1, cancelAnimationFrame() {}});
|
||||
const view = module.createMediaViewer({container:f.container,parentScope,inspectionController:controller,adapter,sender:{open(){}}});
|
||||
try {
|
||||
view.open({items,selectedKey:'c'.repeat(64)});
|
||||
const stageImage = () => f.walk(f.container,n => n.tagName === 'img')[0];
|
||||
view.open({items,selectedKey:'d'.repeat(64)});
|
||||
// Viewer.js's own cancel misses an image whose load event is still queued, so none exists before the load.
|
||||
const unloaded = stageImage(), dropped = [];unloaded.removeAttribute = key => dropped.push(key);
|
||||
assert.equal(instances.length, 0, 'no Viewer.js before the image loads');
|
||||
f.walk(f.container,n => n.attributes['aria-label'] === 'Previous media')[0].fire('click');
|
||||
assert.deepEqual(dropped, ['src'], 'an image left before it loads is released at once');
|
||||
unloaded.fire('load');
|
||||
assert.equal(instances.length, 0, 'its late load builds no Viewer.js');
|
||||
stageImage().fire('load');
|
||||
const [loading] = instances, image = loading.node, removed = [];
|
||||
image.removeAttribute = key => removed.push(key);
|
||||
// Next drops the image whose build is pending; a deferred destroy would let that build run there.
|
||||
f.walk(f.container,n => n.attributes['aria-label'] === 'Next media')[0].fire('click');
|
||||
stageImage().fire('load');assert.equal(instances.length, 2, 'the next image builds its own Viewer.js once loaded');
|
||||
assert.equal(loading.destroyed, 1, 'an unbuilt viewer is destroyed at once');
|
||||
assert.deepEqual(removed, ['src'], 'and its image is released at once');
|
||||
loading.options.viewed.call(loading);
|
||||
assert.equal(loading.destroyed, 1, 'never destroyed twice');
|
||||
} finally { view.dispose(); parentScope.dispose(); f.restore(); Object.assign(globalThis, globals); }
|
||||
});
|
||||
|
||||
test('an inline Viewer still loading its image is destroyed only after it has viewed it', async () => {
|
||||
const {reportedLocationAdapter:adapter} = await import('../../web/js/workspace/history-media.mjs');
|
||||
const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c')];
|
||||
const instances = [];
|
||||
// A built (`ready`) inline Viewer.js's destroy() leaves an in-flight view running; its callback then needs a live instance.
|
||||
f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.ready = true; this.viewed = false; instances.push(this); }
|
||||
destroy() { this.destroyed++; } resize() {} };
|
||||
const globals = {ResizeObserver:globalThis.ResizeObserver, requestAnimationFrame:globalThis.requestAnimationFrame, cancelAnimationFrame:globalThis.cancelAnimationFrame};
|
||||
Object.assign(globalThis, {ResizeObserver:class { observe() {} disconnect() {} }, requestAnimationFrame:() => 1, cancelAnimationFrame() {}});
|
||||
const view = module.createMediaViewer({container:f.container,parentScope,inspectionController:controller,adapter,sender:{open(){}}});
|
||||
try {
|
||||
view.open({items,selectedKey:'c'.repeat(64)});f.walk(f.container,n => n.tagName === 'img')[0].fire('load');
|
||||
const [loading] = instances, image = loading.node, removed = [];
|
||||
image.removeAttribute = key => removed.push(key);
|
||||
f.walk(f.container,n => n.attributes['aria-label'] === 'Close image viewer')[0].fire('click');
|
||||
@@ -148,7 +181,7 @@ test('an inline Viewer still loading its image is destroyed only after it has vi
|
||||
loading.options.viewed.call(loading);
|
||||
assert.equal(loading.destroyed, 1, 'never destroyed twice');
|
||||
|
||||
view.open({items,selectedKey:'c'.repeat(64)});
|
||||
view.open({items,selectedKey:'c'.repeat(64)});f.walk(f.container,n => n.tagName === 'img')[0].fire('load');
|
||||
const shown = instances[1]; shown.viewed = true; shown.options.viewed.call(shown);
|
||||
f.walk(f.container,n => n.attributes['aria-label'] === 'Close image viewer')[0].fire('click');
|
||||
assert.equal(shown.destroyed, 1, 'a viewed image is destroyed at once');
|
||||
@@ -236,7 +269,7 @@ test('viewer uses adapter.mediaUrl for image and video', async () => {
|
||||
const img = f.walk(f.container,n => n.tagName === 'img')[0];
|
||||
assert.equal(img.src, image.url, 'the blob URL is used exactly');
|
||||
assert.doesNotMatch(img.src, /\?/);
|
||||
assert.equal(instances[0].node, img, 'Viewer.js shows the blob image');
|
||||
img.fire('load');assert.equal(instances[0].node, img, 'Viewer.js shows the blob image');
|
||||
const download = f.walk(f.container,n => n.attributes['aria-label'] === 'Download original')[0];
|
||||
assert.equal(download.href, image.url);assert.equal(download.download, 'a.png');
|
||||
f.walk(f.container,n => n.attributes['aria-label'] === 'Next media')[0].fire('click');
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
// Browser display defaults (#249 System S3 Task 7): the workspace reads and writes the legacy gallery's own
|
||||
// `gallerySettings` object (Q3, "One display configuration"), writes only the keys it changed, never consumes the
|
||||
// legacy paging keys, and says when this browser refused to keep a change.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import gallerySettings from '../../web/js/gallery-settings.js';
|
||||
|
||||
const module = await import('../../web/js/workspace/preferences.mjs').catch(error => {
|
||||
if (error.code !== 'ERR_MODULE_NOT_FOUND') throw error;
|
||||
return {};
|
||||
});
|
||||
globalThis.GallerySettings = gallerySettings;
|
||||
const KEY = 'gallerySettings';
|
||||
|
||||
/** A localStorage stand-in holding `text` under the settings key; `setItem` can be made to throw. */
|
||||
function storage(text = null, { quota = false } = {}) {
|
||||
const writes = [];
|
||||
const store = {
|
||||
text,
|
||||
writes,
|
||||
getItem(key) { assert.equal(key, KEY); return store.text; },
|
||||
setItem(key, value) {
|
||||
assert.equal(key, KEY);
|
||||
writes.push(value);
|
||||
if (quota) { const error = new Error('Quota exceeded'); error.name = 'QuotaExceededError'; throw error; }
|
||||
store.text = value;
|
||||
},
|
||||
removeItem() { assert.fail('Preferences never remove the shared settings'); },
|
||||
};
|
||||
return store;
|
||||
}
|
||||
const denied = () => { const error = new Error('Denied'); error.name = 'SecurityError'; throw error; };
|
||||
|
||||
test('normalizes new workspace keys and keeps legacy keys', () => {
|
||||
assert.equal(typeof module.readPreferences, 'function', 'preferences.mjs exports readPreferences');
|
||||
module.resetTabPreferences();
|
||||
const legacy = { defaultViewMode: 'list', gridColumns: 5, defaultLimit: 200, thumbnailPromptDismissed: 'sig', extra: 1 };
|
||||
const store = storage(JSON.stringify({ ...legacy, workspaceLayout: 'masonry', workspaceDensity: 'compact', showFilePaths: false }));
|
||||
const { values, persisted } = module.readPreferences(() => store);
|
||||
assert.equal(persisted, true);
|
||||
assert.equal(values.workspaceLayout, 'masonry');
|
||||
assert.equal(values.workspaceDensity, 'compact');
|
||||
assert.equal(values.showFilePaths, false);
|
||||
// Invalid workspace values fall back to the defaults, as every legacy key does.
|
||||
store.text = JSON.stringify({ workspaceLayout: 'mosaic', workspaceDensity: 7, videoMute: 'no' });
|
||||
const fallback = module.readPreferences(() => store).values;
|
||||
assert.equal(fallback.workspaceLayout, 'grid');
|
||||
assert.equal(fallback.workspaceDensity, 'comfortable');
|
||||
assert.equal(fallback.videoMute, true);
|
||||
// The legacy normalizer knows the new keys and still keeps unknown ones.
|
||||
const normalized = gallerySettings.normalizeGallerySettings({ ...legacy, workspaceLayout: 'list' });
|
||||
assert.equal(normalized.workspaceLayout, 'list');
|
||||
assert.equal(normalized.workspaceDensity, 'comfortable');
|
||||
assert.equal(normalized.extra, 1);
|
||||
assert.equal(normalized.thumbnailPromptDismissed, 'sig');
|
||||
assert.deepEqual(gallerySettings.WORKSPACE_LAYOUTS, ['grid', 'list', 'masonry']);
|
||||
assert.deepEqual(gallerySettings.WORKSPACE_DENSITIES, ['comfortable', 'large', 'compact']);
|
||||
});
|
||||
|
||||
test('write touches only dirty keys', () => {
|
||||
module.resetTabPreferences();
|
||||
const stored = { defaultViewMode: 'list', gridColumns: 5, thumbnailsGenerated: true, extra: { kept: true }, videoMute: false };
|
||||
const store = storage(JSON.stringify(stored));
|
||||
const result = module.writePreferences({ workspaceLayout: 'masonry', showImageInfo: false }, () => store);
|
||||
assert.deepEqual(result, { persisted: true, conflicts: [] });
|
||||
assert.equal(store.writes.length, 1);
|
||||
// Every other key, legacy or unknown, is written back exactly as it was stored: no defaults are filled in.
|
||||
assert.deepEqual(JSON.parse(store.text), { ...stored, workspaceLayout: 'masonry', showImageInfo: false });
|
||||
// It re-reads the stored object, so a key another page wrote after this one read is kept.
|
||||
store.text = JSON.stringify({ ...JSON.parse(store.text), gridColumns: 9 });
|
||||
module.writePreferences({ videoLoop: false }, () => store);
|
||||
assert.equal(JSON.parse(store.text).gridColumns, 9);
|
||||
assert.equal(JSON.parse(store.text).workspaceLayout, 'masonry');
|
||||
// An empty or malformed stored value starts from an empty object, never the defaults.
|
||||
for (const text of [null, '{broken', '[]', '42']) {
|
||||
const fresh = storage(text);
|
||||
assert.equal(module.writePreferences({ cacheMetadata: false }, () => fresh).persisted, true);
|
||||
assert.deepEqual(JSON.parse(fresh.text), { cacheMetadata: false });
|
||||
}
|
||||
// Only workspace keys with valid values are accepted; nothing is written for a bad change.
|
||||
for (const bad of [{ gridColumns: 4 }, { defaultLimit: 50 }, { workspaceLayout: 'mosaic' }, { videoMute: 'yes' }, { imageQuality: 'ultra' }]) {
|
||||
const untouched = storage(JSON.stringify(stored));
|
||||
assert.throws(() => module.writePreferences(bad, () => untouched), TypeError);
|
||||
assert.equal(untouched.writes.length, 0);
|
||||
}
|
||||
// A key changed elsewhere since this tab read it (its baseline) is reported, and the explicit save still wins.
|
||||
const elsewhere = storage(JSON.stringify({ videoLoop: false }));
|
||||
const outcome = module.writePreferences({ videoLoop: true, videoMute: false }, () => elsewhere, { baseline: { videoLoop: true, videoMute: true } });
|
||||
assert.deepEqual(outcome, { persisted: true, conflicts: ['videoLoop'] });
|
||||
assert.equal(JSON.parse(elsewhere.text).videoLoop, true);
|
||||
});
|
||||
|
||||
test('denied storage reports tab-only', () => {
|
||||
module.resetTabPreferences();
|
||||
const read = module.readPreferences(denied);
|
||||
assert.equal(read.persisted, false);
|
||||
assert.equal(read.values.workspaceLayout, 'grid');
|
||||
const result = module.writePreferences({ workspaceLayout: 'list', videoAutoplay: true }, denied);
|
||||
assert.deepEqual(result, { persisted: false, conflicts: [] });
|
||||
// The change is kept for this tab: every reader in it (a new Images visit, the viewer) sees it.
|
||||
const tab = module.readPreferences(denied);
|
||||
assert.equal(tab.persisted, false);
|
||||
assert.equal(tab.values.workspaceLayout, 'list');
|
||||
assert.equal(tab.values.videoAutoplay, true);
|
||||
// A getItem that throws is denied storage too.
|
||||
const throwing = { getItem: denied, setItem: denied };
|
||||
assert.equal(module.readPreferences(() => throwing).values.workspaceLayout, 'list');
|
||||
assert.equal(module.writePreferences({ workspaceDensity: 'large' }, () => throwing).persisted, false);
|
||||
// Once storage accepts a write, the tab copy of that key gives way to the stored value.
|
||||
const working = storage(JSON.stringify({ workspaceLayout: 'masonry' }));
|
||||
assert.equal(module.writePreferences({ workspaceLayout: 'masonry' }, () => working).persisted, true);
|
||||
assert.equal(module.readPreferences(() => working).values.workspaceLayout, 'masonry');
|
||||
module.resetTabPreferences();
|
||||
assert.equal(module.readPreferences(denied).values.videoAutoplay, false);
|
||||
});
|
||||
|
||||
test('quota error keeps previous bytes', () => {
|
||||
module.resetTabPreferences();
|
||||
const before = JSON.stringify({ gridColumns: 6, workspaceDensity: 'large' });
|
||||
const store = storage(before, { quota: true });
|
||||
const result = module.writePreferences({ workspaceDensity: 'compact' }, () => store);
|
||||
assert.deepEqual(result, { persisted: false, conflicts: [] });
|
||||
assert.equal(store.writes.length, 1);
|
||||
assert.equal(store.text, before);
|
||||
// This tab keeps the change it could not store.
|
||||
assert.equal(module.readPreferences(() => store).values.workspaceDensity, 'compact');
|
||||
module.resetTabPreferences();
|
||||
assert.equal(module.readPreferences(() => store).values.workspaceDensity, 'large');
|
||||
});
|
||||
|
||||
/** A window stand-in that records its storage listeners. */
|
||||
function fakeWindow() {
|
||||
const listeners = new Set();
|
||||
return {
|
||||
listeners,
|
||||
addEventListener(type, listener) { assert.equal(type, 'storage'); listeners.add(listener); },
|
||||
removeEventListener(type, listener) { assert.equal(type, 'storage'); listeners.delete(listener); },
|
||||
fire(event) { for (const listener of [...listeners]) listener(event); },
|
||||
};
|
||||
}
|
||||
|
||||
test('storage event updates clean values and flags dirty conflicts', () => {
|
||||
module.resetTabPreferences();
|
||||
const store = storage(JSON.stringify({ workspaceLayout: 'list', videoMute: false }));
|
||||
const win = fakeWindow();
|
||||
const seen = [];
|
||||
const unsubscribe = module.subscribePreferences(win, next => seen.push(next), () => store);
|
||||
assert.equal(win.listeners.size, 1);
|
||||
// Another key, or another storage area's event, is not ours.
|
||||
win.fire({ key: 'pm_update_notice_dismissed', storageArea: store });
|
||||
win.fire({ key: KEY, storageArea: { other: true } });
|
||||
assert.equal(seen.length, 0);
|
||||
win.fire({ key: KEY, storageArea: store });
|
||||
win.fire({ key: null, storageArea: store }); // localStorage.clear() elsewhere
|
||||
assert.equal(seen.length, 2);
|
||||
assert.equal(seen[0].values.workspaceLayout, 'list');
|
||||
unsubscribe();
|
||||
assert.equal(win.listeners.size, 0);
|
||||
// The reconciliation of the System drafts: clean values follow the other page; a key this tab has a draft for and
|
||||
// the other page changed is a conflict, and the draft stays.
|
||||
const baseline = { workspaceLayout: 'grid', videoMute: true, videoLoop: true };
|
||||
const next = { workspaceLayout: 'list', videoMute: true, videoLoop: false };
|
||||
assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { videoMute: false, videoLoop: false }, next }), { conflicts: [] });
|
||||
assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { workspaceLayout: 'masonry' }, next }), { conflicts: ['workspaceLayout'] });
|
||||
assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { workspaceLayout: 'list' }, next }), { conflicts: [] });
|
||||
});
|
||||
|
||||
test('legacy paging keys are never consumed by the workspace', () => {
|
||||
module.resetTabPreferences();
|
||||
const store = storage(JSON.stringify({ defaultLimit: 500, gridColumns: 12, infiniteScroll: true, lazyLoading: false,
|
||||
defaultViewMode: 'list', debugMode: true, apiTimeout: 9, thumbnailsGenerated: true, checkThumbnailsAtStartup: false }));
|
||||
const { values } = module.readPreferences(() => store);
|
||||
assert.deepEqual(Object.keys(values).sort(), [...module.WORKSPACE_PREFERENCE_KEYS].sort());
|
||||
for (const key of ['defaultLimit', 'gridColumns', 'infiniteScroll', 'lazyLoading', 'defaultViewMode', 'debugMode', 'apiTimeout',
|
||||
'thumbnailsGenerated', 'checkThumbnailsAtStartup', 'thumbnailPromptDismissed']) assert.equal(Object.hasOwn(values, key), false, key);
|
||||
// The display defaults are the gallery's own defaults for these keys.
|
||||
const defaults = module.preferenceDefaults();
|
||||
for (const key of module.WORKSPACE_PREFERENCE_KEYS) assert.equal(defaults[key], gallerySettings.DEFAULT_SETTINGS[key], key);
|
||||
});
|
||||
|
||||
test('a later save flushes the changes kept for this tab only', () => {
|
||||
module.resetTabPreferences();
|
||||
const store = storage(JSON.stringify({ gridColumns: 4 }), { quota: true });
|
||||
assert.equal(module.writePreferences({ workspaceLayout: 'list' }, () => store).persisted, false);
|
||||
// Storage accepts writes again: saving another key stores the tab-only change with it, so "Saved" is true for both.
|
||||
const working = storage(store.text);
|
||||
assert.deepEqual(module.writePreferences({ videoLoop: false }, () => working), { persisted: true, conflicts: [] });
|
||||
assert.deepEqual(JSON.parse(working.text), { gridColumns: 4, workspaceLayout: 'list', videoLoop: false });
|
||||
// Nothing is overlaid any more: another page's later change reads through.
|
||||
working.text = JSON.stringify({ ...JSON.parse(working.text), workspaceLayout: 'masonry' });
|
||||
assert.equal(module.readPreferences(() => working).values.workspaceLayout, 'masonry');
|
||||
module.resetTabPreferences();
|
||||
});
|
||||
@@ -23,8 +23,50 @@ test('tags is a native route titled Tags; only unbuilt routes hand off to legacy
|
||||
assert.equal(routeTitle(route), route.label);
|
||||
}
|
||||
assert.equal(routeTitle(resolveRoute('#/library/7')), 'Prompt library');
|
||||
for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id);
|
||||
for (const id of ['system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), true, id);
|
||||
// System is native from S2 (#249 System): its undelivered sections hand off per section (D94), not the route.
|
||||
for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity', 'system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id);
|
||||
});
|
||||
|
||||
test('system resolves its sections; #/overview stays the production hub', async () => {
|
||||
const { resolveRoute, routeTitle, isHandoff, routes } = await source;
|
||||
// `#/system` is the Overview section and keeps the sidebar's hash.
|
||||
const system = resolveRoute('#/system');
|
||||
assert.deepEqual([system.id, system.section, system.hash, routeTitle(system), isHandoff(system)],
|
||||
['system', 'overview', '#/system', 'System', false]);
|
||||
for (const section of ['overview', 'appearance', 'sources', 'integrations', 'tagging', 'maintenance', 'backups', 'diagnostics']) {
|
||||
const route = resolveRoute(`#/system/${section}`);
|
||||
assert.deepEqual([route.id, route.section, route.hash], ['system', section, `#/system/${section}`]);
|
||||
}
|
||||
// An unknown child is the Overview section with a replaced hash.
|
||||
for (const hash of ['#/system/general', '#/system/', '#/system/__proto__', '#/system/backups/1', '#/system/<b>']) {
|
||||
const route = resolveRoute(hash);
|
||||
assert.deepEqual([route.id, route.section, route.hash], ['system', 'overview', '#/system/overview'], hash);
|
||||
}
|
||||
// D28: the production hub is untouched.
|
||||
const hub = resolveRoute('#/overview');
|
||||
assert.deepEqual([hub.id, hub.section], ['overview', undefined]);
|
||||
// Q12: Overview's System card opens the native System Overview, with no legacy href or action.
|
||||
const card = routes.find(route => route.id === 'system');
|
||||
assert.equal(card.href, '#/system/overview');
|
||||
assert.equal(card.native, true);
|
||||
assert.equal(card.action, 'Open System');
|
||||
});
|
||||
|
||||
test('the router renders again when only the System section changes', async () => {
|
||||
const { mountRouter } = await source;
|
||||
const win = new EventTarget();
|
||||
win.location = { hash: '#/system' };
|
||||
const replacements = [];
|
||||
win.history = { state: null, replaceState(state, unused, hash) { replacements.push(hash); win.location.hash = hash; } };
|
||||
const seen = [];
|
||||
const dispose = mountRouter(win, route => seen.push([route.id, route.section]));
|
||||
for (const hash of ['#/system/appearance', '#/system/appearance', '#/system/bogus', '#/system/overview', '#/images']) {
|
||||
win.location.hash = hash;
|
||||
win.dispatchEvent(new Event('hashchange'));
|
||||
}
|
||||
assert.deepEqual(seen, [['system', 'overview'], ['system', 'appearance'], ['system', 'overview'], ['images', undefined]]);
|
||||
assert.deepEqual(replacements, ['#/system/overview']);
|
||||
dispose();
|
||||
});
|
||||
|
||||
test('inspect is the native Inspect a file page with the concept description', async () => {
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
// System › Appearance (#249 System S3 Task 8): the concept's "Browsing & display" rows (app.js 676–678) on the drafts
|
||||
// of the one Save row, their stored values (Q3 mappings), the host timeout's 0–300 whole seconds and the D97 rows.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const gallerySettings = require('../../web/js/gallery-settings.js');
|
||||
|
||||
const appearanceModule = import('../../web/js/workspace/system-appearance.mjs');
|
||||
const stateModule = import('../../web/js/workspace/system-state.mjs');
|
||||
const coreModule = import('../../web/js/shared/toolkit/core.js');
|
||||
globalThis.GallerySettings = gallerySettings;
|
||||
|
||||
/** A minimal DOM whose elements count their live listeners (a listener's signal aborts it, as in a browser). */
|
||||
function fakeDOM() {
|
||||
const previous = global.document;
|
||||
const elements = [];
|
||||
class Element {
|
||||
constructor(tag) {
|
||||
this.tagName = tag.toUpperCase();
|
||||
this.children = [];
|
||||
this.attributes = {};
|
||||
this.listeners = [];
|
||||
this.dataset = {};
|
||||
this.className = '';
|
||||
this.id = '';
|
||||
this.value = '';
|
||||
this._text = '';
|
||||
elements.push(this);
|
||||
}
|
||||
get textContent() { return this._text + this.children.map(child => child.textContent).join(''); }
|
||||
set textContent(text) { this.children = []; this._text = String(text); }
|
||||
append(...nodes) {
|
||||
for (const node of nodes) {
|
||||
const child = typeof node === 'string' ? Object.assign(new Element('#text'), { _text: node }) : node;
|
||||
child.parentNode = this;
|
||||
this.children.push(child);
|
||||
}
|
||||
}
|
||||
remove() { if (this.parentNode) this.parentNode.children = this.parentNode.children.filter(child => child !== this); this.parentNode = null; }
|
||||
replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); }
|
||||
setAttribute(name, value) { this.attributes[name] = String(value); }
|
||||
getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; }
|
||||
removeAttribute(name) { delete this.attributes[name]; }
|
||||
addEventListener(type, callback, options = {}) {
|
||||
if (options.signal?.aborted) return;
|
||||
const entry = { type, callback };
|
||||
this.listeners.push(entry);
|
||||
options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); });
|
||||
}
|
||||
removeEventListener(type, callback) { this.listeners = this.listeners.filter(item => item.type !== type || item.callback !== callback); }
|
||||
dispatch(type) { for (const { callback } of this.listeners.filter(item => item.type === type)) callback({ type, target: this }); }
|
||||
click() { this.dispatch('click'); }
|
||||
}
|
||||
global.document = {
|
||||
createElement: tag => new Element(tag),
|
||||
createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }),
|
||||
};
|
||||
const walk = (node, found = []) => { found.push(node); node.children.forEach(child => walk(child, found)); return found; };
|
||||
return {
|
||||
listenerCount: () => elements.reduce((sum, element) => sum + element.listeners.length, 0),
|
||||
find: (root, predicate) => walk(root).find(predicate),
|
||||
tags: node => node.children.map(child => child.tagName.toLowerCase()),
|
||||
restore() { global.document = previous; },
|
||||
};
|
||||
}
|
||||
|
||||
const BROWSER = { workspaceLayout: 'grid', workspaceDensity: 'comfortable', autoLoadMetadata: true, cacheMetadata: true,
|
||||
imageQuality: 'medium', videoAutoplay: false, videoMute: true, videoLoop: true, showFilePaths: true, showImageInfo: true,
|
||||
sidebarCollapsedByDefault: false };
|
||||
|
||||
async function mounted({ host = { webui_display_mode: 'newtab', result_timeout: 5 }, browser = BROWSER, hostFails = false } = {}) {
|
||||
const { mountAppearance } = await appearanceModule;
|
||||
const { createSystemDrafts } = await stateModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
let hostValues = null;
|
||||
const drafts = createSystemDrafts({
|
||||
readBrowser: () => ({ ...browser }), readRuntime: () => ({}),
|
||||
readHost: async () => { if (hostFails) throw new Error('offline'); return { values: hostValues, revision: 'sha256:one' }; },
|
||||
writeBrowser: async () => ({ persisted: true }), writeHost: async () => ({ success: true, persisted: true }), writeRuntime: async () => ({}),
|
||||
});
|
||||
await drafts.load('browser');
|
||||
const scope = createScope();
|
||||
const [surface] = mountAppearance(null, { scope, drafts });
|
||||
const byId = id => dom.find(surface, node => node.dataset?.setting === id);
|
||||
return { dom, drafts, scope, surface, byId, loadHost: async () => { hostValues = host; await drafts.load('host'); },
|
||||
fixHost: () => { hostFails = false; } };
|
||||
}
|
||||
|
||||
test('the surface is the concept section with the concept rows in order, then the D97 rows', async () => {
|
||||
const { APPEARANCE_ROWS } = await appearanceModule;
|
||||
const { dom, surface, scope } = await mounted();
|
||||
try {
|
||||
assert.equal(surface.className, 'surface');
|
||||
assert.equal(surface.children[0].textContent, 'Browsing & display');
|
||||
assert.equal(surface.children[1].textContent, 'One set of defaults; toolbar changes apply to the current view.');
|
||||
const rows = surface.children.slice(2);
|
||||
const titles = rows.map(row => row.children[0].children[0].textContent);
|
||||
assert.deepEqual(titles, ['Default layout', 'Thumbnail size', 'Continuous browsing', 'Load metadata on selection',
|
||||
'Cache metadata in this browser', 'Image loading quality', 'Autoplay videos', 'Mute videos by default', 'Loop videos',
|
||||
'Show file paths', 'Open from ComfyUI', 'Node search auto-hide (seconds)', 'Show image details on cards',
|
||||
'Collapse viewer details by default', 'Reset display defaults']);
|
||||
const continuous = rows[2];
|
||||
assert.deepEqual(dom.tags(continuous.children[0]), ['h3', 'p']);
|
||||
assert.equal(continuous.children[1].className, 'chip blue');
|
||||
assert.equal(continuous.children[1].textContent, 'Infinite scroll');
|
||||
assert.equal(rows.at(-1).className, 'operation');
|
||||
assert.equal(APPEARANCE_ROWS.filter(Array.isArray).length, 13);
|
||||
} finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); }
|
||||
});
|
||||
|
||||
test('added rows reach their consumers: each control edits its stored key with the Q3 values', async () => {
|
||||
const { dom, drafts, scope, byId, loadHost } = await mounted();
|
||||
try {
|
||||
// Host rows stay disabled until the server settings answered.
|
||||
assert.equal(byId('open-mode').disabled, true);
|
||||
assert.equal(byId('timeout').disabled, true);
|
||||
await loadHost();
|
||||
assert.equal(byId('open-mode').disabled, false);
|
||||
assert.equal(byId('timeout').value, '5');
|
||||
assert.deepEqual(byId('image-quality').children.map(option => [option.value, option.textContent]),
|
||||
[['medium', 'Optimized'], ['high', 'Original'], ['low', 'Fast']]);
|
||||
assert.deepEqual(byId('open-mode').children.map(option => [option.value, option.textContent]),
|
||||
[['newtab', 'New tab'], ['popup', 'Popup window']]);
|
||||
const edits = [['default-view', 'masonry'], ['density', 'large'], ['image-quality', 'low'], ['open-mode', 'popup']];
|
||||
for (const [id, value] of edits) { byId(id).value = value; byId(id).dispatch('change'); }
|
||||
for (const id of ['metadata-auto', 'metadata-cache', 'video-muted', 'video-loop', 'paths', 'card-details']) {
|
||||
byId(id).checked = false; byId(id).dispatch('change');
|
||||
}
|
||||
for (const id of ['video-auto', 'viewer-details']) { byId(id).checked = true; byId(id).dispatch('change'); }
|
||||
assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'masonry', workspaceDensity: 'large', imageQuality: 'low',
|
||||
autoLoadMetadata: false, cacheMetadata: false, videoMute: false, videoLoop: false, showFilePaths: false, showImageInfo: false,
|
||||
videoAutoplay: true, sidebarCollapsedByDefault: true });
|
||||
assert.deepEqual(drafts.changes('host'), { webui_display_mode: 'popup' });
|
||||
} finally { scope.dispose(); dom.restore(); }
|
||||
});
|
||||
|
||||
test('the auto-hide timeout takes whole seconds from 0 to 300 only', async () => {
|
||||
const { resultTimeout } = await appearanceModule;
|
||||
assert.deepEqual(['0', '30', ' 300 ', '301', '-1', '2.5', '', 'abc', '1e2'].map(resultTimeout), [0, 30, 300, null, null, null, null, null, null]);
|
||||
const { dom, drafts, scope, byId, loadHost } = await mounted();
|
||||
try {
|
||||
await loadHost();
|
||||
const timeout = byId('timeout');
|
||||
timeout.value = '45'; timeout.dispatch('change');
|
||||
assert.deepEqual(drafts.changes('host'), { result_timeout: 45 });
|
||||
timeout.value = '301'; timeout.dispatch('change');
|
||||
assert.deepEqual(drafts.changes('host'), {});
|
||||
assert.deepEqual([...drafts.failed('host')], ['result_timeout']);
|
||||
timeout.value = '0'; timeout.dispatch('change');
|
||||
assert.deepEqual(drafts.changes('host'), { result_timeout: 0 });
|
||||
assert.deepEqual([...drafts.failed('host')], []);
|
||||
} finally { scope.dispose(); dom.restore(); }
|
||||
});
|
||||
|
||||
test('reset restores defaults without touching other keys', async () => {
|
||||
const { DISPLAY_KEYS } = await appearanceModule;
|
||||
const changed = { ...BROWSER, workspaceLayout: 'list', videoMute: false, showImageInfo: false, sidebarCollapsedByDefault: true };
|
||||
const { dom, drafts, scope, surface, loadHost } = await mounted({ browser: changed });
|
||||
try {
|
||||
await loadHost();
|
||||
drafts.set('host', 'result_timeout', 9);
|
||||
const reset = dom.find(surface.children.at(-1), node => node.tagName === 'BUTTON');
|
||||
assert.equal(reset.textContent, 'Reset');
|
||||
reset.click();
|
||||
assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'grid', videoMute: true, showImageInfo: true, sidebarCollapsedByDefault: false });
|
||||
// The host fields and every key outside the display defaults are left as they are.
|
||||
assert.deepEqual(drafts.changes('host'), { result_timeout: 9 });
|
||||
assert.deepEqual([...DISPLAY_KEYS].sort(), Object.keys(BROWSER).sort());
|
||||
} finally { scope.dispose(); dom.restore(); }
|
||||
});
|
||||
|
||||
test('host rows that could not be read stay disabled and say so; a later read clears it', async () => {
|
||||
const { HOST_UNAVAILABLE } = await appearanceModule;
|
||||
const { dom, drafts, scope, byId, surface, fixHost } = await mounted({ hostFails: true });
|
||||
const notes = () => surface.children.slice(2).flatMap(row => row.children[0].children.slice(2).map(note => [row.children[0].children[0].textContent, note.className, note.textContent]));
|
||||
try {
|
||||
assert.deepEqual(notes(), []);
|
||||
assert.equal(await drafts.reloadHost(), false);
|
||||
assert.deepEqual(notes(), [['Open from ComfyUI', 'small', HOST_UNAVAILABLE], ['Node search auto-hide (seconds)', 'small', HOST_UNAVAILABLE]]);
|
||||
assert.equal(byId('timeout').disabled, true);
|
||||
// Repaints keep one line per row.
|
||||
drafts.set('browser', 'videoMute', false);
|
||||
assert.equal(notes().length, 2);
|
||||
fixHost();
|
||||
assert.equal(await drafts.reloadHost(), true);
|
||||
assert.deepEqual(notes(), []);
|
||||
} finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); }
|
||||
});
|
||||
@@ -0,0 +1,214 @@
|
||||
// System controls (#249 System S2 Task 5): the concept's `setting()` (app.js 624–634, its toggles hydrated into Toolkit
|
||||
// switches by `hydrateSwitches` 708–717), `operation()` (645–654) and `sectionNav()` (306–308) as DOM builders whose
|
||||
// listeners live on the caller's scope.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const controlsModule = import('../../web/js/workspace/system-controls.mjs');
|
||||
const coreModule = import('../../web/js/shared/toolkit/core.js');
|
||||
|
||||
/** A minimal DOM whose elements count their live listeners (a listener's signal aborts it, as in a browser). */
|
||||
function fakeDOM() {
|
||||
const previous = global.document;
|
||||
const elements = [];
|
||||
class Element {
|
||||
constructor(tag) {
|
||||
this.tagName = tag.toUpperCase();
|
||||
this.children = [];
|
||||
this.attributes = {};
|
||||
this.listeners = [];
|
||||
this.dataset = {};
|
||||
this.className = '';
|
||||
this.id = '';
|
||||
this.value = '';
|
||||
this._text = '';
|
||||
elements.push(this);
|
||||
}
|
||||
get textContent() { return this._text + this.children.map(child => child.textContent).join(''); }
|
||||
set textContent(text) { this.children = []; this._text = String(text); }
|
||||
append(...nodes) {
|
||||
for (const node of nodes) {
|
||||
const child = typeof node === 'string' ? Object.assign(new Element('#text'), { _text: node }) : node;
|
||||
child.parentNode = this;
|
||||
this.children.push(child);
|
||||
}
|
||||
}
|
||||
replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); }
|
||||
setAttribute(name, value) { this.attributes[name] = String(value); }
|
||||
getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; }
|
||||
removeAttribute(name) { delete this.attributes[name]; }
|
||||
addEventListener(type, callback, options = {}) {
|
||||
if (options.signal?.aborted) return;
|
||||
const entry = { type, callback };
|
||||
this.listeners.push(entry);
|
||||
options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); });
|
||||
}
|
||||
removeEventListener(type, callback) { this.listeners = this.listeners.filter(item => item.type !== type || item.callback !== callback); }
|
||||
dispatch(type) { for (const { callback } of this.listeners.filter(item => item.type === type)) callback({ type, target: this }); }
|
||||
click() { this.dispatch('click'); }
|
||||
}
|
||||
global.document = {
|
||||
createElement: tag => new Element(tag),
|
||||
createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }),
|
||||
};
|
||||
const walk = (node, found = []) => { found.push(node); node.children.forEach(child => walk(child, found)); return found; };
|
||||
return {
|
||||
listenerCount: () => elements.reduce((sum, element) => sum + element.listeners.length, 0),
|
||||
find: (root, predicate) => walk(root).find(predicate),
|
||||
tags: node => node.children.map(child => child.tagName.toLowerCase()),
|
||||
restore() { global.document = previous; },
|
||||
};
|
||||
}
|
||||
|
||||
test('settingRow is the concept row: div > label[for] + p, then the control', async () => {
|
||||
const { settingRow } = await controlsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
try {
|
||||
const changes = [];
|
||||
const row = settingRow({ id: 'timeout', label: 'Node search auto-hide (seconds)',
|
||||
description: 'Node search behavior only. Use 0 to keep results visible.',
|
||||
control: { type: 'number', value: 30, min: 0, max: 300 }, scope, onChange: value => changes.push(value) });
|
||||
assert.equal(row.className, 'setting-row');
|
||||
assert.deepEqual(dom.tags(row), ['div', 'input']);
|
||||
const [text, input] = row.children;
|
||||
assert.deepEqual(dom.tags(text), ['label', 'p']);
|
||||
assert.equal(text.children[0].textContent, 'Node search auto-hide (seconds)');
|
||||
assert.equal(text.children[0].htmlFor, 'timeout');
|
||||
assert.equal(text.children[1].textContent, 'Node search behavior only. Use 0 to keep results visible.');
|
||||
assert.deepEqual([input.id, input.type, input.value, input.min, input.max], ['timeout', 'number', '30', '0', '300']);
|
||||
assert.equal(input.dataset.setting, 'timeout');
|
||||
input.value = '45';
|
||||
input.dispatch('change');
|
||||
assert.deepEqual(changes, [45]);
|
||||
|
||||
const select = settingRow({ id: 'open-mode', label: 'Open from ComfyUI', description: 'Keep the node pack’s popup / new-tab preference.',
|
||||
control: { type: 'select', options: [['newtab', 'New tab'], ['popup', 'Popup window']], value: 'popup' }, scope,
|
||||
onChange: value => changes.push(value) }).children[1];
|
||||
assert.equal(select.tagName, 'SELECT');
|
||||
assert.deepEqual(select.children.map(option => [option.value, option.textContent]), [['newtab', 'New tab'], ['popup', 'Popup window']]);
|
||||
assert.equal(select.value, 'popup');
|
||||
select.value = 'newtab';
|
||||
select.dispatch('change');
|
||||
assert.deepEqual(changes, [45, 'newtab']);
|
||||
|
||||
// The concept's range is 1–16 unless the row says otherwise (D113: the CPU count).
|
||||
const range = settingRow({ id: 'workers', label: 'Worker threads', description: 'Balance background scans.',
|
||||
control: { type: 'range', value: 4 } }).children[1];
|
||||
assert.deepEqual([range.type, range.min, range.max, range.value], ['range', '1', '16', '4']);
|
||||
const password = settingRow({ id: 'civitai-key', label: 'CivitAI API key', description: 'Credentials stay on the server.',
|
||||
control: { type: 'password', value: '' } }).children[1];
|
||||
assert.equal(password.type, 'password');
|
||||
|
||||
// A ready-made control is placed as given and takes the row id.
|
||||
const custom = document.createElement('textarea');
|
||||
const customRow = settingRow({ id: 'custom', label: 'Custom', description: 'Any control.', control: custom });
|
||||
assert.equal(customRow.children[1], custom);
|
||||
assert.equal(custom.id, 'custom');
|
||||
assert.throws(() => settingRow({ id: '', label: 'x', description: 'y', control: { type: 'number' } }), TypeError);
|
||||
assert.throws(() => settingRow({ id: 'x', label: 'x', description: 'y', control: { type: 'colour' } }), TypeError);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('a toggle row holds the Toolkit switch, labelled by the row label', async () => {
|
||||
const { settingRow } = await controlsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
try {
|
||||
const changes = [];
|
||||
const row = settingRow({ id: 'video-auto', label: 'Autoplay videos', description: 'Videos stay within the same gallery.',
|
||||
control: { type: 'toggle', checked: false }, scope, onChange: value => changes.push(value) });
|
||||
const field = row.children[1];
|
||||
// createSwitch('', '', id): `label.ca-switch` > input[type=checkbox][role=switch] + an empty label text.
|
||||
assert.equal(field.tagName, 'LABEL');
|
||||
assert.equal(field.className, 'ca-switch');
|
||||
const input = field.children[0];
|
||||
assert.deepEqual([input.type, input.role, input.id, input.checked], ['checkbox', 'switch', 'video-auto', false]);
|
||||
assert.equal(row.children[0].children[0].htmlFor, input.id);
|
||||
assert.equal(input.dataset.setting, 'video-auto');
|
||||
input.checked = true;
|
||||
input.dispatch('change');
|
||||
assert.deepEqual(changes, [true]);
|
||||
// The concept's toggles are on unless configured off (`val !== false`).
|
||||
const on = settingRow({ id: 'video-loop', label: 'Loop videos', description: 'Repeat a clip.', control: { type: 'toggle' } });
|
||||
assert.equal(on.children[1].children[0].checked, true);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('operationRow is the concept operation with a danger class only when flagged', async () => {
|
||||
const { operationRow } = await controlsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
try {
|
||||
const clicks = [];
|
||||
const row = operationRow({ title: 'Missing-file links',
|
||||
description: 'Remove stale database links only. Files on disk are never deleted by this operation.',
|
||||
label: 'Review plan', danger: true, scope, onClick: button => clicks.push(button) });
|
||||
assert.equal(row.className, 'operation');
|
||||
assert.deepEqual(dom.tags(row), ['div', 'div']);
|
||||
assert.deepEqual(dom.tags(row.children[0]), ['h3', 'p']);
|
||||
assert.equal(row.children[0].children[0].textContent, 'Missing-file links');
|
||||
assert.equal(row.children[1].className, 'row-actions');
|
||||
const button = row.children[1].children[0];
|
||||
assert.deepEqual([button.tagName, button.type, button.textContent, button.className], ['BUTTON', 'button', 'Review plan', 'danger']);
|
||||
button.click();
|
||||
assert.deepEqual(clicks, [button]);
|
||||
const plain = operationRow({ title: 'Statistics', description: 'Usage by prompt, category, tag and date.', label: 'Explore',
|
||||
scope, onClick() {} });
|
||||
assert.equal(plain.children[1].children[0].className, '');
|
||||
assert.throws(() => operationRow({ title: 'x', description: 'y', label: 'z', onClick() {} }), TypeError);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('sectionNav marks the active section and reports the chosen one', async () => {
|
||||
const { sectionNav } = await controlsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
try {
|
||||
const chosen = [];
|
||||
const nav = sectionNav({ label: 'System sections', sections: [['overview', 'Overview'], ['appearance', 'Appearance']],
|
||||
active: 'appearance', scope, onSelect: id => chosen.push(id) });
|
||||
assert.equal(nav.className, 'section-nav');
|
||||
assert.equal(nav.getAttribute('role'), 'group');
|
||||
assert.equal(nav.getAttribute('aria-label'), 'System sections');
|
||||
assert.deepEqual(nav.children.map(button => [button.textContent, button.getAttribute('aria-pressed'), button.dataset.value]),
|
||||
[['Overview', 'false', 'overview'], ['Appearance', 'true', 'appearance']]);
|
||||
nav.children[0].click();
|
||||
assert.deepEqual(chosen, ['overview']);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('disposing the scope leaves no listener behind', async () => {
|
||||
const { settingRow, operationRow, sectionNav } = await controlsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
try {
|
||||
settingRow({ id: 'a', label: 'A', description: 'a', control: { type: 'toggle' }, scope, onChange() {} });
|
||||
settingRow({ id: 'b', label: 'B', description: 'b', control: { type: 'select', options: ['Grid', 'List'] }, scope, onChange() {} });
|
||||
operationRow({ title: 'C', description: 'c', label: 'Go', scope, onClick() {} });
|
||||
sectionNav({ label: 'System sections', sections: [['overview', 'Overview']], active: 'overview', scope, onSelect() {} });
|
||||
assert.equal(dom.listenerCount() > 0, true);
|
||||
scope.dispose();
|
||||
assert.equal(dom.listenerCount(), 0);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,255 @@
|
||||
// System › Overview (#249 System S2 Task 6): the concept's Overview copy on real values (D93), the Health report rows
|
||||
// (D95), the Statistics tables (D96) and the health check run that waits for its read job.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const overviewModule = import('../../web/js/workspace/system-overview.mjs');
|
||||
|
||||
const OVERVIEW = Object.freeze({
|
||||
library: { positive_prompts: 1234, negative_variants: 7 },
|
||||
images: { indexed: 24, rated: 20 },
|
||||
health: null,
|
||||
backup: null,
|
||||
needs_review: { pending: null },
|
||||
sources: { configured: 2, effective: 2, auto_detected: false },
|
||||
});
|
||||
|
||||
test('metrics are the server counts and Not checked before any health check', async () => {
|
||||
const { metricValues } = await overviewModule;
|
||||
assert.deepEqual(metricValues(OVERVIEW), [
|
||||
[(1234).toLocaleString(), 'Positive prompts'], ['7', 'Negative variants'], ['24', 'Images'],
|
||||
['Not checked', 'Database integrity'],
|
||||
]);
|
||||
});
|
||||
|
||||
test('metric 4 is the last health result', async () => {
|
||||
const { integrityText } = await overviewModule;
|
||||
assert.equal(integrityText(null), 'Not checked');
|
||||
assert.equal(integrityText({ status: 'passed', issues: 0 }), 'Passed');
|
||||
assert.equal(integrityText({ status: 'issues', issues: 1 }), '1 issue');
|
||||
assert.equal(integrityText({ status: 'issues', issues: 3 }), '3 issues');
|
||||
assert.equal(integrityText({ status: 'incomplete', issues: 0 }), 'Incomplete');
|
||||
// Issues found beside a check that could not run (Needs review, Q6) are still named.
|
||||
assert.equal(integrityText({ status: 'incomplete', issues: 2 }), '2 issues');
|
||||
});
|
||||
|
||||
test('library health rows never invent a review count, a backup or thumbnails', async () => {
|
||||
const { ambiguousTitle, backupText } = await overviewModule;
|
||||
assert.equal(ambiguousTitle({ pending: null }), 'Ambiguous image links');
|
||||
assert.equal(ambiguousTitle(undefined), 'Ambiguous image links');
|
||||
assert.equal(ambiguousTitle({ pending: 1 }), '1 ambiguous image link');
|
||||
assert.equal(ambiguousTitle({ pending: 3 }), '3 ambiguous image links');
|
||||
assert.equal(backupText(null), 'No retained backup record.');
|
||||
assert.equal(backupText({ verified_at: 'not a time' }), 'No retained backup record.');
|
||||
assert.match(backupText({ verified_at: '2026-10-22T09:30:00Z' }), /^Last verified backup .+\.$/);
|
||||
});
|
||||
|
||||
test('the thumbnail row is its own read of the thumbnail status', async () => {
|
||||
const { thumbnailTitle, THUMBNAILS_LOADING } = await overviewModule;
|
||||
// Until `/images/thumbnails/status` answers the row says it is checking; a failed read is unavailable, never 0.
|
||||
assert.equal(THUMBNAILS_LOADING, 'Checking thumbnails…');
|
||||
assert.equal(thumbnailTitle(undefined), THUMBNAILS_LOADING);
|
||||
assert.equal(thumbnailTitle({ pending: 0 }), 'No missing thumbnails');
|
||||
assert.equal(thumbnailTitle({ pending: 1 }), '1 missing thumbnail');
|
||||
assert.equal(thumbnailTitle({ pending: 12 }), '12 missing thumbnails');
|
||||
assert.equal(thumbnailTitle(null), 'Thumbnail status unavailable');
|
||||
assert.equal(thumbnailTitle({ pending: 'x' }), 'Thumbnail status unavailable');
|
||||
// With no image source there is nothing to count.
|
||||
assert.equal(thumbnailTitle({ pending: 0 }, { effective: 0 }), 'Thumbnail status unavailable');
|
||||
assert.equal(thumbnailTitle({ pending: 2 }, { effective: 1 }), '2 missing thumbnails');
|
||||
});
|
||||
|
||||
test('an integrity count SQLite stopped at reads as a lower bound', async () => {
|
||||
const { checkChip } = await overviewModule;
|
||||
assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 100, capped: true }), '100+ problems found');
|
||||
assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 37, capped: false }), '37 problems found');
|
||||
assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 1 }), '1 problem found');
|
||||
});
|
||||
|
||||
test('the sources row tells configured, available and detected sources apart', async () => {
|
||||
const { sourcesCopy } = await overviewModule;
|
||||
const shared = 'Monitoring and enrichment share one source list.';
|
||||
assert.deepEqual(sourcesCopy({ configured: 2, effective: 2, auto_detected: false }), ['2 output sources', shared]);
|
||||
assert.deepEqual(sourcesCopy({ configured: 2, effective: 1, auto_detected: false }),
|
||||
['1 output source', `1 of 2 configured sources available. ${shared}`]);
|
||||
assert.deepEqual(sourcesCopy({ configured: 0, effective: 1, auto_detected: true }),
|
||||
['1 output source', `None configured is available; ComfyUI's output folder is used. ${shared}`]);
|
||||
});
|
||||
|
||||
const REPORT = Object.freeze({
|
||||
checked_at: '2026-10-23T08:00:00Z',
|
||||
status: 'issues',
|
||||
checks: [
|
||||
{ id: 'integrity', label: 'Integrity check', status: 'passed', count: 0 },
|
||||
{ id: 'foreign_keys', label: 'Foreign keys & pair references', status: 'issues', count: 2 },
|
||||
{ id: 'canonical_hashes', label: 'Canonical hashes', status: 'issues', count: 1 },
|
||||
{ id: 'pair_uniqueness', label: 'Pair uniqueness', status: 'failed', count: null },
|
||||
{ id: 'missing_files', label: 'Missing files', status: 'issues', count: 2,
|
||||
sources: [{ name: 'output', status: 'checked', count: 2 }] },
|
||||
{ id: 'ambiguous_metadata', label: 'Ambiguous metadata', status: 'unavailable', count: null },
|
||||
{ id: 'consistency', label: 'Consistency', status: 'issues', count: null },
|
||||
],
|
||||
});
|
||||
|
||||
test('health rows give each check a chip from its real status, in the server order', async () => {
|
||||
const { healthRows } = await overviewModule;
|
||||
assert.deepEqual(healthRows(REPORT), [
|
||||
['Integrity check', 'Passed'],
|
||||
['Foreign keys & pair references', '2 broken references'],
|
||||
['Canonical hashes', '1 hash does not match its text'],
|
||||
['Pair uniqueness', 'Could not check'],
|
||||
['Missing files', '2 links need review'],
|
||||
['Ambiguous metadata', 'Not available yet'],
|
||||
['Consistency', 'Issues found'],
|
||||
]);
|
||||
});
|
||||
|
||||
test('missing files is reported per image source when there are several', async () => {
|
||||
const { healthRows } = await overviewModule;
|
||||
const report = { ...REPORT, checks: [{ id: 'missing_files', label: 'Missing files', status: 'issues', count: 1, partial: true,
|
||||
sources: [{ name: 'output', status: 'checked', count: 1 }, { name: 'input', status: 'checked', count: 0 },
|
||||
{ name: 'archive', status: 'unavailable', count: null }] }] };
|
||||
assert.deepEqual(healthRows(report), [
|
||||
['Missing files · output', '1 link needs review'],
|
||||
['Missing files · input', 'Passed'],
|
||||
['Missing files · archive', 'Not available'],
|
||||
]);
|
||||
const unavailable = { ...REPORT, checks: [{ id: 'missing_files', label: 'Missing files', status: 'unavailable', count: null, sources: [] }] };
|
||||
assert.deepEqual(healthRows(unavailable), [['Missing files', 'Not available']]);
|
||||
});
|
||||
|
||||
test('the report summary says when it was checked and whether it is complete', async () => {
|
||||
const { healthSummary } = await overviewModule;
|
||||
const when = new Date(REPORT.checked_at).toLocaleString();
|
||||
assert.equal(healthSummary({ ...REPORT, status: 'passed' }), `Checked ${when}. Every check passed.`);
|
||||
assert.equal(healthSummary({ ...REPORT, status: 'incomplete' }), `Checked ${when}. Some checks could not run, so this report is incomplete.`);
|
||||
assert.equal(healthSummary({ ...REPORT, checked_at: 'x' }), 'Nothing was changed; review the checks with issues.');
|
||||
});
|
||||
|
||||
test('statistics rows are the real runs and ratings, then the library totals', async () => {
|
||||
const { categoryRows, totalsRows } = await overviewModule;
|
||||
const statistics = {
|
||||
by_category: [{ category: 'Landscapes', runs: 1200, average_rating: 4.5 },
|
||||
{ category: null, runs: 3, average_rating: null }],
|
||||
totals: { total_prompts: 12, total_tags: 15, total_images: 24 },
|
||||
rated_images: 20,
|
||||
};
|
||||
assert.deepEqual(categoryRows(statistics), [['Landscapes', (1200).toLocaleString(), '4.5 / 5'], ['Uncategorized', '3', 'Not rated']]);
|
||||
assert.deepEqual(totalsRows(statistics), [['Prompts', '12'], ['Tags', '15'], ['Images', '24'], ['Rated images', '20']]);
|
||||
});
|
||||
|
||||
/** A fake api answering each `get`/`post` from a queue per URL, recording the calls. */
|
||||
function fakeApi(answers) {
|
||||
const calls = [];
|
||||
const answer = (method, url) => {
|
||||
calls.push(`${method} ${url}`);
|
||||
const queue = answers[`${method} ${url}`];
|
||||
if (!queue?.length) throw new Error(`unexpected ${method} ${url}`);
|
||||
const next = queue.shift();
|
||||
return next instanceof Error ? Promise.reject(next) : Promise.resolve(next);
|
||||
};
|
||||
return { calls, get: url => answer('GET', url), post: url => answer('POST', url) };
|
||||
}
|
||||
|
||||
const JOB = '0123456789abcdef0123456789abcdef';
|
||||
const job = (state, id = JOB, kind = 'health') => ({ id, kind, state });
|
||||
|
||||
/**
|
||||
* A fake Activity store holding `jobs`: `track(job)` lists a started job, `load()` applies the next of `reads` (the
|
||||
* server's job lists) and `push(job)` is a stream update; each notifies as the real store does. It never polls.
|
||||
*/
|
||||
function fakeActivity({ jobs = [], reads = [] } = {}) {
|
||||
const listeners = new Set();
|
||||
const emit = () => { for (const listener of [...listeners]) listener(); };
|
||||
const store = {
|
||||
loads: 0,
|
||||
find: id => jobs.find(item => item.id === id),
|
||||
running: () => jobs.filter(item => item.state === 'running'),
|
||||
subscribe(listener) { listeners.add(listener); return () => listeners.delete(listener); },
|
||||
listeners: () => listeners.size,
|
||||
async load() {
|
||||
store.loads += 1;
|
||||
if (reads.length) jobs = reads.shift();
|
||||
emit();
|
||||
},
|
||||
track(item) {
|
||||
if (!item?.id) throw new TypeError('Invalid job');
|
||||
store.push(item);
|
||||
return item;
|
||||
},
|
||||
push(item) {
|
||||
jobs = [...jobs.filter(other => other.id !== item.id), item];
|
||||
emit();
|
||||
},
|
||||
};
|
||||
return store;
|
||||
}
|
||||
|
||||
test('a health check tracks its job in the Activity store, then reads the report', async () => {
|
||||
const { runHealthCheck } = await overviewModule;
|
||||
const api = fakeApi({
|
||||
'POST /prompt_manager/system/health': [{ job: job('running') }],
|
||||
'GET /prompt_manager/system/health': [{ report: REPORT }],
|
||||
});
|
||||
const activity = fakeActivity();
|
||||
const checking = runHealthCheck(api, { activity });
|
||||
await new Promise(resolve => setImmediate(resolve));
|
||||
// The store follows the started job's event stream; nothing reads or polls the job here.
|
||||
assert.equal(activity.loads, 0);
|
||||
assert.equal(activity.find(JOB).state, 'running');
|
||||
assert.deepEqual(api.calls, ['POST /prompt_manager/system/health']);
|
||||
activity.push(job('finished'));
|
||||
assert.equal(await checking, REPORT);
|
||||
assert.deepEqual(api.calls, ['POST /prompt_manager/system/health', 'GET /prompt_manager/system/health']);
|
||||
assert.equal(activity.listeners(), 0);
|
||||
});
|
||||
|
||||
test('a check already running elsewhere is waited for, not refused', async () => {
|
||||
const { runHealthCheck } = await overviewModule;
|
||||
const busy = Object.assign(new Error('busy'), { status: 409, code: 'busy' });
|
||||
const api = fakeApi({
|
||||
'POST /prompt_manager/system/health': [busy],
|
||||
'GET /prompt_manager/system/health': [{ report: REPORT }],
|
||||
});
|
||||
const activity = fakeActivity({ reads: [[job('running', 'f'.repeat(32), 'scan'), job('running')]] });
|
||||
const checking = runHealthCheck(api, { activity });
|
||||
await new Promise(resolve => setImmediate(resolve));
|
||||
assert.equal(activity.loads, 1);
|
||||
activity.push(job('finished'));
|
||||
assert.equal(await checking, REPORT);
|
||||
// The other check ended between the refusal and the read: its report is the latest.
|
||||
const ended = fakeApi({ 'POST /prompt_manager/system/health': [busy], 'GET /prompt_manager/system/health': [{ report: REPORT }] });
|
||||
assert.equal(await runHealthCheck(ended, { activity: fakeActivity({ reads: [[]] }) }), REPORT);
|
||||
});
|
||||
|
||||
test('a failed, lost or reportless check rejects; other errors pass through', async () => {
|
||||
const { runHealthCheck, waitForJob, HEALTH_FAILED_TEXT } = await overviewModule;
|
||||
const failed = fakeApi({ 'POST /prompt_manager/system/health': [{ job: job('failed') }] });
|
||||
await assert.rejects(runHealthCheck(failed, { activity: fakeActivity() }), { message: HEALTH_FAILED_TEXT });
|
||||
// The store marks a job lost when the server no longer lists it (ComfyUI restarted).
|
||||
const restarted = fakeActivity({ jobs: [job('running')] });
|
||||
const waiting = waitForJob(restarted, JOB);
|
||||
restarted.push(job('lost'));
|
||||
assert.equal(await waiting, 'lost');
|
||||
const empty = fakeApi({ 'POST /prompt_manager/system/health': [{ job: job('finished') }],
|
||||
'GET /prompt_manager/system/health': [{ report: null }] });
|
||||
await assert.rejects(runHealthCheck(empty, { activity: fakeActivity() }), { message: HEALTH_FAILED_TEXT });
|
||||
const broken = fakeApi({ 'POST /prompt_manager/system/health': [Object.assign(new Error('server'), { status: 500 })] });
|
||||
await assert.rejects(runHealthCheck(broken, { activity: fakeActivity() }), { message: 'server' });
|
||||
const invalid = fakeApi({ 'POST /prompt_manager/system/health': [{ job: null }] });
|
||||
await assert.rejects(runHealthCheck(invalid, { activity: fakeActivity() }), { message: 'Invalid job' });
|
||||
});
|
||||
|
||||
test('waiting stops when its signal aborts', async () => {
|
||||
const { waitForJob } = await overviewModule;
|
||||
const controller = new AbortController();
|
||||
const activity = fakeActivity({ jobs: [job('running')] });
|
||||
const waiting = waitForJob(activity, JOB, { signal: controller.signal });
|
||||
await new Promise(resolve => setImmediate(resolve));
|
||||
controller.abort(new Error('left'));
|
||||
await assert.rejects(waiting, { message: 'left' });
|
||||
assert.equal(activity.listeners(), 0);
|
||||
const gone = new AbortController();
|
||||
gone.abort(new Error('already'));
|
||||
await assert.rejects(waitForJob(fakeActivity(), JOB, { signal: gone.signal }), { message: 'already' });
|
||||
});
|
||||
@@ -0,0 +1,145 @@
|
||||
// System › Sources & scans (#249 System S4 Task 11): the concept's "Image sources" list (app.js 679–681) built from the
|
||||
// server's configured, effective and watched sources (GET /settings), the host draft of `gallery_root_paths`, the
|
||||
// 3.2.0 update notice's dismissal and the Refresh files toast (D98).
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const sourcesModule = import('../../web/js/workspace/system-sources.mjs');
|
||||
|
||||
const STATUS = Object.freeze({
|
||||
configured: [
|
||||
{ label: 'ComfyUI output', path: 'output', valid: true, code: null, reason: null },
|
||||
{ label: 'ComfyUI input', path: 'input', valid: true, code: null, reason: null },
|
||||
],
|
||||
effective: [
|
||||
{ label: 'ComfyUI output', path: 'output', root_index: 0, watch: 'watching' },
|
||||
{ label: 'ComfyUI input', path: 'input', root_index: 1, watch: 'manual' },
|
||||
],
|
||||
auto_detected: false,
|
||||
});
|
||||
|
||||
test('each effective source is the concept row: label, "<path> · watching|manual scan", Scan changes or Scan folder', async () => {
|
||||
const { sourceRows } = await sourcesModule;
|
||||
const saved = ['output', 'input'];
|
||||
const { rows, note } = sourceRows({ sources: STATUS, saved, current: saved });
|
||||
assert.equal(note, null);
|
||||
assert.deepEqual(rows, [
|
||||
{ key: 'output', title: 'ComfyUI output', description: 'output · watching', scan: { label: 'Scan changes', root: 0, path: 'output' },
|
||||
remove: 'output' },
|
||||
{ key: 'input', title: 'ComfyUI input', description: 'input · manual scan', scan: { label: 'Scan folder', root: 1, path: 'input' },
|
||||
remove: 'input' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('watching only with a live observer; restart pending otherwise (D98)', async () => {
|
||||
const { sourceRows, WATCH_COPY } = await sourcesModule;
|
||||
assert.deepEqual(WATCH_COPY, { watching: 'watching', stopping: 'watching until restart', manual: 'manual scan',
|
||||
restart_pending: 'restart pending' });
|
||||
const sources = { ...STATUS, effective: STATUS.effective.map(source => ({ ...source, watch: 'restart_pending' })) };
|
||||
const { rows } = sourceRows({ sources, saved: ['output', 'input'], current: ['output', 'input'] });
|
||||
assert.deepEqual(rows.map(row => [row.description, row.scan.label]),
|
||||
[['output · restart pending', 'Scan folder'], ['input · restart pending', 'Scan folder']]);
|
||||
});
|
||||
|
||||
test('a configured source that cannot be used shows the server reason and only Remove', async () => {
|
||||
const { sourceRows } = await sourcesModule;
|
||||
const sources = { ...STATUS, configured: [...STATUS.configured,
|
||||
{ label: 'unmounted-drive', path: 'unmounted-drive', valid: false, code: 'not-found', reason: 'Gallery root does not exist' }] };
|
||||
const saved = ['output', 'input', 'unmounted-drive'];
|
||||
const { rows } = sourceRows({ sources, saved, current: saved });
|
||||
assert.deepEqual(rows.at(-1), { key: 'unmounted-drive', title: 'unmounted-drive',
|
||||
description: 'unmounted-drive · Gallery root does not exist', scan: null, remove: 'unmounted-drive' });
|
||||
});
|
||||
|
||||
test('an added source is listed as not saved yet with its validated label; a removed one leaves the list', async () => {
|
||||
const { sourceRows } = await sourcesModule;
|
||||
const labels = new Map([['output/archive', 'archive']]);
|
||||
const { rows, note } = sourceRows({ sources: STATUS, saved: ['output', 'input'], current: ['input', 'output/archive'], labels });
|
||||
assert.equal(note, null);
|
||||
assert.deepEqual(rows.map(row => [row.title, row.description, row.scan?.label ?? null, row.remove]), [
|
||||
['ComfyUI input', 'input · manual scan', 'Scan folder', 'input'],
|
||||
['archive', 'output/archive · not saved yet', null, 'output/archive'],
|
||||
]);
|
||||
// Without a recorded label (a reload of System), the folder name stands in.
|
||||
assert.equal(sourceRows({ sources: STATUS, saved: [], current: ['output/new-collection'] }).rows[0].title, 'new-collection');
|
||||
});
|
||||
|
||||
test('removing the last entry states auto-detect (D98)', async () => {
|
||||
const { sourceRows, NOTES } = await sourcesModule;
|
||||
const { rows, note } = sourceRows({ sources: STATUS, saved: ['output', 'input'], current: [] });
|
||||
assert.deepEqual(rows, []);
|
||||
assert.equal(note, NOTES.removedAll);
|
||||
assert.equal(NOTES.removedAll, 'No source is configured. After saving, ComfyUI’s output folder is used.');
|
||||
});
|
||||
|
||||
test('auto-detected sources are listed without Remove, with the line that says how they were chosen', async () => {
|
||||
const { sourceRows, NOTES } = await sourcesModule;
|
||||
const auto = { configured: [], effective: [{ label: 'ComfyUI output', path: 'output', root_index: 0, watch: 'manual' }], auto_detected: true };
|
||||
assert.deepEqual(sourceRows({ sources: auto, saved: [], current: [] }), { rows: [
|
||||
{ key: 'auto:0', title: 'ComfyUI output', description: 'output · manual scan', scan: { label: 'Scan folder', root: 0, path: 'output' },
|
||||
remove: null },
|
||||
], note: NOTES.autoDetected });
|
||||
// Every configured source unusable: ComfyUI's output folder is used instead.
|
||||
const fallback = { ...auto, configured: [{ label: 'gone', path: 'gone', valid: false, code: 'not-found', reason: 'Gallery root does not exist' }] };
|
||||
const result = sourceRows({ sources: fallback, saved: ['gone'], current: ['gone'] });
|
||||
assert.deepEqual(result.rows.map(row => [row.title, row.remove]), [['gone', 'gone'], ['ComfyUI output', null]]);
|
||||
assert.equal(result.note, NOTES.fallback);
|
||||
assert.equal(sourceRows({ sources: { configured: [], effective: [], auto_detected: false }, saved: [], current: [] }).note, NOTES.none);
|
||||
});
|
||||
|
||||
test('a path listed twice in the draft is one row', async () => {
|
||||
const { sourceRows } = await sourcesModule;
|
||||
assert.equal(sourceRows({ sources: STATUS, saved: ['output', 'input'], current: ['output', 'output'] }).rows.length, 1);
|
||||
});
|
||||
|
||||
test('roots equal to the saved list are the saved list itself, so the draft is clean again', async () => {
|
||||
const { draftRoots } = await sourcesModule;
|
||||
const saved = ['output', 'input'];
|
||||
assert.equal(draftRoots(['output', 'input'], saved), saved);
|
||||
assert.deepEqual(draftRoots(['input'], saved), ['input']);
|
||||
assert.notEqual(draftRoots(['input', 'output'], saved), saved);
|
||||
assert.deepEqual(draftRoots([], undefined), []);
|
||||
});
|
||||
|
||||
test('notice Later persists the 3.2.0 dismissal (pm_update_notice_dismissed); storage that throws keeps it for this view', async () => {
|
||||
const { noticeDismissed, dismissNotice, NOTICE_KEY, NOTICE_VERSION } = await sourcesModule;
|
||||
assert.deepEqual([NOTICE_KEY, NOTICE_VERSION], ['pm_update_notice_dismissed', '3.2.0']);
|
||||
const values = new Map();
|
||||
const storage = { getItem: key => values.get(key) ?? null, setItem: (key, value) => values.set(key, String(value)) };
|
||||
assert.equal(noticeDismissed(() => storage), false);
|
||||
values.set(NOTICE_KEY, '3.1.0');
|
||||
assert.equal(noticeDismissed(() => storage), false);
|
||||
assert.equal(dismissNotice(() => storage), true);
|
||||
assert.equal(values.get(NOTICE_KEY), '3.2.0');
|
||||
assert.equal(noticeDismissed(() => storage), true);
|
||||
const denied = () => { throw new Error('SecurityError'); };
|
||||
assert.equal(noticeDismissed(denied), false);
|
||||
assert.equal(dismissNotice(denied), false);
|
||||
assert.equal(dismissNotice(() => ({ setItem() { throw new Error('QuotaExceededError'); } })), false);
|
||||
});
|
||||
|
||||
test('Refresh files toasts the rescan counts, or says it finished when they are missing', async () => {
|
||||
const { refreshedText } = await sourcesModule;
|
||||
assert.equal(refreshedText({ total: 1234, roots: 2 }), 'Files refreshed. 1,234 files in 2 output folders.');
|
||||
assert.equal(refreshedText({ total: 1, roots: 1 }), 'Files refreshed. 1 file in 1 output folder.');
|
||||
assert.equal(refreshedText({ total: -1, roots: 1 }), 'Files refreshed.');
|
||||
assert.equal(refreshedText(null), 'Files refreshed.');
|
||||
});
|
||||
|
||||
test('monitoring switched off: a source still watched says so until the restart (D128)', async () => {
|
||||
const { sourceRows } = await sourcesModule;
|
||||
const sources = { ...STATUS, effective: [{ ...STATUS.effective[0], watch: 'stopping' }, STATUS.effective[1]] };
|
||||
const { rows } = sourceRows({ sources, saved: ['output', 'input'], current: ['output', 'input'] });
|
||||
assert.deepEqual(rows.map(row => [row.description, row.scan.label]),
|
||||
[['output · watching until restart', 'Scan changes'], ['input · manual scan', 'Scan folder']]);
|
||||
});
|
||||
|
||||
test('a valid configured source missing from the effective list is another spelling of a listed folder (D128)', async () => {
|
||||
const { sourceRows, DUPLICATE_SOURCE } = await sourcesModule;
|
||||
assert.equal(DUPLICATE_SOURCE, 'same folder as another source');
|
||||
const sources = { ...STATUS, configured: [...STATUS.configured,
|
||||
{ label: 'output', path: 'output/.', valid: true, code: null, reason: null }] };
|
||||
const saved = ['output', 'input', 'output/.'];
|
||||
assert.deepEqual(sourceRows({ sources, saved, current: saved }).rows.at(-1), { key: 'output/.', title: 'output',
|
||||
description: 'output/. · same folder as another source', scan: null, remove: 'output/.' });
|
||||
});
|
||||
@@ -0,0 +1,366 @@
|
||||
// System draft store (#249 System S2 Task 5): one Save row for browser, host and runtime fields (concept app.js
|
||||
// renderSystem `.save-row` 705, `save-settings` 1320). It saves browser, then host with the GET revision, then runtime;
|
||||
// a domain that fails keeps its fields dirty, and the status copy (D92) says what really happened.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const source = import('../../web/js/workspace/system-state.mjs');
|
||||
|
||||
/** Adapters that record every call; `host` answers like POST /prompt_manager/settings. */
|
||||
function adapters({ browser = {}, host = {}, runtime = {}, revision = 'sha256:one' } = {}) {
|
||||
const calls = [];
|
||||
const outcome = { browser: null, host: null, runtime: null };
|
||||
return {
|
||||
calls,
|
||||
outcome,
|
||||
readBrowser: () => { calls.push(['readBrowser']); return { ...browser }; },
|
||||
readHost: async () => { calls.push(['readHost']); return { values: { ...host }, revision }; },
|
||||
readRuntime: async () => { calls.push(['readRuntime']); return { ...runtime }; },
|
||||
writeBrowser: async changes => {
|
||||
calls.push(['writeBrowser', changes]);
|
||||
if (outcome.browser) return outcome.browser(changes);
|
||||
return { persisted: true };
|
||||
},
|
||||
writeHost: async (changes, options) => {
|
||||
calls.push(['writeHost', changes, options]);
|
||||
if (outcome.host) return outcome.host(changes, options);
|
||||
return { success: true, persisted: true, revision: 'sha256:two', runtime_applied: true };
|
||||
},
|
||||
writeRuntime: async changes => {
|
||||
calls.push(['writeRuntime', changes]);
|
||||
if (outcome.runtime) return outcome.runtime(changes);
|
||||
return { success: true };
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function loaded(domains) {
|
||||
const { createSystemDrafts } = await source;
|
||||
const drafts = createSystemDrafts(domains);
|
||||
await Promise.all(['browser', 'host', 'runtime'].map(domain => drafts.load(domain)));
|
||||
domains.calls.length = 0;
|
||||
return drafts;
|
||||
}
|
||||
|
||||
test('sections follow the concept order and the production hashes', async () => {
|
||||
const { SYSTEM_SECTIONS, isSystemSection } = await source;
|
||||
assert.deepEqual(SYSTEM_SECTIONS.map(([id, label]) => [id, label]), [
|
||||
['overview', 'Overview'], ['appearance', 'Appearance'], ['sources', 'Sources & scans'],
|
||||
['integrations', 'Integrations'], ['tagging', 'Auto-tagging'], ['maintenance', 'Maintenance'],
|
||||
['backups', 'Backup & export'], ['diagnostics', 'Diagnostics & logs'],
|
||||
]);
|
||||
assert.equal(isSystemSection('backups'), true);
|
||||
for (const value of ['general', '', 'constructor', '__proto__', undefined]) assert.equal(isSystemSection(value), false, value);
|
||||
});
|
||||
|
||||
test('drafts read baselines and report dirty only for changed values', async () => {
|
||||
const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 } });
|
||||
const drafts = await loaded(domains);
|
||||
assert.equal(drafts.get('browser', 'layout'), 'grid');
|
||||
assert.equal(drafts.get('host', 'result_timeout'), 30);
|
||||
assert.equal(drafts.dirty(), false);
|
||||
drafts.set('host', 'result_timeout', 45);
|
||||
assert.equal(drafts.get('host', 'result_timeout'), 45);
|
||||
assert.equal(drafts.dirty(), true);
|
||||
assert.equal(drafts.dirty('host'), true);
|
||||
assert.equal(drafts.dirty('browser'), false);
|
||||
// Setting the saved value back is clean again.
|
||||
drafts.set('host', 'result_timeout', 30);
|
||||
assert.equal(drafts.dirty(), false);
|
||||
assert.throws(() => drafts.set('elsewhere', 'x', 1), TypeError);
|
||||
assert.throws(() => drafts.get('elsewhere', 'x'), TypeError);
|
||||
});
|
||||
|
||||
test('save commits browser then host then runtime', async () => {
|
||||
const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 }, runtime: { level: 'INFO' } });
|
||||
const drafts = await loaded(domains);
|
||||
drafts.set('runtime', 'level', 'DEBUG');
|
||||
drafts.set('host', 'result_timeout', 45);
|
||||
drafts.set('browser', 'layout', 'list');
|
||||
const outcome = await drafts.save();
|
||||
assert.deepEqual(outcome, { browser: 'saved', host: 'saved', runtime: 'applied' });
|
||||
assert.deepEqual(domains.calls, [
|
||||
['writeBrowser', { layout: 'list' }],
|
||||
['writeHost', { result_timeout: 45 }, { expected_revision: 'sha256:one' }],
|
||||
['writeRuntime', { level: 'DEBUG' }],
|
||||
]);
|
||||
assert.equal(drafts.dirty(), false);
|
||||
// The next host save sends the revision the last one returned.
|
||||
drafts.set('host', 'result_timeout', 60);
|
||||
await drafts.save();
|
||||
assert.deepEqual(domains.calls.at(-1), ['writeHost', { result_timeout: 60 }, { expected_revision: 'sha256:two' }]);
|
||||
});
|
||||
|
||||
test('host failure keeps host fields dirty and browser saved', async () => {
|
||||
const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 } });
|
||||
const drafts = await loaded(domains);
|
||||
domains.outcome.host = () => { throw Object.assign(new Error('Settings could not be written to config.json.'), { code: 'config-write-failed' }); };
|
||||
drafts.set('browser', 'layout', 'list');
|
||||
drafts.set('host', 'result_timeout', 45);
|
||||
const outcome = await drafts.save();
|
||||
assert.deepEqual(outcome, { browser: 'saved', host: 'failed', runtime: 'clean' });
|
||||
assert.equal(drafts.dirty('browser'), false);
|
||||
assert.equal(drafts.dirty('host'), true);
|
||||
assert.equal(drafts.get('host', 'result_timeout'), 45);
|
||||
assert.deepEqual([...drafts.failed('host')], ['result_timeout']);
|
||||
assert.deepEqual([...drafts.failed('browser')], []);
|
||||
assert.equal(drafts.statusText(), 'Some changes were not saved. Review the highlighted settings.');
|
||||
// A save answered `persisted: false` without an error is not saved either.
|
||||
domains.outcome.host = () => ({ success: false, persisted: false });
|
||||
assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'failed', runtime: 'clean' });
|
||||
// Editing the failed field clears its highlight and reads as unsaved again.
|
||||
drafts.set('host', 'result_timeout', 50);
|
||||
assert.deepEqual([...drafts.failed('host')], []);
|
||||
assert.equal(drafts.statusText(), 'Unsaved changes.');
|
||||
});
|
||||
|
||||
test('stale revision reports conflict and keeps the draft', async () => {
|
||||
const domains = adapters({ host: { result_timeout: 30 } });
|
||||
const drafts = await loaded(domains);
|
||||
domains.outcome.host = () => { throw Object.assign(new Error('Settings changed since they were loaded.'), { code: 'settings-changed' }); };
|
||||
drafts.set('host', 'result_timeout', 45);
|
||||
assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'conflict', runtime: 'clean' });
|
||||
assert.equal(drafts.dirty('host'), true);
|
||||
assert.equal(drafts.get('host', 'result_timeout'), 45);
|
||||
assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.');
|
||||
// The conflict read the host values again (their new revision), keeping the draft.
|
||||
assert.deepEqual(domains.calls.slice(-2).map(([name]) => name), ['writeHost', 'readHost']);
|
||||
// Reloading the section reads the new revision and keeps the draft (Q10).
|
||||
domains.outcome.host = null;
|
||||
await drafts.load('host');
|
||||
assert.equal(drafts.get('host', 'result_timeout'), 45);
|
||||
await drafts.save();
|
||||
assert.deepEqual(domains.calls.at(-1)[2], { expected_revision: 'sha256:one' });
|
||||
});
|
||||
|
||||
test('host save without a read revision never posts', async () => {
|
||||
const { createSystemDrafts } = await source;
|
||||
// Never read: an edit made before the host values (and their revision) arrived must not merge blindly.
|
||||
const unread = adapters({ host: { result_timeout: 30 } });
|
||||
const drafts = createSystemDrafts(unread);
|
||||
drafts.set('host', 'result_timeout', 45);
|
||||
assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'failed', runtime: 'clean' });
|
||||
assert.deepEqual(unread.calls, []);
|
||||
assert.equal(drafts.dirty('host'), true);
|
||||
assert.deepEqual([...drafts.failed('host')], ['result_timeout']);
|
||||
// A read of an unreadable configuration has no revision either.
|
||||
const unreadable = adapters({ host: { result_timeout: 30 }, revision: null });
|
||||
const blind = createSystemDrafts(unreadable);
|
||||
await blind.load('host');
|
||||
blind.set('host', 'result_timeout', 45);
|
||||
assert.equal((await blind.save()).host, 'failed');
|
||||
assert.equal(unreadable.calls.filter(([name]) => name === 'writeHost').length, 0);
|
||||
// Once read, the same draft saves with the revision.
|
||||
await drafts.load('host');
|
||||
assert.equal((await drafts.save()).host, 'saved');
|
||||
assert.deepEqual(unread.calls.at(-1), ['writeHost', { result_timeout: 45 }, { expected_revision: 'sha256:one' }]);
|
||||
});
|
||||
|
||||
test('reloadHost reads the host values and says whether it could', async () => {
|
||||
const { createSystemDrafts } = await source;
|
||||
const domains = adapters({ host: { result_timeout: 30 } });
|
||||
const drafts = createSystemDrafts(domains);
|
||||
assert.equal(await drafts.reloadHost(), true);
|
||||
assert.equal(drafts.get('host', 'result_timeout'), 30);
|
||||
const broken = createSystemDrafts({ ...domains, readHost: async () => { throw new Error('offline'); } });
|
||||
assert.equal(await broken.reloadHost(), false);
|
||||
broken.set('host', 'result_timeout', 45);
|
||||
assert.equal((await broken.save()).host, 'failed');
|
||||
});
|
||||
|
||||
test('no domain called when clean', async () => {
|
||||
const domains = adapters({ browser: { layout: 'grid' } });
|
||||
const drafts = await loaded(domains);
|
||||
assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'clean', runtime: 'clean' });
|
||||
assert.deepEqual(domains.calls, []);
|
||||
drafts.set('browser', 'layout', 'list');
|
||||
drafts.discard();
|
||||
assert.equal(drafts.dirty(), false);
|
||||
assert.equal(drafts.get('browser', 'layout'), 'grid');
|
||||
assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'clean', runtime: 'clean' });
|
||||
assert.deepEqual(domains.calls, []);
|
||||
});
|
||||
|
||||
test('status text per outcome', async () => {
|
||||
const { statusText, SETTINGS_STATUS } = await source;
|
||||
assert.deepEqual(SETTINGS_STATUS, {
|
||||
clean: 'No unsaved changes.',
|
||||
dirty: 'Unsaved changes.',
|
||||
browser: 'Saved in this browser.',
|
||||
host: 'Saved to the server configuration.',
|
||||
runtime: 'Applied until the server restarts.',
|
||||
partial: 'Some changes were not saved. Review the highlighted settings.',
|
||||
tabOnly: 'Saved for this tab only; browser storage is unavailable.',
|
||||
conflict: 'Settings changed elsewhere. Reload this section to see them.',
|
||||
restart: 'Restart ComfyUI to apply:',
|
||||
});
|
||||
const clean = { browser: 'clean', host: 'clean', runtime: 'clean' };
|
||||
assert.equal(statusText(clean), 'No unsaved changes.');
|
||||
assert.equal(statusText({ ...clean, browser: 'saved' }), 'Saved in this browser.');
|
||||
assert.equal(statusText({ ...clean, host: 'saved' }), 'Saved to the server configuration.');
|
||||
assert.equal(statusText({ ...clean, runtime: 'applied' }), 'Applied until the server restarts.');
|
||||
assert.equal(statusText({ ...clean, browser: 'tab-only' }), 'Saved for this tab only; browser storage is unavailable.');
|
||||
assert.equal(statusText({ browser: 'saved', host: 'saved', runtime: 'applied' }),
|
||||
'Saved in this browser. Saved to the server configuration. Applied until the server restarts.');
|
||||
assert.equal(statusText({ ...clean, browser: 'saved', runtime: 'failed' }),
|
||||
'Some changes were not saved. Review the highlighted settings.');
|
||||
assert.equal(statusText({ browser: 'failed', host: 'conflict', runtime: 'clean' }),
|
||||
'Settings changed elsewhere. Reload this section to see them.');
|
||||
});
|
||||
|
||||
test('the store says unsaved, then the save outcome, and notifies subscribers', async () => {
|
||||
const domains = adapters({ browser: { layout: 'grid' } });
|
||||
const drafts = await loaded(domains);
|
||||
const seen = [];
|
||||
const unsubscribe = drafts.subscribe(() => seen.push(drafts.statusText()));
|
||||
assert.equal(drafts.statusText(), 'No unsaved changes.');
|
||||
drafts.set('browser', 'layout', 'list');
|
||||
domains.outcome.browser = () => ({ persisted: false });
|
||||
await drafts.save();
|
||||
assert.deepEqual(seen, ['Unsaved changes.', 'Saved for this tab only; browser storage is unavailable.']);
|
||||
unsubscribe();
|
||||
drafts.set('browser', 'layout', 'masonry');
|
||||
assert.equal(seen.length, 2);
|
||||
// The tab-only value is this tab's baseline now.
|
||||
drafts.set('browser', 'layout', 'list');
|
||||
assert.equal(drafts.dirty(), false);
|
||||
});
|
||||
|
||||
test('an edit made while a save runs stays dirty', async () => {
|
||||
const domains = adapters({ browser: { layout: 'grid' } });
|
||||
const drafts = await loaded(domains);
|
||||
let release;
|
||||
domains.outcome.browser = () => new Promise(resolve => { release = () => resolve({ persisted: true }); });
|
||||
drafts.set('browser', 'layout', 'list');
|
||||
const saving = drafts.save();
|
||||
// A second Save while one runs joins it instead of writing twice.
|
||||
assert.equal(drafts.save(), saving);
|
||||
assert.equal(drafts.saving(), true);
|
||||
await new Promise(resolve => setImmediate(resolve));
|
||||
drafts.set('browser', 'layout', 'masonry');
|
||||
release();
|
||||
assert.deepEqual(await saving, { browser: 'saved', host: 'clean', runtime: 'clean' });
|
||||
assert.equal(drafts.saving(), false);
|
||||
assert.equal(drafts.dirty('browser'), true);
|
||||
assert.equal(drafts.get('browser', 'layout'), 'masonry');
|
||||
assert.equal(domains.calls.filter(([name]) => name === 'writeBrowser').length, 1);
|
||||
});
|
||||
|
||||
test('a failed read keeps the store usable and drafts survive a reload (Q10)', async () => {
|
||||
const { createSystemDrafts } = await source;
|
||||
const domains = adapters({ browser: { layout: 'grid' } });
|
||||
const drafts = createSystemDrafts(domains);
|
||||
await drafts.load('browser');
|
||||
drafts.set('browser', 'layout', 'list');
|
||||
await drafts.load('browser');
|
||||
assert.equal(drafts.get('browser', 'layout'), 'list');
|
||||
assert.equal(drafts.dirty(), true);
|
||||
const failing = createSystemDrafts({ ...domains, readHost: async () => { throw new Error('offline'); } });
|
||||
await assert.rejects(failing.load('host'), /offline/);
|
||||
assert.equal(failing.dirty(), false);
|
||||
assert.throws(() => createSystemDrafts({ ...domains, writeHost: null }), TypeError);
|
||||
});
|
||||
|
||||
test('a rejected value drops its draft and is highlighted; a conflict elsewhere keeps the drafts', async () => {
|
||||
const domains = adapters({ browser: { videoMute: true, videoLoop: true }, host: { result_timeout: 5 } });
|
||||
const drafts = await loaded(domains);
|
||||
assert.deepEqual(drafts.saved('browser'), { videoMute: true, videoLoop: true });
|
||||
drafts.set('host', 'result_timeout', 12);
|
||||
assert.equal(drafts.dirty('host'), true);
|
||||
drafts.reject('host', 'result_timeout');
|
||||
assert.equal(drafts.dirty('host'), false);
|
||||
assert.deepEqual([...drafts.failed('host')], ['result_timeout']);
|
||||
assert.equal(drafts.get('host', 'result_timeout'), 5);
|
||||
// A valid value clears the highlight.
|
||||
drafts.set('host', 'result_timeout', 7);
|
||||
assert.deepEqual([...drafts.failed('host')], []);
|
||||
drafts.discard();
|
||||
drafts.set('browser', 'videoMute', false);
|
||||
drafts.conflict('browser', []);
|
||||
assert.equal(drafts.statusText(), 'Unsaved changes.');
|
||||
drafts.conflict('browser', ['videoMute']);
|
||||
assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.');
|
||||
assert.deepEqual([...drafts.failed('browser')], ['videoMute']);
|
||||
assert.equal(drafts.get('browser', 'videoMute'), false);
|
||||
assert.throws(() => drafts.reject('nowhere', 'x'), TypeError);
|
||||
});
|
||||
|
||||
test('a rejected value says why in the status until the next edit', async () => {
|
||||
const drafts = await loaded(adapters({ host: { result_timeout: 5 } }));
|
||||
drafts.reject('host', 'result_timeout');
|
||||
assert.equal(drafts.statusText(), 'Some changes were not saved. Review the highlighted settings.');
|
||||
drafts.set('host', 'result_timeout', 9);
|
||||
assert.equal(drafts.statusText(), 'Unsaved changes.');
|
||||
});
|
||||
|
||||
test('a browser write refused as changed elsewhere keeps the draft and reads the browser values again', async () => {
|
||||
const domains = adapters({ browser: { workspaceDensity: 'comfortable' } });
|
||||
const drafts = await loaded(domains);
|
||||
drafts.set('browser', 'workspaceDensity', 'compact');
|
||||
domains.outcome.browser = () => { throw Object.assign(new Error('changed'), { code: 'settings-changed' }); };
|
||||
assert.deepEqual(await drafts.save(), { browser: 'conflict', host: 'clean', runtime: 'clean' });
|
||||
assert.deepEqual(domains.calls.map(([name]) => name), ['writeBrowser', 'readBrowser']);
|
||||
assert.equal(drafts.get('browser', 'workspaceDensity'), 'compact');
|
||||
assert.deepEqual([...drafts.failed('browser')], ['workspaceDensity']);
|
||||
assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.');
|
||||
});
|
||||
|
||||
test('host read state says whether the server settings could be read', async () => {
|
||||
const { createSystemDrafts } = await source;
|
||||
const domains = adapters({ host: { result_timeout: 5 } });
|
||||
let fail = true;
|
||||
const drafts = createSystemDrafts({ ...domains, readHost: async () => { if (fail) throw new Error('offline'); return { values: {}, revision: 'r' }; } });
|
||||
assert.equal(drafts.hostState(), 'pending');
|
||||
assert.equal(await drafts.reloadHost(), false);
|
||||
assert.equal(drafts.hostState(), 'failed');
|
||||
fail = false;
|
||||
assert.equal(await drafts.reloadHost(), true);
|
||||
assert.equal(drafts.hostState(), 'loaded');
|
||||
});
|
||||
|
||||
test('a host save names the settings that apply after a restart (D128)', async () => {
|
||||
const { statusText, RESTART_LABELS } = await source;
|
||||
assert.deepEqual(RESTART_LABELS, { monitoring_enabled: 'Watch for new outputs' });
|
||||
const saved = { browser: 'clean', host: 'saved', runtime: 'clean' };
|
||||
assert.equal(statusText({ ...saved, restart: ['monitoring_enabled'] }),
|
||||
'Saved to the server configuration. Restart ComfyUI to apply: Watch for new outputs.');
|
||||
// A field without a row label (the sources say "restart pending" per row) adds nothing.
|
||||
assert.equal(statusText({ ...saved, restart: ['gallery_root_paths'] }), 'Saved to the server configuration.');
|
||||
assert.equal(statusText({ ...saved, restart: [] }), 'Saved to the server configuration.');
|
||||
|
||||
const domains = adapters({ host: { monitoring_enabled: true } });
|
||||
const drafts = await loaded(domains);
|
||||
domains.outcome.host = () => ({ success: true, persisted: true, revision: 'sha256:two',
|
||||
restart_required_fields: ['monitoring_enabled', 7] });
|
||||
drafts.set('host', 'monitoring_enabled', false);
|
||||
assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'saved', runtime: 'clean', restart: ['monitoring_enabled'] });
|
||||
assert.equal(drafts.statusText(), 'Saved to the server configuration. Restart ComfyUI to apply: Watch for new outputs.');
|
||||
// The next edit is unsaved again.
|
||||
drafts.set('host', 'monitoring_enabled', true);
|
||||
assert.equal(drafts.statusText(), 'Unsaved changes.');
|
||||
});
|
||||
|
||||
test('a host save refused for one source says the server reason and names it until the next edit', async () => {
|
||||
const domains = adapters({ host: { gallery_root_paths: ['output', 'output/gone'] } });
|
||||
const drafts = await loaded(domains);
|
||||
const message = "output/gone can't be used: Gallery root does not exist. Remove it or reconnect it, then save.";
|
||||
domains.outcome.host = () => { throw Object.assign(new Error(message), { code: 'invalid-source', source: 'output/gone' }); };
|
||||
drafts.set('host', 'gallery_root_paths', ['output/gone']);
|
||||
const outcome = await drafts.save();
|
||||
assert.deepEqual(outcome, { browser: 'clean', host: 'failed', runtime: 'clean',
|
||||
invalid: { source: 'output/gone', message } });
|
||||
assert.equal(drafts.statusText(), message);
|
||||
assert.equal(drafts.invalidSource(), 'output/gone');
|
||||
assert.deepEqual([...drafts.failed('host')], ['gallery_root_paths']);
|
||||
// Any other failure keeps the generic copy and names no source.
|
||||
domains.outcome.host = () => { throw Object.assign(new Error('busy'), { code: 'busy' }); };
|
||||
await drafts.save();
|
||||
assert.equal(drafts.statusText(), 'Some changes were not saved. Review the highlighted settings.');
|
||||
assert.equal(drafts.invalidSource(), null);
|
||||
domains.outcome.host = () => { throw Object.assign(new Error(message), { code: 'invalid-source' }); };
|
||||
await drafts.save();
|
||||
assert.equal(drafts.invalidSource(), null);
|
||||
drafts.set('host', 'gallery_root_paths', ['output']);
|
||||
assert.equal(drafts.invalidSource(), null);
|
||||
assert.equal(drafts.statusText(), 'Unsaved changes.');
|
||||
});
|
||||
@@ -76,6 +76,7 @@ function fakeDOM() {
|
||||
get activeElement() { return active ?? body; },
|
||||
createElement: tag => new Element(tag),
|
||||
createElementNS: (namespace, tag) => new Element(tag),
|
||||
createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }),
|
||||
};
|
||||
const all = (root, predicate, found = []) => {
|
||||
if (predicate(root)) found.push(root);
|
||||
@@ -521,6 +522,10 @@ test('copy fallback selects the text', async () => {
|
||||
assert.equal(note.className, 'small');
|
||||
assert.equal(note.textContent, 'Automatic clipboard access requires a secure browser context.');
|
||||
assert.throws(() => openCopyFallback(42, { scope }), TypeError);
|
||||
// LI-4 (D117): the settings summary names its own dialog; the select-to-copy body is the same.
|
||||
openCopyFallback('Seed: 7', { scope, title: 'Copy settings summary' });
|
||||
assert.equal(byId(dom, 'modal-title').textContent, 'Copy settings summary');
|
||||
assert.equal(byId(dom, 'copy-text').value, 'Seed: 7');
|
||||
} finally {
|
||||
closeWorkspaceDialog();
|
||||
scope.dispose();
|
||||
@@ -786,3 +791,145 @@ test('the Filters button shows the active refinement count', async () => {
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
// #249 System S2 Task 5: the scan review's dialog and backup box (D55) generalised for every record-changing review.
|
||||
const scanModule = import('../../web/js/workspace/activity-scan.mjs');
|
||||
const bodyShape = dom => byId(dom, 'modal-body').children.map(node => [node.tagName.toLowerCase(), node.className].filter(Boolean).join('.'));
|
||||
|
||||
test('backupNotice is the concept .notice with its copy and one Back up now action', async () => {
|
||||
const { backupNotice, SCAN_BACKUP_COPY } = await dialogsModule;
|
||||
const dom = fakeDOM();
|
||||
try {
|
||||
let calls = 0;
|
||||
const notice = backupNotice({ copy: SCAN_BACKUP_COPY, onBackup: () => { calls++; } });
|
||||
assert.equal(notice.className, 'notice');
|
||||
assert.deepEqual(tags(notice), ['div', 'div']);
|
||||
assert.deepEqual(tags(notice.children[0]), ['h3', 'p']);
|
||||
assert.equal(notice.children[0].children[0].textContent, 'Back up the database first (recommended)');
|
||||
assert.equal(notice.children[0].children[1].textContent,
|
||||
'Downloads a copy of the prompts database before the scan changes any records.');
|
||||
assert.equal(notice.children[1].className, 'row-actions');
|
||||
const action = button(notice, 'Back up now');
|
||||
action.click();
|
||||
assert.equal(calls, 1);
|
||||
assert.throws(() => backupNotice({ copy: { heading: '', text: 'x' }, onBackup() {} }), TypeError);
|
||||
assert.throws(() => backupNotice({ copy: SCAN_BACKUP_COPY }), TypeError);
|
||||
} finally {
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('openReviewDialog lays out heading, grid, options, sentence, backup box and actions', async () => {
|
||||
const { openReviewDialog, closeWorkspaceDialog } = await dialogsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
try {
|
||||
const trigger = new dom.Element('button');
|
||||
dom.body.append(trigger);
|
||||
const option = new dom.Element('div');
|
||||
option.className = 'setting-row';
|
||||
const started = [];
|
||||
const review = openReviewDialog({ scope, title: 'Review maintenance plan', heading: 'Missing-file links',
|
||||
grid: [['Scope', '2 stale image links'], ['Media on disk', 'No source files removed']], options: [option],
|
||||
sentence: 'Cancelling leaves all data untouched.', backup: { copy: { heading: 'Back up first', text: 'A copy.' }, onBackup() {} },
|
||||
cancelLabel: 'Cancel', primary: { label: 'Apply reviewed operation' }, onStart: () => started.push('start'), trigger });
|
||||
assert.equal(byId(dom, 'modal-title').textContent, 'Review maintenance plan');
|
||||
assert.deepEqual(bodyShape(dom), ['h3', 'dl.meta-grid', 'div.setting-row', 'p.small', 'div.notice', 'div.dialog-actions']);
|
||||
const grid = byId(dom, 'modal-body').children[1];
|
||||
assert.deepEqual(grid.children.map(entry => [tags(entry), entry.children[0].textContent, entry.children[1].textContent]), [
|
||||
[['dt', 'dd'], 'Scope', '2 stale image links'], [['dt', 'dd'], 'Media on disk', 'No source files removed']]);
|
||||
review.setValue('Scope', '3 stale image links');
|
||||
assert.equal(grid.children[0].children[1].textContent, '3 stale image links');
|
||||
const actions = byId(dom, 'modal-body').children.at(-1);
|
||||
assert.deepEqual(actions.children.map(node => [node.textContent, node.className]),
|
||||
[['Cancel', ''], ['Apply reviewed operation', 'ca-primary']]);
|
||||
actions.children[1].click();
|
||||
assert.deepEqual(started, ['start']);
|
||||
assert.equal(modal(dom).open, false);
|
||||
assert.equal(document.activeElement, trigger);
|
||||
|
||||
// A danger primary (the concept's Restore) has no sentence, heading, options or backup box when none is given.
|
||||
openReviewDialog({ scope, title: 'Restore preview', grid: [['Integrity', 'Verified']], backup: null,
|
||||
cancelLabel: 'Cancel', primary: { label: 'Restore', danger: true }, onStart() {} });
|
||||
assert.deepEqual(bodyShape(dom), ['dl.meta-grid', 'div.dialog-actions']);
|
||||
assert.equal(button(byId(dom, 'modal-body'), 'Restore').className, 'danger');
|
||||
button(byId(dom, 'modal-body'), 'Cancel').click();
|
||||
assert.equal(modal(dom).open, false);
|
||||
assert.throws(() => openReviewDialog({ scope, title: 'x', grid: [['a']], cancelLabel: 'Cancel', primary: { label: 'Go' },
|
||||
onStart() {} }), TypeError);
|
||||
assert.throws(() => openReviewDialog({ scope, title: 'x', grid: [], cancelLabel: 'Cancel', primary: {}, onStart() {} }), TypeError);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
closeWorkspaceDialog();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('review dialog backup box keeps the dialog open and toasts the outcome', async () => {
|
||||
const { openReviewDialog, closeWorkspaceDialog, SCAN_BACKUP_COPY } = await dialogsModule;
|
||||
const { downloadDatabaseBackup, BACKUP_DONE, BACKUP_FAILED } = await scanModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
const created = URL.createObjectURL;
|
||||
const revoked = URL.revokeObjectURL;
|
||||
URL.createObjectURL = () => 'blob:backup';
|
||||
URL.revokeObjectURL = () => {};
|
||||
try {
|
||||
const toasts = [];
|
||||
let answer;
|
||||
const fetchUrl = () => new Promise(resolve => { answer = resolve; });
|
||||
openReviewDialog({ scope, title: 'Review metadata scan', grid: [['Sources', '2 output folders']],
|
||||
backup: { copy: SCAN_BACKUP_COPY, onBackup: () => downloadDatabaseBackup({ fetchUrl, notify: text => toasts.push(text) }) },
|
||||
cancelLabel: 'Cancel', primary: { label: 'Start scan' }, onStart() { throw new Error('not started'); } });
|
||||
const action = button(byId(dom, 'modal-body'), 'Back up now');
|
||||
action.click();
|
||||
assert.equal(action.textContent, 'Backing up…');
|
||||
assert.equal(action.getAttribute('aria-disabled'), 'true');
|
||||
// A second press while the copy downloads is ignored.
|
||||
action.click();
|
||||
answer({ ok: true, blob: async () => new Blob(['db']), headers: { get: () => 'attachment; filename="prompts_backup_1.db"' } });
|
||||
await tick();
|
||||
assert.deepEqual(toasts, [BACKUP_DONE]);
|
||||
assert.equal(modal(dom).open, true);
|
||||
assert.equal(action.textContent, 'Back up now');
|
||||
assert.equal(action.getAttribute('aria-disabled'), null);
|
||||
action.click();
|
||||
answer({ ok: false, status: 500 });
|
||||
await tick();
|
||||
assert.deepEqual(toasts, [BACKUP_DONE, BACKUP_FAILED]);
|
||||
assert.equal(modal(dom).open, true);
|
||||
} finally {
|
||||
URL.createObjectURL = created;
|
||||
URL.revokeObjectURL = revoked;
|
||||
scope.dispose();
|
||||
closeWorkspaceDialog();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('the scan review is a thin openReviewDialog caller with its scope switch', async () => {
|
||||
const { openScanReviewDialog, closeWorkspaceDialog } = await dialogsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
try {
|
||||
const starts = [];
|
||||
openScanReviewDialog({ scope, sources: 2, onBackup() {}, onStart: value => starts.push(value) });
|
||||
assert.equal(byId(dom, 'modal-title').textContent, 'Review metadata scan');
|
||||
assert.deepEqual(bodyShape(dom), ['dl.meta-grid', 'div.setting-row', 'p.small', 'div.notice', 'div.dialog-actions']);
|
||||
const grid = byId(dom, 'modal-body').children[0];
|
||||
assert.deepEqual(grid.children.map(entry => entry.textContent), ['Sources2 output folders', 'ScopeNew and unresolved files']);
|
||||
const allFiles = byId(dom, 'scan-all-files');
|
||||
allFiles.checked = true;
|
||||
allFiles.dispatchEvent(new Event('change'));
|
||||
assert.equal(grid.children[1].children[1].textContent, 'All files');
|
||||
button(byId(dom, 'modal-body'), 'Start scan').click();
|
||||
assert.deepEqual(starts, [{ scope: 'all' }]);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
closeWorkspaceDialog();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,877 @@
|
||||
"""Candidate-selection record sets for the local resolver parity test (PR L3 Task 9).
|
||||
|
||||
``local_resolver_cases`` covers one sampler's tracing. These cases reach the
|
||||
rest of ``_resolve``: several sampler candidates and the identical-branch rule
|
||||
(#270), stages that only reach previews, detailer stages and their upstream
|
||||
samplers (multistage), and A1111 ``parameters`` with every footer shape. The
|
||||
server's ``inspect_prompt_evidence`` is the oracle; nothing here states an
|
||||
expected answer. All content is synthetic.
|
||||
"""
|
||||
|
||||
import json
|
||||
import random
|
||||
|
||||
try:
|
||||
from tests import passthrough_graphs as shapes
|
||||
from tests.local_resolver_cases import EXECUTION_KEY, SPACES, WORDS, _Builder
|
||||
except ImportError: # pragma: no cover - discover -s tests puts tests/ first
|
||||
import passthrough_graphs as shapes
|
||||
from local_resolver_cases import EXECUTION_KEY, SPACES, WORDS, _Builder
|
||||
|
||||
|
||||
def _encode(text):
|
||||
return {"class_type": "CLIPTextEncode", "inputs": {"text": text, "clip": ["4", 1]}}
|
||||
|
||||
|
||||
def _sampler(positive, negative, latent=None, name="KSampler"):
|
||||
inputs = {"seed": 1, "model": ["4", 0], "positive": positive}
|
||||
if negative is not None:
|
||||
inputs["negative"] = negative
|
||||
inputs["latent_image"] = latent or ["5", 0]
|
||||
return {"class_type": name, "inputs": inputs}
|
||||
|
||||
|
||||
def _decode(sampler_id):
|
||||
return {"class_type": "VAEDecode", "inputs": {"samples": [sampler_id, 0]}}
|
||||
|
||||
|
||||
def _save(image, name="SaveImage"):
|
||||
return {"class_type": name, "inputs": {"images": image}}
|
||||
|
||||
|
||||
def _detailer(image, positive, negative, name="FaceDetailer", **extra):
|
||||
inputs = {"image": image, "model": ["4", 0], "positive": positive}
|
||||
inputs["negative"] = negative
|
||||
inputs.update(extra)
|
||||
return {"class_type": name, "inputs": inputs}
|
||||
|
||||
|
||||
def _base(**nodes):
|
||||
graph = {
|
||||
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"5": {"class_type": "EmptyLatentImage", "inputs": {"width": 64}},
|
||||
"6": _encode("a fox"),
|
||||
"7": _encode("blurry"),
|
||||
"8": _encode("a cat"),
|
||||
}
|
||||
graph.update(nodes)
|
||||
return graph
|
||||
|
||||
|
||||
def _samplers():
|
||||
"""Several unanchored samplers: identical, differing and mixed branches."""
|
||||
same = _sampler(["6", 0], ["7", 0])
|
||||
cases = {
|
||||
"two-identical": _base(**{"3": same, "9": same}),
|
||||
"two-differ": _base(**{"3": same, "9": _sampler(["8", 0], ["7", 0])}),
|
||||
"two-negative-differs": _base(**{"3": same, "9": _sampler(["6", 0], ["8", 0])}),
|
||||
"identical-without-negative": _base(
|
||||
**{"3": _sampler(["6", 0], None), "9": _sampler(["6", 0], None)}
|
||||
),
|
||||
"one-unresolved": _base(**{"3": same, "9": _sampler(["missing", 0], ["7", 0])}),
|
||||
"both-unresolved": _base(
|
||||
**{"3": _sampler(["x", 0], ["7", 0]), "9": _sampler(["y", 0], ["7", 0])}
|
||||
),
|
||||
"three-identical-one-advanced": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": same,
|
||||
"10": _sampler(["6", 0], ["7", 0], name="KSamplerAdvanced"),
|
||||
}
|
||||
),
|
||||
# Equal texts, different recipes: a PromptManager pair against an encoder.
|
||||
"same-text-other-recipe": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": _sampler(["11", 0], ["7", 0]),
|
||||
"11": {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {
|
||||
"text": "fox",
|
||||
"prepend_text": "a",
|
||||
"append_text": "",
|
||||
"clip": ["4", 1],
|
||||
},
|
||||
},
|
||||
}
|
||||
),
|
||||
"multistage-latent": _base(
|
||||
**{"3": same, "9": _sampler(["6", 0], ["7", 0], latent=["3", 0])}
|
||||
),
|
||||
}
|
||||
selector = {
|
||||
"class_type": "easy ifElse",
|
||||
"inputs": {"boolean": ["4", 0], "on_true": ["6", 0], "on_false": ["8", 0]},
|
||||
}
|
||||
cases["selector-and-plain"] = _base(
|
||||
**{"3": same, "9": _sampler(["12", 0], ["7", 0]), "12": selector}
|
||||
)
|
||||
return cases
|
||||
|
||||
|
||||
def _previews():
|
||||
"""Stages whose outputs reach only previews are not the saved file's."""
|
||||
same, other = _sampler(["6", 0], ["7", 0]), _sampler(["8", 0], ["7", 0])
|
||||
return {
|
||||
"preview-dropped": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": other,
|
||||
"13": _decode("3"),
|
||||
"14": _decode("9"),
|
||||
"15": _save(["13", 0]),
|
||||
"16": _save(["14", 0], "PreviewImage"),
|
||||
}
|
||||
),
|
||||
"all-previews-kept": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": other,
|
||||
"13": _decode("3"),
|
||||
"14": _decode("9"),
|
||||
"15": _save(["13", 0], "PreviewImage"),
|
||||
"16": _save(["14", 0], "Image Comparer (rgthree)"),
|
||||
}
|
||||
),
|
||||
"preview-and-file": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": other,
|
||||
"13": _decode("3"),
|
||||
"14": _decode("9"),
|
||||
"15": _save(["13", 0]),
|
||||
"16": _save(["14", 0], "PreviewImage"),
|
||||
"17": _save(["14", 0]),
|
||||
}
|
||||
),
|
||||
"unconsumed-stage-kept": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": other,
|
||||
"14": _decode("9"),
|
||||
"16": _save(["14", 0], "PreviewImage"),
|
||||
}
|
||||
),
|
||||
# A list or object class_type is never a preview class (#286).
|
||||
"preview-unhashable-class": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": other,
|
||||
"14": _decode("9"),
|
||||
"16": {"class_type": ["PreviewImage"], "inputs": {"images": ["14", 0]}},
|
||||
"17": {"class_type": {"a": 1}, "inputs": {"images": ["3", 0]}},
|
||||
}
|
||||
),
|
||||
"preview-through-bad-node": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": other,
|
||||
"13": _decode("3"),
|
||||
"15": _save(["13", 0]),
|
||||
"16": _save(["9", 0], "PreviewImage"),
|
||||
"18": "not a node",
|
||||
"19": {"class_type": "Note", "inputs": {"x": ["18", 0]}},
|
||||
}
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _detailers():
|
||||
"""Detailer stages judged with the samplers in their image ancestry."""
|
||||
same = _sampler(["6", 0], ["7", 0])
|
||||
cases = {
|
||||
"detailer-same-prompt": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-differs": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["8", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-and-own-output": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["8", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
"13": _save(["10", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-image-literal": _base(
|
||||
**{
|
||||
"11": _detailer("not a link", ["6", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-cycle": _base(
|
||||
**{
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"10": {"class_type": "ImageScale", "inputs": {"image": ["11", 0]}},
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-chain": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"21": _detailer(
|
||||
["11", 0], ["8", 0], ["7", 0], name="UltimateSDUpscale"
|
||||
),
|
||||
"12": _save(["21", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-chain-same": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"21": _detailer(["11", 0], ["6", 0], ["7", 0], name="DetailerForEach"),
|
||||
"12": _save(["21", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-two-upstream": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"9": _sampler(["8", 0], ["7", 0]),
|
||||
"10": {
|
||||
"class_type": "ImageBlend",
|
||||
"inputs": {"image1": ["13", 0], "image2": ["14", 0]},
|
||||
},
|
||||
"13": _decode("3"),
|
||||
"14": _decode("9"),
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-unsupported-stage": _base(
|
||||
**{
|
||||
"10": {
|
||||
"class_type": "MysterySampler",
|
||||
"inputs": {"positive": ["6", 0], "model": ["4", 0]},
|
||||
},
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-upstream-unresolved": _base(
|
||||
**{
|
||||
"3": _sampler(["gone", 0], ["7", 0]),
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-upstream-selector": _base(
|
||||
**{
|
||||
"3": _sampler(["15", 0], ["7", 0]),
|
||||
"15": {
|
||||
"class_type": "easy ifElse",
|
||||
"inputs": {
|
||||
"boolean": ["4", 0],
|
||||
"on_true": ["6", 0],
|
||||
"on_false": ["8", 0],
|
||||
},
|
||||
},
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-wildcard": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0], wildcard="[x|y]"),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-no-upscaled-image": _base(
|
||||
**{
|
||||
"3": same,
|
||||
"10": _decode("3"),
|
||||
"11": _detailer(
|
||||
["10", 0], ["6", 0], ["7", 0], name="UltimateSDUpscaleNoUpscale"
|
||||
),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
"detailer-no-sampler-upstream": _base(
|
||||
**{
|
||||
"10": {"class_type": "LoadImage", "inputs": {"image": "a.png"}},
|
||||
"11": _detailer(["10", 0], ["6", 0], ["7", 0]),
|
||||
"12": _save(["11", 0]),
|
||||
}
|
||||
),
|
||||
}
|
||||
deep = {"3": _sampler(["6", 0], ["7", 0]), "d0": _decode("3")}
|
||||
for index in range(1, 70):
|
||||
deep[f"d{index}"] = {
|
||||
"class_type": "ImageScale",
|
||||
"inputs": {"image": [f"d{index - 1}", 0]},
|
||||
}
|
||||
deep["11"] = _detailer(["d69", 0], ["6", 0], ["7", 0])
|
||||
deep["12"] = _save(["11", 0])
|
||||
cases["detailer-deep-ancestry"] = _base(**deep)
|
||||
# More nested detailer stages than the depth bound.
|
||||
stacked = {"3": _sampler(["6", 0], ["7", 0]), "s0": _decode("3")}
|
||||
previous = "s0"
|
||||
for index in range(1, 67):
|
||||
stacked[f"s{index}"] = _detailer([previous, 0], ["6", 0], ["7", 0])
|
||||
previous = f"s{index}"
|
||||
stacked["12"] = _save([previous, 0])
|
||||
cases["detailer-stacked-past-depth"] = _base(**stacked)
|
||||
# Each detailer walks a wide shared image ancestry: the total visit budget
|
||||
# runs out across stages before any candidate is read.
|
||||
wide = {
|
||||
f"l{index}": {"class_type": "LoadImage", "inputs": {}} for index in range(1500)
|
||||
}
|
||||
wide["hub"] = {
|
||||
"class_type": "ImageBatchMany",
|
||||
"inputs": {f"i{index}": [f"l{index}", 0] for index in range(1500)},
|
||||
}
|
||||
for index in range(15):
|
||||
wide[f"w{index}"] = _detailer(["hub", 0], ["6", 0], ["7", 0])
|
||||
wide[f"o{index}"] = _save([f"w{index}", 0])
|
||||
cases["detailers-exhaust-total-visits"] = _base(**wide)
|
||||
cases["detailer-mutual-cycle"] = _base(
|
||||
**{
|
||||
"11": _detailer(["21", 0], ["6", 0], ["7", 0]),
|
||||
"21": _detailer(["11", 0], ["6", 0], ["7", 0]),
|
||||
"12": _save(["21", 0]),
|
||||
}
|
||||
)
|
||||
# The sampler's output reaches a preview through more nodes than one
|
||||
# detailer's allowance: the walk runs out, so the sampler is not absorbed.
|
||||
long = {"3": _sampler(["6", 0], ["7", 0]), "10": _decode("3")}
|
||||
previous = "10"
|
||||
for index in range(2100):
|
||||
long[f"c{index}"] = {
|
||||
"class_type": "ImageScale",
|
||||
"inputs": {"image": [previous, 0]},
|
||||
}
|
||||
previous = f"c{index}"
|
||||
long["p"] = _save([previous, 0], "PreviewImage")
|
||||
long["11"] = _detailer(["10", 0], ["6", 0], ["7", 0])
|
||||
long["12"] = _save(["11", 0])
|
||||
cases["detailer-output-walk-exhausted"] = _base(**long)
|
||||
# Every detailer's ancestry walk has its own allowance: the first one is
|
||||
# wide, and the second still absorbs the first.
|
||||
chained = {"3": _sampler(["6", 0], ["7", 0]), "10": _decode("3")}
|
||||
chained["h1"] = {
|
||||
"class_type": "ImageBatchMany",
|
||||
"inputs": {
|
||||
"i": ["10", 0],
|
||||
**{f"i{index}": [f"a{index}", 0] for index in range(1500)},
|
||||
},
|
||||
}
|
||||
chained["h2"] = {
|
||||
"class_type": "ImageBatchMany",
|
||||
"inputs": {
|
||||
"i": ["11", 0],
|
||||
**{f"i{index}": [f"b{index}", 0] for index in range(600)},
|
||||
},
|
||||
}
|
||||
for index in range(1500):
|
||||
chained[f"a{index}"] = {"class_type": "LoadImage", "inputs": {}}
|
||||
for index in range(600):
|
||||
chained[f"b{index}"] = {"class_type": "LoadImage", "inputs": {}}
|
||||
chained["11"] = _detailer(["h1", 0], ["6", 0], ["7", 0])
|
||||
chained["21"] = _detailer(["h2", 0], ["6", 0], ["7", 0])
|
||||
chained["12"] = _save(["21", 0])
|
||||
cases["detailers-each-reset"] = _base(**chained)
|
||||
return cases
|
||||
|
||||
|
||||
_FOOTER = "Steps: 20, Sampler: euler, CFG scale: 7"
|
||||
|
||||
|
||||
def _parameters():
|
||||
"""A1111 ``parameters`` texts: every footer shape and delimiter rule."""
|
||||
texts = {
|
||||
"basic": f"a fox\nNegative prompt: blur\n{_FOOTER}",
|
||||
"no-negative": f"a fox\n{_FOOTER}",
|
||||
"two-delimiters": f"a\nNegative prompt: b\nNegative prompt: c\n{_FOOTER}",
|
||||
"leading-delimiter": f"Negative prompt: blur\n{_FOOTER}",
|
||||
"blank-positive": f" \x85\nNegative prompt: blur\n{_FOOTER}",
|
||||
"footer-not-last": f"a fox\n{_FOOTER}\nmore text",
|
||||
"two-footers": f"a\n{_FOOTER}\n{_FOOTER}",
|
||||
"bare-footer": "a fox\nNegative prompt: blur\nSteps: 20",
|
||||
"bare-and-full": f"a fox\nSteps: 5\n{_FOOTER}",
|
||||
"two-bare": "a\nSteps: 1\nSteps: 2",
|
||||
"template": f"a fox\n{_FOOTER}\nTemplate: {{a|b}}",
|
||||
"both-templates": f"a\n{_FOOTER}\nTemplate: x\nNegative Template: y",
|
||||
"templates-reversed": f"a\n{_FOOTER}\nNegative Template: y\nTemplate: x",
|
||||
"two-templates": f"a\n{_FOOTER}\nTemplate: x\nTemplate: y",
|
||||
"template-multiline": f"a\n{_FOOTER}\nTemplate: x\nmore",
|
||||
"crlf": f"a fox\r\nNegative prompt: blur\r\n{_FOOTER}\r\n",
|
||||
"cr-only": f"a fox\rNegative prompt: blur\r{_FOOTER}",
|
||||
"cr-template": f"a\n{_FOOTER}\rTemplate: x",
|
||||
"line-separator": f"a fox
{_FOOTER}",
|
||||
"next-line": f"a fox\x85{_FOOTER}",
|
||||
"spaced-steps": "a fox\nSteps:\n 20 \n, x",
|
||||
"steps-newline-bare": "a fox\nSteps:\n20\n",
|
||||
"arabic-digits": "a fox\nSteps: ٢٠, Sampler: euler",
|
||||
"fullwidth-digits": "a fox\nSteps: 2, x",
|
||||
"no-digits": "a fox\nSteps: , x",
|
||||
"trailing-spaces": f"a fox \nNegative prompt: blur \n{_FOOTER} \n \n",
|
||||
"empty-negative": f"a fox\nNegative prompt:\n{_FOOTER}",
|
||||
"astral-within-limit": "\U0001f600" * 60_000 + f"\n{_FOOTER}",
|
||||
"over-limit": "x" * 100_001,
|
||||
"at-limit": "x" * (100_000 - len(_FOOTER) - 1) + "\n" + _FOOTER,
|
||||
"empty": "",
|
||||
"only-footer": _FOOTER,
|
||||
"lone-surrogate": f"\ud800 fox\nNegative prompt: \udfff\n{_FOOTER}",
|
||||
}
|
||||
cases = {f"parameters-{name}": {"parameters": text} for name, text in texts.items()}
|
||||
cases["parameters-with-execution"] = {
|
||||
"parameters": texts["basic"],
|
||||
EXECUTION_KEY: json.dumps({"version": 1, "nodes": {}}),
|
||||
}
|
||||
return cases
|
||||
|
||||
|
||||
def _passthrough():
|
||||
"""Chains joined through lineage nodes, and the floor for added conditioning."""
|
||||
cases = {
|
||||
"hires-fix": shapes.hires_fix(),
|
||||
"hires-fix-differs": shapes.hires_fix(second_positive=shapes.OTHER),
|
||||
"hires-fix-unlisted": shapes.hires_fix(filter_class="MysteryImageEffect"),
|
||||
"hires-fix-generated": shapes.generated_text_chain(),
|
||||
"flux-three-stage": shapes.flux_three_stage(),
|
||||
"flux-concat": shapes.flux_three_stage(composite="ConditioningConcat"),
|
||||
"flux-average": shapes.flux_three_stage(composite="ConditioningAverage"),
|
||||
"edit-batch": shapes.edit_batch(3),
|
||||
}
|
||||
graph = shapes.flux_three_stage(combined=(("37", 0), ("31", 0)))
|
||||
graph["37"] = shapes.encode(shapes.OTHER)
|
||||
cases["flux-without-base"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["38"] = shapes.encode("watermark")
|
||||
graph["60"]["inputs"]["negative"] = ["38", 0]
|
||||
cases["flux-other-negative"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["20"]["inputs"]["latent_image"] = ["10", 0]
|
||||
graph["10"]["inputs"]["latent_image"] = ["20", 0]
|
||||
cases["hires-fix-cycle"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["17"]["inputs"]["pixels"] = ["18", 0]
|
||||
graph["18"] = {"class_type": "Reroute", "inputs": {"value": ["16", 0]}}
|
||||
cases["hires-fix-reroute"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["24"] = {"class_type": "SaveImage", "inputs": {"images": ["11", 0]}}
|
||||
cases["hires-fix-both-saved"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["20"]["inputs"]["latent_image"] = ["99", 0]
|
||||
cases["hires-fix-missing-latent"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["38"] = {"class_type": "MysteryNegative", "inputs": {"seed": 1}}
|
||||
graph["60"]["inputs"]["negative"] = ["38", 0]
|
||||
cases["flux-negative-unsupported"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = {"class_type": "Reroute", "inputs": {"input": ["35", 0]}}
|
||||
graph["60"]["inputs"]["positive"] = ["39", 0]
|
||||
cases["flux-reroute-input"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = {
|
||||
"class_type": "CLIPTextEncodeSDXL",
|
||||
"inputs": {"text_g": shapes.POSITIVE, "text_l": shapes.STYLE},
|
||||
}
|
||||
graph["60"]["inputs"]["positive"] = ["39", 0]
|
||||
cases["flux-encoder-parts"] = graph
|
||||
cases.update(_passthrough_edges())
|
||||
cases.update(_runtime_parts())
|
||||
return cases
|
||||
|
||||
|
||||
def _runtime_parts():
|
||||
"""Option A: a PromptManager part fed by a run-time node is left out."""
|
||||
cases = {"trigger-prepend": shapes.hires_fix_with_trigger_words()}
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["2"]["inputs"].update(prepend_text="masterpiece", append_text=["8", 0])
|
||||
graph["20"]["inputs"]["latent_image"] = ["4", 0]
|
||||
cases["trigger-append-single"] = graph
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["8"] = {"class_type": "PrimitiveString", "inputs": {"value": "trig9word"}}
|
||||
cases["traced-prepend"] = graph
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["8"]["class_type"] = "MysteryWords"
|
||||
cases["unknown-prepend"] = graph
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["9"] = {
|
||||
"class_type": "StringConcatenate",
|
||||
"inputs": {"string_a": "kept", "string_b": ["8", 0], "delimiter": ""},
|
||||
}
|
||||
graph["2"]["inputs"]["prepend_text"] = ["9", 0]
|
||||
cases["indirect-prepend"] = graph
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["2"]["inputs"].update(prepend_text="", text=["8", 0])
|
||||
cases["runtime-main-text"] = graph
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["2"]["class_type"] = "PromptManagerPair"
|
||||
graph["2"]["inputs"]["negative_text"] = shapes.NEGATIVE
|
||||
graph["10"]["inputs"]["negative"] = ["2", 1]
|
||||
graph["20"]["inputs"]["negative"] = ["2", 1]
|
||||
cases["trigger-pair"] = graph
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["30"] = {"class_type": "PromptSearchList", "inputs": {"search": "x"}}
|
||||
graph["2"]["inputs"]["append_text"] = ["30", 0]
|
||||
cases["search-list-append"] = graph
|
||||
for shape in ("text-encode", "chained", "concat"):
|
||||
cases[f"through-{shape}"] = shapes.trigger_words_through(shape)
|
||||
cases["branches-with-and-without"] = (
|
||||
shapes.branches_with_and_without_trigger_words()
|
||||
)
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["9"] = shapes.prompt_manager(shapes.POSITIVE)
|
||||
graph["20"]["inputs"]["positive"] = ["9", 0]
|
||||
cases["stage-with-and-without"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["9"] = {
|
||||
"class_type": "JoinStringMulti",
|
||||
"inputs": {
|
||||
"inputcount": 2,
|
||||
"delimiter": " ",
|
||||
"return_list": ["98", 0],
|
||||
"string_1": "static words",
|
||||
"string_2": "more",
|
||||
},
|
||||
}
|
||||
graph["98"] = {"class_type": "PrimitiveBoolean", "inputs": {"value": False}}
|
||||
graph["2"]["inputs"]["prepend_text"] = ["9", 0]
|
||||
cases["join-multi-flag"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["31"] = shapes.encode(shapes.OTHER)
|
||||
cases["flux-other-full-prompt"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["7"] = {
|
||||
"class_type": "Lora Loader (LoraManager)",
|
||||
"inputs": {"model": ["1", 0], "clip": ["2", 0], "text": "x"},
|
||||
}
|
||||
graph["8"] = shapes.trigger_toggle()
|
||||
graph["30"]["inputs"]["prepend_text"] = ["8", 0]
|
||||
cases["flux-trigger"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["60"] = {
|
||||
"class_type": "SamplerCustomAdvanced",
|
||||
"inputs": {"guider": ["63", 0], "latent_image": ["66", 0]},
|
||||
}
|
||||
graph["63"] = {
|
||||
"class_type": "CFGGuider",
|
||||
"inputs": {"positive": ["35", 0], "negative": ["33", 0]},
|
||||
}
|
||||
graph["66"] = {"class_type": "VAEEncode", "inputs": {"pixels": ["51", 0]}}
|
||||
graph["67"] = {"class_type": "VAEDecode", "inputs": {"samples": ["60", 0]}}
|
||||
graph["61"]["inputs"]["images"] = ["67", 0]
|
||||
cases["flux-custom-advanced-chained"] = graph
|
||||
return {f"runtime-{name}": graph for name, graph in cases.items()}
|
||||
|
||||
|
||||
def _passthrough_edges():
|
||||
"""Lineage and floor edges: branching refiners, other slots, cycles, guiders."""
|
||||
cases = {}
|
||||
graph = shapes.hires_fix()
|
||||
graph["30"] = shapes.ksampler(["2", 0], ["3", 0], ["17", 0], denoise=0.4)
|
||||
graph["31"] = {"class_type": "SaveImage", "inputs": {"images": ["30", 0]}}
|
||||
cases["hires-fix-two-refiners"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["10"]["class_type"] = "SamplerCustom"
|
||||
graph["11"]["inputs"]["samples"] = ["10", 1]
|
||||
cases["hires-fix-denoised-slot"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["30"] = shapes.ksampler(["2", 0], ["3", 0], ["4", 0])
|
||||
graph["31"] = {"class_type": "VAEDecode", "inputs": {"samples": ["30", 0]}}
|
||||
graph["32"] = {"class_type": "VAEEncode", "inputs": {"pixels": ["31", 0]}}
|
||||
graph["30"]["inputs"]["latent_image"] = ["32", 0]
|
||||
cases["hires-fix-self-latent"] = graph
|
||||
graph = shapes.hires_fix()
|
||||
graph["30"] = shapes.ksampler(["2", 0], ["3", 0], ["32", 0])
|
||||
graph["31"] = {"class_type": "VAEDecode", "inputs": {"samples": ["32", 0]}}
|
||||
graph["32"] = {"class_type": "VAEEncode", "inputs": {"pixels": ["31", 0]}}
|
||||
graph["33"] = {"class_type": "SaveImage", "inputs": {"images": ["30", 0]}}
|
||||
cases["hires-fix-lineage-cycle"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = shapes.encode(shapes.OTHER)
|
||||
graph["40"]["inputs"]["positive"] = ["39", 0]
|
||||
cases["flux-base-stages-differ"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
del graph["60"]["inputs"]["negative"]
|
||||
cases["flux-no-last-negative"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = {"class_type": "MysteryPrompt", "inputs": {"seed": 1}}
|
||||
graph["40"]["inputs"]["positive"] = ["39", 0]
|
||||
graph["50"]["inputs"]["positive"] = ["39", 0]
|
||||
cases["flux-unreadable-base"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = {
|
||||
"class_type": "MysteryUpscaler",
|
||||
"inputs": {"image": ["51", 0], "model": ["1", 0], "positive": ["34", 0]},
|
||||
}
|
||||
graph["52"]["inputs"]["images"] = ["39", 0]
|
||||
cases["flux-unsupported-ancestry"] = graph
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["60"] = {
|
||||
"class_type": "SamplerCustomAdvanced",
|
||||
"inputs": {
|
||||
"noise": ["62", 0],
|
||||
"guider": ["63", 0],
|
||||
"sampler": ["64", 0],
|
||||
"sigmas": ["65", 0],
|
||||
"latent_image": ["66", 0],
|
||||
},
|
||||
}
|
||||
graph["62"] = {"class_type": "RandomNoise", "inputs": {"noise_seed": 1}}
|
||||
graph["63"] = {
|
||||
"class_type": "CFGGuider",
|
||||
"inputs": {"model": ["1", 0], "positive": ["35", 0], "negative": ["33", 0]},
|
||||
}
|
||||
graph["64"] = {"class_type": "KSamplerSelect", "inputs": {"sampler_name": "euler"}}
|
||||
graph["65"] = {"class_type": "BasicScheduler", "inputs": {"steps": 4}}
|
||||
graph["66"] = {"class_type": "VAEEncode", "inputs": {"pixels": ["52", 0]}}
|
||||
graph["67"] = {"class_type": "VAEDecode", "inputs": {"samples": ["60", 0]}}
|
||||
graph["61"]["inputs"]["images"] = ["67", 0]
|
||||
cases["flux-custom-advanced-last"] = graph
|
||||
return {f"edge-{name}": graph for name, graph in cases.items()}
|
||||
|
||||
|
||||
def selection_cases():
|
||||
"""Named record sets for every Task 9 path, each a ``read_inspection_file`` dict."""
|
||||
cases = {}
|
||||
for group in (_samplers(), _previews(), _detailers(), _passthrough()):
|
||||
for name, graph in group.items():
|
||||
cases[f"select-{name}"] = {"prompt": json.dumps(graph, ensure_ascii=False)}
|
||||
cases.update(_parameters())
|
||||
return cases
|
||||
|
||||
|
||||
class _Selection(_Builder):
|
||||
"""Random graphs with several samplers, outputs, previews and detailers."""
|
||||
|
||||
def sampler(self):
|
||||
rng = self.rng
|
||||
positive = self.conditioning(0)
|
||||
negative = self.conditioning(0) if rng.random() < 0.85 else None
|
||||
name = rng.choice(["KSampler", "KSampler", "KSamplerAdvanced", "SamplerCustom"])
|
||||
node_id = self.new_id()
|
||||
self.nodes[node_id] = _sampler(positive, negative, name=name)
|
||||
return node_id
|
||||
|
||||
def shared(self, ids):
|
||||
"""A copy of an earlier sampler: an identical branch."""
|
||||
node_id = self.new_id()
|
||||
self.nodes[node_id] = json.loads(json.dumps(self.nodes[self.rng.choice(ids)]))
|
||||
return node_id
|
||||
|
||||
def output(self, image):
|
||||
rng = self.rng
|
||||
choice = rng.randrange(5)
|
||||
if choice == 4:
|
||||
return
|
||||
name = ["SaveImage", "SaveImage", "PreviewImage", "PreviewAny"][choice]
|
||||
self.nodes[self.new_id()] = _save(image, name)
|
||||
|
||||
def detailer(self, image):
|
||||
rng = self.rng
|
||||
node_id = self.new_id()
|
||||
positive = self.conditioning(1) if rng.random() < 0.5 else None
|
||||
name = rng.choice(list(_DETAILER_NAMES))
|
||||
node = _detailer(image, positive, self.conditioning(1), name=name)
|
||||
if positive is None:
|
||||
node["inputs"]["positive"] = self.reuse_positive()
|
||||
if name == "UltimateSDUpscaleNoUpscale":
|
||||
node["inputs"]["upscaled_image"] = node["inputs"].pop("image")
|
||||
if name in ("FaceDetailer", "DetailerForEach") and rng.random() < 0.15:
|
||||
node["inputs"]["wildcard"] = rng.choice(["", "x", ["4", 0]])
|
||||
self.nodes[node_id] = node
|
||||
return node_id
|
||||
|
||||
def reuse_positive(self):
|
||||
samplers = [
|
||||
node
|
||||
for node in self.nodes.values()
|
||||
if node.get("class_type")
|
||||
in ("KSampler", "KSamplerAdvanced", "SamplerCustom")
|
||||
]
|
||||
if samplers:
|
||||
return self.rng.choice(samplers)["inputs"]["positive"]
|
||||
return self.conditioning(1)
|
||||
|
||||
def build(self):
|
||||
rng = self.rng
|
||||
ids = [self.sampler()]
|
||||
for _ in range(rng.randrange(3)):
|
||||
ids.append(self.shared(ids) if rng.random() < 0.4 else self.sampler())
|
||||
for sampler_id in list(ids):
|
||||
decoded = self.new_id()
|
||||
self.nodes[decoded] = _decode(sampler_id)
|
||||
image = [decoded, 0]
|
||||
for _ in range(rng.randrange(3)):
|
||||
if rng.random() < 0.5:
|
||||
image = [self.detailer(image), 0]
|
||||
else:
|
||||
self.output(image)
|
||||
self.output(image)
|
||||
if rng.random() < 0.1:
|
||||
self.nodes[sampler_id]["inputs"]["latent_image"] = [rng.choice(ids), 0]
|
||||
graph = {
|
||||
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"5": {"class_type": "EmptyLatentImage", "inputs": {"width": 64}},
|
||||
}
|
||||
graph.update(self.nodes)
|
||||
return graph
|
||||
|
||||
|
||||
_DETAILER_NAMES = (
|
||||
"FaceDetailer",
|
||||
"DetailerForEach",
|
||||
"UltimateSDUpscale",
|
||||
"UltimateSDUpscaleNoUpscale",
|
||||
)
|
||||
_PARAMETER_LINES = [
|
||||
"Negative prompt:",
|
||||
"Negative prompt: blur",
|
||||
"Steps: 20, Sampler: euler",
|
||||
"Steps: 7",
|
||||
"Steps: x",
|
||||
"Template: t",
|
||||
"Negative Template: n",
|
||||
"Steps:٣, a",
|
||||
]
|
||||
|
||||
|
||||
def _parameter_text(rng):
|
||||
parts = []
|
||||
for _ in range(rng.randint(1, 5)):
|
||||
if rng.random() < 0.5:
|
||||
parts.append(rng.choice(_PARAMETER_LINES))
|
||||
else:
|
||||
parts.append(rng.choice(SPACES) + rng.choice(WORDS) + rng.choice(SPACES))
|
||||
joints = ["\n", "\n", "\r\n", "\r", "
", " "]
|
||||
return "".join(part + rng.choice(joints) for part in parts).rstrip(
|
||||
rng.choice(["", "\n", " "])
|
||||
)
|
||||
|
||||
|
||||
def selection_fuzz_cases(seed, count):
|
||||
"""Seeded several-sampler graphs (sometimes corrupted) and parameters texts."""
|
||||
rng = random.Random(seed)
|
||||
cases = {}
|
||||
for index in range(count):
|
||||
if rng.random() < 0.2:
|
||||
cases[f"select-fuzz-{seed}-{index}"] = {"parameters": _parameter_text(rng)}
|
||||
continue
|
||||
builder = _Selection(rng)
|
||||
graph = builder.build()
|
||||
if rng.random() < 0.2:
|
||||
builder.nodes = graph
|
||||
builder.corrupt()
|
||||
records = {"prompt": json.dumps(graph, ensure_ascii=rng.random() < 0.5)}
|
||||
records.update(builder.records)
|
||||
cases[f"select-fuzz-{seed}-{index}"] = records
|
||||
return cases
|
||||
|
||||
|
||||
_LINEAGE_HOPS = (
|
||||
("VAEDecode", "samples"),
|
||||
("VAEDecodeTiled", "samples"),
|
||||
("Image Filter", "images"),
|
||||
("ImageUpscaleWithModel", "image"),
|
||||
("ImageScale", "image"),
|
||||
("ImageScaleBy", "image"),
|
||||
("VAEEncode", "pixels"),
|
||||
("LatentUpscaleBy", "samples"),
|
||||
("Reroute", "value"),
|
||||
("MysteryImageEffect", "image"),
|
||||
)
|
||||
|
||||
|
||||
def passthrough_fuzz_cases(seed, count):
|
||||
"""Seeded sampler chains joined by random lineage (and unlisted) hops.
|
||||
|
||||
Each later stage reads the earlier one's output through zero to four hops,
|
||||
with a shared, a different, an added-to (combine, concat, average) or a
|
||||
run-time positive, and a shared or different negative, under a saver or a
|
||||
preview. The server is the oracle; nothing states the expected answer.
|
||||
"""
|
||||
rng = random.Random(seed)
|
||||
cases = {}
|
||||
for index in range(count):
|
||||
graph = {
|
||||
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"2": shapes.prompt_manager(rng.choice([shapes.POSITIVE, "fox"])),
|
||||
"3": shapes.encode(shapes.NEGATIVE),
|
||||
"4": {"class_type": "EmptyLatentBatch", "inputs": {"batch_size": 2}},
|
||||
"5": shapes.encode(shapes.STYLE),
|
||||
"6": shapes.encode(shapes.OTHER),
|
||||
}
|
||||
previous, next_id = ["4", 0], 10
|
||||
for stage in range(rng.randint(2, 3)):
|
||||
positive = ["2", 0]
|
||||
choice = rng.randrange(8)
|
||||
if stage and choice == 1:
|
||||
positive = ["6", 0]
|
||||
elif stage and choice in (2, 3, 4):
|
||||
name = (
|
||||
"ConditioningCombine",
|
||||
"ConditioningConcat",
|
||||
"ConditioningAverage",
|
||||
)[choice - 2]
|
||||
names = shapes._COMPOSITE_INPUTS[name]
|
||||
parts = [["2", 0], rng.choice([["5", 0], ["6", 0]])]
|
||||
rng.shuffle(parts)
|
||||
graph[str(next_id)] = {
|
||||
"class_type": name,
|
||||
"inputs": dict(zip(names, parts)),
|
||||
}
|
||||
positive, next_id = [str(next_id), 0], next_id + 1
|
||||
elif stage and choice == 5:
|
||||
graph[str(next_id)] = {
|
||||
"class_type": "FluxGuidance",
|
||||
"inputs": {"guidance": 3.5, "conditioning": ["2", 0]},
|
||||
}
|
||||
positive, next_id = [str(next_id), 0], next_id + 1
|
||||
negative = ["6", 0] if stage and rng.random() < 0.15 else ["3", 0]
|
||||
latent = previous
|
||||
if stage:
|
||||
image = previous
|
||||
for _ in range(rng.randrange(5)):
|
||||
name, key = rng.choice(_LINEAGE_HOPS)
|
||||
graph[str(next_id)] = {"class_type": name, "inputs": {key: image}}
|
||||
image, next_id = [str(next_id), 0], next_id + 1
|
||||
latent = image
|
||||
sampler = shapes.ksampler(positive, negative, latent)
|
||||
if stage and rng.random() < 0.2:
|
||||
sampler = {
|
||||
"class_type": "UltimateSDUpscale",
|
||||
"inputs": {
|
||||
"image": latent,
|
||||
"model": ["1", 0],
|
||||
"positive": positive,
|
||||
"negative": negative,
|
||||
},
|
||||
}
|
||||
graph[str(next_id)] = sampler
|
||||
previous, next_id = [str(next_id), 0], next_id + 1
|
||||
if rng.random() < 0.3:
|
||||
saver = rng.choice(["SaveImage", "PreviewImage"])
|
||||
graph[str(next_id)] = {
|
||||
"class_type": saver,
|
||||
"inputs": {"images": previous},
|
||||
}
|
||||
next_id += 1
|
||||
graph[str(next_id)] = {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {"images": previous},
|
||||
}
|
||||
cases[f"select-passthrough-fuzz-{seed}-{index}"] = {
|
||||
"prompt": json.dumps(graph, ensure_ascii=False)
|
||||
}
|
||||
return cases
|
||||
@@ -0,0 +1,332 @@
|
||||
"""Synthetic multi-stage graphs for the pass-through lineage rule.
|
||||
|
||||
Each builder reproduces one graph shape with neutral texts and generic names:
|
||||
a hires-fix or refiner chain (sampler, decode, image filter, model upscale,
|
||||
scale, encode, sampler), a three-stage Flux run whose last stage combines the
|
||||
base prompt with a style trigger, an image-edit batch of samplers with distinct
|
||||
prompts, and a chain whose prompt is generated at run time. They are shared by
|
||||
the resolver tests and the browser-port parity cases; no user file is involved.
|
||||
"""
|
||||
|
||||
import copy
|
||||
|
||||
POSITIVE = "a lighthouse on a cliff, golden hour"
|
||||
NEGATIVE = "blurry, lowres"
|
||||
OTHER = "a red bicycle in the rain"
|
||||
STYLE = "film grain, analog photo"
|
||||
|
||||
|
||||
def encode(text):
|
||||
return {"class_type": "CLIPTextEncode", "inputs": {"text": text, "clip": ["1", 1]}}
|
||||
|
||||
|
||||
def prompt_manager(text, prepend=""):
|
||||
return {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {
|
||||
"text": text,
|
||||
"category": "",
|
||||
"tags": "",
|
||||
"search_text": "",
|
||||
"prepend_text": prepend,
|
||||
"append_text": "",
|
||||
"clip": ["1", 1],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def ksampler(positive, negative, latent, denoise=1.0):
|
||||
return {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"seed": 7,
|
||||
"steps": 20,
|
||||
"cfg": 7.0,
|
||||
"sampler_name": "euler",
|
||||
"scheduler": "normal",
|
||||
"denoise": denoise,
|
||||
"model": ["1", 0],
|
||||
"positive": positive,
|
||||
"negative": negative,
|
||||
"latent_image": latent,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def hires_fix(second_positive=None, filter_class="Image Filter"):
|
||||
"""Two KSamplers joined in image space, like a hires-fix or refiner run.
|
||||
|
||||
Sampler 10 runs on a custom batch latent; its output goes through a tiled
|
||||
decode, an interactive image filter, a model upscale and a scale (sized by a
|
||||
resolution node reading the filtered image), then is encoded again for
|
||||
sampler 20 (denoise 0.25), which is decoded and saved.
|
||||
"""
|
||||
graph = {
|
||||
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"2": prompt_manager(POSITIVE),
|
||||
"3": encode(NEGATIVE),
|
||||
"4": {
|
||||
"class_type": "EmptyLatentBatch",
|
||||
"inputs": {"width": 1024, "height": 512, "batch_size": 4},
|
||||
},
|
||||
"10": ksampler(["2", 0], ["3", 0], ["4", 0]),
|
||||
"11": {
|
||||
"class_type": "VAEDecodeTiled",
|
||||
"inputs": {"tile_size": 512, "samples": ["10", 0], "vae": ["1", 2]},
|
||||
},
|
||||
"12": {
|
||||
"class_type": filter_class,
|
||||
"inputs": {"timeout": 600, "ontimeout": "send none", "images": ["11", 0]},
|
||||
},
|
||||
"13": {"class_type": "UpscaleModelLoader", "inputs": {"model_name": "x4"}},
|
||||
"14": {
|
||||
"class_type": "ImageUpscaleWithModel",
|
||||
"inputs": {"upscale_model": ["13", 0], "image": ["12", 0]},
|
||||
},
|
||||
"15": {
|
||||
"class_type": "ResolutionCalculator",
|
||||
"inputs": {"scale_factor": 1.5, "image": ["12", 0]},
|
||||
},
|
||||
"16": {
|
||||
"class_type": "ImageScale",
|
||||
"inputs": {
|
||||
"upscale_method": "nearest-exact",
|
||||
"width": ["15", 0],
|
||||
"height": ["15", 1],
|
||||
"crop": "disabled",
|
||||
"image": ["14", 0],
|
||||
},
|
||||
},
|
||||
"17": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {"pixels": ["16", 0], "vae": ["1", 2]},
|
||||
},
|
||||
"20": ksampler(["2", 0], ["3", 0], ["17", 0], denoise=0.25),
|
||||
"21": {
|
||||
"class_type": "VAEDecodeTiled",
|
||||
"inputs": {"tile_size": 512, "samples": ["20", 0], "vae": ["1", 2]},
|
||||
},
|
||||
"22": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {"filename_prefix": "hires", "images": ["21", 0]},
|
||||
},
|
||||
"23": {"class_type": "PlaySound|pysssss", "inputs": {"any": ["21", 0]}},
|
||||
}
|
||||
if second_positive is not None:
|
||||
graph["5"] = encode(second_positive)
|
||||
graph["20"]["inputs"]["positive"] = ["5", 0]
|
||||
return graph
|
||||
|
||||
|
||||
_COMPOSITE_INPUTS = {
|
||||
"ConditioningCombine": ("conditioning_1", "conditioning_2"),
|
||||
"ConditioningConcat": ("conditioning_to", "conditioning_from"),
|
||||
"ConditioningAverage": ("conditioning_to", "conditioning_from"),
|
||||
}
|
||||
|
||||
|
||||
def flux_three_stage(combined=(("30", 0), ("31", 0)), composite="ConditioningCombine"):
|
||||
"""FluxGuidance base, a refiner through an image filter, then a tiled upscale.
|
||||
|
||||
Samplers 40 and 50 share FluxGuidance(PromptManager 30) and a zeroed empty
|
||||
negative. Sampler 50 refines 40's filtered image (denoise 0.25). The final
|
||||
UltimateSDUpscale 60 reads 50's image, scaled down, and a positive that is
|
||||
``composite`` of ``combined`` (by default the base PromptManager and a
|
||||
style-trigger encoder 31).
|
||||
"""
|
||||
names = _COMPOSITE_INPUTS[composite]
|
||||
graph = {
|
||||
"1": {"class_type": "UNETLoader", "inputs": {"unet_name": "flux"}},
|
||||
"2": {"class_type": "DualCLIPLoader", "inputs": {"type": "flux"}},
|
||||
"3": {"class_type": "VAELoader", "inputs": {"vae_name": "ae"}},
|
||||
"30": prompt_manager(POSITIVE),
|
||||
"31": encode(STYLE),
|
||||
"32": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": "", "clip": ["2", 0]},
|
||||
},
|
||||
"33": {
|
||||
"class_type": "ConditioningZeroOut",
|
||||
"inputs": {"conditioning": ["32", 0]},
|
||||
},
|
||||
"34": {
|
||||
"class_type": "FluxGuidance",
|
||||
"inputs": {"guidance": 3.5, "conditioning": ["30", 0]},
|
||||
},
|
||||
"35": {
|
||||
"class_type": composite,
|
||||
"inputs": dict(zip(names, map(list, combined))),
|
||||
},
|
||||
"36": {
|
||||
"class_type": "EmptyLatentBatch",
|
||||
"inputs": {"width": 1080, "height": 1920, "batch_size": 4},
|
||||
},
|
||||
"40": ksampler(["34", 0], ["33", 0], ["36", 0]),
|
||||
"41": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {"samples": ["40", 0], "vae": ["3", 0]},
|
||||
},
|
||||
"42": {
|
||||
"class_type": "Image Filter",
|
||||
"inputs": {"timeout": 600, "images": ["41", 0]},
|
||||
},
|
||||
"43": {"class_type": "PreviewImage", "inputs": {"images": ["42", 0]}},
|
||||
"44": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {"pixels": ["42", 0], "vae": ["3", 0]},
|
||||
},
|
||||
"50": ksampler(["34", 0], ["33", 0], ["44", 0], denoise=0.25),
|
||||
"51": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {"samples": ["50", 0], "vae": ["3", 0]},
|
||||
},
|
||||
"52": {
|
||||
"class_type": "ImageScaleDownBy",
|
||||
"inputs": {"scale_by": 0.75, "images": ["51", 0]},
|
||||
},
|
||||
"53": {"class_type": "PreviewImage", "inputs": {"images": ["51", 0]}},
|
||||
"54": {
|
||||
"class_type": "Image Comparer (rgthree)",
|
||||
"inputs": {"image_a": ["51", 0], "image_b": ["42", 0]},
|
||||
},
|
||||
"60": {
|
||||
"class_type": "UltimateSDUpscale",
|
||||
"inputs": {
|
||||
"upscale_by": 2.0,
|
||||
"seed": 7,
|
||||
"steps": 4,
|
||||
"denoise": 0.35,
|
||||
"image": ["52", 0],
|
||||
"model": ["1", 0],
|
||||
"positive": ["35", 0],
|
||||
"negative": ["33", 0],
|
||||
"vae": ["3", 0],
|
||||
},
|
||||
},
|
||||
"61": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {"filename_prefix": "flux", "images": ["60", 0]},
|
||||
},
|
||||
}
|
||||
return graph
|
||||
|
||||
|
||||
def edit_batch(count=3):
|
||||
"""Image-edit samplers with distinct prompts, all saved with one prefix."""
|
||||
graph = {
|
||||
"1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"2": {"class_type": "LoadImage", "inputs": {"image": "ref.png"}},
|
||||
"3": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {"pixels": ["2", 0], "vae": ["1", 2]},
|
||||
},
|
||||
"4": {"class_type": "PrimitiveString", "inputs": {"value": "edit"}},
|
||||
}
|
||||
for index in range(count):
|
||||
base = 100 + index * 10
|
||||
graph[str(base)] = {
|
||||
"class_type": "TextEncodeQwenImageEdit",
|
||||
"inputs": {
|
||||
"prompt": f"edit number {index}: change the pose",
|
||||
"clip": ["1", 1],
|
||||
},
|
||||
}
|
||||
graph[str(base + 1)] = ksampler([str(base), 0], [str(base), 0], ["3", 0])
|
||||
graph[str(base + 2)] = {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {"samples": [str(base + 1), 0], "vae": ["1", 2]},
|
||||
}
|
||||
graph[str(base + 3)] = {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {"filename_prefix": ["4", 0], "images": [str(base + 2), 0]},
|
||||
}
|
||||
return graph
|
||||
|
||||
|
||||
def trigger_toggle(words="trig9word"):
|
||||
"""LoraManager's TriggerWord Toggle: its output is decided at run time."""
|
||||
return {
|
||||
"class_type": "TriggerWord Toggle (LoraManager)",
|
||||
"inputs": {
|
||||
"group_mode": True,
|
||||
"default_active": True,
|
||||
"toggle_trigger_words": {"__value__": [{"text": words, "active": True}]},
|
||||
"orinalMessage": words,
|
||||
"trigger_words": ["7", 2],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def hires_fix_with_trigger_words():
|
||||
"""``hires_fix`` whose PromptManager prepends LoRA trigger words (node 8)."""
|
||||
graph = hires_fix()
|
||||
graph["7"] = {
|
||||
"class_type": "Lora Loader (LoraManager)",
|
||||
"inputs": {"model": ["1", 0], "clip": ["1", 1], "text": "<lora:x:1>"},
|
||||
}
|
||||
graph["8"] = trigger_toggle()
|
||||
graph["2"]["inputs"]["prepend_text"] = ["8", 0]
|
||||
return graph
|
||||
|
||||
|
||||
def prompt_manager_text(text, prepend=""):
|
||||
"""PromptManagerText: a STRING PromptManager (prepend/main/append)."""
|
||||
return {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {
|
||||
"text": text,
|
||||
"category": "",
|
||||
"tags": "",
|
||||
"search_text": "",
|
||||
"prepend_text": prepend,
|
||||
"append_text": "",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def trigger_words_through(shape):
|
||||
"""``hires_fix_with_trigger_words`` with nodes between the toggle and sampler.
|
||||
|
||||
``text-encode``: PromptManagerText (prepend <- toggle) -> CLIPTextEncode.
|
||||
``chained``: a PromptManager whose ``text`` reads that PromptManagerText.
|
||||
``concat``: PromptManagerText -> StringConcatenate(..., "extra") -> encoder.
|
||||
"""
|
||||
graph = hires_fix_with_trigger_words()
|
||||
graph["9"] = prompt_manager_text(POSITIVE, ["8", 0])
|
||||
if shape == "chained":
|
||||
graph["2"]["inputs"].update(prepend_text="", text=["9", 0])
|
||||
elif shape == "text-encode":
|
||||
graph["2"] = {"class_type": "CLIPTextEncode", "inputs": {"text": ["9", 0]}}
|
||||
else:
|
||||
graph["19"] = {
|
||||
"class_type": "StringConcatenate",
|
||||
"inputs": {"string_a": ["9", 0], "string_b": "extra", "delimiter": ", "},
|
||||
}
|
||||
graph["2"] = {"class_type": "CLIPTextEncode", "inputs": {"text": ["19", 0]}}
|
||||
return graph
|
||||
|
||||
|
||||
def branches_with_and_without_trigger_words():
|
||||
"""Two saved branches: one PromptManager prepends trigger words, one does not."""
|
||||
graph = hires_fix_with_trigger_words()
|
||||
graph["9"] = prompt_manager(POSITIVE)
|
||||
graph["20"]["inputs"].update(positive=["9", 0], latent_image=["4", 0])
|
||||
return graph
|
||||
|
||||
|
||||
def generated_text_chain():
|
||||
"""``hires_fix`` whose positive text comes from a run-time text generator."""
|
||||
graph = hires_fix()
|
||||
graph["2"] = {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": ["6", 0], "clip": ["1", 1]},
|
||||
}
|
||||
graph["6"] = {
|
||||
"class_type": "TextGenerate",
|
||||
"inputs": {"prompt": "write a prompt", "model": ["1", 0]},
|
||||
}
|
||||
return graph
|
||||
|
||||
|
||||
def copied(graph):
|
||||
return copy.deepcopy(graph)
|
||||
@@ -30,6 +30,11 @@ FILES = "media_annotation_files"
|
||||
BYTES = "media_annotation_bytes"
|
||||
IMAGE_TAGS = "media_image_tags"
|
||||
TABLES = (METADATA, FILES, BYTES, IMAGE_TAGS)
|
||||
REVIEW_TABLES = (
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_review_metadata",
|
||||
)
|
||||
# The exact DDL the plan pins; the module must produce precisely these objects.
|
||||
DDL = {
|
||||
METADATA: (
|
||||
@@ -202,10 +207,14 @@ class AnnotationSchemaTests(unittest.TestCase):
|
||||
)
|
||||
|
||||
def drop_group(self, path):
|
||||
"""Remove the group from a closed database file (an old backup)."""
|
||||
"""Remove the group from a closed database file (an old backup).
|
||||
|
||||
The review queue group cites annotation bytes and predates nothing, so
|
||||
an old backup lacks it as well.
|
||||
"""
|
||||
with closing(sqlite3.connect(path)) as raw, raw:
|
||||
raw.execute("PRAGMA foreign_keys=OFF")
|
||||
for table in reversed(TABLES):
|
||||
for table in REVIEW_TABLES + tuple(reversed(TABLES)):
|
||||
raw.execute(f"DROP TABLE {table}")
|
||||
|
||||
def test_fresh_install_and_reopen_is_idempotent(self):
|
||||
|
||||
+45
-3
@@ -52,6 +52,7 @@ class AdminAPITestCase(AioHTTPTestCase):
|
||||
self.outside_dir = Path(self.tmpdir) / "outside"
|
||||
self.outside_dir.mkdir()
|
||||
|
||||
self.addAsyncCleanup(self._drain_and_remove)
|
||||
self.config_path = os.path.join(self.tmpdir, "config.json")
|
||||
self._orig_env = {
|
||||
key: os.environ.pop(key, None) for key in (CONFIG_PATH_ENV, EXTRA_ROOTS_ENV)
|
||||
@@ -89,6 +90,26 @@ class AdminAPITestCase(AioHTTPTestCase):
|
||||
sys.modules.pop("folder_paths", None)
|
||||
else:
|
||||
sys.modules["folder_paths"] = self._orig_folder_paths
|
||||
|
||||
async def _drain_and_remove(self):
|
||||
"""The last cleanup: no job outlives the temporary database.
|
||||
|
||||
Registered first, so it runs after every cleanup a test added (which
|
||||
may release its own gated jobs). A job still running then, such as a
|
||||
scan a failed assertion never awaited, is cancelled and awaited
|
||||
before the database and the tree are removed.
|
||||
"""
|
||||
api = getattr(self, "api", None)
|
||||
jobs = api._jobs.list() if api is not None else []
|
||||
tasks = [
|
||||
job.task for job in jobs if job.task is not None and not job.task.done()
|
||||
]
|
||||
for task in tasks:
|
||||
task.cancel()
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
if api is not None:
|
||||
# A scan of an ambiguous file captures it for Needs review.
|
||||
api.close_annotation_captures()
|
||||
PromptModel.close_all_instances() # restore may have swapped api.db
|
||||
assert_closed(self, self.tmpdir)
|
||||
shutil.rmtree(self.tmpdir, ignore_errors=True)
|
||||
@@ -868,6 +889,23 @@ class TestDeleteDuplicatesEndpoint(AdminAPITestCase):
|
||||
self.assertFalse(target.exists())
|
||||
self.assertTrue(thumb_dir.exists())
|
||||
|
||||
async def test_deleted_files_leave_the_cached_gallery_listing(self):
|
||||
# The listing is cached for 30 s; a delete must not leave the gallery
|
||||
# listing (and counting) a file that is gone.
|
||||
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
|
||||
for name in ("keep.png", "gone.png"):
|
||||
(self.output_dir / name).write_bytes(b"same bytes")
|
||||
|
||||
async def listed():
|
||||
resp = await self.client.get("/prompt_manager/images/output")
|
||||
data = await resp.json()
|
||||
return data["total"], sorted(i["filename"] for i in data["images"])
|
||||
|
||||
self.assertEqual(await listed(), (2, ["gone.png", "keep.png"]))
|
||||
resp = await self._post({"image_paths": ["gone.png"]})
|
||||
self.assertEqual((await resp.json())["deleted_count"], 1)
|
||||
self.assertEqual(await listed(), (1, ["keep.png"]))
|
||||
|
||||
def test_unexpected_error_per_file_is_reported_without_path(self):
|
||||
self.api._delete_one_duplicate = lambda *a: (_ for _ in ()).throw(
|
||||
PermissionError(13, "Permission denied", os.path.join(self.tmpdir, "x.png"))
|
||||
@@ -2204,7 +2242,9 @@ class TestActivityJobs(AdminAPITestCase):
|
||||
for name in names:
|
||||
(self.output_dir / name).write_bytes(b"not an image")
|
||||
|
||||
def fake_ingest(media_file, metadata, detailed=False):
|
||||
def fake_ingest(
|
||||
media_file, metadata, detailed=False, ambiguous=None, untraced=None
|
||||
):
|
||||
key = Path(media_file).relative_to(self.output_dir).as_posix()
|
||||
outcome = names[key]
|
||||
if isinstance(outcome, Exception):
|
||||
@@ -2333,8 +2373,10 @@ class TestActivityJobs(AdminAPITestCase):
|
||||
first_done, release = threading.Event(), threading.Event()
|
||||
original = self.api._ingest_scan_batch_sync
|
||||
|
||||
def gated(batch_results, roots=(), outcomes=None):
|
||||
counts = original(batch_results, roots=roots, outcomes=outcomes)
|
||||
def gated(batch_results, roots=(), outcomes=None, review=None):
|
||||
counts = original(
|
||||
batch_results, roots=roots, outcomes=outcomes, review=review
|
||||
)
|
||||
first_done.set()
|
||||
release.wait(5)
|
||||
return counts
|
||||
|
||||
@@ -1045,7 +1045,15 @@ class TestFailures(AnnotationAPITestCase):
|
||||
await self.api.cleanup_annotation_captures(None)
|
||||
self.assertFalse(os.path.exists(directory))
|
||||
self.assertNotIn(identity, observers._restore_observers)
|
||||
# A later observation starts a fresh store.
|
||||
# The shutdown cleanup is final: a later observation (or a monitor
|
||||
# recording still in flight) creates no new store or directory.
|
||||
status, body = await self.observe({"target": self.target("sub/a.png")})
|
||||
self.assertEqual((status, body["code"]), (500, "capture-error"), body)
|
||||
self.assertIsNone(self.api._captures)
|
||||
self.assertNotIn(identity, observers._restore_observers)
|
||||
# An explicit, non-final close still lets a later observation start
|
||||
# a fresh store (the tests' own store swaps rely on it).
|
||||
self.api._captures_closed = False
|
||||
await self.observe_ok()
|
||||
self.assertNotEqual(self.api._captures.directory, directory)
|
||||
|
||||
|
||||
@@ -471,6 +471,19 @@ class TestUiRoutes(CoreRoutesTestCase):
|
||||
self.assertIn(("POST", "/prompt_manager/jobs/scan"), table)
|
||||
self.assertIn(("POST", "/prompt_manager/jobs/image-tags"), table)
|
||||
|
||||
async def test_review_routes_are_registered(self):
|
||||
table = {
|
||||
(route.method, route.resource.canonical)
|
||||
for route in self.app.router.routes()
|
||||
if route.resource is not None
|
||||
}
|
||||
for entry in (
|
||||
("GET", "/prompt_manager/review/items"),
|
||||
("GET", "/prompt_manager/review/items/{item_id}"),
|
||||
("POST", "/prompt_manager/review/items/{item_id}/decision"),
|
||||
):
|
||||
self.assertIn(entry, table)
|
||||
|
||||
|
||||
class TestStaticRoutes(CoreRoutesTestCase):
|
||||
|
||||
@@ -486,6 +499,30 @@ class TestStaticRoutes(CoreRoutesTestCase):
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIn("application/javascript", resp.headers["Content-Type"])
|
||||
|
||||
async def test_inspection_semantics_table_is_the_one_json_served(self):
|
||||
"""Local Inspect reads the node table the server resolves with (#249 PR L3).
|
||||
|
||||
Only that exact file is served; every other JSON stays forbidden.
|
||||
"""
|
||||
resp = await self.client.request(
|
||||
"GET", "/prompt_manager/js/inspection-semantics.json"
|
||||
)
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIn("application/json", resp.headers["Content-Type"])
|
||||
table = Path(__file__).resolve().parents[1] / "web" / "js"
|
||||
self.assertEqual(
|
||||
await resp.read(), (table / "inspection-semantics.json").read_bytes()
|
||||
)
|
||||
for path in (
|
||||
"js/shared/toolkit/provenance.json",
|
||||
"lib/viewerjs/provenance.json",
|
||||
"lib/inspection-semantics.json",
|
||||
"js/workspace/..%2Finspection-semantics.json",
|
||||
"js/INSPECTION-SEMANTICS.json",
|
||||
):
|
||||
resp = await self.client.request("GET", f"/prompt_manager/{path}")
|
||||
self.assertEqual(resp.status, 403, path)
|
||||
|
||||
async def test_lib_served(self):
|
||||
web_dir = self._fake_root()
|
||||
(web_dir / "lib" / "x.css").write_text("body{}")
|
||||
@@ -578,6 +615,28 @@ class TestStaticRoutes(CoreRoutesTestCase):
|
||||
resp = await self.client.request("GET", "/prompt_manager/lib/link.js")
|
||||
self.assertEqual(resp.status, 403)
|
||||
|
||||
async def test_inspection_semantics_symlink_escaping_web_dir_is_403(self):
|
||||
"""The exact-path JSON gets the same realpath containment as any file."""
|
||||
web_dir = self._fake_root()
|
||||
secret = Path(self.tmpdir) / "secret.json"
|
||||
secret.write_text('{"secret": true}')
|
||||
try:
|
||||
os.symlink(secret, web_dir / "js" / "inspection-semantics.json")
|
||||
except (OSError, NotImplementedError):
|
||||
self.skipTest("symlinks not available")
|
||||
resp = await self.client.request(
|
||||
"GET", "/prompt_manager/js/inspection-semantics.json"
|
||||
)
|
||||
self.assertEqual(resp.status, 403)
|
||||
self.assertNotIn("secret", await resp.text())
|
||||
|
||||
async def test_inspection_semantics_missing_is_404(self):
|
||||
self._fake_root()
|
||||
resp = await self.client.request(
|
||||
"GET", "/prompt_manager/js/inspection-semantics.json"
|
||||
)
|
||||
self.assertEqual(resp.status, 404)
|
||||
|
||||
async def test_empty_path_is_not_served(self):
|
||||
for prefix in ("lib", "js"):
|
||||
resp = await self.client.request("GET", f"/prompt_manager/{prefix}/")
|
||||
|
||||
@@ -1643,6 +1643,62 @@ class TestPromptInspector(ImageAPITestCase):
|
||||
str(path), False, report_ambiguity=True
|
||||
)
|
||||
|
||||
async def test_unhashable_workflow_node_type_is_skipped(self):
|
||||
"""A UI workflow node whose ``type`` is a list or dict is skipped (#286).
|
||||
|
||||
The R1 check looks each node's ``type`` up among subgraph IDs; an
|
||||
unhashable ``type`` used to raise there and the route answered 500.
|
||||
Such a node can be neither a subgraph instance nor a PromptManager
|
||||
node, so the walk skips it and a prompt that resolves still resolves.
|
||||
"""
|
||||
source = graph()
|
||||
source["1"]["class_type"] = "PromptManager"
|
||||
for kind in (["PromptManager"], {"name": "PromptManager"}):
|
||||
with self.subTest(kind=type(kind).__name__):
|
||||
workflow = json.dumps(
|
||||
{"nodes": [{"id": 1, "type": kind, "widgets_values": ["x"]}]}
|
||||
)
|
||||
png(
|
||||
self.output_dir / "image.png",
|
||||
prompt=json.dumps(source),
|
||||
workflow=workflow,
|
||||
)
|
||||
response, result = await self.inspect()
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(result["resolution"]["state"], "resolved")
|
||||
self.assertEqual(result["resolution"]["sampler_id"], "3")
|
||||
self.assertEqual(result["prompts"]["positive"]["status"], "resolved")
|
||||
self.assertEqual(result["prompts"]["positive"]["text"], "portrait")
|
||||
self.assertEqual(result["prompts"]["negative"]["text"], "blurry")
|
||||
self.assertNotIn("unhashable", json.dumps(result))
|
||||
|
||||
async def test_unhashable_graph_class_type_is_not_a_preview_output(self):
|
||||
"""A list or object ``class_type`` never names a preview output (#286).
|
||||
|
||||
With several samplers the inspector first drops stages that reach only
|
||||
previews; looking such a ``class_type`` up among the preview classes
|
||||
used to raise, and the route answered 500.
|
||||
"""
|
||||
for kind in (["PreviewImage"], {"name": "PreviewImage"}):
|
||||
with self.subTest(kind=type(kind).__name__):
|
||||
source = graph()
|
||||
source["9"] = {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {"positive": ["8", 0], "negative": ["2", 0]},
|
||||
}
|
||||
source["8"] = {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": "cat"},
|
||||
}
|
||||
source["10"] = {"class_type": kind, "inputs": {"images": ["9", 0]}}
|
||||
png(self.output_dir / "image.png", prompt=json.dumps(source))
|
||||
response, result = await self.inspect()
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(
|
||||
result["resolution"]["reason"], "Multiple sampler branches"
|
||||
)
|
||||
self.assertEqual(result["resolution"]["state"], "ambiguous")
|
||||
|
||||
|
||||
class TestInspectorImageTags(ImageAPITestCase):
|
||||
"""B3: the inspector envelope's ``image_tags`` facet (spec amendment S1)."""
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,234 @@
|
||||
"""A slow page-workflow reset reports where the server waited.
|
||||
|
||||
CI run 39904288262 had one ``/__test__/reset`` wait 16 s with nothing in the
|
||||
server's stderr. tests/browser/slow_reset_watch.py reports any reset past
|
||||
SLOW_RESET_SECONDS with every thread's stack, taken while it still waits.
|
||||
These tests hold a real SQLite write lock behind a real test server's reset.
|
||||
"""
|
||||
|
||||
import glob
|
||||
import io
|
||||
import os
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from tests.browser import slow_reset_watch
|
||||
from tests.browser.slow_reset_watch import REPORT_PREFIX, ResetWatch, is_reset_path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
SERVER_TIMEOUT_S = 60
|
||||
HOLD_SECONDS = slow_reset_watch.SLOW_RESET_SECONDS + 4
|
||||
# The reset may take a little longer than the hold, never the 30 s busy timeout.
|
||||
RESET_MARGIN_SECONDS = 10
|
||||
# While the reset waits, the server still answers other requests this fast.
|
||||
# Reading the waiting connection from the report's thread froze it (the GIL is
|
||||
# held while SQLite's connection mutex is awaited) until the lock was released.
|
||||
RESPONSIVE_SECONDS = 2.0
|
||||
PROBE_INTERVAL_SECONDS = 0.1
|
||||
|
||||
|
||||
def blocked_here(gate: threading.Event) -> None:
|
||||
gate.wait()
|
||||
|
||||
|
||||
class TestResetWatch(unittest.TestCase):
|
||||
def test_reset_paths_are_the_test_reset_routes(self):
|
||||
self.assertTrue(is_reset_path("/__test__/reset"))
|
||||
self.assertTrue(is_reset_path("/__test__/tagging/reset"))
|
||||
self.assertFalse(is_reset_path("/prompt_manager/reset"))
|
||||
self.assertFalse(is_reset_path("/__test__/reset/extra"))
|
||||
|
||||
def test_a_stalled_reset_reports_state_and_stacks_while_it_waits(self):
|
||||
stream = io.StringIO()
|
||||
reported = threading.Event()
|
||||
|
||||
class Stream(io.StringIO):
|
||||
def flush(self):
|
||||
reported.set()
|
||||
|
||||
stream = Stream()
|
||||
gate = threading.Event()
|
||||
worker = threading.Thread(target=blocked_here, args=(gate,), name="held-up")
|
||||
worker.start()
|
||||
self.addCleanup(worker.join)
|
||||
self.addCleanup(gate.set)
|
||||
watch = ResetWatch(
|
||||
"POST /__test__/reset",
|
||||
lambda: ["database: 1 connections"],
|
||||
threshold=0.05,
|
||||
stream=stream,
|
||||
).start()
|
||||
self.assertTrue(reported.wait(SERVER_TIMEOUT_S))
|
||||
report = stream.getvalue().splitlines()
|
||||
self.assertTrue(all(line.startswith(REPORT_PREFIX) for line in report))
|
||||
text = "\n".join(report)
|
||||
self.assertIn("POST /__test__/reset still running after", text)
|
||||
self.assertIn("database: 1 connections", text)
|
||||
self.assertIn(f"thread 0x{worker.ident:016x} is held-up", text)
|
||||
self.assertIn("in blocked_here", text)
|
||||
|
||||
watch.record_call("seed", 0.25, 1.5)
|
||||
watch.finish("answered 200")
|
||||
self.assertIn("POST /__test__/reset answered 200 after", stream.getvalue())
|
||||
self.assertIn(
|
||||
"executor call seed: queued 0.250s, ran 1.500s", stream.getvalue()
|
||||
)
|
||||
|
||||
def test_a_fast_reset_writes_nothing_and_stops_its_timer(self):
|
||||
stream = io.StringIO()
|
||||
watch = ResetWatch("reset", lambda: [], threshold=60, stream=stream).start()
|
||||
watch.finish("answered 200")
|
||||
self.assertTrue(watch._timer.finished.is_set())
|
||||
self.assertEqual(stream.getvalue(), "")
|
||||
|
||||
def test_a_failing_state_probe_still_reports_the_stacks(self):
|
||||
def broken():
|
||||
raise RuntimeError("no state")
|
||||
|
||||
reported = threading.Event()
|
||||
|
||||
class Stream(io.StringIO):
|
||||
def flush(self):
|
||||
reported.set()
|
||||
|
||||
stream = Stream()
|
||||
ResetWatch("reset", broken, threshold=0.01, stream=stream).start()
|
||||
self.assertTrue(reported.wait(SERVER_TIMEOUT_S))
|
||||
self.assertIn("state unavailable: RuntimeError('no state')", stream.getvalue())
|
||||
self.assertIn("most recent call first", stream.getvalue())
|
||||
|
||||
|
||||
class TestSlowResetOnTheServer(unittest.TestCase):
|
||||
"""A reset behind another process's write lock, on tests/browser/server.py."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
env = {**os.environ, "TMPDIR": self.tmp.name}
|
||||
env.update(TEMP=self.tmp.name, TMP=self.tmp.name)
|
||||
self.stderr = tempfile.TemporaryFile()
|
||||
self.addCleanup(self.stderr.close)
|
||||
process = subprocess.Popen(
|
||||
[sys.executable, "-m", "tests.browser.server", "--port", "0"],
|
||||
cwd=REPO_ROOT,
|
||||
env=env,
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=self.stderr,
|
||||
)
|
||||
|
||||
def stop():
|
||||
if process.poll() is None:
|
||||
process.kill()
|
||||
process.wait(timeout=SERVER_TIMEOUT_S)
|
||||
for stream in (process.stdin, process.stdout):
|
||||
stream.close()
|
||||
|
||||
self.addCleanup(stop)
|
||||
line = process.stdout.readline().decode()
|
||||
self.assertTrue(line.startswith("PM_TEST_SERVER_PORT="), line)
|
||||
self.base = f"http://127.0.0.1:{int(line.split('=', 1)[1])}"
|
||||
deadline = time.monotonic() + SERVER_TIMEOUT_S
|
||||
while True:
|
||||
try:
|
||||
with urllib.request.urlopen( # nosec B310
|
||||
self.base + "/prompt_manager/test", timeout=1
|
||||
):
|
||||
break
|
||||
except OSError:
|
||||
self.assertIsNone(process.poll(), "the server exited early")
|
||||
self.assertLess(time.monotonic(), deadline, "never ready")
|
||||
time.sleep(0.1)
|
||||
[self.db_path] = glob.glob(
|
||||
os.path.join(self.tmp.name, "pm-browser-*", "browser.db")
|
||||
)
|
||||
|
||||
def reset(self) -> tuple[int, float]:
|
||||
request = urllib.request.Request(
|
||||
self.base + "/__test__/reset",
|
||||
data=b"{}",
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
start = time.monotonic()
|
||||
with urllib.request.urlopen( # nosec B310
|
||||
request, timeout=SERVER_TIMEOUT_S
|
||||
) as response:
|
||||
return response.status, time.monotonic() - start
|
||||
|
||||
def server_stderr(self) -> str:
|
||||
self.stderr.seek(0)
|
||||
return self.stderr.read().decode("utf-8", "replace")
|
||||
|
||||
def test_a_reset_held_behind_a_write_lock_names_the_wait_and_ends_with_it(self):
|
||||
self.assertEqual(self.reset()[0], 200, self.server_stderr())
|
||||
self.assertNotIn(REPORT_PREFIX, self.server_stderr())
|
||||
|
||||
held = threading.Event()
|
||||
|
||||
def hold_write_lock():
|
||||
holder = sqlite3.connect(self.db_path, timeout=SERVER_TIMEOUT_S)
|
||||
try:
|
||||
holder.execute("BEGIN IMMEDIATE")
|
||||
held.set()
|
||||
time.sleep(HOLD_SECONDS)
|
||||
holder.rollback()
|
||||
finally:
|
||||
holder.close()
|
||||
|
||||
lock = threading.Thread(target=hold_write_lock)
|
||||
lock.start()
|
||||
self.addCleanup(lock.join)
|
||||
self.assertTrue(held.wait(SERVER_TIMEOUT_S))
|
||||
outcome = []
|
||||
resetting = threading.Thread(target=lambda: outcome.append(self.reset()))
|
||||
resetting.start()
|
||||
self.addCleanup(resetting.join)
|
||||
slowest = 0.0
|
||||
while resetting.is_alive():
|
||||
start = time.monotonic()
|
||||
with urllib.request.urlopen( # nosec B310
|
||||
self.base + "/prompt_manager/test", timeout=SERVER_TIMEOUT_S
|
||||
):
|
||||
slowest = max(slowest, time.monotonic() - start)
|
||||
time.sleep(PROBE_INTERVAL_SECONDS)
|
||||
resetting.join()
|
||||
lock.join()
|
||||
self.assertEqual(len(outcome), 1, self.server_stderr())
|
||||
[(status, elapsed)] = outcome
|
||||
|
||||
self.assertLess(slowest, RESPONSIVE_SECONDS)
|
||||
self.assertEqual(status, 200)
|
||||
self.assertGreaterEqual(elapsed, HOLD_SECONDS - 0.5)
|
||||
self.assertLess(elapsed, HOLD_SECONDS + RESET_MARGIN_SECONDS)
|
||||
report = [
|
||||
line[len(REPORT_PREFIX) :]
|
||||
for line in self.server_stderr().splitlines()
|
||||
if line.startswith(REPORT_PREFIX)
|
||||
]
|
||||
text = "\n".join(report)
|
||||
self.assertIn("POST /__test__/reset still running after", text)
|
||||
self.assertIn("default executor:", text)
|
||||
if sys.platform.startswith("linux"):
|
||||
# SQLite's busy handler sleeps between its retries.
|
||||
self.assertRegex(
|
||||
text, r"kernel: asyncio_\d+ \(0x[0-9a-f]{16}\) state \w, waiting in "
|
||||
)
|
||||
# The plain reset's seed() waits on its first DELETE.
|
||||
self.assertRegex(text, r"line \d+ in seed\n")
|
||||
self.assertIn("POST /__test__/reset answered 200 after", text)
|
||||
self.assertRegex(
|
||||
text,
|
||||
r"executor call create_app.<locals>.seed: queued \d+\.\d+s, ran \d+\.\d+s",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -68,6 +68,55 @@ class CardSeed(unittest.TestCase):
|
||||
self.assertIn(expected, (111, 222))
|
||||
self.assertEqual(info_seed(info), str(expected))
|
||||
|
||||
def test_a_chain_the_inspector_names_one_stage_of_shows_no_differing_seed(self):
|
||||
# The inspector shows the named final stage's seed (D19, pass-through
|
||||
# chains); the parser shows another stage's. Never two seeds: none.
|
||||
extra = {"2": {"class_type": "EmptyLatentImage", "inputs": {}}}
|
||||
extra["20"] = {"class_type": "VAEDecode", "inputs": {"samples": ["10", 0]}}
|
||||
extra["21"] = {
|
||||
"class_type": "FaceDetailer",
|
||||
"inputs": {
|
||||
"seed": 333,
|
||||
"image": ["20", 0],
|
||||
"model": ["4", 0],
|
||||
"positive": ["6", 0],
|
||||
"negative": ["7", 0],
|
||||
},
|
||||
}
|
||||
info = graph(seeded(111), extra=extra)
|
||||
self.assertEqual(extract_from_records(info["prompt"], None)["seed"], 111)
|
||||
self.assertIsNone(info_seed(info))
|
||||
# The same chain on one seed keeps it; so does a refiner over a decode.
|
||||
extra["21"]["inputs"]["seed"] = 111
|
||||
self.assertEqual(info_seed(graph(seeded(111), extra=extra)), "111")
|
||||
refiner = seeded(222, latent_image=["22", 0])
|
||||
extra["22"] = {"class_type": "VAEEncode", "inputs": {"pixels": ["20", 0]}}
|
||||
del extra["21"]
|
||||
self.assertIsNone(info_seed(graph(seeded(111), refiner, extra=extra)))
|
||||
refiner["inputs"]["seed"] = 111
|
||||
self.assertEqual(info_seed(graph(seeded(111), refiner, extra=extra)), "111")
|
||||
|
||||
def test_a_seed_that_agrees_with_the_named_stage_is_kept(self):
|
||||
# Review I3: a saved sampler (seed 7) beside a preview-only sampler
|
||||
# (seed 5): the parser and the inspector both show 7, as on main.
|
||||
extra = {"2": {"class_type": "EmptyLatentImage", "inputs": {}}}
|
||||
extra["20"] = {"class_type": "VAEDecode", "inputs": {"samples": ["10", 0]}}
|
||||
extra["21"] = {"class_type": "SaveImage", "inputs": {"images": ["20", 0]}}
|
||||
extra["30"] = {"class_type": "VAEDecode", "inputs": {"samples": ["11", 0]}}
|
||||
extra["31"] = {"class_type": "PreviewImage", "inputs": {"images": ["30", 0]}}
|
||||
info = graph(seeded(7), seeded(5), extra=extra)
|
||||
self.assertEqual(info_seed(info), "7")
|
||||
|
||||
def test_a_named_ambiguous_chain_keeps_the_parser_seed(self):
|
||||
# Review M2: multistage-differs names a sampler, but its facts merge,
|
||||
# and the inspector fills the parser's seed: the card keeps it.
|
||||
extra = {"2": {"class_type": "EmptyLatentImage", "inputs": {}}}
|
||||
extra["8"] = encoder("another scene")
|
||||
extra["20"] = {"class_type": "VAEDecode", "inputs": {"samples": ["10", 0]}}
|
||||
extra["22"] = {"class_type": "VAEEncode", "inputs": {"pixels": ["20", 0]}}
|
||||
refiner = ksampler(("8", 0), ("7", 0), seed=222, latent_image=["22", 0])
|
||||
self.assertEqual(info_seed(graph(seeded(111), refiner, extra=extra)), "111")
|
||||
|
||||
def test_card_seed_linked_input_follows_shared_parser(self):
|
||||
primitive = {"class_type": "PrimitiveNode", "inputs": {"value": 555}}
|
||||
info = graph(seeded(["20", 0]), extra={"20": primitive})
|
||||
@@ -207,6 +256,7 @@ class CardSeed(unittest.TestCase):
|
||||
self.assertIsNone(parsed_positive(unparsable))
|
||||
|
||||
def test_every_shared_case_reads_the_projected_graphs(self):
|
||||
withheld = []
|
||||
for name, case in CASES.items():
|
||||
with self.subTest(case=name):
|
||||
info = {k: v for k, v in case.items() if v is not None}
|
||||
@@ -225,11 +275,18 @@ class CardSeed(unittest.TestCase):
|
||||
except GenerationParamsError:
|
||||
self.assertIsNone(params)
|
||||
continue
|
||||
if name == "shared-prompt-pipeline":
|
||||
# The inspector names the detailer (seed 333); the parser
|
||||
# reads the base sampler (111): the card shows none.
|
||||
self.assertEqual(expected["seed"], 111)
|
||||
expected = {**expected, "seed": None}
|
||||
withheld.append(name)
|
||||
self.assertEqual(params, expected)
|
||||
# Prompt text is the same as from a plain parse of the records.
|
||||
if len(values) == len(info):
|
||||
raw = extract_from_records(case["prompt"], case["workflow"])
|
||||
self.assertEqual(params["positivePrompt"], raw["positivePrompt"])
|
||||
self.assertEqual(withheld, ["shared-prompt-pipeline"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -24,6 +24,9 @@ from database.operations import PromptDatabase
|
||||
from tests.open_handles import assert_closed
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
# A bound on thread hand-offs that only a hung test reaches (never a pacing timer).
|
||||
THREAD_WAIT_SECONDS = 120
|
||||
|
||||
|
||||
def _count_prompts(path):
|
||||
"""Row count read through a brand-new connection (closed before return)."""
|
||||
@@ -350,7 +353,13 @@ class TestRestoreFromFile(BackupTestCase):
|
||||
def worker():
|
||||
self.db.search_prompts(text="old") # opens this thread's connection
|
||||
ready.set()
|
||||
proceed.wait(5)
|
||||
# Wait for the restore itself, not for a timer: a stalled Windows
|
||||
# runner once took longer than 5 s to restore, and the worker then
|
||||
# read the old database before the restore had run.
|
||||
if not proceed.wait(THREAD_WAIT_SECONDS):
|
||||
results["error"] = "the restore did not finish"
|
||||
self.db.close()
|
||||
return
|
||||
try:
|
||||
results["texts"] = [p["text"] for p in self.db.search_prompts()]
|
||||
except Exception as exc: # noqa: BLE001 - reported through results
|
||||
@@ -360,12 +369,15 @@ class TestRestoreFromFile(BackupTestCase):
|
||||
|
||||
thread = threading.Thread(target=worker)
|
||||
thread.start()
|
||||
ready.wait(5)
|
||||
self.assertTrue(ready.wait(THREAD_WAIT_SECONDS))
|
||||
|
||||
self.db.model.restore_from_file(src)
|
||||
proceed.set()
|
||||
thread.join(5)
|
||||
try:
|
||||
self.db.model.restore_from_file(src)
|
||||
finally:
|
||||
proceed.set()
|
||||
thread.join(THREAD_WAIT_SECONDS)
|
||||
|
||||
self.assertFalse(thread.is_alive())
|
||||
self.assertNotIn("error", results)
|
||||
self.assertEqual(results["texts"], ["restored"])
|
||||
self.assertEqual([p["text"] for p in self.db.search_prompts()], ["restored"])
|
||||
@@ -454,6 +466,9 @@ class TestRestoreFromFile(BackupTestCase):
|
||||
|
||||
def test_restore_old_backup_without_annotations(self):
|
||||
tables = (
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_review_metadata",
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
@@ -490,6 +505,70 @@ class TestRestoreFromFile(BackupTestCase):
|
||||
0,
|
||||
)
|
||||
|
||||
def test_restore_annotations_era_backup_without_review_queue(self):
|
||||
tables = (
|
||||
"media_review_decisions",
|
||||
"media_review_items",
|
||||
"media_review_metadata",
|
||||
)
|
||||
src = self._make_source_db(["predates the review queue"])
|
||||
with closing(sqlite3.connect(src)) as conn:
|
||||
for table in tables:
|
||||
conn.execute(f"DROP TABLE {table}")
|
||||
conn.commit()
|
||||
self.assertTrue(PromptModel.verify_database_file(src)[0])
|
||||
ok, reason = PromptModel.verify_database_file(src, require_library_state=True)
|
||||
self.assertFalse(ok)
|
||||
self.assertEqual(reason, "Missing review queue schema")
|
||||
|
||||
self.db.model.restore_from_file(src)
|
||||
|
||||
self.assertEqual(_texts(self.path), ["predates the review queue"])
|
||||
with closing(sqlite3.connect(self.path)) as conn:
|
||||
self.assertEqual(
|
||||
conn.execute("SELECT * FROM media_review_metadata").fetchall(),
|
||||
[(1, 1, 0)],
|
||||
)
|
||||
self.assertEqual(
|
||||
conn.execute("SELECT COUNT(*) FROM media_review_items").fetchone()[0],
|
||||
0,
|
||||
)
|
||||
ok, reason = PromptModel.verify_database_file(
|
||||
self.path, require_library_state=True
|
||||
)
|
||||
self.assertTrue(ok, reason)
|
||||
with closing(sqlite3.connect(src)) as conn: # the source is untouched
|
||||
self.assertEqual(
|
||||
conn.execute(
|
||||
"SELECT COUNT(*) FROM sqlite_master WHERE name LIKE 'media_review%'"
|
||||
).fetchone()[0],
|
||||
0,
|
||||
)
|
||||
|
||||
def test_restore_rejects_review_queue_without_annotations(self):
|
||||
src = self._make_source_db(["review without annotations"])
|
||||
with closing(sqlite3.connect(src)) as conn:
|
||||
conn.execute("PRAGMA foreign_keys=OFF")
|
||||
for table in (
|
||||
"media_image_tags",
|
||||
"media_annotation_bytes",
|
||||
"media_annotation_files",
|
||||
"media_annotation_metadata",
|
||||
):
|
||||
conn.execute(f"DROP TABLE {table}")
|
||||
conn.commit()
|
||||
before = Path(src).read_bytes()
|
||||
live_before = _texts(self.path)
|
||||
ok, reason = PromptModel.verify_database_file(src)
|
||||
self.assertFalse(ok)
|
||||
self.assertIn("image annotation", reason)
|
||||
|
||||
with self.assertRaises(ValueError):
|
||||
self.db.model.restore_from_file(src)
|
||||
|
||||
self.assertEqual(Path(src).read_bytes(), before)
|
||||
self.assertEqual(_texts(self.path), live_before)
|
||||
|
||||
def test_restore_preserves_favorites_and_closes_handles(self):
|
||||
self.db.set_image_favorite("output", "live.png")
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
|
||||
@@ -0,0 +1,614 @@
|
||||
"""The read-only health check (System › Overview and Maintenance).
|
||||
|
||||
``run_health_checks`` reads one snapshot on its own ``mode=ro`` connection:
|
||||
SQLite's integrity and foreign-key checks, stored prompt hashes against
|
||||
``utils.hashing``, duplicate pair rows, the legacy consistency queries, image
|
||||
links whose file is gone (only under readable sources) and the Needs-review
|
||||
count when one is supplied. It never writes, and it never turns a check it
|
||||
could not run into a count.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import ntpath
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from database import health # noqa: E402
|
||||
from database.operations import PromptDatabase # noqa: E402
|
||||
from utils.hashing import generate_prompt_hash # noqa: E402
|
||||
|
||||
try:
|
||||
from tests.open_handles import assert_closed # noqa: E402
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from open_handles import assert_closed # noqa: E402
|
||||
|
||||
CHECK_IDS = [
|
||||
"integrity",
|
||||
"foreign_keys",
|
||||
"canonical_hashes",
|
||||
"pair_uniqueness",
|
||||
"missing_files",
|
||||
"ambiguous_metadata",
|
||||
"consistency",
|
||||
]
|
||||
|
||||
|
||||
def _digest(path):
|
||||
if not os.path.exists(path):
|
||||
return None
|
||||
with open(path, "rb") as handle:
|
||||
return hashlib.sha256(handle.read()).hexdigest()
|
||||
|
||||
|
||||
class HealthTestCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.root = Path(os.path.realpath(self.temp.name))
|
||||
self.output = self.root / "output"
|
||||
self.output.mkdir()
|
||||
self.db_path = str(self.root / "prompts.db")
|
||||
self.db = PromptDatabase(self.db_path)
|
||||
|
||||
def tearDown(self):
|
||||
self.db.close_all()
|
||||
assert_closed(self, self.root)
|
||||
self.temp.cleanup()
|
||||
|
||||
def save(self, text, **metadata):
|
||||
return self.db.save_prompt(
|
||||
text, prompt_hash=generate_prompt_hash(text), **metadata
|
||||
)
|
||||
|
||||
def image(self, prompt_id, name, *, exists=True, folder=None):
|
||||
folder = Path(folder or self.output)
|
||||
path = folder / name
|
||||
if exists:
|
||||
path.write_bytes(b"png")
|
||||
return self.db.link_image_to_prompt(prompt_id, str(path))
|
||||
|
||||
def run_checks(self, roots=None, **kwargs):
|
||||
roots = [str(self.output)] if roots is None else roots
|
||||
return self.db.run_health_checks(roots, **kwargs)
|
||||
|
||||
@staticmethod
|
||||
def check(report, check_id):
|
||||
return next(c for c in report["checks"] if c["id"] == check_id)
|
||||
|
||||
def raw(self):
|
||||
"""A plain connection for planting damage the app itself never writes."""
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
conn.execute("PRAGMA foreign_keys=OFF")
|
||||
return conn
|
||||
|
||||
|
||||
class TestHealthReport(HealthTestCase):
|
||||
def test_clean_library_passes_except_unavailable_review(self):
|
||||
prompt = self.save("A clean prompt")
|
||||
self.db.save_prompt_pair("Cat", "Blur")
|
||||
self.image(prompt, "a.png")
|
||||
report = self.run_checks(review_count=0)
|
||||
self.assertTrue(report["success"])
|
||||
self.assertEqual([c["id"] for c in report["checks"]], CHECK_IDS)
|
||||
self.assertEqual(
|
||||
[c["label"] for c in report["checks"]],
|
||||
[
|
||||
"Integrity check",
|
||||
"Foreign keys & pair references",
|
||||
"Canonical hashes",
|
||||
"Pair uniqueness",
|
||||
"Missing files",
|
||||
"Ambiguous metadata",
|
||||
"Consistency",
|
||||
],
|
||||
)
|
||||
for check in report["checks"]:
|
||||
self.assertEqual(check["status"], "passed", check)
|
||||
self.assertEqual(check["count"], 0, check)
|
||||
self.assertEqual(report["status"], "passed")
|
||||
self.assertRegex(report["checked_at"], r"^\d{4}-\d\d-\d\dT.*\+00:00$")
|
||||
|
||||
def test_health_check_never_writes(self):
|
||||
prompt = self.save("Untouched")
|
||||
self.image(prompt, "kept.png")
|
||||
self.image(prompt, "gone.png", exists=False)
|
||||
with self.raw() as conn:
|
||||
conn.execute("UPDATE prompts SET hash='stale' WHERE id=?", (prompt,))
|
||||
conn.close()
|
||||
# The app's own connection stays open, as in the running server.
|
||||
self.db.get_statistics()
|
||||
files = [self.db_path, self.db_path + "-wal"]
|
||||
before = [_digest(path) for path in files]
|
||||
shm_size = os.path.getsize(self.db_path + "-shm")
|
||||
media = sorted(os.listdir(self.output))
|
||||
opened = []
|
||||
real_connect = sqlite3.connect
|
||||
|
||||
class Recording(sqlite3.Connection):
|
||||
changes_at_close = None
|
||||
|
||||
def close(self):
|
||||
Recording.changes_at_close = self.total_changes
|
||||
super().close()
|
||||
|
||||
def connect(*args, **kwargs):
|
||||
conn = real_connect(*args, factory=Recording, **kwargs)
|
||||
opened.append((args, kwargs, conn))
|
||||
return conn
|
||||
|
||||
with patch.object(health.sqlite3, "connect", side_effect=connect):
|
||||
report = self.run_checks()
|
||||
self.assertEqual(report["status"], "incomplete")
|
||||
self.assertEqual(len(opened), 1)
|
||||
args, kwargs, conn = opened[0]
|
||||
self.assertTrue(str(args[0]).endswith("?mode=ro"), args)
|
||||
self.assertTrue(kwargs.get("uri"))
|
||||
self.assertEqual(Recording.changes_at_close, 0)
|
||||
with self.assertRaises(sqlite3.ProgrammingError): # closed in finally
|
||||
conn.execute("SELECT 1")
|
||||
self.assertEqual([_digest(path) for path in files], before)
|
||||
# The -shm holds WAL read marks every reader updates; it never grows.
|
||||
self.assertEqual(os.path.getsize(self.db_path + "-shm"), shm_size)
|
||||
self.assertEqual(sorted(os.listdir(self.output)), media)
|
||||
with self.db.model.get_connection() as live:
|
||||
stored = live.execute(
|
||||
"SELECT hash FROM prompts WHERE id=?", (prompt,)
|
||||
).fetchone()[0]
|
||||
self.assertEqual(stored, "stale")
|
||||
|
||||
def test_connection_closed_when_a_check_raises(self):
|
||||
self.save("Any")
|
||||
with (
|
||||
patch.object(
|
||||
health,
|
||||
"_canonical_hashes",
|
||||
side_effect=sqlite3.OperationalError("boom"),
|
||||
),
|
||||
self.assertLogs("prompt_manager.database.health", "ERROR"),
|
||||
):
|
||||
report = self.run_checks(review_count=0)
|
||||
canonical = self.check(report, "canonical_hashes")
|
||||
self.assertEqual(canonical["status"], "failed")
|
||||
self.assertIsNone(canonical["count"])
|
||||
self.assertEqual(canonical["detail"], ["The check could not finish."])
|
||||
self.assertEqual(self.check(report, "integrity")["status"], "passed")
|
||||
self.assertEqual(report["status"], "incomplete")
|
||||
self.db.close_all()
|
||||
assert_closed(self, self.root)
|
||||
|
||||
def test_unopenable_database_fails_every_database_check(self):
|
||||
missing = PromptDatabase.__new__(PromptDatabase)
|
||||
missing.model = type("M", (), {"db_path": str(self.root / "nope.db")})()
|
||||
with self.assertLogs("prompt_manager.database.health", "ERROR"):
|
||||
report = missing.run_health_checks([str(self.output)], review_count=0)
|
||||
statuses = {c["id"]: c["status"] for c in report["checks"]}
|
||||
self.assertEqual(statuses["integrity"], "failed")
|
||||
self.assertEqual(statuses["missing_files"], "failed")
|
||||
self.assertEqual(statuses["ambiguous_metadata"], "passed")
|
||||
self.assertEqual(report["status"], "incomplete")
|
||||
self.assertFalse(os.path.exists(self.root / "nope.db"))
|
||||
|
||||
def test_integrity_and_foreign_key_issues_reported(self):
|
||||
prompt = self.save("Linked")
|
||||
with self.raw() as conn:
|
||||
conn.execute("CREATE TABLE probe(a)")
|
||||
conn.execute("CREATE INDEX probe_a ON probe(a)")
|
||||
conn.executemany("INSERT INTO probe VALUES(?)", [(i,) for i in range(16)])
|
||||
conn.execute(
|
||||
"INSERT INTO generated_images(prompt_id,image_path,filename,file_path)"
|
||||
" VALUES(?,?,?,?)",
|
||||
(9999, str(self.output / "orphan.png"), "orphan.png", "orphan"),
|
||||
)
|
||||
conn.execute(
|
||||
"INSERT INTO prompt_tags(prompt_id,tag_id) VALUES(?,?)", (prompt, 777)
|
||||
)
|
||||
conn.close()
|
||||
with self.raw() as conn:
|
||||
conn.execute("PRAGMA writable_schema=ON")
|
||||
conn.execute(
|
||||
"UPDATE sqlite_master SET sql='CREATE INDEX probe_a ON probe(a DESC)'"
|
||||
" WHERE name='probe_a'"
|
||||
)
|
||||
conn.close()
|
||||
self.db.close_all() # the next read sees the edited schema
|
||||
(self.output / "orphan.png").write_bytes(b"png")
|
||||
with self.raw() as conn:
|
||||
expected = [r[0] for r in conn.execute("PRAGMA integrity_check(100)")]
|
||||
conn.close()
|
||||
self.assertGreater(len(expected), 10)
|
||||
report = self.run_checks(review_count=0)
|
||||
integrity = self.check(report, "integrity")
|
||||
self.assertEqual(integrity["status"], "issues")
|
||||
self.assertEqual(integrity["count"], len(expected))
|
||||
self.assertEqual(integrity["detail"], expected[:10])
|
||||
self.assertIn("missing from index probe_a", integrity["detail"][0])
|
||||
foreign = self.check(report, "foreign_keys")
|
||||
self.assertEqual(foreign["status"], "issues")
|
||||
self.assertEqual(foreign["count"], 2)
|
||||
self.assertEqual(
|
||||
sorted(foreign["detail"]),
|
||||
[
|
||||
"generated_images row 1 references a missing prompts row",
|
||||
"prompt_tags row 1 references a missing tags row",
|
||||
],
|
||||
)
|
||||
consistency = self.check(report, "consistency")
|
||||
self.assertEqual(consistency["status"], "issues")
|
||||
self.assertEqual(consistency["count"], 1)
|
||||
self.assertEqual(
|
||||
consistency["detail"], ["Image 1 references non-existent prompt 9999"]
|
||||
)
|
||||
self.assertEqual(report["status"], "issues")
|
||||
|
||||
def test_canonical_hash_mismatch_counts_rows(self):
|
||||
ids = [self.save(f"Prompt {n}") for n in range(14)]
|
||||
with self.raw() as conn:
|
||||
for n, prompt_id in enumerate(ids[:12]):
|
||||
conn.execute(
|
||||
"UPDATE prompts SET hash=? WHERE id=?", (f"bad{n}", prompt_id)
|
||||
)
|
||||
conn.execute("UPDATE prompts SET hash=NULL WHERE id=?", (ids[12],))
|
||||
conn.close()
|
||||
# Normalised spellings hash alike: this one is not a mismatch.
|
||||
self.save(" MIXED Case ")
|
||||
report = self.run_checks(review_count=0)
|
||||
canonical = self.check(report, "canonical_hashes")
|
||||
self.assertEqual(canonical["status"], "issues")
|
||||
self.assertEqual(canonical["count"], 13)
|
||||
self.assertEqual(len(canonical["detail"]), 10)
|
||||
self.assertEqual(
|
||||
canonical["detail"][0],
|
||||
f"Prompt {ids[0]}: the stored hash does not match its text",
|
||||
)
|
||||
|
||||
def test_duplicate_pair_rows_reported(self):
|
||||
pair = self.db.save_prompt_pair("Cat", "")
|
||||
positive = self.db.get_prompt_pair(pair)["positive_prompt_id"]
|
||||
self.db.save_prompt_pair("Dog", "Blur")
|
||||
with self.raw() as conn:
|
||||
# Only the partial index keeps positive-only pairs unique; a restored
|
||||
# or hand-edited database may have lost it.
|
||||
conn.execute("DROP INDEX idx_prompt_pairs_positive_only")
|
||||
conn.executemany(
|
||||
"INSERT INTO prompt_pairs(positive_prompt_id,negative_prompt_id)"
|
||||
" VALUES(?,NULL)",
|
||||
[(positive,), (positive,)],
|
||||
)
|
||||
conn.close()
|
||||
report = self.run_checks(review_count=0)
|
||||
pairs = self.check(report, "pair_uniqueness")
|
||||
self.assertEqual(pairs["status"], "issues")
|
||||
self.assertEqual(pairs["count"], 2)
|
||||
self.assertEqual(
|
||||
pairs["detail"],
|
||||
[f"Positive prompt {positive} with no negative: 3 pair rows"],
|
||||
)
|
||||
|
||||
def test_integrity_count_at_the_limit_is_flagged_capped(self):
|
||||
class Rows:
|
||||
def __init__(self, rows):
|
||||
self.rows = rows
|
||||
|
||||
def execute(self, _sql):
|
||||
return iter([(row,) for row in self.rows])
|
||||
|
||||
capped = health._integrity(Rows(["bad"] * health.INTEGRITY_LIMIT), [])
|
||||
self.assertEqual(capped["status"], "issues")
|
||||
self.assertEqual(capped["count"], health.INTEGRITY_LIMIT)
|
||||
self.assertTrue(capped["capped"])
|
||||
exact = health._integrity(Rows(["bad"] * 3), [])
|
||||
self.assertEqual(exact["count"], 3)
|
||||
self.assertFalse(exact["capped"])
|
||||
self.assertFalse(health._integrity(Rows(["ok"]), [])["capped"])
|
||||
|
||||
def test_an_unexpected_error_fails_only_its_own_check(self):
|
||||
self.save("Any")
|
||||
with (
|
||||
patch.object(health, "_pair_uniqueness", side_effect=KeyError("boom")),
|
||||
self.assertLogs("prompt_manager.database.health", "ERROR"),
|
||||
):
|
||||
report = self.run_checks(review_count=0)
|
||||
statuses = {c["id"]: c["status"] for c in report["checks"]}
|
||||
self.assertEqual(statuses.pop("pair_uniqueness"), "failed")
|
||||
self.assertEqual(set(statuses.values()), {"passed"})
|
||||
|
||||
|
||||
class TestDatabaseUri(unittest.TestCase):
|
||||
"""The ``mode=ro`` URI of the database file, for every path form SQLite opens."""
|
||||
|
||||
def test_posix_path_is_a_local_file_uri(self):
|
||||
self.assertEqual(
|
||||
health.readonly_uri("/data/my db#1?.db", windows=False),
|
||||
"file:///data/my%20db%231%3F.db?mode=ro",
|
||||
)
|
||||
|
||||
def test_windows_drive_path(self):
|
||||
self.assertEqual(
|
||||
health.readonly_uri(r"C:\Users\Me\prompts.db", windows=True),
|
||||
"file:///C:/Users/Me/prompts.db?mode=ro",
|
||||
)
|
||||
|
||||
def test_windows_unc_path_has_an_empty_authority(self):
|
||||
# pathlib spells \\server\share as file://server/share, which SQLite
|
||||
# rejects (a non-local authority); an empty one keeps the UNC path.
|
||||
uri = health.readonly_uri(r"\\server\share\Comfy UI\prompts.db", windows=True)
|
||||
self.assertEqual(uri, "file:////server/share/Comfy%20UI/prompts.db?mode=ro")
|
||||
self.assertTrue(ntpath.isabs(r"\\server\share\Comfy UI\prompts.db"))
|
||||
|
||||
def test_uri_opens_a_path_with_uri_characters(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
# "?" is not a valid Windows file name character; "#", "%" and spaces are.
|
||||
name = "a #b%20 c.db" if os.name == "nt" else "a #b?c%20.db"
|
||||
path = os.path.join(os.path.realpath(temp), name)
|
||||
conn = sqlite3.connect(path)
|
||||
try:
|
||||
conn.execute("CREATE TABLE t(a)")
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
ro = sqlite3.connect(health.readonly_uri(path), uri=True)
|
||||
try:
|
||||
self.assertEqual(ro.execute("SELECT COUNT(*) FROM t").fetchone(), (0,))
|
||||
with self.assertRaises(sqlite3.OperationalError):
|
||||
ro.execute("INSERT INTO t VALUES(1)")
|
||||
finally:
|
||||
ro.close()
|
||||
|
||||
@unittest.skipUnless(os.name == "nt", "a native UNC path exists on Windows only")
|
||||
def test_native_unc_path_opens(self):
|
||||
with tempfile.TemporaryDirectory() as temp:
|
||||
path = os.path.join(os.path.realpath(temp), "prompts.db")
|
||||
conn = sqlite3.connect(path)
|
||||
try:
|
||||
conn.execute("CREATE TABLE t(a)")
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
drive, rest = ntpath.splitdrive(path)
|
||||
unc = "\\\\localhost\\" + drive[0] + "$" + rest
|
||||
if not os.path.exists(unc):
|
||||
self.skipTest("the administrative share is not reachable here")
|
||||
ro = sqlite3.connect(health.readonly_uri(unc), uri=True)
|
||||
try:
|
||||
self.assertEqual(ro.execute("SELECT COUNT(*) FROM t").fetchone(), (0,))
|
||||
finally:
|
||||
ro.close()
|
||||
|
||||
|
||||
class TestMissingFiles(HealthTestCase):
|
||||
def test_missing_files_counted_under_readable_sources_only(self):
|
||||
prompt = self.save("Images")
|
||||
self.image(prompt, "here.png")
|
||||
for n in range(12):
|
||||
self.image(prompt, f"gone-{n}.png", exists=False)
|
||||
elsewhere = self.root / "elsewhere"
|
||||
elsewhere.mkdir()
|
||||
self.image(prompt, "outside.png", exists=False, folder=elsewhere)
|
||||
report = self.run_checks(review_count=0)
|
||||
missing = self.check(report, "missing_files")
|
||||
self.assertEqual(missing["status"], "issues")
|
||||
self.assertEqual(missing["count"], 12)
|
||||
self.assertEqual(len(missing["detail"]), 10)
|
||||
self.assertRegex(missing["detail"][0], r"^Image \d+: gone-0\.png is missing$")
|
||||
self.assertNotIn(str(self.root), " ".join(missing["detail"]))
|
||||
with self.db.model.get_connection() as conn:
|
||||
self.assertEqual(
|
||||
conn.execute("SELECT COUNT(*) FROM generated_images").fetchone()[0],
|
||||
14,
|
||||
)
|
||||
|
||||
def test_links_outside_every_source_are_named_not_counted(self):
|
||||
prompt = self.save("Images")
|
||||
elsewhere = self.root / "elsewhere"
|
||||
elsewhere.mkdir()
|
||||
self.image(prompt, "outside.png", exists=False, folder=elsewhere)
|
||||
missing = self.check(self.run_checks(review_count=0), "missing_files")
|
||||
self.assertEqual(missing["status"], "passed")
|
||||
self.assertEqual(missing["count"], 0)
|
||||
self.assertEqual(
|
||||
missing["detail"],
|
||||
["1 link outside the image sources was not checked."],
|
||||
)
|
||||
|
||||
def test_missing_file_vs_unavailable_root(self):
|
||||
# Per source (S2 ruling): the readable source is counted, the absent
|
||||
# one reports "unavailable" with no count, and the report is incomplete.
|
||||
prompt = self.save("Images")
|
||||
absent = self.root / "unmounted"
|
||||
absent.mkdir()
|
||||
self.image(prompt, "on-nas.png", folder=absent)
|
||||
self.image(prompt, "gone.png", exists=False)
|
||||
os.unlink(absent / "on-nas.png")
|
||||
absent.rmdir()
|
||||
report = self.run_checks([str(self.output), str(absent)], review_count=0)
|
||||
missing = self.check(report, "missing_files")
|
||||
self.assertEqual(missing["status"], "issues")
|
||||
self.assertEqual(missing["count"], 1)
|
||||
self.assertTrue(missing["partial"])
|
||||
self.assertEqual(
|
||||
missing["sources"],
|
||||
[
|
||||
{"name": "output", "status": "checked", "count": 1},
|
||||
{"name": "unmounted", "status": "unavailable", "count": None},
|
||||
],
|
||||
)
|
||||
self.assertEqual(
|
||||
missing["detail"][0],
|
||||
"1 image source is unavailable, so its files were not counted.",
|
||||
)
|
||||
self.assertRegex(missing["detail"][1], r"^Image \d+: gone\.png is missing$")
|
||||
# The link under the unavailable source is neither counted nor "outside".
|
||||
self.assertEqual(len(missing["detail"]), 2)
|
||||
self.assertNotIn(str(self.root), " ".join(missing["detail"]))
|
||||
self.assertEqual(report["status"], "incomplete")
|
||||
|
||||
def test_partial_sources_with_no_missing_file_are_not_passed(self):
|
||||
prompt = self.save("Images")
|
||||
self.image(prompt, "here.png")
|
||||
absent = self.root / "unmounted"
|
||||
missing = self.check(
|
||||
self.run_checks([str(self.output), str(absent)], review_count=0),
|
||||
"missing_files",
|
||||
)
|
||||
self.assertEqual(missing["status"], "unavailable")
|
||||
self.assertEqual(missing["count"], 0)
|
||||
self.assertTrue(missing["partial"])
|
||||
self.assertEqual(
|
||||
[s["status"] for s in missing["sources"]], ["checked", "unavailable"]
|
||||
)
|
||||
|
||||
def test_every_source_unavailable_counts_nothing(self):
|
||||
prompt = self.save("Images")
|
||||
absent = self.root / "unmounted"
|
||||
self.image(prompt, "gone.png", exists=False, folder=self.output)
|
||||
report = self.run_checks([str(absent)], review_count=0)
|
||||
missing = self.check(report, "missing_files")
|
||||
self.assertEqual(missing["status"], "unavailable")
|
||||
self.assertIsNone(missing["count"])
|
||||
self.assertFalse(missing["partial"])
|
||||
self.assertEqual(
|
||||
missing["sources"],
|
||||
[{"name": "unmounted", "status": "unavailable", "count": None}],
|
||||
)
|
||||
self.assertEqual(
|
||||
missing["detail"],
|
||||
["1 image source is unavailable, so missing files were not counted."],
|
||||
)
|
||||
|
||||
def test_each_source_counts_its_own_missing_files(self):
|
||||
prompt = self.save("Images")
|
||||
archive = self.root / "archive"
|
||||
archive.mkdir()
|
||||
self.image(prompt, "a.png", exists=False)
|
||||
self.image(prompt, "b.png", exists=False, folder=archive)
|
||||
self.image(prompt, "c.png", exists=False, folder=archive)
|
||||
missing = self.check(
|
||||
self.run_checks([str(self.output), str(archive)], review_count=0),
|
||||
"missing_files",
|
||||
)
|
||||
self.assertEqual(missing["count"], 3)
|
||||
self.assertFalse(missing["partial"])
|
||||
self.assertEqual(
|
||||
[(s["name"], s["count"]) for s in missing["sources"]],
|
||||
[("output", 1), ("archive", 2)],
|
||||
)
|
||||
|
||||
def test_no_sources_is_unavailable(self):
|
||||
self.save("Images")
|
||||
missing = self.check(self.run_checks([], review_count=0), "missing_files")
|
||||
self.assertEqual(missing["status"], "unavailable")
|
||||
self.assertIsNone(missing["count"])
|
||||
self.assertEqual(missing["detail"], ["No image source is available."])
|
||||
self.assertEqual(missing["sources"], [])
|
||||
self.assertFalse(missing["partial"])
|
||||
|
||||
def test_a_bad_stored_path_fails_only_missing_files(self):
|
||||
prompt = self.save("Images")
|
||||
self.image(prompt, "here.png")
|
||||
with self.raw() as conn:
|
||||
conn.execute(
|
||||
"INSERT INTO generated_images(prompt_id,image_path,filename)"
|
||||
" VALUES(?,?,?)",
|
||||
(prompt, str(self.root / "else\x00where.png"), "bad.png"),
|
||||
)
|
||||
conn.close()
|
||||
real_realpath = os.path.realpath
|
||||
|
||||
def realpath(path, *args, **kwargs):
|
||||
# POSIX (and Windows before Python 3.11) raise for the NUL byte;
|
||||
# newer ntpath returns it unchanged. Raise everywhere, as POSIX does.
|
||||
if "\x00" in str(path):
|
||||
raise ValueError("embedded null byte")
|
||||
return real_realpath(path, *args, **kwargs)
|
||||
|
||||
with (
|
||||
patch.object(health.os.path, "realpath", side_effect=realpath),
|
||||
self.assertLogs("prompt_manager.database.health", "ERROR"),
|
||||
):
|
||||
report = self.run_checks(review_count=0)
|
||||
statuses = {c["id"]: c["status"] for c in report["checks"]}
|
||||
self.assertEqual(statuses.pop("missing_files"), "failed")
|
||||
self.assertEqual(set(statuses.values()), {"passed"})
|
||||
|
||||
def test_files_are_checked_after_the_read_snapshot_ends(self):
|
||||
prompt = self.save("Images")
|
||||
self.image(prompt, "here.png")
|
||||
self.image(prompt, "gone.png", exists=False)
|
||||
opened = []
|
||||
real_connect = sqlite3.connect
|
||||
|
||||
def connect(*args, **kwargs):
|
||||
conn = real_connect(*args, **kwargs)
|
||||
opened.append(conn)
|
||||
return conn
|
||||
|
||||
real_exists = os.path.exists
|
||||
in_snapshot = []
|
||||
|
||||
def exists(path):
|
||||
for conn in opened:
|
||||
try:
|
||||
in_snapshot.append(conn.in_transaction)
|
||||
except sqlite3.ProgrammingError: # closed: no snapshot
|
||||
in_snapshot.append(False)
|
||||
return real_exists(path)
|
||||
|
||||
with (
|
||||
patch.object(health.sqlite3, "connect", side_effect=connect),
|
||||
patch.object(health.os.path, "exists", side_effect=exists),
|
||||
):
|
||||
missing = self.check(self.run_checks(review_count=0), "missing_files")
|
||||
self.assertEqual((missing["status"], missing["count"]), ("issues", 1))
|
||||
self.assertEqual(len(opened), 1)
|
||||
self.assertTrue(in_snapshot)
|
||||
self.assertNotIn(True, in_snapshot)
|
||||
|
||||
|
||||
class TestAmbiguousMetadata(HealthTestCase):
|
||||
def test_ambiguous_metadata_unavailable_without_review_queue(self):
|
||||
self.save("Any")
|
||||
report = self.run_checks()
|
||||
ambiguous = self.check(report, "ambiguous_metadata")
|
||||
self.assertEqual(ambiguous["status"], "unavailable")
|
||||
self.assertIsNone(ambiguous["count"])
|
||||
self.assertEqual(ambiguous["detail"], ["Needs review is not available yet."])
|
||||
self.assertEqual(report["status"], "incomplete")
|
||||
|
||||
def test_review_count_reports_images_needing_review(self):
|
||||
report = self.run_checks(review_count=3)
|
||||
ambiguous = self.check(report, "ambiguous_metadata")
|
||||
self.assertEqual(ambiguous["status"], "issues")
|
||||
self.assertEqual(ambiguous["count"], 3)
|
||||
self.assertEqual(ambiguous["detail"], [])
|
||||
self.assertEqual(report["status"], "issues")
|
||||
|
||||
def test_invalid_review_count_is_rejected(self):
|
||||
for value in (-1, True, "3", 1.5):
|
||||
with self.assertRaises(ValueError):
|
||||
self.run_checks(review_count=value)
|
||||
|
||||
|
||||
class TestOverallStatus(unittest.TestCase):
|
||||
def test_overall_status_precedence(self):
|
||||
def checks(*statuses):
|
||||
return [{"status": status} for status in statuses]
|
||||
|
||||
self.assertEqual(health.overall_status(checks("passed", "passed")), "passed")
|
||||
self.assertEqual(health.overall_status(checks("passed", "issues")), "issues")
|
||||
self.assertEqual(
|
||||
health.overall_status(checks("issues", "unavailable")), "incomplete"
|
||||
)
|
||||
self.assertEqual(health.overall_status(checks("failed")), "incomplete")
|
||||
# A check that found issues under only some sources is incomplete too.
|
||||
partial = [{"status": "issues", "partial": True}, {"status": "passed"}]
|
||||
self.assertEqual(health.overall_status(partial), "incomplete")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -299,45 +299,126 @@ class OutputWriterTests(fixtures.unittest.TestCase):
|
||||
)
|
||||
self.assertEqual(writer.context_budget.charged_bytes, 0)
|
||||
|
||||
def isolated_writer(self) -> HistoryWriter:
|
||||
"""A ready writer whose budget no other writer or test can charge."""
|
||||
writer = HistoryWriter(
|
||||
lambda: self.db, clock=self.clock, context_budget=ContextBudget()
|
||||
)
|
||||
self.writers.append(writer)
|
||||
writer.start()
|
||||
self.wait_for(lambda: writer.status().accepting)
|
||||
return writer
|
||||
|
||||
def rotate_between_transactions(self, writer, start, after, restore, hold=None):
|
||||
"""Wrap ``after`` so a restore or invalidation lands right after it.
|
||||
|
||||
Returns the patch and an event set once that rotation has returned on
|
||||
the worker. A restore is a full verify/migrate/backup/replace cycle, so
|
||||
the handoff waits on this event with the generous ``BLOCK_SECONDS``
|
||||
ceiling instead of the short worker-behavior deadline (TESTING.md,
|
||||
Windows portability preflight).
|
||||
"""
|
||||
backup = os.path.join(self.directory.name, f"restore-{start.run_id}.db")
|
||||
self.assertTrue(self.db.model.backup_database(backup))
|
||||
original = getattr(self.db, after)
|
||||
rotated = threading.Event()
|
||||
|
||||
def write(*args, **kwargs):
|
||||
result = original(*args, **kwargs)
|
||||
try:
|
||||
if hold is not None:
|
||||
entered, release = hold
|
||||
entered.set()
|
||||
release.wait(BLOCK_SECONDS)
|
||||
if restore:
|
||||
self.db.model.restore_from_file(backup)
|
||||
else:
|
||||
writer.invalidate()
|
||||
finally:
|
||||
rotated.set()
|
||||
return result
|
||||
|
||||
return patch.object(self.db, after, write), rotated
|
||||
|
||||
def test_restore_or_generation_rotation_between_transactions_discards_evidence(
|
||||
self,
|
||||
):
|
||||
for after in ("write_execution_finish", "write_execution_contexts_for_finish"):
|
||||
for restore in (False, True):
|
||||
with self.subTest(after=after, restore=restore):
|
||||
writer = self.writer()
|
||||
start = self.start_command(len(self.writers))
|
||||
self.db.write_execution_start(start)
|
||||
backup = os.path.join(self.directory.name, "restore.db")
|
||||
self.assertTrue(self.db.model.backup_database(backup))
|
||||
original = getattr(self.db, after)
|
||||
# Each subtest owns its writer and budget and retires them
|
||||
# before the next starts, so one slow restore cannot leave
|
||||
# in-flight charges or an epoch rotation for the next.
|
||||
writer = self.isolated_writer()
|
||||
try:
|
||||
start = self.start_command(len(self.writers))
|
||||
self.db.write_execution_start(start)
|
||||
rotation, rotated = self.rotate_between_transactions(
|
||||
writer, start, after, restore
|
||||
)
|
||||
with rotation:
|
||||
self.finalization(writer, start)
|
||||
self.assertTrue(rotated.wait(BLOCK_SECONDS))
|
||||
self.wait_for(lambda: writer.status().pending == 0)
|
||||
self.assertEqual(
|
||||
self.db.get_execution_outputs(start.run_id)["state"],
|
||||
"not_captured",
|
||||
)
|
||||
self.assertEqual(
|
||||
self.db.get_execution_contexts(start.run_id)["state"],
|
||||
(
|
||||
"complete"
|
||||
if after == "write_execution_contexts_for_finish"
|
||||
and not restore
|
||||
else "not_captured"
|
||||
),
|
||||
)
|
||||
self.assertEqual(writer.context_budget.charged_bytes, 0)
|
||||
finally:
|
||||
self.assertTrue(writer.close(BLOCK_SECONDS))
|
||||
|
||||
def write(*args, **kwargs):
|
||||
result = original(*args, **kwargs)
|
||||
if restore:
|
||||
self.db.model.restore_from_file(backup)
|
||||
else:
|
||||
writer.invalidate()
|
||||
return result
|
||||
|
||||
with patch.object(self.db, after, write):
|
||||
self.finalization(writer, start)
|
||||
self.wait_for(lambda: writer.status().pending == 0)
|
||||
self.assertEqual(
|
||||
self.db.get_execution_outputs(start.run_id)["state"],
|
||||
"not_captured",
|
||||
)
|
||||
self.assertEqual(
|
||||
self.db.get_execution_contexts(start.run_id)["state"],
|
||||
(
|
||||
"complete"
|
||||
if after == "write_execution_contexts_for_finish"
|
||||
and not restore
|
||||
else "not_captured"
|
||||
),
|
||||
)
|
||||
self.assertEqual(writer.context_budget.charged_bytes, 0)
|
||||
self.assertTrue(writer.close())
|
||||
def test_slow_restore_between_transactions_holds_charge_only_while_in_flight(
|
||||
self,
|
||||
):
|
||||
writer = self.isolated_writer()
|
||||
start = self.start_command()
|
||||
self.db.write_execution_start(start)
|
||||
entered, release = self.block()
|
||||
rotation, rotated = self.rotate_between_transactions(
|
||||
writer, start, "write_execution_finish", True, (entered, release)
|
||||
)
|
||||
with rotation:
|
||||
context = self.sealed(writer)
|
||||
outputs = self.outputs(writer, context.lease.byte_count)
|
||||
charge = writer.context_budget.charged_bytes
|
||||
self.assertGreater(charge, 0)
|
||||
self.assertTrue(
|
||||
writer.submit_finalization(self.finish_command(start), context, outputs)
|
||||
)
|
||||
self.assertTrue(entered.wait(BLOCK_SECONDS))
|
||||
# The worker is between transactions: the payload is owned, not leaked.
|
||||
self.assertEqual(writer.status().pending, 1)
|
||||
self.assertEqual(writer.context_budget.charged_bytes, charge)
|
||||
release.set()
|
||||
self.assertTrue(rotated.wait(BLOCK_SECONDS))
|
||||
self.wait_for(lambda: writer.status().pending == 0)
|
||||
self.assertEqual(writer.context_budget.charged_bytes, 0)
|
||||
self.assertFalse(writer.status().accepting)
|
||||
self.assertEqual(
|
||||
self.db.get_execution_outputs(start.run_id)["state"], "not_captured"
|
||||
)
|
||||
# The restored epoch is probed again and new evidence is accepted.
|
||||
self.clock.advance(5)
|
||||
self.wait_for(lambda: writer.status().accepting)
|
||||
restored = self.start_command(2)
|
||||
self.db.write_execution_start(restored)
|
||||
self.finalization(writer, restored)
|
||||
self.wait_for(lambda: writer.status().pending == 0)
|
||||
self.assertEqual(
|
||||
self.db.get_execution_outputs(restored.run_id)["state"], "complete"
|
||||
)
|
||||
self.assertEqual(writer.context_budget.charged_bytes, 0)
|
||||
self.assertTrue(writer.close(BLOCK_SECONDS))
|
||||
|
||||
def test_invalidation_during_output_preparation_rechecks_actual_commit(self):
|
||||
import database.execution_outputs as outputs_db
|
||||
|
||||
@@ -23,7 +23,11 @@ sys.modules.setdefault("server", _mock_server)
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from py.config import GalleryConfig, PromptManagerConfig # noqa: E402
|
||||
from py.config import ( # noqa: E402
|
||||
OUTSIDE_ALLOWED_REASON,
|
||||
GalleryConfig,
|
||||
PromptManagerConfig,
|
||||
)
|
||||
|
||||
EXTRA_ROOTS_ENV = "PROMPT_MANAGER_EXTRA_GALLERY_ROOTS"
|
||||
|
||||
@@ -158,9 +162,10 @@ class GalleryRootTestCase(unittest.TestCase):
|
||||
class TestValidateGalleryRootRejects(GalleryRootTestCase):
|
||||
|
||||
def test_filesystem_root_rejected(self):
|
||||
"""Refused as outside the allowed folders: no reason names the layout."""
|
||||
ok, reason = GalleryConfig.validate_gallery_root(os.path.abspath(os.sep))
|
||||
self.assertFalse(ok)
|
||||
self.assertIn("root", reason.lower())
|
||||
self.assertEqual(reason, OUTSIDE_ALLOWED_REASON)
|
||||
|
||||
@unittest.skipUnless(os.name == "nt", "drive roots only exist on Windows")
|
||||
def test_windows_drive_root_rejected(self):
|
||||
@@ -170,7 +175,17 @@ class TestValidateGalleryRootRejects(GalleryRootTestCase):
|
||||
def test_home_directory_rejected(self):
|
||||
ok, reason = GalleryConfig.validate_gallery_root(os.path.expanduser("~"))
|
||||
self.assertFalse(ok)
|
||||
self.assertIn("home", reason.lower())
|
||||
self.assertEqual(reason, OUTSIDE_ALLOWED_REASON)
|
||||
|
||||
def test_home_directory_inside_an_allowed_parent_is_still_refused(self):
|
||||
"""Even when the environment opens the home's parent, home answers
|
||||
like any outside path."""
|
||||
home = os.path.expanduser("~")
|
||||
os.environ[EXTRA_ROOTS_ENV] = os.path.dirname(home)
|
||||
self.assertEqual(
|
||||
GalleryConfig.gallery_root_problem(home),
|
||||
("outside-allowed", OUTSIDE_ALLOWED_REASON),
|
||||
)
|
||||
|
||||
def test_directory_outside_comfyui_rejected(self):
|
||||
outside = Path(self.tmpdir) / "elsewhere"
|
||||
|
||||
@@ -5,7 +5,10 @@ returns for the same input (tests/test_generation_params_js_parity.py checks
|
||||
that directly when Node is available).
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess # nosec B404 - runs a fixed Node one-liner
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
@@ -17,6 +20,7 @@ from utils.generation_params import ( # noqa: E402
|
||||
extract_generation_params,
|
||||
extract_from_records,
|
||||
parse_js_json,
|
||||
parse_page_json,
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -219,6 +223,47 @@ class ParsesLikeJavaScript(unittest.TestCase):
|
||||
run("seed-beyond-double-precision")["seed"], 4974052037430000001.0
|
||||
)
|
||||
|
||||
def test_page_parse_reads_nan_numbers_as_null(self):
|
||||
# gallery.js and admin.js replace NaN with null before JSON.parse.
|
||||
self.assertEqual(
|
||||
parse_page_json('{"a": NaN, "b": [NaN], "t": "NaN"}'),
|
||||
{"a": None, "b": [None], "t": "NaN"},
|
||||
)
|
||||
self.assertEqual(
|
||||
parse_page_json('{"seed": 9007199254740993}'), {"seed": 9007199254740992.0}
|
||||
)
|
||||
for text in ('{"a": Infinity}', "[-Infinity]", "not json", "", None):
|
||||
with self.subTest(text=text):
|
||||
self.assertIsNone(parse_page_json(text))
|
||||
|
||||
@unittest.skipUnless(shutil.which("node"), "Node is not installed")
|
||||
def test_page_parse_equals_the_legacy_pages_for_nan_numbers(self):
|
||||
# gallery.js/admin.js: two NaN regexes, then JSON.parse (Infinity throws).
|
||||
texts = [
|
||||
'{"1": {"is_changed": [NaN]}, "2": {"is_changed": NaN, "v": 1.5}}',
|
||||
'{"a":NaN,"b":[NaN, 2]}',
|
||||
'{"a": Infinity}',
|
||||
'{"seed": 9007199254740993}',
|
||||
]
|
||||
script = (
|
||||
"const texts = JSON.parse(require('fs').readFileSync(0, 'utf8'));"
|
||||
"console.log(JSON.stringify(texts.map((t) => { try { return {v: JSON.parse("
|
||||
"t.replace(/:\\s*NaN\\b/g, ': null').replace(/\\bNaN\\b/g, 'null'))}; }"
|
||||
" catch (e) { return null; } })));"
|
||||
)
|
||||
completed = subprocess.run( # nosec B603 B607 - fixed argv, no shell
|
||||
["node", "-e", script],
|
||||
input=json.dumps(texts),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
timeout=60,
|
||||
)
|
||||
expected = json.loads(completed.stdout)
|
||||
for text, page in zip(texts, expected):
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(parse_page_json(text), page and page["v"])
|
||||
|
||||
def test_duplicate_keys_keep_the_last_value(self):
|
||||
self.assertEqual(parse_js_json('{"a": 1, "a": 2}'), {"a": 2})
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import types
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
@@ -368,10 +369,29 @@ class TestConsoleBanner(unittest.TestCase):
|
||||
self.assertIn("Loaded:", banner)
|
||||
|
||||
|
||||
class PriorFileHandler(logging.FileHandler):
|
||||
"""A caller-owned sink that stays closed once its owner closes it.
|
||||
|
||||
``FileHandler.emit`` reopens the file when ``stream`` is None, so a record
|
||||
that selected this handler before ``close()`` would reopen prior.log after
|
||||
it, and Windows then refuses the temp directory cleanup (WinError 32).
|
||||
``Handler.handle`` and ``FileHandler.close`` hold the same handler lock, so
|
||||
checking ``_closed`` in ``emit`` is race free (as in the product's
|
||||
``_OwnedRotatingFileHandler``).
|
||||
"""
|
||||
|
||||
def emit(self, record):
|
||||
if not self._closed:
|
||||
super().emit(record)
|
||||
|
||||
|
||||
class TestPackageLoggerIsolation(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
# Runs after restore_logging, before the cleanup Windows refuses while
|
||||
# prior.log is open; /proc makes Linux fail the same way.
|
||||
self.addCleanup(self.assert_tmp_has_no_open_handles)
|
||||
self.logger = logging.getLogger("prompt_manager")
|
||||
self.previous = (
|
||||
self.logger.handlers[:],
|
||||
@@ -381,7 +401,7 @@ class TestPackageLoggerIsolation(unittest.TestCase):
|
||||
)
|
||||
self.addCleanup(self.restore_logging)
|
||||
self.path = os.path.join(self.tmp.name, "prior.log")
|
||||
self.handler = logging.FileHandler(self.path)
|
||||
self.handler = PriorFileHandler(self.path)
|
||||
self.logger.handlers = [self.handler]
|
||||
self.logger.setLevel(logging.DEBUG)
|
||||
self.logger.propagate = True
|
||||
@@ -398,16 +418,20 @@ class TestPackageLoggerIsolation(unittest.TestCase):
|
||||
self.server.PromptServer.instance.prompt_queue.task_done = lambda *a, **k: None
|
||||
|
||||
def restore_logging(self):
|
||||
for handler in self.logger.handlers:
|
||||
if handler not in self.previous[0]:
|
||||
handler.close()
|
||||
self.handler.close()
|
||||
# Detach before closing, so new records stop selecting these handlers.
|
||||
retired = [h for h in self.logger.handlers if h not in self.previous[0]]
|
||||
(
|
||||
self.logger.handlers,
|
||||
self.logger.level,
|
||||
self.logger.propagate,
|
||||
self.logger.disabled,
|
||||
) = self.previous
|
||||
for handler in retired:
|
||||
handler.close()
|
||||
self.handler.close()
|
||||
|
||||
def assert_tmp_has_no_open_handles(self):
|
||||
self.assertEqual(open_handles_under(self.tmp.name), [])
|
||||
|
||||
def assert_logging_restored(self, handlers, registration):
|
||||
for key, module in self.saved_modules.items():
|
||||
@@ -456,6 +480,49 @@ class TestPackageLoggerIsolation(unittest.TestCase):
|
||||
self.assertEqual(open_at_disposal, [True])
|
||||
self.assert_logging_restored(handlers, registration)
|
||||
|
||||
def test_record_in_flight_at_restore_cannot_reopen_prior_log(self):
|
||||
"""A record that selected prior.log before restore never reopens it.
|
||||
|
||||
Leaked daemon threads of other tests (image workers, watchdog) log
|
||||
under ``prompt_manager.*``. One that reaches the prior handler while
|
||||
restore_logging closes it would, with a plain FileHandler, reopen the
|
||||
file on emit and keep it open through TemporaryDirectory.cleanup().
|
||||
"""
|
||||
message = "record selected before restore_logging"
|
||||
selected = threading.Event()
|
||||
resume = threading.Event()
|
||||
errors = []
|
||||
|
||||
def pause_before_handler_lock(record):
|
||||
if record.getMessage() == message:
|
||||
selected.set()
|
||||
if not resume.wait(5):
|
||||
raise RuntimeError("logging thread was not released")
|
||||
return True
|
||||
|
||||
def log_from_background_thread():
|
||||
try:
|
||||
logging.getLogger("prompt_manager.late_emitter").warning(message)
|
||||
except BaseException as error:
|
||||
errors.append(error)
|
||||
|
||||
self.handler.addFilter(pause_before_handler_lock)
|
||||
worker = threading.Thread(target=log_from_background_thread, daemon=True)
|
||||
worker.start()
|
||||
try:
|
||||
self.assertTrue(selected.wait(5), "record did not select prior.log")
|
||||
self.restore_logging()
|
||||
self.assertIsNone(self.handler.stream)
|
||||
finally:
|
||||
resume.set()
|
||||
worker.join(5)
|
||||
self.assertFalse(worker.is_alive(), "logging thread did not finish")
|
||||
self.assertEqual(errors, [])
|
||||
self.assertIsNone(self.handler.stream)
|
||||
self.assert_tmp_has_no_open_handles()
|
||||
with open(self.path, encoding="utf-8") as log:
|
||||
self.assertNotIn(message, log.read())
|
||||
|
||||
def test_failed_package_import_closes_handlers_and_restores_logging(self):
|
||||
name = "prompt_manager_pkg_logger_failure"
|
||||
captured = {}
|
||||
|
||||
@@ -800,10 +800,15 @@ class TestUpstreamVerifiedNodeSemantics(unittest.TestCase):
|
||||
prompt["31"] = dict(prompt["3"], inputs=dict(prompt["3"]["inputs"]))
|
||||
prompt["31"]["inputs"]["latent_image"] = ["3", 0]
|
||||
prompt["8"]["inputs"]["samples"] = ["31", 0]
|
||||
# Unanchored, two listed samplers are two branches, as for KSampler.
|
||||
self.assertEqual(_inspect({"prompt": prompt})["state"], "ambiguous")
|
||||
# 31 continues 3's latent: the stages are compared, as for KSampler.
|
||||
for anchor in (None, "9"):
|
||||
result = _inspect({"prompt": prompt}, output_node_id=anchor)
|
||||
self.assertEqual(
|
||||
(result["state"], result["sampler_id"]), ("resolved", "31")
|
||||
)
|
||||
prompt["31"]["inputs"]["positive"] = ["7", 0]
|
||||
result = _inspect({"prompt": prompt}, output_node_id="9")
|
||||
self.assertEqual(result["reason"], resolver.MULTISTAGE_UNATTRIBUTED_REASON)
|
||||
self.assertEqual(result["reason_code"], "multistage-differs")
|
||||
|
||||
def test_seed_variance_enhancer_passes_its_conditioning(self):
|
||||
prompt = _f02(
|
||||
|
||||
@@ -55,6 +55,9 @@ def _minimal():
|
||||
"runtime": [],
|
||||
"savers": {},
|
||||
"non_file_outputs": [],
|
||||
"lineage": [
|
||||
{"class": "VAEEncode", "slot": 0, "strategy": "continue", "input": "pixels"}
|
||||
],
|
||||
"stage_inputs": {"shaped": ["positive"], "with_model": ["samples"]},
|
||||
"facts": {
|
||||
"configuration_outputs": {},
|
||||
@@ -199,16 +202,26 @@ class SpecEntriesTest(unittest.TestCase):
|
||||
},
|
||||
)
|
||||
|
||||
# Combine and concat keep every part in effect (additive); an average
|
||||
# blends its parts, so a part may be weighted away.
|
||||
self.assertEqual(
|
||||
table.composites["ConditioningCombine"],
|
||||
{"strategy": "combine", "inputs": ("conditioning_1", "conditioning_2")},
|
||||
{
|
||||
"strategy": "combine",
|
||||
"inputs": ("conditioning_1", "conditioning_2"),
|
||||
"additive": True,
|
||||
},
|
||||
)
|
||||
for name in ("ConditioningConcat", "ConditioningAverage"):
|
||||
for name, additive in (
|
||||
("ConditioningConcat", {"additive": True}),
|
||||
("ConditioningAverage", {}),
|
||||
):
|
||||
self.assertEqual(
|
||||
table.composites[name],
|
||||
{
|
||||
"strategy": "combine",
|
||||
"inputs": ("conditioning_to", "conditioning_from"),
|
||||
**additive,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -257,7 +270,12 @@ class SpecEntriesTest(unittest.TestCase):
|
||||
)
|
||||
|
||||
# D18: decided at run time.
|
||||
self.assertEqual(table.runtime, frozenset({"PromptSearchList"}))
|
||||
# LoraManager's TriggerWord Toggle outputs the LoRA's trigger words as
|
||||
# filtered at run time (option A leaves them out of a PromptManager).
|
||||
self.assertEqual(
|
||||
table.runtime,
|
||||
frozenset({"PromptSearchList", "TriggerWord Toggle (LoraManager)"}),
|
||||
)
|
||||
|
||||
comfy = "{name}_{counter:05}_.{ext}"
|
||||
for name, image in (
|
||||
@@ -464,6 +482,89 @@ class SpecEntriesTest(unittest.TestCase):
|
||||
self.assertIsInstance(table.composites["ConditioningCombine"]["inputs"], tuple)
|
||||
|
||||
|
||||
class LineageTableTest(_TableCase):
|
||||
"""``lineage``: outputs whose image or latent continues one input's."""
|
||||
|
||||
def test_repository_lineage_lists_the_pass_through_nodes(self):
|
||||
table = load_semantics()
|
||||
self.assertEqual(
|
||||
{key: dict(entry) for key, entry in table.lineage.items()},
|
||||
{
|
||||
("VAEDecode", 0): {"strategy": "continue", "input": "samples"},
|
||||
("VAEDecodeTiled", 0): {"strategy": "continue", "input": "samples"},
|
||||
("VAEEncode", 0): {"strategy": "continue", "input": "pixels"},
|
||||
("VAEEncodeTiled", 0): {"strategy": "continue", "input": "pixels"},
|
||||
("ImageUpscaleWithModel", 0): {
|
||||
"strategy": "continue",
|
||||
"input": "image",
|
||||
},
|
||||
("ImageScale", 0): {"strategy": "continue", "input": "image"},
|
||||
("ImageScaleBy", 0): {"strategy": "continue", "input": "image"},
|
||||
("ImageScaleToTotalPixels", 0): {
|
||||
"strategy": "continue",
|
||||
"input": "image",
|
||||
},
|
||||
("LatentUpscale", 0): {"strategy": "continue", "input": "samples"},
|
||||
("LatentUpscaleBy", 0): {"strategy": "continue", "input": "samples"},
|
||||
("Image Filter", 0): {"strategy": "continue", "input": "images"},
|
||||
("Image Filter", 1): {"strategy": "continue", "input": "latents"},
|
||||
},
|
||||
)
|
||||
# Lineage describes no prompt: its classes are not prompt categories.
|
||||
self.assertFalse({name for name, _ in table.lineage} & table.all_classes)
|
||||
|
||||
def test_lineage_rows_are_validated(self):
|
||||
def row(**changes):
|
||||
entry = {
|
||||
"class": "VAEEncode",
|
||||
"slot": 0,
|
||||
"strategy": "continue",
|
||||
"input": "pixels",
|
||||
}
|
||||
entry.update(changes)
|
||||
return entry
|
||||
|
||||
cases = (
|
||||
({"lineage": {}}, "must be a list"),
|
||||
({"lineage": ["VAEEncode"]}, "must be an object"),
|
||||
({"lineage": [row(strategy="passthrough")]}, "unknown strategy"),
|
||||
({"lineage": [row(slot=-1)]}, "slot"),
|
||||
({"lineage": [row(slot=True)]}, "slot"),
|
||||
({"lineage": [row(input="")]}, "input name"),
|
||||
({"lineage": [row(note="x")]}, "field"),
|
||||
({"lineage": [row(), row()]}, "duplicate"),
|
||||
({"lineage": [row(**{"class": ""})]}, "class"),
|
||||
)
|
||||
for index, (change, fragment) in enumerate(cases):
|
||||
with self.subTest(case=index):
|
||||
doc = _minimal()
|
||||
doc.update(change)
|
||||
self.assertRejected(doc, fragment)
|
||||
doc = _minimal()
|
||||
doc["lineage"].append(row(slot=1, input="latents"))
|
||||
self.assertEqual(len(self.load_doc(doc).lineage), 2)
|
||||
|
||||
def test_a_lineage_class_may_also_carry_conditioning(self):
|
||||
# InpaintModelConditioning-like: conditioning slots, a latent slot.
|
||||
doc = _minimal()
|
||||
doc["lineage"].append(
|
||||
{"class": "FluxGuidance", "slot": 1, "strategy": "continue", "input": "x"}
|
||||
)
|
||||
self.assertIn(("FluxGuidance", 1), self.load_doc(doc).lineage)
|
||||
|
||||
def test_composite_additive_must_be_a_boolean(self):
|
||||
doc = _minimal()
|
||||
doc["composites"]["ConditioningCombine"] = {
|
||||
"strategy": "combine",
|
||||
"inputs": ["conditioning_1", "conditioning_2"],
|
||||
"additive": "yes",
|
||||
}
|
||||
self.assertRejected(doc, "additive must be a boolean")
|
||||
doc["composites"]["ConditioningCombine"]["additive"] = True
|
||||
table = self.load_doc(doc)
|
||||
self.assertIs(table.composites["ConditioningCombine"]["additive"], True)
|
||||
|
||||
|
||||
class ValidationTest(_TableCase):
|
||||
def test_minimal_table_loads(self):
|
||||
table = self.load_doc(_minimal())
|
||||
@@ -501,6 +602,7 @@ class ValidationTest(_TableCase):
|
||||
{"literal", "strip-newlines", "join-multi", "concat"}
|
||||
),
|
||||
"savers": frozenset({"comfy-counter", "vhs-counter"}),
|
||||
"lineage": frozenset({"continue"}),
|
||||
},
|
||||
)
|
||||
with self.assertRaises(TypeError):
|
||||
|
||||
@@ -1706,6 +1706,104 @@ class TestLibrary(unittest.TestCase):
|
||||
self.db.list_library_images(positive, pair_id=foreign, archive="all")
|
||||
|
||||
|
||||
class LibraryCounts(unittest.TestCase):
|
||||
"""System › Overview counts and usage statistics follow Library eligibility."""
|
||||
|
||||
setUp = TestLibrary.setUp
|
||||
tearDown = TestLibrary.tearDown
|
||||
save = TestLibrary.save
|
||||
state = TestLibrary.state
|
||||
|
||||
def test_counts_use_library_eligibility(self):
|
||||
self.db.save_prompt_pair("Cat", "Blur")
|
||||
archived = self.db.save_prompt_pair("Cat", "Noise")
|
||||
self.db.archive_prompt_pair(archived)
|
||||
titled_pair = self.db.save_prompt_pair("Dog", "Titled negative")
|
||||
titled = self.db.get_prompt_pair(titled_pair)["negative_prompt_id"]
|
||||
self.state(titled, "A titled parent") # zero pairs of its own: counted
|
||||
legacy = self.save("Legacy")
|
||||
self.db.link_image_to_prompt(legacy, str(Path(self.temp.name) / "a.png"))
|
||||
self.db.link_image_to_prompt(legacy, str(Path(self.temp.name) / "b.png"))
|
||||
counts = self.db.library_counts()
|
||||
# Cat, Dog, the titled negative and Legacy; "Blur" and "Noise" are
|
||||
# negative-only.
|
||||
self.assertEqual(counts["positive_prompts"], 4)
|
||||
# Cat+Blur and Dog+Titled negative; the archived pair is excluded.
|
||||
self.assertEqual(counts["negative_variants"], 2)
|
||||
self.assertEqual(counts["images"], 2)
|
||||
self.assertEqual(counts["rated_images"], 0)
|
||||
listed = self.db.list_library()
|
||||
self.assertEqual(counts["positive_prompts"], listed["total"])
|
||||
self.assertEqual(counts["negative_variants"], listed["variant_total"])
|
||||
|
||||
def test_counts_of_an_empty_library_are_zero(self):
|
||||
self.assertEqual(
|
||||
self.db.library_counts(),
|
||||
{
|
||||
"positive_prompts": 0,
|
||||
"negative_variants": 0,
|
||||
"images": 0,
|
||||
"rated_images": 0,
|
||||
"review_pending": 0,
|
||||
},
|
||||
)
|
||||
|
||||
def test_rated_images_follow_their_linked_prompt_rating(self):
|
||||
# An image carries no rating of its own: it shows its linked prompt's
|
||||
# (the Images "Highest rated" sort), so "rated" counts those images.
|
||||
folder = Path(self.temp.name)
|
||||
rated = self.save("Rated", rating=4)
|
||||
unrated = self.save("Unrated")
|
||||
for name in ("a.png", "b.png", "c.png"):
|
||||
self.db.link_image_to_prompt(rated, str(folder / name))
|
||||
self.db.link_image_to_prompt(unrated, str(folder / "d.png"))
|
||||
counts = self.db.library_counts()
|
||||
self.assertEqual(counts["images"], 4)
|
||||
self.assertEqual(counts["rated_images"], 3)
|
||||
self.db.update_prompt_rating(unrated, 2)
|
||||
self.assertEqual(self.db.library_counts()["rated_images"], 4)
|
||||
|
||||
def test_statistics_sum_real_run_counts(self):
|
||||
a = self.save("Alpha", category="Landscapes", rating=5)
|
||||
b = self.save("Beta", category="Landscapes", rating=4)
|
||||
self.save("Gamma", category="Landscapes") # unrated: not in the average
|
||||
c = self.save("Delta", category="Portraits", rating=3)
|
||||
blank = self.save("Epsilon", category=" ")
|
||||
uncategorised = self.save("Zeta")
|
||||
self.db.record_prompt_use(a, times=3)
|
||||
self.db.record_prompt_use(b, times=2)
|
||||
self.db.record_prompt_use(c)
|
||||
self.db.record_prompt_use(blank)
|
||||
self.db.record_prompt_use(uncategorised, times=4)
|
||||
# A negative-only prompt's runs are the pair's runs: never counted twice.
|
||||
pair = self.db.save_prompt_pair("Alpha", "Only negative")
|
||||
negative = self.db.get_prompt_pair(pair)["negative_prompt_id"]
|
||||
with self.db.model.get_connection() as connection:
|
||||
connection.execute(
|
||||
"UPDATE prompts SET category='Landscapes',run_count=50 WHERE id=?",
|
||||
(negative,),
|
||||
)
|
||||
rows = self.db.usage_by_category()
|
||||
self.assertEqual(
|
||||
rows,
|
||||
[
|
||||
{"category": "Landscapes", "runs": 5, "average_rating": 4.5},
|
||||
{"category": None, "runs": 5, "average_rating": None},
|
||||
{"category": "Portraits", "runs": 1, "average_rating": 3.0},
|
||||
],
|
||||
)
|
||||
|
||||
def test_counts_and_usage_write_nothing(self):
|
||||
self.save("Alpha", category="A")
|
||||
with self.db.model.get_connection() as connection:
|
||||
before = connection.total_changes
|
||||
self.db.library_counts()
|
||||
self.db.usage_by_category()
|
||||
with self.db.model.get_connection() as connection:
|
||||
self.assertEqual(connection.total_changes, before)
|
||||
self.assertFalse(connection.in_transaction)
|
||||
|
||||
|
||||
class DisplayTitleUnits(unittest.TestCase):
|
||||
"""The first-line title is cut like the Library's JavaScript title()."""
|
||||
|
||||
|
||||
@@ -51,12 +51,19 @@ from utils.prompt_inspector import (
|
||||
read_inspection_file,
|
||||
)
|
||||
|
||||
from utils.prompt_inspector import inspect_prompt_evidence, unreadable_evidence
|
||||
from utils.prompt_pair_resolver import ADDED_CONDITIONING_REASON, INSPECTION_PROFILE
|
||||
|
||||
try:
|
||||
from tests import inspection_parity as parity
|
||||
from tests import local_resolver_cases as resolver_cases
|
||||
from tests import local_resolver_selection_cases as selection_cases
|
||||
from tests.open_handles import assert_closed
|
||||
from tests.test_utils_video_metadata import HAVE_FFMPEG, write_video
|
||||
except ImportError: # pragma: no cover - discover -s tests puts tests/ first
|
||||
import inspection_parity as parity
|
||||
import local_resolver_cases as resolver_cases
|
||||
import local_resolver_selection_cases as selection_cases
|
||||
from open_handles import assert_closed
|
||||
from test_utils_video_metadata import HAVE_FFMPEG, write_video
|
||||
|
||||
@@ -1111,5 +1118,333 @@ class LocalProjectionParityTests(unittest.TestCase):
|
||||
self.assertTrue({"parsed", "malformed", "limited"} <= seen)
|
||||
|
||||
|
||||
def _server_evidence(records):
|
||||
return inspect_prompt_evidence(records, None, profile=INSPECTION_PROFILE)
|
||||
|
||||
|
||||
class LocalResolverParityTests(unittest.TestCase):
|
||||
"""``resolveLocalEvidence`` answers as ``inspect_prompt_evidence`` (L3 Tasks 8-9).
|
||||
|
||||
Python is the oracle: every expected value is the server's own answer on the
|
||||
same record texts. Task 8 ported one sampler's tracing; Task 9 ports
|
||||
candidate selection (several samplers, the identical-branch rule of #270,
|
||||
preview-only stages, detailer stages) and A1111 ``parameters``, so no
|
||||
record set is left to the server any more.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
self.addCleanup(assert_closed, self, temporary.name)
|
||||
self.directory = Path(temporary.name)
|
||||
|
||||
def _compare(self, cases):
|
||||
"""Assert each record set's local evidence equal; return the server answers."""
|
||||
observed = _run_oracle(
|
||||
self,
|
||||
[
|
||||
{"name": name, "mode": "resolve", "records": records}
|
||||
for name, records in cases.items()
|
||||
],
|
||||
)
|
||||
answers = {}
|
||||
for name, records in cases.items():
|
||||
with self.subTest(case=name):
|
||||
local = observed[name]
|
||||
server = _server_evidence(records)
|
||||
answers[name] = server
|
||||
self.assertEqual(
|
||||
_canonical_bytes(local["evidence"]), _canonical_bytes(server)
|
||||
)
|
||||
return answers
|
||||
|
||||
def test_local_roles_match_execution_and_single_chain_vectors(self):
|
||||
files = {}
|
||||
for name in parity.COMMITTED:
|
||||
files[name] = parity.fixture_path(name)
|
||||
for name in parity.TEST_TIME:
|
||||
path = parity.fixture_path(name, self.directory)
|
||||
parity.write_png(path, parity.build(name))
|
||||
files[name] = path
|
||||
observed = _run_oracle(
|
||||
self,
|
||||
[
|
||||
{"name": name, "mode": "resolve", "path": str(path)}
|
||||
for name, path in files.items()
|
||||
],
|
||||
)
|
||||
vectors = {
|
||||
vector["fixture"]: vector
|
||||
for vector in json.loads(parity.VECTORS_PATH.read_text(encoding="utf-8"))
|
||||
if vector["profile"] == "inspection"
|
||||
}
|
||||
compared = 0
|
||||
for name, path in files.items():
|
||||
with self.subTest(fixture=name):
|
||||
local = observed[name]
|
||||
try:
|
||||
records = read_inspection_file(path).metadata
|
||||
except MetadataReadError:
|
||||
self.assertTrue(local["unreadable"])
|
||||
continue
|
||||
# The single-chain subset (Task 8); the rest is Task 9's test below.
|
||||
if not resolver_cases.single_chain(records):
|
||||
continue
|
||||
server = _server_evidence(records)
|
||||
self.assertEqual(
|
||||
_canonical_bytes(local["evidence"]), _canonical_bytes(server)
|
||||
)
|
||||
# The committed vector agrees with the server read here.
|
||||
expect = vectors[name]["expect"]
|
||||
self.assertEqual(server["resolution"]["state"], expect["state"])
|
||||
self.assertEqual(
|
||||
server["resolution"]["reason_code"], expect["reason_code"]
|
||||
)
|
||||
compared += 1
|
||||
self.assertGreaterEqual(compared, 20)
|
||||
|
||||
def test_local_roles_match_inspection_vectors(self):
|
||||
"""Every inspection vector: local prompts, resolution and facts equal Images'.
|
||||
|
||||
The facts are ``extractGenerationFacts`` (with the legacy fill) on the
|
||||
local payload, against the facts the committed vectors pin for the
|
||||
server's response on the same file.
|
||||
"""
|
||||
vectors = [
|
||||
vector
|
||||
for vector in json.loads(parity.VECTORS_PATH.read_text(encoding="utf-8"))
|
||||
if vector["profile"] == "inspection"
|
||||
]
|
||||
files = {}
|
||||
for vector in vectors:
|
||||
name = vector["fixture"]
|
||||
if name in parity.TEST_TIME:
|
||||
path = parity.fixture_path(name, self.directory)
|
||||
parity.write_png(path, parity.build(name))
|
||||
else:
|
||||
path = parity.fixture_path(name)
|
||||
files[name] = path
|
||||
observed = _run_oracle(
|
||||
self,
|
||||
[
|
||||
{"name": name, "mode": "resolve", "path": str(path)}
|
||||
for name, path in files.items()
|
||||
],
|
||||
)
|
||||
for vector in vectors:
|
||||
name = vector["fixture"]
|
||||
with self.subTest(fixture=name):
|
||||
local, expect = observed[name], vector["expect"]
|
||||
try:
|
||||
server = _server_evidence(
|
||||
read_inspection_file(files[name]).metadata
|
||||
)
|
||||
except MetadataReadError:
|
||||
server = unreadable_evidence()
|
||||
self.assertTrue(local["unreadable"])
|
||||
self.assertEqual(
|
||||
_canonical_bytes(local["evidence"]), _canonical_bytes(server)
|
||||
)
|
||||
resolution, prompts = (
|
||||
local["evidence"]["resolution"],
|
||||
local["evidence"]["prompts"],
|
||||
)
|
||||
self.assertEqual(
|
||||
(
|
||||
resolution["state"],
|
||||
resolution["reason"],
|
||||
resolution["reason_code"],
|
||||
),
|
||||
(expect["state"], expect["reason"], expect["reason_code"]),
|
||||
)
|
||||
self.assertEqual(
|
||||
[
|
||||
parity._role_text(
|
||||
prompts[role]["status"], prompts[role]["text"]
|
||||
)
|
||||
for role in ("positive", "negative")
|
||||
],
|
||||
[expect["positive"], expect["negative"]],
|
||||
)
|
||||
self.assertEqual(local["facts"], expect["facts"])
|
||||
self.assertEqual(len(vectors), 28)
|
||||
|
||||
def test_local_roles_match_selection_record_sets(self):
|
||||
answers = self._compare(selection_cases.selection_cases())
|
||||
reasons = {answer["resolution"]["reason"] for answer in answers.values()}
|
||||
self.assertTrue(
|
||||
{
|
||||
"Every sampler branch uses the same prompt pair",
|
||||
"Multiple sampler branches",
|
||||
"Sampler stages have different prompts",
|
||||
"Cyclic sampler stage ancestry",
|
||||
"Expected an image link",
|
||||
"Unsupported sampler stage ancestry",
|
||||
"Resolution depth limit reached",
|
||||
"A1111 settings footer is not unique",
|
||||
"Multiple negative prompt delimiters",
|
||||
"Negative prompt delimiter is absent",
|
||||
"Parameters exceed the length limit",
|
||||
"No A1111 parameters",
|
||||
}
|
||||
<= reasons
|
||||
)
|
||||
# A stage reaching only previews is dropped: one sampler is named.
|
||||
self.assertEqual(
|
||||
answers["select-preview-dropped"]["resolution"]["sampler_id"], "3"
|
||||
)
|
||||
# Pass-through lineage: the chain shapes reach every new outcome.
|
||||
self.assertEqual(
|
||||
{
|
||||
name: answers[f"select-{name}"]["resolution"]["reason"]
|
||||
for name in ("hires-fix", "flux-three-stage", "hires-fix-differs")
|
||||
},
|
||||
{
|
||||
"hires-fix": "",
|
||||
"flux-three-stage": ADDED_CONDITIONING_REASON,
|
||||
"hires-fix-differs": "Sampler stages have different prompts",
|
||||
},
|
||||
)
|
||||
|
||||
def test_local_roles_match_fuzzed_pass_through_chains(self):
|
||||
cases = {}
|
||||
for seed in range(4):
|
||||
cases.update(selection_cases.passthrough_fuzz_cases(seed, 120))
|
||||
answers = self._compare(cases)
|
||||
reasons = [answer["resolution"]["reason"] for answer in answers.values()]
|
||||
for reason in (
|
||||
"",
|
||||
"Sampler stages have different prompts",
|
||||
"A later stage adds conditioning to the base prompt",
|
||||
"Combined conditioning has different texts",
|
||||
"Multiple sampler branches",
|
||||
):
|
||||
self.assertGreater(reasons.count(reason), 5, reason)
|
||||
|
||||
def test_local_roles_match_fuzzed_selection_sets(self):
|
||||
cases = {}
|
||||
for seed in range(4):
|
||||
cases.update(selection_cases.selection_fuzz_cases(seed, 150))
|
||||
answers = self._compare(cases)
|
||||
reasons = [answer["resolution"]["reason"] for answer in answers.values()]
|
||||
self.assertGreater(reasons.count("Multiple sampler branches"), 20)
|
||||
self.assertGreater(reasons.count("Sampler stages have different prompts"), 0)
|
||||
self.assertGreater(
|
||||
reasons.count("Every sampler branch uses the same prompt pair"), 0
|
||||
)
|
||||
self.assertGreater(reasons.count("A1111 settings footer is not unique"), 0)
|
||||
|
||||
def test_local_roles_match_golden_record_sets(self):
|
||||
answers = self._compare(resolver_cases.golden_cases())
|
||||
codes = {answer["resolution"]["reason_code"] for answer in answers.values()}
|
||||
self.assertTrue(
|
||||
{
|
||||
"malformed",
|
||||
"unsupported-node",
|
||||
"runtime-value",
|
||||
"composite",
|
||||
"work-limit",
|
||||
"depth-limit",
|
||||
"text-limit",
|
||||
"graph-ceiling",
|
||||
"no-sampler",
|
||||
None,
|
||||
}
|
||||
<= codes
|
||||
)
|
||||
sources = {
|
||||
side["source"]
|
||||
for answer in answers.values()
|
||||
for side in answer["prompts"].values()
|
||||
}
|
||||
self.assertTrue(
|
||||
{"execution", "graph-inferred", "zeroed-conditioning"} <= sources
|
||||
)
|
||||
|
||||
def test_committed_resolver_vectors_are_current(self):
|
||||
"""The Node unit test's committed answers are still the server's."""
|
||||
committed = json.loads(resolver_cases.VECTORS_PATH.read_text(encoding="utf-8"))
|
||||
self.assertEqual(
|
||||
committed,
|
||||
json.loads(json.dumps(resolver_cases.committed_vectors())),
|
||||
"regenerate with: python3 -m tests.local_resolver_cases --write",
|
||||
)
|
||||
|
||||
def test_local_roles_match_at_every_work_bound(self):
|
||||
answers = self._compare(resolver_cases.boundary_cases())
|
||||
for name in ("visits", "depth", "scanned"):
|
||||
self.assertEqual(answers[f"{name}-edge"]["resolution"]["state"], "resolved")
|
||||
self.assertIn(
|
||||
answers[f"{name}-over"]["resolution"]["reason_code"],
|
||||
("work-limit", "depth-limit"),
|
||||
)
|
||||
self.assertEqual(
|
||||
answers["named-astral"]["resolution"]["node"]["class_type"],
|
||||
"\U0001f600" * 64,
|
||||
)
|
||||
|
||||
def test_local_json_depth_cap_at_its_boundary(self):
|
||||
"""Both kinds of deep record at the browser's JSON depth cap (1,000).
|
||||
|
||||
Locally a record nested past the cap is invalid JSON: a prompt is
|
||||
malformed and a workflow is ignored, as the server answers when
|
||||
``json.loads`` raises RecursionError. The server's own limit depends on
|
||||
the Python (TESTING.md, "Deep JSON"): about 990 on 3.10 and 3.11, so it
|
||||
agrees past the cap there; about 10,000 on 3.12 and 3.13 and over
|
||||
50,000 on 3.14, so it agrees up to the cap there and reads deeper.
|
||||
"""
|
||||
cases = resolver_cases.depth_cases()
|
||||
observed = _run_oracle(
|
||||
self,
|
||||
[
|
||||
{"name": name, "mode": "resolve", "records": records}
|
||||
for name, _kind, _depth, records in cases
|
||||
],
|
||||
)
|
||||
newer = sys.version_info >= (3, 12)
|
||||
compared = 0
|
||||
for name, kind, depth, records in cases:
|
||||
with self.subTest(case=name):
|
||||
local = observed[name]["evidence"]["resolution"]
|
||||
within = depth <= resolver_cases.JSON_MAX_DEPTH
|
||||
if kind == "prompt":
|
||||
expected = (
|
||||
("resolved", None) if within else ("unresolved", "malformed")
|
||||
)
|
||||
else:
|
||||
# A workflow that is read finds the overwritten pair (R1).
|
||||
expected = (
|
||||
("unresolved", "malformed") if within else ("resolved", None)
|
||||
)
|
||||
self.assertEqual((local["state"], local["reason_code"]), expected)
|
||||
server = _server_evidence(records)
|
||||
if server["resolution"]["state"] == local["state"]:
|
||||
self.assertEqual(
|
||||
_canonical_bytes(observed[name]["evidence"]),
|
||||
_canonical_bytes(server),
|
||||
)
|
||||
compared += 1
|
||||
else:
|
||||
# The documented divergence only: an older server refuses
|
||||
# a little below the cap, a newer one reads past it.
|
||||
self.assertEqual(within, not newer)
|
||||
self.assertGreaterEqual(compared, 2)
|
||||
|
||||
def test_local_roles_match_fuzzed_record_sets(self):
|
||||
cases = {}
|
||||
for seed in range(4):
|
||||
cases.update(resolver_cases.fuzz_cases(seed, 150))
|
||||
answers = self._compare(cases)
|
||||
states = [answer["resolution"]["state"] for answer in answers.values()]
|
||||
# Coherent graphs reach real results, not only early refusals.
|
||||
self.assertGreater(states.count("resolved"), len(states) // 5)
|
||||
self.assertGreater(states.count("ambiguous"), 0)
|
||||
self.assertGreater(states.count("unresolved"), len(states) // 10)
|
||||
self.assertGreaterEqual(
|
||||
len({answer["resolution"]["reason_code"] for answer in answers.values()}),
|
||||
5,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -471,6 +471,94 @@ class CaptureStoreTestCase(unittest.TestCase):
|
||||
self.assertEqual(results["capture"].size, 60)
|
||||
self.capture(workers, "b.png")
|
||||
|
||||
def _hold_worker(self, store, relative, **options):
|
||||
"""Start a capture that holds a worker until ``proceed`` is set."""
|
||||
entered, proceed = threading.Event(), threading.Event()
|
||||
original = CaptureStore._read
|
||||
results = {}
|
||||
|
||||
def blocking(store_, stream, size):
|
||||
if threading.current_thread() is thread and not entered.is_set():
|
||||
entered.set()
|
||||
self.assertTrue(proceed.wait(WAIT))
|
||||
return original(store_, stream, size)
|
||||
|
||||
def worker():
|
||||
try:
|
||||
results["capture"] = self.capture(store, relative, **options)
|
||||
except Exception as error: # pragma: no cover - reported below
|
||||
results["error"] = error
|
||||
|
||||
patcher = patch.object(CaptureStore, "_read", blocking)
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
thread = threading.Thread(target=worker)
|
||||
thread.start()
|
||||
self.assertTrue(entered.wait(WAIT))
|
||||
|
||||
def finish():
|
||||
proceed.set()
|
||||
thread.join(WAIT)
|
||||
self.assertFalse(thread.is_alive())
|
||||
self.assertNotIn("error", results)
|
||||
return results["capture"]
|
||||
|
||||
self.addCleanup(proceed.set)
|
||||
return finish
|
||||
|
||||
def test_background_capture_never_takes_the_last_worker(self):
|
||||
for name in ("a", "b", "c"):
|
||||
self.write(f"{name}.png", bytes(60))
|
||||
store = self.make_store(max_workers=2)
|
||||
finish = self._hold_worker(store, "a.png", background=True)
|
||||
# The second background capture waits, then fails: one worker is free
|
||||
# but it is the interactive request's.
|
||||
started = time.monotonic()
|
||||
with patch("utils.media_capture.BACKGROUND_WAIT_SECONDS", 0.05):
|
||||
with self.assertRaises(CaptureCapacity):
|
||||
self.capture(store, "b.png", background=True)
|
||||
self.assertGreaterEqual(time.monotonic() - started, 0.04)
|
||||
# An interactive capture (Tag dialog, Apply) still gets the last one.
|
||||
interactive = self.capture(store, "c.png")
|
||||
self.assertEqual(interactive.size, 60)
|
||||
finish()
|
||||
|
||||
def test_background_capture_waits_for_a_worker_to_free(self):
|
||||
for name in ("a", "b"):
|
||||
self.write(f"{name}.png", bytes(60))
|
||||
store = self.make_store(max_workers=2)
|
||||
finish = self._hold_worker(store, "a.png", background=True)
|
||||
waited = {}
|
||||
|
||||
def background():
|
||||
waited["capture"] = self.capture(store, "b.png", background=True)
|
||||
|
||||
thread = threading.Thread(target=background)
|
||||
with patch("utils.media_capture.BACKGROUND_WAIT_SECONDS", WAIT):
|
||||
thread.start()
|
||||
finish()
|
||||
thread.join(WAIT)
|
||||
self.assertFalse(thread.is_alive())
|
||||
self.assertEqual(waited["capture"].size, 60)
|
||||
|
||||
def test_single_worker_store_admits_no_background_capture(self):
|
||||
self.write("a.png", bytes(60))
|
||||
store = self.make_store(max_workers=1)
|
||||
with patch("utils.media_capture.BACKGROUND_WAIT_SECONDS", 0.01):
|
||||
with self.assertRaises(CaptureCapacity):
|
||||
self.capture(store, "a.png", background=True)
|
||||
self.assertEqual(self.capture(store, "a.png").size, 60)
|
||||
|
||||
def test_background_capture_leaves_the_last_capture_slot(self):
|
||||
for name in ("a", "b", "c"):
|
||||
self.write(f"{name}.png", bytes(60))
|
||||
store = self.make_store(max_captures=2)
|
||||
self.capture(store, "a.png")
|
||||
with patch("utils.media_capture.BACKGROUND_WAIT_SECONDS", 0.01):
|
||||
with self.assertRaises(CaptureCapacity):
|
||||
self.capture(store, "b.png", background=True)
|
||||
self.assertEqual(self.capture(store, "c.png").size, 60)
|
||||
|
||||
# -- lifetime ------------------------------------------------------------
|
||||
|
||||
def test_idle_and_hard_expiry_and_in_use_retention(self):
|
||||
|
||||
@@ -0,0 +1,360 @@
|
||||
"""ComfyUI's ``NaN``/``Infinity`` metadata ingests, and is stored as strict JSON.
|
||||
|
||||
ComfyUI serializes the prompt with Python's ``json.dumps``, so a node whose
|
||||
``IS_CHANGED`` returns ``float("nan")`` (the "always re-run" idiom) writes
|
||||
``"is_changed": [NaN]`` into every saved PNG; widgets can hold ``Infinity``.
|
||||
The legacy pages replaced ``NaN`` with ``null`` before ``JSON.parse``, so such
|
||||
files must ingest. Whatever is stored or answered as JSON stays strict (the
|
||||
non-finite numbers become ``null``) and the prompt text and hash never change.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from PIL import Image, PngImagePlugin
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from tests.history_waits import drain_writer
|
||||
from tests.test_api_admin import AdminAPITestCase
|
||||
from tests.test_api_images import ImageAPITestCase
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.image_monitor import ImageGenerationHandler
|
||||
|
||||
try:
|
||||
from tests.open_handles import assert_closed
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from open_handles import assert_closed
|
||||
|
||||
# The words "NaN" and "Infinity" inside the prompt are text, never numbers.
|
||||
POSITIVE = " A cat named NaN beside an Infinity pool, ratio: NaN, [NaN] "
|
||||
NEGATIVE = "blurry"
|
||||
NAN, INF = float("nan"), float("inf")
|
||||
|
||||
|
||||
def graph(positive=POSITIVE, negative=NEGATIVE):
|
||||
"""An API prompt graph with non-finite numbers where ComfyUI writes them."""
|
||||
g = {
|
||||
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": positive}},
|
||||
"2": {"class_type": "CLIPTextEncode", "inputs": {"text": negative}},
|
||||
"3": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {"positive": ["1", 0], "negative": ["2", 0], "model": ["4", 0]},
|
||||
},
|
||||
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"5": {
|
||||
"class_type": "Image Filter",
|
||||
"inputs": {"strength": INF, "floor": -INF},
|
||||
"_meta": {"title": "Image Filter"},
|
||||
"is_changed": [NAN],
|
||||
},
|
||||
}
|
||||
return g
|
||||
|
||||
|
||||
WORKFLOW = {
|
||||
"nodes": [
|
||||
{"id": 5, "type": "Image Filter", "widgets_values": [NAN, INF, -INF]},
|
||||
],
|
||||
"links": [],
|
||||
}
|
||||
|
||||
|
||||
def comfy_text(value):
|
||||
"""The record text ComfyUI writes: Python's default, non-strict JSON."""
|
||||
text = json.dumps(value)
|
||||
assert "NaN" in text and "Infinity" in text
|
||||
return text
|
||||
|
||||
|
||||
def write_png(path, prompt_graph=None, workflow=WORKFLOW):
|
||||
info = PngImagePlugin.PngInfo()
|
||||
info.add_text("prompt", comfy_text(prompt_graph or graph()))
|
||||
if workflow is not None:
|
||||
info.add_text("workflow", comfy_text(workflow))
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with Image.new("RGB", (4, 4)) as image:
|
||||
image.save(path, pnginfo=info)
|
||||
return path
|
||||
|
||||
|
||||
def _reject(name):
|
||||
raise ValueError(f"non-strict JSON constant {name}")
|
||||
|
||||
|
||||
def strict_loads(text):
|
||||
return json.loads(text, parse_constant=_reject)
|
||||
|
||||
|
||||
class StrictRows:
|
||||
"""Assertions over the rows a write left behind."""
|
||||
|
||||
def image_rows(self, db):
|
||||
with db.model.get_connection() as conn:
|
||||
return [
|
||||
dict(row)
|
||||
for row in conn.execute(
|
||||
"SELECT prompt_id, workflow_data, prompt_metadata, parameters "
|
||||
"FROM generated_images ORDER BY id"
|
||||
)
|
||||
]
|
||||
|
||||
def assert_strict_snapshot(self, row, *, workflow=True):
|
||||
prompt = strict_loads(row["prompt_metadata"])
|
||||
self.assertEqual(prompt["5"]["is_changed"], [None])
|
||||
self.assertEqual(prompt["5"]["inputs"], {"strength": None, "floor": None})
|
||||
self.assertEqual(prompt["1"]["inputs"]["text"], POSITIVE)
|
||||
stored_workflow = strict_loads(row["workflow_data"])
|
||||
if workflow:
|
||||
self.assertEqual(
|
||||
stored_workflow["nodes"][0]["widgets_values"], [None, None, None]
|
||||
)
|
||||
if row["parameters"] is not None:
|
||||
strict_loads(row["parameters"])
|
||||
|
||||
def assert_prompt_text(self, db, prompt_id):
|
||||
with db.model.get_connection() as conn:
|
||||
row = conn.execute(
|
||||
"SELECT text, hash FROM prompts WHERE id=?", (prompt_id,)
|
||||
).fetchone()
|
||||
self.assertEqual(row["text"], POSITIVE.strip())
|
||||
self.assertEqual(row["hash"], generate_prompt_hash(POSITIVE))
|
||||
|
||||
|
||||
def dumps_strict(value, **kwargs):
|
||||
from utils.json_strict import dumps_strict as dumps
|
||||
|
||||
return dumps(value, **kwargs)
|
||||
|
||||
|
||||
class TestDumpsStrict(unittest.TestCase):
|
||||
def test_finite_values_serialize_byte_identically(self):
|
||||
value = {"b": [1, 2.5, "NaN", None, True], "a": {"x": "Infinity"}}
|
||||
for kwargs in ({}, {"sort_keys": True, "separators": (",", ":")}):
|
||||
with self.subTest(kwargs=kwargs):
|
||||
self.assertEqual(
|
||||
dumps_strict(value, **kwargs),
|
||||
json.dumps(value, allow_nan=False, **kwargs),
|
||||
)
|
||||
|
||||
def test_non_finite_numbers_become_null_and_strings_stay(self):
|
||||
value = {"n": NAN, "i": [INF, -INF], "s": "NaN Infinity", "t": (NAN, 1)}
|
||||
encoded = dumps_strict(value, sort_keys=True)
|
||||
self.assertEqual(
|
||||
strict_loads(encoded),
|
||||
{"n": None, "i": [None, None], "s": "NaN Infinity", "t": [None, 1]},
|
||||
)
|
||||
|
||||
def test_other_errors_still_raise(self):
|
||||
cycle = []
|
||||
cycle.append(cycle)
|
||||
for value in (cycle, [NAN, cycle]):
|
||||
with self.assertRaisesRegex(ValueError, "Circular"):
|
||||
dumps_strict(value)
|
||||
with self.assertRaises(TypeError):
|
||||
dumps_strict({"x": object()})
|
||||
self.assertEqual(
|
||||
dumps_strict({"x": object()}, default=lambda _: NAN), '{"x": null}'
|
||||
)
|
||||
|
||||
|
||||
class TestScanIngestsNonFiniteMetadata(StrictRows, AdminAPITestCase):
|
||||
def ingest(self, name, prompt_graph=None):
|
||||
path = write_png(self.output_dir / name, prompt_graph)
|
||||
extracted = self.api._extract_comfyui_metadata(str(path))
|
||||
return self.api._ingest_scan_batch_sync([(path, extracted)])
|
||||
|
||||
def test_resolved_pair_with_nan_is_added_with_strict_snapshots(self):
|
||||
counts = self.ingest("nan.png")
|
||||
self.assertEqual(counts["failed"], 0)
|
||||
self.assertEqual(counts["added"], 1)
|
||||
rows = self.image_rows(self.api.db)
|
||||
self.assertEqual(len(rows), 1)
|
||||
self.assert_strict_snapshot(rows[0])
|
||||
self.assert_prompt_text(self.api.db, rows[0]["prompt_id"])
|
||||
pairs = self.api.db.list_prompt_pairs()["pairs"]
|
||||
self.assertEqual(
|
||||
[(p["positive_text"], p["negative_text"]) for p in pairs],
|
||||
[(POSITIVE.strip(), NEGATIVE)],
|
||||
)
|
||||
# A repeated scan compares the same strict snapshot: unchanged.
|
||||
self.assertEqual(self.ingest("nan.png")["unchanged"], 1)
|
||||
|
||||
async def test_scan_route_reports_no_failures(self):
|
||||
from py.config import GalleryConfig
|
||||
|
||||
write_png(self.output_dir / "nan.png")
|
||||
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
|
||||
resp = await self.client.request("POST", "/prompt_manager/scan")
|
||||
body = await resp.text()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIn('"type": "complete"', body)
|
||||
self.assertIn('"added": 1', body)
|
||||
self.assertIn('"failed": 0', body)
|
||||
|
||||
|
||||
class TestMonitorIngestsNonFiniteMetadata(StrictRows, unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.db = PromptDatabase(str(Path(self.temp.name) / "db.sqlite"))
|
||||
self.pid = self.db.save_prompt(
|
||||
POSITIVE.strip(), prompt_hash=generate_prompt_hash(POSITIVE)
|
||||
)
|
||||
self.tracker = SimpleNamespace(
|
||||
pop_next_prompt=lambda: None,
|
||||
get_current_prompt=lambda: None,
|
||||
extend_prompt_timeout=lambda *args: None,
|
||||
)
|
||||
self.handler = ImageGenerationHandler(self.db, self.tracker)
|
||||
self.handler.logger = logging.getLogger("test.nonfinite.monitor")
|
||||
self.handler.logger.disabled = True
|
||||
|
||||
def tearDown(self):
|
||||
self.db.close_all()
|
||||
assert_closed(self, self.temp.name)
|
||||
self.temp.cleanup()
|
||||
|
||||
def test_new_image_with_nan_links_its_pair(self):
|
||||
# The monitor links only images a PromptManager node produced.
|
||||
g = graph()
|
||||
g["1"]["class_type"] = "PromptManager"
|
||||
path = write_png(Path(self.temp.name) / "out" / "nan.png", g)
|
||||
self.handler.process_new_image(str(path))
|
||||
rows = self.image_rows(self.db)
|
||||
self.assertEqual([row["prompt_id"] for row in rows], [self.pid])
|
||||
self.assert_strict_snapshot(rows[0])
|
||||
self.assert_prompt_text(self.db, self.pid)
|
||||
pairs = self.db.list_prompt_pairs()["pairs"]
|
||||
self.assertEqual(len(pairs), 1)
|
||||
self.assertEqual(pairs[0]["image_count"], 1)
|
||||
|
||||
def test_legacy_link_stores_strict_json(self):
|
||||
# The plain prompt link (no exact pair) must not write NaN either.
|
||||
path = str(Path(self.temp.name) / "plain.png")
|
||||
metadata = {"prompt": graph(), "workflow": WORKFLOW, "parameters": {"x": NAN}}
|
||||
self.assertTrue(self.db.link_image_to_prompt(self.pid, path, metadata))
|
||||
row = self.image_rows(self.db)[0]
|
||||
self.assert_strict_snapshot(row)
|
||||
self.assertEqual(strict_loads(row["parameters"]), {"x": None})
|
||||
|
||||
def test_stored_images_read_back_without_non_finite_numbers(self):
|
||||
path = str(Path(self.temp.name) / "legacy.png")
|
||||
self.db.link_image_to_prompt(self.pid, path, {"prompt": graph()})
|
||||
with self.db.model.get_connection() as conn:
|
||||
# A row an older version wrote with Python's default json.dumps.
|
||||
conn.execute(
|
||||
"UPDATE generated_images SET prompt_metadata=?", (comfy_text(graph()),)
|
||||
)
|
||||
conn.commit()
|
||||
images = self.db.get_prompt_images(str(self.pid))
|
||||
strict_loads(json.dumps(images, default=str))
|
||||
|
||||
|
||||
class TestInspectorAnswersStrictJSON(ImageAPITestCase):
|
||||
async def test_inspect_route_resolves_and_answers_strict_json(self):
|
||||
write_png(self.output_dir / "nan.png")
|
||||
response = await self.client.get(
|
||||
"/prompt_manager/images/inspect/nan.png", params={"root": "0"}
|
||||
)
|
||||
self.assertEqual(response.status, 200)
|
||||
result = strict_loads(await response.text())
|
||||
self.assertEqual(result["resolution"]["state"], "resolved")
|
||||
self.assertEqual(
|
||||
result["prompts"]["positive"],
|
||||
{"status": "resolved", "text": POSITIVE, "source": "graph-inferred"},
|
||||
)
|
||||
|
||||
async def test_prompt_images_route_answers_strict_json(self):
|
||||
pid = self.api.db.save_prompt(
|
||||
POSITIVE.strip(), prompt_hash=generate_prompt_hash(POSITIVE)
|
||||
)
|
||||
path = str(write_png(self.output_dir / "linked.png"))
|
||||
self.api.db.link_image_to_prompt(pid, path, {"prompt": graph()})
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE generated_images SET prompt_metadata=?", (comfy_text(graph()),)
|
||||
)
|
||||
conn.commit()
|
||||
response = await self.client.get(f"/prompt_manager/prompts/{pid}/images")
|
||||
self.assertEqual(response.status, 200)
|
||||
result = strict_loads(await response.text())
|
||||
self.assertEqual(len(result["images"]), 1)
|
||||
stored = result["images"][0]["prompt_metadata"]
|
||||
self.assertEqual(stored["1"]["inputs"]["text"], POSITIVE)
|
||||
self.assertEqual(stored["5"]["is_changed"], [None])
|
||||
|
||||
|
||||
class TestHistoryCapturesNonFiniteGraphs(unittest.TestCase):
|
||||
"""The queued graph ComfyUI hands the hook may carry float('nan')."""
|
||||
|
||||
JOB = "a0c26da3-eb26-405c-b143-c662a84c3112"
|
||||
|
||||
def setUp(self):
|
||||
self.directory = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.directory.cleanup)
|
||||
self.db = PromptDatabase(os.path.join(self.directory.name, "history.db"))
|
||||
self.addCleanup(self.db.model.reset_schema_cache)
|
||||
self.addCleanup(self.db.model.close_all)
|
||||
|
||||
def test_queued_graph_with_nan_records_its_run_and_contexts(self):
|
||||
from tests.test_execution_history_hooks import HostQueue, status
|
||||
from utils.execution_context_capture import record_runtime_context
|
||||
from utils.execution_history import register_history_hook
|
||||
|
||||
queue = HostQueue()
|
||||
registration = register_history_hook(
|
||||
SimpleNamespace(prompt_queue=queue), lambda: self.db
|
||||
)
|
||||
self.addCleanup(lambda: registration.dispose(3))
|
||||
self.addCleanup(registration.writer.invalidate)
|
||||
deadline = time.monotonic() + 3
|
||||
while not registration.writer.status().accepting:
|
||||
self.assertLess(time.monotonic(), deadline, "writer never accepted")
|
||||
time.sleep(0.005)
|
||||
g = {
|
||||
"p": {
|
||||
"class_type": "PromptManagerPair",
|
||||
"inputs": {"text": POSITIVE, "negative_text": NEGATIVE},
|
||||
},
|
||||
"s": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {"positive": ["p", 0], "negative": ["p", 1], "cfg": INF},
|
||||
"is_changed": [NAN],
|
||||
},
|
||||
"5": graph()["5"],
|
||||
}
|
||||
extra = {"extra_pnginfo": {"workflow": WORKFLOW}}
|
||||
queue.results.append(((0, self.JOB, g, extra, ["s"]), 1))
|
||||
queue.get()
|
||||
host = types.ModuleType("comfy_execution.utils")
|
||||
host.get_executing_context = lambda: SimpleNamespace(
|
||||
prompt_id=self.JOB, node_id="p", list_index=0
|
||||
)
|
||||
with patch.dict(sys.modules, {"comfy_execution.utils": host}):
|
||||
for role, text in (("positive", POSITIVE), ("negative", NEGATIVE)):
|
||||
record_runtime_context(
|
||||
"PromptManagerPair", text, "", "", text, role=role
|
||||
)
|
||||
queue.task_done(1, {}, status("execution_success", job=self.JOB))
|
||||
drain_writer(registration.writer, clock=time.monotonic)
|
||||
with self.db.model.get_connection() as conn:
|
||||
conn.row_factory = sqlite3.Row
|
||||
runs = [dict(r) for r in conn.execute("SELECT * FROM execution_runs")]
|
||||
self.assertEqual([run["outcome"] for run in runs], ["succeeded"])
|
||||
contexts = self.db.get_execution_contexts(runs[0]["run_id"])
|
||||
self.assertEqual(contexts["state"], "complete")
|
||||
self.assertEqual(contexts["contexts"][0]["positive"]["raw"], POSITIVE)
|
||||
strict_loads(json.dumps(contexts))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -160,3 +160,69 @@ class TestPairIngestion(unittest.TestCase):
|
||||
info = self.db.get_image_prompt_info(self.path)
|
||||
self.assertEqual(info["pair_context"]["positive_text"], "cat, lora_trigger")
|
||||
self.assertEqual(info["pair_context"]["text_source"], "execution")
|
||||
|
||||
def test_public_ingest_resolved_pair_contract_unchanged(self):
|
||||
"""The public ingester keeps its result shape and its own transaction;
|
||||
the extracted ``_ingest_resolved_pair_in`` runs in the caller's."""
|
||||
from database import pair_ingestion
|
||||
|
||||
self.assertTrue(
|
||||
hasattr(pair_ingestion, "_ingest_resolved_pair_in"),
|
||||
"_ingest_resolved_pair_in is not extracted",
|
||||
)
|
||||
for state in ("ambiguous", "unresolved", "conflict"):
|
||||
self.assertEqual(
|
||||
ingest_resolved_pair(self.db, self.path, {}, {"state": state}),
|
||||
{"status": state, "pairs_added": 0, "links_added": 0},
|
||||
)
|
||||
first = ingest_resolved_pair(self.db, self.path, {}, resolved())
|
||||
self.assertEqual(
|
||||
set(first),
|
||||
{"status", "pairs_added", "links_added", "pair_id", "image_id"},
|
||||
)
|
||||
self.assertEqual(first["status"], "added")
|
||||
(link,) = self.db.get_image_prompt_pairs(first["image_id"])
|
||||
self.assertEqual(
|
||||
link["provenance"],
|
||||
{
|
||||
"resolver_version": 1,
|
||||
"positive": resolved()["positive"],
|
||||
"negative": resolved()["negative"],
|
||||
},
|
||||
)
|
||||
conn = self.db.model.get_connection()
|
||||
self.assertFalse(conn.in_transaction)
|
||||
# Inside the caller's transaction, with the caller's provenance; an
|
||||
# exception afterwards rolls the link back with the caller's writes.
|
||||
provenance = {"kind": "user-selection", "resolver_version": 1}
|
||||
with self.assertRaises(RuntimeError):
|
||||
with self.db._pair_transaction() as held:
|
||||
inner = pair_ingestion._ingest_resolved_pair_in(
|
||||
held,
|
||||
self.db,
|
||||
self.path + "2",
|
||||
{},
|
||||
resolved("dog"),
|
||||
provenance=provenance,
|
||||
)
|
||||
self.assertEqual(inner["status"], "added")
|
||||
raise RuntimeError("caller failed")
|
||||
self.assertIsNone(self.db.get_image_prompt_info(self.path + "2"))
|
||||
with self.db._pair_transaction() as held:
|
||||
inner = pair_ingestion._ingest_resolved_pair_in(
|
||||
held,
|
||||
self.db,
|
||||
self.path + "2",
|
||||
{},
|
||||
resolved("dog"),
|
||||
provenance=provenance,
|
||||
)
|
||||
(link,) = self.db.get_image_prompt_pairs(inner["image_id"])
|
||||
self.assertEqual(link["provenance"], provenance)
|
||||
with self.db._pair_transaction() as held:
|
||||
conflict = pair_ingestion._ingest_resolved_pair_in(
|
||||
held, self.db, self.path, {}, resolved("bird")
|
||||
)
|
||||
self.assertEqual(
|
||||
conflict, {"status": "conflict", "pairs_added": 0, "links_added": 0}
|
||||
)
|
||||
|
||||
+120
-2
@@ -82,7 +82,8 @@ class TestPairScan(AdminAPITestCase):
|
||||
"class_type": "BasicGuider",
|
||||
"inputs": {"width": 2048, "height": 2048},
|
||||
}
|
||||
self.assertEqual(self._scanned("untraceable.png", g), "ambiguous")
|
||||
# Ambiguous with nothing to choose: unresolved, never a review item.
|
||||
self.assertEqual(self._scanned("untraceable.png", g), "unresolved")
|
||||
self.assertEqual(self.api.db.list_prompt_pairs()["total"], 0)
|
||||
self.assertEqual(self._texts(), [])
|
||||
|
||||
@@ -159,12 +160,129 @@ class TestPairScan(AdminAPITestCase):
|
||||
"\nSteps: 20, Sampler: Euler"
|
||||
)
|
||||
}
|
||||
# No candidate to choose: reported unresolved, still not guessed.
|
||||
self.assertEqual(
|
||||
self.api._ingest_scanned_file(self.output_dir / "ambiguous.png", meta),
|
||||
"ambiguous",
|
||||
"unresolved",
|
||||
)
|
||||
self.assertEqual(self.api.db.get_recent_prompts()["prompts"], [])
|
||||
|
||||
def _nan_scanned(self, name, prompt_graph):
|
||||
"""Scan ``prompt_graph`` as ComfyUI writes it: ``is_changed`` is NaN."""
|
||||
stamped = {
|
||||
node_id: {**node, "is_changed": [float("nan")]}
|
||||
for node_id, node in prompt_graph.items()
|
||||
}
|
||||
text = json.dumps(stamped)
|
||||
self.assertIn("NaN", text)
|
||||
return self._scanned_text(name, text)
|
||||
|
||||
def _scanned_text(self, name, prompt_text):
|
||||
meta = {"prompt": prompt_text}
|
||||
return self.api._ingest_scanned_file(self.output_dir / name, meta)
|
||||
|
||||
@staticmethod
|
||||
def _hires_fix_graph(positive):
|
||||
"""A PromptManager prompt on a base sampler and a hires-fix sampler.
|
||||
|
||||
The base latent comes from a custom batch node and the hires pass from
|
||||
a custom image filter: no sampler branch traces, so the scan links the
|
||||
prompt the legacy pages show.
|
||||
"""
|
||||
return {
|
||||
"1": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"inputs": {"ckpt_name": "m.safetensors"},
|
||||
},
|
||||
"2": {
|
||||
"class_type": "MyTriggerWords",
|
||||
"inputs": {"trigger_words": ["1", 2]},
|
||||
},
|
||||
"3": {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {
|
||||
"text": positive,
|
||||
"prepend_text": ["2", 0],
|
||||
"append_text": "",
|
||||
"clip": ["1", 1],
|
||||
},
|
||||
},
|
||||
"4": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": "blur", "clip": ["1", 1]},
|
||||
},
|
||||
"5": {"class_type": "MyLatentBatch", "inputs": {"batch_size": 6}},
|
||||
"6": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"seed": 7,
|
||||
"model": ["1", 0],
|
||||
"positive": ["3", 0],
|
||||
"negative": ["4", 0],
|
||||
"latent_image": ["5", 0],
|
||||
},
|
||||
},
|
||||
"7": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {"samples": ["6", 0], "vae": ["1", 2]},
|
||||
},
|
||||
"8": {"class_type": "MyImageFilter", "inputs": {"images": ["7", 0]}},
|
||||
"9": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {"pixels": ["8", 0], "vae": ["1", 2]},
|
||||
},
|
||||
"14": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"seed": 7,
|
||||
"denoise": 0.25,
|
||||
"model": ["1", 0],
|
||||
"positive": ["3", 0],
|
||||
"negative": ["4", 0],
|
||||
"latent_image": ["9", 0],
|
||||
},
|
||||
},
|
||||
"15": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {"samples": ["14", 0], "vae": ["1", 2]},
|
||||
},
|
||||
"16": {"class_type": "SaveImage", "inputs": {"images": ["15", 0]}},
|
||||
}
|
||||
|
||||
def test_hires_fix_prompt_links_with_or_without_nan_is_changed(self):
|
||||
# Legacy parity: the legacy pages read NaN as null before parsing, so they
|
||||
# showed this prompt; the scan left NaN-stamped files in review.
|
||||
for name, scan in (("plain.png", None), ("stamped.png", "nan")):
|
||||
with self.subTest(name=name):
|
||||
g = self._hires_fix_graph("cat on a sofa")
|
||||
if scan:
|
||||
status = self._nan_scanned(name, g)
|
||||
else:
|
||||
status = self._scanned(name, g)
|
||||
self.assertIn(status, ("added", "linked"))
|
||||
self.assertEqual(self._texts(), ["cat on a sofa"])
|
||||
|
||||
def test_untraced_single_sampler_links_the_legacy_prompt_with_nan(self):
|
||||
g = self._hires_fix_graph("cat on a sofa")
|
||||
for node_id in ("7", "8", "9", "14", "15"):
|
||||
del g[node_id]
|
||||
g["16"]["inputs"]["images"] = ["6", 0]
|
||||
self.assertEqual(self._nan_scanned("single.png", g), "added")
|
||||
self.assertEqual(self._texts(), ["cat on a sofa"])
|
||||
|
||||
def test_nan_inside_prompt_text_is_kept(self):
|
||||
# Only the NaN number reads as null; legacy's regex also rewrote text.
|
||||
g = self._hires_fix_graph("NaN cat, value: NaN")
|
||||
self.assertEqual(self._nan_scanned("text.png", g), "added")
|
||||
self.assertEqual(self._texts(), ["NaN cat, value: NaN"])
|
||||
|
||||
def test_infinity_is_still_unreadable_like_the_legacy_pages(self):
|
||||
text = json.dumps(self._hires_fix_graph("cat")).replace(
|
||||
'"seed": 7', '"seed": Infinity'
|
||||
)
|
||||
self.assertIn(self._scanned_text("inf.png", text), ("ambiguous", "unresolved"))
|
||||
self.assertEqual(self._texts(), [])
|
||||
|
||||
def test_raw_png_workflow_is_stored_as_object_and_repeat_is_unchanged(self):
|
||||
path = self.output_dir / "workflow.png"
|
||||
workflow = {"nodes": [{"id": 1, "type": "PromptManagerPair"}]}
|
||||
|
||||
@@ -258,6 +258,10 @@ class PairSchemaTests(unittest.TestCase):
|
||||
],
|
||||
[(1, 1, 0)],
|
||||
)
|
||||
self.assertEqual(
|
||||
[tuple(r) for r in conn.execute("SELECT * FROM media_review_metadata")],
|
||||
[(1, 1, 0)],
|
||||
)
|
||||
self.assertTrue(PromptModel.verify_database_file(live.db_path)[0])
|
||||
self.assertTrue(
|
||||
PromptModel.verify_database_file(live.db_path, require_library_state=True)[
|
||||
|
||||
@@ -1245,7 +1245,7 @@ class TestGuiderSamplers(unittest.TestCase):
|
||||
source["3"]["inputs"]["guider"] = [previous, 0]
|
||||
self.assertEqual(_inspect(source)["reason_code"], "depth-limit")
|
||||
|
||||
def test_custom_advanced_upstream_of_a_sampler_is_multistage(self):
|
||||
def test_custom_advanced_upstream_of_a_sampler_is_a_compared_stage(self):
|
||||
source = _guided()
|
||||
source["41"] = {
|
||||
"class_type": "KSampler",
|
||||
@@ -1256,15 +1256,18 @@ class TestGuiderSamplers(unittest.TestCase):
|
||||
},
|
||||
}
|
||||
source["8"]["inputs"]["samples"] = ["41", 0]
|
||||
# 41 continues 3's latent and reads the same texts as 3's guider.
|
||||
for anchor in ("9", None):
|
||||
result = _inspect(source, output_node_id=anchor)
|
||||
self.assertEqual(
|
||||
(result["state"], result["sampler_id"]), ("resolved", "41")
|
||||
)
|
||||
source["41"]["inputs"]["negative"] = ["6", 0]
|
||||
result = _inspect(source, output_node_id="9")
|
||||
self.assertEqual(result["sampler_id"], "41")
|
||||
self.assertEqual(result["state"], "unresolved")
|
||||
self.assertEqual(result["reason"], "Multistage sampler ancestry is ambiguous")
|
||||
unanchored = _inspect(source)
|
||||
self.assertEqual(unanchored["state"], "ambiguous")
|
||||
self.assertEqual(result["state"], "ambiguous")
|
||||
self.assertEqual(result["reason_code"], "multistage-differs")
|
||||
self.assertEqual(
|
||||
sorted(item["sampler_id"] for item in unanchored["candidates"]),
|
||||
["3", "41"],
|
||||
sorted(item["sampler_id"] for item in result["candidates"]), ["3", "41"]
|
||||
)
|
||||
|
||||
|
||||
@@ -2632,17 +2635,19 @@ class TestLatentAncestry(unittest.TestCase):
|
||||
self.assertEqual(_sides(result), (parity.POS, parity.NEG))
|
||||
self.assertEqual(_inspect(source, output_node_id="9"), result)
|
||||
|
||||
def test_upstream_sampler_through_vaedecode_vaeencode_is_multistage(self):
|
||||
for positive in (OTHER, parity.POS):
|
||||
with self.subTest(positive=positive):
|
||||
result = _inspect(_image_to_image(positive), output_node_id="63")
|
||||
# Until PR3 compares stage texts, never one stage's prompt.
|
||||
self.assertEqual(result["state"], "unresolved")
|
||||
self.assertEqual(result["sampler_id"], "61")
|
||||
self.assertEqual(result["reason"], MULTISTAGE)
|
||||
self.assertIsNone(result["reason_code"])
|
||||
self.assertIsNone(result["positive"])
|
||||
self.assertIsNone(result["negative"])
|
||||
def test_upstream_sampler_through_vaedecode_vaeencode_is_compared(self):
|
||||
# The latent continues the first stage's through lineage nodes: the
|
||||
# stages are compared (spec 9.3), equal prompts resolve, others differ.
|
||||
result = _inspect(_image_to_image(parity.POS), output_node_id="63")
|
||||
self.assertEqual((result["state"], result["sampler_id"]), ("resolved", "61"))
|
||||
self.assertEqual(_sides(result), (parity.POS, parity.NEG))
|
||||
result = _inspect(_image_to_image(OTHER), output_node_id="63")
|
||||
self.assertEqual(result["state"], "ambiguous")
|
||||
self.assertEqual(result["sampler_id"], "61")
|
||||
self.assertEqual(result["reason_code"], "multistage-differs")
|
||||
self.assertEqual(
|
||||
[item["sampler_id"] for item in result["candidates"]], ["61", "3"]
|
||||
)
|
||||
# An upstream sampler reached through a non-latent input is a stage too.
|
||||
source = _latent_from(
|
||||
{
|
||||
@@ -2959,9 +2964,15 @@ class TestLatentAncestry(unittest.TestCase):
|
||||
result = _inspect(source, output_node_id="9")
|
||||
self.assertEqual(result["reason_code"], "unsupported-node")
|
||||
self.assertEqual(result["node"], {"id": "60", "class_type": "MysteryUpscaler"})
|
||||
# The listed stage is now an earlier sampler stage in the ancestry.
|
||||
# The listed stage is now an earlier sampler stage, continued through
|
||||
# VAEEncode: its prompt equals this stage's, so the chain resolves.
|
||||
source["60"]["class_type"] = "UltimateSDUpscale"
|
||||
result = _inspect(source, output_node_id="9")
|
||||
self.assertEqual((result["state"], result["sampler_id"]), ("resolved", "3"))
|
||||
# Reached only through an unlisted image node, it stays unattributed.
|
||||
source["61"] = {"class_type": "MysteryFilter", "inputs": {"image": ["60", 0]}}
|
||||
source["5"]["inputs"]["pixels"] = ["61", 0]
|
||||
result = _inspect(source, output_node_id="9")
|
||||
self.assertEqual(result["reason"], resolver.MULTISTAGE_UNATTRIBUTED_REASON)
|
||||
|
||||
def test_latent_walk_budget_exhaustion_is_work_limit(self):
|
||||
|
||||
@@ -0,0 +1,419 @@
|
||||
"""Multi-stage chains joined through image-space pass-through nodes.
|
||||
|
||||
A hires-fix or refiner run decodes sampler A's latent, filters, upscales and
|
||||
scales the image, then encodes it again for sampler B. The semantics table's
|
||||
``lineage`` section lists the nodes through which the latent or image carries
|
||||
on unchanged in kind. When B's latent descends from A only through those nodes
|
||||
and the stages share one pair, the chain resolves to it. A last stage that adds
|
||||
conditioning to the base prompt (ConditioningCombine of the base and a style
|
||||
trigger) keeps the base pair: the floor legacy linked. Everything else stays
|
||||
ambiguous or unresolved. Ingestion (the default profile) is unchanged (D1).
|
||||
"""
|
||||
|
||||
import json
|
||||
import unittest
|
||||
|
||||
from utils import prompt_pair_resolver as resolver
|
||||
from utils.prompt_inspector import inspect_prompt_evidence
|
||||
from utils.prompt_pair_resolver import resolve_prompt_pair
|
||||
|
||||
try:
|
||||
from tests import passthrough_graphs as shapes
|
||||
except ImportError: # discovered with ``-s tests``
|
||||
import passthrough_graphs as shapes
|
||||
|
||||
|
||||
def _inspect(graph, **options):
|
||||
return resolve_prompt_pair(
|
||||
{"prompt": graph}, profile=resolver.INSPECTION_PROFILE, **options
|
||||
)
|
||||
|
||||
|
||||
def _texts(result):
|
||||
return tuple(
|
||||
result[role]["text"] if isinstance(result[role], dict) else None
|
||||
for role in ("positive", "negative")
|
||||
)
|
||||
|
||||
|
||||
class TestHiresFixChain(unittest.TestCase):
|
||||
def test_shared_pair_resolves_through_image_space_pass_through(self):
|
||||
graph = shapes.hires_fix()
|
||||
for anchor in (None, "22"):
|
||||
with self.subTest(anchor=anchor):
|
||||
result = _inspect(graph, output_node_id=anchor)
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
self.assertEqual(result["sampler_id"], "20")
|
||||
self.assertEqual(_texts(result), (shapes.POSITIVE, shapes.NEGATIVE))
|
||||
self.assertIsNone(result["reason_code"])
|
||||
|
||||
def test_the_earlier_stage_is_judged_in_its_chain(self):
|
||||
# Sampler 10 only feeds sampler 20, so the file is 20's: one candidate.
|
||||
result = _inspect(shapes.hires_fix())
|
||||
self.assertEqual(result["candidates"], [])
|
||||
|
||||
def test_inspector_shows_the_shared_prompt(self):
|
||||
evidence = inspect_prompt_evidence({"prompt": json.dumps(shapes.hires_fix())})
|
||||
self.assertEqual(evidence["resolution"]["state"], "resolved")
|
||||
self.assertEqual(
|
||||
[evidence["prompts"][role]["text"] for role in ("positive", "negative")],
|
||||
[shapes.POSITIVE, shapes.NEGATIVE],
|
||||
)
|
||||
|
||||
def test_different_stage_prompts_are_multistage_ambiguous(self):
|
||||
result = _inspect(shapes.hires_fix(second_positive=shapes.OTHER))
|
||||
self.assertEqual(result["state"], "ambiguous")
|
||||
self.assertEqual(result["reason_code"], "multistage-differs")
|
||||
self.assertEqual(
|
||||
sorted(_texts(item)[0] for item in result["candidates"]),
|
||||
sorted([shapes.POSITIVE, shapes.OTHER]),
|
||||
)
|
||||
|
||||
def test_an_unlisted_image_node_breaks_the_lineage(self):
|
||||
# The walk still finds the earlier sampler, but not through listed
|
||||
# pass-through nodes: the stages are never compared.
|
||||
result = _inspect(shapes.hires_fix(filter_class="MysteryImageEffect"))
|
||||
self.assertEqual(result["state"], "ambiguous")
|
||||
later = next(
|
||||
item for item in result["candidates"] if item["sampler_id"] == "20"
|
||||
)
|
||||
self.assertEqual(later["reason"], resolver.MULTISTAGE_UNATTRIBUTED_REASON)
|
||||
|
||||
def test_a_second_earlier_sampler_in_the_ancestry_stays_unattributed(self):
|
||||
graph = shapes.hires_fix()
|
||||
graph["30"] = shapes.ksampler(["2", 0], ["3", 0], ["4", 0])
|
||||
graph["31"] = {"class_type": "VAEDecode", "inputs": {"samples": ["30", 0]}}
|
||||
graph["32"] = {
|
||||
"class_type": "ImageBlend",
|
||||
"inputs": {"image1": ["14", 0], "image2": ["31", 0], "blend_factor": 0.5},
|
||||
}
|
||||
graph["16"]["inputs"]["image"] = ["32", 0]
|
||||
result = _inspect(graph, output_node_id="22")
|
||||
self.assertEqual(result["state"], "unresolved")
|
||||
self.assertEqual(result["reason"], resolver.MULTISTAGE_UNATTRIBUTED_REASON)
|
||||
|
||||
def test_a_direct_latent_link_is_a_chain_too(self):
|
||||
graph = shapes.hires_fix()
|
||||
graph["20"]["inputs"]["latent_image"] = ["10", 0]
|
||||
result = _inspect(graph, output_node_id="22")
|
||||
self.assertEqual((result["state"], result["sampler_id"]), ("resolved", "20"))
|
||||
|
||||
def test_an_unreadable_earlier_stage_blocks_the_chain(self):
|
||||
graph = shapes.hires_fix(second_positive=shapes.POSITIVE)
|
||||
graph["10"]["inputs"]["positive"] = ["6", 0]
|
||||
graph["6"] = {"class_type": "MysteryPrompt", "inputs": {"seed": 1}}
|
||||
result = _inspect(graph, output_node_id="22")
|
||||
self.assertEqual(result["state"], "unresolved")
|
||||
self.assertEqual(result["reason_code"], "unsupported-node")
|
||||
self.assertEqual(result["node"]["class_type"], "MysteryPrompt")
|
||||
|
||||
def test_generated_text_stays_untraceable(self):
|
||||
result = _inspect(shapes.generated_text_chain())
|
||||
self.assertEqual(result["state"], "unresolved")
|
||||
self.assertEqual(result["reason_code"], "unsupported-node")
|
||||
self.assertEqual(result["node"]["class_type"], "TextGenerate")
|
||||
|
||||
def test_ingestion_is_unchanged(self):
|
||||
# D1: the default profile keeps its historical latent rule.
|
||||
result = resolve_prompt_pair({"prompt": shapes.hires_fix()})
|
||||
self.assertEqual(result["state"], "ambiguous")
|
||||
self.assertEqual(
|
||||
{item["reason"] for item in result["candidates"]},
|
||||
{"Unsupported latent ancestry"},
|
||||
)
|
||||
|
||||
|
||||
class TestFluxThreeStage(unittest.TestCase):
|
||||
def test_added_style_conditioning_keeps_the_base_pair(self):
|
||||
graph = shapes.flux_three_stage()
|
||||
for anchor in (None, "61"):
|
||||
with self.subTest(anchor=anchor):
|
||||
result = _inspect(graph, output_node_id=anchor)
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
self.assertEqual(_texts(result), (shapes.POSITIVE, ""))
|
||||
self.assertEqual(result["reason"], resolver.ADDED_CONDITIONING_REASON)
|
||||
self.assertEqual(result["sampler_id"], "50")
|
||||
# D126: the positive says a later stage added conditioning.
|
||||
self.assertEqual(
|
||||
result["positive"]["provenance"]["text_source"],
|
||||
resolver.ADDED_CONDITIONING_SOURCE,
|
||||
)
|
||||
self.assertEqual(
|
||||
result["negative"]["provenance"]["text_source"],
|
||||
"zeroed-conditioning",
|
||||
)
|
||||
|
||||
def test_any_added_conditioning_is_labelled_never_measured(self):
|
||||
# Ruling I2: a whole other prompt combined in is labelled the same way;
|
||||
# no length or "short trigger" heuristic.
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["31"] = shapes.encode(shapes.OTHER)
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
self.assertEqual(_texts(result)[0], shapes.POSITIVE)
|
||||
self.assertEqual(
|
||||
result["positive"]["provenance"]["text_source"],
|
||||
resolver.ADDED_CONDITIONING_SOURCE,
|
||||
)
|
||||
|
||||
def test_trigger_words_and_added_conditioning_carry_both_labels(self):
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["7"] = {
|
||||
"class_type": "Lora Loader (LoraManager)",
|
||||
"inputs": {"model": ["1", 0], "clip": ["2", 0], "text": "x"},
|
||||
}
|
||||
graph["8"] = shapes.trigger_toggle()
|
||||
graph["30"]["inputs"]["prepend_text"] = ["8", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
self.assertEqual(
|
||||
result["positive"]["provenance"]["text_source"],
|
||||
resolver.RUNTIME_AND_ADDED_SOURCE,
|
||||
)
|
||||
|
||||
def test_a_custom_advanced_last_stage_keeps_its_failure(self):
|
||||
# Review M2: a guider's combined positive is not floored.
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["60"] = {
|
||||
"class_type": "SamplerCustomAdvanced",
|
||||
"inputs": {"guider": ["63", 0], "latent_image": ["66", 0]},
|
||||
}
|
||||
graph["63"] = {
|
||||
"class_type": "CFGGuider",
|
||||
"inputs": {"positive": ["35", 0], "negative": ["33", 0]},
|
||||
}
|
||||
graph["66"] = {"class_type": "VAEEncode", "inputs": {"pixels": ["51", 0]}}
|
||||
graph["67"] = {"class_type": "VAEDecode", "inputs": {"samples": ["60", 0]}}
|
||||
graph["61"]["inputs"]["images"] = ["67", 0]
|
||||
result = _inspect(graph, output_node_id="61")
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason_code"]), ("unresolved", "composite")
|
||||
)
|
||||
|
||||
def test_concat_also_adds_to_the_base_prompt(self):
|
||||
result = _inspect(shapes.flux_three_stage(composite="ConditioningConcat"))
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
self.assertEqual(_texts(result), (shapes.POSITIVE, ""))
|
||||
|
||||
def test_an_average_blends_the_base_away(self):
|
||||
result = _inspect(shapes.flux_three_stage(composite="ConditioningAverage"))
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason_code"]), ("unresolved", "composite")
|
||||
)
|
||||
|
||||
def test_a_combine_without_the_base_prompt_is_not_the_base_pair(self):
|
||||
graph = shapes.flux_three_stage(combined=(("37", 0), ("31", 0)))
|
||||
graph["37"] = shapes.encode(shapes.OTHER)
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason_code"]), ("unresolved", "composite")
|
||||
)
|
||||
|
||||
def test_a_different_last_negative_is_not_the_base_pair(self):
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["38"] = shapes.encode("watermark")
|
||||
graph["60"]["inputs"]["negative"] = ["38", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason_code"]), ("unresolved", "composite")
|
||||
)
|
||||
|
||||
def test_different_base_stages_are_never_floored(self):
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = shapes.encode(shapes.OTHER)
|
||||
graph["40"]["inputs"]["positive"] = ["39", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertNotEqual(result["state"], "resolved")
|
||||
|
||||
def test_a_rerouted_combine_is_followed(self):
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = {"class_type": "Reroute", "inputs": {"input": ["35", 0]}}
|
||||
graph["60"]["inputs"]["positive"] = ["39", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(result["reason"], resolver.ADDED_CONDITIONING_REASON)
|
||||
|
||||
def test_an_unreadable_last_negative_or_encoder_parts_keep_the_failure(self):
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["38"] = {"class_type": "MysteryNegative", "inputs": {"seed": 1}}
|
||||
graph["60"]["inputs"]["negative"] = ["38", 0]
|
||||
self.assertEqual(_inspect(graph)["reason_code"], "composite")
|
||||
# Differing encoder parts are a composite too, but not an added stage.
|
||||
graph = shapes.flux_three_stage()
|
||||
graph["39"] = {
|
||||
"class_type": "CLIPTextEncodeSDXL",
|
||||
"inputs": {"text_g": shapes.POSITIVE, "text_l": shapes.STYLE},
|
||||
}
|
||||
graph["60"]["inputs"]["positive"] = ["39", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason_code"]), ("unresolved", "composite")
|
||||
)
|
||||
self.assertEqual(result["node"]["class_type"], "CLIPTextEncodeSDXL")
|
||||
|
||||
def test_inspector_shows_the_base_prompt(self):
|
||||
evidence = inspect_prompt_evidence(
|
||||
{"prompt": json.dumps(shapes.flux_three_stage())}
|
||||
)
|
||||
self.assertEqual(evidence["resolution"]["state"], "resolved")
|
||||
self.assertEqual(evidence["prompts"]["positive"]["text"], shapes.POSITIVE)
|
||||
|
||||
|
||||
class TestRunTimePromptManagerParts(unittest.TestCase):
|
||||
"""Maintainer option A: a run-time prepend or append is left out, labelled."""
|
||||
|
||||
def test_trigger_words_prepend_resolves_to_the_prompt_manager_text(self):
|
||||
for anchor in (None, "22"):
|
||||
with self.subTest(anchor=anchor):
|
||||
result = _inspect(
|
||||
shapes.hires_fix_with_trigger_words(), output_node_id=anchor
|
||||
)
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
self.assertEqual(_texts(result), (shapes.POSITIVE, shapes.NEGATIVE))
|
||||
self.assertEqual(result["reason"], resolver.RUNTIME_PARTS_REASON)
|
||||
provenance = result["positive"]["provenance"]
|
||||
self.assertEqual(
|
||||
provenance["text_source"], resolver.RUNTIME_PARTS_SOURCE
|
||||
)
|
||||
self.assertEqual(result["positive"]["storage_text"], shapes.POSITIVE)
|
||||
# The negative is untouched by the positive's run-time part.
|
||||
self.assertEqual(
|
||||
result["negative"]["provenance"]["text_source"], "graph-inferred"
|
||||
)
|
||||
|
||||
def test_append_and_a_single_sampler_are_labelled_too(self):
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["2"]["inputs"]["prepend_text"] = "masterpiece"
|
||||
graph["2"]["inputs"]["append_text"] = ["8", 0]
|
||||
graph["20"]["inputs"]["latent_image"] = ["4", 0]
|
||||
result = _inspect(graph, output_node_id="22")
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
# The literal prepend is still combined exactly as before.
|
||||
self.assertEqual(_texts(result)[0], "masterpiece " + shapes.POSITIVE)
|
||||
self.assertEqual(result["reason"], resolver.RUNTIME_PARTS_REASON)
|
||||
|
||||
def test_the_inspector_shows_the_prompt_and_its_source(self):
|
||||
evidence = inspect_prompt_evidence(
|
||||
{"prompt": json.dumps(shapes.hires_fix_with_trigger_words())}
|
||||
)
|
||||
self.assertEqual(evidence["resolution"]["state"], "resolved")
|
||||
self.assertEqual(
|
||||
evidence["prompts"]["positive"],
|
||||
{
|
||||
"status": "resolved",
|
||||
"text": shapes.POSITIVE,
|
||||
"source": resolver.RUNTIME_PARTS_SOURCE,
|
||||
},
|
||||
)
|
||||
|
||||
def test_literal_and_traceable_parts_are_combined_as_before(self):
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["8"] = {"class_type": "PrimitiveString", "inputs": {"value": "trig9word"}}
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(_texts(result)[0], "trig9word " + shapes.POSITIVE)
|
||||
self.assertEqual(result["reason"], "")
|
||||
self.assertEqual(
|
||||
result["positive"]["provenance"]["text_source"], "graph-inferred"
|
||||
)
|
||||
|
||||
def test_only_a_run_time_feeder_is_left_out(self):
|
||||
# An unknown feeder stays unsupported, naming it.
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["8"]["class_type"] = "MysteryWords"
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(result["reason_code"], "unsupported-node")
|
||||
self.assertEqual(result["node"]["class_type"], "MysteryWords")
|
||||
# A run-time node deeper inside the part makes the whole part run time,
|
||||
# but it is not a direct feeder: never leave out a traced literal.
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["9"] = {
|
||||
"class_type": "StringConcatenate",
|
||||
"inputs": {"string_a": "kept", "string_b": ["8", 0], "delimiter": ""},
|
||||
}
|
||||
graph["2"]["inputs"]["prepend_text"] = ["9", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(result["reason_code"], "runtime-value")
|
||||
|
||||
def test_the_main_text_from_a_run_time_node_is_never_left_out(self):
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["2"]["inputs"]["prepend_text"] = ""
|
||||
graph["2"]["inputs"]["text"] = ["8", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(result["reason_code"], "runtime-value")
|
||||
|
||||
def test_the_label_survives_nodes_between_the_prompt_manager_and_sampler(self):
|
||||
# Review C1: encoders, chained PromptManagers and concatenations.
|
||||
expected = {
|
||||
"text-encode": shapes.POSITIVE,
|
||||
"chained": shapes.POSITIVE,
|
||||
"concat": shapes.POSITIVE + ", extra",
|
||||
}
|
||||
for shape, text in expected.items():
|
||||
with self.subTest(shape=shape):
|
||||
result = _inspect(shapes.trigger_words_through(shape))
|
||||
self.assertEqual(result["state"], "resolved")
|
||||
self.assertEqual(_texts(result)[0], text)
|
||||
self.assertEqual(result["reason"], resolver.RUNTIME_PARTS_REASON)
|
||||
self.assertEqual(
|
||||
result["positive"]["provenance"]["text_source"],
|
||||
resolver.RUNTIME_PARTS_SOURCE,
|
||||
)
|
||||
|
||||
def test_a_partial_prompt_never_equals_a_complete_one(self):
|
||||
# Review C2: never collapse a branch that left out trigger words with
|
||||
# one that did not, nor compare such stages as equal.
|
||||
result = _inspect(shapes.branches_with_and_without_trigger_words())
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason"]),
|
||||
("ambiguous", "Multiple sampler branches"),
|
||||
)
|
||||
graph = shapes.hires_fix_with_trigger_words()
|
||||
graph["9"] = shapes.prompt_manager(shapes.POSITIVE)
|
||||
graph["20"]["inputs"]["positive"] = ["9", 0]
|
||||
result = _inspect(graph, output_node_id="22")
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason_code"]),
|
||||
("ambiguous", "multistage-differs"),
|
||||
)
|
||||
# Two PromptManagers leaving out two different toggles differ too.
|
||||
graph["8b"] = shapes.trigger_toggle("otherword")
|
||||
graph["9"]["inputs"]["prepend_text"] = ["8b", 0]
|
||||
result = _inspect(graph, output_node_id="22")
|
||||
self.assertEqual(result["reason_code"], "multistage-differs")
|
||||
|
||||
def test_only_listed_run_time_classes_are_left_out(self):
|
||||
# Review M1: a JoinStringMulti with a linked list flag is run time
|
||||
# too, but it is not a listed run-time node: never left out.
|
||||
graph = shapes.hires_fix()
|
||||
graph["9"] = {
|
||||
"class_type": "JoinStringMulti",
|
||||
"inputs": {
|
||||
"inputcount": 2,
|
||||
"delimiter": " ",
|
||||
"return_list": ["98", 0],
|
||||
"string_1": "static words",
|
||||
"string_2": "more",
|
||||
},
|
||||
}
|
||||
graph["98"] = {"class_type": "PrimitiveBoolean", "inputs": {"value": False}}
|
||||
graph["2"]["inputs"]["prepend_text"] = ["9", 0]
|
||||
result = _inspect(graph)
|
||||
self.assertEqual(
|
||||
(result["state"], result["reason_code"]), ("unresolved", "runtime-value")
|
||||
)
|
||||
self.assertEqual(result["node"]["class_type"], "JoinStringMulti")
|
||||
|
||||
def test_ingestion_is_unchanged(self):
|
||||
result = resolve_prompt_pair({"prompt": shapes.hires_fix_with_trigger_words()})
|
||||
self.assertEqual(result["state"], "ambiguous")
|
||||
|
||||
|
||||
class TestStillAmbiguous(unittest.TestCase):
|
||||
def test_edit_batch_with_distinct_prompts_stays_ambiguous(self):
|
||||
result = _inspect(shapes.edit_batch(4))
|
||||
self.assertEqual(result["state"], "ambiguous")
|
||||
self.assertEqual(result["reason"], "Multiple sampler branches")
|
||||
self.assertEqual(len({_texts(item)[0] for item in result["candidates"]}), 4)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user