Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9ab4b95fb5 | ||
|
|
48b1292115 | ||
|
|
eb309c5737 | ||
|
|
42c96c9ff5 | ||
|
|
fb9134f081 | ||
|
|
9a6a8a5312 | ||
|
|
16e88342ad | ||
|
|
417a05a2db | ||
|
|
4ab0119580 | ||
|
|
b05b5dbfd9 | ||
|
|
628e000765 | ||
|
|
144e6b1145 | ||
|
|
f9835aa4d6 | ||
|
|
863d717435 | ||
|
|
8252d9d226 |
@@ -501,6 +501,8 @@ To add your key:
|
||||
2. Paste it into the **CivitAI API Key** field in the Integrations settings
|
||||
3. Save and re-import to download any previously skipped images
|
||||
|
||||
The key is sent only over HTTPS to CivitAI's own hosts. CivitAI serves images through a redirect to its storage host; PromptManager follows such a redirect only over HTTPS to a host on CivitAI's own `civitai.com`, `civitai.red` or `civitai.green` domains, at most three hops, and never sends the key with it. When downloads fail, the import's status line says how many; when none could be downloaded, it starts with a warning. The server log has one warning per failed image with the reason (HTTP status, refused redirect, size cap, unreadable image, timeout or network error).
|
||||
|
||||
> **Note:** Re-importing is safe — PromptManager skips LoRAs that were already imported. To do a fresh import, the previous `lora-manager` data is cleared automatically before re-scanning.
|
||||
|
||||
### 🌐 Web Interface Features
|
||||
|
||||
+294
-11
@@ -802,8 +802,8 @@ Recorded shell deviations:
|
||||
`file_size` equal to the file's size in this walk. Both paths go through
|
||||
`normalize_image_path` (absolute, normalized, `os.path.normcase`); a row without
|
||||
`file_path` matches by its `image_path`.
|
||||
- Read again: files linked without a pair (the monitor's fallback and FIFO links, the
|
||||
legacy positive-only import), linked to more than one prompt or pair, not linked, a
|
||||
- Read again: files linked without a pair (the monitor's queue and run-key links,
|
||||
fallback links made before #342, the legacy positive-only import), linked to more than one prompt or pair, not linked, a
|
||||
file whose size changed (one replaced at the same path, e.g. a reused ComfyUI
|
||||
counter name, is read and its conflict lands under Needs review), and a row with no
|
||||
stored size (unknown is not known). The scan now stores each file's size when it
|
||||
@@ -2378,7 +2378,8 @@ paints nothing. Until it answers, the section shows "Loading the overview…"; a
|
||||
shows "The overview could not be loaded. Your saved data has not changed." and Retry,
|
||||
never zeros. The thumbnail row is a separate read of `/images/thumbnails/status` (a
|
||||
walk of every source), so the other values never wait for it: "Checking thumbnails…"
|
||||
until it answers. The health check (`POST /prompt_manager/system/health`) is a `read`
|
||||
until it answers. The Backup status row is likewise its own read of
|
||||
`/prompt_manager/backup/status` (S9). The health check (`POST /prompt_manager/system/health`) is a `read`
|
||||
registry job of kind `health`: one at a time, beside a scan, not cancellable and with
|
||||
no file outcomes. The page tracks the started job in the Activity store (its event
|
||||
stream; the store marks it lost if the server stops listing it), never by polling. A press while one runs (another tab) waits
|
||||
@@ -2399,7 +2400,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged
|
||||
`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
|
||||
reader can keep it byte-identical (`test_health_check_never_writes`).
|
||||
|
||||
Recorded deviations (plan D60–D70, D74, D75, D79–D82 renumbered D92–D102, D106, D107 and D111–D114; D121–D131 assigned):
|
||||
Recorded deviations (plan D60–D75, D79–D82 renumbered D92–D107 and D111–D114; D121–D131 assigned):
|
||||
|
||||
- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
|
||||
unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server
|
||||
@@ -2416,7 +2417,10 @@ Recorded deviations (plan D60–D70, D74, D75, D79–D82 renumbered D92–D102,
|
||||
`needs_review.pending`) with the concept's line; its Review metadata action opens
|
||||
`#/review` (the concept's `go-review`). A count
|
||||
that is not an integer reads "Ambiguous image links" / "Not available yet." (Q6).
|
||||
"Backup status" reads "No retained backup record." until verified backups exist (S8).
|
||||
"Backup status" says what `GET /prompt_manager/backup/status` reports, in Backup &
|
||||
export's own status line (D103, S9): "Checking backups…" until its own read answers,
|
||||
then the session's last download and the newest safety backup, "No backup recorded."
|
||||
or "The backup status could not be read.".
|
||||
The thumbnail row is "Checking thumbnails…" until its own read answers, then "No
|
||||
missing thumbnails", "N missing thumbnails" (the pending count of
|
||||
`/images/thumbnails/status`, followed by " · M outdated" when previews are in an
|
||||
@@ -2615,6 +2619,68 @@ Recorded deviations (plan D60–D70, D74, D75, D79–D82 renumbered D92–D102,
|
||||
is D93's style. `maintenance matches the concept` (both themes and sizes) maps only this
|
||||
row; `maintenance comparison detects mutations` proves it is the only difference and
|
||||
catches a removed row, a renamed operation, a lost danger mark and a swapped token.
|
||||
- **D103** (Backup & export, S9): "Back up" (the concept's "Simulate backup") downloads
|
||||
the verified copy of `GET /prompt_manager/backup` through the api client (the same
|
||||
`downloadDatabaseBackup` as every backup box; a copy whose `X-PromptManager-Verify` is
|
||||
not `ok` toasts "Database backup downloaded, but it did not pass verification:
|
||||
<reason>"), then reads the status again. The `p.small#backup-status` under the
|
||||
actions shows only what `GET /prompt_manager/backup/status` reports: "Downloaded
|
||||
backup · <date, time> · verified." (or "not verified") for this session's last
|
||||
download and "Safety backup · <date, time> · N on disk." for the newest
|
||||
`<db>.backup_*`, else "No backup recorded."; "Checking backups…" while read and "The
|
||||
backup status could not be read." when it failed (never "no backup"). Portable
|
||||
export's "Export JSON" (the concept's "Export sample JSON") downloads the Library
|
||||
document (`/library/export`); a second `.row-actions.spaced` adds "Export all prompts
|
||||
JSON" (`/export`, the legacy admin page's Export of every prompt) and "Export saved pairs JSON"
|
||||
(`/pairs/export`), plain buttons as the concept's; a failed export toasts "The export
|
||||
could not be downloaded. Nothing was saved to this device." "Import / inspect file"
|
||||
opens the shared "Import & inspect" (D15, D55), as Images does.
|
||||
- **D104** (Backup & export, S9, Q4): Recovery policy keeps the concept's "Proposed" chip.
|
||||
"Create a safety backup before maintenance" is the server's `safety_backup`
|
||||
(`maintenance.safety_backup`) policy. Its line says what it does ("Keep a verified copy
|
||||
of the database on disk before maintenance, cleanups and LoRA imports change records. A
|
||||
restore always keeps one."); the concept's "Preview a verified backup before applying
|
||||
operations that change records." describes nothing the server does. It is drafted behind the Save row and saved with
|
||||
`expected_revision` (a configuration changed elsewhere is the conflict status); it is
|
||||
disabled with "Server settings could not be read." when the read failed. "Scheduled
|
||||
database backups" offers Manual only; Daily and Weekly stay listed but disabled
|
||||
(there is no scheduler), the row's line is "Daily and Weekly are not available yet:
|
||||
back up with Back up whenever you choose." (the concept's proposal line is
|
||||
prototype copy) and nothing is drafted for it. The warning note reads "Restores are
|
||||
staged, migrated and validated before replacing live data."
|
||||
- **D105** (Backup & export, S9): "Review restore…" picks a file (no type filter: the
|
||||
safety backups `<db>.backup_<time>` have no `.db` extension; the server validates) and stages it
|
||||
(`POST /prompt_manager/restore/preview`, multipart through `fetchUrl`; the button reads
|
||||
"Reviewing…" meanwhile). "Restore preview" is the shared review: no sample note; the
|
||||
concept's `dl.meta-grid` Snapshot (file name · size · date), Integrity ("Verified" plus
|
||||
the stage's document repairs), Migration ("Current schema" / "Upgraded to the current
|
||||
schema"), Safety backup ("Created before replacing (<label>)": the restore always
|
||||
keeps one; the Snapshot size is whole KB below 1 MB), then "In the backup" and "In the library now" (prompts, image links, saved
|
||||
pairs, runs); a `p.small` "Restore replaces the whole library with this backup and
|
||||
runs alone in Activity. Nothing changes until you restore. This review holds for N
|
||||
minutes."; the D55 backup box ("Downloads a copy of the prompts database before the
|
||||
restore replaces it."); Cancel / "Restore" (the concept's "Simulate restore", still
|
||||
`danger`). Leaving it any way but Restore (Cancel, ×, Escape, another dialog, the page
|
||||
leaving) discards the stage (`DELETE …/preview/<token>`). Restore starts the "Database
|
||||
restore" Activity job (exclusive, not cancellable, indeterminate; its descriptor ends
|
||||
with the server's line) and opens its view; a review the server asks to redo (expired,
|
||||
library changed) stages the same file again. A Restore refused as `busy` (another job
|
||||
runs; the server keeps the token) opens the same review again with "Discard" / "Restore":
|
||||
Restore retries, Discard (or leaving) releases it. A job whose leases did not drain
|
||||
hands its review back (`scope.review_kept`, "…The review was reopened in the page that
|
||||
started the restore: restore it again there, or discard it."): the page that started it
|
||||
opens the review again over the page shown now, with Discard / Restore. Every other path
|
||||
that abandons a token after Restore (a used, expired or replaced review, a lost answer)
|
||||
sends its DELETE, which changes nothing for a token the server dropped or a job took. A
|
||||
third review while two are held elsewhere toasts "Two restores are already being
|
||||
reviewed in another window. Discard one there, or try again when it expires (15 minutes
|
||||
after it was reviewed)." An upload whose answer is lost because the section was left
|
||||
mid-response is not discarded by the page: the server's 15-minute expiry removes it. When a restore the page saw running
|
||||
finishes, the cached Images and Library visits are dropped, the sidebar counts and
|
||||
saved views are read again and the open route is shown again (the Activity job view
|
||||
stays). A refused file toasts "This file can't be restored: <validator reason>.". A
|
||||
review value may wrap anywhere (`#modal-body > dl.meta-grid dd`, CSS under D105), so a
|
||||
long file name keeps the concept's two equal columns at phone width.
|
||||
- **D106** (Integrations, S10, Q8): the send bridge always asks for the ComfyUI window
|
||||
and node, and the nodes always follow ComfyUI's own renderer, so "Default send
|
||||
destination" offers only "Ask each time" and "Host compatibility" only "Follow host
|
||||
@@ -3151,7 +3217,14 @@ replace binds the category, not one parameter per prompt, so SQLite's variable l
|
||||
never caps a collection. Cached CivitAI images are written to a `.pm-*.part` file in the
|
||||
same folder and renamed into place, with the umask's mode as a plain save gives; each import
|
||||
first removes `.pm-*.part` files older than an hour from the image cache and its LoRA
|
||||
folders (`TestStaleDownloadSweep`). The legacy `POST /lora/scan` runs the same stages under its SSE
|
||||
folders (`TestStaleDownloadSweep`). A download starts only at an HTTPS `civitai.com`,
|
||||
`civitai.red` or `civitai.green` host from the fixed `CIVITAI_HOSTS` list (the key goes
|
||||
with that first request only); a redirect is followed only over HTTPS to one of those
|
||||
domains or a subdomain (parsed host name, default port, no credentials), at most three
|
||||
hops, without the key (`TestCivitaiRedirects`, `TestCivitaiRedAndGreenDomains`). Each
|
||||
failed download logs one warning naming the URL without its query and a short reason
|
||||
(HTTP status, refused redirect, hop limit, size cap, decode or save error, timeout or
|
||||
network error), never the key (`TestDownloadFailureWarnings`). The legacy `POST /lora/scan` runs the same stages under its SSE
|
||||
event shape; a page that closes its connection stops staging before its next LoRA, and the
|
||||
scan keeps its records hold until staging has stopped, then ends `cancelled` in Activity
|
||||
(`test_a_disconnected_legacy_scan_stops_and_holds_until_it_stopped`) (`test_legacy_scan_events_unchanged_on_success`), holds
|
||||
@@ -3279,14 +3352,14 @@ generation did not change meanwhile (`PromptModel.get_connection`).
|
||||
Writes racing the switch: a background write that was waiting on the old file's lock
|
||||
when the switch closed its connection fails, and run counting (`count_runs`,
|
||||
`count_pair_runs`) and the image monitor's link (`ImageGenerationHandler.
|
||||
link_image_to_prompt`: the resolved-pair ingest and the fallback or recent-prompt link)
|
||||
link_image_to_prompt`: the resolved-pair ingest and the plain prompt link)
|
||||
run it once more through `PromptDatabase.survive_library_switch` when the model's
|
||||
generation changed during the attempt; the retry opens the new library. The monitor
|
||||
links through `PromptDatabase.link_image_to_prompt_or_raise`, which raises database
|
||||
errors (`link_image_to_prompt` keeps its log-and-return-0 contract for other callers);
|
||||
each attempt is one transaction, so a failed attempt rolls back and a committed one
|
||||
does not retry, and the image is linked once. `TestBackgroundWritesSurviveTheSwitch`
|
||||
races a real UPDATE (run counting) and a real fallback image link against the lock and
|
||||
races a real UPDATE (run counting) and a real plain image link against the lock and
|
||||
asserts each lands in the new file only. Any other failure is not retried. Known limit: that
|
||||
blocked writer, and the mover with it, waits out the writer's busy timeout (30 s)
|
||||
before its connection closes: SQLite holds the connection's mutex during the busy
|
||||
@@ -3346,7 +3419,8 @@ the executor. `POST /prompt_manager/restore/apply {token}` rechecks the live fin
|
||||
`plan-changed`, stage dropped) and starts "Database restore": kind `restore`, admission
|
||||
`exclusive` (it runs alone, like Move library and Optimize), not cancellable,
|
||||
indeterminate. Its runner waits for request leases (when they outlast the wait the job
|
||||
fails `busy` and the review is kept under its token), rechecks the fingerprint,
|
||||
fails `busy` and the review is kept under its token, flagged `scope.review_kept` for the
|
||||
page that started it), rechecks the fingerprint,
|
||||
publishes, swaps `self.db`, prunes the pool and logs the safety backup by name, any
|
||||
document repair and the restored counts; a failed page copy keeps the safety backup and
|
||||
says so. Once the copy replaced the library the restore succeeded: pruning, reopening
|
||||
@@ -3375,6 +3449,34 @@ removed. `GET
|
||||
/prompt_manager/backup/status` lists up to ten `<db>.backup_*` files newest first (name,
|
||||
bytes, modified time; no paths) and this session's last verified download, or null.
|
||||
|
||||
System › Backup & export (`system-backups.mjs`, S9) is the concept's "backups" body on
|
||||
these routes (D103–D105 above); Overview's Backup status row reads the same status. The
|
||||
admin page's Backup and Restore are unchanged. `tests/browser/system-backups-concept.spec.mjs`
|
||||
compares the section, the Restore preview and the Import & inspect dialog with the
|
||||
concept (both themes, 1600×1100 and 390×844), mapping only D103–D105 (D15 and D55 for the
|
||||
shared Import dialog); unmapped, exactly those paths differ, and `backup & export
|
||||
comparison detects mutations` renames a label, removes a row, drops a danger mark and
|
||||
swaps tokens. `tests/browser/workspace-system-backups.spec.mjs` covers Back up and its
|
||||
revoked object URL, the status line against the server (none, a safety backup, a failed
|
||||
read), Overview's row, the restore review against the preview (and that it writes
|
||||
nothing), Cancel and Escape discarding the stage, a restore through Activity that
|
||||
reloads the counts and the open Library, a busy refusal reopened and retried, two
|
||||
refusals then a third review, ×, a section and a route leave discarding, a safety-backup
|
||||
file name, an invalid file, the three exports and a
|
||||
failed one, Import / inspect (Rescan while running, Scan for metadata, Inspect), the
|
||||
safety-backup policy with its revision and a conflict, the schedule, a failed settings
|
||||
read and the legacy routes. The fixture (`workspace_system_fixture.py`) writes a safety
|
||||
backup (`/__test__/system/safety-backup`), lists the backups and held stages
|
||||
(`GET /__test__/system/backups`), and its reset drops held reviews, removes the safety
|
||||
backups and forgets the last download.
|
||||
|
||||
```sh
|
||||
node --test tests/js/workspace-system-backups.test.js tests/js/workspace-system-overview.test.js
|
||||
npm run --prefix tools/vendor test:workflows -- workspace-system-backups.spec.mjs \
|
||||
system-backups-concept.spec.mjs \
|
||||
--project chromium --project touch --project firefox --project webkit
|
||||
```
|
||||
|
||||
## Workspace data fields for the concept pages (#249)
|
||||
|
||||
These fields are additive. The legacy `/prompt_manager/images/output` listing (no
|
||||
@@ -3856,10 +3958,191 @@ Viewer, including nested modal focus, mobile layouts and per-role sending.
|
||||
real Classic and Nodes 2.0 widgets and verifies workflow serialization without
|
||||
loading a workflow or queueing generation.
|
||||
|
||||
## Image attribution: certain or unlinked (#342)
|
||||
|
||||
The image monitor and the output scan link an output only to a prompt that
|
||||
provably produced it; anything uncertain stays unlinked and the monitor logs it
|
||||
at info ("No certain prompt for <file>; left unlinked"). There is no recency
|
||||
fallback (`_get_fallback_prompt` is gone) and no unproven tracker entry.
|
||||
|
||||
- **Empty editor.** Prepend and append text are never stored as the prompt
|
||||
(maintainer's product decision; #343 tracks letting a link drive `text`). A
|
||||
PromptManager, Text, Focused, Split or Pair node whose `text` is empty saves,
|
||||
counts and tracks nothing for the positive side, logs at info (no "Could not
|
||||
save or track" warning), and marks its run key empty, so no earlier entry of
|
||||
that key can be used and a batch item keeps its queue place
|
||||
(`PromptTracker.mark_empty`). The queue hook counts nothing for it either. A
|
||||
save or tracking failure of a positive node marks its run key the same way, so
|
||||
an older entry of that key never stands in for the failed run.
|
||||
- **Run keys** (`utils/run_keys.py`): sha256 of a prompt node plus its whole
|
||||
upstream closure (`class_type` and `inputs`) in the API graph. Nodes register
|
||||
tracker entries under it; the monitor computes it from the image's embedded
|
||||
graph. The snapshot taken when the file appears names the key's latest run; a
|
||||
queued entry is taken only by key and from that run (batch items in order, a
|
||||
placeholder leaves its image unlinked, same-key entries of earlier runs ahead
|
||||
of it are stale and dropped, entries of later runs that executed during the
|
||||
monitor's delay wait for their own images). Without a queued entry the
|
||||
snapshot's entry itself is used only when its run id is known
|
||||
(`comfy_execution.utils.get_executing_context`) and that run registered one
|
||||
entry for the key, which covers a cached re-run (sampler seed changed) without
|
||||
applying a batch's last item to every image. When the key's latest entry is a
|
||||
placeholder the image stays unlinked without consulting history (the same
|
||||
rule as a popped placeholder). More than one positive PromptManager node in
|
||||
the image graph, or any positive node without a key: never linked from the
|
||||
tracker. Links are followed exactly as ComfyUI's `is_link` defines them (a
|
||||
`str` node id and an `int` or `float` slot in a two-item list). Equal keys do
|
||||
not always mean equal text: the key can't see `IS_CHANGED`, so an upstream
|
||||
node that forces re-runs gives two runs one key; that is why the run named by
|
||||
the snapshot is required as well. A file event delivered after the next run
|
||||
already re-registered the key is the one remaining exposure (see the ledger).
|
||||
- **Recorded history** (`database/run_attribution.py`): a file is attributed when
|
||||
exactly one recorded run reported its output location (root key, subfolder,
|
||||
name), its mtime lies within that run (dequeue to terminal or callback, +-2 s),
|
||||
and every positive context names the same stored prompt (the live
|
||||
`positive_prompt_id` hint). The monitor tries it last; the scan tries it when the
|
||||
#170 browser-parser floor finds nothing, and also for files with no prompt
|
||||
metadata at all (`--disable-metadata`, savers that embed none); it never
|
||||
creates a prompt. Lookups are batched (`stored_run_prompts`): one
|
||||
parameterized `filename IN (...)` query per chunk of at most 500 names
|
||||
(`LOOKUP_CHUNK`), matched in memory, so a scan batch asks history once per
|
||||
chunk, never once per file. The history schema is unchanged (no extra index),
|
||||
so older builds still open the library.
|
||||
- **After the run ends.** An image with no run keys (no embedded API graph) that
|
||||
nothing identifies waits for its run: when the run's outputs are recorded
|
||||
(`write_execution_outputs_for_finish` notifies `subscribe_outputs_sealed`
|
||||
listeners after commit), each waiting image the run reported gets exactly one
|
||||
history-only re-check on the monitor's worker. At most 1000 images wait, each
|
||||
for at most an hour; shutdown forgets them. The image is registered before the
|
||||
first history check, so a run sealed in between is not missed; a re-check that
|
||||
can't be scheduled yet (the image's own pass still pending, or the pending set
|
||||
full) is kept due and scheduled when a pending image finishes. A keyless image
|
||||
outside ComfyUI's output folder never waits (history can't attribute it).
|
||||
- **Spellings.** History keeps ComfyUI's own spelling of its output folder (the
|
||||
root key). A file is matched through that spelling first, then with both sides
|
||||
resolved (`realpath`), so an output folder reached through a symlink or a
|
||||
Windows junction still attributes the scan's resolved file paths; waiting
|
||||
files are keyed by their resolved path for the same reason. The scan makes
|
||||
its batch's history lookup only when the first file in the batch needs it.
|
||||
- **Blank positive.** A resolved pair whose stored positive is empty (the #342
|
||||
image with its Pair execution record) is `unresolved` for the scan and for
|
||||
`ingest_resolved_pair` (it used to raise in `_save_pair`).
|
||||
- **Parity floor.** Metadata linking, the scan's resolver and parser floor are
|
||||
unchanged for every non-blank result; only guesses are gone.
|
||||
|
||||
Tests: `tests/test_prompt_attribution.py` (nodes, tracker, monitor, run keys),
|
||||
`tests/test_run_attribution.py` (history attribution, monitor and scan),
|
||||
`tests/test_review_image_links_tool.py`, and the real-host
|
||||
`tests/comfyui/image-attribution.spec.mjs` (both renderers): a typed Pair image
|
||||
links, a Pair image whose positive is only linked prepend text from an unknown
|
||||
node stays unlinked with nothing stored, an unknown node feeding `text` links
|
||||
through its run key, and its cached re-run with a new sampler seed links again
|
||||
(the host's `execution_cached` event names the PromptManager node, under the
|
||||
fixture's Classic cache). A second case saves with metadata disabled and holds
|
||||
the run open behind the fixture gate after SaveImage: the output is waiting and
|
||||
unlinked while the run is open, and links to the stored prompt once the run's
|
||||
outputs are recorded. The managed host config disables monitoring, so these
|
||||
cases start and stop the plugin's monitor through the owned fixture route
|
||||
`/pm_execution_fixture/monitor` (its GET lists the waiting files).
|
||||
On origin/main the prepend image was linked to an unrelated prompt.
|
||||
|
||||
Repair: `tools/review_image_links.py <db> --output-dir <ComfyUI output> --ids
|
||||
<ids>` (or `--prompt <id>`) judges links by these rules (`keep`, `relink`,
|
||||
`unlink`, `conflict`, `missing`) on a backup copy and changes nothing.
|
||||
`--apply` writes `relink`/`unlink` only for ids named with `--ids`, after saving
|
||||
`<db>.pre-relink_<stamp>`, and only while each image is still linked to the
|
||||
prompt it was judged on (otherwise it is counted as skipped). `relink` points the
|
||||
link record at the other prompt and drops its pair links. `unlink` removes only
|
||||
the image's link record (its `generated_images` row, the metadata snapshot stored
|
||||
in it and its pair links): that is how the app represents an unlinked image, the
|
||||
same as a file the scan could not attribute, so the file stays on disk and in the
|
||||
gallery as unlinked, and its favorite and tags (kept by file) are untouched. Every
|
||||
run prints this. Queue-linked batch images can't be proven after the fact and read
|
||||
`unlink`, so review before applying. Stop ComfyUI first.
|
||||
## Linked prompt text (#343)
|
||||
|
||||
A link drives `text` on every PromptManager node, and `negative_text` on the Pair,
|
||||
as it drives any core widget. The Focused, Split and Pair editors hide their widgets,
|
||||
and the frontend only draws a widget's socket while the widget is visible, so
|
||||
`web/js/text-links.js` detaches those two inputs from their hidden widgets. They become
|
||||
ordinary sockets; the schema, widget order and saved `widgets_values` do not change. A
|
||||
link dropped on a prompt box (or its link note) fills that box's input: Nodes 2.0 resolves
|
||||
it through the box's `.lg-node-widget` and `[data-slot-key]`, Classic through the node's
|
||||
`getInputOnPos`. An exact STRING link dropped elsewhere on the editor fills the main prompt
|
||||
first, then the negative, before the frontend's own choice (it used to pick `prepend_text`,
|
||||
which caused #342). A link being moved off a text input counts as free, as the frontend
|
||||
counts it, so dropping it back on the editor keeps it on `text`. Wildcard links (a
|
||||
frontend Primitive before it is typed) are not steered.
|
||||
|
||||
The frontend's legacy Primitive is virtual: it writes its value into the target widget,
|
||||
which the editor text no longer is. For the editors' detached inputs the `graphToPrompt`
|
||||
wrapper puts the Primitive's value (through virtual reroutes, also inside subgraphs, where
|
||||
the queued id is `<subgraph node>:<inner node>`) into the queued `text`/`negative_text`,
|
||||
resolved as the host resolves a value written into the widget: the Primitive's text
|
||||
replacement when its "Run widget replace on values" option is on, then the hidden widget's
|
||||
own serializer (dynamic prompts: `{red|blue}` becomes one choice). Each queued output is
|
||||
filled once (the queue hook only fills an output that did not come through the wrapper),
|
||||
so a wildcard is drawn once per run. Inputs still bound to their widget (the Classic
|
||||
nodes) are left as the host resolved them; as on main, the Classic text widget's own
|
||||
serializer queues its text as written, wildcards included (specs wait on the node's
|
||||
`_pmTextSerializerReady` before queuing). A Primitive drives the text as before, also in
|
||||
workflows saved before #343. On every Run the host logs "Invalid widget or widget name"
|
||||
for a Primitive wired to a detached editor input: its own widget write finds no widget
|
||||
binding there. That is expected and harmless. A linked
|
||||
Classic text box gives its canvas row back while linked (the core keeps an empty gap) and
|
||||
gets its size back on disconnect. The metadata parsers map a known node's positional
|
||||
widget values through its widget table when its widget-bound inputs do not line up with
|
||||
it (the editors' text inputs are sockets now).
|
||||
|
||||
Not covered, by ruling: linking a subgraph input to an inner editor's `text` gives the
|
||||
subgraph a plain STRING socket, not a promoted typeable widget as a core widget input
|
||||
would. The socket was unreachable before #343, so nothing regressed.
|
||||
While linked, each editor says the value comes from a link, and the typed text is kept
|
||||
in the workflow for when the link is disconnected. That includes the Classic nodes,
|
||||
which used to save `null` under a link.
|
||||
|
||||
- `tests/js/text-links.test.js`: socket exposure, drop preference (exact STRING, moved
|
||||
links), box hit-test, Primitive values through reroutes, replacement and the widget
|
||||
serializer (bound inputs left to the host), stored values, the Classic row.
|
||||
- `tests/test_generation_params.py` (`workflow-only-detached-*` cases, with JS parity
|
||||
vectors): editor text inputs without widget bindings keep their positional values, and so
|
||||
does a core KSampler whose only widget-bound input (`cfg`) is out of schema order.
|
||||
- `tests/browser/{focused,pair,classic}-node.spec.mjs`: the linked note, typed text
|
||||
kept through save and configure (also with a workflow's widget binding restored), the
|
||||
Classic "use prompt" guard while linked.
|
||||
- `tests/comfyui/text-link.spec.mjs` (both startup projects): a real mouse drag of a
|
||||
STRING output onto each node's prompt box (and onto the Pair's negative socket); the
|
||||
link, the note and the typed text survive renderer switches both ways, a reload,
|
||||
disconnect and node removal. `fixtures/workflow-before-343.json` and
|
||||
`workflow-before-343-primitive.json` were serialized by the pre-#343 code on the pinned
|
||||
host and must load unchanged; the second one's legacy Primitives keep driving the text
|
||||
through a real Run. Also: a Primitive dropped on the Main prompt box and a string on the
|
||||
Negative prompt box, a moved link released on the editor body, paste, clone and undo, a
|
||||
legacy Primitive inside a subgraph, and wildcard Primitives on every node type queued
|
||||
through a real Run as one choice, the same one another `graphToPrompt` wrapper sees.
|
||||
- `tests/comfyui/execution-text-link.spec.mjs` (execution host): workflows built by the
|
||||
real frontend through the sockets, with text from `PrimitiveString`, the legacy
|
||||
Primitive or through `PreviewAny`, for PromptManager, PromptManagerText, Focused, Split
|
||||
and the Pair (positive, negative, both, neither linked, and a Primitive negative). A
|
||||
wildcard Primitive on PromptManager (queued as written) and Focused (one choice) saves
|
||||
and counts what was queued. The linked text is saved, counted once, recorded as
|
||||
the run's raw positive/negative, and after a scan the run's image links to it. Text
|
||||
only known at run time (`PreviewAny`) cannot be read from the image; the case only
|
||||
requires that it is never linked to another prompt (its attribution from the recorded
|
||||
run belongs to #342).
|
||||
|
||||
```sh
|
||||
node --test tests/js/text-links.test.js
|
||||
npm run --prefix tools/vendor test:workflows -- classic-node.spec.mjs focused-node.spec.mjs pair-node.spec.mjs \
|
||||
--project chromium --project touch --project firefox --project webkit
|
||||
PM_COMFY_ROOT=... PM_TEST_PYTHON=... npm run --prefix tools/vendor test:comfyui -- text-link
|
||||
PM_COMFY_ROOT=... PM_TEST_PYTHON=... npm run --prefix tools/vendor test:executions -- execution-text-link
|
||||
```
|
||||
|
||||
## Durable execution ledger
|
||||
|
||||
`npm run --prefix tools/vendor test:executions` runs 46 real execution cases
|
||||
(23 each in Classic and Nodes 2.0, retaining all 42 prior execution cases),
|
||||
`npm run --prefix tools/vendor test:executions` runs 52 real execution cases
|
||||
(26 each in Classic and Nodes 2.0, retaining all 48 prior execution cases and adding
|
||||
the two #342 image-attribution cases),
|
||||
alongside the existing 80 stock host cases (the original 78 plus the native
|
||||
editor host case) and six pack compatibility cases. Run
|
||||
these suites serially because they share report cleanup and owned host resources. Use the same pinned disposable `PM_COMFY_ROOT`,
|
||||
|
||||
@@ -4,6 +4,8 @@ import sqlite3
|
||||
from contextlib import ExitStack
|
||||
from typing import Callable, ContextManager
|
||||
|
||||
from .run_attribution import notify_outputs_sealed
|
||||
|
||||
try:
|
||||
from ..utils.execution_history_events import RunFinish, encode_command
|
||||
from ..utils.execution_outputs import OutputBundle, validate_output_bundle
|
||||
@@ -113,6 +115,10 @@ class ExecutionOutputOperations:
|
||||
"VALUES(?,?,?,?,?,?,?,?)",
|
||||
[(run_id, *reference) for reference in prepared],
|
||||
)
|
||||
# Committed: images waiting for this run may now be attributed (#342)
|
||||
notify_outputs_sealed(
|
||||
run_id, [dict(zip(_REFERENCE_FIELDS, row)) for row in prepared]
|
||||
)
|
||||
return True
|
||||
|
||||
def get_execution_outputs(self, run_id: str) -> dict:
|
||||
|
||||
@@ -23,10 +23,26 @@ except ImportError:
|
||||
from utils.prompt_pair_resolver import RESOLVER_VERSION
|
||||
|
||||
|
||||
def blank_positive(resolution: Dict[str, Any]) -> bool:
|
||||
"""A resolved pair whose stored positive text is empty (#342).
|
||||
|
||||
A PromptManager node whose editor is empty encodes only its prepend and
|
||||
append text, which is never stored as a prompt: such a result names no
|
||||
prompt to link.
|
||||
"""
|
||||
if resolution.get("state") != "resolved":
|
||||
return False
|
||||
positive = resolution.get("positive") or {}
|
||||
text = positive.get("storage_text", positive.get("text"))
|
||||
return not (isinstance(text, str) and text.strip())
|
||||
|
||||
|
||||
def ingest_resolved_pair(
|
||||
db: Any, image_path: str, metadata: Dict[str, Any], resolution: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
state = resolution.get("state", "unresolved")
|
||||
if state == "resolved" and blank_positive(resolution):
|
||||
state = "unresolved"
|
||||
if state != "resolved":
|
||||
return {"status": state, "pairs_added": 0, "links_added": 0}
|
||||
with db._pair_transaction() as conn:
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
"""Attribute an output file to a stored prompt through recorded history (#342).
|
||||
|
||||
Execution history records, per run, the files ComfyUI reported
|
||||
(``execution_run_outputs``) and the PromptManager texts the run used
|
||||
(``execution_run_contexts``, whose ``positive_prompt_id`` hint is set only
|
||||
while the raw text matches exactly one stored prompt). A file is attributed
|
||||
only when all of this holds:
|
||||
|
||||
- exactly one recorded run reported the file's output location (same output
|
||||
root, subfolder and name);
|
||||
- the file was last written while that run was executing, from its dequeue
|
||||
to its terminal event (else the finish callback): a later run that history
|
||||
did not record may have reused the name;
|
||||
- every positive context of that run names the same stored prompt.
|
||||
|
||||
Anything else returns None: the caller leaves the file unlinked. Read-only.
|
||||
|
||||
Lookups are batched (``stored_run_prompts``): one parameterized query per
|
||||
chunk of at most ``LOOKUP_CHUNK`` distinct file names, matched in memory. The
|
||||
history schema is left exactly as it is (no extra index), so older builds can
|
||||
still open the library.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import sqlite3
|
||||
import threading
|
||||
import weakref
|
||||
from typing import Any, Callable, Dict, Iterable, List, Optional
|
||||
|
||||
try:
|
||||
from ..utils.execution_outputs import snapshot_output_roots
|
||||
from ..utils.library_location import import_folder_paths
|
||||
except ImportError:
|
||||
from utils.execution_outputs import snapshot_output_roots
|
||||
from utils.library_location import import_folder_paths
|
||||
|
||||
# File times may trail or lead the recorded run times a little (coarse file
|
||||
# system clocks, the write finishing just after the host reported it).
|
||||
MTIME_SLACK_MS = 2000
|
||||
|
||||
# Distinct file names per lookup query: well under SQLite's bound-parameter
|
||||
# limit (999 on old builds), so a scan batch costs one query per chunk.
|
||||
LOOKUP_CHUNK = 500
|
||||
|
||||
_RUNS_FOR_NAMES = (
|
||||
"SELECT DISTINCT o.filename, o.subfolder, o.root_key, o.path_style, o.run_id, "
|
||||
"r.observed_dequeue_at, COALESCE(r.terminal_event_at, r.observed_callback_at) "
|
||||
"FROM execution_run_outputs o JOIN execution_runs r ON r.run_id = o.run_id "
|
||||
"WHERE o.folder_type = 'output' AND o.filename IN ({marks})"
|
||||
)
|
||||
|
||||
|
||||
def host_output_directory() -> Optional[str]:
|
||||
"""ComfyUI's output directory, or None outside ComfyUI."""
|
||||
folder_paths = import_folder_paths()
|
||||
if folder_paths is None:
|
||||
return None
|
||||
try:
|
||||
value = folder_paths.get_output_directory()
|
||||
except Exception:
|
||||
return None
|
||||
return value if isinstance(value, str) and value else None
|
||||
|
||||
|
||||
def _inside(image_path: str, output_root: str) -> Optional[str]:
|
||||
"""``image_path`` relative to ``output_root``, or None when outside it."""
|
||||
try:
|
||||
relative = os.path.relpath(image_path, output_root)
|
||||
except ValueError: # another drive on Windows
|
||||
return None
|
||||
if os.path.isabs(relative) or relative == os.pardir:
|
||||
return None
|
||||
if relative.startswith(os.pardir + os.sep):
|
||||
return None
|
||||
return relative
|
||||
|
||||
|
||||
def _resolved(path: str) -> str:
|
||||
try:
|
||||
return os.path.realpath(path)
|
||||
except (OSError, ValueError):
|
||||
return os.path.abspath(path)
|
||||
|
||||
|
||||
def _location(image_path: str, output_root: str, path_style: str):
|
||||
"""(root key, subfolder, filename) as history records them, or None.
|
||||
|
||||
The root key is always the host's own spelling of its output folder,
|
||||
because history recorded that one. The file may be spelled through the
|
||||
same folder or through its resolved path (a symlinked or junction output
|
||||
folder, as scan roots are): the lexical spelling is tried first, then
|
||||
both sides resolved.
|
||||
"""
|
||||
root_key = snapshot_output_roots(path_style, output_root, output_root).output_key
|
||||
if root_key is None:
|
||||
return None
|
||||
relative = _inside(os.path.abspath(image_path), os.path.abspath(output_root))
|
||||
if relative is None:
|
||||
relative = _inside(_resolved(image_path), _resolved(output_root))
|
||||
if relative is None:
|
||||
return None
|
||||
subfolder, filename = os.path.split(relative)
|
||||
if not filename:
|
||||
return None
|
||||
return root_key, subfolder, filename
|
||||
|
||||
|
||||
def in_output_folder(image_path: str, output_root: Optional[str]) -> bool:
|
||||
"""Whether history could ever attribute ``image_path`` (inside the host's
|
||||
output folder, in either spelling)."""
|
||||
if not image_path or not output_root:
|
||||
return False
|
||||
style = "windows" if os.name == "nt" else "posix"
|
||||
return _location(image_path, output_root, style) is not None
|
||||
|
||||
|
||||
def _written_during(image_path: str, started: Any, finished: Any) -> bool:
|
||||
if not isinstance(started, int) or not isinstance(finished, int):
|
||||
return False
|
||||
try:
|
||||
written = os.stat(image_path).st_mtime_ns // 1_000_000
|
||||
except OSError:
|
||||
return False
|
||||
return started - MTIME_SLACK_MS <= written <= finished + MTIME_SLACK_MS
|
||||
|
||||
|
||||
def stored_run_prompt(
|
||||
db: Any,
|
||||
image_path: str,
|
||||
output_root: Optional[str],
|
||||
path_style: Optional[str] = None,
|
||||
) -> Optional[int]:
|
||||
"""The stored prompt id history attributes to ``image_path``, else None."""
|
||||
if not image_path:
|
||||
return None
|
||||
return stored_run_prompts(db, [image_path], output_root, path_style).get(image_path)
|
||||
|
||||
|
||||
def _runs_by_location(db: Any, names: List[str]) -> Optional[Dict[tuple, dict]]:
|
||||
"""``{(filename, subfolder, root_key, style): {run_id: (start, end)}}``.
|
||||
|
||||
One query per ``LOOKUP_CHUNK`` names; None without history tables.
|
||||
"""
|
||||
found: Dict[tuple, dict] = {}
|
||||
conn = db.model.get_connection()
|
||||
for start in range(0, len(names), LOOKUP_CHUNK):
|
||||
chunk = names[start : start + LOOKUP_CHUNK]
|
||||
query = _RUNS_FOR_NAMES.format(marks=",".join("?" for _ in chunk))
|
||||
try:
|
||||
rows = conn.execute(query, chunk).fetchall() # nosec B608
|
||||
except sqlite3.Error:
|
||||
return None # no history tables (a library without history)
|
||||
for name, subfolder, root_key, style, run_id, began, ended in rows:
|
||||
runs = found.setdefault((name, subfolder, root_key, style), {})
|
||||
runs[run_id] = (began, ended)
|
||||
return found
|
||||
|
||||
|
||||
def _named_prompt(db: Any, run_id: str) -> Optional[int]:
|
||||
contexts = db.get_execution_contexts(run_id).get("contexts") or []
|
||||
named = {
|
||||
context.get("positive_prompt_id")
|
||||
for context in contexts
|
||||
if context.get("positive") is not None
|
||||
}
|
||||
if len(named) != 1:
|
||||
return None
|
||||
(prompt_id,) = named
|
||||
return prompt_id if isinstance(prompt_id, int) else None
|
||||
|
||||
|
||||
def stored_run_prompts(
|
||||
db: Any,
|
||||
image_paths: Iterable[str],
|
||||
output_root: Optional[str],
|
||||
path_style: Optional[str] = None,
|
||||
) -> Dict[str, Optional[int]]:
|
||||
"""``{path: stored prompt id or None}`` for many files in few queries.
|
||||
|
||||
The output lookup is one query per ``LOOKUP_CHUNK`` distinct names; each
|
||||
run that alone reported a file is then read once for its contexts.
|
||||
"""
|
||||
paths = list(dict.fromkeys(path for path in image_paths if path))
|
||||
result: Dict[str, Optional[int]] = dict.fromkeys(paths)
|
||||
if not paths or not output_root:
|
||||
return result
|
||||
style = path_style or ("windows" if os.name == "nt" else "posix")
|
||||
located = {}
|
||||
for path in paths:
|
||||
location = _location(path, output_root, style)
|
||||
if location is not None:
|
||||
located[path] = location
|
||||
if not located:
|
||||
return result
|
||||
names = sorted({filename for _key, _sub, filename in located.values()})
|
||||
runs = _runs_by_location(db, names)
|
||||
if not runs:
|
||||
return result
|
||||
prompts: Dict[str, Optional[int]] = {}
|
||||
for path, (root_key, subfolder, filename) in located.items():
|
||||
reported = runs.get((filename, subfolder, root_key, style), {})
|
||||
if len(reported) != 1:
|
||||
continue
|
||||
((run_id, (began, ended)),) = reported.items()
|
||||
if not _written_during(path, began, ended):
|
||||
continue
|
||||
if run_id not in prompts:
|
||||
prompts[run_id] = _named_prompt(db, run_id)
|
||||
result[path] = prompts[run_id]
|
||||
return result
|
||||
|
||||
|
||||
_LOGGER = logging.getLogger("prompt_manager.run_attribution")
|
||||
_SEALED_LISTENERS: List[weakref.WeakMethod] = []
|
||||
_SEALED_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def subscribe_outputs_sealed(method: Callable[[str, List[dict]], None]) -> None:
|
||||
"""Call bound ``method(run_id, references)`` after a run's outputs are recorded.
|
||||
|
||||
Held weakly: a listener that is garbage collected is dropped.
|
||||
"""
|
||||
with _SEALED_LOCK:
|
||||
_SEALED_LISTENERS.append(weakref.WeakMethod(method))
|
||||
|
||||
|
||||
def notify_outputs_sealed(run_id: str, references: Iterable[dict]) -> None:
|
||||
"""Tell listeners a run's output references were just committed.
|
||||
|
||||
A listener failure is logged and never reaches the history writer.
|
||||
"""
|
||||
references = list(references)
|
||||
with _SEALED_LOCK:
|
||||
alive = [ref for ref in _SEALED_LISTENERS if ref() is not None]
|
||||
_SEALED_LISTENERS[:] = alive
|
||||
listeners = [ref() for ref in alive]
|
||||
for listener in listeners:
|
||||
if listener is None:
|
||||
continue
|
||||
try:
|
||||
listener(run_id, references)
|
||||
except Exception:
|
||||
_LOGGER.warning("Output seal listener failed", exc_info=True)
|
||||
@@ -238,6 +238,9 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
except Exception as e:
|
||||
# Log error but don't fail the encoding
|
||||
self.logger.warning(f"Failed to save prompt to database: {e}")
|
||||
self._mark_untracked(prompt, unique_id)
|
||||
else:
|
||||
self._skip_empty_prompt(prompt, unique_id)
|
||||
|
||||
# Perform standard CLIP text encoding using the combined text
|
||||
self.logger.debug(
|
||||
|
||||
+56
-1
@@ -24,6 +24,7 @@ try:
|
||||
from .utils.prompt_tracker import get_prompt_tracker
|
||||
from .utils.hashing import generate_prompt_hash
|
||||
from .utils.prompt_graph import resolve_text, run_prompt_nodes
|
||||
from .utils.run_keys import executing_run_id, prompt_node_key
|
||||
from .utils.usage_tracking import PENDING_FIRST_USE
|
||||
except ImportError:
|
||||
import sys
|
||||
@@ -35,6 +36,7 @@ except ImportError:
|
||||
from utils.prompt_tracker import get_prompt_tracker
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.prompt_graph import resolve_text, run_prompt_nodes
|
||||
from utils.run_keys import executing_run_id, prompt_node_key
|
||||
from utils.usage_tracking import PENDING_FIRST_USE
|
||||
|
||||
|
||||
@@ -168,14 +170,67 @@ class PromptManagerBase:
|
||||
prompt_id = self._save_prompt_to_database(
|
||||
text=text, category=category, tags=tags, count_run=batch_item
|
||||
)
|
||||
if prompt_id and role["is_run_prompt"]:
|
||||
if role["is_run_prompt"] and not prompt_id:
|
||||
# The save failed: an older entry of this key must not stand in (#342)
|
||||
self._mark_untracked(prompt_graph, unique_id)
|
||||
elif role["is_run_prompt"]:
|
||||
self.prompt_tracker.set_current_prompt(
|
||||
prompt_text=encoding_text,
|
||||
additional_data={**additional_data, "prompt_id": prompt_id},
|
||||
push_to_queue=batch_item or not role["known"],
|
||||
**self._run_identity(prompt_graph, unique_id),
|
||||
)
|
||||
return prompt_id
|
||||
|
||||
@staticmethod
|
||||
def _run_identity(prompt_graph: Any, unique_id: Any) -> Dict[str, Any]:
|
||||
"""The tracker's proof that an image belongs to this execution (#342)."""
|
||||
return {
|
||||
"run_key": prompt_node_key(prompt_graph, unique_id),
|
||||
"run_id": executing_run_id(),
|
||||
}
|
||||
|
||||
def _skip_empty_prompt(self, prompt_graph: Any, unique_id: Any) -> None:
|
||||
"""The ``text`` editor is empty: save and track nothing (#342).
|
||||
|
||||
Prepend and append text are deliberately never stored as the prompt, so
|
||||
an empty editor has no prompt to save, count or link. For the run's
|
||||
positive node this is logged at info and the node's run key is marked
|
||||
empty (``_mark_untracked``).
|
||||
"""
|
||||
self._mark_untracked(
|
||||
prompt_graph,
|
||||
unique_id,
|
||||
"Prompt text is empty (prepend and append text are not stored): "
|
||||
"nothing saved or tracked, and this run's images stay unlinked",
|
||||
)
|
||||
|
||||
def _mark_untracked(
|
||||
self, prompt_graph: Any, unique_id: Any, message: Optional[str] = None
|
||||
) -> None:
|
||||
"""This execution of a positive node has no prompt to link (#342).
|
||||
|
||||
An empty editor, or a save or tracking failure. The node's run key is
|
||||
marked with a placeholder, so no earlier prompt of that key can be
|
||||
linked to this run's images, and a batch item keeps its queue place.
|
||||
``message`` is logged at info for a positive node.
|
||||
"""
|
||||
try:
|
||||
role = self._prompt_role(prompt_graph, unique_id, "")
|
||||
if not role["is_run_prompt"]:
|
||||
if message:
|
||||
self.logger.debug("Empty text on a non-positive node; not saved")
|
||||
return
|
||||
if message:
|
||||
self.logger.info(message)
|
||||
self.prompt_tracker.mark_empty(
|
||||
push_to_queue=role["known"],
|
||||
**self._run_identity(prompt_graph, unique_id),
|
||||
)
|
||||
except Exception as error:
|
||||
# Tracking must never block generation
|
||||
self.logger.warning(f"Could not mark the untracked prompt: {error}")
|
||||
|
||||
def _record_use(self, prompt_id: int, times: int = 1) -> None:
|
||||
"""Count runs; a tracking failure must never block generation."""
|
||||
try:
|
||||
|
||||
+117
-77
@@ -117,85 +117,29 @@ class PromptManagerPair(PromptManagerFocused):
|
||||
)
|
||||
is_positive = "text" in used_fields
|
||||
pair_used = node_id in intrinsic_pair_nodes(prompt) if known else True
|
||||
try:
|
||||
pair_id = self.db.save_prompt_pair(
|
||||
text,
|
||||
negative_text,
|
||||
category=category.strip() or None,
|
||||
tags=self._parse_tags(tags),
|
||||
)
|
||||
pair = self.db.get_prompt_pair(pair_id)
|
||||
prompt_id = pair["positive_prompt_id"]
|
||||
resolved_positive = (
|
||||
resolve_text_input(prompt, node_id, "text") if known else None
|
||||
)
|
||||
resolved_negative = (
|
||||
resolve_text_input(prompt, node_id, "negative_text") if known else None
|
||||
)
|
||||
positive_seen = resolved_positive is not None and generate_prompt_hash(
|
||||
resolved_positive
|
||||
) == generate_prompt_hash(text)
|
||||
pair_seen = (
|
||||
positive_seen
|
||||
and resolved_negative is not None
|
||||
and generate_prompt_hash(resolved_negative)
|
||||
== generate_prompt_hash(negative_text)
|
||||
)
|
||||
positive_pending = getattr(self, "pending_first_use", PENDING_FIRST_USE)
|
||||
pair_pending = getattr(
|
||||
self, "pending_pair_first_use", PENDING_PAIR_FIRST_USE
|
||||
)
|
||||
counted = set()
|
||||
for field, component_text, resolved_text, component_id in (
|
||||
("text", text, resolved_positive, prompt_id),
|
||||
(
|
||||
"negative_text",
|
||||
if not text.strip():
|
||||
# Nothing to store: prepend/append are never the stored prompt (#342)
|
||||
self._skip_empty_prompt(prompt, unique_id)
|
||||
else:
|
||||
try:
|
||||
pair_id, prompt_id = self._save_and_track_pair(
|
||||
text,
|
||||
negative_text,
|
||||
resolved_negative,
|
||||
pair["negative_prompt_id"],
|
||||
),
|
||||
):
|
||||
component_hash = generate_prompt_hash(component_text)
|
||||
if (
|
||||
field not in used_fields
|
||||
or not component_id
|
||||
or component_hash in counted
|
||||
):
|
||||
continue
|
||||
counted.add(component_hash)
|
||||
seen_at_queue = (
|
||||
resolved_text is not None
|
||||
and generate_prompt_hash(resolved_text) == component_hash
|
||||
positive,
|
||||
category=category,
|
||||
tags=tags,
|
||||
prepend_text=prepend_text,
|
||||
append_text=append_text,
|
||||
prompt=prompt,
|
||||
unique_id=unique_id,
|
||||
used_fields=used_fields,
|
||||
is_positive=is_positive,
|
||||
pair_used=pair_used,
|
||||
)
|
||||
runs = positive_pending.consume(component_hash) + int(not seen_at_queue)
|
||||
if runs:
|
||||
self.db.record_prompt_use(component_id, times=runs)
|
||||
if pair_used:
|
||||
runs = pair_pending.consume(pair_run_key(text, negative_text)) + int(
|
||||
not pair_seen
|
||||
)
|
||||
if runs:
|
||||
self.db.record_prompt_pair_use(pair_id, times=runs)
|
||||
if is_positive:
|
||||
self.prompt_tracker.set_current_prompt(
|
||||
prompt_text=positive,
|
||||
additional_data={
|
||||
"prompt_id": prompt_id,
|
||||
"pair_id": pair_id if pair_used else None,
|
||||
"positive_text": positive,
|
||||
"negative_text": negative_text,
|
||||
"positive_storage_text": text.strip(),
|
||||
"negative_storage_text": negative_text.strip(),
|
||||
"prepend_text": prepend_text.strip(),
|
||||
"append_text": append_text.strip(),
|
||||
"final_text": positive,
|
||||
"category": category.strip() or None,
|
||||
"tags": self._parse_tags(tags) or [],
|
||||
},
|
||||
push_to_queue=not positive_seen or (pair_used and not pair_seen),
|
||||
)
|
||||
except Exception as error:
|
||||
self.logger.warning(f"Could not save or track paired prompt: {error}")
|
||||
except Exception as error:
|
||||
self.logger.warning(f"Could not save or track paired prompt: {error}")
|
||||
if is_positive:
|
||||
self._mark_untracked(prompt, unique_id)
|
||||
try:
|
||||
self.comfyui_integration.register_prompt(
|
||||
f"promptmanager_pair_{time.time_ns()}",
|
||||
@@ -223,6 +167,102 @@ class PromptManagerPair(PromptManagerFocused):
|
||||
)
|
||||
return result
|
||||
|
||||
def _save_and_track_pair(
|
||||
self,
|
||||
text: str,
|
||||
negative_text: str,
|
||||
positive: str,
|
||||
*,
|
||||
category: str,
|
||||
tags: str,
|
||||
prepend_text: str,
|
||||
append_text: str,
|
||||
prompt: Optional[Dict[str, Any]],
|
||||
unique_id: Optional[str],
|
||||
used_fields: set,
|
||||
is_positive: bool,
|
||||
pair_used: bool,
|
||||
) -> Tuple[Optional[int], Optional[int]]:
|
||||
"""Save the recipe, count its runs and register the positive for linking."""
|
||||
known = isinstance(prompt, dict) and unique_id is not None
|
||||
node_id = str(unique_id)
|
||||
pair_id = self.db.save_prompt_pair(
|
||||
text,
|
||||
negative_text,
|
||||
category=category.strip() or None,
|
||||
tags=self._parse_tags(tags),
|
||||
)
|
||||
pair = self.db.get_prompt_pair(pair_id)
|
||||
prompt_id = pair["positive_prompt_id"]
|
||||
resolved_positive = (
|
||||
resolve_text_input(prompt, node_id, "text") if known else None
|
||||
)
|
||||
resolved_negative = (
|
||||
resolve_text_input(prompt, node_id, "negative_text") if known else None
|
||||
)
|
||||
positive_seen = resolved_positive is not None and generate_prompt_hash(
|
||||
resolved_positive
|
||||
) == generate_prompt_hash(text)
|
||||
pair_seen = (
|
||||
positive_seen
|
||||
and resolved_negative is not None
|
||||
and generate_prompt_hash(resolved_negative)
|
||||
== generate_prompt_hash(negative_text)
|
||||
)
|
||||
positive_pending = getattr(self, "pending_first_use", PENDING_FIRST_USE)
|
||||
pair_pending = getattr(self, "pending_pair_first_use", PENDING_PAIR_FIRST_USE)
|
||||
counted = set()
|
||||
for field, component_text, resolved_text, component_id in (
|
||||
("text", text, resolved_positive, prompt_id),
|
||||
(
|
||||
"negative_text",
|
||||
negative_text,
|
||||
resolved_negative,
|
||||
pair["negative_prompt_id"],
|
||||
),
|
||||
):
|
||||
component_hash = generate_prompt_hash(component_text)
|
||||
if (
|
||||
field not in used_fields
|
||||
or not component_id
|
||||
or component_hash in counted
|
||||
):
|
||||
continue
|
||||
counted.add(component_hash)
|
||||
seen_at_queue = (
|
||||
resolved_text is not None
|
||||
and generate_prompt_hash(resolved_text) == component_hash
|
||||
)
|
||||
runs = positive_pending.consume(component_hash) + int(not seen_at_queue)
|
||||
if runs:
|
||||
self.db.record_prompt_use(component_id, times=runs)
|
||||
if pair_used:
|
||||
runs = pair_pending.consume(pair_run_key(text, negative_text)) + int(
|
||||
not pair_seen
|
||||
)
|
||||
if runs:
|
||||
self.db.record_prompt_pair_use(pair_id, times=runs)
|
||||
if is_positive:
|
||||
self.prompt_tracker.set_current_prompt(
|
||||
prompt_text=positive,
|
||||
additional_data={
|
||||
"prompt_id": prompt_id,
|
||||
"pair_id": pair_id if pair_used else None,
|
||||
"positive_text": positive,
|
||||
"negative_text": negative_text,
|
||||
"positive_storage_text": text.strip(),
|
||||
"negative_storage_text": negative_text.strip(),
|
||||
"prepend_text": prepend_text.strip(),
|
||||
"append_text": append_text.strip(),
|
||||
"final_text": positive,
|
||||
"category": category.strip() or None,
|
||||
"tags": self._parse_tags(tags) or [],
|
||||
},
|
||||
push_to_queue=not positive_seen or (pair_used and not pair_seen),
|
||||
**self._run_identity(prompt, unique_id),
|
||||
)
|
||||
return pair_id, prompt_id
|
||||
|
||||
@classmethod
|
||||
def IS_CHANGED(
|
||||
cls,
|
||||
|
||||
@@ -216,6 +216,9 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
except Exception as e:
|
||||
# Log error but don't fail the processing
|
||||
self.logger.warning(f"Failed to save prompt to database: {e}")
|
||||
self._mark_untracked(prompt, unique_id)
|
||||
else:
|
||||
self._skip_empty_prompt(prompt, unique_id)
|
||||
|
||||
# Register with ComfyUI integration for standard metadata compatibility
|
||||
node_id = f"promptmanagertext_{int(time.time() * 1000)}"
|
||||
|
||||
+102
-4
@@ -35,7 +35,12 @@ except ImportError:
|
||||
from utils.parallel import map_parallel
|
||||
|
||||
try:
|
||||
from ...database.pair_ingestion import ingest_resolved_pair
|
||||
from ...database.pair_ingestion import blank_positive, ingest_resolved_pair
|
||||
from ...database.run_attribution import (
|
||||
host_output_directory,
|
||||
stored_run_prompt,
|
||||
stored_run_prompts,
|
||||
)
|
||||
from ...utils.generation_params import (
|
||||
PLACEHOLDERS,
|
||||
GenerationParamsError,
|
||||
@@ -45,7 +50,12 @@ try:
|
||||
from ...utils.prompt_pair_resolver import resolve_prompt_pair
|
||||
from ...utils.review_capture import reviewable
|
||||
except ImportError:
|
||||
from database.pair_ingestion import ingest_resolved_pair
|
||||
from database.pair_ingestion import blank_positive, ingest_resolved_pair
|
||||
from database.run_attribution import (
|
||||
host_output_directory,
|
||||
stored_run_prompt,
|
||||
stored_run_prompts,
|
||||
)
|
||||
from utils.generation_params import (
|
||||
PLACEHOLDERS,
|
||||
GenerationParamsError,
|
||||
@@ -548,6 +558,24 @@ def _current_image_monitor():
|
||||
return getattr(_image_monitor_module(), "_monitor_instance", None)
|
||||
|
||||
|
||||
class _BatchHistory:
|
||||
"""A scan batch's recorded-history lookup, made on the first file needing it.
|
||||
|
||||
Most files link through their metadata and never ask; the first one that
|
||||
does triggers one lookup for the whole batch (``_recorded_runs_for``).
|
||||
"""
|
||||
|
||||
def __init__(self, api, media_files):
|
||||
self._api = api
|
||||
self._files = media_files
|
||||
self._found = None
|
||||
|
||||
def get(self, path):
|
||||
if self._found is None:
|
||||
self._found = self._api._recorded_runs_for(self._files)
|
||||
return self._found.get(path)
|
||||
|
||||
|
||||
class AdminRoutesMixin:
|
||||
"""Mixin providing admin, diagnostics, and maintenance API endpoints."""
|
||||
|
||||
@@ -1564,6 +1592,8 @@ class AdminRoutesMixin:
|
||||
if outcomes is not None:
|
||||
outcomes.append(_scan_outcome(media_file, roots, status, title))
|
||||
|
||||
batch_results = list(batch_results)
|
||||
history = _BatchHistory(self, [media for media, _meta in batch_results])
|
||||
for media_file, metadata in batch_results:
|
||||
if self._scan_job and self._scan_job.cancelled:
|
||||
break
|
||||
@@ -1577,6 +1607,7 @@ class AdminRoutesMixin:
|
||||
ambiguous=ambiguous,
|
||||
untraced=untraced,
|
||||
records=records,
|
||||
history=history,
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error processing {media_file.name}: {e}")
|
||||
@@ -1635,6 +1666,7 @@ class AdminRoutesMixin:
|
||||
ambiguous=None,
|
||||
untraced=None,
|
||||
records=None,
|
||||
history=None,
|
||||
):
|
||||
"""Enrich exact pairs; retain legacy positive-only import for absent pairs.
|
||||
|
||||
@@ -1652,7 +1684,9 @@ class AdminRoutesMixin:
|
||||
a dict.
|
||||
"""
|
||||
if not metadata:
|
||||
return None
|
||||
# No prompt metadata (--disable-metadata, bare savers): only the
|
||||
# recorded run can say which prompt made it (#342)
|
||||
return self._link_recorded_run(media_file, {}, detailed, history)
|
||||
canonical = dict(metadata)
|
||||
for key in ("prompt", "workflow"):
|
||||
if isinstance(canonical.get(key), str):
|
||||
@@ -1662,9 +1696,12 @@ class AdminRoutesMixin:
|
||||
# Preserve malformed metadata for conservative resolver rejection.
|
||||
pass
|
||||
if not canonical.get("prompt") and not canonical.get("parameters"):
|
||||
return None
|
||||
return self._link_recorded_run(media_file, canonical, detailed, history)
|
||||
_with_file_size(canonical, media_file)
|
||||
resolution = resolve_prompt_pair(canonical)
|
||||
if blank_positive(resolution):
|
||||
# Only prepend/append text: no stored prompt can own it (#342)
|
||||
resolution = {**resolution, "state": "unresolved"}
|
||||
# Ambiguous without a choice: unresolved unless something links it.
|
||||
unchoosable = resolution["state"] == "ambiguous" and not reviewable(resolution)
|
||||
if unchoosable:
|
||||
@@ -1693,6 +1730,11 @@ class AdminRoutesMixin:
|
||||
# Graphs without a recognized sampler keep the legacy extractor below.
|
||||
prompt_text = self._graph_prompt_text(metadata)
|
||||
if not prompt_text:
|
||||
linked = self._link_recorded_run(
|
||||
media_file, canonical, detailed, history
|
||||
)
|
||||
if linked is not None:
|
||||
return linked
|
||||
return "unresolved" if unchoosable else resolution["state"]
|
||||
elif resolution["state"] == "ambiguous":
|
||||
return "unresolved" if unchoosable else "ambiguous"
|
||||
@@ -1716,6 +1758,62 @@ class AdminRoutesMixin:
|
||||
return status
|
||||
return {"status": status, "title": self._scan_title(media_file, prompt_text)}
|
||||
|
||||
def _recorded_runs_for(self, media_files):
|
||||
"""``{path: prompt id or None}`` from recorded history for a scan batch.
|
||||
|
||||
One output lookup per chunk of names (``stored_run_prompts``), so a
|
||||
batch never queries history once per file. An empty dict on failure.
|
||||
"""
|
||||
try:
|
||||
return stored_run_prompts(
|
||||
self.db, [str(path) for path in media_files], host_output_directory()
|
||||
)
|
||||
except Exception:
|
||||
self.logger.warning(
|
||||
"History attribution failed for a scan batch", exc_info=True
|
||||
)
|
||||
return {}
|
||||
|
||||
def _link_recorded_run(self, media_file, canonical, detailed=False, history=None):
|
||||
"""Link a file recorded history attributes to one stored prompt (#342).
|
||||
|
||||
Returns the scan status (with ``detailed``, a stored or linked file's
|
||||
``{"status", "title"}``), or None when history names no single stored
|
||||
prompt for the file (``database.run_attribution``). Never creates a
|
||||
prompt. Works without any prompt metadata in the file. ``history`` is
|
||||
the batch's precomputed lookup; without it the file is looked up alone.
|
||||
"""
|
||||
if history is not None:
|
||||
prompt_id = history.get(str(media_file))
|
||||
else:
|
||||
try:
|
||||
prompt_id = stored_run_prompt(
|
||||
self.db, str(media_file), host_output_directory()
|
||||
)
|
||||
except Exception:
|
||||
self.logger.warning(
|
||||
f"History attribution failed for {media_file.name}",
|
||||
exc_info=True,
|
||||
)
|
||||
return None
|
||||
if prompt_id is None:
|
||||
return None
|
||||
try:
|
||||
image_id = self.db.link_image_to_prompt(
|
||||
prompt_id, str(media_file), metadata=canonical
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Failed to link {media_file.name} from history: {e}")
|
||||
return "found"
|
||||
status = "linked" if image_id else "unchanged"
|
||||
if not detailed or status not in SCAN_TITLED_DETAILS:
|
||||
return status
|
||||
prompt = self.db.get_prompt_by_id(prompt_id) or {}
|
||||
return {
|
||||
"status": status,
|
||||
"title": self._scan_title(media_file, prompt.get("text") or ""),
|
||||
}
|
||||
|
||||
def _scan_title(self, media_file, text):
|
||||
"""The linked prompt's Library title (its stored title, else ``text``'s
|
||||
first line), as the Library and the inspector name it."""
|
||||
|
||||
@@ -90,9 +90,13 @@ REQUESTS_BUSY = (
|
||||
"Other requests are still reading the library; nothing was changed. "
|
||||
"Try again in a moment."
|
||||
)
|
||||
# S9 I1: the review goes back to the page that started the restore
|
||||
# (``scope.review_kept``), which opens it again with Restore and Discard; no
|
||||
# other place can start a kept review, so the line says where it is.
|
||||
REVIEW_KEPT = (
|
||||
"Other requests are still reading the library; nothing was changed. "
|
||||
"The review is kept: start the restore again in a moment."
|
||||
"The review was reopened in the page that started the restore: restore it "
|
||||
"again there, or discard it."
|
||||
)
|
||||
VERIFY_HEADER = "X-PromptManager-Verify"
|
||||
SQLITE_HEADER = b"SQLite format 3\x00"
|
||||
@@ -873,6 +877,7 @@ class BackupRestoreRoutesMixin:
|
||||
if token is not None and self._restore_plans.give_back(token, held):
|
||||
keep_stage = True
|
||||
self._schedule_restore_expiry()
|
||||
job.scope["review_kept"] = True
|
||||
return _fail_job(job, "busy", REVIEW_KEPT)
|
||||
return _fail_job(job, "busy", REQUESTS_BUSY)
|
||||
job.update(status_line="Checking the library")
|
||||
|
||||
@@ -100,6 +100,25 @@ def _import_line(result, skipped):
|
||||
return " · ".join(parts)
|
||||
|
||||
|
||||
def _all_downloads_failed(errors):
|
||||
"""The warning when every planned download failed (#337)."""
|
||||
if errors == 1:
|
||||
return "Warning: the example image could not be downloaded"
|
||||
return f"Warning: none of the {errors:,} example images could be downloaded"
|
||||
|
||||
|
||||
def _with_downloads(line, downloaded, errors):
|
||||
"""``line`` with the failed downloads: counted when some failed, led by
|
||||
a warning when every planned download failed (#337). The job still
|
||||
finishes: the import itself succeeded and the downloads are best effort."""
|
||||
if not errors:
|
||||
return line
|
||||
if not downloaded:
|
||||
return f"{_all_downloads_failed(errors)} · {line}"
|
||||
failed = f"{_count(errors, 'example image')} could not be downloaded"
|
||||
return f"{line} · {failed}"
|
||||
|
||||
|
||||
class LoraImportRoutesMixin:
|
||||
"""Mixin providing the reviewed LoRA import (preview, apply, job)."""
|
||||
|
||||
@@ -415,7 +434,12 @@ class LoraImportRoutesMixin:
|
||||
downloaded = staged.stats.get("downloaded", 0)
|
||||
errors = staged.stats.get("download_errors", 0)
|
||||
job.record_counted("downloaded", downloaded)
|
||||
job.record_counted("download_errors", errors)
|
||||
rows = []
|
||||
if errors and not downloaded:
|
||||
warning = _all_downloads_failed(errors)
|
||||
rows = [{"path": "", "detail": f"{warning}; the log has the details"}]
|
||||
job.log_line(warning)
|
||||
job.record_counted("download_errors", errors, rows)
|
||||
if downloaded or errors:
|
||||
job.log_line(
|
||||
f"Downloaded {_count(downloaded, 'example image')}"
|
||||
@@ -451,4 +475,9 @@ class LoraImportRoutesMixin:
|
||||
if result["removed"]:
|
||||
job.log_line(f"Replaced {_count(result['removed'], 'previous import')}")
|
||||
job.log_line(line)
|
||||
line = _with_downloads(
|
||||
line,
|
||||
staged.stats.get("downloaded", 0),
|
||||
staged.stats.get("download_errors", 0),
|
||||
)
|
||||
job.update(progress=100, processed=job.total, status_line=line)
|
||||
|
||||
+168
-31
@@ -8,14 +8,16 @@ return empty results rather than raising.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
from urllib.parse import urlsplit
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
|
||||
try:
|
||||
from ..utils.logging_config import get_logger
|
||||
@@ -419,19 +421,40 @@ def get_preview_image_from_metadata(
|
||||
|
||||
_THUMB_MAX_SIZE = 512
|
||||
|
||||
# CivitAI's own domains (#340: civitai.red and civitai.green serve the same
|
||||
# content as civitai.com).
|
||||
CIVITAI_DOMAINS = ("civitai.com", "civitai.red", "civitai.green")
|
||||
# Only these hosts may ever receive the CivitAI API key, and only over HTTPS.
|
||||
CIVITAI_HOSTS = frozenset(
|
||||
{"civitai.com", "www.civitai.com", "api.civitai.com", "image.civitai.com"}
|
||||
f"{prefix}{domain}"
|
||||
for domain in CIVITAI_DOMAINS
|
||||
for prefix in ("", "www.", "api.", "image.")
|
||||
)
|
||||
MAX_CIVITAI_DOWNLOAD_BYTES = 50 * 1024 * 1024
|
||||
_DOWNLOAD_CHUNK_BYTES = 64 * 1024
|
||||
|
||||
|
||||
# A redirect is followed (#337: image.civitai.com now answers with a redirect
|
||||
# to blobs-b2.civitai.com) only over HTTPS to one of CIVITAI_DOMAINS or a
|
||||
# subdomain of one, at most this many hops, and never with the API key.
|
||||
MAX_CIVITAI_REDIRECTS = 3
|
||||
_REDIRECT_CODES = frozenset({301, 302, 303, 307, 308})
|
||||
_USER_AGENT = "ComfyUI-PromptManager/1.0"
|
||||
|
||||
|
||||
class _DownloadFailed(Exception):
|
||||
"""A download failed for the short, loggable ``reason`` it carries."""
|
||||
|
||||
def __init__(self, reason: str):
|
||||
super().__init__(reason)
|
||||
self.reason = reason
|
||||
|
||||
|
||||
class _RefuseRedirects(urllib.request.HTTPRedirectHandler):
|
||||
"""Never follow a redirect: urllib would re-send the Bearer header to it."""
|
||||
"""Never follow a redirect inside urllib: it would re-send the Bearer
|
||||
header. A 3xx surfaces as ``HTTPError``; ``_download_one`` decides."""
|
||||
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||
logger.warning(f"Refusing redirect from {req.full_url} to {newurl}")
|
||||
return None
|
||||
|
||||
|
||||
@@ -448,6 +471,84 @@ def _open_url(req: urllib.request.Request, timeout: float = 10):
|
||||
return _opener.open(req, timeout=timeout)
|
||||
|
||||
|
||||
def _loggable(url: str) -> str:
|
||||
"""``url`` without its query, fragment or credentials (signed URLs)."""
|
||||
try:
|
||||
parts = urlsplit(url)
|
||||
host = parts.hostname or ""
|
||||
except ValueError:
|
||||
return "<unparsable URL>"
|
||||
return f"{parts.scheme}://{host}{parts.path}"
|
||||
|
||||
|
||||
def _on_civitai_domain(hostname: str) -> bool:
|
||||
"""True when ``hostname`` is one of CIVITAI_DOMAINS or a subdomain of one."""
|
||||
return any(
|
||||
hostname == domain or hostname.endswith(f".{domain}")
|
||||
for domain in CIVITAI_DOMAINS
|
||||
)
|
||||
|
||||
|
||||
def is_civitai_redirect_target(url: str) -> bool:
|
||||
"""True when a redirect to ``url`` may be followed: HTTPS on one of
|
||||
CIVITAI_DOMAINS or a subdomain of one, matched on the parsed host name,
|
||||
default port and no credentials in the URL."""
|
||||
try:
|
||||
parts = urlsplit(url)
|
||||
port = parts.port
|
||||
except ValueError:
|
||||
return False
|
||||
hostname = (parts.hostname or "").lower()
|
||||
return (
|
||||
parts.scheme == "https"
|
||||
and _on_civitai_domain(hostname)
|
||||
and parts.username is None
|
||||
and parts.password is None
|
||||
and port in (None, 443)
|
||||
)
|
||||
|
||||
|
||||
def _redirect_target(url: str, error) -> Optional[str]:
|
||||
"""The URL a 3xx ``error`` for ``url`` points to, or None when absent."""
|
||||
if error.code not in _REDIRECT_CODES:
|
||||
return None
|
||||
location = (error.headers.get("Location") or "").strip()
|
||||
if not location:
|
||||
return None
|
||||
return urljoin(url, location)
|
||||
|
||||
|
||||
def _open_following_civitai_redirects(url: str, api_key: str):
|
||||
"""Open ``url``, following only allowed CivitAI redirects (keyless).
|
||||
|
||||
The first request carries the key; every redirected one carries the
|
||||
User-Agent alone. Returns the open response; raises ``_DownloadFailed``
|
||||
when a redirect is refused or the hop limit is reached. Other HTTP
|
||||
errors propagate.
|
||||
"""
|
||||
headers = {"User-Agent": _USER_AGENT}
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
current = url
|
||||
for hop in range(MAX_CIVITAI_REDIRECTS + 1):
|
||||
req = urllib.request.Request(current, headers=headers)
|
||||
try:
|
||||
return _open_url(req, timeout=10)
|
||||
except urllib.error.HTTPError as error:
|
||||
with error:
|
||||
target = _redirect_target(current, error)
|
||||
if target is None:
|
||||
raise
|
||||
if hop == MAX_CIVITAI_REDIRECTS:
|
||||
raise _DownloadFailed(f"more than {MAX_CIVITAI_REDIRECTS} redirects")
|
||||
if not is_civitai_redirect_target(target):
|
||||
raise _DownloadFailed(f"redirect refused to {_loggable(target)}")
|
||||
logger.debug(f"Following redirect to {_loggable(target)}")
|
||||
current = target
|
||||
headers = {"User-Agent": _USER_AGENT} # never the key across a redirect
|
||||
raise AssertionError("unreachable") # pragma: no cover - loop ends above
|
||||
|
||||
|
||||
def is_civitai_url(url: str) -> bool:
|
||||
"""True when ``url`` is an HTTPS URL on an allow-listed CivitAI host."""
|
||||
try:
|
||||
@@ -488,40 +589,76 @@ def _read_capped(resp, cap: int) -> Optional[bytes]:
|
||||
return b"".join(chunks)
|
||||
|
||||
|
||||
def _network_reason(error: BaseException) -> str:
|
||||
"""A short reason for a failed request: a timeout or a network error."""
|
||||
cause = getattr(error, "reason", error) # URLError wraps the socket error
|
||||
if isinstance(cause, TimeoutError):
|
||||
return "timed out"
|
||||
return f"network error: {cause}"
|
||||
|
||||
|
||||
def _fetch_civitai_body(url: str, api_key: str) -> bytes:
|
||||
"""The body behind ``url`` (redirects as allowed, size capped), or
|
||||
``_DownloadFailed`` with the reason it could not be fetched."""
|
||||
try:
|
||||
with _open_following_civitai_redirects(url, api_key) as resp:
|
||||
raw = _read_capped(resp, MAX_CIVITAI_DOWNLOAD_BYTES)
|
||||
except urllib.error.HTTPError as error:
|
||||
error.close()
|
||||
raise _DownloadFailed(f"HTTP {error.code}") from None
|
||||
except (urllib.error.URLError, OSError, http.client.HTTPException) as error:
|
||||
raise _DownloadFailed(_network_reason(error)) from None
|
||||
if raw is None:
|
||||
cap = MAX_CIVITAI_DOWNLOAD_BYTES
|
||||
size = f"{cap // (1024 * 1024)} MB" if cap >= 1024 * 1024 else f"{cap} byte"
|
||||
raise _DownloadFailed(f"larger than the {size} cap")
|
||||
return raw
|
||||
|
||||
|
||||
def _thumbnail(raw: bytes):
|
||||
"""``raw`` decoded and shrunk to an RGB thumbnail, or ``_DownloadFailed``."""
|
||||
from io import BytesIO
|
||||
|
||||
from PIL import Image
|
||||
|
||||
try:
|
||||
with Image.open(BytesIO(raw)) as img:
|
||||
img.thumbnail((_THUMB_MAX_SIZE, _THUMB_MAX_SIZE), Image.LANCZOS)
|
||||
return img.convert("RGB")
|
||||
except Exception as error: # PIL raises several types for bad data
|
||||
raise _DownloadFailed(
|
||||
f"image could not be decoded ({type(error).__name__})"
|
||||
) from None
|
||||
|
||||
|
||||
def _download_one(url: str, local_path: Path, api_key: str) -> Optional[str]:
|
||||
"""Download a single CivitAI image, resize to thumbnail, save as JPEG.
|
||||
|
||||
Refuses any URL that is not HTTPS on an allow-listed CivitAI host so the
|
||||
API key is never sent elsewhere, and refuses bodies larger than
|
||||
MAX_CIVITAI_DOWNLOAD_BYTES.
|
||||
API key is never sent elsewhere, follows a redirect only as
|
||||
``_open_following_civitai_redirects`` allows (without the key), and
|
||||
refuses bodies larger than MAX_CIVITAI_DOWNLOAD_BYTES. A failure logs
|
||||
one warning with a short reason and the URL without query or
|
||||
credentials (#340), and returns None.
|
||||
"""
|
||||
if not is_civitai_url(url):
|
||||
logger.warning(f"Refusing non-CivitAI download URL: {url}")
|
||||
return None
|
||||
try:
|
||||
headers = {"User-Agent": "ComfyUI-PromptManager/1.0"}
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
req = urllib.request.Request(url, headers=headers)
|
||||
with _open_url(req, timeout=10) as resp:
|
||||
raw = _read_capped(resp, MAX_CIVITAI_DOWNLOAD_BYTES)
|
||||
if raw is None:
|
||||
logger.warning(f"Refusing oversized download: {url}")
|
||||
return None
|
||||
|
||||
# Resize to thumbnail to save disk space
|
||||
from io import BytesIO
|
||||
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(BytesIO(raw)) as img:
|
||||
img.thumbnail((_THUMB_MAX_SIZE, _THUMB_MAX_SIZE), Image.LANCZOS)
|
||||
rgb = img.convert("RGB")
|
||||
_save_atomically(rgb, local_path)
|
||||
if not is_civitai_url(url):
|
||||
raise _DownloadFailed("not an allowed CivitAI URL")
|
||||
image = _thumbnail(_fetch_civitai_body(url, api_key))
|
||||
try:
|
||||
_save_atomically(image, local_path)
|
||||
except OSError as error:
|
||||
raise _DownloadFailed(
|
||||
f"image could not be saved ({error.strerror or type(error).__name__})"
|
||||
) from None
|
||||
except _DownloadFailed as failure:
|
||||
reason = failure.reason
|
||||
except Exception as error: # best effort: an import never fails on one image
|
||||
reason = f"unexpected {type(error).__name__}"
|
||||
else:
|
||||
return str(local_path.resolve())
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to download {url}: {e}")
|
||||
return None
|
||||
logger.warning(f"CivitAI image download failed: {_loggable(url)}: {reason}")
|
||||
return None
|
||||
|
||||
|
||||
_PART_PREFIX = ".pm-"
|
||||
|
||||
@@ -149,3 +149,49 @@ test('Classic test button reports success and failure and releases expired notif
|
||||
await page.clock.fastForward(3001);
|
||||
await expect(page.locator('.pm-classic-notification')).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('Classic shows linked text, keeps the typed text through save and refuses to overwrite it (#343)',async({page})=>{
|
||||
await page.route('**/prompt_manager/recent?**',route=>route.fulfill({json:{success:true,results:[{text:'Saved prompt',tags:[]}]}}));
|
||||
const note=page.locator('.pm-classic').getByText('Text comes from a link. Your typed text is kept for when you disconnect it.',{exact:true});
|
||||
await expect(note).toBeHidden();
|
||||
await page.evaluate(()=>{
|
||||
const widget=node.widgets.find(item=>item.name==='text');widget.value=widget.inputEl.value='Typed classic';
|
||||
node.inputs=[{name:'text',type:'STRING',link:3,widget:{name:'text'}}];node.onConnectionsChange();
|
||||
});
|
||||
await expect(note).toBeVisible();
|
||||
// The link drives the run; the saved workflow keeps what the user typed, not a stale or empty value.
|
||||
expect(await page.evaluate(()=>node.serialize().widgets_values[0])).toBe('Typed classic');
|
||||
await page.getByRole('button',{name:'📋 Recent',exact:true}).click();
|
||||
await page.getByRole('button',{name:/Saved prompt/}).click();
|
||||
expect(await page.evaluate(()=>node.widgets.find(item=>item.name==='text').value)).toBe('Typed classic');
|
||||
await expect(page.getByRole('alert').filter({hasText:'Text comes from a link. Disconnect it to use a saved prompt.'})).toBeVisible();
|
||||
await page.evaluate(()=>{node.inputs[0].link=null;node.onConnectionsChange();});
|
||||
await expect(note).toBeHidden();
|
||||
await page.getByRole('button',{name:/Saved prompt/}).click();
|
||||
expect(await page.evaluate(()=>node.widgets.find(item=>item.name==='text').value)).toBe('Saved prompt');
|
||||
// A workflow load re-evaluates the link state.
|
||||
await page.evaluate(()=>{node.inputs[0].link=8;node.configure({});});
|
||||
await expect(note).toBeVisible();
|
||||
});
|
||||
|
||||
test('PromptManagerText shows linked text in its legacy controls and keeps the typed text (#343)',async({page})=>{
|
||||
await page.evaluate(async()=>{
|
||||
class TextNode {
|
||||
constructor(){this.widgets=['text','category','tags','search_text'].map(name=>({name,value:'',inputEl:document.createElement('textarea')}));this.widgets_values=[];this.inputs=[{name:'text',type:'STRING',link:null,widget:{name:'text'}}];}
|
||||
addDOMWidget(name,type,element){element.id='legacy-text-node';document.body.append(element);return {element};}
|
||||
setSize(){} setDirtyCanvas(){} serialize(){return {properties:{...this.properties}};}
|
||||
}
|
||||
await fixtureApp.extensions[0].beforeRegisterNodeDef(TextNode,{name:'PromptManagerText'},fixtureApp);
|
||||
window.textNode=new TextNode();textNode.onNodeCreated();
|
||||
const widget=textNode.widgets[0];widget.value=widget.inputEl.value='Typed text node';
|
||||
});
|
||||
const note=page.locator('#legacy-text-node').getByText('Text comes from a link. Your typed text is kept for when you disconnect it.',{exact:true});
|
||||
await expect(note).toBeHidden();
|
||||
await page.evaluate(()=>{textNode.inputs[0].link=5;textNode.onConnectionsChange();});
|
||||
await expect(note).toBeVisible();
|
||||
expect(await page.evaluate(()=>textNode.serialize().widgets_values[0])).toBe('Typed text node');
|
||||
await page.evaluate(()=>{textNode.usePrompt({text:'Replacement'},0);});
|
||||
expect(await page.evaluate(()=>textNode.widgets[0].value)).toBe('Typed text node');
|
||||
await page.evaluate(()=>{textNode.inputs[0].link=null;textNode.onConnectionsChange();textNode.onRemoved();});
|
||||
await expect(note).toBeHidden();
|
||||
});
|
||||
|
||||
@@ -26,7 +26,8 @@ test.beforeEach(async({page,browserName})=>{
|
||||
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings:new EventTarget()},graph:{setDirtyCanvas(){}},canvas:{}};
|
||||
await import('/extensions/promptmanager/prompt_manager.js');
|
||||
class Node {
|
||||
constructor(){this.widgets=['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=['clip','prepend_text','append_text'].map(name=>({name,link:null}));this.properties={};this.size=[550,700];}
|
||||
constructor(){this.widgets=['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[550,700];}
|
||||
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
|
||||
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
|
||||
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.height='650px';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
|
||||
setSize(size){this.size=size;}setDirtyCanvas(){}
|
||||
@@ -246,13 +247,39 @@ test(`${variant} restores workflow values and disclosures and protects connected
|
||||
await page.evaluate(()=>{const saved=node.serialize();node.configure(saved);});
|
||||
await expect(page.getByLabel('Category',{exact:true})).toBeVisible();
|
||||
await expect(page.getByLabel('Category',{exact:true})).toHaveValue('Saved category');
|
||||
await page.evaluate(()=>{node.inputs.push({name:'text',link:9});node.onConnectionsChange();});
|
||||
await page.evaluate(()=>{node.inputs.find(input=>input.name==='text').link=9;node.onConnectionsChange();});
|
||||
await expect(page.getByRole('textbox',{name:'Main prompt',exact:true})).toBeHidden();
|
||||
await expect(page.getByText('Text is supplied by an upstream connection.',{exact:true})).toBeVisible();
|
||||
await expect(page.getByText('Main prompt comes from a link. Your typed text is kept for when you disconnect it.',{exact:true})).toBeVisible();
|
||||
await page.evaluate(()=>{node.inputs.find(input=>input.name==='text').link=null;node.onConnectionsChange();});
|
||||
await expect(page.getByRole('textbox',{name:'Main prompt',exact:true})).toHaveValue('Saved draft');
|
||||
});
|
||||
|
||||
test(`${variant} offers text as a link socket, keeps typed text while linked and through a reload (#343)`,async({page})=>{
|
||||
const sockets=()=>page.evaluate(()=>Object.fromEntries(node.inputs.map(input=>[input.name,input.widget?.name??null])));
|
||||
// The hidden text widget no longer owns the input, so the host draws it as a socket; the other widget inputs are unchanged.
|
||||
expect(await sockets()).toEqual({clip:null,prepend_text:null,append_text:null,text:null,category:'category',tags:'tags',search_text:'search_text'});
|
||||
expect(await page.evaluate(()=>node.findInputByType('STRING').slot.name)).toBe('text');
|
||||
expect(await page.evaluate(()=>node.findInputByType('CLIP').slot.name)).toBe('clip');
|
||||
const main=page.getByRole('textbox',{name:'Main prompt',exact:true});
|
||||
await main.fill('Typed main');
|
||||
await page.evaluate(()=>{node.inputs.find(input=>input.name==='text').link=21;node.onConnectionsChange();});
|
||||
await expect(main).toBeHidden();
|
||||
const note=page.getByText('Main prompt comes from a link. Your typed text is kept for when you disconnect it.',{exact:true});
|
||||
await expect(note).toBeVisible();
|
||||
await expect(page.locator('[data-composed-preview]')).toHaveText('[text supplied by connection]');
|
||||
// A second STRING link dropped on the editor falls back to the frontend's own choice.
|
||||
expect(await page.evaluate(()=>node.findInputByType('STRING').slot.name)).toBe('prepend_text');
|
||||
// An older saved workflow brings the widget binding back with its inputs: configure exposes the socket again.
|
||||
const saved=await page.evaluate(()=>{const saved=node.serialize();node.inputs.find(input=>input.name==='text').widget={name:'text'};node.configure(saved);return saved;});
|
||||
expect(saved.widgets_values[0]).toBe('Typed main');
|
||||
expect((await sockets()).text).toBeNull();
|
||||
await expect(note).toBeVisible();
|
||||
await page.evaluate(()=>{node.inputs.find(input=>input.name==='text').link=null;node.onConnectionsChange();});
|
||||
await expect(main).toBeVisible();
|
||||
await expect(main).toHaveValue('Typed main');
|
||||
await expect(note).toBeHidden();
|
||||
});
|
||||
|
||||
test(`${variant} searches quoted tags with Enter and explicitly loads metadata`, async ({page}) => {
|
||||
let requested;
|
||||
await page.route('**/prompt_manager/search?**', async route => {
|
||||
|
||||
@@ -17,7 +17,8 @@ test.beforeEach(async({page,browserName})=>{
|
||||
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings},graph:{_nodes:[],setDirtyCanvas(){}},canvas:{},graphToPrompt:async()=>({})};
|
||||
await import('/extensions/promptmanager/prompt_manager.js');
|
||||
class Node {
|
||||
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=['clip','prepend_text','append_text'].map(name=>({name,link:null}));this.properties={};this.size=[1100,700];}
|
||||
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[1100,700];}
|
||||
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
|
||||
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
|
||||
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.cssText='height:700px;width:1100px;max-width:100%';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
|
||||
setSize(size){this.size=size;}setDirtyCanvas(){}
|
||||
@@ -123,6 +124,46 @@ test('a superseded library response never replaces the latest pair selection',as
|
||||
await expect(page.locator('.pm-editor-result')).not.toContainText('Stale positive');
|
||||
});
|
||||
|
||||
test('paired editor offers positive and negative link sockets and keeps both typed texts (#343)',async({page})=>{
|
||||
await draft(page,'Typed positive','Typed negative');
|
||||
expect(await page.evaluate(()=>node.inputs.filter(input=>!input.widget).map(input=>input.name))).toEqual(['clip','prepend_text','append_text','text','negative_text']);
|
||||
const target=()=>page.evaluate(()=>node.findInputByType('STRING').slot.name);
|
||||
expect(await target()).toBe('text');
|
||||
const positive=page.getByRole('textbox',{name:'Positive prompt',exact:true}),negative=page.getByRole('textbox',{name:'Negative prompt',exact:true});
|
||||
const positiveNote=page.getByText('Positive prompt comes from a link. Your typed text is kept for when you disconnect it.',{exact:true});
|
||||
const negativeNote=page.getByText('Negative prompt comes from a link. Your typed text is kept for when you disconnect it.',{exact:true});
|
||||
const link=(name,value)=>page.evaluate(({name,value})=>{node.inputs.find(input=>input.name===name).link=value;node.onConnectionsChange();},{name,value});
|
||||
for(const [linked,next] of [[['text'],'negative_text'],[['negative_text'],'text'],[['text','negative_text'],'prepend_text'],[[],'text']]){
|
||||
for(const name of ['text','negative_text'])await link(name,linked.includes(name)?30:null);
|
||||
expect(await target()).toBe(next);
|
||||
await expect(positive).toBeVisible({visible:!linked.includes('text')});
|
||||
await expect(negative).toBeVisible({visible:!linked.includes('negative_text')});
|
||||
await expect(positiveNote).toBeVisible({visible:linked.includes('text')});
|
||||
await expect(negativeNote).toBeVisible({visible:linked.includes('negative_text')});
|
||||
const saved=await page.evaluate(()=>{const saved=node.serialize();for(const input of node.inputs)if(['text','negative_text'].includes(input.name))input.widget={name:input.name};node.configure(saved);return saved.widgets_values.slice(0,2);});
|
||||
expect(saved).toEqual(['Typed positive','Typed negative']);
|
||||
expect(await page.evaluate(()=>node.inputs.filter(input=>input.widget&&['text','negative_text'].includes(input.name)).length)).toBe(0);
|
||||
}
|
||||
await expect(positive).toHaveValue('Typed positive');await expect(negative).toHaveValue('Typed negative');
|
||||
});
|
||||
|
||||
test('a link dropped on either prompt box, or its note, fills that box (#343 review M1)',async({page})=>{
|
||||
const keys=await page.evaluate(()=>[...document.querySelectorAll('.pm-pair .lg-node-widget [data-slot-key]')].map(key=>key.dataset.slotKey));
|
||||
// Nodes 2.0 resolves a drop through the nearest .lg-node-widget and its slot key: positive box and note, negative box and note.
|
||||
expect(keys).toEqual(['1-in-3','1-in-3','1-in-4','1-in-4']);
|
||||
const at=(name,connecting=true)=>page.evaluate(({name,connecting})=>{
|
||||
fixtureApp.canvas={linkConnector:{state:{connectingTo:connecting?'input':undefined}},ds:{convertOffsetToCanvas:point=>point},canvas:{getBoundingClientRect:()=>({left:0,top:0})}};
|
||||
const box=[...document.querySelectorAll('.pm-pair .lg-node-widget')].find(element=>element.getClientRects().length&&element.textContent.includes(name)).getBoundingClientRect();
|
||||
return node.getInputOnPos([box.left+box.width/2,box.top+box.height/2])?.name??null;
|
||||
},{name,connecting});
|
||||
expect(await at('Negative prompt')).toBe('negative_text');
|
||||
expect(await at('Positive prompt')).toBe('text');
|
||||
expect(await at('Negative prompt',false)).toBeNull();
|
||||
await page.evaluate(()=>{node.inputs.find(input=>input.name==='negative_text').link=4;node.onConnectionsChange();});
|
||||
expect(await at('Negative prompt comes from a link')).toBe('negative_text');
|
||||
expect(await page.evaluate(()=>{fixtureApp.canvas={};return node.getInputOnPos([1,1])??null;})).toBeNull();
|
||||
});
|
||||
|
||||
test('queue hook synchronizes both editors, preserves links and forwards branch execution arguments',async({page})=>{
|
||||
await draft(page);
|
||||
const args=await page.evaluate(async()=>{
|
||||
|
||||
@@ -0,0 +1,329 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { SIZES, THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate } from './concept-inventory.mjs';
|
||||
|
||||
// #249 System S9 Task 25: System › Backup & export (app.js renderSystem 691–693), its "Restore preview" (`restore`,
|
||||
// 1472–1476) and its "Import & inspect" (`import`, 1483–1487) against the immutable concept, on the concept-shaped
|
||||
// fixture. Each concept measurement is mapped only by the recorded deviations (D103, D104, D105; the Import dialog's
|
||||
// D15 and D55), each commented where it applies; after that the difference must be empty, and a mutated label, row,
|
||||
// danger mark or token must be noticed.
|
||||
test.use({ locale: 'en-US', timezoneId: 'UTC' });
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await seedConcept(request);
|
||||
await expect(await request.post('/__test__/system/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
page.conceptErrors = [];
|
||||
page.on('pageerror', error => page.conceptErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await request.post('/__test__/jobs/restart');
|
||||
await request.post('/__test__/system/reset');
|
||||
expect(page.conceptErrors).toEqual([]);
|
||||
});
|
||||
|
||||
const PREVIEW_URL = '/prompt_manager/restore/preview';
|
||||
|
||||
/** Runs in the page: the section's surfaces under `mainSelector`, their text, controls, geometry and tokens. */
|
||||
function measureBackups(mainSelector) {
|
||||
const layout = document.querySelector(`${mainSelector} .settings-layout`);
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const button = node => ({ text: text(node), className: node.className,
|
||||
...css(node, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') });
|
||||
const base = layout.getBoundingClientRect();
|
||||
const box = node => {
|
||||
const rect = node.getBoundingClientRect();
|
||||
return { left: Math.round(rect.left - base.left), top: Math.round(rect.top - base.top), width: Math.round(rect.width) };
|
||||
};
|
||||
const control = row => {
|
||||
const select = row.querySelector(':scope > select');
|
||||
if (select) return { kind: 'select', id: select.id, options: [...select.options].map(option => [option.textContent, option.disabled]),
|
||||
selected: select.selectedOptions[0]?.textContent ?? null, disabled: select.disabled };
|
||||
const toggle = row.querySelector(':scope > * input[type="checkbox"]');
|
||||
return toggle ? { kind: 'switch', id: toggle.id, wrapper: name(toggle.parentElement), checked: toggle.checked,
|
||||
disabled: toggle.disabled, ...css(toggle.parentElement, 'width', 'height') } : null;
|
||||
};
|
||||
const columns = layout.querySelector(':scope > .two-col');
|
||||
const surfaces = [...columns.querySelectorAll(':scope > section.surface')];
|
||||
const policy = layout.querySelector(':scope > section.surface');
|
||||
const chip = policy.querySelector(':scope > h2 > .chip');
|
||||
const row = policy.querySelector('.setting-row');
|
||||
const status = surfaces[0].querySelector(':scope > p.small');
|
||||
const warning = policy.querySelector(':scope > .warning-note');
|
||||
return {
|
||||
layout: [...layout.children].map(name),
|
||||
columns: { ...css(columns, 'display', 'gridTemplateColumns', 'columnGap'), box: box(columns) },
|
||||
surfaces: surfaces.map(surface => ({ tree: [...surface.children].map(name), heading: text(surface.querySelector(':scope > h2')),
|
||||
intro: text(surface.querySelector(':scope > p:not(.small)')), box: box(surface),
|
||||
actions: [...surface.querySelectorAll(':scope > .row-actions')].map(group => ({ tree: [...group.children].map(name),
|
||||
...css(group, 'display', 'columnGap', 'flexWrap', 'marginTop'),
|
||||
buttons: [...group.children].map(button) })) })),
|
||||
status: { text: text(status), ...css(status, 'fontSize', 'color', 'marginTop') },
|
||||
surface: css(surfaces[0], 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius'),
|
||||
h2: css(surfaces[0].querySelector('h2'), 'fontSize', 'fontWeight', 'color', 'marginBottom'),
|
||||
intro: css(surfaces[0].querySelector(':scope > p'), 'fontSize', 'color', 'lineHeight'),
|
||||
// Its place below the columns (the concept's `.surface + .surface` 18px), whatever the columns' height.
|
||||
policy: { tree: [...policy.children].map(name), heading: text(policy.querySelector(':scope > h2')),
|
||||
box: { left: box(policy).left, width: box(policy).width,
|
||||
gap: Math.round(policy.getBoundingClientRect().top - columns.getBoundingClientRect().bottom) },
|
||||
chip: { text: text(chip), className: chip.className, ...css(chip, 'color', 'fontSize', 'borderTopColor', 'backgroundColor') },
|
||||
rows: [...policy.querySelectorAll(':scope > .setting-row')].map(item => ({ tree: [...item.children].map(name),
|
||||
text: [...item.firstElementChild.children].map(node => [name(node), text(node), node.localName === 'label' ? node.htmlFor : null]),
|
||||
control: control(item) })),
|
||||
warning: { text: text(warning), ...css(warning, 'fontSize', 'color', 'marginTop', 'marginBottom') } },
|
||||
row: css(row, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', 'borderTopWidth', 'borderTopColor',
|
||||
'marginTop'),
|
||||
label: css(row.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
|
||||
description: css(row.querySelector('p'), 'fontSize', 'color', 'marginTop', 'maxWidth'),
|
||||
select: css(policy.querySelector('.setting-row > select'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor', 'color',
|
||||
'borderTopColor'),
|
||||
pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth,
|
||||
};
|
||||
}
|
||||
|
||||
const SCHEDULE_REASON = 'Daily and Weekly are not available yet: back up with Back up whenever you choose.';
|
||||
const SAFETY_DESCRIPTION = 'Keep a verified copy of the database on disk before maintenance, cleanups and LoRA imports change ' +
|
||||
'records. A restore always keeps one.';
|
||||
|
||||
/**
|
||||
* D103: "Back up" (the concept's "Simulate backup"), the status line from /backup/status, "Export JSON" (its "Export
|
||||
* sample JSON") and the two secondary exports in a second `.row-actions.spaced` with the concept's plain buttons.
|
||||
* D104: the toggle's line says what the policy does, the schedule's reason replaces the prototype's proposal line,
|
||||
* Daily and Weekly are disabled, and the warning note is truthful. The fixture has no backup yet.
|
||||
*/
|
||||
function mapBackupsConcept(reference) {
|
||||
const [database, portable] = reference.surfaces;
|
||||
const [backup, restore] = database.actions[0].buttons;
|
||||
const [exportButton, importButton] = portable.actions[0].buttons;
|
||||
const secondary = ['Export all prompts JSON', 'Export saved pairs JSON'].map(label => ({ ...exportButton, text: label }));
|
||||
return {
|
||||
...reference,
|
||||
surfaces: [
|
||||
{ ...database, actions: [{ ...database.actions[0], buttons: [{ ...backup, text: 'Back up' }, restore] }] },
|
||||
{ ...portable, tree: [...portable.tree, 'div.row-actions.spaced'],
|
||||
actions: [{ ...portable.actions[0], buttons: [{ ...exportButton, text: 'Export JSON' }, importButton] },
|
||||
{ ...portable.actions[0], tree: ['button', 'button'], buttons: secondary }] },
|
||||
],
|
||||
status: { ...reference.status, text: 'No backup recorded.' },
|
||||
policy: { ...reference.policy,
|
||||
rows: reference.policy.rows.map(item => (item.control.kind !== 'select'
|
||||
// D104 (M2): what the safety-backup policy really does.
|
||||
? { ...item, text: item.text.map(([node, value, target]) => [node, node === 'p' ? SAFETY_DESCRIPTION : value, target]) }
|
||||
: { ...item,
|
||||
text: item.text.map(([node, value, target]) => [node, node === 'p' ? SCHEDULE_REASON : value, target]),
|
||||
control: { ...item.control, options: item.control.options.map(([label]) => [label, label !== 'Manual only']) } })),
|
||||
warning: { ...reference.policy.warning, text: 'Restores are staged, migrated and validated before replacing live data.' } },
|
||||
};
|
||||
}
|
||||
|
||||
async function conceptBackups(page, theme) {
|
||||
await openConcept(page, 'system', theme);
|
||||
await page.locator('#main .section-nav').getByRole('button', { name: 'Backup & export', exact: true }).click();
|
||||
await expect(page.locator('#main .settings-layout > .two-col > section.surface')).toHaveCount(2);
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureBackups, '#main');
|
||||
}
|
||||
|
||||
async function productionBackups(page, theme, status = 'No backup recorded.') {
|
||||
await openProduction(page, 'system/backups', theme);
|
||||
await expect(page.locator('#workspace-main .settings-layout > .two-col > section.surface')).toHaveCount(2);
|
||||
// The status line and the policy row are their own reads: compare once both answered.
|
||||
await expect(page.locator('#workspace-main #backup-status')).toHaveText(status);
|
||||
await expect(page.locator('#workspace-main #backup-before')).toBeEnabled();
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureBackups, '#workspace-main');
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`backup & export matches the concept ${theme} ${width}x${height}`, async ({ page }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
const concept = await conceptBackups(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-backups-${theme}-${width}.png`), fullPage: true });
|
||||
const production = await productionBackups(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-backups-${theme}-${width}.png`), fullPage: true });
|
||||
expect(diffInventory(mapBackupsConcept(concept), production)).toEqual([]);
|
||||
expect(production.pageScroll).toBe(false);
|
||||
});
|
||||
}
|
||||
|
||||
/** Runs in the page: the open dialog's title, body tree, grid, note, backup box and actions with the concept's tokens. */
|
||||
function measureDialog() {
|
||||
const dialog = document.querySelector('dialog#modal[open]');
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const body = dialog.querySelector('#modal-body');
|
||||
const grid = body.querySelector(':scope > dl.meta-grid');
|
||||
const note = body.querySelector(':scope > p.small');
|
||||
const actions = body.querySelector(':scope > .dialog-actions');
|
||||
const list = body.querySelector(':scope > .operation-list');
|
||||
return {
|
||||
title: text(dialog.querySelector('#modal-title')),
|
||||
tree: [...body.children].map(name),
|
||||
grid: grid && { rows: [...grid.children].map(row => [text(row.querySelector('dt')), text(row.querySelector('dd'))]),
|
||||
...css(grid, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap'),
|
||||
dt: css(grid.querySelector('dt'), 'fontSize', 'color', 'textTransform'), dd: css(grid.querySelector('dd'), 'fontSize', 'color') },
|
||||
note: note && { text: text(note), ...css(note, 'fontSize', 'color') },
|
||||
backup: body.querySelector(':scope > .notice') && { tree: [...body.querySelector(':scope > .notice').children].map(name),
|
||||
text: [...body.querySelectorAll(':scope > .notice > div:first-child > *')].map(node => [node.localName, text(node)]),
|
||||
buttons: [...body.querySelectorAll(':scope > .notice button')].map(text) },
|
||||
list: list && { ...css(list, 'display', 'rowGap'),
|
||||
rows: [...list.children].map(row => ({ tree: [...row.children].map(name), title: text(row.querySelector('h3')),
|
||||
description: text(row.querySelector('p')), buttons: [...row.querySelectorAll('button')].map(text),
|
||||
...css(row, 'display', 'alignItems', 'paddingTop', 'borderTopColor') })) },
|
||||
actions: actions && { tree: [...actions.children].map(name), ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
|
||||
buttons: [...actions.children].map(item => ({ text: text(item), className: item.className,
|
||||
...css(item, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') })) },
|
||||
};
|
||||
}
|
||||
|
||||
const RESTORE_BACKUP_BOX = Object.freeze({ tree: ['div', 'div.row-actions'], text: [['h3', 'Back up the database first (recommended)'],
|
||||
['p', 'Downloads a copy of the prompts database before the restore replaces it.']], buttons: ['Back up now'] });
|
||||
|
||||
/**
|
||||
* D105: the server's review in the concept's "Restore preview": no sample note; the four rows with the review's values,
|
||||
* then "In the backup" and "In the library now"; the hold line in the concept's dialog `p.small` (its Import dialog's
|
||||
* intro, `small`); the D55 backup box before the actions; "Restore" (the concept's "Simulate restore", still danger).
|
||||
*/
|
||||
function mapRestoreConcept(reference, small, rows) {
|
||||
const [cancel, restore] = reference.actions.buttons;
|
||||
return {
|
||||
...reference,
|
||||
tree: ['dl.meta-grid', 'p.small', 'div.notice', 'div.dialog-actions'],
|
||||
grid: { ...reference.grid, rows },
|
||||
note: { ...small, text: 'Restore replaces the whole library with this backup and runs alone in Activity. ' +
|
||||
'Nothing changes until you restore. This review holds for 15 minutes.' },
|
||||
backup: RESTORE_BACKUP_BOX,
|
||||
actions: { ...reference.actions, buttons: [cancel, { ...restore, text: 'Restore' }] },
|
||||
};
|
||||
}
|
||||
|
||||
/** D15 and D55, as on Images: Rescan for "Scan an output folder", then "Scan for metadata"; no portable recipe import. */
|
||||
function mapImportConcept(reference) {
|
||||
return { ...reference,
|
||||
note: { ...reference.note, text: 'Image inspection and output scanning have different effects. Choose the workflow you want.' },
|
||||
list: { ...reference.list, rows: reference.list.rows.filter(row => row.title !== 'Import portable recipes').flatMap(row =>
|
||||
row.title === 'Scan an output folder'
|
||||
? [{ ...row, description: 'Find files added or removed in the output folders.', buttons: ['Rescan'] },
|
||||
{ ...row, title: 'Scan for metadata' }] : [row]) } };
|
||||
}
|
||||
|
||||
async function conceptDialog(page, theme, action, title) {
|
||||
await conceptBackups(page, theme);
|
||||
await page.locator(`#main .settings-layout [data-action="${action}"]`).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText(title);
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureDialog);
|
||||
}
|
||||
|
||||
/** Production's restore review of a fresh backup of this server's library; the review's own answer too. */
|
||||
async function productionRestore(page, request, theme) {
|
||||
const copy = await request.get('/prompt_manager/backup');
|
||||
await expect(copy).toBeOK();
|
||||
// That copy is this session's download now.
|
||||
await productionBackups(page, theme, /^Downloaded backup · .+ · verified\.$/);
|
||||
const chooser = page.waitForEvent('filechooser');
|
||||
const answer = page.waitForResponse(response => response.url().endsWith(PREVIEW_URL));
|
||||
await page.locator('#workspace-main').getByRole('button', { name: 'Review restore…', exact: true }).click();
|
||||
await (await chooser).setFiles({ name: 'prompts_backup_20261025_093000.db', mimeType: 'application/octet-stream',
|
||||
buffer: await copy.body() });
|
||||
const preview = await (await answer).json();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Restore preview');
|
||||
await page.mouse.move(0, 0);
|
||||
return { preview, production: await page.evaluate(measureDialog) };
|
||||
}
|
||||
|
||||
const counts = summary => `${summary.prompts} ${summary.prompts === 1 ? 'prompt' : 'prompts'} · ${summary.image_links} ` +
|
||||
`${summary.image_links === 1 ? 'image link' : 'image links'} · ${summary.pairs} ${summary.pairs === 1 ? 'saved pair' : 'saved pairs'}` +
|
||||
` · ${summary.runs} ${summary.runs === 1 ? 'run' : 'runs'}`;
|
||||
|
||||
function restoreRows(preview, snapshot) {
|
||||
return [['Snapshot', snapshot], ['Integrity', 'Verified'], ['Migration', 'Current schema'],
|
||||
['Safety backup', `Created before replacing (${preview.safety_backup.destination_label})`],
|
||||
['In the backup', counts(preview.summary)], ['In the library now', counts(preview.live)]];
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`restore preview and import dialogs match the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
// The concept's own dialog `p.small` (its Import dialog's intro) is the reference for the review's note.
|
||||
const conceptImport = await conceptDialog(page, theme, 'import', 'Import & inspect');
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-import-${theme}-${width}.png`) });
|
||||
const conceptRestore = await conceptDialog(page, theme, 'restore', 'Restore preview');
|
||||
expect(conceptRestore.tree).toEqual(['p.demo-note', 'dl.meta-grid', 'div.dialog-actions']);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-restore-${theme}-${width}.png`) });
|
||||
const { preview, production } = await productionRestore(page, request, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-restore-${theme}-${width}.png`) });
|
||||
const snapshot = production.grid.rows[0][1];
|
||||
expect(snapshot).toMatch(/^prompts_backup_20261025_093000.db · (\d+ KB|\d+\.\d MB) · .+$/);
|
||||
const { text: _intro, ...small } = conceptImport.note;
|
||||
expect(diffInventory(mapRestoreConcept(conceptRestore, small, restoreRows(preview, snapshot)), production)).toEqual([]);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(page.locator('dialog#modal[open]')).toHaveCount(0);
|
||||
await page.locator('#workspace-main').getByRole('button', { name: 'Import / inspect file', exact: true }).click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Import & inspect');
|
||||
await page.mouse.move(0, 0);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-import-${theme}-${width}.png`) });
|
||||
expect(diffInventory(mapImportConcept(conceptImport), await page.evaluate(measureDialog))).toEqual([]);
|
||||
});
|
||||
}
|
||||
|
||||
test('backup & export comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const concept = await conceptBackups(page, 'dark');
|
||||
const expected = mapBackupsConcept(concept);
|
||||
await productionBackups(page, 'dark');
|
||||
const measure = () => page.evaluate(measureBackups, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
expect(await paths()).toEqual([]);
|
||||
// Unmapped, the section differs exactly on the D103 and D104 paths (the added row's own paths, then the rest).
|
||||
const unmapped = diffInventory(concept, await measure()).map(item => item.path).sort();
|
||||
const added = 'surfaces.1.actions.1.';
|
||||
expect(unmapped.some(path => path.startsWith(added))).toBe(true);
|
||||
expect(unmapped.filter(path => !path.startsWith(added))).toEqual([
|
||||
'policy.rows.0.text.1.1', 'policy.rows.1.control.options.1.1', 'policy.rows.1.control.options.2.1', 'policy.rows.1.text.1.1',
|
||||
'policy.warning.text',
|
||||
'status.text', 'surfaces.0.actions.0.buttons.0.text', 'surfaces.1.actions.0.buttons.0.text', 'surfaces.1.actions.length',
|
||||
'surfaces.1.tree.3', 'surfaces.1.tree.length']);
|
||||
// A renamed label, a removed row, a lost danger mark and a swapped token are noticed.
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .setting-row label').textContent = 'Safety backup'; });
|
||||
expect(await paths()).toEqual(['policy.rows.0.text.0.1']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .setting-row label').textContent = 'Create a safety backup before maintenance'; });
|
||||
const undoRow = await mutate(page, { remove: ['#workspace-main .settings-layout > section.surface > .setting-row:last-of-type'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['policy.rows.length', 'policy.tree.length']));
|
||||
await undoRow();
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main button.danger').classList.remove('danger'); });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['surfaces.0.actions.0.buttons.1.className']));
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main .two-col .row-actions > button:nth-child(2)').classList.add('danger'); });
|
||||
const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['row.borderTopColor']));
|
||||
await undoToken();
|
||||
const undoChip = await mutate(page, { css: '#workspace-main .chip.blue { color: rgb(255, 0, 0); }' });
|
||||
expect(await paths()).toEqual(['policy.chip.color']);
|
||||
await undoChip();
|
||||
expect(await paths()).toEqual([]);
|
||||
|
||||
// The restore review: a renamed term, a lost danger mark and the grid's token are noticed.
|
||||
const conceptImport = await conceptDialog(page, 'dark', 'import', 'Import & inspect');
|
||||
const conceptRestore = await conceptDialog(page, 'dark', 'restore', 'Restore preview');
|
||||
const { preview, production } = await productionRestore(page, request, 'dark');
|
||||
const { text: _intro, ...small } = conceptImport.note;
|
||||
const expectedRestore = mapRestoreConcept(conceptRestore, small, restoreRows(preview, production.grid.rows[0][1]));
|
||||
const restorePaths = async () => diffInventory(expectedRestore, await page.evaluate(measureDialog)).map(item => item.path);
|
||||
expect(await restorePaths()).toEqual([]);
|
||||
await page.evaluate(() => { document.querySelector('dialog#modal[open] dl.meta-grid dt').textContent = 'Backup'; });
|
||||
expect(await restorePaths()).toEqual(['grid.rows.0.0']);
|
||||
await page.evaluate(() => { document.querySelector('dialog#modal[open] dl.meta-grid dt').textContent = 'Snapshot'; });
|
||||
await page.evaluate(() => { document.querySelector('dialog#modal[open] .dialog-actions .danger').classList.remove('danger'); });
|
||||
expect(await restorePaths()).toEqual(expect.arrayContaining(['actions.buttons.1.className']));
|
||||
await page.evaluate(() => { document.querySelector('dialog#modal[open] .dialog-actions button:last-child').classList.add('danger'); });
|
||||
const undoGrid = await mutate(page, { css: 'dialog#modal dl.meta-grid { column-gap: 3px; }' });
|
||||
expect(await restorePaths()).toEqual(expect.arrayContaining(['grid.columnGap']));
|
||||
await undoGrid();
|
||||
expect(await restorePaths()).toEqual([]);
|
||||
});
|
||||
@@ -172,8 +172,8 @@ function mapOverviewConcept(reference, overview) {
|
||||
// D93: the real pending count; Review metadata opens Needs review (the concept's go-review).
|
||||
'3 ambiguous image links': row => ({ ...row,
|
||||
title: `${number(overview.needs_review.pending)} ambiguous image link${overview.needs_review.pending === 1 ? '' : 's'}` }),
|
||||
// D93: no backup has been recorded (verified backups arrive with S8).
|
||||
'Backup status': row => ({ ...row, description: 'No retained backup record.' }),
|
||||
// D93 (S9): what GET /backup/status reports; the fixture has no backup yet.
|
||||
'Backup status': row => ({ ...row, description: 'No backup recorded.' }),
|
||||
// D93: there is no paging, so the shared settings are grid and density.
|
||||
'One display configuration': row => ({ ...row, description: 'Grid and density settings are shared across views.' }),
|
||||
// D93: the real sources, not samples.
|
||||
@@ -219,6 +219,9 @@ async function productionOverview(page, theme) {
|
||||
await expect(page.locator('#workspace-main .summary-metrics')).toBeVisible();
|
||||
// The thumbnail row is its own read (I2): compare once it answered.
|
||||
await expect(page.locator('#workspace-main .operation h3').filter({ hasText: 'thumbnails' })).toHaveText('No missing thumbnails');
|
||||
// So is the backup row (S9).
|
||||
await expect(page.locator('#workspace-main .operation').filter({ has: page.locator('h3', { hasText: 'Backup status' }) })
|
||||
.locator('p')).toHaveText('No backup recorded.');
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureSystem, '#workspace-main');
|
||||
}
|
||||
|
||||
@@ -0,0 +1,489 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { seedConcept } from './concept-inventory.mjs';
|
||||
|
||||
// System › Backup & export (#249 System S9 Task 24) on the S8 backend and the concept-shaped fixture: Back up and its
|
||||
// status line (D103), the restore review (D105) and its job, the exports, Import / inspect and the recovery policy
|
||||
// (D104). A restore here replaces this worker's own temp database; every test starts from a fresh concept fixture.
|
||||
const workspace = '/prompt_manager/workspace';
|
||||
const STATUS_URL = '/prompt_manager/backup/status';
|
||||
const PREVIEW_URL = '/prompt_manager/restore/preview';
|
||||
const APPLY_URL = '/prompt_manager/restore/apply';
|
||||
const WHEN = { dateStyle: 'medium', timeStyle: 'short' };
|
||||
const SAFETY_DESCRIPTION = 'Keep a verified copy of the database on disk before maintenance, cleanups and LoRA imports change ' +
|
||||
'records. A restore always keeps one.';
|
||||
|
||||
test.use({ locale: 'en-US', timezoneId: 'UTC' });
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await seedConcept(request);
|
||||
await expect(await request.post('/__test__/system/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
page.systemErrors = [];
|
||||
page.on('pageerror', error => page.systemErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await request.post('/__test__/release');
|
||||
await request.post('/__test__/jobs/restart');
|
||||
await request.post('/__test__/system/reset');
|
||||
expect(page.systemErrors).toEqual([]);
|
||||
});
|
||||
|
||||
const toast = page => page.locator('#workspace-toast');
|
||||
const layoutOf = page => page.locator('#workspace-main .settings-layout');
|
||||
const dialog = page => page.locator('dialog#modal[open]');
|
||||
const statusLine = page => layoutOf(page).locator('#backup-status');
|
||||
const settingsStatus = page => page.locator('#settings-status');
|
||||
const save = page => page.getByRole('button', { name: 'Save preferences', exact: true });
|
||||
const buttonOf = (page, name) => layoutOf(page).getByRole('button', { name, exact: true });
|
||||
const settingOf = (page, label) => layoutOf(page).locator('.setting-row').filter({ has: page.locator(`label:text-is("${label}")`) });
|
||||
|
||||
async function serverJson(request, url) {
|
||||
const response = await request.get(url);
|
||||
await expect(response).toBeOK();
|
||||
return response.json();
|
||||
}
|
||||
|
||||
async function openBackups(page) {
|
||||
// A fresh page each time: the drafts live for the page session (Q10), and a hash-only goto would not reload.
|
||||
if (page.url().endsWith('#/system/backups')) await page.reload();
|
||||
else await page.goto(`${workspace}#/system/backups`);
|
||||
await expect(layoutOf(page).locator(':scope > .two-col > section.surface')).toHaveCount(2);
|
||||
await expect(statusLine(page)).not.toHaveText('Checking backups…');
|
||||
// The policy row is painted once the server settings were read.
|
||||
await expect(settingOf(page, 'Create a safety backup before maintenance').locator('input')).toBeEnabled();
|
||||
return layoutOf(page);
|
||||
}
|
||||
|
||||
/** The status line the page should show for the server's current status (its own wording, in this locale). */
|
||||
async function expectedStatus(page, request) {
|
||||
const status = await serverJson(request, STATUS_URL);
|
||||
return page.evaluate(([answer, options]) => {
|
||||
const when = value => new Date(value).toLocaleString(undefined, options);
|
||||
const parts = [];
|
||||
if (answer.last_download) parts.push(`Downloaded backup · ${when(answer.last_download.at)} · ${answer.last_download.verified ? 'verified' : 'not verified'}.`);
|
||||
if (answer.safety_backups.length) parts.push(`Safety backup · ${when(answer.safety_backups[0].modified_at)} · ${answer.safety_backups.length} on disk.`);
|
||||
return parts.join(' ') || 'No backup recorded.';
|
||||
}, [status, WHEN]);
|
||||
}
|
||||
|
||||
/** A verified copy of the library as it is now (this records the session's last download, as Back up does). */
|
||||
async function backupBytes(request) {
|
||||
const response = await request.get('/prompt_manager/backup');
|
||||
await expect(response).toBeOK();
|
||||
expect(response.headers()['x-promptmanager-verify']).toBe('ok');
|
||||
return response.body();
|
||||
}
|
||||
|
||||
/** Pick `buffer` as the backup file through Review restore…'s file chooser; resolves with the preview's answer. */
|
||||
async function chooseBackup(page, buffer, name = 'prompts_backup_20261025_093000.db') {
|
||||
const chooser = page.waitForEvent('filechooser');
|
||||
const answer = page.waitForResponse(response => response.url().endsWith(PREVIEW_URL) && response.request().method() === 'POST');
|
||||
await buttonOf(page, 'Review restore…').click();
|
||||
await (await chooser).setFiles({ name, mimeType: 'application/octet-stream', buffer });
|
||||
return answer;
|
||||
}
|
||||
|
||||
const gridOf = page => dialog(page).locator('dl.meta-grid > div').evaluateAll(items => items.map(item =>
|
||||
[item.querySelector('dt').textContent, item.querySelector('dd').textContent]));
|
||||
|
||||
const libraryTotal = async request => (await serverJson(request, '/prompt_manager/library?limit=1')).total;
|
||||
|
||||
test('Back up downloads through the api client and revokes the object URL (D103)', async ({ page, request }) => {
|
||||
await page.addInitScript(() => {
|
||||
window.objectUrls = { created: [], revoked: [] };
|
||||
const create = URL.createObjectURL.bind(URL), revoke = URL.revokeObjectURL.bind(URL);
|
||||
URL.createObjectURL = blob => { const url = create(blob); window.objectUrls.created.push(url); return url; };
|
||||
URL.revokeObjectURL = url => { window.objectUrls.revoked.push(url); revoke(url); };
|
||||
});
|
||||
await openBackups(page);
|
||||
await expect(statusLine(page)).toHaveText('No backup recorded.');
|
||||
const download = page.waitForEvent('download');
|
||||
const answer = page.waitForResponse(response => new URL(response.url()).pathname === '/prompt_manager/backup');
|
||||
await buttonOf(page, 'Back up').click();
|
||||
expect((await answer).headers()['x-promptmanager-verify']).toBe('ok');
|
||||
expect((await download).suggestedFilename()).toMatch(/^prompts_backup_\d{8}_\d{6}\.db$/);
|
||||
await expect(toast(page)).toHaveText('Database backup downloaded.');
|
||||
// The status line is read again: this session's verified download, as the server records it.
|
||||
await expect(statusLine(page)).toHaveText(/^Downloaded backup · .+ · verified\.$/);
|
||||
await expect(statusLine(page)).toHaveText(await expectedStatus(page, request));
|
||||
await expect(buttonOf(page, 'Back up')).not.toHaveAttribute('aria-disabled', 'true');
|
||||
const urls = await page.evaluate(() => window.objectUrls.created);
|
||||
expect(urls).toHaveLength(1);
|
||||
await expect.poll(() => page.evaluate(() => window.objectUrls.revoked)).toEqual(urls);
|
||||
});
|
||||
|
||||
test('status line shows only server-reported backups (D103)', async ({ page, request }) => {
|
||||
await openBackups(page);
|
||||
await expect(statusLine(page)).toHaveText('No backup recorded.');
|
||||
await expect(await request.post('/__test__/system/safety-backup')).toBeOK();
|
||||
await openBackups(page);
|
||||
await expect(statusLine(page)).toHaveText(/^Safety backup · .+ · 1 on disk\.$/);
|
||||
await expect(statusLine(page)).toHaveText(await expectedStatus(page, request));
|
||||
// A status the server could not report is never shown as "no backup".
|
||||
await page.route(`**${STATUS_URL}`, route => route.fulfill({ status: 500, json: { success: false, error: 'Failed' } }));
|
||||
await page.reload();
|
||||
await expect(statusLine(page)).toHaveText('The backup status could not be read.');
|
||||
});
|
||||
|
||||
test('Overview backup row uses the same status (D93)', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/safety-backup')).toBeOK();
|
||||
await backupBytes(request);
|
||||
await page.goto(`${workspace}#/system/overview`);
|
||||
const row = layoutOf(page).locator('.operation').filter({ has: page.locator('h3', { hasText: 'Backup status' }) });
|
||||
const expected = await expectedStatus(page, request);
|
||||
expect(expected).toMatch(/^Downloaded backup · .+ · verified\. Safety backup · .+ · 1 on disk\.$/);
|
||||
await expect(row.locator('p')).toHaveText(expected);
|
||||
await row.getByRole('button', { name: 'View backups', exact: true }).click();
|
||||
await expect(page).toHaveURL(`${workspace}#/system/backups`);
|
||||
await expect(statusLine(page)).toHaveText(expected);
|
||||
});
|
||||
|
||||
test('Review restore stages the chosen file and shows the real summary with the backup box (D105)', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
await expect(await request.post('/prompt_manager/save', { data: { text: 'S9 prompt saved after the backup' } })).toBeOK();
|
||||
const before = await serverJson(request, '/__test__/system/digest');
|
||||
await openBackups(page);
|
||||
const answer = await chooseBackup(page, bytes);
|
||||
const preview = await answer.json();
|
||||
expect(preview.integrity).toBe('ok');
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
const counts = summary => `${summary.prompts.toLocaleString('en-US')} ${summary.prompts === 1 ? 'prompt' : 'prompts'} · ` +
|
||||
`${summary.image_links} ${summary.image_links === 1 ? 'image link' : 'image links'} · ` +
|
||||
`${summary.pairs} ${summary.pairs === 1 ? 'saved pair' : 'saved pairs'} · ${summary.runs} ${summary.runs === 1 ? 'run' : 'runs'}`;
|
||||
const rows = await gridOf(page);
|
||||
expect(rows.map(([term]) => term)).toEqual(['Snapshot', 'Integrity', 'Migration', 'Safety backup', 'In the backup', 'In the library now']);
|
||||
expect(rows[0][1]).toMatch(/^prompts_backup_20261025_093000.db · (\d+ KB|\d+\.\d MB) · .+$/);
|
||||
expect(rows.slice(1)).toEqual([['Integrity', 'Verified'], ['Migration', 'Current schema'],
|
||||
['Safety backup', `Created before replacing (${preview.safety_backup.destination_label})`],
|
||||
['In the backup', counts(preview.summary)], ['In the library now', counts(preview.live)]]);
|
||||
// The prompt saved after the backup is the difference the review shows.
|
||||
expect(preview.live.prompts).toBe(preview.summary.prompts + 1);
|
||||
await expect(dialog(page).locator('#modal-body > p.small')).toHaveText('Restore replaces the whole library with this backup ' +
|
||||
'and runs alone in Activity. Nothing changes until you restore. This review holds for 15 minutes.');
|
||||
await expect(dialog(page).locator('.notice h3')).toHaveText('Back up the database first (recommended)');
|
||||
await expect(dialog(page).locator('.notice p')).toHaveText('Downloads a copy of the prompts database before the restore replaces it.');
|
||||
const actions = dialog(page).locator('.dialog-actions > button');
|
||||
await expect(actions).toHaveText(['Cancel', 'Restore']);
|
||||
await expect(actions.nth(1)).toHaveClass('danger');
|
||||
// The backup box downloads a copy and keeps the review open.
|
||||
const download = page.waitForEvent('download');
|
||||
await dialog(page).getByRole('button', { name: 'Back up now', exact: true }).click();
|
||||
await download;
|
||||
await expect(toast(page)).toHaveText('Database backup downloaded.');
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
// A review writes nothing: the rows and the media are as they were.
|
||||
expect(await serverJson(request, '/__test__/system/digest')).toEqual(before);
|
||||
expect((await serverJson(request, '/__test__/system/backups')).stages).toBe(1);
|
||||
});
|
||||
|
||||
test('Cancel discards the stage; Escape too', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
const before = await serverJson(request, '/__test__/system/digest');
|
||||
await openBackups(page);
|
||||
await chooseBackup(page, bytes);
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
expect((await serverJson(request, '/__test__/system/backups')).stages).toBe(1);
|
||||
let discarded = page.waitForResponse(response => response.url().includes(`${PREVIEW_URL}/`) && response.request().method() === 'DELETE');
|
||||
await dialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
expect((await discarded).status()).toBe(200);
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(buttonOf(page, 'Review restore…')).toBeFocused();
|
||||
expect((await serverJson(request, '/__test__/system/backups')).stages).toBe(0);
|
||||
await chooseBackup(page, bytes);
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
discarded = page.waitForResponse(response => response.url().includes(`${PREVIEW_URL}/`) && response.request().method() === 'DELETE');
|
||||
await page.keyboard.press('Escape');
|
||||
expect((await discarded).status()).toBe(200);
|
||||
expect((await serverJson(request, '/__test__/system/backups')).stages).toBe(0);
|
||||
expect(await serverJson(request, '/__test__/system/digest')).toEqual(before);
|
||||
});
|
||||
|
||||
test('Restore starts the job, opens Activity; views reload after it finishes', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
const restoredTotal = await libraryTotal(request);
|
||||
await expect(await request.post('/prompt_manager/save', { data: { text: 'S9 prompt the restore removes' } })).toBeOK();
|
||||
expect(await libraryTotal(request)).toBe(restoredTotal + 1);
|
||||
await openBackups(page);
|
||||
const libraryCount = page.locator('aside.sidebar a[data-route="library"] .nav-count');
|
||||
await expect(libraryCount).toHaveText((restoredTotal + 1).toLocaleString('en-US'));
|
||||
await chooseBackup(page, bytes);
|
||||
const applied = page.waitForResponse(response => response.url().endsWith(APPLY_URL));
|
||||
await dialog(page).getByRole('button', { name: 'Restore', exact: true }).click();
|
||||
const { job } = await (await applied).json();
|
||||
expect([job.kind, job.title, job.cancellable]).toEqual(['restore', 'Database restore', false]);
|
||||
await expect(page).toHaveURL(`${workspace}#/activity/${job.id}`);
|
||||
await expect(toast(page)).toHaveText(/^Database restore finished · Restored \d+ prompts?, /);
|
||||
// The library is the backup's again, and the page says so without a reload: the sidebar counts were read again.
|
||||
expect(await libraryTotal(request)).toBe(restoredTotal);
|
||||
await expect(libraryCount).toHaveText(restoredTotal.toLocaleString('en-US'));
|
||||
expect((await serverJson(request, '/__test__/system/backups')).safety).toHaveLength(1);
|
||||
// The Activity job view stayed, and Backup & export now names the safety backup the restore kept.
|
||||
await expect(page).toHaveURL(`${workspace}#/activity/${job.id}`);
|
||||
await page.goto(`${workspace}#/system/backups`);
|
||||
await expect(statusLine(page)).toHaveText(/Safety backup · .+ · 1 on disk\.$/);
|
||||
});
|
||||
|
||||
test('a finished restore shows the open page again from the restored data', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
const restoredTotal = await libraryTotal(request);
|
||||
await expect(await request.post('/prompt_manager/save', { data: { text: 'S9 prompt the restore removes' } })).toBeOK();
|
||||
await openBackups(page);
|
||||
await chooseBackup(page, bytes);
|
||||
const preview = page.locator('dialog#modal[open] #modal-title');
|
||||
await expect(preview).toHaveText('Restore preview');
|
||||
// The restore is held at its start, so the reader is on the Library (showing the prompt) when it finishes.
|
||||
await expect(await request.post('/__test__/hold', { data: { path: APPLY_URL } })).toBeOK();
|
||||
const applied = page.waitForResponse(response => response.url().endsWith(APPLY_URL));
|
||||
await dialog(page).getByRole('button', { name: 'Restore', exact: true }).click();
|
||||
await expect.poll(async () => (await serverJson(request, '/__test__/hold')).held[APPLY_URL]).toBe(1);
|
||||
await page.evaluate(() => { window.location.hash = '#/library'; });
|
||||
const card = page.locator('#workspace-main article.prompt-card').filter({ hasText: 'S9 prompt the restore removes' });
|
||||
await expect(card).toHaveCount(1);
|
||||
await expect(await request.post('/__test__/release')).toBeOK();
|
||||
await applied;
|
||||
await expect(toast(page)).toHaveText(/^Database restore finished · /);
|
||||
// The shelf was read again from the restored library, on the same route.
|
||||
await expect(card).toHaveCount(0);
|
||||
await expect(page).toHaveURL(`${workspace}#/library`);
|
||||
expect(await libraryTotal(request)).toBe(restoredTotal);
|
||||
});
|
||||
|
||||
test('invalid file shows the validator message; nothing is staged', async ({ page, request }) => {
|
||||
const before = await serverJson(request, '/__test__/system/digest');
|
||||
await openBackups(page);
|
||||
const answer = await chooseBackup(page, Buffer.from('this is not a database'), 'notes.db');
|
||||
expect(answer.status()).toBe(400);
|
||||
const body = await answer.json();
|
||||
expect(body.code).toBe('invalid-backup');
|
||||
await expect(toast(page)).toHaveText(`This file can't be restored: ${body.error.replace(/\.?$/, '.')}`);
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(buttonOf(page, 'Review restore…')).not.toHaveAttribute('aria-disabled', 'true');
|
||||
expect((await serverJson(request, '/__test__/system/backups')).stages).toBe(0);
|
||||
expect(await serverJson(request, '/__test__/system/digest')).toEqual(before);
|
||||
});
|
||||
|
||||
test('Export JSON downloads /library/export; secondary buttons download all prompts (/export) and /pairs/export (D103)', async ({ page }) => {
|
||||
const layout = await openBackups(page);
|
||||
const portable = layout.locator('section.surface').filter({ has: page.locator('h2', { hasText: 'Portable export' }) });
|
||||
expect(await portable.locator('.row-actions > button').allTextContents())
|
||||
.toEqual(['Export JSON', 'Import / inspect file', 'Export all prompts JSON', 'Export saved pairs JSON']);
|
||||
for (const [label, url, name, kind] of [
|
||||
['Export JSON', '/prompt_manager/library/export', /^prompt-manager-library-v1\.json$/, 'prompt-manager-library'],
|
||||
['Export all prompts JSON', '/prompt_manager/export', /^prompt_manager_\d{8}_\d{6}\.json$/, null],
|
||||
['Export saved pairs JSON', '/prompt_manager/pairs/export', /^prompt-pairs\.json$/, 'prompt-manager-pairs']]) {
|
||||
const download = page.waitForEvent('download');
|
||||
const answer = page.waitForResponse(response => new URL(response.url()).pathname === url);
|
||||
await portable.getByRole('button', { name: label, exact: true }).click();
|
||||
expect((await answer).status()).toBe(200);
|
||||
const file = await download;
|
||||
expect(file.suggestedFilename()).toMatch(name);
|
||||
const parsed = JSON.parse(await (await file.createReadStream()).toArray().then(chunks => Buffer.concat(chunks).toString('utf8')));
|
||||
if (kind) expect(parsed.kind).toBe(kind);
|
||||
else expect(Array.isArray(parsed.prompts)).toBe(true);
|
||||
}
|
||||
// A failed export says so and saves nothing.
|
||||
await page.route('**/prompt_manager/pairs/export', route => route.fulfill({ status: 500, json: { success: false } }));
|
||||
await portable.getByRole('button', { name: 'Export saved pairs JSON', exact: true }).click();
|
||||
await expect(toast(page)).toHaveText('The export could not be downloaded. Nothing was saved to this device.');
|
||||
});
|
||||
|
||||
test('Import / inspect lists Inspect, Rescan and the shared scan review (D15, D55)', async ({ page, request }) => {
|
||||
await openBackups(page);
|
||||
const open = async () => {
|
||||
await buttonOf(page, 'Import / inspect file').click();
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Import & inspect');
|
||||
};
|
||||
await open();
|
||||
const rows = dialog(page).locator('.operation-list > .operation');
|
||||
expect(await rows.locator('h3').allTextContents()).toEqual(['Inspect a local image', 'Scan an output folder', 'Scan for metadata']);
|
||||
expect(await rows.locator('button').allTextContents()).toEqual(['Open inspector', 'Rescan', 'Review scan']);
|
||||
// Rescan reads the output folders again and says what it found; while it runs, a reopened dialog says so.
|
||||
await expect(await request.post('/__test__/hold', { data: { path: '/prompt_manager/gallery/rescan' } })).toBeOK();
|
||||
const rescan = page.waitForResponse(response => response.url().endsWith('/prompt_manager/gallery/rescan'));
|
||||
await rows.getByRole('button', { name: 'Rescan', exact: true }).click();
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(toast(page)).toHaveText('Rescanning output folders…');
|
||||
await open();
|
||||
await expect(rows.nth(1).locator('button')).toHaveText('Rescanning…');
|
||||
await expect(rows.nth(1).locator('button')).toHaveAttribute('aria-disabled', 'true');
|
||||
await expect(await request.post('/__test__/release')).toBeOK();
|
||||
expect((await rescan).status()).toBe(200);
|
||||
await expect(toast(page)).toHaveText(/^Rescan finished\. \d+ files? in \d+ output folders?\.$/);
|
||||
await expect(rows.nth(1).locator('button')).toHaveText('Rescan');
|
||||
// Scan for metadata opens the shared review over the server's output folders; Cancel starts nothing.
|
||||
await dialog(page).getByRole('button', { name: 'Review scan', exact: true }).click();
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Review metadata scan');
|
||||
await expect(dialog(page).locator('dl.meta-grid dd').first()).toHaveText(/^\d+ output folders?$/);
|
||||
await dialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
// Open inspector goes to Inspect a file.
|
||||
await open();
|
||||
await dialog(page).getByRole('button', { name: 'Open inspector', exact: true }).click();
|
||||
await expect(page).toHaveURL(`${workspace}#/inspect`);
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('safety backup toggle saves through Save preferences with revision (D104)', async ({ page, request }) => {
|
||||
const before = await serverJson(request, '/prompt_manager/settings');
|
||||
expect(before.settings.safety_backup).toBe(true);
|
||||
await openBackups(page);
|
||||
const row = settingOf(page, 'Create a safety backup before maintenance');
|
||||
// D104: what the policy really does (the concept's "Preview a verified backup…" line does not say it).
|
||||
await expect(row.locator('p').first()).toHaveText(SAFETY_DESCRIPTION);
|
||||
const toggle = row.locator('input');
|
||||
await expect(toggle).toBeChecked();
|
||||
await toggle.uncheck();
|
||||
await expect(settingsStatus(page)).toHaveText('Unsaved changes.');
|
||||
const saved = page.waitForRequest(item => item.url().endsWith('/prompt_manager/settings') && item.method() === 'POST');
|
||||
await save(page).click();
|
||||
const body = (await saved).postDataJSON();
|
||||
expect(body).toEqual({ safety_backup: false, expected_revision: before.revision });
|
||||
await expect(settingsStatus(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, '/prompt_manager/settings')).settings.safety_backup).toBe(false);
|
||||
// A maintenance review now says so.
|
||||
const plan = await request.post('/prompt_manager/maintenance/preview', { data: { operation: 'optimize', options: {} } });
|
||||
expect((await plan.json()).backup.policy).toBe('off');
|
||||
await openBackups(page);
|
||||
await expect(toggle).not.toBeChecked();
|
||||
// A configuration changed elsewhere is a conflict, never a blind write.
|
||||
await toggle.check();
|
||||
await expect(await request.post('/prompt_manager/settings', { data: { result_timeout: 7,
|
||||
expected_revision: (await serverJson(request, '/prompt_manager/settings')).revision } })).toBeOK();
|
||||
await save(page).click();
|
||||
await expect(settingsStatus(page)).toHaveText('Settings changed elsewhere. Reload this section to see them.');
|
||||
expect((await serverJson(request, '/prompt_manager/settings')).settings.safety_backup).toBe(false);
|
||||
});
|
||||
|
||||
test('schedule offers Manual only; Daily/Weekly disabled with reason (D104)', async ({ page }) => {
|
||||
await openBackups(page);
|
||||
const schedule = settingOf(page, 'Scheduled database backups');
|
||||
expect(await schedule.locator('option').evaluateAll(options => options.map(option => [option.textContent, option.disabled])))
|
||||
.toEqual([['Manual only', false], ['Daily', true], ['Weekly', true]]);
|
||||
await expect(schedule.locator('select')).toHaveValue('Manual only');
|
||||
await expect(schedule.locator('p')).toHaveText('Daily and Weekly are not available yet: back up with Back up whenever you choose.');
|
||||
const policy = layoutOf(page).locator(':scope > section.surface');
|
||||
await expect(policy.locator('h2')).toHaveText('Recovery policy Proposed');
|
||||
await expect(policy.locator('h2 > span.chip.blue')).toHaveText('Proposed');
|
||||
await expect(policy.locator('p.warning-note')).toHaveText('Restores are staged, migrated and validated before replacing live data.');
|
||||
// Nothing on the schedule is ever drafted.
|
||||
await expect(settingsStatus(page)).toHaveText('No unsaved changes.');
|
||||
});
|
||||
|
||||
test('a failed settings read leaves the policy disabled with the reason', async ({ page }) => {
|
||||
await page.route('**/prompt_manager/settings', route => (route.request().method() === 'GET'
|
||||
? route.fulfill({ status: 500, json: { success: false, error: 'Failed' } }) : route.continue()));
|
||||
await page.goto(`${workspace}#/system/backups`);
|
||||
const row = settingOf(page, 'Create a safety backup before maintenance');
|
||||
await expect(row.locator('p.small')).toHaveText('Server settings could not be read.');
|
||||
await expect(row.locator('input')).toBeDisabled();
|
||||
});
|
||||
|
||||
test('the legacy admin Backup and Restore keep working', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
const restored = await request.post('/prompt_manager/restore', { multipart: { database_file: {
|
||||
name: 'prompts_backup.db', mimeType: 'application/octet-stream', buffer: bytes } } });
|
||||
await expect(restored).toBeOK();
|
||||
expect((await restored.json()).success).toBe(true);
|
||||
await page.goto('/prompt_manager/admin');
|
||||
await expect(page.locator('#restoreFileInput')).toHaveCount(1);
|
||||
});
|
||||
|
||||
/** Start a fake running job (a metadata scan another tab started), which an exclusive restore must wait for. */
|
||||
async function runningScan(request) {
|
||||
const response = await request.post('/__test__/jobs/start', { data: { kind: 'scan', title: 'Metadata scan', total: 4 } });
|
||||
await expect(response).toBeOK();
|
||||
return (await response.json()).job;
|
||||
}
|
||||
|
||||
const stagesOf = async request => (await serverJson(request, '/__test__/system/backups')).stages;
|
||||
const deleted = page => page.waitForResponse(response => response.url().includes(`${PREVIEW_URL}/`) &&
|
||||
response.request().method() === 'DELETE');
|
||||
|
||||
test('a refused restore reopens its review: Restore retries it, Discard releases it (I1)', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
await openBackups(page);
|
||||
await chooseBackup(page, bytes);
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
const scan = await runningScan(request);
|
||||
const refused = page.waitForResponse(response => response.url().endsWith(APPLY_URL));
|
||||
await dialog(page).getByRole('button', { name: 'Restore', exact: true }).click();
|
||||
expect((await refused).status()).toBe(409);
|
||||
await expect(toast(page)).toHaveText('Another job is running: Metadata scan.');
|
||||
// The same review is open again, its stage still held: Restore retries, Discard releases it.
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
await expect(dialog(page).locator('.dialog-actions > button')).toHaveText(['Discard', 'Restore']);
|
||||
expect(await stagesOf(request)).toBe(1);
|
||||
await expect(await request.post(`/__test__/jobs/${scan.id}/finish`, { data: { state: 'finished' } })).toBeOK();
|
||||
const applied = page.waitForResponse(response => response.url().endsWith(APPLY_URL));
|
||||
await dialog(page).getByRole('button', { name: 'Restore', exact: true }).click();
|
||||
const { job } = await (await applied).json();
|
||||
await expect(page).toHaveURL(`${workspace}#/activity/${job.id}`);
|
||||
await expect(toast(page)).toHaveText(/^Database restore finished · /);
|
||||
expect(await stagesOf(request)).toBe(0);
|
||||
});
|
||||
|
||||
test('two refused restores in a row, then a third Review restore… still opens the preview (I1)', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
await openBackups(page);
|
||||
await runningScan(request);
|
||||
for (const leave of ['discard', 'escape']) {
|
||||
await chooseBackup(page, bytes);
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
const refused = page.waitForResponse(response => response.url().endsWith(APPLY_URL));
|
||||
await dialog(page).getByRole('button', { name: 'Restore', exact: true }).click();
|
||||
expect((await refused).status()).toBe(409);
|
||||
await expect(dialog(page).locator('.dialog-actions > button')).toHaveText(['Discard', 'Restore']);
|
||||
const released = deleted(page);
|
||||
if (leave === 'discard') await dialog(page).getByRole('button', { name: 'Discard', exact: true }).click();
|
||||
else await page.keyboard.press('Escape');
|
||||
expect((await released).status()).toBe(200);
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
expect(await stagesOf(request)).toBe(0);
|
||||
}
|
||||
const answer = await chooseBackup(page, bytes);
|
||||
expect((await answer).status()).toBe(200);
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
expect(await stagesOf(request)).toBe(1);
|
||||
});
|
||||
|
||||
test('× and leaving the section or the page discard the stage (M1)', async ({ page, request }) => {
|
||||
const bytes = await backupBytes(request);
|
||||
await openBackups(page);
|
||||
const leaves = [
|
||||
['×', () => page.locator('#modal-close').click()],
|
||||
// The modal covers the page, so the section changes as a link or the Back button changes it: by its hash.
|
||||
['section', () => page.evaluate(() => { window.location.hash = '#/system/maintenance'; })],
|
||||
['route', () => page.evaluate(() => { window.location.hash = '#/images'; })],
|
||||
];
|
||||
for (const [name, leave] of leaves) {
|
||||
if (!page.url().endsWith('#/system/backups')) {
|
||||
await page.goto(`${workspace}#/system/backups`);
|
||||
await expect(statusLine(page)).not.toHaveText('Checking backups…');
|
||||
}
|
||||
await chooseBackup(page, bytes);
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
expect(await stagesOf(request), name).toBe(1);
|
||||
// A keepalive DELETE (it may outlive the page) is asserted by its request and the server's stage count.
|
||||
const released = page.waitForRequest(item => item.url().includes(`${PREVIEW_URL}/`) && item.method() === 'DELETE');
|
||||
await leave();
|
||||
await released;
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect.poll(() => stagesOf(request), { message: name }).toBe(0);
|
||||
}
|
||||
});
|
||||
|
||||
test('a safety backup file (no .db extension) can be picked and reviewed (M3)', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/safety-backup')).toBeOK();
|
||||
const bytes = await backupBytes(request);
|
||||
await openBackups(page);
|
||||
// The picker lists every file; the server decides what is a database.
|
||||
await expect(page.locator('#restore-file')).not.toHaveAttribute('accept', /./);
|
||||
const answer = await chooseBackup(page, bytes, 'prompts.db.backup_20261025_093000_000000');
|
||||
expect((await answer).status()).toBe(200);
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Restore preview');
|
||||
expect((await gridOf(page))[0][1]).toMatch(/^prompts\.db\.backup_20261025_093000_000000 · \d+ KB · .+$/);
|
||||
});
|
||||
@@ -36,6 +36,9 @@ const layoutOf = page => page.locator('#workspace-main .settings-layout');
|
||||
async function openOverview(page) {
|
||||
await page.goto(`${workspace}#/system/overview`);
|
||||
await expect(layoutOf(page).locator(':scope > .summary-metrics')).toBeVisible();
|
||||
// The Backup status row is its own read of /backup/status (S9): compare once it answered.
|
||||
await expect(layoutOf(page).locator('.operation').filter({ has: page.locator('h3', { hasText: 'Backup status' }) })
|
||||
.locator('p')).not.toHaveText('Checking backups…');
|
||||
return layoutOf(page);
|
||||
}
|
||||
|
||||
@@ -67,7 +70,7 @@ test('overview shows real counts and Not checked', async ({ page, request }) =>
|
||||
]);
|
||||
expect(await rowsOf(layout, 'Library health')).toEqual([
|
||||
['0 ambiguous image links', 'Review metadata before assigning a positive / negative pair.', 'Review metadata'],
|
||||
['Backup status', 'No retained backup record.', 'View backups'],
|
||||
['Backup status', 'No backup recorded.', 'View backups'],
|
||||
// D129: the fixture's database is configured explicitly, so it reads as config.json's choice and does not move.
|
||||
['Library location', 'browser.db · The database path set in config.json.', 'Details'],
|
||||
['No missing thumbnails', 'Generate only missing previews when necessary.', 'Review thumbnails'],
|
||||
|
||||
@@ -464,17 +464,19 @@ test('System is native: the concept shell, section navigation, sections, hand-of
|
||||
await expect(button).toBeFocused();
|
||||
// S2 Task 6: Overview is native (workspace-system.spec.mjs), and so are Appearance (S3 Task 8) and Sources & scans
|
||||
// (S4 Task 11, workspace-system-sources.spec.mjs, its update notice shown in a fresh browser), Integrations (S10
|
||||
// Task 27, workspace-system-integrations.spec.mjs) and Maintenance (S7 Task 19,
|
||||
// workspace-system-maintenance.spec.mjs); every other section hands off.
|
||||
// Task 27, workspace-system-integrations.spec.mjs), Maintenance (S7 Task 19,
|
||||
// workspace-system-maintenance.spec.mjs) and Backup & export (S9 Task 24, workspace-system-backups.spec.mjs);
|
||||
// every other section hands off.
|
||||
if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible();
|
||||
if (hash === 'sources') await expect(layout.locator('#source-list .operation')).toHaveCount(2);
|
||||
expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`)))
|
||||
.toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'section.surface', 'div.save-row'],
|
||||
sources: ['div.notice', 'section.surface', 'div.save-row'],
|
||||
integrations: ['section.surface', 'section.surface', 'div.save-row'],
|
||||
maintenance: ['div.notice', 'section.surface', 'section.surface', 'div.save-row'] }[hash]
|
||||
maintenance: ['div.notice', 'section.surface', 'section.surface', 'div.save-row'],
|
||||
backups: ['div.two-col', 'section.surface', 'div.save-row'] }[hash]
|
||||
?? ['section.surface handoff', 'div.save-row']);
|
||||
if (['overview', 'appearance', 'sources', 'integrations', 'maintenance'].includes(hash)) continue;
|
||||
if (['overview', 'appearance', 'sources', 'integrations', 'maintenance', 'backups'].includes(hash)) continue;
|
||||
const handoff = layout.locator('section.handoff');
|
||||
await expect(handoff.locator('h2')).toHaveText('Your current tools are ready');
|
||||
expect(await handoff.locator('a.workspace-link').evaluateAll(links => links.map(link => link.getAttribute('href'))))
|
||||
|
||||
@@ -50,6 +50,12 @@ read. These per-server hooks only change what a System test needs on top:
|
||||
rows (a read-only dump) and of every media file under ``output`` and
|
||||
``input`` (not the automatic previews in ``thumbnails``), so a test proves
|
||||
a Cancel wrote nothing.
|
||||
- ``POST /__test__/system/safety-backup`` (S9) writes one verified safety
|
||||
backup beside the database, as a maintenance apply does;
|
||||
``GET /__test__/system/backups`` answers ``{safety: [names], stages: n}``:
|
||||
the ``<db>.backup_*`` files and the held ``.pm-restore-*`` stage folders.
|
||||
Reset drops every held restore review, removes the safety backups and
|
||||
forgets the session's last download.
|
||||
|
||||
Delays use the shared request holds (``request_hold.py``).
|
||||
"""
|
||||
@@ -151,11 +157,45 @@ def register_system_commands(app, routes, api, root: Path):
|
||||
staged.write_bytes(config_bytes)
|
||||
os.replace(staged, config_path)
|
||||
|
||||
def backup_files():
|
||||
db_path = Path(api.db.model.db_path)
|
||||
safety = sorted(
|
||||
path.name
|
||||
for path in db_path.parent.glob(db_path.name + ".backup_*")
|
||||
if path.is_file()
|
||||
)
|
||||
stages = [
|
||||
path
|
||||
for path in db_path.parent.glob(".pm-restore-*")
|
||||
if path.is_dir() and not path.name.startswith(".pm-restore-upload-")
|
||||
]
|
||||
return db_path, safety, stages
|
||||
|
||||
async def reset_backups():
|
||||
await api._discard_held(api._restore_plans.pop_all())
|
||||
db_path, safety, _stages = backup_files()
|
||||
for name in safety:
|
||||
(db_path.parent / name).unlink(missing_ok=True)
|
||||
api._last_backup_download = None
|
||||
|
||||
@routes.post("/__test__/system/reset")
|
||||
async def system_reset(request):
|
||||
reset()
|
||||
await reset_backups()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/safety-backup")
|
||||
async def safety_backup(request):
|
||||
from py.api.maintenance import take_safety_backup
|
||||
|
||||
name = await api._run_in_executor(take_safety_backup, api.db.model)
|
||||
return web.json_response({"success": True, "name": name})
|
||||
|
||||
@routes.get("/__test__/system/backups")
|
||||
async def backups(request):
|
||||
_db_path, safety, stages = backup_files()
|
||||
return web.json_response({"safety": safety, "stages": len(stages)})
|
||||
|
||||
@routes.post("/__test__/system/fail")
|
||||
async def system_fail(request):
|
||||
data = await request.json()
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
// #343: a link drives a PromptManager node's text (and the Pair's negative) on the real pinned host.
|
||||
// The workflow is built by the real frontend through the nodes' sockets and queued as the frontend serializes it;
|
||||
// only inference is replaced (fixtures/ledger_nodes.py). The linked text must be saved, counted, recorded in the
|
||||
// execution history, and the run's image must link to it.
|
||||
import { execFile } from 'node:child_process';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { promisify } from 'node:util';
|
||||
import { expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
|
||||
import { test, readExecutionRuns } from './execution-fixture.mjs';
|
||||
import { installHostHooks } from './host-fixture.mjs';
|
||||
|
||||
// The worker's owned host must be up before the shared host hooks open the page.
|
||||
test.beforeEach(async ({ execution }) => { expect(execution.databasePath).toBeTruthy(); });
|
||||
installHostHooks(test, ['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerPair', 'PMHistoryInputs']);
|
||||
|
||||
// `primitive`: a core string node the queue-time resolver follows. `legacy`: the frontend's virtual Primitive node,
|
||||
// whose value is queued as literal text. `preview`: PrimitiveString -> PreviewAny, the shape of the maintainer's
|
||||
// TextGenerate -> PreviewAny chain, whose text is only known when the node runs.
|
||||
function build(page, { kind, positive, negative, prefix }) {
|
||||
return page.evaluate(async ({ kind, positive, negative, prefix }) => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
app.graph.clear();
|
||||
const add = (type, x, y) => { const node = LiteGraph.createNode(type); app.graph.add(node); node.pos = [x, y]; return node; };
|
||||
const slot = (node, name) => { const index = node.inputs.findIndex(input => input.name === name); if (index < 0) throw new Error(`${node.type} has no ${name} socket`); return index; };
|
||||
const connect = (from, output, to, name) => { if (!from.connect(output, to, slot(to, name))) throw new Error(`${from.type} could not link to ${to.type}.${name}`); };
|
||||
const inputs = add('PMHistoryInputs', 0, 0);
|
||||
const prompt = add(kind, 700, 0);
|
||||
const set = (name, value) => { const widget = prompt.widgets.find(item => item.name === name); widget.value = value; if (widget.inputEl) widget.inputEl.value = value; };
|
||||
set('text', positive.typed);
|
||||
if (negative) set('negative_text', negative.typed);
|
||||
const drive = (name, side, y) => {
|
||||
if (!side.via) return;
|
||||
if (side.via === 'legacy') {
|
||||
// The frontend's own Primitive: a virtual node whose value is applied when the workflow is queued.
|
||||
const primitive = add('PrimitiveNode', 0, y);
|
||||
connect(primitive, 0, prompt, name);
|
||||
primitive.widgets[0].value = side.text;
|
||||
return;
|
||||
}
|
||||
const text = add('PrimitiveString', 0, y);
|
||||
text.widgets.find(widget => widget.name === 'value').value = side.text;
|
||||
let source = text;
|
||||
if (side.via === 'preview') { source = add('PreviewAny', 350, y); connect(text, 0, source, 'source'); }
|
||||
connect(source, 0, prompt, name);
|
||||
};
|
||||
drive('text', positive, 300);
|
||||
if (negative) drive('negative_text', negative, 500);
|
||||
if (kind !== 'PromptManagerText') connect(inputs, 0, prompt, 'clip');
|
||||
const sampler = add('KSampler', 1400, 0);
|
||||
connect(inputs, 1, sampler, 'model');
|
||||
connect(inputs, 2, sampler, 'latent_image');
|
||||
if (kind === 'PromptManagerText') {
|
||||
const encode = add('CLIPTextEncode', 1100, 0);
|
||||
encode.convertWidgetToInput?.(encode.widgets.find(widget => widget.name === 'text'));
|
||||
connect(inputs, 0, encode, 'clip');
|
||||
connect(prompt, 0, encode, 'text');
|
||||
connect(encode, 0, sampler, 'positive');
|
||||
} else connect(prompt, 0, sampler, 'positive');
|
||||
if (kind === 'PromptManagerPair') connect(prompt, 1, sampler, 'negative');
|
||||
else {
|
||||
const empty = add('CLIPTextEncode', 1100, 300);
|
||||
connect(inputs, 0, empty, 'clip');
|
||||
connect(empty, 0, sampler, 'negative');
|
||||
}
|
||||
const images = add('PMHistoryImages', 1800, 0);
|
||||
images.widgets.find(widget => widget.name === 'value').value = 99;
|
||||
connect(sampler, 0, images, 'latent');
|
||||
const save = add('SaveImage', 2100, 0);
|
||||
save.widgets.find(widget => widget.name === 'filename_prefix').value = prefix;
|
||||
connect(images, 0, save, 'images');
|
||||
// The Classic nodes install their own text serializer shortly after creation; queue after it, as a user does.
|
||||
await prompt._pmTextSerializerReady;
|
||||
const { output, workflow } = await app.graphToPrompt();
|
||||
return { output, workflow, promptNode: String(prompt.id) };
|
||||
}, { kind, positive, negative, prefix });
|
||||
}
|
||||
|
||||
async function run(execution, page, testCase) {
|
||||
const prefix = `textlink/${randomUUID()}`;
|
||||
const { output, workflow, promptNode } = await build(page, { ...testCase, prefix });
|
||||
const id = randomUUID();
|
||||
const response = await execution.request.post('/prompt', { data: { prompt: output, prompt_id: id, extra_data: { extra_pnginfo: { workflow } } } });
|
||||
expect(response.ok(), await response.text()).toBe(true);
|
||||
let item;
|
||||
await expect.poll(async () => {
|
||||
item = (await (await execution.request.get(`/history/${id}`)).json())[id];
|
||||
return item?.status?.status_str ?? null;
|
||||
}, { timeout: 20000 }).toBe('success');
|
||||
let result;
|
||||
await expect.poll(async () => {
|
||||
result = await readExecutionRuns(execution.databasePath, id);
|
||||
const run = result.rows[0];
|
||||
return run ? result.captures[run.run_id].state : null;
|
||||
}, { timeout: 10000, message: 'the run and its prompt context reach SQLite' }).toBe('complete');
|
||||
const run = result.rows[0];
|
||||
const files = Object.values(item.outputs).flatMap(value => value.images || []).map(image => image.filename);
|
||||
return { output, promptNode, context: result.captures[run.run_id].contexts[0], usage: result.usage[run.run_id], files };
|
||||
}
|
||||
|
||||
async function scan(execution) {
|
||||
const response = await execution.request.post('/prompt_manager/scan', { timeout: 60000 });
|
||||
expect(response.ok()).toBe(true);
|
||||
const events = (await response.text()).split('\n').filter(line => line.startsWith('data: ')).map(line => JSON.parse(line.slice(6)));
|
||||
expect(events.at(-1)?.type, JSON.stringify(events.at(-1))).toBe('complete');
|
||||
}
|
||||
|
||||
// filename -> linked prompt id (null when the image is not linked), read-only from the owned host's library.
|
||||
const LINKS = `import json, sqlite3, sys, pathlib
|
||||
db = sqlite3.connect(pathlib.Path(sys.argv[1]).resolve().as_uri() + "?mode=ro", uri=True)
|
||||
links = {name: None for name in sys.argv[2:]}
|
||||
for name, prompt in db.execute("SELECT filename, prompt_id FROM generated_images WHERE filename IN (%s)" % ",".join("?" * (len(sys.argv) - 2)), sys.argv[2:]):
|
||||
links[name] = prompt
|
||||
print(json.dumps(links))`;
|
||||
async function linkedPrompts(execution, files) {
|
||||
const { stdout } = await promisify(execFile)(process.env.PM_TEST_PYTHON || 'python', ['-c', LINKS, execution.databasePath, ...files], { timeout: 5000 });
|
||||
return JSON.parse(stdout);
|
||||
}
|
||||
|
||||
// Text a link resolves from the queued graph (a primitive) links its images through their embedded metadata.
|
||||
// Text only known at run time (PreviewAny) cannot be read from the image: its attribution from the recorded run is
|
||||
// #342's. Here it must never be linked to another prompt.
|
||||
async function expectImagesLinked(execution, evidence, viaRuntime) {
|
||||
await scan(execution);
|
||||
const links = await linkedPrompts(execution, evidence.files);
|
||||
expect(evidence.files.length).toBeGreaterThan(0);
|
||||
for (const [file, prompt] of Object.entries(links)) {
|
||||
if (viaRuntime) expect([null, evidence.context.positive_prompt_id], file).toContain(prompt);
|
||||
else expect(prompt, file).toBe(evidence.context.positive_prompt_id);
|
||||
}
|
||||
}
|
||||
|
||||
const unique = label => `${label} ${randomUUID()}`;
|
||||
|
||||
const sources = { primitive: 'PrimitiveString', legacy: 'a legacy Primitive', preview: 'PreviewAny' };
|
||||
for (const kind of ['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit']) {
|
||||
for (const via of Object.keys(sources)) {
|
||||
test(`${kind} saves, counts, records and links images to text linked from ${sources[via]}`, async ({ execution, page }) => {
|
||||
test.setTimeout(90000);
|
||||
const text = unique(`Linked ${kind} ${via}`);
|
||||
const evidence = await run(execution, page, { kind, positive: { typed: 'Typed under the link', text, via } });
|
||||
const source = evidence.output[evidence.promptNode].inputs.text;
|
||||
// A real node is queued as a link; the virtual Primitive as its own value, never the hidden typed text.
|
||||
if (via === 'legacy') expect(source).toBe(text);
|
||||
else expect(Array.isArray(source), 'the frontend queued the link, not the typed text').toBe(true);
|
||||
expect(evidence.context.positive.raw).toBe(text);
|
||||
expect(evidence.context.positive.effective).toBe(text);
|
||||
expect(evidence.usage).toEqual([{ role: 'positive', run_count: 1 }]);
|
||||
const saved = await (await execution.request.get(`/prompt_manager/library/${evidence.context.positive_prompt_id}`)).json();
|
||||
expect(saved.positive.text).toBe(text);
|
||||
await expectImagesLinked(execution, evidence, via === 'preview');
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// A legacy Primitive's wildcard is drawn once when the workflow is queued: the run saves and counts what was queued.
|
||||
for (const kind of ['PromptManager', 'PromptManagerFocused']) {
|
||||
test(`${kind} saves and counts the text a wildcard legacy Primitive queued (#343 re-review R1)`, async ({ execution, page }) => {
|
||||
test.setTimeout(90000);
|
||||
const stem = unique(`Wildcard ${kind}`);
|
||||
const evidence = await run(execution, page, { kind, positive: { typed: 'Typed under the link', text: `${stem} {alpha|beta}`, via: 'legacy' } });
|
||||
const queuedText = evidence.output[evidence.promptNode].inputs.text;
|
||||
// The Classic node keeps its widget binding: the host applies the Primitive, and its own text serializer
|
||||
// queues the text as written, as on main. The Focused editor queues one choice.
|
||||
if (kind === 'PromptManager') expect(queuedText).toBe(`${stem} {alpha|beta}`);
|
||||
else expect([`${stem} alpha`, `${stem} beta`]).toContain(queuedText);
|
||||
expect(evidence.context.positive.raw).toBe(queuedText);
|
||||
expect(evidence.usage).toEqual([{ role: 'positive', run_count: 1 }]);
|
||||
const saved = await (await execution.request.get(`/prompt_manager/library/${evidence.context.positive_prompt_id}`)).json();
|
||||
expect(saved.positive.text).toBe(queuedText);
|
||||
});
|
||||
}
|
||||
|
||||
for (const [label, linked, negativeVia] of [['positive', ['text'], 'primitive'], ['negative', ['negative_text'], 'primitive'],
|
||||
['both', ['text', 'negative_text'], 'primitive'], ['neither', [], 'primitive'], ['negative from a legacy Primitive', ['negative_text'], 'legacy']]) {
|
||||
test(`PromptManagerPair with ${label} linked saves, counts and records the pair and links its images`, async ({ execution, page }) => {
|
||||
test.setTimeout(90000);
|
||||
const sides = {
|
||||
text: { typed: unique('Typed positive'), text: unique('Linked positive'), via: linked.includes('text') ? 'preview' : null },
|
||||
negative_text: { typed: unique('Typed negative'), text: unique('Linked negative'), via: linked.includes('negative_text') ? negativeVia : null },
|
||||
};
|
||||
const evidence = await run(execution, page, { kind: 'PromptManagerPair', positive: sides.text, negative: sides.negative_text });
|
||||
const expected = name => sides[name].via ? sides[name].text : sides[name].typed;
|
||||
for (const name of ['text', 'negative_text']) {
|
||||
const queuedValue = evidence.output[evidence.promptNode].inputs[name];
|
||||
if (sides[name].via === 'legacy') expect(queuedValue, name).toBe(sides[name].text);
|
||||
else expect(Array.isArray(queuedValue), name).toBe(linked.includes(name));
|
||||
}
|
||||
expect(evidence.context.positive.raw).toBe(expected('text'));
|
||||
expect(evidence.context.negative.raw).toBe(expected('negative_text'));
|
||||
expect(evidence.usage).toEqual([{ role: 'positive', run_count: 1 }, { role: 'negative', run_count: 0 }, { role: 'pair', run_count: 1 }]);
|
||||
const pair = await (await execution.request.get(`/prompt_manager/pairs/${evidence.context.pair_id}`)).json();
|
||||
expect(pair.pair).toMatchObject({ positive_text: expected('text'), negative_text: expected('negative_text') });
|
||||
await expectImagesLinked(execution, evidence, sides.text.via === 'preview');
|
||||
});
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Deterministic nodes and queue controls installed only in an owned test host."""
|
||||
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
|
||||
@@ -255,3 +256,50 @@ async def cache_mode(request: web.Request) -> web.Response:
|
||||
raise web.HTTPBadRequest()
|
||||
_CACHE_MODE = mode
|
||||
return web.json_response({"configured": True})
|
||||
|
||||
|
||||
def _plugin_module(suffix: str):
|
||||
"""A loaded PromptManager module (ComfyUI names the package by its path)."""
|
||||
import sys
|
||||
|
||||
for name, module in list(sys.modules.items()):
|
||||
if name.endswith("promptmanager." + suffix):
|
||||
return module
|
||||
raise web.HTTPServiceUnavailable(text=f"PromptManager {suffix} not loaded")
|
||||
|
||||
|
||||
@PromptServer.instance.routes.post("/pm_execution_fixture/monitor")
|
||||
async def image_monitor(request: web.Request) -> web.Response:
|
||||
"""Start or stop PromptManager's image monitor on the host output folder.
|
||||
|
||||
The managed host's config disables monitoring; only the attribution cases
|
||||
(#342) turn it on, and they turn it off again.
|
||||
"""
|
||||
enabled = (await request.json())["enabled"]
|
||||
if type(enabled) is not bool:
|
||||
raise web.HTTPBadRequest()
|
||||
import folder_paths
|
||||
|
||||
monitor_module = _plugin_module("utils.image_monitor")
|
||||
config = _plugin_module("py.config").GalleryConfig
|
||||
monitor = monitor_module._monitor_instance
|
||||
if monitor is None:
|
||||
database = _plugin_module("database.operations").PromptDatabase()
|
||||
tracker = _plugin_module("utils.prompt_tracker").get_prompt_tracker(database)
|
||||
monitor = monitor_module.get_image_monitor(database, tracker)
|
||||
if enabled:
|
||||
config.MONITORING_ENABLED = True
|
||||
monitor.start_monitoring([folder_paths.get_output_directory()])
|
||||
else:
|
||||
monitor.stop_monitoring()
|
||||
config.MONITORING_ENABLED = False
|
||||
return web.json_response({"monitoring": monitor.is_monitoring})
|
||||
|
||||
|
||||
@PromptServer.instance.routes.get("/pm_execution_fixture/monitor")
|
||||
async def image_monitor_state(request: web.Request) -> web.Response:
|
||||
"""File names of keyless images waiting for their run's recorded outputs."""
|
||||
monitor = _plugin_module("utils.image_monitor")._monitor_instance
|
||||
handler = getattr(monitor, "handler", None)
|
||||
waiting = handler.awaiting_history() if handler is not None else []
|
||||
return web.json_response({"awaiting": [os.path.basename(p) for p in waiting]})
|
||||
|
||||
@@ -0,0 +1,379 @@
|
||||
{
|
||||
"id": "37b6baef-94f5-49d2-9b42-1ffc70e7da92",
|
||||
"revision": 0,
|
||||
"last_node_id": 4,
|
||||
"last_link_id": 2,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 1,
|
||||
"type": "PromptManagerFocused",
|
||||
"pos": [
|
||||
600,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
550,
|
||||
650
|
||||
],
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"localized_name": "clip",
|
||||
"name": "clip",
|
||||
"type": "CLIP",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "prepend_text",
|
||||
"name": "prepend_text",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "append_text",
|
||||
"name": "append_text",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "text",
|
||||
"name": "text",
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "text"
|
||||
},
|
||||
"link": 1
|
||||
},
|
||||
{
|
||||
"localized_name": "category",
|
||||
"name": "category",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "category"
|
||||
},
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "tags",
|
||||
"name": "tags",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "tags"
|
||||
},
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "search_text",
|
||||
"name": "search_text",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "search_text"
|
||||
},
|
||||
"link": null
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"localized_name": "CONDITIONING",
|
||||
"name": "CONDITIONING",
|
||||
"type": "CONDITIONING",
|
||||
"links": null
|
||||
},
|
||||
{
|
||||
"localized_name": "STRING",
|
||||
"name": "STRING",
|
||||
"type": "STRING",
|
||||
"links": null
|
||||
}
|
||||
],
|
||||
"title": "Focused fed by a Primitive",
|
||||
"properties": {
|
||||
"Node name for S&R": "PromptManagerFocused",
|
||||
"focusedUI": {
|
||||
"libraryOpen": false,
|
||||
"libraryFilters": {
|
||||
"category": "",
|
||||
"tags": ""
|
||||
},
|
||||
"composition": false,
|
||||
"metadata": false,
|
||||
"preview": false
|
||||
}
|
||||
},
|
||||
"widgets_values": [
|
||||
"Primitive focused text",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
""
|
||||
],
|
||||
"widgets_values_named": {
|
||||
"text": "Primitive focused text",
|
||||
"category": "",
|
||||
"tags": "",
|
||||
"search_text": "",
|
||||
"prepend_text": "",
|
||||
"append_text": "",
|
||||
"prompt_manager_focused_ui": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "PromptManagerPair",
|
||||
"pos": [
|
||||
600,
|
||||
1200
|
||||
],
|
||||
"size": [
|
||||
1100,
|
||||
700
|
||||
],
|
||||
"flags": {},
|
||||
"order": 3,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"localized_name": "clip",
|
||||
"name": "clip",
|
||||
"type": "CLIP",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "prepend_text",
|
||||
"name": "prepend_text",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "append_text",
|
||||
"name": "append_text",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "text",
|
||||
"name": "text",
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "text"
|
||||
},
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "negative_text",
|
||||
"name": "negative_text",
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "negative_text"
|
||||
},
|
||||
"link": 2
|
||||
},
|
||||
{
|
||||
"localized_name": "category",
|
||||
"name": "category",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "category"
|
||||
},
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "tags",
|
||||
"name": "tags",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "tags"
|
||||
},
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "search_text",
|
||||
"name": "search_text",
|
||||
"shape": 7,
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "search_text"
|
||||
},
|
||||
"link": null
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"localized_name": "positive",
|
||||
"name": "positive",
|
||||
"type": "CONDITIONING",
|
||||
"links": null
|
||||
},
|
||||
{
|
||||
"localized_name": "negative",
|
||||
"name": "negative",
|
||||
"type": "CONDITIONING",
|
||||
"links": null
|
||||
},
|
||||
{
|
||||
"localized_name": "positive_text",
|
||||
"name": "positive_text",
|
||||
"type": "STRING",
|
||||
"links": null
|
||||
},
|
||||
{
|
||||
"localized_name": "negative_text",
|
||||
"name": "negative_text",
|
||||
"type": "STRING",
|
||||
"links": null
|
||||
}
|
||||
],
|
||||
"title": "Pair negative fed by a Primitive",
|
||||
"properties": {
|
||||
"Node name for S&R": "PromptManagerPair",
|
||||
"pairUI": {
|
||||
"libraryOpen": true,
|
||||
"libraryFilters": {
|
||||
"category": "",
|
||||
"tags": ""
|
||||
},
|
||||
"composition": false,
|
||||
"metadata": false,
|
||||
"preview": false
|
||||
}
|
||||
},
|
||||
"widgets_values": [
|
||||
"Typed pair positive",
|
||||
"Primitive pair negative",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
""
|
||||
],
|
||||
"widgets_values_named": {
|
||||
"text": "Typed pair positive",
|
||||
"negative_text": "Primitive pair negative",
|
||||
"category": "",
|
||||
"tags": "",
|
||||
"search_text": "",
|
||||
"prepend_text": "",
|
||||
"append_text": "",
|
||||
"prompt_manager_focused_ui": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"type": "PrimitiveNode",
|
||||
"pos": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"size": [
|
||||
210,
|
||||
88
|
||||
],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "STRING",
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "text"
|
||||
},
|
||||
"links": [
|
||||
1
|
||||
]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Run widget replace on values": false
|
||||
},
|
||||
"widgets_values": [
|
||||
"Primitive focused text"
|
||||
],
|
||||
"widgets_values_named": {
|
||||
"text": "Primitive focused text"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"type": "PrimitiveNode",
|
||||
"pos": [
|
||||
0,
|
||||
1200
|
||||
],
|
||||
"size": [
|
||||
210,
|
||||
88
|
||||
],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "STRING",
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "negative_text"
|
||||
},
|
||||
"links": [
|
||||
2
|
||||
]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Run widget replace on values": false
|
||||
},
|
||||
"widgets_values": [
|
||||
"Primitive pair negative"
|
||||
],
|
||||
"widgets_values_named": {
|
||||
"negative_text": "Primitive pair negative"
|
||||
}
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
[
|
||||
1,
|
||||
3,
|
||||
0,
|
||||
1,
|
||||
3,
|
||||
"STRING"
|
||||
],
|
||||
[
|
||||
2,
|
||||
4,
|
||||
0,
|
||||
2,
|
||||
4,
|
||||
"STRING"
|
||||
]
|
||||
],
|
||||
"groups": [],
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ds": {
|
||||
"scale": 0.9,
|
||||
"offset": [
|
||||
416,
|
||||
110
|
||||
]
|
||||
}
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,142 @@
|
||||
// #342 on the real pinned host: the image monitor links an output only to a prompt
|
||||
// that provably produced it. An unknown node (PMHistoryCached) stands in for the
|
||||
// maintainer's LLM; KSampler is the fixture's deterministic stand-in.
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { execFile } from 'node:child_process';
|
||||
import { promisify } from 'node:util';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
|
||||
import { test } from './execution-fixture.mjs';
|
||||
|
||||
const run = promisify(execFile);
|
||||
const reader = fileURLToPath(new URL('./read_execution_history.py', import.meta.url));
|
||||
|
||||
async function readLinks(databasePath, filenames, texts = []) {
|
||||
const { stdout } = await run(process.env.PM_TEST_PYTHON || 'python',
|
||||
[reader, databasePath, '--links', ...filenames, '--texts', ...texts], { timeout: 5000, maxBuffer: 1024 * 1024 });
|
||||
return JSON.parse(stdout);
|
||||
}
|
||||
|
||||
// kind: the positive node; text: a typed editor or 'llm' (linked from the unknown
|
||||
// node); prepend: 'llm' links the unknown node into prepend_text.
|
||||
function graph({ kind = 'PromptManagerPair', text = '', prepend = '', llm = '', seed = 1, prefix }) {
|
||||
const linked = value => value === 'llm' ? ['11', 0] : value;
|
||||
const nodes = {
|
||||
'10': { class_type: 'PMHistoryInputs', inputs: {} },
|
||||
'20': { class_type: kind, inputs: { text: linked(text), clip: ['10', 0], append_text: '',
|
||||
...(prepend ? { prepend_text: linked(prepend) } : {}),
|
||||
...(kind === 'PromptManagerPair' ? { negative_text: '' } : {}) } },
|
||||
'30': { class_type: 'CLIPTextEncode', inputs: { text: '', clip: ['10', 0] } },
|
||||
'40': { class_type: 'KSampler', inputs: { model: ['10', 1], latent_image: ['10', 2],
|
||||
positive: ['20', 0], negative: kind === 'PromptManagerPair' ? ['20', 1] : ['30', 0],
|
||||
seed, steps: 1, cfg: 1, sampler_name: 'euler', scheduler: 'normal', denoise: 1 } },
|
||||
'60': { class_type: 'PMHistoryImages', inputs: { value: 64, batch: 1, latent: ['40', 0] } },
|
||||
'70': { class_type: 'SaveImage', inputs: { images: ['60', 0], filename_prefix: prefix } },
|
||||
};
|
||||
if (llm) nodes['11'] = { class_type: 'PMHistoryCached', inputs: { value: llm } };
|
||||
return nodes;
|
||||
}
|
||||
|
||||
async function saved(request, prompt, history = {}) {
|
||||
const id = randomUUID();
|
||||
const response = await request.post('/prompt', { data: { prompt, prompt_id: id } });
|
||||
expect(response.ok()).toBe(true);
|
||||
let item;
|
||||
await expect.poll(async () => {
|
||||
item = (await (await request.get(`/history/${id}`)).json())[id];
|
||||
return item?.status?.messages?.some(([kind]) => kind === 'execution_success') || false;
|
||||
}, { timeout: 15000 }).toBe(true);
|
||||
const [image] = item.outputs['70'].images;
|
||||
expect(image.type).toBe('output');
|
||||
history.item = item;
|
||||
return image.filename;
|
||||
}
|
||||
|
||||
async function linkedTo(execution, filename, expected) {
|
||||
await expect.poll(async () => (await readLinks(execution.databasePath, [filename])).links[filename],
|
||||
{ timeout: 20000, message: `${filename} is linked by the image monitor` }).toBe(expected);
|
||||
}
|
||||
|
||||
async function monitoring(execution, enabled) {
|
||||
const response = await execution.request.post('/pm_execution_fixture/monitor', { data: { enabled } });
|
||||
expect(response.ok()).toBe(true);
|
||||
expect((await response.json()).monitoring).toBe(enabled);
|
||||
}
|
||||
|
||||
async function cacheMode(execution, mode) {
|
||||
expect((await execution.request.post('/pm_execution_fixture/cache', { data: { mode } })).ok()).toBe(true);
|
||||
}
|
||||
|
||||
test.beforeEach(async ({ execution }) => {
|
||||
expect((await execution.request.post('/pm_execution_fixture/metadata', { data: { disabled: false } })).ok()).toBe(true);
|
||||
await cacheMode(execution, 'CLASSIC'); // keeps the re-run's PromptManager node cached
|
||||
await monitoring(execution, true);
|
||||
});
|
||||
test.afterEach(async ({ execution }) => {
|
||||
await monitoring(execution, false);
|
||||
await cacheMode(execution, 'RAM_PRESSURE');
|
||||
});
|
||||
|
||||
test('an output whose positive is only linked prepend text stays unlinked, never given an earlier prompt', async ({ execution }, testInfo) => {
|
||||
const tag = `${testInfo.project.name} ${randomUUID()}`;
|
||||
const earlier = `a woman holding a coffee cup ${tag}`;
|
||||
const prepended = `llm prepend landscape ${tag}`, llmText = `llm text landscape ${tag}`;
|
||||
|
||||
// A typed prompt runs first: before #342 the next unidentifiable image took it.
|
||||
const first = await saved(execution.request, graph({ text: earlier, prefix: 'misslink/earlier' }));
|
||||
await linkedTo(execution, first, earlier);
|
||||
|
||||
// The maintainer's workflow: empty editor, the generated text arrives in prepend.
|
||||
const orphan = await saved(execution.request,
|
||||
graph({ prepend: 'llm', llm: prepended, prefix: 'misslink/prepend' }));
|
||||
|
||||
// An LLM feeding the editor itself is stored and links through its run key; the
|
||||
// monitor works in creation order, so the prepend image was processed first.
|
||||
const llmGraph = graph({ kind: 'PromptManager', text: 'llm', llm: llmText, prefix: 'misslink/text' });
|
||||
const proven = await saved(execution.request, llmGraph);
|
||||
await linkedTo(execution, proven, llmText);
|
||||
const state = await readLinks(execution.databasePath, [orphan], [prepended]);
|
||||
expect(state).toEqual({ links: { [orphan]: null }, stored: { [prepended]: false } });
|
||||
|
||||
// Only the sampler seed changes: the PromptManager node is served from the cache
|
||||
// without running (the host says so), so no queue entry exists for this run and
|
||||
// only the snapshot's run match can link it.
|
||||
const rerunHistory = {};
|
||||
const rerun = await saved(execution.request, { ...llmGraph, '40': { ...llmGraph['40'],
|
||||
inputs: { ...llmGraph['40'].inputs, seed: 2 } } }, rerunHistory);
|
||||
const cached = rerunHistory.item.status.messages.find(([kind]) => kind === 'execution_cached');
|
||||
expect(cached?.[1].nodes).toEqual(expect.arrayContaining(['11', '20']));
|
||||
await linkedTo(execution, rerun, llmText);
|
||||
});
|
||||
|
||||
test('an output saved without metadata links once its recorded run names a stored prompt', async ({ execution }, testInfo) => {
|
||||
const llmText = `bare output landscape ${testInfo.project.name} ${randomUUID()}`;
|
||||
const request = execution.request;
|
||||
expect((await request.post('/pm_execution_fixture/metadata', { data: { disabled: true } })).ok()).toBe(true);
|
||||
try {
|
||||
// No embedded graph: no run key can prove the tracker entry, so only the
|
||||
// run's recorded history can, and only once the run's outputs are sealed.
|
||||
// The gate after SaveImage holds the run open while the monitor looks.
|
||||
const prompt = graph({ kind: 'PromptManager', text: 'llm', llm: llmText, prefix: 'misslink/bare' });
|
||||
prompt['80'] = { class_type: 'PMHistoryImageOutput', inputs: { images: ['70', 0], mode: 'gate' } };
|
||||
expect((await request.post('/pm_execution_fixture/gate/hold')).ok()).toBe(true);
|
||||
const id = randomUUID();
|
||||
try {
|
||||
expect((await request.post('/prompt', { data: { prompt, prompt_id: id } })).ok()).toBe(true);
|
||||
let waiting = [];
|
||||
await expect.poll(async () => {
|
||||
waiting = (await (await request.get('/pm_execution_fixture/monitor')).json()).awaiting;
|
||||
return waiting.filter(name => name.startsWith('bare_')).length;
|
||||
}, { timeout: 20000, message: 'the monitor processed the bare output while its run was open' }).toBe(1);
|
||||
const [bare] = waiting.filter(name => name.startsWith('bare_'));
|
||||
expect((await readLinks(execution.databasePath, [bare])).links[bare]).toBeNull();
|
||||
expect((await request.post('/pm_execution_fixture/gate/release')).ok()).toBe(true);
|
||||
await linkedTo(execution, bare, llmText);
|
||||
expect((await (await request.get('/pm_execution_fixture/monitor')).json()).awaiting).not.toContain(bare);
|
||||
} finally {
|
||||
expect((await request.post('/pm_execution_fixture/gate/release')).ok()).toBe(true);
|
||||
}
|
||||
} finally {
|
||||
expect((await request.post('/pm_execution_fixture/metadata', { data: { disabled: false } })).ok()).toBe(true);
|
||||
}
|
||||
});
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Bounded read-only SQLite inspection for the owned execution fixture."""
|
||||
|
||||
import argparse
|
||||
from contextlib import closing
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sqlite3
|
||||
@@ -89,14 +90,48 @@ def read_runs(database: str, job_id: str, session_id: str | None = None) -> dict
|
||||
}
|
||||
|
||||
|
||||
def read_links(database: str, filenames: list, texts: list) -> dict:
|
||||
"""Each output file's linked prompt text (None: unlinked), and which texts
|
||||
are stored as prompts (#342)."""
|
||||
uri = Path(database).resolve().as_uri() + "?mode=ro"
|
||||
with closing(sqlite3.connect(uri, uri=True, timeout=0.25)) as connection:
|
||||
connection.execute("PRAGMA query_only=ON")
|
||||
links = {}
|
||||
for name in filenames[:64]:
|
||||
rows = connection.execute(
|
||||
"SELECT p.text FROM generated_images g JOIN prompts p "
|
||||
"ON p.id = g.prompt_id WHERE g.filename = ? LIMIT 2",
|
||||
(name,),
|
||||
).fetchall()
|
||||
values = [row[0] for row in rows]
|
||||
links[name] = values[0] if len(values) == 1 else None
|
||||
if len(values) > 1:
|
||||
links[name] = "<several links>"
|
||||
stored = {
|
||||
text: connection.execute(
|
||||
"SELECT 1 FROM prompts WHERE text = ? LIMIT 1", (text,)
|
||||
).fetchone()
|
||||
is not None
|
||||
for text in texts[:64]
|
||||
}
|
||||
return {"links": links, "stored": stored}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("database")
|
||||
parser.add_argument("job_id")
|
||||
parser.add_argument("job_id", nargs="?")
|
||||
parser.add_argument("--session-id")
|
||||
parser.add_argument("--links", nargs="*", default=None)
|
||||
parser.add_argument("--texts", nargs="*", default=[])
|
||||
arguments = parser.parse_args()
|
||||
print(
|
||||
json.dumps(
|
||||
read_runs(arguments.database, arguments.job_id, arguments.session_id)
|
||||
if arguments.links is not None:
|
||||
print(
|
||||
json.dumps(read_links(arguments.database, arguments.links, arguments.texts))
|
||||
)
|
||||
else:
|
||||
print(
|
||||
json.dumps(
|
||||
read_runs(arguments.database, arguments.job_id, arguments.session_id)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
@@ -0,0 +1,404 @@
|
||||
// #343: on the pinned real host, a STRING link dragged with the mouse drives each PromptManager node's text (and the
|
||||
// Pair's negative) in Classic and Nodes 2.0, across renderer switches, a workflow reload, disconnect and removal.
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
|
||||
import { installHostHooks } from './host-fixture.mjs';
|
||||
|
||||
installHostHooks(test, ['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair']);
|
||||
|
||||
const editors = { PromptManagerFocused: 'Main prompt', PromptManagerSplit: 'Main prompt', PromptManagerPair: 'Positive prompt' };
|
||||
const note = label => `${label} comes from a link. Your typed text is kept for when you disconnect it.`;
|
||||
|
||||
/** Viewport position of a node's slot: the slot dot in Nodes 2.0, the canvas slot position in Classic. */
|
||||
function slotPosition(page, nodeId, kind, name) {
|
||||
return page.evaluate(async ({ nodeId, kind, name }) => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const node = app.graph.getNodeById(nodeId);
|
||||
const slots = kind === 'out' ? node.outputs : node.inputs;
|
||||
const index = typeof name === 'number' ? name : slots.findIndex(slot => slot.name === name);
|
||||
if (LiteGraph.vueNodesMode) {
|
||||
const dot = document.querySelector(`[data-slot-key="${nodeId}-${kind}-${index}"]`);
|
||||
if (!dot) return null;
|
||||
const box = dot.getBoundingClientRect();
|
||||
return [box.x + box.width / 2, box.y + box.height / 2];
|
||||
}
|
||||
const position = app.canvas.ds.convertOffsetToCanvas(kind === 'out' ? node.getOutputPos(index) : node.getInputPos(index));
|
||||
const box = app.canvas.canvas.getBoundingClientRect();
|
||||
return [box.x + position[0], box.y + position[1]];
|
||||
}, { nodeId, kind, name });
|
||||
}
|
||||
|
||||
async function drag(page, from, to) {
|
||||
await page.mouse.move(...from);
|
||||
await page.mouse.down();
|
||||
await page.mouse.move((from[0] + to[0]) / 2, (from[1] + to[1]) / 2, { steps: 5 });
|
||||
await page.mouse.move(...to, { steps: 10 });
|
||||
await page.mouse.up();
|
||||
}
|
||||
|
||||
const center = box => [box.x + box.width / 2, box.y + box.height / 2];
|
||||
const linkedInputs = (page, id) => page.evaluate(async id => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
return app.graph.getNodeById(id).inputs.filter(input => input.link != null).map(input => input.name);
|
||||
}, id);
|
||||
const queued = (page, id) => page.evaluate(async id => (await (await import('/scripts/app.js')).app.graphToPrompt()).output[String(id)].inputs, id);
|
||||
|
||||
async function setup(page, type) {
|
||||
return page.evaluate(async type => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const node = LiteGraph.createNode(type);
|
||||
app.graph.add(node);
|
||||
node.pos = [500, 80];
|
||||
node.setSize([type === 'PromptManagerPair' ? 1100 : 650, 760]);
|
||||
const sources = ['text', 'negative_text'].map((name, index) => {
|
||||
const source = LiteGraph.createNode('PrimitiveString');
|
||||
app.graph.add(source);
|
||||
source.pos = [40, 80 + index * 160];
|
||||
source.widgets.find(widget => widget.name === 'value').value = `Linked ${name} ${type}`;
|
||||
return source.id;
|
||||
});
|
||||
app.canvas.ds.scale = .8;
|
||||
app.canvas.ds.offset = [40, 40];
|
||||
app.canvas.setDirty(true, true);
|
||||
return { id: node.id, sources };
|
||||
}, type);
|
||||
}
|
||||
|
||||
/** The prompt box a user drops the link on: the editor's textarea, or the core text box of the Classic nodes. */
|
||||
async function promptBox(page, type, id) {
|
||||
if (editors[type]) return page.getByRole('textbox', { name: editors[type], exact: true });
|
||||
// The Classic nodes keep the core text widget: Nodes 2.0 draws it as its own textarea, Classic as widget.inputEl.
|
||||
if (await page.evaluate(() => LiteGraph.vueNodesMode)) return page.locator(`[data-node-id="${id}"] textarea`).first();
|
||||
await page.evaluate(async id => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
app.graph.getNodeById(id).widgets.find(widget => widget.name === 'text').inputEl.dataset.pmTestPromptBox = String(id);
|
||||
}, id);
|
||||
return page.locator(`textarea[data-pm-test-prompt-box="${id}"]`);
|
||||
}
|
||||
|
||||
/** What the node shows while its text is linked. */
|
||||
function linkedNote(page, type) {
|
||||
return editors[type] ? page.getByText(note(editors[type]), { exact: true }) : page.locator('.pm-text-link-note').getByText(note('Text'), { exact: true });
|
||||
}
|
||||
|
||||
for (const type of ['PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair', 'PromptManager', 'PromptManagerText']) {
|
||||
test(`${type}: a STRING link dropped on the prompt box drives text and the typed text returns on disconnect`, async ({ page }, testInfo) => {
|
||||
test.setTimeout(90000);
|
||||
const { id, sources } = await setup(page, type);
|
||||
const typed = `Typed ${type} ${testInfo.project.name}`;
|
||||
const box = await promptBox(page, type, id);
|
||||
await box.fill(typed);
|
||||
await expect(linkedNote(page, type)).toBeHidden();
|
||||
await page.screenshot({ path: testInfo.outputPath(`${type}-unlinked.png`) });
|
||||
// The drop lands on text, never on prepend_text (#342).
|
||||
await drag(page, await slotPosition(page, sources[0], 'out', 0), center(await box.boundingBox()));
|
||||
await expect.poll(() => linkedInputs(page, id)).toEqual(['text']);
|
||||
const links = { text: [String(sources[0]), 0] };
|
||||
if (type === 'PromptManagerPair') {
|
||||
await page.getByRole('textbox', { name: 'Negative prompt', exact: true }).fill(`Typed negative ${testInfo.project.name}`);
|
||||
await drag(page, await slotPosition(page, sources[1], 'out', 0), await slotPosition(page, id, 'in', 'negative_text'));
|
||||
await expect.poll(() => linkedInputs(page, id)).toEqual(['text', 'negative_text']);
|
||||
links.negative_text = [String(sources[1]), 0];
|
||||
await expect(page.getByText(note('Negative prompt'), { exact: true })).toBeVisible();
|
||||
}
|
||||
await expect(linkedNote(page, type)).toBeVisible();
|
||||
expect(await queued(page, id)).toMatchObject(links);
|
||||
await page.screenshot({ path: testInfo.outputPath(`${type}-linked.png`) });
|
||||
for (const renderer of [!testInfo.project.metadata.initialRenderer, testInfo.project.metadata.initialRenderer]) {
|
||||
await page.evaluate(async renderer => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
await app.ui.settings.setSettingValue('Comfy.VueNodes.Enabled', renderer);
|
||||
}, renderer);
|
||||
await expect.poll(() => page.evaluate(() => LiteGraph.vueNodesMode)).toBe(renderer);
|
||||
await expect(linkedNote(page, type)).toBeVisible();
|
||||
expect(await queued(page, id)).toMatchObject(links);
|
||||
}
|
||||
const saved = await page.evaluate(async id => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const workflow = app.graph.serialize();
|
||||
await app.loadGraphData(workflow);
|
||||
return workflow.nodes.find(node => String(node.id) === String(id)).widgets_values;
|
||||
}, id);
|
||||
expect(saved[0]).toBe(typed);
|
||||
await expect(linkedNote(page, type)).toBeVisible();
|
||||
expect(await queued(page, id)).toMatchObject(links);
|
||||
const local = await page.evaluate(async id => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const node = app.graph.getNodeById(id);
|
||||
for (const input of [...node.inputs]) if (input.link != null) node.disconnectInput(node.inputs.indexOf(input));
|
||||
return (await app.graphToPrompt()).output[String(id)].inputs;
|
||||
}, id);
|
||||
expect(local.text).toBe(typed);
|
||||
if (type === 'PromptManagerPair') expect(local.negative_text).toBe(`Typed negative ${testInfo.project.name}`);
|
||||
await expect(linkedNote(page, type)).toBeHidden();
|
||||
const restored = await promptBox(page, type, id);
|
||||
await expect(restored).toBeVisible();
|
||||
await expect(restored).toHaveValue(typed);
|
||||
await page.screenshot({ path: testInfo.outputPath(`${type}-disconnected.png`) });
|
||||
await page.evaluate(async id => { const { app } = await import('/scripts/app.js'); app.graph.remove(app.graph.getNodeById(id)); }, id);
|
||||
await expect(page.locator('.pm-focused, .pm-classic, .pm-text-link-note')).toHaveCount(0);
|
||||
});
|
||||
}
|
||||
|
||||
async function addEditors(page) {
|
||||
return page.evaluate(async () => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const add = (type, x, y) => { const node = LiteGraph.createNode(type); app.graph.add(node); node.pos = [x, y]; return node; };
|
||||
const focused = add('PromptManagerFocused', 420, 40);
|
||||
focused.setSize([560, 640]);
|
||||
const pair = add('PromptManagerPair', 1020, 40);
|
||||
pair.setSize([1000, 640]);
|
||||
for (const [node, name, value] of [[focused, 'text', 'Typed focused'], [pair, 'text', 'Typed positive'], [pair, 'negative_text', 'Typed negative']]) {
|
||||
const widget = node.widgets.find(item => item.name === name);
|
||||
widget.value = value;
|
||||
if (widget.inputEl) widget.inputEl.value = value;
|
||||
}
|
||||
const sources = [0, 1].map(index => add(index ? 'PrimitiveString' : 'PrimitiveNode', 20, 60 + index * 300).id);
|
||||
app.canvas.ds.scale = .7;
|
||||
// Clear of the floating graph menu (top left) and the PM run bar (top right).
|
||||
app.canvas.ds.offset = [110, 150];
|
||||
app.canvas.setDirty(true, true);
|
||||
return { focused: focused.id, pair: pair.id, primitive: sources[0], string: sources[1] };
|
||||
});
|
||||
}
|
||||
|
||||
/** The prompt a Run sends to the server, through the real app.queuePrompt path. */
|
||||
async function runPrompt(page) {
|
||||
let captured;
|
||||
const sent = new Promise(resolve => { captured = resolve; });
|
||||
// Installed before the run starts, so the POST can never reach the real host.
|
||||
await page.route('**/prompt', async route => {
|
||||
if (route.request().method() !== 'POST') return route.fallback();
|
||||
captured(route.request().postDataJSON().prompt);
|
||||
return route.fulfill({ json: { prompt_id: 'captured', number: 1, node_errors: {} } });
|
||||
});
|
||||
await page.evaluate(async () => { const { app } = await import('/scripts/app.js'); await app.queuePrompt(0, 1); });
|
||||
const prompt = await sent;
|
||||
await page.unroute('**/prompt');
|
||||
return prompt;
|
||||
}
|
||||
|
||||
const box = (page, name, nth = 0) => page.getByRole('textbox', { name, exact: true }).nth(nth);
|
||||
|
||||
test('a legacy Primitive dropped on an editor box or socket drives the text it shows as linked (#343 review I1, M1)', async ({ page }) => {
|
||||
test.setTimeout(90000);
|
||||
const ids = await addEditors(page);
|
||||
// A wildcard Primitive aimed at the Main prompt box, a PrimitiveString aimed at the Negative prompt box.
|
||||
await drag(page, await slotPosition(page, ids.primitive, 'out', 0), center(await box(page, 'Main prompt').boundingBox()));
|
||||
await expect.poll(() => linkedInputs(page, ids.focused)).toEqual(['text']);
|
||||
await drag(page, await slotPosition(page, ids.string, 'out', 0), center(await box(page, 'Negative prompt').boundingBox()));
|
||||
await expect.poll(() => linkedInputs(page, ids.pair)).toEqual(['negative_text']);
|
||||
await expect(page.getByText(note('Main prompt'), { exact: true })).toBeVisible();
|
||||
await expect(page.getByText(note('Negative prompt'), { exact: true })).toBeVisible();
|
||||
await page.evaluate(async ({ primitive, string }) => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
app.graph.getNodeById(primitive).widgets[0].value = 'From the legacy Primitive';
|
||||
app.graph.getNodeById(string).widgets.find(widget => widget.name === 'value').value = 'From PrimitiveString';
|
||||
}, ids);
|
||||
const prompt = await runPrompt(page);
|
||||
expect(prompt[String(ids.focused)].inputs.text).toBe('From the legacy Primitive');
|
||||
expect(prompt[String(ids.pair)].inputs).toMatchObject({ text: 'Typed positive', negative_text: [String(ids.string), 0] });
|
||||
// The Primitive is re-read on every run; the typed text stays for disconnect.
|
||||
await page.evaluate(async ({ primitive }) => { const { app } = await import('/scripts/app.js'); app.graph.getNodeById(primitive).widgets[0].value = 'Changed in the Primitive'; }, ids);
|
||||
expect((await queued(page, ids.focused)).text).toBe('Changed in the Primitive');
|
||||
await page.evaluate(async ({ focused }) => { const { app } = await import('/scripts/app.js'); const node = app.graph.getNodeById(focused); node.disconnectInput(node.inputs.findIndex(input => input.name === 'text')); }, ids);
|
||||
expect((await queued(page, ids.focused)).text).toBe('Typed focused');
|
||||
});
|
||||
|
||||
test('a wildcard legacy Primitive queues one resolved choice, drawn once per run, on every PromptManager node (#343 re-review R1)', async ({ page }) => {
|
||||
const choices = `{${Array.from({ length: 10 }, (_, index) => `c${index}`).join('|')}}`;
|
||||
const ids = await page.evaluate(async choices => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const add = (type, x, y) => { const node = LiteGraph.createNode(type); app.graph.add(node); node.pos = [x, y]; return node; };
|
||||
// [node, input, Primitive value, its "Run widget replace on values" option]
|
||||
const targets = [['PromptManagerFocused', 'text', `Focused ${choices}`, false], ['PromptManagerSplit', 'text', `Split %date:yyyy% ${choices}`, true],
|
||||
['PromptManagerPair', 'negative_text', `Pair ${choices}`, false], ['PromptManager', 'text', `Classic ${choices}`, false],
|
||||
['PromptManagerText', 'text', `Text ${choices}`, false]];
|
||||
const ids = {};
|
||||
targets.forEach(([type, name, value, replace], index) => {
|
||||
const node = add(type, 500, index * 800);
|
||||
const primitive = add('PrimitiveNode', 20, index * 800);
|
||||
if (!primitive.connect(0, node, node.inputs.findIndex(input => input.name === name))) throw new Error(`no Primitive link to ${type}.${name}`);
|
||||
primitive.widgets[0].value = value;
|
||||
primitive.properties['Run widget replace on values'] = replace;
|
||||
ids[type] = [String(node.id), name];
|
||||
});
|
||||
// The Classic nodes install their own text serializer shortly after creation; run after it, as a user does.
|
||||
await Promise.all(app.graph.nodes.map(node => node._pmTextSerializerReady));
|
||||
// What another extension's graphToPrompt wrapper sees must be what the server gets.
|
||||
const inner = app.graphToPrompt;
|
||||
const seen = window.__pmSeen = [];
|
||||
app.graphToPrompt = async function (...args) { const result = await inner.apply(this, args); seen.push(structuredClone(result.output)); return result; };
|
||||
// Each draw takes the next choice, so a second draw for the same text always changes it.
|
||||
const random = Math.random;
|
||||
let draw = 0;
|
||||
Math.random = () => ((draw++ % 10) + .5) / 10;
|
||||
window.__pmRestore = () => { app.graphToPrompt = inner; Math.random = random; };
|
||||
return ids;
|
||||
}, choices);
|
||||
const prompt = await runPrompt(page);
|
||||
const seen = await page.evaluate(() => { window.__pmRestore(); return window.__pmSeen; });
|
||||
expect(seen).toHaveLength(1);
|
||||
const year = String(new Date().getFullYear());
|
||||
const stems = { PromptManagerFocused: 'Focused', PromptManagerSplit: `Split ${year}`, PromptManagerPair: 'Pair', PromptManager: 'Classic', PromptManagerText: 'Text' };
|
||||
// The Classic nodes keep their widget binding, so the host applies the Primitive and #343 leaves the value alone.
|
||||
// As on main, their text widget's own serializer queues the text as written, wildcard included.
|
||||
const classic = new Set(['PromptManager', 'PromptManagerText']);
|
||||
for (const [type, [id, name]] of Object.entries(ids)) {
|
||||
const value = prompt[id].inputs[name];
|
||||
if (classic.has(type)) expect(value, type).toBe(`${stems[type]} ${choices}`);
|
||||
else expect(value, type).toMatch(new RegExp(`^${stems[type]} c\\d$`));
|
||||
expect(value, `${type}: one draw per run`).toBe(seen[0][id].inputs[name]);
|
||||
}
|
||||
});
|
||||
|
||||
test('a legacy Primitive inside a subgraph drives the inner editor text it shows as linked (#343 re-review R2)', async ({ page }) => {
|
||||
const result = await page.evaluate(async () => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const add = (type, x, y) => { const node = LiteGraph.createNode(type); app.graph.add(node); node.pos = [x, y]; return node; };
|
||||
const focused = add('PromptManagerFocused', 500, 40);
|
||||
focused.widgets.find(widget => widget.name === 'text').value = 'Typed under the link';
|
||||
const primitive = add('PrimitiveNode', 20, 40);
|
||||
if (!primitive.connect(0, focused, focused.inputs.findIndex(input => input.name === 'text'))) throw new Error('no Primitive link');
|
||||
const { node } = app.graph.convertToSubgraph(new Set([focused, primitive]));
|
||||
const inner = node.subgraph.nodes.find(item => item.type === 'PromptManagerFocused');
|
||||
// The value the user gives the Primitive inside the subgraph.
|
||||
node.subgraph.nodes.find(item => item.type === 'PrimitiveNode').widgets[0].value = 'Inner {only choice}';
|
||||
return { host: String(node.id), inner: String(inner.id), linked: inner.inputs.find(input => input.name === 'text').link != null };
|
||||
});
|
||||
expect(result.linked).toBe(true);
|
||||
const prompt = await runPrompt(page);
|
||||
expect(prompt[`${result.host}:${result.inner}`]?.inputs.text).toBe('Inner only choice');
|
||||
});
|
||||
|
||||
test('a workflow saved before #343 with legacy Primitives on editor text keeps running their values (#343 review I1)', async ({ page }) => {
|
||||
const workflow = JSON.parse(await readFile(new URL('./fixtures/workflow-before-343-primitive.json', import.meta.url), 'utf8'));
|
||||
const result = await page.evaluate(async workflow => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
await app.loadGraphData(structuredClone(workflow));
|
||||
const before = (await app.graphToPrompt()).output;
|
||||
app.graph.getNodeById(4).widgets[0].value = 'Edited after the load';
|
||||
const after = (await app.graphToPrompt()).output;
|
||||
const bound = id => app.graph.getNodeById(id).inputs.filter(input => ['text', 'negative_text'].includes(input.name) && input.widget).length;
|
||||
return { before: [before['1'].inputs.text, before['2'].inputs.text, before['2'].inputs.negative_text], after: after['2'].inputs.negative_text, bound: [bound(1), bound(2)],
|
||||
links: [app.graph.getNodeById(1).inputs.find(input => input.name === 'text').link != null, app.graph.getNodeById(2).inputs.find(input => input.name === 'negative_text').link != null] };
|
||||
}, workflow);
|
||||
expect(result).toEqual({ before: ['Primitive focused text', 'Typed pair positive', 'Primitive pair negative'], after: 'Edited after the load', bound: [0, 0], links: [true, true] });
|
||||
const prompt = await runPrompt(page);
|
||||
expect([prompt['1'].inputs.text, prompt['2'].inputs.negative_text]).toEqual(['Primitive focused text', 'Edited after the load']);
|
||||
});
|
||||
|
||||
test('moving a linked text and dropping it on the editor body keeps it on text, never prepend_text (#343 review I2)', async ({ page }) => {
|
||||
test.setTimeout(90000);
|
||||
const ids = await addEditors(page);
|
||||
await page.evaluate(async ({ focused, pair, string }) => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const source = app.graph.getNodeById(string);
|
||||
for (const id of [focused, pair]) { const node = app.graph.getNodeById(id); source.connect(0, node, node.inputs.findIndex(input => input.name === 'text')); }
|
||||
app.canvas.setDirty(true, true);
|
||||
}, ids);
|
||||
for (const [id, title] of [[ids.focused, 0], [ids.pair, 1]]) {
|
||||
// Grab the link at the text socket and release it over the editor's toolbar, away from any prompt box.
|
||||
const toolbar = page.locator('.pm-editor-toolbar strong').nth(title);
|
||||
await drag(page, await slotPosition(page, id, 'in', 'text'), center(await toolbar.boundingBox()));
|
||||
await expect.poll(() => linkedInputs(page, id)).toEqual(['text']);
|
||||
}
|
||||
});
|
||||
|
||||
test('paste, clone and undo keep the editor text a linked socket (#343 review M7)', async ({ page }) => {
|
||||
const ids = await addEditors(page);
|
||||
const state = await page.evaluate(async ({ pair, string }) => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const node = app.graph.getNodeById(pair);
|
||||
app.graph.getNodeById(string).connect(0, node, node.inputs.findIndex(input => input.name === 'text'));
|
||||
const describe = target => ({ bound: target.inputs.filter(input => ['text', 'negative_text'].includes(input.name) && input.widget).length,
|
||||
linked: target.inputs.find(input => input.name === 'text').link != null });
|
||||
app.canvas.selectItems([node]);
|
||||
app.canvas.copyToClipboard();
|
||||
app.canvas.pasteFromClipboard({ connectInputs: true });
|
||||
const pasted = app.graph._nodes.find(item => item.type === 'PromptManagerPair' && item.id !== pair);
|
||||
const clone = node.clone();
|
||||
app.graph.add(clone);
|
||||
const output = (await app.graphToPrompt()).output;
|
||||
return { pasted: describe(pasted), clone: describe(clone), queued: output[String(pasted.id)].inputs.text, pastedId: pasted.id };
|
||||
}, ids);
|
||||
expect(state).toMatchObject({ pasted: { bound: 0, linked: true }, clone: { bound: 0, linked: false }, queued: [String(ids.string), 0] });
|
||||
// Undo restores the graph through a workflow load: the restored link is still a socket link.
|
||||
const undone = await page.evaluate(async ({ pair }) => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const tracker = app.extensionManager?.workflow?.activeWorkflow?.changeTracker;
|
||||
if (!tracker) throw new Error('no change tracker on the active workflow');
|
||||
tracker.checkState();
|
||||
const node = app.graph.getNodeById(pair);
|
||||
node.disconnectInput(node.inputs.findIndex(input => input.name === 'text'));
|
||||
tracker.checkState();
|
||||
await tracker.undo();
|
||||
const restored = app.graph.getNodeById(pair);
|
||||
return { bound: restored.inputs.filter(input => ['text', 'negative_text'].includes(input.name) && input.widget).length,
|
||||
text: (await app.graphToPrompt()).output[String(pair)].inputs.text };
|
||||
}, ids);
|
||||
expect(undone).toEqual({ bound: 0, text: [String(ids.string), 0] });
|
||||
});
|
||||
|
||||
test('PromptSearchList text is a core widget socket and takes a dragged STRING link too', async ({ page }) => {
|
||||
const { id, sources } = await page.evaluate(async () => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
const node = LiteGraph.createNode('PromptSearchList');
|
||||
app.graph.add(node);
|
||||
node.pos = [500, 80];
|
||||
const source = LiteGraph.createNode('PrimitiveString');
|
||||
app.graph.add(source);
|
||||
source.pos = [40, 80];
|
||||
app.canvas.ds.scale = .8;
|
||||
app.canvas.ds.offset = [40, 40];
|
||||
app.canvas.setDirty(true, true);
|
||||
return { id: node.id, sources: [source.id] };
|
||||
});
|
||||
// A core widget's socket shows while a link is dragged; aim at where the frontend places it.
|
||||
await expect.poll(async () => Boolean(await slotPosition(page, id, 'in', 'text'))).toBe(true);
|
||||
await drag(page, await slotPosition(page, sources[0], 'out', 0), await slotPosition(page, id, 'in', 'text'));
|
||||
await expect.poll(() => linkedInputs(page, id)).toEqual(['text']);
|
||||
expect((await queued(page, id)).text).toEqual([String(sources[0]), 0]);
|
||||
});
|
||||
|
||||
test('workflows saved before #343 load unchanged and their linked text keeps working', async ({ page }) => {
|
||||
const workflow = JSON.parse(await readFile(new URL('./fixtures/workflow-before-343.json', import.meta.url), 'utf8'));
|
||||
const result = await page.evaluate(async workflow => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
await app.loadGraphData(structuredClone(workflow));
|
||||
const output = (await app.graphToPrompt()).output;
|
||||
const nodes = Object.fromEntries(app.graph._nodes.map(node => [node.id, node]));
|
||||
const reloaded = app.graph.serialize();
|
||||
return {
|
||||
values: Object.fromEntries(reloaded.nodes.map(node => [node.id, node.widgets_values])),
|
||||
links: Object.fromEntries(Object.entries(output).map(([id, node]) => [id, { text: node.inputs.text, negative_text: node.inputs.negative_text }])),
|
||||
widgetBound: Object.fromEntries(Object.values(nodes).filter(node => node.type.startsWith('PromptManager')).map(node => [node.id, node.inputs.filter(input => ['text', 'negative_text'].includes(input.name) && input.widget).map(input => input.name)])),
|
||||
};
|
||||
}, workflow);
|
||||
for (const node of workflow.nodes) {
|
||||
// Every saved widget value comes back as it was saved, except the typed text main lost under a link (null).
|
||||
if (node.widgets_values?.[0] !== null) expect(result.values[node.id], `${node.type} ${node.id}`).toEqual(node.widgets_values);
|
||||
}
|
||||
expect(result.links).toMatchObject({
|
||||
1: { text: 'Saved PromptManager text' }, 2: { text: 'Saved PromptManagerText text' }, 3: { text: 'Saved PromptManagerFocused text' },
|
||||
4: { text: 'Saved PromptManagerSplit text' }, 5: { text: 'Saved PromptManagerPair text', negative_text: 'Saved pair negative' },
|
||||
6: { text: ['7', 0] }, 8: { text: ['9', 0], negative_text: ['10', 0] }, 11: { text: ['12', 0] },
|
||||
});
|
||||
// The editor nodes' text inputs are sockets now; the Classic nodes keep their core widget sockets.
|
||||
expect(result.widgetBound).toEqual({ 1: ['text'], 2: ['text'], 3: [], 4: [], 5: [], 6: [], 8: [], 11: ['text'] });
|
||||
// The linked sockets are drawn where a user can see and re-drag them, in either renderer.
|
||||
for (const [node, name] of [[6, 'text'], [8, 'text'], [8, 'negative_text']]) {
|
||||
await page.evaluate(async node => { const { app } = await import('/scripts/app.js'); app.canvas.centerOnNode(app.graph.getNodeById(node)); app.canvas.setDirty(true, true); }, node);
|
||||
await expect.poll(async () => {
|
||||
const position = await slotPosition(page, node, 'in', name);
|
||||
return Boolean(position && position[0] > 0 && position[1] > 0 && position[0] < 1600 && position[1] < 1100);
|
||||
}, { message: `${node}.${name} socket on screen` }).toBe(true);
|
||||
}
|
||||
const unplugged = await page.evaluate(async () => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
for (const id of [6, 8]) { const node = app.graph.getNodeById(id); for (const input of [...node.inputs]) if (input.link != null) node.disconnectInput(node.inputs.indexOf(input)); }
|
||||
const output = (await app.graphToPrompt()).output;
|
||||
return { focused: output['6'].inputs.text, positive: output['8'].inputs.text, negative: output['8'].inputs.negative_text };
|
||||
});
|
||||
expect(unplugged).toEqual({ focused: 'Typed under focused link', positive: 'Typed pair positive', negative: 'Typed pair negative' });
|
||||
});
|
||||
+51
@@ -186,6 +186,57 @@
|
||||
"prompt": null,
|
||||
"workflow": "{\"nodes\": [{\"id\": 4, \"type\": \"CheckpointLoaderSimple\", \"mode\": 0, \"inputs\": [], \"widgets_values\": [\"example.safetensors\"]}, {\"id\": 6, \"type\": \"CLIPTextEncode\", \"mode\": 0, \"inputs\": [{\"name\": \"clip\", \"link\": 3}], \"widgets_values\": [\"a tidepool at noon\"]}, {\"id\": 7, \"type\": \"CLIPTextEncode\", \"mode\": 0, \"inputs\": [{\"name\": \"clip\", \"link\": 5}], \"widgets_values\": [\"noisy\"]}, {\"id\": 10, \"type\": \"Reroute\", \"mode\": 0, \"inputs\": [{\"name\": \"\", \"link\": 7}]}, {\"id\": 3, \"type\": \"KSampler\", \"mode\": 0, \"inputs\": [{\"name\": \"model\", \"link\": 1}, {\"name\": \"positive\", \"link\": 8}, {\"name\": \"negative\", \"link\": 4}, {\"name\": \"latent_image\", \"link\": 2}], \"widgets_values\": [314, \"randomize\", 25, 6.5, \"dpmpp_2m\", \"karras\", 1]}, {\"id\": 11, \"type\": \"CLIPTextEncode\", \"mode\": 2, \"inputs\": [], \"widgets_values\": [\"muted node text\"]}], \"links\": [[1, 4, 0, 3, 0, \"MODEL\"], [3, 4, 1, 6, 0, \"CLIP\"], [5, 4, 1, 7, 0, \"CLIP\"], [7, 6, 0, 10, 0, \"CONDITIONING\"], [8, 10, 0, 3, 1, \"CONDITIONING\"], [4, 7, 0, 3, 2, \"CONDITIONING\"], {\"id\": 2, \"origin_id\": 99, \"origin_slot\": 0, \"target_id\": 3, \"target_slot\": 3, \"type\": \"LATENT\"}]}"
|
||||
},
|
||||
{
|
||||
"expected": {
|
||||
"ok": true,
|
||||
"result": {
|
||||
"cfgScale": 5,
|
||||
"checkpoint": "example.safetensors",
|
||||
"negativePrompt": "plain negative",
|
||||
"positivePrompt": "a detached positive",
|
||||
"sampler": "euler",
|
||||
"seed": 7,
|
||||
"steps": 12
|
||||
}
|
||||
},
|
||||
"name": "case:workflow-only-detached-editor-text",
|
||||
"prompt": null,
|
||||
"workflow": "{\"nodes\": [{\"id\": 4, \"type\": \"CheckpointLoaderSimple\", \"mode\": 0, \"inputs\": [], \"widgets_values\": [\"example.safetensors\"]}, {\"id\": 6, \"type\": \"PromptManagerFocused\", \"mode\": 0, \"inputs\": [{\"name\": \"clip\", \"link\": 3}, {\"name\": \"prepend_text\", \"link\": null}, {\"name\": \"append_text\", \"link\": null}, {\"name\": \"text\", \"link\": null}, {\"name\": \"category\", \"widget\": {\"name\": \"category\"}, \"link\": null}, {\"name\": \"tags\", \"widget\": {\"name\": \"tags\"}, \"link\": null}, {\"name\": \"search_text\", \"widget\": {\"name\": \"search_text\"}, \"link\": null}], \"widgets_values\": [\"a detached positive\", \"a category\", \"a tag\", \"\", \"\", \"\"]}, {\"id\": 7, \"type\": \"CLIPTextEncode\", \"mode\": 0, \"inputs\": [{\"name\": \"clip\", \"link\": 5}], \"widgets_values\": [\"plain negative\"]}, {\"id\": 3, \"type\": \"KSampler\", \"mode\": 0, \"inputs\": [{\"name\": \"model\", \"link\": 1}, {\"name\": \"positive\", \"link\": 8}, {\"name\": \"negative\", \"link\": 4}], \"widgets_values\": [7, \"fixed\", 12, 5, \"euler\", \"normal\", 1]}], \"links\": [[1, 4, 0, 3, 0, \"MODEL\"], [3, 4, 1, 6, 0, \"CLIP\"], [5, 4, 1, 7, 0, \"CLIP\"], [8, 6, 0, 3, 1, \"CONDITIONING\"], [4, 7, 0, 3, 2, \"CONDITIONING\"]]}"
|
||||
},
|
||||
{
|
||||
"expected": {
|
||||
"ok": true,
|
||||
"result": {
|
||||
"cfgScale": 5,
|
||||
"checkpoint": "example.safetensors",
|
||||
"negativePrompt": "a detached negative",
|
||||
"positivePrompt": "a detached positive",
|
||||
"sampler": "euler",
|
||||
"seed": 7,
|
||||
"steps": 12
|
||||
}
|
||||
},
|
||||
"name": "case:workflow-only-detached-pair-text",
|
||||
"prompt": null,
|
||||
"workflow": "{\"nodes\": [{\"id\": 4, \"type\": \"CheckpointLoaderSimple\", \"mode\": 0, \"inputs\": [], \"widgets_values\": [\"example.safetensors\"]}, {\"id\": 6, \"type\": \"PromptManagerPair\", \"mode\": 0, \"inputs\": [{\"name\": \"clip\", \"link\": 3}, {\"name\": \"prepend_text\", \"link\": null}, {\"name\": \"append_text\", \"link\": null}, {\"name\": \"text\", \"link\": null}, {\"name\": \"negative_text\", \"link\": null}, {\"name\": \"category\", \"widget\": {\"name\": \"category\"}, \"link\": null}, {\"name\": \"tags\", \"widget\": {\"name\": \"tags\"}, \"link\": null}, {\"name\": \"search_text\", \"widget\": {\"name\": \"search_text\"}, \"link\": null}], \"widgets_values\": [\"a detached positive\", \"a detached negative\", \"a category\", \"a tag\", \"\", \"\", \"\"]}, {\"id\": 3, \"type\": \"KSampler\", \"mode\": 0, \"inputs\": [{\"name\": \"model\", \"link\": 1}, {\"name\": \"positive\", \"link\": 8}, {\"name\": \"negative\", \"link\": 4}], \"widgets_values\": [7, \"fixed\", 12, 5, \"euler\", \"normal\", 1]}], \"links\": [[1, 4, 0, 3, 0, \"MODEL\"], [3, 4, 1, 6, 0, \"CLIP\"], [5, 4, 1, 7, 0, \"CLIP\"], [8, 6, 0, 3, 1, \"CONDITIONING\"], [4, 6, 1, 3, 2, \"CONDITIONING\"]]}"
|
||||
},
|
||||
{
|
||||
"expected": {
|
||||
"ok": true,
|
||||
"result": {
|
||||
"cfgScale": 4.5,
|
||||
"checkpoint": "example.safetensors",
|
||||
"negativePrompt": "plain negative",
|
||||
"positivePrompt": "a detached positive",
|
||||
"sampler": "dpmpp_2m",
|
||||
"seed": 9,
|
||||
"steps": 30
|
||||
}
|
||||
},
|
||||
"name": "case:workflow-only-core-widget-input-out-of-order",
|
||||
"prompt": null,
|
||||
"workflow": "{\"nodes\": [{\"id\": 4, \"type\": \"CheckpointLoaderSimple\", \"mode\": 0, \"inputs\": [], \"widgets_values\": [\"example.safetensors\"]}, {\"id\": 6, \"type\": \"PromptManagerFocused\", \"mode\": 0, \"inputs\": [{\"name\": \"clip\", \"link\": 3}, {\"name\": \"prepend_text\", \"link\": null}, {\"name\": \"append_text\", \"link\": null}, {\"name\": \"text\", \"link\": null}, {\"name\": \"category\", \"widget\": {\"name\": \"category\"}, \"link\": null}, {\"name\": \"tags\", \"widget\": {\"name\": \"tags\"}, \"link\": null}, {\"name\": \"search_text\", \"widget\": {\"name\": \"search_text\"}, \"link\": null}], \"widgets_values\": [\"a detached positive\", \"a category\", \"a tag\", \"\", \"\", \"\"]}, {\"id\": 7, \"type\": \"CLIPTextEncode\", \"mode\": 0, \"inputs\": [{\"name\": \"clip\", \"link\": 5}], \"widgets_values\": [\"plain negative\"]}, {\"id\": 3, \"type\": \"KSampler\", \"mode\": 0, \"inputs\": [{\"name\": \"model\", \"link\": 1}, {\"name\": \"positive\", \"link\": 8}, {\"name\": \"negative\", \"link\": 4}, {\"name\": \"cfg\", \"widget\": {\"name\": \"cfg\"}, \"link\": null}], \"widgets_values\": [9, \"fixed\", 30, 4.5, \"dpmpp_2m\", \"karras\", 1]}], \"links\": [[1, 4, 0, 3, 0, \"MODEL\"], [3, 4, 1, 6, 0, \"CLIP\"], [5, 4, 1, 7, 0, \"CLIP\"], [8, 6, 0, 3, 1, \"CONDITIONING\"], [4, 7, 0, 3, 2, \"CONDITIONING\"]]}"
|
||||
},
|
||||
{
|
||||
"expected": {
|
||||
"ok": true,
|
||||
|
||||
@@ -244,6 +244,90 @@ def _workflow_only():
|
||||
}
|
||||
|
||||
|
||||
def _detached_editor_text(pair):
|
||||
"""#343: editor text inputs are sockets, not widget inputs; no named values."""
|
||||
names = ["text"] + (["negative_text"] if pair else [])
|
||||
sockets = [
|
||||
{"name": "clip", "link": 3},
|
||||
{"name": "prepend_text", "link": None},
|
||||
{"name": "append_text", "link": None},
|
||||
]
|
||||
sockets += [{"name": name, "link": None} for name in names]
|
||||
sockets += [
|
||||
{"name": name, "widget": {"name": name}, "link": None}
|
||||
for name in ("category", "tags", "search_text")
|
||||
]
|
||||
editor = {
|
||||
"id": 6,
|
||||
"type": "PromptManagerPair" if pair else "PromptManagerFocused",
|
||||
"mode": 0,
|
||||
"inputs": sockets,
|
||||
"widgets_values": ["a detached positive"]
|
||||
+ (["a detached negative"] if pair else [])
|
||||
+ ["a category", "a tag", "", "", ""],
|
||||
}
|
||||
negative = (
|
||||
[]
|
||||
if pair
|
||||
else [
|
||||
{
|
||||
"id": 7,
|
||||
"type": "CLIPTextEncode",
|
||||
"mode": 0,
|
||||
"inputs": [{"name": "clip", "link": 5}],
|
||||
"widgets_values": ["plain negative"],
|
||||
}
|
||||
]
|
||||
)
|
||||
return {
|
||||
"nodes": [
|
||||
{
|
||||
"id": 4,
|
||||
"type": "CheckpointLoaderSimple",
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"widgets_values": ["example.safetensors"],
|
||||
},
|
||||
editor,
|
||||
*negative,
|
||||
{
|
||||
"id": 3,
|
||||
"type": "KSampler",
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{"name": "model", "link": 1},
|
||||
{"name": "positive", "link": 8},
|
||||
{"name": "negative", "link": 4},
|
||||
],
|
||||
"widgets_values": [7, "fixed", 12, 5, "euler", "normal", 1],
|
||||
},
|
||||
],
|
||||
"links": [
|
||||
[1, 4, 0, 3, 0, "MODEL"],
|
||||
[3, 4, 1, 6, 0, "CLIP"],
|
||||
[5, 4, 1, 7, 0, "CLIP"],
|
||||
[8, 6, 0, 3, 1, "CONDITIONING"],
|
||||
[4, 6 if pair else 7, 1 if pair else 0, 3, 2, "CONDITIONING"],
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _core_widget_input_out_of_order():
|
||||
"""#343 review R4: a core KSampler whose only widget input is ``cfg``.
|
||||
|
||||
Older frontends saved only converted or linked widget inputs, so the list
|
||||
is not a prefix of the schema and carries no named values; the positional
|
||||
values must still land on the schema's names.
|
||||
"""
|
||||
workflow = _detached_editor_text(False)
|
||||
sampler = workflow["nodes"][-1]
|
||||
sampler["inputs"] = sampler["inputs"] + [
|
||||
{"name": "cfg", "widget": {"name": "cfg"}, "link": None}
|
||||
]
|
||||
sampler["widgets_values"] = [9, "fixed", 30, 4.5, "dpmpp_2m", "karras", 1]
|
||||
return workflow
|
||||
|
||||
|
||||
def _nested(depth):
|
||||
"""A value nested ``depth`` arrays deep, as a crafted file can hold."""
|
||||
value = ["leaf", 7]
|
||||
@@ -389,6 +473,11 @@ CASES = {
|
||||
}
|
||||
),
|
||||
"workflow-only": case(workflow=_workflow_only()),
|
||||
"workflow-only-detached-editor-text": case(workflow=_detached_editor_text(False)),
|
||||
"workflow-only-detached-pair-text": case(workflow=_detached_editor_text(True)),
|
||||
"workflow-only-core-widget-input-out-of-order": case(
|
||||
workflow=_core_widget_input_out_of_order()
|
||||
),
|
||||
"overwritten-prompt-manager-text": case(
|
||||
{
|
||||
"4": CHECKPOINT,
|
||||
|
||||
@@ -0,0 +1,201 @@
|
||||
// #343: a link can drive the editor nodes' text inputs (web/js/text-links.js).
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const source = import('../../web/js/text-links.js');
|
||||
|
||||
// The frontend's own slot order for the Focused/Split/Pair schemas: forceInput sockets first, then the widget sockets.
|
||||
function pairNode() {
|
||||
const inputs = [
|
||||
{name:'clip',type:'CLIP',link:null},
|
||||
{name:'prepend_text',type:'STRING',link:null},
|
||||
{name:'append_text',type:'STRING',link:null},
|
||||
{name:'text',type:'STRING',link:null,widget:{name:'text'},_widget:{name:'text'}},
|
||||
{name:'negative_text',type:'STRING',link:null,widget:{name:'negative_text'}},
|
||||
{name:'category',type:'STRING',link:null,widget:{name:'category'}},
|
||||
];
|
||||
return {inputs};
|
||||
}
|
||||
|
||||
test('the editor note says the value comes from a link and that typed text is kept', async () => {
|
||||
const {linkNoteText} = await source;
|
||||
assert.equal(linkNoteText('Negative prompt'), 'Negative prompt comes from a link. Your typed text is kept for when you disconnect it.');
|
||||
});
|
||||
|
||||
test('a linked widget stores its typed value; an unlinked one its live editor text', async () => {
|
||||
const {storedWidgetValue} = await source;
|
||||
const widget = {name:'text', value:'typed', inputEl:{value:'live'}};
|
||||
assert.equal(storedWidgetValue({inputs:[{name:'text',link:3}]}, widget), 'typed');
|
||||
assert.equal(storedWidgetValue({inputs:[{name:'text',link:null}]}, widget), 'live');
|
||||
assert.equal(storedWidgetValue({}, {name:'tags', value:'a, b'}), 'a, b');
|
||||
});
|
||||
|
||||
test('exposing a socket detaches only the named inputs from their hidden widgets', async () => {
|
||||
const {exposeTextSockets} = await source;
|
||||
const node = pairNode();
|
||||
assert.deepEqual(exposeTextSockets(node, ['text','negative_text']), ['text','negative_text']);
|
||||
const byName = Object.fromEntries(node.inputs.map(input => [input.name,input]));
|
||||
for (const name of ['text','negative_text']) {
|
||||
assert.equal('widget' in byName[name], false);
|
||||
assert.equal(byName[name]._widget, undefined);
|
||||
assert.equal(byName[name].type, 'STRING');
|
||||
}
|
||||
assert.deepEqual(byName.category.widget, {name:'category'});
|
||||
assert.deepEqual(exposeTextSockets(node, ['text','negative_text']), [], 'idempotent');
|
||||
assert.deepEqual(exposeTextSockets({}, ['text']), [], 'a node without inputs is left alone');
|
||||
});
|
||||
|
||||
test('exposing a socket keeps an existing link and the widget value', async () => {
|
||||
const {exposeTextSockets,isLinked} = await source;
|
||||
const node = pairNode();
|
||||
node.inputs[3].link = 12;
|
||||
exposeTextSockets(node, ['text']);
|
||||
assert.equal(node.inputs[3].link, 12);
|
||||
assert.equal(isLinked(node,'text'), true);
|
||||
assert.equal(isLinked(node,'negative_text'), false);
|
||||
assert.equal(isLinked({},'text'), false);
|
||||
});
|
||||
|
||||
test('a STRING link dropped on the editor prefers the first free text input over prepend', async () => {
|
||||
const {preferredTextSlot} = await source;
|
||||
const node = pairNode();
|
||||
assert.deepEqual(preferredTextSlot(node, 'STRING', ['text','negative_text']), {index:3,slot:node.inputs[3]});
|
||||
node.inputs[3].link = 4;
|
||||
assert.deepEqual(preferredTextSlot(node, 'STRING', ['text','negative_text']), {index:4,slot:node.inputs[4]});
|
||||
node.inputs[4].link = 5;
|
||||
assert.equal(preferredTextSlot(node, 'STRING', ['text','negative_text']), undefined, 'both linked: the frontend chooses');
|
||||
});
|
||||
|
||||
test('only exact STRING links are steered; wildcards keep the frontend choice (#343 review M2)', async () => {
|
||||
const {preferredTextSlot} = await source;
|
||||
const node = pairNode();
|
||||
for (const type of ['STRING','STRING,INT',' STRING ']) assert.equal(preferredTextSlot(node, type, ['text'])?.index, 3, type);
|
||||
for (const type of ['*','','CLIP','INT','IMAGE',undefined,null,0]) assert.equal(preferredTextSlot(node, type, ['text']), undefined, String(type));
|
||||
assert.equal(preferredTextSlot({}, 'STRING', ['text']), undefined);
|
||||
});
|
||||
|
||||
test('a link being moved off a text input leaves it free, as the frontend counts it (#343 review I2)', async () => {
|
||||
const {preferredTextSlot} = await source;
|
||||
const links = new Map([[7,{id:7,_dragging:true}],[8,{id:8}]]);
|
||||
const node = Object.assign(pairNode(), {graph:{getLink:id => links.get(id)}});
|
||||
node.inputs[3].link = 7;
|
||||
assert.equal(preferredTextSlot(node, 'STRING', ['text','negative_text']).slot.name, 'text', 'the dragged link does not block text');
|
||||
links.get(7)._dragging = false;
|
||||
assert.equal(preferredTextSlot(node, 'STRING', ['text','negative_text']).slot.name, 'negative_text');
|
||||
node.inputs[4].link = 8;
|
||||
assert.equal(preferredTextSlot(node, 'STRING', ['text','negative_text']), undefined);
|
||||
node.inputs[4].link = 99;
|
||||
assert.equal(preferredTextSlot(node, 'STRING', ['text','negative_text']).slot.name, 'negative_text', 'a link the graph no longer has is free');
|
||||
const legacy = Object.assign(pairNode(), {graph:{links:{5:{id:5,_dragging:true}}}});
|
||||
legacy.inputs[3].link = 5;
|
||||
assert.equal(preferredTextSlot(legacy, 'STRING', ['text']).slot.name, 'text', 'object-keyed link tables too');
|
||||
});
|
||||
|
||||
function primitiveGraph() {
|
||||
const nodes = new Map();
|
||||
const links = new Map();
|
||||
const graph = {getNodeById:id => nodes.get(id), getLink:id => links.get(id)};
|
||||
const add = (id, node) => { nodes.set(id, {id, graph, ...node}); return nodes.get(id); };
|
||||
return {graph, nodes, links, add};
|
||||
}
|
||||
|
||||
test('a legacy frontend Primitive feeding a detached text input supplies its value (#343 review I1)', async () => {
|
||||
const {primitiveTextValue} = await source;
|
||||
const {nodes, links, add} = primitiveGraph();
|
||||
const target = add(1, {inputs:[{name:'text',link:10},{name:'negative_text',link:11},{name:'prepend_text',link:12},{name:'category',link:null}]});
|
||||
add(2, {type:'PrimitiveNode', isVirtualNode:true, widgets:[{value:'From the primitive'}]});
|
||||
add(3, {type:'Reroute', isVirtualNode:true, inputs:[{link:13}]});
|
||||
add(4, {type:'PrimitiveNode', isVirtualNode:true, widgets:[{value:42}]});
|
||||
add(5, {type:'PrimitiveStringMultiline', widgets:[{value:'a real node'}]});
|
||||
links.set(10, {origin_id:2}); links.set(11, {origin_id:3}); links.set(13, {origin_id:4}); links.set(12, {origin_id:5});
|
||||
assert.equal(await primitiveTextValue(target, 'text'), 'From the primitive');
|
||||
assert.equal(await primitiveTextValue(target, 'negative_text'), '42', 'through a virtual reroute');
|
||||
assert.equal(await primitiveTextValue(target, 'prepend_text'), undefined, 'a real node is serialized as a link');
|
||||
assert.equal(await primitiveTextValue(target, 'category'), undefined, 'not linked');
|
||||
assert.equal(await primitiveTextValue(target, 'missing'), undefined);
|
||||
nodes.get(2).widgets = [];
|
||||
assert.equal(await primitiveTextValue(target, 'text'), undefined, 'a primitive without its widget yet');
|
||||
links.set(13, {origin_id:3});
|
||||
assert.equal(await primitiveTextValue(target, 'negative_text'), undefined, 'a reroute cycle ends');
|
||||
assert.equal(await primitiveTextValue({inputs:[{name:'text',link:1}]}, 'text'), undefined, 'no graph');
|
||||
});
|
||||
|
||||
test('an input still bound to its widget is left to the host, which applies the Primitive itself (#343 re-review R1)', async () => {
|
||||
const {primitiveTextValue} = await source;
|
||||
const {links, add} = primitiveGraph();
|
||||
const target = add(1, {inputs:[{name:'text',link:10,widget:{name:'text'}}], widgets:[{name:'text',value:'typed'}]});
|
||||
add(2, {type:'PrimitiveNode', isVirtualNode:true, widgets:[{value:'{red|blue}'}]});
|
||||
links.set(10, {origin_id:2});
|
||||
assert.equal(await primitiveTextValue(target, 'text'), undefined);
|
||||
});
|
||||
|
||||
test('a detached input runs the Primitive value the way the host runs a widget value: replace, then the widget serializer (#343 re-review R1)', async () => {
|
||||
const {primitiveTextValue} = await source;
|
||||
const {graph, links, add} = primitiveGraph();
|
||||
const calls = [];
|
||||
const widget = {name:'text', value:'typed under the link', serializeValue(...args) { calls.push([this.value, args.length]); return this.value.replace('{alpha|beta}', 'alpha'); }};
|
||||
const target = add(1, {inputs:[{name:'text',link:10},{name:'negative_text',link:11}], widgets:[widget]});
|
||||
const primitive = add(2, {type:'PrimitiveNode', isVirtualNode:true, properties:{'Run widget replace on values':false}, widgets:[{value:'a %year% {alpha|beta} car'}]});
|
||||
add(3, {type:'PrimitiveNode', isVirtualNode:true, properties:{'Run widget replace on values':true}, widgets:[{value:'no serializer %year%'}]});
|
||||
links.set(10, {origin_id:2}); links.set(11, {origin_id:3});
|
||||
const replaced = [];
|
||||
const replaceText = (from, value) => { replaced.push(from); return value.replace('%year%', '2026'); };
|
||||
assert.equal(await primitiveTextValue(target, 'text', {replaceText}), 'a %year% alpha car', 'replace stays off unless the Primitive asks for it');
|
||||
assert.deepEqual(calls, [['a %year% {alpha|beta} car', 0]], 'serialized once, without a node, so widgets_values is not touched');
|
||||
assert.equal(widget.value, 'typed under the link', 'the typed text is restored');
|
||||
primitive.properties['Run widget replace on values'] = true;
|
||||
assert.equal(await primitiveTextValue(target, 'text', {replaceText}), 'a 2026 alpha car');
|
||||
assert.deepEqual(replaced, [graph]);
|
||||
assert.equal(await primitiveTextValue(target, 'negative_text', {replaceText}), 'no serializer 2026', 'no widget serializer: the replaced value');
|
||||
assert.equal(await primitiveTextValue(target, 'negative_text'), 'no serializer %year%', 'no replacer: the value as is');
|
||||
const failing = () => { throw new Error('not this host'); };
|
||||
assert.equal(await primitiveTextValue(target, 'negative_text', {replaceText:failing}), 'no serializer %year%', 'a replacer that fails leaves the value');
|
||||
widget.serializeValue = () => { throw new Error('broken serializer'); };
|
||||
await assert.rejects(primitiveTextValue(target, 'text'), /broken serializer/);
|
||||
assert.equal(widget.value, 'typed under the link', 'restored even when the serializer throws');
|
||||
});
|
||||
|
||||
test('installing the preference wraps findInputByType and restores nothing it did not own', async () => {
|
||||
const {preferTextInputs} = await source;
|
||||
const calls = [];
|
||||
const prototype = {findInputByType(type) { calls.push(type); return {index:1,slot:this.inputs[1]}; }};
|
||||
preferTextInputs(prototype, ['text']);
|
||||
const node = Object.assign(Object.create(prototype), pairNode());
|
||||
assert.equal(node.findInputByType('STRING').index, 3);
|
||||
assert.deepEqual(calls, []);
|
||||
assert.equal(node.findInputByType('CLIP').index, 1);
|
||||
node.inputs[3].link = 9;
|
||||
assert.equal(node.findInputByType('STRING').index, 1, 'the main prompt is taken: fall back to the frontend');
|
||||
assert.deepEqual(calls, ['CLIP','STRING']);
|
||||
const bare = {};
|
||||
preferTextInputs(bare, ['text']);
|
||||
assert.equal(bare.findInputByType, undefined, 'an older frontend without findInputByType is left unchanged');
|
||||
});
|
||||
|
||||
test('a drop point over a visible prompt box or its note picks that box input (#343 review M1)', async () => {
|
||||
const {textInputAtPoint} = await source;
|
||||
const box = (left, top, width, height) => ({getBoundingClientRect:() => ({left, top, width, height, right:left + width, bottom:top + height})});
|
||||
const node = pairNode();
|
||||
const targets = [{name:'text',element:box(0,0,100,100)},{name:'negative_text',element:box(120,0,100,100)},{name:'text',element:box(0,0,0,0)},{name:'missing',element:box(300,0,50,50)}];
|
||||
assert.equal(textInputAtPoint(node, [150,50], targets).name, 'negative_text');
|
||||
assert.equal(textInputAtPoint(node, [10,10], targets).name, 'text');
|
||||
assert.equal(textInputAtPoint(node, [110,50], targets), undefined, 'between the boxes');
|
||||
assert.equal(textInputAtPoint(node, [320,10], targets), undefined, 'a target whose input the node lacks');
|
||||
assert.equal(textInputAtPoint(node, [0,0], [{name:'text',element:box(0,0,0,0)}]), undefined, 'a hidden target');
|
||||
});
|
||||
|
||||
test('a linked Classic text box gives its row back and gets its size back on disconnect (#343 review M5)', async () => {
|
||||
const {collapseLinkedWidget} = await source;
|
||||
const own = () => 120;
|
||||
const widget = {options:{getMinHeight:own}};
|
||||
collapseLinkedWidget(widget, true);
|
||||
assert.equal(widget.options.getMinHeight(), 0);
|
||||
assert.equal(widget.options.getMaxHeight(), 0);
|
||||
collapseLinkedWidget(widget, true);
|
||||
collapseLinkedWidget(widget, false);
|
||||
assert.equal(widget.options.getMinHeight, own);
|
||||
assert.equal('getMaxHeight' in widget.options, false, 'an option it did not have is removed again');
|
||||
collapseLinkedWidget(widget, false);
|
||||
assert.equal(widget.options.getMinHeight, own);
|
||||
collapseLinkedWidget(undefined, true);
|
||||
collapseLinkedWidget({}, true);
|
||||
});
|
||||
@@ -126,7 +126,7 @@ test('held legacy jobs read "(admin page)" with no outcomes', async () => {
|
||||
|
||||
test('unknown and generic kinds show no tiles of another kind', async () => {
|
||||
const { kindOf, viewOf } = await source;
|
||||
for (const kind of ['restore', '', 'constructor', '__proto__', 'health']) {
|
||||
for (const kind of ['backup', '', 'constructor', '__proto__', 'health']) {
|
||||
const view = kindOf(kind);
|
||||
assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []], kind);
|
||||
assert.deepEqual([view.log, view.cancelLabel], ['Job log', 'Cancel job']);
|
||||
|
||||
@@ -344,7 +344,7 @@ test('unknown kinds get the generic view without the scan tiles', async () => {
|
||||
['no_metadata', 'No usable metadata'], ['known', 'Already known']]);
|
||||
assert.deepEqual([...scan.openable].sort(), ['added', 'known']);
|
||||
assert.deepEqual([scan.log, scan.cancel], ['Scan log', 'Cancel scan']);
|
||||
for (const kind of ['restore', '', 'constructor', '__proto__']) {
|
||||
for (const kind of ['backup', '', 'constructor', '__proto__']) {
|
||||
const view = jobView(kind);
|
||||
assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [[], [], []]);
|
||||
assert.deepEqual([view.log, view.cancel], ['Job log', 'Cancel job']);
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
// System › Backup & export (#249 System S9 Task 24): the concept's copy (app.js 691–693), the status line from
|
||||
// GET /backup/status (D103), the restore review's rows from the S8 preview (D105), its upload and refusals, the export
|
||||
// list, the restore job's Activity copy and the reload after a finished restore.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { ApiError } = require('../../web/js/api-client.js');
|
||||
|
||||
const backups = import('../../web/js/workspace/system-backups.mjs');
|
||||
const kinds = import('../../web/js/workspace/activity-kinds.mjs');
|
||||
const scan = import('../../web/js/workspace/activity-scan.mjs');
|
||||
const dialogs = import('../../web/js/workspace/workspace-dialogs.mjs');
|
||||
|
||||
const CONCEPT = fs.readFileSync(path.join(__dirname, '../../docs/concepts/workspace/app.js'), 'utf8');
|
||||
// The module's code without its comments (which name the concept's prototype labels they replace).
|
||||
const SOURCE = fs.readFileSync(path.join(__dirname, '../../web/js/workspace/system-backups.mjs'), 'utf8')
|
||||
.replace(/\/\*[\s\S]*?\*\//g, '').replace(/^\s*\/\/.*$/gm, '');
|
||||
|
||||
const conceptSection = () => {
|
||||
const start = CONCEPT.indexOf('case "backups":');
|
||||
return CONCEPT.slice(start, CONCEPT.indexOf('break;', start));
|
||||
};
|
||||
|
||||
test('the section keeps the concept copy: surfaces, descriptions and the two kept labels', () => {
|
||||
const body = conceptSection();
|
||||
for (const text of ['Database backup', 'A complete recovery snapshot, including prompt pairs, metadata and image links. Media files are separate.',
|
||||
'Review restore…', 'Portable export', 'Export reusable recipes and tags as versioned JSON. This is distinct from a complete database backup.',
|
||||
'Import / inspect file', 'Recovery policy', 'Proposed', 'Create a safety backup before maintenance', 'Scheduled database backups',
|
||||
'Manual only', 'Daily', 'Weekly']) {
|
||||
assert.ok(body.includes(text), `concept lacks ${text}`);
|
||||
assert.ok(SOURCE.includes(text), `production lacks ${text}`);
|
||||
}
|
||||
// D103/D104: prototype copy is never shipped.
|
||||
// D104 (M2): the toggle's concept line ("Preview a verified backup…") misstates the policy and is replaced too.
|
||||
for (const sample of ['Simulate backup', 'Sample backup', 'Export sample JSON', 'Production feature proposal', 'A restore would be',
|
||||
'Preview a verified backup before applying operations that change records.']) {
|
||||
assert.ok(body.includes(sample));
|
||||
assert.ok(!SOURCE.includes(sample), `production ships ${sample}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('status line shows only server-reported backups (D103)', async () => {
|
||||
const { backupStatusText, NO_BACKUP, STATUS_LOADING, STATUS_FAILED, whenText } = await backups;
|
||||
assert.equal(backupStatusText(undefined), STATUS_LOADING);
|
||||
assert.equal(backupStatusText(null), STATUS_FAILED);
|
||||
assert.equal(backupStatusText({ success: true, safety_backups: [], last_download: null }), NO_BACKUP);
|
||||
assert.equal(NO_BACKUP, 'No backup recorded.');
|
||||
// A time the server did not send in a readable form names nothing.
|
||||
assert.equal(backupStatusText({ safety_backups: [{ name: 'x', modified_at: 'soon' }], last_download: { at: 7, verified: true } }),
|
||||
NO_BACKUP);
|
||||
const at = '2026-10-25T09:30:00+00:00';
|
||||
assert.equal(backupStatusText({ safety_backups: [], last_download: { at, bytes: 10, verified: true } }),
|
||||
`Downloaded backup · ${whenText(at)} · verified.`);
|
||||
assert.equal(backupStatusText({ safety_backups: [], last_download: { at, bytes: 10, verified: false } }),
|
||||
`Downloaded backup · ${whenText(at)} · not verified.`);
|
||||
const safety = [{ name: 'prompts.db.backup_2', bytes: 1, modified_at: at }, { name: 'prompts.db.backup_1', bytes: 1, modified_at: at }];
|
||||
assert.equal(backupStatusText({ safety_backups: safety, last_download: null }), `Safety backup · ${whenText(at)} · 2 on disk.`);
|
||||
assert.equal(backupStatusText({ safety_backups: safety.slice(0, 1), last_download: { at, verified: true } }),
|
||||
`Downloaded backup · ${whenText(at)} · verified. Safety backup · ${whenText(at)} · 1 on disk.`);
|
||||
assert.equal(whenText(undefined), null);
|
||||
});
|
||||
|
||||
const PREVIEW = Object.freeze({
|
||||
success: true, token: 'r'.repeat(43), expires_at: '2026-10-25T12:15:00+00:00',
|
||||
summary: { prompts: 1234, positives: 900, pairs: 3, archived_pairs: 0, image_links: 56, runs: 1, saved_views: 2 },
|
||||
schema: { migrated: true, library_state: true }, integrity: 'ok', documents: { repaired: 0, cleared: 0, images: 0 },
|
||||
live: { prompts: 1, positives: 1, pairs: 0, archived_pairs: 0, image_links: 0, runs: 0, saved_views: 0 }, size: 4096,
|
||||
safety_backup: { destination_label: 'prompts.db.backup_<date and time>' },
|
||||
});
|
||||
|
||||
test('Review restore shows the real summary in the concept rows (D105)', async () => {
|
||||
const { restoreGrid, restoreNote, countsText, integrityText, snapshotText, whenText } = await backups;
|
||||
const file = { name: 'prompts_backup_20261025_093000.db', size: 3 * 2 ** 20, lastModified: Date.parse('2026-10-25T09:30:00Z') };
|
||||
assert.deepEqual(restoreGrid(PREVIEW, file), [
|
||||
['Snapshot', `prompts_backup_20261025_093000.db · 3.0 MB · ${whenText('2026-10-25T09:30:00.000Z')}`],
|
||||
['Integrity', 'Verified'],
|
||||
['Migration', 'Upgraded to the current schema'],
|
||||
['Safety backup', 'Created before replacing (prompts.db.backup_<date and time>)'],
|
||||
['In the backup', '1,234 prompts · 56 image links · 3 saved pairs · 1 run'],
|
||||
['In the library now', '1 prompt · 0 image links · 0 saved pairs · 0 runs'],
|
||||
]);
|
||||
assert.equal(restoreGrid({ ...PREVIEW, schema: { migrated: false } }, file)[2][1], 'Current schema');
|
||||
assert.equal(restoreGrid({ ...PREVIEW, safety_backup: null }, file)[3][1], 'Created before replacing');
|
||||
assert.equal(countsText({}), 'Not reported');
|
||||
assert.equal(snapshotText({ name: 'a.db' }), 'a.db');
|
||||
assert.equal(integrityText({ integrity: 'ok', documents: { repaired: 2, cleared: 1, images: 1 } }),
|
||||
'Verified · repaired 2 damaged workflow documents · could not repair 1 workflow document: 1 image shows no workflow');
|
||||
assert.equal(integrityText({ integrity: 'ok', documents: { repaired: 1, cleared: 2, images: 3 } }),
|
||||
'Verified · repaired 1 damaged workflow document · could not repair 2 workflow documents: 3 images show no workflow');
|
||||
assert.equal(integrityText({}), 'Not verified');
|
||||
const now = Date.parse('2026-10-25T12:00:00+00:00');
|
||||
assert.equal(restoreNote(PREVIEW, { now }), 'Restore replaces the whole library with this backup and runs alone in Activity. ' +
|
||||
'Nothing changes until you restore. This review holds for 15 minutes.');
|
||||
assert.equal(restoreNote({ expires_at: 'x' }, { now }), 'Restore replaces the whole library with this backup and runs alone ' +
|
||||
'in Activity. Nothing changes until you restore.');
|
||||
});
|
||||
|
||||
function response(status, body, headers = {}) {
|
||||
return { ok: status >= 200 && status < 300, status, json: async () => {
|
||||
if (body === undefined) throw new SyntaxError('not JSON');
|
||||
return body;
|
||||
}, headers: { get: name => headers[name] ?? null }, blob: async () => new Blob(['x']) };
|
||||
}
|
||||
|
||||
test('invalid file shows the validator message; a refusal names its job; a lost answer stages nothing', async () => {
|
||||
const { uploadRestore, reviewRefusal, REVIEW_FAILED, PREVIEW_URL } = await backups;
|
||||
const file = new File(['not a database'], 'bad.db');
|
||||
let seen;
|
||||
const refuse = async (url, init) => { seen = [url, init.method, init.cache]; return response(400,
|
||||
{ success: false, code: 'invalid-backup', error: 'Not a SQLite database file' }); };
|
||||
const invalid = await uploadRestore(file, { fetchUrl: refuse, ApiError }).catch(error => error);
|
||||
assert.deepEqual(seen, [PREVIEW_URL, 'POST', 'no-store']);
|
||||
assert.equal(reviewRefusal(invalid), "This file can't be restored: Not a SQLite database file.");
|
||||
const busy = await uploadRestore(file, { ApiError, fetchUrl: async () => response(409,
|
||||
{ success: false, code: 'busy', error: 'Busy', running: { id: 'j1', title: 'Metadata scan' } }) }).catch(error => error);
|
||||
assert.deepEqual(reviewRefusal(busy), ['Another job is running: ', { text: 'Metadata scan', href: '#/activity/j1' }, '.']);
|
||||
const full = await uploadRestore(file, { ApiError, fetchUrl: async () => response(429,
|
||||
{ success: false, code: 'too-large', error: 'Two restores are already being reviewed. Discard one first.' }) })
|
||||
.catch(error => error);
|
||||
// I1: the server's "Discard one first" names nothing this page can do; the toast says what the user can.
|
||||
assert.equal(reviewRefusal(full), (await backups).REVIEWS_FULL);
|
||||
const proxy = await uploadRestore(file, { ApiError, fetchUrl: async () => response(502, undefined) }).catch(error => error);
|
||||
assert.equal(reviewRefusal(proxy), REVIEW_FAILED);
|
||||
const odd = await uploadRestore(file, { ApiError, fetchUrl: async () => response(200, { success: true }) }).catch(error => error);
|
||||
assert.equal(reviewRefusal(odd), REVIEW_FAILED);
|
||||
assert.equal(reviewRefusal(new Error('offline')), REVIEW_FAILED);
|
||||
assert.deepEqual(await uploadRestore(file, { ApiError, fetchUrl: async () => response(200, PREVIEW) }), PREVIEW);
|
||||
});
|
||||
|
||||
test('Cancel discards the stage through the api client and never throws', async () => {
|
||||
const { discardRestore, PREVIEW_URL } = await backups;
|
||||
const calls = [];
|
||||
await discardRestore('a/b', { fetchUrl: async (url, init) => { calls.push([url, init.method]); return response(200, {}); } });
|
||||
await discardRestore('gone', { fetchUrl: async () => { throw new TypeError('offline'); } });
|
||||
assert.deepEqual(calls, [[`${PREVIEW_URL}/a%2Fb`, 'DELETE']]);
|
||||
});
|
||||
|
||||
test('Export JSON downloads /library/export; secondary buttons download /export and /pairs/export (D103)', async () => {
|
||||
const { EXPORTS } = await backups;
|
||||
assert.deepEqual(EXPORTS.map(([label, url]) => [label, url]), [
|
||||
['Export JSON', '/prompt_manager/library/export'],
|
||||
['Export all prompts JSON', '/prompt_manager/export'],
|
||||
['Export saved pairs JSON', '/prompt_manager/pairs/export'],
|
||||
]);
|
||||
});
|
||||
|
||||
test('a failed export toasts and saves nothing', async () => {
|
||||
const { downloadExport, EXPORTS, EXPORT_FAILED } = await backups;
|
||||
const toasts = [];
|
||||
assert.equal(await downloadExport(EXPORTS[1], { fetchUrl: async () => response(500, {}), notify: text => toasts.push(text) }), false);
|
||||
const aborted = AbortSignal.abort();
|
||||
assert.equal(await downloadExport(EXPORTS[1], { fetchUrl: async () => { throw new Error('x'); }, notify: text => toasts.push(text),
|
||||
signal: aborted }), false);
|
||||
assert.deepEqual(toasts, [EXPORT_FAILED]);
|
||||
});
|
||||
|
||||
test('Restore starts the job: Activity names it and ends with the server line', async () => {
|
||||
const { finishToast, outcomeLine, pillParts, canCancel, startToast } = await kinds;
|
||||
const job = { id: 'j', kind: 'restore', title: 'Database restore', state: 'running', cancellable: false, progress: 0, total: 0,
|
||||
counters: { errors: 0 }, status_line: 'Restoring the database' };
|
||||
assert.equal(startToast(job), 'Database restore started.');
|
||||
assert.equal(pillParts(job).label, 'Restoring the database');
|
||||
assert.equal(canCancel(job), false);
|
||||
assert.equal(outcomeLine(job), '');
|
||||
const done = { ...job, state: 'finished', status_line: 'Restored 3 prompts, 1 image link, 0 saved pairs and 0 runs' };
|
||||
assert.equal(finishToast(done), 'Database restore finished · Restored 3 prompts, 1 image link, 0 saved pairs and 0 runs');
|
||||
assert.equal(outcomeLine(done), done.status_line);
|
||||
const failed = { ...job, state: 'failed', status_line: 'The library changed since the review; nothing was changed. Review the restore again.' };
|
||||
assert.equal(finishToast(failed), `Database restore failed · ${failed.status_line}`);
|
||||
assert.equal(finishToast({ ...job, state: 'failed', status_line: '' }), 'Database restore failed.');
|
||||
});
|
||||
|
||||
test('views reload once after a restore this page saw finish; a failed restore changes nothing', async () => {
|
||||
const { onRestoreFinished } = await backups;
|
||||
let list = [];
|
||||
const listeners = new Set();
|
||||
const activity = { jobs: () => list, subscribe(listener) { listeners.add(listener); return () => listeners.delete(listener); } };
|
||||
const emit = next => { list = next; for (const listener of listeners) listener(); };
|
||||
let reloads = 0;
|
||||
const stop = onRestoreFinished(activity, () => { reloads += 1; });
|
||||
emit([{ id: 'old', kind: 'restore', state: 'finished' }]);
|
||||
assert.equal(reloads, 0, 'a restore that ended before this page saw it run');
|
||||
emit([{ id: 'a', kind: 'restore', state: 'running' }, { id: 's', kind: 'scan', state: 'running' }]);
|
||||
emit([{ id: 'a', kind: 'restore', state: 'finished' }, { id: 's', kind: 'scan', state: 'finished' }]);
|
||||
emit([{ id: 'a', kind: 'restore', state: 'finished' }]);
|
||||
assert.equal(reloads, 1);
|
||||
emit([{ id: 'b', kind: 'restore', state: 'running' }]);
|
||||
emit([{ id: 'b', kind: 'restore', state: 'failed' }]);
|
||||
assert.equal(reloads, 1);
|
||||
stop();
|
||||
assert.equal(listeners.size, 0);
|
||||
});
|
||||
|
||||
test('a downloaded backup the server could not verify says so', async () => {
|
||||
const { downloadDatabaseBackup, BACKUP_DONE, backupUnverified } = await scan;
|
||||
const toasts = [];
|
||||
const body = globalThis.document;
|
||||
globalThis.document = { createElement: () => ({ click() {}, remove() {} }), body: { append() {} } };
|
||||
try {
|
||||
await downloadDatabaseBackup({ notify: text => toasts.push(text),
|
||||
fetchUrl: async () => response(200, {}, { 'X-PromptManager-Verify': 'ok' }) });
|
||||
await downloadDatabaseBackup({ notify: text => toasts.push(text),
|
||||
fetchUrl: async () => response(200, {}, { 'X-PromptManager-Verify': 'Database does not contain a pair table' }) });
|
||||
} finally {
|
||||
globalThis.document = body;
|
||||
}
|
||||
assert.deepEqual(toasts, [BACKUP_DONE, backupUnverified('Database does not contain a pair table')]);
|
||||
assert.equal(toasts[1], 'Database backup downloaded, but it did not pass verification: Database does not contain a pair table');
|
||||
});
|
||||
|
||||
test('the Import dialog shares the rescan copy with Images (D15)', async () => {
|
||||
const { rescanFinished, RESCAN_STARTED, RESCAN_FAILED } = await dialogs;
|
||||
assert.equal(RESCAN_STARTED, 'Rescanning output folders…');
|
||||
assert.equal(RESCAN_FAILED, 'Rescan could not finish. Try again.');
|
||||
assert.equal(rescanFinished({ total: 1, roots: 2 }), 'Rescan finished. 1 file in 2 output folders.');
|
||||
assert.equal(rescanFinished({ total: 1200, roots: 1 }), 'Rescan finished. 1,200 files in 1 output folder.');
|
||||
assert.equal(rescanFinished(null), 'Rescan finished.');
|
||||
});
|
||||
|
||||
test('applyPlan follows a started job even after its section left, and reports the outcome (S9 N5, I1)', async () => {
|
||||
const { applyPlan } = await import('../../web/js/workspace/maintenance-review.mjs');
|
||||
const job = { id: 'a'.repeat(32), kind: 'restore', title: 'Database restore', state: 'running' };
|
||||
const tracked = [];
|
||||
const store = { track: raw => { tracked.push(raw.id); return raw; }, load() {} };
|
||||
const left = new AbortController();
|
||||
left.abort();
|
||||
const toasts = [], routes = [];
|
||||
const outcome = await applyPlan({ api: { post: async () => ({ success: true, job }) }, store, scope: { signal: left.signal },
|
||||
token: 't', notify: text => toasts.push(text), navigate: hash => routes.push(hash) });
|
||||
// The page's store follows it (a finished restore must reload the views); the left section neither toasts nor opens it.
|
||||
assert.deepEqual(tracked, [job.id]);
|
||||
assert.deepEqual([toasts, routes], [[], []]);
|
||||
assert.deepEqual(outcome, { job });
|
||||
const busy = new ApiError(409, 'Busy', 'busy', null, { success: false, code: 'busy', running: { id: 'j1', title: 'Scan' } });
|
||||
const live = { signal: new AbortController().signal };
|
||||
const refused = await applyPlan({ api: { post: async () => { throw busy; } }, store, scope: live, token: 't',
|
||||
notify: text => toasts.push(text), navigate: hash => routes.push(hash) });
|
||||
assert.deepEqual(refused, { refused: busy });
|
||||
assert.deepEqual(toasts, [['Another job is running: ', { text: 'Scan', href: '#/activity/j1' }, '.']]);
|
||||
});
|
||||
|
||||
test('after an apply: follow a started job, reopen a busy refusal, release the token otherwise (I1)', async () => {
|
||||
const { afterApply } = await backups;
|
||||
const refusal = code => ({ refused: new ApiError(409, 'x', code, null, { success: false, code }) });
|
||||
assert.equal(afterApply({ job: { id: 'j' } }), 'follow');
|
||||
assert.equal(afterApply(refusal('busy')), 'reopen');
|
||||
for (const outcome of [refusal('plan-used'), refusal('plan-expired'), refusal('plan-changed'), refusal('storage-changed'),
|
||||
{ refused: new TypeError('offline') }, { lost: true }, undefined]) {
|
||||
assert.equal(afterApply(outcome), 'release');
|
||||
}
|
||||
});
|
||||
|
||||
test('a restore job that hands its review back reopens it once in the page that started it; any other end forgets it (I1)', async () => {
|
||||
const { rememberRestore, onRestoreHandedBack } = await backups;
|
||||
let list = [];
|
||||
const listeners = new Set();
|
||||
const activity = { jobs: () => list, subscribe(listener) { listeners.add(listener); return () => listeners.delete(listener); } };
|
||||
const emit = next => { list = next; for (const listener of listeners) listener(); };
|
||||
const reopened = [];
|
||||
const stop = onRestoreHandedBack(activity, review => reopened.push(review.token));
|
||||
rememberRestore('kept', { token: 'k' });
|
||||
rememberRestore('done', { token: 'd' });
|
||||
rememberRestore('gone', { token: 'g' });
|
||||
emit([{ id: 'kept', kind: 'restore', state: 'running', scope: {} }]);
|
||||
emit([{ id: 'kept', kind: 'restore', state: 'failed', scope: { failure: 'busy', review_kept: true } },
|
||||
{ id: 'done', kind: 'restore', state: 'finished', scope: {} },
|
||||
{ id: 'gone', kind: 'restore', state: 'failed', scope: { failure: 'busy' } }]);
|
||||
emit(list);
|
||||
assert.deepEqual(reopened, ['k']);
|
||||
stop();
|
||||
assert.equal(listeners.size, 0);
|
||||
});
|
||||
|
||||
test('small files show KB; a full review store says what to do (N3, I1)', async () => {
|
||||
const { snapshotText, reviewRefusal, EXPORTS } = await backups;
|
||||
assert.equal(snapshotText({ name: 'a.db', size: 40 * 1024 }), 'a.db · 40 KB');
|
||||
assert.equal(snapshotText({ name: 'a.db', size: 10 }), 'a.db · 1 KB');
|
||||
assert.equal(snapshotText({ name: 'a.db', size: 3 * 2 ** 20 }), 'a.db · 3.0 MB');
|
||||
const full = new ApiError(429, 'Two restores are already being reviewed. Discard one first.', 'too-large', null,
|
||||
{ success: false, code: 'too-large' });
|
||||
assert.equal(reviewRefusal(full), 'Two restores are already being reviewed in another window. Discard one there, ' +
|
||||
'or try again when it expires (15 minutes after it was reviewed).');
|
||||
// N1: /export is the legacy list of every prompt.
|
||||
assert.deepEqual(EXPORTS[1].slice(0, 2), ['Export all prompts JSON', '/prompt_manager/export']);
|
||||
});
|
||||
@@ -34,14 +34,17 @@ test('metric 4 is the last health result', async () => {
|
||||
});
|
||||
|
||||
test('library health rows never invent a review count, a backup or thumbnails', async () => {
|
||||
const { ambiguousTitle, backupText } = await overviewModule;
|
||||
const { ambiguousTitle } = await overviewModule;
|
||||
// S9: the Backup status row is Backup & export's status line (system-backups.mjs, tested there).
|
||||
const { backupStatusText } = await import('../../web/js/workspace/system-backups.mjs');
|
||||
assert.equal(ambiguousTitle({ pending: null }), 'Ambiguous image links');
|
||||
assert.equal(ambiguousTitle(undefined), 'Ambiguous image links');
|
||||
assert.equal(ambiguousTitle({ pending: 1 }), '1 ambiguous image link');
|
||||
assert.equal(ambiguousTitle({ pending: 3 }), '3 ambiguous image links');
|
||||
assert.equal(backupText(null), 'No retained backup record.');
|
||||
assert.equal(backupText({ verified_at: 'not a time' }), 'No retained backup record.');
|
||||
assert.match(backupText({ verified_at: '2026-10-22T09:30:00Z' }), /^Last verified backup .+\.$/);
|
||||
assert.equal(backupStatusText({ safety_backups: [], last_download: null }), 'No backup recorded.');
|
||||
assert.equal(backupStatusText({ safety_backups: [{ modified_at: 'not a time' }], last_download: null }), 'No backup recorded.');
|
||||
assert.match(backupStatusText({ safety_backups: [], last_download: { at: '2026-10-22T09:30:00Z', verified: true } }),
|
||||
/^Downloaded backup · .+ · verified\.$/);
|
||||
});
|
||||
|
||||
test('the thumbnail row is its own read of the thumbnail status', async () => {
|
||||
|
||||
@@ -889,7 +889,7 @@ test('review dialog backup box keeps the dialog open and toasts the outcome', as
|
||||
assert.equal(action.getAttribute('aria-disabled'), 'true');
|
||||
// A second press while the copy downloads is ignored.
|
||||
action.click();
|
||||
answer({ ok: true, blob: async () => new Blob(['db']), headers: { get: () => 'attachment; filename="prompts_backup_1.db"' } });
|
||||
answer({ ok: true, blob: async () => new Blob(['db']), headers: { get: name => (name === 'Content-Disposition' ? 'attachment; filename="prompts_backup_1.db"' : null) } });
|
||||
await tick();
|
||||
assert.deepEqual(toasts, [BACKUP_DONE]);
|
||||
assert.equal(modal(dom).open, true);
|
||||
@@ -933,3 +933,37 @@ test('the scan review is a thin openReviewDialog caller with its scope switch',
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
test('a review dialog reports one dismissal for Cancel, ×, Escape, a replacing dialog and the page leaving, never after its primary (S9 M1)', async () => {
|
||||
const { openReviewDialog, openContentDialog, closeWorkspaceDialog } = await dialogsModule;
|
||||
const { createScope } = await coreModule;
|
||||
const dom = fakeDOM();
|
||||
const scope = createScope();
|
||||
const dismissed = [];
|
||||
const open = (name, primary = 'Restore') => openReviewDialog({ scope, title: 'Restore preview', grid: [['Integrity', 'Verified']],
|
||||
cancelLabel: 'Cancel', primary: { label: primary, danger: true }, onStart() {}, onDismiss: () => dismissed.push(name) });
|
||||
try {
|
||||
open('cancel');
|
||||
button(byId(dom, 'modal-body'), 'Cancel').click();
|
||||
open('close');
|
||||
byId(dom, 'modal-close').click();
|
||||
open('escape');
|
||||
modal(dom).dispatchEvent(new Event('cancel', { cancelable: true }));
|
||||
open('replaced');
|
||||
openContentDialog({ scope, title: 'Other', nodes: [] });
|
||||
closeWorkspaceDialog();
|
||||
open('started');
|
||||
button(byId(dom, 'modal-body'), 'Restore').click();
|
||||
await tick();
|
||||
open('left');
|
||||
scope.dispose();
|
||||
await tick();
|
||||
assert.deepEqual(dismissed, ['cancel', 'close', 'escape', 'replaced', 'left']);
|
||||
assert.throws(() => openReviewDialog({ scope: createScope(), title: 'x', grid: [['a', 'b']], cancelLabel: 'Cancel',
|
||||
primary: { label: 'Go' }, onStart() {}, onDismiss: 'no' }), TypeError);
|
||||
} finally {
|
||||
scope.dispose();
|
||||
closeWorkspaceDialog();
|
||||
dom.restore();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -2466,6 +2466,7 @@ class TestActivityJobs(AdminAPITestCase):
|
||||
ambiguous=None,
|
||||
untraced=None,
|
||||
records=None,
|
||||
history=None,
|
||||
):
|
||||
key = Path(media_file).relative_to(self.output_dir).as_posix()
|
||||
outcome = names[key]
|
||||
|
||||
@@ -82,6 +82,26 @@ class TracesPrompts(unittest.TestCase):
|
||||
self.assertEqual(result["positivePrompt"], "the real positive")
|
||||
self.assertEqual(result["negativePrompt"], "the real negative")
|
||||
|
||||
def test_core_sampler_with_an_out_of_order_widget_input_keeps_positions(self):
|
||||
# #343 review R4: only ``cfg`` is widget-bound, so the bound names are not
|
||||
# a schema prefix; the values follow the KSampler schema positions.
|
||||
result = run("workflow-only-core-widget-input-out-of-order")
|
||||
self.assertEqual(
|
||||
(result["seed"], result["steps"], result["cfgScale"], result["sampler"]),
|
||||
(9, 30, 4.5, "dpmpp_2m"),
|
||||
)
|
||||
|
||||
def test_editor_text_inputs_without_widget_bindings_keep_their_positions(self):
|
||||
# #343: Focused/Split/Pair text inputs are plain sockets. The widget-bound
|
||||
# inputs that remain (category, tags, search_text) must not shift the
|
||||
# positional widget values onto the wrong names.
|
||||
result = run("workflow-only-detached-editor-text")
|
||||
self.assertEqual(result["positivePrompt"], "a detached positive")
|
||||
self.assertEqual(result["negativePrompt"], "plain negative")
|
||||
result = run("workflow-only-detached-pair-text")
|
||||
self.assertEqual(result["positivePrompt"], "a detached positive")
|
||||
self.assertEqual(result["negativePrompt"], "a detached negative")
|
||||
|
||||
|
||||
class ChoosesTheSampler(unittest.TestCase):
|
||||
def test_first_sampler_follows_javascript_key_order(self):
|
||||
|
||||
+17
-13
@@ -12,6 +12,7 @@ from database.operations import PromptDatabase
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.image_monitor import ImageGenerationHandler
|
||||
from utils.prompt_tracker import PromptTracker
|
||||
from utils.run_keys import prompt_node_key
|
||||
|
||||
LOADER = {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
@@ -103,8 +104,11 @@ class TestMetadataLinking(LinkingTestCase):
|
||||
batch_graph = graph()
|
||||
batch_graph["134"]["inputs"]["text"] = ["7", 0]
|
||||
batch_graph["7"] = {"class_type": "PromptSearchList", "inputs": {}}
|
||||
self.tracker.set_current_prompt("item one", {"prompt_id": first})
|
||||
self.tracker.set_current_prompt("item two", {"prompt_id": second})
|
||||
key = prompt_node_key(batch_graph, "134")
|
||||
for text, prompt_id in (("item one", first), ("item two", second)):
|
||||
self.tracker.set_current_prompt(
|
||||
text, {"prompt_id": prompt_id}, run_key=key, run_id="run-1"
|
||||
)
|
||||
|
||||
self.assertEqual(self._process({"prompt": batch_graph}), first)
|
||||
self.assertEqual(self._process({"prompt": batch_graph}), second)
|
||||
@@ -122,9 +126,10 @@ class TestMetadataLinking(LinkingTestCase):
|
||||
self.assertEqual(
|
||||
self.handler._find_prompt_from_metadata(metadata), ("unknown", None)
|
||||
)
|
||||
latest = self._save("most recent prompt") # recency fallback picks this
|
||||
self.assertEqual(self._process(metadata), latest)
|
||||
self.assertNotEqual(latest, negative)
|
||||
self._save("most recent prompt") # never picked by recency (#342)
|
||||
self.assertIsNone(self._process(metadata))
|
||||
self.assertEqual(self.linked, [])
|
||||
self.assertIsNotNone(negative)
|
||||
|
||||
def test_workflow_fallback_still_used_without_an_api_graph(self):
|
||||
positive = self._save("from workflow only")
|
||||
@@ -200,12 +205,14 @@ class TestForeignWorkflows(LinkingTestCase):
|
||||
self.handler._find_prompt_from_metadata(metadata), ("foreign", None)
|
||||
)
|
||||
|
||||
def test_missing_metadata_is_unknown_and_uses_the_queue(self):
|
||||
def test_missing_metadata_is_unknown_and_left_unlinked(self):
|
||||
first = self._queue_batch_item("item one")
|
||||
self.assertEqual(
|
||||
self.handler._find_prompt_from_metadata(None), ("unknown", None)
|
||||
)
|
||||
self.assertEqual(self._process(None), first)
|
||||
# Nothing proves which run made it (#342): unlinked, queue untouched
|
||||
self.assertIsNone(self._process(None))
|
||||
self.assertEqual(self.tracker.pop_next_prompt()["id"], first)
|
||||
|
||||
def test_tri_state_for_linked_unknown_and_foreign(self):
|
||||
positive = self._save("a cat")
|
||||
@@ -224,14 +231,11 @@ class TestForeignWorkflows(LinkingTestCase):
|
||||
("unknown", None),
|
||||
)
|
||||
|
||||
def test_unknown_image_without_any_context_falls_back_to_the_latest_prompt(self):
|
||||
def test_unknown_image_is_never_given_the_latest_prompt(self):
|
||||
self._save("older")
|
||||
latest = self._save("latest")
|
||||
self.assertEqual(self._process(None), latest)
|
||||
|
||||
def test_recent_prompt_fallback_handles_an_empty_database(self):
|
||||
self.assertIsNone(self.handler._get_fallback_prompt())
|
||||
self._save("latest")
|
||||
self.assertIsNone(self._process(None))
|
||||
self.assertIsNone(self._process({"prompt": graph("not saved")}))
|
||||
|
||||
def test_metadata_lookup_errors_are_unknown(self):
|
||||
self._save("a cat")
|
||||
|
||||
@@ -326,7 +326,7 @@ class TestHandlerScheduling(unittest.TestCase):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.tracker = MagicMock()
|
||||
self.tracker.get_current_prompt.return_value = {"id": 7}
|
||||
self.tracker.keyed_snapshot.return_value = {"key": ({"id": 7}, "run", 1)}
|
||||
self.handler = im_mod.ImageGenerationHandler(MagicMock(), self.tracker)
|
||||
self.handler.processing_delay = 0
|
||||
self.handler.settle_interval = 0.01
|
||||
@@ -354,7 +354,14 @@ class TestHandlerScheduling(unittest.TestCase):
|
||||
self.handler.on_created(FakeEvent(os.path.join(self.tmp.name, "c.txt")))
|
||||
self.handler.on_created(FakeEvent(self.tmp.name, is_directory=True))
|
||||
self.assertTrue(wait_for(lambda: self.handler.pending_count() == 0))
|
||||
self.assertEqual(self.processed, [(path, {"id": 7})])
|
||||
self.assertEqual(self.processed, [(path, {"key": ({"id": 7}, "run", 1)})])
|
||||
|
||||
def test_a_failed_snapshot_still_schedules_the_image(self):
|
||||
self.tracker.keyed_snapshot.side_effect = RuntimeError("tracker gone")
|
||||
path = self._image("d.png")
|
||||
self.handler.on_created(FakeEvent(path))
|
||||
self.assertTrue(wait_for(lambda: self.handler.pending_count() == 0))
|
||||
self.assertEqual(self.processed, [(path, None)])
|
||||
|
||||
def test_images_are_processed_in_creation_order(self):
|
||||
paths = [self._image(f"{i}.png") for i in range(5)]
|
||||
|
||||
@@ -239,10 +239,13 @@ class LoraImportAPITestCase(LoraAPITestCase):
|
||||
self.requests = []
|
||||
self.gate = None # a threading.Event a download waits on
|
||||
self.downloading = threading.Event()
|
||||
self.failing = set() # URLs whose download fails
|
||||
|
||||
def fake_open(req, timeout=None):
|
||||
self.requests.append(req)
|
||||
self.downloading.set()
|
||||
if req.full_url in self.failing:
|
||||
raise OSError("connection reset")
|
||||
if self.gate is not None:
|
||||
self.gate.wait(10)
|
||||
return _FakeResponse(_png_bytes())
|
||||
@@ -597,6 +600,114 @@ class LoraImportRouteTests(LoraImportAPITestCase):
|
||||
self.assertEqual(sorted(p["text"] for p in imported), ["neon street", "plain"])
|
||||
|
||||
|
||||
class LoraImportDownloadOutcomeTests(LoraImportAPITestCase):
|
||||
"""#337: failed CivitAI downloads show in the job's status line."""
|
||||
|
||||
URLS = ["https://civitai.com/a.png", "https://civitai.com/b.png"]
|
||||
|
||||
def _lora_with_images(self):
|
||||
_write_metadata(
|
||||
self.loras,
|
||||
"neon",
|
||||
words=["glow"],
|
||||
images=list(self.URLS),
|
||||
prompt="neon street",
|
||||
)
|
||||
_write_metadata(self.loras, "plain")
|
||||
|
||||
async def _run(self):
|
||||
_plan, job = await self._apply_reviewed()
|
||||
await job.task
|
||||
return job
|
||||
|
||||
async def test_status_line_unchanged_when_every_download_succeeds(self):
|
||||
self._lora_with_images()
|
||||
|
||||
job = await self._run()
|
||||
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(job.status_line, "Imported 2 LoRAs")
|
||||
self.assertEqual(job.outcomes["download_errors"], [])
|
||||
|
||||
async def test_status_line_counts_failed_downloads(self):
|
||||
self._lora_with_images()
|
||||
self.failing = {self.URLS[1]}
|
||||
|
||||
job = await self._run()
|
||||
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(
|
||||
(job.counters["downloaded"], job.counters["download_errors"]), (1, 1)
|
||||
)
|
||||
self.assertEqual(
|
||||
job.status_line,
|
||||
"Imported 2 LoRAs · 1 example image could not be downloaded",
|
||||
)
|
||||
self.assertNotIn("Warning", job.status_line)
|
||||
self.assertEqual(job.outcomes["download_errors"], [])
|
||||
|
||||
async def test_every_download_failing_is_a_warning(self):
|
||||
self._lora_with_images()
|
||||
self.failing = set(self.URLS)
|
||||
|
||||
job = await self._run()
|
||||
|
||||
# The import itself succeeded, so the job still finishes.
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(
|
||||
(job.counters["downloaded"], job.counters["download_errors"]), (0, 2)
|
||||
)
|
||||
warning = "Warning: none of the 2 example images could be downloaded"
|
||||
self.assertEqual(job.status_line, f"{warning} · Imported 2 LoRAs")
|
||||
self.assertEqual(
|
||||
job.outcomes["download_errors"],
|
||||
[
|
||||
{
|
||||
"path": "",
|
||||
"root": "",
|
||||
"root_index": None,
|
||||
"detail": f"{warning}; the log has the details",
|
||||
}
|
||||
],
|
||||
)
|
||||
self.assertTrue(any(warning in line for line in job.log), job.log)
|
||||
self.assertEqual(job.summary()["status_line"], job.status_line)
|
||||
self.assertNotIn(SECRET_KEY, json.dumps(job.report()))
|
||||
|
||||
async def test_one_planned_download_failing_reads_in_the_singular(self):
|
||||
_write_metadata(self.loras, "neon", images=[self.URLS[0]], prompt="neon street")
|
||||
self.failing = {self.URLS[0]}
|
||||
|
||||
job = await self._run()
|
||||
|
||||
self.assertEqual(
|
||||
job.status_line,
|
||||
"Warning: the example image could not be downloaded · Imported 1 LoRA",
|
||||
)
|
||||
|
||||
async def test_legacy_scan_events_unchanged_when_every_download_fails(self):
|
||||
self._lora_with_images()
|
||||
self.failing = set(self.URLS)
|
||||
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
events = self._sse(await resp.text())
|
||||
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(
|
||||
events[-1],
|
||||
{
|
||||
"type": "complete",
|
||||
"progress": 100,
|
||||
"total": 2,
|
||||
"imported": 2,
|
||||
"skipped": 0,
|
||||
},
|
||||
)
|
||||
for event in events[:-1]:
|
||||
self.assertEqual(event["type"], "progress")
|
||||
self.assertNotIn("Warning", event.get("status", ""))
|
||||
|
||||
|
||||
class LoraImportRefusalTests(LoraImportAPITestCase):
|
||||
"""Every refusal leaves the previous imports as they were."""
|
||||
|
||||
|
||||
+503
-7
@@ -15,6 +15,7 @@ import threading
|
||||
import time
|
||||
import types
|
||||
import unittest
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import urllib.response
|
||||
@@ -295,12 +296,9 @@ class TestCivitaiUrlPolicy(unittest.TestCase):
|
||||
self.assertEqual(
|
||||
CIVITAI_HOSTS,
|
||||
frozenset(
|
||||
{
|
||||
"civitai.com",
|
||||
"www.civitai.com",
|
||||
"api.civitai.com",
|
||||
"image.civitai.com",
|
||||
}
|
||||
f"{prefix}{domain}"
|
||||
for domain in ("civitai.com", "civitai.red", "civitai.green")
|
||||
for prefix in ("", "www.", "api.", "image.")
|
||||
),
|
||||
)
|
||||
self.assertEqual(MAX_CIVITAI_DOWNLOAD_BYTES, 50 * 1024 * 1024)
|
||||
@@ -333,6 +331,36 @@ class TestCivitaiUrlPolicy(unittest.TestCase):
|
||||
def test_explicit_default_port_is_allowed(self):
|
||||
self.assertTrue(is_civitai_url("https://civitai.com:443/x"))
|
||||
|
||||
def test_red_and_green_starting_hosts_are_allowed(self):
|
||||
# #340: CivitAI's own .red and .green domains serve the same content.
|
||||
for url in (
|
||||
"https://image.civitai.red/xG1/abc/width=450/1.jpeg",
|
||||
"https://civitai.red/api/download/1",
|
||||
"https://civitai.green/api/download/1",
|
||||
"https://image.civitai.green/xG1/abc/1.jpeg",
|
||||
"https://IMAGE.CIVITAI.RED/x",
|
||||
"https://civitai.green:443/x",
|
||||
):
|
||||
self.assertTrue(is_civitai_url(url), url)
|
||||
|
||||
def test_red_and_green_lookalikes_are_refused(self):
|
||||
for url in (
|
||||
"http://image.civitai.red/x",
|
||||
"http://civitai.green/x",
|
||||
"https://civitai.red.evil.example/x",
|
||||
"https://civitai.green.evil.example/x",
|
||||
"https://evilcivitai.red/x",
|
||||
"https://evilcivitai.green/x",
|
||||
"https://civitai.redx/x",
|
||||
"https://civitai.greenx/x",
|
||||
"https://civitai.red@evil.example/x",
|
||||
"https://user:pw@civitai.red/x",
|
||||
"https://civitai.green:8443/x",
|
||||
"https://blobs-b2.civitai.red/x", # starting hosts stay a fixed list
|
||||
"https://evil.example/civitai.red",
|
||||
):
|
||||
self.assertFalse(is_civitai_url(url), url)
|
||||
|
||||
|
||||
class TestLoraManagerPathPolicy(unittest.TestCase):
|
||||
"""Only a direct child of a ComfyUI custom_nodes dir named *lora* qualifies."""
|
||||
@@ -686,7 +714,7 @@ class TestRedirectsAreRefused(unittest.TestCase):
|
||||
if host not in CIVITAI_HOSTS:
|
||||
self.assertIsNone(req.get_header("Authorization"))
|
||||
|
||||
def test_redirect_to_civitai_is_not_followed_either(self):
|
||||
def test_redirect_to_civitai_is_followed_without_the_key(self):
|
||||
handler = _FakeHTTPSHandler("https://image.civitai.com/other.png")
|
||||
opener = lora_utils._build_opener(handler)
|
||||
|
||||
@@ -695,9 +723,477 @@ class TestRedirectsAreRefused(unittest.TestCase):
|
||||
"https://civitai.com/api/download/1", self.tmpdir / "x.jpg", "key"
|
||||
)
|
||||
|
||||
# image.civitai.com redirects again (the fake handler always does),
|
||||
# so the hop limit ends this chain; the first request alone had the key.
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(len(handler.requests), lora_utils.MAX_CIVITAI_REDIRECTS + 1)
|
||||
self.assertEqual(handler.requests[0].get_header("Authorization"), "Bearer key")
|
||||
for req in handler.requests[1:]:
|
||||
self.assertIsNone(req.get_header("Authorization"))
|
||||
|
||||
|
||||
class _ChainHTTPSHandler(urllib.request.HTTPSHandler):
|
||||
"""Serves ``routes`` (url -> (status, Location or None)) through a real
|
||||
urllib opener, recording every request; an unknown URL answers 404."""
|
||||
|
||||
def __init__(self, routes):
|
||||
super().__init__()
|
||||
self.routes = routes
|
||||
self.requests = []
|
||||
|
||||
def https_open(self, req):
|
||||
self.requests.append(req)
|
||||
status, location = self.routes.get(req.full_url, (404, None))
|
||||
if location is not None:
|
||||
headers = email.message_from_string(f"Location: {location}\n")
|
||||
resp = urllib.response.addinfourl(
|
||||
BytesIO(b""), headers, req.full_url, status
|
||||
)
|
||||
resp.msg = "Redirect"
|
||||
return resp
|
||||
body = _png_bytes() if status == 200 else b""
|
||||
headers = email.message_from_string("Content-Type: image/png\n")
|
||||
resp = urllib.response.addinfourl(BytesIO(body), headers, req.full_url, status)
|
||||
resp.msg = "OK" if status == 200 else "Not Found"
|
||||
return resp
|
||||
|
||||
def http_open(self, req): # a plain-HTTP hop must never be requested
|
||||
self.requests.append(req)
|
||||
raise AssertionError(f"plain HTTP requested: {req.full_url}")
|
||||
|
||||
|
||||
START = "https://image.civitai.com/xG1/abc/width=450/1.jpeg"
|
||||
BLOB = "https://blobs-b2.civitai.com/file/civitai-media-cache/1/original"
|
||||
|
||||
|
||||
class TestCivitaiRedirects(unittest.TestCase):
|
||||
"""#337: image.civitai.com now redirects to blobs-b2.civitai.com.
|
||||
|
||||
A redirect is followed only over HTTPS to civitai.com or a subdomain of
|
||||
it, at most MAX_CIVITAI_REDIRECTS hops, and never with the API key.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
self.tmpdir = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, self.tmpdir, True)
|
||||
self.target = self.tmpdir / "x.jpg"
|
||||
|
||||
def _download(self, routes, url=START, api_key="secret-key"):
|
||||
handler = _ChainHTTPSHandler(routes)
|
||||
opener = lora_utils._build_opener(handler)
|
||||
with patch.object(lora_utils, "_opener", opener):
|
||||
result = lora_utils._download_one(url, self.target, api_key)
|
||||
return result, handler.requests
|
||||
|
||||
def _refused(self, location, status=302):
|
||||
result, requests = self._download({START: (status, location)})
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual([r.full_url for r in requests], [START])
|
||||
self.assertFalse(self.target.exists())
|
||||
|
||||
def test_redirect_to_an_allowed_host_succeeds_without_the_key(self):
|
||||
for status in (301, 302, 303, 307, 308):
|
||||
with self.subTest(status=status):
|
||||
if self.target.exists():
|
||||
os.remove(self.target)
|
||||
result, requests = self._download(
|
||||
{START: (status, BLOB), BLOB: (200, None)}
|
||||
)
|
||||
|
||||
self.assertIsNotNone(result)
|
||||
self.assertTrue(self.target.exists())
|
||||
self.assertEqual([r.full_url for r in requests], [START, BLOB])
|
||||
self.assertEqual(
|
||||
requests[0].get_header("Authorization"), "Bearer secret-key"
|
||||
)
|
||||
self.assertIsNone(requests[1].get_header("Authorization"))
|
||||
# urllib adds Host itself; nothing else rides along.
|
||||
self.assertEqual(
|
||||
[h for h in requests[1].header_items() if h[0] != "Host"],
|
||||
[("User-agent", "ComfyUI-PromptManager/1.0")],
|
||||
)
|
||||
|
||||
def test_a_relative_location_resolves_on_the_same_host(self):
|
||||
follow = "https://image.civitai.com/elsewhere/1.jpeg"
|
||||
result, requests = self._download(
|
||||
{START: (302, "/elsewhere/1.jpeg"), follow: (200, None)}
|
||||
)
|
||||
|
||||
self.assertIsNotNone(result)
|
||||
self.assertEqual([r.full_url for r in requests], [START, follow])
|
||||
self.assertIsNone(requests[1].get_header("Authorization"))
|
||||
|
||||
def test_a_plain_http_redirect_is_refused(self):
|
||||
self._refused("http://blobs-b2.civitai.com/file/1")
|
||||
|
||||
def test_a_redirect_to_another_host_is_refused(self):
|
||||
self._refused("https://evil.example/steal")
|
||||
|
||||
def test_lookalike_hosts_are_refused(self):
|
||||
for location in (
|
||||
"https://civitai.com.evil.example/x",
|
||||
"https://evilcivitai.com/x",
|
||||
"https://civitai.com@evil.example/x",
|
||||
"https://user:pw@blobs-b2.civitai.com/x",
|
||||
"https://blobs-b2.civitai.com:8443/x",
|
||||
"https://civitai.co/x",
|
||||
"//evil.example/x",
|
||||
"ftp://blobs-b2.civitai.com/x",
|
||||
"https://[::1]/x",
|
||||
):
|
||||
with self.subTest(location=location):
|
||||
self._refused(location)
|
||||
|
||||
def test_a_redirect_without_a_location_is_refused(self):
|
||||
result, requests = self._download({START: (302, "")})
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(len(requests), 1)
|
||||
|
||||
def test_the_hop_limit_is_enforced(self):
|
||||
limit = lora_utils.MAX_CIVITAI_REDIRECTS
|
||||
self.assertEqual(limit, 3)
|
||||
hops = [f"https://blobs-b2.civitai.com/hop/{i}" for i in range(limit + 1)]
|
||||
|
||||
# Exactly ``limit`` redirects: followed.
|
||||
routes = {START: (302, hops[0])}
|
||||
for i in range(limit - 1):
|
||||
routes[hops[i]] = (302, hops[i + 1])
|
||||
routes[hops[limit - 1]] = (200, None)
|
||||
result, requests = self._download(routes)
|
||||
self.assertIsNotNone(result)
|
||||
self.assertEqual(len(requests), limit + 1)
|
||||
self.assertTrue(
|
||||
all(r.get_header("Authorization") is None for r in requests[1:])
|
||||
)
|
||||
|
||||
# One more redirect: refused, and the next hop is never requested.
|
||||
os.remove(self.target)
|
||||
routes[hops[limit - 1]] = (302, hops[limit])
|
||||
routes[hops[limit]] = (200, None)
|
||||
result, requests = self._download(routes)
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(len(requests), limit + 1)
|
||||
self.assertNotIn(hops[limit], [r.full_url for r in requests])
|
||||
self.assertFalse(self.target.exists())
|
||||
|
||||
def test_a_redirect_loop_ends_at_the_hop_limit(self):
|
||||
result, requests = self._download({START: (302, START)})
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(len(requests), lora_utils.MAX_CIVITAI_REDIRECTS + 1)
|
||||
|
||||
def test_size_cap_applies_after_a_redirect(self):
|
||||
with patch.object(lora_utils, "MAX_CIVITAI_DOWNLOAD_BYTES", 10):
|
||||
result, requests = self._download({START: (302, BLOB), BLOB: (200, None)})
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(len(requests), 2)
|
||||
self.assertFalse(self.target.exists())
|
||||
|
||||
def test_an_error_after_a_redirect_fails_cleanly(self):
|
||||
result, requests = self._download({START: (302, BLOB)}) # BLOB is 404
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(len(requests), 2)
|
||||
self.assertFalse(self.target.exists())
|
||||
|
||||
def test_the_opener_itself_still_refuses_every_redirect(self):
|
||||
handler = _ChainHTTPSHandler({START: (302, BLOB), BLOB: (200, None)})
|
||||
opener = lora_utils._build_opener(handler)
|
||||
req = urllib.request.Request(START, headers={"Authorization": "Bearer k"})
|
||||
|
||||
with self.assertRaises(urllib.error.HTTPError) as caught:
|
||||
opener.open(req, timeout=5)
|
||||
caught.exception.close()
|
||||
self.assertEqual(len(handler.requests), 1)
|
||||
|
||||
def test_redirect_logs_never_carry_the_key_or_the_query(self):
|
||||
signed = BLOB + "?X-Amz-Signature=sig123"
|
||||
with self.assertLogs("prompt_manager.lora_utils", level="DEBUG") as logs:
|
||||
self._download(
|
||||
{START: (302, "https://evil.example/x?token=t0k3n")},
|
||||
api_key="secret-key",
|
||||
)
|
||||
self._download({START: (302, signed), signed: (200, None)})
|
||||
text = "\n".join(logs.output)
|
||||
self.assertNotIn("secret-key", text)
|
||||
self.assertNotIn("t0k3n", text)
|
||||
self.assertNotIn("sig123", text)
|
||||
|
||||
|
||||
RED_BLOB = "https://blobs-b2.civitai.red/file/civitai-media-cache/1/original"
|
||||
GREEN_BLOB = "https://media.civitai.green/file/1/original"
|
||||
|
||||
|
||||
class TestCivitaiRedAndGreenDomains(unittest.TestCase):
|
||||
"""#340: civitai.red and civitai.green are CivitAI's own domains.
|
||||
|
||||
They are redirect targets under the civitai.com rules (HTTPS, default
|
||||
port, no credentials, parsed host name, 3 hops, never the key) and
|
||||
starting hosts whose first request may carry the key.
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
self.tmpdir = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, self.tmpdir, True)
|
||||
self.target = self.tmpdir / "x.jpg"
|
||||
|
||||
def _download(self, routes, url=START, api_key="secret-key"):
|
||||
handler = _ChainHTTPSHandler(routes)
|
||||
opener = lora_utils._build_opener(handler)
|
||||
with patch.object(lora_utils, "_opener", opener):
|
||||
result = lora_utils._download_one(url, self.target, api_key)
|
||||
return result, handler.requests
|
||||
|
||||
def test_redirect_targets_on_red_and_green_are_allowed(self):
|
||||
for url in (
|
||||
"https://civitai.red/x",
|
||||
"https://civitai.green/x",
|
||||
RED_BLOB,
|
||||
GREEN_BLOB,
|
||||
"https://a.b.civitai.red:443/x",
|
||||
"https://CIVITAI.GREEN/x",
|
||||
):
|
||||
self.assertTrue(lora_utils.is_civitai_redirect_target(url), url)
|
||||
|
||||
def test_redirect_target_lookalikes_are_refused(self):
|
||||
for url in (
|
||||
"https://civitai.red.evil.example/x",
|
||||
"https://civitai.green.evil.example/x",
|
||||
"https://evilcivitai.red/x",
|
||||
"https://evilcivitai.green/x",
|
||||
"https://civitai.redx/x",
|
||||
"https://civitai.greenx/x",
|
||||
"https://civitai.red@evil.example/x",
|
||||
"https://user:pw@blobs-b2.civitai.red/x",
|
||||
"https://user@civitai.green/x",
|
||||
"https://blobs-b2.civitai.red:8443/x",
|
||||
"http://blobs-b2.civitai.red/x",
|
||||
"http://civitai.green/x",
|
||||
"ftp://civitai.red/x",
|
||||
"https://evil.example/civitai.red",
|
||||
"https://evil.example/?h=civitai.green",
|
||||
"https://red/x",
|
||||
"https://.civitai.red.evil/x",
|
||||
):
|
||||
self.assertFalse(lora_utils.is_civitai_redirect_target(url), url)
|
||||
|
||||
def test_a_redirect_to_red_or_green_is_followed_without_the_key(self):
|
||||
for blob in (RED_BLOB, GREEN_BLOB):
|
||||
with self.subTest(blob=blob):
|
||||
if self.target.exists():
|
||||
os.remove(self.target)
|
||||
result, requests = self._download(
|
||||
{START: (302, blob), blob: (200, None)}
|
||||
)
|
||||
|
||||
self.assertIsNotNone(result)
|
||||
self.assertEqual([r.full_url for r in requests], [START, blob])
|
||||
self.assertEqual(
|
||||
requests[0].get_header("Authorization"), "Bearer secret-key"
|
||||
)
|
||||
self.assertIsNone(requests[1].get_header("Authorization"))
|
||||
|
||||
def test_lookalike_and_plain_http_redirects_are_refused(self):
|
||||
for location in (
|
||||
"https://civitai.red.evil.example/x",
|
||||
"https://evilcivitai.red/x",
|
||||
"https://civitai.redx/x",
|
||||
"https://civitai.green.evil.example/x",
|
||||
"https://evilcivitai.green/x",
|
||||
"https://civitai.greenx/x",
|
||||
"http://blobs-b2.civitai.red/x",
|
||||
"http://civitai.green/x",
|
||||
):
|
||||
with self.subTest(location=location):
|
||||
result, requests = self._download({START: (302, location)})
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual([r.full_url for r in requests], [START])
|
||||
self.assertFalse(self.target.exists())
|
||||
|
||||
def test_a_red_starting_host_gets_the_key_and_its_redirect_does_not(self):
|
||||
start = "https://image.civitai.red/xG1/abc/width=450/1.jpeg"
|
||||
result, requests = self._download(
|
||||
{start: (302, BLOB), BLOB: (200, None)}, url=start
|
||||
)
|
||||
|
||||
self.assertIsNotNone(result)
|
||||
self.assertEqual([r.full_url for r in requests], [start, BLOB])
|
||||
self.assertEqual(requests[0].get_header("Authorization"), "Bearer secret-key")
|
||||
self.assertIsNone(requests[1].get_header("Authorization"))
|
||||
|
||||
def test_a_green_starting_host_downloads_directly(self):
|
||||
start = "https://civitai.green/api/download/1"
|
||||
result, requests = self._download({start: (200, None)}, url=start)
|
||||
|
||||
self.assertIsNotNone(result)
|
||||
self.assertEqual(requests[0].get_header("Authorization"), "Bearer secret-key")
|
||||
|
||||
def test_the_hop_limit_spans_domains(self):
|
||||
hops = [RED_BLOB, GREEN_BLOB, BLOB, "https://civitai.red/4"]
|
||||
routes = {START: (302, hops[0])}
|
||||
for here, there in zip(hops, hops[1:]):
|
||||
routes[here] = (302, there)
|
||||
routes[hops[-1]] = (200, None)
|
||||
|
||||
result, requests = self._download(routes)
|
||||
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(len(requests), lora_utils.MAX_CIVITAI_REDIRECTS + 1)
|
||||
self.assertNotIn(hops[-1], [r.full_url for r in requests])
|
||||
for req in requests[1:]:
|
||||
self.assertIsNone(req.get_header("Authorization"))
|
||||
|
||||
|
||||
class _RaisingOpener:
|
||||
"""An ``_open_url`` stand-in that raises ``error`` for every request."""
|
||||
|
||||
def __init__(self, error):
|
||||
self.error = error
|
||||
self.requests = []
|
||||
|
||||
def __call__(self, req, timeout=None):
|
||||
self.requests.append(req)
|
||||
raise self.error
|
||||
|
||||
|
||||
class TestDownloadFailureWarnings(unittest.TestCase):
|
||||
"""#340: each failed download logs one warning line with a short reason,
|
||||
the URL without its query or credentials, and never the key."""
|
||||
|
||||
LOGGER = "prompt_manager.lora_utils"
|
||||
SIGNED = START + "?token=t0k3n"
|
||||
|
||||
def setUp(self):
|
||||
self.tmpdir = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, self.tmpdir, True)
|
||||
self.target = self.tmpdir / "x.jpg"
|
||||
|
||||
def _warning(self, run):
|
||||
"""Run ``run`` and return its one WARNING-or-above record's text."""
|
||||
with self.assertLogs(self.LOGGER, level="WARNING") as logs:
|
||||
result = run()
|
||||
self.assertIsNone(result)
|
||||
self.assertFalse(self.target.exists())
|
||||
self.assertEqual(len(logs.records), 1, logs.output)
|
||||
record = logs.records[0]
|
||||
self.assertEqual(record.levelname, "WARNING")
|
||||
text = record.getMessage()
|
||||
self.assertNotIn("secret-key", text)
|
||||
self.assertNotIn("t0k3n", text)
|
||||
self.assertNotIn("sig123", text)
|
||||
self.assertIn(START.split("?")[0], text)
|
||||
return text
|
||||
|
||||
def _chain(self, routes, url=SIGNED):
|
||||
handler = _ChainHTTPSHandler(routes)
|
||||
opener = lora_utils._build_opener(handler)
|
||||
|
||||
def run():
|
||||
with patch.object(lora_utils, "_opener", opener):
|
||||
return lora_utils._download_one(url, self.target, "secret-key")
|
||||
|
||||
return run
|
||||
|
||||
def _fake(self, opener, url=SIGNED):
|
||||
def run():
|
||||
with patch.object(lora_utils, "_open_url", opener):
|
||||
return lora_utils._download_one(url, self.target, "secret-key")
|
||||
|
||||
return run
|
||||
|
||||
def test_http_status(self):
|
||||
text = self._warning(self._chain({})) # unknown URL answers 404
|
||||
self.assertIn("HTTP 404", text)
|
||||
|
||||
def test_http_status_after_a_redirect(self):
|
||||
text = self._warning(self._chain({self.SIGNED: (302, BLOB)}))
|
||||
self.assertIn("HTTP 404", text)
|
||||
|
||||
def test_refused_redirect(self):
|
||||
text = self._warning(
|
||||
self._chain({self.SIGNED: (302, "https://evil.example/x?sig123=1")})
|
||||
)
|
||||
self.assertIn("redirect refused", text)
|
||||
self.assertIn("https://evil.example/x", text)
|
||||
|
||||
def test_hop_limit(self):
|
||||
text = self._warning(self._chain({self.SIGNED: (302, self.SIGNED)}))
|
||||
self.assertIn("more than 3 redirects", text)
|
||||
|
||||
def test_size_cap(self):
|
||||
opener = _FakeOpener(
|
||||
_FakeResponse(_png_bytes(), content_length=MAX_CIVITAI_DOWNLOAD_BYTES + 1)
|
||||
)
|
||||
text = self._warning(self._fake(opener))
|
||||
self.assertIn("larger than the 50 MB cap", text)
|
||||
|
||||
def test_decode_error(self):
|
||||
text = self._warning(self._fake(_FakeOpener(_FakeResponse(b"not an image"))))
|
||||
self.assertIn("image could not be decoded", text)
|
||||
|
||||
def test_timeout(self):
|
||||
for error in (
|
||||
TimeoutError("timed out"),
|
||||
urllib.error.URLError(TimeoutError("timed out")),
|
||||
):
|
||||
with self.subTest(error=type(error).__name__):
|
||||
text = self._warning(self._fake(_RaisingOpener(error)))
|
||||
self.assertIn("timed out", text)
|
||||
|
||||
def test_network_error(self):
|
||||
for error in (
|
||||
urllib.error.URLError("[Errno -2] Name or service not known"),
|
||||
ConnectionResetError(104, "Connection reset by peer"),
|
||||
):
|
||||
with self.subTest(error=type(error).__name__):
|
||||
text = self._warning(self._fake(_RaisingOpener(error)))
|
||||
self.assertIn("network error", text)
|
||||
|
||||
def test_a_timeout_while_reading_the_body(self):
|
||||
class SlowResponse(_FakeResponse):
|
||||
def read(self, amt=-1):
|
||||
raise TimeoutError("timed out")
|
||||
|
||||
text = self._warning(self._fake(_FakeOpener(SlowResponse(b""))))
|
||||
self.assertIn("timed out", text)
|
||||
|
||||
def test_a_failed_save(self):
|
||||
opener = _FakeOpener(_FakeResponse(_png_bytes()))
|
||||
|
||||
def run():
|
||||
with patch.object(
|
||||
lora_utils.os, "replace", side_effect=OSError(28, "No space left")
|
||||
):
|
||||
return self._fake(opener)()
|
||||
|
||||
text = self._warning(run)
|
||||
self.assertIn("could not be saved", text)
|
||||
|
||||
def test_a_refused_starting_url(self):
|
||||
def run():
|
||||
return lora_utils._download_one(
|
||||
"http://image.civitai.com/x?token=t0k3n", self.target, "secret-key"
|
||||
)
|
||||
|
||||
with self.assertLogs(self.LOGGER, level="WARNING") as logs:
|
||||
self.assertIsNone(run())
|
||||
self.assertEqual(len(logs.records), 1)
|
||||
text = logs.records[0].getMessage()
|
||||
self.assertIn("not an allowed CivitAI URL", text)
|
||||
self.assertNotIn("t0k3n", text)
|
||||
|
||||
def test_an_unexpected_error_names_only_its_type(self):
|
||||
error = ValueError("unknown url type: https://x?token=t0k3n")
|
||||
text = self._warning(self._fake(_RaisingOpener(error)))
|
||||
self.assertIn("unexpected ValueError", text)
|
||||
|
||||
def test_the_line_names_the_download(self):
|
||||
text = self._warning(self._chain({}))
|
||||
self.assertTrue(text.startswith("CivitAI image download failed"), text)
|
||||
|
||||
def test_a_successful_download_logs_no_warning(self):
|
||||
with self.assertNoLogs(self.LOGGER, level="WARNING"):
|
||||
result = self._fake(_FakeOpener(_FakeResponse(_png_bytes())))()
|
||||
self.assertIsNotNone(result)
|
||||
|
||||
|
||||
class TestDownloadCivitaiImages(unittest.TestCase):
|
||||
|
||||
|
||||
@@ -0,0 +1,662 @@
|
||||
"""#342: an image is linked only to a prompt that provably produced it.
|
||||
|
||||
The maintainer's run: an LLM writes the prompt into a PromptManagerPair's
|
||||
``prepend_text`` while its ``text`` editor is empty. Prepend and append text are
|
||||
never stored as the prompt (product decision), so nothing is saved, and the
|
||||
image must stay unlinked instead of taking the newest prompt in the database or
|
||||
a stale tracker entry.
|
||||
"""
|
||||
|
||||
import copy
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
try:
|
||||
from tests.open_handles import assert_closed
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from open_handles import assert_closed
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from prompt_manager import PromptManager
|
||||
from prompt_manager_pair import PromptManagerPair
|
||||
from prompt_manager_text import PromptManagerText
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.image_monitor import ImageGenerationHandler
|
||||
from utils.prompt_tracker import PromptTracker
|
||||
from utils.run_keys import image_run_keys, prompt_node_key
|
||||
from utils.usage_tracking import (
|
||||
PendingFirstUse,
|
||||
queued_pair_runs,
|
||||
queued_prompt_runs,
|
||||
)
|
||||
|
||||
LLM_TEXT = "Photorealistic aerial landscape of misty mountains at dawn"
|
||||
EVIDENCE_KEY = "comfyassets_prompt_pairs"
|
||||
|
||||
|
||||
def issue_graph(text="", seed=1):
|
||||
"""The workflow from #342 (subgraph ids kept), PromptManagerPair positive."""
|
||||
return {
|
||||
"18:35": {
|
||||
"class_type": "PromptManagerPair",
|
||||
"inputs": {
|
||||
"text": text,
|
||||
"negative_text": "",
|
||||
"prepend_text": ["18:13", 0],
|
||||
"append_text": "",
|
||||
"category": "",
|
||||
"tags": "",
|
||||
"clip": ["4", 1],
|
||||
},
|
||||
},
|
||||
"18:13": {"class_type": "PreviewAny", "inputs": {"source": ["18:16", 0]}},
|
||||
"18:16": {
|
||||
"class_type": "ComfySwitchNode",
|
||||
"inputs": {
|
||||
"switch": True,
|
||||
"on_false": ["18:17", 0],
|
||||
"on_true": ["18:11", 0],
|
||||
},
|
||||
},
|
||||
"18:11": {
|
||||
"class_type": "TextGenerate",
|
||||
"inputs": {"prompt": ["18:17", 0], "seed": 7},
|
||||
},
|
||||
"18:17": {
|
||||
"class_type": "PrimitiveStringMultiline",
|
||||
"inputs": {"value": "Describe an aerial mountain landscape"},
|
||||
},
|
||||
"18:5": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": "blurry", "clip": ["4", 1]},
|
||||
},
|
||||
"4": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"inputs": {"ckpt_name": "moss.safetensors"},
|
||||
},
|
||||
"3": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"model": ["4", 0],
|
||||
"positive": ["18:35", 0],
|
||||
"negative": ["18:5", 0],
|
||||
"latent_image": ["6", 0],
|
||||
"seed": seed,
|
||||
},
|
||||
},
|
||||
"6": {"class_type": "EmptyLatentImage", "inputs": {"batch_size": 1}},
|
||||
}
|
||||
|
||||
|
||||
def llm_text_graph(seed=1):
|
||||
"""An LLM feeding a PromptManager's ``text``: stored, but not in the metadata."""
|
||||
graph = issue_graph(seed=seed)
|
||||
graph["18:35"] = {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {"text": ["18:13", 0], "clip": ["4", 1]},
|
||||
}
|
||||
return graph
|
||||
|
||||
|
||||
def pair_evidence(text="", prepend=LLM_TEXT):
|
||||
"""The execution record PromptManagerPair embeds in the saved image."""
|
||||
return {
|
||||
"version": 1,
|
||||
"nodes": {
|
||||
"18:35": {
|
||||
"records": [
|
||||
{
|
||||
"text": text,
|
||||
"negative_text": "",
|
||||
"positive_text": " ".join(p for p in (prepend, text) if p),
|
||||
"negative_effective_text": "",
|
||||
"prepend_text": prepend,
|
||||
"append_text": "",
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def make_tracker(db):
|
||||
"""A real tracker without its cleanup thread or config import."""
|
||||
with mock.patch.object(PromptTracker, "__init__", lambda self, db: None):
|
||||
tracker = PromptTracker.__new__(PromptTracker)
|
||||
tracker.logger = logging.getLogger("test.prompt_attribution.tracker")
|
||||
tracker.db_manager = db
|
||||
tracker._local = threading.local()
|
||||
tracker.active_prompts = {}
|
||||
tracker.lock = threading.Lock()
|
||||
tracker._prompt_queue = []
|
||||
tracker._queue_lock = threading.Lock()
|
||||
tracker.cleanup_interval = 300
|
||||
tracker.prompt_timeout = 600
|
||||
return tracker
|
||||
|
||||
|
||||
class DatabaseCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
self.addCleanup(assert_closed, self, self.tmp.name)
|
||||
self.db = PromptDatabase(os.path.join(self.tmp.name, "prompts.db"))
|
||||
self.addCleanup(self.db.close_all)
|
||||
self.tracker = make_tracker(self.db)
|
||||
|
||||
def save(self, text):
|
||||
return self.db.save_prompt(text=text, prompt_hash=generate_prompt_hash(text))
|
||||
|
||||
|
||||
class NodeCase(DatabaseCase):
|
||||
def node(self, cls):
|
||||
node = cls.__new__(cls)
|
||||
node.db = self.db
|
||||
node.logger = logging.getLogger("test.prompt_attribution.node")
|
||||
node.prompt_tracker = self.tracker
|
||||
node.comfyui_integration = mock.Mock()
|
||||
node._inject_lora_trigger_words = lambda text: text
|
||||
node.pending_first_use = PendingFirstUse()
|
||||
node.pending_pair_first_use = PendingFirstUse()
|
||||
return node
|
||||
|
||||
def clip(self):
|
||||
clip = mock.Mock()
|
||||
clip.tokenize.side_effect = lambda text: text
|
||||
clip.encode_from_tokens_scheduled.side_effect = lambda t: ("cond", t)
|
||||
return clip
|
||||
|
||||
def count_rows(self, table):
|
||||
conn = self.db.model.get_connection()
|
||||
return conn.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0]
|
||||
|
||||
|
||||
class TestEmptyEditorSavesNothing(NodeCase):
|
||||
"""Prepend/append are never the stored prompt; an empty editor saves nothing."""
|
||||
|
||||
def test_pair_with_only_prepend_text_saves_nothing_and_logs_info(self):
|
||||
graph = issue_graph()
|
||||
with self.assertLogs("test.prompt_attribution.node", "INFO") as logs:
|
||||
result = self.node(PromptManagerPair).encode_prompt(
|
||||
self.clip(),
|
||||
"",
|
||||
negative_text="",
|
||||
prepend_text=LLM_TEXT,
|
||||
prompt=graph,
|
||||
unique_id="18:35",
|
||||
)
|
||||
self.assertEqual(result[2], LLM_TEXT, "the composed text is still encoded")
|
||||
self.assertEqual(self.count_rows("prompts"), 0)
|
||||
self.assertEqual(self.count_rows("prompt_pairs"), 0)
|
||||
self.assertFalse(
|
||||
any(record.levelno >= logging.WARNING for record in logs.records),
|
||||
[record.getMessage() for record in logs.records],
|
||||
)
|
||||
self.assertTrue(any("Prompt text is empty" in line for line in logs.output))
|
||||
|
||||
def test_prompt_manager_and_text_nodes_with_only_prepend_save_nothing(self):
|
||||
graph = issue_graph()
|
||||
graph["18:35"] = {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {"text": "", "prepend_text": ["18:13", 0], "clip": ["4", 1]},
|
||||
}
|
||||
self.node(PromptManager).encode_prompt(
|
||||
self.clip(), "", prepend_text=LLM_TEXT, prompt=graph, unique_id="18:35"
|
||||
)
|
||||
graph["18:35"]["class_type"] = "PromptManagerText"
|
||||
self.node(PromptManagerText).process_text(
|
||||
"", prepend_text=LLM_TEXT, prompt=graph, unique_id="18:35"
|
||||
)
|
||||
self.assertEqual(self.count_rows("prompts"), 0)
|
||||
|
||||
def test_empty_editor_marks_its_run_key_so_no_older_entry_is_used(self):
|
||||
graph = issue_graph()
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
older = self.save("a woman holding a coffee cup")
|
||||
self.tracker.set_current_prompt(
|
||||
"a woman holding a coffee cup",
|
||||
{"prompt_id": older},
|
||||
push_to_queue=True,
|
||||
run_key=key,
|
||||
run_id="run-1",
|
||||
)
|
||||
self.node(PromptManagerPair).encode_prompt(
|
||||
self.clip(), "", prepend_text=LLM_TEXT, prompt=graph, unique_id="18:35"
|
||||
)
|
||||
latest, _run, _count = self.tracker.keyed_snapshot()[key]
|
||||
self.assertTrue(latest.get("placeholder"))
|
||||
self.assertIsNone(self.tracker.pop_for_key(key, "another run"))
|
||||
popped = self.tracker.pop_for_key(key)
|
||||
self.assertEqual(popped["id"], older, "an earlier queued item stays first")
|
||||
self.assertTrue(self.tracker.pop_for_key(key).get("placeholder"))
|
||||
|
||||
def test_empty_negative_node_is_not_marked(self):
|
||||
graph = issue_graph(text="a typed positive")
|
||||
graph["18:5"] = {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {"text": "", "clip": ["4", 1]},
|
||||
}
|
||||
self.node(PromptManager).encode_prompt(
|
||||
self.clip(), "", prompt=graph, unique_id="18:5"
|
||||
)
|
||||
self.assertEqual(self.tracker.keyed_snapshot(), {})
|
||||
|
||||
def test_queue_hook_counts_nothing_for_an_empty_editor(self):
|
||||
graph = issue_graph()
|
||||
self.assertEqual(queued_prompt_runs(graph), [])
|
||||
self.assertEqual(queued_pair_runs(graph), [])
|
||||
|
||||
def test_typed_text_with_prepend_still_stores_only_the_editor_text(self):
|
||||
graph = issue_graph(text="mountains")
|
||||
self.node(PromptManagerPair).encode_prompt(
|
||||
self.clip(),
|
||||
"mountains",
|
||||
prepend_text=LLM_TEXT,
|
||||
prompt=graph,
|
||||
unique_id="18:35",
|
||||
)
|
||||
self.assertIsNotNone(
|
||||
self.db.get_prompt_by_hash(generate_prompt_hash("mountains"))
|
||||
)
|
||||
self.assertIsNone(self.db.get_prompt_by_hash(generate_prompt_hash(LLM_TEXT)))
|
||||
context, _run, _count = self.tracker.keyed_snapshot()[
|
||||
prompt_node_key(graph, "18:35")
|
||||
]
|
||||
self.assertEqual(context["text"], f"{LLM_TEXT} mountains")
|
||||
|
||||
|
||||
class MonitorCase(DatabaseCase):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
fd, self.image = tempfile.mkstemp(suffix=".png", dir=self.tmp.name)
|
||||
os.close(fd)
|
||||
self.handler = ImageGenerationHandler(self.db, self.tracker)
|
||||
self.linked = []
|
||||
self.handler.link_image_to_prompt = lambda path, prompt, meta: (
|
||||
self.linked.append(prompt["id"])
|
||||
)
|
||||
|
||||
def process(self, metadata, snapshot=None):
|
||||
before = len(self.linked)
|
||||
with mock.patch.object(
|
||||
self.handler.metadata_extractor, "extract_metadata", return_value=metadata
|
||||
):
|
||||
self.handler.process_new_image(self.image, prompt_snapshot=snapshot)
|
||||
return self.linked[-1] if len(self.linked) > before else None
|
||||
|
||||
|
||||
class TestIssueReproduction(MonitorCase):
|
||||
def test_llm_prepend_image_is_left_unlinked_not_given_the_newest_prompt(self):
|
||||
"""Failing before #342: linked to the newest prompt in the database."""
|
||||
self.save("a woman holding a coffee cup")
|
||||
metadata = {"prompt": issue_graph(), EVIDENCE_KEY: pair_evidence()}
|
||||
with self.assertLogs("prompt_manager.image_monitor", "INFO") as logs:
|
||||
self.assertIsNone(self.process(metadata))
|
||||
self.assertTrue(any("left unlinked" in line for line in logs.output))
|
||||
|
||||
def test_a_stale_entry_of_another_workflow_is_never_used(self):
|
||||
"""Failing before #342: the live tracker's entry was taken."""
|
||||
stale = self.save("a woman holding a coffee cup")
|
||||
other = issue_graph(text="a woman holding a coffee cup")
|
||||
self.tracker.set_current_prompt(
|
||||
"a woman holding a coffee cup",
|
||||
{"prompt_id": stale},
|
||||
push_to_queue=True,
|
||||
run_key=prompt_node_key(other, "18:35"),
|
||||
run_id="run-0",
|
||||
)
|
||||
metadata = {"prompt": issue_graph(), EVIDENCE_KEY: pair_evidence()}
|
||||
self.assertIsNone(self.process(metadata, self.tracker.keyed_snapshot()))
|
||||
self.assertIsNone(self.process(metadata))
|
||||
self.assertEqual(len(self.tracker._prompt_queue), 1, "not consumed")
|
||||
|
||||
def test_an_entry_without_a_run_key_is_never_used(self):
|
||||
stale = self.save("previous run")
|
||||
self.tracker.set_current_prompt("previous run", {"prompt_id": stale})
|
||||
self.assertIsNone(self.process({"prompt": llm_text_graph()}))
|
||||
|
||||
def test_an_image_without_metadata_is_left_unlinked(self):
|
||||
"""Failing before #342: popped the queue, else took the newest prompt."""
|
||||
self.save("latest")
|
||||
self.assertIsNone(self.process(None))
|
||||
|
||||
def test_the_recency_fallback_is_gone(self):
|
||||
self.assertFalse(hasattr(self.handler, "_get_fallback_prompt"))
|
||||
|
||||
|
||||
class TestProvenRuns(MonitorCase):
|
||||
"""Entries the image's own graph proves still link (LLM into ``text``)."""
|
||||
|
||||
def track(self, graph, text, run_id="run-1", push=True):
|
||||
prompt_id = (
|
||||
self.save(text)
|
||||
if not self.db.get_prompt_by_hash(generate_prompt_hash(text))
|
||||
else self.db.get_prompt_by_hash(generate_prompt_hash(text))["id"]
|
||||
)
|
||||
self.tracker.set_current_prompt(
|
||||
text,
|
||||
{"prompt_id": prompt_id},
|
||||
push_to_queue=push,
|
||||
run_key=prompt_node_key(graph, "18:35"),
|
||||
run_id=run_id,
|
||||
)
|
||||
return prompt_id
|
||||
|
||||
def test_queued_entry_of_the_same_run_key_links(self):
|
||||
graph = llm_text_graph()
|
||||
prompt_id = self.track(graph, LLM_TEXT)
|
||||
self.assertEqual(self.process({"prompt": graph}), prompt_id)
|
||||
|
||||
def test_cached_node_rerun_links_through_the_snapshot(self):
|
||||
graph = llm_text_graph(seed=1)
|
||||
prompt_id = self.track(graph, LLM_TEXT)
|
||||
self.assertEqual(self.process({"prompt": graph}), prompt_id)
|
||||
# Only the sampler seed changed: the node was cached and did not run,
|
||||
# but its upstream (the run key) is unchanged.
|
||||
rerun = llm_text_graph(seed=2)
|
||||
self.assertEqual(
|
||||
prompt_node_key(rerun, "18:35"), prompt_node_key(graph, "18:35")
|
||||
)
|
||||
self.assertEqual(
|
||||
self.process({"prompt": rerun}, self.tracker.keyed_snapshot()), prompt_id
|
||||
)
|
||||
|
||||
def test_snapshot_needs_a_known_run(self):
|
||||
graph = llm_text_graph()
|
||||
self.track(graph, LLM_TEXT, run_id=None, push=False)
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
|
||||
def test_batch_items_link_in_order_and_the_latest_alone_proves_nothing(self):
|
||||
graph = llm_text_graph()
|
||||
first = self.track(graph, "item one")
|
||||
second = self.track(graph, "item two")
|
||||
self.assertEqual(self.process({"prompt": graph}), first)
|
||||
self.assertEqual(self.process({"prompt": graph}), second)
|
||||
# Queue drained: two entries in one run, so the latest is not proof
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
|
||||
def test_an_empty_batch_item_keeps_the_queue_aligned(self):
|
||||
graph = llm_text_graph()
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
first = self.track(graph, "item one")
|
||||
self.tracker.mark_empty(key, "run-1", push_to_queue=True)
|
||||
third = self.track(graph, "item three")
|
||||
self.assertEqual(self.process({"prompt": graph}), first)
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
self.assertEqual(self.process({"prompt": graph}), third)
|
||||
|
||||
def test_a_newer_run_drops_stale_entries_of_the_same_key(self):
|
||||
graph = llm_text_graph()
|
||||
# run-1 saved no image, so its entry was never taken
|
||||
self.track(graph, "earlier run text", run_id="run-1")
|
||||
newer = self.track(graph, "newer run text", run_id="run-2")
|
||||
self.assertEqual(self.process({"prompt": graph}), newer)
|
||||
self.assertEqual(self.tracker._prompt_queue, [])
|
||||
|
||||
def test_a_later_run_queued_before_processing_keeps_its_own_entry(self):
|
||||
# The image appears (snapshot), the next queued run executes the same
|
||||
# node again before the monitor's delay is over, then the image is
|
||||
# processed: it takes its own run's entry, the later run keeps its own.
|
||||
graph = llm_text_graph()
|
||||
own = self.track(graph, "first run text", run_id="run-1")
|
||||
snapshot = self.tracker.keyed_snapshot()
|
||||
later = self.track(graph, "second run text", run_id="run-2")
|
||||
self.assertEqual(self.process({"prompt": graph}, snapshot), own)
|
||||
self.assertEqual(self.process({"prompt": graph}), later)
|
||||
|
||||
def test_no_entry_when_the_image_appeared_means_no_queue_match(self):
|
||||
graph = llm_text_graph()
|
||||
empty_snapshot = self.tracker.keyed_snapshot()
|
||||
self.track(graph, "queued after the image appeared")
|
||||
self.assertIsNone(self.process({"prompt": graph}, empty_snapshot))
|
||||
self.assertEqual(len(self.tracker._prompt_queue), 1)
|
||||
|
||||
def test_hosts_without_run_ids_still_link_queued_entries_in_order(self):
|
||||
graph = llm_text_graph()
|
||||
first = self.track(graph, "item one", run_id=None)
|
||||
second = self.track(graph, "item two", run_id=None)
|
||||
self.assertEqual(self.process({"prompt": graph}), first)
|
||||
self.assertEqual(self.process({"prompt": graph}), second)
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
|
||||
def test_other_keys_stay_queued_for_their_own_images(self):
|
||||
mine, other = llm_text_graph(), copy.deepcopy(llm_text_graph())
|
||||
other["18:17"]["inputs"]["value"] = "Describe a beach"
|
||||
theirs = self.track(other, "a beach")
|
||||
prompt_id = self.track(mine, LLM_TEXT)
|
||||
self.assertEqual(self.process({"prompt": mine}), prompt_id)
|
||||
self.assertEqual(self.process({"prompt": other}), theirs)
|
||||
|
||||
def test_two_positive_prompt_nodes_are_not_guessed(self):
|
||||
graph = llm_text_graph()
|
||||
self.track(graph, LLM_TEXT)
|
||||
graph["30"] = {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {"text": ["18:13", 0], "clip": ["4", 1]},
|
||||
}
|
||||
graph["31"] = dict(graph["3"])
|
||||
graph["31"]["inputs"] = {**graph["3"]["inputs"], "positive": ["30", 0]}
|
||||
self.assertEqual(len(image_run_keys({"prompt": graph})), 2)
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
|
||||
def test_metadata_still_wins_and_leaves_the_queue_alone(self):
|
||||
typed = self.save("a typed prompt")
|
||||
graph = issue_graph(text="a typed prompt")
|
||||
self.tracker.set_current_prompt(
|
||||
"x",
|
||||
{"prompt_id": typed},
|
||||
run_key=prompt_node_key(graph, "18:35"),
|
||||
run_id="r",
|
||||
)
|
||||
self.assertEqual(self.process({"prompt": graph}), typed)
|
||||
self.assertEqual(len(self.tracker._prompt_queue), 1)
|
||||
|
||||
|
||||
class TestRunKeys(unittest.TestCase):
|
||||
def test_key_covers_the_upstream_and_ignores_downstream_and_titles(self):
|
||||
graph = issue_graph()
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
changed = copy.deepcopy(graph)
|
||||
changed["3"]["inputs"]["seed"] = 99
|
||||
changed["18:35"]["_meta"] = {"title": "renamed"}
|
||||
self.assertEqual(prompt_node_key(changed, "18:35"), key)
|
||||
changed["18:11"]["inputs"]["seed"] = 8
|
||||
self.assertNotEqual(prompt_node_key(changed, "18:35"), key)
|
||||
|
||||
def test_unknowable_keys_are_none(self):
|
||||
graph = issue_graph()
|
||||
self.assertIsNone(prompt_node_key(None, "1"))
|
||||
self.assertIsNone(prompt_node_key(graph, None))
|
||||
self.assertIsNone(prompt_node_key(graph, "missing"))
|
||||
broken = copy.deepcopy(graph)
|
||||
broken["18:13"]["inputs"]["source"] = ["gone", 0]
|
||||
self.assertIsNone(prompt_node_key(broken, "18:35"))
|
||||
broken = copy.deepcopy(graph)
|
||||
broken["18:17"]["inputs"]["value"] = object()
|
||||
self.assertIsNone(prompt_node_key(broken, "18:35"))
|
||||
|
||||
def test_image_keys_read_string_graphs_and_reject_garbage(self):
|
||||
import json
|
||||
|
||||
graph = issue_graph()
|
||||
self.assertEqual(
|
||||
image_run_keys({"prompt": json.dumps(graph)}),
|
||||
{"18:35": prompt_node_key(graph, "18:35")},
|
||||
)
|
||||
self.assertEqual(image_run_keys({"prompt": "{not json"}), {})
|
||||
self.assertEqual(image_run_keys(None), {})
|
||||
self.assertEqual(image_run_keys({"prompt": []}), {})
|
||||
|
||||
def test_cyclic_and_huge_closures_have_no_key(self):
|
||||
from utils.prompt_graph import MAX_VISITS
|
||||
|
||||
cyclic = {
|
||||
"1": {"class_type": "A", "inputs": {"x": ["2", 0]}},
|
||||
"2": {"class_type": "B", "inputs": {"x": ["1", 0]}},
|
||||
}
|
||||
self.assertIsNotNone(prompt_node_key(cyclic, "1"))
|
||||
chain = {
|
||||
str(i): {"class_type": "N", "inputs": {"x": [str(i + 1), 0]}}
|
||||
for i in range(MAX_VISITS + 1)
|
||||
}
|
||||
chain[str(MAX_VISITS + 1)] = {"class_type": "N", "inputs": {}}
|
||||
self.assertIsNone(prompt_node_key(chain, "0"))
|
||||
|
||||
|
||||
class TestReviewFindings(MonitorCase, NodeCase):
|
||||
"""Final-review findings M1-M4 (#344)."""
|
||||
|
||||
def test_m1_a_positive_node_without_a_key_makes_the_image_unkeyable(self):
|
||||
graph = llm_text_graph()
|
||||
graph["30"] = {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {"text": ["missing", 0], "clip": ["4", 1]},
|
||||
}
|
||||
graph["31"] = {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {**graph["3"]["inputs"], "positive": ["30", 0]},
|
||||
}
|
||||
self.assertIsNone(prompt_node_key(graph, "30"))
|
||||
self.assertEqual(image_run_keys({"prompt": graph}), {})
|
||||
prompt_id = self.save(LLM_TEXT)
|
||||
self.tracker.set_current_prompt(
|
||||
LLM_TEXT,
|
||||
{"prompt_id": prompt_id},
|
||||
run_key=prompt_node_key(graph, "18:35"),
|
||||
run_id="run-1",
|
||||
)
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
|
||||
def test_m2_links_are_exactly_comfyui_is_link(self):
|
||||
from utils.run_keys import _links
|
||||
|
||||
inputs = {
|
||||
"string_slot_int": ["7", 0],
|
||||
"string_slot_float": ["7", 0.0],
|
||||
"string_slot_bool": ["7", True], # bool is an int to ComfyUI too
|
||||
"int_source": [7, 0],
|
||||
"tuple": ("7", 0),
|
||||
"three": ["7", 0, 1],
|
||||
"slot_text": ["7", "0"],
|
||||
}
|
||||
self.assertEqual(sorted(_links(inputs)), ["7", "7", "7"])
|
||||
widget = issue_graph()
|
||||
widget["18:17"]["inputs"]["size"] = [512, 768] # a value, not a link
|
||||
self.assertIsNotNone(prompt_node_key(widget, "18:35"))
|
||||
floated = issue_graph()
|
||||
floated["18:35"]["inputs"]["prepend_text"] = ["18:13", 0.0]
|
||||
changed = copy.deepcopy(floated)
|
||||
changed["18:17"]["inputs"]["value"] = "another instruction"
|
||||
self.assertNotEqual(
|
||||
prompt_node_key(floated, "18:35"), prompt_node_key(changed, "18:35")
|
||||
)
|
||||
|
||||
def _failing_save(self, node):
|
||||
def explode(*args, **kwargs):
|
||||
raise RuntimeError("database is locked")
|
||||
|
||||
node.db = mock.Mock(wraps=self.db)
|
||||
for name in ("save_prompt", "save_prompt_pair", "get_prompt_by_hash"):
|
||||
setattr(node.db, name, mock.Mock(side_effect=explode))
|
||||
return node
|
||||
|
||||
def test_m3_a_failed_save_takes_over_its_run_key(self):
|
||||
graph = llm_text_graph()
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
older = self.save("previous run text")
|
||||
self.tracker.set_current_prompt(
|
||||
"previous run text", {"prompt_id": older}, run_key=key, run_id="run-1"
|
||||
)
|
||||
node = self._failing_save(self.node(PromptManager))
|
||||
with mock.patch("prompt_manager_base.executing_run_id", return_value="run-2"):
|
||||
node.encode_prompt(
|
||||
self.clip(), "new run text", prompt=graph, unique_id="18:35"
|
||||
)
|
||||
latest, run_id, _count = self.tracker.keyed_snapshot()[key]
|
||||
self.assertTrue(latest.get("placeholder"))
|
||||
self.assertEqual(run_id, "run-2")
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
|
||||
def test_m3_a_failed_pair_save_takes_over_its_run_key(self):
|
||||
graph = issue_graph(text="typed")
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
node = self._failing_save(self.node(PromptManagerPair))
|
||||
with self.assertLogs("test.prompt_attribution.node", "WARNING"):
|
||||
node.encode_prompt(self.clip(), "typed", prompt=graph, unique_id="18:35")
|
||||
self.assertTrue(self.tracker.keyed_snapshot()[key][0].get("placeholder"))
|
||||
|
||||
def test_m3_a_failed_text_node_save_takes_over_its_run_key(self):
|
||||
graph = issue_graph()
|
||||
graph["18:35"] = {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": "typed"},
|
||||
}
|
||||
graph["40"] = {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": ["18:35", 0], "clip": ["4", 1]},
|
||||
}
|
||||
graph["3"]["inputs"]["positive"] = ["40", 0]
|
||||
node = self._failing_save(self.node(PromptManagerText))
|
||||
with mock.patch.object(
|
||||
node, "_track_prompt_execution", side_effect=RuntimeError("boom")
|
||||
):
|
||||
node.process_text("typed", prompt=graph, unique_id="18:35")
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
self.assertTrue(self.tracker.keyed_snapshot()[key][0].get("placeholder"))
|
||||
|
||||
def test_m4_a_placeholder_never_falls_through_to_history(self):
|
||||
graph = llm_text_graph()
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
self.tracker.mark_empty(key, "run-1", push_to_queue=False)
|
||||
with mock.patch.object(
|
||||
self.handler, "_history_prompt", return_value=99
|
||||
) as history:
|
||||
self.assertIsNone(self.process({"prompt": graph})) # snapshot form
|
||||
self.tracker.mark_empty(key, "run-2", push_to_queue=True)
|
||||
self.assertIsNone(self.process({"prompt": graph})) # popped form
|
||||
history.assert_not_called()
|
||||
|
||||
|
||||
class TestRereviewFindings(MonitorCase, NodeCase):
|
||||
"""Re-review findings R4 and R5 (#344)."""
|
||||
|
||||
def test_r4_a_logging_failure_after_tracking_never_doubles_a_queue_slot(self):
|
||||
graph = llm_text_graph()
|
||||
key = prompt_node_key(graph, "18:35")
|
||||
node = self.node(PromptManager)
|
||||
logger = mock.Mock(wraps=self.tracker.logger)
|
||||
logger.debug.side_effect = RuntimeError("log handler broke")
|
||||
self.tracker.logger = logger
|
||||
node.encode_prompt(self.clip(), LLM_TEXT, prompt=graph, unique_id="18:35")
|
||||
queued = [ctx for ctx in self.tracker._prompt_queue if ctx["run_key"] == key]
|
||||
self.assertEqual(len(queued), 1)
|
||||
|
||||
def test_r5_a_keyless_image_outside_the_output_folder_does_not_wait(self):
|
||||
outside = tempfile.mkdtemp()
|
||||
self.addCleanup(lambda: __import__("shutil").rmtree(outside, True))
|
||||
stray = os.path.join(outside, "copied.png")
|
||||
with open(stray, "wb") as handle:
|
||||
handle.write(b"png")
|
||||
with mock.patch(
|
||||
"utils.image_monitor.host_output_directory", return_value=self.tmp.name
|
||||
):
|
||||
with mock.patch.object(
|
||||
self.handler.metadata_extractor, "extract_metadata", return_value=None
|
||||
):
|
||||
self.handler.process_new_image(stray)
|
||||
self.handler.process_new_image(self.image) # inside: waits
|
||||
self.assertEqual(
|
||||
self.handler.awaiting_history(),
|
||||
[os.path.normcase(os.path.realpath(self.image))],
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -205,13 +205,39 @@ class TestPreviewLease(RestoreTestCase):
|
||||
await lease.__aexit__(None, None, None)
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertEqual(job.last_event["code"], "busy")
|
||||
self.assertIn("review is kept", job.status_line)
|
||||
# S9 I1: the page that started it reopens the kept review (Restore again
|
||||
# or Discard); the flag tells it so, and the line says where the review
|
||||
# is, never "start the restore again" from a place that cannot.
|
||||
self.assertIs(job.scope.get("review_kept"), True)
|
||||
self.assertEqual(job.status_line, backup_restore.REVIEW_KEPT)
|
||||
self.assertIn("reopened in the page that started", job.status_line)
|
||||
self.assertEqual(len(self.stages()), 1)
|
||||
# The same review applies once the requests are done.
|
||||
job = await self.finish((await self.apply(body["token"]))["job"]["id"])
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(_texts(self.db_path), ["restored"])
|
||||
|
||||
async def test_a_review_that_cannot_be_kept_is_not_flagged_kept(self):
|
||||
self.save("live")
|
||||
body = await self.preview(self.upload(["restored"]))
|
||||
lease = self.api._jobs.lease("Stuck preview")
|
||||
await lease.__aenter__()
|
||||
try:
|
||||
with (
|
||||
patch.object(jobs, "LEASE_DRAIN_SECONDS", 0),
|
||||
patch.object(
|
||||
backup_restore.RestorePlanStore, "give_back", lambda *_: False
|
||||
),
|
||||
):
|
||||
started = await self.apply(body["token"])
|
||||
job = await self.finish(started["job"]["id"])
|
||||
finally:
|
||||
await lease.__aexit__(None, None, None)
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertNotIn("review_kept", job.scope)
|
||||
self.assertEqual(job.status_line, backup_restore.REQUESTS_BUSY)
|
||||
self.assertEqual(len(self.stages()), 0)
|
||||
|
||||
|
||||
class TestStoreOnTheLoop(RestoreTestCase):
|
||||
"""M3: entries leave the store on the loop; only file removal is threaded."""
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
"""tools/review_image_links.py: a dry run by default, changes only named images."""
|
||||
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
|
||||
from PIL import Image
|
||||
from PIL.PngImagePlugin import PngInfo
|
||||
|
||||
try:
|
||||
from tests.open_handles import assert_closed
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from open_handles import assert_closed
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from tests.test_api_images import ImageAPITestCase
|
||||
from tests.test_prompt_attribution import issue_graph, pair_evidence
|
||||
from tools import review_image_links as tool
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
|
||||
def png(path, prompt=None, evidence=None):
|
||||
info = PngInfo()
|
||||
if prompt is not None:
|
||||
info.add_text("prompt", json.dumps(prompt))
|
||||
if evidence is not None:
|
||||
info.add_text("comfyassets_prompt_pairs", json.dumps(evidence))
|
||||
Image.new("RGB", (4, 4)).save(path, pnginfo=info)
|
||||
|
||||
|
||||
class ToolFixture(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.mkdtemp()
|
||||
self.addCleanup(shutil.rmtree, self.tmp, True)
|
||||
self.addCleanup(assert_closed, self, self.tmp)
|
||||
self.output = os.path.join(self.tmp, "output")
|
||||
os.makedirs(self.output)
|
||||
self.db_path = os.path.join(self.tmp, "prompts.db")
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
save = lambda text: db.save_prompt( # noqa: E731
|
||||
text=text, prompt_hash=generate_prompt_hash(text)
|
||||
)
|
||||
self.coffee = save("a woman holding a coffee cup")
|
||||
self.typed = save("snowy peaks")
|
||||
# The six #342 images: only prepend text, linked by the old fallback
|
||||
self.guessed = []
|
||||
for n in range(9, 12):
|
||||
path = os.path.join(self.output, f"moss_{n:05}_.png")
|
||||
png(path, issue_graph(), pair_evidence())
|
||||
self.guessed.append(
|
||||
db.link_image_to_prompt(self.coffee, path, metadata={})
|
||||
)
|
||||
# A typed image linked to the wrong prompt, and a correct one
|
||||
wrong = os.path.join(self.output, "typed_wrong.png")
|
||||
png(wrong, issue_graph(text="snowy peaks"))
|
||||
self.wrong = db.link_image_to_prompt(self.coffee, wrong, metadata={})
|
||||
right = os.path.join(self.output, "typed_right.png")
|
||||
png(right, issue_graph(text="snowy peaks"))
|
||||
self.right = db.link_image_to_prompt(self.typed, right, metadata={})
|
||||
gone = os.path.join(self.output, "gone.png")
|
||||
self.gone = db.link_image_to_prompt(self.coffee, gone, metadata={})
|
||||
finally:
|
||||
db.close_all()
|
||||
|
||||
def run_tool(self, *args):
|
||||
out = io.StringIO()
|
||||
code = tool.main([self.db_path, "--output-dir", self.output, *args], out=out)
|
||||
return code, out.getvalue()
|
||||
|
||||
def links(self):
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
rows = db.model.get_connection().execute(
|
||||
"SELECT id, prompt_id FROM generated_images"
|
||||
)
|
||||
return {row[0]: row[1] for row in rows}
|
||||
finally:
|
||||
db.close_all()
|
||||
|
||||
|
||||
class ReviewToolTests(ToolFixture):
|
||||
def test_dry_run_judges_and_changes_nothing(self):
|
||||
before = self.links()
|
||||
ids = f"{self.guessed[0]}-{self.gone}"
|
||||
code, out = self.run_tool("--ids", ids, "--json")
|
||||
self.assertEqual(code, 0)
|
||||
verdicts = {v["image_id"]: v for v in json.loads(out.split("--apply, for")[0])}
|
||||
for image_id in self.guessed:
|
||||
self.assertEqual(verdicts[image_id]["verdict"], "unlink")
|
||||
self.assertEqual(verdicts[image_id]["sources"], {})
|
||||
self.assertEqual(verdicts[self.wrong]["verdict"], "relink")
|
||||
self.assertEqual(verdicts[self.wrong]["target"], self.typed)
|
||||
self.assertEqual(verdicts[self.right]["verdict"], "keep")
|
||||
self.assertEqual(verdicts[self.gone]["verdict"], "missing")
|
||||
self.assertIn("Dry run: nothing was changed.", out)
|
||||
self.assertEqual(self.links(), before)
|
||||
self.assertEqual(
|
||||
sorted(os.listdir(self.tmp)), ["output", "prompts.db"], "no side files"
|
||||
)
|
||||
|
||||
def test_prompt_selection_and_text_report(self):
|
||||
code, out = self.run_tool("--prompt", str(self.coffee))
|
||||
self.assertEqual(code, 0)
|
||||
self.assertIn("unlink: 3", out)
|
||||
self.assertIn("relink: 1", out)
|
||||
self.assertIn("missing: 1", out)
|
||||
self.assertIn(f"-> {self.typed}", out)
|
||||
|
||||
def test_apply_backs_up_then_changes_only_named_images(self):
|
||||
named = f"{self.guessed[0]}-{self.guessed[1]},{self.wrong}"
|
||||
code, out = self.run_tool(
|
||||
"--ids", named, "--prompt", str(self.coffee), "--apply"
|
||||
)
|
||||
self.assertEqual(code, 0)
|
||||
self.assertIn("Applied: 1 relinked, 2 unlinked, 0 skipped", out)
|
||||
links = self.links()
|
||||
self.assertNotIn(self.guessed[0], links)
|
||||
self.assertNotIn(self.guessed[1], links)
|
||||
self.assertEqual(links[self.guessed[2]], self.coffee, "not named: kept")
|
||||
self.assertEqual(links[self.wrong], self.typed)
|
||||
self.assertEqual(links[self.gone], self.coffee, "missing files are kept")
|
||||
backups = [n for n in os.listdir(self.tmp) if ".pre-relink_" in n]
|
||||
self.assertEqual(len(backups), 1)
|
||||
restored = PromptDatabase(os.path.join(self.tmp, backups[0]))
|
||||
try:
|
||||
count = (
|
||||
restored.model.get_connection()
|
||||
.execute("SELECT COUNT(*) FROM generated_images")
|
||||
.fetchone()[0]
|
||||
)
|
||||
finally:
|
||||
restored.close_all()
|
||||
self.assertEqual(count, 6)
|
||||
|
||||
def test_arguments_are_validated(self):
|
||||
for args in (
|
||||
[],
|
||||
["--ids", "5-2"],
|
||||
["--ids", "x"],
|
||||
["--prompt", "1", "--apply"],
|
||||
):
|
||||
with self.subTest(args=args), self.assertRaises(SystemExit):
|
||||
with unittest.mock.patch("sys.stderr", io.StringIO()):
|
||||
self.run_tool(*args)
|
||||
with (
|
||||
self.assertRaises(SystemExit),
|
||||
unittest.mock.patch("sys.stderr", io.StringIO()),
|
||||
):
|
||||
tool.main(
|
||||
[os.path.join(self.tmp, "none.db"), "--output-dir", "x", "--ids", "1"]
|
||||
)
|
||||
self.assertEqual(tool.parse_ids(" 3, 1-2 ,"), [1, 2, 3])
|
||||
|
||||
def test_conflicting_sources_are_left_to_the_maintainer(self):
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
handler = tool.ImageGenerationHandler(db, None)
|
||||
row = {
|
||||
"id": 1,
|
||||
"prompt_id": self.coffee,
|
||||
"image_path": os.path.join(self.output, "typed_right.png"),
|
||||
}
|
||||
with unittest.mock.patch.object(
|
||||
tool, "certain_prompts", return_value={"scan": 1, "history": 2}
|
||||
):
|
||||
verdict = tool.judge(db, handler, row, self.output)
|
||||
finally:
|
||||
db.close_all()
|
||||
self.assertEqual(verdict["verdict"], "conflict")
|
||||
|
||||
|
||||
class ApplyGuardTests(ToolFixture):
|
||||
"""M6: unlink clears only the link, and only the link it was judged on."""
|
||||
|
||||
def test_unlink_skips_a_row_linked_elsewhere_since_the_review(self):
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
verdicts = tool.review(db, [self.guessed[0]], None, self.output)
|
||||
self.assertEqual(verdicts[0]["verdict"], "unlink")
|
||||
db.model.get_connection().execute(
|
||||
"UPDATE generated_images SET prompt_id=? WHERE id=?",
|
||||
(self.typed, self.guessed[0]),
|
||||
)
|
||||
db.model.get_connection().commit()
|
||||
result = tool.apply(db, verdicts, [self.guessed[0]])
|
||||
finally:
|
||||
db.close_all()
|
||||
self.assertEqual(result["unlinked"], 0)
|
||||
self.assertEqual(self.links()[self.guessed[0]], self.typed)
|
||||
|
||||
def test_unlink_keeps_the_file_and_its_favorite(self):
|
||||
path = os.path.join(self.output, "moss_00009_.png")
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
db.set_image_favorite("a" * 64, "moss_00009_.png")
|
||||
finally:
|
||||
db.close_all()
|
||||
self.run_tool("--ids", str(self.guessed[0]), "--apply")
|
||||
self.assertTrue(os.path.isfile(path))
|
||||
db = PromptDatabase(self.db_path)
|
||||
try:
|
||||
self.assertEqual(db.get_root_favorites("a" * 64), {"moss_00009_.png"})
|
||||
summary = db.get_image_link_summaries([path]).get(path) or {}
|
||||
finally:
|
||||
db.close_all()
|
||||
self.assertNotEqual(summary.get("status"), "linked")
|
||||
|
||||
def test_dry_run_states_what_apply_does(self):
|
||||
_code, out = self.run_tool("--ids", str(self.guessed[0]))
|
||||
self.assertIn("removes only the image's link record", out)
|
||||
self.assertIn("file stays on disk and in the gallery as unlinked", out)
|
||||
|
||||
|
||||
class GalleryAfterUnlinkTests(ImageAPITestCase):
|
||||
"""M6: after --apply the file is still listed, as an unlinked image."""
|
||||
|
||||
async def test_unlinked_file_stays_in_the_workspace_gallery(self):
|
||||
db = self.api.db
|
||||
prompt = db.save_prompt(
|
||||
text="a woman holding a coffee cup",
|
||||
prompt_hash=generate_prompt_hash("a woman holding a coffee cup"),
|
||||
)
|
||||
path = self.output_dir / "moss_00009_.png"
|
||||
png(str(path), issue_graph(), pair_evidence())
|
||||
image_id = db.link_image_to_prompt(prompt, str(path), metadata={})
|
||||
out = io.StringIO()
|
||||
tool.main(
|
||||
[
|
||||
db.model.db_path,
|
||||
"--output-dir",
|
||||
str(self.output_dir),
|
||||
"--ids",
|
||||
str(image_id),
|
||||
"--apply",
|
||||
],
|
||||
out=out,
|
||||
)
|
||||
self.assertIn("Applied: 0 relinked, 1 unlinked", out.getvalue())
|
||||
response = await self.client.get(
|
||||
"/prompt_manager/images/output", params={"mode": "workspace"}
|
||||
)
|
||||
self.assertEqual(response.status, 200)
|
||||
images = {i["filename"]: i for i in (await response.json())["images"]}
|
||||
self.assertIn("moss_00009_.png", images)
|
||||
link = images["moss_00009_.png"].get("linked") or {}
|
||||
self.assertNotEqual(link.get("status"), "linked")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,674 @@
|
||||
"""#342: recorded history attributes a file only to one certain, stored prompt."""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from database.run_attribution import stored_run_prompt
|
||||
from tests import test_execution_history_database as history_tests
|
||||
from tests.test_api_admin import AdminAPITestCase
|
||||
from tests.test_execution_context_database import snapshot
|
||||
from tests.test_prompt_attribution import issue_graph, llm_text_graph, make_tracker
|
||||
from utils.execution_contexts import ContextBundle, RecipeContext
|
||||
from utils.execution_outputs import (
|
||||
OutputBundle,
|
||||
OutputReference,
|
||||
snapshot_output_roots,
|
||||
)
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.image_monitor import ImageGenerationHandler
|
||||
|
||||
STYLE = "windows" if os.name == "nt" else "posix"
|
||||
LLM_TEXT = "Photorealistic aerial landscape of misty mountains at dawn"
|
||||
|
||||
|
||||
def now_ms():
|
||||
return time.time_ns() // 1_000_000
|
||||
|
||||
|
||||
class RecordedRuns:
|
||||
"""Write runs into execution history the way the capture pipeline does."""
|
||||
|
||||
start = history_tests.HistoryDatabaseTests.start
|
||||
finish = history_tests.HistoryDatabaseTests.finish
|
||||
|
||||
def record_run(self, db, output_root, files, positives, item=0, window=None):
|
||||
started, finished = window or (now_ms() - 5000, now_ms() + 5000)
|
||||
start = self.start(db, item, started)
|
||||
db.write_execution_start(start)
|
||||
finish = self.finish(start, time=finished)
|
||||
db.write_execution_finish(finish)
|
||||
db.write_execution_contexts(
|
||||
start.run_id,
|
||||
start.storage_epoch,
|
||||
ContextBundle(
|
||||
tuple(
|
||||
RecipeContext(str(4 + i), snapshot(text), snapshot(""), "inferred")
|
||||
for i, text in enumerate(positives)
|
||||
),
|
||||
"complete",
|
||||
"captured",
|
||||
),
|
||||
)
|
||||
root = snapshot_output_roots(STYLE, output_root, output_root).output_key
|
||||
references = tuple(
|
||||
OutputReference("9", i, "output", sub, name, STYLE, root)
|
||||
for i, (sub, name) in enumerate(files)
|
||||
)
|
||||
db.write_execution_outputs_for_finish(
|
||||
finish, OutputBundle(references, "complete", "reported")
|
||||
)
|
||||
return start.run_id
|
||||
|
||||
|
||||
class TestStoredRunPrompt(RecordedRuns, unittest.TestCase):
|
||||
setUp = history_tests.HistoryDatabaseTests.setUp
|
||||
close_databases = history_tests.HistoryDatabaseTests.close_databases
|
||||
database = history_tests.HistoryDatabaseTests.database
|
||||
|
||||
def setUp(self): # noqa: F811 - the shared temp dir and database list first
|
||||
history_tests.HistoryDatabaseTests.setUp(self)
|
||||
self.db = self.database()
|
||||
self.output = str(Path(self.temp.name) / "output")
|
||||
os.makedirs(os.path.join(self.output, "sub"))
|
||||
self.image = os.path.join(self.output, "sub", "img_00001_.png")
|
||||
Path(self.image).write_bytes(b"png")
|
||||
|
||||
def save(self, text):
|
||||
return self.db.save_prompt(text=text, prompt_hash=generate_prompt_hash(text))
|
||||
|
||||
def attribute(self, path=None):
|
||||
return stored_run_prompt(self.db, path or self.image, self.output)
|
||||
|
||||
def test_one_run_naming_one_stored_prompt_attributes_the_file(self):
|
||||
prompt_id = self.save("an llm prompt")
|
||||
self.record_run(
|
||||
self.db, self.output, [("sub", "img_00001_.png")], ["an llm prompt"]
|
||||
)
|
||||
self.assertEqual(self.attribute(), prompt_id)
|
||||
|
||||
def test_text_that_is_not_stored_attributes_nothing(self):
|
||||
"""The #342 run: raw editor text was empty, nothing was stored."""
|
||||
self.save("a woman holding a coffee cup")
|
||||
self.record_run(self.db, self.output, [("sub", "img_00001_.png")], [""])
|
||||
self.assertIsNone(self.attribute())
|
||||
|
||||
def test_two_runs_reporting_the_same_file_attribute_nothing(self):
|
||||
self.save("first")
|
||||
self.save("second")
|
||||
self.record_run(self.db, self.output, [("sub", "img_00001_.png")], ["first"])
|
||||
self.record_run(
|
||||
self.db, self.output, [("sub", "img_00001_.png")], ["second"], item=1
|
||||
)
|
||||
self.assertIsNone(self.attribute())
|
||||
|
||||
def test_a_file_written_outside_the_run_attributes_nothing(self):
|
||||
self.save("an llm prompt")
|
||||
self.record_run(
|
||||
self.db,
|
||||
self.output,
|
||||
[("sub", "img_00001_.png")],
|
||||
["an llm prompt"],
|
||||
window=(1_000_000, 2_000_000),
|
||||
)
|
||||
self.assertIsNone(self.attribute())
|
||||
|
||||
def test_contexts_naming_different_prompts_attribute_nothing(self):
|
||||
self.save("left")
|
||||
self.save("right")
|
||||
self.record_run(
|
||||
self.db, self.output, [("sub", "img_00001_.png")], ["left", "right"]
|
||||
)
|
||||
self.assertIsNone(self.attribute())
|
||||
|
||||
def test_files_outside_the_output_root_or_unrecorded_attribute_nothing(self):
|
||||
self.save("an llm prompt")
|
||||
self.record_run(
|
||||
self.db, self.output, [("sub", "img_00001_.png")], ["an llm prompt"]
|
||||
)
|
||||
outside = os.path.join(self.temp.name, "img_00001_.png")
|
||||
Path(outside).write_bytes(b"png")
|
||||
self.assertIsNone(self.attribute(outside))
|
||||
other = os.path.join(self.output, "sub", "other.png")
|
||||
Path(other).write_bytes(b"png")
|
||||
self.assertIsNone(self.attribute(other))
|
||||
self.assertIsNone(stored_run_prompt(self.db, self.image, None))
|
||||
self.assertIsNone(stored_run_prompt(self.db, self.image, "relative/root"))
|
||||
|
||||
def test_a_library_without_history_tables_attributes_nothing(self):
|
||||
broken = mock.Mock()
|
||||
broken.model.get_connection.return_value.execute.side_effect = (
|
||||
sqlite3.OperationalError("no such table")
|
||||
)
|
||||
self.assertIsNone(stored_run_prompt(broken, self.image, self.output))
|
||||
|
||||
def test_the_monitor_links_from_history_when_nothing_else_is_certain(self):
|
||||
prompt_id = self.save(LLM_TEXT)
|
||||
self.record_run(self.db, self.output, [("sub", "img_00001_.png")], [LLM_TEXT])
|
||||
handler = ImageGenerationHandler(self.db, make_tracker(self.db))
|
||||
linked = []
|
||||
handler.link_image_to_prompt = lambda path, prompt, meta: linked.append(
|
||||
prompt["id"]
|
||||
)
|
||||
with (
|
||||
mock.patch(
|
||||
"utils.image_monitor.host_output_directory", return_value=self.output
|
||||
),
|
||||
mock.patch.object(
|
||||
handler.metadata_extractor,
|
||||
"extract_metadata",
|
||||
return_value={"prompt": llm_text_graph()},
|
||||
),
|
||||
):
|
||||
handler.process_new_image(self.image)
|
||||
self.assertEqual(linked, [prompt_id])
|
||||
|
||||
|
||||
class TestScanUsesHistory(RecordedRuns, AdminAPITestCase):
|
||||
"""The output scan applies the same rules as the monitor."""
|
||||
|
||||
def linked_prompt(self, path):
|
||||
row = (
|
||||
self.api.db.model.get_connection()
|
||||
.execute(
|
||||
"SELECT prompt_id FROM generated_images WHERE image_path=?",
|
||||
(str(path),),
|
||||
)
|
||||
.fetchone()
|
||||
)
|
||||
return row[0] if row else None
|
||||
|
||||
def write(self, name):
|
||||
path = self.output_dir / name
|
||||
path.write_bytes(b"png")
|
||||
return path
|
||||
|
||||
def test_an_llm_text_file_links_to_the_prompt_its_run_stored(self):
|
||||
db = self.api.db
|
||||
prompt_id = db.save_prompt(
|
||||
text=LLM_TEXT, prompt_hash=generate_prompt_hash(LLM_TEXT)
|
||||
)
|
||||
path = self.write("llm_00001_.png")
|
||||
self.record_run(db, str(self.output_dir), [("", path.name)], [LLM_TEXT])
|
||||
meta = {"prompt": json.dumps(llm_text_graph())}
|
||||
self.assertEqual(self.api._ingest_scanned_file(path, meta), "linked")
|
||||
self.assertEqual(self.linked_prompt(path), prompt_id)
|
||||
|
||||
def test_a_metadata_less_file_links_through_its_recorded_run(self):
|
||||
"""I1: --disable-metadata outputs carry no prompt; history still proves one."""
|
||||
db = self.api.db
|
||||
prompt_id = db.save_prompt(
|
||||
text=LLM_TEXT, prompt_hash=generate_prompt_hash(LLM_TEXT)
|
||||
)
|
||||
path = self.write("bare_00001_.png")
|
||||
self.record_run(db, str(self.output_dir), [("", path.name)], [LLM_TEXT])
|
||||
self.assertEqual(self.api._ingest_scanned_file(path, {}), "linked")
|
||||
self.assertEqual(self.linked_prompt(path), prompt_id)
|
||||
self.assertEqual(self.api._ingest_scanned_file(path, None), "unchanged")
|
||||
detailed = self.api._ingest_scanned_file(path, {}, detailed=True)
|
||||
self.assertEqual(detailed["status"], "unchanged")
|
||||
self.assertTrue(detailed["title"])
|
||||
|
||||
def test_a_metadata_less_file_without_a_recorded_run_is_still_skipped(self):
|
||||
path = self.write("bare_00002_.png")
|
||||
self.assertIsNone(self.api._ingest_scanned_file(path, {}))
|
||||
self.assertIsNone(self.api._ingest_scanned_file(path, {"workflow": {}}))
|
||||
self.assertIsNone(self.linked_prompt(path))
|
||||
|
||||
def test_the_issue_file_stays_unlinked_even_with_its_pair_record(self):
|
||||
"""Resolved with an empty stored positive: before #342 the scan raised."""
|
||||
db = self.api.db
|
||||
db.save_prompt(
|
||||
text="a woman holding a coffee cup",
|
||||
prompt_hash=generate_prompt_hash("a woman holding a coffee cup"),
|
||||
)
|
||||
path = self.write("moss_00009_.png")
|
||||
self.record_run(db, str(self.output_dir), [("", path.name)], [""])
|
||||
evidence = {
|
||||
"version": 1,
|
||||
"nodes": {
|
||||
"18:35": {
|
||||
"records": [
|
||||
{
|
||||
"text": "",
|
||||
"negative_text": "",
|
||||
"positive_text": LLM_TEXT,
|
||||
"negative_effective_text": "",
|
||||
"prepend_text": LLM_TEXT,
|
||||
"append_text": "",
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
}
|
||||
meta = {
|
||||
"prompt": json.dumps(issue_graph()),
|
||||
"comfyassets_prompt_pairs": json.dumps(evidence),
|
||||
}
|
||||
self.assertEqual(self.api._ingest_scanned_file(path, meta), "unresolved")
|
||||
self.assertIsNone(self.linked_prompt(path))
|
||||
self.assertEqual(db.list_prompt_pairs()["pairs"], [])
|
||||
|
||||
|
||||
class TestHostHelpers(unittest.TestCase):
|
||||
"""The host lookups degrade to "no proof", never to an error."""
|
||||
|
||||
def test_host_output_directory(self):
|
||||
import sys
|
||||
import types
|
||||
|
||||
from database import run_attribution
|
||||
|
||||
cases = (
|
||||
(types.SimpleNamespace(get_output_directory=lambda: "/out"), "/out"),
|
||||
(types.SimpleNamespace(get_output_directory=lambda: ""), None),
|
||||
(types.SimpleNamespace(get_output_directory=lambda: 1 / 0), None),
|
||||
)
|
||||
original = sys.modules.get("folder_paths")
|
||||
try:
|
||||
for module, expected in cases:
|
||||
sys.modules["folder_paths"] = module
|
||||
self.assertEqual(run_attribution.host_output_directory(), expected)
|
||||
sys.modules["folder_paths"] = None # import fails: outside ComfyUI
|
||||
self.assertIsNone(run_attribution.host_output_directory())
|
||||
finally:
|
||||
if original is None:
|
||||
sys.modules.pop("folder_paths", None)
|
||||
else:
|
||||
sys.modules["folder_paths"] = original
|
||||
|
||||
def test_run_window_needs_integer_times_and_a_readable_file(self):
|
||||
from database.run_attribution import _written_during
|
||||
|
||||
self.assertFalse(_written_during(__file__, None, 5))
|
||||
self.assertFalse(_written_during(os.path.join("no", "such.png"), 0, 5))
|
||||
|
||||
def test_executing_run_id(self):
|
||||
import sys
|
||||
import types
|
||||
|
||||
from utils.run_keys import executing_run_id
|
||||
|
||||
def host(context):
|
||||
def get_executing_context():
|
||||
if isinstance(context, Exception):
|
||||
raise context
|
||||
return context
|
||||
|
||||
return types.SimpleNamespace(get_executing_context=get_executing_context)
|
||||
|
||||
original = sys.modules.get("comfy_execution.utils")
|
||||
try:
|
||||
for context, expected in (
|
||||
(types.SimpleNamespace(prompt_id="run-1"), "run-1"),
|
||||
(types.SimpleNamespace(prompt_id=""), None),
|
||||
(None, None),
|
||||
(RuntimeError("no context"), None),
|
||||
):
|
||||
sys.modules["comfy_execution.utils"] = host(context)
|
||||
self.assertEqual(executing_run_id(), expected)
|
||||
sys.modules["comfy_execution.utils"] = None
|
||||
self.assertIsNone(executing_run_id())
|
||||
finally:
|
||||
if original is None:
|
||||
sys.modules.pop("comfy_execution.utils", None)
|
||||
else:
|
||||
sys.modules["comfy_execution.utils"] = original
|
||||
|
||||
|
||||
class TestMonitorHistoryRecheck(RecordedRuns, unittest.TestCase):
|
||||
"""I1: a keyless image gets one history-only re-check when its run is sealed."""
|
||||
|
||||
def setUp(self):
|
||||
history_tests.HistoryDatabaseTests.setUp(self)
|
||||
self.db = history_tests.HistoryDatabaseTests.database(self)
|
||||
self.output = str(Path(self.temp.name) / "output")
|
||||
os.makedirs(self.output)
|
||||
self.handler = ImageGenerationHandler(self.db, make_tracker(self.db))
|
||||
self.handler.processing_delay = 0
|
||||
self.handler.settle_interval = 0.01
|
||||
self.addCleanup(self.handler.shutdown)
|
||||
self.linked = []
|
||||
self.handler.link_image_to_prompt = lambda path, prompt, meta: (
|
||||
self.linked.append((os.path.basename(path), prompt["id"]))
|
||||
)
|
||||
patches = (
|
||||
mock.patch(
|
||||
"utils.image_monitor.host_output_directory", return_value=self.output
|
||||
),
|
||||
mock.patch.object(
|
||||
self.handler.metadata_extractor, "extract_metadata", return_value=None
|
||||
),
|
||||
)
|
||||
for patch in patches:
|
||||
patch.start()
|
||||
self.addCleanup(patch.stop)
|
||||
|
||||
close_databases = history_tests.HistoryDatabaseTests.close_databases
|
||||
|
||||
def image(self, name):
|
||||
path = os.path.join(self.output, name)
|
||||
Path(path).write_bytes(b"png")
|
||||
return path
|
||||
|
||||
def save(self, text):
|
||||
return self.db.save_prompt(text=text, prompt_hash=generate_prompt_hash(text))
|
||||
|
||||
def wait_for(self, predicate, timeout=5.0):
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if predicate():
|
||||
return True
|
||||
time.sleep(0.01)
|
||||
return predicate()
|
||||
|
||||
def test_the_run_ending_after_processing_links_the_image_once(self):
|
||||
prompt_id = self.save(LLM_TEXT)
|
||||
path = self.image("late_00001_.png")
|
||||
self.handler.process_new_image(path) # the run is still executing
|
||||
self.assertEqual(self.linked, [])
|
||||
self.assertIn(
|
||||
os.path.normcase(os.path.realpath(path)), self.handler.awaiting_history()
|
||||
)
|
||||
self.record_run(self.db, self.output, [("", "late_00001_.png")], [LLM_TEXT])
|
||||
self.assertTrue(self.wait_for(lambda: self.linked))
|
||||
self.assertTrue(self.wait_for(lambda: self.handler.pending_count() == 0))
|
||||
self.assertEqual(self.linked, [("late_00001_.png", prompt_id)])
|
||||
self.assertEqual(self.handler.awaiting_history(), [])
|
||||
# Another run reporting the same name later is not a second re-check
|
||||
self.record_run(
|
||||
self.db, self.output, [("", "late_00001_.png")], [LLM_TEXT], item=1
|
||||
)
|
||||
self.assertTrue(self.wait_for(lambda: self.handler.pending_count() == 0))
|
||||
self.assertEqual(len(self.linked), 1)
|
||||
|
||||
def test_a_seal_during_the_first_pass_is_not_lost(self):
|
||||
"""R1: the run seals while the image's own pass is still pending."""
|
||||
prompt_id = self.save(LLM_TEXT)
|
||||
path = self.image("race_00001_.png")
|
||||
real = self.handler._history_prompt
|
||||
sealed = []
|
||||
|
||||
def history(image_path):
|
||||
result = real(image_path) # before the run's outputs commit: None
|
||||
if not sealed:
|
||||
sealed.append(True)
|
||||
# The writer commits and notifies now; the path is still pending
|
||||
self.record_run(
|
||||
self.db, self.output, [("", "race_00001_.png")], [LLM_TEXT]
|
||||
)
|
||||
return result
|
||||
|
||||
self.handler._history_prompt = history
|
||||
self.assertTrue(self.handler.schedule(path))
|
||||
self.assertTrue(self.wait_for(lambda: self.linked))
|
||||
self.assertTrue(self.wait_for(lambda: self.handler.pending_count() == 0))
|
||||
self.assertEqual(self.linked, [("race_00001_.png", prompt_id)])
|
||||
|
||||
def test_a_seal_while_the_pending_set_is_full_is_rechecked_later(self):
|
||||
from utils import image_monitor
|
||||
|
||||
prompt_id = self.save(LLM_TEXT)
|
||||
path = self.image("full_00001_.png")
|
||||
self.handler.process_new_image(path) # waiting, unlinked
|
||||
blocker = self.image("blocker.png")
|
||||
release = threading.Event()
|
||||
original = self.handler.process_new_image
|
||||
|
||||
def slow(image_path, prompt_snapshot=None):
|
||||
if image_path == blocker:
|
||||
release.wait(5)
|
||||
return original(image_path, prompt_snapshot=prompt_snapshot)
|
||||
|
||||
self.handler.process_new_image = slow
|
||||
with mock.patch.object(image_monitor, "MAX_PENDING_IMAGES", 1):
|
||||
self.assertTrue(self.handler.schedule(blocker))
|
||||
self.assertTrue(self.wait_for(lambda: self.handler.pending_count() == 1))
|
||||
self.record_run(self.db, self.output, [("", "full_00001_.png")], [LLM_TEXT])
|
||||
self.assertEqual(self.linked, [], "not scheduled while the set is full")
|
||||
release.set()
|
||||
self.assertTrue(self.wait_for(lambda: self.linked))
|
||||
self.assertTrue(self.wait_for(lambda: self.handler.pending_count() == 0))
|
||||
self.assertEqual(self.linked, [("full_00001_.png", prompt_id)])
|
||||
|
||||
def test_a_run_naming_nothing_stored_leaves_the_image_unlinked(self):
|
||||
path = self.image("empty_00001_.png")
|
||||
self.handler.process_new_image(path)
|
||||
self.record_run(self.db, self.output, [("", "empty_00001_.png")], [""])
|
||||
self.assertTrue(self.wait_for(lambda: not self.handler.awaiting_history()))
|
||||
self.assertTrue(self.wait_for(lambda: self.handler.pending_count() == 0))
|
||||
self.assertEqual(self.linked, [])
|
||||
|
||||
def test_history_found_at_once_leaves_nothing_awaiting(self):
|
||||
prompt_id = self.save(LLM_TEXT)
|
||||
self.record_run(self.db, self.output, [("", "done_00001_.png")], [LLM_TEXT])
|
||||
self.handler.process_new_image(self.image("done_00001_.png"))
|
||||
self.assertEqual(self.linked, [("done_00001_.png", prompt_id)])
|
||||
self.assertEqual(self.handler.awaiting_history(), [])
|
||||
|
||||
def test_keyed_images_do_not_wait_for_history(self):
|
||||
with mock.patch.object(
|
||||
self.handler.metadata_extractor,
|
||||
"extract_metadata",
|
||||
return_value={"prompt": llm_text_graph()},
|
||||
):
|
||||
self.handler.process_new_image(self.image("keyed_00001_.png"))
|
||||
self.assertEqual(self.handler.awaiting_history(), [])
|
||||
|
||||
def test_the_waiting_set_is_bounded_and_expires(self):
|
||||
from utils import image_monitor
|
||||
|
||||
with mock.patch.object(image_monitor, "MAX_AWAITING_HISTORY", 2):
|
||||
for n in range(3):
|
||||
self.handler.process_new_image(self.image(f"b_{n}.png"))
|
||||
self.assertEqual(
|
||||
[os.path.basename(p) for p in self.handler.awaiting_history()],
|
||||
["b_1.png", "b_2.png"],
|
||||
)
|
||||
later = time.monotonic() + image_monitor.AWAIT_HISTORY_SECONDS + 1
|
||||
with mock.patch.object(image_monitor.time, "monotonic", return_value=later):
|
||||
self.assertEqual(self.handler.awaiting_history(), [])
|
||||
|
||||
def test_shutdown_forgets_waiting_images_and_a_dead_handler_is_dropped(self):
|
||||
import gc
|
||||
|
||||
from database import run_attribution
|
||||
|
||||
self.handler.process_new_image(self.image("gone_00001_.png"))
|
||||
self.handler.shutdown()
|
||||
self.assertEqual(self.handler.awaiting_history(), [])
|
||||
other = ImageGenerationHandler(self.db, make_tracker(self.db))
|
||||
with mock.patch.object(
|
||||
other.metadata_extractor, "extract_metadata", return_value=None
|
||||
):
|
||||
other.process_new_image(self.image("gone_00002_.png"))
|
||||
del other
|
||||
gc.collect()
|
||||
run_attribution.notify_outputs_sealed("run", []) # no dead callback raises
|
||||
|
||||
|
||||
class TestBoundedLookup(RecordedRuns, unittest.TestCase):
|
||||
"""M5: history lookups are batched, and the history schema is unchanged."""
|
||||
|
||||
setUp = history_tests.HistoryDatabaseTests.setUp
|
||||
close_databases = history_tests.HistoryDatabaseTests.close_databases
|
||||
database = history_tests.HistoryDatabaseTests.database
|
||||
|
||||
def output_lookups(self, db, action):
|
||||
statements = []
|
||||
conn = db.model.get_connection()
|
||||
conn.set_trace_callback(statements.append)
|
||||
try:
|
||||
result = action()
|
||||
finally:
|
||||
conn.set_trace_callback(None)
|
||||
return result, [s for s in statements if "FROM execution_run_outputs" in s]
|
||||
|
||||
def test_one_query_per_chunk_not_per_file_and_same_answers(self):
|
||||
from database import run_attribution
|
||||
|
||||
db = self.database()
|
||||
output = str(Path(self.temp.name) / "output")
|
||||
os.makedirs(output)
|
||||
prompt_id = db.save_prompt(
|
||||
text=LLM_TEXT, prompt_hash=generate_prompt_hash(LLM_TEXT)
|
||||
)
|
||||
names = [f"img_{n:05}_.png" for n in range(25)]
|
||||
paths = [os.path.join(output, name) for name in names]
|
||||
for path in paths:
|
||||
Path(path).write_bytes(b"png")
|
||||
self.record_run(db, output, [("", n) for n in names[:10]], [LLM_TEXT])
|
||||
self.record_run(db, output, [("", n) for n in names[5:10]], [""], item=1)
|
||||
with mock.patch.object(run_attribution, "LOOKUP_CHUNK", 10):
|
||||
result, lookups = self.output_lookups(
|
||||
db, lambda: run_attribution.stored_run_prompts(db, paths, output)
|
||||
)
|
||||
self.assertEqual(len(lookups), 3) # 25 names in chunks of 10
|
||||
expected = {path: None for path in paths}
|
||||
expected.update({path: prompt_id for path in paths[:5]})
|
||||
self.assertEqual(result, expected)
|
||||
for path in paths[:12]:
|
||||
self.assertEqual(
|
||||
run_attribution.stored_run_prompt(db, path, output), expected[path]
|
||||
)
|
||||
|
||||
def test_the_history_schema_gains_no_index(self):
|
||||
from database.execution_history_schema import history_schema_error
|
||||
|
||||
db = self.database()
|
||||
conn = db.model.get_connection()
|
||||
indexes = [
|
||||
row[1]
|
||||
for row in conn.execute("PRAGMA index_list(execution_run_outputs)")
|
||||
if row[3] != "pk"
|
||||
]
|
||||
self.assertEqual(indexes, [])
|
||||
self.assertIsNone(history_schema_error(conn, allow_missing=False))
|
||||
|
||||
|
||||
class TestScanBatchLookup(RecordedRuns, AdminAPITestCase):
|
||||
"""M5: a scan batch asks history once per chunk of names, not per file."""
|
||||
|
||||
def test_a_batch_its_metadata_links_never_asks_history(self):
|
||||
"""R3: the batch lookup is made only when a file needs it."""
|
||||
files = []
|
||||
for n in range(3):
|
||||
path = self.output_dir / f"typed_{n:05}_.png"
|
||||
path.write_bytes(b"png")
|
||||
files.append(path)
|
||||
meta = {"prompt": json.dumps(issue_graph(text="typed mountains"))}
|
||||
statements = []
|
||||
conn = self.api.db.model.get_connection()
|
||||
conn.set_trace_callback(statements.append)
|
||||
try:
|
||||
counts = self.api._ingest_scan_batch_sync([(f, meta) for f in files])
|
||||
finally:
|
||||
conn.set_trace_callback(None)
|
||||
self.assertEqual(counts["added"] + counts["linked"], 3, counts)
|
||||
lookups = [s for s in statements if "FROM execution_run_outputs" in s]
|
||||
self.assertEqual(lookups, [])
|
||||
|
||||
def test_metadata_less_batch_links_with_one_history_lookup(self):
|
||||
db = self.api.db
|
||||
prompt_id = db.save_prompt(
|
||||
text=LLM_TEXT, prompt_hash=generate_prompt_hash(LLM_TEXT)
|
||||
)
|
||||
files = []
|
||||
for n in range(6):
|
||||
path = self.output_dir / f"bare_{n:05}_.png"
|
||||
path.write_bytes(b"png")
|
||||
files.append(path)
|
||||
self.record_run(
|
||||
db, str(self.output_dir), [("", f.name) for f in files[:4]], [LLM_TEXT]
|
||||
)
|
||||
statements = []
|
||||
conn = db.model.get_connection()
|
||||
conn.set_trace_callback(statements.append)
|
||||
try:
|
||||
counts = self.api._ingest_scan_batch_sync([(f, {}) for f in files])
|
||||
finally:
|
||||
conn.set_trace_callback(None)
|
||||
lookups = [s for s in statements if "FROM execution_run_outputs" in s]
|
||||
self.assertEqual(len(lookups), 1)
|
||||
self.assertEqual(counts["linked"], 4)
|
||||
linked = {
|
||||
row[0]
|
||||
for row in conn.execute(
|
||||
"SELECT filename FROM generated_images WHERE prompt_id=?",
|
||||
(prompt_id,),
|
||||
)
|
||||
}
|
||||
self.assertEqual(linked, {f.name for f in files[:4]})
|
||||
|
||||
|
||||
class TestLinkedOutputRoot(RecordedRuns, unittest.TestCase):
|
||||
"""R2: ComfyUI's output folder reached through a symlink or junction."""
|
||||
|
||||
setUp = history_tests.HistoryDatabaseTests.setUp
|
||||
close_databases = history_tests.HistoryDatabaseTests.close_databases
|
||||
database = history_tests.HistoryDatabaseTests.database
|
||||
|
||||
def linked_root(self):
|
||||
self.real = str(Path(self.temp.name) / "big_drive" / "output")
|
||||
os.makedirs(os.path.join(self.real, "sub"))
|
||||
link = str(Path(self.temp.name) / "output_link")
|
||||
try:
|
||||
os.symlink(self.real, link, target_is_directory=True)
|
||||
except (OSError, NotImplementedError) as error:
|
||||
self.skipTest(f"symlinks unavailable: {error}")
|
||||
return link
|
||||
|
||||
def test_either_spelling_of_the_file_is_attributed(self):
|
||||
db = self.database()
|
||||
link = self.linked_root()
|
||||
prompt_id = db.save_prompt(
|
||||
text=LLM_TEXT, prompt_hash=generate_prompt_hash(LLM_TEXT)
|
||||
)
|
||||
real_file = os.path.join(self.real, "sub", "img_00001_.png")
|
||||
Path(real_file).write_bytes(b"png")
|
||||
# History keeps the host's spelling of its output folder (the link)
|
||||
self.record_run(db, link, [("sub", "img_00001_.png")], [LLM_TEXT])
|
||||
link_file = os.path.join(link, "sub", "img_00001_.png")
|
||||
self.assertEqual(stored_run_prompt(db, real_file, link), prompt_id)
|
||||
self.assertEqual(stored_run_prompt(db, link_file, link), prompt_id)
|
||||
# The real folder named as the root is another root key: not attributed
|
||||
self.assertIsNone(stored_run_prompt(db, real_file, self.real))
|
||||
|
||||
def test_the_monitor_rechecks_a_file_seen_through_the_real_path(self):
|
||||
db = self.database()
|
||||
link = self.linked_root()
|
||||
prompt_id = db.save_prompt(
|
||||
text=LLM_TEXT, prompt_hash=generate_prompt_hash(LLM_TEXT)
|
||||
)
|
||||
real_file = os.path.join(self.real, "late_00001_.png")
|
||||
Path(real_file).write_bytes(b"png")
|
||||
handler = ImageGenerationHandler(db, make_tracker(db))
|
||||
handler.processing_delay = 0
|
||||
handler.settle_interval = 0.01
|
||||
self.addCleanup(handler.shutdown)
|
||||
linked = []
|
||||
handler.link_image_to_prompt = lambda path, prompt, meta: linked.append(
|
||||
prompt["id"]
|
||||
)
|
||||
with (
|
||||
mock.patch("utils.image_monitor.host_output_directory", return_value=link),
|
||||
mock.patch.object(
|
||||
handler.metadata_extractor, "extract_metadata", return_value=None
|
||||
),
|
||||
):
|
||||
handler.process_new_image(real_file)
|
||||
self.assertEqual(linked, [])
|
||||
self.record_run(db, link, [("", "late_00001_.png")], [LLM_TEXT])
|
||||
deadline = time.monotonic() + 5
|
||||
while not linked and time.monotonic() < deadline:
|
||||
time.sleep(0.01)
|
||||
self.assertEqual(linked, [prompt_id])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -477,11 +477,20 @@ class TestPendingFirstUse(unittest.TestCase):
|
||||
class FakeTracker:
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
self.runs = []
|
||||
|
||||
def set_current_prompt(self, prompt_text, additional_data=None, push_to_queue=True):
|
||||
def set_current_prompt(
|
||||
self, prompt_text, additional_data=None, push_to_queue=True, **run
|
||||
):
|
||||
self.calls.append({"text": prompt_text, "push_to_queue": push_to_queue})
|
||||
self.runs.append(run)
|
||||
return "exec-1"
|
||||
|
||||
def mark_empty(self, run_key=None, run_id=None, push_to_queue=False):
|
||||
self.calls.append(
|
||||
{"empty": True, "run_key": run_key, "push_to_queue": push_to_queue}
|
||||
)
|
||||
|
||||
|
||||
class TestNodeRoles(DbTestCase):
|
||||
def _node(self):
|
||||
|
||||
Vendored
+8
-2
@@ -23,19 +23,25 @@ test('execution script discovers only the new matrix and preserves all 78 origin
|
||||
env: { ...process.env, PM_COMFY_ROOT: '/unused-pinned-checkout' } });
|
||||
const stock = list('tools/vendor/comfyui.config.mjs');
|
||||
// + local Inspect host case (inspect-sender.spec.mjs, 2 projects), + the S10 renderer case (prompt-bridge.spec.mjs, 2 projects)
|
||||
assert.match(stock, /Total: 84 tests in 8 files/);
|
||||
// + #343 linked text: five node types, PromptSearchList, the pre-#343 workflows, legacy Primitive, moved link,
|
||||
// paste/clone/undo, wildcard Primitive, subgraph Primitive (text-link.spec.mjs, 2 projects)
|
||||
assert.match(stock, /Total: 110 tests in 9 files/);
|
||||
assert.doesNotMatch(stock, /execution-history/);
|
||||
assert.doesNotMatch(stock, /execution-outputs/);
|
||||
assert.doesNotMatch(stock, /library-location/); // it starts its own fresh host (#249 S6b)
|
||||
assert.doesNotMatch(stock, /image-attribution/); // needs the owned fixture pack (#342)
|
||||
const executions = list('tools/vendor/playwright.execution.config.mjs');
|
||||
assert.match(executions, /\[classic-start\]/);
|
||||
assert.match(executions, /\[nodes2-start\]/);
|
||||
assert.match(executions, /Total: 48 tests in 3 files/); // + the fresh-host library location case
|
||||
// + the fresh-host library location case, + #342 image attribution, + #343 linked text
|
||||
assert.match(executions, /Total: 90 tests in 5 files/);
|
||||
for (const project of ['classic-start', 'nodes2-start']) {
|
||||
const cases = executions.split('\n').filter(line => line.includes(`[${project}]`));
|
||||
assert.equal(cases.filter(line => line.includes('execution-history.spec.mjs')).length, 15);
|
||||
assert.equal(cases.filter(line => line.includes('execution-outputs.spec.mjs')).length, 8);
|
||||
assert.equal(cases.filter(line => line.includes('library-location.spec.mjs')).length, 1);
|
||||
assert.equal(cases.filter(line => line.includes('image-attribution.spec.mjs')).length, 2);
|
||||
assert.equal(cases.filter(line => line.includes('execution-text-link.spec.mjs')).length, 19);
|
||||
}
|
||||
assert.doesNotMatch(executions, /prompt-manager.spec/);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
"""Review, and on request repair, image links that nothing certain supports (#342).
|
||||
|
||||
Before #342 the image monitor linked an image it could not identify to the
|
||||
newest prompt in the database or to a stale tracker entry. This tool judges the
|
||||
selected images again by today's rules, using only what is certain:
|
||||
|
||||
- the image's own metadata, as the monitor reads it;
|
||||
- the output scan's resolver, and its legacy browser-parser floor (#170);
|
||||
- recorded execution history (``database.run_attribution``).
|
||||
|
||||
Each image gets one verdict:
|
||||
|
||||
``keep`` a certain source names the prompt it is linked to
|
||||
``relink`` every certain source names one other stored prompt
|
||||
``unlink`` nothing certain names a prompt (the link was a guess)
|
||||
``conflict`` certain sources disagree: decide by hand
|
||||
``missing`` the file is gone: leave it to Maintenance's missing links
|
||||
|
||||
Dry run by default: the database is copied (SQLite backup from a read-only
|
||||
connection) into a temporary folder and judged there, so the library is never
|
||||
written. ``--apply`` writes the ``relink`` and ``unlink`` verdicts of the
|
||||
selected images, after saving a backup beside the database
|
||||
(``<db>.pre-relink_<stamp>``). Only ids named on the command line are ever
|
||||
changed, and only while each is still linked to the prompt it was judged on.
|
||||
|
||||
What ``--apply`` does (APPLY_EFFECTS below, printed by every run):
|
||||
|
||||
- ``relink`` points the image's link record at the other prompt and drops its
|
||||
pair links (they named the old positive).
|
||||
- ``unlink`` removes only the image's link record (its ``generated_images`` row
|
||||
with the metadata snapshot stored in it, and its pair links). That is how the
|
||||
app represents an unlinked image, exactly like a file the scan could not
|
||||
attribute: the file stays on disk and in the gallery as unlinked, and its
|
||||
favorite star and tags (kept by file, not by link) are untouched.
|
||||
|
||||
A batch item linked through the monitor's queue (PromptSearchList) can't be
|
||||
proven after the fact and reads ``unlink``: review before applying.
|
||||
|
||||
python3 tools/review_image_links.py <db> --output-dir <ComfyUI output> \\
|
||||
--ids 58005-58010
|
||||
python3 tools/review_image_links.py <db> --output-dir <dir> --prompt 5278
|
||||
python3 tools/review_image_links.py <db> --output-dir <dir> \\
|
||||
--ids 58005-58010 --apply
|
||||
|
||||
Stop ComfyUI before ``--apply``. Run from this repository with its venv; the
|
||||
package's logger may write to its usual log file.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
import tempfile
|
||||
from contextlib import closing
|
||||
from pathlib import Path
|
||||
|
||||
_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_ROOT))
|
||||
|
||||
from database.maintenance import _delete_ids # noqa: E402
|
||||
from database.operations import PromptDatabase # noqa: E402
|
||||
from database.pair_ingestion import blank_positive # noqa: E402
|
||||
from database.run_attribution import stored_run_prompt # noqa: E402
|
||||
from utils.generation_params import ( # noqa: E402
|
||||
PLACEHOLDERS,
|
||||
GenerationParamsError,
|
||||
extract_generation_params,
|
||||
)
|
||||
from utils.hashing import generate_prompt_hash # noqa: E402
|
||||
from utils.image_monitor import ImageGenerationHandler # noqa: E402
|
||||
from utils.prompt_pair_resolver import resolve_prompt_pair # noqa: E402
|
||||
|
||||
VERDICTS = ("keep", "relink", "unlink", "conflict", "missing")
|
||||
APPLY_EFFECTS = (
|
||||
"--apply, for ids named with --ids only, while each is still linked to the "
|
||||
"prompt it was judged on:\n"
|
||||
" relink: points the image's link record at the other prompt and drops "
|
||||
"its pair links;\n"
|
||||
" unlink: removes only the image's link record (its stored metadata "
|
||||
"snapshot and pair links go with it); the file stays on disk and in the "
|
||||
"gallery as unlinked, with its favorite and tags.\n"
|
||||
)
|
||||
# Ids per query (well under SQLite's bound-parameter limit)
|
||||
ID_BATCH = 500
|
||||
|
||||
|
||||
def parse_ids(text):
|
||||
"""``"1,4-6"`` -> ``[1, 4, 5, 6]``; raises ValueError on anything else."""
|
||||
ids = set()
|
||||
for part in (text or "").split(","):
|
||||
part = part.strip()
|
||||
if not part:
|
||||
continue
|
||||
low, _, high = part.partition("-")
|
||||
first, last = int(low), int(high or low)
|
||||
if first < 1 or last < first or last - first > 100_000:
|
||||
raise ValueError(f"Bad id range: {part}")
|
||||
ids.update(range(first, last + 1))
|
||||
return sorted(ids)
|
||||
|
||||
|
||||
def _stored_id(db, text):
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
return None
|
||||
row = db.get_prompt_by_hash(generate_prompt_hash(text.strip()))
|
||||
return row["id"] if row else None
|
||||
|
||||
|
||||
def _parser_text(metadata):
|
||||
"""The legacy browser parser's positive prompt (the scan's floor), or None."""
|
||||
records = {key: metadata.get(key) for key in ("prompt", "workflow")}
|
||||
try:
|
||||
positive = extract_generation_params(records)["positivePrompt"]
|
||||
except (GenerationParamsError, ValueError, TypeError, KeyError):
|
||||
return None
|
||||
if not isinstance(positive, str) or positive == PLACEHOLDERS["positivePrompt"]:
|
||||
return None
|
||||
return positive.strip() or None
|
||||
|
||||
|
||||
def certain_prompts(db, handler, path, metadata, output_dir):
|
||||
"""``{source: prompt id}`` of every certain source that names a stored prompt."""
|
||||
found = {}
|
||||
if metadata:
|
||||
state, prompt = handler._find_prompt_from_metadata(metadata)
|
||||
if state == "linked" and prompt:
|
||||
found["metadata"] = prompt["id"]
|
||||
resolution = resolve_prompt_pair(metadata)
|
||||
if resolution["state"] == "resolved" and not blank_positive(resolution):
|
||||
positive = resolution["positive"]
|
||||
scanned = _stored_id(db, positive.get("storage_text", positive["text"]))
|
||||
else:
|
||||
scanned = _stored_id(db, _parser_text(metadata))
|
||||
if scanned is not None:
|
||||
found["scan"] = scanned
|
||||
recorded = stored_run_prompt(db, path, output_dir)
|
||||
if recorded is not None:
|
||||
found["history"] = recorded
|
||||
return found
|
||||
|
||||
|
||||
def judge(db, handler, row, output_dir):
|
||||
"""One image's verdict dict."""
|
||||
path = row["image_path"]
|
||||
verdict = {
|
||||
"image_id": row["id"],
|
||||
"file": path,
|
||||
"linked_prompt": row["prompt_id"],
|
||||
"sources": {},
|
||||
"target": None,
|
||||
}
|
||||
if not os.path.isfile(path):
|
||||
return {**verdict, "verdict": "missing"}
|
||||
try:
|
||||
metadata = handler.metadata_extractor.extract_metadata(path)
|
||||
except Exception:
|
||||
metadata = None
|
||||
sources = certain_prompts(db, handler, path, metadata, output_dir)
|
||||
named = set(sources.values())
|
||||
verdict["sources"] = sources
|
||||
if not named:
|
||||
return {**verdict, "verdict": "unlink"}
|
||||
if len(named) > 1:
|
||||
return {**verdict, "verdict": "conflict"}
|
||||
(target,) = named
|
||||
if target == row["prompt_id"]:
|
||||
return {**verdict, "verdict": "keep", "target": target}
|
||||
return {**verdict, "verdict": "relink", "target": target}
|
||||
|
||||
|
||||
def selected_rows(db, ids, prompt_id):
|
||||
"""The named images and every image of ``prompt_id``, by id."""
|
||||
conn = db.model.get_connection()
|
||||
columns = "SELECT id, prompt_id, image_path FROM generated_images "
|
||||
rows = {}
|
||||
for start in range(0, len(ids), ID_BATCH):
|
||||
batch = ids[start : start + ID_BATCH]
|
||||
marks = ",".join("?" for _ in batch)
|
||||
query = columns + f"WHERE id IN ({marks})" # only ? marks
|
||||
rows.update((row["id"], row) for row in conn.execute(query, batch))
|
||||
if prompt_id is not None:
|
||||
found = conn.execute(columns + "WHERE prompt_id = ?", (prompt_id,))
|
||||
rows.update((row["id"], row) for row in found)
|
||||
return [rows[key] for key in sorted(rows)]
|
||||
|
||||
|
||||
def review(db, ids, prompt_id, output_dir):
|
||||
handler = ImageGenerationHandler(db, None)
|
||||
return [
|
||||
judge(db, handler, row, output_dir) for row in selected_rows(db, ids, prompt_id)
|
||||
]
|
||||
|
||||
|
||||
def apply(db, verdicts, ids):
|
||||
"""Write the relink/unlink verdicts of explicitly named ids; returns counts.
|
||||
|
||||
An image whose link changed since it was judged (now another prompt, or
|
||||
gone) is skipped.
|
||||
"""
|
||||
allowed = set(ids)
|
||||
chosen = [
|
||||
v
|
||||
for v in verdicts
|
||||
if v["verdict"] in ("relink", "unlink") and v["image_id"] in allowed
|
||||
]
|
||||
with db._pair_transaction() as conn:
|
||||
current = [
|
||||
item
|
||||
for item in chosen
|
||||
if (
|
||||
row := conn.execute(
|
||||
"SELECT prompt_id FROM generated_images WHERE id=?",
|
||||
(item["image_id"],),
|
||||
).fetchone()
|
||||
)
|
||||
and row[0] == item["linked_prompt"]
|
||||
]
|
||||
relink = [item for item in current if item["verdict"] == "relink"]
|
||||
for item in relink:
|
||||
# A pair link records the old positive: it goes with the old link
|
||||
conn.execute(
|
||||
"DELETE FROM image_prompt_pairs WHERE image_id=?", (item["image_id"],)
|
||||
)
|
||||
conn.execute(
|
||||
"UPDATE generated_images SET prompt_id=? WHERE id=? AND prompt_id=?",
|
||||
(item["target"], item["image_id"], item["linked_prompt"]),
|
||||
)
|
||||
removed = _delete_ids(
|
||||
conn,
|
||||
"generated_images",
|
||||
[item["image_id"] for item in current if item["verdict"] == "unlink"],
|
||||
)
|
||||
return {
|
||||
"relinked": len(relink),
|
||||
"unlinked": removed,
|
||||
"skipped": len(chosen) - len(current),
|
||||
}
|
||||
|
||||
|
||||
def backup_database(source, target):
|
||||
"""SQLite online backup of ``source`` (opened read-only) into ``target``."""
|
||||
uri = Path(os.path.abspath(source)).as_uri() + "?mode=ro"
|
||||
with (
|
||||
closing(sqlite3.connect(uri, uri=True)) as src,
|
||||
closing(sqlite3.connect(target)) as dst,
|
||||
):
|
||||
src.backup(dst)
|
||||
|
||||
|
||||
def _print(verdicts, out):
|
||||
for item in verdicts:
|
||||
sources = ", ".join(f"{k}={v}" for k, v in sorted(item["sources"].items()))
|
||||
target = f" -> {item['target']}" if item["verdict"] == "relink" else ""
|
||||
linked = item["linked_prompt"]
|
||||
out.write(
|
||||
f"{item['image_id']:>8} {item['verdict']:<8} prompt {linked}{target}"
|
||||
f" [{sources or 'nothing certain'}] {item['file']}\n"
|
||||
)
|
||||
counts = {name: sum(v["verdict"] == name for v in verdicts) for name in VERDICTS}
|
||||
out.write(" ".join(f"{name}: {count}" for name, count in counts.items()) + "\n")
|
||||
|
||||
|
||||
def main(argv=None, out=sys.stdout):
|
||||
parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
|
||||
parser.add_argument("database", help="the library database (prompts.db)")
|
||||
parser.add_argument(
|
||||
"--output-dir", required=True, help="ComfyUI's output directory (history)"
|
||||
)
|
||||
parser.add_argument("--ids", default="", help="image ids, e.g. 58005-58010,58020")
|
||||
parser.add_argument("--prompt", type=int, help="every image linked to this prompt")
|
||||
parser.add_argument(
|
||||
"--apply", action="store_true", help="write relink/unlink for --ids images"
|
||||
)
|
||||
parser.add_argument("--json", action="store_true", help="print JSON verdicts")
|
||||
args = parser.parse_args(argv)
|
||||
try:
|
||||
ids = parse_ids(args.ids)
|
||||
except ValueError as error:
|
||||
parser.error(str(error))
|
||||
if not ids and args.prompt is None:
|
||||
parser.error("select images with --ids and/or --prompt")
|
||||
if args.apply and not ids:
|
||||
parser.error("--apply changes only images named with --ids")
|
||||
if not os.path.isfile(args.database):
|
||||
parser.error(f"no database at {args.database}")
|
||||
|
||||
if not args.apply:
|
||||
with tempfile.TemporaryDirectory() as scratch:
|
||||
copy = os.path.join(scratch, "review.db")
|
||||
backup_database(args.database, copy)
|
||||
db = PromptDatabase(copy)
|
||||
try:
|
||||
verdicts = review(db, ids, args.prompt, args.output_dir)
|
||||
finally:
|
||||
db.close_all()
|
||||
_report(verdicts, args.json, out)
|
||||
out.write(APPLY_EFFECTS)
|
||||
out.write("Dry run: nothing was changed.\n")
|
||||
return 0
|
||||
|
||||
stamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
backup = f"{args.database}.pre-relink_{stamp}"
|
||||
backup_database(args.database, backup)
|
||||
out.write(f"Backup: {backup}\n")
|
||||
db = PromptDatabase(args.database)
|
||||
try:
|
||||
verdicts = review(db, ids, args.prompt, args.output_dir)
|
||||
_report(verdicts, args.json, out)
|
||||
result = apply(db, verdicts, ids)
|
||||
finally:
|
||||
db.close_all()
|
||||
out.write(
|
||||
f"Applied: {result['relinked']} relinked, {result['unlinked']} unlinked, "
|
||||
f"{result['skipped']} skipped (link changed since review)\n"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _report(verdicts, as_json, out):
|
||||
if as_json:
|
||||
out.write(json.dumps(verdicts, indent=2, sort_keys=True) + "\n")
|
||||
else:
|
||||
_print(verdicts, out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Vendored
+2
-1
@@ -19,7 +19,8 @@ const servers = [{
|
||||
|
||||
export default defineConfig({
|
||||
testDir: '../../tests/comfyui',
|
||||
testIgnore: ['execution-history.spec.mjs', 'execution-outputs.spec.mjs', 'library-location.spec.mjs'],
|
||||
testIgnore: ['execution-history.spec.mjs', 'execution-outputs.spec.mjs', 'execution-text-link.spec.mjs', 'library-location.spec.mjs',
|
||||
'image-attribution.spec.mjs'],
|
||||
outputDir: './test-results/comfyui',
|
||||
timeout: 60000,
|
||||
workers: 1,
|
||||
|
||||
+15
-3
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"admin.js": {
|
||||
"lines": 32,
|
||||
"branches": 58,
|
||||
"branches": 61,
|
||||
"functions": 27,
|
||||
"statements": 32
|
||||
},
|
||||
@@ -149,6 +149,12 @@
|
||||
"functions": 56,
|
||||
"statements": 57
|
||||
},
|
||||
"text-links.js": {
|
||||
"lines": 98,
|
||||
"branches": 98,
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
"theme-bridge.js": {
|
||||
"lines": 90,
|
||||
"branches": 53,
|
||||
@@ -175,7 +181,7 @@
|
||||
},
|
||||
"workspace/activity-scan.mjs": {
|
||||
"lines": 96,
|
||||
"branches": 78,
|
||||
"branches": 87,
|
||||
"functions": 98,
|
||||
"statements": 96
|
||||
},
|
||||
@@ -415,7 +421,7 @@
|
||||
},
|
||||
"workspace/maintenance-review.mjs": {
|
||||
"lines": 94,
|
||||
"branches": 85,
|
||||
"branches": 86,
|
||||
"functions": 90,
|
||||
"statements": 94
|
||||
},
|
||||
@@ -497,6 +503,12 @@
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
"workspace/system-backups.mjs": {
|
||||
"lines": 95,
|
||||
"branches": 88,
|
||||
"functions": 94,
|
||||
"statements": 95
|
||||
},
|
||||
"workspace/system-controls.mjs": {
|
||||
"lines": 98,
|
||||
"branches": 89,
|
||||
|
||||
+1
-1
@@ -6,7 +6,7 @@ if (host.href !== 'http://127.0.0.1:8196/') throw new Error('Execution tests req
|
||||
|
||||
export default defineConfig({
|
||||
testDir: '../../tests/comfyui',
|
||||
testMatch: ['execution-history.spec.mjs', 'execution-outputs.spec.mjs', 'library-location.spec.mjs'],
|
||||
testMatch: ['execution-history.spec.mjs', 'execution-outputs.spec.mjs', 'execution-text-link.spec.mjs', 'library-location.spec.mjs', 'image-attribution.spec.mjs'],
|
||||
outputDir: './test-results/executions/cases',
|
||||
timeout: 60000,
|
||||
workers: 1,
|
||||
|
||||
@@ -762,9 +762,15 @@ def _widget_names(node):
|
||||
and _truthy(_get(item, "widget"))
|
||||
and _truthy(_get(_get(item, "widget"), "name"))
|
||||
]
|
||||
if from_inputs:
|
||||
return from_inputs
|
||||
node_type = _get(node, "type")
|
||||
if from_inputs:
|
||||
# A known node's widget-bound inputs must line up with its widget order.
|
||||
# Sockets that are not widget inputs (#343: the editors' text) would shift
|
||||
# every positional value onto the wrong name; use the table then.
|
||||
known = WIDGET_POSITIONS.get(node_type) if isinstance(node_type, str) else None
|
||||
if known is not None and list(known[: len(from_inputs)]) != from_inputs:
|
||||
return list(known)
|
||||
return from_inputs
|
||||
key = _js_string(node_type)
|
||||
if key in _OBJECT_PROTOTYPE_KEYS:
|
||||
raise GenerationParamsError(f"widget positions of {key!r} are not iterable")
|
||||
|
||||
+254
-82
@@ -18,8 +18,9 @@ Typical usage:
|
||||
The system automatically:
|
||||
- Detects new image files in monitored directories
|
||||
- Extracts ComfyUI workflow metadata from PNG chunks
|
||||
- Links images to active prompts using the prompt tracker
|
||||
- Handles fallback linking when no active prompt is available
|
||||
- Links images to tracked prompts only when the image's own graph proves the
|
||||
tracked entry belongs to its run (``utils.run_keys``)
|
||||
- Leaves an image unlinked when nothing certain identifies its prompt (#342)
|
||||
- Provides status information and monitoring control
|
||||
"""
|
||||
|
||||
@@ -27,7 +28,8 @@ import os
|
||||
import queue
|
||||
import time
|
||||
import threading
|
||||
from typing import Callable, Optional, Dict, Any, Tuple
|
||||
from collections import OrderedDict
|
||||
from typing import Callable, Optional, Dict, Any, List, Tuple
|
||||
from watchdog.observers import Observer
|
||||
from watchdog.events import FileSystemEventHandler
|
||||
|
||||
@@ -36,18 +38,38 @@ from .metadata_extractor import ComfyUIMetadataExtractor
|
||||
from .logging_config import get_logger
|
||||
from .prompt_graph import resolve_text, run_prompt_nodes
|
||||
from .prompt_pair_resolver import resolve_prompt_pair
|
||||
from .run_keys import image_run_keys
|
||||
from .usage_tracking import survive_switch
|
||||
|
||||
try:
|
||||
from ..database.pair_ingestion import ingest_resolved_pair
|
||||
from ..database.run_attribution import (
|
||||
host_output_directory,
|
||||
in_output_folder,
|
||||
stored_run_prompt,
|
||||
subscribe_outputs_sealed,
|
||||
)
|
||||
except ImportError:
|
||||
from database.pair_ingestion import ingest_resolved_pair
|
||||
from database.run_attribution import (
|
||||
host_output_directory,
|
||||
in_output_folder,
|
||||
stored_run_prompt,
|
||||
subscribe_outputs_sealed,
|
||||
)
|
||||
|
||||
# Upper bound on images waiting to be processed (a burst beyond this is skipped)
|
||||
MAX_PENDING_IMAGES = 1000
|
||||
# A file counts as fully written once its size is unchanged across two reads
|
||||
SETTLE_INTERVAL_SECONDS = 0.1
|
||||
SETTLE_ATTEMPTS = 20
|
||||
# Keyless images (no embedded API graph) left unlinked wait for their run's
|
||||
# recorded outputs for one history-only re-check (#342): at most this many,
|
||||
# each for at most this long.
|
||||
MAX_AWAITING_HISTORY = 1000
|
||||
AWAIT_HISTORY_SECONDS = 3600
|
||||
# ``schedule`` snapshot meaning "history-only re-check" (not a new image)
|
||||
HISTORY_ONLY = object()
|
||||
|
||||
|
||||
class ImageGenerationHandler(FileSystemEventHandler):
|
||||
@@ -109,6 +131,14 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
self._work: "queue.Queue[Optional[str]]" = queue.Queue()
|
||||
self._stop = threading.Event()
|
||||
self._worker: Optional[threading.Thread] = None
|
||||
# Keyless unlinked images awaiting their run's recorded outputs:
|
||||
# normalized path -> (path, monotonic deadline), oldest first.
|
||||
self._awaiting: "OrderedDict[str, Tuple[str, float]]" = OrderedDict()
|
||||
# Sealed re-checks ``schedule`` refused (the path still pending, or the
|
||||
# pending set full): retried each time a pending image finishes.
|
||||
self._history_due: "OrderedDict[str, None]" = OrderedDict()
|
||||
self._awaiting_lock = threading.Lock()
|
||||
self._sealed_subscribed = False
|
||||
|
||||
def on_created(self, event):
|
||||
"""Handle filesystem creation events.
|
||||
@@ -126,15 +156,17 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
if event.is_directory or not self.is_image_file(event.src_path):
|
||||
return
|
||||
self.logger.info(f"New image detected: {event.src_path}")
|
||||
# Snapshot prompt context NOW before the delay — in batch workflows
|
||||
# the tracker advances to the next prompt before images are processed.
|
||||
prompt_snapshot = self.prompt_tracker.get_current_prompt()
|
||||
if prompt_snapshot:
|
||||
self.logger.debug(
|
||||
f"Snapshot prompt {prompt_snapshot.get('id', '?')} for "
|
||||
f"{os.path.basename(event.src_path)}"
|
||||
)
|
||||
self.schedule(event.src_path, prompt_snapshot)
|
||||
# Snapshot the tracked runs NOW, before the delay: a later run may
|
||||
# execute the same node again before this image is processed.
|
||||
self.schedule(event.src_path, self._keyed_snapshot())
|
||||
|
||||
def _keyed_snapshot(self):
|
||||
"""The tracker's latest entry per run key, or None when unavailable."""
|
||||
try:
|
||||
return self.prompt_tracker.keyed_snapshot()
|
||||
except Exception as error:
|
||||
self.logger.warning(f"Could not snapshot tracked prompts: {error}")
|
||||
return None
|
||||
|
||||
def schedule(self, image_path: str, prompt_snapshot=None) -> bool:
|
||||
"""Queue an image for processing on the worker thread, once per path.
|
||||
@@ -174,6 +206,9 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
and exits; the next ``schedule`` starts a fresh worker with its own
|
||||
queue. Safe to call repeatedly and before anything was scheduled.
|
||||
"""
|
||||
with self._awaiting_lock:
|
||||
self._awaiting.clear()
|
||||
self._history_due.clear()
|
||||
with self._pending_lock:
|
||||
self._pending.clear()
|
||||
worker, self._worker = self._worker, None
|
||||
@@ -229,6 +264,7 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
finally:
|
||||
with self._pending_lock:
|
||||
self._pending.pop(image_path, None)
|
||||
self._drain_history_due()
|
||||
|
||||
def _queue_thumbnail(self, image_path: str) -> None:
|
||||
"""Queue a settled output's preview, best effort (linked or not).
|
||||
@@ -290,15 +326,20 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
This method handles the complete processing pipeline for a new image:
|
||||
1. Verifies the file still exists
|
||||
2. Extracts ComfyUI metadata from the image
|
||||
3. Tries to identify the correct prompt via metadata-based lookup
|
||||
4. Falls back to the prompt snapshot captured at event time
|
||||
5. Links the image to the appropriate prompt in the database
|
||||
3. Identifies the prompt from the image's metadata, else from a tracked
|
||||
run the image's graph proves (``_find_prompt_from_run``)
|
||||
4. Links the image to that prompt, or leaves it unlinked when nothing
|
||||
certain identifies it (logged at info; #342)
|
||||
|
||||
Args:
|
||||
image_path: Full path to the newly created image file
|
||||
prompt_snapshot: Prompt context captured at file-creation time (optional)
|
||||
prompt_snapshot: The tracker's keyed snapshot taken when the image
|
||||
appeared (``PromptTracker.keyed_snapshot``); None reads it now
|
||||
"""
|
||||
try:
|
||||
if prompt_snapshot is HISTORY_ONLY:
|
||||
self._recheck_history(image_path)
|
||||
return
|
||||
self.logger.info(f"Processing image: {image_path}")
|
||||
|
||||
if not os.path.exists(image_path):
|
||||
@@ -329,43 +370,17 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
f"Metadata match: prompt {current_prompt['id']} for "
|
||||
f"{os.path.basename(image_path)}"
|
||||
)
|
||||
|
||||
# Strategy 2: Pop from batch queue. Batch items (text supplied by
|
||||
# PromptSearchList) share one graph, so metadata can't tell them apart;
|
||||
# they are queued in encoding order and images save in the same order.
|
||||
if not current_prompt:
|
||||
current_prompt = self.prompt_tracker.pop_next_prompt()
|
||||
if current_prompt:
|
||||
self.logger.info(
|
||||
f"Queue match: prompt {current_prompt['id']} for "
|
||||
f"{os.path.basename(image_path)}"
|
||||
)
|
||||
|
||||
# Strategy 3: Use snapshot captured at file-creation time
|
||||
if not current_prompt and prompt_snapshot:
|
||||
current_prompt = prompt_snapshot
|
||||
self.logger.info(
|
||||
f"Snapshot match: prompt {current_prompt.get('id', 'unknown')}"
|
||||
else:
|
||||
current_prompt = self._find_prompt_from_run(
|
||||
image_path, metadata, prompt_snapshot
|
||||
)
|
||||
|
||||
# Strategy 4: Check live prompt tracker
|
||||
if not current_prompt:
|
||||
current_prompt = self.prompt_tracker.get_current_prompt()
|
||||
if current_prompt:
|
||||
self.logger.info(
|
||||
f"Live tracker match: prompt {current_prompt['id']}"
|
||||
)
|
||||
|
||||
# Strategy 5: Fallback to most recent prompt in DB
|
||||
if not current_prompt:
|
||||
current_prompt = self._get_fallback_prompt()
|
||||
if current_prompt:
|
||||
self.logger.debug(f"Fallback match: prompt {current_prompt['id']}")
|
||||
else:
|
||||
self.logger.warning(
|
||||
f"No prompt context available, skipping image: {image_path}"
|
||||
)
|
||||
return
|
||||
# Nothing certain: never guess from recency or a stale entry (#342)
|
||||
self.logger.info(
|
||||
"No certain prompt for "
|
||||
f"{os.path.basename(image_path)}; left unlinked"
|
||||
)
|
||||
return
|
||||
|
||||
# Extend timeout if we have an active execution
|
||||
if current_prompt.get("execution_id"):
|
||||
@@ -393,6 +408,193 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
|
||||
self.logger.error(traceback.format_exc())
|
||||
|
||||
def _find_prompt_from_run(
|
||||
self, image_path: str, metadata, prompt_snapshot=None
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""The prompt a tracked run proves for an image whose metadata can't say.
|
||||
|
||||
Only an entry registered under the run key of the image's own positive
|
||||
node counts (``utils.run_keys``): never the newest prompt in the
|
||||
database and never an entry of another graph (#342).
|
||||
|
||||
The snapshot taken when the image appeared names the key's latest run.
|
||||
|
||||
1. Batch queue: the oldest queued entry of that key from that run.
|
||||
Batch items (text from PromptSearchList) share one key and are
|
||||
linked in order; an empty item's placeholder leaves its image
|
||||
unlinked; entries queued by later runs stay for their own images.
|
||||
2. The snapshot's entry itself, when its run is known and registered
|
||||
exactly one entry for the key (a batch's latest item says nothing
|
||||
about the other images). This covers a cached node that did not
|
||||
run again.
|
||||
3. Recorded execution history: the run that wrote this file names one
|
||||
stored prompt (``database.run_attribution``).
|
||||
"""
|
||||
keys = image_run_keys(metadata)
|
||||
name = os.path.basename(image_path)
|
||||
if not keys:
|
||||
# No tracked run can be proven: history now, and once more after
|
||||
# the run's outputs are recorded (``_on_outputs_sealed``). A file
|
||||
# outside ComfyUI's output folder can never be attributed, so it
|
||||
# does not take a waiting slot.
|
||||
if not in_output_folder(image_path, host_output_directory()):
|
||||
return None
|
||||
self._await_history(image_path)
|
||||
prompt_id = self._history_prompt(image_path)
|
||||
if prompt_id is None:
|
||||
return None
|
||||
self._stop_awaiting(image_path)
|
||||
self.logger.info(f"History match: prompt {prompt_id} for {name}")
|
||||
return {"id": prompt_id, "from_history": True}
|
||||
if len(keys) == 1:
|
||||
(key,) = keys.values()
|
||||
if prompt_snapshot is None:
|
||||
prompt_snapshot = self._keyed_snapshot()
|
||||
# No entry of the key when the image appeared: no tracked run made it
|
||||
entry = (prompt_snapshot or {}).get(key)
|
||||
if entry is not None:
|
||||
context, run_id, count = entry
|
||||
queued = self.prompt_tracker.pop_for_key(key, run_id)
|
||||
if queued is not None:
|
||||
if queued.get("placeholder"):
|
||||
self.logger.info(f"Queue entry for {name} has no prompt text")
|
||||
return None
|
||||
self.logger.info(f"Queue match: prompt {queued['id']} for {name}")
|
||||
return queued
|
||||
if context.get("placeholder"):
|
||||
# Same rule as a popped placeholder: that run had no prompt
|
||||
self.logger.info(f"Latest run for {name} had no prompt text")
|
||||
return None
|
||||
if run_id and count == 1:
|
||||
self.logger.info(f"Run match: prompt {context['id']} for {name}")
|
||||
return context
|
||||
else:
|
||||
self.logger.debug(f"{len(keys)} positive prompt nodes for {name}")
|
||||
prompt_id = self._history_prompt(image_path)
|
||||
if prompt_id is None:
|
||||
return None
|
||||
self.logger.info(f"History match: prompt {prompt_id} for {name}")
|
||||
return {"id": prompt_id, "from_history": True}
|
||||
|
||||
@staticmethod
|
||||
def _awaiting_key(image_path: str) -> str:
|
||||
"""One spelling for a waiting file, through symlinks and junctions."""
|
||||
try:
|
||||
resolved = os.path.realpath(image_path)
|
||||
except (OSError, ValueError):
|
||||
resolved = os.path.abspath(image_path)
|
||||
return os.path.normcase(resolved)
|
||||
|
||||
def _prune_awaiting(self) -> None:
|
||||
"""Drop expired and surplus entries (``_awaiting_lock`` held)."""
|
||||
now = time.monotonic()
|
||||
for key in [k for k, (_p, due) in self._awaiting.items() if due <= now]:
|
||||
del self._awaiting[key]
|
||||
while len(self._awaiting) > MAX_AWAITING_HISTORY:
|
||||
self._awaiting.popitem(last=False)
|
||||
|
||||
def _await_history(self, image_path: str) -> None:
|
||||
"""Re-check ``image_path`` once, history only, when its run is sealed."""
|
||||
if not self._sealed_subscribed:
|
||||
subscribe_outputs_sealed(self._on_outputs_sealed)
|
||||
self._sealed_subscribed = True
|
||||
key = self._awaiting_key(image_path)
|
||||
with self._awaiting_lock:
|
||||
self._awaiting.pop(key, None)
|
||||
self._awaiting[key] = (
|
||||
image_path,
|
||||
time.monotonic() + AWAIT_HISTORY_SECONDS,
|
||||
)
|
||||
self._prune_awaiting()
|
||||
|
||||
def _stop_awaiting(self, image_path: str) -> None:
|
||||
with self._awaiting_lock:
|
||||
self._awaiting.pop(self._awaiting_key(image_path), None)
|
||||
|
||||
def awaiting_history(self) -> List[str]:
|
||||
"""Normalized paths still waiting for their run's recorded outputs."""
|
||||
with self._awaiting_lock:
|
||||
self._prune_awaiting()
|
||||
return list(self._awaiting)
|
||||
|
||||
def _on_outputs_sealed(self, run_id: str, references: List[dict]) -> None:
|
||||
"""A run's outputs were recorded: re-check its waiting images once.
|
||||
|
||||
Called on the history writer's thread, so it only schedules the work.
|
||||
"""
|
||||
root = host_output_directory()
|
||||
if not root:
|
||||
return
|
||||
due = []
|
||||
with self._awaiting_lock:
|
||||
self._prune_awaiting()
|
||||
for reference in references:
|
||||
if reference.get("folder_type") != "output":
|
||||
continue
|
||||
key = self._awaiting_key(
|
||||
os.path.join(
|
||||
root, reference.get("subfolder", ""), reference["filename"]
|
||||
)
|
||||
)
|
||||
waiting = self._awaiting.pop(key, None)
|
||||
if waiting is not None:
|
||||
due.append(waiting[0])
|
||||
for image_path in due:
|
||||
self._schedule_recheck(image_path)
|
||||
|
||||
def _schedule_recheck(self, image_path: str) -> None:
|
||||
"""Schedule the history-only re-check, or keep it due for later."""
|
||||
if self.schedule(image_path, HISTORY_ONLY):
|
||||
return
|
||||
with self._awaiting_lock:
|
||||
self._history_due.pop(image_path, None)
|
||||
self._history_due[image_path] = None
|
||||
while len(self._history_due) > MAX_AWAITING_HISTORY:
|
||||
self._history_due.popitem(last=False)
|
||||
|
||||
def _drain_history_due(self) -> None:
|
||||
"""Schedule due re-checks now that a pending slot or path is free."""
|
||||
with self._awaiting_lock:
|
||||
if not self._history_due:
|
||||
return
|
||||
due = list(self._history_due)
|
||||
for image_path in due:
|
||||
if self.schedule(image_path, HISTORY_ONLY):
|
||||
with self._awaiting_lock:
|
||||
self._history_due.pop(image_path, None)
|
||||
|
||||
def _recheck_history(self, image_path: str) -> None:
|
||||
"""The one history-only re-check of a keyless image (#342)."""
|
||||
name = os.path.basename(image_path)
|
||||
if not os.path.exists(image_path):
|
||||
return
|
||||
prompt_id = self._history_prompt(image_path)
|
||||
if prompt_id is None:
|
||||
self.logger.info(
|
||||
f"No certain prompt for {name} after its run ended; left unlinked"
|
||||
)
|
||||
return
|
||||
self.logger.info(f"History match after its run ended: {prompt_id} for {name}")
|
||||
try:
|
||||
metadata = self.metadata_extractor.extract_metadata(image_path)
|
||||
except Exception:
|
||||
metadata = None
|
||||
self.link_image_to_prompt(
|
||||
image_path,
|
||||
{"id": prompt_id, "from_history": True},
|
||||
metadata or {"file_info": self.get_basic_file_info(image_path)},
|
||||
)
|
||||
|
||||
def _history_prompt(self, image_path: str) -> Optional[int]:
|
||||
"""The stored prompt recorded history attributes to this file, if one."""
|
||||
try:
|
||||
return stored_run_prompt(
|
||||
self.db_manager, image_path, host_output_directory()
|
||||
)
|
||||
except Exception as error:
|
||||
self.logger.warning(f"History attribution failed: {error}")
|
||||
return None
|
||||
|
||||
def _find_prompt_from_metadata(self, metadata) -> Tuple[str, Optional[Dict]]:
|
||||
"""Identify the prompt that produced an image from its embedded metadata.
|
||||
|
||||
@@ -542,35 +744,6 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
self.logger.error(f"Error getting file info: {e}")
|
||||
return {}
|
||||
|
||||
def _get_fallback_prompt(self) -> Optional[Dict[str, Any]]:
|
||||
"""Get the most recent prompt from database as fallback.
|
||||
|
||||
When no active prompt is available from the tracker, this method attempts
|
||||
to find the most recently created prompt in the database to use as a fallback
|
||||
for image linking.
|
||||
|
||||
Returns:
|
||||
Dictionary containing prompt information with 'fallback' flag set to True,
|
||||
or None if no recent prompt is available
|
||||
"""
|
||||
try:
|
||||
recent = self.db_manager.get_recent_prompts(limit=1)
|
||||
# Paginated dict ({"prompts": [...]}) or a plain list
|
||||
recent_prompts = (
|
||||
recent.get("prompts") if isinstance(recent, dict) else recent
|
||||
)
|
||||
if recent_prompts:
|
||||
prompt = recent_prompts[0]
|
||||
return {
|
||||
"id": prompt["id"],
|
||||
"text": prompt["text"],
|
||||
"timestamp": prompt.get("created_at"),
|
||||
"fallback": True,
|
||||
}
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error getting fallback prompt: {e}")
|
||||
return None
|
||||
|
||||
def link_image_to_prompt(
|
||||
self, image_path: str, prompt_context: Dict, metadata: Dict
|
||||
):
|
||||
@@ -615,10 +788,9 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
if image_id is None:
|
||||
return
|
||||
if image_id: # 0: nothing new was linked (logged by the database)
|
||||
fallback_note = " (fallback)" if prompt_context.get("fallback") else ""
|
||||
self.logger.debug(
|
||||
f"Successfully linked image {image_id} to prompt "
|
||||
f"{prompt_context['id']}{fallback_note}"
|
||||
f"{prompt_context['id']}"
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Failed to link image to prompt: {e}")
|
||||
|
||||
+151
-9
@@ -28,7 +28,8 @@ Or using the context manager:
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from typing import Optional, Dict, Any
|
||||
from collections import OrderedDict
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
|
||||
# Import logging system
|
||||
try:
|
||||
@@ -103,6 +104,8 @@ class PromptTracker:
|
||||
prompt_text: str,
|
||||
additional_data: Optional[Dict[str, Any]] = None,
|
||||
push_to_queue: bool = True,
|
||||
run_key: Optional[str] = None,
|
||||
run_id: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Set the current prompt for this thread for image tracking.
|
||||
@@ -120,6 +123,10 @@ class PromptTracker:
|
||||
push_to_queue: Also push onto the FIFO batch queue. Only batch items whose
|
||||
text comes from another node need it; typed prompts are linked from
|
||||
the image's own metadata instead.
|
||||
run_key: The node's run key (``utils.run_keys.prompt_node_key``). The
|
||||
image monitor links an image to a tracked prompt only when the
|
||||
image's own graph gives the same key (#342).
|
||||
run_id: The ComfyUI prompt id executing the node, when the host says.
|
||||
|
||||
Returns:
|
||||
Unique execution ID for this prompt execution
|
||||
@@ -157,6 +164,8 @@ class PromptTracker:
|
||||
"timestamp": time.time(),
|
||||
"thread_id": threading.current_thread().ident,
|
||||
"additional_data": additional_data or {},
|
||||
"run_key": run_key,
|
||||
"run_id": run_id,
|
||||
}
|
||||
|
||||
# Store in thread-local storage
|
||||
@@ -166,20 +175,20 @@ class PromptTracker:
|
||||
with self.lock:
|
||||
self.active_prompts[execution_id] = execution_context
|
||||
|
||||
# Push to batch queue for ordered image linking
|
||||
if push_to_queue:
|
||||
with self._queue_lock:
|
||||
self._prompt_queue.append(execution_context)
|
||||
self.logger.debug(
|
||||
f"Queue size: {len(self._prompt_queue)} after push "
|
||||
f"for prompt {prompt_id}"
|
||||
)
|
||||
self._remember_keyed(execution_context)
|
||||
|
||||
self.logger.debug(
|
||||
f"Set current prompt: {execution_id} -> {prompt_text[:50]}... "
|
||||
f"(thread: {threading.current_thread().ident})"
|
||||
)
|
||||
self.logger.debug(f"Active prompts count: {len(self.active_prompts)}")
|
||||
|
||||
# Push to batch queue for ordered image linking. Last, with nothing
|
||||
# after it that can raise: a caller that marks a failed execution
|
||||
# (#342) must never add a second queue slot for the same item.
|
||||
if push_to_queue:
|
||||
with self._queue_lock:
|
||||
self._prompt_queue.append(execution_context)
|
||||
return execution_id
|
||||
|
||||
def get_current_prompt(self) -> Optional[Dict[str, Any]]:
|
||||
@@ -225,6 +234,99 @@ class PromptTracker:
|
||||
)
|
||||
return None
|
||||
|
||||
def mark_empty(
|
||||
self,
|
||||
run_key: Optional[str],
|
||||
run_id: Optional[str] = None,
|
||||
push_to_queue: bool = False,
|
||||
) -> None:
|
||||
"""Record that a positive node ran with nothing to save (#342).
|
||||
|
||||
The run key's latest entry becomes a placeholder, so no earlier prompt of
|
||||
that key can be linked to this run's images; with ``push_to_queue`` the
|
||||
placeholder also holds this item's place in the batch queue, so a later
|
||||
item's image never takes the wrong queue entry.
|
||||
"""
|
||||
if not run_key:
|
||||
return
|
||||
placeholder = {
|
||||
"id": None,
|
||||
"placeholder": True,
|
||||
"execution_id": self.generate_execution_id(),
|
||||
"text": "",
|
||||
"timestamp": time.time(),
|
||||
"additional_data": {},
|
||||
"run_key": run_key,
|
||||
"run_id": run_id,
|
||||
}
|
||||
self._remember_keyed(placeholder)
|
||||
if push_to_queue:
|
||||
with self._queue_lock:
|
||||
self._prompt_queue.append(placeholder)
|
||||
|
||||
def _keyed_runs(self) -> "KeyedRuns":
|
||||
keyed = getattr(self, "_keyed", None)
|
||||
if keyed is None:
|
||||
with _KEYED_INIT_LOCK:
|
||||
keyed = getattr(self, "_keyed", None)
|
||||
if keyed is None:
|
||||
keyed = self._keyed = KeyedRuns()
|
||||
return keyed
|
||||
|
||||
def _remember_keyed(self, context: Dict[str, Any]) -> None:
|
||||
if context.get("run_key"):
|
||||
self._keyed_runs().record(context)
|
||||
|
||||
def keyed_snapshot(self) -> Dict[str, Tuple[Dict[str, Any], Optional[str], int]]:
|
||||
"""``{run key: (latest entry, its run id, entries of that run)}`` now.
|
||||
|
||||
The image monitor captures it when an image appears, before later runs
|
||||
can move on.
|
||||
"""
|
||||
return self._keyed_runs().snapshot(self.prompt_timeout)
|
||||
|
||||
def pop_for_key(
|
||||
self, run_key: Optional[str], run_id: Optional[str] = None
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""Pop the oldest queued entry of ``run_key`` (FIFO), from ``run_id`` if set.
|
||||
|
||||
``run_id`` is the run the key's latest entry came from when the image
|
||||
appeared (``keyed_snapshot``): entries of that key queued by earlier
|
||||
runs ahead of it are stale and dropped, while entries of later runs
|
||||
(queued after the image appeared) stay for their own images. Entries
|
||||
of other keys stay queued; expired entries are discarded.
|
||||
"""
|
||||
if not run_key:
|
||||
return None
|
||||
now = time.time()
|
||||
with self._queue_lock:
|
||||
live = [
|
||||
ctx
|
||||
for ctx in self._prompt_queue
|
||||
if now - ctx.get("timestamp", 0) < self.prompt_timeout
|
||||
]
|
||||
index = next(
|
||||
(
|
||||
i
|
||||
for i, ctx in enumerate(live)
|
||||
if ctx.get("run_key") == run_key
|
||||
and (run_id is None or ctx.get("run_id") == run_id)
|
||||
),
|
||||
None,
|
||||
)
|
||||
if index is None:
|
||||
self._prompt_queue = live
|
||||
return None
|
||||
found = live[index]
|
||||
# Same-key entries ahead of it came from earlier runs: stale
|
||||
stale_or_found = {
|
||||
i for i in range(index + 1) if live[i].get("run_key") == run_key
|
||||
}
|
||||
self._prompt_queue = [
|
||||
ctx for i, ctx in enumerate(live) if i not in stale_or_found
|
||||
]
|
||||
return found
|
||||
|
||||
def pop_next_prompt(self) -> Optional[Dict[str, Any]]:
|
||||
"""Pop the next prompt from the batch queue (FIFO).
|
||||
|
||||
@@ -463,6 +565,46 @@ class PromptTracker:
|
||||
}
|
||||
|
||||
|
||||
_KEYED_INIT_LOCK = threading.Lock()
|
||||
# Run keys remembered for the image monitor (oldest forgotten first)
|
||||
MAX_KEYED_RUNS = 256
|
||||
|
||||
|
||||
class KeyedRuns:
|
||||
"""The latest tracked entry per run key, and how many its run produced.
|
||||
|
||||
More than one entry for a key in one run (a batch fed by a list) means the
|
||||
latest one alone cannot say which item an image belongs to.
|
||||
"""
|
||||
|
||||
def __init__(self, max_keys: int = MAX_KEYED_RUNS):
|
||||
self._max = max_keys
|
||||
self._latest: "OrderedDict[str, Tuple[Dict[str, Any], Optional[str], int]]"
|
||||
self._latest = OrderedDict()
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def record(self, context: Dict[str, Any]) -> None:
|
||||
key, run_id = context["run_key"], context.get("run_id")
|
||||
with self._lock:
|
||||
previous = self._latest.pop(key, None)
|
||||
same_run = (
|
||||
previous is not None and run_id is not None and previous[1] == run_id
|
||||
)
|
||||
count = previous[2] + 1 if same_run else 1
|
||||
self._latest[key] = (context, run_id, count)
|
||||
while len(self._latest) > self._max:
|
||||
self._latest.popitem(last=False)
|
||||
|
||||
def snapshot(self, timeout: float) -> Dict[str, Tuple[Dict[str, Any], Any, int]]:
|
||||
now = time.time()
|
||||
with self._lock:
|
||||
return {
|
||||
key: entry
|
||||
for key, entry in self._latest.items()
|
||||
if now - entry[0].get("timestamp", 0) < timeout
|
||||
}
|
||||
|
||||
|
||||
class PromptExecutionContext:
|
||||
"""Context manager for prompt executions.
|
||||
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
"""Prove that a tracked prompt belongs to the run that saved an image (#342).
|
||||
|
||||
A PromptManager node registers the text it executed under a *run key*: a digest
|
||||
of the node and everything upstream of it in the queued API graph. The image
|
||||
embeds the same graph, so the image monitor can compute the same key for the
|
||||
image's positive prompt node. Equal keys mean the node ran with exactly the
|
||||
inputs (and upstream nodes) that produced this image.
|
||||
|
||||
Equal keys do not always mean equal text. ComfyUI's cache signature also folds
|
||||
in each node's ``IS_CHANGED``, which the key cannot see: an upstream node that
|
||||
forces a re-run (a random line, an "always run" LLM) can give two runs one key
|
||||
and two texts. The key is therefore never used alone: the monitor also requires
|
||||
the entry to come from the run the snapshot taken when the image appeared names
|
||||
(``PromptTracker.pop_for_key``, ``ImageGenerationHandler._find_prompt_from_run``).
|
||||
|
||||
Pure functions; the only host dependency (the executing run id) is optional.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
try:
|
||||
from .prompt_graph import MAX_VISITS, run_prompt_nodes
|
||||
except ImportError:
|
||||
from utils.prompt_graph import MAX_VISITS, run_prompt_nodes
|
||||
|
||||
RUN_KEY_VERSION = 1
|
||||
|
||||
|
||||
def is_link(value: Any) -> bool:
|
||||
"""ComfyUI's own test (``comfy_execution.graph_utils.is_link``), exactly.
|
||||
|
||||
A two-item list of a ``str`` node id and an ``int`` or ``float`` slot (a
|
||||
``bool`` slot passes too, as it does in ComfyUI). Tuples are values.
|
||||
"""
|
||||
if not isinstance(value, list):
|
||||
return False
|
||||
if len(value) != 2:
|
||||
return False
|
||||
if not isinstance(value[0], str):
|
||||
return False
|
||||
if not isinstance(value[1], int) and not isinstance(value[1], float):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _links(inputs: Any) -> List[str]:
|
||||
if not isinstance(inputs, dict):
|
||||
return []
|
||||
return [value[0] for value in inputs.values() if is_link(value)]
|
||||
|
||||
|
||||
def prompt_node_key(graph: Any, node_id: Any) -> Optional[str]:
|
||||
"""Digest of ``node_id`` and its whole upstream closure, or None.
|
||||
|
||||
Only ``class_type`` and ``inputs`` count (titles and other UI fields do not
|
||||
change what runs). None when the graph or node is missing, an upstream link
|
||||
points at a missing node, the closure exceeds ``MAX_VISITS`` nodes, or a
|
||||
value cannot be serialized: no key means no proof.
|
||||
"""
|
||||
if not isinstance(graph, dict) or node_id is None:
|
||||
return None
|
||||
start = str(node_id)
|
||||
closure: Dict[str, Any] = {}
|
||||
stack = [start]
|
||||
while stack:
|
||||
current = stack.pop()
|
||||
if current in closure:
|
||||
continue
|
||||
node = graph.get(current)
|
||||
if not isinstance(node, dict):
|
||||
return None
|
||||
if len(closure) >= MAX_VISITS:
|
||||
return None
|
||||
inputs = node.get("inputs")
|
||||
closure[current] = [node.get("class_type"), inputs]
|
||||
stack.extend(_links(inputs))
|
||||
try:
|
||||
encoded = json.dumps(
|
||||
[RUN_KEY_VERSION, start, closure],
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=True,
|
||||
)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return hashlib.sha256(encoded.encode("ascii")).hexdigest()
|
||||
|
||||
|
||||
def image_run_keys(metadata: Any) -> Dict[str, str]:
|
||||
"""``{node_id: run key}`` for each positive PromptManager node of an image.
|
||||
|
||||
Empty without an embedded API graph, and empty when any positive node has
|
||||
no key: one unkeyable node must not leave another looking like the only
|
||||
positive prompt (a two-pass workflow would link the wrong pass).
|
||||
"""
|
||||
graph = metadata.get("prompt") if isinstance(metadata, dict) else None
|
||||
if isinstance(graph, str):
|
||||
try:
|
||||
graph = json.loads(graph)
|
||||
except (TypeError, ValueError, RecursionError):
|
||||
return {}
|
||||
if not isinstance(graph, dict):
|
||||
return {}
|
||||
keys = {}
|
||||
for node_id in run_prompt_nodes(graph):
|
||||
key = prompt_node_key(graph, node_id)
|
||||
if key is None:
|
||||
return {}
|
||||
keys[node_id] = key
|
||||
return keys
|
||||
|
||||
|
||||
def executing_run_id() -> Optional[str]:
|
||||
"""The ComfyUI prompt id currently executing on this thread, if the host says.
|
||||
|
||||
None outside ComfyUI or on hosts without ``get_executing_context``.
|
||||
"""
|
||||
try:
|
||||
from comfy_execution.utils import get_executing_context
|
||||
except Exception:
|
||||
return None
|
||||
try:
|
||||
context = get_executing_context()
|
||||
except Exception:
|
||||
return None
|
||||
run_id = getattr(context, "prompt_id", None)
|
||||
return run_id if isinstance(run_id, str) and run_id else None
|
||||
+130
-56
@@ -5,6 +5,7 @@ import { api } from "../../scripts/api.js";
|
||||
const { installClassicControls } = await import(api.fileURL('/prompt_manager/js/classic-node.js'));
|
||||
const { installFocusedNode, installSplitNode, installPairNode } = await import(api.fileURL('/prompt_manager/js/focused-node.js'));
|
||||
const { installPromptBridge } = await import(api.fileURL('/prompt_manager/js/prompt-bridge-host.js'));
|
||||
const { collapseLinkedWidget, isLinked, linkNoteText, primitiveTextValue, storedWidgetValue } = await import(api.fileURL('/prompt_manager/js/text-links.js'));
|
||||
const promptManagerTypes = new Set(['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair']);
|
||||
|
||||
// ============================================================================
|
||||
@@ -208,15 +209,10 @@ function syncPromptManagerWidgets() {
|
||||
|
||||
node.widgets.forEach((widget, idx) => {
|
||||
if (widget) {
|
||||
// Skip widgets whose input is connected to another node —
|
||||
// the connected value takes precedence over the DOM widget value
|
||||
const isConnected = node.inputs?.some(
|
||||
i => i.name === widget.name && i.link != null
|
||||
);
|
||||
if (isConnected) return;
|
||||
|
||||
// A linked widget keeps the text the user typed (#343): the link drives the run,
|
||||
// the workflow keeps the typed text for when the link is disconnected.
|
||||
// For non-connected widgets, inputEl.value is the true source
|
||||
const actualValue = widget.inputEl ? widget.inputEl.value : widget.value;
|
||||
const actualValue = storedWidgetValue(node, widget);
|
||||
if (actualValue !== undefined) {
|
||||
node.widgets_values[idx] = actualValue;
|
||||
|
||||
@@ -232,6 +228,45 @@ function syncPromptManagerWidgets() {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* #343: a legacy frontend Primitive linked to a PromptManager editor text input runs with the Primitive's value.
|
||||
* The Primitive writes into the target's widget, but the editor text inputs are no longer bound to their hidden
|
||||
* widgets, so the queued value would otherwise be the hidden typed text while the editor says "linked". Inputs still
|
||||
* bound to a widget (the Classic nodes) are left as the host resolved them.
|
||||
*
|
||||
* A wildcard (`{red|blue}`) draws a new choice each time it is resolved, so each queued output is filled once: by the
|
||||
* graphToPrompt wrapper, or by the queue hook when a caller queues an output that did not come through it.
|
||||
*/
|
||||
const primitiveTextApplied = new WeakSet();
|
||||
const MAX_SUBGRAPH_DEPTH = 32;
|
||||
|
||||
/** Every node with the id it is queued under: a node inside a subgraph is `<subgraph node id>:<inner id>`. */
|
||||
function* queuedNodes(graph, prefix = '', depth = 0) {
|
||||
for (const node of graph?._nodes || graph?.nodes || []) {
|
||||
yield [`${prefix}${node.id}`, node];
|
||||
if (depth < MAX_SUBGRAPH_DEPTH && node.isSubgraphNode?.() && node.subgraph) {
|
||||
yield* queuedNodes(node.subgraph, `${prefix}${node.id}:`, depth + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function applyPrimitiveText(output) {
|
||||
if (!output || typeof output !== 'object' || primitiveTextApplied.has(output)) return;
|
||||
primitiveTextApplied.add(output);
|
||||
const replaceText = window.comfyAPI?.utils?.applyTextReplacements;
|
||||
for (const [id, node] of queuedNodes(app.graph)) {
|
||||
const queued = promptManagerTypes.has(node.type) ? output[id] : null;
|
||||
// Only an output of this graph: the same id must hold the same node type.
|
||||
const inputs = queued?.class_type === node.type ? queued.inputs : null;
|
||||
if (!inputs) continue;
|
||||
for (const name of ['text', 'negative_text']) {
|
||||
if (Array.isArray(inputs[name])) continue;
|
||||
const value = await primitiveTextValue(node, name, { replaceText });
|
||||
if (value !== undefined) inputs[name] = value;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "PromptManager.UI",
|
||||
|
||||
@@ -247,7 +282,9 @@ app.registerExtension({
|
||||
const originalGraphToPrompt = app.graphToPrompt.bind(app);
|
||||
app.graphToPrompt = async function(...args) {
|
||||
syncPromptManagerWidgets();
|
||||
return originalGraphToPrompt(...args);
|
||||
const result = await originalGraphToPrompt(...args);
|
||||
await applyPrimitiveText(result?.output);
|
||||
return result;
|
||||
};
|
||||
|
||||
// CRITICAL FIX: Intercept api.queuePrompt to fix prompt data at the LAST moment
|
||||
@@ -274,6 +311,7 @@ app.registerExtension({
|
||||
}
|
||||
}
|
||||
|
||||
await applyPrimitiveText(output);
|
||||
return originalQueuePrompt(number, { output, workflow }, ...rest);
|
||||
};
|
||||
},
|
||||
@@ -304,53 +342,60 @@ app.registerExtension({
|
||||
// CRITICAL FIX: Ensure text widget always returns current value during serialization
|
||||
// This fixes the issue where graphToPrompt reads stale cached values
|
||||
const self = this;
|
||||
// Settles once the serializer below is in place (or skipped), for callers that must queue after it.
|
||||
let textSerializerSettled;
|
||||
this._pmTextSerializerReady = new Promise((resolve) => { textSerializerSettled = resolve; });
|
||||
this._textSyncTimer = setTimeout(() => {
|
||||
if (self._classicScope?.signal.aborted) return;
|
||||
const textWidget = self.widgets?.find((w) => w.name === "text");
|
||||
if (textWidget) {
|
||||
// Store reference to the widget for the closure
|
||||
const widgetRef = textWidget;
|
||||
const originalSerializeValue = textWidget.serializeValue;
|
||||
try {
|
||||
if (self._classicScope?.signal.aborted) return;
|
||||
const textWidget = self.widgets?.find((w) => w.name === "text");
|
||||
if (textWidget) {
|
||||
// Store reference to the widget for the closure
|
||||
const widgetRef = textWidget;
|
||||
const originalSerializeValue = textWidget.serializeValue;
|
||||
|
||||
// Override serializeValue - use arrow function to capture widgetRef
|
||||
textWidget.serializeValue = (nodeId, widgetIndex) => {
|
||||
// Always return the current DOM value using our stored reference
|
||||
if (widgetRef.inputEl) {
|
||||
const currentValue = widgetRef.inputEl.value;
|
||||
// Sync widget.value as well
|
||||
widgetRef.value = currentValue;
|
||||
return currentValue;
|
||||
}
|
||||
return widgetRef.value;
|
||||
};
|
||||
|
||||
// Add aggressive sync listeners for all value change events
|
||||
if (textWidget.inputEl) {
|
||||
const syncValue = function() {
|
||||
const currentVal = textWidget.inputEl.value;
|
||||
textWidget.value = currentVal;
|
||||
// Also update widgets_values array
|
||||
const widgetIndex = self.widgets?.indexOf(textWidget);
|
||||
if (self.widgets_values && widgetIndex >= 0) {
|
||||
self.widgets_values[widgetIndex] = currentVal;
|
||||
// Override serializeValue - use arrow function to capture widgetRef
|
||||
textWidget.serializeValue = (nodeId, widgetIndex) => {
|
||||
// Always return the current DOM value using our stored reference
|
||||
if (widgetRef.inputEl) {
|
||||
const currentValue = widgetRef.inputEl.value;
|
||||
// Sync widget.value as well
|
||||
widgetRef.value = currentValue;
|
||||
return currentValue;
|
||||
}
|
||||
return widgetRef.value;
|
||||
};
|
||||
|
||||
// Sync on every keystroke
|
||||
const listen = self._classicScope
|
||||
? (event) => self._classicScope.listen(textWidget.inputEl, event, syncValue)
|
||||
: (event) => textWidget.inputEl.addEventListener(event, syncValue);
|
||||
listen('input');
|
||||
// Sync on change (for paste, autocomplete, etc.)
|
||||
listen('change');
|
||||
// Sync when focus leaves the field
|
||||
listen('blur');
|
||||
// Sync on keyup as backup
|
||||
listen('keyup');
|
||||
// Add aggressive sync listeners for all value change events
|
||||
if (textWidget.inputEl) {
|
||||
const syncValue = function() {
|
||||
const currentVal = textWidget.inputEl.value;
|
||||
textWidget.value = currentVal;
|
||||
// Also update widgets_values array
|
||||
const widgetIndex = self.widgets?.indexOf(textWidget);
|
||||
if (self.widgets_values && widgetIndex >= 0) {
|
||||
self.widgets_values[widgetIndex] = currentVal;
|
||||
}
|
||||
};
|
||||
|
||||
// Initial sync
|
||||
syncValue();
|
||||
// Sync on every keystroke
|
||||
const listen = self._classicScope
|
||||
? (event) => self._classicScope.listen(textWidget.inputEl, event, syncValue)
|
||||
: (event) => textWidget.inputEl.addEventListener(event, syncValue);
|
||||
listen('input');
|
||||
// Sync on change (for paste, autocomplete, etc.)
|
||||
listen('change');
|
||||
// Sync when focus leaves the field
|
||||
listen('blur');
|
||||
// Sync on keyup as backup
|
||||
listen('keyup');
|
||||
|
||||
// Initial sync
|
||||
syncValue();
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
textSerializerSettled();
|
||||
}
|
||||
}, 50); // Shorter delay to beat other extensions
|
||||
|
||||
@@ -443,13 +488,8 @@ app.registerExtension({
|
||||
}
|
||||
this.widgets.forEach((widget, idx) => {
|
||||
if (widget) {
|
||||
// Skip connected widgets — their value comes from the link
|
||||
const isConnected = self.inputs?.some(
|
||||
i => i.name === widget.name && i.link != null
|
||||
);
|
||||
if (isConnected) return;
|
||||
|
||||
const actualValue = widget.inputEl ? widget.inputEl.value : widget.value;
|
||||
// A linked widget stores its typed text, not the live editor (#343)
|
||||
const actualValue = storedWidgetValue(self, widget);
|
||||
if (actualValue !== undefined) {
|
||||
this.widgets_values[idx] = actualValue;
|
||||
if (widget.inputEl && widget.value !== actualValue) {
|
||||
@@ -488,11 +528,36 @@ app.registerExtension({
|
||||
if (data._userHasResized) {
|
||||
this._userHasResized = data._userHasResized;
|
||||
}
|
||||
this.renderTextLink();
|
||||
};
|
||||
|
||||
this.setDirtyCanvas(true, true);
|
||||
};
|
||||
|
||||
// #343: text can come from a link like any core widget. The host hides the text box;
|
||||
// the controls say why, and the typed text is kept for when the link is disconnected.
|
||||
const onConnectionsChange = nodeType.prototype.onConnectionsChange;
|
||||
nodeType.prototype.onConnectionsChange = function (...args) {
|
||||
const result = onConnectionsChange?.apply(this, args);
|
||||
this.renderTextLink();
|
||||
return result;
|
||||
};
|
||||
nodeType.prototype.createTextLinkNote = function () {
|
||||
const note = document.createElement("p");
|
||||
note.className = "pm-text-link-note";
|
||||
note.setAttribute("role", "status");
|
||||
note.textContent = linkNoteText("Text");
|
||||
this._textLinkNote = note;
|
||||
this.renderTextLink();
|
||||
return note;
|
||||
};
|
||||
nodeType.prototype.renderTextLink = function () {
|
||||
const linked = isLinked(this, "text");
|
||||
if (this._textLinkNote) this._textLinkNote.hidden = !linked;
|
||||
collapseLinkedWidget(this.widgets?.find((w) => w.name === "text"), linked);
|
||||
this.setDirtyCanvas?.(true, true);
|
||||
};
|
||||
|
||||
// Override onRemoved to cleanup styles
|
||||
const onRemoved = nodeType.prototype.onRemoved;
|
||||
nodeType.prototype.onRemoved = function () {
|
||||
@@ -526,6 +591,10 @@ app.registerExtension({
|
||||
searchSection.style.height = "100%";
|
||||
searchSection.style.gap = "8px";
|
||||
|
||||
const linkNote = this.createTextLinkNote();
|
||||
linkNote.style.cssText = "margin: 0; color: var(--fg-color); font-size: 12px;";
|
||||
searchSection.appendChild(linkNote);
|
||||
|
||||
// Search buttons row
|
||||
const buttonRow = document.createElement("div");
|
||||
buttonRow.style.display = "flex";
|
||||
@@ -876,6 +945,11 @@ app.registerExtension({
|
||||
|
||||
// Method to use a selected prompt
|
||||
nodeType.prototype.usePrompt = function (prompt, index) {
|
||||
// The typed text under a link is kept for when it is disconnected (#343)
|
||||
if (isLinked(this, "text")) {
|
||||
this.showNotification("Text comes from a link. Disconnect it to use a saved prompt.", "error");
|
||||
return;
|
||||
}
|
||||
// Find the text widget and set its value
|
||||
const textWidget = this.widgets?.find((w) => w.name === "text");
|
||||
if (textWidget) {
|
||||
|
||||
@@ -12,3 +12,5 @@
|
||||
.pm-classic-restore { white-space: normal !important; }
|
||||
.pm-classic-notification { position: fixed; inset-block-start: 20px; inset-inline-end: 20px; padding: 10px 15px; border: 1px solid var(--ca-border); border-radius: 5px; z-index: 10000; max-inline-size: min(300px, calc(100vw - 40px)); overflow-wrap: anywhere; }
|
||||
.pm-classic-notification[data-kind="error"] { color: var(--ca-error-text); background: var(--ca-error-bg); border-color: var(--ca-error); }
|
||||
.pm-classic .pm-classic-link { margin: 0; padding: 6px 8px; border: 1px solid var(--ca-line); border-radius: 5px; color: var(--ca-text); background: var(--ca-field); overflow-wrap: anywhere; }
|
||||
.pm-classic .pm-classic-link[hidden] { display: none; }
|
||||
|
||||
@@ -44,6 +44,9 @@ export function installClassicControls(nodeType, app, parseSearchQuery, api) {
|
||||
this._classicContainer = container;
|
||||
const wrapper = document.createElement('div');
|
||||
wrapper.className = 'pm-classic-workspace';
|
||||
const linkNote = this.createTextLinkNote();
|
||||
linkNote.classList.add('pm-classic-link');
|
||||
wrapper.append(linkNote);
|
||||
const buttons = document.createElement('div');
|
||||
buttons.className = 'pm-classic-toolbar';
|
||||
for (const [label, action, primary] of [
|
||||
|
||||
@@ -275,7 +275,11 @@
|
||||
|
||||
function widgetNames(n) {
|
||||
const fromInputs = (n.inputs || []).filter((i) => i && i.widget && i.widget.name).map((i) => i.widget.name);
|
||||
if (fromInputs.length) return fromInputs;
|
||||
if (fromInputs.length) {
|
||||
// Socket inputs that are not widget inputs (#343: the editors' text) would shift the positional values.
|
||||
const known = typeof n.type === "string" && Object.prototype.hasOwnProperty.call(WIDGET_POSITIONS, n.type) ? WIDGET_POSITIONS[n.type] : null;
|
||||
return known && fromInputs.some((name, index) => known[index] !== name) ? known : fromInputs;
|
||||
}
|
||||
if (WIDGET_POSITIONS[n.type]) return WIDGET_POSITIONS[n.type];
|
||||
if (/checkpointloader/i.test(n.type || "")) return ["ckpt_name"];
|
||||
if (/unet/i.test(n.type || "")) return ["unet_name"];
|
||||
|
||||
+40
-3
@@ -1,6 +1,7 @@
|
||||
import {createScope, createButton, createField, mountTheme} from './shared/toolkit/core.js';
|
||||
import {createPromptLibrary} from './prompt-library.js';
|
||||
import {attachMetadataAutocomplete} from './metadata-autocomplete.js';
|
||||
import {exposeTextSockets, linkNoteText, preferTextInputs, textInputAtPoint} from './text-links.js';
|
||||
|
||||
const names = ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text'];
|
||||
let nextEditorId = 0;
|
||||
@@ -23,12 +24,16 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
const pair = layout === 'pair';
|
||||
const split = layout !== 'focused';
|
||||
const uiKey = pair ? 'pairUI' : split ? 'splitUI' : 'focusedUI';
|
||||
const textInputs = pair ? ['text','negative_text'] : ['text'];
|
||||
const prototype = nodeType.prototype;
|
||||
preferTextInputs(prototype, textInputs);
|
||||
const created = prototype.onNodeCreated;
|
||||
const removed = prototype.onRemoved;
|
||||
const configure = prototype.configure;
|
||||
const serialize = prototype.serialize;
|
||||
const connections = prototype.onConnectionsChange;
|
||||
const inputOnPos = prototype.getInputOnPos;
|
||||
const added = prototype.onAdded;
|
||||
prototype.onNodeCreated = function (...args) {
|
||||
created?.apply(this, args);
|
||||
const scope = createScope(); this._focusedScope = scope;
|
||||
@@ -41,6 +46,7 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
widget.hidden = true; widget.type = 'hidden'; widget.computeSize = () => [0, -4];
|
||||
widgets.set(name, widget);
|
||||
}
|
||||
exposeTextSockets(this, textInputs);
|
||||
const element = document.createElement('div'); element.className = `ca-tools pm-focused${split ? ' pm-split' : ''}${pair ? ' pm-pair' : ''}`;
|
||||
element.setAttribute('aria-label',pair ? 'Paired prompt editor' : split ? 'Split prompt editor' : 'Focused prompt editor');
|
||||
loadStyles(); mountTheme(element, app.ui?.settings, scope);
|
||||
@@ -63,6 +69,7 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
const mode = data?.settings?.webui_display_mode;
|
||||
if (!scope.signal.aborted && data?.success && ['popup','newtab'].includes(mode)) webDisplayMode = mode;
|
||||
}).catch(() => {});
|
||||
const dropTargets = [];
|
||||
function textControl(name, label) {
|
||||
const widget = widgets.get(name);
|
||||
const nativeInput = widget.inputEl;
|
||||
@@ -78,7 +85,14 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
const field = createField(label,input,`${editorId}-${name}`);
|
||||
field.classList.add('pm-editor-main');
|
||||
const note = document.createElement('p'); note.className = 'pm-editor-status';
|
||||
note.textContent = `${pair ? label : 'Text'} is supplied by an upstream connection.`;
|
||||
note.textContent = linkNoteText(label);
|
||||
// The prompt box and its link note are drop targets for this input (#343): Nodes 2.0 resolves a drop
|
||||
// through the nearest `.lg-node-widget` and its `[data-slot-key]`; Classic asks getInputOnPos.
|
||||
for (const target of [field,note]) {
|
||||
const key = document.createElement('span'); key.hidden = true; key.dataset.slotKey = '';
|
||||
target.classList.add('lg-node-widget'); target.append(key);
|
||||
dropTargets.push({name,element:target,key});
|
||||
}
|
||||
return {input,field,note};
|
||||
}
|
||||
const positive = textControl('text',pair ? 'Positive prompt' : 'Main prompt');
|
||||
@@ -143,7 +157,12 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
widget.callback?.(text, app.canvas, thisNode);
|
||||
thisNode.setDirtyCanvas(true,true); app.graph?.setDirtyCanvas(true,true);
|
||||
}
|
||||
// The node id is assigned when the node joins a graph (and changes on paste), so keys follow it.
|
||||
function syncSlotKeys() {
|
||||
for (const target of dropTargets) target.key.dataset.slotKey = `${thisNode.id}-in-${thisNode.inputs?.findIndex(input => input.name === target.name)}`;
|
||||
}
|
||||
function render() {
|
||||
syncSlotKeys();
|
||||
mainField.hidden = Boolean(isConnected('text')); connected.hidden = !isConnected('text');
|
||||
if (negative) { negative.field.hidden = Boolean(isConnected('negative_text')); negative.note.hidden = !isConnected('negative_text'); }
|
||||
if (save) { save.disabled = saving || hasConnectedPairValues(); saveConnectionNotice.hidden = !hasConnectedPairValues(); }
|
||||
@@ -214,15 +233,33 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
footer.setAttribute('role',opened ? 'status' : 'alert');
|
||||
footer.textContent = opened ? 'Web UI opened.' : 'Allow popups to open Web UI, then try again.';
|
||||
});
|
||||
this._focusedView = {element,render,restore,snapshot};
|
||||
function inputAt(position) {
|
||||
const canvas = app.canvas;
|
||||
if (canvas?.linkConnector?.state?.connectingTo !== 'input' || !canvas.ds || !canvas.canvas) return undefined;
|
||||
const [x,y] = canvas.ds.convertOffsetToCanvas(position);
|
||||
const box = canvas.canvas.getBoundingClientRect();
|
||||
return textInputAtPoint(thisNode, [box.left + x, box.top + y], dropTargets);
|
||||
}
|
||||
this._focusedView = {element,render,restore,snapshot,inputAt,syncSlotKeys};
|
||||
this.addDOMWidget('prompt_manager_focused_ui','div',element,{serialize:false, hideOnZoom:false, getMinHeight:() => 360});
|
||||
this.size = [pair ? 1100 : split ? 900 : 550,pair ? 700 : 650]; this.setSize(this.size); restore();
|
||||
};
|
||||
prototype.configure = function (...args) { const result = configure?.apply(this,args); this._focusedView?.restore(); return result; };
|
||||
prototype.configure = function (...args) {
|
||||
const result = configure?.apply(this,args);
|
||||
// A saved workflow restores its inputs with their widget bindings; keep the text inputs as sockets.
|
||||
exposeTextSockets(this, textInputs);
|
||||
this._focusedView?.restore();
|
||||
return result;
|
||||
};
|
||||
prototype.serialize = function (...args) {
|
||||
if (this._focusedView) this.properties[uiKey] = this._focusedView.snapshot();
|
||||
return serialize?.apply(this,args);
|
||||
};
|
||||
prototype.onConnectionsChange = function (...args) { const result = connections?.apply(this,args); this._focusedView?.render(); return result; };
|
||||
prototype.onRemoved = function (...args) { this._focusedScope?.dispose(); return removed?.apply(this,args); };
|
||||
prototype.onAdded = function (...args) { const result = added?.apply(this,args); this._focusedView?.syncSlotKeys(); return result; };
|
||||
// Classic: a link dropped on a prompt box (or its link note) fills that box's input.
|
||||
prototype.getInputOnPos = function (...args) {
|
||||
return inputOnPos?.apply(this,args) ?? this._focusedView?.inputAt(args[0]);
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
// #343: let a link drive the editor nodes' prompt text, the way a link drives any core widget.
|
||||
//
|
||||
// The Focused, Split and Pair editors hide their widgets to draw their own editor. ComfyUI's frontend draws
|
||||
// and hit-tests a widget's input socket only while the widget itself is visible, so `text` (and the Pair's
|
||||
// `negative_text`) had no socket, and a STRING link dropped on the editor fell through to the first free
|
||||
// STRING input, `prepend_text` (#342). Detaching those inputs from their hidden widgets makes them ordinary
|
||||
// sockets in both renderers; the schema, the widgets and their saved values are unchanged.
|
||||
|
||||
/** What an editor shows in place of a text field whose value comes from a link. */
|
||||
export function linkNoteText(label) {
|
||||
return `${label} comes from a link. Your typed text is kept for when you disconnect it.`;
|
||||
}
|
||||
|
||||
export function isLinked(node, name) {
|
||||
return Boolean(node?.inputs?.some(input => input.name === name && input.link != null));
|
||||
}
|
||||
|
||||
/** Detach the named inputs from their hidden widgets so the frontend shows them as sockets. Returns the names changed. */
|
||||
export function exposeTextSockets(node, names) {
|
||||
const exposed = [];
|
||||
for (const input of node?.inputs || []) {
|
||||
if (!names.includes(input.name) || !('widget' in input || input._widget)) continue;
|
||||
delete input.widget;
|
||||
if ('_widget' in input) input._widget = undefined;
|
||||
exposed.push(input.name);
|
||||
}
|
||||
return exposed;
|
||||
}
|
||||
|
||||
function acceptsString(type) {
|
||||
// Only exact STRING links: a wildcard (a legacy Primitive before it is typed, an untyped reroute) keeps the
|
||||
// frontend's own choice, and is aimed at a prompt by dropping it on the prompt box or its socket.
|
||||
return type != null && String(type).split(',').some(part => part.trim() === 'STRING');
|
||||
}
|
||||
|
||||
function linkOf(graph, id) {
|
||||
if (id == null || !graph) return undefined;
|
||||
return graph.getLink?.(id) ?? graph.links?.get?.(id) ?? graph.links?.[id];
|
||||
}
|
||||
|
||||
// The frontend's own free test: an input whose link is being moved (`_dragging`), or whose link the graph no
|
||||
// longer has, takes a drop. Without a graph only the input's own link field is known.
|
||||
function isFree(node, input) {
|
||||
if (input.link == null) return true;
|
||||
if (!node.graph) return false;
|
||||
const link = linkOf(node.graph, input.link);
|
||||
return link == null || Boolean(link._dragging);
|
||||
}
|
||||
|
||||
/** The first free text input, in editor order, for a link of `type`; undefined leaves the choice to the frontend. */
|
||||
export function preferredTextSlot(node, type, names) {
|
||||
if (!acceptsString(type)) return undefined;
|
||||
const inputs = node?.inputs || [];
|
||||
for (const name of names) {
|
||||
const index = inputs.findIndex(input => input.name === name);
|
||||
if (index >= 0 && isFree(node, inputs[index])) return {index, slot:inputs[index]};
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const MAX_VIRTUAL_HOPS = 32;
|
||||
const REPLACE_OPTION = 'Run widget replace on values';
|
||||
|
||||
/** The legacy frontend Primitive (a virtual node) at the source of a link, following virtual reroutes. */
|
||||
function primitiveSource(graph, id) {
|
||||
let link = linkOf(graph, id);
|
||||
for (let hop = 0; link && hop < MAX_VIRTUAL_HOPS; hop++) {
|
||||
const origin = graph.getNodeById?.(link.origin_id);
|
||||
if (!origin?.isVirtualNode) return undefined;
|
||||
if (origin.type === 'PrimitiveNode') return origin;
|
||||
link = linkOf(graph, origin.inputs?.[0]?.link);
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** A widget's queued value for `value`: its own serializer (the host's dynamic prompts) without touching its text. */
|
||||
function serializeAs(widget, value) {
|
||||
if (typeof widget?.serializeValue !== 'function') return value;
|
||||
const typed = widget.value;
|
||||
widget.value = value;
|
||||
try {
|
||||
// No node argument: the host would also write the result into the workflow's widgets_values.
|
||||
return widget.serializeValue();
|
||||
} finally {
|
||||
widget.value = typed;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The value a legacy frontend Primitive supplies to a linked editor text input, resolved the way the host resolves a
|
||||
* value the Primitive writes into a widget: its text replacement when its "Run widget replace on values" option is on,
|
||||
* then the input's own widget serializer (`{red|blue}` becomes one choice). Undefined when the input is unlinked, its
|
||||
* source is a real node (serialized as a link), or the input is still bound to its widget, where the host already
|
||||
* applied the Primitive (#343 re-review R1).
|
||||
*
|
||||
* The editor text inputs are no longer bound to their hidden widgets, so the Primitive cannot write into them and the
|
||||
* queue path fills the value in instead (#343 review I1). Each call draws a new wildcard choice: call it once per run.
|
||||
*/
|
||||
export async function primitiveTextValue(node, name, {replaceText} = {}) {
|
||||
const input = node?.inputs?.find(item => item.name === name);
|
||||
const graph = node?.graph;
|
||||
if (input?.link == null || input.widget || !graph) return undefined;
|
||||
const primitive = primitiveSource(graph, input.link);
|
||||
let value = primitive?.widgets?.[0]?.value;
|
||||
if (value == null) return undefined;
|
||||
value = String(value);
|
||||
if (value && primitive.properties?.[REPLACE_OPTION] && replaceText) {
|
||||
try {
|
||||
value = replaceText(graph, value);
|
||||
} catch (error) {
|
||||
console.warn('[PromptManager] text replacement skipped for a linked Primitive', error);
|
||||
}
|
||||
}
|
||||
return serializeAs(node.widgets?.find(widget => widget.name === name), value);
|
||||
}
|
||||
|
||||
/** A link dropped on the editor body (not on a socket) fills the main prompt first, then the negative. */
|
||||
export function preferTextInputs(prototype, names) {
|
||||
const original = prototype.findInputByType;
|
||||
if (typeof original !== 'function') return;
|
||||
prototype.findInputByType = function (type, ...rest) {
|
||||
return preferredTextSlot(this, type, names) ?? original.call(this, type, ...rest);
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The value a workflow stores for a widget. A linked widget keeps what the user typed (its value), so the text
|
||||
* returns when the link is disconnected; an unlinked one stores its live editor text.
|
||||
*/
|
||||
export function storedWidgetValue(node, widget) {
|
||||
if (isLinked(node, widget.name)) return widget.value;
|
||||
return widget.inputEl ? widget.inputEl.value : widget.value;
|
||||
}
|
||||
|
||||
/** The text input whose visible drop target (a prompt box or its link note) contains a viewport point. */
|
||||
export function textInputAtPoint(node, [x, y], targets) {
|
||||
for (const {name, element} of targets) {
|
||||
const box = element.getBoundingClientRect();
|
||||
if (!box.width || !box.height || x < box.left || x > box.right || y < box.top || y > box.bottom) continue;
|
||||
const input = node?.inputs?.find(item => item.name === name);
|
||||
if (input) return input;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const LAYOUT_OPTIONS = ['getMinHeight','getMaxHeight'];
|
||||
|
||||
/**
|
||||
* While its input is linked, the frontend hides a core text box but keeps its row, leaving an empty gap in the
|
||||
* Classic renderer. Give the row back while linked and restore the widget's own sizing on disconnect.
|
||||
*/
|
||||
export function collapseLinkedWidget(widget, linked) {
|
||||
const options = widget?.options;
|
||||
if (!options) return;
|
||||
if (linked && !widget._pmLinkedLayout) {
|
||||
widget._pmLinkedLayout = Object.fromEntries(LAYOUT_OPTIONS.filter(name => name in options).map(name => [name, options[name]]));
|
||||
for (const name of LAYOUT_OPTIONS) options[name] = () => 0;
|
||||
} else if (!linked && widget._pmLinkedLayout) {
|
||||
for (const name of LAYOUT_OPTIONS) {
|
||||
if (name in widget._pmLinkedLayout) options[name] = widget._pmLinkedLayout[name];
|
||||
else delete options[name];
|
||||
}
|
||||
delete widget._pmLinkedLayout;
|
||||
}
|
||||
}
|
||||
@@ -14,7 +14,8 @@
|
||||
* Errors (red). System S7: `maintenance` (a reviewed plan's apply) counts Changed, Protected and Errors and, when it
|
||||
* failed, ends with the server's reason; `duplicates` (the search) counts Groups found; `duplicate_cleanup` counts
|
||||
* Removed, Kept and Errors; S10: `lora_import` counts Imported, Linked to existing, Skipped, Images downloaded, Download
|
||||
* errors and Errors and, when it failed, ends with the server's reason (the previous imports were kept). Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
|
||||
* errors and Errors and, when it failed, ends with the server's reason (the previous imports were kept); S9: `restore`
|
||||
* ends with the server's line (what was restored, or why nothing changed). Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
|
||||
* the generic descriptor: title-based copy and no tiles or outcomes, so it never shows another kind's zeros (S1 review
|
||||
* M6, S2 review M3). `viewOf(job)` keeps only the groups the job itself counts, so a hold (no groups) has none either.
|
||||
*
|
||||
@@ -190,6 +191,16 @@ export const KINDS = Object.freeze({
|
||||
openable: groups('kept'),
|
||||
log: 'Cleanup log',
|
||||
}),
|
||||
// System S9: the reviewed database restore (exclusive, never cancellable, indeterminate); it ends with the server's
|
||||
// line: what was restored, or why nothing was changed.
|
||||
restore: Object.freeze({
|
||||
...GENERIC,
|
||||
pillVerb: 'Restoring the database',
|
||||
finishToast: job => (job.status_line && job.state !== 'cancelled'
|
||||
? `${job.title} ${job.state === 'failed' ? 'failed' : 'finished'} · ${job.status_line}` : titleFinished(job)),
|
||||
line: job => (job.state === 'running' ? '' : job.status_line || ''),
|
||||
log: 'Restore log',
|
||||
}),
|
||||
// System S10: the reviewed LoRA reference import; cancellable only while it downloads (the server says when).
|
||||
lora_import: Object.freeze({
|
||||
...GENERIC,
|
||||
|
||||
@@ -12,6 +12,10 @@ import { navigateTo } from './state.mjs';
|
||||
|
||||
export const BACKUP_URL = '/prompt_manager/backup';
|
||||
export const BACKUP_DONE = 'Database backup downloaded.';
|
||||
// S8 M5: a copy that failed a semantic check is still sent, with the reason in this header ('ok' otherwise).
|
||||
export const VERIFY_HEADER = 'X-PromptManager-Verify';
|
||||
/** The toast for a downloaded copy the server could not verify (S9). */
|
||||
export const backupUnverified = reason => `Database backup downloaded, but it did not pass verification: ${reason}`;
|
||||
export const BACKUP_FAILED = 'The database could not be backed up. Try again.';
|
||||
export const REPORT_DONE = 'Job report downloaded.';
|
||||
export const REPORT_FAILED = 'The report could not be downloaded. Try again.';
|
||||
@@ -28,8 +32,8 @@ export function backupFilename(disposition, fallback = FALLBACK_NAME) {
|
||||
return match ? match[1] : fallback;
|
||||
}
|
||||
|
||||
/** Save a fetched response through a temporary link, named by `filename`. */
|
||||
function saveResponse(blob, filename) {
|
||||
/** Save a fetched response through a temporary link, named by `filename` (also System's exports, S9). */
|
||||
export function saveResponse(blob, filename) {
|
||||
const url = URL.createObjectURL(blob);
|
||||
const link = document.createElement('a');
|
||||
link.href = url;
|
||||
@@ -40,14 +44,18 @@ function saveResponse(blob, filename) {
|
||||
setTimeout(() => URL.revokeObjectURL(url), 1000);
|
||||
}
|
||||
|
||||
/** Download the prompts database copy the legacy admin page also uses; toasts the outcome. */
|
||||
/**
|
||||
* Download the prompts database copy the legacy admin page also uses; toasts the outcome, and says so when the server
|
||||
* sent a copy it could not verify (S9).
|
||||
*/
|
||||
export async function downloadDatabaseBackup({ fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast } = {}) {
|
||||
try {
|
||||
const response = await fetchUrl(BACKUP_URL);
|
||||
if (!response.ok) throw new Error(`Backup failed with ${response.status}`);
|
||||
const blob = await response.blob();
|
||||
saveResponse(blob, backupFilename(response.headers.get('Content-Disposition')));
|
||||
notify(BACKUP_DONE);
|
||||
const verify = response.headers.get(VERIFY_HEADER);
|
||||
notify(verify && verify !== 'ok' ? backupUnverified(verify) : BACKUP_DONE);
|
||||
return true;
|
||||
} catch {
|
||||
notify(BACKUP_FAILED);
|
||||
|
||||
@@ -12,7 +12,7 @@ import { createMediaViewer } from './media-viewer.mjs';
|
||||
import { noun } from './plural.mjs';
|
||||
import { pageHeading } from './heading.mjs';
|
||||
import { mountToolbar } from './toolbar.mjs';
|
||||
import { openImportDialog, openRefineDialog } from './workspace-dialogs.mjs';
|
||||
import { RESCAN_FAILED, RESCAN_STARTED, openImportDialog, openRefineDialog, rescanFinished } from './workspace-dialogs.mjs';
|
||||
import { openMetadataScan } from './activity-scan.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
import { imagesFilters, imagesViewState } from './saved-views.mjs';
|
||||
@@ -70,15 +70,6 @@ function loadFailure(sourcesRetry, categoriesFailed) {
|
||||
if (sourcesRetry) return `Sources and folders could not be loaded. ${sourcesRetry}`;
|
||||
return categoriesFailed ? `Categories could not be loaded. ${RETRY_RESCAN}` : '';
|
||||
}
|
||||
// Import → Rescan (D15): the concept has no rescan copy, so its toasts follow the workspace pattern. The answer
|
||||
// counts every listed file across the output roots it walked.
|
||||
const RESCAN_STARTED = 'Rescanning output folders…';
|
||||
const RESCAN_FAILED = 'Rescan could not finish. Try again.';
|
||||
function rescanFinished(result) {
|
||||
const {total, roots} = result ?? {};
|
||||
if (![total, roots].every(value => Number.isSafeInteger(value) && value >= 0)) return 'Rescan finished.';
|
||||
return `Rescan finished. ${total.toLocaleString('en-US')} ${noun(total, 'file')} in ${roots.toLocaleString('en-US')} output ${noun(roots, 'folder')}.`;
|
||||
}
|
||||
// The concept card's first `.media-info` reservation and list row (image-layout.mjs); real rows are measured.
|
||||
const INITIAL_METRICS = Object.freeze({columns:1, gap:13, footerHeight:54, listHeight:97});
|
||||
|
||||
@@ -90,6 +81,10 @@ setFavoriteVisitTarget((key, favorite) => {
|
||||
|
||||
// A tag save made while Images is left (Tags › Auto-tag images) may change what a filtered visit lists: it reloads (M6).
|
||||
onImageTagsSaved(() => { if (lastVisit?.filters.tags.length || lastVisit?.filters.q) lastVisit = {...lastVisit, stale: true}; });
|
||||
/** Forget the last Images visit (its listing, scroll and selection): the next visit reads everything again (S9). */
|
||||
export function forgetImagesVisit() {
|
||||
lastVisit = null;
|
||||
}
|
||||
/** The image selected in Images when it was last left (the concept's `state.selectedImage`, for Tags › Auto-tag images). */
|
||||
export function selectedImage() {
|
||||
const key = lastVisit?.selection;
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import '../api-client.js';
|
||||
import { createButton, createScope } from '../shared/toolkit/core.js';
|
||||
import { routes, mountRouter, summarizeStats, routeTitle, loadNavCounts,
|
||||
import { routes, mountRouter, summarizeStats, routeTitle, loadNavCounts, navigateTo,
|
||||
LIBRARY_WRITE_COUNTS, NAV_COUNT_ROUTES, FAVORITE_WRITE_COUNTS, FAVORITE_COUNT_DELAY_MS, REVIEW_WRITE_COUNTS } from './state.mjs';
|
||||
import { mountWorkspaceTheme } from './theme.mjs';
|
||||
import { mountImages, onFavoriteChange } from './images.mjs';
|
||||
import { mountLibrary, mountNewPromptAction, onLibraryWrite } from './library.mjs';
|
||||
import { forgetImagesVisit, mountImages, onFavoriteChange } from './images.mjs';
|
||||
import { forgetLibraryVisit, mountLibrary, mountNewPromptAction, onLibraryWrite } from './library.mjs';
|
||||
import { mountVocabulary } from './vocabulary.mjs';
|
||||
import { createWorkspaceIcon } from './icons.mjs';
|
||||
import { bindCanvasLink, canvasHref } from './topbar.mjs';
|
||||
@@ -18,6 +18,7 @@ import { mountInspect } from './inspect.mjs';
|
||||
import { mountReview } from './review.mjs';
|
||||
import { createPromptSender } from './prompt-sender.mjs';
|
||||
import { mountSystem } from './system.mjs';
|
||||
import { onRestoreFinished, onRestoreHandedBack, showRestoreReview } from './system-backups.mjs';
|
||||
import { mountLibraryBanner } from './library-location.mjs';
|
||||
import { detectJxlSupport } from './thumbnail-support.mjs';
|
||||
|
||||
@@ -159,6 +160,20 @@ function mountWorkspace() {
|
||||
if (ended) refreshCounts(REVIEW_WRITE_COUNTS);
|
||||
}));
|
||||
refreshCounts(NAV_COUNT_ROUTES);
|
||||
// S9: a finished restore replaced the library. The cached visits are dropped, the counts and saved views read
|
||||
// again, and the open page is shown again from the restored data; the Activity job view the restore opened stays.
|
||||
scope.cleanup(onRestoreFinished(activity, () => {
|
||||
forgetImagesVisit();
|
||||
forgetLibraryVisit();
|
||||
refreshCounts(NAV_COUNT_ROUTES);
|
||||
savedViews.load();
|
||||
if (currentKey !== 'activity') navigateTo(window.location.hash);
|
||||
}));
|
||||
// S9 I1: a restore this page started that could not run yet (other requests kept reading) hands its review back:
|
||||
// it opens again over the page shown now, with Restore (again) and Discard; leaving it any other way discards it.
|
||||
scope.cleanup(onRestoreHandedBack(activity, review => {
|
||||
if (viewScope && !viewScope.signal.aborted) showRestoreReview({ api, store: activity, scope: viewScope, ...review, again: true });
|
||||
}));
|
||||
let viewScope;
|
||||
let firstRender = true;
|
||||
// D62: a page whose teardown must wait (Inspect, while a video is still starting) holds the next route until it
|
||||
|
||||
@@ -65,6 +65,10 @@ export function invalidateLibraryAfterWrite(commit) {
|
||||
if(shelfVisit) shelfVisit={filters:shelfVisit.filters,anchor:shelfVisit.layout?.anchor ?? shelfVisit.anchor ?? null};
|
||||
for(const listener of [...writeListeners]) listener(commit);
|
||||
}
|
||||
/** Forget the shelf's last visit (filters, items and position): the next visit starts fresh (S9: a restored library). */
|
||||
export function forgetLibraryVisit() {
|
||||
writeGeneration++;shelfVisit=null;
|
||||
}
|
||||
/** Observe confirmed Library and vocabulary writes (the sidebar re-reads its counts); returns the unsubscribe. */
|
||||
export function onLibraryWrite(listener) {
|
||||
if(typeof listener!=='function') throw new TypeError('Invalid Library write listener');
|
||||
|
||||
@@ -155,7 +155,7 @@ export function reviewRefusal(error) {
|
||||
}
|
||||
|
||||
/** Hold `trigger` aria-disabled with `busyText` while `work()` runs; a second press is ignored. */
|
||||
async function holding(trigger, busyText, work) {
|
||||
export async function holding(trigger, busyText, work) {
|
||||
if (trigger?.getAttribute('aria-disabled') === 'true') return;
|
||||
const text = trigger?.textContent;
|
||||
trigger?.setAttribute('aria-disabled', 'true');
|
||||
@@ -169,9 +169,11 @@ async function holding(trigger, busyText, work) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply a reviewed plan's `token` (at `url`: maintenance's apply, or the LoRA import's, S10); the job opens in Activity.
|
||||
* `reviewAgain()` reviews the operation once more when the server says the review expired, the database was replaced
|
||||
* or (the LoRA import) its files changed.
|
||||
* Apply a reviewed plan's `token` (at `url`: maintenance's apply, the LoRA import's (S10) or the restore's (S9)); the
|
||||
* job opens in Activity. `reviewAgain()` reviews the operation once more when the server says the review expired, the
|
||||
* database was replaced or (the LoRA import) its files changed. Resolves with the outcome, which maintenance and the
|
||||
* LoRA import ignore and the restore acts on (S9 I1): `{job}` once started, `{refused: error}` when the server or the
|
||||
* network refused it, `{lost: true}` when the answer could not be read (the job may be running).
|
||||
*/
|
||||
export async function applyPlan({ api, store, scope, token, reviewAgain, notify = showWorkspaceToast, navigate = navigateTo,
|
||||
url = APPLY_URL }) {
|
||||
@@ -179,24 +181,28 @@ export async function applyPlan({ api, store, scope, token, reviewAgain, notify
|
||||
try {
|
||||
result = await api.post(url, { token });
|
||||
} catch (error) {
|
||||
if (scope.signal.aborted) return;
|
||||
if (scope.signal.aborted) return { refused: error };
|
||||
const refusal = applyRefusal(error);
|
||||
notify(refusal.toast);
|
||||
if (refusal.lost) store.load();
|
||||
if (refusal.review) reviewAgain?.();
|
||||
return;
|
||||
return refusal.lost ? { lost: true } : { refused: error };
|
||||
}
|
||||
if (scope.signal.aborted) return;
|
||||
let job;
|
||||
try {
|
||||
// The page's store follows the started job even when its section was left meanwhile (S9: a restore that ends
|
||||
// must still reload the views), but only a section still shown toasts and opens it.
|
||||
job = store.track(result?.job);
|
||||
} catch {
|
||||
notify(NOT_CONFIRMED);
|
||||
if (!scope.signal.aborted) notify(NOT_CONFIRMED);
|
||||
store.load();
|
||||
return;
|
||||
return { lost: true };
|
||||
}
|
||||
notify(`${job.title} started.`);
|
||||
navigate(`#/activity/${job.id}`);
|
||||
if (!scope.signal.aborted) {
|
||||
notify(`${job.title} started.`);
|
||||
navigate(`#/activity/${job.id}`);
|
||||
}
|
||||
return { job };
|
||||
}
|
||||
|
||||
/** Show the reviewed `plan` for `operation`; Apply applies exactly this token. */
|
||||
|
||||
@@ -0,0 +1,464 @@
|
||||
/**
|
||||
* System › Backup & export (#249 System S9 Task 24): the concept's `renderSystem` "backups" body (app.js 691–693), its
|
||||
* `restore` dialog ("Restore preview", a1472–1476) and its `import` dialog (a1483–1487) on the S8 backend.
|
||||
*
|
||||
* - "Database backup": D103 "Back up" (the concept's "Simulate backup") downloads the verified copy of
|
||||
* `GET /prompt_manager/backup` through the api client (`downloadDatabaseBackup`, the same download as every backup
|
||||
* box), then reads the status again. The `p.small` under the actions is `GET /prompt_manager/backup/status` only:
|
||||
* this session's last download (verified or not) and the newest safety backup on disk, else "No backup recorded."
|
||||
* Overview's "Backup status" row says the same (`backupStatusText`).
|
||||
* - "Review restore…" picks a `.db` file and stages it (`POST /prompt_manager/restore/preview`); D105: "Restore
|
||||
* preview" shows the server's answer in the shared review (`openReviewDialog`): the concept's Snapshot / Integrity /
|
||||
* Migration / Safety backup rows with real values, plus what the backup holds and what the library holds now, the
|
||||
* hold line, the D55 backup box and Cancel / "Restore" (`danger`). Leaving it any way but Restore discards the stage
|
||||
* (`DELETE /prompt_manager/restore/preview/<token>`). Restore starts the "Database restore" Activity job (exclusive,
|
||||
* not cancellable) and opens its view; a finished restore reloads the open views (`onRestoreFinished`, index.mjs).
|
||||
* I1: a Restore the server refuses as busy reopens the same review (Restore retries, Discard releases); a job that
|
||||
* hands its review back (the leases did not drain) reopens it in this page (`onRestoreHandedBack`, index.mjs);
|
||||
* every other path that abandons the token discards it. The picker has no type filter (M3: the safety backups have
|
||||
* no `.db` extension); a file the server refuses toasts its validator's reason, and nothing is staged.
|
||||
* - "Portable export": D103 "Export JSON" downloads the Library document (`/library/export`); the secondary row
|
||||
* downloads every prompt (`/export`, the legacy admin Export) and the saved pairs (`/pairs/export`). "Import / inspect file"
|
||||
* opens the shared "Import & inspect" (D15, D55): Inspect a local image, Rescan and Scan for metadata.
|
||||
* - D104 "Recovery policy" (the concept's "Proposed" chip kept, Q4): "Create a safety backup before maintenance" is
|
||||
* the server's `safety_backup` policy (its line says what it does), drafted behind the Save row and saved with the
|
||||
* revision; "Scheduled database
|
||||
* backups" offers Manual only, Daily and Weekly disabled with the reason; the warning note is truthful.
|
||||
*
|
||||
* Every request goes through the api client (or `fetchUrl` for a download or an upload); every listener sits on the
|
||||
* section's `scope`.
|
||||
*/
|
||||
import { createButton } from '../shared/toolkit/core.js';
|
||||
import { navigateTo } from './state.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
import { settingRow } from './system-controls.mjs';
|
||||
import { busyToast, downloadDatabaseBackup, openMetadataScan, saveResponse } from './activity-scan.mjs';
|
||||
import { applyPlan, holding } from './maintenance-review.mjs';
|
||||
import { RESCAN_FAILED, RESCAN_STARTED, openImportDialog, openReviewDialog, rescanFinished } from './workspace-dialogs.mjs';
|
||||
import { noun } from './plural.mjs';
|
||||
|
||||
export const STATUS_URL = '/prompt_manager/backup/status';
|
||||
export const PREVIEW_URL = '/prompt_manager/restore/preview';
|
||||
export const APPLY_URL = '/prompt_manager/restore/apply';
|
||||
const SETTINGS_URL = '/prompt_manager/settings';
|
||||
const RESCAN_URL = '/prompt_manager/gallery/rescan';
|
||||
export const RESTORE_KIND = 'restore';
|
||||
export const RESTORE_TITLE = 'Restore preview';
|
||||
export const RESTORE_LABEL = 'Restore';
|
||||
|
||||
export const STATUS_LOADING = 'Checking backups…';
|
||||
export const STATUS_FAILED = 'The backup status could not be read.';
|
||||
export const NO_BACKUP = 'No backup recorded.';
|
||||
export const REVIEW_FAILED = 'The backup could not be reviewed. Nothing was changed.';
|
||||
export const EXPORT_FAILED = 'The export could not be downloaded. Nothing was saved to this device.';
|
||||
export const SOURCES_FAILED = 'The output folders could not be read. Try again.';
|
||||
export const HOST_UNAVAILABLE = 'Server settings could not be read.';
|
||||
export const SCHEDULE_REASON = 'Daily and Weekly are not available yet: back up with Back up whenever you choose.';
|
||||
export const WARNING_NOTE = 'Restores are staged, migrated and validated before replacing live data.';
|
||||
// D104 (M2): what the safety-backup policy does (the concept's "Preview a verified backup…" line misstates it).
|
||||
export const SAFETY_DESCRIPTION = 'Keep a verified copy of the database on disk before maintenance, cleanups and LoRA ' +
|
||||
'imports change records. A restore always keeps one.';
|
||||
// I1: a third review while two are held (another window's): what the user can do about it.
|
||||
export const REVIEWS_FULL = 'Two restores are already being reviewed in another window. Discard one there, or try again ' +
|
||||
'when it expires (15 minutes after it was reviewed).';
|
||||
export const RESTORE_BACKUP_COPY = Object.freeze({ heading: 'Back up the database first (recommended)',
|
||||
text: 'Downloads a copy of the prompts database before the restore replaces it.' });
|
||||
|
||||
/** D103: the concept's Export plus the two secondary exports: `[label, url, fallback file name]`. */
|
||||
export const EXPORTS = Object.freeze([
|
||||
['Export JSON', '/prompt_manager/library/export', 'prompt-manager-library-v1.json'],
|
||||
// N1: the legacy admin page's Export: every prompt, as `{export_date, total_prompts, prompts}`.
|
||||
['Export all prompts JSON', '/prompt_manager/export', 'prompt_manager_prompts.json'],
|
||||
['Export saved pairs JSON', '/prompt_manager/pairs/export', 'prompt-pairs.json'],
|
||||
]);
|
||||
|
||||
const number = value => value.toLocaleString('en-US');
|
||||
const WHEN = Object.freeze({ dateStyle: 'medium', timeStyle: 'short' });
|
||||
|
||||
function el(tag, text, className) {
|
||||
const node = document.createElement(tag);
|
||||
if (text !== undefined) node.textContent = text;
|
||||
if (className) node.className = className;
|
||||
return node;
|
||||
}
|
||||
|
||||
/** A server time as the reader's local date and time, or null when it is not one. */
|
||||
export function whenText(value) {
|
||||
const when = new Date(value);
|
||||
return typeof value === 'string' && !Number.isNaN(when.getTime()) ? when.toLocaleString(undefined, WHEN) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The status line (and Overview's Backup status) from `GET /backup/status`: undefined while it is read, null when it
|
||||
* failed. Only what the server reports is named: this session's last download and the newest safety backup on disk.
|
||||
*/
|
||||
export function backupStatusText(status) {
|
||||
if (status === undefined) return STATUS_LOADING;
|
||||
if (status === null || typeof status !== 'object') return STATUS_FAILED;
|
||||
const parts = [];
|
||||
const download = status.last_download;
|
||||
const downloaded = whenText(download?.at);
|
||||
if (downloaded) parts.push(`Downloaded backup · ${downloaded} · ${download.verified === true ? 'verified' : 'not verified'}.`);
|
||||
const safety = Array.isArray(status.safety_backups) ? status.safety_backups : [];
|
||||
const newest = whenText(safety[0]?.modified_at);
|
||||
if (newest) parts.push(`Safety backup · ${newest} · ${number(safety.length)} on disk.`);
|
||||
return parts.join(' ') || NO_BACKUP;
|
||||
}
|
||||
|
||||
/** "1,234 prompts · 56 image links · 3 saved pairs · 10 runs" for a restore summary (backup or live library). */
|
||||
export function countsText(counts) {
|
||||
const parts = [['prompts', 'prompt'], ['image_links', 'image link'], ['pairs', 'saved pair'], ['runs', 'run']]
|
||||
.filter(([key]) => Number.isSafeInteger(counts?.[key]))
|
||||
.map(([key, word]) => `${number(counts[key])} ${noun(counts[key], word)}`);
|
||||
return parts.join(' · ') || 'Not reported';
|
||||
}
|
||||
|
||||
/** The Integrity value: verified, with what the stage repaired or could not (S8's `documents_line`). */
|
||||
export function integrityText(preview) {
|
||||
const verified = preview?.integrity === 'ok' ? 'Verified' : 'Not verified';
|
||||
const documents = preview?.documents ?? {};
|
||||
const repaired = Number.isSafeInteger(documents.repaired) ? documents.repaired : 0;
|
||||
const cleared = Number.isSafeInteger(documents.cleared) ? documents.cleared : 0;
|
||||
const parts = [verified];
|
||||
if (repaired) parts.push(`repaired ${number(repaired)} damaged workflow ${noun(repaired, 'document')}`);
|
||||
if (cleared) {
|
||||
const images = Number.isSafeInteger(documents.images) ? documents.images : 0;
|
||||
parts.push(`could not repair ${number(cleared)} workflow ${noun(cleared, 'document')}: ${number(images)} ` +
|
||||
`${noun(images, 'image')} ${images === 1 ? 'shows' : 'show'} no workflow`);
|
||||
}
|
||||
return parts.join(' · ');
|
||||
}
|
||||
|
||||
/** A file size: whole KB below 1 MB (never "0.0 MB"), else MB with one decimal. */
|
||||
function sizeText(bytes) {
|
||||
if (bytes < 2 ** 20) return `${number(Math.max(1, Math.round(bytes / 1024)))} KB`;
|
||||
return `${(bytes / 2 ** 20).toLocaleString('en-US', { minimumFractionDigits: 1, maximumFractionDigits: 1 })} MB`;
|
||||
}
|
||||
|
||||
/** The Snapshot value: the chosen file's name, size and date. */
|
||||
export function snapshotText(file) {
|
||||
const size = Number.isFinite(file?.size) ? ` · ${sizeText(file.size)}` : '';
|
||||
const when = Number.isFinite(file?.lastModified) ? whenText(new Date(file.lastModified).toISOString()) : null;
|
||||
return `${file?.name ?? 'Backup file'}${size}${when ? ` · ${when}` : ''}`;
|
||||
}
|
||||
|
||||
/** D105: the concept's four rows with the review's values, then the two the decision needs. */
|
||||
export function restoreGrid(preview, file) {
|
||||
const destination = preview?.safety_backup?.destination_label;
|
||||
return [
|
||||
['Snapshot', snapshotText(file)],
|
||||
['Integrity', integrityText(preview)],
|
||||
['Migration', preview?.schema?.migrated === true ? 'Upgraded to the current schema' : 'Current schema'],
|
||||
['Safety backup', `Created before replacing${destination ? ` (${destination})` : ''}`],
|
||||
['In the backup', countsText(preview?.summary)],
|
||||
['In the library now', countsText(preview?.live)],
|
||||
];
|
||||
}
|
||||
|
||||
/** D105: the `p.small` line: what Restore does and until when the review holds. */
|
||||
export function restoreNote(preview, { now = Date.now() } = {}) {
|
||||
const parts = ['Restore replaces the whole library with this backup and runs alone in Activity.'];
|
||||
const expires = new Date(preview?.expires_at);
|
||||
if (!Number.isNaN(expires.getTime())) {
|
||||
const minutes = Math.max(1, Math.round((expires.getTime() - now) / 60000));
|
||||
parts.push(`Nothing changes until you restore. This review holds for ${number(minutes)} ${noun(minutes, 'minute')}.`);
|
||||
} else {
|
||||
parts.push('Nothing changes until you restore.');
|
||||
}
|
||||
return parts.join(' ');
|
||||
}
|
||||
|
||||
/** The toast for a review the server refused (its reason, a running job) or that could not be read. */
|
||||
export function reviewRefusal(error) {
|
||||
if (error?.code === 'busy') return busyToast(error.running);
|
||||
if (error?.status === 429) return REVIEWS_FULL;
|
||||
if (error?.refused === true && typeof error.message === 'string' && error.message) {
|
||||
if (error.code === 'invalid-backup') return `This file can't be restored: ${error.message.replace(/\.?$/, '.')}`;
|
||||
return error.message;
|
||||
}
|
||||
return REVIEW_FAILED;
|
||||
}
|
||||
|
||||
/** Stage `file` for review; the server's answer, or an ApiError naming its refusal. */
|
||||
export async function uploadRestore(file, { fetchUrl = window.ApiClient.fetchUrl, ApiError = window.ApiClient.ApiError,
|
||||
signal } = {}) {
|
||||
const form = new FormData();
|
||||
form.append('database_file', file, file.name);
|
||||
const response = await fetchUrl(PREVIEW_URL, { method: 'POST', body: form, cache: 'no-store', credentials: 'same-origin',
|
||||
signal });
|
||||
let data = null;
|
||||
try {
|
||||
data = await response.json();
|
||||
} catch {
|
||||
data = null;
|
||||
}
|
||||
const message = data && typeof data === 'object' ? data.error : undefined;
|
||||
if (!response.ok || !data || data.success === false || typeof data.token !== 'string') {
|
||||
throw new ApiError(response.status, message || `HTTP ${response.status}`, data?.code, null, data);
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
/** Discard a staged review; a failure only leaves it to expire on the server. */
|
||||
export function discardRestore(token, { fetchUrl = window.ApiClient.fetchUrl } = {}) {
|
||||
return fetchUrl(`${PREVIEW_URL}/${encodeURIComponent(token)}`, { method: 'DELETE', keepalive: true, credentials: 'same-origin' })
|
||||
.then(() => {}, () => {});
|
||||
}
|
||||
|
||||
/**
|
||||
* I1: what to do with a reviewed token once Restore was pressed: `follow` the started job (the page keeps the review
|
||||
* until it ends, in case the job hands it back), `reopen` the review after a busy refusal (the server keeps the token:
|
||||
* Restore retries, Discard releases), and `release` it on every other path (a refusal that dropped it, a used one, a
|
||||
* lost answer): a DELETE of a token the server no longer holds, or a job took, changes nothing.
|
||||
*/
|
||||
export function afterApply(outcome) {
|
||||
if (outcome?.job) return 'follow';
|
||||
if (outcome?.refused?.code === 'busy') return 'reopen';
|
||||
return 'release';
|
||||
}
|
||||
|
||||
// I1: the reviews whose restore job is running, by job id, so a job that hands its review back reopens it here.
|
||||
const startedReviews = new Map();
|
||||
|
||||
/** Keep `review` (`{token, preview, file}`) until the restore job `jobId` ends. */
|
||||
export function rememberRestore(jobId, review) {
|
||||
startedReviews.set(jobId, review);
|
||||
}
|
||||
|
||||
/**
|
||||
* Call `onKept(review)` when a restore job this page started fails and the server kept its review
|
||||
* (`scope.review_kept`, the leases did not drain); any other end forgets the review (the server discarded or used its
|
||||
* stage). Returns the unsubscribe.
|
||||
*/
|
||||
export function onRestoreHandedBack(activity, onKept) {
|
||||
return activity.subscribe(() => {
|
||||
for (const job of activity.jobs()) {
|
||||
if (job.kind !== RESTORE_KIND || job.state === 'running' || !startedReviews.has(job.id)) continue;
|
||||
const review = startedReviews.get(job.id);
|
||||
startedReviews.delete(job.id);
|
||||
if (job.state === 'failed' && job.scope?.review_kept === true) onKept(review);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* D105: "Restore preview" for a staged `preview` of `file`. Leaving it any way but Restore discards the stage; after a
|
||||
* refusal or a handed-back job (`again`), Cancel reads "Discard". Restore applies exactly this token and acts on the
|
||||
* outcome (`afterApply`).
|
||||
*/
|
||||
export function showRestoreReview({ api, store, scope, preview, file, trigger = null, onBackup = () => downloadDatabaseBackup(),
|
||||
again = false, reviewAgain = null, notify = showWorkspaceToast, navigate = navigateTo, fetchUrl = window.ApiClient.fetchUrl }) {
|
||||
const token = preview.token;
|
||||
const show = (retry, shownScope = scope) => showRestoreReview({ api, store, scope: shownScope, preview, file, trigger,
|
||||
onBackup, again: retry, reviewAgain, notify, navigate, fetchUrl });
|
||||
openReviewDialog({ scope, trigger, title: RESTORE_TITLE, grid: restoreGrid(preview, file),
|
||||
sentence: restoreNote(preview), backup: { copy: RESTORE_BACKUP_COPY, onBackup },
|
||||
cancelLabel: again ? 'Discard' : 'Cancel', primary: { label: RESTORE_LABEL, danger: true },
|
||||
onDismiss: () => discardRestore(token, { fetchUrl }),
|
||||
onStart: async () => {
|
||||
const outcome = await applyPlan({ api, store, scope, token, reviewAgain, notify, navigate, url: APPLY_URL });
|
||||
const next = afterApply(outcome);
|
||||
if (next === 'follow') rememberRestore(outcome.job.id, { token, preview, file });
|
||||
else if (next === 'reopen' && !scope.signal.aborted) show(true);
|
||||
else discardRestore(token, { fetchUrl });
|
||||
} });
|
||||
}
|
||||
|
||||
/**
|
||||
* Stage `file`, then show "Restore preview"; Restore applies exactly that token. A review the server asks to redo (it
|
||||
* expired, or the library changed) stages the same file again.
|
||||
*/
|
||||
export async function reviewRestore({ api, store, scope, file, trigger = null, hold = trigger, onBackup,
|
||||
notify = showWorkspaceToast, navigate = navigateTo, fetchUrl = window.ApiClient.fetchUrl }) {
|
||||
await holding(hold, 'Reviewing…', async () => {
|
||||
let preview;
|
||||
try {
|
||||
preview = await uploadRestore(file, { fetchUrl, signal: scope.signal });
|
||||
} catch (error) {
|
||||
if (!scope.signal.aborted) notify(reviewRefusal(error));
|
||||
return;
|
||||
}
|
||||
if (scope.signal.aborted) {
|
||||
discardRestore(preview.token, { fetchUrl });
|
||||
return;
|
||||
}
|
||||
const reviewAgain = () => reviewRestore({ api, store, scope, file, trigger, onBackup, notify, navigate, fetchUrl });
|
||||
showRestoreReview({ api, store, scope, preview, file, trigger, onBackup, reviewAgain, notify, navigate, fetchUrl });
|
||||
});
|
||||
}
|
||||
|
||||
/** The file name a download's Content-Disposition names, or `fallback`. */
|
||||
function filenameOf(response, fallback) {
|
||||
const match = /filename="?([^";]+)"?/i.exec(response.headers.get('Content-Disposition') || '');
|
||||
return match ? match[1] : fallback;
|
||||
}
|
||||
|
||||
/** Download one export through `fetchUrl`; toasts a failure, and nothing is saved then. */
|
||||
export async function downloadExport([, url, fallback], { fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast,
|
||||
signal } = {}) {
|
||||
try {
|
||||
const response = await fetchUrl(url, { cache: 'no-store', credentials: 'same-origin', signal });
|
||||
if (!response.ok) throw new Error(`Export failed with ${response.status}`);
|
||||
const blob = await response.blob();
|
||||
if (signal?.aborted) return false;
|
||||
saveResponse(blob, filenameOf(response, fallback));
|
||||
return true;
|
||||
} catch {
|
||||
if (!signal?.aborted) notify(EXPORT_FAILED);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `onFinished()` once each time a restore this page saw running ends finished (the library was replaced). A
|
||||
* failed restore changed nothing. Returns the unsubscribe.
|
||||
*/
|
||||
export function onRestoreFinished(activity, onFinished) {
|
||||
const running = new Set();
|
||||
return activity.subscribe(() => {
|
||||
let finished = false;
|
||||
for (const job of activity.jobs()) {
|
||||
if (job.kind !== RESTORE_KIND) continue;
|
||||
if (job.state === 'running') running.add(job.id);
|
||||
else if (running.delete(job.id) && job.state === 'finished') finished = true;
|
||||
}
|
||||
if (finished) onFinished();
|
||||
});
|
||||
}
|
||||
|
||||
function surface(heading, ...nodes) {
|
||||
const section = el('section', undefined, 'surface');
|
||||
section.append(heading, ...nodes);
|
||||
return section;
|
||||
}
|
||||
|
||||
/** The Backup & export section's mount (`mountSystem`'s `sections.backups`). */
|
||||
export function mountBackups(container, { api, activity, scope, drafts, notify = showWorkspaceToast, navigate = navigateTo,
|
||||
fetchUrl = window.ApiClient.fetchUrl, latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) {
|
||||
const statusRead = latestOnly({ lifetime: scope.signal });
|
||||
const status = el('p', STATUS_LOADING, 'small');
|
||||
status.id = 'backup-status';
|
||||
status.setAttribute('role', 'status');
|
||||
async function loadStatus() {
|
||||
let answer;
|
||||
try {
|
||||
answer = await statusRead(signal => api.get(STATUS_URL, { signal, cache: 'no-store' }));
|
||||
} catch {
|
||||
answer = null;
|
||||
}
|
||||
if (answer === STALE || scope.signal.aborted) return;
|
||||
status.textContent = backupStatusText(answer);
|
||||
}
|
||||
const backupNow = async () => {
|
||||
const done = await downloadDatabaseBackup({ fetchUrl, notify });
|
||||
if (done && !scope.signal.aborted) loadStatus();
|
||||
return done;
|
||||
};
|
||||
|
||||
// Database backup: Back up, Review restore… and the status line.
|
||||
const backup = createButton('Back up', 'ca-primary');
|
||||
const restore = createButton('Review restore…', 'danger');
|
||||
const picker = el('input');
|
||||
picker.type = 'file';
|
||||
// M3: no type filter: the safety backups (`prompts.db.backup_<time>`) have no `.db` extension; the server validates.
|
||||
picker.hidden = true;
|
||||
picker.id = 'restore-file';
|
||||
picker.setAttribute('aria-label', 'Backup file to restore');
|
||||
// Outside the concept's surface, for as long as the section is shown.
|
||||
document.body.append(picker);
|
||||
scope.cleanup(() => picker.remove());
|
||||
scope.listen(backup, 'click', () => holding(backup, 'Backing up…', backupNow));
|
||||
scope.listen(restore, 'click', () => { if (restore.getAttribute('aria-disabled') !== 'true') picker.click(); });
|
||||
scope.listen(picker, 'change', () => {
|
||||
const [file] = picker.files ?? [];
|
||||
picker.value = '';
|
||||
if (!file) return;
|
||||
reviewRestore({ api, store: activity, scope, file, trigger: restore, onBackup: backupNow, notify, navigate, fetchUrl });
|
||||
});
|
||||
const backupActions = el('div', undefined, 'row-actions spaced');
|
||||
backupActions.append(backup, restore);
|
||||
const database = surface(el('h2', 'Database backup'),
|
||||
el('p', 'A complete recovery snapshot, including prompt pairs, metadata and image links. Media files are separate.'),
|
||||
backupActions, status);
|
||||
|
||||
// Portable export: Export JSON and Import / inspect file, then the two secondary exports (D103).
|
||||
const exportButtons = EXPORTS.map(entry => {
|
||||
const button = createButton(entry[0]);
|
||||
scope.listen(button, 'click', () => holding(button, 'Exporting…',
|
||||
() => downloadExport(entry, { fetchUrl, notify, signal: scope.signal })));
|
||||
return button;
|
||||
});
|
||||
const importButton = createButton('Import / inspect file');
|
||||
let rescanning = false;
|
||||
let importDialog = null;
|
||||
async function rescan() {
|
||||
if (rescanning || scope.signal.aborted) return;
|
||||
rescanning = true;
|
||||
notify(RESCAN_STARTED);
|
||||
try {
|
||||
const result = await api.post(RESCAN_URL, {}, { signal: scope.signal });
|
||||
if (!scope.signal.aborted) notify(rescanFinished(result));
|
||||
} catch {
|
||||
if (!scope.signal.aborted) notify(RESCAN_FAILED);
|
||||
} finally {
|
||||
rescanning = false;
|
||||
importDialog?.setRescanBusy(false);
|
||||
}
|
||||
}
|
||||
async function metadataScan(trigger) {
|
||||
let settings;
|
||||
try {
|
||||
settings = await api.get(SETTINGS_URL, { signal: scope.signal, cache: 'no-store' });
|
||||
} catch {
|
||||
if (!scope.signal.aborted) notify(SOURCES_FAILED);
|
||||
return;
|
||||
}
|
||||
if (scope.signal.aborted) return;
|
||||
const sources = Array.isArray(settings?.sources?.effective) ? settings.sources.effective.length : 0;
|
||||
openMetadataScan({ scope, store: activity, sources, trigger });
|
||||
}
|
||||
scope.listen(importButton, 'click', () => {
|
||||
importDialog = openImportDialog({ scope, trigger: importButton, rescanBusy: rescanning,
|
||||
onInspect: () => navigate('#/inspect'), onRescan: rescan, onMetadataScan: metadataScan });
|
||||
});
|
||||
const exportActions = el('div', undefined, 'row-actions spaced');
|
||||
exportActions.append(exportButtons[0], importButton);
|
||||
const moreExports = el('div', undefined, 'row-actions spaced');
|
||||
moreExports.append(...exportButtons.slice(1));
|
||||
const portable = surface(el('h2', 'Portable export'),
|
||||
el('p', 'Export reusable recipes and tags as versioned JSON. This is distinct from a complete database backup.'),
|
||||
exportActions, moreExports);
|
||||
const columns = el('div', undefined, 'two-col');
|
||||
columns.append(database, portable);
|
||||
|
||||
// Recovery policy (D104): the real safety-backup policy and the schedule the server does not have yet.
|
||||
const heading = el('h2', 'Recovery policy ');
|
||||
heading.append(el('span', 'Proposed', 'chip blue'));
|
||||
const safety = settingRow({ id: 'backup-before', label: 'Create a safety backup before maintenance',
|
||||
description: SAFETY_DESCRIPTION,
|
||||
control: { type: 'toggle' }, scope, onChange: value => drafts.set('host', 'safety_backup', value) });
|
||||
const safetyInput = safety.querySelector('#backup-before');
|
||||
safetyInput.dataset.key = 'safety_backup';
|
||||
const schedule = settingRow({ id: 'backup-schedule', label: 'Scheduled database backups', description: SCHEDULE_REASON,
|
||||
control: { type: 'select', options: ['Manual only', 'Daily', 'Weekly'], value: 'Manual only' } });
|
||||
for (const option of schedule.querySelectorAll('option')) option.disabled = option.value !== 'Manual only';
|
||||
const policy = surface(heading, safety, schedule, el('p', WARNING_NOTE, 'warning-note'));
|
||||
let hostNote = null;
|
||||
function paintPolicy() {
|
||||
const value = drafts.get('host', 'safety_backup');
|
||||
safetyInput.disabled = typeof value !== 'boolean';
|
||||
if (typeof value === 'boolean') safetyInput.checked = value;
|
||||
const unavailable = drafts.hostState?.() === 'failed' && typeof value !== 'boolean';
|
||||
if (unavailable && !hostNote) {
|
||||
hostNote = el('p', HOST_UNAVAILABLE, 'small');
|
||||
safety.children[0].append(hostNote);
|
||||
} else if (!unavailable && hostNote) {
|
||||
hostNote.remove();
|
||||
hostNote = null;
|
||||
}
|
||||
}
|
||||
scope.cleanup(drafts.subscribe(paintPolicy));
|
||||
paintPolicy();
|
||||
loadStatus();
|
||||
return [columns, policy];
|
||||
}
|
||||
@@ -7,6 +7,7 @@
|
||||
* N issues / Incomplete); the review, backup, thumbnail and sources rows say what the server reports, and nothing
|
||||
* reads 0 before the read answers: a load shows a status line, a failure an alert and Retry. The thumbnail row is
|
||||
* its own read of `/images/thumbnails/status` (a walk of every source): "Checking thumbnails…" until it answers.
|
||||
* S9: the Backup status row is its own read of `/backup/status`, worded as Backup & export's status line.
|
||||
* D99: its Review thumbnails opens the same "Thumbnail maintenance" dialog as Sources (thumbnail-review.mjs).
|
||||
* #317: it reads "N missing · M outdated · K failed"; while previews are outdated (another format than the
|
||||
* configured one) a second button, "Convert outdated", opens that dialog on its Convert outdated scope (D131).
|
||||
@@ -25,6 +26,7 @@ import { noun } from './plural.mjs';
|
||||
import { operationRow } from './system-controls.mjs';
|
||||
import { openThumbnailReview } from './thumbnail-review.mjs';
|
||||
import { isLocation, locationRow, onMoveEnded } from './library-location.mjs';
|
||||
import { STATUS_LOADING, STATUS_URL, backupStatusText } from './system-backups.mjs';
|
||||
|
||||
export const OVERVIEW_URL = '/prompt_manager/system/overview';
|
||||
export const STATISTICS_URL = '/prompt_manager/system/statistics';
|
||||
@@ -75,12 +77,6 @@ export function ambiguousTitle(needsReview) {
|
||||
return `${number(pending)} ambiguous image ${noun(pending, 'link')}`;
|
||||
}
|
||||
|
||||
/** D93: the backup row; only a recorded, verified backup is ever named. */
|
||||
export function backupText(backup) {
|
||||
if (!backup) return 'No retained backup record.';
|
||||
const when = new Date(backup.verified_at ?? backup.created_at);
|
||||
return Number.isNaN(when.getTime()) ? 'No retained backup record.' : `Last verified backup ${when.toLocaleString()}.`;
|
||||
}
|
||||
|
||||
/**
|
||||
* D93: the thumbnail row's title from the `/images/thumbnails/status` answer: undefined while it is read, null when
|
||||
@@ -366,10 +362,12 @@ function overviewNodes(overview, { api, activity, notify, scope, navigate, openS
|
||||
const [sourcesTitle, sourcesDescription] = sourcesCopy(overview.sources);
|
||||
const thumbnails = operationRow({ scope, title: THUMBNAILS_LOADING, description: 'Generate only missing previews when necessary.',
|
||||
label: 'Review thumbnails', onClick: openThumbnails });
|
||||
const backupRow = operationRow({ scope, title: 'Backup status', description: STATUS_LOADING, label: 'View backups',
|
||||
onClick: go('backups') });
|
||||
const health = surface('Library health', [
|
||||
review,
|
||||
operationRow({ scope, title: 'Backup status', description: backupText(overview.backup), label: 'View backups',
|
||||
onClick: go('backups') }),
|
||||
// S9: what GET /backup/status reports, as Backup & export's status line says it (its own read).
|
||||
backupRow,
|
||||
// D129: where the library lives and why; Move library when it is in the node folder.
|
||||
...(isLocation(overview.location)
|
||||
? [locationRow(overview.location, { api, activity, scope, notify, navigate })] : []),
|
||||
@@ -385,7 +383,7 @@ function overviewNodes(overview, { api, activity, notify, scope, navigate, openS
|
||||
]);
|
||||
const columns = el('div', undefined, 'two-col');
|
||||
columns.append(health, preferences);
|
||||
return { nodes: [metricsNode(overview), columns], thumbnails };
|
||||
return { nodes: [metricsNode(overview), columns], thumbnails, backup: backupRow };
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -398,7 +396,14 @@ export function mountOverview(container, { api, activity, scope, navigate = navi
|
||||
const overviewRead = latestOnly({ lifetime: scope.signal });
|
||||
const statisticsRead = latestOnly({ lifetime: scope.signal });
|
||||
const thumbnailsRead = latestOnly({ lifetime: scope.signal });
|
||||
const backupRead = latestOnly({ lifetime: scope.signal });
|
||||
let shown = [];
|
||||
// The backup status (undefined while read, null when it failed), painted into its row once both are there (S9).
|
||||
let backupStatus;
|
||||
let backupRow = null;
|
||||
const paintBackup = () => {
|
||||
if (backupRow) backupRow.querySelector('p').textContent = backupStatusText(backupStatus);
|
||||
};
|
||||
// The thumbnail status (undefined while read, null when it failed), painted into the row whenever both are there.
|
||||
let thumbnailStatus;
|
||||
let thumbnailRow = null;
|
||||
@@ -438,12 +443,14 @@ export function mountOverview(container, { api, activity, scope, navigate = navi
|
||||
try {
|
||||
const overview = await overviewRead(signal => api.get(OVERVIEW_URL, { signal, cache: 'no-store' }));
|
||||
if (overview === STALE || scope.signal.aborted) return;
|
||||
const { nodes, thumbnails } = overviewNodes(overview, { api, activity, notify, scope, navigate, openStatistics,
|
||||
const { nodes, thumbnails, backup } = overviewNodes(overview, { api, activity, notify, scope, navigate, openStatistics,
|
||||
openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate }) });
|
||||
thumbnailRow = thumbnails;
|
||||
backupRow = backup;
|
||||
sources = overview.sources;
|
||||
swap(nodes);
|
||||
paintThumbnails();
|
||||
paintBackup();
|
||||
} catch (error) {
|
||||
if (scope.signal.aborted) return;
|
||||
const alert = el('p', LOAD_FAILED_TEXT);
|
||||
@@ -453,6 +460,7 @@ export function mountOverview(container, { api, activity, scope, navigate = navi
|
||||
swap(loading());
|
||||
load();
|
||||
loadThumbnails();
|
||||
loadBackup();
|
||||
});
|
||||
const panel = el('section', undefined, 'surface');
|
||||
panel.append(alert, retry);
|
||||
@@ -470,10 +478,22 @@ export function mountOverview(container, { api, activity, scope, navigate = navi
|
||||
thumbnailStatus = status ?? null;
|
||||
paintThumbnails();
|
||||
}
|
||||
async function loadBackup() {
|
||||
let status;
|
||||
try {
|
||||
status = await backupRead(signal => api.get(STATUS_URL, { signal, cache: 'no-store' }));
|
||||
} catch (error) {
|
||||
status = null;
|
||||
}
|
||||
if (status === STALE || scope.signal.aborted) return;
|
||||
backupStatus = status ?? null;
|
||||
paintBackup();
|
||||
}
|
||||
const first = swap(loading());
|
||||
// D129: a finished Move library changes where the library lives.
|
||||
scope.cleanup(onMoveEnded(activity, load));
|
||||
load();
|
||||
loadThumbnails();
|
||||
loadBackup();
|
||||
return first;
|
||||
}
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
* (`span.small#settings-status` + "Save preferences"). D92: no "Sample configuration" chip, the status copy is
|
||||
* truthful and "Save preferences" is aria-disabled while nothing is unsaved. A section not delivered yet shows the
|
||||
* per-section hand-off to its legacy tool (D94), never mock placeholders. Overview (system-overview.mjs),
|
||||
* Appearance (system-appearance.mjs), Sources & scans (system-sources.mjs), Integrations (system-integrations.mjs) and
|
||||
* Maintenance (system-maintenance.mjs) are native.
|
||||
* Appearance (system-appearance.mjs), Sources & scans (system-sources.mjs), Integrations (system-integrations.mjs),
|
||||
* Maintenance (system-maintenance.mjs) and Backup & export (system-backups.mjs) are native.
|
||||
*
|
||||
* Switching sections keeps the shell (and focus on the pressed button) and disposes the old section's scope, so its
|
||||
* listeners and timers never outlive it. The drafts are module-level (Q10): they live for the page session.
|
||||
@@ -19,6 +19,7 @@ import { mountOverview } from './system-overview.mjs';
|
||||
import { mountAppearance } from './system-appearance.mjs';
|
||||
import { mountSources } from './system-sources.mjs';
|
||||
import { mountMaintenance } from './system-maintenance.mjs';
|
||||
import { mountBackups } from './system-backups.mjs';
|
||||
import { LORA_ENABLE_URL, LORA_STATUS_URL, loraValues, mountIntegrations, writeHostInParts } from './system-integrations.mjs';
|
||||
import { busyToast } from './activity-scan.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
@@ -33,7 +34,6 @@ const ADMIN = Object.freeze({ href: '/prompt_manager/admin', label: 'Open existi
|
||||
*/
|
||||
export const SECTION_HANDOFFS = Object.freeze({
|
||||
tagging: [ADMIN],
|
||||
backups: [ADMIN],
|
||||
diagnostics: [ADMIN],
|
||||
});
|
||||
export const NOT_DELIVERED = 'This setting is not available here yet.';
|
||||
@@ -128,7 +128,7 @@ function markFailed(layout, store) {
|
||||
|
||||
/**
|
||||
* The sections delivered so far: Overview (S2 Task 6), Appearance (S3 Task 8), Sources & scans (S4 Task 11),
|
||||
* Integrations (S10 Task 27) and Maintenance (S7 Task 19).
|
||||
* Integrations (S10 Task 27), Maintenance (S7 Task 19) and Backup & export (S9 Task 24).
|
||||
*/
|
||||
export const DELIVERED_SECTIONS = Object.freeze({
|
||||
overview: mountOverview,
|
||||
@@ -136,6 +136,7 @@ export const DELIVERED_SECTIONS = Object.freeze({
|
||||
sources: mountSources,
|
||||
integrations: mountIntegrations,
|
||||
maintenance: mountMaintenance,
|
||||
backups: mountBackups,
|
||||
});
|
||||
|
||||
/**
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
* Every opener takes the page `scope`: disposing it closes and removes the shell, so a dialog never outlives its page.
|
||||
*/
|
||||
import { createButton, createSwitch } from '../shared/toolkit/core.js';
|
||||
import { noun } from './plural.mjs';
|
||||
|
||||
const FIELD_LABELS = Object.freeze({ source: 'Source', folder: 'Folder', media: 'Media', tags: 'Tags' });
|
||||
const FIELD_ORDER = Object.keys(FIELD_LABELS);
|
||||
@@ -255,6 +256,17 @@ function operation(title, description, label, onAction) {
|
||||
|
||||
const RESCAN_LABEL = 'Rescan';
|
||||
const RESCAN_BUSY_LABEL = 'Rescanning…';
|
||||
// Import → Rescan (D15): the concept has no rescan copy, so its toasts follow the workspace pattern. The answer counts
|
||||
// every listed file across the output roots it walked. Images and System › Backup & export (S9) share them.
|
||||
export const RESCAN_STARTED = 'Rescanning output folders…';
|
||||
export const RESCAN_FAILED = 'Rescan could not finish. Try again.';
|
||||
/** The toast after a rescan answered `{total, roots}`. */
|
||||
export function rescanFinished(result) {
|
||||
const { total, roots } = result ?? {};
|
||||
if (![total, roots].every(value => Number.isSafeInteger(value) && value >= 0)) return 'Rescan finished.';
|
||||
return `Rescan finished. ${total.toLocaleString('en-US')} ${noun(total, 'file')} in ` +
|
||||
`${roots.toLocaleString('en-US')} output ${noun(roots, 'folder')}.`;
|
||||
}
|
||||
|
||||
/**
|
||||
* "Import & inspect" with the real operations (D15). While `rescanBusy`, Rescan reads "Rescanning…" and is
|
||||
@@ -415,12 +427,14 @@ function requireGrid(grid) {
|
||||
* `h3` heading, the `dl.meta-grid` of `[term, value]` rows, page-built `options` nodes, the optional `p.small`
|
||||
* sentence, the D55 backup box when `backup` is `{copy, onBackup}`, then `.dialog-actions` with `cancelLabel` and the
|
||||
* primary (`ca-primary`, or the concept's `danger` when `primary.danger`). The primary closes the dialog, then calls
|
||||
* `onStart()`. Returns `{setValue(term, text)}` for a grid value that follows an option.
|
||||
* `onStart()`; `onDismiss()` runs once when the dialog closes any other way (Cancel, ×, Escape, a replacing dialog, the
|
||||
* page leaving), so a staged review is discarded (S9). Returns `{setValue(term, text)}` for a grid value that follows
|
||||
* an option.
|
||||
*/
|
||||
export function openReviewDialog({ scope, title, heading = null, grid, options = [], sentence = null, backup = null,
|
||||
cancelLabel, primary, onStart, trigger = null }) {
|
||||
cancelLabel, primary, onStart, trigger = null, onDismiss = null }) {
|
||||
requireScope(scope);
|
||||
requireCallbacks({ onStart });
|
||||
requireCallbacks({ onStart, ...(onDismiss ? { onDismiss } : {}) });
|
||||
for (const [name, text] of Object.entries({ title, cancelLabel, label: primary?.label })) {
|
||||
if (typeof text !== 'string' || !text) throw new TypeError(`The review needs its ${name}`);
|
||||
}
|
||||
@@ -444,15 +458,18 @@ export function openReviewDialog({ scope, title, heading = null, grid, options =
|
||||
if (backup) nodes.push(backupNotice(backup));
|
||||
const cancel = createButton(cancelLabel);
|
||||
const start = createButton(primary.label, primary.danger ? 'danger' : 'ca-primary');
|
||||
let started = false;
|
||||
cancel.addEventListener('click', () => finish());
|
||||
start.addEventListener('click', () => {
|
||||
started = true;
|
||||
finish();
|
||||
onStart();
|
||||
});
|
||||
const actions = el('div', undefined, 'dialog-actions');
|
||||
actions.append(cancel, start);
|
||||
nodes.push(actions);
|
||||
show(title, nodes, { scope, trigger });
|
||||
// Cancel, ×, Escape, a replacing dialog or the page leaving: everything but the primary is a dismissal.
|
||||
show(title, nodes, { scope, trigger, onClose: onDismiss ? () => { if (!started) onDismiss(); } : null });
|
||||
return {
|
||||
setValue(term, text) {
|
||||
const dd = values.get(term);
|
||||
|
||||
@@ -423,6 +423,9 @@ pre.prompt-text { font: inherit; font-size: 12px; line-height: 1.75; margin-bloc
|
||||
/* D101: a duplicate copy's Keep label is its source and path, often long names with no space to break at; it wraps
|
||||
inside its `.compare-item` so the review never scrolls sideways. */
|
||||
.duplicate-groups .compare-item label { overflow-wrap: anywhere; /* D101 */ }
|
||||
/* D105: a review's grid values (the restore's backup file name, a safety backup's name) may be long names with no space
|
||||
to break at; they wrap inside their column, so the concept's two equal columns hold at phone width. */
|
||||
#modal-body > dl.meta-grid dd { overflow-wrap: anywhere; /* D105 */ }
|
||||
/* D106: Integrations' two selects keep their chosen option readable: they never shrink for the row's reason lines, and
|
||||
at phone width they may grow past the concept's 120px cap (which cuts "Follow host renderer") up to 160px. */
|
||||
.setting-row :is(#send-destination, #renderer) { flex-shrink: 0; /* D106 */ }
|
||||
|
||||
Reference in New Issue
Block a user