From c86739178d89b26d578b2a31502ba306d7a49ecf Mon Sep 17 00:00:00 2001 From: Vito Date: Fri, 23 Oct 2026 04:59:18 -0700 Subject: [PATCH 1/7] feat(system): System page shell, review dialog and Overview (#298) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(system): read-only health check and truthful overview counts POST /prompt_manager/system/health runs the health check off-loop as a read registry job of kind "health" (one at a time, beside a scan, not cancellable, no file outcomes) and keeps the last report; GET returns it or null. The check reads one snapshot on its own mode=ro connection, closed in finally: integrity_check(100), foreign_key_check, stored hashes against utils.hashing, duplicate pair rows, the legacy consistency queries (now shared with check_consistency), and image links whose file is gone, counted only under readable sources. An unavailable source, or the absent Needs-review queue, makes its check unavailable and the report incomplete instead of producing a count. GET /prompt_manager/system/overview returns Library-eligible positive prompts and active negative variants, indexed images, the last health summary or null, backup null, the Needs-review count null, the effective sources with whether they were auto-detected, and the thumbnail plan's missing count. GET /prompt_manager/system/statistics lists runs and average rating per category of Library parents, with the /stats totals. * feat(system): native System route, drafts and shared review dialog - #/system/
renders the concept's System shell: section nav, settings layout and Save row with truthful status (D92); each section not delivered yet hands off to its legacy tools (D94). Overview's System card opens #/system/overview (Q12). - A module-level draft store saves browser, then host (expected_revision), then runtime; failed domains stay dirty, a stale revision is a conflict. - System row builders (setting, operation, section nav) on the scope. - openReviewDialog and backupNotice extracted from the scan review, which is now a thin caller with identical markup. - Overview images gain a "rated" count (images whose linked prompt is rated); the missing-files health check reports per source, so partial availability is visible. * feat(system): native Overview with health report and statistics System > Overview renders the concept's metrics and its Library health and Workspace preferences surfaces from GET /system/overview: real counts, "Not checked" until a health check ran, no invented review count, backup or thumbnail status (D93). Explore opens "Usage statistics" with the real runs and ratings per category and the Library totals (D96); the statistics route now also returns rated_images. Maintenance shows the concept's notice with Run health checks above its hand-off: the read-only check runs as a registry read job and opens the "Health report" with a chip per real result, missing files per source (D95). A load failure shows Retry and no zeros; late answers never repaint a newer section. Adds the System concept comparison (shell, Overview, both dialogs, the notice, mutation checks), the System workflow spec and its fixture hooks, and the TESTING.md "Native System" section. * test(system): coverage floors for the native Overview Adds the system-overview.mjs baseline and raises the system.mjs floors to their measured coverage; no other floor changes. * fix(system): keep health reports per database and off the event loop - A restore that replaces the database forgets the stored health report, and a check that started before the swap is not kept (database generation stamped at start). - The Overview no longer walks every source for the missing-thumbnail count; the page reads /images/thumbnails/status on its own. - Image sources for the health check and the Overview resolve in the executor; a failure to start the check answers the coded envelope. - Any exception fails only its own check; missing files reads its rows in the snapshot and stats the files after it ended. - The read-only URI keeps Windows UNC paths (empty authority). - An integrity count at SQLite's limit is flagged as capped. * fix(system): thumbnail row read, tracked health job and guarded host save - The Overview's thumbnail row is its own read of the thumbnail status: "Checking thumbnails…" until it answers, unavailable when it fails. - The health job is tracked in the Activity store (new store.track) and followed through its event stream instead of polling every 150 ms. - A host save before a read returned a revision fails without a request; System reads the host values on mount and after a conflict. - "100+ problems found" when the integrity count is capped. - D-NEW-S2-5 is D123 everywhere; the unused Overview hand-off is gone. - The concept spec builds the health report from the concept's own rows and asserts the full unmapped path list; surface tops pin the 18px offset. - TESTING.md: D96's reason, the read-only (WAL/SHM) standard, the UNC URI and the new behavior. Coverage floors raised for the touched files. * fix(system): make two health tests portable to Windows - Windows forbids "?" in file names, so the URI-character open test uses "#", "%" and a space there. - ntpath.realpath on Python 3.11+ returns a NUL-byte path instead of raising, so the bad-path test raises as POSIX does to reach the missing-files failure path on every platform. --- TESTING.md | 136 ++++ database/health.py | 354 +++++++++++ database/library.py | 51 ++ database/operations.py | 34 +- py/api/__init__.py | 27 +- py/api/admin.py | 1 + py/api/jobs.py | 3 + py/api/system.py | 207 +++++++ tests/browser/server.py | 2 + tests/browser/system-concept.spec.mjs | 410 +++++++++++++ tests/browser/workspace-system.spec.mjs | 252 ++++++++ tests/browser/workspace.spec.mjs | 97 ++- tests/browser/workspace_system_fixture.py | 116 ++++ tests/js/workspace-activity-store.test.js | 18 + tests/js/workspace-state.test.js | 46 +- tests/js/workspace-system-controls.test.js | 214 +++++++ tests/js/workspace-system-overview.test.js | 255 ++++++++ tests/js/workspace-system-state.test.js | 261 ++++++++ tests/js/workspace-toolbar.test.js | 143 +++++ tests/test_database_health.py | 614 +++++++++++++++++++ tests/test_library.py | 97 +++ tests/test_system_api.py | 422 +++++++++++++ tools/vendor/frontend-coverage-baseline.json | 24 + web/js/workspace/activity-store.mjs | 10 + web/js/workspace/index.mjs | 29 +- web/js/workspace/state.mjs | 34 +- web/js/workspace/system-controls.mjs | 132 ++++ web/js/workspace/system-overview.mjs | 432 +++++++++++++ web/js/workspace/system-state.mjs | 201 ++++++ web/js/workspace/system.mjs | 148 +++++ web/js/workspace/workspace-dialogs.mjs | 143 +++-- web/js/workspace/workspace.css | 9 + 32 files changed, 4831 insertions(+), 91 deletions(-) create mode 100644 database/health.py create mode 100644 py/api/system.py create mode 100644 tests/browser/system-concept.spec.mjs create mode 100644 tests/browser/workspace-system.spec.mjs create mode 100644 tests/browser/workspace_system_fixture.py create mode 100644 tests/js/workspace-system-controls.test.js create mode 100644 tests/js/workspace-system-overview.test.js create mode 100644 tests/js/workspace-system-state.test.js create mode 100644 tests/test_database_health.py create mode 100644 tests/test_system_api.py create mode 100644 web/js/workspace/system-controls.mjs create mode 100644 web/js/workspace/system-overview.mjs create mode 100644 web/js/workspace/system-state.mjs create mode 100644 web/js/workspace/system.mjs diff --git a/TESTING.md b/TESTING.md index cab865a9..208e190c 100644 --- a/TESTING.md +++ b/TESTING.md @@ -1816,6 +1816,142 @@ npm run --prefix tools/vendor test:workflows -- inspect-concept.spec.mjs workspa npm run --prefix tools/vendor test:comfyui -- inspect-sender.spec.mjs ``` +### Native System (#249 System) + +`#/system/
` is a native workspace route rendered by `web/js/workspace/system.mjs` +on the concept's `renderSystem` markup (app.js a698–706; `concept.css` 709–895). `#/system` +is Overview under the sidebar's own hash; `#/system/overview|appearance|sources| +integrations|tagging|maintenance|backups|diagnostics` select a section, and an unknown +child (the concept's `#/system/general`) is replaced by `#/system/overview`. The +production hub `#/overview` is unchanged (D28); its System card opens +`#/system/overview` (Q12). Switching sections keeps the shell, the drafts and focus on +the pressed section button and disposes the previous section's scope, so its reads, +listeners and dialogs never outlive it. + +Drafts (`system-state.mjs`) live for the page session (Q10). Save writes the dirty +browser fields, then the host fields (`POST /prompt_manager/settings` with +`expected_revision`, #220), then the runtime fields. A failed domain keeps its drafts and +highlights their controls; `settings-changed` is a conflict and keeps the draft. System +reads the host values on mount and again after a conflict; until a read returned a +revision, a host save fails without a request, so it never sends a blind +`expected_revision: null` (which S1 merges into the file on disk). No delivered section +edits a setting yet, so Save stays disabled until Appearance (S3). + +Overview (`system-overview.mjs`) reads `GET /prompt_manager/system/overview` once per +visit through `latestOnly()` on the section scope; a late answer after the section left +paints nothing. Until it answers, the section shows "Loading the overview…"; a failure +shows "The overview could not be loaded. Your saved data has not changed." and Retry, +never zeros. The thumbnail row is a separate read of `/images/thumbnails/status` (a +walk of every source), so the other values never wait for it: "Checking thumbnails…" +until it answers. The health check (`POST /prompt_manager/system/health`) is a `read` +registry job of kind `health`: one at a time, beside a scan, not cancellable and with +no file outcomes. The page tracks the started job in the Activity store (its event +stream; the store marks it lost if the server stops listing it), never by polling. A press while one runs (another tab) waits +for that one. The last report lives for this server session (`GET +/prompt_manager/system/health`) and describes the database it was taken from: a +restore that replaces the database forgets it, and a check that started before the +swap is not kept. The check's own connection opens a `mode=ro` URI that keeps a Windows +UNC path (`file:////server/share/prompts.db`; pathlib's `file://server/share/...` is a +remote authority SQLite refuses). A check that raises fails only itself, and the +missing-files check reads its rows inside the snapshot but stats the files after it +ended, so a slow source never pins the WAL. Statistics +read `GET /prompt_manager/system/statistics` (`by_category`, the legacy `/stats` totals +and `rated_images`). + +Read-only standard for a check or preview (S2's health check; S6 and S8 previews use +the same): the database and `-wal` bytes are unchanged, the `-shm` size is unchanged, +the connection's `total_changes` is 0 at close and the data reads back unchanged. The +`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no +reader can keep it byte-identical (`test_health_check_never_writes`). + +Recorded deviations (plan D60–D64 renumbered D92–D96; D121, D122 and D123 assigned): + +- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No + unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server + configuration." / "Applied until the server restarts.", "Some changes were not saved. + Review the highlighted settings.", "Saved for this tab only; browser storage is + unavailable." and "Settings changed elsewhere. Reload this section to see them." + "Save preferences" is `aria-disabled` (opacity .55, `not-allowed`) while clean or + saving; a setting the last save could not save gets a red outline. CSS under D92. +- **D93** (Overview): real values. The metrics are the Library's eligible positives + and active saved negatives and the indexed `generated_images` rows (Q2). "Database + integrity" is "Not checked" until a check ran this session, then "N issues" (checks + that found issues, even beside a check that could not run), "Passed" or "Incomplete". + "Ambiguous image links" reads "Not available yet." with its Review metadata action + `aria-disabled` (CSS under D93) until the Needs review queue reports a count (Q6). + "Backup status" reads "No retained backup record." until verified backups exist (S8). + The thumbnail row is "Checking thumbnails…" until its own read answers, then "No + missing thumbnails", "N missing thumbnails" (the pending count of + `/images/thumbnails/status`) or "Thumbnail status unavailable"; Review thumbnails + opens Sources until the thumbnail dialog (S5). "One display configuration" says "Grid + and density settings are shared across views." (no paging). The sources row is + "N output sources" with "Monitoring and enrichment share one source list.", prefixed + by "M of N configured sources available." or "None configured is available; ComfyUI's + output folder is used." when that is the case. +- **D94** (all sections): a section not delivered yet shows the hand-off panel "Your + current tools are ready" with a link to each legacy tool that still owns it (the + plan's legacy coverage table); retired section by section, gone after S12. +- **D95** (health report): the concept's `health-check` dialog titled "Health report", + with no sample note. It lists every check in the server's order: Integrity check, + Foreign keys & pair references, Canonical hashes, Pair uniqueness (added), Missing + files, Ambiguous metadata, Consistency (added). Missing files has one row per image + source when there are several ("Missing files · output"). Each chip is the real + result: "Passed", a count with what was found ("2 broken references", "1 link needs + review", the concept's "N links need review" and "N images need review"; "100+ + problems found" when SQLite's integrity check stopped at its limit), "Not + available" (an unavailable source), "Not available yet" (Ambiguous metadata, Q6) or + "Could not check". A `p.small` after the list says when it was checked and whether + every check passed, some could not run, or issues need review. Close / Review missing + links (to Maintenance, its hand-off until S7). Nothing is changed by a check. +- **D96** (Statistics): "Usage statistics" without the sample note: the concept's + "Runs by category" table (Category / Runs / Avg. rating) with the real runs and + average rating of eligible parents per category ("Uncategorized", "Not rated"), + then a second table "Library totals" (Total / Count: Prompts, Tags, Images from the + legacy statistics, and Rated images: indexed images whose linked prompt is rated). + The Overview row says "Runs and ratings by category, with library totals." because + that is what the dialog shows (the concept's own dialog shows only "Runs by + category"); per-prompt run counts, prompt tags and dated runs are recorded, but no + table of them exists yet. A failed read toasts "Statistics could not + be loaded. Your saved data has not changed." +- **D121** (production hub): the Overview hub's System card reads the concept's System + heading and "Open System"; the concept has no hub card. +- **D122** (shell): the section nav's `aria-label` is "System sections"; the concept's + literal label is its prototype action name "system-tab sections". +- **D123** (Maintenance): the concept's Maintenance notice + ("Check first. Review changes. Then apply." + Run health checks) is delivered ahead of + the rest of Maintenance, above its D94 hand-off, as the native place that runs the + health check. A press shows "Checking…" (`aria-disabled`) and opens the report; a + failed check toasts "The health check could not finish. Nothing was changed." + +`tests/browser/system-concept.spec.mjs` compares the shell and Overview with the concept +(dark and light, 1600×1100 and 390×844), mapping only the deviations above; unmapped, +the difference is exactly the D92, D93, D96 and D122 paths. Each surface's `top` is +measured, which pins the concept's `.surface + .surface` 18px offset that concept.css +also applies inside `.two-col` (copied as is). `statistics and health dialogs match +the concept` maps D95 from the concept's own five rows (labels verbatim): it inserts +Pair uniqueness after Canonical hashes and Consistency last, splits Missing files per +source when there are several, replaces only the chips with the fixture's expected +results, and asserts the full unmapped path list; it maps D96 and compares the +Maintenance notice unchanged. `system comparison detects mutations` removes a row, +renames a metric label, swaps the `--line` token, recolours a chip, renames a check, +removes the totals table and renames a caption. Paired captures (`{concept,production}-system-{dark,light}-{1600,390}.png`, +`{concept,production}-{statistics,health}-{dark,light}.png`) are test artifacts; there +are no PNG baselines. `tests/browser/workspace-system.spec.mjs` covers the real counts, +the health check and metric 4, a failed check, the thumbnail row against the status +endpoint (painted after the rest when its read is held, unavailable when it fails), the review row (Q6), the section links, the Statistics dialog against the +route, a load failure with Retry and late answers after leaving. Its hooks are +`tests/browser/workspace_system_fixture.py` (`/__test__/system/*`: reset, injected +failures, an absent source, a deleted linked file, unthumbnailed files) on the +concept-shaped fixture, with delays through the shared request holds. + +```sh +python3 -m unittest tests.test_database_health tests.test_system_api tests.test_library -v +node --test tests/js/workspace-system-state.test.js tests/js/workspace-system-controls.test.js \ + tests/js/workspace-system-overview.test.js tests/js/workspace-state.test.js +npm run --prefix tools/vendor test:workflows -- system-concept.spec.mjs workspace-system.spec.mjs \ + --project chromium --project touch --project firefox --project webkit +``` + ## Workspace data fields for the concept pages (#249) These fields are additive. The legacy `/prompt_manager/images/output` listing (no diff --git a/database/health.py b/database/health.py new file mode 100644 index 00000000..7feedda3 --- /dev/null +++ b/database/health.py @@ -0,0 +1,354 @@ +"""The read-only health check behind System › Overview and Maintenance. + +``run_health_checks`` opens its own ``mode=ro`` connection, reads one +snapshot and closes the connection in ``finally``; it never writes. Each +check reports ``passed``, ``issues`` (with an exact ``count``; the +integrity check stops at ``INTEGRITY_LIMIT`` problems and then says +``capped``), +``unavailable`` (it could not be checked here, so ``count`` is None, or +``missing_files`` checked only some sources and found nothing) or ``failed`` +(it raised; the other checks still run). ``missing_files`` also reports each +source and whether the check was ``partial``; it reads the image rows inside +the snapshot and checks their files after the snapshot ended, so a slow source +never holds the WAL. At most ``MAX_DETAIL`` detail strings are returned per +check, and none of them carries a server path. +""" + +import datetime +import logging +import ntpath +import os +import sqlite3 +from pathlib import PurePosixPath, PureWindowsPath +from urllib.parse import quote + +try: + from ..utils.hashing import generate_prompt_hash + from .models import normalize_image_path +except ImportError: + from utils.hashing import generate_prompt_hash + from database.models import normalize_image_path + +logger = logging.getLogger("prompt_manager.database.health") + +MAX_DETAIL = 10 # detail strings returned per check +INTEGRITY_LIMIT = 100 # PRAGMA integrity_check(N): problems SQLite reports +BATCH_ROWS = 100 # image rows read per fetch +REVIEW_UNAVAILABLE = "Needs review is not available yet." +CHECK_FAILED = "The check could not finish." +# Check ids and the labels of the concept's health report, in its order; +# pair uniqueness and consistency are the added rows (D63, now D95). +CHECKS = ( + ("integrity", "Integrity check"), + ("foreign_keys", "Foreign keys & pair references"), + ("canonical_hashes", "Canonical hashes"), + ("pair_uniqueness", "Pair uniqueness"), + ("missing_files", "Missing files"), + ("ambiguous_metadata", "Ambiguous metadata"), + ("consistency", "Consistency"), +) +_DATABASE_CHECKS = ( + "integrity", + "foreign_keys", + "canonical_hashes", + "pair_uniqueness", + "missing_files", + "consistency", +) + + +def iter_consistency_issues(conn): + """The legacy consistency issues: tag and image rows of a missing prompt.""" + for ref in conn.execute( + "SELECT pt.prompt_id FROM prompt_tags pt " + "LEFT JOIN prompts p ON pt.prompt_id = p.id WHERE p.id IS NULL" + ): + yield f"prompt_tags entry references non-existent prompt {ref[0]}" + for ref in conn.execute( + "SELECT gi.id, gi.prompt_id FROM generated_images gi " + "LEFT JOIN prompts p ON gi.prompt_id = p.id WHERE p.id IS NULL" + ): + yield f"Image {ref[0]} references non-existent prompt {ref[1]}" + + +def _result(count, detail, *, status=None): + return { + "status": status or ("issues" if count else "passed"), + "count": count, + "detail": detail[:MAX_DETAIL], + } + + +def _counted(lines): + """(count, first MAX_DETAIL lines) of an iterable, consumed once.""" + count, detail = 0, [] + for line in lines: + count += 1 + if len(detail) < MAX_DETAIL: + detail.append(line) + return _result(count, detail) + + +def readonly_uri(db_path, *, windows=None): + """The ``mode=ro`` SQLite URI of *db_path*, on this platform or the given one. + + pathlib spells a UNC path ``\\\\server\\share\\x.db`` as + ``file://server/share/x.db``, a remote authority SQLite refuses to open. + With an empty authority (``file:////server/share/x.db``) SQLite hands + Windows the ``//server/share`` path, which it opens as the UNC path. + """ + if windows is None: + windows = os.name == "nt" + db_path = os.path.abspath(db_path) + if not windows: + return PurePosixPath(db_path).as_uri() + "?mode=ro" + path = PureWindowsPath(db_path) + if ntpath.splitdrive(str(path))[0].startswith("\\\\"): + return "file:" + quote("//" + path.as_posix()) + "?mode=ro" + return path.as_uri() + "?mode=ro" + + +def _integrity(conn, _roots): + rows = [ + row[0] for row in conn.execute(f"PRAGMA integrity_check({INTEGRITY_LIMIT})") + ] + if rows == ["ok"]: + return {**_result(0, []), "capped": False} + # SQLite stops listing at the limit: there may be more problems than that. + return {**_result(len(rows), rows), "capped": len(rows) >= INTEGRITY_LIMIT} + + +def _foreign_keys(conn, _roots): + return _counted( + f"{table} row {rowid} references a missing {parent} row" + for table, rowid, parent, _fkid in conn.execute("PRAGMA foreign_key_check") + ) + + +def _prompt_hash(text): + return generate_prompt_hash(text) if isinstance(text, str) else None + + +def _canonical_hashes(conn, _roots): + conn.create_function("pm_prompt_hash", 1, _prompt_hash, deterministic=True) + return _counted( + f"Prompt {row[0]}: the stored hash does not match its text" + for row in conn.execute( + "SELECT id FROM prompts WHERE hash IS NOT pm_prompt_hash(text) ORDER BY id" + ) + ) + + +def _pair_uniqueness(conn, _roots): + groups = conn.execute( + "SELECT positive_prompt_id,negative_prompt_id,COUNT(*) FROM prompt_pairs " + "GROUP BY positive_prompt_id,negative_prompt_id HAVING COUNT(*)>1 " + "ORDER BY positive_prompt_id,negative_prompt_id" + ).fetchall() + detail = [ + f"Positive prompt {positive} with " + + ("no negative" if negative is None else f"negative {negative}") + + f": {rows} pair rows" + for positive, negative, rows in groups + ] + # The surplus rows: what keeping one row per pair would leave out. + return _result(sum(rows - 1 for _p, _n, rows in groups), detail) + + +def _consistency(conn, _roots): + return _counted(iter_consistency_issues(conn)) + + +def _readable(root): + return os.path.isdir(root) and os.access(root, os.R_OK | os.X_OK) + + +def _root_prefix(root): + return normalize_image_path(root).rstrip("\\/") + os.sep + + +def _source_name(root, index): + """A source's folder name for the report (never its server path).""" + return os.path.basename(str(root).rstrip("\\/")) or f"Source {index + 1}" + + +def _owner(path, prefixes): + """Index of the longest source prefix *path* lies under, or None.""" + owner, longest = None, -1 + for index, prefix in enumerate(prefixes): + if path.startswith(prefix) and len(prefix) > longest: + owner, longest = index, len(prefix) + return owner + + +def _image_owner(image_path, file_path, prefixes): + spelled = file_path or normalize_image_path(image_path) + owner = _owner(spelled, prefixes) + if owner is None: + owner = _owner(normalize_image_path(os.path.realpath(image_path)), prefixes) + return owner + + +def _unavailable_line(unavailable, *, partial): + noun = "source is" if unavailable == 1 else "sources are" + effect = "its files were" if unavailable == 1 else "their files were" + if not partial: + effect = "missing files were" + return f"{unavailable} image {noun} unavailable, so {effect} not counted." + + +def _missing_files(conn, roots): + """Links whose file is gone, counted per source (only readable ones). + + Each source reports ``checked`` with its own count, or ``unavailable`` + (an unmounted drive) with no count: its links are never turned into a + count, while the readable sources still are. The check's ``count`` is the + readable sources' total (None when none is readable) and ``partial`` says + some sources were not checked. Links outside every source are named in the + detail, not counted. + + Inside the snapshot this only reads the image rows; it returns the step + that checks their files, which runs after the snapshot ended. + """ + if not roots: + result = _result(None, ["No image source is available."], status="unavailable") + return {**result, "partial": False, "sources": []} + rows = [] + cursor = conn.execute( + "SELECT id,image_path,filename,file_path FROM generated_images ORDER BY id" + ) + while batch := cursor.fetchmany(BATCH_ROWS): + rows.extend(batch) + return lambda: _check_files(rows, roots) + + +def _check_files(rows, roots): + """The ``missing_files`` result of the image *rows*, from the file system.""" + readable = [_readable(root) for root in roots] + sources = [ + { + "name": _source_name(root, index), + "status": "checked" if ok else "unavailable", + "count": 0 if ok else None, + } + for index, (root, ok) in enumerate(zip(roots, readable)) + ] + unavailable = readable.count(False) + if not any(readable): + result = _result( + None, [_unavailable_line(unavailable, partial=False)], status="unavailable" + ) + return {**result, "partial": False, "sources": sources} + prefixes = [_root_prefix(root) for root in roots] + outside, missing = 0, [] + for image_id, image_path, filename, file_path in rows: + owner = _image_owner(image_path, file_path, prefixes) + if owner is None: + outside += 1 + elif readable[owner] and not os.path.exists(image_path): + sources[owner]["count"] += 1 + if len(missing) < MAX_DETAIL: + name = filename or os.path.basename(image_path) + missing.append(f"Image {image_id}: {name} is missing") + count = sum(source["count"] or 0 for source in sources) + detail = [_unavailable_line(unavailable, partial=True)] if unavailable else [] + detail += missing + if outside: + links = "link" if outside == 1 else "links" + verb = "was" if outside == 1 else "were" + detail.append( + f"{outside} {links} outside the image sources {verb} not checked." + ) + status = None if count or not unavailable else "unavailable" + result = _result(count, detail, status=status) + return {**result, "partial": bool(unavailable), "sources": sources} + + +def _ambiguous(review_count): + if review_count is None: + return _result(None, [REVIEW_UNAVAILABLE], status="unavailable") + return _result(review_count, []) + + +def overall_status(checks) -> str: + """``passed`` only when every check passed; any check not run is incomplete.""" + statuses = {check["status"] for check in checks} + if statuses & {"unavailable", "failed"} or any( + check.get("partial") for check in checks + ): + return "incomplete" + return "issues" if "issues" in statuses else "passed" + + +def _failed(): + return _result(None, [CHECK_FAILED], status="failed") + + +def _run_check(check_id, step, *args): + """One check's result, or None (reported as failed) when it raised.""" + try: + return step(*args) + except Exception: # any error fails this check only; the others still run + logger.exception(f"Health check {check_id} failed") + return None + + +def _run_database_checks(db_path, roots): + """Each database check's result (None when it raised), from one snapshot. + + A check may return a step instead of its result: the step runs after the + snapshot ended and the connection closed (``missing_files``' file checks). + """ + results = dict.fromkeys(_DATABASE_CHECKS) + try: + conn = sqlite3.connect(readonly_uri(db_path), uri=True) + except sqlite3.Error: + logger.exception("Health check could not open the database") + return results + try: + conn.execute("BEGIN") # one read snapshot for every check + for check_id in _DATABASE_CHECKS: + runner = globals()["_" + check_id] # looked up per run (tests patch it) + results[check_id] = _run_check(check_id, runner, conn, roots) + except sqlite3.Error: + logger.exception("Health check could not read the database") + finally: + conn.close() + for check_id, result in results.items(): + if callable(result): + results[check_id] = _run_check(check_id, result) + return results + + +class HealthOperations: + """Mixin for PromptDatabase: the read-only System health check.""" + + def run_health_checks(self, roots, *, review_count=None) -> dict: + """Run every check and return a ``HealthReport``. + + Args: + roots: The image source directories to judge missing files under: + the effective sources plus any configured source that is not + available (reported as an unavailable source of ``missing_files``). + review_count: The Needs-review pending count, or None while no + review queue exists (``ambiguous_metadata`` is then unavailable). + """ + if review_count is not None and ( + type(review_count) is not int or review_count < 0 + ): + raise ValueError("review_count must be a non-negative integer or None") + roots = [str(root) for root in roots] + database = _run_database_checks(self.model.db_path, roots) + checks = [] + for check_id, label in CHECKS: + if check_id == "ambiguous_metadata": + result = _ambiguous(review_count) + else: + result = database[check_id] or _failed() + checks.append({"id": check_id, "label": label, **result}) + return { + "success": True, + "checked_at": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "status": overall_status(checks), + "checks": checks, + } diff --git a/database/library.py b/database/library.py index 4e532b95..7393e6df 100644 --- a/database/library.py +++ b/database/library.py @@ -628,3 +628,54 @@ class LibraryOperations: "kind": "prompt-manager-library", "positives": [owners[key] for key in order], } + + def library_counts(self) -> dict: + """System › Overview counts from one snapshot, writing nothing. + + ``positive_prompts`` and ``negative_variants`` are the unfiltered + Library's ``total`` and ``variant_total``: eligible parents and active + pairs (every pair's positive is eligible). ``images`` counts indexed + ``generated_images`` rows, not files on disk. An image has no rating + of its own (it shows its linked prompt's), so ``rated_images`` counts + the indexed images whose linked prompt is rated. + """ + with self.library_snapshot() as (conn, _epoch, _ceiling): + row = conn.execute( + "SELECT (SELECT COUNT(*) FROM prompts p WHERE " + _ELIGIBLE + ")," + "(SELECT COUNT(*) FROM prompt_pairs WHERE archived=0)," + "(SELECT COUNT(*) FROM generated_images)," + "(SELECT COUNT(*) FROM generated_images gi JOIN prompts r " + "ON r.id=gi.prompt_id WHERE r.rating IS NOT NULL)" + ).fetchone() + return { + "positive_prompts": row[0], + "negative_variants": row[1], + "images": row[2], + "rated_images": row[3], + } + + def usage_by_category(self) -> list[dict]: + """Runs and average rating of the Library's parents, per category. + + Only eligible parents count, so a negative-only prompt's runs (which + repeat its pairs' runs) are never added twice. A blank category groups + with no category as ``None``. Unrated parents are left out of the + average, which is ``None`` when no parent is rated. Most runs first. + """ + with self.library_snapshot() as (conn, _epoch, _ceiling): + rows = conn.execute( + "SELECT * FROM (SELECT NULLIF(TRIM(p.category),'') AS category," + "COALESCE(SUM(p.run_count),0) AS runs,AVG(p.rating) AS rating " + "FROM prompts p WHERE " + _ELIGIBLE + " GROUP BY 1) " + "ORDER BY runs DESC,category IS NULL,category" + ).fetchall() + return [ + { + "category": row["category"], + "runs": row["runs"], + "average_rating": ( + None if row["rating"] is None else round(row["rating"], 2) + ), + } + for row in rows + ] diff --git a/database/operations.py b/database/operations.py index 28bf35db..8424e880 100644 --- a/database/operations.py +++ b/database/operations.py @@ -20,6 +20,7 @@ from .execution_history import ExecutionHistoryOperations from .execution_contexts import ExecutionContextOperations, LIVE_HINT_CTES from .execution_outputs import ExecutionOutputOperations from .history_reads import HistoryReadOperations +from .health import HealthOperations, iter_consistency_issues # Import logging system try: @@ -342,6 +343,7 @@ class PromptDatabase( ImageFavoriteOperations, AnnotationOperations, SavedViewOperations, + HealthOperations, ): """Database operations class for managing prompts.""" @@ -2570,33 +2572,13 @@ class PromptDatabase( return cursor.rowcount def check_consistency(self) -> List[str]: - """Run consistency checks and return a list of issue descriptions.""" - issues: List[str] = [] - with self.model.get_connection() as conn: - # Check for orphaned prompt_tags entries - cursor = conn.execute(""" - SELECT pt.prompt_id, pt.tag_id FROM prompt_tags pt - LEFT JOIN prompts p ON pt.prompt_id = p.id - WHERE p.id IS NULL - """) - for ref in cursor.fetchall(): - issues.append( - "prompt_tags entry references non-existent prompt " - f"{ref['prompt_id']}" - ) + """Run consistency checks and return a list of issue descriptions. - # Check for orphaned image entries - cursor = conn.execute(""" - SELECT gi.id, gi.prompt_id FROM generated_images gi - LEFT JOIN prompts p ON gi.prompt_id = p.id - WHERE p.id IS NULL - """) - for ref in cursor.fetchall(): - issues.append( - f"Image {ref['id']} references non-existent prompt " - f"{ref['prompt_id']}" - ) - return issues + The same queries back the System health check's Consistency row + (database/health.py). + """ + with self.model.get_connection() as conn: + return list(iter_consistency_issues(conn)) def _image_row_to_dict(self, row: sqlite3.Row) -> Dict[str, Any]: """ diff --git a/py/api/__init__.py b/py/api/__init__.py index d97b3597..fbff121f 100644 --- a/py/api/__init__.py +++ b/py/api/__init__.py @@ -8,6 +8,7 @@ Split into domain-specific mixins for maintainability: - AutotagRoutesMixin: auto-tagging model management and tagging - JobRoutesMixin: the session job registry behind Activity (py/api/jobs.py) - ImageTagJobsMixin: bulk Add tags / Auto-tag as image-tags jobs +- SystemRoutesMixin: System health check, Overview and usage statistics """ import asyncio @@ -35,6 +36,7 @@ from .admin import AdminRoutesMixin from .settings_routes import SettingsRoutesMixin from .job_routes import JobRoutesMixin from .image_tag_jobs import ImageTagJobsMixin +from .system import SystemRoutesMixin from .jobs import JobRegistry from .logging_routes import LoggingRoutesMixin from .autotag_routes import AutotagRoutesMixin @@ -269,6 +271,7 @@ class PromptManagerAPI( SettingsRoutesMixin, JobRoutesMixin, ImageTagJobsMixin, + SystemRoutesMixin, LoggingRoutesMixin, AutotagRoutesMixin, LoraIntegrationMixin, @@ -325,6 +328,11 @@ class PromptManagerAPI( self._scan_job = None # The current duplicate scan (a py/api/jobs.Job outside the registry). self._duplicate_job = None + # The last System health report this session and the database + # generation it describes, bumped when a restore replaces self.db + # (py/api/system.py). + self._health_report = None + self._db_generation = 0 # Run cleanup on initialization to remove any existing duplicates try: @@ -482,6 +490,7 @@ class PromptManagerAPI( self._register_admin_routes(routes) self._register_settings_routes(routes) self._register_job_routes(routes) + self._register_system_routes(routes) self._register_logging_routes(routes) self._register_autotag_routes(routes) self._register_lora_routes(routes) @@ -728,6 +737,13 @@ class PromptManagerAPI( Returns: List[Path]: Validated directory paths, possibly empty. """ + return self._output_dirs_with_origin()[0] + + def _output_dirs_with_origin(self): + """``(dirs, auto_detected)``: the ``_get_all_output_dirs`` list, and + whether it comes from detection or ComfyUI's output rather than from + a valid configured root (False when there is no directory at all). + """ from ..config import GalleryConfig configured = list(GalleryConfig.MONITORING_DIRECTORIES) @@ -746,15 +762,16 @@ class PromptManagerAPI( output_dirs.append(Path(real)) if output_dirs: - return output_dirs + return output_dirs, False if configured: # Every configured root was rejected: never let auto-detection # resurrect it, serve ComfyUI's own output directory instead. - return self._comfyui_output_dir_list() - - fallback = self._find_comfyui_output_dir() - return [Path(fallback)] if fallback else [] + fallback_dirs = self._comfyui_output_dir_list() + else: + fallback = self._find_comfyui_output_dir() + fallback_dirs = [Path(fallback)] if fallback else [] + return fallback_dirs, bool(fallback_dirs) def _comfyui_output_dir_list(self): """ComfyUI's output directory as a one-element list, or [] if unknown.""" diff --git a/py/api/admin.py b/py/api/admin.py index 7ad05da6..64104893 100644 --- a/py/api/admin.py +++ b/py/api/admin.py @@ -1271,6 +1271,7 @@ class AdminRoutesMixin: self.logger.info(f"Removed {len(removed)} old safety backups") previous = self.db self.db = PromptDatabase(model.db_path) + self._database_replaced() # a health report described the old one # Its connections would otherwise linger (a file lock on Windows) previous.close_all() prompt_count = self.db.model.get_database_info().get("total_prompts", 0) diff --git a/py/api/jobs.py b/py/api/jobs.py index 568a99e6..09ab15aa 100644 --- a/py/api/jobs.py +++ b/py/api/jobs.py @@ -88,6 +88,9 @@ DUPLICATE_CLEANUP_JOB_KIND = "duplicate_cleanup" THUMBNAILS_JOB_KIND = "thumbnails" LORA_IMPORT_JOB_KIND = "lora_import" AUTOTAG_BATCH_JOB_KIND = "autotag" +# The System health check (py/api/system.py): a read-only ``read`` job with +# no file outcomes, so it may run beside a scan. +HEALTH_JOB_KIND = "health" def resolve_admission(admission=None, record_changing=None) -> str: diff --git a/py/api/system.py b/py/api/system.py new file mode 100644 index 00000000..9539ae91 --- /dev/null +++ b/py/api/system.py @@ -0,0 +1,207 @@ +"""System routes: the read-only health check, the Overview and usage statistics. + +- ``POST /prompt_manager/system/health`` starts the health check + (``database/health.py``) off-loop as a ``read`` job of kind ``health``: one + at a time, beside a scan, not cancellable and with no file outcomes. The + API keeps the last report for this session; ``GET`` returns it or null. + A report describes the database it was taken from: replacing the database + (a restore) forgets it, and a check that started before the swap is not + kept. +- ``GET /prompt_manager/system/overview`` is the System › Overview aggregate. + Every value is real or null: there is no backup status until backups report + one, no health result until a check ran, and no Needs-review count until + the review queue exists. The missing-thumbnail count is not part of it: + the page reads ``/images/thumbnails/status``, which walks every source. +- ``GET /prompt_manager/system/statistics`` lists runs and ratings per + category, with the legacy ``/stats`` totals and the Overview's count of + rated images (images whose linked prompt is rated). + +New routes answer errors with ``{success:false, code, error}``. +""" + +import asyncio +import os +from collections import namedtuple + +from aiohttp import web + +from .job_routes import busy_response +from .jobs import HEALTH_JOB_KIND, JobBusy + +HEALTH_JOB_TITLE = "Health check" +_HEALTH_STATUS_LINES = { + "passed": "Health check passed", + "issues": "Health check found issues", + "incomplete": "Health check incomplete", +} + + +# The last health report and the database generation it was taken from. +_StoredReport = namedtuple("_StoredReport", "generation report") + + +def _failure(message, status=500, code=None): + return web.json_response( + {"success": False, "code": code, "error": message}, status=status + ) + + +def health_summary(report): + """What the Overview shows of a report: when, the outcome, checks with issues.""" + if report is None: + return None + return { + "checked_at": report["checked_at"], + "status": report["status"], + "issues": sum(1 for check in report["checks"] if check["status"] == "issues"), + } + + +class SystemRoutesMixin: + """Mixin providing the /prompt_manager/system endpoints.""" + + def _register_system_routes(self, routes): + @routes.post("/prompt_manager/system/health") + async def start_health_check_route(request): + return await self.start_health_check(request) + + @routes.get("/prompt_manager/system/health") + async def health_report_route(request): + return await self.get_health_report(request) + + @routes.get("/prompt_manager/system/overview") + async def system_overview_route(request): + return await self.get_system_overview(request) + + @routes.get("/prompt_manager/system/statistics") + async def system_statistics_route(request): + return await self.get_system_statistics(request) + + # ── health ──────────────────────────────────────────────────────── + + def _health_roots(self): + """The sources missing files are judged under. + + The effective sources, plus every configured source that is not an + existing directory (an unmounted drive): the check then reports + missing files as unavailable instead of counting that source's links. + """ + from ..config import GalleryConfig + + roots = [str(path) for path in self._get_all_output_dirs()] + for configured in GalleryConfig.MONITORING_DIRECTORIES: + try: + resolved = GalleryConfig.resolve_gallery_root(configured) + except (OSError, TypeError, ValueError): + continue + if not os.path.isdir(resolved): + roots.append(resolved) + return roots + + def _database_replaced(self): + """Forget what described the previous database (call after swapping ``db``).""" + self._db_generation += 1 + self._health_report = None + + def _current_health_report(self): + """The last report, or None when none was kept for the current database.""" + stored = self._health_report + generation = self._db_generation + if isinstance(stored, _StoredReport) and stored.generation == generation: + return stored.report + return None + + async def start_health_check(self, request): + """Start the health check job (202), or 409 naming the running check.""" + generation = self._db_generation + db = self.db + try: + roots = await self._run_in_executor(self._health_roots) + except Exception: + self.logger.exception("Health check could not resolve the image sources") + return _failure("Failed to start the health check") + + async def run(job): + job.update(status_line="Checking the database and image links") + report = await self._run_in_executor(db.run_health_checks, roots) + if generation == self._db_generation: + self._health_report = _StoredReport(generation, report) + else: + self.logger.info("Health report dropped: the database was replaced") + job.update(progress=100, status_line=_HEALTH_STATUS_LINES[report["status"]]) + + try: + job = self._jobs.start( + HEALTH_JOB_KIND, + HEALTH_JOB_TITLE, + admission="read", + cancellable=False, + scope={}, + runner=run, + ) + except JobBusy as busy: + return busy_response(busy) + return web.json_response({"success": True, "job": job.summary()}, status=202) + + async def get_health_report(self, request): + return web.json_response( + {"success": True, "report": self._current_health_report()} + ) + + # ── overview ────────────────────────────────────────────────────── + + async def get_system_overview(self, request): + from ..config import GalleryConfig + + try: + (output_dirs, auto_detected), counts = await asyncio.gather( + self._run_in_executor(self._output_dirs_with_origin), + self._run_in_executor(self.db.library_counts), + ) + except Exception: + self.logger.exception("System overview failed") + return _failure("Failed to read the overview") + return web.json_response( + { + "success": True, + "library": { + "positive_prompts": counts["positive_prompts"], + "negative_variants": counts["negative_variants"], + }, + "images": { + "indexed": counts["images"], + "rated": counts["rated_images"], + }, + "health": health_summary(self._current_health_report()), + "backup": None, # no verified backup status exists yet (S8) + "needs_review": {"pending": None}, # no review queue yet (Q6) + "sources": { + "configured": len(GalleryConfig.MONITORING_DIRECTORIES), + "effective": len(output_dirs), + "auto_detected": auto_detected, + }, + }, + headers={"Cache-Control": "no-store"}, + ) + + # ── statistics ──────────────────────────────────────────────────── + + async def get_system_statistics(self, request): + try: + by_category, totals, counts = await asyncio.gather( + self._run_in_executor(self.db.usage_by_category), + self._run_in_executor(self.db.get_statistics), + self._run_in_executor(self.db.library_counts), + ) + except Exception: + self.logger.exception("System statistics failed") + return _failure("Failed to read statistics") + return web.json_response( + { + "success": True, + "by_category": by_category, + "totals": totals, + "rated_images": counts["rated_images"], + }, + headers={"Cache-Control": "no-store"}, + ) diff --git a/tests/browser/server.py b/tests/browser/server.py index 9b60d7b1..9926c269 100644 --- a/tests/browser/server.py +++ b/tests/browser/server.py @@ -21,6 +21,7 @@ from PIL.PngImagePlugin import PngInfo from tests.browser.request_hold import register_request_holds from tests.browser import slow_reset_watch from tests.browser.workspace_activity_fixture import register_activity_commands +from tests.browser.workspace_system_fixture import register_system_commands from tests.browser.workspace_concept_fixture import ( MAX_EXTRA_IMAGES as CONCEPT_MAX_EXTRA_IMAGES, MAX_SEEDLESS_IMAGES as CONCEPT_MAX_SEEDLESS_IMAGES, @@ -198,6 +199,7 @@ def create_app(root: Path) -> web.Application: register_tagging_commands(routes, api, root, model) reveal_stub = register_reveal_stub(routes, api, root) register_activity_commands(app, routes, api) + register_system_commands(app, routes, api, root) request_holds = register_request_holds(app, routes) slow_reset_watch.install(app, api) diff --git a/tests/browser/system-concept.spec.mjs b/tests/browser/system-concept.spec.mjs new file mode 100644 index 00000000..c32a62e6 --- /dev/null +++ b/tests/browser/system-concept.spec.mjs @@ -0,0 +1,410 @@ +import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs'; +import { test, expect } from './fixtures.mjs'; +import { SIZES, THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate } from './concept-inventory.mjs'; + +// #249 System S2 Task 6: System's shell and its Overview section (app.js renderSystem 658–706), the Statistics dialog +// ("Usage statistics", a1600–1604) and the health report (a1439–1456) against the immutable concept, on the +// concept-shaped real fixture. Each concept measurement is mapped only by the recorded deviations (D92–D96, D121, +// D122, D123), each commented where it applies; after that the difference must be empty, and a mutation of a +// row, a label or a token must be noticed. +test.use({ locale: 'en-US', timezoneId: 'UTC' }); + +test.beforeEach(async ({ page, request, browserName }) => { + await startCoverage(page, browserName); + await seedConcept(request); + await expect(await request.post('/__test__/system/reset')).toBeOK(); + await expect(await request.post('/__test__/jobs/restart')).toBeOK(); + page.conceptErrors = []; + page.on('pageerror', error => page.conceptErrors.push(error.message)); +}); + +test.afterEach(async ({ page, request, browserName }) => { + await saveCoverage(page, browserName); + await request.post('/__test__/system/reset'); + expect(page.conceptErrors).toEqual([]); +}); + +/** Runs in the page; self-contained. System's heading, section nav, Overview body and Save row under `mainSelector`. */ +function measureSystem(mainSelector) { + const main = document.querySelector(mainSelector); + const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null; + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const box = (node, origin = main) => { + const rect = node.getBoundingClientRect(), base = origin.getBoundingClientRect(); + return { left: Math.round(rect.left - base.left), width: Math.round(rect.width) }; + }; + // A surface's top: from `.two-col` beside its neighbour, else from the bottom of the surface above it. This pins the + // concept's `.surface + .surface` 18px offset, which concept.css applies inside `.two-col` too (copied, not ruled). + const top = node => { + const rect = node.getBoundingClientRect(), previous = node.previousElementSibling?.getBoundingClientRect(); + if (previous && rect.top >= previous.bottom) return Math.round(rect.top - previous.bottom); + return Math.round(rect.top - node.parentElement.getBoundingClientRect().top); + }; + const text = node => node?.textContent.trim() ?? null; + const button = node => ({ text: text(node), className: node.className, + ...css(node, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopWidth', 'borderTopColor', + 'borderTopLeftRadius', 'opacity', 'cursor') }); + const heading = main.querySelector(':scope > .heading'); + const nav = main.querySelector(':scope > .section-nav'); + const layout = main.querySelector(':scope > .settings-layout'); + const metrics = layout.querySelector(':scope > .summary-metrics'); + const columns = layout.querySelector(':scope > .two-col'); + const save = layout.querySelector(':scope > .save-row'); + const firstMetric = metrics.querySelector('.metric'); + const firstSurface = columns.querySelector('.surface'); + const firstOperation = firstSurface.querySelector('.operation'); + return { + tree: [...main.children].map(name), + heading: { + title: text(heading.querySelector('h1')), description: text(heading.querySelector('p.muted')), + actions: [...(heading.querySelector('.heading-actions')?.children ?? [])].map(name), + }, + nav: { + role: nav.getAttribute('role'), label: nav.getAttribute('aria-label'), + ...css(nav, 'display', 'flexWrap', 'columnGap', 'rowGap', 'paddingBottom', 'marginBottom', 'borderBottomWidth', 'borderBottomColor'), + box: box(nav), + buttons: [...nav.children].map(item => ({ pressed: item.getAttribute('aria-pressed'), ...button(item) })), + }, + layout: { tree: [...layout.children].map(name), ...css(layout, 'maxWidth'), box: box(layout) }, + metrics: { + ...css(metrics, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap', 'marginBottom'), box: box(metrics), + items: [...metrics.children].map(item => ({ tree: [...item.children].map(name), value: text(item.querySelector('strong')), + label: text(item.querySelector('span')), box: box(item) })), + metric: css(firstMetric, 'paddingTop', 'paddingLeft', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius', 'backgroundColor'), + strong: css(firstMetric.querySelector('strong'), 'display', 'fontSize', 'fontWeight', 'color'), + span: css(firstMetric.querySelector('span'), 'fontSize', 'color', 'marginTop'), + }, + columns: { + ...css(columns, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap'), box: box(columns), + surfaces: [...columns.children].map(surface => ({ tree: [...surface.children].map(name), box: box(surface), top: top(surface), + heading: text(surface.querySelector(':scope > h2')), + rows: [...surface.querySelectorAll(':scope > .operation')].map(row => ({ tree: [...row.children].map(name), + title: text(row.querySelector('h3')), description: text(row.querySelector('p')), + action: button(row.querySelector('.row-actions > button')) })) })), + surface: css(firstSurface, 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius'), + h2: css(firstSurface.querySelector('h2'), 'fontSize', 'fontWeight', 'color', 'marginBottom'), + operation: css(firstOperation, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', + 'borderTopWidth', 'borderTopColor'), + h3: css(firstOperation.querySelector('h3'), 'fontSize', 'fontWeight', 'color'), + p: css(firstOperation.querySelector('p'), 'fontSize', 'color', 'marginTop'), + actions: css(firstOperation.querySelector('.row-actions'), 'display', 'columnGap', 'flexShrink'), + }, + save: { + tree: [...save.children].map(name), ...css(save, 'display', 'alignItems', 'justifyContent', 'columnGap', 'marginTop', 'paddingTop'), + status: { text: text(save.querySelector('#settings-status')), ...css(save.querySelector('#settings-status'), 'fontSize', 'color') }, + button: button(save.querySelector('button')), + }, + pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth, + }; +} + +/** Runs in the page: the open dialog's title, body sequence, tables, operation rows and actions. */ +function measureDialog() { + const dialog = document.querySelector('dialog#modal[open]'); + const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null; + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const body = dialog.querySelector('#modal-body'); + const firstCell = selector => body.querySelector(selector); + const list = body.querySelector(':scope > .operation-list'); + const actions = body.querySelector(':scope > .dialog-actions'); + return { + title: dialog.querySelector('#modal-title').textContent, + body: [...body.children].map(name), + tables: [...body.querySelectorAll(':scope > table')].map(table => ({ + caption: table.caption?.textContent ?? null, + head: [...table.tHead.rows[0].cells].map(cell => [cell.localName, cell.scope, cell.textContent]), + rows: [...table.tBodies[0].rows].map(row => [...row.cells].map(cell => cell.textContent)), + })), + tableStyle: firstCell(':scope > table') && { + table: css(firstCell(':scope > table'), 'width', 'borderCollapse', 'fontSize', 'marginTop'), + caption: css(firstCell(':scope > table caption'), 'textAlign', 'fontSize', 'fontWeight', 'color', 'paddingBottom'), + th: css(firstCell(':scope > table thead th'), 'textAlign', 'fontSize', 'fontWeight', 'color', 'paddingTop', 'paddingLeft', + 'borderBottomWidth', 'borderBottomColor'), + td: css(firstCell(':scope > table tbody td'), 'textAlign', 'fontSize', 'color', 'paddingTop', 'paddingLeft', + 'borderBottomWidth', 'borderBottomColor'), + }, + operations: list && { + ...css(list, 'display', 'rowGap', 'marginTop'), + rows: [...list.children].map(row => ({ tree: [...row.children].map(name), title: row.querySelector('h3').textContent, + chip: row.querySelector('.chip').textContent })), + operation: css(list.querySelector('.operation'), 'display', 'alignItems', 'justifyContent', 'paddingTop', 'borderTopWidth'), + h3: css(list.querySelector('.operation h3'), 'fontSize', 'fontWeight', 'color'), + chip: css(list.querySelector('.operation .chip'), 'fontSize', 'color', 'backgroundColor', 'borderTopLeftRadius', 'paddingLeft'), + }, + actions: actions && { + ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'), + buttons: [...actions.children].map(item => ({ text: item.textContent, className: item.className, + ...css(item, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft') })), + }, + }; +} + +/** Runs in the page: a `.notice` (Maintenance's "Check first…") with its copy. */ +function measureNotice(mainSelector) { + const notice = document.querySelector(`${mainSelector} .settings-layout > .notice`); + const css = (node, ...keys) => Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])); + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const run = notice.querySelector(':scope > button'); + return { + tree: [...notice.children].map(name), text: [...notice.querySelectorAll(':scope > div > *')].map(node => [node.localName, node.textContent]), + frame: css(notice, 'display', 'alignItems', 'justifyContent', 'columnGap', 'borderTopWidth', 'borderTopColor', 'backgroundColor', + 'borderTopLeftRadius', 'paddingTop', 'paddingLeft', 'marginBottom'), + h2: css(notice.querySelector('h2'), 'fontSize', 'fontWeight', 'color'), + p: css(notice.querySelector('p'), 'fontSize', 'color', 'marginTop'), + run: { text: run.textContent, className: run.className, ...css(run, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft') }, + }; +} + +async function serverJson(request, url) { + const response = await request.get(url); + await expect(response).toBeOK(); + return response.json(); +} + +const number = value => value.toLocaleString('en-US'); +const ratingText = value => value === null ? 'Not rated' : `${number(value)} / 5`; + +/** The concept's shell and Overview with the recorded deviations applied, given the server's real overview. */ +function mapOverviewConcept(reference, overview) { + const relabel = { 'Database integrity': 'Not checked', 'Positive prompts': number(overview.library.positive_prompts), + 'Negative variants': number(overview.library.negative_variants), Images: number(overview.images.indexed) }; + const rows = { + // D93 (Q6): no review queue yet, so no count and the action is unavailable (dimmed, as the Save row's D92). + '3 ambiguous image links': row => ({ ...row, title: 'Ambiguous image links', description: 'Not available yet.', + action: { ...row.action, opacity: '0.55', cursor: 'not-allowed' } }), + // D93: no backup has been recorded (verified backups arrive with S8). + 'Backup status': row => ({ ...row, description: 'No retained backup record.' }), + // D93: there is no paging, so the shared settings are grid and density. + 'One display configuration': row => ({ ...row, description: 'Grid and density settings are shared across views.' }), + // D93: the real sources, not samples. + '2 sample output sources': row => ({ ...row, title: `${number(overview.sources.effective)} output sources` }), + // D96: the dialog shows runs and ratings by category and library totals, not prompt, tag or date usage. + Statistics: row => ({ ...row, description: 'Runs and ratings by category, with library totals.' }), + }; + return { + ...reference, + // D92: no "Sample configuration" chip. + heading: { ...reference.heading, actions: [] }, + // D122: the concept's literal label is its prototype action name ("system-tab sections"). + nav: { ...reference.nav, label: 'System sections' }, + metrics: { ...reference.metrics, items: reference.metrics.items.map(item => ({ ...item, value: relabel[item.label] })) }, + columns: { ...reference.columns, surfaces: reference.columns.surfaces.map(surface => ({ ...surface, + rows: surface.rows.map(row => (rows[row.title] ?? (value => value))(row)) })) }, + save: { ...reference.save, + // D92: truthful status copy, and "Save preferences" is aria-disabled (dimmed) while nothing is unsaved. + status: { ...reference.save.status, text: 'No unsaved changes.' }, + button: { ...reference.save.button, opacity: '0.55', cursor: 'not-allowed' } }, + }; +} + +async function conceptOverview(page, theme) { + await openConcept(page, 'system', theme); + await expect(page.locator('#main .summary-metrics')).toBeVisible(); + await page.mouse.move(0, 0); + return page.evaluate(measureSystem, '#main'); +} + +async function productionOverview(page, theme) { + await openProduction(page, 'system/overview', theme); + await expect(page.locator('#workspace-main .summary-metrics')).toBeVisible(); + // The thumbnail row is its own read (I2): compare once it answered. + await expect(page.locator('#workspace-main .operation h3').filter({ hasText: 'thumbnails' })).toHaveText('No missing thumbnails'); + await page.mouse.move(0, 0); + return page.evaluate(measureSystem, '#workspace-main'); +} + +for (const theme of THEMES) for (const [width, height] of SIZES) { + test(`system shell and overview match the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => { + await page.setViewportSize({ width, height }); + const overview = await serverJson(request, '/prompt_manager/system/overview'); + // The fixture's thumbnails all exist, so the concept's "No missing thumbnails" is also the real title. + expect((await serverJson(request, '/prompt_manager/images/thumbnails/status')).pending).toBe(0); + const concept = await conceptOverview(page, theme); + expect(concept.heading.actions).toEqual(['span.chip']); + // Paired captures are artifacts only; there are no PNG baselines. + await page.screenshot({ path: testInfo.outputPath(`concept-system-${theme}-${width}.png`), fullPage: true }); + const production = await productionOverview(page, theme); + await page.screenshot({ path: testInfo.outputPath(`production-system-${theme}-${width}.png`), fullPage: true }); + expect(diffInventory(mapOverviewConcept(concept, overview), production)).toEqual([]); + expect(production.pageScroll).toBe(false); + }); +} + +// The recorded differences, by path: D93 the review row (title, Not available yet, dimmed action), the backup row, the +// preferences row and the sources row, and metric 4; D96 the statistics row; D92 the chip, the status and the dimmed +// Save; D122 the nav label. (The fixture's first three metrics are the concept's own counts.) +const UNMAPPED_PATHS = [ + 'columns.surfaces.0.rows.0.action.cursor', 'columns.surfaces.0.rows.0.action.opacity', + 'columns.surfaces.0.rows.0.description', 'columns.surfaces.0.rows.0.title', 'columns.surfaces.0.rows.1.description', + 'columns.surfaces.1.rows.0.description', 'columns.surfaces.1.rows.1.title', 'columns.surfaces.1.rows.2.description', + 'heading.actions.0', 'heading.actions.length', 'metrics.items.3.value', 'nav.label', + 'save.button.cursor', 'save.button.opacity', 'save.status.text', +]; + +test('system comparison detects mutations', async ({ page, request }) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const overview = await serverJson(request, '/prompt_manager/system/overview'); + const concept = await conceptOverview(page, 'dark'); + const expected = mapOverviewConcept(concept, overview); + const production = await productionOverview(page, 'dark'); + expect(diffInventory(expected, production)).toEqual([]); + // Unmapped, the concept differs exactly where a deviation is recorded, and nowhere else. + expect(diffInventory(concept, production).map(item => item.path).sort()).toEqual(UNMAPPED_PATHS); + const measure = () => page.evaluate(measureSystem, '#workspace-main'); + const paths = async () => diffInventory(expected, await measure()).map(item => item.path); + + // A removed row. + const undoRow = await mutate(page, { remove: ['#workspace-main .two-col > .surface:first-child > .operation:nth-of-type(2)'] }); + expect(await paths()).toEqual(expect.arrayContaining(['columns.surfaces.0.rows.length', 'columns.surfaces.0.tree.length'])); + await undoRow(); + // A changed label. + await page.evaluate(() => { document.querySelector('#workspace-main .metric span').textContent = 'Positive prompt'; }); + expect(await paths()).toEqual(['metrics.items.0.label']); + await page.evaluate(() => { document.querySelector('#workspace-main .metric span').textContent = 'Positive prompts'; }); + // A swapped token. + const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' }); + expect(await paths()).toEqual(expect.arrayContaining(['metrics.metric.borderTopColor', 'columns.surface.borderTopColor'])); + await undoToken(); + expect(diffInventory(expected, await measure())).toEqual([]); +}); + +/** D96: the concept's table, its rows the real ones; then "Library totals" in the same table markup. */ +function mapStatisticsConcept(reference, statistics) { + const [runs] = reference.tables; + return { + ...reference, + // No sample note (prototype copy); a second table follows the first. + body: ['table', 'table'], + tables: [ + { ...runs, rows: statistics.by_category.map(row => [row.category ?? 'Uncategorized', number(row.runs), ratingText(row.average_rating)]) }, + { caption: 'Library totals', head: [['th', 'col', 'Total'], ['th', 'col', 'Count']], + rows: [['Prompts', number(statistics.totals.total_prompts)], ['Tags', number(statistics.totals.total_tags)], + ['Images', number(statistics.totals.total_images)], ['Rated images', number(statistics.rated_images)]] }, + ], + }; +} + +// The fixture's image sources and each check's real chip on it: nothing is damaged, Needs review does not exist yet (Q6). +const FIXTURE_SOURCES = ['output', 'input']; +const FIXTURE_CHIPS = Object.freeze({ 'Integrity check': 'Passed', 'Foreign keys & pair references': 'Passed', + 'Canonical hashes': 'Passed', 'Pair uniqueness': 'Passed', 'Missing files': 'Passed', 'Ambiguous metadata': 'Not available yet', + Consistency: 'Passed' }); +const chipOf = title => { + if (!(title in FIXTURE_CHIPS)) throw new Error(`No expected chip for ${title}`); + return FIXTURE_CHIPS[title]; +}; + +/** + * D95: the concept's health report from its own five rows, labels verbatim, titled "Health report". Only the D95 rows + * are inserted where they are recorded (Pair uniqueness after Canonical hashes, Consistency last); Missing files has a + * row per image source when there are several; each chip is the fixture's expected result. + */ +function mapHealthConcept(reference, sources = FIXTURE_SOURCES) { + const [first] = reference.operations.rows; + const added = title => ({ ...first, title, chip: chipOf(title) }); + const rows = reference.operations.rows.flatMap(row => { + const mapped = row.title === 'Missing files' && sources.length > 1 + ? sources.map(source => ({ ...row, title: `Missing files · ${source}`, chip: chipOf(row.title) })) + : [{ ...row, chip: chipOf(row.title) }]; + return row.title === 'Canonical hashes' ? [...mapped, added('Pair uniqueness')] : mapped; + }); + return { + ...reference, + title: 'Health report', + // No sample note (prototype copy); D95 adds the checked line after the list. + body: ['div.operation-list', 'p.small', 'div.dialog-actions'], + operations: { ...reference.operations, rows: [...rows, added('Consistency')] }, + }; +} + +// Unmapped, the health report differs exactly where D95 is recorded: the title, the body sequence (no sample note, the +// checked line), the rows after Canonical hashes (Pair uniqueness inserted, Missing files per source) and the chips of +// the concept's sample findings. +const UNMAPPED_HEALTH_PATHS = [ + 'body.0', 'body.1', + ...[3, 4].flatMap(row => [`operations.rows.${row}.chip`, `operations.rows.${row}.title`]), + // The concept's five rows end at row 4; rows 5–7 exist only with the D95 rows and the second source. + ...[5, 6, 7].flatMap(row => ['chip', 'title', 'tree.0', 'tree.1', 'tree.length'].map(key => `operations.rows.${row}.${key}`)), + 'operations.rows.length', 'title', +]; + +async function conceptDialogs(page, theme, testInfo) { + await openConcept(page, 'system', theme); + await page.locator('#main').getByRole('button', { name: 'Explore', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics'); + await page.mouse.move(0, 0); + const statistics = await page.evaluate(measureDialog); + await page.screenshot({ path: testInfo.outputPath(`concept-statistics-${theme}.png`) }); + await page.keyboard.press('Escape'); + await page.locator('#main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click(); + const notice = await page.evaluate(measureNotice, '#main'); + await page.locator('#main .notice').getByRole('button', { name: 'Run health checks', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Sample health report'); + await page.mouse.move(0, 0); + await page.screenshot({ path: testInfo.outputPath(`concept-health-${theme}.png`) }); + return { statistics, notice, health: await page.evaluate(measureDialog) }; +} + +async function productionDialogs(page, theme, testInfo) { + await openProduction(page, 'system/overview', theme); + await page.locator('#workspace-main').getByRole('button', { name: 'Explore', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics'); + await page.mouse.move(0, 0); + const statistics = await page.evaluate(measureDialog); + await page.screenshot({ path: testInfo.outputPath(`production-statistics-${theme}.png`) }); + await page.keyboard.press('Escape'); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click(); + await expect(page.locator('#workspace-main .notice')).toBeVisible(); + const notice = await page.evaluate(measureNotice, '#workspace-main'); + await page.locator('#workspace-main .notice').getByRole('button', { name: 'Run health checks', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Health report'); + await page.mouse.move(0, 0); + await page.screenshot({ path: testInfo.outputPath(`production-health-${theme}.png`) }); + return { statistics, notice, health: await page.evaluate(measureDialog) }; +} + +for (const theme of THEMES) test(`statistics and health dialogs match the concept ${theme}`, async ({ page, request }, testInfo) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const concept = await conceptDialogs(page, theme, testInfo); + expect(concept.statistics.body).toEqual(['p.demo-note', 'table']); + expect(concept.health.body).toEqual(['p.demo-note', 'div.operation-list', 'div.dialog-actions']); + const production = await productionDialogs(page, theme, testInfo); + const statistics = await serverJson(request, '/prompt_manager/system/statistics'); + const { report } = await serverJson(request, '/prompt_manager/system/health'); + // The fixture the expected chips describe: its two sources, and the concept's checks plus the D95 ones. + const missing = report.checks.find(check => check.id === 'missing_files'); + expect(missing.sources.map(source => source.name)).toEqual(FIXTURE_SOURCES); + expect(diffInventory(mapStatisticsConcept(concept.statistics, statistics), production.statistics)).toEqual([]); + expect(diffInventory(mapHealthConcept(concept.health), production.health)).toEqual([]); + // D123: Maintenance's notice is the concept's own, delivered ahead of the rest of the section. + expect(diffInventory(concept.notice, production.notice)).toEqual([]); + + // Unmapped, the dialogs differ only where D95 and D96 are recorded. + expect(diffInventory(concept.health, production.health).map(item => item.path).sort()).toEqual(UNMAPPED_HEALTH_PATHS); + expect(diffInventory(concept.statistics, production.statistics).map(item => item.path.split('.')[0]) + .filter((part, index, all) => all.indexOf(part) === index).sort()).toEqual(['body', 'tables']); + + // Mutation check: a changed chip token in the health report is noticed, then undone to an exact match again. + const undo = await mutate(page, { css: '#modal .operation .chip { background: rgb(255, 0, 0); }' }); + expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog)).map(item => item.path)) + .toEqual(['operations.chip.backgroundColor']); + await undo(); + expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog))).toEqual([]); + // A renamed check (database/health.py's label) is noticed against the concept's own label. + await page.evaluate(() => { document.querySelector('#modal .operation h3').textContent = 'Integrity checks'; }); + expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog)).map(item => item.path)) + .toEqual(['operations.rows.0.title']); + await page.evaluate(() => { document.querySelector('#modal .operation h3').textContent = 'Integrity check'; }); + // And a removed "Library totals" table and a changed caption in the statistics dialog. + await page.keyboard.press('Escape'); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click(); + await page.locator('#workspace-main').getByRole('button', { name: 'Explore', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics'); + const expectedStatistics = mapStatisticsConcept(concept.statistics, statistics); + const undoTable = await mutate(page, { remove: ['#modal-body > table:nth-of-type(2)'] }); + expect(diffInventory(expectedStatistics, await page.evaluate(measureDialog)).map(item => item.path)) + .toEqual(expect.arrayContaining(['body.length', 'tables.length'])); + await undoTable(); + await page.evaluate(() => { document.querySelector('#modal-body caption').textContent = 'Runs per category'; }); + expect(diffInventory(expectedStatistics, await page.evaluate(measureDialog)).map(item => item.path)).toEqual(['tables.0.caption']); +}); diff --git a/tests/browser/workspace-system.spec.mjs b/tests/browser/workspace-system.spec.mjs new file mode 100644 index 00000000..75d2069b --- /dev/null +++ b/tests/browser/workspace-system.spec.mjs @@ -0,0 +1,252 @@ +import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs'; +import { test, expect } from './fixtures.mjs'; +import { seedConcept } from './concept-inventory.mjs'; + +// System › Overview (#249 System S2 Task 6) on the concept-shaped real fixture: the metrics, rows, Statistics dialog and +// Health report read the server, never sample values (D93, D95, D96). The hooks are workspace_system_fixture.py's. +const workspace = '/prompt_manager/workspace'; +const OVERVIEW_URL = '/prompt_manager/system/overview'; +const THUMBNAILS_URL = '/prompt_manager/images/thumbnails/status'; +const number = value => value.toLocaleString('en-US'); + +test.beforeEach(async ({ page, request, browserName }) => { + await startCoverage(page, browserName); + await seedConcept(request); + await expect(await request.post('/__test__/system/reset')).toBeOK(); + await expect(await request.post('/__test__/jobs/restart')).toBeOK(); + page.systemErrors = []; + page.on('pageerror', error => page.systemErrors.push(error.message)); +}); + +test.afterEach(async ({ page, request, browserName }) => { + await saveCoverage(page, browserName); + await request.post('/__test__/release'); + await request.post('/__test__/system/reset'); + expect(page.systemErrors).toEqual([]); +}); + +async function serverJson(request, url) { + const response = await request.get(url); + await expect(response).toBeOK(); + return response.json(); +} + +const layoutOf = page => page.locator('#workspace-main .settings-layout'); + +async function openOverview(page) { + await page.goto(`${workspace}#/system/overview`); + await expect(layoutOf(page).locator(':scope > .summary-metrics')).toBeVisible(); + return layoutOf(page); +} + +/** The rows of one Overview surface: `[h3, p, button]`. */ +function rowsOf(layout, heading) { + return layout.locator('.two-col > section.surface').filter({ has: layout.page().getByRole('heading', { name: heading, exact: true }) }) + .locator('.operation').evaluateAll(rows => rows.map(row => [row.querySelector('h3').textContent, + row.querySelector('p').textContent, row.querySelector('.row-actions button').textContent])); +} + +test('overview shows real counts and Not checked', async ({ page, request }) => { + const writes = []; + page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) writes.push(item.url()); }); + const overview = await serverJson(request, OVERVIEW_URL); + const thumbnails = await serverJson(request, THUMBNAILS_URL); + // The concept fixture: six positives, their saved negatives and 24 linked images, all with thumbnails. + expect([overview.library.positive_prompts, overview.images.indexed, thumbnails.pending]).toEqual([6, 24, 0]); + // I2: the Overview never walks the sources for thumbnails; the row reads the status route itself. + expect(overview).not.toHaveProperty('thumbnails'); + const layout = await openOverview(page); + expect(await layout.locator('.summary-metrics > .metric').evaluateAll(items => items.map(item => + [item.querySelector('strong').textContent, item.querySelector('span').textContent]))).toEqual([ + [number(overview.library.positive_prompts), 'Positive prompts'], + [number(overview.library.negative_variants), 'Negative variants'], + [number(overview.images.indexed), 'Images'], + ['Not checked', 'Database integrity'], + ]); + expect(await rowsOf(layout, 'Library health')).toEqual([ + ['Ambiguous image links', 'Not available yet.', 'Review metadata'], + ['Backup status', 'No retained backup record.', 'View backups'], + ['No missing thumbnails', 'Generate only missing previews when necessary.', 'Review thumbnails'], + ]); + expect(await rowsOf(layout, 'Workspace preferences')).toEqual([ + ['One display configuration', 'Grid and density settings are shared across views.', 'Appearance'], + ['2 output sources', 'Monitoring and enrichment share one source list.', 'Manage sources'], + ['Statistics', 'Runs and ratings by category, with library totals.', 'Explore'], + ]); + expect(writes).toEqual([]); +}); + +test('Run health checks opens Health report and updates metric 4', async ({ page, request }) => { + await page.goto(`${workspace}#/system/maintenance`); + const layout = layoutOf(page); + await expect(layout.locator(':scope > .notice h2')).toHaveText('Check first. Review changes. Then apply.'); + // The rest of Maintenance is still the existing tools' (D94). + await expect(layout.locator(':scope > section.handoff h2')).toHaveText('Your current tools are ready'); + const run = layout.getByRole('button', { name: 'Run health checks', exact: true }); + await run.click(); + const dialog = page.locator('dialog#modal[open]'); + await expect(dialog.locator('#modal-title')).toHaveText('Health report'); + const { report } = await serverJson(request, '/prompt_manager/system/health'); + expect(report.status).toBe('incomplete'); + expect(await dialog.locator('.operation-list > .operation').evaluateAll(rows => rows.map(row => + [row.querySelector('h3').textContent, row.querySelector('span.chip').textContent]))).toEqual([ + ['Integrity check', 'Passed'], ['Foreign keys & pair references', 'Passed'], ['Canonical hashes', 'Passed'], + ['Pair uniqueness', 'Passed'], ['Missing files · output', 'Passed'], ['Missing files · input', 'Passed'], + ['Ambiguous metadata', 'Not available yet'], ['Consistency', 'Passed'], + ]); + await expect(dialog.locator('#modal-body > p.small')).toHaveText(/^Checked .+\. Some checks could not run, so this report is incomplete\.$/); + await expect(dialog.locator('.dialog-actions button')).toHaveText(['Close', 'Review missing links']); + await dialog.getByRole('button', { name: 'Close', exact: true }).click(); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); + await expect(run).toBeFocused(); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click(); + await expect(layout.locator('.metric').nth(3).locator('strong')).toHaveText('Incomplete'); + + // A missing file and an unmounted source: the report counts the readable sources and names the other one. + await expect(await request.post('/__test__/system/missing-file')).toBeOK(); + await expect(await request.post('/__test__/system/missing-source')).toBeOK(); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click(); + await layout.getByRole('button', { name: 'Run health checks', exact: true }).click(); + await expect(dialog.locator('.operation').filter({ hasText: 'Missing files' }).locator('.chip')) + .toHaveText(['1 link needs review', 'Passed', 'Not available']); + await expect(dialog.locator('.operation h3').filter({ hasText: 'Missing files' })) + .toHaveText(['Missing files · output', 'Missing files · input', 'Missing files · unmounted-drive']); + // Review missing links goes to Maintenance, where the cleanup's existing tool is linked. + await dialog.getByRole('button', { name: 'Review missing links', exact: true }).click(); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); + await expect(page).toHaveURL(`${workspace}#/system/maintenance`); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click(); + await expect(layout.locator('.metric').nth(3).locator('strong')).toHaveText('1 issue'); +}); + +test('a health check that fails says so and keeps the last result', async ({ page, request }) => { + await expect(await request.post('/__test__/system/fail', { data: { path: '/prompt_manager/system/health' } })).toBeOK(); + await page.goto(`${workspace}#/system/maintenance`); + const run = layoutOf(page).getByRole('button', { name: 'Run health checks', exact: true }); + await run.click(); + await expect(page.locator('#workspace-toast')).toHaveText('The health check could not finish. Nothing was changed.'); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); + await expect(run).not.toHaveAttribute('aria-disabled', 'true'); + await openOverview(page); + await expect(layoutOf(page).locator('.metric').nth(3).locator('strong')).toHaveText('Not checked'); +}); + +test('thumbnail row reflects the status endpoint', async ({ page, request }) => { + await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 3 } })).toBeOK(); + const status = await serverJson(request, '/prompt_manager/images/thumbnails/status'); + expect(status.pending).toBe(3); + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) }); + await expect(row.locator('h3')).toHaveText('3 missing thumbnails'); + // Until the thumbnail review dialog (S5), Review thumbnails opens Sources. + await row.getByRole('button', { name: 'Review thumbnails', exact: true }).click(); + await expect(page).toHaveURL(`${workspace}#/system/sources`); +}); + +test('the thumbnail row paints when its own status read answers', async ({ page, request }) => { + const held = async () => (await serverJson(request, '/__test__/hold')).held[THUMBNAILS_URL] ?? 0; + await expect(await request.post('/__test__/hold', { data: { path: THUMBNAILS_URL } })).toBeOK(); + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) }); + // The other values do not wait for the walk of every source. + await expect(row.locator('h3')).toHaveText('Checking thumbnails…'); + await expect(layout.locator('.summary-metrics > .metric')).toHaveCount(4); + await expect.poll(held).toBe(1); + await expect(await request.post('/__test__/release')).toBeOK(); + await expect(row.locator('h3')).toHaveText('No missing thumbnails'); +}); + +test('a failed thumbnail read is unavailable, never zero', async ({ page }) => { + await page.route(`**${THUMBNAILS_URL}`, route => route.fulfill({ status: 500, contentType: 'application/json', + body: JSON.stringify({ success: false, error: 'Injected failure' }) })); + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) }); + await expect(row.locator('h3')).toHaveText('Thumbnail status unavailable'); + await expect(layout.getByRole('alert')).toHaveCount(0); + await expect(layout.locator('.metric').nth(2).locator('strong')).toHaveText('24'); +}); + +test('ambiguous row reads Not available yet (Q6)', async ({ page }) => { + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ hasText: 'Ambiguous image links' }); + await expect(row.locator('p')).toHaveText('Not available yet.'); + const review = row.getByRole('button', { name: 'Review metadata', exact: true }); + await expect(review).toHaveAttribute('aria-disabled', 'true'); + await review.click({ force: true }); + await expect(page).toHaveURL(`${workspace}#/system/overview`); +}); + +test('View backups / Appearance / Manage sources switch sections', async ({ page }) => { + for (const [label, section, pressed] of [['View backups', 'backups', 'Backup & export'], ['Appearance', 'appearance', 'Appearance'], + ['Manage sources', 'sources', 'Sources & scans']]) { + const layout = await openOverview(page); + await layout.getByRole('button', { name: label, exact: true }).click(); + await expect(page).toHaveURL(`${workspace}#/system/${section}`); + await expect(page.locator('#workspace-main .section-nav [aria-pressed="true"]')).toHaveText(pressed); + await expect(layout.locator(':scope > .summary-metrics')).toHaveCount(0); + } +}); + +test('statistics dialog lists runs by category and library totals', async ({ page, request }) => { + const statistics = await serverJson(request, '/prompt_manager/system/statistics'); + expect(statistics.by_category.length).toBeGreaterThan(1); + const layout = await openOverview(page); + const explore = layout.getByRole('button', { name: 'Explore', exact: true }); + await explore.click(); + const dialog = page.locator('dialog#modal[open]'); + await expect(dialog.locator('#modal-title')).toHaveText('Usage statistics'); + const tables = dialog.locator('#modal-body > table'); + await expect(tables.locator('caption')).toHaveText(['Runs by category', 'Library totals']); + await expect(tables.first().locator('thead th')).toHaveText(['Category', 'Runs', 'Avg. rating']); + const cells = table => table.locator('tbody tr').evaluateAll(rows => rows.map(row => [...row.cells].map(cell => cell.textContent))); + expect(await cells(tables.first())).toEqual(statistics.by_category.map(row => [row.category ?? 'Uncategorized', + number(row.runs), row.average_rating === null ? 'Not rated' : `${number(row.average_rating)} / 5`])); + expect(await cells(tables.nth(1))).toEqual([['Prompts', number(statistics.totals.total_prompts)], + ['Tags', number(statistics.totals.total_tags)], ['Images', number(statistics.totals.total_images)], + ['Rated images', number(statistics.rated_images)]]); + await page.keyboard.press('Escape'); + await expect(explore).toBeFocused(); + // A statistics read that fails says so; the Overview stays. + await expect(await request.post('/__test__/system/fail', { data: { path: '/prompt_manager/system/statistics' } })).toBeOK(); + await explore.click(); + await expect(page.locator('#workspace-toast')).toHaveText('Statistics could not be loaded. Your saved data has not changed.'); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); +}); + +test('load failure shows Retry and no zeros', async ({ page, request }) => { + await expect(await request.post('/__test__/system/fail', { data: { path: OVERVIEW_URL } })).toBeOK(); + await page.goto(`${workspace}#/system/overview`); + const layout = layoutOf(page); + await expect(layout.getByRole('alert')).toHaveText('The overview could not be loaded. Your saved data has not changed.'); + await expect(layout.locator('.metric')).toHaveCount(0); + await expect(layout).not.toContainText('0'); + await layout.getByRole('button', { name: 'Retry', exact: true }).click(); + await expect(layout.locator('.summary-metrics > .metric')).toHaveCount(4); + await expect(layout.getByRole('alert')).toHaveCount(0); +}); + +test('out-of-order overview responses never repaint a newer section', async ({ page, request }) => { + const held = async () => (await serverJson(request, '/__test__/hold')).held[OVERVIEW_URL] ?? 0; + await expect(await request.post('/__test__/hold', { data: { path: OVERVIEW_URL } })).toBeOK(); + await page.goto(`${workspace}#/system/overview`); + const layout = layoutOf(page); + await expect(layout.getByRole('status')).toHaveText('Loading the overview…'); + await expect.poll(held).toBe(1); + // Leave for Appearance, come back (a second read), then leave again before either answers. + const nav = page.locator('#workspace-main .section-nav'); + await nav.getByRole('button', { name: 'Appearance', exact: true }).click(); + await nav.getByRole('button', { name: 'Overview', exact: true }).click(); + await expect.poll(held).toBe(2); + await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click(); + await expect(await request.post('/__test__/release')).toBeOK(); + await expect.poll(held).toBe(0); + // Both late answers reached a section that had left: Sources keeps its own body. + await expect(layout.locator(':scope > section.handoff')).toBeVisible(); + await page.evaluate(() => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve)))); + await expect(layout.locator('.summary-metrics')).toHaveCount(0); + expect(await layout.evaluate(node => [...node.children].map(child => child.className))).toEqual(['surface handoff', 'save-row']); + // Back on Overview, one fresh read paints one set of metrics. + await nav.getByRole('button', { name: 'Overview', exact: true }).click(); + await expect(layout.locator('.summary-metrics')).toHaveCount(1); + await expect(layout.locator('.two-col')).toHaveCount(1); +}); diff --git a/tests/browser/workspace.spec.mjs b/tests/browser/workspace.spec.mjs index fff42a62..f065b4c5 100644 --- a/tests/browser/workspace.spec.mjs +++ b/tests/browser/workspace.spec.mjs @@ -18,7 +18,8 @@ const routes = [ ['tags', 'Tags & categories', null], // #249 local Inspect: the native Inspect a file page; Overview's tool card still links the legacy metadata page. ['inspect', 'Inspect a file', null], - ['system', 'System', '/prompt_manager/admin'], + // #249 System S2: native System; its undelivered sections hand off per section (D94, asserted below). + ['system', 'System', null], // #249 D51: SAVED VIEWS' Favorites, Images filtered to favorites. ['favorites', 'Favorites', null], // #249 Activity-1: the concept's Activity page in the tools-and-system group. @@ -409,6 +410,8 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true); await sidebarLink(page, 'System').click(); + // System's Overview is native (S2 Task 6); a section not delivered yet still hands off (D94). + await main.locator('.section-nav').getByRole('button', { name: 'Integrations', exact: true }).click(); const handoff = main.locator('section.handoff'); await expect(handoff).toHaveClass(/\bsurface\b/); expect(await handoff.evaluate(surfaceStyle)).toMatchObject(SURFACE); @@ -417,6 +420,98 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true); }); +// #249 System S2 Task 5: the concept's renderSystem shell (app.js 698–706) at `#/system/
`, every section a D94 +// hand-off until its PR, and the Save row's truthful status (D92). +const SYSTEM_SECTION_LABELS = ['Overview', 'Appearance', 'Sources & scans', 'Integrations', 'Auto-tagging', 'Maintenance', + 'Backup & export', 'Diagnostics & logs']; +const SYSTEM_HASHES = ['overview', 'appearance', 'sources', 'integrations', 'tagging', 'maintenance', 'backups', 'diagnostics']; +const SYSTEM_HANDOFFS = { appearance: ['/prompt_manager/gallery', '/prompt_manager/admin'], sources: ['/prompt_manager/admin', '/prompt_manager/gallery'], + maintenance: ['/prompt_manager/admin', '/prompt_manager/gallery'] }; + +test('System is native: the concept shell, section navigation, sections, hand-offs and the Save row', async ({ page }) => { + const mutations = []; + page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) mutations.push(item.url()); }); + await page.goto(`${workspace}#/system`); + const main = page.locator('#workspace-main'); + await expect(main.locator('h1')).toHaveText('System'); + await expect(main.locator('.heading p.muted')).toHaveText('Configuration, maintenance and recovery—with browsing kept in the workspace.'); + // D92: no "Sample configuration" chip. + await expect(main.locator('.heading .chip')).toHaveCount(0); + expect(await main.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`))) + .toEqual(['div.heading', 'div.section-nav', 'div.settings-layout']); + const nav = main.getByRole('group', { name: 'System sections', exact: true }); + await expect(nav.getByRole('button')).toHaveText(SYSTEM_SECTION_LABELS); + await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true'); + await expect(nav.locator('[aria-pressed="true"]')).toHaveCount(1); + await expect(page.locator('aside.sidebar a[href="#/system"]')).toHaveAttribute('aria-current', 'page'); + const layout = main.locator('.settings-layout'); + await expect(layout.locator(':scope > .save-row > span.small#settings-status')).toHaveText('No unsaved changes.'); + const save = layout.locator(':scope > .save-row > button.ca-primary'); + await expect(save).toHaveText('Save preferences'); + await expect(save).toHaveAttribute('aria-disabled', 'true'); + // Playwright waits on an aria-disabled control, so the ignored press is forced. + await save.click({ force: true }); + await expect(layout.locator('#settings-status')).toHaveText('No unsaved changes.'); + + for (const [index, label] of SYSTEM_SECTION_LABELS.entries()) { + const hash = SYSTEM_HASHES[index]; + const button = nav.getByRole('button', { name: label, exact: true }); + await button.click(); + await expect(page).toHaveURL(`${workspace}#/system/${hash}`); + await expect(button).toHaveAttribute('aria-pressed', 'true'); + await expect(nav.locator('[aria-pressed="true"]')).toHaveCount(1); + // The section changes in place: focus stays on the pressed section, not the page title. + await expect(button).toBeFocused(); + // S2 Task 6: Overview is native (workspace-system.spec.mjs); Maintenance shows its health-check notice above + // the hand-off (D123); every other section hands off. + if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible(); + expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`))) + .toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], + maintenance: ['div.notice', 'section.surface handoff', 'div.save-row'] }[hash] + ?? ['section.surface handoff', 'div.save-row']); + if (hash === 'overview') continue; + const handoff = layout.locator('section.handoff'); + await expect(handoff.locator('h2')).toHaveText('Your current tools are ready'); + expect(await handoff.locator('a.workspace-link').evaluateAll(links => links.map(link => link.getAttribute('href')))) + .toEqual(SYSTEM_HANDOFFS[hash] ?? ['/prompt_manager/admin']); + await expect(layout.locator('#settings-status')).toHaveText('No unsaved changes.'); + await expect(page.locator('aside.sidebar a[href="#/system"]')).toHaveAttribute('aria-current', 'page'); + } + // Deep links, reload and history keep the section; an unknown section is Overview under its own hash. + await page.reload(); + await expect(nav.getByRole('button', { name: 'Diagnostics & logs', exact: true })).toHaveAttribute('aria-pressed', 'true'); + await page.goBack(); + await expect(page).toHaveURL(`${workspace}#/system/backups`); + await expect(nav.getByRole('button', { name: 'Backup & export', exact: true })).toHaveAttribute('aria-pressed', 'true'); + await page.goto(`${workspace}#/system/general`); + await expect(page).toHaveURL(`${workspace}#/system/overview`); + await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true'); + // The sidebar's System returns to the Overview section. + await nav.getByRole('button', { name: 'Integrations', exact: true }).click(); + await sidebarLink(page, 'System').click(); + await expect(page).toHaveURL(`${workspace}#/system`); + await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true'); + // Leaving System and coming back renders it once, with a fresh section. + await sidebarLink(page, 'Images').click(); + await expect(main.locator('h1')).toHaveText('Images'); + await expect(main.locator('.section-nav')).toHaveCount(0); + await page.goBack(); + await expect(main.locator('.section-nav')).toHaveCount(1); + await expect(main.locator('.settings-layout')).toHaveCount(1); + expect(mutations).toEqual([]); +}); + +test('Overview opens native System from its System card (Q12)', async ({ page }) => { + await page.goto(`${workspace}#/overview`); + const card = page.locator('#workspace-main .tool-card').filter({ has: page.getByRole('heading', { name: 'System', exact: true }) }); + await expect(card.locator('p.small')).toHaveText('Configuration, maintenance and recovery—with browsing kept in the workspace.'); + const link = card.getByRole('link', { name: 'Open System', exact: true }); + await expect(link).toHaveAttribute('href', '#/system/overview'); + await link.click(); + await expect(page.locator('#workspace-main h1')).toHaveText('System'); + await expect(page.locator('#workspace-main .section-nav [aria-pressed="true"]')).toHaveText('Overview'); +}); + // Task A3: the concept sidebar (index.html h16-41, app.js nav()) rendered from real data (D27, D28). const NAV_TOTALS = [ ['images', 'Images', '/prompt_manager/images/output?mode=workspace&limit=1'], diff --git a/tests/browser/workspace_system_fixture.py b/tests/browser/workspace_system_fixture.py new file mode 100644 index 00000000..6027c64d --- /dev/null +++ b/tests/browser/workspace_system_fixture.py @@ -0,0 +1,116 @@ +"""System test hooks for the browser tests (#249 System S2 Task 6). + +The data is the concept-shaped real fixture (``POST /__test__/concept/reset``, +``workspace_concept_fixture.py``): its parents, saved pairs, images, ratings +and runs are what the Overview, the Statistics dialog and the health check +read. These per-server hooks only change what a System test needs on top: + +- ``POST /__test__/system/reset`` restores the configured sources, forgets the + session's health report and ends every injected failure; +- ``POST /__test__/system/fail`` ``{path, times}`` answers the next ``times`` + requests for ``path`` (a ``/prompt_manager/system/`` route) with the routes' + 500 error envelope; +- ``POST /__test__/system/missing-source`` configures one more source that + does not exist (an unmounted drive); +- ``POST /__test__/system/missing-file`` deletes the newest linked image's + file from disk, leaving its database row (a missing-file link); +- ``POST /__test__/system/unthumbnailed`` ``{count}`` writes ``count`` plain + PNGs with no thumbnail into the output folder. + +Delays use the shared request holds (``request_hold.py``). +""" + +import os +from pathlib import Path + +from aiohttp import web +from PIL import Image + +FAILABLE_PREFIX = "/prompt_manager/system/" +MAX_UNTHUMBNAILED = 50 + + +def register_system_commands(app, routes, api, root: Path): + from py.config import GalleryConfig + + configured = list(GalleryConfig.MONITORING_DIRECTORIES) + failures: dict[str, int] = {} + + @web.middleware + async def inject_failures(request, handler): + left = failures.get(request.path, 0) + if left <= 0: + return await handler(request) + failures[request.path] = left - 1 + return web.json_response( + {"success": False, "code": None, "error": "Injected failure"}, status=500 + ) + + app.middlewares.append(inject_failures) + + def reset(): + failures.clear() + GalleryConfig.MONITORING_DIRECTORIES = list(configured) + api._health_report = None + + @routes.post("/__test__/system/reset") + async def system_reset(request): + reset() + return web.json_response({"success": True}) + + @routes.post("/__test__/system/fail") + async def system_fail(request): + data = await request.json() + path, times = data.get("path"), data.get("times", 1) + if ( + not isinstance(path, str) + or not path.startswith(FAILABLE_PREFIX) + or isinstance(times, bool) + or not isinstance(times, int) + or not 1 <= times <= 10 + ): + raise web.HTTPBadRequest() + failures[path] = times + return web.json_response({"success": True}) + + @routes.post("/__test__/system/missing-source") + async def missing_source(request): + absent = str(root / "unmounted-drive") + if os.path.exists(absent): + raise web.HTTPConflict() + GalleryConfig.MONITORING_DIRECTORIES = [*configured, absent] + return web.json_response( + {"success": True, "configured": len(GalleryConfig.MONITORING_DIRECTORIES)} + ) + + @routes.post("/__test__/system/missing-file") + async def missing_file(request): + with api.db.model.get_connection() as conn: + row = conn.execute( + "SELECT file_path FROM generated_images ORDER BY id DESC LIMIT 1" + ).fetchone() + if row is None: + raise web.HTTPNotFound() + Path(row[0]).unlink() + api.invalidate_gallery_cache() + return web.json_response({"success": True, "filename": Path(row[0]).name}) + + @routes.post("/__test__/system/unthumbnailed") + async def unthumbnailed(request): + data = await request.json() + count = data.get("count") + if ( + isinstance(count, bool) + or not isinstance(count, int) + or not 1 <= count <= MAX_UNTHUMBNAILED + ): + raise web.HTTPBadRequest() + folder = root / "output" / "unthumbnailed" + folder.mkdir(exist_ok=True) + for number in range(count): + with Image.new("RGB", (64, 64), "#2b3442") as image: + image.save(folder / f"plain_{number + 1:03}.png") + api.invalidate_gallery_cache() + return web.json_response({"success": True}) + + return reset diff --git a/tests/js/workspace-activity-store.test.js b/tests/js/workspace-activity-store.test.js index 5fce2cda..82b69dac 100644 --- a/tests/js/workspace-activity-store.test.js +++ b/tests/js/workspace-activity-store.test.js @@ -182,6 +182,24 @@ test('start returns busy with the running job', async () => { world.dispose(); }); +test('track follows a job another route started, with no read of its own', async () => { + const { createActivityStore } = await source; + const { world, ...deps } = harness(); + const store = createActivityStore(deps); + const seen = []; + store.subscribe(() => seen.push(store.find(ID_A)?.state)); + const job = store.track(summary({ kind: 'health', title: 'Health check' })); + assert.equal(job.id, ID_A); + assert.equal(world.gets, 0, 'the started job is known without a list read'); + assert.deepEqual(world.open().map(stream => stream.url), [`/prompt_manager/jobs/${ID_A}/events`]); + assert.deepEqual(world.toasts, [], 'the route that started it says so itself'); + send(world.open()[0], summary({ kind: 'health', title: 'Health check', state: 'finished', progress: 100 })); + assert.deepEqual(seen, ['running', 'finished']); + assert.equal(world.open().length, 0, 'an ended job is no longer followed'); + assert.throws(() => store.track({ id: 'nope' }), /Invalid job/); + world.dispose(); +}); + test('a list that is not a job list counts as a failed read', async () => { const { createActivityStore } = await source; const { world, api, ...deps } = harness(); diff --git a/tests/js/workspace-state.test.js b/tests/js/workspace-state.test.js index a02129a1..c05714bb 100644 --- a/tests/js/workspace-state.test.js +++ b/tests/js/workspace-state.test.js @@ -23,8 +23,50 @@ test('tags is a native route titled Tags; only unbuilt routes hand off to legacy assert.equal(routeTitle(route), route.label); } assert.equal(routeTitle(resolveRoute('#/library/7')), 'Prompt library'); - for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id); - for (const id of ['system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), true, id); + // System is native from S2 (#249 System): its undelivered sections hand off per section (D94), not the route. + for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity', 'system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id); +}); + +test('system resolves its sections; #/overview stays the production hub', async () => { + const { resolveRoute, routeTitle, isHandoff, routes } = await source; + // `#/system` is the Overview section and keeps the sidebar's hash. + const system = resolveRoute('#/system'); + assert.deepEqual([system.id, system.section, system.hash, routeTitle(system), isHandoff(system)], + ['system', 'overview', '#/system', 'System', false]); + for (const section of ['overview', 'appearance', 'sources', 'integrations', 'tagging', 'maintenance', 'backups', 'diagnostics']) { + const route = resolveRoute(`#/system/${section}`); + assert.deepEqual([route.id, route.section, route.hash], ['system', section, `#/system/${section}`]); + } + // An unknown child is the Overview section with a replaced hash. + for (const hash of ['#/system/general', '#/system/', '#/system/__proto__', '#/system/backups/1', '#/system/']) { + const route = resolveRoute(hash); + assert.deepEqual([route.id, route.section, route.hash], ['system', 'overview', '#/system/overview'], hash); + } + // D28: the production hub is untouched. + const hub = resolveRoute('#/overview'); + assert.deepEqual([hub.id, hub.section], ['overview', undefined]); + // Q12: Overview's System card opens the native System Overview, with no legacy href or action. + const card = routes.find(route => route.id === 'system'); + assert.equal(card.href, '#/system/overview'); + assert.equal(card.native, true); + assert.equal(card.action, 'Open System'); +}); + +test('the router renders again when only the System section changes', async () => { + const { mountRouter } = await source; + const win = new EventTarget(); + win.location = { hash: '#/system' }; + const replacements = []; + win.history = { state: null, replaceState(state, unused, hash) { replacements.push(hash); win.location.hash = hash; } }; + const seen = []; + const dispose = mountRouter(win, route => seen.push([route.id, route.section])); + for (const hash of ['#/system/appearance', '#/system/appearance', '#/system/bogus', '#/system/overview', '#/images']) { + win.location.hash = hash; + win.dispatchEvent(new Event('hashchange')); + } + assert.deepEqual(seen, [['system', 'overview'], ['system', 'appearance'], ['system', 'overview'], ['images', undefined]]); + assert.deepEqual(replacements, ['#/system/overview']); + dispose(); }); test('inspect is the native Inspect a file page with the concept description', async () => { diff --git a/tests/js/workspace-system-controls.test.js b/tests/js/workspace-system-controls.test.js new file mode 100644 index 00000000..19a99b84 --- /dev/null +++ b/tests/js/workspace-system-controls.test.js @@ -0,0 +1,214 @@ +// System controls (#249 System S2 Task 5): the concept's `setting()` (app.js 624–634, its toggles hydrated into Toolkit +// switches by `hydrateSwitches` 708–717), `operation()` (645–654) and `sectionNav()` (306–308) as DOM builders whose +// listeners live on the caller's scope. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const controlsModule = import('../../web/js/workspace/system-controls.mjs'); +const coreModule = import('../../web/js/shared/toolkit/core.js'); + +/** A minimal DOM whose elements count their live listeners (a listener's signal aborts it, as in a browser). */ +function fakeDOM() { + const previous = global.document; + const elements = []; + class Element { + constructor(tag) { + this.tagName = tag.toUpperCase(); + this.children = []; + this.attributes = {}; + this.listeners = []; + this.dataset = {}; + this.className = ''; + this.id = ''; + this.value = ''; + this._text = ''; + elements.push(this); + } + get textContent() { return this._text + this.children.map(child => child.textContent).join(''); } + set textContent(text) { this.children = []; this._text = String(text); } + append(...nodes) { + for (const node of nodes) { + const child = typeof node === 'string' ? Object.assign(new Element('#text'), { _text: node }) : node; + child.parentNode = this; + this.children.push(child); + } + } + replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); } + setAttribute(name, value) { this.attributes[name] = String(value); } + getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; } + removeAttribute(name) { delete this.attributes[name]; } + addEventListener(type, callback, options = {}) { + if (options.signal?.aborted) return; + const entry = { type, callback }; + this.listeners.push(entry); + options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); }); + } + removeEventListener(type, callback) { this.listeners = this.listeners.filter(item => item.type !== type || item.callback !== callback); } + dispatch(type) { for (const { callback } of this.listeners.filter(item => item.type === type)) callback({ type, target: this }); } + click() { this.dispatch('click'); } + } + global.document = { + createElement: tag => new Element(tag), + createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }), + }; + const walk = (node, found = []) => { found.push(node); node.children.forEach(child => walk(child, found)); return found; }; + return { + listenerCount: () => elements.reduce((sum, element) => sum + element.listeners.length, 0), + find: (root, predicate) => walk(root).find(predicate), + tags: node => node.children.map(child => child.tagName.toLowerCase()), + restore() { global.document = previous; }, + }; +} + +test('settingRow is the concept row: div > label[for] + p, then the control', async () => { + const { settingRow } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const changes = []; + const row = settingRow({ id: 'timeout', label: 'Node search auto-hide (seconds)', + description: 'Node search behavior only. Use 0 to keep results visible.', + control: { type: 'number', value: 30, min: 0, max: 300 }, scope, onChange: value => changes.push(value) }); + assert.equal(row.className, 'setting-row'); + assert.deepEqual(dom.tags(row), ['div', 'input']); + const [text, input] = row.children; + assert.deepEqual(dom.tags(text), ['label', 'p']); + assert.equal(text.children[0].textContent, 'Node search auto-hide (seconds)'); + assert.equal(text.children[0].htmlFor, 'timeout'); + assert.equal(text.children[1].textContent, 'Node search behavior only. Use 0 to keep results visible.'); + assert.deepEqual([input.id, input.type, input.value, input.min, input.max], ['timeout', 'number', '30', '0', '300']); + assert.equal(input.dataset.setting, 'timeout'); + input.value = '45'; + input.dispatch('change'); + assert.deepEqual(changes, [45]); + + const select = settingRow({ id: 'open-mode', label: 'Open from ComfyUI', description: 'Keep the node pack’s popup / new-tab preference.', + control: { type: 'select', options: [['newtab', 'New tab'], ['popup', 'Popup window']], value: 'popup' }, scope, + onChange: value => changes.push(value) }).children[1]; + assert.equal(select.tagName, 'SELECT'); + assert.deepEqual(select.children.map(option => [option.value, option.textContent]), [['newtab', 'New tab'], ['popup', 'Popup window']]); + assert.equal(select.value, 'popup'); + select.value = 'newtab'; + select.dispatch('change'); + assert.deepEqual(changes, [45, 'newtab']); + + // The concept's range is 1–16 unless the row says otherwise (D113: the CPU count). + const range = settingRow({ id: 'workers', label: 'Worker threads', description: 'Balance background scans.', + control: { type: 'range', value: 4 } }).children[1]; + assert.deepEqual([range.type, range.min, range.max, range.value], ['range', '1', '16', '4']); + const password = settingRow({ id: 'civitai-key', label: 'CivitAI API key', description: 'Credentials stay on the server.', + control: { type: 'password', value: '' } }).children[1]; + assert.equal(password.type, 'password'); + + // A ready-made control is placed as given and takes the row id. + const custom = document.createElement('textarea'); + const customRow = settingRow({ id: 'custom', label: 'Custom', description: 'Any control.', control: custom }); + assert.equal(customRow.children[1], custom); + assert.equal(custom.id, 'custom'); + assert.throws(() => settingRow({ id: '', label: 'x', description: 'y', control: { type: 'number' } }), TypeError); + assert.throws(() => settingRow({ id: 'x', label: 'x', description: 'y', control: { type: 'colour' } }), TypeError); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('a toggle row holds the Toolkit switch, labelled by the row label', async () => { + const { settingRow } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const changes = []; + const row = settingRow({ id: 'video-auto', label: 'Autoplay videos', description: 'Videos stay within the same gallery.', + control: { type: 'toggle', checked: false }, scope, onChange: value => changes.push(value) }); + const field = row.children[1]; + // createSwitch('', '', id): `label.ca-switch` > input[type=checkbox][role=switch] + an empty label text. + assert.equal(field.tagName, 'LABEL'); + assert.equal(field.className, 'ca-switch'); + const input = field.children[0]; + assert.deepEqual([input.type, input.role, input.id, input.checked], ['checkbox', 'switch', 'video-auto', false]); + assert.equal(row.children[0].children[0].htmlFor, input.id); + assert.equal(input.dataset.setting, 'video-auto'); + input.checked = true; + input.dispatch('change'); + assert.deepEqual(changes, [true]); + // The concept's toggles are on unless configured off (`val !== false`). + const on = settingRow({ id: 'video-loop', label: 'Loop videos', description: 'Repeat a clip.', control: { type: 'toggle' } }); + assert.equal(on.children[1].children[0].checked, true); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('operationRow is the concept operation with a danger class only when flagged', async () => { + const { operationRow } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const clicks = []; + const row = operationRow({ title: 'Missing-file links', + description: 'Remove stale database links only. Files on disk are never deleted by this operation.', + label: 'Review plan', danger: true, scope, onClick: button => clicks.push(button) }); + assert.equal(row.className, 'operation'); + assert.deepEqual(dom.tags(row), ['div', 'div']); + assert.deepEqual(dom.tags(row.children[0]), ['h3', 'p']); + assert.equal(row.children[0].children[0].textContent, 'Missing-file links'); + assert.equal(row.children[1].className, 'row-actions'); + const button = row.children[1].children[0]; + assert.deepEqual([button.tagName, button.type, button.textContent, button.className], ['BUTTON', 'button', 'Review plan', 'danger']); + button.click(); + assert.deepEqual(clicks, [button]); + const plain = operationRow({ title: 'Statistics', description: 'Usage by prompt, category, tag and date.', label: 'Explore', + scope, onClick() {} }); + assert.equal(plain.children[1].children[0].className, ''); + assert.throws(() => operationRow({ title: 'x', description: 'y', label: 'z', onClick() {} }), TypeError); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('sectionNav marks the active section and reports the chosen one', async () => { + const { sectionNav } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const chosen = []; + const nav = sectionNav({ label: 'System sections', sections: [['overview', 'Overview'], ['appearance', 'Appearance']], + active: 'appearance', scope, onSelect: id => chosen.push(id) }); + assert.equal(nav.className, 'section-nav'); + assert.equal(nav.getAttribute('role'), 'group'); + assert.equal(nav.getAttribute('aria-label'), 'System sections'); + assert.deepEqual(nav.children.map(button => [button.textContent, button.getAttribute('aria-pressed'), button.dataset.value]), + [['Overview', 'false', 'overview'], ['Appearance', 'true', 'appearance']]); + nav.children[0].click(); + assert.deepEqual(chosen, ['overview']); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('disposing the scope leaves no listener behind', async () => { + const { settingRow, operationRow, sectionNav } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + settingRow({ id: 'a', label: 'A', description: 'a', control: { type: 'toggle' }, scope, onChange() {} }); + settingRow({ id: 'b', label: 'B', description: 'b', control: { type: 'select', options: ['Grid', 'List'] }, scope, onChange() {} }); + operationRow({ title: 'C', description: 'c', label: 'Go', scope, onClick() {} }); + sectionNav({ label: 'System sections', sections: [['overview', 'Overview']], active: 'overview', scope, onSelect() {} }); + assert.equal(dom.listenerCount() > 0, true); + scope.dispose(); + assert.equal(dom.listenerCount(), 0); + } finally { + scope.dispose(); + dom.restore(); + } +}); diff --git a/tests/js/workspace-system-overview.test.js b/tests/js/workspace-system-overview.test.js new file mode 100644 index 00000000..bafcd0f9 --- /dev/null +++ b/tests/js/workspace-system-overview.test.js @@ -0,0 +1,255 @@ +// System › Overview (#249 System S2 Task 6): the concept's Overview copy on real values (D93), the Health report rows +// (D95), the Statistics tables (D96) and the health check run that waits for its read job. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const overviewModule = import('../../web/js/workspace/system-overview.mjs'); + +const OVERVIEW = Object.freeze({ + library: { positive_prompts: 1234, negative_variants: 7 }, + images: { indexed: 24, rated: 20 }, + health: null, + backup: null, + needs_review: { pending: null }, + sources: { configured: 2, effective: 2, auto_detected: false }, +}); + +test('metrics are the server counts and Not checked before any health check', async () => { + const { metricValues } = await overviewModule; + assert.deepEqual(metricValues(OVERVIEW), [ + [(1234).toLocaleString(), 'Positive prompts'], ['7', 'Negative variants'], ['24', 'Images'], + ['Not checked', 'Database integrity'], + ]); +}); + +test('metric 4 is the last health result', async () => { + const { integrityText } = await overviewModule; + assert.equal(integrityText(null), 'Not checked'); + assert.equal(integrityText({ status: 'passed', issues: 0 }), 'Passed'); + assert.equal(integrityText({ status: 'issues', issues: 1 }), '1 issue'); + assert.equal(integrityText({ status: 'issues', issues: 3 }), '3 issues'); + assert.equal(integrityText({ status: 'incomplete', issues: 0 }), 'Incomplete'); + // Issues found beside a check that could not run (Needs review, Q6) are still named. + assert.equal(integrityText({ status: 'incomplete', issues: 2 }), '2 issues'); +}); + +test('library health rows never invent a review count, a backup or thumbnails', async () => { + const { ambiguousTitle, backupText } = await overviewModule; + assert.equal(ambiguousTitle({ pending: null }), 'Ambiguous image links'); + assert.equal(ambiguousTitle(undefined), 'Ambiguous image links'); + assert.equal(ambiguousTitle({ pending: 1 }), '1 ambiguous image link'); + assert.equal(ambiguousTitle({ pending: 3 }), '3 ambiguous image links'); + assert.equal(backupText(null), 'No retained backup record.'); + assert.equal(backupText({ verified_at: 'not a time' }), 'No retained backup record.'); + assert.match(backupText({ verified_at: '2026-10-22T09:30:00Z' }), /^Last verified backup .+\.$/); +}); + +test('the thumbnail row is its own read of the thumbnail status', async () => { + const { thumbnailTitle, THUMBNAILS_LOADING } = await overviewModule; + // Until `/images/thumbnails/status` answers the row says it is checking; a failed read is unavailable, never 0. + assert.equal(THUMBNAILS_LOADING, 'Checking thumbnails…'); + assert.equal(thumbnailTitle(undefined), THUMBNAILS_LOADING); + assert.equal(thumbnailTitle({ pending: 0 }), 'No missing thumbnails'); + assert.equal(thumbnailTitle({ pending: 1 }), '1 missing thumbnail'); + assert.equal(thumbnailTitle({ pending: 12 }), '12 missing thumbnails'); + assert.equal(thumbnailTitle(null), 'Thumbnail status unavailable'); + assert.equal(thumbnailTitle({ pending: 'x' }), 'Thumbnail status unavailable'); + // With no image source there is nothing to count. + assert.equal(thumbnailTitle({ pending: 0 }, { effective: 0 }), 'Thumbnail status unavailable'); + assert.equal(thumbnailTitle({ pending: 2 }, { effective: 1 }), '2 missing thumbnails'); +}); + +test('an integrity count SQLite stopped at reads as a lower bound', async () => { + const { checkChip } = await overviewModule; + assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 100, capped: true }), '100+ problems found'); + assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 37, capped: false }), '37 problems found'); + assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 1 }), '1 problem found'); +}); + +test('the sources row tells configured, available and detected sources apart', async () => { + const { sourcesCopy } = await overviewModule; + const shared = 'Monitoring and enrichment share one source list.'; + assert.deepEqual(sourcesCopy({ configured: 2, effective: 2, auto_detected: false }), ['2 output sources', shared]); + assert.deepEqual(sourcesCopy({ configured: 2, effective: 1, auto_detected: false }), + ['1 output source', `1 of 2 configured sources available. ${shared}`]); + assert.deepEqual(sourcesCopy({ configured: 0, effective: 1, auto_detected: true }), + ['1 output source', `None configured is available; ComfyUI's output folder is used. ${shared}`]); +}); + +const REPORT = Object.freeze({ + checked_at: '2026-10-23T08:00:00Z', + status: 'issues', + checks: [ + { id: 'integrity', label: 'Integrity check', status: 'passed', count: 0 }, + { id: 'foreign_keys', label: 'Foreign keys & pair references', status: 'issues', count: 2 }, + { id: 'canonical_hashes', label: 'Canonical hashes', status: 'issues', count: 1 }, + { id: 'pair_uniqueness', label: 'Pair uniqueness', status: 'failed', count: null }, + { id: 'missing_files', label: 'Missing files', status: 'issues', count: 2, + sources: [{ name: 'output', status: 'checked', count: 2 }] }, + { id: 'ambiguous_metadata', label: 'Ambiguous metadata', status: 'unavailable', count: null }, + { id: 'consistency', label: 'Consistency', status: 'issues', count: null }, + ], +}); + +test('health rows give each check a chip from its real status, in the server order', async () => { + const { healthRows } = await overviewModule; + assert.deepEqual(healthRows(REPORT), [ + ['Integrity check', 'Passed'], + ['Foreign keys & pair references', '2 broken references'], + ['Canonical hashes', '1 hash does not match its text'], + ['Pair uniqueness', 'Could not check'], + ['Missing files', '2 links need review'], + ['Ambiguous metadata', 'Not available yet'], + ['Consistency', 'Issues found'], + ]); +}); + +test('missing files is reported per image source when there are several', async () => { + const { healthRows } = await overviewModule; + const report = { ...REPORT, checks: [{ id: 'missing_files', label: 'Missing files', status: 'issues', count: 1, partial: true, + sources: [{ name: 'output', status: 'checked', count: 1 }, { name: 'input', status: 'checked', count: 0 }, + { name: 'archive', status: 'unavailable', count: null }] }] }; + assert.deepEqual(healthRows(report), [ + ['Missing files · output', '1 link needs review'], + ['Missing files · input', 'Passed'], + ['Missing files · archive', 'Not available'], + ]); + const unavailable = { ...REPORT, checks: [{ id: 'missing_files', label: 'Missing files', status: 'unavailable', count: null, sources: [] }] }; + assert.deepEqual(healthRows(unavailable), [['Missing files', 'Not available']]); +}); + +test('the report summary says when it was checked and whether it is complete', async () => { + const { healthSummary } = await overviewModule; + const when = new Date(REPORT.checked_at).toLocaleString(); + assert.equal(healthSummary({ ...REPORT, status: 'passed' }), `Checked ${when}. Every check passed.`); + assert.equal(healthSummary({ ...REPORT, status: 'incomplete' }), `Checked ${when}. Some checks could not run, so this report is incomplete.`); + assert.equal(healthSummary({ ...REPORT, checked_at: 'x' }), 'Nothing was changed; review the checks with issues.'); +}); + +test('statistics rows are the real runs and ratings, then the library totals', async () => { + const { categoryRows, totalsRows } = await overviewModule; + const statistics = { + by_category: [{ category: 'Landscapes', runs: 1200, average_rating: 4.5 }, + { category: null, runs: 3, average_rating: null }], + totals: { total_prompts: 12, total_tags: 15, total_images: 24 }, + rated_images: 20, + }; + assert.deepEqual(categoryRows(statistics), [['Landscapes', (1200).toLocaleString(), '4.5 / 5'], ['Uncategorized', '3', 'Not rated']]); + assert.deepEqual(totalsRows(statistics), [['Prompts', '12'], ['Tags', '15'], ['Images', '24'], ['Rated images', '20']]); +}); + +/** A fake api answering each `get`/`post` from a queue per URL, recording the calls. */ +function fakeApi(answers) { + const calls = []; + const answer = (method, url) => { + calls.push(`${method} ${url}`); + const queue = answers[`${method} ${url}`]; + if (!queue?.length) throw new Error(`unexpected ${method} ${url}`); + const next = queue.shift(); + return next instanceof Error ? Promise.reject(next) : Promise.resolve(next); + }; + return { calls, get: url => answer('GET', url), post: url => answer('POST', url) }; +} + +const JOB = '0123456789abcdef0123456789abcdef'; +const job = (state, id = JOB, kind = 'health') => ({ id, kind, state }); + +/** + * A fake Activity store holding `jobs`: `track(job)` lists a started job, `load()` applies the next of `reads` (the + * server's job lists) and `push(job)` is a stream update; each notifies as the real store does. It never polls. + */ +function fakeActivity({ jobs = [], reads = [] } = {}) { + const listeners = new Set(); + const emit = () => { for (const listener of [...listeners]) listener(); }; + const store = { + loads: 0, + find: id => jobs.find(item => item.id === id), + running: () => jobs.filter(item => item.state === 'running'), + subscribe(listener) { listeners.add(listener); return () => listeners.delete(listener); }, + listeners: () => listeners.size, + async load() { + store.loads += 1; + if (reads.length) jobs = reads.shift(); + emit(); + }, + track(item) { + if (!item?.id) throw new TypeError('Invalid job'); + store.push(item); + return item; + }, + push(item) { + jobs = [...jobs.filter(other => other.id !== item.id), item]; + emit(); + }, + }; + return store; +} + +test('a health check tracks its job in the Activity store, then reads the report', async () => { + const { runHealthCheck } = await overviewModule; + const api = fakeApi({ + 'POST /prompt_manager/system/health': [{ job: job('running') }], + 'GET /prompt_manager/system/health': [{ report: REPORT }], + }); + const activity = fakeActivity(); + const checking = runHealthCheck(api, { activity }); + await new Promise(resolve => setImmediate(resolve)); + // The store follows the started job's event stream; nothing reads or polls the job here. + assert.equal(activity.loads, 0); + assert.equal(activity.find(JOB).state, 'running'); + assert.deepEqual(api.calls, ['POST /prompt_manager/system/health']); + activity.push(job('finished')); + assert.equal(await checking, REPORT); + assert.deepEqual(api.calls, ['POST /prompt_manager/system/health', 'GET /prompt_manager/system/health']); + assert.equal(activity.listeners(), 0); +}); + +test('a check already running elsewhere is waited for, not refused', async () => { + const { runHealthCheck } = await overviewModule; + const busy = Object.assign(new Error('busy'), { status: 409, code: 'busy' }); + const api = fakeApi({ + 'POST /prompt_manager/system/health': [busy], + 'GET /prompt_manager/system/health': [{ report: REPORT }], + }); + const activity = fakeActivity({ reads: [[job('running', 'f'.repeat(32), 'scan'), job('running')]] }); + const checking = runHealthCheck(api, { activity }); + await new Promise(resolve => setImmediate(resolve)); + assert.equal(activity.loads, 1); + activity.push(job('finished')); + assert.equal(await checking, REPORT); + // The other check ended between the refusal and the read: its report is the latest. + const ended = fakeApi({ 'POST /prompt_manager/system/health': [busy], 'GET /prompt_manager/system/health': [{ report: REPORT }] }); + assert.equal(await runHealthCheck(ended, { activity: fakeActivity({ reads: [[]] }) }), REPORT); +}); + +test('a failed, lost or reportless check rejects; other errors pass through', async () => { + const { runHealthCheck, waitForJob, HEALTH_FAILED_TEXT } = await overviewModule; + const failed = fakeApi({ 'POST /prompt_manager/system/health': [{ job: job('failed') }] }); + await assert.rejects(runHealthCheck(failed, { activity: fakeActivity() }), { message: HEALTH_FAILED_TEXT }); + // The store marks a job lost when the server no longer lists it (ComfyUI restarted). + const restarted = fakeActivity({ jobs: [job('running')] }); + const waiting = waitForJob(restarted, JOB); + restarted.push(job('lost')); + assert.equal(await waiting, 'lost'); + const empty = fakeApi({ 'POST /prompt_manager/system/health': [{ job: job('finished') }], + 'GET /prompt_manager/system/health': [{ report: null }] }); + await assert.rejects(runHealthCheck(empty, { activity: fakeActivity() }), { message: HEALTH_FAILED_TEXT }); + const broken = fakeApi({ 'POST /prompt_manager/system/health': [Object.assign(new Error('server'), { status: 500 })] }); + await assert.rejects(runHealthCheck(broken, { activity: fakeActivity() }), { message: 'server' }); + const invalid = fakeApi({ 'POST /prompt_manager/system/health': [{ job: null }] }); + await assert.rejects(runHealthCheck(invalid, { activity: fakeActivity() }), { message: 'Invalid job' }); +}); + +test('waiting stops when its signal aborts', async () => { + const { waitForJob } = await overviewModule; + const controller = new AbortController(); + const activity = fakeActivity({ jobs: [job('running')] }); + const waiting = waitForJob(activity, JOB, { signal: controller.signal }); + await new Promise(resolve => setImmediate(resolve)); + controller.abort(new Error('left')); + await assert.rejects(waiting, { message: 'left' }); + assert.equal(activity.listeners(), 0); + const gone = new AbortController(); + gone.abort(new Error('already')); + await assert.rejects(waitForJob(fakeActivity(), JOB, { signal: gone.signal }), { message: 'already' }); +}); diff --git a/tests/js/workspace-system-state.test.js b/tests/js/workspace-system-state.test.js new file mode 100644 index 00000000..a9e09118 --- /dev/null +++ b/tests/js/workspace-system-state.test.js @@ -0,0 +1,261 @@ +// System draft store (#249 System S2 Task 5): one Save row for browser, host and runtime fields (concept app.js +// renderSystem `.save-row` 705, `save-settings` 1320). It saves browser, then host with the GET revision, then runtime; +// a domain that fails keeps its fields dirty, and the status copy (D92) says what really happened. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const source = import('../../web/js/workspace/system-state.mjs'); + +/** Adapters that record every call; `host` answers like POST /prompt_manager/settings. */ +function adapters({ browser = {}, host = {}, runtime = {}, revision = 'sha256:one' } = {}) { + const calls = []; + const outcome = { browser: null, host: null, runtime: null }; + return { + calls, + outcome, + readBrowser: () => { calls.push(['readBrowser']); return { ...browser }; }, + readHost: async () => { calls.push(['readHost']); return { values: { ...host }, revision }; }, + readRuntime: async () => { calls.push(['readRuntime']); return { ...runtime }; }, + writeBrowser: async changes => { + calls.push(['writeBrowser', changes]); + if (outcome.browser) return outcome.browser(changes); + return { persisted: true }; + }, + writeHost: async (changes, options) => { + calls.push(['writeHost', changes, options]); + if (outcome.host) return outcome.host(changes, options); + return { success: true, persisted: true, revision: 'sha256:two', runtime_applied: true }; + }, + writeRuntime: async changes => { + calls.push(['writeRuntime', changes]); + if (outcome.runtime) return outcome.runtime(changes); + return { success: true }; + }, + }; +} + +async function loaded(domains) { + const { createSystemDrafts } = await source; + const drafts = createSystemDrafts(domains); + await Promise.all(['browser', 'host', 'runtime'].map(domain => drafts.load(domain))); + domains.calls.length = 0; + return drafts; +} + +test('sections follow the concept order and the production hashes', async () => { + const { SYSTEM_SECTIONS, isSystemSection } = await source; + assert.deepEqual(SYSTEM_SECTIONS.map(([id, label]) => [id, label]), [ + ['overview', 'Overview'], ['appearance', 'Appearance'], ['sources', 'Sources & scans'], + ['integrations', 'Integrations'], ['tagging', 'Auto-tagging'], ['maintenance', 'Maintenance'], + ['backups', 'Backup & export'], ['diagnostics', 'Diagnostics & logs'], + ]); + assert.equal(isSystemSection('backups'), true); + for (const value of ['general', '', 'constructor', '__proto__', undefined]) assert.equal(isSystemSection(value), false, value); +}); + +test('drafts read baselines and report dirty only for changed values', async () => { + const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 } }); + const drafts = await loaded(domains); + assert.equal(drafts.get('browser', 'layout'), 'grid'); + assert.equal(drafts.get('host', 'result_timeout'), 30); + assert.equal(drafts.dirty(), false); + drafts.set('host', 'result_timeout', 45); + assert.equal(drafts.get('host', 'result_timeout'), 45); + assert.equal(drafts.dirty(), true); + assert.equal(drafts.dirty('host'), true); + assert.equal(drafts.dirty('browser'), false); + // Setting the saved value back is clean again. + drafts.set('host', 'result_timeout', 30); + assert.equal(drafts.dirty(), false); + assert.throws(() => drafts.set('elsewhere', 'x', 1), TypeError); + assert.throws(() => drafts.get('elsewhere', 'x'), TypeError); +}); + +test('save commits browser then host then runtime', async () => { + const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 }, runtime: { level: 'INFO' } }); + const drafts = await loaded(domains); + drafts.set('runtime', 'level', 'DEBUG'); + drafts.set('host', 'result_timeout', 45); + drafts.set('browser', 'layout', 'list'); + const outcome = await drafts.save(); + assert.deepEqual(outcome, { browser: 'saved', host: 'saved', runtime: 'applied' }); + assert.deepEqual(domains.calls, [ + ['writeBrowser', { layout: 'list' }], + ['writeHost', { result_timeout: 45 }, { expected_revision: 'sha256:one' }], + ['writeRuntime', { level: 'DEBUG' }], + ]); + assert.equal(drafts.dirty(), false); + // The next host save sends the revision the last one returned. + drafts.set('host', 'result_timeout', 60); + await drafts.save(); + assert.deepEqual(domains.calls.at(-1), ['writeHost', { result_timeout: 60 }, { expected_revision: 'sha256:two' }]); +}); + +test('host failure keeps host fields dirty and browser saved', async () => { + const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 } }); + const drafts = await loaded(domains); + domains.outcome.host = () => { throw Object.assign(new Error('Settings could not be written to config.json.'), { code: 'config-write-failed' }); }; + drafts.set('browser', 'layout', 'list'); + drafts.set('host', 'result_timeout', 45); + const outcome = await drafts.save(); + assert.deepEqual(outcome, { browser: 'saved', host: 'failed', runtime: 'clean' }); + assert.equal(drafts.dirty('browser'), false); + assert.equal(drafts.dirty('host'), true); + assert.equal(drafts.get('host', 'result_timeout'), 45); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + assert.deepEqual([...drafts.failed('browser')], []); + assert.equal(drafts.statusText(), 'Some changes were not saved. Review the highlighted settings.'); + // A save answered `persisted: false` without an error is not saved either. + domains.outcome.host = () => ({ success: false, persisted: false }); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'failed', runtime: 'clean' }); + // Editing the failed field clears its highlight and reads as unsaved again. + drafts.set('host', 'result_timeout', 50); + assert.deepEqual([...drafts.failed('host')], []); + assert.equal(drafts.statusText(), 'Unsaved changes.'); +}); + +test('stale revision reports conflict and keeps the draft', async () => { + const domains = adapters({ host: { result_timeout: 30 } }); + const drafts = await loaded(domains); + domains.outcome.host = () => { throw Object.assign(new Error('Settings changed since they were loaded.'), { code: 'settings-changed' }); }; + drafts.set('host', 'result_timeout', 45); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'conflict', runtime: 'clean' }); + assert.equal(drafts.dirty('host'), true); + assert.equal(drafts.get('host', 'result_timeout'), 45); + assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.'); + // The conflict read the host values again (their new revision), keeping the draft. + assert.deepEqual(domains.calls.slice(-2).map(([name]) => name), ['writeHost', 'readHost']); + // Reloading the section reads the new revision and keeps the draft (Q10). + domains.outcome.host = null; + await drafts.load('host'); + assert.equal(drafts.get('host', 'result_timeout'), 45); + await drafts.save(); + assert.deepEqual(domains.calls.at(-1)[2], { expected_revision: 'sha256:one' }); +}); + +test('host save without a read revision never posts', async () => { + const { createSystemDrafts } = await source; + // Never read: an edit made before the host values (and their revision) arrived must not merge blindly. + const unread = adapters({ host: { result_timeout: 30 } }); + const drafts = createSystemDrafts(unread); + drafts.set('host', 'result_timeout', 45); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'failed', runtime: 'clean' }); + assert.deepEqual(unread.calls, []); + assert.equal(drafts.dirty('host'), true); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + // A read of an unreadable configuration has no revision either. + const unreadable = adapters({ host: { result_timeout: 30 }, revision: null }); + const blind = createSystemDrafts(unreadable); + await blind.load('host'); + blind.set('host', 'result_timeout', 45); + assert.equal((await blind.save()).host, 'failed'); + assert.equal(unreadable.calls.filter(([name]) => name === 'writeHost').length, 0); + // Once read, the same draft saves with the revision. + await drafts.load('host'); + assert.equal((await drafts.save()).host, 'saved'); + assert.deepEqual(unread.calls.at(-1), ['writeHost', { result_timeout: 45 }, { expected_revision: 'sha256:one' }]); +}); + +test('reloadHost reads the host values and says whether it could', async () => { + const { createSystemDrafts } = await source; + const domains = adapters({ host: { result_timeout: 30 } }); + const drafts = createSystemDrafts(domains); + assert.equal(await drafts.reloadHost(), true); + assert.equal(drafts.get('host', 'result_timeout'), 30); + const broken = createSystemDrafts({ ...domains, readHost: async () => { throw new Error('offline'); } }); + assert.equal(await broken.reloadHost(), false); + broken.set('host', 'result_timeout', 45); + assert.equal((await broken.save()).host, 'failed'); +}); + +test('no domain called when clean', async () => { + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = await loaded(domains); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'clean', runtime: 'clean' }); + assert.deepEqual(domains.calls, []); + drafts.set('browser', 'layout', 'list'); + drafts.discard(); + assert.equal(drafts.dirty(), false); + assert.equal(drafts.get('browser', 'layout'), 'grid'); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'clean', runtime: 'clean' }); + assert.deepEqual(domains.calls, []); +}); + +test('status text per outcome', async () => { + const { statusText, SETTINGS_STATUS } = await source; + assert.deepEqual(SETTINGS_STATUS, { + clean: 'No unsaved changes.', + dirty: 'Unsaved changes.', + browser: 'Saved in this browser.', + host: 'Saved to the server configuration.', + runtime: 'Applied until the server restarts.', + partial: 'Some changes were not saved. Review the highlighted settings.', + tabOnly: 'Saved for this tab only; browser storage is unavailable.', + conflict: 'Settings changed elsewhere. Reload this section to see them.', + }); + const clean = { browser: 'clean', host: 'clean', runtime: 'clean' }; + assert.equal(statusText(clean), 'No unsaved changes.'); + assert.equal(statusText({ ...clean, browser: 'saved' }), 'Saved in this browser.'); + assert.equal(statusText({ ...clean, host: 'saved' }), 'Saved to the server configuration.'); + assert.equal(statusText({ ...clean, runtime: 'applied' }), 'Applied until the server restarts.'); + assert.equal(statusText({ ...clean, browser: 'tab-only' }), 'Saved for this tab only; browser storage is unavailable.'); + assert.equal(statusText({ browser: 'saved', host: 'saved', runtime: 'applied' }), + 'Saved in this browser. Saved to the server configuration. Applied until the server restarts.'); + assert.equal(statusText({ ...clean, browser: 'saved', runtime: 'failed' }), + 'Some changes were not saved. Review the highlighted settings.'); + assert.equal(statusText({ browser: 'failed', host: 'conflict', runtime: 'clean' }), + 'Settings changed elsewhere. Reload this section to see them.'); +}); + +test('the store says unsaved, then the save outcome, and notifies subscribers', async () => { + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = await loaded(domains); + const seen = []; + const unsubscribe = drafts.subscribe(() => seen.push(drafts.statusText())); + assert.equal(drafts.statusText(), 'No unsaved changes.'); + drafts.set('browser', 'layout', 'list'); + domains.outcome.browser = () => ({ persisted: false }); + await drafts.save(); + assert.deepEqual(seen, ['Unsaved changes.', 'Saved for this tab only; browser storage is unavailable.']); + unsubscribe(); + drafts.set('browser', 'layout', 'masonry'); + assert.equal(seen.length, 2); + // The tab-only value is this tab's baseline now. + drafts.set('browser', 'layout', 'list'); + assert.equal(drafts.dirty(), false); +}); + +test('an edit made while a save runs stays dirty', async () => { + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = await loaded(domains); + let release; + domains.outcome.browser = () => new Promise(resolve => { release = () => resolve({ persisted: true }); }); + drafts.set('browser', 'layout', 'list'); + const saving = drafts.save(); + // A second Save while one runs joins it instead of writing twice. + assert.equal(drafts.save(), saving); + assert.equal(drafts.saving(), true); + await new Promise(resolve => setImmediate(resolve)); + drafts.set('browser', 'layout', 'masonry'); + release(); + assert.deepEqual(await saving, { browser: 'saved', host: 'clean', runtime: 'clean' }); + assert.equal(drafts.saving(), false); + assert.equal(drafts.dirty('browser'), true); + assert.equal(drafts.get('browser', 'layout'), 'masonry'); + assert.equal(domains.calls.filter(([name]) => name === 'writeBrowser').length, 1); +}); + +test('a failed read keeps the store usable and drafts survive a reload (Q10)', async () => { + const { createSystemDrafts } = await source; + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = createSystemDrafts(domains); + await drafts.load('browser'); + drafts.set('browser', 'layout', 'list'); + await drafts.load('browser'); + assert.equal(drafts.get('browser', 'layout'), 'list'); + assert.equal(drafts.dirty(), true); + const failing = createSystemDrafts({ ...domains, readHost: async () => { throw new Error('offline'); } }); + await assert.rejects(failing.load('host'), /offline/); + assert.equal(failing.dirty(), false); + assert.throws(() => createSystemDrafts({ ...domains, writeHost: null }), TypeError); +}); diff --git a/tests/js/workspace-toolbar.test.js b/tests/js/workspace-toolbar.test.js index 91e1c3b9..052bc0be 100644 --- a/tests/js/workspace-toolbar.test.js +++ b/tests/js/workspace-toolbar.test.js @@ -76,6 +76,7 @@ function fakeDOM() { get activeElement() { return active ?? body; }, createElement: tag => new Element(tag), createElementNS: (namespace, tag) => new Element(tag), + createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }), }; const all = (root, predicate, found = []) => { if (predicate(root)) found.push(root); @@ -790,3 +791,145 @@ test('the Filters button shows the active refinement count', async () => { dom.restore(); } }); + +// #249 System S2 Task 5: the scan review's dialog and backup box (D55) generalised for every record-changing review. +const scanModule = import('../../web/js/workspace/activity-scan.mjs'); +const bodyShape = dom => byId(dom, 'modal-body').children.map(node => [node.tagName.toLowerCase(), node.className].filter(Boolean).join('.')); + +test('backupNotice is the concept .notice with its copy and one Back up now action', async () => { + const { backupNotice, SCAN_BACKUP_COPY } = await dialogsModule; + const dom = fakeDOM(); + try { + let calls = 0; + const notice = backupNotice({ copy: SCAN_BACKUP_COPY, onBackup: () => { calls++; } }); + assert.equal(notice.className, 'notice'); + assert.deepEqual(tags(notice), ['div', 'div']); + assert.deepEqual(tags(notice.children[0]), ['h3', 'p']); + assert.equal(notice.children[0].children[0].textContent, 'Back up the database first (recommended)'); + assert.equal(notice.children[0].children[1].textContent, + 'Downloads a copy of the prompts database before the scan changes any records.'); + assert.equal(notice.children[1].className, 'row-actions'); + const action = button(notice, 'Back up now'); + action.click(); + assert.equal(calls, 1); + assert.throws(() => backupNotice({ copy: { heading: '', text: 'x' }, onBackup() {} }), TypeError); + assert.throws(() => backupNotice({ copy: SCAN_BACKUP_COPY }), TypeError); + } finally { + dom.restore(); + } +}); + +test('openReviewDialog lays out heading, grid, options, sentence, backup box and actions', async () => { + const { openReviewDialog, closeWorkspaceDialog } = await dialogsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const trigger = new dom.Element('button'); + dom.body.append(trigger); + const option = new dom.Element('div'); + option.className = 'setting-row'; + const started = []; + const review = openReviewDialog({ scope, title: 'Review maintenance plan', heading: 'Missing-file links', + grid: [['Scope', '2 stale image links'], ['Media on disk', 'No source files removed']], options: [option], + sentence: 'Cancelling leaves all data untouched.', backup: { copy: { heading: 'Back up first', text: 'A copy.' }, onBackup() {} }, + cancelLabel: 'Cancel', primary: { label: 'Apply reviewed operation' }, onStart: () => started.push('start'), trigger }); + assert.equal(byId(dom, 'modal-title').textContent, 'Review maintenance plan'); + assert.deepEqual(bodyShape(dom), ['h3', 'dl.meta-grid', 'div.setting-row', 'p.small', 'div.notice', 'div.dialog-actions']); + const grid = byId(dom, 'modal-body').children[1]; + assert.deepEqual(grid.children.map(entry => [tags(entry), entry.children[0].textContent, entry.children[1].textContent]), [ + [['dt', 'dd'], 'Scope', '2 stale image links'], [['dt', 'dd'], 'Media on disk', 'No source files removed']]); + review.setValue('Scope', '3 stale image links'); + assert.equal(grid.children[0].children[1].textContent, '3 stale image links'); + const actions = byId(dom, 'modal-body').children.at(-1); + assert.deepEqual(actions.children.map(node => [node.textContent, node.className]), + [['Cancel', ''], ['Apply reviewed operation', 'ca-primary']]); + actions.children[1].click(); + assert.deepEqual(started, ['start']); + assert.equal(modal(dom).open, false); + assert.equal(document.activeElement, trigger); + + // A danger primary (the concept's Restore) has no sentence, heading, options or backup box when none is given. + openReviewDialog({ scope, title: 'Restore preview', grid: [['Integrity', 'Verified']], backup: null, + cancelLabel: 'Cancel', primary: { label: 'Restore', danger: true }, onStart() {} }); + assert.deepEqual(bodyShape(dom), ['dl.meta-grid', 'div.dialog-actions']); + assert.equal(button(byId(dom, 'modal-body'), 'Restore').className, 'danger'); + button(byId(dom, 'modal-body'), 'Cancel').click(); + assert.equal(modal(dom).open, false); + assert.throws(() => openReviewDialog({ scope, title: 'x', grid: [['a']], cancelLabel: 'Cancel', primary: { label: 'Go' }, + onStart() {} }), TypeError); + assert.throws(() => openReviewDialog({ scope, title: 'x', grid: [], cancelLabel: 'Cancel', primary: {}, onStart() {} }), TypeError); + } finally { + scope.dispose(); + closeWorkspaceDialog(); + dom.restore(); + } +}); + +test('review dialog backup box keeps the dialog open and toasts the outcome', async () => { + const { openReviewDialog, closeWorkspaceDialog, SCAN_BACKUP_COPY } = await dialogsModule; + const { downloadDatabaseBackup, BACKUP_DONE, BACKUP_FAILED } = await scanModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + const created = URL.createObjectURL; + const revoked = URL.revokeObjectURL; + URL.createObjectURL = () => 'blob:backup'; + URL.revokeObjectURL = () => {}; + try { + const toasts = []; + let answer; + const fetchUrl = () => new Promise(resolve => { answer = resolve; }); + openReviewDialog({ scope, title: 'Review metadata scan', grid: [['Sources', '2 output folders']], + backup: { copy: SCAN_BACKUP_COPY, onBackup: () => downloadDatabaseBackup({ fetchUrl, notify: text => toasts.push(text) }) }, + cancelLabel: 'Cancel', primary: { label: 'Start scan' }, onStart() { throw new Error('not started'); } }); + const action = button(byId(dom, 'modal-body'), 'Back up now'); + action.click(); + assert.equal(action.textContent, 'Backing up…'); + assert.equal(action.getAttribute('aria-disabled'), 'true'); + // A second press while the copy downloads is ignored. + action.click(); + answer({ ok: true, blob: async () => new Blob(['db']), headers: { get: () => 'attachment; filename="prompts_backup_1.db"' } }); + await tick(); + assert.deepEqual(toasts, [BACKUP_DONE]); + assert.equal(modal(dom).open, true); + assert.equal(action.textContent, 'Back up now'); + assert.equal(action.getAttribute('aria-disabled'), null); + action.click(); + answer({ ok: false, status: 500 }); + await tick(); + assert.deepEqual(toasts, [BACKUP_DONE, BACKUP_FAILED]); + assert.equal(modal(dom).open, true); + } finally { + URL.createObjectURL = created; + URL.revokeObjectURL = revoked; + scope.dispose(); + closeWorkspaceDialog(); + dom.restore(); + } +}); + +test('the scan review is a thin openReviewDialog caller with its scope switch', async () => { + const { openScanReviewDialog, closeWorkspaceDialog } = await dialogsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const starts = []; + openScanReviewDialog({ scope, sources: 2, onBackup() {}, onStart: value => starts.push(value) }); + assert.equal(byId(dom, 'modal-title').textContent, 'Review metadata scan'); + assert.deepEqual(bodyShape(dom), ['dl.meta-grid', 'div.setting-row', 'p.small', 'div.notice', 'div.dialog-actions']); + const grid = byId(dom, 'modal-body').children[0]; + assert.deepEqual(grid.children.map(entry => entry.textContent), ['Sources2 output folders', 'ScopeNew and unresolved files']); + const allFiles = byId(dom, 'scan-all-files'); + allFiles.checked = true; + allFiles.dispatchEvent(new Event('change')); + assert.equal(grid.children[1].children[1].textContent, 'All files'); + button(byId(dom, 'modal-body'), 'Start scan').click(); + assert.deepEqual(starts, [{ scope: 'all' }]); + } finally { + scope.dispose(); + closeWorkspaceDialog(); + dom.restore(); + } +}); diff --git a/tests/test_database_health.py b/tests/test_database_health.py new file mode 100644 index 00000000..2c0b40f0 --- /dev/null +++ b/tests/test_database_health.py @@ -0,0 +1,614 @@ +"""The read-only health check (System › Overview and Maintenance). + +``run_health_checks`` reads one snapshot on its own ``mode=ro`` connection: +SQLite's integrity and foreign-key checks, stored prompt hashes against +``utils.hashing``, duplicate pair rows, the legacy consistency queries, image +links whose file is gone (only under readable sources) and the Needs-review +count when one is supplied. It never writes, and it never turns a check it +could not run into a count. +""" + +import hashlib +import ntpath +import os +import sqlite3 +import sys +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +from database import health # noqa: E402 +from database.operations import PromptDatabase # noqa: E402 +from utils.hashing import generate_prompt_hash # noqa: E402 + +try: + from tests.open_handles import assert_closed # noqa: E402 +except ImportError: # discovered with tests/ as the top-level directory + from open_handles import assert_closed # noqa: E402 + +CHECK_IDS = [ + "integrity", + "foreign_keys", + "canonical_hashes", + "pair_uniqueness", + "missing_files", + "ambiguous_metadata", + "consistency", +] + + +def _digest(path): + if not os.path.exists(path): + return None + with open(path, "rb") as handle: + return hashlib.sha256(handle.read()).hexdigest() + + +class HealthTestCase(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.root = Path(os.path.realpath(self.temp.name)) + self.output = self.root / "output" + self.output.mkdir() + self.db_path = str(self.root / "prompts.db") + self.db = PromptDatabase(self.db_path) + + def tearDown(self): + self.db.close_all() + assert_closed(self, self.root) + self.temp.cleanup() + + def save(self, text, **metadata): + return self.db.save_prompt( + text, prompt_hash=generate_prompt_hash(text), **metadata + ) + + def image(self, prompt_id, name, *, exists=True, folder=None): + folder = Path(folder or self.output) + path = folder / name + if exists: + path.write_bytes(b"png") + return self.db.link_image_to_prompt(prompt_id, str(path)) + + def run_checks(self, roots=None, **kwargs): + roots = [str(self.output)] if roots is None else roots + return self.db.run_health_checks(roots, **kwargs) + + @staticmethod + def check(report, check_id): + return next(c for c in report["checks"] if c["id"] == check_id) + + def raw(self): + """A plain connection for planting damage the app itself never writes.""" + conn = sqlite3.connect(self.db_path) + conn.execute("PRAGMA foreign_keys=OFF") + return conn + + +class TestHealthReport(HealthTestCase): + def test_clean_library_passes_except_unavailable_review(self): + prompt = self.save("A clean prompt") + self.db.save_prompt_pair("Cat", "Blur") + self.image(prompt, "a.png") + report = self.run_checks(review_count=0) + self.assertTrue(report["success"]) + self.assertEqual([c["id"] for c in report["checks"]], CHECK_IDS) + self.assertEqual( + [c["label"] for c in report["checks"]], + [ + "Integrity check", + "Foreign keys & pair references", + "Canonical hashes", + "Pair uniqueness", + "Missing files", + "Ambiguous metadata", + "Consistency", + ], + ) + for check in report["checks"]: + self.assertEqual(check["status"], "passed", check) + self.assertEqual(check["count"], 0, check) + self.assertEqual(report["status"], "passed") + self.assertRegex(report["checked_at"], r"^\d{4}-\d\d-\d\dT.*\+00:00$") + + def test_health_check_never_writes(self): + prompt = self.save("Untouched") + self.image(prompt, "kept.png") + self.image(prompt, "gone.png", exists=False) + with self.raw() as conn: + conn.execute("UPDATE prompts SET hash='stale' WHERE id=?", (prompt,)) + conn.close() + # The app's own connection stays open, as in the running server. + self.db.get_statistics() + files = [self.db_path, self.db_path + "-wal"] + before = [_digest(path) for path in files] + shm_size = os.path.getsize(self.db_path + "-shm") + media = sorted(os.listdir(self.output)) + opened = [] + real_connect = sqlite3.connect + + class Recording(sqlite3.Connection): + changes_at_close = None + + def close(self): + Recording.changes_at_close = self.total_changes + super().close() + + def connect(*args, **kwargs): + conn = real_connect(*args, factory=Recording, **kwargs) + opened.append((args, kwargs, conn)) + return conn + + with patch.object(health.sqlite3, "connect", side_effect=connect): + report = self.run_checks() + self.assertEqual(report["status"], "incomplete") + self.assertEqual(len(opened), 1) + args, kwargs, conn = opened[0] + self.assertTrue(str(args[0]).endswith("?mode=ro"), args) + self.assertTrue(kwargs.get("uri")) + self.assertEqual(Recording.changes_at_close, 0) + with self.assertRaises(sqlite3.ProgrammingError): # closed in finally + conn.execute("SELECT 1") + self.assertEqual([_digest(path) for path in files], before) + # The -shm holds WAL read marks every reader updates; it never grows. + self.assertEqual(os.path.getsize(self.db_path + "-shm"), shm_size) + self.assertEqual(sorted(os.listdir(self.output)), media) + with self.db.model.get_connection() as live: + stored = live.execute( + "SELECT hash FROM prompts WHERE id=?", (prompt,) + ).fetchone()[0] + self.assertEqual(stored, "stale") + + def test_connection_closed_when_a_check_raises(self): + self.save("Any") + with ( + patch.object( + health, + "_canonical_hashes", + side_effect=sqlite3.OperationalError("boom"), + ), + self.assertLogs("prompt_manager.database.health", "ERROR"), + ): + report = self.run_checks(review_count=0) + canonical = self.check(report, "canonical_hashes") + self.assertEqual(canonical["status"], "failed") + self.assertIsNone(canonical["count"]) + self.assertEqual(canonical["detail"], ["The check could not finish."]) + self.assertEqual(self.check(report, "integrity")["status"], "passed") + self.assertEqual(report["status"], "incomplete") + self.db.close_all() + assert_closed(self, self.root) + + def test_unopenable_database_fails_every_database_check(self): + missing = PromptDatabase.__new__(PromptDatabase) + missing.model = type("M", (), {"db_path": str(self.root / "nope.db")})() + with self.assertLogs("prompt_manager.database.health", "ERROR"): + report = missing.run_health_checks([str(self.output)], review_count=0) + statuses = {c["id"]: c["status"] for c in report["checks"]} + self.assertEqual(statuses["integrity"], "failed") + self.assertEqual(statuses["missing_files"], "failed") + self.assertEqual(statuses["ambiguous_metadata"], "passed") + self.assertEqual(report["status"], "incomplete") + self.assertFalse(os.path.exists(self.root / "nope.db")) + + def test_integrity_and_foreign_key_issues_reported(self): + prompt = self.save("Linked") + with self.raw() as conn: + conn.execute("CREATE TABLE probe(a)") + conn.execute("CREATE INDEX probe_a ON probe(a)") + conn.executemany("INSERT INTO probe VALUES(?)", [(i,) for i in range(16)]) + conn.execute( + "INSERT INTO generated_images(prompt_id,image_path,filename,file_path)" + " VALUES(?,?,?,?)", + (9999, str(self.output / "orphan.png"), "orphan.png", "orphan"), + ) + conn.execute( + "INSERT INTO prompt_tags(prompt_id,tag_id) VALUES(?,?)", (prompt, 777) + ) + conn.close() + with self.raw() as conn: + conn.execute("PRAGMA writable_schema=ON") + conn.execute( + "UPDATE sqlite_master SET sql='CREATE INDEX probe_a ON probe(a DESC)'" + " WHERE name='probe_a'" + ) + conn.close() + self.db.close_all() # the next read sees the edited schema + (self.output / "orphan.png").write_bytes(b"png") + with self.raw() as conn: + expected = [r[0] for r in conn.execute("PRAGMA integrity_check(100)")] + conn.close() + self.assertGreater(len(expected), 10) + report = self.run_checks(review_count=0) + integrity = self.check(report, "integrity") + self.assertEqual(integrity["status"], "issues") + self.assertEqual(integrity["count"], len(expected)) + self.assertEqual(integrity["detail"], expected[:10]) + self.assertIn("missing from index probe_a", integrity["detail"][0]) + foreign = self.check(report, "foreign_keys") + self.assertEqual(foreign["status"], "issues") + self.assertEqual(foreign["count"], 2) + self.assertEqual( + sorted(foreign["detail"]), + [ + "generated_images row 1 references a missing prompts row", + "prompt_tags row 1 references a missing tags row", + ], + ) + consistency = self.check(report, "consistency") + self.assertEqual(consistency["status"], "issues") + self.assertEqual(consistency["count"], 1) + self.assertEqual( + consistency["detail"], ["Image 1 references non-existent prompt 9999"] + ) + self.assertEqual(report["status"], "issues") + + def test_canonical_hash_mismatch_counts_rows(self): + ids = [self.save(f"Prompt {n}") for n in range(14)] + with self.raw() as conn: + for n, prompt_id in enumerate(ids[:12]): + conn.execute( + "UPDATE prompts SET hash=? WHERE id=?", (f"bad{n}", prompt_id) + ) + conn.execute("UPDATE prompts SET hash=NULL WHERE id=?", (ids[12],)) + conn.close() + # Normalised spellings hash alike: this one is not a mismatch. + self.save(" MIXED Case ") + report = self.run_checks(review_count=0) + canonical = self.check(report, "canonical_hashes") + self.assertEqual(canonical["status"], "issues") + self.assertEqual(canonical["count"], 13) + self.assertEqual(len(canonical["detail"]), 10) + self.assertEqual( + canonical["detail"][0], + f"Prompt {ids[0]}: the stored hash does not match its text", + ) + + def test_duplicate_pair_rows_reported(self): + pair = self.db.save_prompt_pair("Cat", "") + positive = self.db.get_prompt_pair(pair)["positive_prompt_id"] + self.db.save_prompt_pair("Dog", "Blur") + with self.raw() as conn: + # Only the partial index keeps positive-only pairs unique; a restored + # or hand-edited database may have lost it. + conn.execute("DROP INDEX idx_prompt_pairs_positive_only") + conn.executemany( + "INSERT INTO prompt_pairs(positive_prompt_id,negative_prompt_id)" + " VALUES(?,NULL)", + [(positive,), (positive,)], + ) + conn.close() + report = self.run_checks(review_count=0) + pairs = self.check(report, "pair_uniqueness") + self.assertEqual(pairs["status"], "issues") + self.assertEqual(pairs["count"], 2) + self.assertEqual( + pairs["detail"], + [f"Positive prompt {positive} with no negative: 3 pair rows"], + ) + + def test_integrity_count_at_the_limit_is_flagged_capped(self): + class Rows: + def __init__(self, rows): + self.rows = rows + + def execute(self, _sql): + return iter([(row,) for row in self.rows]) + + capped = health._integrity(Rows(["bad"] * health.INTEGRITY_LIMIT), []) + self.assertEqual(capped["status"], "issues") + self.assertEqual(capped["count"], health.INTEGRITY_LIMIT) + self.assertTrue(capped["capped"]) + exact = health._integrity(Rows(["bad"] * 3), []) + self.assertEqual(exact["count"], 3) + self.assertFalse(exact["capped"]) + self.assertFalse(health._integrity(Rows(["ok"]), [])["capped"]) + + def test_an_unexpected_error_fails_only_its_own_check(self): + self.save("Any") + with ( + patch.object(health, "_pair_uniqueness", side_effect=KeyError("boom")), + self.assertLogs("prompt_manager.database.health", "ERROR"), + ): + report = self.run_checks(review_count=0) + statuses = {c["id"]: c["status"] for c in report["checks"]} + self.assertEqual(statuses.pop("pair_uniqueness"), "failed") + self.assertEqual(set(statuses.values()), {"passed"}) + + +class TestDatabaseUri(unittest.TestCase): + """The ``mode=ro`` URI of the database file, for every path form SQLite opens.""" + + def test_posix_path_is_a_local_file_uri(self): + self.assertEqual( + health.readonly_uri("/data/my db#1?.db", windows=False), + "file:///data/my%20db%231%3F.db?mode=ro", + ) + + def test_windows_drive_path(self): + self.assertEqual( + health.readonly_uri(r"C:\Users\Me\prompts.db", windows=True), + "file:///C:/Users/Me/prompts.db?mode=ro", + ) + + def test_windows_unc_path_has_an_empty_authority(self): + # pathlib spells \\server\share as file://server/share, which SQLite + # rejects (a non-local authority); an empty one keeps the UNC path. + uri = health.readonly_uri(r"\\server\share\Comfy UI\prompts.db", windows=True) + self.assertEqual(uri, "file:////server/share/Comfy%20UI/prompts.db?mode=ro") + self.assertTrue(ntpath.isabs(r"\\server\share\Comfy UI\prompts.db")) + + def test_uri_opens_a_path_with_uri_characters(self): + with tempfile.TemporaryDirectory() as temp: + # "?" is not a valid Windows file name character; "#", "%" and spaces are. + name = "a #b%20 c.db" if os.name == "nt" else "a #b?c%20.db" + path = os.path.join(os.path.realpath(temp), name) + conn = sqlite3.connect(path) + try: + conn.execute("CREATE TABLE t(a)") + conn.commit() + finally: + conn.close() + ro = sqlite3.connect(health.readonly_uri(path), uri=True) + try: + self.assertEqual(ro.execute("SELECT COUNT(*) FROM t").fetchone(), (0,)) + with self.assertRaises(sqlite3.OperationalError): + ro.execute("INSERT INTO t VALUES(1)") + finally: + ro.close() + + @unittest.skipUnless(os.name == "nt", "a native UNC path exists on Windows only") + def test_native_unc_path_opens(self): + with tempfile.TemporaryDirectory() as temp: + path = os.path.join(os.path.realpath(temp), "prompts.db") + conn = sqlite3.connect(path) + try: + conn.execute("CREATE TABLE t(a)") + conn.commit() + finally: + conn.close() + drive, rest = ntpath.splitdrive(path) + unc = "\\\\localhost\\" + drive[0] + "$" + rest + if not os.path.exists(unc): + self.skipTest("the administrative share is not reachable here") + ro = sqlite3.connect(health.readonly_uri(unc), uri=True) + try: + self.assertEqual(ro.execute("SELECT COUNT(*) FROM t").fetchone(), (0,)) + finally: + ro.close() + + +class TestMissingFiles(HealthTestCase): + def test_missing_files_counted_under_readable_sources_only(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + for n in range(12): + self.image(prompt, f"gone-{n}.png", exists=False) + elsewhere = self.root / "elsewhere" + elsewhere.mkdir() + self.image(prompt, "outside.png", exists=False, folder=elsewhere) + report = self.run_checks(review_count=0) + missing = self.check(report, "missing_files") + self.assertEqual(missing["status"], "issues") + self.assertEqual(missing["count"], 12) + self.assertEqual(len(missing["detail"]), 10) + self.assertRegex(missing["detail"][0], r"^Image \d+: gone-0\.png is missing$") + self.assertNotIn(str(self.root), " ".join(missing["detail"])) + with self.db.model.get_connection() as conn: + self.assertEqual( + conn.execute("SELECT COUNT(*) FROM generated_images").fetchone()[0], + 14, + ) + + def test_links_outside_every_source_are_named_not_counted(self): + prompt = self.save("Images") + elsewhere = self.root / "elsewhere" + elsewhere.mkdir() + self.image(prompt, "outside.png", exists=False, folder=elsewhere) + missing = self.check(self.run_checks(review_count=0), "missing_files") + self.assertEqual(missing["status"], "passed") + self.assertEqual(missing["count"], 0) + self.assertEqual( + missing["detail"], + ["1 link outside the image sources was not checked."], + ) + + def test_missing_file_vs_unavailable_root(self): + # Per source (S2 ruling): the readable source is counted, the absent + # one reports "unavailable" with no count, and the report is incomplete. + prompt = self.save("Images") + absent = self.root / "unmounted" + absent.mkdir() + self.image(prompt, "on-nas.png", folder=absent) + self.image(prompt, "gone.png", exists=False) + os.unlink(absent / "on-nas.png") + absent.rmdir() + report = self.run_checks([str(self.output), str(absent)], review_count=0) + missing = self.check(report, "missing_files") + self.assertEqual(missing["status"], "issues") + self.assertEqual(missing["count"], 1) + self.assertTrue(missing["partial"]) + self.assertEqual( + missing["sources"], + [ + {"name": "output", "status": "checked", "count": 1}, + {"name": "unmounted", "status": "unavailable", "count": None}, + ], + ) + self.assertEqual( + missing["detail"][0], + "1 image source is unavailable, so its files were not counted.", + ) + self.assertRegex(missing["detail"][1], r"^Image \d+: gone\.png is missing$") + # The link under the unavailable source is neither counted nor "outside". + self.assertEqual(len(missing["detail"]), 2) + self.assertNotIn(str(self.root), " ".join(missing["detail"])) + self.assertEqual(report["status"], "incomplete") + + def test_partial_sources_with_no_missing_file_are_not_passed(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + absent = self.root / "unmounted" + missing = self.check( + self.run_checks([str(self.output), str(absent)], review_count=0), + "missing_files", + ) + self.assertEqual(missing["status"], "unavailable") + self.assertEqual(missing["count"], 0) + self.assertTrue(missing["partial"]) + self.assertEqual( + [s["status"] for s in missing["sources"]], ["checked", "unavailable"] + ) + + def test_every_source_unavailable_counts_nothing(self): + prompt = self.save("Images") + absent = self.root / "unmounted" + self.image(prompt, "gone.png", exists=False, folder=self.output) + report = self.run_checks([str(absent)], review_count=0) + missing = self.check(report, "missing_files") + self.assertEqual(missing["status"], "unavailable") + self.assertIsNone(missing["count"]) + self.assertFalse(missing["partial"]) + self.assertEqual( + missing["sources"], + [{"name": "unmounted", "status": "unavailable", "count": None}], + ) + self.assertEqual( + missing["detail"], + ["1 image source is unavailable, so missing files were not counted."], + ) + + def test_each_source_counts_its_own_missing_files(self): + prompt = self.save("Images") + archive = self.root / "archive" + archive.mkdir() + self.image(prompt, "a.png", exists=False) + self.image(prompt, "b.png", exists=False, folder=archive) + self.image(prompt, "c.png", exists=False, folder=archive) + missing = self.check( + self.run_checks([str(self.output), str(archive)], review_count=0), + "missing_files", + ) + self.assertEqual(missing["count"], 3) + self.assertFalse(missing["partial"]) + self.assertEqual( + [(s["name"], s["count"]) for s in missing["sources"]], + [("output", 1), ("archive", 2)], + ) + + def test_no_sources_is_unavailable(self): + self.save("Images") + missing = self.check(self.run_checks([], review_count=0), "missing_files") + self.assertEqual(missing["status"], "unavailable") + self.assertIsNone(missing["count"]) + self.assertEqual(missing["detail"], ["No image source is available."]) + self.assertEqual(missing["sources"], []) + self.assertFalse(missing["partial"]) + + def test_a_bad_stored_path_fails_only_missing_files(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + with self.raw() as conn: + conn.execute( + "INSERT INTO generated_images(prompt_id,image_path,filename)" + " VALUES(?,?,?)", + (prompt, str(self.root / "else\x00where.png"), "bad.png"), + ) + conn.close() + real_realpath = os.path.realpath + + def realpath(path, *args, **kwargs): + # POSIX (and Windows before Python 3.11) raise for the NUL byte; + # newer ntpath returns it unchanged. Raise everywhere, as POSIX does. + if "\x00" in str(path): + raise ValueError("embedded null byte") + return real_realpath(path, *args, **kwargs) + + with ( + patch.object(health.os.path, "realpath", side_effect=realpath), + self.assertLogs("prompt_manager.database.health", "ERROR"), + ): + report = self.run_checks(review_count=0) + statuses = {c["id"]: c["status"] for c in report["checks"]} + self.assertEqual(statuses.pop("missing_files"), "failed") + self.assertEqual(set(statuses.values()), {"passed"}) + + def test_files_are_checked_after_the_read_snapshot_ends(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + self.image(prompt, "gone.png", exists=False) + opened = [] + real_connect = sqlite3.connect + + def connect(*args, **kwargs): + conn = real_connect(*args, **kwargs) + opened.append(conn) + return conn + + real_exists = os.path.exists + in_snapshot = [] + + def exists(path): + for conn in opened: + try: + in_snapshot.append(conn.in_transaction) + except sqlite3.ProgrammingError: # closed: no snapshot + in_snapshot.append(False) + return real_exists(path) + + with ( + patch.object(health.sqlite3, "connect", side_effect=connect), + patch.object(health.os.path, "exists", side_effect=exists), + ): + missing = self.check(self.run_checks(review_count=0), "missing_files") + self.assertEqual((missing["status"], missing["count"]), ("issues", 1)) + self.assertEqual(len(opened), 1) + self.assertTrue(in_snapshot) + self.assertNotIn(True, in_snapshot) + + +class TestAmbiguousMetadata(HealthTestCase): + def test_ambiguous_metadata_unavailable_without_review_queue(self): + self.save("Any") + report = self.run_checks() + ambiguous = self.check(report, "ambiguous_metadata") + self.assertEqual(ambiguous["status"], "unavailable") + self.assertIsNone(ambiguous["count"]) + self.assertEqual(ambiguous["detail"], ["Needs review is not available yet."]) + self.assertEqual(report["status"], "incomplete") + + def test_review_count_reports_images_needing_review(self): + report = self.run_checks(review_count=3) + ambiguous = self.check(report, "ambiguous_metadata") + self.assertEqual(ambiguous["status"], "issues") + self.assertEqual(ambiguous["count"], 3) + self.assertEqual(ambiguous["detail"], []) + self.assertEqual(report["status"], "issues") + + def test_invalid_review_count_is_rejected(self): + for value in (-1, True, "3", 1.5): + with self.assertRaises(ValueError): + self.run_checks(review_count=value) + + +class TestOverallStatus(unittest.TestCase): + def test_overall_status_precedence(self): + def checks(*statuses): + return [{"status": status} for status in statuses] + + self.assertEqual(health.overall_status(checks("passed", "passed")), "passed") + self.assertEqual(health.overall_status(checks("passed", "issues")), "issues") + self.assertEqual( + health.overall_status(checks("issues", "unavailable")), "incomplete" + ) + self.assertEqual(health.overall_status(checks("failed")), "incomplete") + # A check that found issues under only some sources is incomplete too. + partial = [{"status": "issues", "partial": True}, {"status": "passed"}] + self.assertEqual(health.overall_status(partial), "incomplete") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_library.py b/tests/test_library.py index fd83cb7c..242b76d4 100644 --- a/tests/test_library.py +++ b/tests/test_library.py @@ -1706,6 +1706,103 @@ class TestLibrary(unittest.TestCase): self.db.list_library_images(positive, pair_id=foreign, archive="all") +class LibraryCounts(unittest.TestCase): + """System › Overview counts and usage statistics follow Library eligibility.""" + + setUp = TestLibrary.setUp + tearDown = TestLibrary.tearDown + save = TestLibrary.save + state = TestLibrary.state + + def test_counts_use_library_eligibility(self): + self.db.save_prompt_pair("Cat", "Blur") + archived = self.db.save_prompt_pair("Cat", "Noise") + self.db.archive_prompt_pair(archived) + titled_pair = self.db.save_prompt_pair("Dog", "Titled negative") + titled = self.db.get_prompt_pair(titled_pair)["negative_prompt_id"] + self.state(titled, "A titled parent") # zero pairs of its own: counted + legacy = self.save("Legacy") + self.db.link_image_to_prompt(legacy, str(Path(self.temp.name) / "a.png")) + self.db.link_image_to_prompt(legacy, str(Path(self.temp.name) / "b.png")) + counts = self.db.library_counts() + # Cat, Dog, the titled negative and Legacy; "Blur" and "Noise" are + # negative-only. + self.assertEqual(counts["positive_prompts"], 4) + # Cat+Blur and Dog+Titled negative; the archived pair is excluded. + self.assertEqual(counts["negative_variants"], 2) + self.assertEqual(counts["images"], 2) + self.assertEqual(counts["rated_images"], 0) + listed = self.db.list_library() + self.assertEqual(counts["positive_prompts"], listed["total"]) + self.assertEqual(counts["negative_variants"], listed["variant_total"]) + + def test_counts_of_an_empty_library_are_zero(self): + self.assertEqual( + self.db.library_counts(), + { + "positive_prompts": 0, + "negative_variants": 0, + "images": 0, + "rated_images": 0, + }, + ) + + def test_rated_images_follow_their_linked_prompt_rating(self): + # An image carries no rating of its own: it shows its linked prompt's + # (the Images "Highest rated" sort), so "rated" counts those images. + folder = Path(self.temp.name) + rated = self.save("Rated", rating=4) + unrated = self.save("Unrated") + for name in ("a.png", "b.png", "c.png"): + self.db.link_image_to_prompt(rated, str(folder / name)) + self.db.link_image_to_prompt(unrated, str(folder / "d.png")) + counts = self.db.library_counts() + self.assertEqual(counts["images"], 4) + self.assertEqual(counts["rated_images"], 3) + self.db.update_prompt_rating(unrated, 2) + self.assertEqual(self.db.library_counts()["rated_images"], 4) + + def test_statistics_sum_real_run_counts(self): + a = self.save("Alpha", category="Landscapes", rating=5) + b = self.save("Beta", category="Landscapes", rating=4) + self.save("Gamma", category="Landscapes") # unrated: not in the average + c = self.save("Delta", category="Portraits", rating=3) + blank = self.save("Epsilon", category=" ") + uncategorised = self.save("Zeta") + self.db.record_prompt_use(a, times=3) + self.db.record_prompt_use(b, times=2) + self.db.record_prompt_use(c) + self.db.record_prompt_use(blank) + self.db.record_prompt_use(uncategorised, times=4) + # A negative-only prompt's runs are the pair's runs: never counted twice. + pair = self.db.save_prompt_pair("Alpha", "Only negative") + negative = self.db.get_prompt_pair(pair)["negative_prompt_id"] + with self.db.model.get_connection() as connection: + connection.execute( + "UPDATE prompts SET category='Landscapes',run_count=50 WHERE id=?", + (negative,), + ) + rows = self.db.usage_by_category() + self.assertEqual( + rows, + [ + {"category": "Landscapes", "runs": 5, "average_rating": 4.5}, + {"category": None, "runs": 5, "average_rating": None}, + {"category": "Portraits", "runs": 1, "average_rating": 3.0}, + ], + ) + + def test_counts_and_usage_write_nothing(self): + self.save("Alpha", category="A") + with self.db.model.get_connection() as connection: + before = connection.total_changes + self.db.library_counts() + self.db.usage_by_category() + with self.db.model.get_connection() as connection: + self.assertEqual(connection.total_changes, before) + self.assertFalse(connection.in_transaction) + + class DisplayTitleUnits(unittest.TestCase): """The first-line title is cut like the Library's JavaScript title().""" diff --git a/tests/test_system_api.py b/tests/test_system_api.py new file mode 100644 index 00000000..f72e8d4f --- /dev/null +++ b/tests/test_system_api.py @@ -0,0 +1,422 @@ +"""System routes (py/api/system.py): health check job, Overview and statistics. + +The health check runs off-loop as a ``read`` registry job of kind ``health`` +(one at a time, beside a scan), and the API keeps its last report. The +Overview reads only real values: Library-eligible counts, indexed images, the +last health result or null, no backup until one exists, and the effective +sources and whether they were auto-detected. The missing-thumbnail count is not +part of it: the Overview reads ``/images/thumbnails/status`` on its own (its +walk of every source must not hold the other values back). +""" + +import asyncio +import os +import sqlite3 +import sys +import threading +from pathlib import Path +from unittest.mock import MagicMock, patch + +_mock_server = MagicMock() +_mock_server.PromptServer.instance.routes = MagicMock() +sys.modules.setdefault("server", _mock_server) + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +from py.api.jobs import HEALTH_JOB_KIND # noqa: E402 +from py.config import GalleryConfig # noqa: E402 +from utils.hashing import generate_prompt_hash # noqa: E402 + +try: + from tests.test_api_admin import AdminAPITestCase # noqa: E402 +except ImportError: # discovered with tests/ as the top-level directory + from test_api_admin import AdminAPITestCase # noqa: E402 + +WAIT = 5 + + +class SystemAPITestCase(AdminAPITestCase): + async def get_application(self): + app = await super().get_application() + # Auto-detection searches the disk; here it finds the fake ComfyUI only. + self.api._find_comfyui_output_dir = lambda: ( + str(self.output_dir) if self.output_dir.is_dir() else None + ) + return app + + def save(self, text, **metadata): + return self.api.db.save_prompt( + text, prompt_hash=generate_prompt_hash(text), **metadata + ) + + def image(self, prompt_id, name, *, exists=True, folder=None): + path = Path(folder or self.output_dir) / name + if exists: + path.write_bytes(b"png") + return self.api.db.link_image_to_prompt(prompt_id, str(path)) + + async def json(self, method, url, status=200): + response = await self.client.request(method, url) + self.assertEqual(response.status, status, await response.text()) + return await response.json() + + async def restore_snapshot(self): + """Restore the live database from a copy of itself through the legacy route.""" + from contextlib import closing + + from aiohttp import FormData + + snapshot = os.path.join(self.tmpdir, "snapshot.db") + with closing(sqlite3.connect(self.api.db.model.db_path)) as src: + with closing(sqlite3.connect(snapshot)) as dst: + src.backup(dst) + with open(snapshot, "rb") as handle: + upload = handle.read() + form = FormData() + form.add_field("database_file", upload, filename="upload.db") + response = await self.client.request( + "POST", "/prompt_manager/restore", data=form + ) + self.assertEqual(response.status, 200, await response.text()) + + async def run_health(self): + body = await self.json("POST", "/prompt_manager/system/health", 202) + job = self.api._jobs.get(body["job"]["id"]) + await asyncio.wait_for(asyncio.shield(job.task), WAIT) + return body["job"], job + + +class TestHealthRoute(SystemAPITestCase): + async def test_no_report_before_a_check(self): + body = await self.json("GET", "/prompt_manager/system/health") + self.assertEqual(body, {"success": True, "report": None}) + + async def test_health_check_runs_as_a_read_job_and_keeps_its_report(self): + prompt = self.save("Images") + self.image(prompt, "kept.png") + self.image(prompt, "gone.png", exists=False) + summary, job = await self.run_health() + self.assertEqual(summary["kind"], HEALTH_JOB_KIND) + self.assertEqual(summary["title"], "Health check") + self.assertEqual(summary["admission"], "read") + self.assertFalse(summary["cancellable"]) + self.assertEqual(summary["counters"], {}) + self.assertEqual(job.state, "finished") + self.assertEqual(job.progress, 100) + self.assertEqual(job.status_line, "Health check incomplete") + report = (await self.json("GET", "/prompt_manager/system/health"))["report"] + self.assertEqual(report["status"], "incomplete") + missing = next(c for c in report["checks"] if c["id"] == "missing_files") + self.assertEqual((missing["status"], missing["count"]), ("issues", 1)) + ambiguous = next(c for c in report["checks"] if c["id"] == "ambiguous_metadata") + self.assertEqual(ambiguous["status"], "unavailable") + self.assertNotIn(self.tmpdir, str(report)) + + async def test_health_check_is_single_flight_and_runs_beside_a_scan(self): + release = threading.Event() + entered = threading.Event() + real = self.api.db.run_health_checks + + def gated(*args, **kwargs): + entered.set() + release.wait(WAIT) + return real(*args, **kwargs) + + scan_gate = asyncio.Event() + + async def scanning(job): + await scan_gate.wait() + + scan = self.api._jobs.start( + "scan", "Metadata scan", admission="records", runner=scanning + ) + with patch.object(self.api.db, "run_health_checks", side_effect=gated): + first = await self.json("POST", "/prompt_manager/system/health", 202) + loop = asyncio.get_running_loop() + await loop.run_in_executor(None, entered.wait, WAIT) + busy = await self.json("POST", "/prompt_manager/system/health", 409) + self.assertEqual(busy["code"], "busy") + self.assertEqual( + busy["running"], {"id": first["job"]["id"], "title": "Health check"} + ) + release.set() + await asyncio.wait_for(self.api._jobs.get(first["job"]["id"]).task, WAIT) + scan_gate.set() + await asyncio.wait_for(scan.task, WAIT) + + async def test_failed_check_keeps_the_previous_report(self): + await self.run_health() + before = (await self.json("GET", "/prompt_manager/system/health"))["report"] + with ( + patch.object( + self.api.db, "run_health_checks", side_effect=RuntimeError("boom") + ), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("POST", "/prompt_manager/system/health", 202) + job = self.api._jobs.get(body["job"]["id"]) + await asyncio.wait_for(asyncio.shield(job.task), WAIT) + self.assertEqual(job.state, "failed") + after = (await self.json("GET", "/prompt_manager/system/health"))["report"] + self.assertEqual(after, before) + + async def test_absent_configured_source_is_reported_per_source(self): + prompt = self.save("Images") + self.image(prompt, "gone.png", exists=False) + absent = self.comfy_dir / "output" / "unmounted" + GalleryConfig.MONITORING_DIRECTORIES = [ + self._root(self.output_dir), + str(absent), + ] + await self.run_health() + report = (await self.json("GET", "/prompt_manager/system/health"))["report"] + missing = next(c for c in report["checks"] if c["id"] == "missing_files") + # Per source: the readable output is counted, the absent one is not. + self.assertEqual(missing["status"], "issues") + self.assertEqual(missing["count"], 1) + self.assertTrue(missing["partial"]) + self.assertEqual( + [(s["status"], s["count"]) for s in missing["sources"]], + [("checked", 1), ("unavailable", None)], + ) + self.assertEqual(report["status"], "incomplete") + + async def test_restore_clears_health_report(self): + self.save("Any") + await self.run_health() + self.assertIsNotNone( + (await self.json("GET", "/prompt_manager/system/health"))["report"] + ) + await self.restore_snapshot() + body = await self.json("GET", "/prompt_manager/system/health") + self.assertEqual(body, {"success": True, "report": None}) + overview = await self.json("GET", "/prompt_manager/system/overview") + self.assertIsNone(overview["health"]) + + async def test_check_finishing_after_a_restore_is_not_kept(self): + # A check that started on the old database describes it, not the new one. + self.save("Any") + release = threading.Event() + entered = threading.Event() + real = self.api.db.run_health_checks + + def gated(*args, **kwargs): + entered.set() + release.wait(WAIT) + return real(*args, **kwargs) + + with patch.object(self.api.db, "run_health_checks", side_effect=gated): + body = await self.json("POST", "/prompt_manager/system/health", 202) + loop = asyncio.get_running_loop() + await loop.run_in_executor(None, entered.wait, WAIT) + await self.restore_snapshot() + release.set() + job = self.api._jobs.get(body["job"]["id"]) + await asyncio.wait_for(asyncio.shield(job.task), WAIT) + self.assertEqual(job.state, "finished") + body = await self.json("GET", "/prompt_manager/system/health") + self.assertEqual(body, {"success": True, "report": None}) + # A check started on the restored database is kept again. + await self.run_health() + self.assertIsNotNone( + (await self.json("GET", "/prompt_manager/system/health"))["report"] + ) + + async def test_health_sources_resolve_off_the_event_loop(self): + loop_thread = threading.get_ident() + threads = [] + real = self.api._health_roots + + def recording(): + threads.append(threading.get_ident()) + return real() + + with patch.object(self.api, "_health_roots", side_effect=recording): + await self.run_health() + self.assertEqual(len(threads), 1) + self.assertNotEqual(threads[0], loop_thread) + + async def test_health_start_failure_is_an_error_envelope(self): + with ( + patch.object(self.api, "_health_roots", side_effect=RuntimeError("secret")), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("POST", "/prompt_manager/system/health", 500) + self.assertEqual( + body, + { + "success": False, + "code": None, + "error": "Failed to start the health check", + }, + ) + self.assertEqual(self.api._jobs.list(), []) + + +class TestOverviewRoute(SystemAPITestCase): + async def test_overview_reports_real_counts(self): + self.api.db.save_prompt_pair("Cat", "Blur") + archived = self.api.db.save_prompt_pair("Cat", "Noise") + self.api.db.archive_prompt_pair(archived) + legacy = self.save("Legacy") + self.image(legacy, "a.png") + self.image(legacy, "b.png") + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertTrue(body["success"]) + self.assertEqual( + body["library"], {"positive_prompts": 2, "negative_variants": 1} + ) + self.assertEqual(body["images"], {"indexed": 2, "rated": 0}) + self.assertEqual( + set(body), + { + "success", + "library", + "images", + "health", + "backup", + "needs_review", + "sources", + }, + ) + + async def test_overview_rated_counts_images_of_rated_prompts(self): + rated = self.save("Rated", rating=5) + self.image(rated, "a.png") + self.image(rated, "b.png") + self.image(self.save("Unrated"), "c.png") + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual(body["images"], {"indexed": 3, "rated": 2}) + + async def test_overview_never_fabricates_backup_or_health(self): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertIsNone(body["health"]) + self.assertIsNone(body["backup"]) + self.assertEqual(body["needs_review"], {"pending": None}) + self.save("Any") + await self.run_health() + report = (await self.json("GET", "/prompt_manager/system/health"))["report"] + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["health"], + {"checked_at": report["checked_at"], "status": "incomplete", "issues": 0}, + ) + self.assertIsNone(body["backup"]) + + async def test_overview_health_counts_checks_with_issues(self): + prompt = self.save("Images") + self.image(prompt, "gone.png", exists=False) + self.image(prompt, "gone-too.png", exists=False) + await self.run_health() + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual(body["health"]["issues"], 1) + + async def test_overview_sources_distinguish_auto_detect(self): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["sources"], {"configured": 0, "effective": 1, "auto_detected": True} + ) + second = self.output_dir / "archive" + second.mkdir() + GalleryConfig.MONITORING_DIRECTORIES = [ + self._root(self.output_dir), + self._root(second), + ] + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["sources"], {"configured": 2, "effective": 2, "auto_detected": False} + ) + # Every configured source unavailable: ComfyUI's output is used instead. + GalleryConfig.MONITORING_DIRECTORIES = [str(self.comfy_dir / "absent")] + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["sources"], {"configured": 1, "effective": 1, "auto_detected": True} + ) + + async def test_overview_never_walks_the_sources_for_thumbnails(self): + (self.output_dir / "one.png").write_bytes(b"png") + with patch.object( + self.api, + "_plan_gallery_thumbnails", + side_effect=AssertionError("walked"), + ): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertTrue(body["success"]) + self.assertNotIn("thumbnails", body) + + async def test_overview_sources_resolve_off_the_event_loop(self): + loop_thread = threading.get_ident() + threads = [] + real = self.api._output_dirs_with_origin + + def recording(): + threads.append(threading.get_ident()) + return real() + + with patch.object(self.api, "_output_dirs_with_origin", side_effect=recording): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual(body["sources"]["effective"], 1) + self.assertEqual(len(threads), 1) + self.assertNotEqual(threads[0], loop_thread) + + async def test_overview_failure_is_an_error_envelope(self): + with ( + patch.object( + self.api.db, "library_counts", side_effect=RuntimeError("secret") + ), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("GET", "/prompt_manager/system/overview", 500) + self.assertEqual( + body, + {"success": False, "code": None, "error": "Failed to read the overview"}, + ) + + +class TestStatisticsRoute(SystemAPITestCase): + async def test_statistics_sum_real_run_counts(self): + a = self.save("Alpha", category="Landscapes", rating=5) + b = self.save("Beta", category="Portraits", rating=2) + self.api.db.record_prompt_use(a, times=3) + self.api.db.record_prompt_use(b) + body = await self.json("GET", "/prompt_manager/system/statistics") + self.assertEqual( + body["by_category"], + [ + {"category": "Landscapes", "runs": 3, "average_rating": 5.0}, + {"category": "Portraits", "runs": 1, "average_rating": 2.0}, + ], + ) + self.assertEqual(body["totals"], self.api.db.get_statistics()) + self.assertTrue(body["success"]) + + async def test_statistics_lists_rated_images_for_library_totals(self): + # D96: the Statistics dialog's "Library totals" reads the rated count + # with the runs (one read), from the same images the Overview counts. + rated = self.save("Rated", rating=4) + self.image(rated, "a.png") + self.image(self.save("Unrated"), "b.png") + body = await self.json("GET", "/prompt_manager/system/statistics") + self.assertEqual(body["rated_images"], 1) + self.assertEqual( + set(body), {"success", "by_category", "totals", "rated_images"} + ) + + async def test_statistics_failure_is_an_error_envelope(self): + with ( + patch.object( + self.api.db, "usage_by_category", side_effect=RuntimeError("secret") + ), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("GET", "/prompt_manager/system/statistics", 500) + self.assertEqual( + body, + {"success": False, "code": None, "error": "Failed to read statistics"}, + ) + + +if __name__ == "__main__": + import unittest + + unittest.main() diff --git a/tools/vendor/frontend-coverage-baseline.json b/tools/vendor/frontend-coverage-baseline.json index 7d829414..88dfa16e 100644 --- a/tools/vendor/frontend-coverage-baseline.json +++ b/tools/vendor/frontend-coverage-baseline.json @@ -443,6 +443,30 @@ "functions": 98, "statements": 98 }, + "workspace/system-controls.mjs": { + "lines": 98, + "branches": 87, + "functions": 89, + "statements": 98 + }, + "workspace/system-overview.mjs": { + "lines": 96, + "branches": 92, + "functions": 98, + "statements": 96 + }, + "workspace/system-state.mjs": { + "lines": 98, + "branches": 92, + "functions": 92, + "statements": 98 + }, + "workspace/system.mjs": { + "lines": 93, + "branches": 85, + "functions": 69, + "statements": 93 + }, "workspace/theme.mjs": { "lines": 98, "branches": 93, diff --git a/web/js/workspace/activity-store.mjs b/web/js/workspace/activity-store.mjs index 72eb6aba..0729d449 100644 --- a/web/js/workspace/activity-store.mjs +++ b/web/js/workspace/activity-store.mjs @@ -336,6 +336,16 @@ export function createActivityStore({ api, scope, connect = defaultConnect, noti return { busy: store.running()[0] ?? null, message: error.message }; } }, + /** + * Follow a job another route started (its summary from that route's answer, e.g. the health check's + * `POST /system/health`): it is listed and its stream opened at once, with no read and no start toast. + */ + track(raw) { + const job = parseJob(raw); + if (!job) throw new TypeError('Invalid job'); + upsert(job); + return job; + }, /** Ask a running job to stop at its next file boundary; false once it has ended. */ async cancel(id) { const result = await api.post(`${JOBS_URL}/${encodeURIComponent(id)}/cancel`, {}, { signal: scope.signal }); diff --git a/web/js/workspace/index.mjs b/web/js/workspace/index.mjs index 70adf1a4..138a6ef7 100644 --- a/web/js/workspace/index.mjs +++ b/web/js/workspace/index.mjs @@ -1,6 +1,6 @@ import '../api-client.js'; import { createButton, createScope } from '../shared/toolkit/core.js'; -import { routes, mountRouter, summarizeStats, routeTitle, isHandoff, loadNavCounts, +import { routes, mountRouter, summarizeStats, routeTitle, loadNavCounts, LIBRARY_WRITE_COUNTS, NAV_COUNT_ROUTES, FAVORITE_WRITE_COUNTS, FAVORITE_COUNT_DELAY_MS } from './state.mjs'; import { mountWorkspaceTheme } from './theme.mjs'; import { mountImages, onFavoriteChange } from './images.mjs'; @@ -16,6 +16,7 @@ import { mountActivityPage, mountActivityTopbar } from './activity.mjs'; import { mountActivityJobView } from './activity-job-view.mjs'; import { mountInspect } from './inspect.mjs'; import { createPromptSender } from './prompt-sender.mjs'; +import { mountSystem } from './system.mjs'; const { createApiClient, resolveUrl } = window.ApiClient; @@ -28,7 +29,8 @@ function element(tag, text, className) { function legacyLink(route) { const link = element('a', route.action, 'workspace-link'); - link.href = resolveUrl(route.href); + // A native card (System, Q12) links its workspace hash; a legacy tool's page goes through the mount prefix. + link.href = route.native ? route.href : resolveUrl(route.href); return link; } @@ -137,6 +139,9 @@ function mountWorkspace() { let leaveGuard = null, renderToken = 0; scope.cleanup(() => viewScope?.dispose()); + // The mounted System page: another of its sections is shown in place, keeping the shell and the drafts. + let system = null; + function render(route) { const token = ++renderToken; const hold = leaveGuard?.(); @@ -151,6 +156,11 @@ function mountWorkspace() { leaveGuard = null; // A shared page dialog never stays open over the next route. closeWorkspaceDialog(); + if (route.id === 'system' && system && currentKey === 'system') { + system.show(route.section); + return; + } + system = null; viewScope?.dispose(); images = null; viewScope = createScope(); @@ -251,21 +261,18 @@ function mountWorkspace() { else mountActivityPage(main, { store: activity, scope: view }); return; } + if (route.id === 'system') { + // The concept's System (app.js renderSystem): its heading description, section nav and Save row. + description.textContent = route.description; + system = mountSystem(main, api, view, { section: route.section, activity }); + return; + } if (route.id === 'inspect') { // The concept's Inspect a file (app.js renderMetadata): a local file, read only in this browser. description.textContent = route.description; ({ leaveGuard } = mountInspect(main, { parentScope: view, api, sender: createPromptSender({ scope: view }) })); return; } - if (isHandoff(route)) { - description.textContent = route.description; - const panel = element('section', undefined, 'surface handoff'); - panel.append(element('h2', 'Your current tools are ready'), - element('p', 'This workspace is being introduced in stages. Continue in the existing tool with your saved data.'), - legacyLink(route)); - main.append(panel); - return; - } description.textContent = 'A home for your prompts, images and creative history.'; const stats = element('section', undefined, 'overview-stats'); stats.setAttribute('aria-label', 'Library statistics'); diff --git a/web/js/workspace/state.mjs b/web/js/workspace/state.mjs index 7957f4dd..3255381a 100644 --- a/web/js/workspace/state.mjs +++ b/web/js/workspace/state.mjs @@ -14,11 +14,29 @@ export const routes = [ { id: 'favorites', label: 'Favorites' }, // The concept's Activity (app.js renderActivity 732–738): this server session's jobs. No href: no Overview card. { id: 'activity', label: 'Activity' }, - { id: 'system', label: 'System', href: '/prompt_manager/admin', - description: 'Open the current dashboard, then choose Settings or Maintenance for configuration, scans, backups and diagnostics.', action: 'Open existing administration' }, + // Native System (#249 System S2): `#/system/
`. Q12: Overview's card opens its Overview section (`native`: + // a workspace hash, not a legacy page); each undelivered section hands off to its legacy tool (D94). The + // description is the concept's System heading (app.js renderSystem 700). + { id: 'system', label: 'System', href: '#/system/overview', native: true, + description: 'Configuration, maintenance and recovery—with browsing kept in the workspace.', action: 'Open System' }, ]; -const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'inspect', 'overview', 'saved', 'favorites', 'activity']); +const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'inspect', 'overview', 'saved', 'favorites', 'activity', 'system']); + +/** + * System's sections in the concept's order (app.js `systemSections` 635–644); the concept's `general` is production's + * `overview`, so `#/system/overview` sits beside the production hub `#/overview` (D28). + */ +export const SYSTEM_SECTIONS = Object.freeze([ + ['overview', 'Overview'], ['appearance', 'Appearance'], ['sources', 'Sources & scans'], ['integrations', 'Integrations'], + ['tagging', 'Auto-tagging'], ['maintenance', 'Maintenance'], ['backups', 'Backup & export'], ['diagnostics', 'Diagnostics & logs'], +].map(section => Object.freeze(section))); +const SYSTEM_SECTION_IDS = new Set(SYSTEM_SECTIONS.map(([id]) => id)); + +/** Whether `id` names a System section. */ +export function isSystemSection(id) { + return typeof id === 'string' && SYSTEM_SECTION_IDS.has(id); +} /** A saved view (#249 D16): the shell resolves its id to the view's page, name and state. */ const SAVED_ROUTE = Object.freeze({ id: 'saved', label: 'Saved view' }); @@ -39,6 +57,14 @@ export function resolveRoute(hash) { // One job of the Activity page (its id is the registry's uuid4 hex). const job = /^#\/activity\/([0-9a-f]{32})$/.exec(hash); if (job) return {...routes.find(route => route.id === 'activity'), jobId:job[1], hash}; + // `#/system` is the Overview section under the sidebar's own hash; an unknown child is Overview, its hash replaced. + const system = /^#\/system(?:\/(.*))?$/.exec(hash); + if (system) { + const route = routes.find(item => item.id === 'system'); + if (system[1] === undefined) return {...route, section:'overview', hash}; + const section = isSystemSection(system[1]) ? system[1] : 'overview'; + return {...route, section, hash:`#/system/${section}`}; + } const match = /^#\/library\/([1-9]\d*)$/.exec(hash); if (match && Number.isSafeInteger(Number(match[1]))) { return {...routes.find(route => route.id === 'library'), positiveId:Number(match[1]), hash}; @@ -117,7 +143,7 @@ export function mountRouter(win, onRoute) { const hash = route.hash || `#/${route.id}`; if (win.location.hash !== hash) win.history.replaceState(win.history.state, '', hash); if (current?.id === route.id && current?.positiveId === route.positiveId && current?.savedId === route.savedId && - current?.jobId === route.jobId) return; + current?.jobId === route.jobId && current?.section === route.section) return; current = route; onRoute(route); }; diff --git a/web/js/workspace/system-controls.mjs b/web/js/workspace/system-controls.mjs new file mode 100644 index 00000000..79bf6183 --- /dev/null +++ b/web/js/workspace/system-controls.mjs @@ -0,0 +1,132 @@ +/** + * System's row builders on the concept's markup (#249 System S2 Task 5), built with the DOM, never HTML strings: + * - `settingRow` is the concept's `setting(id, label, description, type, options)` (app.js 624–634): `.setting-row` > + * `div` > `label[for]` + `p`, then the control. A toggle is the Toolkit switch, as the concept's `hydrateSwitches` + * (708–717) makes it; a range is the concept's `min=1 max=16` unless the row gives its own bounds. + * - `operationRow` is `operation(title, desc, action, label, extra, danger)` (645–654): `.operation` > `div` > `h3` + `p`, + * then `.row-actions` with one button, `danger` when flagged. + * - `sectionNav` is `sectionNav(items, active, action)` (306–308): `.section-nav[role=group]` buttons with `aria-pressed`. + * Every listener is registered on the caller's `scope`, so disposing it leaves none behind. + */ +import { createButton, createSwitch } from '../shared/toolkit/core.js'; + +const INPUT_TYPES = new Set(['number', 'password', 'text', 'range']); +const RANGE_DEFAULTS = Object.freeze({ min: 1, max: 16 }); + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +function requireText(values) { + for (const [name, value] of Object.entries(values)) { + if (typeof value !== 'string' || !value) throw new TypeError(`The row needs a ${name}`); + } +} + +function requireScope(scope, callback, name) { + if (callback === undefined) return; + if (typeof callback !== 'function') throw new TypeError(`The row ${name} must be a function`); + if (typeof scope?.listen !== 'function') throw new TypeError('The row needs the section scope for its listener'); +} + +/** `[value, label]` pairs from strings or pairs. */ +function optionPairs(options) { + if (!Array.isArray(options) || options.length === 0) throw new TypeError('A select row needs its options'); + return options.map(option => (Array.isArray(option) ? option.map(String) : [String(option), String(option)])); +} + +/** The control a spec describes, and how to read its value. */ +function buildControl(id, spec) { + if (spec?.type === 'toggle') { + const field = createSwitch('', '', id); + const input = field.children[0]; + // The concept's toggles are on unless configured off (`val !== false`). + input.checked = spec.checked !== false; + return { node: field, input, read: () => input.checked }; + } + if (spec?.type === 'select') { + const select = el('select'); + for (const [value, label] of optionPairs(spec.options)) { + const option = el('option', label); + option.value = value; + select.append(option); + } + if (spec.value !== undefined) select.value = String(spec.value); + return { node: select, input: select, read: () => select.value }; + } + if (INPUT_TYPES.has(spec?.type)) { + const input = el('input'); + input.type = spec.type; + const bounds = spec.type === 'range' ? { ...RANGE_DEFAULTS, ...spec } : spec; + for (const key of ['min', 'max', 'step']) if (bounds[key] !== undefined) input[key] = String(bounds[key]); + input.value = spec.value === undefined || spec.value === null ? '' : String(spec.value); + const numeric = spec.type === 'number' || spec.type === 'range'; + return { node: input, input, read: () => (numeric ? Number(input.value) : input.value) }; + } + throw new TypeError(`Unknown setting control: ${spec?.type}`); +} + +/** + * One `.setting-row`. `control` is a spec (`{type:'toggle', checked}`, `{type:'select', options, value}` with string or + * `[value, label]` options, `{type:'number'|'password'|'text'|'range', value, min, max, step}`) or a ready-made node, + * which takes `id`. `onChange(value)` (on `scope`) reports the checked state, a number for number and range, else the + * text. + */ +export function settingRow({ id, label, description, control, scope, onChange }) { + requireText({ id, label, description }); + requireScope(scope, onChange, 'onChange'); + let built; + if (control && typeof control === 'object' && typeof control.type !== 'string' && 'tagName' in control) { + control.id = id; + built = { node: control, input: control, read: () => control.value }; + } else { + built = buildControl(id, control); + built.input.id = id; + } + built.input.dataset.setting = id; + const row = el('div', undefined, 'setting-row'); + const text = el('div'); + const caption = el('label', label); + caption.htmlFor = id; + text.append(caption, el('p', description)); + row.append(text, built.node); + if (onChange) scope.listen(built.input, 'change', () => onChange(built.read())); + return row; +} + +/** One `.operation` row; `onClick(button)` runs on `scope`. */ +export function operationRow({ title, description, label, danger = false, onClick, scope }) { + requireText({ title, description, label }); + if (typeof onClick !== 'function') throw new TypeError('The operation needs onClick'); + requireScope(scope, onClick, 'onClick'); + const row = el('div', undefined, 'operation'); + const text = el('div'); + text.append(el('h3', title), el('p', description)); + const actions = el('div', undefined, 'row-actions'); + const action = createButton(label, danger ? 'danger' : ''); + scope.listen(action, 'click', () => onClick(action)); + actions.append(action); + row.append(text, actions); + return row; +} + +/** The `.section-nav` group: one button per `[id, title]`, the `active` one pressed; `onSelect(id)` runs on `scope`. */ +export function sectionNav({ label, sections, active, onSelect, scope }) { + requireText({ label }); + if (typeof onSelect !== 'function') throw new TypeError('The section nav needs onSelect'); + requireScope(scope, onSelect, 'onSelect'); + const nav = el('div', undefined, 'section-nav'); + nav.setAttribute('role', 'group'); + nav.setAttribute('aria-label', label); + for (const [id, title] of sections) { + const item = createButton(title); + item.dataset.value = id; + item.setAttribute('aria-pressed', String(id === active)); + scope.listen(item, 'click', () => onSelect(id)); + nav.append(item); + } + return nav; +} diff --git a/web/js/workspace/system-overview.mjs b/web/js/workspace/system-overview.mjs new file mode 100644 index 00000000..ac902064 --- /dev/null +++ b/web/js/workspace/system-overview.mjs @@ -0,0 +1,432 @@ +/** + * System › Overview (#249 System S2 Task 6): the concept's `renderSystem` "general" body (app.js 658–675) on real + * values, its Statistics dialog ("Usage statistics", a1600–1604) and its health report (`health-check`, a1439–1456). + * + * - The four `.summary-metrics` and the two `.two-col` surfaces ("Library health", "Workspace preferences") come from + * one `GET /prompt_manager/system/overview`. D93: metric 4 is the last health result (Not checked / Passed / + * N issues / Incomplete); the review, backup, thumbnail and sources rows say what the server reports, and nothing + * reads 0 before the read answers: a load shows a status line, a failure an alert and Retry. The thumbnail row is + * its own read of `/images/thumbnails/status` (a walk of every source): "Checking thumbnails…" until it answers. + * - D96: the Statistics dialog lists the real runs and average rating per category, then "Library totals". + * - D95: the "Health report" lists every check the server ran, Missing files per image source, with a chip per real + * result and when it was checked. The check runs from Maintenance's notice ("Run health checks", D123) as a + * read job of the registry, tracked in the Activity store (its event stream); the report opens when it ends. + * + * Reads run through `latestOnly()` on the section's scope, so a late answer never repaints a newer section. + */ +import { createButton } from '../shared/toolkit/core.js'; +import { navigateTo } from './state.mjs'; +import { closeWorkspaceDialog, openContentDialog } from './workspace-dialogs.mjs'; +import { showWorkspaceToast } from './toast.mjs'; +import { noun } from './plural.mjs'; +import { operationRow } from './system-controls.mjs'; + +export const OVERVIEW_URL = '/prompt_manager/system/overview'; +export const STATISTICS_URL = '/prompt_manager/system/statistics'; +export const HEALTH_URL = '/prompt_manager/system/health'; +export const THUMBNAILS_URL = '/prompt_manager/images/thumbnails/status'; +export const HEALTH_KIND = 'health'; + +export const LOADING_TEXT = 'Loading the overview…'; +export const THUMBNAILS_LOADING = 'Checking thumbnails…'; +export const LOAD_FAILED_TEXT = 'The overview could not be loaded. Your saved data has not changed.'; +export const STATISTICS_FAILED_TEXT = 'Statistics could not be loaded. Your saved data has not changed.'; +export const HEALTH_FAILED_TEXT = 'The health check could not finish. Nothing was changed.'; +export const NOT_AVAILABLE = 'Not available yet.'; + +const number = value => Number(value).toLocaleString(); + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +/** + * D93: metric 4, the last health result of this server session: the checks that found issues, even when another + * check could not run; otherwise Passed, or Incomplete while a check was unavailable or failed. + */ +export function integrityText(health) { + if (!health) return 'Not checked'; + if (health.issues > 0) return `${number(health.issues)} ${noun(health.issues, 'issue')}`; + return health.status === 'passed' ? 'Passed' : 'Incomplete'; +} + +/** The concept's four metrics `[value, label]`, in its order. */ +export function metricValues(overview) { + return [ + [number(overview.library.positive_prompts), 'Positive prompts'], + [number(overview.library.negative_variants), 'Negative variants'], + [number(overview.images.indexed), 'Images'], + [integrityText(overview.health), 'Database integrity'], + ]; +} + +/** D93 (Q6): the review row's title; a count only once the Needs review queue reports one. */ +export function ambiguousTitle(needsReview) { + const pending = needsReview?.pending; + if (!Number.isSafeInteger(pending)) return 'Ambiguous image links'; + return `${number(pending)} ambiguous image ${noun(pending, 'link')}`; +} + +/** D93: the backup row; only a recorded, verified backup is ever named. */ +export function backupText(backup) { + if (!backup) return 'No retained backup record.'; + const when = new Date(backup.verified_at ?? backup.created_at); + return Number.isNaN(when.getTime()) ? 'No retained backup record.' : `Last verified backup ${when.toLocaleString()}.`; +} + +/** + * D93: the thumbnail row's title from the `/images/thumbnails/status` answer: undefined while it is read, null when + * the read failed; with no image source (`sources.effective` 0) there is nothing to count. + */ +export function thumbnailTitle(status, sources) { + if (status === undefined) return THUMBNAILS_LOADING; + const missing = status?.pending; + if (!Number.isSafeInteger(missing) || sources?.effective === 0) return 'Thumbnail status unavailable'; + if (missing === 0) return 'No missing thumbnails'; + return `${number(missing)} missing ${noun(missing, 'thumbnail')}`; +} + +const SOURCES_SHARED = 'Monitoring and enrichment share one source list.'; + +/** D93: the sources row, `[title, description]`: the sources in use and how they were chosen. */ +export function sourcesCopy(sources) { + const { configured, effective, auto_detected: auto } = sources; + const title = `${number(effective)} output ${noun(effective, 'source')}`; + if (auto) return [title, `None configured is available; ComfyUI's output folder is used. ${SOURCES_SHARED}`]; + if (configured !== effective) return [title, `${number(effective)} of ${number(configured)} configured ${noun(configured, 'source')} available. ${SOURCES_SHARED}`]; + return [title, SOURCES_SHARED]; +} + +// D95: an issue chip names what was found; the concept's own wording for missing files and ambiguous metadata. +const ISSUE_NOUNS = Object.freeze({ + integrity: ['problem found', 'problems found'], + foreign_keys: ['broken reference', 'broken references'], + canonical_hashes: ['hash does not match its text', 'hashes do not match their text'], + pair_uniqueness: ['duplicate pair row', 'duplicate pair rows'], + missing_files: ['link needs review', 'links need review'], + ambiguous_metadata: ['image needs review', 'images need review'], + consistency: ['inconsistency', 'inconsistencies'], +}); + +/** D95: a check's chip from its real status. */ +export function checkChip(check) { + if (check.status === 'passed') return 'Passed'; + if (check.status === 'failed') return 'Could not check'; + if (check.status === 'unavailable') return check.id === 'ambiguous_metadata' ? NOT_AVAILABLE.slice(0, -1) : 'Not available'; + const [one, many] = ISSUE_NOUNS[check.id] ?? ['issue', 'issues']; + if (!Number.isSafeInteger(check.count)) return 'Issues found'; + // SQLite stops listing integrity problems at its limit, so a capped count is a lower bound ("100+"). + if (check.capped === true) return `${number(check.count)}+ ${many}`; + return `${number(check.count)} ${check.count === 1 ? one : many}`; +} + +function sourceChip(source) { + if (source.status !== 'checked') return 'Not available'; + return source.count === 0 ? 'Passed' : checkChip({ id: 'missing_files', status: 'issues', count: source.count }); +} + +/** D95: the report's rows `[label, chip]` in the server's order; Missing files once per image source when several. */ +export function healthRows(report) { + return report.checks.flatMap(check => { + const sources = check.id === 'missing_files' && Array.isArray(check.sources) ? check.sources : []; + if (sources.length < 2) return [[check.label, checkChip(check)]]; + return sources.map(source => [`${check.label} · ${source.name}`, sourceChip(source)]); + }); +} + +/** D95: when the report was taken, and whether it is complete. */ +export function healthSummary(report) { + const when = new Date(report.checked_at); + const time = Number.isNaN(when.getTime()) ? '' : `Checked ${when.toLocaleString()}. `; + if (report.status === 'passed') return `${time}Every check passed.`; + if (report.status === 'incomplete') return `${time}Some checks could not run, so this report is incomplete.`; + return `${time}Nothing was changed; review the checks with issues.`; +} + +/** D96: the concept's "Runs by category" rows `[category, runs, average rating]`. */ +export function categoryRows(statistics) { + return statistics.by_category.map(row => [ + row.category ?? 'Uncategorized', + number(row.runs), + row.average_rating === null || row.average_rating === undefined ? 'Not rated' + : `${Number(row.average_rating).toLocaleString(undefined, { maximumFractionDigits: 2 })} / 5`, + ]); +} + +/** D96: the added "Library totals" rows `[label, count]` (the legacy statistics and the rated images). */ +export function totalsRows(statistics) { + const { totals } = statistics; + return [['Prompts', totals.total_prompts], ['Tags', totals.total_tags], ['Images', totals.total_images], + ['Rated images', statistics.rated_images]].map(([label, value]) => [label, number(value)]); +} + +function table(caption, head, rows, empty) { + const node = el('table'); + const header = el('tr'); + for (const text of head) { + const cell = el('th', text); + cell.scope = 'col'; + header.append(cell); + } + const thead = el('thead'); + thead.append(header); + const tbody = el('tbody'); + for (const values of rows) { + const row = el('tr'); + row.append(...values.map(text => el('td', text))); + tbody.append(row); + } + if (rows.length === 0) { + const row = el('tr'); + const cell = el('td', empty); + cell.colSpan = head.length; + row.append(cell); + tbody.append(row); + } + node.append(el('caption', caption), thead, tbody); + return node; +} + +/** The Statistics dialog: "Runs by category" (the concept's table), then "Library totals" (D96). */ +export function statisticsNodes(statistics) { + return [ + table('Runs by category', ['Category', 'Runs', 'Avg. rating'], categoryRows(statistics), 'No prompts in the library yet.'), + table('Library totals', ['Total', 'Count'], totalsRows(statistics), ''), + ]; +} + +/** The concept's health report body: `.operation-list` of `h3` + chip, the D95 summary, then Close / Review missing links. */ +export function healthNodes(report, { scope, onReviewMissing }) { + const list = el('div', undefined, 'operation-list'); + for (const [label, chip] of healthRows(report)) { + const row = el('div', undefined, 'operation'); + row.append(el('h3', label), el('span', chip, 'chip')); + list.append(row); + } + const close = createButton('Close'); + const review = createButton('Review missing links', 'ca-primary'); + scope.listen(close, 'click', () => closeWorkspaceDialog()); + scope.listen(review, 'click', () => onReviewMissing()); + const actions = el('div', undefined, 'dialog-actions'); + actions.append(close, review); + return [list, el('p', healthSummary(report), 'small'), actions]; +} + +/** + * Wait until the job `id` has ended, following it in the Activity store (`activity`), which streams the job's events + * and marks it lost when the server no longer lists it; nothing here polls. Resolves its final state ('finished', + * 'failed', 'cancelled' or 'lost'). + */ +export function waitForJob(activity, id, { signal } = {}) { + if (signal?.aborted) return Promise.reject(signal.reason); + return new Promise((resolve, reject) => { + const check = () => { + const state = activity.find(id)?.state; + if (state && state !== 'running') finish(resolve, state); + }; + const onAbort = () => finish(reject, signal.reason); + const unsubscribe = activity.subscribe(check); + function finish(settle, value) { + unsubscribe(); + signal?.removeEventListener('abort', onAbort); + settle(value); + } + signal?.addEventListener('abort', onAbort, { once: true }); + check(); + }); +} + +/** + * Run the health check and resolve its report. The started job is tracked in the Activity store; a check already + * running (409 busy, another tab) is found in the store's job list and waited for instead of refused. Rejects when the + * check fails or ends without a report. + */ +export async function runHealthCheck(api, { activity, signal } = {}) { + let id; + try { + const { job } = await api.post(HEALTH_URL, {}, { signal }); + id = activity.track(job).id; + } catch (error) { + if (error?.status !== 409 || error?.code !== 'busy') throw error; + await activity.load(); + id = activity.running().find(job => job.kind === HEALTH_KIND)?.id; + // The other check ended between the refusal and this read: its report is the latest. + } + const state = id ? await waitForJob(activity, id, { signal }) : 'finished'; + if (state !== 'finished') throw new Error(HEALTH_FAILED_TEXT); + const { report } = await api.get(HEALTH_URL, { signal, cache: 'no-store' }); + if (!report) throw new Error(HEALTH_FAILED_TEXT); + return report; +} + +/** Open the Health report for `report` (the Maintenance notice's and, from S7, its Check action's dialog). */ +export function openHealthReport(report, { scope, trigger = null, navigate = navigateTo }) { + openContentDialog({ scope, title: 'Health report', trigger, nodes: healthNodes(report, { scope, + // Missing-link cleanup lives in Maintenance (its D94 hand-off until S7). + onReviewMissing: () => { closeWorkspaceDialog(); navigate('#/system/maintenance'); } }) }); +} + +/** + * The concept's Maintenance notice ("Check first. Review changes. Then apply." + "Run health checks", a689), which + * S2 delivers ahead of the rest of Maintenance (D123): the press runs the read-only check, then opens the report. + */ +export function healthNotice({ api, activity, scope, navigate = navigateTo, notify = showWorkspaceToast }) { + const notice = el('div', undefined, 'notice'); + const text = el('div'); + text.append(el('h2', 'Check first. Review changes. Then apply.'), + el('p', 'One home for database and media maintenance. Scanning is separate from changing or removing records.')); + const run = createButton('Run health checks', 'ca-primary'); + notice.append(text, run); + scope.listen(run, 'click', async () => { + if (run.getAttribute('aria-disabled') === 'true') return; + run.setAttribute('aria-disabled', 'true'); + run.textContent = 'Checking…'; + try { + const report = await runHealthCheck(api, { activity, signal: scope.signal }); + if (scope.signal.aborted) return; + openHealthReport(report, { scope, trigger: run, navigate }); + } catch (error) { + if (!scope.signal.aborted) notify(HEALTH_FAILED_TEXT); + } finally { + if (!scope.signal.aborted) { + run.removeAttribute('aria-disabled'); + run.textContent = 'Run health checks'; + } + } + }); + return notice; +} + +function metricsNode(overview) { + const metrics = el('div', undefined, 'summary-metrics'); + for (const [value, label] of metricValues(overview)) { + const metric = el('div', undefined, 'metric'); + metric.append(el('strong', value), el('span', label)); + metrics.append(metric); + } + return metrics; +} + +function surface(heading, rows) { + const section = el('section', undefined, 'surface'); + section.append(el('h2', heading), ...rows); + return section; +} + +/** The Overview's nodes, and its thumbnail row (painted by its own read). */ +function overviewNodes(overview, { scope, navigate, openStatistics }) { + const go = section => () => navigate(`#/system/${section}`); + const review = operationRow({ scope, title: ambiguousTitle(overview.needs_review), + description: Number.isSafeInteger(overview.needs_review?.pending) + ? 'Review metadata before assigning a positive / negative pair.' : NOT_AVAILABLE, + label: 'Review metadata', onClick: () => {} }); + // D93 (Q6): the review queue (Needs review) is not part of the workspace yet. + review.querySelector('button').setAttribute('aria-disabled', 'true'); + const [sourcesTitle, sourcesDescription] = sourcesCopy(overview.sources); + const thumbnails = operationRow({ scope, title: THUMBNAILS_LOADING, description: 'Generate only missing previews when necessary.', + label: 'Review thumbnails', onClick: go('sources') }); + const health = surface('Library health', [ + review, + operationRow({ scope, title: 'Backup status', description: backupText(overview.backup), label: 'View backups', + onClick: go('backups') }), + // Until the thumbnail review dialog (S5), Review thumbnails opens Sources, where thumbnail generation lives. + thumbnails, + ]); + const preferences = surface('Workspace preferences', [ + operationRow({ scope, title: 'One display configuration', description: 'Grid and density settings are shared across views.', + label: 'Appearance', onClick: go('appearance') }), + operationRow({ scope, title: sourcesTitle, description: sourcesDescription, label: 'Manage sources', onClick: go('sources') }), + operationRow({ scope, title: 'Statistics', description: 'Runs and ratings by category, with library totals.', label: 'Explore', + onClick: openStatistics }), + ]); + const columns = el('div', undefined, 'two-col'); + columns.append(health, preferences); + return { nodes: [metricsNode(overview), columns], thumbnails }; +} + +/** + * The Overview section's mount (`mountSystem`'s `sections.overview`): returns its loading line, which the server's + * answer replaces in place (the concept's metrics and columns sit directly in `.settings-layout`). A read that a newer + * one superseded, or one answering after the section left, paints nothing. + */ +export function mountOverview(container, { api, scope, navigate = navigateTo, notify = showWorkspaceToast, + latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) { + const overviewRead = latestOnly({ lifetime: scope.signal }); + const statisticsRead = latestOnly({ lifetime: scope.signal }); + const thumbnailsRead = latestOnly({ lifetime: scope.signal }); + let shown = []; + // The thumbnail status (undefined while read, null when it failed), painted into the row whenever both are there. + let thumbnailStatus; + let thumbnailRow = null; + let sources = null; + const paintThumbnails = () => { + if (thumbnailRow) thumbnailRow.querySelector('h3').textContent = thumbnailTitle(thumbnailStatus, sources); + }; + + function swap(nodes) { + const anchor = shown.find(node => node.isConnected); + if (anchor) anchor.before(...nodes); + for (const node of shown) node.remove(); + shown = nodes; + return nodes; + } + + async function openStatistics(button) { + try { + const statistics = await statisticsRead(signal => api.get(STATISTICS_URL, { signal, cache: 'no-store' })); + if (statistics === STALE || scope.signal.aborted) return; + openContentDialog({ scope, title: 'Usage statistics', trigger: button, nodes: statisticsNodes(statistics) }); + } catch (error) { + if (!scope.signal.aborted) notify(STATISTICS_FAILED_TEXT); + } + } + + function loading() { + const status = el('p', LOADING_TEXT, 'small'); + status.setAttribute('role', 'status'); + return [status]; + } + + async function load() { + try { + const overview = await overviewRead(signal => api.get(OVERVIEW_URL, { signal, cache: 'no-store' })); + if (overview === STALE || scope.signal.aborted) return; + const { nodes, thumbnails } = overviewNodes(overview, { scope, navigate, openStatistics }); + thumbnailRow = thumbnails; + sources = overview.sources; + swap(nodes); + paintThumbnails(); + } catch (error) { + if (scope.signal.aborted) return; + const alert = el('p', LOAD_FAILED_TEXT); + alert.setAttribute('role', 'alert'); + const retry = createButton('Retry'); + scope.listen(retry, 'click', () => { + swap(loading()); + load(); + loadThumbnails(); + }); + const panel = el('section', undefined, 'surface'); + panel.append(alert, retry); + swap([panel]); + } + } + async function loadThumbnails() { + let status; + try { + status = await thumbnailsRead(signal => api.get(THUMBNAILS_URL, { signal, cache: 'no-store' })); + } catch (error) { + status = null; + } + if (status === STALE || scope.signal.aborted) return; + thumbnailStatus = status ?? null; + paintThumbnails(); + } + const first = swap(loading()); + load(); + loadThumbnails(); + return first; +} diff --git a/web/js/workspace/system-state.mjs b/web/js/workspace/system-state.mjs new file mode 100644 index 00000000..2c24e67e --- /dev/null +++ b/web/js/workspace/system-state.mjs @@ -0,0 +1,201 @@ +/** + * The System draft store (#249 System S2 Task 5): every System section edits drafts behind the one Save row (the + * concept's `.save-row`, app.js renderSystem 705), in three domains: + * - `browser`: preferences kept in this browser (Appearance); + * - `host`: the server configuration, saved with the `revision` its read returned (`expected_revision`, #220); + * - `runtime`: values applied to the running server only (the log level, Q10b). + * + * `save()` writes browser, then host, then runtime, and only the domains with changes. A domain that fails keeps its + * drafts dirty and its keys `failed()`, so the section highlights them; the others are saved. A host save refused + * because the configuration changed elsewhere (`settings-changed`) is a `conflict` and keeps the draft. A host save before + * `load('host')` returned a revision fails without a request: a blind write would merge over a configuration changed + * elsewhere (S1 reads a null `expected_revision` as "merge into what is on disk"). `reloadHost()` reads the host values + * again (System calls it on mount); a host conflict reloads them itself, so the next save carries the current revision. + * Drafts live for + * the page session (Q10a): a section left and opened again shows them, and a reload of its values keeps them. + * + * Adapters: `readBrowser()` and `readRuntime()` return the values; `readHost()` returns `{values, revision}`. + * `writeBrowser(changes)` returns `{persisted}` (false: kept for this tab only), `writeHost(changes, + * {expected_revision})` the settings response (`{success, persisted, revision}`), `writeRuntime(changes)` anything; + * a write that throws failed. + */ +export { SYSTEM_SECTIONS, isSystemSection } from './state.mjs'; + +/** D92: the Save row's status copy; never the concept's prototype text. */ +export const SETTINGS_STATUS = Object.freeze({ + clean: 'No unsaved changes.', + dirty: 'Unsaved changes.', + browser: 'Saved in this browser.', + host: 'Saved to the server configuration.', + runtime: 'Applied until the server restarts.', + partial: 'Some changes were not saved. Review the highlighted settings.', + tabOnly: 'Saved for this tab only; browser storage is unavailable.', + conflict: 'Settings changed elsewhere. Reload this section to see them.', +}); + +export const DRAFT_DOMAINS = Object.freeze(['browser', 'host', 'runtime']); +const CONFLICT_CODE = 'settings-changed'; +const ADAPTERS = ['readBrowser', 'readHost', 'readRuntime', 'writeBrowser', 'writeHost', 'writeRuntime']; + +/** The status copy of one `save()` outcome (`{browser, host, runtime}`, each clean/saved/tab-only/applied/failed/conflict). */ +export function statusText(outcome) { + const results = DRAFT_DOMAINS.map(domain => outcome?.[domain]); + if (results.includes('conflict')) return SETTINGS_STATUS.conflict; + if (results.includes('failed')) return SETTINGS_STATUS.partial; + const saved = []; + if (outcome?.browser === 'saved') saved.push(SETTINGS_STATUS.browser); + if (outcome?.browser === 'tab-only') saved.push(SETTINGS_STATUS.tabOnly); + if (outcome?.host === 'saved') saved.push(SETTINGS_STATUS.host); + if (outcome?.runtime === 'applied') saved.push(SETTINGS_STATUS.runtime); + return saved.join(' ') || SETTINGS_STATUS.clean; +} + +function requireDomain(domain) { + if (!DRAFT_DOMAINS.includes(domain)) throw new TypeError(`Unknown settings domain: ${domain}`); +} + +/** Create the store; see the module comment for the adapters. */ +export function createSystemDrafts(adapters) { + for (const name of ADAPTERS) { + if (typeof adapters?.[name] !== 'function') throw new TypeError(`System drafts need ${name}`); + } + const baseline = { browser: {}, host: {}, runtime: {} }; + const drafts = { browser: new Map(), host: new Map(), runtime: new Map() }; + const failedKeys = { browser: new Set(), host: new Set(), runtime: new Set() }; + const listeners = new Set(); + let revision = null; + let outcome = null; + let saving = null; + + const notify = () => { for (const listener of [...listeners]) listener(); }; + const changes = domain => Object.fromEntries(drafts[domain]); + + /** Read a domain's saved values; drafts are kept, and a draft equal to the new value is clean. */ + async function load(domain) { + requireDomain(domain); + let values; + if (domain === 'host') { + const read = await adapters.readHost(); + values = read?.values ?? {}; + revision = read?.revision ?? null; + } else { + values = await (domain === 'browser' ? adapters.readBrowser() : adapters.readRuntime()); + } + baseline[domain] = { ...(values ?? {}) }; + for (const [key, value] of drafts[domain]) if (Object.is(baseline[domain][key], value)) drafts[domain].delete(key); + notify(); + return { ...baseline[domain] }; + } + + function get(domain, key) { + requireDomain(domain); + return drafts[domain].has(key) ? drafts[domain].get(key) : baseline[domain][key]; + } + + function set(domain, key, value) { + requireDomain(domain); + if (Object.is(baseline[domain][key], value)) drafts[domain].delete(key); + else drafts[domain].set(key, value); + failedKeys[domain].delete(key); + outcome = null; + notify(); + } + + function dirty(domain) { + if (domain !== undefined) { + requireDomain(domain); + return drafts[domain].size > 0; + } + return DRAFT_DOMAINS.some(name => drafts[name].size > 0); + } + + function discard(domain) { + for (const name of domain === undefined ? DRAFT_DOMAINS : [domain]) { + requireDomain(name); + drafts[name].clear(); + failedKeys[name].clear(); + } + outcome = null; + notify(); + } + + /** The values of one domain as written: they become its saved values, and drafts still equal to them are clean. */ + function commit(domain, written) { + for (const [key, value] of written) { + baseline[domain][key] = value; + if (drafts[domain].has(key) && Object.is(drafts[domain].get(key), value)) drafts[domain].delete(key); + } + } + + function fail(domain, written) { + for (const [key] of written) failedKeys[domain].add(key); + } + + async function saveDomain(domain) { + const written = new Map(drafts[domain]); + if (written.size === 0) return 'clean'; + const values = Object.fromEntries(written); + try { + if (domain === 'browser') { + const result = await adapters.writeBrowser(values); + commit(domain, written); + return result?.persisted === false ? 'tab-only' : 'saved'; + } + if (domain === 'host') { + if (typeof revision !== 'string') { fail(domain, written); return 'failed'; } + const result = await adapters.writeHost(values, { expected_revision: revision }); + if (result?.code === CONFLICT_CODE) { fail(domain, written); return 'conflict'; } + if (!result?.success || result.persisted !== true) { fail(domain, written); return 'failed'; } + if (typeof result.revision === 'string') revision = result.revision; + commit(domain, written); + return 'saved'; + } + await adapters.writeRuntime(values); + commit(domain, written); + return 'applied'; + } catch (error) { + fail(domain, written); + return error?.code === CONFLICT_CODE ? 'conflict' : 'failed'; + } + } + + /** Read the host values again; a failed read leaves host saves refused until one succeeds. */ + const reloadHost = () => load('host').then(() => true, () => false); + + /** Save the dirty domains in order; a Save pressed while one runs joins it. */ + function save() { + saving ??= (async () => { + for (const domain of DRAFT_DOMAINS) failedKeys[domain].clear(); + const result = {}; + for (const domain of DRAFT_DOMAINS) result[domain] = await saveDomain(domain); + // Settings changed elsewhere: read them (and their revision), keeping the drafts (Q10). + if (result.host === 'conflict') await reloadHost(); + return result; + })().then(result => { + saving = null; + outcome = result; + notify(); + return result; + }); + return saving; + } + + return { + load, + reloadHost, + get, + set, + dirty, + discard, + save, + changes, + saving: () => saving !== null, + failed: domain => { requireDomain(domain); return new Set(failedKeys[domain]); }, + /** The Save row's text: the last save's outcome until the next edit, else unsaved or clean. */ + statusText: () => (outcome ? statusText(outcome) : dirty() ? SETTINGS_STATUS.dirty : SETTINGS_STATUS.clean), + subscribe(listener) { + listeners.add(listener); + return () => listeners.delete(listener); + }, + }; +} diff --git a/web/js/workspace/system.mjs b/web/js/workspace/system.mjs new file mode 100644 index 00000000..0fa76585 --- /dev/null +++ b/web/js/workspace/system.mjs @@ -0,0 +1,148 @@ +/** + * Native System (#249 System S2 Task 5): the concept's `renderSystem` shell (app.js 698–706) at `#/system/
`. + * Under the page heading it renders the section nav (`.section-nav`, one button per section, the shown one + * `aria-pressed`; a press navigates to its hash), then `.settings-layout` with the section's body and the Save row + * (`span.small#settings-status` + "Save preferences"). D92: no "Sample configuration" chip, the status copy is + * truthful and "Save preferences" is aria-disabled while nothing is unsaved. A section not delivered yet shows the + * per-section hand-off to its legacy tool (D94), never mock placeholders. Overview is native (system-overview.mjs). + * + * Switching sections keeps the shell (and focus on the pressed button) and disposes the old section's scope, so its + * listeners and timers never outlive it. The drafts are module-level (Q10): they live for the page session. + */ +import { createButton, createScope } from '../shared/toolkit/core.js'; +import { SYSTEM_SECTIONS, navigateTo } from './state.mjs'; +import { createSystemDrafts, DRAFT_DOMAINS } from './system-state.mjs'; +import { sectionNav } from './system-controls.mjs'; +import { healthNotice, mountOverview } from './system-overview.mjs'; + +const SETTINGS_URL = '/prompt_manager/settings'; +const ADMIN = Object.freeze({ href: '/prompt_manager/admin', label: 'Open existing administration' }); +const GALLERY = Object.freeze({ href: '/prompt_manager/gallery', label: 'Open existing image gallery' }); +/** + * D94: where each section's capabilities live until its PR (the plan's legacy capability coverage): the admin page's + * dashboard, Settings, Maintenance, Backup, Restore, Scan, Logs, Diagnostics and Auto-tag tools, and the gallery's + * settings, thumbnails, duplicates and cache tools. + */ +export const SECTION_HANDOFFS = Object.freeze({ + appearance: [GALLERY, ADMIN], + sources: [ADMIN, GALLERY], + integrations: [ADMIN], + tagging: [ADMIN], + maintenance: [ADMIN, GALLERY], + backups: [ADMIN], + diagnostics: [ADMIN], +}); +export const NOT_DELIVERED = 'This setting is not available here yet.'; + +let drafts = null; + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +/** A domain whose settings no delivered section edits yet: nothing can be drafted, and a write would say it failed. */ +const undelivered = async () => { throw new Error(NOT_DELIVERED); }; + +/** The page session's drafts (Q10); the host domain reads and saves the server settings with their revision (#220). */ +export function systemDrafts(api) { + drafts ??= createSystemDrafts({ + readBrowser: () => ({}), + readRuntime: () => ({}), + writeBrowser: undelivered, + writeRuntime: undelivered, + async readHost() { + const result = await api.get(SETTINGS_URL, { cache: 'no-store' }); + // The store reads a missing value as none and a missing revision as unreadable (no host save). + return { values: result.settings, revision: result.revision }; + }, + writeHost: (changes, { expected_revision }) => api.post(SETTINGS_URL, { ...changes, expected_revision }), + }); + return drafts; +} + +/** D94: the shared hand-off panel, linking each legacy tool that still owns this section. */ +function handoff(section) { + const panel = el('section', undefined, 'surface handoff'); + panel.append(el('h2', 'Your current tools are ready'), + el('p', 'This workspace is being introduced in stages. Continue in the existing tool with your saved data.')); + for (const tool of SECTION_HANDOFFS[section]) { + const link = el('a', tool.label, 'workspace-link'); + link.href = window.ApiClient.resolveUrl(tool.href); + panel.append(link); + } + return [panel]; +} + +/** D92: highlight the controls of the settings the last save could not save. */ +function markFailed(layout, store) { + const failed = new Set(DRAFT_DOMAINS.flatMap(domain => [...store.failed(domain)])); + for (const control of layout.querySelectorAll('[data-setting]')) { + if (failed.has(control.dataset.setting)) control.setAttribute('aria-invalid', 'true'); + else control.removeAttribute('aria-invalid'); + } +} + +/** + * The sections delivered so far: Overview (S2 Task 6), and Maintenance's notice with "Run health checks" above the + * hand-off to the maintenance tools that are not native yet (D123). + */ +export const DELIVERED_SECTIONS = Object.freeze({ + overview: mountOverview, + maintenance: (container, { api, activity, scope }) => [healthNotice({ api, activity, scope }), ...handoff('maintenance')], +}); + +/** + * Mount System under the heading `renderHeading` wrote in `main`. Returns `{show(section)}`, which shows another + * section in place. `scope` is the page's view scope; `sections` maps a section id to a delivered section's mount + * (`(container, {api, activity, scope, drafts}) => nodes`), and every other section hands off. `activity` is the page's + * Activity store, which follows the jobs a section starts. + * + * The host values are read on mount (and by the store again after a conflict), so a save carries the current revision; + * until a read returned one, a host save fails without a request (system-state.mjs). + */ +export function mountSystem(main, api, scope, { section, activity, sections = DELIVERED_SECTIONS, navigate = navigateTo }) { + const store = systemDrafts(api); + store.reloadHost(); + const nav = sectionNav({ label: 'System sections', sections: SYSTEM_SECTIONS, active: section, scope, + onSelect: id => navigate(`#/system/${id}`) }); + const layout = el('div', undefined, 'settings-layout'); + const saveRow = el('div', undefined, 'save-row'); + const status = el('span', undefined, 'small'); + status.id = 'settings-status'; + const save = createButton('Save preferences', 'ca-primary'); + saveRow.append(status, save); + layout.append(saveRow); + main.append(nav, layout); + + const paintStatus = () => { + status.textContent = store.statusText(); + if (store.dirty() && !store.saving()) save.removeAttribute('aria-disabled'); + else save.setAttribute('aria-disabled', 'true'); + markFailed(layout, store); + }; + scope.cleanup(store.subscribe(paintStatus)); + scope.listen(save, 'click', () => { + if (save.getAttribute('aria-disabled') === 'true') return; + const saving = store.save(); + paintStatus(); + saving.then(() => { if (!scope.signal.aborted) paintStatus(); }); + }); + + let sectionScope = null; + scope.cleanup(() => sectionScope?.dispose()); + function show(next) { + sectionScope?.dispose(); + sectionScope = createScope(); + for (const button of nav.children) button.setAttribute('aria-pressed', String(button.dataset.value === next)); + while (layout.firstChild !== saveRow) layout.firstChild.remove(); + const mount = sections[next]; + const nodes = mount ? mount(layout, { api, activity, scope: sectionScope, drafts: store }) : handoff(next); + layout.prepend(...nodes); + paintStatus(); + } + show(section); + return { show }; +} diff --git a/web/js/workspace/workspace-dialogs.mjs b/web/js/workspace/workspace-dialogs.mjs index 39b35e61..da53853e 100644 --- a/web/js/workspace/workspace-dialogs.mjs +++ b/web/js/workspace/workspace-dialogs.mjs @@ -11,7 +11,8 @@ * Rescan; no "Import portable recipes" (D15). Rescan shows its running state (aria-disabled "Rescanning…"). D55: the * concept's own "Scan an output folder → Review scan" row follows as "Scan for metadata". * - "Review metadata scan" (a1428–1432 `scan-review`, D55): Sources and Scope in a `dl.meta-grid`, the concept's - * sentence, a backup `.notice` ("Back up now") and Cancel / Start scan; no sample-plan note. + * sentence, a backup `.notice` ("Back up now") and Cancel / Start scan; no sample-plan note. It is one caller of + * `openReviewDialog`, the review every record-changing System operation shares, with its `backupNotice`. * - "Copy prompt" (a1223–1241): the select-to-copy fallback when the clipboard is unavailable (D22). * - One-value input (a1138 `simpleInput`): a "Category"-style field (`input#action-input`) and Cancel / a primary * action; used by the Images selection bar's "Set category" (#249 D8) and the toolbar's Save view. @@ -367,6 +368,98 @@ const BACKUP_BUSY_LABEL = 'Backing up…'; export const SCAN_SCOPE_LABELS = { new: 'New and unresolved files', all: 'All files' }; export const SCAN_ALL_FILES_HINT = 'Also read the files already linked to their prompt again.'; +/** The scan review's backup box copy (D55). */ +export const SCAN_BACKUP_COPY = Object.freeze({ heading: 'Back up the database first (recommended)', + text: 'Downloads a copy of the prompts database before the scan changes any records.' }); + +/** + * The D55 backup box: the concept's System `.notice` (`div` > `h3` + `p`, then `.row-actions` with "Back up now"). + * `onBackup` runs the backup and toasts its outcome; until it settles the button reads "Backing up…" and is + * aria-disabled, so a second press is ignored. It never closes the dialog it sits in. + */ +export function backupNotice({ copy, onBackup }) { + if (typeof copy?.heading !== 'string' || !copy.heading || typeof copy.text !== 'string' || !copy.text) { + throw new TypeError('The backup box needs its heading and text'); + } + requireCallbacks({ onBackup }); + const notice = el('div', undefined, 'notice'); + const text = el('div'); + text.append(el('h3', copy.heading), el('p', copy.text)); + const actions = el('div', undefined, 'row-actions'); + const backup = createButton(BACKUP_LABEL); + actions.append(backup); + notice.append(text, actions); + backup.addEventListener('click', async () => { + if (backup.getAttribute('aria-disabled') === 'true') return; + backup.setAttribute('aria-disabled', 'true'); + backup.textContent = BACKUP_BUSY_LABEL; + try { + await onBackup(); + } finally { + backup.removeAttribute('aria-disabled'); + backup.textContent = BACKUP_LABEL; + } + }); + return notice; +} + +function requireGrid(grid) { + if (!Array.isArray(grid) || grid.length === 0 || grid.some(row => !Array.isArray(row) || row.length !== 2 || + typeof row[0] !== 'string' || !row[0] || typeof row[1] !== 'string')) { + throw new TypeError('The review needs its [term, value] grid'); + } +} + +/** + * The shared record-changing review (the concept's `scan-review`, maintenance plan and restore preview): an optional + * `h3` heading, the `dl.meta-grid` of `[term, value]` rows, page-built `options` nodes, the optional `p.small` + * sentence, the D55 backup box when `backup` is `{copy, onBackup}`, then `.dialog-actions` with `cancelLabel` and the + * primary (`ca-primary`, or the concept's `danger` when `primary.danger`). The primary closes the dialog, then calls + * `onStart()`. Returns `{setValue(term, text)}` for a grid value that follows an option. + */ +export function openReviewDialog({ scope, title, heading = null, grid, options = [], sentence = null, backup = null, + cancelLabel, primary, onStart, trigger = null }) { + requireScope(scope); + requireCallbacks({ onStart }); + for (const [name, text] of Object.entries({ title, cancelLabel, label: primary?.label })) { + if (typeof text !== 'string' || !text) throw new TypeError(`The review needs its ${name}`); + } + requireGrid(grid); + if (!Array.isArray(options) || options.some(node => !node || typeof node !== 'object')) { + throw new TypeError('The review options must be nodes'); + } + const nodes = []; + if (heading) nodes.push(el('h3', heading)); + const list = el('dl', undefined, 'meta-grid'); + const values = new Map(); + for (const [term, value] of grid) { + const entry = el('div'); + const dd = el('dd', value); + values.set(term, dd); + entry.append(el('dt', term), dd); + list.append(entry); + } + nodes.push(list, ...options); + if (sentence) nodes.push(el('p', sentence, 'small')); + if (backup) nodes.push(backupNotice(backup)); + const cancel = createButton(cancelLabel); + const start = createButton(primary.label, primary.danger ? 'danger' : 'ca-primary'); + cancel.addEventListener('click', () => finish()); + start.addEventListener('click', () => { + finish(); + onStart(); + }); + const actions = el('div', undefined, 'dialog-actions'); + actions.append(cancel, start); + nodes.push(actions); + show(title, nodes, { scope, trigger }); + return { + setValue(term, text) { + const dd = values.get(term); + if (dd) dd.textContent = text; + }, + }; +} /** * The concept's "Review metadata scan" (`scan-review`, D55). `sources` is the number of output folders the scan reads. @@ -379,18 +472,9 @@ export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigge requireScope(scope); requireCallbacks({ onBackup, onStart }); if (!Number.isSafeInteger(sources) || sources < 0) throw new TypeError('The scan review needs its source count'); - const grid = el('dl', undefined, 'meta-grid'); - const values = {}; - for (const [term, value] of [['Sources', `${sources.toLocaleString('en-US')} output ${sources === 1 ? 'folder' : 'folders'}`], - ['Scope', SCAN_SCOPE_LABELS.new]]) { - const entry = el('div'); - values[term] = el('dd', value); - entry.append(el('dt', term), values[term]); - grid.append(entry); - } // The concept's System toggle: a `.setting-row` whose checkbox is the Toolkit switch (concept `hydrateSwitches`). const toggle = createSwitch('', '', 'scan-all-files'); - const allFiles = toggle.querySelector('input'); + const allFiles = toggle.children[0]; allFiles.checked = false; const option = el('div', undefined, 'setting-row'); const optionText = el('div'); @@ -402,35 +486,10 @@ export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigge optionText.append(optionLabel, hint); option.append(optionText, toggle); const chosenScope = () => (allFiles.checked ? 'all' : 'new'); - allFiles.addEventListener('change', () => { values.Scope.textContent = SCAN_SCOPE_LABELS[chosenScope()]; }); - const notice = el('div', undefined, 'notice'); - const copy = el('div'); - copy.append(el('h3', 'Back up the database first (recommended)'), - el('p', 'Downloads a copy of the prompts database before the scan changes any records.')); - const backupActions = el('div', undefined, 'row-actions'); - const backup = createButton(BACKUP_LABEL); - backupActions.append(backup); - notice.append(copy, backupActions); - backup.addEventListener('click', async () => { - if (backup.getAttribute('aria-disabled') === 'true') return; - backup.setAttribute('aria-disabled', 'true'); - backup.textContent = BACKUP_BUSY_LABEL; - try { - await onBackup(); - } finally { - backup.removeAttribute('aria-disabled'); - backup.textContent = BACKUP_LABEL; - } - }); - const cancel = createButton('Cancel'); - const start = createButton('Start scan', 'ca-primary'); - cancel.addEventListener('click', () => finish()); - start.addEventListener('click', () => { - const files = chosenScope(); - finish(); - onStart({ scope: files }); - }); - const actions = el('div', undefined, 'dialog-actions'); - actions.append(cancel, start); - show('Review metadata scan', [grid, option, el('p', SCAN_REVIEW_SENTENCE, 'small'), notice, actions], { scope, trigger }); + const review = openReviewDialog({ scope, trigger, title: 'Review metadata scan', + grid: [['Sources', `${sources.toLocaleString('en-US')} output ${sources === 1 ? 'folder' : 'folders'}`], + ['Scope', SCAN_SCOPE_LABELS.new]], + options: [option], sentence: SCAN_REVIEW_SENTENCE, backup: { copy: SCAN_BACKUP_COPY, onBackup }, + cancelLabel: 'Cancel', primary: { label: 'Start scan' }, onStart: () => onStart({ scope: chosenScope() }) }); + allFiles.addEventListener('change', () => review.setValue('Scope', SCAN_SCOPE_LABELS[chosenScope()])); } diff --git a/web/js/workspace/workspace.css b/web/js/workspace/workspace.css index 71173760..c7088b31 100644 --- a/web/js/workspace/workspace.css +++ b/web/js/workspace/workspace.css @@ -82,6 +82,15 @@ main:focus-visible { outline: 2px solid var(--ca-focus); outline-offset: 4px; } .tool-card p { flex: 1; margin: 0 0 12px; } .workspace-link { display: inline-flex; align-items: center; min-height: 44px; padding: 8px 0; font-size: 12px; overflow-wrap: anywhere; } .handoff .workspace-link { margin-top: 12px; } +/* D94: a System section not delivered yet links every legacy tool that still owns it, side by side. */ +.handoff .workspace-link + .workspace-link { margin-inline-start: 18px; /* D94 */ } + +/* D92: System's Save row is aria-disabled while nothing is unsaved (or a save runs), and a setting the last save could + not save is highlighted, as its "Review the highlighted settings" status says. */ +.save-row button[aria-disabled="true"] { opacity: .55; cursor: not-allowed; /* D92 */ } +.setting-row [aria-invalid="true"] { outline: 2px solid var(--red); outline-offset: 2px; /* D92 */ } +/* D93: an Overview row whose action is not available yet (the review queue, Q6) keeps its button, disabled. */ +.operation .row-actions button[aria-disabled="true"] { opacity: .55; cursor: not-allowed; /* D93 */ } @media (max-width: 700px) { .stat-grid { gap: 8px; } .stat { padding: 16px; } From b0a025a326b523ab88ebb3b41bc71e204e4a91b9 Mon Sep 17 00:00:00 2001 From: Vito Date: Fri, 23 Oct 2026 05:17:10 -0700 Subject: [PATCH 2/7] fix(resolver): read NaN as null in the scan's legacy-parser fallback (#304) ComfyUI stamps "is_changed": [NaN] on nodes that always re-run, the PromptManager node among them. The legacy gallery and admin pages replaced NaN with null before JSON.parse, so they traced these files' prompts. The scan's fallback parsed records strictly, so every NaN-stamped file the resolver could not decide (hires-fix and refiner chains, custom latent nodes) fell to "Needs review" although legacy linked its prompt. parse_page_json parses as the legacy pages did, but only the NaN number reads as null: the word inside prompt text is kept. Infinity still makes the record unreadable, as it did in the pages. --- py/api/admin.py | 11 +-- tests/test_generation_params.py | 45 +++++++++++++ tests/test_pair_scan.py | 116 ++++++++++++++++++++++++++++++++ utils/generation_params.py | 25 +++++++ 4 files changed, 193 insertions(+), 4 deletions(-) diff --git a/py/api/admin.py b/py/api/admin.py index 64104893..2b58b59f 100644 --- a/py/api/admin.py +++ b/py/api/admin.py @@ -41,7 +41,7 @@ try: PLACEHOLDERS, GenerationParamsError, extract_generation_params, - parse_js_json, + parse_page_json, ) from ...utils.prompt_pair_resolver import resolve_prompt_pair except ImportError: @@ -51,7 +51,7 @@ except ImportError: PLACEHOLDERS, GenerationParamsError, extract_generation_params, - parse_js_json, + parse_page_json, ) from utils.prompt_pair_resolver import resolve_prompt_pair @@ -1603,11 +1603,14 @@ class AdminRoutesMixin: @staticmethod def _graph_prompt_text(metadata): - """The positive prompt the browser parser traces from the graphs, or None.""" + """The positive prompt the browser parser traces from the graphs, or None. + + Records parse as the legacy pages parsed them (NaN reads as null). + """ records = {} for key in ("prompt", "workflow"): value = metadata.get(key) - records[key] = parse_js_json(value) if isinstance(value, str) else value + records[key] = parse_page_json(value) if isinstance(value, str) else value try: positive = extract_generation_params(records)["positivePrompt"] except GenerationParamsError: diff --git a/tests/test_generation_params.py b/tests/test_generation_params.py index 990d3a60..507cea29 100644 --- a/tests/test_generation_params.py +++ b/tests/test_generation_params.py @@ -5,7 +5,10 @@ returns for the same input (tests/test_generation_params_js_parity.py checks that directly when Node is available). """ +import json import os +import shutil +import subprocess # nosec B404 - runs a fixed Node one-liner import sys import unittest @@ -17,6 +20,7 @@ from utils.generation_params import ( # noqa: E402 extract_generation_params, extract_from_records, parse_js_json, + parse_page_json, ) try: @@ -219,6 +223,47 @@ class ParsesLikeJavaScript(unittest.TestCase): run("seed-beyond-double-precision")["seed"], 4974052037430000001.0 ) + def test_page_parse_reads_nan_numbers_as_null(self): + # gallery.js and admin.js replace NaN with null before JSON.parse. + self.assertEqual( + parse_page_json('{"a": NaN, "b": [NaN], "t": "NaN"}'), + {"a": None, "b": [None], "t": "NaN"}, + ) + self.assertEqual( + parse_page_json('{"seed": 9007199254740993}'), {"seed": 9007199254740992.0} + ) + for text in ('{"a": Infinity}', "[-Infinity]", "not json", "", None): + with self.subTest(text=text): + self.assertIsNone(parse_page_json(text)) + + @unittest.skipUnless(shutil.which("node"), "Node is not installed") + def test_page_parse_equals_the_legacy_pages_for_nan_numbers(self): + # gallery.js/admin.js: two NaN regexes, then JSON.parse (Infinity throws). + texts = [ + '{"1": {"is_changed": [NaN]}, "2": {"is_changed": NaN, "v": 1.5}}', + '{"a":NaN,"b":[NaN, 2]}', + '{"a": Infinity}', + '{"seed": 9007199254740993}', + ] + script = ( + "const texts = JSON.parse(require('fs').readFileSync(0, 'utf8'));" + "console.log(JSON.stringify(texts.map((t) => { try { return {v: JSON.parse(" + "t.replace(/:\\s*NaN\\b/g, ': null').replace(/\\bNaN\\b/g, 'null'))}; }" + " catch (e) { return null; } })));" + ) + completed = subprocess.run( # nosec B603 B607 - fixed argv, no shell + ["node", "-e", script], + input=json.dumps(texts), + capture_output=True, + text=True, + check=True, + timeout=60, + ) + expected = json.loads(completed.stdout) + for text, page in zip(texts, expected): + with self.subTest(text=text): + self.assertEqual(parse_page_json(text), page and page["v"]) + def test_duplicate_keys_keep_the_last_value(self): self.assertEqual(parse_js_json('{"a": 1, "a": 2}'), {"a": 2}) diff --git a/tests/test_pair_scan.py b/tests/test_pair_scan.py index 8b65fb67..b8d914d1 100644 --- a/tests/test_pair_scan.py +++ b/tests/test_pair_scan.py @@ -165,6 +165,122 @@ class TestPairScan(AdminAPITestCase): ) self.assertEqual(self.api.db.get_recent_prompts()["prompts"], []) + def _nan_scanned(self, name, prompt_graph): + """Scan ``prompt_graph`` as ComfyUI writes it: ``is_changed`` is NaN.""" + stamped = { + node_id: {**node, "is_changed": [float("nan")]} + for node_id, node in prompt_graph.items() + } + text = json.dumps(stamped) + self.assertIn("NaN", text) + return self._scanned_text(name, text) + + def _scanned_text(self, name, prompt_text): + meta = {"prompt": prompt_text} + return self.api._ingest_scanned_file(self.output_dir / name, meta) + + @staticmethod + def _hires_fix_graph(positive): + """A PromptManager prompt on a base sampler and a hires-fix sampler. + + The base latent comes from a custom batch node and the hires pass from + a custom image filter: no sampler branch traces, so the scan links the + prompt the legacy pages show. + """ + return { + "1": { + "class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "m.safetensors"}, + }, + "2": { + "class_type": "MyTriggerWords", + "inputs": {"trigger_words": ["1", 2]}, + }, + "3": { + "class_type": "PromptManager", + "inputs": { + "text": positive, + "prepend_text": ["2", 0], + "append_text": "", + "clip": ["1", 1], + }, + }, + "4": { + "class_type": "CLIPTextEncode", + "inputs": {"text": "blur", "clip": ["1", 1]}, + }, + "5": {"class_type": "MyLatentBatch", "inputs": {"batch_size": 6}}, + "6": { + "class_type": "KSampler", + "inputs": { + "seed": 7, + "model": ["1", 0], + "positive": ["3", 0], + "negative": ["4", 0], + "latent_image": ["5", 0], + }, + }, + "7": { + "class_type": "VAEDecode", + "inputs": {"samples": ["6", 0], "vae": ["1", 2]}, + }, + "8": {"class_type": "MyImageFilter", "inputs": {"images": ["7", 0]}}, + "9": { + "class_type": "VAEEncode", + "inputs": {"pixels": ["8", 0], "vae": ["1", 2]}, + }, + "14": { + "class_type": "KSampler", + "inputs": { + "seed": 7, + "denoise": 0.25, + "model": ["1", 0], + "positive": ["3", 0], + "negative": ["4", 0], + "latent_image": ["9", 0], + }, + }, + "15": { + "class_type": "VAEDecode", + "inputs": {"samples": ["14", 0], "vae": ["1", 2]}, + }, + "16": {"class_type": "SaveImage", "inputs": {"images": ["15", 0]}}, + } + + def test_hires_fix_prompt_links_with_or_without_nan_is_changed(self): + # Legacy parity: the legacy pages read NaN as null before parsing, so they + # showed this prompt; the scan left NaN-stamped files in review. + for name, scan in (("plain.png", None), ("stamped.png", "nan")): + with self.subTest(name=name): + g = self._hires_fix_graph("cat on a sofa") + if scan: + status = self._nan_scanned(name, g) + else: + status = self._scanned(name, g) + self.assertIn(status, ("added", "linked")) + self.assertEqual(self._texts(), ["cat on a sofa"]) + + def test_untraced_single_sampler_links_the_legacy_prompt_with_nan(self): + g = self._hires_fix_graph("cat on a sofa") + for node_id in ("7", "8", "9", "14", "15"): + del g[node_id] + g["16"]["inputs"]["images"] = ["6", 0] + self.assertEqual(self._nan_scanned("single.png", g), "added") + self.assertEqual(self._texts(), ["cat on a sofa"]) + + def test_nan_inside_prompt_text_is_kept(self): + # Only the NaN number reads as null; legacy's regex also rewrote text. + g = self._hires_fix_graph("NaN cat, value: NaN") + self.assertEqual(self._nan_scanned("text.png", g), "added") + self.assertEqual(self._texts(), ["NaN cat, value: NaN"]) + + def test_infinity_is_still_unreadable_like_the_legacy_pages(self): + text = json.dumps(self._hires_fix_graph("cat")).replace( + '"seed": 7', '"seed": Infinity' + ) + self.assertIn(self._scanned_text("inf.png", text), ("ambiguous", "unresolved")) + self.assertEqual(self._texts(), []) + def test_raw_png_workflow_is_stored_as_object_and_repeat_is_unchanged(self): path = self.output_dir / "workflow.png" workflow = {"nodes": [{"id": 1, "type": "PromptManagerPair"}]} diff --git a/utils/generation_params.py b/utils/generation_params.py index e65ecd1e..42b7c9b3 100644 --- a/utils/generation_params.py +++ b/utils/generation_params.py @@ -28,6 +28,7 @@ __all__ = [ "extract_generation_params", "extract_from_records", "parse_js_json", + "parse_page_json", ] PLACEHOLDERS = MappingProxyType( @@ -953,6 +954,30 @@ def parse_js_json(text): return None +def _nan_as_null(name): + """The legacy pages' NaN cleaning; Infinity still throws in ``JSON.parse``.""" + if name == "NaN": + return None + raise ValueError(f"{name} is not JSON") + + +def parse_page_json(text): + """A metadata record as the legacy gallery and admin pages parsed it. + + They replace ``NaN`` with ``null`` and then ``JSON.parse`` (ComfyUI writes + ``"is_changed": [NaN]`` for nodes that always re-run, PromptManager among + them). Only the NaN number reads as null here: the pages' regex also + rewrote the word inside strings, which changed prompt text. None where the + parse throws. + """ + if not isinstance(text, str): + return None + try: + return json.loads(text, parse_constant=_nan_as_null, parse_int=_double_integer) + except (ValueError, RecursionError): + return None + + def extract_from_records(prompt_text, workflow_text): """``extract_generation_params`` for raw ``prompt``/``workflow`` record text. From 88e5b78207bc1581103c16b698cc2e6afd8e5974 Mon Sep 17 00:00:00 2001 From: Vito Date: Fri, 23 Oct 2026 05:42:59 -0700 Subject: [PATCH 3/7] fix(gallery): check duplicate deletes in the scanned root and refresh the listing (#305) The duplicate cleanup workflow failed in CI with the deleted file still served (200). The delete was real: the scan reads only the output root, and the browser fixture's input root holds a different shared/same.png. The test checked a bare /images/serve/ URL, which tries every gallery root, so after the output copy went it answered with the input copy. That only happened when the kernel stamped a bulk copy a fraction of a millisecond before the wall-clock stamp on shared/same.png, which made shared/same.png a Delete candidate (CI trace: 0000 .8605533, same .8606777, 0001 .8618972). The workflow now checks each file in its own root (?root=N), checks the input copy survives, and a new workflow forces the CI order through the scan response so the collision runs every time. The server log in that failure also showed the gallery listing a file the cleanup had deleted: delete_duplicate_images left the 30 s listing cache, so the gallery kept counting the file. A successful delete now invalidates the gallery cache. --- py/api/admin.py | 3 + tests/browser/workflows.spec.mjs | 100 +++++++++++++++++++++++-------- tests/test_api_admin.py | 17 ++++++ 3 files changed, 96 insertions(+), 24 deletions(-) diff --git a/py/api/admin.py b/py/api/admin.py index 2b58b59f..dc4e9bcb 100644 --- a/py/api/admin.py +++ b/py/api/admin.py @@ -804,6 +804,9 @@ class AdminRoutesMixin: result = await self._run_in_executor( self._delete_duplicate_images_sync, image_paths, output_dir ) + if result["deleted_count"]: + # The cached listing would keep (and count) the deleted files. + self.invalidate_gallery_cache() response_data = { "success": True, diff --git a/tests/browser/workflows.spec.mjs b/tests/browser/workflows.spec.mjs index 0ee4620c..ad42f378 100644 --- a/tests/browser/workflows.spec.mjs +++ b/tests/browser/workflows.spec.mjs @@ -354,39 +354,48 @@ test('prompt deletion requires confirmation and can retry after a network failur } }); -test('duplicate cleanup cancels safely and deletes only the selected file after a retry', async ({ page, request }) => { +// The duplicate scan reads the output root only, but the fixture's input root holds a different +// shared/same.png. A bare /images/serve/ URL tries every gallery root in turn, so once the +// output copy is deleted it answers 200 with the input copy: check each file in its own root. +const serveIn = (root, path) => `/prompt_manager/images/serve/${path.split(/[\\/]/).map(encodeURIComponent).join('/')}?root=${root}`; +const rootOf = url => new URL(url, 'http://gallery.test').searchParams.get('root'); + +// Which copy the scan keeps depends on file times: the fixture stamps shared/same.png with the +// wall clock, and the kernel can stamp its bulk copies a fraction of a millisecond earlier. +// listOrder rewrites the finished scan's groups so a test can pick the order it needs. +async function openDuplicateScan(page, request, listOrder = null) { await expect(await request.post('/__test__/reset', { data: { extra_images: 2 } })).toBeOK(); + if (listOrder) { + await page.route('**/prompt_manager/scan_duplicates/stream', async route => { + const response = await route.fetch(); + const body = (await response.text()).replace(/^data: (.*)$/gm, (line, json) => { + const event = JSON.parse(json); + if (event.type !== 'complete') return line; + for (const group of event.duplicates) group.images = listOrder(group.images); + return `data: ${JSON.stringify(event)}`; + }); + await route.fulfill({ response, body }); + }); + } await page.goto('/prompt_manager/gallery'); await expect(page.locator('#galleryGrid .image-item')).toHaveCount(4); const originals = await page.locator('#galleryGrid img[data-original]').evaluateAll(images => images.map(image => image.dataset.original)); + const output = rootOf(originals.find(url => url.includes('/bulk/'))); + const input = rootOf(originals.find(url => url.includes('/shared/same.png') && rootOf(url) !== output)); + expect(output).not.toBeNull(); + expect(input).not.toBeNull(); await page.locator('#settingsBtn').click(); await page.locator('#scanDuplicatesBtn').click(); await expect(page.locator('#duplicatesCount')).toHaveText('1'); const candidates = page.locator('.duplicate-checkbox'); await expect(candidates).toHaveCount(2); - const selected = await candidates.first().getAttribute('data-image-path'); - const retained = await candidates.last().getAttribute('data-image-path'); - const serve = path => `/prompt_manager/images/serve/${path.split(/[\\/]/).map(encodeURIComponent).join('/')}`; - await candidates.first().check(); - let deletes = 0; - page.on('request', request => { if (request.url().endsWith('/delete_duplicate_images')) deletes++; }); - confirmNextDialog(page, false, 'permanently delete 1 duplicate image files'); - await page.locator('#removeDuplicatesBtn').click(); - expect(deletes).toBe(0); - await expect(await request.get(serve(selected))).toBeOK(); - await page.route('**/prompt_manager/delete_duplicate_images', route => route.abort('failed'), { times: 1 }); - confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); - await page.locator('#removeDuplicatesBtn').click(); - await expect(page.getByText('Failed to delete duplicate images', { exact: true })).toBeVisible(); - await expect(page.locator('#removeDuplicatesBtn')).toBeEnabled(); - await expect(await request.get(serve(selected))).toBeOK(); - confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); - const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images')); - await page.locator('#removeDuplicatesBtn').click(); - expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 }); - await expect(page.locator('#duplicatesModal')).toBeHidden(); - expect((await request.get(serve(selected))).status()).toBe(404); - await expect(await request.get(serve(retained))).toBeOK(); + return { originals, output, input, candidates }; +} + +async function expectOnlySelectedGone(page, request, { originals, output, input }, selected, retained) { + expect((await request.get(serveIn(output, selected))).status()).toBe(404); + await expect(await request.get(serveIn(output, retained))).toBeOK(); + await expect(await request.get(serveIn(input, 'shared/same.png'))).toBeOK(); await page.reload(); await expect(page.locator('#galleryGrid .image-item')).toHaveCount(3); const remaining = await page.locator('#galleryGrid img[data-original]').evaluateAll(images => images.map(image => image.dataset.original)); @@ -395,4 +404,47 @@ test('duplicate cleanup cancels safely and deletes only the selected file after expect(originals).toContain(url); await expect(await request.get(url)).toBeOK(); } +} + +test('duplicate cleanup cancels safely and deletes only the selected file after a retry', async ({ page, request }) => { + const scan = await openDuplicateScan(page, request); + const { output, candidates } = scan; + const selected = await candidates.first().getAttribute('data-image-path'); + const retained = await candidates.last().getAttribute('data-image-path'); + await candidates.first().check(); + let deletes = 0; + page.on('request', request => { if (request.url().endsWith('/delete_duplicate_images')) deletes++; }); + confirmNextDialog(page, false, 'permanently delete 1 duplicate image files'); + await page.locator('#removeDuplicatesBtn').click(); + expect(deletes).toBe(0); + await expect(await request.get(serveIn(output, selected))).toBeOK(); + await page.route('**/prompt_manager/delete_duplicate_images', route => route.abort('failed'), { times: 1 }); + confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); + await page.locator('#removeDuplicatesBtn').click(); + await expect(page.getByText('Failed to delete duplicate images', { exact: true })).toBeVisible(); + await expect(page.locator('#removeDuplicatesBtn')).toBeEnabled(); + await expect(await request.get(serveIn(output, selected))).toBeOK(); + confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); + const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images')); + await page.locator('#removeDuplicatesBtn').click(); + expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 }); + await expect(page.locator('#duplicatesModal')).toBeHidden(); + await expectOnlySelectedGone(page, request, scan, selected, retained); +}); + +test('deleting a duplicate whose path another gallery root also holds removes only the scanned copy', async ({ page, request }) => { + // The order CI saw: a bulk copy stamped first, so shared/same.png is the first Delete candidate. + const ciOrder = ['bulk/0000.png', 'shared/same.png', 'bulk/0001.png']; + const rank = image => ciOrder.indexOf(image.path.replace(/\\/g, '/')); + const scan = await openDuplicateScan(page, request, images => [...images].sort((a, b) => rank(a) - rank(b))); + const selected = await scan.candidates.first().getAttribute('data-image-path'); + const retained = await scan.candidates.last().getAttribute('data-image-path'); + expect(selected.replace(/\\/g, '/')).toBe('shared/same.png'); + await scan.candidates.first().check(); + confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); + const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images')); + await page.locator('#removeDuplicatesBtn').click(); + expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 }); + await expect(page.locator('#duplicatesModal')).toBeHidden(); + await expectOnlySelectedGone(page, request, scan, selected, retained); }); diff --git a/tests/test_api_admin.py b/tests/test_api_admin.py index 61d703b1..b34c0df9 100644 --- a/tests/test_api_admin.py +++ b/tests/test_api_admin.py @@ -868,6 +868,23 @@ class TestDeleteDuplicatesEndpoint(AdminAPITestCase): self.assertFalse(target.exists()) self.assertTrue(thumb_dir.exists()) + async def test_deleted_files_leave_the_cached_gallery_listing(self): + # The listing is cached for 30 s; a delete must not leave the gallery + # listing (and counting) a file that is gone. + GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)] + for name in ("keep.png", "gone.png"): + (self.output_dir / name).write_bytes(b"same bytes") + + async def listed(): + resp = await self.client.get("/prompt_manager/images/output") + data = await resp.json() + return data["total"], sorted(i["filename"] for i in data["images"]) + + self.assertEqual(await listed(), (2, ["gone.png", "keep.png"])) + resp = await self._post({"image_paths": ["gone.png"]}) + self.assertEqual((await resp.json())["deleted_count"], 1) + self.assertEqual(await listed(), (1, ["keep.png"])) + def test_unexpected_error_per_file_is_reported_without_path(self): self.api._delete_one_duplicate = lambda *a: (_ for _ in ()).throw( PermissionError(13, "Permission denied", os.path.join(self.tmpdir, "x.png")) From add035d94061956f957b6533848e00e8dce94d8d Mon Sep 17 00:00:00 2001 From: Vito Date: Fri, 23 Oct 2026 06:08:44 -0700 Subject: [PATCH 4/7] fix(images): release a collection viewer image Viewer.js has not built yet (#293) * fix(images): release a collection viewer image Viewer.js has not built yet Leaving a viewer image while it was still loading (Previous, Next, the arrow keys, Retry) deferred the inline Viewer.js destroy() until the image's view settled, failed or a 10 s fallback ran. Before its image loads Viewer.js has not built, so nothing could settle: the stage dropped the image, the late load ran Viewer.js's build() on an image without a parent, and getComputedStyle(null) threw a page error ("Argument 1 is not an object" in Firefox, "must be an instance of Element" in WebKit). Under CI load this failed "collection viewer follows filtered loaded order beyond mounted cards". A viewer that is not ready yet is now destroyed at once, which cancels its pending build, and its image src is released. A built viewer with a still-loading view keeps the existing deferral. * fix(images): build Viewer.js only on a loaded viewer image Destroying an unbuilt inline Viewer.js at once (429b3a8) is not safe in one state. Viewer.js's own cancel removes its image listeners only while `image.complete` is false, so an image that had finished downloading with its load event still queued kept them. Closing the viewer then released the src, the queued load built Viewer.js anyway, and its first view found no listed image: "can't access property querySelector, c is undefined" (workspace-history.spec.mjs:222 on Firefox in CI). The viewer now constructs Viewer.js from its own load listener, which the media scope removes when the image is left. Before the load no Viewer.js exists, so leaving the image only releases its src; after it Viewer.js sees a complete image and only its zero-delay build is pending, which destroy() cancels. Transforms report Viewer.js as unavailable only when it did not load at all. * fix(images): describe transforms by the Viewer.js an image was built with 1086436 creates Viewer.js only once the image has loaded, so the "Viewer.js could not load" reason moved from the instance to the live window.Viewer. An image shown natively while Viewer.js was missing then offered transforms as soon as Viewer.js reappeared, which full-inspector.spec.mjs:854 ("collection viewer preserves original download and rooted retries") caught on every project. Each rebuild now captures the Viewer.js it builds with. The reason, the native fallback and the later construction all use that capture. --- tests/browser/workspace-images.spec.mjs | 44 +++++++++++++++++++++++ tests/js/workspace-media-viewer.test.js | 47 +++++++++++++++++++++---- web/js/workspace/media-viewer.mjs | 34 ++++++++++++------ 3 files changed, 107 insertions(+), 18 deletions(-) diff --git a/tests/browser/workspace-images.spec.mjs b/tests/browser/workspace-images.spec.mjs index 4f544cf3..5727a80b 100644 --- a/tests/browser/workspace-images.spec.mjs +++ b/tests/browser/workspace-images.spec.mjs @@ -463,6 +463,50 @@ test('collection viewer ignores delayed media callbacks and keeps inspection ret await page.evaluate(()=>{location.hash='/overview';});await expect(page.locator('.viewer-container,#viewer-dialog')).toHaveCount(0); }); +test('collection viewer leaving an image Viewer.js has not built yet releases it at once',async({page})=>{ + await page.goto(workspace);await expect(cards(page)).toHaveCount(2); + let release,held;const waiting=new Promise(resolve=>{held=resolve;});const pending=new Promise(resolve=>{release=resolve;}); + await page.route('**/images/serve/shared/same.png?*',async route=>{ + const url=new URL(route.request().url()); + if(url.searchParams.get('root')==='0'&&url.searchParams.has('_pm_viewer')) {held();await pending;} + await route.continue().catch(()=>{}); + }); + await cards(page).first().getByRole('button',{name:'Open same.png',exact:true}).click();await waiting; + const preview=page.locator('dialog#viewer-dialog'); + await page.evaluate(()=>{window.unbuiltImage=document.querySelector('#viewer-stage > img');}); + await preview.getByRole('button',{name:'Next media',exact:true}).click(); + await expect(preview.getByRole('button',{name:'Zoom in',exact:true})).toHaveAttribute('aria-disabled','false'); + // The left image is off the stage. Viewer.js has not built it, so the viewer is destroyed and the src released at + // once: its late load must never build Viewer.js into an image without a parent (getComputedStyle(null)). + release(); + await expect.poll(()=>page.evaluate(()=>[window.unbuiltImage.isConnected,window.unbuiltImage.hasAttribute('src'),!!window.unbuiltImage.viewer])).toEqual([false,false,false]); + await page.evaluate(()=>{location.hash='/overview';});await expect(page.locator('.viewer-container,#viewer-dialog')).toHaveCount(0); +}); + +test('collection viewer closed while its image finishes loading never builds Viewer.js afterwards',async({page})=>{ + await page.goto(workspace);await expect(cards(page)).toHaveCount(2); + let held;const waiting=new Promise(resolve=>{held=resolve;}); + // The image is held for the whole test: its late load is delivered below, at the moment Viewer.js cannot cancel. + await page.route('**/images/serve/shared/same.png?*',async route=>{ + const url=new URL(route.request().url()); + if(url.searchParams.get('root')==='0'&&url.searchParams.has('_pm_viewer')) {held();return;} + await route.continue().catch(()=>{}); + }); + await cards(page).first().getByRole('button',{name:'Open same.png',exact:true}).click();await waiting; + const preview=page.locator('dialog#viewer-dialog'); + // Viewer.js's own cancel removes its image listeners only while `image.complete` is false. An image that has + // finished downloading but whose load event is still queued is already complete: close in that state, then deliver it. + await page.evaluate(()=>{window.finishingImage=document.querySelector('#viewer-stage > img');Object.defineProperty(window.finishingImage,'complete',{get:()=>true});}); + await page.keyboard.press('Escape');await expect(preview).toHaveCount(0); + const after=await page.evaluate(async()=>{ + const image=window.finishingImage;image.dispatchEvent(new Event('load')); + // Viewer.js builds an inline viewer in a zero-delay timeout after its image loads; this runs after it. + await new Promise(resolve=>setTimeout(resolve)); + return {src:image.hasAttribute('src'),built:!!image.parentNode?.querySelector('.viewer-container')}; + }); + expect(after).toEqual({src:false,built:false}); +}); + test('collection viewer describes video transforms and releases native playback on navigation',async({page,request})=>{ await expect(await request.post('/__test__/reset',{data:{include_video:true}})).toBeOK();await page.goto(workspace); await page.locator('article[data-image-key="0:clip.webm"] .image-open').click(); diff --git a/tests/js/workspace-media-viewer.test.js b/tests/js/workspace-media-viewer.test.js index 749694a5..d375903f 100644 --- a/tests/js/workspace-media-viewer.test.js +++ b/tests/js/workspace-media-viewer.test.js @@ -124,18 +124,51 @@ test('real inspection navigation preserves slideshow across two advances and def } finally {view.dispose();controller.dispose();parentScope.dispose();globalThis.setTimeout=oldSetTimeout;globalThis.clearTimeout=oldClearTimeout;f.restore();} }); -test('an inline Viewer still loading its image is destroyed only after it has viewed it', async () => { +test('Viewer.js is built only on a loaded image and an unbuilt viewer is released at once', async () => { const {reportedLocationAdapter:adapter} = await import('../../web/js/workspace/history-media.mjs'); - const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c')]; + const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c'),occurrenceMedia('2',0,'d')]; const instances = []; - // Viewer.js's inline destroy() leaves an in-flight view running; its callback then needs a live instance. - f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.viewed = false; instances.push(this); } + // On a loaded image Viewer.js schedules its inline build; until it runs `ready` is false and destroy() cancels it. + f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.ready = false; this.viewed = false; instances.push(this); } destroy() { this.destroyed++; } resize() {} }; const globals = {ResizeObserver:globalThis.ResizeObserver, requestAnimationFrame:globalThis.requestAnimationFrame, cancelAnimationFrame:globalThis.cancelAnimationFrame}; Object.assign(globalThis, {ResizeObserver:class { observe() {} disconnect() {} }, requestAnimationFrame:() => 1, cancelAnimationFrame() {}}); const view = module.createMediaViewer({container:f.container,parentScope,inspectionController:controller,adapter,sender:{open(){}}}); try { - view.open({items,selectedKey:'c'.repeat(64)}); + const stageImage = () => f.walk(f.container,n => n.tagName === 'img')[0]; + view.open({items,selectedKey:'d'.repeat(64)}); + // Viewer.js's own cancel misses an image whose load event is still queued, so none exists before the load. + const unloaded = stageImage(), dropped = [];unloaded.removeAttribute = key => dropped.push(key); + assert.equal(instances.length, 0, 'no Viewer.js before the image loads'); + f.walk(f.container,n => n.attributes['aria-label'] === 'Previous media')[0].fire('click'); + assert.deepEqual(dropped, ['src'], 'an image left before it loads is released at once'); + unloaded.fire('load'); + assert.equal(instances.length, 0, 'its late load builds no Viewer.js'); + stageImage().fire('load'); + const [loading] = instances, image = loading.node, removed = []; + image.removeAttribute = key => removed.push(key); + // Next drops the image whose build is pending; a deferred destroy would let that build run there. + f.walk(f.container,n => n.attributes['aria-label'] === 'Next media')[0].fire('click'); + stageImage().fire('load');assert.equal(instances.length, 2, 'the next image builds its own Viewer.js once loaded'); + assert.equal(loading.destroyed, 1, 'an unbuilt viewer is destroyed at once'); + assert.deepEqual(removed, ['src'], 'and its image is released at once'); + loading.options.viewed.call(loading); + assert.equal(loading.destroyed, 1, 'never destroyed twice'); + } finally { view.dispose(); parentScope.dispose(); f.restore(); Object.assign(globalThis, globals); } +}); + +test('an inline Viewer still loading its image is destroyed only after it has viewed it', async () => { + const {reportedLocationAdapter:adapter} = await import('../../web/js/workspace/history-media.mjs'); + const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c')]; + const instances = []; + // A built (`ready`) inline Viewer.js's destroy() leaves an in-flight view running; its callback then needs a live instance. + f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.ready = true; this.viewed = false; instances.push(this); } + destroy() { this.destroyed++; } resize() {} }; + const globals = {ResizeObserver:globalThis.ResizeObserver, requestAnimationFrame:globalThis.requestAnimationFrame, cancelAnimationFrame:globalThis.cancelAnimationFrame}; + Object.assign(globalThis, {ResizeObserver:class { observe() {} disconnect() {} }, requestAnimationFrame:() => 1, cancelAnimationFrame() {}}); + const view = module.createMediaViewer({container:f.container,parentScope,inspectionController:controller,adapter,sender:{open(){}}}); + try { + view.open({items,selectedKey:'c'.repeat(64)});f.walk(f.container,n => n.tagName === 'img')[0].fire('load'); const [loading] = instances, image = loading.node, removed = []; image.removeAttribute = key => removed.push(key); f.walk(f.container,n => n.attributes['aria-label'] === 'Close image viewer')[0].fire('click'); @@ -148,7 +181,7 @@ test('an inline Viewer still loading its image is destroyed only after it has vi loading.options.viewed.call(loading); assert.equal(loading.destroyed, 1, 'never destroyed twice'); - view.open({items,selectedKey:'c'.repeat(64)}); + view.open({items,selectedKey:'c'.repeat(64)});f.walk(f.container,n => n.tagName === 'img')[0].fire('load'); const shown = instances[1]; shown.viewed = true; shown.options.viewed.call(shown); f.walk(f.container,n => n.attributes['aria-label'] === 'Close image viewer')[0].fire('click'); assert.equal(shown.destroyed, 1, 'a viewed image is destroyed at once'); @@ -236,7 +269,7 @@ test('viewer uses adapter.mediaUrl for image and video', async () => { const img = f.walk(f.container,n => n.tagName === 'img')[0]; assert.equal(img.src, image.url, 'the blob URL is used exactly'); assert.doesNotMatch(img.src, /\?/); - assert.equal(instances[0].node, img, 'Viewer.js shows the blob image'); + img.fire('load');assert.equal(instances[0].node, img, 'Viewer.js shows the blob image'); const download = f.walk(f.container,n => n.attributes['aria-label'] === 'Download original')[0]; assert.equal(download.href, image.url);assert.equal(download.download, 'a.png'); f.walk(f.container,n => n.attributes['aria-label'] === 'Next media')[0].fire('click'); diff --git a/web/js/workspace/media-viewer.mjs b/web/js/workspace/media-viewer.mjs index 4427dd64..bd1b3233 100644 --- a/web/js/workspace/media-viewer.mjs +++ b/web/js/workspace/media-viewer.mjs @@ -35,7 +35,7 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle if (scope.signal.aborted) return; active?.close(false); const {items,selectedKey,trigger = document.activeElement} = options; - let mounted = false, initial, entry, currentKey, viewer = null, mediaScope = null, closed = false; + let mounted = false, initial, entry, currentKey, viewer = null, withoutViewer = false, mediaScope = null, closed = false; let originalDownload = null, inspectionState = inspectionController.state; const session = createViewerSession({adapter,items,initialKey:selectedKey,onChange:(state,selected) => { if (!mounted) { initial = [state,selected]; return; } @@ -115,7 +115,9 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle mediaScope?.dispose();mediaScope = createScope();viewer = null; for (const child of [...stage.children]) if (child !== failure) child.remove(); failure.hidden = true; - const mediaLocal = mediaScope, key = session.state.key; + const mediaLocal = mediaScope, key = session.state.key, ViewerJs = window.Viewer; + // Transforms follow the Viewer.js this media was built with, which builds once the image has loaded. + withoutViewer = !ViewerJs; const current = () => !local.signal.aborted && !mediaLocal.signal.aborted && session.state.key === key; // An adapter's own media URL (a session-owned blob:) is used exactly, so a retry reuses it; a server // original gets a fresh cache-buster on every build. @@ -142,15 +144,23 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle },{once:true}); } else { node.alt = selected.filename;node.decoding = 'async'; - node.hidden = !!window.Viewer; + node.hidden = !!ViewerJs; // With Viewer.js the src is released after the viewer (see release below). - if (!window.Viewer) { + if (!ViewerJs) { mediaLocal.listen(node,'load',() => {if (current()) session.ready(key);}); mediaLocal.cleanup(() => node.removeAttribute('src')); } } node.src = src;stage.append(node); - if (!selected.is_video && window.Viewer) { + if (!selected.is_video && ViewerJs) { + // Viewer.js is built on an image that has loaded. Its own cancel removes its image listeners only + // while `image.complete` is false, so an image left with its load event still queued would build a + // viewer into the released image after destroy() (#293). Until then no viewer exists to cancel. + let mountedViewer = false; + mediaLocal.listen(node,'load',() => { if (current()) { mountedViewer = true;mountViewer(); } },{once:true}); + mediaLocal.cleanup(() => { if (!mountedViewer) node.removeAttribute('src'); }); + } + function mountViewer() { let fitFrame = 0, fitWidth, fitHeight, viewSettled = false, released = false, fallback = 0; const release = () => { if (released) return; @@ -166,7 +176,7 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle fitWidth = width;fitHeight = height;instance.resize(); }); }; - const instance = new window.Viewer(node,{inline:true,toolbar:false,navbar:false,title:false,button:false, + const instance = new ViewerJs(node,{inline:true,toolbar:false,navbar:false,title:false,button:false, keyboard:false,fullscreen:false,transition:false,viewed() { viewSettled = true; if (mediaLocal.signal.aborted) { release(); return; } @@ -176,10 +186,12 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle const observer = new ResizeObserver(scheduleFit);observer.observe(stage); viewer = instance;mediaLocal.cleanup(() => { observer.disconnect();cancelAnimationFrame(fitFrame); - // Inline Viewer.js keeps viewing a still-loading image after destroy(), and that view - // rebuilds its list from images that still have a src; keep both until the view - // settles, fails or times out. - if (viewSettled) { release(); return; } + // Before Viewer.js has built (its zero-delay build is pending), destroy() cancels that build; + // deferring it would let the build run into the image the stage has just dropped. + // Once built, inline Viewer.js keeps viewing a still-loading image after destroy(), and that + // view rebuilds its list from images that still have a src; keep both until the view settles, + // fails or times out. + if (viewSettled || !instance.ready) { release(); return; } node.addEventListener('error',release,{once:true}); fallback = setTimeout(release,PENDING_VIEW_DESTROY_MS); }); @@ -188,7 +200,7 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle } function updateControls(state) { const unavailable = inspectionState.refreshRequired ? inspectionState.message : state.phase === 'error' ? 'Media could not be loaded. Retry this file.' : entry.is_video ? - 'Image transforms are unavailable for video.' : !viewer ? + 'Image transforms are unavailable for video.' : withoutViewer ? 'Image transforms are unavailable because Viewer.js could not load.' : !state.mediaReady ? 'Image transforms are available when the image is ready.' : ''; reason.textContent = unavailable;reason.hidden = !unavailable; for (const node of transforms) { From 7511a0fd3837502b2fa680c12ff18d977b432034 Mon Sep 17 00:00:00 2001 From: Vito Date: Fri, 23 Oct 2026 06:29:45 -0700 Subject: [PATCH 5/7] fix(activity): no Cancel on jobs that ended before Activity opened (#306) * fix(activity): drop Cancel job from a job that had ended before Activity opened Since #289 the Activity list removed an item's actions only when they were isConnected. On the first render the item is not in the page yet, so a job that had already ended when Activity opened kept its "Cancel job" (and a running job's progress was moved after its actions, then back). The list now asks parentNode instead. The review spec "a scan job with review items offers Review metadata in Activity" failed on this before the change (Expected 0 buttons on a finished job, Received 1) and passes after it. * test(backup): wait for the restore, not a 5 s timer, in the thread test test_database_is_usable_from_every_thread_after_restore let its worker wait 5 s for the restore. On a stalled Windows runner (py3.11, where the previous restore test also took 25 s instead of 0.3 s) the restore took longer, the wait timed out, and the worker read the old database before the restore ran. The worker now waits for the event under a 120 s bound that only a hung test reaches, reports a timeout as an error, and the main thread releases it even when the restore raises. With a restore delayed by 6 s the old test fails and the new one passes. * test(activity): no Cancel job on jobs that ended before Activity opened Since #289 a list item dropped its actions only when they were isConnected, so the first render of jobs the store already knew (Activity opened from another page) kept "Cancel job" on a finished or cancelled job and moved a running job's progress after its actions. Activity's own list read renders again a moment later and hides it, so the new spec holds that read and checks the first render: no progress or Cancel job on the ended jobs, and a running job's children in order heading, detail, progress, actions. With the parentNode fix reverted it fails on chromium and touch (Expected 0, Received 1 on the finished job); with it, it passes. --- tests/browser/workspace-activity.spec.mjs | 36 +++++++++++++++++++++++ tests/test_database_backup.py | 22 ++++++++++---- web/js/workspace/activity.mjs | 8 +++-- 3 files changed, 58 insertions(+), 8 deletions(-) diff --git a/tests/browser/workspace-activity.spec.mjs b/tests/browser/workspace-activity.spec.mjs index cbd848c0..a3a16165 100644 --- a/tests/browser/workspace-activity.spec.mjs +++ b/tests/browser/workspace-activity.spec.mjs @@ -724,6 +724,42 @@ test('a held legacy job shows no Cancel in the list or its view', async ({ page, await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']); }); +// Since #289 an item dropped its Cancel job only when it was already in the page, so the first render of jobs the +// store already knew (Activity opened from another page) kept a Cancel job on a job that had ended, and moved a +// running job's progress after its actions. The read Activity starts on opening is held, so that first render is +// what the page shows (the read's own render would otherwise repair it a moment later). +test('jobs that ended before Activity opened show no Cancel job, a running one keeps its progress first', async ({ page, request }) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const finished = await startJob(request, { title: 'Finished scan' }); + await finish(request, finished.id); + const cancelled = await startJob(request, { title: 'Cancelled scan' }); + await finish(request, cancelled.id, 'cancelled'); + await startJob(request, { title: 'Running scan' }); + // The page's first list read has all three jobs once the running one shows in the pill. + await page.goto(`${workspace}#/images`); + await expect(pill(page)).toBeVisible(); + let hold; + const held = new Promise(resolve => { hold = resolve; }); + // Only the first read is held; any later one goes through. + await page.route(url => url.pathname === '/prompt_manager/jobs', route => { + if (!hold) return route.continue(); + hold(route); + hold = null; + }); + await activityButton(page).click(); + const route = await held; + await expect(items(page)).toHaveCount(3); + const item = title => items(page).filter({ has: page.getByRole('link', { name: new RegExp(`${title}$`) }) }); + for (const ended of ['Finished scan', 'Cancelled scan']) { + await expect(item(ended).locator('progress, .row-actions')).toHaveCount(0); + await expect(item(ended).getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(0); + } + await expect(item('Running scan').getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(1); + expect(await item('Running scan').evaluate(node => [...node.children].map(child => child.localName + (child.className ? `.${child.className}` : '')))) + .toEqual(['div.panel-heading', 'p', 'progress', 'div.row-actions']); + await route.continue(); +}); + test('busy start names the running job', async ({ page, request }) => { await page.setViewportSize({ width: 1600, height: 1100 }); // The page's timers are Playwright's, so the toast's lifetime below is run out rather than waited for. diff --git a/tests/test_database_backup.py b/tests/test_database_backup.py index deb1346f..93686971 100644 --- a/tests/test_database_backup.py +++ b/tests/test_database_backup.py @@ -24,6 +24,9 @@ from database.operations import PromptDatabase from tests.open_handles import assert_closed from utils.hashing import generate_prompt_hash +# A bound on thread hand-offs that only a hung test reaches (never a pacing timer). +THREAD_WAIT_SECONDS = 120 + def _count_prompts(path): """Row count read through a brand-new connection (closed before return).""" @@ -350,7 +353,13 @@ class TestRestoreFromFile(BackupTestCase): def worker(): self.db.search_prompts(text="old") # opens this thread's connection ready.set() - proceed.wait(5) + # Wait for the restore itself, not for a timer: a stalled Windows + # runner once took longer than 5 s to restore, and the worker then + # read the old database before the restore had run. + if not proceed.wait(THREAD_WAIT_SECONDS): + results["error"] = "the restore did not finish" + self.db.close() + return try: results["texts"] = [p["text"] for p in self.db.search_prompts()] except Exception as exc: # noqa: BLE001 - reported through results @@ -360,12 +369,15 @@ class TestRestoreFromFile(BackupTestCase): thread = threading.Thread(target=worker) thread.start() - ready.wait(5) + self.assertTrue(ready.wait(THREAD_WAIT_SECONDS)) - self.db.model.restore_from_file(src) - proceed.set() - thread.join(5) + try: + self.db.model.restore_from_file(src) + finally: + proceed.set() + thread.join(THREAD_WAIT_SECONDS) + self.assertFalse(thread.is_alive()) self.assertNotIn("error", results) self.assertEqual(results["texts"], ["restored"]) self.assertEqual([p["text"] for p in self.db.search_prompts()], ["restored"]) diff --git a/web/js/workspace/activity.mjs b/web/js/workspace/activity.mjs index eb7bc14b..f4deedee 100644 --- a/web/js/workspace/activity.mjs +++ b/web/js/workspace/activity.mjs @@ -148,18 +148,20 @@ function itemFor(job, scope) { chip.textContent = state; detail.textContent = jobDetail(next); const running = next.state === 'running'; + // parentNode, not isConnected: on the first render the item is not in the page yet, so a job that had already + // ended when Activity opened kept its Cancel job (and a running one's progress moved after its actions). if (running) { progress.value = next.progress; progress.textContent = `${next.progress}%`; progress.setAttribute('aria-label', `${next.title} progress`); - if (!progress.isConnected) item.insertBefore(progress, actions.isConnected ? actions : null); + if (!progress.parentNode) item.insertBefore(progress, actions.parentNode ? actions : null); } else { progress.remove(); } // Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel job. if (running && next.cancellable) { - if (!actions.isConnected) item.append(actions); - } else if (actions.isConnected) { + if (!actions.parentNode) item.append(actions); + } else if (actions.parentNode) { // A focused Cancel job that goes away hands its focus to the item's title link, never to the page body. if (actions.contains(document.activeElement)) link.focus(); actions.remove(); From 966a25d1293c2c9b59278059452b9a27a072979c Mon Sep 17 00:00:00 2001 From: Vito Date: Fri, 23 Oct 2026 06:48:21 -0700 Subject: [PATCH 6/7] fix(scan): ingest metadata whose graph holds NaN or Infinity (#303) * fix(scan): ingest metadata whose graph holds NaN or Infinity ComfyUI writes its prompt with Python's default json.dumps, so a node whose IS_CHANGED returns float("nan") saves "is_changed": [NaN] in every PNG. The exact-pair ingestion stored the snapshots with allow_nan=False, so the scan failed those files ("Out of range float values are not JSON compliant: nan") and the monitor dropped their link. The legacy pages replaced NaN with null before JSON.parse and ingested them. - utils/json_strict.dumps_strict: strict JSON with non-finite numbers as null; identical bytes for every other value. - Pair snapshots and provenance, and the plain prompt link, store strict JSON (the legacy link wrote bare NaN into the database). - The browser-parser scan fallback parses records as the legacy pages do (parse_page_json), so a graph with NaN still traces its prompt. - The prompt images route answers strict JSON instead of regex-replacing "NaN" (which also rewrote prompt text such as "ratio: NaN"); the NaN cleaners also null Infinity. Prompt text and its hash are unchanged: only numbers become null. * fix(scan): leave the browser-parser NaN parse to the resolver parity fix The scan fallback's legacy-page parse (parse_page_json in _graph_prompt_text) is owned by the resolver parity change, which adds the same helper with the legacy pages' exact rule (NaN reads as null, Infinity still rejected). This branch keeps only the strict serialization of stored and answered JSON. --- database/operations.py | 13 +- database/pair_ingestion.py | 10 +- py/api/__init__.py | 5 +- py/api/images.py | 30 +-- tests/test_nonfinite_metadata.py | 360 +++++++++++++++++++++++++++++++ utils/json_strict.py | 42 ++++ 6 files changed, 427 insertions(+), 33 deletions(-) create mode 100644 tests/test_nonfinite_metadata.py create mode 100644 utils/json_strict.py diff --git a/database/operations.py b/database/operations.py index 8424e880..024dc867 100644 --- a/database/operations.py +++ b/database/operations.py @@ -4,6 +4,7 @@ Database operations for KikoTextEncode prompt storage and retrieval. import sqlite3 import json +import math import datetime import os from typing import Optional, List, Dict, Any, Set, Tuple @@ -26,6 +27,7 @@ from .health import HealthOperations, iter_consistency_issues try: from ..utils.logging_config import get_logger from ..utils.hashing import generate_prompt_hash + from ..utils.json_strict import dumps_strict except ImportError: import sys @@ -33,6 +35,7 @@ except ImportError: sys.path.insert(0, current_dir) from utils.logging_config import get_logger from utils.hashing import generate_prompt_hash + from utils.json_strict import dumps_strict # Subquery to fetch tags from junction table, embedded in SELECT statements @@ -1697,9 +1700,9 @@ class PromptDatabase( width, height, image_format, - json.dumps(metadata.get("workflow", {})), - json.dumps(metadata.get("prompt", {})), - json.dumps(metadata.get("parameters", {})), + dumps_strict(metadata.get("workflow", {})), + dumps_strict(metadata.get("prompt", {})), + dumps_strict(metadata.get("parameters", {})), ), ) conn.commit() @@ -1904,13 +1907,13 @@ class PromptDatabase( obj: The object to clean (dict, list, or scalar) Returns: - Cleaned object with NaN values replaced by None + Cleaned object with NaN and Infinity values replaced by None """ if isinstance(obj, dict): return {key: self._clean_nan_values(value) for key, value in obj.items()} elif isinstance(obj, list): return [self._clean_nan_values(item) for item in obj] - elif isinstance(obj, float) and str(obj) == "nan": + elif isinstance(obj, float) and not math.isfinite(obj): return None else: return obj diff --git a/database/pair_ingestion.py b/database/pair_ingestion.py index cad3641a..27781987 100644 --- a/database/pair_ingestion.py +++ b/database/pair_ingestion.py @@ -1,6 +1,5 @@ """Atomic metadata enrichment for a single known generation context.""" -import json import os from typing import Any, Dict @@ -9,8 +8,10 @@ from .operations import _coerce_file_info try: from ..utils.hashing import generate_prompt_hash + from ..utils.json_strict import dumps_strict except ImportError: from utils.hashing import generate_prompt_hash + from utils.json_strict import dumps_strict def ingest_resolved_pair( @@ -24,9 +25,7 @@ def ingest_resolved_pair( negative_text = negative.get("storage_text", negative["text"]) normalized = normalize_image_path(image_path) provenance = {"resolver_version": 1, "positive": positive, "negative": negative} - encoded = json.dumps( - provenance, sort_keys=True, separators=(",", ":"), allow_nan=False - ) + encoded = dumps_strict(provenance, sort_keys=True, separators=(",", ":")) with db._pair_transaction() as conn: images = conn.execute( "SELECT gi.*,p.hash AS positive_hash FROM generated_images gi " @@ -67,8 +66,9 @@ def ingest_resolved_pair( ).fetchone() pairs_added = int(prior_pair is None) file_size, width, height, image_format = _coerce_file_info(metadata) + # ComfyUI's NaN/Infinity are stored as null: the snapshot is strict JSON. snapshots = { - column: json.dumps(metadata[key], sort_keys=True, allow_nan=False) + column: dumps_strict(metadata[key], sort_keys=True) for column, key in ( ("workflow_data", "workflow"), ("prompt_metadata", "prompt"), diff --git a/py/api/__init__.py b/py/api/__init__.py index fbff121f..34ae5dac 100644 --- a/py/api/__init__.py +++ b/py/api/__init__.py @@ -16,6 +16,7 @@ import datetime import functools import gzip as gzip_module import json +import math import ntpath import os import re @@ -609,12 +610,12 @@ class PromptManagerAPI( return prompts def _clean_nan_recursive(self, obj): - """Recursively clean NaN values from nested data structures.""" + """Recursively replace NaN and Infinity in nested data with None.""" if isinstance(obj, dict): return {key: self._clean_nan_recursive(value) for key, value in obj.items()} elif isinstance(obj, list): return [self._clean_nan_recursive(item) for item in obj] - elif isinstance(obj, float) and str(obj) == "nan": + elif isinstance(obj, float) and not math.isfinite(obj): return None else: return obj diff --git a/py/api/images.py b/py/api/images.py index 69ad96fa..3bf0a341 100644 --- a/py/api/images.py +++ b/py/api/images.py @@ -3,6 +3,7 @@ import asyncio import base64 import binascii +import functools import hashlib import hmac import json @@ -37,6 +38,7 @@ try: from ...database.models import normalize_image_path from ...utils.card_seed import card_seed, header_params, parsed_positive from ...utils.parallel import map_parallel + from ...utils.json_strict import dumps_strict from ...utils.inspection_response import ( encode_inspection_response, library_metadata, @@ -56,6 +58,7 @@ except ImportError: from database.models import normalize_image_path from utils.card_seed import card_seed, header_params, parsed_positive from utils.parallel import map_parallel + from utils.json_strict import dumps_strict from utils.inspection_response import ( encode_inspection_response, library_metadata, @@ -982,27 +985,12 @@ class ImageRoutesMixin: # Add url and thumbnail_url so the frontend can serve them cleaned_images = self._present_images(cleaned_images) - # Additional fallback: convert to JSON string and clean NaN values manually - try: - response_data = {"success": True, "images": cleaned_images} - # Convert to JSON string - json_str = json.dumps(response_data, default=str) - - # Clean any remaining NaN values with regex - json_str = re.sub(r":\s*NaN", ": null", json_str) - json_str = re.sub(r"\[\s*NaN\s*\]", "[null]", json_str) - json_str = re.sub(r",\s*NaN\s*,", ", null,", json_str) - json_str = re.sub(r",\s*NaN\s*\]", ", null]", json_str) - json_str = re.sub(r"\[\s*NaN\s*,", "[null,", json_str) - - # Parse back to verify it's valid JSON - cleaned_data = json.loads(json_str) - - return web.json_response(cleaned_data) - except Exception as json_error: - self.logger.error(f"JSON cleaning error: {json_error}") - # Fallback to original response - return web.json_response({"success": True, "images": cleaned_images}) + # Rows an older version stored may still hold NaN/Infinity: answer + # strict JSON, and leave prompt text that spells "NaN" untouched. + return web.json_response( + {"success": True, "images": cleaned_images}, + dumps=functools.partial(dumps_strict, default=str), + ) except Exception as e: self.logger.error(f"Get prompt images error: {e}") diff --git a/tests/test_nonfinite_metadata.py b/tests/test_nonfinite_metadata.py new file mode 100644 index 00000000..ff6abcd5 --- /dev/null +++ b/tests/test_nonfinite_metadata.py @@ -0,0 +1,360 @@ +"""ComfyUI's ``NaN``/``Infinity`` metadata ingests, and is stored as strict JSON. + +ComfyUI serializes the prompt with Python's ``json.dumps``, so a node whose +``IS_CHANGED`` returns ``float("nan")`` (the "always re-run" idiom) writes +``"is_changed": [NaN]`` into every saved PNG; widgets can hold ``Infinity``. +The legacy pages replaced ``NaN`` with ``null`` before ``JSON.parse``, so such +files must ingest. Whatever is stored or answered as JSON stays strict (the +non-finite numbers become ``null``) and the prompt text and hash never change. +""" + +import json +import logging +import os +import sqlite3 +import sys +import tempfile +import time +import types +import unittest +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import patch + +from PIL import Image, PngImagePlugin + +from database.operations import PromptDatabase +from tests.history_waits import drain_writer +from tests.test_api_admin import AdminAPITestCase +from tests.test_api_images import ImageAPITestCase +from utils.hashing import generate_prompt_hash +from utils.image_monitor import ImageGenerationHandler + +try: + from tests.open_handles import assert_closed +except ImportError: # discovered with tests/ as the top-level directory + from open_handles import assert_closed + +# The words "NaN" and "Infinity" inside the prompt are text, never numbers. +POSITIVE = " A cat named NaN beside an Infinity pool, ratio: NaN, [NaN] " +NEGATIVE = "blurry" +NAN, INF = float("nan"), float("inf") + + +def graph(positive=POSITIVE, negative=NEGATIVE): + """An API prompt graph with non-finite numbers where ComfyUI writes them.""" + g = { + "1": {"class_type": "CLIPTextEncode", "inputs": {"text": positive}}, + "2": {"class_type": "CLIPTextEncode", "inputs": {"text": negative}}, + "3": { + "class_type": "KSampler", + "inputs": {"positive": ["1", 0], "negative": ["2", 0], "model": ["4", 0]}, + }, + "4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}}, + "5": { + "class_type": "Image Filter", + "inputs": {"strength": INF, "floor": -INF}, + "_meta": {"title": "Image Filter"}, + "is_changed": [NAN], + }, + } + return g + + +WORKFLOW = { + "nodes": [ + {"id": 5, "type": "Image Filter", "widgets_values": [NAN, INF, -INF]}, + ], + "links": [], +} + + +def comfy_text(value): + """The record text ComfyUI writes: Python's default, non-strict JSON.""" + text = json.dumps(value) + assert "NaN" in text and "Infinity" in text + return text + + +def write_png(path, prompt_graph=None, workflow=WORKFLOW): + info = PngImagePlugin.PngInfo() + info.add_text("prompt", comfy_text(prompt_graph or graph())) + if workflow is not None: + info.add_text("workflow", comfy_text(workflow)) + path.parent.mkdir(parents=True, exist_ok=True) + with Image.new("RGB", (4, 4)) as image: + image.save(path, pnginfo=info) + return path + + +def _reject(name): + raise ValueError(f"non-strict JSON constant {name}") + + +def strict_loads(text): + return json.loads(text, parse_constant=_reject) + + +class StrictRows: + """Assertions over the rows a write left behind.""" + + def image_rows(self, db): + with db.model.get_connection() as conn: + return [ + dict(row) + for row in conn.execute( + "SELECT prompt_id, workflow_data, prompt_metadata, parameters " + "FROM generated_images ORDER BY id" + ) + ] + + def assert_strict_snapshot(self, row, *, workflow=True): + prompt = strict_loads(row["prompt_metadata"]) + self.assertEqual(prompt["5"]["is_changed"], [None]) + self.assertEqual(prompt["5"]["inputs"], {"strength": None, "floor": None}) + self.assertEqual(prompt["1"]["inputs"]["text"], POSITIVE) + stored_workflow = strict_loads(row["workflow_data"]) + if workflow: + self.assertEqual( + stored_workflow["nodes"][0]["widgets_values"], [None, None, None] + ) + if row["parameters"] is not None: + strict_loads(row["parameters"]) + + def assert_prompt_text(self, db, prompt_id): + with db.model.get_connection() as conn: + row = conn.execute( + "SELECT text, hash FROM prompts WHERE id=?", (prompt_id,) + ).fetchone() + self.assertEqual(row["text"], POSITIVE.strip()) + self.assertEqual(row["hash"], generate_prompt_hash(POSITIVE)) + + +def dumps_strict(value, **kwargs): + from utils.json_strict import dumps_strict as dumps + + return dumps(value, **kwargs) + + +class TestDumpsStrict(unittest.TestCase): + def test_finite_values_serialize_byte_identically(self): + value = {"b": [1, 2.5, "NaN", None, True], "a": {"x": "Infinity"}} + for kwargs in ({}, {"sort_keys": True, "separators": (",", ":")}): + with self.subTest(kwargs=kwargs): + self.assertEqual( + dumps_strict(value, **kwargs), + json.dumps(value, allow_nan=False, **kwargs), + ) + + def test_non_finite_numbers_become_null_and_strings_stay(self): + value = {"n": NAN, "i": [INF, -INF], "s": "NaN Infinity", "t": (NAN, 1)} + encoded = dumps_strict(value, sort_keys=True) + self.assertEqual( + strict_loads(encoded), + {"n": None, "i": [None, None], "s": "NaN Infinity", "t": [None, 1]}, + ) + + def test_other_errors_still_raise(self): + cycle = [] + cycle.append(cycle) + for value in (cycle, [NAN, cycle]): + with self.assertRaisesRegex(ValueError, "Circular"): + dumps_strict(value) + with self.assertRaises(TypeError): + dumps_strict({"x": object()}) + self.assertEqual( + dumps_strict({"x": object()}, default=lambda _: NAN), '{"x": null}' + ) + + +class TestScanIngestsNonFiniteMetadata(StrictRows, AdminAPITestCase): + def ingest(self, name, prompt_graph=None): + path = write_png(self.output_dir / name, prompt_graph) + extracted = self.api._extract_comfyui_metadata(str(path)) + return self.api._ingest_scan_batch_sync([(path, extracted)]) + + def test_resolved_pair_with_nan_is_added_with_strict_snapshots(self): + counts = self.ingest("nan.png") + self.assertEqual(counts["failed"], 0) + self.assertEqual(counts["added"], 1) + rows = self.image_rows(self.api.db) + self.assertEqual(len(rows), 1) + self.assert_strict_snapshot(rows[0]) + self.assert_prompt_text(self.api.db, rows[0]["prompt_id"]) + pairs = self.api.db.list_prompt_pairs()["pairs"] + self.assertEqual( + [(p["positive_text"], p["negative_text"]) for p in pairs], + [(POSITIVE.strip(), NEGATIVE)], + ) + # A repeated scan compares the same strict snapshot: unchanged. + self.assertEqual(self.ingest("nan.png")["unchanged"], 1) + + async def test_scan_route_reports_no_failures(self): + from py.config import GalleryConfig + + write_png(self.output_dir / "nan.png") + GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)] + resp = await self.client.request("POST", "/prompt_manager/scan") + body = await resp.text() + self.assertEqual(resp.status, 200) + self.assertIn('"type": "complete"', body) + self.assertIn('"added": 1', body) + self.assertIn('"failed": 0', body) + + +class TestMonitorIngestsNonFiniteMetadata(StrictRows, unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.db = PromptDatabase(str(Path(self.temp.name) / "db.sqlite")) + self.pid = self.db.save_prompt( + POSITIVE.strip(), prompt_hash=generate_prompt_hash(POSITIVE) + ) + self.tracker = SimpleNamespace( + pop_next_prompt=lambda: None, + get_current_prompt=lambda: None, + extend_prompt_timeout=lambda *args: None, + ) + self.handler = ImageGenerationHandler(self.db, self.tracker) + self.handler.logger = logging.getLogger("test.nonfinite.monitor") + self.handler.logger.disabled = True + + def tearDown(self): + self.db.close_all() + assert_closed(self, self.temp.name) + self.temp.cleanup() + + def test_new_image_with_nan_links_its_pair(self): + # The monitor links only images a PromptManager node produced. + g = graph() + g["1"]["class_type"] = "PromptManager" + path = write_png(Path(self.temp.name) / "out" / "nan.png", g) + self.handler.process_new_image(str(path)) + rows = self.image_rows(self.db) + self.assertEqual([row["prompt_id"] for row in rows], [self.pid]) + self.assert_strict_snapshot(rows[0]) + self.assert_prompt_text(self.db, self.pid) + pairs = self.db.list_prompt_pairs()["pairs"] + self.assertEqual(len(pairs), 1) + self.assertEqual(pairs[0]["image_count"], 1) + + def test_legacy_link_stores_strict_json(self): + # The plain prompt link (no exact pair) must not write NaN either. + path = str(Path(self.temp.name) / "plain.png") + metadata = {"prompt": graph(), "workflow": WORKFLOW, "parameters": {"x": NAN}} + self.assertTrue(self.db.link_image_to_prompt(self.pid, path, metadata)) + row = self.image_rows(self.db)[0] + self.assert_strict_snapshot(row) + self.assertEqual(strict_loads(row["parameters"]), {"x": None}) + + def test_stored_images_read_back_without_non_finite_numbers(self): + path = str(Path(self.temp.name) / "legacy.png") + self.db.link_image_to_prompt(self.pid, path, {"prompt": graph()}) + with self.db.model.get_connection() as conn: + # A row an older version wrote with Python's default json.dumps. + conn.execute( + "UPDATE generated_images SET prompt_metadata=?", (comfy_text(graph()),) + ) + conn.commit() + images = self.db.get_prompt_images(str(self.pid)) + strict_loads(json.dumps(images, default=str)) + + +class TestInspectorAnswersStrictJSON(ImageAPITestCase): + async def test_inspect_route_resolves_and_answers_strict_json(self): + write_png(self.output_dir / "nan.png") + response = await self.client.get( + "/prompt_manager/images/inspect/nan.png", params={"root": "0"} + ) + self.assertEqual(response.status, 200) + result = strict_loads(await response.text()) + self.assertEqual(result["resolution"]["state"], "resolved") + self.assertEqual( + result["prompts"]["positive"], + {"status": "resolved", "text": POSITIVE, "source": "graph-inferred"}, + ) + + async def test_prompt_images_route_answers_strict_json(self): + pid = self.api.db.save_prompt( + POSITIVE.strip(), prompt_hash=generate_prompt_hash(POSITIVE) + ) + path = str(write_png(self.output_dir / "linked.png")) + self.api.db.link_image_to_prompt(pid, path, {"prompt": graph()}) + with self.api.db.model.get_connection() as conn: + conn.execute( + "UPDATE generated_images SET prompt_metadata=?", (comfy_text(graph()),) + ) + conn.commit() + response = await self.client.get(f"/prompt_manager/prompts/{pid}/images") + self.assertEqual(response.status, 200) + result = strict_loads(await response.text()) + self.assertEqual(len(result["images"]), 1) + stored = result["images"][0]["prompt_metadata"] + self.assertEqual(stored["1"]["inputs"]["text"], POSITIVE) + self.assertEqual(stored["5"]["is_changed"], [None]) + + +class TestHistoryCapturesNonFiniteGraphs(unittest.TestCase): + """The queued graph ComfyUI hands the hook may carry float('nan').""" + + JOB = "a0c26da3-eb26-405c-b143-c662a84c3112" + + def setUp(self): + self.directory = tempfile.TemporaryDirectory() + self.addCleanup(self.directory.cleanup) + self.db = PromptDatabase(os.path.join(self.directory.name, "history.db")) + self.addCleanup(self.db.model.reset_schema_cache) + self.addCleanup(self.db.model.close_all) + + def test_queued_graph_with_nan_records_its_run_and_contexts(self): + from tests.test_execution_history_hooks import HostQueue, status + from utils.execution_context_capture import record_runtime_context + from utils.execution_history import register_history_hook + + queue = HostQueue() + registration = register_history_hook( + SimpleNamespace(prompt_queue=queue), lambda: self.db + ) + self.addCleanup(lambda: registration.dispose(3)) + self.addCleanup(registration.writer.invalidate) + deadline = time.monotonic() + 3 + while not registration.writer.status().accepting: + self.assertLess(time.monotonic(), deadline, "writer never accepted") + time.sleep(0.005) + g = { + "p": { + "class_type": "PromptManagerPair", + "inputs": {"text": POSITIVE, "negative_text": NEGATIVE}, + }, + "s": { + "class_type": "KSampler", + "inputs": {"positive": ["p", 0], "negative": ["p", 1], "cfg": INF}, + "is_changed": [NAN], + }, + "5": graph()["5"], + } + extra = {"extra_pnginfo": {"workflow": WORKFLOW}} + queue.results.append(((0, self.JOB, g, extra, ["s"]), 1)) + queue.get() + host = types.ModuleType("comfy_execution.utils") + host.get_executing_context = lambda: SimpleNamespace( + prompt_id=self.JOB, node_id="p", list_index=0 + ) + with patch.dict(sys.modules, {"comfy_execution.utils": host}): + for role, text in (("positive", POSITIVE), ("negative", NEGATIVE)): + record_runtime_context( + "PromptManagerPair", text, "", "", text, role=role + ) + queue.task_done(1, {}, status("execution_success", job=self.JOB)) + drain_writer(registration.writer, clock=time.monotonic) + with self.db.model.get_connection() as conn: + conn.row_factory = sqlite3.Row + runs = [dict(r) for r in conn.execute("SELECT * FROM execution_runs")] + self.assertEqual([run["outcome"] for run in runs], ["succeeded"]) + contexts = self.db.get_execution_contexts(runs[0]["run_id"]) + self.assertEqual(contexts["state"], "complete") + self.assertEqual(contexts["contexts"][0]["positive"]["raw"], POSITIVE) + strict_loads(json.dumps(contexts)) + + +if __name__ == "__main__": + unittest.main() diff --git a/utils/json_strict.py b/utils/json_strict.py new file mode 100644 index 00000000..7eeb8aed --- /dev/null +++ b/utils/json_strict.py @@ -0,0 +1,42 @@ +"""Strict JSON output for values that may carry ComfyUI's non-finite numbers. + +ComfyUI writes its metadata with Python's default ``json.dumps``, so a graph can +hold ``NaN``, ``Infinity`` and ``-Infinity`` (an ``IS_CHANGED`` that returns +``float("nan")`` puts ``"is_changed": [NaN]`` into every saved prompt). Those +literals are not JSON: browsers' ``JSON.parse`` and ``allow_nan=False`` reject +them. Reading follows the legacy pages (``parse_page_json`` where the browser +parser is mirrored; Python's ``json.loads`` elsewhere). Whatever PromptManager +stores or answers as JSON goes through ``dumps_strict``, which writes each +non-finite number as ``null`` (the legacy pages' own replacement for NaN) and +leaves strings, including the words ``NaN`` or ``Infinity`` inside prompt +text, untouched. +""" + +import json +from typing import Any + +__all__ = ["dumps_strict"] + + +def _null(_constant: str) -> None: + return None + + +def dumps_strict(value: Any, **kwargs: Any) -> str: + """``json.dumps(value, allow_nan=False, **kwargs)`` with non-finite numbers null. + + A value without non-finite numbers serializes byte for byte as the plain + call does. Any other error (a cycle, an unserializable object) still raises. + """ + kwargs.pop("allow_nan", None) + try: + return json.dumps(value, allow_nan=False, **kwargs) + except ValueError as error: + if "Out of range float" not in str(error): + raise + # The non-strict text spells each non-finite number as a bare literal, never + # inside a string, so parsing it back maps exactly those numbers to null. + lenient = json.dumps(value, allow_nan=True, default=kwargs.get("default")) + return json.dumps( + json.loads(lenient, parse_constant=_null), allow_nan=False, **kwargs + ) From 785b7445547de6c8e5225e1b29d4788e9ae57955 Mon Sep 17 00:00:00 2001 From: Vito Date: Fri, 23 Oct 2026 07:49:39 -0700 Subject: [PATCH 7/7] feat(system): Appearance section and browser display defaults (#307) * feat(system): share browser display defaults The workspace reads and writes the legacy gallery's gallerySettings object through preferences.mjs: workspaceLayout and workspaceDensity join the shared keys, a write re-reads the stored object and writes only the changed keys, denied storage or a full quota keeps the change for this tab and says so, and storage events report changes from other pages. The legacy gallery Save keeps keys it does not edit, and the viewer reads its video defaults through the shared module. * feat(system): Appearance defaults used by Images and the viewer System > Appearance is the concept's Browsing & display surface on real values: the browser rows edit this browser's display defaults through the shared gallerySettings object, and Open from ComfyUI and the node search auto-hide save to the server settings with their revision. A new Images visit starts from the default layout and size (the current view and saved views keep theirs); metadata off waits for Load metadata; cache off re-reads the shown file; paths off hides the Path row; the viewer reads the video defaults. D97 adds card details, collapsed viewer details and Reset display defaults; storage refused says Saved for this tab only. * fix(system): flag changes made while System was closed and keep embedded galleries fixed Final review fixes for System S3 (Appearance): - I1: the on-demand metadata deviation is D124 (TESTING.md, inspector markers); TESTING.md names the spec that pins it, since no concept comparison renders that state. - I2: System checks the display defaults for conflicts on mount, not only on storage events, and a browser write refuses a draft another page overtook, so a draft is never saved over another tab's value without the conflict status first. New two-tab spec. - I3: the layout and size defaults start the Images page only; embedded galleries (a prompt history's Shared gallery and dialogs) keep Grid/Comfortable. - M1: host rows whose /settings read failed say "Server settings could not be read." - M2: a rejected timeout shows the "Some changes were not saved" status. - M3: the browser loads on mount and on storage events never reject unhandled. - M4: the next accepted write stores the tab-only changes and clears the overlay. - M5: cache off never re-reads a source-changed selection, and refresh() never loads a deferred one. - M6: the System fixture refuses a config.json outside its temp root and restores it with os.replace. - M7: the inspection read counts wait on a request barrier, not frames. - M8 recorded as an open saved-views question in TESTING.md. * fix(system): answer a failed System read only after its other reads finish CI (py3.10 on ubuntu) crashed with a segfault inside SQLite in test_statistics_failure_is_an_error_envelope: the statistics and overview routes used asyncio.gather without return_exceptions, so the 500 answered on the first failure while get_statistics still ran in the executor, and the test tore the database down under it. Both routes now await every read before raising the first failure (failing before, passing after: TestFailedReadsSettleFirst). The frontend coverage merge failed on workspace/activity.mjs branches (86.3 < 87): no test asserted Activity's empty and failed-read states, so their branch was only reached in passing, depending on timing. A spec now asserts both. --- TESTING.md | 90 ++++- py/api/system.py | 17 +- tests/browser/system-concept.spec.mjs | 125 +++++++ tests/browser/workspace-activity.spec.mjs | 13 + tests/browser/workspace-system.spec.mjs | 367 ++++++++++++++++++- tests/browser/workspace.spec.mjs | 10 +- tests/browser/workspace_system_fixture.py | 30 +- tests/js/gallery-settings.test.js | 47 +++ tests/js/workspace-images-state.test.js | 16 + tests/js/workspace-inspection-state.test.js | 70 ++++ tests/js/workspace-preferences.test.js | 193 ++++++++++ tests/js/workspace-system-appearance.test.js | 188 ++++++++++ tests/js/workspace-system-state.test.js | 57 +++ tests/test_system_api.py | 57 +++ tests/test_workspace_system_fixture.py | 38 ++ tools/vendor/frontend-coverage-baseline.json | 30 +- web/js/gallery-settings.js | 12 +- web/js/gallery.js | 5 +- web/js/workspace/file-inspector.mjs | 20 +- web/js/workspace/images-state.mjs | 11 + web/js/workspace/images.mjs | 18 +- web/js/workspace/inspection-state.mjs | 22 +- web/js/workspace/media-viewer.mjs | 22 +- web/js/workspace/preferences.mjs | 112 ++++++ web/js/workspace/system-appearance.mjs | 143 ++++++++ web/js/workspace/system-state.mjs | 45 ++- web/js/workspace/system.mjs | 45 ++- web/js/workspace/workspace.css | 4 + 28 files changed, 1750 insertions(+), 57 deletions(-) create mode 100644 tests/js/workspace-preferences.test.js create mode 100644 tests/js/workspace-system-appearance.test.js create mode 100644 tests/test_workspace_system_fixture.py create mode 100644 web/js/workspace/preferences.mjs create mode 100644 web/js/workspace/system-appearance.mjs diff --git a/TESTING.md b/TESTING.md index 208e190c..16cf4cc5 100644 --- a/TESTING.md +++ b/TESTING.md @@ -1834,8 +1834,36 @@ browser fields, then the host fields (`POST /prompt_manager/settings` with highlights their controls; `settings-changed` is a conflict and keeps the draft. System reads the host values on mount and again after a conflict; until a read returned a revision, a host save fails without a request, so it never sends a blind -`expected_revision: null` (which S1 merges into the file on disk). No delivered section -edits a setting yet, so Save stays disabled until Appearance (S3). +`expected_revision: null` (which S1 merges into the file on disk). Save stays disabled +until a section has an unsaved change. + +Appearance (`system-appearance.mjs`, S3) is the concept's "Browsing & display" surface +on real values. Its browser rows are this browser's display defaults in the legacy +gallery's own `gallerySettings` object (`web/js/workspace/preferences.mjs`, Q3): Default +layout and Thumbnail size (`workspaceLayout` grid/list/masonry, `workspaceDensity` +comfortable/large/compact, new keys), Load metadata on selection (`autoLoadMetadata`), +Cache metadata in this browser (`cacheMetadata`), Image loading quality (`imageQuality`: +Optimized→`medium`, Original→`high`, Fast→`low`; the thumbnail dialog's preselected quality +once S5 lands), the three video rows (`videoAutoplay`, `videoMute`, `videoLoop`) and Show +file paths (`showFilePaths`). A write re-reads the stored object and writes back only the +changed keys, every other key (legacy paging, thumbnail state, unknown keys) as stored; +the legacy gallery's Save also keeps the keys it does not edit, so the two pages stay in +sync. The workspace never reads the legacy paging keys (`defaultLimit`, `gridColumns`, +`infiniteScroll`; README continuous-browsing ruling). A browser that refuses storage +(denied, or a full quota that leaves the stored bytes as they were) keeps the change for +this tab only and the status says "Saved for this tab only; browser storage is +unavailable.", never "Saved in this browser.". Another tab's change (a `storage` event) +repaints the clean controls; a draft it overtook stays, highlighted, with the conflict +status. Open from ComfyUI (New tab→`newtab`, Popup window→`popup`) and Node search +auto-hide (seconds) (a whole number 0–300; anything else is highlighted and never sent) +are the server's `webui_display_mode` and `result_timeout`, saved through +`POST /prompt_manager/settings` with the revision. The consumers read the defaults on +their next visit, never the current view: a new Images visit starts from the layout and +size (the last visit keeps its own, a saved view opens at Grid and Comfortable as +before); with metadata off a selection reads nothing until "Load metadata"; with the +cache off selecting the shown file again re-reads it (on, the one current record is +reused); paths off leaves out only the Path row of File & provenance; the viewer reads +the video defaults. Overview (`system-overview.mjs`) reads `GET /prompt_manager/system/overview` once per visit through `latestOnly()` on the section scope; a late answer after the section left @@ -1864,7 +1892,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged `-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no reader can keep it byte-identical (`test_health_check_never_writes`). -Recorded deviations (plan D60–D64 renumbered D92–D96; D121, D122 and D123 assigned): +Recorded deviations (plan D60–D65 renumbered D92–D97; D121, D122, D123 and D124 assigned): - **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server @@ -1922,6 +1950,36 @@ Recorded deviations (plan D60–D64 renumbered D92–D96; D121, D122 and D123 as the rest of Maintenance, above its D94 hand-off, as the native place that runs the health check. A press shows "Checking…" (`aria-disabled`) and opens the report; a failed check toasts "The health check could not finish. Nothing was changed." +- **D97** (Appearance): added rows for the legacy gallery settings the concept leaves + out, after its own rows: "Show image details on cards" (`showImageInfo`; off hides a + card's title and file line, Info and Tag stay at the end of the row), "Collapse viewer + details by default" (`sidebarCollapsedByDefault`; the viewer opens with + `#viewer-details` hidden and a "Show details"/"Hide details" toggle with + `aria-expanded`, the stage taking the whole width), and the operation "Reset display + defaults" ("Restore this browser’s display defaults, then save. Other settings are + kept.", button "Reset"), which drafts the gallery's defaults for the browser rows only. + The node search timeout shows the server's real value, not the sample 30. CSS under D97. +- **D124** (inspector): with "Load metadata on selection" off, the Images panel and + the viewer's details show "Metadata loads when you ask for it." and a "Load metadata" + button instead of reading the file. The concept has no such state, so no concept + comparison renders it: `metadata off shows an explicit load and loads once on request` + (workspace-system.spec) pins it instead. + +Appearance scope and states. The Default layout and Thumbnail size defaults start the +Images page only (a new visit, not the current view or a saved view); the embedded +galleries (a prompt history's Shared gallery and its All images / Repeat run dialogs) +have no toolbar, so they keep Grid/Comfortable, while the metadata, cache and card +defaults apply there too. A host row whose `/settings` read failed stays disabled with +"Server settings could not be read."; a rejected timeout says "Some changes were not +saved. Review the highlighted settings." A display default another page saved under one +of this tab's drafts flags that draft with the conflict status, whether it arrived while +System was mounted (a `storage` event) or while it was not (checked again on mount, and +before a write); the draft is never written over the other page's value without that +status first. A change kept for this tab only is stored by the next write storage accepts. + +Open question (saved-views follow-up, M8): a saved view stores no layout or size and +always opens at Grid/Comfortable, even when the Default layout is Masonry. Should saved +views store a layout, or follow the default? `tests/browser/system-concept.spec.mjs` compares the shell and Overview with the concept (dark and light, 1600×1100 and 390×844), mapping only the deviations above; unmapped, @@ -1934,20 +1992,34 @@ source when there are several, replaces only the chips with the fixture's expect results, and asserts the full unmapped path list; it maps D96 and compares the Maintenance notice unchanged. `system comparison detects mutations` removes a row, renames a metric label, swaps the `--line` token, recolours a chip, renames a check, -removes the totals table and renames a caption. Paired captures (`{concept,production}-system-{dark,light}-{1600,390}.png`, -`{concept,production}-{statistics,health}-{dark,light}.png`) are test artifacts; there +removes the totals table and renames a caption. `appearance matches the concept` maps +only D97 and the real timeout (unmapped, exactly those paths differ), and `appearance +comparison detects mutations` removes a row, renames a label and an option and swaps +the `--line` token. Paired captures (`{concept,production}-system-{dark,light}-{1600,390}.png`, +`{concept,production}-{statistics,health}-{dark,light}.png`, +`{concept,production}-appearance-{dark,light}-{1600,390}.png`) are test artifacts; there are no PNG baselines. `tests/browser/workspace-system.spec.mjs` covers the real counts, the health check and metric 4, a failed check, the thumbnail row against the status endpoint (painted after the rest when its read is held, unavailable when it fails), the review row (Q6), the section links, the Statistics dialog against the -route, a load failure with Retry and late answers after leaving. Its hooks are +route, a load failure with Retry and late answers after leaving, and Appearance: new +visits versus the current view and a saved view, the embedded gallery keeping Grid, +metadata off and on request, the cache (counted up to a request barrier), hidden paths, +the video defaults, the host fields with their revision, a stale one and a failed read, +reload and a second tab, a second tab's save while System was closed, storage refused, +the D97 rows and reset. Its hooks are `tests/browser/workspace_system_fixture.py` (`/__test__/system/*`: reset, injected failures, an absent source, a deleted linked file, unthumbnailed files) on the -concept-shaped fixture, with delays through the shared request holds. +concept-shaped fixture, with delays through the shared request holds; reset also restores +the server's host settings and its `config.json`, which must be the one inside the server's +temp root (`tests/test_workspace_system_fixture.py`). ```sh -python3 -m unittest tests.test_database_health tests.test_system_api tests.test_library -v +python3 -m unittest tests.test_database_health tests.test_system_api tests.test_library \ + tests.test_workspace_system_fixture -v node --test tests/js/workspace-system-state.test.js tests/js/workspace-system-controls.test.js \ - tests/js/workspace-system-overview.test.js tests/js/workspace-state.test.js + tests/js/workspace-system-overview.test.js tests/js/workspace-state.test.js \ + tests/js/workspace-system-appearance.test.js tests/js/workspace-preferences.test.js \ + tests/js/workspace-inspection-state.test.js tests/js/workspace-images-state.test.js npm run --prefix tools/vendor test:workflows -- system-concept.spec.mjs workspace-system.spec.mjs \ --project chromium --project touch --project firefox --project webkit ``` diff --git a/py/api/system.py b/py/api/system.py index 9539ae91..16a62d71 100644 --- a/py/api/system.py +++ b/py/api/system.py @@ -40,6 +40,19 @@ _HEALTH_STATUS_LINES = { _StoredReport = namedtuple("_StoredReport", "generation report") +async def _gather_settled(*reads): + """Await every read, then raise the first failure. + + Unlike a plain gather, a failure never answers while the other executor + reads still run against the database. + """ + results = await asyncio.gather(*reads, return_exceptions=True) + for result in results: + if isinstance(result, BaseException): + raise result + return results + + def _failure(message, status=500, code=None): return web.json_response( {"success": False, "code": code, "error": message}, status=status @@ -154,7 +167,7 @@ class SystemRoutesMixin: from ..config import GalleryConfig try: - (output_dirs, auto_detected), counts = await asyncio.gather( + (output_dirs, auto_detected), counts = await _gather_settled( self._run_in_executor(self._output_dirs_with_origin), self._run_in_executor(self.db.library_counts), ) @@ -188,7 +201,7 @@ class SystemRoutesMixin: async def get_system_statistics(self, request): try: - by_category, totals, counts = await asyncio.gather( + by_category, totals, counts = await _gather_settled( self._run_in_executor(self.db.usage_by_category), self._run_in_executor(self.db.get_statistics), self._run_in_executor(self.db.library_counts), diff --git a/tests/browser/system-concept.spec.mjs b/tests/browser/system-concept.spec.mjs index c32a62e6..761134b9 100644 --- a/tests/browser/system-concept.spec.mjs +++ b/tests/browser/system-concept.spec.mjs @@ -408,3 +408,128 @@ for (const theme of THEMES) test(`statistics and health dialogs match the concep await page.evaluate(() => { document.querySelector('#modal-body caption').textContent = 'Runs per category'; }); expect(diffInventory(expectedStatistics, await page.evaluate(measureDialog)).map(item => item.path)).toEqual(['tables.0.caption']); }); + +/** Runs in the page: Appearance's surface, every row's text and control, and the concept's row geometry and tokens. */ +function measureAppearance(mainSelector) { + const layout = document.querySelector(`${mainSelector} .settings-layout`); + const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null; + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const text = node => node?.textContent.trim() ?? null; + const surface = layout.querySelector(':scope > section.surface'); + const control = row => { + const select = row.querySelector(':scope > select'); + if (select) return { kind: 'select', id: select.id, options: [...select.options].map(option => option.textContent), + selected: select.selectedOptions[0]?.textContent ?? null }; + const input = row.querySelector(':scope > input'); + if (input) return { kind: `input.${input.type}`, id: input.id, value: input.value }; + const toggle = row.querySelector(':scope > * input[type="checkbox"]'); + if (toggle) return { kind: 'switch', id: toggle.id, wrapper: name(toggle.parentElement), checked: toggle.checked, + ...css(toggle.parentElement, 'width', 'height') }; + const chip = row.querySelector(':scope > .chip'); + return chip ? { kind: name(chip), text: text(chip), ...css(chip, 'color', 'backgroundColor', 'fontSize') } : null; + }; + const rows = [...surface.querySelectorAll(':scope > .setting-row')]; + const operations = [...surface.querySelectorAll(':scope > .operation')]; + const firstRow = rows[0]; + return { + layout: [...layout.children].map(name), + surface: { tree: [...surface.children].map(name), heading: text(surface.querySelector(':scope > h2')), + intro: text(surface.querySelector(':scope > p')), + ...css(surface, 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius') }, + rows: rows.map(row => ({ tree: [...row.children].map(name), text: [...row.firstElementChild.children].map(node => [node.localName, text(node), + node.localName === 'label' ? node.htmlFor : null]), control: control(row) })), + operations: operations.map(row => ({ tree: [...row.children].map(name), title: text(row.querySelector('h3')), + description: text(row.querySelector('p')), action: text(row.querySelector('.row-actions > button')) })), + row: css(firstRow, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', 'borderTopWidth', + 'borderTopColor', 'marginTop'), + label: css(firstRow.querySelector('label'), 'fontSize', 'fontWeight', 'color'), + description: css(firstRow.querySelector('p'), 'fontSize', 'color', 'marginTop', 'maxWidth'), + select: css(firstRow.querySelector('select'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor', 'color', 'borderTopColor'), + heading3: css(surface.querySelector('.setting-row h3'), 'fontSize', 'fontWeight', 'color'), + number: css(surface.querySelector('.setting-row input[type="number"]'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor'), + pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth, + }; +} + +// D97: the rows Appearance adds for the legacy gallery settings, after the concept's own, as production renders them. +const ADDED_APPEARANCE_ROWS = [ + ['card-details', 'Show image details on cards', 'Show each card’s title and file details under the preview.', true], + ['viewer-details', 'Collapse viewer details by default', 'Open the image viewer with its details panel hidden.', false], +]; + +/** The concept's Appearance with the recorded deviations applied, given the server's real settings. */ +function mapAppearanceConcept(reference, settings) { + const toggleRow = reference.rows.find(row => row.control?.kind === 'switch'); + const added = ADDED_APPEARANCE_ROWS.map(([id, label, description, checked]) => ({ + tree: toggleRow.tree, text: [['label', label, id], ['p', description, null]], + control: { ...toggleRow.control, id, checked } })); + return { + ...reference, + // D97: two added toggles and the "Reset display defaults" operation close the surface. + surface: { ...reference.surface, tree: [...reference.surface.tree, 'div.setting-row', 'div.setting-row', 'div.operation'] }, + rows: [...reference.rows.map(row => row.control?.id === 'timeout' + // The node search timeout is the server's real `result_timeout`, not the concept's sample 30. + ? { ...row, control: { ...row.control, value: String(settings.result_timeout) } } : row), ...added], + operations: [{ tree: ['div', 'div.row-actions'], title: 'Reset display defaults', + description: 'Restore this browser’s display defaults, then save. Other settings are kept.', action: 'Reset' }], + }; +} + +async function conceptAppearance(page, theme) { + await openConcept(page, 'system', theme); + await page.locator('#main .section-nav').getByRole('button', { name: 'Appearance', exact: true }).click(); + await expect(page.locator('#main .settings-layout > section.surface > h2')).toHaveText('Browsing & display'); + await page.mouse.move(0, 0); + return page.evaluate(measureAppearance, '#main'); +} + +async function productionAppearance(page, theme) { + await openProduction(page, 'system/appearance', theme); + await expect(page.locator('#workspace-main #timeout')).toBeEnabled(); + await page.mouse.move(0, 0); + return page.evaluate(measureAppearance, '#workspace-main'); +} + +for (const theme of THEMES) for (const [width, height] of SIZES) { + test(`appearance matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => { + await page.setViewportSize({ width, height }); + const { settings } = await serverJson(request, '/prompt_manager/settings'); + const concept = await conceptAppearance(page, theme); + await page.screenshot({ path: testInfo.outputPath(`concept-appearance-${theme}-${width}.png`), fullPage: true }); + const production = await productionAppearance(page, theme); + await page.screenshot({ path: testInfo.outputPath(`production-appearance-${theme}-${width}.png`), fullPage: true }); + expect(diffInventory(mapAppearanceConcept(concept, settings), production)).toEqual([]); + expect(production.pageScroll).toBe(false); + }); +} + +test('appearance comparison detects mutations', async ({ page, request }) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const { settings } = await serverJson(request, '/prompt_manager/settings'); + const concept = await conceptAppearance(page, 'dark'); + const expected = mapAppearanceConcept(concept, settings); + const production = await productionAppearance(page, 'dark'); + expect(diffInventory(expected, production)).toEqual([]); + // Unmapped, the concept differs only where D97 and the real timeout are recorded. + expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 2).join('.')) + .filter((path, index, all) => all.indexOf(path) === index).sort()) + .toEqual(['operations.0', 'operations.length', 'rows.11', 'rows.12', 'rows.13', 'rows.length', 'surface.tree'].sort()); + const measure = () => page.evaluate(measureAppearance, '#workspace-main'); + const paths = async () => diffInventory(expected, await measure()).map(item => item.path); + // A removed row. + const undoRow = await mutate(page, { remove: ['#workspace-main .setting-row:nth-of-type(3)'] }); + expect(await paths()).toEqual(expect.arrayContaining(['rows.length', 'surface.tree.length'])); + await undoRow(); + // A changed label and a changed option. + await page.evaluate(() => { document.querySelector('#workspace-main label[for="density"]').textContent = 'Thumbnail sizes'; }); + expect(await paths()).toEqual(['rows.1.text.0.1']); + await page.evaluate(() => { document.querySelector('#workspace-main label[for="density"]').textContent = 'Thumbnail size'; }); + await page.evaluate(() => { document.querySelector('#workspace-main #image-quality option').textContent = 'Medium'; }); + expect(await paths()).toEqual(expect.arrayContaining(['rows.5.control.options.0'])); + await page.evaluate(() => { document.querySelector('#workspace-main #image-quality option').textContent = 'Optimized'; }); + // A swapped token. + const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' }); + expect(await paths()).toEqual(expect.arrayContaining(['row.borderTopColor'])); + await undoToken(); + expect(diffInventory(expected, await measure())).toEqual([]); +}); diff --git a/tests/browser/workspace-activity.spec.mjs b/tests/browser/workspace-activity.spec.mjs index a3a16165..72e95f28 100644 --- a/tests/browser/workspace-activity.spec.mjs +++ b/tests/browser/workspace-activity.spec.mjs @@ -284,6 +284,19 @@ test('reload during a scan reattaches to the running job', async ({ page, reques await expect(pill(page)).toBeHidden(); }); +test('an empty Activity says no job has run, and a list that cannot be read says so', async ({ page }) => { + // The empty and failed states were only reached in passing by other tests, so their coverage depended on timing. + const empty = main(page).locator('section.surface > p.empty-state'); + await page.goto(`${workspace}#/activity`); + await expect(empty).toHaveText('No jobs have run since ComfyUI started. Scans and other long operations appear here.'); + await expect(empty).not.toHaveAttribute('role'); + await expect(items(page)).toHaveCount(0); + await page.route(url => url.pathname === '/prompt_manager/jobs', route => route.abort()); + await page.reload(); + await expect(empty).toHaveText('Activity could not be read. It is tried again shortly.'); + await expect(empty).toHaveAttribute('role', 'alert'); +}); + test('lost server clears the pill and marks jobs ended', async ({ page, request }) => { // A poll interval after the restart, then failed re-reads backing off for 1 + 2 + 4 s. test.setTimeout(90_000); diff --git a/tests/browser/workspace-system.spec.mjs b/tests/browser/workspace-system.spec.mjs index 75d2069b..ca2da6e7 100644 --- a/tests/browser/workspace-system.spec.mjs +++ b/tests/browser/workspace-system.spec.mjs @@ -11,7 +11,7 @@ const number = value => value.toLocaleString('en-US'); test.beforeEach(async ({ page, request, browserName }) => { await startCoverage(page, browserName); - await seedConcept(request); + page.manifest = await seedConcept(request); await expect(await request.post('/__test__/system/reset')).toBeOK(); await expect(await request.post('/__test__/jobs/restart')).toBeOK(); page.systemErrors = []; @@ -250,3 +250,368 @@ test('out-of-order overview responses never repaint a newer section', async ({ p await expect(layout.locator('.summary-metrics')).toHaveCount(1); await expect(layout.locator('.two-col')).toHaveCount(1); }); + +// System › Appearance (#249 System S3 Task 8): the browser display defaults (`gallerySettings`, preferences.mjs) and the +// two host fields behind the one Save row, and the consumers that read them on their next visit. +const SETTINGS_URL = '/prompt_manager/settings'; +const INSPECT_PATH = '/prompt_manager/images/inspect/'; +const statusOf = page => page.locator('#workspace-main #settings-status'); +const saveOf = page => page.locator('#workspace-main .save-row > button'); +const storedSettings = page => page.evaluate(() => JSON.parse(localStorage.getItem('gallerySettings') || 'null')); +const imageCards = page => page.locator('#workspace-main article.media-card'); +const panel = page => page.getByRole('complementary', { name: 'Selected image details', exact: true }); + +async function openAppearance(page) { + await page.goto(`${workspace}#/system/appearance`); + await expect(page.locator('#workspace-main .settings-layout > section.surface > h2')).toHaveText('Browsing & display'); + // The host rows are enabled once the server settings answered. + await expect(page.locator('#timeout')).toBeEnabled(); +} + +async function saveAppearance(page, text) { + await expect(saveOf(page)).not.toHaveAttribute('aria-disabled', 'true'); + await saveOf(page).click(); + await expect(statusOf(page)).toHaveText(text); +} + +/** Set toggles (by row id) through their Toolkit switches. */ +async function toggle(page, values) { + for (const [id, checked] of Object.entries(values)) await page.locator(`#${id}`).setChecked(checked, { force: true }); +} + +/** + * A request barrier: one request issued and answered after everything the page has asked for so far, so a read the + * last action started has been seen (Playwright reports a page's requests in order). + */ +async function requestBarrier(page) { + expect(await page.evaluate(url => fetch(url, { cache: 'no-store' }).then(response => response.ok), SETTINGS_URL)).toBe(true); +} + +async function openImages(page) { + await page.goto(`${workspace}#/images`); + await expect(imageCards(page).first()).toBeVisible(); +} + +test('defaults apply to a new visit but not to the current view or a saved view', async ({ page, request }) => { + const created = await request.post('/prompt_manager/saved-views', { data: { page: 'images', name: 'Everything', + state: { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false } } }); + expect(created.status()).toBe(201); + const view = (await created.json()).view; + try { + await openImages(page); + // The current view: the toolbar sets List. + await page.getByRole('group', { name: 'Image layout' }).getByRole('button', { name: 'List', exact: true }).click(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); + await openAppearance(page); + await expect(page.locator('#default-view')).toHaveValue('grid'); + await page.locator('#default-view').selectOption('masonry'); + await page.locator('#density').selectOption('compact'); + await expect(statusOf(page)).toHaveText('Unsaved changes.'); + await saveAppearance(page, 'Saved in this browser.'); + const stored = await storedSettings(page); + expect([stored.workspaceLayout, stored.workspaceDensity]).toEqual(['masonry', 'compact']); + // Back in Images (the page session's last visit): the current view keeps its own layout and size. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'comfortable'); + // The saved view opens as before. + await page.goto(`${workspace}#/saved/${view.id}`); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'grid'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'comfortable'); + // A new visit starts from the defaults. + await openImages(page); + await page.reload(); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'masonry'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'compact'); + await expect(page.getByRole('group', { name: 'Image layout' }).getByRole('button', { name: 'Masonry', exact: true })) + .toHaveAttribute('aria-pressed', 'true'); + } finally { + await request.delete(`/prompt_manager/saved-views/${view.id}`); + } +}); + +test('the defaults reach the Images page only; embedded galleries keep their own presentation', async ({ page }) => { + await openAppearance(page); + await page.locator('#default-view').selectOption('list'); + await page.locator('#density').selectOption('compact'); + await saveAppearance(page, 'Saved in this browser.'); + // The prompt history's shared gallery has no toolbar to change them, so it opens as it always did. + await page.goto(`${workspace}#/library/${page.manifest.prompts[0].positive_id}`); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('#workspace-main .image-grid')).toHaveAttribute('data-layout', 'grid'); + await expect(page.locator('#workspace-main .image-grid')).toHaveAttribute('data-density', 'comfortable'); + // The Images page starts from them. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'compact'); +}); + +test('metadata off shows an explicit load and loads once on request', async ({ page }) => { + await openAppearance(page); + await toggle(page, { 'metadata-auto': false }); + await saveAppearance(page, 'Saved in this browser.'); + const reads = []; + page.on('request', item => { if (new URL(item.url()).pathname.startsWith(INSPECT_PATH)) reads.push(item.url()); }); + await openImages(page); + const first = imageCards(page).first(); + await first.locator('.image-card-actions button').first().click(); + await expect(panel(page).getByRole('status')).toHaveText('Metadata loads when you ask for it.'); + const load = panel(page).getByRole('button', { name: 'Load metadata', exact: true }); + await expect(load).toBeVisible(); + expect(reads).toEqual([]); + await load.click(); + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + expect(reads).toHaveLength(1); + // The same file again reads nothing more (the cache is on). + await first.locator('.image-card-actions button').first().click(); + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + await requestBarrier(page); + expect(reads).toHaveLength(1); + // The viewer's details use the same inspector: another file waits there too, and loads on request. + await imageCards(page).nth(1).locator('.image-open').click(); + const viewerDetails = page.locator('#viewer-details'); + await expect(viewerDetails.getByRole('status')).toHaveText('Metadata loads when you ask for it.'); + expect(reads).toHaveLength(1); + await viewerDetails.getByRole('button', { name: 'Load metadata', exact: true }).click(); + await expect(viewerDetails.locator('.panel-heading h2')).toBeVisible(); + expect(reads).toHaveLength(2); +}); + +test('cache off rereads the current selection; on reuses the one current record', async ({ page }) => { + const reads = []; + page.on('request', item => { if (new URL(item.url()).pathname.startsWith(INSPECT_PATH)) reads.push(item.url()); }); + for (const cache of [true, false]) { + if (!cache) { + await openAppearance(page); + await toggle(page, { 'metadata-cache': false }); + await saveAppearance(page, 'Saved in this browser.'); + await page.reload(); + } + await openImages(page); + const info = imageCards(page).first().locator('.image-card-actions button').first(); + await info.click(); + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + await expect.poll(() => reads.length).toBeGreaterThan(0); + await requestBarrier(page); + reads.length = 0; + // Inspect the selected file twice more. + for (const _ of [1, 2]) { + const answered = cache ? null : page.waitForResponse(response => new URL(response.url()).pathname.startsWith(INSPECT_PATH)); + await info.click(); + if (answered) await answered; + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + } + await requestBarrier(page); + expect(reads, `cache ${cache}`).toHaveLength(cache ? 0 : 2); + } +}); + +test('paths hidden only changes display', async ({ page }) => { + const rows = async () => page.locator('#viewer-details details').filter({ hasText: 'File & provenance' }).locator('dt') + .evaluateAll(terms => terms.map(term => term.textContent)); + const openViewer = async () => { + await openImages(page); + await imageCards(page).first().locator('.image-open').click(); + const provenance = page.locator('#viewer-details details').filter({ hasText: 'File & provenance' }); + await provenance.locator('summary').click(); + await expect(provenance.locator('dl')).toBeVisible(); + return provenance; + }; + let provenance = await openViewer(); + expect(await rows()).toContain('Path'); + const before = { show: await provenance.getByRole('button', { name: /^Show .* in folder$/ }).count(), + download: await provenance.getByRole('link', { name: 'Download original', exact: true }).getAttribute('href') }; + await page.keyboard.press('Escape'); + await openAppearance(page); + await toggle(page, { paths: false }); + await saveAppearance(page, 'Saved in this browser.'); + provenance = await openViewer(); + const shown = await rows(); + expect(shown).not.toContain('Path'); + expect(shown).toEqual(expect.arrayContaining(['Source', 'Size'])); + // Everything else is as it was: the folder action and the original download stay. + expect(await provenance.getByRole('button', { name: /^Show .* in folder$/ }).count()).toBe(before.show); + await expect(provenance.getByRole('link', { name: 'Download original', exact: true })).toHaveAttribute('href', before.download); +}); + +test('video defaults reach the viewer', async ({ page }) => { + await openAppearance(page); + await toggle(page, { 'video-auto': true, 'video-muted': false, 'video-loop': false }); + await saveAppearance(page, 'Saved in this browser.'); + expect(await page.evaluate(async () => (await import('/prompt_manager/js/workspace/media-viewer.mjs')).readViewerPreferences())) + .toEqual({ videoAutoplay: true, videoMute: false, videoLoop: false }); + const stored = await storedSettings(page); + expect([stored.videoAutoplay, stored.videoMute, stored.videoLoop]).toEqual([true, false, false]); +}); + +test('host fields round-trip through /settings with revision', async ({ page, request }) => { + const posts = []; + page.on('request', item => { if (item.method() === 'POST' && new URL(item.url()).pathname === SETTINGS_URL) posts.push(item.postDataJSON()); }); + const before = await serverJson(request, SETTINGS_URL); + await openAppearance(page); + await expect(page.locator('#open-mode')).toHaveValue(before.settings.webui_display_mode); + await expect(page.locator('#timeout')).toHaveValue(String(before.settings.result_timeout)); + // Only whole seconds from 0 to 300: anything else is highlighted and never sent. + for (const bad of ['301', '-1', '2.5']) { + await page.locator('#timeout').fill(bad); + await page.locator('#timeout').press('Tab'); + await expect(page.locator('#timeout')).toHaveAttribute('aria-invalid', 'true'); + await expect(statusOf(page)).toHaveText('Some changes were not saved. Review the highlighted settings.'); + await expect(saveOf(page)).toHaveAttribute('aria-disabled', 'true'); + } + await page.locator('#timeout').fill('0'); + await page.locator('#timeout').press('Tab'); + await expect(page.locator('#timeout')).not.toHaveAttribute('aria-invalid'); + await page.locator('#open-mode').selectOption('popup'); + await saveAppearance(page, 'Saved to the server configuration.'); + expect(posts).toEqual([{ webui_display_mode: 'popup', result_timeout: 0, expected_revision: before.revision }]); + const after = await serverJson(request, SETTINGS_URL); + expect([after.settings.webui_display_mode, after.settings.result_timeout]).toEqual(['popup', 0]); + expect(after.revision).not.toBe(before.revision); + // No browser key was written for a host field. + expect(await storedSettings(page)).toBeNull(); +}); + +test('server settings that cannot be read say so in their rows', async ({ page }) => { + await page.route(url => url.pathname === SETTINGS_URL, route => (route.request().method() === 'GET' + ? route.fulfill({ status: 500, contentType: 'application/json', body: JSON.stringify({ success: false, error: 'Injected failure' }) }) + : route.continue())); + await page.goto(`${workspace}#/system/appearance`); + for (const id of ['open-mode', 'timeout']) { + const row = page.locator('#workspace-main .setting-row').filter({ has: page.locator(`#${id}`) }); + await expect(row.locator('p.small')).toHaveText('Server settings could not be read.'); + await expect(page.locator(`#${id}`)).toBeDisabled(); + } + // The browser rows are this browser's, and work as before. + await expect(page.locator('#default-view')).toBeEnabled(); + await expect(page.locator('#workspace-main .setting-row p.small')).toHaveCount(2); +}); + +test('stale revision shows the conflict status', async ({ page, request }) => { + await openAppearance(page); + // Another page saves after this one read the settings. + const current = await serverJson(request, SETTINGS_URL); + await expect(await request.post(SETTINGS_URL, { data: { result_timeout: 42, expected_revision: current.revision } })).toBeOK(); + await page.locator('#open-mode').selectOption('popup'); + await saveAppearance(page, 'Settings changed elsewhere. Reload this section to see them.'); + await expect(page.locator('#open-mode')).toHaveAttribute('aria-invalid', 'true'); + // The draft is kept and the section now shows the other page's value. + await expect(page.locator('#open-mode')).toHaveValue('popup'); + await expect(page.locator('#timeout')).toHaveValue('42'); + expect((await serverJson(request, SETTINGS_URL)).settings.webui_display_mode).toBe('newtab'); + // With the reloaded revision, Save succeeds. + await saveAppearance(page, 'Saved to the server configuration.'); + expect((await serverJson(request, SETTINGS_URL)).settings.webui_display_mode).toBe('popup'); +}); + +test('reload keeps saved values; a new tab reads them', async ({ page, context }) => { + await openAppearance(page); + await page.locator('#image-quality').selectOption('high'); + await toggle(page, { 'card-details': false }); + await page.locator('#open-mode').selectOption('popup'); + await saveAppearance(page, 'Saved in this browser. Saved to the server configuration.'); + await page.reload(); + await expect(page.locator('#timeout')).toBeEnabled(); + await expect(page.locator('#image-quality')).toHaveValue('high'); + await expect(page.locator('#card-details')).not.toBeChecked(); + await expect(page.locator('#open-mode')).toHaveValue('popup'); + const other = await context.newPage(); + await openAppearance(other); + await expect(other.locator('#image-quality')).toHaveValue('high'); + await expect(other.locator('#card-details')).not.toBeChecked(); + // A change saved in one tab reaches the other's clean controls; a draft it overtakes is flagged and kept. + await other.locator('#density').selectOption('large'); + await page.locator('#density').selectOption('compact'); + await saveAppearance(page, 'Saved in this browser.'); + await expect(other.locator('#density')).toHaveValue('large'); + await expect(other.locator('#settings-status')).toHaveText('Settings changed elsewhere. Reload this section to see them.'); + await expect(other.locator('#density')).toHaveAttribute('aria-invalid', 'true'); + await page.locator('#image-quality').selectOption('low'); + await saveAppearance(page, 'Saved in this browser.'); + await expect(other.locator('#image-quality')).toHaveValue('low'); + await other.close(); +}); + +test('a change another tab saved while System was closed is a conflict on return, never saved over silently', async ({ page, context }) => { + await openAppearance(page); + await page.locator('#density').selectOption('compact'); + await expect(statusOf(page)).toHaveText('Unsaved changes.'); + // Leave System for Images: the draft stays for the page session, but System no longer listens. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(imageCards(page).first()).toBeVisible(); + const other = await context.newPage(); + await openAppearance(other); + await other.locator('#density').selectOption('large'); + await saveAppearance(other, 'Saved in this browser.'); + await other.close(); + // Back in System: the draft is kept and flagged, and nothing was written over the other tab's value. + await page.evaluate(() => { location.hash = '#/system/appearance'; }); + await expect(page.locator('#workspace-main .settings-layout > section.surface > h2')).toHaveText('Browsing & display'); + await expect(statusOf(page)).toHaveText('Settings changed elsewhere. Reload this section to see them.'); + await expect(page.locator('#density')).toHaveAttribute('aria-invalid', 'true'); + await expect(page.locator('#density')).toHaveValue('compact'); + expect((await storedSettings(page)).workspaceDensity).toBe('large'); + // Saving after the conflict was shown is an explicit choice: it wins. + await saveAppearance(page, 'Saved in this browser.'); + expect((await storedSettings(page)).workspaceDensity).toBe('compact'); + await expect(page.locator('#density')).not.toHaveAttribute('aria-invalid'); +}); + +test('a browser that refuses storage is saved for this tab only, never "Saved"', async ({ page }) => { + await openAppearance(page); + await page.evaluate(() => { + Storage.prototype.setItem = function setItem() { throw new DOMException('Quota exceeded', 'QuotaExceededError'); }; + }); + await page.locator('#default-view').selectOption('list'); + await saveAppearance(page, 'Saved for this tab only; browser storage is unavailable.'); + expect(await storedSettings(page)).toBeNull(); + // This tab's next Images visit still uses it. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); +}); + +test('added rows reach their consumers', async ({ page }) => { + await openImages(page); + await expect(imageCards(page).first().locator('.media-info h3')).toBeVisible(); + await openAppearance(page); + await toggle(page, { 'card-details': false, 'viewer-details': true }); + await saveAppearance(page, 'Saved in this browser.'); + await page.reload(); + await openImages(page); + // Show image details on cards off: no title or file line; Info and Tag stay. + const card = imageCards(page).first(); + await expect(card.locator('.media-info h3')).toBeHidden(); + await expect(card.locator('.image-card-actions button')).toHaveCount(2); + // Collapse viewer details by default: the viewer opens with its details hidden behind one toggle. + await card.locator('.image-open').click(); + const details = page.locator('#viewer-details'); + await expect(page.locator('#viewer-dialog')).toBeVisible(); + await expect(details).toBeHidden(); + const show = page.locator('#viewer-toolbar').getByRole('button', { name: 'Show details', exact: true }); + await expect(show).toHaveAttribute('aria-expanded', 'false'); + await show.click(); + await expect(details).toBeVisible(); + await expect(page.locator('#viewer-toolbar').getByRole('button', { name: 'Hide details', exact: true })).toHaveAttribute('aria-expanded', 'true'); +}); + +test('reset restores defaults without touching other keys', async ({ page }) => { + await page.goto(`${workspace}#/system/appearance`); + await page.evaluate(() => localStorage.setItem('gallerySettings', JSON.stringify({ gridColumns: 5, defaultLimit: 200, + thumbnailPromptDismissed: 'kept', workspaceLayout: 'list', videoMute: false, showImageInfo: false }))); + await page.reload(); + await expect(page.locator('#timeout')).toBeEnabled(); + await expect(page.locator('#default-view')).toHaveValue('list'); + const reset = page.locator('#workspace-main .operation').filter({ hasText: 'Reset display defaults' }).getByRole('button', { name: 'Reset', exact: true }); + await reset.click(); + await expect(page.locator('#default-view')).toHaveValue('grid'); + await expect(page.locator('#video-muted')).toBeChecked(); + await expect(statusOf(page)).toHaveText('Unsaved changes.'); + await saveAppearance(page, 'Saved in this browser.'); + const stored = await storedSettings(page); + expect(stored).toEqual({ gridColumns: 5, defaultLimit: 200, thumbnailPromptDismissed: 'kept', workspaceLayout: 'grid', + videoMute: true, showImageInfo: true }); +}); diff --git a/tests/browser/workspace.spec.mjs b/tests/browser/workspace.spec.mjs index f065b4c5..8d847a33 100644 --- a/tests/browser/workspace.spec.mjs +++ b/tests/browser/workspace.spec.mjs @@ -425,7 +425,7 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept const SYSTEM_SECTION_LABELS = ['Overview', 'Appearance', 'Sources & scans', 'Integrations', 'Auto-tagging', 'Maintenance', 'Backup & export', 'Diagnostics & logs']; const SYSTEM_HASHES = ['overview', 'appearance', 'sources', 'integrations', 'tagging', 'maintenance', 'backups', 'diagnostics']; -const SYSTEM_HANDOFFS = { appearance: ['/prompt_manager/gallery', '/prompt_manager/admin'], sources: ['/prompt_manager/admin', '/prompt_manager/gallery'], +const SYSTEM_HANDOFFS = { sources: ['/prompt_manager/admin', '/prompt_manager/gallery'], maintenance: ['/prompt_manager/admin', '/prompt_manager/gallery'] }; test('System is native: the concept shell, section navigation, sections, hand-offs and the Save row', async ({ page }) => { @@ -462,14 +462,14 @@ test('System is native: the concept shell, section navigation, sections, hand-of await expect(nav.locator('[aria-pressed="true"]')).toHaveCount(1); // The section changes in place: focus stays on the pressed section, not the page title. await expect(button).toBeFocused(); - // S2 Task 6: Overview is native (workspace-system.spec.mjs); Maintenance shows its health-check notice above - // the hand-off (D123); every other section hands off. + // S2 Task 6: Overview is native (workspace-system.spec.mjs), and so is Appearance (S3 Task 8); Maintenance shows + // its health-check notice above the hand-off (D123); every other section hands off. if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible(); expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`))) - .toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], + .toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'div.save-row'], maintenance: ['div.notice', 'section.surface handoff', 'div.save-row'] }[hash] ?? ['section.surface handoff', 'div.save-row']); - if (hash === 'overview') continue; + if (hash === 'overview' || hash === 'appearance') continue; const handoff = layout.locator('section.handoff'); await expect(handoff.locator('h2')).toHaveText('Your current tools are ready'); expect(await handoff.locator('a.workspace-link').evaluateAll(links => links.map(link => link.getAttribute('href')))) diff --git a/tests/browser/workspace_system_fixture.py b/tests/browser/workspace_system_fixture.py index 6027c64d..2413c1f0 100644 --- a/tests/browser/workspace_system_fixture.py +++ b/tests/browser/workspace_system_fixture.py @@ -31,9 +31,27 @@ MAX_UNTHUMBNAILED = 50 def register_system_commands(app, routes, api, root: Path): - from py.config import GalleryConfig + # System › Appearance saves the host fields to this server's config.json (S3); + # reset restores both. The file is this server's own, inside its temp root: + # reset rewrites or deletes it, so a server whose configuration points + # anywhere else (a real config.json) refuses to register these hooks. + config_path = root / "config.json" + configured_path = os.environ.get("PROMPT_MANAGER_CONFIG_PATH") + if ( + configured_path is None + or Path(configured_path).resolve() != config_path.resolve() + ): + raise RuntimeError( + f"PROMPT_MANAGER_CONFIG_PATH must be {config_path}, not {configured_path!r}" + ) + from py.config import GalleryConfig, PromptManagerConfig configured = list(GalleryConfig.MONITORING_DIRECTORIES) + config_bytes = config_path.read_bytes() if config_path.exists() else None + host_values = ( + PromptManagerConfig.RESULT_TIMEOUT, + PromptManagerConfig.WEBUI_DISPLAY_MODE, + ) failures: dict[str, int] = {} @web.middleware @@ -52,6 +70,16 @@ def register_system_commands(app, routes, api, root: Path): failures.clear() GalleryConfig.MONITORING_DIRECTORIES = list(configured) api._health_report = None + PromptManagerConfig.RESULT_TIMEOUT, PromptManagerConfig.WEBUI_DISPLAY_MODE = ( + host_values + ) + if config_bytes is None: + config_path.unlink(missing_ok=True) + else: + # Replace in one step, as the server writes it (Windows preflight). + staged = config_path.with_name(config_path.name + ".reset") + staged.write_bytes(config_bytes) + os.replace(staged, config_path) @routes.post("/__test__/system/reset") async def system_reset(request): diff --git a/tests/js/gallery-settings.test.js b/tests/js/gallery-settings.test.js index c3681a29..a8fbf658 100644 --- a/tests/js/gallery-settings.test.js +++ b/tests/js/gallery-settings.test.js @@ -61,3 +61,50 @@ test("infinite scrolling is off by default and must be a boolean", () => { assert.equal(normalizeGallerySettings({ infiniteScroll: true }).infiniteScroll, true); assert.equal(normalizeGallerySettings({ infiniteScroll: "yes" }).infiniteScroll, false); }); + +test("the workspace keys have their own defaults and validation", () => { + const { WORKSPACE_LAYOUTS, WORKSPACE_DENSITIES } = require("../../web/js/gallery-settings.js"); + assert.equal(DEFAULT_SETTINGS.workspaceLayout, "grid"); + assert.equal(DEFAULT_SETTINGS.workspaceDensity, "comfortable"); + assert.deepEqual(WORKSPACE_LAYOUTS, ["grid", "list", "masonry"]); + assert.deepEqual(WORKSPACE_DENSITIES, ["comfortable", "large", "compact"]); + assert.equal(normalizeGallerySettings({ workspaceLayout: "masonry" }).workspaceLayout, "masonry"); + assert.equal(normalizeGallerySettings({ workspaceLayout: "mosaic" }).workspaceLayout, "grid"); + assert.equal(normalizeGallerySettings({ workspaceDensity: "large" }).workspaceDensity, "large"); + assert.equal(normalizeGallerySettings({ workspaceDensity: 3 }).workspaceDensity, "comfortable"); + // The legacy gallery's own view mode is separate: it has no masonry. + assert.equal(normalizeGallerySettings({ defaultViewMode: "masonry" }).defaultViewMode, "grid"); +}); + +test("a legacy gallery Save keeps the workspace keys and every other stored key", () => { + const fs = require("node:fs"); + const path = require("node:path"); + const vm = require("node:vm"); + const settings = require("../../web/js/gallery-settings.js"); + const storage = new Map([["gallerySettings", JSON.stringify({ workspaceLayout: "masonry", workspaceDensity: "compact", someOtherTool: 1 })]]); + const controls = { + lazyLoadingToggle: { checked: true }, imageQualitySelect: { value: "high" }, defaultViewModeSelect: { value: "list" }, + defaultLimitSelect: { value: "200" }, gridColumnsSelect: { value: "6" }, showImageInfoToggle: { checked: false }, + autoLoadMetadataToggle: { checked: true }, cacheMetadataToggle: { checked: true }, showFilePathsToggle: { checked: false }, + sidebarCollapsedByDefaultToggle: { checked: true }, infiniteScrollToggle: { checked: false }, debugModeToggle: { checked: false }, + apiTimeoutInput: { value: "30" }, checkThumbnailsAtStartup: { checked: true }, videoAutoplayToggle: { checked: true }, + videoMuteToggle: { checked: false }, videoLoopToggle: { checked: true }, + }; + const context = vm.createContext({ + document: { addEventListener() {}, getElementById: id => controls[id] }, + window: {}, console: { debug() {}, warn() {}, error() {} }, GallerySettings: settings, + localStorage: { getItem: key => storage.get(key) ?? null, setItem: (key, value) => storage.set(key, value) }, + }); + const file = path.join(__dirname, "../../web/js/gallery.js"); + const Manager = vm.runInContext(`${fs.readFileSync(file, "utf8")}\nGalleryManager;`, context, { filename: file }); + const manager = Object.create(Manager.prototype); + Object.assign(manager, { showNotification() {}, hideSettings() {}, applySettings() {} }); + manager.saveSettings(); + const saved = JSON.parse(storage.get("gallerySettings")); + assert.equal(saved.workspaceLayout, "masonry"); + assert.equal(saved.workspaceDensity, "compact"); + assert.equal(saved.someOtherTool, 1); + assert.equal(saved.defaultViewMode, "list"); + assert.equal(saved.showImageInfo, false); + assert.equal(saved.videoMute, false); +}); diff --git a/tests/js/workspace-images-state.test.js b/tests/js/workspace-images-state.test.js index b17aa9bf..bb81a6e9 100644 --- a/tests/js/workspace-images-state.test.js +++ b/tests/js/workspace-images-state.test.js @@ -478,3 +478,19 @@ test('a refresh deferred behind a page load is dropped when the stream is dispos await new Promise(resolve=>setImmediate(resolve)); assert.equal(requests.length,2,'no refresh runs after dispose'); }); + +test('defaults apply to a new visit but not to the current view or a saved view', async () => { + const { initialImagesView } = await source; + assert.equal(typeof initialImagesView, 'function', 'initialImagesView is exported'); + const preferences = { workspaceLayout: 'masonry', workspaceDensity: 'compact' }; + // A new visit starts from System › Appearance. + assert.deepEqual(initialImagesView({ preferences }), { layout: 'masonry', density: 'compact' }); + // The current view (the last visit) keeps its own toolbar choices. + assert.deepEqual(initialImagesView({ prior: { layout: 'list', density: 'large' }, preferences }), { layout: 'list', density: 'large' }); + assert.deepEqual(initialImagesView({ prior: {}, preferences }), { layout: 'grid', density: 'comfortable' }); + // A saved view opens as it always did. + assert.deepEqual(initialImagesView({ saved: { q: '' }, preferences }), { layout: 'grid', density: 'comfortable' }); + // Without readable preferences nothing changes. + assert.deepEqual(initialImagesView({}), { layout: 'grid', density: 'comfortable' }); + assert.deepEqual(initialImagesView({ preferences: {} }), { layout: 'grid', density: 'comfortable' }); +}); diff --git a/tests/js/workspace-inspection-state.test.js b/tests/js/workspace-inspection-state.test.js index 82ea79d1..27d095e9 100644 --- a/tests/js/workspace-inspection-state.test.js +++ b/tests/js/workspace-inspection-state.test.js @@ -825,3 +825,73 @@ test('refresh re-reads the selected file and keeps the shown record until the ne assert.equal(controller.state.record.identity.path, 'folder/b.png'); controller.dispose(); assert.equal(await controller.refresh(), undefined); }); + +test('metadata off shows an explicit load and loads once on request', async () => { + const { api, requests } = pendingApi(), updates = []; + const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, autoLoad: false }); + controller.subscribe(state => updates.push(state)); + // Selecting reads nothing: the state says the metadata waits for a request. + assert.equal(controller.select(image()), undefined); + assert.equal(requests.length, 0); + assert.equal(controller.state.deferred, true); + assert.equal(controller.state.loading, false); + assert.equal(controller.state.record, null); + // Selecting the same file again still reads nothing. + controller.select({ ...image() }); + assert.equal(requests.length, 0); + const load = controller.load(); + assert.equal(requests.length, 1); + assert.equal(controller.state.loading, true); + assert.equal(controller.state.deferred, false); + // A second press while it reads, and the same file selected again once it loaded, read nothing more. + controller.load(); + assert.equal(requests.length, 1); + requests[0].resolve(envelope()); await load; + assert.equal(controller.state.record.generation.seed.value, '9007199254740993'); + controller.load(); controller.select(image()); + assert.equal(requests.length, 1); + // Another file waits for its own request again. + controller.select({ ...image(), path: 'folder/b.png', filename: 'b.png', url: image().url.replace('a%20%23%25', 'b') }); + assert.equal(controller.state.deferred, true); + assert.equal(requests.length, 1); + // The default controller loads on selection, as before. + const eager = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts }); + eager.select(image()); + assert.equal(requests.length, 2); + assert.equal(eager.state.deferred, false); + eager.dispose(); controller.dispose(); + assert.equal(controller.load(), undefined); +}); + +test('cache off rereads the current selection; on reuses the one current record', async () => { + for (const cache of [true, false]) { + const { api, requests } = pendingApi(); + const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, cache }); + const first = controller.select(image()); + // A read in flight is shared either way. + controller.select(image()); + assert.equal(requests.length, 1); + requests[0].resolve(envelope()); await first; + const again = controller.select(image()); + assert.equal(requests.length, cache ? 1 : 2, `cache ${cache}`); + if (!cache) { requests[1].resolve(envelope()); await again; assert.equal(controller.state.record.identity.path, 'folder/a #%.png'); } + controller.dispose(); + } +}); + +test('cache off never rereads a selection whose source changed, and refresh never loads a deferred one', async () => { + const { api, requests } = pendingApi(); + const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, cache: false }); + const first = controller.select(image()); + requests[0].reject(new ApiError(409, refreshMessage, 'source-changed')); await first; + assert.equal(controller.state.refreshRequired, true); + controller.select(image()); + assert.equal(requests.length, 1); + controller.dispose(); + const deferred = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, autoLoad: false }); + deferred.select(image()); + assert.equal(deferred.refresh(), undefined); + assert.equal(requests.length, 1); + assert.equal(deferred.state.deferred, true); + deferred.dispose(); +}); diff --git a/tests/js/workspace-preferences.test.js b/tests/js/workspace-preferences.test.js new file mode 100644 index 00000000..07b18cad --- /dev/null +++ b/tests/js/workspace-preferences.test.js @@ -0,0 +1,193 @@ +// Browser display defaults (#249 System S3 Task 7): the workspace reads and writes the legacy gallery's own +// `gallerySettings` object (Q3, "One display configuration"), writes only the keys it changed, never consumes the +// legacy paging keys, and says when this browser refused to keep a change. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import gallerySettings from '../../web/js/gallery-settings.js'; + +const module = await import('../../web/js/workspace/preferences.mjs').catch(error => { + if (error.code !== 'ERR_MODULE_NOT_FOUND') throw error; + return {}; +}); +globalThis.GallerySettings = gallerySettings; +const KEY = 'gallerySettings'; + +/** A localStorage stand-in holding `text` under the settings key; `setItem` can be made to throw. */ +function storage(text = null, { quota = false } = {}) { + const writes = []; + const store = { + text, + writes, + getItem(key) { assert.equal(key, KEY); return store.text; }, + setItem(key, value) { + assert.equal(key, KEY); + writes.push(value); + if (quota) { const error = new Error('Quota exceeded'); error.name = 'QuotaExceededError'; throw error; } + store.text = value; + }, + removeItem() { assert.fail('Preferences never remove the shared settings'); }, + }; + return store; +} +const denied = () => { const error = new Error('Denied'); error.name = 'SecurityError'; throw error; }; + +test('normalizes new workspace keys and keeps legacy keys', () => { + assert.equal(typeof module.readPreferences, 'function', 'preferences.mjs exports readPreferences'); + module.resetTabPreferences(); + const legacy = { defaultViewMode: 'list', gridColumns: 5, defaultLimit: 200, thumbnailPromptDismissed: 'sig', extra: 1 }; + const store = storage(JSON.stringify({ ...legacy, workspaceLayout: 'masonry', workspaceDensity: 'compact', showFilePaths: false })); + const { values, persisted } = module.readPreferences(() => store); + assert.equal(persisted, true); + assert.equal(values.workspaceLayout, 'masonry'); + assert.equal(values.workspaceDensity, 'compact'); + assert.equal(values.showFilePaths, false); + // Invalid workspace values fall back to the defaults, as every legacy key does. + store.text = JSON.stringify({ workspaceLayout: 'mosaic', workspaceDensity: 7, videoMute: 'no' }); + const fallback = module.readPreferences(() => store).values; + assert.equal(fallback.workspaceLayout, 'grid'); + assert.equal(fallback.workspaceDensity, 'comfortable'); + assert.equal(fallback.videoMute, true); + // The legacy normalizer knows the new keys and still keeps unknown ones. + const normalized = gallerySettings.normalizeGallerySettings({ ...legacy, workspaceLayout: 'list' }); + assert.equal(normalized.workspaceLayout, 'list'); + assert.equal(normalized.workspaceDensity, 'comfortable'); + assert.equal(normalized.extra, 1); + assert.equal(normalized.thumbnailPromptDismissed, 'sig'); + assert.deepEqual(gallerySettings.WORKSPACE_LAYOUTS, ['grid', 'list', 'masonry']); + assert.deepEqual(gallerySettings.WORKSPACE_DENSITIES, ['comfortable', 'large', 'compact']); +}); + +test('write touches only dirty keys', () => { + module.resetTabPreferences(); + const stored = { defaultViewMode: 'list', gridColumns: 5, thumbnailsGenerated: true, extra: { kept: true }, videoMute: false }; + const store = storage(JSON.stringify(stored)); + const result = module.writePreferences({ workspaceLayout: 'masonry', showImageInfo: false }, () => store); + assert.deepEqual(result, { persisted: true, conflicts: [] }); + assert.equal(store.writes.length, 1); + // Every other key, legacy or unknown, is written back exactly as it was stored: no defaults are filled in. + assert.deepEqual(JSON.parse(store.text), { ...stored, workspaceLayout: 'masonry', showImageInfo: false }); + // It re-reads the stored object, so a key another page wrote after this one read is kept. + store.text = JSON.stringify({ ...JSON.parse(store.text), gridColumns: 9 }); + module.writePreferences({ videoLoop: false }, () => store); + assert.equal(JSON.parse(store.text).gridColumns, 9); + assert.equal(JSON.parse(store.text).workspaceLayout, 'masonry'); + // An empty or malformed stored value starts from an empty object, never the defaults. + for (const text of [null, '{broken', '[]', '42']) { + const fresh = storage(text); + assert.equal(module.writePreferences({ cacheMetadata: false }, () => fresh).persisted, true); + assert.deepEqual(JSON.parse(fresh.text), { cacheMetadata: false }); + } + // Only workspace keys with valid values are accepted; nothing is written for a bad change. + for (const bad of [{ gridColumns: 4 }, { defaultLimit: 50 }, { workspaceLayout: 'mosaic' }, { videoMute: 'yes' }, { imageQuality: 'ultra' }]) { + const untouched = storage(JSON.stringify(stored)); + assert.throws(() => module.writePreferences(bad, () => untouched), TypeError); + assert.equal(untouched.writes.length, 0); + } + // A key changed elsewhere since this tab read it (its baseline) is reported, and the explicit save still wins. + const elsewhere = storage(JSON.stringify({ videoLoop: false })); + const outcome = module.writePreferences({ videoLoop: true, videoMute: false }, () => elsewhere, { baseline: { videoLoop: true, videoMute: true } }); + assert.deepEqual(outcome, { persisted: true, conflicts: ['videoLoop'] }); + assert.equal(JSON.parse(elsewhere.text).videoLoop, true); +}); + +test('denied storage reports tab-only', () => { + module.resetTabPreferences(); + const read = module.readPreferences(denied); + assert.equal(read.persisted, false); + assert.equal(read.values.workspaceLayout, 'grid'); + const result = module.writePreferences({ workspaceLayout: 'list', videoAutoplay: true }, denied); + assert.deepEqual(result, { persisted: false, conflicts: [] }); + // The change is kept for this tab: every reader in it (a new Images visit, the viewer) sees it. + const tab = module.readPreferences(denied); + assert.equal(tab.persisted, false); + assert.equal(tab.values.workspaceLayout, 'list'); + assert.equal(tab.values.videoAutoplay, true); + // A getItem that throws is denied storage too. + const throwing = { getItem: denied, setItem: denied }; + assert.equal(module.readPreferences(() => throwing).values.workspaceLayout, 'list'); + assert.equal(module.writePreferences({ workspaceDensity: 'large' }, () => throwing).persisted, false); + // Once storage accepts a write, the tab copy of that key gives way to the stored value. + const working = storage(JSON.stringify({ workspaceLayout: 'masonry' })); + assert.equal(module.writePreferences({ workspaceLayout: 'masonry' }, () => working).persisted, true); + assert.equal(module.readPreferences(() => working).values.workspaceLayout, 'masonry'); + module.resetTabPreferences(); + assert.equal(module.readPreferences(denied).values.videoAutoplay, false); +}); + +test('quota error keeps previous bytes', () => { + module.resetTabPreferences(); + const before = JSON.stringify({ gridColumns: 6, workspaceDensity: 'large' }); + const store = storage(before, { quota: true }); + const result = module.writePreferences({ workspaceDensity: 'compact' }, () => store); + assert.deepEqual(result, { persisted: false, conflicts: [] }); + assert.equal(store.writes.length, 1); + assert.equal(store.text, before); + // This tab keeps the change it could not store. + assert.equal(module.readPreferences(() => store).values.workspaceDensity, 'compact'); + module.resetTabPreferences(); + assert.equal(module.readPreferences(() => store).values.workspaceDensity, 'large'); +}); + +/** A window stand-in that records its storage listeners. */ +function fakeWindow() { + const listeners = new Set(); + return { + listeners, + addEventListener(type, listener) { assert.equal(type, 'storage'); listeners.add(listener); }, + removeEventListener(type, listener) { assert.equal(type, 'storage'); listeners.delete(listener); }, + fire(event) { for (const listener of [...listeners]) listener(event); }, + }; +} + +test('storage event updates clean values and flags dirty conflicts', () => { + module.resetTabPreferences(); + const store = storage(JSON.stringify({ workspaceLayout: 'list', videoMute: false })); + const win = fakeWindow(); + const seen = []; + const unsubscribe = module.subscribePreferences(win, next => seen.push(next), () => store); + assert.equal(win.listeners.size, 1); + // Another key, or another storage area's event, is not ours. + win.fire({ key: 'pm_update_notice_dismissed', storageArea: store }); + win.fire({ key: KEY, storageArea: { other: true } }); + assert.equal(seen.length, 0); + win.fire({ key: KEY, storageArea: store }); + win.fire({ key: null, storageArea: store }); // localStorage.clear() elsewhere + assert.equal(seen.length, 2); + assert.equal(seen[0].values.workspaceLayout, 'list'); + unsubscribe(); + assert.equal(win.listeners.size, 0); + // The reconciliation of the System drafts: clean values follow the other page; a key this tab has a draft for and + // the other page changed is a conflict, and the draft stays. + const baseline = { workspaceLayout: 'grid', videoMute: true, videoLoop: true }; + const next = { workspaceLayout: 'list', videoMute: true, videoLoop: false }; + assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { videoMute: false, videoLoop: false }, next }), { conflicts: [] }); + assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { workspaceLayout: 'masonry' }, next }), { conflicts: ['workspaceLayout'] }); + assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { workspaceLayout: 'list' }, next }), { conflicts: [] }); +}); + +test('legacy paging keys are never consumed by the workspace', () => { + module.resetTabPreferences(); + const store = storage(JSON.stringify({ defaultLimit: 500, gridColumns: 12, infiniteScroll: true, lazyLoading: false, + defaultViewMode: 'list', debugMode: true, apiTimeout: 9, thumbnailsGenerated: true, checkThumbnailsAtStartup: false })); + const { values } = module.readPreferences(() => store); + assert.deepEqual(Object.keys(values).sort(), [...module.WORKSPACE_PREFERENCE_KEYS].sort()); + for (const key of ['defaultLimit', 'gridColumns', 'infiniteScroll', 'lazyLoading', 'defaultViewMode', 'debugMode', 'apiTimeout', + 'thumbnailsGenerated', 'checkThumbnailsAtStartup', 'thumbnailPromptDismissed']) assert.equal(Object.hasOwn(values, key), false, key); + // The display defaults are the gallery's own defaults for these keys. + const defaults = module.preferenceDefaults(); + for (const key of module.WORKSPACE_PREFERENCE_KEYS) assert.equal(defaults[key], gallerySettings.DEFAULT_SETTINGS[key], key); +}); + +test('a later save flushes the changes kept for this tab only', () => { + module.resetTabPreferences(); + const store = storage(JSON.stringify({ gridColumns: 4 }), { quota: true }); + assert.equal(module.writePreferences({ workspaceLayout: 'list' }, () => store).persisted, false); + // Storage accepts writes again: saving another key stores the tab-only change with it, so "Saved" is true for both. + const working = storage(store.text); + assert.deepEqual(module.writePreferences({ videoLoop: false }, () => working), { persisted: true, conflicts: [] }); + assert.deepEqual(JSON.parse(working.text), { gridColumns: 4, workspaceLayout: 'list', videoLoop: false }); + // Nothing is overlaid any more: another page's later change reads through. + working.text = JSON.stringify({ ...JSON.parse(working.text), workspaceLayout: 'masonry' }); + assert.equal(module.readPreferences(() => working).values.workspaceLayout, 'masonry'); + module.resetTabPreferences(); +}); diff --git a/tests/js/workspace-system-appearance.test.js b/tests/js/workspace-system-appearance.test.js new file mode 100644 index 00000000..497df7f6 --- /dev/null +++ b/tests/js/workspace-system-appearance.test.js @@ -0,0 +1,188 @@ +// System › Appearance (#249 System S3 Task 8): the concept's "Browsing & display" rows (app.js 676–678) on the drafts +// of the one Save row, their stored values (Q3 mappings), the host timeout's 0–300 whole seconds and the D97 rows. +const test = require('node:test'); +const assert = require('node:assert/strict'); +const gallerySettings = require('../../web/js/gallery-settings.js'); + +const appearanceModule = import('../../web/js/workspace/system-appearance.mjs'); +const stateModule = import('../../web/js/workspace/system-state.mjs'); +const coreModule = import('../../web/js/shared/toolkit/core.js'); +globalThis.GallerySettings = gallerySettings; + +/** A minimal DOM whose elements count their live listeners (a listener's signal aborts it, as in a browser). */ +function fakeDOM() { + const previous = global.document; + const elements = []; + class Element { + constructor(tag) { + this.tagName = tag.toUpperCase(); + this.children = []; + this.attributes = {}; + this.listeners = []; + this.dataset = {}; + this.className = ''; + this.id = ''; + this.value = ''; + this._text = ''; + elements.push(this); + } + get textContent() { return this._text + this.children.map(child => child.textContent).join(''); } + set textContent(text) { this.children = []; this._text = String(text); } + append(...nodes) { + for (const node of nodes) { + const child = typeof node === 'string' ? Object.assign(new Element('#text'), { _text: node }) : node; + child.parentNode = this; + this.children.push(child); + } + } + remove() { if (this.parentNode) this.parentNode.children = this.parentNode.children.filter(child => child !== this); this.parentNode = null; } + replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); } + setAttribute(name, value) { this.attributes[name] = String(value); } + getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; } + removeAttribute(name) { delete this.attributes[name]; } + addEventListener(type, callback, options = {}) { + if (options.signal?.aborted) return; + const entry = { type, callback }; + this.listeners.push(entry); + options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); }); + } + removeEventListener(type, callback) { this.listeners = this.listeners.filter(item => item.type !== type || item.callback !== callback); } + dispatch(type) { for (const { callback } of this.listeners.filter(item => item.type === type)) callback({ type, target: this }); } + click() { this.dispatch('click'); } + } + global.document = { + createElement: tag => new Element(tag), + createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }), + }; + const walk = (node, found = []) => { found.push(node); node.children.forEach(child => walk(child, found)); return found; }; + return { + listenerCount: () => elements.reduce((sum, element) => sum + element.listeners.length, 0), + find: (root, predicate) => walk(root).find(predicate), + tags: node => node.children.map(child => child.tagName.toLowerCase()), + restore() { global.document = previous; }, + }; +} + +const BROWSER = { workspaceLayout: 'grid', workspaceDensity: 'comfortable', autoLoadMetadata: true, cacheMetadata: true, + imageQuality: 'medium', videoAutoplay: false, videoMute: true, videoLoop: true, showFilePaths: true, showImageInfo: true, + sidebarCollapsedByDefault: false }; + +async function mounted({ host = { webui_display_mode: 'newtab', result_timeout: 5 }, browser = BROWSER, hostFails = false } = {}) { + const { mountAppearance } = await appearanceModule; + const { createSystemDrafts } = await stateModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + let hostValues = null; + const drafts = createSystemDrafts({ + readBrowser: () => ({ ...browser }), readRuntime: () => ({}), + readHost: async () => { if (hostFails) throw new Error('offline'); return { values: hostValues, revision: 'sha256:one' }; }, + writeBrowser: async () => ({ persisted: true }), writeHost: async () => ({ success: true, persisted: true }), writeRuntime: async () => ({}), + }); + await drafts.load('browser'); + const scope = createScope(); + const [surface] = mountAppearance(null, { scope, drafts }); + const byId = id => dom.find(surface, node => node.dataset?.setting === id); + return { dom, drafts, scope, surface, byId, loadHost: async () => { hostValues = host; await drafts.load('host'); }, + fixHost: () => { hostFails = false; } }; +} + +test('the surface is the concept section with the concept rows in order, then the D97 rows', async () => { + const { APPEARANCE_ROWS } = await appearanceModule; + const { dom, surface, scope } = await mounted(); + try { + assert.equal(surface.className, 'surface'); + assert.equal(surface.children[0].textContent, 'Browsing & display'); + assert.equal(surface.children[1].textContent, 'One set of defaults; toolbar changes apply to the current view.'); + const rows = surface.children.slice(2); + const titles = rows.map(row => row.children[0].children[0].textContent); + assert.deepEqual(titles, ['Default layout', 'Thumbnail size', 'Continuous browsing', 'Load metadata on selection', + 'Cache metadata in this browser', 'Image loading quality', 'Autoplay videos', 'Mute videos by default', 'Loop videos', + 'Show file paths', 'Open from ComfyUI', 'Node search auto-hide (seconds)', 'Show image details on cards', + 'Collapse viewer details by default', 'Reset display defaults']); + const continuous = rows[2]; + assert.deepEqual(dom.tags(continuous.children[0]), ['h3', 'p']); + assert.equal(continuous.children[1].className, 'chip blue'); + assert.equal(continuous.children[1].textContent, 'Infinite scroll'); + assert.equal(rows.at(-1).className, 'operation'); + assert.equal(APPEARANCE_ROWS.filter(Array.isArray).length, 13); + } finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); } +}); + +test('added rows reach their consumers: each control edits its stored key with the Q3 values', async () => { + const { dom, drafts, scope, byId, loadHost } = await mounted(); + try { + // Host rows stay disabled until the server settings answered. + assert.equal(byId('open-mode').disabled, true); + assert.equal(byId('timeout').disabled, true); + await loadHost(); + assert.equal(byId('open-mode').disabled, false); + assert.equal(byId('timeout').value, '5'); + assert.deepEqual(byId('image-quality').children.map(option => [option.value, option.textContent]), + [['medium', 'Optimized'], ['high', 'Original'], ['low', 'Fast']]); + assert.deepEqual(byId('open-mode').children.map(option => [option.value, option.textContent]), + [['newtab', 'New tab'], ['popup', 'Popup window']]); + const edits = [['default-view', 'masonry'], ['density', 'large'], ['image-quality', 'low'], ['open-mode', 'popup']]; + for (const [id, value] of edits) { byId(id).value = value; byId(id).dispatch('change'); } + for (const id of ['metadata-auto', 'metadata-cache', 'video-muted', 'video-loop', 'paths', 'card-details']) { + byId(id).checked = false; byId(id).dispatch('change'); + } + for (const id of ['video-auto', 'viewer-details']) { byId(id).checked = true; byId(id).dispatch('change'); } + assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'masonry', workspaceDensity: 'large', imageQuality: 'low', + autoLoadMetadata: false, cacheMetadata: false, videoMute: false, videoLoop: false, showFilePaths: false, showImageInfo: false, + videoAutoplay: true, sidebarCollapsedByDefault: true }); + assert.deepEqual(drafts.changes('host'), { webui_display_mode: 'popup' }); + } finally { scope.dispose(); dom.restore(); } +}); + +test('the auto-hide timeout takes whole seconds from 0 to 300 only', async () => { + const { resultTimeout } = await appearanceModule; + assert.deepEqual(['0', '30', ' 300 ', '301', '-1', '2.5', '', 'abc', '1e2'].map(resultTimeout), [0, 30, 300, null, null, null, null, null, null]); + const { dom, drafts, scope, byId, loadHost } = await mounted(); + try { + await loadHost(); + const timeout = byId('timeout'); + timeout.value = '45'; timeout.dispatch('change'); + assert.deepEqual(drafts.changes('host'), { result_timeout: 45 }); + timeout.value = '301'; timeout.dispatch('change'); + assert.deepEqual(drafts.changes('host'), {}); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + timeout.value = '0'; timeout.dispatch('change'); + assert.deepEqual(drafts.changes('host'), { result_timeout: 0 }); + assert.deepEqual([...drafts.failed('host')], []); + } finally { scope.dispose(); dom.restore(); } +}); + +test('reset restores defaults without touching other keys', async () => { + const { DISPLAY_KEYS } = await appearanceModule; + const changed = { ...BROWSER, workspaceLayout: 'list', videoMute: false, showImageInfo: false, sidebarCollapsedByDefault: true }; + const { dom, drafts, scope, surface, loadHost } = await mounted({ browser: changed }); + try { + await loadHost(); + drafts.set('host', 'result_timeout', 9); + const reset = dom.find(surface.children.at(-1), node => node.tagName === 'BUTTON'); + assert.equal(reset.textContent, 'Reset'); + reset.click(); + assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'grid', videoMute: true, showImageInfo: true, sidebarCollapsedByDefault: false }); + // The host fields and every key outside the display defaults are left as they are. + assert.deepEqual(drafts.changes('host'), { result_timeout: 9 }); + assert.deepEqual([...DISPLAY_KEYS].sort(), Object.keys(BROWSER).sort()); + } finally { scope.dispose(); dom.restore(); } +}); + +test('host rows that could not be read stay disabled and say so; a later read clears it', async () => { + const { HOST_UNAVAILABLE } = await appearanceModule; + const { dom, drafts, scope, byId, surface, fixHost } = await mounted({ hostFails: true }); + const notes = () => surface.children.slice(2).flatMap(row => row.children[0].children.slice(2).map(note => [row.children[0].children[0].textContent, note.className, note.textContent])); + try { + assert.deepEqual(notes(), []); + assert.equal(await drafts.reloadHost(), false); + assert.deepEqual(notes(), [['Open from ComfyUI', 'small', HOST_UNAVAILABLE], ['Node search auto-hide (seconds)', 'small', HOST_UNAVAILABLE]]); + assert.equal(byId('timeout').disabled, true); + // Repaints keep one line per row. + drafts.set('browser', 'videoMute', false); + assert.equal(notes().length, 2); + fixHost(); + assert.equal(await drafts.reloadHost(), true); + assert.deepEqual(notes(), []); + } finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); } +}); diff --git a/tests/js/workspace-system-state.test.js b/tests/js/workspace-system-state.test.js index a9e09118..af60efb0 100644 --- a/tests/js/workspace-system-state.test.js +++ b/tests/js/workspace-system-state.test.js @@ -259,3 +259,60 @@ test('a failed read keeps the store usable and drafts survive a reload (Q10)', a assert.equal(failing.dirty(), false); assert.throws(() => createSystemDrafts({ ...domains, writeHost: null }), TypeError); }); + +test('a rejected value drops its draft and is highlighted; a conflict elsewhere keeps the drafts', async () => { + const domains = adapters({ browser: { videoMute: true, videoLoop: true }, host: { result_timeout: 5 } }); + const drafts = await loaded(domains); + assert.deepEqual(drafts.saved('browser'), { videoMute: true, videoLoop: true }); + drafts.set('host', 'result_timeout', 12); + assert.equal(drafts.dirty('host'), true); + drafts.reject('host', 'result_timeout'); + assert.equal(drafts.dirty('host'), false); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + assert.equal(drafts.get('host', 'result_timeout'), 5); + // A valid value clears the highlight. + drafts.set('host', 'result_timeout', 7); + assert.deepEqual([...drafts.failed('host')], []); + drafts.discard(); + drafts.set('browser', 'videoMute', false); + drafts.conflict('browser', []); + assert.equal(drafts.statusText(), 'Unsaved changes.'); + drafts.conflict('browser', ['videoMute']); + assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.'); + assert.deepEqual([...drafts.failed('browser')], ['videoMute']); + assert.equal(drafts.get('browser', 'videoMute'), false); + assert.throws(() => drafts.reject('nowhere', 'x'), TypeError); +}); + +test('a rejected value says why in the status until the next edit', async () => { + const drafts = await loaded(adapters({ host: { result_timeout: 5 } })); + drafts.reject('host', 'result_timeout'); + assert.equal(drafts.statusText(), 'Some changes were not saved. Review the highlighted settings.'); + drafts.set('host', 'result_timeout', 9); + assert.equal(drafts.statusText(), 'Unsaved changes.'); +}); + +test('a browser write refused as changed elsewhere keeps the draft and reads the browser values again', async () => { + const domains = adapters({ browser: { workspaceDensity: 'comfortable' } }); + const drafts = await loaded(domains); + drafts.set('browser', 'workspaceDensity', 'compact'); + domains.outcome.browser = () => { throw Object.assign(new Error('changed'), { code: 'settings-changed' }); }; + assert.deepEqual(await drafts.save(), { browser: 'conflict', host: 'clean', runtime: 'clean' }); + assert.deepEqual(domains.calls.map(([name]) => name), ['writeBrowser', 'readBrowser']); + assert.equal(drafts.get('browser', 'workspaceDensity'), 'compact'); + assert.deepEqual([...drafts.failed('browser')], ['workspaceDensity']); + assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.'); +}); + +test('host read state says whether the server settings could be read', async () => { + const { createSystemDrafts } = await source; + const domains = adapters({ host: { result_timeout: 5 } }); + let fail = true; + const drafts = createSystemDrafts({ ...domains, readHost: async () => { if (fail) throw new Error('offline'); return { values: {}, revision: 'r' }; } }); + assert.equal(drafts.hostState(), 'pending'); + assert.equal(await drafts.reloadHost(), false); + assert.equal(drafts.hostState(), 'failed'); + fail = false; + assert.equal(await drafts.reloadHost(), true); + assert.equal(drafts.hostState(), 'loaded'); +}); diff --git a/tests/test_system_api.py b/tests/test_system_api.py index f72e8d4f..0a902085 100644 --- a/tests/test_system_api.py +++ b/tests/test_system_api.py @@ -416,6 +416,63 @@ class TestStatisticsRoute(SystemAPITestCase): ) +class TestFailedReadsSettleFirst(SystemAPITestCase): + """A failed read answers only after the route's other reads finished. + + Answering on the first failure (``asyncio.gather`` without + ``return_exceptions``) left the other executor reads running against a + database the caller may already be closing; on Python 3.10 CI that + crashed the test process inside SQLite. + """ + + async def assert_answers_after_reads(self, url, failing, slow, error): + raised, release, finished = (threading.Event() for _ in range(3)) + real = getattr(*slow) + + def fail(*args, **kwargs): + raised.set() + raise RuntimeError("secret") + + def held(*args, **kwargs): + release.wait() + try: + return real(*args, **kwargs) + finally: + finished.set() + + with ( + patch.object(*failing, side_effect=fail), + patch.object(*slow, side_effect=held), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + answer = asyncio.ensure_future(self.json("GET", url, 500)) + try: + await asyncio.get_running_loop().run_in_executor(None, raised.wait) + done, _ = await asyncio.wait({answer}, timeout=0.5) + self.assertEqual(done, set(), "answered while a read still ran") + finally: + release.set() + body = await answer + self.assertTrue(finished.is_set()) + self.assertEqual(body, {"success": False, "code": None, "error": error}) + + async def test_statistics(self): + await self.assert_answers_after_reads( + "/prompt_manager/system/statistics", + (self.api.db, "usage_by_category"), + (self.api.db, "get_statistics"), + "Failed to read statistics", + ) + + async def test_overview(self): + await self.assert_answers_after_reads( + "/prompt_manager/system/overview", + (self.api.db, "library_counts"), + (self.api, "_output_dirs_with_origin"), + "Failed to read the overview", + ) + + if __name__ == "__main__": import unittest diff --git a/tests/test_workspace_system_fixture.py b/tests/test_workspace_system_fixture.py new file mode 100644 index 00000000..a43ec854 --- /dev/null +++ b/tests/test_workspace_system_fixture.py @@ -0,0 +1,38 @@ +"""The System browser hooks only ever reset their own server's config.json (#249 S3).""" + +import os +import tempfile +import unittest +from pathlib import Path +from unittest import mock + +from tests.browser.workspace_system_fixture import register_system_commands + + +class TestWorkspaceSystemFixture(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.root = Path(self.tmp.name) / "server" + self.root.mkdir() + + def test_refuses_a_config_path_outside_its_root(self): + elsewhere = Path(self.tmp.name) / "config.json" + elsewhere.write_bytes(b'{"real": true}') + for value in (str(elsewhere), None): + env = { + k: v for k, v in os.environ.items() if k != "PROMPT_MANAGER_CONFIG_PATH" + } + if value is not None: + env["PROMPT_MANAGER_CONFIG_PATH"] = value + with ( + self.subTest(value=value), + mock.patch.dict(os.environ, env, clear=True), + ): + with self.assertRaisesRegex(RuntimeError, "PROMPT_MANAGER_CONFIG_PATH"): + register_system_commands(None, None, None, self.root) + self.assertEqual(elsewhere.read_bytes(), b'{"real": true}') + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/vendor/frontend-coverage-baseline.json b/tools/vendor/frontend-coverage-baseline.json index 88dfa16e..177df6b1 100644 --- a/tools/vendor/frontend-coverage-baseline.json +++ b/tools/vendor/frontend-coverage-baseline.json @@ -37,15 +37,15 @@ }, "gallery-settings.js": { "lines": 98, - "branches": 90, + "branches": 94, "functions": 98, "statements": 98 }, "gallery.js": { - "lines": 40, + "lines": 41, "branches": 63, "functions": 44, - "statements": 40 + "statements": 41 }, "html-escape.js": { "lines": 98, @@ -401,6 +401,12 @@ "functions": 98, "statements": 98 }, + "workspace/preferences.mjs": { + "lines": 98, + "branches": 96, + "functions": 98, + "statements": 98 + }, "workspace/prompt-blocks.mjs": { "lines": 90, "branches": 92, @@ -443,6 +449,12 @@ "functions": 98, "statements": 98 }, + "workspace/system-appearance.mjs": { + "lines": 98, + "branches": 93, + "functions": 98, + "statements": 98 + }, "workspace/system-controls.mjs": { "lines": 98, "branches": 87, @@ -457,15 +469,15 @@ }, "workspace/system-state.mjs": { "lines": 98, - "branches": 92, - "functions": 92, + "branches": 94, + "functions": 98, "statements": 98 }, "workspace/system.mjs": { - "lines": 93, - "branches": 85, - "functions": 69, - "statements": 93 + "lines": 98, + "branches": 92, + "functions": 85, + "statements": 98 }, "workspace/theme.mjs": { "lines": 98, diff --git a/web/js/gallery-settings.js b/web/js/gallery-settings.js index 4c21dd62..3ea591d0 100644 --- a/web/js/gallery-settings.js +++ b/web/js/gallery-settings.js @@ -4,6 +4,9 @@ * * Settings are per browser (localStorage key "gallerySettings"). Anything * stored can be stale or hand-edited, so every value is checked before use. + * The workspace shares this object ("One display configuration", #249 System + * S3): it adds workspaceLayout and workspaceDensity, and every key either page + * does not know is kept. * * Loadable in the browser (window.GallerySettings) and in Node for tests. */ @@ -11,6 +14,9 @@ "use strict"; const VIEW_MODES = ["grid", "list"]; + // The workspace Images defaults; the legacy gallery's defaultViewMode has no masonry. + const WORKSPACE_LAYOUTS = ["grid", "list", "masonry"]; + const WORKSPACE_DENSITIES = ["comfortable", "large", "compact"]; const LIMIT_OPTIONS = [50, 100, 200, 500]; const MIN_COLUMNS = 2; const MAX_COLUMNS = 12; @@ -34,6 +40,8 @@ videoAutoplay: false, videoMute: true, videoLoop: true, + workspaceLayout: "grid", + workspaceDensity: "comfortable", }); function toInt(value) { @@ -72,6 +80,8 @@ videoAutoplay: bool(merged.videoAutoplay, DEFAULT_SETTINGS.videoAutoplay), videoMute: bool(merged.videoMute, DEFAULT_SETTINGS.videoMute), videoLoop: bool(merged.videoLoop, DEFAULT_SETTINGS.videoLoop), + workspaceLayout: WORKSPACE_LAYOUTS.includes(merged.workspaceLayout) ? merged.workspaceLayout : DEFAULT_SETTINGS.workspaceLayout, + workspaceDensity: WORKSPACE_DENSITIES.includes(merged.workspaceDensity) ? merged.workspaceDensity : DEFAULT_SETTINGS.workspaceDensity, }; } @@ -81,7 +91,7 @@ return `repeat(${n}, minmax(0, 1fr))`; } - const api = { DEFAULT_SETTINGS, LIMIT_OPTIONS, VIEW_MODES, normalizeGallerySettings, gridColumnsStyle }; + const api = { DEFAULT_SETTINGS, LIMIT_OPTIONS, VIEW_MODES, WORKSPACE_LAYOUTS, WORKSPACE_DENSITIES, normalizeGallerySettings, gridColumnsStyle }; if (typeof module !== "undefined" && module.exports) module.exports = api; else root.GallerySettings = api; })(typeof window !== "undefined" ? window : globalThis); diff --git a/web/js/gallery.js b/web/js/gallery.js index ee9f24bd..cad3c2ca 100644 --- a/web/js/gallery.js +++ b/web/js/gallery.js @@ -1768,7 +1768,10 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`; } saveSettings() { + // The workspace shares this object (its layout and density, #249 System S3), so a Save starts from what is + // stored and keeps every key this dialog does not edit. const settings = { + ...this.getSettings(), lazyLoading: document.getElementById('lazyLoadingToggle').checked, imageQuality: document.getElementById('imageQualitySelect').value, defaultViewMode: document.getElementById('defaultViewModeSelect').value, @@ -1782,8 +1785,6 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`; infiniteScroll: document.getElementById('infiniteScrollToggle').checked, debugMode: document.getElementById('debugModeToggle').checked, apiTimeout: parseInt(document.getElementById('apiTimeoutInput').value), - thumbnailsGenerated: this.getSettings().thumbnailsGenerated, // Preserve this - thumbnailPromptDismissed: this.getSettings().thumbnailPromptDismissed, checkThumbnailsAtStartup: document.getElementById('checkThumbnailsAtStartup').checked, // Video settings videoAutoplay: document.getElementById('videoAutoplayToggle').checked, diff --git a/web/js/workspace/file-inspector.mjs b/web/js/workspace/file-inspector.mjs index 2d7a032d..80465acd 100644 --- a/web/js/workspace/file-inspector.mjs +++ b/web/js/workspace/file-inspector.mjs @@ -6,6 +6,7 @@ import { createWorkspaceIcon } from './icons.mjs'; import { navigateTo } from './state.mjs'; import { showWorkspaceToast } from './toast.mjs'; import { openCopyFallback } from './workspace-dialogs.mjs'; +import { displayPreferences } from './preferences.mjs'; const RAW_KEYS = ['prompt', 'workflow', 'parameters', 'comfyassets_prompt_pairs']; const RAW_LIMIT = 'Raw metadata exceeds the response limit. Download the original file to inspect it externally.'; @@ -196,7 +197,10 @@ function prefixedChip(prefix, text) { * `variant: 'full'` (the viewer's `#viewer-details`, imageDetails(i, true)) leaves out the cover and adds "File & * provenance" and "Workflow & raw metadata" with the downloads. `inline: true` means `panel`; neither means `full`. */ -export function mountFileInspector(container, {api, sender, parentScope, variant, inline = false, promptContext = '', onTagFilter, onRetry, onRefresh, onInteraction = () => {}, copyFallback = openCopyFallback}) { +/** D124: the inspector's line while "Load metadata on selection" is off. */ +export const DEFERRED_TEXT = 'Metadata loads when you ask for it.'; + +export function mountFileInspector(container, {api, sender, parentScope, variant, inline = false, promptContext = '', onTagFilter, onRetry, onRefresh, onLoad, onInteraction = () => {}, copyFallback = openCopyFallback}) { if (parentScope.signal.aborted) return {setState() {}, dispose() {}}; const scope = childScope(parentScope); let contentScope; @@ -235,6 +239,13 @@ export function mountFileInspector(container, {api, sender, parentScope, variant function render(state) { const inner = el('div', undefined, full ? undefined : 'detail-inner'); if (state.loading || state.error) {container.append(inner); statusView(state, inner); return;} + // D124: "Load metadata on selection" is off, so the file's details wait for an explicit request. + if (state.deferred) { + container.append(inner); + const status = el('p', DEFERRED_TEXT, 'small'); status.setAttribute('role', 'status'); inner.append(status); + if (onLoad) {const load = createButton('Load metadata'); contentScope.listen(load, 'click', onLoad); inner.append(load);} + return; + } const record = state.record; if (!record) return; const {file, library} = record; @@ -326,8 +337,10 @@ export function mountFileInspector(container, {api, sender, parentScope, variant section.append(el('p', reported ? 'Current file at the reported location · identity unverified' : local ? 'Selected local file · read in this browser' : 'Current file at this source; historical bytes are not verified.', 'small')); const path = el('dd', record.identity.path ?? 'Not available'); + // System › Appearance "Show file paths" off: the Path row is left out; nothing else changes. + const showPath = !local && displayPreferences()?.showFilePaths !== false; const facts = file && definitionList([ - ['Source', file.source_label], ...(local ? [] : [['Path', path]]), ['Size', formatBytes(file.size)], + ['Source', file.source_label], ...(showPath ? [['Path', path]] : []), ['Size', formatBytes(file.size)], ['Dimensions', file.width && file.height ? `${file.width} × ${file.height}` : null], ['Format', file.format || `${extensionOf(record)} extension`], ['Media type', file.media_type], ['Modified', file.modified_time !== null ? new Date(file.modified_time * 1000).toLocaleString() : null], @@ -335,8 +348,9 @@ export function mountFileInspector(container, {api, sender, parentScope, variant if (facts) { // D38: "Show in folder" follows the Path value. const target = api ? revealTarget(record.identity) : null; - if (target) path.append(' ', revealButton(target)); section.append(facts); + if (target && showPath) path.append(' ', revealButton(target)); + else if (target) section.append(revealButton(target)); } else section.append(el('p', 'Current file observations are unavailable for this inspection response.', 'small')); // The concept's "Prompt evidence: execution recorded" line, in the D21 badge's words; no badge, no line. if (badge) section.append(el('p', `Prompt evidence: ${badge.toLowerCase()}`, 'small')); diff --git a/web/js/workspace/images-state.mjs b/web/js/workspace/images-state.mjs index d133eba0..c61daf46 100644 --- a/web/js/workspace/images-state.mjs +++ b/web/js/workspace/images-state.mjs @@ -1,6 +1,17 @@ import { noun } from './plural.mjs'; /** Identity is the source and relative path; content hashes may be shared by files. */ +/** + * The layout and thumbnail size an Images visit starts with (#249 System S3). The last visit (the current view) keeps + * its own; a saved view starts as before, at Grid and Comfortable; any other new visit starts from System › + * Appearance's defaults (`preferences`, preferences.mjs values, or null when unavailable). + */ +export function initialImagesView({ prior = null, saved = null, preferences = null } = {}) { + if (prior) return { layout: prior.layout || 'grid', density: prior.density || 'comfortable' }; + if (saved || !preferences) return { layout: 'grid', density: 'comfortable' }; + return { layout: preferences.workspaceLayout || 'grid', density: preferences.workspaceDensity || 'comfortable' }; +} + export function imageKey(image) { return `${image.root_index}:${image.path}`; } diff --git a/web/js/workspace/images.mjs b/web/js/workspace/images.mjs index 1ac497a6..b96dab12 100644 --- a/web/js/workspace/images.mjs +++ b/web/js/workspace/images.mjs @@ -1,5 +1,5 @@ import { createScope, createButton } from '../shared/toolkit/core.js'; -import { createImageStream, withItem, imageKey, countLine, captionDate, scopeLabel, activeRefinements, cardTitle, cardSubtitle, mediaRatio } from './images-state.mjs'; +import { createImageStream, withItem, imageKey, countLine, captionDate, scopeLabel, activeRefinements, cardTitle, cardSubtitle, mediaRatio, initialImagesView } from './images-state.mjs'; import { layoutImages, visibleImages, gridMetrics, stacksCardInfo } from './image-layout.mjs'; import { mountFileInspector } from './file-inspector.mjs'; import { createInspectionController } from './inspection-state.mjs'; @@ -15,6 +15,7 @@ import { openImportDialog, openRefineDialog } from './workspace-dialogs.mjs'; import { openMetadataScan } from './activity-scan.mjs'; import { showWorkspaceToast } from './toast.mjs'; import { imagesFilters, imagesViewState } from './saved-views.mjs'; +import { displayPreferences } from './preferences.mjs'; import { createTagButton, onImageTagsSaved } from './image-tagging.mjs'; import { followImageTagSaves, openCardTagging } from './images-tagging.mjs'; import { favoriteBusy, paintStar, receiveFavorites, setFavoriteVisitTarget, toggleFavorite } from './images-favorites.mjs'; @@ -111,7 +112,9 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage const { resolveUrl } = window.ApiClient; const sender = createPromptSender({scope}); let detailInspector; - const inspection = createInspectionController({api}); + // System › Appearance: the display defaults this visit starts from (preferences.mjs); null without gallery-settings.js. + const preferences = displayPreferences(); + const inspection = createInspectionController({api, autoLoad:preferences?.autoLoadMetadata !== false, cache:preferences?.cacheMetadata !== false}); scope.cleanup(inspection.subscribe(next => detailInspector?.setState(next))); scope.cleanup(() => inspection.dispose()); scope.cleanup(() => detailInspector?.dispose()); @@ -119,9 +122,12 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage const prior = embedded || saved || favorites ? null : lastVisit; // A stale visit keeps its filters, layout and size, and lists again instead of restoring (M6). const visit = prior?.stale ? null : prior; - let density = prior?.density || 'comfortable'; + // The layout and size defaults are the Images page's: an embedded gallery (a prompt history's, its dialogs) has no + // toolbar to change them, so it keeps its fixed Grid/Comfortable. The metadata and card defaults apply everywhere. + const start = initialImagesView({prior, saved, preferences:embedded ? null : preferences}); + let density = start.density; let metrics = {...INITIAL_METRICS}; - let layout = prior?.layout || 'grid', selection = visit?.selection || null, frame = 0, loadFrame = 0, restoreFrame = 0, observer; + let layout = start.layout, selection = visit?.selection || null, frame = 0, loadFrame = 0, restoreFrame = 0, observer; let state, geometry = { positions: [], columns: 1, height: 0, maxHeight: 0 }, galleryWidth = 0, optionsRequest, categoryRequest, recovering = false, rescanning = false; // Discovered sources (each with its own folders) and every folder, for "Refine this view" and the scope chip. let roots = [], folders = []; @@ -287,7 +293,7 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage selection = imageKey(image); for (const video of details.querySelectorAll('video')) releaseVideo(video); detailInspector = mountFileInspector(details, {sender, parentScope:scope, variant:'panel', - onTagFilter:embedded ? undefined : filterByTag, onRetry:() => inspection.retry(), onRefresh:refreshInspectionCollection}); + onTagFilter:embedded ? undefined : filterByTag, onRetry:() => inspection.retry(), onLoad:() => inspection.load(), onRefresh:refreshInspectionCollection}); detailInspector.setState(inspection.state); inspection.select(image); for (const [key, card] of rendered) markSelected(card, key === selection); @@ -331,6 +337,8 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage const text = el('div'); const title = el('h3', cardTitle(image)); title.title = image.path; text.append(title, el('p', cardSubtitle(image))); + // D97: "Show image details on cards" off hides the title and file line; Info and Tag stay. + if (preferences?.showImageInfo === false) text.hidden = true; const actions = el('div', undefined, 'image-card-actions'); const inspectButton = createButton('', 'icon-btn'); inspectButton.append(createWorkspaceIcon('info')); diff --git a/web/js/workspace/inspection-state.mjs b/web/js/workspace/inspection-state.mjs index 290797b9..f9567ed1 100644 --- a/web/js/workspace/inspection-state.mjs +++ b/web/js/workspace/inspection-state.mjs @@ -344,9 +344,16 @@ export function createInspectionLoader(api, onChange, { extractFacts, adapter = }; } -const emptyState = () => ({ loading: false, error: false, record: null, prompts: undefined, message: '', refreshRequired: false, code: null, recovery: null }); +const emptyState = () => ({ loading: false, error: false, deferred: false, record: null, prompts: undefined, message: '', refreshRequired: false, code: null, recovery: null }); -export function createInspectionController({ api, extractFacts, adapter = rootedFileAdapter }) { +/** + * One selection's inspection. #249 System S3 (Appearance): `autoLoad: false` ("Load metadata on selection" off) makes a + * selection publish `deferred` instead of reading, and `load()` reads it once on request; `cache: false` ("Cache + * metadata in this browser" off) re-reads the selected file when it is selected again, where the default reuses the one + * current record. A read in flight is shared either way, and a file whose source changed is never re-read by selecting + * it again (as `retry()` and `refresh()` refuse). + */ +export function createInspectionController({ api, extractFacts, adapter = rootedFileAdapter, autoLoad = true, cache = true }) { let state = Object.freeze(emptyState()), current = null, pending = null, disposed = false; let attempted, hasAttempt = false; const listeners = new Set(); @@ -369,13 +376,19 @@ export function createInspectionController({ api, extractFacts, adapter = rooted try { adapter.validate(image); const scope = adapter.scope(image), key = adapter.key(image), path = adapter.inspectionPath(image); - if (current?.key === key && current.scope === scope && current.path === path) return pending; + if (current?.key === key && current.scope === scope && current.path === path && (cache || state.loading || state.deferred || state.refreshRequired)) return pending; selected = { key, scope, path }; } catch { // The loader publishes validation errors after invalidating the previous source. } loader.dispose(); loader = makeLoader(); current = selected; attempted = object(image) ? { ...image, ...(adapter !== rootedFileAdapter ? {occurrence:{...image.occurrence}, reference:{...image.reference}} : {}) } : image; hasAttempt = true; + if (!autoLoad && selected) { pending = null; publish({ deferred: true }); return undefined; } + return read(attempted); + }, + /** Read a deferred selection (the inspector's "Load metadata"); a read already made or running is reused. */ + load() { + if (disposed || !hasAttempt || !state.deferred) return pending ?? undefined; return read(attempted); }, retry() { @@ -384,7 +397,8 @@ export function createInspectionController({ api, extractFacts, adapter = rooted }, /** Re-read the selected file keeping the shown record until the answer (a retry when none is shown). */ refresh() { - if (disposed || !hasAttempt || state.refreshRequired) return; + // A deferred selection loads only through `load()` (the user's "Load metadata"). + if (disposed || !hasAttempt || state.refreshRequired || state.deferred) return; return read(attempted, { quiet: Boolean(state.record) && !state.loading && !state.error }); }, clear() { diff --git a/web/js/workspace/media-viewer.mjs b/web/js/workspace/media-viewer.mjs index bd1b3233..834c9957 100644 --- a/web/js/workspace/media-viewer.mjs +++ b/web/js/workspace/media-viewer.mjs @@ -2,14 +2,14 @@ import {createScope, createButton} from '../shared/toolkit/core.js'; import {createViewerSession} from './viewer-state.mjs'; import {rootedFileAdapter} from './inspection-state.mjs'; import {mountFileInspector} from './file-inspector.mjs'; +import {displayPreferences, readPreferences} from './preferences.mjs'; let viewerId = 0; // Upper bound on waiting for a closed Viewer's in-flight image view before destroying it. const PENDING_VIEW_DESTROY_MS = 10000; +/** The video defaults, read through the shared display defaults (preferences.mjs, #249 System S3). */ export function readViewerPreferences(getStorage = () => window.localStorage, normalize = globalThis.GallerySettings.normalizeGallerySettings) { - let stored; - try { stored = JSON.parse(getStorage().getItem('gallerySettings')); } catch { stored = null; } - const {videoAutoplay,videoMute,videoLoop} = normalize(stored); + const {videoAutoplay,videoMute,videoLoop} = readPreferences(getStorage,normalize).values; return {videoAutoplay,videoMute,videoLoop}; } function element(tag, className, text) { @@ -74,15 +74,31 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle local.listen(download,'click',event => {session.stop();if (download.getAttribute('aria-disabled') === 'true') event.preventDefault();}); controls.append(download); const next = button('Next media','Next →',() => session.navigate(1)); + // D97: "Collapse viewer details by default" opens the viewer with its details hidden behind one toggle. + const collapsed = displayPreferences()?.sidebarCollapsedByDefault === true; + let detailsToggle = null; + if (collapsed) { + detailsToggle = createButton('Show details');detailsToggle.setAttribute('aria-expanded','false'); + detailsToggle.setAttribute('aria-controls','viewer-details');controls.append(detailsToggle); + } const position = element('p','media-position muted');position.id = `media-viewer-position-${++viewerId}`; position.setAttribute('role','status'); const reason = element('p','media-reason muted');reason.id = `media-viewer-reason-${viewerId}`; const slideshowReason = element('p','media-slideshow-reason muted');slideshowReason.id = `media-viewer-slideshow-${viewerId}`; controls.append(position,reason,slideshowReason);left.append(stage,controls); const details = element('aside');details.id = 'viewer-details';details.setAttribute('aria-label','Image metadata'); + if (detailsToggle) { + details.hidden = true; + local.listen(detailsToggle,'click',() => { + details.hidden = !details.hidden; + detailsToggle.textContent = details.hidden ? 'Show details' : 'Hide details'; + detailsToggle.setAttribute('aria-expanded',String(!details.hidden)); + }); + } body.append(left,details);dialog.append(header,body);container.append(dialog); const localSender = {open(roles,trigger) {session.stop();sender.open(roles,trigger);}}; const inspector = mountFileInspector(details,{api,sender:localSender,parentScope:local,variant:'full',onInteraction:() => session.stop(),onRetry:() => inspectionController.retry(), + onLoad:() => inspectionController.load?.(), onRefresh:onRefresh ? () => { closeViewer();onRefresh(); } : undefined}); function closeViewer(restore = true) { if (closed) return; diff --git a/web/js/workspace/preferences.mjs b/web/js/workspace/preferences.mjs new file mode 100644 index 00000000..c964d846 --- /dev/null +++ b/web/js/workspace/preferences.mjs @@ -0,0 +1,112 @@ +/** + * Browser display defaults (#249 System S3 Task 7). The workspace keeps them in the legacy gallery's own + * `gallerySettings` object (Q3, "One display configuration"), so the gallery, the workspace and the viewer read one + * configuration: the new `workspaceLayout` and `workspaceDensity` beside the legacy keys they share (metadata, paths, + * card details, viewer details, image quality and the video defaults). The legacy paging keys (`defaultLimit`, + * `gridColumns`, `infiniteScroll`, …) belong to the legacy gallery and are never read here (README continuous-browsing + * ruling). + * + * - `readPreferences(getStorage, normalize)` → `{values, persisted}`: the workspace keys, normalized by `gallery-settings.js`; + * `persisted` is false when this browser refused its storage. + * - `writePreferences(dirty, getStorage, {baseline})` → `{persisted, conflicts}`: re-reads the stored object and writes + * back only the dirty keys, every other key byte-for-byte as stored. A write the browser refused (denied storage, a + * full quota) leaves the stored bytes as they were and keeps the change for this tab only, so every reader in this + * tab still sees it; the next write storage accepts stores those changes too. `conflicts` lists the dirty keys another page changed since `baseline` was read; the explicit + * save still wins. + * - `subscribePreferences(win, onChange, getStorage)` → unsubscribe: `onChange(readPreferences())` after another page + * changed the settings (a `storage` event). + * - `reconcilePreferences({baseline, drafts, next})` → `{conflicts}`: which drafts such a change overtook. + * + * Every storage access is in try/catch: a private window, blocked site data or a malformed value never breaks a page. + */ +export const PREFERENCES_KEY = 'gallerySettings'; +export const WORKSPACE_PREFERENCE_KEYS = Object.freeze(['workspaceLayout', 'workspaceDensity', 'autoLoadMetadata', 'cacheMetadata', + 'imageQuality', 'videoAutoplay', 'videoMute', 'videoLoop', 'showFilePaths', 'showImageInfo', 'sidebarCollapsedByDefault']); + +// The changes this tab could not store, overlaid on every read until storage accepts them. +const tabOnly = new Map(); +const defaultStorage = () => globalThis.localStorage; +const normalizer = () => globalThis.GallerySettings.normalizeGallerySettings; +const isObject = value => value !== null && typeof value === 'object' && !Array.isArray(value); + +/** Forget what this tab kept without storage (tests, and nothing else). */ +export function resetTabPreferences() { + tabOnly.clear(); +} + +const pick = settings => Object.fromEntries(WORKSPACE_PREFERENCE_KEYS.map(key => [key, settings[key]])); +/** The gallery's defaults for the workspace keys. */ +export function preferenceDefaults() { + return pick(normalizer()(null)); +} + +/** The stored object (`{}` when absent or malformed), or null when storage is unavailable. */ +function storedObject(storage) { + let text; + try { text = storage.getItem(PREFERENCES_KEY); } catch { return null; } + try { + const parsed = text ? JSON.parse(text) : null; + return isObject(parsed) ? parsed : {}; + } catch { + return {}; + } +} + +function storageOf(getStorage) { + try { return getStorage() ?? null; } catch { return null; } +} + +/** `normalize` is gallery-settings.js's `normalizeGallerySettings` (the page's `window.GallerySettings` by default). */ +export function readPreferences(getStorage = defaultStorage, normalize = normalizer()) { + const storage = storageOf(getStorage); + const stored = storage ? storedObject(storage) : null; + const values = pick(normalize({ ...(stored ?? {}), ...Object.fromEntries(tabOnly) })); + return { values, persisted: stored !== null }; +} + +/** A change is only a workspace key with a value the gallery's normalizer keeps as given. */ +function validate(dirty) { + if (!isObject(dirty)) throw new TypeError('Preferences need an object of changes'); + for (const [key, value] of Object.entries(dirty)) { + if (!WORKSPACE_PREFERENCE_KEYS.includes(key)) throw new TypeError(`Not a workspace display preference: ${key}`); + if (normalizer()({ [key]: value })[key] !== value) throw new TypeError(`Invalid value for ${key}`); + } +} + +export function writePreferences(dirty, getStorage = defaultStorage, { baseline = null } = {}) { + validate(dirty); + const keep = () => { for (const [key, value] of Object.entries(dirty)) tabOnly.set(key, value); }; + const storage = storageOf(getStorage); + const stored = storage ? storedObject(storage) : null; + if (stored === null) { keep(); return { persisted: false, conflicts: [] }; } + const current = pick(normalizer()(stored)); + const conflicts = baseline ? Object.keys(dirty).filter(key => Object.hasOwn(baseline, key) && current[key] !== baseline[key]) : []; + try { + // The changes this tab could not store before are stored with this one, so "Saved" is true for all of them. + storage.setItem(PREFERENCES_KEY, JSON.stringify({ ...stored, ...Object.fromEntries(tabOnly), ...dirty })); + } catch { + keep(); + return { persisted: false, conflicts }; + } + tabOnly.clear(); + return { persisted: true, conflicts }; +} + +/** The current values for a display consumer, or null on a page without gallery-settings.js (a unit test). */ +export function displayPreferences(getStorage = defaultStorage) { + try { return readPreferences(getStorage).values; } catch { return null; } +} + +export function subscribePreferences(win, onChange, getStorage = defaultStorage) { + const listener = event => { + if (event.key !== null && event.key !== PREFERENCES_KEY) return; + if (event.storageArea && event.storageArea !== storageOf(getStorage)) return; + onChange(readPreferences(getStorage)); + }; + win.addEventListener('storage', listener); + return () => win.removeEventListener('storage', listener); +} + +export function reconcilePreferences({ baseline, drafts, next }) { + return { conflicts: Object.keys(drafts).filter(key => next[key] !== baseline[key] && next[key] !== drafts[key]) }; +} diff --git a/web/js/workspace/system-appearance.mjs b/web/js/workspace/system-appearance.mjs new file mode 100644 index 00000000..290bf778 --- /dev/null +++ b/web/js/workspace/system-appearance.mjs @@ -0,0 +1,143 @@ +/** + * System › Appearance (#249 System S3 Task 8): the concept's `renderSystem` "appearance" body (app.js 676–678), one + * `section.surface` "Browsing & display" of `.setting-row`s whose values are the drafts of the one Save row. + * + * - The browser rows are the display defaults this browser keeps in the shared `gallerySettings` object + * (preferences.mjs, Q3): Default layout and Thumbnail size (`workspaceLayout`, `workspaceDensity`), the metadata, + * quality (Optimized→`medium`, Original→`high`, Fast→`low`), video and path rows. Images, the inspector and the + * viewer read them on their next visit; a toolbar change still applies to the current view only. + * - "Open from ComfyUI" (New tab→`newtab`, Popup window→`popup`) and "Node search auto-hide (seconds)" (a whole number + * 0–300) are the server's `webui_display_mode` and `result_timeout`, saved with the settings revision (#220). + * - "Continuous browsing" is the concept's fixed row: there is no paging or column control (README ruling). + * - D97: added rows for the legacy gallery settings the concept leaves out: "Show image details on cards" + * (`showImageInfo`), "Collapse viewer details by default" (`sidebarCollapsedByDefault`) and the operation "Reset + * display defaults", which drafts the gallery's defaults for these browser rows and touches no other key. + * + * Controls follow the store: a value read later (the host settings, another tab's change) repaints them. A host row + * whose value could not be read stays disabled and says so ("Server settings could not be read."). + */ +import { settingRow, operationRow } from './system-controls.mjs'; +import { preferenceDefaults } from './preferences.mjs'; + +export const RESULT_TIMEOUT_MAX = 300; +/** A host row's line while the server settings could not be read (its control stays disabled). */ +export const HOST_UNAVAILABLE = 'Server settings could not be read.'; +const BOOLEAN = Object.freeze({ type: 'toggle' }); + +/** + * The concept's rows in its order, then the D97 rows: `[id, label, description, domain, key, control]`. A select's + * options are `[stored value, concept label]`. + */ +export const APPEARANCE_ROWS = Object.freeze([ + ['default-view', 'Default layout', 'Choose your starting image layout.', 'browser', 'workspaceLayout', + { type: 'select', options: [['grid', 'Grid'], ['list', 'List'], ['masonry', 'Masonry']] }], + ['density', 'Thumbnail size', 'Readable previews by default; no fixed column-count setting.', 'browser', 'workspaceDensity', + { type: 'select', options: [['comfortable', 'Comfortable'], ['large', 'Large'], ['compact', 'Compact']] }], + 'continuous', + ['metadata-auto', 'Load metadata on selection', 'Use the same inspector in the gallery and image viewer.', 'browser', 'autoLoadMetadata', BOOLEAN], + ['metadata-cache', 'Cache metadata in this browser', 'Reuse fetched metadata until it changes or you clear this cache.', 'browser', 'cacheMetadata', BOOLEAN], + ['image-quality', 'Image loading quality', 'Preserve original / optimized / fast image-loading preferences.', 'browser', 'imageQuality', + { type: 'select', options: [['medium', 'Optimized'], ['high', 'Original'], ['low', 'Fast']] }], + ['video-auto', 'Autoplay videos', 'Videos stay within the same gallery and use native playback controls.', 'browser', 'videoAutoplay', BOOLEAN], + ['video-muted', 'Mute videos by default', 'Keep playback quiet while browsing.', 'browser', 'videoMute', BOOLEAN], + ['video-loop', 'Loop videos', 'Repeat a selected clip until you navigate away.', 'browser', 'videoLoop', BOOLEAN], + ['paths', 'Show file paths', 'Paths appear under File & provenance, not across every card.', 'browser', 'showFilePaths', BOOLEAN], + ['open-mode', 'Open from ComfyUI', 'Keep the node pack’s popup / new-tab preference.', 'host', 'webui_display_mode', + { type: 'select', options: [['newtab', 'New tab'], ['popup', 'Popup window']] }], + ['timeout', 'Node search auto-hide (seconds)', 'Node search behavior only. Use 0 to keep results visible.', 'host', 'result_timeout', + { type: 'number' }], + // D97: the legacy gallery's card and viewer settings. + ['card-details', 'Show image details on cards', 'Show each card’s title and file details under the preview.', 'browser', 'showImageInfo', BOOLEAN], + ['viewer-details', 'Collapse viewer details by default', 'Open the image viewer with its details panel hidden.', 'browser', + 'sidebarCollapsedByDefault', BOOLEAN], +]); + +/** The browser keys "Reset display defaults" restores (D97). */ +export const DISPLAY_KEYS = Object.freeze(APPEARANCE_ROWS.filter(row => Array.isArray(row) && row[3] === 'browser').map(row => row[4])); + +/** A whole number of seconds from 0 to 300 (the server's `validate_result_timeout`), else null. */ +export function resultTimeout(text) { + const value = String(text).trim(); + if (!/^\d+$/.test(value)) return null; + const seconds = Number(value); + return seconds <= RESULT_TIMEOUT_MAX ? seconds : null; +} + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +/** The concept's fixed "Continuous browsing" row: `div > h3 + p` and the "Infinite scroll" chip. */ +function continuousRow() { + const row = el('div', undefined, 'setting-row'); + const text = el('div'); + text.append(el('h3', 'Continuous browsing'), + el('p', 'Images load automatically as you scroll in Grid, List and Masonry. Batch sizing is handled internally.')); + row.append(text, el('span', 'Infinite scroll', 'chip blue')); + return row; +} + +/** The row's control (`data-setting`), wherever the Toolkit switch nests it. */ +function controlOf(node, id) { + if (node.dataset?.setting === id) return node; + for (const child of node.children ?? []) { + const found = controlOf(child, id); + if (found) return found; + } + return null; +} + +/** Show a stored value in its control; a value not read yet leaves the control disabled. */ +function paint(input, control, value) { + input.disabled = value === undefined; + if (value === undefined) return; + if (control.type === 'toggle') input.checked = value === true; + else if (document.activeElement !== input || control.type === 'select') input.value = String(value); +} + +/** Mount Appearance in `container` (System's `.settings-layout`); returns the section's nodes. */ +export function mountAppearance(container, { scope, drafts }) { + const surface = el('section', undefined, 'surface'); + surface.append(el('h2', 'Browsing & display'), el('p', 'One set of defaults; toolbar changes apply to the current view.')); + const controls = []; + for (const row of APPEARANCE_ROWS) { + if (row === 'continuous') { surface.append(continuousRow()); continue; } + const [id, label, description, domain, key, control] = row; + const onChange = value => { + if (key === 'result_timeout') { + const seconds = resultTimeout(input.value); + if (seconds === null) { drafts.reject(domain, key); return; } + value = seconds; + } + drafts.set(domain, key, value); + }; + const node = settingRow({ id, label, description, control, scope, onChange }); + surface.append(node); + const input = controlOf(node, id); + input.dataset.key = key; + controls.push({ input, control, domain, key, text: node.children[0], note: null }); + } + surface.append(operationRow({ title: 'Reset display defaults', + description: 'Restore this browser’s display defaults, then save. Other settings are kept.', label: 'Reset', scope, + onClick: () => { + const defaults = preferenceDefaults(); + for (const key of DISPLAY_KEYS) drafts.set('browser', key, defaults[key]); + } })); + const repaint = () => { + const unavailable = drafts.hostState?.() === 'failed'; + for (const row of controls) { + const value = drafts.get(row.domain, row.key); + paint(row.input, row.control, value); + if (row.domain !== 'host') continue; + if (unavailable && value === undefined) { + if (!row.note) { row.note = el('p', HOST_UNAVAILABLE, 'small'); row.text.append(row.note); } + } else if (row.note) { row.note.remove(); row.note = null; } + } + }; + scope.cleanup(drafts.subscribe(repaint)); + repaint(); + return [surface]; +} diff --git a/web/js/workspace/system-state.mjs b/web/js/workspace/system-state.mjs index 2c24e67e..b827dc3e 100644 --- a/web/js/workspace/system-state.mjs +++ b/web/js/workspace/system-state.mjs @@ -15,8 +15,9 @@ * the page session (Q10a): a section left and opened again shows them, and a reload of its values keeps them. * * Adapters: `readBrowser()` and `readRuntime()` return the values; `readHost()` returns `{values, revision}`. - * `writeBrowser(changes)` returns `{persisted}` (false: kept for this tab only), `writeHost(changes, - * {expected_revision})` the settings response (`{success, persisted, revision}`), `writeRuntime(changes)` anything; + * `writeBrowser(changes)` returns `{persisted}` (false: kept for this tab only) and throws an error with `code: + * 'settings-changed'` when another page changed a drafted value (a conflict: the browser values are read again), + * `writeHost(changes, {expected_revision})` the settings response (`{success, persisted, revision}`), `writeRuntime(changes)` anything; * a write that throws failed. */ export { SYSTEM_SECTIONS, isSystemSection } from './state.mjs'; @@ -64,6 +65,7 @@ export function createSystemDrafts(adapters) { const failedKeys = { browser: new Set(), host: new Set(), runtime: new Set() }; const listeners = new Set(); let revision = null; + let hostRead = 'pending'; let outcome = null; let saving = null; @@ -78,6 +80,7 @@ export function createSystemDrafts(adapters) { const read = await adapters.readHost(); values = read?.values ?? {}; revision = read?.revision ?? null; + hostRead = 'loaded'; } else { values = await (domain === 'browser' ? adapters.readBrowser() : adapters.readRuntime()); } @@ -159,8 +162,11 @@ export function createSystemDrafts(adapters) { } } - /** Read the host values again; a failed read leaves host saves refused until one succeeds. */ - const reloadHost = () => load('host').then(() => true, () => false); + /** + * Read the host values again; a failed read leaves host saves refused until one succeeds, and `hostState()` says + * so (System S3: the host rows show that the server settings could not be read). + */ + const reloadHost = () => load('host').then(() => true, () => { hostRead = 'failed'; notify(); return false; }); /** Save the dirty domains in order; a Save pressed while one runs joins it. */ function save() { @@ -170,6 +176,7 @@ export function createSystemDrafts(adapters) { for (const domain of DRAFT_DOMAINS) result[domain] = await saveDomain(domain); // Settings changed elsewhere: read them (and their revision), keeping the drafts (Q10). if (result.host === 'conflict') await reloadHost(); + if (result.browser === 'conflict') await load('browser').catch(() => {}); return result; })().then(result => { saving = null; @@ -180,9 +187,39 @@ export function createSystemDrafts(adapters) { return saving; } + /** + * A value the control could not take (System S3: a timeout outside 0–300): no draft, the control is highlighted and + * the status says a change was not saved. + */ + function reject(domain, key) { + requireDomain(domain); + drafts[domain].delete(key); + failedKeys[domain].add(key); + outcome = { [domain]: 'failed' }; + notify(); + } + + /** + * Another page changed saved values this tab has drafts for (a browser `storage` event, System S3): the drafts stay + * and are highlighted, and the status says the settings changed elsewhere. + */ + function conflict(domain, keys) { + requireDomain(domain); + if (!keys.length) return; + for (const key of keys) failedKeys[domain].add(key); + outcome = { [domain]: 'conflict' }; + notify(); + } + return { load, reloadHost, + /** Whether the host values were read: `pending`, `loaded`, or `failed` (the last read failed). */ + hostState: () => hostRead, + reject, + conflict, + /** The saved values of a domain (its last read or save), not the drafts. */ + saved: domain => { requireDomain(domain); return { ...baseline[domain] }; }, get, set, dirty, diff --git a/web/js/workspace/system.mjs b/web/js/workspace/system.mjs index 0fa76585..27e1a54a 100644 --- a/web/js/workspace/system.mjs +++ b/web/js/workspace/system.mjs @@ -4,7 +4,8 @@ * `aria-pressed`; a press navigates to its hash), then `.settings-layout` with the section's body and the Save row * (`span.small#settings-status` + "Save preferences"). D92: no "Sample configuration" chip, the status copy is * truthful and "Save preferences" is aria-disabled while nothing is unsaved. A section not delivered yet shows the - * per-section hand-off to its legacy tool (D94), never mock placeholders. Overview is native (system-overview.mjs). + * per-section hand-off to its legacy tool (D94), never mock placeholders. Overview (system-overview.mjs) and + * Appearance (system-appearance.mjs) are native. * * Switching sections keeps the shell (and focus on the pressed button) and disposes the old section's scope, so its * listeners and timers never outlive it. The drafts are module-level (Q10): they live for the page session. @@ -14,6 +15,8 @@ import { SYSTEM_SECTIONS, navigateTo } from './state.mjs'; import { createSystemDrafts, DRAFT_DOMAINS } from './system-state.mjs'; import { sectionNav } from './system-controls.mjs'; import { healthNotice, mountOverview } from './system-overview.mjs'; +import { mountAppearance } from './system-appearance.mjs'; +import { displayPreferences, readPreferences, reconcilePreferences, subscribePreferences, writePreferences } from './preferences.mjs'; const SETTINGS_URL = '/prompt_manager/settings'; const ADMIN = Object.freeze({ href: '/prompt_manager/admin', label: 'Open existing administration' }); @@ -24,7 +27,6 @@ const GALLERY = Object.freeze({ href: '/prompt_manager/gallery', label: 'Open ex * settings, thumbnails, duplicates and cache tools. */ export const SECTION_HANDOFFS = Object.freeze({ - appearance: [GALLERY, ADMIN], sources: [ADMIN, GALLERY], integrations: [ADMIN], tagging: [ADMIN], @@ -46,12 +48,23 @@ function el(tag, text, className) { /** A domain whose settings no delivered section edits yet: nothing can be drafted, and a write would say it failed. */ const undelivered = async () => { throw new Error(NOT_DELIVERED); }; -/** The page session's drafts (Q10); the host domain reads and saves the server settings with their revision (#220). */ +/** + * The page session's drafts (Q10). The browser domain is this browser's display defaults (preferences.mjs): a write + * says whether storage kept it, so the status never claims "Saved in this browser." for a change only this tab holds. + * The host domain reads and saves the server settings with their revision (#220). + */ export function systemDrafts(api) { drafts ??= createSystemDrafts({ - readBrowser: () => ({}), + readBrowser: () => readPreferences().values, readRuntime: () => ({}), - writeBrowser: undelivered, + writeBrowser: changes => { + // Another page changed a drafted value since this tab read it, and no storage event told it yet: refuse, + // so the store keeps the draft, flags it and reads the values again; the next Save is an explicit choice. + const baseline = drafts.saved('browser'); + const { conflicts } = reconcilePreferences({ baseline, drafts: changes, next: readPreferences().values }); + if (conflicts.length) throw Object.assign(new Error('Settings changed elsewhere'), { code: 'settings-changed' }); + return { persisted: writePreferences(changes, undefined, { baseline }).persisted }; + }, writeRuntime: undelivered, async readHost() { const result = await api.get(SETTINGS_URL, { cache: 'no-store' }); @@ -80,17 +93,19 @@ function handoff(section) { function markFailed(layout, store) { const failed = new Set(DRAFT_DOMAINS.flatMap(domain => [...store.failed(domain)])); for (const control of layout.querySelectorAll('[data-setting]')) { - if (failed.has(control.dataset.setting)) control.setAttribute('aria-invalid', 'true'); + // A row's control names its stored key when that differs from the row id (Appearance's `data-key`). + if (failed.has(control.dataset.key ?? control.dataset.setting)) control.setAttribute('aria-invalid', 'true'); else control.removeAttribute('aria-invalid'); } } /** - * The sections delivered so far: Overview (S2 Task 6), and Maintenance's notice with "Run health checks" above the + * The sections delivered so far: Overview (S2 Task 6), Appearance (S3 Task 8), and Maintenance's notice with "Run health checks" above the * hand-off to the maintenance tools that are not native yet (D123). */ export const DELIVERED_SECTIONS = Object.freeze({ overview: mountOverview, + appearance: mountAppearance, maintenance: (container, { api, activity, scope }) => [healthNotice({ api, activity, scope }), ...handoff('maintenance')], }); @@ -101,11 +116,25 @@ export const DELIVERED_SECTIONS = Object.freeze({ * Activity store, which follows the jobs a section starts. * * The host values are read on mount (and by the store again after a conflict), so a save carries the current revision; - * until a read returned one, a host save fails without a request (system-state.mjs). + * until a read returned one, a host save fails without a request (system-state.mjs). The browser values are read on + * mount and again whenever another page changes them (a `storage` event). Either way a value another page changed + * under one of this tab's drafts flags that draft (D92's conflict status), including a change made while System was + * not mounted (the drafts outlive it, Q10). */ export function mountSystem(main, api, scope, { section, activity, sections = DELIVERED_SECTIONS, navigate = navigateTo }) { const store = systemDrafts(api); store.reloadHost(); + // Another tab or the legacy gallery changed the display defaults: clean values follow it; a draft it overtook stays + // and is flagged (D92's conflict status). `next` is null when the values cannot be read (no gallery-settings.js). + const reconcile = next => { + const baseline = store.saved('browser'); + // The first mount has read nothing yet, so nothing can have been overtaken. + const conflicts = next && Object.keys(baseline).length + ? reconcilePreferences({ baseline, drafts: store.changes('browser'), next }).conflicts : []; + store.load('browser').then(() => store.conflict('browser', conflicts), () => {}); + }; + reconcile(displayPreferences()); + scope.cleanup(subscribePreferences(window, next => reconcile(next.values))); const nav = sectionNav({ label: 'System sections', sections: SYSTEM_SECTIONS, active: section, scope, onSelect: id => navigate(`#/system/${id}`) }); const layout = el('div', undefined, 'settings-layout'); diff --git a/web/js/workspace/workspace.css b/web/js/workspace/workspace.css index c7088b31..ace96b00 100644 --- a/web/js/workspace/workspace.css +++ b/web/js/workspace/workspace.css @@ -91,6 +91,10 @@ main:focus-visible { outline: 2px solid var(--ca-focus); outline-offset: 4px; } .setting-row [aria-invalid="true"] { outline: 2px solid var(--red); outline-offset: 2px; /* D92 */ } /* D93: an Overview row whose action is not available yet (the review queue, Q6) keeps its button, disabled. */ .operation .row-actions button[aria-disabled="true"] { opacity: .55; cursor: not-allowed; /* D93 */ } +/* D97: Appearance's legacy display rows. "Show image details on cards" off keeps a card's Info and Tag at the end of + its row; "Collapse viewer details by default" gives the viewer's stage the whole width while its details are hidden. */ +.media-info > div[hidden] + .image-card-actions { margin-inline-start: auto; /* D97 */ } +.viewer-layout:has(> #viewer-details[hidden]) { grid-template-columns: minmax(0, 1fr); /* D97 */ } @media (max-width: 700px) { .stat-grid { gap: 8px; } .stat { padding: 16px; }