diff --git a/TESTING.md b/TESTING.md index e73c6ebd..7d6543d2 100644 --- a/TESTING.md +++ b/TESTING.md @@ -1854,6 +1854,214 @@ npm run --prefix tools/vendor test:workflows -- inspect-concept.spec.mjs workspa npm run --prefix tools/vendor test:comfyui -- inspect-sender.spec.mjs ``` +### Native System (#249 System) + +`#/system/
` is a native workspace route rendered by `web/js/workspace/system.mjs` +on the concept's `renderSystem` markup (app.js a698–706; `concept.css` 709–895). `#/system` +is Overview under the sidebar's own hash; `#/system/overview|appearance|sources| +integrations|tagging|maintenance|backups|diagnostics` select a section, and an unknown +child (the concept's `#/system/general`) is replaced by `#/system/overview`. The +production hub `#/overview` is unchanged (D28); its System card opens +`#/system/overview` (Q12). Switching sections keeps the shell, the drafts and focus on +the pressed section button and disposes the previous section's scope, so its reads, +listeners and dialogs never outlive it. + +Drafts (`system-state.mjs`) live for the page session (Q10). Save writes the dirty +browser fields, then the host fields (`POST /prompt_manager/settings` with +`expected_revision`, #220), then the runtime fields. A failed domain keeps its drafts and +highlights their controls; `settings-changed` is a conflict and keeps the draft. System +reads the host values on mount and again after a conflict; until a read returned a +revision, a host save fails without a request, so it never sends a blind +`expected_revision: null` (which S1 merges into the file on disk). Save stays disabled +until a section has an unsaved change. + +Appearance (`system-appearance.mjs`, S3) is the concept's "Browsing & display" surface +on real values. Its browser rows are this browser's display defaults in the legacy +gallery's own `gallerySettings` object (`web/js/workspace/preferences.mjs`, Q3): Default +layout and Thumbnail size (`workspaceLayout` grid/list/masonry, `workspaceDensity` +comfortable/large/compact, new keys), Load metadata on selection (`autoLoadMetadata`), +Cache metadata in this browser (`cacheMetadata`), Image loading quality (`imageQuality`: +Optimized→`medium`, Original→`high`, Fast→`low`; the thumbnail dialog's preselected quality +once S5 lands), the three video rows (`videoAutoplay`, `videoMute`, `videoLoop`) and Show +file paths (`showFilePaths`). A write re-reads the stored object and writes back only the +changed keys, every other key (legacy paging, thumbnail state, unknown keys) as stored; +the legacy gallery's Save also keeps the keys it does not edit, so the two pages stay in +sync. The workspace never reads the legacy paging keys (`defaultLimit`, `gridColumns`, +`infiniteScroll`; README continuous-browsing ruling). A browser that refuses storage +(denied, or a full quota that leaves the stored bytes as they were) keeps the change for +this tab only and the status says "Saved for this tab only; browser storage is +unavailable.", never "Saved in this browser.". Another tab's change (a `storage` event) +repaints the clean controls; a draft it overtook stays, highlighted, with the conflict +status. Open from ComfyUI (New tab→`newtab`, Popup window→`popup`) and Node search +auto-hide (seconds) (a whole number 0–300; anything else is highlighted and never sent) +are the server's `webui_display_mode` and `result_timeout`, saved through +`POST /prompt_manager/settings` with the revision. The consumers read the defaults on +their next visit, never the current view: a new Images visit starts from the layout and +size (the last visit keeps its own, a saved view opens at Grid and Comfortable as +before); with metadata off a selection reads nothing until "Load metadata"; with the +cache off selecting the shown file again re-reads it (on, the one current record is +reused); paths off leaves out only the Path row of File & provenance; the viewer reads +the video defaults. + +Overview (`system-overview.mjs`) reads `GET /prompt_manager/system/overview` once per +visit through `latestOnly()` on the section scope; a late answer after the section left +paints nothing. Until it answers, the section shows "Loading the overview…"; a failure +shows "The overview could not be loaded. Your saved data has not changed." and Retry, +never zeros. The thumbnail row is a separate read of `/images/thumbnails/status` (a +walk of every source), so the other values never wait for it: "Checking thumbnails…" +until it answers. The health check (`POST /prompt_manager/system/health`) is a `read` +registry job of kind `health`: one at a time, beside a scan, not cancellable and with +no file outcomes. The page tracks the started job in the Activity store (its event +stream; the store marks it lost if the server stops listing it), never by polling. A press while one runs (another tab) waits +for that one. The last report lives for this server session (`GET +/prompt_manager/system/health`) and describes the database it was taken from: a +restore that replaces the database forgets it, and a check that started before the +swap is not kept. The check's own connection opens a `mode=ro` URI that keeps a Windows +UNC path (`file:////server/share/prompts.db`; pathlib's `file://server/share/...` is a +remote authority SQLite refuses). A check that raises fails only itself, and the +missing-files check reads its rows inside the snapshot but stats the files after it +ended, so a slow source never pins the WAL. Statistics +read `GET /prompt_manager/system/statistics` (`by_category`, the legacy `/stats` totals +and `rated_images`). + +Read-only standard for a check or preview (S2's health check; S6 and S8 previews use +the same): the database and `-wal` bytes are unchanged, the `-shm` size is unchanged, +the connection's `total_changes` is 0 at close and the data reads back unchanged. The +`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no +reader can keep it byte-identical (`test_health_check_never_writes`). + +Recorded deviations (plan D60–D65 renumbered D92–D97; D121, D122, D123 and D124 assigned): + +- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No + unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server + configuration." / "Applied until the server restarts.", "Some changes were not saved. + Review the highlighted settings.", "Saved for this tab only; browser storage is + unavailable." and "Settings changed elsewhere. Reload this section to see them." + "Save preferences" is `aria-disabled` (opacity .55, `not-allowed`) while clean or + saving; a setting the last save could not save gets a red outline. CSS under D92. +- **D93** (Overview): real values. The metrics are the Library's eligible positives + and active saved negatives and the indexed `generated_images` rows (Q2). "Database + integrity" is "Not checked" until a check ran this session, then "N issues" (checks + that found issues, even beside a check that could not run), "Passed" or "Incomplete". + "Ambiguous image links" reads "Not available yet." with its Review metadata action + `aria-disabled` (CSS under D93) until the Needs review queue reports a count (Q6). + "Backup status" reads "No retained backup record." until verified backups exist (S8). + The thumbnail row is "Checking thumbnails…" until its own read answers, then "No + missing thumbnails", "N missing thumbnails" (the pending count of + `/images/thumbnails/status`) or "Thumbnail status unavailable"; Review thumbnails + opens Sources until the thumbnail dialog (S5). "One display configuration" says "Grid + and density settings are shared across views." (no paging). The sources row is + "N output sources" with "Monitoring and enrichment share one source list.", prefixed + by "M of N configured sources available." or "None configured is available; ComfyUI's + output folder is used." when that is the case. +- **D94** (all sections): a section not delivered yet shows the hand-off panel "Your + current tools are ready" with a link to each legacy tool that still owns it (the + plan's legacy coverage table); retired section by section, gone after S12. +- **D95** (health report): the concept's `health-check` dialog titled "Health report", + with no sample note. It lists every check in the server's order: Integrity check, + Foreign keys & pair references, Canonical hashes, Pair uniqueness (added), Missing + files, Ambiguous metadata, Consistency (added). Missing files has one row per image + source when there are several ("Missing files · output"). Each chip is the real + result: "Passed", a count with what was found ("2 broken references", "1 link needs + review", the concept's "N links need review" and "N images need review"; "100+ + problems found" when SQLite's integrity check stopped at its limit), "Not + available" (an unavailable source), "Not available yet" (Ambiguous metadata, Q6) or + "Could not check". A `p.small` after the list says when it was checked and whether + every check passed, some could not run, or issues need review. Close / Review missing + links (to Maintenance, its hand-off until S7). Nothing is changed by a check. +- **D96** (Statistics): "Usage statistics" without the sample note: the concept's + "Runs by category" table (Category / Runs / Avg. rating) with the real runs and + average rating of eligible parents per category ("Uncategorized", "Not rated"), + then a second table "Library totals" (Total / Count: Prompts, Tags, Images from the + legacy statistics, and Rated images: indexed images whose linked prompt is rated). + The Overview row says "Runs and ratings by category, with library totals." because + that is what the dialog shows (the concept's own dialog shows only "Runs by + category"); per-prompt run counts, prompt tags and dated runs are recorded, but no + table of them exists yet. A failed read toasts "Statistics could not + be loaded. Your saved data has not changed." +- **D121** (production hub): the Overview hub's System card reads the concept's System + heading and "Open System"; the concept has no hub card. +- **D122** (shell): the section nav's `aria-label` is "System sections"; the concept's + literal label is its prototype action name "system-tab sections". +- **D123** (Maintenance): the concept's Maintenance notice + ("Check first. Review changes. Then apply." + Run health checks) is delivered ahead of + the rest of Maintenance, above its D94 hand-off, as the native place that runs the + health check. A press shows "Checking…" (`aria-disabled`) and opens the report; a + failed check toasts "The health check could not finish. Nothing was changed." +- **D97** (Appearance): added rows for the legacy gallery settings the concept leaves + out, after its own rows: "Show image details on cards" (`showImageInfo`; off hides a + card's title and file line, Info and Tag stay at the end of the row), "Collapse viewer + details by default" (`sidebarCollapsedByDefault`; the viewer opens with + `#viewer-details` hidden and a "Show details"/"Hide details" toggle with + `aria-expanded`, the stage taking the whole width), and the operation "Reset display + defaults" ("Restore this browser’s display defaults, then save. Other settings are + kept.", button "Reset"), which drafts the gallery's defaults for the browser rows only. + The node search timeout shows the server's real value, not the sample 30. CSS under D97. +- **D124** (inspector): with "Load metadata on selection" off, the Images panel and + the viewer's details show "Metadata loads when you ask for it." and a "Load metadata" + button instead of reading the file. The concept has no such state, so no concept + comparison renders it: `metadata off shows an explicit load and loads once on request` + (workspace-system.spec) pins it instead. + +Appearance scope and states. The Default layout and Thumbnail size defaults start the +Images page only (a new visit, not the current view or a saved view); the embedded +galleries (a prompt history's Shared gallery and its All images / Repeat run dialogs) +have no toolbar, so they keep Grid/Comfortable, while the metadata, cache and card +defaults apply there too. A host row whose `/settings` read failed stays disabled with +"Server settings could not be read."; a rejected timeout says "Some changes were not +saved. Review the highlighted settings." A display default another page saved under one +of this tab's drafts flags that draft with the conflict status, whether it arrived while +System was mounted (a `storage` event) or while it was not (checked again on mount, and +before a write); the draft is never written over the other page's value without that +status first. A change kept for this tab only is stored by the next write storage accepts. + +Open question (saved-views follow-up, M8): a saved view stores no layout or size and +always opens at Grid/Comfortable, even when the Default layout is Masonry. Should saved +views store a layout, or follow the default? + +`tests/browser/system-concept.spec.mjs` compares the shell and Overview with the concept +(dark and light, 1600×1100 and 390×844), mapping only the deviations above; unmapped, +the difference is exactly the D92, D93, D96 and D122 paths. Each surface's `top` is +measured, which pins the concept's `.surface + .surface` 18px offset that concept.css +also applies inside `.two-col` (copied as is). `statistics and health dialogs match +the concept` maps D95 from the concept's own five rows (labels verbatim): it inserts +Pair uniqueness after Canonical hashes and Consistency last, splits Missing files per +source when there are several, replaces only the chips with the fixture's expected +results, and asserts the full unmapped path list; it maps D96 and compares the +Maintenance notice unchanged. `system comparison detects mutations` removes a row, +renames a metric label, swaps the `--line` token, recolours a chip, renames a check, +removes the totals table and renames a caption. `appearance matches the concept` maps +only D97 and the real timeout (unmapped, exactly those paths differ), and `appearance +comparison detects mutations` removes a row, renames a label and an option and swaps +the `--line` token. Paired captures (`{concept,production}-system-{dark,light}-{1600,390}.png`, +`{concept,production}-{statistics,health}-{dark,light}.png`, +`{concept,production}-appearance-{dark,light}-{1600,390}.png`) are test artifacts; there +are no PNG baselines. `tests/browser/workspace-system.spec.mjs` covers the real counts, +the health check and metric 4, a failed check, the thumbnail row against the status +endpoint (painted after the rest when its read is held, unavailable when it fails), the review row (Q6), the section links, the Statistics dialog against the +route, a load failure with Retry and late answers after leaving, and Appearance: new +visits versus the current view and a saved view, the embedded gallery keeping Grid, +metadata off and on request, the cache (counted up to a request barrier), hidden paths, +the video defaults, the host fields with their revision, a stale one and a failed read, +reload and a second tab, a second tab's save while System was closed, storage refused, +the D97 rows and reset. Its hooks are +`tests/browser/workspace_system_fixture.py` (`/__test__/system/*`: reset, injected +failures, an absent source, a deleted linked file, unthumbnailed files) on the +concept-shaped fixture, with delays through the shared request holds; reset also restores +the server's host settings and its `config.json`, which must be the one inside the server's +temp root (`tests/test_workspace_system_fixture.py`). + +```sh +python3 -m unittest tests.test_database_health tests.test_system_api tests.test_library \ + tests.test_workspace_system_fixture -v +node --test tests/js/workspace-system-state.test.js tests/js/workspace-system-controls.test.js \ + tests/js/workspace-system-overview.test.js tests/js/workspace-state.test.js \ + tests/js/workspace-system-appearance.test.js tests/js/workspace-preferences.test.js \ + tests/js/workspace-inspection-state.test.js tests/js/workspace-images-state.test.js +npm run --prefix tools/vendor test:workflows -- system-concept.spec.mjs workspace-system.spec.mjs \ + --project chromium --project touch --project firefox --project webkit +``` + ## Workspace data fields for the concept pages (#249) These fields are additive. The legacy `/prompt_manager/images/output` listing (no diff --git a/database/health.py b/database/health.py new file mode 100644 index 00000000..7feedda3 --- /dev/null +++ b/database/health.py @@ -0,0 +1,354 @@ +"""The read-only health check behind System › Overview and Maintenance. + +``run_health_checks`` opens its own ``mode=ro`` connection, reads one +snapshot and closes the connection in ``finally``; it never writes. Each +check reports ``passed``, ``issues`` (with an exact ``count``; the +integrity check stops at ``INTEGRITY_LIMIT`` problems and then says +``capped``), +``unavailable`` (it could not be checked here, so ``count`` is None, or +``missing_files`` checked only some sources and found nothing) or ``failed`` +(it raised; the other checks still run). ``missing_files`` also reports each +source and whether the check was ``partial``; it reads the image rows inside +the snapshot and checks their files after the snapshot ended, so a slow source +never holds the WAL. At most ``MAX_DETAIL`` detail strings are returned per +check, and none of them carries a server path. +""" + +import datetime +import logging +import ntpath +import os +import sqlite3 +from pathlib import PurePosixPath, PureWindowsPath +from urllib.parse import quote + +try: + from ..utils.hashing import generate_prompt_hash + from .models import normalize_image_path +except ImportError: + from utils.hashing import generate_prompt_hash + from database.models import normalize_image_path + +logger = logging.getLogger("prompt_manager.database.health") + +MAX_DETAIL = 10 # detail strings returned per check +INTEGRITY_LIMIT = 100 # PRAGMA integrity_check(N): problems SQLite reports +BATCH_ROWS = 100 # image rows read per fetch +REVIEW_UNAVAILABLE = "Needs review is not available yet." +CHECK_FAILED = "The check could not finish." +# Check ids and the labels of the concept's health report, in its order; +# pair uniqueness and consistency are the added rows (D63, now D95). +CHECKS = ( + ("integrity", "Integrity check"), + ("foreign_keys", "Foreign keys & pair references"), + ("canonical_hashes", "Canonical hashes"), + ("pair_uniqueness", "Pair uniqueness"), + ("missing_files", "Missing files"), + ("ambiguous_metadata", "Ambiguous metadata"), + ("consistency", "Consistency"), +) +_DATABASE_CHECKS = ( + "integrity", + "foreign_keys", + "canonical_hashes", + "pair_uniqueness", + "missing_files", + "consistency", +) + + +def iter_consistency_issues(conn): + """The legacy consistency issues: tag and image rows of a missing prompt.""" + for ref in conn.execute( + "SELECT pt.prompt_id FROM prompt_tags pt " + "LEFT JOIN prompts p ON pt.prompt_id = p.id WHERE p.id IS NULL" + ): + yield f"prompt_tags entry references non-existent prompt {ref[0]}" + for ref in conn.execute( + "SELECT gi.id, gi.prompt_id FROM generated_images gi " + "LEFT JOIN prompts p ON gi.prompt_id = p.id WHERE p.id IS NULL" + ): + yield f"Image {ref[0]} references non-existent prompt {ref[1]}" + + +def _result(count, detail, *, status=None): + return { + "status": status or ("issues" if count else "passed"), + "count": count, + "detail": detail[:MAX_DETAIL], + } + + +def _counted(lines): + """(count, first MAX_DETAIL lines) of an iterable, consumed once.""" + count, detail = 0, [] + for line in lines: + count += 1 + if len(detail) < MAX_DETAIL: + detail.append(line) + return _result(count, detail) + + +def readonly_uri(db_path, *, windows=None): + """The ``mode=ro`` SQLite URI of *db_path*, on this platform or the given one. + + pathlib spells a UNC path ``\\\\server\\share\\x.db`` as + ``file://server/share/x.db``, a remote authority SQLite refuses to open. + With an empty authority (``file:////server/share/x.db``) SQLite hands + Windows the ``//server/share`` path, which it opens as the UNC path. + """ + if windows is None: + windows = os.name == "nt" + db_path = os.path.abspath(db_path) + if not windows: + return PurePosixPath(db_path).as_uri() + "?mode=ro" + path = PureWindowsPath(db_path) + if ntpath.splitdrive(str(path))[0].startswith("\\\\"): + return "file:" + quote("//" + path.as_posix()) + "?mode=ro" + return path.as_uri() + "?mode=ro" + + +def _integrity(conn, _roots): + rows = [ + row[0] for row in conn.execute(f"PRAGMA integrity_check({INTEGRITY_LIMIT})") + ] + if rows == ["ok"]: + return {**_result(0, []), "capped": False} + # SQLite stops listing at the limit: there may be more problems than that. + return {**_result(len(rows), rows), "capped": len(rows) >= INTEGRITY_LIMIT} + + +def _foreign_keys(conn, _roots): + return _counted( + f"{table} row {rowid} references a missing {parent} row" + for table, rowid, parent, _fkid in conn.execute("PRAGMA foreign_key_check") + ) + + +def _prompt_hash(text): + return generate_prompt_hash(text) if isinstance(text, str) else None + + +def _canonical_hashes(conn, _roots): + conn.create_function("pm_prompt_hash", 1, _prompt_hash, deterministic=True) + return _counted( + f"Prompt {row[0]}: the stored hash does not match its text" + for row in conn.execute( + "SELECT id FROM prompts WHERE hash IS NOT pm_prompt_hash(text) ORDER BY id" + ) + ) + + +def _pair_uniqueness(conn, _roots): + groups = conn.execute( + "SELECT positive_prompt_id,negative_prompt_id,COUNT(*) FROM prompt_pairs " + "GROUP BY positive_prompt_id,negative_prompt_id HAVING COUNT(*)>1 " + "ORDER BY positive_prompt_id,negative_prompt_id" + ).fetchall() + detail = [ + f"Positive prompt {positive} with " + + ("no negative" if negative is None else f"negative {negative}") + + f": {rows} pair rows" + for positive, negative, rows in groups + ] + # The surplus rows: what keeping one row per pair would leave out. + return _result(sum(rows - 1 for _p, _n, rows in groups), detail) + + +def _consistency(conn, _roots): + return _counted(iter_consistency_issues(conn)) + + +def _readable(root): + return os.path.isdir(root) and os.access(root, os.R_OK | os.X_OK) + + +def _root_prefix(root): + return normalize_image_path(root).rstrip("\\/") + os.sep + + +def _source_name(root, index): + """A source's folder name for the report (never its server path).""" + return os.path.basename(str(root).rstrip("\\/")) or f"Source {index + 1}" + + +def _owner(path, prefixes): + """Index of the longest source prefix *path* lies under, or None.""" + owner, longest = None, -1 + for index, prefix in enumerate(prefixes): + if path.startswith(prefix) and len(prefix) > longest: + owner, longest = index, len(prefix) + return owner + + +def _image_owner(image_path, file_path, prefixes): + spelled = file_path or normalize_image_path(image_path) + owner = _owner(spelled, prefixes) + if owner is None: + owner = _owner(normalize_image_path(os.path.realpath(image_path)), prefixes) + return owner + + +def _unavailable_line(unavailable, *, partial): + noun = "source is" if unavailable == 1 else "sources are" + effect = "its files were" if unavailable == 1 else "their files were" + if not partial: + effect = "missing files were" + return f"{unavailable} image {noun} unavailable, so {effect} not counted." + + +def _missing_files(conn, roots): + """Links whose file is gone, counted per source (only readable ones). + + Each source reports ``checked`` with its own count, or ``unavailable`` + (an unmounted drive) with no count: its links are never turned into a + count, while the readable sources still are. The check's ``count`` is the + readable sources' total (None when none is readable) and ``partial`` says + some sources were not checked. Links outside every source are named in the + detail, not counted. + + Inside the snapshot this only reads the image rows; it returns the step + that checks their files, which runs after the snapshot ended. + """ + if not roots: + result = _result(None, ["No image source is available."], status="unavailable") + return {**result, "partial": False, "sources": []} + rows = [] + cursor = conn.execute( + "SELECT id,image_path,filename,file_path FROM generated_images ORDER BY id" + ) + while batch := cursor.fetchmany(BATCH_ROWS): + rows.extend(batch) + return lambda: _check_files(rows, roots) + + +def _check_files(rows, roots): + """The ``missing_files`` result of the image *rows*, from the file system.""" + readable = [_readable(root) for root in roots] + sources = [ + { + "name": _source_name(root, index), + "status": "checked" if ok else "unavailable", + "count": 0 if ok else None, + } + for index, (root, ok) in enumerate(zip(roots, readable)) + ] + unavailable = readable.count(False) + if not any(readable): + result = _result( + None, [_unavailable_line(unavailable, partial=False)], status="unavailable" + ) + return {**result, "partial": False, "sources": sources} + prefixes = [_root_prefix(root) for root in roots] + outside, missing = 0, [] + for image_id, image_path, filename, file_path in rows: + owner = _image_owner(image_path, file_path, prefixes) + if owner is None: + outside += 1 + elif readable[owner] and not os.path.exists(image_path): + sources[owner]["count"] += 1 + if len(missing) < MAX_DETAIL: + name = filename or os.path.basename(image_path) + missing.append(f"Image {image_id}: {name} is missing") + count = sum(source["count"] or 0 for source in sources) + detail = [_unavailable_line(unavailable, partial=True)] if unavailable else [] + detail += missing + if outside: + links = "link" if outside == 1 else "links" + verb = "was" if outside == 1 else "were" + detail.append( + f"{outside} {links} outside the image sources {verb} not checked." + ) + status = None if count or not unavailable else "unavailable" + result = _result(count, detail, status=status) + return {**result, "partial": bool(unavailable), "sources": sources} + + +def _ambiguous(review_count): + if review_count is None: + return _result(None, [REVIEW_UNAVAILABLE], status="unavailable") + return _result(review_count, []) + + +def overall_status(checks) -> str: + """``passed`` only when every check passed; any check not run is incomplete.""" + statuses = {check["status"] for check in checks} + if statuses & {"unavailable", "failed"} or any( + check.get("partial") for check in checks + ): + return "incomplete" + return "issues" if "issues" in statuses else "passed" + + +def _failed(): + return _result(None, [CHECK_FAILED], status="failed") + + +def _run_check(check_id, step, *args): + """One check's result, or None (reported as failed) when it raised.""" + try: + return step(*args) + except Exception: # any error fails this check only; the others still run + logger.exception(f"Health check {check_id} failed") + return None + + +def _run_database_checks(db_path, roots): + """Each database check's result (None when it raised), from one snapshot. + + A check may return a step instead of its result: the step runs after the + snapshot ended and the connection closed (``missing_files``' file checks). + """ + results = dict.fromkeys(_DATABASE_CHECKS) + try: + conn = sqlite3.connect(readonly_uri(db_path), uri=True) + except sqlite3.Error: + logger.exception("Health check could not open the database") + return results + try: + conn.execute("BEGIN") # one read snapshot for every check + for check_id in _DATABASE_CHECKS: + runner = globals()["_" + check_id] # looked up per run (tests patch it) + results[check_id] = _run_check(check_id, runner, conn, roots) + except sqlite3.Error: + logger.exception("Health check could not read the database") + finally: + conn.close() + for check_id, result in results.items(): + if callable(result): + results[check_id] = _run_check(check_id, result) + return results + + +class HealthOperations: + """Mixin for PromptDatabase: the read-only System health check.""" + + def run_health_checks(self, roots, *, review_count=None) -> dict: + """Run every check and return a ``HealthReport``. + + Args: + roots: The image source directories to judge missing files under: + the effective sources plus any configured source that is not + available (reported as an unavailable source of ``missing_files``). + review_count: The Needs-review pending count, or None while no + review queue exists (``ambiguous_metadata`` is then unavailable). + """ + if review_count is not None and ( + type(review_count) is not int or review_count < 0 + ): + raise ValueError("review_count must be a non-negative integer or None") + roots = [str(root) for root in roots] + database = _run_database_checks(self.model.db_path, roots) + checks = [] + for check_id, label in CHECKS: + if check_id == "ambiguous_metadata": + result = _ambiguous(review_count) + else: + result = database[check_id] or _failed() + checks.append({"id": check_id, "label": label, **result}) + return { + "success": True, + "checked_at": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "status": overall_status(checks), + "checks": checks, + } diff --git a/database/library.py b/database/library.py index 4e532b95..7393e6df 100644 --- a/database/library.py +++ b/database/library.py @@ -628,3 +628,54 @@ class LibraryOperations: "kind": "prompt-manager-library", "positives": [owners[key] for key in order], } + + def library_counts(self) -> dict: + """System › Overview counts from one snapshot, writing nothing. + + ``positive_prompts`` and ``negative_variants`` are the unfiltered + Library's ``total`` and ``variant_total``: eligible parents and active + pairs (every pair's positive is eligible). ``images`` counts indexed + ``generated_images`` rows, not files on disk. An image has no rating + of its own (it shows its linked prompt's), so ``rated_images`` counts + the indexed images whose linked prompt is rated. + """ + with self.library_snapshot() as (conn, _epoch, _ceiling): + row = conn.execute( + "SELECT (SELECT COUNT(*) FROM prompts p WHERE " + _ELIGIBLE + ")," + "(SELECT COUNT(*) FROM prompt_pairs WHERE archived=0)," + "(SELECT COUNT(*) FROM generated_images)," + "(SELECT COUNT(*) FROM generated_images gi JOIN prompts r " + "ON r.id=gi.prompt_id WHERE r.rating IS NOT NULL)" + ).fetchone() + return { + "positive_prompts": row[0], + "negative_variants": row[1], + "images": row[2], + "rated_images": row[3], + } + + def usage_by_category(self) -> list[dict]: + """Runs and average rating of the Library's parents, per category. + + Only eligible parents count, so a negative-only prompt's runs (which + repeat its pairs' runs) are never added twice. A blank category groups + with no category as ``None``. Unrated parents are left out of the + average, which is ``None`` when no parent is rated. Most runs first. + """ + with self.library_snapshot() as (conn, _epoch, _ceiling): + rows = conn.execute( + "SELECT * FROM (SELECT NULLIF(TRIM(p.category),'') AS category," + "COALESCE(SUM(p.run_count),0) AS runs,AVG(p.rating) AS rating " + "FROM prompts p WHERE " + _ELIGIBLE + " GROUP BY 1) " + "ORDER BY runs DESC,category IS NULL,category" + ).fetchall() + return [ + { + "category": row["category"], + "runs": row["runs"], + "average_rating": ( + None if row["rating"] is None else round(row["rating"], 2) + ), + } + for row in rows + ] diff --git a/database/operations.py b/database/operations.py index 28bf35db..024dc867 100644 --- a/database/operations.py +++ b/database/operations.py @@ -4,6 +4,7 @@ Database operations for KikoTextEncode prompt storage and retrieval. import sqlite3 import json +import math import datetime import os from typing import Optional, List, Dict, Any, Set, Tuple @@ -20,11 +21,13 @@ from .execution_history import ExecutionHistoryOperations from .execution_contexts import ExecutionContextOperations, LIVE_HINT_CTES from .execution_outputs import ExecutionOutputOperations from .history_reads import HistoryReadOperations +from .health import HealthOperations, iter_consistency_issues # Import logging system try: from ..utils.logging_config import get_logger from ..utils.hashing import generate_prompt_hash + from ..utils.json_strict import dumps_strict except ImportError: import sys @@ -32,6 +35,7 @@ except ImportError: sys.path.insert(0, current_dir) from utils.logging_config import get_logger from utils.hashing import generate_prompt_hash + from utils.json_strict import dumps_strict # Subquery to fetch tags from junction table, embedded in SELECT statements @@ -342,6 +346,7 @@ class PromptDatabase( ImageFavoriteOperations, AnnotationOperations, SavedViewOperations, + HealthOperations, ): """Database operations class for managing prompts.""" @@ -1695,9 +1700,9 @@ class PromptDatabase( width, height, image_format, - json.dumps(metadata.get("workflow", {})), - json.dumps(metadata.get("prompt", {})), - json.dumps(metadata.get("parameters", {})), + dumps_strict(metadata.get("workflow", {})), + dumps_strict(metadata.get("prompt", {})), + dumps_strict(metadata.get("parameters", {})), ), ) conn.commit() @@ -1902,13 +1907,13 @@ class PromptDatabase( obj: The object to clean (dict, list, or scalar) Returns: - Cleaned object with NaN values replaced by None + Cleaned object with NaN and Infinity values replaced by None """ if isinstance(obj, dict): return {key: self._clean_nan_values(value) for key, value in obj.items()} elif isinstance(obj, list): return [self._clean_nan_values(item) for item in obj] - elif isinstance(obj, float) and str(obj) == "nan": + elif isinstance(obj, float) and not math.isfinite(obj): return None else: return obj @@ -2570,33 +2575,13 @@ class PromptDatabase( return cursor.rowcount def check_consistency(self) -> List[str]: - """Run consistency checks and return a list of issue descriptions.""" - issues: List[str] = [] - with self.model.get_connection() as conn: - # Check for orphaned prompt_tags entries - cursor = conn.execute(""" - SELECT pt.prompt_id, pt.tag_id FROM prompt_tags pt - LEFT JOIN prompts p ON pt.prompt_id = p.id - WHERE p.id IS NULL - """) - for ref in cursor.fetchall(): - issues.append( - "prompt_tags entry references non-existent prompt " - f"{ref['prompt_id']}" - ) + """Run consistency checks and return a list of issue descriptions. - # Check for orphaned image entries - cursor = conn.execute(""" - SELECT gi.id, gi.prompt_id FROM generated_images gi - LEFT JOIN prompts p ON gi.prompt_id = p.id - WHERE p.id IS NULL - """) - for ref in cursor.fetchall(): - issues.append( - f"Image {ref['id']} references non-existent prompt " - f"{ref['prompt_id']}" - ) - return issues + The same queries back the System health check's Consistency row + (database/health.py). + """ + with self.model.get_connection() as conn: + return list(iter_consistency_issues(conn)) def _image_row_to_dict(self, row: sqlite3.Row) -> Dict[str, Any]: """ diff --git a/database/pair_ingestion.py b/database/pair_ingestion.py index cad3641a..27781987 100644 --- a/database/pair_ingestion.py +++ b/database/pair_ingestion.py @@ -1,6 +1,5 @@ """Atomic metadata enrichment for a single known generation context.""" -import json import os from typing import Any, Dict @@ -9,8 +8,10 @@ from .operations import _coerce_file_info try: from ..utils.hashing import generate_prompt_hash + from ..utils.json_strict import dumps_strict except ImportError: from utils.hashing import generate_prompt_hash + from utils.json_strict import dumps_strict def ingest_resolved_pair( @@ -24,9 +25,7 @@ def ingest_resolved_pair( negative_text = negative.get("storage_text", negative["text"]) normalized = normalize_image_path(image_path) provenance = {"resolver_version": 1, "positive": positive, "negative": negative} - encoded = json.dumps( - provenance, sort_keys=True, separators=(",", ":"), allow_nan=False - ) + encoded = dumps_strict(provenance, sort_keys=True, separators=(",", ":")) with db._pair_transaction() as conn: images = conn.execute( "SELECT gi.*,p.hash AS positive_hash FROM generated_images gi " @@ -67,8 +66,9 @@ def ingest_resolved_pair( ).fetchone() pairs_added = int(prior_pair is None) file_size, width, height, image_format = _coerce_file_info(metadata) + # ComfyUI's NaN/Infinity are stored as null: the snapshot is strict JSON. snapshots = { - column: json.dumps(metadata[key], sort_keys=True, allow_nan=False) + column: dumps_strict(metadata[key], sort_keys=True) for column, key in ( ("workflow_data", "workflow"), ("prompt_metadata", "prompt"), diff --git a/py/api/__init__.py b/py/api/__init__.py index d97b3597..34ae5dac 100644 --- a/py/api/__init__.py +++ b/py/api/__init__.py @@ -8,6 +8,7 @@ Split into domain-specific mixins for maintainability: - AutotagRoutesMixin: auto-tagging model management and tagging - JobRoutesMixin: the session job registry behind Activity (py/api/jobs.py) - ImageTagJobsMixin: bulk Add tags / Auto-tag as image-tags jobs +- SystemRoutesMixin: System health check, Overview and usage statistics """ import asyncio @@ -15,6 +16,7 @@ import datetime import functools import gzip as gzip_module import json +import math import ntpath import os import re @@ -35,6 +37,7 @@ from .admin import AdminRoutesMixin from .settings_routes import SettingsRoutesMixin from .job_routes import JobRoutesMixin from .image_tag_jobs import ImageTagJobsMixin +from .system import SystemRoutesMixin from .jobs import JobRegistry from .logging_routes import LoggingRoutesMixin from .autotag_routes import AutotagRoutesMixin @@ -269,6 +272,7 @@ class PromptManagerAPI( SettingsRoutesMixin, JobRoutesMixin, ImageTagJobsMixin, + SystemRoutesMixin, LoggingRoutesMixin, AutotagRoutesMixin, LoraIntegrationMixin, @@ -325,6 +329,11 @@ class PromptManagerAPI( self._scan_job = None # The current duplicate scan (a py/api/jobs.Job outside the registry). self._duplicate_job = None + # The last System health report this session and the database + # generation it describes, bumped when a restore replaces self.db + # (py/api/system.py). + self._health_report = None + self._db_generation = 0 # Run cleanup on initialization to remove any existing duplicates try: @@ -482,6 +491,7 @@ class PromptManagerAPI( self._register_admin_routes(routes) self._register_settings_routes(routes) self._register_job_routes(routes) + self._register_system_routes(routes) self._register_logging_routes(routes) self._register_autotag_routes(routes) self._register_lora_routes(routes) @@ -600,12 +610,12 @@ class PromptManagerAPI( return prompts def _clean_nan_recursive(self, obj): - """Recursively clean NaN values from nested data structures.""" + """Recursively replace NaN and Infinity in nested data with None.""" if isinstance(obj, dict): return {key: self._clean_nan_recursive(value) for key, value in obj.items()} elif isinstance(obj, list): return [self._clean_nan_recursive(item) for item in obj] - elif isinstance(obj, float) and str(obj) == "nan": + elif isinstance(obj, float) and not math.isfinite(obj): return None else: return obj @@ -728,6 +738,13 @@ class PromptManagerAPI( Returns: List[Path]: Validated directory paths, possibly empty. """ + return self._output_dirs_with_origin()[0] + + def _output_dirs_with_origin(self): + """``(dirs, auto_detected)``: the ``_get_all_output_dirs`` list, and + whether it comes from detection or ComfyUI's output rather than from + a valid configured root (False when there is no directory at all). + """ from ..config import GalleryConfig configured = list(GalleryConfig.MONITORING_DIRECTORIES) @@ -746,15 +763,16 @@ class PromptManagerAPI( output_dirs.append(Path(real)) if output_dirs: - return output_dirs + return output_dirs, False if configured: # Every configured root was rejected: never let auto-detection # resurrect it, serve ComfyUI's own output directory instead. - return self._comfyui_output_dir_list() - - fallback = self._find_comfyui_output_dir() - return [Path(fallback)] if fallback else [] + fallback_dirs = self._comfyui_output_dir_list() + else: + fallback = self._find_comfyui_output_dir() + fallback_dirs = [Path(fallback)] if fallback else [] + return fallback_dirs, bool(fallback_dirs) def _comfyui_output_dir_list(self): """ComfyUI's output directory as a one-element list, or [] if unknown.""" diff --git a/py/api/admin.py b/py/api/admin.py index 7ad05da6..dc4e9bcb 100644 --- a/py/api/admin.py +++ b/py/api/admin.py @@ -41,7 +41,7 @@ try: PLACEHOLDERS, GenerationParamsError, extract_generation_params, - parse_js_json, + parse_page_json, ) from ...utils.prompt_pair_resolver import resolve_prompt_pair except ImportError: @@ -51,7 +51,7 @@ except ImportError: PLACEHOLDERS, GenerationParamsError, extract_generation_params, - parse_js_json, + parse_page_json, ) from utils.prompt_pair_resolver import resolve_prompt_pair @@ -804,6 +804,9 @@ class AdminRoutesMixin: result = await self._run_in_executor( self._delete_duplicate_images_sync, image_paths, output_dir ) + if result["deleted_count"]: + # The cached listing would keep (and count) the deleted files. + self.invalidate_gallery_cache() response_data = { "success": True, @@ -1271,6 +1274,7 @@ class AdminRoutesMixin: self.logger.info(f"Removed {len(removed)} old safety backups") previous = self.db self.db = PromptDatabase(model.db_path) + self._database_replaced() # a health report described the old one # Its connections would otherwise linger (a file lock on Windows) previous.close_all() prompt_count = self.db.model.get_database_info().get("total_prompts", 0) @@ -1602,11 +1606,14 @@ class AdminRoutesMixin: @staticmethod def _graph_prompt_text(metadata): - """The positive prompt the browser parser traces from the graphs, or None.""" + """The positive prompt the browser parser traces from the graphs, or None. + + Records parse as the legacy pages parsed them (NaN reads as null). + """ records = {} for key in ("prompt", "workflow"): value = metadata.get(key) - records[key] = parse_js_json(value) if isinstance(value, str) else value + records[key] = parse_page_json(value) if isinstance(value, str) else value try: positive = extract_generation_params(records)["positivePrompt"] except GenerationParamsError: diff --git a/py/api/images.py b/py/api/images.py index 69ad96fa..3bf0a341 100644 --- a/py/api/images.py +++ b/py/api/images.py @@ -3,6 +3,7 @@ import asyncio import base64 import binascii +import functools import hashlib import hmac import json @@ -37,6 +38,7 @@ try: from ...database.models import normalize_image_path from ...utils.card_seed import card_seed, header_params, parsed_positive from ...utils.parallel import map_parallel + from ...utils.json_strict import dumps_strict from ...utils.inspection_response import ( encode_inspection_response, library_metadata, @@ -56,6 +58,7 @@ except ImportError: from database.models import normalize_image_path from utils.card_seed import card_seed, header_params, parsed_positive from utils.parallel import map_parallel + from utils.json_strict import dumps_strict from utils.inspection_response import ( encode_inspection_response, library_metadata, @@ -982,27 +985,12 @@ class ImageRoutesMixin: # Add url and thumbnail_url so the frontend can serve them cleaned_images = self._present_images(cleaned_images) - # Additional fallback: convert to JSON string and clean NaN values manually - try: - response_data = {"success": True, "images": cleaned_images} - # Convert to JSON string - json_str = json.dumps(response_data, default=str) - - # Clean any remaining NaN values with regex - json_str = re.sub(r":\s*NaN", ": null", json_str) - json_str = re.sub(r"\[\s*NaN\s*\]", "[null]", json_str) - json_str = re.sub(r",\s*NaN\s*,", ", null,", json_str) - json_str = re.sub(r",\s*NaN\s*\]", ", null]", json_str) - json_str = re.sub(r"\[\s*NaN\s*,", "[null,", json_str) - - # Parse back to verify it's valid JSON - cleaned_data = json.loads(json_str) - - return web.json_response(cleaned_data) - except Exception as json_error: - self.logger.error(f"JSON cleaning error: {json_error}") - # Fallback to original response - return web.json_response({"success": True, "images": cleaned_images}) + # Rows an older version stored may still hold NaN/Infinity: answer + # strict JSON, and leave prompt text that spells "NaN" untouched. + return web.json_response( + {"success": True, "images": cleaned_images}, + dumps=functools.partial(dumps_strict, default=str), + ) except Exception as e: self.logger.error(f"Get prompt images error: {e}") diff --git a/py/api/jobs.py b/py/api/jobs.py index 568a99e6..09ab15aa 100644 --- a/py/api/jobs.py +++ b/py/api/jobs.py @@ -88,6 +88,9 @@ DUPLICATE_CLEANUP_JOB_KIND = "duplicate_cleanup" THUMBNAILS_JOB_KIND = "thumbnails" LORA_IMPORT_JOB_KIND = "lora_import" AUTOTAG_BATCH_JOB_KIND = "autotag" +# The System health check (py/api/system.py): a read-only ``read`` job with +# no file outcomes, so it may run beside a scan. +HEALTH_JOB_KIND = "health" def resolve_admission(admission=None, record_changing=None) -> str: diff --git a/py/api/system.py b/py/api/system.py new file mode 100644 index 00000000..16a62d71 --- /dev/null +++ b/py/api/system.py @@ -0,0 +1,220 @@ +"""System routes: the read-only health check, the Overview and usage statistics. + +- ``POST /prompt_manager/system/health`` starts the health check + (``database/health.py``) off-loop as a ``read`` job of kind ``health``: one + at a time, beside a scan, not cancellable and with no file outcomes. The + API keeps the last report for this session; ``GET`` returns it or null. + A report describes the database it was taken from: replacing the database + (a restore) forgets it, and a check that started before the swap is not + kept. +- ``GET /prompt_manager/system/overview`` is the System › Overview aggregate. + Every value is real or null: there is no backup status until backups report + one, no health result until a check ran, and no Needs-review count until + the review queue exists. The missing-thumbnail count is not part of it: + the page reads ``/images/thumbnails/status``, which walks every source. +- ``GET /prompt_manager/system/statistics`` lists runs and ratings per + category, with the legacy ``/stats`` totals and the Overview's count of + rated images (images whose linked prompt is rated). + +New routes answer errors with ``{success:false, code, error}``. +""" + +import asyncio +import os +from collections import namedtuple + +from aiohttp import web + +from .job_routes import busy_response +from .jobs import HEALTH_JOB_KIND, JobBusy + +HEALTH_JOB_TITLE = "Health check" +_HEALTH_STATUS_LINES = { + "passed": "Health check passed", + "issues": "Health check found issues", + "incomplete": "Health check incomplete", +} + + +# The last health report and the database generation it was taken from. +_StoredReport = namedtuple("_StoredReport", "generation report") + + +async def _gather_settled(*reads): + """Await every read, then raise the first failure. + + Unlike a plain gather, a failure never answers while the other executor + reads still run against the database. + """ + results = await asyncio.gather(*reads, return_exceptions=True) + for result in results: + if isinstance(result, BaseException): + raise result + return results + + +def _failure(message, status=500, code=None): + return web.json_response( + {"success": False, "code": code, "error": message}, status=status + ) + + +def health_summary(report): + """What the Overview shows of a report: when, the outcome, checks with issues.""" + if report is None: + return None + return { + "checked_at": report["checked_at"], + "status": report["status"], + "issues": sum(1 for check in report["checks"] if check["status"] == "issues"), + } + + +class SystemRoutesMixin: + """Mixin providing the /prompt_manager/system endpoints.""" + + def _register_system_routes(self, routes): + @routes.post("/prompt_manager/system/health") + async def start_health_check_route(request): + return await self.start_health_check(request) + + @routes.get("/prompt_manager/system/health") + async def health_report_route(request): + return await self.get_health_report(request) + + @routes.get("/prompt_manager/system/overview") + async def system_overview_route(request): + return await self.get_system_overview(request) + + @routes.get("/prompt_manager/system/statistics") + async def system_statistics_route(request): + return await self.get_system_statistics(request) + + # ── health ──────────────────────────────────────────────────────── + + def _health_roots(self): + """The sources missing files are judged under. + + The effective sources, plus every configured source that is not an + existing directory (an unmounted drive): the check then reports + missing files as unavailable instead of counting that source's links. + """ + from ..config import GalleryConfig + + roots = [str(path) for path in self._get_all_output_dirs()] + for configured in GalleryConfig.MONITORING_DIRECTORIES: + try: + resolved = GalleryConfig.resolve_gallery_root(configured) + except (OSError, TypeError, ValueError): + continue + if not os.path.isdir(resolved): + roots.append(resolved) + return roots + + def _database_replaced(self): + """Forget what described the previous database (call after swapping ``db``).""" + self._db_generation += 1 + self._health_report = None + + def _current_health_report(self): + """The last report, or None when none was kept for the current database.""" + stored = self._health_report + generation = self._db_generation + if isinstance(stored, _StoredReport) and stored.generation == generation: + return stored.report + return None + + async def start_health_check(self, request): + """Start the health check job (202), or 409 naming the running check.""" + generation = self._db_generation + db = self.db + try: + roots = await self._run_in_executor(self._health_roots) + except Exception: + self.logger.exception("Health check could not resolve the image sources") + return _failure("Failed to start the health check") + + async def run(job): + job.update(status_line="Checking the database and image links") + report = await self._run_in_executor(db.run_health_checks, roots) + if generation == self._db_generation: + self._health_report = _StoredReport(generation, report) + else: + self.logger.info("Health report dropped: the database was replaced") + job.update(progress=100, status_line=_HEALTH_STATUS_LINES[report["status"]]) + + try: + job = self._jobs.start( + HEALTH_JOB_KIND, + HEALTH_JOB_TITLE, + admission="read", + cancellable=False, + scope={}, + runner=run, + ) + except JobBusy as busy: + return busy_response(busy) + return web.json_response({"success": True, "job": job.summary()}, status=202) + + async def get_health_report(self, request): + return web.json_response( + {"success": True, "report": self._current_health_report()} + ) + + # ── overview ────────────────────────────────────────────────────── + + async def get_system_overview(self, request): + from ..config import GalleryConfig + + try: + (output_dirs, auto_detected), counts = await _gather_settled( + self._run_in_executor(self._output_dirs_with_origin), + self._run_in_executor(self.db.library_counts), + ) + except Exception: + self.logger.exception("System overview failed") + return _failure("Failed to read the overview") + return web.json_response( + { + "success": True, + "library": { + "positive_prompts": counts["positive_prompts"], + "negative_variants": counts["negative_variants"], + }, + "images": { + "indexed": counts["images"], + "rated": counts["rated_images"], + }, + "health": health_summary(self._current_health_report()), + "backup": None, # no verified backup status exists yet (S8) + "needs_review": {"pending": None}, # no review queue yet (Q6) + "sources": { + "configured": len(GalleryConfig.MONITORING_DIRECTORIES), + "effective": len(output_dirs), + "auto_detected": auto_detected, + }, + }, + headers={"Cache-Control": "no-store"}, + ) + + # ── statistics ──────────────────────────────────────────────────── + + async def get_system_statistics(self, request): + try: + by_category, totals, counts = await _gather_settled( + self._run_in_executor(self.db.usage_by_category), + self._run_in_executor(self.db.get_statistics), + self._run_in_executor(self.db.library_counts), + ) + except Exception: + self.logger.exception("System statistics failed") + return _failure("Failed to read statistics") + return web.json_response( + { + "success": True, + "by_category": by_category, + "totals": totals, + "rated_images": counts["rated_images"], + }, + headers={"Cache-Control": "no-store"}, + ) diff --git a/tests/browser/server.py b/tests/browser/server.py index 9b60d7b1..9926c269 100644 --- a/tests/browser/server.py +++ b/tests/browser/server.py @@ -21,6 +21,7 @@ from PIL.PngImagePlugin import PngInfo from tests.browser.request_hold import register_request_holds from tests.browser import slow_reset_watch from tests.browser.workspace_activity_fixture import register_activity_commands +from tests.browser.workspace_system_fixture import register_system_commands from tests.browser.workspace_concept_fixture import ( MAX_EXTRA_IMAGES as CONCEPT_MAX_EXTRA_IMAGES, MAX_SEEDLESS_IMAGES as CONCEPT_MAX_SEEDLESS_IMAGES, @@ -198,6 +199,7 @@ def create_app(root: Path) -> web.Application: register_tagging_commands(routes, api, root, model) reveal_stub = register_reveal_stub(routes, api, root) register_activity_commands(app, routes, api) + register_system_commands(app, routes, api, root) request_holds = register_request_holds(app, routes) slow_reset_watch.install(app, api) diff --git a/tests/browser/system-concept.spec.mjs b/tests/browser/system-concept.spec.mjs new file mode 100644 index 00000000..761134b9 --- /dev/null +++ b/tests/browser/system-concept.spec.mjs @@ -0,0 +1,535 @@ +import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs'; +import { test, expect } from './fixtures.mjs'; +import { SIZES, THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate } from './concept-inventory.mjs'; + +// #249 System S2 Task 6: System's shell and its Overview section (app.js renderSystem 658–706), the Statistics dialog +// ("Usage statistics", a1600–1604) and the health report (a1439–1456) against the immutable concept, on the +// concept-shaped real fixture. Each concept measurement is mapped only by the recorded deviations (D92–D96, D121, +// D122, D123), each commented where it applies; after that the difference must be empty, and a mutation of a +// row, a label or a token must be noticed. +test.use({ locale: 'en-US', timezoneId: 'UTC' }); + +test.beforeEach(async ({ page, request, browserName }) => { + await startCoverage(page, browserName); + await seedConcept(request); + await expect(await request.post('/__test__/system/reset')).toBeOK(); + await expect(await request.post('/__test__/jobs/restart')).toBeOK(); + page.conceptErrors = []; + page.on('pageerror', error => page.conceptErrors.push(error.message)); +}); + +test.afterEach(async ({ page, request, browserName }) => { + await saveCoverage(page, browserName); + await request.post('/__test__/system/reset'); + expect(page.conceptErrors).toEqual([]); +}); + +/** Runs in the page; self-contained. System's heading, section nav, Overview body and Save row under `mainSelector`. */ +function measureSystem(mainSelector) { + const main = document.querySelector(mainSelector); + const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null; + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const box = (node, origin = main) => { + const rect = node.getBoundingClientRect(), base = origin.getBoundingClientRect(); + return { left: Math.round(rect.left - base.left), width: Math.round(rect.width) }; + }; + // A surface's top: from `.two-col` beside its neighbour, else from the bottom of the surface above it. This pins the + // concept's `.surface + .surface` 18px offset, which concept.css applies inside `.two-col` too (copied, not ruled). + const top = node => { + const rect = node.getBoundingClientRect(), previous = node.previousElementSibling?.getBoundingClientRect(); + if (previous && rect.top >= previous.bottom) return Math.round(rect.top - previous.bottom); + return Math.round(rect.top - node.parentElement.getBoundingClientRect().top); + }; + const text = node => node?.textContent.trim() ?? null; + const button = node => ({ text: text(node), className: node.className, + ...css(node, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopWidth', 'borderTopColor', + 'borderTopLeftRadius', 'opacity', 'cursor') }); + const heading = main.querySelector(':scope > .heading'); + const nav = main.querySelector(':scope > .section-nav'); + const layout = main.querySelector(':scope > .settings-layout'); + const metrics = layout.querySelector(':scope > .summary-metrics'); + const columns = layout.querySelector(':scope > .two-col'); + const save = layout.querySelector(':scope > .save-row'); + const firstMetric = metrics.querySelector('.metric'); + const firstSurface = columns.querySelector('.surface'); + const firstOperation = firstSurface.querySelector('.operation'); + return { + tree: [...main.children].map(name), + heading: { + title: text(heading.querySelector('h1')), description: text(heading.querySelector('p.muted')), + actions: [...(heading.querySelector('.heading-actions')?.children ?? [])].map(name), + }, + nav: { + role: nav.getAttribute('role'), label: nav.getAttribute('aria-label'), + ...css(nav, 'display', 'flexWrap', 'columnGap', 'rowGap', 'paddingBottom', 'marginBottom', 'borderBottomWidth', 'borderBottomColor'), + box: box(nav), + buttons: [...nav.children].map(item => ({ pressed: item.getAttribute('aria-pressed'), ...button(item) })), + }, + layout: { tree: [...layout.children].map(name), ...css(layout, 'maxWidth'), box: box(layout) }, + metrics: { + ...css(metrics, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap', 'marginBottom'), box: box(metrics), + items: [...metrics.children].map(item => ({ tree: [...item.children].map(name), value: text(item.querySelector('strong')), + label: text(item.querySelector('span')), box: box(item) })), + metric: css(firstMetric, 'paddingTop', 'paddingLeft', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius', 'backgroundColor'), + strong: css(firstMetric.querySelector('strong'), 'display', 'fontSize', 'fontWeight', 'color'), + span: css(firstMetric.querySelector('span'), 'fontSize', 'color', 'marginTop'), + }, + columns: { + ...css(columns, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap'), box: box(columns), + surfaces: [...columns.children].map(surface => ({ tree: [...surface.children].map(name), box: box(surface), top: top(surface), + heading: text(surface.querySelector(':scope > h2')), + rows: [...surface.querySelectorAll(':scope > .operation')].map(row => ({ tree: [...row.children].map(name), + title: text(row.querySelector('h3')), description: text(row.querySelector('p')), + action: button(row.querySelector('.row-actions > button')) })) })), + surface: css(firstSurface, 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius'), + h2: css(firstSurface.querySelector('h2'), 'fontSize', 'fontWeight', 'color', 'marginBottom'), + operation: css(firstOperation, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', + 'borderTopWidth', 'borderTopColor'), + h3: css(firstOperation.querySelector('h3'), 'fontSize', 'fontWeight', 'color'), + p: css(firstOperation.querySelector('p'), 'fontSize', 'color', 'marginTop'), + actions: css(firstOperation.querySelector('.row-actions'), 'display', 'columnGap', 'flexShrink'), + }, + save: { + tree: [...save.children].map(name), ...css(save, 'display', 'alignItems', 'justifyContent', 'columnGap', 'marginTop', 'paddingTop'), + status: { text: text(save.querySelector('#settings-status')), ...css(save.querySelector('#settings-status'), 'fontSize', 'color') }, + button: button(save.querySelector('button')), + }, + pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth, + }; +} + +/** Runs in the page: the open dialog's title, body sequence, tables, operation rows and actions. */ +function measureDialog() { + const dialog = document.querySelector('dialog#modal[open]'); + const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null; + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const body = dialog.querySelector('#modal-body'); + const firstCell = selector => body.querySelector(selector); + const list = body.querySelector(':scope > .operation-list'); + const actions = body.querySelector(':scope > .dialog-actions'); + return { + title: dialog.querySelector('#modal-title').textContent, + body: [...body.children].map(name), + tables: [...body.querySelectorAll(':scope > table')].map(table => ({ + caption: table.caption?.textContent ?? null, + head: [...table.tHead.rows[0].cells].map(cell => [cell.localName, cell.scope, cell.textContent]), + rows: [...table.tBodies[0].rows].map(row => [...row.cells].map(cell => cell.textContent)), + })), + tableStyle: firstCell(':scope > table') && { + table: css(firstCell(':scope > table'), 'width', 'borderCollapse', 'fontSize', 'marginTop'), + caption: css(firstCell(':scope > table caption'), 'textAlign', 'fontSize', 'fontWeight', 'color', 'paddingBottom'), + th: css(firstCell(':scope > table thead th'), 'textAlign', 'fontSize', 'fontWeight', 'color', 'paddingTop', 'paddingLeft', + 'borderBottomWidth', 'borderBottomColor'), + td: css(firstCell(':scope > table tbody td'), 'textAlign', 'fontSize', 'color', 'paddingTop', 'paddingLeft', + 'borderBottomWidth', 'borderBottomColor'), + }, + operations: list && { + ...css(list, 'display', 'rowGap', 'marginTop'), + rows: [...list.children].map(row => ({ tree: [...row.children].map(name), title: row.querySelector('h3').textContent, + chip: row.querySelector('.chip').textContent })), + operation: css(list.querySelector('.operation'), 'display', 'alignItems', 'justifyContent', 'paddingTop', 'borderTopWidth'), + h3: css(list.querySelector('.operation h3'), 'fontSize', 'fontWeight', 'color'), + chip: css(list.querySelector('.operation .chip'), 'fontSize', 'color', 'backgroundColor', 'borderTopLeftRadius', 'paddingLeft'), + }, + actions: actions && { + ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'), + buttons: [...actions.children].map(item => ({ text: item.textContent, className: item.className, + ...css(item, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft') })), + }, + }; +} + +/** Runs in the page: a `.notice` (Maintenance's "Check first…") with its copy. */ +function measureNotice(mainSelector) { + const notice = document.querySelector(`${mainSelector} .settings-layout > .notice`); + const css = (node, ...keys) => Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])); + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const run = notice.querySelector(':scope > button'); + return { + tree: [...notice.children].map(name), text: [...notice.querySelectorAll(':scope > div > *')].map(node => [node.localName, node.textContent]), + frame: css(notice, 'display', 'alignItems', 'justifyContent', 'columnGap', 'borderTopWidth', 'borderTopColor', 'backgroundColor', + 'borderTopLeftRadius', 'paddingTop', 'paddingLeft', 'marginBottom'), + h2: css(notice.querySelector('h2'), 'fontSize', 'fontWeight', 'color'), + p: css(notice.querySelector('p'), 'fontSize', 'color', 'marginTop'), + run: { text: run.textContent, className: run.className, ...css(run, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft') }, + }; +} + +async function serverJson(request, url) { + const response = await request.get(url); + await expect(response).toBeOK(); + return response.json(); +} + +const number = value => value.toLocaleString('en-US'); +const ratingText = value => value === null ? 'Not rated' : `${number(value)} / 5`; + +/** The concept's shell and Overview with the recorded deviations applied, given the server's real overview. */ +function mapOverviewConcept(reference, overview) { + const relabel = { 'Database integrity': 'Not checked', 'Positive prompts': number(overview.library.positive_prompts), + 'Negative variants': number(overview.library.negative_variants), Images: number(overview.images.indexed) }; + const rows = { + // D93 (Q6): no review queue yet, so no count and the action is unavailable (dimmed, as the Save row's D92). + '3 ambiguous image links': row => ({ ...row, title: 'Ambiguous image links', description: 'Not available yet.', + action: { ...row.action, opacity: '0.55', cursor: 'not-allowed' } }), + // D93: no backup has been recorded (verified backups arrive with S8). + 'Backup status': row => ({ ...row, description: 'No retained backup record.' }), + // D93: there is no paging, so the shared settings are grid and density. + 'One display configuration': row => ({ ...row, description: 'Grid and density settings are shared across views.' }), + // D93: the real sources, not samples. + '2 sample output sources': row => ({ ...row, title: `${number(overview.sources.effective)} output sources` }), + // D96: the dialog shows runs and ratings by category and library totals, not prompt, tag or date usage. + Statistics: row => ({ ...row, description: 'Runs and ratings by category, with library totals.' }), + }; + return { + ...reference, + // D92: no "Sample configuration" chip. + heading: { ...reference.heading, actions: [] }, + // D122: the concept's literal label is its prototype action name ("system-tab sections"). + nav: { ...reference.nav, label: 'System sections' }, + metrics: { ...reference.metrics, items: reference.metrics.items.map(item => ({ ...item, value: relabel[item.label] })) }, + columns: { ...reference.columns, surfaces: reference.columns.surfaces.map(surface => ({ ...surface, + rows: surface.rows.map(row => (rows[row.title] ?? (value => value))(row)) })) }, + save: { ...reference.save, + // D92: truthful status copy, and "Save preferences" is aria-disabled (dimmed) while nothing is unsaved. + status: { ...reference.save.status, text: 'No unsaved changes.' }, + button: { ...reference.save.button, opacity: '0.55', cursor: 'not-allowed' } }, + }; +} + +async function conceptOverview(page, theme) { + await openConcept(page, 'system', theme); + await expect(page.locator('#main .summary-metrics')).toBeVisible(); + await page.mouse.move(0, 0); + return page.evaluate(measureSystem, '#main'); +} + +async function productionOverview(page, theme) { + await openProduction(page, 'system/overview', theme); + await expect(page.locator('#workspace-main .summary-metrics')).toBeVisible(); + // The thumbnail row is its own read (I2): compare once it answered. + await expect(page.locator('#workspace-main .operation h3').filter({ hasText: 'thumbnails' })).toHaveText('No missing thumbnails'); + await page.mouse.move(0, 0); + return page.evaluate(measureSystem, '#workspace-main'); +} + +for (const theme of THEMES) for (const [width, height] of SIZES) { + test(`system shell and overview match the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => { + await page.setViewportSize({ width, height }); + const overview = await serverJson(request, '/prompt_manager/system/overview'); + // The fixture's thumbnails all exist, so the concept's "No missing thumbnails" is also the real title. + expect((await serverJson(request, '/prompt_manager/images/thumbnails/status')).pending).toBe(0); + const concept = await conceptOverview(page, theme); + expect(concept.heading.actions).toEqual(['span.chip']); + // Paired captures are artifacts only; there are no PNG baselines. + await page.screenshot({ path: testInfo.outputPath(`concept-system-${theme}-${width}.png`), fullPage: true }); + const production = await productionOverview(page, theme); + await page.screenshot({ path: testInfo.outputPath(`production-system-${theme}-${width}.png`), fullPage: true }); + expect(diffInventory(mapOverviewConcept(concept, overview), production)).toEqual([]); + expect(production.pageScroll).toBe(false); + }); +} + +// The recorded differences, by path: D93 the review row (title, Not available yet, dimmed action), the backup row, the +// preferences row and the sources row, and metric 4; D96 the statistics row; D92 the chip, the status and the dimmed +// Save; D122 the nav label. (The fixture's first three metrics are the concept's own counts.) +const UNMAPPED_PATHS = [ + 'columns.surfaces.0.rows.0.action.cursor', 'columns.surfaces.0.rows.0.action.opacity', + 'columns.surfaces.0.rows.0.description', 'columns.surfaces.0.rows.0.title', 'columns.surfaces.0.rows.1.description', + 'columns.surfaces.1.rows.0.description', 'columns.surfaces.1.rows.1.title', 'columns.surfaces.1.rows.2.description', + 'heading.actions.0', 'heading.actions.length', 'metrics.items.3.value', 'nav.label', + 'save.button.cursor', 'save.button.opacity', 'save.status.text', +]; + +test('system comparison detects mutations', async ({ page, request }) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const overview = await serverJson(request, '/prompt_manager/system/overview'); + const concept = await conceptOverview(page, 'dark'); + const expected = mapOverviewConcept(concept, overview); + const production = await productionOverview(page, 'dark'); + expect(diffInventory(expected, production)).toEqual([]); + // Unmapped, the concept differs exactly where a deviation is recorded, and nowhere else. + expect(diffInventory(concept, production).map(item => item.path).sort()).toEqual(UNMAPPED_PATHS); + const measure = () => page.evaluate(measureSystem, '#workspace-main'); + const paths = async () => diffInventory(expected, await measure()).map(item => item.path); + + // A removed row. + const undoRow = await mutate(page, { remove: ['#workspace-main .two-col > .surface:first-child > .operation:nth-of-type(2)'] }); + expect(await paths()).toEqual(expect.arrayContaining(['columns.surfaces.0.rows.length', 'columns.surfaces.0.tree.length'])); + await undoRow(); + // A changed label. + await page.evaluate(() => { document.querySelector('#workspace-main .metric span').textContent = 'Positive prompt'; }); + expect(await paths()).toEqual(['metrics.items.0.label']); + await page.evaluate(() => { document.querySelector('#workspace-main .metric span').textContent = 'Positive prompts'; }); + // A swapped token. + const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' }); + expect(await paths()).toEqual(expect.arrayContaining(['metrics.metric.borderTopColor', 'columns.surface.borderTopColor'])); + await undoToken(); + expect(diffInventory(expected, await measure())).toEqual([]); +}); + +/** D96: the concept's table, its rows the real ones; then "Library totals" in the same table markup. */ +function mapStatisticsConcept(reference, statistics) { + const [runs] = reference.tables; + return { + ...reference, + // No sample note (prototype copy); a second table follows the first. + body: ['table', 'table'], + tables: [ + { ...runs, rows: statistics.by_category.map(row => [row.category ?? 'Uncategorized', number(row.runs), ratingText(row.average_rating)]) }, + { caption: 'Library totals', head: [['th', 'col', 'Total'], ['th', 'col', 'Count']], + rows: [['Prompts', number(statistics.totals.total_prompts)], ['Tags', number(statistics.totals.total_tags)], + ['Images', number(statistics.totals.total_images)], ['Rated images', number(statistics.rated_images)]] }, + ], + }; +} + +// The fixture's image sources and each check's real chip on it: nothing is damaged, Needs review does not exist yet (Q6). +const FIXTURE_SOURCES = ['output', 'input']; +const FIXTURE_CHIPS = Object.freeze({ 'Integrity check': 'Passed', 'Foreign keys & pair references': 'Passed', + 'Canonical hashes': 'Passed', 'Pair uniqueness': 'Passed', 'Missing files': 'Passed', 'Ambiguous metadata': 'Not available yet', + Consistency: 'Passed' }); +const chipOf = title => { + if (!(title in FIXTURE_CHIPS)) throw new Error(`No expected chip for ${title}`); + return FIXTURE_CHIPS[title]; +}; + +/** + * D95: the concept's health report from its own five rows, labels verbatim, titled "Health report". Only the D95 rows + * are inserted where they are recorded (Pair uniqueness after Canonical hashes, Consistency last); Missing files has a + * row per image source when there are several; each chip is the fixture's expected result. + */ +function mapHealthConcept(reference, sources = FIXTURE_SOURCES) { + const [first] = reference.operations.rows; + const added = title => ({ ...first, title, chip: chipOf(title) }); + const rows = reference.operations.rows.flatMap(row => { + const mapped = row.title === 'Missing files' && sources.length > 1 + ? sources.map(source => ({ ...row, title: `Missing files · ${source}`, chip: chipOf(row.title) })) + : [{ ...row, chip: chipOf(row.title) }]; + return row.title === 'Canonical hashes' ? [...mapped, added('Pair uniqueness')] : mapped; + }); + return { + ...reference, + title: 'Health report', + // No sample note (prototype copy); D95 adds the checked line after the list. + body: ['div.operation-list', 'p.small', 'div.dialog-actions'], + operations: { ...reference.operations, rows: [...rows, added('Consistency')] }, + }; +} + +// Unmapped, the health report differs exactly where D95 is recorded: the title, the body sequence (no sample note, the +// checked line), the rows after Canonical hashes (Pair uniqueness inserted, Missing files per source) and the chips of +// the concept's sample findings. +const UNMAPPED_HEALTH_PATHS = [ + 'body.0', 'body.1', + ...[3, 4].flatMap(row => [`operations.rows.${row}.chip`, `operations.rows.${row}.title`]), + // The concept's five rows end at row 4; rows 5–7 exist only with the D95 rows and the second source. + ...[5, 6, 7].flatMap(row => ['chip', 'title', 'tree.0', 'tree.1', 'tree.length'].map(key => `operations.rows.${row}.${key}`)), + 'operations.rows.length', 'title', +]; + +async function conceptDialogs(page, theme, testInfo) { + await openConcept(page, 'system', theme); + await page.locator('#main').getByRole('button', { name: 'Explore', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics'); + await page.mouse.move(0, 0); + const statistics = await page.evaluate(measureDialog); + await page.screenshot({ path: testInfo.outputPath(`concept-statistics-${theme}.png`) }); + await page.keyboard.press('Escape'); + await page.locator('#main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click(); + const notice = await page.evaluate(measureNotice, '#main'); + await page.locator('#main .notice').getByRole('button', { name: 'Run health checks', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Sample health report'); + await page.mouse.move(0, 0); + await page.screenshot({ path: testInfo.outputPath(`concept-health-${theme}.png`) }); + return { statistics, notice, health: await page.evaluate(measureDialog) }; +} + +async function productionDialogs(page, theme, testInfo) { + await openProduction(page, 'system/overview', theme); + await page.locator('#workspace-main').getByRole('button', { name: 'Explore', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics'); + await page.mouse.move(0, 0); + const statistics = await page.evaluate(measureDialog); + await page.screenshot({ path: testInfo.outputPath(`production-statistics-${theme}.png`) }); + await page.keyboard.press('Escape'); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click(); + await expect(page.locator('#workspace-main .notice')).toBeVisible(); + const notice = await page.evaluate(measureNotice, '#workspace-main'); + await page.locator('#workspace-main .notice').getByRole('button', { name: 'Run health checks', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Health report'); + await page.mouse.move(0, 0); + await page.screenshot({ path: testInfo.outputPath(`production-health-${theme}.png`) }); + return { statistics, notice, health: await page.evaluate(measureDialog) }; +} + +for (const theme of THEMES) test(`statistics and health dialogs match the concept ${theme}`, async ({ page, request }, testInfo) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const concept = await conceptDialogs(page, theme, testInfo); + expect(concept.statistics.body).toEqual(['p.demo-note', 'table']); + expect(concept.health.body).toEqual(['p.demo-note', 'div.operation-list', 'div.dialog-actions']); + const production = await productionDialogs(page, theme, testInfo); + const statistics = await serverJson(request, '/prompt_manager/system/statistics'); + const { report } = await serverJson(request, '/prompt_manager/system/health'); + // The fixture the expected chips describe: its two sources, and the concept's checks plus the D95 ones. + const missing = report.checks.find(check => check.id === 'missing_files'); + expect(missing.sources.map(source => source.name)).toEqual(FIXTURE_SOURCES); + expect(diffInventory(mapStatisticsConcept(concept.statistics, statistics), production.statistics)).toEqual([]); + expect(diffInventory(mapHealthConcept(concept.health), production.health)).toEqual([]); + // D123: Maintenance's notice is the concept's own, delivered ahead of the rest of the section. + expect(diffInventory(concept.notice, production.notice)).toEqual([]); + + // Unmapped, the dialogs differ only where D95 and D96 are recorded. + expect(diffInventory(concept.health, production.health).map(item => item.path).sort()).toEqual(UNMAPPED_HEALTH_PATHS); + expect(diffInventory(concept.statistics, production.statistics).map(item => item.path.split('.')[0]) + .filter((part, index, all) => all.indexOf(part) === index).sort()).toEqual(['body', 'tables']); + + // Mutation check: a changed chip token in the health report is noticed, then undone to an exact match again. + const undo = await mutate(page, { css: '#modal .operation .chip { background: rgb(255, 0, 0); }' }); + expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog)).map(item => item.path)) + .toEqual(['operations.chip.backgroundColor']); + await undo(); + expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog))).toEqual([]); + // A renamed check (database/health.py's label) is noticed against the concept's own label. + await page.evaluate(() => { document.querySelector('#modal .operation h3').textContent = 'Integrity checks'; }); + expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog)).map(item => item.path)) + .toEqual(['operations.rows.0.title']); + await page.evaluate(() => { document.querySelector('#modal .operation h3').textContent = 'Integrity check'; }); + // And a removed "Library totals" table and a changed caption in the statistics dialog. + await page.keyboard.press('Escape'); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click(); + await page.locator('#workspace-main').getByRole('button', { name: 'Explore', exact: true }).click(); + await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Usage statistics'); + const expectedStatistics = mapStatisticsConcept(concept.statistics, statistics); + const undoTable = await mutate(page, { remove: ['#modal-body > table:nth-of-type(2)'] }); + expect(diffInventory(expectedStatistics, await page.evaluate(measureDialog)).map(item => item.path)) + .toEqual(expect.arrayContaining(['body.length', 'tables.length'])); + await undoTable(); + await page.evaluate(() => { document.querySelector('#modal-body caption').textContent = 'Runs per category'; }); + expect(diffInventory(expectedStatistics, await page.evaluate(measureDialog)).map(item => item.path)).toEqual(['tables.0.caption']); +}); + +/** Runs in the page: Appearance's surface, every row's text and control, and the concept's row geometry and tokens. */ +function measureAppearance(mainSelector) { + const layout = document.querySelector(`${mainSelector} .settings-layout`); + const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null; + const name = node => [node.localName, ...[...node.classList].sort()].join('.'); + const text = node => node?.textContent.trim() ?? null; + const surface = layout.querySelector(':scope > section.surface'); + const control = row => { + const select = row.querySelector(':scope > select'); + if (select) return { kind: 'select', id: select.id, options: [...select.options].map(option => option.textContent), + selected: select.selectedOptions[0]?.textContent ?? null }; + const input = row.querySelector(':scope > input'); + if (input) return { kind: `input.${input.type}`, id: input.id, value: input.value }; + const toggle = row.querySelector(':scope > * input[type="checkbox"]'); + if (toggle) return { kind: 'switch', id: toggle.id, wrapper: name(toggle.parentElement), checked: toggle.checked, + ...css(toggle.parentElement, 'width', 'height') }; + const chip = row.querySelector(':scope > .chip'); + return chip ? { kind: name(chip), text: text(chip), ...css(chip, 'color', 'backgroundColor', 'fontSize') } : null; + }; + const rows = [...surface.querySelectorAll(':scope > .setting-row')]; + const operations = [...surface.querySelectorAll(':scope > .operation')]; + const firstRow = rows[0]; + return { + layout: [...layout.children].map(name), + surface: { tree: [...surface.children].map(name), heading: text(surface.querySelector(':scope > h2')), + intro: text(surface.querySelector(':scope > p')), + ...css(surface, 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius') }, + rows: rows.map(row => ({ tree: [...row.children].map(name), text: [...row.firstElementChild.children].map(node => [node.localName, text(node), + node.localName === 'label' ? node.htmlFor : null]), control: control(row) })), + operations: operations.map(row => ({ tree: [...row.children].map(name), title: text(row.querySelector('h3')), + description: text(row.querySelector('p')), action: text(row.querySelector('.row-actions > button')) })), + row: css(firstRow, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', 'borderTopWidth', + 'borderTopColor', 'marginTop'), + label: css(firstRow.querySelector('label'), 'fontSize', 'fontWeight', 'color'), + description: css(firstRow.querySelector('p'), 'fontSize', 'color', 'marginTop', 'maxWidth'), + select: css(firstRow.querySelector('select'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor', 'color', 'borderTopColor'), + heading3: css(surface.querySelector('.setting-row h3'), 'fontSize', 'fontWeight', 'color'), + number: css(surface.querySelector('.setting-row input[type="number"]'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor'), + pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth, + }; +} + +// D97: the rows Appearance adds for the legacy gallery settings, after the concept's own, as production renders them. +const ADDED_APPEARANCE_ROWS = [ + ['card-details', 'Show image details on cards', 'Show each card’s title and file details under the preview.', true], + ['viewer-details', 'Collapse viewer details by default', 'Open the image viewer with its details panel hidden.', false], +]; + +/** The concept's Appearance with the recorded deviations applied, given the server's real settings. */ +function mapAppearanceConcept(reference, settings) { + const toggleRow = reference.rows.find(row => row.control?.kind === 'switch'); + const added = ADDED_APPEARANCE_ROWS.map(([id, label, description, checked]) => ({ + tree: toggleRow.tree, text: [['label', label, id], ['p', description, null]], + control: { ...toggleRow.control, id, checked } })); + return { + ...reference, + // D97: two added toggles and the "Reset display defaults" operation close the surface. + surface: { ...reference.surface, tree: [...reference.surface.tree, 'div.setting-row', 'div.setting-row', 'div.operation'] }, + rows: [...reference.rows.map(row => row.control?.id === 'timeout' + // The node search timeout is the server's real `result_timeout`, not the concept's sample 30. + ? { ...row, control: { ...row.control, value: String(settings.result_timeout) } } : row), ...added], + operations: [{ tree: ['div', 'div.row-actions'], title: 'Reset display defaults', + description: 'Restore this browser’s display defaults, then save. Other settings are kept.', action: 'Reset' }], + }; +} + +async function conceptAppearance(page, theme) { + await openConcept(page, 'system', theme); + await page.locator('#main .section-nav').getByRole('button', { name: 'Appearance', exact: true }).click(); + await expect(page.locator('#main .settings-layout > section.surface > h2')).toHaveText('Browsing & display'); + await page.mouse.move(0, 0); + return page.evaluate(measureAppearance, '#main'); +} + +async function productionAppearance(page, theme) { + await openProduction(page, 'system/appearance', theme); + await expect(page.locator('#workspace-main #timeout')).toBeEnabled(); + await page.mouse.move(0, 0); + return page.evaluate(measureAppearance, '#workspace-main'); +} + +for (const theme of THEMES) for (const [width, height] of SIZES) { + test(`appearance matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => { + await page.setViewportSize({ width, height }); + const { settings } = await serverJson(request, '/prompt_manager/settings'); + const concept = await conceptAppearance(page, theme); + await page.screenshot({ path: testInfo.outputPath(`concept-appearance-${theme}-${width}.png`), fullPage: true }); + const production = await productionAppearance(page, theme); + await page.screenshot({ path: testInfo.outputPath(`production-appearance-${theme}-${width}.png`), fullPage: true }); + expect(diffInventory(mapAppearanceConcept(concept, settings), production)).toEqual([]); + expect(production.pageScroll).toBe(false); + }); +} + +test('appearance comparison detects mutations', async ({ page, request }) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const { settings } = await serverJson(request, '/prompt_manager/settings'); + const concept = await conceptAppearance(page, 'dark'); + const expected = mapAppearanceConcept(concept, settings); + const production = await productionAppearance(page, 'dark'); + expect(diffInventory(expected, production)).toEqual([]); + // Unmapped, the concept differs only where D97 and the real timeout are recorded. + expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 2).join('.')) + .filter((path, index, all) => all.indexOf(path) === index).sort()) + .toEqual(['operations.0', 'operations.length', 'rows.11', 'rows.12', 'rows.13', 'rows.length', 'surface.tree'].sort()); + const measure = () => page.evaluate(measureAppearance, '#workspace-main'); + const paths = async () => diffInventory(expected, await measure()).map(item => item.path); + // A removed row. + const undoRow = await mutate(page, { remove: ['#workspace-main .setting-row:nth-of-type(3)'] }); + expect(await paths()).toEqual(expect.arrayContaining(['rows.length', 'surface.tree.length'])); + await undoRow(); + // A changed label and a changed option. + await page.evaluate(() => { document.querySelector('#workspace-main label[for="density"]').textContent = 'Thumbnail sizes'; }); + expect(await paths()).toEqual(['rows.1.text.0.1']); + await page.evaluate(() => { document.querySelector('#workspace-main label[for="density"]').textContent = 'Thumbnail size'; }); + await page.evaluate(() => { document.querySelector('#workspace-main #image-quality option').textContent = 'Medium'; }); + expect(await paths()).toEqual(expect.arrayContaining(['rows.5.control.options.0'])); + await page.evaluate(() => { document.querySelector('#workspace-main #image-quality option').textContent = 'Optimized'; }); + // A swapped token. + const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' }); + expect(await paths()).toEqual(expect.arrayContaining(['row.borderTopColor'])); + await undoToken(); + expect(diffInventory(expected, await measure())).toEqual([]); +}); diff --git a/tests/browser/workflows.spec.mjs b/tests/browser/workflows.spec.mjs index 0ee4620c..ad42f378 100644 --- a/tests/browser/workflows.spec.mjs +++ b/tests/browser/workflows.spec.mjs @@ -354,39 +354,48 @@ test('prompt deletion requires confirmation and can retry after a network failur } }); -test('duplicate cleanup cancels safely and deletes only the selected file after a retry', async ({ page, request }) => { +// The duplicate scan reads the output root only, but the fixture's input root holds a different +// shared/same.png. A bare /images/serve/ URL tries every gallery root in turn, so once the +// output copy is deleted it answers 200 with the input copy: check each file in its own root. +const serveIn = (root, path) => `/prompt_manager/images/serve/${path.split(/[\\/]/).map(encodeURIComponent).join('/')}?root=${root}`; +const rootOf = url => new URL(url, 'http://gallery.test').searchParams.get('root'); + +// Which copy the scan keeps depends on file times: the fixture stamps shared/same.png with the +// wall clock, and the kernel can stamp its bulk copies a fraction of a millisecond earlier. +// listOrder rewrites the finished scan's groups so a test can pick the order it needs. +async function openDuplicateScan(page, request, listOrder = null) { await expect(await request.post('/__test__/reset', { data: { extra_images: 2 } })).toBeOK(); + if (listOrder) { + await page.route('**/prompt_manager/scan_duplicates/stream', async route => { + const response = await route.fetch(); + const body = (await response.text()).replace(/^data: (.*)$/gm, (line, json) => { + const event = JSON.parse(json); + if (event.type !== 'complete') return line; + for (const group of event.duplicates) group.images = listOrder(group.images); + return `data: ${JSON.stringify(event)}`; + }); + await route.fulfill({ response, body }); + }); + } await page.goto('/prompt_manager/gallery'); await expect(page.locator('#galleryGrid .image-item')).toHaveCount(4); const originals = await page.locator('#galleryGrid img[data-original]').evaluateAll(images => images.map(image => image.dataset.original)); + const output = rootOf(originals.find(url => url.includes('/bulk/'))); + const input = rootOf(originals.find(url => url.includes('/shared/same.png') && rootOf(url) !== output)); + expect(output).not.toBeNull(); + expect(input).not.toBeNull(); await page.locator('#settingsBtn').click(); await page.locator('#scanDuplicatesBtn').click(); await expect(page.locator('#duplicatesCount')).toHaveText('1'); const candidates = page.locator('.duplicate-checkbox'); await expect(candidates).toHaveCount(2); - const selected = await candidates.first().getAttribute('data-image-path'); - const retained = await candidates.last().getAttribute('data-image-path'); - const serve = path => `/prompt_manager/images/serve/${path.split(/[\\/]/).map(encodeURIComponent).join('/')}`; - await candidates.first().check(); - let deletes = 0; - page.on('request', request => { if (request.url().endsWith('/delete_duplicate_images')) deletes++; }); - confirmNextDialog(page, false, 'permanently delete 1 duplicate image files'); - await page.locator('#removeDuplicatesBtn').click(); - expect(deletes).toBe(0); - await expect(await request.get(serve(selected))).toBeOK(); - await page.route('**/prompt_manager/delete_duplicate_images', route => route.abort('failed'), { times: 1 }); - confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); - await page.locator('#removeDuplicatesBtn').click(); - await expect(page.getByText('Failed to delete duplicate images', { exact: true })).toBeVisible(); - await expect(page.locator('#removeDuplicatesBtn')).toBeEnabled(); - await expect(await request.get(serve(selected))).toBeOK(); - confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); - const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images')); - await page.locator('#removeDuplicatesBtn').click(); - expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 }); - await expect(page.locator('#duplicatesModal')).toBeHidden(); - expect((await request.get(serve(selected))).status()).toBe(404); - await expect(await request.get(serve(retained))).toBeOK(); + return { originals, output, input, candidates }; +} + +async function expectOnlySelectedGone(page, request, { originals, output, input }, selected, retained) { + expect((await request.get(serveIn(output, selected))).status()).toBe(404); + await expect(await request.get(serveIn(output, retained))).toBeOK(); + await expect(await request.get(serveIn(input, 'shared/same.png'))).toBeOK(); await page.reload(); await expect(page.locator('#galleryGrid .image-item')).toHaveCount(3); const remaining = await page.locator('#galleryGrid img[data-original]').evaluateAll(images => images.map(image => image.dataset.original)); @@ -395,4 +404,47 @@ test('duplicate cleanup cancels safely and deletes only the selected file after expect(originals).toContain(url); await expect(await request.get(url)).toBeOK(); } +} + +test('duplicate cleanup cancels safely and deletes only the selected file after a retry', async ({ page, request }) => { + const scan = await openDuplicateScan(page, request); + const { output, candidates } = scan; + const selected = await candidates.first().getAttribute('data-image-path'); + const retained = await candidates.last().getAttribute('data-image-path'); + await candidates.first().check(); + let deletes = 0; + page.on('request', request => { if (request.url().endsWith('/delete_duplicate_images')) deletes++; }); + confirmNextDialog(page, false, 'permanently delete 1 duplicate image files'); + await page.locator('#removeDuplicatesBtn').click(); + expect(deletes).toBe(0); + await expect(await request.get(serveIn(output, selected))).toBeOK(); + await page.route('**/prompt_manager/delete_duplicate_images', route => route.abort('failed'), { times: 1 }); + confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); + await page.locator('#removeDuplicatesBtn').click(); + await expect(page.getByText('Failed to delete duplicate images', { exact: true })).toBeVisible(); + await expect(page.locator('#removeDuplicatesBtn')).toBeEnabled(); + await expect(await request.get(serveIn(output, selected))).toBeOK(); + confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); + const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images')); + await page.locator('#removeDuplicatesBtn').click(); + expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 }); + await expect(page.locator('#duplicatesModal')).toBeHidden(); + await expectOnlySelectedGone(page, request, scan, selected, retained); +}); + +test('deleting a duplicate whose path another gallery root also holds removes only the scanned copy', async ({ page, request }) => { + // The order CI saw: a bulk copy stamped first, so shared/same.png is the first Delete candidate. + const ciOrder = ['bulk/0000.png', 'shared/same.png', 'bulk/0001.png']; + const rank = image => ciOrder.indexOf(image.path.replace(/\\/g, '/')); + const scan = await openDuplicateScan(page, request, images => [...images].sort((a, b) => rank(a) - rank(b))); + const selected = await scan.candidates.first().getAttribute('data-image-path'); + const retained = await scan.candidates.last().getAttribute('data-image-path'); + expect(selected.replace(/\\/g, '/')).toBe('shared/same.png'); + await scan.candidates.first().check(); + confirmNextDialog(page, true, 'permanently delete 1 duplicate image files'); + const deleted = page.waitForResponse(response => response.url().endsWith('/delete_duplicate_images')); + await page.locator('#removeDuplicatesBtn').click(); + expect((await (await deleted).json())).toMatchObject({ success: true, deleted_count: 1, failed_count: 0 }); + await expect(page.locator('#duplicatesModal')).toBeHidden(); + await expectOnlySelectedGone(page, request, scan, selected, retained); }); diff --git a/tests/browser/workspace-activity.spec.mjs b/tests/browser/workspace-activity.spec.mjs index cbd848c0..72e95f28 100644 --- a/tests/browser/workspace-activity.spec.mjs +++ b/tests/browser/workspace-activity.spec.mjs @@ -284,6 +284,19 @@ test('reload during a scan reattaches to the running job', async ({ page, reques await expect(pill(page)).toBeHidden(); }); +test('an empty Activity says no job has run, and a list that cannot be read says so', async ({ page }) => { + // The empty and failed states were only reached in passing by other tests, so their coverage depended on timing. + const empty = main(page).locator('section.surface > p.empty-state'); + await page.goto(`${workspace}#/activity`); + await expect(empty).toHaveText('No jobs have run since ComfyUI started. Scans and other long operations appear here.'); + await expect(empty).not.toHaveAttribute('role'); + await expect(items(page)).toHaveCount(0); + await page.route(url => url.pathname === '/prompt_manager/jobs', route => route.abort()); + await page.reload(); + await expect(empty).toHaveText('Activity could not be read. It is tried again shortly.'); + await expect(empty).toHaveAttribute('role', 'alert'); +}); + test('lost server clears the pill and marks jobs ended', async ({ page, request }) => { // A poll interval after the restart, then failed re-reads backing off for 1 + 2 + 4 s. test.setTimeout(90_000); @@ -724,6 +737,42 @@ test('a held legacy job shows no Cancel in the list or its view', async ({ page, await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']); }); +// Since #289 an item dropped its Cancel job only when it was already in the page, so the first render of jobs the +// store already knew (Activity opened from another page) kept a Cancel job on a job that had ended, and moved a +// running job's progress after its actions. The read Activity starts on opening is held, so that first render is +// what the page shows (the read's own render would otherwise repair it a moment later). +test('jobs that ended before Activity opened show no Cancel job, a running one keeps its progress first', async ({ page, request }) => { + await page.setViewportSize({ width: 1600, height: 1100 }); + const finished = await startJob(request, { title: 'Finished scan' }); + await finish(request, finished.id); + const cancelled = await startJob(request, { title: 'Cancelled scan' }); + await finish(request, cancelled.id, 'cancelled'); + await startJob(request, { title: 'Running scan' }); + // The page's first list read has all three jobs once the running one shows in the pill. + await page.goto(`${workspace}#/images`); + await expect(pill(page)).toBeVisible(); + let hold; + const held = new Promise(resolve => { hold = resolve; }); + // Only the first read is held; any later one goes through. + await page.route(url => url.pathname === '/prompt_manager/jobs', route => { + if (!hold) return route.continue(); + hold(route); + hold = null; + }); + await activityButton(page).click(); + const route = await held; + await expect(items(page)).toHaveCount(3); + const item = title => items(page).filter({ has: page.getByRole('link', { name: new RegExp(`${title}$`) }) }); + for (const ended of ['Finished scan', 'Cancelled scan']) { + await expect(item(ended).locator('progress, .row-actions')).toHaveCount(0); + await expect(item(ended).getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(0); + } + await expect(item('Running scan').getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(1); + expect(await item('Running scan').evaluate(node => [...node.children].map(child => child.localName + (child.className ? `.${child.className}` : '')))) + .toEqual(['div.panel-heading', 'p', 'progress', 'div.row-actions']); + await route.continue(); +}); + test('busy start names the running job', async ({ page, request }) => { await page.setViewportSize({ width: 1600, height: 1100 }); // The page's timers are Playwright's, so the toast's lifetime below is run out rather than waited for. diff --git a/tests/browser/workspace-images.spec.mjs b/tests/browser/workspace-images.spec.mjs index 4f544cf3..5727a80b 100644 --- a/tests/browser/workspace-images.spec.mjs +++ b/tests/browser/workspace-images.spec.mjs @@ -463,6 +463,50 @@ test('collection viewer ignores delayed media callbacks and keeps inspection ret await page.evaluate(()=>{location.hash='/overview';});await expect(page.locator('.viewer-container,#viewer-dialog')).toHaveCount(0); }); +test('collection viewer leaving an image Viewer.js has not built yet releases it at once',async({page})=>{ + await page.goto(workspace);await expect(cards(page)).toHaveCount(2); + let release,held;const waiting=new Promise(resolve=>{held=resolve;});const pending=new Promise(resolve=>{release=resolve;}); + await page.route('**/images/serve/shared/same.png?*',async route=>{ + const url=new URL(route.request().url()); + if(url.searchParams.get('root')==='0'&&url.searchParams.has('_pm_viewer')) {held();await pending;} + await route.continue().catch(()=>{}); + }); + await cards(page).first().getByRole('button',{name:'Open same.png',exact:true}).click();await waiting; + const preview=page.locator('dialog#viewer-dialog'); + await page.evaluate(()=>{window.unbuiltImage=document.querySelector('#viewer-stage > img');}); + await preview.getByRole('button',{name:'Next media',exact:true}).click(); + await expect(preview.getByRole('button',{name:'Zoom in',exact:true})).toHaveAttribute('aria-disabled','false'); + // The left image is off the stage. Viewer.js has not built it, so the viewer is destroyed and the src released at + // once: its late load must never build Viewer.js into an image without a parent (getComputedStyle(null)). + release(); + await expect.poll(()=>page.evaluate(()=>[window.unbuiltImage.isConnected,window.unbuiltImage.hasAttribute('src'),!!window.unbuiltImage.viewer])).toEqual([false,false,false]); + await page.evaluate(()=>{location.hash='/overview';});await expect(page.locator('.viewer-container,#viewer-dialog')).toHaveCount(0); +}); + +test('collection viewer closed while its image finishes loading never builds Viewer.js afterwards',async({page})=>{ + await page.goto(workspace);await expect(cards(page)).toHaveCount(2); + let held;const waiting=new Promise(resolve=>{held=resolve;}); + // The image is held for the whole test: its late load is delivered below, at the moment Viewer.js cannot cancel. + await page.route('**/images/serve/shared/same.png?*',async route=>{ + const url=new URL(route.request().url()); + if(url.searchParams.get('root')==='0'&&url.searchParams.has('_pm_viewer')) {held();return;} + await route.continue().catch(()=>{}); + }); + await cards(page).first().getByRole('button',{name:'Open same.png',exact:true}).click();await waiting; + const preview=page.locator('dialog#viewer-dialog'); + // Viewer.js's own cancel removes its image listeners only while `image.complete` is false. An image that has + // finished downloading but whose load event is still queued is already complete: close in that state, then deliver it. + await page.evaluate(()=>{window.finishingImage=document.querySelector('#viewer-stage > img');Object.defineProperty(window.finishingImage,'complete',{get:()=>true});}); + await page.keyboard.press('Escape');await expect(preview).toHaveCount(0); + const after=await page.evaluate(async()=>{ + const image=window.finishingImage;image.dispatchEvent(new Event('load')); + // Viewer.js builds an inline viewer in a zero-delay timeout after its image loads; this runs after it. + await new Promise(resolve=>setTimeout(resolve)); + return {src:image.hasAttribute('src'),built:!!image.parentNode?.querySelector('.viewer-container')}; + }); + expect(after).toEqual({src:false,built:false}); +}); + test('collection viewer describes video transforms and releases native playback on navigation',async({page,request})=>{ await expect(await request.post('/__test__/reset',{data:{include_video:true}})).toBeOK();await page.goto(workspace); await page.locator('article[data-image-key="0:clip.webm"] .image-open').click(); diff --git a/tests/browser/workspace-system.spec.mjs b/tests/browser/workspace-system.spec.mjs new file mode 100644 index 00000000..ca2da6e7 --- /dev/null +++ b/tests/browser/workspace-system.spec.mjs @@ -0,0 +1,617 @@ +import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs'; +import { test, expect } from './fixtures.mjs'; +import { seedConcept } from './concept-inventory.mjs'; + +// System › Overview (#249 System S2 Task 6) on the concept-shaped real fixture: the metrics, rows, Statistics dialog and +// Health report read the server, never sample values (D93, D95, D96). The hooks are workspace_system_fixture.py's. +const workspace = '/prompt_manager/workspace'; +const OVERVIEW_URL = '/prompt_manager/system/overview'; +const THUMBNAILS_URL = '/prompt_manager/images/thumbnails/status'; +const number = value => value.toLocaleString('en-US'); + +test.beforeEach(async ({ page, request, browserName }) => { + await startCoverage(page, browserName); + page.manifest = await seedConcept(request); + await expect(await request.post('/__test__/system/reset')).toBeOK(); + await expect(await request.post('/__test__/jobs/restart')).toBeOK(); + page.systemErrors = []; + page.on('pageerror', error => page.systemErrors.push(error.message)); +}); + +test.afterEach(async ({ page, request, browserName }) => { + await saveCoverage(page, browserName); + await request.post('/__test__/release'); + await request.post('/__test__/system/reset'); + expect(page.systemErrors).toEqual([]); +}); + +async function serverJson(request, url) { + const response = await request.get(url); + await expect(response).toBeOK(); + return response.json(); +} + +const layoutOf = page => page.locator('#workspace-main .settings-layout'); + +async function openOverview(page) { + await page.goto(`${workspace}#/system/overview`); + await expect(layoutOf(page).locator(':scope > .summary-metrics')).toBeVisible(); + return layoutOf(page); +} + +/** The rows of one Overview surface: `[h3, p, button]`. */ +function rowsOf(layout, heading) { + return layout.locator('.two-col > section.surface').filter({ has: layout.page().getByRole('heading', { name: heading, exact: true }) }) + .locator('.operation').evaluateAll(rows => rows.map(row => [row.querySelector('h3').textContent, + row.querySelector('p').textContent, row.querySelector('.row-actions button').textContent])); +} + +test('overview shows real counts and Not checked', async ({ page, request }) => { + const writes = []; + page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) writes.push(item.url()); }); + const overview = await serverJson(request, OVERVIEW_URL); + const thumbnails = await serverJson(request, THUMBNAILS_URL); + // The concept fixture: six positives, their saved negatives and 24 linked images, all with thumbnails. + expect([overview.library.positive_prompts, overview.images.indexed, thumbnails.pending]).toEqual([6, 24, 0]); + // I2: the Overview never walks the sources for thumbnails; the row reads the status route itself. + expect(overview).not.toHaveProperty('thumbnails'); + const layout = await openOverview(page); + expect(await layout.locator('.summary-metrics > .metric').evaluateAll(items => items.map(item => + [item.querySelector('strong').textContent, item.querySelector('span').textContent]))).toEqual([ + [number(overview.library.positive_prompts), 'Positive prompts'], + [number(overview.library.negative_variants), 'Negative variants'], + [number(overview.images.indexed), 'Images'], + ['Not checked', 'Database integrity'], + ]); + expect(await rowsOf(layout, 'Library health')).toEqual([ + ['Ambiguous image links', 'Not available yet.', 'Review metadata'], + ['Backup status', 'No retained backup record.', 'View backups'], + ['No missing thumbnails', 'Generate only missing previews when necessary.', 'Review thumbnails'], + ]); + expect(await rowsOf(layout, 'Workspace preferences')).toEqual([ + ['One display configuration', 'Grid and density settings are shared across views.', 'Appearance'], + ['2 output sources', 'Monitoring and enrichment share one source list.', 'Manage sources'], + ['Statistics', 'Runs and ratings by category, with library totals.', 'Explore'], + ]); + expect(writes).toEqual([]); +}); + +test('Run health checks opens Health report and updates metric 4', async ({ page, request }) => { + await page.goto(`${workspace}#/system/maintenance`); + const layout = layoutOf(page); + await expect(layout.locator(':scope > .notice h2')).toHaveText('Check first. Review changes. Then apply.'); + // The rest of Maintenance is still the existing tools' (D94). + await expect(layout.locator(':scope > section.handoff h2')).toHaveText('Your current tools are ready'); + const run = layout.getByRole('button', { name: 'Run health checks', exact: true }); + await run.click(); + const dialog = page.locator('dialog#modal[open]'); + await expect(dialog.locator('#modal-title')).toHaveText('Health report'); + const { report } = await serverJson(request, '/prompt_manager/system/health'); + expect(report.status).toBe('incomplete'); + expect(await dialog.locator('.operation-list > .operation').evaluateAll(rows => rows.map(row => + [row.querySelector('h3').textContent, row.querySelector('span.chip').textContent]))).toEqual([ + ['Integrity check', 'Passed'], ['Foreign keys & pair references', 'Passed'], ['Canonical hashes', 'Passed'], + ['Pair uniqueness', 'Passed'], ['Missing files · output', 'Passed'], ['Missing files · input', 'Passed'], + ['Ambiguous metadata', 'Not available yet'], ['Consistency', 'Passed'], + ]); + await expect(dialog.locator('#modal-body > p.small')).toHaveText(/^Checked .+\. Some checks could not run, so this report is incomplete\.$/); + await expect(dialog.locator('.dialog-actions button')).toHaveText(['Close', 'Review missing links']); + await dialog.getByRole('button', { name: 'Close', exact: true }).click(); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); + await expect(run).toBeFocused(); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click(); + await expect(layout.locator('.metric').nth(3).locator('strong')).toHaveText('Incomplete'); + + // A missing file and an unmounted source: the report counts the readable sources and names the other one. + await expect(await request.post('/__test__/system/missing-file')).toBeOK(); + await expect(await request.post('/__test__/system/missing-source')).toBeOK(); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click(); + await layout.getByRole('button', { name: 'Run health checks', exact: true }).click(); + await expect(dialog.locator('.operation').filter({ hasText: 'Missing files' }).locator('.chip')) + .toHaveText(['1 link needs review', 'Passed', 'Not available']); + await expect(dialog.locator('.operation h3').filter({ hasText: 'Missing files' })) + .toHaveText(['Missing files · output', 'Missing files · input', 'Missing files · unmounted-drive']); + // Review missing links goes to Maintenance, where the cleanup's existing tool is linked. + await dialog.getByRole('button', { name: 'Review missing links', exact: true }).click(); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); + await expect(page).toHaveURL(`${workspace}#/system/maintenance`); + await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click(); + await expect(layout.locator('.metric').nth(3).locator('strong')).toHaveText('1 issue'); +}); + +test('a health check that fails says so and keeps the last result', async ({ page, request }) => { + await expect(await request.post('/__test__/system/fail', { data: { path: '/prompt_manager/system/health' } })).toBeOK(); + await page.goto(`${workspace}#/system/maintenance`); + const run = layoutOf(page).getByRole('button', { name: 'Run health checks', exact: true }); + await run.click(); + await expect(page.locator('#workspace-toast')).toHaveText('The health check could not finish. Nothing was changed.'); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); + await expect(run).not.toHaveAttribute('aria-disabled', 'true'); + await openOverview(page); + await expect(layoutOf(page).locator('.metric').nth(3).locator('strong')).toHaveText('Not checked'); +}); + +test('thumbnail row reflects the status endpoint', async ({ page, request }) => { + await expect(await request.post('/__test__/system/unthumbnailed', { data: { count: 3 } })).toBeOK(); + const status = await serverJson(request, '/prompt_manager/images/thumbnails/status'); + expect(status.pending).toBe(3); + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) }); + await expect(row.locator('h3')).toHaveText('3 missing thumbnails'); + // Until the thumbnail review dialog (S5), Review thumbnails opens Sources. + await row.getByRole('button', { name: 'Review thumbnails', exact: true }).click(); + await expect(page).toHaveURL(`${workspace}#/system/sources`); +}); + +test('the thumbnail row paints when its own status read answers', async ({ page, request }) => { + const held = async () => (await serverJson(request, '/__test__/hold')).held[THUMBNAILS_URL] ?? 0; + await expect(await request.post('/__test__/hold', { data: { path: THUMBNAILS_URL } })).toBeOK(); + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) }); + // The other values do not wait for the walk of every source. + await expect(row.locator('h3')).toHaveText('Checking thumbnails…'); + await expect(layout.locator('.summary-metrics > .metric')).toHaveCount(4); + await expect.poll(held).toBe(1); + await expect(await request.post('/__test__/release')).toBeOK(); + await expect(row.locator('h3')).toHaveText('No missing thumbnails'); +}); + +test('a failed thumbnail read is unavailable, never zero', async ({ page }) => { + await page.route(`**${THUMBNAILS_URL}`, route => route.fulfill({ status: 500, contentType: 'application/json', + body: JSON.stringify({ success: false, error: 'Injected failure' }) })); + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ has: page.getByRole('button', { name: 'Review thumbnails', exact: true }) }); + await expect(row.locator('h3')).toHaveText('Thumbnail status unavailable'); + await expect(layout.getByRole('alert')).toHaveCount(0); + await expect(layout.locator('.metric').nth(2).locator('strong')).toHaveText('24'); +}); + +test('ambiguous row reads Not available yet (Q6)', async ({ page }) => { + const layout = await openOverview(page); + const row = layout.locator('.operation').filter({ hasText: 'Ambiguous image links' }); + await expect(row.locator('p')).toHaveText('Not available yet.'); + const review = row.getByRole('button', { name: 'Review metadata', exact: true }); + await expect(review).toHaveAttribute('aria-disabled', 'true'); + await review.click({ force: true }); + await expect(page).toHaveURL(`${workspace}#/system/overview`); +}); + +test('View backups / Appearance / Manage sources switch sections', async ({ page }) => { + for (const [label, section, pressed] of [['View backups', 'backups', 'Backup & export'], ['Appearance', 'appearance', 'Appearance'], + ['Manage sources', 'sources', 'Sources & scans']]) { + const layout = await openOverview(page); + await layout.getByRole('button', { name: label, exact: true }).click(); + await expect(page).toHaveURL(`${workspace}#/system/${section}`); + await expect(page.locator('#workspace-main .section-nav [aria-pressed="true"]')).toHaveText(pressed); + await expect(layout.locator(':scope > .summary-metrics')).toHaveCount(0); + } +}); + +test('statistics dialog lists runs by category and library totals', async ({ page, request }) => { + const statistics = await serverJson(request, '/prompt_manager/system/statistics'); + expect(statistics.by_category.length).toBeGreaterThan(1); + const layout = await openOverview(page); + const explore = layout.getByRole('button', { name: 'Explore', exact: true }); + await explore.click(); + const dialog = page.locator('dialog#modal[open]'); + await expect(dialog.locator('#modal-title')).toHaveText('Usage statistics'); + const tables = dialog.locator('#modal-body > table'); + await expect(tables.locator('caption')).toHaveText(['Runs by category', 'Library totals']); + await expect(tables.first().locator('thead th')).toHaveText(['Category', 'Runs', 'Avg. rating']); + const cells = table => table.locator('tbody tr').evaluateAll(rows => rows.map(row => [...row.cells].map(cell => cell.textContent))); + expect(await cells(tables.first())).toEqual(statistics.by_category.map(row => [row.category ?? 'Uncategorized', + number(row.runs), row.average_rating === null ? 'Not rated' : `${number(row.average_rating)} / 5`])); + expect(await cells(tables.nth(1))).toEqual([['Prompts', number(statistics.totals.total_prompts)], + ['Tags', number(statistics.totals.total_tags)], ['Images', number(statistics.totals.total_images)], + ['Rated images', number(statistics.rated_images)]]); + await page.keyboard.press('Escape'); + await expect(explore).toBeFocused(); + // A statistics read that fails says so; the Overview stays. + await expect(await request.post('/__test__/system/fail', { data: { path: '/prompt_manager/system/statistics' } })).toBeOK(); + await explore.click(); + await expect(page.locator('#workspace-toast')).toHaveText('Statistics could not be loaded. Your saved data has not changed.'); + await expect(page.locator('dialog#modal[open]')).toHaveCount(0); +}); + +test('load failure shows Retry and no zeros', async ({ page, request }) => { + await expect(await request.post('/__test__/system/fail', { data: { path: OVERVIEW_URL } })).toBeOK(); + await page.goto(`${workspace}#/system/overview`); + const layout = layoutOf(page); + await expect(layout.getByRole('alert')).toHaveText('The overview could not be loaded. Your saved data has not changed.'); + await expect(layout.locator('.metric')).toHaveCount(0); + await expect(layout).not.toContainText('0'); + await layout.getByRole('button', { name: 'Retry', exact: true }).click(); + await expect(layout.locator('.summary-metrics > .metric')).toHaveCount(4); + await expect(layout.getByRole('alert')).toHaveCount(0); +}); + +test('out-of-order overview responses never repaint a newer section', async ({ page, request }) => { + const held = async () => (await serverJson(request, '/__test__/hold')).held[OVERVIEW_URL] ?? 0; + await expect(await request.post('/__test__/hold', { data: { path: OVERVIEW_URL } })).toBeOK(); + await page.goto(`${workspace}#/system/overview`); + const layout = layoutOf(page); + await expect(layout.getByRole('status')).toHaveText('Loading the overview…'); + await expect.poll(held).toBe(1); + // Leave for Appearance, come back (a second read), then leave again before either answers. + const nav = page.locator('#workspace-main .section-nav'); + await nav.getByRole('button', { name: 'Appearance', exact: true }).click(); + await nav.getByRole('button', { name: 'Overview', exact: true }).click(); + await expect.poll(held).toBe(2); + await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click(); + await expect(await request.post('/__test__/release')).toBeOK(); + await expect.poll(held).toBe(0); + // Both late answers reached a section that had left: Sources keeps its own body. + await expect(layout.locator(':scope > section.handoff')).toBeVisible(); + await page.evaluate(() => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve)))); + await expect(layout.locator('.summary-metrics')).toHaveCount(0); + expect(await layout.evaluate(node => [...node.children].map(child => child.className))).toEqual(['surface handoff', 'save-row']); + // Back on Overview, one fresh read paints one set of metrics. + await nav.getByRole('button', { name: 'Overview', exact: true }).click(); + await expect(layout.locator('.summary-metrics')).toHaveCount(1); + await expect(layout.locator('.two-col')).toHaveCount(1); +}); + +// System › Appearance (#249 System S3 Task 8): the browser display defaults (`gallerySettings`, preferences.mjs) and the +// two host fields behind the one Save row, and the consumers that read them on their next visit. +const SETTINGS_URL = '/prompt_manager/settings'; +const INSPECT_PATH = '/prompt_manager/images/inspect/'; +const statusOf = page => page.locator('#workspace-main #settings-status'); +const saveOf = page => page.locator('#workspace-main .save-row > button'); +const storedSettings = page => page.evaluate(() => JSON.parse(localStorage.getItem('gallerySettings') || 'null')); +const imageCards = page => page.locator('#workspace-main article.media-card'); +const panel = page => page.getByRole('complementary', { name: 'Selected image details', exact: true }); + +async function openAppearance(page) { + await page.goto(`${workspace}#/system/appearance`); + await expect(page.locator('#workspace-main .settings-layout > section.surface > h2')).toHaveText('Browsing & display'); + // The host rows are enabled once the server settings answered. + await expect(page.locator('#timeout')).toBeEnabled(); +} + +async function saveAppearance(page, text) { + await expect(saveOf(page)).not.toHaveAttribute('aria-disabled', 'true'); + await saveOf(page).click(); + await expect(statusOf(page)).toHaveText(text); +} + +/** Set toggles (by row id) through their Toolkit switches. */ +async function toggle(page, values) { + for (const [id, checked] of Object.entries(values)) await page.locator(`#${id}`).setChecked(checked, { force: true }); +} + +/** + * A request barrier: one request issued and answered after everything the page has asked for so far, so a read the + * last action started has been seen (Playwright reports a page's requests in order). + */ +async function requestBarrier(page) { + expect(await page.evaluate(url => fetch(url, { cache: 'no-store' }).then(response => response.ok), SETTINGS_URL)).toBe(true); +} + +async function openImages(page) { + await page.goto(`${workspace}#/images`); + await expect(imageCards(page).first()).toBeVisible(); +} + +test('defaults apply to a new visit but not to the current view or a saved view', async ({ page, request }) => { + const created = await request.post('/prompt_manager/saved-views', { data: { page: 'images', name: 'Everything', + state: { q: '', category: '', sort: 'newest', source: 'all', folder: '', media: 'all', tags: [], favorites: false } } }); + expect(created.status()).toBe(201); + const view = (await created.json()).view; + try { + await openImages(page); + // The current view: the toolbar sets List. + await page.getByRole('group', { name: 'Image layout' }).getByRole('button', { name: 'List', exact: true }).click(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); + await openAppearance(page); + await expect(page.locator('#default-view')).toHaveValue('grid'); + await page.locator('#default-view').selectOption('masonry'); + await page.locator('#density').selectOption('compact'); + await expect(statusOf(page)).toHaveText('Unsaved changes.'); + await saveAppearance(page, 'Saved in this browser.'); + const stored = await storedSettings(page); + expect([stored.workspaceLayout, stored.workspaceDensity]).toEqual(['masonry', 'compact']); + // Back in Images (the page session's last visit): the current view keeps its own layout and size. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'comfortable'); + // The saved view opens as before. + await page.goto(`${workspace}#/saved/${view.id}`); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'grid'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'comfortable'); + // A new visit starts from the defaults. + await openImages(page); + await page.reload(); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'masonry'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'compact'); + await expect(page.getByRole('group', { name: 'Image layout' }).getByRole('button', { name: 'Masonry', exact: true })) + .toHaveAttribute('aria-pressed', 'true'); + } finally { + await request.delete(`/prompt_manager/saved-views/${view.id}`); + } +}); + +test('the defaults reach the Images page only; embedded galleries keep their own presentation', async ({ page }) => { + await openAppearance(page); + await page.locator('#default-view').selectOption('list'); + await page.locator('#density').selectOption('compact'); + await saveAppearance(page, 'Saved in this browser.'); + // The prompt history's shared gallery has no toolbar to change them, so it opens as it always did. + await page.goto(`${workspace}#/library/${page.manifest.prompts[0].positive_id}`); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('#workspace-main .image-grid')).toHaveAttribute('data-layout', 'grid'); + await expect(page.locator('#workspace-main .image-grid')).toHaveAttribute('data-density', 'comfortable'); + // The Images page starts from them. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); + await expect(page.locator('.image-grid')).toHaveAttribute('data-density', 'compact'); +}); + +test('metadata off shows an explicit load and loads once on request', async ({ page }) => { + await openAppearance(page); + await toggle(page, { 'metadata-auto': false }); + await saveAppearance(page, 'Saved in this browser.'); + const reads = []; + page.on('request', item => { if (new URL(item.url()).pathname.startsWith(INSPECT_PATH)) reads.push(item.url()); }); + await openImages(page); + const first = imageCards(page).first(); + await first.locator('.image-card-actions button').first().click(); + await expect(panel(page).getByRole('status')).toHaveText('Metadata loads when you ask for it.'); + const load = panel(page).getByRole('button', { name: 'Load metadata', exact: true }); + await expect(load).toBeVisible(); + expect(reads).toEqual([]); + await load.click(); + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + expect(reads).toHaveLength(1); + // The same file again reads nothing more (the cache is on). + await first.locator('.image-card-actions button').first().click(); + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + await requestBarrier(page); + expect(reads).toHaveLength(1); + // The viewer's details use the same inspector: another file waits there too, and loads on request. + await imageCards(page).nth(1).locator('.image-open').click(); + const viewerDetails = page.locator('#viewer-details'); + await expect(viewerDetails.getByRole('status')).toHaveText('Metadata loads when you ask for it.'); + expect(reads).toHaveLength(1); + await viewerDetails.getByRole('button', { name: 'Load metadata', exact: true }).click(); + await expect(viewerDetails.locator('.panel-heading h2')).toBeVisible(); + expect(reads).toHaveLength(2); +}); + +test('cache off rereads the current selection; on reuses the one current record', async ({ page }) => { + const reads = []; + page.on('request', item => { if (new URL(item.url()).pathname.startsWith(INSPECT_PATH)) reads.push(item.url()); }); + for (const cache of [true, false]) { + if (!cache) { + await openAppearance(page); + await toggle(page, { 'metadata-cache': false }); + await saveAppearance(page, 'Saved in this browser.'); + await page.reload(); + } + await openImages(page); + const info = imageCards(page).first().locator('.image-card-actions button').first(); + await info.click(); + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + await expect.poll(() => reads.length).toBeGreaterThan(0); + await requestBarrier(page); + reads.length = 0; + // Inspect the selected file twice more. + for (const _ of [1, 2]) { + const answered = cache ? null : page.waitForResponse(response => new URL(response.url()).pathname.startsWith(INSPECT_PATH)); + await info.click(); + if (answered) await answered; + await expect(panel(page).locator('.panel-heading h2')).toBeVisible(); + } + await requestBarrier(page); + expect(reads, `cache ${cache}`).toHaveLength(cache ? 0 : 2); + } +}); + +test('paths hidden only changes display', async ({ page }) => { + const rows = async () => page.locator('#viewer-details details').filter({ hasText: 'File & provenance' }).locator('dt') + .evaluateAll(terms => terms.map(term => term.textContent)); + const openViewer = async () => { + await openImages(page); + await imageCards(page).first().locator('.image-open').click(); + const provenance = page.locator('#viewer-details details').filter({ hasText: 'File & provenance' }); + await provenance.locator('summary').click(); + await expect(provenance.locator('dl')).toBeVisible(); + return provenance; + }; + let provenance = await openViewer(); + expect(await rows()).toContain('Path'); + const before = { show: await provenance.getByRole('button', { name: /^Show .* in folder$/ }).count(), + download: await provenance.getByRole('link', { name: 'Download original', exact: true }).getAttribute('href') }; + await page.keyboard.press('Escape'); + await openAppearance(page); + await toggle(page, { paths: false }); + await saveAppearance(page, 'Saved in this browser.'); + provenance = await openViewer(); + const shown = await rows(); + expect(shown).not.toContain('Path'); + expect(shown).toEqual(expect.arrayContaining(['Source', 'Size'])); + // Everything else is as it was: the folder action and the original download stay. + expect(await provenance.getByRole('button', { name: /^Show .* in folder$/ }).count()).toBe(before.show); + await expect(provenance.getByRole('link', { name: 'Download original', exact: true })).toHaveAttribute('href', before.download); +}); + +test('video defaults reach the viewer', async ({ page }) => { + await openAppearance(page); + await toggle(page, { 'video-auto': true, 'video-muted': false, 'video-loop': false }); + await saveAppearance(page, 'Saved in this browser.'); + expect(await page.evaluate(async () => (await import('/prompt_manager/js/workspace/media-viewer.mjs')).readViewerPreferences())) + .toEqual({ videoAutoplay: true, videoMute: false, videoLoop: false }); + const stored = await storedSettings(page); + expect([stored.videoAutoplay, stored.videoMute, stored.videoLoop]).toEqual([true, false, false]); +}); + +test('host fields round-trip through /settings with revision', async ({ page, request }) => { + const posts = []; + page.on('request', item => { if (item.method() === 'POST' && new URL(item.url()).pathname === SETTINGS_URL) posts.push(item.postDataJSON()); }); + const before = await serverJson(request, SETTINGS_URL); + await openAppearance(page); + await expect(page.locator('#open-mode')).toHaveValue(before.settings.webui_display_mode); + await expect(page.locator('#timeout')).toHaveValue(String(before.settings.result_timeout)); + // Only whole seconds from 0 to 300: anything else is highlighted and never sent. + for (const bad of ['301', '-1', '2.5']) { + await page.locator('#timeout').fill(bad); + await page.locator('#timeout').press('Tab'); + await expect(page.locator('#timeout')).toHaveAttribute('aria-invalid', 'true'); + await expect(statusOf(page)).toHaveText('Some changes were not saved. Review the highlighted settings.'); + await expect(saveOf(page)).toHaveAttribute('aria-disabled', 'true'); + } + await page.locator('#timeout').fill('0'); + await page.locator('#timeout').press('Tab'); + await expect(page.locator('#timeout')).not.toHaveAttribute('aria-invalid'); + await page.locator('#open-mode').selectOption('popup'); + await saveAppearance(page, 'Saved to the server configuration.'); + expect(posts).toEqual([{ webui_display_mode: 'popup', result_timeout: 0, expected_revision: before.revision }]); + const after = await serverJson(request, SETTINGS_URL); + expect([after.settings.webui_display_mode, after.settings.result_timeout]).toEqual(['popup', 0]); + expect(after.revision).not.toBe(before.revision); + // No browser key was written for a host field. + expect(await storedSettings(page)).toBeNull(); +}); + +test('server settings that cannot be read say so in their rows', async ({ page }) => { + await page.route(url => url.pathname === SETTINGS_URL, route => (route.request().method() === 'GET' + ? route.fulfill({ status: 500, contentType: 'application/json', body: JSON.stringify({ success: false, error: 'Injected failure' }) }) + : route.continue())); + await page.goto(`${workspace}#/system/appearance`); + for (const id of ['open-mode', 'timeout']) { + const row = page.locator('#workspace-main .setting-row').filter({ has: page.locator(`#${id}`) }); + await expect(row.locator('p.small')).toHaveText('Server settings could not be read.'); + await expect(page.locator(`#${id}`)).toBeDisabled(); + } + // The browser rows are this browser's, and work as before. + await expect(page.locator('#default-view')).toBeEnabled(); + await expect(page.locator('#workspace-main .setting-row p.small')).toHaveCount(2); +}); + +test('stale revision shows the conflict status', async ({ page, request }) => { + await openAppearance(page); + // Another page saves after this one read the settings. + const current = await serverJson(request, SETTINGS_URL); + await expect(await request.post(SETTINGS_URL, { data: { result_timeout: 42, expected_revision: current.revision } })).toBeOK(); + await page.locator('#open-mode').selectOption('popup'); + await saveAppearance(page, 'Settings changed elsewhere. Reload this section to see them.'); + await expect(page.locator('#open-mode')).toHaveAttribute('aria-invalid', 'true'); + // The draft is kept and the section now shows the other page's value. + await expect(page.locator('#open-mode')).toHaveValue('popup'); + await expect(page.locator('#timeout')).toHaveValue('42'); + expect((await serverJson(request, SETTINGS_URL)).settings.webui_display_mode).toBe('newtab'); + // With the reloaded revision, Save succeeds. + await saveAppearance(page, 'Saved to the server configuration.'); + expect((await serverJson(request, SETTINGS_URL)).settings.webui_display_mode).toBe('popup'); +}); + +test('reload keeps saved values; a new tab reads them', async ({ page, context }) => { + await openAppearance(page); + await page.locator('#image-quality').selectOption('high'); + await toggle(page, { 'card-details': false }); + await page.locator('#open-mode').selectOption('popup'); + await saveAppearance(page, 'Saved in this browser. Saved to the server configuration.'); + await page.reload(); + await expect(page.locator('#timeout')).toBeEnabled(); + await expect(page.locator('#image-quality')).toHaveValue('high'); + await expect(page.locator('#card-details')).not.toBeChecked(); + await expect(page.locator('#open-mode')).toHaveValue('popup'); + const other = await context.newPage(); + await openAppearance(other); + await expect(other.locator('#image-quality')).toHaveValue('high'); + await expect(other.locator('#card-details')).not.toBeChecked(); + // A change saved in one tab reaches the other's clean controls; a draft it overtakes is flagged and kept. + await other.locator('#density').selectOption('large'); + await page.locator('#density').selectOption('compact'); + await saveAppearance(page, 'Saved in this browser.'); + await expect(other.locator('#density')).toHaveValue('large'); + await expect(other.locator('#settings-status')).toHaveText('Settings changed elsewhere. Reload this section to see them.'); + await expect(other.locator('#density')).toHaveAttribute('aria-invalid', 'true'); + await page.locator('#image-quality').selectOption('low'); + await saveAppearance(page, 'Saved in this browser.'); + await expect(other.locator('#image-quality')).toHaveValue('low'); + await other.close(); +}); + +test('a change another tab saved while System was closed is a conflict on return, never saved over silently', async ({ page, context }) => { + await openAppearance(page); + await page.locator('#density').selectOption('compact'); + await expect(statusOf(page)).toHaveText('Unsaved changes.'); + // Leave System for Images: the draft stays for the page session, but System no longer listens. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(imageCards(page).first()).toBeVisible(); + const other = await context.newPage(); + await openAppearance(other); + await other.locator('#density').selectOption('large'); + await saveAppearance(other, 'Saved in this browser.'); + await other.close(); + // Back in System: the draft is kept and flagged, and nothing was written over the other tab's value. + await page.evaluate(() => { location.hash = '#/system/appearance'; }); + await expect(page.locator('#workspace-main .settings-layout > section.surface > h2')).toHaveText('Browsing & display'); + await expect(statusOf(page)).toHaveText('Settings changed elsewhere. Reload this section to see them.'); + await expect(page.locator('#density')).toHaveAttribute('aria-invalid', 'true'); + await expect(page.locator('#density')).toHaveValue('compact'); + expect((await storedSettings(page)).workspaceDensity).toBe('large'); + // Saving after the conflict was shown is an explicit choice: it wins. + await saveAppearance(page, 'Saved in this browser.'); + expect((await storedSettings(page)).workspaceDensity).toBe('compact'); + await expect(page.locator('#density')).not.toHaveAttribute('aria-invalid'); +}); + +test('a browser that refuses storage is saved for this tab only, never "Saved"', async ({ page }) => { + await openAppearance(page); + await page.evaluate(() => { + Storage.prototype.setItem = function setItem() { throw new DOMException('Quota exceeded', 'QuotaExceededError'); }; + }); + await page.locator('#default-view').selectOption('list'); + await saveAppearance(page, 'Saved for this tab only; browser storage is unavailable.'); + expect(await storedSettings(page)).toBeNull(); + // This tab's next Images visit still uses it. + await page.locator('#primary-nav a[data-route="images"]').click(); + await expect(imageCards(page).first()).toBeVisible(); + await expect(page.locator('.image-grid')).toHaveAttribute('data-layout', 'list'); +}); + +test('added rows reach their consumers', async ({ page }) => { + await openImages(page); + await expect(imageCards(page).first().locator('.media-info h3')).toBeVisible(); + await openAppearance(page); + await toggle(page, { 'card-details': false, 'viewer-details': true }); + await saveAppearance(page, 'Saved in this browser.'); + await page.reload(); + await openImages(page); + // Show image details on cards off: no title or file line; Info and Tag stay. + const card = imageCards(page).first(); + await expect(card.locator('.media-info h3')).toBeHidden(); + await expect(card.locator('.image-card-actions button')).toHaveCount(2); + // Collapse viewer details by default: the viewer opens with its details hidden behind one toggle. + await card.locator('.image-open').click(); + const details = page.locator('#viewer-details'); + await expect(page.locator('#viewer-dialog')).toBeVisible(); + await expect(details).toBeHidden(); + const show = page.locator('#viewer-toolbar').getByRole('button', { name: 'Show details', exact: true }); + await expect(show).toHaveAttribute('aria-expanded', 'false'); + await show.click(); + await expect(details).toBeVisible(); + await expect(page.locator('#viewer-toolbar').getByRole('button', { name: 'Hide details', exact: true })).toHaveAttribute('aria-expanded', 'true'); +}); + +test('reset restores defaults without touching other keys', async ({ page }) => { + await page.goto(`${workspace}#/system/appearance`); + await page.evaluate(() => localStorage.setItem('gallerySettings', JSON.stringify({ gridColumns: 5, defaultLimit: 200, + thumbnailPromptDismissed: 'kept', workspaceLayout: 'list', videoMute: false, showImageInfo: false }))); + await page.reload(); + await expect(page.locator('#timeout')).toBeEnabled(); + await expect(page.locator('#default-view')).toHaveValue('list'); + const reset = page.locator('#workspace-main .operation').filter({ hasText: 'Reset display defaults' }).getByRole('button', { name: 'Reset', exact: true }); + await reset.click(); + await expect(page.locator('#default-view')).toHaveValue('grid'); + await expect(page.locator('#video-muted')).toBeChecked(); + await expect(statusOf(page)).toHaveText('Unsaved changes.'); + await saveAppearance(page, 'Saved in this browser.'); + const stored = await storedSettings(page); + expect(stored).toEqual({ gridColumns: 5, defaultLimit: 200, thumbnailPromptDismissed: 'kept', workspaceLayout: 'grid', + videoMute: true, showImageInfo: true }); +}); diff --git a/tests/browser/workspace.spec.mjs b/tests/browser/workspace.spec.mjs index fff42a62..8d847a33 100644 --- a/tests/browser/workspace.spec.mjs +++ b/tests/browser/workspace.spec.mjs @@ -18,7 +18,8 @@ const routes = [ ['tags', 'Tags & categories', null], // #249 local Inspect: the native Inspect a file page; Overview's tool card still links the legacy metadata page. ['inspect', 'Inspect a file', null], - ['system', 'System', '/prompt_manager/admin'], + // #249 System S2: native System; its undelivered sections hand off per section (D94, asserted below). + ['system', 'System', null], // #249 D51: SAVED VIEWS' Favorites, Images filtered to favorites. ['favorites', 'Favorites', null], // #249 Activity-1: the concept's Activity page in the tools-and-system group. @@ -409,6 +410,8 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true); await sidebarLink(page, 'System').click(); + // System's Overview is native (S2 Task 6); a section not delivered yet still hands off (D94). + await main.locator('.section-nav').getByRole('button', { name: 'Integrations', exact: true }).click(); const handoff = main.locator('section.handoff'); await expect(handoff).toHaveClass(/\bsurface\b/); expect(await handoff.evaluate(surfaceStyle)).toMatchObject(SURFACE); @@ -417,6 +420,98 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true); }); +// #249 System S2 Task 5: the concept's renderSystem shell (app.js 698–706) at `#/system/
`, every section a D94 +// hand-off until its PR, and the Save row's truthful status (D92). +const SYSTEM_SECTION_LABELS = ['Overview', 'Appearance', 'Sources & scans', 'Integrations', 'Auto-tagging', 'Maintenance', + 'Backup & export', 'Diagnostics & logs']; +const SYSTEM_HASHES = ['overview', 'appearance', 'sources', 'integrations', 'tagging', 'maintenance', 'backups', 'diagnostics']; +const SYSTEM_HANDOFFS = { sources: ['/prompt_manager/admin', '/prompt_manager/gallery'], + maintenance: ['/prompt_manager/admin', '/prompt_manager/gallery'] }; + +test('System is native: the concept shell, section navigation, sections, hand-offs and the Save row', async ({ page }) => { + const mutations = []; + page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) mutations.push(item.url()); }); + await page.goto(`${workspace}#/system`); + const main = page.locator('#workspace-main'); + await expect(main.locator('h1')).toHaveText('System'); + await expect(main.locator('.heading p.muted')).toHaveText('Configuration, maintenance and recovery—with browsing kept in the workspace.'); + // D92: no "Sample configuration" chip. + await expect(main.locator('.heading .chip')).toHaveCount(0); + expect(await main.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`))) + .toEqual(['div.heading', 'div.section-nav', 'div.settings-layout']); + const nav = main.getByRole('group', { name: 'System sections', exact: true }); + await expect(nav.getByRole('button')).toHaveText(SYSTEM_SECTION_LABELS); + await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true'); + await expect(nav.locator('[aria-pressed="true"]')).toHaveCount(1); + await expect(page.locator('aside.sidebar a[href="#/system"]')).toHaveAttribute('aria-current', 'page'); + const layout = main.locator('.settings-layout'); + await expect(layout.locator(':scope > .save-row > span.small#settings-status')).toHaveText('No unsaved changes.'); + const save = layout.locator(':scope > .save-row > button.ca-primary'); + await expect(save).toHaveText('Save preferences'); + await expect(save).toHaveAttribute('aria-disabled', 'true'); + // Playwright waits on an aria-disabled control, so the ignored press is forced. + await save.click({ force: true }); + await expect(layout.locator('#settings-status')).toHaveText('No unsaved changes.'); + + for (const [index, label] of SYSTEM_SECTION_LABELS.entries()) { + const hash = SYSTEM_HASHES[index]; + const button = nav.getByRole('button', { name: label, exact: true }); + await button.click(); + await expect(page).toHaveURL(`${workspace}#/system/${hash}`); + await expect(button).toHaveAttribute('aria-pressed', 'true'); + await expect(nav.locator('[aria-pressed="true"]')).toHaveCount(1); + // The section changes in place: focus stays on the pressed section, not the page title. + await expect(button).toBeFocused(); + // S2 Task 6: Overview is native (workspace-system.spec.mjs), and so is Appearance (S3 Task 8); Maintenance shows + // its health-check notice above the hand-off (D123); every other section hands off. + if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible(); + expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`))) + .toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'div.save-row'], + maintenance: ['div.notice', 'section.surface handoff', 'div.save-row'] }[hash] + ?? ['section.surface handoff', 'div.save-row']); + if (hash === 'overview' || hash === 'appearance') continue; + const handoff = layout.locator('section.handoff'); + await expect(handoff.locator('h2')).toHaveText('Your current tools are ready'); + expect(await handoff.locator('a.workspace-link').evaluateAll(links => links.map(link => link.getAttribute('href')))) + .toEqual(SYSTEM_HANDOFFS[hash] ?? ['/prompt_manager/admin']); + await expect(layout.locator('#settings-status')).toHaveText('No unsaved changes.'); + await expect(page.locator('aside.sidebar a[href="#/system"]')).toHaveAttribute('aria-current', 'page'); + } + // Deep links, reload and history keep the section; an unknown section is Overview under its own hash. + await page.reload(); + await expect(nav.getByRole('button', { name: 'Diagnostics & logs', exact: true })).toHaveAttribute('aria-pressed', 'true'); + await page.goBack(); + await expect(page).toHaveURL(`${workspace}#/system/backups`); + await expect(nav.getByRole('button', { name: 'Backup & export', exact: true })).toHaveAttribute('aria-pressed', 'true'); + await page.goto(`${workspace}#/system/general`); + await expect(page).toHaveURL(`${workspace}#/system/overview`); + await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true'); + // The sidebar's System returns to the Overview section. + await nav.getByRole('button', { name: 'Integrations', exact: true }).click(); + await sidebarLink(page, 'System').click(); + await expect(page).toHaveURL(`${workspace}#/system`); + await expect(nav.getByRole('button', { name: 'Overview', exact: true })).toHaveAttribute('aria-pressed', 'true'); + // Leaving System and coming back renders it once, with a fresh section. + await sidebarLink(page, 'Images').click(); + await expect(main.locator('h1')).toHaveText('Images'); + await expect(main.locator('.section-nav')).toHaveCount(0); + await page.goBack(); + await expect(main.locator('.section-nav')).toHaveCount(1); + await expect(main.locator('.settings-layout')).toHaveCount(1); + expect(mutations).toEqual([]); +}); + +test('Overview opens native System from its System card (Q12)', async ({ page }) => { + await page.goto(`${workspace}#/overview`); + const card = page.locator('#workspace-main .tool-card').filter({ has: page.getByRole('heading', { name: 'System', exact: true }) }); + await expect(card.locator('p.small')).toHaveText('Configuration, maintenance and recovery—with browsing kept in the workspace.'); + const link = card.getByRole('link', { name: 'Open System', exact: true }); + await expect(link).toHaveAttribute('href', '#/system/overview'); + await link.click(); + await expect(page.locator('#workspace-main h1')).toHaveText('System'); + await expect(page.locator('#workspace-main .section-nav [aria-pressed="true"]')).toHaveText('Overview'); +}); + // Task A3: the concept sidebar (index.html h16-41, app.js nav()) rendered from real data (D27, D28). const NAV_TOTALS = [ ['images', 'Images', '/prompt_manager/images/output?mode=workspace&limit=1'], diff --git a/tests/browser/workspace_system_fixture.py b/tests/browser/workspace_system_fixture.py new file mode 100644 index 00000000..2413c1f0 --- /dev/null +++ b/tests/browser/workspace_system_fixture.py @@ -0,0 +1,144 @@ +"""System test hooks for the browser tests (#249 System S2 Task 6). + +The data is the concept-shaped real fixture (``POST /__test__/concept/reset``, +``workspace_concept_fixture.py``): its parents, saved pairs, images, ratings +and runs are what the Overview, the Statistics dialog and the health check +read. These per-server hooks only change what a System test needs on top: + +- ``POST /__test__/system/reset`` restores the configured sources, forgets the + session's health report and ends every injected failure; +- ``POST /__test__/system/fail`` ``{path, times}`` answers the next ``times`` + requests for ``path`` (a ``/prompt_manager/system/`` route) with the routes' + 500 error envelope; +- ``POST /__test__/system/missing-source`` configures one more source that + does not exist (an unmounted drive); +- ``POST /__test__/system/missing-file`` deletes the newest linked image's + file from disk, leaving its database row (a missing-file link); +- ``POST /__test__/system/unthumbnailed`` ``{count}`` writes ``count`` plain + PNGs with no thumbnail into the output folder. + +Delays use the shared request holds (``request_hold.py``). +""" + +import os +from pathlib import Path + +from aiohttp import web +from PIL import Image + +FAILABLE_PREFIX = "/prompt_manager/system/" +MAX_UNTHUMBNAILED = 50 + + +def register_system_commands(app, routes, api, root: Path): + # System › Appearance saves the host fields to this server's config.json (S3); + # reset restores both. The file is this server's own, inside its temp root: + # reset rewrites or deletes it, so a server whose configuration points + # anywhere else (a real config.json) refuses to register these hooks. + config_path = root / "config.json" + configured_path = os.environ.get("PROMPT_MANAGER_CONFIG_PATH") + if ( + configured_path is None + or Path(configured_path).resolve() != config_path.resolve() + ): + raise RuntimeError( + f"PROMPT_MANAGER_CONFIG_PATH must be {config_path}, not {configured_path!r}" + ) + from py.config import GalleryConfig, PromptManagerConfig + + configured = list(GalleryConfig.MONITORING_DIRECTORIES) + config_bytes = config_path.read_bytes() if config_path.exists() else None + host_values = ( + PromptManagerConfig.RESULT_TIMEOUT, + PromptManagerConfig.WEBUI_DISPLAY_MODE, + ) + failures: dict[str, int] = {} + + @web.middleware + async def inject_failures(request, handler): + left = failures.get(request.path, 0) + if left <= 0: + return await handler(request) + failures[request.path] = left - 1 + return web.json_response( + {"success": False, "code": None, "error": "Injected failure"}, status=500 + ) + + app.middlewares.append(inject_failures) + + def reset(): + failures.clear() + GalleryConfig.MONITORING_DIRECTORIES = list(configured) + api._health_report = None + PromptManagerConfig.RESULT_TIMEOUT, PromptManagerConfig.WEBUI_DISPLAY_MODE = ( + host_values + ) + if config_bytes is None: + config_path.unlink(missing_ok=True) + else: + # Replace in one step, as the server writes it (Windows preflight). + staged = config_path.with_name(config_path.name + ".reset") + staged.write_bytes(config_bytes) + os.replace(staged, config_path) + + @routes.post("/__test__/system/reset") + async def system_reset(request): + reset() + return web.json_response({"success": True}) + + @routes.post("/__test__/system/fail") + async def system_fail(request): + data = await request.json() + path, times = data.get("path"), data.get("times", 1) + if ( + not isinstance(path, str) + or not path.startswith(FAILABLE_PREFIX) + or isinstance(times, bool) + or not isinstance(times, int) + or not 1 <= times <= 10 + ): + raise web.HTTPBadRequest() + failures[path] = times + return web.json_response({"success": True}) + + @routes.post("/__test__/system/missing-source") + async def missing_source(request): + absent = str(root / "unmounted-drive") + if os.path.exists(absent): + raise web.HTTPConflict() + GalleryConfig.MONITORING_DIRECTORIES = [*configured, absent] + return web.json_response( + {"success": True, "configured": len(GalleryConfig.MONITORING_DIRECTORIES)} + ) + + @routes.post("/__test__/system/missing-file") + async def missing_file(request): + with api.db.model.get_connection() as conn: + row = conn.execute( + "SELECT file_path FROM generated_images ORDER BY id DESC LIMIT 1" + ).fetchone() + if row is None: + raise web.HTTPNotFound() + Path(row[0]).unlink() + api.invalidate_gallery_cache() + return web.json_response({"success": True, "filename": Path(row[0]).name}) + + @routes.post("/__test__/system/unthumbnailed") + async def unthumbnailed(request): + data = await request.json() + count = data.get("count") + if ( + isinstance(count, bool) + or not isinstance(count, int) + or not 1 <= count <= MAX_UNTHUMBNAILED + ): + raise web.HTTPBadRequest() + folder = root / "output" / "unthumbnailed" + folder.mkdir(exist_ok=True) + for number in range(count): + with Image.new("RGB", (64, 64), "#2b3442") as image: + image.save(folder / f"plain_{number + 1:03}.png") + api.invalidate_gallery_cache() + return web.json_response({"success": True}) + + return reset diff --git a/tests/js/gallery-settings.test.js b/tests/js/gallery-settings.test.js index c3681a29..a8fbf658 100644 --- a/tests/js/gallery-settings.test.js +++ b/tests/js/gallery-settings.test.js @@ -61,3 +61,50 @@ test("infinite scrolling is off by default and must be a boolean", () => { assert.equal(normalizeGallerySettings({ infiniteScroll: true }).infiniteScroll, true); assert.equal(normalizeGallerySettings({ infiniteScroll: "yes" }).infiniteScroll, false); }); + +test("the workspace keys have their own defaults and validation", () => { + const { WORKSPACE_LAYOUTS, WORKSPACE_DENSITIES } = require("../../web/js/gallery-settings.js"); + assert.equal(DEFAULT_SETTINGS.workspaceLayout, "grid"); + assert.equal(DEFAULT_SETTINGS.workspaceDensity, "comfortable"); + assert.deepEqual(WORKSPACE_LAYOUTS, ["grid", "list", "masonry"]); + assert.deepEqual(WORKSPACE_DENSITIES, ["comfortable", "large", "compact"]); + assert.equal(normalizeGallerySettings({ workspaceLayout: "masonry" }).workspaceLayout, "masonry"); + assert.equal(normalizeGallerySettings({ workspaceLayout: "mosaic" }).workspaceLayout, "grid"); + assert.equal(normalizeGallerySettings({ workspaceDensity: "large" }).workspaceDensity, "large"); + assert.equal(normalizeGallerySettings({ workspaceDensity: 3 }).workspaceDensity, "comfortable"); + // The legacy gallery's own view mode is separate: it has no masonry. + assert.equal(normalizeGallerySettings({ defaultViewMode: "masonry" }).defaultViewMode, "grid"); +}); + +test("a legacy gallery Save keeps the workspace keys and every other stored key", () => { + const fs = require("node:fs"); + const path = require("node:path"); + const vm = require("node:vm"); + const settings = require("../../web/js/gallery-settings.js"); + const storage = new Map([["gallerySettings", JSON.stringify({ workspaceLayout: "masonry", workspaceDensity: "compact", someOtherTool: 1 })]]); + const controls = { + lazyLoadingToggle: { checked: true }, imageQualitySelect: { value: "high" }, defaultViewModeSelect: { value: "list" }, + defaultLimitSelect: { value: "200" }, gridColumnsSelect: { value: "6" }, showImageInfoToggle: { checked: false }, + autoLoadMetadataToggle: { checked: true }, cacheMetadataToggle: { checked: true }, showFilePathsToggle: { checked: false }, + sidebarCollapsedByDefaultToggle: { checked: true }, infiniteScrollToggle: { checked: false }, debugModeToggle: { checked: false }, + apiTimeoutInput: { value: "30" }, checkThumbnailsAtStartup: { checked: true }, videoAutoplayToggle: { checked: true }, + videoMuteToggle: { checked: false }, videoLoopToggle: { checked: true }, + }; + const context = vm.createContext({ + document: { addEventListener() {}, getElementById: id => controls[id] }, + window: {}, console: { debug() {}, warn() {}, error() {} }, GallerySettings: settings, + localStorage: { getItem: key => storage.get(key) ?? null, setItem: (key, value) => storage.set(key, value) }, + }); + const file = path.join(__dirname, "../../web/js/gallery.js"); + const Manager = vm.runInContext(`${fs.readFileSync(file, "utf8")}\nGalleryManager;`, context, { filename: file }); + const manager = Object.create(Manager.prototype); + Object.assign(manager, { showNotification() {}, hideSettings() {}, applySettings() {} }); + manager.saveSettings(); + const saved = JSON.parse(storage.get("gallerySettings")); + assert.equal(saved.workspaceLayout, "masonry"); + assert.equal(saved.workspaceDensity, "compact"); + assert.equal(saved.someOtherTool, 1); + assert.equal(saved.defaultViewMode, "list"); + assert.equal(saved.showImageInfo, false); + assert.equal(saved.videoMute, false); +}); diff --git a/tests/js/workspace-activity-store.test.js b/tests/js/workspace-activity-store.test.js index 5fce2cda..82b69dac 100644 --- a/tests/js/workspace-activity-store.test.js +++ b/tests/js/workspace-activity-store.test.js @@ -182,6 +182,24 @@ test('start returns busy with the running job', async () => { world.dispose(); }); +test('track follows a job another route started, with no read of its own', async () => { + const { createActivityStore } = await source; + const { world, ...deps } = harness(); + const store = createActivityStore(deps); + const seen = []; + store.subscribe(() => seen.push(store.find(ID_A)?.state)); + const job = store.track(summary({ kind: 'health', title: 'Health check' })); + assert.equal(job.id, ID_A); + assert.equal(world.gets, 0, 'the started job is known without a list read'); + assert.deepEqual(world.open().map(stream => stream.url), [`/prompt_manager/jobs/${ID_A}/events`]); + assert.deepEqual(world.toasts, [], 'the route that started it says so itself'); + send(world.open()[0], summary({ kind: 'health', title: 'Health check', state: 'finished', progress: 100 })); + assert.deepEqual(seen, ['running', 'finished']); + assert.equal(world.open().length, 0, 'an ended job is no longer followed'); + assert.throws(() => store.track({ id: 'nope' }), /Invalid job/); + world.dispose(); +}); + test('a list that is not a job list counts as a failed read', async () => { const { createActivityStore } = await source; const { world, api, ...deps } = harness(); diff --git a/tests/js/workspace-images-state.test.js b/tests/js/workspace-images-state.test.js index b17aa9bf..bb81a6e9 100644 --- a/tests/js/workspace-images-state.test.js +++ b/tests/js/workspace-images-state.test.js @@ -478,3 +478,19 @@ test('a refresh deferred behind a page load is dropped when the stream is dispos await new Promise(resolve=>setImmediate(resolve)); assert.equal(requests.length,2,'no refresh runs after dispose'); }); + +test('defaults apply to a new visit but not to the current view or a saved view', async () => { + const { initialImagesView } = await source; + assert.equal(typeof initialImagesView, 'function', 'initialImagesView is exported'); + const preferences = { workspaceLayout: 'masonry', workspaceDensity: 'compact' }; + // A new visit starts from System › Appearance. + assert.deepEqual(initialImagesView({ preferences }), { layout: 'masonry', density: 'compact' }); + // The current view (the last visit) keeps its own toolbar choices. + assert.deepEqual(initialImagesView({ prior: { layout: 'list', density: 'large' }, preferences }), { layout: 'list', density: 'large' }); + assert.deepEqual(initialImagesView({ prior: {}, preferences }), { layout: 'grid', density: 'comfortable' }); + // A saved view opens as it always did. + assert.deepEqual(initialImagesView({ saved: { q: '' }, preferences }), { layout: 'grid', density: 'comfortable' }); + // Without readable preferences nothing changes. + assert.deepEqual(initialImagesView({}), { layout: 'grid', density: 'comfortable' }); + assert.deepEqual(initialImagesView({ preferences: {} }), { layout: 'grid', density: 'comfortable' }); +}); diff --git a/tests/js/workspace-inspection-state.test.js b/tests/js/workspace-inspection-state.test.js index 82ea79d1..27d095e9 100644 --- a/tests/js/workspace-inspection-state.test.js +++ b/tests/js/workspace-inspection-state.test.js @@ -825,3 +825,73 @@ test('refresh re-reads the selected file and keeps the shown record until the ne assert.equal(controller.state.record.identity.path, 'folder/b.png'); controller.dispose(); assert.equal(await controller.refresh(), undefined); }); + +test('metadata off shows an explicit load and loads once on request', async () => { + const { api, requests } = pendingApi(), updates = []; + const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, autoLoad: false }); + controller.subscribe(state => updates.push(state)); + // Selecting reads nothing: the state says the metadata waits for a request. + assert.equal(controller.select(image()), undefined); + assert.equal(requests.length, 0); + assert.equal(controller.state.deferred, true); + assert.equal(controller.state.loading, false); + assert.equal(controller.state.record, null); + // Selecting the same file again still reads nothing. + controller.select({ ...image() }); + assert.equal(requests.length, 0); + const load = controller.load(); + assert.equal(requests.length, 1); + assert.equal(controller.state.loading, true); + assert.equal(controller.state.deferred, false); + // A second press while it reads, and the same file selected again once it loaded, read nothing more. + controller.load(); + assert.equal(requests.length, 1); + requests[0].resolve(envelope()); await load; + assert.equal(controller.state.record.generation.seed.value, '9007199254740993'); + controller.load(); controller.select(image()); + assert.equal(requests.length, 1); + // Another file waits for its own request again. + controller.select({ ...image(), path: 'folder/b.png', filename: 'b.png', url: image().url.replace('a%20%23%25', 'b') }); + assert.equal(controller.state.deferred, true); + assert.equal(requests.length, 1); + // The default controller loads on selection, as before. + const eager = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts }); + eager.select(image()); + assert.equal(requests.length, 2); + assert.equal(eager.state.deferred, false); + eager.dispose(); controller.dispose(); + assert.equal(controller.load(), undefined); +}); + +test('cache off rereads the current selection; on reuses the one current record', async () => { + for (const cache of [true, false]) { + const { api, requests } = pendingApi(); + const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, cache }); + const first = controller.select(image()); + // A read in flight is shared either way. + controller.select(image()); + assert.equal(requests.length, 1); + requests[0].resolve(envelope()); await first; + const again = controller.select(image()); + assert.equal(requests.length, cache ? 1 : 2, `cache ${cache}`); + if (!cache) { requests[1].resolve(envelope()); await again; assert.equal(controller.state.record.identity.path, 'folder/a #%.png'); } + controller.dispose(); + } +}); + +test('cache off never rereads a selection whose source changed, and refresh never loads a deferred one', async () => { + const { api, requests } = pendingApi(); + const controller = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, cache: false }); + const first = controller.select(image()); + requests[0].reject(new ApiError(409, refreshMessage, 'source-changed')); await first; + assert.equal(controller.state.refreshRequired, true); + controller.select(image()); + assert.equal(requests.length, 1); + controller.dispose(); + const deferred = contract('createInspectionController')({ api, extractFacts: extractGenerationFacts, autoLoad: false }); + deferred.select(image()); + assert.equal(deferred.refresh(), undefined); + assert.equal(requests.length, 1); + assert.equal(deferred.state.deferred, true); + deferred.dispose(); +}); diff --git a/tests/js/workspace-media-viewer.test.js b/tests/js/workspace-media-viewer.test.js index 749694a5..d375903f 100644 --- a/tests/js/workspace-media-viewer.test.js +++ b/tests/js/workspace-media-viewer.test.js @@ -124,18 +124,51 @@ test('real inspection navigation preserves slideshow across two advances and def } finally {view.dispose();controller.dispose();parentScope.dispose();globalThis.setTimeout=oldSetTimeout;globalThis.clearTimeout=oldClearTimeout;f.restore();} }); -test('an inline Viewer still loading its image is destroyed only after it has viewed it', async () => { +test('Viewer.js is built only on a loaded image and an unbuilt viewer is released at once', async () => { const {reportedLocationAdapter:adapter} = await import('../../web/js/workspace/history-media.mjs'); - const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c')]; + const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c'),occurrenceMedia('2',0,'d')]; const instances = []; - // Viewer.js's inline destroy() leaves an in-flight view running; its callback then needs a live instance. - f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.viewed = false; instances.push(this); } + // On a loaded image Viewer.js schedules its inline build; until it runs `ready` is false and destroy() cancels it. + f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.ready = false; this.viewed = false; instances.push(this); } destroy() { this.destroyed++; } resize() {} }; const globals = {ResizeObserver:globalThis.ResizeObserver, requestAnimationFrame:globalThis.requestAnimationFrame, cancelAnimationFrame:globalThis.cancelAnimationFrame}; Object.assign(globalThis, {ResizeObserver:class { observe() {} disconnect() {} }, requestAnimationFrame:() => 1, cancelAnimationFrame() {}}); const view = module.createMediaViewer({container:f.container,parentScope,inspectionController:controller,adapter,sender:{open(){}}}); try { - view.open({items,selectedKey:'c'.repeat(64)}); + const stageImage = () => f.walk(f.container,n => n.tagName === 'img')[0]; + view.open({items,selectedKey:'d'.repeat(64)}); + // Viewer.js's own cancel misses an image whose load event is still queued, so none exists before the load. + const unloaded = stageImage(), dropped = [];unloaded.removeAttribute = key => dropped.push(key); + assert.equal(instances.length, 0, 'no Viewer.js before the image loads'); + f.walk(f.container,n => n.attributes['aria-label'] === 'Previous media')[0].fire('click'); + assert.deepEqual(dropped, ['src'], 'an image left before it loads is released at once'); + unloaded.fire('load'); + assert.equal(instances.length, 0, 'its late load builds no Viewer.js'); + stageImage().fire('load'); + const [loading] = instances, image = loading.node, removed = []; + image.removeAttribute = key => removed.push(key); + // Next drops the image whose build is pending; a deferred destroy would let that build run there. + f.walk(f.container,n => n.attributes['aria-label'] === 'Next media')[0].fire('click'); + stageImage().fire('load');assert.equal(instances.length, 2, 'the next image builds its own Viewer.js once loaded'); + assert.equal(loading.destroyed, 1, 'an unbuilt viewer is destroyed at once'); + assert.deepEqual(removed, ['src'], 'and its image is released at once'); + loading.options.viewed.call(loading); + assert.equal(loading.destroyed, 1, 'never destroyed twice'); + } finally { view.dispose(); parentScope.dispose(); f.restore(); Object.assign(globalThis, globals); } +}); + +test('an inline Viewer still loading its image is destroyed only after it has viewed it', async () => { + const {reportedLocationAdapter:adapter} = await import('../../web/js/workspace/history-media.mjs'); + const f = mountedDom(), parentScope = createScope(), controller = borrowedController(), items = [occurrenceMedia('1',1,'c')]; + const instances = []; + // A built (`ready`) inline Viewer.js's destroy() leaves an in-flight view running; its callback then needs a live instance. + f.win.Viewer = class { constructor(node,options) { this.node = node; this.options = options; this.destroyed = 0; this.ready = true; this.viewed = false; instances.push(this); } + destroy() { this.destroyed++; } resize() {} }; + const globals = {ResizeObserver:globalThis.ResizeObserver, requestAnimationFrame:globalThis.requestAnimationFrame, cancelAnimationFrame:globalThis.cancelAnimationFrame}; + Object.assign(globalThis, {ResizeObserver:class { observe() {} disconnect() {} }, requestAnimationFrame:() => 1, cancelAnimationFrame() {}}); + const view = module.createMediaViewer({container:f.container,parentScope,inspectionController:controller,adapter,sender:{open(){}}}); + try { + view.open({items,selectedKey:'c'.repeat(64)});f.walk(f.container,n => n.tagName === 'img')[0].fire('load'); const [loading] = instances, image = loading.node, removed = []; image.removeAttribute = key => removed.push(key); f.walk(f.container,n => n.attributes['aria-label'] === 'Close image viewer')[0].fire('click'); @@ -148,7 +181,7 @@ test('an inline Viewer still loading its image is destroyed only after it has vi loading.options.viewed.call(loading); assert.equal(loading.destroyed, 1, 'never destroyed twice'); - view.open({items,selectedKey:'c'.repeat(64)}); + view.open({items,selectedKey:'c'.repeat(64)});f.walk(f.container,n => n.tagName === 'img')[0].fire('load'); const shown = instances[1]; shown.viewed = true; shown.options.viewed.call(shown); f.walk(f.container,n => n.attributes['aria-label'] === 'Close image viewer')[0].fire('click'); assert.equal(shown.destroyed, 1, 'a viewed image is destroyed at once'); @@ -236,7 +269,7 @@ test('viewer uses adapter.mediaUrl for image and video', async () => { const img = f.walk(f.container,n => n.tagName === 'img')[0]; assert.equal(img.src, image.url, 'the blob URL is used exactly'); assert.doesNotMatch(img.src, /\?/); - assert.equal(instances[0].node, img, 'Viewer.js shows the blob image'); + img.fire('load');assert.equal(instances[0].node, img, 'Viewer.js shows the blob image'); const download = f.walk(f.container,n => n.attributes['aria-label'] === 'Download original')[0]; assert.equal(download.href, image.url);assert.equal(download.download, 'a.png'); f.walk(f.container,n => n.attributes['aria-label'] === 'Next media')[0].fire('click'); diff --git a/tests/js/workspace-preferences.test.js b/tests/js/workspace-preferences.test.js new file mode 100644 index 00000000..07b18cad --- /dev/null +++ b/tests/js/workspace-preferences.test.js @@ -0,0 +1,193 @@ +// Browser display defaults (#249 System S3 Task 7): the workspace reads and writes the legacy gallery's own +// `gallerySettings` object (Q3, "One display configuration"), writes only the keys it changed, never consumes the +// legacy paging keys, and says when this browser refused to keep a change. +import test from 'node:test'; +import assert from 'node:assert/strict'; +import gallerySettings from '../../web/js/gallery-settings.js'; + +const module = await import('../../web/js/workspace/preferences.mjs').catch(error => { + if (error.code !== 'ERR_MODULE_NOT_FOUND') throw error; + return {}; +}); +globalThis.GallerySettings = gallerySettings; +const KEY = 'gallerySettings'; + +/** A localStorage stand-in holding `text` under the settings key; `setItem` can be made to throw. */ +function storage(text = null, { quota = false } = {}) { + const writes = []; + const store = { + text, + writes, + getItem(key) { assert.equal(key, KEY); return store.text; }, + setItem(key, value) { + assert.equal(key, KEY); + writes.push(value); + if (quota) { const error = new Error('Quota exceeded'); error.name = 'QuotaExceededError'; throw error; } + store.text = value; + }, + removeItem() { assert.fail('Preferences never remove the shared settings'); }, + }; + return store; +} +const denied = () => { const error = new Error('Denied'); error.name = 'SecurityError'; throw error; }; + +test('normalizes new workspace keys and keeps legacy keys', () => { + assert.equal(typeof module.readPreferences, 'function', 'preferences.mjs exports readPreferences'); + module.resetTabPreferences(); + const legacy = { defaultViewMode: 'list', gridColumns: 5, defaultLimit: 200, thumbnailPromptDismissed: 'sig', extra: 1 }; + const store = storage(JSON.stringify({ ...legacy, workspaceLayout: 'masonry', workspaceDensity: 'compact', showFilePaths: false })); + const { values, persisted } = module.readPreferences(() => store); + assert.equal(persisted, true); + assert.equal(values.workspaceLayout, 'masonry'); + assert.equal(values.workspaceDensity, 'compact'); + assert.equal(values.showFilePaths, false); + // Invalid workspace values fall back to the defaults, as every legacy key does. + store.text = JSON.stringify({ workspaceLayout: 'mosaic', workspaceDensity: 7, videoMute: 'no' }); + const fallback = module.readPreferences(() => store).values; + assert.equal(fallback.workspaceLayout, 'grid'); + assert.equal(fallback.workspaceDensity, 'comfortable'); + assert.equal(fallback.videoMute, true); + // The legacy normalizer knows the new keys and still keeps unknown ones. + const normalized = gallerySettings.normalizeGallerySettings({ ...legacy, workspaceLayout: 'list' }); + assert.equal(normalized.workspaceLayout, 'list'); + assert.equal(normalized.workspaceDensity, 'comfortable'); + assert.equal(normalized.extra, 1); + assert.equal(normalized.thumbnailPromptDismissed, 'sig'); + assert.deepEqual(gallerySettings.WORKSPACE_LAYOUTS, ['grid', 'list', 'masonry']); + assert.deepEqual(gallerySettings.WORKSPACE_DENSITIES, ['comfortable', 'large', 'compact']); +}); + +test('write touches only dirty keys', () => { + module.resetTabPreferences(); + const stored = { defaultViewMode: 'list', gridColumns: 5, thumbnailsGenerated: true, extra: { kept: true }, videoMute: false }; + const store = storage(JSON.stringify(stored)); + const result = module.writePreferences({ workspaceLayout: 'masonry', showImageInfo: false }, () => store); + assert.deepEqual(result, { persisted: true, conflicts: [] }); + assert.equal(store.writes.length, 1); + // Every other key, legacy or unknown, is written back exactly as it was stored: no defaults are filled in. + assert.deepEqual(JSON.parse(store.text), { ...stored, workspaceLayout: 'masonry', showImageInfo: false }); + // It re-reads the stored object, so a key another page wrote after this one read is kept. + store.text = JSON.stringify({ ...JSON.parse(store.text), gridColumns: 9 }); + module.writePreferences({ videoLoop: false }, () => store); + assert.equal(JSON.parse(store.text).gridColumns, 9); + assert.equal(JSON.parse(store.text).workspaceLayout, 'masonry'); + // An empty or malformed stored value starts from an empty object, never the defaults. + for (const text of [null, '{broken', '[]', '42']) { + const fresh = storage(text); + assert.equal(module.writePreferences({ cacheMetadata: false }, () => fresh).persisted, true); + assert.deepEqual(JSON.parse(fresh.text), { cacheMetadata: false }); + } + // Only workspace keys with valid values are accepted; nothing is written for a bad change. + for (const bad of [{ gridColumns: 4 }, { defaultLimit: 50 }, { workspaceLayout: 'mosaic' }, { videoMute: 'yes' }, { imageQuality: 'ultra' }]) { + const untouched = storage(JSON.stringify(stored)); + assert.throws(() => module.writePreferences(bad, () => untouched), TypeError); + assert.equal(untouched.writes.length, 0); + } + // A key changed elsewhere since this tab read it (its baseline) is reported, and the explicit save still wins. + const elsewhere = storage(JSON.stringify({ videoLoop: false })); + const outcome = module.writePreferences({ videoLoop: true, videoMute: false }, () => elsewhere, { baseline: { videoLoop: true, videoMute: true } }); + assert.deepEqual(outcome, { persisted: true, conflicts: ['videoLoop'] }); + assert.equal(JSON.parse(elsewhere.text).videoLoop, true); +}); + +test('denied storage reports tab-only', () => { + module.resetTabPreferences(); + const read = module.readPreferences(denied); + assert.equal(read.persisted, false); + assert.equal(read.values.workspaceLayout, 'grid'); + const result = module.writePreferences({ workspaceLayout: 'list', videoAutoplay: true }, denied); + assert.deepEqual(result, { persisted: false, conflicts: [] }); + // The change is kept for this tab: every reader in it (a new Images visit, the viewer) sees it. + const tab = module.readPreferences(denied); + assert.equal(tab.persisted, false); + assert.equal(tab.values.workspaceLayout, 'list'); + assert.equal(tab.values.videoAutoplay, true); + // A getItem that throws is denied storage too. + const throwing = { getItem: denied, setItem: denied }; + assert.equal(module.readPreferences(() => throwing).values.workspaceLayout, 'list'); + assert.equal(module.writePreferences({ workspaceDensity: 'large' }, () => throwing).persisted, false); + // Once storage accepts a write, the tab copy of that key gives way to the stored value. + const working = storage(JSON.stringify({ workspaceLayout: 'masonry' })); + assert.equal(module.writePreferences({ workspaceLayout: 'masonry' }, () => working).persisted, true); + assert.equal(module.readPreferences(() => working).values.workspaceLayout, 'masonry'); + module.resetTabPreferences(); + assert.equal(module.readPreferences(denied).values.videoAutoplay, false); +}); + +test('quota error keeps previous bytes', () => { + module.resetTabPreferences(); + const before = JSON.stringify({ gridColumns: 6, workspaceDensity: 'large' }); + const store = storage(before, { quota: true }); + const result = module.writePreferences({ workspaceDensity: 'compact' }, () => store); + assert.deepEqual(result, { persisted: false, conflicts: [] }); + assert.equal(store.writes.length, 1); + assert.equal(store.text, before); + // This tab keeps the change it could not store. + assert.equal(module.readPreferences(() => store).values.workspaceDensity, 'compact'); + module.resetTabPreferences(); + assert.equal(module.readPreferences(() => store).values.workspaceDensity, 'large'); +}); + +/** A window stand-in that records its storage listeners. */ +function fakeWindow() { + const listeners = new Set(); + return { + listeners, + addEventListener(type, listener) { assert.equal(type, 'storage'); listeners.add(listener); }, + removeEventListener(type, listener) { assert.equal(type, 'storage'); listeners.delete(listener); }, + fire(event) { for (const listener of [...listeners]) listener(event); }, + }; +} + +test('storage event updates clean values and flags dirty conflicts', () => { + module.resetTabPreferences(); + const store = storage(JSON.stringify({ workspaceLayout: 'list', videoMute: false })); + const win = fakeWindow(); + const seen = []; + const unsubscribe = module.subscribePreferences(win, next => seen.push(next), () => store); + assert.equal(win.listeners.size, 1); + // Another key, or another storage area's event, is not ours. + win.fire({ key: 'pm_update_notice_dismissed', storageArea: store }); + win.fire({ key: KEY, storageArea: { other: true } }); + assert.equal(seen.length, 0); + win.fire({ key: KEY, storageArea: store }); + win.fire({ key: null, storageArea: store }); // localStorage.clear() elsewhere + assert.equal(seen.length, 2); + assert.equal(seen[0].values.workspaceLayout, 'list'); + unsubscribe(); + assert.equal(win.listeners.size, 0); + // The reconciliation of the System drafts: clean values follow the other page; a key this tab has a draft for and + // the other page changed is a conflict, and the draft stays. + const baseline = { workspaceLayout: 'grid', videoMute: true, videoLoop: true }; + const next = { workspaceLayout: 'list', videoMute: true, videoLoop: false }; + assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { videoMute: false, videoLoop: false }, next }), { conflicts: [] }); + assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { workspaceLayout: 'masonry' }, next }), { conflicts: ['workspaceLayout'] }); + assert.deepEqual(module.reconcilePreferences({ baseline, drafts: { workspaceLayout: 'list' }, next }), { conflicts: [] }); +}); + +test('legacy paging keys are never consumed by the workspace', () => { + module.resetTabPreferences(); + const store = storage(JSON.stringify({ defaultLimit: 500, gridColumns: 12, infiniteScroll: true, lazyLoading: false, + defaultViewMode: 'list', debugMode: true, apiTimeout: 9, thumbnailsGenerated: true, checkThumbnailsAtStartup: false })); + const { values } = module.readPreferences(() => store); + assert.deepEqual(Object.keys(values).sort(), [...module.WORKSPACE_PREFERENCE_KEYS].sort()); + for (const key of ['defaultLimit', 'gridColumns', 'infiniteScroll', 'lazyLoading', 'defaultViewMode', 'debugMode', 'apiTimeout', + 'thumbnailsGenerated', 'checkThumbnailsAtStartup', 'thumbnailPromptDismissed']) assert.equal(Object.hasOwn(values, key), false, key); + // The display defaults are the gallery's own defaults for these keys. + const defaults = module.preferenceDefaults(); + for (const key of module.WORKSPACE_PREFERENCE_KEYS) assert.equal(defaults[key], gallerySettings.DEFAULT_SETTINGS[key], key); +}); + +test('a later save flushes the changes kept for this tab only', () => { + module.resetTabPreferences(); + const store = storage(JSON.stringify({ gridColumns: 4 }), { quota: true }); + assert.equal(module.writePreferences({ workspaceLayout: 'list' }, () => store).persisted, false); + // Storage accepts writes again: saving another key stores the tab-only change with it, so "Saved" is true for both. + const working = storage(store.text); + assert.deepEqual(module.writePreferences({ videoLoop: false }, () => working), { persisted: true, conflicts: [] }); + assert.deepEqual(JSON.parse(working.text), { gridColumns: 4, workspaceLayout: 'list', videoLoop: false }); + // Nothing is overlaid any more: another page's later change reads through. + working.text = JSON.stringify({ ...JSON.parse(working.text), workspaceLayout: 'masonry' }); + assert.equal(module.readPreferences(() => working).values.workspaceLayout, 'masonry'); + module.resetTabPreferences(); +}); diff --git a/tests/js/workspace-state.test.js b/tests/js/workspace-state.test.js index a02129a1..c05714bb 100644 --- a/tests/js/workspace-state.test.js +++ b/tests/js/workspace-state.test.js @@ -23,8 +23,50 @@ test('tags is a native route titled Tags; only unbuilt routes hand off to legacy assert.equal(routeTitle(route), route.label); } assert.equal(routeTitle(resolveRoute('#/library/7')), 'Prompt library'); - for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id); - for (const id of ['system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), true, id); + // System is native from S2 (#249 System): its undelivered sections hand off per section (D94), not the route. + for (const id of ['images', 'library', 'tags', 'inspect', 'overview', 'activity', 'system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id); +}); + +test('system resolves its sections; #/overview stays the production hub', async () => { + const { resolveRoute, routeTitle, isHandoff, routes } = await source; + // `#/system` is the Overview section and keeps the sidebar's hash. + const system = resolveRoute('#/system'); + assert.deepEqual([system.id, system.section, system.hash, routeTitle(system), isHandoff(system)], + ['system', 'overview', '#/system', 'System', false]); + for (const section of ['overview', 'appearance', 'sources', 'integrations', 'tagging', 'maintenance', 'backups', 'diagnostics']) { + const route = resolveRoute(`#/system/${section}`); + assert.deepEqual([route.id, route.section, route.hash], ['system', section, `#/system/${section}`]); + } + // An unknown child is the Overview section with a replaced hash. + for (const hash of ['#/system/general', '#/system/', '#/system/__proto__', '#/system/backups/1', '#/system/']) { + const route = resolveRoute(hash); + assert.deepEqual([route.id, route.section, route.hash], ['system', 'overview', '#/system/overview'], hash); + } + // D28: the production hub is untouched. + const hub = resolveRoute('#/overview'); + assert.deepEqual([hub.id, hub.section], ['overview', undefined]); + // Q12: Overview's System card opens the native System Overview, with no legacy href or action. + const card = routes.find(route => route.id === 'system'); + assert.equal(card.href, '#/system/overview'); + assert.equal(card.native, true); + assert.equal(card.action, 'Open System'); +}); + +test('the router renders again when only the System section changes', async () => { + const { mountRouter } = await source; + const win = new EventTarget(); + win.location = { hash: '#/system' }; + const replacements = []; + win.history = { state: null, replaceState(state, unused, hash) { replacements.push(hash); win.location.hash = hash; } }; + const seen = []; + const dispose = mountRouter(win, route => seen.push([route.id, route.section])); + for (const hash of ['#/system/appearance', '#/system/appearance', '#/system/bogus', '#/system/overview', '#/images']) { + win.location.hash = hash; + win.dispatchEvent(new Event('hashchange')); + } + assert.deepEqual(seen, [['system', 'overview'], ['system', 'appearance'], ['system', 'overview'], ['images', undefined]]); + assert.deepEqual(replacements, ['#/system/overview']); + dispose(); }); test('inspect is the native Inspect a file page with the concept description', async () => { diff --git a/tests/js/workspace-system-appearance.test.js b/tests/js/workspace-system-appearance.test.js new file mode 100644 index 00000000..497df7f6 --- /dev/null +++ b/tests/js/workspace-system-appearance.test.js @@ -0,0 +1,188 @@ +// System › Appearance (#249 System S3 Task 8): the concept's "Browsing & display" rows (app.js 676–678) on the drafts +// of the one Save row, their stored values (Q3 mappings), the host timeout's 0–300 whole seconds and the D97 rows. +const test = require('node:test'); +const assert = require('node:assert/strict'); +const gallerySettings = require('../../web/js/gallery-settings.js'); + +const appearanceModule = import('../../web/js/workspace/system-appearance.mjs'); +const stateModule = import('../../web/js/workspace/system-state.mjs'); +const coreModule = import('../../web/js/shared/toolkit/core.js'); +globalThis.GallerySettings = gallerySettings; + +/** A minimal DOM whose elements count their live listeners (a listener's signal aborts it, as in a browser). */ +function fakeDOM() { + const previous = global.document; + const elements = []; + class Element { + constructor(tag) { + this.tagName = tag.toUpperCase(); + this.children = []; + this.attributes = {}; + this.listeners = []; + this.dataset = {}; + this.className = ''; + this.id = ''; + this.value = ''; + this._text = ''; + elements.push(this); + } + get textContent() { return this._text + this.children.map(child => child.textContent).join(''); } + set textContent(text) { this.children = []; this._text = String(text); } + append(...nodes) { + for (const node of nodes) { + const child = typeof node === 'string' ? Object.assign(new Element('#text'), { _text: node }) : node; + child.parentNode = this; + this.children.push(child); + } + } + remove() { if (this.parentNode) this.parentNode.children = this.parentNode.children.filter(child => child !== this); this.parentNode = null; } + replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); } + setAttribute(name, value) { this.attributes[name] = String(value); } + getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; } + removeAttribute(name) { delete this.attributes[name]; } + addEventListener(type, callback, options = {}) { + if (options.signal?.aborted) return; + const entry = { type, callback }; + this.listeners.push(entry); + options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); }); + } + removeEventListener(type, callback) { this.listeners = this.listeners.filter(item => item.type !== type || item.callback !== callback); } + dispatch(type) { for (const { callback } of this.listeners.filter(item => item.type === type)) callback({ type, target: this }); } + click() { this.dispatch('click'); } + } + global.document = { + createElement: tag => new Element(tag), + createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }), + }; + const walk = (node, found = []) => { found.push(node); node.children.forEach(child => walk(child, found)); return found; }; + return { + listenerCount: () => elements.reduce((sum, element) => sum + element.listeners.length, 0), + find: (root, predicate) => walk(root).find(predicate), + tags: node => node.children.map(child => child.tagName.toLowerCase()), + restore() { global.document = previous; }, + }; +} + +const BROWSER = { workspaceLayout: 'grid', workspaceDensity: 'comfortable', autoLoadMetadata: true, cacheMetadata: true, + imageQuality: 'medium', videoAutoplay: false, videoMute: true, videoLoop: true, showFilePaths: true, showImageInfo: true, + sidebarCollapsedByDefault: false }; + +async function mounted({ host = { webui_display_mode: 'newtab', result_timeout: 5 }, browser = BROWSER, hostFails = false } = {}) { + const { mountAppearance } = await appearanceModule; + const { createSystemDrafts } = await stateModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + let hostValues = null; + const drafts = createSystemDrafts({ + readBrowser: () => ({ ...browser }), readRuntime: () => ({}), + readHost: async () => { if (hostFails) throw new Error('offline'); return { values: hostValues, revision: 'sha256:one' }; }, + writeBrowser: async () => ({ persisted: true }), writeHost: async () => ({ success: true, persisted: true }), writeRuntime: async () => ({}), + }); + await drafts.load('browser'); + const scope = createScope(); + const [surface] = mountAppearance(null, { scope, drafts }); + const byId = id => dom.find(surface, node => node.dataset?.setting === id); + return { dom, drafts, scope, surface, byId, loadHost: async () => { hostValues = host; await drafts.load('host'); }, + fixHost: () => { hostFails = false; } }; +} + +test('the surface is the concept section with the concept rows in order, then the D97 rows', async () => { + const { APPEARANCE_ROWS } = await appearanceModule; + const { dom, surface, scope } = await mounted(); + try { + assert.equal(surface.className, 'surface'); + assert.equal(surface.children[0].textContent, 'Browsing & display'); + assert.equal(surface.children[1].textContent, 'One set of defaults; toolbar changes apply to the current view.'); + const rows = surface.children.slice(2); + const titles = rows.map(row => row.children[0].children[0].textContent); + assert.deepEqual(titles, ['Default layout', 'Thumbnail size', 'Continuous browsing', 'Load metadata on selection', + 'Cache metadata in this browser', 'Image loading quality', 'Autoplay videos', 'Mute videos by default', 'Loop videos', + 'Show file paths', 'Open from ComfyUI', 'Node search auto-hide (seconds)', 'Show image details on cards', + 'Collapse viewer details by default', 'Reset display defaults']); + const continuous = rows[2]; + assert.deepEqual(dom.tags(continuous.children[0]), ['h3', 'p']); + assert.equal(continuous.children[1].className, 'chip blue'); + assert.equal(continuous.children[1].textContent, 'Infinite scroll'); + assert.equal(rows.at(-1).className, 'operation'); + assert.equal(APPEARANCE_ROWS.filter(Array.isArray).length, 13); + } finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); } +}); + +test('added rows reach their consumers: each control edits its stored key with the Q3 values', async () => { + const { dom, drafts, scope, byId, loadHost } = await mounted(); + try { + // Host rows stay disabled until the server settings answered. + assert.equal(byId('open-mode').disabled, true); + assert.equal(byId('timeout').disabled, true); + await loadHost(); + assert.equal(byId('open-mode').disabled, false); + assert.equal(byId('timeout').value, '5'); + assert.deepEqual(byId('image-quality').children.map(option => [option.value, option.textContent]), + [['medium', 'Optimized'], ['high', 'Original'], ['low', 'Fast']]); + assert.deepEqual(byId('open-mode').children.map(option => [option.value, option.textContent]), + [['newtab', 'New tab'], ['popup', 'Popup window']]); + const edits = [['default-view', 'masonry'], ['density', 'large'], ['image-quality', 'low'], ['open-mode', 'popup']]; + for (const [id, value] of edits) { byId(id).value = value; byId(id).dispatch('change'); } + for (const id of ['metadata-auto', 'metadata-cache', 'video-muted', 'video-loop', 'paths', 'card-details']) { + byId(id).checked = false; byId(id).dispatch('change'); + } + for (const id of ['video-auto', 'viewer-details']) { byId(id).checked = true; byId(id).dispatch('change'); } + assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'masonry', workspaceDensity: 'large', imageQuality: 'low', + autoLoadMetadata: false, cacheMetadata: false, videoMute: false, videoLoop: false, showFilePaths: false, showImageInfo: false, + videoAutoplay: true, sidebarCollapsedByDefault: true }); + assert.deepEqual(drafts.changes('host'), { webui_display_mode: 'popup' }); + } finally { scope.dispose(); dom.restore(); } +}); + +test('the auto-hide timeout takes whole seconds from 0 to 300 only', async () => { + const { resultTimeout } = await appearanceModule; + assert.deepEqual(['0', '30', ' 300 ', '301', '-1', '2.5', '', 'abc', '1e2'].map(resultTimeout), [0, 30, 300, null, null, null, null, null, null]); + const { dom, drafts, scope, byId, loadHost } = await mounted(); + try { + await loadHost(); + const timeout = byId('timeout'); + timeout.value = '45'; timeout.dispatch('change'); + assert.deepEqual(drafts.changes('host'), { result_timeout: 45 }); + timeout.value = '301'; timeout.dispatch('change'); + assert.deepEqual(drafts.changes('host'), {}); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + timeout.value = '0'; timeout.dispatch('change'); + assert.deepEqual(drafts.changes('host'), { result_timeout: 0 }); + assert.deepEqual([...drafts.failed('host')], []); + } finally { scope.dispose(); dom.restore(); } +}); + +test('reset restores defaults without touching other keys', async () => { + const { DISPLAY_KEYS } = await appearanceModule; + const changed = { ...BROWSER, workspaceLayout: 'list', videoMute: false, showImageInfo: false, sidebarCollapsedByDefault: true }; + const { dom, drafts, scope, surface, loadHost } = await mounted({ browser: changed }); + try { + await loadHost(); + drafts.set('host', 'result_timeout', 9); + const reset = dom.find(surface.children.at(-1), node => node.tagName === 'BUTTON'); + assert.equal(reset.textContent, 'Reset'); + reset.click(); + assert.deepEqual(drafts.changes('browser'), { workspaceLayout: 'grid', videoMute: true, showImageInfo: true, sidebarCollapsedByDefault: false }); + // The host fields and every key outside the display defaults are left as they are. + assert.deepEqual(drafts.changes('host'), { result_timeout: 9 }); + assert.deepEqual([...DISPLAY_KEYS].sort(), Object.keys(BROWSER).sort()); + } finally { scope.dispose(); dom.restore(); } +}); + +test('host rows that could not be read stay disabled and say so; a later read clears it', async () => { + const { HOST_UNAVAILABLE } = await appearanceModule; + const { dom, drafts, scope, byId, surface, fixHost } = await mounted({ hostFails: true }); + const notes = () => surface.children.slice(2).flatMap(row => row.children[0].children.slice(2).map(note => [row.children[0].children[0].textContent, note.className, note.textContent])); + try { + assert.deepEqual(notes(), []); + assert.equal(await drafts.reloadHost(), false); + assert.deepEqual(notes(), [['Open from ComfyUI', 'small', HOST_UNAVAILABLE], ['Node search auto-hide (seconds)', 'small', HOST_UNAVAILABLE]]); + assert.equal(byId('timeout').disabled, true); + // Repaints keep one line per row. + drafts.set('browser', 'videoMute', false); + assert.equal(notes().length, 2); + fixHost(); + assert.equal(await drafts.reloadHost(), true); + assert.deepEqual(notes(), []); + } finally { scope.dispose(); assert.equal(dom.listenerCount(), 0); dom.restore(); } +}); diff --git a/tests/js/workspace-system-controls.test.js b/tests/js/workspace-system-controls.test.js new file mode 100644 index 00000000..19a99b84 --- /dev/null +++ b/tests/js/workspace-system-controls.test.js @@ -0,0 +1,214 @@ +// System controls (#249 System S2 Task 5): the concept's `setting()` (app.js 624–634, its toggles hydrated into Toolkit +// switches by `hydrateSwitches` 708–717), `operation()` (645–654) and `sectionNav()` (306–308) as DOM builders whose +// listeners live on the caller's scope. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const controlsModule = import('../../web/js/workspace/system-controls.mjs'); +const coreModule = import('../../web/js/shared/toolkit/core.js'); + +/** A minimal DOM whose elements count their live listeners (a listener's signal aborts it, as in a browser). */ +function fakeDOM() { + const previous = global.document; + const elements = []; + class Element { + constructor(tag) { + this.tagName = tag.toUpperCase(); + this.children = []; + this.attributes = {}; + this.listeners = []; + this.dataset = {}; + this.className = ''; + this.id = ''; + this.value = ''; + this._text = ''; + elements.push(this); + } + get textContent() { return this._text + this.children.map(child => child.textContent).join(''); } + set textContent(text) { this.children = []; this._text = String(text); } + append(...nodes) { + for (const node of nodes) { + const child = typeof node === 'string' ? Object.assign(new Element('#text'), { _text: node }) : node; + child.parentNode = this; + this.children.push(child); + } + } + replaceChildren(...nodes) { this.children = []; this._text = ''; this.append(...nodes); } + setAttribute(name, value) { this.attributes[name] = String(value); } + getAttribute(name) { return Object.hasOwn(this.attributes, name) ? this.attributes[name] : null; } + removeAttribute(name) { delete this.attributes[name]; } + addEventListener(type, callback, options = {}) { + if (options.signal?.aborted) return; + const entry = { type, callback }; + this.listeners.push(entry); + options.signal?.addEventListener('abort', () => { this.listeners = this.listeners.filter(item => item !== entry); }); + } + removeEventListener(type, callback) { this.listeners = this.listeners.filter(item => item.type !== type || item.callback !== callback); } + dispatch(type) { for (const { callback } of this.listeners.filter(item => item.type === type)) callback({ type, target: this }); } + click() { this.dispatch('click'); } + } + global.document = { + createElement: tag => new Element(tag), + createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }), + }; + const walk = (node, found = []) => { found.push(node); node.children.forEach(child => walk(child, found)); return found; }; + return { + listenerCount: () => elements.reduce((sum, element) => sum + element.listeners.length, 0), + find: (root, predicate) => walk(root).find(predicate), + tags: node => node.children.map(child => child.tagName.toLowerCase()), + restore() { global.document = previous; }, + }; +} + +test('settingRow is the concept row: div > label[for] + p, then the control', async () => { + const { settingRow } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const changes = []; + const row = settingRow({ id: 'timeout', label: 'Node search auto-hide (seconds)', + description: 'Node search behavior only. Use 0 to keep results visible.', + control: { type: 'number', value: 30, min: 0, max: 300 }, scope, onChange: value => changes.push(value) }); + assert.equal(row.className, 'setting-row'); + assert.deepEqual(dom.tags(row), ['div', 'input']); + const [text, input] = row.children; + assert.deepEqual(dom.tags(text), ['label', 'p']); + assert.equal(text.children[0].textContent, 'Node search auto-hide (seconds)'); + assert.equal(text.children[0].htmlFor, 'timeout'); + assert.equal(text.children[1].textContent, 'Node search behavior only. Use 0 to keep results visible.'); + assert.deepEqual([input.id, input.type, input.value, input.min, input.max], ['timeout', 'number', '30', '0', '300']); + assert.equal(input.dataset.setting, 'timeout'); + input.value = '45'; + input.dispatch('change'); + assert.deepEqual(changes, [45]); + + const select = settingRow({ id: 'open-mode', label: 'Open from ComfyUI', description: 'Keep the node pack’s popup / new-tab preference.', + control: { type: 'select', options: [['newtab', 'New tab'], ['popup', 'Popup window']], value: 'popup' }, scope, + onChange: value => changes.push(value) }).children[1]; + assert.equal(select.tagName, 'SELECT'); + assert.deepEqual(select.children.map(option => [option.value, option.textContent]), [['newtab', 'New tab'], ['popup', 'Popup window']]); + assert.equal(select.value, 'popup'); + select.value = 'newtab'; + select.dispatch('change'); + assert.deepEqual(changes, [45, 'newtab']); + + // The concept's range is 1–16 unless the row says otherwise (D113: the CPU count). + const range = settingRow({ id: 'workers', label: 'Worker threads', description: 'Balance background scans.', + control: { type: 'range', value: 4 } }).children[1]; + assert.deepEqual([range.type, range.min, range.max, range.value], ['range', '1', '16', '4']); + const password = settingRow({ id: 'civitai-key', label: 'CivitAI API key', description: 'Credentials stay on the server.', + control: { type: 'password', value: '' } }).children[1]; + assert.equal(password.type, 'password'); + + // A ready-made control is placed as given and takes the row id. + const custom = document.createElement('textarea'); + const customRow = settingRow({ id: 'custom', label: 'Custom', description: 'Any control.', control: custom }); + assert.equal(customRow.children[1], custom); + assert.equal(custom.id, 'custom'); + assert.throws(() => settingRow({ id: '', label: 'x', description: 'y', control: { type: 'number' } }), TypeError); + assert.throws(() => settingRow({ id: 'x', label: 'x', description: 'y', control: { type: 'colour' } }), TypeError); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('a toggle row holds the Toolkit switch, labelled by the row label', async () => { + const { settingRow } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const changes = []; + const row = settingRow({ id: 'video-auto', label: 'Autoplay videos', description: 'Videos stay within the same gallery.', + control: { type: 'toggle', checked: false }, scope, onChange: value => changes.push(value) }); + const field = row.children[1]; + // createSwitch('', '', id): `label.ca-switch` > input[type=checkbox][role=switch] + an empty label text. + assert.equal(field.tagName, 'LABEL'); + assert.equal(field.className, 'ca-switch'); + const input = field.children[0]; + assert.deepEqual([input.type, input.role, input.id, input.checked], ['checkbox', 'switch', 'video-auto', false]); + assert.equal(row.children[0].children[0].htmlFor, input.id); + assert.equal(input.dataset.setting, 'video-auto'); + input.checked = true; + input.dispatch('change'); + assert.deepEqual(changes, [true]); + // The concept's toggles are on unless configured off (`val !== false`). + const on = settingRow({ id: 'video-loop', label: 'Loop videos', description: 'Repeat a clip.', control: { type: 'toggle' } }); + assert.equal(on.children[1].children[0].checked, true); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('operationRow is the concept operation with a danger class only when flagged', async () => { + const { operationRow } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const clicks = []; + const row = operationRow({ title: 'Missing-file links', + description: 'Remove stale database links only. Files on disk are never deleted by this operation.', + label: 'Review plan', danger: true, scope, onClick: button => clicks.push(button) }); + assert.equal(row.className, 'operation'); + assert.deepEqual(dom.tags(row), ['div', 'div']); + assert.deepEqual(dom.tags(row.children[0]), ['h3', 'p']); + assert.equal(row.children[0].children[0].textContent, 'Missing-file links'); + assert.equal(row.children[1].className, 'row-actions'); + const button = row.children[1].children[0]; + assert.deepEqual([button.tagName, button.type, button.textContent, button.className], ['BUTTON', 'button', 'Review plan', 'danger']); + button.click(); + assert.deepEqual(clicks, [button]); + const plain = operationRow({ title: 'Statistics', description: 'Usage by prompt, category, tag and date.', label: 'Explore', + scope, onClick() {} }); + assert.equal(plain.children[1].children[0].className, ''); + assert.throws(() => operationRow({ title: 'x', description: 'y', label: 'z', onClick() {} }), TypeError); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('sectionNav marks the active section and reports the chosen one', async () => { + const { sectionNav } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const chosen = []; + const nav = sectionNav({ label: 'System sections', sections: [['overview', 'Overview'], ['appearance', 'Appearance']], + active: 'appearance', scope, onSelect: id => chosen.push(id) }); + assert.equal(nav.className, 'section-nav'); + assert.equal(nav.getAttribute('role'), 'group'); + assert.equal(nav.getAttribute('aria-label'), 'System sections'); + assert.deepEqual(nav.children.map(button => [button.textContent, button.getAttribute('aria-pressed'), button.dataset.value]), + [['Overview', 'false', 'overview'], ['Appearance', 'true', 'appearance']]); + nav.children[0].click(); + assert.deepEqual(chosen, ['overview']); + } finally { + scope.dispose(); + dom.restore(); + } +}); + +test('disposing the scope leaves no listener behind', async () => { + const { settingRow, operationRow, sectionNav } = await controlsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + settingRow({ id: 'a', label: 'A', description: 'a', control: { type: 'toggle' }, scope, onChange() {} }); + settingRow({ id: 'b', label: 'B', description: 'b', control: { type: 'select', options: ['Grid', 'List'] }, scope, onChange() {} }); + operationRow({ title: 'C', description: 'c', label: 'Go', scope, onClick() {} }); + sectionNav({ label: 'System sections', sections: [['overview', 'Overview']], active: 'overview', scope, onSelect() {} }); + assert.equal(dom.listenerCount() > 0, true); + scope.dispose(); + assert.equal(dom.listenerCount(), 0); + } finally { + scope.dispose(); + dom.restore(); + } +}); diff --git a/tests/js/workspace-system-overview.test.js b/tests/js/workspace-system-overview.test.js new file mode 100644 index 00000000..bafcd0f9 --- /dev/null +++ b/tests/js/workspace-system-overview.test.js @@ -0,0 +1,255 @@ +// System › Overview (#249 System S2 Task 6): the concept's Overview copy on real values (D93), the Health report rows +// (D95), the Statistics tables (D96) and the health check run that waits for its read job. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const overviewModule = import('../../web/js/workspace/system-overview.mjs'); + +const OVERVIEW = Object.freeze({ + library: { positive_prompts: 1234, negative_variants: 7 }, + images: { indexed: 24, rated: 20 }, + health: null, + backup: null, + needs_review: { pending: null }, + sources: { configured: 2, effective: 2, auto_detected: false }, +}); + +test('metrics are the server counts and Not checked before any health check', async () => { + const { metricValues } = await overviewModule; + assert.deepEqual(metricValues(OVERVIEW), [ + [(1234).toLocaleString(), 'Positive prompts'], ['7', 'Negative variants'], ['24', 'Images'], + ['Not checked', 'Database integrity'], + ]); +}); + +test('metric 4 is the last health result', async () => { + const { integrityText } = await overviewModule; + assert.equal(integrityText(null), 'Not checked'); + assert.equal(integrityText({ status: 'passed', issues: 0 }), 'Passed'); + assert.equal(integrityText({ status: 'issues', issues: 1 }), '1 issue'); + assert.equal(integrityText({ status: 'issues', issues: 3 }), '3 issues'); + assert.equal(integrityText({ status: 'incomplete', issues: 0 }), 'Incomplete'); + // Issues found beside a check that could not run (Needs review, Q6) are still named. + assert.equal(integrityText({ status: 'incomplete', issues: 2 }), '2 issues'); +}); + +test('library health rows never invent a review count, a backup or thumbnails', async () => { + const { ambiguousTitle, backupText } = await overviewModule; + assert.equal(ambiguousTitle({ pending: null }), 'Ambiguous image links'); + assert.equal(ambiguousTitle(undefined), 'Ambiguous image links'); + assert.equal(ambiguousTitle({ pending: 1 }), '1 ambiguous image link'); + assert.equal(ambiguousTitle({ pending: 3 }), '3 ambiguous image links'); + assert.equal(backupText(null), 'No retained backup record.'); + assert.equal(backupText({ verified_at: 'not a time' }), 'No retained backup record.'); + assert.match(backupText({ verified_at: '2026-10-22T09:30:00Z' }), /^Last verified backup .+\.$/); +}); + +test('the thumbnail row is its own read of the thumbnail status', async () => { + const { thumbnailTitle, THUMBNAILS_LOADING } = await overviewModule; + // Until `/images/thumbnails/status` answers the row says it is checking; a failed read is unavailable, never 0. + assert.equal(THUMBNAILS_LOADING, 'Checking thumbnails…'); + assert.equal(thumbnailTitle(undefined), THUMBNAILS_LOADING); + assert.equal(thumbnailTitle({ pending: 0 }), 'No missing thumbnails'); + assert.equal(thumbnailTitle({ pending: 1 }), '1 missing thumbnail'); + assert.equal(thumbnailTitle({ pending: 12 }), '12 missing thumbnails'); + assert.equal(thumbnailTitle(null), 'Thumbnail status unavailable'); + assert.equal(thumbnailTitle({ pending: 'x' }), 'Thumbnail status unavailable'); + // With no image source there is nothing to count. + assert.equal(thumbnailTitle({ pending: 0 }, { effective: 0 }), 'Thumbnail status unavailable'); + assert.equal(thumbnailTitle({ pending: 2 }, { effective: 1 }), '2 missing thumbnails'); +}); + +test('an integrity count SQLite stopped at reads as a lower bound', async () => { + const { checkChip } = await overviewModule; + assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 100, capped: true }), '100+ problems found'); + assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 37, capped: false }), '37 problems found'); + assert.equal(checkChip({ id: 'integrity', status: 'issues', count: 1 }), '1 problem found'); +}); + +test('the sources row tells configured, available and detected sources apart', async () => { + const { sourcesCopy } = await overviewModule; + const shared = 'Monitoring and enrichment share one source list.'; + assert.deepEqual(sourcesCopy({ configured: 2, effective: 2, auto_detected: false }), ['2 output sources', shared]); + assert.deepEqual(sourcesCopy({ configured: 2, effective: 1, auto_detected: false }), + ['1 output source', `1 of 2 configured sources available. ${shared}`]); + assert.deepEqual(sourcesCopy({ configured: 0, effective: 1, auto_detected: true }), + ['1 output source', `None configured is available; ComfyUI's output folder is used. ${shared}`]); +}); + +const REPORT = Object.freeze({ + checked_at: '2026-10-23T08:00:00Z', + status: 'issues', + checks: [ + { id: 'integrity', label: 'Integrity check', status: 'passed', count: 0 }, + { id: 'foreign_keys', label: 'Foreign keys & pair references', status: 'issues', count: 2 }, + { id: 'canonical_hashes', label: 'Canonical hashes', status: 'issues', count: 1 }, + { id: 'pair_uniqueness', label: 'Pair uniqueness', status: 'failed', count: null }, + { id: 'missing_files', label: 'Missing files', status: 'issues', count: 2, + sources: [{ name: 'output', status: 'checked', count: 2 }] }, + { id: 'ambiguous_metadata', label: 'Ambiguous metadata', status: 'unavailable', count: null }, + { id: 'consistency', label: 'Consistency', status: 'issues', count: null }, + ], +}); + +test('health rows give each check a chip from its real status, in the server order', async () => { + const { healthRows } = await overviewModule; + assert.deepEqual(healthRows(REPORT), [ + ['Integrity check', 'Passed'], + ['Foreign keys & pair references', '2 broken references'], + ['Canonical hashes', '1 hash does not match its text'], + ['Pair uniqueness', 'Could not check'], + ['Missing files', '2 links need review'], + ['Ambiguous metadata', 'Not available yet'], + ['Consistency', 'Issues found'], + ]); +}); + +test('missing files is reported per image source when there are several', async () => { + const { healthRows } = await overviewModule; + const report = { ...REPORT, checks: [{ id: 'missing_files', label: 'Missing files', status: 'issues', count: 1, partial: true, + sources: [{ name: 'output', status: 'checked', count: 1 }, { name: 'input', status: 'checked', count: 0 }, + { name: 'archive', status: 'unavailable', count: null }] }] }; + assert.deepEqual(healthRows(report), [ + ['Missing files · output', '1 link needs review'], + ['Missing files · input', 'Passed'], + ['Missing files · archive', 'Not available'], + ]); + const unavailable = { ...REPORT, checks: [{ id: 'missing_files', label: 'Missing files', status: 'unavailable', count: null, sources: [] }] }; + assert.deepEqual(healthRows(unavailable), [['Missing files', 'Not available']]); +}); + +test('the report summary says when it was checked and whether it is complete', async () => { + const { healthSummary } = await overviewModule; + const when = new Date(REPORT.checked_at).toLocaleString(); + assert.equal(healthSummary({ ...REPORT, status: 'passed' }), `Checked ${when}. Every check passed.`); + assert.equal(healthSummary({ ...REPORT, status: 'incomplete' }), `Checked ${when}. Some checks could not run, so this report is incomplete.`); + assert.equal(healthSummary({ ...REPORT, checked_at: 'x' }), 'Nothing was changed; review the checks with issues.'); +}); + +test('statistics rows are the real runs and ratings, then the library totals', async () => { + const { categoryRows, totalsRows } = await overviewModule; + const statistics = { + by_category: [{ category: 'Landscapes', runs: 1200, average_rating: 4.5 }, + { category: null, runs: 3, average_rating: null }], + totals: { total_prompts: 12, total_tags: 15, total_images: 24 }, + rated_images: 20, + }; + assert.deepEqual(categoryRows(statistics), [['Landscapes', (1200).toLocaleString(), '4.5 / 5'], ['Uncategorized', '3', 'Not rated']]); + assert.deepEqual(totalsRows(statistics), [['Prompts', '12'], ['Tags', '15'], ['Images', '24'], ['Rated images', '20']]); +}); + +/** A fake api answering each `get`/`post` from a queue per URL, recording the calls. */ +function fakeApi(answers) { + const calls = []; + const answer = (method, url) => { + calls.push(`${method} ${url}`); + const queue = answers[`${method} ${url}`]; + if (!queue?.length) throw new Error(`unexpected ${method} ${url}`); + const next = queue.shift(); + return next instanceof Error ? Promise.reject(next) : Promise.resolve(next); + }; + return { calls, get: url => answer('GET', url), post: url => answer('POST', url) }; +} + +const JOB = '0123456789abcdef0123456789abcdef'; +const job = (state, id = JOB, kind = 'health') => ({ id, kind, state }); + +/** + * A fake Activity store holding `jobs`: `track(job)` lists a started job, `load()` applies the next of `reads` (the + * server's job lists) and `push(job)` is a stream update; each notifies as the real store does. It never polls. + */ +function fakeActivity({ jobs = [], reads = [] } = {}) { + const listeners = new Set(); + const emit = () => { for (const listener of [...listeners]) listener(); }; + const store = { + loads: 0, + find: id => jobs.find(item => item.id === id), + running: () => jobs.filter(item => item.state === 'running'), + subscribe(listener) { listeners.add(listener); return () => listeners.delete(listener); }, + listeners: () => listeners.size, + async load() { + store.loads += 1; + if (reads.length) jobs = reads.shift(); + emit(); + }, + track(item) { + if (!item?.id) throw new TypeError('Invalid job'); + store.push(item); + return item; + }, + push(item) { + jobs = [...jobs.filter(other => other.id !== item.id), item]; + emit(); + }, + }; + return store; +} + +test('a health check tracks its job in the Activity store, then reads the report', async () => { + const { runHealthCheck } = await overviewModule; + const api = fakeApi({ + 'POST /prompt_manager/system/health': [{ job: job('running') }], + 'GET /prompt_manager/system/health': [{ report: REPORT }], + }); + const activity = fakeActivity(); + const checking = runHealthCheck(api, { activity }); + await new Promise(resolve => setImmediate(resolve)); + // The store follows the started job's event stream; nothing reads or polls the job here. + assert.equal(activity.loads, 0); + assert.equal(activity.find(JOB).state, 'running'); + assert.deepEqual(api.calls, ['POST /prompt_manager/system/health']); + activity.push(job('finished')); + assert.equal(await checking, REPORT); + assert.deepEqual(api.calls, ['POST /prompt_manager/system/health', 'GET /prompt_manager/system/health']); + assert.equal(activity.listeners(), 0); +}); + +test('a check already running elsewhere is waited for, not refused', async () => { + const { runHealthCheck } = await overviewModule; + const busy = Object.assign(new Error('busy'), { status: 409, code: 'busy' }); + const api = fakeApi({ + 'POST /prompt_manager/system/health': [busy], + 'GET /prompt_manager/system/health': [{ report: REPORT }], + }); + const activity = fakeActivity({ reads: [[job('running', 'f'.repeat(32), 'scan'), job('running')]] }); + const checking = runHealthCheck(api, { activity }); + await new Promise(resolve => setImmediate(resolve)); + assert.equal(activity.loads, 1); + activity.push(job('finished')); + assert.equal(await checking, REPORT); + // The other check ended between the refusal and the read: its report is the latest. + const ended = fakeApi({ 'POST /prompt_manager/system/health': [busy], 'GET /prompt_manager/system/health': [{ report: REPORT }] }); + assert.equal(await runHealthCheck(ended, { activity: fakeActivity({ reads: [[]] }) }), REPORT); +}); + +test('a failed, lost or reportless check rejects; other errors pass through', async () => { + const { runHealthCheck, waitForJob, HEALTH_FAILED_TEXT } = await overviewModule; + const failed = fakeApi({ 'POST /prompt_manager/system/health': [{ job: job('failed') }] }); + await assert.rejects(runHealthCheck(failed, { activity: fakeActivity() }), { message: HEALTH_FAILED_TEXT }); + // The store marks a job lost when the server no longer lists it (ComfyUI restarted). + const restarted = fakeActivity({ jobs: [job('running')] }); + const waiting = waitForJob(restarted, JOB); + restarted.push(job('lost')); + assert.equal(await waiting, 'lost'); + const empty = fakeApi({ 'POST /prompt_manager/system/health': [{ job: job('finished') }], + 'GET /prompt_manager/system/health': [{ report: null }] }); + await assert.rejects(runHealthCheck(empty, { activity: fakeActivity() }), { message: HEALTH_FAILED_TEXT }); + const broken = fakeApi({ 'POST /prompt_manager/system/health': [Object.assign(new Error('server'), { status: 500 })] }); + await assert.rejects(runHealthCheck(broken, { activity: fakeActivity() }), { message: 'server' }); + const invalid = fakeApi({ 'POST /prompt_manager/system/health': [{ job: null }] }); + await assert.rejects(runHealthCheck(invalid, { activity: fakeActivity() }), { message: 'Invalid job' }); +}); + +test('waiting stops when its signal aborts', async () => { + const { waitForJob } = await overviewModule; + const controller = new AbortController(); + const activity = fakeActivity({ jobs: [job('running')] }); + const waiting = waitForJob(activity, JOB, { signal: controller.signal }); + await new Promise(resolve => setImmediate(resolve)); + controller.abort(new Error('left')); + await assert.rejects(waiting, { message: 'left' }); + assert.equal(activity.listeners(), 0); + const gone = new AbortController(); + gone.abort(new Error('already')); + await assert.rejects(waitForJob(fakeActivity(), JOB, { signal: gone.signal }), { message: 'already' }); +}); diff --git a/tests/js/workspace-system-state.test.js b/tests/js/workspace-system-state.test.js new file mode 100644 index 00000000..af60efb0 --- /dev/null +++ b/tests/js/workspace-system-state.test.js @@ -0,0 +1,318 @@ +// System draft store (#249 System S2 Task 5): one Save row for browser, host and runtime fields (concept app.js +// renderSystem `.save-row` 705, `save-settings` 1320). It saves browser, then host with the GET revision, then runtime; +// a domain that fails keeps its fields dirty, and the status copy (D92) says what really happened. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const source = import('../../web/js/workspace/system-state.mjs'); + +/** Adapters that record every call; `host` answers like POST /prompt_manager/settings. */ +function adapters({ browser = {}, host = {}, runtime = {}, revision = 'sha256:one' } = {}) { + const calls = []; + const outcome = { browser: null, host: null, runtime: null }; + return { + calls, + outcome, + readBrowser: () => { calls.push(['readBrowser']); return { ...browser }; }, + readHost: async () => { calls.push(['readHost']); return { values: { ...host }, revision }; }, + readRuntime: async () => { calls.push(['readRuntime']); return { ...runtime }; }, + writeBrowser: async changes => { + calls.push(['writeBrowser', changes]); + if (outcome.browser) return outcome.browser(changes); + return { persisted: true }; + }, + writeHost: async (changes, options) => { + calls.push(['writeHost', changes, options]); + if (outcome.host) return outcome.host(changes, options); + return { success: true, persisted: true, revision: 'sha256:two', runtime_applied: true }; + }, + writeRuntime: async changes => { + calls.push(['writeRuntime', changes]); + if (outcome.runtime) return outcome.runtime(changes); + return { success: true }; + }, + }; +} + +async function loaded(domains) { + const { createSystemDrafts } = await source; + const drafts = createSystemDrafts(domains); + await Promise.all(['browser', 'host', 'runtime'].map(domain => drafts.load(domain))); + domains.calls.length = 0; + return drafts; +} + +test('sections follow the concept order and the production hashes', async () => { + const { SYSTEM_SECTIONS, isSystemSection } = await source; + assert.deepEqual(SYSTEM_SECTIONS.map(([id, label]) => [id, label]), [ + ['overview', 'Overview'], ['appearance', 'Appearance'], ['sources', 'Sources & scans'], + ['integrations', 'Integrations'], ['tagging', 'Auto-tagging'], ['maintenance', 'Maintenance'], + ['backups', 'Backup & export'], ['diagnostics', 'Diagnostics & logs'], + ]); + assert.equal(isSystemSection('backups'), true); + for (const value of ['general', '', 'constructor', '__proto__', undefined]) assert.equal(isSystemSection(value), false, value); +}); + +test('drafts read baselines and report dirty only for changed values', async () => { + const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 } }); + const drafts = await loaded(domains); + assert.equal(drafts.get('browser', 'layout'), 'grid'); + assert.equal(drafts.get('host', 'result_timeout'), 30); + assert.equal(drafts.dirty(), false); + drafts.set('host', 'result_timeout', 45); + assert.equal(drafts.get('host', 'result_timeout'), 45); + assert.equal(drafts.dirty(), true); + assert.equal(drafts.dirty('host'), true); + assert.equal(drafts.dirty('browser'), false); + // Setting the saved value back is clean again. + drafts.set('host', 'result_timeout', 30); + assert.equal(drafts.dirty(), false); + assert.throws(() => drafts.set('elsewhere', 'x', 1), TypeError); + assert.throws(() => drafts.get('elsewhere', 'x'), TypeError); +}); + +test('save commits browser then host then runtime', async () => { + const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 }, runtime: { level: 'INFO' } }); + const drafts = await loaded(domains); + drafts.set('runtime', 'level', 'DEBUG'); + drafts.set('host', 'result_timeout', 45); + drafts.set('browser', 'layout', 'list'); + const outcome = await drafts.save(); + assert.deepEqual(outcome, { browser: 'saved', host: 'saved', runtime: 'applied' }); + assert.deepEqual(domains.calls, [ + ['writeBrowser', { layout: 'list' }], + ['writeHost', { result_timeout: 45 }, { expected_revision: 'sha256:one' }], + ['writeRuntime', { level: 'DEBUG' }], + ]); + assert.equal(drafts.dirty(), false); + // The next host save sends the revision the last one returned. + drafts.set('host', 'result_timeout', 60); + await drafts.save(); + assert.deepEqual(domains.calls.at(-1), ['writeHost', { result_timeout: 60 }, { expected_revision: 'sha256:two' }]); +}); + +test('host failure keeps host fields dirty and browser saved', async () => { + const domains = adapters({ browser: { layout: 'grid' }, host: { result_timeout: 30 } }); + const drafts = await loaded(domains); + domains.outcome.host = () => { throw Object.assign(new Error('Settings could not be written to config.json.'), { code: 'config-write-failed' }); }; + drafts.set('browser', 'layout', 'list'); + drafts.set('host', 'result_timeout', 45); + const outcome = await drafts.save(); + assert.deepEqual(outcome, { browser: 'saved', host: 'failed', runtime: 'clean' }); + assert.equal(drafts.dirty('browser'), false); + assert.equal(drafts.dirty('host'), true); + assert.equal(drafts.get('host', 'result_timeout'), 45); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + assert.deepEqual([...drafts.failed('browser')], []); + assert.equal(drafts.statusText(), 'Some changes were not saved. Review the highlighted settings.'); + // A save answered `persisted: false` without an error is not saved either. + domains.outcome.host = () => ({ success: false, persisted: false }); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'failed', runtime: 'clean' }); + // Editing the failed field clears its highlight and reads as unsaved again. + drafts.set('host', 'result_timeout', 50); + assert.deepEqual([...drafts.failed('host')], []); + assert.equal(drafts.statusText(), 'Unsaved changes.'); +}); + +test('stale revision reports conflict and keeps the draft', async () => { + const domains = adapters({ host: { result_timeout: 30 } }); + const drafts = await loaded(domains); + domains.outcome.host = () => { throw Object.assign(new Error('Settings changed since they were loaded.'), { code: 'settings-changed' }); }; + drafts.set('host', 'result_timeout', 45); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'conflict', runtime: 'clean' }); + assert.equal(drafts.dirty('host'), true); + assert.equal(drafts.get('host', 'result_timeout'), 45); + assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.'); + // The conflict read the host values again (their new revision), keeping the draft. + assert.deepEqual(domains.calls.slice(-2).map(([name]) => name), ['writeHost', 'readHost']); + // Reloading the section reads the new revision and keeps the draft (Q10). + domains.outcome.host = null; + await drafts.load('host'); + assert.equal(drafts.get('host', 'result_timeout'), 45); + await drafts.save(); + assert.deepEqual(domains.calls.at(-1)[2], { expected_revision: 'sha256:one' }); +}); + +test('host save without a read revision never posts', async () => { + const { createSystemDrafts } = await source; + // Never read: an edit made before the host values (and their revision) arrived must not merge blindly. + const unread = adapters({ host: { result_timeout: 30 } }); + const drafts = createSystemDrafts(unread); + drafts.set('host', 'result_timeout', 45); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'failed', runtime: 'clean' }); + assert.deepEqual(unread.calls, []); + assert.equal(drafts.dirty('host'), true); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + // A read of an unreadable configuration has no revision either. + const unreadable = adapters({ host: { result_timeout: 30 }, revision: null }); + const blind = createSystemDrafts(unreadable); + await blind.load('host'); + blind.set('host', 'result_timeout', 45); + assert.equal((await blind.save()).host, 'failed'); + assert.equal(unreadable.calls.filter(([name]) => name === 'writeHost').length, 0); + // Once read, the same draft saves with the revision. + await drafts.load('host'); + assert.equal((await drafts.save()).host, 'saved'); + assert.deepEqual(unread.calls.at(-1), ['writeHost', { result_timeout: 45 }, { expected_revision: 'sha256:one' }]); +}); + +test('reloadHost reads the host values and says whether it could', async () => { + const { createSystemDrafts } = await source; + const domains = adapters({ host: { result_timeout: 30 } }); + const drafts = createSystemDrafts(domains); + assert.equal(await drafts.reloadHost(), true); + assert.equal(drafts.get('host', 'result_timeout'), 30); + const broken = createSystemDrafts({ ...domains, readHost: async () => { throw new Error('offline'); } }); + assert.equal(await broken.reloadHost(), false); + broken.set('host', 'result_timeout', 45); + assert.equal((await broken.save()).host, 'failed'); +}); + +test('no domain called when clean', async () => { + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = await loaded(domains); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'clean', runtime: 'clean' }); + assert.deepEqual(domains.calls, []); + drafts.set('browser', 'layout', 'list'); + drafts.discard(); + assert.equal(drafts.dirty(), false); + assert.equal(drafts.get('browser', 'layout'), 'grid'); + assert.deepEqual(await drafts.save(), { browser: 'clean', host: 'clean', runtime: 'clean' }); + assert.deepEqual(domains.calls, []); +}); + +test('status text per outcome', async () => { + const { statusText, SETTINGS_STATUS } = await source; + assert.deepEqual(SETTINGS_STATUS, { + clean: 'No unsaved changes.', + dirty: 'Unsaved changes.', + browser: 'Saved in this browser.', + host: 'Saved to the server configuration.', + runtime: 'Applied until the server restarts.', + partial: 'Some changes were not saved. Review the highlighted settings.', + tabOnly: 'Saved for this tab only; browser storage is unavailable.', + conflict: 'Settings changed elsewhere. Reload this section to see them.', + }); + const clean = { browser: 'clean', host: 'clean', runtime: 'clean' }; + assert.equal(statusText(clean), 'No unsaved changes.'); + assert.equal(statusText({ ...clean, browser: 'saved' }), 'Saved in this browser.'); + assert.equal(statusText({ ...clean, host: 'saved' }), 'Saved to the server configuration.'); + assert.equal(statusText({ ...clean, runtime: 'applied' }), 'Applied until the server restarts.'); + assert.equal(statusText({ ...clean, browser: 'tab-only' }), 'Saved for this tab only; browser storage is unavailable.'); + assert.equal(statusText({ browser: 'saved', host: 'saved', runtime: 'applied' }), + 'Saved in this browser. Saved to the server configuration. Applied until the server restarts.'); + assert.equal(statusText({ ...clean, browser: 'saved', runtime: 'failed' }), + 'Some changes were not saved. Review the highlighted settings.'); + assert.equal(statusText({ browser: 'failed', host: 'conflict', runtime: 'clean' }), + 'Settings changed elsewhere. Reload this section to see them.'); +}); + +test('the store says unsaved, then the save outcome, and notifies subscribers', async () => { + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = await loaded(domains); + const seen = []; + const unsubscribe = drafts.subscribe(() => seen.push(drafts.statusText())); + assert.equal(drafts.statusText(), 'No unsaved changes.'); + drafts.set('browser', 'layout', 'list'); + domains.outcome.browser = () => ({ persisted: false }); + await drafts.save(); + assert.deepEqual(seen, ['Unsaved changes.', 'Saved for this tab only; browser storage is unavailable.']); + unsubscribe(); + drafts.set('browser', 'layout', 'masonry'); + assert.equal(seen.length, 2); + // The tab-only value is this tab's baseline now. + drafts.set('browser', 'layout', 'list'); + assert.equal(drafts.dirty(), false); +}); + +test('an edit made while a save runs stays dirty', async () => { + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = await loaded(domains); + let release; + domains.outcome.browser = () => new Promise(resolve => { release = () => resolve({ persisted: true }); }); + drafts.set('browser', 'layout', 'list'); + const saving = drafts.save(); + // A second Save while one runs joins it instead of writing twice. + assert.equal(drafts.save(), saving); + assert.equal(drafts.saving(), true); + await new Promise(resolve => setImmediate(resolve)); + drafts.set('browser', 'layout', 'masonry'); + release(); + assert.deepEqual(await saving, { browser: 'saved', host: 'clean', runtime: 'clean' }); + assert.equal(drafts.saving(), false); + assert.equal(drafts.dirty('browser'), true); + assert.equal(drafts.get('browser', 'layout'), 'masonry'); + assert.equal(domains.calls.filter(([name]) => name === 'writeBrowser').length, 1); +}); + +test('a failed read keeps the store usable and drafts survive a reload (Q10)', async () => { + const { createSystemDrafts } = await source; + const domains = adapters({ browser: { layout: 'grid' } }); + const drafts = createSystemDrafts(domains); + await drafts.load('browser'); + drafts.set('browser', 'layout', 'list'); + await drafts.load('browser'); + assert.equal(drafts.get('browser', 'layout'), 'list'); + assert.equal(drafts.dirty(), true); + const failing = createSystemDrafts({ ...domains, readHost: async () => { throw new Error('offline'); } }); + await assert.rejects(failing.load('host'), /offline/); + assert.equal(failing.dirty(), false); + assert.throws(() => createSystemDrafts({ ...domains, writeHost: null }), TypeError); +}); + +test('a rejected value drops its draft and is highlighted; a conflict elsewhere keeps the drafts', async () => { + const domains = adapters({ browser: { videoMute: true, videoLoop: true }, host: { result_timeout: 5 } }); + const drafts = await loaded(domains); + assert.deepEqual(drafts.saved('browser'), { videoMute: true, videoLoop: true }); + drafts.set('host', 'result_timeout', 12); + assert.equal(drafts.dirty('host'), true); + drafts.reject('host', 'result_timeout'); + assert.equal(drafts.dirty('host'), false); + assert.deepEqual([...drafts.failed('host')], ['result_timeout']); + assert.equal(drafts.get('host', 'result_timeout'), 5); + // A valid value clears the highlight. + drafts.set('host', 'result_timeout', 7); + assert.deepEqual([...drafts.failed('host')], []); + drafts.discard(); + drafts.set('browser', 'videoMute', false); + drafts.conflict('browser', []); + assert.equal(drafts.statusText(), 'Unsaved changes.'); + drafts.conflict('browser', ['videoMute']); + assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.'); + assert.deepEqual([...drafts.failed('browser')], ['videoMute']); + assert.equal(drafts.get('browser', 'videoMute'), false); + assert.throws(() => drafts.reject('nowhere', 'x'), TypeError); +}); + +test('a rejected value says why in the status until the next edit', async () => { + const drafts = await loaded(adapters({ host: { result_timeout: 5 } })); + drafts.reject('host', 'result_timeout'); + assert.equal(drafts.statusText(), 'Some changes were not saved. Review the highlighted settings.'); + drafts.set('host', 'result_timeout', 9); + assert.equal(drafts.statusText(), 'Unsaved changes.'); +}); + +test('a browser write refused as changed elsewhere keeps the draft and reads the browser values again', async () => { + const domains = adapters({ browser: { workspaceDensity: 'comfortable' } }); + const drafts = await loaded(domains); + drafts.set('browser', 'workspaceDensity', 'compact'); + domains.outcome.browser = () => { throw Object.assign(new Error('changed'), { code: 'settings-changed' }); }; + assert.deepEqual(await drafts.save(), { browser: 'conflict', host: 'clean', runtime: 'clean' }); + assert.deepEqual(domains.calls.map(([name]) => name), ['writeBrowser', 'readBrowser']); + assert.equal(drafts.get('browser', 'workspaceDensity'), 'compact'); + assert.deepEqual([...drafts.failed('browser')], ['workspaceDensity']); + assert.equal(drafts.statusText(), 'Settings changed elsewhere. Reload this section to see them.'); +}); + +test('host read state says whether the server settings could be read', async () => { + const { createSystemDrafts } = await source; + const domains = adapters({ host: { result_timeout: 5 } }); + let fail = true; + const drafts = createSystemDrafts({ ...domains, readHost: async () => { if (fail) throw new Error('offline'); return { values: {}, revision: 'r' }; } }); + assert.equal(drafts.hostState(), 'pending'); + assert.equal(await drafts.reloadHost(), false); + assert.equal(drafts.hostState(), 'failed'); + fail = false; + assert.equal(await drafts.reloadHost(), true); + assert.equal(drafts.hostState(), 'loaded'); +}); diff --git a/tests/js/workspace-toolbar.test.js b/tests/js/workspace-toolbar.test.js index 91e1c3b9..052bc0be 100644 --- a/tests/js/workspace-toolbar.test.js +++ b/tests/js/workspace-toolbar.test.js @@ -76,6 +76,7 @@ function fakeDOM() { get activeElement() { return active ?? body; }, createElement: tag => new Element(tag), createElementNS: (namespace, tag) => new Element(tag), + createTextNode: text => Object.assign(new Element('#text'), { _text: String(text) }), }; const all = (root, predicate, found = []) => { if (predicate(root)) found.push(root); @@ -790,3 +791,145 @@ test('the Filters button shows the active refinement count', async () => { dom.restore(); } }); + +// #249 System S2 Task 5: the scan review's dialog and backup box (D55) generalised for every record-changing review. +const scanModule = import('../../web/js/workspace/activity-scan.mjs'); +const bodyShape = dom => byId(dom, 'modal-body').children.map(node => [node.tagName.toLowerCase(), node.className].filter(Boolean).join('.')); + +test('backupNotice is the concept .notice with its copy and one Back up now action', async () => { + const { backupNotice, SCAN_BACKUP_COPY } = await dialogsModule; + const dom = fakeDOM(); + try { + let calls = 0; + const notice = backupNotice({ copy: SCAN_BACKUP_COPY, onBackup: () => { calls++; } }); + assert.equal(notice.className, 'notice'); + assert.deepEqual(tags(notice), ['div', 'div']); + assert.deepEqual(tags(notice.children[0]), ['h3', 'p']); + assert.equal(notice.children[0].children[0].textContent, 'Back up the database first (recommended)'); + assert.equal(notice.children[0].children[1].textContent, + 'Downloads a copy of the prompts database before the scan changes any records.'); + assert.equal(notice.children[1].className, 'row-actions'); + const action = button(notice, 'Back up now'); + action.click(); + assert.equal(calls, 1); + assert.throws(() => backupNotice({ copy: { heading: '', text: 'x' }, onBackup() {} }), TypeError); + assert.throws(() => backupNotice({ copy: SCAN_BACKUP_COPY }), TypeError); + } finally { + dom.restore(); + } +}); + +test('openReviewDialog lays out heading, grid, options, sentence, backup box and actions', async () => { + const { openReviewDialog, closeWorkspaceDialog } = await dialogsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const trigger = new dom.Element('button'); + dom.body.append(trigger); + const option = new dom.Element('div'); + option.className = 'setting-row'; + const started = []; + const review = openReviewDialog({ scope, title: 'Review maintenance plan', heading: 'Missing-file links', + grid: [['Scope', '2 stale image links'], ['Media on disk', 'No source files removed']], options: [option], + sentence: 'Cancelling leaves all data untouched.', backup: { copy: { heading: 'Back up first', text: 'A copy.' }, onBackup() {} }, + cancelLabel: 'Cancel', primary: { label: 'Apply reviewed operation' }, onStart: () => started.push('start'), trigger }); + assert.equal(byId(dom, 'modal-title').textContent, 'Review maintenance plan'); + assert.deepEqual(bodyShape(dom), ['h3', 'dl.meta-grid', 'div.setting-row', 'p.small', 'div.notice', 'div.dialog-actions']); + const grid = byId(dom, 'modal-body').children[1]; + assert.deepEqual(grid.children.map(entry => [tags(entry), entry.children[0].textContent, entry.children[1].textContent]), [ + [['dt', 'dd'], 'Scope', '2 stale image links'], [['dt', 'dd'], 'Media on disk', 'No source files removed']]); + review.setValue('Scope', '3 stale image links'); + assert.equal(grid.children[0].children[1].textContent, '3 stale image links'); + const actions = byId(dom, 'modal-body').children.at(-1); + assert.deepEqual(actions.children.map(node => [node.textContent, node.className]), + [['Cancel', ''], ['Apply reviewed operation', 'ca-primary']]); + actions.children[1].click(); + assert.deepEqual(started, ['start']); + assert.equal(modal(dom).open, false); + assert.equal(document.activeElement, trigger); + + // A danger primary (the concept's Restore) has no sentence, heading, options or backup box when none is given. + openReviewDialog({ scope, title: 'Restore preview', grid: [['Integrity', 'Verified']], backup: null, + cancelLabel: 'Cancel', primary: { label: 'Restore', danger: true }, onStart() {} }); + assert.deepEqual(bodyShape(dom), ['dl.meta-grid', 'div.dialog-actions']); + assert.equal(button(byId(dom, 'modal-body'), 'Restore').className, 'danger'); + button(byId(dom, 'modal-body'), 'Cancel').click(); + assert.equal(modal(dom).open, false); + assert.throws(() => openReviewDialog({ scope, title: 'x', grid: [['a']], cancelLabel: 'Cancel', primary: { label: 'Go' }, + onStart() {} }), TypeError); + assert.throws(() => openReviewDialog({ scope, title: 'x', grid: [], cancelLabel: 'Cancel', primary: {}, onStart() {} }), TypeError); + } finally { + scope.dispose(); + closeWorkspaceDialog(); + dom.restore(); + } +}); + +test('review dialog backup box keeps the dialog open and toasts the outcome', async () => { + const { openReviewDialog, closeWorkspaceDialog, SCAN_BACKUP_COPY } = await dialogsModule; + const { downloadDatabaseBackup, BACKUP_DONE, BACKUP_FAILED } = await scanModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + const created = URL.createObjectURL; + const revoked = URL.revokeObjectURL; + URL.createObjectURL = () => 'blob:backup'; + URL.revokeObjectURL = () => {}; + try { + const toasts = []; + let answer; + const fetchUrl = () => new Promise(resolve => { answer = resolve; }); + openReviewDialog({ scope, title: 'Review metadata scan', grid: [['Sources', '2 output folders']], + backup: { copy: SCAN_BACKUP_COPY, onBackup: () => downloadDatabaseBackup({ fetchUrl, notify: text => toasts.push(text) }) }, + cancelLabel: 'Cancel', primary: { label: 'Start scan' }, onStart() { throw new Error('not started'); } }); + const action = button(byId(dom, 'modal-body'), 'Back up now'); + action.click(); + assert.equal(action.textContent, 'Backing up…'); + assert.equal(action.getAttribute('aria-disabled'), 'true'); + // A second press while the copy downloads is ignored. + action.click(); + answer({ ok: true, blob: async () => new Blob(['db']), headers: { get: () => 'attachment; filename="prompts_backup_1.db"' } }); + await tick(); + assert.deepEqual(toasts, [BACKUP_DONE]); + assert.equal(modal(dom).open, true); + assert.equal(action.textContent, 'Back up now'); + assert.equal(action.getAttribute('aria-disabled'), null); + action.click(); + answer({ ok: false, status: 500 }); + await tick(); + assert.deepEqual(toasts, [BACKUP_DONE, BACKUP_FAILED]); + assert.equal(modal(dom).open, true); + } finally { + URL.createObjectURL = created; + URL.revokeObjectURL = revoked; + scope.dispose(); + closeWorkspaceDialog(); + dom.restore(); + } +}); + +test('the scan review is a thin openReviewDialog caller with its scope switch', async () => { + const { openScanReviewDialog, closeWorkspaceDialog } = await dialogsModule; + const { createScope } = await coreModule; + const dom = fakeDOM(); + const scope = createScope(); + try { + const starts = []; + openScanReviewDialog({ scope, sources: 2, onBackup() {}, onStart: value => starts.push(value) }); + assert.equal(byId(dom, 'modal-title').textContent, 'Review metadata scan'); + assert.deepEqual(bodyShape(dom), ['dl.meta-grid', 'div.setting-row', 'p.small', 'div.notice', 'div.dialog-actions']); + const grid = byId(dom, 'modal-body').children[0]; + assert.deepEqual(grid.children.map(entry => entry.textContent), ['Sources2 output folders', 'ScopeNew and unresolved files']); + const allFiles = byId(dom, 'scan-all-files'); + allFiles.checked = true; + allFiles.dispatchEvent(new Event('change')); + assert.equal(grid.children[1].children[1].textContent, 'All files'); + button(byId(dom, 'modal-body'), 'Start scan').click(); + assert.deepEqual(starts, [{ scope: 'all' }]); + } finally { + scope.dispose(); + closeWorkspaceDialog(); + dom.restore(); + } +}); diff --git a/tests/test_api_admin.py b/tests/test_api_admin.py index 61d703b1..b34c0df9 100644 --- a/tests/test_api_admin.py +++ b/tests/test_api_admin.py @@ -868,6 +868,23 @@ class TestDeleteDuplicatesEndpoint(AdminAPITestCase): self.assertFalse(target.exists()) self.assertTrue(thumb_dir.exists()) + async def test_deleted_files_leave_the_cached_gallery_listing(self): + # The listing is cached for 30 s; a delete must not leave the gallery + # listing (and counting) a file that is gone. + GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)] + for name in ("keep.png", "gone.png"): + (self.output_dir / name).write_bytes(b"same bytes") + + async def listed(): + resp = await self.client.get("/prompt_manager/images/output") + data = await resp.json() + return data["total"], sorted(i["filename"] for i in data["images"]) + + self.assertEqual(await listed(), (2, ["gone.png", "keep.png"])) + resp = await self._post({"image_paths": ["gone.png"]}) + self.assertEqual((await resp.json())["deleted_count"], 1) + self.assertEqual(await listed(), (1, ["keep.png"])) + def test_unexpected_error_per_file_is_reported_without_path(self): self.api._delete_one_duplicate = lambda *a: (_ for _ in ()).throw( PermissionError(13, "Permission denied", os.path.join(self.tmpdir, "x.png")) diff --git a/tests/test_database_backup.py b/tests/test_database_backup.py index deb1346f..93686971 100644 --- a/tests/test_database_backup.py +++ b/tests/test_database_backup.py @@ -24,6 +24,9 @@ from database.operations import PromptDatabase from tests.open_handles import assert_closed from utils.hashing import generate_prompt_hash +# A bound on thread hand-offs that only a hung test reaches (never a pacing timer). +THREAD_WAIT_SECONDS = 120 + def _count_prompts(path): """Row count read through a brand-new connection (closed before return).""" @@ -350,7 +353,13 @@ class TestRestoreFromFile(BackupTestCase): def worker(): self.db.search_prompts(text="old") # opens this thread's connection ready.set() - proceed.wait(5) + # Wait for the restore itself, not for a timer: a stalled Windows + # runner once took longer than 5 s to restore, and the worker then + # read the old database before the restore had run. + if not proceed.wait(THREAD_WAIT_SECONDS): + results["error"] = "the restore did not finish" + self.db.close() + return try: results["texts"] = [p["text"] for p in self.db.search_prompts()] except Exception as exc: # noqa: BLE001 - reported through results @@ -360,12 +369,15 @@ class TestRestoreFromFile(BackupTestCase): thread = threading.Thread(target=worker) thread.start() - ready.wait(5) + self.assertTrue(ready.wait(THREAD_WAIT_SECONDS)) - self.db.model.restore_from_file(src) - proceed.set() - thread.join(5) + try: + self.db.model.restore_from_file(src) + finally: + proceed.set() + thread.join(THREAD_WAIT_SECONDS) + self.assertFalse(thread.is_alive()) self.assertNotIn("error", results) self.assertEqual(results["texts"], ["restored"]) self.assertEqual([p["text"] for p in self.db.search_prompts()], ["restored"]) diff --git a/tests/test_database_health.py b/tests/test_database_health.py new file mode 100644 index 00000000..2c0b40f0 --- /dev/null +++ b/tests/test_database_health.py @@ -0,0 +1,614 @@ +"""The read-only health check (System › Overview and Maintenance). + +``run_health_checks`` reads one snapshot on its own ``mode=ro`` connection: +SQLite's integrity and foreign-key checks, stored prompt hashes against +``utils.hashing``, duplicate pair rows, the legacy consistency queries, image +links whose file is gone (only under readable sources) and the Needs-review +count when one is supplied. It never writes, and it never turns a check it +could not run into a count. +""" + +import hashlib +import ntpath +import os +import sqlite3 +import sys +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +from database import health # noqa: E402 +from database.operations import PromptDatabase # noqa: E402 +from utils.hashing import generate_prompt_hash # noqa: E402 + +try: + from tests.open_handles import assert_closed # noqa: E402 +except ImportError: # discovered with tests/ as the top-level directory + from open_handles import assert_closed # noqa: E402 + +CHECK_IDS = [ + "integrity", + "foreign_keys", + "canonical_hashes", + "pair_uniqueness", + "missing_files", + "ambiguous_metadata", + "consistency", +] + + +def _digest(path): + if not os.path.exists(path): + return None + with open(path, "rb") as handle: + return hashlib.sha256(handle.read()).hexdigest() + + +class HealthTestCase(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.root = Path(os.path.realpath(self.temp.name)) + self.output = self.root / "output" + self.output.mkdir() + self.db_path = str(self.root / "prompts.db") + self.db = PromptDatabase(self.db_path) + + def tearDown(self): + self.db.close_all() + assert_closed(self, self.root) + self.temp.cleanup() + + def save(self, text, **metadata): + return self.db.save_prompt( + text, prompt_hash=generate_prompt_hash(text), **metadata + ) + + def image(self, prompt_id, name, *, exists=True, folder=None): + folder = Path(folder or self.output) + path = folder / name + if exists: + path.write_bytes(b"png") + return self.db.link_image_to_prompt(prompt_id, str(path)) + + def run_checks(self, roots=None, **kwargs): + roots = [str(self.output)] if roots is None else roots + return self.db.run_health_checks(roots, **kwargs) + + @staticmethod + def check(report, check_id): + return next(c for c in report["checks"] if c["id"] == check_id) + + def raw(self): + """A plain connection for planting damage the app itself never writes.""" + conn = sqlite3.connect(self.db_path) + conn.execute("PRAGMA foreign_keys=OFF") + return conn + + +class TestHealthReport(HealthTestCase): + def test_clean_library_passes_except_unavailable_review(self): + prompt = self.save("A clean prompt") + self.db.save_prompt_pair("Cat", "Blur") + self.image(prompt, "a.png") + report = self.run_checks(review_count=0) + self.assertTrue(report["success"]) + self.assertEqual([c["id"] for c in report["checks"]], CHECK_IDS) + self.assertEqual( + [c["label"] for c in report["checks"]], + [ + "Integrity check", + "Foreign keys & pair references", + "Canonical hashes", + "Pair uniqueness", + "Missing files", + "Ambiguous metadata", + "Consistency", + ], + ) + for check in report["checks"]: + self.assertEqual(check["status"], "passed", check) + self.assertEqual(check["count"], 0, check) + self.assertEqual(report["status"], "passed") + self.assertRegex(report["checked_at"], r"^\d{4}-\d\d-\d\dT.*\+00:00$") + + def test_health_check_never_writes(self): + prompt = self.save("Untouched") + self.image(prompt, "kept.png") + self.image(prompt, "gone.png", exists=False) + with self.raw() as conn: + conn.execute("UPDATE prompts SET hash='stale' WHERE id=?", (prompt,)) + conn.close() + # The app's own connection stays open, as in the running server. + self.db.get_statistics() + files = [self.db_path, self.db_path + "-wal"] + before = [_digest(path) for path in files] + shm_size = os.path.getsize(self.db_path + "-shm") + media = sorted(os.listdir(self.output)) + opened = [] + real_connect = sqlite3.connect + + class Recording(sqlite3.Connection): + changes_at_close = None + + def close(self): + Recording.changes_at_close = self.total_changes + super().close() + + def connect(*args, **kwargs): + conn = real_connect(*args, factory=Recording, **kwargs) + opened.append((args, kwargs, conn)) + return conn + + with patch.object(health.sqlite3, "connect", side_effect=connect): + report = self.run_checks() + self.assertEqual(report["status"], "incomplete") + self.assertEqual(len(opened), 1) + args, kwargs, conn = opened[0] + self.assertTrue(str(args[0]).endswith("?mode=ro"), args) + self.assertTrue(kwargs.get("uri")) + self.assertEqual(Recording.changes_at_close, 0) + with self.assertRaises(sqlite3.ProgrammingError): # closed in finally + conn.execute("SELECT 1") + self.assertEqual([_digest(path) for path in files], before) + # The -shm holds WAL read marks every reader updates; it never grows. + self.assertEqual(os.path.getsize(self.db_path + "-shm"), shm_size) + self.assertEqual(sorted(os.listdir(self.output)), media) + with self.db.model.get_connection() as live: + stored = live.execute( + "SELECT hash FROM prompts WHERE id=?", (prompt,) + ).fetchone()[0] + self.assertEqual(stored, "stale") + + def test_connection_closed_when_a_check_raises(self): + self.save("Any") + with ( + patch.object( + health, + "_canonical_hashes", + side_effect=sqlite3.OperationalError("boom"), + ), + self.assertLogs("prompt_manager.database.health", "ERROR"), + ): + report = self.run_checks(review_count=0) + canonical = self.check(report, "canonical_hashes") + self.assertEqual(canonical["status"], "failed") + self.assertIsNone(canonical["count"]) + self.assertEqual(canonical["detail"], ["The check could not finish."]) + self.assertEqual(self.check(report, "integrity")["status"], "passed") + self.assertEqual(report["status"], "incomplete") + self.db.close_all() + assert_closed(self, self.root) + + def test_unopenable_database_fails_every_database_check(self): + missing = PromptDatabase.__new__(PromptDatabase) + missing.model = type("M", (), {"db_path": str(self.root / "nope.db")})() + with self.assertLogs("prompt_manager.database.health", "ERROR"): + report = missing.run_health_checks([str(self.output)], review_count=0) + statuses = {c["id"]: c["status"] for c in report["checks"]} + self.assertEqual(statuses["integrity"], "failed") + self.assertEqual(statuses["missing_files"], "failed") + self.assertEqual(statuses["ambiguous_metadata"], "passed") + self.assertEqual(report["status"], "incomplete") + self.assertFalse(os.path.exists(self.root / "nope.db")) + + def test_integrity_and_foreign_key_issues_reported(self): + prompt = self.save("Linked") + with self.raw() as conn: + conn.execute("CREATE TABLE probe(a)") + conn.execute("CREATE INDEX probe_a ON probe(a)") + conn.executemany("INSERT INTO probe VALUES(?)", [(i,) for i in range(16)]) + conn.execute( + "INSERT INTO generated_images(prompt_id,image_path,filename,file_path)" + " VALUES(?,?,?,?)", + (9999, str(self.output / "orphan.png"), "orphan.png", "orphan"), + ) + conn.execute( + "INSERT INTO prompt_tags(prompt_id,tag_id) VALUES(?,?)", (prompt, 777) + ) + conn.close() + with self.raw() as conn: + conn.execute("PRAGMA writable_schema=ON") + conn.execute( + "UPDATE sqlite_master SET sql='CREATE INDEX probe_a ON probe(a DESC)'" + " WHERE name='probe_a'" + ) + conn.close() + self.db.close_all() # the next read sees the edited schema + (self.output / "orphan.png").write_bytes(b"png") + with self.raw() as conn: + expected = [r[0] for r in conn.execute("PRAGMA integrity_check(100)")] + conn.close() + self.assertGreater(len(expected), 10) + report = self.run_checks(review_count=0) + integrity = self.check(report, "integrity") + self.assertEqual(integrity["status"], "issues") + self.assertEqual(integrity["count"], len(expected)) + self.assertEqual(integrity["detail"], expected[:10]) + self.assertIn("missing from index probe_a", integrity["detail"][0]) + foreign = self.check(report, "foreign_keys") + self.assertEqual(foreign["status"], "issues") + self.assertEqual(foreign["count"], 2) + self.assertEqual( + sorted(foreign["detail"]), + [ + "generated_images row 1 references a missing prompts row", + "prompt_tags row 1 references a missing tags row", + ], + ) + consistency = self.check(report, "consistency") + self.assertEqual(consistency["status"], "issues") + self.assertEqual(consistency["count"], 1) + self.assertEqual( + consistency["detail"], ["Image 1 references non-existent prompt 9999"] + ) + self.assertEqual(report["status"], "issues") + + def test_canonical_hash_mismatch_counts_rows(self): + ids = [self.save(f"Prompt {n}") for n in range(14)] + with self.raw() as conn: + for n, prompt_id in enumerate(ids[:12]): + conn.execute( + "UPDATE prompts SET hash=? WHERE id=?", (f"bad{n}", prompt_id) + ) + conn.execute("UPDATE prompts SET hash=NULL WHERE id=?", (ids[12],)) + conn.close() + # Normalised spellings hash alike: this one is not a mismatch. + self.save(" MIXED Case ") + report = self.run_checks(review_count=0) + canonical = self.check(report, "canonical_hashes") + self.assertEqual(canonical["status"], "issues") + self.assertEqual(canonical["count"], 13) + self.assertEqual(len(canonical["detail"]), 10) + self.assertEqual( + canonical["detail"][0], + f"Prompt {ids[0]}: the stored hash does not match its text", + ) + + def test_duplicate_pair_rows_reported(self): + pair = self.db.save_prompt_pair("Cat", "") + positive = self.db.get_prompt_pair(pair)["positive_prompt_id"] + self.db.save_prompt_pair("Dog", "Blur") + with self.raw() as conn: + # Only the partial index keeps positive-only pairs unique; a restored + # or hand-edited database may have lost it. + conn.execute("DROP INDEX idx_prompt_pairs_positive_only") + conn.executemany( + "INSERT INTO prompt_pairs(positive_prompt_id,negative_prompt_id)" + " VALUES(?,NULL)", + [(positive,), (positive,)], + ) + conn.close() + report = self.run_checks(review_count=0) + pairs = self.check(report, "pair_uniqueness") + self.assertEqual(pairs["status"], "issues") + self.assertEqual(pairs["count"], 2) + self.assertEqual( + pairs["detail"], + [f"Positive prompt {positive} with no negative: 3 pair rows"], + ) + + def test_integrity_count_at_the_limit_is_flagged_capped(self): + class Rows: + def __init__(self, rows): + self.rows = rows + + def execute(self, _sql): + return iter([(row,) for row in self.rows]) + + capped = health._integrity(Rows(["bad"] * health.INTEGRITY_LIMIT), []) + self.assertEqual(capped["status"], "issues") + self.assertEqual(capped["count"], health.INTEGRITY_LIMIT) + self.assertTrue(capped["capped"]) + exact = health._integrity(Rows(["bad"] * 3), []) + self.assertEqual(exact["count"], 3) + self.assertFalse(exact["capped"]) + self.assertFalse(health._integrity(Rows(["ok"]), [])["capped"]) + + def test_an_unexpected_error_fails_only_its_own_check(self): + self.save("Any") + with ( + patch.object(health, "_pair_uniqueness", side_effect=KeyError("boom")), + self.assertLogs("prompt_manager.database.health", "ERROR"), + ): + report = self.run_checks(review_count=0) + statuses = {c["id"]: c["status"] for c in report["checks"]} + self.assertEqual(statuses.pop("pair_uniqueness"), "failed") + self.assertEqual(set(statuses.values()), {"passed"}) + + +class TestDatabaseUri(unittest.TestCase): + """The ``mode=ro`` URI of the database file, for every path form SQLite opens.""" + + def test_posix_path_is_a_local_file_uri(self): + self.assertEqual( + health.readonly_uri("/data/my db#1?.db", windows=False), + "file:///data/my%20db%231%3F.db?mode=ro", + ) + + def test_windows_drive_path(self): + self.assertEqual( + health.readonly_uri(r"C:\Users\Me\prompts.db", windows=True), + "file:///C:/Users/Me/prompts.db?mode=ro", + ) + + def test_windows_unc_path_has_an_empty_authority(self): + # pathlib spells \\server\share as file://server/share, which SQLite + # rejects (a non-local authority); an empty one keeps the UNC path. + uri = health.readonly_uri(r"\\server\share\Comfy UI\prompts.db", windows=True) + self.assertEqual(uri, "file:////server/share/Comfy%20UI/prompts.db?mode=ro") + self.assertTrue(ntpath.isabs(r"\\server\share\Comfy UI\prompts.db")) + + def test_uri_opens_a_path_with_uri_characters(self): + with tempfile.TemporaryDirectory() as temp: + # "?" is not a valid Windows file name character; "#", "%" and spaces are. + name = "a #b%20 c.db" if os.name == "nt" else "a #b?c%20.db" + path = os.path.join(os.path.realpath(temp), name) + conn = sqlite3.connect(path) + try: + conn.execute("CREATE TABLE t(a)") + conn.commit() + finally: + conn.close() + ro = sqlite3.connect(health.readonly_uri(path), uri=True) + try: + self.assertEqual(ro.execute("SELECT COUNT(*) FROM t").fetchone(), (0,)) + with self.assertRaises(sqlite3.OperationalError): + ro.execute("INSERT INTO t VALUES(1)") + finally: + ro.close() + + @unittest.skipUnless(os.name == "nt", "a native UNC path exists on Windows only") + def test_native_unc_path_opens(self): + with tempfile.TemporaryDirectory() as temp: + path = os.path.join(os.path.realpath(temp), "prompts.db") + conn = sqlite3.connect(path) + try: + conn.execute("CREATE TABLE t(a)") + conn.commit() + finally: + conn.close() + drive, rest = ntpath.splitdrive(path) + unc = "\\\\localhost\\" + drive[0] + "$" + rest + if not os.path.exists(unc): + self.skipTest("the administrative share is not reachable here") + ro = sqlite3.connect(health.readonly_uri(unc), uri=True) + try: + self.assertEqual(ro.execute("SELECT COUNT(*) FROM t").fetchone(), (0,)) + finally: + ro.close() + + +class TestMissingFiles(HealthTestCase): + def test_missing_files_counted_under_readable_sources_only(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + for n in range(12): + self.image(prompt, f"gone-{n}.png", exists=False) + elsewhere = self.root / "elsewhere" + elsewhere.mkdir() + self.image(prompt, "outside.png", exists=False, folder=elsewhere) + report = self.run_checks(review_count=0) + missing = self.check(report, "missing_files") + self.assertEqual(missing["status"], "issues") + self.assertEqual(missing["count"], 12) + self.assertEqual(len(missing["detail"]), 10) + self.assertRegex(missing["detail"][0], r"^Image \d+: gone-0\.png is missing$") + self.assertNotIn(str(self.root), " ".join(missing["detail"])) + with self.db.model.get_connection() as conn: + self.assertEqual( + conn.execute("SELECT COUNT(*) FROM generated_images").fetchone()[0], + 14, + ) + + def test_links_outside_every_source_are_named_not_counted(self): + prompt = self.save("Images") + elsewhere = self.root / "elsewhere" + elsewhere.mkdir() + self.image(prompt, "outside.png", exists=False, folder=elsewhere) + missing = self.check(self.run_checks(review_count=0), "missing_files") + self.assertEqual(missing["status"], "passed") + self.assertEqual(missing["count"], 0) + self.assertEqual( + missing["detail"], + ["1 link outside the image sources was not checked."], + ) + + def test_missing_file_vs_unavailable_root(self): + # Per source (S2 ruling): the readable source is counted, the absent + # one reports "unavailable" with no count, and the report is incomplete. + prompt = self.save("Images") + absent = self.root / "unmounted" + absent.mkdir() + self.image(prompt, "on-nas.png", folder=absent) + self.image(prompt, "gone.png", exists=False) + os.unlink(absent / "on-nas.png") + absent.rmdir() + report = self.run_checks([str(self.output), str(absent)], review_count=0) + missing = self.check(report, "missing_files") + self.assertEqual(missing["status"], "issues") + self.assertEqual(missing["count"], 1) + self.assertTrue(missing["partial"]) + self.assertEqual( + missing["sources"], + [ + {"name": "output", "status": "checked", "count": 1}, + {"name": "unmounted", "status": "unavailable", "count": None}, + ], + ) + self.assertEqual( + missing["detail"][0], + "1 image source is unavailable, so its files were not counted.", + ) + self.assertRegex(missing["detail"][1], r"^Image \d+: gone\.png is missing$") + # The link under the unavailable source is neither counted nor "outside". + self.assertEqual(len(missing["detail"]), 2) + self.assertNotIn(str(self.root), " ".join(missing["detail"])) + self.assertEqual(report["status"], "incomplete") + + def test_partial_sources_with_no_missing_file_are_not_passed(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + absent = self.root / "unmounted" + missing = self.check( + self.run_checks([str(self.output), str(absent)], review_count=0), + "missing_files", + ) + self.assertEqual(missing["status"], "unavailable") + self.assertEqual(missing["count"], 0) + self.assertTrue(missing["partial"]) + self.assertEqual( + [s["status"] for s in missing["sources"]], ["checked", "unavailable"] + ) + + def test_every_source_unavailable_counts_nothing(self): + prompt = self.save("Images") + absent = self.root / "unmounted" + self.image(prompt, "gone.png", exists=False, folder=self.output) + report = self.run_checks([str(absent)], review_count=0) + missing = self.check(report, "missing_files") + self.assertEqual(missing["status"], "unavailable") + self.assertIsNone(missing["count"]) + self.assertFalse(missing["partial"]) + self.assertEqual( + missing["sources"], + [{"name": "unmounted", "status": "unavailable", "count": None}], + ) + self.assertEqual( + missing["detail"], + ["1 image source is unavailable, so missing files were not counted."], + ) + + def test_each_source_counts_its_own_missing_files(self): + prompt = self.save("Images") + archive = self.root / "archive" + archive.mkdir() + self.image(prompt, "a.png", exists=False) + self.image(prompt, "b.png", exists=False, folder=archive) + self.image(prompt, "c.png", exists=False, folder=archive) + missing = self.check( + self.run_checks([str(self.output), str(archive)], review_count=0), + "missing_files", + ) + self.assertEqual(missing["count"], 3) + self.assertFalse(missing["partial"]) + self.assertEqual( + [(s["name"], s["count"]) for s in missing["sources"]], + [("output", 1), ("archive", 2)], + ) + + def test_no_sources_is_unavailable(self): + self.save("Images") + missing = self.check(self.run_checks([], review_count=0), "missing_files") + self.assertEqual(missing["status"], "unavailable") + self.assertIsNone(missing["count"]) + self.assertEqual(missing["detail"], ["No image source is available."]) + self.assertEqual(missing["sources"], []) + self.assertFalse(missing["partial"]) + + def test_a_bad_stored_path_fails_only_missing_files(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + with self.raw() as conn: + conn.execute( + "INSERT INTO generated_images(prompt_id,image_path,filename)" + " VALUES(?,?,?)", + (prompt, str(self.root / "else\x00where.png"), "bad.png"), + ) + conn.close() + real_realpath = os.path.realpath + + def realpath(path, *args, **kwargs): + # POSIX (and Windows before Python 3.11) raise for the NUL byte; + # newer ntpath returns it unchanged. Raise everywhere, as POSIX does. + if "\x00" in str(path): + raise ValueError("embedded null byte") + return real_realpath(path, *args, **kwargs) + + with ( + patch.object(health.os.path, "realpath", side_effect=realpath), + self.assertLogs("prompt_manager.database.health", "ERROR"), + ): + report = self.run_checks(review_count=0) + statuses = {c["id"]: c["status"] for c in report["checks"]} + self.assertEqual(statuses.pop("missing_files"), "failed") + self.assertEqual(set(statuses.values()), {"passed"}) + + def test_files_are_checked_after_the_read_snapshot_ends(self): + prompt = self.save("Images") + self.image(prompt, "here.png") + self.image(prompt, "gone.png", exists=False) + opened = [] + real_connect = sqlite3.connect + + def connect(*args, **kwargs): + conn = real_connect(*args, **kwargs) + opened.append(conn) + return conn + + real_exists = os.path.exists + in_snapshot = [] + + def exists(path): + for conn in opened: + try: + in_snapshot.append(conn.in_transaction) + except sqlite3.ProgrammingError: # closed: no snapshot + in_snapshot.append(False) + return real_exists(path) + + with ( + patch.object(health.sqlite3, "connect", side_effect=connect), + patch.object(health.os.path, "exists", side_effect=exists), + ): + missing = self.check(self.run_checks(review_count=0), "missing_files") + self.assertEqual((missing["status"], missing["count"]), ("issues", 1)) + self.assertEqual(len(opened), 1) + self.assertTrue(in_snapshot) + self.assertNotIn(True, in_snapshot) + + +class TestAmbiguousMetadata(HealthTestCase): + def test_ambiguous_metadata_unavailable_without_review_queue(self): + self.save("Any") + report = self.run_checks() + ambiguous = self.check(report, "ambiguous_metadata") + self.assertEqual(ambiguous["status"], "unavailable") + self.assertIsNone(ambiguous["count"]) + self.assertEqual(ambiguous["detail"], ["Needs review is not available yet."]) + self.assertEqual(report["status"], "incomplete") + + def test_review_count_reports_images_needing_review(self): + report = self.run_checks(review_count=3) + ambiguous = self.check(report, "ambiguous_metadata") + self.assertEqual(ambiguous["status"], "issues") + self.assertEqual(ambiguous["count"], 3) + self.assertEqual(ambiguous["detail"], []) + self.assertEqual(report["status"], "issues") + + def test_invalid_review_count_is_rejected(self): + for value in (-1, True, "3", 1.5): + with self.assertRaises(ValueError): + self.run_checks(review_count=value) + + +class TestOverallStatus(unittest.TestCase): + def test_overall_status_precedence(self): + def checks(*statuses): + return [{"status": status} for status in statuses] + + self.assertEqual(health.overall_status(checks("passed", "passed")), "passed") + self.assertEqual(health.overall_status(checks("passed", "issues")), "issues") + self.assertEqual( + health.overall_status(checks("issues", "unavailable")), "incomplete" + ) + self.assertEqual(health.overall_status(checks("failed")), "incomplete") + # A check that found issues under only some sources is incomplete too. + partial = [{"status": "issues", "partial": True}, {"status": "passed"}] + self.assertEqual(health.overall_status(partial), "incomplete") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_generation_params.py b/tests/test_generation_params.py index 990d3a60..507cea29 100644 --- a/tests/test_generation_params.py +++ b/tests/test_generation_params.py @@ -5,7 +5,10 @@ returns for the same input (tests/test_generation_params_js_parity.py checks that directly when Node is available). """ +import json import os +import shutil +import subprocess # nosec B404 - runs a fixed Node one-liner import sys import unittest @@ -17,6 +20,7 @@ from utils.generation_params import ( # noqa: E402 extract_generation_params, extract_from_records, parse_js_json, + parse_page_json, ) try: @@ -219,6 +223,47 @@ class ParsesLikeJavaScript(unittest.TestCase): run("seed-beyond-double-precision")["seed"], 4974052037430000001.0 ) + def test_page_parse_reads_nan_numbers_as_null(self): + # gallery.js and admin.js replace NaN with null before JSON.parse. + self.assertEqual( + parse_page_json('{"a": NaN, "b": [NaN], "t": "NaN"}'), + {"a": None, "b": [None], "t": "NaN"}, + ) + self.assertEqual( + parse_page_json('{"seed": 9007199254740993}'), {"seed": 9007199254740992.0} + ) + for text in ('{"a": Infinity}', "[-Infinity]", "not json", "", None): + with self.subTest(text=text): + self.assertIsNone(parse_page_json(text)) + + @unittest.skipUnless(shutil.which("node"), "Node is not installed") + def test_page_parse_equals_the_legacy_pages_for_nan_numbers(self): + # gallery.js/admin.js: two NaN regexes, then JSON.parse (Infinity throws). + texts = [ + '{"1": {"is_changed": [NaN]}, "2": {"is_changed": NaN, "v": 1.5}}', + '{"a":NaN,"b":[NaN, 2]}', + '{"a": Infinity}', + '{"seed": 9007199254740993}', + ] + script = ( + "const texts = JSON.parse(require('fs').readFileSync(0, 'utf8'));" + "console.log(JSON.stringify(texts.map((t) => { try { return {v: JSON.parse(" + "t.replace(/:\\s*NaN\\b/g, ': null').replace(/\\bNaN\\b/g, 'null'))}; }" + " catch (e) { return null; } })));" + ) + completed = subprocess.run( # nosec B603 B607 - fixed argv, no shell + ["node", "-e", script], + input=json.dumps(texts), + capture_output=True, + text=True, + check=True, + timeout=60, + ) + expected = json.loads(completed.stdout) + for text, page in zip(texts, expected): + with self.subTest(text=text): + self.assertEqual(parse_page_json(text), page and page["v"]) + def test_duplicate_keys_keep_the_last_value(self): self.assertEqual(parse_js_json('{"a": 1, "a": 2}'), {"a": 2}) diff --git a/tests/test_library.py b/tests/test_library.py index fd83cb7c..242b76d4 100644 --- a/tests/test_library.py +++ b/tests/test_library.py @@ -1706,6 +1706,103 @@ class TestLibrary(unittest.TestCase): self.db.list_library_images(positive, pair_id=foreign, archive="all") +class LibraryCounts(unittest.TestCase): + """System › Overview counts and usage statistics follow Library eligibility.""" + + setUp = TestLibrary.setUp + tearDown = TestLibrary.tearDown + save = TestLibrary.save + state = TestLibrary.state + + def test_counts_use_library_eligibility(self): + self.db.save_prompt_pair("Cat", "Blur") + archived = self.db.save_prompt_pair("Cat", "Noise") + self.db.archive_prompt_pair(archived) + titled_pair = self.db.save_prompt_pair("Dog", "Titled negative") + titled = self.db.get_prompt_pair(titled_pair)["negative_prompt_id"] + self.state(titled, "A titled parent") # zero pairs of its own: counted + legacy = self.save("Legacy") + self.db.link_image_to_prompt(legacy, str(Path(self.temp.name) / "a.png")) + self.db.link_image_to_prompt(legacy, str(Path(self.temp.name) / "b.png")) + counts = self.db.library_counts() + # Cat, Dog, the titled negative and Legacy; "Blur" and "Noise" are + # negative-only. + self.assertEqual(counts["positive_prompts"], 4) + # Cat+Blur and Dog+Titled negative; the archived pair is excluded. + self.assertEqual(counts["negative_variants"], 2) + self.assertEqual(counts["images"], 2) + self.assertEqual(counts["rated_images"], 0) + listed = self.db.list_library() + self.assertEqual(counts["positive_prompts"], listed["total"]) + self.assertEqual(counts["negative_variants"], listed["variant_total"]) + + def test_counts_of_an_empty_library_are_zero(self): + self.assertEqual( + self.db.library_counts(), + { + "positive_prompts": 0, + "negative_variants": 0, + "images": 0, + "rated_images": 0, + }, + ) + + def test_rated_images_follow_their_linked_prompt_rating(self): + # An image carries no rating of its own: it shows its linked prompt's + # (the Images "Highest rated" sort), so "rated" counts those images. + folder = Path(self.temp.name) + rated = self.save("Rated", rating=4) + unrated = self.save("Unrated") + for name in ("a.png", "b.png", "c.png"): + self.db.link_image_to_prompt(rated, str(folder / name)) + self.db.link_image_to_prompt(unrated, str(folder / "d.png")) + counts = self.db.library_counts() + self.assertEqual(counts["images"], 4) + self.assertEqual(counts["rated_images"], 3) + self.db.update_prompt_rating(unrated, 2) + self.assertEqual(self.db.library_counts()["rated_images"], 4) + + def test_statistics_sum_real_run_counts(self): + a = self.save("Alpha", category="Landscapes", rating=5) + b = self.save("Beta", category="Landscapes", rating=4) + self.save("Gamma", category="Landscapes") # unrated: not in the average + c = self.save("Delta", category="Portraits", rating=3) + blank = self.save("Epsilon", category=" ") + uncategorised = self.save("Zeta") + self.db.record_prompt_use(a, times=3) + self.db.record_prompt_use(b, times=2) + self.db.record_prompt_use(c) + self.db.record_prompt_use(blank) + self.db.record_prompt_use(uncategorised, times=4) + # A negative-only prompt's runs are the pair's runs: never counted twice. + pair = self.db.save_prompt_pair("Alpha", "Only negative") + negative = self.db.get_prompt_pair(pair)["negative_prompt_id"] + with self.db.model.get_connection() as connection: + connection.execute( + "UPDATE prompts SET category='Landscapes',run_count=50 WHERE id=?", + (negative,), + ) + rows = self.db.usage_by_category() + self.assertEqual( + rows, + [ + {"category": "Landscapes", "runs": 5, "average_rating": 4.5}, + {"category": None, "runs": 5, "average_rating": None}, + {"category": "Portraits", "runs": 1, "average_rating": 3.0}, + ], + ) + + def test_counts_and_usage_write_nothing(self): + self.save("Alpha", category="A") + with self.db.model.get_connection() as connection: + before = connection.total_changes + self.db.library_counts() + self.db.usage_by_category() + with self.db.model.get_connection() as connection: + self.assertEqual(connection.total_changes, before) + self.assertFalse(connection.in_transaction) + + class DisplayTitleUnits(unittest.TestCase): """The first-line title is cut like the Library's JavaScript title().""" diff --git a/tests/test_nonfinite_metadata.py b/tests/test_nonfinite_metadata.py new file mode 100644 index 00000000..ff6abcd5 --- /dev/null +++ b/tests/test_nonfinite_metadata.py @@ -0,0 +1,360 @@ +"""ComfyUI's ``NaN``/``Infinity`` metadata ingests, and is stored as strict JSON. + +ComfyUI serializes the prompt with Python's ``json.dumps``, so a node whose +``IS_CHANGED`` returns ``float("nan")`` (the "always re-run" idiom) writes +``"is_changed": [NaN]`` into every saved PNG; widgets can hold ``Infinity``. +The legacy pages replaced ``NaN`` with ``null`` before ``JSON.parse``, so such +files must ingest. Whatever is stored or answered as JSON stays strict (the +non-finite numbers become ``null``) and the prompt text and hash never change. +""" + +import json +import logging +import os +import sqlite3 +import sys +import tempfile +import time +import types +import unittest +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import patch + +from PIL import Image, PngImagePlugin + +from database.operations import PromptDatabase +from tests.history_waits import drain_writer +from tests.test_api_admin import AdminAPITestCase +from tests.test_api_images import ImageAPITestCase +from utils.hashing import generate_prompt_hash +from utils.image_monitor import ImageGenerationHandler + +try: + from tests.open_handles import assert_closed +except ImportError: # discovered with tests/ as the top-level directory + from open_handles import assert_closed + +# The words "NaN" and "Infinity" inside the prompt are text, never numbers. +POSITIVE = " A cat named NaN beside an Infinity pool, ratio: NaN, [NaN] " +NEGATIVE = "blurry" +NAN, INF = float("nan"), float("inf") + + +def graph(positive=POSITIVE, negative=NEGATIVE): + """An API prompt graph with non-finite numbers where ComfyUI writes them.""" + g = { + "1": {"class_type": "CLIPTextEncode", "inputs": {"text": positive}}, + "2": {"class_type": "CLIPTextEncode", "inputs": {"text": negative}}, + "3": { + "class_type": "KSampler", + "inputs": {"positive": ["1", 0], "negative": ["2", 0], "model": ["4", 0]}, + }, + "4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}}, + "5": { + "class_type": "Image Filter", + "inputs": {"strength": INF, "floor": -INF}, + "_meta": {"title": "Image Filter"}, + "is_changed": [NAN], + }, + } + return g + + +WORKFLOW = { + "nodes": [ + {"id": 5, "type": "Image Filter", "widgets_values": [NAN, INF, -INF]}, + ], + "links": [], +} + + +def comfy_text(value): + """The record text ComfyUI writes: Python's default, non-strict JSON.""" + text = json.dumps(value) + assert "NaN" in text and "Infinity" in text + return text + + +def write_png(path, prompt_graph=None, workflow=WORKFLOW): + info = PngImagePlugin.PngInfo() + info.add_text("prompt", comfy_text(prompt_graph or graph())) + if workflow is not None: + info.add_text("workflow", comfy_text(workflow)) + path.parent.mkdir(parents=True, exist_ok=True) + with Image.new("RGB", (4, 4)) as image: + image.save(path, pnginfo=info) + return path + + +def _reject(name): + raise ValueError(f"non-strict JSON constant {name}") + + +def strict_loads(text): + return json.loads(text, parse_constant=_reject) + + +class StrictRows: + """Assertions over the rows a write left behind.""" + + def image_rows(self, db): + with db.model.get_connection() as conn: + return [ + dict(row) + for row in conn.execute( + "SELECT prompt_id, workflow_data, prompt_metadata, parameters " + "FROM generated_images ORDER BY id" + ) + ] + + def assert_strict_snapshot(self, row, *, workflow=True): + prompt = strict_loads(row["prompt_metadata"]) + self.assertEqual(prompt["5"]["is_changed"], [None]) + self.assertEqual(prompt["5"]["inputs"], {"strength": None, "floor": None}) + self.assertEqual(prompt["1"]["inputs"]["text"], POSITIVE) + stored_workflow = strict_loads(row["workflow_data"]) + if workflow: + self.assertEqual( + stored_workflow["nodes"][0]["widgets_values"], [None, None, None] + ) + if row["parameters"] is not None: + strict_loads(row["parameters"]) + + def assert_prompt_text(self, db, prompt_id): + with db.model.get_connection() as conn: + row = conn.execute( + "SELECT text, hash FROM prompts WHERE id=?", (prompt_id,) + ).fetchone() + self.assertEqual(row["text"], POSITIVE.strip()) + self.assertEqual(row["hash"], generate_prompt_hash(POSITIVE)) + + +def dumps_strict(value, **kwargs): + from utils.json_strict import dumps_strict as dumps + + return dumps(value, **kwargs) + + +class TestDumpsStrict(unittest.TestCase): + def test_finite_values_serialize_byte_identically(self): + value = {"b": [1, 2.5, "NaN", None, True], "a": {"x": "Infinity"}} + for kwargs in ({}, {"sort_keys": True, "separators": (",", ":")}): + with self.subTest(kwargs=kwargs): + self.assertEqual( + dumps_strict(value, **kwargs), + json.dumps(value, allow_nan=False, **kwargs), + ) + + def test_non_finite_numbers_become_null_and_strings_stay(self): + value = {"n": NAN, "i": [INF, -INF], "s": "NaN Infinity", "t": (NAN, 1)} + encoded = dumps_strict(value, sort_keys=True) + self.assertEqual( + strict_loads(encoded), + {"n": None, "i": [None, None], "s": "NaN Infinity", "t": [None, 1]}, + ) + + def test_other_errors_still_raise(self): + cycle = [] + cycle.append(cycle) + for value in (cycle, [NAN, cycle]): + with self.assertRaisesRegex(ValueError, "Circular"): + dumps_strict(value) + with self.assertRaises(TypeError): + dumps_strict({"x": object()}) + self.assertEqual( + dumps_strict({"x": object()}, default=lambda _: NAN), '{"x": null}' + ) + + +class TestScanIngestsNonFiniteMetadata(StrictRows, AdminAPITestCase): + def ingest(self, name, prompt_graph=None): + path = write_png(self.output_dir / name, prompt_graph) + extracted = self.api._extract_comfyui_metadata(str(path)) + return self.api._ingest_scan_batch_sync([(path, extracted)]) + + def test_resolved_pair_with_nan_is_added_with_strict_snapshots(self): + counts = self.ingest("nan.png") + self.assertEqual(counts["failed"], 0) + self.assertEqual(counts["added"], 1) + rows = self.image_rows(self.api.db) + self.assertEqual(len(rows), 1) + self.assert_strict_snapshot(rows[0]) + self.assert_prompt_text(self.api.db, rows[0]["prompt_id"]) + pairs = self.api.db.list_prompt_pairs()["pairs"] + self.assertEqual( + [(p["positive_text"], p["negative_text"]) for p in pairs], + [(POSITIVE.strip(), NEGATIVE)], + ) + # A repeated scan compares the same strict snapshot: unchanged. + self.assertEqual(self.ingest("nan.png")["unchanged"], 1) + + async def test_scan_route_reports_no_failures(self): + from py.config import GalleryConfig + + write_png(self.output_dir / "nan.png") + GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)] + resp = await self.client.request("POST", "/prompt_manager/scan") + body = await resp.text() + self.assertEqual(resp.status, 200) + self.assertIn('"type": "complete"', body) + self.assertIn('"added": 1', body) + self.assertIn('"failed": 0', body) + + +class TestMonitorIngestsNonFiniteMetadata(StrictRows, unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.db = PromptDatabase(str(Path(self.temp.name) / "db.sqlite")) + self.pid = self.db.save_prompt( + POSITIVE.strip(), prompt_hash=generate_prompt_hash(POSITIVE) + ) + self.tracker = SimpleNamespace( + pop_next_prompt=lambda: None, + get_current_prompt=lambda: None, + extend_prompt_timeout=lambda *args: None, + ) + self.handler = ImageGenerationHandler(self.db, self.tracker) + self.handler.logger = logging.getLogger("test.nonfinite.monitor") + self.handler.logger.disabled = True + + def tearDown(self): + self.db.close_all() + assert_closed(self, self.temp.name) + self.temp.cleanup() + + def test_new_image_with_nan_links_its_pair(self): + # The monitor links only images a PromptManager node produced. + g = graph() + g["1"]["class_type"] = "PromptManager" + path = write_png(Path(self.temp.name) / "out" / "nan.png", g) + self.handler.process_new_image(str(path)) + rows = self.image_rows(self.db) + self.assertEqual([row["prompt_id"] for row in rows], [self.pid]) + self.assert_strict_snapshot(rows[0]) + self.assert_prompt_text(self.db, self.pid) + pairs = self.db.list_prompt_pairs()["pairs"] + self.assertEqual(len(pairs), 1) + self.assertEqual(pairs[0]["image_count"], 1) + + def test_legacy_link_stores_strict_json(self): + # The plain prompt link (no exact pair) must not write NaN either. + path = str(Path(self.temp.name) / "plain.png") + metadata = {"prompt": graph(), "workflow": WORKFLOW, "parameters": {"x": NAN}} + self.assertTrue(self.db.link_image_to_prompt(self.pid, path, metadata)) + row = self.image_rows(self.db)[0] + self.assert_strict_snapshot(row) + self.assertEqual(strict_loads(row["parameters"]), {"x": None}) + + def test_stored_images_read_back_without_non_finite_numbers(self): + path = str(Path(self.temp.name) / "legacy.png") + self.db.link_image_to_prompt(self.pid, path, {"prompt": graph()}) + with self.db.model.get_connection() as conn: + # A row an older version wrote with Python's default json.dumps. + conn.execute( + "UPDATE generated_images SET prompt_metadata=?", (comfy_text(graph()),) + ) + conn.commit() + images = self.db.get_prompt_images(str(self.pid)) + strict_loads(json.dumps(images, default=str)) + + +class TestInspectorAnswersStrictJSON(ImageAPITestCase): + async def test_inspect_route_resolves_and_answers_strict_json(self): + write_png(self.output_dir / "nan.png") + response = await self.client.get( + "/prompt_manager/images/inspect/nan.png", params={"root": "0"} + ) + self.assertEqual(response.status, 200) + result = strict_loads(await response.text()) + self.assertEqual(result["resolution"]["state"], "resolved") + self.assertEqual( + result["prompts"]["positive"], + {"status": "resolved", "text": POSITIVE, "source": "graph-inferred"}, + ) + + async def test_prompt_images_route_answers_strict_json(self): + pid = self.api.db.save_prompt( + POSITIVE.strip(), prompt_hash=generate_prompt_hash(POSITIVE) + ) + path = str(write_png(self.output_dir / "linked.png")) + self.api.db.link_image_to_prompt(pid, path, {"prompt": graph()}) + with self.api.db.model.get_connection() as conn: + conn.execute( + "UPDATE generated_images SET prompt_metadata=?", (comfy_text(graph()),) + ) + conn.commit() + response = await self.client.get(f"/prompt_manager/prompts/{pid}/images") + self.assertEqual(response.status, 200) + result = strict_loads(await response.text()) + self.assertEqual(len(result["images"]), 1) + stored = result["images"][0]["prompt_metadata"] + self.assertEqual(stored["1"]["inputs"]["text"], POSITIVE) + self.assertEqual(stored["5"]["is_changed"], [None]) + + +class TestHistoryCapturesNonFiniteGraphs(unittest.TestCase): + """The queued graph ComfyUI hands the hook may carry float('nan').""" + + JOB = "a0c26da3-eb26-405c-b143-c662a84c3112" + + def setUp(self): + self.directory = tempfile.TemporaryDirectory() + self.addCleanup(self.directory.cleanup) + self.db = PromptDatabase(os.path.join(self.directory.name, "history.db")) + self.addCleanup(self.db.model.reset_schema_cache) + self.addCleanup(self.db.model.close_all) + + def test_queued_graph_with_nan_records_its_run_and_contexts(self): + from tests.test_execution_history_hooks import HostQueue, status + from utils.execution_context_capture import record_runtime_context + from utils.execution_history import register_history_hook + + queue = HostQueue() + registration = register_history_hook( + SimpleNamespace(prompt_queue=queue), lambda: self.db + ) + self.addCleanup(lambda: registration.dispose(3)) + self.addCleanup(registration.writer.invalidate) + deadline = time.monotonic() + 3 + while not registration.writer.status().accepting: + self.assertLess(time.monotonic(), deadline, "writer never accepted") + time.sleep(0.005) + g = { + "p": { + "class_type": "PromptManagerPair", + "inputs": {"text": POSITIVE, "negative_text": NEGATIVE}, + }, + "s": { + "class_type": "KSampler", + "inputs": {"positive": ["p", 0], "negative": ["p", 1], "cfg": INF}, + "is_changed": [NAN], + }, + "5": graph()["5"], + } + extra = {"extra_pnginfo": {"workflow": WORKFLOW}} + queue.results.append(((0, self.JOB, g, extra, ["s"]), 1)) + queue.get() + host = types.ModuleType("comfy_execution.utils") + host.get_executing_context = lambda: SimpleNamespace( + prompt_id=self.JOB, node_id="p", list_index=0 + ) + with patch.dict(sys.modules, {"comfy_execution.utils": host}): + for role, text in (("positive", POSITIVE), ("negative", NEGATIVE)): + record_runtime_context( + "PromptManagerPair", text, "", "", text, role=role + ) + queue.task_done(1, {}, status("execution_success", job=self.JOB)) + drain_writer(registration.writer, clock=time.monotonic) + with self.db.model.get_connection() as conn: + conn.row_factory = sqlite3.Row + runs = [dict(r) for r in conn.execute("SELECT * FROM execution_runs")] + self.assertEqual([run["outcome"] for run in runs], ["succeeded"]) + contexts = self.db.get_execution_contexts(runs[0]["run_id"]) + self.assertEqual(contexts["state"], "complete") + self.assertEqual(contexts["contexts"][0]["positive"]["raw"], POSITIVE) + strict_loads(json.dumps(contexts)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_pair_scan.py b/tests/test_pair_scan.py index 8b65fb67..b8d914d1 100644 --- a/tests/test_pair_scan.py +++ b/tests/test_pair_scan.py @@ -165,6 +165,122 @@ class TestPairScan(AdminAPITestCase): ) self.assertEqual(self.api.db.get_recent_prompts()["prompts"], []) + def _nan_scanned(self, name, prompt_graph): + """Scan ``prompt_graph`` as ComfyUI writes it: ``is_changed`` is NaN.""" + stamped = { + node_id: {**node, "is_changed": [float("nan")]} + for node_id, node in prompt_graph.items() + } + text = json.dumps(stamped) + self.assertIn("NaN", text) + return self._scanned_text(name, text) + + def _scanned_text(self, name, prompt_text): + meta = {"prompt": prompt_text} + return self.api._ingest_scanned_file(self.output_dir / name, meta) + + @staticmethod + def _hires_fix_graph(positive): + """A PromptManager prompt on a base sampler and a hires-fix sampler. + + The base latent comes from a custom batch node and the hires pass from + a custom image filter: no sampler branch traces, so the scan links the + prompt the legacy pages show. + """ + return { + "1": { + "class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "m.safetensors"}, + }, + "2": { + "class_type": "MyTriggerWords", + "inputs": {"trigger_words": ["1", 2]}, + }, + "3": { + "class_type": "PromptManager", + "inputs": { + "text": positive, + "prepend_text": ["2", 0], + "append_text": "", + "clip": ["1", 1], + }, + }, + "4": { + "class_type": "CLIPTextEncode", + "inputs": {"text": "blur", "clip": ["1", 1]}, + }, + "5": {"class_type": "MyLatentBatch", "inputs": {"batch_size": 6}}, + "6": { + "class_type": "KSampler", + "inputs": { + "seed": 7, + "model": ["1", 0], + "positive": ["3", 0], + "negative": ["4", 0], + "latent_image": ["5", 0], + }, + }, + "7": { + "class_type": "VAEDecode", + "inputs": {"samples": ["6", 0], "vae": ["1", 2]}, + }, + "8": {"class_type": "MyImageFilter", "inputs": {"images": ["7", 0]}}, + "9": { + "class_type": "VAEEncode", + "inputs": {"pixels": ["8", 0], "vae": ["1", 2]}, + }, + "14": { + "class_type": "KSampler", + "inputs": { + "seed": 7, + "denoise": 0.25, + "model": ["1", 0], + "positive": ["3", 0], + "negative": ["4", 0], + "latent_image": ["9", 0], + }, + }, + "15": { + "class_type": "VAEDecode", + "inputs": {"samples": ["14", 0], "vae": ["1", 2]}, + }, + "16": {"class_type": "SaveImage", "inputs": {"images": ["15", 0]}}, + } + + def test_hires_fix_prompt_links_with_or_without_nan_is_changed(self): + # Legacy parity: the legacy pages read NaN as null before parsing, so they + # showed this prompt; the scan left NaN-stamped files in review. + for name, scan in (("plain.png", None), ("stamped.png", "nan")): + with self.subTest(name=name): + g = self._hires_fix_graph("cat on a sofa") + if scan: + status = self._nan_scanned(name, g) + else: + status = self._scanned(name, g) + self.assertIn(status, ("added", "linked")) + self.assertEqual(self._texts(), ["cat on a sofa"]) + + def test_untraced_single_sampler_links_the_legacy_prompt_with_nan(self): + g = self._hires_fix_graph("cat on a sofa") + for node_id in ("7", "8", "9", "14", "15"): + del g[node_id] + g["16"]["inputs"]["images"] = ["6", 0] + self.assertEqual(self._nan_scanned("single.png", g), "added") + self.assertEqual(self._texts(), ["cat on a sofa"]) + + def test_nan_inside_prompt_text_is_kept(self): + # Only the NaN number reads as null; legacy's regex also rewrote text. + g = self._hires_fix_graph("NaN cat, value: NaN") + self.assertEqual(self._nan_scanned("text.png", g), "added") + self.assertEqual(self._texts(), ["NaN cat, value: NaN"]) + + def test_infinity_is_still_unreadable_like_the_legacy_pages(self): + text = json.dumps(self._hires_fix_graph("cat")).replace( + '"seed": 7', '"seed": Infinity' + ) + self.assertIn(self._scanned_text("inf.png", text), ("ambiguous", "unresolved")) + self.assertEqual(self._texts(), []) + def test_raw_png_workflow_is_stored_as_object_and_repeat_is_unchanged(self): path = self.output_dir / "workflow.png" workflow = {"nodes": [{"id": 1, "type": "PromptManagerPair"}]} diff --git a/tests/test_system_api.py b/tests/test_system_api.py new file mode 100644 index 00000000..0a902085 --- /dev/null +++ b/tests/test_system_api.py @@ -0,0 +1,479 @@ +"""System routes (py/api/system.py): health check job, Overview and statistics. + +The health check runs off-loop as a ``read`` registry job of kind ``health`` +(one at a time, beside a scan), and the API keeps its last report. The +Overview reads only real values: Library-eligible counts, indexed images, the +last health result or null, no backup until one exists, and the effective +sources and whether they were auto-detected. The missing-thumbnail count is not +part of it: the Overview reads ``/images/thumbnails/status`` on its own (its +walk of every source must not hold the other values back). +""" + +import asyncio +import os +import sqlite3 +import sys +import threading +from pathlib import Path +from unittest.mock import MagicMock, patch + +_mock_server = MagicMock() +_mock_server.PromptServer.instance.routes = MagicMock() +sys.modules.setdefault("server", _mock_server) + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + +from py.api.jobs import HEALTH_JOB_KIND # noqa: E402 +from py.config import GalleryConfig # noqa: E402 +from utils.hashing import generate_prompt_hash # noqa: E402 + +try: + from tests.test_api_admin import AdminAPITestCase # noqa: E402 +except ImportError: # discovered with tests/ as the top-level directory + from test_api_admin import AdminAPITestCase # noqa: E402 + +WAIT = 5 + + +class SystemAPITestCase(AdminAPITestCase): + async def get_application(self): + app = await super().get_application() + # Auto-detection searches the disk; here it finds the fake ComfyUI only. + self.api._find_comfyui_output_dir = lambda: ( + str(self.output_dir) if self.output_dir.is_dir() else None + ) + return app + + def save(self, text, **metadata): + return self.api.db.save_prompt( + text, prompt_hash=generate_prompt_hash(text), **metadata + ) + + def image(self, prompt_id, name, *, exists=True, folder=None): + path = Path(folder or self.output_dir) / name + if exists: + path.write_bytes(b"png") + return self.api.db.link_image_to_prompt(prompt_id, str(path)) + + async def json(self, method, url, status=200): + response = await self.client.request(method, url) + self.assertEqual(response.status, status, await response.text()) + return await response.json() + + async def restore_snapshot(self): + """Restore the live database from a copy of itself through the legacy route.""" + from contextlib import closing + + from aiohttp import FormData + + snapshot = os.path.join(self.tmpdir, "snapshot.db") + with closing(sqlite3.connect(self.api.db.model.db_path)) as src: + with closing(sqlite3.connect(snapshot)) as dst: + src.backup(dst) + with open(snapshot, "rb") as handle: + upload = handle.read() + form = FormData() + form.add_field("database_file", upload, filename="upload.db") + response = await self.client.request( + "POST", "/prompt_manager/restore", data=form + ) + self.assertEqual(response.status, 200, await response.text()) + + async def run_health(self): + body = await self.json("POST", "/prompt_manager/system/health", 202) + job = self.api._jobs.get(body["job"]["id"]) + await asyncio.wait_for(asyncio.shield(job.task), WAIT) + return body["job"], job + + +class TestHealthRoute(SystemAPITestCase): + async def test_no_report_before_a_check(self): + body = await self.json("GET", "/prompt_manager/system/health") + self.assertEqual(body, {"success": True, "report": None}) + + async def test_health_check_runs_as_a_read_job_and_keeps_its_report(self): + prompt = self.save("Images") + self.image(prompt, "kept.png") + self.image(prompt, "gone.png", exists=False) + summary, job = await self.run_health() + self.assertEqual(summary["kind"], HEALTH_JOB_KIND) + self.assertEqual(summary["title"], "Health check") + self.assertEqual(summary["admission"], "read") + self.assertFalse(summary["cancellable"]) + self.assertEqual(summary["counters"], {}) + self.assertEqual(job.state, "finished") + self.assertEqual(job.progress, 100) + self.assertEqual(job.status_line, "Health check incomplete") + report = (await self.json("GET", "/prompt_manager/system/health"))["report"] + self.assertEqual(report["status"], "incomplete") + missing = next(c for c in report["checks"] if c["id"] == "missing_files") + self.assertEqual((missing["status"], missing["count"]), ("issues", 1)) + ambiguous = next(c for c in report["checks"] if c["id"] == "ambiguous_metadata") + self.assertEqual(ambiguous["status"], "unavailable") + self.assertNotIn(self.tmpdir, str(report)) + + async def test_health_check_is_single_flight_and_runs_beside_a_scan(self): + release = threading.Event() + entered = threading.Event() + real = self.api.db.run_health_checks + + def gated(*args, **kwargs): + entered.set() + release.wait(WAIT) + return real(*args, **kwargs) + + scan_gate = asyncio.Event() + + async def scanning(job): + await scan_gate.wait() + + scan = self.api._jobs.start( + "scan", "Metadata scan", admission="records", runner=scanning + ) + with patch.object(self.api.db, "run_health_checks", side_effect=gated): + first = await self.json("POST", "/prompt_manager/system/health", 202) + loop = asyncio.get_running_loop() + await loop.run_in_executor(None, entered.wait, WAIT) + busy = await self.json("POST", "/prompt_manager/system/health", 409) + self.assertEqual(busy["code"], "busy") + self.assertEqual( + busy["running"], {"id": first["job"]["id"], "title": "Health check"} + ) + release.set() + await asyncio.wait_for(self.api._jobs.get(first["job"]["id"]).task, WAIT) + scan_gate.set() + await asyncio.wait_for(scan.task, WAIT) + + async def test_failed_check_keeps_the_previous_report(self): + await self.run_health() + before = (await self.json("GET", "/prompt_manager/system/health"))["report"] + with ( + patch.object( + self.api.db, "run_health_checks", side_effect=RuntimeError("boom") + ), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("POST", "/prompt_manager/system/health", 202) + job = self.api._jobs.get(body["job"]["id"]) + await asyncio.wait_for(asyncio.shield(job.task), WAIT) + self.assertEqual(job.state, "failed") + after = (await self.json("GET", "/prompt_manager/system/health"))["report"] + self.assertEqual(after, before) + + async def test_absent_configured_source_is_reported_per_source(self): + prompt = self.save("Images") + self.image(prompt, "gone.png", exists=False) + absent = self.comfy_dir / "output" / "unmounted" + GalleryConfig.MONITORING_DIRECTORIES = [ + self._root(self.output_dir), + str(absent), + ] + await self.run_health() + report = (await self.json("GET", "/prompt_manager/system/health"))["report"] + missing = next(c for c in report["checks"] if c["id"] == "missing_files") + # Per source: the readable output is counted, the absent one is not. + self.assertEqual(missing["status"], "issues") + self.assertEqual(missing["count"], 1) + self.assertTrue(missing["partial"]) + self.assertEqual( + [(s["status"], s["count"]) for s in missing["sources"]], + [("checked", 1), ("unavailable", None)], + ) + self.assertEqual(report["status"], "incomplete") + + async def test_restore_clears_health_report(self): + self.save("Any") + await self.run_health() + self.assertIsNotNone( + (await self.json("GET", "/prompt_manager/system/health"))["report"] + ) + await self.restore_snapshot() + body = await self.json("GET", "/prompt_manager/system/health") + self.assertEqual(body, {"success": True, "report": None}) + overview = await self.json("GET", "/prompt_manager/system/overview") + self.assertIsNone(overview["health"]) + + async def test_check_finishing_after_a_restore_is_not_kept(self): + # A check that started on the old database describes it, not the new one. + self.save("Any") + release = threading.Event() + entered = threading.Event() + real = self.api.db.run_health_checks + + def gated(*args, **kwargs): + entered.set() + release.wait(WAIT) + return real(*args, **kwargs) + + with patch.object(self.api.db, "run_health_checks", side_effect=gated): + body = await self.json("POST", "/prompt_manager/system/health", 202) + loop = asyncio.get_running_loop() + await loop.run_in_executor(None, entered.wait, WAIT) + await self.restore_snapshot() + release.set() + job = self.api._jobs.get(body["job"]["id"]) + await asyncio.wait_for(asyncio.shield(job.task), WAIT) + self.assertEqual(job.state, "finished") + body = await self.json("GET", "/prompt_manager/system/health") + self.assertEqual(body, {"success": True, "report": None}) + # A check started on the restored database is kept again. + await self.run_health() + self.assertIsNotNone( + (await self.json("GET", "/prompt_manager/system/health"))["report"] + ) + + async def test_health_sources_resolve_off_the_event_loop(self): + loop_thread = threading.get_ident() + threads = [] + real = self.api._health_roots + + def recording(): + threads.append(threading.get_ident()) + return real() + + with patch.object(self.api, "_health_roots", side_effect=recording): + await self.run_health() + self.assertEqual(len(threads), 1) + self.assertNotEqual(threads[0], loop_thread) + + async def test_health_start_failure_is_an_error_envelope(self): + with ( + patch.object(self.api, "_health_roots", side_effect=RuntimeError("secret")), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("POST", "/prompt_manager/system/health", 500) + self.assertEqual( + body, + { + "success": False, + "code": None, + "error": "Failed to start the health check", + }, + ) + self.assertEqual(self.api._jobs.list(), []) + + +class TestOverviewRoute(SystemAPITestCase): + async def test_overview_reports_real_counts(self): + self.api.db.save_prompt_pair("Cat", "Blur") + archived = self.api.db.save_prompt_pair("Cat", "Noise") + self.api.db.archive_prompt_pair(archived) + legacy = self.save("Legacy") + self.image(legacy, "a.png") + self.image(legacy, "b.png") + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertTrue(body["success"]) + self.assertEqual( + body["library"], {"positive_prompts": 2, "negative_variants": 1} + ) + self.assertEqual(body["images"], {"indexed": 2, "rated": 0}) + self.assertEqual( + set(body), + { + "success", + "library", + "images", + "health", + "backup", + "needs_review", + "sources", + }, + ) + + async def test_overview_rated_counts_images_of_rated_prompts(self): + rated = self.save("Rated", rating=5) + self.image(rated, "a.png") + self.image(rated, "b.png") + self.image(self.save("Unrated"), "c.png") + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual(body["images"], {"indexed": 3, "rated": 2}) + + async def test_overview_never_fabricates_backup_or_health(self): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertIsNone(body["health"]) + self.assertIsNone(body["backup"]) + self.assertEqual(body["needs_review"], {"pending": None}) + self.save("Any") + await self.run_health() + report = (await self.json("GET", "/prompt_manager/system/health"))["report"] + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["health"], + {"checked_at": report["checked_at"], "status": "incomplete", "issues": 0}, + ) + self.assertIsNone(body["backup"]) + + async def test_overview_health_counts_checks_with_issues(self): + prompt = self.save("Images") + self.image(prompt, "gone.png", exists=False) + self.image(prompt, "gone-too.png", exists=False) + await self.run_health() + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual(body["health"]["issues"], 1) + + async def test_overview_sources_distinguish_auto_detect(self): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["sources"], {"configured": 0, "effective": 1, "auto_detected": True} + ) + second = self.output_dir / "archive" + second.mkdir() + GalleryConfig.MONITORING_DIRECTORIES = [ + self._root(self.output_dir), + self._root(second), + ] + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["sources"], {"configured": 2, "effective": 2, "auto_detected": False} + ) + # Every configured source unavailable: ComfyUI's output is used instead. + GalleryConfig.MONITORING_DIRECTORIES = [str(self.comfy_dir / "absent")] + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual( + body["sources"], {"configured": 1, "effective": 1, "auto_detected": True} + ) + + async def test_overview_never_walks_the_sources_for_thumbnails(self): + (self.output_dir / "one.png").write_bytes(b"png") + with patch.object( + self.api, + "_plan_gallery_thumbnails", + side_effect=AssertionError("walked"), + ): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertTrue(body["success"]) + self.assertNotIn("thumbnails", body) + + async def test_overview_sources_resolve_off_the_event_loop(self): + loop_thread = threading.get_ident() + threads = [] + real = self.api._output_dirs_with_origin + + def recording(): + threads.append(threading.get_ident()) + return real() + + with patch.object(self.api, "_output_dirs_with_origin", side_effect=recording): + body = await self.json("GET", "/prompt_manager/system/overview") + self.assertEqual(body["sources"]["effective"], 1) + self.assertEqual(len(threads), 1) + self.assertNotEqual(threads[0], loop_thread) + + async def test_overview_failure_is_an_error_envelope(self): + with ( + patch.object( + self.api.db, "library_counts", side_effect=RuntimeError("secret") + ), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("GET", "/prompt_manager/system/overview", 500) + self.assertEqual( + body, + {"success": False, "code": None, "error": "Failed to read the overview"}, + ) + + +class TestStatisticsRoute(SystemAPITestCase): + async def test_statistics_sum_real_run_counts(self): + a = self.save("Alpha", category="Landscapes", rating=5) + b = self.save("Beta", category="Portraits", rating=2) + self.api.db.record_prompt_use(a, times=3) + self.api.db.record_prompt_use(b) + body = await self.json("GET", "/prompt_manager/system/statistics") + self.assertEqual( + body["by_category"], + [ + {"category": "Landscapes", "runs": 3, "average_rating": 5.0}, + {"category": "Portraits", "runs": 1, "average_rating": 2.0}, + ], + ) + self.assertEqual(body["totals"], self.api.db.get_statistics()) + self.assertTrue(body["success"]) + + async def test_statistics_lists_rated_images_for_library_totals(self): + # D96: the Statistics dialog's "Library totals" reads the rated count + # with the runs (one read), from the same images the Overview counts. + rated = self.save("Rated", rating=4) + self.image(rated, "a.png") + self.image(self.save("Unrated"), "b.png") + body = await self.json("GET", "/prompt_manager/system/statistics") + self.assertEqual(body["rated_images"], 1) + self.assertEqual( + set(body), {"success", "by_category", "totals", "rated_images"} + ) + + async def test_statistics_failure_is_an_error_envelope(self): + with ( + patch.object( + self.api.db, "usage_by_category", side_effect=RuntimeError("secret") + ), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + body = await self.json("GET", "/prompt_manager/system/statistics", 500) + self.assertEqual( + body, + {"success": False, "code": None, "error": "Failed to read statistics"}, + ) + + +class TestFailedReadsSettleFirst(SystemAPITestCase): + """A failed read answers only after the route's other reads finished. + + Answering on the first failure (``asyncio.gather`` without + ``return_exceptions``) left the other executor reads running against a + database the caller may already be closing; on Python 3.10 CI that + crashed the test process inside SQLite. + """ + + async def assert_answers_after_reads(self, url, failing, slow, error): + raised, release, finished = (threading.Event() for _ in range(3)) + real = getattr(*slow) + + def fail(*args, **kwargs): + raised.set() + raise RuntimeError("secret") + + def held(*args, **kwargs): + release.wait() + try: + return real(*args, **kwargs) + finally: + finished.set() + + with ( + patch.object(*failing, side_effect=fail), + patch.object(*slow, side_effect=held), + self.assertLogs("prompt_manager.api", "ERROR"), + ): + answer = asyncio.ensure_future(self.json("GET", url, 500)) + try: + await asyncio.get_running_loop().run_in_executor(None, raised.wait) + done, _ = await asyncio.wait({answer}, timeout=0.5) + self.assertEqual(done, set(), "answered while a read still ran") + finally: + release.set() + body = await answer + self.assertTrue(finished.is_set()) + self.assertEqual(body, {"success": False, "code": None, "error": error}) + + async def test_statistics(self): + await self.assert_answers_after_reads( + "/prompt_manager/system/statistics", + (self.api.db, "usage_by_category"), + (self.api.db, "get_statistics"), + "Failed to read statistics", + ) + + async def test_overview(self): + await self.assert_answers_after_reads( + "/prompt_manager/system/overview", + (self.api.db, "library_counts"), + (self.api, "_output_dirs_with_origin"), + "Failed to read the overview", + ) + + +if __name__ == "__main__": + import unittest + + unittest.main() diff --git a/tests/test_workspace_system_fixture.py b/tests/test_workspace_system_fixture.py new file mode 100644 index 00000000..a43ec854 --- /dev/null +++ b/tests/test_workspace_system_fixture.py @@ -0,0 +1,38 @@ +"""The System browser hooks only ever reset their own server's config.json (#249 S3).""" + +import os +import tempfile +import unittest +from pathlib import Path +from unittest import mock + +from tests.browser.workspace_system_fixture import register_system_commands + + +class TestWorkspaceSystemFixture(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.root = Path(self.tmp.name) / "server" + self.root.mkdir() + + def test_refuses_a_config_path_outside_its_root(self): + elsewhere = Path(self.tmp.name) / "config.json" + elsewhere.write_bytes(b'{"real": true}') + for value in (str(elsewhere), None): + env = { + k: v for k, v in os.environ.items() if k != "PROMPT_MANAGER_CONFIG_PATH" + } + if value is not None: + env["PROMPT_MANAGER_CONFIG_PATH"] = value + with ( + self.subTest(value=value), + mock.patch.dict(os.environ, env, clear=True), + ): + with self.assertRaisesRegex(RuntimeError, "PROMPT_MANAGER_CONFIG_PATH"): + register_system_commands(None, None, None, self.root) + self.assertEqual(elsewhere.read_bytes(), b'{"real": true}') + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/vendor/frontend-coverage-baseline.json b/tools/vendor/frontend-coverage-baseline.json index 7d829414..177df6b1 100644 --- a/tools/vendor/frontend-coverage-baseline.json +++ b/tools/vendor/frontend-coverage-baseline.json @@ -37,15 +37,15 @@ }, "gallery-settings.js": { "lines": 98, - "branches": 90, + "branches": 94, "functions": 98, "statements": 98 }, "gallery.js": { - "lines": 40, + "lines": 41, "branches": 63, "functions": 44, - "statements": 40 + "statements": 41 }, "html-escape.js": { "lines": 98, @@ -401,6 +401,12 @@ "functions": 98, "statements": 98 }, + "workspace/preferences.mjs": { + "lines": 98, + "branches": 96, + "functions": 98, + "statements": 98 + }, "workspace/prompt-blocks.mjs": { "lines": 90, "branches": 92, @@ -443,6 +449,36 @@ "functions": 98, "statements": 98 }, + "workspace/system-appearance.mjs": { + "lines": 98, + "branches": 93, + "functions": 98, + "statements": 98 + }, + "workspace/system-controls.mjs": { + "lines": 98, + "branches": 87, + "functions": 89, + "statements": 98 + }, + "workspace/system-overview.mjs": { + "lines": 96, + "branches": 92, + "functions": 98, + "statements": 96 + }, + "workspace/system-state.mjs": { + "lines": 98, + "branches": 94, + "functions": 98, + "statements": 98 + }, + "workspace/system.mjs": { + "lines": 98, + "branches": 92, + "functions": 85, + "statements": 98 + }, "workspace/theme.mjs": { "lines": 98, "branches": 93, diff --git a/utils/generation_params.py b/utils/generation_params.py index e65ecd1e..42b7c9b3 100644 --- a/utils/generation_params.py +++ b/utils/generation_params.py @@ -28,6 +28,7 @@ __all__ = [ "extract_generation_params", "extract_from_records", "parse_js_json", + "parse_page_json", ] PLACEHOLDERS = MappingProxyType( @@ -953,6 +954,30 @@ def parse_js_json(text): return None +def _nan_as_null(name): + """The legacy pages' NaN cleaning; Infinity still throws in ``JSON.parse``.""" + if name == "NaN": + return None + raise ValueError(f"{name} is not JSON") + + +def parse_page_json(text): + """A metadata record as the legacy gallery and admin pages parsed it. + + They replace ``NaN`` with ``null`` and then ``JSON.parse`` (ComfyUI writes + ``"is_changed": [NaN]`` for nodes that always re-run, PromptManager among + them). Only the NaN number reads as null here: the pages' regex also + rewrote the word inside strings, which changed prompt text. None where the + parse throws. + """ + if not isinstance(text, str): + return None + try: + return json.loads(text, parse_constant=_nan_as_null, parse_int=_double_integer) + except (ValueError, RecursionError): + return None + + def extract_from_records(prompt_text, workflow_text): """``extract_generation_params`` for raw ``prompt``/``workflow`` record text. diff --git a/utils/json_strict.py b/utils/json_strict.py new file mode 100644 index 00000000..7eeb8aed --- /dev/null +++ b/utils/json_strict.py @@ -0,0 +1,42 @@ +"""Strict JSON output for values that may carry ComfyUI's non-finite numbers. + +ComfyUI writes its metadata with Python's default ``json.dumps``, so a graph can +hold ``NaN``, ``Infinity`` and ``-Infinity`` (an ``IS_CHANGED`` that returns +``float("nan")`` puts ``"is_changed": [NaN]`` into every saved prompt). Those +literals are not JSON: browsers' ``JSON.parse`` and ``allow_nan=False`` reject +them. Reading follows the legacy pages (``parse_page_json`` where the browser +parser is mirrored; Python's ``json.loads`` elsewhere). Whatever PromptManager +stores or answers as JSON goes through ``dumps_strict``, which writes each +non-finite number as ``null`` (the legacy pages' own replacement for NaN) and +leaves strings, including the words ``NaN`` or ``Infinity`` inside prompt +text, untouched. +""" + +import json +from typing import Any + +__all__ = ["dumps_strict"] + + +def _null(_constant: str) -> None: + return None + + +def dumps_strict(value: Any, **kwargs: Any) -> str: + """``json.dumps(value, allow_nan=False, **kwargs)`` with non-finite numbers null. + + A value without non-finite numbers serializes byte for byte as the plain + call does. Any other error (a cycle, an unserializable object) still raises. + """ + kwargs.pop("allow_nan", None) + try: + return json.dumps(value, allow_nan=False, **kwargs) + except ValueError as error: + if "Out of range float" not in str(error): + raise + # The non-strict text spells each non-finite number as a bare literal, never + # inside a string, so parsing it back maps exactly those numbers to null. + lenient = json.dumps(value, allow_nan=True, default=kwargs.get("default")) + return json.dumps( + json.loads(lenient, parse_constant=_null), allow_nan=False, **kwargs + ) diff --git a/web/js/gallery-settings.js b/web/js/gallery-settings.js index 4c21dd62..3ea591d0 100644 --- a/web/js/gallery-settings.js +++ b/web/js/gallery-settings.js @@ -4,6 +4,9 @@ * * Settings are per browser (localStorage key "gallerySettings"). Anything * stored can be stale or hand-edited, so every value is checked before use. + * The workspace shares this object ("One display configuration", #249 System + * S3): it adds workspaceLayout and workspaceDensity, and every key either page + * does not know is kept. * * Loadable in the browser (window.GallerySettings) and in Node for tests. */ @@ -11,6 +14,9 @@ "use strict"; const VIEW_MODES = ["grid", "list"]; + // The workspace Images defaults; the legacy gallery's defaultViewMode has no masonry. + const WORKSPACE_LAYOUTS = ["grid", "list", "masonry"]; + const WORKSPACE_DENSITIES = ["comfortable", "large", "compact"]; const LIMIT_OPTIONS = [50, 100, 200, 500]; const MIN_COLUMNS = 2; const MAX_COLUMNS = 12; @@ -34,6 +40,8 @@ videoAutoplay: false, videoMute: true, videoLoop: true, + workspaceLayout: "grid", + workspaceDensity: "comfortable", }); function toInt(value) { @@ -72,6 +80,8 @@ videoAutoplay: bool(merged.videoAutoplay, DEFAULT_SETTINGS.videoAutoplay), videoMute: bool(merged.videoMute, DEFAULT_SETTINGS.videoMute), videoLoop: bool(merged.videoLoop, DEFAULT_SETTINGS.videoLoop), + workspaceLayout: WORKSPACE_LAYOUTS.includes(merged.workspaceLayout) ? merged.workspaceLayout : DEFAULT_SETTINGS.workspaceLayout, + workspaceDensity: WORKSPACE_DENSITIES.includes(merged.workspaceDensity) ? merged.workspaceDensity : DEFAULT_SETTINGS.workspaceDensity, }; } @@ -81,7 +91,7 @@ return `repeat(${n}, minmax(0, 1fr))`; } - const api = { DEFAULT_SETTINGS, LIMIT_OPTIONS, VIEW_MODES, normalizeGallerySettings, gridColumnsStyle }; + const api = { DEFAULT_SETTINGS, LIMIT_OPTIONS, VIEW_MODES, WORKSPACE_LAYOUTS, WORKSPACE_DENSITIES, normalizeGallerySettings, gridColumnsStyle }; if (typeof module !== "undefined" && module.exports) module.exports = api; else root.GallerySettings = api; })(typeof window !== "undefined" ? window : globalThis); diff --git a/web/js/gallery.js b/web/js/gallery.js index ee9f24bd..cad3c2ca 100644 --- a/web/js/gallery.js +++ b/web/js/gallery.js @@ -1768,7 +1768,10 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`; } saveSettings() { + // The workspace shares this object (its layout and density, #249 System S3), so a Save starts from what is + // stored and keeps every key this dialog does not edit. const settings = { + ...this.getSettings(), lazyLoading: document.getElementById('lazyLoadingToggle').checked, imageQuality: document.getElementById('imageQualitySelect').value, defaultViewMode: document.getElementById('defaultViewModeSelect').value, @@ -1782,8 +1785,6 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`; infiniteScroll: document.getElementById('infiniteScrollToggle').checked, debugMode: document.getElementById('debugModeToggle').checked, apiTimeout: parseInt(document.getElementById('apiTimeoutInput').value), - thumbnailsGenerated: this.getSettings().thumbnailsGenerated, // Preserve this - thumbnailPromptDismissed: this.getSettings().thumbnailPromptDismissed, checkThumbnailsAtStartup: document.getElementById('checkThumbnailsAtStartup').checked, // Video settings videoAutoplay: document.getElementById('videoAutoplayToggle').checked, diff --git a/web/js/workspace/activity-store.mjs b/web/js/workspace/activity-store.mjs index 72eb6aba..0729d449 100644 --- a/web/js/workspace/activity-store.mjs +++ b/web/js/workspace/activity-store.mjs @@ -336,6 +336,16 @@ export function createActivityStore({ api, scope, connect = defaultConnect, noti return { busy: store.running()[0] ?? null, message: error.message }; } }, + /** + * Follow a job another route started (its summary from that route's answer, e.g. the health check's + * `POST /system/health`): it is listed and its stream opened at once, with no read and no start toast. + */ + track(raw) { + const job = parseJob(raw); + if (!job) throw new TypeError('Invalid job'); + upsert(job); + return job; + }, /** Ask a running job to stop at its next file boundary; false once it has ended. */ async cancel(id) { const result = await api.post(`${JOBS_URL}/${encodeURIComponent(id)}/cancel`, {}, { signal: scope.signal }); diff --git a/web/js/workspace/activity.mjs b/web/js/workspace/activity.mjs index eb7bc14b..f4deedee 100644 --- a/web/js/workspace/activity.mjs +++ b/web/js/workspace/activity.mjs @@ -148,18 +148,20 @@ function itemFor(job, scope) { chip.textContent = state; detail.textContent = jobDetail(next); const running = next.state === 'running'; + // parentNode, not isConnected: on the first render the item is not in the page yet, so a job that had already + // ended when Activity opened kept its Cancel job (and a running one's progress moved after its actions). if (running) { progress.value = next.progress; progress.textContent = `${next.progress}%`; progress.setAttribute('aria-label', `${next.title} progress`); - if (!progress.isConnected) item.insertBefore(progress, actions.isConnected ? actions : null); + if (!progress.parentNode) item.insertBefore(progress, actions.parentNode ? actions : null); } else { progress.remove(); } // Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel job. if (running && next.cancellable) { - if (!actions.isConnected) item.append(actions); - } else if (actions.isConnected) { + if (!actions.parentNode) item.append(actions); + } else if (actions.parentNode) { // A focused Cancel job that goes away hands its focus to the item's title link, never to the page body. if (actions.contains(document.activeElement)) link.focus(); actions.remove(); diff --git a/web/js/workspace/file-inspector.mjs b/web/js/workspace/file-inspector.mjs index 2d7a032d..80465acd 100644 --- a/web/js/workspace/file-inspector.mjs +++ b/web/js/workspace/file-inspector.mjs @@ -6,6 +6,7 @@ import { createWorkspaceIcon } from './icons.mjs'; import { navigateTo } from './state.mjs'; import { showWorkspaceToast } from './toast.mjs'; import { openCopyFallback } from './workspace-dialogs.mjs'; +import { displayPreferences } from './preferences.mjs'; const RAW_KEYS = ['prompt', 'workflow', 'parameters', 'comfyassets_prompt_pairs']; const RAW_LIMIT = 'Raw metadata exceeds the response limit. Download the original file to inspect it externally.'; @@ -196,7 +197,10 @@ function prefixedChip(prefix, text) { * `variant: 'full'` (the viewer's `#viewer-details`, imageDetails(i, true)) leaves out the cover and adds "File & * provenance" and "Workflow & raw metadata" with the downloads. `inline: true` means `panel`; neither means `full`. */ -export function mountFileInspector(container, {api, sender, parentScope, variant, inline = false, promptContext = '', onTagFilter, onRetry, onRefresh, onInteraction = () => {}, copyFallback = openCopyFallback}) { +/** D124: the inspector's line while "Load metadata on selection" is off. */ +export const DEFERRED_TEXT = 'Metadata loads when you ask for it.'; + +export function mountFileInspector(container, {api, sender, parentScope, variant, inline = false, promptContext = '', onTagFilter, onRetry, onRefresh, onLoad, onInteraction = () => {}, copyFallback = openCopyFallback}) { if (parentScope.signal.aborted) return {setState() {}, dispose() {}}; const scope = childScope(parentScope); let contentScope; @@ -235,6 +239,13 @@ export function mountFileInspector(container, {api, sender, parentScope, variant function render(state) { const inner = el('div', undefined, full ? undefined : 'detail-inner'); if (state.loading || state.error) {container.append(inner); statusView(state, inner); return;} + // D124: "Load metadata on selection" is off, so the file's details wait for an explicit request. + if (state.deferred) { + container.append(inner); + const status = el('p', DEFERRED_TEXT, 'small'); status.setAttribute('role', 'status'); inner.append(status); + if (onLoad) {const load = createButton('Load metadata'); contentScope.listen(load, 'click', onLoad); inner.append(load);} + return; + } const record = state.record; if (!record) return; const {file, library} = record; @@ -326,8 +337,10 @@ export function mountFileInspector(container, {api, sender, parentScope, variant section.append(el('p', reported ? 'Current file at the reported location · identity unverified' : local ? 'Selected local file · read in this browser' : 'Current file at this source; historical bytes are not verified.', 'small')); const path = el('dd', record.identity.path ?? 'Not available'); + // System › Appearance "Show file paths" off: the Path row is left out; nothing else changes. + const showPath = !local && displayPreferences()?.showFilePaths !== false; const facts = file && definitionList([ - ['Source', file.source_label], ...(local ? [] : [['Path', path]]), ['Size', formatBytes(file.size)], + ['Source', file.source_label], ...(showPath ? [['Path', path]] : []), ['Size', formatBytes(file.size)], ['Dimensions', file.width && file.height ? `${file.width} × ${file.height}` : null], ['Format', file.format || `${extensionOf(record)} extension`], ['Media type', file.media_type], ['Modified', file.modified_time !== null ? new Date(file.modified_time * 1000).toLocaleString() : null], @@ -335,8 +348,9 @@ export function mountFileInspector(container, {api, sender, parentScope, variant if (facts) { // D38: "Show in folder" follows the Path value. const target = api ? revealTarget(record.identity) : null; - if (target) path.append(' ', revealButton(target)); section.append(facts); + if (target && showPath) path.append(' ', revealButton(target)); + else if (target) section.append(revealButton(target)); } else section.append(el('p', 'Current file observations are unavailable for this inspection response.', 'small')); // The concept's "Prompt evidence: execution recorded" line, in the D21 badge's words; no badge, no line. if (badge) section.append(el('p', `Prompt evidence: ${badge.toLowerCase()}`, 'small')); diff --git a/web/js/workspace/images-state.mjs b/web/js/workspace/images-state.mjs index d133eba0..c61daf46 100644 --- a/web/js/workspace/images-state.mjs +++ b/web/js/workspace/images-state.mjs @@ -1,6 +1,17 @@ import { noun } from './plural.mjs'; /** Identity is the source and relative path; content hashes may be shared by files. */ +/** + * The layout and thumbnail size an Images visit starts with (#249 System S3). The last visit (the current view) keeps + * its own; a saved view starts as before, at Grid and Comfortable; any other new visit starts from System › + * Appearance's defaults (`preferences`, preferences.mjs values, or null when unavailable). + */ +export function initialImagesView({ prior = null, saved = null, preferences = null } = {}) { + if (prior) return { layout: prior.layout || 'grid', density: prior.density || 'comfortable' }; + if (saved || !preferences) return { layout: 'grid', density: 'comfortable' }; + return { layout: preferences.workspaceLayout || 'grid', density: preferences.workspaceDensity || 'comfortable' }; +} + export function imageKey(image) { return `${image.root_index}:${image.path}`; } diff --git a/web/js/workspace/images.mjs b/web/js/workspace/images.mjs index 1ac497a6..b96dab12 100644 --- a/web/js/workspace/images.mjs +++ b/web/js/workspace/images.mjs @@ -1,5 +1,5 @@ import { createScope, createButton } from '../shared/toolkit/core.js'; -import { createImageStream, withItem, imageKey, countLine, captionDate, scopeLabel, activeRefinements, cardTitle, cardSubtitle, mediaRatio } from './images-state.mjs'; +import { createImageStream, withItem, imageKey, countLine, captionDate, scopeLabel, activeRefinements, cardTitle, cardSubtitle, mediaRatio, initialImagesView } from './images-state.mjs'; import { layoutImages, visibleImages, gridMetrics, stacksCardInfo } from './image-layout.mjs'; import { mountFileInspector } from './file-inspector.mjs'; import { createInspectionController } from './inspection-state.mjs'; @@ -15,6 +15,7 @@ import { openImportDialog, openRefineDialog } from './workspace-dialogs.mjs'; import { openMetadataScan } from './activity-scan.mjs'; import { showWorkspaceToast } from './toast.mjs'; import { imagesFilters, imagesViewState } from './saved-views.mjs'; +import { displayPreferences } from './preferences.mjs'; import { createTagButton, onImageTagsSaved } from './image-tagging.mjs'; import { followImageTagSaves, openCardTagging } from './images-tagging.mjs'; import { favoriteBusy, paintStar, receiveFavorites, setFavoriteVisitTarget, toggleFavorite } from './images-favorites.mjs'; @@ -111,7 +112,9 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage const { resolveUrl } = window.ApiClient; const sender = createPromptSender({scope}); let detailInspector; - const inspection = createInspectionController({api}); + // System › Appearance: the display defaults this visit starts from (preferences.mjs); null without gallery-settings.js. + const preferences = displayPreferences(); + const inspection = createInspectionController({api, autoLoad:preferences?.autoLoadMetadata !== false, cache:preferences?.cacheMetadata !== false}); scope.cleanup(inspection.subscribe(next => detailInspector?.setState(next))); scope.cleanup(() => inspection.dispose()); scope.cleanup(() => detailInspector?.dispose()); @@ -119,9 +122,12 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage const prior = embedded || saved || favorites ? null : lastVisit; // A stale visit keeps its filters, layout and size, and lists again instead of restoring (M6). const visit = prior?.stale ? null : prior; - let density = prior?.density || 'comfortable'; + // The layout and size defaults are the Images page's: an embedded gallery (a prompt history's, its dialogs) has no + // toolbar to change them, so it keeps its fixed Grid/Comfortable. The metadata and card defaults apply everywhere. + const start = initialImagesView({prior, saved, preferences:embedded ? null : preferences}); + let density = start.density; let metrics = {...INITIAL_METRICS}; - let layout = prior?.layout || 'grid', selection = visit?.selection || null, frame = 0, loadFrame = 0, restoreFrame = 0, observer; + let layout = start.layout, selection = visit?.selection || null, frame = 0, loadFrame = 0, restoreFrame = 0, observer; let state, geometry = { positions: [], columns: 1, height: 0, maxHeight: 0 }, galleryWidth = 0, optionsRequest, categoryRequest, recovering = false, rescanning = false; // Discovered sources (each with its own folders) and every folder, for "Refine this view" and the scope chip. let roots = [], folders = []; @@ -287,7 +293,7 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage selection = imageKey(image); for (const video of details.querySelectorAll('video')) releaseVideo(video); detailInspector = mountFileInspector(details, {sender, parentScope:scope, variant:'panel', - onTagFilter:embedded ? undefined : filterByTag, onRetry:() => inspection.retry(), onRefresh:refreshInspectionCollection}); + onTagFilter:embedded ? undefined : filterByTag, onRetry:() => inspection.retry(), onLoad:() => inspection.load(), onRefresh:refreshInspectionCollection}); detailInspector.setState(inspection.state); inspection.select(image); for (const [key, card] of rendered) markSelected(card, key === selection); @@ -331,6 +337,8 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage const text = el('div'); const title = el('h3', cardTitle(image)); title.title = image.path; text.append(title, el('p', cardSubtitle(image))); + // D97: "Show image details on cards" off hides the title and file line; Info and Tag stay. + if (preferences?.showImageInfo === false) text.hidden = true; const actions = el('div', undefined, 'image-card-actions'); const inspectButton = createButton('', 'icon-btn'); inspectButton.append(createWorkspaceIcon('info')); diff --git a/web/js/workspace/index.mjs b/web/js/workspace/index.mjs index 70adf1a4..138a6ef7 100644 --- a/web/js/workspace/index.mjs +++ b/web/js/workspace/index.mjs @@ -1,6 +1,6 @@ import '../api-client.js'; import { createButton, createScope } from '../shared/toolkit/core.js'; -import { routes, mountRouter, summarizeStats, routeTitle, isHandoff, loadNavCounts, +import { routes, mountRouter, summarizeStats, routeTitle, loadNavCounts, LIBRARY_WRITE_COUNTS, NAV_COUNT_ROUTES, FAVORITE_WRITE_COUNTS, FAVORITE_COUNT_DELAY_MS } from './state.mjs'; import { mountWorkspaceTheme } from './theme.mjs'; import { mountImages, onFavoriteChange } from './images.mjs'; @@ -16,6 +16,7 @@ import { mountActivityPage, mountActivityTopbar } from './activity.mjs'; import { mountActivityJobView } from './activity-job-view.mjs'; import { mountInspect } from './inspect.mjs'; import { createPromptSender } from './prompt-sender.mjs'; +import { mountSystem } from './system.mjs'; const { createApiClient, resolveUrl } = window.ApiClient; @@ -28,7 +29,8 @@ function element(tag, text, className) { function legacyLink(route) { const link = element('a', route.action, 'workspace-link'); - link.href = resolveUrl(route.href); + // A native card (System, Q12) links its workspace hash; a legacy tool's page goes through the mount prefix. + link.href = route.native ? route.href : resolveUrl(route.href); return link; } @@ -137,6 +139,9 @@ function mountWorkspace() { let leaveGuard = null, renderToken = 0; scope.cleanup(() => viewScope?.dispose()); + // The mounted System page: another of its sections is shown in place, keeping the shell and the drafts. + let system = null; + function render(route) { const token = ++renderToken; const hold = leaveGuard?.(); @@ -151,6 +156,11 @@ function mountWorkspace() { leaveGuard = null; // A shared page dialog never stays open over the next route. closeWorkspaceDialog(); + if (route.id === 'system' && system && currentKey === 'system') { + system.show(route.section); + return; + } + system = null; viewScope?.dispose(); images = null; viewScope = createScope(); @@ -251,21 +261,18 @@ function mountWorkspace() { else mountActivityPage(main, { store: activity, scope: view }); return; } + if (route.id === 'system') { + // The concept's System (app.js renderSystem): its heading description, section nav and Save row. + description.textContent = route.description; + system = mountSystem(main, api, view, { section: route.section, activity }); + return; + } if (route.id === 'inspect') { // The concept's Inspect a file (app.js renderMetadata): a local file, read only in this browser. description.textContent = route.description; ({ leaveGuard } = mountInspect(main, { parentScope: view, api, sender: createPromptSender({ scope: view }) })); return; } - if (isHandoff(route)) { - description.textContent = route.description; - const panel = element('section', undefined, 'surface handoff'); - panel.append(element('h2', 'Your current tools are ready'), - element('p', 'This workspace is being introduced in stages. Continue in the existing tool with your saved data.'), - legacyLink(route)); - main.append(panel); - return; - } description.textContent = 'A home for your prompts, images and creative history.'; const stats = element('section', undefined, 'overview-stats'); stats.setAttribute('aria-label', 'Library statistics'); diff --git a/web/js/workspace/inspection-state.mjs b/web/js/workspace/inspection-state.mjs index 290797b9..f9567ed1 100644 --- a/web/js/workspace/inspection-state.mjs +++ b/web/js/workspace/inspection-state.mjs @@ -344,9 +344,16 @@ export function createInspectionLoader(api, onChange, { extractFacts, adapter = }; } -const emptyState = () => ({ loading: false, error: false, record: null, prompts: undefined, message: '', refreshRequired: false, code: null, recovery: null }); +const emptyState = () => ({ loading: false, error: false, deferred: false, record: null, prompts: undefined, message: '', refreshRequired: false, code: null, recovery: null }); -export function createInspectionController({ api, extractFacts, adapter = rootedFileAdapter }) { +/** + * One selection's inspection. #249 System S3 (Appearance): `autoLoad: false` ("Load metadata on selection" off) makes a + * selection publish `deferred` instead of reading, and `load()` reads it once on request; `cache: false` ("Cache + * metadata in this browser" off) re-reads the selected file when it is selected again, where the default reuses the one + * current record. A read in flight is shared either way, and a file whose source changed is never re-read by selecting + * it again (as `retry()` and `refresh()` refuse). + */ +export function createInspectionController({ api, extractFacts, adapter = rootedFileAdapter, autoLoad = true, cache = true }) { let state = Object.freeze(emptyState()), current = null, pending = null, disposed = false; let attempted, hasAttempt = false; const listeners = new Set(); @@ -369,13 +376,19 @@ export function createInspectionController({ api, extractFacts, adapter = rooted try { adapter.validate(image); const scope = adapter.scope(image), key = adapter.key(image), path = adapter.inspectionPath(image); - if (current?.key === key && current.scope === scope && current.path === path) return pending; + if (current?.key === key && current.scope === scope && current.path === path && (cache || state.loading || state.deferred || state.refreshRequired)) return pending; selected = { key, scope, path }; } catch { // The loader publishes validation errors after invalidating the previous source. } loader.dispose(); loader = makeLoader(); current = selected; attempted = object(image) ? { ...image, ...(adapter !== rootedFileAdapter ? {occurrence:{...image.occurrence}, reference:{...image.reference}} : {}) } : image; hasAttempt = true; + if (!autoLoad && selected) { pending = null; publish({ deferred: true }); return undefined; } + return read(attempted); + }, + /** Read a deferred selection (the inspector's "Load metadata"); a read already made or running is reused. */ + load() { + if (disposed || !hasAttempt || !state.deferred) return pending ?? undefined; return read(attempted); }, retry() { @@ -384,7 +397,8 @@ export function createInspectionController({ api, extractFacts, adapter = rooted }, /** Re-read the selected file keeping the shown record until the answer (a retry when none is shown). */ refresh() { - if (disposed || !hasAttempt || state.refreshRequired) return; + // A deferred selection loads only through `load()` (the user's "Load metadata"). + if (disposed || !hasAttempt || state.refreshRequired || state.deferred) return; return read(attempted, { quiet: Boolean(state.record) && !state.loading && !state.error }); }, clear() { diff --git a/web/js/workspace/media-viewer.mjs b/web/js/workspace/media-viewer.mjs index 4427dd64..834c9957 100644 --- a/web/js/workspace/media-viewer.mjs +++ b/web/js/workspace/media-viewer.mjs @@ -2,14 +2,14 @@ import {createScope, createButton} from '../shared/toolkit/core.js'; import {createViewerSession} from './viewer-state.mjs'; import {rootedFileAdapter} from './inspection-state.mjs'; import {mountFileInspector} from './file-inspector.mjs'; +import {displayPreferences, readPreferences} from './preferences.mjs'; let viewerId = 0; // Upper bound on waiting for a closed Viewer's in-flight image view before destroying it. const PENDING_VIEW_DESTROY_MS = 10000; +/** The video defaults, read through the shared display defaults (preferences.mjs, #249 System S3). */ export function readViewerPreferences(getStorage = () => window.localStorage, normalize = globalThis.GallerySettings.normalizeGallerySettings) { - let stored; - try { stored = JSON.parse(getStorage().getItem('gallerySettings')); } catch { stored = null; } - const {videoAutoplay,videoMute,videoLoop} = normalize(stored); + const {videoAutoplay,videoMute,videoLoop} = readPreferences(getStorage,normalize).values; return {videoAutoplay,videoMute,videoLoop}; } function element(tag, className, text) { @@ -35,7 +35,7 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle if (scope.signal.aborted) return; active?.close(false); const {items,selectedKey,trigger = document.activeElement} = options; - let mounted = false, initial, entry, currentKey, viewer = null, mediaScope = null, closed = false; + let mounted = false, initial, entry, currentKey, viewer = null, withoutViewer = false, mediaScope = null, closed = false; let originalDownload = null, inspectionState = inspectionController.state; const session = createViewerSession({adapter,items,initialKey:selectedKey,onChange:(state,selected) => { if (!mounted) { initial = [state,selected]; return; } @@ -74,15 +74,31 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle local.listen(download,'click',event => {session.stop();if (download.getAttribute('aria-disabled') === 'true') event.preventDefault();}); controls.append(download); const next = button('Next media','Next →',() => session.navigate(1)); + // D97: "Collapse viewer details by default" opens the viewer with its details hidden behind one toggle. + const collapsed = displayPreferences()?.sidebarCollapsedByDefault === true; + let detailsToggle = null; + if (collapsed) { + detailsToggle = createButton('Show details');detailsToggle.setAttribute('aria-expanded','false'); + detailsToggle.setAttribute('aria-controls','viewer-details');controls.append(detailsToggle); + } const position = element('p','media-position muted');position.id = `media-viewer-position-${++viewerId}`; position.setAttribute('role','status'); const reason = element('p','media-reason muted');reason.id = `media-viewer-reason-${viewerId}`; const slideshowReason = element('p','media-slideshow-reason muted');slideshowReason.id = `media-viewer-slideshow-${viewerId}`; controls.append(position,reason,slideshowReason);left.append(stage,controls); const details = element('aside');details.id = 'viewer-details';details.setAttribute('aria-label','Image metadata'); + if (detailsToggle) { + details.hidden = true; + local.listen(detailsToggle,'click',() => { + details.hidden = !details.hidden; + detailsToggle.textContent = details.hidden ? 'Show details' : 'Hide details'; + detailsToggle.setAttribute('aria-expanded',String(!details.hidden)); + }); + } body.append(left,details);dialog.append(header,body);container.append(dialog); const localSender = {open(roles,trigger) {session.stop();sender.open(roles,trigger);}}; const inspector = mountFileInspector(details,{api,sender:localSender,parentScope:local,variant:'full',onInteraction:() => session.stop(),onRetry:() => inspectionController.retry(), + onLoad:() => inspectionController.load?.(), onRefresh:onRefresh ? () => { closeViewer();onRefresh(); } : undefined}); function closeViewer(restore = true) { if (closed) return; @@ -115,7 +131,9 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle mediaScope?.dispose();mediaScope = createScope();viewer = null; for (const child of [...stage.children]) if (child !== failure) child.remove(); failure.hidden = true; - const mediaLocal = mediaScope, key = session.state.key; + const mediaLocal = mediaScope, key = session.state.key, ViewerJs = window.Viewer; + // Transforms follow the Viewer.js this media was built with, which builds once the image has loaded. + withoutViewer = !ViewerJs; const current = () => !local.signal.aborted && !mediaLocal.signal.aborted && session.state.key === key; // An adapter's own media URL (a session-owned blob:) is used exactly, so a retry reuses it; a server // original gets a fresh cache-buster on every build. @@ -142,15 +160,23 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle },{once:true}); } else { node.alt = selected.filename;node.decoding = 'async'; - node.hidden = !!window.Viewer; + node.hidden = !!ViewerJs; // With Viewer.js the src is released after the viewer (see release below). - if (!window.Viewer) { + if (!ViewerJs) { mediaLocal.listen(node,'load',() => {if (current()) session.ready(key);}); mediaLocal.cleanup(() => node.removeAttribute('src')); } } node.src = src;stage.append(node); - if (!selected.is_video && window.Viewer) { + if (!selected.is_video && ViewerJs) { + // Viewer.js is built on an image that has loaded. Its own cancel removes its image listeners only + // while `image.complete` is false, so an image left with its load event still queued would build a + // viewer into the released image after destroy() (#293). Until then no viewer exists to cancel. + let mountedViewer = false; + mediaLocal.listen(node,'load',() => { if (current()) { mountedViewer = true;mountViewer(); } },{once:true}); + mediaLocal.cleanup(() => { if (!mountedViewer) node.removeAttribute('src'); }); + } + function mountViewer() { let fitFrame = 0, fitWidth, fitHeight, viewSettled = false, released = false, fallback = 0; const release = () => { if (released) return; @@ -166,7 +192,7 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle fitWidth = width;fitHeight = height;instance.resize(); }); }; - const instance = new window.Viewer(node,{inline:true,toolbar:false,navbar:false,title:false,button:false, + const instance = new ViewerJs(node,{inline:true,toolbar:false,navbar:false,title:false,button:false, keyboard:false,fullscreen:false,transition:false,viewed() { viewSettled = true; if (mediaLocal.signal.aborted) { release(); return; } @@ -176,10 +202,12 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle const observer = new ResizeObserver(scheduleFit);observer.observe(stage); viewer = instance;mediaLocal.cleanup(() => { observer.disconnect();cancelAnimationFrame(fitFrame); - // Inline Viewer.js keeps viewing a still-loading image after destroy(), and that view - // rebuilds its list from images that still have a src; keep both until the view - // settles, fails or times out. - if (viewSettled) { release(); return; } + // Before Viewer.js has built (its zero-delay build is pending), destroy() cancels that build; + // deferring it would let the build run into the image the stage has just dropped. + // Once built, inline Viewer.js keeps viewing a still-loading image after destroy(), and that + // view rebuilds its list from images that still have a src; keep both until the view settles, + // fails or times out. + if (viewSettled || !instance.ready) { release(); return; } node.addEventListener('error',release,{once:true}); fallback = setTimeout(release,PENDING_VIEW_DESTROY_MS); }); @@ -188,7 +216,7 @@ export function createMediaViewer({api,container,parentScope,inspectionControlle } function updateControls(state) { const unavailable = inspectionState.refreshRequired ? inspectionState.message : state.phase === 'error' ? 'Media could not be loaded. Retry this file.' : entry.is_video ? - 'Image transforms are unavailable for video.' : !viewer ? + 'Image transforms are unavailable for video.' : withoutViewer ? 'Image transforms are unavailable because Viewer.js could not load.' : !state.mediaReady ? 'Image transforms are available when the image is ready.' : ''; reason.textContent = unavailable;reason.hidden = !unavailable; for (const node of transforms) { diff --git a/web/js/workspace/preferences.mjs b/web/js/workspace/preferences.mjs new file mode 100644 index 00000000..c964d846 --- /dev/null +++ b/web/js/workspace/preferences.mjs @@ -0,0 +1,112 @@ +/** + * Browser display defaults (#249 System S3 Task 7). The workspace keeps them in the legacy gallery's own + * `gallerySettings` object (Q3, "One display configuration"), so the gallery, the workspace and the viewer read one + * configuration: the new `workspaceLayout` and `workspaceDensity` beside the legacy keys they share (metadata, paths, + * card details, viewer details, image quality and the video defaults). The legacy paging keys (`defaultLimit`, + * `gridColumns`, `infiniteScroll`, …) belong to the legacy gallery and are never read here (README continuous-browsing + * ruling). + * + * - `readPreferences(getStorage, normalize)` → `{values, persisted}`: the workspace keys, normalized by `gallery-settings.js`; + * `persisted` is false when this browser refused its storage. + * - `writePreferences(dirty, getStorage, {baseline})` → `{persisted, conflicts}`: re-reads the stored object and writes + * back only the dirty keys, every other key byte-for-byte as stored. A write the browser refused (denied storage, a + * full quota) leaves the stored bytes as they were and keeps the change for this tab only, so every reader in this + * tab still sees it; the next write storage accepts stores those changes too. `conflicts` lists the dirty keys another page changed since `baseline` was read; the explicit + * save still wins. + * - `subscribePreferences(win, onChange, getStorage)` → unsubscribe: `onChange(readPreferences())` after another page + * changed the settings (a `storage` event). + * - `reconcilePreferences({baseline, drafts, next})` → `{conflicts}`: which drafts such a change overtook. + * + * Every storage access is in try/catch: a private window, blocked site data or a malformed value never breaks a page. + */ +export const PREFERENCES_KEY = 'gallerySettings'; +export const WORKSPACE_PREFERENCE_KEYS = Object.freeze(['workspaceLayout', 'workspaceDensity', 'autoLoadMetadata', 'cacheMetadata', + 'imageQuality', 'videoAutoplay', 'videoMute', 'videoLoop', 'showFilePaths', 'showImageInfo', 'sidebarCollapsedByDefault']); + +// The changes this tab could not store, overlaid on every read until storage accepts them. +const tabOnly = new Map(); +const defaultStorage = () => globalThis.localStorage; +const normalizer = () => globalThis.GallerySettings.normalizeGallerySettings; +const isObject = value => value !== null && typeof value === 'object' && !Array.isArray(value); + +/** Forget what this tab kept without storage (tests, and nothing else). */ +export function resetTabPreferences() { + tabOnly.clear(); +} + +const pick = settings => Object.fromEntries(WORKSPACE_PREFERENCE_KEYS.map(key => [key, settings[key]])); +/** The gallery's defaults for the workspace keys. */ +export function preferenceDefaults() { + return pick(normalizer()(null)); +} + +/** The stored object (`{}` when absent or malformed), or null when storage is unavailable. */ +function storedObject(storage) { + let text; + try { text = storage.getItem(PREFERENCES_KEY); } catch { return null; } + try { + const parsed = text ? JSON.parse(text) : null; + return isObject(parsed) ? parsed : {}; + } catch { + return {}; + } +} + +function storageOf(getStorage) { + try { return getStorage() ?? null; } catch { return null; } +} + +/** `normalize` is gallery-settings.js's `normalizeGallerySettings` (the page's `window.GallerySettings` by default). */ +export function readPreferences(getStorage = defaultStorage, normalize = normalizer()) { + const storage = storageOf(getStorage); + const stored = storage ? storedObject(storage) : null; + const values = pick(normalize({ ...(stored ?? {}), ...Object.fromEntries(tabOnly) })); + return { values, persisted: stored !== null }; +} + +/** A change is only a workspace key with a value the gallery's normalizer keeps as given. */ +function validate(dirty) { + if (!isObject(dirty)) throw new TypeError('Preferences need an object of changes'); + for (const [key, value] of Object.entries(dirty)) { + if (!WORKSPACE_PREFERENCE_KEYS.includes(key)) throw new TypeError(`Not a workspace display preference: ${key}`); + if (normalizer()({ [key]: value })[key] !== value) throw new TypeError(`Invalid value for ${key}`); + } +} + +export function writePreferences(dirty, getStorage = defaultStorage, { baseline = null } = {}) { + validate(dirty); + const keep = () => { for (const [key, value] of Object.entries(dirty)) tabOnly.set(key, value); }; + const storage = storageOf(getStorage); + const stored = storage ? storedObject(storage) : null; + if (stored === null) { keep(); return { persisted: false, conflicts: [] }; } + const current = pick(normalizer()(stored)); + const conflicts = baseline ? Object.keys(dirty).filter(key => Object.hasOwn(baseline, key) && current[key] !== baseline[key]) : []; + try { + // The changes this tab could not store before are stored with this one, so "Saved" is true for all of them. + storage.setItem(PREFERENCES_KEY, JSON.stringify({ ...stored, ...Object.fromEntries(tabOnly), ...dirty })); + } catch { + keep(); + return { persisted: false, conflicts }; + } + tabOnly.clear(); + return { persisted: true, conflicts }; +} + +/** The current values for a display consumer, or null on a page without gallery-settings.js (a unit test). */ +export function displayPreferences(getStorage = defaultStorage) { + try { return readPreferences(getStorage).values; } catch { return null; } +} + +export function subscribePreferences(win, onChange, getStorage = defaultStorage) { + const listener = event => { + if (event.key !== null && event.key !== PREFERENCES_KEY) return; + if (event.storageArea && event.storageArea !== storageOf(getStorage)) return; + onChange(readPreferences(getStorage)); + }; + win.addEventListener('storage', listener); + return () => win.removeEventListener('storage', listener); +} + +export function reconcilePreferences({ baseline, drafts, next }) { + return { conflicts: Object.keys(drafts).filter(key => next[key] !== baseline[key] && next[key] !== drafts[key]) }; +} diff --git a/web/js/workspace/state.mjs b/web/js/workspace/state.mjs index 7957f4dd..3255381a 100644 --- a/web/js/workspace/state.mjs +++ b/web/js/workspace/state.mjs @@ -14,11 +14,29 @@ export const routes = [ { id: 'favorites', label: 'Favorites' }, // The concept's Activity (app.js renderActivity 732–738): this server session's jobs. No href: no Overview card. { id: 'activity', label: 'Activity' }, - { id: 'system', label: 'System', href: '/prompt_manager/admin', - description: 'Open the current dashboard, then choose Settings or Maintenance for configuration, scans, backups and diagnostics.', action: 'Open existing administration' }, + // Native System (#249 System S2): `#/system/
`. Q12: Overview's card opens its Overview section (`native`: + // a workspace hash, not a legacy page); each undelivered section hands off to its legacy tool (D94). The + // description is the concept's System heading (app.js renderSystem 700). + { id: 'system', label: 'System', href: '#/system/overview', native: true, + description: 'Configuration, maintenance and recovery—with browsing kept in the workspace.', action: 'Open System' }, ]; -const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'inspect', 'overview', 'saved', 'favorites', 'activity']); +const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'inspect', 'overview', 'saved', 'favorites', 'activity', 'system']); + +/** + * System's sections in the concept's order (app.js `systemSections` 635–644); the concept's `general` is production's + * `overview`, so `#/system/overview` sits beside the production hub `#/overview` (D28). + */ +export const SYSTEM_SECTIONS = Object.freeze([ + ['overview', 'Overview'], ['appearance', 'Appearance'], ['sources', 'Sources & scans'], ['integrations', 'Integrations'], + ['tagging', 'Auto-tagging'], ['maintenance', 'Maintenance'], ['backups', 'Backup & export'], ['diagnostics', 'Diagnostics & logs'], +].map(section => Object.freeze(section))); +const SYSTEM_SECTION_IDS = new Set(SYSTEM_SECTIONS.map(([id]) => id)); + +/** Whether `id` names a System section. */ +export function isSystemSection(id) { + return typeof id === 'string' && SYSTEM_SECTION_IDS.has(id); +} /** A saved view (#249 D16): the shell resolves its id to the view's page, name and state. */ const SAVED_ROUTE = Object.freeze({ id: 'saved', label: 'Saved view' }); @@ -39,6 +57,14 @@ export function resolveRoute(hash) { // One job of the Activity page (its id is the registry's uuid4 hex). const job = /^#\/activity\/([0-9a-f]{32})$/.exec(hash); if (job) return {...routes.find(route => route.id === 'activity'), jobId:job[1], hash}; + // `#/system` is the Overview section under the sidebar's own hash; an unknown child is Overview, its hash replaced. + const system = /^#\/system(?:\/(.*))?$/.exec(hash); + if (system) { + const route = routes.find(item => item.id === 'system'); + if (system[1] === undefined) return {...route, section:'overview', hash}; + const section = isSystemSection(system[1]) ? system[1] : 'overview'; + return {...route, section, hash:`#/system/${section}`}; + } const match = /^#\/library\/([1-9]\d*)$/.exec(hash); if (match && Number.isSafeInteger(Number(match[1]))) { return {...routes.find(route => route.id === 'library'), positiveId:Number(match[1]), hash}; @@ -117,7 +143,7 @@ export function mountRouter(win, onRoute) { const hash = route.hash || `#/${route.id}`; if (win.location.hash !== hash) win.history.replaceState(win.history.state, '', hash); if (current?.id === route.id && current?.positiveId === route.positiveId && current?.savedId === route.savedId && - current?.jobId === route.jobId) return; + current?.jobId === route.jobId && current?.section === route.section) return; current = route; onRoute(route); }; diff --git a/web/js/workspace/system-appearance.mjs b/web/js/workspace/system-appearance.mjs new file mode 100644 index 00000000..290bf778 --- /dev/null +++ b/web/js/workspace/system-appearance.mjs @@ -0,0 +1,143 @@ +/** + * System › Appearance (#249 System S3 Task 8): the concept's `renderSystem` "appearance" body (app.js 676–678), one + * `section.surface` "Browsing & display" of `.setting-row`s whose values are the drafts of the one Save row. + * + * - The browser rows are the display defaults this browser keeps in the shared `gallerySettings` object + * (preferences.mjs, Q3): Default layout and Thumbnail size (`workspaceLayout`, `workspaceDensity`), the metadata, + * quality (Optimized→`medium`, Original→`high`, Fast→`low`), video and path rows. Images, the inspector and the + * viewer read them on their next visit; a toolbar change still applies to the current view only. + * - "Open from ComfyUI" (New tab→`newtab`, Popup window→`popup`) and "Node search auto-hide (seconds)" (a whole number + * 0–300) are the server's `webui_display_mode` and `result_timeout`, saved with the settings revision (#220). + * - "Continuous browsing" is the concept's fixed row: there is no paging or column control (README ruling). + * - D97: added rows for the legacy gallery settings the concept leaves out: "Show image details on cards" + * (`showImageInfo`), "Collapse viewer details by default" (`sidebarCollapsedByDefault`) and the operation "Reset + * display defaults", which drafts the gallery's defaults for these browser rows and touches no other key. + * + * Controls follow the store: a value read later (the host settings, another tab's change) repaints them. A host row + * whose value could not be read stays disabled and says so ("Server settings could not be read."). + */ +import { settingRow, operationRow } from './system-controls.mjs'; +import { preferenceDefaults } from './preferences.mjs'; + +export const RESULT_TIMEOUT_MAX = 300; +/** A host row's line while the server settings could not be read (its control stays disabled). */ +export const HOST_UNAVAILABLE = 'Server settings could not be read.'; +const BOOLEAN = Object.freeze({ type: 'toggle' }); + +/** + * The concept's rows in its order, then the D97 rows: `[id, label, description, domain, key, control]`. A select's + * options are `[stored value, concept label]`. + */ +export const APPEARANCE_ROWS = Object.freeze([ + ['default-view', 'Default layout', 'Choose your starting image layout.', 'browser', 'workspaceLayout', + { type: 'select', options: [['grid', 'Grid'], ['list', 'List'], ['masonry', 'Masonry']] }], + ['density', 'Thumbnail size', 'Readable previews by default; no fixed column-count setting.', 'browser', 'workspaceDensity', + { type: 'select', options: [['comfortable', 'Comfortable'], ['large', 'Large'], ['compact', 'Compact']] }], + 'continuous', + ['metadata-auto', 'Load metadata on selection', 'Use the same inspector in the gallery and image viewer.', 'browser', 'autoLoadMetadata', BOOLEAN], + ['metadata-cache', 'Cache metadata in this browser', 'Reuse fetched metadata until it changes or you clear this cache.', 'browser', 'cacheMetadata', BOOLEAN], + ['image-quality', 'Image loading quality', 'Preserve original / optimized / fast image-loading preferences.', 'browser', 'imageQuality', + { type: 'select', options: [['medium', 'Optimized'], ['high', 'Original'], ['low', 'Fast']] }], + ['video-auto', 'Autoplay videos', 'Videos stay within the same gallery and use native playback controls.', 'browser', 'videoAutoplay', BOOLEAN], + ['video-muted', 'Mute videos by default', 'Keep playback quiet while browsing.', 'browser', 'videoMute', BOOLEAN], + ['video-loop', 'Loop videos', 'Repeat a selected clip until you navigate away.', 'browser', 'videoLoop', BOOLEAN], + ['paths', 'Show file paths', 'Paths appear under File & provenance, not across every card.', 'browser', 'showFilePaths', BOOLEAN], + ['open-mode', 'Open from ComfyUI', 'Keep the node pack’s popup / new-tab preference.', 'host', 'webui_display_mode', + { type: 'select', options: [['newtab', 'New tab'], ['popup', 'Popup window']] }], + ['timeout', 'Node search auto-hide (seconds)', 'Node search behavior only. Use 0 to keep results visible.', 'host', 'result_timeout', + { type: 'number' }], + // D97: the legacy gallery's card and viewer settings. + ['card-details', 'Show image details on cards', 'Show each card’s title and file details under the preview.', 'browser', 'showImageInfo', BOOLEAN], + ['viewer-details', 'Collapse viewer details by default', 'Open the image viewer with its details panel hidden.', 'browser', + 'sidebarCollapsedByDefault', BOOLEAN], +]); + +/** The browser keys "Reset display defaults" restores (D97). */ +export const DISPLAY_KEYS = Object.freeze(APPEARANCE_ROWS.filter(row => Array.isArray(row) && row[3] === 'browser').map(row => row[4])); + +/** A whole number of seconds from 0 to 300 (the server's `validate_result_timeout`), else null. */ +export function resultTimeout(text) { + const value = String(text).trim(); + if (!/^\d+$/.test(value)) return null; + const seconds = Number(value); + return seconds <= RESULT_TIMEOUT_MAX ? seconds : null; +} + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +/** The concept's fixed "Continuous browsing" row: `div > h3 + p` and the "Infinite scroll" chip. */ +function continuousRow() { + const row = el('div', undefined, 'setting-row'); + const text = el('div'); + text.append(el('h3', 'Continuous browsing'), + el('p', 'Images load automatically as you scroll in Grid, List and Masonry. Batch sizing is handled internally.')); + row.append(text, el('span', 'Infinite scroll', 'chip blue')); + return row; +} + +/** The row's control (`data-setting`), wherever the Toolkit switch nests it. */ +function controlOf(node, id) { + if (node.dataset?.setting === id) return node; + for (const child of node.children ?? []) { + const found = controlOf(child, id); + if (found) return found; + } + return null; +} + +/** Show a stored value in its control; a value not read yet leaves the control disabled. */ +function paint(input, control, value) { + input.disabled = value === undefined; + if (value === undefined) return; + if (control.type === 'toggle') input.checked = value === true; + else if (document.activeElement !== input || control.type === 'select') input.value = String(value); +} + +/** Mount Appearance in `container` (System's `.settings-layout`); returns the section's nodes. */ +export function mountAppearance(container, { scope, drafts }) { + const surface = el('section', undefined, 'surface'); + surface.append(el('h2', 'Browsing & display'), el('p', 'One set of defaults; toolbar changes apply to the current view.')); + const controls = []; + for (const row of APPEARANCE_ROWS) { + if (row === 'continuous') { surface.append(continuousRow()); continue; } + const [id, label, description, domain, key, control] = row; + const onChange = value => { + if (key === 'result_timeout') { + const seconds = resultTimeout(input.value); + if (seconds === null) { drafts.reject(domain, key); return; } + value = seconds; + } + drafts.set(domain, key, value); + }; + const node = settingRow({ id, label, description, control, scope, onChange }); + surface.append(node); + const input = controlOf(node, id); + input.dataset.key = key; + controls.push({ input, control, domain, key, text: node.children[0], note: null }); + } + surface.append(operationRow({ title: 'Reset display defaults', + description: 'Restore this browser’s display defaults, then save. Other settings are kept.', label: 'Reset', scope, + onClick: () => { + const defaults = preferenceDefaults(); + for (const key of DISPLAY_KEYS) drafts.set('browser', key, defaults[key]); + } })); + const repaint = () => { + const unavailable = drafts.hostState?.() === 'failed'; + for (const row of controls) { + const value = drafts.get(row.domain, row.key); + paint(row.input, row.control, value); + if (row.domain !== 'host') continue; + if (unavailable && value === undefined) { + if (!row.note) { row.note = el('p', HOST_UNAVAILABLE, 'small'); row.text.append(row.note); } + } else if (row.note) { row.note.remove(); row.note = null; } + } + }; + scope.cleanup(drafts.subscribe(repaint)); + repaint(); + return [surface]; +} diff --git a/web/js/workspace/system-controls.mjs b/web/js/workspace/system-controls.mjs new file mode 100644 index 00000000..79bf6183 --- /dev/null +++ b/web/js/workspace/system-controls.mjs @@ -0,0 +1,132 @@ +/** + * System's row builders on the concept's markup (#249 System S2 Task 5), built with the DOM, never HTML strings: + * - `settingRow` is the concept's `setting(id, label, description, type, options)` (app.js 624–634): `.setting-row` > + * `div` > `label[for]` + `p`, then the control. A toggle is the Toolkit switch, as the concept's `hydrateSwitches` + * (708–717) makes it; a range is the concept's `min=1 max=16` unless the row gives its own bounds. + * - `operationRow` is `operation(title, desc, action, label, extra, danger)` (645–654): `.operation` > `div` > `h3` + `p`, + * then `.row-actions` with one button, `danger` when flagged. + * - `sectionNav` is `sectionNav(items, active, action)` (306–308): `.section-nav[role=group]` buttons with `aria-pressed`. + * Every listener is registered on the caller's `scope`, so disposing it leaves none behind. + */ +import { createButton, createSwitch } from '../shared/toolkit/core.js'; + +const INPUT_TYPES = new Set(['number', 'password', 'text', 'range']); +const RANGE_DEFAULTS = Object.freeze({ min: 1, max: 16 }); + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +function requireText(values) { + for (const [name, value] of Object.entries(values)) { + if (typeof value !== 'string' || !value) throw new TypeError(`The row needs a ${name}`); + } +} + +function requireScope(scope, callback, name) { + if (callback === undefined) return; + if (typeof callback !== 'function') throw new TypeError(`The row ${name} must be a function`); + if (typeof scope?.listen !== 'function') throw new TypeError('The row needs the section scope for its listener'); +} + +/** `[value, label]` pairs from strings or pairs. */ +function optionPairs(options) { + if (!Array.isArray(options) || options.length === 0) throw new TypeError('A select row needs its options'); + return options.map(option => (Array.isArray(option) ? option.map(String) : [String(option), String(option)])); +} + +/** The control a spec describes, and how to read its value. */ +function buildControl(id, spec) { + if (spec?.type === 'toggle') { + const field = createSwitch('', '', id); + const input = field.children[0]; + // The concept's toggles are on unless configured off (`val !== false`). + input.checked = spec.checked !== false; + return { node: field, input, read: () => input.checked }; + } + if (spec?.type === 'select') { + const select = el('select'); + for (const [value, label] of optionPairs(spec.options)) { + const option = el('option', label); + option.value = value; + select.append(option); + } + if (spec.value !== undefined) select.value = String(spec.value); + return { node: select, input: select, read: () => select.value }; + } + if (INPUT_TYPES.has(spec?.type)) { + const input = el('input'); + input.type = spec.type; + const bounds = spec.type === 'range' ? { ...RANGE_DEFAULTS, ...spec } : spec; + for (const key of ['min', 'max', 'step']) if (bounds[key] !== undefined) input[key] = String(bounds[key]); + input.value = spec.value === undefined || spec.value === null ? '' : String(spec.value); + const numeric = spec.type === 'number' || spec.type === 'range'; + return { node: input, input, read: () => (numeric ? Number(input.value) : input.value) }; + } + throw new TypeError(`Unknown setting control: ${spec?.type}`); +} + +/** + * One `.setting-row`. `control` is a spec (`{type:'toggle', checked}`, `{type:'select', options, value}` with string or + * `[value, label]` options, `{type:'number'|'password'|'text'|'range', value, min, max, step}`) or a ready-made node, + * which takes `id`. `onChange(value)` (on `scope`) reports the checked state, a number for number and range, else the + * text. + */ +export function settingRow({ id, label, description, control, scope, onChange }) { + requireText({ id, label, description }); + requireScope(scope, onChange, 'onChange'); + let built; + if (control && typeof control === 'object' && typeof control.type !== 'string' && 'tagName' in control) { + control.id = id; + built = { node: control, input: control, read: () => control.value }; + } else { + built = buildControl(id, control); + built.input.id = id; + } + built.input.dataset.setting = id; + const row = el('div', undefined, 'setting-row'); + const text = el('div'); + const caption = el('label', label); + caption.htmlFor = id; + text.append(caption, el('p', description)); + row.append(text, built.node); + if (onChange) scope.listen(built.input, 'change', () => onChange(built.read())); + return row; +} + +/** One `.operation` row; `onClick(button)` runs on `scope`. */ +export function operationRow({ title, description, label, danger = false, onClick, scope }) { + requireText({ title, description, label }); + if (typeof onClick !== 'function') throw new TypeError('The operation needs onClick'); + requireScope(scope, onClick, 'onClick'); + const row = el('div', undefined, 'operation'); + const text = el('div'); + text.append(el('h3', title), el('p', description)); + const actions = el('div', undefined, 'row-actions'); + const action = createButton(label, danger ? 'danger' : ''); + scope.listen(action, 'click', () => onClick(action)); + actions.append(action); + row.append(text, actions); + return row; +} + +/** The `.section-nav` group: one button per `[id, title]`, the `active` one pressed; `onSelect(id)` runs on `scope`. */ +export function sectionNav({ label, sections, active, onSelect, scope }) { + requireText({ label }); + if (typeof onSelect !== 'function') throw new TypeError('The section nav needs onSelect'); + requireScope(scope, onSelect, 'onSelect'); + const nav = el('div', undefined, 'section-nav'); + nav.setAttribute('role', 'group'); + nav.setAttribute('aria-label', label); + for (const [id, title] of sections) { + const item = createButton(title); + item.dataset.value = id; + item.setAttribute('aria-pressed', String(id === active)); + scope.listen(item, 'click', () => onSelect(id)); + nav.append(item); + } + return nav; +} diff --git a/web/js/workspace/system-overview.mjs b/web/js/workspace/system-overview.mjs new file mode 100644 index 00000000..ac902064 --- /dev/null +++ b/web/js/workspace/system-overview.mjs @@ -0,0 +1,432 @@ +/** + * System › Overview (#249 System S2 Task 6): the concept's `renderSystem` "general" body (app.js 658–675) on real + * values, its Statistics dialog ("Usage statistics", a1600–1604) and its health report (`health-check`, a1439–1456). + * + * - The four `.summary-metrics` and the two `.two-col` surfaces ("Library health", "Workspace preferences") come from + * one `GET /prompt_manager/system/overview`. D93: metric 4 is the last health result (Not checked / Passed / + * N issues / Incomplete); the review, backup, thumbnail and sources rows say what the server reports, and nothing + * reads 0 before the read answers: a load shows a status line, a failure an alert and Retry. The thumbnail row is + * its own read of `/images/thumbnails/status` (a walk of every source): "Checking thumbnails…" until it answers. + * - D96: the Statistics dialog lists the real runs and average rating per category, then "Library totals". + * - D95: the "Health report" lists every check the server ran, Missing files per image source, with a chip per real + * result and when it was checked. The check runs from Maintenance's notice ("Run health checks", D123) as a + * read job of the registry, tracked in the Activity store (its event stream); the report opens when it ends. + * + * Reads run through `latestOnly()` on the section's scope, so a late answer never repaints a newer section. + */ +import { createButton } from '../shared/toolkit/core.js'; +import { navigateTo } from './state.mjs'; +import { closeWorkspaceDialog, openContentDialog } from './workspace-dialogs.mjs'; +import { showWorkspaceToast } from './toast.mjs'; +import { noun } from './plural.mjs'; +import { operationRow } from './system-controls.mjs'; + +export const OVERVIEW_URL = '/prompt_manager/system/overview'; +export const STATISTICS_URL = '/prompt_manager/system/statistics'; +export const HEALTH_URL = '/prompt_manager/system/health'; +export const THUMBNAILS_URL = '/prompt_manager/images/thumbnails/status'; +export const HEALTH_KIND = 'health'; + +export const LOADING_TEXT = 'Loading the overview…'; +export const THUMBNAILS_LOADING = 'Checking thumbnails…'; +export const LOAD_FAILED_TEXT = 'The overview could not be loaded. Your saved data has not changed.'; +export const STATISTICS_FAILED_TEXT = 'Statistics could not be loaded. Your saved data has not changed.'; +export const HEALTH_FAILED_TEXT = 'The health check could not finish. Nothing was changed.'; +export const NOT_AVAILABLE = 'Not available yet.'; + +const number = value => Number(value).toLocaleString(); + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +/** + * D93: metric 4, the last health result of this server session: the checks that found issues, even when another + * check could not run; otherwise Passed, or Incomplete while a check was unavailable or failed. + */ +export function integrityText(health) { + if (!health) return 'Not checked'; + if (health.issues > 0) return `${number(health.issues)} ${noun(health.issues, 'issue')}`; + return health.status === 'passed' ? 'Passed' : 'Incomplete'; +} + +/** The concept's four metrics `[value, label]`, in its order. */ +export function metricValues(overview) { + return [ + [number(overview.library.positive_prompts), 'Positive prompts'], + [number(overview.library.negative_variants), 'Negative variants'], + [number(overview.images.indexed), 'Images'], + [integrityText(overview.health), 'Database integrity'], + ]; +} + +/** D93 (Q6): the review row's title; a count only once the Needs review queue reports one. */ +export function ambiguousTitle(needsReview) { + const pending = needsReview?.pending; + if (!Number.isSafeInteger(pending)) return 'Ambiguous image links'; + return `${number(pending)} ambiguous image ${noun(pending, 'link')}`; +} + +/** D93: the backup row; only a recorded, verified backup is ever named. */ +export function backupText(backup) { + if (!backup) return 'No retained backup record.'; + const when = new Date(backup.verified_at ?? backup.created_at); + return Number.isNaN(when.getTime()) ? 'No retained backup record.' : `Last verified backup ${when.toLocaleString()}.`; +} + +/** + * D93: the thumbnail row's title from the `/images/thumbnails/status` answer: undefined while it is read, null when + * the read failed; with no image source (`sources.effective` 0) there is nothing to count. + */ +export function thumbnailTitle(status, sources) { + if (status === undefined) return THUMBNAILS_LOADING; + const missing = status?.pending; + if (!Number.isSafeInteger(missing) || sources?.effective === 0) return 'Thumbnail status unavailable'; + if (missing === 0) return 'No missing thumbnails'; + return `${number(missing)} missing ${noun(missing, 'thumbnail')}`; +} + +const SOURCES_SHARED = 'Monitoring and enrichment share one source list.'; + +/** D93: the sources row, `[title, description]`: the sources in use and how they were chosen. */ +export function sourcesCopy(sources) { + const { configured, effective, auto_detected: auto } = sources; + const title = `${number(effective)} output ${noun(effective, 'source')}`; + if (auto) return [title, `None configured is available; ComfyUI's output folder is used. ${SOURCES_SHARED}`]; + if (configured !== effective) return [title, `${number(effective)} of ${number(configured)} configured ${noun(configured, 'source')} available. ${SOURCES_SHARED}`]; + return [title, SOURCES_SHARED]; +} + +// D95: an issue chip names what was found; the concept's own wording for missing files and ambiguous metadata. +const ISSUE_NOUNS = Object.freeze({ + integrity: ['problem found', 'problems found'], + foreign_keys: ['broken reference', 'broken references'], + canonical_hashes: ['hash does not match its text', 'hashes do not match their text'], + pair_uniqueness: ['duplicate pair row', 'duplicate pair rows'], + missing_files: ['link needs review', 'links need review'], + ambiguous_metadata: ['image needs review', 'images need review'], + consistency: ['inconsistency', 'inconsistencies'], +}); + +/** D95: a check's chip from its real status. */ +export function checkChip(check) { + if (check.status === 'passed') return 'Passed'; + if (check.status === 'failed') return 'Could not check'; + if (check.status === 'unavailable') return check.id === 'ambiguous_metadata' ? NOT_AVAILABLE.slice(0, -1) : 'Not available'; + const [one, many] = ISSUE_NOUNS[check.id] ?? ['issue', 'issues']; + if (!Number.isSafeInteger(check.count)) return 'Issues found'; + // SQLite stops listing integrity problems at its limit, so a capped count is a lower bound ("100+"). + if (check.capped === true) return `${number(check.count)}+ ${many}`; + return `${number(check.count)} ${check.count === 1 ? one : many}`; +} + +function sourceChip(source) { + if (source.status !== 'checked') return 'Not available'; + return source.count === 0 ? 'Passed' : checkChip({ id: 'missing_files', status: 'issues', count: source.count }); +} + +/** D95: the report's rows `[label, chip]` in the server's order; Missing files once per image source when several. */ +export function healthRows(report) { + return report.checks.flatMap(check => { + const sources = check.id === 'missing_files' && Array.isArray(check.sources) ? check.sources : []; + if (sources.length < 2) return [[check.label, checkChip(check)]]; + return sources.map(source => [`${check.label} · ${source.name}`, sourceChip(source)]); + }); +} + +/** D95: when the report was taken, and whether it is complete. */ +export function healthSummary(report) { + const when = new Date(report.checked_at); + const time = Number.isNaN(when.getTime()) ? '' : `Checked ${when.toLocaleString()}. `; + if (report.status === 'passed') return `${time}Every check passed.`; + if (report.status === 'incomplete') return `${time}Some checks could not run, so this report is incomplete.`; + return `${time}Nothing was changed; review the checks with issues.`; +} + +/** D96: the concept's "Runs by category" rows `[category, runs, average rating]`. */ +export function categoryRows(statistics) { + return statistics.by_category.map(row => [ + row.category ?? 'Uncategorized', + number(row.runs), + row.average_rating === null || row.average_rating === undefined ? 'Not rated' + : `${Number(row.average_rating).toLocaleString(undefined, { maximumFractionDigits: 2 })} / 5`, + ]); +} + +/** D96: the added "Library totals" rows `[label, count]` (the legacy statistics and the rated images). */ +export function totalsRows(statistics) { + const { totals } = statistics; + return [['Prompts', totals.total_prompts], ['Tags', totals.total_tags], ['Images', totals.total_images], + ['Rated images', statistics.rated_images]].map(([label, value]) => [label, number(value)]); +} + +function table(caption, head, rows, empty) { + const node = el('table'); + const header = el('tr'); + for (const text of head) { + const cell = el('th', text); + cell.scope = 'col'; + header.append(cell); + } + const thead = el('thead'); + thead.append(header); + const tbody = el('tbody'); + for (const values of rows) { + const row = el('tr'); + row.append(...values.map(text => el('td', text))); + tbody.append(row); + } + if (rows.length === 0) { + const row = el('tr'); + const cell = el('td', empty); + cell.colSpan = head.length; + row.append(cell); + tbody.append(row); + } + node.append(el('caption', caption), thead, tbody); + return node; +} + +/** The Statistics dialog: "Runs by category" (the concept's table), then "Library totals" (D96). */ +export function statisticsNodes(statistics) { + return [ + table('Runs by category', ['Category', 'Runs', 'Avg. rating'], categoryRows(statistics), 'No prompts in the library yet.'), + table('Library totals', ['Total', 'Count'], totalsRows(statistics), ''), + ]; +} + +/** The concept's health report body: `.operation-list` of `h3` + chip, the D95 summary, then Close / Review missing links. */ +export function healthNodes(report, { scope, onReviewMissing }) { + const list = el('div', undefined, 'operation-list'); + for (const [label, chip] of healthRows(report)) { + const row = el('div', undefined, 'operation'); + row.append(el('h3', label), el('span', chip, 'chip')); + list.append(row); + } + const close = createButton('Close'); + const review = createButton('Review missing links', 'ca-primary'); + scope.listen(close, 'click', () => closeWorkspaceDialog()); + scope.listen(review, 'click', () => onReviewMissing()); + const actions = el('div', undefined, 'dialog-actions'); + actions.append(close, review); + return [list, el('p', healthSummary(report), 'small'), actions]; +} + +/** + * Wait until the job `id` has ended, following it in the Activity store (`activity`), which streams the job's events + * and marks it lost when the server no longer lists it; nothing here polls. Resolves its final state ('finished', + * 'failed', 'cancelled' or 'lost'). + */ +export function waitForJob(activity, id, { signal } = {}) { + if (signal?.aborted) return Promise.reject(signal.reason); + return new Promise((resolve, reject) => { + const check = () => { + const state = activity.find(id)?.state; + if (state && state !== 'running') finish(resolve, state); + }; + const onAbort = () => finish(reject, signal.reason); + const unsubscribe = activity.subscribe(check); + function finish(settle, value) { + unsubscribe(); + signal?.removeEventListener('abort', onAbort); + settle(value); + } + signal?.addEventListener('abort', onAbort, { once: true }); + check(); + }); +} + +/** + * Run the health check and resolve its report. The started job is tracked in the Activity store; a check already + * running (409 busy, another tab) is found in the store's job list and waited for instead of refused. Rejects when the + * check fails or ends without a report. + */ +export async function runHealthCheck(api, { activity, signal } = {}) { + let id; + try { + const { job } = await api.post(HEALTH_URL, {}, { signal }); + id = activity.track(job).id; + } catch (error) { + if (error?.status !== 409 || error?.code !== 'busy') throw error; + await activity.load(); + id = activity.running().find(job => job.kind === HEALTH_KIND)?.id; + // The other check ended between the refusal and this read: its report is the latest. + } + const state = id ? await waitForJob(activity, id, { signal }) : 'finished'; + if (state !== 'finished') throw new Error(HEALTH_FAILED_TEXT); + const { report } = await api.get(HEALTH_URL, { signal, cache: 'no-store' }); + if (!report) throw new Error(HEALTH_FAILED_TEXT); + return report; +} + +/** Open the Health report for `report` (the Maintenance notice's and, from S7, its Check action's dialog). */ +export function openHealthReport(report, { scope, trigger = null, navigate = navigateTo }) { + openContentDialog({ scope, title: 'Health report', trigger, nodes: healthNodes(report, { scope, + // Missing-link cleanup lives in Maintenance (its D94 hand-off until S7). + onReviewMissing: () => { closeWorkspaceDialog(); navigate('#/system/maintenance'); } }) }); +} + +/** + * The concept's Maintenance notice ("Check first. Review changes. Then apply." + "Run health checks", a689), which + * S2 delivers ahead of the rest of Maintenance (D123): the press runs the read-only check, then opens the report. + */ +export function healthNotice({ api, activity, scope, navigate = navigateTo, notify = showWorkspaceToast }) { + const notice = el('div', undefined, 'notice'); + const text = el('div'); + text.append(el('h2', 'Check first. Review changes. Then apply.'), + el('p', 'One home for database and media maintenance. Scanning is separate from changing or removing records.')); + const run = createButton('Run health checks', 'ca-primary'); + notice.append(text, run); + scope.listen(run, 'click', async () => { + if (run.getAttribute('aria-disabled') === 'true') return; + run.setAttribute('aria-disabled', 'true'); + run.textContent = 'Checking…'; + try { + const report = await runHealthCheck(api, { activity, signal: scope.signal }); + if (scope.signal.aborted) return; + openHealthReport(report, { scope, trigger: run, navigate }); + } catch (error) { + if (!scope.signal.aborted) notify(HEALTH_FAILED_TEXT); + } finally { + if (!scope.signal.aborted) { + run.removeAttribute('aria-disabled'); + run.textContent = 'Run health checks'; + } + } + }); + return notice; +} + +function metricsNode(overview) { + const metrics = el('div', undefined, 'summary-metrics'); + for (const [value, label] of metricValues(overview)) { + const metric = el('div', undefined, 'metric'); + metric.append(el('strong', value), el('span', label)); + metrics.append(metric); + } + return metrics; +} + +function surface(heading, rows) { + const section = el('section', undefined, 'surface'); + section.append(el('h2', heading), ...rows); + return section; +} + +/** The Overview's nodes, and its thumbnail row (painted by its own read). */ +function overviewNodes(overview, { scope, navigate, openStatistics }) { + const go = section => () => navigate(`#/system/${section}`); + const review = operationRow({ scope, title: ambiguousTitle(overview.needs_review), + description: Number.isSafeInteger(overview.needs_review?.pending) + ? 'Review metadata before assigning a positive / negative pair.' : NOT_AVAILABLE, + label: 'Review metadata', onClick: () => {} }); + // D93 (Q6): the review queue (Needs review) is not part of the workspace yet. + review.querySelector('button').setAttribute('aria-disabled', 'true'); + const [sourcesTitle, sourcesDescription] = sourcesCopy(overview.sources); + const thumbnails = operationRow({ scope, title: THUMBNAILS_LOADING, description: 'Generate only missing previews when necessary.', + label: 'Review thumbnails', onClick: go('sources') }); + const health = surface('Library health', [ + review, + operationRow({ scope, title: 'Backup status', description: backupText(overview.backup), label: 'View backups', + onClick: go('backups') }), + // Until the thumbnail review dialog (S5), Review thumbnails opens Sources, where thumbnail generation lives. + thumbnails, + ]); + const preferences = surface('Workspace preferences', [ + operationRow({ scope, title: 'One display configuration', description: 'Grid and density settings are shared across views.', + label: 'Appearance', onClick: go('appearance') }), + operationRow({ scope, title: sourcesTitle, description: sourcesDescription, label: 'Manage sources', onClick: go('sources') }), + operationRow({ scope, title: 'Statistics', description: 'Runs and ratings by category, with library totals.', label: 'Explore', + onClick: openStatistics }), + ]); + const columns = el('div', undefined, 'two-col'); + columns.append(health, preferences); + return { nodes: [metricsNode(overview), columns], thumbnails }; +} + +/** + * The Overview section's mount (`mountSystem`'s `sections.overview`): returns its loading line, which the server's + * answer replaces in place (the concept's metrics and columns sit directly in `.settings-layout`). A read that a newer + * one superseded, or one answering after the section left, paints nothing. + */ +export function mountOverview(container, { api, scope, navigate = navigateTo, notify = showWorkspaceToast, + latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) { + const overviewRead = latestOnly({ lifetime: scope.signal }); + const statisticsRead = latestOnly({ lifetime: scope.signal }); + const thumbnailsRead = latestOnly({ lifetime: scope.signal }); + let shown = []; + // The thumbnail status (undefined while read, null when it failed), painted into the row whenever both are there. + let thumbnailStatus; + let thumbnailRow = null; + let sources = null; + const paintThumbnails = () => { + if (thumbnailRow) thumbnailRow.querySelector('h3').textContent = thumbnailTitle(thumbnailStatus, sources); + }; + + function swap(nodes) { + const anchor = shown.find(node => node.isConnected); + if (anchor) anchor.before(...nodes); + for (const node of shown) node.remove(); + shown = nodes; + return nodes; + } + + async function openStatistics(button) { + try { + const statistics = await statisticsRead(signal => api.get(STATISTICS_URL, { signal, cache: 'no-store' })); + if (statistics === STALE || scope.signal.aborted) return; + openContentDialog({ scope, title: 'Usage statistics', trigger: button, nodes: statisticsNodes(statistics) }); + } catch (error) { + if (!scope.signal.aborted) notify(STATISTICS_FAILED_TEXT); + } + } + + function loading() { + const status = el('p', LOADING_TEXT, 'small'); + status.setAttribute('role', 'status'); + return [status]; + } + + async function load() { + try { + const overview = await overviewRead(signal => api.get(OVERVIEW_URL, { signal, cache: 'no-store' })); + if (overview === STALE || scope.signal.aborted) return; + const { nodes, thumbnails } = overviewNodes(overview, { scope, navigate, openStatistics }); + thumbnailRow = thumbnails; + sources = overview.sources; + swap(nodes); + paintThumbnails(); + } catch (error) { + if (scope.signal.aborted) return; + const alert = el('p', LOAD_FAILED_TEXT); + alert.setAttribute('role', 'alert'); + const retry = createButton('Retry'); + scope.listen(retry, 'click', () => { + swap(loading()); + load(); + loadThumbnails(); + }); + const panel = el('section', undefined, 'surface'); + panel.append(alert, retry); + swap([panel]); + } + } + async function loadThumbnails() { + let status; + try { + status = await thumbnailsRead(signal => api.get(THUMBNAILS_URL, { signal, cache: 'no-store' })); + } catch (error) { + status = null; + } + if (status === STALE || scope.signal.aborted) return; + thumbnailStatus = status ?? null; + paintThumbnails(); + } + const first = swap(loading()); + load(); + loadThumbnails(); + return first; +} diff --git a/web/js/workspace/system-state.mjs b/web/js/workspace/system-state.mjs new file mode 100644 index 00000000..b827dc3e --- /dev/null +++ b/web/js/workspace/system-state.mjs @@ -0,0 +1,238 @@ +/** + * The System draft store (#249 System S2 Task 5): every System section edits drafts behind the one Save row (the + * concept's `.save-row`, app.js renderSystem 705), in three domains: + * - `browser`: preferences kept in this browser (Appearance); + * - `host`: the server configuration, saved with the `revision` its read returned (`expected_revision`, #220); + * - `runtime`: values applied to the running server only (the log level, Q10b). + * + * `save()` writes browser, then host, then runtime, and only the domains with changes. A domain that fails keeps its + * drafts dirty and its keys `failed()`, so the section highlights them; the others are saved. A host save refused + * because the configuration changed elsewhere (`settings-changed`) is a `conflict` and keeps the draft. A host save before + * `load('host')` returned a revision fails without a request: a blind write would merge over a configuration changed + * elsewhere (S1 reads a null `expected_revision` as "merge into what is on disk"). `reloadHost()` reads the host values + * again (System calls it on mount); a host conflict reloads them itself, so the next save carries the current revision. + * Drafts live for + * the page session (Q10a): a section left and opened again shows them, and a reload of its values keeps them. + * + * Adapters: `readBrowser()` and `readRuntime()` return the values; `readHost()` returns `{values, revision}`. + * `writeBrowser(changes)` returns `{persisted}` (false: kept for this tab only) and throws an error with `code: + * 'settings-changed'` when another page changed a drafted value (a conflict: the browser values are read again), + * `writeHost(changes, {expected_revision})` the settings response (`{success, persisted, revision}`), `writeRuntime(changes)` anything; + * a write that throws failed. + */ +export { SYSTEM_SECTIONS, isSystemSection } from './state.mjs'; + +/** D92: the Save row's status copy; never the concept's prototype text. */ +export const SETTINGS_STATUS = Object.freeze({ + clean: 'No unsaved changes.', + dirty: 'Unsaved changes.', + browser: 'Saved in this browser.', + host: 'Saved to the server configuration.', + runtime: 'Applied until the server restarts.', + partial: 'Some changes were not saved. Review the highlighted settings.', + tabOnly: 'Saved for this tab only; browser storage is unavailable.', + conflict: 'Settings changed elsewhere. Reload this section to see them.', +}); + +export const DRAFT_DOMAINS = Object.freeze(['browser', 'host', 'runtime']); +const CONFLICT_CODE = 'settings-changed'; +const ADAPTERS = ['readBrowser', 'readHost', 'readRuntime', 'writeBrowser', 'writeHost', 'writeRuntime']; + +/** The status copy of one `save()` outcome (`{browser, host, runtime}`, each clean/saved/tab-only/applied/failed/conflict). */ +export function statusText(outcome) { + const results = DRAFT_DOMAINS.map(domain => outcome?.[domain]); + if (results.includes('conflict')) return SETTINGS_STATUS.conflict; + if (results.includes('failed')) return SETTINGS_STATUS.partial; + const saved = []; + if (outcome?.browser === 'saved') saved.push(SETTINGS_STATUS.browser); + if (outcome?.browser === 'tab-only') saved.push(SETTINGS_STATUS.tabOnly); + if (outcome?.host === 'saved') saved.push(SETTINGS_STATUS.host); + if (outcome?.runtime === 'applied') saved.push(SETTINGS_STATUS.runtime); + return saved.join(' ') || SETTINGS_STATUS.clean; +} + +function requireDomain(domain) { + if (!DRAFT_DOMAINS.includes(domain)) throw new TypeError(`Unknown settings domain: ${domain}`); +} + +/** Create the store; see the module comment for the adapters. */ +export function createSystemDrafts(adapters) { + for (const name of ADAPTERS) { + if (typeof adapters?.[name] !== 'function') throw new TypeError(`System drafts need ${name}`); + } + const baseline = { browser: {}, host: {}, runtime: {} }; + const drafts = { browser: new Map(), host: new Map(), runtime: new Map() }; + const failedKeys = { browser: new Set(), host: new Set(), runtime: new Set() }; + const listeners = new Set(); + let revision = null; + let hostRead = 'pending'; + let outcome = null; + let saving = null; + + const notify = () => { for (const listener of [...listeners]) listener(); }; + const changes = domain => Object.fromEntries(drafts[domain]); + + /** Read a domain's saved values; drafts are kept, and a draft equal to the new value is clean. */ + async function load(domain) { + requireDomain(domain); + let values; + if (domain === 'host') { + const read = await adapters.readHost(); + values = read?.values ?? {}; + revision = read?.revision ?? null; + hostRead = 'loaded'; + } else { + values = await (domain === 'browser' ? adapters.readBrowser() : adapters.readRuntime()); + } + baseline[domain] = { ...(values ?? {}) }; + for (const [key, value] of drafts[domain]) if (Object.is(baseline[domain][key], value)) drafts[domain].delete(key); + notify(); + return { ...baseline[domain] }; + } + + function get(domain, key) { + requireDomain(domain); + return drafts[domain].has(key) ? drafts[domain].get(key) : baseline[domain][key]; + } + + function set(domain, key, value) { + requireDomain(domain); + if (Object.is(baseline[domain][key], value)) drafts[domain].delete(key); + else drafts[domain].set(key, value); + failedKeys[domain].delete(key); + outcome = null; + notify(); + } + + function dirty(domain) { + if (domain !== undefined) { + requireDomain(domain); + return drafts[domain].size > 0; + } + return DRAFT_DOMAINS.some(name => drafts[name].size > 0); + } + + function discard(domain) { + for (const name of domain === undefined ? DRAFT_DOMAINS : [domain]) { + requireDomain(name); + drafts[name].clear(); + failedKeys[name].clear(); + } + outcome = null; + notify(); + } + + /** The values of one domain as written: they become its saved values, and drafts still equal to them are clean. */ + function commit(domain, written) { + for (const [key, value] of written) { + baseline[domain][key] = value; + if (drafts[domain].has(key) && Object.is(drafts[domain].get(key), value)) drafts[domain].delete(key); + } + } + + function fail(domain, written) { + for (const [key] of written) failedKeys[domain].add(key); + } + + async function saveDomain(domain) { + const written = new Map(drafts[domain]); + if (written.size === 0) return 'clean'; + const values = Object.fromEntries(written); + try { + if (domain === 'browser') { + const result = await adapters.writeBrowser(values); + commit(domain, written); + return result?.persisted === false ? 'tab-only' : 'saved'; + } + if (domain === 'host') { + if (typeof revision !== 'string') { fail(domain, written); return 'failed'; } + const result = await adapters.writeHost(values, { expected_revision: revision }); + if (result?.code === CONFLICT_CODE) { fail(domain, written); return 'conflict'; } + if (!result?.success || result.persisted !== true) { fail(domain, written); return 'failed'; } + if (typeof result.revision === 'string') revision = result.revision; + commit(domain, written); + return 'saved'; + } + await adapters.writeRuntime(values); + commit(domain, written); + return 'applied'; + } catch (error) { + fail(domain, written); + return error?.code === CONFLICT_CODE ? 'conflict' : 'failed'; + } + } + + /** + * Read the host values again; a failed read leaves host saves refused until one succeeds, and `hostState()` says + * so (System S3: the host rows show that the server settings could not be read). + */ + const reloadHost = () => load('host').then(() => true, () => { hostRead = 'failed'; notify(); return false; }); + + /** Save the dirty domains in order; a Save pressed while one runs joins it. */ + function save() { + saving ??= (async () => { + for (const domain of DRAFT_DOMAINS) failedKeys[domain].clear(); + const result = {}; + for (const domain of DRAFT_DOMAINS) result[domain] = await saveDomain(domain); + // Settings changed elsewhere: read them (and their revision), keeping the drafts (Q10). + if (result.host === 'conflict') await reloadHost(); + if (result.browser === 'conflict') await load('browser').catch(() => {}); + return result; + })().then(result => { + saving = null; + outcome = result; + notify(); + return result; + }); + return saving; + } + + /** + * A value the control could not take (System S3: a timeout outside 0–300): no draft, the control is highlighted and + * the status says a change was not saved. + */ + function reject(domain, key) { + requireDomain(domain); + drafts[domain].delete(key); + failedKeys[domain].add(key); + outcome = { [domain]: 'failed' }; + notify(); + } + + /** + * Another page changed saved values this tab has drafts for (a browser `storage` event, System S3): the drafts stay + * and are highlighted, and the status says the settings changed elsewhere. + */ + function conflict(domain, keys) { + requireDomain(domain); + if (!keys.length) return; + for (const key of keys) failedKeys[domain].add(key); + outcome = { [domain]: 'conflict' }; + notify(); + } + + return { + load, + reloadHost, + /** Whether the host values were read: `pending`, `loaded`, or `failed` (the last read failed). */ + hostState: () => hostRead, + reject, + conflict, + /** The saved values of a domain (its last read or save), not the drafts. */ + saved: domain => { requireDomain(domain); return { ...baseline[domain] }; }, + get, + set, + dirty, + discard, + save, + changes, + saving: () => saving !== null, + failed: domain => { requireDomain(domain); return new Set(failedKeys[domain]); }, + /** The Save row's text: the last save's outcome until the next edit, else unsaved or clean. */ + statusText: () => (outcome ? statusText(outcome) : dirty() ? SETTINGS_STATUS.dirty : SETTINGS_STATUS.clean), + subscribe(listener) { + listeners.add(listener); + return () => listeners.delete(listener); + }, + }; +} diff --git a/web/js/workspace/system.mjs b/web/js/workspace/system.mjs new file mode 100644 index 00000000..27e1a54a --- /dev/null +++ b/web/js/workspace/system.mjs @@ -0,0 +1,177 @@ +/** + * Native System (#249 System S2 Task 5): the concept's `renderSystem` shell (app.js 698–706) at `#/system/
`. + * Under the page heading it renders the section nav (`.section-nav`, one button per section, the shown one + * `aria-pressed`; a press navigates to its hash), then `.settings-layout` with the section's body and the Save row + * (`span.small#settings-status` + "Save preferences"). D92: no "Sample configuration" chip, the status copy is + * truthful and "Save preferences" is aria-disabled while nothing is unsaved. A section not delivered yet shows the + * per-section hand-off to its legacy tool (D94), never mock placeholders. Overview (system-overview.mjs) and + * Appearance (system-appearance.mjs) are native. + * + * Switching sections keeps the shell (and focus on the pressed button) and disposes the old section's scope, so its + * listeners and timers never outlive it. The drafts are module-level (Q10): they live for the page session. + */ +import { createButton, createScope } from '../shared/toolkit/core.js'; +import { SYSTEM_SECTIONS, navigateTo } from './state.mjs'; +import { createSystemDrafts, DRAFT_DOMAINS } from './system-state.mjs'; +import { sectionNav } from './system-controls.mjs'; +import { healthNotice, mountOverview } from './system-overview.mjs'; +import { mountAppearance } from './system-appearance.mjs'; +import { displayPreferences, readPreferences, reconcilePreferences, subscribePreferences, writePreferences } from './preferences.mjs'; + +const SETTINGS_URL = '/prompt_manager/settings'; +const ADMIN = Object.freeze({ href: '/prompt_manager/admin', label: 'Open existing administration' }); +const GALLERY = Object.freeze({ href: '/prompt_manager/gallery', label: 'Open existing image gallery' }); +/** + * D94: where each section's capabilities live until its PR (the plan's legacy capability coverage): the admin page's + * dashboard, Settings, Maintenance, Backup, Restore, Scan, Logs, Diagnostics and Auto-tag tools, and the gallery's + * settings, thumbnails, duplicates and cache tools. + */ +export const SECTION_HANDOFFS = Object.freeze({ + sources: [ADMIN, GALLERY], + integrations: [ADMIN], + tagging: [ADMIN], + maintenance: [ADMIN, GALLERY], + backups: [ADMIN], + diagnostics: [ADMIN], +}); +export const NOT_DELIVERED = 'This setting is not available here yet.'; + +let drafts = null; + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +/** A domain whose settings no delivered section edits yet: nothing can be drafted, and a write would say it failed. */ +const undelivered = async () => { throw new Error(NOT_DELIVERED); }; + +/** + * The page session's drafts (Q10). The browser domain is this browser's display defaults (preferences.mjs): a write + * says whether storage kept it, so the status never claims "Saved in this browser." for a change only this tab holds. + * The host domain reads and saves the server settings with their revision (#220). + */ +export function systemDrafts(api) { + drafts ??= createSystemDrafts({ + readBrowser: () => readPreferences().values, + readRuntime: () => ({}), + writeBrowser: changes => { + // Another page changed a drafted value since this tab read it, and no storage event told it yet: refuse, + // so the store keeps the draft, flags it and reads the values again; the next Save is an explicit choice. + const baseline = drafts.saved('browser'); + const { conflicts } = reconcilePreferences({ baseline, drafts: changes, next: readPreferences().values }); + if (conflicts.length) throw Object.assign(new Error('Settings changed elsewhere'), { code: 'settings-changed' }); + return { persisted: writePreferences(changes, undefined, { baseline }).persisted }; + }, + writeRuntime: undelivered, + async readHost() { + const result = await api.get(SETTINGS_URL, { cache: 'no-store' }); + // The store reads a missing value as none and a missing revision as unreadable (no host save). + return { values: result.settings, revision: result.revision }; + }, + writeHost: (changes, { expected_revision }) => api.post(SETTINGS_URL, { ...changes, expected_revision }), + }); + return drafts; +} + +/** D94: the shared hand-off panel, linking each legacy tool that still owns this section. */ +function handoff(section) { + const panel = el('section', undefined, 'surface handoff'); + panel.append(el('h2', 'Your current tools are ready'), + el('p', 'This workspace is being introduced in stages. Continue in the existing tool with your saved data.')); + for (const tool of SECTION_HANDOFFS[section]) { + const link = el('a', tool.label, 'workspace-link'); + link.href = window.ApiClient.resolveUrl(tool.href); + panel.append(link); + } + return [panel]; +} + +/** D92: highlight the controls of the settings the last save could not save. */ +function markFailed(layout, store) { + const failed = new Set(DRAFT_DOMAINS.flatMap(domain => [...store.failed(domain)])); + for (const control of layout.querySelectorAll('[data-setting]')) { + // A row's control names its stored key when that differs from the row id (Appearance's `data-key`). + if (failed.has(control.dataset.key ?? control.dataset.setting)) control.setAttribute('aria-invalid', 'true'); + else control.removeAttribute('aria-invalid'); + } +} + +/** + * The sections delivered so far: Overview (S2 Task 6), Appearance (S3 Task 8), and Maintenance's notice with "Run health checks" above the + * hand-off to the maintenance tools that are not native yet (D123). + */ +export const DELIVERED_SECTIONS = Object.freeze({ + overview: mountOverview, + appearance: mountAppearance, + maintenance: (container, { api, activity, scope }) => [healthNotice({ api, activity, scope }), ...handoff('maintenance')], +}); + +/** + * Mount System under the heading `renderHeading` wrote in `main`. Returns `{show(section)}`, which shows another + * section in place. `scope` is the page's view scope; `sections` maps a section id to a delivered section's mount + * (`(container, {api, activity, scope, drafts}) => nodes`), and every other section hands off. `activity` is the page's + * Activity store, which follows the jobs a section starts. + * + * The host values are read on mount (and by the store again after a conflict), so a save carries the current revision; + * until a read returned one, a host save fails without a request (system-state.mjs). The browser values are read on + * mount and again whenever another page changes them (a `storage` event). Either way a value another page changed + * under one of this tab's drafts flags that draft (D92's conflict status), including a change made while System was + * not mounted (the drafts outlive it, Q10). + */ +export function mountSystem(main, api, scope, { section, activity, sections = DELIVERED_SECTIONS, navigate = navigateTo }) { + const store = systemDrafts(api); + store.reloadHost(); + // Another tab or the legacy gallery changed the display defaults: clean values follow it; a draft it overtook stays + // and is flagged (D92's conflict status). `next` is null when the values cannot be read (no gallery-settings.js). + const reconcile = next => { + const baseline = store.saved('browser'); + // The first mount has read nothing yet, so nothing can have been overtaken. + const conflicts = next && Object.keys(baseline).length + ? reconcilePreferences({ baseline, drafts: store.changes('browser'), next }).conflicts : []; + store.load('browser').then(() => store.conflict('browser', conflicts), () => {}); + }; + reconcile(displayPreferences()); + scope.cleanup(subscribePreferences(window, next => reconcile(next.values))); + const nav = sectionNav({ label: 'System sections', sections: SYSTEM_SECTIONS, active: section, scope, + onSelect: id => navigate(`#/system/${id}`) }); + const layout = el('div', undefined, 'settings-layout'); + const saveRow = el('div', undefined, 'save-row'); + const status = el('span', undefined, 'small'); + status.id = 'settings-status'; + const save = createButton('Save preferences', 'ca-primary'); + saveRow.append(status, save); + layout.append(saveRow); + main.append(nav, layout); + + const paintStatus = () => { + status.textContent = store.statusText(); + if (store.dirty() && !store.saving()) save.removeAttribute('aria-disabled'); + else save.setAttribute('aria-disabled', 'true'); + markFailed(layout, store); + }; + scope.cleanup(store.subscribe(paintStatus)); + scope.listen(save, 'click', () => { + if (save.getAttribute('aria-disabled') === 'true') return; + const saving = store.save(); + paintStatus(); + saving.then(() => { if (!scope.signal.aborted) paintStatus(); }); + }); + + let sectionScope = null; + scope.cleanup(() => sectionScope?.dispose()); + function show(next) { + sectionScope?.dispose(); + sectionScope = createScope(); + for (const button of nav.children) button.setAttribute('aria-pressed', String(button.dataset.value === next)); + while (layout.firstChild !== saveRow) layout.firstChild.remove(); + const mount = sections[next]; + const nodes = mount ? mount(layout, { api, activity, scope: sectionScope, drafts: store }) : handoff(next); + layout.prepend(...nodes); + paintStatus(); + } + show(section); + return { show }; +} diff --git a/web/js/workspace/workspace-dialogs.mjs b/web/js/workspace/workspace-dialogs.mjs index 39b35e61..da53853e 100644 --- a/web/js/workspace/workspace-dialogs.mjs +++ b/web/js/workspace/workspace-dialogs.mjs @@ -11,7 +11,8 @@ * Rescan; no "Import portable recipes" (D15). Rescan shows its running state (aria-disabled "Rescanning…"). D55: the * concept's own "Scan an output folder → Review scan" row follows as "Scan for metadata". * - "Review metadata scan" (a1428–1432 `scan-review`, D55): Sources and Scope in a `dl.meta-grid`, the concept's - * sentence, a backup `.notice` ("Back up now") and Cancel / Start scan; no sample-plan note. + * sentence, a backup `.notice` ("Back up now") and Cancel / Start scan; no sample-plan note. It is one caller of + * `openReviewDialog`, the review every record-changing System operation shares, with its `backupNotice`. * - "Copy prompt" (a1223–1241): the select-to-copy fallback when the clipboard is unavailable (D22). * - One-value input (a1138 `simpleInput`): a "Category"-style field (`input#action-input`) and Cancel / a primary * action; used by the Images selection bar's "Set category" (#249 D8) and the toolbar's Save view. @@ -367,6 +368,98 @@ const BACKUP_BUSY_LABEL = 'Backing up…'; export const SCAN_SCOPE_LABELS = { new: 'New and unresolved files', all: 'All files' }; export const SCAN_ALL_FILES_HINT = 'Also read the files already linked to their prompt again.'; +/** The scan review's backup box copy (D55). */ +export const SCAN_BACKUP_COPY = Object.freeze({ heading: 'Back up the database first (recommended)', + text: 'Downloads a copy of the prompts database before the scan changes any records.' }); + +/** + * The D55 backup box: the concept's System `.notice` (`div` > `h3` + `p`, then `.row-actions` with "Back up now"). + * `onBackup` runs the backup and toasts its outcome; until it settles the button reads "Backing up…" and is + * aria-disabled, so a second press is ignored. It never closes the dialog it sits in. + */ +export function backupNotice({ copy, onBackup }) { + if (typeof copy?.heading !== 'string' || !copy.heading || typeof copy.text !== 'string' || !copy.text) { + throw new TypeError('The backup box needs its heading and text'); + } + requireCallbacks({ onBackup }); + const notice = el('div', undefined, 'notice'); + const text = el('div'); + text.append(el('h3', copy.heading), el('p', copy.text)); + const actions = el('div', undefined, 'row-actions'); + const backup = createButton(BACKUP_LABEL); + actions.append(backup); + notice.append(text, actions); + backup.addEventListener('click', async () => { + if (backup.getAttribute('aria-disabled') === 'true') return; + backup.setAttribute('aria-disabled', 'true'); + backup.textContent = BACKUP_BUSY_LABEL; + try { + await onBackup(); + } finally { + backup.removeAttribute('aria-disabled'); + backup.textContent = BACKUP_LABEL; + } + }); + return notice; +} + +function requireGrid(grid) { + if (!Array.isArray(grid) || grid.length === 0 || grid.some(row => !Array.isArray(row) || row.length !== 2 || + typeof row[0] !== 'string' || !row[0] || typeof row[1] !== 'string')) { + throw new TypeError('The review needs its [term, value] grid'); + } +} + +/** + * The shared record-changing review (the concept's `scan-review`, maintenance plan and restore preview): an optional + * `h3` heading, the `dl.meta-grid` of `[term, value]` rows, page-built `options` nodes, the optional `p.small` + * sentence, the D55 backup box when `backup` is `{copy, onBackup}`, then `.dialog-actions` with `cancelLabel` and the + * primary (`ca-primary`, or the concept's `danger` when `primary.danger`). The primary closes the dialog, then calls + * `onStart()`. Returns `{setValue(term, text)}` for a grid value that follows an option. + */ +export function openReviewDialog({ scope, title, heading = null, grid, options = [], sentence = null, backup = null, + cancelLabel, primary, onStart, trigger = null }) { + requireScope(scope); + requireCallbacks({ onStart }); + for (const [name, text] of Object.entries({ title, cancelLabel, label: primary?.label })) { + if (typeof text !== 'string' || !text) throw new TypeError(`The review needs its ${name}`); + } + requireGrid(grid); + if (!Array.isArray(options) || options.some(node => !node || typeof node !== 'object')) { + throw new TypeError('The review options must be nodes'); + } + const nodes = []; + if (heading) nodes.push(el('h3', heading)); + const list = el('dl', undefined, 'meta-grid'); + const values = new Map(); + for (const [term, value] of grid) { + const entry = el('div'); + const dd = el('dd', value); + values.set(term, dd); + entry.append(el('dt', term), dd); + list.append(entry); + } + nodes.push(list, ...options); + if (sentence) nodes.push(el('p', sentence, 'small')); + if (backup) nodes.push(backupNotice(backup)); + const cancel = createButton(cancelLabel); + const start = createButton(primary.label, primary.danger ? 'danger' : 'ca-primary'); + cancel.addEventListener('click', () => finish()); + start.addEventListener('click', () => { + finish(); + onStart(); + }); + const actions = el('div', undefined, 'dialog-actions'); + actions.append(cancel, start); + nodes.push(actions); + show(title, nodes, { scope, trigger }); + return { + setValue(term, text) { + const dd = values.get(term); + if (dd) dd.textContent = text; + }, + }; +} /** * The concept's "Review metadata scan" (`scan-review`, D55). `sources` is the number of output folders the scan reads. @@ -379,18 +472,9 @@ export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigge requireScope(scope); requireCallbacks({ onBackup, onStart }); if (!Number.isSafeInteger(sources) || sources < 0) throw new TypeError('The scan review needs its source count'); - const grid = el('dl', undefined, 'meta-grid'); - const values = {}; - for (const [term, value] of [['Sources', `${sources.toLocaleString('en-US')} output ${sources === 1 ? 'folder' : 'folders'}`], - ['Scope', SCAN_SCOPE_LABELS.new]]) { - const entry = el('div'); - values[term] = el('dd', value); - entry.append(el('dt', term), values[term]); - grid.append(entry); - } // The concept's System toggle: a `.setting-row` whose checkbox is the Toolkit switch (concept `hydrateSwitches`). const toggle = createSwitch('', '', 'scan-all-files'); - const allFiles = toggle.querySelector('input'); + const allFiles = toggle.children[0]; allFiles.checked = false; const option = el('div', undefined, 'setting-row'); const optionText = el('div'); @@ -402,35 +486,10 @@ export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigge optionText.append(optionLabel, hint); option.append(optionText, toggle); const chosenScope = () => (allFiles.checked ? 'all' : 'new'); - allFiles.addEventListener('change', () => { values.Scope.textContent = SCAN_SCOPE_LABELS[chosenScope()]; }); - const notice = el('div', undefined, 'notice'); - const copy = el('div'); - copy.append(el('h3', 'Back up the database first (recommended)'), - el('p', 'Downloads a copy of the prompts database before the scan changes any records.')); - const backupActions = el('div', undefined, 'row-actions'); - const backup = createButton(BACKUP_LABEL); - backupActions.append(backup); - notice.append(copy, backupActions); - backup.addEventListener('click', async () => { - if (backup.getAttribute('aria-disabled') === 'true') return; - backup.setAttribute('aria-disabled', 'true'); - backup.textContent = BACKUP_BUSY_LABEL; - try { - await onBackup(); - } finally { - backup.removeAttribute('aria-disabled'); - backup.textContent = BACKUP_LABEL; - } - }); - const cancel = createButton('Cancel'); - const start = createButton('Start scan', 'ca-primary'); - cancel.addEventListener('click', () => finish()); - start.addEventListener('click', () => { - const files = chosenScope(); - finish(); - onStart({ scope: files }); - }); - const actions = el('div', undefined, 'dialog-actions'); - actions.append(cancel, start); - show('Review metadata scan', [grid, option, el('p', SCAN_REVIEW_SENTENCE, 'small'), notice, actions], { scope, trigger }); + const review = openReviewDialog({ scope, trigger, title: 'Review metadata scan', + grid: [['Sources', `${sources.toLocaleString('en-US')} output ${sources === 1 ? 'folder' : 'folders'}`], + ['Scope', SCAN_SCOPE_LABELS.new]], + options: [option], sentence: SCAN_REVIEW_SENTENCE, backup: { copy: SCAN_BACKUP_COPY, onBackup }, + cancelLabel: 'Cancel', primary: { label: 'Start scan' }, onStart: () => onStart({ scope: chosenScope() }) }); + allFiles.addEventListener('change', () => review.setValue('Scope', SCAN_SCOPE_LABELS[chosenScope()])); } diff --git a/web/js/workspace/workspace.css b/web/js/workspace/workspace.css index 71173760..ace96b00 100644 --- a/web/js/workspace/workspace.css +++ b/web/js/workspace/workspace.css @@ -82,6 +82,19 @@ main:focus-visible { outline: 2px solid var(--ca-focus); outline-offset: 4px; } .tool-card p { flex: 1; margin: 0 0 12px; } .workspace-link { display: inline-flex; align-items: center; min-height: 44px; padding: 8px 0; font-size: 12px; overflow-wrap: anywhere; } .handoff .workspace-link { margin-top: 12px; } +/* D94: a System section not delivered yet links every legacy tool that still owns it, side by side. */ +.handoff .workspace-link + .workspace-link { margin-inline-start: 18px; /* D94 */ } + +/* D92: System's Save row is aria-disabled while nothing is unsaved (or a save runs), and a setting the last save could + not save is highlighted, as its "Review the highlighted settings" status says. */ +.save-row button[aria-disabled="true"] { opacity: .55; cursor: not-allowed; /* D92 */ } +.setting-row [aria-invalid="true"] { outline: 2px solid var(--red); outline-offset: 2px; /* D92 */ } +/* D93: an Overview row whose action is not available yet (the review queue, Q6) keeps its button, disabled. */ +.operation .row-actions button[aria-disabled="true"] { opacity: .55; cursor: not-allowed; /* D93 */ } +/* D97: Appearance's legacy display rows. "Show image details on cards" off keeps a card's Info and Tag at the end of + its row; "Collapse viewer details by default" gives the viewer's stage the whole width while its details are hidden. */ +.media-info > div[hidden] + .image-card-actions { margin-inline-start: auto; /* D97 */ } +.viewer-layout:has(> #viewer-details[hidden]) { grid-template-columns: minmax(0, 1fr); /* D97 */ } @media (max-width: 700px) { .stat-grid { gap: 8px; } .stat { padding: 16px; }