Compare commits

..
Author SHA1 Message Date
Vito Sansevero 82ec8782b4 fix(browser): server shutdown and its tests on Windows, Python < 3.13 and CI
Three CI failures on ed809f7:

- Windows: server.py exited 1 on stdin EOF because create_app works with
  the temp root as its cwd, and Windows cannot remove a process's cwd.
  main() now restores the original cwd before the root is removed.
- Python 3.10 to 3.12: the lifetime test's cleanup called communicate()
  after closing stdin, which flushes the closed pipe (ValueError); 3.13
  tolerates it. It now waits and closes the pipes itself, and the exit
  assertion shows the server's stderr. Under 3.12: errors=2 before, OK
  after.
- Tooling (CI container): the SIGKILLed-worker tests treated an exited
  server as alive, because the container's PID 1 does not reap orphans
  and kill(pid, 0) succeeds on a zombie. Reproduced locally with docker
  without --init (2 failed); a zombie now counts as exited (2 passed).
2026-10-21 22:57:03 -07:00
Vito Sansevero 14cfb8aedf docs(testing): server lifetime, stderr attachments and the final measured CI times
TESTING.md describes the stdin-EOF shutdown and the server-stderr
attachment, states that the npm cache key names the job's image tag
(now tested), quotes run 39725176320 at cbc8a61 (8m38s) and explains the
20-minute browser job limit. The workflow timing comment quotes the same
run.
2026-10-21 22:37:49 -07:00
Vito Sansevero ed809f7fa4 test(browser): server stderr on failures, no orphan servers, waits on state
Final review fixes for the per-worker test servers (#274):

- A failed test gets the stderr its server wrote during that test
  attached as `server-stderr`, so an aiohttp traceback behind a 500
  reaches the CI report. The launcher keeps the newest 256 KB with a
  stream position per test (stderrMark/stderrSince), and the fixture
  attaches it only when the test did not end as expected.
- The port announcement counts only as a complete line; a chunk ending
  mid-number was read as a shorter port.
- server.py --exit-on-stdin-eof shuts down gracefully (database closed,
  temp root removed) when stdin closes. The launcher holds that pipe,
  so a SIGKILLed worker leaves no server behind, on every OS.
- settled() gives up after 600 frames with the observation stats
  instead of hanging to the test timeout.
- The Classic text-sync test waits for the page's 50 ms timer to
  install serializeValue instead of sleeping 60 ms. With that timer run
  100 ms late (a scratch copy), the old wait failed 20/20 and the poll
  passed 20/20; the spec passed 240/240 at 8 workers x10 and 240/240
  at 1 CPU with 4 workers.
- The workflow test checks each npm cache key names its job's pinned
  image tag.

Every new unit test failed before its change (missing export, the old
regex timing out on the split announcement, an unknown server flag, a
hung settle, a mismatched cache key).
2026-10-21 22:37:48 -07:00
Vito Sansevero cbc8a61b1d test(browser): stable under parallel workers
Document how to run the browser suites in parallel and sharded, with the
measured times, and what keeps specs stable beside each other: wait for
a rendering frame, a response or a visible state, never for time.

With b665aa6 the full workflows suite passed on chromium, touch, firefox
and webkit with 8 workers, each test 3 times (6132 passed, 0 failed,
24 minutes), and once at CI size (docker --cpus=4, 3 workers: 2044
passed, 25.5 minutes).

The npm download cache removed the npm ci stall: 0 to 2 s in every job
of run 39722172529, against about 300 s in most jobs before. The
workflow's timing comment now records that run; the whole workflow took
8m54s against about 33 minutes before sharding. Shard imbalance (shard 1
carries the concept-comparison specs) is documented, not tuned.
2026-10-21 21:55:21 -07:00
Vito Sansevero b665aa687d test(browser): wait for rendering frames, not timers or luck, under load
Two Library workflows failed in CI's webkit shard 2/2 (run 39714840809)
under parallel load:

- "library infinite loading ..." counted cards right after the last
  page hid "Load more". That render queues the window update for the
  next frame, so a slow frame left the 120 cards of the all-rows
  layout in place (received 120, expected < 80). It now waits for the
  next frame, which runs after every callback the page queued.
- "row measurement preserves anchor ..." settled with a 60 ms timer. A
  scroll event is delivered in a rendering frame, so on a starved
  renderer the timer ended before the scroll's frame and no card was in
  view (null anchor). settleLibrary now waits for two consecutive
  native frames with no Library frame queued, checked inside the frame
  callback.

Before/after: a scratch copy that delays every frame by 120 ms (as a
starved runner) failed the old helpers 19 of 40 runs, at the CI lines,
and passed the new ones 40 of 40 on chromium and webkit. Throttled to
1 CPU with 4 workers, the original spec failed 1 of 42 for each test.

The Tags viewer workflow closed the viewer as soon as its image was
visible, before ViewerJS rendered the title, so CI's coverage never ran
the page's title callback (tags-page.js functions 26/46 = 56.52%, one
function above the floor of 56; locally 27/46). The test now asserts the
title names the shown file, which always runs the callback.
2026-10-21 21:22:52 -07:00
Vito Sansevero f85b45d9a8 ci(viewerjs): cache npm downloads and size timeouts from the first sharded run
The first sharded run lost a webkit shard to its 12-minute limit after all
267 tests passed, and npm ci stalled for 5 minutes in six of eight jobs. Each
npm ci now restores npm's download cache keyed on the vendor lockfile, and
browser job timeouts are the measured times plus at least 50%, never under 20
minutes. The workflow comment records the measurements.
2026-10-21 20:03:13 -07:00
Vito Sansevero 21a2cc0932 ci(viewerjs): run browser suites with workers and shards
Frontend coverage runs as three shards (PM_TEST_WORKERS=3 each) that upload
their raw V8 files as coverage-raw-N; a merge job downloads them, fails unless
all three arrived with raw files and no clashing names, then runs
coverage:merge. Firefox and WebKit page workflows run as two shards each with
three workers. The required viewerjs context keeps its name and now needs the
shards, the merge and the browser shards, failing unless every one succeeded.
2026-10-21 19:41:54 -07:00
Vito Sansevero 3ebf5c48cf ci(coverage): shard and merge modes for the frontend coverage run
frontend-coverage.mjs now plans its steps from the command line.
No arguments still runs everything, then reports and checks.
--shard I/N runs one slice of the chromium+touch workflows into
coverage/js/raw without wiping it or reporting; shard 1 also runs the
node units and the viewer suite. --merge-only reports and checks over raw
coverage the shards wrote, with --update-baseline as before, and refuses
an empty or missing raw dir. npm scripts: test:coverage:shard and
coverage:merge.
2026-10-21 19:34:08 -07:00
Vito Sansevero 0156d3fadb test(browser): test servers bind an OS-chosen port themselves
tests/browser/server.py accepts --port 0: it binds 127.0.0.1:0 before
building the app, holds that socket for its whole life and prints
PM_TEST_SERVER_PORT=<n> to stdout. startBrowserServer defaults to port 0
and reads the announced port, so no port is reserved and released before
the server binds and two launchers can no longer race for one. The
"address already in use" retry and free-port.mjs are gone; --port N still
works and is checked against the announcement.
2026-10-21 18:53:35 -07:00
Vito Sansevero 357c0f09e7 test(browser): one test server per Playwright worker
The page workflows started one global webServer and ran with one worker.
Each worker now owns its own tests/browser/server.py on an OS-chosen port,
with its own temp root and database, through a shared `test` in
tests/browser/fixtures.mjs. baseURL, and so page, request and
browser.newContext, point at that worker's server. A server that dies
fails the running test with its stderr tail and the next test gets a
fresh one. The worker count comes from PM_TEST_WORKERS, else 1 under CI,
else a quarter of the CPUs up to 8.

Chromium workflows in the pinned image: 510 tests in 606 s with one
worker on main, 511 (one new isolation spec) in 119 s with eight.
2026-10-21 18:30:22 -07:00
Vito 2cecfcf0ac feat(activity): scan new and unresolved files by default, and download a job report (#273)
* feat(activity): scan new and unresolved files by default, all files on request

Scan for metadata now skips, before reading any metadata, the files the
Library already links with a resolved link (one generated_images row with
exactly one pair link), found with one streamed query that keeps only the
scanned paths and compares paths with os.path.normcase on both sides.
Skipped files count as Already known ("Skipped · already linked").
Unlinked, fallback-linked, ambiguous and unresolved files are read again.

POST /prompt_manager/jobs/scan takes {scope: "new" | "all"} (default
"new", anything else is a 400). The review dialog shows the concept's
"New and unresolved files" and adds an "All files" switch (D58). The
legacy /prompt_manager/scan route keeps scanning all files with unchanged
SSE payloads.

* feat(activity): download a job report

GET /prompt_manager/jobs/{id}/report returns the summary, counters, every
outcome group's listed rows with truncated flags, the scope and the log as a
JSON attachment (prompt-manager-<kind>-<YYYYMMDD-HHMM>.json), with no absolute
paths. The job view's Download report saves it through the api-client URL, and
the scan log opens at its newest line and follows new lines only while the
reader is at the bottom.

* fix(activity): re-read known files whose size changed, and review follow-ups

The new scope now skips a known, resolved file only while its stored
generated_images.file_size equals its size on disk; a NULL stored size is
re-read. A file replaced at the same path is read again and its conflict
shows under Needs review. The scan stores each file's size when it ingests
it, and both sides of the known-set match go through normalize_image_path.

A failed known-set read logs and falls back to reading every file, and an
empty request body asks for the default scope. The All files switch is
described by its hint, a lost job's Download report is disabled, the report
toast is skipped once the view is gone, and the scan log opens at its
newest line every time. Fixtures use the real scope shape.

* test(activity): wait for the job view's settled detail read, not a fixed delay

The 200-row cap test now starts counting detail reads once one read has
returned the full listing and none is in flight. The job view re-reads after
an in-flight read only when the summary still lists more than that read
returned, so no redundant read is left due after opening.
2026-10-21 18:07:16 -07:00
Vito a6f6697c50 feat(activity): shared job registry, Activity page and Scan for metadata (#269)
* feat(activity): shared session job registry with the scan as its first kind

The output scan now runs as the first kind of an in-memory JobRegistry
(py/api/jobs.py, replacing py/api/scan_job.py). Each job keeps exact
counters, at most 200 outcome rows per group and a log capped at 2,000
lines, with root-relative paths and public root labels only. New routes
list, inspect, follow (SSE) and cancel jobs and start the scan
(POST /prompt_manager/jobs/scan, 409 busy while another record-changing
job runs). The legacy /prompt_manager/scan routes attach to the same job
and stream unchanged payloads.

* feat(activity): concept Activity page, top-bar button and progress pill

The workspace gains the concept's Activity page (renderActivity), its
top-bar button and sidebar row, fed by one session job store: a /jobs
read, one SSE stream per running job, a 1-30 s capped backoff after a
dropped stream and a 15 s list poll while visible. A job the server no
longer lists shows as ended, so no pill stays stuck after a restart.

D52: the counts show running jobs and hide at 0; the phone top-bar row
may wrap. D53: the mockup's progress pill links the running job. D54:
real job items with state chips, detail lines and Cancel job. Start and
finish toasts follow the spec copy. Fake browser jobs come from
tests/browser/workspace_activity_fixture.py.

* feat(activity): Scan for metadata with its review dialog and job view

Images -> Import keeps Rescan and adds the concept's "Review scan" row as
"Scan for metadata". It opens the concept's "Review metadata scan" with the
real source count, the fixed "All files" scope, a backup box that downloads
the existing database backup, and Cancel / Start scan. Start scan starts the
session scan job and opens its view; a start refused by another
record-changing job toasts its title as a link.

#/activity/<id> is now the approved job-view mockup: the heading with
Download report (disabled until Activity-2) and Cancel scan, the Progress
surface with the five counter tiles, the Results surface with the outcome
list, path filter and capped rows (Added and Already known open the viewer
through the listing identity), and the collapsed Scan log. Activity item
titles link their views. The scan's progress line groups thousands.

Deviations D55-D57 are recorded in TESTING.md.

* fix(activity): stop the job store on pagehide and let proxied routes outlive teardown

CI's touch coverage run failed "a resolver failure for the ComfyUI link never
stops the shell from mounting" with "route.fulfill: Fetch response has been
disposed": its trace shows two Library thumbnail requests started by the last
route still in the spec's proxy-mount handler while the page fixture was torn
down.

- The proxy-mount handler ignores only that teardown error, and the workspace
  and Activity specs unroute with ignoreErrors before teardown.
- The Activity store now ends its work on pagehide: the poll and backoff timers
  stop, the job streams close and a list read in flight is aborted (an older
  read is also aborted when a newer one starts). A pageshow from the
  back-forward cache reads the list again.

* test(activity): compare the job view log's line height to the hundredth of a pixel

WebKit reports the mockup's 11px/1.7 line height as 18.700001px, so the job view comparison failed on WebKit only. The measurement now rounds it, as other engine-dependent values are normalized.

* fix(activity): final review round for the job store, job view and scan rows

- Store: one failed list read no longer marks running jobs as ended; two in
  a row, or a good list without them, do (M1). A good read resets the
  backoff, except for a stream that keeps dropping while the list reads fine
  (M2). Finished jobs the server pruned are dropped (M3). Numbers group as
  en-US, like the server's lines (M13).
- Job view: a job the store's list no longer has shows the not-found state
  (M3). Focus moves to the job heading when Cancel scan goes away, and to the
  item's title link in the list (M4). The detail is read only when its listed
  rows change or the open log grows, using the summary's new listed and
  log_length (M12).
- Viewer rows open by exact identity: each scan row records its root's
  listing index, the source must still carry the row's label, and the
  workspace listing gains an exact `path` filter (M8, M9).
- Scan rows name the prompt by its Library title, stored title first (M10).
- A toast with a link stays 20 s, and any toast stays while hovered or
  focused (M14).
- A real two-batch scan cancelled after batch 1 keeps partial counts (M7).
- TESTING.md: D54, D56 and D57 updated (M5, M6).

* test(activity): cover the held and linked toast and a job view pruned by the server

Unit tests for the toast's 6.5 s and 20 s timeouts and its hover and focus hold, and a browser test for a finished job's open view after the server stops listing it. Coverage floors are only raised.
2026-10-21 14:22:30 -07:00
Vito 6e9d8f81fd ci(coverage): fast check that every frontend file has a coverage baseline (#271)
The frontend coverage job only reports a missing baseline entry after its
full browser run. coverage:baseline-check compares the files the gate
measures (same discoverSources) with frontend-coverage-baseline.json in
milliseconds and also flags entries for files that no longer exist. npm test
runs the same check.
2026-10-21 13:08:38 -07:00
Vito 454ac12e1d fix(images): list sources and folders that load after Refine opened (#272)
* fix(images): list sources and folders that load after Refine opened

"Refine this view" copied the discovered sources and folders when it
opened, so opening Filters before /gallery/subfolders answered left
Source and Folder at "All sources" / "All output folders" until the
dialog was reopened. On a slow CI runner this stalled the touch
"collection viewer follows filtered loaded order beyond mounted cards"
workflow in selectOption('0') until the 30s timeout.

openRefineDialog now returns setOptions(key, options); the Images page
calls it when discovery settles, and the open dialog refills Source and
its Folder list, showing the value it opened with (or the one chosen
since) once offered. It does nothing after the dialog closed or was
replaced.

* test(library): read filtered shelf titles after the cards' frame

The Library concept comparison read card titles as soon as the caption
changed and the card count dropped below six. Production empties the
shelf when a filter applies, writes the new caption when the page
answers and places the cards in the next animation frame, so on a slow
CI runner the empty shelf satisfied "fewer than six" and the titles
read as []. Settle one frame pair before reading, as the Images and
Inspector comparisons already do.
2026-10-21 12:36:38 -07:00
Vito 27b4895795 fix(resolver): identical sampler branches resolve to their shared prompt pair (#270)
* fix(resolver): identical sampler branches resolve to their shared prompt pair

A LoRA epoch-compare grid runs one sampler per epoch, every one fed by the
same positive encode and the same ConditioningZeroOut negative, and saves
one concatenated image. `_resolve` returned "Multiple sampler branches"
(ambiguous) whenever more than one sampler was found, even though every
branch used the same pair, so the workspace inspector fell back to the
legacy parser and the image could not be paired.

When every branch resolves to identical texts and stored recipes, the
result is now that pair: state resolved, no sampler named (none alone made
the output), reason "Every sampler branch uses the same prompt pair", the
first branch's sides and provenance, and every branch kept in `candidates`
as evidence. A branch that differs, cannot be read, or is not resolved
keeps the result ambiguous. Both profiles share the rule; the default
profile still cannot read ConditioningZeroOut, so the zero-out grid stays
ambiguous for ingestion and keeps the scan's legacy-parser fallback.

Workspace facts: a resolved result without a named sampler over several
listed samplers merges facts per field, as an ambiguous one does (the
grid's shared seed/steps are known, the per-epoch model is ambiguous).

Tests that encoded the old behavior (identical branches => ambiguous):
- test_prompt_pair_resolver: test_multiple_samplers_are_ambiguous_even_when_texts_match
  -> test_multiple_samplers_with_matching_texts_resolve_to_the_shared_pair
  (plus a new differing-texts case that stays ambiguous);
  test_raw_and_detailed_saves_stay_ambiguous_without_an_anchor (same prompt)
  -> ..._of_one_prompt_share_it_without_an_anchor (anchored cases unchanged).
- test_pair_scan: two samplers sharing one prompt now ingest the pair
  ("cat"/"blur") instead of the positive-only fallback.
- test_api_prompt_inspector, test_inspection_outcomes, test_pair_image_linking:
  their "ambiguous" fixtures now use samplers with different prompts.
No parity vector changes (f01/f09 branches are not all resolved).

* fix(inspector): never show the positive as a zeroed negative from the legacy fill

When the server leaves the negative undecided, the workspace fills it from
the legacy browser parser (comfy-metadata.js), which follows
ConditioningZeroOut back to its input and so reads the positive text as
the negative. The workspace now checks the prompt graph first:

- every sampler's negative is a ConditioningZeroOut (Reroutes allowed):
  the negative shows the server's explicitly empty zeroed state
  ({status: resolved, text: "", source: zeroed-conditioning});
- a zero-out elsewhere in a negative path (one branch zeroed, another
  encoded; a zero-out combined with other conditioning), or a
  workflow-only file holding one: the negative is not filled and keeps
  the server's status;
- graphs without a zero-out keep the legacy fill unchanged.

comfy-metadata.js is untouched; the legacy pages keep their reading.

* fix(images): explain why Set category has nothing to set

Set category on a selection where no image is linked to a Library prompt
said only "None of the selected images has a linked prompt." It now says
why and what it means: "The selected images aren't in your Library yet,
so there is no prompt category to set." Unit and page-workflow tests
follow the copy.

* fix(inspector): check only the negative the legacy parse read for a zero-out

Review I1/M2/M3/M5 of the zero-out legacy-fill guard. The guard decided
over every node with a `negative` input, so a graph whose read sampler
encodes "blurry" lost that legacy-filled negative when another branch
zeroed its own (fewer than the legacy pages: below the parity floor).

- comfy-metadata.js gains `legacyNegativeSource`, which returns the graph
  and sampler `negative` link that `extractGenerationParams` reads from.
  It reuses the parser's own sampler pick (`selectSampler`, factored out
  of `fromPromptGraph` without changing any reading). The workspace
  checks only that link: a direct zero-out shows the zeroed empty
  negative, a zero-out further up the path leaves it unfilled, and
  anything else keeps the legacy fill.
- Workflow-only files go through the parser's own workflow conversion,
  so bypassed (mode 4) and muted (mode 2) zero-outs and unconnected ones
  behave as the legacy reading does. A subgraph instance whose definition
  runs a zero-out (directly or nested) leaves the negative unfilled.
- One bounded walk with an index pointer replaces a walk per negative
  input with queue.shift().
- An ambiguous fact that the legacy parse knows (a grid's per-branch
  model) is still filled, documented: the legacy pages show it and this
  page never shows less.

Tests: the reversed mixed graph shows "blurry"; the mixed graph whose
read sampler is zeroed, the zero-out-only grid and a direct workflow
zero-out show the zeroed empty negative; a combined zero-out and a
zero-out subgraph instance stay unfilled; bypassed and unconnected
workflow zero-outs keep the legacy reading. An API-graph Reroute in
front of a zero-out gives the legacy parse no negative, so the server
answer stays (that expectation changed from zeroed).

* fix(images): name the missing link in the Set category toast

Review M4: say that no selected image is linked to a saved prompt, in
the same terms as the partial-success toast and right for one image.

Also review M1: a monitor test that two parallel samplers sharing one
positive and one negative write that pair (the resolver collapse).
2026-10-21 10:55:35 -07:00
75 changed files with 7155 additions and 395 deletions
+132 -22
View File
@@ -16,9 +16,19 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
# Timeouts are the measured job time plus at least 50%, and never under 20 minutes for a browser job,
# because runner speed varies. Before sharding (main 2cecfcf, one worker): frontend coverage 32m10s,
# page workflows firefox 18m59s, webkit 19m26s. First sharded run (PR #274 run 39712909047, 3 workers,
# no npm cache, where npm ci stalled for 5m in six of eight jobs): coverage shards 11m19s/10m47s/10m49s
# (tests 5m20s/4m42s/4m46s), merge 5m52s (report and check 4s), firefox shards 7m53s/9m27s (tests
# 4m41s/3m29s), webkit shards 12m25s+ (cancelled at the old 12-minute limit after its 267 tests passed in
# 6m29s) and 4m53s (tests 3m08s). With the npm cache hit (run 39725176320 at cbc8a61, npm ci 0-2s):
# coverage shards 7m34s/6m04s/5m22s (tests 6m38s/4m53s/4m04s), merge 54s, firefox 7m57s/5m21s (tests
# 7m08s/4m34s), webkit 8m16s/5m03s (tests 7m26s/3m58s); a cache miss adds up to 5m to npm ci, which
# the 20-minute limits absorb.
jobs:
frontend-coverage:
name: frontend coverage (chromium and touch)
name: frontend coverage (shard ${{ matrix.shard }}/3)
runs-on: ubuntu-latest
# Layout comparisons depend on the fonts each browser falls back to, so browser jobs run in the
# same pinned Playwright image as local runs (tag = @playwright/test in tools/vendor/package-lock.json;
@@ -26,12 +36,15 @@ jobs:
container:
image: mcr.microsoft.com/playwright:v1.64.1-noble
options: --user 1001
# This job runs every page workflow twice (chromium, then touch) on one worker with V8 coverage on, so it
# takes about twice a single-browser job. Measured 2026-10-20: whole job 18m26s-24m57s (main 5ded3cf
# 22m15s; PR head 6359999 was cancelled at the old 25-minute limit while still passing), coverage step
# alone 20m55s-22m17s+, against ~12-15 minutes for the firefox/webkit jobs. 40 minutes leaves headroom
# for suite growth while still stopping a hung run; re-measure if a run passes 30 minutes.
timeout-minutes: 40
timeout-minutes: 20
# Public-repo ubuntu-latest runners have 4 vCPUs: three Playwright workers, each with its own test
# server, leave one CPU for the servers and the runner.
env:
PM_TEST_WORKERS: 3
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3]
steps:
- uses: actions/checkout@v7
# The image's own Python 3.12 and Node 24: setup-python/setup-node install the runner's toolcache
@@ -50,24 +63,107 @@ jobs:
"$venv/bin/python" -m pip install --no-cache-dir -r requirements-dev.txt
echo "$venv/bin" >> "$GITHUB_PATH"
- run: python --version && node --version && node --version | grep -q '^v24\.'
# npm's download cache, keyed on the lockfile; inside the container ~ is the job's HOME (/github/home),
# which npm also uses. node_modules itself is still built fresh by npm ci.
- name: Cache npm downloads
uses: actions/cache@v6
with:
path: ~/.npm
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
- run: npm ci --prefix tools/vendor --ignore-scripts
- run: npm run --prefix tools/vendor check
- run: npm run --prefix tools/vendor test
- name: Frontend unit, viewer, workflow, and coverage checks
run: npm run --prefix tools/vendor test:coverage
- name: Upload frontend coverage and Chromium traces
# Shard 1 also runs the node units and the viewer suite, so the vendor pins and the tooling tests
# run on the lightest shard instead.
- if: matrix.shard == 3
run: npm run --prefix tools/vendor check
- if: matrix.shard == 3
run: npm run --prefix tools/vendor test
- name: Page workflows with coverage, shard ${{ matrix.shard }}/3
run: npm run --prefix tools/vendor test:coverage:shard -- ${{ matrix.shard }}/3
- name: Upload raw coverage
uses: actions/upload-artifact@v7
with:
name: coverage-raw-${{ matrix.shard }}
path: coverage/js/raw/
if-no-files-found: error
retention-days: 3
- name: Upload Chromium and touch traces
if: always()
uses: actions/upload-artifact@v7
with:
name: frontend-traces-shard-${{ matrix.shard }}
path: tools/vendor/test-results/
retention-days: 14
frontend-coverage-merge:
name: frontend coverage (merge)
needs: frontend-coverage
runs-on: ubuntu-latest
# Same image as the shards: the raw files name sources by the container's workspace path.
container:
image: mcr.microsoft.com/playwright:v1.64.1-noble
options: --user 1001
timeout-minutes: 10
steps:
- uses: actions/checkout@v7
- run: node --version && node --version | grep -q '^v24\.'
# npm's download cache, keyed on the lockfile; inside the container ~ is the job's HOME (/github/home),
# which npm also uses. node_modules itself is still built fresh by npm ci.
- name: Cache npm downloads
uses: actions/cache@v6
with:
path: ~/.npm
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
- run: npm ci --prefix tools/vendor --ignore-scripts
# Each artifact lands in its own coverage-raw-N directory, so the next step can count them.
- uses: actions/download-artifact@v8
with:
pattern: coverage-raw-*
path: ${{ runner.temp }}/coverage-raw
# coverage:merge only checks that raw files exist; merging two of three shards would report
# partial coverage, so a missing or empty shard artifact fails here.
- name: Verify every shard's raw coverage arrived
shell: bash
env:
STAGED_RAW: ${{ runner.temp }}/coverage-raw
EXPECTED_SHARDS: 3
run: |
found=0
for shard in $(seq 1 "$EXPECTED_SHARDS"); do
directory="$STAGED_RAW/coverage-raw-$shard"
count=0
if [ -d "$directory" ]; then count=$(find "$directory" -maxdepth 1 -type f -name '*.json' | wc -l); fi
echo "coverage-raw-$shard: $count raw files"
if [ "$count" -gt 0 ]; then found=$((found + 1)); fi
done
echo "shard artifacts with raw coverage: $found of $EXPECTED_SHARDS"
if [ "$found" -ne "$EXPECTED_SHARDS" ]; then
echo "::error::Expected raw coverage from $EXPECTED_SHARDS shards, got $found of $EXPECTED_SHARDS"
exit 1
fi
staged=$(find "$STAGED_RAW" -mindepth 2 -maxdepth 2 -type f -name '*.json' | wc -l)
mkdir -p coverage/js/raw
# cp -n never overwrites; a clash shows up as fewer merged files than staged ones.
find "$STAGED_RAW" -mindepth 2 -maxdepth 2 -type f -name '*.json' -exec cp -n -t coverage/js/raw {} + || true
merged=$(find coverage/js/raw -maxdepth 1 -type f -name '*.json' | wc -l)
if [ "$merged" -ne "$staged" ]; then
echo "::error::Shards wrote clashing raw file names: $staged staged, $merged merged"
exit 1
fi
echo "merged $merged raw files"
- name: Frontend coverage report and baseline check
run: npm run --prefix tools/vendor coverage:merge
- name: Upload frontend coverage report
if: always()
uses: actions/upload-artifact@v7
with:
name: frontend-test-reports
path: |
tools/vendor/test-results/
coverage/js/
!coverage/js/raw/**
retention-days: 14
browser-workflows:
name: page workflows (${{ matrix.browser }})
name: page workflows (${{ matrix.browser }}, shard ${{ matrix.shard }}/2)
runs-on: ubuntu-latest
# Layout comparisons depend on the fonts each browser falls back to, so browser jobs run in the
# same pinned Playwright image as local runs (tag = @playwright/test in tools/vendor/package-lock.json;
@@ -75,11 +171,16 @@ jobs:
container:
image: mcr.microsoft.com/playwright:v1.64.1-noble
options: --user 1001
timeout-minutes: 25
timeout-minutes: 20
# Public-repo ubuntu-latest runners have 4 vCPUs: three Playwright workers, each with its own test
# server, leave one CPU for the servers and the runner.
env:
PM_TEST_WORKERS: 3
strategy:
fail-fast: false
matrix:
browser: [firefox, webkit]
shard: [1, 2]
steps:
- uses: actions/checkout@v7
# The image's own Python 3.12 and Node 24: setup-python/setup-node install the runner's toolcache
@@ -98,22 +199,29 @@ jobs:
"$venv/bin/python" -m pip install --no-cache-dir -r requirements-dev.txt
echo "$venv/bin" >> "$GITHUB_PATH"
- run: python --version && node --version && node --version | grep -q '^v24\.'
# npm's download cache, keyed on the lockfile; inside the container ~ is the job's HOME (/github/home),
# which npm also uses. node_modules itself is still built fresh by npm ci.
- name: Cache npm downloads
uses: actions/cache@v6
with:
path: ~/.npm
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
- run: npm ci --prefix tools/vendor --ignore-scripts
- name: Page workflows in ${{ matrix.browser }}
run: npm run --prefix tools/vendor test:workflows -- --project ${{ matrix.browser }}
- name: Page workflows in ${{ matrix.browser }}, shard ${{ matrix.shard }}/2
run: npm run --prefix tools/vendor test:workflows -- --project ${{ matrix.browser }} --shard ${{ matrix.shard }}/2
- name: Upload browser traces
if: always()
uses: actions/upload-artifact@v7
with:
name: browser-traces-${{ matrix.browser }}
name: browser-traces-${{ matrix.browser }}-shard-${{ matrix.shard }}
path: tools/vendor/test-results/
retention-days: 14
# Branch protection requires the exact context "viewerjs". This job keeps that
# name and fails unless every frontend job above succeeded, including when one
# was cancelled or skipped.
# name and fails unless every frontend job above succeeded (every shard, the
# coverage merge and every browser shard), including when one was cancelled or skipped.
viewerjs:
needs: [frontend-coverage, browser-workflows]
needs: [frontend-coverage, frontend-coverage-merge, browser-workflows]
if: always()
runs-on: ubuntu-latest
timeout-minutes: 5
@@ -121,8 +229,10 @@ jobs:
- name: Require every frontend job to succeed
env:
COVERAGE_RESULT: ${{ needs.frontend-coverage.result }}
MERGE_RESULT: ${{ needs.frontend-coverage-merge.result }}
WORKFLOWS_RESULT: ${{ needs.browser-workflows.result }}
run: |
echo "frontend-coverage: $COVERAGE_RESULT"
echo "frontend-coverage-merge: $MERGE_RESULT"
echo "browser-workflows: $WORKFLOWS_RESULT"
test "$COVERAGE_RESULT" = success && test "$WORKFLOWS_RESULT" = success
test "$COVERAGE_RESULT" = success && test "$MERGE_RESULT" = success && test "$WORKFLOWS_RESULT" = success
+274 -17
View File
@@ -39,9 +39,44 @@ npm run --prefix tools/vendor test:coverage
npm run --prefix tools/vendor test:workflows -- --project firefox --project webkit
```
`PM_TEST_PYTHON` selects the Python executable; `PM_TEST_PORT` selects the local
port (default 8765). Set these environment variables when Python is outside PATH
or the port is occupied. Test servers are never reused. On unsupported Linux
`PM_TEST_PYTHON` selects the Python executable; set it when Python is outside
PATH. Every Playwright worker starts its own `tests/browser/server.py --port 0`,
which binds a port the OS picks and prints it as `PM_TEST_SERVER_PORT=<n>`, with
its own temp root and database (`tests/browser/fixtures.mjs`), so
parallel workers and concurrent runs never share state. `PM_TEST_WORKERS` sets the
worker count (default 1 under `CI`, otherwise a quarter of the CPUs, at most 8).
Test servers are never reused across runs. The launcher holds each server's
stdin open and the server (`--exit-on-stdin-eof`) shuts down cleanly, removing its
temp root, when that pipe closes, so a worker killed without teardown leaves no
server behind. A test that fails gets the stderr its server wrote during the test
attached to the report as `server-stderr` (an aiohttp traceback behind a 500).
The coverage run can be split the way CI splits it, and each shard keeps the raw
V8 files it writes under `coverage/js/raw`:
```sh
PM_TEST_WORKERS=8 npm run --prefix tools/vendor test:coverage # whole run, one report
npm run --prefix tools/vendor test:coverage:shard -- 1/3 # node units, viewer suite, workflows 1/3
npm run --prefix tools/vendor test:coverage:shard -- 2/3 # workflows 2/3 (likewise 3/3)
npm run --prefix tools/vendor coverage:merge # report and baseline check of every raw file
npm run --prefix tools/vendor test:workflows -- --project webkit --shard 2/2
```
`coverage:merge` refuses an empty raw directory; it does not count shards, so run
every shard first (CI's merge job checks all three arrived). Measured on a 32-CPU
machine in the pinned image (2026-10-21): `test:coverage` takes about 5 minutes
with 8 workers and 24 minutes with 1; all four projects, 8 workers, each test 3
times (6132 tests), 24 minutes. A CI-sized run (`docker run --cpus=4`,
`PM_TEST_WORKERS=3`, all four projects once, 2044 tests) takes 25.5 minutes.
Specs must pass beside each other, not only alone. Wait for the page's own
signal, never for time: a rendering frame (`requestAnimationFrame`) for layout
work queued by a render or a scroll, a response or a visible state for data. A
timer that "usually" covers a frame is a race on a loaded runner. Library
virtualization specs settle with `libraryObservation.settled()` (two consecutive
frames with no Library frame queued). To look for such races, repeat a spec under
load (`--workers 8 --repeat-each 10`, and in the image with `--cpus=1` or `2`).
On unsupported Linux
distributions, browser system dependencies may need manual installation. For a
quick browser run, use `test:workflows -- --project chromium`.
@@ -109,14 +144,44 @@ the baseline updater. Raise floors with `npm run --prefix tools/vendor
coverage:update`; it reruns the suites and cannot lower an existing floor.
Review baseline changes like code changes.
`npm run --prefix tools/vendor coverage:baseline-check` runs in well under a
second without a browser and fails, naming the files, when a measured
`web/js` file has no baseline entry or the baseline lists a file that no longer
exists. It uses the gate's own `discoverSources`, and `npm test` runs the same
check, so run it before pushing a new or deleted frontend file instead of
waiting for the `frontend coverage` jobs.
The `viewerjs` check runs on every PR, including backend-only changes. It is an
aggregate job that passes only when the `frontend coverage (chromium and touch)`
job and both `page workflows (firefox)` and `page workflows (webkit)` jobs
succeed; the browsers run on separate runners in parallel. GitHub artifacts
named `frontend-test-reports` contain HTML coverage, a JSON summary, and the
Chromium/touch failure screenshots/traces; `browser-traces-firefox` and
`browser-traces-webkit` hold the other browsers' traces. Open traces with
Playwright's `show-trace` tool.
aggregate job that passes only when every frontend job succeeds: the three
`frontend coverage (shard N/3)` jobs, `frontend coverage (merge)`, and the four
`page workflows (firefox|webkit, shard N/2)` jobs. A failed, cancelled or skipped
job fails it. Every browser job runs with `PM_TEST_WORKERS: 3` (the public
runners have 4 vCPUs). Each coverage shard runs `test:coverage:shard -- N/3`
(shard 1 also runs the node units and the viewer suite, shard 3 the vendor pins
and tooling tests) and uploads its raw V8 files as `coverage-raw-N` (kept 3
days). The merge job downloads them, fails unless all three arrived with raw
files and without clashing names, then runs `coverage:merge` (report plus
baseline check). `tests/vendor/ci-workflow.test.mjs` checks this shape. GitHub
artifacts named `frontend-test-reports` contain the merged HTML coverage and
JSON summary; `frontend-traces-shard-N` hold the Chromium/touch failure
screenshots/traces and `browser-traces-<browser>-shard-N` the other browsers'
traces. Open traces with Playwright's `show-trace` tool.
Every `npm ci` first restores npm's download cache (`~/.npm`), keyed on the vendor
lockfile and the job's pinned image tag (`tests/vendor/ci-workflow.test.mjs` checks
the key names the same tag as the job's `container.image`). Without it, `npm ci` stalled for about 300 s
(npm's fetch timeout) in most of the eight concurrent jobs; with a hit it takes
0 to 2 s. A lockfile change misses the cache once, so expect one slow run after it.
Measured on PR #274 (run 39725176320 at cbc8a61, cache hit): the whole workflow
took 8m38s, against about 33 minutes with one worker and no shards. Jobs: coverage
shards 7m34s/6m04s/5m22s (tests 6m38s/4m53s/4m04s), merge 54s, firefox shards
7m57s/5m21s (tests 7m08s/4m34s), webkit shards 8m16s/5m03s (tests 7m26s/3m58s).
Browser jobs keep a 20-minute limit: the slowest job is about 8.5 minutes, a cache
miss adds up to 5 to `npm ci`, and runner speed varies (the first sharded run was
cancelled at a 12-minute limit).
Playwright shards by test count, whole files at a time, so shard 1 carries the
slow concept-comparison specs (`*-concept.spec.mjs`, alphabetically first) and
runs longer; the remedy, if it starts to matter, is more shards, not reordering.
The real-host suite covers both Classic and Nodes 2.0 startup, switching both
ways, actual database prompt selection, execution-text serialization, workflow
@@ -430,10 +495,11 @@ the first nav link and heading button.
Recorded shell deviations:
- **D4**: a "Theme" select with Dark, Light and System. System follows the ComfyUI
palette, then the OS.
- **D5**: no Activity button or Activity row until an activity center exists.
- **D5**: retired by Activity (#249 Activity-1). The concept's Activity button and
Activity row are shown; their differences are D52 to D54.
- **D28**: no COLLECTIONS, and no Design review. The bottom nav is
System plus the production Overview hub (its own outline home glyph, not the concept's
`activity` pulse, and no count).
the concept's Activity and System, plus the production Overview hub (its own
outline home glyph, not the concept's `activity` pulse, and no count).
The connection reads "Your library, on your server" with a neutral muted dot.
- **D49** (saved views, #249 D16): the SAVED VIEWS group label has no "PROPOSED"
badge, because saved views are persisted (`/prompt_manager/saved-views`). Its rows are
@@ -484,6 +550,196 @@ Recorded shell deviations:
as deleted. `tests/browser/workspace-saved-views.spec.mjs` covers the row, count,
route, star updates, Save view on Favorites, Edit, and the failure paths;
`tests/test_image_favorites.py` covers `favorites=1`.
- **D52** (Activity, #249 Activity-1): the top-bar `button#activity-open` and the
sidebar's Activity row (the concept's `activity` pulse, first in `system-nav`) count
this session's *running* jobs. The concept counts
every sample job. `span.count#activity-count` is `hidden` at 0 (its text stays "0"),
and the row has no `.nav-count` then. The Activity heading has no "Unified activity ·
proposed" chip (prototype copy). At `max-width: 700px` the full-width `.top-actions`
row may wrap (`flex-wrap: wrap`), so the button, Theme and ComfyUI stay inside a
zoomed phone. The shell comparison hides the concept's count at runtime
(`#activity-count { display: none }`), so the concept's own CSS lays out its button
without it, and maps the row's count to none, the button's count to "0" and the
phone row's `flexWrap` to `wrap`.
- **D53**: while a job runs, the approved Activity mockup's progress pill sits before
the Activity button on every page: `a#activity-pill.activity-pill`, "Scanning · 41% ·
8,240 / 19,622" (the verb by job kind, the server's floored percentage, and the file
counts once the total is known), a 70px mini bar, `href="#/activity/<id>"` and the same
text as its `aria-label`. With no running job it is `hidden`, which the shell
comparison maps as a hidden item before the button. At `max-width: 700px` it keeps the
verb and percentage only; on coarse pointers it is 40px tall like the concept's
buttons. Its rules are in `workspace.css` under D53.
- **D54**: each `.activity-item` is a real job of this session (`GET
/prompt_manager/jobs`), running first, then finished newest first. The concept's
structure and CSS stay: `.panel-heading` > `h3` "✓ <title>" for a completed job and
"○ <title>" otherwise, plus `span.chip`, then `p`, and for a running job `<progress
max=100>` ("<title> progress") and `.row-actions`. The chip is green "Completed" or
blue "Running", "Cancelled", "Failed" or "Ended". The line reads "Started HH:MM · K
output folders · closing this view does not cancel it" while running, "Finished HH:MM
· N added · M errors", "Cancelled HH:MM at a file boundary · N added · completed work
is kept", "Stopped on an error at HH:MM · N added · M errors", or "This job ended when
ComfyUI restarted." The running actions are "Cancel job" only (no "Simulate
completion"); a failed cancel toasts "The job could not be cancelled. Try again.". With
no job the section says "No jobs have run since ComfyUI started. Scans and other long
operations appear here." `#/activity/<id>` is that job's own view (D56). When a job
ends while its Cancel job has focus, focus moves to the item's title link.
The Activity store reads the list once, then follows each running job over
`/prompt_manager/jobs/<id>/events`. A dropped stream re-reads the list after 1 s,
doubling to at most 30 s; a received event resets the wait, and so does a good list
read after failed ones or after any read that was not for a dropped stream. A running
job the server no longer lists, or one whose server could not be read twice in a row,
is shown as ended, so no pill stays; one failed read (a proxy's 502) changes nothing.
A finished job the server no longer lists (it keeps 20) is dropped. Numbers are
grouped "en-US", as the server's lines are. While the tab is visible the list is read again every 15 s (and whenever
Activity opens), so a job started in another tab or on the legacy admin page appears
without a reload. Toasts: "Scan started." after this tab's own start; "Scan finished
· N added · M errors", "Scan cancelled · N added" or "Scan failed · N added · M
errors" when a job this tab saw running ends (plural helper for "error").
`tests/js/workspace-activity-store.test.js` covers the store, its backoff and the
copy; `tests/browser/workspace-activity.spec.mjs` covers the list against the
concept's own items (`renderActivity` and `startJob`), the pill and counts, a phone,
reload and a second tab, Cancel job, a restart and an unreachable server, and the
toasts, with fake jobs from `tests/browser/workspace_activity_fixture.py`
(`/__test__/jobs/*`, including a simulated restart that drops the job streams).
- **D55** (Scan for metadata, Activity-1 Task 3): Images → Import keeps Rescan as the
quick refresh (D15) and then shows the concept's own "Scan an output folder → Review
scan" row as "Scan for metadata" ("Discover and enrich existing files using the
configured sources.", button "Review scan"). It opens the concept's `scan-review`,
"Review metadata scan": no sample-plan `.demo-note` (prototype copy); Sources and
Scope in a `dl.meta-grid` (the concept's maintenance plan uses `dl`; `scan-review`
uses a `div`) reading "N output folders" (the discovered sources) and the concept's
"New and unresolved files" (the D58 checkbox switches it to "All files"); the
concept's sentence; then
a backup `.notice` (the concept's System notice box): `h3` "Back up the database first
(recommended)", "Downloads a copy of the prompts database before the scan changes any
records." and "Back up now", which downloads the existing `/prompt_manager/backup`
copy (as the legacy admin page does), reads "Backing up…" while aria-disabled, keeps
the dialog open and toasts "Database backup downloaded." or "The database could not be
backed up. Try again."; then Cancel / Start scan (primary; the concept's "Simulate
scan"). Start scan closes the dialog, starts the session scan job (`POST
/prompt_manager/jobs/scan`), toasts "Scan started." and opens `#/activity/<id>`; a
failed start toasts "The scan could not start. Try again.". The scan's progress line
now groups thousands ("Processing file 8,240 of 19,622"), as the mockup does.
`tests/browser/activity-concept.spec.mjs` compares the dialog with the concept's
`scan-review` and its box with the concept's System `.notice` (dark and light,
Chromium and touch, with a CSS and a DOM mutation check); `images-concept.spec.mjs`
maps the Import row; `workspace-images.spec.mjs` covers the dialog, the backup and
Cancel.
- **D56**: one job's view at `#/activity/<id>`, the approved mockup
(`activity-scan.html`, "100% spot on"), which the concept does not have. The page
heading names the job (h1, document title "<title> · Prompt Manager"; the breadcrumb
stays "Activity"), its line is the D54 item line, and its actions are Download report
(disabled until Activity-2) and, while the job runs, "Cancel scan" (`.danger`; "Cancel
job" for another kind; a failed request toasts the D54 message). Then `section.surface`
"Progress": `.panel-heading` (h2, `span.small` with the server's "Processing file N of
M · about X min left", or after the end "N of M files processed", "Cancelled after N
of M files", the error line, or the restart line), `<progress max=100>` ("Scan
progress") and `dl.stat-row` tiles in order: Added to Library (`.added`, green),
Already known, Needs review (`.warn`, warning), No usable metadata, Errors (`.bad`,
red). Then `section.surface` "Results": `label.search-field` "Filter by file path…"
(`aria-label` "Filter results", case-folded match on the root label and path), and
`.outcomes` with `.outcome-list` (`button[aria-pressed]` label + count, in the
mockup's order Added, Needs review, Errors, No usable metadata, Already known; the
view opens on the first with files) beside `.outcome-files`: one `.file-row` (`.path`
root-relative, `.why` the server's detail) per listed file. The detail wording is the
builder's (approved with Task 1): "Added to the Library as “title”", "Linked to
“title”", "Already linked to “title”", "Needs review: …", "Error: …", where the title
is the prompt's Library title (its stored title, else its text's first line), not the
mockup's sample "→ “title” · new positive + negative". Each log line is "HH:MM:SS
<prefix> <root label>/<path> · <detail>" rather than the mockup's "[HH:MM:SS] + path →
prompt #4812 (new)". The header's actions sit in the page heading's
`.heading-actions`, where the mockup has a `.panel-heading > .row-actions`. An
empty outcome shows "No files in this group."
or "No listed file matches this filter.", and the note "Showing N of M" when fewer
than the counter are shown (the server lists 200 per outcome), with "each row opens
the image in the viewer" for Added and Already known. Then `details.term`, collapsed:
"● Scan log — raw detail, click to show" and `pre[tabindex=0]` with the raw lines.
The summary follows the store and carries `listed` (rows per outcome) and
`log_length`; the rows and log are read from `GET /prompt_manager/jobs/<id>` once on
opening, then at most once a second only while what it lists changes (or, with the
log open, its length), so a job whose lists are capped and whose log is closed is not
re-read; opening the log reads its new lines. When the job ends while Cancel scan has
focus, focus moves to the job's heading. An unknown id, or a job the store's list no
longer has, shows "This job is not in this session’s Activity. Jobs are kept until
ComfyUI restarts." and "All activity"; a job lost with a restart keeps what was read.
At `max-width: 700px` the tiles take two columns and the outcome list stacks above
the files. Each Activity item's title is now a link to its view, in the title's
colour. The rules are in `workspace.css` under D56. `activity-concept.spec.mjs`
restates the mockup's structure and CSS values and checks them with mutations;
`workspace-activity.spec.mjs` covers the counters, live updates, viewer rows, the
filter, the 200-row cap, Cancel scan, the busy start and an unknown or lost job.
- **D57**: rows under Added and Already known are `button.file-row`s that open the
existing media viewer on that one file, by the identity Images opens it by
(`root_index:path`). Each scan row records its root's listing index (`root_index`,
the place in `_get_all_output_dirs`) beside its label. The row's source is that index
in `/prompt_manager/gallery/subfolders?mode=workspace` and must still carry the row's
label, so a reordered or replaced source never opens another root's file; the
workspace listing's new `path` filter (one exact relative path, case-folded, at most
1,024 characters, no traversal) returns that file. A row without an index, a changed
source or a file the listing no longer has toasts "This file is no longer in Images.
Rescan from Import and try again."; other rows are plain text, with no new file
route. A start refused because another record-changing job runs toasts "Another job
is running: <title>." with the title linking that job's view (the shared toast now
accepts link parts). A toast with a link stays 20 s instead of 6.5 s, and any toast
stays while the pointer or focus is on it.
- **D58** (scan scope, Activity-2 Task 4): the concept's `scan-review` states a fixed
scope, "New and unresolved files". Production keeps that value as the default and adds
the concept's own System toggle after the `dl.meta-grid`: a `.setting-row` with the
label "All files", the hint "Also read the files already linked to their prompt
again." (the switch's `aria-describedby`) and the Toolkit switch (`label.ca-switch`, a
checkbox with `role="switch"`, as the concept's `hydrateSwitches` renders it),
unchecked. Checking it switches the Scope value to "All files"; Start scan sends
`POST /prompt_manager/jobs/scan {scope: "new" | "all"}` (no body, an empty body or no
`scope` is "new"; another value is a 400).
- In "new" the scan skips, before reading any metadata, the files the Library already
links with a resolved link and that are unchanged on disk: exactly one
`generated_images` row for the file, exactly one distinct pair link on it
(`image_prompt_pairs`, written only by resolved ingestion), and a stored
`file_size` equal to the file's size in this walk. Both paths go through
`normalize_image_path` (absolute, normalized, `os.path.normcase`); a row without
`file_path` matches by its `image_path`.
- Read again: files linked without a pair (the monitor's fallback and FIFO links, the
legacy positive-only import), linked to more than one prompt or pair, not linked, a
file whose size changed (one replaced at the same path, e.g. a reused ComfyUI
counter name, is read and its conflict lands under Needs review), and a row with no
stored size (unknown is not known). The scan now stores each file's size when it
ingests it, so a file read once is skipped by the next "new" pass; the first pass
after the upgrade fills the missing sizes, which reports those files as enriched
(Added) instead of unchanged.
- The known set is one streamed query that keeps only the scanned paths. If it cannot
be read, the scan logs "Known files could not be read · reading every file" and
reads every file.
- A skipped file counts as processed and unchanged in the legacy counts (not as
found) and under Already known, with the detail "Skipped · already linked" and the
log line "= <root>/<path> · Skipped · already linked"; the log's first line names
the scope ("Scan started · N output folders · new and unresolved files" or "· all
files"). A legacy-page scan that attaches to a running "new" job therefore shows
the skipped files in its `unchanged` count.
- "All files" reads every file, as the scan did in Activity-1, and the legacy
`/prompt_manager/scan` route always does when it starts the scan (its SSE payload
keys are unchanged).
- `activity-concept.spec.mjs` compares the switch with the concept's System → Sources
toggle ("Watch for new outputs"), without its place-dependent top margin;
`workspace-activity.spec.mjs` covers the Scope value, the description and the
request both ways; `tests/test_scan_scope.py` covers the skip, the size rule, the
replaced file, the re-read of unresolved files, "all", the legacy route, the
fallback, the empty body, the 400 and Windows case folding.
- **D59** (job report and log, Activity-2 Task 5): the job view's "Download report" is
enabled. It fetches `GET /prompt_manager/jobs/<id>/report` through the api-client URL
(so a proxy mount works) and saves a JSON attachment named
`prompt-manager-<kind>-<YYYYMMDD-HHMM>.json` (server local time), then toasts "Job
report downloaded." (or "The report could not be downloaded. Try again."). The report
holds the summary, every counter, each outcome group's exact `count`, its listed
`rows` (at most 200) and a `truncated` flag, the scope and the log; rows and log lines
carry root-relative paths only, never an absolute path (an unknown id is a 404).
A job lost with a restart has no report on the server, so its Download report is
disabled; a view left while the report downloads still saves it but shows no toast.
The scan log opens at its newest line every time it is opened and follows new lines
only while the reader is at the bottom; a reader who scrolled up keeps the position
while it stays open (checked on Chromium, Firefox and WebKit).
`test_report_is_attachment_without_absolute_paths` and the browser tests `download
report saves the job JSON` and `log keeps the reader's scroll position` cover it.
- **D29**: no prototype copy. There is no "Interactive concept" badge and no
"What changes & why" button. The footer reads "ComfyAssets · Local tools. Your
library.", and the title has no " concept" suffix. Save view's dialog action reads
@@ -570,7 +826,8 @@ Recorded deviations on this page (spec §8):
footer, masonry tops may differ by sub-pixel snapping (at most one layout unit per
card above in the column), and a List row's info takes its natural height, centred
beside the 160×95 art.
- **D5**, **D29**: no Activity button and no prototype copy (shell).
- **D29**: no prototype copy (shell). The Activity button and pill are the shell's
(D52, D53).
- **D7**: the star is a real persisted favorite with "Added to favorites." /
"Removed from favorites." and a rollback toast on failure.
- **D8**: every Images card has the concept's select checkbox before the star; the
@@ -1357,7 +1614,7 @@ bytes at the same path without changing the immutable receipt.
The focused acceptance command retains all four projects:
```sh
PM_TEST_PORT=8766 npm run --prefix tools/vendor test:workflows -- \
npm run --prefix tools/vendor test:workflows -- \
workspace-history.spec.mjs workspace-visual.spec.mjs \
--project chromium --project touch --project firefox --project webkit
npm run --prefix tools/vendor test:executions
@@ -1450,7 +1707,7 @@ acceptance; a baseline is added only after the maintainer accepts them.
Focused commands:
```sh
PM_TEST_PORT=8766 npm run --prefix tools/vendor test:workflows -- \
npm run --prefix tools/vendor test:workflows -- \
workspace-library-editor.spec.mjs library-editor-visual.spec.mjs \
--project chromium --project touch --project firefox --project webkit
npm run --prefix tools/vendor test:comfyui # library-sender.spec.mjs host regression
@@ -1550,7 +1807,7 @@ mobile, the concept geometry/token/control comparison and its mutation check),
```sh
node --test tests/js/workspace-vocabulary-state.test.js tests/js/workspace-state.test.js
PM_TEST_PORT=8766 npm run --prefix tools/vendor test:workflows -- \
npm run --prefix tools/vendor test:workflows -- \
workspace-vocabulary.spec.mjs vocabulary-visual.spec.mjs focused-node.spec.mjs \
--project chromium --project touch --project firefox --project webkit
```
+61
View File
@@ -181,6 +181,21 @@ AND p.id NOT IN (SELECT prompt_id FROM live_hint_prompts)
"""
# The scan's known set (#249 Activity-2): every image row, with its stored file
# size and whether its pair link is resolved: exactly one distinct pair, the rule
# _LINK_SUMMARY_SQL and get_image_prompt_info use. A file is known and resolved
# when it has exactly one image row, that row is resolved, and the row's stored
# size equals the file's size now; several rows (one file linked to more than
# one prompt) are ambiguous, and an unknown (NULL) stored size is not known. The
# rows are streamed in fetches of KNOWN_PATHS_FETCH_SIZE and only the scanned
# paths are kept, so memory follows the scan (at most MAX_SCAN_FILES paths), not
# the table.
_KNOWN_PATHS_SQL = """SELECT gi.file_path, gi.image_path, gi.file_size,
(SELECT COUNT(DISTINCT ip.pair_id) FROM image_prompt_pairs ip
WHERE ip.image_id = gi.id) = 1
FROM generated_images gi"""
KNOWN_PATHS_FETCH_SIZE = 1000
# One SELECT reads the link summaries of at most this many paths: three bound
# parameters per path stays under SQLite's historical 999-variable limit.
LINK_SUMMARY_BATCH = 200
@@ -2280,6 +2295,52 @@ class PromptDatabase(
"parameters": row[8] if len(row) > 8 else None,
}
def get_known_resolved_paths(self, files: Dict[str, Optional[int]]) -> Set[str]:
"""The paths in ``files`` the Library already links with a resolved link.
``files`` maps each scanned path to its size on disk (None when it could
not be read). A path qualifies when exactly one ``generated_images`` row
claims the file, that row has exactly one pair link
(``image_prompt_pairs``, written only by resolved ingestion), and its
stored ``file_size`` is not NULL and equals the given size, so a file
replaced at the same path is read again. Files linked without a pair
(the monitor's fallback and FIFO links, the legacy positive-only
import), linked to several prompts or pairs, or not linked at all do
not qualify. Both sides go through ``normalize_image_path`` (absolute,
normalized, ``os.path.normcase``), so a Windows row stored in another
case or spelling still matches. One query, streamed; returns the
matching keys of ``files``.
"""
wanted: Dict[str, List[Tuple[str, Optional[int]]]] = {}
for path, size in files.items():
key = normalize_image_path(str(path))
wanted.setdefault(key, []).append((str(path), size))
if not wanted:
return set()
rows_seen: Dict[str, int] = {}
stored_sizes: Dict[str, int] = {}
with self.model.get_connection() as conn:
cursor = conn.execute(_KNOWN_PATHS_SQL)
while True:
rows = cursor.fetchmany(KNOWN_PATHS_FETCH_SIZE)
if not rows:
break
for file_path, image_path, file_size, is_resolved in rows:
# image_path is NOT NULL; file_path is NULL only on old rows.
key = normalize_image_path(file_path or image_path)
if key not in wanted:
continue
rows_seen[key] = rows_seen.get(key, 0) + 1
if is_resolved and file_size is not None:
stored_sizes[key] = file_size
return {
path
for key, stored_size in stored_sizes.items()
if rows_seen[key] == 1
for path, size in wanted[key]
if size is not None and size == stored_size
}
def get_image_link_summaries(
self, paths: List[str], allow_filename_fallback: bool = True
) -> Dict[str, Dict[str, Any]]:
+10 -2
View File
@@ -6,6 +6,7 @@ Split into domain-specific mixins for maintainability:
- AdminRoutesMixin: duplicates, stats, settings, diagnostics, maintenance, backup, scan
- LoggingRoutesMixin: log management endpoints
- AutotagRoutesMixin: auto-tagging model management and tagging
- JobRoutesMixin: the session job registry behind Activity (py/api/jobs.py)
"""
import asyncio
@@ -29,6 +30,8 @@ from .saved_views import SavedViewRoutesMixin
from .history import HistoryRoutesMixin
from .images import ImageRoutesMixin
from .admin import AdminRoutesMixin
from .job_routes import JobRoutesMixin
from .jobs import JobRegistry
from .logging_routes import LoggingRoutesMixin
from .autotag_routes import AutotagRoutesMixin
from .lora_integration import LoraIntegrationMixin
@@ -254,6 +257,7 @@ class PromptManagerAPI(
PromptRoutesMixin,
ImageRoutesMixin,
AdminRoutesMixin,
JobRoutesMixin,
LoggingRoutesMixin,
AutotagRoutesMixin,
LoraIntegrationMixin,
@@ -299,9 +303,12 @@ class PromptManagerAPI(
# _acquire_long_job(). Created lazily so no lock binds to an event
# loop before the server's loop exists.
self._long_jobs: dict = {}
# The current output scan (py/api/scan_job.ScanJob), if one was started.
# This session's jobs (py/api/jobs.py): the metadata scan, and later
# every other long operation the Activity page tracks.
self._jobs = JobRegistry(self.logger)
# The latest metadata scan job (also in self._jobs), if one was started.
self._scan_job = None
# The current duplicate scan (also a ScanJob), if one was started.
# The current duplicate scan (a py/api/jobs.Job outside the registry).
self._duplicate_job = None
# Run cleanup on initialization to remove any existing duplicates
@@ -439,6 +446,7 @@ class PromptManagerAPI(
self._register_prompt_routes(routes)
self._register_image_routes(routes)
self._register_admin_routes(routes)
self._register_job_routes(routes)
self._register_logging_routes(routes)
self._register_autotag_routes(routes)
self._register_lora_routes(routes)
+305 -53
View File
@@ -6,16 +6,25 @@ from contextlib import closing
import datetime
import hashlib
import json
import math
import os
import sqlite3
import tempfile
import threading
import time
from pathlib import Path
from aiohttp import web
from .prompts import thumbnail_url_for
from .scan_job import ScanJob
from .jobs import (
DEFAULT_SCAN_SCOPE,
SCAN_GROUP_NAMES,
SCAN_GROUPS,
SCAN_SCOPES,
Job,
JobBusy,
)
try:
from ...utils.parallel import map_parallel
@@ -47,8 +56,10 @@ except ImportError:
try:
from ...database.library import display_title
from ...utils.hashing import generate_prompt_hash
except ImportError:
from database.library import display_title
from utils.hashing import generate_prompt_hash
IMAGE_EXTENSIONS = (".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff")
@@ -138,6 +149,127 @@ def _sse(payload):
return f"data: {json.dumps(payload)}\n\n"
SCAN_JOB_KIND = "scan"
SCAN_JOB_TITLE = "Scan for metadata"
# The legacy final event type of a scan and the job state it ends in.
SCAN_FINAL_STATES = {
"complete": "finished",
"cancelled": "cancelled",
"error": "failed",
}
# The scope as the scan log's first line names it (the review dialog's words).
SCAN_SCOPE_LOG = {"new": "new and unresolved files", "all": "all files"}
MIN_BATCHES_FOR_ESTIMATE = 2 # "about X min left" waits for this many batches
# Log line prefix per outcome group (spec: + added, = known, ? review, ! error,
# - no usable metadata).
SCAN_LOG_PREFIXES = {
"added": "+",
"known": "=",
"review": "?",
"errors": "!",
"no_metadata": "-",
}
# What an outcome row says about a file: where it went (with the prompt's
# title) or why it was skipped. "error" is a file whose ingestion raised;
# None is a file without prompt metadata.
SCAN_TITLED_DETAILS = {
"added": "Added to the Library as “{title}”",
"linked": "Linked to “{title}”",
"enriched": "Metadata added to “{title}”",
"unchanged": "Already linked to “{title}”",
}
SCAN_DETAILS = {
"added": "Added to the Library",
"linked": "Linked to an existing prompt",
"enriched": "Metadata added to its prompt",
"unchanged": "Already linked to its prompt",
"skipped": "Skipped · already linked",
"ambiguous": "Needs review: more than one prompt could match",
"unresolved": "Needs review: the prompt could not be traced in the workflow",
"conflict": "Needs review: already linked to a different prompt",
"failed": "Error: the prompt could not be stored",
"error": "Error: the file could not be processed (see the server log)",
"found": "Metadata found, but no prompt text to store",
None: "No prompt metadata in the file",
}
def _plural(count, word):
return f"{count} {word}" if count == 1 else f"{count} {word}s"
def _scan_status_line(done, total, batches, elapsed):
""" "Processing file N of M", with an estimate once enough batches ran."""
line = f"Processing file {done:,} of {total:,}"
if batches < MIN_BATCHES_FOR_ESTIMATE or done <= 0 or done >= total:
return line
minutes = max(1, math.ceil(elapsed / done * (total - done) / 60))
return f"{line} · about {minutes} min left"
def _file_size(path):
"""``path``'s size on disk, or None when it cannot be read."""
try:
return os.path.getsize(path)
except OSError:
return None
def _with_file_size(canonical, media_file):
"""Record the file's size in ``file_info`` unless the metadata carries one.
Stored as ``generated_images.file_size``; the "new" scan scope skips a known
file only while its size is unchanged.
"""
info = canonical.get("file_info")
info = dict(info) if isinstance(info, dict) else {}
if info.get("size") is None:
size = _file_size(media_file)
if size is not None:
info["size"] = size
if info:
canonical["file_info"] = info
def _scan_roots(output_dirs):
"""(absolute root, public label, listing index), the most specific first.
The index is the root's place in ``output_dirs`` (``_get_all_output_dirs``),
the ``root_index`` the workspace listing names it by.
"""
public_path, _ = _public_helpers()
roots = [
(os.path.abspath(str(d)), public_path(d), index)
for index, d in enumerate(output_dirs)
]
return sorted(roots, key=lambda root: len(root[0]), reverse=True)
def _scan_location(media_file, roots):
"""``media_file`` relative to its output root, that root's label and index."""
full = os.path.abspath(str(media_file))
folded = os.path.normcase(full)
for root, label, index in roots:
base = os.path.normcase(root).rstrip("\\/") + os.sep
if folded.startswith(base):
return Path(os.path.relpath(full, root)).as_posix(), label, index
return Path(full).name, "", None
def _scan_outcome(media_file, roots, status, title=None):
"""The (group, row, log line) one scanned file records in its job."""
if status in SCAN_GROUPS:
group = SCAN_GROUPS[status]
else:
group = "errors" if status == "error" else "no_metadata"
template = SCAN_TITLED_DETAILS.get(status) if title else None
detail = template.format(title=title) if template else SCAN_DETAILS[status]
path, label, index = _scan_location(media_file, roots)
row = {"path": path, "root": label, "root_index": index, "detail": detail}
where = f"{label}/{path}" if label else path
return group, row, f"{SCAN_LOG_PREFIXES[group]} {where} · {detail}"
def _inspect_restore_upload(path):
"""Checks on a restore upload beyond PromptModel.verify_database_file.
@@ -437,7 +569,7 @@ class AdminRoutesMixin:
guard = self._acquire_long_job(self.LONG_JOB_DUPLICATES)
if guard is None:
return self._long_job_busy_response("Duplicate scan or delete")
job = ScanJob()
job = Job("duplicates", "Duplicate scan")
self._duplicate_job = job
job.task = asyncio.ensure_future(self._run_duplicate_job(job, guard))
return await self._stream_scan_job(request, job)
@@ -1319,14 +1451,18 @@ class AdminRoutesMixin:
async def scan_images(self, request):
"""Scan ComfyUI output images for prompt metadata and add them to the database.
The scan runs as a background task that outlives this request, so a
closed tab never leaves a half-finished scan. Progress streams as
server-sent events; while a scan is running, a new request attaches
to it instead of starting another one.
The scan runs as the registry's ``scan`` job, which outlives this
request, so a closed tab never leaves a half-finished scan. Progress
streams as the legacy server-sent events; while a scan is running
(started here or from the workspace), a new request attaches to it
instead of starting another one.
"""
job = self._scan_job
if job is None or job.finished:
job = self._start_scan_job()
job = self._jobs.active(SCAN_JOB_KIND)
if job is None:
try:
job = self._start_scan_job("all")
except JobBusy as busy:
return self._long_job_busy_response(busy.running.title)
return await self._stream_scan_job(request, job)
async def cancel_scan(self, request):
@@ -1348,19 +1484,37 @@ class AdminRoutesMixin:
}
)
def _start_scan_job(self):
"""Create the job record synchronously, then run the scan as a task."""
job = ScanJob()
def _start_scan_job(self, files=DEFAULT_SCAN_SCOPE):
"""Register the scan job synchronously; its runner starts on the loop.
``files`` is the scope (``SCAN_SCOPES``). Raises JobBusy while another
record-changing job runs.
"""
if files not in SCAN_SCOPES:
raise ValueError(f"Unknown scan scope: {files!r}")
job = self._jobs.start(
SCAN_JOB_KIND,
SCAN_JOB_TITLE,
record_changing=True,
scope={"files": files},
runner=self._run_scan_job,
groups=SCAN_GROUP_NAMES,
)
self._scan_job = job
job.task = asyncio.ensure_future(self._run_scan_job(job))
return job
async def _run_scan_job(self, job):
try:
async for payload in self._scan_images_events():
job.publish(payload)
finally:
job.finish()
final = None
async for payload in self._scan_images_events(job):
job.publish(payload)
final = payload.get("type")
state = SCAN_FINAL_STATES.get(final, "finished")
if state == "failed":
job.log_line(f"Scan failed · {job.status_line}")
else:
verb = "cancelled" if state == "cancelled" else "finished"
job.log_line(f"Scan {verb} · {_plural(job.processed, 'file')} processed")
job.finish(state)
async def _stream_scan_job(self, request, job):
"""Stream ``job`` as SSE; stop quietly if the client goes away."""
@@ -1386,21 +1540,31 @@ class AdminRoutesMixin:
)
return response
async def _scan_images_events(self):
"""Yield progress payloads while scanning every configured output directory."""
async def _scan_images_events(self, job):
"""Yield progress payloads while scanning every configured output directory.
The payloads are the legacy SSE events, unchanged; ``job`` also gets
its status line, totals and one outcome per processed file.
"""
try:
self.logger.info("Starting image scan operation")
output_dirs = self._get_all_output_dirs()
if not output_dirs:
self.logger.error("No output directories found")
yield {
"type": "error",
"message": "No output directories found. "
"Configure scan directories in Settings.",
}
message = (
"No output directories found. "
"Configure scan directories in Settings."
)
job.update(status_line=message)
yield {"type": "error", "message": message}
return
dir_names = [self._public_path(d) for d in output_dirs]
folders = _plural(len(output_dirs), "output folder")
job.scope["output_folders"] = len(output_dirs)
scope = SCAN_SCOPE_LOG[job.scope.get("files", "all")]
job.log_line(f"Scan started · {folders} · {scope}")
job.update(status_line=f"Looking for media files in {folders}")
yield self._scan_progress(
0, f"Scanning {len(output_dirs)} directory(ies) for media files..."
)
@@ -1409,11 +1573,21 @@ class AdminRoutesMixin:
self._collect_output_media_sync, output_dirs
)
counts = dict.fromkeys(SCAN_COUNT_KEYS, 0)
job.update(
total=len(media_files),
status_line=_scan_status_line(0, len(media_files), 0, 0),
)
if media_files:
yield self._scan_progress(
5, f"Found {len(media_files)} media files to process...", counts
)
async for event in self._scan_batches(media_files, counts):
roots = _scan_roots(output_dirs)
known = set()
if media_files and job.scope.get("files") != "all":
known = await self._known_scan_files(job, media_files)
async for event in self._scan_batches(
media_files, counts, job, roots, known
):
yield event
self.logger.info(
@@ -1421,37 +1595,78 @@ class AdminRoutesMixin:
f"found={counts['found']}, new_prompts_added={counts['added']}, "
f"images_linked_to_existing={counts['linked']}"
)
job = self._scan_job
final_type = "cancelled" if job and job.cancelled else "complete"
final_type = "cancelled" if job.cancelled else "complete"
if final_type == "complete":
job.update(progress=100)
yield {"type": final_type, **counts, "directories": dir_names}
except Exception:
self.logger.exception("Scan error")
job.update(status_line="The scan stopped on an internal error.")
yield {
"type": "error",
"message": "An internal error occurred. Check server logs for details.",
}
async def _scan_batches(self, media_files, counts):
async def _known_scan_files(self, job, media_files):
"""The "new" scope's skip set; a failed read falls back to reading all."""
try:
return await self._run_in_executor(self._known_scan_files_sync, media_files)
except Exception:
self.logger.warning(
"Known files could not be read; the scan reads every file",
exc_info=True,
)
job.log_line("Known files could not be read · reading every file")
return set()
def _known_scan_files_sync(self, media_files):
"""Stat each scanned file and ask the database which are known (blocking)."""
sizes = {str(media_file): _file_size(media_file) for media_file in media_files}
return self.db.get_known_resolved_paths(sizes)
async def _scan_batches(self, media_files, counts, job, roots=(), known=()):
"""Process ``media_files`` in executor batches, updating ``counts`` in place.
Yields one progress event per batch.
Records each file's outcome in ``job`` and yields one progress event
per batch. Files in ``known`` (path strings) are not read: they count
as processed and unchanged and are recorded as skipped.
"""
total = len(media_files)
started = time.monotonic()
batches = 0
for batch_start in range(0, total, SCAN_BATCH_SIZE):
if self._scan_job and self._scan_job.cancelled:
if job.cancelled:
return
batch = media_files[batch_start : batch_start + SCAN_BATCH_SIZE]
batch_results = await self._run_in_executor(
self._extract_batch_metadata_sync, batch
)
batch_counts = await self._run_in_executor(
self._ingest_scan_batch_sync, batch_results
)
for key in counts:
counts[key] += batch_counts[key]
skipped = [f for f in batch if str(f) in known]
to_read = [f for f in batch if str(f) not in known]
outcomes = [_scan_outcome(f, roots, "skipped") for f in skipped]
counts["processed"] += len(skipped)
counts["unchanged"] += len(skipped)
if to_read:
batch_results = await self._run_in_executor(
self._extract_batch_metadata_sync, to_read
)
batch_counts = await self._run_in_executor(
self._ingest_scan_batch_sync,
batch_results,
roots=roots,
outcomes=outcomes,
)
for key in counts:
counts[key] += batch_counts[key]
for group, row, log_line in outcomes:
job.record(group, row, log_line)
batches += 1
done = min(batch_start + len(batch), total)
job.update(
processed=counts["processed"],
status_line=_scan_status_line(
done, total, batches, time.monotonic() - started
),
)
yield self._scan_progress(
int(done / total * 100), f"Processing file {done}/{total}...", counts
)
@@ -1490,9 +1705,19 @@ class AdminRoutesMixin:
return map_parallel(extract, file_batch, PromptManagerConfig.WORKER_THREADS)
def _ingest_scan_batch_sync(self, batch_results):
"""Store the prompts found in a metadata batch (blocking DB work)."""
def _ingest_scan_batch_sync(self, batch_results, roots=(), outcomes=None):
"""Store the prompts found in a metadata batch (blocking DB work).
Returns the legacy counts. When ``outcomes`` is a list, every processed
file appends its ``(group, row, log_line)`` for the scan job, with the
path relative to its root in ``roots`` (see ``_scan_roots``).
"""
counts = dict.fromkeys(SCAN_COUNT_KEYS, 0)
def note(media_file, status, title=None):
if outcomes is not None:
outcomes.append(_scan_outcome(media_file, roots, status, title))
for media_file, metadata in batch_results:
if self._scan_job and self._scan_job.cancelled:
break
@@ -1502,25 +1727,21 @@ class AdminRoutesMixin:
except Exception as e:
self.logger.error(f"Error processing {media_file.name}: {e}")
counts["failed"] += 1
note(media_file, "error")
continue
if outcome is None:
note(media_file, None)
continue
counts["found"] += 1
status = outcome["status"] if isinstance(outcome, dict) else outcome
if status in (
"added",
"linked",
"enriched",
"unchanged",
"ambiguous",
"unresolved",
"conflict",
"failed",
):
if status in SCAN_GROUPS:
counts[status] += 1
if isinstance(outcome, dict):
counts["pairs_added"] += outcome.get("pairs_added", 0)
counts["pair_links_added"] += outcome.get("links_added", 0)
note(media_file, status, outcome.get("title"))
else:
note(media_file, status)
return counts
@staticmethod
@@ -1539,7 +1760,13 @@ class AdminRoutesMixin:
return positive.strip() or None
def _ingest_scanned_file(self, media_file, metadata, detailed=False):
"""Enrich exact pairs; retain legacy positive-only import for absent pairs."""
"""Enrich exact pairs; retain legacy positive-only import for absent pairs.
Returns the file's scan status, or None without prompt metadata. With
``detailed``, a stored or linked file returns a dict instead: its
``status``, the prompt's display ``title`` and, for a resolved pair,
the pair counts.
"""
if not metadata:
return None
canonical = dict(metadata)
@@ -1552,12 +1779,18 @@ class AdminRoutesMixin:
pass
if not canonical.get("prompt") and not canonical.get("parameters"):
return None
_with_file_size(canonical, media_file)
resolution = resolve_prompt_pair(canonical)
if resolution["state"] == "resolved":
result = ingest_resolved_pair(
self.db, str(media_file), canonical, resolution
)
return result if detailed else result["status"]
if not detailed:
return result["status"]
return {
**result,
"title": self._scan_title(media_file, resolution["positive"]["text"]),
}
if canonical.get("prompt") and resolution["state"] in (
"ambiguous",
"unresolved",
@@ -1585,6 +1818,25 @@ class AdminRoutesMixin:
return "found"
prompt_text = prompt_text.strip()
status = self._store_scanned_prompt(media_file, canonical, prompt_text)
if not detailed or status not in SCAN_TITLED_DETAILS:
return status
return {"status": status, "title": self._scan_title(media_file, prompt_text)}
def _scan_title(self, media_file, text):
"""The linked prompt's Library title (its stored title, else ``text``'s
first line), as the Library and the inspector name it."""
try:
path = str(media_file)
summary = self.db.get_image_link_summaries([path], False).get(path)
if summary and summary.get("title"):
return summary["title"]
except Exception:
self.logger.warning("Scan row title could not be read; using its text")
return display_title((), text)
def _store_scanned_prompt(self, media_file, canonical, prompt_text):
"""Link ``media_file`` to ``prompt_text``'s prompt, saving it when new."""
prompt_hash = generate_prompt_hash(prompt_text)
existing = self.db.get_prompt_by_hash(prompt_hash)
if existing:
+9
View File
@@ -705,6 +705,8 @@ def _workspace_filters(query: Mapping) -> dict:
media = query.get("media", "all")
order = query.get("sort", "newest")
folder = query.get("subfolder", "").strip().replace("\\", "/")
# ``path``: one exact relative path (Activity opens a scan row by it, D57).
exact = query.get("path", "").strip().replace("\\", "/")
category = query.get("category", "").strip() or None
tags = sorted(
{tag.strip() for tag in query.get("tags", "").split(",") if tag.strip()}
@@ -723,12 +725,15 @@ def _workspace_filters(query: Mapping) -> dict:
raise ValueError("Invalid gallery filters")
if folder and (_has_traversal(folder) or len(folder) > 1024):
raise ValueError("Invalid gallery folder")
if exact and (_has_traversal(exact) or len(exact) > 1024):
raise ValueError("Invalid gallery path")
return {
"text": text,
"query": folded,
"media": media,
"order": order,
"folder": folder,
"path": exact.casefold(),
"category": category,
"tags": tags,
"favorites": favorites == "1",
@@ -762,7 +767,10 @@ def _filtered_page(
native_folder = folder.replace("/", os.sep)
nested = native_folder + os.sep
total, page, positives = 0, [], set()
exact = filters.get("path", "")
for position, entry in enumerate(entries):
if exact and entry.folded != exact:
continue
if folder and not (
entry.folder == native_folder or entry.folder.startswith(nested)
):
@@ -1155,6 +1163,7 @@ class ImageRoutesMixin:
filters["media"],
filters["order"],
filters["folder"],
filters["path"],
root_index,
[str(root) for root in output_dirs],
filters["category"],
+161
View File
@@ -0,0 +1,161 @@
"""Session job routes (Activity): list, inspect, follow and cancel jobs.
The registry itself is ``py/api/jobs.py``; the scan job's runner is in
``py/api/admin.py``. Every response leaves out absolute server paths: job
rows already carry root-relative paths and public root labels.
"""
import contextlib
import json
from aiohttp import web
from .jobs import DEFAULT_SCAN_SCOPE, SCAN_SCOPES, JobBusy
def _not_found():
return web.json_response({"success": False, "error": "Job not found"}, status=404)
def _bad_request(message):
return web.json_response({"success": False, "error": message}, status=400)
async def _scan_scope(request):
"""The requested scan scope ("new" or "all"), or None when the body is invalid.
An empty body (also a chunked one with no bytes), or one without
``scope``, asks for the default.
"""
if not request.can_read_body:
return DEFAULT_SCAN_SCOPE
text = await request.text()
if not text.strip(): # e.g. a chunked POST that sent no bytes
return DEFAULT_SCAN_SCOPE
try:
body = json.loads(text)
except ValueError:
return None
if not isinstance(body, dict):
return None
scope = body.get("scope", DEFAULT_SCAN_SCOPE)
return scope if scope in SCAN_SCOPES else None
def _summary_event(job):
return f"data: {json.dumps(job.summary())}\n\n".encode("utf-8")
class JobRoutesMixin:
"""Mixin providing the /prompt_manager/jobs endpoints."""
def _register_job_routes(self, routes):
@routes.get("/prompt_manager/jobs")
async def list_jobs_route(request):
return await self.list_jobs(request)
@routes.post("/prompt_manager/jobs/scan")
async def start_scan_job_route(request):
return await self.start_scan_job(request)
@routes.get("/prompt_manager/jobs/{job_id}")
async def job_detail_route(request):
return await self.job_detail(request)
@routes.get("/prompt_manager/jobs/{job_id}/report")
async def job_report_route(request):
return await self.job_report(request)
@routes.get("/prompt_manager/jobs/{job_id}/events")
async def job_events_route(request):
return await self.job_events(request)
@routes.post("/prompt_manager/jobs/{job_id}/cancel")
async def cancel_job_route(request):
return await self.cancel_job(request)
async def list_jobs(self, request):
"""This session's jobs: running first, then finished, newest first."""
jobs = [job.summary() for job in self._jobs.list()]
return web.json_response({"success": True, "jobs": jobs})
async def job_detail(self, request):
job = self._jobs.get(request.match_info["job_id"])
if job is None:
return _not_found()
return web.json_response({"success": True, "job": job.detail()})
async def job_report(self, request):
"""The job's whole record as a JSON attachment (no absolute paths)."""
job = self._jobs.get(request.match_info["job_id"])
if job is None:
return _not_found()
body = json.dumps(job.report(), indent=2)
disposition = f'attachment; filename="{job.report_filename()}"'
return web.Response(
text=body,
content_type="application/json",
headers={
"Content-Disposition": disposition,
"Cache-Control": "no-store",
},
)
async def cancel_job(self, request):
"""Stop a job at its next file boundary; a finished job is a no-op."""
job_id = request.match_info["job_id"]
if self._jobs.get(job_id) is None:
return _not_found()
requested = self._jobs.cancel(job_id)
return web.json_response({"success": True, "cancellation_requested": requested})
async def start_scan_job(self, request):
"""Start the metadata scan as a job (202), or 409 naming the busy job.
The body's ``scope`` is "new" (the default: new and unresolved files)
or "all"; anything else is a 400.
"""
files = await _scan_scope(request)
if files is None:
return _bad_request('The scan scope must be "new" or "all"')
try:
job = self._start_scan_job(files)
except JobBusy as busy:
running = busy.running
return web.json_response(
{
"success": False,
"code": "busy",
"error": str(busy),
"running": {"id": running.id, "title": running.title},
},
status=409,
)
return web.json_response({"success": True, "job": job.summary()}, status=202)
async def job_events(self, request):
"""Stream the job's summary as SSE: now, after each event, and at the end."""
job = self._jobs.get(request.match_info["job_id"])
if job is None:
return _not_found()
response = web.StreamResponse(
status=200,
reason="OK",
headers={
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
},
)
await response.prepare(request)
try:
await response.write(_summary_event(job))
async with contextlib.aclosing(job.events()) as events:
async for _payload in events:
await response.write(_summary_event(job))
# The last event is published before the job takes its final state.
await response.write(_summary_event(job))
await response.write_eof()
except Exception as e: # the socket closed under us; the job goes on
self.logger.info(f"Job progress client disconnected ({e})")
return response
+359
View File
@@ -0,0 +1,359 @@
"""The session job registry: every long operation as one tracked job.
A job runs as an asyncio task that outlives the HTTP request that started it,
so an accidental tab close never abandons half-finished work. Each job keeps
its counters, a capped list of file outcomes per outcome group and a capped
log, and fans its progress out to SSE subscribers: the request that started
it, a reconnecting page, a second tab. A subscriber receives the latest event
as a snapshot and then every event published after it, until the job ends.
Jobs live only in server memory and are gone after a restart. The registry
keeps the running jobs and this session's last ``MAX_FINISHED_JOBS`` finished
ones. Record-changing jobs (the metadata scan, and later maintenance, cleanup
and restore) never run together, and one job of a kind runs at a time.
"""
import asyncio
import itertools
import logging
import re
import time
import uuid
from pathlib import PurePosixPath, PureWindowsPath
logger = logging.getLogger(__name__)
# Start and finish order: wall-clock times can tie (coarse Windows clocks).
_sequence = itertools.count()
JOB_STATES = ("running", "finished", "cancelled", "failed")
FINAL_STATES = JOB_STATES[1:]
MAX_OUTCOME_ROWS = 200 # rows listed per outcome group; counters stay exact
MAX_LOG_LINES = 2000
LOG_CAP_LINE = "[Log capped at 2,000 lines]"
MAX_FINISHED_JOBS = 20 # finished jobs this session keeps
# The metadata scan's outcome groups, in counter-tile order of the spec's
# mapping, and the per-file scan status each group collects. Files without
# usable metadata have no status and go to "no_metadata".
# Which files a scan reads (#249 Activity-2): "new" skips the files the Library
# already links with a resolved link (get_known_resolved_paths) before reading
# them, so new, unlinked, unresolved and ambiguous files are read; "all" reads
# every file again. The workspace job defaults to "new"; legacy /scan is "all".
SCAN_SCOPES = ("new", "all")
DEFAULT_SCAN_SCOPE = "new"
SCAN_GROUP_NAMES = ("added", "known", "review", "no_metadata", "errors")
SCAN_GROUPS = {
"added": "added",
"linked": "added",
"enriched": "added",
"unchanged": "known",
"skipped": "known", # "new" scope: known and resolved, not read again
"ambiguous": "review",
"conflict": "review",
"unresolved": "review",
"failed": "errors",
}
def _relative_path(path) -> str:
"""``path`` as a root-relative POSIX path; an absolute one keeps its name."""
text = str(path or "")
if (
PurePosixPath(text).is_absolute()
or PureWindowsPath(text).is_absolute()
or PureWindowsPath(text).drive
):
return re.split(r"[\\/]", text)[-1]
return text.replace("\\", "/")
class JobBusy(Exception):
"""A job could not start because ``running`` holds its slot."""
def __init__(self, running):
super().__init__(f"{running.title} is already running")
self.running = running
class Job:
"""One tracked operation: state, counters, outcomes, log and subscribers."""
def __init__(
self, kind, title, *, record_changing=False, scope=None, groups=(), clock=None
):
self._clock = clock or time.time
self.id = uuid.uuid4().hex
self.kind = kind
self.title = title
self.record_changing = record_changing
self.scope = dict(scope or {})
self.state = "running"
self.started_at = self._clock()
self._started_seq = next(_sequence)
self._finished_seq = None
self.finished_at = None
self.progress = 0
self.status_line = ""
self.processed = 0
self.total = 0
self.counters = dict.fromkeys(groups, 0)
self.outcomes = {group: [] for group in groups}
self.log = []
self.cancelled = False
self.last_event = None
self.task = None
self._subscribers = set()
@property
def running(self):
return self.state == "running"
@property
def finished(self):
return not self.running
@property
def subscriber_count(self):
return len(self._subscribers)
# ── progress ──────────────────────────────────────────────────────
def update(self, *, progress=None, status_line=None, processed=None, total=None):
"""Set the summary's progress fields; ``None`` leaves one unchanged."""
if progress is not None:
self.progress = max(0, min(100, int(progress)))
if status_line is not None:
self.status_line = status_line
if processed is not None:
self.processed = processed
if total is not None:
self.total = total
def publish(self, payload):
"""Record ``payload`` as the latest event and hand it to every subscriber."""
if isinstance(payload.get("progress"), (int, float)):
self.update(progress=payload["progress"])
self.last_event = payload
for queue in list(self._subscribers):
queue.put_nowait(payload)
def log_line(self, text):
"""Append a timestamped log line, up to MAX_LOG_LINES and one cap line."""
if len(self.log) > MAX_LOG_LINES:
return
if len(self.log) == MAX_LOG_LINES:
self.log.append(LOG_CAP_LINE)
return
stamp = time.strftime("%H:%M:%S", time.localtime(self._clock()))
self.log.append(f"{stamp} {text}")
def record(self, group, row, log_line):
"""Count one file outcome in ``group``, list its row and log it.
``row`` is ``{path, root, root_index, detail}``: the path relative to
its output root, the root's public label and its listing index (the
identity Images opens it by; None when unknown). An absolute path is
cut to its file name, so a row never reveals the server's layout.
"""
if group not in self.counters:
raise ValueError(f"Unknown outcome group {group!r} for {self.kind}")
self.counters[group] += 1
raw = str(row.get("path") or "")
path = _relative_path(raw)
if raw and raw != path:
log_line = log_line.replace(raw, path)
rows = self.outcomes[group]
index = row.get("root_index")
if isinstance(index, bool) or not isinstance(index, int) or index < 0:
index = None
if len(rows) < MAX_OUTCOME_ROWS:
rows.append(
{
"path": path,
"root": str(row.get("root") or ""),
"root_index": index,
"detail": str(row.get("detail") or ""),
}
)
self.log_line(log_line)
def finish(self, state="finished"):
"""End the job in ``state`` and release every subscriber.
Each subscriber still gets the events queued before the release. A
second finish is a no-op, so the first final state stands.
"""
if state not in FINAL_STATES:
raise ValueError(f"{state!r} is not a final job state")
if self.finished:
return
self.state = state
self.finished_at = self._clock()
self._finished_seq = next(_sequence)
for queue in list(self._subscribers):
queue.put_nowait(None)
async def events(self):
"""Yield the latest event, then each later one, until the job finishes.
Consume it under ``contextlib.aclosing`` so a consumer that stops early
(a closed socket) drops its queue at once instead of at garbage collection.
"""
queue = asyncio.Queue()
self._subscribers.add(queue)
try:
if self.last_event is not None:
yield self.last_event
if self.finished:
return
while True:
payload = await queue.get()
if payload is None:
return
yield payload
finally:
self._subscribers.discard(queue)
# ── views ─────────────────────────────────────────────────────────
def summary(self):
"""The job as the Activity list and pill show it."""
return {
"id": self.id,
"kind": self.kind,
"title": self.title,
"state": self.state,
"started_at": self.started_at,
"finished_at": self.finished_at,
"progress": self.progress,
"status_line": self.status_line,
"processed": self.processed,
"total": self.total,
"counters": dict(self.counters),
"scope": dict(self.scope),
# What detail() lists, so a view re-reads it only when it changed.
"listed": {group: len(rows) for group, rows in self.outcomes.items()},
"log_length": len(self.log),
}
def detail(self):
"""The summary plus the listed outcome rows and the log."""
return {
**self.summary(),
"outcomes": {
group: [dict(row) for row in rows]
for group, rows in self.outcomes.items()
},
"log": list(self.log),
}
def report(self):
"""The downloadable report: the summary, every outcome group and the log.
Each group carries its exact count, its listed rows (at most
MAX_OUTCOME_ROWS) and whether the list is cut short. Rows and log lines
already hold root-relative paths only.
"""
summary = self.summary()
summary.pop("listed", None)
summary.pop("log_length", None)
return {
"report_version": 1,
"generated_at": self._clock(),
"job": summary,
"outcomes": {
group: {
"count": self.counters.get(group, len(rows)),
"rows": [dict(row) for row in rows],
"truncated": self.counters.get(group, len(rows)) > len(rows),
}
for group, rows in self.outcomes.items()
},
"log": list(self.log),
}
def report_filename(self):
"""The download name: prompt-manager-KIND-YYYYMMDD-HHMM.json (local time)."""
stamp = time.strftime("%Y%m%d-%H%M", time.localtime(self._clock()))
return f"prompt-manager-{self.kind}-{stamp}.json"
class JobRegistry:
"""Every job of this server session, running and recently finished."""
def __init__(self, logger_=None):
self._logger = logger_ or logger
self._jobs = {}
def start(self, kind, title, *, record_changing, scope, runner, groups=()) -> Job:
"""Create a job and run ``runner(job)`` as a task.
Raises JobBusy, naming the job in the way, when a job of ``kind`` is
running or when ``record_changing`` and another record-changing job
runs. Must be called from the event loop.
"""
same_kind = self.active(kind)
if same_kind is not None:
raise JobBusy(same_kind)
if record_changing:
for job in self._jobs.values():
if job.running and job.record_changing:
raise JobBusy(job)
self._prune()
job = Job(
kind, title, record_changing=record_changing, scope=scope, groups=groups
)
self._jobs[job.id] = job
job.task = asyncio.ensure_future(self._run(job, runner))
return job
async def _run(self, job, runner):
try:
await runner(job)
except asyncio.CancelledError:
job.finish("cancelled")
raise
except Exception:
self._logger.exception(f"Job {job.kind} failed")
job.finish("failed")
finally:
if job.running:
job.finish("cancelled" if job.cancelled else "finished")
def _prune(self):
"""Drop the oldest finished jobs so a new one keeps the cap."""
finished = [job for job in self._jobs.values() if job.finished]
finished.sort(key=lambda job: job._finished_seq)
for job in finished[: max(0, len(finished) - MAX_FINISHED_JOBS + 1)]:
del self._jobs[job.id]
def get(self, job_id):
return self._jobs.get(job_id)
def active(self, kind):
"""The running job of ``kind``, or None."""
for job in self._jobs.values():
if job.kind == kind and job.running:
return job
return None
def list(self):
"""Running jobs first, then the last finished ones, newest first."""
jobs = list(self._jobs.values())
running = [job for job in jobs if job.running]
finished = [job for job in jobs if job.finished]
running.sort(key=lambda job: job._started_seq, reverse=True)
finished.sort(key=lambda job: job._finished_seq, reverse=True)
return running + finished[:MAX_FINISHED_JOBS]
def cancel(self, job_id) -> bool:
"""Ask a running job to stop at its next safe boundary.
Returns whether a cancellation was requested: False for a finished or
unknown job, so a cancel after the end is a no-op.
"""
job = self._jobs.get(job_id)
if job is None or job.finished:
return False
job.cancelled = True
return True
-62
View File
@@ -1,62 +0,0 @@
"""Progress fan-out for the background output scan.
The scan runs as an asyncio task that outlives the HTTP request that started
it, so an accidental tab close does not abandon a half-finished scan. Every
SSE subscriber (the request that started the scan, a reconnecting page, a
second tab) receives the latest event as a snapshot and then every event
published after it, until the job finishes.
"""
import asyncio
class ScanJob:
"""One scan run: its latest event, completion flag and live subscribers."""
def __init__(self):
self.last_event = None
self.finished = False
self.cancelled = False
self.task = None
self._subscribers = set()
@property
def running(self):
return not self.finished
@property
def subscriber_count(self):
return len(self._subscribers)
def publish(self, payload):
"""Record ``payload`` as the latest event and hand it to every subscriber."""
self.last_event = payload
for queue in list(self._subscribers):
queue.put_nowait(payload)
def finish(self):
"""Mark the job done and release every subscriber after its queued events."""
self.finished = True
for queue in list(self._subscribers):
queue.put_nowait(None)
async def events(self):
"""Yield the latest event, then each later one, until the job finishes.
Consume it under ``contextlib.aclosing`` so a consumer that stops early
(a closed socket) drops its queue at once instead of at garbage collection.
"""
queue = asyncio.Queue()
self._subscribers.add(queue)
try:
if self.last_event is not None:
yield self.last_event
if self.finished:
return
while True:
payload = await queue.get()
if payload is None:
return
yield payload
finally:
self._subscribers.discard(queue)
+296
View File
@@ -0,0 +1,296 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from './fixtures.mjs';
import { THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate } from './concept-inventory.mjs';
// Activity-1 Task 3. The review dialog is compared with the concept's own `scan-review` ("Review metadata scan",
// app.js 1428–1432) and its backup box with the concept's System `.notice`; the differences are D55. The job view
// has no concept counterpart: it is compared with the approved mockup `activity-scan.html` (a local, ignored file),
// whose structure and CSS values are restated below (D56). Each comparison is shown to notice a mutation.
const workspace = '/prompt_manager/workspace';
test.beforeEach(async ({ page, request, browserName }) => {
await startCoverage(page, browserName);
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
page.conceptErrors = [];
page.on('pageerror', error => page.conceptErrors.push(error.message));
});
test.afterEach(async ({ page, request, browserName }) => {
await saveCoverage(page, browserName);
expect(page.conceptErrors).toEqual([]);
await request.post('/__test__/jobs/restart');
});
/** The open review dialog: its body's element sequence, the meta grid, the sentence, the notice and the actions. */
function measureReview() {
const dialog = document.querySelector('dialog#modal[open]');
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
const grid = dialog.querySelector('#modal-body > .meta-grid');
const sentence = dialog.querySelector('#modal-body > p.small');
const actions = dialog.querySelector('#modal-body > .dialog-actions');
return {
title: dialog.querySelector('#modal-title').textContent,
body: [...dialog.querySelector('#modal-body').children].map(name),
grid: {
tag: grid.localName,
...css(grid, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap', 'marginTop', 'marginBottom', 'fontSize'),
entries: [...grid.children].map(entry => ({
tag: entry.localName, dt: entry.querySelector('dt').textContent, dd: entry.querySelector('dd').textContent,
dtStyle: css(entry.querySelector('dt'), 'color', 'fontSize'),
ddStyle: css(entry.querySelector('dd'), 'marginTop', 'marginLeft', 'fontSize', 'color'),
})),
},
sentence: { text: sentence.textContent, ...css(sentence, 'fontSize', 'color', 'lineHeight', 'marginTop') },
actions: {
...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
buttons: [...actions.children].map(button => ({ text: button.textContent, className: button.className,
...css(button, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopLeftRadius') })),
},
};
}
/** A `.notice` box: its frame and its text and actions columns, without the copy. */
function measureNotice(selector) {
const notice = document.querySelector(selector);
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
return {
frame: css(notice, 'display', 'alignItems', 'justifyContent', 'columnGap', 'borderTopWidth', 'borderTopStyle',
'borderTopColor', 'backgroundColor', 'borderTopLeftRadius', 'paddingTop', 'paddingLeft'),
columns: [...notice.children].map(node => node.localName + (node.className ? `.${node.className}` : '')),
text: css(notice.querySelector(':scope > div > p'), 'fontSize', 'color', 'marginTop'),
actions: css(notice.querySelector(':scope > .row-actions'), 'display', 'flexShrink'),
};
}
/** A concept `.setting-row` switch: its frame, its label and hint, and its checkbox (not its place-dependent margin). */
function measureToggle(selector) {
const row = document.querySelector(selector);
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
return {
children: [...row.children].map(node => [node.localName, ...node.classList].join('.')),
frame: css(row, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom',
'borderTopWidth', 'borderTopStyle', 'borderTopColor'),
label: css(row.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
hint: css(row.querySelector('p'), 'fontSize', 'color', 'marginTop'),
box: css(row.querySelector('input[type="checkbox"]'), 'accentColor', 'width', 'height', 'appearance',
'backgroundColor', 'borderTopLeftRadius'),
role: row.querySelector('input').getAttribute('role'),
labelled: row.querySelector('label').htmlFor === row.querySelector('input').id,
};
}
/** D55 and D58: the concept's dialog with the production differences applied. */
function mapReviewConcept(reference) {
return {
...reference,
// No sample-plan note (prototype copy); the D58 "All files" toggle follows the grid; the backup box sits
// before the actions.
body: [...reference.body.filter(item => item !== 'p.demo-note').slice(0, -1), 'div.notice', reference.body.at(-1)]
.flatMap(item => item === 'div.meta-grid' ? ['dl.meta-grid', 'div.setting-row'] : [item]),
// A `dl`, as in the concept's maintenance plan; real sources. The scope keeps the concept's
// "New and unresolved files" until "All files" is checked.
grid: { ...reference.grid, tag: 'dl', entries: reference.grid.entries.map(entry => ({ ...entry,
dd: { Sources: '2 output folders', Scope: 'New and unresolved files' }[entry.dt] })) },
actions: { ...reference.actions, buttons: reference.actions.buttons.map(button => button.text === 'Simulate scan'
? { ...button, text: 'Start scan' } : button) },
};
}
async function conceptReview(page, theme) {
await openConcept(page, 'images', theme);
await page.locator('#main .heading-actions').getByRole('button', { name: 'Import', exact: true }).click();
await page.locator('dialog#modal[open] .operation').filter({ hasText: 'Scan an output folder' })
.getByRole('button', { name: 'Review scan', exact: true }).click();
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review metadata scan');
await page.mouse.move(0, 0);
const review = await page.evaluate(measureReview);
await page.keyboard.press('Escape');
// The concept's System → Sources notice ("Bring older images into your prompt history").
await page.goto(`/__test__/canonical/workspace#system`);
await page.getByRole('combobox', { name: 'Theme', exact: true }).selectOption(theme);
await page.locator('#main').getByRole('button', { name: 'Manage sources', exact: true }).click();
await expect(page.locator('#main .notice')).toBeVisible();
return { review, notice: await page.evaluate(measureNotice, '#main .notice'),
toggle: await page.evaluate(measureToggle, '#main .setting-row:has(input[type="checkbox"])') };
}
async function productionReview(page, theme) {
await openProduction(page, 'images', theme);
await page.locator('#workspace-main .heading-actions').getByRole('button', { name: 'Import', exact: true }).click();
await page.locator('dialog#modal[open] .operation').filter({ hasText: 'Scan for metadata' })
.getByRole('button', { name: 'Review scan', exact: true }).click();
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review metadata scan');
// The swapped dialog can open under the pointer that pressed Review scan; measure it at rest.
await page.mouse.move(0, 0);
const review = await page.evaluate(measureReview);
const notice = await page.evaluate(measureNotice, 'dialog#modal[open] .notice');
// The concept's switch is on; compare ours on too, then turn it off again (the default).
const allFiles = page.getByRole('switch', { name: 'All files', exact: true });
await allFiles.check();
await page.mouse.move(0, 0);
const toggle = await page.evaluate(measureToggle, 'dialog#modal[open] .setting-row');
await allFiles.uncheck();
await page.mouse.move(0, 0);
return { review, notice, toggle };
}
for (const theme of THEMES) test(`review dialog matches the concept scan-review ${theme}`, async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await seedConcept(request);
const concept = await conceptReview(page, theme);
expect(concept.review.body).toEqual(['p.demo-note', 'div.meta-grid', 'p.small', 'div.dialog-actions']);
const production = await productionReview(page, theme);
expect(diffInventory(mapReviewConcept(concept.review), production.review)).toEqual([]);
expect(diffInventory(concept.notice, production.notice)).toEqual([]);
expect(diffInventory(concept.toggle, production.toggle)).toEqual([]);
await expect(page.locator('dialog#modal[open] .setting-row > div > label')).toHaveText('All files');
await expect(page.locator('dialog#modal[open] .setting-row input')).not.toBeChecked();
await expect(page.getByRole('switch', { name: 'All files', exact: true })).toBeVisible();
await expect(page.locator('dialog#modal[open] .notice h3')).toHaveText('Back up the database first (recommended)');
await expect(page.locator('dialog#modal[open] .notice .row-actions button')).toHaveText(['Back up now']);
// Mutation check: one CSS change and one removed element are both noticed, then undone to an exact match again.
const undoCss = await mutate(page, { css: '#modal .meta-grid { gap: 3px; } #modal .notice { padding: 2px; } #modal .setting-row { gap: 3px; }' });
const cssMutated = await page.evaluate(measureReview);
const noticeMutated = await page.evaluate(measureNotice, 'dialog#modal[open] .notice');
await page.getByRole('switch', { name: 'All files', exact: true }).check();
await page.mouse.move(0, 0);
const toggleMutated = await page.evaluate(measureToggle, 'dialog#modal[open] .setting-row');
await page.getByRole('switch', { name: 'All files', exact: true }).uncheck();
await page.mouse.move(0, 0);
expect(diffInventory(concept.toggle, toggleMutated).map(item => item.path)).toEqual(['frame.columnGap']);
expect(diffInventory(mapReviewConcept(concept.review), cssMutated).map(item => item.path)).toEqual(expect.arrayContaining(['grid.columnGap', 'grid.rowGap']));
expect(diffInventory(concept.notice, noticeMutated).map(item => item.path)).toEqual(['frame.paddingTop', 'frame.paddingLeft']);
await undoCss();
const undoDom = await mutate(page, { remove: ['#modal-body > p.small'] });
const domMutated = await page.evaluate(() => [...document.querySelectorAll('#modal-body > *')].map(node => node.localName));
expect(domMutated).toEqual(['dl', 'div', 'div', 'div']);
await undoDom();
expect(diffInventory(mapReviewConcept(concept.review), await page.evaluate(measureReview))).toEqual([]);
});
// The approved mockup's job view, restated: structure first, then the values of its CSS rules.
const MOCKUP_TREE = ['div.heading', 'section.surface', 'section.surface', 'details.term'];
const MOCKUP_PROGRESS = ['div.panel-heading(h2,span.small)', 'progress', 'dl.stat-row(div.added.stat,div.stat,div.stat.warn,div.stat,div.bad.stat)'];
const MOCKUP_RESULTS = ['div.panel-heading(h2,label.search-field)', 'div.outcomes(div.outcome-list,div.outcome-files)'];
const MOCKUP_STYLES = {
statRow: { display: 'grid', columnGap: '10px', columns: 5, marginTop: '16px', marginBottom: '16px' },
stat: { borderTopWidth: '1px', borderTopLeftRadius: '7px', paddingTop: '12px', paddingLeft: '14px' },
dt: { fontSize: '11px' },
dd: { fontSize: '20px', fontWeight: '600', marginTop: '6px', marginLeft: '0px' },
outcomes: { display: 'grid', columnGap: '16px', first: '220px', marginTop: '6px' },
outcomeButton: { display: 'flex', justifyContent: 'space-between', marginBottom: '6px', textAlign: 'left' },
fileRow: { display: 'grid', columnGap: '12px', paddingTop: '9px', paddingLeft: '4px', borderBottomWidth: '1px', fontSize: '12px' },
path: { monospace: true, overflowWrap: 'anywhere' },
search: { maxWidth: '320px' },
term: { marginTop: '16px', borderTopWidth: '1px', borderTopLeftRadius: '7px', overflow: 'hidden' },
summary: { paddingTop: '11px', paddingLeft: '14px', fontSize: '12px', cursor: 'pointer' },
pre: { marginTop: '0px', paddingTop: '12px', paddingLeft: '14px', maxHeight: '180px', fontSize: '11px', lineHeight: '18.7px', overflow: 'auto' },
};
function measureJobView() {
const main = document.querySelector('#workspace-main');
const css = (node, ...keys) => Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]]));
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
// The rows and sections the mockup nests are listed with their children.
const shape = node => node.matches('.panel-heading, .outcomes, dl.stat-row')
? `${name(node)}(${[...node.children].map(name).join(',')})` : name(node);
const [progress, results] = main.querySelectorAll(':scope > section.surface');
const statRow = progress.querySelector('dl.stat-row');
const stat = statRow.querySelector('.stat');
const outcomes = results.querySelector('.outcomes');
const button = outcomes.querySelector('.outcome-list > button');
const row = outcomes.querySelector('.file-row');
const term = main.querySelector(':scope > details.term');
const pre = term.querySelector('pre');
const columns = getComputedStyle(statRow).gridTemplateColumns.split(' ');
return {
tree: [...main.children].map(name),
progress: [...progress.children].map(shape),
results: [...results.children].map(shape),
h2: [...main.querySelectorAll(':scope > section.surface > .panel-heading > h2')].map(node => node.textContent),
summary: term.querySelector('summary').textContent,
styles: {
statRow: { ...css(statRow, 'display', 'columnGap', 'marginTop', 'marginBottom'), columns: columns.length },
stat: css(stat, 'borderTopWidth', 'borderTopLeftRadius', 'paddingTop', 'paddingLeft'),
dt: css(stat.querySelector('dt'), 'fontSize'),
dd: css(stat.querySelector('dd'), 'fontSize', 'fontWeight', 'marginTop', 'marginLeft'),
outcomes: { ...css(outcomes, 'display', 'columnGap', 'marginTop'), first: getComputedStyle(outcomes).gridTemplateColumns.split(' ')[0] },
outcomeButton: css(button, 'display', 'justifyContent', 'marginBottom', 'textAlign'),
fileRow: css(row, 'display', 'columnGap', 'paddingTop', 'paddingLeft', 'borderBottomWidth', 'fontSize'),
path: { monospace: /monospace/.test(getComputedStyle(row.querySelector('.path')).fontFamily),
overflowWrap: getComputedStyle(row.querySelector('.path')).overflowWrap },
search: css(results.querySelector('.search-field'), 'maxWidth'),
term: css(term, 'marginTop', 'borderTopWidth', 'borderTopLeftRadius', 'overflow'),
summary: css(term.querySelector('summary'), 'paddingTop', 'paddingLeft', 'fontSize', 'cursor'),
// WebKit reports the 1.7 line height of 11px as 18.700001px; compare it to the hundredth of a pixel.
pre: { ...css(pre, 'marginTop', 'paddingTop', 'paddingLeft', 'maxHeight', 'fontSize', 'overflow'),
lineHeight: `${Math.round(parseFloat(getComputedStyle(pre).lineHeight) * 100) / 100}px` },
},
// Mockup colours: the pressed outcome takes the accent border on the soft blue, the tiles their tone.
pressed: (() => {
const probe = document.createElement('span');
main.append(probe);
const tone = name => { probe.style.color = `var(${name})`; return getComputedStyle(probe).color; };
const values = { accent: tone('--ca-accent'), soft: tone('--soft-blue'), green: tone('--green'), red: tone('--red'), warning: tone('--warning') };
probe.remove();
const on = outcomes.querySelector('.outcome-list > button[aria-pressed="true"]');
const dd = selector => getComputedStyle(statRow.querySelector(`${selector} dd`)).color;
return { border: getComputedStyle(on).borderTopColor === values.accent, background: getComputedStyle(on).backgroundColor === values.soft,
added: dd('.stat.added') === values.green, bad: dd('.stat.bad') === values.red, warn: dd('.stat.warn') === values.warning };
})(),
};
}
const EXPECTED_JOB_VIEW = {
tree: MOCKUP_TREE, progress: MOCKUP_PROGRESS, results: MOCKUP_RESULTS, h2: ['Progress', 'Results'],
summary: '● Scan log — raw detail, click to show', styles: MOCKUP_STYLES,
pressed: { border: true, background: true, added: true, bad: true, warn: true },
};
async function openRunningJob(page, request, theme, viewport = { width: 1600, height: 1100 }) {
await page.setViewportSize(viewport);
const started = await request.post('/__test__/jobs/start', { data: { total: 19622 } });
await expect(started).toBeOK();
const { job } = await started.json();
await expect(await request.post(`/__test__/jobs/${job.id}/step`, { data: { processed: 8240,
counters: { known: 6511, review: 37, no_metadata: 254, errors: 18 },
records: [{ group: 'added', path: 'CCC/mabes9sans-dataset-v1/IMG_GEN_00369_.png', detail: 'Added to the Library as “Mabes portrait”' }] } })).toBeOK();
await openProduction(page, `activity/${job.id}`, theme);
await expect(page.locator('#workspace-main .file-row')).toHaveCount(1);
return job;
}
for (const theme of THEMES) test(`job view follows the approved mockup ${theme}`, async ({ page, request }) => {
await seedConcept(request);
await openRunningJob(page, request, theme);
expect(await page.evaluate(measureJobView)).toEqual(EXPECTED_JOB_VIEW);
// Mutation check: a CSS change (tile gap) and a removed element (the outcome list) are both noticed.
const undoCss = await mutate(page, { css: '#workspace-main .stat-row { gap: 3px; }' });
expect(diffInventory(EXPECTED_JOB_VIEW, await page.evaluate(measureJobView)).map(item => item.path)).toEqual(['styles.statRow.columnGap']);
await undoCss();
const undoDom = await mutate(page, { remove: ['#workspace-main details.term'] });
expect(await page.evaluate(() => [...document.querySelector('#workspace-main').children].map(node => node.localName)))
.toEqual(['div', 'section', 'section']);
await undoDom();
expect(await page.evaluate(measureJobView)).toEqual(EXPECTED_JOB_VIEW);
});
test('job view fits a phone without page scroll', async ({ page, request }) => {
await seedConcept(request);
await openRunningJob(page, request, 'dark', { width: 390, height: 844 });
const layout = await page.evaluate(() => {
const main = document.querySelector('#workspace-main');
const columns = node => getComputedStyle(node).gridTemplateColumns.split(' ').length;
return {
scroll: document.documentElement.scrollWidth <= document.documentElement.clientWidth,
tiles: columns(main.querySelector('dl.stat-row')),
outcomes: columns(main.querySelector('.outcomes')),
inside: [...main.querySelectorAll('.stat, .outcome-list button, .file-row, .search-field')]
.every(node => node.getBoundingClientRect().right <= main.getBoundingClientRect().right + 0.5),
};
});
expect(layout).toEqual({ scroll: true, tiles: 2, outcomes: 1, inside: true });
});
+3 -2
View File
@@ -1,5 +1,5 @@
import { readFileSync } from 'node:fs';
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
test.beforeEach(async ({page, browserName}) => {
@@ -57,7 +57,8 @@ test('Classic keeps immediate selection, search query and metadata serialization
});
test('Classic detaches native text synchronization on removal',async({page})=>{
await page.waitForTimeout(60);
// The page installs the text sync (serializeValue and its listeners) in a 50 ms timer; wait for it, not for time.
await expect.poll(()=>page.evaluate(()=>typeof node.widgets.find(item=>item.name==='text').serializeValue)).toBe('function');
const values=await page.evaluate(()=>{
const widget=node.widgets.find(item=>item.name==='text');
widget.inputEl.value='Before removal';widget.inputEl.dispatchEvent(new Event('input'));
+1 -1
View File
@@ -2,7 +2,7 @@
// same concept selectors in the immutable concept (/__test__/canonical/workspace) and in production, which now renders
// the concept's classes. The concept is never rewritten: each page's map<Page>Concept(reference) applies only the
// recorded spec-§8 deviations, each commented with its D-number, and diffInventory must then report nothing.
import {expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {expect} from './fixtures.mjs';
export {diffInventory} from './library-editor-canonical-comparison.mjs';
export const CONCEPT_URL='/__test__/canonical/workspace';
+33
View File
@@ -0,0 +1,33 @@
// Shared `test` for the page workflows: every Playwright worker owns one
// tests/browser/server.py process on a port the OS chose for it, with its own temp root
// and database. `baseURL` (and so `page`, `request` and `browser.newContext`)
// points at that worker's server. A server that dies fails the running test
// with its stderr tail, and the next test in the worker gets a fresh server.
// A test that fails gets the stderr its server wrote during it attached as
// `server-stderr`.
import { test as base, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { attachServerStderr, createServerSupervisor, startBrowserServer } from '../../tools/vendor/start-browser-server.mjs';
const SERVER_FIXTURE_TIMEOUT_MS = 90_000;
export const test = base.extend({
testServer: [async ({}, use) => {
const supervisor = createServerSupervisor({ start: () => startBrowserServer() });
try {
await use(supervisor);
} finally {
await supervisor.stop();
}
}, { scope: 'worker', timeout: SERVER_FIXTURE_TIMEOUT_MS }],
workerServer: [async ({ testServer }, use, testInfo) => {
const server = await testServer.ensure();
const mark = server.stderrMark();
await use(server);
await attachServerStderr(testInfo, server, mark);
const crash = testServer.crash(server);
if (crash) throw crash;
}, { auto: true, timeout: SERVER_FIXTURE_TIMEOUT_MS }],
baseURL: async ({ workerServer }, use) => use(workerServer.url),
});
export { expect };
+1 -1
View File
@@ -1,5 +1,5 @@
import {readFileSync} from 'node:fs';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
for (const variant of ['Focused', 'Split']) {
+1 -1
View File
@@ -1,5 +1,5 @@
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
import { seedConcept, openConcept, openProduction, SIZES } from './concept-inventory.mjs';
// D10/D20: the heading is the linked positive's title (its confirmed first line here), never the pair's
// "Inspector recipe", which names the negative variant.
@@ -1,5 +1,5 @@
import { writeFile } from 'node:fs/promises';
import { expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { expect } from './fixtures.mjs';
// Content, row counts, and image bytes come from independent synthetic datasets.
// Saved choices and actual executions are separate production sections. The concept's
+1 -1
View File
@@ -1,5 +1,5 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact} from './concept-inventory.mjs';
+8 -5
View File
@@ -1,5 +1,5 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact,countCopy,reflow,withoutItems} from './concept-inventory.mjs';
@@ -7,7 +7,7 @@ import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,
// images of the concept's six prompts, its seeds, sizes and favorites). Heading, toolbar, filter row, caption, grid,
// the first six cards, the continuation and the panel box are compared at 1600x1100 and 390x844 in both themes, and the
// page's behaviors under normal and reduced motion; grid, cards and panel also at 1200x900 and 1800x1100 (2 and 4
// columns, the 275px and 310px panel). The topbar and sidebar (D5, D28, D29) are the shell spec's.
// columns, the 275px and 310px panel). The topbar and sidebar (D28, D29, D52, D53) are the shell spec's.
// Paired captures are artifacts only; there are no PNG baselines.
test.use({locale:'en-US',timezoneId:'UTC'});
@@ -149,12 +149,15 @@ function compareLoadingRow(reference,actual,count) {
return diffInventory(unplaced(mapContinuation(reference,{loaded,total})),unplaced(actual));
}
/** D15: the Import dialog's two real operations, with the ruled intro and the Rescan wording. */
/**
* D15: the Import dialog's real operations, with the ruled intro and the Rescan wording. D55: the concept's own "Scan an
* output folder → Review scan" row follows Rescan as "Scan for metadata", opening the concept's "Review metadata scan".
*/
function mapImportConcept(reference) {
return {...reference,
intro:{...reference.intro,text:'Image inspection and output scanning have different effects. Choose the workflow you want.'},
rows:reference.rows.filter(row=>row.title!=='Import portable recipes').map(row=>row.title==='Scan an output folder'
?{...row,description:'Find files added or removed in the output folders.',buttons:['Rescan']}:row)};
rows:reference.rows.filter(row=>row.title!=='Import portable recipes').flatMap(row=>row.title==='Scan an output folder'
?[{...row,description:'Find files added or removed in the output folders.',buttons:['Rescan']},{...row,title:'Scan for metadata'}]:[row])};
}
const SECTIONS=['heading','toolbar','filterRow','bulkbar','caption','grid','cards','continuation','panel','noPageScroll'];
+1 -1
View File
@@ -1,5 +1,5 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {SIZES,THEMES,seedConcept,openConcept,openProduction,inspectorOf,diffInventory,mutate,countCopy} from './concept-inventory.mjs';
+5 -1
View File
@@ -1,5 +1,5 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,mutate,stripExact,countCopy,reflow} from './concept-inventory.mjs';
@@ -158,6 +158,10 @@ async function filteredOf(page,apply,undo) {
await apply();
await expect.poll(()=>captionOf(page)).not.toBe(before);
await expect.poll(()=>page.locator(CARDS).count()).toBeLessThan(CONCEPT_PROMPTS);
// Production empties the shelf when the filter applies, writes the answer's caption at once and places its cards
// in the next animation frame (one paint for a viewer), so the empty shelf already counts as "fewer": read the
// titles only after that frame, as the Images and Inspector comparisons do.
await settle(page);
const result={caption:await captionOf(page),titles:await titlesOf(page)};
await undo();
await expect.poll(()=>captionOf(page)).toBe(before);
+1 -1
View File
@@ -1,4 +1,4 @@
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
// Artifact-only captures of the editor states that have no concept counterpart (archive confirmation, Show archived,
+1 -1
View File
@@ -1,5 +1,5 @@
import {readFileSync} from 'node:fs';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
const pair={id:4,positive_text:'Library positive',negative_text:'Library negative',category:'Portrait',tags:['studio']};
+1 -1
View File
@@ -1,4 +1,4 @@
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
const moduleUrl = '/prompt_manager/js/shared/toolkit/prompt-bridge.js';
+1 -1
View File
@@ -1,5 +1,5 @@
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
const workspace='/prompt_manager/workspace#/images';
const details=page=>page.getByRole('complementary',{name:'Selected image details',exact:true});
+75 -6
View File
@@ -1,19 +1,24 @@
"""Serve production pages and API routes with disposable state for browser tests."""
import argparse
import asyncio
import json
import os
import shutil
import socket
from pathlib import Path
import sys
import tempfile
import threading
import time
import types
from aiohttp import web
from aiohttp.web_runner import GracefulExit
from PIL import Image
from PIL.PngImagePlugin import PngInfo
from tests.browser.workspace_activity_fixture import register_activity_commands
from tests.browser.workspace_concept_fixture import (
MAX_EXTRA_IMAGES as CONCEPT_MAX_EXTRA_IMAGES,
MAX_SEEDLESS_IMAGES as CONCEPT_MAX_SEEDLESS_IMAGES,
@@ -159,6 +164,7 @@ def create_app(root: Path) -> web.Application:
library_editor_commands = register_library_editor_commands(routes, api, root)
register_vocabulary_commands(routes, api, root)
reveal_stub = register_reveal_stub(routes, api, root)
register_activity_commands(app, routes, api)
async def canonical_asset(request: web.Request) -> web.Response:
filename, content_type = CANONICAL_ASSETS[request.path]
@@ -513,14 +519,77 @@ def create_app(root: Path) -> web.Application:
return app
def main() -> None:
HOST = "127.0.0.1"
PORT_ANNOUNCEMENT = "PM_TEST_SERVER_PORT="
def bind_listener(port: int) -> socket.socket:
"""Bind the server's socket before the app is built.
``port`` 0 lets the OS choose a free port, which this process then holds,
so parallel launchers never race for one between choosing and binding.
"""
listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
if port and os.name != "nt":
# aiohttp's own default on POSIX; on Windows it would let a
# second process share the port.
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listener.bind((HOST, port))
listener.listen(128)
except OSError:
listener.close()
raise
return listener
def _graceful_exit() -> None:
raise GracefulExit()
def exit_when_stdin_closes(loop: asyncio.AbstractEventLoop) -> threading.Thread:
"""Stop the app gracefully once stdin reaches end of file.
The launcher keeps the write end of this process's stdin pipe, and the OS
closes it however the launcher dies, SIGKILL included. GracefulExit runs
aiohttp's cleanup (closing the database) and lets the temp root be removed,
on every OS, where a signal from a dead parent would not.
"""
def watch() -> None:
while sys.stdin.buffer.read(65536):
pass
loop.call_soon_threadsafe(_graceful_exit)
thread = threading.Thread(target=watch, name="stdin-eof", daemon=True)
thread.start()
return thread
def main(argv: list[str] | None = None) -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, default=8765)
args = parser.parse_args()
parser.add_argument(
"--port", type=int, default=8765, help="0 binds an OS-chosen free port"
)
parser.add_argument(
"--exit-on-stdin-eof",
action="store_true",
help="shut down when stdin closes (the launcher's parent-death guard)",
)
args = parser.parse_args(argv)
listener = bind_listener(args.port)
# One machine-readable line tells the launcher which port this process holds.
print(f"{PORT_ANNOUNCEMENT}{listener.getsockname()[1]}", flush=True)
loop = asyncio.new_event_loop()
if args.exit_on_stdin_eof:
exit_when_stdin_closes(loop)
original_cwd = os.getcwd()
with tempfile.TemporaryDirectory(prefix="pm-browser-") as directory:
web.run_app(
create_app(Path(directory).resolve()), host="127.0.0.1", port=args.port
)
try:
web.run_app(create_app(Path(directory).resolve()), sock=listener, loop=loop)
finally:
# create_app works inside the root; Windows cannot remove a process's cwd.
os.chdir(original_cwd)
if __name__ == "__main__":
+25 -10
View File
@@ -1,7 +1,7 @@
import {writeFile} from 'node:fs/promises';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept,openProduction,inventoryOf,diffInventory,focusRingOf,hoverStateOf,mutate} from './concept-inventory.mjs';
import {SIZES,THEMES,MOTIONS,seedConcept,openConcept as openConceptPage,openProduction,inventoryOf,diffInventory,focusRingOf,hoverStateOf,mutate} from './concept-inventory.mjs';
// Task A6 of #249: the production shell (sidebar, topbar, heading, footer) against the immutable concept, with the
// concept-shaped real fixture so the sidebar counts are the concept's own (24 images, 6 prompts, 15 tags). Both themes,
@@ -16,7 +16,16 @@ const FULL_HEADING=new Set(['tags']);
const rowNamed=(nav,text)=>nav.rows.find(row=>row.text===text);
const removedItem=item=>String(item.className).includes('concept-badge')||(item.tag==='button'&&item.text.startsWith('Activity'));
const removedItem=item=>String(item.className).includes('concept-badge');
/**
* The concept with its Activity count hidden (D52): production counts running jobs only and hides the count at 0, as
* the comparison fixture has none. The concept's own CSS then lays out its button without the count.
*/
async function openConcept(page,route,theme,motion) {
await openConceptPage(page,route,theme,motion);
await mutate(page,{css:'#activity-count { display: none }'});
}
/** Left offsets of flex items packed along a row of `width` by its `justify-content` and `gap`. */
function packedLefts(items,width,justify,gap) {
@@ -37,10 +46,12 @@ function remainingTopActions(actions) {
const gap=parseFloat(actions.columnGap);
// A full-width row (<=700px) keeps its width; otherwise it shrinks to its remaining items.
const width=actions.fullWidth?actions.width:shown.reduce((sum,item)=>sum+item.width,0)+gap*(shown.length-1);
// At <=700px the concept spreads three items with `space-between`, so its Theme select sits in the middle; with
// Activity removed (D5) the same rule puts Theme at the row's start. Offsets come from the concept's own sizes.
// At <=700px the concept spreads its shown items with `space-between`; the hidden pill (D53) takes no place.
// Offsets come from the concept's own sizes.
const lefts=new Map(shown.map((item,index)=>[item,packedLefts(shown,width,actions.justifyContent,gap)[index]]));
return {...actions,height,width,
// D52: the full-width row (<=700px) wraps its items when a zoomed phone is narrower than they are.
flexWrap:actions.fullWidth?'wrap':actions.flexWrap,
items:kept.map(item=>item.shown?{...item,top:Math.round((height-item.height)/2),left:lefts.get(item)}:item)};
}
@@ -53,7 +64,7 @@ function remainingFooter(footer,spans) {
}
// D28 ruling (final review A, Minor 1): Overview gets a distinct outline home glyph. The concept has no home icon, and
// its `activity` pulse stays reserved for the Activity row D5 omits, so the two entries never share one glyph.
// its `activity` pulse belongs to the Activity row, so the two entries never share one glyph.
const OVERVIEW_GLYPH='M3 11 12 4l9 7M5 10v10h14V10M10 20v-6h4v6';
/**
@@ -76,7 +87,9 @@ function mapShellConcept(reference) {
if(nav.id!=='system-nav')return nav;
const system=rowNamed(nav,'System');
return {...nav,rows:[
// D5: no Activity row until an activity center exists. D28: no Design review (prototype page).
// D52: the Activity row counts running jobs only, so with none it has no count.
{...rowNamed(nav,'Activity'),count:null},
// D28: no Design review (prototype page).
system,
// D28: the production Overview hub follows System, as a concept nav row without a count. Its icon is
// the ruling's own outline home glyph (OVERVIEW_GLYPH), never the concept's `activity` pulse.
@@ -95,9 +108,11 @@ function mapShellConcept(reference) {
small:'Your library, on your server',
// D28: no fake connection state, so the dot is the neutral muted token (the concept's label color).
dot:{...chrome.connection.dot,backgroundColor:chrome.labels[0].color}},
// D29: no "Interactive concept" badge. D5: no Activity button. The row's size and the remaining items'
// centered offsets follow from the concept's own item sizes.
topActions:remainingTopActions(chrome.topActions),
// D29: no "Interactive concept" badge. D53: the progress pill before the Activity button, hidden with no running
// job. D52: the button's count is hidden at 0. The row's size and the remaining items' centered offsets follow
// from the concept's own item sizes.
topActions:remainingTopActions({...chrome.topActions,items:chrome.topActions.items.flatMap(item=>
item.tag==='button'&&item.text==='Activity'?[{tag:'a',className:'activity-pill',shown:false},{...item,count:'0'}]:[item])}),
footer:(()=>{
// D29: production footer copy (one line, as the concept's own copy is), and no "What changes & why"
// (a prototype review page); the footer then holds only its span.
+1 -1
View File
@@ -1,4 +1,4 @@
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
// Artifact-only captures of Tags & categories states that have no concept counterpart: the Manage categories
+35
View File
@@ -0,0 +1,35 @@
// Each Playwright worker owns its server: `request`, `page` and manual
// `browser.newContext()` all reach it through `baseURL`, and another server
// process never sees its database.
import { test, expect } from './fixtures.mjs';
import { startBrowserServer } from '../../tools/vendor/start-browser-server.mjs';
const search = text => `/prompt_manager/search?text=${encodeURIComponent(text)}`;
test('every fixture reaches this worker\'s own server and another server keeps its own database', async ({ request, page, browser, baseURL, workerServer }, testInfo) => {
expect(baseURL).toBe(workerServer.url);
await expect(await request.post('/__test__/reset')).toBeOK();
const marker = `worker ${testInfo.workerIndex} isolation ${testInfo.project.name} ${testInfo.repeatEachIndex}`;
await expect(await request.post('/prompt_manager/save', { data: { text: marker } })).toBeOK();
expect((await (await request.get(search(marker))).json()).count).toBe(1);
const viaPage = await page.request.get(search(marker));
expect(viaPage.url().startsWith(`${baseURL}/`)).toBe(true);
expect((await viaPage.json()).count).toBe(1);
const context = await browser.newContext();
try {
const response = await (await context.newPage()).goto(search(marker));
expect(response.url().startsWith(`${baseURL}/`)).toBe(true);
expect((await response.json()).count).toBe(1);
} finally {
await context.close();
}
const other = await startBrowserServer();
try {
expect(other.url).not.toBe(baseURL);
expect((await (await fetch(`${other.url}${search(marker)}`)).json()).count).toBe(0);
} finally {
await other.stop();
}
});
+6 -1
View File
@@ -1,5 +1,5 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
test.beforeEach(async ({ request, page, browserName }) => {
await startCoverage(page, browserName);
@@ -64,11 +64,16 @@ test('actual Tags page reloads the selected filter and opens its image viewer',
await page.goto('/prompt_manager/admin#/tags');
await page.locator('[data-tag-name="cats"]').click();
await expect(page.locator('#tagPromptsGrid')).toContainText('Cat portrait');
const dogs = page.waitForResponse(response => response.url().includes('/tags/dogs/prompts?'));
await page.locator('[data-tag-name="dogs"]').click();
const [shown] = (await (await dogs).json()).prompts[0].images;
await expect(page.locator('#tagPromptsGrid')).toContainText('Dog landscape');
await expect(page.locator('#tagPromptsGrid')).not.toContainText('Cat portrait');
await page.locator('#tagPromptsGrid img').first().click();
await expect(page.locator('.viewer-canvas img')).toBeVisible();
// The viewer's title names the shown image through the page's title callback. ViewerJS renders it only once the
// image is viewed, so closing at "visible" alone left the callback unrun on slower hosts (CI coverage missed it).
await expect(page.locator('.viewer-title')).toHaveText(shown.filename);
await page.locator('.viewer-container').getByRole('button', { name: 'Close', exact: true }).click();
await expect(page.locator('.viewer-container')).toHaveCount(0);
});
+801
View File
@@ -0,0 +1,801 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from './fixtures.mjs';
import { seedConcept, openConcept, openProduction } from './concept-inventory.mjs';
// Activity (#249 Activity-1): the concept's Activity page (app.js renderActivity 732–738), its top-bar button
// (index.html 51–52) and sidebar row (app.js ~341), with the recorded D52 (counts of running jobs), D53 (the progress
// pill) and D54 (real job items). Fake jobs come from tests/browser/workspace_activity_fixture.py.
const workspace = '/prompt_manager/workspace';
const main = page => page.locator('#workspace-main');
const pill = page => page.locator('.top-actions #activity-pill');
const activityButton = page => page.locator('.top-actions button#activity-open');
const topCount = page => page.locator('#activity-count');
const navRow = page => page.locator('#system-nav a[data-route="activity"]');
const toast = page => page.locator('#workspace-toast');
const items = page => main(page).locator('section.surface > .activity-item');
const LOST = 'This job ended when ComfyUI restarted.';
// The job view (Task 3, D56): the page heading, then the Progress and Results surfaces and the Scan log.
const heading = page => main(page).locator(':scope > .heading');
const progressSection = page => main(page).getByRole('region', { name: 'Progress', exact: true });
const resultsSection = page => main(page).getByRole('region', { name: 'Results', exact: true });
const tiles = page => progressSection(page).locator('dl.stat-row > div.stat');
const outcomeButtons = page => resultsSection(page).locator('.outcome-list > button');
const fileRows = page => resultsSection(page).locator('.outcome-files > .file-row');
const resultsNote = page => resultsSection(page).locator('.outcome-files > p.small');
const filter = page => resultsSection(page).getByRole('searchbox', { name: 'Filter results', exact: true });
const scanLog = page => main(page).locator('details.term');
const TILE_LABELS = ['Added to Library', 'Already known', 'Needs review', 'No usable metadata', 'Errors'];
const VIEWER_HINT = 'each row opens the image in the viewer';
async function tileValues(page) {
return tiles(page).evaluateAll(nodes => nodes.map(node => [node.className, node.querySelector('dt').textContent,
node.querySelector('dd').textContent]));
}
async function openJob(page, id) {
await page.goto(`${workspace}#/activity/${id}`);
await expect(progressSection(page)).toBeVisible();
}
async function startJob(request, data = {}) {
const response = await request.post('/__test__/jobs/start', { data: { total: 19622, ...data } });
await expect(response).toBeOK();
return (await response.json()).job;
}
async function step(request, id, data) {
await expect(await request.post(`/__test__/jobs/${id}/step`, { data })).toBeOK();
}
async function finish(request, id, state = 'finished') {
const response = await request.post(`/__test__/jobs/${id}/finish`, { data: { state } });
await expect(response).toBeOK();
return (await response.json()).job;
}
async function listed(request) {
const response = await request.get('/prompt_manager/jobs');
await expect(response).toBeOK();
return (await response.json()).jobs;
}
test.beforeEach(async ({ page, request, browserName }) => {
await startCoverage(page, browserName);
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
await expect(await request.post('/__test__/reset')).toBeOK();
page.activityErrors = [];
page.on('pageerror', error => page.activityErrors.push(error.message));
});
test.afterEach(async ({ page, request, browserName }) => {
// The store's own reads can meet a route handler at teardown; that must not fail the test it outlived.
await page.unrouteAll({ behavior: 'ignoreErrors' });
await saveCoverage(page, browserName);
expect(page.activityErrors).toEqual([]);
await request.post('/__test__/jobs/restart');
});
/** The concept's item structure and the computed styles its CSS gives it, measured the same way in both documents. */
function measureItems(selector) {
return [...document.querySelectorAll(selector)].map(item => {
const css = (node, ...keys) => node && Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]]));
const tree = node => ({ tag: node.localName, className: [...node.classList].sort().join(' '), children: [...node.children].map(tree) });
const progress = item.querySelector('progress');
return {
tree: tree(item),
title: item.querySelector('h3').textContent,
chip: item.querySelector('.chip').textContent,
buttons: [...item.querySelectorAll('.row-actions button')].map(button => button.textContent),
styles: {
item: css(item, 'paddingTop', 'paddingBottom', 'borderBottomWidth', 'borderBottomStyle', 'borderBottomColor'),
heading: css(item.querySelector('.panel-heading'), 'display', 'justifyContent', 'alignItems', 'columnGap'),
h3: css(item.querySelector('h3'), 'fontSize', 'fontWeight', 'color', 'marginTop', 'marginBottom'),
chip: css(item.querySelector('.chip'), 'fontSize', 'color', 'backgroundColor', 'paddingTop', 'paddingLeft', 'borderTopLeftRadius'),
p: css(item.querySelector('p'), 'fontSize', 'color', 'marginTop', 'lineHeight'),
progress: progress && { ...css(progress, 'height', 'marginTop', 'marginBottom', 'accentColor'),
fills: Math.abs(progress.getBoundingClientRect().width - item.getBoundingClientRect().width) < 1,
max: progress.max, label: Boolean(progress.getAttribute('aria-label')) },
actions: css(item.querySelector('.row-actions'), 'display', 'columnGap', 'marginTop'),
button: css(item.querySelector('.row-actions button'), 'fontSize', 'paddingTop', 'paddingLeft', 'minHeight', 'borderTopLeftRadius'),
},
};
});
}
/** D56: an item tree whose `h3` holds the link to the job's view. */
function withTitleLink(tree) {
const [heading, ...rest] = tree.children;
const link = { tag: 'a', className: '', children: [] };
return { ...tree, children: [{ ...heading, children: heading.children.map(node => node.tag === 'h3' ? { ...node, children: [link] } : node) }, ...rest] };
}
function measurePage(mainSelector) {
const root = document.querySelector(mainSelector);
const section = root.querySelector(':scope > section.surface');
return {
h1: root.querySelector('.heading h1').textContent,
description: root.querySelector('.heading p.muted').textContent,
actions: [...root.querySelector('.heading .heading-actions').children].map(node => node.textContent),
h2: section.querySelector(':scope > h2').textContent,
surface: Object.fromEntries(['paddingTop', 'borderTopWidth', 'backgroundColor', 'borderTopLeftRadius']
.map(key => [key, getComputedStyle(section)[key]])),
};
}
for (const theme of ['dark', 'light']) test(`activity page matches the concept list markup ${theme}`, async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await seedConcept(request);
// The concept: its finished sample, then a running job started from System (startJob, app.js 940–952).
await openConcept(page, 'activity', theme);
const conceptPage = await page.evaluate(measurePage, '#main');
const [conceptDone] = await page.evaluate(measureItems, '#main .activity-item');
await page.goto('/__test__/canonical/workspace#system');
await page.getByRole('combobox', { name: 'Theme', exact: true }).selectOption(theme);
await page.getByRole('button', { name: 'Review thumbnails', exact: true }).click();
await expect(page.locator('#main .activity-item progress')).toHaveCount(1);
const [conceptRunning] = await page.evaluate(measureItems, '#main .activity-item');
const done = await startJob(request);
await step(request, done.id, { processed: 19622, counters: { added: 1420, errors: 18 } });
await finish(request, done.id);
const running = await startJob(request);
await step(request, running.id, { processed: 8240 });
await openProduction(page, 'activity', theme);
await expect(items(page)).toHaveCount(2);
const productionPage = await page.evaluate(measurePage, '#workspace-main');
const [productionRunning, productionDone] = await page.evaluate(measureItems, '#workspace-main .activity-item');
// D52: no "Unified activity · proposed" chip (prototype copy); the rest of the heading and the section are the concept's.
expect(conceptPage.actions).toEqual(['Unified activity · proposed']);
expect(productionPage).toEqual({ ...conceptPage, actions: [] });
// A completed job is the concept's completed item; D56: its title is a link to the job's view, in the title's colour.
expect(conceptDone.title).toBe('✓ Library metadata enriched');
expect(productionDone).toEqual({ ...conceptDone, tree: withTitleLink(conceptDone.tree), title: '✓ Scan for metadata' });
const links = await items(page).locator('h3 > a').evaluateAll(nodes => nodes.map(node => [node.getAttribute('href'),
getComputedStyle(node).color === getComputedStyle(node.parentElement).color, getComputedStyle(node).textDecorationLine]));
expect(links).toEqual([[`#/activity/${running.id}`, true, 'none'], [`#/activity/${done.id}`, true, 'none']]);
// D54: a running job's chip names its state ("Running", where the concept's sample reads "Needs attention"), and
// its actions are Cancel job only (no "Simulate completion").
expect(conceptRunning.buttons).toEqual(['Simulate completion', 'Cancel job']);
const [, ...cancelOnly] = conceptRunning.tree.children.at(-1).children;
const conceptActions = { ...conceptRunning.tree.children.at(-1), children: cancelOnly };
expect(productionRunning).toEqual({ ...conceptRunning,
tree: withTitleLink({ ...conceptRunning.tree, children: [...conceptRunning.tree.children.slice(0, -1), conceptActions] }),
title: '○ Scan for metadata', chip: 'Running', buttons: ['Cancel job'] });
await expect(items(page).nth(1).locator('p')).toHaveText(/^Finished \d\d:\d\d · 1,420 added · 18 errors$/);
await expect(items(page).first().locator('p')).toHaveText(/^Started \d\d:\d\d · 2 output folders · closing this view does not cancel it$/);
await expect(items(page).first().locator('progress')).toHaveAttribute('aria-label', 'Scan for metadata progress');
expect(await items(page).first().locator('progress').evaluate(node => node.value)).toBe(41);
});
test('pill shows while a job runs and opens its view', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await page.goto(`${workspace}#/images`);
// D52: no running job, so neither count shows; the button is the concept's.
await expect(activityButton(page)).toHaveAccessibleName('Activity');
await expect(topCount(page)).toBeHidden();
await expect(navRow(page)).toHaveText('Activity');
await expect(navRow(page).locator('.nav-count')).toHaveCount(0);
await expect(pill(page)).toBeHidden();
const job = await startJob(request);
// The list is read again on its own (15 s), or at once on a reload or when Activity opens.
await page.reload();
await expect(pill(page)).toBeVisible();
await expect(pill(page)).toHaveText('Scanning · 0% · 0 / 19,622');
await step(request, job.id, { processed: 8240 });
const text = 'Scanning · 41% · 8,240 / 19,622';
await expect(pill(page)).toHaveText(text);
await expect(pill(page)).toHaveAttribute('aria-label', text);
await expect(pill(page)).toHaveAttribute('href', `#/activity/${job.id}`);
expect(await pill(page).locator('.activity-pill-bar i').evaluate(node => node.style.width)).toBe('41%');
// The pill sits before the button, in the mockup's place, and both counts read 1.
expect(await page.locator('.top-actions > *').evaluateAll(nodes => nodes.map(node => node.id || node.className)))
.toEqual(['activity-pill', 'activity-open', 'theme-label', 'canvas-link']);
await expect(topCount(page)).toBeVisible();
await expect(topCount(page)).toHaveText('1');
await expect(navRow(page).locator('.nav-count')).toHaveText('1');
// The pill opens the job's own view (Task 3, D56), named by the job.
await pill(page).click();
await expect(page).toHaveURL(new RegExp(`#/activity/${job.id}$`));
await expect(main(page).getByRole('heading', { level: 1 })).toHaveText('Scan for metadata');
await expect(page.locator('#workspace-breadcrumb')).toHaveText('Activity');
await expect(page).toHaveTitle('Scan for metadata · Prompt Manager');
await expect(navRow(page)).toHaveAttribute('aria-current', 'page');
await expect(progressSection(page).locator('progress')).toHaveJSProperty('value', 41);
// The Activity button opens the list itself from any page.
await page.locator('nav [data-route="library"]').click();
await activityButton(page).click();
await expect(page).toHaveURL(/#\/activity$/);
await expect(items(page)).toHaveCount(1);
await finish(request, job.id);
await expect(pill(page)).toBeHidden();
await expect(topCount(page)).toBeHidden();
await expect(navRow(page).locator('.nav-count')).toHaveCount(0);
await expect(items(page).first().locator('.chip.green')).toHaveText('Completed');
await expect(items(page).first().locator('progress, .row-actions')).toHaveCount(0);
// A finished job's title still opens its view (D56).
await items(page).first().locator('h3 > a').click();
await expect(page).toHaveURL(new RegExp(`#/activity/${job.id}$`));
await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d · 0 added · 0 errors$/);
});
test('a phone keeps the pill and its controls inside the topbar', async ({ page, request }) => {
await page.setViewportSize({ width: 390, height: 844 });
const job = await startJob(request);
await step(request, job.id, { processed: 8240 });
await page.goto(`${workspace}#/images`);
await expect(pill(page)).toHaveText('Scanning · 41% · 8,240 / 19,622');
await expect(pill(page)).toHaveAttribute('aria-label', 'Scanning · 41% · 8,240 / 19,622');
await expect(pill(page).locator('.activity-pill-files, .activity-pill-bar')).toHaveCount(2);
await expect(pill(page).locator('.activity-pill-files')).toBeHidden();
const layout = await page.locator('.top-actions').evaluate(row => ({
scroll: document.documentElement.scrollWidth <= document.documentElement.clientWidth,
inside: [...row.children].filter(node => node.getBoundingClientRect().width)
.every(node => node.getBoundingClientRect().right <= row.getBoundingClientRect().right + 0.5
&& node.getBoundingClientRect().left >= row.getBoundingClientRect().left - 0.5),
}));
expect(layout).toEqual({ scroll: true, inside: true });
});
test('reload during a scan reattaches to the running job', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 1000 });
await page.goto(`${workspace}#/activity`);
await expect(items(page)).toHaveCount(1);
await expect(pill(page)).toHaveText('Scanning · 5% · 1,000 / 19,622');
await page.reload();
await expect(pill(page)).toHaveText('Scanning · 5% · 1,000 / 19,622');
// The reloaded page follows the same job live and started nothing.
await step(request, job.id, { processed: 9811 });
await expect(pill(page)).toHaveText('Scanning · 50% · 9,811 / 19,622');
expect(await items(page).first().locator('progress').evaluate(node => node.value)).toBe(50);
await expect(items(page)).toHaveCount(1);
expect((await listed(request)).map(item => [item.id, item.state])).toEqual([[job.id, 'running']]);
// A second tab attaches to the same job and sees the same progress.
const other = await page.context().newPage();
await other.goto(`${workspace}#/activity`);
await expect(pill(other)).toHaveText('Scanning · 50% · 9,811 / 19,622');
await expect(other.locator('#workspace-main .activity-item')).toHaveCount(1);
await other.close();
expect((await listed(request)).length).toBe(1);
// Cancel job asks the server to stop at a file boundary; the job ends cancelled with its partial counts.
await step(request, job.id, { counters: { added: 7 } });
await items(page).first().getByRole('button', { name: 'Cancel job', exact: true }).focus();
await page.keyboard.press('Enter');
await expect(toast(page)).toHaveText('Scan cancelled · 7 added');
// The removed Cancel job hands its focus to the item's title link (review M4).
await expect(items(page).first().locator('h3 > a')).toBeFocused();
await expect(items(page).first().locator('.chip.blue')).toHaveText('Cancelled');
await expect(items(page).first().locator('p')).toHaveText(/^Cancelled \d\d:\d\d at a file boundary · 7 added · completed work is kept$/);
await expect(pill(page)).toBeHidden();
});
test('lost server clears the pill and marks jobs ended', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 4000 });
await page.goto(`${workspace}#/activity`);
await expect(pill(page)).toBeVisible();
await expect(topCount(page)).toHaveText('1');
const reads = [];
page.on('request', item => { if (new URL(item.url()).pathname === '/prompt_manager/jobs') reads.push(Date.now()); });
// ComfyUI restarts: the stream drops without a final event and the new registry is empty.
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
await expect(pill(page)).toBeHidden();
await expect(topCount(page)).toBeHidden();
await expect(items(page).first().locator('.chip.blue')).toHaveText('Ended');
await expect(items(page).first().locator('p')).toHaveText(LOST);
await expect(items(page).first().locator('progress, .row-actions')).toHaveCount(0);
await expect(navRow(page).locator('.nav-count')).toHaveCount(0);
// One re-read found the restart; there is no retry storm and no finish toast for a job that only vanished.
await page.waitForTimeout(2500);
expect(reads.length).toBeLessThanOrEqual(2);
await expect(toast(page)).toHaveText('');
// A server that cannot be read at all also ends the running job, then is read again with a growing backoff.
const second = await startJob(request);
await page.reload();
await expect(pill(page)).toBeVisible();
const failed = [];
await page.route('**/prompt_manager/jobs**', route => { failed.push(Date.now()); return route.abort(); });
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
await expect(pill(page)).toBeHidden();
await expect(items(page).filter({ hasText: LOST })).toHaveCount(1);
await page.waitForTimeout(3500);
// 1 s, then 2 s: at most two re-reads in 3.5 s.
expect(failed.length).toBeLessThanOrEqual(2);
await page.unroute('**/prompt_manager/jobs**');
expect(second.id).not.toBe(job.id);
});
test('start and finish toasts', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await page.goto(`${workspace}#/images`);
await expect(pill(page)).toBeVisible();
await step(request, job.id, { processed: 19622, counters: { added: 1420, errors: 18 } });
await finish(request, job.id);
await expect(toast(page)).toHaveText('Scan finished · 1,420 added · 18 errors');
await expect(pill(page)).toBeHidden();
const single = await startJob(request);
await page.reload();
await expect(pill(page)).toBeVisible();
await step(request, single.id, { processed: 19622, counters: { added: 1, errors: 1 } });
await finish(request, single.id, 'failed');
await expect(toast(page)).toHaveText('Scan failed · 1 added · 1 error');
// The store's own start: the real metadata scan over this server's outputs, announced, followed and finished.
await page.goto(`${workspace}#/activity`);
const started = await page.evaluate(async () => {
const { createActivityStore } = await import('/prompt_manager/js/workspace/activity-store.mjs');
const controller = new AbortController();
const store = createActivityStore({ api: window.ApiClient.createApiClient(), pollMs: 0,
scope: { signal: controller.signal, cleanup: () => {} } });
const result = await store.start('scan');
// Read before the busy attempt: its list read may already see the small scan finished and toast that.
const toast = document.getElementById('workspace-toast').textContent;
const busy = await store.start('scan').catch(error => ({ error: error.message }));
return { id: result.job.id, title: result.job.title, toast, busy: busy.busy?.id ?? null, message: busy.message ?? null };
});
expect(started.title).toBe('Scan for metadata');
expect(started.toast).toBe('Scan started.');
// A second start while the scan runs is refused and names it (unless the small fixture scan already ended).
if (started.busy !== null) expect(started.busy).toBe(started.id);
await expect(toast(page)).toHaveText(/^Scan (started\.|finished · \d+ added · \d+ errors?)$/);
await expect.poll(async () => (await listed(request)).find(item => item.id === started.id)?.state, { timeout: 15000 }).toBe('finished');
});
// ── Task 3: Scan for metadata, its review dialog and the job view ─────────────────────────────
async function startFromImport(page) {
await page.goto(`${workspace}#/images`);
await page.getByRole('button', { name: 'Import', exact: true }).click();
const importDialog = page.getByRole('dialog', { name: 'Import & inspect', exact: true });
const row = importDialog.locator('.operation').filter({ hasText: 'Scan for metadata' });
await row.getByRole('button', { name: 'Review scan', exact: true }).click();
const review = page.getByRole('dialog', { name: 'Review metadata scan', exact: true });
await expect(review).toBeVisible();
await review.getByRole('button', { name: 'Start scan', exact: true }).click();
await expect(review).toHaveCount(0);
}
test('start scan creates a job, toasts, and opens its view', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const posted = [];
page.on('request', item => { if (item.method() === 'POST' && new URL(item.url()).pathname === '/prompt_manager/jobs/scan') posted.push(item); });
await startFromImport(page);
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
const id = page.url().split('/').at(-1);
expect(posted).toHaveLength(1);
// The fixture scan is small and may already have finished; the start toast came first either way.
await expect(toast(page)).toHaveText(/^Scan (started\.|finished · \d+ added · \d+ errors?)$/);
await expect(heading(page).locator('h1')).toHaveText('Scan for metadata');
await expect(heading(page).locator('h1')).toBeFocused();
expect((await listed(request)).map(item => item.id)).toEqual([id]);
await expect.poll(async () => (await listed(request))[0].state, { timeout: 15000 }).toBe('finished');
await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d · \d+ added · \d+ errors?$/);
// The finished scan's tiles add up to the files it processed, and its log is there to open.
const job = (await listed(request))[0];
const values = await tileValues(page);
expect(values.map(([, label]) => label)).toEqual(TILE_LABELS);
expect(values.reduce((sum, [, , value]) => sum + Number(value.replaceAll(',', '')), 0)).toBe(job.processed);
// Opening the log reads its last lines (review M12).
await scanLog(page).locator('summary').click();
await expect(scanLog(page).locator('pre')).toContainText('Scan finished');
});
// ── Task 4 (Activity-2): scan scope, new and unresolved files by default ─────────────────────
test('all files checkbox changes the scope line and the request', async ({ page }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const bodies = [];
page.on('request', item => {
if (item.method() === 'POST' && new URL(item.url()).pathname === '/prompt_manager/jobs/scan') bodies.push(item.postDataJSON());
});
const review = page.getByRole('dialog', { name: 'Review metadata scan', exact: true });
const scopeValue = review.locator('.meta-grid > div').filter({ has: page.locator('dt', { hasText: 'Scope' }) }).locator('dd');
// The concept's switch: a checkbox with role=switch.
const allFiles = review.getByRole('switch', { name: 'All files', exact: true });
const openReview = async () => {
await page.goto(`${workspace}#/images`);
await page.getByRole('button', { name: 'Import', exact: true }).click();
await page.getByRole('dialog', { name: 'Import & inspect', exact: true }).locator('.operation')
.filter({ hasText: 'Scan for metadata' }).getByRole('button', { name: 'Review scan', exact: true }).click();
await expect(review).toBeVisible();
};
// The default: the concept's "New and unresolved files", the box unchecked; the request asks for "new".
await openReview();
await expect(scopeValue).toHaveText('New and unresolved files');
await expect(allFiles).not.toBeChecked();
await expect(review.locator('.setting-row p')).toHaveText('Also read the files already linked to their prompt again.');
await expect(allFiles).toHaveAccessibleDescription('Also read the files already linked to their prompt again.');
// Checking and unchecking switches the Scope value both ways.
await allFiles.check();
await expect(scopeValue).toHaveText('All files');
await allFiles.uncheck();
await expect(scopeValue).toHaveText('New and unresolved files');
await review.getByRole('button', { name: 'Start scan', exact: true }).click();
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
await expect.poll(() => bodies.length).toBe(1);
expect(bodies[0]).toEqual({ scope: 'new' });
await expect(heading(page).locator('h1')).toHaveText('Scan for metadata');
// Wait for the first scan to end so the second is not refused as busy.
await scanLog(page).locator('summary').click();
await expect(scanLog(page).locator('pre')).toContainText('Scan finished', { timeout: 15000 });
await expect(scanLog(page).locator('pre')).toContainText('new and unresolved files');
// Checked: "All files", and the request asks for "all".
await openReview();
await allFiles.check();
await expect(scopeValue).toHaveText('All files');
await review.getByRole('button', { name: 'Start scan', exact: true }).click();
await expect.poll(() => bodies.length).toBe(2);
expect(bodies[1]).toEqual({ scope: 'all' });
await scanLog(page).locator('summary').click();
await expect(scanLog(page).locator('pre')).toContainText('Scan started · ', { timeout: 15000 });
await expect(scanLog(page).locator('pre')).toContainText('all files');
});
test('job view counters follow the mapping and update live', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 8240, counters: { added: 1420, known: 6511, review: 37, no_metadata: 254, errors: 18 } });
await openJob(page, job.id);
// The header: the title, the detail line, then Download report (enabled in Task 5) and Cancel scan (danger).
await expect(heading(page).locator('h1')).toHaveText('Scan for metadata');
await expect(heading(page).locator('p.muted')).toHaveText(/^Started \d\d:\d\d · 2 output folders · closing this view does not cancel it$/);
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report', 'Cancel scan']);
await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeEnabled();
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).toHaveClass(/danger/);
// Progress: the bar, the server's line and the five tiles in order and colour.
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Processing file 8,240 of 19,622');
await expect(progressSection(page).locator('progress')).toHaveAttribute('aria-label', 'Scan progress');
await expect(progressSection(page).locator('progress')).toHaveJSProperty('value', 41);
expect(await tileValues(page)).toEqual([
['stat added', 'Added to Library', '1,420'], ['stat', 'Already known', '6,511'], ['stat warn', 'Needs review', '37'],
['stat', 'No usable metadata', '254'], ['stat bad', 'Errors', '18'],
]);
const colours = await tiles(page).evaluateAll(nodes => nodes.map(node => getComputedStyle(node.querySelector('dd')).color));
const expected = await page.evaluate(() => ['--green', '--ca-text', '--warning', '--ca-text', '--red'].map(name => {
const probe = document.createElement('span');
probe.style.color = `var(${name})`;
document.querySelector('#workspace-main').append(probe);
const colour = getComputedStyle(probe).color;
probe.remove();
return colour;
}));
expect([colours[0], colours[2], colours[4]]).toEqual([expected[0], expected[2], expected[4]]);
expect(colours[1]).toBe(colours[3]);
// The outcome list follows the mockup's order with the same counts.
await expect(outcomeButtons(page)).toHaveText(['Added to Library1,420', 'Needs review37', 'Errors18', 'No usable metadata254', 'Already known6,511']);
// Live: the next step updates the bar, the line and the tiles without a reload.
await step(request, job.id, { processed: 9811, counters: { added: 1500, errors: 19 }, records: [
{ group: 'added', path: 'live/new.png', detail: 'Added to the Library as “Live”' },
] });
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Processing file 9,811 of 19,622');
await expect(progressSection(page).locator('progress')).toHaveJSProperty('value', 50);
await expect(tiles(page).first().locator('dd')).toHaveText('1,500');
await expect(tiles(page).last().locator('dd')).toHaveText('19');
await expect(fileRows(page)).toHaveCount(1);
// A focused row keeps its focus while later reads append rows.
await fileRows(page).first().focus();
await step(request, job.id, { records: [{ group: 'added', path: 'live/next.png', detail: 'Added' }] });
await expect(fileRows(page)).toHaveCount(2);
await expect(fileRows(page).first()).toBeFocused();
// The log is collapsed by default and holds the raw timestamped lines.
await expect(scanLog(page)).not.toHaveAttribute('open', '');
await expect(scanLog(page).locator('summary')).toContainText('Scan log');
await scanLog(page).locator('summary').click();
await expect(scanLog(page).locator('pre')).toHaveAttribute('tabindex', '0');
await expect(scanLog(page).locator('pre')).toHaveText(/^\d\d:\d\d:\d\d \+ live\/new\.png · Added to the Library as “Live”\n\d\d:\d\d:\d\d \+ live\/next\.png · Added$/);
// Finished: the header says so, Cancel scan is gone and the counters stay.
await finish(request, job.id);
await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d · 1,501 added · 19 errors$/);
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
await expect(tiles(page).first().locator('dd')).toHaveText('1,501');
});
test('added rows open the viewer, error rows are plain', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
// The fixture's two listed files share a name in the output and input roots; the row's root label tells them apart.
await step(request, job.id, { processed: 3, records: [
{ group: 'added', path: 'shared/same.png', root: 'output', detail: 'Added to the Library as “Cat portrait”' },
{ group: 'known', path: 'shared/same.png', root: 'input', detail: 'Already linked to “Dog portrait”' },
{ group: 'errors', path: 'broken/truncated.png', root: 'output', detail: 'Error: the file could not be processed (see the server log)' },
] });
await openJob(page, job.id);
await expect(outcomeButtons(page).first()).toHaveAttribute('aria-pressed', 'true');
await expect(fileRows(page)).toHaveCount(1);
const added = fileRows(page).first();
expect(await added.evaluate(node => node.localName)).toBe('button');
await expect(added.locator('.path')).toHaveText('shared/same.png');
await expect(added.locator('.why')).toHaveText('Added to the Library as “Cat portrait”');
await expect(resultsNote(page)).toHaveText('Each row opens the image in the viewer.');
await added.click();
const viewer = page.locator('dialog#viewer-dialog[open]');
await expect(viewer).toBeVisible();
await expect(viewer.locator('#viewer-title')).toContainText('same.png');
await expect(viewer.locator('#viewer-stage img').first()).toHaveAttribute('src', /[?&]root=0(&|$)/);
await page.keyboard.press('Escape');
await expect(viewer).toHaveCount(0);
await expect(added).toBeFocused();
// Already known opens the input root's file.
await outcomeButtons(page).filter({ hasText: 'Already known' }).click();
await expect(outcomeButtons(page).filter({ hasText: 'Already known' })).toHaveAttribute('aria-pressed', 'true');
await fileRows(page).first().click();
await expect(page.locator('dialog#viewer-dialog[open] #viewer-stage img').first()).toHaveAttribute('src', /[?&]root=1(&|$)/);
await page.keyboard.press('Escape');
// Errors: plain text rows, no viewer hint.
await outcomeButtons(page).filter({ hasText: 'Errors' }).click();
await expect(fileRows(page)).toHaveCount(1);
expect(await fileRows(page).first().evaluate(node => node.localName)).toBe('div');
await expect(fileRows(page).first().locator('.path')).toHaveText('broken/truncated.png');
await expect(resultsNote(page)).toHaveCount(0);
// An empty group says so.
await outcomeButtons(page).filter({ hasText: 'Needs review' }).click();
await expect(resultsSection(page).locator('.outcome-files > p.empty-state')).toHaveText('No files in this group.');
// A listed row whose file has left the listing toasts instead of opening an empty viewer.
await step(request, job.id, { records: [{ group: 'added', path: 'gone/missing.png', root: 'output', detail: 'Added' }] });
await outcomeButtons(page).first().click();
await expect(fileRows(page)).toHaveCount(2);
await fileRows(page).nth(1).click();
await expect(toast(page)).toHaveText('This file is no longer in Images. Rescan from Import and try again.');
await expect(page.locator('dialog#viewer-dialog')).toHaveCount(0);
// A row opens only its own root's file: a root whose label no longer matches (a replaced source) never opens the
// other root's same path, and a row without a root index opens nothing (review M8, M9).
const lookups = [];
page.on('request', item => { const url = new URL(item.url()); if (url.pathname === '/prompt_manager/images/output' && url.searchParams.get('path')) lookups.push(url.search); });
await step(request, job.id, { records: [
{ group: 'added', path: 'shared/same.png', root: 'output', root_index: 1, detail: 'Added' },
{ group: 'added', path: 'shared/same.png', root: 'output', root_index: null, detail: 'Added' },
] });
await expect(fileRows(page)).toHaveCount(4);
for (const index of [2, 3]) {
await fileRows(page).nth(index).click();
await expect(toast(page)).toHaveText('This file is no longer in Images. Rescan from Import and try again.');
await expect(page.locator('dialog#viewer-dialog')).toHaveCount(0);
}
expect(lookups).toEqual([]);
// The lookup is the exact identity, not a name search.
await fileRows(page).first().click();
await expect(page.locator('dialog#viewer-dialog[open]')).toBeVisible();
expect(lookups).toEqual(['?mode=workspace&root=0&path=shared%2Fsame.png&limit=5']);
await page.keyboard.press('Escape');
});
test('filter narrows the selected outcome', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 4, records: [
{ group: 'added', path: 'CCC/portrait_0001.png', detail: 'Added' },
{ group: 'added', path: 'CCC/portrait_0002.png', detail: 'Added' },
{ group: 'added', path: 'forest/pines_0008.png', detail: 'Added' },
{ group: 'errors', path: 'CCC/broken.png', detail: 'Error' },
] });
await openJob(page, job.id);
await expect(fileRows(page)).toHaveCount(3);
await expect(filter(page)).toHaveAttribute('placeholder', 'Filter by file path…');
await filter(page).fill('ccc/');
await expect(fileRows(page).locator('.path')).toHaveText(['CCC/portrait_0001.png', 'CCC/portrait_0002.png']);
await expect(resultsNote(page)).toHaveText(`Showing 2 of 3 · ${VIEWER_HINT}`);
// The filter stays while another outcome is chosen.
await outcomeButtons(page).filter({ hasText: 'Errors' }).click();
await expect(fileRows(page).locator('.path')).toHaveText(['CCC/broken.png']);
await filter(page).fill('pines');
await expect(fileRows(page)).toHaveCount(0);
await expect(resultsSection(page).locator('.outcome-files > p.empty-state')).toHaveText('No listed file matches this filter.');
await outcomeButtons(page).first().click();
await expect(fileRows(page).locator('.path')).toHaveText(['forest/pines_0008.png']);
await filter(page).fill('');
await expect(fileRows(page)).toHaveCount(3);
});
test('job view renders 200-row cap note', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 260, bulk: { group: 'added', count: 250 }, records: [
...Array.from({ length: 10 }, (_, index) => ({ group: 'no_metadata', path: `misc/plain_${index}.png`, detail: 'No prompt metadata in the file' })),
] });
// Every detail read of this job: in flight, and whether one has returned the full listing.
const detailUrl = `/prompt_manager/jobs/${job.id}`;
const isDetail = item => new URL(item.url()).pathname === detailUrl;
const detailReads = { inFlight: 0, full: false };
page.on('request', item => { if (isDetail(item)) detailReads.inFlight += 1; });
page.on('requestfailed', item => { if (isDetail(item)) detailReads.inFlight -= 1; });
page.on('response', async response => {
if (!isDetail(response)) return;
const body = await response.json().catch(() => null);
const outcomes = body?.job?.outcomes ?? {};
if (outcomes.added?.length === 200 && outcomes.no_metadata?.length === 10) detailReads.full = true;
detailReads.inFlight -= 1;
});
await openJob(page, job.id);
await expect(tiles(page).first().locator('dd')).toHaveText('250');
await expect(fileRows(page)).toHaveCount(200);
await expect(resultsNote(page)).toHaveText(`Showing 200 of 250 · ${VIEWER_HINT}`);
await outcomeButtons(page).filter({ hasText: 'No usable metadata' }).click();
await expect(fileRows(page)).toHaveCount(10);
await expect(resultsNote(page)).toHaveCount(0);
// Every listed row is in the page and it stays responsive: a filter answers at once.
await outcomeButtons(page).first().click();
await filter(page).fill('file_00199');
await expect(fileRows(page)).toHaveCount(1);
await expect(resultsNote(page)).toHaveText(`Showing 1 of 250 · ${VIEWER_HINT}`);
// Review M12: with the list capped and the log closed, more files only update the counters; the detail is read
// again only when the log opens with new lines.
// The reads from opening the view have settled: one returned the full listing and none is in flight. The view
// re-reads only while what the summary lists differs from the last read, so no further read is due.
await expect.poll(() => detailReads.full && detailReads.inFlight === 0).toBe(true);
const reads = [];
page.on('request', item => { if (new URL(item.url()).pathname === `/prompt_manager/jobs/${job.id}`) reads.push(Date.now()); });
await step(request, job.id, { processed: 300, bulk: { group: 'added', count: 40 } });
await expect(tiles(page).first().locator('dd')).toHaveText('290');
await page.waitForTimeout(1500);
expect(reads).toEqual([]);
await scanLog(page).locator('summary').click();
await expect.poll(() => reads.length).toBe(1);
await expect(scanLog(page).locator('pre')).toContainText('bulk/file_00039.png');
});
test('cancel scan ends cancelled with partial counts', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 9, counters: { added: 7, errors: 2 } });
await openJob(page, job.id);
// From the keyboard: when the job ends and Cancel scan goes away, focus moves to the job's heading (review M4).
await heading(page).getByRole('button', { name: 'Cancel scan', exact: true }).focus();
await page.keyboard.press('Enter');
await expect(toast(page)).toHaveText('Scan cancelled · 7 added');
await expect(heading(page).locator('h1')).toBeFocused();
await expect(heading(page).locator('p.muted')).toHaveText(/^Cancelled \d\d:\d\d at a file boundary · 7 added · completed work is kept$/);
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).toHaveCount(0);
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Cancelled after 9 of 19,622 files');
expect((await tileValues(page)).map(([, , value]) => value)).toEqual(['7', '0', '0', '0', '2']);
expect((await listed(request))[0].state).toBe('cancelled');
// A failed cancel request keeps the button and says so.
const second = await startJob(request);
await openJob(page, second.id);
await page.route('**/cancel', route => route.fulfill({ status: 500, json: { success: false, error: 'nope' } }));
await heading(page).getByRole('button', { name: 'Cancel scan', exact: true }).click();
await expect(toast(page)).toHaveText('The job could not be cancelled. Try again.');
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).not.toHaveAttribute('aria-disabled', 'true');
});
test('busy start names the running job', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const running = await startJob(request, { title: 'Database maintenance', kind: 'maintenance' });
await startFromImport(page);
await expect(toast(page)).toHaveText('Another job is running: Database maintenance.');
await expect(toast(page).getByRole('link', { name: 'Database maintenance', exact: true })).toHaveAttribute('href', `#/activity/${running.id}`);
await expect(page).toHaveURL(/#\/images$/);
expect((await listed(request)).map(item => item.id)).toEqual([running.id]);
// Review M14: a toast with a link outlasts the plain 6.5 s, so the link stays reachable.
await page.waitForTimeout(7000);
await expect(toast(page).getByRole('link', { name: 'Database maintenance', exact: true })).toBeVisible();
await toast(page).getByRole('link', { name: 'Database maintenance', exact: true }).click();
await expect(heading(page).locator('h1')).toHaveText('Database maintenance');
// A job of another kind cancels as "Cancel job".
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report', 'Cancel job']);
});
test('an unknown or restarted job view says so', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await page.goto(`${workspace}#/activity/${'0'.repeat(32)}`);
await expect(main(page).locator('section.surface p.empty-state')).toHaveText('This job is not in this session’s Activity. Jobs are kept until ComfyUI restarts.');
await expect(main(page).getByRole('link', { name: 'All activity', exact: true })).toHaveAttribute('href', '#/activity');
const job = await startJob(request);
await step(request, job.id, { processed: 5, records: [{ group: 'added', path: 'kept/row.png', detail: 'Added' }] });
await openJob(page, job.id);
await expect(fileRows(page)).toHaveCount(1);
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
await expect(heading(page).locator('p.muted')).toHaveText(LOST);
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).toHaveCount(0);
// A lost job has no report on the server (review M9): Download report is disabled rather than failing.
await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeDisabled();
// What was read before the restart stays on the page.
await expect(fileRows(page)).toHaveCount(1);
await expect(pill(page)).toBeHidden();
});
// Review M3: a finished job the server no longer lists is gone from this tab too, and its open view says so.
test('a finished job the server pruned leaves its open view honest', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 1, records: [{ group: 'added', path: 'kept/row.png', detail: 'Added' }] });
await finish(request, job.id);
await openJob(page, job.id);
await expect(fileRows(page)).toHaveCount(1);
await filter(page).focus();
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
// The next periodic list read (15 s) no longer has the finished job.
test.slow();
await expect(main(page).locator('section.surface p.empty-state')).toHaveText('This job is not in this session’s Activity. Jobs are kept until ComfyUI restarts.', { timeout: 20000 });
await expect(progressSection(page)).toHaveCount(0);
await expect(heading(page).locator('h1')).toHaveText('Activity');
await expect(heading(page).locator('h1')).toBeFocused();
await expect(page).toHaveTitle('Activity · Prompt Manager');
});
// Activity-2 Task 5 (D59): Download report and the log's scroll position.
test('download report saves the job JSON', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 2, records: [
{ group: 'added', path: 'sub/a.png', root: 'output', detail: 'Added' },
{ group: 'errors', path: 'broken/b.png', root: 'output', detail: 'Error' },
] });
await finish(request, job.id);
await openJob(page, job.id);
const button = heading(page).getByRole('button', { name: 'Download report', exact: true });
await expect(button).toBeEnabled();
const [download] = await Promise.all([page.waitForEvent('download'), button.click()]);
expect(download.suggestedFilename()).toMatch(/^prompt-manager-scan-\d{8}-\d{4}\.json$/);
const path = await download.path();
const report = JSON.parse(await (await import('node:fs/promises')).readFile(path, 'utf8'));
expect(report.job.id).toBe(job.id);
expect(report.outcomes.added.rows.map(row => row.path)).toEqual(['sub/a.png']);
expect(report.outcomes.errors.truncated).toBe(false);
expect(report.log.length).toBeGreaterThan(0);
await expect(toast(page)).toHaveText('Job report downloaded.');
});
test('log keeps the reader\'s scroll position', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
const more = (from, count) => Array.from({ length: count }, (_, index) => ({ group: 'known', path: `log/f_${from + index}.png`,
root: 'output', detail: 'Known' }));
await step(request, job.id, { processed: 60, records: more(0, 60) });
await openJob(page, job.id);
await scanLog(page).locator('summary').click();
const pre = scanLog(page).locator('pre');
await expect(pre).toContainText('log/f_59.png');
const metrics = () => pre.evaluate(node => ({ top: node.scrollTop, max: node.scrollHeight - node.clientHeight }));
// Opening starts at the newest line.
await expect.poll(async () => { const m = await metrics(); return m.max > 0 && m.top >= m.max - 2; }).toBe(true);
// Pinned to the bottom: new lines are followed.
await step(request, job.id, { processed: 90, records: more(60, 30) });
await expect(pre).toContainText('log/f_89.png');
await expect.poll(async () => { const m = await metrics(); return m.top >= m.max - 2; }).toBe(true);
// Scrolled up: the position stays while more lines arrive.
await pre.evaluate(node => { node.scrollTop = 40; });
await step(request, job.id, { processed: 120, records: more(90, 30) });
await expect(pre).toContainText('log/f_119.png');
expect((await metrics()).top).toBe(40);
// Closed and opened again (review M6): every open starts at the newest line, also after lines arrived meanwhile.
await scanLog(page).locator('summary').click();
await expect(scanLog(page)).not.toHaveAttribute('open', '');
await step(request, job.id, { processed: 150, records: more(120, 30) });
await scanLog(page).locator('summary').click();
await expect(pre).toContainText('log/f_149.png');
await expect.poll(async () => { const m = await metrics(); return m.max > 0 && m.top >= m.max - 2; }).toBe(true);
});
test('a failed report download says so and leaves the button usable', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await finish(request, job.id);
await openJob(page, job.id);
await page.route(`**/prompt_manager/jobs/${job.id}/report`, route => route.fulfill({ status: 500, body: 'no' }));
const button = heading(page).getByRole('button', { name: 'Download report', exact: true });
await button.click();
await expect(toast(page)).toHaveText('The report could not be downloaded. Try again.');
await expect(button).not.toHaveAttribute('aria-disabled', 'true');
});
+1 -1
View File
@@ -1,5 +1,5 @@
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {readFile,writeFile} from 'node:fs/promises';
import {createHash} from 'node:crypto';
+72 -1
View File
@@ -1,5 +1,5 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
import { seedConcept, openConcept, openProduction } from './concept-inventory.mjs';
const workspace = '/prompt_manager/workspace#/images';
@@ -1487,6 +1487,30 @@ test('a star press leaves the continuation status, count and caption date untouc
expect(await page.evaluate(() => window.__statusMutations)).toEqual([]);
});
// Opened before the source and folder discovery answered (a slow server), Refine lists them once it does.
test('Refine opened before discovery settles lists the discovered sources and folders', async ({ page }) => {
let release, started;
const held = new Promise(resolve => { release = resolve; }), hit = new Promise(resolve => { started = resolve; });
await page.route('**/prompt_manager/gallery/subfolders?*', async route => { started(); await held; await route.continue(); }, { times: 1 });
await page.goto(workspace);
await expect(cards(page)).toHaveCount(2);
await hit;
await filtersButton(page).click();
const source = refineDialog(page).getByLabel('Source', { exact: true });
const folder = refineDialog(page).getByLabel('Folder', { exact: true });
const values = select => select.locator('option').evaluateAll(options => options.map(option => option.value));
expect(await values(source)).toEqual(['all']);
expect(await values(folder)).toEqual(['']);
release();
await expect.poll(() => values(source)).toEqual(['all', '0', '1']);
expect(await values(folder)).toEqual(['', 'shared']);
await expect(source).toHaveValue('all');
await source.selectOption('1');
await refineDialog(page).getByRole('button', { name: 'Apply filters', exact: true }).click();
await expect(cards(page)).toHaveCount(1);
await expect(cards(page)).toHaveAttribute('data-image-key', '1:shared/same.png');
});
// #249 final review m1: as in the old field row, a Source offers only its own folders.
test('Refine offers only the chosen source folders', async ({ page, request }) => {
await expect(await request.post('/__test__/add-nested-image')).toBeOK();
@@ -2013,3 +2037,50 @@ test('Set category limits the field to the server length, shows the server count
await expect(bulkbar(page)).toBeEmpty();
expect(answers).toHaveLength(0);
});
// Activity-1 Task 3 (D55): Import → "Scan for metadata" → Review scan opens the concept's "Review metadata scan".
const scanReview = page => page.getByRole('dialog', { name: 'Review metadata scan', exact: true });
async function openScanReview(page) {
await page.getByRole('button', { name: 'Import', exact: true }).click();
const row = importDialog(page).locator('.operation').filter({ has: page.getByRole('heading', { name: 'Scan for metadata', exact: true }) });
await row.getByRole('button', { name: 'Review scan', exact: true }).click();
await expect(scanReview(page)).toBeVisible();
}
test('scan for metadata opens the review dialog from Import', async ({ page, request }) => {
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
await page.setViewportSize({ width: 1600, height: 1100 });
await page.goto(workspace);
await expect(cards(page)).toHaveCount(2);
// Rescan stays the quick refresh; the metadata scan is its own row under it.
await page.getByRole('button', { name: 'Import', exact: true }).click();
await expect(importDialog(page).locator('.operation h3')).toHaveText(['Inspect a local image', 'Scan an output folder', 'Scan for metadata']);
await expect(importDialog(page).locator('.operation').nth(2).locator('p')).toHaveText('Discover and enrich existing files using the configured sources.');
await page.keyboard.press('Escape');
await openScanReview(page);
const dialog = scanReview(page);
await expect(dialog.locator('dl.meta-grid > div > dt')).toHaveText(['Sources', 'Scope']);
await expect(dialog.locator('dl.meta-grid > div > dd')).toHaveText(['2 output folders', 'New and unresolved files']);
await expect(dialog.locator('#modal-body > p.small')).toHaveText('Enrich positive / negative associations, reuse canonical text and report unchanged files. Do not inflate run counts or recreate existing thumbnails.');
await expect(dialog.locator('.notice h3')).toHaveText('Back up the database first (recommended)');
await expect(dialog.locator('.dialog-actions button')).toHaveText(['Cancel', 'Start scan']);
await expect(dialog.getByRole('button', { name: 'Start scan', exact: true })).toHaveClass(/ca-primary/);
// Back up now downloads the existing backup, toasts, and keeps the dialog open for the scan.
const download = page.waitForEvent('download');
await dialog.getByRole('button', { name: 'Back up now', exact: true }).click();
expect((await download).suggestedFilename()).toMatch(/^prompts_backup_\d{8}_\d{6}\.db$/);
await expect(toast(page)).toHaveText('Database backup downloaded.');
await expect(dialog).toBeVisible();
// A failed backup says so and downloads nothing.
await page.route('**/prompt_manager/backup', route => route.fulfill({ status: 500, json: { success: false, error: 'disk full' } }));
await dialog.getByRole('button', { name: 'Back up now', exact: true }).click();
await expect(toast(page)).toHaveText('The database could not be backed up. Try again.');
// Cancel closes it and starts nothing.
await dialog.getByRole('button', { name: 'Cancel', exact: true }).click();
await expect(scanReview(page)).toHaveCount(0);
await expect(page.getByRole('button', { name: 'Import', exact: true })).toBeFocused();
const jobs = await (await request.get('/prompt_manager/jobs')).json();
expect(jobs.jobs).toEqual([]);
});
@@ -1,6 +1,6 @@
import {writeFile} from 'node:fs/promises';
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {inventoryOf,mapConcept,diffInventory} from './library-editor-canonical-comparison.mjs';
+23 -5
View File
@@ -1,5 +1,5 @@
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
const route='/prompt_manager/workspace#/library';
const cards=page=>page.locator('article[data-prompt-id]');
@@ -25,6 +25,8 @@ async function loadShelfPages(page,more,pages) {
await more.evaluate(node=>node.click());await response;await expect(more).toHaveAttribute('aria-disabled','false');
}
}
/** Resolves in the rendering frame after every frame callback the page had already queued (callbacks run in request order). */
const nextFrame=page=>page.evaluate(()=>new Promise(resolve=>requestAnimationFrame(()=>resolve())));
async function openHistory(page,text='Cat portrait') {
await cards(page).filter({has:page.getByText(text,{exact:true})}).getByRole('button',{name:'Explore history',exact:true}).click();
await expect(history(page)).toBeVisible();
@@ -103,7 +105,8 @@ test('library infinite loading keeps unique parents and keyboard-accessible hist
const appended=await (await nextPage).json();expect(appended.items.length).toBeGreaterThan(0);hasMore=appended.has_more;
if(hasMore)await expect(more).toHaveAttribute('aria-disabled','false');
}
await expect(more).toBeHidden();expect(await cards(page).count()).toBeLessThan(80);
// The last page's render queued the window update for the next frame; count after that frame, not before it.
await expect(more).toBeHidden();await nextFrame(page);expect(await cards(page).count()).toBeLessThan(80);
const ids=await cards(page).evaluateAll(nodes=>nodes.map(node=>node.dataset.promptId));expect(new Set(ids).size).toBe(ids.length);
await expect(page.getByRole('button',{name:/next page|previous page/i})).toHaveCount(0);
const firstId=await cards(page).first().getAttribute('data-prompt-id');
@@ -376,14 +379,18 @@ async function observeLibrary(page, hold=false) {
holdFrames(){holdingFrames=true;},
releaseFrames(){holdingFrames=false;const callbacks=[...heldFrames.values()];heldFrames.clear();for(const callback of callbacks)callback();},
equal(){for(const record of records){const entries=[...record.latest.values()].filter(entry=>record.targets.has(entry.target));if(entries.length)record.callback(entries,record.native);}},
// Resolves once two consecutive rendering frames find no Library frame queued. Each check runs inside a native
// frame callback, so a frame the page queues later in that update (a scroll event, a ResizeObserver delivery) is
// still pending there. A timer is not a frame: under CPU load a 60 ms wait could end before the scroll's frame.
// A frame that never drains (a rAF loop, or settling while frames are held) fails after `limit` frames with the stats.
settled(limit=600){return new Promise((resolve,reject)=>{let quiet=0,frames=0;const check=()=>{frames++;quiet=pending.size?0:quiet+1;
if(quiet>=2)resolve();else if(frames>=limit)reject(new Error(`Library never settled within ${limit} frames: ${JSON.stringify(window.libraryObservation.stats())}`));else raf(check);};raf(check);});},
stats(){return {requested,pending:pending.size,heldFrames:heldFrames.size,measurements:records.reduce((sum,record)=>sum+record.entries.size,0),mutations,unmounted,targets:records.reduce((sum,record)=>sum+record.targets.size,0)};},
};
},{hold});
}
async function settleLibrary(page) {
await expect.poll(async()=>page.evaluate(()=>window.libraryObservation.stats().pending)).toBe(0);
await page.evaluate(()=>new Promise(resolve=>setTimeout(resolve,60)));
await expect.poll(async()=>page.evaluate(()=>window.libraryObservation.stats().pending)).toBe(0);
await page.evaluate(()=>window.libraryObservation.settled());
}
async function firstVisibleParent(page) {
return cards(page).evaluateAll(nodes=>{
@@ -683,6 +690,17 @@ test('title and Explore history are buttons that open history',async({page,reque
await page.keyboard.press('ArrowLeft');await expect(cards(page).nth(1).locator('.card-title > button')).toBeFocused();
});
test('settling fails with the observation stats when a Library frame stays queued',async({page,request})=>{
await request.post('/__test__/reset',{data:{native_library:true,extra_prompts:200}});
await observeLibrary(page);await page.addInitScript(()=>{window.IntersectionObserver=undefined;});
await page.goto(route);await expect(cards(page).first()).toBeVisible();await settleLibrary(page);
await page.evaluate(()=>{window.libraryObservation.holdFrames();dispatchEvent(new Event('scroll'));});
await expect.poll(()=>page.evaluate(()=>window.libraryObservation.stats().heldFrames)).toBeGreaterThan(0);
const failure=await page.evaluate(()=>window.libraryObservation.settled(30).then(()=>'settled',error=>error.message));
expect(failure).toMatch(/^Library never settled within 30 frames: \{.*"heldFrames":[1-9]/);
await page.evaluate(()=>window.libraryObservation.releaseFrames());await settleLibrary(page);
});
test('virtualized rows keep measured heights with the clamp',async({page,request,isMobile})=>{
await request.post('/__test__/reset',{data:{native_library:true,extra_prompts:200}});
await observeLibrary(page);await page.addInitScript(()=>{window.IntersectionObserver=undefined;});
+1 -1
View File
@@ -1,5 +1,5 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
// Saved views (#249 D16, maintainer ruling 2026-10-21): the toolbar's Save view (concept app.js a359, a1330–1346),
// the sidebar SAVED VIEWS rows (a331–335) and `#/saved/<id>` (a1158–1162), on Images and the Prompt library.
+5 -4
View File
@@ -3,12 +3,12 @@ import { readFile } from 'node:fs/promises';
import { createHash } from 'node:crypto';
import { captureCanonicalState, historyGeometry, compareCanonicalHistory, nativeDialogState, exerciseCurrentPairSend, assertRecordedHistory } from './history-canonical-comparison.mjs';
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
test.use({ locale: 'en-US', timezoneId: 'UTC', reducedMotion: 'reduce' });
const workspace = '/prompt_manager/workspace';
const primary = ['images', 'library', 'tags', 'inspect'];
const tools = ['system', 'overview'];
const tools = ['activity', 'system', 'overview'];
async function noOverflow(page) {
expect(await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth)).toBeLessThanOrEqual(1);
@@ -556,7 +556,7 @@ test('approved icons are decorative safe SVGs and reject unknown vocabulary', as
await page.goto(`${workspace}#/images`);
const icons = await page.evaluate(async () => {
const { createWorkspaceIcon } = await import('/prompt_manager/js/workspace/icons.mjs');
const names = ['images', 'library', 'tags', 'upload', 'overview', 'settings', 'grid', 'list', 'masonry', 'search', 'info', 'copy', 'branch', 'star', 'tagImage', 'folder', 'history', 'review'];
const names = ['images', 'library', 'tags', 'upload', 'overview', 'activity', 'settings', 'grid', 'list', 'masonry', 'search', 'info', 'copy', 'branch', 'star', 'tagImage', 'folder', 'history', 'review'];
const otherDocument = document.implementation.createHTMLDocument();
const rendered = names.map(name => {
const icon = createWorkspaceIcon(name, otherDocument);
@@ -575,7 +575,8 @@ test('approved icons are decorative safe SVGs and reject unknown vocabulary', as
expect(icon).toMatchObject({ svg: true, className: 'icon', owner: true, viewBox: '0 0 24 24', hidden: 'true', focusable: 'false', fill: 'none', stroke: 'currentColor', width: '1.65', children: 1 });
expect(icon.path.length).toBeGreaterThan(10);
}
// D28 ruling: Overview's own outline home glyph, not the concept's `activity` pulse ('M3 12h4l3-8 4 16 3-8h4').
// D28 ruling: Overview's own outline home glyph; the concept's `activity` pulse is the Activity row's.
expect(icons.rendered.find(icon => icon.name === 'activity').path).toBe('M3 12h4l3-8 4 16 3-8h4');
expect(icons.rendered.find(icon => icon.name === 'overview').path).toBe('M3 11 12 4l9 7M5 10v10h14V10M10 20v-6h4v6');
expect(new Set(icons.rendered.map(icon => icon.path)).size).toBe(icons.rendered.length);
});
+1 -1
View File
@@ -1,6 +1,6 @@
import {startCoverage,saveCoverage} from '../../tools/vendor/browser-coverage.mjs';
import {writeFile} from 'node:fs/promises';
import {test,expect} from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import {test,expect} from './fixtures.mjs';
import {captureCanonicalState} from './history-canonical-comparison.mjs';
import {pageInventoryOf,inventoryOf,diffInventory,mapVocabularyConcept,mapVocabularyDialog} from './vocabulary-canonical-comparison.mjs';
+30 -13
View File
@@ -1,5 +1,5 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from '../../tools/vendor/node_modules/@playwright/test/index.mjs';
import { test, expect } from './fixtures.mjs';
const workspace = '/prompt_manager/workspace';
const statistic = (page, label) => page.locator('#workspace-main dt')
@@ -20,6 +20,8 @@ const routes = [
['system', 'System', '/prompt_manager/admin'],
// #249 D51: SAVED VIEWS' Favorites, Images filtered to favorites.
['favorites', 'Favorites', null],
// #249 Activity-1: the concept's Activity page in the tools-and-system group.
['activity', 'Activity', null],
];
async function countThemeWrites(page) {
@@ -46,6 +48,8 @@ test.beforeEach(async ({ page, request, browserName }) => {
});
test.afterEach(async ({ page, browserName }) => {
// A route handler still running at teardown must not fail the test it outlived.
await page.unrouteAll({ behavior: 'ignoreErrors' });
await saveCoverage(page, browserName);
expect(page.workspaceErrors).toEqual([]);
});
@@ -176,7 +180,8 @@ test('theme System follows ComfyUI and persists an explicit light or dark prefer
await expect(theme).toHaveAttribute('data-ca-theme', 'light');
});
// Task A4 (D4, D5): the concept topbar (index.html 45-63, style.css 225-277) without the prototype badge or Activity.
// Task A4 (D4, D29): the concept topbar (index.html 45-63, style.css 225-277) without the prototype badge; Activity
// (D52, D53) shows its button, and its pill only while a job runs.
test('topbar matches the concept actions', async ({ page }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await page.goto(`${workspace}#/images`);
@@ -204,7 +209,7 @@ test('topbar matches the concept actions', async ({ page }) => {
expect(shape).toEqual({
children: ['div.breadcrumbs', 'div.top-actions'],
crumbs: ['ComfyAssets', 'span:/', 'span#workspace-breadcrumb:Images'],
actions: ['label.theme-label', 'a.canvas-link'],
actions: ['a.activity-pill', 'button.', 'label.theme-label', 'a.canvas-link'],
label: 'Theme', labelFor: 'theme',
options: [['dark', 'Dark'], ['light', 'Light'], ['system', 'System']],
arrow: [['↗', 'true']],
@@ -218,11 +223,12 @@ test('topbar matches the concept actions', async ({ page }) => {
await expect(link).toHaveAttribute('href', '/');
expect(await link.evaluate(node => node.href === `${location.origin}/`)).toBe(true);
await expect(page.getByRole('combobox', { name: 'Appearance' })).toHaveCount(0);
await expect(page.getByRole('button', { name: /^Activity/ })).toHaveCount(0);
await expect(page.locator('.concept-badge, #activity-open')).toHaveCount(0);
await expect(page.getByRole('button', { name: 'Activity', exact: true })).toHaveCount(1);
await expect(page.locator('#activity-pill')).toBeHidden();
await expect(page.locator('.concept-badge')).toHaveCount(0);
await page.locator('nav [data-route="library"]').click();
await expect(page.locator('#workspace-breadcrumb')).toHaveText('Prompt library');
await expect(topbar.locator('.top-actions > *')).toHaveCount(2);
await expect(topbar.locator('.top-actions > *')).toHaveCount(4);
});
// Concept style.css 1375-1391: at <=700px the breadcrumbs hide, the actions spread across the bar and the Theme
@@ -242,13 +248,18 @@ test('mobile topbar hides breadcrumbs and spreads actions', async ({ page }) =>
expected: bar.width - 28,
label: getComputedStyle(node.querySelector('.theme-label')).fontSize,
select: getComputedStyle(node.querySelector('#theme')).fontSize,
labelLeft: box(node.querySelector('.theme-label')).left - bar.left,
activityLeft: box(node.querySelector('#activity-open')).left - bar.left,
actionsGap: [box(node.querySelector('#activity-open')).right, box(node.querySelector('.theme-label')).left,
box(node.querySelector('.theme-label')).right, box(node.querySelector('.canvas-link')).left],
linkRight: bar.right - box(node.querySelector('.canvas-link')).right,
};
});
expect(layout).toMatchObject({ padding: '14px', actions: { justify: 'space-between', gap: '8px' }, label: '0px', select: '11px' });
expect(Math.abs(layout.actions.width - layout.expected)).toBeLessThanOrEqual(1);
expect(Math.abs(layout.labelLeft - 14)).toBeLessThanOrEqual(1);
// The concept's three items spread with space-between: Activity first, Theme between equal gaps, ComfyUI last.
expect(Math.abs(layout.activityLeft - 14)).toBeLessThanOrEqual(1);
const [activityRight, themeLeft, themeRight, linkLeft] = layout.actionsGap;
expect(Math.abs((themeLeft - activityRight) - (linkLeft - themeRight))).toBeLessThanOrEqual(1);
expect(Math.abs(layout.linkRight - 14)).toBeLessThanOrEqual(1);
await expect(page.getByRole('link', { name: 'ComfyUI', exact: true })).toBeVisible();
await theme.selectOption('light');
@@ -506,18 +517,18 @@ test('a slow Images count does not delay the Library and Tags badges', async ({
await expect(navCount(page, 'images')).toHaveText(String(totals.images));
});
test('sidebar bottom holds System and Overview with a neutral connection', async ({ page }) => {
test('sidebar bottom holds Activity, System and Overview with a neutral connection', async ({ page }) => {
await page.goto(`${workspace}#/system`);
const bottom = page.locator('aside.sidebar > .sidebar-bottom');
expect(await childShape(bottom)).toEqual(['nav#system-nav', 'div.connection']);
const system = page.locator('nav#system-nav');
await expect(system).toHaveAttribute('aria-label', 'Tools and system');
expect(await system.locator(':scope > a').evaluateAll(nodes => nodes.map(node => [node.getAttribute('href'), node.textContent]))).toEqual([
['#/system', 'System'], ['#/overview', 'Overview'],
['#/activity', 'Activity'], ['#/system', 'System'], ['#/overview', 'Overview'],
]);
await expect(system.locator('.nav-count')).toHaveCount(0);
await expect(system.locator('a[href="#/system"]')).toHaveAttribute('aria-current', 'page');
await expect(system.locator('svg.icon')).toHaveCount(2);
await expect(system.locator('svg.icon')).toHaveCount(3);
const connection = bottom.locator('.connection');
expect(await childShape(connection)).toEqual(['i', 'div']);
await expect(connection.locator(':scope > div')).toHaveText('ComfyUI workspaceYour library, on your server');
@@ -679,8 +690,14 @@ async function serveUnderProxyMount(page) {
await page.route(url => url.pathname.startsWith(`${PROXY_MOUNT}/`), async route => {
const url = new URL(route.request().url());
url.pathname = url.pathname.slice(PROXY_MOUNT.length);
const response = await route.fetch({ url: url.href });
await route.fulfill({ response });
// A request still in flight when the test ends (a thumbnail the last route started) meets a page that is
// closing: its fetched response is disposed under it. Only that teardown error is ignored.
try {
const response = await route.fetch({ url: url.href });
await route.fulfill({ response });
} catch (error) {
if (!/has been disposed|has been closed|Target page, context or browser has been closed/.test(String(error?.message))) throw error;
}
});
return escaped;
}
+194
View File
@@ -0,0 +1,194 @@
"""Fake Activity jobs for the browser tests (#249 Activity-1).
A fake job is a real registry job whose runner waits for commands instead of
reading files, so a test sets its progress and counters exactly:
- ``POST /__test__/jobs/start`` starts one (``kind``, ``title``, ``total``,
``output_folders``) and returns its summary, or 409 while one runs;
- ``POST /__test__/jobs/{id}/step`` records ``records`` (``{group, path,
root, root_index, detail}`` file outcomes, counted, listed and logged as the scan
records them) and ``bulk`` (``{group, count, root}``: ``count`` numbered
rows), sets ``processed`` (and ``total``) and ``counters``, then publishes
the progress to every subscriber;
- ``POST /__test__/jobs/{id}/finish`` ends it as ``finished``, ``cancelled``
or ``failed``; a cancel through the real route ends it ``cancelled``;
- ``POST /__test__/jobs/restart`` acts out a ComfyUI restart: the open job
streams drop without a final event and the registry starts empty.
"""
import asyncio
import re
from aiohttp import web
EVENTS_PATH = re.compile(r"^/prompt_manager/jobs/[0-9a-f]{32}/events$")
FINAL_STATES = {"finished", "cancelled", "failed"}
POLL_SECONDS = 0.02
PREFIXES = {"added": "+", "known": "=", "review": "?", "errors": "!"}
PREFIXES["no_metadata"] = "-"
ROOT_INDEXES = {"output": 0, "input": 1}
def outcome_records(data):
"""The step's file outcomes: its ``records``, then its ``bulk`` rows."""
records = data.get("records", [])
bulk = data.get("bulk")
if not isinstance(records, list):
raise web.HTTPBadRequest()
rows = []
for record in records:
if not isinstance(record, dict) or record.get("group") not in PREFIXES:
raise web.HTTPBadRequest()
root = str(record.get("root", "output"))
rows.append(
{
"group": record["group"],
"path": str(record.get("path", "")),
"root": root,
# The fixture's listing order: output is root 0, input root 1.
"root_index": record.get("root_index", ROOT_INDEXES.get(root)),
"detail": str(record.get("detail", "")),
}
)
if bulk is not None:
if not isinstance(bulk, dict) or bulk.get("group") not in PREFIXES:
raise web.HTTPBadRequest()
total = bulk.get("count")
if isinstance(total, bool) or not isinstance(total, int) or total < 0:
raise web.HTTPBadRequest()
rows.extend(
{
"group": bulk["group"],
"path": f"bulk/file_{index:05d}.png",
"root": str(bulk.get("root", "output")),
"root_index": 0,
"detail": f"Bulk row {index}",
}
for index in range(total)
)
return rows
def register_activity_commands(app, routes, api):
from py.api.jobs import SCAN_GROUP_NAMES, JobBusy, JobRegistry
streams = set()
endings = {}
@web.middleware
async def track_streams(request, handler):
if not EVENTS_PATH.match(request.path):
return await handler(request)
transport = request.transport
streams.add(transport)
try:
return await handler(request)
finally:
streams.discard(transport)
app.middlewares.append(track_streams)
def fake_runner(ending):
async def run(job):
while not ending.done() and not job.cancelled:
await asyncio.sleep(POLL_SECONDS)
state = ending.result() if ending.done() else "cancelled"
job.publish({"type": "complete", "progress": job.progress})
job.finish(state)
return run
def job_or_404(request):
job = api._jobs.get(request.match_info["job_id"])
if job is None or job.id not in endings:
raise web.HTTPNotFound()
return job
async def body(request):
data = await request.json() if request.can_read_body else {}
if not isinstance(data, dict):
raise web.HTTPBadRequest()
return data
def count(value, default=0):
if value is None:
return default
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
raise web.HTTPBadRequest()
return value
@routes.post("/__test__/jobs/start")
async def start_job(request):
data = await body(request)
ending = asyncio.get_running_loop().create_future()
try:
job = api._jobs.start(
str(data.get("kind", "scan")),
str(data.get("title", "Scan for metadata")),
record_changing=True,
scope={
"files": str(data.get("files", "new")),
"output_folders": count(data.get("output_folders"), 2),
},
runner=fake_runner(ending),
groups=SCAN_GROUP_NAMES,
)
except JobBusy:
raise web.HTTPConflict()
job.update(total=count(data.get("total")))
endings[job.id] = ending
return web.json_response({"success": True, "job": job.summary()})
@routes.post("/__test__/jobs/{job_id}/step")
async def step_job(request):
job = job_or_404(request)
data = await body(request)
total = count(data.get("total"), job.total)
processed = count(data.get("processed"), job.processed)
counters = data.get("counters", {})
if not isinstance(counters, dict) or set(counters) - set(job.counters):
raise web.HTTPBadRequest()
for record in outcome_records(data):
group = record["group"]
path = record["path"]
detail = record["detail"]
job.record(group, record, f"{PREFIXES[group]} {path} · {detail}")
for key, value in counters.items():
job.counters[key] = count(value)
progress = processed * 100 // total if total else 0
job.update(
progress=progress,
processed=processed,
total=total,
status_line=f"Processing file {processed:,} of {total:,}",
)
job.publish({"type": "progress", "progress": progress})
return web.json_response({"success": True, "job": job.summary()})
@routes.post("/__test__/jobs/{job_id}/finish")
async def finish_job(request):
job = job_or_404(request)
state = (await body(request)).get("state", "finished")
if state not in FINAL_STATES:
raise web.HTTPBadRequest()
ending = endings[job.id]
if not ending.done():
ending.set_result(state)
while job.running:
await asyncio.sleep(POLL_SECONDS)
return web.json_response({"success": True, "job": job.summary()})
@routes.post("/__test__/jobs/restart")
async def restart(request):
old = api._jobs
api._jobs = JobRegistry(api.logger)
endings.clear()
for transport in list(streams):
transport.close()
tasks = [job.task for job in old.list() if job.task and not job.task.done()]
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
return web.json_response({"success": True})
+371
View File
@@ -0,0 +1,371 @@
// Activity (#249 Activity-1): the session job store behind the top-bar button and pill and the Activity page.
const test = require('node:test');
const assert = require('node:assert/strict');
const source = import('../../web/js/workspace/activity-store.mjs');
const ID_A = 'a'.repeat(32);
const ID_B = 'b'.repeat(32);
const ID_C = 'c'.repeat(32);
function summary(overrides = {}) {
return { id: ID_A, kind: 'scan', title: 'Scan for metadata', state: 'running', started_at: 1000, finished_at: null,
progress: 0, status_line: '', processed: 0, total: 0,
counters: { added: 0, known: 0, review: 0, no_metadata: 0, errors: 0 }, scope: { files: 'new', output_folders: 2 },
...overrides };
}
function apiError(status, message, code) {
return Object.assign(new Error(message), { status, code });
}
/** A fake world: the API's answers, every opened stream, the pending timers and the toasts. */
function harness() {
const world = { list: [], listError: null, posts: [], postResult: null, postError: null, streams: [], timers: [],
toasts: [], gets: 0 };
const cleanups = [];
const controller = new AbortController();
const scope = { signal: controller.signal, cleanup: fn => cleanups.push(fn) };
const api = {
async get(url, options) {
assert.equal(url, '/prompt_manager/jobs');
assert.equal(options.cache, 'no-store');
world.gets += 1;
if (world.listError) throw world.listError;
return { success: true, jobs: world.list };
},
async post(url, body) {
world.posts.push([url, body]);
if (world.postError) throw world.postError;
return world.postResult;
},
};
const connect = url => {
const stream = { url, closed: false, onmessage: null, onerror: null, close() { this.closed = true; } };
world.streams.push(stream);
return stream;
};
const timers = {
setTimeout(fn, delay) { const timer = { fn, delay, cleared: false }; world.timers.push(timer); return timer; },
clearTimeout(timer) { timer.cleared = true; },
};
world.dispose = () => { controller.abort(); cleanups.forEach(fn => fn()); };
world.fire = async () => {
const timer = world.timers.find(item => !item.cleared && !item.fired);
timer.fired = true;
timer.fn();
await new Promise(resolve => setImmediate(resolve));
return timer.delay;
};
world.open = () => world.streams.filter(stream => !stream.closed);
return { world, api, scope, connect, timers, pollMs: 0, notify: message => world.toasts.push(message) };
}
const send = (stream, job) => stream.onmessage({ data: JSON.stringify(job) });
test('store merges list and SSE updates and caps backoff', async () => {
const { createActivityStore, BACKOFF_MAX_MS, LOST_DETAIL } = await source;
const { world, ...deps } = harness();
world.list = [summary(), summary({ id: ID_B, state: 'finished', finished_at: 900, counters: { added: 3, errors: 0 } })];
const store = createActivityStore(deps);
let changes = 0;
store.subscribe(() => { changes += 1; });
await store.load();
assert.equal(store.status, 'ready');
assert.deepEqual(store.jobs().map(job => [job.id, job.state]), [[ID_A, 'running'], [ID_B, 'finished']]);
assert.deepEqual(store.running().map(job => job.id), [ID_A]);
// One stream per running job only; a finished job is not followed.
assert.deepEqual(world.streams.map(stream => stream.url), [`/prompt_manager/jobs/${ID_A}/events`]);
assert.ok(changes >= 1);
// An SSE summary replaces the listed one; garbage and another job's summary are ignored.
const [stream] = world.streams;
send(stream, summary({ progress: 42, processed: 8240, total: 19622 }));
stream.onmessage({ data: 'not json' });
send(stream, summary({ id: ID_B, progress: 99 }));
assert.equal(store.find(ID_A).progress, 42);
assert.equal(store.find(ID_B).state, 'finished');
assert.deepEqual(world.toasts, []);
// The end of the job: the finish toast, and the stream is closed.
send(stream, summary({ state: 'finished', progress: 100, finished_at: 2000, counters: { added: 1420, errors: 18 } }));
assert.deepEqual(world.toasts, ['Scan finished · 1,420 added · 18 errors']);
assert.equal(stream.closed, true);
assert.deepEqual(store.running(), []);
// A new running job from another tab, then the server goes away: every re-read waits longer, up to 30 s.
world.list = [summary({ id: ID_C, state: 'running' })];
await store.load();
const second = world.open()[0];
assert.equal(second.url, `/prompt_manager/jobs/${ID_C}/events`);
world.listError = new Error('connection refused');
second.onerror();
assert.equal(second.closed, true);
const delays = [];
for (let index = 0; index < 8; index += 1) delays.push(await world.fire());
assert.deepEqual(delays, [1000, 2000, 4000, 8000, 16000, 30000, 30000, 30000]);
assert.ok(delays.every(delay => delay <= BACKOFF_MAX_MS));
// While the server cannot be read the running job shows as ended, so no pill stays.
assert.equal(store.status, 'error');
assert.equal(store.find(ID_C).state, 'lost');
assert.deepEqual(store.running(), []);
assert.equal(typeof LOST_DETAIL, 'string');
// The server is back without the job (a restart): it stays ended; finished jobs the server no longer lists are
// dropped, so the list never outgrows the server's (review M3).
world.listError = null;
world.list = [];
await world.fire();
assert.equal(store.status, 'ready');
assert.deepEqual(store.jobs().map(job => [job.id, job.state]), [[ID_C, 'lost']]);
assert.equal(world.timers.filter(timer => !timer.cleared && !timer.fired).length, 0, 'no retry after a good read');
assert.equal(world.toasts.length, 1, 'an ended job has no finish toast');
world.dispose();
});
test('a dropped stream of a still running job reattaches after the backoff, then resets it', async () => {
const { createActivityStore } = await source;
const { world, ...deps } = harness();
world.list = [summary()];
const store = createActivityStore(deps);
await store.load();
world.streams[0].onerror();
assert.equal(await world.fire(), 1000);
const again = world.open();
assert.equal(again.length, 1);
assert.equal(store.find(ID_A).state, 'running');
again[0].onerror();
assert.equal(await world.fire(), 2000);
send(world.open()[0], summary({ progress: 5 }));
world.open()[0].onerror();
assert.equal(await world.fire(), 1000, 'a received message resets the backoff');
// A second drop while a re-read is already scheduled does not schedule another.
world.open()[0].onerror();
store.load();
assert.equal(world.timers.filter(timer => !timer.cleared && !timer.fired).length, 1);
await new Promise(resolve => setImmediate(resolve));
world.dispose();
assert.ok(world.streams.every(stream => stream.closed), 'dispose closes every stream');
assert.ok(world.timers.every(timer => timer.cleared || timer.fired), 'dispose clears the pending re-read');
});
test('start returns busy with the running job', async () => {
const { createActivityStore } = await source;
const { world, ...deps } = harness();
const store = createActivityStore(deps);
world.postResult = { success: true, job: summary() };
const started = await store.start('scan');
assert.equal(started.job.id, ID_A);
assert.deepEqual(world.posts, [['/prompt_manager/jobs/scan', {}]]);
assert.deepEqual(world.toasts, ['Scan started.']);
assert.equal(world.open().length, 1, 'the started job is followed');
world.postError = apiError(409, 'Scan for metadata is already running', 'busy');
world.list = [summary()];
const busy = await store.start('scan');
assert.equal(busy.busy.id, ID_A);
assert.equal(busy.busy.title, 'Scan for metadata');
assert.equal(busy.message, 'Scan for metadata is already running');
assert.equal(world.toasts.length, 1, 'a refused start has no start toast');
world.postError = apiError(500, 'boom', null);
await assert.rejects(store.start('scan'), /boom/);
world.postError = null;
world.postResult = { success: true, job: { id: 'nope' } };
await assert.rejects(store.start('scan'), /Invalid job/);
world.postResult = { success: true, cancellation_requested: true };
assert.equal(await store.cancel(ID_A), true);
assert.deepEqual(world.posts.at(-1), [`/prompt_manager/jobs/${ID_A}/cancel`, {}]);
world.postResult = { success: true, cancellation_requested: false };
assert.equal(await store.cancel(ID_A), false);
world.dispose();
});
test('a list that is not a job list counts as a failed read', async () => {
const { createActivityStore } = await source;
const { world, api, ...deps } = harness();
const store = createActivityStore({ ...deps, api: { ...api, get: async () => ({ success: true }) } });
await store.load();
assert.equal(store.status, 'error');
assert.equal(world.timers.length, 1);
world.dispose();
await store.load();
assert.equal(store.status, 'error', 'a disposed store reads nothing');
});
test('a visible tab reads the list again every 15 s; a hidden one skips its turn', async () => {
const { createActivityStore, POLL_MS } = await source;
const { world, ...deps } = harness();
let visible = true;
const store = createActivityStore({ ...deps, pollMs: POLL_MS, isVisible: () => visible });
await store.load();
assert.equal(world.gets, 1);
// A job started in another tab appears on the next read.
world.list = [summary()];
assert.equal(await world.fire(), 15000);
assert.equal(world.gets, 2);
assert.deepEqual(store.running().map(job => job.id), [ID_A]);
visible = false;
assert.equal(await world.fire(), 15000);
assert.equal(world.gets, 2, 'a hidden tab does not read');
visible = true;
await world.fire();
assert.equal(world.gets, 3);
// A manual read replaces the pending poll rather than adding one.
await store.load();
assert.equal(world.timers.filter(timer => !timer.cleared && !timer.fired).length, 1);
world.dispose();
assert.ok(world.timers.every(timer => timer.cleared || timer.fired));
});
test('a late or older answer never brings an ended job back, so it is announced once', async () => {
const { createActivityStore } = await source;
const { world, api, ...deps } = harness();
const answers = [];
const slowApi = { ...api, get: () => new Promise(resolve => answers.push(resolve)) };
const store = createActivityStore({ ...deps, api: slowApi });
const first = store.load();
const second = store.load();
// The newer read answers first; the older one, answering later with stale data, is dropped.
answers[1]({ success: true, jobs: [summary()] });
await second;
answers[0]({ success: true, jobs: [] });
await first;
assert.deepEqual(store.jobs().map(job => job.state), ['running']);
send(world.open()[0], summary({ state: 'finished', counters: { added: 2, errors: 0 } }));
assert.deepEqual(world.toasts, ['Scan finished · 2 added · 0 errors']);
// A list read sent before the end still says running: the job stays finished and no stream reopens.
const third = store.load();
answers[2]({ success: true, jobs: [summary()] });
await third;
assert.equal(store.find(ID_A).state, 'finished');
assert.equal(world.open().length, 0);
assert.equal(world.toasts.length, 1);
// A stale failure is dropped too.
const failing = store.load();
const newest = store.load();
answers[4]({ success: true, jobs: [] });
await newest;
answers[3](Promise.reject(new Error('late failure')));
await failing;
assert.equal(store.status, 'ready');
world.dispose();
});
test('summaries are validated and the copy follows the spec', async () => {
const { parseJob, finishToast, startToast, pillParts, outcomeLine, backoffDelay } = await source;
for (const raw of [null, [], { id: 'x', state: 'running' }, summary({ state: 'lost' }), summary({ id: ID_A.toUpperCase() })]) {
assert.equal(parseJob(raw), null);
}
const loose = parseJob({ id: ID_A, state: 'running', progress: 140.4, processed: -1, total: 1.5, counters: { added: 'x', errors: 2 },
title: ' ', started_at: 'x', scope: [] });
assert.deepEqual(loose, { id: ID_A, kind: '', title: 'Background job', state: 'running', started_at: null, finished_at: null,
progress: 100, status_line: '', processed: 0, total: 0, counters: { added: 0, errors: 2 }, scope: {}, listed: {}, log_length: 0 });
assert.equal(parseJob(summary({ progress: -3 })).progress, 0);
const scan = parseJob(summary({ progress: 42, processed: 8240, total: 19622, counters: { added: 1, errors: 1 } }));
assert.deepEqual(pillParts(scan), { label: 'Scanning · 42%', files: ' · 8,240 / 19,622', text: 'Scanning · 42% · 8,240 / 19,622' });
assert.equal(pillParts({ ...scan, total: 0 }).text, 'Scanning · 42%');
assert.equal(pillParts({ ...scan, kind: 'thumbnails', title: 'Thumbnails' }).text, 'Thumbnails · 42% · 8,240 / 19,622');
assert.equal(finishToast({ ...scan, state: 'finished' }), 'Scan finished · 1 added · 1 error');
assert.equal(finishToast({ ...scan, state: 'cancelled' }), 'Scan cancelled · 1 added');
assert.equal(finishToast({ ...scan, state: 'failed' }), 'Scan failed · 1 added · 1 error');
assert.equal(finishToast({ ...scan, counters: {}, state: 'finished' }), 'Scan finished · 0 added · 0 errors');
const other = { ...scan, kind: 'backup', title: 'Backup' };
assert.equal(finishToast({ ...other, state: 'finished' }), 'Backup finished.');
assert.equal(finishToast({ ...other, state: 'cancelled' }), 'Backup was cancelled.');
assert.equal(finishToast({ ...other, state: 'failed' }), 'Backup failed.');
assert.equal(startToast(scan), 'Scan started.');
assert.equal(startToast(other), 'Backup started.');
assert.equal(outcomeLine(scan), '1 added · 1 error');
assert.equal(outcomeLine(other), '');
assert.deepEqual([-1, 0, 1, 4, 5, 6, 1e9, 0.5].map(backoffDelay), [1000, 1000, 2000, 16000, 30000, 30000, 30000, 1000]);
});
test('pagehide stops polling, closes streams and aborts the in-flight read; pageshow resumes', async () => {
const { createActivityStore } = await source;
const { world, api, ...deps } = harness();
const lifecycle = new EventTarget();
const signals = [];
let answer = null;
const pendingApi = { ...api, get: (url, options) => {
signals.push(options.signal);
world.gets += 1;
if (world.gets === 1) return Promise.resolve({ success: true, jobs: [summary()] });
return new Promise((resolve, reject) => {
answer = resolve;
options.signal.addEventListener('abort', () => reject(Object.assign(new Error('aborted'), { name: 'AbortError' })));
});
} };
const store = createActivityStore({ ...deps, api: pendingApi, pollMs: 15000, lifecycle });
await store.load();
assert.equal(world.open().length, 1);
const poll = world.timers.find(timer => !timer.cleared && timer.delay === 15000);
assert.ok(poll, 'a visible tab polls');
// A read in flight when the page goes away (a closed tab, a test teardown) is aborted, nothing is scheduled again.
const read = store.load();
lifecycle.dispatchEvent(new Event('pagehide'));
await read;
assert.equal(signals.at(-1).aborted, true);
assert.equal(world.open().length, 0);
assert.ok(world.timers.every(timer => timer.cleared || timer.fired));
assert.equal(store.find(ID_A).state, 'running', 'a hidden page marks nothing lost');
await store.load();
assert.equal(world.gets, 2, 'a suspended store reads nothing');
// Back from the cache, it reads the list again and follows the job.
const resumed = Object.assign(new Event('pageshow'), { persisted: true });
lifecycle.dispatchEvent(resumed);
await new Promise(resolve => setImmediate(resolve));
assert.equal(world.gets, 3);
answer({ success: true, jobs: [summary()] });
await new Promise(resolve => setImmediate(resolve));
assert.equal(world.open().length, 1);
world.dispose();
assert.equal(signals.at(-1).aborted, true, 'disposing aborts the store reads');
assert.equal(world.open().length, 0);
});
test('one failed read is not a restart; two in a row are, and a good read resets the backoff', async () => {
const { createActivityStore } = await source;
const { world, ...deps } = harness();
world.list = [summary(), summary({ id: ID_B, state: 'finished', finished_at: 900 })];
const store = createActivityStore(deps);
await store.load();
// A single failed read (a proxy's 502) keeps the job running and its stream open (review M1).
world.listError = apiError(502, 'Bad gateway', null);
await store.load();
assert.equal(store.status, 'error');
assert.equal(store.find(ID_A).state, 'running');
assert.equal(world.open().length, 1);
assert.equal(await world.fire(), 1000);
// The second failure in a row: the server is gone, so the running job ended with it.
assert.equal(store.find(ID_A).state, 'lost');
assert.equal(world.open().length, 0);
for (let index = 0; index < 3; index += 1) await world.fire();
// The server is back with a new running job: the backoff starts again at 1 s (review M2).
world.listError = null;
world.list = [summary({ id: ID_C })];
await world.fire();
assert.equal(store.status, 'ready');
assert.deepEqual(store.jobs().map(job => [job.id, job.state]), [[ID_C, 'running'], [ID_A, 'lost']]);
world.open()[0].onerror();
assert.equal(await world.fire(), 1000);
// A summary carries what its detail lists, for the job view's reads (review M12).
send(world.open()[0], summary({ id: ID_C, listed: { added: 3, errors: -1 }, log_length: 7 }));
assert.deepEqual([store.find(ID_C).listed, store.find(ID_C).log_length], [{ added: 3, errors: 0 }, 7]);
world.dispose();
});
test('numbers group the way the server writes them, whatever the browser locale', async () => {
const { pillParts, finishToast, parseJob } = await source;
const original = Number.prototype.toLocaleString;
Number.prototype.toLocaleString = function (locale, ...rest) { return original.call(this, locale ?? 'de-DE', ...rest); };
try {
const job = parseJob(summary({ progress: 41, processed: 8240, total: 19622, counters: { added: 1420, errors: 18 } }));
assert.equal(pillParts(job).text, 'Scanning · 41% · 8,240 / 19,622');
assert.equal(finishToast({ ...job, state: 'finished' }), 'Scan finished · 1,420 added · 18 errors');
} finally {
Number.prototype.toLocaleString = original;
}
});
+26 -3
View File
@@ -4,10 +4,10 @@ const source = import('../../web/js/workspace/state.mjs');
test('workspace routes resolve only supported deep links', async () => {
const { resolveRoute } = await source;
for (const route of ['overview', 'images', 'library', 'tags', 'inspect', 'system']) {
for (const route of ['overview', 'images', 'library', 'tags', 'inspect', 'system', 'activity']) {
assert.equal(resolveRoute(`#/${route}`).id, route);
}
for (const hash of ['', '#', '#/unknown', '#/images/other', '#/<script>']) {
for (const hash of ['', '#', '#/unknown', '#/images/other', '#/<script>', '#/activity/abc', `#/activity/${'A'.repeat(32)}`]) {
assert.equal(resolveRoute(hash).id, 'images');
}
});
@@ -23,10 +23,33 @@ test('tags is a native route titled Tags; only unbuilt routes hand off to legacy
assert.equal(routeTitle(route), route.label);
}
assert.equal(routeTitle(resolveRoute('#/library/7')), 'Prompt library');
for (const id of ['images', 'library', 'tags', 'overview']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id);
for (const id of ['images', 'library', 'tags', 'overview', 'activity']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), false, id);
for (const id of ['inspect', 'system']) assert.equal(isHandoff(resolveRoute(`#/${id}`)), true, id);
});
test('a job of the Activity page is its own deep link', async () => {
const { resolveRoute, routeTitle, isHandoff } = await source;
const id = '0123456789abcdef'.repeat(2);
const route = resolveRoute(`#/activity/${id}`);
assert.deepEqual([route.id, route.jobId, route.hash, routeTitle(route), isHandoff(route)],
['activity', id, `#/activity/${id}`, 'Activity', false]);
assert.equal(resolveRoute('#/activity').jobId, undefined);
assert.equal(route.href, undefined, 'Activity has no legacy card on Overview');
// The router renders again when only the job changes, and not for the same job.
const { mountRouter } = await source;
const other = 'f'.repeat(32);
const win = new EventTarget();
win.location = { hash: `#/activity/${id}` };
win.history = { state: null, replaceState(state, unused, hash) { win.location.hash = hash; } };
const seen = [];
const dispose = mountRouter(win, next => seen.push([next.id, next.jobId]));
win.dispatchEvent(new Event('hashchange'));
win.location.hash = `#/activity/${other}`; win.dispatchEvent(new Event('hashchange'));
win.location.hash = '#/activity'; win.dispatchEvent(new Event('hashchange'));
assert.deepEqual(seen, [['activity', id], ['activity', other], ['activity', undefined]]);
dispose();
});
test('statistics preserve zero and reject incomplete or invalid counts', async () => {
const { summarizeStats } = await source;
assert.deepEqual(summarizeStats({total_prompts:0, total_images:12, total_tags:3}), [
+96
View File
@@ -0,0 +1,96 @@
// The shell's shared toast: plain messages clear after 6.5 s; a message with a link stays 20 s (D57), and any toast
// stays while the pointer or keyboard focus rests on it.
const test = require('node:test');
const assert = require('node:assert/strict');
const source = import('../../web/js/workspace/toast.mjs');
class FakeToast extends EventTarget {
constructor() { super(); this.held = false; this.nodes = []; this.text = ''; }
get textContent() { return this.text; }
set textContent(value) { this.text = value; this.nodes = []; }
replaceChildren(...nodes) { this.nodes = nodes; this.text = nodes.map(node => node.textContent).join(''); }
matches(selector) { assert.equal(selector, ':hover, :focus-within'); return this.held; }
}
function install() {
const toast = new FakeToast();
const previous = global.document;
global.document = {
getElementById: id => id === 'workspace-toast' ? toast : null,
createTextNode: text => ({ textContent: text }),
createElement: tag => ({ tag, href: '', textContent: '' }),
};
return { toast, restore: () => { global.document = previous; } };
}
test('a plain toast clears after 6.5 s and a linked one after 20 s', async t => {
const { showWorkspaceToast, TOAST_MS, LINK_TOAST_MS } = await source;
t.mock.timers.enable({ apis: ['setTimeout'] });
const { toast, restore } = install();
try {
showWorkspaceToast('Scan started.');
assert.equal(toast.textContent, 'Scan started.');
t.mock.timers.tick(TOAST_MS);
assert.equal(toast.textContent, '');
showWorkspaceToast(['Another job is running: ', { text: 'Scan', href: '#/activity/x' }, '.']);
assert.equal(toast.textContent, 'Another job is running: Scan.');
assert.deepEqual(toast.nodes[1], { tag: 'a', href: '#/activity/x', textContent: 'Scan' });
t.mock.timers.tick(TOAST_MS);
assert.equal(toast.textContent, 'Another job is running: Scan.');
t.mock.timers.tick(LINK_TOAST_MS - TOAST_MS);
assert.equal(toast.textContent, '');
} finally {
restore();
}
});
test('a toast under the pointer or focus waits until they leave', async t => {
const { showWorkspaceToast, TOAST_MS } = await source;
t.mock.timers.enable({ apis: ['setTimeout'] });
const { toast, restore } = install();
try {
showWorkspaceToast('Prompt copied.');
toast.held = true;
t.mock.timers.tick(TOAST_MS);
assert.equal(toast.textContent, 'Prompt copied.', 'held: it stays');
// Focus moving inside the toast keeps it; leaving it starts a fresh 6.5 s.
toast.dispatchEvent(new Event('focusout'));
t.mock.timers.tick(0);
toast.dispatchEvent(new Event('pointerleave'));
assert.equal(toast.textContent, 'Prompt copied.');
toast.held = false;
toast.dispatchEvent(new Event('focusout'));
t.mock.timers.tick(0);
t.mock.timers.tick(TOAST_MS - 1);
assert.equal(toast.textContent, 'Prompt copied.');
t.mock.timers.tick(1);
assert.equal(toast.textContent, '');
// A newer toast replaces a held one and stops waiting for the old one's pointer.
showWorkspaceToast('First.');
toast.held = true;
t.mock.timers.tick(TOAST_MS);
showWorkspaceToast('Second.');
toast.held = false;
toast.dispatchEvent(new Event('pointerleave'));
t.mock.timers.tick(TOAST_MS - 1);
assert.equal(toast.textContent, 'Second.');
t.mock.timers.tick(1);
assert.equal(toast.textContent, '');
} finally {
restore();
}
});
test('a page without the toast shows nothing and does not fail', async () => {
const { showWorkspaceToast } = await source;
const previous = global.document;
global.document = { getElementById: () => null };
try {
assert.doesNotThrow(() => showWorkspaceToast('Nothing.'));
} finally {
global.document = previous;
}
});
+50
View File
@@ -342,6 +342,56 @@ test('refine dialog round-trips values and clear resets them', async () => {
}
});
test('Refine lists options discovered after it opened and keeps the wanted values', async () => {
const { openRefineDialog, openImportDialog, closeWorkspaceDialog } = await dialogsModule;
const dom = fakeDOM();
const scope = (await coreModule).createScope();
try {
let bySource = {};
const optionsFor = source => [['', 'All output folders'], ...(bySource[source] ?? []).map(path => [path, path])];
const fields = [{ key: 'source', options: [['all', 'All sources']] }, { key: 'folder', options: optionsFor('all'), optionsFor },
REFINE_FIELDS[2]];
const applied = [];
const refine = openRefineDialog({ scope, fields, values: { source: '1', folder: 'temp-only', media: 'video' },
onApply: value => applied.push(value), onClear() {} });
const [source, folder, media] = byId(dom, 'modal-body').children.filter(child => child.className === 'field')
.map(field => field.children[1]);
// Opened before discovery: only the cleared values.
assert.deepEqual(optionPairs(source), [['all', 'All sources']]);
assert.equal(source.value, 'all');
assert.equal(folder.value, '');
bySource = { all: ['portraits', 'temp-only'], 0: ['portraits'], 1: ['temp-only'] };
refine.setOptions('source', [['all', 'All sources'], ['0', 'output'], ['1', 'archive']]);
// The values it opened with, now offered, and the chosen Source's folders.
assert.deepEqual(optionPairs(source).map(([value]) => value), ['all', '0', '1']);
assert.equal(source.value, '1');
assert.deepEqual(optionPairs(folder), [['', 'All output folders'], ['temp-only', 'temp-only']]);
assert.equal(folder.value, 'temp-only');
// A choice made since wins over the opening value; tags and unknown keys are ignored.
source.value = '0';
source.dispatchEvent(new Event('change'));
refine.setOptions('source', [['all', 'All sources'], ['0', 'output'], ['1', 'archive']]);
refine.setOptions('tags', [['x', 'x']]);
assert.equal(source.value, '0');
assert.deepEqual(optionPairs(folder).map(([value]) => value), ['', 'portraits']);
assert.equal(media.value, 'video');
button(byId(dom, 'modal-body'), 'Apply filters').click();
assert.deepEqual(applied, [{ source: '0', folder: '', media: 'video' }]);
// Closed: a late discovery leaves the shell alone.
refine.setOptions('source', [['all', 'All sources']]);
assert.equal(source.options.length, 3);
// Replaced by another dialog: the same.
const again = openRefineDialog({ scope, fields, values: {}, onApply() {}, onClear() {} });
openImportDialog({ scope, onInspect() {}, onRescan() {} });
again.setOptions('source', [['all', 'All sources'], ['9', 'late']]);
assert.equal(byId(dom, 'modal-title').textContent, 'Import & inspect');
} finally {
closeWorkspaceDialog();
scope.dispose();
dom.restore();
}
});
test('the Folder list follows the chosen Source', async () => {
const { openRefineDialog, closeWorkspaceDialog } = await dialogsModule;
const dom = fakeDOM();
+413
View File
@@ -1965,6 +1965,419 @@ class TestOutputScanIsABackgroundJob(AdminAPITestCase):
self.assertEqual(data["last_event"]["type"], "error")
class TestActivityJobs(AdminAPITestCase):
"""The scan runs as a registry job; legacy routes keep their payloads."""
def _gate_collect(self):
import threading
gate = threading.Event()
calls = []
original = self.api._collect_output_media_sync
def gated(output_dirs):
calls.append(output_dirs)
gate.wait(5)
return original(output_dirs)
self.api._collect_output_media_sync = gated
return gate, calls
async def _json(self, method, url):
resp = await self.client.request(method, url)
return resp.status, await resp.json()
async def test_legacy_scan_route_reuses_registry_job_and_payload_shape(self):
from py.api.admin import SCAN_COUNT_KEYS
_write_png(self.output_dir / "gen.png", prompt_text="a red square")
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
gate, calls = self._gate_collect()
# The workspace starts the job; the legacy page then attaches to it.
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
self.assertEqual(status, 202)
self.assertTrue(started["success"])
legacy = asyncio.ensure_future(
self.client.request("POST", "/prompt_manager/scan", data="{}")
)
await asyncio.sleep(0.05)
gate.set()
body = await (await legacy).text()
await asyncio.wait_for(self.api._scan_job.task, 10)
self.assertEqual(len(calls), 1)
payloads = _sse_payloads(body)
progress = [p for p in payloads if p["type"] == "progress"]
self.assertTrue(progress)
legacy_keys = set(SCAN_COUNT_KEYS) | {"type", "progress", "status", "processed"}
for payload in progress:
self.assertEqual(set(payload), legacy_keys)
final = payloads[-1]
self.assertEqual(final["type"], "complete")
self.assertEqual(set(final), set(SCAN_COUNT_KEYS) | {"type", "directories"})
self.assertEqual(final["added"], 1)
self.assertEqual(final["directories"], ["output"])
status, listed = await self._json("GET", "/prompt_manager/jobs")
self.assertEqual(status, 200)
self.assertEqual(len(listed["jobs"]), 1)
job = listed["jobs"][0]
self.assertEqual(job["id"], started["job"]["id"])
self.assertEqual(job["id"], self.api._scan_job.id)
self.assertEqual((job["kind"], job["state"]), ("scan", "finished"))
self.assertEqual(job["counters"]["added"], 1)
self.assertEqual(job["progress"], 100)
self.assertEqual(job["scope"]["output_folders"], 1)
async def test_scan_records_outcomes_by_group(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
(self.output_dir / "sub").mkdir()
names = {
"sub/added.png": {"status": "added", "title": "A red fox"},
"linked.png": {"status": "linked", "title": "A grey cat"},
"unchanged.png": {"status": "unchanged", "title": "A brown dog"},
"ambiguous.png": {"status": "ambiguous"},
"failed.png": RuntimeError(f"{self.tmpdir}/secret boom"),
"none.png": None,
}
for name in names:
(self.output_dir / name).write_bytes(b"not an image")
def fake_ingest(media_file, metadata, detailed=False):
key = Path(media_file).relative_to(self.output_dir).as_posix()
outcome = names[key]
if isinstance(outcome, Exception):
raise outcome
return outcome
self.api._ingest_scanned_file = fake_ingest
with self.assertLogs("prompt_manager.api", level="ERROR"):
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
self.assertEqual(status, 202)
await asyncio.wait_for(self.api._scan_job.task, 10)
resp = await self.client.request(
"GET", f"/prompt_manager/jobs/{started['job']['id']}"
)
text = await resp.text()
self.assertEqual(resp.status, 200)
self.assertNotIn(self.tmpdir, text)
job = json.loads(text)["job"]
self.assertEqual(
job["counters"],
{"added": 2, "known": 1, "review": 1, "no_metadata": 1, "errors": 1},
)
outcomes = job["outcomes"]
self.assertEqual(
{k: len(v) for k, v in outcomes.items()},
{"added": 2, "known": 1, "review": 1, "no_metadata": 1, "errors": 1},
)
added = {row["path"]: row for row in outcomes["added"]}
self.assertEqual(set(added), {"sub/added.png", "linked.png"})
self.assertIn("A red fox", added["sub/added.png"]["detail"])
self.assertIn("A grey cat", added["linked.png"]["detail"])
self.assertEqual(added["linked.png"]["root"], "output")
self.assertIn("A brown dog", outcomes["known"][0]["detail"])
self.assertIn("review", outcomes["review"][0]["detail"].lower())
self.assertEqual(outcomes["errors"][0]["path"], "failed.png")
self.assertTrue(outcomes["errors"][0]["detail"])
self.assertEqual(outcomes["no_metadata"][0]["path"], "none.png")
self.assertTrue(outcomes["no_metadata"][0]["detail"])
prefixes = {line.split(" ", 2)[1] for line in job["log"][1:-1]}
self.assertEqual(prefixes, {"+", "=", "?", "!", "-"})
self.assertEqual(job["state"], "finished")
self.assertTrue(job["status_line"].startswith("Processing file 6 of 6"))
legacy = self.api._scan_job.last_event
self.assertEqual(
(legacy["processed"], legacy["found"], legacy["failed"]), (6, 4, 1)
)
async def test_report_is_attachment_without_absolute_paths(self):
"""Task 5: the report downloads as JSON, whole and without server paths."""
import re
from py.api.jobs import MAX_OUTCOME_ROWS
async def runner(job):
absolute = f"{self.tmpdir}/output/deep/abs.png"
job.record(
"added",
{"path": absolute, "root": "output", "root_index": 0, "detail": "ok"},
f"+ {absolute} · ok",
)
for index in range(MAX_OUTCOME_ROWS + 5):
job.record(
"known",
{"path": f"k/{index}.png", "root": "output", "detail": "x"},
f"= k/{index}.png",
)
job.update(processed=MAX_OUTCOME_ROWS + 6, total=MAX_OUTCOME_ROWS + 6)
job = self.api._jobs.start(
"scan",
"Scan for metadata",
record_changing=True,
scope={"files": "new", "output_folders": 1},
runner=runner,
groups=("added", "known", "review", "no_metadata", "errors"),
)
await asyncio.wait_for(job.task, 10)
resp = await self.client.request("GET", f"/prompt_manager/jobs/{job.id}/report")
text = await resp.text()
self.assertEqual(resp.status, 200)
self.assertIn("application/json", resp.headers["Content-Type"])
disposition = resp.headers["Content-Disposition"]
self.assertRegex(
disposition,
r'^attachment; filename="prompt-manager-scan-\d{8}-\d{4}\.json"$',
)
self.assertNotIn(self.tmpdir, text)
self.assertIsNone(
re.search(r"[A-Za-z]:[\\/]|(?<![\w.])/(?:tmp|home|var)/", text)
)
report = json.loads(text)
self.assertEqual(report["job"]["id"], job.id)
self.assertEqual(report["job"]["scope"]["files"], "new")
self.assertEqual(report["job"]["counters"]["known"], MAX_OUTCOME_ROWS + 5)
added = report["outcomes"]["added"]
self.assertEqual(
(added["count"], len(added["rows"]), added["truncated"]), (1, 1, False)
)
self.assertEqual(added["rows"][0]["path"], "abs.png")
known = report["outcomes"]["known"]
self.assertEqual(
(known["count"], len(known["rows"]), known["truncated"]),
(MAX_OUTCOME_ROWS + 5, MAX_OUTCOME_ROWS, True),
)
self.assertEqual(report["outcomes"]["errors"]["rows"], [])
self.assertFalse(report["outcomes"]["errors"]["truncated"])
self.assertEqual(report["log"], job.detail()["log"])
self.assertIn("generated_at", report)
resp = await self.client.request("GET", "/prompt_manager/jobs/nope/report")
self.assertEqual(resp.status, 404)
async def test_cancel_after_first_batch_keeps_partial_counts(self):
"""Review M7: the real scan stops at the batch boundary, keeping batch 1."""
import threading
from unittest import mock
from py.api import admin
_write_png(self.output_dir / "a.png", prompt_text="a red square")
_write_png(self.output_dir / "b.png", prompt_text="a blue circle")
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
first_done, release = threading.Event(), threading.Event()
original = self.api._ingest_scan_batch_sync
def gated(batch_results, roots=(), outcomes=None):
counts = original(batch_results, roots=roots, outcomes=outcomes)
first_done.set()
release.wait(5)
return counts
self.api._ingest_scan_batch_sync = gated
with mock.patch.object(admin, "SCAN_BATCH_SIZE", 1):
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
self.assertEqual(status, 202)
job_id = started["job"]["id"]
await asyncio.get_running_loop().run_in_executor(None, first_done.wait, 5)
status, cancelled = await self._json(
"POST", f"/prompt_manager/jobs/{job_id}/cancel"
)
self.assertEqual((status, cancelled["cancellation_requested"]), (200, True))
release.set()
await asyncio.wait_for(self.api._scan_job.task, 10)
status, body = await self._json("GET", f"/prompt_manager/jobs/{job_id}")
job = body["job"]
self.assertEqual(job["state"], "cancelled")
self.assertEqual((job["processed"], job["total"]), (1, 2))
self.assertEqual(sum(job["counters"].values()), job["processed"])
self.assertEqual(job["counters"]["added"], 1)
listed = [row["path"] for rows in job["outcomes"].values() for row in rows]
self.assertEqual(len(listed), 1)
self.assertIn(listed[0], {"a.png", "b.png"})
self.assertIn("Scan cancelled · 1 file processed", job["log"][-1])
# A cancel after the end is a no-op.
status, again = await self._json(
"POST", f"/prompt_manager/jobs/{job_id}/cancel"
)
self.assertEqual((status, again["cancellation_requested"]), (200, False))
async def test_scan_rows_carry_their_root_index(self):
"""Review M9: two roots labelled "output" stay apart by listing index."""
second = self.outside_dir / "output"
second.mkdir(parents=True)
os.environ[EXTRA_ROOTS_ENV] = str(second)
_write_png(self.output_dir / "same.png", prompt_text="a red square")
_write_png(second / "same.png", prompt_text="a blue circle")
GalleryConfig.MONITORING_DIRECTORIES = [
self._root(self.output_dir),
self._root(second),
]
output_dirs = [Path(d) for d in self.api._get_all_output_dirs()]
self.assertEqual(len(output_dirs), 2)
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
await asyncio.wait_for(self.api._scan_job.task, 10)
status, body = await self._json(
"GET", f"/prompt_manager/jobs/{started['job']['id']}"
)
rows = body["job"]["outcomes"]["added"]
self.assertEqual(len(rows), 2)
self.assertEqual({row["root"] for row in rows}, {"output"})
by_index = {row["root_index"]: row for row in rows}
self.assertEqual(set(by_index), {0, 1})
for index, row in by_index.items():
self.assertEqual(row["path"], "same.png")
self.assertTrue((output_dirs[index] / row["path"]).is_file())
texts = {
index: (
"red"
if output_dirs[index] == Path(self._root(self.output_dir))
else "blue"
)
for index in by_index
}
for index, row in by_index.items():
self.assertIn(texts[index], row["detail"])
async def test_scan_row_names_the_prompt_by_its_stored_title(self):
"""Review M10: a row names the prompt as the Library does."""
_write_png(self.output_dir / "gen.png", prompt_text="a red square\nmore")
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
summaries = self.api.db.get_image_link_summaries
def titled(paths, allow_filename_fallback=True):
found = summaries(paths, allow_filename_fallback)
return {
path: {**summary, "title": "Stored square title"}
for path, summary in found.items()
}
self.api.db.get_image_link_summaries = titled
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
await asyncio.wait_for(self.api._scan_job.task, 10)
status, body = await self._json(
"GET", f"/prompt_manager/jobs/{started['job']['id']}"
)
(row,) = body["job"]["outcomes"]["added"]
self.assertIn("“Stored square title”", row["detail"])
# Without a stored title (or an unreadable one): the text's first line.
def broken(paths, allow_filename_fallback=True):
raise RuntimeError("unreadable")
self.api.db.get_image_link_summaries = broken
_write_png(self.output_dir / "next.png", prompt_text="a green cone\nmore")
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
await asyncio.wait_for(self.api._scan_job.task, 10)
status, body = await self._json(
"GET", f"/prompt_manager/jobs/{started['job']['id']}"
)
details = {r["path"]: r["detail"] for r in body["job"]["outcomes"]["added"]}
self.assertIn("“a green cone”", details["next.png"])
async def test_jobs_routes_404_and_409(self):
_write_png(self.output_dir / "gen.png", prompt_text="a red square")
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
for method, url in (
("GET", "/prompt_manager/jobs/nope"),
("GET", "/prompt_manager/jobs/nope/events"),
("POST", "/prompt_manager/jobs/nope/cancel"),
):
status, data = await self._json(method, url)
self.assertEqual(status, 404, url)
self.assertFalse(data["success"])
gate, calls = self._gate_collect()
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
self.assertEqual(status, 202)
job_id = started["job"]["id"]
self.assertEqual(started["job"]["state"], "running")
status, busy = await self._json("POST", "/prompt_manager/jobs/scan")
self.assertEqual(status, 409)
self.assertEqual(
busy["running"], {"id": job_id, "title": started["job"]["title"]}
)
self.assertEqual((busy["success"], busy["code"]), (False, "busy"))
watcher = asyncio.ensure_future(
self.client.request("GET", f"/prompt_manager/jobs/{job_id}/events")
)
await asyncio.sleep(0.05)
status, cancel = await self._json(
"POST", f"/prompt_manager/jobs/{job_id}/cancel"
)
self.assertEqual((status, cancel["cancellation_requested"]), (200, True))
gate.set()
events = _sse_payloads(await (await watcher).text())
await asyncio.wait_for(self.api._scan_job.task, 10)
self.assertEqual(len(calls), 1)
self.assertEqual(events[0]["id"], job_id)
self.assertEqual(events[-1]["state"], "cancelled")
status, cancel = await self._json(
"POST", f"/prompt_manager/jobs/{job_id}/cancel"
)
self.assertEqual((status, cancel["cancellation_requested"]), (200, False))
status, legacy = await self._json("GET", "/prompt_manager/scan/status")
self.assertEqual(legacy["last_event"]["type"], "cancelled")
async def test_scan_without_directories_fails_the_job(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.outside_dir)]
sys.modules.pop("folder_paths", None)
self.api._find_comfyui_output_dir = lambda: None
status, started = await self._json("POST", "/prompt_manager/jobs/scan")
await asyncio.wait_for(self.api._scan_job.task, 10)
status, data = await self._json(
"GET", f"/prompt_manager/jobs/{started['job']['id']}"
)
self.assertEqual(data["job"]["state"], "failed")
self.assertIn("No output directories", data["job"]["status_line"])
async def test_legacy_scan_refused_while_another_record_job_runs(self):
hold = asyncio.Event()
async def runner(job):
await hold.wait()
other = self.api._jobs.start(
"maintenance", "Maintenance", record_changing=True, scope={}, runner=runner
)
resp = await self.client.request("POST", "/prompt_manager/scan", data="{}")
self.assertEqual(resp.status, 409)
self.assertIn("Maintenance", (await resp.json())["error"])
hold.set()
await asyncio.wait_for(other.task, 5)
class TestScanStatusLine(unittest.TestCase):
def test_estimate_appears_after_two_batches(self):
from py.api.admin import _scan_status_line
self.assertEqual(
_scan_status_line(50, 200, 1, 10.0), "Processing file 50 of 200"
)
self.assertEqual(
_scan_status_line(100, 1000, 2, 30.0),
"Processing file 100 of 1,000 · about 5 min left",
)
self.assertEqual(
_scan_status_line(100, 120, 2, 1.0),
"Processing file 100 of 120 · about 1 min left",
)
self.assertEqual(
_scan_status_line(120, 120, 3, 9.0), "Processing file 120 of 120"
)
class TestWorkerThreadsSetting(AdminAPITestCase):
"""The worker-thread count is exposed, validated, persisted and used by the scan."""
+20
View File
@@ -190,6 +190,26 @@ class TestWorkspaceImages(ImageAPITestCase):
self.assertEqual(data["images"], [])
self.assertIsNone(data["next_cursor"])
async def test_path_filter_lists_only_that_exact_file(self):
# Activity (D57): a scan row opens its file by root and exact relative path,
# never by a substring that also matches the same name in other folders.
self.media("ComfyUI_00001_.png")
for day in range(3):
self.media(f"2026-10-0{day + 1}/ComfyUI_00001_.png")
self.media("2026-10-01/ComfyUI_00001_.png.bak.png")
_, data = await self.get(path="ComfyUI_00001_.png")
self.assertEqual([i["path"] for i in data["images"]], ["ComfyUI_00001_.png"])
_, data = await self.get(path="2026-10-02/ComfyUI_00001_.png")
self.assertEqual(
[i["path"] for i in data["images"]], ["2026-10-02/ComfyUI_00001_.png"]
)
_, data = await self.get(path="missing.png")
self.assertEqual(data["images"], [])
for bad in ("../outside.png", "a" * 1025):
with self.subTest(bad=bad[:12]):
response, _ = await self.get(path=bad)
self.assertEqual(response.status, 400)
async def test_cursor_cannot_be_reused_with_changed_filters_or_tampered(self):
self.media("a.png")
self.media("b.png")
+83
View File
@@ -0,0 +1,83 @@
"""The page-workflow test server ends with its launcher (#274 review M2).
A Playwright worker that is SIGKILLed never runs its fixture teardown, so the
server it started must notice on its own: with ``--exit-on-stdin-eof`` it shuts
down when the launcher's end of its stdin pipe closes, and still removes its
temp root. This runs on every OS in CI, where Windows is the strict case.
"""
import os
import subprocess
import sys
import tempfile
import time
import unittest
import urllib.request
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
SERVER_TIMEOUT_S = 60
class TestBrowserServerLifetime(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
def start(self, *extra):
env = {**os.environ, "TMPDIR": self.tmp.name}
env.update(TEMP=self.tmp.name, TMP=self.tmp.name)
process = subprocess.Popen(
[sys.executable, "-m", "tests.browser.server", "--port", "0", *extra],
cwd=REPO_ROOT,
env=env,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
def stop():
if process.poll() is None:
process.kill()
process.wait(timeout=SERVER_TIMEOUT_S)
# communicate() would flush a closed stdin, which Python < 3.13 refuses.
for stream in (process.stdin, process.stdout, process.stderr):
stream.close()
self.addCleanup(stop)
line = process.stdout.readline().decode()
self.assertTrue(line.startswith("PM_TEST_SERVER_PORT="), line)
url = f"http://127.0.0.1:{int(line.split('=', 1)[1])}/prompt_manager/test"
deadline = time.monotonic() + SERVER_TIMEOUT_S
while True:
try:
with urllib.request.urlopen(url, timeout=1) as response: # nosec B310
self.assertEqual(response.status, 200)
return process
except OSError:
self.assertIsNone(process.poll(), "the server exited early")
self.assertLess(time.monotonic(), deadline, "never ready")
time.sleep(0.1)
def roots(self):
return sorted(path.name for path in Path(self.tmp.name).glob("pm-browser-*"))
def test_server_exits_and_removes_its_temp_root_when_stdin_closes(self):
process = self.start("--exit-on-stdin-eof")
self.assertEqual(len(self.roots()), 1, "the server works in its own temp root")
process.stdin.close()
code = process.wait(timeout=SERVER_TIMEOUT_S)
self.assertEqual(
code, 0, process.stderr.read().decode(errors="replace")[-4000:]
)
self.assertEqual(self.roots(), [], "the temp root and its database are removed")
def test_server_keeps_running_with_stdin_closed_without_the_flag(self):
process = self.start()
process.stdin.close()
with self.assertRaises(subprocess.TimeoutExpired):
process.wait(timeout=1)
if __name__ == "__main__":
unittest.main()
+436
View File
@@ -0,0 +1,436 @@
"""Unit tests for py/api/jobs.py: the in-memory session job registry.
Fake runners only: no files, no database. Ordering is driven by events and
awaited tasks, never by wall-clock bounds.
"""
import asyncio
import contextlib
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from py.api.jobs import ( # noqa: E402
DEFAULT_SCAN_SCOPE,
LOG_CAP_LINE,
MAX_FINISHED_JOBS,
MAX_LOG_LINES,
MAX_OUTCOME_ROWS,
SCAN_GROUP_NAMES,
SCAN_GROUPS,
SCAN_SCOPES,
Job,
JobBusy,
JobRegistry,
)
async def _collect(job, stop_after=None):
seen = []
async with contextlib.aclosing(job.events()) as events:
async for payload in events:
seen.append(payload)
if stop_after is not None and len(seen) >= stop_after:
break
return seen
def _gated_runner(gate, publish=None):
"""A runner that publishes ``publish`` (if any) and waits for ``gate``."""
async def runner(job):
if publish is not None:
job.publish(publish)
await gate.wait()
return runner
async def _noop(job):
return None
class TestJobFanOut(unittest.IsolatedAsyncioTestCase):
"""The SSE fan-out the scan job always had, now on every Job."""
async def test_new_job_is_running_with_no_event(self):
job = Job("scan", "Scan")
self.assertTrue(job.running)
self.assertIsNone(job.last_event)
self.assertEqual(job.state, "running")
self.assertEqual(len(job.id), 32)
async def test_subscriber_receives_events_until_finish(self):
job = Job("scan", "Scan")
consumer = asyncio.ensure_future(_collect(job))
await asyncio.sleep(0)
job.publish({"type": "progress", "progress": 10})
job.publish({"type": "complete"})
job.finish("finished")
self.assertEqual(
await asyncio.wait_for(consumer, 2),
[{"type": "progress", "progress": 10}, {"type": "complete"}],
)
self.assertFalse(job.running)
async def test_late_subscriber_gets_latest_snapshot_then_live_events(self):
job = Job("scan", "Scan")
job.publish({"type": "progress", "progress": 10})
job.publish({"type": "progress", "progress": 40})
consumer = asyncio.ensure_future(_collect(job))
await asyncio.sleep(0)
job.publish({"type": "complete"})
job.finish()
self.assertEqual(
await asyncio.wait_for(consumer, 2),
[{"type": "progress", "progress": 40}, {"type": "complete"}],
)
async def test_subscriber_after_finish_gets_only_the_final_event(self):
job = Job("scan", "Scan")
job.publish({"type": "progress", "progress": 99})
job.publish({"type": "complete"})
job.finish()
self.assertEqual(await _collect(job), [{"type": "complete"}])
async def test_departed_subscriber_is_forgotten(self):
job = Job("scan", "Scan")
job.publish({"type": "progress", "progress": 1})
await _collect(job, stop_after=1)
self.assertEqual(job.subscriber_count, 0)
job.publish({"type": "complete"}) # must not raise or queue anywhere
job.finish()
async def test_second_subscriber_attaches_to_running_job(self):
registry = JobRegistry()
gate = asyncio.Event()
job = registry.start(
"scan",
"Scan",
record_changing=True,
scope={},
runner=_gated_runner(gate, {"type": "progress", "progress": 30}),
)
await asyncio.sleep(0) # the runner publishes its first event
first = asyncio.ensure_future(_collect(job))
second = asyncio.ensure_future(_collect(job))
await asyncio.sleep(0)
self.assertEqual(job.subscriber_count, 2)
self.assertIs(registry.active("scan"), job)
job.publish({"type": "progress", "progress": 60})
gate.set()
await asyncio.wait_for(job.task, 2)
expected = [
{"type": "progress", "progress": 30},
{"type": "progress", "progress": 60},
]
self.assertEqual(await asyncio.wait_for(first, 2), expected)
self.assertEqual(await asyncio.wait_for(second, 2), expected)
self.assertEqual(job.summary()["progress"], 60)
async def test_finish_rejects_unknown_state_and_is_final(self):
job = Job("scan", "Scan")
with self.assertRaises(ValueError):
job.finish("running")
job.finish("cancelled")
job.finish("finished") # a second finish keeps the first state
self.assertEqual(job.state, "cancelled")
self.assertIsNotNone(job.finished_at)
class TestJobRegistry(unittest.IsolatedAsyncioTestCase):
async def test_start_lists_and_finishes_a_job(self):
registry = JobRegistry()
gate = asyncio.Event()
job = registry.start(
"scan",
"Scan for metadata",
record_changing=True,
scope={"files": "new", "output_folders": 2},
runner=_gated_runner(gate),
)
self.assertEqual(job.state, "running")
self.assertIs(registry.get(job.id), job)
self.assertIsNone(registry.get("missing"))
summary = job.summary()
self.assertEqual(
set(summary),
{
"id",
"kind",
"title",
"state",
"started_at",
"finished_at",
"progress",
"status_line",
"processed",
"total",
"counters",
"scope",
"listed",
"log_length",
},
)
self.assertEqual(summary["scope"], {"files": "new", "output_folders": 2})
# What the detail would list, so a view reads it only when that changed.
self.assertEqual((summary["listed"], summary["log_length"]), ({}, 0))
self.assertIsNone(summary["finished_at"])
gate.set()
await asyncio.wait_for(job.task, 2)
self.assertEqual(job.state, "finished")
self.assertIsNone(registry.active("scan"))
# Running jobs list first, then finished ones newest first.
older = registry.start(
"thumbs", "Thumbnails", record_changing=False, scope={}, runner=_noop
)
await asyncio.wait_for(older.task, 2)
hold = asyncio.Event()
running = registry.start(
"other",
"Other",
record_changing=False,
scope={},
runner=_gated_runner(hold),
)
listed = registry.list()
self.assertEqual([j.id for j in listed], [running.id, older.id, job.id])
# Only the last MAX_FINISHED_JOBS finished jobs are kept.
for index in range(MAX_FINISHED_JOBS + 5):
done = registry.start(
f"k{index}", "Fake", record_changing=False, scope={}, runner=_noop
)
await asyncio.wait_for(done.task, 2)
listed = registry.list()
finished = [j for j in listed if not j.running]
self.assertEqual(len(finished), MAX_FINISHED_JOBS)
self.assertEqual(finished[0].id, done.id)
self.assertIsNone(registry.get(job.id))
self.assertIs(listed[0], running)
hold.set()
await asyncio.wait_for(running.task, 2)
async def test_record_changing_jobs_are_exclusive(self):
registry = JobRegistry()
gate = asyncio.Event()
first = registry.start(
"scan",
"Scan for metadata",
record_changing=True,
scope={},
runner=_gated_runner(gate),
)
with self.assertRaises(JobBusy) as raised:
registry.start(
"maintenance",
"Maintenance",
record_changing=True,
scope={},
runner=_noop,
)
self.assertIs(raised.exception.running, first)
self.assertIn("Scan for metadata", str(raised.exception))
reader = registry.start(
"report", "Report", record_changing=False, scope={}, runner=_noop
)
await asyncio.wait_for(reader.task, 2)
self.assertEqual(reader.state, "finished")
# One job per kind, even a read-only one.
hold = asyncio.Event()
registry.start(
"report",
"Report",
record_changing=False,
scope={},
runner=_gated_runner(hold),
)
with self.assertRaises(JobBusy):
registry.start(
"report", "Report", record_changing=False, scope={}, runner=_noop
)
gate.set()
hold.set()
await asyncio.wait_for(first.task, 2)
async def test_cancel_is_idempotent_and_keeps_partial_counts(self):
registry = JobRegistry()
reached = asyncio.Event()
proceed = asyncio.Event()
async def runner(job):
for index in range(3):
if job.cancelled:
return
job.record("added", {"path": f"a{index}.png", "root": "output"}, "+")
if index == 0:
reached.set()
await proceed.wait()
job = registry.start(
"scan",
"Scan",
record_changing=True,
scope={},
runner=runner,
groups=SCAN_GROUP_NAMES,
)
await asyncio.wait_for(reached.wait(), 2)
self.assertTrue(registry.cancel(job.id))
self.assertTrue(registry.cancel(job.id)) # still running: still requested
proceed.set()
await asyncio.wait_for(job.task, 2)
self.assertEqual(job.state, "cancelled")
self.assertEqual(job.counters["added"], 1) # the work done before the stop
self.assertFalse(registry.cancel(job.id)) # finished: a no-op
self.assertFalse(registry.cancel("missing"))
self.assertEqual(job.state, "cancelled")
async def test_failing_runner_ends_failed_and_frees_the_slot(self):
registry = JobRegistry()
async def runner(job):
raise RuntimeError("boom")
job = registry.start(
"scan", "Scan", record_changing=True, scope={}, runner=runner
)
with self.assertLogs("py.api.jobs", level="ERROR"):
await asyncio.wait_for(job.task, 2)
self.assertEqual(job.state, "failed")
self.assertIsNone(registry.active("scan"))
async def test_runner_that_finishes_its_own_job_keeps_its_state(self):
registry = JobRegistry()
async def runner(job):
job.finish("failed")
job = registry.start(
"scan", "Scan", record_changing=True, scope={}, runner=runner
)
await asyncio.wait_for(job.task, 2)
self.assertEqual(job.state, "failed")
class TestJobOutcomesAndLog(unittest.TestCase):
def test_outcome_and_log_caps(self):
job = Job("scan", "Scan", groups=SCAN_GROUP_NAMES)
for index in range(MAX_OUTCOME_ROWS + 50):
job.record("errors", {"path": f"f{index}.png", "root": "output"}, "!")
self.assertEqual(len(job.outcomes["errors"]), MAX_OUTCOME_ROWS)
self.assertEqual(job.counters["errors"], MAX_OUTCOME_ROWS + 50)
self.assertEqual(job.outcomes["errors"][0]["path"], "f0.png")
job = Job("scan", "Scan")
for index in range(MAX_LOG_LINES + 100):
job.log_line(f"line {index}")
self.assertEqual(len(job.log), MAX_LOG_LINES + 1)
self.assertEqual(job.log[-1], LOG_CAP_LINE)
self.assertTrue(job.log[0].endswith("line 0"))
self.assertEqual(LOG_CAP_LINE, "[Log capped at 2,000 lines]")
def test_paths_are_root_relative(self):
job = Job("scan", "Scan", groups=SCAN_GROUP_NAMES)
absolute = os.path.abspath(os.path.join("outputs", "sub", "a.png"))
job.record(
"added",
{"path": absolute, "root": "output", "detail": "x"},
f"+ {absolute}",
)
job.record("known", {"path": "sub/b.png", "root": "output"}, "= sub/b.png")
job.record("review", {"path": "C:\\out\\c.png", "root": "output"}, "?")
rows = [row for group in job.outcomes.values() for row in group]
for row in rows:
self.assertFalse(os.path.isabs(row["path"]), row)
self.assertNotIn(":", row["path"])
self.assertEqual(job.outcomes["added"][0]["path"], "a.png")
self.assertEqual(job.outcomes["known"][0]["path"], "sub/b.png")
self.assertEqual(job.outcomes["review"][0]["path"], "c.png")
self.assertEqual(job.outcomes["known"][0]["detail"], "")
self.assertNotIn(absolute, "\n".join(job.log))
def test_record_rejects_unknown_group(self):
job = Job("scan", "Scan", groups=SCAN_GROUP_NAMES)
with self.assertRaises(ValueError):
job.record("bogus", {"path": "a.png", "root": "output"}, "x")
def test_detail_and_progress(self):
job = Job(
"scan",
"Scan",
scope={"files": "new", "output_folders": 2},
groups=SCAN_GROUP_NAMES,
)
job.update(progress=150, status_line="Processing file 3 of 4", processed=3)
job.update(total=4)
job.record("known", {"path": "a.png", "root": "output"}, "= a.png")
detail = job.detail()
self.assertEqual(detail["progress"], 100)
self.assertEqual(detail["status_line"], "Processing file 3 of 4")
self.assertEqual((detail["processed"], detail["total"]), (3, 4))
self.assertEqual(detail["counters"]["known"], 1)
self.assertEqual(detail["outcomes"]["known"][0]["path"], "a.png")
self.assertEqual(len(detail["log"]), 1)
detail["outcomes"]["known"].clear() # a copy, not the live list
self.assertEqual(len(job.outcomes["known"]), 1)
job.publish({"type": "progress", "progress": -5})
self.assertEqual(job.summary()["progress"], 0)
def test_scan_groups_follow_the_counter_mapping(self):
self.assertEqual(
SCAN_GROUPS,
{
"added": "added",
"linked": "added",
"enriched": "added",
"unchanged": "known",
# Activity-2: a file the "new" scope skipped is Already known.
"skipped": "known",
"ambiguous": "review",
"conflict": "review",
"unresolved": "review",
"failed": "errors",
},
)
self.assertEqual((SCAN_SCOPES, DEFAULT_SCAN_SCOPE), (("new", "all"), "new"))
self.assertEqual(
SCAN_GROUP_NAMES, ("added", "known", "review", "no_metadata", "errors")
)
if __name__ == "__main__":
unittest.main()
class TestListedSummary(unittest.TestCase):
def test_summary_counts_listed_rows_and_log_lines(self):
job = Job("scan", "Scan", groups=("added", "errors"))
job.record(
"added", {"path": "a.png", "root": "output", "root_index": 0}, "+ a.png"
)
job.record("errors", {"path": "b.png", "root_index": "x"}, "! b.png")
summary = job.summary()
self.assertEqual(summary["listed"], {"added": 1, "errors": 1})
self.assertEqual(summary["log_length"], 2)
rows = job.detail()["outcomes"]
self.assertEqual(rows["added"][0]["root_index"], 0)
# Only a whole non-negative number is a root index.
self.assertIsNone(rows["errors"][0]["root_index"])
+3 -3
View File
@@ -120,9 +120,9 @@ class TestPairScan(AdminAPITestCase):
self.assertEqual(self.api._ingest_scanned_file(path, meta), "unchanged")
async def test_cancel_finishes_at_safe_boundary_and_restart_is_idempotent(self):
from py.api.scan_job import ScanJob
from py.api.jobs import Job
job = ScanJob()
job = Job("scan", "Scan")
self.api._scan_job = job
response = await self.client.post("/prompt_manager/scan/cancel")
self.assertEqual(response.status, 200)
@@ -133,7 +133,7 @@ class TestPairScan(AdminAPITestCase):
self.assertEqual(counts["processed"], 0)
self.assertEqual(self.api.db.list_prompt_pairs()["total"], 0)
job.finish()
self.api._scan_job = ScanJob()
self.api._scan_job = Job("scan", "Scan")
first = self.api._ingest_scan_batch_sync(
[(self.output_dir / "cancel.png", {"prompt": json.dumps(graph())})]
)
-92
View File
@@ -1,92 +0,0 @@
"""Unit tests for py/api/scan_job.py: fan-out of scan progress to SSE subscribers."""
import asyncio
import contextlib
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from py.api.scan_job import ScanJob # noqa: E402
async def _collect(job, stop_after=None):
seen = []
async with contextlib.aclosing(job.events()) as events:
async for payload in events:
seen.append(payload)
if stop_after is not None and len(seen) >= stop_after:
break
return seen
class TestScanJob(unittest.IsolatedAsyncioTestCase):
async def test_new_job_is_running_with_no_event(self):
job = ScanJob()
self.assertTrue(job.running)
self.assertIsNone(job.last_event)
async def test_subscriber_receives_events_until_finish(self):
job = ScanJob()
consumer = asyncio.ensure_future(_collect(job))
await asyncio.sleep(0)
job.publish({"type": "progress", "progress": 10})
job.publish({"type": "complete"})
job.finish()
self.assertEqual(
await asyncio.wait_for(consumer, 2),
[{"type": "progress", "progress": 10}, {"type": "complete"}],
)
self.assertFalse(job.running)
async def test_late_subscriber_gets_latest_snapshot_then_live_events(self):
job = ScanJob()
job.publish({"type": "progress", "progress": 10})
job.publish({"type": "progress", "progress": 40})
consumer = asyncio.ensure_future(_collect(job))
await asyncio.sleep(0)
job.publish({"type": "complete"})
job.finish()
self.assertEqual(
await asyncio.wait_for(consumer, 2),
[{"type": "progress", "progress": 40}, {"type": "complete"}],
)
async def test_subscriber_after_finish_gets_only_the_final_event(self):
job = ScanJob()
job.publish({"type": "progress", "progress": 99})
job.publish({"type": "complete"})
job.finish()
self.assertEqual(await _collect(job), [{"type": "complete"}])
async def test_two_subscribers_both_see_every_event(self):
job = ScanJob()
first = asyncio.ensure_future(_collect(job))
second = asyncio.ensure_future(_collect(job))
await asyncio.sleep(0)
job.publish({"type": "progress", "progress": 1})
job.finish()
self.assertEqual(await first, [{"type": "progress", "progress": 1}])
self.assertEqual(await second, [{"type": "progress", "progress": 1}])
async def test_departed_subscriber_is_forgotten(self):
job = ScanJob()
job.publish({"type": "progress", "progress": 1})
await _collect(job, stop_after=1)
self.assertEqual(job.subscriber_count, 0)
job.publish({"type": "complete"}) # must not raise or queue anywhere
job.finish()
if __name__ == "__main__":
unittest.main()
+390
View File
@@ -0,0 +1,390 @@
"""Scan scope (#249 Activity-2 Task 4): new and unresolved files by default.
A "Scan for metadata" job started without ``scope`` (or with ``"new"``) skips
the files the Library already links with a resolved link, before their
metadata is read, and counts them as Already known. ``"all"`` reads every file
again, as the legacy ``/prompt_manager/scan`` route always does.
"""
import asyncio
import json
import os
import sqlite3
import unittest
from unittest.mock import patch
try:
from tests.test_api_admin import AdminAPITestCase, _sse_payloads, _write_png
except ImportError: # discovered with tests/ as the top-level directory
from test_api_admin import AdminAPITestCase, _sse_payloads, _write_png
from database import operations # noqa: E402
from database.pair_ingestion import ingest_resolved_pair # noqa: E402
from py.config import GalleryConfig # noqa: E402
from utils.prompt_pair_resolver import resolve_prompt_pair # noqa: E402
SKIPPED_DETAIL = "Skipped · already linked"
ON_DISK = object() # _link_resolved: store the file's size on disk
def _graph(text):
return {
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": text}},
"2": {"class_type": "CLIPTextEncode", "inputs": {"text": "bad"}},
"3": {
"class_type": "KSampler",
"inputs": {"positive": ["1", 0], "negative": ["2", 0]},
},
}
class ScanScopeTestCase(AdminAPITestCase):
"""Helpers: link files the way the scan and the monitor do, spy on reads."""
def _link_resolved(self, path, text, size=ON_DISK):
"""``path`` linked with a resolved pair link (what a resolved scan writes).
The row stores ``size`` as its file size: by default the file's size on
disk (None when the file does not exist).
"""
if size is ON_DISK:
size = os.path.getsize(path) if os.path.exists(path) else None
metadata = {"prompt": _graph(text)}
if size is not None:
metadata["file_info"] = {"size": size}
result = ingest_resolved_pair(
self.api.db, str(path), metadata, resolve_prompt_pair(metadata)
)
self.assertEqual(result["status"], "added")
def _link_without_pair(self, path, text):
"""``path`` linked to a prompt with no pair link (fallback / FIFO link)."""
prompt_id = self.api.db.save_prompt(text, None, [], None, None)
self.assertTrue(self.api.db.link_image_to_prompt(prompt_id, str(path)))
return prompt_id
def _spy_reads(self):
reads = []
original = self.api._extract_comfyui_metadata
def spy(path):
reads.append(os.path.basename(path))
return original(path)
self.api._extract_comfyui_metadata = spy
return reads
async def _scan(self, body=None):
kwargs = {} if body is None else {"json": body}
resp = await self.client.request("POST", "/prompt_manager/jobs/scan", **kwargs)
data = await resp.json()
self.assertEqual(resp.status, 202, data)
await asyncio.wait_for(self.api._scan_job.task, 10)
detail = await self.client.request(
"GET", f"/prompt_manager/jobs/{data['job']['id']}"
)
return (await detail.json())["job"]
class TestScanScope(ScanScopeTestCase):
async def test_new_scope_skips_known_resolved_files_without_reading_them(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
_write_png(self.output_dir / "new.png", prompt_text="a blue circle")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(reads, ["new.png"])
self.assertEqual(job["scope"]["files"], "new")
self.assertEqual(job["counters"]["known"], 1)
self.assertEqual(job["counters"]["added"], 1)
self.assertEqual(job["processed"], 2)
self.assertEqual(
job["outcomes"]["known"],
[
{
"path": "known.png",
"root": "output",
"root_index": 0,
"detail": SKIPPED_DETAIL,
}
],
)
self.assertIn(f"= output/known.png · {SKIPPED_DETAIL}", "\n".join(job["log"]))
self.assertTrue(
job["log"][0].endswith(
"Scan started · 1 output folder · new and unresolved files"
)
)
legacy = self.api._scan_job.last_event
self.assertEqual((legacy["processed"], legacy["unchanged"]), (2, 1))
self.assertTrue(job["status_line"].startswith("Processing file 2 of 2"))
async def test_whole_known_batch_skips_the_executor_reads(self):
from py.api import admin
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
for name in ("a.png", "b.png"):
_write_png(self.output_dir / name, prompt_text=f"prompt {name}")
self._link_resolved(self.output_dir / name, f"prompt {name}")
_write_png(self.output_dir / "c.png", prompt_text="prompt c")
reads = self._spy_reads()
with patch.object(admin, "SCAN_BATCH_SIZE", 1):
job = await self._scan({"scope": "new"})
self.assertEqual(reads, ["c.png"])
self.assertEqual(job["counters"]["known"], 2)
self.assertEqual(job["counters"]["added"], 1)
async def test_unresolved_files_are_rescanned_in_new_scope(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
names = ("fallback.png", "conflict.png", "unlinked.png")
for name in names:
_write_png(self.output_dir / name, prompt_text=f"text of {name}")
# Linked without a resolved pair link (the monitor's fallback / FIFO link).
self._link_without_pair(self.output_dir / "fallback.png", "text of fallback")
# Resolved once, then also linked to a second prompt: ambiguous.
conflict = self.output_dir / "conflict.png"
self._link_resolved(conflict, "text of conflict.png")
self._link_without_pair(conflict, "another prompt")
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(sorted(reads), sorted(names))
self.assertNotIn(SKIPPED_DETAIL, json.dumps(job["outcomes"]))
async def test_all_scope_reads_everything(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
job = await self._scan({"scope": "all"})
self.assertEqual(reads, ["known.png"])
self.assertEqual(job["scope"]["files"], "all")
self.assertEqual(job["counters"]["known"], 1)
self.assertNotEqual(job["outcomes"]["known"][0]["detail"], SKIPPED_DETAIL)
self.assertTrue(
job["log"][0].endswith("Scan started · 1 output folder · all files")
)
async def test_legacy_scan_route_reads_everything(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
resp = await self.client.request("POST", "/prompt_manager/scan", data="{}")
payloads = _sse_payloads(await resp.text())
self.assertEqual(reads, ["known.png"])
self.assertEqual(self.api._scan_job.scope["files"], "all")
self.assertEqual(payloads[-1]["type"], "complete")
self.assertEqual(payloads[-1]["unchanged"], 1)
async def test_file_replaced_at_a_known_path_is_reread_and_needs_review(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
path = self.output_dir / "out_00012.png"
_write_png(path, prompt_text="the first prompt")
self._link_resolved(path, "the first prompt")
before = os.path.getsize(path)
# ComfyUI reused the name for a new image with another prompt.
_write_png(path, prompt_text="a much longer second prompt for the same name")
self.assertNotEqual(os.path.getsize(path), before)
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(reads, ["out_00012.png"])
self.assertEqual(job["counters"]["review"], 1)
self.assertEqual(job["counters"]["known"], 0)
self.assertIn("different prompt", job["outcomes"]["review"][0]["detail"])
async def test_scanned_files_store_their_size_so_the_next_new_scan_skips(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
_write_png(self.output_dir / "fresh.png", prompt_text="a fresh prompt")
first = await self._scan()
self.assertEqual(first["counters"]["added"], 1)
reads = self._spy_reads()
second = await self._scan()
self.assertEqual(reads, [])
self.assertEqual(second["outcomes"]["known"][0]["detail"], SKIPPED_DETAIL)
async def test_stored_null_size_is_reread(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
path = self.output_dir / "old.png"
_write_png(path, prompt_text="an old prompt")
self._link_resolved(path, "an old prompt", size=None)
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(reads, ["old.png"])
# The read stores the size it was missing, so the file is known next time.
self.assertEqual(job["counters"]["added"], 1)
self.assertEqual(
self.api.db.get_known_resolved_paths({str(path): os.path.getsize(path)}),
{str(path)},
)
async def test_known_set_failure_falls_back_to_reading_every_file(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
def broken(files):
raise sqlite3.OperationalError("database is locked")
self.api.db.get_known_resolved_paths = broken
with self.assertLogs("prompt_manager.api", level="WARNING"):
job = await self._scan()
self.assertEqual(job["state"], "finished")
self.assertEqual(reads, ["known.png"])
self.assertIn(
"Known files could not be read · reading every file", "\n".join(job["log"])
)
async def test_empty_read_body_asks_for_the_default_scope(self):
"""A chunked POST with no bytes can read a body: still the default."""
from py.api.job_routes import _scan_scope
class Request:
can_read_body = True
def __init__(self, text):
self._text = text
async def text(self):
return self._text
self.assertEqual(await _scan_scope(Request("")), "new")
self.assertEqual(await _scan_scope(Request(" \n")), "new")
self.assertEqual(await _scan_scope(Request('{"scope": "all"}')), "all")
self.assertIsNone(await _scan_scope(Request("{")))
async def test_invalid_scope_is_refused(self):
for body in ({"scope": "everything"}, {"scope": 1}, ["new"]):
with self.subTest(body=body):
resp = await self.client.request(
"POST", "/prompt_manager/jobs/scan", json=body
)
self.assertEqual(resp.status, 400)
data = await resp.json()
self.assertFalse(data["success"])
resp = await self.client.request(
"POST", "/prompt_manager/jobs/scan", data="not json"
)
self.assertEqual(resp.status, 400)
self.assertIsNone(self.api._scan_job)
class TestKnownResolvedPaths(ScanScopeTestCase):
"""``PromptDatabase.get_known_resolved_paths``: one streamed query."""
async def test_returns_the_given_paths_that_are_known_and_resolved(self):
known, fallback = self.output_dir / "known.png", self.output_dir / "fb.png"
self._link_resolved(known, "a red square", size=10)
self._link_without_pair(fallback, "a fallback")
asked = {str(known): 10, str(fallback): 10, str(self.output_dir / "n.png"): 10}
self.assertEqual(self.api.db.get_known_resolved_paths(asked), {str(known)})
self.assertEqual(self.api.db.get_known_resolved_paths({}), set())
async def test_size_must_match_and_a_stored_null_size_is_not_known(self):
same, other = self.output_dir / "same.png", self.output_dir / "other.png"
unknown = self.output_dir / "unknown.png"
self._link_resolved(same, "same size", size=10)
self._link_resolved(other, "other size", size=10)
self._link_resolved(unknown, "no stored size", size=None)
asked = {str(same): 10, str(other): 11, str(unknown): 10}
self.assertEqual(self.api.db.get_known_resolved_paths(asked), {str(same)})
# A file the walk could not stat is not known either.
self.assertEqual(self.api.db.get_known_resolved_paths({str(same): None}), set())
async def test_row_without_file_path_matches_through_image_path(self):
known = self.output_dir / "legacy.png"
self._link_resolved(known, "a legacy row", size=10)
with sqlite3.connect(self.api.db.model.db_path) as conn:
conn.execute("UPDATE generated_images SET file_path=NULL")
conn.close()
found = self.api.db.get_known_resolved_paths({str(known): 10})
self.assertEqual(found, {str(known)})
async def test_stored_spelling_is_normalized_like_the_scanned_path(self):
known = self.output_dir / "sub" / "known.png"
self._link_resolved(known, "a red square", size=10)
spelled = os.path.join(str(self.output_dir), "sub", "..", "sub", "known.png")
with sqlite3.connect(self.api.db.model.db_path) as conn:
conn.execute("UPDATE generated_images SET file_path=?", (spelled,))
conn.close()
found = self.api.db.get_known_resolved_paths({str(known): 10})
self.assertEqual(found, {str(known)})
async def test_one_query_streamed_in_small_fetches(self):
paths = [self.output_dir / f"{index}.png" for index in range(5)]
for path in paths:
self._link_resolved(path, f"prompt {path.name}", size=10)
original = self.api.db.model.get_connection
opened = []
def counting():
opened.append(1)
return original()
self.api.db.model.get_connection = counting
with patch.object(operations, "KNOWN_PATHS_FETCH_SIZE", 2):
found = self.api.db.get_known_resolved_paths({str(p): 10 for p in paths})
self.assertEqual(found, {str(p) for p in paths})
self.assertEqual(len(opened), 1)
async def test_windows_path_case_matches_known_set(self):
stored = self.output_dir / "Sub" / "Known.PNG"
self._link_resolved(stored, "a red square", size=10)
# A row written before paths were case-folded keeps its own spelling.
with sqlite3.connect(self.api.db.model.db_path) as conn:
conn.execute(
"UPDATE generated_images SET file_path=?", (str(stored).upper(),)
)
conn.close()
asked = str(self.output_dir / "sub" / "known.png")
with patch("os.path.normcase", lambda path: path.lower()):
self.assertEqual(self.api.db.get_known_resolved_paths({asked: 10}), {asked})
if os.path.normcase("A") == "A": # a case-sensitive platform
self.assertEqual(self.api.db.get_known_resolved_paths({asked: 10}), set())
class TestSkippedOutcome(unittest.TestCase):
def test_skipped_file_is_already_known_with_its_own_detail(self):
from py.api.admin import _scan_outcome
root = os.path.abspath("out")
group, row, line = _scan_outcome(
os.path.join(root, "a", "b.png"), [(root, "output", 0)], "skipped"
)
self.assertEqual(group, "known")
self.assertEqual(row["detail"], SKIPPED_DETAIL)
self.assertEqual(row["path"], "a/b.png")
self.assertEqual(line, f"= output/a/b.png · {SKIPPED_DETAIL}")
if __name__ == "__main__":
unittest.main()
+234
View File
@@ -0,0 +1,234 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { createServer } from 'node:net';
import { spawn } from 'node:child_process';
import { once } from 'node:events';
import { readFileSync } from 'node:fs';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { setTimeout as delay } from 'node:timers/promises';
import { startBrowserServer, createServerSupervisor, attachServerStderr } from '../../tools/vendor/start-browser-server.mjs';
import { workflowWorkers } from '../../tools/vendor/test-workers.mjs';
const fakeServer = fileURLToPath(new URL('./fake-browser-server.mjs', import.meta.url));
const fake = (...extra) => ({ command: process.execPath, args: port => [fakeServer, '--port', String(port), ...extra] });
const REAL_SERVER_TIMEOUT_MS = 90_000;
const launcher = fileURLToPath(new URL('../../tools/vendor/start-browser-server.mjs', import.meta.url));
/** Polls `check` until it is truthy, failing with `what` after `ms`. */
async function waitFor(check, what, ms = 5_000) {
const deadline = Date.now() + ms;
while (!check()) {
if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`);
await delay(20);
}
}
async function writeStderr(server, text) {
await (await fetch(`${server.url}/__stderr?text=${encodeURIComponent(text)}`)).text();
await waitFor(() => server.stderrTail().endsWith(text.trimEnd()), `stderr ${JSON.stringify(text)}`);
}
/** A process that exited counts as gone even while a non-reaping PID 1 (CI's container) leaves it a zombie. */
function alive(pid) {
try { process.kill(pid, 0); } catch { return false; }
try { return readFileSync(`/proc/${pid}/stat`, 'utf8').replace(/^.*\) /s, '')[0] !== 'Z'; } catch { return true; }
}
/** Starts a server from a separate Node worker process, SIGKILLs that worker and returns the server's pid. */
async function serverOfKilledWorker(options) {
const script = `import { startBrowserServer } from ${JSON.stringify(pathToFileURL(launcher).href)};
const server = await startBrowserServer(${options});
console.log('SERVER_PID=' + server.pid);
setInterval(() => {}, 1000);`;
const worker = spawn(process.execPath, ['--input-type=module', '-e', script], { stdio: ['ignore', 'pipe', 'inherit'] });
let out = '';
worker.stdout.setEncoding('utf8');
worker.stdout.on('data', chunk => { out += chunk; });
await waitFor(() => /SERVER_PID=\d+\n/.test(out) || worker.exitCode !== null, 'the worker to start its server', REAL_SERVER_TIMEOUT_MS);
const pid = Number(/SERVER_PID=(\d+)/.exec(out)?.[1]);
assert.ok(pid > 0, `the worker started a server (output: ${out})`);
assert.ok(alive(pid), 'the server runs while its worker lives');
worker.kill('SIGKILL');
await once(worker, 'exit');
return pid;
}
function listen(port = 0) {
return new Promise((resolve, reject) => {
const server = createServer();
server.once('error', reject);
server.listen(port, '127.0.0.1', () => resolve(server));
});
}
async function releasedPort() {
const probe = await listen();
const { port } = probe.address();
await new Promise(resolve => probe.close(resolve));
return port;
}
test('free port allocation never reuses a bound port', async t => {
const held = await listen();
t.after(() => held.close());
const heldPort = held.address().port;
const servers = await Promise.all(Array.from({ length: 6 }, () => startBrowserServer(fake())));
t.after(() => Promise.all(servers.map(server => server.stop())));
const ports = servers.map(server => server.port);
for (const server of servers) {
assert.ok(Number.isInteger(server.port) && server.port > 0 && server.port < 65536, `port ${server.port} is a TCP port`);
assert.equal(server.url, `http://127.0.0.1:${server.port}`);
assert.equal(await (await fetch(`${server.url}/prompt_manager/test`)).text(), 'ready');
}
assert.ok(!ports.includes(heldPort), 'a port another process holds is never handed out');
assert.equal(new Set(ports).size, ports.length, 'concurrent servers never share a port');
});
test('startBrowserServer asks the server to bind port 0 and reads the port it announces', async () => {
const asked = [];
const server = await startBrowserServer({ command: process.execPath, args: port => { asked.push(port); return fake().args(port); } });
try {
assert.deepEqual(asked, [0], 'no port is reserved and released before the server binds');
assert.ok(server.port > 0);
} finally {
await server.stop();
}
});
test('startBrowserServer serves the readiness page on a requested port and stops cleanly', async () => {
const port = await releasedPort();
const server = await startBrowserServer({ port, ...fake() });
assert.equal(server.port, port);
assert.equal(server.url, `http://127.0.0.1:${port}`);
assert.equal(await (await fetch(`${server.url}/prompt_manager/test`)).text(), 'ready');
await server.stop();
await assert.rejects(fetch(`${server.url}/prompt_manager/test`));
});
test('startBrowserServer reports a startup failure with the server\'s stderr tail', async () => {
await assert.rejects(startBrowserServer(fake('--fail')), error => {
assert.match(error.message, /exited with code 4 before it was ready/);
assert.match(error.message, /last fake stderr line/);
return true;
});
});
test('startBrowserServer gives up on a server that never announces a port or becomes ready', async () => {
const silent = { command: process.execPath, args: () => ['-e', 'setInterval(() => {}, 1000)'] };
await assert.rejects(startBrowserServer({ ...silent, timeoutMs: 500 }), /not ready .* within 500 ms/);
});
test('worker server restarts after an unexpected exit', async t => {
const supervisor = createServerSupervisor({ start: () => startBrowserServer(fake()) });
t.after(() => supervisor.stop());
const first = await supervisor.ensure();
assert.equal(supervisor.crash(first), null, 'a running server reports no crash');
await fetch(`${first.url}/__exit`);
await first.exited;
const crash = supervisor.crash(first);
assert.match(crash.message, /test server on port \d+ exited unexpectedly \(code 3\)/);
assert.match(crash.message, /crashed on purpose/);
const second = await supervisor.ensure();
assert.notEqual(second, first, 'the next test gets a fresh server');
assert.equal(await (await fetch(`${second.url}/prompt_manager/test`)).text(), 'ready');
assert.equal(await supervisor.ensure(), second, 'a healthy server is reused');
});
test('supervisor reports a startup failure once and tries again on the next ensure', async t => {
let attempts = 0;
const supervisor = createServerSupervisor({
start: async () => {
attempts += 1;
if (attempts === 1) throw new Error('ImportError: no module');
return startBrowserServer(fake());
},
});
t.after(() => supervisor.stop());
await assert.rejects(supervisor.ensure(), /ImportError/);
assert.equal(attempts, 1, 'a failed startup is not retried behind the caller\'s back');
const server = await supervisor.ensure();
assert.equal(attempts, 2);
assert.equal(await (await fetch(`${server.url}/prompt_manager/test`)).text(), 'ready');
});
test('the port announcement counts only once its line is complete', async t => {
const server = await startBrowserServer({ ...fake('--split-announce'), timeoutMs: 5_000 });
t.after(() => server.stop());
assert.equal(await (await fetch(`${server.url}/prompt_manager/test`)).text(), 'ready');
assert.equal(server.url, `http://127.0.0.1:${server.port}`);
});
test('server stderr since a mark holds only what came after it, bounded to the newest output', async t => {
const server = await startBrowserServer({ ...fake(), stderrLogChars: 64 });
t.after(() => server.stop());
await writeStderr(server, 'earlier test\n');
const mark = server.stderrMark();
assert.equal(server.stderrSince(mark), '');
await writeStderr(server, 'Traceback: this test\n');
assert.equal(server.stderrSince(mark), 'Traceback: this test\n');
const flood = `${'x'.repeat(80)}newest\n`;
await writeStderr(server, flood);
const kept = server.stderrSince(mark);
assert.match(kept, /^\[earlier server stderr dropped\]\n/);
assert.ok(kept.endsWith('newest\n'), 'the newest output is kept');
assert.ok(kept.length <= 64 + '[earlier server stderr dropped]\n'.length, `bounded, got ${kept.length} chars`);
});
test('a failed test gets the server stderr written during it attached; a passing one does not', async t => {
const server = await startBrowserServer(fake());
t.after(() => server.stop());
const attached = [];
const info = status => ({ status, expectedStatus: 'passed', attach: async (name, options) => { attached.push({ name, ...options }); } });
await writeStderr(server, 'before the test\n');
const mark = server.stderrMark();
await attachServerStderr(info('failed'), server, mark);
assert.deepEqual(attached, [], 'nothing to attach when the server wrote nothing');
await writeStderr(server, 'Traceback (most recent call last): boom\n');
await attachServerStderr(info('passed'), server, mark);
assert.deepEqual(attached, [], 'a passing test attaches nothing');
await attachServerStderr(info('timedOut'), server, mark);
assert.equal(attached.length, 1);
assert.equal(attached[0].name, 'server-stderr');
assert.equal(attached[0].contentType, 'text/plain');
assert.equal(String(attached[0].body), 'Traceback (most recent call last): boom\n');
});
test('a server exits with its worker even when the worker is SIGKILLed', async () => {
const pid = await serverOfKilledWorker(`{ command: process.execPath, args: port => [${JSON.stringify(fakeServer)}, '--port', String(port), '--exit-on-stdin-eof'] }`);
await waitFor(() => !alive(pid), `server ${pid} to exit after its worker died`);
});
test('tests/browser/server.py exits with a SIGKILLed worker', { timeout: 2 * REAL_SERVER_TIMEOUT_MS }, async () => {
const pid = await serverOfKilledWorker(`{ timeoutMs: ${REAL_SERVER_TIMEOUT_MS} }`);
await waitFor(() => !alive(pid), `server.py ${pid} to exit after its worker died`, 15_000);
});
test('tests/browser/server.py binds an OS-chosen port for --port 0 and still honours --port N', { timeout: 2 * REAL_SERVER_TIMEOUT_MS }, async t => {
const held = await listen();
t.after(() => held.close());
const chosen = await startBrowserServer({ timeoutMs: REAL_SERVER_TIMEOUT_MS });
try {
assert.notEqual(chosen.port, held.address().port);
assert.equal((await fetch(`${chosen.url}/prompt_manager/test`)).status, 200);
} finally {
await chosen.stop();
}
const port = await releasedPort();
const fixed = await startBrowserServer({ port, timeoutMs: REAL_SERVER_TIMEOUT_MS });
try {
assert.equal(fixed.port, port);
assert.equal((await fetch(`http://127.0.0.1:${port}/prompt_manager/test`)).status, 200);
} finally {
await fixed.stop();
}
});
test('worker count follows PM_TEST_WORKERS, then one in CI, then a quarter of the CPUs up to eight', () => {
assert.equal(workflowWorkers({ env: { PM_TEST_WORKERS: '3', CI: '1' }, cpus: 32 }), 3);
assert.equal(workflowWorkers({ env: { CI: '1' }, cpus: 32 }), 1);
assert.equal(workflowWorkers({ env: {}, cpus: 32 }), 8);
assert.equal(workflowWorkers({ env: {}, cpus: 12 }), 3);
assert.equal(workflowWorkers({ env: {}, cpus: 2 }), 1);
assert.throws(() => workflowWorkers({ env: { PM_TEST_WORKERS: 'many' }, cpus: 8 }), /PM_TEST_WORKERS must be a positive integer/);
assert.throws(() => workflowWorkers({ env: { PM_TEST_WORKERS: '0' }, cpus: 8 }), /PM_TEST_WORKERS must be a positive integer/);
});
+223
View File
@@ -0,0 +1,223 @@
// Structural checks on .github/workflows/viewerjs-check.yml. The tooling has no YAML parser, so these
// read the workflow's fixed two-space layout: jobs at two spaces, their keys at four, steps at six.
import test from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { mkdtemp, mkdir, readdir, rm, writeFile } from 'node:fs/promises';
import { readFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const workflow = readFileSync(new URL('../../.github/workflows/viewerjs-check.yml', import.meta.url), 'utf8');
const PINNED_IMAGE = /^\s+image: mcr\.microsoft\.com\/playwright:v\d+\.\d+\.\d+-noble$/m;
function jobs(text) {
const lines = text.split('\n');
const start = lines.indexOf('jobs:');
assert.notEqual(start, -1, 'workflow has a jobs section');
const found = new Map();
let current = null;
for (const line of lines.slice(start + 1)) {
const header = /^ {2}([\w-]+):\s*$/.exec(line);
if (header) {
current = header[1];
found.set(current, []);
} else if (current && (line.startsWith(' ') || line.trim() === '' || /^ {2}#/.test(line))) {
found.get(current).push(line);
} else if (current && /^\S/.test(line)) {
break;
}
}
return new Map([...found].map(([id, body]) => [id, body.join('\n')]));
}
function steps(jobText) {
const parts = jobText.split(/^(?= {6}- )/m);
return parts.slice(1);
}
function runScript(stepText) {
const lines = stepText.split('\n');
const index = lines.findIndex(line => /^ {8}run: \|\s*$/.test(line));
assert.notEqual(index, -1, 'step has a block run script');
const body = [];
for (const line of lines.slice(index + 1)) {
if (line.trim() !== '' && !line.startsWith(' ')) break;
body.push(line.slice(10));
}
return body.join('\n');
}
function needsOf(jobText) {
const match = /^ {4}needs: \[([^\]]*)\]\s*$/m.exec(jobText) ?? /^ {4}needs: ([\w-]+)\s*$/m.exec(jobText);
assert.ok(match, 'job declares needs');
return match[1].split(',').map(value => value.trim()).filter(Boolean);
}
function matrixList(jobText, key) {
const match = new RegExp(`^ {8}${key}: \\[([^\\]]*)\\]\\s*$`, 'm').exec(jobText);
assert.ok(match, `job has a ${key} matrix`);
return match[1].split(',').map(value => value.trim());
}
const allJobs = jobs(workflow);
const browserJobIds = [...allJobs].filter(([, body]) => /npm run --prefix tools\/vendor test:(workflows|coverage)/.test(body))
.map(([id]) => id);
const mergeJob = () => allJobs.get('frontend-coverage-merge');
const verifyStep = () => steps(mergeJob()).find(step => /name: Verify every shard's raw coverage arrived/.test(step));
test('viewerjs keeps its name and needs every frontend job', () => {
const viewerjs = allJobs.get('viewerjs');
assert.ok(viewerjs, 'the required context is the job id viewerjs');
assert.doesNotMatch(viewerjs, /^ {4}name:/m, 'no display name may replace the context "viewerjs"');
const needs = needsOf(viewerjs);
const others = [...allJobs.keys()].filter(id => id !== 'viewerjs');
assert.deepEqual([...needs].sort(), [...others].sort(), 'viewerjs needs every other job in the workflow');
assert.deepEqual([...needs].sort(), ['browser-workflows', 'frontend-coverage', 'frontend-coverage-merge']);
assert.match(viewerjs, /^ {4}if: always\(\)$/m, 'viewerjs runs (and fails) when a needed job failed or was skipped');
const script = runScript(steps(viewerjs).find(step => /run: \|/.test(step)));
for (const id of needs) {
const variable = new RegExp(`^ {10}([A-Z_]+): \\$\\{\\{ needs\\.${id}\\.result \\}\\}$`, 'm').exec(viewerjs);
assert.ok(variable, `${id}'s result reaches the check`);
assert.match(script, new RegExp(`test "\\$${variable[1]}" = success`), `${id} must be exactly success`);
}
});
test('viewerjs fails unless every needed job succeeded', () => {
const viewerjs = allJobs.get('viewerjs');
const script = runScript(steps(viewerjs).find(step => /run: \|/.test(step)));
const variables = [...viewerjs.matchAll(/^ {10}([A-Z_]+): \$\{\{ needs\.[\w-]+\.result \}\}$/gm)].map(match => match[1]);
const run = env => {
try {
execFileSync('bash', ['--noprofile', '--norc', '-eo', 'pipefail', '-c', script], { env: { PATH: process.env.PATH, ...env }, stdio: 'pipe' });
return true;
} catch {
return false;
}
};
const allSuccess = Object.fromEntries(variables.map(name => [name, 'success']));
assert.equal(run(allSuccess), true);
for (const name of variables) {
for (const result of ['failure', 'cancelled', 'skipped']) {
assert.equal(run({ ...allSuccess, [name]: result }), false, `${name}=${result} must fail viewerjs`);
}
}
});
test('coverage runs as three shards that each upload their raw coverage', () => {
const coverage = allJobs.get('frontend-coverage');
assert.deepEqual(matrixList(coverage, 'shard'), ['1', '2', '3']);
assert.match(coverage, /^ {4}name: frontend coverage \(shard \$\{\{ matrix\.shard \}\}\/3\)$/m);
assert.match(coverage, /npm run --prefix tools\/vendor test:coverage:shard -- \$\{\{ matrix\.shard \}\}\/3/);
const upload = steps(coverage).find(step => /name: coverage-raw-\$\{\{ matrix\.shard \}\}/.test(step));
assert.ok(upload, 'each shard uploads coverage-raw-N');
assert.match(upload, /^ {10}path: coverage\/js\/raw\/?$/m);
assert.match(upload, /^ {10}retention-days: 3$/m);
assert.match(upload, /^ {10}if-no-files-found: error$/m);
});
test('merge job downloads every shard and runs the merge-only report and check', () => {
const merge = mergeJob();
assert.ok(merge, 'frontend-coverage-merge exists');
assert.deepEqual(needsOf(merge), ['frontend-coverage']);
assert.match(merge, /^ {4}name: frontend coverage \(merge\)$/m);
const download = steps(merge).find(step => /uses: actions\/download-artifact@/.test(step));
assert.ok(download, 'merge downloads the shard artifacts');
assert.match(download, /^ {10}pattern: coverage-raw-\*$/m);
assert.doesNotMatch(download, /merge-multiple: true/, 'each artifact keeps its own directory so the check can count them');
assert.ok(verifyStep(), 'merge has an explicit artifact count check');
assert.match(verifyStep(), new RegExp(`^ {10}EXPECTED_SHARDS: ${matrixList(allJobs.get('frontend-coverage'), 'shard').length}$`, 'm'),
'the expected count matches the shard matrix');
const order = steps(merge).map(step => (/name: (.*)/.exec(step) ?? [, step.trim().split('\n')[0]])[1]);
const verifyAt = order.findIndex(name => /Verify every shard/.test(name));
const mergeAt = steps(merge).findIndex(step => /npm run --prefix tools\/vendor coverage:merge/.test(step));
assert.ok(verifyAt >= 0 && mergeAt > verifyAt, 'the count check runs before coverage:merge');
const report = steps(merge).find(step => /name: frontend-test-reports$/m.test(step));
assert.ok(report, 'merge uploads the HTML report as frontend-test-reports');
assert.match(report, /!coverage\/js\/raw\/\*\*/);
});
async function stagedShards(root, shards) {
const staged = join(root, 'staged');
for (const [shard, files] of shards) {
const directory = join(staged, `coverage-raw-${shard}`);
await mkdir(directory, { recursive: true });
for (const file of files) await writeFile(join(directory, file), '{"result":[]}');
}
await mkdir(staged, { recursive: true });
return staged;
}
async function verify(t, shards) {
const root = await mkdtemp(join(tmpdir(), 'pm-ci-merge-'));
t.after(() => rm(root, { recursive: true, force: true }));
const staged = await stagedShards(root, shards);
try {
execFileSync('bash', ['--noprofile', '--norc', '-eo', 'pipefail', '-c', runScript(verifyStep())], {
cwd: root, stdio: 'pipe',
env: { PATH: process.env.PATH, STAGED_RAW: staged, EXPECTED_SHARDS: '3' },
});
} catch (error) {
return { ok: false, output: `${error.stdout}${error.stderr}` };
}
return { ok: true, raw: (await readdir(join(root, 'coverage/js/raw'))).sort() };
}
test('merge refuses when a shard\'s raw coverage is missing', async t => {
const missing = await verify(t, [[1, ['a.json']], [3, ['c.json']]]);
assert.equal(missing.ok, false);
assert.match(missing.output, /2 of 3/);
const empty = await verify(t, [[1, ['a.json']], [2, []], [3, ['c.json']]]);
assert.equal(empty.ok, false, 'an artifact without raw files counts as missing');
const collision = await verify(t, [[1, ['a.json']], [2, ['a.json']], [3, ['c.json']]]);
assert.equal(collision.ok, false, 'two shards writing the same file name must not silently drop one');
const complete = await verify(t, [[1, ['a.json', 'node.json']], [2, ['b.json']], [3, ['c.json']]]);
assert.deepEqual(complete, { ok: true, raw: ['a.json', 'b.json', 'c.json', 'node.json'] });
});
test('every browser job sets PM_TEST_WORKERS and runs in the pinned image', () => {
assert.deepEqual([...browserJobIds].sort(), ['browser-workflows', 'frontend-coverage']);
for (const id of browserJobIds) {
const body = allJobs.get(id);
assert.match(body, /^ {6}PM_TEST_WORKERS: 3$/m, `${id} sets PM_TEST_WORKERS for the whole job`);
assert.match(body, PINNED_IMAGE, `${id} keeps the pinned Playwright image`);
assert.match(body, /PIP_SHA256: [0-9a-f]{64}/, `${id} keeps the pinned pip bootstrap`);
assert.match(body, /^ {4}timeout-minutes: \d+$/m);
}
assert.match(mergeJob(), PINNED_IMAGE, 'the merge sees the same workspace paths as the shards');
});
test('page workflows run each browser in two shards', () => {
const workflows = allJobs.get('browser-workflows');
assert.deepEqual(matrixList(workflows, 'browser'), ['firefox', 'webkit']);
assert.deepEqual(matrixList(workflows, 'shard'), ['1', '2']);
assert.match(workflows, /test:workflows -- --project \$\{\{ matrix\.browser \}\} --shard \$\{\{ matrix\.shard \}\}\/2/);
assert.match(workflows, /^ {6}fail-fast: false$/m);
assert.match(workflows, /name: browser-traces-\$\{\{ matrix\.browser \}\}-shard-\$\{\{ matrix\.shard \}\}/);
});
test('every npm ci restores the npm download cache keyed on the vendor lockfile first', () => {
const installing = [...allJobs].filter(([, body]) => /npm ci --prefix tools\/vendor/.test(body));
assert.deepEqual(installing.map(([id]) => id).sort(), ['browser-workflows', 'frontend-coverage', 'frontend-coverage-merge']);
for (const [id, body] of installing) {
const jobSteps = steps(body);
const cacheAt = jobSteps.findIndex(step => /uses: actions\/cache@v\d+/.test(step));
const installAt = jobSteps.findIndex(step => /npm ci --prefix tools\/vendor/.test(step));
assert.ok(cacheAt >= 0 && cacheAt < installAt, `${id} restores the npm cache before npm ci`);
assert.match(jobSteps[cacheAt], /^ {10}path: ~\/\.npm$/m, `${id} caches npm's download cache, not node_modules`);
assert.match(jobSteps[cacheAt], /hashFiles\('tools\/vendor\/package-lock\.json'\)/, `${id} keys the cache on the lockfile`);
assert.match(jobSteps[installAt], /npm ci --prefix tools\/vendor --ignore-scripts/, `${id} still installs without scripts`);
// The key names the job's own pinned image, so an image bump starts a fresh cache.
const image = /^ {6}image: mcr\.microsoft\.com\/playwright:(v\d+\.\d+\.\d+)-noble$/m.exec(body)?.[1];
assert.ok(image, `${id} runs in the pinned Playwright image`);
assert.match(jobSteps[cacheAt], new RegExp(`key: npm-\\$\\{\\{ runner\\.os \\}\\}-playwright-${image.replaceAll('.', '\\.')}-\\$\\{\\{ hashFiles`),
`${id} keys the cache on its image ${image}`);
}
});
test('browser job timeouts keep a real margin over measured runs', () => {
for (const id of ['frontend-coverage', 'browser-workflows']) {
const minutes = Number(/^ {4}timeout-minutes: (\d+)$/m.exec(allJobs.get(id))[1]);
assert.ok(minutes >= 20, `${id} allows at least 20 minutes, since runner speed varies`);
}
});
+33
View File
@@ -0,0 +1,33 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { baselineGaps, measuredSources, readBaseline } from '../../tools/vendor/coverage-baseline-check.mjs';
const floor = { lines: 80, branches: 80, functions: 80, statements: 80 };
test('baseline gaps name measured files without a baseline entry', () => {
const gaps = baselineGaps(['a.js', 'nested/new.mjs'], { 'a.js': floor });
assert.deepEqual(gaps.missing, ['nested/new.mjs']);
assert.deepEqual(gaps.stale, []);
});
test('baseline gaps name baseline entries whose file no longer exists', () => {
const gaps = baselineGaps(['a.js'], { 'a.js': floor, 'gone.js': floor });
assert.deepEqual(gaps.missing, []);
assert.deepEqual(gaps.stale, ['gone.js']);
});
test('baseline gaps are empty when files and baseline agree', () => {
assert.deepEqual(baselineGaps(['a.js'], { 'a.js': floor }), { missing: [], stale: [] });
});
test('the committed baseline covers exactly the measured frontend sources', () => {
const gaps = baselineGaps(measuredSources(), readBaseline());
assert.deepEqual(gaps.missing, [], `Missing baseline (run coverage:update): ${gaps.missing.join(', ')}`);
assert.deepEqual(gaps.stale, [], `Stale baseline entries (file removed): ${gaps.stale.join(', ')}`);
});
test('the real gate and this check select files through the same function', () => {
const gate = readFileSync(new URL('../../tools/vendor/frontend-coverage.mjs', import.meta.url), 'utf8');
assert.match(gate, /discoverSources\(sourceRoot\)/);
});
+44
View File
@@ -0,0 +1,44 @@
// A stand-in for tests/browser/server.py: `--port N` (0 = an OS-chosen port)
// serves the readiness page and announces the bound port on stdout as
// `PM_TEST_SERVER_PORT=<n>`, `--fail` writes to stderr and exits before
// listening, and GET /__exit makes the process die unexpectedly after writing a
// last stderr line. GET /__stderr?text=T writes T to stderr before answering.
// `--split-announce` writes the announcement in two chunks, the first ending
// mid-number. `--exit-on-stdin-eof` exits when its stdin closes, as server.py does.
import { createServer } from 'node:http';
const args = process.argv.slice(2);
const port = Number(args[args.indexOf('--port') + 1]);
if (args.includes('--fail')) {
process.stderr.write('fake server: refusing to start\nlast fake stderr line\n');
process.exit(4);
}
const server = createServer((request, response) => {
const url = new URL(request.url, 'http://fake');
if (url.pathname === '/__stderr') {
process.stderr.write(url.searchParams.get('text'));
response.end('written');
return;
}
if (request.url === '/__exit') {
process.stderr.write('fake server: crashed on purpose\n');
response.end('bye', () => process.exit(3));
return;
}
response.end(request.url === '/prompt_manager/test' ? 'ready' : `pid ${process.pid}`);
});
server.listen(port, '127.0.0.1', () => {
const line = `PM_TEST_SERVER_PORT=${server.address().port}\n`;
if (!args.includes('--split-announce')) {
process.stdout.write(`fake server: listening\n${line}`);
return;
}
// The first chunk is a complete-looking number, one digit short of the real port.
process.stdout.write(`fake server: listening\n${line.slice(0, -2)}`);
setTimeout(() => process.stdout.write(line.slice(-2)), 300);
});
if (args.includes('--exit-on-stdin-eof')) {
process.stdin.resume();
process.stdin.once('end', () => process.exit(0));
}
for (const signal of ['SIGTERM', 'SIGINT']) process.once(signal, () => server.close(() => process.exit(0)));
+93
View File
@@ -0,0 +1,93 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { existsSync, mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { plan, runCoverage } from '../../tools/vendor/frontend-coverage.mjs';
const steps = argv => plan(argv).map(({ step, ...options }) => (Object.keys(options).length ? { step, ...options } : step));
function tempPaths(t) {
const root = mkdtempSync(join(tmpdir(), 'pm-coverage-cli-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const reportRoot = join(root, 'coverage/js');
return { root, paths: { repositoryRoot: root, tools: join(root, 'tools'), reportRoot, raw: join(reportRoot, 'raw'), sourceRoot: join(root, 'web/js') } };
}
function recorder() {
const calls = [];
const run = (args, env = {}) => { calls.push({ args, env }); };
return { calls, run };
}
test('no arguments plans the unchanged full run', () => {
assert.deepEqual(steps([]), [
'clean', 'node-units', 'viewer', { step: 'workflows', shard: null }, 'report', { step: 'check', updateBaseline: false },
]);
assert.deepEqual(steps(['--update-baseline']), [
'clean', 'node-units', 'viewer', { step: 'workflows', shard: null }, 'report', { step: 'check', updateBaseline: true },
]);
});
test('--shard 2/3 plans only workflows with --shard 2/3 and no report', () => {
assert.deepEqual(steps(['--shard', '2/3']), ['prepare', { step: 'workflows', shard: '2/3' }]);
assert.deepEqual(steps(['--shard=3/3']), ['prepare', { step: 'workflows', shard: '3/3' }]);
});
test('--shard 1/N also runs the node units and the viewer suite', () => {
assert.deepEqual(steps(['--shard', '1/3']), ['prepare', 'node-units', 'viewer', { step: 'workflows', shard: '1/3' }]);
assert.deepEqual(steps(['--shard', '1/1']), ['prepare', 'node-units', 'viewer', { step: 'workflows', shard: '1/1' }]);
});
test('--merge-only plans the report and the check over existing raw coverage', () => {
assert.deepEqual(steps(['--merge-only']), ['require-raw', 'report', { step: 'check', updateBaseline: false }]);
assert.deepEqual(steps(['--merge-only', '--update-baseline']), ['require-raw', 'report', { step: 'check', updateBaseline: true }]);
});
test('invalid shard and mode combinations are refused', () => {
for (const argv of [['--shard'], ['--shard', '0/3'], ['--shard', '4/3'], ['--shard', '2'], ['--shard', 'a/b'], ['--shard', '1/0'], ['--shard', '1.5/3']]) {
assert.throws(() => plan(argv), /--shard needs I\/N/, argv.join(' '));
}
assert.throws(() => plan(['--shard', '1/2', '--merge-only']), /either --shard or --merge-only/);
assert.throws(() => plan(['--shard', '1/2', '--update-baseline']), /--update-baseline .* --merge-only or a full run/);
assert.throws(() => plan(['--shard', '1/2', '--shard', '2/2']), /--shard once/);
assert.throws(() => plan(['--report']), /Unknown argument: --report/);
});
test('--merge-only refuses an empty raw dir', async t => {
const { paths } = tempPaths(t);
const { calls, run } = recorder();
await assert.rejects(runCoverage({ argv: ['--merge-only'], run, paths }), /no raw coverage .*coverage\/js\/raw.* run the --shard steps first/i);
mkdirSync(paths.raw, { recursive: true });
await assert.rejects(runCoverage({ argv: ['--merge-only'], run, paths }), /no raw coverage/i);
writeFileSync(join(paths.raw, 'notes.txt'), 'not coverage');
await assert.rejects(runCoverage({ argv: ['--merge-only'], run, paths }), /no raw coverage/i);
assert.deepEqual(calls, [], 'nothing is reported or checked over partial or empty coverage');
});
test('a shard keeps raw coverage other shards wrote and passes its shard to Playwright', async t => {
const { paths } = tempPaths(t);
mkdirSync(paths.raw, { recursive: true });
const earlier = join(paths.raw, 'browser-earlier-shard.json');
writeFileSync(earlier, '{"result":[]}');
const { calls, run } = recorder();
await runCoverage({ argv: ['--shard', '2/3'], run, paths });
assert.ok(existsSync(earlier), 'raw files from another shard survive');
assert.equal(calls.length, 1, 'shard 2 runs only the workflows');
const [{ args, env }] = calls;
assert.deepEqual(args.slice(-6), ['--project', 'chromium', '--project', 'touch', '--shard', '2/3']);
assert.equal(env.PM_JS_COVERAGE, paths.raw);
});
test('shard 1 runs the node units into the shared raw dir, then the viewer suite, then its workflows', async t => {
const { root, paths } = tempPaths(t);
mkdirSync(join(root, 'tests/js'), { recursive: true });
writeFileSync(join(root, 'tests/js/a.test.js'), '');
const { calls, run } = recorder();
await runCoverage({ argv: ['--shard', '1/3'], run, paths });
assert.equal(calls.length, 3);
assert.deepEqual(calls[0].args, ['--test', 'tests/js/a.test.js']);
assert.equal(calls[0].env.NODE_V8_COVERAGE, paths.raw);
assert.ok(calls[1].args.includes(join(paths.tools, 'playwright.config.mjs')));
assert.deepEqual(calls[2].args.slice(-2), ['--shard', '1/3']);
});
+38
View File
@@ -0,0 +1,38 @@
// Fast, browser-free check that frontend-coverage-baseline.json lists exactly the
// files the coverage gate measures. CI's `frontend coverage` job reports a missing
// entry only after ~20 minutes; this reports it in milliseconds.
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { repositoryRoot } from './vendor-viewerjs.mjs';
import { discoverSources } from './frontend-coverage.mjs';
const baselinePath = join(dirname(fileURLToPath(import.meta.url)), 'frontend-coverage-baseline.json');
export function measuredSources() {
return discoverSources(join(repositoryRoot, 'web/js'));
}
export function readBaseline() {
return JSON.parse(readFileSync(baselinePath, 'utf8'));
}
export function baselineGaps(files, baseline) {
const measured = new Set(files);
return {
missing: files.filter(file => !baseline[file]),
stale: Object.keys(baseline).filter(file => !measured.has(file)).sort(),
};
}
function main() {
const { missing, stale } = baselineGaps(measuredSources(), readBaseline());
if (!missing.length && !stale.length) {
console.log('Coverage baseline matches the measured frontend files.');
return;
}
if (missing.length) console.error(`Missing baseline for:\n ${missing.join('\n ')}\nRun: npm run --prefix tools/vendor coverage:update`);
if (stale.length) console.error(`Baseline lists files that no longer exist:\n ${stale.join('\n ')}\nRemove them from tools/vendor/frontend-coverage-baseline.json`);
process.exitCode = 1;
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) main();
+38 -14
View File
@@ -155,12 +155,36 @@
"functions": 98,
"statements": 98
},
"workspace/file-inspector.mjs": {
"workspace/activity-job-view.mjs": {
"lines": 97,
"branches": 83,
"functions": 98,
"statements": 97
},
"workspace/activity-scan.mjs": {
"lines": 95,
"branches": 92,
"functions": 88,
"branches": 64,
"functions": 98,
"statements": 95
},
"workspace/activity-store.mjs": {
"lines": 98,
"branches": 95,
"functions": 98,
"statements": 98
},
"workspace/activity.mjs": {
"lines": 95,
"branches": 87,
"functions": 98,
"statements": 95
},
"workspace/file-inspector.mjs": {
"lines": 96,
"branches": 92,
"functions": 92,
"statements": 96
},
"workspace/heading.mjs": {
"lines": 98,
"branches": 98,
@@ -229,7 +253,7 @@
},
"workspace/index.mjs": {
"lines": 98,
"branches": 96,
"branches": 97,
"functions": 98,
"statements": 98
},
@@ -264,10 +288,10 @@
"statements": 98
},
"workspace/library.mjs": {
"lines": 95,
"lines": 96,
"branches": 85,
"functions": 96,
"statements": 95
"statements": 96
},
"workspace/media-viewer.mjs": {
"lines": 98,
@@ -282,10 +306,10 @@
"statements": 98
},
"workspace/prompt-blocks.mjs": {
"lines": 89,
"branches": 89,
"lines": 90,
"branches": 92,
"functions": 98,
"statements": 89
"statements": 90
},
"workspace/prompt-inspector.mjs": {
"lines": 98,
@@ -302,7 +326,7 @@
"workspace/recorded-run.mjs": {
"lines": 95,
"branches": 83,
"functions": 77,
"functions": 80,
"statements": 95
},
"workspace/saved-views.mjs": {
@@ -325,15 +349,15 @@
},
"workspace/toast.mjs": {
"lines": 98,
"branches": 64,
"branches": 98,
"functions": 98,
"statements": 98
},
"workspace/toolbar.mjs": {
"lines": 94,
"branches": 93,
"lines": 96,
"branches": 96,
"functions": 98,
"statements": 94
"statements": 96
},
"workspace/topbar.mjs": {
"lines": 98,
+99 -20
View File
@@ -1,4 +1,4 @@
import { readFileSync, readdirSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
import { existsSync, readFileSync, readdirSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
import { dirname, join, relative } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { spawnSync } from 'node:child_process';
@@ -7,33 +7,69 @@ import { repositoryRoot } from './vendor-viewerjs.mjs';
import { discoverSources, checkCoverage } from '@comfyassets/toolkit/testing/coverage';
export { discoverSources, checkCoverage };
const metrics = ['lines', 'branches', 'functions', 'statements'];
const SHARD = /^([1-9]\d*)\/([1-9]\d*)$/;
const USAGE = 'usage: frontend-coverage.mjs [--update-baseline] | --shard I/N | --merge-only [--update-baseline]';
function run(args, env = {}) {
function runNode(args, env = {}) {
const result = spawnSync(process.execPath, args, { cwd: repositoryRoot, stdio: 'inherit',
env: { ...process.env, ...env } });
if (result.status !== 0) throw new Error(`Command failed (${result.status}): ${args.join(' ')}`);
}
async function main() {
const tools = dirname(fileURLToPath(import.meta.url));
function defaultPaths() {
const reportRoot = join(repositoryRoot, 'coverage/js');
const raw = join(reportRoot, 'raw');
const sourceRoot = join(repositoryRoot, 'web/js');
return { repositoryRoot, tools: dirname(fileURLToPath(import.meta.url)), reportRoot,
raw: join(reportRoot, 'raw'), sourceRoot: join(repositoryRoot, 'web/js') };
}
function parseShard(value) {
const match = SHARD.exec(value ?? '');
if (!match || Number(match[1]) > Number(match[2])) {
throw new Error(`--shard needs I/N with 1 <= I <= N, got ${value ?? 'nothing'}\n${USAGE}`);
}
return { index: Number(match[1]), total: Number(match[2]), value };
}
// Turn the command line into the ordered steps of one coverage run:
// - no mode: the full run (clean, node units, viewer suite, chromium+touch
// workflows, report, check), exactly as before shards existed;
// - --shard I/N: one slice of the workflows into coverage/js/raw, without wiping
// it or reporting; shard 1 also runs the node units and the viewer suite;
// - --merge-only: report and check over raw coverage the shards already wrote.
export function plan(argv) {
let shard = null;
let mergeOnly = false;
let updateBaseline = false;
for (let index = 0; index < argv.length; index += 1) {
const arg = argv[index];
if (arg === '--update-baseline') updateBaseline = true;
else if (arg === '--merge-only') mergeOnly = true;
else if (arg === '--shard' || arg.startsWith('--shard=')) {
if (shard) throw new Error(`Pass --shard once\n${USAGE}`);
shard = parseShard(arg === '--shard' ? argv[(index += 1)] : arg.slice('--shard='.length));
} else throw new Error(`Unknown argument: ${arg}\n${USAGE}`);
}
if (shard && mergeOnly) throw new Error(`Pass either --shard or --merge-only\n${USAGE}`);
if (shard && updateBaseline) throw new Error(`--update-baseline needs --merge-only or a full run, not a shard\n${USAGE}`);
const check = { step: 'check', updateBaseline };
if (mergeOnly) return [{ step: 'require-raw' }, { step: 'report' }, check];
if (shard) {
const first = shard.index === 1 ? [{ step: 'node-units' }, { step: 'viewer' }] : [];
return [{ step: 'prepare' }, ...first, { step: 'workflows', shard: shard.value }];
}
return [{ step: 'clean' }, { step: 'node-units' }, { step: 'viewer' }, { step: 'workflows', shard: null },
{ step: 'report' }, check];
}
function rawCoverageFiles(raw) {
return existsSync(raw) ? readdirSync(raw).filter(file => file.endsWith('.json')) : [];
}
function checkSummary({ tools, reportRoot, sourceRoot }, updateBaseline) {
const files = discoverSources(sourceRoot);
rmSync(reportRoot, { recursive: true, force: true });
mkdirSync(raw, { recursive: true });
const tests = readdirSync(join(repositoryRoot, 'tests/js')).filter(file => file.endsWith('.test.js')).map(file => `tests/js/${file}`);
run(['--test', ...tests], { NODE_V8_COVERAGE: raw });
const playwright = join(tools, 'node_modules/@playwright/test/cli.js');
run([playwright, 'test', '--config', join(tools, 'playwright.config.mjs')]);
run([playwright, 'test', '--config', join(tools, 'playwright.workflows.config.mjs'),
'--project', 'chromium', '--project', 'touch'], { PM_JS_COVERAGE: raw });
run([join(tools, 'node_modules/c8/bin/c8.js'), 'report', '--all', '--src', sourceRoot,
'--include', 'web/js/**/*.js', '--include', 'web/js/**/*.mjs', '--temp-directory', raw, '--reports-dir', reportRoot,
'--reporter', 'text', '--reporter', 'json-summary', '--reporter', 'html']);
const summary = JSON.parse(readFileSync(join(reportRoot, 'coverage-summary.json'), 'utf8'));
const baselinePath = join(tools, 'frontend-coverage-baseline.json');
if (process.argv.includes('--update-baseline')) {
if (updateBaseline) {
const previous = JSON.parse(readFileSync(baselinePath, 'utf8'));
const failures = checkCoverage(summary, previous, sourceRoot, files.filter(file => previous[file]));
if (failures.length) throw new Error(`Baseline updates cannot lower coverage:\n${failures.join('\n')}`);
@@ -53,8 +89,51 @@ async function main() {
const failures = checkCoverage(summary, baseline, sourceRoot, files);
if (failures.length) throw new Error(`Frontend coverage regressed:\n${failures.join('\n')}`);
}
console.log(`Frontend coverage passed for ${files.length} files. Report: ${relative(repositoryRoot, reportRoot)}/index.html`);
return files.length;
}
export async function runCoverage({ argv = process.argv.slice(2), run = runNode, paths = defaultPaths() } = {}) {
const { repositoryRoot: root, tools, reportRoot, raw, sourceRoot } = paths;
const playwright = join(tools, 'node_modules/@playwright/test/cli.js');
const actions = {
clean() {
rmSync(reportRoot, { recursive: true, force: true });
mkdirSync(raw, { recursive: true });
},
prepare() {
mkdirSync(raw, { recursive: true });
},
'require-raw'() {
if (!rawCoverageFiles(raw).length) {
throw new Error(`No raw coverage in ${relative(root, raw) || raw}: run the --shard steps first, `
+ 'or put every shard\'s raw files there');
}
},
'node-units'() {
const tests = readdirSync(join(root, 'tests/js')).filter(file => file.endsWith('.test.js'))
.map(file => `tests/js/${file}`);
run(['--test', ...tests], { NODE_V8_COVERAGE: raw });
},
viewer() {
run([playwright, 'test', '--config', join(tools, 'playwright.config.mjs')]);
},
workflows({ shard }) {
run([playwright, 'test', '--config', join(tools, 'playwright.workflows.config.mjs'),
'--project', 'chromium', '--project', 'touch', ...(shard ? ['--shard', shard] : [])], { PM_JS_COVERAGE: raw });
},
report() {
run([join(tools, 'node_modules/c8/bin/c8.js'), 'report', '--all', '--src', sourceRoot,
'--include', 'web/js/**/*.js', '--include', 'web/js/**/*.mjs', '--temp-directory', raw, '--reports-dir', reportRoot,
'--reporter', 'text', '--reporter', 'json-summary', '--reporter', 'html']);
},
check({ updateBaseline }) {
const count = checkSummary(paths, updateBaseline);
console.log(`Frontend coverage passed for ${count} files. Report: ${relative(root, reportRoot)}/index.html`);
},
};
for (const { step, ...options } of plan(argv)) await actions[step](options);
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
main().catch(error => { console.error(error.message); process.exitCode = 1; });
runCoverage().catch(error => { console.error(error.message); process.exitCode = 1; });
}
+3
View File
@@ -15,6 +15,9 @@
"test:comfyui": "playwright test --config comfyui.config.mjs",
"test:executions": "playwright test --config playwright.execution.config.mjs",
"test:coverage": "node frontend-coverage.mjs",
"test:coverage:shard": "node frontend-coverage.mjs --shard",
"coverage:merge": "node frontend-coverage.mjs --merge-only",
"coverage:baseline-check": "node coverage-baseline-check.mjs",
"coverage:update": "node frontend-coverage.mjs --update-baseline",
"test:compat": "playwright test --config pack-compat.config.mjs"
},
+6 -11
View File
@@ -1,23 +1,18 @@
import { defineConfig, devices } from '@playwright/test';
import { repositoryRoot } from './vendor-viewerjs.mjs';
const port = Number(process.env.PM_TEST_PORT || 8765);
import { workflowWorkers } from './test-workers.mjs';
// No global webServer: tests/browser/fixtures.mjs starts one server per worker
// on an OS-chosen port. Files run in parallel; tests inside a file keep their order.
export default defineConfig({
testDir: '../../tests/browser',
outputDir: './test-results/workflows',
testMatch: '*.spec.mjs',
workers: 1,
workers: workflowWorkers(),
fullyParallel: false,
forbidOnly: Boolean(process.env.CI),
retries: 0,
reporter: 'list',
use: { baseURL: `http://127.0.0.1:${port}`, trace: 'retain-on-failure', screenshot: 'only-on-failure' },
webServer: {
command: `node tools/vendor/start-browser-server.mjs --port ${port}`,
cwd: repositoryRoot,
url: `http://127.0.0.1:${port}/prompt_manager/test`,
reuseExistingServer: false,
gracefulShutdown: { signal: 'SIGTERM', timeout: 5000 },
},
use: { trace: 'retain-on-failure', screenshot: 'only-on-failure' },
projects: [
{ name: 'chromium', outputDir: './test-results/workflows/chromium', use: { browserName: 'chromium' } },
{ name: 'firefox', outputDir: './test-results/workflows/firefox', use: { browserName: 'firefox' } },
+187 -6
View File
@@ -1,8 +1,189 @@
import { spawn } from 'node:child_process';
const child = spawn(process.env.PM_TEST_PYTHON || 'python',
['-m', 'tests.browser.server', ...process.argv.slice(2)], { stdio: 'inherit' });
child.on('error', error => { console.error(error.message); process.exitCode = 1; });
child.on('exit', code => { process.exitCode = code ?? 1; });
for (const signal of ['SIGTERM', 'SIGINT']) {
process.once(signal, () => child.kill(signal));
import { resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
import { setTimeout as delay } from 'node:timers/promises';
import { repositoryRoot } from './vendor-viewerjs.mjs';
const READY_PATH = '/prompt_manager/test';
const STARTUP_TIMEOUT_MS = 60_000;
const STOP_TIMEOUT_MS = 5_000;
const POLL_INTERVAL_MS = 100;
const PROBE_TIMEOUT_MS = 1_000;
const STDERR_TAIL_BYTES = 8_192;
// Server stderr kept for the failing test's attachment; the newest output wins.
const STDERR_LOG_CHARS = 256 * 1024;
const STDERR_DROPPED = '[earlier server stderr dropped]\n';
const HOST = '127.0.0.1';
// tests/browser/server.py prints this once it holds its listening socket. Only a
// complete line counts: a chunk may end mid-number.
const PORT_ANNOUNCEMENT = /^PM_TEST_SERVER_PORT=(\d+)\r?\n/m;
// The server exits when its stdin closes, so a worker that dies without teardown
// (SIGKILL) leaves no server behind; the launcher holds the pipe's other end.
const pythonServer = port => ['-m', 'tests.browser.server', '--port', String(port), '--exit-on-stdin-eof'];
function describeExit({ code, signal }) {
return signal ? `signal ${signal}` : `code ${code}`;
}
// Start one disposable tests/browser/server.py and wait until it serves the
// readiness page. `port` 0 (the default) lets the server bind an OS-chosen port
// itself and announce it on stdout, so nothing can take the port between
// choosing and binding. Resolves to {port, url, exited, exitStatus,
// pid, stderrTail(), stderrMark(), stderrSince(mark), stop()}; rejects with the
// server's stderr tail when it exits or stays unready. stdin is a pipe this
// process holds open for the server's life.
export async function startBrowserServer({
port = 0,
command = process.env.PM_TEST_PYTHON || 'python',
args = pythonServer,
cwd = repositoryRoot,
timeoutMs = STARTUP_TIMEOUT_MS,
stderrLogChars = STDERR_LOG_CHARS,
} = {}) {
if (!Number.isInteger(port) || port < 0 || port > 65535) {
throw new Error(`startBrowserServer needs a port from 0 to 65535, got ${port}`);
}
const child = spawn(command, args(port), { cwd, stdio: ['pipe', 'pipe', 'pipe'] });
// Never written; closing it (or this process dying) tells the server to exit.
child.stdin.on('error', () => {});
let stderr = '';
let stderrTotal = 0;
child.stderr.setEncoding('utf8');
child.stderr.on('data', chunk => {
stderr = (stderr + chunk).slice(-Math.max(stderrLogChars, STDERR_TAIL_BYTES));
stderrTotal += chunk.length;
});
let stdout = '';
let announced = null;
child.stdout.setEncoding('utf8');
// Keep draining stdout for the server's whole life so it never blocks on a full pipe.
child.stdout.on('data', chunk => {
if (announced !== null) return;
stdout = (stdout + chunk).slice(-STDERR_TAIL_BYTES);
const match = PORT_ANNOUNCEMENT.exec(stdout);
if (match) announced = Number(match[1]);
});
const killOnExit = () => child.kill('SIGKILL');
process.once('exit', killOnExit);
const server = {
pid: child.pid,
port: port || null,
url: port ? `http://${HOST}:${port}` : null,
exitStatus: null,
stopping: false,
stderrTail: () => stderr.slice(-STDERR_TAIL_BYTES).trimEnd(),
// A position in the server's stderr stream, for stderrSince().
stderrMark: () => stderrTotal,
// What the server wrote after `mark`, at most stderrLogChars of the newest of it.
stderrSince(mark) {
const kept = Math.min(stderr.length, stderrLogChars);
const keptFrom = stderrTotal - kept;
const log = stderr.slice(stderr.length - kept);
return mark >= keptFrom ? log.slice(mark - keptFrom) : STDERR_DROPPED + log;
},
exited: new Promise(done => {
child.once('error', error => {
stderr += `\n${error.message}`;
server.exitStatus = { code: null, signal: null, error };
done(server.exitStatus);
});
child.once('exit', (code, signal) => {
server.exitStatus = { code, signal };
done(server.exitStatus);
});
}),
async stop() {
server.stopping = true;
child.stdin.destroy();
if (server.exitStatus === null) {
child.kill('SIGTERM');
const stopped = await Promise.race([server.exited, delay(STOP_TIMEOUT_MS).then(() => false)]);
if (!stopped) {
child.kill('SIGKILL');
await server.exited;
}
}
process.removeListener('exit', killOnExit);
},
};
server.exited.then(() => process.removeListener('exit', killOnExit));
const failure = async reason => {
await server.stop();
const tail = server.stderrTail();
return new Error(`test server on port ${server.port ?? '(not yet announced)'} ${reason}`
+ (tail ? `\n--- server stderr tail ---\n${tail}` : ''));
};
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (server.exitStatus !== null) {
throw await failure(`exited with ${describeExit(server.exitStatus)} before it was ready`);
}
if (announced !== null && server.url === null) {
server.port = announced;
server.url = `http://${HOST}:${announced}`;
}
if (announced !== null && announced !== server.port) {
throw await failure(`announced port ${announced} instead`);
}
if (server.url === null) {
await delay(POLL_INTERVAL_MS);
continue;
}
try {
const response = await fetch(`${server.url}${READY_PATH}`, { signal: AbortSignal.timeout(PROBE_TIMEOUT_MS) });
await response.arrayBuffer();
if (response.ok) return server;
} catch {
// Not listening yet.
}
await delay(POLL_INTERVAL_MS);
}
throw await failure(`was not ready at ${READY_PATH} within ${timeoutMs} ms`);
}
// After a test: when it did not end as expected, attach what the server wrote to
// stderr since `mark` (an aiohttp traceback names the cause a "got 500" hides).
export async function attachServerStderr(testInfo, server, mark) {
if (testInfo.status === testInfo.expectedStatus) return;
const log = server.stderrSince(mark);
if (log) await testInfo.attach('server-stderr', { body: log, contentType: 'text/plain' });
}
// Keep one live server for a Playwright worker. ensure() returns the running
// server, or starts a fresh one when the last one died (a failed start is
// reported, not retried); crash(server) turns an unexpected exit into an error
// carrying the stderr tail.
export function createServerSupervisor({ start }) {
let current = null;
return {
async ensure() {
if (current && current.exitStatus === null) return current;
current = null;
current = await start();
return current;
},
crash(server) {
if (!server || server.stopping || server.exitStatus === null) return null;
const tail = server.stderrTail();
return new Error(`test server on port ${server.port} exited unexpectedly (${describeExit(server.exitStatus)})`
+ (tail ? `\n--- server stderr tail ---\n${tail}` : ''));
},
async stop() {
const server = current;
current = null;
await server?.stop();
},
};
}
// CLI: `node start-browser-server.mjs --port N` runs one server in the foreground.
if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) {
const child = spawn(process.env.PM_TEST_PYTHON || 'python',
['-m', 'tests.browser.server', ...process.argv.slice(2)], { stdio: 'inherit' });
child.on('error', error => { console.error(error.message); process.exitCode = 1; });
child.on('exit', code => { process.exitCode = code ?? 1; });
for (const signal of ['SIGTERM', 'SIGINT']) {
process.once(signal, () => child.kill(signal));
}
}
+18
View File
@@ -0,0 +1,18 @@
import { availableParallelism } from 'node:os';
const MAX_LOCAL_WORKERS = 8;
const CPUS_PER_WORKER = 4;
// PM_TEST_WORKERS wins; CI without it keeps one worker, so CI only changes
// through explicit workflow edits; locally a quarter of the CPUs, at most eight.
export function workflowWorkers({ env = process.env, cpus = availableParallelism() } = {}) {
if (env.PM_TEST_WORKERS !== undefined) {
const workers = Number(env.PM_TEST_WORKERS);
if (!Number.isInteger(workers) || workers < 1) {
throw new Error(`PM_TEST_WORKERS must be a positive integer, got "${env.PM_TEST_WORKERS}"`);
}
return workers;
}
if (env.CI) return 1;
return Math.max(1, Math.min(MAX_LOCAL_WORKERS, Math.floor(cpus / CPUS_PER_WORKER)));
}
+1 -1
View File
@@ -1,7 +1,7 @@
/**
* State of the output-image scan as seen by the dashboard.
*
* The scan runs on the server as a background job (see py/api/scan_job.py) and
* The scan runs on the server as a background job (see py/api/jobs.py) and
* reports progress over SSE. This module turns those events into a small
* immutable state object and answers the UI questions that depend on it:
* what dismissing the modal should do, what the minimized pill should say, and
+445
View File
@@ -0,0 +1,445 @@
/**
* One job's Activity view (#249 Activity-1, D56): `#/activity/<id>`, the approved mockup `activity-scan.html` in the
* concept's classes.
*
* - Header: the page heading names the job; its line is the Activity item's (D54), and its actions are Download report
* (the job's JSON report, D59) and, while the job runs, "Cancel scan" (danger).
* - Progress (`section.surface`): `<progress>`, the server's "Processing file N of M · about X min left", and the
* outcome counters as `dl.stat-row` tiles: Added to Library (green), Already known, Needs review (warning), No usable
* metadata, Errors (red).
* - Results (`section.surface`): the outcome list (`button[aria-pressed]` with counts, in the mockup's order), a path
* filter, and the selected outcome's listed rows (at most 200 per outcome, so "Showing N of M"). Rows under Added and
* Already known open the file in the existing viewer through its listing identity (D57); the others are plain text.
* - Log: `details.term` "Scan log", collapsed, holding the raw timestamped lines in `pre[tabindex=0]`.
*
* The summary follows the Activity store (one SSE stream per running job); the listed rows and the log are read from
* `GET /prompt_manager/jobs/<id>` at most once a second while the job runs, and once more when it ends.
*/
import { createButton } from '../shared/toolkit/core.js';
import { pageHeading } from './heading.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { noun } from './plural.mjs';
import { parseJob, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
import { jobDetail, CANCEL_FAILED_TOAST } from './activity.mjs';
import { downloadJobReport } from './activity-scan.mjs';
import { createMediaViewer } from './media-viewer.mjs';
import { createInspectionController } from './inspection-state.mjs';
import { createPromptSender } from './prompt-sender.mjs';
import { imageKey } from './images-state.mjs';
export const DETAIL_INTERVAL_MS = 1000;
export const NOT_FOUND = 'This job is not in this session’s Activity. Jobs are kept until ComfyUI restarts.';
export const EMPTY_GROUP = 'No files in this group.';
export const NO_MATCH = 'No listed file matches this filter.';
export const VIEWER_HINT = 'each row opens the image in the viewer';
export const FILE_GONE = 'This file is no longer in Images. Rescan from Import and try again.';
export const OPEN_FAILED = 'The file could not be opened. Try again.';
const LISTING_LIMIT = 5;
/** The counter tiles, in the spec's order and the mockup's tones. */
export const TILES = Object.freeze([
['added', 'Added to Library', 'added'], ['known', 'Already known', ''], ['review', 'Needs review', 'warn'],
['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad'],
]);
/** The outcome list, in the mockup's order: what got added and what went wrong first. */
export const OUTCOMES = Object.freeze([
['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
['no_metadata', 'No usable metadata'], ['known', 'Already known'],
]);
const OPENABLE = new Set(['added', 'known']);
function element(tag, text, className) {
const node = document.createElement(tag);
if (text !== undefined) node.textContent = text;
if (className) node.className = className;
return node;
}
// Grouped as the server writes its lines, whatever the browser's locale (as images.mjs does).
const number = value => value.toLocaleString('en-US');
const isText = value => typeof value === 'string';
/** A job's detail read: the summary plus its listed rows per outcome and its log; null when it is not one. */
export function parseDetail(raw) {
const job = parseJob(raw);
if (!job) return null;
const outcomes = {};
const groups = raw.outcomes !== null && typeof raw.outcomes === 'object' ? raw.outcomes : {};
for (const [group, rows] of Object.entries(groups)) {
if (!Array.isArray(rows)) continue;
outcomes[group] = rows.filter(row => row && isText(row.path) && row.path)
.map(row => ({ path: row.path, root: isText(row.root) ? row.root : '',
root_index: Number.isSafeInteger(row.root_index) && row.root_index >= 0 ? row.root_index : null,
detail: isText(row.detail) ? row.detail : '' }));
}
const log = Array.isArray(raw.log) ? raw.log.filter(isText) : [];
return { job, outcomes, log };
}
/** The progress line: the server's while running, then what the job got through. */
export function progressLine(job) {
const files = `${number(job.processed)} of ${number(job.total)} ${noun(job.total, 'file')}`;
if (job.state === 'running') return job.status_line || `${job.progress}%`;
if (job.state === 'finished') return `${files} processed`;
if (job.state === 'cancelled') return `Cancelled after ${files}`;
if (job.state === 'lost') return LOST_DETAIL;
return job.status_line || 'The job stopped on an error.';
}
/** The note under the rows: "Showing N of M" when not every file is listed, and the viewer hint for openable rows. */
export function resultsNote(shown, total, openable) {
const parts = [];
if (shown < total) parts.push(`Showing ${number(shown)} of ${number(total)}`);
if (openable && shown > 0) parts.push(VIEWER_HINT);
if (!parts.length) return '';
const text = parts.join(' · ');
return parts.length === 1 && openable && shown >= total ? `${text[0].toUpperCase()}${text.slice(1)}.` : text;
}
/** Rows whose path (or root label and path) holds the case-folded filter. */
export function filterRows(rows, filter) {
const wanted = filter.trim().toLocaleLowerCase();
if (!wanted) return rows;
return rows.filter(row => `${row.root}/${row.path}`.toLocaleLowerCase().includes(wanted));
}
/** The first outcome with files, so a view opens on what happened; Added when nothing has yet. */
export function defaultOutcome(counters) {
return OUTCOMES.find(([group]) => (counters[group] ?? 0) > 0)?.[0] ?? 'added';
}
function section(title, id) {
const node = element('section', undefined, 'surface');
const heading = element('div', undefined, 'panel-heading');
const h2 = element('h2', title);
h2.id = id;
node.setAttribute('aria-labelledby', id);
heading.append(h2);
node.append(heading);
return { node, heading };
}
/** `#/activity/<id>`: one job's view. */
export function mountActivityJobView(main, { store, scope, api, jobId }) {
const parts = pageHeading(main);
const download = createButton('Download report');
const cancel = createButton('Cancel scan', 'danger');
const progressPart = section('Progress', 'job-progress-title');
const status = element('span', '', 'small');
progressPart.heading.append(status);
const bar = element('progress');
bar.max = 100;
const stats = element('dl', undefined, 'stat-row');
const tileValues = new Map();
for (const [group, label, tone] of TILES) {
const tile = element('div', undefined, tone ? `stat ${tone}` : 'stat');
const value = element('dd', '0');
tile.append(element('dt', label), value);
tileValues.set(group, value);
stats.append(tile);
}
progressPart.node.append(bar, stats);
const resultsPart = section('Results', 'job-results-title');
const search = element('label', undefined, 'search-field');
const input = element('input');
input.type = 'search';
input.placeholder = 'Filter by file path…';
input.setAttribute('aria-label', 'Filter results');
search.append(input);
resultsPart.heading.append(search);
const outcomes = element('div', undefined, 'outcomes');
const list = element('div', undefined, 'outcome-list');
list.setAttribute('role', 'group');
list.setAttribute('aria-label', 'Outcomes');
const files = element('div', undefined, 'outcome-files');
const buttons = new Map();
for (const [group, label] of OUTCOMES) {
const button = createButton('');
const count = element('span', '0');
button.append(element('span', label), count);
button.setAttribute('aria-pressed', 'false');
button.count = count;
buttons.set(group, button);
list.append(button);
}
outcomes.append(list, files);
resultsPart.node.append(outcomes);
const log = element('details', undefined, 'term');
const summary = element('summary');
const logLabel = document.createTextNode('');
summary.append(logLabel, element('span', ' — raw detail, click to show'));
const pre = element('pre', '');
pre.tabIndex = 0;
log.append(summary, pre);
const missing = element('section', undefined, 'surface');
const back = element('a', 'All activity');
back.href = '#/activity';
missing.append(element('p', NOT_FOUND, 'empty-state'), back);
let detail = null;
let selected = null;
let mounted = false;
let lastRead = 0;
let reading = false;
let readAgain = false;
let readTimer = null;
let roots = null;
let lastState = null;
// Set once this page's own list read answered, or the detail read said the job is unknown.
let listedOnce = false;
let notFound = false;
// The store is the truth once it has read the list: a job it no longer has was pruned by the server.
const current = () => store.find(jobId) ?? (listedOnce ? null : detail?.job ?? null);
const inspection = createInspectionController({ api });
scope.cleanup(() => inspection.dispose());
const viewer = createMediaViewer({ api, container: main, parentScope: scope, inspectionController: inspection,
sender: createPromptSender({ scope }), onSelect: image => inspection.select(image) });
function showBody(job) {
if (mounted) return;
mounted = true;
missing.remove();
parts.actions.append(download, cancel);
main.append(progressPart.node, resultsPart.node, log);
selected ??= defaultOutcome(job.counters);
}
/** The job is not (or no longer) in this session: say so instead of an empty view. */
function showMissing() {
if (mounted) {
mounted = false;
const focused = main.contains(document.activeElement) && document.activeElement !== parts.h1;
download.remove();
cancel.remove();
progressPart.node.remove();
resultsPart.node.remove();
log.remove();
parts.h1.textContent = 'Activity';
parts.description.textContent = '';
document.title = 'Activity · Prompt Manager';
if (focused) parts.h1.focus();
}
if (!missing.isConnected) main.append(missing);
}
// Listed rows only ever grow, so a row keeps its node across reads: a focused row (or the viewer's trigger) stays.
let rowNodes = new Map();
function rowNode(row, index, openable) {
const key = `${selected}\n${index}\n${row.root}\n${row.path}`;
let node = rowNodes.get(key);
if (!node) {
node = openable ? createButton('', 'file-row') : element('div', undefined, 'file-row');
node.append(element('span', row.path, 'path'), element('span', row.detail, 'why'));
if (openable) node.row = row;
}
return [key, node];
}
function renderRows(job) {
const rows = detail?.outcomes[selected] ?? [];
const openable = OPENABLE.has(selected);
const wanted = filterRows(rows.map((row, index) => ({ ...row, index })), input.value);
const kept = new Map(wanted.map(row => rowNode(row, row.index, openable)));
rowNodes = kept;
const nodes = [...kept.values()];
// The counter is exact; at most 200 of its files are listed, and the filter narrows those.
const total = Math.max(job.counters[selected] ?? 0, rows.length);
if (!nodes.length) nodes.push(element('p', rows.length ? NO_MATCH : EMPTY_GROUP, 'empty-state'));
const note = wanted.length ? resultsNote(wanted.length, total, openable) : '';
if (note) nodes.push(element('p', note, 'small'));
for (const child of [...files.children]) if (!nodes.includes(child)) child.remove();
nodes.forEach((node, index) => {
// Only a node out of place moves, so focus survives a re-read.
if (files.children[index] !== node) files.insertBefore(node, files.children[index] ?? null);
});
}
function render() {
const job = current();
if (!job) {
if (listedOnce || notFound) showMissing();
return;
}
showBody(job);
const scan = job.kind === 'scan';
parts.h1.textContent = job.title;
document.title = `${job.title} · Prompt Manager`;
parts.description.textContent = jobDetail(job);
cancel.textContent = scan ? 'Cancel scan' : 'Cancel job';
// A job lost with the server has no report there (review M9).
const lost = job.state === 'lost';
if (lost && document.activeElement === download) parts.h1.focus();
download.disabled = lost;
if (job.state === 'running') {
if (!cancel.isConnected) parts.actions.append(cancel);
} else if (cancel.isConnected) {
// A focused Cancel that goes away hands its focus to the job's heading, never to the page body.
if (document.activeElement === cancel) parts.h1.focus();
cancel.remove();
}
status.textContent = progressLine(job);
bar.value = job.progress;
bar.textContent = `${job.progress}%`;
bar.setAttribute('aria-label', scan ? 'Scan progress' : `${job.title} progress`);
for (const [group, value] of tileValues) value.textContent = number(job.counters[group] ?? 0);
for (const [group, button] of buttons) {
button.count.textContent = number(job.counters[group] ?? 0);
button.setAttribute('aria-pressed', String(group === selected));
}
logLabel.textContent = `● ${scan ? 'Scan' : 'Job'} log`;
const text = detail?.log.length ? detail.log.join('\n') : 'No log lines yet.';
if (pre.textContent !== text) {
// Replacing the text clamps the scroll position, so keep it: pinned readers follow the end, others stay.
const { scrollTop } = pre;
const atEnd = pre.scrollTop + pre.clientHeight >= pre.scrollHeight - 2;
pre.textContent = text;
pre.scrollTop = atEnd ? pre.scrollHeight : scrollTop;
}
renderRows(job);
}
/** Read the rows and the log, at most once a second; a request while one is in flight reads once more after it. */
function scheduleRead() {
if (scope.signal.aborted) return;
if (reading) { readAgain = true; return; }
if (readTimer !== null) return;
const wait = Math.max(0, lastRead + DETAIL_INTERVAL_MS - Date.now());
readTimer = setTimeout(() => { readTimer = null; read(); }, wait);
}
scope.cleanup(() => { if (readTimer !== null) clearTimeout(readTimer); });
async function read() {
reading = true;
lastRead = Date.now();
try {
const result = await api.get(`${JOBS_URL}/${jobId}`, { signal: scope.signal, cache: 'no-store' });
if (scope.signal.aborted) return;
const parsed = parseDetail(result?.job);
if (parsed && parsed.job.id === jobId) detail = parsed;
} catch (error) {
if (scope.signal.aborted) return;
// An unknown job: the page says so unless the store still knows it (then the rows wait for a later read).
if (error?.status === 404) notFound = true;
} finally {
reading = false;
}
render();
// Read once more only if the summary moved past what this read returned (review: no redundant re-read).
if (readAgain) {
readAgain = false;
const job = store.find(jobId);
if (job && job.state !== 'lost' && detailChanged(job)) scheduleRead();
}
}
/**
* Whether the detail read would change the page: more listed rows in any outcome, or (with the log open) more log
* lines. Counters come with the summary, so a job whose lists are capped and whose log is closed is not re-read.
*/
function detailChanged(job) {
if (!detail) return true;
const groups = new Set([...Object.keys(job.listed), ...Object.keys(detail.outcomes)]);
for (const group of groups) if ((job.listed[group] ?? 0) !== (detail.outcomes[group]?.length ?? 0)) return true;
return log.open && job.log_length !== detail.log.length;
}
function follow() {
const job = store.find(jobId);
render();
if (!job) return;
// A job lost with the server cannot be read; any other is read while what it lists changes.
if (job.state !== 'lost' && (job.state === 'running' || lastState === 'running') && detailChanged(job)) scheduleRead();
lastState = job.state;
}
scope.listen(log, 'toggle', () => {
// Every open starts at the newest line: while closed the log has no layout box (Firefox, WebKit), so a kept
// position would be lost anyway (review M6).
if (log.open) pre.scrollTop = pre.scrollHeight;
const job = store.find(jobId);
if (log.open && job && job.state !== 'lost' && detailChanged(job)) scheduleRead();
});
scope.listen(download, 'click', async () => {
if (download.disabled || download.getAttribute('aria-disabled') === 'true') return;
download.setAttribute('aria-disabled', 'true');
try {
// A view left while the report downloads shows no toast (review M10).
await downloadJobReport(jobId, { signal: scope.signal });
} finally {
download.removeAttribute('aria-disabled');
}
});
scope.listen(cancel, 'click', async () => {
if (cancel.getAttribute('aria-disabled') === 'true') return;
cancel.setAttribute('aria-disabled', 'true');
try {
await store.cancel(jobId);
} catch {
if (!scope.signal.aborted) showWorkspaceToast(CANCEL_FAILED_TOAST);
} finally {
cancel.removeAttribute('aria-disabled');
}
});
scope.listen(list, 'click', event => {
const button = event.target.closest('button');
if (!button) return;
selected = [...buttons].find(([, node]) => node === button)?.[0] ?? selected;
render();
});
scope.listen(input, 'input', () => { const job = current(); if (job) renderRows(job); });
scope.listen(files, 'click', event => {
const button = event.target.closest('button.file-row');
if (button?.row) openRow(button.row, button);
});
/**
* The listing's own item for a row, by its exact identity (D57): the root index the scan recorded, still carrying
* the row's label (so a reordered or replaced source never opens another root's file), and its exact relative path.
*/
async function listed(row) {
if (row.root_index === null) return null;
if (!roots) {
const data = await api.get('/prompt_manager/gallery/subfolders?mode=workspace', { signal: scope.signal });
roots = Array.isArray(data?.roots) ? data.roots : [];
}
const root = roots.find(item => item?.index === row.root_index);
if (!root || root.label !== row.root) return null;
const params = new URLSearchParams({ mode: 'workspace', root: String(root.index), path: row.path,
limit: String(LISTING_LIMIT) });
const page = await api.get(`/prompt_manager/images/output?${params}`, { signal: scope.signal });
return (Array.isArray(page?.images) ? page.images : []).find(item => item?.path === row.path
&& item.root_index === root.index) ?? null;
}
async function openRow(row, trigger) {
if (trigger.getAttribute('aria-disabled') === 'true') return;
trigger.setAttribute('aria-disabled', 'true');
try {
const item = await listed(row);
if (scope.signal.aborted) return;
if (!item) { showWorkspaceToast(FILE_GONE); return; }
viewer.open({ items: [item], selectedKey: imageKey(item), trigger });
} catch {
if (!scope.signal.aborted) showWorkspaceToast(OPEN_FAILED);
} finally {
trigger.removeAttribute('aria-disabled');
}
}
lastState = store.find(jobId)?.state ?? null;
scope.cleanup(store.subscribe(follow));
render();
read();
// A job that finished before this page loaded is only in the list read; once it answered, the store decides.
store.load().then(() => {
if (scope.signal.aborted || store.status !== 'ready') return;
listedOnce = true;
render();
});
}
+105
View File
@@ -0,0 +1,105 @@
/**
* Scan for metadata (#249 Activity-1, D55): Images → Import → "Scan for metadata" → Review scan opens the concept's
* "Review metadata scan". Its backup box downloads the existing `/prompt_manager/backup` copy and toasts the outcome;
* Start scan starts the session job through the Activity store over the chosen scope (new and unresolved files, or
* all files with the D58 checkbox), which toasts "Scan started.", and opens the job's view.
* A start refused because another record-changing job runs toasts "Another job is running: <title>." with a link to
* that job (D57).
*/
import { openScanReviewDialog } from './workspace-dialogs.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { navigateTo } from './state.mjs';
export const BACKUP_URL = '/prompt_manager/backup';
export const BACKUP_DONE = 'Database backup downloaded.';
export const BACKUP_FAILED = 'The database could not be backed up. Try again.';
export const REPORT_DONE = 'Job report downloaded.';
export const REPORT_FAILED = 'The report could not be downloaded. Try again.';
const JOBS_REPORT_BASE = '/prompt_manager/jobs';
export const START_FAILED = 'The scan could not start. Try again.';
const FALLBACK_NAME = 'prompts_backup.db';
const REPORT_FALLBACK_NAME = 'prompt-manager-report.json';
/** The download name from a Content-Disposition header, or `fallback`. */
export function backupFilename(disposition, fallback = FALLBACK_NAME) {
const match = /filename="?([^";]+)"?/i.exec(disposition || '');
return match ? match[1] : fallback;
}
/** Save a fetched response through a temporary link, named by `filename`. */
function saveResponse(blob, filename) {
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = filename;
document.body.append(link);
link.click();
link.remove();
setTimeout(() => URL.revokeObjectURL(url), 1000);
}
/** Download the prompts database copy the legacy admin page also uses; toasts the outcome. */
export async function downloadDatabaseBackup({ fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast } = {}) {
try {
const response = await fetchUrl(BACKUP_URL);
if (!response.ok) throw new Error(`Backup failed with ${response.status}`);
const blob = await response.blob();
saveResponse(blob, backupFilename(response.headers.get('Content-Disposition')));
notify(BACKUP_DONE);
return true;
} catch {
notify(BACKUP_FAILED);
return false;
}
}
/**
* Download a job's JSON report (Activity-2, D59) through the api-client URL, so a proxy mount works; toasts the outcome
* unless `signal` (the view's scope) was aborted meanwhile.
*/
export async function downloadJobReport(jobId, { fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast,
signal = null } = {}) {
try {
const response = await fetchUrl(`${JOBS_REPORT_BASE}/${encodeURIComponent(jobId)}/report`);
if (!response.ok) throw new Error(`Report failed with ${response.status}`);
const blob = await response.blob();
saveResponse(blob, backupFilename(response.headers.get('Content-Disposition'), REPORT_FALLBACK_NAME));
// The file is saved either way; a view left meanwhile (`signal` aborted) shows no toast.
if (!signal?.aborted) notify(REPORT_DONE);
return true;
} catch {
if (!signal?.aborted) notify(REPORT_FAILED);
return false;
}
}
/** The toast for a refused start, linking the job that holds the slot. */
export function busyToast(running) {
if (!running) return 'Another job is running. Try again when it ends.';
return ['Another job is running: ', { text: running.title, href: `#/activity/${running.id}` }, '.'];
}
/**
* Start the scan job over `files` ('new': new and unresolved files, or 'all') and open its view; a refused or failed
* start says why and stays on the page.
*/
export async function startMetadataScan({ store, scope, files = 'new', notify = showWorkspaceToast, navigate = navigateTo }) {
try {
const result = await store.start('scan', { scope: files });
if (scope.signal.aborted) return;
if (result.busy !== undefined) {
notify(busyToast(result.busy));
return;
}
navigate(`#/activity/${result.job.id}`);
} catch {
if (!scope.signal.aborted) notify(START_FAILED);
}
}
/** Open "Review metadata scan" for `sources` output folders. */
export function openMetadataScan({ scope, store, sources, trigger = null }) {
openScanReviewDialog({ scope, sources, trigger,
onBackup: () => downloadDatabaseBackup(),
onStart: ({ scope: files }) => startMetadataScan({ store, scope, files }) });
}
+313
View File
@@ -0,0 +1,313 @@
/**
* The Activity store (#249 Activity-1): this server session's jobs, for the top-bar button and pill and the Activity
* page. One `GET /prompt_manager/jobs` read, then one SSE stream (`/jobs/<id>/events`) per running job.
*
* - A dropped stream re-reads the list after a backoff that doubles from 1 s and is capped at 30 s, so a stopped
* server never sees a retry storm. A successful stream message resets the backoff.
* - While the tab is visible the list is read again every 15 s, so a job started elsewhere (another tab, the legacy
* admin page) appears without a reload.
* - Jobs live only in server memory. A running job the server no longer lists (ComfyUI restarted), or one whose
* server cannot be read at all, is shown as ended (`lost`), so no pill stays stuck.
* - Toasts announce what happened; Activity is where progress lives. "Scan started." follows this tab's own start;
* the finish toast follows any job this store saw running when it ends.
*/
import { showWorkspaceToast } from './toast.mjs';
import { noun } from './plural.mjs';
export const JOBS_URL = '/prompt_manager/jobs';
export const BACKOFF_FIRST_MS = 1000;
export const BACKOFF_MAX_MS = 30000;
export const POLL_MS = 15000;
export const LOST_AFTER_FAILURES = 2;
export const LOST_DETAIL = 'This job ended when ComfyUI restarted.';
const SERVER_STATES = new Set(['running', 'finished', 'cancelled', 'failed']);
const FINAL_STATES = new Set(['finished', 'cancelled', 'failed']);
const JOB_ID = /^[0-9a-f]{32}$/;
// The pill's verb for each job kind; a kind without one shows its title.
const PILL_VERBS = Object.freeze({ scan: 'Scanning' });
const START_TOASTS = Object.freeze({ scan: 'Scan started.' });
/** The wait before the `attempt`-th re-read (0-based): 1 s, 2 s, 4 s … never more than 30 s. */
export function backoffDelay(attempt) {
const step = Number.isSafeInteger(attempt) && attempt > 0 ? Math.min(attempt, 16) : 0;
return Math.min(BACKOFF_MAX_MS, BACKOFF_FIRST_MS * 2 ** step);
}
const count = value => Number.isSafeInteger(value) && value >= 0 ? value : 0;
const time = value => typeof value === 'number' && Number.isFinite(value) ? value : null;
const plainObject = value => value !== null && typeof value === 'object' && !Array.isArray(value);
/** A job summary from the API, or null when it is not one (unknown state, bad id). */
export function parseJob(raw) {
if (!plainObject(raw) || typeof raw.id !== 'string' || !JOB_ID.test(raw.id) || !SERVER_STATES.has(raw.state)) return null;
const counters = {};
if (plainObject(raw.counters)) for (const [key, value] of Object.entries(raw.counters)) counters[key] = count(value);
// What the job's detail lists per outcome, and its log length: a view re-reads the detail only when they change.
const listed = {};
if (plainObject(raw.listed)) for (const [key, value] of Object.entries(raw.listed)) listed[key] = count(value);
const progress = typeof raw.progress === 'number' && Number.isFinite(raw.progress) ? Math.round(raw.progress) : 0;
return {
id: raw.id,
kind: typeof raw.kind === 'string' ? raw.kind : '',
title: typeof raw.title === 'string' && raw.title.trim() ? raw.title : 'Background job',
state: raw.state,
started_at: time(raw.started_at),
finished_at: time(raw.finished_at),
progress: Math.max(0, Math.min(100, progress)),
status_line: typeof raw.status_line === 'string' ? raw.status_line : '',
processed: count(raw.processed),
total: count(raw.total),
counters,
scope: plainObject(raw.scope) ? { ...raw.scope } : {},
listed,
log_length: count(raw.log_length),
};
}
// Grouped as the server writes its status lines ("8,240 of 19,622"), whatever the browser's locale.
const number = value => value.toLocaleString('en-US');
const errorsOf = job => `${number(job.counters.errors ?? 0)} ${noun(job.counters.errors ?? 0, 'error')}`;
/** "1,420 added". */
export const addedOf = job => `${number(job.counters.added ?? 0)} added`;
/** The toast when a job ends: "Scan finished · N added · M errors", "Scan cancelled · N added". */
export function finishToast(job) {
if (job.kind === 'scan') {
if (job.state === 'cancelled') return `Scan cancelled · ${addedOf(job)}`;
if (job.state === 'failed') return `Scan failed · ${addedOf(job)} · ${errorsOf(job)}`;
return `Scan finished · ${addedOf(job)} · ${errorsOf(job)}`;
}
if (job.state === 'cancelled') return `${job.title} was cancelled.`;
if (job.state === 'failed') return `${job.title} failed.`;
return `${job.title} finished.`;
}
/** The toast after this tab starts a job. */
export function startToast(job) {
return START_TOASTS[job.kind] ?? `${job.title} started.`;
}
/** The pill's text: "Scanning · 42% · 8,240 / 19,622"; the file counts only once the total is known. */
export function pillParts(job) {
const label = `${PILL_VERBS[job.kind] ?? job.title} · ${job.progress}%`;
const files = job.total > 0 ? ` · ${number(job.processed)} / ${number(job.total)}` : '';
return { label, files, text: label + files };
}
/** The scan's counters as one line: "N added · M errors"; other kinds have none yet. */
export function outcomeLine(job) {
return job.kind === 'scan' ? `${addedOf(job)} · ${errorsOf(job)}` : '';
}
function defaultConnect(url) {
const resolve = globalThis.ApiClient?.resolveUrl ?? (value => value);
return new EventSource(resolve(url));
}
/**
* @param {{api: {get: Function, post: Function}, scope: {signal: AbortSignal, cleanup: Function},
* connect?: (url: string) => {onmessage: Function|null, onerror: Function|null, close: Function},
* notify?: (message: string) => void, timers?: {setTimeout: Function, clearTimeout: Function},
* pollMs?: number, isVisible?: () => boolean, lifecycle?: EventTarget}} options
*
* `lifecycle` (the window) ends the store's work on `pagehide`: the periodic and backoff reads stop, the streams close
* and a read in flight is aborted, so nothing answers into a page that is going away (a closed tab, a test teardown).
* A `pageshow` from the back-forward cache reads the list again.
*/
export function createActivityStore({ api, scope, connect = defaultConnect, notify = showWorkspaceToast,
timers = globalThis, pollMs = POLL_MS, isVisible = () => globalThis.document?.visibilityState !== 'hidden',
lifecycle = globalThis.window ?? null } = {}) {
let jobs = [];
let status = 'idle';
let attempt = 0;
// Failed list reads in a row: one (a proxy's 502, a dropped poll) is not yet a restart.
let failures = 0;
let retryTimer = null;
let pollTimer = null;
let reads = 0;
let disposed = false;
let suspended = false;
let reading = null;
const streams = new Map();
const listeners = new Set();
const emit = () => { for (const listener of [...listeners]) listener(store); };
const find = id => jobs.find(job => job.id === id);
function announce(previous, next) {
if (previous?.state === 'running' && FINAL_STATES.has(next.state)) notify(finishToast(next));
}
function closeStream(id) {
const source = streams.get(id);
if (!source) return;
streams.delete(id);
source.onmessage = null;
source.onerror = null;
source.close();
}
function follow() {
for (const id of [...streams.keys()]) if (find(id)?.state !== 'running') closeStream(id);
for (const job of jobs) if (job.state === 'running' && !streams.has(job.id)) openStream(job.id);
}
function openStream(id) {
const source = connect(`${JOBS_URL}/${id}/events`);
streams.set(id, source);
source.onmessage = event => {
let job;
try { job = parseJob(JSON.parse(event.data)); } catch { job = null; }
if (!job || job.id !== id) return;
attempt = 0;
upsert(job);
};
source.onerror = () => {
closeStream(id);
if (find(id)?.state === 'running') scheduleResync({ drop: true });
};
}
/** A job never runs again once it has ended, whatever an older read in flight still says. */
const settled = (previous, next) => FINAL_STATES.has(previous?.state) && next.state === 'running' ? previous : next;
function upsert(job) {
const previous = find(job.id);
job = settled(previous, job);
jobs = previous ? jobs.map(item => item.id === job.id ? job : item) : [job, ...jobs];
announce(previous, job);
follow();
emit();
}
/**
* Replace the jobs with the server's list. A running job it no longer has ended with the server and stays as
* ended; a finished job it no longer lists was pruned there and is dropped here too.
*/
function merge(list) {
const next = list.map(parseJob).filter(Boolean).map(job => settled(find(job.id), job));
const ids = new Set(next.map(job => job.id));
const carried = jobs.filter(job => !ids.has(job.id) && (job.state === 'running' || job.state === 'lost'))
.map(job => ({ ...job, state: 'lost' }));
for (const job of next) announce(find(job.id), job);
jobs = [...next, ...carried];
follow();
}
function markLost() {
jobs = jobs.map(job => job.state === 'running' ? { ...job, state: 'lost' } : job);
follow();
}
/** Re-read after the backoff; `drop` marks a re-read for a dropped stream, which keeps the backoff growing. */
function scheduleResync({ drop = false } = {}) {
if (disposed || suspended || retryTimer !== null) return;
const delay = backoffDelay(attempt);
attempt += 1;
retryTimer = timers.setTimeout(() => {
retryTimer = null;
load({ drop });
}, delay);
}
/** The next periodic read; a hidden tab skips its turn. */
function schedulePoll() {
if (disposed || suspended || pollMs <= 0) return;
if (pollTimer !== null) timers.clearTimeout(pollTimer);
pollTimer = timers.setTimeout(() => {
pollTimer = null;
if (isVisible()) load();
else schedulePoll();
}, pollMs);
}
/** Read this session's jobs. A failed read marks running jobs ended and tries again after the backoff. */
async function load({ drop = false } = {}) {
if (disposed || suspended) return;
const read = ++reads;
// Only the newest read applies, so an older one still in flight is aborted.
reading?.abort();
const controller = new AbortController();
reading = controller;
try {
const result = await api.get(JOBS_URL, { signal: controller.signal, cache: 'no-store' });
// An older read answering late is dropped.
if (disposed || suspended || read !== reads) return;
if (!Array.isArray(result?.jobs)) throw new TypeError('Invalid job list');
merge(result.jobs);
status = 'ready';
// A good read ends the backoff (the server is back, or nothing dropped), except for a stream that keeps
// dropping while the list reads fine: that waits longer each time, and a received message resets it.
if (failures > 0 || !drop) attempt = 0;
failures = 0;
schedulePoll();
} catch (error) {
if (disposed || suspended || scope.signal?.aborted || controller.signal.aborted || read !== reads) return;
failures += 1;
if (failures >= LOST_AFTER_FAILURES) markLost();
status = 'error';
scheduleResync();
}
emit();
}
const store = {
get status() { return status; },
jobs: () => [...jobs],
running: () => jobs.filter(job => job.state === 'running'),
find,
load,
subscribe(listener) {
listeners.add(listener);
return () => listeners.delete(listener);
},
/** Start a job of `kind`: {job}, or {busy, message} naming the running job that holds its slot. */
async start(kind, body = {}) {
try {
const result = await api.post(`${JOBS_URL}/${encodeURIComponent(kind)}`, body, { signal: scope.signal });
const job = parseJob(result?.job);
if (!job) throw new TypeError('Invalid job');
upsert(job);
notify(startToast(job));
return { job };
} catch (error) {
if (error?.status !== 409 || error?.code !== 'busy') throw error;
await load();
return { busy: store.running()[0] ?? null, message: error.message };
}
},
/** Ask a running job to stop at its next file boundary; false once it has ended. */
async cancel(id) {
const result = await api.post(`${JOBS_URL}/${encodeURIComponent(id)}/cancel`, {}, { signal: scope.signal });
return result?.cancellation_requested === true;
},
};
/** Stop every timer, stream and read in flight. */
function halt() {
if (retryTimer !== null) timers.clearTimeout(retryTimer);
if (pollTimer !== null) timers.clearTimeout(pollTimer);
retryTimer = null;
pollTimer = null;
reading?.abort();
reading = null;
for (const id of [...streams.keys()]) closeStream(id);
}
const onPageHide = () => { suspended = true; halt(); };
const onPageShow = event => {
if (!suspended || !event.persisted) return;
suspended = false;
load();
};
lifecycle?.addEventListener('pagehide', onPageHide);
lifecycle?.addEventListener('pageshow', onPageShow);
scope.cleanup(() => {
disposed = true;
halt();
lifecycle?.removeEventListener('pagehide', onPageHide);
lifecycle?.removeEventListener('pageshow', onPageShow);
listeners.clear();
});
return store;
}
+192
View File
@@ -0,0 +1,192 @@
/**
* Activity (#249 Activity-1): the concept's Activity page (app.js renderActivity 732–738), its top-bar button
* (`button#activity-open` with `span.count#activity-count`, index.html 51–52) and its sidebar row (app.js ~341), fed
* by the session job store (activity-store.mjs).
*
* - D52: both counts show the running jobs only and are hidden at 0; the heading has no "Unified activity · proposed"
* chip (prototype copy).
* - D53: while a job runs, the progress pill (`a.activity-pill`, the approved mockup's) sits before the button on every
* page: "Scanning · 42% · 8,240 / 19,622" and a mini bar, linking `#/activity/<id>`.
* - D56: each item's title links its job's own view (activity-job-view.mjs).
* - D54: each `.activity-item` is a real job. Its chip names the state (green "Completed", or blue "Running",
* "Cancelled", "Failed", "Ended"), its line says when and what, and a running job shows its `<progress>` and
* "Cancel job" (no "Simulate completion").
*/
import { createButton } from '../shared/toolkit/core.js';
import { navigateTo } from './state.mjs';
import { pageHeading } from './heading.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { countOf } from './plural.mjs';
import { pillParts, outcomeLine, addedOf, LOST_DETAIL } from './activity-store.mjs';
export const ACTIVITY_DESCRIPTION = 'Scans, tagging, downloads and maintenance share one history.';
export const EMPTY_ACTIVITY = 'No jobs have run since ComfyUI started. Scans and other long operations appear here.';
export const LOAD_FAILED = 'Activity could not be read. It is tried again shortly.';
export const CANCEL_FAILED_TOAST = 'The job could not be cancelled. Try again.';
const CHIPS = Object.freeze({
running: ['blue', 'Running'], finished: ['green', 'Completed'], cancelled: ['blue', 'Cancelled'],
failed: ['blue', 'Failed'], lost: ['blue', 'Ended'],
});
function element(tag, text, className) {
const node = document.createElement(tag);
if (text !== undefined) node.textContent = text;
if (className) node.className = className;
return node;
}
/** "09:41" in the browser's local time; an unknown time is empty. */
export function clockTime(seconds) {
if (typeof seconds !== 'number' || !Number.isFinite(seconds)) return '';
const date = new Date(seconds * 1000);
return `${String(date.getHours()).padStart(2, '0')}:${String(date.getMinutes()).padStart(2, '0')}`;
}
const join = parts => parts.filter(Boolean).join(' · ');
/** What a job covers: "2 output folders" for a scan. */
function scopeLine(job) {
const folders = job.scope?.output_folders;
return Number.isSafeInteger(folders) && folders >= 0 ? countOf(folders, 'output folder') : '';
}
/** The item's line under its title, by state. */
export function jobDetail(job) {
const at = clockTime(job.finished_at);
if (job.state === 'running') {
return join([`Started ${clockTime(job.started_at)}`, scopeLine(job), 'closing this view does not cancel it']);
}
if (job.state === 'lost') return LOST_DETAIL;
if (job.state === 'cancelled') {
return join([`Cancelled ${at} at a file boundary`, job.kind === 'scan' ? addedOf(job) : '',
'completed work is kept']);
}
if (job.state === 'failed') return join([`Stopped on an error at ${at}`, outcomeLine(job)]);
return join([`Finished ${at}`, outcomeLine(job)]);
}
/** The shell's Activity indicators: the top-bar count and pill, and the sidebar row's count. */
export function mountActivityTopbar({ store, scope, doc = document }) {
const button = doc.getElementById('activity-open');
const count = doc.getElementById('activity-count');
const pill = doc.getElementById('activity-pill');
scope.listen(button, 'click', () => navigateTo('#/activity'));
const label = element('span', '', 'activity-pill-label');
const files = element('span', '', 'activity-pill-files');
const bar = element('span', undefined, 'activity-pill-bar');
const fill = element('i');
bar.setAttribute('aria-hidden', 'true');
bar.append(fill);
pill.replaceChildren(label, files, bar);
const render = () => {
const running = store.running();
count.textContent = String(running.length);
count.hidden = running.length === 0;
const row = doc.querySelector('#system-nav a[data-route="activity"]');
row?.querySelector('.nav-count')?.remove();
if (row && running.length) row.append(element('span', running.length.toLocaleString(), 'nav-count'));
const [job] = running;
pill.hidden = !job;
if (!job) {
pill.removeAttribute('href');
pill.removeAttribute('aria-label');
return;
}
const parts = pillParts(job);
label.textContent = parts.label;
files.textContent = parts.files;
fill.style.width = `${job.progress}%`;
pill.href = `#/activity/${job.id}`;
pill.setAttribute('aria-label', parts.text);
};
scope.cleanup(store.subscribe(render));
render();
}
function itemFor(job, scope) {
const item = element('div', undefined, 'activity-item');
item.dataset.jobId = job.id;
const heading = element('div', undefined, 'panel-heading');
const title = element('h3');
// D56: the title opens the job's own view.
const link = element('a');
link.href = `#/activity/${job.id}`;
title.append(link);
const chip = element('span');
heading.append(title, chip);
const detail = element('p');
const progress = element('progress');
progress.max = 100;
const actions = element('div', undefined, 'row-actions');
const cancel = createButton('Cancel job');
actions.append(cancel);
item.append(heading, detail, progress, actions);
scope.listen(cancel, 'click', async () => {
if (cancel.getAttribute('aria-disabled') === 'true') return;
cancel.setAttribute('aria-disabled', 'true');
try {
await item.cancelJob();
} catch (error) {
if (scope.signal.aborted) return;
cancel.removeAttribute('aria-disabled');
showWorkspaceToast(CANCEL_FAILED_TOAST);
}
});
item.update = next => {
const [tone, state] = CHIPS[next.state];
link.textContent = `${next.state === 'finished' ? '✓' : '○'} ${next.title}`;
chip.className = `chip ${tone}`;
chip.textContent = state;
detail.textContent = jobDetail(next);
const running = next.state === 'running';
if (running) {
progress.value = next.progress;
progress.textContent = `${next.progress}%`;
progress.setAttribute('aria-label', `${next.title} progress`);
if (!progress.isConnected) item.insertBefore(progress, actions.isConnected ? actions : null);
if (!actions.isConnected) item.append(actions);
} else {
// A focused Cancel job that goes away hands its focus to the item's title link, never to the page body.
if (actions.contains(document.activeElement)) link.focus();
progress.remove();
actions.remove();
}
};
return item;
}
/** The Activity page: the concept's "Recent work" list of this session's jobs (one job's view is D56). */
export function mountActivityPage(main, { store, scope }) {
pageHeading(main).description.textContent = ACTIVITY_DESCRIPTION;
const section = element('section', undefined, 'surface');
const status = element('p', undefined, 'empty-state');
section.append(element('h2', 'Recent work'));
main.append(section);
const items = new Map();
const render = () => {
const jobs = store.jobs();
for (const [id, item] of items) if (!jobs.some(job => job.id === id)) { item.remove(); items.delete(id); }
jobs.forEach((job, index) => {
let item = items.get(job.id);
if (!item) {
item = itemFor(job, scope);
item.cancelJob = () => store.cancel(job.id);
items.set(job.id, item);
}
item.update(job);
// Only a node out of place moves, so a focused Cancel job keeps its focus across updates.
const at = section.children[index + 1];
if (at !== item) section.insertBefore(item, at ?? null);
});
const failed = store.status === 'error' && !jobs.length;
status.textContent = failed ? LOAD_FAILED : EMPTY_ACTIVITY;
if (failed) status.setAttribute('role', 'alert');
else status.removeAttribute('role');
if (jobs.length || store.status === 'idle') status.remove();
else if (!status.isConnected) section.append(status);
};
scope.cleanup(store.subscribe(render));
render();
// Opening Activity reads the list at once instead of waiting for the next periodic read.
store.load();
}
+2 -1
View File
@@ -3,8 +3,9 @@ const paths = Object.freeze({
"library": "M4 4h5v16H4z M11 4h4v16h-4z M17 5l3-1 3 15-3 1z",
"tags": "M3 3h8l10 10-8 8L3 11V3ZM7 7h.01",
"upload": "M12 16V3m-5 5 5-5 5 5M3 15v6h18v-6",
// D28 ruling: an outline home glyph of its own; the concept's `activity` pulse stays with the Activity row D5 omits.
// D28 ruling: an outline home glyph of its own; the concept's `activity` pulse belongs to the Activity row.
"overview": "M3 11 12 4l9 7M5 10v10h14V10M10 20v-6h4v6",
"activity": "M3 12h4l3-8 4 16 3-8h4",
"settings": "M12 8a4 4 0 1 0 0 8 4 4 0 0 0 0-8ZM12 2v3m0 14v3M2 12h3m14 0h3M5 5l2 2m10 10 2 2M5 19l2-2M17 7l2-2",
"grid": "M3 3h7v7H3z M14 3h7v7h-7z M3 14h7v7H3z M14 14h7v7h-7z",
"list": "M9 5h12M9 12h12M9 19h12M3 5h.01M3 12h.01M3 19h.01",
+13 -4
View File
@@ -12,6 +12,7 @@ import { noun } from './plural.mjs';
import { pageHeading } from './heading.mjs';
import { mountToolbar } from './toolbar.mjs';
import { openImportDialog, openRefineDialog } from './workspace-dialogs.mjs';
import { openMetadataScan } from './activity-scan.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { imagesFilters, imagesViewState } from './saved-views.mjs';
@@ -112,7 +113,7 @@ export function onFavoriteChange(listener) {
* Images last visit.
*/
export function mountImages(main, api, parentScope, {embedded = false, fetchPage, emptyText, focusTarget = null, savedState = null,
favorites = false, onSaveView = () => {}} = {}) {
favorites = false, onSaveView = () => {}, activity = null} = {}) {
const scope = createScope();
const dispose = () => scope.dispose();
parentScope.signal.addEventListener('abort', dispose, {once:true});
@@ -133,6 +134,8 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
let state, geometry = { positions: [], columns: 1, height: 0, maxHeight: 0 }, galleryWidth = 0, optionsRequest, categoryRequest, recovering = false, rescanning = false;
// Discovered sources (each with its own folders) and every folder, for "Refine this view" and the scope chip.
let roots = [], folders = [];
// The open "Refine this view", so a discovery that settles after it opened still lists its sources and folders.
let refine = null;
const rendered = new Map();
const itemIndexes = new Map();
const filters = { root: 'all', q: '', category: '', tags: [], subfolder: '', media: 'all', sort: 'newest', ...visit?.filters, ...saved,
@@ -163,7 +166,9 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
scope.cleanup(() => importButton.remove());
scope.listen(importButton, 'click', () => {
importDialog = openImportDialog({scope, trigger:importButton, rescanBusy:rescanning,
onInspect:() => { window.location.hash = '#/inspect'; }, onRescan:rescan});
onInspect:() => { window.location.hash = '#/inspect'; }, onRescan:rescan,
// D55: Scan for metadata reviews the session scan job over every discovered source.
onMetadataScan:activity ? trigger => openMetadataScan({scope, store:activity, sources:roots.length, trigger}) : null});
});
}
// The shared concept toolbar: search (filename, linked prompt text, title and tags), category and sort (D12–D14).
@@ -643,8 +648,8 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
/** "Refine this view": Source, Folder, Media and Tags, the filters the old field row held (D14). */
function openFilters(trigger) {
if (recovering) return;
openRefineDialog({scope, trigger, fields:[
{key:'source', options:[['all', 'All sources'], ...roots.map(root => [String(root.index), root.label])]},
refine = openRefineDialog({scope, trigger, fields:[
{key:'source', options:sourceOptions()},
{key:'folder', options:folderOptions('all'), optionsFor:folderOptions},
{key:'media', options:MEDIA_OPTIONS},
{key:'tags'},
@@ -654,6 +659,9 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
applyRefinement(cleared);
}});
}
function sourceOptions() {
return [['all', 'All sources'], ...roots.map(root => [String(root.index), root.label])];
}
/** The chosen Source's folders, as the old field row reloaded them; every folder under All sources. */
function folderOptions(source) {
const root = source === 'all' ? null : roots.find(item => String(item.index) === source);
@@ -750,6 +758,7 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
roots = data.roots.filter(root => Number.isSafeInteger(root?.index) && typeof root.label === 'string')
.map(root => ({index:root.index, label:root.label, subfolders:paths(root.subfolders)}));
folders = paths(data.subfolders);
refine?.setOptions('source', sourceOptions());
updateScope();
sourcesRetry = null;
return true;
+18 -4
View File
@@ -11,6 +11,9 @@ import { bindCanvasLink, canvasHref } from './topbar.mjs';
import { renderHeading, pageHeading } from './heading.mjs';
import { closeWorkspaceDialog } from './workspace-dialogs.mjs';
import { createSavedViews, openSaveViewDialog, openEditViewDialog, savedViewHash } from './saved-views.mjs';
import { createActivityStore } from './activity-store.mjs';
import { mountActivityPage, mountActivityTopbar } from './activity.mjs';
import { mountActivityJobView } from './activity-job-view.mjs';
const { createApiClient, resolveUrl } = window.ApiClient;
@@ -43,7 +46,7 @@ function mountWorkspace() {
const preference = document.getElementById('theme');
preference.value = theme.preference;
scope.listen(preference, 'change', () => theme.setPreference(preference.value));
// D6: ComfyUI ↗ opens the canvas; on Images with a resolved selection it sends that pair (D5: no Activity).
// D6: ComfyUI ↗ opens the canvas; on Images with a resolved selection it sends that pair (a send is not an Activity job).
let images = null;
scope.cleanup(bindCanvasLink(document.querySelector('.topbar .canvas-link'), {
href: canvasHref(resolveUrl, window.location.pathname),
@@ -60,8 +63,13 @@ function mountWorkspace() {
return link;
});
primaryNav.replaceChildren(...navLinks([['images', 'images'], ['library', 'library'], ['tags', 'tags'], ['inspect', 'upload']]));
// D28: System, then the production Overview hub, in the concept's tools-and-system group.
systemNav.replaceChildren(...navLinks([['system', 'settings'], ['overview', 'overview']]));
// The concept's tools-and-system group: Activity, System, then (D28) the production Overview hub.
systemNav.replaceChildren(...navLinks([['activity', 'activity'], ['system', 'settings'], ['overview', 'overview']]));
// Activity (#249 Activity-1): one store of this session's jobs feeds the top-bar button and pill, the sidebar
// row's count and the Activity page, on every route.
const activity = createActivityStore({ api, scope, connect: url => new EventSource(resolveUrl(url)) });
mountActivityTopbar({ store: activity, scope });
activity.load();
const sidebar = document.querySelector('aside.sidebar');
// The route key a nav link carries in `data-route`: a saved view's is `saved/<id>`.
const routeKey = route => route.id === 'saved' ? `saved/${route.savedId}` : route.id;
@@ -210,7 +218,7 @@ function mountWorkspace() {
if (route.id === 'images' || route.id === 'favorites') {
mountNewPromptAction(main, api, view);
images = mountImages(main, api, view, {savedState:saved?.state, favorites:route.id === 'favorites',
onSaveView:saveView('images', view)});
onSaveView:saveView('images', view), activity});
return;
}
if (route.id === 'library') {
@@ -221,6 +229,12 @@ function mountWorkspace() {
mountVocabulary(main, api, view);
return;
}
if (route.id === 'activity') {
// D56: `#/activity/<id>` is that job's own view; `#/activity` lists the session's jobs.
if (route.jobId) mountActivityJobView(main, { store: activity, scope: view, api, jobId: route.jobId });
else mountActivityPage(main, { store: activity, scope: view });
return;
}
if (isHandoff(route)) {
description.textContent = route.description;
const panel = element('section', undefined, 'surface handoff');
+8 -2
View File
@@ -11,11 +11,13 @@ export const routes = [
{ id: 'overview', label: 'Overview' },
// SAVED VIEWS' first row (#249 D51): Images filtered to favorites. No href, so Overview lists no card for it.
{ id: 'favorites', label: 'Favorites' },
// The concept's Activity (app.js renderActivity 732–738): this server session's jobs. No href: no Overview card.
{ id: 'activity', label: 'Activity' },
{ id: 'system', label: 'System', href: '/prompt_manager/admin',
description: 'Open the current dashboard, then choose Settings or Maintenance for configuration, scans, backups and diagnostics.', action: 'Open existing administration' },
];
const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'overview', 'saved', 'favorites']);
const NATIVE_ROUTES = new Set(['images', 'library', 'tags', 'overview', 'saved', 'favorites', 'activity']);
/** A saved view (#249 D16): the shell resolves its id to the view's page, name and state. */
const SAVED_ROUTE = Object.freeze({ id: 'saved', label: 'Saved view' });
@@ -33,6 +35,9 @@ export function isHandoff(route) {
export function resolveRoute(hash) {
const saved = /^#\/saved\/([1-9]\d*)$/.exec(hash);
if (saved && Number.isSafeInteger(Number(saved[1]))) return {...SAVED_ROUTE, savedId:Number(saved[1]), hash};
// One job of the Activity page (its id is the registry's uuid4 hex).
const job = /^#\/activity\/([0-9a-f]{32})$/.exec(hash);
if (job) return {...routes.find(route => route.id === 'activity'), jobId:job[1], hash};
const match = /^#\/library\/([1-9]\d*)$/.exec(hash);
if (match && Number.isSafeInteger(Number(match[1]))) {
return {...routes.find(route => route.id === 'library'), positiveId:Number(match[1]), hash};
@@ -110,7 +115,8 @@ export function mountRouter(win, onRoute) {
const route = resolveRoute(win.location.hash);
const hash = route.hash || `#/${route.id}`;
if (win.location.hash !== hash) win.history.replaceState(win.history.state, '', hash);
if (current?.id === route.id && current?.positiveId === route.positiveId && current?.savedId === route.savedId) return;
if (current?.id === route.id && current?.positiveId === route.positiveId && current?.savedId === route.savedId &&
current?.jobId === route.jobId) return;
current = route;
onRoute(route);
};
+53 -5
View File
@@ -1,12 +1,60 @@
/** The shell's shared toast (`#workspace-toast`, role="status"), used by every workspace page. */
const TOAST_MS = 6500;
export const TOAST_MS = 6500;
/** A toast with a link stays longer, so a keyboard user can reach it (D57, WCAG 2.2.1). */
export const LINK_TOAST_MS = 20000;
let toastTimer;
// Stops waiting for the pointer or focus to leave a toast that a newer one replaced.
let releaseWait = null;
/** Show one outcome in the shared workspace toast; it clears itself like the concept's. */
/** Whether the pointer or keyboard focus rests on the toast. */
const held = toast => typeof toast.matches === 'function' && toast.matches(':hover, :focus-within');
/** Clear the toast when its time is up, unless the pointer or focus is on it: then wait until they leave. */
function scheduleClear(toast, delay) {
clearTimeout(toastTimer);
toastTimer = setTimeout(() => {
if (held(toast)) {
const release = () => {
toast.removeEventListener('pointerleave', leave);
toast.removeEventListener('focusout', later);
releaseWait = null;
};
const leave = () => {
if (held(toast)) return;
release();
scheduleClear(toast, TOAST_MS);
};
// focusout fires before the next element takes focus, so it checks once focus has moved.
const later = () => setTimeout(leave);
toast.addEventListener('pointerleave', leave);
toast.addEventListener('focusout', later);
releaseWait = release;
return;
}
toast.textContent = '';
}, delay);
}
/**
* Show one outcome in the shared workspace toast; it clears itself like the concept's. A message given as parts
* (`['Another job is running: ', {text, href}, '.']`) renders each `{text, href}` as a link (D57); such a toast stays
* for LINK_TOAST_MS and while the pointer or focus is on it.
*/
export function showWorkspaceToast(message) {
const toast = document.getElementById('workspace-toast');
if (!toast) return;
toast.textContent = message;
clearTimeout(toastTimer);
toastTimer = setTimeout(() => {toast.textContent = '';}, TOAST_MS);
releaseWait?.();
if (Array.isArray(message)) {
toast.replaceChildren(...message.map(part => {
if (typeof part === 'string') return document.createTextNode(part);
const link = document.createElement('a');
link.href = part.href;
link.textContent = part.text;
return link;
}));
} else {
toast.textContent = message;
}
const linked = Array.isArray(message) && message.some(part => typeof part !== 'string');
scheduleClear(toast, linked ? LINK_TOAST_MS : TOAST_MS);
}
+111 -9
View File
@@ -8,7 +8,10 @@
* mock-status note because every field is a real filter, and no "Prompt evidence" field because evidence is only
* known per file (D14). Clear also clears the toolbar category, as the concept's does; the page applies it.
* - "Import & inspect" (a1483–1491): only the real operations, Inspect a local image and Scan an output folder →
* Rescan; no "Import portable recipes" (D15). Rescan shows its running state (aria-disabled "Rescanning…").
* Rescan; no "Import portable recipes" (D15). Rescan shows its running state (aria-disabled "Rescanning…"). D55: the
* concept's own "Scan an output folder → Review scan" row follows as "Scan for metadata".
* - "Review metadata scan" (a1428–1432 `scan-review`, D55): Sources and Scope in a `dl.meta-grid`, the concept's
* sentence, a backup `.notice` ("Back up now") and Cancel / Start scan; no sample-plan note.
* - "Copy prompt" (a1223–1241): the select-to-copy fallback when the clipboard is unavailable (D22).
* - One-value input (a1138 `simpleInput`): a "Category"-style field (`input#action-input`) and Cancel / a primary
* action; used by the Images selection bar's "Set category" (#249 D8) and the toolbar's Save view.
@@ -17,7 +20,7 @@
*
* Every opener takes the page `scope`: disposing it closes and removes the shell, so a dialog never outlives its page.
*/
import { createButton } from '../shared/toolkit/core.js';
import { createButton, createSwitch } from '../shared/toolkit/core.js';
const FIELD_LABELS = Object.freeze({ source: 'Source', folder: 'Folder', media: 'Media', tags: 'Tags' });
const FIELD_ORDER = Object.keys(FIELD_LABELS);
@@ -152,6 +155,9 @@ function choose(select, value) {
* Tags; Folder may add `optionsFor(source)`, the options of one Source, so its list follows the chosen Source and a
* folder that Source lacks returns to the first option. `values` holds the current value per key. Apply reports the edited values; Clear resets every field and
* reports the cleared values plus `category: ''`, so the page clears its toolbar category too. Both close first.
* Returns `{setOptions(key, options)}` for options discovered after the dialog opened (a slow folder discovery): the
* select is refilled and shows the value the field opened with, or the one chosen since, once it is offered; a Folder
* that follows its Source is refilled from `optionsFor` again. It does nothing once this dialog closed or was replaced.
*/
export function openRefineDialog({ scope, fields, values = {}, onApply, onClear, trigger = null }) {
requireScope(scope);
@@ -172,16 +178,21 @@ export function openRefineDialog({ scope, fields, values = {}, onApply, onClear,
choose(select, values[key]);
return [{ key, control: select, cleared: item.options[0][0], optionsFor: item.optionsFor }];
});
// Each select's wanted value: the one it opened with until the person picks another (see setOptions).
const wanted = new Map(controls.map(({ key }) => [key, values[key]]));
for (const { key, control } of controls) {
if (key !== 'tags') control.addEventListener('change', () => wanted.set(key, control.value));
}
const source = controls.find(item => item.key === 'source');
const folder = controls.find(item => item.key === 'folder' && typeof item.optionsFor === 'function');
const followSource = () => {
const chosen = folder.control.value;
const followSource = (chosen = folder.control.value) => {
fillSelect(folder.control, 'folder', folder.optionsFor(source.control.value));
choose(folder.control, chosen);
};
if (source && folder) {
followSource();
source.control.addEventListener('change', followSource);
followSource(values.folder);
// A Source that lacks the chosen folder clears it, and that cleared folder is the wanted one from then on.
source.control.addEventListener('change', () => { followSource(); wanted.set('folder', folder.control.value); });
}
const read = () => Object.fromEntries(controls.map(({ key, control }) =>
[key, key === 'tags' ? control.value.trim() : control.value]));
@@ -207,6 +218,15 @@ export function openRefineDialog({ scope, fields, values = {}, onApply, onClear,
...controls.map(({ key, control }) => field(FIELD_LABELS[key], control, `filter-${key}`)),
actions,
], { scope, trigger });
const opened = session;
function setOptions(key, options) {
const item = controls.find(candidate => candidate.key === key);
if (session !== opened || !item || key === 'tags') return;
fillSelect(item.control, key, options);
choose(item.control, wanted.get(key));
if (source && folder && (item === source || item === folder)) followSource(wanted.get('folder'));
}
return { setOptions };
}
/** One operation row; a press on its busy (aria-disabled) action is ignored and keeps the dialog open. */
@@ -230,16 +250,23 @@ const RESCAN_LABEL = 'Rescan';
const RESCAN_BUSY_LABEL = 'Rescanning…';
/**
* "Import & inspect" with the two real operations (D15). While `rescanBusy`, Rescan reads "Rescanning…" and is
* "Import & inspect" with the real operations (D15). While `rescanBusy`, Rescan reads "Rescanning…" and is
* aria-disabled, so a second press is visibly ignored. Returns `{setRescanBusy}` for the page to end that state.
* With `onMetadataScan`, the concept's "Review scan" row follows as "Scan for metadata" (D55).
*/
export function openImportDialog({ scope, onInspect, onRescan, trigger = null, rescanBusy = false }) {
export function openImportDialog({ scope, onInspect, onRescan, onMetadataScan = null, trigger = null, rescanBusy = false }) {
requireScope(scope);
requireCallbacks({ onInspect, onRescan });
requireCallbacks({ onInspect, onRescan, ...(onMetadataScan ? { onMetadataScan } : {}) });
const list = el('div', undefined, 'operation-list');
const inspect = operation('Inspect a local image', 'Read embedded metadata before deciding to add it.', 'Open inspector', onInspect);
const rescan = operation('Scan an output folder', 'Find files added or removed in the output folders.', RESCAN_LABEL, onRescan);
list.append(inspect.row, rescan.row);
if (onMetadataScan) {
// The opened review dialog replaces this one in the shell, so its Cancel returns focus to Import.
const review = operation('Scan for metadata', 'Discover and enrich existing files using the configured sources.',
'Review scan', () => onMetadataScan(trigger));
list.append(review.row);
}
function setRescanBusy(busy) {
rescan.action.textContent = busy ? RESCAN_BUSY_LABEL : RESCAN_LABEL;
if (busy) rescan.action.setAttribute('aria-disabled', 'true');
@@ -309,3 +336,78 @@ export function openContentDialog({ scope, title, nodes, onClose = null, trigger
if (onClose !== null && typeof onClose !== 'function') throw new TypeError('The dialog onClose must be a function');
show(title, nodes, { scope, trigger, onClose });
}
export const SCAN_REVIEW_SENTENCE = 'Enrich positive / negative associations, reuse canonical text and report unchanged files. ' +
'Do not inflate run counts or recreate existing thumbnails.';
const BACKUP_LABEL = 'Back up now';
const BACKUP_BUSY_LABEL = 'Backing up…';
export const SCAN_SCOPE_LABELS = { new: 'New and unresolved files', all: 'All files' };
export const SCAN_ALL_FILES_HINT = 'Also read the files already linked to their prompt again.';
/**
* The concept's "Review metadata scan" (`scan-review`, D55). `sources` is the number of output folders the scan reads.
* The scope is the concept's "New and unresolved files"; the "All files" checkbox (D58, the concept's `.setting-row`
* switch, unchecked) switches the Scope value to "All files". `onBackup` runs the database backup (the button is
* aria-disabled and reads "Backing up…" until it settles, and the dialog stays open); Start scan closes the dialog, then
* calls `onStart({scope: 'new' | 'all'})`.
*/
export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigger = null }) {
requireScope(scope);
requireCallbacks({ onBackup, onStart });
if (!Number.isSafeInteger(sources) || sources < 0) throw new TypeError('The scan review needs its source count');
const grid = el('dl', undefined, 'meta-grid');
const values = {};
for (const [term, value] of [['Sources', `${sources.toLocaleString('en-US')} output ${sources === 1 ? 'folder' : 'folders'}`],
['Scope', SCAN_SCOPE_LABELS.new]]) {
const entry = el('div');
values[term] = el('dd', value);
entry.append(el('dt', term), values[term]);
grid.append(entry);
}
// The concept's System toggle: a `.setting-row` whose checkbox is the Toolkit switch (concept `hydrateSwitches`).
const toggle = createSwitch('', '', 'scan-all-files');
const allFiles = toggle.querySelector('input');
allFiles.checked = false;
const option = el('div', undefined, 'setting-row');
const optionText = el('div');
const optionLabel = el('label', SCAN_SCOPE_LABELS.all);
optionLabel.htmlFor = allFiles.id;
const hint = el('p', SCAN_ALL_FILES_HINT);
hint.id = 'scan-all-files-hint';
allFiles.setAttribute('aria-describedby', hint.id);
optionText.append(optionLabel, hint);
option.append(optionText, toggle);
const chosenScope = () => (allFiles.checked ? 'all' : 'new');
allFiles.addEventListener('change', () => { values.Scope.textContent = SCAN_SCOPE_LABELS[chosenScope()]; });
const notice = el('div', undefined, 'notice');
const copy = el('div');
copy.append(el('h3', 'Back up the database first (recommended)'),
el('p', 'Downloads a copy of the prompts database before the scan changes any records.'));
const backupActions = el('div', undefined, 'row-actions');
const backup = createButton(BACKUP_LABEL);
backupActions.append(backup);
notice.append(copy, backupActions);
backup.addEventListener('click', async () => {
if (backup.getAttribute('aria-disabled') === 'true') return;
backup.setAttribute('aria-disabled', 'true');
backup.textContent = BACKUP_BUSY_LABEL;
try {
await onBackup();
} finally {
backup.removeAttribute('aria-disabled');
backup.textContent = BACKUP_LABEL;
}
});
const cancel = createButton('Cancel');
const start = createButton('Start scan', 'ca-primary');
cancel.addEventListener('click', () => finish());
start.addEventListener('click', () => {
const files = chosenScope();
finish();
onStart({ scope: files });
});
const actions = el('div', undefined, 'dialog-actions');
actions.append(cancel, start);
show('Review metadata scan', [grid, option, el('p', SCAN_REVIEW_SENTENCE, 'small'), notice, actions], { scope, trigger });
}
+52
View File
@@ -17,6 +17,58 @@ main:focus-visible { outline: 2px solid var(--ca-focus); outline-offset: 4px; }
nav a[aria-current="page"] { border: 2px solid Highlight; color: LinkText; }
}
/* D53: the running job's progress pill before the Activity button (the approved Activity mockup's `.pill`), in the
concept's tokens. At <=700px it keeps its verb and percentage; the file counts and bar stay in its accessible name. */
.activity-pill:not([hidden]) { display: inline-flex; align-items: center; gap: 8px; padding: 6px 12px; border: 1px solid var(--line);
border-radius: 999px; font-size: 12px; white-space: nowrap; color: var(--ca-text); background: var(--ca-surface); /* D53 */ }
.activity-pill:hover { border-color: var(--ca-accent); /* D53 */ }
.activity-pill-bar { display: block; inline-size: 70px; block-size: 5px; border-radius: 3px; background: var(--line); overflow: hidden; /* D53 */ }
.activity-pill-bar i { display: block; block-size: 100%; inline-size: 0; background: var(--ca-accent); /* D53 */ }
@media (pointer: coarse) { .activity-pill:not([hidden]) { min-block-size: 40px; /* D53 */ } }
@media (max-width: 700px) {
/* D52: with the Activity button the full-width row can be wider than a zoomed phone, so its items wrap. */
.top-actions { flex-wrap: wrap; /* D52 */ }
.activity-pill:not([hidden]) { gap: 0; padding-inline: 10px; /* D53 */ }
.activity-pill-files, .activity-pill-bar { display: none; /* D53 */ }
}
/* D56: each Activity item's title links its job's view in the title's own colour. */
.activity-item h3 a { color: inherit; text-decoration: none; /* D56 */ }
.activity-item h3 a:hover { text-decoration: underline; /* D56 */ }
/* D56: one job's view, the approved Activity mockup (activity-scan.html) in the concept's tokens: the counter tiles,
the outcome list beside the selected outcome's files, and the collapsible log. */
.stat-row { display: grid; grid-template-columns: repeat(5, minmax(0, 1fr)); gap: 10px; margin: 16px 0; /* D56 */ }
.stat-row .stat { border: 1px solid var(--line); border-radius: 7px; padding: 12px 14px; background: var(--ca-surface); min-inline-size: 0; /* D56 */ }
.stat-row .stat dt { font-size: 11px; color: var(--ca-muted); /* D56 */ }
.stat-row .stat dd { margin: 6px 0 0; font-size: 20px; font-weight: 600; /* D56 */ }
.stat-row .stat.added dd { color: var(--green); /* D56 */ }
.stat-row .stat.warn dd { color: var(--warning); /* D56 */ }
.stat-row .stat.bad dd { color: var(--red); /* D56 */ }
.surface > .panel-heading > .search-field { max-width: 320px; /* D56 */ }
.outcomes { display: grid; grid-template-columns: 220px minmax(0, 1fr); gap: 16px; margin-top: 6px; /* D56 */ }
.ca-tools .outcome-list button { display: flex; justify-content: space-between; width: 100%; margin-bottom: 6px; text-align: left; /* D56 */ }
.ca-tools .outcome-list button[aria-pressed="true"] { border-color: var(--ca-accent); background: var(--soft-blue); /* D56 */ }
.outcome-files .file-row { display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 12px; padding: 9px 4px;
border-bottom: 1px solid var(--line); font-size: 12px; /* D56 */ }
.ca-tools button.file-row { width: 100%; min-height: 0; justify-content: normal; align-items: start; border: 0;
border-bottom: 1px solid var(--line); border-radius: 0; background: transparent; color: inherit; text-align: left;
white-space: normal; font: inherit; font-size: 12px; /* D56 */ }
.ca-tools button.file-row:hover:not(:disabled) { background: var(--ca-hover); /* D56 */ }
.file-row .path { font-family: ui-monospace, monospace; overflow-wrap: anywhere; /* D56 */ }
.file-row .why { color: var(--ca-muted); /* D56 */ }
.outcome-files > p.small { color: var(--ca-muted); margin-top: 10px; /* D56 */ }
.term { margin-top: 16px; border: 1px solid var(--line); border-radius: 7px; overflow: hidden; /* D56 */ }
.term summary { padding: 11px 14px; cursor: pointer; font-size: 12px; background: var(--ca-surface); /* D56 */ }
.term summary span { color: var(--ca-muted); /* D56 */ }
.term pre { margin: 0; padding: 12px 14px; max-height: 180px; overflow: auto; font: 11px/1.7 ui-monospace, monospace;
color: var(--ca-muted); background: var(--page); white-space: pre-wrap; overflow-wrap: anywhere; /* D56 */ }
@media (max-width: 700px) {
.stat-row { grid-template-columns: repeat(2, minmax(0, 1fr)); /* D56 */ }
.outcomes { grid-template-columns: minmax(0, 1fr); /* D56 */ }
.outcome-files .file-row { grid-template-columns: minmax(0, 1fr); gap: 4px; /* D56 */ }
}
/* Overview: real library statistics and the legacy tool cards inside a concept `.surface`. */
.stat-grid { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 16px; margin: 0 0 26px; }
.stat { padding: 22px; background: var(--ca-surface); border: 1px solid var(--line); border-radius: 7px; }
+1 -1
View File
@@ -32,7 +32,7 @@
<div class="workspace">
<header class="topbar">
<div class="breadcrumbs">ComfyAssets <span>/</span> <span id="workspace-breadcrumb">Images</span></div>
<div class="top-actions"><label class="theme-label" for="theme">Theme
<div class="top-actions"><!-- D53: the progress pill, shown while a job runs. D52: the count shows running jobs, hidden at 0. --><a id="activity-pill" class="activity-pill" hidden></a><button id="activity-open" type="button">Activity <span class="count" id="activity-count" hidden>0</span></button><label class="theme-label" for="theme">Theme
<select id="theme" aria-label="Theme">
<option value="dark">Dark</option>
<option value="light">Light</option>