diff --git a/TESTING.md b/TESTING.md
index 8069def..21aaa78 100644
--- a/TESTING.md
+++ b/TESTING.md
@@ -550,8 +550,9 @@ Recorded shell deviations:
configured sources.", button "Review scan"). It opens the concept's `scan-review`,
"Review metadata scan": no sample-plan `.demo-note` (prototype copy); Sources and
Scope in a `dl.meta-grid` (the concept's maintenance plan uses `dl`; `scan-review`
- uses a `div`) reading "N output folders" (the discovered sources) and "All files"
- (fixed in Activity-1; Activity-2 adds the scope option); the concept's sentence; then
+ uses a `div`) reading "N output folders" (the discovered sources) and the concept's
+ "New and unresolved files" (the D58 checkbox switches it to "All files"); the
+ concept's sentence; then
a backup `.notice` (the concept's System notice box): `h3` "Back up the database first
(recommended)", "Downloads a copy of the prompts database before the scan changes any
records." and "Back up now", which downloads the existing `/prompt_manager/backup`
@@ -624,6 +625,63 @@ Recorded shell deviations:
is running:
." with the title linking that job's view (the shared toast now
accepts link parts). A toast with a link stays 20 s instead of 6.5 s, and any toast
stays while the pointer or focus is on it.
+- **D58** (scan scope, Activity-2 Task 4): the concept's `scan-review` states a fixed
+ scope, "New and unresolved files". Production keeps that value as the default and adds
+ the concept's own System toggle after the `dl.meta-grid`: a `.setting-row` with the
+ label "All files", the hint "Also read the files already linked to their prompt
+ again." (the switch's `aria-describedby`) and the Toolkit switch (`label.ca-switch`, a
+ checkbox with `role="switch"`, as the concept's `hydrateSwitches` renders it),
+ unchecked. Checking it switches the Scope value to "All files"; Start scan sends
+ `POST /prompt_manager/jobs/scan {scope: "new" | "all"}` (no body, an empty body or no
+ `scope` is "new"; another value is a 400).
+ - In "new" the scan skips, before reading any metadata, the files the Library already
+ links with a resolved link and that are unchanged on disk: exactly one
+ `generated_images` row for the file, exactly one distinct pair link on it
+ (`image_prompt_pairs`, written only by resolved ingestion), and a stored
+ `file_size` equal to the file's size in this walk. Both paths go through
+ `normalize_image_path` (absolute, normalized, `os.path.normcase`); a row without
+ `file_path` matches by its `image_path`.
+ - Read again: files linked without a pair (the monitor's fallback and FIFO links, the
+ legacy positive-only import), linked to more than one prompt or pair, not linked, a
+ file whose size changed (one replaced at the same path, e.g. a reused ComfyUI
+ counter name, is read and its conflict lands under Needs review), and a row with no
+ stored size (unknown is not known). The scan now stores each file's size when it
+ ingests it, so a file read once is skipped by the next "new" pass; the first pass
+ after the upgrade fills the missing sizes, which reports those files as enriched
+ (Added) instead of unchanged.
+ - The known set is one streamed query that keeps only the scanned paths. If it cannot
+ be read, the scan logs "Known files could not be read · reading every file" and
+ reads every file.
+ - A skipped file counts as processed and unchanged in the legacy counts (not as
+ found) and under Already known, with the detail "Skipped · already linked" and the
+ log line "= / · Skipped · already linked"; the log's first line names
+ the scope ("Scan started · N output folders · new and unresolved files" or "· all
+ files"). A legacy-page scan that attaches to a running "new" job therefore shows
+ the skipped files in its `unchanged` count.
+ - "All files" reads every file, as the scan did in Activity-1, and the legacy
+ `/prompt_manager/scan` route always does when it starts the scan (its SSE payload
+ keys are unchanged).
+ - `activity-concept.spec.mjs` compares the switch with the concept's System → Sources
+ toggle ("Watch for new outputs"), without its place-dependent top margin;
+ `workspace-activity.spec.mjs` covers the Scope value, the description and the
+ request both ways; `tests/test_scan_scope.py` covers the skip, the size rule, the
+ replaced file, the re-read of unresolved files, "all", the legacy route, the
+ fallback, the empty body, the 400 and Windows case folding.
+- **D59** (job report and log, Activity-2 Task 5): the job view's "Download report" is
+ enabled. It fetches `GET /prompt_manager/jobs//report` through the api-client URL
+ (so a proxy mount works) and saves a JSON attachment named
+ `prompt-manager--.json` (server local time), then toasts "Job
+ report downloaded." (or "The report could not be downloaded. Try again."). The report
+ holds the summary, every counter, each outcome group's exact `count`, its listed
+ `rows` (at most 200) and a `truncated` flag, the scope and the log; rows and log lines
+ carry root-relative paths only, never an absolute path (an unknown id is a 404).
+ A job lost with a restart has no report on the server, so its Download report is
+ disabled; a view left while the report downloads still saves it but shows no toast.
+ The scan log opens at its newest line every time it is opened and follows new lines
+ only while the reader is at the bottom; a reader who scrolled up keeps the position
+ while it stays open (checked on Chromium, Firefox and WebKit).
+ `test_report_is_attachment_without_absolute_paths` and the browser tests `download
+ report saves the job JSON` and `log keeps the reader's scroll position` cover it.
- **D29**: no prototype copy. There is no "Interactive concept" badge and no
"What changes & why" button. The footer reads "ComfyAssets · Local tools. Your
library.", and the title has no " concept" suffix. Save view's dialog action reads
diff --git a/database/operations.py b/database/operations.py
index 2584498..26b28d1 100644
--- a/database/operations.py
+++ b/database/operations.py
@@ -181,6 +181,21 @@ AND p.id NOT IN (SELECT prompt_id FROM live_hint_prompts)
"""
+# The scan's known set (#249 Activity-2): every image row, with its stored file
+# size and whether its pair link is resolved: exactly one distinct pair, the rule
+# _LINK_SUMMARY_SQL and get_image_prompt_info use. A file is known and resolved
+# when it has exactly one image row, that row is resolved, and the row's stored
+# size equals the file's size now; several rows (one file linked to more than
+# one prompt) are ambiguous, and an unknown (NULL) stored size is not known. The
+# rows are streamed in fetches of KNOWN_PATHS_FETCH_SIZE and only the scanned
+# paths are kept, so memory follows the scan (at most MAX_SCAN_FILES paths), not
+# the table.
+_KNOWN_PATHS_SQL = """SELECT gi.file_path, gi.image_path, gi.file_size,
+ (SELECT COUNT(DISTINCT ip.pair_id) FROM image_prompt_pairs ip
+ WHERE ip.image_id = gi.id) = 1
+FROM generated_images gi"""
+KNOWN_PATHS_FETCH_SIZE = 1000
+
# One SELECT reads the link summaries of at most this many paths: three bound
# parameters per path stays under SQLite's historical 999-variable limit.
LINK_SUMMARY_BATCH = 200
@@ -2280,6 +2295,52 @@ class PromptDatabase(
"parameters": row[8] if len(row) > 8 else None,
}
+ def get_known_resolved_paths(self, files: Dict[str, Optional[int]]) -> Set[str]:
+ """The paths in ``files`` the Library already links with a resolved link.
+
+ ``files`` maps each scanned path to its size on disk (None when it could
+ not be read). A path qualifies when exactly one ``generated_images`` row
+ claims the file, that row has exactly one pair link
+ (``image_prompt_pairs``, written only by resolved ingestion), and its
+ stored ``file_size`` is not NULL and equals the given size, so a file
+ replaced at the same path is read again. Files linked without a pair
+ (the monitor's fallback and FIFO links, the legacy positive-only
+ import), linked to several prompts or pairs, or not linked at all do
+ not qualify. Both sides go through ``normalize_image_path`` (absolute,
+ normalized, ``os.path.normcase``), so a Windows row stored in another
+ case or spelling still matches. One query, streamed; returns the
+ matching keys of ``files``.
+ """
+ wanted: Dict[str, List[Tuple[str, Optional[int]]]] = {}
+ for path, size in files.items():
+ key = normalize_image_path(str(path))
+ wanted.setdefault(key, []).append((str(path), size))
+ if not wanted:
+ return set()
+ rows_seen: Dict[str, int] = {}
+ stored_sizes: Dict[str, int] = {}
+ with self.model.get_connection() as conn:
+ cursor = conn.execute(_KNOWN_PATHS_SQL)
+ while True:
+ rows = cursor.fetchmany(KNOWN_PATHS_FETCH_SIZE)
+ if not rows:
+ break
+ for file_path, image_path, file_size, is_resolved in rows:
+ # image_path is NOT NULL; file_path is NULL only on old rows.
+ key = normalize_image_path(file_path or image_path)
+ if key not in wanted:
+ continue
+ rows_seen[key] = rows_seen.get(key, 0) + 1
+ if is_resolved and file_size is not None:
+ stored_sizes[key] = file_size
+ return {
+ path
+ for key, stored_size in stored_sizes.items()
+ if rows_seen[key] == 1
+ for path, size in wanted[key]
+ if size is not None and size == stored_size
+ }
+
def get_image_link_summaries(
self, paths: List[str], allow_filename_fallback: bool = True
) -> Dict[str, Dict[str, Any]]:
diff --git a/py/api/admin.py b/py/api/admin.py
index 8519cd1..bc900ee 100644
--- a/py/api/admin.py
+++ b/py/api/admin.py
@@ -17,7 +17,14 @@ from pathlib import Path
from aiohttp import web
from .prompts import thumbnail_url_for
-from .jobs import SCAN_GROUP_NAMES, SCAN_GROUPS, Job, JobBusy
+from .jobs import (
+ DEFAULT_SCAN_SCOPE,
+ SCAN_GROUP_NAMES,
+ SCAN_GROUPS,
+ SCAN_SCOPES,
+ Job,
+ JobBusy,
+)
try:
from ...utils.parallel import map_parallel
@@ -150,6 +157,8 @@ SCAN_FINAL_STATES = {
"cancelled": "cancelled",
"error": "failed",
}
+# The scope as the scan log's first line names it (the review dialog's words).
+SCAN_SCOPE_LOG = {"new": "new and unresolved files", "all": "all files"}
MIN_BATCHES_FOR_ESTIMATE = 2 # "about X min left" waits for this many batches
# Log line prefix per outcome group (spec: + added, = known, ? review, ! error,
# - no usable metadata).
@@ -174,6 +183,7 @@ SCAN_DETAILS = {
"linked": "Linked to an existing prompt",
"enriched": "Metadata added to its prompt",
"unchanged": "Already linked to its prompt",
+ "skipped": "Skipped · already linked",
"ambiguous": "Needs review: more than one prompt could match",
"unresolved": "Needs review: the prompt could not be traced in the workflow",
"conflict": "Needs review: already linked to a different prompt",
@@ -197,6 +207,30 @@ def _scan_status_line(done, total, batches, elapsed):
return f"{line} · about {minutes} min left"
+def _file_size(path):
+ """``path``'s size on disk, or None when it cannot be read."""
+ try:
+ return os.path.getsize(path)
+ except OSError:
+ return None
+
+
+def _with_file_size(canonical, media_file):
+ """Record the file's size in ``file_info`` unless the metadata carries one.
+
+ Stored as ``generated_images.file_size``; the "new" scan scope skips a known
+ file only while its size is unchanged.
+ """
+ info = canonical.get("file_info")
+ info = dict(info) if isinstance(info, dict) else {}
+ if info.get("size") is None:
+ size = _file_size(media_file)
+ if size is not None:
+ info["size"] = size
+ if info:
+ canonical["file_info"] = info
+
+
def _scan_roots(output_dirs):
"""(absolute root, public label, listing index), the most specific first.
@@ -1426,7 +1460,7 @@ class AdminRoutesMixin:
job = self._jobs.active(SCAN_JOB_KIND)
if job is None:
try:
- job = self._start_scan_job()
+ job = self._start_scan_job("all")
except JobBusy as busy:
return self._long_job_busy_response(busy.running.title)
return await self._stream_scan_job(request, job)
@@ -1450,16 +1484,19 @@ class AdminRoutesMixin:
}
)
- def _start_scan_job(self):
+ def _start_scan_job(self, files=DEFAULT_SCAN_SCOPE):
"""Register the scan job synchronously; its runner starts on the loop.
- Raises JobBusy while another record-changing job runs.
+ ``files`` is the scope (``SCAN_SCOPES``). Raises JobBusy while another
+ record-changing job runs.
"""
+ if files not in SCAN_SCOPES:
+ raise ValueError(f"Unknown scan scope: {files!r}")
job = self._jobs.start(
SCAN_JOB_KIND,
SCAN_JOB_TITLE,
record_changing=True,
- scope={"mode": "all"},
+ scope={"files": files},
runner=self._run_scan_job,
groups=SCAN_GROUP_NAMES,
)
@@ -1525,7 +1562,8 @@ class AdminRoutesMixin:
dir_names = [self._public_path(d) for d in output_dirs]
folders = _plural(len(output_dirs), "output folder")
job.scope["output_folders"] = len(output_dirs)
- job.log_line(f"Scan started · {folders}")
+ scope = SCAN_SCOPE_LOG[job.scope.get("files", "all")]
+ job.log_line(f"Scan started · {folders} · {scope}")
job.update(status_line=f"Looking for media files in {folders}")
yield self._scan_progress(
0, f"Scanning {len(output_dirs)} directory(ies) for media files..."
@@ -1544,7 +1582,12 @@ class AdminRoutesMixin:
5, f"Found {len(media_files)} media files to process...", counts
)
roots = _scan_roots(output_dirs)
- async for event in self._scan_batches(media_files, counts, job, roots):
+ known = set()
+ if media_files and job.scope.get("files") != "all":
+ known = await self._known_scan_files(job, media_files)
+ async for event in self._scan_batches(
+ media_files, counts, job, roots, known
+ ):
yield event
self.logger.info(
@@ -1565,11 +1608,29 @@ class AdminRoutesMixin:
"message": "An internal error occurred. Check server logs for details.",
}
- async def _scan_batches(self, media_files, counts, job, roots=()):
+ async def _known_scan_files(self, job, media_files):
+ """The "new" scope's skip set; a failed read falls back to reading all."""
+ try:
+ return await self._run_in_executor(self._known_scan_files_sync, media_files)
+ except Exception:
+ self.logger.warning(
+ "Known files could not be read; the scan reads every file",
+ exc_info=True,
+ )
+ job.log_line("Known files could not be read · reading every file")
+ return set()
+
+ def _known_scan_files_sync(self, media_files):
+ """Stat each scanned file and ask the database which are known (blocking)."""
+ sizes = {str(media_file): _file_size(media_file) for media_file in media_files}
+ return self.db.get_known_resolved_paths(sizes)
+
+ async def _scan_batches(self, media_files, counts, job, roots=(), known=()):
"""Process ``media_files`` in executor batches, updating ``counts`` in place.
Records each file's outcome in ``job`` and yields one progress event
- per batch.
+ per batch. Files in ``known`` (path strings) are not read: they count
+ as processed and unchanged and are recorded as skipped.
"""
total = len(media_files)
started = time.monotonic()
@@ -1578,18 +1639,23 @@ class AdminRoutesMixin:
if job.cancelled:
return
batch = media_files[batch_start : batch_start + SCAN_BATCH_SIZE]
- batch_results = await self._run_in_executor(
- self._extract_batch_metadata_sync, batch
- )
- outcomes = []
- batch_counts = await self._run_in_executor(
- self._ingest_scan_batch_sync,
- batch_results,
- roots=roots,
- outcomes=outcomes,
- )
- for key in counts:
- counts[key] += batch_counts[key]
+ skipped = [f for f in batch if str(f) in known]
+ to_read = [f for f in batch if str(f) not in known]
+ outcomes = [_scan_outcome(f, roots, "skipped") for f in skipped]
+ counts["processed"] += len(skipped)
+ counts["unchanged"] += len(skipped)
+ if to_read:
+ batch_results = await self._run_in_executor(
+ self._extract_batch_metadata_sync, to_read
+ )
+ batch_counts = await self._run_in_executor(
+ self._ingest_scan_batch_sync,
+ batch_results,
+ roots=roots,
+ outcomes=outcomes,
+ )
+ for key in counts:
+ counts[key] += batch_counts[key]
for group, row, log_line in outcomes:
job.record(group, row, log_line)
@@ -1713,6 +1779,7 @@ class AdminRoutesMixin:
pass
if not canonical.get("prompt") and not canonical.get("parameters"):
return None
+ _with_file_size(canonical, media_file)
resolution = resolve_prompt_pair(canonical)
if resolution["state"] == "resolved":
result = ingest_resolved_pair(
diff --git a/py/api/job_routes.py b/py/api/job_routes.py
index 1d63eb4..71b35cb 100644
--- a/py/api/job_routes.py
+++ b/py/api/job_routes.py
@@ -10,13 +10,38 @@ import json
from aiohttp import web
-from .jobs import JobBusy
+from .jobs import DEFAULT_SCAN_SCOPE, SCAN_SCOPES, JobBusy
def _not_found():
return web.json_response({"success": False, "error": "Job not found"}, status=404)
+def _bad_request(message):
+ return web.json_response({"success": False, "error": message}, status=400)
+
+
+async def _scan_scope(request):
+ """The requested scan scope ("new" or "all"), or None when the body is invalid.
+
+ An empty body (also a chunked one with no bytes), or one without
+ ``scope``, asks for the default.
+ """
+ if not request.can_read_body:
+ return DEFAULT_SCAN_SCOPE
+ text = await request.text()
+ if not text.strip(): # e.g. a chunked POST that sent no bytes
+ return DEFAULT_SCAN_SCOPE
+ try:
+ body = json.loads(text)
+ except ValueError:
+ return None
+ if not isinstance(body, dict):
+ return None
+ scope = body.get("scope", DEFAULT_SCAN_SCOPE)
+ return scope if scope in SCAN_SCOPES else None
+
+
def _summary_event(job):
return f"data: {json.dumps(job.summary())}\n\n".encode("utf-8")
@@ -37,6 +62,10 @@ class JobRoutesMixin:
async def job_detail_route(request):
return await self.job_detail(request)
+ @routes.get("/prompt_manager/jobs/{job_id}/report")
+ async def job_report_route(request):
+ return await self.job_report(request)
+
@routes.get("/prompt_manager/jobs/{job_id}/events")
async def job_events_route(request):
return await self.job_events(request)
@@ -56,6 +85,22 @@ class JobRoutesMixin:
return _not_found()
return web.json_response({"success": True, "job": job.detail()})
+ async def job_report(self, request):
+ """The job's whole record as a JSON attachment (no absolute paths)."""
+ job = self._jobs.get(request.match_info["job_id"])
+ if job is None:
+ return _not_found()
+ body = json.dumps(job.report(), indent=2)
+ disposition = f'attachment; filename="{job.report_filename()}"'
+ return web.Response(
+ text=body,
+ content_type="application/json",
+ headers={
+ "Content-Disposition": disposition,
+ "Cache-Control": "no-store",
+ },
+ )
+
async def cancel_job(self, request):
"""Stop a job at its next file boundary; a finished job is a no-op."""
job_id = request.match_info["job_id"]
@@ -65,9 +110,16 @@ class JobRoutesMixin:
return web.json_response({"success": True, "cancellation_requested": requested})
async def start_scan_job(self, request):
- """Start the metadata scan as a job (202), or 409 naming the busy job."""
+ """Start the metadata scan as a job (202), or 409 naming the busy job.
+
+ The body's ``scope`` is "new" (the default: new and unresolved files)
+ or "all"; anything else is a 400.
+ """
+ files = await _scan_scope(request)
+ if files is None:
+ return _bad_request('The scan scope must be "new" or "all"')
try:
- job = self._start_scan_job()
+ job = self._start_scan_job(files)
except JobBusy as busy:
running = busy.running
return web.json_response(
diff --git a/py/api/jobs.py b/py/api/jobs.py
index 4918e4c..a5a4b0e 100644
--- a/py/api/jobs.py
+++ b/py/api/jobs.py
@@ -35,12 +35,19 @@ MAX_FINISHED_JOBS = 20 # finished jobs this session keeps
# The metadata scan's outcome groups, in counter-tile order of the spec's
# mapping, and the per-file scan status each group collects. Files without
# usable metadata have no status and go to "no_metadata".
+# Which files a scan reads (#249 Activity-2): "new" skips the files the Library
+# already links with a resolved link (get_known_resolved_paths) before reading
+# them, so new, unlinked, unresolved and ambiguous files are read; "all" reads
+# every file again. The workspace job defaults to "new"; legacy /scan is "all".
+SCAN_SCOPES = ("new", "all")
+DEFAULT_SCAN_SCOPE = "new"
SCAN_GROUP_NAMES = ("added", "known", "review", "no_metadata", "errors")
SCAN_GROUPS = {
"added": "added",
"linked": "added",
"enriched": "added",
"unchanged": "known",
+ "skipped": "known", # "new" scope: known and resolved, not read again
"ambiguous": "review",
"conflict": "review",
"unresolved": "review",
@@ -240,6 +247,36 @@ class Job:
"log": list(self.log),
}
+ def report(self):
+ """The downloadable report: the summary, every outcome group and the log.
+
+ Each group carries its exact count, its listed rows (at most
+ MAX_OUTCOME_ROWS) and whether the list is cut short. Rows and log lines
+ already hold root-relative paths only.
+ """
+ summary = self.summary()
+ summary.pop("listed", None)
+ summary.pop("log_length", None)
+ return {
+ "report_version": 1,
+ "generated_at": self._clock(),
+ "job": summary,
+ "outcomes": {
+ group: {
+ "count": self.counters.get(group, len(rows)),
+ "rows": [dict(row) for row in rows],
+ "truncated": self.counters.get(group, len(rows)) > len(rows),
+ }
+ for group, rows in self.outcomes.items()
+ },
+ "log": list(self.log),
+ }
+
+ def report_filename(self):
+ """The download name: prompt-manager-KIND-YYYYMMDD-HHMM.json (local time)."""
+ stamp = time.strftime("%Y%m%d-%H%M", time.localtime(self._clock()))
+ return f"prompt-manager-{self.kind}-{stamp}.json"
+
class JobRegistry:
"""Every job of this server session, running and recently finished."""
diff --git a/tests/browser/activity-concept.spec.mjs b/tests/browser/activity-concept.spec.mjs
index 4d6d37b..e3c8f9f 100644
--- a/tests/browser/activity-concept.spec.mjs
+++ b/tests/browser/activity-concept.spec.mjs
@@ -63,16 +63,35 @@ function measureNotice(selector) {
};
}
-/** D55: the concept's dialog with the production differences applied. */
+/** A concept `.setting-row` switch: its frame, its label and hint, and its checkbox (not its place-dependent margin). */
+function measureToggle(selector) {
+ const row = document.querySelector(selector);
+ const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
+ return {
+ children: [...row.children].map(node => [node.localName, ...node.classList].join('.')),
+ frame: css(row, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom',
+ 'borderTopWidth', 'borderTopStyle', 'borderTopColor'),
+ label: css(row.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
+ hint: css(row.querySelector('p'), 'fontSize', 'color', 'marginTop'),
+ box: css(row.querySelector('input[type="checkbox"]'), 'accentColor', 'width', 'height', 'appearance',
+ 'backgroundColor', 'borderTopLeftRadius'),
+ role: row.querySelector('input').getAttribute('role'),
+ labelled: row.querySelector('label').htmlFor === row.querySelector('input').id,
+ };
+}
+
+/** D55 and D58: the concept's dialog with the production differences applied. */
function mapReviewConcept(reference) {
return {
...reference,
- // No sample-plan note (prototype copy); the backup box sits before the actions.
+ // No sample-plan note (prototype copy); the D58 "All files" toggle follows the grid; the backup box sits
+ // before the actions.
body: [...reference.body.filter(item => item !== 'p.demo-note').slice(0, -1), 'div.notice', reference.body.at(-1)]
- .map(item => item === 'div.meta-grid' ? 'dl.meta-grid' : item),
- // A `dl`, as in the concept's maintenance plan; real sources and the fixed Activity-1 scope.
+ .flatMap(item => item === 'div.meta-grid' ? ['dl.meta-grid', 'div.setting-row'] : [item]),
+ // A `dl`, as in the concept's maintenance plan; real sources. The scope keeps the concept's
+ // "New and unresolved files" until "All files" is checked.
grid: { ...reference.grid, tag: 'dl', entries: reference.grid.entries.map(entry => ({ ...entry,
- dd: { Sources: '2 output folders', Scope: 'All files' }[entry.dt] })) },
+ dd: { Sources: '2 output folders', Scope: 'New and unresolved files' }[entry.dt] })) },
actions: { ...reference.actions, buttons: reference.actions.buttons.map(button => button.text === 'Simulate scan'
? { ...button, text: 'Start scan' } : button) },
};
@@ -92,7 +111,8 @@ async function conceptReview(page, theme) {
await page.getByRole('combobox', { name: 'Theme', exact: true }).selectOption(theme);
await page.locator('#main').getByRole('button', { name: 'Manage sources', exact: true }).click();
await expect(page.locator('#main .notice')).toBeVisible();
- return { review, notice: await page.evaluate(measureNotice, '#main .notice') };
+ return { review, notice: await page.evaluate(measureNotice, '#main .notice'),
+ toggle: await page.evaluate(measureToggle, '#main .setting-row:has(input[type="checkbox"])') };
}
async function productionReview(page, theme) {
@@ -103,7 +123,16 @@ async function productionReview(page, theme) {
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review metadata scan');
// The swapped dialog can open under the pointer that pressed Review scan; measure it at rest.
await page.mouse.move(0, 0);
- return { review: await page.evaluate(measureReview), notice: await page.evaluate(measureNotice, 'dialog#modal[open] .notice') };
+ const review = await page.evaluate(measureReview);
+ const notice = await page.evaluate(measureNotice, 'dialog#modal[open] .notice');
+ // The concept's switch is on; compare ours on too, then turn it off again (the default).
+ const allFiles = page.getByRole('switch', { name: 'All files', exact: true });
+ await allFiles.check();
+ await page.mouse.move(0, 0);
+ const toggle = await page.evaluate(measureToggle, 'dialog#modal[open] .setting-row');
+ await allFiles.uncheck();
+ await page.mouse.move(0, 0);
+ return { review, notice, toggle };
}
for (const theme of THEMES) test(`review dialog matches the concept scan-review ${theme}`, async ({ page, request }) => {
@@ -114,19 +143,29 @@ for (const theme of THEMES) test(`review dialog matches the concept scan-review
const production = await productionReview(page, theme);
expect(diffInventory(mapReviewConcept(concept.review), production.review)).toEqual([]);
expect(diffInventory(concept.notice, production.notice)).toEqual([]);
+ expect(diffInventory(concept.toggle, production.toggle)).toEqual([]);
+ await expect(page.locator('dialog#modal[open] .setting-row > div > label')).toHaveText('All files');
+ await expect(page.locator('dialog#modal[open] .setting-row input')).not.toBeChecked();
+ await expect(page.getByRole('switch', { name: 'All files', exact: true })).toBeVisible();
await expect(page.locator('dialog#modal[open] .notice h3')).toHaveText('Back up the database first (recommended)');
await expect(page.locator('dialog#modal[open] .notice .row-actions button')).toHaveText(['Back up now']);
// Mutation check: one CSS change and one removed element are both noticed, then undone to an exact match again.
- const undoCss = await mutate(page, { css: '#modal .meta-grid { gap: 3px; } #modal .notice { padding: 2px; }' });
+ const undoCss = await mutate(page, { css: '#modal .meta-grid { gap: 3px; } #modal .notice { padding: 2px; } #modal .setting-row { gap: 3px; }' });
const cssMutated = await page.evaluate(measureReview);
const noticeMutated = await page.evaluate(measureNotice, 'dialog#modal[open] .notice');
+ await page.getByRole('switch', { name: 'All files', exact: true }).check();
+ await page.mouse.move(0, 0);
+ const toggleMutated = await page.evaluate(measureToggle, 'dialog#modal[open] .setting-row');
+ await page.getByRole('switch', { name: 'All files', exact: true }).uncheck();
+ await page.mouse.move(0, 0);
+ expect(diffInventory(concept.toggle, toggleMutated).map(item => item.path)).toEqual(['frame.columnGap']);
expect(diffInventory(mapReviewConcept(concept.review), cssMutated).map(item => item.path)).toEqual(expect.arrayContaining(['grid.columnGap', 'grid.rowGap']));
expect(diffInventory(concept.notice, noticeMutated).map(item => item.path)).toEqual(['frame.paddingTop', 'frame.paddingLeft']);
await undoCss();
const undoDom = await mutate(page, { remove: ['#modal-body > p.small'] });
const domMutated = await page.evaluate(() => [...document.querySelectorAll('#modal-body > *')].map(node => node.localName));
- expect(domMutated).toEqual(['dl', 'div', 'div']);
+ expect(domMutated).toEqual(['dl', 'div', 'div', 'div']);
await undoDom();
expect(diffInventory(mapReviewConcept(concept.review), await page.evaluate(measureReview))).toEqual([]);
});
diff --git a/tests/browser/workspace-activity.spec.mjs b/tests/browser/workspace-activity.spec.mjs
index f61eac5..3970de0 100644
--- a/tests/browser/workspace-activity.spec.mjs
+++ b/tests/browser/workspace-activity.spec.mjs
@@ -392,16 +392,69 @@ test('start scan creates a job, toasts, and opens its view', async ({ page, requ
await expect(scanLog(page).locator('pre')).toContainText('Scan finished');
});
+// ── Task 4 (Activity-2): scan scope, new and unresolved files by default ─────────────────────
+
+test('all files checkbox changes the scope line and the request', async ({ page }) => {
+ await page.setViewportSize({ width: 1600, height: 1100 });
+ const bodies = [];
+ page.on('request', item => {
+ if (item.method() === 'POST' && new URL(item.url()).pathname === '/prompt_manager/jobs/scan') bodies.push(item.postDataJSON());
+ });
+ const review = page.getByRole('dialog', { name: 'Review metadata scan', exact: true });
+ const scopeValue = review.locator('.meta-grid > div').filter({ has: page.locator('dt', { hasText: 'Scope' }) }).locator('dd');
+ // The concept's switch: a checkbox with role=switch.
+ const allFiles = review.getByRole('switch', { name: 'All files', exact: true });
+ const openReview = async () => {
+ await page.goto(`${workspace}#/images`);
+ await page.getByRole('button', { name: 'Import', exact: true }).click();
+ await page.getByRole('dialog', { name: 'Import & inspect', exact: true }).locator('.operation')
+ .filter({ hasText: 'Scan for metadata' }).getByRole('button', { name: 'Review scan', exact: true }).click();
+ await expect(review).toBeVisible();
+ };
+
+ // The default: the concept's "New and unresolved files", the box unchecked; the request asks for "new".
+ await openReview();
+ await expect(scopeValue).toHaveText('New and unresolved files');
+ await expect(allFiles).not.toBeChecked();
+ await expect(review.locator('.setting-row p')).toHaveText('Also read the files already linked to their prompt again.');
+ await expect(allFiles).toHaveAccessibleDescription('Also read the files already linked to their prompt again.');
+ // Checking and unchecking switches the Scope value both ways.
+ await allFiles.check();
+ await expect(scopeValue).toHaveText('All files');
+ await allFiles.uncheck();
+ await expect(scopeValue).toHaveText('New and unresolved files');
+ await review.getByRole('button', { name: 'Start scan', exact: true }).click();
+ await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
+ await expect.poll(() => bodies.length).toBe(1);
+ expect(bodies[0]).toEqual({ scope: 'new' });
+ await expect(heading(page).locator('h1')).toHaveText('Scan for metadata');
+ // Wait for the first scan to end so the second is not refused as busy.
+ await scanLog(page).locator('summary').click();
+ await expect(scanLog(page).locator('pre')).toContainText('Scan finished', { timeout: 15000 });
+ await expect(scanLog(page).locator('pre')).toContainText('new and unresolved files');
+
+ // Checked: "All files", and the request asks for "all".
+ await openReview();
+ await allFiles.check();
+ await expect(scopeValue).toHaveText('All files');
+ await review.getByRole('button', { name: 'Start scan', exact: true }).click();
+ await expect.poll(() => bodies.length).toBe(2);
+ expect(bodies[1]).toEqual({ scope: 'all' });
+ await scanLog(page).locator('summary').click();
+ await expect(scanLog(page).locator('pre')).toContainText('Scan started · ', { timeout: 15000 });
+ await expect(scanLog(page).locator('pre')).toContainText('all files');
+});
+
test('job view counters follow the mapping and update live', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 8240, counters: { added: 1420, known: 6511, review: 37, no_metadata: 254, errors: 18 } });
await openJob(page, job.id);
- // The header: the title, the detail line, then Download report (Task 5) and Cancel scan (danger).
+ // The header: the title, the detail line, then Download report (enabled in Task 5) and Cancel scan (danger).
await expect(heading(page).locator('h1')).toHaveText('Scan for metadata');
await expect(heading(page).locator('p.muted')).toHaveText(/^Started \d\d:\d\d · 2 output folders · closing this view does not cancel it$/);
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report', 'Cancel scan']);
- await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeDisabled();
+ await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeEnabled();
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).toHaveClass(/danger/);
// Progress: the bar, the server's line and the five tiles in order and colour.
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Processing file 8,240 of 19,622');
@@ -558,6 +611,19 @@ test('job view renders 200-row cap note', async ({ page, request }) => {
await step(request, job.id, { processed: 260, bulk: { group: 'added', count: 250 }, records: [
...Array.from({ length: 10 }, (_, index) => ({ group: 'no_metadata', path: `misc/plain_${index}.png`, detail: 'No prompt metadata in the file' })),
] });
+ // Every detail read of this job: in flight, and whether one has returned the full listing.
+ const detailUrl = `/prompt_manager/jobs/${job.id}`;
+ const isDetail = item => new URL(item.url()).pathname === detailUrl;
+ const detailReads = { inFlight: 0, full: false };
+ page.on('request', item => { if (isDetail(item)) detailReads.inFlight += 1; });
+ page.on('requestfailed', item => { if (isDetail(item)) detailReads.inFlight -= 1; });
+ page.on('response', async response => {
+ if (!isDetail(response)) return;
+ const body = await response.json().catch(() => null);
+ const outcomes = body?.job?.outcomes ?? {};
+ if (outcomes.added?.length === 200 && outcomes.no_metadata?.length === 10) detailReads.full = true;
+ detailReads.inFlight -= 1;
+ });
await openJob(page, job.id);
await expect(tiles(page).first().locator('dd')).toHaveText('250');
await expect(fileRows(page)).toHaveCount(200);
@@ -573,6 +639,9 @@ test('job view renders 200-row cap note', async ({ page, request }) => {
// Review M12: with the list capped and the log closed, more files only update the counters; the detail is read
// again only when the log opens with new lines.
+ // The reads from opening the view have settled: one returned the full listing and none is in flight. The view
+ // re-reads only while what the summary lists differs from the last read, so no further read is due.
+ await expect.poll(() => detailReads.full && detailReads.inFlight === 0).toBe(true);
const reads = [];
page.on('request', item => { if (new URL(item.url()).pathname === `/prompt_manager/jobs/${job.id}`) reads.push(Date.now()); });
await step(request, job.id, { processed: 300, bulk: { group: 'added', count: 40 } });
@@ -639,6 +708,8 @@ test('an unknown or restarted job view says so', async ({ page, request }) => {
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
await expect(heading(page).locator('p.muted')).toHaveText(LOST);
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).toHaveCount(0);
+ // A lost job has no report on the server (review M9): Download report is disabled rather than failing.
+ await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeDisabled();
// What was read before the restart stays on the page.
await expect(fileRows(page)).toHaveCount(1);
await expect(pill(page)).toBeHidden();
@@ -662,3 +733,69 @@ test('a finished job the server pruned leaves its open view honest', async ({ pa
await expect(heading(page).locator('h1')).toBeFocused();
await expect(page).toHaveTitle('Activity · Prompt Manager');
});
+
+// Activity-2 Task 5 (D59): Download report and the log's scroll position.
+test('download report saves the job JSON', async ({ page, request }) => {
+ await page.setViewportSize({ width: 1600, height: 1100 });
+ const job = await startJob(request);
+ await step(request, job.id, { processed: 2, records: [
+ { group: 'added', path: 'sub/a.png', root: 'output', detail: 'Added' },
+ { group: 'errors', path: 'broken/b.png', root: 'output', detail: 'Error' },
+ ] });
+ await finish(request, job.id);
+ await openJob(page, job.id);
+ const button = heading(page).getByRole('button', { name: 'Download report', exact: true });
+ await expect(button).toBeEnabled();
+ const [download] = await Promise.all([page.waitForEvent('download'), button.click()]);
+ expect(download.suggestedFilename()).toMatch(/^prompt-manager-scan-\d{8}-\d{4}\.json$/);
+ const path = await download.path();
+ const report = JSON.parse(await (await import('node:fs/promises')).readFile(path, 'utf8'));
+ expect(report.job.id).toBe(job.id);
+ expect(report.outcomes.added.rows.map(row => row.path)).toEqual(['sub/a.png']);
+ expect(report.outcomes.errors.truncated).toBe(false);
+ expect(report.log.length).toBeGreaterThan(0);
+ await expect(toast(page)).toHaveText('Job report downloaded.');
+});
+
+test('log keeps the reader\'s scroll position', async ({ page, request }) => {
+ await page.setViewportSize({ width: 1600, height: 1100 });
+ const job = await startJob(request);
+ const more = (from, count) => Array.from({ length: count }, (_, index) => ({ group: 'known', path: `log/f_${from + index}.png`,
+ root: 'output', detail: 'Known' }));
+ await step(request, job.id, { processed: 60, records: more(0, 60) });
+ await openJob(page, job.id);
+ await scanLog(page).locator('summary').click();
+ const pre = scanLog(page).locator('pre');
+ await expect(pre).toContainText('log/f_59.png');
+ const metrics = () => pre.evaluate(node => ({ top: node.scrollTop, max: node.scrollHeight - node.clientHeight }));
+ // Opening starts at the newest line.
+ await expect.poll(async () => { const m = await metrics(); return m.max > 0 && m.top >= m.max - 2; }).toBe(true);
+ // Pinned to the bottom: new lines are followed.
+ await step(request, job.id, { processed: 90, records: more(60, 30) });
+ await expect(pre).toContainText('log/f_89.png');
+ await expect.poll(async () => { const m = await metrics(); return m.top >= m.max - 2; }).toBe(true);
+ // Scrolled up: the position stays while more lines arrive.
+ await pre.evaluate(node => { node.scrollTop = 40; });
+ await step(request, job.id, { processed: 120, records: more(90, 30) });
+ await expect(pre).toContainText('log/f_119.png');
+ expect((await metrics()).top).toBe(40);
+ // Closed and opened again (review M6): every open starts at the newest line, also after lines arrived meanwhile.
+ await scanLog(page).locator('summary').click();
+ await expect(scanLog(page)).not.toHaveAttribute('open', '');
+ await step(request, job.id, { processed: 150, records: more(120, 30) });
+ await scanLog(page).locator('summary').click();
+ await expect(pre).toContainText('log/f_149.png');
+ await expect.poll(async () => { const m = await metrics(); return m.max > 0 && m.top >= m.max - 2; }).toBe(true);
+});
+
+test('a failed report download says so and leaves the button usable', async ({ page, request }) => {
+ await page.setViewportSize({ width: 1600, height: 1100 });
+ const job = await startJob(request);
+ await finish(request, job.id);
+ await openJob(page, job.id);
+ await page.route(`**/prompt_manager/jobs/${job.id}/report`, route => route.fulfill({ status: 500, body: 'no' }));
+ const button = heading(page).getByRole('button', { name: 'Download report', exact: true });
+ await button.click();
+ await expect(toast(page)).toHaveText('The report could not be downloaded. Try again.');
+ await expect(button).not.toHaveAttribute('aria-disabled', 'true');
+});
diff --git a/tests/browser/workspace-images.spec.mjs b/tests/browser/workspace-images.spec.mjs
index 0e799ed..99dbbba 100644
--- a/tests/browser/workspace-images.spec.mjs
+++ b/tests/browser/workspace-images.spec.mjs
@@ -2060,7 +2060,7 @@ test('scan for metadata opens the review dialog from Import', async ({ page, req
await openScanReview(page);
const dialog = scanReview(page);
await expect(dialog.locator('dl.meta-grid > div > dt')).toHaveText(['Sources', 'Scope']);
- await expect(dialog.locator('dl.meta-grid > div > dd')).toHaveText(['2 output folders', 'All files']);
+ await expect(dialog.locator('dl.meta-grid > div > dd')).toHaveText(['2 output folders', 'New and unresolved files']);
await expect(dialog.locator('#modal-body > p.small')).toHaveText('Enrich positive / negative associations, reuse canonical text and report unchanged files. Do not inflate run counts or recreate existing thumbnails.');
await expect(dialog.locator('.notice h3')).toHaveText('Back up the database first (recommended)');
await expect(dialog.locator('.dialog-actions button')).toHaveText(['Cancel', 'Start scan']);
diff --git a/tests/browser/workspace_activity_fixture.py b/tests/browser/workspace_activity_fixture.py
index 40a9054..77f102a 100644
--- a/tests/browser/workspace_activity_fixture.py
+++ b/tests/browser/workspace_activity_fixture.py
@@ -129,7 +129,7 @@ def register_activity_commands(app, routes, api):
str(data.get("title", "Scan for metadata")),
record_changing=True,
scope={
- "mode": "all",
+ "files": str(data.get("files", "new")),
"output_folders": count(data.get("output_folders"), 2),
},
runner=fake_runner(ending),
diff --git a/tests/js/workspace-activity-store.test.js b/tests/js/workspace-activity-store.test.js
index 211825c..89eb72d 100644
--- a/tests/js/workspace-activity-store.test.js
+++ b/tests/js/workspace-activity-store.test.js
@@ -11,7 +11,7 @@ const ID_C = 'c'.repeat(32);
function summary(overrides = {}) {
return { id: ID_A, kind: 'scan', title: 'Scan for metadata', state: 'running', started_at: 1000, finished_at: null,
progress: 0, status_line: '', processed: 0, total: 0,
- counters: { added: 0, known: 0, review: 0, no_metadata: 0, errors: 0 }, scope: { mode: 'all', output_folders: 2 },
+ counters: { added: 0, known: 0, review: 0, no_metadata: 0, errors: 0 }, scope: { files: 'new', output_folders: 2 },
...overrides };
}
diff --git a/tests/test_api_admin.py b/tests/test_api_admin.py
index e7cc02b..363a9b1 100644
--- a/tests/test_api_admin.py
+++ b/tests/test_api_admin.py
@@ -2094,6 +2094,72 @@ class TestActivityJobs(AdminAPITestCase):
(legacy["processed"], legacy["found"], legacy["failed"]), (6, 4, 1)
)
+ async def test_report_is_attachment_without_absolute_paths(self):
+ """Task 5: the report downloads as JSON, whole and without server paths."""
+ import re
+
+ from py.api.jobs import MAX_OUTCOME_ROWS
+
+ async def runner(job):
+ absolute = f"{self.tmpdir}/output/deep/abs.png"
+ job.record(
+ "added",
+ {"path": absolute, "root": "output", "root_index": 0, "detail": "ok"},
+ f"+ {absolute} · ok",
+ )
+ for index in range(MAX_OUTCOME_ROWS + 5):
+ job.record(
+ "known",
+ {"path": f"k/{index}.png", "root": "output", "detail": "x"},
+ f"= k/{index}.png",
+ )
+ job.update(processed=MAX_OUTCOME_ROWS + 6, total=MAX_OUTCOME_ROWS + 6)
+
+ job = self.api._jobs.start(
+ "scan",
+ "Scan for metadata",
+ record_changing=True,
+ scope={"files": "new", "output_folders": 1},
+ runner=runner,
+ groups=("added", "known", "review", "no_metadata", "errors"),
+ )
+ await asyncio.wait_for(job.task, 10)
+
+ resp = await self.client.request("GET", f"/prompt_manager/jobs/{job.id}/report")
+ text = await resp.text()
+ self.assertEqual(resp.status, 200)
+ self.assertIn("application/json", resp.headers["Content-Type"])
+ disposition = resp.headers["Content-Disposition"]
+ self.assertRegex(
+ disposition,
+ r'^attachment; filename="prompt-manager-scan-\d{8}-\d{4}\.json"$',
+ )
+ self.assertNotIn(self.tmpdir, text)
+ self.assertIsNone(
+ re.search(r"[A-Za-z]:[\\/]|(?`, the server's "Processing file N of M · about X min left", and the
* outcome counters as `dl.stat-row` tiles: Added to Library (green), Already known, Needs review (warning), No usable
* metadata, Errors (red).
@@ -21,6 +21,7 @@ import { showWorkspaceToast } from './toast.mjs';
import { noun } from './plural.mjs';
import { parseJob, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
import { jobDetail, CANCEL_FAILED_TOAST } from './activity.mjs';
+import { downloadJobReport } from './activity-scan.mjs';
import { createMediaViewer } from './media-viewer.mjs';
import { createInspectionController } from './inspection-state.mjs';
import { createPromptSender } from './prompt-sender.mjs';
@@ -122,7 +123,6 @@ function section(title, id) {
export function mountActivityJobView(main, { store, scope, api, jobId }) {
const parts = pageHeading(main);
const download = createButton('Download report');
- download.disabled = true;
const cancel = createButton('Cancel scan', 'danger');
const progressPart = section('Progress', 'job-progress-title');
@@ -271,6 +271,10 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
document.title = `${job.title} · Prompt Manager`;
parts.description.textContent = jobDetail(job);
cancel.textContent = scan ? 'Cancel scan' : 'Cancel job';
+ // A job lost with the server has no report there (review M9).
+ const lost = job.state === 'lost';
+ if (lost && document.activeElement === download) parts.h1.focus();
+ download.disabled = lost;
if (job.state === 'running') {
if (!cancel.isConnected) parts.actions.append(cancel);
} else if (cancel.isConnected) {
@@ -290,9 +294,11 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
logLabel.textContent = `● ${scan ? 'Scan' : 'Job'} log`;
const text = detail?.log.length ? detail.log.join('\n') : 'No log lines yet.';
if (pre.textContent !== text) {
+ // Replacing the text clamps the scroll position, so keep it: pinned readers follow the end, others stay.
+ const { scrollTop } = pre;
const atEnd = pre.scrollTop + pre.clientHeight >= pre.scrollHeight - 2;
pre.textContent = text;
- if (atEnd) pre.scrollTop = pre.scrollHeight;
+ pre.scrollTop = atEnd ? pre.scrollHeight : scrollTop;
}
renderRows(job);
}
@@ -323,7 +329,12 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
reading = false;
}
render();
- if (readAgain) { readAgain = false; scheduleRead(); }
+ // Read once more only if the summary moved past what this read returned (review: no redundant re-read).
+ if (readAgain) {
+ readAgain = false;
+ const job = store.find(jobId);
+ if (job && job.state !== 'lost' && detailChanged(job)) scheduleRead();
+ }
}
/**
@@ -347,10 +358,23 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
}
scope.listen(log, 'toggle', () => {
+ // Every open starts at the newest line: while closed the log has no layout box (Firefox, WebKit), so a kept
+ // position would be lost anyway (review M6).
+ if (log.open) pre.scrollTop = pre.scrollHeight;
const job = store.find(jobId);
if (log.open && job && job.state !== 'lost' && detailChanged(job)) scheduleRead();
});
+ scope.listen(download, 'click', async () => {
+ if (download.disabled || download.getAttribute('aria-disabled') === 'true') return;
+ download.setAttribute('aria-disabled', 'true');
+ try {
+ // A view left while the report downloads shows no toast (review M10).
+ await downloadJobReport(jobId, { signal: scope.signal });
+ } finally {
+ download.removeAttribute('aria-disabled');
+ }
+ });
scope.listen(cancel, 'click', async () => {
if (cancel.getAttribute('aria-disabled') === 'true') return;
cancel.setAttribute('aria-disabled', 'true');
diff --git a/web/js/workspace/activity-scan.mjs b/web/js/workspace/activity-scan.mjs
index 9fdfd09..8ec2c87 100644
--- a/web/js/workspace/activity-scan.mjs
+++ b/web/js/workspace/activity-scan.mjs
@@ -1,7 +1,8 @@
/**
* Scan for metadata (#249 Activity-1, D55): Images → Import → "Scan for metadata" → Review scan opens the concept's
* "Review metadata scan". Its backup box downloads the existing `/prompt_manager/backup` copy and toasts the outcome;
- * Start scan starts the session job through the Activity store, which toasts "Scan started.", and opens the job's view.
+ * Start scan starts the session job through the Activity store over the chosen scope (new and unresolved files, or
+ * all files with the D58 checkbox), which toasts "Scan started.", and opens the job's view.
* A start refused because another record-changing job runs toasts "Another job is running: ." with a link to
* that job (D57).
*/
@@ -12,13 +13,29 @@ import { navigateTo } from './state.mjs';
export const BACKUP_URL = '/prompt_manager/backup';
export const BACKUP_DONE = 'Database backup downloaded.';
export const BACKUP_FAILED = 'The database could not be backed up. Try again.';
+export const REPORT_DONE = 'Job report downloaded.';
+export const REPORT_FAILED = 'The report could not be downloaded. Try again.';
+const JOBS_REPORT_BASE = '/prompt_manager/jobs';
export const START_FAILED = 'The scan could not start. Try again.';
const FALLBACK_NAME = 'prompts_backup.db';
+const REPORT_FALLBACK_NAME = 'prompt-manager-report.json';
-/** The download name from a Content-Disposition header, or the fallback. */
-export function backupFilename(disposition) {
+/** The download name from a Content-Disposition header, or `fallback`. */
+export function backupFilename(disposition, fallback = FALLBACK_NAME) {
const match = /filename="?([^";]+)"?/i.exec(disposition || '');
- return match ? match[1] : FALLBACK_NAME;
+ return match ? match[1] : fallback;
+}
+
+/** Save a fetched response through a temporary link, named by `filename`. */
+function saveResponse(blob, filename) {
+ const url = URL.createObjectURL(blob);
+ const link = document.createElement('a');
+ link.href = url;
+ link.download = filename;
+ document.body.append(link);
+ link.click();
+ link.remove();
+ setTimeout(() => URL.revokeObjectURL(url), 1000);
}
/** Download the prompts database copy the legacy admin page also uses; toasts the outcome. */
@@ -27,14 +44,7 @@ export async function downloadDatabaseBackup({ fetchUrl = window.ApiClient.fetch
const response = await fetchUrl(BACKUP_URL);
if (!response.ok) throw new Error(`Backup failed with ${response.status}`);
const blob = await response.blob();
- const url = URL.createObjectURL(blob);
- const link = document.createElement('a');
- link.href = url;
- link.download = backupFilename(response.headers.get('Content-Disposition'));
- document.body.append(link);
- link.click();
- link.remove();
- setTimeout(() => URL.revokeObjectURL(url), 1000);
+ saveResponse(blob, backupFilename(response.headers.get('Content-Disposition')));
notify(BACKUP_DONE);
return true;
} catch {
@@ -43,16 +53,39 @@ export async function downloadDatabaseBackup({ fetchUrl = window.ApiClient.fetch
}
}
+/**
+ * Download a job's JSON report (Activity-2, D59) through the api-client URL, so a proxy mount works; toasts the outcome
+ * unless `signal` (the view's scope) was aborted meanwhile.
+ */
+export async function downloadJobReport(jobId, { fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast,
+ signal = null } = {}) {
+ try {
+ const response = await fetchUrl(`${JOBS_REPORT_BASE}/${encodeURIComponent(jobId)}/report`);
+ if (!response.ok) throw new Error(`Report failed with ${response.status}`);
+ const blob = await response.blob();
+ saveResponse(blob, backupFilename(response.headers.get('Content-Disposition'), REPORT_FALLBACK_NAME));
+ // The file is saved either way; a view left meanwhile (`signal` aborted) shows no toast.
+ if (!signal?.aborted) notify(REPORT_DONE);
+ return true;
+ } catch {
+ if (!signal?.aborted) notify(REPORT_FAILED);
+ return false;
+ }
+}
+
/** The toast for a refused start, linking the job that holds the slot. */
export function busyToast(running) {
if (!running) return 'Another job is running. Try again when it ends.';
return ['Another job is running: ', { text: running.title, href: `#/activity/${running.id}` }, '.'];
}
-/** Start the scan job and open its view; a refused or failed start says why and stays on the page. */
-export async function startMetadataScan({ store, scope, notify = showWorkspaceToast, navigate = navigateTo }) {
+/**
+ * Start the scan job over `files` ('new': new and unresolved files, or 'all') and open its view; a refused or failed
+ * start says why and stays on the page.
+ */
+export async function startMetadataScan({ store, scope, files = 'new', notify = showWorkspaceToast, navigate = navigateTo }) {
try {
- const result = await store.start('scan');
+ const result = await store.start('scan', { scope: files });
if (scope.signal.aborted) return;
if (result.busy !== undefined) {
notify(busyToast(result.busy));
@@ -68,5 +101,5 @@ export async function startMetadataScan({ store, scope, notify = showWorkspaceTo
export function openMetadataScan({ scope, store, sources, trigger = null }) {
openScanReviewDialog({ scope, sources, trigger,
onBackup: () => downloadDatabaseBackup(),
- onStart: () => startMetadataScan({ store, scope }) });
+ onStart: ({ scope: files }) => startMetadataScan({ store, scope, files }) });
}
diff --git a/web/js/workspace/workspace-dialogs.mjs b/web/js/workspace/workspace-dialogs.mjs
index 6872a68..72b3e86 100644
--- a/web/js/workspace/workspace-dialogs.mjs
+++ b/web/js/workspace/workspace-dialogs.mjs
@@ -20,7 +20,7 @@
*
* Every opener takes the page `scope`: disposing it closes and removes the shell, so a dialog never outlives its page.
*/
-import { createButton } from '../shared/toolkit/core.js';
+import { createButton, createSwitch } from '../shared/toolkit/core.js';
const FIELD_LABELS = Object.freeze({ source: 'Source', folder: 'Folder', media: 'Media', tags: 'Tags' });
const FIELD_ORDER = Object.keys(FIELD_LABELS);
@@ -342,22 +342,44 @@ export const SCAN_REVIEW_SENTENCE = 'Enrich positive / negative associations, re
const BACKUP_LABEL = 'Back up now';
const BACKUP_BUSY_LABEL = 'Backing up…';
+export const SCAN_SCOPE_LABELS = { new: 'New and unresolved files', all: 'All files' };
+export const SCAN_ALL_FILES_HINT = 'Also read the files already linked to their prompt again.';
+
/**
- * The concept's "Review metadata scan" (`scan-review`, D55). `sources` is the number of output folders the scan reads;
- * the scope is fixed to "All files" in Activity-1. `onBackup` runs the database backup (the button is aria-disabled
- * and reads "Backing up…" until it settles, and the dialog stays open); Start scan closes the dialog, then `onStart`.
+ * The concept's "Review metadata scan" (`scan-review`, D55). `sources` is the number of output folders the scan reads.
+ * The scope is the concept's "New and unresolved files"; the "All files" checkbox (D58, the concept's `.setting-row`
+ * switch, unchecked) switches the Scope value to "All files". `onBackup` runs the database backup (the button is
+ * aria-disabled and reads "Backing up…" until it settles, and the dialog stays open); Start scan closes the dialog, then
+ * calls `onStart({scope: 'new' | 'all'})`.
*/
export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigger = null }) {
requireScope(scope);
requireCallbacks({ onBackup, onStart });
if (!Number.isSafeInteger(sources) || sources < 0) throw new TypeError('The scan review needs its source count');
const grid = el('dl', undefined, 'meta-grid');
+ const values = {};
for (const [term, value] of [['Sources', `${sources.toLocaleString('en-US')} output ${sources === 1 ? 'folder' : 'folders'}`],
- ['Scope', 'All files']]) {
+ ['Scope', SCAN_SCOPE_LABELS.new]]) {
const entry = el('div');
- entry.append(el('dt', term), el('dd', value));
+ values[term] = el('dd', value);
+ entry.append(el('dt', term), values[term]);
grid.append(entry);
}
+ // The concept's System toggle: a `.setting-row` whose checkbox is the Toolkit switch (concept `hydrateSwitches`).
+ const toggle = createSwitch('', '', 'scan-all-files');
+ const allFiles = toggle.querySelector('input');
+ allFiles.checked = false;
+ const option = el('div', undefined, 'setting-row');
+ const optionText = el('div');
+ const optionLabel = el('label', SCAN_SCOPE_LABELS.all);
+ optionLabel.htmlFor = allFiles.id;
+ const hint = el('p', SCAN_ALL_FILES_HINT);
+ hint.id = 'scan-all-files-hint';
+ allFiles.setAttribute('aria-describedby', hint.id);
+ optionText.append(optionLabel, hint);
+ option.append(optionText, toggle);
+ const chosenScope = () => (allFiles.checked ? 'all' : 'new');
+ allFiles.addEventListener('change', () => { values.Scope.textContent = SCAN_SCOPE_LABELS[chosenScope()]; });
const notice = el('div', undefined, 'notice');
const copy = el('div');
copy.append(el('h3', 'Back up the database first (recommended)'),
@@ -381,10 +403,11 @@ export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigge
const start = createButton('Start scan', 'ca-primary');
cancel.addEventListener('click', () => finish());
start.addEventListener('click', () => {
+ const files = chosenScope();
finish();
- onStart();
+ onStart({ scope: files });
});
const actions = el('div', undefined, 'dialog-actions');
actions.append(cancel, start);
- show('Review metadata scan', [grid, el('p', SCAN_REVIEW_SENTENCE, 'small'), notice, actions], { scope, trigger });
+ show('Review metadata scan', [grid, option, el('p', SCAN_REVIEW_SENTENCE, 'small'), notice, actions], { scope, trigger });
}