Compare commits

...
Author SHA1 Message Date
Vito Sansevero e2c1cc8fc9 test(activity): wait for the job view's settled detail read, not a fixed delay
The 200-row cap test now starts counting detail reads once one read has
returned the full listing and none is in flight. The job view re-reads after
an in-flight read only when the summary still lists more than that read
returned, so no redundant read is left due after opening.
2026-10-21 17:34:02 -07:00
Vito Sansevero 66b3d078db fix(activity): re-read known files whose size changed, and review follow-ups
The new scope now skips a known, resolved file only while its stored
generated_images.file_size equals its size on disk; a NULL stored size is
re-read. A file replaced at the same path is read again and its conflict
shows under Needs review. The scan stores each file's size when it ingests
it, and both sides of the known-set match go through normalize_image_path.

A failed known-set read logs and falls back to reading every file, and an
empty request body asks for the default scope. The All files switch is
described by its hint, a lost job's Download report is disabled, the report
toast is skipped once the view is gone, and the scan log opens at its
newest line every time. Fixtures use the real scope shape.
2026-10-21 16:38:38 -07:00
Vito Sansevero f95c7adfc9 feat(activity): download a job report
GET /prompt_manager/jobs/{id}/report returns the summary, counters, every
outcome group's listed rows with truncated flags, the scope and the log as a
JSON attachment (prompt-manager-<kind>-<YYYYMMDD-HHMM>.json), with no absolute
paths. The job view's Download report saves it through the api-client URL, and
the scan log opens at its newest line and follows new lines only while the
reader is at the bottom.
2026-10-21 15:56:49 -07:00
Vito Sansevero a32e66729b feat(activity): scan new and unresolved files by default, all files on request
Scan for metadata now skips, before reading any metadata, the files the
Library already links with a resolved link (one generated_images row with
exactly one pair link), found with one streamed query that keeps only the
scanned paths and compares paths with os.path.normcase on both sides.
Skipped files count as Already known ("Skipped · already linked").
Unlinked, fallback-linked, ambiguous and unresolved files are read again.

POST /prompt_manager/jobs/scan takes {scope: "new" | "all"} (default
"new", anything else is a 400). The review dialog shows the concept's
"New and unresolved files" and adds an "All files" switch (D58). The
legacy /prompt_manager/scan route keeps scanning all files with unchanged
SSE payloads.
2026-10-21 14:55:11 -07:00
16 changed files with 1068 additions and 71 deletions
+60 -2
View File
@@ -550,8 +550,9 @@ Recorded shell deviations:
configured sources.", button "Review scan"). It opens the concept's `scan-review`,
"Review metadata scan": no sample-plan `.demo-note` (prototype copy); Sources and
Scope in a `dl.meta-grid` (the concept's maintenance plan uses `dl`; `scan-review`
uses a `div`) reading "N output folders" (the discovered sources) and "All files"
(fixed in Activity-1; Activity-2 adds the scope option); the concept's sentence; then
uses a `div`) reading "N output folders" (the discovered sources) and the concept's
"New and unresolved files" (the D58 checkbox switches it to "All files"); the
concept's sentence; then
a backup `.notice` (the concept's System notice box): `h3` "Back up the database first
(recommended)", "Downloads a copy of the prompts database before the scan changes any
records." and "Back up now", which downloads the existing `/prompt_manager/backup`
@@ -624,6 +625,63 @@ Recorded shell deviations:
is running: <title>." with the title linking that job's view (the shared toast now
accepts link parts). A toast with a link stays 20 s instead of 6.5 s, and any toast
stays while the pointer or focus is on it.
- **D58** (scan scope, Activity-2 Task 4): the concept's `scan-review` states a fixed
scope, "New and unresolved files". Production keeps that value as the default and adds
the concept's own System toggle after the `dl.meta-grid`: a `.setting-row` with the
label "All files", the hint "Also read the files already linked to their prompt
again." (the switch's `aria-describedby`) and the Toolkit switch (`label.ca-switch`, a
checkbox with `role="switch"`, as the concept's `hydrateSwitches` renders it),
unchecked. Checking it switches the Scope value to "All files"; Start scan sends
`POST /prompt_manager/jobs/scan {scope: "new" | "all"}` (no body, an empty body or no
`scope` is "new"; another value is a 400).
- In "new" the scan skips, before reading any metadata, the files the Library already
links with a resolved link and that are unchanged on disk: exactly one
`generated_images` row for the file, exactly one distinct pair link on it
(`image_prompt_pairs`, written only by resolved ingestion), and a stored
`file_size` equal to the file's size in this walk. Both paths go through
`normalize_image_path` (absolute, normalized, `os.path.normcase`); a row without
`file_path` matches by its `image_path`.
- Read again: files linked without a pair (the monitor's fallback and FIFO links, the
legacy positive-only import), linked to more than one prompt or pair, not linked, a
file whose size changed (one replaced at the same path, e.g. a reused ComfyUI
counter name, is read and its conflict lands under Needs review), and a row with no
stored size (unknown is not known). The scan now stores each file's size when it
ingests it, so a file read once is skipped by the next "new" pass; the first pass
after the upgrade fills the missing sizes, which reports those files as enriched
(Added) instead of unchanged.
- The known set is one streamed query that keeps only the scanned paths. If it cannot
be read, the scan logs "Known files could not be read · reading every file" and
reads every file.
- A skipped file counts as processed and unchanged in the legacy counts (not as
found) and under Already known, with the detail "Skipped · already linked" and the
log line "= <root>/<path> · Skipped · already linked"; the log's first line names
the scope ("Scan started · N output folders · new and unresolved files" or "· all
files"). A legacy-page scan that attaches to a running "new" job therefore shows
the skipped files in its `unchanged` count.
- "All files" reads every file, as the scan did in Activity-1, and the legacy
`/prompt_manager/scan` route always does when it starts the scan (its SSE payload
keys are unchanged).
- `activity-concept.spec.mjs` compares the switch with the concept's System → Sources
toggle ("Watch for new outputs"), without its place-dependent top margin;
`workspace-activity.spec.mjs` covers the Scope value, the description and the
request both ways; `tests/test_scan_scope.py` covers the skip, the size rule, the
replaced file, the re-read of unresolved files, "all", the legacy route, the
fallback, the empty body, the 400 and Windows case folding.
- **D59** (job report and log, Activity-2 Task 5): the job view's "Download report" is
enabled. It fetches `GET /prompt_manager/jobs/<id>/report` through the api-client URL
(so a proxy mount works) and saves a JSON attachment named
`prompt-manager-<kind>-<YYYYMMDD-HHMM>.json` (server local time), then toasts "Job
report downloaded." (or "The report could not be downloaded. Try again."). The report
holds the summary, every counter, each outcome group's exact `count`, its listed
`rows` (at most 200) and a `truncated` flag, the scope and the log; rows and log lines
carry root-relative paths only, never an absolute path (an unknown id is a 404).
A job lost with a restart has no report on the server, so its Download report is
disabled; a view left while the report downloads still saves it but shows no toast.
The scan log opens at its newest line every time it is opened and follows new lines
only while the reader is at the bottom; a reader who scrolled up keeps the position
while it stays open (checked on Chromium, Firefox and WebKit).
`test_report_is_attachment_without_absolute_paths` and the browser tests `download
report saves the job JSON` and `log keeps the reader's scroll position` cover it.
- **D29**: no prototype copy. There is no "Interactive concept" badge and no
"What changes & why" button. The footer reads "ComfyAssets · Local tools. Your
library.", and the title has no " concept" suffix. Save view's dialog action reads
+61
View File
@@ -181,6 +181,21 @@ AND p.id NOT IN (SELECT prompt_id FROM live_hint_prompts)
"""
# The scan's known set (#249 Activity-2): every image row, with its stored file
# size and whether its pair link is resolved: exactly one distinct pair, the rule
# _LINK_SUMMARY_SQL and get_image_prompt_info use. A file is known and resolved
# when it has exactly one image row, that row is resolved, and the row's stored
# size equals the file's size now; several rows (one file linked to more than
# one prompt) are ambiguous, and an unknown (NULL) stored size is not known. The
# rows are streamed in fetches of KNOWN_PATHS_FETCH_SIZE and only the scanned
# paths are kept, so memory follows the scan (at most MAX_SCAN_FILES paths), not
# the table.
_KNOWN_PATHS_SQL = """SELECT gi.file_path, gi.image_path, gi.file_size,
(SELECT COUNT(DISTINCT ip.pair_id) FROM image_prompt_pairs ip
WHERE ip.image_id = gi.id) = 1
FROM generated_images gi"""
KNOWN_PATHS_FETCH_SIZE = 1000
# One SELECT reads the link summaries of at most this many paths: three bound
# parameters per path stays under SQLite's historical 999-variable limit.
LINK_SUMMARY_BATCH = 200
@@ -2280,6 +2295,52 @@ class PromptDatabase(
"parameters": row[8] if len(row) > 8 else None,
}
def get_known_resolved_paths(self, files: Dict[str, Optional[int]]) -> Set[str]:
"""The paths in ``files`` the Library already links with a resolved link.
``files`` maps each scanned path to its size on disk (None when it could
not be read). A path qualifies when exactly one ``generated_images`` row
claims the file, that row has exactly one pair link
(``image_prompt_pairs``, written only by resolved ingestion), and its
stored ``file_size`` is not NULL and equals the given size, so a file
replaced at the same path is read again. Files linked without a pair
(the monitor's fallback and FIFO links, the legacy positive-only
import), linked to several prompts or pairs, or not linked at all do
not qualify. Both sides go through ``normalize_image_path`` (absolute,
normalized, ``os.path.normcase``), so a Windows row stored in another
case or spelling still matches. One query, streamed; returns the
matching keys of ``files``.
"""
wanted: Dict[str, List[Tuple[str, Optional[int]]]] = {}
for path, size in files.items():
key = normalize_image_path(str(path))
wanted.setdefault(key, []).append((str(path), size))
if not wanted:
return set()
rows_seen: Dict[str, int] = {}
stored_sizes: Dict[str, int] = {}
with self.model.get_connection() as conn:
cursor = conn.execute(_KNOWN_PATHS_SQL)
while True:
rows = cursor.fetchmany(KNOWN_PATHS_FETCH_SIZE)
if not rows:
break
for file_path, image_path, file_size, is_resolved in rows:
# image_path is NOT NULL; file_path is NULL only on old rows.
key = normalize_image_path(file_path or image_path)
if key not in wanted:
continue
rows_seen[key] = rows_seen.get(key, 0) + 1
if is_resolved and file_size is not None:
stored_sizes[key] = file_size
return {
path
for key, stored_size in stored_sizes.items()
if rows_seen[key] == 1
for path, size in wanted[key]
if size is not None and size == stored_size
}
def get_image_link_summaries(
self, paths: List[str], allow_filename_fallback: bool = True
) -> Dict[str, Dict[str, Any]]:
+88 -21
View File
@@ -17,7 +17,14 @@ from pathlib import Path
from aiohttp import web
from .prompts import thumbnail_url_for
from .jobs import SCAN_GROUP_NAMES, SCAN_GROUPS, Job, JobBusy
from .jobs import (
DEFAULT_SCAN_SCOPE,
SCAN_GROUP_NAMES,
SCAN_GROUPS,
SCAN_SCOPES,
Job,
JobBusy,
)
try:
from ...utils.parallel import map_parallel
@@ -150,6 +157,8 @@ SCAN_FINAL_STATES = {
"cancelled": "cancelled",
"error": "failed",
}
# The scope as the scan log's first line names it (the review dialog's words).
SCAN_SCOPE_LOG = {"new": "new and unresolved files", "all": "all files"}
MIN_BATCHES_FOR_ESTIMATE = 2 # "about X min left" waits for this many batches
# Log line prefix per outcome group (spec: + added, = known, ? review, ! error,
# - no usable metadata).
@@ -174,6 +183,7 @@ SCAN_DETAILS = {
"linked": "Linked to an existing prompt",
"enriched": "Metadata added to its prompt",
"unchanged": "Already linked to its prompt",
"skipped": "Skipped · already linked",
"ambiguous": "Needs review: more than one prompt could match",
"unresolved": "Needs review: the prompt could not be traced in the workflow",
"conflict": "Needs review: already linked to a different prompt",
@@ -197,6 +207,30 @@ def _scan_status_line(done, total, batches, elapsed):
return f"{line} · about {minutes} min left"
def _file_size(path):
"""``path``'s size on disk, or None when it cannot be read."""
try:
return os.path.getsize(path)
except OSError:
return None
def _with_file_size(canonical, media_file):
"""Record the file's size in ``file_info`` unless the metadata carries one.
Stored as ``generated_images.file_size``; the "new" scan scope skips a known
file only while its size is unchanged.
"""
info = canonical.get("file_info")
info = dict(info) if isinstance(info, dict) else {}
if info.get("size") is None:
size = _file_size(media_file)
if size is not None:
info["size"] = size
if info:
canonical["file_info"] = info
def _scan_roots(output_dirs):
"""(absolute root, public label, listing index), the most specific first.
@@ -1426,7 +1460,7 @@ class AdminRoutesMixin:
job = self._jobs.active(SCAN_JOB_KIND)
if job is None:
try:
job = self._start_scan_job()
job = self._start_scan_job("all")
except JobBusy as busy:
return self._long_job_busy_response(busy.running.title)
return await self._stream_scan_job(request, job)
@@ -1450,16 +1484,19 @@ class AdminRoutesMixin:
}
)
def _start_scan_job(self):
def _start_scan_job(self, files=DEFAULT_SCAN_SCOPE):
"""Register the scan job synchronously; its runner starts on the loop.
Raises JobBusy while another record-changing job runs.
``files`` is the scope (``SCAN_SCOPES``). Raises JobBusy while another
record-changing job runs.
"""
if files not in SCAN_SCOPES:
raise ValueError(f"Unknown scan scope: {files!r}")
job = self._jobs.start(
SCAN_JOB_KIND,
SCAN_JOB_TITLE,
record_changing=True,
scope={"mode": "all"},
scope={"files": files},
runner=self._run_scan_job,
groups=SCAN_GROUP_NAMES,
)
@@ -1525,7 +1562,8 @@ class AdminRoutesMixin:
dir_names = [self._public_path(d) for d in output_dirs]
folders = _plural(len(output_dirs), "output folder")
job.scope["output_folders"] = len(output_dirs)
job.log_line(f"Scan started · {folders}")
scope = SCAN_SCOPE_LOG[job.scope.get("files", "all")]
job.log_line(f"Scan started · {folders} · {scope}")
job.update(status_line=f"Looking for media files in {folders}")
yield self._scan_progress(
0, f"Scanning {len(output_dirs)} directory(ies) for media files..."
@@ -1544,7 +1582,12 @@ class AdminRoutesMixin:
5, f"Found {len(media_files)} media files to process...", counts
)
roots = _scan_roots(output_dirs)
async for event in self._scan_batches(media_files, counts, job, roots):
known = set()
if media_files and job.scope.get("files") != "all":
known = await self._known_scan_files(job, media_files)
async for event in self._scan_batches(
media_files, counts, job, roots, known
):
yield event
self.logger.info(
@@ -1565,11 +1608,29 @@ class AdminRoutesMixin:
"message": "An internal error occurred. Check server logs for details.",
}
async def _scan_batches(self, media_files, counts, job, roots=()):
async def _known_scan_files(self, job, media_files):
"""The "new" scope's skip set; a failed read falls back to reading all."""
try:
return await self._run_in_executor(self._known_scan_files_sync, media_files)
except Exception:
self.logger.warning(
"Known files could not be read; the scan reads every file",
exc_info=True,
)
job.log_line("Known files could not be read · reading every file")
return set()
def _known_scan_files_sync(self, media_files):
"""Stat each scanned file and ask the database which are known (blocking)."""
sizes = {str(media_file): _file_size(media_file) for media_file in media_files}
return self.db.get_known_resolved_paths(sizes)
async def _scan_batches(self, media_files, counts, job, roots=(), known=()):
"""Process ``media_files`` in executor batches, updating ``counts`` in place.
Records each file's outcome in ``job`` and yields one progress event
per batch.
per batch. Files in ``known`` (path strings) are not read: they count
as processed and unchanged and are recorded as skipped.
"""
total = len(media_files)
started = time.monotonic()
@@ -1578,18 +1639,23 @@ class AdminRoutesMixin:
if job.cancelled:
return
batch = media_files[batch_start : batch_start + SCAN_BATCH_SIZE]
batch_results = await self._run_in_executor(
self._extract_batch_metadata_sync, batch
)
outcomes = []
batch_counts = await self._run_in_executor(
self._ingest_scan_batch_sync,
batch_results,
roots=roots,
outcomes=outcomes,
)
for key in counts:
counts[key] += batch_counts[key]
skipped = [f for f in batch if str(f) in known]
to_read = [f for f in batch if str(f) not in known]
outcomes = [_scan_outcome(f, roots, "skipped") for f in skipped]
counts["processed"] += len(skipped)
counts["unchanged"] += len(skipped)
if to_read:
batch_results = await self._run_in_executor(
self._extract_batch_metadata_sync, to_read
)
batch_counts = await self._run_in_executor(
self._ingest_scan_batch_sync,
batch_results,
roots=roots,
outcomes=outcomes,
)
for key in counts:
counts[key] += batch_counts[key]
for group, row, log_line in outcomes:
job.record(group, row, log_line)
@@ -1713,6 +1779,7 @@ class AdminRoutesMixin:
pass
if not canonical.get("prompt") and not canonical.get("parameters"):
return None
_with_file_size(canonical, media_file)
resolution = resolve_prompt_pair(canonical)
if resolution["state"] == "resolved":
result = ingest_resolved_pair(
+55 -3
View File
@@ -10,13 +10,38 @@ import json
from aiohttp import web
from .jobs import JobBusy
from .jobs import DEFAULT_SCAN_SCOPE, SCAN_SCOPES, JobBusy
def _not_found():
return web.json_response({"success": False, "error": "Job not found"}, status=404)
def _bad_request(message):
return web.json_response({"success": False, "error": message}, status=400)
async def _scan_scope(request):
"""The requested scan scope ("new" or "all"), or None when the body is invalid.
An empty body (also a chunked one with no bytes), or one without
``scope``, asks for the default.
"""
if not request.can_read_body:
return DEFAULT_SCAN_SCOPE
text = await request.text()
if not text.strip(): # e.g. a chunked POST that sent no bytes
return DEFAULT_SCAN_SCOPE
try:
body = json.loads(text)
except ValueError:
return None
if not isinstance(body, dict):
return None
scope = body.get("scope", DEFAULT_SCAN_SCOPE)
return scope if scope in SCAN_SCOPES else None
def _summary_event(job):
return f"data: {json.dumps(job.summary())}\n\n".encode("utf-8")
@@ -37,6 +62,10 @@ class JobRoutesMixin:
async def job_detail_route(request):
return await self.job_detail(request)
@routes.get("/prompt_manager/jobs/{job_id}/report")
async def job_report_route(request):
return await self.job_report(request)
@routes.get("/prompt_manager/jobs/{job_id}/events")
async def job_events_route(request):
return await self.job_events(request)
@@ -56,6 +85,22 @@ class JobRoutesMixin:
return _not_found()
return web.json_response({"success": True, "job": job.detail()})
async def job_report(self, request):
"""The job's whole record as a JSON attachment (no absolute paths)."""
job = self._jobs.get(request.match_info["job_id"])
if job is None:
return _not_found()
body = json.dumps(job.report(), indent=2)
disposition = f'attachment; filename="{job.report_filename()}"'
return web.Response(
text=body,
content_type="application/json",
headers={
"Content-Disposition": disposition,
"Cache-Control": "no-store",
},
)
async def cancel_job(self, request):
"""Stop a job at its next file boundary; a finished job is a no-op."""
job_id = request.match_info["job_id"]
@@ -65,9 +110,16 @@ class JobRoutesMixin:
return web.json_response({"success": True, "cancellation_requested": requested})
async def start_scan_job(self, request):
"""Start the metadata scan as a job (202), or 409 naming the busy job."""
"""Start the metadata scan as a job (202), or 409 naming the busy job.
The body's ``scope`` is "new" (the default: new and unresolved files)
or "all"; anything else is a 400.
"""
files = await _scan_scope(request)
if files is None:
return _bad_request('The scan scope must be "new" or "all"')
try:
job = self._start_scan_job()
job = self._start_scan_job(files)
except JobBusy as busy:
running = busy.running
return web.json_response(
+37
View File
@@ -35,12 +35,19 @@ MAX_FINISHED_JOBS = 20 # finished jobs this session keeps
# The metadata scan's outcome groups, in counter-tile order of the spec's
# mapping, and the per-file scan status each group collects. Files without
# usable metadata have no status and go to "no_metadata".
# Which files a scan reads (#249 Activity-2): "new" skips the files the Library
# already links with a resolved link (get_known_resolved_paths) before reading
# them, so new, unlinked, unresolved and ambiguous files are read; "all" reads
# every file again. The workspace job defaults to "new"; legacy /scan is "all".
SCAN_SCOPES = ("new", "all")
DEFAULT_SCAN_SCOPE = "new"
SCAN_GROUP_NAMES = ("added", "known", "review", "no_metadata", "errors")
SCAN_GROUPS = {
"added": "added",
"linked": "added",
"enriched": "added",
"unchanged": "known",
"skipped": "known", # "new" scope: known and resolved, not read again
"ambiguous": "review",
"conflict": "review",
"unresolved": "review",
@@ -240,6 +247,36 @@ class Job:
"log": list(self.log),
}
def report(self):
"""The downloadable report: the summary, every outcome group and the log.
Each group carries its exact count, its listed rows (at most
MAX_OUTCOME_ROWS) and whether the list is cut short. Rows and log lines
already hold root-relative paths only.
"""
summary = self.summary()
summary.pop("listed", None)
summary.pop("log_length", None)
return {
"report_version": 1,
"generated_at": self._clock(),
"job": summary,
"outcomes": {
group: {
"count": self.counters.get(group, len(rows)),
"rows": [dict(row) for row in rows],
"truncated": self.counters.get(group, len(rows)) > len(rows),
}
for group, rows in self.outcomes.items()
},
"log": list(self.log),
}
def report_filename(self):
"""The download name: prompt-manager-KIND-YYYYMMDD-HHMM.json (local time)."""
stamp = time.strftime("%Y%m%d-%H%M", time.localtime(self._clock()))
return f"prompt-manager-{self.kind}-{stamp}.json"
class JobRegistry:
"""Every job of this server session, running and recently finished."""
+48 -9
View File
@@ -63,16 +63,35 @@ function measureNotice(selector) {
};
}
/** D55: the concept's dialog with the production differences applied. */
/** A concept `.setting-row` switch: its frame, its label and hint, and its checkbox (not its place-dependent margin). */
function measureToggle(selector) {
const row = document.querySelector(selector);
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
return {
children: [...row.children].map(node => [node.localName, ...node.classList].join('.')),
frame: css(row, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom',
'borderTopWidth', 'borderTopStyle', 'borderTopColor'),
label: css(row.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
hint: css(row.querySelector('p'), 'fontSize', 'color', 'marginTop'),
box: css(row.querySelector('input[type="checkbox"]'), 'accentColor', 'width', 'height', 'appearance',
'backgroundColor', 'borderTopLeftRadius'),
role: row.querySelector('input').getAttribute('role'),
labelled: row.querySelector('label').htmlFor === row.querySelector('input').id,
};
}
/** D55 and D58: the concept's dialog with the production differences applied. */
function mapReviewConcept(reference) {
return {
...reference,
// No sample-plan note (prototype copy); the backup box sits before the actions.
// No sample-plan note (prototype copy); the D58 "All files" toggle follows the grid; the backup box sits
// before the actions.
body: [...reference.body.filter(item => item !== 'p.demo-note').slice(0, -1), 'div.notice', reference.body.at(-1)]
.map(item => item === 'div.meta-grid' ? 'dl.meta-grid' : item),
// A `dl`, as in the concept's maintenance plan; real sources and the fixed Activity-1 scope.
.flatMap(item => item === 'div.meta-grid' ? ['dl.meta-grid', 'div.setting-row'] : [item]),
// A `dl`, as in the concept's maintenance plan; real sources. The scope keeps the concept's
// "New and unresolved files" until "All files" is checked.
grid: { ...reference.grid, tag: 'dl', entries: reference.grid.entries.map(entry => ({ ...entry,
dd: { Sources: '2 output folders', Scope: 'All files' }[entry.dt] })) },
dd: { Sources: '2 output folders', Scope: 'New and unresolved files' }[entry.dt] })) },
actions: { ...reference.actions, buttons: reference.actions.buttons.map(button => button.text === 'Simulate scan'
? { ...button, text: 'Start scan' } : button) },
};
@@ -92,7 +111,8 @@ async function conceptReview(page, theme) {
await page.getByRole('combobox', { name: 'Theme', exact: true }).selectOption(theme);
await page.locator('#main').getByRole('button', { name: 'Manage sources', exact: true }).click();
await expect(page.locator('#main .notice')).toBeVisible();
return { review, notice: await page.evaluate(measureNotice, '#main .notice') };
return { review, notice: await page.evaluate(measureNotice, '#main .notice'),
toggle: await page.evaluate(measureToggle, '#main .setting-row:has(input[type="checkbox"])') };
}
async function productionReview(page, theme) {
@@ -103,7 +123,16 @@ async function productionReview(page, theme) {
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review metadata scan');
// The swapped dialog can open under the pointer that pressed Review scan; measure it at rest.
await page.mouse.move(0, 0);
return { review: await page.evaluate(measureReview), notice: await page.evaluate(measureNotice, 'dialog#modal[open] .notice') };
const review = await page.evaluate(measureReview);
const notice = await page.evaluate(measureNotice, 'dialog#modal[open] .notice');
// The concept's switch is on; compare ours on too, then turn it off again (the default).
const allFiles = page.getByRole('switch', { name: 'All files', exact: true });
await allFiles.check();
await page.mouse.move(0, 0);
const toggle = await page.evaluate(measureToggle, 'dialog#modal[open] .setting-row');
await allFiles.uncheck();
await page.mouse.move(0, 0);
return { review, notice, toggle };
}
for (const theme of THEMES) test(`review dialog matches the concept scan-review ${theme}`, async ({ page, request }) => {
@@ -114,19 +143,29 @@ for (const theme of THEMES) test(`review dialog matches the concept scan-review
const production = await productionReview(page, theme);
expect(diffInventory(mapReviewConcept(concept.review), production.review)).toEqual([]);
expect(diffInventory(concept.notice, production.notice)).toEqual([]);
expect(diffInventory(concept.toggle, production.toggle)).toEqual([]);
await expect(page.locator('dialog#modal[open] .setting-row > div > label')).toHaveText('All files');
await expect(page.locator('dialog#modal[open] .setting-row input')).not.toBeChecked();
await expect(page.getByRole('switch', { name: 'All files', exact: true })).toBeVisible();
await expect(page.locator('dialog#modal[open] .notice h3')).toHaveText('Back up the database first (recommended)');
await expect(page.locator('dialog#modal[open] .notice .row-actions button')).toHaveText(['Back up now']);
// Mutation check: one CSS change and one removed element are both noticed, then undone to an exact match again.
const undoCss = await mutate(page, { css: '#modal .meta-grid { gap: 3px; } #modal .notice { padding: 2px; }' });
const undoCss = await mutate(page, { css: '#modal .meta-grid { gap: 3px; } #modal .notice { padding: 2px; } #modal .setting-row { gap: 3px; }' });
const cssMutated = await page.evaluate(measureReview);
const noticeMutated = await page.evaluate(measureNotice, 'dialog#modal[open] .notice');
await page.getByRole('switch', { name: 'All files', exact: true }).check();
await page.mouse.move(0, 0);
const toggleMutated = await page.evaluate(measureToggle, 'dialog#modal[open] .setting-row');
await page.getByRole('switch', { name: 'All files', exact: true }).uncheck();
await page.mouse.move(0, 0);
expect(diffInventory(concept.toggle, toggleMutated).map(item => item.path)).toEqual(['frame.columnGap']);
expect(diffInventory(mapReviewConcept(concept.review), cssMutated).map(item => item.path)).toEqual(expect.arrayContaining(['grid.columnGap', 'grid.rowGap']));
expect(diffInventory(concept.notice, noticeMutated).map(item => item.path)).toEqual(['frame.paddingTop', 'frame.paddingLeft']);
await undoCss();
const undoDom = await mutate(page, { remove: ['#modal-body > p.small'] });
const domMutated = await page.evaluate(() => [...document.querySelectorAll('#modal-body > *')].map(node => node.localName));
expect(domMutated).toEqual(['dl', 'div', 'div']);
expect(domMutated).toEqual(['dl', 'div', 'div', 'div']);
await undoDom();
expect(diffInventory(mapReviewConcept(concept.review), await page.evaluate(measureReview))).toEqual([]);
});
+139 -2
View File
@@ -392,16 +392,69 @@ test('start scan creates a job, toasts, and opens its view', async ({ page, requ
await expect(scanLog(page).locator('pre')).toContainText('Scan finished');
});
// ── Task 4 (Activity-2): scan scope, new and unresolved files by default ─────────────────────
test('all files checkbox changes the scope line and the request', async ({ page }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const bodies = [];
page.on('request', item => {
if (item.method() === 'POST' && new URL(item.url()).pathname === '/prompt_manager/jobs/scan') bodies.push(item.postDataJSON());
});
const review = page.getByRole('dialog', { name: 'Review metadata scan', exact: true });
const scopeValue = review.locator('.meta-grid > div').filter({ has: page.locator('dt', { hasText: 'Scope' }) }).locator('dd');
// The concept's switch: a checkbox with role=switch.
const allFiles = review.getByRole('switch', { name: 'All files', exact: true });
const openReview = async () => {
await page.goto(`${workspace}#/images`);
await page.getByRole('button', { name: 'Import', exact: true }).click();
await page.getByRole('dialog', { name: 'Import & inspect', exact: true }).locator('.operation')
.filter({ hasText: 'Scan for metadata' }).getByRole('button', { name: 'Review scan', exact: true }).click();
await expect(review).toBeVisible();
};
// The default: the concept's "New and unresolved files", the box unchecked; the request asks for "new".
await openReview();
await expect(scopeValue).toHaveText('New and unresolved files');
await expect(allFiles).not.toBeChecked();
await expect(review.locator('.setting-row p')).toHaveText('Also read the files already linked to their prompt again.');
await expect(allFiles).toHaveAccessibleDescription('Also read the files already linked to their prompt again.');
// Checking and unchecking switches the Scope value both ways.
await allFiles.check();
await expect(scopeValue).toHaveText('All files');
await allFiles.uncheck();
await expect(scopeValue).toHaveText('New and unresolved files');
await review.getByRole('button', { name: 'Start scan', exact: true }).click();
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
await expect.poll(() => bodies.length).toBe(1);
expect(bodies[0]).toEqual({ scope: 'new' });
await expect(heading(page).locator('h1')).toHaveText('Scan for metadata');
// Wait for the first scan to end so the second is not refused as busy.
await scanLog(page).locator('summary').click();
await expect(scanLog(page).locator('pre')).toContainText('Scan finished', { timeout: 15000 });
await expect(scanLog(page).locator('pre')).toContainText('new and unresolved files');
// Checked: "All files", and the request asks for "all".
await openReview();
await allFiles.check();
await expect(scopeValue).toHaveText('All files');
await review.getByRole('button', { name: 'Start scan', exact: true }).click();
await expect.poll(() => bodies.length).toBe(2);
expect(bodies[1]).toEqual({ scope: 'all' });
await scanLog(page).locator('summary').click();
await expect(scanLog(page).locator('pre')).toContainText('Scan started · ', { timeout: 15000 });
await expect(scanLog(page).locator('pre')).toContainText('all files');
});
test('job view counters follow the mapping and update live', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 8240, counters: { added: 1420, known: 6511, review: 37, no_metadata: 254, errors: 18 } });
await openJob(page, job.id);
// The header: the title, the detail line, then Download report (Task 5) and Cancel scan (danger).
// The header: the title, the detail line, then Download report (enabled in Task 5) and Cancel scan (danger).
await expect(heading(page).locator('h1')).toHaveText('Scan for metadata');
await expect(heading(page).locator('p.muted')).toHaveText(/^Started \d\d:\d\d · 2 output folders · closing this view does not cancel it$/);
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report', 'Cancel scan']);
await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeDisabled();
await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeEnabled();
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).toHaveClass(/danger/);
// Progress: the bar, the server's line and the five tiles in order and colour.
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Processing file 8,240 of 19,622');
@@ -558,6 +611,19 @@ test('job view renders 200-row cap note', async ({ page, request }) => {
await step(request, job.id, { processed: 260, bulk: { group: 'added', count: 250 }, records: [
...Array.from({ length: 10 }, (_, index) => ({ group: 'no_metadata', path: `misc/plain_${index}.png`, detail: 'No prompt metadata in the file' })),
] });
// Every detail read of this job: in flight, and whether one has returned the full listing.
const detailUrl = `/prompt_manager/jobs/${job.id}`;
const isDetail = item => new URL(item.url()).pathname === detailUrl;
const detailReads = { inFlight: 0, full: false };
page.on('request', item => { if (isDetail(item)) detailReads.inFlight += 1; });
page.on('requestfailed', item => { if (isDetail(item)) detailReads.inFlight -= 1; });
page.on('response', async response => {
if (!isDetail(response)) return;
const body = await response.json().catch(() => null);
const outcomes = body?.job?.outcomes ?? {};
if (outcomes.added?.length === 200 && outcomes.no_metadata?.length === 10) detailReads.full = true;
detailReads.inFlight -= 1;
});
await openJob(page, job.id);
await expect(tiles(page).first().locator('dd')).toHaveText('250');
await expect(fileRows(page)).toHaveCount(200);
@@ -573,6 +639,9 @@ test('job view renders 200-row cap note', async ({ page, request }) => {
// Review M12: with the list capped and the log closed, more files only update the counters; the detail is read
// again only when the log opens with new lines.
// The reads from opening the view have settled: one returned the full listing and none is in flight. The view
// re-reads only while what the summary lists differs from the last read, so no further read is due.
await expect.poll(() => detailReads.full && detailReads.inFlight === 0).toBe(true);
const reads = [];
page.on('request', item => { if (new URL(item.url()).pathname === `/prompt_manager/jobs/${job.id}`) reads.push(Date.now()); });
await step(request, job.id, { processed: 300, bulk: { group: 'added', count: 40 } });
@@ -639,6 +708,8 @@ test('an unknown or restarted job view says so', async ({ page, request }) => {
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
await expect(heading(page).locator('p.muted')).toHaveText(LOST);
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).toHaveCount(0);
// A lost job has no report on the server (review M9): Download report is disabled rather than failing.
await expect(heading(page).getByRole('button', { name: 'Download report', exact: true })).toBeDisabled();
// What was read before the restart stays on the page.
await expect(fileRows(page)).toHaveCount(1);
await expect(pill(page)).toBeHidden();
@@ -662,3 +733,69 @@ test('a finished job the server pruned leaves its open view honest', async ({ pa
await expect(heading(page).locator('h1')).toBeFocused();
await expect(page).toHaveTitle('Activity · Prompt Manager');
});
// Activity-2 Task 5 (D59): Download report and the log's scroll position.
test('download report saves the job JSON', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await step(request, job.id, { processed: 2, records: [
{ group: 'added', path: 'sub/a.png', root: 'output', detail: 'Added' },
{ group: 'errors', path: 'broken/b.png', root: 'output', detail: 'Error' },
] });
await finish(request, job.id);
await openJob(page, job.id);
const button = heading(page).getByRole('button', { name: 'Download report', exact: true });
await expect(button).toBeEnabled();
const [download] = await Promise.all([page.waitForEvent('download'), button.click()]);
expect(download.suggestedFilename()).toMatch(/^prompt-manager-scan-\d{8}-\d{4}\.json$/);
const path = await download.path();
const report = JSON.parse(await (await import('node:fs/promises')).readFile(path, 'utf8'));
expect(report.job.id).toBe(job.id);
expect(report.outcomes.added.rows.map(row => row.path)).toEqual(['sub/a.png']);
expect(report.outcomes.errors.truncated).toBe(false);
expect(report.log.length).toBeGreaterThan(0);
await expect(toast(page)).toHaveText('Job report downloaded.');
});
test('log keeps the reader\'s scroll position', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
const more = (from, count) => Array.from({ length: count }, (_, index) => ({ group: 'known', path: `log/f_${from + index}.png`,
root: 'output', detail: 'Known' }));
await step(request, job.id, { processed: 60, records: more(0, 60) });
await openJob(page, job.id);
await scanLog(page).locator('summary').click();
const pre = scanLog(page).locator('pre');
await expect(pre).toContainText('log/f_59.png');
const metrics = () => pre.evaluate(node => ({ top: node.scrollTop, max: node.scrollHeight - node.clientHeight }));
// Opening starts at the newest line.
await expect.poll(async () => { const m = await metrics(); return m.max > 0 && m.top >= m.max - 2; }).toBe(true);
// Pinned to the bottom: new lines are followed.
await step(request, job.id, { processed: 90, records: more(60, 30) });
await expect(pre).toContainText('log/f_89.png');
await expect.poll(async () => { const m = await metrics(); return m.top >= m.max - 2; }).toBe(true);
// Scrolled up: the position stays while more lines arrive.
await pre.evaluate(node => { node.scrollTop = 40; });
await step(request, job.id, { processed: 120, records: more(90, 30) });
await expect(pre).toContainText('log/f_119.png');
expect((await metrics()).top).toBe(40);
// Closed and opened again (review M6): every open starts at the newest line, also after lines arrived meanwhile.
await scanLog(page).locator('summary').click();
await expect(scanLog(page)).not.toHaveAttribute('open', '');
await step(request, job.id, { processed: 150, records: more(120, 30) });
await scanLog(page).locator('summary').click();
await expect(pre).toContainText('log/f_149.png');
await expect.poll(async () => { const m = await metrics(); return m.max > 0 && m.top >= m.max - 2; }).toBe(true);
});
test('a failed report download says so and leaves the button usable', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
const job = await startJob(request);
await finish(request, job.id);
await openJob(page, job.id);
await page.route(`**/prompt_manager/jobs/${job.id}/report`, route => route.fulfill({ status: 500, body: 'no' }));
const button = heading(page).getByRole('button', { name: 'Download report', exact: true });
await button.click();
await expect(toast(page)).toHaveText('The report could not be downloaded. Try again.');
await expect(button).not.toHaveAttribute('aria-disabled', 'true');
});
+1 -1
View File
@@ -2060,7 +2060,7 @@ test('scan for metadata opens the review dialog from Import', async ({ page, req
await openScanReview(page);
const dialog = scanReview(page);
await expect(dialog.locator('dl.meta-grid > div > dt')).toHaveText(['Sources', 'Scope']);
await expect(dialog.locator('dl.meta-grid > div > dd')).toHaveText(['2 output folders', 'All files']);
await expect(dialog.locator('dl.meta-grid > div > dd')).toHaveText(['2 output folders', 'New and unresolved files']);
await expect(dialog.locator('#modal-body > p.small')).toHaveText('Enrich positive / negative associations, reuse canonical text and report unchanged files. Do not inflate run counts or recreate existing thumbnails.');
await expect(dialog.locator('.notice h3')).toHaveText('Back up the database first (recommended)');
await expect(dialog.locator('.dialog-actions button')).toHaveText(['Cancel', 'Start scan']);
+1 -1
View File
@@ -129,7 +129,7 @@ def register_activity_commands(app, routes, api):
str(data.get("title", "Scan for metadata")),
record_changing=True,
scope={
"mode": "all",
"files": str(data.get("files", "new")),
"output_folders": count(data.get("output_folders"), 2),
},
runner=fake_runner(ending),
+1 -1
View File
@@ -11,7 +11,7 @@ const ID_C = 'c'.repeat(32);
function summary(overrides = {}) {
return { id: ID_A, kind: 'scan', title: 'Scan for metadata', state: 'running', started_at: 1000, finished_at: null,
progress: 0, status_line: '', processed: 0, total: 0,
counters: { added: 0, known: 0, review: 0, no_metadata: 0, errors: 0 }, scope: { mode: 'all', output_folders: 2 },
counters: { added: 0, known: 0, review: 0, no_metadata: 0, errors: 0 }, scope: { files: 'new', output_folders: 2 },
...overrides };
}
+66
View File
@@ -2094,6 +2094,72 @@ class TestActivityJobs(AdminAPITestCase):
(legacy["processed"], legacy["found"], legacy["failed"]), (6, 4, 1)
)
async def test_report_is_attachment_without_absolute_paths(self):
"""Task 5: the report downloads as JSON, whole and without server paths."""
import re
from py.api.jobs import MAX_OUTCOME_ROWS
async def runner(job):
absolute = f"{self.tmpdir}/output/deep/abs.png"
job.record(
"added",
{"path": absolute, "root": "output", "root_index": 0, "detail": "ok"},
f"+ {absolute} · ok",
)
for index in range(MAX_OUTCOME_ROWS + 5):
job.record(
"known",
{"path": f"k/{index}.png", "root": "output", "detail": "x"},
f"= k/{index}.png",
)
job.update(processed=MAX_OUTCOME_ROWS + 6, total=MAX_OUTCOME_ROWS + 6)
job = self.api._jobs.start(
"scan",
"Scan for metadata",
record_changing=True,
scope={"files": "new", "output_folders": 1},
runner=runner,
groups=("added", "known", "review", "no_metadata", "errors"),
)
await asyncio.wait_for(job.task, 10)
resp = await self.client.request("GET", f"/prompt_manager/jobs/{job.id}/report")
text = await resp.text()
self.assertEqual(resp.status, 200)
self.assertIn("application/json", resp.headers["Content-Type"])
disposition = resp.headers["Content-Disposition"]
self.assertRegex(
disposition,
r'^attachment; filename="prompt-manager-scan-\d{8}-\d{4}\.json"$',
)
self.assertNotIn(self.tmpdir, text)
self.assertIsNone(
re.search(r"[A-Za-z]:[\\/]|(?<![\w.])/(?:tmp|home|var)/", text)
)
report = json.loads(text)
self.assertEqual(report["job"]["id"], job.id)
self.assertEqual(report["job"]["scope"]["files"], "new")
self.assertEqual(report["job"]["counters"]["known"], MAX_OUTCOME_ROWS + 5)
added = report["outcomes"]["added"]
self.assertEqual(
(added["count"], len(added["rows"]), added["truncated"]), (1, 1, False)
)
self.assertEqual(added["rows"][0]["path"], "abs.png")
known = report["outcomes"]["known"]
self.assertEqual(
(known["count"], len(known["rows"]), known["truncated"]),
(MAX_OUTCOME_ROWS + 5, MAX_OUTCOME_ROWS, True),
)
self.assertEqual(report["outcomes"]["errors"]["rows"], [])
self.assertFalse(report["outcomes"]["errors"]["truncated"])
self.assertEqual(report["log"], job.detail()["log"])
self.assertIn("generated_at", report)
resp = await self.client.request("GET", "/prompt_manager/jobs/nope/report")
self.assertEqual(resp.status, 404)
async def test_cancel_after_first_batch_keeps_partial_counts(self):
"""Review M7: the real scan stops at the batch boundary, keeping batch 1."""
import threading
+13 -3
View File
@@ -13,12 +13,14 @@ import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from py.api.jobs import ( # noqa: E402
DEFAULT_SCAN_SCOPE,
LOG_CAP_LINE,
MAX_FINISHED_JOBS,
MAX_LOG_LINES,
MAX_OUTCOME_ROWS,
SCAN_GROUP_NAMES,
SCAN_GROUPS,
SCAN_SCOPES,
Job,
JobBusy,
JobRegistry,
@@ -155,7 +157,7 @@ class TestJobRegistry(unittest.IsolatedAsyncioTestCase):
"scan",
"Scan for metadata",
record_changing=True,
scope={"mode": "all"},
scope={"files": "new", "output_folders": 2},
runner=_gated_runner(gate),
)
self.assertEqual(job.state, "running")
@@ -181,7 +183,7 @@ class TestJobRegistry(unittest.IsolatedAsyncioTestCase):
"log_length",
},
)
self.assertEqual(summary["scope"], {"mode": "all"})
self.assertEqual(summary["scope"], {"files": "new", "output_folders": 2})
# What the detail would list, so a view reads it only when that changed.
self.assertEqual((summary["listed"], summary["log_length"]), ({}, 0))
self.assertIsNone(summary["finished_at"])
@@ -371,7 +373,12 @@ class TestJobOutcomesAndLog(unittest.TestCase):
job.record("bogus", {"path": "a.png", "root": "output"}, "x")
def test_detail_and_progress(self):
job = Job("scan", "Scan", scope={"mode": "all"}, groups=SCAN_GROUP_NAMES)
job = Job(
"scan",
"Scan",
scope={"files": "new", "output_folders": 2},
groups=SCAN_GROUP_NAMES,
)
job.update(progress=150, status_line="Processing file 3 of 4", processed=3)
job.update(total=4)
job.record("known", {"path": "a.png", "root": "output"}, "= a.png")
@@ -395,12 +402,15 @@ class TestJobOutcomesAndLog(unittest.TestCase):
"linked": "added",
"enriched": "added",
"unchanged": "known",
# Activity-2: a file the "new" scope skipped is Already known.
"skipped": "known",
"ambiguous": "review",
"conflict": "review",
"unresolved": "review",
"failed": "errors",
},
)
self.assertEqual((SCAN_SCOPES, DEFAULT_SCAN_SCOPE), (("new", "all"), "new"))
self.assertEqual(
SCAN_GROUP_NAMES, ("added", "known", "review", "no_metadata", "errors")
)
+390
View File
@@ -0,0 +1,390 @@
"""Scan scope (#249 Activity-2 Task 4): new and unresolved files by default.
A "Scan for metadata" job started without ``scope`` (or with ``"new"``) skips
the files the Library already links with a resolved link, before their
metadata is read, and counts them as Already known. ``"all"`` reads every file
again, as the legacy ``/prompt_manager/scan`` route always does.
"""
import asyncio
import json
import os
import sqlite3
import unittest
from unittest.mock import patch
try:
from tests.test_api_admin import AdminAPITestCase, _sse_payloads, _write_png
except ImportError: # discovered with tests/ as the top-level directory
from test_api_admin import AdminAPITestCase, _sse_payloads, _write_png
from database import operations # noqa: E402
from database.pair_ingestion import ingest_resolved_pair # noqa: E402
from py.config import GalleryConfig # noqa: E402
from utils.prompt_pair_resolver import resolve_prompt_pair # noqa: E402
SKIPPED_DETAIL = "Skipped · already linked"
ON_DISK = object() # _link_resolved: store the file's size on disk
def _graph(text):
return {
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": text}},
"2": {"class_type": "CLIPTextEncode", "inputs": {"text": "bad"}},
"3": {
"class_type": "KSampler",
"inputs": {"positive": ["1", 0], "negative": ["2", 0]},
},
}
class ScanScopeTestCase(AdminAPITestCase):
"""Helpers: link files the way the scan and the monitor do, spy on reads."""
def _link_resolved(self, path, text, size=ON_DISK):
"""``path`` linked with a resolved pair link (what a resolved scan writes).
The row stores ``size`` as its file size: by default the file's size on
disk (None when the file does not exist).
"""
if size is ON_DISK:
size = os.path.getsize(path) if os.path.exists(path) else None
metadata = {"prompt": _graph(text)}
if size is not None:
metadata["file_info"] = {"size": size}
result = ingest_resolved_pair(
self.api.db, str(path), metadata, resolve_prompt_pair(metadata)
)
self.assertEqual(result["status"], "added")
def _link_without_pair(self, path, text):
"""``path`` linked to a prompt with no pair link (fallback / FIFO link)."""
prompt_id = self.api.db.save_prompt(text, None, [], None, None)
self.assertTrue(self.api.db.link_image_to_prompt(prompt_id, str(path)))
return prompt_id
def _spy_reads(self):
reads = []
original = self.api._extract_comfyui_metadata
def spy(path):
reads.append(os.path.basename(path))
return original(path)
self.api._extract_comfyui_metadata = spy
return reads
async def _scan(self, body=None):
kwargs = {} if body is None else {"json": body}
resp = await self.client.request("POST", "/prompt_manager/jobs/scan", **kwargs)
data = await resp.json()
self.assertEqual(resp.status, 202, data)
await asyncio.wait_for(self.api._scan_job.task, 10)
detail = await self.client.request(
"GET", f"/prompt_manager/jobs/{data['job']['id']}"
)
return (await detail.json())["job"]
class TestScanScope(ScanScopeTestCase):
async def test_new_scope_skips_known_resolved_files_without_reading_them(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
_write_png(self.output_dir / "new.png", prompt_text="a blue circle")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(reads, ["new.png"])
self.assertEqual(job["scope"]["files"], "new")
self.assertEqual(job["counters"]["known"], 1)
self.assertEqual(job["counters"]["added"], 1)
self.assertEqual(job["processed"], 2)
self.assertEqual(
job["outcomes"]["known"],
[
{
"path": "known.png",
"root": "output",
"root_index": 0,
"detail": SKIPPED_DETAIL,
}
],
)
self.assertIn(f"= output/known.png · {SKIPPED_DETAIL}", "\n".join(job["log"]))
self.assertTrue(
job["log"][0].endswith(
"Scan started · 1 output folder · new and unresolved files"
)
)
legacy = self.api._scan_job.last_event
self.assertEqual((legacy["processed"], legacy["unchanged"]), (2, 1))
self.assertTrue(job["status_line"].startswith("Processing file 2 of 2"))
async def test_whole_known_batch_skips_the_executor_reads(self):
from py.api import admin
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
for name in ("a.png", "b.png"):
_write_png(self.output_dir / name, prompt_text=f"prompt {name}")
self._link_resolved(self.output_dir / name, f"prompt {name}")
_write_png(self.output_dir / "c.png", prompt_text="prompt c")
reads = self._spy_reads()
with patch.object(admin, "SCAN_BATCH_SIZE", 1):
job = await self._scan({"scope": "new"})
self.assertEqual(reads, ["c.png"])
self.assertEqual(job["counters"]["known"], 2)
self.assertEqual(job["counters"]["added"], 1)
async def test_unresolved_files_are_rescanned_in_new_scope(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
names = ("fallback.png", "conflict.png", "unlinked.png")
for name in names:
_write_png(self.output_dir / name, prompt_text=f"text of {name}")
# Linked without a resolved pair link (the monitor's fallback / FIFO link).
self._link_without_pair(self.output_dir / "fallback.png", "text of fallback")
# Resolved once, then also linked to a second prompt: ambiguous.
conflict = self.output_dir / "conflict.png"
self._link_resolved(conflict, "text of conflict.png")
self._link_without_pair(conflict, "another prompt")
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(sorted(reads), sorted(names))
self.assertNotIn(SKIPPED_DETAIL, json.dumps(job["outcomes"]))
async def test_all_scope_reads_everything(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
job = await self._scan({"scope": "all"})
self.assertEqual(reads, ["known.png"])
self.assertEqual(job["scope"]["files"], "all")
self.assertEqual(job["counters"]["known"], 1)
self.assertNotEqual(job["outcomes"]["known"][0]["detail"], SKIPPED_DETAIL)
self.assertTrue(
job["log"][0].endswith("Scan started · 1 output folder · all files")
)
async def test_legacy_scan_route_reads_everything(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
resp = await self.client.request("POST", "/prompt_manager/scan", data="{}")
payloads = _sse_payloads(await resp.text())
self.assertEqual(reads, ["known.png"])
self.assertEqual(self.api._scan_job.scope["files"], "all")
self.assertEqual(payloads[-1]["type"], "complete")
self.assertEqual(payloads[-1]["unchanged"], 1)
async def test_file_replaced_at_a_known_path_is_reread_and_needs_review(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
path = self.output_dir / "out_00012.png"
_write_png(path, prompt_text="the first prompt")
self._link_resolved(path, "the first prompt")
before = os.path.getsize(path)
# ComfyUI reused the name for a new image with another prompt.
_write_png(path, prompt_text="a much longer second prompt for the same name")
self.assertNotEqual(os.path.getsize(path), before)
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(reads, ["out_00012.png"])
self.assertEqual(job["counters"]["review"], 1)
self.assertEqual(job["counters"]["known"], 0)
self.assertIn("different prompt", job["outcomes"]["review"][0]["detail"])
async def test_scanned_files_store_their_size_so_the_next_new_scan_skips(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
_write_png(self.output_dir / "fresh.png", prompt_text="a fresh prompt")
first = await self._scan()
self.assertEqual(first["counters"]["added"], 1)
reads = self._spy_reads()
second = await self._scan()
self.assertEqual(reads, [])
self.assertEqual(second["outcomes"]["known"][0]["detail"], SKIPPED_DETAIL)
async def test_stored_null_size_is_reread(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
path = self.output_dir / "old.png"
_write_png(path, prompt_text="an old prompt")
self._link_resolved(path, "an old prompt", size=None)
reads = self._spy_reads()
job = await self._scan()
self.assertEqual(reads, ["old.png"])
# The read stores the size it was missing, so the file is known next time.
self.assertEqual(job["counters"]["added"], 1)
self.assertEqual(
self.api.db.get_known_resolved_paths({str(path): os.path.getsize(path)}),
{str(path)},
)
async def test_known_set_failure_falls_back_to_reading_every_file(self):
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.output_dir)]
known = self.output_dir / "known.png"
_write_png(known, prompt_text="a red square")
self._link_resolved(known, "a red square")
reads = self._spy_reads()
def broken(files):
raise sqlite3.OperationalError("database is locked")
self.api.db.get_known_resolved_paths = broken
with self.assertLogs("prompt_manager.api", level="WARNING"):
job = await self._scan()
self.assertEqual(job["state"], "finished")
self.assertEqual(reads, ["known.png"])
self.assertIn(
"Known files could not be read · reading every file", "\n".join(job["log"])
)
async def test_empty_read_body_asks_for_the_default_scope(self):
"""A chunked POST with no bytes can read a body: still the default."""
from py.api.job_routes import _scan_scope
class Request:
can_read_body = True
def __init__(self, text):
self._text = text
async def text(self):
return self._text
self.assertEqual(await _scan_scope(Request("")), "new")
self.assertEqual(await _scan_scope(Request(" \n")), "new")
self.assertEqual(await _scan_scope(Request('{"scope": "all"}')), "all")
self.assertIsNone(await _scan_scope(Request("{")))
async def test_invalid_scope_is_refused(self):
for body in ({"scope": "everything"}, {"scope": 1}, ["new"]):
with self.subTest(body=body):
resp = await self.client.request(
"POST", "/prompt_manager/jobs/scan", json=body
)
self.assertEqual(resp.status, 400)
data = await resp.json()
self.assertFalse(data["success"])
resp = await self.client.request(
"POST", "/prompt_manager/jobs/scan", data="not json"
)
self.assertEqual(resp.status, 400)
self.assertIsNone(self.api._scan_job)
class TestKnownResolvedPaths(ScanScopeTestCase):
"""``PromptDatabase.get_known_resolved_paths``: one streamed query."""
async def test_returns_the_given_paths_that_are_known_and_resolved(self):
known, fallback = self.output_dir / "known.png", self.output_dir / "fb.png"
self._link_resolved(known, "a red square", size=10)
self._link_without_pair(fallback, "a fallback")
asked = {str(known): 10, str(fallback): 10, str(self.output_dir / "n.png"): 10}
self.assertEqual(self.api.db.get_known_resolved_paths(asked), {str(known)})
self.assertEqual(self.api.db.get_known_resolved_paths({}), set())
async def test_size_must_match_and_a_stored_null_size_is_not_known(self):
same, other = self.output_dir / "same.png", self.output_dir / "other.png"
unknown = self.output_dir / "unknown.png"
self._link_resolved(same, "same size", size=10)
self._link_resolved(other, "other size", size=10)
self._link_resolved(unknown, "no stored size", size=None)
asked = {str(same): 10, str(other): 11, str(unknown): 10}
self.assertEqual(self.api.db.get_known_resolved_paths(asked), {str(same)})
# A file the walk could not stat is not known either.
self.assertEqual(self.api.db.get_known_resolved_paths({str(same): None}), set())
async def test_row_without_file_path_matches_through_image_path(self):
known = self.output_dir / "legacy.png"
self._link_resolved(known, "a legacy row", size=10)
with sqlite3.connect(self.api.db.model.db_path) as conn:
conn.execute("UPDATE generated_images SET file_path=NULL")
conn.close()
found = self.api.db.get_known_resolved_paths({str(known): 10})
self.assertEqual(found, {str(known)})
async def test_stored_spelling_is_normalized_like_the_scanned_path(self):
known = self.output_dir / "sub" / "known.png"
self._link_resolved(known, "a red square", size=10)
spelled = os.path.join(str(self.output_dir), "sub", "..", "sub", "known.png")
with sqlite3.connect(self.api.db.model.db_path) as conn:
conn.execute("UPDATE generated_images SET file_path=?", (spelled,))
conn.close()
found = self.api.db.get_known_resolved_paths({str(known): 10})
self.assertEqual(found, {str(known)})
async def test_one_query_streamed_in_small_fetches(self):
paths = [self.output_dir / f"{index}.png" for index in range(5)]
for path in paths:
self._link_resolved(path, f"prompt {path.name}", size=10)
original = self.api.db.model.get_connection
opened = []
def counting():
opened.append(1)
return original()
self.api.db.model.get_connection = counting
with patch.object(operations, "KNOWN_PATHS_FETCH_SIZE", 2):
found = self.api.db.get_known_resolved_paths({str(p): 10 for p in paths})
self.assertEqual(found, {str(p) for p in paths})
self.assertEqual(len(opened), 1)
async def test_windows_path_case_matches_known_set(self):
stored = self.output_dir / "Sub" / "Known.PNG"
self._link_resolved(stored, "a red square", size=10)
# A row written before paths were case-folded keeps its own spelling.
with sqlite3.connect(self.api.db.model.db_path) as conn:
conn.execute(
"UPDATE generated_images SET file_path=?", (str(stored).upper(),)
)
conn.close()
asked = str(self.output_dir / "sub" / "known.png")
with patch("os.path.normcase", lambda path: path.lower()):
self.assertEqual(self.api.db.get_known_resolved_paths({asked: 10}), {asked})
if os.path.normcase("A") == "A": # a case-sensitive platform
self.assertEqual(self.api.db.get_known_resolved_paths({asked: 10}), set())
class TestSkippedOutcome(unittest.TestCase):
def test_skipped_file_is_already_known_with_its_own_detail(self):
from py.api.admin import _scan_outcome
root = os.path.abspath("out")
group, row, line = _scan_outcome(
os.path.join(root, "a", "b.png"), [(root, "output", 0)], "skipped"
)
self.assertEqual(group, "known")
self.assertEqual(row["detail"], SKIPPED_DETAIL)
self.assertEqual(row["path"], "a/b.png")
self.assertEqual(line, f"= output/a/b.png · {SKIPPED_DETAIL}")
if __name__ == "__main__":
unittest.main()
+28 -4
View File
@@ -3,7 +3,7 @@
* concept's classes.
*
* - Header: the page heading names the job; its line is the Activity item's (D54), and its actions are Download report
* (disabled until Activity-2) and, while the job runs, "Cancel scan" (danger).
* (the job's JSON report, D59) and, while the job runs, "Cancel scan" (danger).
* - Progress (`section.surface`): `<progress>`, the server's "Processing file N of M · about X min left", and the
* outcome counters as `dl.stat-row` tiles: Added to Library (green), Already known, Needs review (warning), No usable
* metadata, Errors (red).
@@ -21,6 +21,7 @@ import { showWorkspaceToast } from './toast.mjs';
import { noun } from './plural.mjs';
import { parseJob, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
import { jobDetail, CANCEL_FAILED_TOAST } from './activity.mjs';
import { downloadJobReport } from './activity-scan.mjs';
import { createMediaViewer } from './media-viewer.mjs';
import { createInspectionController } from './inspection-state.mjs';
import { createPromptSender } from './prompt-sender.mjs';
@@ -122,7 +123,6 @@ function section(title, id) {
export function mountActivityJobView(main, { store, scope, api, jobId }) {
const parts = pageHeading(main);
const download = createButton('Download report');
download.disabled = true;
const cancel = createButton('Cancel scan', 'danger');
const progressPart = section('Progress', 'job-progress-title');
@@ -271,6 +271,10 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
document.title = `${job.title} · Prompt Manager`;
parts.description.textContent = jobDetail(job);
cancel.textContent = scan ? 'Cancel scan' : 'Cancel job';
// A job lost with the server has no report there (review M9).
const lost = job.state === 'lost';
if (lost && document.activeElement === download) parts.h1.focus();
download.disabled = lost;
if (job.state === 'running') {
if (!cancel.isConnected) parts.actions.append(cancel);
} else if (cancel.isConnected) {
@@ -290,9 +294,11 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
logLabel.textContent = `● ${scan ? 'Scan' : 'Job'} log`;
const text = detail?.log.length ? detail.log.join('\n') : 'No log lines yet.';
if (pre.textContent !== text) {
// Replacing the text clamps the scroll position, so keep it: pinned readers follow the end, others stay.
const { scrollTop } = pre;
const atEnd = pre.scrollTop + pre.clientHeight >= pre.scrollHeight - 2;
pre.textContent = text;
if (atEnd) pre.scrollTop = pre.scrollHeight;
pre.scrollTop = atEnd ? pre.scrollHeight : scrollTop;
}
renderRows(job);
}
@@ -323,7 +329,12 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
reading = false;
}
render();
if (readAgain) { readAgain = false; scheduleRead(); }
// Read once more only if the summary moved past what this read returned (review: no redundant re-read).
if (readAgain) {
readAgain = false;
const job = store.find(jobId);
if (job && job.state !== 'lost' && detailChanged(job)) scheduleRead();
}
}
/**
@@ -347,10 +358,23 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
}
scope.listen(log, 'toggle', () => {
// Every open starts at the newest line: while closed the log has no layout box (Firefox, WebKit), so a kept
// position would be lost anyway (review M6).
if (log.open) pre.scrollTop = pre.scrollHeight;
const job = store.find(jobId);
if (log.open && job && job.state !== 'lost' && detailChanged(job)) scheduleRead();
});
scope.listen(download, 'click', async () => {
if (download.disabled || download.getAttribute('aria-disabled') === 'true') return;
download.setAttribute('aria-disabled', 'true');
try {
// A view left while the report downloads shows no toast (review M10).
await downloadJobReport(jobId, { signal: scope.signal });
} finally {
download.removeAttribute('aria-disabled');
}
});
scope.listen(cancel, 'click', async () => {
if (cancel.getAttribute('aria-disabled') === 'true') return;
cancel.setAttribute('aria-disabled', 'true');
+49 -16
View File
@@ -1,7 +1,8 @@
/**
* Scan for metadata (#249 Activity-1, D55): Images → Import → "Scan for metadata" → Review scan opens the concept's
* "Review metadata scan". Its backup box downloads the existing `/prompt_manager/backup` copy and toasts the outcome;
* Start scan starts the session job through the Activity store, which toasts "Scan started.", and opens the job's view.
* Start scan starts the session job through the Activity store over the chosen scope (new and unresolved files, or
* all files with the D58 checkbox), which toasts "Scan started.", and opens the job's view.
* A start refused because another record-changing job runs toasts "Another job is running: <title>." with a link to
* that job (D57).
*/
@@ -12,13 +13,29 @@ import { navigateTo } from './state.mjs';
export const BACKUP_URL = '/prompt_manager/backup';
export const BACKUP_DONE = 'Database backup downloaded.';
export const BACKUP_FAILED = 'The database could not be backed up. Try again.';
export const REPORT_DONE = 'Job report downloaded.';
export const REPORT_FAILED = 'The report could not be downloaded. Try again.';
const JOBS_REPORT_BASE = '/prompt_manager/jobs';
export const START_FAILED = 'The scan could not start. Try again.';
const FALLBACK_NAME = 'prompts_backup.db';
const REPORT_FALLBACK_NAME = 'prompt-manager-report.json';
/** The download name from a Content-Disposition header, or the fallback. */
export function backupFilename(disposition) {
/** The download name from a Content-Disposition header, or `fallback`. */
export function backupFilename(disposition, fallback = FALLBACK_NAME) {
const match = /filename="?([^";]+)"?/i.exec(disposition || '');
return match ? match[1] : FALLBACK_NAME;
return match ? match[1] : fallback;
}
/** Save a fetched response through a temporary link, named by `filename`. */
function saveResponse(blob, filename) {
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = filename;
document.body.append(link);
link.click();
link.remove();
setTimeout(() => URL.revokeObjectURL(url), 1000);
}
/** Download the prompts database copy the legacy admin page also uses; toasts the outcome. */
@@ -27,14 +44,7 @@ export async function downloadDatabaseBackup({ fetchUrl = window.ApiClient.fetch
const response = await fetchUrl(BACKUP_URL);
if (!response.ok) throw new Error(`Backup failed with ${response.status}`);
const blob = await response.blob();
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = backupFilename(response.headers.get('Content-Disposition'));
document.body.append(link);
link.click();
link.remove();
setTimeout(() => URL.revokeObjectURL(url), 1000);
saveResponse(blob, backupFilename(response.headers.get('Content-Disposition')));
notify(BACKUP_DONE);
return true;
} catch {
@@ -43,16 +53,39 @@ export async function downloadDatabaseBackup({ fetchUrl = window.ApiClient.fetch
}
}
/**
* Download a job's JSON report (Activity-2, D59) through the api-client URL, so a proxy mount works; toasts the outcome
* unless `signal` (the view's scope) was aborted meanwhile.
*/
export async function downloadJobReport(jobId, { fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast,
signal = null } = {}) {
try {
const response = await fetchUrl(`${JOBS_REPORT_BASE}/${encodeURIComponent(jobId)}/report`);
if (!response.ok) throw new Error(`Report failed with ${response.status}`);
const blob = await response.blob();
saveResponse(blob, backupFilename(response.headers.get('Content-Disposition'), REPORT_FALLBACK_NAME));
// The file is saved either way; a view left meanwhile (`signal` aborted) shows no toast.
if (!signal?.aborted) notify(REPORT_DONE);
return true;
} catch {
if (!signal?.aborted) notify(REPORT_FAILED);
return false;
}
}
/** The toast for a refused start, linking the job that holds the slot. */
export function busyToast(running) {
if (!running) return 'Another job is running. Try again when it ends.';
return ['Another job is running: ', { text: running.title, href: `#/activity/${running.id}` }, '.'];
}
/** Start the scan job and open its view; a refused or failed start says why and stays on the page. */
export async function startMetadataScan({ store, scope, notify = showWorkspaceToast, navigate = navigateTo }) {
/**
* Start the scan job over `files` ('new': new and unresolved files, or 'all') and open its view; a refused or failed
* start says why and stays on the page.
*/
export async function startMetadataScan({ store, scope, files = 'new', notify = showWorkspaceToast, navigate = navigateTo }) {
try {
const result = await store.start('scan');
const result = await store.start('scan', { scope: files });
if (scope.signal.aborted) return;
if (result.busy !== undefined) {
notify(busyToast(result.busy));
@@ -68,5 +101,5 @@ export async function startMetadataScan({ store, scope, notify = showWorkspaceTo
export function openMetadataScan({ scope, store, sources, trigger = null }) {
openScanReviewDialog({ scope, sources, trigger,
onBackup: () => downloadDatabaseBackup(),
onStart: () => startMetadataScan({ store, scope }) });
onStart: ({ scope: files }) => startMetadataScan({ store, scope, files }) });
}
+31 -8
View File
@@ -20,7 +20,7 @@
*
* Every opener takes the page `scope`: disposing it closes and removes the shell, so a dialog never outlives its page.
*/
import { createButton } from '../shared/toolkit/core.js';
import { createButton, createSwitch } from '../shared/toolkit/core.js';
const FIELD_LABELS = Object.freeze({ source: 'Source', folder: 'Folder', media: 'Media', tags: 'Tags' });
const FIELD_ORDER = Object.keys(FIELD_LABELS);
@@ -342,22 +342,44 @@ export const SCAN_REVIEW_SENTENCE = 'Enrich positive / negative associations, re
const BACKUP_LABEL = 'Back up now';
const BACKUP_BUSY_LABEL = 'Backing up…';
export const SCAN_SCOPE_LABELS = { new: 'New and unresolved files', all: 'All files' };
export const SCAN_ALL_FILES_HINT = 'Also read the files already linked to their prompt again.';
/**
* The concept's "Review metadata scan" (`scan-review`, D55). `sources` is the number of output folders the scan reads;
* the scope is fixed to "All files" in Activity-1. `onBackup` runs the database backup (the button is aria-disabled
* and reads "Backing up…" until it settles, and the dialog stays open); Start scan closes the dialog, then `onStart`.
* The concept's "Review metadata scan" (`scan-review`, D55). `sources` is the number of output folders the scan reads.
* The scope is the concept's "New and unresolved files"; the "All files" checkbox (D58, the concept's `.setting-row`
* switch, unchecked) switches the Scope value to "All files". `onBackup` runs the database backup (the button is
* aria-disabled and reads "Backing up…" until it settles, and the dialog stays open); Start scan closes the dialog, then
* calls `onStart({scope: 'new' | 'all'})`.
*/
export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigger = null }) {
requireScope(scope);
requireCallbacks({ onBackup, onStart });
if (!Number.isSafeInteger(sources) || sources < 0) throw new TypeError('The scan review needs its source count');
const grid = el('dl', undefined, 'meta-grid');
const values = {};
for (const [term, value] of [['Sources', `${sources.toLocaleString('en-US')} output ${sources === 1 ? 'folder' : 'folders'}`],
['Scope', 'All files']]) {
['Scope', SCAN_SCOPE_LABELS.new]]) {
const entry = el('div');
entry.append(el('dt', term), el('dd', value));
values[term] = el('dd', value);
entry.append(el('dt', term), values[term]);
grid.append(entry);
}
// The concept's System toggle: a `.setting-row` whose checkbox is the Toolkit switch (concept `hydrateSwitches`).
const toggle = createSwitch('', '', 'scan-all-files');
const allFiles = toggle.querySelector('input');
allFiles.checked = false;
const option = el('div', undefined, 'setting-row');
const optionText = el('div');
const optionLabel = el('label', SCAN_SCOPE_LABELS.all);
optionLabel.htmlFor = allFiles.id;
const hint = el('p', SCAN_ALL_FILES_HINT);
hint.id = 'scan-all-files-hint';
allFiles.setAttribute('aria-describedby', hint.id);
optionText.append(optionLabel, hint);
option.append(optionText, toggle);
const chosenScope = () => (allFiles.checked ? 'all' : 'new');
allFiles.addEventListener('change', () => { values.Scope.textContent = SCAN_SCOPE_LABELS[chosenScope()]; });
const notice = el('div', undefined, 'notice');
const copy = el('div');
copy.append(el('h3', 'Back up the database first (recommended)'),
@@ -381,10 +403,11 @@ export function openScanReviewDialog({ scope, sources, onBackup, onStart, trigge
const start = createButton('Start scan', 'ca-primary');
cancel.addEventListener('click', () => finish());
start.addEventListener('click', () => {
const files = chosenScope();
finish();
onStart();
onStart({ scope: files });
});
const actions = el('div', undefined, 'dialog-actions');
actions.append(cancel, start);
show('Review metadata scan', [grid, el('p', SCAN_REVIEW_SENTENCE, 'small'), notice, actions], { scope, trigger });
show('Review metadata scan', [grid, option, el('p', SCAN_REVIEW_SENTENCE, 'small'), notice, actions], { scope, trigger });
}