Files
Vito 69de20a427 feat(system): Sources & scans (#309)
* feat(system): report configured, effective and watched sources

GET /prompt_manager/settings adds settings.monitoring_enabled, a sources
block (configured roots with valid/code/reason, effective roots with their
listing index and watch state, auto_detected) and the monitor state
(present, observer_alive, scheduled). Every path is a public path.

A source is "watching" only while a live observer scheduled it, "manual"
when monitoring is off or the process has no monitor, and otherwise
"restart_pending".

POST accepts monitoring_enabled as a strict bool, saved under
gallery.monitoring.enabled; restart_required_fields names it when the
running observer does not already match. A save that changes the sources
runs as a "Source change" records hold, so it is refused with 409 busy
while a scan runs. Case variants of one folder are saved once.

POST /prompt_manager/settings/validate-source {path} checks a path
lexically (no "..", no control characters, native path forms only) and
then against the gallery-root rules, answering a code and reason, and
writes nothing.

* fix(system): no existence oracle in source validation

A gallery root's containment in the allowed ComfyUI folders is now
decided before its existence and type, so a path outside them answers
"outside-allowed" whether or not it exists. validate-source also folds
the filesystem-root and home-directory codes into "outside-allowed", with
one reason, and logs the detail server-side only. Paths inside the
allowed folders still say "not-found" or "not-directory". The legacy
settings save shares the same check.

* feat(scan): scan one selected source

POST /prompt_manager/jobs/scan accepts roots: indexes into the effective
sources, strict non-negative ints (no bools or floats), deduplicated, at
most 64, non-empty when present. An index past the sources is a 400
"unknown-source" and starts nothing. The job walks only the selected
sources, resolved when it was requested, and keeps each file's listing
root_index. Its scope records files, roots (null for every source) and
output_folders, the number of folders walked, which the log's first line
counts. The legacy /scan route still scans every source, and attaching to
a scoped job keeps its payload keys.

* feat(system): manage sources and start scans natively

System > Sources & scans renders the concept's section on the server's
sources: one row per effective source with its watch state and a scan
scoped to that source, the 3.2.0 update notice with the legacy dismissal,
Add source checked by validate-source, Remove, monitoring and worker
threads as host drafts behind the one Save row, and Refresh files.
A save refused by a running scan names the job. Thumbnail generation
hands off to the legacy gallery until S5.

Deviations D98, D113 and D-NEW-S4-1 are recorded in TESTING.md.

* fix(security): decide source containment as text before any filesystem access

Every caller (validate-source, POST /settings, config.json at load) now
refuses a source through one check, sources.source_problem: the form
check first, then GalleryConfig.gallery_root_problem, which compares the
normalised spelling with the allowed folders before anything resolves,
stats or opens the path. A UNC share, the \\?\ and \\.\ namespaces, a
device or any outside folder is refused with no SMB request, no
executor stall and no existence or timing oracle; only then is the path
resolved and contained again, which still catches a symlink out. The
filesystem root and the home directory answer outside-allowed with the
same reason as any outside path, so no caller learns the home path.

Also:
- POST /settings runs the form checks and parses the roots off the loop;
  an unusable source is a coded 400 invalid-source naming its public
  path (as text) and its index.
- ComfyUI folders outside the base directory are path anchors, so their
  sources round-trip and never collide by folder name.
- A per-source scan sends each index with its public path; a stale or
  shifted index is 409 sources-changed and starts nothing.
- A source still watched after monitoring was switched off is
  "stopping".
- Roots are compared as folders, not spellings, so an unchanged legacy
  resend takes no source-change hold.
- Tests: no realpath/stat/lstat of an outside path (native and Windows
  forms with ntpath fakes), the alike answers with /, ~ and a link to ~,
  an event instead of a timer wait, and a teardown that drains jobs
  before the temporary database goes.

* fix(system): say what Sources saved, refused and still watches

- A per-source scan sends the source's public path; when the sources
  changed since they were read the toast says the server's reason and
  the list and saved values are read again.
- "watching until restart" for a source the running monitor still
  watches after monitoring was saved off, and the status adds "Restart
  ComfyUI to apply: Watch for new outputs." from restart_required_fields.
- A save refused for one unusable source marks that row and the status
  is the server's message naming it.
- A valid configured source the effective list holds under another
  spelling reads "same folder as another source".
- The busy toast names the job the 409 named (ApiError keeps a
  validated running job and refused source), falling back to the list.
- The Add source dialog's listeners live on the page scope and Enter
  adds; its markup is compared with the concept (D127).
- TESTING.md: D-NEW-S4-1 is D127; D-NEW-S4-2 records the new copy; the
  Sources contract describes the lexical containment, invalid-source and
  sources-changed.

* fix(security): guard the Windows form probes on a token of their own

The no-filesystem guard matched "C:", which on a Windows runner is also
the temporary ComfyUI tree the save may resolve for its trusted first
entry. The probe paths now carry a token no trusted directory holds.

* docs(system): number the Sources states deviation D128
2026-10-23 13:15:06 -07:00

120 lines
5.6 KiB
JavaScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* Scan for metadata (#249 Activity-1, D55): Images → Import → "Scan for metadata" → Review scan opens the concept's
* "Review metadata scan". Its backup box downloads the existing `/prompt_manager/backup` copy and toasts the outcome;
* Start scan starts the session job through the Activity store over the chosen scope (new and unresolved files, or
* all files with the D58 checkbox), which toasts "Scan started.", and opens the job's view.
* A start refused because another record-changing job runs toasts "Another job is running: <title>." with a link to
* that job (D57).
*/
import { openScanReviewDialog } from './workspace-dialogs.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { navigateTo } from './state.mjs';
export const BACKUP_URL = '/prompt_manager/backup';
export const BACKUP_DONE = 'Database backup downloaded.';
export const BACKUP_FAILED = 'The database could not be backed up. Try again.';
export const REPORT_DONE = 'Job report downloaded.';
export const REPORT_FAILED = 'The report could not be downloaded. Try again.';
const JOBS_REPORT_BASE = '/prompt_manager/jobs';
export const START_FAILED = 'The scan could not start. Try again.';
// A per-source scan whose sources changed since they were read (S4): the server's 409 says so.
const SOURCES_CHANGED_CODE = 'sources-changed';
const FALLBACK_NAME = 'prompts_backup.db';
const REPORT_FALLBACK_NAME = 'prompt-manager-report.json';
/** The download name from a Content-Disposition header, or `fallback`. */
export function backupFilename(disposition, fallback = FALLBACK_NAME) {
const match = /filename="?([^";]+)"?/i.exec(disposition || '');
return match ? match[1] : fallback;
}
/** Save a fetched response through a temporary link, named by `filename`. */
function saveResponse(blob, filename) {
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = filename;
document.body.append(link);
link.click();
link.remove();
setTimeout(() => URL.revokeObjectURL(url), 1000);
}
/** Download the prompts database copy the legacy admin page also uses; toasts the outcome. */
export async function downloadDatabaseBackup({ fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast } = {}) {
try {
const response = await fetchUrl(BACKUP_URL);
if (!response.ok) throw new Error(`Backup failed with ${response.status}`);
const blob = await response.blob();
saveResponse(blob, backupFilename(response.headers.get('Content-Disposition')));
notify(BACKUP_DONE);
return true;
} catch {
notify(BACKUP_FAILED);
return false;
}
}
/**
* Download a job's JSON report (Activity-2, D59) through the api-client URL, so a proxy mount works; toasts the outcome
* unless `signal` (the view's scope) was aborted meanwhile.
*/
export async function downloadJobReport(jobId, { fetchUrl = window.ApiClient.fetchUrl, notify = showWorkspaceToast,
signal = null } = {}) {
try {
const response = await fetchUrl(`${JOBS_REPORT_BASE}/${encodeURIComponent(jobId)}/report`);
if (!response.ok) throw new Error(`Report failed with ${response.status}`);
const blob = await response.blob();
saveResponse(blob, backupFilename(response.headers.get('Content-Disposition'), REPORT_FALLBACK_NAME));
// The file is saved either way; a view left meanwhile (`signal` aborted) shows no toast.
if (!signal?.aborted) notify(REPORT_DONE);
return true;
} catch {
if (!signal?.aborted) notify(REPORT_FAILED);
return false;
}
}
/** The toast for a refused start, linking the job that holds the slot. */
export function busyToast(running) {
if (!running) return 'Another job is running. Try again when it ends.';
return ['Another job is running: ', { text: running.title, href: `#/activity/${running.id}` }, '.'];
}
/**
* Start the scan job over `files` ('new': new and unresolved files, or 'all') and open its view; a refused or failed
* start says why and stays on the page. `roots` (System › Sources, S4) limits it to those effective source indexes, and
* `paths` names the public path read for each: when the sources changed since (409 `sources-changed`) nothing starts,
* the toast says the server's reason and `onStale` (which every caller with `roots` passes) reads them again.
*/
export async function startMetadataScan({ store, scope, files = 'new', roots = null, paths = null, onStale,
notify = showWorkspaceToast, navigate = navigateTo }) {
try {
const result = await store.start('scan', roots ? { scope: files, roots, paths } : { scope: files });
if (scope.signal.aborted) return;
if (result.busy !== undefined) {
notify(busyToast(result.busy));
return;
}
navigate(`#/activity/${result.job.id}`);
} catch (error) {
if (scope.signal.aborted) return;
if (error?.code === SOURCES_CHANGED_CODE) {
notify(error.message);
onStale();
return;
}
notify(START_FAILED);
}
}
/**
* Open "Review metadata scan" for `sources` output folders; with `roots` and `paths` (one source's index and public
* path, System › Sources) the scan reads only those, and `onStale` runs when they changed since they were read.
*/
export function openMetadataScan({ scope, store, sources, roots = null, paths = null, onStale, trigger = null }) {
openScanReviewDialog({ scope, sources, trigger,
onBackup: () => downloadDatabaseBackup(),
onStart: ({ scope: files }) => startMetadataScan({ store, scope, files, roots, paths, onStale }) });
}