Vito
69de20a427
* feat(system): report configured, effective and watched sources
GET /prompt_manager/settings adds settings.monitoring_enabled, a sources
block (configured roots with valid/code/reason, effective roots with their
listing index and watch state, auto_detected) and the monitor state
(present, observer_alive, scheduled). Every path is a public path.
A source is "watching" only while a live observer scheduled it, "manual"
when monitoring is off or the process has no monitor, and otherwise
"restart_pending".
POST accepts monitoring_enabled as a strict bool, saved under
gallery.monitoring.enabled; restart_required_fields names it when the
running observer does not already match. A save that changes the sources
runs as a "Source change" records hold, so it is refused with 409 busy
while a scan runs. Case variants of one folder are saved once.
POST /prompt_manager/settings/validate-source {path} checks a path
lexically (no "..", no control characters, native path forms only) and
then against the gallery-root rules, answering a code and reason, and
writes nothing.
* fix(system): no existence oracle in source validation
A gallery root's containment in the allowed ComfyUI folders is now
decided before its existence and type, so a path outside them answers
"outside-allowed" whether or not it exists. validate-source also folds
the filesystem-root and home-directory codes into "outside-allowed", with
one reason, and logs the detail server-side only. Paths inside the
allowed folders still say "not-found" or "not-directory". The legacy
settings save shares the same check.
* feat(scan): scan one selected source
POST /prompt_manager/jobs/scan accepts roots: indexes into the effective
sources, strict non-negative ints (no bools or floats), deduplicated, at
most 64, non-empty when present. An index past the sources is a 400
"unknown-source" and starts nothing. The job walks only the selected
sources, resolved when it was requested, and keeps each file's listing
root_index. Its scope records files, roots (null for every source) and
output_folders, the number of folders walked, which the log's first line
counts. The legacy /scan route still scans every source, and attaching to
a scoped job keeps its payload keys.
* feat(system): manage sources and start scans natively
System > Sources & scans renders the concept's section on the server's
sources: one row per effective source with its watch state and a scan
scoped to that source, the 3.2.0 update notice with the legacy dismissal,
Add source checked by validate-source, Remove, monitoring and worker
threads as host drafts behind the one Save row, and Refresh files.
A save refused by a running scan names the job. Thumbnail generation
hands off to the legacy gallery until S5.
Deviations D98, D113 and D-NEW-S4-1 are recorded in TESTING.md.
* fix(security): decide source containment as text before any filesystem access
Every caller (validate-source, POST /settings, config.json at load) now
refuses a source through one check, sources.source_problem: the form
check first, then GalleryConfig.gallery_root_problem, which compares the
normalised spelling with the allowed folders before anything resolves,
stats or opens the path. A UNC share, the \\?\ and \\.\ namespaces, a
device or any outside folder is refused with no SMB request, no
executor stall and no existence or timing oracle; only then is the path
resolved and contained again, which still catches a symlink out. The
filesystem root and the home directory answer outside-allowed with the
same reason as any outside path, so no caller learns the home path.
Also:
- POST /settings runs the form checks and parses the roots off the loop;
an unusable source is a coded 400 invalid-source naming its public
path (as text) and its index.
- ComfyUI folders outside the base directory are path anchors, so their
sources round-trip and never collide by folder name.
- A per-source scan sends each index with its public path; a stale or
shifted index is 409 sources-changed and starts nothing.
- A source still watched after monitoring was switched off is
"stopping".
- Roots are compared as folders, not spellings, so an unchanged legacy
resend takes no source-change hold.
- Tests: no realpath/stat/lstat of an outside path (native and Windows
forms with ntpath fakes), the alike answers with /, ~ and a link to ~,
an event instead of a timer wait, and a teardown that drains jobs
before the temporary database goes.
* fix(system): say what Sources saved, refused and still watches
- A per-source scan sends the source's public path; when the sources
changed since they were read the toast says the server's reason and
the list and saved values are read again.
- "watching until restart" for a source the running monitor still
watches after monitoring was saved off, and the status adds "Restart
ComfyUI to apply: Watch for new outputs." from restart_required_fields.
- A save refused for one unusable source marks that row and the status
is the server's message naming it.
- A valid configured source the effective list holds under another
spelling reads "same folder as another source".
- The busy toast names the job the 409 named (ApiError keeps a
validated running job and refused source), falling back to the list.
- The Add source dialog's listeners live on the page scope and Enter
adds; its markup is compared with the concept (D127).
- TESTING.md: D-NEW-S4-1 is D127; D-NEW-S4-2 records the new copy; the
Sources contract describes the lexical containment, invalid-source and
sources-changed.
* fix(security): guard the Windows form probes on a token of their own
The no-filesystem guard matched "C:", which on a Windows runner is also
the temporary ComfyUI tree the save may resolve for its trusted first
entry. The probe paths now carry a token no trusted directory holds.
* docs(system): number the Sources states deviation D128
146 lines
8.9 KiB
JavaScript
146 lines
8.9 KiB
JavaScript
// System › Sources & scans (#249 System S4 Task 11): the concept's "Image sources" list (app.js 679–681) built from the
|
||
// server's configured, effective and watched sources (GET /settings), the host draft of `gallery_root_paths`, the
|
||
// 3.2.0 update notice's dismissal and the Refresh files toast (D98).
|
||
const test = require('node:test');
|
||
const assert = require('node:assert/strict');
|
||
|
||
const sourcesModule = import('../../web/js/workspace/system-sources.mjs');
|
||
|
||
const STATUS = Object.freeze({
|
||
configured: [
|
||
{ label: 'ComfyUI output', path: 'output', valid: true, code: null, reason: null },
|
||
{ label: 'ComfyUI input', path: 'input', valid: true, code: null, reason: null },
|
||
],
|
||
effective: [
|
||
{ label: 'ComfyUI output', path: 'output', root_index: 0, watch: 'watching' },
|
||
{ label: 'ComfyUI input', path: 'input', root_index: 1, watch: 'manual' },
|
||
],
|
||
auto_detected: false,
|
||
});
|
||
|
||
test('each effective source is the concept row: label, "<path> · watching|manual scan", Scan changes or Scan folder', async () => {
|
||
const { sourceRows } = await sourcesModule;
|
||
const saved = ['output', 'input'];
|
||
const { rows, note } = sourceRows({ sources: STATUS, saved, current: saved });
|
||
assert.equal(note, null);
|
||
assert.deepEqual(rows, [
|
||
{ key: 'output', title: 'ComfyUI output', description: 'output · watching', scan: { label: 'Scan changes', root: 0, path: 'output' },
|
||
remove: 'output' },
|
||
{ key: 'input', title: 'ComfyUI input', description: 'input · manual scan', scan: { label: 'Scan folder', root: 1, path: 'input' },
|
||
remove: 'input' },
|
||
]);
|
||
});
|
||
|
||
test('watching only with a live observer; restart pending otherwise (D98)', async () => {
|
||
const { sourceRows, WATCH_COPY } = await sourcesModule;
|
||
assert.deepEqual(WATCH_COPY, { watching: 'watching', stopping: 'watching until restart', manual: 'manual scan',
|
||
restart_pending: 'restart pending' });
|
||
const sources = { ...STATUS, effective: STATUS.effective.map(source => ({ ...source, watch: 'restart_pending' })) };
|
||
const { rows } = sourceRows({ sources, saved: ['output', 'input'], current: ['output', 'input'] });
|
||
assert.deepEqual(rows.map(row => [row.description, row.scan.label]),
|
||
[['output · restart pending', 'Scan folder'], ['input · restart pending', 'Scan folder']]);
|
||
});
|
||
|
||
test('a configured source that cannot be used shows the server reason and only Remove', async () => {
|
||
const { sourceRows } = await sourcesModule;
|
||
const sources = { ...STATUS, configured: [...STATUS.configured,
|
||
{ label: 'unmounted-drive', path: 'unmounted-drive', valid: false, code: 'not-found', reason: 'Gallery root does not exist' }] };
|
||
const saved = ['output', 'input', 'unmounted-drive'];
|
||
const { rows } = sourceRows({ sources, saved, current: saved });
|
||
assert.deepEqual(rows.at(-1), { key: 'unmounted-drive', title: 'unmounted-drive',
|
||
description: 'unmounted-drive · Gallery root does not exist', scan: null, remove: 'unmounted-drive' });
|
||
});
|
||
|
||
test('an added source is listed as not saved yet with its validated label; a removed one leaves the list', async () => {
|
||
const { sourceRows } = await sourcesModule;
|
||
const labels = new Map([['output/archive', 'archive']]);
|
||
const { rows, note } = sourceRows({ sources: STATUS, saved: ['output', 'input'], current: ['input', 'output/archive'], labels });
|
||
assert.equal(note, null);
|
||
assert.deepEqual(rows.map(row => [row.title, row.description, row.scan?.label ?? null, row.remove]), [
|
||
['ComfyUI input', 'input · manual scan', 'Scan folder', 'input'],
|
||
['archive', 'output/archive · not saved yet', null, 'output/archive'],
|
||
]);
|
||
// Without a recorded label (a reload of System), the folder name stands in.
|
||
assert.equal(sourceRows({ sources: STATUS, saved: [], current: ['output/new-collection'] }).rows[0].title, 'new-collection');
|
||
});
|
||
|
||
test('removing the last entry states auto-detect (D98)', async () => {
|
||
const { sourceRows, NOTES } = await sourcesModule;
|
||
const { rows, note } = sourceRows({ sources: STATUS, saved: ['output', 'input'], current: [] });
|
||
assert.deepEqual(rows, []);
|
||
assert.equal(note, NOTES.removedAll);
|
||
assert.equal(NOTES.removedAll, 'No source is configured. After saving, ComfyUI’s output folder is used.');
|
||
});
|
||
|
||
test('auto-detected sources are listed without Remove, with the line that says how they were chosen', async () => {
|
||
const { sourceRows, NOTES } = await sourcesModule;
|
||
const auto = { configured: [], effective: [{ label: 'ComfyUI output', path: 'output', root_index: 0, watch: 'manual' }], auto_detected: true };
|
||
assert.deepEqual(sourceRows({ sources: auto, saved: [], current: [] }), { rows: [
|
||
{ key: 'auto:0', title: 'ComfyUI output', description: 'output · manual scan', scan: { label: 'Scan folder', root: 0, path: 'output' },
|
||
remove: null },
|
||
], note: NOTES.autoDetected });
|
||
// Every configured source unusable: ComfyUI's output folder is used instead.
|
||
const fallback = { ...auto, configured: [{ label: 'gone', path: 'gone', valid: false, code: 'not-found', reason: 'Gallery root does not exist' }] };
|
||
const result = sourceRows({ sources: fallback, saved: ['gone'], current: ['gone'] });
|
||
assert.deepEqual(result.rows.map(row => [row.title, row.remove]), [['gone', 'gone'], ['ComfyUI output', null]]);
|
||
assert.equal(result.note, NOTES.fallback);
|
||
assert.equal(sourceRows({ sources: { configured: [], effective: [], auto_detected: false }, saved: [], current: [] }).note, NOTES.none);
|
||
});
|
||
|
||
test('a path listed twice in the draft is one row', async () => {
|
||
const { sourceRows } = await sourcesModule;
|
||
assert.equal(sourceRows({ sources: STATUS, saved: ['output', 'input'], current: ['output', 'output'] }).rows.length, 1);
|
||
});
|
||
|
||
test('roots equal to the saved list are the saved list itself, so the draft is clean again', async () => {
|
||
const { draftRoots } = await sourcesModule;
|
||
const saved = ['output', 'input'];
|
||
assert.equal(draftRoots(['output', 'input'], saved), saved);
|
||
assert.deepEqual(draftRoots(['input'], saved), ['input']);
|
||
assert.notEqual(draftRoots(['input', 'output'], saved), saved);
|
||
assert.deepEqual(draftRoots([], undefined), []);
|
||
});
|
||
|
||
test('notice Later persists the 3.2.0 dismissal (pm_update_notice_dismissed); storage that throws keeps it for this view', async () => {
|
||
const { noticeDismissed, dismissNotice, NOTICE_KEY, NOTICE_VERSION } = await sourcesModule;
|
||
assert.deepEqual([NOTICE_KEY, NOTICE_VERSION], ['pm_update_notice_dismissed', '3.2.0']);
|
||
const values = new Map();
|
||
const storage = { getItem: key => values.get(key) ?? null, setItem: (key, value) => values.set(key, String(value)) };
|
||
assert.equal(noticeDismissed(() => storage), false);
|
||
values.set(NOTICE_KEY, '3.1.0');
|
||
assert.equal(noticeDismissed(() => storage), false);
|
||
assert.equal(dismissNotice(() => storage), true);
|
||
assert.equal(values.get(NOTICE_KEY), '3.2.0');
|
||
assert.equal(noticeDismissed(() => storage), true);
|
||
const denied = () => { throw new Error('SecurityError'); };
|
||
assert.equal(noticeDismissed(denied), false);
|
||
assert.equal(dismissNotice(denied), false);
|
||
assert.equal(dismissNotice(() => ({ setItem() { throw new Error('QuotaExceededError'); } })), false);
|
||
});
|
||
|
||
test('Refresh files toasts the rescan counts, or says it finished when they are missing', async () => {
|
||
const { refreshedText } = await sourcesModule;
|
||
assert.equal(refreshedText({ total: 1234, roots: 2 }), 'Files refreshed. 1,234 files in 2 output folders.');
|
||
assert.equal(refreshedText({ total: 1, roots: 1 }), 'Files refreshed. 1 file in 1 output folder.');
|
||
assert.equal(refreshedText({ total: -1, roots: 1 }), 'Files refreshed.');
|
||
assert.equal(refreshedText(null), 'Files refreshed.');
|
||
});
|
||
|
||
test('monitoring switched off: a source still watched says so until the restart (D128)', async () => {
|
||
const { sourceRows } = await sourcesModule;
|
||
const sources = { ...STATUS, effective: [{ ...STATUS.effective[0], watch: 'stopping' }, STATUS.effective[1]] };
|
||
const { rows } = sourceRows({ sources, saved: ['output', 'input'], current: ['output', 'input'] });
|
||
assert.deepEqual(rows.map(row => [row.description, row.scan.label]),
|
||
[['output · watching until restart', 'Scan changes'], ['input · manual scan', 'Scan folder']]);
|
||
});
|
||
|
||
test('a valid configured source missing from the effective list is another spelling of a listed folder (D128)', async () => {
|
||
const { sourceRows, DUPLICATE_SOURCE } = await sourcesModule;
|
||
assert.equal(DUPLICATE_SOURCE, 'same folder as another source');
|
||
const sources = { ...STATUS, configured: [...STATUS.configured,
|
||
{ label: 'output', path: 'output/.', valid: true, code: null, reason: null }] };
|
||
const saved = ['output', 'input', 'output/.'];
|
||
assert.deepEqual(sourceRows({ sources, saved, current: saved }).rows.at(-1), { key: 'output/.', title: 'output',
|
||
description: 'output/. · same folder as another source', scan: null, remove: 'output/.' });
|
||
});
|