* fix(gallery): clear only Prompt Manager cache entries (#222) The legacy gallery's Clear cache deleted every localStorage key starting pm_ or promptManager_ (the workspace theme and the dismissed update notice among them) and every CacheStorage cache whose name held gallery or thumbnail, which on ComfyUI's shared origin can belong to another tool. web/js/cache-ownership.js names what a clear may remove from an inventory of the real writers: every key Prompt Manager writes is a preference and no page opens CacheStorage or IndexedDB, so nothing is removed and the toast says so. A test fails when a new storage writer is not classified. The server thumbnail clear stays its own request; a refused or unreachable server changes nothing and is reported, and refused browser storage is reported as such. * feat(system): run reviewed maintenance natively System › Maintenance is native: the concept's notice and its Database and Images & cache surfaces replace the hand-off. - Integrity & relationship checks runs the health check; the report's Review missing links reviews a fresh Missing-file links plan. - Review plan shows the server's read-only plan (D100): scope, protected, backup policy and media lines, the listed records, how long the review holds and the backup box; Apply reviewed operation sends only its token and opens the job in Activity. busy names the running job, plan-used links the job that applied it, an expired or replaced review is reviewed again, and a lost answer says Outcome not confirmed and points to Activity. - Find duplicates starts the search; its view offers Review groups (D101): ten groups at a time, Keep radios per copy, each keeper sent as {group_id, root_index, path}; the dialog says History keeps the removed paths in its run records. - Thumbnail cache opens the thumbnail dialog; Metadata cache is disabled with its reason while nothing is owned (D102, Q5). - Activity descriptors for maintenance, duplicates and duplicate_cleanup. - The legacy duplicates dialog's View Full names its source (?root=). * test(system): compare Maintenance with the concept system-maintenance-concept.spec compares System › Maintenance, the plan dialog of each operation and the duplicates dialog with the concept in both themes and both viewports, mapped only by D100–D102, and proves a mutated row, label, danger mark or token is noticed. The duplicates dialog's Review duplicate cleanup is the concept's plain button. * test(system): Maintenance no longer hands off; raise coverage floors The System shell test expected Maintenance's D94 hand-off; the section is native now. coverage:update sets the new modules' floors from the measured run (cache-ownership 94, maintenance-review 94, system-maintenance 92 lines) and raises the floors other files passed. * test(admin): close the preview worker after the last job in teardown A scan publishes its last event, which ends the stream the test reads, before it hands its missing previews to the automatic worker. That hand-off can run after tearDownAsync closed the worker, so it starts a new one whose .part files were still open when the final drain checked for open handles (py3.10 on ubuntu, test_scan_reads_metadata_on_several_threads). The drain now closes the worker for good once every job has ended. * fix(system): address the S7 final review - I1: an apply answer is a refusal only when our server sent it (a known code, or a 4xx whose body says success:false); a proxy's 5xx, an unreadable 2xx or a dropped connection says Outcome not confirmed and reads Activity again. ApiError gains 'refused'; the server's own pre-start failure is coded 'apply-failed'. - M1: the lost-response test lets the apply run, then drops its answer. - M2: a failed apply records its code in the job's scope; Activity offers Review again for an overtaken plan (Maintenance for a duplicate cleanup whose sources changed). - M3: a failed duplicate cleanup toasts the server's reason. - M4: a review button holds while its preview is read (one preview per double press). - M5: the storage guard also catches bracket and property assignment and CacheStorage/IndexedDB opens; why the Toolkit build is skipped is written down. - M6: a refused storage listing says so on the Metadata cache row. - M7, M8: D100 text corrected; workflow_storage has its title. - M9: a busy start names the job the server named. * test: follow #319 in the Maintenance tests - The thumbnail dialog has only Scope since #319. - The admin teardown refuses new preview workers on its own API instead of closing with final=True, which since #319 also shuts the process-wide encoders that later test modules use. * fix(system): keep the server's search-again line on a duplicate review A duplicate cleanup review the server refuses with 409 sources-changed showed the generic "could not be reviewed" message; it now shows the server's "The sources changed since the search. Search for duplicates again." * fix(system): a plan with nothing to do offers only Close A reviewed maintenance plan whose scope is 0 records ("No duplicate prompt text") offered Apply reviewed operation, which started an empty job. It now shows its rows, "There is nothing to apply; nothing was changed." and a Close button. * fix(system): wrap long copy paths in the duplicates review A Keep label is the copy's source and path, and dataset folders often have long names with no space to break at, so the review scrolled sideways. The label now wraps anywhere inside its compare item (D101).
190 lines
11 KiB
JavaScript
190 lines
11 KiB
JavaScript
// #222: what "clear the cache" may remove from this browser. Only entries Prompt Manager itself writes as a cache count,
|
|
// from an inventory of the real writers (never a guessed prefix): preferences (the workspace theme, the update notice,
|
|
// the display defaults) and other tools' storage and caches on the shared ComfyUI origin always stay.
|
|
const test = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('node:fs');
|
|
const path = require('node:path');
|
|
|
|
const CacheOwnership = require('../../web/js/cache-ownership.js');
|
|
|
|
const WEB = path.join(__dirname, '../../web');
|
|
|
|
function memoryStorage(entries) {
|
|
const map = new Map(Object.entries(entries));
|
|
return {
|
|
map,
|
|
get length() { return map.size; },
|
|
key: index => [...map.keys()][index] ?? null,
|
|
getItem: key => (map.has(key) ? map.get(key) : null),
|
|
setItem: (key, value) => map.set(key, String(value)),
|
|
removeItem: key => map.delete(key),
|
|
};
|
|
}
|
|
|
|
function memoryCaches(names, { failDelete = new Set() } = {}) {
|
|
const set = new Set(names);
|
|
return {
|
|
set,
|
|
keys: async () => [...set],
|
|
delete: async name => {
|
|
if (failDelete.has(name)) throw new Error('denied');
|
|
return set.delete(name);
|
|
},
|
|
};
|
|
}
|
|
|
|
const PREFERENCES = { gallerySettings: '{"viewMode":"grid"}', pm_workspace_theme: 'light', pm_update_notice_dismissed: '3.3.0' };
|
|
const OWNED = Object.freeze({ storageKeys: ['pm-owned-test-entry'], cacheNames: ['pm-owned-test-cache'] });
|
|
|
|
test('keeps pm_workspace_theme, pm_update_notice_dismissed and gallerySettings', async () => {
|
|
const storage = memoryStorage({ ...PREFERENCES, 'pm-owned-test-entry': 'x', pm_cached: '1', promptManager_x: '1' });
|
|
const result = await CacheOwnership.clearOwned({ storage, cacheStorage: null, owned: OWNED });
|
|
assert.deepEqual(result.removed, ['pm-owned-test-entry']);
|
|
for (const [key, value] of Object.entries(PREFERENCES)) assert.equal(storage.getItem(key), value, key);
|
|
// The old prefix rule's matches are not Prompt Manager caches either: nothing writes them.
|
|
assert.equal(storage.getItem('pm_cached'), '1');
|
|
assert.equal(storage.getItem('promptManager_x'), '1');
|
|
for (const key of Object.keys(PREFERENCES)) assert.ok(CacheOwnership.PREFERENCE_KEYS.includes(key), key);
|
|
});
|
|
|
|
test("keeps other tools' CacheStorage named gallery/thumbnail", async () => {
|
|
const caches = memoryCaches(['other-gallery', 'thumbnail-cache', 'my-gallery-thumbnails', 'pm-owned-test-cache']);
|
|
const result = await CacheOwnership.clearOwned({ storage: memoryStorage({}), cacheStorage: caches, owned: OWNED });
|
|
assert.deepEqual([...caches.set].sort(), ['my-gallery-thumbnails', 'other-gallery', 'thumbnail-cache']);
|
|
assert.deepEqual(result.removed, ['pm-owned-test-cache']);
|
|
});
|
|
|
|
test('removes only inventoried owned keys', async () => {
|
|
// The shipped inventory: no Prompt Manager writer stores a cache entry in this browser.
|
|
assert.deepEqual(CacheOwnership.OWNED, { storageKeys: [], cacheNames: [] });
|
|
assert.ok(Object.isFrozen(CacheOwnership.OWNED.storageKeys) && Object.isFrozen(CacheOwnership.OWNED.cacheNames));
|
|
const storage = memoryStorage({ ...PREFERENCES, gallery_cache_1: 'a', metadata_cache_2: 'b', other: 'c' });
|
|
const caches = memoryCaches(['gallery', 'thumbnail']);
|
|
const result = await CacheOwnership.clearOwned({ storage, cacheStorage: caches });
|
|
assert.deepEqual(result, { removed: [], failed: [], denied: false, partial: false });
|
|
assert.equal(storage.map.size, 6);
|
|
assert.equal(caches.set.size, 2);
|
|
assert.equal(CacheOwnership.clearMessage(result), CacheOwnership.NOTHING_CACHED);
|
|
});
|
|
|
|
/**
|
|
* Every browser-storage write in one file's text: `{keys}` (setItem keys, as written) and `problems` (writes no key
|
|
* can be read from: a bracket or property assignment on a Storage, a computed setItem key, an IndexedDB or
|
|
* CacheStorage open or write). The scan is textual, so it errs on the side of reporting.
|
|
*/
|
|
function storageWrites(text) {
|
|
const keys = [];
|
|
const problems = [];
|
|
for (const match of text.matchAll(/\.setItem\(\s*([^,)]+)/g)) {
|
|
const key = match[1].trim();
|
|
const literal = /^(?:'([^']+)'|"([^"]+)")$/.exec(key);
|
|
if (literal) keys.push(literal[1] ?? literal[2]);
|
|
else if (/^[A-Z_]+_KEY$/.test(key)) keys.push(`$${key}`);
|
|
else problems.push(`a computed setItem key: ${key}`);
|
|
}
|
|
if (/\b\w*[sS]torage\s*\[[^\]]+\]\s*=(?!=)/.test(text)) problems.push('a bracket assignment on a Storage');
|
|
if (/\b(?:local|session)Storage\s*\.\s*(?!getItem|setItem|removeItem|key\b|length\b|clear\b)\w+\s*=(?!=)/.test(text)) {
|
|
problems.push('a property assignment on a Storage');
|
|
}
|
|
if (/\bcaches\s*\.\s*open\s*\(/.test(text)) problems.push('opens a CacheStorage cache');
|
|
if (/\bindexedDB\s*\.\s*open\s*\(/.test(text)) problems.push('opens IndexedDB');
|
|
return { keys, problems };
|
|
}
|
|
|
|
test('the storage scan reports every way of writing browser storage (S7 review M5)', () => {
|
|
assert.deepEqual(storageWrites("localStorage.setItem('gallerySettings', x); storage.setItem(NOTICE_KEY, v);").keys,
|
|
['gallerySettings', '$NOTICE_KEY']);
|
|
for (const text of ["localStorage['pm_cache'] = 1;", 'sessionStorage[key] = value;', 'storage[name] = x;',
|
|
'localStorage.pm_cache = 1;', 'sessionStorage.someKey = "v";', 'localStorage.setItem(key, 1);',
|
|
"caches.open('pm').then(cache => cache.put(url, response));", "indexedDB.open('pm-cache', 1);"]) {
|
|
assert.equal(storageWrites(text).problems.length, 1, text);
|
|
}
|
|
for (const text of ["if (localStorage['pm'] === x) {}", 'const value = localStorage.getItem(KEY);',
|
|
'localStorage.removeItem(KEY);', 'const n = localStorage.length;']) {
|
|
assert.deepEqual(storageWrites(text), { keys: [], problems: [] }, text);
|
|
}
|
|
});
|
|
|
|
test('the inventory covers every browser storage writer in web/', () => {
|
|
// Every key a Prompt Manager page writes is a preference or an inventoried cache entry; a new writer must be
|
|
// classified here before a cache clear may remove it. Skipped: web/lib (third-party Viewer.js and Tailwind
|
|
// output) and web/js/shared/toolkit (the pinned ComfyAssets Toolkit build, never hand-edited here and verified by
|
|
// `npm run --prefix tools/vendor check`). Neither writes browser storage today; the Toolkit's own repository owns
|
|
// its storage, and a new pin is reviewed as its own change.
|
|
const files = [];
|
|
const walk = dir => {
|
|
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
|
|
const full = path.join(dir, entry.name);
|
|
if (entry.isDirectory()) { if (!['lib', 'toolkit'].includes(entry.name)) walk(full); }
|
|
else if (/\.(m?js|html)$/.test(entry.name)) files.push(full);
|
|
}
|
|
};
|
|
walk(WEB);
|
|
const written = new Set();
|
|
const constants = {};
|
|
for (const file of files) {
|
|
const text = fs.readFileSync(file, 'utf8');
|
|
for (const match of text.matchAll(/(?:const|export const)\s+([A-Z_]+_KEY)\s*=\s*'([^']+)'/g)) constants[match[1]] = match[2];
|
|
const { keys, problems } = storageWrites(text);
|
|
assert.deepEqual(problems, [], path.relative(WEB, file));
|
|
for (const key of keys) written.add(key);
|
|
}
|
|
const keys = [...written].map(key => (key.startsWith('$') ? constants[key.slice(1)] : key));
|
|
assert.ok(keys.length >= 3, 'the writers were found');
|
|
for (const key of keys) {
|
|
assert.ok(key, 'every written key resolves');
|
|
assert.ok(CacheOwnership.PREFERENCE_KEYS.includes(key) || CacheOwnership.OWNED.storageKeys.includes(key), key);
|
|
}
|
|
});
|
|
|
|
test('storage denial reports partial', async () => {
|
|
const denied = { get length() { throw new Error('SecurityError'); }, key() { throw new Error('SecurityError'); },
|
|
getItem() { throw new Error('SecurityError'); }, removeItem() { throw new Error('SecurityError'); } };
|
|
const result = await CacheOwnership.clearOwned({ storage: denied, cacheStorage: null, owned: OWNED });
|
|
assert.equal(result.denied, true);
|
|
assert.equal(result.partial, true);
|
|
assert.equal(CacheOwnership.clearMessage(result), CacheOwnership.STORAGE_DENIED);
|
|
|
|
const caches = memoryCaches(['pm-owned-test-cache'], { failDelete: new Set(['pm-owned-test-cache']) });
|
|
const storage = memoryStorage({ 'pm-owned-test-entry': 'x' });
|
|
const partial = await CacheOwnership.clearOwned({ storage, cacheStorage: caches, owned: OWNED });
|
|
assert.deepEqual(partial.removed, ['pm-owned-test-entry']);
|
|
assert.deepEqual(partial.failed, ['pm-owned-test-cache']);
|
|
assert.equal(partial.partial, true);
|
|
assert.equal(CacheOwnership.clearMessage(partial), 'Cleared 1 of 2 cached entries; this browser refused the rest.');
|
|
|
|
const keysDenied = { keys: async () => { throw new Error('denied'); } };
|
|
const unreadable = await CacheOwnership.clearOwned({ storage: memoryStorage({}), cacheStorage: keysDenied, owned: OWNED });
|
|
assert.equal(unreadable.denied, true);
|
|
});
|
|
|
|
test('server thumbnail clear stays a separate explicit call', async () => {
|
|
const calls = [];
|
|
const original = globalThis.fetch;
|
|
globalThis.fetch = async (...args) => { calls.push(args); return { ok: true }; };
|
|
try {
|
|
const storage = memoryStorage({ 'pm-owned-test-entry': 'x' });
|
|
const result = await CacheOwnership.clearOwned({ storage, cacheStorage: memoryCaches([]), owned: OWNED });
|
|
assert.deepEqual(result.removed, ['pm-owned-test-entry']);
|
|
assert.equal(CacheOwnership.clearMessage(result), 'Cleared 1 cached entry from this browser.');
|
|
} finally {
|
|
globalThis.fetch = original;
|
|
}
|
|
assert.deepEqual(calls, []);
|
|
assert.equal(/fetch|clear-thumbnails|ApiClient/.test(fs.readFileSync(path.join(WEB, 'js/cache-ownership.js'), 'utf8')), false);
|
|
});
|
|
|
|
test('ownedEntries lists what a clear would remove, reading nothing it cannot', async () => {
|
|
const storage = memoryStorage({ ...PREFERENCES, 'pm-owned-test-entry': 'x' });
|
|
const caches = memoryCaches(['pm-owned-test-cache', 'gallery']);
|
|
assert.deepEqual(await CacheOwnership.ownedEntries({ storage, cacheStorage: caches, owned: OWNED }),
|
|
{ keys: ['pm-owned-test-entry'], caches: ['pm-owned-test-cache'], denied: false });
|
|
assert.deepEqual(await CacheOwnership.ownedEntries({ storage, cacheStorage: caches }),
|
|
{ keys: [], caches: [], denied: false });
|
|
assert.deepEqual(await CacheOwnership.ownedEntries({ storage: null, cacheStorage: undefined, owned: OWNED }),
|
|
{ keys: [], caches: [], denied: false });
|
|
assert.equal(CacheOwnership.clearMessage({ removed: ['a', 'b'], failed: [], denied: false, partial: false }),
|
|
'Cleared 2 cached entries from this browser.');
|
|
});
|