Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
975ae2ac49 | ||
|
|
93bf24f47d | ||
|
|
e29c23de8d | ||
|
|
b82e68a9d0 | ||
|
|
fb02066cda | ||
|
|
ab17818792 |
+25
-7
@@ -345,8 +345,10 @@ and the time left. The steps, in order:
|
||||
monitor's callbacks (Needs review recorder, preview queue, Images listing
|
||||
listener).
|
||||
2. `ImageMonitor.stop_monitoring` works against one 5 s deadline. Closing the
|
||||
image handler joins the image worker; watchdog's `observer.stop()` (not
|
||||
bounded here) and the observer join get what is left, which can be 0 s.
|
||||
image handler joins the image worker; watchdog's `observer.stop()` and the
|
||||
observer join get what is left, which can be 0 s. `stop()` joins watchdog's
|
||||
emitters without a limit, so it runs on a daemon thread and is waited for
|
||||
only until the deadline; a later stop waits for that same call (#561).
|
||||
3. `close_auto_thumbnails(final=True)` stops the preview worker (30 s) and
|
||||
kills ffmpeg.
|
||||
4. `PromptTracker.shutdown` stops the prompt cleanup thread (2 s).
|
||||
@@ -361,6 +363,15 @@ stored, with the parts not reached False, and the `atexit` callback is
|
||||
removed: the later steps are skipped, no summary line is logged, and a later
|
||||
call returns that result and runs nothing.
|
||||
|
||||
The observer is started from a short-lived daemon thread, so every thread it
|
||||
creates is a daemon (a thread inherits that flag from the thread that creates
|
||||
it). watchdog 7 on Windows starts a plain `threading.Thread` per watch for its
|
||||
`ReadDirectoryChangesW` reader; created from ComfyUI's main thread it would not
|
||||
be a daemon, and the interpreter waits for every such thread before it runs any
|
||||
`atexit` callback, so the teardown would never start (#561).
|
||||
`tests/test_package_exit.py` runs a child whose observer does both (a plain
|
||||
reader thread, a `stop()` that never returns) and requires its exit.
|
||||
|
||||
ComfyUI never runs aiohttp's `on_cleanup` hooks, because it never calls
|
||||
`runner.cleanup()`. So `atexit` is the live host's path. It runs on Ctrl+C
|
||||
(`main.py` catches `KeyboardInterrupt` and returns) and on a normal exit.
|
||||
@@ -1408,6 +1419,8 @@ is shorter than the row it reserves.
|
||||
Each Overview card is the count of the page it names: the shell's Library, Images and Tags sidebar counts, one source,
|
||||
so a card and its badge cannot differ. `/prompt_manager/stats` is for the legacy page and counts raw rows. A visit re-reads
|
||||
the three unless a read is in flight. An unread or failed card shows a dash; Retry re-reads only the failed counts.
|
||||
System › Overview's Images metric reads the same shell count (#555), so it, the Overview card and the Images badge show
|
||||
one number; its Statistics dialog labels the legacy row counts as "Database records".
|
||||
Tests: `tests/js/workspace-state.test.js`, `tests/browser/workspace.spec.mjs`, `tests/comfyui/workspace.spec.mjs`.
|
||||
|
||||
### Images page controls and recorded deviations (#249 PR C)
|
||||
@@ -4007,7 +4020,11 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
"Save preferences" is `aria-disabled` (opacity .55, `not-allowed`) while clean or
|
||||
saving; a setting the last save could not save gets a red outline. CSS under D92.
|
||||
- **D93** (Overview): real values. The metrics are the Library's eligible positives
|
||||
and active saved negatives and the indexed `generated_images` rows (Q2). "Database
|
||||
and active saved negatives and the Images count (#555, amending Q2's indexed
|
||||
`generated_images` rows): the shell's Images sidebar count, the Images page's own
|
||||
total, which also lists files no row links, as the concept's `images.length` is
|
||||
both. A visit re-reads it unless a read is in flight; "–" until read or after a
|
||||
failed read. `images.indexed` stays in the API and is not shown. "Database
|
||||
integrity" is "Not checked" until a check ran this session, then "N issues" (checks
|
||||
that found issues, even beside a check that could not run), "Passed" or "Incomplete".
|
||||
The review row is "N ambiguous image links" (the Needs review queue's pending count,
|
||||
@@ -4045,10 +4062,11 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
- **D96** (Statistics; amended by D153): "Usage statistics" without the sample note: the
|
||||
concept's "Runs by category" table without its Avg. rating column (Category / Runs)
|
||||
with the real runs of eligible parents per category ("Uncategorized"), then a second
|
||||
table "Library totals" (Total / Count: Prompts, Tags, Images from the legacy
|
||||
statistics; no Rated images). The API still reports `average_rating` and
|
||||
table "Database records" (Total / Count: Prompt records, Tags on prompts, Image
|
||||
records: the raw row counts of the legacy statistics, labelled as rows so they never
|
||||
read as the Library or Images page counts, #555; no Rated images). The API still reports `average_rating` and
|
||||
`rated_images`; the workspace does not show them.
|
||||
The Overview row says "Runs by category, with library totals." because
|
||||
The Overview row says "Runs by category, with database record counts." because
|
||||
that is what the dialog shows (the concept's own dialog shows only "Runs by
|
||||
category"); per-prompt run counts, prompt tags and dated runs are recorded, but no
|
||||
table of them exists yet. A failed read toasts "Statistics could not
|
||||
@@ -5414,7 +5432,7 @@ list stay unscoped: any row's control proves them.
|
||||
| Worker threads | `admin.html:437–445` | S4 Sources | covered | `sources` · slider · "Worker threads" | worker range max is cpu_count (workspace-system-sources.spec.mjs) |
|
||||
| LoraManager badge, enable, path, trigger words, CivitAI key, clear key | `admin.html:448–490`; `admin.js:1052–1156` | S10 Integrations | covered | `integrations` · switch · "LoraManager integration" | LoRA rows show detected / enabled / cache-loaded separately (workspace-system-integrations.spec.mjs) |
|
||||
| LoRA data import with progress | `admin.html:492–498,517–541`; `admin.js:1157` | S10 Integrations | covered | `integrations` · button · "Review import" | Review import shows replacement, blocked items and network effects with the backup box; Start opens the job (D107) (workspace-system-integrations.spec.mjs) |
|
||||
| Statistics | `admin.html:159`; `admin.js:1627` | S2 Overview | covered | `overview` · button · "Explore" | statistics dialog lists runs by category and library totals (workspace-system.spec.mjs) |
|
||||
| Statistics | `admin.html:159`; `admin.js:1627` | S2 Overview | covered | `overview` · button · "Explore" | statistics dialog lists runs by category and database record counts (workspace-system.spec.mjs) |
|
||||
| Export components JSON `/export` | `admin.html:163`; `admin.js:1607`; `py/api/prompts.py:238` | S9 Backup & export | covered | `backups` · button · "Export all prompts JSON" | Export JSON downloads /library/export; secondary buttons download all prompts (/export) and /pairs/export (D103) (workspace-system-backups.spec.mjs) |
|
||||
| Saved pairs export | `py/api/pairs.py:84` | S9 Backup & export | covered | `backups` · button · "Export saved pairs JSON" | Export JSON downloads /library/export; secondary buttons download all prompts (/export) and /pairs/export (D103) (workspace-system-backups.spec.mjs) |
|
||||
| Library export | `py/api/library.py:223` | S9 Backup & export | covered | `backups` · button · "Export JSON" | Export JSON downloads /library/export; secondary buttons download all prompts (/export) and /pairs/export (D103) (workspace-system-backups.spec.mjs) |
|
||||
|
||||
+6
-9
@@ -435,12 +435,9 @@ except Exception as e:
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
||||
|
||||
# Print startup message with loaded tools
|
||||
print()
|
||||
print(f"\033[94m[ComfyUI-PromptManager] Version:\033[0m {get_version()}")
|
||||
for node_key, display_name in NODE_DISPLAY_NAME_MAPPINGS.items():
|
||||
# ASCII only: a Windows console using a legacy code page cannot print emoji,
|
||||
# and a UnicodeEncodeError here would abort loading the whole node.
|
||||
print(f" \033[94mLoaded:\033[0m {display_name}")
|
||||
print(f"\033[94mTotal: {len(NODE_CLASS_MAPPINGS)} tools loaded\033[0m")
|
||||
print()
|
||||
# One startup line. ASCII only: a Windows console using a legacy code page
|
||||
# cannot print emoji, and a UnicodeEncodeError here would abort loading the node.
|
||||
print(
|
||||
f"\033[94m[ComfyUI-PromptManager] Version:\033[0m {get_version()}"
|
||||
f" - {len(NODE_CLASS_MAPPINGS)} tools loaded"
|
||||
)
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ description = "A powerful ComfyUI custom node that extends the standard text enc
|
||||
version = "3.4.1"
|
||||
license = {file = "LICENSE"}
|
||||
requires-python = ">=3.12"
|
||||
dependencies = ["watchdog>=2.1.0,<7", "Pillow>=8.0.0"]
|
||||
dependencies = ["watchdog>=2.1.0", "Pillow>=8.0.0"]
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=84.2.1"]
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Runtime dependencies
|
||||
watchdog>=6.0.0,<7
|
||||
watchdog>=6.0.0
|
||||
Pillow>=12.3.0
|
||||
aiohttp>=3.14.5
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Runtime dependencies for PromptManager.
|
||||
# aiohttp, numpy and torch come from ComfyUI itself and are not listed here.
|
||||
watchdog>=2.1.0,<7 # Output directory monitoring
|
||||
watchdog>=2.1.0 # Output directory monitoring
|
||||
Pillow>=8.0.0 # Image metadata extraction and thumbnails
|
||||
|
||||
@@ -162,13 +162,15 @@ async function serverJson(request, url) {
|
||||
}
|
||||
|
||||
const number = value => value.toLocaleString('en-US');
|
||||
// The sidebar's Images count (state.mjs), which System's Images metric shows (#555).
|
||||
const IMAGES_COUNT_URL = '/prompt_manager/images/output?mode=workspace&limit=1';
|
||||
|
||||
/** The concept's shell and Overview with the recorded deviations applied, given the server's real overview. */
|
||||
function mapOverviewConcept(reference, overview) {
|
||||
function mapOverviewConcept(reference, overview, images) {
|
||||
// D165: the discreet legacy-pages line follows the settings layout.
|
||||
reference = { ...reference, tree: [...reference.tree, 'p.legacy-pages.small'] };
|
||||
const relabel = { 'Database integrity': 'Not checked', 'Positive prompts': number(overview.library.positive_prompts),
|
||||
'Negative variants': number(overview.library.negative_variants), Images: number(overview.images.indexed) };
|
||||
'Negative variants': number(overview.library.negative_variants), Images: number(images) };
|
||||
const rows = {
|
||||
// D93: the real pending count; Review metadata opens Needs review (the concept's go-review).
|
||||
'3 ambiguous image links': row => ({ ...row,
|
||||
@@ -179,8 +181,9 @@ function mapOverviewConcept(reference, overview) {
|
||||
'One display configuration': row => ({ ...row, description: 'Grid and density settings are shared across views.' }),
|
||||
// D93: the real sources, not samples.
|
||||
'2 sample output sources': row => ({ ...row, title: `${number(overview.sources.effective)} output sources` }),
|
||||
// D96: the dialog shows runs by category and library totals, not prompt, tag or date usage (D153: no ratings).
|
||||
Statistics: row => ({ ...row, description: 'Runs by category, with library totals.' }),
|
||||
// D96: the dialog shows runs by category and database record counts, not prompt, tag or date usage (D153: no
|
||||
// ratings; #555: the counts are labelled as records, not as the Library or Images page counts).
|
||||
Statistics: row => ({ ...row, description: 'Runs by category, with database record counts.' }),
|
||||
};
|
||||
const surfaces = reference.columns.surfaces.map(surface => ({ ...surface,
|
||||
rows: surface.rows.map(row => (rows[row.title] ?? (value => value))(row)) }));
|
||||
@@ -218,6 +221,8 @@ async function conceptOverview(page, theme) {
|
||||
async function productionOverview(page, theme) {
|
||||
await openProduction(page, 'system/overview', theme);
|
||||
await expect(page.locator('#workspace-main .summary-metrics')).toBeVisible();
|
||||
// #555: the Images metric is the shell's Images count: compare once it was read.
|
||||
await expect(page.locator('#workspace-main .summary-metrics > .metric:nth-child(3) strong')).toHaveText(/^[\d,]+$/);
|
||||
// The thumbnail row is its own read (I2): compare once it answered.
|
||||
await expect(page.locator('#workspace-main .operation h3').filter({ hasText: 'thumbnails' })).toHaveText('No missing thumbnails');
|
||||
// So is the backup row (S9).
|
||||
@@ -231,6 +236,7 @@ for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`system shell and overview match the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
const overview = await serverJson(request, '/prompt_manager/system/overview');
|
||||
const images = (await serverJson(request, IMAGES_COUNT_URL)).total;
|
||||
// The fixture's thumbnails all exist, so the concept's "No missing thumbnails" is also the real title.
|
||||
expect((await serverJson(request, '/prompt_manager/images/thumbnails/status')).pending).toBe(0);
|
||||
const concept = await conceptOverview(page, theme);
|
||||
@@ -239,7 +245,7 @@ for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-system-${theme}-${width}.png`), fullPage: true });
|
||||
const production = await productionOverview(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-system-${theme}-${width}.png`), fullPage: true });
|
||||
expect(diffInventory(mapOverviewConcept(concept, overview), production)).toEqual([]);
|
||||
expect(diffInventory(mapOverviewConcept(concept, overview, images), production)).toEqual([]);
|
||||
expect(production.pageScroll).toBe(false);
|
||||
});
|
||||
}
|
||||
@@ -265,8 +271,9 @@ const UNMAPPED_PATHS = [
|
||||
test('system comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const overview = await serverJson(request, '/prompt_manager/system/overview');
|
||||
const images = (await serverJson(request, IMAGES_COUNT_URL)).total;
|
||||
const concept = await conceptOverview(page, 'dark');
|
||||
const expected = mapOverviewConcept(concept, overview);
|
||||
const expected = mapOverviewConcept(concept, overview, images);
|
||||
const production = await productionOverview(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the concept differs exactly where a deviation is recorded, and nowhere else.
|
||||
@@ -290,7 +297,7 @@ test('system comparison detects mutations', async ({ page, request }) => {
|
||||
});
|
||||
|
||||
/**
|
||||
* D96: the concept's table, its rows the real ones; then "Library totals" in the same table markup. D153 (#447): the
|
||||
* D96: the concept's table, its rows the real ones; then "Database records" (#555) in the same table markup. D153 (#447): the
|
||||
* concept's "Avg. rating" column and a "Rated images" total are not drawn.
|
||||
*/
|
||||
function mapStatisticsConcept(reference, statistics) {
|
||||
@@ -302,9 +309,10 @@ function mapStatisticsConcept(reference, statistics) {
|
||||
tables: [
|
||||
{ ...runs, head: runs.head.filter(([, , text]) => text !== 'Avg. rating'),
|
||||
rows: statistics.by_category.map(row => [row.category ?? 'Uncategorized', number(row.runs)]) },
|
||||
{ caption: 'Library totals', head: [['th', 'col', 'Total'], ['th', 'col', 'Count']],
|
||||
rows: [['Prompts', number(statistics.totals.total_prompts)], ['Tags', number(statistics.totals.total_tags)],
|
||||
['Images', number(statistics.totals.total_images)]] },
|
||||
{ caption: 'Database records', head: [['th', 'col', 'Total'], ['th', 'col', 'Count']],
|
||||
rows: [['Prompt records', number(statistics.totals.total_prompts)],
|
||||
['Tags on prompts', number(statistics.totals.total_tags)],
|
||||
['Image records', number(statistics.totals.total_images)]] },
|
||||
],
|
||||
};
|
||||
}
|
||||
@@ -420,7 +428,7 @@ for (const theme of THEMES) test(`statistics and health dialogs match the concep
|
||||
expect(diffInventory(mapHealthConcept(concept.health), await page.evaluate(measureDialog)).map(item => item.path))
|
||||
.toEqual(['operations.rows.0.title']);
|
||||
await page.evaluate(() => { document.querySelector('#modal .operation h3').textContent = 'Integrity check'; });
|
||||
// And a removed "Library totals" table and a changed caption in the statistics dialog.
|
||||
// And a removed "Database records" table and a changed caption in the statistics dialog.
|
||||
await page.keyboard.press('Escape');
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await page.locator('#workspace-main').getByRole('button', { name: 'Explore', exact: true }).click();
|
||||
|
||||
@@ -8,6 +8,8 @@ import { readCoverageTable, parseControl, CONTROL_EXEMPTIONS } from '../../tools
|
||||
const workspace = '/prompt_manager/workspace';
|
||||
const OVERVIEW_URL = '/prompt_manager/system/overview';
|
||||
const THUMBNAILS_URL = '/prompt_manager/images/thumbnails/status';
|
||||
// The sidebar's Images count (state.mjs NAV_COUNT_SOURCES): the Images page's own total.
|
||||
const IMAGES_COUNT_URL = '/prompt_manager/images/output?mode=workspace&limit=1';
|
||||
const number = value => value.toLocaleString('en-US');
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
@@ -55,6 +57,7 @@ test('overview shows real counts and Not checked', async ({ page, request }) =>
|
||||
page.on('request', item => { if (!['GET', 'HEAD', 'OPTIONS'].includes(item.method())) writes.push(item.url()); });
|
||||
const overview = await serverJson(request, OVERVIEW_URL);
|
||||
const thumbnails = await serverJson(request, THUMBNAILS_URL);
|
||||
const images = (await serverJson(request, IMAGES_COUNT_URL)).total;
|
||||
// The concept fixture: six positives, their saved negatives and 24 linked images, all with thumbnails.
|
||||
expect([overview.library.positive_prompts, overview.images.indexed, thumbnails.pending]).toEqual([6, 24, 0]);
|
||||
// I2: the Overview never walks the sources for thumbnails; the row reads the status route itself.
|
||||
@@ -66,7 +69,8 @@ test('overview shows real counts and Not checked', async ({ page, request }) =>
|
||||
[item.querySelector('strong').textContent, item.querySelector('span').textContent]))).toEqual([
|
||||
[number(overview.library.positive_prompts), 'Positive prompts'],
|
||||
[number(overview.library.negative_variants), 'Negative variants'],
|
||||
[number(overview.images.indexed), 'Images'],
|
||||
// #555: the Images page's own count, the same number as its sidebar badge.
|
||||
[number(images), 'Images'],
|
||||
['Not checked', 'Database integrity'],
|
||||
]);
|
||||
expect(await rowsOf(layout, 'Library health')).toEqual([
|
||||
@@ -79,11 +83,31 @@ test('overview shows real counts and Not checked', async ({ page, request }) =>
|
||||
expect(await rowsOf(layout, 'Workspace preferences')).toEqual([
|
||||
['One display configuration', 'Grid and density settings are shared across views.', 'Appearance'],
|
||||
['2 output sources', 'Monitoring and enrichment share one source list.', 'Manage sources'],
|
||||
['Statistics', 'Runs by category, with library totals.', 'Explore'],
|
||||
['Statistics', 'Runs by category, with database record counts.', 'Explore'],
|
||||
]);
|
||||
expect(writes).toEqual([]);
|
||||
});
|
||||
|
||||
test('#555: the Images metric is the sidebar Images count, not the indexed image rows', async ({ page, request }) => {
|
||||
const overview = await serverJson(request, OVERVIEW_URL);
|
||||
// The Images page lists files no row links (the demo library: 122 files, 110 rows); the listing says so.
|
||||
const listed = overview.images.indexed + 12;
|
||||
await page.route(`**${IMAGES_COUNT_URL}`, async route => {
|
||||
const response = await route.fetch();
|
||||
await route.fulfill({ response, json: { ...(await response.json()), total: listed } });
|
||||
});
|
||||
const layout = await openOverview(page);
|
||||
const imagesMetric = layout.locator('.summary-metrics > .metric').filter({ has: page.locator('span', { hasText: /^Images$/ }) });
|
||||
await expect(imagesMetric.locator('strong')).toHaveText(number(listed));
|
||||
await expect(page.locator('#primary-nav a[data-route="images"] .nav-count')).toHaveText(number(listed));
|
||||
// A failed count reads as a dash, never the row count.
|
||||
await page.unroute(`**${IMAGES_COUNT_URL}`);
|
||||
await page.route(`**${IMAGES_COUNT_URL}`, route => route.fulfill({ status: 503, json: { success: false, error: 'Unavailable' } }));
|
||||
await page.goto(`${workspace}#/system/appearance`);
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await expect(imagesMetric.locator('strong')).toHaveText('–');
|
||||
});
|
||||
|
||||
// D129 (#249 S6b): where the library lives. The hook makes the fixture's database read as a node-folder library; its
|
||||
// Move library is a stand-in that touches no file (workspace_system_fixture.py).
|
||||
const banner = page => page.locator('#library-banner');
|
||||
@@ -364,7 +388,7 @@ test('View backups / Appearance / Manage sources switch sections', async ({ page
|
||||
}
|
||||
});
|
||||
|
||||
test('statistics dialog lists runs by category and library totals', async ({ page, request }) => {
|
||||
test('statistics dialog lists runs by category and database record counts', async ({ page, request }) => {
|
||||
const statistics = await serverJson(request, '/prompt_manager/system/statistics');
|
||||
expect(statistics.by_category.length).toBeGreaterThan(1);
|
||||
const layout = await openOverview(page);
|
||||
@@ -373,13 +397,14 @@ test('statistics dialog lists runs by category and library totals', async ({ pag
|
||||
const dialog = page.locator('dialog#modal[open]');
|
||||
await expect(dialog.locator('#modal-title')).toHaveText('Usage statistics');
|
||||
const tables = dialog.locator('#modal-body > table');
|
||||
await expect(tables.locator('caption')).toHaveText(['Runs by category', 'Library totals']);
|
||||
await expect(tables.locator('caption')).toHaveText(['Runs by category', 'Database records']);
|
||||
// D153 (#447): no rating column and no rated-images total.
|
||||
await expect(tables.first().locator('thead th')).toHaveText(['Category', 'Runs']);
|
||||
const cells = table => table.locator('tbody tr').evaluateAll(rows => rows.map(row => [...row.cells].map(cell => cell.textContent)));
|
||||
expect(await cells(tables.first())).toEqual(statistics.by_category.map(row => [row.category ?? 'Uncategorized', number(row.runs)]));
|
||||
expect(await cells(tables.nth(1))).toEqual([['Prompts', number(statistics.totals.total_prompts)],
|
||||
['Tags', number(statistics.totals.total_tags)], ['Images', number(statistics.totals.total_images)]]);
|
||||
// #555: raw rows of the legacy /stats, labelled as rows so they never read as the Library or Images page counts.
|
||||
expect(await cells(tables.nth(1))).toEqual([['Prompt records', number(statistics.totals.total_prompts)],
|
||||
['Tags on prompts', number(statistics.totals.total_tags)], ['Image records', number(statistics.totals.total_images)]]);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(explore).toBeFocused();
|
||||
// A statistics read that fails says so; the Overview stays.
|
||||
|
||||
@@ -145,9 +145,10 @@ test('long vocabularies append and Load more is keyboard reachable',async({page,
|
||||
await expect(loadMore(page)).toHaveCount(0);
|
||||
await expect(rows(page).nth(200).getByRole('button',{name:'Browse',exact:true})).toBeFocused();
|
||||
await expect(rows(page).locator('td:nth-child(1)')).toHaveText(manifest.tags);
|
||||
// Reload with the window at its top: Firefox restores a scrolled window, whose end then appends a page before the check.
|
||||
await main(page).locator('.two-col .table-wrap').evaluate(node=>new Promise(resolve=>{node.scrollTop=0;requestAnimationFrame(()=>requestAnimationFrame(resolve));}));
|
||||
await page.reload();
|
||||
// Open the screen as a new history entry, not a reload: Firefox restores a reloaded page's scrolled tag window, and
|
||||
// its end then appends a page before the check (#567). Scrolling the window to its top first does not always stick.
|
||||
await page.goto('about:blank');
|
||||
await page.goto(route);
|
||||
await expect(rows(page)).toHaveCount(100);
|
||||
// D183: the end of the list arrives inside the tag window.
|
||||
const scroll=top=>main(page).locator('.two-col .table-wrap').evaluate((node,y)=>new Promise(resolve=>{node.scrollTop=y ?? node.scrollHeight;requestAnimationFrame(()=>requestAnimationFrame(resolve));}),top);
|
||||
|
||||
@@ -16,12 +16,22 @@ const OVERVIEW = Object.freeze({
|
||||
|
||||
test('metrics are the server counts and Not checked before any health check', async () => {
|
||||
const { metricValues } = await overviewModule;
|
||||
assert.deepEqual(metricValues(OVERVIEW), [
|
||||
assert.deepEqual(metricValues(OVERVIEW, 24), [
|
||||
[(1234).toLocaleString(), 'Positive prompts'], ['7', 'Negative variants'], ['24', 'Images'],
|
||||
['Not checked', 'Database integrity'],
|
||||
]);
|
||||
});
|
||||
|
||||
test('#555: the Images metric is the Images page count (the sidebar badge), not the indexed rows', async () => {
|
||||
const { metricValues } = await overviewModule;
|
||||
// The demo library: 110 indexed rows, 122 files on the Images page (unlinked ones included).
|
||||
const overview = { ...OVERVIEW, images: { indexed: 110, rated: 0 } };
|
||||
assert.deepEqual(metricValues(overview, 122)[2], [(122).toLocaleString(), 'Images']);
|
||||
// Not read yet (undefined) or failed (null): a dash, as on the workspace Overview, never the row count.
|
||||
assert.deepEqual(metricValues(overview, undefined)[2], ['–', 'Images']);
|
||||
assert.deepEqual(metricValues(overview, null)[2], ['–', 'Images']);
|
||||
});
|
||||
|
||||
test('metric 4 is the last health result', async () => {
|
||||
const { integrityText } = await overviewModule;
|
||||
assert.equal(integrityText(null), 'Not checked');
|
||||
@@ -150,7 +160,7 @@ test('statistics rows are the real runs, then the library totals, without rating
|
||||
rated_images: 20,
|
||||
};
|
||||
assert.deepEqual(categoryRows(statistics), [['Landscapes', (1200).toLocaleString()], ['Uncategorized', '3']]);
|
||||
assert.deepEqual(totalsRows(statistics), [['Prompts', '12'], ['Tags', '15'], ['Images', '24']]);
|
||||
assert.deepEqual(totalsRows(statistics), [['Prompt records', '12'], ['Tags on prompts', '15'], ['Image records', '24']]);
|
||||
});
|
||||
|
||||
/** A fake api answering each `get`/`post` from a queue per URL, recording the calls. */
|
||||
|
||||
@@ -392,7 +392,10 @@ class TestStartStop(unittest.TestCase):
|
||||
self.assertLogs("prompt_manager.image_monitor", "WARNING") as logs,
|
||||
):
|
||||
self.assertFalse(self.monitor.stop_monitoring(timeout=5.0))
|
||||
self.assertEqual(fake.calls[-1], ("join", 0.0))
|
||||
# The stop runs on its own thread and gets 0 s too: it may not have
|
||||
# returned yet, and then nothing is joined at all.
|
||||
joins = [call for call in fake.calls if call[0] == "join"]
|
||||
self.assertTrue(all(call == ("join", 0.0) for call in joins), fake.calls)
|
||||
self.assertIn(
|
||||
"watchdog observer still running after 0.0 s", "\n".join(logs.output)
|
||||
)
|
||||
@@ -401,6 +404,80 @@ class TestStartStop(unittest.TestCase):
|
||||
def test_stop_monitoring_without_an_observer_returns_true(self):
|
||||
self.assertTrue(self.monitor.stop_monitoring())
|
||||
|
||||
def test_threads_the_observer_starts_are_daemons(self):
|
||||
# watchdog 7 on Windows starts a plain threading.Thread per watch in
|
||||
# observer.start(); a non-daemon one holds the interpreter's exit
|
||||
# before any atexit callback runs (#561).
|
||||
fake = ReaderObserver()
|
||||
self.addCleanup(fake.release.set)
|
||||
self._start_with(fake)
|
||||
self.assertIsNotNone(fake.reader)
|
||||
self.assertTrue(fake.reader.daemon)
|
||||
self.assertFalse(threading.current_thread().daemon)
|
||||
|
||||
def test_an_observer_that_fails_to_start_on_its_thread_is_reported(self):
|
||||
fake = FakeObserver()
|
||||
fake.start = MagicMock(side_effect=OSError("handle refused"))
|
||||
with self.assertLogs("prompt_manager.image_monitor", "ERROR") as logs:
|
||||
with patch.object(im_mod, "Observer", return_value=fake):
|
||||
self.monitor.start_monitoring([self.tmp.name])
|
||||
self.assertIn("handle refused", "\n".join(logs.output))
|
||||
self.assertFalse(self.monitor.is_monitoring)
|
||||
|
||||
def test_an_observer_stop_that_never_returns_is_bounded(self):
|
||||
# watchdog's observer.stop() joins its emitters (and, in 7.x on
|
||||
# Windows, their reader threads) without a limit (#561).
|
||||
fake = StuckStopObserver()
|
||||
self.addCleanup(fake.release.set)
|
||||
self._start_with(fake)
|
||||
with self.assertLogs("prompt_manager.image_monitor", "WARNING") as logs:
|
||||
self.assertFalse(self.monitor.stop_monitoring(timeout=0.05))
|
||||
self.assertIn("watchdog observer still running", "\n".join(logs.output))
|
||||
self.assertTrue(fake.entered.wait(30))
|
||||
self.assertFalse(self.monitor.is_monitoring)
|
||||
# A second stop waits for the stop already running; it never calls
|
||||
# observer.stop() a second time.
|
||||
fake.release.set()
|
||||
self.assertTrue(self.monitor.stop_monitoring())
|
||||
self.assertEqual(fake.stop_calls, 1)
|
||||
self.assertFalse(self.monitor._unfinished)
|
||||
|
||||
|
||||
class ReaderObserver(FakeObserver):
|
||||
"""A FakeObserver whose start() starts a plain thread, as watchdog 7's
|
||||
Windows emitter does for its ReadDirectoryChangesW reader."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.release = threading.Event()
|
||||
self.reader = None
|
||||
|
||||
def start(self):
|
||||
self.reader = threading.Thread(target=self.release.wait)
|
||||
self.reader.start()
|
||||
super().start()
|
||||
|
||||
def stop(self):
|
||||
self.release.set()
|
||||
self.reader.join(30)
|
||||
super().stop()
|
||||
|
||||
|
||||
class StuckStopObserver(FakeObserver):
|
||||
"""A FakeObserver whose stop() does not return until ``release``."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.entered = threading.Event()
|
||||
self.release = threading.Event()
|
||||
self.stop_calls = 0
|
||||
|
||||
def stop(self):
|
||||
self.stop_calls += 1
|
||||
self.entered.set()
|
||||
self.release.wait(30)
|
||||
super().stop()
|
||||
|
||||
|
||||
class FakeEvent:
|
||||
def __init__(self, path, is_directory=False):
|
||||
|
||||
@@ -136,6 +136,11 @@ def cleanup_package(package, require_stopped=True):
|
||||
if registration is not None:
|
||||
assert registration.dispose(3), "package history worker leaked"
|
||||
close_every_database()
|
||||
# The package's exit actions outlive it otherwise: at interpreter exit
|
||||
# they would import from a package no longer in sys.modules (#580).
|
||||
exit_signal = getattr(package, "_exit_signal", None)
|
||||
if exit_signal is not None:
|
||||
exit_signal.reset()
|
||||
sys.modules.pop(package.__name__, None)
|
||||
|
||||
|
||||
@@ -390,7 +395,39 @@ class TestConsoleBanner(unittest.TestCase):
|
||||
|
||||
stream.flush()
|
||||
banner = stream.buffer.getvalue().decode("cp1252")
|
||||
self.assertIn("Loaded:", banner)
|
||||
lines = [
|
||||
line
|
||||
for line in banner.splitlines()
|
||||
if "PromptManager" in line or "loaded" in line
|
||||
]
|
||||
self.assertEqual(len(lines), 1, banner)
|
||||
self.assertRegex(
|
||||
lines[0], r"\[ComfyUI-PromptManager\] Version:.* \S+ - \d+ tools loaded$"
|
||||
)
|
||||
self.assertNotIn("Loaded:", banner)
|
||||
|
||||
|
||||
class TestFixtureExitActions(unittest.TestCase):
|
||||
"""A disposed fixture package runs nothing at interpreter exit (#580)."""
|
||||
|
||||
def test_a_disposed_package_leaves_no_exit_action(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.addCleanup(close_every_database)
|
||||
output_dir = os.path.join(tmp.name, "output")
|
||||
os.makedirs(output_dir)
|
||||
name = "prompt_manager_pkg_exit_actions"
|
||||
self.addCleanup(sys.modules.pop, name, None)
|
||||
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
with load_package(name, os.path.join(tmp.name, "p.db"), output_dir) as (
|
||||
package,
|
||||
_,
|
||||
):
|
||||
signal = package._exit_signal
|
||||
self.assertTrue(signal._on_exit, "the package installs exit actions")
|
||||
|
||||
self.assertEqual(signal._on_exit, [])
|
||||
|
||||
|
||||
class PriorFileHandler(logging.FileHandler):
|
||||
@@ -553,7 +590,7 @@ class TestPackageLoggerIsolation(unittest.TestCase):
|
||||
original_print = print
|
||||
|
||||
def fail_banner(*args, **kwargs):
|
||||
if any("Total:" in str(arg) for arg in args):
|
||||
if any("tools loaded" in str(arg) for arg in args):
|
||||
captured["package"] = sys.modules[name]
|
||||
captured["handlers"] = [
|
||||
h
|
||||
@@ -587,13 +624,16 @@ PACKAGE_THREAD_NAMES = ("PromptManagerPromptCleanup", "PromptManagerImageWorker"
|
||||
|
||||
def package_threads():
|
||||
"""Live threads this design owns: the tracker, the image worker, and the
|
||||
watchdog observer and emitters (which have no name of their own)."""
|
||||
watchdog observer and emitters (which have no name of their own).
|
||||
watchdog 7's InotifyFD, one daemon reader per process that no observer
|
||||
stop ends, is not the package's."""
|
||||
from watchdog.utils import BaseThread
|
||||
|
||||
return {
|
||||
thread
|
||||
for thread in threading.enumerate()
|
||||
if thread.name in PACKAGE_THREAD_NAMES or isinstance(thread, BaseThread)
|
||||
if thread.name in PACKAGE_THREAD_NAMES
|
||||
or (isinstance(thread, BaseThread) and type(thread).__name__ != "InotifyFD")
|
||||
}
|
||||
|
||||
|
||||
|
||||
+107
-44
@@ -31,22 +31,65 @@ CHILD = textwrap.dedent("""
|
||||
import threading
|
||||
|
||||
root, tmp = sys.argv[1], sys.argv[2]
|
||||
stuck = sys.argv[3:] == ["stuck-observer"]
|
||||
sys.path.insert(0, root)
|
||||
state = {}
|
||||
|
||||
# watchdog 7's Windows observer at its worst (#561): start() starts a
|
||||
# plain reader thread, and stop() waits for it, which never ends.
|
||||
class StuckObserver:
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
self._never = threading.Event()
|
||||
self._reader = None
|
||||
|
||||
def schedule(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def start(self):
|
||||
self._reader = threading.Thread(
|
||||
target=self._never.wait, name="StuckReader"
|
||||
)
|
||||
self._reader.start()
|
||||
|
||||
def stop(self):
|
||||
self._reader.join()
|
||||
|
||||
def join(self, timeout=None):
|
||||
self._reader.join(timeout)
|
||||
|
||||
def is_alive(self):
|
||||
return self._reader.is_alive()
|
||||
|
||||
if stuck:
|
||||
import watchdog.observers
|
||||
|
||||
watchdog.observers.Observer = StuckObserver
|
||||
|
||||
def report():
|
||||
from watchdog.utils import BaseThread
|
||||
|
||||
package = state.get("package")
|
||||
result = getattr(package, "_background_work_result", None)
|
||||
# The image worker, the tracker, and watchdog's observer and emitters.
|
||||
# watchdog 7's InotifyFD is one daemon reader per process that no
|
||||
# observer stop ends, by design; it is not the package's.
|
||||
alive = sorted(
|
||||
t.name
|
||||
for t in threading.enumerate()
|
||||
if t.name in ("PromptManagerImageWorker", "PromptManagerPromptCleanup")
|
||||
or isinstance(t, BaseThread)
|
||||
or (isinstance(t, BaseThread) and type(t).__name__ != "InotifyFD")
|
||||
)
|
||||
# Threads that would hold the exit before any atexit callback (#561).
|
||||
holding = sorted(
|
||||
t.name
|
||||
for t in threading.enumerate()
|
||||
if not t.daemon and t is not threading.main_thread()
|
||||
)
|
||||
print(
|
||||
"PM_EXIT_REPORT "
|
||||
+ json.dumps({"result": result, "alive": alive, "holding": holding})
|
||||
)
|
||||
print("PM_EXIT_REPORT " + json.dumps({"result": result, "alive": alive}))
|
||||
sys.stdout.flush()
|
||||
|
||||
# First registered, so it runs last: after the package's teardown and
|
||||
@@ -64,6 +107,10 @@ CHILD = textwrap.dedent("""
|
||||
fake_server_module(queue_task_done=True),
|
||||
)
|
||||
state["package"] = package
|
||||
if stuck:
|
||||
# The stop gives up after this long instead of the full 5 s.
|
||||
monitor_module = sys.modules[package.__name__ + ".utils.image_monitor"]
|
||||
monitor_module.OBSERVER_JOIN_SECONDS = 0.5
|
||||
monitor = package._global_image_monitor
|
||||
handler = monitor.handler if monitor is not None else None
|
||||
if handler is None:
|
||||
@@ -78,47 +125,52 @@ CHILD = textwrap.dedent("""
|
||||
""")
|
||||
|
||||
|
||||
def run_child(testcase, *args):
|
||||
"""Run CHILD with ``args``; returns (completed, report, detail)."""
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
os.makedirs(os.path.join(tmp, "output"))
|
||||
script = os.path.join(tmp, "child.py")
|
||||
with open(script, "w", encoding="utf-8") as handle:
|
||||
handle.write(CHILD)
|
||||
env = dict(os.environ)
|
||||
env.pop("PROMPT_MANAGER_CONFIG_PATH", None)
|
||||
# The child's library and log file live in its own folder, never
|
||||
# in the checkout's logs/ (which this runner may hold open) or a
|
||||
# data folder the caller set.
|
||||
env["PROMPT_MANAGER_DATA_DIR"] = tmp
|
||||
env["PYTHONIOENCODING"] = "utf-8"
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, script, ROOT, tmp, *args],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
timeout=CHILD_CEILING_SECONDS,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
except subprocess.TimeoutExpired as hung:
|
||||
testcase.fail(
|
||||
f"the child did not exit within {CHILD_CEILING_SECONDS} s\n"
|
||||
f"stdout:\n{hung.stdout}\nstderr:\n{hung.stderr}"
|
||||
)
|
||||
# Leaving the block removes the folder: on Windows that fails
|
||||
# if the child left a handle open, which it cannot once it exited,
|
||||
# so the check here is that the child exited at all.
|
||||
detail = f"stdout:\n{completed.stdout}\nstderr:\n{completed.stderr}"
|
||||
testcase.assertEqual(completed.returncode, 0, detail)
|
||||
reports = [
|
||||
line[len(REPORT_PREFIX) :]
|
||||
for line in completed.stdout.splitlines()
|
||||
if line.startswith(REPORT_PREFIX)
|
||||
]
|
||||
testcase.assertEqual(len(reports), 1, detail)
|
||||
return completed, json.loads(reports[0]), detail
|
||||
|
||||
|
||||
class TestPackageExit(unittest.TestCase):
|
||||
def test_teardown_runs_at_a_normal_exit_with_the_worker_alive(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
os.makedirs(os.path.join(tmp, "output"))
|
||||
script = os.path.join(tmp, "child.py")
|
||||
with open(script, "w", encoding="utf-8") as handle:
|
||||
handle.write(CHILD)
|
||||
env = dict(os.environ)
|
||||
env.pop("PROMPT_MANAGER_CONFIG_PATH", None)
|
||||
# The child's library and log file live in its own folder, never
|
||||
# in the checkout's logs/ (which this runner may hold open) or a
|
||||
# data folder the caller set.
|
||||
env["PROMPT_MANAGER_DATA_DIR"] = tmp
|
||||
env["PYTHONIOENCODING"] = "utf-8"
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, script, ROOT, tmp],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
timeout=CHILD_CEILING_SECONDS,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
except subprocess.TimeoutExpired as hung:
|
||||
self.fail(
|
||||
f"the child did not exit within {CHILD_CEILING_SECONDS} s\n"
|
||||
f"stdout:\n{hung.stdout}\nstderr:\n{hung.stderr}"
|
||||
)
|
||||
# Leaving the block removes the folder: on Windows that fails
|
||||
# if the child left a handle open, which it cannot once it exited,
|
||||
# so the check here is that the child exited at all.
|
||||
detail = f"stdout:\n{completed.stdout}\nstderr:\n{completed.stderr}"
|
||||
self.assertEqual(completed.returncode, 0, detail)
|
||||
reports = [
|
||||
line[len(REPORT_PREFIX) :]
|
||||
for line in completed.stdout.splitlines()
|
||||
if line.startswith(REPORT_PREFIX)
|
||||
]
|
||||
self.assertEqual(len(reports), 1, detail)
|
||||
report = json.loads(reports[0])
|
||||
completed, report, detail = run_child(self)
|
||||
# "thumbnails" is True here because no preview worker started; its
|
||||
# close result is tested in test_thumbnail_auto and test_init_package.
|
||||
self.assertEqual(
|
||||
@@ -127,6 +179,7 @@ class TestPackageExit(unittest.TestCase):
|
||||
detail,
|
||||
)
|
||||
self.assertEqual(report["alive"], [], detail)
|
||||
self.assertEqual(report["holding"], [], detail)
|
||||
# The teardown's own warnings. Import-time warnings (the API's
|
||||
# output-folder search finds no ComfyUI here) are not about exit.
|
||||
for text in (
|
||||
@@ -137,6 +190,16 @@ class TestPackageExit(unittest.TestCase):
|
||||
):
|
||||
self.assertNotIn(text, completed.stderr, detail)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
def test_an_observer_that_never_stops_does_not_hold_the_exit(self):
|
||||
# Before #561 the reader thread kept the interpreter waiting before
|
||||
# any atexit callback ran, so the teardown never started.
|
||||
completed, report, detail = run_child(self, "stuck-observer")
|
||||
self.assertEqual(
|
||||
report["result"],
|
||||
{"monitor": False, "thumbnails": True, "tracker": True},
|
||||
detail,
|
||||
)
|
||||
# The stuck reader is still alive, but as a daemon it holds nothing.
|
||||
self.assertEqual(report["holding"], [], detail)
|
||||
self.assertIn("watchdog observer still running", completed.stderr, detail)
|
||||
self.assertIn("Background work not stopped in time", completed.stderr, detail)
|
||||
|
||||
+87
-11
@@ -96,6 +96,65 @@ WORKER_JOIN_SECONDS = 5.0
|
||||
OBSERVER_JOIN_SECONDS = 5.0
|
||||
|
||||
|
||||
def _start_daemon_observer(observer) -> None:
|
||||
"""``observer.start()`` on a short-lived daemon thread (#561).
|
||||
|
||||
A thread is a daemon when the thread that creates it is one. watchdog
|
||||
7 on Windows starts a plain ``threading.Thread`` per watch inside
|
||||
``start()``; created from ComfyUI's main thread it is not a daemon, and
|
||||
the interpreter waits for every such thread before it runs any
|
||||
``atexit`` callback, so the teardown that would stop it never runs.
|
||||
Started from here, every thread the observer creates is a daemon.
|
||||
Waits for the start without a limit (start-up, not exit); re-raises
|
||||
what ``start()`` raised.
|
||||
"""
|
||||
raised: List[BaseException] = []
|
||||
|
||||
def start() -> None:
|
||||
try:
|
||||
observer.start()
|
||||
except BaseException as e: # re-raised on the caller's thread
|
||||
raised.append(e)
|
||||
|
||||
starter = threading.Thread(
|
||||
target=start, name="PromptManagerObserverStart", daemon=True
|
||||
)
|
||||
starter.start()
|
||||
starter.join()
|
||||
if raised:
|
||||
raise raised[0]
|
||||
|
||||
|
||||
class _ObserverStop:
|
||||
"""One ``observer.stop()`` running on a daemon thread (#561).
|
||||
|
||||
watchdog's ``stop()`` joins its emitters (and, in 7.x on Windows, their
|
||||
reader threads) without a limit. Run here, a stop that never returns
|
||||
holds neither the caller past its deadline nor the interpreter's exit.
|
||||
"""
|
||||
|
||||
def __init__(self, observer) -> None:
|
||||
self.error: Optional[Exception] = None
|
||||
self._thread = threading.Thread(
|
||||
target=self._run,
|
||||
args=(observer,),
|
||||
name="PromptManagerObserverStop",
|
||||
daemon=True,
|
||||
)
|
||||
self._thread.start()
|
||||
|
||||
def _run(self, observer) -> None:
|
||||
try:
|
||||
observer.stop()
|
||||
except Exception as e: # reported by the stop that waits for it
|
||||
self.error = e
|
||||
|
||||
def wait(self, timeout: float) -> bool:
|
||||
"""True once ``stop()`` has returned or raised."""
|
||||
self._thread.join(timeout)
|
||||
return not self._thread.is_alive()
|
||||
|
||||
|
||||
class ImageGenerationHandler(FileSystemEventHandler):
|
||||
"""Filesystem event handler for detecting new image generation.
|
||||
|
||||
@@ -1170,6 +1229,9 @@ class ImageMonitor:
|
||||
# (observer, handler) pairs a stop left running: the next stop waits
|
||||
# for them again instead of reporting True.
|
||||
self._unfinished: List[Tuple[Any, Any]] = []
|
||||
# Observer stops still running on their thread, by observer: a later
|
||||
# stop waits for the same call instead of starting another (#561).
|
||||
self._observer_stops: Dict[Any, _ObserverStop] = {}
|
||||
self.logger = get_logger("prompt_manager.image_monitor")
|
||||
self.review_recorder: Optional[Callable[[str], str]] = None
|
||||
self.thumbnail_queue: Optional[Callable[[str], str]] = None
|
||||
@@ -1273,7 +1335,7 @@ class ImageMonitor:
|
||||
self.logger.warning("No directory could be watched")
|
||||
return
|
||||
try:
|
||||
observer.start()
|
||||
_start_daemon_observer(observer)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Image monitoring could not start: {e}")
|
||||
return
|
||||
@@ -1303,12 +1365,12 @@ class ImageMonitor:
|
||||
|
||||
Closes the handler first (its worker is joined and its dispatch
|
||||
returns at once), then stops the watchdog observer and joins it. The
|
||||
worker and the observer share the one deadline: the observer's join
|
||||
gets what the worker left, possibly 0 s.
|
||||
watchdog's ``observer.stop()`` joins its emitters without a limit;
|
||||
it counts against the deadline but is not bounded here. A part an
|
||||
earlier call left running is waited for again. Safe to call when
|
||||
monitoring never started and more than once.
|
||||
worker and the observer share the one deadline: the observer's stop
|
||||
and join get what the worker left, possibly 0 s. watchdog's
|
||||
``observer.stop()`` joins its emitters without a limit, so it runs on
|
||||
a daemon thread and is waited for only until the deadline (#561). A
|
||||
part an earlier call left running is waited for again. Safe to call
|
||||
when monitoring never started and more than once.
|
||||
|
||||
Returns:
|
||||
True when every worker and observer it stopped has ended.
|
||||
@@ -1349,13 +1411,27 @@ class ImageMonitor:
|
||||
return False
|
||||
|
||||
def _stop_observer(self, observer, timeout: float) -> bool:
|
||||
"""``observer.stop()`` (safe to repeat), then a join of ``timeout``
|
||||
seconds; a join that times out is logged with the time it got."""
|
||||
"""``observer.stop()`` on its own daemon thread, then a join, both
|
||||
within ``timeout`` seconds; a stop or join still running then is
|
||||
logged with the time it got. A stop still running from an earlier
|
||||
call is waited for, never started again."""
|
||||
if observer is None:
|
||||
return True
|
||||
deadline = time.monotonic() + timeout
|
||||
stop = self._observer_stops.get(observer)
|
||||
if stop is None:
|
||||
stop = self._observer_stops[observer] = _ObserverStop(observer)
|
||||
if not stop.wait(timeout):
|
||||
self.logger.warning(
|
||||
f"watchdog observer still running after {timeout:.1f} s"
|
||||
)
|
||||
return False
|
||||
del self._observer_stops[observer]
|
||||
if stop.error is not None:
|
||||
self.logger.warning(f"Error while stopping image monitoring: {stop.error}")
|
||||
return False
|
||||
try:
|
||||
observer.stop()
|
||||
observer.join(timeout)
|
||||
observer.join(max(0.0, deadline - time.monotonic()))
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Error while stopping image monitoring: {e}")
|
||||
return False
|
||||
|
||||
@@ -176,6 +176,12 @@ function mountWorkspace() {
|
||||
function notifyCounts() { for (const listener of countListeners) listener(); }
|
||||
/** Re-read the given counts unless a read is already in flight. */
|
||||
function freshenCounts(ids) { refreshCounts(ids.filter(id => !countRequests.has(id))); }
|
||||
/** #555: the counts as System › Overview reads them, so its Images metric and the Images badge cannot differ. */
|
||||
const sharedCounts = Object.freeze({
|
||||
get: id => knownCounts.get(id),
|
||||
subscribe(listener) { countListeners.add(listener); return () => countListeners.delete(listener); },
|
||||
freshen: freshenCounts,
|
||||
});
|
||||
/** A count a page has just read itself: it replaces the badge, and an older read in flight no longer lands. */
|
||||
function setCount(id, count) {
|
||||
countRequests.get(id)?.abort();
|
||||
@@ -488,7 +494,7 @@ function mountWorkspace() {
|
||||
if (route.id === 'system') {
|
||||
// The concept's System (app.js renderSystem): its heading description, section nav and Save row.
|
||||
description.textContent = route.description;
|
||||
system = mountSystem(main, api, view, { section: route.section, activity });
|
||||
system = mountSystem(main, api, view, { section: route.section, activity, counts: sharedCounts });
|
||||
return;
|
||||
}
|
||||
if (route.id === 'inspect') {
|
||||
|
||||
@@ -11,8 +11,10 @@
|
||||
* D99: its Review thumbnails opens the same "Thumbnail maintenance" dialog as Sources (thumbnail-review.mjs).
|
||||
* #317: it reads "N missing · M outdated · K failed"; while previews are outdated (another format than the
|
||||
* configured one) a second button, "Convert outdated", opens that dialog on its Convert outdated scope (D131).
|
||||
* - D96: the Statistics dialog lists the real runs per category, then "Library totals" (D153: no rating column or
|
||||
* rated-images total).
|
||||
* - #555: metric 3, "Images", is the shell's Images count (the Images page's total, its sidebar badge), not the
|
||||
* overview's indexed rows, which leave out files no row links: a dash until it is read or when its read failed.
|
||||
* - D96: the Statistics dialog lists the real runs per category, then "Database records" (D153: no rating column or
|
||||
* rated-images total; #555: the legacy row counts, labelled as records).
|
||||
* - D95: the "Health report" lists every check the server ran, Missing files per image source, with a chip per real
|
||||
* result and when it was checked. The check runs from Maintenance's notice ("Run health checks", D123) as a
|
||||
* read job of the registry, tracked in the Activity store (its event stream); the report opens when it ends.
|
||||
@@ -61,12 +63,20 @@ export function integrityText(health) {
|
||||
return health.status === 'passed' ? 'Passed' : 'Incomplete';
|
||||
}
|
||||
|
||||
/** The concept's four metrics `[value, label]`, in its order. */
|
||||
export function metricValues(overview) {
|
||||
/** #555: the Images metric's value from the shell's Images count; a dash while unread (undefined) or failed (null). */
|
||||
export function imagesText(images) {
|
||||
return Number.isSafeInteger(images) && images >= 0 ? number(images) : '–';
|
||||
}
|
||||
|
||||
/**
|
||||
* The concept's four metrics `[value, label]`, in its order. `images` is the shell's Images count (#555): the concept's
|
||||
* `images.length`, the same number as the Images sidebar badge.
|
||||
*/
|
||||
export function metricValues(overview, images) {
|
||||
return [
|
||||
[number(overview.library.positive_prompts), 'Positive prompts'],
|
||||
[number(overview.library.negative_variants), 'Negative variants'],
|
||||
[number(overview.images.indexed), 'Images'],
|
||||
[imagesText(images), 'Images'],
|
||||
[integrityText(overview.health), 'Database integrity'],
|
||||
];
|
||||
}
|
||||
@@ -174,10 +184,15 @@ export function categoryRows(statistics) {
|
||||
return statistics.by_category.map(row => [row.category ?? 'Uncategorized', number(row.runs)]);
|
||||
}
|
||||
|
||||
/** D96: the added "Library totals" rows `[label, count]` (the legacy statistics; D153 drops the rated images). */
|
||||
/**
|
||||
* D96: the added "Database records" rows `[label, count]` (the legacy statistics; D153 drops the rated images). #555:
|
||||
* they count raw rows (every prompt row, tags on a prompt, every image row), so they say so and never read as the
|
||||
* Library or Images page counts.
|
||||
*/
|
||||
export function totalsRows(statistics) {
|
||||
const { totals } = statistics;
|
||||
return [['Prompts', totals.total_prompts], ['Tags', totals.total_tags], ['Images', totals.total_images]]
|
||||
return [['Prompt records', totals.total_prompts], ['Tags on prompts', totals.total_tags],
|
||||
['Image records', totals.total_images]]
|
||||
.map(([label, value]) => [label, number(value)]);
|
||||
}
|
||||
|
||||
@@ -208,11 +223,11 @@ function table(caption, head, rows, empty) {
|
||||
return node;
|
||||
}
|
||||
|
||||
/** The Statistics dialog: "Runs by category" (the concept's table), then "Library totals" (D96). */
|
||||
/** The Statistics dialog: "Runs by category" (the concept's table), then "Database records" (D96, #555). */
|
||||
export function statisticsNodes(statistics) {
|
||||
return [
|
||||
table('Runs by category', ['Category', 'Runs'], categoryRows(statistics), 'No prompts in the library yet.'),
|
||||
table('Library totals', ['Total', 'Count'], totalsRows(statistics), ''),
|
||||
table('Database records', ['Total', 'Count'], totalsRows(statistics), ''),
|
||||
];
|
||||
}
|
||||
|
||||
@@ -332,9 +347,9 @@ export function healthNotice({ api, activity, scope, navigate = navigateTo, noti
|
||||
return notice;
|
||||
}
|
||||
|
||||
function metricsNode(overview) {
|
||||
function metricsNode(overview, images) {
|
||||
const metrics = el('div', undefined, 'summary-metrics');
|
||||
for (const [value, label] of metricValues(overview)) {
|
||||
for (const [value, label] of metricValues(overview, images)) {
|
||||
const metric = el('div', undefined, 'metric');
|
||||
metric.append(el('strong', value), el('span', label));
|
||||
metrics.append(metric);
|
||||
@@ -349,7 +364,7 @@ function surface(heading, rows) {
|
||||
}
|
||||
|
||||
/** The Overview's nodes, and its thumbnail row (painted by its own read). */
|
||||
function overviewNodes(overview, { api, activity, notify, scope, navigate, openStatistics, openThumbnails }) {
|
||||
function overviewNodes(overview, { api, activity, notify, scope, navigate, openStatistics, openThumbnails, images }) {
|
||||
const go = section => () => navigate(`#/system/${section}`);
|
||||
const review = operationRow({ scope, title: ambiguousTitle(overview.needs_review),
|
||||
description: Number.isSafeInteger(overview.needs_review?.pending)
|
||||
@@ -374,26 +389,36 @@ function overviewNodes(overview, { api, activity, notify, scope, navigate, openS
|
||||
operationRow({ scope, title: 'One display configuration', description: 'Grid and density settings are shared across views.',
|
||||
label: 'Appearance', onClick: go('appearance') }),
|
||||
operationRow({ scope, title: sourcesTitle, description: sourcesDescription, label: 'Manage sources', onClick: go('sources') }),
|
||||
operationRow({ scope, title: 'Statistics', description: 'Runs by category, with library totals.', label: 'Explore',
|
||||
operationRow({ scope, title: 'Statistics', description: 'Runs by category, with database record counts.', label: 'Explore',
|
||||
onClick: openStatistics }),
|
||||
]);
|
||||
const columns = el('div', undefined, 'two-col');
|
||||
columns.append(health, preferences);
|
||||
return { nodes: [metricsNode(overview), columns], thumbnails, backup: backupRow };
|
||||
const metrics = metricsNode(overview, images);
|
||||
return { nodes: [metrics, columns], thumbnails, backup: backupRow, metrics };
|
||||
}
|
||||
|
||||
/**
|
||||
* The Overview section's mount (`mountSystem`'s `sections.overview`): returns its loading line, which the server's
|
||||
* answer replaces in place (the concept's metrics and columns sit directly in `.settings-layout`). A read that a newer
|
||||
* one superseded, or one answering after the section left, paints nothing.
|
||||
*
|
||||
* `counts` is the shell's sidebar counts (`{get(id), subscribe(listener), freshen(ids)}`, index.mjs): the Images metric
|
||||
* shows its `images` count and follows it, and a visit re-reads it as the workspace Overview does (#555).
|
||||
*/
|
||||
export function mountOverview(container, { api, activity, scope, navigate = navigateTo, notify = showWorkspaceToast,
|
||||
latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) {
|
||||
export function mountOverview(container, { api, activity, scope, counts = null, navigate = navigateTo,
|
||||
notify = showWorkspaceToast, latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) {
|
||||
const overviewRead = latestOnly({ lifetime: scope.signal });
|
||||
const statisticsRead = latestOnly({ lifetime: scope.signal });
|
||||
const thumbnailsRead = latestOnly({ lifetime: scope.signal });
|
||||
const backupRead = latestOnly({ lifetime: scope.signal });
|
||||
let shown = [];
|
||||
// #555: the Images metric's value node, repainted whenever the shell's Images count changes.
|
||||
let imagesValue = null;
|
||||
const imagesCount = () => counts?.get('images');
|
||||
const paintImages = () => {
|
||||
if (imagesValue) imagesValue.textContent = imagesText(imagesCount());
|
||||
};
|
||||
// The backup status (undefined while read, null when it failed), painted into its row once both are there (S9).
|
||||
let backupStatus;
|
||||
let backupRow = null;
|
||||
@@ -439,8 +464,10 @@ export function mountOverview(container, { api, activity, scope, navigate = navi
|
||||
try {
|
||||
const overview = await overviewRead(signal => api.get(OVERVIEW_URL, { signal, cache: 'no-store' }));
|
||||
if (overview === STALE || scope.signal.aborted) return;
|
||||
const { nodes, thumbnails, backup } = overviewNodes(overview, { api, activity, notify, scope, navigate, openStatistics,
|
||||
const { nodes, thumbnails, backup, metrics } = overviewNodes(overview, { api, activity, notify, scope, navigate,
|
||||
openStatistics, images: imagesCount(),
|
||||
openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate }) });
|
||||
imagesValue = metrics.children[2].querySelector('strong');
|
||||
thumbnailRow = thumbnails;
|
||||
backupRow = backup;
|
||||
sources = overview.sources;
|
||||
@@ -488,6 +515,10 @@ export function mountOverview(container, { api, activity, scope, navigate = navi
|
||||
const first = swap(loading());
|
||||
// D129: a finished Move library changes where the library lives.
|
||||
scope.cleanup(onMoveEnded(activity, load));
|
||||
if (counts) {
|
||||
scope.cleanup(counts.subscribe(paintImages));
|
||||
counts.freshen(['images']);
|
||||
}
|
||||
load();
|
||||
loadThumbnails();
|
||||
loadBackup();
|
||||
|
||||
@@ -135,7 +135,7 @@ export const DELIVERED_SECTIONS = Object.freeze({
|
||||
/**
|
||||
* Mount System under the heading `renderHeading` wrote in `main`. Returns `{show(section)}`, which shows another
|
||||
* section in place. `scope` is the page's view scope; `sections` maps a section id to a delivered section's mount
|
||||
* (`(container, {api, activity, scope, drafts}) => nodes`) for every section. `activity` is the page's Activity store, which follows the jobs a section starts.
|
||||
* (`(container, {api, activity, scope, drafts, counts}) => nodes`; `counts` is the shell's sidebar counts, #555) for every section. `activity` is the page's Activity store, which follows the jobs a section starts.
|
||||
*
|
||||
* The host values are read on mount (and by the store again after a conflict), so a save carries the current revision;
|
||||
* until a read returned one, a host save fails without a request (system-state.mjs). The browser values are read on
|
||||
@@ -159,7 +159,8 @@ function legacyPagesLine() {
|
||||
return line;
|
||||
}
|
||||
|
||||
export function mountSystem(main, api, scope, { section, activity, sections = DELIVERED_SECTIONS, navigate = navigateTo }) {
|
||||
export function mountSystem(main, api, scope, { section, activity, counts = null, sections = DELIVERED_SECTIONS,
|
||||
navigate = navigateTo }) {
|
||||
const store = systemDrafts(api);
|
||||
jobs = activity;
|
||||
store.reloadHost();
|
||||
@@ -208,7 +209,7 @@ export function mountSystem(main, api, scope, { section, activity, sections = DE
|
||||
for (const button of nav.children) button.setAttribute('aria-pressed', String(button.dataset.value === next));
|
||||
while (layout.firstChild !== saveRow) layout.firstChild.remove();
|
||||
// Every section is delivered (S12), and the router only names known sections (state.mjs).
|
||||
const nodes = sections[next](layout, { api, activity, scope: sectionScope, drafts: store });
|
||||
const nodes = sections[next](layout, { api, activity, scope: sectionScope, drafts: store, counts });
|
||||
layout.prepend(...nodes);
|
||||
if (next === 'overview') main.append(legacyPages);
|
||||
else legacyPages.remove();
|
||||
|
||||
Reference in New Issue
Block a user