Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5ec3178195 | ||
|
|
00d31614e8 | ||
|
|
c3ef047341 |
@@ -221,10 +221,7 @@ jobs:
|
||||
- uses: actions/setup-python@v7
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: requirements-dev.txt
|
||||
# The exact black and flake8 pins from requirements-dev.txt, so a new release cannot turn CI red.
|
||||
- run: pip install $(grep -E '^(black|flake8)==' requirements-dev.txt)
|
||||
- run: pip install black flake8
|
||||
- run: black --check .
|
||||
- run: flake8 .
|
||||
|
||||
@@ -238,8 +235,6 @@ jobs:
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: "24"
|
||||
cache: npm
|
||||
cache-dependency-path: tools/vendor/package-lock.json
|
||||
|
||||
- name: Check shipped extension and page modules parse
|
||||
run: |
|
||||
|
||||
@@ -13,11 +13,9 @@ repos:
|
||||
language: system
|
||||
types: [python]
|
||||
|
||||
# Fails at the severity CI enforces (code-quality.yml: -lll, high only), never stricter.
|
||||
# pre-commit appends the staged files, so no shell wrapper can drop the first one.
|
||||
- id: bandit
|
||||
name: bandit
|
||||
entry: bandit -lll -q
|
||||
entry: bash -c 'bandit "$@" -ll -q 2>&1; true'
|
||||
language: system
|
||||
types: [python]
|
||||
exclude: ^(tests/|reference/)
|
||||
|
||||
-27
@@ -4103,33 +4103,6 @@ list stay unscoped: any row's control proves them.
|
||||
| API timeout `apiTimeout` | `gallery-settings.js:37` | Legacy gallery (S12-10; read by no request) | kept in legacy | — | defaults are a complete, sane settings object (gallery-settings.test.js) |
|
||||
| config.json-only keys | `py/config.py:212–234,722–739`; `py/api/diagnostics_report.py:185` | S12 report (read-only) | covered | `diagnostics` · button · "Download diagnostic report" | test_report_lists_config_only_keys (test_api_diagnostics_report.py) |
|
||||
|
||||
### Legacy System UI removed by S13
|
||||
|
||||
System S13 (#330) points the legacy pages' System buttons at native System (a link that
|
||||
opens `workspace#/system/<id>` in a new tab) and removes the legacy copy of a capability
|
||||
only when its row above is `covered`. One row per removed capability: Capability is the
|
||||
exact Capability cell of its coverage row, Page is `admin` or `gallery`, Removed ids and
|
||||
Removed methods list what went (`—` for none), and Native test copies that row's Test
|
||||
cell. An id that stays on a System link is not listed, only its handler.
|
||||
|
||||
`tests/vendor/legacy-coverage.test.mjs` fails when a row cites a capability that is not
|
||||
`covered` or copies another test, when an id or method is listed twice for one page, or
|
||||
when a listed id is still on its page (`id="…"` in the markup or the script, a
|
||||
`getElementById`/`querySelector('#…')` lookup, an entry of a modal-id list) or a listed
|
||||
method is still defined or called (`.name(`). A deletion listed nowhere is caught by the
|
||||
kept-flow tests (`kept admin flows still work` in `legacy-system-links.spec.mjs` and the
|
||||
legacy page workflows), not by this check.
|
||||
|
||||
| Capability | Page | Removed ids | Removed methods | Native test |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Diagnostics run | admin | `#diagnosticsModal`, `#runDiagnosticsBtn`, `#diagnosticsContent` | `showDiagnosticsModal()`, `closeDiagnostics()`, `runDiagnostics()`, `renderDiagnostics()` | Run checks opens the shared Health report (workspace-system-diagnostics.spec.mjs) |
|
||||
| Synthetic image-link test | admin | `#testImageLinkBtn` | `testImageLink()` | Create test link confirms first and names the prompt (workspace-system-diagnostics.spec.mjs) |
|
||||
| View logs with level and limit | admin | `#logsModal`, `#logLevel`, `#logLimit`, `#refreshLogsBtn`, `#logsContainer` | `showLogsModal()`, `closeLogs()`, `refreshLogs()`, `displayLogs()` | level filter is minimum-level over all six choices (workspace-system-diagnostics.spec.mjs) |
|
||||
| Log statistics | admin | `#logStatsTotal`, `#logStatsErrors`, `#logStatsWarnings`, `#logStatsLevel`, `#logStatsSize` | `loadLogStats()`, `formatBytes()` | log statistics show entries, errors, warnings, files and size (workspace-system-diagnostics.spec.mjs) |
|
||||
| Log files list and download | admin | `#downloadLogsBtn`, `#logFilesList` | `loadLogFiles()`, `downloadLogs()`, `downloadLogFile()` | each listed file downloads by its returned name and a link has no Download (workspace-system-diagnostics.spec.mjs) |
|
||||
| Clear/truncate logs | admin | `#clearLogsBtn` | `clearLogs()` | Clear all logs states both counts, confirms first and reads everything again (workspace-system-diagnostics.spec.mjs) |
|
||||
| Set log level (incl. CRITICAL), console logging | admin | `#updateLogConfigBtn`, `#setLogLevel`, `#consoleLogging` | `loadLogConfig()`, `updateLogConfig()` | runtime level (including CRITICAL) and console logging save and say they apply until the server restarts (workspace-system-diagnostics.spec.mjs) |
|
||||
|
||||
### Browser cache ownership (#222)
|
||||
|
||||
The legacy gallery's Clear cache used to delete every localStorage key starting `pm_` or
|
||||
|
||||
@@ -483,59 +483,13 @@ class BackupRestoreRoutesMixin:
|
||||
self.logger.info(f"Removed {len(removed)} old safety backups")
|
||||
|
||||
async def _publish(self, model, staged):
|
||||
"""Publish ``staged``; on a page-copy failure prune and re-raise.
|
||||
|
||||
A cancel (server stop) waits for the publishing thread (#360): the
|
||||
caller's cleanup and its registry admission must outlive it. When the
|
||||
copy then succeeded the library was replaced, so it is adopted
|
||||
(``_after_publish``, itself run to its end) before the cancel goes
|
||||
on; a copy that failed is handled as usual, then the cancel goes on.
|
||||
"""
|
||||
future = self._in_executor_task(model.publish_staged, staged)
|
||||
cancelled = await _outlive_cancel(future)
|
||||
"""Publish ``staged``; on a page-copy failure prune and re-raise."""
|
||||
try:
|
||||
backup_path = future.result()
|
||||
return await self._run_in_executor(model.publish_staged, staged)
|
||||
except sqlite3.OperationalError as exc:
|
||||
if getattr(exc, "_prompt_manager_restore_backup_created", False) is True:
|
||||
await self._prune_after(model.db_path, failed=True)
|
||||
if cancelled:
|
||||
raise asyncio.CancelledError from exc
|
||||
raise
|
||||
except Exception as exc:
|
||||
if cancelled:
|
||||
raise asyncio.CancelledError from exc
|
||||
raise
|
||||
if cancelled:
|
||||
self.logger.info("The restore finished while the server was stopping")
|
||||
# A second cancel (the loop's own shutdown) must not cut it short.
|
||||
adopted = asyncio.ensure_future(self._after_publish(model.db_path))
|
||||
await _outlive_cancel(adopted)
|
||||
if not adopted.cancelled() and adopted.exception() is not None:
|
||||
self.logger.error(
|
||||
"Could not adopt the restored library",
|
||||
exc_info=adopted.exception(),
|
||||
)
|
||||
raise asyncio.CancelledError
|
||||
return backup_path
|
||||
|
||||
async def _stage_to_end(self, model, temp_path):
|
||||
"""Stage ``temp_path``; a cancel waits for the worker (#360).
|
||||
|
||||
The upload stays until the staging thread is done with it, and a
|
||||
stage it finished after the cancel is removed before the cancel goes
|
||||
on.
|
||||
"""
|
||||
future = self._in_executor_task(model.stage_restore, temp_path)
|
||||
if await _outlive_cancel(future):
|
||||
if not future.cancelled() and future.exception() is None:
|
||||
await self._discard_held([_Orphan(future.result())])
|
||||
raise asyncio.CancelledError
|
||||
return future.result()
|
||||
|
||||
def _in_executor_task(self, func, *args):
|
||||
"""``_run_in_executor(func, *args)`` as a task ``_outlive_cancel`` can
|
||||
wait for (shielded, the executor call is never cancelled)."""
|
||||
return asyncio.ensure_future(self._run_in_executor(func, *args))
|
||||
|
||||
async def _after_publish(self, db_path):
|
||||
"""Everything after the copy replaced the library: best effort, logged.
|
||||
@@ -655,7 +609,7 @@ class BackupRestoreRoutesMixin:
|
||||
{"success": False, "error": REQUESTS_BUSY}, status=409
|
||||
)
|
||||
model = self.db.model
|
||||
staged = await self._stage_to_end(model, temp_path)
|
||||
staged = await self._run_in_executor(model.stage_restore, temp_path)
|
||||
try:
|
||||
backup_path = await self._publish(model, staged)
|
||||
finally:
|
||||
@@ -973,30 +927,6 @@ class BackupRestoreRoutesMixin:
|
||||
job.update(progress=100, status_line=line)
|
||||
|
||||
|
||||
async def _outlive_cancel(future):
|
||||
"""Await ``future`` (an executor call's task) to its end, even through
|
||||
cancels (#360).
|
||||
|
||||
A cancelled task cannot stop the worker thread, so the files it uses and
|
||||
the registry admission it runs under must outlive it: the cancel is held
|
||||
until the worker returned. Returns whether the task was cancelled
|
||||
meanwhile; the caller reads the outcome from ``future`` and re-raises
|
||||
``CancelledError``. A future cancelled itself (an executor shut down) is
|
||||
not a cancel of the task: ``future.result()`` raises it. Never catches
|
||||
``GeneratorExit``: a closed loop still closes the coroutine.
|
||||
"""
|
||||
cancelled = False
|
||||
while not future.done():
|
||||
try:
|
||||
await asyncio.shield(future)
|
||||
except asyncio.CancelledError:
|
||||
if not future.cancelled():
|
||||
cancelled = True
|
||||
except Exception:
|
||||
pass # the caller reads it from the future
|
||||
return cancelled
|
||||
|
||||
|
||||
class _Orphan:
|
||||
"""A stage with no store entry, in the shape ``_discard_held`` takes."""
|
||||
|
||||
|
||||
+11
-23
@@ -88,9 +88,9 @@ from .thumbnail_encode import shutdown_encoders, writer_settings
|
||||
from .thumbnail_jobs import ThumbnailOptions
|
||||
|
||||
try:
|
||||
from ...utils.parallel import map_parallel, note_refusal, refused_at_shutdown
|
||||
from ...utils.parallel import map_parallel
|
||||
except ImportError: # tests import the package top-level
|
||||
from utils.parallel import map_parallel, note_refusal, refused_at_shutdown
|
||||
from utils.parallel import map_parallel
|
||||
|
||||
# The placeholder frame a video without a preview is answered with (JPEG
|
||||
# bytes, never written to disk); written previews follow the settings.
|
||||
@@ -410,8 +410,7 @@ class AutoThumbnailer:
|
||||
|
||||
def enqueue(self, src, dst, is_video):
|
||||
"""Queue *dst*'s preview for the worker; False when already queued,
|
||||
full, closed or its worker is refused at interpreter exit. Never
|
||||
blocks on a write."""
|
||||
full or closed. Never blocks on a write."""
|
||||
key = _dst_key(dst)
|
||||
with self._changed:
|
||||
if self._closed or key in self._queued:
|
||||
@@ -421,41 +420,30 @@ class AutoThumbnailer:
|
||||
self._queued.add(key)
|
||||
self._queue.append((key, Path(src), Path(dst), bool(is_video)))
|
||||
self._busy = True
|
||||
started = True
|
||||
if self._start_worker and not self._worker_running:
|
||||
started = self._start_worker_locked()
|
||||
if not started:
|
||||
# Refused at interpreter exit: no worker will take it (#375).
|
||||
self._queue.pop()
|
||||
self._queued.discard(key)
|
||||
self._start_worker_locked()
|
||||
self._changed.notify_all()
|
||||
return started
|
||||
return True
|
||||
|
||||
def _start_worker_locked(self):
|
||||
"""Start the worker (lock held); False only when the interpreter is
|
||||
exiting (#375: logged once at DEBUG, the caller drops the file).
|
||||
Another start failure leaves the file queued and the next enqueue
|
||||
tries again."""
|
||||
"""Start the worker (lock held). A start that fails leaves the file
|
||||
queued and the next enqueue tries again."""
|
||||
worker = threading.Thread(target=self._run, name=WORKER_NAME, daemon=True)
|
||||
worker.pm_preview_token = _MODULE_TOKEN
|
||||
worker.pm_preview_close = self.close
|
||||
self._worker_running = True
|
||||
try:
|
||||
worker.start()
|
||||
except RuntimeError as error:
|
||||
except RuntimeError:
|
||||
self._worker_running = False
|
||||
self._busy = False
|
||||
if refused_at_shutdown(error):
|
||||
note_refusal(self._logger, "The automatic preview worker")
|
||||
return False
|
||||
self._logger.warning(
|
||||
"The automatic preview worker could not start; the next output "
|
||||
"tries again",
|
||||
exc_info=True,
|
||||
)
|
||||
return True
|
||||
return
|
||||
self._worker = worker
|
||||
return True
|
||||
|
||||
def _run(self):
|
||||
try:
|
||||
@@ -802,8 +790,8 @@ class AutoThumbnailMixin:
|
||||
|
||||
Returns ``queued``, ``fresh`` (its preview is current) or ``skipped``
|
||||
(not a media file under a current root, a link, a preview itself,
|
||||
already queued, after the app's cleanup, or when its worker is refused
|
||||
at interpreter exit). Never writes on the caller's thread.
|
||||
already queued, or after the app's cleanup). Never writes on the
|
||||
caller's thread.
|
||||
"""
|
||||
if self._auto_thumbnails_shut:
|
||||
return "skipped"
|
||||
|
||||
@@ -8,8 +8,8 @@ coverage>=7.16.2
|
||||
numpy>=2.2.6 # WD14 engine tests
|
||||
|
||||
# Code quality
|
||||
black==26.10.1
|
||||
flake8==7.4.1
|
||||
black>=26.10.1
|
||||
flake8>=7.4.1
|
||||
|
||||
# Security scanning
|
||||
bandit[toml]>=1.9.4
|
||||
|
||||
@@ -286,8 +286,7 @@ const continuationText=page=>page.locator('.infinite-continuation > p').first().
|
||||
async function searchOf(page,term) {
|
||||
const before=await countText(page);
|
||||
await page.locator('#search').fill(term);
|
||||
// Production's caption reads "Loading images…" until the search's listing answers (#391); the concept's is instant.
|
||||
await expect.poll(async()=>{const text=await countText(page);return text!==before&&text!=='Loading images…';}).toBe(true);
|
||||
await expect.poll(()=>countText(page)).not.toBe(before);
|
||||
await expect.poll(()=>page.locator(CARDS).count()).toBeLessThan(FIXTURE_IMAGES);
|
||||
await settle(page);
|
||||
const result={count:await countText(page),continuation:await continuationText(page),cards:await page.locator(`${CARDS} .image-open`).evaluateAll(nodes=>nodes.map(node=>node.getAttribute('aria-label')))};
|
||||
|
||||
@@ -228,8 +228,7 @@ async function productionSendOf(page,context) {
|
||||
async function tagOf(page) {
|
||||
const before=await page.locator('#result-count').textContent();
|
||||
await panelOf(page).locator('.detail-inner > .tags').last().getByRole('button',{name:'cinematic',exact:true}).click();
|
||||
// Production's caption reads "Loading images…" until the tag's listing answers (#391); the concept's search is instant.
|
||||
await expect.poll(async()=>{const text=await page.locator('#result-count').textContent();return text!==before&&text!=='Loading images…';}).toBe(true);
|
||||
await expect.poll(()=>page.locator('#result-count').textContent()).not.toBe(before);
|
||||
await settle(page);
|
||||
return {count:await page.locator('#result-count').textContent(),
|
||||
cards:await page.locator(`${CARDS} .image-open`).evaluateAll(nodes=>nodes.map(node=>node.getAttribute('aria-label')))};
|
||||
@@ -329,25 +328,6 @@ test('a recipe-linked image without recipe metadata shows its positive prompt me
|
||||
await expect(panelOf(page).locator('.detail-footer button')).toHaveText(['Send to ComfyUI','Prompt history']);
|
||||
});
|
||||
|
||||
// #391: a tag chip replaces the collection; until its listing answers the caption says it is loading, never that the
|
||||
// filter matched nothing ("0 of 0 images" was painted before the response and read by a fast reader).
|
||||
test('a tag chip collection reads as loading, not empty, until its listing answers (#391)',async({page})=>{
|
||||
await page.setViewportSize({width:1600,height:1100});
|
||||
await openProductionPanel(page,'dark');
|
||||
let release,arrived;
|
||||
const held=new Promise(resolve=>{release=resolve;}),reached=new Promise(resolve=>{arrived=resolve;});
|
||||
await page.route(url=>url.pathname==='/prompt_manager/images/output'&&url.searchParams.get('tags')==='cinematic',
|
||||
async route=>{arrived();await held;await route.continue();},{times:1});
|
||||
const count=page.locator('#result-count');
|
||||
await panelOf(page).locator('.detail-inner > .tags').last().getByRole('button',{name:'cinematic',exact:true}).click();
|
||||
await reached;
|
||||
await expect(count).toHaveText('Loading images…');
|
||||
await expect(page.locator(CARDS)).toHaveCount(0);
|
||||
release();
|
||||
await expect(count).toHaveText(countCopy('8 of 8 images · 2 positive prompts'));
|
||||
await expect(page.locator(CARDS)).toHaveCount(8);
|
||||
});
|
||||
|
||||
// Task C3 (D88, #249 PR C): the concept's chip row is the image's own tags (`i.tags`, app.js a402). With those names
|
||||
// saved as the file's current image tags, production draws the row from them (Tags filter buttons in the panel, plain
|
||||
// in the viewer, the linked positive's equal tags not repeated) and still equals the concept, in both sizes.
|
||||
|
||||
@@ -1,163 +0,0 @@
|
||||
// System S13 (#330): the legacy admin and gallery pages open native System for their System tools. Each replaced
|
||||
// control is a plain link (`workspace#/system/<id>`, a new tab), and the legacy flows S13 keeps still work.
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
|
||||
const PAGES = Object.freeze({ admin: '/prompt_manager/admin', gallery: '/prompt_manager/gallery' });
|
||||
const LABELS = Object.freeze({
|
||||
overview: 'Overview',
|
||||
appearance: 'Appearance',
|
||||
sources: 'Sources & scans',
|
||||
integrations: 'Integrations',
|
||||
tagging: 'Auto-tagging',
|
||||
maintenance: 'Maintenance',
|
||||
backups: 'Backup & export',
|
||||
diagnostics: 'Diagnostics & logs',
|
||||
});
|
||||
|
||||
// One row per System link of the design's §2 map: the page, the control's label, how to find it, the section it opens
|
||||
// and what a test must arrange before the control shows.
|
||||
const LINKS = Object.freeze([
|
||||
{ page: 'admin', label: 'Diagnostics', locate: page => page.locator('#diagnosticsBtn'), section: 'diagnostics' },
|
||||
{ page: 'admin', label: 'View Logs', locate: page => page.locator('#logsBtn'), section: 'diagnostics' },
|
||||
]);
|
||||
|
||||
async function post(request, url, data) {
|
||||
await expect(await request.post(url, data === undefined ? {} : { data })).toBeOK();
|
||||
}
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await post(request, '/__test__/reset');
|
||||
await post(request, '/__test__/system/reset');
|
||||
await post(request, '/__test__/tagging/reset');
|
||||
// The 3.2.0 update notice would cover the admin page's toolbar.
|
||||
await page.addInitScript(() => { localStorage.setItem('pm_update_notice_dismissed', '3.2.0'); });
|
||||
page.legacyErrors = [];
|
||||
page.on('pageerror', error => page.legacyErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await post(request, '/__test__/tagging/reset');
|
||||
await post(request, '/__test__/system/reset');
|
||||
expect(page.legacyErrors).toEqual([]);
|
||||
});
|
||||
|
||||
/** Opens the legacy page with the link showing; returns the link. */
|
||||
async function openLink(page, request, link) {
|
||||
if (link.arrange) await link.arrange(page, request);
|
||||
await page.goto(PAGES[link.page]);
|
||||
if (link.reveal) await link.reveal(page);
|
||||
const control = link.locate(page);
|
||||
await expect(control).toBeVisible();
|
||||
return control;
|
||||
}
|
||||
|
||||
for (const link of LINKS) {
|
||||
test(`legacy ${link.page} ${link.label} opens System ${link.section}`, async ({ page, context, request }) => {
|
||||
const control = await openLink(page, request, link);
|
||||
const legacyUrl = page.url();
|
||||
await expect(control).toHaveAttribute('href', `workspace#/system/${link.section}`);
|
||||
await expect(control).toHaveAttribute('target', '_blank');
|
||||
await expect(control).toHaveAttribute('rel', 'noopener');
|
||||
const opened = context.waitForEvent('page');
|
||||
await control.click();
|
||||
const system = await opened;
|
||||
await expect(system).toHaveURL(new RegExp(`/prompt_manager/workspace#/system/${link.section}$`));
|
||||
// System's page heading is "System"; the open section is the pressed button of its section switcher.
|
||||
const main = system.locator('#workspace-main');
|
||||
await expect(main.getByRole('heading', { level: 1, name: 'System', exact: true })).toBeVisible();
|
||||
await expect(main.getByRole('group', { name: 'System sections', exact: true })
|
||||
.getByRole('button', { name: LABELS[link.section], exact: true })).toHaveAttribute('aria-pressed', 'true');
|
||||
expect(page.url()).toBe(legacyUrl);
|
||||
await system.close();
|
||||
});
|
||||
}
|
||||
|
||||
test('a System link sends no request from the legacy page', async ({ page, context, request }) => {
|
||||
for (const link of LINKS) {
|
||||
const control = await openLink(page, request, link);
|
||||
// The page's own start-up reads (list, film strips, settings) finish first; then the click must add none.
|
||||
await page.waitForLoadState('networkidle');
|
||||
const sent = [];
|
||||
const record = item => sent.push(`${item.method()} ${item.url()}`);
|
||||
page.on('request', record);
|
||||
const opened = context.waitForEvent('page');
|
||||
await control.click();
|
||||
const system = await opened;
|
||||
await system.waitForLoadState('domcontentloaded');
|
||||
page.off('request', record);
|
||||
expect(sent, `${link.page} ${link.label}`).toEqual([]);
|
||||
await system.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('kept admin flows still work', async ({ page, context }) => {
|
||||
await page.goto(PAGES.admin);
|
||||
const texts = page.locator('#resultsList .prompt-text');
|
||||
await expect(texts.first()).toBeVisible();
|
||||
|
||||
// K1: sorting reorders the list.
|
||||
for (const [value, order] of [['text_asc', 1], ['text_desc', -1]]) {
|
||||
const listed = page.waitForResponse(item => /\/prompt_manager\/(recent|search)\?/.test(item.url()) && item.url().includes(`sort=${value}`));
|
||||
await page.locator('#sortBy').selectOption(value);
|
||||
await listed;
|
||||
await expect.poll(async () => {
|
||||
const shown = await texts.allTextContents();
|
||||
// The server sorts case-insensitively (SQLite NOCASE), not by locale.
|
||||
const key = text => text.toLowerCase();
|
||||
const sorted = [...shown].sort((a, b) => order * (key(a) < key(b) ? -1 : key(a) > key(b) ? 1 : 0));
|
||||
return shown.length > 1 && shown.join('\n') === sorted.join('\n');
|
||||
}).toBe(true);
|
||||
}
|
||||
// K3: Add Prompt saves a prompt the list then shows.
|
||||
await page.locator('#addPromptBtn').click();
|
||||
await page.locator('#addPromptText').fill('Kept flow lighthouse');
|
||||
const saved = page.waitForResponse(item => item.request().method() === 'POST' && item.url().endsWith('/prompt_manager/save'));
|
||||
await page.locator('#saveNewPromptBtn').click();
|
||||
expect((await saved).ok()).toBe(true);
|
||||
await expect(page.locator('#addPromptModal')).toBeHidden();
|
||||
await expect(texts.filter({ hasText: 'Kept flow lighthouse' })).toBeVisible();
|
||||
|
||||
// K1: a search narrows the list, and a selection enables the bulk buttons.
|
||||
const all = await texts.allTextContents();
|
||||
const word = all[0].trim().split(/\s+/)[0];
|
||||
const searched = page.waitForResponse(item => item.url().includes('/prompt_manager/search?') && item.url().includes('text='));
|
||||
await page.locator('#searchText').fill(word);
|
||||
await page.locator('#searchBtn').click();
|
||||
await searched;
|
||||
await expect.poll(async () => {
|
||||
const shown = await texts.allTextContents();
|
||||
return shown.length > 0 && shown.length < all.length && shown.every(text => text.toLowerCase().includes(word.toLowerCase()));
|
||||
}).toBe(true);
|
||||
await expect(page.locator('#bulkDeleteBtn')).toBeDisabled();
|
||||
await page.locator('#resultsList .prompt-checkbox').first().check();
|
||||
for (const id of ['#bulkDeleteBtn', '#bulkTagBtn', '#bulkCategoryBtn']) await expect(page.locator(id)).toBeEnabled();
|
||||
|
||||
// K4: Stats reads the statistics again; Export downloads the prompts.
|
||||
const stats = page.waitForResponse(item => item.url().endsWith('/prompt_manager/stats') && item.ok());
|
||||
await page.locator('#statsBtn').click();
|
||||
await stats;
|
||||
await expect(page.getByText('Stats refreshed successfully!')).toBeVisible();
|
||||
const download = page.waitForEvent('download');
|
||||
await page.locator('#exportBtn').click();
|
||||
expect((await download).suggestedFilename()).toMatch(/^prompts-\d{4}-\d{2}-\d{2}\.json$/);
|
||||
|
||||
// K5: Scan Images opens its dialog (with the quick backup) without starting a scan.
|
||||
await page.locator('#scanBtn').click();
|
||||
await expect(page.locator('#scanModal')).toBeVisible();
|
||||
await expect(page.locator('#quickBackupBtn')).toBeVisible();
|
||||
await page.locator('#cancelScan').click();
|
||||
await expect(page.locator('#scanModal')).toBeHidden();
|
||||
|
||||
// K7: Metadata and Gallery open their pages in a new tab.
|
||||
for (const [id, path] of [['#metadataBtn', '/prompt_manager/gallery.html'], ['#galleryBtn', '/prompt_manager/gallery']]) {
|
||||
const opened = context.waitForEvent('page');
|
||||
await page.locator(id).click();
|
||||
const tab = await opened;
|
||||
await tab.waitForLoadState('domcontentloaded');
|
||||
expect(new URL(tab.url()).pathname).toBe(path);
|
||||
await tab.close();
|
||||
}
|
||||
});
|
||||
@@ -47,17 +47,6 @@ async function serverJson(request, url) {
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/**
|
||||
* The restore job's own end (#361): its progress stream closes when the job takes its final state, however long the
|
||||
* disk-bound publish takes (bounded by the test, not by a 5 s expect), and the job finished. The toast the page shows
|
||||
* for it is then the page's reaction alone, checked with the default timeout.
|
||||
*/
|
||||
async function restoreFinished(request, id) {
|
||||
await expect(await request.get(`/prompt_manager/jobs/${id}/events`)).toBeOK();
|
||||
const { job } = await serverJson(request, `/prompt_manager/jobs/${id}`);
|
||||
expect([job.kind, job.state]).toEqual(['restore', 'finished']);
|
||||
}
|
||||
|
||||
async function openBackups(page) {
|
||||
// A fresh page each time: the drafts live for the page session (Q10), and a hash-only goto would not reload.
|
||||
if (page.url().endsWith('#/system/backups')) await page.reload();
|
||||
@@ -227,7 +216,6 @@ test('Restore starts the job, opens Activity; views reload after it finishes', a
|
||||
const { job } = await (await applied).json();
|
||||
expect([job.kind, job.title, job.cancellable]).toEqual(['restore', 'Database restore', false]);
|
||||
await expect(page).toHaveURL(`${workspace}#/activity/${job.id}`);
|
||||
await restoreFinished(request, job.id);
|
||||
await expect(toast(page)).toHaveText(/^Database restore finished · Restored \d+ prompts?, /);
|
||||
// The library is the backup's again, and the page says so without a reload: the sidebar counts were read again.
|
||||
expect(await libraryTotal(request)).toBe(restoredTotal);
|
||||
@@ -256,8 +244,7 @@ test('a finished restore shows the open page again from the restored data', asyn
|
||||
const card = page.locator('#workspace-main article.prompt-card').filter({ hasText: 'S9 prompt the restore removes' });
|
||||
await expect(card).toHaveCount(1);
|
||||
await expect(await request.post('/__test__/release')).toBeOK();
|
||||
const { job } = await (await applied).json();
|
||||
await restoreFinished(request, job.id);
|
||||
await applied;
|
||||
await expect(toast(page)).toHaveText(/^Database restore finished · /);
|
||||
// The shelf was read again from the restored library, on the same route.
|
||||
await expect(card).toHaveCount(0);
|
||||
@@ -435,7 +422,6 @@ test('a refused restore reopens its review: Restore retries it, Discard releases
|
||||
await dialog(page).getByRole('button', { name: 'Restore', exact: true }).click();
|
||||
const { job } = await (await applied).json();
|
||||
await expect(page).toHaveURL(`${workspace}#/activity/${job.id}`);
|
||||
await restoreFinished(request, job.id);
|
||||
await expect(toast(page)).toHaveText(/^Database restore finished · /);
|
||||
expect(await stagesOf(request)).toBe(0);
|
||||
});
|
||||
|
||||
@@ -35,8 +35,7 @@ read. These per-server hooks only change what a System test needs on top:
|
||||
previews for Convert outdated;
|
||||
- ``POST /__test__/system/jxl-encoder`` ``{available}`` (#317) makes the
|
||||
server read as having (``cjxl``) or lacking a JPEG XL encoder without
|
||||
probing, whatever the host has installed (#371: ``false`` stands in for the
|
||||
detector itself); reset restores the thumbnail settings and the real detection.
|
||||
probing; reset restores the thumbnail settings and the real detection.
|
||||
|
||||
- ``POST /__test__/system/duplicate-prompts`` (S7) adds two prompts whose
|
||||
text differs only in case and spacing (a duplicate prompt text group);
|
||||
@@ -119,11 +118,6 @@ def register_system_commands(app, routes, api, root: Path):
|
||||
}
|
||||
from py.api import thumbnail_encode
|
||||
|
||||
real_detect = thumbnail_encode._detect
|
||||
|
||||
def _no_encoder():
|
||||
return None
|
||||
|
||||
monitor = getattr(image_monitor, "_monitor_instance", None)
|
||||
failures: dict[str, int] = {}
|
||||
|
||||
@@ -153,7 +147,6 @@ def register_system_commands(app, routes, api, root: Path):
|
||||
api._health_report = None
|
||||
for (owner, name), value in host_values.items():
|
||||
setattr(owner, name, value)
|
||||
thumbnail_encode._detect = real_detect
|
||||
thumbnail_encode.reset_encoders()
|
||||
image_monitor._monitor_instance = monitor
|
||||
api.__dict__.pop("_library_resolver", None)
|
||||
@@ -305,12 +298,7 @@ def register_system_commands(app, routes, api, root: Path):
|
||||
available = data.get("available")
|
||||
if not isinstance(available, bool):
|
||||
raise web.HTTPBadRequest()
|
||||
# Test-only: the fixture, not the host, decides (#371). Without an encoder the
|
||||
# detector itself stands in, so a re-probe (a "none found" is probed again on
|
||||
# every settings read) cannot find the host's real encoder; with one the cached
|
||||
# answer stands, and a failed write still probes the real detector as before.
|
||||
# Reset restores the real detector.
|
||||
thumbnail_encode._detect = real_detect if available else _no_encoder
|
||||
# Test-only: stand in the cached detection (no probe).
|
||||
with thumbnail_encode._detect_lock:
|
||||
thumbnail_encode._detected = "cjxl" if available else None
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
"""Hang guards for waits on disk-bound work (tests only).
|
||||
|
||||
A test that waits for a job or handoff whose work is file and SQLite I/O
|
||||
(a safety backup with fsync, a restore, hashing and unlinking files) waits
|
||||
on that state; the ceiling below only ends a real hang. Hosted Windows
|
||||
runners stall disk I/O for tens of seconds at a time: a restore took over
|
||||
10 s (merge queue, #382), and on PR #405 every test setup in a 50 s window
|
||||
took up to 9 s against 0.1-0.6 s around it while the test bodies stayed
|
||||
fast, so a one-file duplicate cleanup missed a 10 s bound (#409).
|
||||
|
||||
The ceiling is per wait and stays under the suite's 120 s per-test hang
|
||||
watchdog. A test with several such waits in a row can reach the watchdog
|
||||
first; either way a real hang fails, the watchdog with every thread's stack.
|
||||
"""
|
||||
|
||||
DISK_BOUND_SECONDS = 90
|
||||
Vendored
-16
@@ -113,19 +113,3 @@ by their own suites. The expectations start as the observed baseline. A change
|
||||
that alters an outcome edits the vector in the same commit.
|
||||
`tests/js/legacy-metadata-characterization.test.js` pins what
|
||||
the legacy viewers show on the same files.
|
||||
|
||||
# Legacy contract snapshot
|
||||
|
||||
`legacy_contract_snapshot.json` records what the legacy admin and gallery pages
|
||||
talk to while System S13 (#330) moves their tools to native System: every
|
||||
registered route as `"<METHOD> <path>"` (`routes`) and the key paths of
|
||||
`GET /prompt_manager/settings` to depth 2 (`settings_keys`; list items give the
|
||||
union of their keys as `name[].key`). `tests/test_legacy_contract_snapshot.py`
|
||||
compares a temp-database API against it. Record it again only for an intended
|
||||
change, and say why in the PR body:
|
||||
|
||||
```sh
|
||||
PM_RECORD_LEGACY_SNAPSHOT=1 python3 -m unittest tests.test_legacy_contract_snapshot
|
||||
```
|
||||
|
||||
Cutover (#334) removes the test and this fixture.
|
||||
|
||||
-200
@@ -1,200 +0,0 @@
|
||||
{
|
||||
"routes": [
|
||||
"DELETE /prompt_manager/delete/{prompt_id}",
|
||||
"DELETE /prompt_manager/image-annotations/capture/{token}",
|
||||
"DELETE /prompt_manager/images/favorite",
|
||||
"DELETE /prompt_manager/images/{image_id}",
|
||||
"DELETE /prompt_manager/pairs/{pair_id}",
|
||||
"DELETE /prompt_manager/prompts/{prompt_id}/tags",
|
||||
"DELETE /prompt_manager/restore/preview/{token}",
|
||||
"DELETE /prompt_manager/saved-views/{view_id}",
|
||||
"DELETE /prompt_manager/tags/{tag_name}",
|
||||
"GET /prompt_manager/admin",
|
||||
"GET /prompt_manager/autotag/models",
|
||||
"GET /prompt_manager/backup",
|
||||
"GET /prompt_manager/backup/status",
|
||||
"GET /prompt_manager/categories",
|
||||
"GET /prompt_manager/diagnostics",
|
||||
"GET /prompt_manager/export",
|
||||
"GET /prompt_manager/gallery",
|
||||
"GET /prompt_manager/gallery.html",
|
||||
"GET /prompt_manager/gallery/subfolders",
|
||||
"GET /prompt_manager/history/runs/{run_id}",
|
||||
"GET /prompt_manager/history/runs/{run_id}/output/current/inspection",
|
||||
"GET /prompt_manager/history/runs/{run_id}/output/current/media",
|
||||
"GET /prompt_manager/history/runs/{run_id}/output/current/thumbnail",
|
||||
"GET /prompt_manager/history/runs/{run_id}/outputs",
|
||||
"GET /prompt_manager/image-annotations/capture/{token}",
|
||||
"GET /prompt_manager/images/all",
|
||||
"GET /prompt_manager/images/inspect/{image_path}",
|
||||
"GET /prompt_manager/images/output",
|
||||
"GET /prompt_manager/images/prompt/{image_path}",
|
||||
"GET /prompt_manager/images/recent",
|
||||
"GET /prompt_manager/images/search",
|
||||
"GET /prompt_manager/images/serve/{filepath}",
|
||||
"GET /prompt_manager/images/thumbnails/plan",
|
||||
"GET /prompt_manager/images/thumbnails/status",
|
||||
"GET /prompt_manager/images/{image_id}/file",
|
||||
"GET /prompt_manager/jobs",
|
||||
"GET /prompt_manager/jobs/{job_id}",
|
||||
"GET /prompt_manager/jobs/{job_id}/events",
|
||||
"GET /prompt_manager/jobs/{job_id}/report",
|
||||
"GET /prompt_manager/js/{filepath}",
|
||||
"GET /prompt_manager/lib/{filepath}",
|
||||
"GET /prompt_manager/library",
|
||||
"GET /prompt_manager/library/export",
|
||||
"GET /prompt_manager/library/options",
|
||||
"GET /prompt_manager/library/{positive_id}",
|
||||
"GET /prompt_manager/library/{positive_id}/history",
|
||||
"GET /prompt_manager/library/{positive_id}/history/pairs/{pair_id}/newest",
|
||||
"GET /prompt_manager/library/{positive_id}/images",
|
||||
"GET /prompt_manager/logs",
|
||||
"GET /prompt_manager/logs/config",
|
||||
"GET /prompt_manager/logs/download/{filename}",
|
||||
"GET /prompt_manager/logs/files",
|
||||
"GET /prompt_manager/logs/stats",
|
||||
"GET /prompt_manager/lora/detect",
|
||||
"GET /prompt_manager/lora/status",
|
||||
"GET /prompt_manager/lora/trigger-words",
|
||||
"GET /prompt_manager/maintenance/duplicates/{job_id}",
|
||||
"GET /prompt_manager/pairs",
|
||||
"GET /prompt_manager/pairs/export",
|
||||
"GET /prompt_manager/pairs/{pair_id}",
|
||||
"GET /prompt_manager/prompts/{prompt_id}/images",
|
||||
"GET /prompt_manager/recent",
|
||||
"GET /prompt_manager/review/events",
|
||||
"GET /prompt_manager/review/items",
|
||||
"GET /prompt_manager/review/items/{item_id}",
|
||||
"GET /prompt_manager/saved-views",
|
||||
"GET /prompt_manager/scan/status",
|
||||
"GET /prompt_manager/scan_duplicates",
|
||||
"GET /prompt_manager/scan_duplicates/status",
|
||||
"GET /prompt_manager/scan_output_dir",
|
||||
"GET /prompt_manager/search",
|
||||
"GET /prompt_manager/settings",
|
||||
"GET /prompt_manager/stats",
|
||||
"GET /prompt_manager/subfolders",
|
||||
"GET /prompt_manager/system/diagnostics",
|
||||
"GET /prompt_manager/system/health",
|
||||
"GET /prompt_manager/system/library",
|
||||
"GET /prompt_manager/system/overview",
|
||||
"GET /prompt_manager/system/statistics",
|
||||
"GET /prompt_manager/tags",
|
||||
"GET /prompt_manager/tags/filter",
|
||||
"GET /prompt_manager/tags/stats",
|
||||
"GET /prompt_manager/tags/suggest",
|
||||
"GET /prompt_manager/tags/{tag_name}/prompts",
|
||||
"GET /prompt_manager/test",
|
||||
"GET /prompt_manager/vocabulary/categories",
|
||||
"GET /prompt_manager/vocabulary/tags",
|
||||
"GET /prompt_manager/vocabulary/tags/{tag_id}/impact",
|
||||
"GET /prompt_manager/web",
|
||||
"GET /prompt_manager/workspace",
|
||||
"PATCH /prompt_manager/library/{positive_id}",
|
||||
"PATCH /prompt_manager/pairs/{pair_id}",
|
||||
"PATCH /prompt_manager/saved-views/{view_id}",
|
||||
"PATCH /prompt_manager/vocabulary/tags/{tag_id}",
|
||||
"POST /prompt_manager/autotag/apply",
|
||||
"POST /prompt_manager/autotag/batch/preview",
|
||||
"POST /prompt_manager/autotag/download/{model_type}",
|
||||
"POST /prompt_manager/autotag/single",
|
||||
"POST /prompt_manager/autotag/start",
|
||||
"POST /prompt_manager/autotag/unload",
|
||||
"POST /prompt_manager/bulk/category",
|
||||
"POST /prompt_manager/bulk/delete",
|
||||
"POST /prompt_manager/bulk/tags",
|
||||
"POST /prompt_manager/cleanup",
|
||||
"POST /prompt_manager/delete_duplicate_images",
|
||||
"POST /prompt_manager/diagnostics/test-link",
|
||||
"POST /prompt_manager/gallery/rescan",
|
||||
"POST /prompt_manager/image-annotations/observe",
|
||||
"POST /prompt_manager/image-annotations/suggest",
|
||||
"POST /prompt_manager/images/clear-thumbnails",
|
||||
"POST /prompt_manager/images/generate-thumbnails",
|
||||
"POST /prompt_manager/images/generate-thumbnails/progress",
|
||||
"POST /prompt_manager/images/link",
|
||||
"POST /prompt_manager/images/reveal",
|
||||
"POST /prompt_manager/jobs/autotag",
|
||||
"POST /prompt_manager/jobs/duplicates",
|
||||
"POST /prompt_manager/jobs/image-tags",
|
||||
"POST /prompt_manager/jobs/model-download",
|
||||
"POST /prompt_manager/jobs/scan",
|
||||
"POST /prompt_manager/jobs/thumbnails",
|
||||
"POST /prompt_manager/jobs/{job_id}/cancel",
|
||||
"POST /prompt_manager/library",
|
||||
"POST /prompt_manager/library/{positive_id}/variants",
|
||||
"POST /prompt_manager/logs/config",
|
||||
"POST /prompt_manager/logs/truncate",
|
||||
"POST /prompt_manager/lora/enable",
|
||||
"POST /prompt_manager/lora/import/apply",
|
||||
"POST /prompt_manager/lora/import/preview",
|
||||
"POST /prompt_manager/lora/refresh-cache",
|
||||
"POST /prompt_manager/lora/scan",
|
||||
"POST /prompt_manager/maintenance",
|
||||
"POST /prompt_manager/maintenance/apply",
|
||||
"POST /prompt_manager/maintenance/preview",
|
||||
"POST /prompt_manager/pairs",
|
||||
"POST /prompt_manager/prompts/tags",
|
||||
"POST /prompt_manager/prompts/{prompt_id}/tags",
|
||||
"POST /prompt_manager/restore",
|
||||
"POST /prompt_manager/restore/apply",
|
||||
"POST /prompt_manager/restore/preview",
|
||||
"POST /prompt_manager/review/items/{item_id}/decision",
|
||||
"POST /prompt_manager/save",
|
||||
"POST /prompt_manager/saved-views",
|
||||
"POST /prompt_manager/scan",
|
||||
"POST /prompt_manager/scan/cancel",
|
||||
"POST /prompt_manager/scan_duplicates/stream",
|
||||
"POST /prompt_manager/settings",
|
||||
"POST /prompt_manager/settings/validate-source",
|
||||
"POST /prompt_manager/system/health",
|
||||
"POST /prompt_manager/system/library/keep",
|
||||
"POST /prompt_manager/system/library/move",
|
||||
"POST /prompt_manager/tags/merge",
|
||||
"POST /prompt_manager/vocabulary/categories/apply",
|
||||
"POST /prompt_manager/vocabulary/categories/preview",
|
||||
"POST /prompt_manager/vocabulary/tags",
|
||||
"POST /prompt_manager/vocabulary/tags/{tag_id}/delete",
|
||||
"PUT /prompt_manager/image-annotations/tags",
|
||||
"PUT /prompt_manager/images/favorite",
|
||||
"PUT /prompt_manager/prompts/{prompt_id}",
|
||||
"PUT /prompt_manager/prompts/{prompt_id}/rating",
|
||||
"PUT /prompt_manager/tags/{tag_name}"
|
||||
],
|
||||
"settings_keys": [
|
||||
"monitor",
|
||||
"monitor.observer_alive",
|
||||
"monitor.present",
|
||||
"monitor.scheduled",
|
||||
"revision",
|
||||
"settings",
|
||||
"settings.autotag_character_threshold",
|
||||
"settings.autotag_existing",
|
||||
"settings.autotag_general_threshold",
|
||||
"settings.autotag_model_type",
|
||||
"settings.autotag_suggestion_limit",
|
||||
"settings.autotag_template",
|
||||
"settings.cpu_count",
|
||||
"settings.gallery_root_path",
|
||||
"settings.gallery_root_paths",
|
||||
"settings.infinite_scroll",
|
||||
"settings.monitored_directories",
|
||||
"settings.monitoring_enabled",
|
||||
"settings.result_timeout",
|
||||
"settings.safety_backup",
|
||||
"settings.thumbnail_edge",
|
||||
"settings.thumbnail_format",
|
||||
"settings.thumbnail_quality",
|
||||
"settings.webui_display_mode",
|
||||
"settings.worker_threads",
|
||||
"sources",
|
||||
"sources.auto_detected",
|
||||
"sources.configured",
|
||||
"sources.effective",
|
||||
"success",
|
||||
"thumbnail_encoders",
|
||||
"thumbnail_encoders.jpeg",
|
||||
"thumbnail_encoders.jxl",
|
||||
"thumbnail_encoders.webp"
|
||||
]
|
||||
}
|
||||
@@ -234,23 +234,6 @@ test('the count line reads N of M images and K positive prompts only when known'
|
||||
assert.equal(countLine({loaded:1200,total:3400,positiveCount:0}),'1,200 of 3,400 images · 0 positive prompts');
|
||||
});
|
||||
|
||||
test('the count line reads as loading, never as an empty result, until a first page answers (#391)', async () => {
|
||||
const {countLine,createImageStream}=await source;
|
||||
assert.equal(countLine({loaded:0,total:0,positiveCount:null,loading:true,error:null}),'Loading images…');
|
||||
// A failed first page and a settled empty one say what the server said; a later page keeps the loaded count.
|
||||
assert.equal(countLine({loaded:0,total:0,positiveCount:null,loading:false,error:new Error('x')}),'0 of 0 images');
|
||||
assert.equal(countLine({loaded:0,total:0,positiveCount:0,loading:false,error:null}),'0 of 0 images · 0 positive prompts');
|
||||
assert.equal(countLine({loaded:40,total:102,positiveCount:2,loading:true,error:null}),'40 of 102 images · 2 positive prompts');
|
||||
// While a reset's first page is in flight the line reads as loading (the cleared state before it is replaced in
|
||||
// the same task, so it is never painted).
|
||||
let answer;const lines=[];
|
||||
const stream=createImageStream({api:{get:()=>new Promise(resolve=>{answer=resolve;})},onChange:state=>lines.push(countLine(state))});
|
||||
const loading=stream.reset({tags:['cinematic']});
|
||||
assert.equal(lines.at(-1),'Loading images…');
|
||||
answer({images:[{path:'a.png',root_index:0}],total:1,has_more:false,next_cursor:null,positive_count:1});await loading;
|
||||
assert.equal(lines.at(-1),'1 of 1 image · 1 positive prompt');
|
||||
});
|
||||
|
||||
test('the caption date names the newest result day under Newest first only', async () => {
|
||||
const {captionDate}=await source;
|
||||
const now=new Date(2026,9,17,15,0);
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
"""A temp SQLite database in a folder no other test process shares (#346).
|
||||
|
||||
``PromptManagerAPI`` sweeps its database's folder at startup for the stages,
|
||||
uploads and safety-backup copies an interrupted restore or backup left. A
|
||||
fixture database directly in the system temp folder made every other test
|
||||
process that started an API (another worktree's suite, a pre-push hook) delete
|
||||
this process's in-flight restore files, and left safety backups behind there.
|
||||
"""
|
||||
|
||||
import tempfile
|
||||
|
||||
|
||||
def private_temp_db(case):
|
||||
"""Return a closed ``NamedTemporaryFile`` handle for an empty ``.db`` file.
|
||||
|
||||
The file lives in a new folder that ``case`` removes at cleanup, after its
|
||||
own teardown has closed and unlinked the database (cleanups run after
|
||||
``tearDown``), together with anything a restore or backup left next to it.
|
||||
"""
|
||||
folder = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
|
||||
case.addCleanup(folder.cleanup)
|
||||
handle = tempfile.NamedTemporaryFile(delete=False, suffix=".db", dir=folder.name)
|
||||
handle.close()
|
||||
return handle
|
||||
+3
-2
@@ -8,6 +8,7 @@ without requiring a full ComfyUI server.
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
@@ -17,7 +18,6 @@ from aiohttp.test_utils import AioHTTPTestCase
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from py.api import PromptManagerAPI
|
||||
from tests.private_temp import private_temp_db
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
|
||||
@@ -25,7 +25,8 @@ class APITestCase(AioHTTPTestCase):
|
||||
"""Base class that stands up a test aiohttp app with PromptManager routes."""
|
||||
|
||||
async def get_application(self):
|
||||
self._temp_db = private_temp_db(self)
|
||||
self._temp_db = tempfile.NamedTemporaryFile(delete=False, suffix=".db")
|
||||
self._temp_db.close()
|
||||
|
||||
app = web.Application()
|
||||
routes = web.RouteTableDef()
|
||||
|
||||
@@ -35,7 +35,6 @@ from aiohttp.test_utils import AioHTTPTestCase
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from py.api import PromptManagerAPI
|
||||
from tests.private_temp import private_temp_db
|
||||
from py.api.autotag_routes import MODEL_BUSY_ERROR
|
||||
from py.api.jobs import JobBusy
|
||||
from utils.hashing import generate_prompt_hash
|
||||
@@ -144,7 +143,8 @@ class AutotagAPITestCase(AioHTTPTestCase):
|
||||
fake engine."""
|
||||
|
||||
async def get_application(self):
|
||||
self._temp_db = private_temp_db(self)
|
||||
self._temp_db = tempfile.NamedTemporaryFile(delete=False, suffix=".db")
|
||||
self._temp_db.close()
|
||||
self.tmp_root = Path(tempfile.mkdtemp())
|
||||
self.output_dir = self.tmp_root / "output"
|
||||
self.output_dir.mkdir()
|
||||
|
||||
@@ -18,19 +18,15 @@ from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
from tests.disk_waits import DISK_BOUND_SECONDS
|
||||
from tests.open_handles import open_handles_under
|
||||
from tests.test_api_maintenance import ApplyTestCase, _digest
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from disk_waits import DISK_BOUND_SECONDS
|
||||
from open_handles import open_handles_under
|
||||
from test_api_maintenance import ApplyTestCase, _digest
|
||||
|
||||
from py.api import duplicate_files # noqa: E402
|
||||
from py.config import GalleryConfig, PromptManagerConfig # noqa: E402
|
||||
|
||||
# Gates and server-state polls. Waits on a find or cleanup job's task use
|
||||
# DISK_BOUND_SECONDS: those jobs hash, back up and unlink files (#409).
|
||||
WAIT = 10
|
||||
FIND = "/prompt_manager/jobs/duplicates"
|
||||
GROUPS = "/prompt_manager/maintenance/duplicates/"
|
||||
@@ -87,7 +83,7 @@ class DuplicateFilesTestCase(ApplyTestCase):
|
||||
self.assertEqual(response.status, 202, text)
|
||||
self.assertNotIn(self.tmpdir, text)
|
||||
job = self.api._jobs.get(json.loads(text)["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(job.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
return job
|
||||
|
||||
async def groups(self, job_id, status=200, **query):
|
||||
@@ -131,7 +127,7 @@ class DuplicateFilesTestCase(ApplyTestCase):
|
||||
plan = await self.review(job.id, [self.keep_of(group, path)])
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
return plan, cleanup
|
||||
|
||||
async def until(self, predicate):
|
||||
@@ -182,7 +178,7 @@ class TestFindJob(DuplicateFilesTestCase):
|
||||
gate.set()
|
||||
body = await (await legacy).text()
|
||||
job = self.api._jobs.get(summary["id"])
|
||||
await asyncio.wait_for(asyncio.shield(job.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
events = _sse_payloads(body)
|
||||
done = events[-1]
|
||||
self.assertEqual(done["type"], "complete")
|
||||
@@ -245,7 +241,7 @@ class TestFindJob(DuplicateFilesTestCase):
|
||||
self.assertTrue((await cancel.json())["cancellation_requested"])
|
||||
gate.set()
|
||||
job = self.api._jobs.get(job_id)
|
||||
await asyncio.wait_for(asyncio.shield(job.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
self.assertEqual(job.state, "cancelled")
|
||||
last = _sse_payloads(await (await stream).text())[-1]
|
||||
self.assertEqual(last, {"type": "error", "message": "Duplicate scan cancelled"})
|
||||
@@ -345,8 +341,6 @@ class TestReview(DuplicateFilesTestCase):
|
||||
self.assertEqual(plan["protected"], {"count": 0, "reasons": []})
|
||||
|
||||
async def test_cannot_delete_keeper_or_outside_group(self):
|
||||
# About which files may go, not the backup: skip its fsync-heavy copy.
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
self.trio()
|
||||
self.write("e.png", b"other pair")
|
||||
self.write("f.png", b"other pair")
|
||||
@@ -378,7 +372,7 @@ class TestReview(DuplicateFilesTestCase):
|
||||
self.assertEqual([row["label"] for row in plan["details"]], ["b.png"])
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.counters["removed"], 1)
|
||||
self.assertTrue((self.output_dir / "a.png").is_file())
|
||||
self.assertTrue((self.output_dir / "sub" / "c.png").is_file())
|
||||
@@ -502,7 +496,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
plan = await self.review(job.id, keep)
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertTrue(kept.is_file())
|
||||
self.assertFalse(gone.exists())
|
||||
self.assertFalse(video_thumb.exists())
|
||||
@@ -514,7 +508,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
self.write("b.png", b"new bytes") # changed since the review
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.counters, {"removed": 1, "kept": 2, "errors": 0})
|
||||
self.assertEqual((self.output_dir / "b.png").read_bytes(), b"new bytes")
|
||||
self.assertFalse((self.output_dir / "sub" / "c.png").exists())
|
||||
@@ -528,7 +522,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
self.write("a.png", b"edited keeper")
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.counters["removed"], 0)
|
||||
self.assertTrue((self.output_dir / "b.png").is_file())
|
||||
self.assertTrue((self.output_dir / "sub" / "c.png").is_file())
|
||||
@@ -555,7 +549,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
with patch.object(duplicate_files, "_unlink", side_effect=unlink):
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.state, "failed")
|
||||
self.assertEqual(cleanup.counters, {"removed": 1, "kept": 1, "errors": 1})
|
||||
self.assertEqual(
|
||||
@@ -590,7 +584,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
):
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.counters, {"removed": 0, "kept": 3, "errors": 1})
|
||||
self.assertEqual(
|
||||
cleanup.outcomes["errors"][0]["detail"], duplicate_files.DELETE_FAILED
|
||||
@@ -625,7 +619,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
with context:
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
finally:
|
||||
if handle is not None: # pragma: no cover - Windows
|
||||
handle.close()
|
||||
@@ -648,7 +642,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
with patch.object(duplicate_files, "_unlink", side_effect=unlink):
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.counters["removed"], 2)
|
||||
self.assertEqual(open_at_unlink, [])
|
||||
|
||||
@@ -709,7 +703,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
with patch.object(duplicate_files, "_unlink", side_effect=unlink):
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
# Its records moved to the identical kept copy before the unlink failed.
|
||||
[image] = self.api.db.get_prompt_images(prompt)
|
||||
self.assertTrue(image["image_path"].endswith("a.png"))
|
||||
@@ -725,7 +719,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
self.assertEqual((busy["code"], busy["running"]["id"]), ("busy", find["id"]))
|
||||
gate.set()
|
||||
await asyncio.wait_for(
|
||||
asyncio.shield(self.api._jobs.get(find["id"]).task), DISK_BOUND_SECONDS
|
||||
asyncio.shield(self.api._jobs.get(find["id"]).task), WAIT
|
||||
)
|
||||
# The token survived the refusal; the cleanup holds off a new find.
|
||||
release = asyncio.Event()
|
||||
@@ -743,7 +737,7 @@ class TestCleanupJob(DuplicateFilesTestCase):
|
||||
self.assertEqual(legacy.status, 409)
|
||||
release.set()
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
|
||||
async def test_cleanup_refused_up_front_when_the_backup_does_not_fit(self):
|
||||
from py.api import maintenance
|
||||
@@ -971,7 +965,7 @@ class TestReviewHelpers(DuplicateFilesTestCase):
|
||||
GalleryConfig.MONITORING_DIRECTORIES = [self._root(self.second_source())]
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.state, "failed")
|
||||
self.assertEqual(cleanup.last_event["code"], "sources-changed")
|
||||
self.assertEqual(self.backups(), [])
|
||||
@@ -991,7 +985,7 @@ class TestReviewHelpers(DuplicateFilesTestCase):
|
||||
):
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
self.assertEqual(cleanup.state, "failed")
|
||||
self.assertEqual(
|
||||
cleanup.outcomes["errors"][-1]["detail"], duplicate_files.RECORDS_FAILED
|
||||
|
||||
@@ -19,17 +19,17 @@ from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
from tests.disk_waits import DISK_BOUND_SECONDS
|
||||
from tests.open_handles import open_handles_under
|
||||
from tests import test_api_duplicate_files as dup_tests
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
import test_api_duplicate_files as dup_tests
|
||||
from disk_waits import DISK_BOUND_SECONDS
|
||||
from open_handles import open_handles_under
|
||||
|
||||
from py.api import duplicate_files # noqa: E402
|
||||
from py.config import GalleryConfig # noqa: E402
|
||||
|
||||
WAIT = 10
|
||||
|
||||
|
||||
def _locator(path):
|
||||
return os.path.normcase(os.path.realpath(str(path)))
|
||||
@@ -49,7 +49,7 @@ class SafetyTestCase(dup_tests.DuplicateFilesTestCase):
|
||||
async def apply_plan(self, plan):
|
||||
body = await self.apply(plan["token"])
|
||||
cleanup = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), DISK_BOUND_SECONDS)
|
||||
await asyncio.wait_for(asyncio.shield(cleanup.task), WAIT)
|
||||
return cleanup
|
||||
|
||||
|
||||
|
||||
@@ -1169,47 +1169,3 @@ class HistoryHostCleanupTests(unittest.IsolatedAsyncioTestCase):
|
||||
fence.validate(epoch, generation)
|
||||
with self.assertRaises(sqlite3.Error):
|
||||
api._history_fence()
|
||||
|
||||
|
||||
class SharedTempFolderRestoreTests(ImageAPITestCase):
|
||||
"""#346: a restore survives another suite starting in the shared temp folder.
|
||||
|
||||
Every ``PromptManagerAPI`` sweeps its database's folder for stages and
|
||||
uploads an interrupted restore left. A fixture database directly in the
|
||||
shared temp folder let any other test process (another worktree's suite,
|
||||
a pre-push hook) delete this restore's files mid-request. The shared
|
||||
folder here is a scratch stand-in for the system temp folder, so the test
|
||||
never sweeps the real one.
|
||||
"""
|
||||
|
||||
async def get_application(self):
|
||||
shared = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
|
||||
# Cleanups run last-in first-out: this one after the fixture's own.
|
||||
self.addCleanup(shared.cleanup)
|
||||
self.shared_tmp = shared.name
|
||||
with patch.object(tempfile, "tempdir", shared.name):
|
||||
return await super().get_application()
|
||||
|
||||
async def test_restore_survives_another_suite_sweeping_the_shared_folder(self):
|
||||
from database import restore_stage
|
||||
from database.models import PromptModel
|
||||
|
||||
self.api.db.save_prompt("kept by the upload")
|
||||
source = self.comfy_dir / "upload.db"
|
||||
self.assertTrue(self.api.db.model.backup_database(str(source)))
|
||||
stage = PromptModel.stage_restore
|
||||
swept = []
|
||||
|
||||
def stage_then_other_suite_starts(model, src_path):
|
||||
staged = stage(model, src_path)
|
||||
# What another suite's PromptManagerAPI does at startup.
|
||||
other = os.path.join(self.shared_tmp, "other-suite.db")
|
||||
swept.extend(restore_stage.sweep_restore_temps(other))
|
||||
return staged
|
||||
|
||||
form = FormData()
|
||||
form.add_field("database_file", source.read_bytes(), filename="upload.db")
|
||||
with patch.object(PromptModel, "stage_restore", stage_then_other_suite_starts):
|
||||
response = await self.client.post("/prompt_manager/restore", data=form)
|
||||
self.assertEqual(response.status, 200, await response.text())
|
||||
self.assertEqual(swept, [], "another suite's sweep reached this restore")
|
||||
|
||||
@@ -42,7 +42,6 @@ from py import lora_utils # noqa: E402
|
||||
from py import lora_manager_layout as layout_module # noqa: E402
|
||||
from py.api.thumbnail_settings import ThumbnailSettings # noqa: E402
|
||||
from py.api import PromptManagerAPI # noqa: E402
|
||||
from tests.private_temp import private_temp_db # noqa: E402
|
||||
from py.api.prompts import MAX_PAGE_LIMIT # noqa: E402
|
||||
from py.config import GalleryConfig # noqa: E402
|
||||
from utils.hashing import generate_prompt_hash # noqa: E402
|
||||
@@ -61,7 +60,8 @@ class ImageAPITestCase(AioHTTPTestCase):
|
||||
"""App with PromptManager routes, a temp DB and a temp output directory."""
|
||||
|
||||
async def get_application(self):
|
||||
self._temp_db = private_temp_db(self)
|
||||
self._temp_db = tempfile.NamedTemporaryFile(delete=False, suffix=".db")
|
||||
self._temp_db.close()
|
||||
|
||||
self._output_tmp = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
|
||||
# A fake ComfyUI tree: <tmp>/output is the gallery root and <tmp> is the
|
||||
|
||||
@@ -36,7 +36,6 @@ from aiohttp.test_utils import AioHTTPTestCase
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from py.api import PromptManagerAPI
|
||||
from tests.private_temp import private_temp_db
|
||||
from py import config_store, lora_import, lora_utils
|
||||
from py.config import IntegrationConfig, PromptManagerConfig
|
||||
from py.api import settings_routes as settings_module
|
||||
@@ -58,7 +57,8 @@ class LoraAPITestCase(AioHTTPTestCase):
|
||||
"""Stands up the PromptManager routes with a temp DB and neutral LoRA config."""
|
||||
|
||||
async def get_application(self):
|
||||
self._temp_db = private_temp_db(self)
|
||||
self._temp_db = tempfile.NamedTemporaryFile(delete=False, suffix=".db")
|
||||
self._temp_db.close()
|
||||
self.tmp_root = Path(tempfile.mkdtemp())
|
||||
self.custom_nodes = self.tmp_root / "custom_nodes"
|
||||
self.custom_nodes.mkdir()
|
||||
|
||||
@@ -5,6 +5,7 @@ Route tests for the prompt API (py/api/prompts.py).
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
@@ -28,7 +29,6 @@ except ImportError: # discovered with tests/ as the top-level directory
|
||||
|
||||
from database.operations import PromptDatabase # noqa: E402
|
||||
from py.api import PromptManagerAPI # noqa: E402
|
||||
from tests.private_temp import private_temp_db # noqa: E402
|
||||
import py.api.prompts as prompts_module # noqa: E402
|
||||
from py.api.prompts import ( # noqa: E402
|
||||
BULK_CATEGORY_MAX_PROMPTS,
|
||||
@@ -43,7 +43,8 @@ class PromptAPITestCase(AioHTTPTestCase):
|
||||
"""App with PromptManager routes and a temporary SQLite database."""
|
||||
|
||||
async def get_application(self):
|
||||
self._temp_db = private_temp_db(self)
|
||||
self._temp_db = tempfile.NamedTemporaryFile(delete=False, suffix=".db")
|
||||
self._temp_db.close()
|
||||
|
||||
app = web.Application()
|
||||
routes = web.RouteTableDef()
|
||||
|
||||
@@ -31,12 +31,6 @@ from utils.execution_history_writer import HistoryWriter
|
||||
# 0.19 s on macOS py3.13 (PR #351).
|
||||
NON_BLOCKING_CEILING = 1.0
|
||||
|
||||
# Saturation tests block the worker while they queue 1023 more items. The
|
||||
# block must hold until the test releases it (#346/#352): a 3 s timer let a
|
||||
# loaded runner unblock the worker mid-fill and drain it. This is only a hang
|
||||
# guard; the test, or cleanup, always sets the release.
|
||||
SATURATION_HOLD_SECONDS = 60
|
||||
|
||||
|
||||
class ManualClock:
|
||||
def __init__(self):
|
||||
@@ -90,26 +84,6 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
threading.Event().wait(0.005)
|
||||
self.fail("finite deadline expired waiting for worker behavior")
|
||||
|
||||
def wait_for_worker_pass(self, writer):
|
||||
"""Wait until the idle worker has run one whole loop pass from now.
|
||||
|
||||
A worker with no accepted work ends every pass by parking on its wake
|
||||
event. The first park counted may end a pass already under way; the
|
||||
second ends a pass that began after this call, so whatever the worker
|
||||
decides at the current clock value has happened (state, not a
|
||||
wall-clock pause that a loaded runner can outlast).
|
||||
"""
|
||||
wake = writer._wake
|
||||
park = wake.wait
|
||||
parks = []
|
||||
|
||||
def counted(timeout=None):
|
||||
parks.append(timeout)
|
||||
return park(timeout)
|
||||
|
||||
with patch.object(wake, "wait", counted):
|
||||
self.wait_for(lambda: len(parks) >= 2)
|
||||
|
||||
def writer(self, factory=None, ready=True):
|
||||
writer = HistoryWriter(factory or (lambda: self.db), clock=self.clock)
|
||||
self.writers.append(writer)
|
||||
@@ -327,7 +301,7 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
|
||||
def write(command):
|
||||
entered.set()
|
||||
release.wait(SATURATION_HOLD_SECONDS)
|
||||
release.wait(3)
|
||||
return original(command)
|
||||
|
||||
with patch.object(self.db, "write_execution_start", write):
|
||||
@@ -374,12 +348,9 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
first = self.sealed(writer)
|
||||
charge = first.lease.byte_count
|
||||
|
||||
left = threading.Event()
|
||||
|
||||
def write(*args, **kwargs):
|
||||
entered.set()
|
||||
release.wait(3)
|
||||
left.set()
|
||||
return original(*args, **kwargs)
|
||||
|
||||
with patch.object(self.db, "write_execution_contexts_for_finish", write):
|
||||
@@ -425,21 +396,8 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
before = monotonic()
|
||||
self.assertFalse(writer.close(0.02))
|
||||
elapsed = monotonic() - before
|
||||
# close() waits until its own monotonic deadline, so the lower
|
||||
# bound holds on any runner.
|
||||
self.assertGreaterEqual(elapsed, 0.02)
|
||||
if early_join:
|
||||
# Manual clock: only the patched join/sleep advance it, so this
|
||||
# exact bound proves early joins do not spin forever.
|
||||
self.assertLess(elapsed, 0.2)
|
||||
else:
|
||||
# Real clock: a loaded runner may run late, but close() must
|
||||
# not wait anywhere near the 3 s the blocked write holds.
|
||||
self.assertLess(elapsed, 0.02 + NON_BLOCKING_CEILING)
|
||||
# Bounded on the real clock too: close() gave up while the
|
||||
# in-flight write was still blocked, instead of waiting it out.
|
||||
self.assertFalse(left.is_set())
|
||||
self.assertTrue(writer.status().alive)
|
||||
self.assertLess(elapsed, 0.2)
|
||||
self.assertEqual(pool.charged_bytes, charge)
|
||||
replacement = HistoryWriter(lambda: self.db, context_budget=pool)
|
||||
self.writers.append(replacement)
|
||||
@@ -813,7 +771,7 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
self.wait_for(lambda: not writer.status().accepting)
|
||||
self.assertFalse(writer.submit(fresh))
|
||||
self.clock.advance(4.99)
|
||||
self.wait_for_worker_pass(writer)
|
||||
threading.Event().wait(0.08)
|
||||
self.assertEqual(probes, [0])
|
||||
self.clock.advance(0.01)
|
||||
self.wait_for(lambda: writer.status().accepting)
|
||||
@@ -836,12 +794,12 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
writer = self.writer(factory, ready=False)
|
||||
self.wait_for(lambda: len(calls) == 1)
|
||||
self.clock.advance(4.99)
|
||||
self.wait_for_worker_pass(writer)
|
||||
threading.Event().wait(0.08)
|
||||
self.assertEqual(calls, [0])
|
||||
self.clock.advance(0.01)
|
||||
self.wait_for(lambda: len(calls) == 2)
|
||||
self.clock.advance(4.99)
|
||||
self.wait_for_worker_pass(writer)
|
||||
threading.Event().wait(0.08)
|
||||
self.assertEqual(calls, [0, 5])
|
||||
self.clock.advance(0.01)
|
||||
self.wait_for(lambda: writer.status().accepting)
|
||||
@@ -895,7 +853,7 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
)
|
||||
writer.invalidate()
|
||||
release.set()
|
||||
self.wait_for_worker_pass(writer)
|
||||
threading.Event().wait(0.08)
|
||||
self.assertFalse(writer.status().accepting)
|
||||
self.assertIsNone(writer.status().epoch)
|
||||
self.assertFalse(writer.submit(self.start_command(epoch=stale_epoch)))
|
||||
@@ -913,7 +871,7 @@ class HistoryWriterTests(unittest.TestCase):
|
||||
lock.execute("BEGIN IMMEDIATE")
|
||||
before = time.monotonic()
|
||||
self.assertTrue(writer.submit(command))
|
||||
self.assertLess(time.monotonic() - before, NON_BLOCKING_CEILING)
|
||||
self.assertLess(time.monotonic() - before, 0.1)
|
||||
self.wait_for(lambda: not writer.status().accepting)
|
||||
lock.rollback()
|
||||
self.clock.advance(5)
|
||||
|
||||
@@ -18,7 +18,6 @@ from utils.execution_output_capture import seal_outputs
|
||||
from utils.execution_outputs import snapshot_output_roots
|
||||
from utils.execution_history_writer import HistoryWriter
|
||||
from tests.history_waits import drain_writer
|
||||
from tests.disk_waits import DISK_BOUND_SECONDS
|
||||
|
||||
try:
|
||||
from tests import test_execution_history_writer as fixtures
|
||||
@@ -29,17 +28,11 @@ except ImportError:
|
||||
# Upper bound a blocked fake write waits for its release. Tests prove ordering
|
||||
# with events, so this only caps a failing run and never decides a pass.
|
||||
BLOCK_SECONDS = 10
|
||||
|
||||
# Saturation tests block the worker while they queue 1023 more items. The
|
||||
# block must hold until the test releases it (#346/#352): a 3 s timer let a
|
||||
# loaded runner unblock the worker mid-fill and drain it. This is only a hang
|
||||
# guard; the test, or cleanup, always sets the release.
|
||||
SATURATION_HOLD_SECONDS = 60
|
||||
|
||||
# A restore is a full verify/migrate/backup/replace cycle and is bound by the
|
||||
# disk: a busy Windows runner took over 10 s (merge queue, #382). Waits that
|
||||
# span a restore use the shared disk-bound hang guard (tests/disk_waits.py).
|
||||
RESTORE_SECONDS = DISK_BOUND_SECONDS
|
||||
# span a restore use this ceiling, which stays under the runner's 120 s hang
|
||||
# watchdog, so a real hang still fails with the thread dump.
|
||||
RESTORE_SECONDS = 90
|
||||
|
||||
|
||||
class OutputWriterTests(fixtures.unittest.TestCase):
|
||||
@@ -598,7 +591,7 @@ class OutputWriterTests(fixtures.unittest.TestCase):
|
||||
|
||||
def write(command):
|
||||
entered.set()
|
||||
release.wait(SATURATION_HOLD_SECONDS)
|
||||
release.wait(3)
|
||||
return original(command)
|
||||
|
||||
with patch.object(self.db, "write_execution_start", write):
|
||||
|
||||
@@ -1,266 +0,0 @@
|
||||
"""A thread or pool refused at interpreter shutdown is quiet (#375).
|
||||
|
||||
When the interpreter exits, ``threading._shutdown`` first shuts every
|
||||
``ThreadPoolExecutor`` (``submit`` then raises "cannot schedule new futures
|
||||
after interpreter shutdown") and joins the non-daemon threads; only then do
|
||||
the ``atexit`` callbacks (the package teardown) run. Our daemon threads keep
|
||||
working meanwhile. ``Thread.start`` is refused ("can't create new thread at
|
||||
interpreter shutdown") only later, once the interpreter is finalizing; the
|
||||
unit tests cover that case, the child process the pool case. Such a refusal
|
||||
is logged once at DEBUG, with no traceback, and leaves nothing pending.
|
||||
|
||||
``threading.Thread`` is replaced only in the module under test (a copy of
|
||||
the ``threading`` namespace), never process-wide. The child-process test runs
|
||||
the real exit: a daemon thread queues previews while the main thread returns.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import textwrap
|
||||
import threading
|
||||
import types
|
||||
import unittest
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from py.api import thumbnail_auto
|
||||
from utils import image_monitor, parallel
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
CHILD_CEILING_SECONDS = 60 # the hang watchdog allows 120 s per test
|
||||
REPORT_PREFIX = "PM_EXIT_REPORT "
|
||||
THREAD_REFUSAL = "can't create new thread at interpreter shutdown"
|
||||
POOL_REFUSAL = "cannot schedule new futures after interpreter shutdown"
|
||||
|
||||
|
||||
def _refusing_threading():
|
||||
"""A ``threading`` namespace whose ``Thread.start`` is refused."""
|
||||
|
||||
class RefusedThread(threading.Thread):
|
||||
def start(self):
|
||||
raise RuntimeError(THREAD_REFUSAL)
|
||||
|
||||
namespace = types.SimpleNamespace(**vars(threading))
|
||||
namespace.Thread = RefusedThread
|
||||
return namespace
|
||||
|
||||
|
||||
class _ShutPool(ThreadPoolExecutor):
|
||||
def submit(self, *args, **kwargs):
|
||||
raise RuntimeError(POOL_REFUSAL)
|
||||
|
||||
|
||||
class _Capture(logging.Handler):
|
||||
def __init__(self):
|
||||
super().__init__(logging.DEBUG)
|
||||
self.records = []
|
||||
|
||||
def emit(self, record):
|
||||
self.records.append(record)
|
||||
|
||||
|
||||
def _capture(testcase, name):
|
||||
parallel._noted.clear() # "once" is per process: start each test afresh
|
||||
logger = logging.getLogger(name)
|
||||
handler = _Capture()
|
||||
previous = logger.level
|
||||
logger.addHandler(handler)
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
def restore():
|
||||
logger.removeHandler(handler)
|
||||
logger.setLevel(previous)
|
||||
|
||||
testcase.addCleanup(restore)
|
||||
return handler.records
|
||||
|
||||
|
||||
def _assert_one_debug_line(testcase, records):
|
||||
"""Exactly one refusal line, at DEBUG and without a traceback."""
|
||||
refusals = [r for r in records if "interpreter is exiting" in r.getMessage()]
|
||||
testcase.assertEqual(len(refusals), 1, [r.getMessage() for r in records])
|
||||
testcase.assertEqual(refusals[0].levelno, logging.DEBUG)
|
||||
testcase.assertEqual([r for r in records if r.levelno > logging.DEBUG], [])
|
||||
testcase.assertTrue(all(r.exc_info is None for r in records))
|
||||
|
||||
|
||||
class TestScheduleRefusedAtShutdown(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.handler = image_monitor.ImageGenerationHandler(
|
||||
mock.MagicMock(), mock.MagicMock()
|
||||
)
|
||||
self.addCleanup(self.handler.close)
|
||||
|
||||
def test_schedule_returns_false_and_leaves_nothing_pending(self):
|
||||
records = _capture(self, "prompt_manager.image_monitor")
|
||||
with mock.patch.object(image_monitor, "threading", _refusing_threading()):
|
||||
self.assertFalse(self.handler.schedule("/out/a.png"))
|
||||
self.assertFalse(self.handler.schedule("/out/b.png"))
|
||||
self.assertEqual(self.handler.pending_count(), 0)
|
||||
_assert_one_debug_line(self, records)
|
||||
|
||||
def test_another_start_failure_still_raises_and_leaves_nothing_pending(self):
|
||||
class Failing(threading.Thread):
|
||||
def start(self):
|
||||
raise RuntimeError("can't start new thread")
|
||||
|
||||
namespace = types.SimpleNamespace(**vars(threading))
|
||||
namespace.Thread = Failing
|
||||
with mock.patch.object(image_monitor, "threading", namespace):
|
||||
with self.assertRaises(RuntimeError):
|
||||
self.handler.schedule("/out/a.png")
|
||||
self.assertEqual(self.handler.pending_count(), 0)
|
||||
|
||||
def test_schedule_before_shutdown_is_unchanged(self):
|
||||
self.handler.processing_delay = 3600
|
||||
self.assertTrue(self.handler.schedule("/out/a.png"))
|
||||
self.assertEqual(self.handler.pending_count(), 1)
|
||||
|
||||
|
||||
class TestPreviewQueueRefusedAtShutdown(unittest.TestCase):
|
||||
def thumbnailer(self, logger):
|
||||
made = thumbnail_auto.AutoThumbnailer(
|
||||
lambda *a, **k: {"action": "generated"}, lambda: True, logger
|
||||
)
|
||||
self.addCleanup(made.close)
|
||||
return made
|
||||
|
||||
def test_enqueue_is_refused_quietly_and_drops_the_file(self):
|
||||
logger = logging.getLogger("prompt_manager.test_exit_window")
|
||||
records = _capture(self, logger.name)
|
||||
thumbnailer = self.thumbnailer(logger)
|
||||
src = Path("/out/a.png")
|
||||
with mock.patch.object(thumbnail_auto, "threading", _refusing_threading()):
|
||||
self.assertFalse(thumbnailer.enqueue(src, Path("/out/t/a.webp"), False))
|
||||
self.assertFalse(thumbnailer.enqueue(src, Path("/out/t/b.webp"), False))
|
||||
self.assertFalse(thumbnailer._worker_running)
|
||||
self.assertEqual(len(thumbnailer._queue), 0)
|
||||
self.assertEqual(thumbnailer._queued, set())
|
||||
self.assertTrue(thumbnailer.wait_idle(0))
|
||||
_assert_one_debug_line(self, records)
|
||||
|
||||
|
||||
class TestMapParallelRefusedAtShutdown(unittest.TestCase):
|
||||
def test_a_shut_pool_runs_the_items_on_the_calling_thread(self):
|
||||
records = _capture(self, "prompt_manager.parallel")
|
||||
with mock.patch.object(parallel, "ThreadPoolExecutor", _ShutPool):
|
||||
self.assertEqual(
|
||||
parallel.map_parallel(lambda n: n * n, [1, 2, 3], 4), [1, 4, 9]
|
||||
)
|
||||
_assert_one_debug_line(self, records)
|
||||
|
||||
def test_another_pool_error_still_raises(self):
|
||||
class Broken(ThreadPoolExecutor):
|
||||
def submit(self, *args, **kwargs):
|
||||
raise RuntimeError("something else")
|
||||
|
||||
with mock.patch.object(parallel, "ThreadPoolExecutor", Broken):
|
||||
with self.assertRaises(RuntimeError):
|
||||
parallel.map_parallel(lambda n: n, [1, 2], 2)
|
||||
|
||||
|
||||
CHILD = textwrap.dedent("""
|
||||
import threading
|
||||
|
||||
# Registered before concurrent.futures is imported, so it runs after the
|
||||
# pool's own exit hook: from here on, a pool refuses new work.
|
||||
exiting = threading.Event()
|
||||
threading._register_atexit(exiting.set)
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
root, tmp = sys.argv[1], sys.argv[2]
|
||||
sys.path.insert(0, root)
|
||||
|
||||
from py.api.thumbnail_auto import AutoThumbnailer
|
||||
|
||||
logging.basicConfig(level=logging.WARNING, stream=sys.stderr)
|
||||
logger = logging.getLogger("prompt_manager.exit_child")
|
||||
|
||||
def generate(src, dst, is_video, *, stop, guard, track):
|
||||
Path(dst).write_bytes(b"preview")
|
||||
return {"action": "generated"}
|
||||
|
||||
thumbnailer = AutoThumbnailer(
|
||||
generate, lambda: True, logger, workers=lambda: 4
|
||||
)
|
||||
queued = threading.Event()
|
||||
|
||||
def late_monitor():
|
||||
exiting.wait()
|
||||
# One batch of two: the worker maps it on a pool.
|
||||
with thumbnailer._changed:
|
||||
for name in ("a", "b"):
|
||||
src = Path(tmp, name + ".png")
|
||||
src.write_bytes(b"x")
|
||||
thumbnailer.enqueue(src, Path(tmp, name + "_thumb.webp"), False)
|
||||
queued.set()
|
||||
|
||||
def holder():
|
||||
# Non-daemon: the interpreter waits for it, inside the exit window.
|
||||
exiting.wait()
|
||||
queued.wait(30)
|
||||
idle = thumbnailer.wait_idle(30)
|
||||
written = sorted(
|
||||
n for n in os.listdir(tmp) if n.endswith("_thumb.webp")
|
||||
)
|
||||
print("PM_EXIT_REPORT " + json.dumps({"idle": idle, "written": written}))
|
||||
sys.stdout.flush()
|
||||
|
||||
threading.Thread(target=late_monitor, daemon=True).start()
|
||||
threading.Thread(target=holder).start()
|
||||
# Return normally, as ComfyUI does after Ctrl+C.
|
||||
""")
|
||||
|
||||
|
||||
class TestExitWindowChild(unittest.TestCase):
|
||||
def test_previews_queued_in_the_exit_window_log_no_traceback(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
script = os.path.join(tmp, "child.py")
|
||||
with open(script, "w", encoding="utf-8") as handle:
|
||||
handle.write(CHILD)
|
||||
env = dict(os.environ)
|
||||
env.pop("PROMPT_MANAGER_CONFIG_PATH", None)
|
||||
env["PROMPT_MANAGER_DATA_DIR"] = tmp
|
||||
env["PYTHONIOENCODING"] = "utf-8"
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, script, ROOT, tmp],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
timeout=CHILD_CEILING_SECONDS,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
except subprocess.TimeoutExpired as hung:
|
||||
self.fail(
|
||||
f"the child did not exit within {CHILD_CEILING_SECONDS} s\n"
|
||||
f"stdout:\n{hung.stdout}\nstderr:\n{hung.stderr}"
|
||||
)
|
||||
detail = f"stdout:\n{completed.stdout}\nstderr:\n{completed.stderr}"
|
||||
self.assertEqual(completed.returncode, 0, detail)
|
||||
reports = [
|
||||
line[len(REPORT_PREFIX) :]
|
||||
for line in completed.stdout.splitlines()
|
||||
if line.startswith(REPORT_PREFIX)
|
||||
]
|
||||
self.assertEqual(len(reports), 1, detail)
|
||||
report = json.loads(reports[0])
|
||||
self.assertTrue(report["idle"], detail)
|
||||
# The batch still ran: on the worker's own thread.
|
||||
self.assertEqual(report["written"], ["a_thumb.webp", "b_thumb.webp"], detail)
|
||||
self.assertNotIn("Traceback", completed.stderr, detail)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,136 +0,0 @@
|
||||
"""The legacy pages' backend contract does not move under System S13 (#330).
|
||||
|
||||
S13 points the legacy admin and gallery buttons at native System and removes
|
||||
their duplicate dialogs, changing no backend file. This snapshot pins what the
|
||||
legacy pages talk to: every registered route (method and path) and the key
|
||||
paths of ``GET /prompt_manager/settings``, the one route whose legacy request
|
||||
body S13 shrinks. Record again only with a reason in the PR body:
|
||||
|
||||
PM_RECORD_LEGACY_SNAPSHOT=1 python3 -m unittest tests.test_legacy_contract_snapshot
|
||||
|
||||
Cutover (#334) removes this test and its fixture.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from tests.test_api_admin import AdminAPITestCase
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_api_admin import AdminAPITestCase
|
||||
|
||||
from py.config import GalleryConfig # noqa: E402
|
||||
|
||||
SNAPSHOT = (
|
||||
Path(__file__).resolve().parent / "fixtures" / "legacy_contract_snapshot.json"
|
||||
)
|
||||
RECORD_ENV = "PM_RECORD_LEGACY_SNAPSHOT"
|
||||
SETTINGS = "/prompt_manager/settings"
|
||||
DEPTH = 2
|
||||
CHANGED = (
|
||||
"the legacy contract changed; if the change is intended (a new route, a new"
|
||||
" settings key), record it with PM_RECORD_LEGACY_SNAPSHOT=1 and say why in"
|
||||
" the PR body"
|
||||
)
|
||||
|
||||
|
||||
def key_paths(value, prefix="", depth=1):
|
||||
"""The key paths of a JSON object to ``DEPTH`` levels.
|
||||
|
||||
``{"a": {"b": 1}}`` gives ``a`` and ``a.b``; the items of a list of
|
||||
objects give the union of their keys as ``a[].b``.
|
||||
"""
|
||||
paths = set()
|
||||
for key, child in value.items():
|
||||
path = f"{prefix}{key}"
|
||||
paths.add(path)
|
||||
if depth >= DEPTH:
|
||||
continue
|
||||
if isinstance(child, dict):
|
||||
paths |= key_paths(child, f"{path}.", depth + 1)
|
||||
elif isinstance(child, list):
|
||||
for item in child:
|
||||
if isinstance(item, dict):
|
||||
paths |= key_paths(item, f"{path}[].", depth + 1)
|
||||
return paths
|
||||
|
||||
|
||||
class TestLegacyContractSnapshot(AdminAPITestCase):
|
||||
"""Routes and settings keys against tests/fixtures/legacy_contract_snapshot.json.
|
||||
|
||||
The base case isolates config.json (``PROMPT_MANAGER_CONFIG_PATH`` in the
|
||||
temp tree), stubs ComfyUI's ``folder_paths`` and resets the gallery roots;
|
||||
nothing here reads the maintainer's install.
|
||||
"""
|
||||
|
||||
async def asyncSetUp(self):
|
||||
await super().asyncSetUp()
|
||||
self.api._get_all_output_dirs = lambda: [str(self.output_dir)]
|
||||
GalleryConfig.MONITORING_DIRECTORIES = [str(self.output_dir)]
|
||||
|
||||
def _routes(self):
|
||||
return sorted(
|
||||
{
|
||||
f"{route.method} {route.resource.canonical}"
|
||||
for route in self.app.router.routes()
|
||||
if route.method != "HEAD"
|
||||
}
|
||||
)
|
||||
|
||||
async def _settings_keys(self):
|
||||
resp = await self.client.request("GET", SETTINGS)
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200, body)
|
||||
return sorted(key_paths(body))
|
||||
|
||||
def _snapshot(self):
|
||||
with open(SNAPSHOT, encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
def _record(self, key, value):
|
||||
recorded = self._snapshot() if SNAPSHOT.exists() else {}
|
||||
recorded[key] = value
|
||||
with open(SNAPSHOT, "w", encoding="utf-8", newline="\n") as f:
|
||||
json.dump(recorded, f, indent=2)
|
||||
f.write("\n")
|
||||
|
||||
async def test_routes_match_the_snapshot(self):
|
||||
routes = self._routes()
|
||||
if os.environ.get(RECORD_ENV) == "1":
|
||||
self._record("routes", routes)
|
||||
self.assertGreater(len(routes), 50)
|
||||
self.assertEqual(routes, self._snapshot()["routes"], CHANGED)
|
||||
|
||||
async def test_settings_response_keeps_its_keys(self):
|
||||
keys = await self._settings_keys()
|
||||
if os.environ.get(RECORD_ENV) == "1":
|
||||
self._record("settings_keys", keys)
|
||||
self.assertIn("settings.infinite_scroll", keys)
|
||||
self.assertEqual(keys, self._snapshot()["settings_keys"], CHANGED)
|
||||
|
||||
|
||||
class TestKeyPaths(unittest.TestCase):
|
||||
def test_depth_and_list_items(self):
|
||||
value = {
|
||||
"success": True,
|
||||
"settings": {"a": 1, "deep": {"hidden": 1}},
|
||||
"rows": [{"path": "x"}, {"label": "y"}],
|
||||
}
|
||||
self.assertEqual(
|
||||
sorted(key_paths(value)),
|
||||
[
|
||||
"rows",
|
||||
"rows[].label",
|
||||
"rows[].path",
|
||||
"settings",
|
||||
"settings.a",
|
||||
"settings.deep",
|
||||
"success",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,233 +0,0 @@
|
||||
"""A restore cut off by a server stop waits for its worker thread (#360).
|
||||
|
||||
A cancelled handler (or job) task cannot stop the executor thread that
|
||||
stages or publishes. Its cleanup (the stage, the upload) and the registry's
|
||||
``restore`` admission must outlive that thread, and the live library ends as
|
||||
the old or the restored one, never partial, with the safety backup kept.
|
||||
|
||||
Every positive wait is on an event. The one bounded wait (``SETTLE``) is a
|
||||
negative check: the cancelled task must still be pending while its worker is
|
||||
held.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import shutil
|
||||
import sqlite3
|
||||
import threading
|
||||
from unittest.mock import patch
|
||||
|
||||
from database import restore_stage
|
||||
from database.models import PromptModel
|
||||
from py.api import admin as admin_mod
|
||||
from py.api.jobs import RESTORE_JOB_KIND
|
||||
|
||||
try:
|
||||
from tests.test_restore_plans import RestoreTestCase, _texts
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_restore_plans import RestoreTestCase, _texts
|
||||
|
||||
WAIT = 60 # a ceiling only a hung test reaches
|
||||
SETTLE = 0.3 # the negative check's bound: the old code's cleanup takes ms
|
||||
|
||||
|
||||
class _Gate:
|
||||
"""Hold one blocking call on its worker thread until released."""
|
||||
|
||||
def __init__(self, real):
|
||||
self.real = real
|
||||
self.entered = threading.Event()
|
||||
self.release = threading.Event()
|
||||
self.done = threading.Event()
|
||||
|
||||
def method(self):
|
||||
"""A function to patch onto the class (bound like the real one)."""
|
||||
|
||||
def held(*args):
|
||||
return self(*args)
|
||||
|
||||
return held
|
||||
|
||||
def __call__(self, *args):
|
||||
self.entered.set()
|
||||
self.release.wait(WAIT)
|
||||
try:
|
||||
return self.real(*args)
|
||||
finally:
|
||||
self.done.set()
|
||||
|
||||
|
||||
class ShutdownRestoreCase(RestoreTestCase):
|
||||
def gate(self, real):
|
||||
gate = _Gate(real)
|
||||
# A failed check never leaves the worker held until WAIT.
|
||||
self._gates = getattr(self, "_gates", []) + [gate]
|
||||
self.addCleanup(gate.release.set)
|
||||
return gate
|
||||
|
||||
def stages(self):
|
||||
"""The stage folders (the upload's prefix starts the same way)."""
|
||||
return [
|
||||
name
|
||||
for name in super().stages()
|
||||
if os.path.isdir(os.path.join(self.tmpdir, name))
|
||||
]
|
||||
|
||||
def uploads(self):
|
||||
"""The upload files (not the -wal/-shm its readers may add)."""
|
||||
return sorted(
|
||||
name
|
||||
for name in os.listdir(self.tmpdir)
|
||||
if name.startswith(restore_stage.UPLOAD_PREFIX) and name.endswith(".tmp")
|
||||
)
|
||||
|
||||
def fake_upload(self, texts):
|
||||
"""Make ``_receive_restore_upload`` hand over a prepared upload."""
|
||||
self.upload(texts, "source.db")
|
||||
source = os.path.join(self.tmpdir, "uploads", "source.db")
|
||||
target = os.path.join(self.tmpdir, restore_stage.UPLOAD_PREFIX + "x.tmp")
|
||||
shutil.copyfile(source, target)
|
||||
|
||||
async def receive(request, uploads):
|
||||
uploads.append(target)
|
||||
return target, None
|
||||
|
||||
return patch.object(self.api, "_receive_restore_upload", receive)
|
||||
|
||||
async def wait_thread(self, event):
|
||||
loop = asyncio.get_running_loop()
|
||||
self.assertTrue(await loop.run_in_executor(None, event.wait, WAIT))
|
||||
|
||||
async def assert_still_pending(self, task):
|
||||
done, _ = await asyncio.wait({task}, timeout=SETTLE)
|
||||
if done:
|
||||
for gate in getattr(self, "_gates", ()):
|
||||
gate.release.set() # let the worker end before teardown
|
||||
self.fail("the cancelled task ended while its worker still ran")
|
||||
|
||||
|
||||
class TestLegacyRestoreAtShutdown(ShutdownRestoreCase):
|
||||
async def test_cancel_during_publish_waits_for_the_worker(self):
|
||||
self.save("old")
|
||||
old_db = self.api.db
|
||||
gate = self.gate(PromptModel.publish_staged)
|
||||
with (
|
||||
self.fake_upload(["restored"]),
|
||||
patch.object(PromptModel, "publish_staged", gate.method()),
|
||||
):
|
||||
task = asyncio.ensure_future(self.api.restore_database(None))
|
||||
await self.wait_thread(gate.entered)
|
||||
self.assertEqual(len(self.stages()), 1)
|
||||
task.cancel()
|
||||
await self.assert_still_pending(task)
|
||||
# The worker still has its stage and upload, and runs alone.
|
||||
self.assertEqual(len(self.stages()), 1)
|
||||
self.assertEqual(len(self.uploads()), 1)
|
||||
self.assertIsNotNone(self.api._jobs.active(RESTORE_JOB_KIND))
|
||||
gate.release.set()
|
||||
with self.assertRaises(asyncio.CancelledError):
|
||||
await asyncio.wait_for(task, WAIT)
|
||||
self.assertTrue(gate.done.is_set())
|
||||
self.assertEqual(_texts(self.db_path), ["restored"])
|
||||
self.assertIsNot(self.api.db, old_db) # the restored library adopted
|
||||
self.assertEqual(len(self.backups()), 1) # the safety backup is kept
|
||||
self.assertEqual(self.stages(), [])
|
||||
self.assertEqual(self.uploads(), [])
|
||||
self.assertIsNone(self.api._jobs.active(RESTORE_JOB_KIND))
|
||||
|
||||
async def test_cancel_during_staging_waits_and_changes_nothing(self):
|
||||
self.save("old")
|
||||
gate = self.gate(PromptModel.stage_restore)
|
||||
with (
|
||||
self.fake_upload(["restored"]),
|
||||
patch.object(PromptModel, "stage_restore", gate.method()),
|
||||
):
|
||||
task = asyncio.ensure_future(self.api.restore_database(None))
|
||||
await self.wait_thread(gate.entered)
|
||||
task.cancel()
|
||||
await self.assert_still_pending(task)
|
||||
self.assertEqual(len(self.uploads()), 1) # still being read
|
||||
self.assertIsNotNone(self.api._jobs.active(RESTORE_JOB_KIND))
|
||||
gate.release.set()
|
||||
with self.assertRaises(asyncio.CancelledError):
|
||||
await asyncio.wait_for(task, WAIT)
|
||||
self.assertEqual(_texts(self.db_path), ["old"])
|
||||
self.assertEqual(self.backups(), [])
|
||||
self.assertEqual(self.stages(), []) # the finished stage is removed
|
||||
self.assertEqual(self.uploads(), [])
|
||||
self.assertIsNone(self.api._jobs.active(RESTORE_JOB_KIND))
|
||||
|
||||
async def test_a_publish_that_fails_after_the_cancel_still_cancels(self):
|
||||
self.save("old")
|
||||
|
||||
def refuse(model, staged):
|
||||
raise sqlite3.OperationalError("refused")
|
||||
|
||||
gate = self.gate(refuse)
|
||||
with (
|
||||
self.fake_upload(["restored"]),
|
||||
patch.object(PromptModel, "publish_staged", gate.method()),
|
||||
):
|
||||
task = asyncio.ensure_future(self.api.restore_database(None))
|
||||
await self.wait_thread(gate.entered)
|
||||
task.cancel()
|
||||
await self.assert_still_pending(task)
|
||||
gate.release.set()
|
||||
# Not answered as a 500: the cancel goes on.
|
||||
with self.assertRaises(asyncio.CancelledError):
|
||||
await asyncio.wait_for(task, WAIT)
|
||||
self.assertEqual(_texts(self.db_path), ["old"])
|
||||
self.assertEqual(self.stages(), [])
|
||||
self.assertEqual(self.uploads(), [])
|
||||
self.assertIsNone(self.api._jobs.active(RESTORE_JOB_KIND))
|
||||
|
||||
async def test_a_second_cancel_still_adopts_the_restored_library(self):
|
||||
self.save("old")
|
||||
old_db = self.api.db
|
||||
publish = self.gate(PromptModel.publish_staged)
|
||||
prune = self.gate(admin_mod._prune_safety_backups)
|
||||
with (
|
||||
self.fake_upload(["restored"]),
|
||||
patch.object(PromptModel, "publish_staged", publish.method()),
|
||||
patch.object(admin_mod, "_prune_safety_backups", prune),
|
||||
):
|
||||
task = asyncio.ensure_future(self.api.restore_database(None))
|
||||
await self.wait_thread(publish.entered)
|
||||
task.cancel()
|
||||
await self.assert_still_pending(task)
|
||||
publish.release.set()
|
||||
await self.wait_thread(prune.entered) # adopting the library
|
||||
task.cancel() # the loop's own shutdown cancels again
|
||||
await self.assert_still_pending(task)
|
||||
prune.release.set()
|
||||
with self.assertRaises(asyncio.CancelledError):
|
||||
await asyncio.wait_for(task, WAIT)
|
||||
self.assertEqual(_texts(self.db_path), ["restored"])
|
||||
self.assertIsNot(self.api.db, old_db)
|
||||
self.assertEqual(len(self.backups()), 1)
|
||||
self.assertEqual(self.stages(), [])
|
||||
self.assertIsNone(self.api._jobs.active(RESTORE_JOB_KIND))
|
||||
|
||||
|
||||
class TestReviewedRestoreAtShutdown(ShutdownRestoreCase):
|
||||
async def test_cancelled_job_waits_for_the_publish(self):
|
||||
self.save("old")
|
||||
old_db = self.api.db
|
||||
body = await self.preview(self.upload(["restored"]))
|
||||
gate = self.gate(PromptModel.publish_staged)
|
||||
with patch.object(PromptModel, "publish_staged", gate.method()):
|
||||
started = await self.apply(body["token"])
|
||||
job = self.api._jobs.get(started["job"]["id"])
|
||||
await self.wait_thread(gate.entered)
|
||||
job.task.cancel()
|
||||
await self.assert_still_pending(job.task)
|
||||
self.assertEqual(len(self.stages()), 1)
|
||||
self.assertTrue(job.running) # its exclusive admission holds
|
||||
gate.release.set()
|
||||
with self.assertRaises(asyncio.CancelledError):
|
||||
await asyncio.wait_for(job.task, WAIT)
|
||||
self.assertEqual(_texts(self.db_path), ["restored"])
|
||||
self.assertIsNot(self.api.db, old_db)
|
||||
self.assertEqual(len(self.backups()), 1)
|
||||
self.assertEqual(self.stages(), [])
|
||||
self.assertFalse(job.running)
|
||||
@@ -978,27 +978,6 @@ class TestThumbnailOnCapture(AutoThumbnailCase):
|
||||
(self.output_dir / "thumbnails" / f"{name}_thumb.webp").is_file()
|
||||
)
|
||||
|
||||
async def test_a_worker_refused_at_interpreter_exit_skips_quietly(self):
|
||||
"""#375: refused once the interpreter exits; nothing stays queued."""
|
||||
|
||||
class Refused(threading.Thread):
|
||||
def start(self):
|
||||
raise RuntimeError("can't create new thread at interpreter shutdown")
|
||||
|
||||
namespace = type(sys)("threading_copy")
|
||||
namespace.__dict__.update(vars(threading))
|
||||
namespace.Thread = Refused
|
||||
thumbnailer = self.api._auto_thumbnails()
|
||||
path = make_png(self.output_dir / "late.png", size=(32, 32))
|
||||
with (
|
||||
patch.object(auto, "threading", namespace),
|
||||
self.assertNoLogs("prompt_manager.api", "INFO"),
|
||||
):
|
||||
self.assertEqual(self.api.queue_capture_thumbnail(str(path)), "skipped")
|
||||
self.assertFalse(thumbnailer._worker_running)
|
||||
self.assertEqual(len(thumbnailer._queue), 0)
|
||||
self.assertTrue(thumbnailer.wait_idle(0))
|
||||
|
||||
def test_queue_is_bounded(self):
|
||||
thumbnailer = auto.AutoThumbnailer(
|
||||
lambda *a: {"action": "generated"},
|
||||
|
||||
@@ -1,26 +1,12 @@
|
||||
"""The System browser hooks only ever reset their own server's config.json (#249 S3)
|
||||
and decide JPEG XL availability whatever the host has installed (#371)."""
|
||||
"""The System browser hooks only ever reset their own server's config.json (#249 S3)."""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from aiohttp import web
|
||||
from aiohttp.test_utils import TestClient, TestServer
|
||||
|
||||
_mock_server = mock.MagicMock()
|
||||
_mock_server.PromptServer.instance.routes = mock.MagicMock()
|
||||
sys.modules.setdefault("server", _mock_server)
|
||||
|
||||
from py.api import thumbnail_encode # noqa: E402
|
||||
from tests.browser.workspace_system_fixture import ( # noqa: E402
|
||||
register_system_commands,
|
||||
)
|
||||
from tests.browser.workspace_system_fixture import register_system_commands
|
||||
|
||||
|
||||
class TestWorkspaceSystemFixture(unittest.TestCase):
|
||||
@@ -47,48 +33,6 @@ class TestWorkspaceSystemFixture(unittest.TestCase):
|
||||
register_system_commands(None, None, None, self.root)
|
||||
self.assertEqual(elsewhere.read_bytes(), b'{"real": true}')
|
||||
|
||||
def test_no_encoder_holds_even_when_the_host_has_one(self):
|
||||
"""``{available: false}`` survives the settings read's re-probe (#371)."""
|
||||
config = self.root / "config.json"
|
||||
env = {**os.environ, "PROMPT_MANAGER_CONFIG_PATH": str(config)}
|
||||
api = types.SimpleNamespace()
|
||||
app, routes = web.Application(), web.RouteTableDef()
|
||||
real_detect = thumbnail_encode._detect
|
||||
with mock.patch.dict(os.environ, env, clear=True):
|
||||
reset = register_system_commands(app, routes, api, self.root)
|
||||
app.add_routes(routes)
|
||||
self.addCleanup(thumbnail_encode.reset_encoders)
|
||||
self.addCleanup(setattr, thumbnail_encode, "_detect", real_detect)
|
||||
|
||||
async def scenario():
|
||||
async with TestClient(TestServer(app)) as client:
|
||||
|
||||
async def post(available):
|
||||
response = await client.post(
|
||||
"/__test__/system/jxl-encoder", json={"available": available}
|
||||
)
|
||||
self.assertEqual(response.status, 200)
|
||||
|
||||
await post(False)
|
||||
status = thumbnail_encode.encoder_status()["jxl"]
|
||||
self.assertFalse(status["available"])
|
||||
self.assertIsNone(
|
||||
thumbnail_encode.detect_jxl_encoder(recheck_missing=True)
|
||||
)
|
||||
await post(True)
|
||||
status = thumbnail_encode.encoder_status()["jxl"]
|
||||
self.assertEqual(status["encoder"], "cjxl")
|
||||
await post(False)
|
||||
|
||||
# The host's real detector finds an encoder (Pillow can save JPEG XL).
|
||||
with mock.patch.object(thumbnail_encode, "_pillow_can_save_jxl", lambda: True):
|
||||
asyncio.run(scenario())
|
||||
# Reset gives detection back to the host.
|
||||
reset()
|
||||
self.assertIs(thumbnail_encode._detect, real_detect)
|
||||
status = thumbnail_encode.encoder_status()["jxl"]
|
||||
self.assertEqual(status["encoder"], "pillow")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Vendored
-29
@@ -247,32 +247,3 @@ test('the Python matrix and coverage run the suite through the watchdog runner',
|
||||
assert.doesNotMatch(found.get(id), /unittest discover/, `${id} no longer runs the bare unittest discover`);
|
||||
}
|
||||
});
|
||||
|
||||
// CI hygiene (#352): lint tools are pinned, the bandit hook can fail, the JS job caches npm.
|
||||
const readRepoFile = path => readFileSync(new URL(`../../${path}`, import.meta.url), 'utf8');
|
||||
|
||||
test('black and flake8 are pinned exactly and the lint job installs those pins', () => {
|
||||
const requirements = readRepoFile('requirements-dev.txt');
|
||||
assert.match(requirements, /^black==\d+\.\d+\.\d+$/m, 'black is pinned exactly');
|
||||
assert.match(requirements, /^flake8==\d+\.\d+\.\d+$/m, 'flake8 is pinned exactly');
|
||||
const lint = jobs(readWorkflow('test.yml')).get('lint');
|
||||
assert.match(lint, /pip install \$\(grep -E '\^\(black\|flake8\)==' requirements-dev\.txt\)/);
|
||||
assert.doesNotMatch(lint, /pip install black flake8/, 'lint no longer installs the latest releases');
|
||||
});
|
||||
|
||||
test('the frontend modules job caches npm keyed on the vendor lockfile', () => {
|
||||
const setupNode = steps(jobs(readWorkflow('test.yml')).get('js-test'))
|
||||
.find(step => /uses: actions\/setup-node@v\d+/.test(step));
|
||||
assert.match(setupNode, /^ {10}cache: npm$/m);
|
||||
assert.match(setupNode, /^ {10}cache-dependency-path: tools\/vendor\/package-lock\.json$/m);
|
||||
});
|
||||
|
||||
test('the bandit pre-commit hook fails at the severity CI enforces and passes every file', () => {
|
||||
const quality = readWorkflow('code-quality.yml');
|
||||
assert.match(quality, /^ {8}run: bandit -r \. -lll -q --exclude /m, 'CI fails on high severity');
|
||||
const hook = /- id: bandit\n([\s\S]*?)(?=\n {6}(?:- id:|#)|\n*$)/.exec(readRepoFile('.pre-commit-config.yaml'))?.[1];
|
||||
assert.ok(hook, 'the bandit hook exists');
|
||||
assert.match(hook, /^ {8}entry: bandit -lll -q$/m, 'the hook enforces the same -lll level as CI');
|
||||
assert.doesNotMatch(hook, /true|exit-zero|bash -c/, 'nothing masks the exit code or drops a file');
|
||||
assert.doesNotMatch(hook, /pass_filenames: false/, 'the hook scans the staged files');
|
||||
});
|
||||
|
||||
+1
-98
@@ -1,7 +1,6 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { COLUMNS, CONTROL_EXEMPTIONS, PAGES, REMOVAL_COLUMNS, STATUSES, declaredTitles, duplicateRemovals, idUses, methodUses,
|
||||
parseControl, parseCoverageTable, parseRemovalTable, readCoverageTable, readPage, readRemovalTable, removalProblems,
|
||||
import { COLUMNS, CONTROL_EXEMPTIONS, STATUSES, declaredTitles, parseControl, parseCoverageTable, readCoverageTable,
|
||||
rowProblems } from '../../tools/vendor/legacy-coverage.mjs';
|
||||
|
||||
// TESTING.md "Legacy System capability coverage" (#249 System S12, #329): every legacy System capability names an
|
||||
@@ -115,99 +114,3 @@ test('a covered row names its native control unless it is exempt', () => {
|
||||
assert.deepEqual([row.Status, row['Native control']], ['covered', '—']);
|
||||
}
|
||||
});
|
||||
|
||||
// TESTING.md "Legacy System UI removed by S13" (#330): a legacy element is removed only when its capability is
|
||||
// `covered`, the removal copies that row's native test, and the element is really gone from its page.
|
||||
const COVERAGE_HEADER = '| Capability | Legacy anchor | Owner | Status | Native control | Test |\n| --- | --- | --- | --- | --- | --- |';
|
||||
const REMOVAL_HEADER = '| Capability | Page | Removed ids | Removed methods | Native test |\n| --- | --- | --- | --- | --- |';
|
||||
const FIXTURE_COVERAGE = parseCoverageTable(['### Legacy System capability coverage', '', COVERAGE_HEADER,
|
||||
'| View logs | `admin.html:1` | S12 Logs | covered | `diagnostics` · combobox · "Level" | level filter works (logs.spec.mjs) |',
|
||||
'| Infinite scrolling default | `admin.html:2` | Legacy prompt list | kept in legacy | — | scrolls (list-state.test.js) |',
|
||||
''].join('\n'));
|
||||
const removalRow = cells => parseRemovalTable(['### Legacy System UI removed by S13', '', REMOVAL_HEADER, `| ${cells} |`, '']
|
||||
.join('\n'))[0];
|
||||
const EMPTY_PAGE = () => ({ html: '<button id="kept">Kept</button>', script: 'class Admin {\n kept() {}\n}\n' });
|
||||
|
||||
test('every removed legacy element cites a covered capability and its native test', () => {
|
||||
const coverage = readCoverageTable();
|
||||
const rows = readRemovalTable();
|
||||
for (const row of rows) {
|
||||
const cited = coverage.find(item => item.Capability === row.Capability);
|
||||
assert.ok(cited, `TESTING.md:${row.line} cites "${row.Capability}"`);
|
||||
assert.equal(cited.Status, 'covered', `TESTING.md:${row.line} ${row.Capability}`);
|
||||
assert.equal(row['Native test'], cited.Test, `TESTING.md:${row.line} ${row.Capability}`);
|
||||
assert.ok(Object.keys(PAGES).includes(row.Page), `TESTING.md:${row.line} page ${row.Page}`);
|
||||
}
|
||||
assert.deepEqual(duplicateRemovals(rows), []);
|
||||
assert.deepEqual(REMOVAL_COLUMNS, ['Capability', 'Page', 'Removed ids', 'Removed methods', 'Native test']);
|
||||
});
|
||||
|
||||
test('a removed id or method is gone from its page', () => {
|
||||
const coverage = readCoverageTable();
|
||||
for (const row of readRemovalTable()) {
|
||||
assert.deepEqual(removalProblems(row, coverage, page => readPage(page)), [], `TESTING.md:${row.line} ${row.Capability}`);
|
||||
}
|
||||
// Each way a page can still hold an id or a method is found; a call through another object, or a longer name, is not
|
||||
// a definition.
|
||||
const page = {
|
||||
html: '<div id="logsModal"></div><button onclick="window.admin.closeLogs()">x</button>',
|
||||
script: [
|
||||
"document.getElementById('logLevel').value;",
|
||||
"this.showModal('diagnosticsModal');",
|
||||
'<span data-id="logsContainer"></span>',
|
||||
'document.querySelector("#logLimit option");',
|
||||
"const backdrops = ['settingsModal', 'duplicatesModal'];",
|
||||
'class Admin {',
|
||||
' async refreshLogs(level) {',
|
||||
' }',
|
||||
'} displayLogs(logs) {',
|
||||
' }',
|
||||
'this.loadLogStats();',
|
||||
'// downloadLogs() {',
|
||||
'other.formatBytesLater(1);',
|
||||
].join('\n'),
|
||||
};
|
||||
assert.deepEqual(idUses('logsModal', page), ['id="logsModal" in the markup']);
|
||||
assert.deepEqual(idUses('logLevel', page), ["getElementById('logLevel')"]);
|
||||
assert.deepEqual(idUses('logLimit', page), ["querySelector('#logLimit')"]);
|
||||
assert.deepEqual(idUses('duplicatesModal', page), ["'duplicatesModal' in a modal list"]);
|
||||
assert.deepEqual(idUses('logLevelX', page), []);
|
||||
assert.deepEqual(idUses('diagnosticsModal', page), ["a …Modal('diagnosticsModal') call"]);
|
||||
assert.deepEqual(idUses('logsContainer', page), []);
|
||||
assert.deepEqual(methodUses('closeLogs', page), ['.closeLogs( called in the markup']);
|
||||
assert.deepEqual(methodUses('refreshLogs', page), ['refreshLogs() defined']);
|
||||
assert.deepEqual(methodUses('displayLogs', page), ['displayLogs() defined']);
|
||||
assert.deepEqual(methodUses('loadLogStats', page), ['.loadLogStats( called']);
|
||||
assert.deepEqual(methodUses('downloadLogs', page), []);
|
||||
assert.deepEqual(methodUses('formatBytes', page), []);
|
||||
// One id or method is listed on one row per page.
|
||||
const twice = [removalRow('View logs | admin | `#logsModal` | — | level filter works (logs.spec.mjs)'),
|
||||
{ ...removalRow('View logs | admin | `#logsModal` | — | level filter works (logs.spec.mjs)'), line: 9 }];
|
||||
assert.match(duplicateRemovals(twice).join(), /admin #logsModal is listed on rows/);
|
||||
assert.deepEqual(removalProblems(removalRow('View logs | admin | `logsModal`, `#ok` | `close` | level filter works (logs.spec.mjs)'),
|
||||
FIXTURE_COVERAGE, EMPTY_PAGE), ['malformed entry "logsModal"', 'malformed entry "close"']);
|
||||
});
|
||||
|
||||
test('a removal citing a kept-in-legacy capability fails', () => {
|
||||
const row = removalRow('Infinite scrolling default | admin | `#infiniteScrollDefault` | — | scrolls (list-state.test.js)');
|
||||
assert.deepEqual(removalProblems(row, FIXTURE_COVERAGE, EMPTY_PAGE), ['cites a kept in legacy capability']);
|
||||
const unknown = removalRow('Something else | admin | — | — | scrolls (list-state.test.js)');
|
||||
assert.deepEqual(removalProblems(unknown, FIXTURE_COVERAGE, EMPTY_PAGE), ['cites no single coverage row "Something else"']);
|
||||
});
|
||||
|
||||
test('a removal whose native test differs fails', () => {
|
||||
const row = removalRow('View logs | admin | `#logsModal` | `showLogsModal()` | another title (logs.spec.mjs)');
|
||||
assert.deepEqual(removalProblems(row, FIXTURE_COVERAGE, EMPTY_PAGE),
|
||||
['its native test differs from the coverage row\'s "level filter works (logs.spec.mjs)"']);
|
||||
assert.deepEqual(removalProblems({ ...row, 'Native test': 'level filter works (logs.spec.mjs)' }, FIXTURE_COVERAGE, EMPTY_PAGE), []);
|
||||
});
|
||||
|
||||
test('a removed id still on its page fails', () => {
|
||||
const row = removalRow('View logs | admin | `#logsModal` | `showLogsModal()` | level filter works (logs.spec.mjs)');
|
||||
const page = () => ({ html: '<div id="logsModal" class="hidden"></div>', script: 'class Admin {\n showLogsModal() {\n }\n}\n' });
|
||||
assert.deepEqual(removalProblems(row, FIXTURE_COVERAGE, page), [
|
||||
'#logsModal is still on the admin page: id="logsModal" in the markup',
|
||||
'showLogsModal() is still on the admin page: showLogsModal() defined',
|
||||
]);
|
||||
assert.deepEqual(removalProblems({ ...row, Page: 'metadata' }, FIXTURE_COVERAGE, page), ['unknown page "metadata"']);
|
||||
});
|
||||
|
||||
Vendored
+8
-140
@@ -42,30 +42,25 @@ function cells(line) {
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The rows, keyed by `columns`, of the first Markdown table under `heading` (`label` names it in errors). */
|
||||
function parseTable(markdown, heading, columns, label) {
|
||||
/** The table's rows as objects keyed by COLUMNS, from the section that starts at HEADING. */
|
||||
export function parseCoverageTable(markdown) {
|
||||
const lines = markdown.split('\n');
|
||||
const start = lines.findIndex(line => line.trim() === heading);
|
||||
if (start === -1) throw new Error(`TESTING.md has no "${heading}" section`);
|
||||
const start = lines.findIndex(line => line.trim() === HEADING);
|
||||
if (start === -1) throw new Error(`TESTING.md has no "${HEADING}" section`);
|
||||
const header = lines.findIndex((line, index) => index > start && line.trim().startsWith('|'));
|
||||
if (header === -1 || lines.slice(start + 1, header).some(line => /^#{1,3} /.test(line))) {
|
||||
throw new Error(`The ${label} section has no table`);
|
||||
throw new Error('The coverage section has no table');
|
||||
}
|
||||
const found = cells(lines[header]);
|
||||
if (found.join('|') !== columns.join('|')) throw new Error(`The ${label} table's columns are ${found.join(', ')}`);
|
||||
const columns = cells(lines[header]);
|
||||
if (columns.join('|') !== COLUMNS.join('|')) throw new Error(`The coverage table's columns are ${columns.join(', ')}`);
|
||||
const rows = [];
|
||||
for (let index = header + 2; index < lines.length && lines[index].trim().startsWith('|'); index += 1) {
|
||||
const values = cells(lines[index]);
|
||||
rows.push({ line: index + 1, ...Object.fromEntries(columns.map((column, at) => [column, values[at] ?? ''])) });
|
||||
rows.push({ line: index + 1, ...Object.fromEntries(COLUMNS.map((column, at) => [column, values[at] ?? ''])) });
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
/** The table's rows as objects keyed by COLUMNS, from the section that starts at HEADING. */
|
||||
export function parseCoverageTable(markdown) {
|
||||
return parseTable(markdown, HEADING, COLUMNS, 'coverage');
|
||||
}
|
||||
|
||||
/**
|
||||
* A Native control cell: `` `<section>` · <role> · "<name>" ``, optionally ` · in "<row title>"` when the name is shared
|
||||
* by several rows of the section → {section, role, name, row}; `—` → null; else it throws.
|
||||
@@ -218,130 +213,3 @@ export function rowProblems(row, { root = ROOT, locate = file => testFile(file,
|
||||
export function readCoverageTable(root = ROOT) {
|
||||
return parseCoverageTable(readFileSync(join(root, 'TESTING.md'), 'utf8'));
|
||||
}
|
||||
|
||||
// ── Legacy System UI removed by S13 (#330) ──────────────────────────────────────────────────────────────────────────
|
||||
// TESTING.md "Legacy System UI removed by S13": one row per covered capability whose legacy UI S13 removed, with the
|
||||
// element ids and methods that went. A row must cite a `covered` coverage row and copy its Test cell, and none of its
|
||||
// ids or methods may still be on its page.
|
||||
export const REMOVAL_HEADING = '### Legacy System UI removed by S13';
|
||||
export const REMOVAL_COLUMNS = Object.freeze(['Capability', 'Page', 'Removed ids', 'Removed methods', 'Native test']);
|
||||
/** Each legacy page's markup and script. */
|
||||
export const PAGES = Object.freeze({
|
||||
admin: Object.freeze({ html: 'web/admin.html', script: 'web/js/admin.js' }),
|
||||
gallery: Object.freeze({ html: 'web/gallery.html', script: 'web/js/gallery.js' }),
|
||||
});
|
||||
|
||||
/** The removal table's rows, from the section that starts at REMOVAL_HEADING. */
|
||||
export function parseRemovalTable(markdown) {
|
||||
return parseTable(markdown, REMOVAL_HEADING, REMOVAL_COLUMNS, 'removal');
|
||||
}
|
||||
|
||||
/** The removal table read from TESTING.md. */
|
||||
export function readRemovalTable(root = ROOT) {
|
||||
return parseRemovalTable(readFileSync(join(root, 'TESTING.md'), 'utf8'));
|
||||
}
|
||||
|
||||
/** A Removed ids or Removed methods cell → its entries without backticks (`—` → none). */
|
||||
export function listCell(cell) {
|
||||
if (cell.trim() === NONE) return [];
|
||||
return cell.split(',').map(entry => entry.trim().replace(/^`|`$/g, '')).filter(Boolean);
|
||||
}
|
||||
|
||||
/** The removed ids of a row (without `#`) and its malformed entries. */
|
||||
function removedIds(row) {
|
||||
const entries = listCell(row['Removed ids']);
|
||||
return { ids: entries.filter(entry => /^#[\w-]+$/.test(entry)).map(entry => entry.slice(1)),
|
||||
bad: entries.filter(entry => !/^#[\w-]+$/.test(entry)) };
|
||||
}
|
||||
|
||||
/** The removed methods of a row (without `()`) and its malformed entries. */
|
||||
function removedMethods(row) {
|
||||
const entries = listCell(row['Removed methods']);
|
||||
return { names: entries.filter(entry => /^[A-Za-z_$][\w$]*\(\)$/.test(entry)).map(entry => entry.slice(0, -2)),
|
||||
bad: entries.filter(entry => !/^[A-Za-z_$][\w$]*\(\)$/.test(entry)) };
|
||||
}
|
||||
|
||||
const escapeRegExp = text => text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
|
||||
/**
|
||||
* Where `id` is still on a page: `id="<id>"` in its markup or script; in its script a lookup, a `…Modal("<id>")` call
|
||||
* (`showModal`, `hideModal`) or a modal-list entry.
|
||||
*/
|
||||
export function idUses(id, { html, script }) {
|
||||
const name = escapeRegExp(id);
|
||||
const code = stripJsComments(script);
|
||||
const uses = [];
|
||||
// `id=`, not `data-id=`: the attribute starts after a space or a tag's `<name`.
|
||||
const attribute = new RegExp(`(?:^|\\s)id=["']${name}["']`, 'm');
|
||||
if (attribute.test(html)) uses.push(`id="${id}" in the markup`);
|
||||
if (attribute.test(code)) uses.push(`id="${id}" in the script`);
|
||||
if (new RegExp(`getElementById\\(\\s*["'\`]${name}["'\`]\\s*\\)`).test(code)) uses.push(`getElementById('${id}')`);
|
||||
if (new RegExp(`querySelector(?:All)?\\(\\s*["'\`]#${name}\\b`).test(code)) uses.push(`querySelector('#${id}')`);
|
||||
if (new RegExp(`\\w*Modal\\(\\s*["'\`]${name}["'\`]`).test(code)) uses.push(`a …Modal('${id}') call`);
|
||||
// A string list of modal ids (the gallery's backdrop list): the entry next to another `…Modal` entry.
|
||||
for (const list of code.matchAll(/\[[^\[\]]*\]/g)) {
|
||||
const entry = new RegExp(`["']${name}["']`);
|
||||
if (entry.test(list[0]) && /["'][\w-]*Modal["']/.test(list[0].replace(entry, ''))) {
|
||||
uses.push(`'${id}' in a modal list`);
|
||||
break;
|
||||
}
|
||||
}
|
||||
return uses;
|
||||
}
|
||||
|
||||
/**
|
||||
* Where method `name` is still on a page: a definition in its script, or a `.<name>(` call (`this.`, `window.admin.`)
|
||||
* in its script or markup.
|
||||
*/
|
||||
export function methodUses(name, { html, script }) {
|
||||
const method = escapeRegExp(name);
|
||||
const code = stripJsComments(script);
|
||||
const uses = [];
|
||||
if (new RegExp(`(?:^|[\\s;}])(?:async\\s+)?${method}\\s*\\([^)]*\\)\\s*\\{`, 'm').test(code)) uses.push(`${name}() defined`);
|
||||
if (new RegExp(`\\.${method}\\s*\\(`).test(code)) uses.push(`.${name}( called`);
|
||||
if (new RegExp(`\\.${method}\\s*\\(`).test(html)) uses.push(`.${name}( called in the markup`);
|
||||
return uses;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every problem of one removal row (empty: sound). `coverage` is the coverage table's rows; `readPage(page)` returns
|
||||
* that page's `{html, script}`.
|
||||
*/
|
||||
export function removalProblems(row, coverage, readPage) {
|
||||
const problems = [];
|
||||
const cited = coverage.filter(item => item.Capability === row.Capability);
|
||||
if (cited.length !== 1) problems.push(`cites no single coverage row "${row.Capability}"`);
|
||||
else if (cited[0].Status !== 'covered') problems.push(`cites a ${cited[0].Status} capability`);
|
||||
else if (cited[0].Test !== row['Native test']) problems.push(`its native test differs from the coverage row's "${cited[0].Test}"`);
|
||||
if (!Object.hasOwn(PAGES, row.Page)) {
|
||||
problems.push(`unknown page "${row.Page}"`);
|
||||
return problems;
|
||||
}
|
||||
const { ids, bad: badIds } = removedIds(row);
|
||||
const { names, bad: badNames } = removedMethods(row);
|
||||
for (const entry of [...badIds, ...badNames]) problems.push(`malformed entry "${entry}"`);
|
||||
const page = readPage(row.Page);
|
||||
for (const id of ids) for (const use of idUses(id, page)) problems.push(`#${id} is still on the ${row.Page} page: ${use}`);
|
||||
for (const name of names) for (const use of methodUses(name, page)) problems.push(`${name}() is still on the ${row.Page} page: ${use}`);
|
||||
return problems;
|
||||
}
|
||||
|
||||
/** Ids and methods listed on more than one row of the same page. */
|
||||
export function duplicateRemovals(rows) {
|
||||
const seen = new Map();
|
||||
const duplicates = [];
|
||||
for (const row of rows) {
|
||||
for (const entry of [...listCell(row['Removed ids']), ...listCell(row['Removed methods'])]) {
|
||||
const key = `${row.Page} ${entry}`;
|
||||
if (seen.has(key)) duplicates.push(`${key} is listed on rows ${seen.get(key)} and ${row.line}`);
|
||||
else seen.set(key, row.line);
|
||||
}
|
||||
}
|
||||
return duplicates;
|
||||
}
|
||||
|
||||
/** A page's markup and script from the repository. */
|
||||
export function readPage(page, root = ROOT) {
|
||||
const files = PAGES[page];
|
||||
return { html: readFileSync(join(root, files.html), 'utf8'), script: readFileSync(join(root, files.script), 'utf8') };
|
||||
}
|
||||
|
||||
+3
-13
@@ -35,7 +35,6 @@ from watchdog.events import FileSystemEventHandler
|
||||
|
||||
from .hashing import generate_prompt_hash
|
||||
from .metadata_extractor import ComfyUIMetadataExtractor
|
||||
from .parallel import note_refusal, refused_at_shutdown
|
||||
from .logging_config import get_logger
|
||||
from .prompt_graph import resolve_text, run_prompt_nodes
|
||||
from .prompt_pair_resolver import resolve_prompt_pair
|
||||
@@ -195,9 +194,8 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
and once its size has stopped changing.
|
||||
|
||||
Returns:
|
||||
False when the path is already pending, the pending set is full,
|
||||
the handler is closed or the interpreter is exiting (its worker
|
||||
cannot start)
|
||||
False when the path is already pending, the pending set is full or
|
||||
the handler is closed
|
||||
"""
|
||||
with self._pending_lock:
|
||||
if self._closed:
|
||||
@@ -213,15 +211,7 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
return False
|
||||
due = time.monotonic() + self.processing_delay
|
||||
self._pending[image_path] = (prompt_snapshot, due)
|
||||
try:
|
||||
self._ensure_worker()
|
||||
except RuntimeError as error:
|
||||
# No worker will ever take it: never leave it pending (#375).
|
||||
self._pending.pop(image_path, None)
|
||||
if not refused_at_shutdown(error):
|
||||
raise
|
||||
note_refusal(self.logger, "The image worker")
|
||||
return False
|
||||
self._ensure_worker()
|
||||
self._work.put(image_path)
|
||||
return True
|
||||
|
||||
|
||||
+2
-67
@@ -5,51 +5,17 @@ a CUDA-initialised, multi-threaded process is unsafe and the spawn start
|
||||
method would re-import ComfyUI's main module in every child. Pillow releases
|
||||
the GIL while decoding, resizing and encoding, so threads still scale close
|
||||
to linearly for both jobs.
|
||||
|
||||
Interpreter exit (#375): ``threading._shutdown`` shuts every thread pool
|
||||
(``submit`` then raises "cannot schedule new futures after interpreter
|
||||
shutdown") before it joins the non-daemon threads and long before the package
|
||||
teardown runs, while our daemon threads keep working. Measured on 3.12 and
|
||||
3.14, ``Thread.start`` still succeeds in that window; it is refused ("can't
|
||||
create new thread at interpreter shutdown") only once the interpreter is
|
||||
marked finalizing, after the ``atexit`` callbacks. ``refused_at_shutdown``
|
||||
recognises both; ``note_refusal`` logs one DEBUG line per place, with no
|
||||
traceback.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
SHUTDOWN_REFUSAL = "interpreter shutdown"
|
||||
_logger = logging.getLogger("prompt_manager.parallel")
|
||||
_noted = set()
|
||||
_noted_lock = threading.Lock()
|
||||
|
||||
|
||||
def refused_at_shutdown(error):
|
||||
"""Whether ``error`` is a thread or pool start refused at interpreter exit."""
|
||||
return isinstance(error, RuntimeError) and SHUTDOWN_REFUSAL in str(error)
|
||||
|
||||
|
||||
def note_refusal(logger, what):
|
||||
"""Log once per ``what``, at DEBUG and without a traceback, that ``what``
|
||||
was refused because the interpreter is exiting."""
|
||||
key = (logger.name, what)
|
||||
with _noted_lock:
|
||||
if key in _noted:
|
||||
return
|
||||
_noted.add(key)
|
||||
logger.debug(f"{what}: refused, the interpreter is exiting")
|
||||
|
||||
|
||||
def map_parallel(func, items, workers):
|
||||
"""Apply ``func`` to every item on up to ``workers`` threads, keeping order.
|
||||
|
||||
``workers`` of 1 (or fewer) runs on the calling thread with no pool. An
|
||||
exception in ``func`` propagates to the caller, so callers that must
|
||||
survive bad input wrap ``func`` themselves. A pool refused at interpreter
|
||||
exit runs the items it did not start on the calling thread (#375).
|
||||
survive bad input wrap ``func`` themselves.
|
||||
"""
|
||||
items = list(items)
|
||||
workers = max(1, int(workers or 1))
|
||||
@@ -57,35 +23,4 @@ def map_parallel(func, items, workers):
|
||||
return [func(item) for item in items]
|
||||
max_workers = min(workers, len(items))
|
||||
with ThreadPoolExecutor(max_workers, thread_name_prefix="pm-worker") as pool:
|
||||
futures = []
|
||||
try:
|
||||
for item in items:
|
||||
futures.append(pool.submit(func, item))
|
||||
except RuntimeError as error:
|
||||
if not refused_at_shutdown(error):
|
||||
raise
|
||||
note_refusal(_logger, "A worker pool")
|
||||
# Drop what was queued but not started; it runs here instead.
|
||||
# (A refused Thread.start leaves its item queued without a future
|
||||
# here: it is cancelled too, unless a busy pool thread took it in
|
||||
# between, the one case where an item can run twice.)
|
||||
pool.shutdown(wait=True, cancel_futures=True)
|
||||
else:
|
||||
return [future.result() for future in futures]
|
||||
return _run_rest(func, items, futures)
|
||||
|
||||
|
||||
def _run_rest(func, items, futures):
|
||||
"""The pool's finished results, and every item it did not run, run here.
|
||||
|
||||
After ``shutdown(cancel_futures=True)`` each submitted future either ran
|
||||
(its result or exception is kept) or was cancelled before it started.
|
||||
"""
|
||||
results = []
|
||||
for index, item in enumerate(items):
|
||||
future = futures[index] if index < len(futures) else None
|
||||
if future is not None and not future.cancelled():
|
||||
results.append(future.result())
|
||||
else:
|
||||
results.append(func(item))
|
||||
return results
|
||||
return list(pool.map(func, items))
|
||||
|
||||
+161
-8
@@ -168,11 +168,10 @@
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Settings
|
||||
</button>
|
||||
<a id="diagnosticsBtn" href="workspace#/system/diagnostics" target="_blank" rel="noopener"
|
||||
title="Opens System › Diagnostics & logs in a new tab"
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
<button id="diagnosticsBtn"
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Diagnostics
|
||||
</a>
|
||||
</button>
|
||||
<button id="maintenanceBtn"
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Maintenance
|
||||
@@ -193,11 +192,10 @@
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Auto Tag
|
||||
</button>
|
||||
<a id="logsBtn" href="workspace#/system/diagnostics" target="_blank" rel="noopener"
|
||||
title="Opens System › Diagnostics & logs in a new tab"
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
<button id="logsBtn"
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
View Logs
|
||||
</a>
|
||||
</button>
|
||||
<button id="metadataBtn"
|
||||
class="px-2.5 py-1 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Metadata
|
||||
@@ -643,6 +641,37 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Diagnostics Modal -->
|
||||
<div id="diagnosticsModal" class="fixed inset-0 bg-black/60 backdrop-blur-[2px] hidden items-center justify-center z-50">
|
||||
<div class="bg-pm-surface rounded-pm-lg max-w-4xl w-full max-h-[90vh] overflow-hidden mx-4 border border-pm shadow-pm">
|
||||
<div class="p-4 border-b border-pm">
|
||||
<div class="flex items-center justify-between">
|
||||
<h3 class="text-sm font-semibold text-pm">Gallery System Diagnostics</h3>
|
||||
<button onclick="window.admin.closeDiagnostics()" class="p-1 rounded-pm-sm hover:bg-pm-hover transition-colors">
|
||||
<svg class="w-4 h-4 text-pm-secondary" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path>
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="p-4 overflow-y-auto max-h-[calc(90vh-120px)]">
|
||||
<div class="flex items-center justify-between mb-3">
|
||||
<button id="runDiagnosticsBtn" class="px-2.5 py-1.5 bg-pm-accent hover:bg-pm-accent-hover text-pm-accent-fg text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Run Diagnostics
|
||||
</button>
|
||||
<button id="testImageLinkBtn" class="px-2.5 py-1.5 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Test Image Link
|
||||
</button>
|
||||
</div>
|
||||
<div id="diagnosticsContent" class="space-y-3">
|
||||
<div class="text-center py-6 text-pm-secondary text-xs">
|
||||
Click "Run Diagnostics" to check the gallery system status
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Maintenance Modal -->
|
||||
<div id="maintenanceModal" class="fixed inset-0 bg-black/60 backdrop-blur-[2px] hidden items-center justify-center z-50">
|
||||
<div class="bg-pm-surface rounded-pm-lg max-w-4xl w-full max-h-[90vh] overflow-hidden mx-4 border border-pm shadow-pm">
|
||||
@@ -835,6 +864,130 @@
|
||||
<span class="inline-block w-1.5 h-1.5 rounded-full bg-pm-accent mr-1.5 align-middle animate-pulse"></span><span id="scanPillText" class="align-middle">Scanning...</span>
|
||||
</button>
|
||||
|
||||
<!-- Logs Modal -->
|
||||
<div id="logsModal" class="fixed inset-0 bg-black/60 backdrop-blur-[2px] hidden items-center justify-center z-50">
|
||||
<div class="bg-pm-surface rounded-pm-lg max-w-6xl w-full max-h-[90vh] overflow-hidden mx-4 border border-pm shadow-pm">
|
||||
<div class="p-4 border-b border-pm">
|
||||
<div class="flex items-center justify-between">
|
||||
<h3 class="text-sm font-semibold text-pm">System Logs</h3>
|
||||
<button onclick="window.admin.closeLogs()" class="p-1 rounded-pm-sm hover:bg-pm-hover transition-colors">
|
||||
<svg class="w-4 h-4 text-pm-secondary" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path>
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Log Controls -->
|
||||
<div class="p-4 border-b border-pm">
|
||||
<div class="grid grid-cols-1 md:grid-cols-5 gap-3 mb-3">
|
||||
<div>
|
||||
<label class="block text-xs font-medium text-pm-secondary mb-1">Log Level</label>
|
||||
<select id="logLevel" class="w-full px-2.5 py-1.5 bg-pm-input border border-pm rounded-pm-sm text-pm text-[13px] focus:outline-none">
|
||||
<option value="">All Levels</option>
|
||||
<option value="DEBUG">Debug</option>
|
||||
<option value="INFO">Info</option>
|
||||
<option value="WARNING">Warning</option>
|
||||
<option value="ERROR">Error</option>
|
||||
<option value="CRITICAL">Critical</option>
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-xs font-medium text-pm-secondary mb-1">Limit</label>
|
||||
<select id="logLimit" class="w-full px-2.5 py-1.5 bg-pm-input border border-pm rounded-pm-sm text-pm text-[13px] focus:outline-none">
|
||||
<option value="50">Last 50</option>
|
||||
<option value="100" selected>Last 100</option>
|
||||
<option value="200">Last 200</option>
|
||||
<option value="500">Last 500</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="flex items-end">
|
||||
<button id="refreshLogsBtn" class="w-full px-2.5 py-1.5 bg-pm-accent hover:bg-pm-accent-hover text-pm-accent-fg text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Refresh
|
||||
</button>
|
||||
</div>
|
||||
<div class="flex items-end">
|
||||
<button id="downloadLogsBtn" class="w-full px-2.5 py-1.5 bg-pm-surface border border-pm hover:bg-pm-hover text-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Download
|
||||
</button>
|
||||
</div>
|
||||
<div class="flex items-end">
|
||||
<button id="clearLogsBtn" class="w-full px-2.5 py-1.5 text-pm-error bg-transparent hover:bg-pm-error-tint border border-pm text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Clear Logs
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Log Stats -->
|
||||
<div class="grid grid-cols-2 md:grid-cols-5 gap-3 mb-3">
|
||||
<div class="bg-pm-input rounded-pm-md p-2">
|
||||
<div class="text-[11px] text-pm-secondary">Total Logs</div>
|
||||
<div class="text-sm font-semibold text-pm-accent" id="logStatsTotal">-</div>
|
||||
</div>
|
||||
<div class="bg-pm-input rounded-pm-md p-2">
|
||||
<div class="text-[11px] text-pm-secondary">Errors</div>
|
||||
<div class="text-sm font-semibold text-pm-error" id="logStatsErrors">-</div>
|
||||
</div>
|
||||
<div class="bg-pm-input rounded-pm-md p-2">
|
||||
<div class="text-[11px] text-pm-secondary">Warnings</div>
|
||||
<div class="text-sm font-semibold text-pm-warning" id="logStatsWarnings">-</div>
|
||||
</div>
|
||||
<div class="bg-pm-input rounded-pm-md p-2">
|
||||
<div class="text-[11px] text-pm-secondary">Current Level</div>
|
||||
<div class="text-sm font-semibold text-pm-success" id="logStatsLevel">-</div>
|
||||
</div>
|
||||
<div class="bg-pm-input rounded-pm-md p-2">
|
||||
<div class="text-[11px] text-pm-secondary">Log Size</div>
|
||||
<div class="text-sm font-semibold text-pm" id="logStatsSize">-</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Log Settings -->
|
||||
<div class="grid grid-cols-1 md:grid-cols-3 gap-3">
|
||||
<div>
|
||||
<label class="block text-xs font-medium text-pm-secondary mb-1">Set Log Level</label>
|
||||
<select id="setLogLevel" class="w-full px-2.5 py-1.5 bg-pm-input border border-pm rounded-pm-sm text-pm text-[13px] focus:outline-none">
|
||||
<option value="DEBUG">Debug</option>
|
||||
<option value="INFO" selected>Info</option>
|
||||
<option value="WARNING">Warning</option>
|
||||
<option value="ERROR">Error</option>
|
||||
<option value="CRITICAL">Critical</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="flex items-center">
|
||||
<label class="flex items-center gap-1.5 text-pm">
|
||||
<input type="checkbox" id="consoleLogging" checked class="w-3.5 h-3.5 accent-[var(--pm-accent)]">
|
||||
<span class="text-xs">Console Logging</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="flex items-end">
|
||||
<button id="updateLogConfigBtn" class="w-full px-2.5 py-1.5 bg-pm-accent hover:bg-pm-accent-hover text-pm-accent-fg text-xs font-medium rounded-pm-sm transition-colors">
|
||||
Update Config
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Log Viewer -->
|
||||
<div class="flex-1 overflow-hidden">
|
||||
<div class="h-96 overflow-y-auto bg-pm-primary font-mono text-xs">
|
||||
<div id="logsContainer" class="p-3 space-y-0.5">
|
||||
<div class="text-center py-6 text-pm-muted text-xs">
|
||||
Click "Refresh" to load logs
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Log Files -->
|
||||
<div class="p-4 border-t border-pm">
|
||||
<h4 class="text-xs font-semibold text-pm-secondary uppercase tracking-wide mb-3">Log Files</h4>
|
||||
<div id="logFilesList" class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-2">
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Add Prompt Modal -->
|
||||
<div id="addPromptModal" class="fixed inset-0 bg-black/60 backdrop-blur-[2px] hidden items-center justify-center z-50">
|
||||
<div class="bg-pm-surface rounded-pm-lg p-4 max-w-2xl w-full mx-4 border border-pm max-h-[90vh] overflow-y-auto shadow-pm">
|
||||
|
||||
+443
@@ -47,6 +47,7 @@
|
||||
edit: ({ promptId }) => this.editPrompt(promptId),
|
||||
delete: ({ promptId }) => this.deletePrompt(promptId),
|
||||
"open-film": ({ promptId, index }) => this.openFilmStripViewer(promptId, index),
|
||||
"download-log": ({ filename }) => this.downloadLogFile(filename),
|
||||
"remove-review-tag": ({ index }) => this.removeReviewTag(index),
|
||||
"pick-tag-suggestion": ({ tag }) => this.pickTagSuggestion(tag),
|
||||
};
|
||||
@@ -127,12 +128,14 @@
|
||||
document.getElementById("exportBtn").addEventListener("click", () => this.exportPrompts());
|
||||
document.getElementById("settingsBtn").addEventListener("click", () => this.showSettingsModal());
|
||||
document.getElementById("statsBtn").addEventListener("click", () => this.showStats());
|
||||
document.getElementById("diagnosticsBtn").addEventListener("click", () => this.showDiagnosticsModal());
|
||||
document.getElementById("maintenanceBtn").addEventListener("click", () => this.showMaintenanceModal());
|
||||
document.getElementById("backupBtn").addEventListener("click", () => this.backupDatabase());
|
||||
document.getElementById("restoreBtn").addEventListener("click", () => this.showRestoreModal());
|
||||
document.getElementById("scanBtn").addEventListener("click", () => this.showScanModal());
|
||||
document.getElementById("autoTagBtn").addEventListener("click", () => this.showAutoTagModal());
|
||||
document.getElementById("addPromptBtn").addEventListener("click", () => this.showAddPromptModal());
|
||||
document.getElementById("logsBtn").addEventListener("click", () => this.showLogsModal());
|
||||
document.getElementById("metadataBtn").addEventListener("click", () => this.openMetadataViewer());
|
||||
document.getElementById("galleryBtn").addEventListener("click", () => this.openGallery());
|
||||
|
||||
@@ -212,6 +215,10 @@
|
||||
document.getElementById("confirmRestore").addEventListener("click", () => this.confirmRestore());
|
||||
document.getElementById("cancelRestore").addEventListener("click", () => this.hideModal("restoreModal"));
|
||||
|
||||
// Diagnostics modal
|
||||
document.getElementById("runDiagnosticsBtn").addEventListener("click", () => this.runDiagnostics());
|
||||
document.getElementById("testImageLinkBtn").addEventListener("click", () => this.testImageLink());
|
||||
|
||||
// Maintenance modal
|
||||
document.getElementById("runMaintenanceBtn").addEventListener("click", () => this.runMaintenance());
|
||||
document.getElementById("selectAllMaintenanceBtn").addEventListener("click", () => this.selectAllMaintenance());
|
||||
@@ -223,6 +230,12 @@
|
||||
document.getElementById("scanPill").addEventListener("click", () => this.showScanModal());
|
||||
document.getElementById("quickBackupBtn").addEventListener("click", () => this.quickBackup());
|
||||
|
||||
// Logs modal
|
||||
document.getElementById("refreshLogsBtn").addEventListener("click", () => this.refreshLogs());
|
||||
document.getElementById("downloadLogsBtn").addEventListener("click", () => this.downloadLogs());
|
||||
document.getElementById("clearLogsBtn").addEventListener("click", () => this.clearLogs());
|
||||
document.getElementById("updateLogConfigBtn").addEventListener("click", () => this.updateLogConfig());
|
||||
|
||||
// Auto Tag modals
|
||||
document.getElementById("startAutoTagBtn").addEventListener("click", () => this.startAutoTag());
|
||||
document.getElementById("startReviewBtn").addEventListener("click", () => this.startReview());
|
||||
@@ -1788,6 +1801,15 @@
|
||||
return `${size.toFixed(1)} ${units[unitIndex]}`;
|
||||
}
|
||||
|
||||
// Diagnostics functionality
|
||||
showDiagnosticsModal() {
|
||||
this.showModal("diagnosticsModal");
|
||||
}
|
||||
|
||||
closeDiagnostics() {
|
||||
this.hideModal("diagnosticsModal");
|
||||
}
|
||||
|
||||
// Maintenance functionality
|
||||
showMaintenanceModal() {
|
||||
this.showModal("maintenanceModal");
|
||||
@@ -1959,6 +1981,160 @@
|
||||
resultsContainer.innerHTML = html;
|
||||
}
|
||||
|
||||
async runDiagnostics() {
|
||||
const content = document.getElementById("diagnosticsContent");
|
||||
content.innerHTML = `
|
||||
<div class="text-center py-4">
|
||||
<div class="w-12 h-12 border-4 border-orange-500/30 border-t-orange-500 rounded-full animate-spin mx-auto mb-4"></div>
|
||||
<p class="text-pm-secondary">Running diagnostics...</p>
|
||||
</div>
|
||||
`;
|
||||
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl('/prompt_manager/diagnostics');
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
if (data.success) {
|
||||
this.renderDiagnostics(data.diagnostics);
|
||||
} else {
|
||||
throw new Error(data.error || 'Diagnostics failed');
|
||||
}
|
||||
} else {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Diagnostics error:', error);
|
||||
content.innerHTML = `
|
||||
<div class="bg-pm-error-tint border border-pm-error rounded-pm-sm p-4">
|
||||
<h4 class="text-pm-error font-medium mb-2">❌ Diagnostics Failed</h4>
|
||||
<p class="text-pm-secondary">${escapeHtml(error.message)}</p>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
}
|
||||
|
||||
renderDiagnostics(diagnostics) {
|
||||
const content = document.getElementById("diagnosticsContent");
|
||||
|
||||
let html = '<div class="space-y-4">';
|
||||
|
||||
// Summary
|
||||
html += '<div class="bg-pm-surface rounded-pm-sm p-4">';
|
||||
html += '<h4 class="text-pm font-medium mb-3">📋 Diagnostic Summary</h4>';
|
||||
html += '<div class="grid grid-cols-1 md:grid-cols-2 gap-3">';
|
||||
|
||||
for (const [category, result] of Object.entries(diagnostics)) {
|
||||
const status = result.status === 'ok' ? '✅ PASS' : (result.status === 'warning' ? '⚠️ WARNING' : '❌ FAIL');
|
||||
const statusColor = result.status === 'ok' ? 'text-pm-success' : (result.status === 'warning' ? 'text-pm-warning' : 'text-pm-error');
|
||||
|
||||
html += `
|
||||
<div class="flex justify-between items-center p-2 bg-pm-input rounded">
|
||||
<span class="text-pm-secondary capitalize">${escapeHtml(category)}</span>
|
||||
<span class="${statusColor} font-mono text-sm">${status}</span>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
html += '</div></div>';
|
||||
|
||||
// Detailed results
|
||||
for (const [category, result] of Object.entries(diagnostics)) {
|
||||
const bgColor = result.status === 'ok' ? 'bg-pm-success-tint border-pm-success' :
|
||||
(result.status === 'warning' ? 'bg-pm-warning/20 border-pm-warning' : 'bg-pm-error-tint border-pm-error');
|
||||
|
||||
html += `<div class="${bgColor} border rounded-pm-sm p-4">`;
|
||||
html += `<h4 class="text-pm font-medium mb-2 capitalize">${escapeHtml(category)}</h4>`;
|
||||
|
||||
if (result.message) {
|
||||
html += `<p class="text-pm-secondary mb-2">${escapeHtml(result.message)}</p>`;
|
||||
}
|
||||
|
||||
// Show specific details based on category
|
||||
if (category === 'database' && result.status === 'ok') {
|
||||
html += `<div class="text-sm text-pm-secondary">`;
|
||||
html += `<p>Prompts: ${escapeHtml(result.prompt_count || 0)}</p>`;
|
||||
html += `<p>Images table: ${result.has_images_table ? 'Yes' : 'No'}</p>`;
|
||||
html += `</div>`;
|
||||
}
|
||||
|
||||
if (category === 'images_table' && result.status === 'ok') {
|
||||
html += `<div class="text-sm text-pm-secondary">`;
|
||||
html += `<p>Images: ${escapeHtml(result.image_count || 0)}</p>`;
|
||||
if (result.recent_images && result.recent_images.length > 0) {
|
||||
html += `<p>Recent images:</p>`;
|
||||
html += `<ul class="ml-4 list-disc">`;
|
||||
result.recent_images.slice(0, 3).forEach(img => {
|
||||
html += `<li>${escapeHtml(img.filename)} → Prompt ${escapeHtml(img.prompt_id)}</li>`;
|
||||
});
|
||||
html += `</ul>`;
|
||||
}
|
||||
html += `</div>`;
|
||||
}
|
||||
|
||||
if (category === 'comfyui_output' && result.output_dirs) {
|
||||
html += `<div class="text-sm text-pm-secondary">`;
|
||||
html += `<p>Output directories found:</p>`;
|
||||
html += `<ul class="ml-4 list-disc">`;
|
||||
result.output_dirs.forEach(dir => {
|
||||
html += `<li>${escapeHtml(dir)}</li>`;
|
||||
});
|
||||
html += `</ul>`;
|
||||
html += `</div>`;
|
||||
}
|
||||
|
||||
if (category === 'dependencies' && result.dependencies) {
|
||||
html += `<div class="text-sm text-pm-secondary">`;
|
||||
for (const [dep, available] of Object.entries(result.dependencies)) {
|
||||
const status = available ? '✅' : '❌';
|
||||
html += `<p>${status} ${escapeHtml(dep)}</p>`;
|
||||
}
|
||||
html += `</div>`;
|
||||
}
|
||||
|
||||
html += '</div>';
|
||||
}
|
||||
|
||||
html += '</div>';
|
||||
content.innerHTML = html;
|
||||
}
|
||||
|
||||
async testImageLink() {
|
||||
if (this.prompts.length === 0) {
|
||||
this.showNotification('No prompts available for testing', 'warning');
|
||||
return;
|
||||
}
|
||||
|
||||
// Use the first prompt for testing
|
||||
const testPrompt = this.prompts[0];
|
||||
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl('/prompt_manager/diagnostics/test-link', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
prompt_id: testPrompt.id,
|
||||
image_path: '/test/fake/image.png'
|
||||
})
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
if (data.success) {
|
||||
this.showNotification('✅ Test image link created successfully!', 'success');
|
||||
// Refresh the gallery to show the test image
|
||||
setTimeout(() => this.refreshList(), 1000);
|
||||
} else {
|
||||
throw new Error(data.result?.message || 'Test failed');
|
||||
}
|
||||
} else {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Test link error:', error);
|
||||
this.showNotification(`❌ Test failed: ${error.message}`, 'error');
|
||||
}
|
||||
}
|
||||
|
||||
// Database backup and restore functionality
|
||||
async backupDatabase() {
|
||||
try {
|
||||
@@ -2321,6 +2497,273 @@
|
||||
this.showNotification(`Scan failed: ${state.message}`, "error");
|
||||
}
|
||||
|
||||
// Logs functionality
|
||||
showLogsModal() {
|
||||
this.showModal("logsModal");
|
||||
this.loadLogStats();
|
||||
this.loadLogFiles();
|
||||
this.loadLogConfig();
|
||||
}
|
||||
|
||||
closeLogs() {
|
||||
this.hideModal("logsModal");
|
||||
}
|
||||
|
||||
async loadLogStats() {
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl('/prompt_manager/logs/stats');
|
||||
const data = await response.json();
|
||||
|
||||
if (data.success) {
|
||||
const stats = data.stats;
|
||||
document.getElementById("logStatsTotal").textContent = stats.buffer_count || 0;
|
||||
document.getElementById("logStatsErrors").textContent = stats.level_counts?.ERROR || 0;
|
||||
document.getElementById("logStatsWarnings").textContent = stats.level_counts?.WARNING || 0;
|
||||
document.getElementById("logStatsLevel").textContent = stats.current_level || "INFO";
|
||||
document.getElementById("logStatsSize").textContent = this.formatBytes(stats.total_log_size || 0);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to load log stats:", error);
|
||||
}
|
||||
}
|
||||
|
||||
async loadLogConfig() {
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl('/prompt_manager/logs/config');
|
||||
const data = await response.json();
|
||||
|
||||
if (data.success) {
|
||||
const config = data.config;
|
||||
document.getElementById("setLogLevel").value = config.level || "INFO";
|
||||
document.getElementById("consoleLogging").checked = config.console_logging !== false;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to load log config:", error);
|
||||
}
|
||||
}
|
||||
|
||||
async loadLogFiles() {
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl('/prompt_manager/logs/files');
|
||||
const data = await response.json();
|
||||
|
||||
if (data.success) {
|
||||
const container = document.getElementById("logFilesList");
|
||||
|
||||
if (data.files.length === 0) {
|
||||
container.innerHTML = '<div class="col-span-full text-center text-pm-muted">No log files found</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
container.innerHTML = data.files.map(file => `
|
||||
<div class="bg-pm-surface rounded-pm-sm p-3">
|
||||
<div class="flex items-center justify-between mb-2">
|
||||
<span class="text-sm font-medium text-pm">${escapeHtml(file.filename)}</span>
|
||||
${file.is_main ? '<span class="bg-pm-accent text-xs px-2 py-1 rounded">Active</span>' : ''}
|
||||
</div>
|
||||
<div class="text-xs text-pm-secondary mb-2">
|
||||
Size: ${this.formatBytes(file.size)} | Modified: ${new Date(file.modified).toLocaleString()}
|
||||
</div>
|
||||
<button data-action="download-log" data-filename="${escapeHtml(file.filename)}"
|
||||
class="w-full px-3 py-1 bg-pm-success hover:bg-pm-success text-pm text-xs rounded transition-colors">
|
||||
📥 Download
|
||||
</button>
|
||||
</div>
|
||||
`).join('');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to load log files:", error);
|
||||
}
|
||||
}
|
||||
|
||||
async refreshLogs() {
|
||||
const level = document.getElementById("logLevel").value;
|
||||
const limit = parseInt(document.getElementById("logLimit").value);
|
||||
|
||||
try {
|
||||
const url = new URL(ApiClient.resolveUrl('/prompt_manager/logs'), window.location.origin);
|
||||
if (level) url.searchParams.set('level', level);
|
||||
url.searchParams.set('limit', limit.toString());
|
||||
|
||||
const response = await ApiClient.fetchUrl(url);
|
||||
const data = await response.json();
|
||||
|
||||
if (data.success) {
|
||||
this.displayLogs(data.logs);
|
||||
this.showNotification(`Loaded ${data.logs.length} log entries`, "success");
|
||||
} else {
|
||||
this.showNotification(`Failed to load logs: ${data.error}`, "error");
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to refresh logs:", error);
|
||||
this.showNotification("Failed to refresh logs", "error");
|
||||
}
|
||||
}
|
||||
|
||||
displayLogs(logs) {
|
||||
const container = document.getElementById("logsContainer");
|
||||
|
||||
if (logs.length === 0) {
|
||||
container.innerHTML = '<div class="text-center py-4 text-pm-muted">No logs found</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
container.innerHTML = logs.map(log => {
|
||||
const levelColors = {
|
||||
DEBUG: 'text-pm-secondary',
|
||||
INFO: 'text-pm-accent',
|
||||
WARNING: 'text-pm-warning',
|
||||
ERROR: 'text-pm-error',
|
||||
CRITICAL: 'text-pm-error'
|
||||
};
|
||||
|
||||
const levelColor = levelColors[log.level] || 'text-pm-secondary';
|
||||
const timestamp = new Date(log.timestamp).toLocaleString();
|
||||
|
||||
return `
|
||||
<div class="border-l-2 border-pm pl-3 py-1 hover:bg-pm-surface transition-colors">
|
||||
<div class="flex items-start space-x-2 text-sm">
|
||||
<span class="text-pm-muted text-xs font-mono w-24 flex-shrink-0">${escapeHtml(timestamp.split(' ')[1])}</span>
|
||||
<span class="${levelColor} font-semibold w-16 flex-shrink-0">${escapeHtml(log.level)}</span>
|
||||
<span class="text-pm-accent text-xs w-32 flex-shrink-0">${escapeHtml(log.logger)}</span>
|
||||
<span class="text-pm-secondary flex-1">${escapeHtml(log.message)}</span>
|
||||
</div>
|
||||
<div class="text-xs text-pm-muted ml-44">
|
||||
${escapeHtml(log.filename)}:${escapeHtml(log.lineno)}
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
|
||||
// Auto-scroll to bottom
|
||||
container.scrollTop = container.scrollHeight;
|
||||
}
|
||||
|
||||
async downloadLogs() {
|
||||
const level = document.getElementById("logLevel").value;
|
||||
const limit = parseInt(document.getElementById("logLimit").value);
|
||||
|
||||
try {
|
||||
const url = new URL(ApiClient.resolveUrl('/prompt_manager/logs'), window.location.origin);
|
||||
if (level) url.searchParams.set('level', level);
|
||||
url.searchParams.set('limit', limit.toString());
|
||||
|
||||
const response = await ApiClient.fetchUrl(url);
|
||||
const data = await response.json();
|
||||
|
||||
if (data.success) {
|
||||
const logText = data.logs.map(log =>
|
||||
`${log.timestamp} [${log.level}] ${log.logger}: ${log.message} (${log.filename}:${log.lineno})`
|
||||
).join('\n');
|
||||
|
||||
const blob = new Blob([logText], { type: 'text/plain' });
|
||||
const url2 = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url2;
|
||||
a.download = `prompt_manager_logs_${new Date().toISOString().slice(0, 19).replace(/:/g, '-')}.txt`;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
URL.revokeObjectURL(url2);
|
||||
|
||||
this.showNotification("Logs downloaded successfully", "success");
|
||||
} else {
|
||||
this.showNotification(`Failed to download logs: ${data.error}`, "error");
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to download logs:", error);
|
||||
this.showNotification("Failed to download logs", "error");
|
||||
}
|
||||
}
|
||||
|
||||
async downloadLogFile(filename) {
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl(`/prompt_manager/logs/download/${encodeURIComponent(filename)}`);
|
||||
|
||||
if (response.ok) {
|
||||
const blob = await response.blob();
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = filename;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
URL.revokeObjectURL(url);
|
||||
|
||||
this.showNotification(`Downloaded ${filename}`, "success");
|
||||
} else {
|
||||
this.showNotification(`Failed to download ${filename}`, "error");
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to download log file:", error);
|
||||
this.showNotification("Failed to download log file", "error");
|
||||
}
|
||||
}
|
||||
|
||||
async clearLogs() {
|
||||
if (!confirm("Are you sure you want to clear all log files? This cannot be undone.")) {
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl('/prompt_manager/logs/truncate', {
|
||||
method: 'POST'
|
||||
});
|
||||
const data = await response.json();
|
||||
|
||||
if (data.success) {
|
||||
this.showNotification(data.message, "success");
|
||||
this.refreshLogs();
|
||||
this.loadLogStats();
|
||||
this.loadLogFiles();
|
||||
} else {
|
||||
this.showNotification(`Failed to clear logs: ${data.error}`, "error");
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to clear logs:", error);
|
||||
this.showNotification("Failed to clear logs", "error");
|
||||
}
|
||||
}
|
||||
|
||||
async updateLogConfig() {
|
||||
const level = document.getElementById("setLogLevel").value;
|
||||
const consoleLogging = document.getElementById("consoleLogging").checked;
|
||||
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl('/prompt_manager/logs/config', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
body: JSON.stringify({
|
||||
level: level,
|
||||
console_logging: consoleLogging
|
||||
})
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (data.success) {
|
||||
this.showNotification("Log configuration updated", "success");
|
||||
this.loadLogStats();
|
||||
} else {
|
||||
this.showNotification(`Failed to update config: ${data.error}`, "error");
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Failed to update log config:", error);
|
||||
this.showNotification("Failed to update log configuration", "error");
|
||||
}
|
||||
}
|
||||
|
||||
formatBytes(bytes) {
|
||||
if (bytes === 0) return '0 B';
|
||||
const k = 1024;
|
||||
const sizes = ['B', 'KB', 'MB', 'GB'];
|
||||
const i = Math.floor(Math.log(bytes) / Math.log(k));
|
||||
return parseFloat((bytes / Math.pow(k, i)).toFixed(1)) + ' ' + sizes[i];
|
||||
}
|
||||
|
||||
// Pagination methods
|
||||
updatePaginationControls() {
|
||||
const { currentPage, totalPages, total, limit } = this.pagination;
|
||||
|
||||
@@ -275,13 +275,8 @@ export function createImageStream({ api, onChange = () => {}, batchSize = 40, re
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The concept's count line (app.js a446): "N of M images · K positive prompts", without the count while unknown, and
|
||||
* the page's first "Loading images…" while a first page loads.
|
||||
*/
|
||||
export function countLine({ loaded, total, positiveCount, loading = false, error = null }) {
|
||||
// A first page in flight has no count yet: "0 of 0 images" would claim the filter matched nothing (#391).
|
||||
if (loading && !loaded && !error) return 'Loading images…';
|
||||
/** The concept's count line (app.js a446): "N of M images · K positive prompts", without the count while unknown. */
|
||||
export function countLine({ loaded, total, positiveCount }) {
|
||||
const images = `${loaded.toLocaleString('en-US')} of ${total.toLocaleString('en-US')} ${noun(total, 'image')}`;
|
||||
return isCount(positiveCount) ? `${images} · ${positiveCount.toLocaleString('en-US')} ${noun(positiveCount, 'positive prompt')}` : images;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user