Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1fb4ad5495 | ||
|
|
92be8770c3 | ||
|
|
60ce03ba4b | ||
|
|
c4eb487d02 |
@@ -41,13 +41,9 @@ concurrency:
|
||||
# 7.3m-12.2m), because --shard splits by test count and the slow concept specs sort into shard 1. The page
|
||||
# workflows now run in three shards per browser with the limits unchanged; the per-shard times are
|
||||
# re-measured after three green main runs.
|
||||
# #563: coverage shards reached their 18-minute step limit three times on 2026-10-31 with no failing
|
||||
# test (PR #560 shard 2/3: 814 passed in 17.9m; queue run 41206495451 shard 1/3: 827 passed; PR #564
|
||||
# shard 2/3 twice), against recent green shards of 11m-17m. Coverage now runs in four shards with the
|
||||
# limits unchanged; the per-shard times are re-measured after three green main runs.
|
||||
jobs:
|
||||
frontend-coverage:
|
||||
name: frontend coverage (shard ${{ matrix.shard }}/4)
|
||||
name: frontend coverage (shard ${{ matrix.shard }}/3)
|
||||
runs-on: ubuntu-latest
|
||||
# Layout comparisons depend on the fonts each browser falls back to, so browser jobs run in the
|
||||
# same pinned Playwright image as local runs (tag = @playwright/test in tools/vendor/package-lock.json;
|
||||
@@ -63,7 +59,7 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
shard: [1, 2, 3, 4]
|
||||
shard: [1, 2, 3]
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
# The image's own Python 3.12 and Node 24: setup-python/setup-node install the runner's toolcache
|
||||
@@ -92,13 +88,13 @@ jobs:
|
||||
- run: npm ci --prefix tools/vendor --ignore-scripts
|
||||
# Shard 1 also runs the node units and the viewer suite, so the vendor pins and the tooling tests
|
||||
# run on the lightest shard instead.
|
||||
- if: matrix.shard == 4
|
||||
- if: matrix.shard == 3
|
||||
run: npm run --prefix tools/vendor check
|
||||
- if: matrix.shard == 4
|
||||
- if: matrix.shard == 3
|
||||
run: npm run --prefix tools/vendor test
|
||||
- name: Page workflows with coverage, shard ${{ matrix.shard }}/4
|
||||
- name: Page workflows with coverage, shard ${{ matrix.shard }}/3
|
||||
timeout-minutes: 18
|
||||
run: npm run --prefix tools/vendor test:coverage:shard -- ${{ matrix.shard }}/4
|
||||
run: npm run --prefix tools/vendor test:coverage:shard -- ${{ matrix.shard }}/3
|
||||
- name: Upload raw coverage
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
@@ -140,13 +136,13 @@ jobs:
|
||||
with:
|
||||
pattern: coverage-raw-*
|
||||
path: ${{ runner.temp }}/coverage-raw
|
||||
# coverage:merge only checks that raw files exist; merging three of four shards would report
|
||||
# coverage:merge only checks that raw files exist; merging two of three shards would report
|
||||
# partial coverage, so a missing or empty shard artifact fails here.
|
||||
- name: Verify every shard's raw coverage arrived
|
||||
shell: bash
|
||||
env:
|
||||
STAGED_RAW: ${{ runner.temp }}/coverage-raw
|
||||
EXPECTED_SHARDS: 4
|
||||
EXPECTED_SHARDS: 3
|
||||
run: |
|
||||
found=0
|
||||
for shard in $(seq 1 "$EXPECTED_SHARDS"); do
|
||||
|
||||
+9
-20
@@ -74,14 +74,14 @@ V8 files it writes under `coverage/js/raw`:
|
||||
|
||||
```sh
|
||||
PM_TEST_WORKERS=8 npm run --prefix tools/vendor test:coverage # whole run, one report
|
||||
npm run --prefix tools/vendor test:coverage:shard -- 1/4 # node units, viewer suite, workflows 1/4
|
||||
npm run --prefix tools/vendor test:coverage:shard -- 2/4 # workflows 2/4 (likewise 3/4, 4/4)
|
||||
npm run --prefix tools/vendor test:coverage:shard -- 1/3 # node units, viewer suite, workflows 1/3
|
||||
npm run --prefix tools/vendor test:coverage:shard -- 2/3 # workflows 2/3 (likewise 3/3)
|
||||
npm run --prefix tools/vendor coverage:merge # report and baseline check of every raw file
|
||||
npm run --prefix tools/vendor test:workflows -- --project webkit --shard 2/2
|
||||
```
|
||||
|
||||
`coverage:merge` refuses an empty raw directory; it does not count shards, so run
|
||||
every shard first (CI's merge job checks all four arrived). Measured on a 32-CPU
|
||||
every shard first (CI's merge job checks all three arrived). Measured on a 32-CPU
|
||||
machine in the pinned image (2026-10-21): `test:coverage` takes about 5 minutes
|
||||
with 8 workers and 24 minutes with 1; all four projects, 8 workers, each test 3
|
||||
times (6132 tests), 24 minutes. A CI-sized run (`docker run --cpus=4`,
|
||||
@@ -189,14 +189,14 @@ check, so run it before pushing a new or deleted frontend file instead of
|
||||
waiting for the `frontend coverage` jobs.
|
||||
|
||||
The `viewerjs` check runs on every PR, including backend-only changes. It is an
|
||||
aggregate job. It requires the four `frontend coverage (shard N/4)` jobs and
|
||||
aggregate job. It requires the three `frontend coverage (shard N/3)` jobs and
|
||||
`frontend coverage (merge)` to succeed on every event. The four
|
||||
`page workflows (firefox|webkit, shard N/2)` jobs run only in the merge queue,
|
||||
on `main` and on `workflow_dispatch` (see "Pull requests and the merge queue").
|
||||
`viewerjs` requires them to be skipped on a pull request and to succeed
|
||||
everywhere else. Any other failed, cancelled or skipped job fails it. Every browser job runs with `PM_TEST_WORKERS: 3` (the public
|
||||
runners have 4 vCPUs). Each coverage shard runs `test:coverage:shard -- N/4`
|
||||
(shard 1 also runs the node units and the viewer suite, shard 4 the vendor pins
|
||||
runners have 4 vCPUs). Each coverage shard runs `test:coverage:shard -- N/3`
|
||||
(shard 1 also runs the node units and the viewer suite, shard 3 the vendor pins
|
||||
and tooling tests) and uploads its raw V8 files as `coverage-raw-N` (kept 3
|
||||
days). The merge job downloads them, fails unless all three arrived with raw
|
||||
files and without clashing names, then runs `coverage:merge` (report plus
|
||||
@@ -345,10 +345,8 @@ and the time left. The steps, in order:
|
||||
monitor's callbacks (Needs review recorder, preview queue, Images listing
|
||||
listener).
|
||||
2. `ImageMonitor.stop_monitoring` works against one 5 s deadline. Closing the
|
||||
image handler joins the image worker; watchdog's `observer.stop()` and the
|
||||
observer join get what is left, which can be 0 s. `stop()` joins watchdog's
|
||||
emitters without a limit, so it runs on a daemon thread and is waited for
|
||||
only until the deadline; a later stop waits for that same call (#561).
|
||||
image handler joins the image worker; watchdog's `observer.stop()` (not
|
||||
bounded here) and the observer join get what is left, which can be 0 s.
|
||||
3. `close_auto_thumbnails(final=True)` stops the preview worker (30 s) and
|
||||
kills ffmpeg.
|
||||
4. `PromptTracker.shutdown` stops the prompt cleanup thread (2 s).
|
||||
@@ -363,15 +361,6 @@ stored, with the parts not reached False, and the `atexit` callback is
|
||||
removed: the later steps are skipped, no summary line is logged, and a later
|
||||
call returns that result and runs nothing.
|
||||
|
||||
The observer is started from a short-lived daemon thread, so every thread it
|
||||
creates is a daemon (a thread inherits that flag from the thread that creates
|
||||
it). watchdog 7 on Windows starts a plain `threading.Thread` per watch for its
|
||||
`ReadDirectoryChangesW` reader; created from ComfyUI's main thread it would not
|
||||
be a daemon, and the interpreter waits for every such thread before it runs any
|
||||
`atexit` callback, so the teardown would never start (#561).
|
||||
`tests/test_package_exit.py` runs a child whose observer does both (a plain
|
||||
reader thread, a `stop()` that never returns) and requires its exit.
|
||||
|
||||
ComfyUI never runs aiohttp's `on_cleanup` hooks, because it never calls
|
||||
`runner.cleanup()`. So `atexit` is the live host's path. It runs on Ctrl+C
|
||||
(`main.py` catches `KeyboardInterrupt` and returns) and on a normal exit.
|
||||
@@ -675,7 +664,7 @@ what runs:
|
||||
| `py3.13 on ubuntu-latest`, `py3.14 on ubuntu-latest` | not created | run |
|
||||
| `real ComfyUI compatibility` | skipped | runs |
|
||||
| `coverage gate (85%)`, `black and flake8`, `frontend modules`, `bandit (high severity)` | run | run |
|
||||
| `frontend coverage (shard N/4)`, `frontend coverage (merge)` (vendor pins and tooling tests on shard 4) | run | run |
|
||||
| `frontend coverage (shard N/3)`, `frontend coverage (merge)` (vendor pins and tooling tests on shard 3) | run | run |
|
||||
| `page workflows (firefox\|webkit, shard N/2)` | skipped | run |
|
||||
| `all tests`, `viewerjs` (gates) | run | run |
|
||||
| `perf budget` | skipped | push and dispatch only (not in the queue) |
|
||||
|
||||
@@ -120,7 +120,6 @@ except Exception as e:
|
||||
logger = get_logger("prompt_manager.init")
|
||||
logger.error(f"Failed to register API routes: {e}")
|
||||
except Exception:
|
||||
# logging failed too; a report must never stop the node loading
|
||||
pass
|
||||
|
||||
# Shared database, set up with the monitoring system below. Stays None when that
|
||||
@@ -412,7 +411,6 @@ except Exception as e:
|
||||
f"Failed to register the exit signal: {e}"
|
||||
)
|
||||
except Exception:
|
||||
# logging failed too; a report must never stop the node loading
|
||||
pass
|
||||
|
||||
try:
|
||||
@@ -430,7 +428,6 @@ except Exception as e:
|
||||
f"Failed to register the background work teardown: {e}"
|
||||
)
|
||||
except Exception:
|
||||
# logging failed too; a report must never stop the node loading
|
||||
pass
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
||||
|
||||
@@ -590,7 +590,6 @@ class _Move:
|
||||
try:
|
||||
os.rmdir(self.plan.target_dir)
|
||||
except OSError:
|
||||
# not empty or already gone: the undo leaves it alone
|
||||
pass
|
||||
|
||||
|
||||
|
||||
@@ -35,51 +35,38 @@ from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import logging
|
||||
import operator
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import weakref
|
||||
from collections import deque
|
||||
from typing import Any, Callable, Iterable, Iterator, List, Optional
|
||||
from typing import Iterable, Iterator, List, Optional
|
||||
|
||||
import sqlite3
|
||||
|
||||
CLOSED_MESSAGE = "Cannot operate on a closed database."
|
||||
# Rows fetched per step when a cursor is iterated: the connection is marked
|
||||
# in use once per chunk instead of once per row (iteration costs about what
|
||||
# a plain cursor costs). Prefetched rows belong to the cursor so iteration
|
||||
# and explicit fetches always advance the same position.
|
||||
# a plain cursor costs). A loop that stops early leaves up to this many rows
|
||||
# fetched; ``fetchone``/``fetchmany`` after a partial loop skip them.
|
||||
ITER_CHUNK = 128
|
||||
|
||||
|
||||
class ManagedCursor(sqlite3.Cursor):
|
||||
"""A cursor whose calls into SQLite count as its connection being in use."""
|
||||
|
||||
def __init__(self, *args: Any, **kwargs: Any) -> None:
|
||||
super().__init__(*args, **kwargs)
|
||||
self._pending = deque()
|
||||
self._closed = False
|
||||
|
||||
def execute(self, sql, parameters=(), /):
|
||||
return self._run_statement(super().execute, sql, parameters)
|
||||
|
||||
def executemany(self, sql, seq_of_parameters, /):
|
||||
return self._run_statement(super().executemany, sql, seq_of_parameters)
|
||||
|
||||
def _run_statement(self, operation: Callable, *args: Any) -> ManagedCursor:
|
||||
conn = self.connection
|
||||
conn._enter()
|
||||
description = self.description
|
||||
completed = False
|
||||
try:
|
||||
result = operation(*args)
|
||||
completed = True
|
||||
return result
|
||||
return super().execute(sql, parameters)
|
||||
finally:
|
||||
conn._leave()
|
||||
|
||||
def executemany(self, sql, seq_of_parameters, /):
|
||||
conn = self.connection
|
||||
conn._enter()
|
||||
try:
|
||||
return super().executemany(sql, seq_of_parameters)
|
||||
finally:
|
||||
# Argument parsing can fail before SQLite replaces the old statement.
|
||||
if completed or self.description is not description:
|
||||
self._pending.clear()
|
||||
conn._leave()
|
||||
|
||||
def executescript(self, sql_script, /):
|
||||
@@ -94,8 +81,6 @@ class ManagedCursor(sqlite3.Cursor):
|
||||
conn = self.connection
|
||||
conn._enter()
|
||||
try:
|
||||
if self._pending and not self._closed:
|
||||
return self._pending.popleft()
|
||||
return super().fetchone()
|
||||
finally:
|
||||
conn._leave()
|
||||
@@ -104,20 +89,7 @@ class ManagedCursor(sqlite3.Cursor):
|
||||
conn = self.connection
|
||||
conn._enter()
|
||||
try:
|
||||
# Let SQLite validate the original arguments, including version-specific
|
||||
# handling of zero/negative sizes, before touching prefetched rows.
|
||||
rows = super().fetchmany(*args, **kwargs)
|
||||
if not self._pending:
|
||||
return rows
|
||||
size = operator.index(
|
||||
kwargs.get("size", args[0] if args else self.arraysize)
|
||||
)
|
||||
# Python 3.12 reads all rows for size=0; 3.13+ reads none.
|
||||
if size == 0 and sys.version_info >= (3, 13):
|
||||
return rows
|
||||
self._pending.extend(rows)
|
||||
count = len(self._pending) if size <= 0 else min(size, len(self._pending))
|
||||
return [self._pending.popleft() for _ in range(count)]
|
||||
return super().fetchmany(*args, **kwargs)
|
||||
finally:
|
||||
conn._leave()
|
||||
|
||||
@@ -125,31 +97,25 @@ class ManagedCursor(sqlite3.Cursor):
|
||||
conn = self.connection
|
||||
conn._enter()
|
||||
try:
|
||||
rows = super().fetchall()
|
||||
pending = list(self._pending)
|
||||
self._pending.clear()
|
||||
return pending + rows
|
||||
return super().fetchall()
|
||||
finally:
|
||||
conn._leave()
|
||||
|
||||
def __iter__(self) -> Iterator:
|
||||
return self
|
||||
return self._chunks()
|
||||
|
||||
def _chunks(self) -> Iterator:
|
||||
while True:
|
||||
rows = self.fetchmany(ITER_CHUNK)
|
||||
if not rows:
|
||||
return
|
||||
yield from rows
|
||||
|
||||
def __next__(self):
|
||||
conn = self.connection
|
||||
pending = self._pending
|
||||
# Buffered Python rows need no SQLite lock, but retirement and cursor
|
||||
# close must still stop an iterator obtained before either operation.
|
||||
if conn._retired or self._closed:
|
||||
raise sqlite3.ProgrammingError(CLOSED_MESSAGE)
|
||||
if pending:
|
||||
return pending.popleft()
|
||||
conn._enter()
|
||||
try:
|
||||
pending.extend(super().fetchmany(ITER_CHUNK))
|
||||
if not pending:
|
||||
raise StopIteration
|
||||
return pending.popleft()
|
||||
return super().__next__()
|
||||
finally:
|
||||
conn._leave()
|
||||
|
||||
@@ -160,10 +126,7 @@ class ManagedCursor(sqlite3.Cursor):
|
||||
except sqlite3.ProgrammingError:
|
||||
return super().close() # the connection is closed or closing
|
||||
try:
|
||||
result = super().close()
|
||||
self._pending = deque()
|
||||
self._closed = True
|
||||
return result
|
||||
return super().close()
|
||||
finally:
|
||||
conn._leave()
|
||||
|
||||
@@ -330,9 +293,6 @@ class ManagedConnection(sqlite3.Connection):
|
||||
cursors = list(self._cursors)
|
||||
try:
|
||||
for cursor in cursors:
|
||||
if isinstance(cursor, ManagedCursor):
|
||||
cursor._pending = deque()
|
||||
cursor._closed = True
|
||||
with contextlib.suppress(sqlite3.Error):
|
||||
sqlite3.Cursor.close(cursor)
|
||||
super().close()
|
||||
|
||||
@@ -34,7 +34,7 @@ from .history_reads import HistoryReadOperations
|
||||
from .health import HealthOperations, iter_consistency_issues
|
||||
from .duplicate_copies import DuplicateCopyOperations
|
||||
from .workflow_storage import WorkflowStorageOperations
|
||||
from .category_import import CategoryImportOperations
|
||||
from .category_import import CategoryImportOperations, EmptyReplacement # noqa: F401
|
||||
from .image_categories import ImageCategoryOperations
|
||||
from .workflow_documents import (
|
||||
DocumentReader,
|
||||
|
||||
@@ -46,7 +46,6 @@ def remove_quietly(*paths):
|
||||
if path and os.path.exists(path):
|
||||
os.remove(path)
|
||||
except OSError:
|
||||
# cleanup after a failure must not replace that failure
|
||||
pass
|
||||
|
||||
|
||||
|
||||
@@ -66,6 +66,7 @@ def _tag_rows(page: str, order: str = "page.id") -> str:
|
||||
)
|
||||
|
||||
|
||||
_TAG_PAGE = _tag_rows("SELECT id,name FROM tags WHERE id>? ORDER BY id LIMIT ?")
|
||||
# #444 tag sorts: sort -> (key expression on tags/counted, direction). Ties and
|
||||
# "created" follow the id, so every order is total and keyset pages are exact.
|
||||
TAG_SORTS = {
|
||||
|
||||
+1
-2
@@ -176,7 +176,7 @@ def _physical_file_key(path, stat):
|
||||
volumes on Windows) falls back to the resolved, case-folded path.
|
||||
"""
|
||||
if stat.st_ino:
|
||||
return ("inode", (stat.st_dev, stat.st_ino))
|
||||
return ("inode", stat.st_dev, stat.st_ino)
|
||||
return ("path", os.path.normcase(os.path.realpath(str(path))))
|
||||
|
||||
|
||||
@@ -539,7 +539,6 @@ def _diagnose_output_dirs(configured_dirs, public_path=None):
|
||||
|
||||
candidates.append(folder_paths.get_output_directory())
|
||||
except ImportError:
|
||||
# outside ComfyUI there is no output folder to add
|
||||
pass
|
||||
|
||||
output_dirs = []
|
||||
|
||||
@@ -13,6 +13,7 @@ import math
|
||||
from ..tagging_defaults import ( # noqa: F401 - re-exported for the route
|
||||
LLM_MODEL_TYPES as LLM_MODELS,
|
||||
MAX_INSTRUCTION_BYTES,
|
||||
MAX_INSTRUCTION_CHARS,
|
||||
MODEL_TYPES,
|
||||
WD14_MODEL_TYPES as WD14_MODELS,
|
||||
validate_template,
|
||||
|
||||
@@ -424,7 +424,6 @@ class AutotagRoutesMixin:
|
||||
title = f"Tagging model download · {label}"
|
||||
_download_titles.clear()
|
||||
_download_titles[owner] = title
|
||||
admitted = False
|
||||
try:
|
||||
job = self._jobs.start(
|
||||
MODEL_DOWNLOAD_JOB_KIND,
|
||||
@@ -440,13 +439,13 @@ class AutotagRoutesMixin:
|
||||
self._run_model_download, service, model_type, owner
|
||||
),
|
||||
)
|
||||
admitted = True
|
||||
except JobBusy as refused:
|
||||
return None, busy_response(refused)
|
||||
finally:
|
||||
if not admitted: # JobBusy, or anything else: give the gate back.
|
||||
model_gate.release(owner)
|
||||
_download_titles.pop(owner, None)
|
||||
except BaseException as refused:
|
||||
# JobBusy, or anything else the registry raised: give the gate back.
|
||||
model_gate.release(owner)
|
||||
_download_titles.pop(owner, None)
|
||||
if isinstance(refused, JobBusy):
|
||||
return None, busy_response(refused)
|
||||
raise
|
||||
|
||||
def ended(_task):
|
||||
model_gate.release(owner)
|
||||
@@ -1042,12 +1041,12 @@ class AutotagRoutesMixin:
|
||||
return self._bad_request('existing must be "skip" or "merge"')
|
||||
try:
|
||||
options = self._single_options(data, model_type)
|
||||
except OptionError as error:
|
||||
return self._bad_request(str(error))
|
||||
except (ValueError, TypeError):
|
||||
return self._bad_request(
|
||||
"general_threshold and character_threshold must be numeric"
|
||||
)
|
||||
except OptionError as error:
|
||||
return self._bad_request(str(error))
|
||||
target, error = await self._select_autotag_target(data)
|
||||
if error is not None:
|
||||
return error
|
||||
|
||||
@@ -971,7 +971,6 @@ class BackupRestoreRoutesMixin:
|
||||
line = restored_line(staged.summary)
|
||||
job.log_line(line)
|
||||
job.update(progress=100, status_line=line)
|
||||
return None
|
||||
|
||||
|
||||
async def _outlive_cancel(future):
|
||||
|
||||
@@ -574,13 +574,12 @@ class DuplicateFilesMixin:
|
||||
return _fail_job(job, "sources-changed", SOURCES_CHANGED)
|
||||
ok, backup = await self._safety_backup_step(job, db, backup_on)
|
||||
if not ok:
|
||||
return None
|
||||
return
|
||||
try:
|
||||
await self._remove_duplicates(job, db, plan, backup)
|
||||
finally:
|
||||
if backup:
|
||||
await self._prune_backup_pool(job, db)
|
||||
return None
|
||||
|
||||
async def _remove_duplicates(self, job, db, plan, backup):
|
||||
targets = plan.payload["targets"]
|
||||
|
||||
@@ -27,7 +27,7 @@ def _identity(path):
|
||||
except OSError:
|
||||
return None
|
||||
if info.st_ino:
|
||||
return ("inode", (info.st_dev, info.st_ino))
|
||||
return ("inode", info.st_dev, info.st_ino)
|
||||
return ("path", os.path.normcase(os.path.realpath(str(path))))
|
||||
|
||||
|
||||
|
||||
@@ -330,13 +330,12 @@ class ImageDeleteMixin:
|
||||
return _fail_job(job, "sources-changed", SOURCES_CHANGED)
|
||||
ok, backup = await self._safety_backup_step(job, db, backup_on)
|
||||
if not ok:
|
||||
return None
|
||||
return
|
||||
try:
|
||||
await self._delete_images(job, db, targets, mode, backup)
|
||||
finally:
|
||||
if backup:
|
||||
await self._prune_backup_pool(job, db)
|
||||
return None
|
||||
|
||||
async def _delete_images(self, job, db, targets, mode, backup):
|
||||
job.update(total=len(targets), status_line=TITLES[mode])
|
||||
|
||||
@@ -531,7 +531,6 @@ class ImageTagJobsMixin:
|
||||
run = AutotagRun(
|
||||
model_type, existing, options, keep_in_memory, backup_on, owner
|
||||
)
|
||||
admitted = False
|
||||
try:
|
||||
job = self._jobs.start(
|
||||
AUTOTAG_BATCH_JOB_KIND,
|
||||
@@ -542,12 +541,11 @@ class ImageTagJobsMixin:
|
||||
groups=AUTOTAG_GROUP_NAMES,
|
||||
runner=functools.partial(self._run_autotag_batch, run),
|
||||
)
|
||||
admitted = True
|
||||
except JobBusy as refused:
|
||||
return None, _busy_refusal(refused)
|
||||
finally:
|
||||
if not admitted:
|
||||
model_gate.release(owner)
|
||||
except BaseException as refused:
|
||||
model_gate.release(owner)
|
||||
if isinstance(refused, JobBusy):
|
||||
return None, _busy_refusal(refused)
|
||||
raise
|
||||
job.task.add_done_callback(lambda _task: model_gate.release(owner))
|
||||
return job, None
|
||||
|
||||
|
||||
+7
-2
@@ -39,8 +39,10 @@ from .thumbnail_encode import (
|
||||
PREVIEW_HELD,
|
||||
EncoderClosed,
|
||||
PreviewWriteHeld,
|
||||
replace_unless_held,
|
||||
run_encoder,
|
||||
sweep_stray_temps,
|
||||
temp_sibling,
|
||||
thumbnail_frame,
|
||||
writer_settings,
|
||||
write_thumbnail,
|
||||
@@ -337,6 +339,11 @@ def _thumbnail_batch_size(workers):
|
||||
return max(4, workers * 2)
|
||||
|
||||
|
||||
# Preview files are written by thumbnail_encode in the configured format
|
||||
# (thumbnail_settings, #317); these names stay importable from here.
|
||||
_temp_sibling = temp_sibling
|
||||
_replace_unless_held = replace_unless_held
|
||||
|
||||
# A preview that could not be replaced (#249 S5 M6). Windows refuses to rename
|
||||
# over a file another process has open (WinError 5 or 32), such as a preview
|
||||
# the gallery is serving; the old preview stays and the file counts as an error.
|
||||
@@ -389,7 +396,6 @@ def _video_placeholder_image(thumbnail_size):
|
||||
font=font,
|
||||
)
|
||||
except (OSError, AttributeError):
|
||||
# a placeholder without its label still works
|
||||
pass
|
||||
return img
|
||||
|
||||
@@ -493,7 +499,6 @@ def _preview_names(folders, folder):
|
||||
except OSError:
|
||||
continue
|
||||
except OSError:
|
||||
# an unreadable folder lists as empty; nothing there to cache
|
||||
pass
|
||||
folders[folder] = names
|
||||
return names
|
||||
|
||||
@@ -250,7 +250,7 @@ class JobRoutesMixin:
|
||||
try:
|
||||
await response.write(_summary_event(job))
|
||||
async with contextlib.aclosing(job.events()) as events:
|
||||
async for _ in events:
|
||||
async for _payload in events:
|
||||
await response.write(_summary_event(job))
|
||||
# The last event is published before the job takes its final state.
|
||||
await response.write(_summary_event(job))
|
||||
|
||||
@@ -207,7 +207,6 @@ class LibraryLocationRoutesMixin:
|
||||
try:
|
||||
size = os.path.getsize(location.db_path)
|
||||
except OSError:
|
||||
# no library file yet: report size 0
|
||||
pass
|
||||
return {
|
||||
"path": self._library_public_path(location.db_path),
|
||||
@@ -374,7 +373,6 @@ class LibraryLocationRoutesMixin:
|
||||
line += ". Logs move at the next restart."
|
||||
job.log_line(line)
|
||||
job.update(progress=100, status_line=line)
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _fail_library_move(job, code, message):
|
||||
|
||||
@@ -205,4 +205,3 @@ class LogCleanupMixin:
|
||||
line += f", {failed} could not be deleted"
|
||||
job.log_line(line)
|
||||
job.update(processed=len(files), progress=100, status_line=line)
|
||||
return None
|
||||
|
||||
@@ -436,7 +436,7 @@ class LoraImportRoutesMixin:
|
||||
job.log_line(f"{CANCELLED} {CACHE_NOTE}")
|
||||
job.update(status_line=CANCELLED)
|
||||
job.finish("cancelled")
|
||||
return None
|
||||
return
|
||||
except Exception:
|
||||
self.logger.exception("LoRA import staging failed")
|
||||
return _fail_job(job, "import-failed", f"{FAILED} {CACHE_NOTE}")
|
||||
@@ -455,10 +455,10 @@ class LoraImportRoutesMixin:
|
||||
job.log_line(f"{CANCELLED} {CACHE_NOTE}")
|
||||
job.update(status_line=CANCELLED)
|
||||
job.finish("cancelled")
|
||||
return None
|
||||
return
|
||||
ok, _backup = await self._safety_backup_step(job, db, backup_on)
|
||||
if not ok:
|
||||
return None
|
||||
return
|
||||
# I1: the files are listed and fingerprinted again right before the
|
||||
# replace, so a file changed, removed or added while the downloads ran
|
||||
# (LoraManager refreshing its CivitAI info) aborts with nothing deleted.
|
||||
@@ -466,7 +466,6 @@ class LoraImportRoutesMixin:
|
||||
return _fail_job(job, "plan-changed", f"{SOURCE_CHANGED} {CACHE_NOTE}")
|
||||
job.update(status_line="Replacing the LoRA collection", progress=95)
|
||||
await self._replace_in_job(job, db, plan, staged)
|
||||
return None
|
||||
|
||||
def _count_downloads(self, job, staged):
|
||||
downloaded = staged.stats.get("downloaded", 0)
|
||||
@@ -519,4 +518,3 @@ class LoraImportRoutesMixin:
|
||||
staged.stats.get("download_errors", 0),
|
||||
)
|
||||
job.update(progress=100, processed=job.total, status_line=line)
|
||||
return None
|
||||
|
||||
@@ -44,7 +44,6 @@ _FILES_CHANGED = (
|
||||
"The LoRA metadata files changed during the scan; the previous imports "
|
||||
"were kept. Run the scan again."
|
||||
)
|
||||
_LORA_SCAN_FAILED = "The LoRA import failed. Check the server log for details."
|
||||
_IMPORTS_CHANGED = (
|
||||
"The imported LoRA prompts changed during the scan; the previous imports "
|
||||
"were kept. Run the scan again."
|
||||
@@ -408,21 +407,10 @@ class LoraIntegrationMixin:
|
||||
return
|
||||
if isinstance(error, PlanChanged):
|
||||
message = _IMPORTS_CHANGED
|
||||
elif isinstance(
|
||||
error,
|
||||
(
|
||||
LoraFilesChanged,
|
||||
LoraSettingsUnusable,
|
||||
EmptyReplacement,
|
||||
PairDependencyError,
|
||||
),
|
||||
):
|
||||
elif isinstance(error, (LoraFilesChanged, LoraSettingsUnusable)):
|
||||
message = str(error)
|
||||
elif isinstance(error, OSError):
|
||||
# Reduced to its reason and the file's basename.
|
||||
message = self._public_error(error)
|
||||
else:
|
||||
message = _LORA_SCAN_FAILED
|
||||
message = self._public_error(error)
|
||||
if isinstance(
|
||||
error,
|
||||
(
|
||||
@@ -548,7 +536,6 @@ class LoraIntegrationMixin:
|
||||
except asyncio.CancelledError:
|
||||
interrupted = True
|
||||
except Exception:
|
||||
# the await after this loop raises it for the caller
|
||||
pass
|
||||
if interrupted:
|
||||
raise asyncio.CancelledError
|
||||
|
||||
@@ -673,7 +673,6 @@ class MaintenanceRoutesMixin:
|
||||
line = _changed_line(plan.operation, result["changed"], result.get("marks", 0))
|
||||
job.log_line(line)
|
||||
job.update(progress=100, status_line=f"{line} · {backup_note(backup)}")
|
||||
return None
|
||||
|
||||
def _maintenance_rows(self, operation, described):
|
||||
"""Outcome rows for the changed records (root-relative paths only)."""
|
||||
|
||||
@@ -533,7 +533,6 @@ class AutoThumbnailer:
|
||||
try:
|
||||
process.kill()
|
||||
except OSError:
|
||||
# the process has already exited
|
||||
pass
|
||||
if worker is None:
|
||||
return True
|
||||
|
||||
@@ -158,7 +158,6 @@ def _unlink_quietly(path):
|
||||
try:
|
||||
Path(path).unlink()
|
||||
except FileNotFoundError:
|
||||
# already gone, which is what was wanted
|
||||
pass
|
||||
except OSError as exc:
|
||||
logger.debug(f"Could not remove temp file {Path(path).name}: {exc}")
|
||||
@@ -222,7 +221,6 @@ def shutdown_encoders():
|
||||
try:
|
||||
process.kill()
|
||||
except OSError:
|
||||
# the process has already exited
|
||||
pass
|
||||
|
||||
|
||||
@@ -361,12 +359,10 @@ def thumbnail_frame(img, edge):
|
||||
try:
|
||||
img.seek(0)
|
||||
except (EOFError, AttributeError, ValueError):
|
||||
# not seekable or a single frame: use the current one
|
||||
pass
|
||||
try:
|
||||
img.draft("RGB", (edge, edge)) # JPEG: decode at a reduced scale
|
||||
except (AttributeError, ValueError, OSError):
|
||||
# draft is only a speedup; the full decode below still works
|
||||
pass
|
||||
frame = img.convert("RGB") if img.mode != "RGB" else img.copy()
|
||||
frame.thumbnail((edge, edge), Image.Resampling.LANCZOS)
|
||||
|
||||
@@ -275,4 +275,3 @@ class WorkflowStorageRoutesMixin:
|
||||
job.record_counted("changed", 1)
|
||||
job.log_line(line)
|
||||
job.update(progress=100, status_line=line)
|
||||
return None
|
||||
|
||||
@@ -109,7 +109,6 @@ def _write_atomically(path: str, payload: bytes) -> None:
|
||||
try:
|
||||
os.unlink(temp_path)
|
||||
except OSError:
|
||||
# nothing to clean up; the original error is raised below
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
@@ -642,7 +642,6 @@ def layout_summary(layout: LoraManagerLayout, home: Path) -> Dict:
|
||||
relative = Path(settings_file).relative_to(home)
|
||||
settings_file = os.path.join("~", str(relative))
|
||||
except ValueError:
|
||||
# outside the home folder: show the full path
|
||||
pass
|
||||
return {
|
||||
"source": layout.source,
|
||||
|
||||
@@ -585,7 +585,6 @@ def _read_capped(resp, cap: int) -> Optional[bytes]:
|
||||
if int(declared) > cap:
|
||||
return None
|
||||
except ValueError:
|
||||
# a bad header says nothing; the read below enforces the cap
|
||||
pass
|
||||
|
||||
chunks = []
|
||||
@@ -708,7 +707,6 @@ def _save_atomically(image, local_path: Path) -> None:
|
||||
try:
|
||||
os.remove(temp)
|
||||
except OSError:
|
||||
# nothing to clean up; the original error is raised below
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "promptmanager"
|
||||
description = "A powerful ComfyUI custom node that extends the standard text encoder with persistent prompt storage, advanced search capabilities, and an automatic image gallery system using SQLite."
|
||||
version = "3.4.1"
|
||||
version = "3.4.0"
|
||||
license = {file = "LICENSE"}
|
||||
requires-python = ">=3.12"
|
||||
dependencies = ["watchdog>=2.1.0", "Pillow>=8.0.0"]
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Runtime dependencies for PromptManager.
|
||||
# aiohttp, numpy and torch come from ComfyUI itself and are not listed here.
|
||||
watchdog>=2.1.0 # Output directory monitoring
|
||||
watchdog>=2.1.0 # Output directory monitoring
|
||||
Pillow>=8.0.0 # Image metadata extraction and thumbnails
|
||||
|
||||
@@ -6,6 +6,7 @@ import { THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate
|
||||
// app.js 1428–1432) and its backup box with the concept's System `.notice`; the differences are D55. The job view
|
||||
// has no concept counterpart: it is compared with the approved mockup `activity-scan.html` (a local, ignored file),
|
||||
// whose structure and CSS values are restated below (D56). Each comparison is shown to notice a mutation.
|
||||
const workspace = '/prompt_manager/workspace';
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
|
||||
@@ -109,7 +109,6 @@ class ResetWatch:
|
||||
stream.write(text)
|
||||
stream.flush()
|
||||
except (OSError, ValueError):
|
||||
# the stream closed; a watcher must not fail the server
|
||||
pass
|
||||
|
||||
|
||||
|
||||
@@ -156,9 +156,8 @@ test('at 390px the window fits the screen, actions stay reachable and the page s
|
||||
expect(frame.bottom-frame.top).toBeLessThan(844);
|
||||
expect(frame.bottom-frame.top).toBeGreaterThanOrEqual(280);
|
||||
expect(await page.evaluate(()=>document.documentElement.scrollWidth)).toBeLessThanOrEqual(390);
|
||||
// The first page's last row's actions can be scrolled to inside the window and fit its width.
|
||||
// Pinned to row 100: reaching it loads the next page (D183), and last() would then follow row 200.
|
||||
const last=rows(page).nth(99).getByRole('button',{name:'Rename / merge',exact:true});
|
||||
// The last row's actions can be scrolled to inside the window and fit its width.
|
||||
const last=rows(page).last().getByRole('button',{name:'Rename / merge',exact:true});
|
||||
await last.scrollIntoViewIfNeeded();await expect(last).toBeInViewport();
|
||||
const [button]=await boxes(last);const [after]=await boxes(tagWindow(page));
|
||||
expect(button.right).toBeLessThanOrEqual(after.right);
|
||||
|
||||
@@ -233,7 +233,7 @@ test('a view stored under older rules still lists and opens with its usable valu
|
||||
// D153 (#447): the workspace no longer offers "Highest rated"; the API still stores such a view as it was saved.
|
||||
test('a view saved under Highest rated opens on Recently added on Images and the Library', async ({ page, request }) => {
|
||||
const images = await create(request, { page: 'images', name: 'Rated cats', state: { ...IMAGE_DEFAULTS, q: 'cat', sort: 'rating' } });
|
||||
await create(request, { page: 'library', name: 'Rated dogs', state: { q: 'dog', category: '', sort: 'rating', tags: [], tag_mode: 'all', folder: '' } });
|
||||
const library = await create(request, { page: 'library', name: 'Rated dogs', state: { q: 'dog', category: '', sort: 'rating', tags: [], tag_mode: 'all', folder: '' } });
|
||||
expect((await listed(request)).map(view => view.state.sort)).toEqual(['rating', 'rating']);
|
||||
const listings = [];
|
||||
page.on('request', item => {
|
||||
|
||||
@@ -60,7 +60,7 @@ async function openReview(page, request, items = 30) {
|
||||
const inViewport = (rect, page) => rect.top >= 0 && rect.bottom <= page.viewportSize().height;
|
||||
|
||||
test.beforeEach(async ({ page, browserName }) => { await startCoverage(page, browserName); });
|
||||
test.afterEach(async ({ page, browserName }) => { await saveCoverage(page, browserName); });
|
||||
test.afterEach(async ({ page, browserName }, testInfo) => { await saveCoverage(page, browserName, testInfo); });
|
||||
|
||||
for (const viewport of [null, PHONE]) {
|
||||
const label = viewport ? ' at 390px' : '';
|
||||
|
||||
@@ -11,6 +11,7 @@ const REPORT_URL = '/prompt_manager/system/diagnostics';
|
||||
const LOGS_URL = '/prompt_manager/logs';
|
||||
const CONFIG_URL = '/prompt_manager/logs/config';
|
||||
const PREVIEW_URL = '/prompt_manager/maintenance/preview';
|
||||
const APPLY_URL = '/prompt_manager/maintenance/apply';
|
||||
const TEST_LINK_URL = '/prompt_manager/diagnostics/test-link';
|
||||
const CHANGED = 'The log files changed since the review; nothing was deleted. Review again.';
|
||||
const KEY = 'sk-s12-browser-SECRET-0123456789';
|
||||
|
||||
@@ -21,6 +21,14 @@ function jsonResponse(status, body) {
|
||||
json: async () => body };
|
||||
}
|
||||
|
||||
function sseResponse(frames) {
|
||||
const bytes = new TextEncoder().encode(frames.map(frame => `data: ${JSON.stringify(frame)}\n\n`).join(''));
|
||||
let sent = false;
|
||||
return { ok: true, status: 200, body: { getReader: () => ({
|
||||
read: async () => (sent ? { done: true } : (sent = true, { done: false, value: bytes })),
|
||||
}) } };
|
||||
}
|
||||
|
||||
function load(file, className, fetchUrl, extra = {}, dom = elements()) {
|
||||
const timers = [];
|
||||
const notifications = [];
|
||||
@@ -55,6 +63,8 @@ function load(file, className, fetchUrl, extra = {}, dom = elements()) {
|
||||
return { page, dom, timers, notifications, storage };
|
||||
}
|
||||
|
||||
const BUSY = 'Scan for metadata already running';
|
||||
|
||||
test('admin settings save shows why config.json refused it', async () => {
|
||||
const message = 'config.json cannot be read, so nothing was saved. Fix or remove the file, then save again.';
|
||||
const { page, notifications } = load('admin.js', 'PromptAdmin',
|
||||
|
||||
@@ -2584,7 +2584,7 @@ class TestActivityJobs(AdminAPITestCase):
|
||||
async def test_cancel_after_first_batch_keeps_partial_counts(self):
|
||||
"""Review M7: the real scan stops at the batch boundary, keeping batch 1."""
|
||||
import threading
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from py.api import admin
|
||||
|
||||
|
||||
@@ -600,33 +600,6 @@ class TestAutotagSingleEdgeCases(AutotagAPITestCase):
|
||||
{"path": str(image), "general_threshold": "high"},
|
||||
)
|
||||
self.assertEqual(resp.status, 400)
|
||||
self.assertEqual(
|
||||
(await resp.json())["error"],
|
||||
"general_threshold and character_threshold must be numeric",
|
||||
)
|
||||
|
||||
async def test_invalid_single_options_report_the_validation_error(self) -> None:
|
||||
image = self._make_image(self.output_dir)
|
||||
cases = (
|
||||
(
|
||||
{"model_type": "wd14-vit", "general_threshold": 2},
|
||||
"general_threshold must be between 0 and 1",
|
||||
),
|
||||
(
|
||||
{"model_type": "gguf", "prompt": "x" * 4001},
|
||||
"instructions must be at most 4000 characters",
|
||||
),
|
||||
)
|
||||
for options, message in cases:
|
||||
with self.subTest(options=options["model_type"]):
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/autotag/single",
|
||||
{"path": str(image), **options},
|
||||
)
|
||||
self.assertEqual(resp.status, 400)
|
||||
self.assertEqual(
|
||||
await resp.json(), {"success": False, "error": message}
|
||||
)
|
||||
|
||||
async def test_prompt_lookup_failure_still_returns_tags(self):
|
||||
image = self._make_image(self.output_dir)
|
||||
@@ -1178,9 +1151,7 @@ class TestModelStatus(AutotagAPITestCase):
|
||||
)
|
||||
# Kept fields legacy pages and the workspace read.
|
||||
for model in models.values():
|
||||
self.assertLessEqual(
|
||||
{"name", "description", "size", "downloaded"}, set(model)
|
||||
)
|
||||
self.assertTrue({"name", "description", "size", "downloaded"} <= set(model))
|
||||
self.assertTrue(data["model_loaded"])
|
||||
self.assertEqual(data["loaded_model_type"], "wd14-vit")
|
||||
|
||||
|
||||
@@ -351,6 +351,8 @@ class TestLongJobLock(unittest.TestCase):
|
||||
self.api = PromptManagerAPI(db=database)
|
||||
|
||||
def _run(self, coro):
|
||||
import asyncio
|
||||
|
||||
return asyncio.run(coro)
|
||||
|
||||
def test_second_acquire_is_refused_until_first_releases(self):
|
||||
@@ -823,6 +825,8 @@ class TestInstanceHelpers(FolderPathsFixture):
|
||||
self.addCleanup(setattr, GalleryConfig, "MONITORING_DIRECTORIES", orig_dirs)
|
||||
|
||||
def _run(self, coro):
|
||||
import asyncio
|
||||
|
||||
return asyncio.run(coro)
|
||||
|
||||
def test_run_in_executor_with_kwargs(self):
|
||||
|
||||
@@ -16,7 +16,7 @@ import re
|
||||
import sqlite3
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
_mock_server = MagicMock()
|
||||
|
||||
@@ -381,7 +381,7 @@ class TestAnotherMount(SafetyTestCase):
|
||||
self.assertTrue(other.is_file() and keeper.is_file())
|
||||
|
||||
async def test_a_volume_without_file_ids_is_named_truthfully(self):
|
||||
self.mount("x.png", "x.png")
|
||||
keeper, other, patches = self.mount("x.png", "x.png")
|
||||
|
||||
def zero(path, *args, **kwargs):
|
||||
return _fake_stat(self.real_stat(path), st_dev=0, st_ino=0)
|
||||
|
||||
@@ -3713,7 +3713,7 @@ class TestSharedListing(ImageAPITestCase):
|
||||
|
||||
async def test_a_root_read_again_after_the_roots_change_listens_again(self):
|
||||
make_png(self.output_dir / "a.png")
|
||||
await self._listed()
|
||||
record = await self._listed()
|
||||
self.api.reset_gallery_roots_cache()
|
||||
record = await self._listed() # still configured: read and walked
|
||||
self._interval(0)
|
||||
|
||||
@@ -840,23 +840,6 @@ class TestLoraScanRoute(LoraAPITestCase):
|
||||
("error", error["message"], 0),
|
||||
)
|
||||
|
||||
async def test_unexpected_scan_error_is_generic_in_the_stream(self):
|
||||
self._enable()
|
||||
secret = str(self.tmp_root / "x.db")
|
||||
err = RuntimeError(f"could not open {secret}")
|
||||
with patch.object(self.api.db, "replace_category_prompts", side_effect=err):
|
||||
with self.assertLogs("prompt_manager.api", level="ERROR") as logs:
|
||||
resp = await self.client.request("POST", "/prompt_manager/lora/scan")
|
||||
body = await resp.text() # the stream ends inside the patch
|
||||
self.assertNotIn("x.db", body)
|
||||
error = self._sse_events(body)[-1]
|
||||
self.assertEqual(
|
||||
(error["type"], error["status"], error["progress"]),
|
||||
("error", error["message"], 0),
|
||||
)
|
||||
self.assertIn("server log", error["message"])
|
||||
self.assertIn(secret, "\n".join(logs.output))
|
||||
|
||||
@staticmethod
|
||||
def _sse_events(text):
|
||||
return [
|
||||
|
||||
@@ -675,7 +675,7 @@ class TestBackupPoolAndDiskSpace(ApplyTestCase):
|
||||
self.assertEqual(len(backups), MAX_SAFETY_BACKUPS)
|
||||
self.assertNotIn(restore_style[0], backups)
|
||||
self.assertNotIn(restore_style[1], backups)
|
||||
self.assertLessEqual(set(maintenance_style), set(backups))
|
||||
self.assertTrue(set(maintenance_style) <= set(backups))
|
||||
# A restore prunes the same pool, maintenance backups included.
|
||||
self.old_backup("prompts.db.backup_20020101_000000", 3000)
|
||||
_prune_safety_backups(self.api.db.model.db_path)
|
||||
|
||||
@@ -11,7 +11,7 @@ import time
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
try:
|
||||
from tests.open_handles import assert_closed
|
||||
|
||||
@@ -6,7 +6,7 @@ Large frees then go back to the OS instead of fragmenting per-thread arenas.
|
||||
import ctypes
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from tests.browser import malloc_tuning
|
||||
|
||||
|
||||
@@ -46,6 +46,7 @@ class TestResetWatch(unittest.TestCase):
|
||||
self.assertFalse(is_reset_path("/__test__/reset/extra"))
|
||||
|
||||
def test_a_stalled_reset_reports_state_and_stacks_while_it_waits(self):
|
||||
stream = io.StringIO()
|
||||
reported = threading.Event()
|
||||
|
||||
class Stream(io.StringIO):
|
||||
|
||||
@@ -385,8 +385,8 @@ class TestArchivedReaders(LibraryDeleteCase):
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.kept, _, _ = self.paired("Kept cat", "kept blur")
|
||||
self.gone, _, _ = self.paired("Gone cat", "gone blur")
|
||||
self.kept, _kn, _kp = self.paired("Kept cat", "kept blur")
|
||||
self.gone, _gn, _gp = self.paired("Gone cat", "gone blur")
|
||||
for pid, category, tag in ((self.kept, "keep", "k"), (self.gone, "gone", "g")):
|
||||
self.db.update_prompt_metadata(pid, category=category, tags=[tag])
|
||||
self.apply([self.gone])
|
||||
@@ -642,7 +642,7 @@ class TestPairedDelete(LibraryDeleteCase):
|
||||
# The negatives stay, and the image file stays on disk.
|
||||
self.assertTrue((self.root / "owl.png").exists())
|
||||
texts = [row[0] for row in rows(self.db, "SELECT text FROM prompts")]
|
||||
self.assertLessEqual({"dark", "fog", "Bystander"}, set(texts))
|
||||
self.assertTrue({"dark", "fog", "Bystander"} <= set(texts))
|
||||
|
||||
def test_a_negative_used_by_another_positive_is_archived_never_deleted(self):
|
||||
shared = save(self.db, "Shared")
|
||||
|
||||
@@ -14,7 +14,7 @@ import threading
|
||||
import types
|
||||
import unittest
|
||||
from collections import namedtuple
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -140,10 +140,8 @@ class TestIdempotentMigrations(MigrationTestCase):
|
||||
def test_stale_prompts_new_from_a_failed_rebuild_does_not_block_startup(self):
|
||||
self._legacy_db(
|
||||
[
|
||||
(
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')"
|
||||
),
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')",
|
||||
"CREATE TABLE prompts_new (id INTEGER PRIMARY KEY, junk TEXT)",
|
||||
"INSERT INTO prompts_new (junk) VALUES ('leftover')",
|
||||
]
|
||||
@@ -159,14 +157,10 @@ class TestIdempotentMigrations(MigrationTestCase):
|
||||
# DROP TABLE prompts with foreign keys ON cascades into generated_images
|
||||
self._legacy_db(
|
||||
[
|
||||
(
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')"
|
||||
),
|
||||
(
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/a.png', 'a.png')"
|
||||
),
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')",
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/a.png', 'a.png')",
|
||||
]
|
||||
)
|
||||
|
||||
@@ -178,15 +172,11 @@ class TestIdempotentMigrations(MigrationTestCase):
|
||||
def test_rebuild_keeps_library_state_rows(self):
|
||||
self._legacy_db(
|
||||
[
|
||||
(
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')"
|
||||
),
|
||||
(
|
||||
"CREATE TABLE prompt_library_state (prompt_id INTEGER PRIMARY KEY,"
|
||||
" title TEXT, FOREIGN KEY(prompt_id) REFERENCES prompts(id)"
|
||||
" ON DELETE CASCADE)"
|
||||
),
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')",
|
||||
"CREATE TABLE prompt_library_state (prompt_id INTEGER PRIMARY KEY,"
|
||||
" title TEXT, FOREIGN KEY(prompt_id) REFERENCES prompts(id)"
|
||||
" ON DELETE CASCADE)",
|
||||
"INSERT INTO prompt_library_state VALUES (1, 'kept')",
|
||||
]
|
||||
)
|
||||
@@ -242,59 +232,43 @@ class TestIdempotentMigrations(MigrationTestCase):
|
||||
byte_id = "6a1b2c3d-4e5f-4a6b-8c9d-0e1f2a3b4c5d"
|
||||
self._legacy_db(
|
||||
[
|
||||
(
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')"
|
||||
),
|
||||
(
|
||||
"CREATE TABLE tags (id INTEGER PRIMARY KEY AUTOINCREMENT,"
|
||||
" name TEXT NOT NULL UNIQUE)"
|
||||
),
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')",
|
||||
"CREATE TABLE tags (id INTEGER PRIMARY KEY AUTOINCREMENT,"
|
||||
" name TEXT NOT NULL UNIQUE)",
|
||||
"INSERT INTO tags (id, name) VALUES (3, 'img-only')",
|
||||
(
|
||||
"CREATE TABLE media_annotation_metadata (singleton_id INTEGER"
|
||||
" PRIMARY KEY CHECK(singleton_id=1), schema_version INTEGER NOT"
|
||||
" NULL CHECK(schema_version=1), image_index_revision INTEGER NOT"
|
||||
" NULL DEFAULT 0 CHECK(image_index_revision>=0))"
|
||||
),
|
||||
"CREATE TABLE media_annotation_metadata (singleton_id INTEGER"
|
||||
" PRIMARY KEY CHECK(singleton_id=1), schema_version INTEGER NOT"
|
||||
" NULL CHECK(schema_version=1), image_index_revision INTEGER NOT"
|
||||
" NULL DEFAULT 0 CHECK(image_index_revision>=0))",
|
||||
"INSERT INTO media_annotation_metadata VALUES (1, 1, 4)",
|
||||
(
|
||||
"CREATE TABLE media_annotation_files (id TEXT PRIMARY KEY"
|
||||
" CHECK(length(id)=36), locator TEXT NOT NULL UNIQUE,"
|
||||
" created_at TEXT NOT NULL)"
|
||||
),
|
||||
"CREATE TABLE media_annotation_files (id TEXT PRIMARY KEY"
|
||||
" CHECK(length(id)=36), locator TEXT NOT NULL UNIQUE,"
|
||||
" created_at TEXT NOT NULL)",
|
||||
f"INSERT INTO media_annotation_files VALUES ('{file_id}',"
|
||||
" '/out/a.png', '2026-10-22T00:00:00.000000+00:00')",
|
||||
(
|
||||
"CREATE TABLE media_annotation_bytes (id TEXT PRIMARY KEY"
|
||||
" CHECK(length(id)=36), file_id TEXT NOT NULL REFERENCES"
|
||||
" media_annotation_files(id) ON DELETE CASCADE, sha256 TEXT NOT"
|
||||
" NULL CHECK(length(sha256)=64 AND sha256 NOT GLOB"
|
||||
" '*[^0-9a-f]*'), size INTEGER NOT NULL CHECK(size>=0), mtime_ns"
|
||||
" INTEGER NOT NULL, verified_at TEXT NOT NULL, first_seen_at TEXT"
|
||||
" NOT NULL, tag_revision INTEGER NOT NULL DEFAULT 0"
|
||||
" CHECK(tag_revision>=0), UNIQUE(file_id, sha256))"
|
||||
),
|
||||
(
|
||||
"CREATE INDEX idx_media_annotation_bytes_fingerprint ON"
|
||||
" media_annotation_bytes(file_id, size, mtime_ns)"
|
||||
),
|
||||
"CREATE TABLE media_annotation_bytes (id TEXT PRIMARY KEY"
|
||||
" CHECK(length(id)=36), file_id TEXT NOT NULL REFERENCES"
|
||||
" media_annotation_files(id) ON DELETE CASCADE, sha256 TEXT NOT"
|
||||
" NULL CHECK(length(sha256)=64 AND sha256 NOT GLOB"
|
||||
" '*[^0-9a-f]*'), size INTEGER NOT NULL CHECK(size>=0), mtime_ns"
|
||||
" INTEGER NOT NULL, verified_at TEXT NOT NULL, first_seen_at TEXT"
|
||||
" NOT NULL, tag_revision INTEGER NOT NULL DEFAULT 0"
|
||||
" CHECK(tag_revision>=0), UNIQUE(file_id, sha256))",
|
||||
"CREATE INDEX idx_media_annotation_bytes_fingerprint ON"
|
||||
" media_annotation_bytes(file_id, size, mtime_ns)",
|
||||
f"INSERT INTO media_annotation_bytes VALUES ('{byte_id}',"
|
||||
f" '{file_id}', '{'a' * 64}', 3, 1700000000000000000,"
|
||||
" '2026-10-22T00:00:00.000000+00:00',"
|
||||
" '2026-10-22T00:00:00.000000+00:00', 2)",
|
||||
(
|
||||
"CREATE TABLE media_image_tags (byte_id TEXT NOT NULL REFERENCES"
|
||||
" media_annotation_bytes(id) ON DELETE CASCADE, tag_id INTEGER"
|
||||
" NOT NULL REFERENCES tags(id) ON DELETE CASCADE, position"
|
||||
" INTEGER NOT NULL CHECK(position>=0), PRIMARY KEY(byte_id,"
|
||||
" tag_id))"
|
||||
),
|
||||
"CREATE TABLE media_image_tags (byte_id TEXT NOT NULL REFERENCES"
|
||||
" media_annotation_bytes(id) ON DELETE CASCADE, tag_id INTEGER"
|
||||
" NOT NULL REFERENCES tags(id) ON DELETE CASCADE, position"
|
||||
" INTEGER NOT NULL CHECK(position>=0), PRIMARY KEY(byte_id,"
|
||||
" tag_id))",
|
||||
"CREATE INDEX idx_media_image_tags_tag ON media_image_tags(tag_id)",
|
||||
(
|
||||
"CREATE UNIQUE INDEX idx_media_image_tags_position ON"
|
||||
" media_image_tags(byte_id, position)"
|
||||
),
|
||||
"CREATE UNIQUE INDEX idx_media_image_tags_position ON"
|
||||
" media_image_tags(byte_id, position)",
|
||||
f"INSERT INTO media_image_tags VALUES ('{byte_id}', 3, 0)",
|
||||
]
|
||||
)
|
||||
@@ -393,28 +367,18 @@ class TestIdempotentMigrations(MigrationTestCase):
|
||||
def test_image_table_is_rebuilt_with_uniqueness_by_full_path(self):
|
||||
self._legacy_db(
|
||||
[
|
||||
(
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')"
|
||||
),
|
||||
"INSERT INTO prompts (text, hash, workflow_name)"
|
||||
" VALUES ('a', 'h1', 'wf')",
|
||||
# same basename in two folders: both must survive
|
||||
(
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/2026-01-01/a.png', 'a.png')"
|
||||
),
|
||||
(
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/2026-01-02/a.png', 'a.png')"
|
||||
),
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/2026-01-01/a.png', 'a.png')",
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/2026-01-02/a.png', 'a.png')",
|
||||
# two spellings of one file: only the newest row survives
|
||||
(
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/x/../b.png', 'b.png')"
|
||||
),
|
||||
(
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/b.png', 'b.png')"
|
||||
),
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/x/../b.png', 'b.png')",
|
||||
"INSERT INTO generated_images (prompt_id, image_path, filename)"
|
||||
" VALUES (1, '/out/b.png', 'b.png')",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@@ -182,7 +182,7 @@ class TestAdminEndpointsUseConfigPath(unittest.TestCase):
|
||||
self.assertIsNone(warning)
|
||||
self.assertEqual(os.path.dirname(snapshot), folder)
|
||||
with open(snapshot, "rb") as handle:
|
||||
self.assertEqual(handle.read(16), b"SQLite format 3\x00")
|
||||
self.assertTrue(handle.read(16) == b"SQLite format 3\x00")
|
||||
finally:
|
||||
os.unlink(snapshot)
|
||||
finally:
|
||||
|
||||
@@ -31,20 +31,6 @@ def message(kind, job_id=JOB, **fields):
|
||||
return (kind, {"prompt_id": job_id, **fields})
|
||||
|
||||
|
||||
def hash_recording_kind():
|
||||
"""A str subclass for event kinds, and the list of hashes taken of it."""
|
||||
hashed = []
|
||||
|
||||
class OversizedKind(str):
|
||||
__eq__ = str.__eq__
|
||||
|
||||
def __hash__(self):
|
||||
hashed.append(len(self))
|
||||
return str.__hash__(self)
|
||||
|
||||
return OversizedKind, hashed
|
||||
|
||||
|
||||
def finish(messages, completed=True):
|
||||
return make_finish(SESSION, 0, EPOCH, Status("success", completed, messages), 500)
|
||||
|
||||
@@ -360,10 +346,13 @@ class ExecutionHistoryEventsTests(unittest.TestCase):
|
||||
self.assertLessEqual(len(encode_command(command)), 2048)
|
||||
|
||||
def test_oversized_unknown_kind_is_ignored_without_hashing(self):
|
||||
kind_class, hashed = hash_recording_kind()
|
||||
class OversizedKind(str):
|
||||
def __hash__(self):
|
||||
raise AssertionError("Oversized event kind must not be hashed")
|
||||
|
||||
command = finish(
|
||||
[
|
||||
message(kind_class("x" * 1_000_000)),
|
||||
message(OversizedKind("x" * 1_000_000)),
|
||||
message("execution_success", timestamp=300),
|
||||
]
|
||||
)
|
||||
@@ -372,16 +361,19 @@ class ExecutionHistoryEventsTests(unittest.TestCase):
|
||||
(result.outcome, result.reason), ("succeeded", "terminal_success")
|
||||
)
|
||||
self.assertLessEqual(len(encode_command(command)), 2048)
|
||||
self.assertEqual(hashed, [], "Oversized event kind must not be hashed")
|
||||
|
||||
def test_encoder_rejects_oversized_kind_without_hashing(self):
|
||||
kind_class, hashed = hash_recording_kind()
|
||||
class OversizedKind(str):
|
||||
def __hash__(self):
|
||||
raise AssertionError("Oversized event kind must not be hashed")
|
||||
|
||||
command = finish([message("execution_success")])
|
||||
event = LifecycleEvent(kind_class("x" * 1_000_000), JOB, None, None, None, None)
|
||||
event = LifecycleEvent(
|
||||
OversizedKind("x" * 1_000_000), JOB, None, None, None, None
|
||||
)
|
||||
with self.assertRaises(ValueError):
|
||||
encode_command(
|
||||
replace(
|
||||
command, evidence=TerminalEvidence((event,), False, False, False)
|
||||
)
|
||||
)
|
||||
self.assertEqual(hashed, [], "Oversized event kind must not be hashed")
|
||||
|
||||
@@ -17,7 +17,7 @@ class Hostile:
|
||||
raise AssertionError("host scalar coercion")
|
||||
|
||||
def __bool__(self):
|
||||
raise TypeError("host truth test")
|
||||
raise AssertionError("host truth test")
|
||||
|
||||
def __eq__(self, other):
|
||||
raise AssertionError("host equality")
|
||||
|
||||
@@ -24,7 +24,7 @@ import tempfile
|
||||
import textwrap
|
||||
import threading
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
|
||||
@@ -26,7 +26,7 @@ import types
|
||||
import unittest
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from py.api import thumbnail_auto
|
||||
from utils import image_monitor, parallel
|
||||
|
||||
@@ -7,7 +7,7 @@ I/O: history comes in through ``lookup``.
|
||||
|
||||
import json
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from database.run_attribution import RecordedOutput
|
||||
from tests.test_pair_scan import flux_graph, sampled_graph
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
import logging
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from prompt_manager import PromptManager, PromptManagerFocused
|
||||
from utils.prompt_graph import positive_prompt_nodes, run_prompt_nodes
|
||||
|
||||
@@ -312,7 +312,7 @@ class RootContainsTests(unittest.TestCase):
|
||||
root = os.path.normcase(os.path.realpath(tempfile.gettempdir()))
|
||||
inside = os.path.join(root, "x.png")
|
||||
self.assertTrue(root_contains(root, inside))
|
||||
self.assertEqual(root_contains(root, inside.upper()), os.name == "nt")
|
||||
self.assertTrue(root_contains(root, inside.upper()) == (os.name == "nt"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -392,10 +392,7 @@ class TestStartStop(unittest.TestCase):
|
||||
self.assertLogs("prompt_manager.image_monitor", "WARNING") as logs,
|
||||
):
|
||||
self.assertFalse(self.monitor.stop_monitoring(timeout=5.0))
|
||||
# The stop runs on its own thread and gets 0 s too: it may not have
|
||||
# returned yet, and then nothing is joined at all.
|
||||
joins = [call for call in fake.calls if call[0] == "join"]
|
||||
self.assertTrue(all(call == ("join", 0.0) for call in joins), fake.calls)
|
||||
self.assertEqual(fake.calls[-1], ("join", 0.0))
|
||||
self.assertIn(
|
||||
"watchdog observer still running after 0.0 s", "\n".join(logs.output)
|
||||
)
|
||||
@@ -404,80 +401,6 @@ class TestStartStop(unittest.TestCase):
|
||||
def test_stop_monitoring_without_an_observer_returns_true(self):
|
||||
self.assertTrue(self.monitor.stop_monitoring())
|
||||
|
||||
def test_threads_the_observer_starts_are_daemons(self):
|
||||
# watchdog 7 on Windows starts a plain threading.Thread per watch in
|
||||
# observer.start(); a non-daemon one holds the interpreter's exit
|
||||
# before any atexit callback runs (#561).
|
||||
fake = ReaderObserver()
|
||||
self.addCleanup(fake.release.set)
|
||||
self._start_with(fake)
|
||||
self.assertIsNotNone(fake.reader)
|
||||
self.assertTrue(fake.reader.daemon)
|
||||
self.assertFalse(threading.current_thread().daemon)
|
||||
|
||||
def test_an_observer_that_fails_to_start_on_its_thread_is_reported(self):
|
||||
fake = FakeObserver()
|
||||
fake.start = MagicMock(side_effect=OSError("handle refused"))
|
||||
with self.assertLogs("prompt_manager.image_monitor", "ERROR") as logs:
|
||||
with patch.object(im_mod, "Observer", return_value=fake):
|
||||
self.monitor.start_monitoring([self.tmp.name])
|
||||
self.assertIn("handle refused", "\n".join(logs.output))
|
||||
self.assertFalse(self.monitor.is_monitoring)
|
||||
|
||||
def test_an_observer_stop_that_never_returns_is_bounded(self):
|
||||
# watchdog's observer.stop() joins its emitters (and, in 7.x on
|
||||
# Windows, their reader threads) without a limit (#561).
|
||||
fake = StuckStopObserver()
|
||||
self.addCleanup(fake.release.set)
|
||||
self._start_with(fake)
|
||||
with self.assertLogs("prompt_manager.image_monitor", "WARNING") as logs:
|
||||
self.assertFalse(self.monitor.stop_monitoring(timeout=0.05))
|
||||
self.assertIn("watchdog observer still running", "\n".join(logs.output))
|
||||
self.assertTrue(fake.entered.wait(30))
|
||||
self.assertFalse(self.monitor.is_monitoring)
|
||||
# A second stop waits for the stop already running; it never calls
|
||||
# observer.stop() a second time.
|
||||
fake.release.set()
|
||||
self.assertTrue(self.monitor.stop_monitoring())
|
||||
self.assertEqual(fake.stop_calls, 1)
|
||||
self.assertFalse(self.monitor._unfinished)
|
||||
|
||||
|
||||
class ReaderObserver(FakeObserver):
|
||||
"""A FakeObserver whose start() starts a plain thread, as watchdog 7's
|
||||
Windows emitter does for its ReadDirectoryChangesW reader."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.release = threading.Event()
|
||||
self.reader = None
|
||||
|
||||
def start(self):
|
||||
self.reader = threading.Thread(target=self.release.wait)
|
||||
self.reader.start()
|
||||
super().start()
|
||||
|
||||
def stop(self):
|
||||
self.release.set()
|
||||
self.reader.join(30)
|
||||
super().stop()
|
||||
|
||||
|
||||
class StuckStopObserver(FakeObserver):
|
||||
"""A FakeObserver whose stop() does not return until ``release``."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.entered = threading.Event()
|
||||
self.release = threading.Event()
|
||||
self.stop_calls = 0
|
||||
|
||||
def stop(self):
|
||||
self.stop_calls += 1
|
||||
self.entered.set()
|
||||
self.release.wait(30)
|
||||
super().stop()
|
||||
|
||||
|
||||
class FakeEvent:
|
||||
def __init__(self, path, is_directory=False):
|
||||
|
||||
@@ -527,7 +527,7 @@ class TestPackageLoggerIsolation(unittest.TestCase):
|
||||
def log_from_background_thread():
|
||||
try:
|
||||
logging.getLogger("prompt_manager.late_emitter").warning(message)
|
||||
except Exception as error:
|
||||
except BaseException as error:
|
||||
errors.append(error)
|
||||
|
||||
self.handler.addFilter(pause_before_handler_lock)
|
||||
@@ -587,16 +587,13 @@ PACKAGE_THREAD_NAMES = ("PromptManagerPromptCleanup", "PromptManagerImageWorker"
|
||||
|
||||
def package_threads():
|
||||
"""Live threads this design owns: the tracker, the image worker, and the
|
||||
watchdog observer and emitters (which have no name of their own).
|
||||
watchdog 7's InotifyFD, one daemon reader per process that no observer
|
||||
stop ends, is not the package's."""
|
||||
watchdog observer and emitters (which have no name of their own)."""
|
||||
from watchdog.utils import BaseThread
|
||||
|
||||
return {
|
||||
thread
|
||||
for thread in threading.enumerate()
|
||||
if thread.name in PACKAGE_THREAD_NAMES
|
||||
or (isinstance(thread, BaseThread) and type(thread).__name__ != "InotifyFD")
|
||||
if thread.name in PACKAGE_THREAD_NAMES or isinstance(thread, BaseThread)
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -143,6 +143,8 @@ class TestLibrary(unittest.TestCase):
|
||||
self.db.get_library_positive(positive, variant_limit=201)
|
||||
|
||||
def test_sorts_keysets_survive_anchor_deletion_and_insertions_without_writes(self):
|
||||
import sqlite3
|
||||
|
||||
a = self.save("Alpha", rating=2, tags=["tag"])
|
||||
b = self.save("beta", rating=5)
|
||||
c = self.save("charlie")
|
||||
|
||||
@@ -297,7 +297,7 @@ class TestLibraryDeleteRoutes(LibraryDeleteRouteCase):
|
||||
)
|
||||
page = await response.json()
|
||||
self.assertEqual(page["archived_total"], 3)
|
||||
self.assertLessEqual(set(item["id"] for item in page["items"]), set(kept))
|
||||
self.assertTrue(set(item["id"] for item in page["items"]) <= set(kept))
|
||||
response = await self.client.request(
|
||||
"GET",
|
||||
"/prompt_manager/library?limit=2&shelf=archived"
|
||||
|
||||
@@ -135,7 +135,6 @@ class TestLogsAndCache(_EnvCase):
|
||||
|
||||
get_logger_manager()._setup_loggers()
|
||||
except Exception:
|
||||
# best effort restore in teardown; must not hide the test result
|
||||
pass
|
||||
|
||||
def test_lora_image_cache_follows_the_data_folder(self):
|
||||
|
||||
@@ -18,7 +18,7 @@ import tempfile
|
||||
import threading
|
||||
import types
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ import sqlite3
|
||||
import sys
|
||||
import threading
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -1112,10 +1112,10 @@ class LocalProjectionParityTests(unittest.TestCase):
|
||||
self.assertEqual(_canonical_bytes(local), _canonical_bytes(server))
|
||||
seen.update(server["record_formats"].values())
|
||||
seen.add(server["workflow_graph"]["status"])
|
||||
self.assertLessEqual(
|
||||
{"reserialized-json", "original-text", "invalid", "limited"}, seen
|
||||
self.assertTrue(
|
||||
{"reserialized-json", "original-text", "invalid", "limited"} <= seen
|
||||
)
|
||||
self.assertLessEqual({"parsed", "malformed", "limited"}, seen)
|
||||
self.assertTrue({"parsed", "malformed", "limited"} <= seen)
|
||||
|
||||
|
||||
def _server_evidence(records, file_name=None):
|
||||
@@ -1287,7 +1287,7 @@ class LocalResolverParityTests(unittest.TestCase):
|
||||
def test_local_roles_match_selection_record_sets(self):
|
||||
answers = self._compare(selection_cases.selection_cases())
|
||||
reasons = {answer["resolution"]["reason"] for answer in answers.values()}
|
||||
self.assertLessEqual(
|
||||
self.assertTrue(
|
||||
{
|
||||
"Every sampler branch uses the same prompt pair",
|
||||
"Multiple sampler branches",
|
||||
@@ -1301,8 +1301,8 @@ class LocalResolverParityTests(unittest.TestCase):
|
||||
"Negative prompt delimiter is absent",
|
||||
"Parameters exceed the length limit",
|
||||
"No A1111 parameters",
|
||||
},
|
||||
reasons,
|
||||
}
|
||||
<= reasons
|
||||
)
|
||||
# A stage reaching only previews is dropped: one sampler is named.
|
||||
self.assertEqual(
|
||||
@@ -1352,7 +1352,7 @@ class LocalResolverParityTests(unittest.TestCase):
|
||||
def test_local_roles_match_golden_record_sets(self):
|
||||
answers = self._compare(resolver_cases.golden_cases())
|
||||
codes = {answer["resolution"]["reason_code"] for answer in answers.values()}
|
||||
self.assertLessEqual(
|
||||
self.assertTrue(
|
||||
{
|
||||
"malformed",
|
||||
"unsupported-node",
|
||||
@@ -1364,16 +1364,16 @@ class LocalResolverParityTests(unittest.TestCase):
|
||||
"graph-ceiling",
|
||||
"no-sampler",
|
||||
None,
|
||||
},
|
||||
codes,
|
||||
}
|
||||
<= codes
|
||||
)
|
||||
sources = {
|
||||
side["source"]
|
||||
for answer in answers.values()
|
||||
for side in answer["prompts"].values()
|
||||
}
|
||||
self.assertLessEqual(
|
||||
{"execution", "graph-inferred", "zeroed-conditioning"}, sources
|
||||
self.assertTrue(
|
||||
{"execution", "graph-inferred", "zeroed-conditioning"} <= sources
|
||||
)
|
||||
|
||||
def test_committed_resolver_vectors_are_current(self):
|
||||
|
||||
@@ -197,7 +197,7 @@ class TestGetPromptSubfolders(LoraDBTestCase):
|
||||
|
||||
def test_subfolders_use_forward_slashes_on_every_platform(self):
|
||||
"""Clients split folders on "/" whatever separator the OS produced."""
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
import database.operations as operations
|
||||
|
||||
|
||||
@@ -584,6 +584,9 @@ class CentralizedDiscoveryTests(LayoutTestCase):
|
||||
def test_an_unlistable_folder_raises(self):
|
||||
layout = self.load()
|
||||
|
||||
def refuse(error):
|
||||
raise error
|
||||
|
||||
def failing_walk(top, onerror=None, followlinks=False):
|
||||
onerror(PermissionError(13, "denied", top))
|
||||
return iter(())
|
||||
|
||||
@@ -15,7 +15,7 @@ import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
import weakref
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
@@ -154,116 +154,6 @@ class TestRetire(_Case):
|
||||
|
||||
|
||||
class TestCalls(_Case):
|
||||
def test_iteration_and_fetches_share_one_position(self) -> None:
|
||||
conn = self.connect()
|
||||
conn.executemany("INSERT INTO t VALUES (?)", [(x,) for x in range(1, 260)])
|
||||
cursor = conn.execute("SELECT x FROM t ORDER BY x")
|
||||
self.assertIs(iter(cursor), cursor)
|
||||
self.assertEqual(next(cursor), (0,))
|
||||
self.assertEqual(cursor.fetchone(), (1,))
|
||||
self.assertEqual(cursor.fetchmany(2), [(2,), (3,)])
|
||||
self.assertEqual(next(iter(cursor)), (4,))
|
||||
self.assertEqual(cursor.fetchall(), [(x,) for x in range(5, 260)])
|
||||
self.assertEqual(list(cursor), [])
|
||||
|
||||
def test_a_partial_loop_can_resume_without_skipping_rows(self) -> None:
|
||||
cursor = self.connect().execute(
|
||||
"SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3"
|
||||
)
|
||||
for row in cursor:
|
||||
self.assertEqual(row, (1,))
|
||||
break
|
||||
self.assertEqual(list(cursor), [(2,), (3,)])
|
||||
|
||||
def test_an_existing_iterator_refuses_reads_after_retire_or_close(self) -> None:
|
||||
for retire in (False, True):
|
||||
with self.subTest(retire=retire):
|
||||
conn = self.connect()
|
||||
cursor = conn.execute("SELECT 1 UNION ALL SELECT 2")
|
||||
iterator = iter(cursor)
|
||||
self.assertEqual(next(iterator), (1,))
|
||||
if retire:
|
||||
conn.retire()
|
||||
else:
|
||||
cursor.close()
|
||||
with self.assertRaises(sqlite3.ProgrammingError):
|
||||
next(iterator)
|
||||
|
||||
def test_buffered_fetchmany_matches_sqlite_sizes_and_validation(self) -> None:
|
||||
conn = self.connect()
|
||||
conn.executemany("INSERT INTO t VALUES (?)", [(x,) for x in range(1, 260)])
|
||||
conn.commit()
|
||||
with contextlib.closing(sqlite3.connect(self.path)) as plain:
|
||||
for size in (None, 0, 1, 2, 127, 128, 130, 300, -1):
|
||||
with self.subTest(size=size):
|
||||
cursors = [
|
||||
c.execute("SELECT x FROM t ORDER BY x") for c in (plain, conn)
|
||||
]
|
||||
for cursor in cursors:
|
||||
cursor.arraysize = 2
|
||||
self.assertEqual(next(iter(cursor)), (0,))
|
||||
results = []
|
||||
for cursor in cursors:
|
||||
try:
|
||||
rows = (
|
||||
cursor.fetchmany()
|
||||
if size is None
|
||||
else cursor.fetchmany(size)
|
||||
)
|
||||
except ValueError as error: # Negative sizes on Python 3.14+.
|
||||
rows = type(error)
|
||||
results.append((rows, cursor.fetchall()))
|
||||
self.assertEqual(*results)
|
||||
for args, kwargs in ((("bad",), {}), ((1, 2), {}), ((), {"unknown": 1})):
|
||||
cursor = conn.execute("SELECT 1 UNION ALL SELECT 2")
|
||||
self.assertEqual(next(iter(cursor)), (1,))
|
||||
with self.assertRaises(TypeError):
|
||||
cursor.fetchmany(*args, **kwargs)
|
||||
self.assertEqual(cursor.fetchone(), (2,))
|
||||
|
||||
def test_reexecuting_a_cursor_matches_sqlite_result_lifetime(self) -> None:
|
||||
conn = self.connect()
|
||||
conn.row_factory = sqlite3.Row
|
||||
for method, args in (
|
||||
("execute", ("SELECT 9 AS x",)),
|
||||
("executemany", ("INSERT INTO t VALUES (?)", [(9,)])),
|
||||
("executescript", ("SELECT 9 AS x;",)),
|
||||
):
|
||||
with self.subTest(method=method):
|
||||
cursor = conn.execute("SELECT 1 AS x UNION ALL SELECT 2")
|
||||
self.assertEqual(next(iter(cursor))["x"], 1)
|
||||
self.assertIs(getattr(cursor, method)(*args), cursor)
|
||||
if method == "execute":
|
||||
self.assertEqual(cursor.fetchone()["x"], 9)
|
||||
elif method == "executescript":
|
||||
self.assertEqual(cursor.fetchone()["x"], 2)
|
||||
self.assertEqual(list(cursor), [])
|
||||
|
||||
def test_failed_execution_matches_sqlite_result_lifetime(self) -> None:
|
||||
conn = self.connect()
|
||||
cases = (
|
||||
("execute", (None,)),
|
||||
("execute", ("invalid sql",)),
|
||||
("execute", ("SELECT ?", ())),
|
||||
("executemany", (None, [])),
|
||||
("executemany", ("SELECT 9", [])),
|
||||
("executescript", (None,)),
|
||||
("executescript", ("invalid sql;",)),
|
||||
)
|
||||
with contextlib.closing(sqlite3.connect(self.path)) as plain:
|
||||
for method, args in cases:
|
||||
with self.subTest(method=method, args=args):
|
||||
results = []
|
||||
for connection in (plain, conn):
|
||||
cursor = connection.execute("SELECT 1 UNION ALL SELECT 2")
|
||||
self.assertEqual(next(iter(cursor)), (1,))
|
||||
with self.assertRaises(
|
||||
sqlite3.Error if isinstance(args[0], str) else TypeError
|
||||
):
|
||||
getattr(cursor, method)(*args)
|
||||
results.append(cursor.fetchall())
|
||||
self.assertEqual(*results)
|
||||
|
||||
def test_every_cursor_call_works_and_is_refused_once_retired(self):
|
||||
conn = self.connect()
|
||||
conn.row_factory = sqlite3.Row
|
||||
|
||||
@@ -347,7 +347,7 @@ class CaptureStoreTestCase(unittest.TestCase):
|
||||
with patch.object(CaptureStore, "_read", rewriting):
|
||||
try:
|
||||
cap = store.capture(self.root, "a.png", epoch=EPOCH, scope=SCOPE)
|
||||
except capture_module.CaptureChanged:
|
||||
except CaptureChanged:
|
||||
detected = True
|
||||
else:
|
||||
detected = False
|
||||
|
||||
+44
-107
@@ -31,65 +31,22 @@ CHILD = textwrap.dedent("""
|
||||
import threading
|
||||
|
||||
root, tmp = sys.argv[1], sys.argv[2]
|
||||
stuck = sys.argv[3:] == ["stuck-observer"]
|
||||
sys.path.insert(0, root)
|
||||
state = {}
|
||||
|
||||
# watchdog 7's Windows observer at its worst (#561): start() starts a
|
||||
# plain reader thread, and stop() waits for it, which never ends.
|
||||
class StuckObserver:
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
self._never = threading.Event()
|
||||
self._reader = None
|
||||
|
||||
def schedule(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def start(self):
|
||||
self._reader = threading.Thread(
|
||||
target=self._never.wait, name="StuckReader"
|
||||
)
|
||||
self._reader.start()
|
||||
|
||||
def stop(self):
|
||||
self._reader.join()
|
||||
|
||||
def join(self, timeout=None):
|
||||
self._reader.join(timeout)
|
||||
|
||||
def is_alive(self):
|
||||
return self._reader.is_alive()
|
||||
|
||||
if stuck:
|
||||
import watchdog.observers
|
||||
|
||||
watchdog.observers.Observer = StuckObserver
|
||||
|
||||
def report():
|
||||
from watchdog.utils import BaseThread
|
||||
|
||||
package = state.get("package")
|
||||
result = getattr(package, "_background_work_result", None)
|
||||
# The image worker, the tracker, and watchdog's observer and emitters.
|
||||
# watchdog 7's InotifyFD is one daemon reader per process that no
|
||||
# observer stop ends, by design; it is not the package's.
|
||||
alive = sorted(
|
||||
t.name
|
||||
for t in threading.enumerate()
|
||||
if t.name in ("PromptManagerImageWorker", "PromptManagerPromptCleanup")
|
||||
or (isinstance(t, BaseThread) and type(t).__name__ != "InotifyFD")
|
||||
)
|
||||
# Threads that would hold the exit before any atexit callback (#561).
|
||||
holding = sorted(
|
||||
t.name
|
||||
for t in threading.enumerate()
|
||||
if not t.daemon and t is not threading.main_thread()
|
||||
)
|
||||
print(
|
||||
"PM_EXIT_REPORT "
|
||||
+ json.dumps({"result": result, "alive": alive, "holding": holding})
|
||||
or isinstance(t, BaseThread)
|
||||
)
|
||||
print("PM_EXIT_REPORT " + json.dumps({"result": result, "alive": alive}))
|
||||
sys.stdout.flush()
|
||||
|
||||
# First registered, so it runs last: after the package's teardown and
|
||||
@@ -107,10 +64,6 @@ CHILD = textwrap.dedent("""
|
||||
fake_server_module(queue_task_done=True),
|
||||
)
|
||||
state["package"] = package
|
||||
if stuck:
|
||||
# The stop gives up after this long instead of the full 5 s.
|
||||
monitor_module = sys.modules[package.__name__ + ".utils.image_monitor"]
|
||||
monitor_module.OBSERVER_JOIN_SECONDS = 0.5
|
||||
monitor = package._global_image_monitor
|
||||
handler = monitor.handler if monitor is not None else None
|
||||
if handler is None:
|
||||
@@ -125,52 +78,47 @@ CHILD = textwrap.dedent("""
|
||||
""")
|
||||
|
||||
|
||||
def run_child(testcase, *args):
|
||||
"""Run CHILD with ``args``; returns (completed, report, detail)."""
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
os.makedirs(os.path.join(tmp, "output"))
|
||||
script = os.path.join(tmp, "child.py")
|
||||
with open(script, "w", encoding="utf-8") as handle:
|
||||
handle.write(CHILD)
|
||||
env = dict(os.environ)
|
||||
env.pop("PROMPT_MANAGER_CONFIG_PATH", None)
|
||||
# The child's library and log file live in its own folder, never
|
||||
# in the checkout's logs/ (which this runner may hold open) or a
|
||||
# data folder the caller set.
|
||||
env["PROMPT_MANAGER_DATA_DIR"] = tmp
|
||||
env["PYTHONIOENCODING"] = "utf-8"
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, script, ROOT, tmp, *args],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
timeout=CHILD_CEILING_SECONDS,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
except subprocess.TimeoutExpired as hung:
|
||||
testcase.fail(
|
||||
f"the child did not exit within {CHILD_CEILING_SECONDS} s\n"
|
||||
f"stdout:\n{hung.stdout}\nstderr:\n{hung.stderr}"
|
||||
)
|
||||
# Leaving the block removes the folder: on Windows that fails
|
||||
# if the child left a handle open, which it cannot once it exited,
|
||||
# so the check here is that the child exited at all.
|
||||
detail = f"stdout:\n{completed.stdout}\nstderr:\n{completed.stderr}"
|
||||
testcase.assertEqual(completed.returncode, 0, detail)
|
||||
reports = [
|
||||
line[len(REPORT_PREFIX) :]
|
||||
for line in completed.stdout.splitlines()
|
||||
if line.startswith(REPORT_PREFIX)
|
||||
]
|
||||
testcase.assertEqual(len(reports), 1, detail)
|
||||
return completed, json.loads(reports[0]), detail
|
||||
|
||||
|
||||
class TestPackageExit(unittest.TestCase):
|
||||
def test_teardown_runs_at_a_normal_exit_with_the_worker_alive(self):
|
||||
completed, report, detail = run_child(self)
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
os.makedirs(os.path.join(tmp, "output"))
|
||||
script = os.path.join(tmp, "child.py")
|
||||
with open(script, "w", encoding="utf-8") as handle:
|
||||
handle.write(CHILD)
|
||||
env = dict(os.environ)
|
||||
env.pop("PROMPT_MANAGER_CONFIG_PATH", None)
|
||||
# The child's library and log file live in its own folder, never
|
||||
# in the checkout's logs/ (which this runner may hold open) or a
|
||||
# data folder the caller set.
|
||||
env["PROMPT_MANAGER_DATA_DIR"] = tmp
|
||||
env["PYTHONIOENCODING"] = "utf-8"
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
[sys.executable, script, ROOT, tmp],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
timeout=CHILD_CEILING_SECONDS,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
except subprocess.TimeoutExpired as hung:
|
||||
self.fail(
|
||||
f"the child did not exit within {CHILD_CEILING_SECONDS} s\n"
|
||||
f"stdout:\n{hung.stdout}\nstderr:\n{hung.stderr}"
|
||||
)
|
||||
# Leaving the block removes the folder: on Windows that fails
|
||||
# if the child left a handle open, which it cannot once it exited,
|
||||
# so the check here is that the child exited at all.
|
||||
detail = f"stdout:\n{completed.stdout}\nstderr:\n{completed.stderr}"
|
||||
self.assertEqual(completed.returncode, 0, detail)
|
||||
reports = [
|
||||
line[len(REPORT_PREFIX) :]
|
||||
for line in completed.stdout.splitlines()
|
||||
if line.startswith(REPORT_PREFIX)
|
||||
]
|
||||
self.assertEqual(len(reports), 1, detail)
|
||||
report = json.loads(reports[0])
|
||||
# "thumbnails" is True here because no preview worker started; its
|
||||
# close result is tested in test_thumbnail_auto and test_init_package.
|
||||
self.assertEqual(
|
||||
@@ -179,7 +127,6 @@ class TestPackageExit(unittest.TestCase):
|
||||
detail,
|
||||
)
|
||||
self.assertEqual(report["alive"], [], detail)
|
||||
self.assertEqual(report["holding"], [], detail)
|
||||
# The teardown's own warnings. Import-time warnings (the API's
|
||||
# output-folder search finds no ComfyUI here) are not about exit.
|
||||
for text in (
|
||||
@@ -190,16 +137,6 @@ class TestPackageExit(unittest.TestCase):
|
||||
):
|
||||
self.assertNotIn(text, completed.stderr, detail)
|
||||
|
||||
def test_an_observer_that_never_stops_does_not_hold_the_exit(self):
|
||||
# Before #561 the reader thread kept the interpreter waiting before
|
||||
# any atexit callback ran, so the teardown never started.
|
||||
completed, report, detail = run_child(self, "stuck-observer")
|
||||
self.assertEqual(
|
||||
report["result"],
|
||||
{"monitor": False, "thumbnails": True, "tracker": True},
|
||||
detail,
|
||||
)
|
||||
# The stuck reader is still alive, but as a daemon it holds nothing.
|
||||
self.assertEqual(report["holding"], [], detail)
|
||||
self.assertIn("watchdog observer still running", completed.stderr, detail)
|
||||
self.assertIn("Background work not stopped in time", completed.stderr, detail)
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -14,7 +14,7 @@ import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ import os
|
||||
import sys
|
||||
import time
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ import tempfile
|
||||
import unittest
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from utils import inspection_semantics
|
||||
from utils import prompt_pair_resolver as resolver
|
||||
|
||||
@@ -286,7 +286,7 @@ class TestNodeRecordsCachedRunReceipt(UsageTestCase):
|
||||
}
|
||||
|
||||
def _node(self):
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from prompt_manager_text import PromptManagerText
|
||||
|
||||
|
||||
@@ -13,19 +13,19 @@ class FakeQueue:
|
||||
def __init__(self):
|
||||
self.result = ((0, "job", {}, {}, [], {}), 0)
|
||||
self.calls = []
|
||||
self.failures = ()
|
||||
self.failure = None
|
||||
self.done_result = object()
|
||||
|
||||
def get(self, *args, **kwargs):
|
||||
self.calls.append(("get", args, kwargs))
|
||||
for failure in self.failures:
|
||||
raise failure
|
||||
if self.failure:
|
||||
raise self.failure
|
||||
return self.result
|
||||
|
||||
def task_done(self, *args, **kwargs):
|
||||
self.calls.append(("done", args, kwargs))
|
||||
for failure in self.failures:
|
||||
raise failure
|
||||
if self.failure:
|
||||
raise self.failure
|
||||
return self.done_result
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ class QueueObserversTests(unittest.TestCase):
|
||||
self.assertIsNone(host.get())
|
||||
self.assertEqual(seen, [original])
|
||||
for error in (queue.Empty(), RuntimeError("host failure")):
|
||||
host.failures = (error,)
|
||||
host.failure = error
|
||||
with self.assertRaises(type(error)) as raised:
|
||||
host.get(timeout=sentinel)
|
||||
self.assertIs(raised.exception, error)
|
||||
@@ -59,7 +59,7 @@ class QueueObserversTests(unittest.TestCase):
|
||||
self.assertIs(host.task_done(0, sentinel, status=sentinel), host.done_result)
|
||||
self.assertEqual(seen, [((0, sentinel), {"status": sentinel})])
|
||||
error = ValueError("host failure")
|
||||
host.failures = (error,)
|
||||
host.failure = error
|
||||
with self.assertRaises(ValueError) as raised:
|
||||
host.task_done(1)
|
||||
self.assertIs(raised.exception, error)
|
||||
|
||||
@@ -357,7 +357,7 @@ class TestRestoreAdmission(RestoreTestCase):
|
||||
gate.set()
|
||||
await health.task
|
||||
|
||||
self.gated_job("restore", "exclusive", RESTORE_JOB_TITLE)
|
||||
restore, release = self.gated_job("restore", "exclusive", RESTORE_JOB_TITLE)
|
||||
resp = await self.client.post("/prompt_manager/system/health")
|
||||
refusal = await resp.json()
|
||||
self.assertEqual((resp.status, refusal["code"]), (409, "busy"))
|
||||
@@ -369,7 +369,7 @@ class TestRestoreAdmission(RestoreTestCase):
|
||||
self.assertEqual(body["running"]["title"], "Move library")
|
||||
|
||||
self.api._jobs.list()[0].finish()
|
||||
await self.preview(self.upload(["restored"]))
|
||||
body = await self.preview(self.upload(["restored"]))
|
||||
self.gated_job("restore", "exclusive", RESTORE_JOB_TITLE)
|
||||
resp = await self.client.post("/prompt_manager/system/library/move")
|
||||
self.assertEqual(resp.status, 409)
|
||||
|
||||
@@ -116,7 +116,7 @@ class ReviewToolTests(ToolFixture):
|
||||
self.assertEqual(code, 0)
|
||||
(verdict,) = json.loads(out.split("--apply, for")[0])
|
||||
self.assertEqual(verdict["verdict"], "manual")
|
||||
self.run_tool("--ids", str(self.guessed[0]), "--apply")
|
||||
code, _out = self.run_tool("--ids", str(self.guessed[0]), "--apply")
|
||||
self.assertEqual(self.links(), before)
|
||||
|
||||
def test_prompt_selection_and_text_report(self):
|
||||
|
||||
@@ -337,7 +337,7 @@ class TestDefaultProfileStillDecides(InspectionTestCase):
|
||||
side_effect=AssertionError("inspection choices were asked"),
|
||||
),
|
||||
):
|
||||
_, (status, data) = await self._apply("cat")
|
||||
_item, (status, data) = await self._apply("cat")
|
||||
self.assertEqual(status, 200, data)
|
||||
# The saver anchor check (#390): with the file's name, nothing matches.
|
||||
self.assertEqual([c[1] for c in calls], ["two.png"])
|
||||
@@ -383,7 +383,7 @@ class TestApplyInspectionCandidate(InspectionTestCase):
|
||||
return real(metadata, file_name)
|
||||
|
||||
with patch.object(review_capture, "resolve_inspection", side_effect=spy):
|
||||
_, (status, data) = await self._apply("make it night")
|
||||
_item, (status, data) = await self._apply("make it night")
|
||||
|
||||
self.assertEqual(status, 200, data)
|
||||
self.assertEqual(len(calls), 1)
|
||||
@@ -566,7 +566,7 @@ class TestVersionChanges(InspectionTestCase):
|
||||
self.assertNotEqual(item["resolver_version"], old["resolver_version"])
|
||||
self.assertEqual(job["scope"]["review_items"], 1)
|
||||
self.assertEqual(self._pending(), 1)
|
||||
_, (status, data) = await self._apply("make it night")
|
||||
_item, (status, data) = await self._apply("make it night")
|
||||
self.assertEqual(status, 200, data)
|
||||
self.assertEqual(self._linked_text(path), "make it night")
|
||||
|
||||
@@ -885,7 +885,7 @@ class TestInspectionResolverVersion(InspectionTestCase):
|
||||
await self._scan("all")
|
||||
(item,) = self._items()
|
||||
self.assertNotEqual(item["id"], old["id"])
|
||||
_, (status, data) = await self._apply("add a red hat")
|
||||
_item, (status, data) = await self._apply("add a red hat")
|
||||
|
||||
self.assertEqual(status, 200, data)
|
||||
self.assertEqual(self._linked_text(path), "add a red hat")
|
||||
|
||||
@@ -169,7 +169,7 @@ class TestInspectionItemsAnchor(InspectionTestCase):
|
||||
path = self._write("branch_1_00001_.png", graph=branch_graph(*PROMPTS))
|
||||
await self._scan()
|
||||
|
||||
_, (status, data) = await self._apply("add a red hat")
|
||||
_item, (status, data) = await self._apply("add a red hat")
|
||||
|
||||
self.assertEqual(status, 200, data)
|
||||
self.assertEqual(self._linked_text(path), "add a red hat")
|
||||
@@ -316,7 +316,7 @@ class TestDefaultItemsAnchor(InspectionTestCase):
|
||||
with patch.object(review_capture, "_captured_inspection", return_value=None):
|
||||
await self._scan()
|
||||
|
||||
_, (status, data) = await self._apply("cat")
|
||||
_item, (status, data) = await self._apply("cat")
|
||||
|
||||
self.assertEqual(status, 200, data)
|
||||
self.assertEqual(self._linked_text(path), "cat")
|
||||
|
||||
@@ -7,7 +7,7 @@ import threading
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from database.run_attribution import (
|
||||
RecordedOutput,
|
||||
@@ -86,10 +86,11 @@ class RecordedRuns:
|
||||
|
||||
|
||||
class TestStoredRunPrompt(RecordedRuns, unittest.TestCase):
|
||||
setUp = history_tests.HistoryDatabaseTests.setUp
|
||||
close_databases = history_tests.HistoryDatabaseTests.close_databases
|
||||
database = history_tests.HistoryDatabaseTests.database
|
||||
|
||||
def setUp(self): # the shared temp dir and database list first
|
||||
def setUp(self): # noqa: F811 - the shared temp dir and database list first
|
||||
history_tests.HistoryDatabaseTests.setUp(self)
|
||||
self.db = self.database()
|
||||
self.output = str(Path(self.temp.name) / "output")
|
||||
|
||||
@@ -10,7 +10,7 @@ import os
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ import threading
|
||||
import unittest
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
try:
|
||||
from tests.open_handles import assert_closed
|
||||
|
||||
@@ -894,7 +894,7 @@ class TestLegacyStreamOnTheJob(ThumbnailJobCase):
|
||||
self.assertEqual(resp.status, 200)
|
||||
await self._wait(first)
|
||||
job = self.api._jobs.active("thumbnails")
|
||||
_, state["left"] = self._watch_subscribers(job)
|
||||
_subscribed, state["left"] = self._watch_subscribers(job)
|
||||
resp.close() # the gallery tab went away
|
||||
closed.set()
|
||||
await self._wait(state["left"]) # the server saw the client go
|
||||
|
||||
@@ -12,7 +12,7 @@ import stat
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -998,7 +998,7 @@ class TestConfiguration(IsolatedLoggerTestCase):
|
||||
def log_from_background_thread() -> None:
|
||||
try:
|
||||
self.manager.logger.info(message)
|
||||
except Exception as error:
|
||||
except BaseException as error:
|
||||
errors.append(error)
|
||||
|
||||
# Allow this record through the fixture's normal main-thread filter.
|
||||
|
||||
@@ -15,7 +15,7 @@ import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
import unittest.mock as mock
|
||||
from unittest import mock
|
||||
|
||||
from aiohttp import web
|
||||
from aiohttp.test_utils import TestClient, TestServer
|
||||
|
||||
Vendored
+4
-4
@@ -129,11 +129,11 @@ test('Firefox/WebKit page workflows run only outside pull requests; coverage run
|
||||
}
|
||||
});
|
||||
|
||||
test('coverage runs as four shards that each upload their raw coverage', () => {
|
||||
test('coverage runs as three shards that each upload their raw coverage', () => {
|
||||
const coverage = allJobs.get('frontend-coverage');
|
||||
assert.deepEqual(matrixList(coverage, 'shard'), ['1', '2', '3', '4']);
|
||||
assert.match(coverage, /^ {4}name: frontend coverage \(shard \$\{\{ matrix\.shard \}\}\/4\)$/m);
|
||||
assert.match(coverage, /npm run --prefix tools\/vendor test:coverage:shard -- \$\{\{ matrix\.shard \}\}\/4/);
|
||||
assert.deepEqual(matrixList(coverage, 'shard'), ['1', '2', '3']);
|
||||
assert.match(coverage, /^ {4}name: frontend coverage \(shard \$\{\{ matrix\.shard \}\}\/3\)$/m);
|
||||
assert.match(coverage, /npm run --prefix tools\/vendor test:coverage:shard -- \$\{\{ matrix\.shard \}\}\/3/);
|
||||
const upload = steps(coverage).find(step => /name: coverage-raw-\$\{\{ matrix\.shard \}\}/.test(step));
|
||||
assert.ok(upload, 'each shard uploads coverage-raw-N');
|
||||
assert.match(upload, /^ {10}path: coverage\/js\/raw\/?$/m);
|
||||
|
||||
@@ -506,10 +506,8 @@ def markdown(plan, prompt_texts):
|
||||
"",
|
||||
f"## Relink ({sum(i['action'] == 'relink' for i in rows)})",
|
||||
"",
|
||||
(
|
||||
"| row | file | linked prompt | linked text | target | target hash "
|
||||
"| target text |"
|
||||
),
|
||||
"| row | file | linked prompt | linked text | target | target hash "
|
||||
"| target text |",
|
||||
"|---:|---|---:|---|---|---|---|",
|
||||
]
|
||||
for item in (i for i in rows if i["action"] == "relink"):
|
||||
|
||||
@@ -386,9 +386,14 @@ class GalleryDiagnostics:
|
||||
except ImportError:
|
||||
self.logger.error(" [FAIL] PIL (Pillow): NOT AVAILABLE")
|
||||
|
||||
# sqlite3 is imported at the top of this module, so it is always here.
|
||||
dependencies["sqlite3"] = True
|
||||
self.logger.info(f" [PASS] sqlite3: {sqlite3.sqlite_version}")
|
||||
# Check sqlite3
|
||||
try:
|
||||
import sqlite3
|
||||
|
||||
dependencies["sqlite3"] = True
|
||||
self.logger.info(f" [PASS] sqlite3: {sqlite3.sqlite_version}")
|
||||
except ImportError:
|
||||
self.logger.error(" [FAIL] sqlite3: NOT AVAILABLE")
|
||||
|
||||
all_deps_ok = all(dependencies.values())
|
||||
|
||||
|
||||
@@ -398,7 +398,6 @@ class HistoryWriter:
|
||||
try:
|
||||
command = self._queue.get_nowait()
|
||||
except queue.Empty:
|
||||
# no command waiting; the loop below handles command None
|
||||
pass
|
||||
if probe:
|
||||
try:
|
||||
|
||||
+15
-91
@@ -96,65 +96,6 @@ WORKER_JOIN_SECONDS = 5.0
|
||||
OBSERVER_JOIN_SECONDS = 5.0
|
||||
|
||||
|
||||
def _start_daemon_observer(observer) -> None:
|
||||
"""``observer.start()`` on a short-lived daemon thread (#561).
|
||||
|
||||
A thread is a daemon when the thread that creates it is one. watchdog
|
||||
7 on Windows starts a plain ``threading.Thread`` per watch inside
|
||||
``start()``; created from ComfyUI's main thread it is not a daemon, and
|
||||
the interpreter waits for every such thread before it runs any
|
||||
``atexit`` callback, so the teardown that would stop it never runs.
|
||||
Started from here, every thread the observer creates is a daemon.
|
||||
Waits for the start without a limit (start-up, not exit); re-raises
|
||||
what ``start()`` raised.
|
||||
"""
|
||||
raised: List[BaseException] = []
|
||||
|
||||
def start() -> None:
|
||||
try:
|
||||
observer.start()
|
||||
except BaseException as e: # re-raised on the caller's thread
|
||||
raised.append(e)
|
||||
|
||||
starter = threading.Thread(
|
||||
target=start, name="PromptManagerObserverStart", daemon=True
|
||||
)
|
||||
starter.start()
|
||||
starter.join()
|
||||
if raised:
|
||||
raise raised[0]
|
||||
|
||||
|
||||
class _ObserverStop:
|
||||
"""One ``observer.stop()`` running on a daemon thread (#561).
|
||||
|
||||
watchdog's ``stop()`` joins its emitters (and, in 7.x on Windows, their
|
||||
reader threads) without a limit. Run here, a stop that never returns
|
||||
holds neither the caller past its deadline nor the interpreter's exit.
|
||||
"""
|
||||
|
||||
def __init__(self, observer) -> None:
|
||||
self.error: Optional[Exception] = None
|
||||
self._thread = threading.Thread(
|
||||
target=self._run,
|
||||
args=(observer,),
|
||||
name="PromptManagerObserverStop",
|
||||
daemon=True,
|
||||
)
|
||||
self._thread.start()
|
||||
|
||||
def _run(self, observer) -> None:
|
||||
try:
|
||||
observer.stop()
|
||||
except Exception as e: # reported by the stop that waits for it
|
||||
self.error = e
|
||||
|
||||
def wait(self, timeout: float) -> bool:
|
||||
"""True once ``stop()`` has returned or raised."""
|
||||
self._thread.join(timeout)
|
||||
return not self._thread.is_alive()
|
||||
|
||||
|
||||
class ImageGenerationHandler(FileSystemEventHandler):
|
||||
"""Filesystem event handler for detecting new image generation.
|
||||
|
||||
@@ -1107,8 +1048,8 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
with Image.open(image_path) as img:
|
||||
file_info["dimensions"] = list(img.size)
|
||||
file_info["format"] = img.format
|
||||
except Exception as error:
|
||||
self.logger.debug(f"No dimensions for {image_path}: {error}")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return file_info
|
||||
except Exception as e:
|
||||
@@ -1229,9 +1170,6 @@ class ImageMonitor:
|
||||
# (observer, handler) pairs a stop left running: the next stop waits
|
||||
# for them again instead of reporting True.
|
||||
self._unfinished: List[Tuple[Any, Any]] = []
|
||||
# Observer stops still running on their thread, by observer: a later
|
||||
# stop waits for the same call instead of starting another (#561).
|
||||
self._observer_stops: Dict[Any, _ObserverStop] = {}
|
||||
self.logger = get_logger("prompt_manager.image_monitor")
|
||||
self.review_recorder: Optional[Callable[[str], str]] = None
|
||||
self.thumbnail_queue: Optional[Callable[[str], str]] = None
|
||||
@@ -1335,7 +1273,7 @@ class ImageMonitor:
|
||||
self.logger.warning("No directory could be watched")
|
||||
return
|
||||
try:
|
||||
_start_daemon_observer(observer)
|
||||
observer.start()
|
||||
except Exception as e:
|
||||
self.logger.error(f"Image monitoring could not start: {e}")
|
||||
return
|
||||
@@ -1365,12 +1303,12 @@ class ImageMonitor:
|
||||
|
||||
Closes the handler first (its worker is joined and its dispatch
|
||||
returns at once), then stops the watchdog observer and joins it. The
|
||||
worker and the observer share the one deadline: the observer's stop
|
||||
and join get what the worker left, possibly 0 s. watchdog's
|
||||
``observer.stop()`` joins its emitters without a limit, so it runs on
|
||||
a daemon thread and is waited for only until the deadline (#561). A
|
||||
part an earlier call left running is waited for again. Safe to call
|
||||
when monitoring never started and more than once.
|
||||
worker and the observer share the one deadline: the observer's join
|
||||
gets what the worker left, possibly 0 s.
|
||||
watchdog's ``observer.stop()`` joins its emitters without a limit;
|
||||
it counts against the deadline but is not bounded here. A part an
|
||||
earlier call left running is waited for again. Safe to call when
|
||||
monitoring never started and more than once.
|
||||
|
||||
Returns:
|
||||
True when every worker and observer it stopped has ended.
|
||||
@@ -1411,27 +1349,13 @@ class ImageMonitor:
|
||||
return False
|
||||
|
||||
def _stop_observer(self, observer, timeout: float) -> bool:
|
||||
"""``observer.stop()`` on its own daemon thread, then a join, both
|
||||
within ``timeout`` seconds; a stop or join still running then is
|
||||
logged with the time it got. A stop still running from an earlier
|
||||
call is waited for, never started again."""
|
||||
"""``observer.stop()`` (safe to repeat), then a join of ``timeout``
|
||||
seconds; a join that times out is logged with the time it got."""
|
||||
if observer is None:
|
||||
return True
|
||||
deadline = time.monotonic() + timeout
|
||||
stop = self._observer_stops.get(observer)
|
||||
if stop is None:
|
||||
stop = self._observer_stops[observer] = _ObserverStop(observer)
|
||||
if not stop.wait(timeout):
|
||||
self.logger.warning(
|
||||
f"watchdog observer still running after {timeout:.1f} s"
|
||||
)
|
||||
return False
|
||||
del self._observer_stops[observer]
|
||||
if stop.error is not None:
|
||||
self.logger.warning(f"Error while stopping image monitoring: {stop.error}")
|
||||
return False
|
||||
try:
|
||||
observer.join(max(0.0, deadline - time.monotonic()))
|
||||
observer.stop()
|
||||
observer.join(timeout)
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Error while stopping image monitoring: {e}")
|
||||
return False
|
||||
@@ -1500,8 +1424,8 @@ class ImageMonitor:
|
||||
if self.observer is not None:
|
||||
try:
|
||||
observer_alive = self.observer.is_alive()
|
||||
except Exception as error:
|
||||
self.logger.debug(f"Could not read the observer state: {error}")
|
||||
except Exception:
|
||||
pass
|
||||
return {
|
||||
"running": self.observer is not None,
|
||||
"monitored_directories": self.monitored_directories,
|
||||
|
||||
@@ -466,7 +466,6 @@ def ensure_private_dir(path: str) -> str:
|
||||
try:
|
||||
os.chmod(path, 0o700)
|
||||
except OSError:
|
||||
# the folder exists; a filesystem without modes keeps its own
|
||||
pass
|
||||
return path
|
||||
|
||||
|
||||
@@ -181,7 +181,6 @@ def _unlink_quietly(path: str) -> None:
|
||||
try:
|
||||
os.unlink(path)
|
||||
except FileNotFoundError:
|
||||
# already gone, which is what was wanted
|
||||
pass
|
||||
except OSError as error:
|
||||
logger.warning("Capture file %s could not be removed: %s", path, error)
|
||||
@@ -391,7 +390,6 @@ class CaptureStore:
|
||||
try:
|
||||
os.rmdir(self.directory)
|
||||
except FileNotFoundError:
|
||||
# already gone, which is what was wanted
|
||||
pass
|
||||
except OSError as error:
|
||||
logger.warning(
|
||||
|
||||
@@ -339,7 +339,6 @@ class FreedesktopBackend:
|
||||
try:
|
||||
os.mkdir(name, 0o700, dir_fd=parent_fd)
|
||||
except FileExistsError:
|
||||
# the folder already exists; the checks below still vet it
|
||||
pass
|
||||
except OSError as error:
|
||||
raise TrashRefused(NO_TRASH, str(error)) from error
|
||||
|
||||
@@ -114,7 +114,6 @@ def inside_any_root(path: str, roots) -> bool:
|
||||
if _identity(current) in identities:
|
||||
return True
|
||||
except OSError:
|
||||
# an unreadable folder cannot be a trash; try its parent
|
||||
pass
|
||||
parent = os.path.dirname(current)
|
||||
if parent == current:
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user