* feat(system): report configured, effective and watched sources
GET /prompt_manager/settings adds settings.monitoring_enabled, a sources
block (configured roots with valid/code/reason, effective roots with their
listing index and watch state, auto_detected) and the monitor state
(present, observer_alive, scheduled). Every path is a public path.
A source is "watching" only while a live observer scheduled it, "manual"
when monitoring is off or the process has no monitor, and otherwise
"restart_pending".
POST accepts monitoring_enabled as a strict bool, saved under
gallery.monitoring.enabled; restart_required_fields names it when the
running observer does not already match. A save that changes the sources
runs as a "Source change" records hold, so it is refused with 409 busy
while a scan runs. Case variants of one folder are saved once.
POST /prompt_manager/settings/validate-source {path} checks a path
lexically (no "..", no control characters, native path forms only) and
then against the gallery-root rules, answering a code and reason, and
writes nothing.
* fix(system): no existence oracle in source validation
A gallery root's containment in the allowed ComfyUI folders is now
decided before its existence and type, so a path outside them answers
"outside-allowed" whether or not it exists. validate-source also folds
the filesystem-root and home-directory codes into "outside-allowed", with
one reason, and logs the detail server-side only. Paths inside the
allowed folders still say "not-found" or "not-directory". The legacy
settings save shares the same check.
* feat(scan): scan one selected source
POST /prompt_manager/jobs/scan accepts roots: indexes into the effective
sources, strict non-negative ints (no bools or floats), deduplicated, at
most 64, non-empty when present. An index past the sources is a 400
"unknown-source" and starts nothing. The job walks only the selected
sources, resolved when it was requested, and keeps each file's listing
root_index. Its scope records files, roots (null for every source) and
output_folders, the number of folders walked, which the log's first line
counts. The legacy /scan route still scans every source, and attaching to
a scoped job keeps its payload keys.
* feat(system): manage sources and start scans natively
System > Sources & scans renders the concept's section on the server's
sources: one row per effective source with its watch state and a scan
scoped to that source, the 3.2.0 update notice with the legacy dismissal,
Add source checked by validate-source, Remove, monitoring and worker
threads as host drafts behind the one Save row, and Refresh files.
A save refused by a running scan names the job. Thumbnail generation
hands off to the legacy gallery until S5.
Deviations D98, D113 and D-NEW-S4-1 are recorded in TESTING.md.
* fix(security): decide source containment as text before any filesystem access
Every caller (validate-source, POST /settings, config.json at load) now
refuses a source through one check, sources.source_problem: the form
check first, then GalleryConfig.gallery_root_problem, which compares the
normalised spelling with the allowed folders before anything resolves,
stats or opens the path. A UNC share, the \\?\ and \\.\ namespaces, a
device or any outside folder is refused with no SMB request, no
executor stall and no existence or timing oracle; only then is the path
resolved and contained again, which still catches a symlink out. The
filesystem root and the home directory answer outside-allowed with the
same reason as any outside path, so no caller learns the home path.
Also:
- POST /settings runs the form checks and parses the roots off the loop;
an unusable source is a coded 400 invalid-source naming its public
path (as text) and its index.
- ComfyUI folders outside the base directory are path anchors, so their
sources round-trip and never collide by folder name.
- A per-source scan sends each index with its public path; a stale or
shifted index is 409 sources-changed and starts nothing.
- A source still watched after monitoring was switched off is
"stopping".
- Roots are compared as folders, not spellings, so an unchanged legacy
resend takes no source-change hold.
- Tests: no realpath/stat/lstat of an outside path (native and Windows
forms with ntpath fakes), the alike answers with /, ~ and a link to ~,
an event instead of a timer wait, and a teardown that drains jobs
before the temporary database goes.
* fix(system): say what Sources saved, refused and still watches
- A per-source scan sends the source's public path; when the sources
changed since they were read the toast says the server's reason and
the list and saved values are read again.
- "watching until restart" for a source the running monitor still
watches after monitoring was saved off, and the status adds "Restart
ComfyUI to apply: Watch for new outputs." from restart_required_fields.
- A save refused for one unusable source marks that row and the status
is the server's message naming it.
- A valid configured source the effective list holds under another
spelling reads "same folder as another source".
- The busy toast names the job the 409 named (ApiError keeps a
validated running job and refused source), falling back to the list.
- The Add source dialog's listeners live on the page scope and Enter
adds; its markup is compared with the concept (D127).
- TESTING.md: D-NEW-S4-1 is D127; D-NEW-S4-2 records the new copy; the
Sources contract describes the lexical containment, invalid-source and
sources-changed.
* fix(security): guard the Windows form probes on a token of their own
The no-filesystem guard matched "C:", which on a Windows runner is also
the temporary ComfyUI tree the save may resolve for its trusted first
entry. The probe paths now carry a token no trusted directory holds.
* docs(system): number the Sources states deviation D128
207 lines
7.9 KiB
Python
207 lines
7.9 KiB
Python
"""System test hooks for the browser tests (#249 System S2 Task 6).
|
||
|
||
The data is the concept-shaped real fixture (``POST /__test__/concept/reset``,
|
||
``workspace_concept_fixture.py``): its parents, saved pairs, images, ratings
|
||
and runs are what the Overview, the Statistics dialog and the health check
|
||
read. These per-server hooks only change what a System test needs on top:
|
||
|
||
- ``POST /__test__/system/reset`` restores the configured sources, forgets the
|
||
session's health report and ends every injected failure;
|
||
- ``POST /__test__/system/fail`` ``{path, times}`` answers the next ``times``
|
||
requests for ``path`` (a ``/prompt_manager/system/`` route) with the routes'
|
||
500 error envelope;
|
||
- ``POST /__test__/system/missing-source`` configures one more source that
|
||
does not exist (an unmounted drive); with ``{"inside": true}`` it is an
|
||
existing folder inside the output folder (``output/unplugged``), which
|
||
``POST /__test__/system/unplug`` then removes (a drive unplugged while the
|
||
page shows it);
|
||
- ``POST /__test__/system/missing-file`` deletes the newest linked image's
|
||
file from disk, leaving its database row (a missing-file link);
|
||
- ``POST /__test__/system/unthumbnailed`` ``{count}`` writes ``count`` plain
|
||
PNGs with no thumbnail into the output folder;
|
||
- ``POST /__test__/system/monitor`` ``{watching, alive}`` stands in a fake
|
||
image monitor whose observer is ``alive`` (default true) and schedules the
|
||
named folders of this server's root (``["output"]``), so Sources reads
|
||
watching, manual scan and restart pending states (S4).
|
||
|
||
Delays use the shared request holds (``request_hold.py``).
|
||
"""
|
||
|
||
import os
|
||
from pathlib import Path
|
||
|
||
from aiohttp import web
|
||
from PIL import Image
|
||
|
||
FAILABLE_PREFIX = "/prompt_manager/system/"
|
||
MAX_UNTHUMBNAILED = 50
|
||
|
||
|
||
def register_system_commands(app, routes, api, root: Path):
|
||
# System › Appearance saves the host fields to this server's config.json (S3);
|
||
# reset restores both. The file is this server's own, inside its temp root:
|
||
# reset rewrites or deletes it, so a server whose configuration points
|
||
# anywhere else (a real config.json) refuses to register these hooks.
|
||
config_path = root / "config.json"
|
||
configured_path = os.environ.get("PROMPT_MANAGER_CONFIG_PATH")
|
||
if (
|
||
configured_path is None
|
||
or Path(configured_path).resolve() != config_path.resolve()
|
||
):
|
||
raise RuntimeError(
|
||
f"PROMPT_MANAGER_CONFIG_PATH must be {config_path}, not {configured_path!r}"
|
||
)
|
||
from py.config import GalleryConfig, PromptManagerConfig
|
||
from utils import image_monitor
|
||
|
||
configured = list(GalleryConfig.MONITORING_DIRECTORIES)
|
||
config_bytes = config_path.read_bytes() if config_path.exists() else None
|
||
host_values = (
|
||
PromptManagerConfig.RESULT_TIMEOUT,
|
||
PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||
PromptManagerConfig.WORKER_THREADS,
|
||
GalleryConfig.MONITORING_ENABLED,
|
||
)
|
||
monitor = getattr(image_monitor, "_monitor_instance", None)
|
||
failures: dict[str, int] = {}
|
||
|
||
@web.middleware
|
||
async def inject_failures(request, handler):
|
||
left = failures.get(request.path, 0)
|
||
if left <= 0:
|
||
return await handler(request)
|
||
failures[request.path] = left - 1
|
||
return web.json_response(
|
||
{"success": False, "code": None, "error": "Injected failure"}, status=500
|
||
)
|
||
|
||
app.middlewares.append(inject_failures)
|
||
|
||
unplugged = root / "output" / "unplugged"
|
||
|
||
def reset():
|
||
failures.clear()
|
||
if unplugged.is_dir():
|
||
unplugged.rmdir()
|
||
GalleryConfig.MONITORING_DIRECTORIES = list(configured)
|
||
api._health_report = None
|
||
(
|
||
PromptManagerConfig.RESULT_TIMEOUT,
|
||
PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||
PromptManagerConfig.WORKER_THREADS,
|
||
GalleryConfig.MONITORING_ENABLED,
|
||
) = host_values
|
||
image_monitor._monitor_instance = monitor
|
||
if config_bytes is None:
|
||
config_path.unlink(missing_ok=True)
|
||
else:
|
||
# Replace in one step, as the server writes it (Windows preflight).
|
||
staged = config_path.with_name(config_path.name + ".reset")
|
||
staged.write_bytes(config_bytes)
|
||
os.replace(staged, config_path)
|
||
|
||
@routes.post("/__test__/system/reset")
|
||
async def system_reset(request):
|
||
reset()
|
||
return web.json_response({"success": True})
|
||
|
||
@routes.post("/__test__/system/fail")
|
||
async def system_fail(request):
|
||
data = await request.json()
|
||
path, times = data.get("path"), data.get("times", 1)
|
||
if (
|
||
not isinstance(path, str)
|
||
or not path.startswith(FAILABLE_PREFIX)
|
||
or isinstance(times, bool)
|
||
or not isinstance(times, int)
|
||
or not 1 <= times <= 10
|
||
):
|
||
raise web.HTTPBadRequest()
|
||
failures[path] = times
|
||
return web.json_response({"success": True})
|
||
|
||
@routes.post("/__test__/system/missing-source")
|
||
async def missing_source(request):
|
||
data = await request.json() if request.can_read_body else {}
|
||
if data.get("inside") is True:
|
||
unplugged.mkdir(exist_ok=True)
|
||
absent = str(unplugged)
|
||
else:
|
||
absent = str(root / "unmounted-drive")
|
||
if os.path.exists(absent):
|
||
raise web.HTTPConflict()
|
||
GalleryConfig.MONITORING_DIRECTORIES = [*configured, absent]
|
||
return web.json_response(
|
||
{"success": True, "configured": len(GalleryConfig.MONITORING_DIRECTORIES)}
|
||
)
|
||
|
||
@routes.post("/__test__/system/unplug")
|
||
async def unplug(request):
|
||
if not unplugged.is_dir():
|
||
raise web.HTTPConflict()
|
||
unplugged.rmdir()
|
||
return web.json_response({"success": True})
|
||
|
||
@routes.post("/__test__/system/missing-file")
|
||
async def missing_file(request):
|
||
with api.db.model.get_connection() as conn:
|
||
row = conn.execute(
|
||
"SELECT file_path FROM generated_images ORDER BY id DESC LIMIT 1"
|
||
).fetchone()
|
||
if row is None:
|
||
raise web.HTTPNotFound()
|
||
Path(row[0]).unlink()
|
||
api.invalidate_gallery_cache()
|
||
return web.json_response({"success": True, "filename": Path(row[0]).name})
|
||
|
||
@routes.post("/__test__/system/unthumbnailed")
|
||
async def unthumbnailed(request):
|
||
data = await request.json()
|
||
count = data.get("count")
|
||
if (
|
||
isinstance(count, bool)
|
||
or not isinstance(count, int)
|
||
or not 1 <= count <= MAX_UNTHUMBNAILED
|
||
):
|
||
raise web.HTTPBadRequest()
|
||
folder = root / "output" / "unthumbnailed"
|
||
folder.mkdir(exist_ok=True)
|
||
for number in range(count):
|
||
with Image.new("RGB", (64, 64), "#2b3442") as image:
|
||
image.save(folder / f"plain_{number + 1:03}.png")
|
||
api.invalidate_gallery_cache()
|
||
return web.json_response({"success": True})
|
||
|
||
@routes.post("/__test__/system/monitor")
|
||
async def fake_monitor(request):
|
||
data = await request.json()
|
||
watching, alive = data.get("watching", []), data.get("alive", True)
|
||
if (
|
||
not isinstance(watching, list)
|
||
or not all(name in ("output", "input") for name in watching)
|
||
or not isinstance(alive, bool)
|
||
):
|
||
raise web.HTTPBadRequest()
|
||
image_monitor._monitor_instance = _FakeMonitor(
|
||
[str(root / name) for name in watching], alive
|
||
)
|
||
return web.json_response({"success": True})
|
||
|
||
return reset
|
||
|
||
|
||
class _FakeMonitor:
|
||
"""The image monitor's status surface (``get_status``) with no observer thread."""
|
||
|
||
def __init__(self, directories, alive):
|
||
self.monitored_directories = directories
|
||
self._alive = alive
|
||
|
||
def get_status(self):
|
||
return {
|
||
"running": True,
|
||
"monitored_directories": list(self.monitored_directories),
|
||
"handler_active": True,
|
||
"observer_alive": self._alive,
|
||
}
|