Files
ComfyAssets-ComfyUI_PromptM…/tests/browser/workspace_system_fixture.py
T
Vito 69de20a427 feat(system): Sources & scans (#309)
* feat(system): report configured, effective and watched sources

GET /prompt_manager/settings adds settings.monitoring_enabled, a sources
block (configured roots with valid/code/reason, effective roots with their
listing index and watch state, auto_detected) and the monitor state
(present, observer_alive, scheduled). Every path is a public path.

A source is "watching" only while a live observer scheduled it, "manual"
when monitoring is off or the process has no monitor, and otherwise
"restart_pending".

POST accepts monitoring_enabled as a strict bool, saved under
gallery.monitoring.enabled; restart_required_fields names it when the
running observer does not already match. A save that changes the sources
runs as a "Source change" records hold, so it is refused with 409 busy
while a scan runs. Case variants of one folder are saved once.

POST /prompt_manager/settings/validate-source {path} checks a path
lexically (no "..", no control characters, native path forms only) and
then against the gallery-root rules, answering a code and reason, and
writes nothing.

* fix(system): no existence oracle in source validation

A gallery root's containment in the allowed ComfyUI folders is now
decided before its existence and type, so a path outside them answers
"outside-allowed" whether or not it exists. validate-source also folds
the filesystem-root and home-directory codes into "outside-allowed", with
one reason, and logs the detail server-side only. Paths inside the
allowed folders still say "not-found" or "not-directory". The legacy
settings save shares the same check.

* feat(scan): scan one selected source

POST /prompt_manager/jobs/scan accepts roots: indexes into the effective
sources, strict non-negative ints (no bools or floats), deduplicated, at
most 64, non-empty when present. An index past the sources is a 400
"unknown-source" and starts nothing. The job walks only the selected
sources, resolved when it was requested, and keeps each file's listing
root_index. Its scope records files, roots (null for every source) and
output_folders, the number of folders walked, which the log's first line
counts. The legacy /scan route still scans every source, and attaching to
a scoped job keeps its payload keys.

* feat(system): manage sources and start scans natively

System > Sources & scans renders the concept's section on the server's
sources: one row per effective source with its watch state and a scan
scoped to that source, the 3.2.0 update notice with the legacy dismissal,
Add source checked by validate-source, Remove, monitoring and worker
threads as host drafts behind the one Save row, and Refresh files.
A save refused by a running scan names the job. Thumbnail generation
hands off to the legacy gallery until S5.

Deviations D98, D113 and D-NEW-S4-1 are recorded in TESTING.md.

* fix(security): decide source containment as text before any filesystem access

Every caller (validate-source, POST /settings, config.json at load) now
refuses a source through one check, sources.source_problem: the form
check first, then GalleryConfig.gallery_root_problem, which compares the
normalised spelling with the allowed folders before anything resolves,
stats or opens the path. A UNC share, the \\?\ and \\.\ namespaces, a
device or any outside folder is refused with no SMB request, no
executor stall and no existence or timing oracle; only then is the path
resolved and contained again, which still catches a symlink out. The
filesystem root and the home directory answer outside-allowed with the
same reason as any outside path, so no caller learns the home path.

Also:
- POST /settings runs the form checks and parses the roots off the loop;
  an unusable source is a coded 400 invalid-source naming its public
  path (as text) and its index.
- ComfyUI folders outside the base directory are path anchors, so their
  sources round-trip and never collide by folder name.
- A per-source scan sends each index with its public path; a stale or
  shifted index is 409 sources-changed and starts nothing.
- A source still watched after monitoring was switched off is
  "stopping".
- Roots are compared as folders, not spellings, so an unchanged legacy
  resend takes no source-change hold.
- Tests: no realpath/stat/lstat of an outside path (native and Windows
  forms with ntpath fakes), the alike answers with /, ~ and a link to ~,
  an event instead of a timer wait, and a teardown that drains jobs
  before the temporary database goes.

* fix(system): say what Sources saved, refused and still watches

- A per-source scan sends the source's public path; when the sources
  changed since they were read the toast says the server's reason and
  the list and saved values are read again.
- "watching until restart" for a source the running monitor still
  watches after monitoring was saved off, and the status adds "Restart
  ComfyUI to apply: Watch for new outputs." from restart_required_fields.
- A save refused for one unusable source marks that row and the status
  is the server's message naming it.
- A valid configured source the effective list holds under another
  spelling reads "same folder as another source".
- The busy toast names the job the 409 named (ApiError keeps a
  validated running job and refused source), falling back to the list.
- The Add source dialog's listeners live on the page scope and Enter
  adds; its markup is compared with the concept (D127).
- TESTING.md: D-NEW-S4-1 is D127; D-NEW-S4-2 records the new copy; the
  Sources contract describes the lexical containment, invalid-source and
  sources-changed.

* fix(security): guard the Windows form probes on a token of their own

The no-filesystem guard matched "C:", which on a Windows runner is also
the temporary ComfyUI tree the save may resolve for its trusted first
entry. The probe paths now carry a token no trusted directory holds.

* docs(system): number the Sources states deviation D128
2026-10-23 13:15:06 -07:00

207 lines
7.9 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""System test hooks for the browser tests (#249 System S2 Task 6).
The data is the concept-shaped real fixture (``POST /__test__/concept/reset``,
``workspace_concept_fixture.py``): its parents, saved pairs, images, ratings
and runs are what the Overview, the Statistics dialog and the health check
read. These per-server hooks only change what a System test needs on top:
- ``POST /__test__/system/reset`` restores the configured sources, forgets the
session's health report and ends every injected failure;
- ``POST /__test__/system/fail`` ``{path, times}`` answers the next ``times``
requests for ``path`` (a ``/prompt_manager/system/`` route) with the routes'
500 error envelope;
- ``POST /__test__/system/missing-source`` configures one more source that
does not exist (an unmounted drive); with ``{"inside": true}`` it is an
existing folder inside the output folder (``output/unplugged``), which
``POST /__test__/system/unplug`` then removes (a drive unplugged while the
page shows it);
- ``POST /__test__/system/missing-file`` deletes the newest linked image's
file from disk, leaving its database row (a missing-file link);
- ``POST /__test__/system/unthumbnailed`` ``{count}`` writes ``count`` plain
PNGs with no thumbnail into the output folder;
- ``POST /__test__/system/monitor`` ``{watching, alive}`` stands in a fake
image monitor whose observer is ``alive`` (default true) and schedules the
named folders of this server's root (``["output"]``), so Sources reads
watching, manual scan and restart pending states (S4).
Delays use the shared request holds (``request_hold.py``).
"""
import os
from pathlib import Path
from aiohttp import web
from PIL import Image
FAILABLE_PREFIX = "/prompt_manager/system/"
MAX_UNTHUMBNAILED = 50
def register_system_commands(app, routes, api, root: Path):
# System › Appearance saves the host fields to this server's config.json (S3);
# reset restores both. The file is this server's own, inside its temp root:
# reset rewrites or deletes it, so a server whose configuration points
# anywhere else (a real config.json) refuses to register these hooks.
config_path = root / "config.json"
configured_path = os.environ.get("PROMPT_MANAGER_CONFIG_PATH")
if (
configured_path is None
or Path(configured_path).resolve() != config_path.resolve()
):
raise RuntimeError(
f"PROMPT_MANAGER_CONFIG_PATH must be {config_path}, not {configured_path!r}"
)
from py.config import GalleryConfig, PromptManagerConfig
from utils import image_monitor
configured = list(GalleryConfig.MONITORING_DIRECTORIES)
config_bytes = config_path.read_bytes() if config_path.exists() else None
host_values = (
PromptManagerConfig.RESULT_TIMEOUT,
PromptManagerConfig.WEBUI_DISPLAY_MODE,
PromptManagerConfig.WORKER_THREADS,
GalleryConfig.MONITORING_ENABLED,
)
monitor = getattr(image_monitor, "_monitor_instance", None)
failures: dict[str, int] = {}
@web.middleware
async def inject_failures(request, handler):
left = failures.get(request.path, 0)
if left <= 0:
return await handler(request)
failures[request.path] = left - 1
return web.json_response(
{"success": False, "code": None, "error": "Injected failure"}, status=500
)
app.middlewares.append(inject_failures)
unplugged = root / "output" / "unplugged"
def reset():
failures.clear()
if unplugged.is_dir():
unplugged.rmdir()
GalleryConfig.MONITORING_DIRECTORIES = list(configured)
api._health_report = None
(
PromptManagerConfig.RESULT_TIMEOUT,
PromptManagerConfig.WEBUI_DISPLAY_MODE,
PromptManagerConfig.WORKER_THREADS,
GalleryConfig.MONITORING_ENABLED,
) = host_values
image_monitor._monitor_instance = monitor
if config_bytes is None:
config_path.unlink(missing_ok=True)
else:
# Replace in one step, as the server writes it (Windows preflight).
staged = config_path.with_name(config_path.name + ".reset")
staged.write_bytes(config_bytes)
os.replace(staged, config_path)
@routes.post("/__test__/system/reset")
async def system_reset(request):
reset()
return web.json_response({"success": True})
@routes.post("/__test__/system/fail")
async def system_fail(request):
data = await request.json()
path, times = data.get("path"), data.get("times", 1)
if (
not isinstance(path, str)
or not path.startswith(FAILABLE_PREFIX)
or isinstance(times, bool)
or not isinstance(times, int)
or not 1 <= times <= 10
):
raise web.HTTPBadRequest()
failures[path] = times
return web.json_response({"success": True})
@routes.post("/__test__/system/missing-source")
async def missing_source(request):
data = await request.json() if request.can_read_body else {}
if data.get("inside") is True:
unplugged.mkdir(exist_ok=True)
absent = str(unplugged)
else:
absent = str(root / "unmounted-drive")
if os.path.exists(absent):
raise web.HTTPConflict()
GalleryConfig.MONITORING_DIRECTORIES = [*configured, absent]
return web.json_response(
{"success": True, "configured": len(GalleryConfig.MONITORING_DIRECTORIES)}
)
@routes.post("/__test__/system/unplug")
async def unplug(request):
if not unplugged.is_dir():
raise web.HTTPConflict()
unplugged.rmdir()
return web.json_response({"success": True})
@routes.post("/__test__/system/missing-file")
async def missing_file(request):
with api.db.model.get_connection() as conn:
row = conn.execute(
"SELECT file_path FROM generated_images ORDER BY id DESC LIMIT 1"
).fetchone()
if row is None:
raise web.HTTPNotFound()
Path(row[0]).unlink()
api.invalidate_gallery_cache()
return web.json_response({"success": True, "filename": Path(row[0]).name})
@routes.post("/__test__/system/unthumbnailed")
async def unthumbnailed(request):
data = await request.json()
count = data.get("count")
if (
isinstance(count, bool)
or not isinstance(count, int)
or not 1 <= count <= MAX_UNTHUMBNAILED
):
raise web.HTTPBadRequest()
folder = root / "output" / "unthumbnailed"
folder.mkdir(exist_ok=True)
for number in range(count):
with Image.new("RGB", (64, 64), "#2b3442") as image:
image.save(folder / f"plain_{number + 1:03}.png")
api.invalidate_gallery_cache()
return web.json_response({"success": True})
@routes.post("/__test__/system/monitor")
async def fake_monitor(request):
data = await request.json()
watching, alive = data.get("watching", []), data.get("alive", True)
if (
not isinstance(watching, list)
or not all(name in ("output", "input") for name in watching)
or not isinstance(alive, bool)
):
raise web.HTTPBadRequest()
image_monitor._monitor_instance = _FakeMonitor(
[str(root / name) for name in watching], alive
)
return web.json_response({"success": True})
return reset
class _FakeMonitor:
"""The image monitor's status surface (``get_status``) with no observer thread."""
def __init__(self, directories, alive):
self.monitored_directories = directories
self._alive = alive
def get_status(self):
return {
"running": True,
"monitored_directories": list(self.monitored_directories),
"handler_active": True,
"observer_alive": self._alive,
}