* feat(system): report configured, effective and watched sources
GET /prompt_manager/settings adds settings.monitoring_enabled, a sources
block (configured roots with valid/code/reason, effective roots with their
listing index and watch state, auto_detected) and the monitor state
(present, observer_alive, scheduled). Every path is a public path.
A source is "watching" only while a live observer scheduled it, "manual"
when monitoring is off or the process has no monitor, and otherwise
"restart_pending".
POST accepts monitoring_enabled as a strict bool, saved under
gallery.monitoring.enabled; restart_required_fields names it when the
running observer does not already match. A save that changes the sources
runs as a "Source change" records hold, so it is refused with 409 busy
while a scan runs. Case variants of one folder are saved once.
POST /prompt_manager/settings/validate-source {path} checks a path
lexically (no "..", no control characters, native path forms only) and
then against the gallery-root rules, answering a code and reason, and
writes nothing.
* fix(system): no existence oracle in source validation
A gallery root's containment in the allowed ComfyUI folders is now
decided before its existence and type, so a path outside them answers
"outside-allowed" whether or not it exists. validate-source also folds
the filesystem-root and home-directory codes into "outside-allowed", with
one reason, and logs the detail server-side only. Paths inside the
allowed folders still say "not-found" or "not-directory". The legacy
settings save shares the same check.
* feat(scan): scan one selected source
POST /prompt_manager/jobs/scan accepts roots: indexes into the effective
sources, strict non-negative ints (no bools or floats), deduplicated, at
most 64, non-empty when present. An index past the sources is a 400
"unknown-source" and starts nothing. The job walks only the selected
sources, resolved when it was requested, and keeps each file's listing
root_index. Its scope records files, roots (null for every source) and
output_folders, the number of folders walked, which the log's first line
counts. The legacy /scan route still scans every source, and attaching to
a scoped job keeps its payload keys.
* feat(system): manage sources and start scans natively
System > Sources & scans renders the concept's section on the server's
sources: one row per effective source with its watch state and a scan
scoped to that source, the 3.2.0 update notice with the legacy dismissal,
Add source checked by validate-source, Remove, monitoring and worker
threads as host drafts behind the one Save row, and Refresh files.
A save refused by a running scan names the job. Thumbnail generation
hands off to the legacy gallery until S5.
Deviations D98, D113 and D-NEW-S4-1 are recorded in TESTING.md.
* fix(security): decide source containment as text before any filesystem access
Every caller (validate-source, POST /settings, config.json at load) now
refuses a source through one check, sources.source_problem: the form
check first, then GalleryConfig.gallery_root_problem, which compares the
normalised spelling with the allowed folders before anything resolves,
stats or opens the path. A UNC share, the \\?\ and \\.\ namespaces, a
device or any outside folder is refused with no SMB request, no
executor stall and no existence or timing oracle; only then is the path
resolved and contained again, which still catches a symlink out. The
filesystem root and the home directory answer outside-allowed with the
same reason as any outside path, so no caller learns the home path.
Also:
- POST /settings runs the form checks and parses the roots off the loop;
an unusable source is a coded 400 invalid-source naming its public
path (as text) and its index.
- ComfyUI folders outside the base directory are path anchors, so their
sources round-trip and never collide by folder name.
- A per-source scan sends each index with its public path; a stale or
shifted index is 409 sources-changed and starts nothing.
- A source still watched after monitoring was switched off is
"stopping".
- Roots are compared as folders, not spellings, so an unchanged legacy
resend takes no source-change hold.
- Tests: no realpath/stat/lstat of an outside path (native and Windows
forms with ntpath fakes), the alike answers with /, ~ and a link to ~,
an event instead of a timer wait, and a teardown that drains jobs
before the temporary database goes.
* fix(system): say what Sources saved, refused and still watches
- A per-source scan sends the source's public path; when the sources
changed since they were read the toast says the server's reason and
the list and saved values are read again.
- "watching until restart" for a source the running monitor still
watches after monitoring was saved off, and the status adds "Restart
ComfyUI to apply: Watch for new outputs." from restart_required_fields.
- A save refused for one unusable source marks that row and the status
is the server's message naming it.
- A valid configured source the effective list holds under another
spelling reads "same folder as another source".
- The busy toast names the job the 409 named (ApiError keeps a
validated running job and refused source), falling back to the list.
- The Add source dialog's listeners live on the page scope and Enter
adds; its markup is compared with the concept (D127).
- TESTING.md: D-NEW-S4-1 is D127; D-NEW-S4-2 records the new copy; the
Sources contract describes the lexical containment, invalid-source and
sources-changed.
* fix(security): guard the Windows form probes on a token of their own
The no-filesystem guard matched "C:", which on a Windows runner is also
the temporary ComfyUI tree the save may resolve for its trusted first
entry. The probe paths now carry a token no trusted directory holds.
* docs(system): number the Sources states deviation D128
281 lines
9.8 KiB
Python
281 lines
9.8 KiB
Python
"""Image sources as System › Sources & scans shows them (#249 S4).
|
||
|
||
- ``source_form_error`` is the lexical check a submitted source path passes
|
||
before any filesystem access: no control characters, no ``..`` part, and
|
||
only the native path forms (on Windows a drive or a UNC share, never the
|
||
``\\\\?\\`` and ``\\\\.\\`` namespaces, a drive-relative path, an alternate
|
||
data stream, a device name or a part Windows would silently trim; on POSIX
|
||
no Windows drive or backslash form).
|
||
- ``source_problem`` is the one refusal every caller answers (validate-source
|
||
and the settings save): the form check, then
|
||
``GalleryConfig.gallery_root_problem``, which decides containment as text
|
||
before it touches the filesystem.
|
||
- ``describe_sources`` (blocking: it resolves paths) reports the configured
|
||
sources with their validity, the effective sources with their listing
|
||
index and watch state, and the image monitor's state. Every path in it is
|
||
a public path (``_public_path``), never an absolute one.
|
||
|
||
A source's watch state is ``watching`` when the monitor's observer is alive,
|
||
scheduled that directory and monitoring is on; ``stopping`` when it still
|
||
watches it but monitoring was switched off (it stops at the next restart);
|
||
``manual`` when monitoring is off or this process has no monitor; else
|
||
``restart_pending`` (a source added since the monitor started, monitoring
|
||
switched on, or an observer that stopped).
|
||
"""
|
||
|
||
import logging
|
||
import ntpath
|
||
import os
|
||
import re
|
||
|
||
logger = logging.getLogger("prompt_manager.api")
|
||
|
||
MAX_SOURCE_PATH = 4096 # characters
|
||
FORM_REASONS = {
|
||
"empty": "Gallery root must be a non-empty path",
|
||
"invalid-form": "Gallery root is not a folder path this server can use",
|
||
"traversal": "Gallery root must not contain '..'",
|
||
}
|
||
_DRIVE = re.compile(r"[A-Za-z]:")
|
||
_POSIX_FOREIGN = re.compile(r"^[A-Za-z]:")
|
||
# Reserved device names: Windows opens the device for any folder of that name.
|
||
_DEVICE_NAME = re.compile(r"^(con|prn|aux|nul|com[0-9¹²³]|lpt[0-9¹²³])(\..*)?$", re.I)
|
||
# folder_paths getters and the label of the directory each one reports.
|
||
_COMFYUI_LABELS = (
|
||
("get_output_directory", "ComfyUI output"),
|
||
("get_input_directory", "ComfyUI input"),
|
||
("get_temp_directory", "ComfyUI temp"),
|
||
("get_user_directory", "ComfyUI user"),
|
||
)
|
||
WATCH_STATES = ("watching", "stopping", "manual", "restart_pending")
|
||
|
||
|
||
def source_form_error(text, *, windows=None):
|
||
"""The form code that refuses ``text`` as a source path, or None.
|
||
|
||
``windows`` selects the path rules (default: this OS). No filesystem
|
||
access happens here.
|
||
"""
|
||
if windows is None:
|
||
windows = os.name == "nt"
|
||
if not isinstance(text, str) or len(text) > MAX_SOURCE_PATH:
|
||
return "invalid-form"
|
||
if any(ord(char) < 32 or ord(char) == 127 for char in text):
|
||
return "invalid-form"
|
||
text = text.strip()
|
||
if not text:
|
||
return "empty"
|
||
return _windows_form_error(text) if windows else _posix_form_error(text)
|
||
|
||
|
||
def _posix_form_error(text):
|
||
if "\\" in text or _POSIX_FOREIGN.match(text):
|
||
return "invalid-form" # a Windows path names nothing on this server
|
||
return "traversal" if ".." in text.split("/") else None
|
||
|
||
|
||
def _windows_form_error(text):
|
||
portable = text.replace("/", "\\")
|
||
if portable.startswith(("\\\\?\\", "\\\\.\\")):
|
||
return "invalid-form" # verbatim and device namespaces
|
||
if portable.startswith("\\\\"):
|
||
drive, rest = ntpath.splitdrive(portable)
|
||
share = drive[2:].split("\\")
|
||
if len(share) != 2 or any(part in ("", ".", "..") for part in share):
|
||
return "invalid-form"
|
||
else:
|
||
drive, rest = ntpath.splitdrive(portable)
|
||
if drive and (not _DRIVE.fullmatch(drive) or not rest.startswith("\\")):
|
||
return "invalid-form" # "C:folder" is relative to that drive's cwd
|
||
if ":" in rest:
|
||
return "invalid-form" # an alternate data stream
|
||
parts = rest.split("\\")
|
||
if ".." in parts:
|
||
return "traversal"
|
||
for part in parts:
|
||
if part in ("", "."):
|
||
continue
|
||
if part[-1] in ". " or _DEVICE_NAME.match(part):
|
||
return "invalid-form"
|
||
return None
|
||
|
||
|
||
def source_problem(text):
|
||
"""``(code, reason)`` that refuses ``text`` as a source, else ``(None, "")``.
|
||
|
||
The form check runs first and touches nothing; then containment is
|
||
decided as text before any filesystem access (see
|
||
``GalleryConfig.gallery_root_problem``). Every caller answers alike.
|
||
"""
|
||
from ..config import GalleryConfig
|
||
|
||
code = source_form_error(text)
|
||
if code is not None:
|
||
return code, FORM_REASONS[code]
|
||
return GalleryConfig.gallery_root_problem(text)
|
||
|
||
|
||
def public_spelling(text):
|
||
"""How a refused source is named in a response, computed as text only.
|
||
|
||
Relative to the path anchor it lies under (its public path), else its
|
||
last part: the caller sent the path, so naming it reveals nothing, and a
|
||
refused path is never resolved to name it.
|
||
"""
|
||
from ..config import GalleryConfig, _lexical_path, _lexically_inside
|
||
|
||
text = str(text).strip()
|
||
try:
|
||
lexical = os.path.normpath(text)
|
||
key = _lexical_path(text) # the same length as ``lexical``
|
||
if not os.path.isabs(text):
|
||
return lexical.replace("\\", "/")
|
||
for anchor in GalleryConfig.path_anchors():
|
||
if _lexically_inside(key, [anchor]):
|
||
rest = lexical[len(anchor.rstrip(os.sep)) :].lstrip(os.sep)
|
||
return rest.replace(os.sep, "/") or "."
|
||
return os.path.basename(lexical) or lexical
|
||
except (TypeError, ValueError):
|
||
return ""
|
||
|
||
|
||
def identity(path):
|
||
"""The comparison key of a directory: resolved and case-folded (Windows)."""
|
||
return os.path.normcase(os.path.realpath(str(path)))
|
||
|
||
|
||
def _comfyui_labels():
|
||
"""``{identity: label}`` of the directories ComfyUI reports."""
|
||
try:
|
||
import folder_paths
|
||
except ImportError:
|
||
return {}
|
||
labels = {}
|
||
for getter_name, label in _COMFYUI_LABELS:
|
||
getter = getattr(folder_paths, getter_name, None)
|
||
if getter is None:
|
||
continue
|
||
try:
|
||
directory = getter()
|
||
except Exception:
|
||
continue
|
||
if isinstance(directory, str) and directory:
|
||
labels.setdefault(identity(directory), label)
|
||
return labels
|
||
|
||
|
||
def source_label(path, labels=None):
|
||
""" "ComfyUI output" for ComfyUI's own directories, else the folder name."""
|
||
if labels is None:
|
||
labels = _comfyui_labels()
|
||
label = labels.get(identity(path))
|
||
if label:
|
||
return label
|
||
return os.path.basename(os.path.normpath(str(path))) or str(path)
|
||
|
||
|
||
def _configured(entry, labels, public_path):
|
||
from ..config import GalleryConfig
|
||
|
||
code, reason = GalleryConfig.gallery_root_problem(entry)
|
||
try:
|
||
resolved = GalleryConfig.resolve_gallery_root(entry)
|
||
except (OSError, TypeError, ValueError):
|
||
resolved = str(entry)
|
||
return {
|
||
"label": source_label(resolved, labels),
|
||
"path": public_path(resolved),
|
||
"valid": code is None,
|
||
"code": code,
|
||
"reason": reason or None,
|
||
}
|
||
|
||
|
||
def monitor_state(monitor):
|
||
"""``(present, observer_alive, scheduled directories)`` of the image monitor."""
|
||
if monitor is None:
|
||
return False, False, []
|
||
status = monitor.get_status()
|
||
scheduled = [str(d) for d in status.get("monitored_directories") or []]
|
||
return True, bool(status.get("observer_alive")), scheduled
|
||
|
||
|
||
def watch_state(path, *, present, alive, scheduled_keys, enabled):
|
||
"""One of ``WATCH_STATES`` (see the module docstring)."""
|
||
if alive and identity(path) in scheduled_keys:
|
||
return "watching" if enabled else "stopping"
|
||
if not present or not enabled:
|
||
return "manual"
|
||
return "restart_pending"
|
||
|
||
|
||
def describe_sources(*, configured, output_dirs, auto_detected, monitor, enabled):
|
||
"""``(sources, monitor)`` for GET /settings (blocking; public paths only).
|
||
|
||
``configured`` is the configured roots, ``output_dirs`` and
|
||
``auto_detected`` what ``_output_dirs_with_origin`` returned, ``monitor``
|
||
the running ImageMonitor or None, ``enabled`` the monitoring setting.
|
||
"""
|
||
from . import _public_path
|
||
|
||
labels = _comfyui_labels()
|
||
present, alive, scheduled = monitor_state(monitor)
|
||
scheduled_keys = {identity(path) for path in scheduled}
|
||
effective = [
|
||
{
|
||
"label": source_label(path, labels),
|
||
"path": _public_path(path),
|
||
"root_index": index,
|
||
"watch": watch_state(
|
||
path,
|
||
present=present,
|
||
alive=alive,
|
||
scheduled_keys=scheduled_keys,
|
||
enabled=enabled,
|
||
),
|
||
}
|
||
for index, path in enumerate(output_dirs)
|
||
]
|
||
return (
|
||
{
|
||
"configured": [
|
||
_configured(entry, labels, _public_path) for entry in configured
|
||
],
|
||
"effective": effective,
|
||
"auto_detected": bool(auto_detected),
|
||
},
|
||
{
|
||
"present": present,
|
||
"observer_alive": alive,
|
||
"scheduled": [_public_path(path) for path in scheduled],
|
||
},
|
||
)
|
||
|
||
|
||
def validate_source(text):
|
||
"""``{valid, label, path, code, reason}`` for one submitted path (blocking).
|
||
|
||
Writes nothing. An invalid path answers its code and reason with no
|
||
label or path, so a response never echoes what was typed.
|
||
"""
|
||
from ..config import GalleryConfig
|
||
from . import _public_path
|
||
|
||
code, reason = source_problem(text)
|
||
if code is not None:
|
||
return {
|
||
"valid": False,
|
||
"label": None,
|
||
"path": None,
|
||
"code": code,
|
||
"reason": reason,
|
||
}
|
||
resolved = GalleryConfig.resolve_gallery_root(text)
|
||
return {
|
||
"valid": True,
|
||
"label": source_label(resolved),
|
||
"path": _public_path(resolved),
|
||
"code": None,
|
||
"reason": None,
|
||
}
|