Files
ComfyAssets-ComfyUI_PromptM…/__init__.py
T
Vito c7ba481d3c perf(images): refresh the listing when the monitor sees a new output (#348, PR 2) (#353)
* perf(images): refresh the listing when the monitor sees a new output (#348, PR 2)

The image monitor now reports every settled output (linked or not, also
when linking raised) with its resolved path through set_output_listener.
The API's notify_output_settled hops from the monitor's worker thread onto
the server's loop (call_soon_threadsafe) and does nothing before the loop is
known or after cleanup. mark_root_dirty matches the path against each
walk's resolved_root (normcase(realpath), computed in the executor by
walk_root, now its fourth value) and marks the root dirty.

A dirty root gets one background walk, no sooner than
rewalk_interval = min(15, max(5, 5 x last walk's duration)) after the
previous walk started (last_started_at / last_duration are recorded on
every walk). _schedule_walk is shared with PR 1's delayed stale refresh:
one deferred walk per root, never armed while a walk runs (the walk's end
re-checks dirty: the trailing walk), at least 0.5 s away, the earlier
deadline wins, and every walk start cancels it. A failed walk clears dirty.
Rescan semantics are unchanged.

* fix(images): keep removed roots and cancelled walks from walking again (#348, PR 2)

Review findings on the monitor-triggered walks:

- reset_gallery_roots_cache now retires every listing record: its pending
  timer is cancelled, dirty is cleared, and no mark, timer or trailing walk
  starts a walk of it until a reader or the warm-up asks for that root
  again. Before, a timer or trailing walk pending when the roots changed
  stored a listing for the removed root, and the monitor (which watches the
  old roots until restart) kept it walking on every new output.
- A walk task cancelled outside cleanup_gallery_listings no longer starts a
  trailing walk while the loop shuts down.
- root_contains ignores a Windows extended-length prefix (\\?\ and
  \\?\UNC\) on either side, which realpath can keep on a file past MAX_PATH.
- The REWALK comment states the ~20% GIL share only for walks up to 3 s.

* test(images): prove a cancelled walk starts no trailing walk (#348, PR 2)

The previous assertions held even with the trailing walk, because that
walk finished before they ran; assert the walk number and the scan count.

* perf(images): add a settled output to the listing without a walk (#348, PR 2)

Live test: each landing in a 20,000-file folder cost a full walk (~0.4 s of
GIL) plus a full workspace rebuild, so Images reads during a run took up to
2.1 s. The monitor's report now patches the stored listing instead.

- output_settled (the monitor hop's target) slices the file's part below
  the root in its own case (relative_part; Windows compares folded), and
  _run_adds places one root's pending files in one executor batch:
  _settled_entry applies the walk's filters (suffix, thumbnails, depth,
  symlink by lstat; macOS on-disk spelling), insert_entry puts the
  (Path, mtime) entry where the newest-first sort does, add_folder copies
  the folder sets. Stored only when no walk started and no drop happened
  meanwhile; walked_at is untouched, so the 30 s refresh still walks.
- Anything inexact marks the root dirty and walks as before: a running
  walk, an unsliceable path, a failed lstat, a tie in mtime, the cap, an
  unexpected error. Rescan, the refresh, the warm-up and invalidation stay
  full walks; a walk start forgets the pending adds.
- The workspace caches follow a patched listing by entry identity
  (listing_delta): only added values are built, the newest order places
  them by bisection, the favorites lookup is copied and extended.

Synthetic 20,000 files: per landing 252 ms (walk + full build) -> 18 ms
(3 ms add + 15 ms warm build).

* fix(images): review fixes for adds without a walk (#348, PR 2)

- _newest_step steps the cached newest order only when the per-file cache
  still holds exactly the values this call built. A stale page read racing
  a newer warm build (both run in executor threads) stored an order
  labelled with older values than it held, and the next read inserted the
  new file a second time. It also sorts in full when a roots reset dropped
  the per-file values meanwhile.
- The stored spelling is checked on Windows too (nt._getfinalpathname:
  realpath keeps the given case when it cannot open the file), and before
  the name filters, which compare the walk's stored spelling. macOS opens
  non-blocking, so a FIFO named like an image never holds the executor.
- An output in a folder the walk cannot list (no read permission) is not
  added, as the walk skips it.
- Windows-portable unit tests: POSIX paths in the insert tests, and a
  str.lower fold for the length guard (Windows' normcase keeps lengths).

* test(images): reproduce the newest-order race and fix a Windows clock flake (#348, PR 2)

- The race test now has the newer build step from the values the cached
  order was built from, which is the interleaving that inserted a file
  twice; it fails without the parts check in _newest_step.
- test_a_failed_walk_records_its_timing_and_clears_dirty compared two
  monotonic readings that Windows' ~15.6 ms clock can make equal; the
  previous start is moved back instead.
- TESTING.md: os.access ignores ACLs on Windows.
2026-10-26 16:14:50 -07:00

247 lines
9.3 KiB
Python

"""
ComfyUI_PromptManager: A comprehensive ComfyUI custom node that extends text encoding
with persistent prompt storage, advanced search capabilities, and an automatic image
gallery system.
This module provides prompt node types:
- PromptManager: CLIP encoding node that outputs CONDITIONING
- PromptManagerFocused: Editor-first layout using the same encoding contract
- PromptManagerSplit: Side-by-side editor and library using the same contract
- PromptManagerText: Text-only node that outputs STRING with prepend/append
functionality
The nodes share the same database backend for persistent prompt storage and include
an automatic image gallery system that monitors ComfyUI output directories and links
generated images to their source prompts.
Features:
- SQLite-based persistent prompt storage with deduplication
- Advanced search and categorization system
- Real-time image gallery with metadata extraction
- Web-based admin dashboard for prompt management
- Comprehensive logging and diagnostics
"""
import re
from pathlib import Path
def get_version():
"""Parse version from pyproject.toml"""
try:
pyproject_path = Path(__file__).parent / "pyproject.toml"
if pyproject_path.exists():
content = pyproject_path.read_text()
match = re.search(r'version\s*=\s*["\']([^"\']+)["\']', content)
if match:
return match.group(1)
except Exception:
pass
return "unknown"
from .prompt_manager import ( # noqa: E402
PromptManager,
PromptManagerFocused,
PromptManagerSplit,
)
from .prompt_manager_pair import PromptManagerPair # noqa: E402
from .prompt_manager_text import PromptManagerText # noqa: E402
from .prompt_search_list import PromptSearchList # noqa: E402
NODE_CLASS_MAPPINGS = {
"PromptManager": PromptManager,
"PromptManagerFocused": PromptManagerFocused,
"PromptManagerSplit": PromptManagerSplit,
"PromptManagerPair": PromptManagerPair,
"PromptManagerText": PromptManagerText,
"PromptSearchList": PromptSearchList,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"PromptManager": "Prompt Manager (Classic)",
"PromptManagerFocused": "Prompt Manager (Focused)",
"PromptManagerSplit": "Prompt Manager (Split)",
"PromptManagerPair": "Prompt Manager (+/-)",
"PromptManagerText": "Prompt Manager Text",
"PromptSearchList": "Prompt Search List",
}
# Canvas extension only. ComfyUI loads every .js under this directory into the
# canvas page, so the admin/gallery bundles and vendored libraries under web/
# are served by our own routes instead (see py/api).
WEB_DIRECTORY = "web/comfy"
# Add API routes (same pattern as ComfyUI_Assets)
try:
# Set extension URI
import os
from .py import config
from .py.api import PromptManagerAPI
extension_uri = os.path.dirname(__file__)
config.extension_uri = extension_uri
# Register API routes using the same pattern as ComfyUI_Assets
routes = config.routes
api = PromptManagerAPI()
api.add_routes(routes)
# Temp files of preview writes a crash interrupted (#317), off the
# import path: the walk may cover a large thumbnails folder.
import threading
threading.Thread(
target=api.sweep_thumbnail_temps,
name="PromptManagerPreviewTempSweep",
daemon=True,
).start()
host_app = getattr(config.server_instance, "app", None)
if host_app is not None and hasattr(host_app, "on_cleanup"):
host_app.on_cleanup.append(api.cleanup_history_reads)
host_app.on_cleanup.append(api.cleanup_annotation_captures)
host_app.on_cleanup.append(api.cleanup_gallery_listings)
if host_app is not None and hasattr(host_app, "on_startup"):
# Build each output root's Images listing once the server's loop runs,
# so the first open does not wait for the walk (#348).
host_app.on_startup.append(api.warm_gallery_listings)
if host_app is not None and hasattr(host_app, "on_shutdown"):
# Open Needs review change streams end before the server waits on them.
host_app.on_shutdown.append(api.close_review_events)
except Exception as e:
# Log to internal logging system without console spam
try:
from .utils.logging_config import get_logger
logger = get_logger("prompt_manager.init")
logger.error(f"Failed to register API routes: {e}")
except Exception:
pass
# Shared database, set up with the monitoring system below. Stays None when that
# fails so the queue hook can report the database as unavailable instead of crashing.
_global_db = None
# Start image monitoring globally at module import time
# This ensures images are linked regardless of which node is used in the workflow
try:
from .database.operations import PromptDatabase
from .utils.image_monitor import get_image_monitor
from .utils.prompt_tracker import get_prompt_tracker
from .utils.logging_config import get_logger
_init_logger = get_logger("prompt_manager.init")
_init_logger.info("Starting global image monitoring system...")
# Initialize database and get singleton instances
_global_db = PromptDatabase()
_global_prompt_tracker = get_prompt_tracker(_global_db)
_global_image_monitor = get_image_monitor(_global_db, _global_prompt_tracker)
# The Needs review pass for ambiguous new outputs runs through the API's
# capture store (the one the scan and the Tag dialog use): a linked output
# is never queued and its pending items leave the queue. It stops before
# that store is closed at shutdown.
_review_api = globals().get("api")
if _review_api is not None:
_global_image_monitor.set_review_recorder(_review_api.record_review_file)
# Every settled output's preview is queued for the API's background
# worker (#249): the workspace never asks to generate thumbnails.
_global_image_monitor.set_thumbnail_queue(_review_api.queue_capture_thumbnail)
# Every settled output walks its Images listing again (#348): the
# API hops from the monitor's thread onto the server's loop.
_global_image_monitor.set_output_listener(_review_api.notify_output_settled)
_review_host = getattr(config.server_instance, "app", None)
if _review_host is not None and hasattr(_review_host, "on_cleanup"):
async def _stop_review_recording(
app, monitor=_global_image_monitor, api=_review_api
):
monitor.set_review_recorder(None)
monitor.set_thumbnail_queue(None)
monitor.set_output_listener(None)
api.close_auto_thumbnails(final=True)
_review_host.on_cleanup.insert(0, _stop_review_recording)
# Start monitoring
_global_image_monitor.start_monitoring()
_init_logger.info("Global image monitoring system started successfully")
except Exception as e:
try:
from .utils.logging_config import get_logger
_init_logger = get_logger("prompt_manager.init")
_init_logger.error(f"Failed to start image monitoring: {e}")
except Exception:
print(f"[ComfyUI-PromptManager] Warning: Failed to start image monitoring: {e}")
def _db_factory():
"""The shared database for the queue hook; called lazily per request."""
if _global_db is None:
raise RuntimeError("PromptManager database unavailable")
return _global_db
# Count prompt runs when ComfyUI dequeues workflows (it skips unchanged nodes, so
# node execution alone misses re-runs). Separate from monitoring so either can fail.
try:
from .py.config import server_instance
from .utils.usage_tracking import register_queue_hook
register_queue_hook(server_instance, _db_factory)
except Exception as e:
try:
from .utils.logging_config import get_logger
get_logger("prompt_manager.init").error(f"Failed to register usage hook: {e}")
except Exception:
import logging
logging.getLogger("prompt_manager.init").error(
f"Failed to register usage hook: {e}"
)
def _history_db_factory() -> "PromptDatabase":
"""Initialize history on its worker even if monitoring initialization failed."""
if _global_db is not None:
return _global_db
from .database.operations import PromptDatabase
return PromptDatabase()
_history_registration = None
try:
from .py.config import server_instance
from .utils.execution_history import register_history_hook
_history_registration = register_history_hook(server_instance, _history_db_factory)
except Exception:
try:
from .utils.logging_config import get_logger
get_logger("prompt_manager.init").error("Failed to register execution history")
except Exception:
import logging
logging.getLogger("prompt_manager.init").error(
"Failed to register execution history"
)
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
# Print startup message with loaded tools
print()
print(f"\033[94m[ComfyUI-PromptManager] Version:\033[0m {get_version()}")
for node_key, display_name in NODE_DISPLAY_NAME_MAPPINGS.items():
# ASCII only: a Windows console using a legacy code page cannot print emoji,
# and a UnicodeEncodeError here would abort loading the whole node.
print(f" \033[94mLoaded:\033[0m {display_name}")
print(f"\033[94mTotal: {len(NODE_CLASS_MAPPINGS)} tools loaded\033[0m")
print()