Files
ComfyAssets-ComfyUI_PromptM…/utils/diagnostics.py
T
Vito 8bf759d6f8 feat(system): keep the library in ComfyUI's user folder (#318)
* feat(system): resolve every library path in one place

utils/library_location.py decides where the library lives: the
PROMPT_MANAGER_DATA_DIR folder (validated: absolute, never inside a folder
ComfyUI or Prompt Manager serves), an explicit database.default_path as
before, a node-folder library that was not moved, else ComfyUI's
user/__promptmanager. The database, diagnostics, config.json, logs and the
LoRA image cache all ask it, so a new install never writes into the node
folder and an existing one keeps every path until the user moves it.
"Keep it here" is persisted as library.keep_in_node_folder.

* feat(system): move the library as a verified Activity job

POST /prompt_manager/system/library/move starts "Move library", an
exclusive, non-cancellable job (database/library_move.py). It checks the
target volume's free space, copies the database with SQLite's backup API
from one read snapshot, verifies the copy (integrity check, every workflow
document, the same rows in every table), copies the safety and
pre-migration backups and config.json (owner-only, checked by size and
SHA-256), then publishes under the old file's write lock and switches the
live process (every PromptModel, the review and history observers, the
API's plans and health report). The old prompts.db and config.json are
renamed *.moved-to-<folder> only after the switch. A failure removes
everything it wrote and leaves the old library in use; a marker in the new
folder makes a restart after any crash pick the right library.

GET /prompt_manager/system/library and the Overview's "location" say where
the library lives and why; POST .../keep persists "Keep it here".

* feat(system): show where the library lives and offer the move

D129. System > Overview's Library health gains a "Library location" row
(the database path, why it is there, the "Keep it here" choice and any
PROMPT_MANAGER_DATA_DIR problem) with Move library for a node-folder
library and Details otherwise. While a node-folder library is not kept
there, the workspace shows the concept's .notice above every page:
"Keep it here" saves the choice on the server and stops the notice;
"Move library" opens the shared review (From, To, Size, what moves, what
happens to the old files) and starts the exclusive job in Activity. The
notice and the Overview read the location again when the job ends.

The Overview comparison maps the inserted row; a new comparison holds the
notice to the concept's Sources notice (markup, tokens, alignment, both
themes and widths) with a mutation check.

* feat(system): close the browser server's log files before its root goes

The logs now live in the library's data folder, which for the browser test
server is <root>/user/__promptmanager; Windows could not remove the temp
root while the server still held prompt_manager.log open. The API test's
node folder is spelled in lower case because public paths are case-folded
on Windows.

* feat(system): show the library location on the legacy admin page

A line under the admin page's statistics names the database path and why it is there (ComfyUI's user folder, PROMPT_MANAGER_DATA_DIR, config.json, or the node folder with a pointer to Move library), read once from GET /prompt_manager/system/library.

* feat(system): prove a fresh host keeps its library in user/__promptmanager

The execution suite starts one pinned CPU ComfyUI with no configuration at all on an OS-chosen port: the library is created in user/__promptmanager (database and logs), nothing but byte code is added to the node folder, the location route reports the default folder, and ComfyUI's /userdata API serves none of it (traversal spellings, a listing, a system-user header).

* feat(system): document where the library lives and how it moves

README: where the library lives, the upgrade path (Move library / Keep it here), PROMPT_MANAGER_DATA_DIR, explicit paths unchanged, and the way back to an older version. TESTING.md: D129 and the library location contract (resolution order, move steps, crash windows, coded refusals, Windows handling).

* wip(system): library move fix round in progress

* fix(system): retry a fallback image link that races the library switch

The image monitor's fallback and recent-prompt links went through
PromptDatabase.link_image_to_prompt, which logs and returns 0 on any
error, so survive_switch never saw a "database is locked" or closed
connection during Move library and the link was lost in both files.
The monitor now links through link_image_to_prompt_or_raise (one
transaction per attempt); link_image_to_prompt keeps its
log-and-return-0 contract for other callers. A link that returns 0 is
no longer logged as "Successfully linked image 0".

A switch that fails after repointing the models now closes the old
file's detached connections instead of leaving them to GC: switch()
adds them to a list the mover owns.

The mocked retry test is replaced by a real-lock race asserting the
image row lands in the new library only.

* fix(system): say which source is the default and where a symlinked library lives

Sources & scans: with no source configured, ComfyUI's output folder now
reads "ComfyUI/output · watching · default" on its own row instead of a
separate note line. GET /settings marks each effective source with
in_comfyui (its public path is relative to the ComfyUI base) so only
those get the "ComfyUI/" prefix; the scan still sends the server's
public path.

Library location: a library in the node folder reads through the node
folder's unresolved place in custom_nodes, so a node folder symlinked to
a checkout outside ComfyUI shows custom_nodes/promptmanager/prompts.db
instead of prompts.db (System library row, Move review, admin Library
line). Never absolute or "..", else the shared _public_path, which
sources and jobs keep using unchanged.
2026-10-24 12:54:09 -07:00

483 lines
18 KiB
Python

"""Diagnostic utilities for troubleshooting the PromptManager gallery system.
This module provides comprehensive diagnostic tools to help identify and resolve
issues with the PromptManager system, particularly the gallery and image monitoring
functionality. It performs systematic checks of all system components and provides
clear feedback on potential problems.
The diagnostic system checks:
- Database connectivity and schema integrity
- Image table structure and data
- File system permissions and access
- ComfyUI output directory detection
- Required Python dependencies
- System integration points
Typical usage:
from utils.diagnostics import GalleryDiagnostics, run_diagnostics
# Run full diagnostic suite
results = run_diagnostics()
# Or create custom diagnostic instance
diagnostics = GalleryDiagnostics("custom_db.db")
database_status = diagnostics.check_database()
The diagnostics provide:
- Clear pass/fail status for each component
- Detailed error messages and remediation suggestions
- Statistics and system information
- Test utilities for verifying functionality
"""
import os
import sqlite3
from contextlib import closing
from pathlib import Path
from typing import Dict, Any
from .logging_config import get_logger
try:
from ..database.workflow_documents import DocumentReader, resolve_image_row
except ImportError: # loaded as a top-level ``utils`` package
from database.workflow_documents import DocumentReader, resolve_image_row
class GalleryDiagnostics:
"""Diagnostics for the gallery system.
This class provides comprehensive diagnostic capabilities for the PromptManager
gallery system. It systematically checks all components and dependencies to
identify potential issues and provide actionable feedback.
The diagnostics cover:
- Database structure and connectivity
- Image tracking table integrity
- File system access and permissions
- ComfyUI integration points
- Python dependency availability
Each diagnostic method returns a standardized result dictionary with:
- status: 'ok', 'warning', or 'error'
- message: Descriptive message (for warnings/errors)
- Additional data specific to the diagnostic
"""
def __init__(self, db_path: str = None):
"""Initialize the diagnostics system.
Args:
db_path: Path to the SQLite database file to diagnose.
If None, the library's database (see
``utils.library_location``).
"""
if db_path is None:
db_path = self._resolve_db_path()
self.db_path = db_path
self.logger = get_logger("prompt_manager.diagnostics")
@staticmethod
def _resolve_db_path() -> str:
"""The library's database: the same resolver as PromptDatabase."""
try:
from ..database.operations import resolve_library_location
except ImportError:
from database.operations import resolve_library_location
return resolve_library_location().db_path
def run_full_diagnostic(self) -> Dict[str, Any]:
"""Run a complete diagnostic check.
Executes all diagnostic checks in sequence and provides a comprehensive
report of system status. Logs detailed information during the process.
Returns:
Dictionary mapping diagnostic categories to their results:
- database: Database connectivity and structure check
- images_table: Image tracking table specific check
- file_system: File system access and permissions check
- comfyui_output: ComfyUI output directory detection
- dependencies: Python dependency availability check
"""
self.logger.info("\n" + "=" * 60)
self.logger.info("[DIAG] PROMPTMANAGER GALLERY DIAGNOSTICS")
self.logger.info("=" * 60)
results = {
"database": self.check_database(),
"images_table": self.check_images_table(),
"file_system": self.check_file_system(),
"comfyui_output": self.check_comfyui_output(),
"dependencies": self.check_dependencies(),
}
self.logger.info("\n" + "=" * 60)
self.logger.info("[SUMMARY] DIAGNOSTIC SUMMARY")
self.logger.info("=" * 60)
for category, result in results.items():
status = "[PASS] PASS" if result["status"] == "ok" else "[FAIL] FAIL"
self.logger.info(f"{category.upper():<20} {status}")
if result["status"] != "ok":
self.logger.warning(f" Issue: {result['message']}")
self.logger.info("\n" + "=" * 60)
return results
def check_database(self) -> Dict[str, Any]:
"""Check database connection and structure.
Verifies that the database file exists, is accessible, and contains
the expected prompt table structure.
Returns:
Dictionary containing:
- status: 'ok' or 'error'
- message: Error description (if status is 'error')
- prompt_count: Number of prompts in database (if successful)
- has_images_table: Whether generated_images table exists (if successful)
"""
self.logger.info("\n[DB] Checking Database...")
try:
if not os.path.exists(self.db_path):
return {
"status": "error",
"message": f"Database file not found: {self.db_path}",
}
with closing(sqlite3.connect(self.db_path)) as conn:
conn.row_factory = sqlite3.Row
# Check prompts table
cursor = conn.execute("SELECT COUNT(*) as count FROM prompts")
prompt_count = cursor.fetchone()["count"]
self.logger.info(f" [NOTE] Prompts in database: {prompt_count}")
# Check if generated_images table exists
cursor = conn.execute("""
SELECT name FROM sqlite_master
WHERE type='table' AND name='generated_images'
""")
has_images_table = cursor.fetchone() is not None
self.logger.info(f" [IMG] Images table exists: {has_images_table}")
return {
"status": "ok",
"prompt_count": prompt_count,
"has_images_table": has_images_table,
}
except Exception as e:
return {"status": "error", "message": f"Database error: {str(e)}"}
def check_images_table(self) -> Dict[str, Any]:
"""Check the generated_images table specifically.
Examines the generated_images table structure and content to verify
the gallery system can function properly.
Returns:
Dictionary containing:
- status: 'ok' or 'error'
- message: Error description (if table missing or inaccessible)
- image_count: Number of images in table (if successful)
- recent_images: List of recent image records (if successful)
"""
self.logger.info("\n[IMG] Checking Images Table...")
try:
with closing(sqlite3.connect(self.db_path)) as conn:
conn.row_factory = sqlite3.Row
# Check if table exists
cursor = conn.execute("""
SELECT name FROM sqlite_master
WHERE type='table' AND name='generated_images'
""")
if not cursor.fetchone():
return {
"status": "error",
"message": (
"generated_images table does not exist - "
"run the updated code to create it"
),
}
# Check image records
cursor = conn.execute("SELECT COUNT(*) as count FROM generated_images")
image_count = cursor.fetchone()["count"]
self.logger.info(f" [STATS] Images in database: {image_count}")
# Get recent images
cursor = conn.execute("""
SELECT gi.*, p.text
FROM generated_images gi
LEFT JOIN prompts p ON gi.prompt_id = p.id
ORDER BY gi.generation_time DESC
LIMIT 5
""")
reader = DocumentReader(conn)
recent_images = [resolve_image_row(reader, row) for row in cursor]
self.logger.info(f" [TIME] Recent images: {len(recent_images)}")
for img in recent_images:
self.logger.info(
f" - {img['filename']} -> Prompt {img['prompt_id']}"
)
return {
"status": "ok",
"image_count": image_count,
"recent_images": recent_images,
}
except Exception as e:
return {"status": "error", "message": f"Images table error: {str(e)}"}
def check_file_system(self) -> Dict[str, Any]:
"""Check file system and permissions.
Verifies that the application has appropriate file system access
for reading images and writing database files.
Returns:
Dictionary containing:
- status: 'ok' or 'error'
- message: Error description (if issues detected)
- current_dir: Current working directory path
- can_write: Whether write access is available
"""
self.logger.info("\n[DIR] Checking File System...")
try:
# Check current directory
current_dir = os.getcwd()
self.logger.info(f" [FOLDER] Current directory: {current_dir}")
# Check if we can write to current directory
test_file = "test_write.tmp"
try:
with open(test_file, "w") as f:
f.write("test")
os.remove(test_file)
can_write = True
except (IOError, OSError):
can_write = False
self.logger.info(f" [EDIT] Can write to directory: {can_write}")
return {"status": "ok", "current_dir": current_dir, "can_write": can_write}
except Exception as e:
return {"status": "error", "message": f"File system error: {str(e)}"}
def check_comfyui_output(self) -> Dict[str, Any]:
"""Check ComfyUI output directories.
Attempts to locate ComfyUI output directories where generated images
would be stored. Checks both common locations and ComfyUI's configured paths.
Returns:
Dictionary containing:
- status: 'ok' if directories found, 'warning' if none found
- message: Warning message (if no directories found)
- output_dirs: List of detected output directory paths
"""
self.logger.info("\n[STYLE] Checking ComfyUI Output...")
output_dirs = []
def _add_dir(abs_path, source):
if abs_path not in output_dirs:
output_dirs.append(abs_path)
self.logger.info(f" [DIR] {source}: {abs_path}")
try:
image_files = []
for ext in [".png", ".jpg", ".jpeg", ".webp"]:
image_files.extend(Path(abs_path).rglob(f"*{ext}"))
self.logger.info(f" [IMG] Images found: {len(image_files)}")
except Exception as e:
self.logger.error(f" [FAIL] Error scanning: {e}")
# Check user-configured gallery directories from config.json
try:
from py.config import GalleryConfig
for cfg_dir in GalleryConfig.MONITORING_DIRECTORIES:
abs_path = os.path.abspath(cfg_dir)
if os.path.exists(abs_path):
_add_dir(abs_path, "Configured directory")
except Exception as e:
self.logger.debug(f" [NOTE] Could not load gallery config: {e}")
# Check ComfyUI's own output directory (respects --output-directory)
try:
import folder_paths
comfyui_output = folder_paths.get_output_directory()
if comfyui_output and os.path.exists(comfyui_output):
_add_dir(os.path.abspath(comfyui_output), "ComfyUI output dir")
except ImportError:
self.logger.warning(" [WARN] ComfyUI folder_paths not available")
# Fallback: check common relative paths
if not output_dirs:
for dir_path in [
"output",
"../output",
"../../output",
"ComfyUI/output",
"../ComfyUI/output",
"../../ComfyUI/output",
]:
abs_path = os.path.abspath(dir_path)
if os.path.exists(abs_path):
_add_dir(abs_path, "Found output dir")
return {
"status": "ok" if output_dirs else "warning",
"message": "No output directories found" if not output_dirs else None,
"output_dirs": output_dirs,
}
def check_dependencies(self) -> Dict[str, Any]:
"""Check required dependencies.
Verifies that all required Python packages are available for the
PromptManager system to function properly.
Returns:
Dictionary containing:
- status: 'ok' if all dependencies available, 'error' if any missing
- message: Error description (if dependencies missing)
- dependencies: Dictionary mapping package names to availability status
"""
self.logger.info("\n[PKG] Checking Dependencies...")
dependencies = {"watchdog": False, "PIL": False, "sqlite3": False}
# Check watchdog
try:
import watchdog
dependencies["watchdog"] = True
version = getattr(watchdog, "__version__", "unknown version")
self.logger.info(f" [PASS] watchdog: {version}")
except ImportError:
self.logger.error(" [FAIL] watchdog: NOT INSTALLED")
# Check PIL
try:
from PIL import Image # noqa: F401 - availability probe
dependencies["PIL"] = True
self.logger.info(" [PASS] PIL (Pillow): Available")
except ImportError:
self.logger.error(" [FAIL] PIL (Pillow): NOT AVAILABLE")
# Check sqlite3
try:
import sqlite3
dependencies["sqlite3"] = True
self.logger.info(f" [PASS] sqlite3: {sqlite3.sqlite_version}")
except ImportError:
self.logger.error(" [FAIL] sqlite3: NOT AVAILABLE")
all_deps_ok = all(dependencies.values())
return {
"status": "ok" if all_deps_ok else "error",
"message": "Missing dependencies" if not all_deps_ok else None,
"dependencies": dependencies,
}
def create_test_image_link(
self, prompt_id: int, test_image_path: str = None
) -> Dict[str, Any]:
"""Create a test image link to verify the system works.
Creates a test entry in the generated_images table to verify that
the image linking functionality is working correctly.
Args:
prompt_id: ID of an existing prompt to link the test image to
test_image_path: Optional path for the test image (uses fake path if None)
Returns:
Dictionary containing:
- status: 'ok' if test link created successfully, 'error' otherwise
- message: Success or error message
- image_id: ID of created test image record (if successful)
"""
self.logger.info(f"\n[TEST] Creating test image link for prompt {prompt_id}...")
try:
# Import database operations
import sys
import os
sys.path.insert(
0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
from database.operations import PromptDatabase
db = PromptDatabase()
# Create a fake image path if none provided
if not test_image_path:
test_image_path = "/fake/test/image.png"
# Create test metadata
test_metadata = {
"file_info": {
"size": 1024000,
"dimensions": [512, 512],
"format": "PNG",
},
"workflow": {"test": True},
"prompt": {"test_prompt": "This is a test"},
}
# Link the test image
image_id = db.link_image_to_prompt(
prompt_id=str(prompt_id),
image_path=test_image_path,
metadata=test_metadata,
)
self.logger.info(f" [PASS] Test image linked with ID: {image_id}")
return {
"status": "ok",
"image_id": image_id,
"message": f"Test image linked successfully with ID {image_id}",
}
except Exception as e:
return {
"status": "error",
"message": f"Failed to create test link: {str(e)}",
}
def run_diagnostics() -> Dict[str, Any]:
"""Run diagnostics from command line.
Convenience function to create a GalleryDiagnostics instance and run
the full diagnostic suite with default settings.
Returns:
Complete diagnostic results dictionary
"""
diagnostics = GalleryDiagnostics()
return diagnostics.run_full_diagnostic()
if __name__ == "__main__":
run_diagnostics()