* fix(lora): read LoraManager's settings and storage layout the way it does (#355) Add py/lora_manager_layout.py: the settings file in LoraManager's lookup order (environment folder, portable plugin file, user config folder), only the seven keys the import needs, the LoRA roots, the centralized sidecar root, the example-images root and the active library, each folder through one guard. Sidecar discovery for both storage modes and the local preview and example-image lookup live there too. The test runner and the browser test server point LoraManager's settings lookup at a disposable folder, so no test reads the real settings. * fix(lora): import the sidecars and local images LoraManager really uses (#355) The import, the legacy scan, the trigger-word cache and the image server now share LoraManager's layout: sidecars beside the models (alongside) or under the sidecar root (centralized), kept only when their model exists inside a LoRA root. Each LoRA links its LoraManager preview and example images where they are; CivitAI downloads run only for a LoRA without any local image. The review adds skipped, with_local_media and the layout, and offers no token when LoraManager's settings cannot be used; the legacy scan reports the same reason without replacing anything. find_lora_directories becomes default_lora_roots (no scan of the LoraManager install); the preview and example-folder helpers it replaced are removed. The trigger cache never raises into a running node. * fix(lora): never serve a LoraManager folder that holds the home folder (#355) The folder guard also rejects ancestors of the home folder (/home, C:\Users), compared in the realpath + normcase form, so a LoraManager setting cannot widen the image server's allow-list to every user's files. * fix(lora): show linked LoraManager videos as videos (#355) _enrich_images marks every image without is_video by its extension, so an .mp4 preview outside the output folders shows the Video preview label and plays in the inspector. Tests cover serving the sidecar and example-image folders only while the integration is on, and a filesystem root as the sidecar folder adding nothing. * feat(lora): the import review names the LoraManager settings it read (#355) The review gains a first row, LoraManager ("<source> settings · <mode> storage · library <name>", the settings file as its hover title), and its Found row counts metadata files, LoRAs and LoRAs with local images. A plan whose LoraManager settings cannot be used takes the close-only path: "LoraManager settings: not usable: <reason>. Nothing was changed." The browser fixture writes each LoRA's model file and can leave out the local preview, so the download, gate and cancel specs still exercise CivitAI. * docs(lora): describe how the import finds LoraManager's files (#355) * test(lora): set the settings folder relative to the process folder (#355) os.path.relpath fails across Windows drives (the temp folder on C:, the checkout on D:); the test now changes into its temp folder instead. * fix(lora): review findings for the LoraManager layout (#355) - The example-images library subfolder goes through the folder guard like its root, so its canonical (normcased) form matches the media check on Windows for a library name with capitals. - A failed trigger-word load is retried by a running node only after TRIGGER_RETRY_SECONDS, not on every prompt run. - A review whose discovery cannot list a folder says so (folder-unreadable, the folder's name only) instead of a generic error. * test(images): hold the listing clock in the deferred-walk test test_a_deferred_walk_waits_at_least_half_a_second set the previous walk 29.99 s back against a 30 s interval and then read the real clock again: a runner slower than 10 ms between the two lines passed the interval and the walk started at once (failed on macOS 3.13 and Windows 3.11). The module's clock is now held for the mark, so the deferred walk is always the one measured. * test(lora): write the unkept-settings fixture as raw text CodeQL read the dict with a credential-named key flowing into write_text as clear-text storage of a secret. The value is a marker, so the fixture now writes the JSON text directly and names the value MARKER. * fix(lora): link each LoRA's cover as its newest image (#355) Workspace lists show a prompt's media newest link first, so the import links a record's images last to first: the card shows the cover (a still in front of a leading video) the import put first. * fix(lora): show LoRA-linked media on the Library page and in the inspector (#355) Library previews, the positive detail and the per-prompt images page list a prompt's linked media under the output folders plus, while the LoRA integration is on, LoraManager's guarded folders and the download cache (_media_roots; output root indices keep their meaning). That media keeps its rooted identity, so Info and the media viewer open it; the path-based serve route answers for those roots only with a file the database records (either path spelling), never serves or makes a preview there, and output requests never read LoraManager's layout. Such media is marked linked_only: no preview file, no favorite star, no Tag action. Show in folder accepts those roots; the download cache is named, not created. * fix(lora): expose only linked files under LoraManager's folders (#355) Serve, inspect and Show in folder share one gate for a LoRA-linked root (_media_roots past the output folders): a file the database does not record as a linked image is refused with 403, checked by its given and its real path. Before, inspect and reveal read any file under those folders by path. * fix(lora): check the file Show in folder would open under a LoRA root (#355) reveal_image resolved the requested path through the client-path resolver, which can fall back to a path anchor; under a LoRA-linked root the file that would open now passes the same recorded-media gate, so only a linked file is ever revealed there.
372 lines
15 KiB
Python
372 lines
15 KiB
Python
"""
|
|
PromptManagerBase: Shared logic for PromptManager node variants.
|
|
|
|
Provides database initialization, prompt saving, hashing, tag parsing,
|
|
search, gallery system management, and cleanup — extracted from the
|
|
duplicate code in prompt_manager.py and prompt_manager_text.py.
|
|
"""
|
|
|
|
import os
|
|
from typing import Any, Dict, List, Optional
|
|
|
|
try:
|
|
from .utils.logging_config import get_logger
|
|
except ImportError:
|
|
import sys
|
|
|
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
from utils.logging_config import get_logger
|
|
|
|
try:
|
|
from .database.operations import PromptDatabase
|
|
from .utils.comfyui_integration import get_comfyui_integration
|
|
from .utils.image_monitor import get_image_monitor
|
|
from .utils.prompt_tracker import get_prompt_tracker
|
|
from .utils.hashing import generate_prompt_hash
|
|
from .utils.prompt_graph import resolve_text, run_prompt_nodes
|
|
from .utils.run_keys import executing_run_id, prompt_node_key
|
|
from .utils.usage_tracking import PENDING_FIRST_USE
|
|
except ImportError:
|
|
import sys
|
|
|
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
from database.operations import PromptDatabase
|
|
from utils.comfyui_integration import get_comfyui_integration
|
|
from utils.image_monitor import get_image_monitor
|
|
from utils.prompt_tracker import get_prompt_tracker
|
|
from utils.hashing import generate_prompt_hash
|
|
from utils.prompt_graph import resolve_text, run_prompt_nodes
|
|
from utils.run_keys import executing_run_id, prompt_node_key
|
|
from utils.usage_tracking import PENDING_FIRST_USE
|
|
|
|
|
|
class PromptManagerBase:
|
|
"""Mixin providing shared prompt management logic for ComfyUI nodes.
|
|
|
|
Handles database connection, prompt saving with deduplication,
|
|
tag parsing, search, gallery system lifecycle, and cleanup.
|
|
Subclasses only need to define ComfyUI-specific class attributes
|
|
(INPUT_TYPES, RETURN_TYPES, FUNCTION) and their execution method.
|
|
"""
|
|
|
|
def __init__(self, logger_name: str = "prompt_manager.node"):
|
|
self.logger = get_logger(logger_name)
|
|
self.logger.debug(f"Initializing {self.__class__.__name__} node")
|
|
|
|
self.db = PromptDatabase()
|
|
self.prompt_tracker = get_prompt_tracker(self.db)
|
|
self.image_monitor = get_image_monitor(self.db, self.prompt_tracker)
|
|
self.comfyui_integration = get_comfyui_integration()
|
|
|
|
self._start_gallery_system()
|
|
self.logger.debug(f"{self.__class__.__name__} node initialization completed")
|
|
|
|
def _save_prompt_to_database(
|
|
self,
|
|
text: str,
|
|
category: Optional[str] = None,
|
|
tags: Optional[list] = None,
|
|
count_run: bool = False,
|
|
) -> Optional[int]:
|
|
"""Save the prompt to the SQLite database.
|
|
|
|
Runs are normally counted by the queue hook (utils/usage_tracking.py), so a
|
|
save only counts one when asked to (batch items whose text the hook could not
|
|
see) or when the hook queued this prompt before it existed.
|
|
|
|
Args:
|
|
text: The prompt text
|
|
category: Optional category
|
|
tags: List of tags
|
|
count_run: Count this execution as a run of the prompt
|
|
|
|
Returns:
|
|
The prompt ID if saved successfully, None otherwise
|
|
"""
|
|
try:
|
|
prompt_hash = self._generate_hash(text)
|
|
self.logger.debug(f"Generated hash for prompt: {prompt_hash[:16]}...")
|
|
|
|
existing = self.db.get_prompt_by_hash(prompt_hash)
|
|
if existing:
|
|
self.logger.info(
|
|
f"Found existing prompt with ID {existing['id']}, updating metadata"
|
|
)
|
|
if any([category, tags]):
|
|
self.db.update_prompt_metadata(
|
|
prompt_id=existing["id"], category=category, tags=tags
|
|
)
|
|
self.logger.debug("Updated metadata for existing prompt")
|
|
if count_run:
|
|
self._record_use(existing["id"])
|
|
return existing["id"]
|
|
|
|
self.logger.debug(
|
|
f"Saving new prompt with category: {category}, tags: {tags}"
|
|
)
|
|
prompt_id = self.db.save_prompt(
|
|
text=text, category=category, tags=tags, prompt_hash=prompt_hash
|
|
)
|
|
|
|
if prompt_id:
|
|
self.logger.debug(f"Successfully saved new prompt with ID: {prompt_id}")
|
|
pending = getattr(self, "pending_first_use", PENDING_FIRST_USE)
|
|
runs = pending.consume(prompt_hash) + (1 if count_run else 0)
|
|
if runs:
|
|
self._record_use(prompt_id, times=runs)
|
|
else:
|
|
self.logger.warning("Failed to save prompt - no ID returned")
|
|
|
|
return prompt_id
|
|
|
|
except Exception as e:
|
|
self.logger.error(f"Error saving prompt to database: {e}")
|
|
return None
|
|
|
|
def _prompt_role(
|
|
self, prompt_graph: Any, unique_id: Any, text: str
|
|
) -> Dict[str, bool]:
|
|
"""How this node's execution relates to the run, from the queued graph.
|
|
|
|
Returns flags:
|
|
is_run_prompt: feeds a sampler's positive input (images belong to it)
|
|
seen_at_queue: the queue hook resolved exactly this text, so it already
|
|
counted the run and the image's metadata can identify the prompt
|
|
known: the graph and node id were available
|
|
"""
|
|
if not isinstance(prompt_graph, dict) or unique_id is None:
|
|
return {"is_run_prompt": True, "seen_at_queue": False, "known": False}
|
|
node_id = str(unique_id)
|
|
resolved = resolve_text(prompt_graph, node_id)
|
|
return {
|
|
"is_run_prompt": node_id in run_prompt_nodes(prompt_graph),
|
|
"seen_at_queue": resolved is not None
|
|
and generate_prompt_hash(resolved) == generate_prompt_hash(text),
|
|
"known": True,
|
|
}
|
|
|
|
def _track_prompt_execution(
|
|
self,
|
|
text: str,
|
|
encoding_text: str,
|
|
category: Optional[str],
|
|
tags: Optional[list],
|
|
additional_data: Dict[str, Any],
|
|
prompt_graph: Any = None,
|
|
unique_id: Any = None,
|
|
) -> Optional[int]:
|
|
"""Save the prompt and, for the run's positive prompt, register it for linking.
|
|
|
|
Negative-prompt nodes are saved but never become the prompt their images are
|
|
linked to. Text the queue hook could resolve is counted there and linked from
|
|
the image's own metadata; anything else (PromptSearchList batches, unknown
|
|
text nodes, or a resolution that doesn't match) is counted here and pushed
|
|
onto the FIFO queue. Without graph context the pre-3.2.4 behaviour is kept.
|
|
"""
|
|
role = self._prompt_role(prompt_graph, unique_id, text)
|
|
batch_item = (
|
|
role["known"] and role["is_run_prompt"] and not role["seen_at_queue"]
|
|
)
|
|
prompt_id = self._save_prompt_to_database(
|
|
text=text, category=category, tags=tags, count_run=batch_item
|
|
)
|
|
if role["is_run_prompt"] and not prompt_id:
|
|
# The save failed: an older entry of this key must not stand in (#342)
|
|
self._mark_untracked(prompt_graph, unique_id)
|
|
elif role["is_run_prompt"]:
|
|
self.prompt_tracker.set_current_prompt(
|
|
prompt_text=encoding_text,
|
|
additional_data={**additional_data, "prompt_id": prompt_id},
|
|
push_to_queue=batch_item or not role["known"],
|
|
**self._run_identity(prompt_graph, unique_id),
|
|
)
|
|
return prompt_id
|
|
|
|
@staticmethod
|
|
def _run_identity(prompt_graph: Any, unique_id: Any) -> Dict[str, Any]:
|
|
"""The tracker's proof that an image belongs to this execution (#342)."""
|
|
return {
|
|
"run_key": prompt_node_key(prompt_graph, unique_id),
|
|
"run_id": executing_run_id(),
|
|
}
|
|
|
|
def _skip_empty_prompt(self, prompt_graph: Any, unique_id: Any) -> None:
|
|
"""The ``text`` editor is empty: save and track nothing (#342).
|
|
|
|
Prepend and append text are deliberately never stored as the prompt, so
|
|
an empty editor has no prompt to save, count or link. For the run's
|
|
positive node this is logged at info and the node's run key is marked
|
|
empty (``_mark_untracked``).
|
|
"""
|
|
self._mark_untracked(
|
|
prompt_graph,
|
|
unique_id,
|
|
"Prompt text is empty (prepend and append text are not stored): "
|
|
"nothing saved or tracked, and this run's images stay unlinked",
|
|
)
|
|
|
|
def _mark_untracked(
|
|
self, prompt_graph: Any, unique_id: Any, message: Optional[str] = None
|
|
) -> None:
|
|
"""This execution of a positive node has no prompt to link (#342).
|
|
|
|
An empty editor, or a save or tracking failure. The node's run key is
|
|
marked with a placeholder, so no earlier prompt of that key can be
|
|
linked to this run's images, and a batch item keeps its queue place.
|
|
``message`` is logged at info for a positive node.
|
|
"""
|
|
try:
|
|
role = self._prompt_role(prompt_graph, unique_id, "")
|
|
if not role["is_run_prompt"]:
|
|
if message:
|
|
self.logger.debug("Empty text on a non-positive node; not saved")
|
|
return
|
|
if message:
|
|
self.logger.info(message)
|
|
self.prompt_tracker.mark_empty(
|
|
push_to_queue=role["known"],
|
|
**self._run_identity(prompt_graph, unique_id),
|
|
)
|
|
except Exception as error:
|
|
# Tracking must never block generation
|
|
self.logger.warning(f"Could not mark the untracked prompt: {error}")
|
|
|
|
def _record_use(self, prompt_id: int, times: int = 1) -> None:
|
|
"""Count runs; a tracking failure must never block generation."""
|
|
try:
|
|
self.db.record_prompt_use(prompt_id, times=times)
|
|
except Exception as e:
|
|
self.logger.warning(f"Could not record use of prompt {prompt_id}: {e}")
|
|
|
|
def _inject_lora_trigger_words(self, text: str) -> str:
|
|
"""Append LoRA trigger words if integration is enabled.
|
|
|
|
Returns the text unchanged if the integration is disabled or
|
|
LoraManager is not installed.
|
|
"""
|
|
try:
|
|
from .py.config import IntegrationConfig
|
|
except ImportError:
|
|
try:
|
|
from py.config import IntegrationConfig
|
|
except ImportError:
|
|
return text
|
|
|
|
if (
|
|
not IntegrationConfig.LORA_MANAGER_ENABLED
|
|
or not IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED
|
|
):
|
|
return text
|
|
|
|
try:
|
|
from .py.lora_utils import get_trigger_cache, inject_trigger_words
|
|
except ImportError:
|
|
try:
|
|
from py.lora_utils import get_trigger_cache, inject_trigger_words
|
|
except ImportError:
|
|
return text
|
|
|
|
cache = get_trigger_cache()
|
|
|
|
# Lazy-load cache on first use; a failed load waits before the next try
|
|
# (#355: never a rescan of LoraManager's folders on every run).
|
|
retry_due = getattr(cache, "retry_due", lambda: True)
|
|
if not cache.is_loaded and IntegrationConfig.LORA_MANAGER_PATH and retry_due():
|
|
cache.load(IntegrationConfig.LORA_MANAGER_PATH)
|
|
|
|
modified, injected = inject_trigger_words(text, cache)
|
|
if injected:
|
|
self.logger.info(f"Injected trigger words: {', '.join(injected)}")
|
|
return modified
|
|
|
|
def _generate_hash(self, text: str) -> str:
|
|
"""The prompt's deduplication hash (see utils.hashing.generate_prompt_hash)."""
|
|
return generate_prompt_hash(text)
|
|
|
|
def _parse_tags(self, tags_string: str) -> Optional[list]:
|
|
"""Parse comma-separated tags string into a list.
|
|
|
|
Args:
|
|
tags_string: Comma-separated string of tags
|
|
|
|
Returns:
|
|
List of parsed tags, or None if no valid tags found
|
|
"""
|
|
if not tags_string or not tags_string.strip():
|
|
return None
|
|
|
|
tags = [tag.strip() for tag in tags_string.split(",") if tag.strip()]
|
|
return tags if tags else None
|
|
|
|
def _search_prompts(self, search_text: str = "") -> List[Dict[str, Any]]:
|
|
"""Search for past prompts by text content.
|
|
|
|
Args:
|
|
search_text: Text to search for in prompt database
|
|
|
|
Returns:
|
|
List of matching prompt dictionaries with metadata
|
|
"""
|
|
try:
|
|
if not search_text or not search_text.strip():
|
|
return []
|
|
|
|
try:
|
|
from .py.config import PromptManagerConfig
|
|
|
|
max_results = PromptManagerConfig.MAX_SEARCH_RESULTS
|
|
except Exception:
|
|
max_results = 100
|
|
|
|
results = self.db.search_prompts(
|
|
text=search_text.strip(),
|
|
category=None,
|
|
tags=None,
|
|
rating_min=None,
|
|
limit=max_results,
|
|
)
|
|
|
|
return results
|
|
|
|
except Exception as e:
|
|
self.logger.error(f"Error searching prompts: {e}")
|
|
return []
|
|
|
|
def search_prompts_api(self, search_text: str = "") -> List[Dict[str, Any]]:
|
|
"""API method for JavaScript UI to search prompts."""
|
|
return self._search_prompts(search_text=search_text)
|
|
|
|
def get_recent_prompts_api(self, limit: int = 20) -> List[Dict[str, Any]]:
|
|
"""API method for JavaScript UI to get recent prompts."""
|
|
try:
|
|
return self.db.get_recent_prompts(limit=limit)
|
|
except Exception as e:
|
|
self.logger.error(f"Error getting recent prompts: {e}")
|
|
return []
|
|
|
|
def _start_gallery_system(self):
|
|
"""Initialize and start the gallery monitoring system."""
|
|
try:
|
|
self.logger.debug("Starting gallery system...")
|
|
self.image_monitor.start_monitoring()
|
|
self.logger.debug("Gallery system started successfully")
|
|
except Exception as e:
|
|
self.logger.error(f"Failed to start gallery system: {e}")
|
|
self.logger.warning("Gallery features will be disabled")
|
|
|
|
def get_gallery_status(self) -> Dict[str, Any]:
|
|
"""Get status of the gallery system."""
|
|
return {
|
|
"image_monitor": self.image_monitor.get_status(),
|
|
"prompt_tracker": self.prompt_tracker.get_status(),
|
|
}
|
|
|
|
def cleanup_gallery_system(self):
|
|
"""Clean up gallery system resources.
|
|
|
|
Note: The image monitor is a singleton shared across all node instances,
|
|
so we intentionally do NOT stop it here. It should run for the entire
|
|
ComfyUI session. Stopping it would kill monitoring for all nodes.
|
|
"""
|
|
self.logger.debug("Gallery system cleaned up (monitor continues running)")
|