Second CI run on Windows:
- gallery roots are stored resolved and case-folded there; the admin API
tests now compare against that form instead of the mixed-case temp path
- the subfolder fallback for images outside every root also uses "/", and
the test expects it
- a relative diagnostics db path is normalised after joining
- "database is locked" under many writers on a slow disk: connections now
wait up to 30 seconds (connect timeout and busy_timeout), long enough for
a scan or thumbnail run while ComfyUI saves images
The first push of the branch ran the Tests matrix on all three operating
systems. Every Python job failed; the causes were:
- numpy missing in CI: the WD14 engine tests import it (requirements-dev)
- macOS: temp directories resolve through /private; the admin API tests
now use the real path like the gallery roots they compare against
- Windows: the startup banner printed an emoji, which raises
UnicodeEncodeError on a legacy-code-page console and aborted loading
the whole node; the banner is ASCII now and a test loads the package
under a cp1252 stdout
- Windows: gallery subfolders came back with backslashes, which neither
the folder tree nor the search filter understand; they are "/" joined
on every platform. A relative DEFAULT_DB_PATH kept its forward slashes
after being joined to the extension root; it is normalised now
- Windows: sqlite connections opened with `with sqlite3.connect()` are
not closed by the context manager and linger until garbage collection,
which holds a file lock there. Production sites (vacuum, diagnostics,
restore inspection) close explicitly; a restore that swaps the API's
database object closes the previous one; PromptModel keeps a registry
so a shutdown or a test can close every instance
- Windows: served files were still open when their temp directory was
removed (tests now read the body), a log download test compared text
mode newlines, two tests compared paths with a hard-coded separator
tests/open_handles.py makes the Linux run fail the way Windows would:
every database test base checks, right before deleting its files, that
nothing in the process still holds them open.
The output scan handed every .mp4 to Pillow and logged one ERROR per file.
ComfyUI's video savers store the queued graph as JSON in the container's
comment tag, so those files carry prompts worth linking.
- utils/video_metadata.py reads the comment/description tag through
ffprobe with a timeout, normalised to the PNG text-chunk shape so the
existing parser handles both; without ffprobe videos are skipped and a
single INFO notice is logged
- _extract_comfyui_metadata routes videos there and logs unreadable images
as warnings, since the scan carries on past them
Test hygiene: every aiohttp test base class overrode tearDownAsync without
calling super(), so no test client was ever closed. That produced thousands
of ResourceWarnings, "Unclosed client session" errors from asyncio that
broke log assertions, and doubled the suite's run time.
Both jobs processed one file at a time on a single core. Pillow decodes the
whole image to reach trailing PNG text chunks and releases the GIL while it
does, so a bounded thread pool scales almost linearly: on a 1 GB sample of
400 PNGs the metadata pass dropped from 18 s to 2.5 s at 8 threads, and 120
thumbnails from 4.6 s to 0.6 s.
- utils/parallel.py: map_parallel(func, items, workers) keeps order, runs on
the caller's thread when workers is 1, and caps the pool at the item count
- PromptManagerConfig.WORKER_THREADS defaults to half the detected cores,
is clamped to max_worker_threads() when loaded from config.json, and is
persisted under "performance"
- settings API returns worker_threads and cpu_count and validates saves;
the settings modal gets a "Worker threads" slider capped at the core count
- the scan reads each metadata batch on the configured threads; thumbnail
generation processes batches of files per executor job, still emitting
one progress event per file
Threads rather than processes: this runs inside ComfyUI's server, where
forking a CUDA-initialised process is unsafe and the spawn start method
would re-import ComfyUI's main module in every child.
ImageGenerationHandler.shutdown() clears the pending set, sets a stop
event the worker's delay and settle waits listen to, and puts a None
sentinel on the queue; the worker exits within the current wait. Each
worker owns its queue and stop event so a draining worker never swallows
an image scheduled after shutdown. ImageMonitor.stop_monitoring calls it.
A root written straight into config.json skipped the containment check
the settings endpoint applies. GalleryConfig.update_config (and so
load_from_file) now drops entries that fail validate_gallery_root with a
warning, _find_comfyui_output_dir validates the configured directory
before using it, and ImageMonitor.start_monitoring validates every
directory before scheduling the observer.
The duplicate cleaner only deletes files with a media extension and
never anything under a thumbnails directory. _collect_media_files walks
with os.walk(followlinks=False), skips symlinked files and directories,
prunes thumbnails, stops MAX_SCAN_DEPTH (12) levels down and after
MAX_SCAN_FILES (50000) files.
A per-path visited set alone makes a chain of diamonds (node i reads node
i+1 on both inputs) exponential in depth. Each node's text is now
memoised so it is expanded once, every expansion is charged to the shared
MAX_VISITS budget, chains deeper than MAX_DEPTH (64) are unknowable so
recursion stays bounded, and text longer than MAX_TEXT_LENGTH (100k) is
unknowable so StringConcatenate doubling cannot exhaust memory.
The cycle guard used one visited set for the whole walk, so a string node
read twice (a diamond, or StringConcatenate with the same source on both
inputs) made the second branch None and the prompt text unknowable. Cycle
detection is now scoped to the current path; a separate visit counter
keeps MAX_VISITS as the bound on total work.
Re-wrap module docstrings, comments and f-string log messages; remove the
unused typing.List and PngInfo imports; mark the PIL availability probe in
diagnostics with noqa: F401; turn two placeholder-less f-strings into plain
strings; strip trailing whitespace inside the diagnostics SQL.
get_recent_images and search_images_by_prompt take limit/offset so the recent
and search image routes no longer over-fetch a 5000-row window and slice it.
generate_content_hash, PromptModel.migrate_database and find_duplicates had no
callers outside their own tests. The isolated logger tests re-apply their
thread filter after update_config rebuilds the handlers. Long lines in the
touched database and image modules are wrapped so the full flake8 hook passes.
truncate_logs emptied prompt_manager.log through a second open(), leaving the
rotating handler's stream at its old offset: the next record was written past
the new end (zero padding on POSIX) and tell() still reported the old size, so
the file rolled over and resurrected the backup that had just been deleted.
It also logged once per file from inside the deletion loop, which could do the
same with a small size limit. The main log is now truncated through the
handler's own stream and a single summary line is logged afterwards.
check_dependencies read watchdog.__version__ for its log line; recent
watchdog releases have no such attribute, so the AttributeError escaped and
took the whole diagnostics run down. Read it with a default.
Also covers utils/diagnostics.py: database and images-table checks against
a real temporary database, file system check, ComfyUI output detection
(folder_paths, configured directories, relative fallback, nothing found),
dependencies, the full run and the test image link.
get_recent_prompts returns {"prompts": [...]} but _get_fallback_prompt
indexed it like a list, so the last-resort link to the most recent prompt
always failed silently. Accept both shapes.
utils.hashing.generate_prompt_hash is now the only prompt hash:
PromptManagerBase._generate_hash delegates to it and the inline
hashlib.sha256 copies in the prompt tracker and image monitor are gone.
Remove PromptManagerBase._open_web_interface (it targeted a web_interface
folder that never existed) and the unused restart_gallery.py script.
IS_CHANGED on PromptManager and PromptManagerText now includes category and
tags, so editing either re-runs the node and updates the saved metadata.
start_monitoring assigned self.observer before checking that any directory
existed, so with no valid directory the monitor was stuck "already running"
and stop_monitoring raised on join of a never-started observer. Directories
are validated first and only a started observer is kept; stop_monitoring is
safe to call at any time and always joins. New is_monitoring property.
The per-image threading.Timer is replaced by one daemon worker: paths are
deduplicated and bounded (MAX_PENDING_IMAGES), processed in creation order
(batch images pop the prompt queue in order) after the processing delay and
once the file size is unchanged across two reads.
The metadata strategy returned None for images whose embedded workflow has
no positive PromptManager node, so the handler popped the batch FIFO for
them and every later batch image linked to the wrong prompt. It now returns
a tri-state: linked (prompt found), foreign (no PromptManager prompt in the
workflow, so the image is skipped and no fallback applies) or unknown (only
then the queue, snapshot, tracker and recent-prompt fallbacks run).
The on-prompt hook runs before ComfyUI validates the request, so counting
there bumped run_count for prompts that were rejected or deleted from the
queue. The hook now only resolves the positive PromptManager texts, mints a
UUID4 prompt_id when the client sent none, and stashes the hashes under that
id (QueuedRuns, bounded to 1000 entries). PromptQueue.get is wrapped
idempotently; when a prompt is dequeued for execution its stashed runs are
counted (or remembered as pending first use). Servers without a prompt_queue
fall back to counting at submit time with one warning.
__init__ defines _global_db before the monitoring block and passes a factory
that raises RuntimeError while the database is unavailable; the package is
now imported under test with server and folder_paths stubbed.
_TEXT_KEY matched input names such as text_negative, so a styler node feeding
a sampler's positive input pulled its negative PromptManager node into the
positive trace. Input names matching (^|_)neg(ative)?($|_) are now skipped
(new follows_input helper).
run_prompt_nodes fell back to every PromptManager node whenever no positive
node was traced, including graphs that do have a sampler and whose only
PromptManager node feeds its negative input. Fall back only when the graph
has no recognisable sampler (new has_sampler helper). Samplers named like
one are recognised even without a model link. Adversarial graphs (cycles,
10k-node fan-out, non-dict inputs) are covered.
* chore: start 3.2.4
* feat: track prompt usage (run_count, last_used_at) with server-side sorting
- Migration adds last_used_at and run_count; existing prompts are estimated
from linked images (count, newest image time, else created_at). The
backfill only touches NULL rows, so it self-heals rows written after a
downgrade or an interrupted upgrade.
- Every node run records a use, including re-runs of an existing prompt;
a tracking failure is logged and never blocks generation.
- search_prompts/get_recent_prompts accept a whitelisted sort key
(last_used, created, run_count, rating, text); default stays newest first.
Refs #90
* feat: sort param on /recent and /search; microsecond last_used_at
- /prompt_manager/recent and /search accept ?sort= (whitelisted in the DB
layer; unknown values fall back to newest first).
- last_used_at is written with microseconds from Python so a re-run in the
same millisecond as another save still sorts first; ordering stays correct
against millisecond values from the backfill.
Refs #90
* feat: server-side sort in admin UI with Recently Used / Most Used and run badges
- New web/js/prompt-list-sort.js owns the sort options, validation and
request URLs; admin.js fills the dropdown from it and re-fetches from the
server on change, so every sort spans all pages (the old client-side sort
only reordered the current page).
- Default sort is Recently Used; prompt cards show a run-count badge with
the last-used time on hover.
- The node's Recent button now lists recently used prompts.
- Contract test keeps the UI sort keys and the node's key in sync with the
server whitelist.
Refs #90
* fix: link images and count runs by the positive prompt, not node execution
With a negative-prompt PromptManager node in the workflow, every executed
PromptManager node pushed onto the image-linking queue, so images could be
linked to the negative prompt or to a previous run. ComfyUI also skips
unchanged nodes, so re-runs of the same prompt were never counted.
- utils/prompt_graph.py finds the PromptManager nodes feeding a sampler's
positive input (KSampler, CFG/Basic guiders, ControlNet and combine
pass-throughs, PromptManagerText into encoders), falling back to all
PromptManager nodes when none can be traced.
- Runs are counted by a ComfyUI on-prompt hook when a workflow is queued;
prompts not yet saved are counted when the node first saves them.
- Nodes receive the queued graph (hidden PROMPT/UNIQUE_ID). Only the run's
positive prompt becomes the current prompt; only batch items with linked
text are pushed onto the FIFO queue.
- The image monitor links from the image's own metadata first, using the
positive node, and falls back to the queue for batch items.
Refs #90
* fix: address review — count cached runs of linked text, keep repeated pending runs
- Text from known pure string nodes (core PrimitiveString/StringConcatenate,
WAS Text Concatenate/Multiline, pysssss ShowText) is resolved at queue
time, so cached re-runs of such prompts are counted and their images link
from metadata. Unknown nodes are never guessed; the node compares its real
text with the queue-time resolution and falls back to counting and
queueing itself on any mismatch.
- A new prompt queued several times before its first run keeps every
pending run; the first save records all of them.
- Registration flag is a threading.Event; consistent unittest imports;
logging instead of print in the init fallback.
* refactor: unquote PendingFirstUse annotation so Tuple usage is visible
* fix: release 3.2.3 — metadata parsing, null-safe saves, auto-hide, tag search
- fix(#75): stop SaveImage patch from writing the last-run prompt into every
PromptManager node (negative prompts were saved as copies of the positive)
- fix(#75): replace four duplicated metadata parsers with a shared
web/js/comfy-metadata.js that traces sampler links (KSampler, CFGGuider /
SamplerCustomAdvanced, LoRA chains, ControlNet pass-through, any checkpoint
loader) and recovers per-node text for images saved by older versions
- fix(#143): treat null JSON body fields as empty in prompt API handlers
- fix(#149): keep auto-hide timeout 0 (disabled) instead of falling back to
the default; validate result_timeout on save
- feat(#150): support tag:name filters in the node search box
- docs(#142): scan dialog mentions A1111/Forge metadata support
- docs(#144): fix ComfyUI-Lora-Manager links
- ci: run JS tests with node --test
* fix: bound metadata graph traversal to prevent tab freeze on crafted PNGs
Link resolution tried every linked input recursively with only a depth cap,
so a PNG whose prompt graph has cycles and fan-out could force ~inputs^25
paths and hang the gallery/metadata page. Each lookup now shares a visited
set (each node expanded once) with a hard cap of 500 visits.
* fix: read the matching output of multi-output settings nodes
KSampler settings linked to a node like SamplerCombo all resolved to its
first value, so steps, cfg and sampler showed the sampler name. Scalar
resolution now prefers the linked node's input with the same field name.
* fix: address review — trust API text, restore workflow-only parsing
- Workflow widget text now only replaces API text for metadata showing the
pre-3.2.3 corruption signature (2+ PromptManager nodes with identical API
text but different widgets). Otherwise the API graph wins, so expanded
dynamic prompts and linked text display what actually ran.
- Images with only a UI workflow are converted to the API graph shape and
traced the same way: prompts, KSamplerAdvanced, UNET/checkpoint loaders,
KJNodes Set/Get, Reroute, bypassed and muted nodes.
* fix(search): PromptSearchList crash, filters, and batch output (#52)
- Fix empty list crash (OUTPUT_IS_LIST requires at least one element)
- Add partial tag matching (LIKE instead of exact)
- Add preview and count outputs for batch visibility
- Add skip_multipart filter for Clip_N video prompts
- Add LoRA-only prompt filter
- Collapse newlines for StringOutputList compatibility
* feat(db): add tag_partial parameter for LIKE tag matching (#52)
Allows search_prompts to use LIKE matching for tags instead of exact.
Used by PromptSearchList for discovery-style search.
Existing callers default to exact matching (unchanged behavior).
* feat(tracker): add FIFO prompt queue for batch image linking (#52)
Batch workflows encode all prompts first, then save all images later.
The queue preserves encode order so each image links to the correct
prompt by position (pop on save matches push on encode).
* fix(monitor): multi-strategy image-to-prompt linking (#52)
Previous approach relied on live prompt tracker state, which was
always stale for batch workflows (all images linked to last prompt).
New 5-strategy priority chain:
1. FIFO queue pop (batch-correct by position)
2. Metadata-based DB lookup (extracts prompt from PNG)
3. Prompt snapshot at file-creation time
4. Live prompt tracker
5. Most recent DB prompt fallback
* test: add tests for prompt queue and search filters (#52)
- 6 queue tests: FIFO ordering, batch simulation, exhaustion
- 15 filter tests: newline collapse, Clip_ filter, LoRA-only filter
- Fix _make_tracker helper to include queue fields
* chore: bump version to 3.2.2
* fix: address code review — regex backtracking and queue expiry (#52)
- Replace backtracking-prone LoRA-only regex with subtraction approach
(re.sub removes tags, check if anything remains). Fixes CodeQL alert.
- Add expiry check to pop_next_prompt — skips stale entries to prevent
incorrect linking from accumulated queue entries across sessions.
- Cleanup thread now also prunes expired queue entries.
* fix: diagnostics, backup, and restore now respect config.json db path (#132)
* test: add regression tests for config-aware diagnostics and backup (#132)
* fix(test): use fresh event loop to avoid aiohttp test pollution
* fix: don't kill singleton image monitor when node instance is garbage collected
The __del__ method was calling stop_monitoring() on the shared singleton
ImageMonitor, killing it for ALL nodes whenever any single node instance
was garbage collected between workflow runs. This caused the monitor to
restart on every execution and miss image save events due to race conditions.
* test: add regression tests for image monitor singleton lifecycle
Ensures cleanup/del never stops the shared monitor, singleton is
thread-safe, and observer survives across node instance lifecycles.
* style: address PR review — log suppressed exceptions, remove unused imports
start_monitoring() used an absolute import (from py.config) to read
MONITORING_DIRECTORIES, which either failed with ImportError or loaded
a separate module instance with empty defaults. The configured custom
directory was never used, and auto-detection always kicked in.
Fixed by consolidating into the single relative import (from ..py.config)
that was already used for the MONITORING_ENABLED check.
- Fix AttributeError: 'str' has no attribute 'as_posix' on thumbnail_rel_path
in get_output_images and find_duplicates (thumbnail_rel_path is already a
string from f-string construction, not a Path object)
- Fix NameError: total_images undefined in generate_thumbnails (was defined as
total_media, causing progress logging and response to fail)
- Add URL encoding for paths in scan_output_dir for filenames with spaces
- Exclude thumbnails directory from image monitor to prevent linking thumbnail
files to prompts via fallback mechanism
- Bump version to 3.0.31
Add ability to configure a custom image scan directory in Settings:
- New "Image Scan Directory" field in Settings modal
- "Currently Monitoring" display shows active directory path
- Settings API now returns/saves gallery_root_path and monitored_directories
- Image monitor checks GalleryConfig before auto-detecting
- Restart notification when gallery path changes
Closes#76
- Add Google-style docstrings to main node files (prompt_manager.py, prompt_manager_text.py)
- Enhance database module docstrings with detailed parameter and return documentation
- Add comprehensive docstrings to all utils modules with usage examples
- Document API endpoints and configuration classes thoroughly
- Update module-level docstrings in all __init__.py files
- Ensure consistent naming (ComfyUI_PromptManager) across all documentation
- Follow Python best practices for docstring formatting
Vito Sansevero