19 Commits

Include renames

Author	SHA1	Message	Date
Vito	3be0ada6a1	fix(ingestion): link an image only to a prompt that provably produced it (#344) ... * fix(ingestion): link an image only to a prompt that provably produced it An LLM fed a PromptManagerPair's prepend_text while its text editor was empty. The node refused to save (an empty positive raised and logged a warning), and the image monitor then linked the image to a stale tracker entry or to the newest prompt in the database. - Nodes: an empty text editor saves and tracks nothing (prepend and append text are never the stored prompt), logs at info, and marks the node's run key empty so no earlier entry can be used for its images. - Tracker entries carry a run key: a digest of the prompt node and its whole upstream graph, the identity ComfyUI's cache uses. The monitor computes the same key from the image's embedded graph and uses a queued or snapshot entry only when the keys match (a snapshot also needs a known run with one entry, so a batch's last item is never applied to every image). - The recency fallback is gone; anything uncertain stays unlinked and is logged at info. - Recorded execution history attributes a file when exactly one run reported it, the file was written during that run, and the run's positive contexts name one stored prompt. The monitor and the output scan use it as their last certain source. - The scan and pair ingestion treat a resolved pair with an empty stored positive as unresolved instead of raising. - tools/review_image_links.py reviews selected links by these rules; dry run by default, --apply changes only named ids after a backup. Closes #342 * test(ingestion): real-host attribution case and the #342 testing contract The managed host disables the image monitor, so the owned fixture pack gains a route that starts and stops the plugin's monitor singleton for the attribution case only. The case runs in both renderers: a typed Pair image links, a Pair image whose positive is only linked prepend text stays unlinked with nothing stored, an unknown node feeding text links through its run key, and its cached re-run links again. * fix(ingestion): take a queued entry only from the run the image appeared in Dropping a key's older queue entries whenever the node ran again let a later queued run, executing during the monitor's processing delay, take the earlier image's entry. The snapshot taken when the image appears now names the key's latest run, and the queue is popped by key and run: same-key entries of earlier runs ahead of it are stale, entries of later runs wait for their own images. Also keeps the attribution case out of the stock real-host suite. * test(ingestion): host lookups degrade to no proof * fix(ingestion): address the #344 review (history re-check, keys, repair) - Images without an embedded API graph: the scan now asks recorded history before skipping a file with no prompt metadata, and the monitor gives a keyless unlinked image one history-only re-check when its run's outputs are recorded (a seal notification from the outputs write; at most 1000 waiting images, each for an hour). - Run keys: an image is unkeyable when any positive node has no key, so a two-pass graph is never linked through the other pass; links follow ComfyUI's is_link exactly. - A failed save marks the node's run key like an empty editor, so an older entry of that key can't stand in for this run. - A placeholder is the latest entry: the image stays unlinked without falling through to history, the same rule as a popped placeholder. - An index on execution_run_outputs(filename, subfolder, root_key, path_style) for the history lookup, added by migration and accepted by the output schema check (any other index is still refused). - Repair tool: unlink removes only the link record and only while the image is still linked to the prompt it was judged on; every run prints what --apply does. - Real-host case proves the re-run's PromptManager node was cached. * test(ingestion): real-host bare-output re-check and the review-round contract A metadata-disabled save, held open behind the fixture gate after SaveImage, waits unlinked while its run is open and links once the run's outputs are recorded. The fixture's monitor route lists the waiting files. TESTING.md covers the history re-check, the read index and what the repair tool's --apply does. * fix(ingestion): batch history lookups instead of adding an index An index on execution_run_outputs would make the library unopenable by older builds, whose schema check refuses any extra index there. The migration and the schema-check allowance are reverted, so the history schema is exactly main's. stored_run_prompts instead looks up a batch of files with one parameterized filename IN (...) query per chunk of at most 500 names and matches in memory; the scan computes it once per scan batch. * fix(ingestion): address the #344 re-review (seal race, linked roots) - A sealed history re-check that can't be scheduled yet (the image's own pass still pending, or the pending set full) is kept due and scheduled when a pending image finishes, instead of being dropped. - History matches a file through ComfyUI's spelling of its output folder first, then with both sides resolved, so a symlinked or junction output folder still attributes the scan's resolved paths; waiting files are keyed by their resolved path. - The scan makes its batch history lookup only when a file needs it. - set_current_prompt pushes to the queue last, so marking a failed execution never adds a second queue slot. - A keyless image outside the output folder doesn't take a waiting slot.	2026-10-25 19:03:35 -07:00
Vito	8bf759d6f8	feat(system): keep the library in ComfyUI's user folder (#318) ... * feat(system): resolve every library path in one place utils/library_location.py decides where the library lives: the PROMPT_MANAGER_DATA_DIR folder (validated: absolute, never inside a folder ComfyUI or Prompt Manager serves), an explicit database.default_path as before, a node-folder library that was not moved, else ComfyUI's user/__promptmanager. The database, diagnostics, config.json, logs and the LoRA image cache all ask it, so a new install never writes into the node folder and an existing one keeps every path until the user moves it. "Keep it here" is persisted as library.keep_in_node_folder. * feat(system): move the library as a verified Activity job POST /prompt_manager/system/library/move starts "Move library", an exclusive, non-cancellable job (database/library_move.py). It checks the target volume's free space, copies the database with SQLite's backup API from one read snapshot, verifies the copy (integrity check, every workflow document, the same rows in every table), copies the safety and pre-migration backups and config.json (owner-only, checked by size and SHA-256), then publishes under the old file's write lock and switches the live process (every PromptModel, the review and history observers, the API's plans and health report). The old prompts.db and config.json are renamed *.moved-to-<folder> only after the switch. A failure removes everything it wrote and leaves the old library in use; a marker in the new folder makes a restart after any crash pick the right library. GET /prompt_manager/system/library and the Overview's "location" say where the library lives and why; POST .../keep persists "Keep it here". * feat(system): show where the library lives and offer the move D129. System > Overview's Library health gains a "Library location" row (the database path, why it is there, the "Keep it here" choice and any PROMPT_MANAGER_DATA_DIR problem) with Move library for a node-folder library and Details otherwise. While a node-folder library is not kept there, the workspace shows the concept's .notice above every page: "Keep it here" saves the choice on the server and stops the notice; "Move library" opens the shared review (From, To, Size, what moves, what happens to the old files) and starts the exclusive job in Activity. The notice and the Overview read the location again when the job ends. The Overview comparison maps the inserted row; a new comparison holds the notice to the concept's Sources notice (markup, tokens, alignment, both themes and widths) with a mutation check. * feat(system): close the browser server's log files before its root goes The logs now live in the library's data folder, which for the browser test server is <root>/user/__promptmanager; Windows could not remove the temp root while the server still held prompt_manager.log open. The API test's node folder is spelled in lower case because public paths are case-folded on Windows. * feat(system): show the library location on the legacy admin page A line under the admin page's statistics names the database path and why it is there (ComfyUI's user folder, PROMPT_MANAGER_DATA_DIR, config.json, or the node folder with a pointer to Move library), read once from GET /prompt_manager/system/library. * feat(system): prove a fresh host keeps its library in user/__promptmanager The execution suite starts one pinned CPU ComfyUI with no configuration at all on an OS-chosen port: the library is created in user/__promptmanager (database and logs), nothing but byte code is added to the node folder, the location route reports the default folder, and ComfyUI's /userdata API serves none of it (traversal spellings, a listing, a system-user header). * feat(system): document where the library lives and how it moves README: where the library lives, the upgrade path (Move library / Keep it here), PROMPT_MANAGER_DATA_DIR, explicit paths unchanged, and the way back to an older version. TESTING.md: D129 and the library location contract (resolution order, move steps, crash windows, coded refusals, Windows handling). * wip(system): library move fix round in progress * fix(system): retry a fallback image link that races the library switch The image monitor's fallback and recent-prompt links went through PromptDatabase.link_image_to_prompt, which logs and returns 0 on any error, so survive_switch never saw a "database is locked" or closed connection during Move library and the link was lost in both files. The monitor now links through link_image_to_prompt_or_raise (one transaction per attempt); link_image_to_prompt keeps its log-and-return-0 contract for other callers. A link that returns 0 is no longer logged as "Successfully linked image 0". A switch that fails after repointing the models now closes the old file's detached connections instead of leaving them to GC: switch() adds them to a list the mover owns. The mocked retry test is replaced by a real-lock race asserting the image row lands in the new library only. * fix(system): say which source is the default and where a symlinked library lives Sources & scans: with no source configured, ComfyUI's output folder now reads "ComfyUI/output · watching · default" on its own row instead of a separate note line. GET /settings marks each effective source with in_comfyui (its public path is relative to the ComfyUI base) so only those get the "ComfyUI/" prefix; the scan still sends the server's public path. Library location: a library in the node folder reads through the node folder's unresolved place in custom_nodes, so a node folder symlinked to a checkout outside ComfyUI shows custom_nodes/promptmanager/prompts.db instead of prompts.db (System library row, Move review, admin Library line). Never absolute or "..", else the shared _public_path, which sources and jobs keep using unchanged.	2026-10-24 12:54:09 -07:00
Vito	fc3266d455	fix(thumbnails): generate thumbnails automatically and use them everywhere (#316) ... * fix(thumbnails): generate thumbnails automatically and use them everywhere New outputs never got a preview, so the workspace grid loaded full-size originals (844 of 19,907 images on a live install, exactly the newest). - Listings always give images and videos a thumbnail URL; a missing preview is made by the serve route on first view (bounded, single flight per file) and falls back to the original, remembering files that cannot be decoded. - The image monitor queues each settled output's preview on one background worker that follows Worker threads. - A scan that leaves 200 or more previews missing starts the visible thumbnail job; fewer go through the worker. - Automatic previews wait for a restore, a duplicate removal, a sources change, the gallery's clear and exclusive jobs. - The inspector cover uses the preview; the Overview row counts previews that failed. * fix(thumbnails): keep the video fallback tested and stop the preview worker before cleanup A video listed without a preview still renders its own element and is released on refresh. Test teardowns and the browser reset stop the automatic preview worker before the temp tree goes, so no preview lands after cleanup (Windows). TESTING.md documents automatic previews. * fix(thumbnails): admit previews on the loop and never send a video to an img Final review fixes for automatic previews: - A video preview that cannot be made is the placeholder frame (no-store), never the video; OpenCV failures fall through to ffmpeg and the placeholder. Cards and the inspector cover swap back to the video element when the preview image fails. - View requests are admitted on the event loop (3 writers, one flight per file, at most 64 waiting; past that the fallback at once). Writes run on a small dedicated pool, never the default executor. The history route shares the admission, flight key and link checks. - Drive, UNC, empty and dot segments are refused as text; the folder listing runs inside the admission. - Holds are checked after the slot and again before the rename; only decode failures are remembered. - Bounded, logged shutdown: ffmpeg is killed, no worker restarts after cleanup, and a reload stops the old worker. - The listing says whether the preview file exists (thumbnail_ready) for the legacy gallery's dot; media in a nested thumbnails folder keeps its original; file names the server quotes match in the inspector. * fix(thumbnails): cover the image fallback card and document the admission An image listed without a preview (a nested thumbnails folder) still shows its original; TESTING.md describes the view admission, the video placeholder, the holds before the rename and the bounded shutdown. * fix(thumbnails): let an idle preview worker leave and fit case-insensitive filesystems The capture worker exits once its queue is empty, so no thread outlives its work; the next output starts a fresh one. The mixed-case source test expects no folder listing where the exact name already resolves (macOS, Windows). * fix(thumbnails): answer shared flights with the outcome only and restart a failed worker The history and serve routes share a flight key, so a flight now answers only the ensure outcome whichever route started it; the serve route derives the preview path and, for a fallback, looks its source up again on the view pool. A view joining a history flight no longer gets a 500 or an image answered with the video placeholder. The capture worker clears its running state however its loop ends, and a worker that cannot start leaves the file queued for the next enqueue.	2026-10-24 08:35:14 -07:00
Vito	3f11ea6168	feat(annotations): Needs review queue backend (#284) ... * feat(review): add review queue schema with restore validation Add the Needs review schema group (media_review_metadata, media_review_items, media_review_decisions) beside the image annotation group it cites. review_schema_error validates the catalog, columns, foreign keys, row values and canonical 8 MiB evidence read-only; migrate_review_state installs it in a SAVEPOINT after the annotation group, and verify_database_file requires it under require_library_state. A review group without a valid annotation group is refused. Decisions null their pair and image navigation through indexes when those are deleted. Old backups and annotation-era backups restore and gain the group; browser fixtures clear the new tables on reset. * feat(review): persist ambiguous sampler evidence from scans and new outputs The output scan and the live monitor now queue every file the ingestion resolver finds ambiguous for Needs review. The file is captured, its metadata is read again from the capture and resolved with the default profile, and only an ambiguous result is stored, bound to the capture's annotation byte record and fenced by its storage epoch. Evidence holds one hashed projection per selectable candidate; none, more than 16 or over 8 MiB is a limited item. Recording is idempotent per bytes and evidence and never touches a decision. Scan statuses, counters, outcome rows and legacy SSE events are unchanged; the job scope gains review_items/review_errors and the log a Needs review line. The monitor records after its fallback link, best effort. * feat(review): explicit association decisions with compatible linking Needs review gains its queue reads and decisions: - GET /prompt_manager/review/items lists pending items oldest first with the exact pending total and an HMAC keyset cursor bound to the storage epoch and the review index revision. - GET /prompt_manager/review/items/{id} adds the candidates (role texts, label, selectability), the dependency fingerprint and the digest. - POST /prompt_manager/review/items/{id}/decision applies a candidate only for a capture of the item's bytes that the current file still hashes to and whose evidence is unchanged, through the ingester's compatible-only rules in the same transaction as the decision row; a conflict writes nothing. Leave unresolved records the decision and links nothing. ingest_resolved_pair keeps its contract; its body moves into _ingest_resolved_pair_in, which runs in the caller's transaction. Items keep the file's recorded spelling so targets keep their case on Windows, and a rescan skips the copy and hash of a file whose size and mtime match the bytes an item already cites. * fix(review): build the spelling validation test from native paths On Windows the locator is case-folded and uses backslashes, so the fixed POSIX pair /out/a.png no longer folded to itself and the new spelling check refused the good case. The test now spells a mixed-case native path and folds it with os.path.normcase, and also refuses a locator of another file. * fix(review): Apply sees the file under every spelling and rechecks only its choice Apply now finds the legacy rows of the same physical file however its root is reached. The roots are real paths while the monitor links under the directory it watches, so a symlinked or junctioned root, subst or a mapped drive used to hide the monitor's row and Apply added a second, differently linked row. Same-name rows are resolved to their physical identity (normcase(realpath), else samefile); the dependency fingerprint covers all of them, two rows refuse the choice, one row is enriched under the ingester's compatible-only rules, and a same-name row the request did not classify is a changed association. Apply re-verifies only what the decision depends on: the bytes are still ambiguous and still offer the chosen candidate. A resolver change that rewords the reason or reorders candidates leaves pending items applicable. The resolver version is one constant in prompt_pair_resolver, shared by ingestion provenance and Needs review. The transaction's byte check receives the re-hashed digest (or the body's for Leave unresolved), every Apply refusal branch is tested with zero writes, and the root listing and realpath calls run on a worker thread. * fix(review): validate stored review evidence once per startup Startup used to run the full review validation twice (the early guard and the install) and the annotation group's row checks inside each, so every evidence document was parsed and re-encoded twice. The early guard is now catalog-only (a review group without annotation tables), and the install validates the group once, trusting the annotation group it has just validated. The evidence predicate no longer pattern-scans every document in SQL, because the streamed canonical pass proves it, and the codec is built once. Measured on a database with 40,000 items of 4 KiB evidence: startup open went from 7.7 s to 2.5 s. A test pins one evidence pass per reopen. Restore and verify_database_file keep the full validation. * fix(review): keep one capture worker for interactive requests Needs review recording from the scan and the monitor shared the capture store's two workers with the Tag dialog and Apply, so a busy scan could answer a user's preview or decision with "Too many previews are open". Background captures now never take the last worker or capture slot, and wait up to two seconds for room before counting the file as not added. Interactive captures are admitted or refused at once, as before. * fix(review): refresh the recorded spelling when a file is seen under a new one A case-only rename on Windows or macOS keeps size and mtime_ns, so the rescan skipped the file and the item kept the old spelling. Its target was then formatted from a name the directory no longer lists, the capture refused it, and the item stayed pending with no way to apply it. The skip path and a duplicate insert now record the current spelling for every item at the locator and move the review index revision; item revisions do not change, so an open decision stays valid. Tested under case-folding semantics on every platform. * fix(review): final capture store close and recoverable stale cursors Closing the capture store at shutdown now runs under the store lock and is final: a monitor recording or scan batch still in flight gets a capture error instead of creating a new temp directory and restore subscription after the cleanup. A well-formed queue cursor this process cannot verify (opened before a ComfyUI restart, whose cursor secret is per process) now answers 409 review-conflict with the current total, like any other stale page, instead of an uncoded 400. Malformed cursors stay 400. * fix(review): make only the shutdown cleanup final The previous commit made every close of the capture store final, which also refused the explicit closes that swap in a fresh store. Only the host's on_cleanup hook is final now; an observe after it answers a coded capture error instead of an unhandled 500. * refactor(admin): move Needs review recording into its own mixin _record_scanned_review, _record_review_sync, record_review_file and _log_review_items move unchanged from py/api/admin.py to py/api/review_recording.py (ReviewRecordingMixin), keeping admin.py from growing further past the size ceiling. No behaviour change. * fix(review): record the review fix round and the skip's residual risks in TESTING.md The Needs review paragraphs now state the rescan skip's residual risks (an in-place rewrite that keeps size and mtime_ns is never re-queued; a deleted or moved file leaves its pending item counted), the physical identity rule for Apply, the candidate-only recheck and shared resolver version, the spelling refresh, background capture admission, the final shutdown close, the stale-cursor answer and the measured startup validation cost. * fix(review): refuse monitor recording beside record-changing jobs System S1 (#289) gives every record change a registry admission class. The scan already records Needs review items under its own records job. The monitor's recording lasts one file and is not a job, so it now checks the records admission: while a records or guarded job or hold runs it records nothing, logs that the file waits for the next scan, and answers deferred. The fallback link stands and the next "new" scan reads the ambiguous file again, because a fallback link is not a resolved one. JobRegistry.blocker and active iterate a snapshot of the jobs, since the monitor's worker thread now asks them too. * test(backup): wait for the restore, not a 5 s timer, in the thread test test_database_is_usable_from_every_thread_after_restore let its worker wait 5 s for the restore. On a stalled Windows runner (py3.11, where the previous restore test also took 25 s instead of 0.3 s) the restore took longer, the wait timed out, and the worker read the old database before the restore ran. The worker now waits for the event under a 120 s bound that only a hung test reaches, reports a timeout as an error, and the main thread releases it even when the restore raises. With a restore delayed by 6 s the old test fails and the new one passes. * fix(review): store non-finite evidence as null and count pending items canonical_evidence now follows main's non-finite policy (#303): NaN and ±Infinity are written as null instead of raising, so a NaN-stamped graph never turns a review item limited. The System Overview reports the review queue's pending total in needs_review.pending, read in the same snapshot as the Library counts. * fix(review): show the pending count on the System Overview review row The Overview's review row now reads "N ambiguous image links" with the concept's line, from needs_review.pending. Its Review metadata action stays aria-disabled until the Needs review page ships. The system specs and their concept comparison follow the real count. * fix(system): answer the Overview and Statistics only after every read ended asyncio.gather answered at the first failed read while its sibling executor reads were still inside SQLite. In the suite the test teardown then deleted the database under the running read, and py3.10 on Ubuntu segfaulted in get_statistics (test_statistics_failure_is_an_error_envelope). Both routes now wait for all their reads before answering. * fix(review): queue only ambiguous files that nothing links, and heal the queue The maintainer's live scan (19,908 files, scope "new") reported 4,677 files under Needs review, but the queue held 10,232 items: every file the strict resolver found ambiguous was queued, including the ones the legacy browser-parser fallback then linked (all sampled "added" rows) and the ones already linked to a known prompt (135 of 200 sampled "known" rows). A file is now queued only when its final outcome is ambiguous and unlinked: the resolver finds it ambiguous, the fallback links nothing and no saved row links it already. An ambiguous file that a saved row links is reported as unchanged ("Already linked to its prompt"), not as Needs review. The queue heals itself. After each batch, the scan removes the pending items of every listed file (read, or skipped as known) that a saved row now links. The monitor does the same for each ambiguous output it links, and never queues one. Only an undecided item is removed (pending, revision 0, no decision row), so a decision is never deleted or changed and the schema needs no new state. The scan job's scope.review_items now counts this scan's files that are in the pending queue after it, new or already there, so it agrees with the review group's ambiguous rows and with the queue total. review_added and review_settled are new, and the log gains "already in Needs review" and "left Needs review · now linked" lines. A duplicate whose item is decided is reported as "decided" and is not counted. * fix(review): never queue an ambiguous file with nothing to choose The maintainer's live queue held 39 items whose candidates all read "unresolved" ("No positive / No negative"). An ambiguous result is now reviewable only when at least two candidates are resolved with non-blank positive text. With none the file could not be traced; with exactly one, the untraced branches may still have made the file, so that text is not certain either. Both are reported unresolved, never linked and never queued (also the A1111 two-delimiter case, which has no candidates). A rescan removes the pending items of such files whose own evidence offers no choice, with the same safe delete as the linked settle (pending, revision 0, no decision row). A limited item (over 16 candidates) that holds at least two candidates is kept. The job scope gains review_untraced and the log "= N images left Needs review · nothing to choose". * test(scan): pin only the per-source scope keys beside the review counters * test(system): match the scan scope keys beside the review counters	2026-10-23 14:19:11 -07:00
Vito	caa9649f32	feat: reuse positive and negative prompt combinations (#195) ... * feat(db): add pair schema and stage restore migrations for #193 * feat(database): persist reusable ordered prompt pairs (#193) * feat(api): expose reusable pairs and atomic metadata enrichment (#193) * feat(metadata): resolve canonical prompt pairs with bounded provenance (#193) * feat(scan): enrich image pairs without guessing or duplicate links (#193) * fix(database): preserve pair tags during global maintenance (#193) * feat(tracking): enrich live image pairs and cancel scans safely (#193) * feat(nodes): encode paired prompts and count cached recipe runs (#193) * fix(database): validate restored pair ownership and provenance (#193) * feat(admin): report pair scan outcomes and cancellation (#193) * feat(ui): add paired prompt editor and explicit recipe library for #193 * fix(autocomplete): include reusable pair metadata vocabulary (#193) * fix(pairs): honor recent sorting and conservative scan outcomes (#193) * fix(ui): protect paired draft saves and clarify negative variants (#193) * fix(metadata): preserve bounded paired execution evidence (#193) * fix(tracking): follow actual pair output roles and prefer execution evidence (#193) * docs(pairs): document live validation and retain renderer screenshots (#193) * test(api): inject disposable databases before API startup (#194) * test(api): canonicalize portable temporary gallery roots (#194) * docs(ui): prototype branching negative variant history (#193) * docs(ui): show selected negative beneath positive in timeline concept (#193) * docs: retain approved timeline for later gallery redesign (#193) * test(db): close restore fixture connections before cleanup (#194) * test(db): reuse existing handle guards in all pair fixtures (#194) * chore: clarify rollback test and intentional exception handling (#193)	2026-10-17 08:00:42 -07:00
Vito	da322f8c94	Hardening: security fixes, queue-time run counting, background jobs, 85%+ test coverage (#153) ... * chore: valid packaging metadata, flake8 config file, ignore local config with secrets pyproject.toml carried requirements.txt comment lines as dependencies, which fails PEP 508 parsing. flake8 never read [tool.flake8] from pyproject, so the settings move to .flake8 and pre-commit runs the full rule set. config.json (holds the CivitAI API key and machine paths) and other local artifacts are now ignored. coverage joins the dev requirements. * ci: test on Linux, Windows and macOS with a coverage gate and lint job * refactor: drop validate_workflow_name, the column it validated no longer exists * chore: keep agent worktrees out of coverage measurement * fix: image routes serve only media files inside allowed directories serve_image and serve_output_image now reject anything whose extension is not in IMAGE_EXTENSIONS, fail closed when no output directory is configured, and answer 403 (not 404) for dot-dot, absolute and symlink-escaping paths. Containment is checked after realpath+normcase so it behaves the same on Windows, macOS and Linux. 500 responses use safe_error_message so OSError text no longer leaks absolute paths. * fix: bound page sizes on list endpoints /search, /recent and /images/recent share parse_page_params: limit is clamped to [1, MAX_PAGE_LIMIT=500], offset to >= 0, and non-integer values answer 400 instead of 500. /search now honours offset and all three return a pagination block with the effective limit and offset. * perf: thumbnail generation no longer blocks the event loop generate_thumbnails_with_progress is split into _thumbnail_targets (scan, executor), _generate_one (one PIL/ffmpeg job, executor) and an async driver that awaits each file via _run_in_executor, emits one progress frame per file and yields between files. Per-file failures are reported as file_error events (error stays fatal for the UI) and the loop continues. The wildcard Access-Control-Allow-Origin header is gone. The blocking POST variant reuses the same helpers. The scan never follows symlinks and is capped by MAX_SCAN_DEPTH / MAX_THUMBNAIL_FILES. * fix: cap every request-driven resource in the image routes - /images/recent, /images/all and /images/output parse limit/offset through parse_page_params on every path (absent, 0, negative, float strings), and the clamped values are what reach the DB. /images/all gets its own MAX_BULK_LIMIT ceiling instead of an unbounded query; /images/recent caps its fetch window at MAX_RECENT_IMAGES_WINDOW. - parse_page_params also clamps offset to MAX_PAGE_OFFSET so values past int64 no longer surface as a 500 from SQLite. - Gallery and thumbnail scans share _iter_media_files: no symlink following, MAX_SCAN_DEPTH, and MAX_GALLERY_FILES / MAX_THUMBNAIL_FILES caps. Thumbnail size comes only from named presets. - JSON bodies on /images/link and /images/generate-thumbnails are read in bounded chunks; over MAX_JSON_BODY_BYTES answers 413, malformed or non-object JSON answers 400. - /images/link only accepts media paths inside the allowed directories (403 otherwise), closing a path-existence oracle. * test: cover image and prompt routes Every route in py/api/images.py and py/api/prompts.py now has a success path and an error path test: save/update/delete/rate/tag/category/bulk endpoints, image list/link/unlink/delete/metadata/scan endpoints, the thumbnail SSE stream, clear-thumbnails, and the video thumbnail chain (fake cv2 success/failure, ffmpeg absent or failing, PIL placeholder). Combined statement+branch coverage: images.py 97.7%, prompts.py 97.4%. * fix: autotag endpoints no longer read arbitrary paths or run on GET * fix: CivitAI key is never returned or sent off-host * fix: autotag/single checks real paths and DB rows against output dirs * fix: CivitAI downloads never follow redirects * refactor: autotag model loading is injectable * test: cover autotag and lora integration * fix: saving LoRA settings without a key keeps the stored CivitAI key * refactor: share the internal error message in autotag routes * fix: never treat negative-only PromptManager nodes as the run prompt run_prompt_nodes fell back to every PromptManager node whenever no positive node was traced, including graphs that do have a sampler and whose only PromptManager node feeds its negative input. Fall back only when the graph has no recognisable sampler (new has_sampler helper). Samplers named like one are recognised even without a model link. Adversarial graphs (cycles, 10k-node fan-out, non-dict inputs) are covered. * fix: do not follow negative text inputs when tracing the positive prompt _TEXT_KEY matched input names such as text_negative, so a styler node feeding a sampler's positive input pulled its negative PromptManager node into the positive trace. Input names matching (^|_)neg(ative)?($|_) are now skipped (new follows_input helper). * fix: count prompt runs when ComfyUI dequeues the prompt, not when it is submitted The on-prompt hook runs before ComfyUI validates the request, so counting there bumped run_count for prompts that were rejected or deleted from the queue. The hook now only resolves the positive PromptManager texts, mints a UUID4 prompt_id when the client sent none, and stashes the hashes under that id (QueuedRuns, bounded to 1000 entries). PromptQueue.get is wrapped idempotently; when a prompt is dequeued for execution its stashed runs are counted (or remembered as pending first use). Servers without a prompt_queue fall back to counting at submit time with one warning. __init__ defines _global_db before the monitoring block and passes a factory that raises RuntimeError while the database is unavailable; the package is now imported under test with server and folder_paths stubbed. * fix: keep batch image queue aligned when unrelated workflows save images The metadata strategy returned None for images whose embedded workflow has no positive PromptManager node, so the handler popped the batch FIFO for them and every later batch image linked to the wrong prompt. It now returns a tri-state: linked (prompt found), foreign (no PromptManager prompt in the workflow, so the image is skipped and no fallback applies) or unknown (only then the queue, snapshot, tracker and recent-prompt fallbacks run). * fix: image monitor validates directories before creating the observer start_monitoring assigned self.observer before checking that any directory existed, so with no valid directory the monitor was stuck "already running" and stop_monitoring raised on join of a never-started observer. Directories are validated first and only a started observer is kept; stop_monitoring is safe to call at any time and always joins. New is_monitoring property. The per-image threading.Timer is replaced by one daemon worker: paths are deduplicated and bounded (MAX_PENDING_IMAGES), processed in creation order (batch images pop the prompt queue in order) after the processing delay and once the file size is unchanged across two reads. * refactor: one prompt hash implementation utils.hashing.generate_prompt_hash is now the only prompt hash: PromptManagerBase._generate_hash delegates to it and the inline hashlib.sha256 copies in the prompt tracker and image monitor are gone. Remove PromptManagerBase._open_web_interface (it targeted a web_interface folder that never existed) and the unused restart_gallery.py script. IS_CHANGED on PromptManager and PromptManagerText now includes category and tags, so editing either re-runs the node and updates the saved metadata. * fix: stop ComfyUI from loading the admin bundle into the canvas page WEB_DIRECTORY = "web" made ComfyUI inject every .js under web/ (admin.js, gallery.js, vendored Tailwind and ViewerJS) into the canvas page. The node extension moves to web/comfy/prompt_manager.js (content unchanged; its ../../scripts/app.js import resolves against the served URL, which is the same) and WEB_DIRECTORY points at that folder only. Admin and gallery assets are still served from web/ by the extension's own routes. tests/test_sort_contract.py still reads web/prompt_manager.js and must be updated to the new path by its owner. * fix: image monitor recent-prompt fallback reads the paginated result get_recent_prompts returns {"prompts": [...]} but _get_fallback_prompt indexed it like a list, so the last-resort link to the most recent prompt always failed silently. Accept both shapes. * test: cover prompt_manager_base, prompt_manager and prompt_search_list Node base: init wiring, metadata update on re-save, run counting, database error paths, tag parsing, search and recent API wrappers, gallery start failure, LoRA trigger-word injection (disabled, missing modules, enabled). PromptManager: missing CLIP error, encoding of the combined text with a fake CLIP, tracking failure tolerance, empty text. PromptManagerText: tracking failure tolerance. PromptSearchList: filters, multipart and LoRA-only skipping, newline collapsing, preview, empty and error results. * fix: diagnostics dependency check survives a watchdog without __version__ check_dependencies read watchdog.__version__ for its log line; recent watchdog releases have no such attribute, so the AttributeError escaped and took the whole diagnostics run down. Read it with a default. Also covers utils/diagnostics.py: database and images-table checks against a real temporary database, file system check, ComfyUI output detection (folder_paths, configured directories, relative fallback, nothing found), dependencies, the full run and the test image link. * test: cover utils/logging_config Isolated logger manager on a temporary log directory: memory buffer order, level filter and 1000-entry cap, rotation and backup listing, tail and whole file reads, path traversal rejection, truncation (including removal errors), level changes across prompt_manager loggers, handler rebuilds. * test: cover utils/comfyui_integration Singleton re-initialisation keeps registrations, latest prompt per thread, cross-thread fallback with the five-minute recency limit, global lookup for the current thread, and cleanup of old registrations. * test: cover __init__ Version parsing from pyproject (and the unknown fallback), and a server whose add_on_prompt_handler raises: nodes and monitoring still come up. * docs: point at the moved canvas extension under web/comfy * fix: give every thread its own sqlite connection PromptModel.get_connection() handed every thread the same check_same_thread=False connection, so a commit or rollback from the watchdog, executor or queue-hook thread could land on another thread's half-done write (sqlite3 raised 'bad parameter or other API misuse' under concurrent saves). Connections now live in threading.local, each with WAL, Row factory, foreign_keys and busy_timeout, and are tracked so they can be closed. PromptModel.close() / PromptDatabase.close() close the calling thread's connection so tests and restores can delete the file on Windows. * fix: saving an existing prompt concurrently returns its id Node execution, the queue hook and the image monitor all do get_prompt_by_hash() then save_prompt(). When two of them race on the same text the second INSERT failed on UNIQUE(hash) and surfaced an IntegrityError. save_prompt now catches it, re-fetches by hash and returns the existing id, so eight threads saving one prompt end with a single row and every record_prompt_use counted. * fix: backups include WAL data and restores are integrity-checked The database runs in WAL mode, so a shutil.copy2 of prompts.db silently dropped every committed row still in prompts.db-wal; the backup download and the pre-restore safety copy both did that. Restore also overwrote the live file with connections open, read the whole upload into memory before checking the 100 MB cap and never ran PRAGMA integrity_check. PromptModel.backup_database now uses the SQLite online backup API, verify_database_file opens a candidate read-only and checks header, integrity and the prompts table, and restore_from_file backs up the live database, closes every thread's connection, copies the file, removes stale -wal/-shm and re-runs migrations. The admin handlers stream the upload to a temp file with an early size cutoff, reject anything verify_database_file refuses with a 400 and keep the {success: ...} JSON shape. The schema-init connection is now closed instead of leaking. * fix: migrations are idempotent and schema init runs once per process Every PromptDatabase() (nodes, image monitor, API, diagnostics) re-ran table creation and all migrations. Schema init is now cached per normalised path (class-level set under a lock) and re-run only when the file disappeared, after a restore, or after reset_schema_cache(). The copy-and-swap migrations drop a stale *_new table left by an interrupted run, execute inside an explicit transaction with PRAGMA foreign_keys = OFF (DROP TABLE prompts used to cascade-delete every generated_images row), roll back cleanly and log and re-raise sqlite3.Error instead of swallowing it. * fix: image uniqueness by full path, escaped LIKE search, no usage stamp on save generated_images was UNIQUE(prompt_id, filename) on the basename, so ComfyUI's same-named files in different date folders were silently ignored by INSERT OR IGNORE, and the lastrowid == 0 branch after it never ran. The table now carries file_path (normcase/normpath/abspath of the image path) with UNIQUE(prompt_id, file_path); the migration collapses rows that normalise to one file and supersedes the old basename migration, which would otherwise undo it on every start. search_prompts and the tag search escaped nothing, so 100% matched '100 percent' and a_b matched aXb; user text now goes through escape_like with ESCAPE. save_prompt no longer stamps last_used_at, the usage backfill runs only on the upgrade that adds the columns, and the Recently Used / Most Used orders put never-run prompts (NULL) after run ones instead of first. * fix: escape LIKE wildcards in image prompt search search_images_by_prompt interpolated the user's text into LIKE unescaped, so '50%' also matched '50 percent'. It now goes through the same escape_like helper as search_prompts. Also drops an unused import and an unused cursor assignment flagged by pyflakes. * test: cover database operations Behaviour tests for export (json/csv/errors), duplicate detection and merging, image metadata storage and lookup, missing-file cleanup, statistics, bulk operations, orphan pruning, consistency checks, subfolder listing, tag queries and management edge cases, legacy tag column parsing, config-driven path resolution, model housekeeping and the TEXT -> INTEGER prompt_id migration. database/operations.py and database/models.py are both above 85% statement+branch coverage. * test: isolate the clock and log buffer from background threads of other tests * fix: gallery roots are restricted to ComfyUI directories A gallery root could be set to '/' (or 'C:\') through POST /prompt_manager/settings because the deny list only matched a few system prefixes. The root was persisted to config.json and fed into the image-serving containment check, which then allowed any file on disk to be read through /prompt_manager/images/serve/. - GalleryConfig.validate_gallery_root() accepts a directory only when its realpath lies inside a ComfyUI directory (folder_paths output/input/ temp/user) or a directory listed in PROMPT_MANAGER_EXTRA_GALLERY_ROOTS; filesystem roots, the home directory, files and missing paths are rejected. - save_settings returns 400 and persists nothing when any root fails. - _get_all_output_dirs() filters configured roots through the validator and never falls back to a rejected root. - PROMPT_MANAGER_CONFIG_PATH relocates config.json; load/save default to it. * fix: log config accepts only known keys POST /prompt_manager/logs/config forwarded the whole request body into PromptManagerLogger.update_config(), so any key could be injected into the logging configuration. The handler now accepts only LOG_CONFIG_KEYS (level, max_file_size, backup_count, console_logging, file_logging, buffer_size), type-checks each value (level name, non-negative int, bool) and answers 400 listing unknown keys. Malformed or non-object bodies are 400 instead of 500. * fix: no absolute server paths in API responses Settings, diagnostics, duplicate/output scans and the logging endpoints returned absolute filesystem paths, and several error envelopes echoed str(e) which for OSError carries the full path. - _public_path() renders a path relative to the ComfyUI base directory (folder_paths.base_path, else the output dir's parent) or to the parent of an opted-in extra gallery root, falling back to the basename; _public_error() reduces OSError to strerror + basename. - GET /settings returns gallery roots and monitored directories in that form; POST /settings resolves relative roots against the same anchors (GalleryConfig.resolve_gallery_root) so the GET output round-trips. - Diagnostics output_dirs, monitored_directories and the missing-DB message no longer carry absolute paths; the image monitor is reached through the package-or-top-level import fallback instead of a three-dot relative import that failed outside the package. - Duplicate scan 'path' is now relative to the output root and the delete endpoint accepts relative paths. - /logs/files 'path' and /logs/stats 'log_directory' use the public form; unexpected errors answer with a generic message. * perf: run duplicate and output scans off the event loop The duplicate scan walked the output tree and hashed every file on the aiohttp event loop, and the duplicate delete removed files inline; the output scan ran its per-file database work one executor hop at a time. - _find_duplicate_images_sync / _describe_media_file do the walk and hashing; find_duplicate_images() runs them via _run_in_executor. - _delete_duplicate_images_sync / _delete_one_duplicate do the removal (absolute or output-relative paths) off the loop. - The output scan is split into _collect_output_media_sync, _extract_batch_metadata_sync and _ingest_scan_batch_sync (one executor call per batch for all DB work) with a small SSE generator. - Media collection is shared and de-duplicates by normcase'd path. * refactor: settings reads the image monitor through its module get_settings located the ImageMonitor singleton by scanning sys.modules for any module with 'image_monitor' in its name, a workaround for the package being importable under two identities. It now uses the same try-package / except-top-level import as the rest of the API (_current_image_monitor) and falls back to the configured roots when no monitor is running. * fix: clamp the logs limit and reject non-integers GET /prompt_manager/logs accepted any limit up to 1000 and answered 500 when the value was not an integer. The limit is now clamped to [1, 500] and a non-integer value is a 400. * test: cover core API module and gzip middleware * refactor: split diagnostics, maintenance and HTML serving into helpers - run_diagnostics composes four module-level probes (_diagnose_database, _diagnose_dependencies, _diagnose_output_dirs, _diagnose_image_monitor). - run_maintenance is table-driven: _maintenance_runners maps each operation name to a callable and _run_maintenance_operations reports per-operation success or a public error message. - _read_json answers 400 for malformed or non-object JSON bodies in settings, duplicate delete, test-link and maintenance instead of 500. - test_image_link delegates to _link_test_image. - The four HTML page routes share _serve_html_page. * test: cover admin and core API routes Route tests for stats, cleanup, duplicate scan/delete, settings, diagnostics, test-link, maintenance (every operation, success and failure), the output scan internals, and contract-level tests for backup/restore (status codes and envelope only). * test: make admin, logging and diagnostics tests independent of test order The diagnostics message no longer carries the absolute path, the restore test closes its raw sqlite connections before the live file is replaced, the logs tests declare their own folder_paths base directory, and the isolated logger filters every handler to the test thread. * fix: truncating logs no longer rolls the main log over or pads it with zeros truncate_logs emptied prompt_manager.log through a second open(), leaving the rotating handler's stream at its old offset: the next record was written past the new end (zero padding on POSIX) and tell() still reported the old size, so the file rolled over and resurrected the backup that had just been deleted. It also logged once per file from inside the deletion loop, which could do the same with a small size limit. The main log is now truncated through the handler's own stream and a single summary line is logged afterwards. * fix: HTML escaping covers attribute context escapeHtml now escapes & < > " ' (shared web/js/html-escape.js, loaded by admin, gallery and metadata pages) instead of the textContent/innerHTML round-trip that left quotes intact, so a tag, category, checkpoint or file name containing a quote can no longer break out of a data-*, value or title attribute. Inline onclick handlers that interpolated server data now carry data-action and data-* attributes and are dispatched through one delegated listener per page (web/js/data-actions.js); the img onerror fallback uses data-fallback-src with a capturing error listener. Every remaining template value written into an attribute or text node (prompts, checkpoints, file paths, log lines, directory names, categories, suggestions) is escaped. metadata.html no longer builds an "explorer /select" shell command from the file name (Electron-only branch); the path is copied to the clipboard. * refactor: remove shadowed and dead frontend code admin.js defined parsePNGMetadata, extractComfyUIData, showFullPrompt, showWorkflowData, downloadWorkflowJSON and copyAllMetadata twice; the first definitions were unreachable because the later ones shadow them. updateMetadataPanel, showMetadataError, copyPrompt and tryFallbackMetadata only ever wrote to #metadataContent, an element no page creates, so they are gone too. gallery.js defined escapeHtml and showNotification twice. The admin keydown/resize handlers referenced closeImageViewer, previousImage, nextImage and applyImageSizing, none of which exist, and the imageViewMode state they used; only the Escape-closes-gallery branch remains. Debug console.log calls are removed; the ones that were the only statement of a catch block are console.warn so parse failures stay visible. * fix: stale responses, double submits and leaked listeners web/js/api-client.js wraps fetch with one failure path (ApiError for non-2xx, success:false and non-JSON bodies) and latestOnly(), which aborts the previous in-flight request and resolves superseded calls to STALE. The admin search and recent-page loads share one guard, the gallery page load has its own, so fast typing or clicking can no longer paint an older result over a newer one. In-place prompt editing used a blur listener with {once:true} plus a keydown listener added on every edit, which sent the PUT twice and stacked handlers. The list now has one keydown/focusout pair bound at construction and finishEdit() clears the edit state before touching the element, so a save is sent exactly once. Mutations re-fetch the page currently shown (refreshList) instead of calling search(), and delete, bulk delete, edit, restore and scan no longer reload the whole page. Pagination state is committed only after a successful fetch via the pure ListState.nextPageState(current, response) in both admin and gallery, so a failed or aborted page request leaves the current page intact. The gallery lightbox removed its keydown handlers on a non-existent 'remove' event; a MutationObserver now drops them when the modal leaves the document. web/js/sse-stream.js reads server-sent events over fetch so POST-only progress endpoints can stream. * fix: follow the POST-only autotag streams and the new LoRA and list contracts autotag/download and autotag/start are POST-only now; EventSource can only GET, so both pages stream their progress through SseStream.connect(url, {method: 'POST'}), which keeps the onmessage/onerror/close surface the handlers already use. autotag/single is called with image_id and the re-tag dialog shows the server's filename instead of splitting an absolute path. lora/status reports has_civitai_api_key instead of the key: the field shows a 'key is stored' placeholder, an empty field keeps the key on save and a 'Clear the stored key on save' checkbox sends clear_civitai_api_key. Thumbnail generation reports per-file failures as file_error events; both progress views count them (and show error_count from progress) without closing the stream. The auto-tag review follows pagination.has_more on the capped /images/all endpoint, and /search is paged like /recent so paging in search mode no longer switches the list back to recent prompts. * refactor: one PNG metadata parser admin.js, gallery.js and metadata.html each carried a copy of the PNG chunk walker that reads ComfyUI's tEXt/iTXt entries (admin.js had two until the shadowed one was removed). web/js/png-metadata.js is now the only parser: it checks the signature, bounds-checks every chunk so a truncated file or a bogus length cannot throw, decodes tEXt as Latin-1 and iTXt as UTF-8, skips compressed zTXt/iTXt payloads instead of decoding them as garbage, and stops at IEND. The three page-level parsePNGMetadata functions delegate to it. * fix: tags deep link renders and admin modals close admin.html loaded tags-page.js after admin.js, so opening the page at #/tags found no TagsPageManager and rendered nothing; the script order is now theme-bridge, html-escape, api-client, png-metadata, (list-state, sse-stream, data-actions), viewer, comfy-metadata, prompt-list-sort, view-router, tags-page, admin. Route resolution lives in the pure ViewRouter.resolveView(). The auto-tag modal's close buttons called a bare hideModal(), which only gallery.js defines; they call window.admin.hideModal(). PromptAdmin is constructed from startAdmin() on DOMContentLoaded (or immediately when the document is already parsed) instead of at script evaluation time. * test: cover frontend modules browser-globals.test.js loads every page module through vm with a fake window, the way the pages do, and asserts the global each one exposes (HtmlEscape/escapeHtml, DataActions, ApiClient, ListState, SseStream, PngMetadata, ViewRouter, PromptListSort, ComfyMetadata). sse-stream gains cases for a POST body, missing handlers and close() before or during delivery. Every pure module is above 85% line coverage under node --test --experimental-test-coverage. * fix: page image listings in SQL and drop three unused helpers get_recent_images and search_images_by_prompt take limit/offset so the recent and search image routes no longer over-fetch a 5000-row window and slice it. generate_content_hash, PromptModel.migrate_database and find_duplicates had no callers outside their own tests. The isolated logger tests re-apply their thread filter after update_config rebuilds the handlers. Long lines in the touched database and image modules are wrapped so the full flake8 hook passes. * fix: restore through SQLite's backup API, close_all for shared files, safer output scans restore_from_file copied the uploaded file over the live database and deleted its WAL while other PromptModel instances (nodes, image monitor, queue hook) still held connections, which corrupts on POSIX and fails on Windows. It now copies pages through the online backup API into the live connection so every open handle sees the new content. PromptDatabase.close_all() closes the connections of every thread, which tests need before deleting a database. Output scans skip symlinked files, filesystem entries get a stable digest id instead of a per-process hash(), and thumbnail streaming stops when the client disconnects. * style: wrap long lines in node modules Split tooltip, description and docstring strings that exceed 88 columns using implicit concatenation; string values are unchanged. Drop the unused typing imports from prompt_search_list and mark the deliberately late node imports in __init__ with noqa: E402. * style: wrap long lines in the API layer, autotag and config Split long f-string log messages, response messages, URL builders and docstrings; DEFAULT_PROMPT and SYSTEM_PROMPT are rewritten as implicit concatenations with identical values. Drop the unused shutil import in admin and the unused HFCacheInfo import in autotag; keep the hf_hub_download/snapshot_download availability probe with noqa: F401 and mark the deliberately late imports in config with noqa: E402. * style: wrap long lines and drop unused imports in utils Re-wrap module docstrings, comments and f-string log messages; remove the unused typing.List and PngInfo imports; mark the PIL availability probe in diagnostics with noqa: F401; turn two placeholder-less f-strings into plain strings; strip trailing whitespace inside the diagnostics SQL. * style: wrap long lines in tests Split long docstrings and SQL literals (values unchanged) and drop the unused os import from test_path_security. * chore: flake8 ignores the gitignored standalone tagger script * fix: escape image dimensions and workflow JSON before rendering openGalleryModal interpolated width, height and file_size (stored from a POST body, so possibly strings) straight into data-caption and the card text. showWorkflowData wrote JSON.stringify(workflow) raw into a blank same-origin window, so a workflow containing </pre><script> ran. Both now go through ImageHelpers (new pure module, node:test covered); a string file_size no longer reaches formatFileSize's toFixed either, and a blocked popup shows a notice instead of throwing. * fix: escape remaining server values rendered through innerHTML Maintenance and diagnostics results, review-tag errors, video info and metadata error messages, consistency issues, recent image filenames, output directory names, the prompt rating attribute, gallery media types and duplicate group headers were interpolated unescaped. The copy fallback modal also inlined the prompt text inside <textarea>, which a closing tag could break out of; it now sets textarea.value. * fix: address gallery auto-tag review images by path, not digest id The gallery review flow builds its list from /prompt_manager/images/output, whose id is a 16-hex path digest, but posted it as image_id to autotag/single, so the server looked up a database row and every request 404ed. autotagSingleBody (pure, tested) now sends path for output-folder entries and keeps image_id for database rows, which the admin review flow (/images/all) still uses. * fix: read thumbnail generation progress over POST with SseStream images/generate-thumbnails/progress is becoming POST-only, which the browser EventSource cannot request. Both thumbnail progress consumers in the gallery now use SseStream.connect(url, {method: 'POST'}) exactly as the autotag streams do. SseStream gains addEventListener/removeEventListener for named event: frames (tested) so the status, start, progress, file_error, complete and error listeners stay as they were, and an onclose guard rejects when the stream ends without a terminal frame. * fix: drop the unused copy-text data-action handler Nothing in the admin markup emits data-action="copy-text"; the prompt copy buttons call copyToClipboard directly. * fix(lora): confine LoraManager paths to custom_nodes and hide server paths - is_civitai_url also refuses URLs with userinfo or a non-443 port, so https://evil.com\@civitai.com/x and https://civitai.com:8443/x are rejected. - resolve_lora_manager_path / custom_nodes_directories: lora/enable and lora/scan only accept a LoraManager directory that is a direct child of ComfyUI's custom_nodes (folder_paths base_path or registered custom_nodes folders, else this package's parent) and whose name contains "lora", compared via realpath+normcase; relative public paths resolve against the API path anchors. Anything else is 400, so find_lora_directories can no longer widen the image allow-list with arbitrary trees. - lora/enable persists through PromptManagerConfig.save_to_file(), which honours PROMPT_MANAGER_CONFIG_PATH, instead of a hard-coded config.json. - detect/status/enable return path and detected_path through _public_path; every lora handler reports exceptions via _public_error, and lora/scan errors raised after the SSE stream opened become an error event. - Tests close the database before unlinking it. * fix(autotag): contain batch rows, require media extensions, bound the scan - The batch tagger runs every generated_images row through _resolve_allowed_image_path; rows that exist but lie outside the served output directories are skipped and counted in skipped_outside_roots on progress and complete events, and the engine receives the resolved path. - _resolve_allowed_image_path requires a media extension (.png .jpg .jpeg .webp .gif .bmp .tiff .tif), so autotag/single returns 403 for other files. - autotag/start holds a module-level lock; a concurrent start gets 409 {"success": false, "error": "Auto-tagging is already running"}. - scan_output_dir walks with a 50,000-file and 12-level cap, skips symlinked files and thumbnails, returns path through _public_path, relative_path in POSIX form, and a truncated flag. - Autotag handlers report exceptions via the shared _public_error; the local copy is removed. Tests close the database before unlinking it. * test: close every database before unlinking and anchor public paths in image tests Each owned test that opens a PromptDatabase now calls close_all() before removing the file, and temp directories holding a database use ignore_cleanup_errors, so the suite tears down cleanly on Windows. The image API fixture lays out a fake ComfyUI tree (<tmp>/output) and pins GalleryConfig.path_anchors to it so responses can be checked for root-relative paths. * fix: validate image file_info before it reaches the generated_images columns link_image_to_prompt stored size, dimensions and format straight from the request body, so a string such as '"><img src=x>' ended up in INTEGER columns and was echoed back to the gallery. Size and dimensions are now coerced with int() and bounded (size 0..10^12, width/height 0..65535), format is matched against a whitelist of media formats, and anything else is stored as NULL while the image still links. Non-dict metadata or file_info no longer aborts the link. * fix: clamp paging on tag listings and the page-derived recent offset tags/stats, tags/{tag}/prompts and tags/filter parsed limit/offset with a bare int(), so limit=999999 was passed straight to SQLite and a huge offset raised OverflowError (500). They now share parse_page_params (limit 1..500, offset 0..MAX_PAGE_OFFSET, 400 on non-integers), and /recent clamps the offset it derives from ?page= the same way. * fix: make the thumbnail progress stream POST-only GET /prompt_manager/images/generate-thumbnails/progress wrote thumbnails to disk, so any prefetch, link preview or <img src> could start a full output-directory scan. The route now accepts POST only (GET answers 405); quality still comes from the query string so the SSE client can keep the same URL shape. * fix: describe images relative to the ComfyUI tree in API responses Gallery and prompt responses carried absolute server paths: /images/output entries (path, root_dir), thumbnails_path, the images/prompt lookup, every image list (image_path plus its normalised file_path duplicate) and the preview images attached to prompt listings. They now go through _public_path: /images/output 'path' is relative to its root (what the delete and autotag endpoints resolve against), 'root_dir' and every other image_path are relative to the ComfyUI base, and file_path is dropped. recent/all/search image lists also gain url/thumbnail_url via the same enrichment the prompt-images route already had. Remaining str(e) error bodies in both modules use the basename-only error text. * fix: check thumbnail directory containment with a real path test clear_thumbnails compared resolved paths with str.startswith, so a thumbnails directory that resolved to a sibling with a shared prefix (out vs out2) passed the safety check and was emptied. It now uses the module's _is_within (realpath + normcase + Path.is_relative_to). * fix: decode tag names from the URL only once The tag lookup, rename and delete routes ran urllib.parse.unquote on match_info values that aiohttp had already percent-decoded, so a tag literally named '%41' (requested as %2541) was handled as 'A'. The second decode is dropped. * fix: validate tags on every tag mutation route add_prompt_tag, add_tags_to_prompt, bulk_add_tags, merge_tags and rename_tag accepted whatever the body carried; a non-string tag raised AttributeError (500) and over-long or control-character tags were stored. They now run utils.validators.validate_tags through a shared tags_error helper and answer 400 with the validator's message. Blank entries in add_tags_to_prompt are still skipped rather than rejected. * fix: export every prompt by paging instead of truncating at 10000 /prompt_manager/export fetched a single page of 10000 prompts and silently dropped the rest. It now walks the table in EXPORT_PAGE_SIZE pages (1000) inside one executor job until a short page is returned. * fix: gate the LoraManager serving roots and accept relative link paths A LoraManager path hand-edited into config.json went straight into find_lora_directories and so into the image-serving allow-list. When a path is configured it must now pass lora_utils.resolve_lora_manager_path (a custom_nodes child whose name contains 'lora'); otherwise, or when the resolver is unavailable, no LoRA directories are served. An empty path keeps the custom_nodes-only auto-detection. POST /images/link now also accepts the relative forms the API hands out (root-relative 'sub/a.png' and ComfyUI-relative 'output/sub/a.png'), resolving them against the allowed roots and anchors; '..' segments in a relative path are refused and unknown files are 404. * fix: review flow uses the served image url and the lora-dirs test pins its config Image paths in API responses are now relative to the ComfyUI tree, so the auto-tag review page can no longer derive a serve URL from image_path; the server always provides url. The lora-dirs serving test assumed an empty LoraManager path, which only held in a checkout without a local config.json. * test: stop leaking a mocked utils.comfyui_integration and mock the IS_CHANGED clock test_image_monitor_singleton replaced utils.comfyui_integration in sys.modules with a MagicMock and never restored it, so running it before test_comfyui_integration failed eight tests. The mock was unused; drop it and install the server stub with setdefault in every owned test module. The PromptSearchList IS_CHANGED test slept 10 ms between two time.time() reads; the Windows clock ticks every 15.6 ms, so it was flaky there. Mock the clock with an incrementing counter instead. * fix: resolve_text follows a node shared by two branches The cycle guard used one visited set for the whole walk, so a string node read twice (a diamond, or StringConcatenate with the same source on both inputs) made the second branch None and the prompt text unknowable. Cycle detection is now scoped to the current path; a separate visit counter keeps MAX_VISITS as the bound on total work. * fix: bound resolve_text on diamond chains with memoisation, depth and length caps A per-path visited set alone makes a chain of diamonds (node i reads node i+1 on both inputs) exponential in depth. Each node's text is now memoised so it is expanded once, every expansion is charged to the shared MAX_VISITS budget, chains deeper than MAX_DEPTH (64) are unknowable so recursion stays bounded, and text longer than MAX_TEXT_LENGTH (100k) is unknowable so StringConcatenate doubling cannot exhaust memory. * fix: static routes resolve under web/ with an extension allow-list /prompt_manager/lib/{filepath} and /js/{filepath} only rejected '..' and a leading '/'. On Windows os.path.join(root, 'C:/x') or a leading backslash re-anchors the path, so any file on the host could be read. Requests are now refused when the path is absolute on any platform (slash, backslash, drive letter, UNC prefix), carries an empty, '.' or '..' segment, has an extension outside the allow-list, or resolves (symlinks included) outside the static directory; the resolved real path is served with its MIME type. * feat: capped JSON body reader and single-flight lock for long admin jobs Add _read_json_body(request, max_bytes=1_000_000) to py/api: reads the body in chunks, answers 413 past the cap (declared or streamed) and 400 for invalid or non-object JSON. Every JSON POST in the admin and logging handlers goes through it; it is exposed as PromptManagerAPI._read_json_body for the other route mixins. Add _acquire_long_job(name) / _long_job_busy_response(label) on the API object: one asyncio.Lock per job family, handed out as an async context manager or None when busy, so a handler answers 409 instead of starting a second scan. The duplicate scan and delete share the 'duplicates' family. * fix: harden database restore and keep server paths out of admin responses Restore: verify_database_file and restore_from_file run through _run_in_executor; the upload is also opened read-only with PRAGMA trusted_schema=OFF and refused when sqlite_master lists a trigger or a view (a prompts database has neither, a crafted one could use them to run SQL against the live data); at most MAX_SAFETY_BACKUPS (10) '<db>.backup_*' files are kept next to the live database, oldest first. Responses: backup_created and the maintenance statistics database_path go through _public_path, and backup/restore failure messages through _public_error, so no absolute server path appears in an admin body. * fix: validate hand-edited gallery roots everywhere and bound output scans A root written straight into config.json skipped the containment check the settings endpoint applies. GalleryConfig.update_config (and so load_from_file) now drops entries that fail validate_gallery_root with a warning, _find_comfyui_output_dir validates the configured directory before using it, and ImageMonitor.start_monitoring validates every directory before scheduling the observer. The duplicate cleaner only deletes files with a media extension and never anything under a thumbnails directory. _collect_media_files walks with os.walk(followlinks=False), skips symlinked files and directories, prunes thumbnails, stops MAX_SCAN_DEPTH (12) levels down and after MAX_SCAN_FILES (50000) files. * fix: bound log file size and backup count in the logging config endpoint max_file_size must be between 1 MB and 1024 MB (bytes) and backup_count between 0 and 50; anything else is a 400. Also close every database connection in the logging test teardown. * fix: private config.json, merged settings save, synthetic test-link path PromptManagerConfig.save_to_file and the settings endpoint write config.json through write_private_json: created with mode 0600 (chmod on POSIX, best effort on Windows) because the file can hold an API key. _persist_settings merges web_ui and gallery.monitoring.directories into the existing file instead of overwriting it, so the integrations section and any other key survive a settings save. The test-link diagnostic no longer stores a client-supplied path; it records a marker file name under the output directory. * fix: image worker thread exits on shutdown and restarts cleanly ImageGenerationHandler.shutdown() clears the pending set, sets a stop event the worker's delay and settle waits listen to, and puts a None sentinel on the queue; the worker exits within the current wait. Each worker owns its queue and stop event so a draining worker never swallows an image scheduled after shutdown. ImageMonitor.stop_monitoring calls it. * test: close all database connections before deleting temp files Windows refuses to unlink an open SQLite file and executor threads keep their own connections, so every owned test that creates a PromptDatabase calls close_all() before cleanup; the diagnostics temp dir ignores cleanup errors. * fix: output scan runs as a background job that survives the browser going away Closing the dashboard tab or clicking outside the scan modal raised ClientConnectionResetError out of scan_images and abandoned the scan mid-batch, leaving a half-scanned database. Two overlapping scans could also run at once. - py/api/scan_job.py: ScanJob fans progress out to any number of SSE subscribers (latest snapshot first, then live events) - POST /prompt_manager/scan starts the scan as an asyncio task, or attaches to the one already running; a write to a dead socket ends that stream quietly while the scan continues - GET /prompt_manager/scan/status reports the running flag and last event - dashboard: dismissing the modal while a scan runs minimizes it to a progress pill, the page re-attaches to a running scan after a reload, a lost stream reconnects, and leaving the page mid-scan asks first * feat: worker-thread setting speeds up the output scan and thumbnail generation Both jobs processed one file at a time on a single core. Pillow decodes the whole image to reach trailing PNG text chunks and releases the GIL while it does, so a bounded thread pool scales almost linearly: on a 1 GB sample of 400 PNGs the metadata pass dropped from 18 s to 2.5 s at 8 threads, and 120 thumbnails from 4.6 s to 0.6 s. - utils/parallel.py: map_parallel(func, items, workers) keeps order, runs on the caller's thread when workers is 1, and caps the pool at the item count - PromptManagerConfig.WORKER_THREADS defaults to half the detected cores, is clamped to max_worker_threads() when loaded from config.json, and is persisted under "performance" - settings API returns worker_threads and cpu_count and validates saves; the settings modal gets a "Worker threads" slider capped at the core count - the scan reads each metadata batch on the configured threads; thumbnail generation processes batches of files per executor job, still emitting one progress event per file Threads rather than processes: this runs inside ComfyUI's server, where forking a CUDA-initialised process is unsafe and the spawn start method would re-import ComfyUI's main module in every child. * fix: scan modal ignores backdrop clicks so a focus click cannot dismiss it Only its own buttons dismiss it: Cancel before a scan starts, Hide (minimize to the progress pill) while one runs. * feat: scan reads prompts embedded in videos; unreadable files no longer log errors The output scan handed every .mp4 to Pillow and logged one ERROR per file. ComfyUI's video savers store the queued graph as JSON in the container's comment tag, so those files carry prompts worth linking. - utils/video_metadata.py reads the comment/description tag through ffprobe with a timeout, normalised to the PNG text-chunk shape so the existing parser handles both; without ffprobe videos are skipped and a single INFO notice is logged - _extract_comfyui_metadata routes videos there and logs unreadable images as warnings, since the scan carries on past them Test hygiene: every aiohttp test base class overrode tearDownAsync without calling super(), so no test client was ever closed. That produced thousands of ResourceWarnings, "Unclosed client session" errors from asyncio that broke log assertions, and doubled the suite's run time. * fix: thumbnail runs skip fresh thumbnails before the cap and report what remains The per-run cap of 10,000 applied to the directory walk itself, so a library with more files than that never had thumbnails generated for the rest, and a run that found 9,987 existing thumbnails spent its whole budget on skipping them. - _plan_thumbnails walks the whole tree (bounded like the gallery listing), counts files whose thumbnail is already newer as skipped without queuing them, and caps only the pending files; the leftover count is returned - start and complete payloads carry skipped and remaining; total_images counts every file considered and the processing rate uses the files actually worked on - the gallery page shows a warning naming the pending count instead of a success toast when a run stopped at the cap * feat(gallery): warn with the pending count when a thumbnail run stops at the cap * fix: thumbnails are written atomically and served under mtime-versioned urls Cards showed "Failed to load" for thumbnails that were valid on disk. The browser had fetched them while the thumbnail run was still writing the file and, because image responses are cached for an hour, kept the broken bytes. The card also had no fallback to the original. - image and video thumbnails are written to a hidden temp sibling and renamed into place, so a partial file is never visible at the final path - every thumbnail_url (gallery listing, prompt images, duplicates, autotag listing) carries ?v=<mtime>, so a regenerated thumbnail gets a new URL instead of the cached old one - a 404 from the image serve route is marked no-store; ComfyUI's cache middleware would otherwise let browsers cache the miss for an hour - image-helpers gains fallbackImageSource(): the original is tried once before a card gives up Test hygiene: the autotag tests now stub ComfyUI's server module like the other API test modules, so the module passes when run on its own. * fix(gallery): display settings apply at load, columns work, media info toggles Default View Mode, Items per Page and Grid Columns were only applied when Save was pressed, never when the page loaded; the column setting swapped a Tailwind class that was never compiled (only 3, 6 and 8 exist) and a wider breakpoint class overrode it anyway; Show Media Info was saved and never read. - gallery-settings.js validates what localStorage holds (view mode, limit option, column range, booleans) and falls back to defaults per value - settings are applied in the constructor before the first load; changing the page size reloads, toggling media info re-renders - the column count sets an explicit grid template through a CSS variable that wins on desktop widths - media info lines honour the toggle in grid and list views - image load failures are handled by one delegated listener that tries the original before showing the failed-to-load card * feat(gallery): duplicate scan runs as a background job with progress; infinite scrolling The duplicate scan was one blocking request with a spinner: no progress, a modal that could not be put away, and a result lost if the tab closed. - POST /prompt_manager/scan_duplicates/stream starts the scan as a background job (same ScanJob fan-out as the output scan) or attaches to the running one; GET /prompt_manager/scan_duplicates/status reports it. The job holds the duplicates lock, so deletes are refused while it runs. - hashing runs on the configured worker threads and reports progress about every percent - the gallery modal shows a progress bar, closing it minimizes to a pill that reopens it, results wait in the pill if the modal was closed, and a reload re-attaches to a running scan Infinite scrolling is a new Display Setting (off by default): the next page is appended when the viewport nears the bottom, pagination controls hide, and the load-more rule lives in list-state.js where it is unit tested. Thumb-size buttons and the column setting no longer override each other. * fix(gallery): Rescan Output Folder drops the server listing cache and rewalks The button only re-requested the listing, which the server answers from a 30-second in-memory cache, so a rescan showed exactly what was already on screen. - POST /prompt_manager/gallery/rescan invalidates the gallery cache, rewalks every output root and returns the file and root counts - the button calls it, closes the settings modal, reloads the folder tree and the first page, and reports what it found * feat: tag search box autocompletes, narrowed by the tags already typed - PromptDatabase.suggest_tags(prefix, with_tags, limit): tags starting with the prefix, counted over the prompts that carry every context tag, context tags excluded, ordered by count then name; LIKE wildcards in the prefix are escaped - GET /prompt_manager/tags/suggest?q=&with=&limit= exposes it (limit clamped to 1..50, context capped at 10 tags, registered before the {tag_name} routes) - tag-autocomplete.js splits the comma-separated input into committed tags and the one being typed, builds the query, applies a pick and handles wrap-around keyboard selection - the dashboard Tags box shows suggestions with counts on focus and while typing, arrow keys and Enter/Tab pick, Escape closes, and each pick immediately offers the next tag narrowed by the previous ones * feat(dashboard): infinite scrolling for the prompt list, saved default plus session toggle - PromptManagerConfig.INFINITE_SCROLL (web_ui.infinite_scroll) is reported and saved through the settings API; only booleans are accepted - Settings modal gains an "Infinite Scrolling" default; saving applies it at once - the results header gains an "Infinite scroll" checkbox that overrides the default for the current session without saving - when on, the next page is appended as the list nears its bottom (same load-more rule as the gallery), page controls hide, a short page keeps loading until the list fills, and a refresh after an edit restarts from the top * fix: first CI run on Windows and macOS; tests now enforce closed database files The first push of the branch ran the Tests matrix on all three operating systems. Every Python job failed; the causes were: - numpy missing in CI: the WD14 engine tests import it (requirements-dev) - macOS: temp directories resolve through /private; the admin API tests now use the real path like the gallery roots they compare against - Windows: the startup banner printed an emoji, which raises UnicodeEncodeError on a legacy-code-page console and aborted loading the whole node; the banner is ASCII now and a test loads the package under a cp1252 stdout - Windows: gallery subfolders came back with backslashes, which neither the folder tree nor the search filter understand; they are "/" joined on every platform. A relative DEFAULT_DB_PATH kept its forward slashes after being joined to the extension root; it is normalised now - Windows: sqlite connections opened with `with sqlite3.connect()` are not closed by the context manager and linger until garbage collection, which holds a file lock there. Production sites (vacuum, diagnostics, restore inspection) close explicitly; a restore that swaps the API's database object closes the previous one; PromptModel keeps a registry so a shutdown or a test can close every instance - Windows: served files were still open when their temp directory was removed (tests now read the body), a log download test compared text mode newlines, two tests compared paths with a hard-coded separator tests/open_handles.py makes the Linux run fail the way Windows would: every database test base checks, right before deleting its files, that nothing in the process still holds them open. * fix: Windows case-folded gallery roots in tests, "/" subfolders everywhere, longer busy timeout Second CI run on Windows: - gallery roots are stored resolved and case-folded there; the admin API tests now compare against that form instead of the mixed-case temp path - the subfolder fallback for images outside every root also uses "/", and the test expects it - a relative diagnostics db path is normalised after joining - "database is locked" under many writers on a slow disk: connections now wait up to 30 seconds (connect timeout and busy_timeout), long enough for a scan or thumbnail run while ComfyUI saves images * fix: serve every gallery video format, validate settings before applying, dedupe output dirs Review findings on #153 (Codex): - the image serve route allowed only three video types while the gallery lists seven, so .avi/.mkv/.m4v/.wmv cards opened to a 403; the serve allow-list now covers every gallery video format - a settings payload with one invalid field still applied the valid ones before returning 400; the whole payload is validated first and nothing changes on failure. webui_display_mode is validated too Third Windows CI run: - the same output directory spelled twice (different case or a link) was listed twice in diagnostics and served roots; both dedupe on the resolved, case-folded path - the root-less subfolder test expects the "/" form Code-quality bot: index definitions are (name, target) pairs instead of implicitly concatenated strings; redundant local asyncio imports and two trivial lambdas in test stubs removed. * docs: version 3.3.0, README changelog and refreshed sections Bump to 3.3.0: new features (worker threads, background scans, tag autocomplete, infinite scrolling, video prompt scanning) and API contract changes (POST-only side-effect routes, relative image paths, paginated search) go beyond a patch release. README: changelog entries for 3.2.4 and 3.3.0, refreshed bulk scanning, gallery settings, search, configuration and testing sections, and the roadmap's recently-completed list.	2026-10-08 18:53:11 -07:00
Vito	6c7e4d5b35	feat: release 3.2.4 — recently used sorting, run counts, positive-prompt image linking (#152) ... * chore: start 3.2.4 * feat: track prompt usage (run_count, last_used_at) with server-side sorting - Migration adds last_used_at and run_count; existing prompts are estimated from linked images (count, newest image time, else created_at). The backfill only touches NULL rows, so it self-heals rows written after a downgrade or an interrupted upgrade. - Every node run records a use, including re-runs of an existing prompt; a tracking failure is logged and never blocks generation. - search_prompts/get_recent_prompts accept a whitelisted sort key (last_used, created, run_count, rating, text); default stays newest first. Refs #90 * feat: sort param on /recent and /search; microsecond last_used_at - /prompt_manager/recent and /search accept ?sort= (whitelisted in the DB layer; unknown values fall back to newest first). - last_used_at is written with microseconds from Python so a re-run in the same millisecond as another save still sorts first; ordering stays correct against millisecond values from the backfill. Refs #90 * feat: server-side sort in admin UI with Recently Used / Most Used and run badges - New web/js/prompt-list-sort.js owns the sort options, validation and request URLs; admin.js fills the dropdown from it and re-fetches from the server on change, so every sort spans all pages (the old client-side sort only reordered the current page). - Default sort is Recently Used; prompt cards show a run-count badge with the last-used time on hover. - The node's Recent button now lists recently used prompts. - Contract test keeps the UI sort keys and the node's key in sync with the server whitelist. Refs #90 * fix: link images and count runs by the positive prompt, not node execution With a negative-prompt PromptManager node in the workflow, every executed PromptManager node pushed onto the image-linking queue, so images could be linked to the negative prompt or to a previous run. ComfyUI also skips unchanged nodes, so re-runs of the same prompt were never counted. - utils/prompt_graph.py finds the PromptManager nodes feeding a sampler's positive input (KSampler, CFG/Basic guiders, ControlNet and combine pass-throughs, PromptManagerText into encoders), falling back to all PromptManager nodes when none can be traced. - Runs are counted by a ComfyUI on-prompt hook when a workflow is queued; prompts not yet saved are counted when the node first saves them. - Nodes receive the queued graph (hidden PROMPT/UNIQUE_ID). Only the run's positive prompt becomes the current prompt; only batch items with linked text are pushed onto the FIFO queue. - The image monitor links from the image's own metadata first, using the positive node, and falls back to the queue for batch items. Refs #90 * fix: address review — count cached runs of linked text, keep repeated pending runs - Text from known pure string nodes (core PrimitiveString/StringConcatenate, WAS Text Concatenate/Multiline, pysssss ShowText) is resolved at queue time, so cached re-runs of such prompts are counted and their images link from metadata. Unknown nodes are never guessed; the node compares its real text with the queue-time resolution and falls back to counting and queueing itself on any mismatch. - A new prompt queued several times before its first run keeps every pending run; the first save records all of them. - Registration flag is a threading.Event; consistent unittest imports; logging instead of print in the init fallback. * refactor: unquote PendingFirstUse annotation so Tuple usage is visible	2026-10-07 13:52:09 -07:00
Vito	8146250274	feat: PromptSearchList batch workflow support & image linking v3.2.2 (#139) ... * fix(search): PromptSearchList crash, filters, and batch output (#52) - Fix empty list crash (OUTPUT_IS_LIST requires at least one element) - Add partial tag matching (LIKE instead of exact) - Add preview and count outputs for batch visibility - Add skip_multipart filter for Clip_N video prompts - Add LoRA-only prompt filter - Collapse newlines for StringOutputList compatibility * feat(db): add tag_partial parameter for LIKE tag matching (#52) Allows search_prompts to use LIKE matching for tags instead of exact. Used by PromptSearchList for discovery-style search. Existing callers default to exact matching (unchanged behavior). * feat(tracker): add FIFO prompt queue for batch image linking (#52) Batch workflows encode all prompts first, then save all images later. The queue preserves encode order so each image links to the correct prompt by position (pop on save matches push on encode). * fix(monitor): multi-strategy image-to-prompt linking (#52) Previous approach relied on live prompt tracker state, which was always stale for batch workflows (all images linked to last prompt). New 5-strategy priority chain: 1. FIFO queue pop (batch-correct by position) 2. Metadata-based DB lookup (extracts prompt from PNG) 3. Prompt snapshot at file-creation time 4. Live prompt tracker 5. Most recent DB prompt fallback * test: add tests for prompt queue and search filters (#52) - 6 queue tests: FIFO ordering, batch simulation, exhaustion - 15 filter tests: newline collapse, Clip_ filter, LoRA-only filter - Fix _make_tracker helper to include queue fields * chore: bump version to 3.2.2 * fix: address code review — regex backtracking and queue expiry (#52) - Replace backtracking-prone LoRA-only regex with subtraction approach (re.sub removes tags, check if anything remains). Fixes CodeQL alert. - Add expiry check to pop_next_prompt — skips stale entries to prevent incorrect linking from accumulated queue entries across sessions. - Cleanup thread now also prunes expired queue entries.	2026-04-10 12:49:59 -07:00
Vito Sansevero	c9ca4be8b8	fix: custom image scan directory ignored due to wrong import (#108) ... start_monitoring() used an absolute import (from py.config) to read MONITORING_DIRECTORIES, which either failed with ImportError or loaded a separate module instance with empty defaults. The configured custom directory was never used, and auto-detection always kicked in. Fixed by consolidating into the single relative import (from ..py.config) that was already used for the MONITORING_ENABLED check.	2026-02-08 05:38:42 -08:00
Vito Sansevero	2737a4b1ef	style: apply Black formatter (line-length=88) to all Python files ... Automated formatting pass across 30 files to establish consistent code style enforced by CI. No logic changes.	2026-02-07 08:30:09 -08:00
Vito Sansevero	4f62066c63	fix: harden nodes, validators, and utility modules ... Phase 0-3 fixes from codebase audit: - Extract shared node logic into prompt_manager_base.py (DRY) - Add input validation guards to validators.py - Fix logging_config.py buffer management - Harden image_monitor.py against race conditions - Fix prompt_tracker.py cleanup edge cases - Fix diagnostics.py import path - Wire config.py server instance correctly	2026-02-07 07:02:10 -08:00
Vito Sansevero	f485825bfb	fix: resolve gallery crash when thumbnails exist and prevent thumbnail-as-image linking ... - Fix AttributeError: 'str' has no attribute 'as_posix' on thumbnail_rel_path in get_output_images and find_duplicates (thumbnail_rel_path is already a string from f-string construction, not a Path object) - Fix NameError: total_images undefined in generate_thumbnails (was defined as total_media, causing progress logging and response to fail) - Add URL encoding for paths in scan_output_dir for filenames with spaces - Exclude thumbnails directory from image monitor to prevent linking thumbnail files to prompts via fallback mechanism - Bump version to 3.0.31	2026-02-03 14:07:23 -08:00
Vito Sansevero	a7d842a51c	feat(settings): add configurable gallery scan directory ... Add ability to configure a custom image scan directory in Settings: - New "Image Scan Directory" field in Settings modal - "Currently Monitoring" display shows active directory path - Settings API now returns/saves gallery_root_path and monitored_directories - Image monitor checks GalleryConfig before auto-detecting - Restart notification when gallery path changes Closes #76	2026-02-02 15:58:54 -08:00
Vito Sansevero	c50a7292c7	refactor(image_monitor): use info log level instead of debug	2026-02-02 15:09:48 -08:00
Vito Sansevero	22593e8514	docs: add comprehensive docstrings to all Python modules ... - Add Google-style docstrings to main node files (prompt_manager.py, prompt_manager_text.py) - Enhance database module docstrings with detailed parameter and return documentation - Add comprehensive docstrings to all utils modules with usage examples - Document API endpoints and configuration classes thoroughly - Update module-level docstrings in all __init__.py files - Ensure consistent naming (ComfyUI_PromptManager) across all documentation - Follow Python best practices for docstring formatting	2025-08-12 06:51:36 -07:00
Vito Sansevero	ecbd2b4ca2	style: Change log level from info to debug	2025-06-19 13:06:09 -07:00
Vito Sansevero	1ea4ac7565	refactor(image_monitor): Replace print with logger	2025-05-28 08:28:18 -07:00
Vito Sansevero	777d5e6a6a	feat(image_monitor): add fallback prompt handling	2025-05-27 10:59:01 -07:00
Vito Sansevero	de1fe46563	feat(utils): add utility modules for diagnostics, hashing, image monitoring, metadata extraction, prompt tracking, and validation	2025-05-24 12:26:31 -07:00