main
49
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vito
|
2cecfcf0ac |
feat(activity): scan new and unresolved files by default, and download a job report (#273)
* feat(activity): scan new and unresolved files by default, all files on request
Scan for metadata now skips, before reading any metadata, the files the
Library already links with a resolved link (one generated_images row with
exactly one pair link), found with one streamed query that keeps only the
scanned paths and compares paths with os.path.normcase on both sides.
Skipped files count as Already known ("Skipped · already linked").
Unlinked, fallback-linked, ambiguous and unresolved files are read again.
POST /prompt_manager/jobs/scan takes {scope: "new" | "all"} (default
"new", anything else is a 400). The review dialog shows the concept's
"New and unresolved files" and adds an "All files" switch (D58). The
legacy /prompt_manager/scan route keeps scanning all files with unchanged
SSE payloads.
* feat(activity): download a job report
GET /prompt_manager/jobs/{id}/report returns the summary, counters, every
outcome group's listed rows with truncated flags, the scope and the log as a
JSON attachment (prompt-manager-<kind>-<YYYYMMDD-HHMM>.json), with no absolute
paths. The job view's Download report saves it through the api-client URL, and
the scan log opens at its newest line and follows new lines only while the
reader is at the bottom.
* fix(activity): re-read known files whose size changed, and review follow-ups
The new scope now skips a known, resolved file only while its stored
generated_images.file_size equals its size on disk; a NULL stored size is
re-read. A file replaced at the same path is read again and its conflict
shows under Needs review. The scan stores each file's size when it ingests
it, and both sides of the known-set match go through normalize_image_path.
A failed known-set read logs and falls back to reading every file, and an
empty request body asks for the default scope. The All files switch is
described by its hint, a lost job's Download report is disabled, the report
toast is skipped once the view is gone, and the scan log opens at its
newest line every time. Fixtures use the real scope shape.
* test(activity): wait for the job view's settled detail read, not a fixed delay
The 200-row cap test now starts counting detail reads once one read has
returned the full listing and none is in flight. The job view re-reads after
an in-flight read only when the summary still lists more than that read
returned, so no redundant read is left due after opening.
|
||
|
Vito
|
e9996996e2 |
feat(views): saved views and Favorites on Images and Library (#268)
* feat(views): saved-views store and API
Add a saved_views table (page images|library, trimmed 1-100 character
name, whitelisted JSON state of at most 4 KB, position, timestamps)
installed through the same validated-savepoint migration as image
favorites. It is optional in old backups and required in a migrated
restore stage; restore rejects a tampered definition, extra indexes and
any row that would fail write validation.
Routes under /prompt_manager/saved-views: GET lists views by position,
POST {page,name,state} returns 201 {success, view}, PATCH /{id} {name}
renames and DELETE /{id} removes, both 404 for an unknown id. The state
must hold exactly the page's toolbar and Refine-this-view keys (Images:
q, category, sort, source, folder, media, tags; Library: q, category,
sort, tags) with their types and sort/media values; anything else is a
400.
Refs #249
* feat(views): Save view, the SAVED VIEWS sidebar and opening a view
The shared toolbar gains the concept's Save view between sort and Filters
on Images and the Prompt library. It opens the concept's simpleInput
("Save this view", "View name", default "Search: <q>" or "My workspace
view", Cancel / Save); an empty name is "Saved view". A save posts the
page's full whitelisted state (toolbar value plus Refine filters) and
toasts "Saved view added to the sidebar.".
The shell adds the concept's SAVED VIEWS label and nav#saved-nav, loaded
with the shell and re-read after every save. Rows use the concept's
links() markup (search icon, name, no count) and link #/saved/<id>.
Opening one mounts its page with the saved state through the page's own
filters, so the toolbar and "Filters · N" (D44) show it; the h1,
breadcrumb and title are the view's name. A removed Source or Folder is
dropped to its default after discovery, so no listing read is rejected;
an unknown id re-reads the list once, then opens Images; a failed list
read offers Retry.
D16 is retired: images-concept and library-concept no longer map Save
view out. New deviations: D49 (no PROPOSED badge) and D50 (no Needs
review row); shell-concept maps Favorites out until Task 3 and compares
a saved row against the concept's. Browser fixtures now wipe
saved_views.
* feat(views): Edit rename/delete and the Favorites saved view
- Favorites is the first SAVED VIEWS row (star icon, name, real count).
The count is the Favorites listing's own limit=1 total, follows the
nav-count rules and is re-read after every confirmed star or unstar.
- #/favorites opens Images filtered to favorites. The workspace listing
takes favorites=1: each selected root's favorites are read through the
image_favorites primary key and matched against the cached scan with the
toggle's case folding; a failed read is an error, never an unfiltered
page. The cursor scope includes the flag.
- Saved Images state gains a required favorites boolean, so Save view on
Favorites reopens the favorites.
- An open saved view's heading leads with Edit: "Edit saved view" with
View name, Delete... (danger), Cancel and Save. Delete... confirms in the
same dialog (never window.confirm). Toasts "Saved view renamed." and
"Saved view deleted."; a delete returns to the view's plain page.
- shell-concept no longer maps Favorites out (Needs review stays, D50).
Recorded as D51 in TESTING.md.
* fix(views): open stale saved views safely and finish the review fixes
Startup and restore now check only the saved_views structure (table,
columns, types, sizes, a JSON object state) and never the value rules,
so a row saved under older rules no longer stops the database from
opening. Reads normalize each view: missing keys take their defaults,
unknown keys are dropped, invalid values default and the name is made
displayable. POST and PATCH still validate strictly.
Only request validation (SavedViewInvalid) maps to 400; database and
internal errors are a logged generic 500. A saved view no longer
replaces the plain Images or Library return state, a toolbar change
while sources load keeps a saved Source and Folder, a state over 4 KB
is refused in the browser with its own toast, the delete confirmation
skips its failure toast after the page is gone, the Edit dialog reuses
the shared field helper, and quick star toggles coalesce into one
Favorites count read.
* test(views): cover leaving a pending saved view and add its coverage floor
A saved view left while the list or a rename is still pending leaves
the next page alone; the shell drops two unreachable heading guards.
workspace/saved-views.mjs gets its frontend coverage baseline entry;
no existing floor changes.
* fix(views): a corrupt saved-view state no longer blocks startup
At startup a saved_views row whose state is not a JSON object is
logged as a warning by id only and left in place. The listing and
rename skip it, so its #/saved/<id> link behaves like a missing view.
Restore stays strict and still rejects a backup holding such a row.
|
||
|
Vito
|
1ff2ddce05 |
fix(inspector): fall back to the positive's category and tags for recipe-linked images (#267)
* fix(inspector): fall back to the positive's category and tags for recipe-linked images Maintainer ruling D39 (2026-10-21): a recipe-linked image shows the recipe's own category and tags when set, otherwise the linked positive prompt's, each field on its own and without merging tag sets. Set category (#259) and the Images category and Tags filters work on the positive, so the panel now agrees with them. The inspection library replaces metadata_scope with category_scope and tags_scope so the category chip's hidden scope prefix (D21) and the tag chips' filter-button rule (#260 D2) follow the value actually shown. History's media inspector uses the same shared rule. The concept fixture's positive_links workaround is gone: the comparison now matches the concept without it. * test(inspector): assert the D39 fallback tag chips in their saved order * fix(inspector): treat Unicode whitespace as a blank History recipe category * test(inspector): cover mixed-scope panels, invalid library scopes and the skipped positive tags query |
||
|
Vito
|
10a4847a6b |
feat(history): concept heading, negative variants and compare (#265)
* feat(history): concept heading and summary The Prompt history page takes the approved mock's heading and summary surface (docs/concepts/workspace/app.js renderHistory a591-598, #249 F1): - Heading description is "Prompt history · one positive, one shared gallery"; the actions stay "← Prompt library", "Edit metadata" and button.send "Send to ComfyUI" (aria-label "Send prompt pair to ComfyUI: <title>"), on one row at 1600. Send is disabled only while the selected pair's roles are loading (D24). - The summary is article.surface > span.chip category, the positive and selected negative prompt blocks, then .card-meta "N tracked runs · N images · N negative variants". Tracked runs come from the recorded history window's recorded_run_count (hidden until a read completes, kept while a refresh is in flight); images and variants from the saved positive's image_count and variant_count. - The summary's copy/send actions are named after the prompt, as in the concept ("Copy positive prompt: <title>"); mountPromptBlocks gains setTitle so an in-place metadata edit renames them without a remount. - mountRecordedHistory gains an optional onRecordedCount callback. Selector mapping: - .library-surface (history summary) -> .library-history article.surface - .library-surface .library-chip -> article.surface > span.chip - "Copy|Send <role> prompt[ to ComfyUI]" in "Saved prompt texts" -> "...: <title>" - heading p.muted "One positive prompt. Reusable negative variants. One shared gallery." -> "Prompt history · one positive, one shared gallery" - Unchanged: region "Saved prompt texts", .library-variants, .library-shared-gallery, .recorded-history (still .library-surface). Tests: workspace-history "history heading actions stay on one row at 1600 and are enabled after load" and "summary meta uses recorded runs" (both failed before), a prompt-blocks setTitle unit, and the mapped selectors in workspace-visual, workspace-library-editor and the canonical comparison. * feat(history): concept negative variants panel The history's Negative variants panel is the concept's (app.js a598, style.css 1059-1154): section.surface.timeline-panel with the panel heading (h2, p.small "Choose a red dot to change the negative and filter this gallery."), row actions Compare negatives (aria-pressed; its section is Task F3), + Add negative and the approved Show archived; a .timeline of .positive-lane and .variant-lane rows (--lane-offset 38 + 47i) with the "−" .variant-dot, "<name> · N runs" and a .run-dot when the pair's run_count > 1; and .timeline-lower with the legend and All images for this positive. D26: All images for this positive opens the concept dialog with the embedded, bounded (Load more) media gallery instead of filtering the shared gallery. The run dot opens Recorded run for the pair's newest recorded execution, else a Repeat run dialog with the pair's newest linked image. Backing data: - library variants carry prompt_pairs.run_count; - GET /prompt_manager/library/{positive_id}/history/pairs/{pair_id}/newest reads the newest run in the positive's exact raw-text scope whose current-epoch saved-pair hint still resolves to that pair (the get_history_run hint rule), inside the fenced history snapshot. - workspace-dialogs gains openContentDialog (onClose releases the body). - mountRecordedHistory returns openRun. D45 (new deviations): - a dot is named "Select negative <title>" when no other loaded lane has that title, else "Select negative variant N", so names stay unique; - the bounded "N of M saved negative variants" count stays a visually hidden status; "Saved combinations, not an execution timeline." is gone; - archived lanes (#235) are concept lanes marked .library-variant-archived at .6 emphasis with their Restore button; coarse dots keep a 47px target; the heading, legend row and All images button wrap at 390px and 200% zoom. Retired CSS: .library-branches, .library-positive-lane, .library-variant-lane, .library-variant-dot, .library-variants, .library-panel-heading, .library-variant-footer, .library-variant-summary. Selector mapping: - .library-variants -> .library-history .timeline-panel (region "Saved negative variants" unchanged) - .library-branches -> .timeline; .library-positive-lane -> .positive-lane - .library-variant-lane -> .variant-lane; .library-variant-dot -> .variant-dot - .library-variant-archived and .library-variant-restore unchanged - .library-variant-footer / button:last-child -> .timeline-lower / button - .library-variants > p.muted -> .timeline-panel .panel-heading .small - "Select negative variant N" -> "Select negative <title>" for titled pairs (history, native library, inspector, visual and real-host fixtures); untitled pairs unchanged - lane text "· N linked image(s)" -> "· N run(s)" - All images for this positive: aria-pressed filter -> dialog "All images for this positive"; tests that used it for the whole gallery select the lane linking the image, or read the dialog - variant panel row actions gain Compare negatives first Tests: new workspace-history "variants panel matches the concept", "run dot appears only for pairs with more than one run and opens its run", "all images dialog lists every image for the positive"; new workspace-library-editor "show archived and restore still work" (all four failed before the change). Python: variants carry run_count, newest pair run (DB and route). Unit: content dialog. The history fixture counts two canonical pairs and links one image to Clean geometry. * feat(history): compare negative variants Compare negatives now opens the concept's comparison (app.js a599-616, style.css 1155-1179) as a section.surface right after the timeline panel: h2 "Compare negative variants", p "Baseline compared with <name>. Choose another red dot to change the comparison.", a .compare-grid of two .compare-item negative blocks (baseline, selected), p.small "Added terms", then span.diff-added "+ term" per added term or p.small "No added terms.". Choosing another red dot updates the same section in place; pressing Compare negatives again removes it. D25: the baseline is the oldest active variant; with no selection the selected side falls back to it, as the concept's v does. The pure addedTerms(base, selected) in history-state.mjs is the concept's rule: comma split, trim, drop empty terms, exact case-sensitive difference that keeps the selected order and repeats. D46 (new deviations): - lanes are newest first, so the baseline is the lowest-id active variant, not the first lane; while the variant page has more, the oldest is not loaded and the section says "Load more negative variants to reach the baseline." instead of guessing; - the two blocks are named "Baseline" and "Selected" (regions and copy/send actions), so no name repeats on the page; - Compare negatives is disabled when there is no active variant. Selector mapping: none changed; the section is new (.library-history .timeline-panel + section.surface). Tests: unit "added terms split, trim and exact-diff like the concept"; workspace-history "compare toggles, follows the selected dot and stacks at 390" and "compare waits for the page holding the oldest active negative" (all failed before the change). * test(history): guard the newest-pair route and record D45/D46 TESTING.md lists the Prompt history page's spec decisions (D24-D26) and its new deviations D45 and D46 beside D23 and D40-D44, notes under D46 that production opens on the newest lane while the concept opens on its baseline, adds GET /prompt_manager/library/{positive_id}/history/pairs/ {pair_id}/newest to the history route list with its guards, and documents the detail variants' run_count. Tests: the newest-pair read skips a stale reference_epoch row, stops at 16 stale-hint candidates (17 newer stale rows hide an older valid run and report {run: null}), and rejects a negative-only or deleted positive; the route answers 404 for an ineligible positive, 503 storage-unavailable for storage failures without leaking details, and 409 history-changed for a restore during the snapshot. * fix(history): run-dot touch target, unique dot names and dialog cleanup - D45: under pointer: coarse the 12px run dot gets a 44px ::after target, and while a run dot shows the lane text wraps within 139px before it. - D45: dot names compare trimmed titles and never use a title that reads like a fallback "variant N", so names stay unique. - All images and Repeat run share one dialog-scope slot: each open replaces the previous scope instead of adding a cleanup per open. - A run-dot lookup that settles after the view went stale opens nothing. - Retire the unused .library-main .library-chip rule and the stale "Saved combinations" comparison note. Tests (workspace-history, Chromium and touch): the coarse run-dot target and lane width; a failed newest lookup shows the toast and Repeat run; Recorded run that cannot open (recorded history failed) falls back to Repeat run; an archived pair's Repeat run reads archive=archived; Compare negatives is disabled with no active variant; dot names stay unique. * fix(workspace): one inert handle for aborted history and prompt-block mounts An aborted parent no longer gets a separate never-called stub per method: mountRecordedHistory returns its hoisted openRun (it answers false before reading later state), and mountPromptBlocks returns one shared no-op for update, setState, setTitle and dispose. This keeps the history.mjs and prompt-blocks.mjs function-coverage floors (98) without lowering them. Tests: test:coverage (Chromium and touch, 922 workflows) passes for all 61 files; it failed before on history.mjs 94.44 and prompt-blocks.mjs 93.75 function coverage. |
||
|
Vito
|
86d82186ee |
feat(images): concept Images page — toolbar, cards, star, viewer, compared against the mock (#258)
* feat(workspace): shared concept toolbar and dialogs
Add the concept's shared page toolbar (search.toolbar with the search
field, category filter, sort and Filters) and its dialogs in one modal
shell: "Refine this view" (Source / Folder / Media / Tags, Apply filters),
"Import & inspect" with only the real operations (Open inspector, Rescan),
and the "Copy prompt" select-to-copy fallback. The sort lists the
concept's three options first (D14); there is no Save view (D16). Every
dialog returns focus to its trigger on close or Escape.
Refs #249
* fix(workspace): bind shared dialogs to the page and its route
- Drop the invented Refine note; every field is real (D14).
- Ignore a late close event once the shell was reopened or replaced, so
an action that opens the next dialog keeps it open.
- Dialog openers require the page scope; disposing it closes and removes
the shell unless a newer page's dialog is open. Every route change
closes the shared dialog before the next page mounts.
- Clear filters also reports category '' so the page clears the toolbar
category, as the concept does.
- The toolbar requires its scope and falls back to the first sort for an
unknown value; dialog callbacks are validated up front.
Refs #249
* feat(images): concept toolbar, filter row and caption
The Images page now renders the approved mock's heading, shared toolbar,
filter row, section caption and infinite continuation (#249, Task C2).
- Heading: "Your generations, their prompts, and the next idea." with
Import (upload icon) before New prompt (primary).
- Toolbar (shared mountToolbar): search over filenames, linked prompt
text, titles and tags; category (the Library's categories); sort
Recently added / Name A–Z / Highest rated, then Oldest first and
Name Z–A (D14).
- Filter row: Grid/List/Masonry segmented control, Thumbnail size,
the scope chip ("All output folders" or "<source> · <folder>") and
"Group by positive prompt →".
- Caption: "N of M images · K positive prompts" from positive_count; a
plain cursor page keeps the first page's count and an unknown count
hides the "· K positive prompts" part. The date span shows
"<Today|Yesterday|weekday> · <Month D>" of the newest loaded result
under a date sort and is hidden otherwise (D17).
- Continuation: "N of M images · more load as you scroll" with Load
more, the concept's end copy, Retry for a failed page, and the
concept's "K more images loaded. N of M." toast.
- The listing sends category, tags and sort=rating (PR B params).
Old control -> new control:
- "Search filenames" field -> toolbar search "Search workspace"
(placeholder "Search prompts, tags, filenames…"), also matching
prompt text and tags
- Source select -> Filters -> "Refine this view" Source
- Folder select -> Refine Folder (all sources' folders and ancestors)
- Media select -> Refine Media
- (new) Tags -> Refine Tags; Category -> toolbar category select
- Sort select (Newest/Oldest/Filename A–Z/Z–A) -> toolbar sort
(Recently added/Oldest first/Name A–Z/Name Z–A, plus Highest rated)
- Layout buttons -> .segmented Grid/List/Masonry in the filter row
- Thumbnail size field -> label.small + select#image-size
- "Group by positive prompt" link -> button.chip "Group by positive
prompt →" (#/library)
- Refresh -> Import -> "Scan an output folder" -> Rescan (D15)
- "Current gallery" link -> removed; Overview's tool cards link the
legacy gallery (D29)
- "All images" h2 + count -> .section-caption #result-count
- .image-continuation -> .infinite-continuation (Retry kept)
Legacy behavior kept: video items, keyboard grid navigation, scroll and
filter restoration, bounded DOM, continuation retry, source-change
recovery (now disabling the toolbar, Filters and Import). The Filters
and Import dialogs close on route change and browser Back.
The shared toolbar (C1) called .some() on select.options, which is an
HTMLOptionsCollection in a browser (only the unit tests' fake DOM gave
an array); setValue and setCategories now spread it first. The layout
segmented control may wrap at extreme zoom (D31) instead of scrolling
the 390px page sideways.
The retired .image-filters, .image-view-controls, .image-toolbar,
.image-layouts, .image-collection, .image-continuation and duplicated
.image-workspace grid rules are gone from images.css.
* test(workspace): model select.options as a browser collection
The toolbar units' fake DOM returned select.options as an Array, so the
toolbar's .some() call passed there while it threw in every browser
(fixed in
|
||
|
Vito
|
6c82098207 |
feat(images): data for the concept cards — linked titles, seed, search and filters, counts, favorites (#257)
* feat(images): link summary on workspace image items
* fix(images): carry title source and cut first-line titles like the Library
Summaries gain title_source (stored or text). The inspector always shows a
stored title and confirms only a first-line title against the file, so a
stored title equal to the first line is no longer withheld. A link-title read
failure no longer marks a readable link unavailable. First-line titles over
80 characters end with an ellipsis, as the Library's title() does. The card
gate for unconfirmed first-line titles is pinned as an expected failure for B2.
* feat(images): unambiguous seed on workspace items
Workspace items gain `seed`, read with the shared generation-parameter
parser from the prompt/workflow records the header open already loaded
(each at most 2 MiB; no extra file reads, videos give None). A
placeholder or non-integer seed gives None. JSON integers beyond 2**53
are recovered exactly, so the card prints the digits the inspector shows.
The same parse confirms first-line card titles: a `text` title shows only
when the file's parsed positive gives the same title. One helper applies
that rule on the card and in the inspector, which now reads the positive
from its own inspection records with the same parser.
* fix(images): a deeply nested record no longer fails the workspace listing
The Python parser recurses over nested arrays where JavaScript String()
does not, so a crafted 2 KB PNG turned every workspace page holding it
(and its inspect route) into a 500. Card facts now treat RecursionError
like any record the parser cannot use: no seed and no confirmed title.
Also: Library previews skip the card parse they never show; the parse
runs after the file handle closes; the exact-seed read uses the one JSON
reader (parse_js_json(..., exact_integers=True)) and the public
number_string; record size is decided from the text length unless only
the UTF-8 byte count can tell.
* feat(images): search and filter workspace images by linked prompts
The workspace listing now takes category (exact), tags (comma list, all
required) and sort=rating, and q also matches the linked positive's text,
Library title, pair title and tag names. One bounded, parameterized SELECT
(get_image_link_matches / get_image_paths_matching) returns normalized
paths of unambiguously linked images; the listing is intersected with it.
Unlinked and ambiguous files are excluded by a category or tag filter and
sort last by rating, with a newest-first tie-break. The cursor scope covers
the new filters and sort; unknown values are still rejected.
The derived listing values and sort orders are cached per gallery scan so a
50,000-file listing with 20,000 links pages well under 150 ms at p95.
* test(images): count the filter budget everywhere, time it in one job
The 150 ms wall-clock budget failed under the CI coverage tracer, so it
is split. test_filter_work_is_bounded runs on every job and under
coverage: one filter SELECT per request, the ambiguity probe is an index
search (EXPLAIN QUERY PLAN), the per-scan values are built on the cold
request only, and a cursor page calls the sort key at most
ceil(log2 n)+3 times. test_filter_budget runs only with
PROMPTMANAGER_PERF_BUDGET=1 and no tracer, asserts the best of three
rounds' p95, and has its own non-required "perf budget" job. TESTING.md
records the convention.
* fix(images): rating sort works however many link rows the database holds
The filter SELECT counted every generated_images row against a 500,000
row cap and materialized all of them first, so a large history made
sort=rating answer 400 with nothing to narrow. The rows are now streamed
and only paths in the listing's normalized set are kept, so memory is
bounded by the listing; the cap and LinkFilterTooBroad are gone.
* fix(images): rebuild workspace listing values only where the scan changed
A TTL rescan that finds the same files reuses the previous values and
sort orders; one that finds a few changes rebuilds only those files.
Gallery invalidation and a gallery-roots change drop the derived values,
so old roots no longer hold memory. category is trimmed like tags.
* fix(images): a monitor invalidation rebuilds only the new files' values
The image monitor invalidates the gallery whenever it sees a new file,
and that dropped every derived workspace value, so the next request paid
a full cold build. Invalidation now keeps the per-file values and drops
only the sort orders; the rescan reuses every unchanged file and builds
only new or changed ones. A gallery-roots change still drops everything.
* feat(library): expose run count, last use and variant totals
* fix(library): an out-of-range last use reads as unknown, not a failed list
* perf(images): read positive counts only on a plain listing's first page
* feat(images): persist image favorites
Favorites are stored in image_favorites keyed by the gallery root label
and the relative path, set and cleared through PUT/DELETE
/prompt_manager/images/favorite with a {root_index, path} body validated
against the rooted listing. Workspace image cards carry favorite: bool.
The table is created by migration, travels with backups, is optional in
old backups and required after a restore's migration.
* fix(images): key favorites by the real root and case-normalized path
Folder names are not unique across gallery roots, so two roots shown as
"output" shared each other's stars. Favorites are now keyed by the
root's real, case-normalized path (server-side only) and a
case-normalized relative path, so a Windows case-only rename keeps its
star while distinct Linux files stay distinct. Toggles look the file up
in a per-scan dictionary instead of walking the listing.
* fix(images): favorites lookup follows monitor invalidation and the stored key bound
A monitor invalidation now also drops the favorites lookup, which held the
previous scan list until the next toggle. PUT refuses a favorite whose root
or relative key would exceed the schema's 4096-character bound, so the live
database keeps passing startup and restore validation.
* docs(testing): workspace data fields for the concept pages
* fix(images): card facts read the inspector's projected graphs
The inspector hands its browser parser only graphs the inspection
projection parsed (strict JSON, depth 64, node ceiling); a deeper record
is "limited" there, so the legacy fill never runs and no seed shows. The
card parsed the raw records instead and showed a seed the inspector did
not, e.g. for a 900-deep class_type beside a normal sampler.
project_graph() is now the one projection step, used by the inspection
response and by the card. The card runs the shared parser on the
projected graphs, the same values the browser receives: big integers
arrive as their exact digits, so the separate exact-integer re-parse and
the parse_js_json flag it needed are gone.
* fix(images): send the card seed as a decimal string
JSON.parse rounds a number past 2**53, so a listing seed such as
9223372036854788153 reached the page as 9223372036854788000 while the
inspector showed the file's digits. Every non-null card seed is now its
decimal string.
The Node parity harness now JSON.parses the real listing response and
compares String(seed) with the seed the inspector shows, including a
seed past 2**53. TESTING.md describes the string contract and drops the
removed exact re-read.
* fix(images): bound digit-string seeds and isolate each card's parse
A seed recorded as a 5000-digit string reached int(), which raises
ValueError past Python's integer string limit, and the listing answered
500. A digit-string seed now has at most 20 digits (2**64 fits), and the
conversion falls back to no seed on ValueError.
Each card's parse is also guarded in the page builder: an unexpected
error in one file's records logs a warning and gives that card no seed
and no first-line title, and the rest of the page still lists.
* fix(images): filename search survives a failed prompt-link read
On main a workspace q matched only file paths, so search worked with a
broken database. Since search also matches linked prompts, any q turned
a failed link read into a 500. A q without category, tags or the rating
sort now falls back to filename matches with a null positive_count and
logs the failure with its traceback; those linked filters still fail.
* fix(images): card reads PNG records with the inspector's reader
The card read PNG text through Pillow's header info, where a later
duplicate prompt chunk silently wins and text after IDAT is missed. The
inspector's bounded reader refuses conflicting duplicates and reads text
after IDAT, so a file with two different prompt chunks showed a card
seed the inspector refused.
PNG cards now read their records with read_inspection_file, the reader
the inspector uses. When it refuses a file the card keeps its
dimensions from Pillow and shows no seed. Other formats still use
Pillow's header info, as the inspector's header read does.
* perf(images): read each card's header facts once per gallery scan
Every page parsed each card's prompt and workflow records again, about
3.4 ms per 50 KB record set and 25 ms per 400 KB workflow, inside
ComfyUI's server process, on every scroll-back and refresh.
Dimensions, the seed and the parsed positive's display title are now
cached per root by (path, mtime), beside the scan's per-file values and
invalidated the same way: a rescan keeps unchanged files and prunes the
rest, a roots change drops them all, and only files of the current scan
are stored. An unreadable header is not stored. Links and favorites
are still read on every page.
* ci: run the perf budget on main pushes and on demand only
The wall-clock budget has under 2x margin on shared ubuntu runners, and
a red exact-head check blocks a merge that must never be rerun to green.
The job now runs on each push to main and on workflow_dispatch, not on
pull requests. TESTING.md says a red run on main means fix the code.
* fix(library): cut display titles at 80 UTF-16 units, never inside a pair
The server counted code points while the Library's JavaScript counts UTF-16
units, so a first line of emoji kept 80 emoji on cards but showed 40 in the
Library. Both now cut at 80 UTF-16 units and drop a character that would
straddle the limit instead of leaving half a surrogate pair.
|
||
|
Vito
|
1ba9e22824 |
fix(database): stop SQLite write-lock starvation under concurrent saves (#246)
* test(database): require identical WAL/NORMAL settings on managed connections (#242) Concurrent writers failed with 'database is locked' on Windows runners despite the 30 s busy timeout: every WAL commit flushed to disk while it held the write lock, so polling waiters starved. These tests pin the shared per-connection settings, including synchronous=NORMAL in WAL mode. * fix(database): stop WAL commits from holding the write lock across a flush (#242) Every managed connection now goes through configure_connection: WAL, foreign keys, the 30 s busy timeout and, when WAL is active, synchronous=NORMAL. Under FULL each commit flushed to disk while it held SQLite's only write lock; on slow Windows disks a burst of writers (a scan beside ComfyUI saves, or the concurrency tests) starved a polling waiter past the busy timeout and it failed with 'database is locked'. With a 60 ms emulated flush on Linux, 8 threads x 50 save+use calls went from 49.6 s with a 32.1 s worst call (and 'database is locked' in the unit test) to 0.7 s with a 0.4 s worst call. NORMAL in WAL mode keeps the database consistent after an application crash; only an OS crash or power loss can drop the most recent commits. In-memory or non-WAL databases keep SQLite's FULL default. * test(history): hold the first concurrent writer open past the bounded wait (#242) The concurrent replay tests expected every peer to succeed, but each peer only waits 250 ms for BEGIN IMMEDIATE by design. On a slow Windows runner three queued commits exceeded that and a peer raised 'database is locked'. Holding the first writer's transaction open for 500 ms makes the same failure deterministic on any host (20/20 locally). * test(history): let concurrent replay peers queue like ordinary writers (#242) History writes deliberately give up after a bounded 250 ms wait so they never stall ComfyUI; the existing bounded-timeout test still proves that. The concurrent replay tests assert that every peer succeeds, which is a serialization and first-capture property, so they now raise the budget to the shared 30 s busy timeout while the first writer is held open. The budget is one named constant, EXECUTION_BUSY_TIMEOUT_MS, shared by the transaction helper and the background writer. Before: 20/20 'database is locked' with the held writer. After: 0/20. * test(history): wait on restore handoff events, not a 5 s clock (#242) The retired-owner projection tests park a history read at a barrier while the test performs a full multipart restore, then release it. The barrier gave up after 5 s and raised inside the route handler, so on a slow Windows runner (a 694 s suite run) aiohttp answered with its text/plain 500 and the test reported 'history route missing'. Every wait already ends on an event; the ceiling now only stops a broken test from hanging (60 s). With restore slowed by 5.5 s locally, all 8 subtests failed with text/plain before and pass after. * docs(testing): add the concurrent SQLite test rule to the Windows preflight (#242) |
||
|
Vito
|
ab333d3838 |
feat(vocabulary): backend for native tags and categories (#241)
* feat(tags): keep unused tags in the shared vocabulary * feat(vocabulary): add bounded tag and category reads * feat(vocabulary): guard tag create, rename and merge * feat(vocabulary): preview and apply category consolidation * refactor(vocabulary): share canonical token hashing * test(vocabulary): prove retained tags survive restore * test(vocabulary): cover category paging tie-break and clarify contract docs * test(vocabulary): assert rejected tag creations write nothing |
||
|
Vito
|
39dcc40114 |
fix(maintenance): keep titled and history-referenced prompts when pruning (#239)
* fix(maintenance): keep titled and history-referenced prompts when pruning prune_orphaned_prompts now also keeps prompts with a prompt_library_state row (titled or explicit Library parents, whose state would cascade away) and prompts that are the currently valid navigation hint of a recorded run (execution_run_contexts positive/negative_prompt_id in the current storage epoch whose exact raw snapshot text still matches). Stale hints that history reads already ignore do not block pruning. Fixes #237 * fix(maintenance): mirror unique-text hint rule and compute live hints once The prune exempted a hinted prompt even when another prompt shared its exact raw text, although history reads treat such a hint as dead. The live-hint set now applies the reader's rule (current storage epoch, exact raw text matching exactly one prompt, which is the hinted one) and lives beside _prompt_hint in execution_contexts as LIVE_HINT_CTES. The set is materialized once per statement and excluded with NOT IN, so the prune no longer scans execution_run_contexts once per prompt. No schema or index change. Refs #237 |
||
|
Vito
|
e5da4e1d9d |
feat(library): backend for native recipe editing, pair archive browsing and portable export (#233)
* feat(library): persist recoverable parent state * feat(library): expose versioned parent metadata reads * feat(library): protect parent metadata and archive writes * feat(library): save native recipes with atomic outcomes * feat(library): download complete portable library json * fix(library): name export owners list positives per spec * fix(library): normalise native category and harden restore and id errors * fix(library): measure stored titles past embedded NULs |
||
|
Vito
|
ecd21165a7 |
feat(history): add chronological runs and exact output inspection (#227)
* feat(history): add bounded chronological read projections (#209) * fix(history): validate internal continuation positions (#209) * feat(history): expose fenced history windows and run details (#209) * fix(history): release retired projection connections (#209) * feat(history): resolve reported outputs through current authorization (#209) * fix(history): reject invalid inspection targets (#209) * feat(history): add scoped history state and current-file adapter (#209) * fix(history): preserve slideshow and clear stale reentry actions (#209) * feat(history): render recorded executions and frozen run details (#209) * fix(history): recover receipts and refresh capture evidence (#209) * test(history): verify chronological history across restore and host modes (#209) * fix(history): enforce chronological marker visual parity (#209) * fix(history): resolve final chronological review findings Address #209 recovery, bounded visit restoration, exact receipt availability, and portable regression coverage. * test(history): repair Windows fixtures and review feedback Address #209 and PR #227 native path, cleanup, media and review-comment expectations. Document Windows portability preflight without changing production behavior. * ci: raise the viewerjs job budget for the larger browser suite The cross-browser workflows suite grew from 396 to 496 tests on this branch, which pushed the whole viewerjs job to about 25.5 minutes against the 25 minute cap set in #181. Two exact-head runs on this branch timed out without any test failing: run 39173019283 was killed on test 474 of 496, and run 39175125474 printed "496 passed (12.8m)" and was then cancelled during Playwright shutdown. Raise only the job budget. Browser projects, workers and per-test timeouts are unchanged. Splitting the browsers into parallel jobs is tracked separately. |
||
|
Vito
|
10197026be |
fix(database): refuse unsafe restore file fallback (#226)
* fix(database): refuse unsafe restore file fallback (#223) * fix(database): retain safety backup limit after restore refusal (#223) |
||
|
Vito
|
d9537a726c |
feat(workspace): complete shared inspector and collection viewer (#224)
* feat(inspector): expose bounded physical file evidence * fix(inspector): distinguish ambiguous saved associations * feat(inspector): derive conservative generation facts * fix(inspector): guard fact ancestry and inspection state * feat(inspector): share complete file details and exports * fix(inspector): preserve fact states and touch targets * feat(viewer): navigate the loaded media collection Refs #216 * fix(viewer): refit after current stage geometry changes Ref #216. Observe the current media stage after readiness and wrapped title changes without rebuilding focused controls or resetting transforms at unchanged dimensions. * feat(viewer): add lifecycle-safe slideshow and media preferences * fix(viewer): resume gallery continuation after close (#216) * fix(images): guard source recovery and wrap viewer actions Clear stale root and folder intent, cancel old pages, and wait for current source discovery before loading a refreshed collection. Preserve safe retry through failed, superseded, and disposed discovery. Wrap narrow viewer footer actions and wait for measured gallery viewport settling in responsive checks. Refs #216; addresses PR #224 review. * test(viewer): await gallery layout after viewport changes Require actual inner, root client, and document widths to agree after each responsive-case viewport and CSS zoom transition. Keep the existing exact frame, overflow, focus, and coarse target assertions unchanged. Observed PR #224 CI initial mobile resize at537px before pending card relayout settled to356px. Refs #216. |
||
|
Vito
|
4ef236aaf7 |
feat(history): persist exact run output references (#214)
* feat(history): define bounded output reference contracts * feat(database): migrate execution output evidence to schema v3 * feat(history): persist immutable run output references * test(history): isolate mocked node gallery monitor * feat(history): capture bounded output descriptors with shared leases * fix(history): authenticate payload owners and release failed captures * feat(history): attach output evidence to exact finalized attempts * test(history): verify run output evidence on real ComfyUI * fix(history): release idle worker payload references * docs(history): clarify defensive lease cleanup * test(history): align output fixtures with native path styles * test(history): separate restore completion from writer drain * test(history): bound batch drains by progress and total time |
||
|
Vito
|
bae542c9ac |
feat(history): persist exact execution prompt contexts (#213)
* feat(history): define exact bounded execution contexts * fix(history): retain observable sources and validate evidence * feat(history): migrate execution contexts to schema v2 * feat(history): persist immutable execution context snapshots * fix(history): preserve package identity for execution contexts * feat(history): bound context payloads beside lifecycle finalization * fix(history): fence context commits and resolve stored run identity * fix(history): bound persisted identifier bytes before restore reads * feat(history): bind exact runtime context to execution attempts * test(history): discover owner invalidation callback regression * feat(history): capture cached prompt strings without mutating host caches * test(history): verify durable contexts on real ComfyUI hosts * fix(history): scope transforms to captured source paths * test(ci): update execution context tooling fixtures (#209) * fix(history): harden context validation and worker fixtures (#209) * fix(history): honor close deadlines after early joins (#209) |
||
|
Vito
|
763e6b37b1 |
feat(history): record durable ComfyUI execution attempts (#212)
* feat(history): normalize bounded execution evidence (#209) * fix(history): bound lifecycle kind checks before hashing (#209) * feat(history): persist execution attempts with restore fencing (#209) * test(api): bootstrap standalone backup restore fixtures (#209) * feat(history): isolate bounded asynchronous ledger writes (#209) * feat(history): observe actual queue attempts without altering execution (#209) * fix(database): honor package-scoped configuration (#211) * test(history): exercise real cached and headless executions (#209) * fix(history): reject unsupported uniqueness and isolate package fixtures (#209) * test(history): support Python 3.10 package fixture cleanup (#209) |
||
|
Vito
|
fdba6fd012 |
feat(workspace): group saved negatives in the native Prompt Library (#210)
* feat(workspace): group saved negative variants in native library (#208) * fix(library): preserve full name order and batch previews (#208) |
||
|
Vito
|
caa9649f32 |
feat: reuse positive and negative prompt combinations (#195)
* feat(db): add pair schema and stage restore migrations for #193 * feat(database): persist reusable ordered prompt pairs (#193) * feat(api): expose reusable pairs and atomic metadata enrichment (#193) * feat(metadata): resolve canonical prompt pairs with bounded provenance (#193) * feat(scan): enrich image pairs without guessing or duplicate links (#193) * fix(database): preserve pair tags during global maintenance (#193) * feat(tracking): enrich live image pairs and cancel scans safely (#193) * feat(nodes): encode paired prompts and count cached recipe runs (#193) * fix(database): validate restored pair ownership and provenance (#193) * feat(admin): report pair scan outcomes and cancellation (#193) * feat(ui): add paired prompt editor and explicit recipe library for #193 * fix(autocomplete): include reusable pair metadata vocabulary (#193) * fix(pairs): honor recent sorting and conservative scan outcomes (#193) * fix(ui): protect paired draft saves and clarify negative variants (#193) * fix(metadata): preserve bounded paired execution evidence (#193) * fix(tracking): follow actual pair output roles and prefer execution evidence (#193) * docs(pairs): document live validation and retain renderer screenshots (#193) * test(api): inject disposable databases before API startup (#194) * test(api): canonicalize portable temporary gallery roots (#194) * docs(ui): prototype branching negative variant history (#193) * docs(ui): show selected negative beneath positive in timeline concept (#193) * docs: retain approved timeline for later gallery redesign (#193) * test(db): close restore fixture connections before cleanup (#194) * test(db): reuse existing handle guards in all pair fixtures (#194) * chore: clarify rollback test and intentional exception handling (#193) |
||
|
Vito
|
e9fe586442 |
fix: repair prompt edit hashes, auto-tag review, and stale UI responses (#180)
* fix: keep edited prompt hashes consistent Fixes #176 * fix: align gallery auto-tag review with scoped image APIs Fixes #177 * fix: keep Tags prompt results on the current filter Fixes #178 * fix: ignore stale metadata in admin image viewers Fixes #179 * test: compare canonical gallery paths on macOS |
||
|
Vito
|
6d9f3fc8fb |
fix: preserve gallery image identity and ignore stale metadata (#175)
* fix: preserve gallery root identity across media and metadata Fixes #173 * fix: ignore stale gallery metadata responses Fixes #174 * test: use platform-valid gallery filenames on Windows |
||
|
ghosttigerllc-bitandkai
|
2763e9f9fa |
Read A1111/Forge generation parameters: fix the SELECT and parse multi-line prompts (#165)
* get_image_prompt_info: include gi.parameters in SELECT and return value The function selects workflow_data, prompt_metadata and generation_time from generated_images but omits the parameters column, so the A1111/Forge generation-parameters string (steps, sampler, CFG, seed, LoRA list) never reaches the API even when present in the database. Add it to the SELECT and include it in the returned dict. * parse A1111/Forge parameters: multi-line prompts, structured settings _parse_comfyui_prompt took lines[0] of the parameters chunk as the whole positive prompt, truncating every multi-line prompt to its first line, and matched 'negative prompt:' only as a full line start. Split on the trailing 'Steps: ...' settings line instead, keep the full multi-line positive prompt, and extract Steps / Sampler / Schedule type / CFG scale / Seed as structured fields alongside the raw string. Together with the get_image_prompt_info SELECT fix (previous commit), the full A1111/Forge generation recipe now flows from the database to the API. * style: apply black to the A1111 parameters parser CI's black --check flagged py/api/__init__.py (slice spacing and two long lines in the settings extraction). Formatting only; the AST is unchanged. --------- Co-authored-by: kai <kai@hermes.local> |
||
|
Vito
|
da322f8c94 |
Hardening: security fixes, queue-time run counting, background jobs, 85%+ test coverage (#153)
* chore: valid packaging metadata, flake8 config file, ignore local config with secrets
pyproject.toml carried requirements.txt comment lines as dependencies, which
fails PEP 508 parsing. flake8 never read [tool.flake8] from pyproject, so the
settings move to .flake8 and pre-commit runs the full rule set. config.json
(holds the CivitAI API key and machine paths) and other local artifacts are
now ignored. coverage joins the dev requirements.
* ci: test on Linux, Windows and macOS with a coverage gate and lint job
* refactor: drop validate_workflow_name, the column it validated no longer exists
* chore: keep agent worktrees out of coverage measurement
* fix: image routes serve only media files inside allowed directories
serve_image and serve_output_image now reject anything whose extension
is not in IMAGE_EXTENSIONS, fail closed when no output directory is
configured, and answer 403 (not 404) for dot-dot, absolute and
symlink-escaping paths. Containment is checked after realpath+normcase
so it behaves the same on Windows, macOS and Linux. 500 responses use
safe_error_message so OSError text no longer leaks absolute paths.
* fix: bound page sizes on list endpoints
/search, /recent and /images/recent share parse_page_params: limit is
clamped to [1, MAX_PAGE_LIMIT=500], offset to >= 0, and non-integer
values answer 400 instead of 500. /search now honours offset and all
three return a pagination block with the effective limit and offset.
* perf: thumbnail generation no longer blocks the event loop
generate_thumbnails_with_progress is split into _thumbnail_targets
(scan, executor), _generate_one (one PIL/ffmpeg job, executor) and an
async driver that awaits each file via _run_in_executor, emits one
progress frame per file and yields between files. Per-file failures
are reported as file_error events (error stays fatal for the UI) and
the loop continues. The wildcard Access-Control-Allow-Origin header is
gone. The blocking POST variant reuses the same helpers. The scan never
follows symlinks and is capped by MAX_SCAN_DEPTH / MAX_THUMBNAIL_FILES.
* fix: cap every request-driven resource in the image routes
- /images/recent, /images/all and /images/output parse limit/offset
through parse_page_params on every path (absent, 0, negative, float
strings), and the clamped values are what reach the DB. /images/all
gets its own MAX_BULK_LIMIT ceiling instead of an unbounded query;
/images/recent caps its fetch window at MAX_RECENT_IMAGES_WINDOW.
- parse_page_params also clamps offset to MAX_PAGE_OFFSET so values past
int64 no longer surface as a 500 from SQLite.
- Gallery and thumbnail scans share _iter_media_files: no symlink
following, MAX_SCAN_DEPTH, and MAX_GALLERY_FILES / MAX_THUMBNAIL_FILES
caps. Thumbnail size comes only from named presets.
- JSON bodies on /images/link and /images/generate-thumbnails are read
in bounded chunks; over MAX_JSON_BODY_BYTES answers 413, malformed or
non-object JSON answers 400.
- /images/link only accepts media paths inside the allowed directories
(403 otherwise), closing a path-existence oracle.
* test: cover image and prompt routes
Every route in py/api/images.py and py/api/prompts.py now has a success
path and an error path test: save/update/delete/rate/tag/category/bulk
endpoints, image list/link/unlink/delete/metadata/scan endpoints, the
thumbnail SSE stream, clear-thumbnails, and the video thumbnail chain
(fake cv2 success/failure, ffmpeg absent or failing, PIL placeholder).
Combined statement+branch coverage: images.py 97.7%, prompts.py 97.4%.
* fix: autotag endpoints no longer read arbitrary paths or run on GET
* fix: CivitAI key is never returned or sent off-host
* fix: autotag/single checks real paths and DB rows against output dirs
* fix: CivitAI downloads never follow redirects
* refactor: autotag model loading is injectable
* test: cover autotag and lora integration
* fix: saving LoRA settings without a key keeps the stored CivitAI key
* refactor: share the internal error message in autotag routes
* fix: never treat negative-only PromptManager nodes as the run prompt
run_prompt_nodes fell back to every PromptManager node whenever no positive
node was traced, including graphs that do have a sampler and whose only
PromptManager node feeds its negative input. Fall back only when the graph
has no recognisable sampler (new has_sampler helper). Samplers named like
one are recognised even without a model link. Adversarial graphs (cycles,
10k-node fan-out, non-dict inputs) are covered.
* fix: do not follow negative text inputs when tracing the positive prompt
_TEXT_KEY matched input names such as text_negative, so a styler node feeding
a sampler's positive input pulled its negative PromptManager node into the
positive trace. Input names matching (^|_)neg(ative)?($|_) are now skipped
(new follows_input helper).
* fix: count prompt runs when ComfyUI dequeues the prompt, not when it is submitted
The on-prompt hook runs before ComfyUI validates the request, so counting
there bumped run_count for prompts that were rejected or deleted from the
queue. The hook now only resolves the positive PromptManager texts, mints a
UUID4 prompt_id when the client sent none, and stashes the hashes under that
id (QueuedRuns, bounded to 1000 entries). PromptQueue.get is wrapped
idempotently; when a prompt is dequeued for execution its stashed runs are
counted (or remembered as pending first use). Servers without a prompt_queue
fall back to counting at submit time with one warning.
__init__ defines _global_db before the monitoring block and passes a factory
that raises RuntimeError while the database is unavailable; the package is
now imported under test with server and folder_paths stubbed.
* fix: keep batch image queue aligned when unrelated workflows save images
The metadata strategy returned None for images whose embedded workflow has
no positive PromptManager node, so the handler popped the batch FIFO for
them and every later batch image linked to the wrong prompt. It now returns
a tri-state: linked (prompt found), foreign (no PromptManager prompt in the
workflow, so the image is skipped and no fallback applies) or unknown (only
then the queue, snapshot, tracker and recent-prompt fallbacks run).
* fix: image monitor validates directories before creating the observer
start_monitoring assigned self.observer before checking that any directory
existed, so with no valid directory the monitor was stuck "already running"
and stop_monitoring raised on join of a never-started observer. Directories
are validated first and only a started observer is kept; stop_monitoring is
safe to call at any time and always joins. New is_monitoring property.
The per-image threading.Timer is replaced by one daemon worker: paths are
deduplicated and bounded (MAX_PENDING_IMAGES), processed in creation order
(batch images pop the prompt queue in order) after the processing delay and
once the file size is unchanged across two reads.
* refactor: one prompt hash implementation
utils.hashing.generate_prompt_hash is now the only prompt hash:
PromptManagerBase._generate_hash delegates to it and the inline
hashlib.sha256 copies in the prompt tracker and image monitor are gone.
Remove PromptManagerBase._open_web_interface (it targeted a web_interface
folder that never existed) and the unused restart_gallery.py script.
IS_CHANGED on PromptManager and PromptManagerText now includes category and
tags, so editing either re-runs the node and updates the saved metadata.
* fix: stop ComfyUI from loading the admin bundle into the canvas page
WEB_DIRECTORY = "web" made ComfyUI inject every .js under web/ (admin.js,
gallery.js, vendored Tailwind and ViewerJS) into the canvas page. The node
extension moves to web/comfy/prompt_manager.js (content unchanged; its
../../scripts/app.js import resolves against the served URL, which is the
same) and WEB_DIRECTORY points at that folder only. Admin and gallery assets
are still served from web/ by the extension's own routes.
tests/test_sort_contract.py still reads web/prompt_manager.js and must be
updated to the new path by its owner.
* fix: image monitor recent-prompt fallback reads the paginated result
get_recent_prompts returns {"prompts": [...]} but _get_fallback_prompt
indexed it like a list, so the last-resort link to the most recent prompt
always failed silently. Accept both shapes.
* test: cover prompt_manager_base, prompt_manager and prompt_search_list
Node base: init wiring, metadata update on re-save, run counting, database
error paths, tag parsing, search and recent API wrappers, gallery start
failure, LoRA trigger-word injection (disabled, missing modules, enabled).
PromptManager: missing CLIP error, encoding of the combined text with a fake
CLIP, tracking failure tolerance, empty text. PromptManagerText: tracking
failure tolerance. PromptSearchList: filters, multipart and LoRA-only
skipping, newline collapsing, preview, empty and error results.
* fix: diagnostics dependency check survives a watchdog without __version__
check_dependencies read watchdog.__version__ for its log line; recent
watchdog releases have no such attribute, so the AttributeError escaped and
took the whole diagnostics run down. Read it with a default.
Also covers utils/diagnostics.py: database and images-table checks against
a real temporary database, file system check, ComfyUI output detection
(folder_paths, configured directories, relative fallback, nothing found),
dependencies, the full run and the test image link.
* test: cover utils/logging_config
Isolated logger manager on a temporary log directory: memory buffer order,
level filter and 1000-entry cap, rotation and backup listing, tail and whole
file reads, path traversal rejection, truncation (including removal
errors), level changes across prompt_manager loggers, handler rebuilds.
* test: cover utils/comfyui_integration
Singleton re-initialisation keeps registrations, latest prompt per thread,
cross-thread fallback with the five-minute recency limit, global lookup for
the current thread, and cleanup of old registrations.
* test: cover __init__
Version parsing from pyproject (and the unknown fallback), and a server
whose add_on_prompt_handler raises: nodes and monitoring still come up.
* docs: point at the moved canvas extension under web/comfy
* fix: give every thread its own sqlite connection
PromptModel.get_connection() handed every thread the same
check_same_thread=False connection, so a commit or rollback from the
watchdog, executor or queue-hook thread could land on another thread's
half-done write (sqlite3 raised 'bad parameter or other API misuse'
under concurrent saves). Connections now live in threading.local, each
with WAL, Row factory, foreign_keys and busy_timeout, and are tracked so
they can be closed. PromptModel.close() / PromptDatabase.close() close
the calling thread's connection so tests and restores can delete the
file on Windows.
* fix: saving an existing prompt concurrently returns its id
Node execution, the queue hook and the image monitor all do
get_prompt_by_hash() then save_prompt(). When two of them race on the
same text the second INSERT failed on UNIQUE(hash) and surfaced an
IntegrityError. save_prompt now catches it, re-fetches by hash and
returns the existing id, so eight threads saving one prompt end with a
single row and every record_prompt_use counted.
* fix: backups include WAL data and restores are integrity-checked
The database runs in WAL mode, so a shutil.copy2 of prompts.db silently
dropped every committed row still in prompts.db-wal; the backup
download and the pre-restore safety copy both did that. Restore also
overwrote the live file with connections open, read the whole upload
into memory before checking the 100 MB cap and never ran
PRAGMA integrity_check.
PromptModel.backup_database now uses the SQLite online backup API,
verify_database_file opens a candidate read-only and checks header,
integrity and the prompts table, and restore_from_file backs up the
live database, closes every thread's connection, copies the file,
removes stale -wal/-shm and re-runs migrations. The admin handlers
stream the upload to a temp file with an early size cutoff, reject
anything verify_database_file refuses with a 400 and keep the
{success: ...} JSON shape. The schema-init connection is now closed
instead of leaking.
* fix: migrations are idempotent and schema init runs once per process
Every PromptDatabase() (nodes, image monitor, API, diagnostics) re-ran
table creation and all migrations. Schema init is now cached per
normalised path (class-level set under a lock) and re-run only when the
file disappeared, after a restore, or after reset_schema_cache().
The copy-and-swap migrations drop a stale *_new table left by an
interrupted run, execute inside an explicit transaction with
PRAGMA foreign_keys = OFF (DROP TABLE prompts used to cascade-delete
every generated_images row), roll back cleanly and log and re-raise
sqlite3.Error instead of swallowing it.
* fix: image uniqueness by full path, escaped LIKE search, no usage stamp on save
generated_images was UNIQUE(prompt_id, filename) on the basename, so
ComfyUI's same-named files in different date folders were silently
ignored by INSERT OR IGNORE, and the lastrowid == 0 branch after it
never ran. The table now carries file_path (normcase/normpath/abspath
of the image path) with UNIQUE(prompt_id, file_path); the migration
collapses rows that normalise to one file and supersedes the old
basename migration, which would otherwise undo it on every start.
search_prompts and the tag search escaped nothing, so 100% matched
'100 percent' and a_b matched aXb; user text now goes through
escape_like with ESCAPE. save_prompt no longer stamps last_used_at, the
usage backfill runs only on the upgrade that adds the columns, and the
Recently Used / Most Used orders put never-run prompts (NULL) after run
ones instead of first.
* fix: escape LIKE wildcards in image prompt search
search_images_by_prompt interpolated the user's text into LIKE
unescaped, so '50%' also matched '50 percent'. It now goes through the
same escape_like helper as search_prompts. Also drops an unused import
and an unused cursor assignment flagged by pyflakes.
* test: cover database operations
Behaviour tests for export (json/csv/errors), duplicate detection and
merging, image metadata storage and lookup, missing-file cleanup,
statistics, bulk operations, orphan pruning, consistency checks,
subfolder listing, tag queries and management edge cases, legacy tag
column parsing, config-driven path resolution, model housekeeping and
the TEXT -> INTEGER prompt_id migration. database/operations.py and
database/models.py are both above 85% statement+branch coverage.
* test: isolate the clock and log buffer from background threads of other tests
* fix: gallery roots are restricted to ComfyUI directories
A gallery root could be set to '/' (or 'C:\') through POST
/prompt_manager/settings because the deny list only matched a few
system prefixes. The root was persisted to config.json and fed into the
image-serving containment check, which then allowed any file on disk to
be read through /prompt_manager/images/serve/.
- GalleryConfig.validate_gallery_root() accepts a directory only when its
realpath lies inside a ComfyUI directory (folder_paths output/input/
temp/user) or a directory listed in PROMPT_MANAGER_EXTRA_GALLERY_ROOTS;
filesystem roots, the home directory, files and missing paths are
rejected.
- save_settings returns 400 and persists nothing when any root fails.
- _get_all_output_dirs() filters configured roots through the validator
and never falls back to a rejected root.
- PROMPT_MANAGER_CONFIG_PATH relocates config.json; load/save default
to it.
* fix: log config accepts only known keys
POST /prompt_manager/logs/config forwarded the whole request body into
PromptManagerLogger.update_config(), so any key could be injected into
the logging configuration. The handler now accepts only LOG_CONFIG_KEYS
(level, max_file_size, backup_count, console_logging, file_logging,
buffer_size), type-checks each value (level name, non-negative int,
bool) and answers 400 listing unknown keys. Malformed or non-object
bodies are 400 instead of 500.
* fix: no absolute server paths in API responses
Settings, diagnostics, duplicate/output scans and the logging endpoints
returned absolute filesystem paths, and several error envelopes echoed
str(e) which for OSError carries the full path.
- _public_path() renders a path relative to the ComfyUI base directory
(folder_paths.base_path, else the output dir's parent) or to the
parent of an opted-in extra gallery root, falling back to the
basename; _public_error() reduces OSError to strerror + basename.
- GET /settings returns gallery roots and monitored directories in that
form; POST /settings resolves relative roots against the same anchors
(GalleryConfig.resolve_gallery_root) so the GET output round-trips.
- Diagnostics output_dirs, monitored_directories and the missing-DB
message no longer carry absolute paths; the image monitor is reached
through the package-or-top-level import fallback instead of a
three-dot relative import that failed outside the package.
- Duplicate scan 'path' is now relative to the output root and the
delete endpoint accepts relative paths.
- /logs/files 'path' and /logs/stats 'log_directory' use the public
form; unexpected errors answer with a generic message.
* perf: run duplicate and output scans off the event loop
The duplicate scan walked the output tree and hashed every file on the
aiohttp event loop, and the duplicate delete removed files inline; the
output scan ran its per-file database work one executor hop at a time.
- _find_duplicate_images_sync / _describe_media_file do the walk and
hashing; find_duplicate_images() runs them via _run_in_executor.
- _delete_duplicate_images_sync / _delete_one_duplicate do the removal
(absolute or output-relative paths) off the loop.
- The output scan is split into _collect_output_media_sync,
_extract_batch_metadata_sync and _ingest_scan_batch_sync (one
executor call per batch for all DB work) with a small SSE generator.
- Media collection is shared and de-duplicates by normcase'd path.
* refactor: settings reads the image monitor through its module
get_settings located the ImageMonitor singleton by scanning sys.modules
for any module with 'image_monitor' in its name, a workaround for the
package being importable under two identities. It now uses the same
try-package / except-top-level import as the rest of the API
(_current_image_monitor) and falls back to the configured roots when no
monitor is running.
* fix: clamp the logs limit and reject non-integers
GET /prompt_manager/logs accepted any limit up to 1000 and answered 500
when the value was not an integer. The limit is now clamped to [1, 500]
and a non-integer value is a 400.
* test: cover core API module and gzip middleware
* refactor: split diagnostics, maintenance and HTML serving into helpers
- run_diagnostics composes four module-level probes (_diagnose_database,
_diagnose_dependencies, _diagnose_output_dirs, _diagnose_image_monitor).
- run_maintenance is table-driven: _maintenance_runners maps each
operation name to a callable and _run_maintenance_operations reports
per-operation success or a public error message.
- _read_json answers 400 for malformed or non-object JSON bodies in
settings, duplicate delete, test-link and maintenance instead of 500.
- test_image_link delegates to _link_test_image.
- The four HTML page routes share _serve_html_page.
* test: cover admin and core API routes
Route tests for stats, cleanup, duplicate scan/delete, settings,
diagnostics, test-link, maintenance (every operation, success and
failure), the output scan internals, and contract-level tests for
backup/restore (status codes and envelope only).
* test: make admin, logging and diagnostics tests independent of test order
The diagnostics message no longer carries the absolute path, the restore test
closes its raw sqlite connections before the live file is replaced, the logs
tests declare their own folder_paths base directory, and the isolated logger
filters every handler to the test thread.
* fix: truncating logs no longer rolls the main log over or pads it with zeros
truncate_logs emptied prompt_manager.log through a second open(), leaving the
rotating handler's stream at its old offset: the next record was written past
the new end (zero padding on POSIX) and tell() still reported the old size, so
the file rolled over and resurrected the backup that had just been deleted.
It also logged once per file from inside the deletion loop, which could do the
same with a small size limit. The main log is now truncated through the
handler's own stream and a single summary line is logged afterwards.
* fix: HTML escaping covers attribute context
escapeHtml now escapes & < > " ' (shared web/js/html-escape.js, loaded by
admin, gallery and metadata pages) instead of the textContent/innerHTML
round-trip that left quotes intact, so a tag, category, checkpoint or file
name containing a quote can no longer break out of a data-*, value or title
attribute.
Inline onclick handlers that interpolated server data now carry data-action
and data-* attributes and are dispatched through one delegated listener per
page (web/js/data-actions.js); the img onerror fallback uses
data-fallback-src with a capturing error listener. Every remaining template
value written into an attribute or text node (prompts, checkpoints, file
paths, log lines, directory names, categories, suggestions) is escaped.
metadata.html no longer builds an "explorer /select" shell command from the
file name (Electron-only branch); the path is copied to the clipboard.
* refactor: remove shadowed and dead frontend code
admin.js defined parsePNGMetadata, extractComfyUIData, showFullPrompt,
showWorkflowData, downloadWorkflowJSON and copyAllMetadata twice; the first
definitions were unreachable because the later ones shadow them.
updateMetadataPanel, showMetadataError, copyPrompt and tryFallbackMetadata
only ever wrote to #metadataContent, an element no page creates, so they are
gone too. gallery.js defined escapeHtml and showNotification twice. The admin
keydown/resize handlers referenced closeImageViewer, previousImage, nextImage
and applyImageSizing, none of which exist, and the imageViewMode state they
used; only the Escape-closes-gallery branch remains.
Debug console.log calls are removed; the ones that were the only statement of
a catch block are console.warn so parse failures stay visible.
* fix: stale responses, double submits and leaked listeners
web/js/api-client.js wraps fetch with one failure path (ApiError for non-2xx,
success:false and non-JSON bodies) and latestOnly(), which aborts the previous
in-flight request and resolves superseded calls to STALE. The admin search and
recent-page loads share one guard, the gallery page load has its own, so fast
typing or clicking can no longer paint an older result over a newer one.
In-place prompt editing used a blur listener with {once:true} plus a keydown
listener added on every edit, which sent the PUT twice and stacked handlers.
The list now has one keydown/focusout pair bound at construction and
finishEdit() clears the edit state before touching the element, so a save is
sent exactly once. Mutations re-fetch the page currently shown (refreshList)
instead of calling search(), and delete, bulk delete, edit, restore and scan no
longer reload the whole page.
Pagination state is committed only after a successful fetch via the pure
ListState.nextPageState(current, response) in both admin and gallery, so a
failed or aborted page request leaves the current page intact. The gallery
lightbox removed its keydown handlers on a non-existent 'remove' event; a
MutationObserver now drops them when the modal leaves the document.
web/js/sse-stream.js reads server-sent events over fetch so POST-only progress
endpoints can stream.
* fix: follow the POST-only autotag streams and the new LoRA and list contracts
autotag/download and autotag/start are POST-only now; EventSource can only
GET, so both pages stream their progress through SseStream.connect(url,
{method: 'POST'}), which keeps the onmessage/onerror/close surface the
handlers already use. autotag/single is called with image_id and the re-tag
dialog shows the server's filename instead of splitting an absolute path.
lora/status reports has_civitai_api_key instead of the key: the field shows a
'key is stored' placeholder, an empty field keeps the key on save and a
'Clear the stored key on save' checkbox sends clear_civitai_api_key.
Thumbnail generation reports per-file failures as file_error events; both
progress views count them (and show error_count from progress) without
closing the stream. The auto-tag review follows pagination.has_more on the
capped /images/all endpoint, and /search is paged like /recent so paging in
search mode no longer switches the list back to recent prompts.
* refactor: one PNG metadata parser
admin.js, gallery.js and metadata.html each carried a copy of the PNG chunk
walker that reads ComfyUI's tEXt/iTXt entries (admin.js had two until the
shadowed one was removed). web/js/png-metadata.js is now the only parser:
it checks the signature, bounds-checks every chunk so a truncated file or a
bogus length cannot throw, decodes tEXt as Latin-1 and iTXt as UTF-8, skips
compressed zTXt/iTXt payloads instead of decoding them as garbage, and stops
at IEND. The three page-level parsePNGMetadata functions delegate to it.
* fix: tags deep link renders and admin modals close
admin.html loaded tags-page.js after admin.js, so opening the page at #/tags
found no TagsPageManager and rendered nothing; the script order is now
theme-bridge, html-escape, api-client, png-metadata, (list-state, sse-stream,
data-actions), viewer, comfy-metadata, prompt-list-sort, view-router,
tags-page, admin. Route resolution lives in the pure ViewRouter.resolveView().
The auto-tag modal's close buttons called a bare hideModal(), which only
gallery.js defines; they call window.admin.hideModal(). PromptAdmin is
constructed from startAdmin() on DOMContentLoaded (or immediately when the
document is already parsed) instead of at script evaluation time.
* test: cover frontend modules
browser-globals.test.js loads every page module through vm with a fake
window, the way the pages do, and asserts the global each one exposes
(HtmlEscape/escapeHtml, DataActions, ApiClient, ListState, SseStream,
PngMetadata, ViewRouter, PromptListSort, ComfyMetadata). sse-stream gains
cases for a POST body, missing handlers and close() before or during
delivery. Every pure module is above 85% line coverage under
node --test --experimental-test-coverage.
* fix: page image listings in SQL and drop three unused helpers
get_recent_images and search_images_by_prompt take limit/offset so the recent
and search image routes no longer over-fetch a 5000-row window and slice it.
generate_content_hash, PromptModel.migrate_database and find_duplicates had no
callers outside their own tests. The isolated logger tests re-apply their
thread filter after update_config rebuilds the handlers. Long lines in the
touched database and image modules are wrapped so the full flake8 hook passes.
* fix: restore through SQLite's backup API, close_all for shared files, safer output scans
restore_from_file copied the uploaded file over the live database and deleted
its WAL while other PromptModel instances (nodes, image monitor, queue hook)
still held connections, which corrupts on POSIX and fails on Windows. It now
copies pages through the online backup API into the live connection so every
open handle sees the new content. PromptDatabase.close_all() closes the
connections of every thread, which tests need before deleting a database.
Output scans skip symlinked files, filesystem entries get a stable digest id
instead of a per-process hash(), and thumbnail streaming stops when the
client disconnects.
* style: wrap long lines in node modules
Split tooltip, description and docstring strings that exceed 88 columns
using implicit concatenation; string values are unchanged. Drop the unused
typing imports from prompt_search_list and mark the deliberately late node
imports in __init__ with noqa: E402.
* style: wrap long lines in the API layer, autotag and config
Split long f-string log messages, response messages, URL builders and
docstrings; DEFAULT_PROMPT and SYSTEM_PROMPT are rewritten as implicit
concatenations with identical values. Drop the unused shutil import in
admin and the unused HFCacheInfo import in autotag; keep the
hf_hub_download/snapshot_download availability probe with noqa: F401 and
mark the deliberately late imports in config with noqa: E402.
* style: wrap long lines and drop unused imports in utils
Re-wrap module docstrings, comments and f-string log messages; remove the
unused typing.List and PngInfo imports; mark the PIL availability probe in
diagnostics with noqa: F401; turn two placeholder-less f-strings into plain
strings; strip trailing whitespace inside the diagnostics SQL.
* style: wrap long lines in tests
Split long docstrings and SQL literals (values unchanged) and drop the
unused os import from test_path_security.
* chore: flake8 ignores the gitignored standalone tagger script
* fix: escape image dimensions and workflow JSON before rendering
openGalleryModal interpolated width, height and file_size (stored from a
POST body, so possibly strings) straight into data-caption and the card
text. showWorkflowData wrote JSON.stringify(workflow) raw into a blank
same-origin window, so a workflow containing </pre><script> ran. Both now
go through ImageHelpers (new pure module, node:test covered); a string
file_size no longer reaches formatFileSize's toFixed either, and a blocked
popup shows a notice instead of throwing.
* fix: escape remaining server values rendered through innerHTML
Maintenance and diagnostics results, review-tag errors, video info and
metadata error messages, consistency issues, recent image filenames,
output directory names, the prompt rating attribute, gallery media types
and duplicate group headers were interpolated unescaped. The copy
fallback modal also inlined the prompt text inside <textarea>, which a
closing tag could break out of; it now sets textarea.value.
* fix: address gallery auto-tag review images by path, not digest id
The gallery review flow builds its list from /prompt_manager/images/output,
whose id is a 16-hex path digest, but posted it as image_id to
autotag/single, so the server looked up a database row and every request
404ed. autotagSingleBody (pure, tested) now sends path for output-folder
entries and keeps image_id for database rows, which the admin review flow
(/images/all) still uses.
* fix: read thumbnail generation progress over POST with SseStream
images/generate-thumbnails/progress is becoming POST-only, which the
browser EventSource cannot request. Both thumbnail progress consumers in
the gallery now use SseStream.connect(url, {method: 'POST'}) exactly as
the autotag streams do. SseStream gains addEventListener/removeEventListener
for named event: frames (tested) so the status, start, progress,
file_error, complete and error listeners stay as they were, and an onclose
guard rejects when the stream ends without a terminal frame.
* fix: drop the unused copy-text data-action handler
Nothing in the admin markup emits data-action="copy-text"; the prompt
copy buttons call copyToClipboard directly.
* fix(lora): confine LoraManager paths to custom_nodes and hide server paths
- is_civitai_url also refuses URLs with userinfo or a non-443 port, so
https://evil.com\@civitai.com/x and https://civitai.com:8443/x are rejected.
- resolve_lora_manager_path / custom_nodes_directories: lora/enable and
lora/scan only accept a LoraManager directory that is a direct child of
ComfyUI's custom_nodes (folder_paths base_path or registered custom_nodes
folders, else this package's parent) and whose name contains "lora",
compared via realpath+normcase; relative public paths resolve against the
API path anchors. Anything else is 400, so find_lora_directories can no
longer widen the image allow-list with arbitrary trees.
- lora/enable persists through PromptManagerConfig.save_to_file(), which
honours PROMPT_MANAGER_CONFIG_PATH, instead of a hard-coded config.json.
- detect/status/enable return path and detected_path through _public_path;
every lora handler reports exceptions via _public_error, and lora/scan
errors raised after the SSE stream opened become an error event.
- Tests close the database before unlinking it.
* fix(autotag): contain batch rows, require media extensions, bound the scan
- The batch tagger runs every generated_images row through
_resolve_allowed_image_path; rows that exist but lie outside the served
output directories are skipped and counted in skipped_outside_roots on
progress and complete events, and the engine receives the resolved path.
- _resolve_allowed_image_path requires a media extension (.png .jpg .jpeg
.webp .gif .bmp .tiff .tif), so autotag/single returns 403 for other files.
- autotag/start holds a module-level lock; a concurrent start gets 409
{"success": false, "error": "Auto-tagging is already running"}.
- scan_output_dir walks with a 50,000-file and 12-level cap, skips symlinked
files and thumbnails, returns path through _public_path, relative_path in
POSIX form, and a truncated flag.
- Autotag handlers report exceptions via the shared _public_error; the
local copy is removed. Tests close the database before unlinking it.
* test: close every database before unlinking and anchor public paths in image tests
Each owned test that opens a PromptDatabase now calls close_all() before
removing the file, and temp directories holding a database use
ignore_cleanup_errors, so the suite tears down cleanly on Windows. The
image API fixture lays out a fake ComfyUI tree (<tmp>/output) and pins
GalleryConfig.path_anchors to it so responses can be checked for
root-relative paths.
* fix: validate image file_info before it reaches the generated_images columns
link_image_to_prompt stored size, dimensions and format straight from the
request body, so a string such as '"><img src=x>' ended up in INTEGER
columns and was echoed back to the gallery. Size and dimensions are now
coerced with int() and bounded (size 0..10^12, width/height 0..65535),
format is matched against a whitelist of media formats, and anything
else is stored as NULL while the image still links. Non-dict metadata or
file_info no longer aborts the link.
* fix: clamp paging on tag listings and the page-derived recent offset
tags/stats, tags/{tag}/prompts and tags/filter parsed limit/offset with a
bare int(), so limit=999999 was passed straight to SQLite and a huge
offset raised OverflowError (500). They now share parse_page_params
(limit 1..500, offset 0..MAX_PAGE_OFFSET, 400 on non-integers), and
/recent clamps the offset it derives from ?page= the same way.
* fix: make the thumbnail progress stream POST-only
GET /prompt_manager/images/generate-thumbnails/progress wrote thumbnails
to disk, so any prefetch, link preview or <img src> could start a full
output-directory scan. The route now accepts POST only (GET answers
405); quality still comes from the query string so the SSE client can
keep the same URL shape.
* fix: describe images relative to the ComfyUI tree in API responses
Gallery and prompt responses carried absolute server paths: /images/output
entries (path, root_dir), thumbnails_path, the images/prompt lookup, every
image list (image_path plus its normalised file_path duplicate) and the
preview images attached to prompt listings. They now go through
_public_path: /images/output 'path' is relative to its root (what the
delete and autotag endpoints resolve against), 'root_dir' and every
other image_path are relative to the ComfyUI base, and file_path is
dropped. recent/all/search image lists also gain url/thumbnail_url via
the same enrichment the prompt-images route already had. Remaining
str(e) error bodies in both modules use the basename-only error text.
* fix: check thumbnail directory containment with a real path test
clear_thumbnails compared resolved paths with str.startswith, so a
thumbnails directory that resolved to a sibling with a shared prefix
(out vs out2) passed the safety check and was emptied. It now uses the
module's _is_within (realpath + normcase + Path.is_relative_to).
* fix: decode tag names from the URL only once
The tag lookup, rename and delete routes ran urllib.parse.unquote on
match_info values that aiohttp had already percent-decoded, so a tag
literally named '%41' (requested as %2541) was handled as 'A'. The
second decode is dropped.
* fix: validate tags on every tag mutation route
add_prompt_tag, add_tags_to_prompt, bulk_add_tags, merge_tags and
rename_tag accepted whatever the body carried; a non-string tag raised
AttributeError (500) and over-long or control-character tags were
stored. They now run utils.validators.validate_tags through a shared
tags_error helper and answer 400 with the validator's message. Blank
entries in add_tags_to_prompt are still skipped rather than rejected.
* fix: export every prompt by paging instead of truncating at 10000
/prompt_manager/export fetched a single page of 10000 prompts and
silently dropped the rest. It now walks the table in EXPORT_PAGE_SIZE
pages (1000) inside one executor job until a short page is returned.
* fix: gate the LoraManager serving roots and accept relative link paths
A LoraManager path hand-edited into config.json went straight into
find_lora_directories and so into the image-serving allow-list. When a
path is configured it must now pass lora_utils.resolve_lora_manager_path
(a custom_nodes child whose name contains 'lora'); otherwise, or when the
resolver is unavailable, no LoRA directories are served. An empty path
keeps the custom_nodes-only auto-detection.
POST /images/link now also accepts the relative forms the API hands out
(root-relative 'sub/a.png' and ComfyUI-relative 'output/sub/a.png'),
resolving them against the allowed roots and anchors; '..' segments in
a relative path are refused and unknown files are 404.
* fix: review flow uses the served image url and the lora-dirs test pins its config
Image paths in API responses are now relative to the ComfyUI tree, so the
auto-tag review page can no longer derive a serve URL from image_path; the
server always provides url. The lora-dirs serving test assumed an empty
LoraManager path, which only held in a checkout without a local config.json.
* test: stop leaking a mocked utils.comfyui_integration and mock the IS_CHANGED clock
test_image_monitor_singleton replaced utils.comfyui_integration in
sys.modules with a MagicMock and never restored it, so running it before
test_comfyui_integration failed eight tests. The mock was unused; drop it
and install the server stub with setdefault in every owned test module.
The PromptSearchList IS_CHANGED test slept 10 ms between two time.time()
reads; the Windows clock ticks every 15.6 ms, so it was flaky there. Mock
the clock with an incrementing counter instead.
* fix: resolve_text follows a node shared by two branches
The cycle guard used one visited set for the whole walk, so a string node
read twice (a diamond, or StringConcatenate with the same source on both
inputs) made the second branch None and the prompt text unknowable. Cycle
detection is now scoped to the current path; a separate visit counter
keeps MAX_VISITS as the bound on total work.
* fix: bound resolve_text on diamond chains with memoisation, depth and length caps
A per-path visited set alone makes a chain of diamonds (node i reads node
i+1 on both inputs) exponential in depth. Each node's text is now
memoised so it is expanded once, every expansion is charged to the shared
MAX_VISITS budget, chains deeper than MAX_DEPTH (64) are unknowable so
recursion stays bounded, and text longer than MAX_TEXT_LENGTH (100k) is
unknowable so StringConcatenate doubling cannot exhaust memory.
* fix: static routes resolve under web/ with an extension allow-list
/prompt_manager/lib/{filepath} and /js/{filepath} only rejected '..' and a
leading '/'. On Windows os.path.join(root, 'C:/x') or a leading backslash
re-anchors the path, so any file on the host could be read. Requests are
now refused when the path is absolute on any platform (slash, backslash,
drive letter, UNC prefix), carries an empty, '.' or '..' segment, has an
extension outside the allow-list, or resolves (symlinks included) outside
the static directory; the resolved real path is served with its MIME type.
* feat: capped JSON body reader and single-flight lock for long admin jobs
Add _read_json_body(request, max_bytes=1_000_000) to py/api: reads the
body in chunks, answers 413 past the cap (declared or streamed) and 400
for invalid or non-object JSON. Every JSON POST in the admin and logging
handlers goes through it; it is exposed as PromptManagerAPI._read_json_body
for the other route mixins.
Add _acquire_long_job(name) / _long_job_busy_response(label) on the API
object: one asyncio.Lock per job family, handed out as an async context
manager or None when busy, so a handler answers 409 instead of starting a
second scan. The duplicate scan and delete share the 'duplicates' family.
* fix: harden database restore and keep server paths out of admin responses
Restore: verify_database_file and restore_from_file run through
_run_in_executor; the upload is also opened read-only with
PRAGMA trusted_schema=OFF and refused when sqlite_master lists a trigger
or a view (a prompts database has neither, a crafted one could use them
to run SQL against the live data); at most MAX_SAFETY_BACKUPS (10)
'<db>.backup_*' files are kept next to the live database, oldest first.
Responses: backup_created and the maintenance statistics database_path
go through _public_path, and backup/restore failure messages through
_public_error, so no absolute server path appears in an admin body.
* fix: validate hand-edited gallery roots everywhere and bound output scans
A root written straight into config.json skipped the containment check
the settings endpoint applies. GalleryConfig.update_config (and so
load_from_file) now drops entries that fail validate_gallery_root with a
warning, _find_comfyui_output_dir validates the configured directory
before using it, and ImageMonitor.start_monitoring validates every
directory before scheduling the observer.
The duplicate cleaner only deletes files with a media extension and
never anything under a thumbnails directory. _collect_media_files walks
with os.walk(followlinks=False), skips symlinked files and directories,
prunes thumbnails, stops MAX_SCAN_DEPTH (12) levels down and after
MAX_SCAN_FILES (50000) files.
* fix: bound log file size and backup count in the logging config endpoint
max_file_size must be between 1 MB and 1024 MB (bytes) and backup_count
between 0 and 50; anything else is a 400. Also close every database
connection in the logging test teardown.
* fix: private config.json, merged settings save, synthetic test-link path
PromptManagerConfig.save_to_file and the settings endpoint write
config.json through write_private_json: created with mode 0600 (chmod on
POSIX, best effort on Windows) because the file can hold an API key.
_persist_settings merges web_ui and gallery.monitoring.directories into
the existing file instead of overwriting it, so the integrations section
and any other key survive a settings save.
The test-link diagnostic no longer stores a client-supplied path; it
records a marker file name under the output directory.
* fix: image worker thread exits on shutdown and restarts cleanly
ImageGenerationHandler.shutdown() clears the pending set, sets a stop
event the worker's delay and settle waits listen to, and puts a None
sentinel on the queue; the worker exits within the current wait. Each
worker owns its queue and stop event so a draining worker never swallows
an image scheduled after shutdown. ImageMonitor.stop_monitoring calls it.
* test: close all database connections before deleting temp files
Windows refuses to unlink an open SQLite file and executor threads keep
their own connections, so every owned test that creates a PromptDatabase
calls close_all() before cleanup; the diagnostics temp dir ignores
cleanup errors.
* fix: output scan runs as a background job that survives the browser going away
Closing the dashboard tab or clicking outside the scan modal raised
ClientConnectionResetError out of scan_images and abandoned the scan
mid-batch, leaving a half-scanned database. Two overlapping scans could
also run at once.
- py/api/scan_job.py: ScanJob fans progress out to any number of SSE
subscribers (latest snapshot first, then live events)
- POST /prompt_manager/scan starts the scan as an asyncio task, or attaches
to the one already running; a write to a dead socket ends that stream
quietly while the scan continues
- GET /prompt_manager/scan/status reports the running flag and last event
- dashboard: dismissing the modal while a scan runs minimizes it to a
progress pill, the page re-attaches to a running scan after a reload,
a lost stream reconnects, and leaving the page mid-scan asks first
* feat: worker-thread setting speeds up the output scan and thumbnail generation
Both jobs processed one file at a time on a single core. Pillow decodes the
whole image to reach trailing PNG text chunks and releases the GIL while it
does, so a bounded thread pool scales almost linearly: on a 1 GB sample of
400 PNGs the metadata pass dropped from 18 s to 2.5 s at 8 threads, and 120
thumbnails from 4.6 s to 0.6 s.
- utils/parallel.py: map_parallel(func, items, workers) keeps order, runs on
the caller's thread when workers is 1, and caps the pool at the item count
- PromptManagerConfig.WORKER_THREADS defaults to half the detected cores,
is clamped to max_worker_threads() when loaded from config.json, and is
persisted under "performance"
- settings API returns worker_threads and cpu_count and validates saves;
the settings modal gets a "Worker threads" slider capped at the core count
- the scan reads each metadata batch on the configured threads; thumbnail
generation processes batches of files per executor job, still emitting
one progress event per file
Threads rather than processes: this runs inside ComfyUI's server, where
forking a CUDA-initialised process is unsafe and the spawn start method
would re-import ComfyUI's main module in every child.
* fix: scan modal ignores backdrop clicks so a focus click cannot dismiss it
Only its own buttons dismiss it: Cancel before a scan starts, Hide (minimize
to the progress pill) while one runs.
* feat: scan reads prompts embedded in videos; unreadable files no longer log errors
The output scan handed every .mp4 to Pillow and logged one ERROR per file.
ComfyUI's video savers store the queued graph as JSON in the container's
comment tag, so those files carry prompts worth linking.
- utils/video_metadata.py reads the comment/description tag through
ffprobe with a timeout, normalised to the PNG text-chunk shape so the
existing parser handles both; without ffprobe videos are skipped and a
single INFO notice is logged
- _extract_comfyui_metadata routes videos there and logs unreadable images
as warnings, since the scan carries on past them
Test hygiene: every aiohttp test base class overrode tearDownAsync without
calling super(), so no test client was ever closed. That produced thousands
of ResourceWarnings, "Unclosed client session" errors from asyncio that
broke log assertions, and doubled the suite's run time.
* fix: thumbnail runs skip fresh thumbnails before the cap and report what remains
The per-run cap of 10,000 applied to the directory walk itself, so a
library with more files than that never had thumbnails generated for the
rest, and a run that found 9,987 existing thumbnails spent its whole
budget on skipping them.
- _plan_thumbnails walks the whole tree (bounded like the gallery listing),
counts files whose thumbnail is already newer as skipped without queuing
them, and caps only the pending files; the leftover count is returned
- start and complete payloads carry skipped and remaining; total_images
counts every file considered and the processing rate uses the files
actually worked on
- the gallery page shows a warning naming the pending count instead of a
success toast when a run stopped at the cap
* feat(gallery): warn with the pending count when a thumbnail run stops at the cap
* fix: thumbnails are written atomically and served under mtime-versioned urls
Cards showed "Failed to load" for thumbnails that were valid on disk. The
browser had fetched them while the thumbnail run was still writing the
file and, because image responses are cached for an hour, kept the broken
bytes. The card also had no fallback to the original.
- image and video thumbnails are written to a hidden temp sibling and
renamed into place, so a partial file is never visible at the final path
- every thumbnail_url (gallery listing, prompt images, duplicates, autotag
listing) carries ?v=<mtime>, so a regenerated thumbnail gets a new URL
instead of the cached old one
- a 404 from the image serve route is marked no-store; ComfyUI's cache
middleware would otherwise let browsers cache the miss for an hour
- image-helpers gains fallbackImageSource(): the original is tried once
before a card gives up
Test hygiene: the autotag tests now stub ComfyUI's server module like the
other API test modules, so the module passes when run on its own.
* fix(gallery): display settings apply at load, columns work, media info toggles
Default View Mode, Items per Page and Grid Columns were only applied when
Save was pressed, never when the page loaded; the column setting swapped a
Tailwind class that was never compiled (only 3, 6 and 8 exist) and a wider
breakpoint class overrode it anyway; Show Media Info was saved and never
read.
- gallery-settings.js validates what localStorage holds (view mode, limit
option, column range, booleans) and falls back to defaults per value
- settings are applied in the constructor before the first load; changing
the page size reloads, toggling media info re-renders
- the column count sets an explicit grid template through a CSS variable
that wins on desktop widths
- media info lines honour the toggle in grid and list views
- image load failures are handled by one delegated listener that tries the
original before showing the failed-to-load card
* feat(gallery): duplicate scan runs as a background job with progress; infinite scrolling
The duplicate scan was one blocking request with a spinner: no progress,
a modal that could not be put away, and a result lost if the tab closed.
- POST /prompt_manager/scan_duplicates/stream starts the scan as a
background job (same ScanJob fan-out as the output scan) or attaches to
the running one; GET /prompt_manager/scan_duplicates/status reports it.
The job holds the duplicates lock, so deletes are refused while it runs.
- hashing runs on the configured worker threads and reports progress
about every percent
- the gallery modal shows a progress bar, closing it minimizes to a pill
that reopens it, results wait in the pill if the modal was closed, and a
reload re-attaches to a running scan
Infinite scrolling is a new Display Setting (off by default): the next page
is appended when the viewport nears the bottom, pagination controls hide,
and the load-more rule lives in list-state.js where it is unit tested.
Thumb-size buttons and the column setting no longer override each other.
* fix(gallery): Rescan Output Folder drops the server listing cache and rewalks
The button only re-requested the listing, which the server answers from a
30-second in-memory cache, so a rescan showed exactly what was already on
screen.
- POST /prompt_manager/gallery/rescan invalidates the gallery cache,
rewalks every output root and returns the file and root counts
- the button calls it, closes the settings modal, reloads the folder tree
and the first page, and reports what it found
* feat: tag search box autocompletes, narrowed by the tags already typed
- PromptDatabase.suggest_tags(prefix, with_tags, limit): tags starting
with the prefix, counted over the prompts that carry every context tag,
context tags excluded, ordered by count then name; LIKE wildcards in the
prefix are escaped
- GET /prompt_manager/tags/suggest?q=&with=&limit= exposes it (limit
clamped to 1..50, context capped at 10 tags, registered before the
{tag_name} routes)
- tag-autocomplete.js splits the comma-separated input into committed tags
and the one being typed, builds the query, applies a pick and handles
wrap-around keyboard selection
- the dashboard Tags box shows suggestions with counts on focus and while
typing, arrow keys and Enter/Tab pick, Escape closes, and each pick
immediately offers the next tag narrowed by the previous ones
* feat(dashboard): infinite scrolling for the prompt list, saved default plus session toggle
- PromptManagerConfig.INFINITE_SCROLL (web_ui.infinite_scroll) is reported
and saved through the settings API; only booleans are accepted
- Settings modal gains an "Infinite Scrolling" default; saving applies it
at once
- the results header gains an "Infinite scroll" checkbox that overrides the
default for the current session without saving
- when on, the next page is appended as the list nears its bottom (same
load-more rule as the gallery), page controls hide, a short page keeps
loading until the list fills, and a refresh after an edit restarts from
the top
* fix: first CI run on Windows and macOS; tests now enforce closed database files
The first push of the branch ran the Tests matrix on all three operating
systems. Every Python job failed; the causes were:
- numpy missing in CI: the WD14 engine tests import it (requirements-dev)
- macOS: temp directories resolve through /private; the admin API tests
now use the real path like the gallery roots they compare against
- Windows: the startup banner printed an emoji, which raises
UnicodeEncodeError on a legacy-code-page console and aborted loading
the whole node; the banner is ASCII now and a test loads the package
under a cp1252 stdout
- Windows: gallery subfolders came back with backslashes, which neither
the folder tree nor the search filter understand; they are "/" joined
on every platform. A relative DEFAULT_DB_PATH kept its forward slashes
after being joined to the extension root; it is normalised now
- Windows: sqlite connections opened with `with sqlite3.connect()` are
not closed by the context manager and linger until garbage collection,
which holds a file lock there. Production sites (vacuum, diagnostics,
restore inspection) close explicitly; a restore that swaps the API's
database object closes the previous one; PromptModel keeps a registry
so a shutdown or a test can close every instance
- Windows: served files were still open when their temp directory was
removed (tests now read the body), a log download test compared text
mode newlines, two tests compared paths with a hard-coded separator
tests/open_handles.py makes the Linux run fail the way Windows would:
every database test base checks, right before deleting its files, that
nothing in the process still holds them open.
* fix: Windows case-folded gallery roots in tests, "/" subfolders everywhere, longer busy timeout
Second CI run on Windows:
- gallery roots are stored resolved and case-folded there; the admin API
tests now compare against that form instead of the mixed-case temp path
- the subfolder fallback for images outside every root also uses "/", and
the test expects it
- a relative diagnostics db path is normalised after joining
- "database is locked" under many writers on a slow disk: connections now
wait up to 30 seconds (connect timeout and busy_timeout), long enough for
a scan or thumbnail run while ComfyUI saves images
* fix: serve every gallery video format, validate settings before applying, dedupe output dirs
Review findings on #153 (Codex):
- the image serve route allowed only three video types while the gallery
lists seven, so .avi/.mkv/.m4v/.wmv cards opened to a 403; the serve
allow-list now covers every gallery video format
- a settings payload with one invalid field still applied the valid ones
before returning 400; the whole payload is validated first and nothing
changes on failure. webui_display_mode is validated too
Third Windows CI run:
- the same output directory spelled twice (different case or a link) was
listed twice in diagnostics and served roots; both dedupe on the
resolved, case-folded path
- the root-less subfolder test expects the "/" form
Code-quality bot: index definitions are (name, target) pairs instead of
implicitly concatenated strings; redundant local asyncio imports and
two trivial lambdas in test stubs removed.
* docs: version 3.3.0, README changelog and refreshed sections
Bump to 3.3.0: new features (worker threads, background scans, tag
autocomplete, infinite scrolling, video prompt scanning) and API contract
changes (POST-only side-effect routes, relative image paths, paginated
search) go beyond a patch release.
README: changelog entries for 3.2.4 and 3.3.0, refreshed bulk scanning,
gallery settings, search, configuration and testing sections, and the
roadmap's recently-completed list.
|
||
|
Vito
|
6c7e4d5b35 |
feat: release 3.2.4 — recently used sorting, run counts, positive-prompt image linking (#152)
* chore: start 3.2.4 * feat: track prompt usage (run_count, last_used_at) with server-side sorting - Migration adds last_used_at and run_count; existing prompts are estimated from linked images (count, newest image time, else created_at). The backfill only touches NULL rows, so it self-heals rows written after a downgrade or an interrupted upgrade. - Every node run records a use, including re-runs of an existing prompt; a tracking failure is logged and never blocks generation. - search_prompts/get_recent_prompts accept a whitelisted sort key (last_used, created, run_count, rating, text); default stays newest first. Refs #90 * feat: sort param on /recent and /search; microsecond last_used_at - /prompt_manager/recent and /search accept ?sort= (whitelisted in the DB layer; unknown values fall back to newest first). - last_used_at is written with microseconds from Python so a re-run in the same millisecond as another save still sorts first; ordering stays correct against millisecond values from the backfill. Refs #90 * feat: server-side sort in admin UI with Recently Used / Most Used and run badges - New web/js/prompt-list-sort.js owns the sort options, validation and request URLs; admin.js fills the dropdown from it and re-fetches from the server on change, so every sort spans all pages (the old client-side sort only reordered the current page). - Default sort is Recently Used; prompt cards show a run-count badge with the last-used time on hover. - The node's Recent button now lists recently used prompts. - Contract test keeps the UI sort keys and the node's key in sync with the server whitelist. Refs #90 * fix: link images and count runs by the positive prompt, not node execution With a negative-prompt PromptManager node in the workflow, every executed PromptManager node pushed onto the image-linking queue, so images could be linked to the negative prompt or to a previous run. ComfyUI also skips unchanged nodes, so re-runs of the same prompt were never counted. - utils/prompt_graph.py finds the PromptManager nodes feeding a sampler's positive input (KSampler, CFG/Basic guiders, ControlNet and combine pass-throughs, PromptManagerText into encoders), falling back to all PromptManager nodes when none can be traced. - Runs are counted by a ComfyUI on-prompt hook when a workflow is queued; prompts not yet saved are counted when the node first saves them. - Nodes receive the queued graph (hidden PROMPT/UNIQUE_ID). Only the run's positive prompt becomes the current prompt; only batch items with linked text are pushed onto the FIFO queue. - The image monitor links from the image's own metadata first, using the positive node, and falls back to the queue for batch items. Refs #90 * fix: address review — count cached runs of linked text, keep repeated pending runs - Text from known pure string nodes (core PrimitiveString/StringConcatenate, WAS Text Concatenate/Multiline, pysssss ShowText) is resolved at queue time, so cached re-runs of such prompts are counted and their images link from metadata. Unknown nodes are never guessed; the node compares its real text with the queue-time resolution and falls back to counting and queueing itself on any mismatch. - A new prompt queued several times before its first run keeps every pending run; the first save records all of them. - Registration flag is a threading.Event; consistent unittest imports; logging instead of print in the init fallback. * refactor: unquote PendingFirstUse annotation so Tuple usage is visible |
||
|
|
c01d8e3c12 |
feat: recursive subfolder filtering + gallery tree navigation (#140)
* feat: recursive subfolder filtering + gallery tree navigation + thumbnail sizing - Recursive subfolder filtering: selecting an intermediate folder shows all child images - Gallery tree navigation panel with collapsible folder hierarchy (sticky, resizable) - Thumbnail size controls (S/M/L) for adjustable grid density - Fix path collision when multiple gallery roots share overlapping relative paths - Fix auto-detect mode collapsing nested folders to ambiguous basenames - Security test suite for path traversal edge cases Addresses feedback from #126. Fixes code review items P1/P2 from #135. Co-Authored-By: Kai Bowie <kaigtb@proton.me> * style: fix Black formatting in test_path_security.py * feat(gallery): add folder tree sidebar with hierarchical navigation (#126) - Add collapsible folder tree sidebar to gallery view - Support hierarchical year/month/date folder navigation - Include ancestor path expansion for proper tree structure - Add subfolder filtering to gallery image queries - Update admin subfolder dropdown with ancestor support - Fix trailing whitespace in SQL queries Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Kai <noreply@gtb.ai> Co-authored-by: Kai Bowie <kaigtb@proton.me> Co-authored-by: Kai <kai@tachi-02.localdomain> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vito
|
8146250274 |
feat: PromptSearchList batch workflow support & image linking v3.2.2 (#139)
* fix(search): PromptSearchList crash, filters, and batch output (#52) - Fix empty list crash (OUTPUT_IS_LIST requires at least one element) - Add partial tag matching (LIKE instead of exact) - Add preview and count outputs for batch visibility - Add skip_multipart filter for Clip_N video prompts - Add LoRA-only prompt filter - Collapse newlines for StringOutputList compatibility * feat(db): add tag_partial parameter for LIKE tag matching (#52) Allows search_prompts to use LIKE matching for tags instead of exact. Used by PromptSearchList for discovery-style search. Existing callers default to exact matching (unchanged behavior). * feat(tracker): add FIFO prompt queue for batch image linking (#52) Batch workflows encode all prompts first, then save all images later. The queue preserves encode order so each image links to the correct prompt by position (pop on save matches push on encode). * fix(monitor): multi-strategy image-to-prompt linking (#52) Previous approach relied on live prompt tracker state, which was always stale for batch workflows (all images linked to last prompt). New 5-strategy priority chain: 1. FIFO queue pop (batch-correct by position) 2. Metadata-based DB lookup (extracts prompt from PNG) 3. Prompt snapshot at file-creation time 4. Live prompt tracker 5. Most recent DB prompt fallback * test: add tests for prompt queue and search filters (#52) - 6 queue tests: FIFO ordering, batch simulation, exhaustion - 15 filter tests: newline collapse, Clip_ filter, LoRA-only filter - Fix _make_tracker helper to include queue fields * chore: bump version to 3.2.2 * fix: address code review — regex backtracking and queue expiry (#52) - Replace backtracking-prone LoRA-only regex with subtraction approach (re.sub removes tags, check if anything remains). Fixes CodeQL alert. - Add expiry check to pop_next_prompt — skips stale entries to prevent incorrect linking from accumulated queue entries across sessions. - Cleanup thread now also prunes expired queue entries. |
||
|
Vito
|
00284d92da |
feat: LoraManager integration — trigger words & example import (#52) (#136)
* feat: add LoRA utility module for LoraManager integration (#52) Detection, metadata reading, trigger word cache, and prompt injection for ComfyUI-Lora-Manager. All functions return empty results when LoraManager is not installed. * feat: add IntegrationConfig for opt-in third-party integrations (#52) * feat(api): add LoraManager integration endpoints (#52) Detection, enable/disable, scan/import, trigger word lookup, and cache refresh endpoints under /prompt_manager/lora/*. * feat: inject LoRA trigger words into prompts at encoding time (#52) When enabled, scans for <lora:NAME:WEIGHT> tags in prompt text and appends trigger words from LoraManager metadata. Behind config toggle. * feat(ui): add LoraManager integration settings and import UI (#52) Integrations section in settings with auto-detection badge, enable toggle, trigger word toggle, and Import LoRA Data button. * fix: detect LoraManager with case-insensitive directory scan (#52) * fix: simplify LoraManager heuristic to match real install structure (#52) * fix: find ComfyUI root via folder_paths and handle symlinked installs (#52) * fix(ui): add pointer-events-none to toggle switch overlays (#52) The styled div was intercepting clicks meant for the sr-only checkbox input, preventing toggle switches from being clickable. * fix: use lora-manager category and tag for imported LoRA data (#52) * fix: use rglob to find metadata in lora subdirectories (#52) * feat: discover LoRA dirs from extra_model_paths.yaml and folder_paths (#52) Only models/loras under the ComfyUI root was checked. Now also parses extra_model_paths.yaml and uses folder_paths.get_folder_paths('loras') at runtime to find all configured LoRA directories. * fix: use example prompts instead of trigger words, serve LoRA preview images (#52) - Prompt text now uses civitai example prompts (images[].meta.prompt) when available, falling back to model name instead of trigger words - Image serving now allows paths within LoRA directories when the integration is enabled, fixing 403 errors on preview images * fix: handle null civitai field, link all preview images (#52) - Guard against civitai: null in metadata (was crashing the scan) - Link all local preview images per LoRA, not just the first - Add get_civitai_image_urls() for future remote image support * feat: reimport clears previous lora-manager data first (#52) Clicking Import LoRA Data now deletes all existing lora-manager category prompts before scanning, ensuring a clean reimport. * feat: download civitai example images during LoRA import (#52) Images from civitai.images[] are downloaded to data/lora_images/ cache and linked to prompts alongside local preview files. Cached files are reused on reimport. Image serving allows the cache directory. * feat: add CivitAI API key setting for authenticated image downloads (#52) Most civitai example images (especially NSFW) require authentication. Adds API key field to the Integrations settings panel, passed as Bearer token when downloading example images. * feat(ui): add progress modal for LoRA import (#52) Closes settings modal and shows a dedicated progress modal with progress bar, status text, and processed/imported counts during LoRA import. Auto-closes after completion. * fix: reduce download timeout to 5s, update progress per LoRA (#52) 5K images at 15s timeout was painfully slow. Reduced to 5s fail-fast. Progress now updates for every LoRA with image count, not every 5th. * perf: use 512px thumbnails and parallel downloads for LoRA images (#52) Full-size civitai images averaged 5.6MB each (27GB total for 5K images). Now requests /width=512/ thumbnails (~50-100KB) and downloads 8 in parallel per LoRA. Expected speedup: ~100x smaller + 8x parallel. * fix: resize downloaded images to 512px thumbnails locally (#52) Civitai CDN returns 401 for /width=N/ thumbnail URLs with API key auth. Instead, download the original and resize to 512px via PIL before saving. Reduces disk usage from ~5MB to ~30-50KB per image. * docs: update README for v3.2.1 LoRA Manager integration (#52) - Add LoRA Manager integration section with setup guide and CivitAI key docs - Add folder filter section with rescan note for existing libraries - Add v3.2.1 changelog entry, split from v3.2.0 - Add WIP notice for LoRA Manager feature - Update AutoTag section to include WD14 models - Fix stale references (KikoTextEncode, outdated file structure) - Remove dated v2 development note - Add screenshots for settings, integration, and filtered results - Fix code review items: remove unused constant, add comments to empty excepts - Bump version to 3.2.1 * test: add unit tests for LoRA integration and folder filter (#52) - test_lora_utils.py: 32 tests covering trigger word extraction, example prompts, image URLs, metadata parsing, cache dir, TriggerWordCache - test_lora_database.py: 17 tests covering delete_prompts_by_category, folder filter search, get_prompt_subfolders, LoRA import workflow - test_config.py: 7 new IntegrationConfig tests for structure, enable, partial update, reset, and roundtrip * fix: remove unused MagicMock import in test_lora_utils (#52) |
||
|
Vito
|
6fc6e683eb |
feat: multi-directory gallery scan with folder filter (#126) (#135)
* refactor(api): add multi-root output dir infrastructure (#126) - Change _gallery_cache from None to {} (dict keyed by path string) - Update invalidate_gallery_cache() to clear dict instead of setting None - Add _get_all_output_dirs() helper as single source of truth for resolving configured + fallback output directories - Update _enrich_prompt_images() to iterate all output roots when computing relative paths and thumbnail URLs * feat(api): support multiple gallery scan paths in settings endpoint (#126) * feat(ui): dynamic multi-path settings with add/remove buttons (#126) * cleanup: remove dead galleryRootPath assignment in loadSettings * feat(api): multi-directory gallery scan with subfolder filter and multi-root serve (#126) * feat(db): add folder filter to search_prompts and get_prompt_subfolders (#126) * feat(api): add subfolders endpoint and folder filter to prompt search (#126) * feat(ui): add folder dropdown filter to search bar (#126) * fix: escape LIKE wildcards in folder filter and remove dead _gallery_cache_time field * feat(scan): scan all media types across all output dirs with batched extraction (#126) * chore: gitignore SQLite WAL-mode temp files * fix: enrich prompt images with URLs and check all output dirs for serving - Extract _enrich_images() for flat image lists, used by get_prompt_images - get_prompt_images now adds url/thumbnail_url (fixes stuck "Loading images...") - serve_image checks all output dirs not just first (fixes 403 on multi-path) - Fix diagnostics monitor check to read current module attribute, not stale import * feat(ui): add version-scoped update notice popup for rescan (#126) * chore: bump version to 3.2.0 |
||
|
Vito
|
ad32a51497 |
fix: database path config option now works (#119) (#130)
* fix: resolve database path from config.json (#119) PromptDatabase() was hardcoded to "prompts.db" relative to CWD, ignoring the DEFAULT_DB_PATH setting from config.json. Added _resolve_db_path() that reads from PromptManagerConfig when available and resolves relative paths against the extension root directory. This fixes Docker deployments where the CWD differs from the extension. * test: add unit tests for database path resolution (#119) Cover explicit absolute/relative paths, config fallback, relative config paths resolved to extension root, and PromptDatabase constructor integration with _resolve_db_path. * docs: update database path configuration instructions (#119) Replace outdated advice to edit py/config.py with correct config.json approach. Add Docker-specific guidance for absolute paths. * chore: bump version to 3.1.5 * fix: remove unused import and variables from test (#119) Address code review: remove unused MagicMock import and unused db local variables in constructor tests. |
||
|
Vito Sansevero
|
2737a4b1ef |
style: apply Black formatter (line-length=88) to all Python files
Automated formatting pass across 30 files to establish consistent code style enforced by CI. No logic changes. |
||
|
Vito Sansevero
|
4deda8542d |
feat: normalize tag storage with junction tables
Phase 5 scalability and polish: - Add tags + prompt_tags junction tables replacing JSON column - Migrate existing JSON tags via json_each() on startup - Rewrite all tag queries to use junction tables (O(1) vs O(n)) - Simplify rename/delete/merge tag operations to single SQL - Batch-attach preview images to prompt list responses (fix N+1) - Fix fetchone() fragility under thread contention (defensive defaults) |
||
|
Vito Sansevero
|
9e337ed02a | fix(database): handle skipped prompts in tag operations | ||
|
Vito Sansevero
|
760b2a0f48 |
feat(tags): add Phase 2 tag management enhancements
- Bulk tag operations: rename, delete, merge across all prompts - Right-click context menu on tags (rename/delete/merge) - Click prompt card → dashboard search, click thumbnail → ViewerJS - Usage bars, prompt count badge, staggered card animations - URL-synced tag selection (#/tags/landscape,portrait?mode=or) - Untagged prompts filter with [untagged] entry |
||
|
Vito Sansevero
|
d4f13e40b6 | fix(database): skip duplicates in image linking | ||
|
Vito Sansevero
|
86815913c1 | feat(models): add UNIQUE constraint for prompt_id, filename | ||
|
Vito Sansevero
|
ffaf5fd3d8 | feat(database): add method to retrieve all images with prompts | ||
|
Vito Sansevero
|
22593e8514 |
docs: add comprehensive docstrings to all Python modules
- Add Google-style docstrings to main node files (prompt_manager.py, prompt_manager_text.py) - Enhance database module docstrings with detailed parameter and return documentation - Add comprehensive docstrings to all utils modules with usage examples - Document API endpoints and configuration classes thoroughly - Update module-level docstrings in all __init__.py files - Ensure consistent naming (ComfyUI_PromptManager) across all documentation - Follow Python best practices for docstring formatting |
||
|
Vito Sansevero
|
f161fc4c1f |
fix(database): resolve SQLite ORDER BY syntax error in GROUP_CONCAT
Fix compatibility issue with older SQLite versions that don't support ORDER BY clause inside GROUP_CONCAT function. - Remove ORDER BY from GROUP_CONCAT in SQL queries - Add GROUP_CONCAT for created_at dates - Sort IDs manually in Python after fetching results - Maintain same behavior of keeping oldest prompt as primary This resolves the startup error: "sqlite3.OperationalError: near 'ORDER': syntax error" The fix ensures compatibility with all SQLite versions while preserving the original functionality of keeping the oldest duplicate. |
||
|
Vito Sansevero
|
caa48d7cdf | refactor(database): enhance prompt ID validation logic | ||
|
Vito Sansevero
|
2c890f7eaa | style(models): Change log level from info to debug | ||
|
Vito Sansevero
|
52ffdeda2e | feat(database): add method to find duplicate prompts | ||
|
Vito Sansevero
|
44afbbcd58 | fix(database): handle empty categories in query | ||
|
Vito Sansevero
|
92e904767d | feat(database): add pagination to get_recent_prompts | ||
|
Vito Sansevero
|
ee21a695cd | refactor(database): improve duplicate cleanup logic | ||
|
Vito Sansevero
|
3b158d0ec3 | feat(models): integrate logging for database operations | ||
|
Vito Sansevero
|
850900fcd7 | feat(logging): integrate logging in database operations | ||
|
Vito Sansevero
|
c639fa79a7 | fix(database): delete related images before prompts | ||
|
Vito Sansevero
|
c22cd36e7c | fix(models): correct foreign key type mismatch | ||
|
Vito Sansevero
|
e0ac927a0d | fix(database): resolve foreign key constraint error | ||
|
Vito Sansevero
|
d4c1689ce4 | feat(database): add prompt storage and retrieval system |