68a914da5dbfdafdd5067079a650efd02f925a9f
555
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vito Sansevero
|
68a914da5d | Merge remote-tracking branch 'origin/main' into feat/247-foreign-workflows-pr3 | ||
|
Vito
|
2941a9b748 |
feat(inspect): choosing a file opens the viewer with its details (#448) (#458)
* fix(inspector): give image tags their own labelled section (#449) The inspector drew a file's tags as a bare chip row right under the negative prompt, so they read as part of it. The side panel, the viewer details and History's current file now draw them in labelled sections in the prompt sections' frame: "Image tags · N" for the file's own tags, then "Prompt tags · N" or "Recipe tags · N" for the linked tags. Each count is true: repeats are removed within a source only. Recorded as D154. * feat(inspect): choosing a file opens the viewer with its details (#448) On Inspect a file, choosing or dropping a file now opens the shared viewer on it at once, details showing, instead of previewing it and waiting for "Open file metadata". Details show on this page even when "Collapse viewer details by default" is on. An unreadable file or an AVI/WMV note stays on the page; the heading action reopens any selection. A new selection that starts while the viewer shows the previous file closes it first, so its blob URL is never revoked under an open element. Recorded as D155. * fix(inspect): open a video's viewer only once its preview settles (#448) Replacing a video while both the inline preview and the auto-opened viewer's video were still starting deadlocked WebKit (D62). The viewer now opens for a video once its inline preview reports metadata or an error (or after 15 s). * test(inspect): cover the AVI note and an open dialog holding the auto-open back (#448) Frontend coverage for inspect.mjs fell below its floor (branches 91.04 < 92, functions 92.85 < 93): no spec reached the AVI toast or the open-dialog guard, and the video wait had a page-end callback no spec reached. The wait now ends through its own callback at page end, and two specs cover the AVI note and a selection landing under another dialog. * test(comfyui): the Inspect sender spec takes the viewer choosing a file opens (#448) The real-host spec chose a file and then pressed "Open file metadata"; with D155 the viewer is already open, so the press waited behind the modal until the test timed out in the merge queue (classic-start and nodes2-start). |
||
|
Vito
|
505c16a724 |
test(collections): order the review M1 reads by events, not the 300 ms timer (#459) (#463)
The M1 test let the rename answer at once and changed the hash only after the toast, so on a loaded host the rename's 300 ms count read could start before Fresh's page read; the page then waited on that one read and the poll for two held reads failed. Hold the rename's answer until Fresh's read is held. |
||
|
Vito
|
66e567cc95 |
fix(workspace): remove the 1-5 prompt rating from the workspace (#447) (#461)
The star is only for favorites (maintainer ruling 2026-10-28). The workspace no longer shows or edits a prompt rating: no Rating field in the Library editor, no stars on Library cards, no "Highest rated" sort on Images or the Library, and no rating column or rated-images total in System > Statistics. The editor never sends `rating`, so a workspace save keeps a stored rating as it is. A saved view or Library visit with the retired rating sort opens on Recently added. Data and API are unchanged; legacy pages keep their ratings. Recorded as deviation D153 (amends D14, D23, D96); concept comparisons map the concept through it. |
||
|
Vito Sansevero
|
35b17a3b49 | Merge remote-tracking branch 'origin/main' into feat/247-foreign-workflows-pr3 | ||
|
Vito
|
dff08bc161 |
fix(images): open Masonry by default, keep a saved Default layout, crop portraits from the top (#451) (#455)
- A Default layout or Thumbnail size saved in System now reaches the next Images visit in an open tab (this tab or another): the last visit records the defaults it started from, and a value its toolbar never changed follows newer defaults. A toolbar choice stays the current view's own. - Images opens in Masonry when the browser has no stored Default layout (gallery-settings.js DEFAULT_SETTINGS.workspaceLayout); stored choices are kept. - Grid/List card art, Library card art, the inspector cover and the Tag image preview crop with object-position 50% 20%, so portrait images keep their top. - D157 in TESTING.md; concept comparisons start from the concept's stored Grid. |
||
|
Vito
|
aad3a37bcf |
fix(inspector): give image tags their own labelled section (#449) (#454)
The inspector drew a file's tags as a bare chip row right under the negative prompt, so they read as part of it. The side panel, the viewer details and History's current file now draw them in labelled sections in the prompt sections' frame: "Image tags · N" for the file's own tags, then "Prompt tags · N" or "Recipe tags · N" for the linked tags. Each count is true: repeats are removed within a source only. Recorded as D154. |
||
|
Vito
|
d03d0ed1c5 |
fix(history): count cached re-runs of runtime-linked text as tracked runs (#403) (#445)
* fix(history): count cached re-runs of runtime-linked text as tracked runs (#403) Since #347 a cached re-run of runtime-linked prompt text (PreviewAny or an LLM into a PromptManager text input) counts in the Library, but its history capture had no raw text, so the history page's "tracked runs" stayed at the number of jobs in which the node executed. The #347 receipt now also keeps the raw, prepend and append text the node gave the history capture. When a cached node's graph inference leaves a field unknown, the capture takes it from the receipt that matches the cache entry, labeled inferred/graph_semantics, so the job joins the prompt's history scope and gets its prompt hint. No schema change and no new evidence vocabulary; no backfill. Fails closed on no receipt, a disagreeing known field, equal outputs with different fields, oversized or invalid text and a full budget (least recently used receipts give up their fields first; counting stays). * docs(testing): link the whitespace history gap to #438 |
||
|
Vito
|
02190d03a8 |
feat(library): add tags to many selected prompts and select every loaded prompt (#442) (#453)
* feat(library): set the category on many selected prompts (#437) The Prompt library shelf gets the concept's Images selection checkbox on each card and its .bulkbar (N selected, Set category, Clear), recorded as D148. Set category opens the shared one-value dialog and posts {prompt_ids, category, expected_epoch} to POST /prompt_manager/bulk/category: the server checks the storage epoch the shelf was read under, then sets the category on the Library parents in one BEGIN IMMEDIATE transaction, writing only rows that change. A restored storage answers 409 storage-changed and writes nothing. The legacy admin and images_filter bodies are unchanged. This replaces the legacy admin page's bulk Set Category, a cutover prerequisite (#334). Closes #437 * feat(library): add tags to many selected prompts and select every loaded prompt (#442) The Library selection bar (D148) gets Add tags and select-all of the loaded shelf, the legacy admin page's bulk Add Tags and select-all checkbox. Add tags posts {prompt_ids, tags, expected_epoch} to POST /prompt_manager/bulk/tags: in one BEGIN IMMEDIATE transaction the server checks the storage epoch, normalizes the names as image tags are, gives each Library parent only the names it lacks (casefold), finds or inserts tag rows by exact name only when some prompt gets them, and skips prompts that would pass 20 tags. Prompt tags only. Both bulk/tags forms refuse a non-object body with 400. Ctrl/Cmd+A in the grid and "Select all N loaded" in the bar select every loaded prompt; the offer follows each loaded batch. Closes #442 * docs(library): record Add tags and select-all as D151 (#442) |
||
|
Vito
|
40dc0e5417 |
feat(collections): add to and remove from collections in the selection bar and the inspector (#331, PR 3) (#435)
* feat(saved-views): keep a collection in a saved view Images saved views gain a collection key (#331 D140): empty or a collection id of 1 to 9 digits. Writes refuse favorites together with a collection; older rows read an empty collection, and a stored row with both opens the collection, as the listing does for a URL with both. Part of #331 * feat(collections): store collections in the library database * feat(collections): add, remove and rename collections and keep members through duplicate cleanup * feat(collections): add collection routes and the Images collection filter * feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * fix(collections): address review findings * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * feat(collections): browse a collection from the sidebar The sidebar gains the concept's COLLECTIONS group (D136): one row per collection with its folder icon, name and listed-member count, without the PROPOSED badge; the label shows once a collection exists. A row opens #/collection/<id>, the Images page filtered to that collection (D137): the collection's name as h1, Collection as breadcrumb and title, Edit to rename or delete it through the saved views' Edit dialog with collection nouns, and a note counting members the sources do not list now. An unknown or deleted collection shows a page that says so with a link to Images. A saved view on a deleted collection lists nothing and says so (MC6, D140). The list is read again after a restore and the counts 300 ms after a change. Part of #331 * test(collections): compare the COLLECTIONS group with the concept The concept hides COLLECTIONS at 950 px and below, which would leave a phone no way to open a collection; production shows the rows there in the wrapped layout of every other sidebar nav (D136). The shell comparison now compares the group instead of filtering it out: the concept fixture seeds Landscapes (12) and Experiments (8), the rows compare by name order, and at phone width the nav compares against the concept's own sidebar-nav layout. The collection rows are checked at 1600x1100 and 390x844 in both themes. Part of #331 * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(browser): install the slideshow clock before wrapping setTimeout The slideshow fixture wrapped window.setTimeout before page.clock.install(), so the viewer's timer ran on real time and clock.runFor() did not drive it; a late WebKit timer failed 'slideshow exposes exact state labels' on CI. * feat(collections): add and remove from the selection bar Add to collection follows Set category in the Images selection bar and opens the concept's simpleInput ("Add selected images to collection", "Collection name", Add) with the existing names as a datalist. Remove from collection follows it while a collection filter is active and reloads the listing in place, keeping the checked files still listed. Every outcome uses the design's strings (section 7.4). The routes take at most 500 explicit files, so an "all N matching" selection is sent only when every match is loaded, checked and at most 500; otherwise the action toasts "Select at most 500 images." (D138, #331). * fix(collections): address review findings - A list read superseded by a newer one (the 300 ms count read, a restore) no longer shows "could not be loaded": the collection and saved-view pages wait for the list to settle. - A listing that finds the open collection gone shows the deleted page once the list no longer holds it; when the list read fails or still holds it, the page keeps the listing's error and Retry instead of staying blank, and each failed read is reported again. - The missing-members note follows every list read and goes away at zero. - A rename that settles after its Edit dialog closed leaves any dialog opened since alone. - The deleted-collection, list-failed and saved-view failure pages focus their title after a route change, as every page does. Part of #331 * feat(collections): add and remove from the inspector and the viewer The shared inspector's footer ends with Collections… (folder icon) after Prompt history for a listed file, in the Images side panel and the viewer. Its dialog reads the file's membership, lists every collection as the concept's checkbox rows, takes an optional new name, and Save sends one request per change (new name, added, removed). A failure keeps the dialog with the membership read again. On a collection page a change to the open collection reloads the listing at once from the side panel and only when the viewer closes from the viewer. The inspector concept comparison draws the button into the concept after Prompt history (D139, #331). * test(collections): compare the collection page and dialog with the concept images-concept compares Add to collection in place in the selection bar (only Remove links… stays omitted) and the collection page's heading, toolbar and filter row against the concept's #collection-landscapes, with D137's Edit drawn into the concept. collections-concept compares the Add to collection dialog with the concept's simpleInput (D138: "Add", an empty field, the datalist of names), in both themes at 1600 and 390 px, with a mutation check. TESTING.md gains the Collections contract (database, keys, missing members, duplicate move, restore, routes, rules, limits, the PR 1 rulings and the all-matching ruling), D136-D140, and amends D28 and D8; docs/application-redesign.md gets the delivery-status line. The coverage baseline adds the two new modules; optional collection callbacks are null rather than no-op defaults so they do not count as uncovered functions. * test(coverage): keep main's floors for files this PR does not touch coverage:update raised nine floors of untouched files from one run; metadata-autocomplete.js branches then measured 84.84 against 85. * fix(collections): address review findings - M1: after a failed Save the Collections dialog shows the membership loading and keeps Save disabled until the re-read lands, so a stale plan cannot be sent. - M2: the changes a Save made reach the page only when the dialog closes, so a collection page's reload cannot remove the dialog before its failure is read. - L4: a New collection name equal to a listed collection ignoring case is that collection's checkbox, checked; never an add then a remove. - L5: a name that cannot be a collection name is refused on the field; nothing is sent, the checkbox edits stay, no re-read. - L6: focus starts on the field and moves to the first checkbox once listed; closing after a re-render replaced the trigger focuses the inspector's current Collections… button or its heading (openContentDialog takes an optional focusTarget). - L3, L7: TESTING.md D138/D139 describe the stale-source and deleted collection outcomes and the above; the viewer spec asserts no listing request while it is open, and a browser test covers M2. * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. * test(duplicates): a removed copy with a collection and a category moves both (#331, #332) * test(collections): map the Add to collection suggestions to the shared dialog's datalist (#331) After the merge with #332 PR 3 the input dialog places its datalist after the field with the id workspace-input-suggestions; the concept mapping follows. |
||
|
Vito
|
ac378f07d3 |
feat(library): set the category on many selected prompts (#437) (#446)
The Prompt library shelf gets the concept's Images selection checkbox on each
card and its .bulkbar (N selected, Set category, Clear), recorded as D148.
Set category opens the shared one-value dialog and posts
{prompt_ids, category, expected_epoch} to POST /prompt_manager/bulk/category:
the server checks the storage epoch the shelf was read under, then sets the
category on the Library parents in one BEGIN IMMEDIATE transaction, writing
only rows that change. A restored storage answers 409 storage-changed and
writes nothing. The legacy admin and images_filter bodies are unchanged.
This replaces the legacy admin page's bulk Set Category, a cutover
prerequisite (#334).
Closes #437
|
||
|
Vito
|
827de84f40 |
fix(workspace): keep an open dialog when a restore it saw finishes (#415) (#441)
When the page's Activity store saw a restore run (its 15 s job-list poll landed inside it), the S9 reload on finish re-rendered the open page and closed any open dialog, dropping what was typed without notice. Whether that happened depended on the poll's timing. Now the S9 reload waits while any dialog is open and runs once the last one closes (createRestoreReload, a MutationObserver on dialog[open]), so the dialog keeps its entries and its next write meets the existing restored-storage refusal with Reload, as when the page never saw the restore. An open record-changing review is the exception: its staged plan no longer describes storage, so it closes as dismissed, as before (closeReviewDialog). Tests: followRestore makes the page see a restore run and finish without timers; browser cases for Create tag, New prompt and the restore review; unit tests for the deferral and closeReviewDialog. |
||
|
Vito
|
9eb5e9453d |
docs(redesign): record Copy settings summary and Local Inspect PR L3 as shipped (#436) (#440)
* docs(redesign): record Copy settings summary and Local Inspect PR L3 as shipped (#436) * docs(redesign): cite D117 and fix the stale Task 9 tense in local-resolver.mjs (#436) * docs(inspect): local-resolver.mjs header says Task 9 shipped (#436) |
||
|
Vito
|
f5856076ce |
fix(test): own /dev/shm reset scratch by a flock, not a pid (#419) (#439)
A browser server inside an --ipc=host container shares the host's /dev/shm but not its PID namespace, so the pid probe saw live servers' scratch folders as orphans and deleted them. A live owner now holds a flock on owner.lock in its folder; the kernel drops it when the owner dies, in any namespace. The cleaner removes only held-named folders whose lock is free and leaves legacy and staging folders alone. Without flock (Windows) the scratch stays on disk. |
||
|
Vito
|
4040007264 |
fix(workspace): keep open pages valid across a restart and recover a changed source (#430) (#434)
The gallery root scope and the Images listing cursor were keyed with a per-process random key, so every ComfyUI restart made an open tab's scopes answer 409 source-changed (blank Tag image preview, a Generate that did nothing) and its cursors fail Load more. - utils/workspace_keys.py: one random secret in workspace-scope.key beside the database (owner-only, never config.json or the database), two derived keys; per-process fallback with a warning. Move library copies it. - Cursor v2 binds the listing scope in its MAC instead of sending an unkeyed hash of the root paths; an unverifiable cursor answers 400 cursor-stale and Images offers Refresh instead of a dead Retry. - POST /prompt_manager/images/rescope maps a stale target to its current root only when the held scope was issued for it. - Images finds a renumbered file again from the inspector, the viewer and the Tag image dialog (the dialog stays open, draft kept); nothing else is kept by an old root_index:path key. - Tag image dialog: a state that blocks Save shows its reason above the actions with Refresh, disabled actions look disabled, and an empty preview says "Preview unavailable" (D147). |
||
|
Vito Sansevero
|
176558ea4d |
test(tools): the #247 repair never relinks a switched-off node's text (#90)
The plan decides every file through decide_file_prompt, so a not_saved or not_saved_wired file is a skip. Cover an off node behind a pass-through, an off node in the recorded saver's branch (literal and wired switch), an off node on another branch, and an off node wired only to the negative (still repaired). |
||
|
Vito Sansevero
|
aa799d9606 | Merge remote-tracking branch 'origin/main' into feat/247-foreign-workflows-pr3 | ||
|
Vito Sansevero
|
ab6001917c | Merge commit '264d9c3' into feat/247-foreign-workflows-pr3 | ||
|
Vito
|
56d0a4904c |
feat(collections): COLLECTIONS sidebar, collection page and saved views (#331, PR 2) (#410)
* feat(saved-views): keep a collection in a saved view Images saved views gain a collection key (#331 D140): empty or a collection id of 1 to 9 digits. Writes refuse favorites together with a collection; older rows read an empty collection, and a stored row with both opens the collection, as the listing does for a URL with both. Part of #331 * feat(collections): store collections in the library database * feat(collections): add, remove and rename collections and keep members through duplicate cleanup * feat(collections): add collection routes and the Images collection filter * feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * fix(collections): address review findings * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * feat(collections): browse a collection from the sidebar The sidebar gains the concept's COLLECTIONS group (D136): one row per collection with its folder icon, name and listed-member count, without the PROPOSED badge; the label shows once a collection exists. A row opens #/collection/<id>, the Images page filtered to that collection (D137): the collection's name as h1, Collection as breadcrumb and title, Edit to rename or delete it through the saved views' Edit dialog with collection nouns, and a note counting members the sources do not list now. An unknown or deleted collection shows a page that says so with a link to Images. A saved view on a deleted collection lists nothing and says so (MC6, D140). The list is read again after a restore and the counts 300 ms after a change. Part of #331 * test(collections): compare the COLLECTIONS group with the concept The concept hides COLLECTIONS at 950 px and below, which would leave a phone no way to open a collection; production shows the rows there in the wrapped layout of every other sidebar nav (D136). The shell comparison now compares the group instead of filtering it out: the concept fixture seeds Landscapes (12) and Experiments (8), the rows compare by name order, and at phone width the nav compares against the concept's own sidebar-nav layout. The collection rows are checked at 1600x1100 and 390x844 in both themes. Part of #331 * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(browser): install the slideshow clock before wrapping setTimeout The slideshow fixture wrapped window.setTimeout before page.clock.install(), so the viewer's timer ran on real time and clock.runFor() did not drive it; a late WebKit timer failed 'slideshow exposes exact state labels' on CI. * fix(collections): address review findings - A list read superseded by a newer one (the 300 ms count read, a restore) no longer shows "could not be loaded": the collection and saved-view pages wait for the list to settle. - A listing that finds the open collection gone shows the deleted page once the list no longer holds it; when the list read fails or still holds it, the page keeps the listing's error and Retry instead of staying blank, and each failed read is reported again. - The missing-members note follows every list read and goes away at zero. - A rename that settles after its Edit dialog closed leaves any dialog opened since alone. - The deleted-collection, list-failed and saved-view failure pages focus their title after a route change, as every page does. Part of #331 * test(coverage): keep main's floors for files this PR does not touch coverage:update raised nine floors of untouched files from one run; metadata-autocomplete.js branches then measured 84.84 against 85. * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. * test(duplicates): a removed copy with a collection and a category moves both (#331, #332) |
||
|
Vito
|
e86c0dcf05 |
fix(ci): write browser coverage without each script's source (#432) (#433)
Every raw browser coverage file repeated the full source of each mapped script, about 84% of its bytes, and the frontend coverage merge ran out of Node's heap at 1,884 files. c8 reads the source from the mapped file on disk, so the entries keep only the URL, script id and functions. |
||
|
Vito
|
39877202b2 |
feat(ingestion): one prompt decision for scan and monitor; scans link each dataset branch's own prompt (#247, PR 1) (#422)
* feat(inspector): use recorded run history when the graph alone can't name the prompt (#386) When a file's own graph leaves the prompt open, both inspect routes ask recorded run history about the file (one run reported it and it was written during that run). The save node that run reported it from anchors the walk like a file-name match, so dataset branches that share one saver prefix show their own prompt, labelled "Recorded run". Still open at one sampler whose PromptManager text was recorded at execution (a language model's output), the recorded text shows as "Execution recorded". Results the file resolves alone never change; a failed lookup judges the file alone. * fix(inspector): recorded texts follow history's saver; conflicts anchor nothing (#386) Review follow-ups: the texts PromptManager recorded are looked up at the sampler history's saver leads to, never at a sampler the unanchored walk named; a file name naming a different saver than history anchors nothing; outputs and texts are read in one transaction; "Recorded run" ranks below every workflow-inferred source; unexpected lookup errors log at error level. * fix(history): look up recorded runs by history's own spelling of the file (#386) The history route inspects a case-normalized target on Windows, while history matches file names exactly, so the occurrence's reference names the file. * refactor(inspector): share the inspector's resolve as inspection_resolution inspect_prompt_evidence now calls inspection_resolution, which returns the resolution to use, the anchored result and whether the recorded saver chose it. Ingestion can call the same helper for files the default profile cannot decide (#247). * feat(ingestion): one per-file prompt decision for the scan and the monitor utils/file_prompt.py holds decide_file_prompt, the ladder the output scan runs today plus the inspector's rule (inspection profile, then the recorded saver) before the legacy parser, with a SCAN and a MONITOR policy. Junk text (blank, numbers, LoRA tags only, model file names) is never stored. graph_prompt_text and parse_comfyui_prompt move here; the parser accepts parsed metadata (#247). * feat(ingestion): store a file's prompt in one transaction, one prompt per file database/scanned_prompts.py links a file to the prompt of one text, creating it when new, inside one BEGIN IMMEDIATE transaction: a file linked to another prompt is a conflict, a failed link leaves no prompt behind and two writers store one prompt. pair_ingestion gains file_link_rows, link_conflict and Marks/mark_created; ingest_resolved_pair takes optional marks for the components it creates. recorded_runs and named_prompt in run_attribution become public (#247). * feat(scan): decide each scanned file with the shared ladder (#247) _ingest_scanned_file calls decide_file_prompt (scan policy) and stores the result: pairs through ingest_resolved_pair and positives through store_scanned_prompt, both with the scan marks. Files the default profile leaves undecided now get the inspector's resolve, with the saver recorded history names, before the legacy parser, so each branch of a shared-prefix dataset run links its own prompt. Junk text is never stored, a file linked to another prompt is a conflict instead of a second link, and a failed link leaves no prompt behind. _BatchHistory reads the stored prompt and the recorded saver from one recorded_runs pass. _parse_comfyui_prompt calls the moved parse_comfyui_prompt. TESTING.md documents the decision. * fix(ingestion): keep the file-name saver anchor out of the ladder (#390) The #390 ruling queues a file that only its saver's file name resolves for Needs review, never linking it. inspection_resolution gains name_anchor; the ladder passes False, so such a file falls to the legacy floor in the scan (and to Needs review) and is never stored by the monitor, while the recorded saver still decides. The inspector keeps the name anchor. * fix(ingestion): no joined PromptManager text from the legacy floor; junk keeps its step's status The legacy parser returns prepend + text + append, so its answer is refused when a PromptManager-family node has a prepend or append part (#343). Junk found at the undecided-graph step keeps that step's unlinked status and review, so an ambiguous file is still queued for Needs review. The monitor policy's ambiguous file no longer asks for the recorded run, and the file size may be read lazily. * fix(ingestion): keep the legacy answer that is a PromptManager node's own text The joined-text refusal also refused the #170 parity files whose PromptManager prepend is a link the browser parser does not follow: its answer is then the node's literal text, which is what the node stores. Refuse only an answer that is not such a node's own text. * fix(scan): one prompt per file for recorded-run links; writes survive a library move The recorded run's stored prompt now links in one transaction with the file's link rows, so a file linked to another prompt is a conflict instead of a second link. The scan's stores run through survive_library_switch. A single file asks history once (a one-file _BatchHistory), the scan passes in_output=False and reads the file size only for files with prompt metadata. A conflict row reads "Kept: already linked to a different prompt". TESTING.md records the rules. * fix(ingestion): match a file's link rows under its resolved spelling too The scan stores realpath spellings and the monitor the watcher's raw path, so a file reached through a symlinked or junction output folder could get a second generated_images row. file_link_rows now queries the path and its realpath (each normalized and raw) and dedupes by row id. TESTING.md notes the reverse order is matched only when the raw spelling is the realpath. * fix(ingestion): a switched-off PromptManager node behind a pass-through stores nothing (#90) The #90 graph walk follows only prompt-like inputs, so a PromptManagerText with Save to library off reaching the encoder through another input name (easy showAnything.anything, a custom node's value) was invisible to it, and the inspector's resolve stored its text. The ladder now checks the positive every step names: a resolved or inspected positive whose path passes a node that is off, or a legacy answer that is such a node's own text, makes the file not_saved. * fix(ingestion): any switched-off node upstream of a positive stores nothing (#90) The ladder walks every input upstream of each sampler or guider positive (not negative, model, clip, latent or image inputs) for PromptManager nodes whose Save to library is off or wired from an unreadable node, so the sampler-less step, an off node with a prepend part and a PromptManagerPair behind a pass-through no longer store its text. The legacy-floor guard reads a node's resolved text. Every guard reports not_saved for a switch that is off and not_saved_wired for one it cannot read. * fix(ingestion): the switch walk skips negative inputs by #90's own rule The upstream walk treated only names containing "negative" as negative, so an off node on a neg, neg_text or text_neg input blocked an enabled positive. prompt_graph gains is_negative_input (the _NEGATIVE_KEY rule, also used by follows_input) and the walk uses it. The guards accept a node with null inputs and a side whose provenance is not an object. * fix(ingestion): the switch walk also skips any input named with "negative" #90's rule matches only whole underscore-separated words, so negative2, negativeprompt and NegativePrompt counted as positive inputs and an off node wired only to one of them blocked the file. The walk skips those too; follows_input is unchanged. |
||
|
Vito Sansevero
|
264d9c3791 |
fix(ingestion): the switch walk also skips any input named with "negative"
#90's rule matches only whole underscore-separated words, so negative2, negativeprompt and NegativePrompt counted as positive inputs and an off node wired only to one of them blocked the file. The walk skips those too; follows_input is unchanged. |
||
|
Vito Sansevero
|
dc0df5760f |
fix(ingestion): the switch walk skips negative inputs by #90's own rule
The upstream walk treated only names containing "negative" as negative, so an off node on a neg, neg_text or text_neg input blocked an enabled positive. prompt_graph gains is_negative_input (the _NEGATIVE_KEY rule, also used by follows_input) and the walk uses it. The guards accept a node with null inputs and a side whose provenance is not an object. |
||
|
Vito
|
41027aa1b1 |
fix(autotag): report a missing backend package instead of offering the model as ready (#426)
Model status now says, per backend, which Python packages cannot be imported (importlib.util.find_spec, nothing imported): llama-cpp-python for GGUF, torch/transformers/accelerate/bitsandbytes for HuggingFace, onnxruntime for WD14, with the pip install command and a note (the CUDA CMAKE_ARGS for llama-cpp-python, onnxruntime-gpu for WD14). Nothing is installed. load_model refuses such a model before the loader with MissingBackendPackages (one WARNING line, no traceback); a ModuleNotFoundError from a loader becomes the same refusal. The suggest route, the legacy single and batch routes and both tagging jobs answer 409 model-unavailable with that sentence. System > Auto-tagging reads "Downloaded. Needs llama-cpp-python: pip install llama-cpp-python. ..." instead of "Downloaded. Not loaded.", Review explains instead of reviewing, and the bulk tagging dialog keeps Start disabled with the sentence. "downloaded" keeps its files-only meaning, so the legacy pages are unchanged. Closes #424 |
||
|
Vito Sansevero
|
982464b7f2 |
fix(ingestion): any switched-off node upstream of a positive stores nothing (#90)
The ladder walks every input upstream of each sampler or guider positive (not negative, model, clip, latent or image inputs) for PromptManager nodes whose Save to library is off or wired from an unreadable node, so the sampler-less step, an off node with a prepend part and a PromptManagerPair behind a pass-through no longer store its text. The legacy-floor guard reads a node's resolved text. Every guard reports not_saved for a switch that is off and not_saved_wired for one it cannot read. |
||
|
Vito Sansevero
|
cf371094b7 |
fix(ingestion): a switched-off PromptManager node behind a pass-through stores nothing (#90)
The #90 graph walk follows only prompt-like inputs, so a PromptManagerText with Save to library off reaching the encoder through another input name (easy showAnything.anything, a custom node's value) was invisible to it, and the inspector's resolve stored its text. The ladder now checks the positive every step names: a resolved or inspected positive whose path passes a node that is off, or a legacy answer that is such a node's own text, makes the file not_saved. |
||
|
Vito
|
c7375573ed |
feat(images): set an image's own category, show its origin and count image categories (#332, PR 3) (#428)
* feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * feat(vocabulary): count and rename image categories (#332) Tags & categories lists every category value with image_count, the stored image_categories rows holding it. Values only images hold follow every prompt-backed value (first_pid at SQLite's INTEGER maximum), so keyset cursors cross the boundary. /library/options keeps its own query and never lists image-only values. Manage category counts image rows (impact.images), treats an image-held target as existing, renames image rows in the same transaction, allows renaming an image-only value, and hashes the sorted image rowids into the impact token so a change between preview and apply is refused with vocabulary-conflict. Every image-category statement reads idx_image_categories_category. * feat(images): the category filter matches each image's effective category (#332) An image's own category wins; without one it inherits its link's: the recipe's category when set, else the positive prompt's (MC4, one rule with the inspector chip). Ambiguous and unlinked images inherit nothing, so an unlinked image with its own category is now listed. The link read no longer filters by category in SQL; the page filter, select all N matching and saved views compare the effective value. A failed own-category read answers the existing listing-failed 500. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * feat(inspector): show an image's own category in the inspector and History (#332) The inspector's library facts gain category_scope "image": an image's own category overrides the inherited one for any link status (linked, ambiguous, none). Images reads it for output sources only; History reads it once per inspection inside its snapshot, keyed by the same file key from the symlink-resolved output folder. A failed read hides the chip rather than show a possibly wrong value. The chip's hidden prefix names the origin ("Image category: "); the visible origin text comes in PR 3. * perf(images): keep the category filter's link read narrow; record D141, D142, D145 (#332) Under a category the link read keeps only links whose raw recipe or positive category is the value (Python still decides the inherited one), the links of images whose own category is the value are read by path, and plain reads skip the inherited-category work. The own-category map is keyed by normalized path. TESTING.md records D141, D142 and D145 and amends D13, D36 and D39; the filter budget seeds 2,000 own categories. * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * fix(images): Windows path keys for own categories; plan check for the category read (#332) Review fixes: on Windows the own-category map keys each file with normalize_image_path (abspath drops trailing dots and spaces), and the plan test also covers the category's narrowing clause. * feat(images): Set category sets each image's own category; the chip names its origin (#332) The selection bar's Set category writes the selected images' own category (linked or not, up to 500, or all N matching up to 500), with the toolbar categories as suggestions; Use prompt category removes it after a confirmation that states the count. It never writes a prompt (#259's prompt action is retired from Images; the Library editor and the legacy admin Set Category still change prompt categories). The toolbar lists the listed images' own categories beside the prompt and recipe ones. The inspector chip shows its origin: "· this image", "· prompt" or "· recipe". Tags & categories counts images and opens Images for a category only images use. * test(images): compare the category chip's visible origin against the concept (#332) D146: the inspector comparison measures each chip's span.chip-source and lays the concept's own chip out with the recorded rule and origin added at runtime (withChipSource), in the side panel and the viewer's details. The D39 recipe case asserts the visible " · prompt"; the mutation check removes the origin. * test(images): compare Set category and the category options against the concept (#332) With one image's own category set through the real route, the toolbar's #category-filter options are compared with D141 mapped (the image category sorted in with the Library's, D36), and the Set category dialog against the concept's simpleInput laid out as recorded at runtime: D140 (empty field, datalist#workspace-input-suggestions of the toolbar's categories, Use prompt category between Cancel and Apply) and D8 (Apply). Unmapped, the dialog differs only in those parts. * test(history): compare the History media inspector's category chip against the concept (#332) D145: with an own category set on the baseline's newest output, the Recorded run viewer's chip is compared with the concept viewer's category chip laid out with the D146 origin and the own category at runtime; D21 maps its hidden "Image category: " prefix. Unmapped, the two differ only in the text, the prefix, the width and span.chip-source. * test(images): Set category sets each image's own category in page workflows (#332) Replace the #259 prompt-category tests with CUJ 1, 2, 3, 8 and 10, the suggestions datalist, the 500-image limit, the source-changed path, D142 and coded refusals; Images never posts /prompt_manager/bulk/category. * test(inspector): the category chip names its origin in the panel, viewer and History (#332) CUJ 4 and 5: "· this image", "· prompt" and "· recipe" chips, an unlinked file's own category, and History's current-file inspector. * test(vocabulary): category rename covers image categories; image-only rows browse Images (#332) CUJ 6 and 7 with own categories set through the API on two output files. * test(vocabulary): compare image-category rows and the Manage category preview against the concept (#332) The vocabulary fixture seeds own image categories on request (image_categories: Landscapes on two images, Portraits on one image only). mapVocabularyConcept maps D143 ("N positive prompts · M images", image-only rows before Manage category), a model checked against the concept page with those rows added at runtime, and mapVocabularyDialog takes the preview sentence, so the Manage category preview is compared with D9 and D144's images. The default fixture and mapping are unchanged. * test(vocabulary): park the pointer before measuring the category rows (#332) Firefox kept the pointer where the concept's Review was pressed, over the production Portraits row's Browse, so that button was measured hovered. * docs(testing): record D140, D143, D144 and D146 and the image category UI (#332) D140 (Set category writes images' own categories), D146 (the chip's visible origin) beside D141, D142 and D145; D143 and D144 in the vocabulary section. D8, D21 (the D20/D21 side panel) and D39 amended; the D8, the "all N matching" and the Native vocabulary UI paragraphs name the Prompt library editor and the legacy admin "Set Category" for prompt categories; the #332 section lists the interface and its comparisons. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * fix(images): focus after a changed source; Browse is a fresh Images visit (#332) Review fixes: a 409 source-changed reload moves focus to the heading as a success does; a selection the listing no longer holds says none could be found instead of sending nothing; a Tags & categories Browse opens a plain Images visit filtered by the category (remembered and with the Appearance defaults) rather than a saved-view visit. * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. * refactor(images): take the selection bar's categories from its one caller (#332) The getCategories default was never used (images.mjs always passes the toolbar's categories) and left images-selection.mjs function coverage at 97.05 (floor 98). |
||
|
Vito Sansevero
|
02ce5e79bc |
Merge remote-tracking branch 'origin/main' into feat/247-foreign-workflows
# Conflicts: # py/api/admin.py |
||
|
Vito
|
4f37af209c |
feat(collections): store collections and add their routes and Images filter (#331, PR 1) (#400)
* feat(collections): store collections in the library database * feat(collections): add, remove and rename collections and keep members through duplicate cleanup * feat(collections): add collection routes and the Images collection filter * feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * fix(collections): address review findings * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(browser): install the slideshow clock before wrapping setTimeout The slideshow fixture wrapped window.setTimeout before page.clock.install(), so the viewer's timer ran on real time and clock.runFor() did not drive it; a late WebKit timer failed 'slideshow exposes exact state labels' on CI. * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. * test(duplicates): a removed copy with a collection and a category moves both (#331, #332) |
||
|
Vito
|
3ebd5e4166 |
ci(browser): a hung browser step fails with its running tests named and its traces kept (#425) (#429)
* ci(browser): stop a hung browser run before the job limit and name it A browser job killed at its 20-minute limit keeps no log and skips its upload steps (merge-queue run 40723857746, firefox shard 2/2), so nothing showed which test hung. - Each browser test step gets timeout-minutes 18, below the job's 20, so the step fails and the always() uploads still run. - Under CI, Playwright's globalTimeout stops the run first: PM_TEST_GLOBAL_TIMEOUT_MINUTES 17 for page workflows, 15 for coverage workflows, a fixed 3 for the viewer suite. Locally there is no limit unless the variable is set. Each test keeps the 30 s default. - running-tests-reporter.mjs prints the tests still running when the run ends; Playwright's own reporters cannot, since those tests never end. - A blob report goes to tools/vendor/test-results/blob/, inside the uploaded traces path. - ci-workflow.test.mjs pins globalTimeout < step limit < job limit and the always() uploads; no existing limit is raised. Closes #425 * ci(browser): rely on the step limit, report the running tests on SIGTERM Maintainer ruling on #425: no Playwright globalTimeout. The 18-minute step limit under the unchanged 20-minute job limit stays, as do the always() uploads and the CI blob report. - Drop globalTimeout and PM_TEST_GLOBAL_TIMEOUT_MINUTES with its helper and tests. - A step that reaches its limit gets SIGTERM, which ended Playwright with no report and no blob. The running-tests reporter now catches the first SIGTERM, prints the tests still running and sends Playwright its own SIGINT, so the run ends interrupted and the blob report is written. A second SIGTERM kills as usual. - Test paths are relative to the suite's rootDir instead of the cwd, so `npm run --prefix` no longer prints ../../ prefixes. - The workflow comment and TESTING.md say the guarantee assumes an npm cache hit and that coverage shard 1 is tight in its 18-minute step. * ci(browser): forward SIGTERM and SIGINT from frontend-coverage.mjs to its child In the coverage job npm signals only frontend-coverage.mjs. It blocked in spawnSync with no listener, so Playwright never saw a step timeout. It now runs each step's child with async spawn, in its own process group, and forwards SIGTERM and SIGINT to the running child. Step order, exit codes and the "Command failed (N)" message are unchanged. Locally, in the pinned image, `timeout -s TERM` on test:coverage printed the still-running tests and wrote both blob reports. TESTING.md records that a SIGTERM to npm alone stops at npm's `sh -c`, and that delivery through `docker exec` in CI is not yet observed. Refs #425 * ci(browser): load the running-tests reporter by absolute path A config imported from another directory (the workflow-artifacts test copies it to a temp root) resolved './running-tests-reporter.mjs' against that root under CI and failed to load it. Resolve it from this module instead. |
||
|
Vito
|
d9f168c317 |
feat(nodes): per-node Save to library switch (#90) (#427)
* feat(nodes): per-node Save to library switch (#90) Every PromptManager node (Classic, Text, Focused, Split, Pair) gets an optional save_to_library BOOLEAN, default on and declared last. Off, the node still returns its normal outputs but PromptManager ignores the run: - the node saves, counts, tracks and records nothing; - the queue hook reads the switch from the queued graph (literal, or a linked PrimitiveBoolean), so a cached node is not counted either; a switch wired from an unreadable node is left to the node itself; - an opted-out node is no run prompt, so no tracker entry, run key, embedded-metadata match or history attribution links its images, and a run with any opted-out positive node links none of its files; - the scan reports such files as not_saved (never stored or queued for Needs review), and a file whose only opted-out node is its negative is stored positive-only, without the pair; - execution history records no prompt text for an opted-out role. PromptSearchList only reads the library and gets no switch. The switch takes the widget index older workflows used for the editor's own DOM slot (''), so a configure step turns any non-boolean value on: workflows saved before the switch load with it on and no earlier widget value moves. The Focused, Split and Pair editors draw it as a "Save to library" checkbox in their toolbar; Classic and Text show the native toggle. * fix(nodes): Save to library review fixes (#90) - A switch wired from a node the graph cannot read and off at run time marks the node's run key, so an earlier run's tracker entry never links this run's images. - The scan reports such files as not_saved_wired ("only the run itself can link it") instead of not_saved. - Workflow-only metadata ignores muted and bypassed PromptManager nodes and reads nodes inside subgraph definitions. - The Focused, Split and Pair footer says when the switch comes from a link. * fix(usage): count cached re-runs of runtime-linked prompt text (#347) A PromptManager-family node whose text comes from a runtime link (PreviewAny, an LLM, a list node) counted only the job that executed it; later jobs served from ComfyUI's output cache counted nothing. Each execution now records a receipt (digests of its STRING outputs and hashes of the text it stores) per run key and queue item. On execution_cached the receipt whose digests equal the cache entry's text is replayed once for the job, with the node's role and what the queue hook already saw recomputed from the job's own graph. The executing node and the replay count through one shared own_counts. Unmatched cache entries count nothing, as before; a node that executes after its replay in the same job skips its own count. No schema change, no backfill. * fix(usage): never replay a node twice in one job; dedupe receipts once complete (#347) A repeated execution_cached id or message for the same job now counts the node once. Receipts replace equal older ones only when their job finishes (or the next job starts), so a list still recording [A] of [A, B] no longer drops an older [A] receipt. Node-level tests check the receipt each execution records. * test(coverage): baseline for the Save to library module (#90) * test(usage): disabled node then cached jobs counts 0 (#90, #347) A PromptManager node with Save to library off records no cached-run receipt and is no run prompt in the replayed job's graph, so ComfyUI serving it from its cache on later jobs counts nothing. The prompt is saved first by an enabled run, so a replayed count would show. Covers PromptManager, Focused, Split, Text and the Pair. * fix(nodes): hide the editor's native widgets through one helper (#90) The Save to library switch copied the text fallbacks' zero-height computeSize arrow, and neither copy was ever called, so focused-node.js function coverage fell to 87.5 (floor 90). One hideNativeWidget helper now hides both, and the spec checks that the hidden switch and a text fallback take no node height. |
||
|
Vito
|
cdb771d722 |
feat(images): filter, inspect and count by each image's effective category (#332, PR 2) (#406)
* feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * feat(vocabulary): count and rename image categories (#332) Tags & categories lists every category value with image_count, the stored image_categories rows holding it. Values only images hold follow every prompt-backed value (first_pid at SQLite's INTEGER maximum), so keyset cursors cross the boundary. /library/options keeps its own query and never lists image-only values. Manage category counts image rows (impact.images), treats an image-held target as existing, renames image rows in the same transaction, allows renaming an image-only value, and hashes the sorted image rowids into the impact token so a change between preview and apply is refused with vocabulary-conflict. Every image-category statement reads idx_image_categories_category. * feat(images): the category filter matches each image's effective category (#332) An image's own category wins; without one it inherits its link's: the recipe's category when set, else the positive prompt's (MC4, one rule with the inspector chip). Ambiguous and unlinked images inherit nothing, so an unlinked image with its own category is now listed. The link read no longer filters by category in SQL; the page filter, select all N matching and saved views compare the effective value. A failed own-category read answers the existing listing-failed 500. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * feat(inspector): show an image's own category in the inspector and History (#332) The inspector's library facts gain category_scope "image": an image's own category overrides the inherited one for any link status (linked, ambiguous, none). Images reads it for output sources only; History reads it once per inspection inside its snapshot, keyed by the same file key from the symlink-resolved output folder. A failed read hides the chip rather than show a possibly wrong value. The chip's hidden prefix names the origin ("Image category: "); the visible origin text comes in PR 3. * perf(images): keep the category filter's link read narrow; record D141, D142, D145 (#332) Under a category the link read keeps only links whose raw recipe or positive category is the value (Python still decides the inherited one), the links of images whose own category is the value are read by path, and plain reads skip the inherited-category work. The own-category map is keyed by normalized path. TESTING.md records D141, D142 and D145 and amends D13, D36 and D39; the filter budget seeds 2,000 own categories. * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * fix(images): Windows path keys for own categories; plan check for the category read (#332) Review fixes: on Windows the own-category map keys each file with normalize_image_path (abspath drops trailing dots and spaces), and the plan test also covers the category's narrowing clause. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. |
||
|
Vito
|
55fa2c4870 |
fix(runs): count cached PromptManager nodes fed by a runtime text link on every job (#347) (#423)
* fix(usage): count cached re-runs of runtime-linked prompt text (#347) A PromptManager-family node whose text comes from a runtime link (PreviewAny, an LLM, a list node) counted only the job that executed it; later jobs served from ComfyUI's output cache counted nothing. Each execution now records a receipt (digests of its STRING outputs and hashes of the text it stores) per run key and queue item. On execution_cached the receipt whose digests equal the cache entry's text is replayed once for the job, with the node's role and what the queue hook already saw recomputed from the job's own graph. The executing node and the replay count through one shared own_counts. Unmatched cache entries count nothing, as before; a node that executes after its replay in the same job skips its own count. No schema change, no backfill. * fix(usage): never replay a node twice in one job; dedupe receipts once complete (#347) A repeated execution_cached id or message for the same job now counts the node once. Receipts replace equal older ones only when their job finishes (or the next job starts), so a list still recording [A] of [A, B] no longer drops an older [A] receipt. Node-level tests check the receipt each execution records. * test(comfyui): count #347's cached-run cases in the execution suite pin (#347) The pin listed 19 execution-text-link cases per project; #347 adds five node kinds and eight one-renderer cases, so the suite lists 32 per project and 118 in total with #332's image-attribution case from main. |
||
|
Vito
|
11422d4584 |
perf(images): read prompt links in one SQLite step (#354) (#418)
* perf(images): read prompt links in one SQLite step (#354) get_image_link_matches streamed one sqlite3_step per linked row, and Python's sqlite3 releases and retakes the GIL around every step. While ComfyUI generates, each retake waits on the busy thread, so the first-page link read of the workspace Images page stretched from tens of milliseconds to seconds. SQLite now returns the unchanged filter SELECT's rows as one JSON value (json_group_array), decoded with json.loads: a constant two or three GIL hand-backs instead of one per row. Nothing is cached. Any OperationalError from the wrapped statement, or a BLOB file_path or rating, makes that read stream as before with one warning per process; a missing JSON function stops the attempt for the rest of the process. tests/test_database_link_matches.py proves the result equals the streamed read on every filter combination and on NULL ratings, quoted, backslashed, non-ASCII and multi-line paths, ambiguous and pair links, an empty database and both fallbacks; the structural budget test also checks the wrapped query plan. * fix(images): keep the maintenance link reader, refine the one-step fallback (#354) The one-step reader was named _link_rows, which shadowed MaintenanceOperations._link_rows on PromptDatabase and broke the Missing links preview and apply; it is now _one_step_link_rows, and a test pins that the maintenance reader is not shadowed. An interrupted read (Move library retiring the connection) is raised as before instead of falling back and spending the one warning; a JSON text over SQLite's length limit (DataError) streams like the other value failures. The equivalence test compares value types per path rather than by row order, and a test checks that the read leaves no WAL snapshot pinned. * docs(testing): name every one-step link read fallback (#354) |
||
|
Vito Sansevero
|
81e1c685f0 |
Merge remote-tracking branch 'origin/main' into feat/247-foreign-workflows
# Conflicts: # database/run_attribution.py # tests/test_recorded_saver_anchor.py # tests/test_run_attribution.py # utils/prompt_inspector.py |
||
|
Vito
|
b5a56bee0b |
feat(images): store images' own categories and set them by route (#332, PR 1) (#401)
* feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. |
||
|
Vito Sansevero
|
f6e9de6483 | Merge commit '3dfb828' into feat/247-foreign-workflows-pr3 | ||
|
Vito Sansevero
|
3dfb828ae5 |
fix(ingestion): match a file's link rows under its resolved spelling too
The scan stores realpath spellings and the monitor the watcher's raw path, so a file reached through a symlinked or junction output folder could get a second generated_images row. file_link_rows now queries the path and its realpath (each normalized and raw) and dedupes by row id. TESTING.md notes the reverse order is matched only when the raw spelling is the realpath. |
||
|
Vito Sansevero
|
1a90d3b0ae |
fix(tools): harden the #247 repair apply after review
- refuse while any other process holds the library, its -wal or -shm open (tools/comfyui_stopped.py), also in a rehearsal; without /proc only with --i-stopped-comfyui; require main.py in --comfyui-dir; check again just before BEGIN IMMEDIATE - the plan records its library (normcase realpath) and max row id; apply refuses on another library - refuse when opening the library changes its schema (run from a checkout at the live code version) - verify the backup (integrity_check and counts) and never overwrite one - target_text is a drift key and must match its hash; keep compares the linked text's hash, not the stored column - the dry run rehearses the orphan deletes when --delete-orphans is chosen - a .md plan path is refused; the list is <stem>.list.md - any error during apply rolls back, names the backup, exits 2 - relpath on another drive falls back to the full path |
||
|
Vito Sansevero
|
b7cf8ef221 | fix(tools): the repair list keeps whole file paths and shows each target hash | ||
|
Vito Sansevero
|
6b568de09b |
feat(tools): reviewed repair of the foreign dataset links (#247)
tools/repair_foreign_links.py judges every link row of a file that only foreign runs reported with the shared ladder (scan policy, recorded saver). The dry run judges a copy, writes a JSON plan and a Markdown list, and rehearses the apply on that copy to audit the prompts that would lose every link (deleting them is off unless chosen). --apply refuses while ComfyUI runs, takes a backup, re-judges every row inside one BEGIN IMMEDIATE transaction, creates the targets with the scan's marks, relinks and drops pair links, keeps UNIQUE(prompt_id, file_path), and rolls back with exit 2 on any drift. |
||
|
Vito Sansevero
|
b107de70f8 |
fix(scan): one prompt per file for recorded-run links; writes survive a library move
The recorded run's stored prompt now links in one transaction with the file's link rows, so a file linked to another prompt is a conflict instead of a second link. The scan's stores run through survive_library_switch. A single file asks history once (a one-file _BatchHistory), the scan passes in_output=False and reads the file size only for files with prompt metadata. A conflict row reads "Kept: already linked to a different prompt". TESTING.md records the rules. |
||
|
Vito Sansevero
|
a6d634509b |
fix(ingestion): keep the legacy answer that is a PromptManager node's own text
The joined-text refusal also refused the #170 parity files whose PromptManager prepend is a link the browser parser does not follow: its answer is then the node's literal text, which is what the node stores. Refuse only an answer that is not such a node's own text. |
||
|
Vito Sansevero
|
e137270f45 |
fix(ingestion): no joined PromptManager text from the legacy floor; junk keeps its step's status
The legacy parser returns prepend + text + append, so its answer is refused when a PromptManager-family node has a prepend or append part (#343). Junk found at the undecided-graph step keeps that step's unlinked status and review, so an ambiguous file is still queued for Needs review. The monitor policy's ambiguous file no longer asks for the recorded run, and the file size may be read lazily. |
||
|
Vito
|
95586e6395 |
refactor(gallery): open System for gallery settings, maintenance and tagging models (#330, PR 3) (#408)
* test(legacy): snapshot the routes and the settings response Pins every registered route and the key paths of GET /prompt_manager/settings while System S13 (#330) moves the legacy pages' tools to native System. Recorded on the unchanged main; cutover (#334) removes it. * test(legacy): prove each removed legacy control has native coverage TESTING.md gains the "Legacy System UI removed by S13" table (no rows yet). legacy-coverage.test.mjs fails when a removal cites a capability that is not covered, copies another native test, lists an id or method twice, or leaves the id or method on its page. legacy-system-links.spec.mjs holds the System link table and guards the kept admin flows. * refactor(admin): open System for Diagnostics and logs The admin page's Diagnostics and View Logs buttons become links that open System › Diagnostics & logs in a new tab. The legacy Diagnostics and System Logs dialogs and their methods go; TESTING.md lists them against their covered capabilities (rows B1–B7). * refactor(admin): open System for settings and keep page options Settings opens System › Overview in a new tab. The settings dialog becomes Page options (opened from the list header) with only the Infinite Scrolling default, and its Save posts {"infinite_scroll": …} alone, so a stale legacy form can no longer overwrite scan folders or other values edited in System (#330 S13-9). Scan folders, monitoring status, auto-hide, display mode, worker threads, LoraManager and the LoRA import leave the page (rows B8–B14). * refactor(admin): open System for maintenance Maintenance opens System › Maintenance in a new tab, where each operation runs as a reviewed Activity job. The one-click legacy Database Maintenance dialog and its methods go (rows B15–B18); formatFileSize stays for the image modal. * refactor(admin): open System for backup and restore Backup DB and Restore DB open System › Backup & export in a new tab, where a restore is staged, reviewed and run as an Activity job. The one-click Restore dialog and its methods go (rows B19–B20); the scan dialog's quick backup stays. The removed browser tests' data checks live on at the route: a new round trip test restores every prompt with its tags and image links. * test(legacy): find a removed modal id passed to showModal and settle the links spec The removal check also finds an id passed as a string to a ...Modal() call and no longer reads data-id= as id=. The no-request test waits for the page's own start-up reads to finish, the sort check compares as the server sorts, and the kept-flow test checks the added prompt shows in the list. * test(admin): open Page options after the saved default is read Drops the leftover LoraManager section comment above the Page options save. * refactor(gallery): open System for settings and maintenance The gallery's Settings opens System › Appearance in a new tab. Its settings dialog becomes Page options (a new header button) with only the page's own settings: thumbnail startup check, lazy loading, items per page, grid columns, infinite scrolling, debug mode and API timeout. Save and Reset keep every other stored key, so the Appearance values System saves survive (S13-13). Generate Thumbnails, Clear cache, Rescan and Find duplicates leave the page; Page options links System › Maintenance and Sources & scans (rows B21–B28). * refactor(autotag): manage tagging models in System In both auto-tag dialogs a model that is not downloaded shows a Manage in System link to System › Auto-tagging (a new tab) instead of a Download button, and the gallery's Unload Model becomes the same link. The legacy download dialog and the download, cancel and unload methods go (rows B29–B31); the batch and review stay. A test hook marks a fake model loaded. * docs(system): name System as the home for settings and maintenance README and TESTING.md send settings, maintenance, backups, diagnostics, the gallery's tools and auto-tag model downloads to their System section and name each legacy page's Page options. * fix(gallery): keep the default view in Page options and save viewer choices alone System › Appearance's Default layout is the workspace's own key, so the legacy gallery's Default View Mode stays a page option instead of losing its only control. The viewer's mute and loop buttons remember just that choice instead of saving the Page options dialog's unfilled inputs over the stored page options. The pending-thumbnails toast names System › Sources & scans. * test(legacy): check each System link sends no request in its own test The no-request check runs inside each link's test, so a slow page no longer times out the whole table, and it ignores lazy thumbnail loads the layout starts on its own. * test(legacy): a lazy thumbnail load is not a System link's request * fix(gallery): let the header buttons wrap on a narrow screen With Page options beside Settings the header no longer fits a phone's width; it wrapped off the page and covered the Page options dialog in the touch runs. * feat(gallery): remove Default View Mode from Page options (#330) System > Appearance's Default layout is the one default-layout control, as the S13 design's G5 says. The legacy gallery's Page options no longer show or save a default view and Reset no longer resets it. A stored legacy defaultViewMode still picks the view the gallery opens in (grid otherwise), and Save and Reset keep that key for the workspace-preferences migration. |
||
|
Vito
|
5583159497 |
feat(inspector): use recorded run history when the graph alone can't name the prompt (#386) (#396)
* feat(inspector): use recorded run history when the graph alone can't name the prompt (#386) When a file's own graph leaves the prompt open, both inspect routes ask recorded run history about the file (one run reported it and it was written during that run). The save node that run reported it from anchors the walk like a file-name match, so dataset branches that share one saver prefix show their own prompt, labelled "Recorded run". Still open at one sampler whose PromptManager text was recorded at execution (a language model's output), the recorded text shows as "Execution recorded". Results the file resolves alone never change; a failed lookup judges the file alone. * fix(inspector): recorded texts follow history's saver; conflicts anchor nothing (#386) Review follow-ups: the texts PromptManager recorded are looked up at the sampler history's saver leads to, never at a sampler the unanchored walk named; a file name naming a different saver than history anchors nothing; outputs and texts are read in one transaction; "Recorded run" ranks below every workflow-inferred source; unexpected lookup errors log at error level. * fix(history): look up recorded runs by history's own spelling of the file (#386) The history route inspects a case-normalized target on Windows, while history matches file names exactly, so the occurrence's reference names the file. |
||
|
Vito Sansevero
|
4b4fa7f8ff |
fix(ingestion): keep the file-name saver anchor out of the ladder (#390)
The #390 ruling queues a file that only its saver's file name resolves for Needs review, never linking it. inspection_resolution gains name_anchor; the ladder passes False, so such a file falls to the legacy floor in the scan (and to Needs review) and is never stored by the monitor, while the recorded saver still decides. The inspector keeps the name anchor. |
||
|
Vito Sansevero
|
4a90a5dd83 |
feat(scan): decide each scanned file with the shared ladder (#247)
_ingest_scanned_file calls decide_file_prompt (scan policy) and stores the result: pairs through ingest_resolved_pair and positives through store_scanned_prompt, both with the scan marks. Files the default profile leaves undecided now get the inspector's resolve, with the saver recorded history names, before the legacy parser, so each branch of a shared-prefix dataset run links its own prompt. Junk text is never stored, a file linked to another prompt is a conflict instead of a second link, and a failed link leaves no prompt behind. _BatchHistory reads the stored prompt and the recorded saver from one recorded_runs pass. _parse_comfyui_prompt calls the moved parse_comfyui_prompt. TESTING.md documents the decision. |
||
|
Vito Sansevero
|
e3191853c8 |
feat(ingestion): store a file's prompt in one transaction, one prompt per file
database/scanned_prompts.py links a file to the prompt of one text, creating it when new, inside one BEGIN IMMEDIATE transaction: a file linked to another prompt is a conflict, a failed link leaves no prompt behind and two writers store one prompt. pair_ingestion gains file_link_rows, link_conflict and Marks/mark_created; ingest_resolved_pair takes optional marks for the components it creates. recorded_runs and named_prompt in run_attribution become public (#247). |
||
|
Vito Sansevero
|
9bf8362d22 |
feat(ingestion): one per-file prompt decision for the scan and the monitor
utils/file_prompt.py holds decide_file_prompt, the ladder the output scan runs today plus the inspector's rule (inspection profile, then the recorded saver) before the legacy parser, with a SCAN and a MONITOR policy. Junk text (blank, numbers, LoRA tags only, model file names) is never stored. graph_prompt_text and parse_comfyui_prompt move here; the parser accepts parsed metadata (#247). |