a20e59f6744f9b5ec96fe8cd0b2a84dfcee20fcd
20
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vito Sansevero
|
a20e59f674 |
feat(tagging): skipped tags hide matching suggestions (#431)
A Skipped tags list in System > Auto-tagging (D202) hides every suggested tag whose words contain an entry's words, on every suggestion path (Tag image, Auto-tag selected, the System batch, the legacy single and batch routes), before the limit. Jobs snapshot the list at their start. A read-only preview route counts what a list hides among the library's tags and the downloaded WD14 label lists. The draft store compares list and object drafts by value (sameDraft). |
||
|
Vito Sansevero
|
f53a43152d |
feat(review): link an image to a library prompt from Needs review (#536) (#540)
Closes #536 |
||
|
Vito
|
9af3271dbb |
feat(images): Show removed images and Return to library (#498) (#514)
* fix(images): background jobs skip files hidden by the removal mark; walkers skip trash folders inside output roots (#501) - Thumbnail jobs and the legacy scan_output_dir listing leave library-removed files out (one mark check, mark_hides). - Missing-file links also forgets removal marks that hide nothing any more (file gone from a folder that exists, or changed), only under a readable, non-empty source, re-checked at apply. - The sidebar Images, Favorites and Needs review counts and the collection counts are read again when an image_delete job ends. - utils/media_folders.py: one skip for thumbnails and trash folders (.Trash, .Trash-<uid>, .Trashes, $RECYCLE.BIN) used by every walker, the image monitor and the delete review. * feat(images): Show removed images and Return to library (#498) - GET /images/output?mode=workspace&removed=1 lists only files a removal mark hides; every workspace page carries removed_total. - POST /images/removed/return {items | filter} starts a scan job, "Return images to the library": the marks that still hide their file go in one transaction, then exactly those files are scanned, so their prompt links come back from their metadata. Tags, categories, collections and favorites do not come back. - The marks generation moves again after the commit, so a reader that cached the marks mid-transaction reloads them. - Images: Show removed images (N) heading toggle (D162), Removed chips, a bar with only Return to library and Clear, Return to library in the side panel; the view is never saved or restored. * test(images): record the return route in the legacy contract snapshot; removed_total in the workspace keys (#498) The scan runner passes returned only when set, so a stand-in events function without that keyword still runs. * test(browser): wait for the answers two specs raced in the merge queue (#501) - System diagnostics "level filter": wait for each level's own log answer, and poll the shown levels; the CRITICAL line is in the previous level's answer too, so it proved nothing (firefox). - Library editor "restored storage after a confirmed save": wait for the PATCH answer, which waits for a whole restore in the test's route, instead of the 5 s expect default (webkit). * fix(monitor): skip trash folders only below the watched folder (#501 review) A source named .Trash-1000 (or another trash name) was ignored by the image monitor while every walker still listed it; the monitor now tests the folders below the longest watched folder holding the new file. * fix(images): no default removedView function in the selection bar (#498) The default arrow was never called, so images-selection.mjs fell under its function coverage floor (97.29 < 98). * fix(images): a category or tags chosen in Show removed images ends the view (#498 review) The toolbar's Category and Refine's Tags stayed usable in the removed view, and the listing then refused the combination with a page error. Choosing one now leaves the view, as a removed file has neither. |
||
|
Vito
|
d02a911748 |
feat(library): delete or archive selected prompts, Show archived prompts and Restore (#443) (#470)
* feat(library): set the category on many selected prompts (#437) The Prompt library shelf gets the concept's Images selection checkbox on each card and its .bulkbar (N selected, Set category, Clear), recorded as D148. Set category opens the shared one-value dialog and posts {prompt_ids, category, expected_epoch} to POST /prompt_manager/bulk/category: the server checks the storage epoch the shelf was read under, then sets the category on the Library parents in one BEGIN IMMEDIATE transaction, writing only rows that change. A restored storage answers 409 storage-changed and writes nothing. The legacy admin and images_filter bodies are unchanged. This replaces the legacy admin page's bulk Set Category, a cutover prerequisite (#334). Closes #437 * feat(library): add tags to many selected prompts and select every loaded prompt (#442) The Library selection bar (D148) gets Add tags and select-all of the loaded shelf, the legacy admin page's bulk Add Tags and select-all checkbox. Add tags posts {prompt_ids, tags, expected_epoch} to POST /prompt_manager/bulk/tags: in one BEGIN IMMEDIATE transaction the server checks the storage epoch, normalizes the names as image tags are, gives each Library parent only the names it lacks (casefold), finds or inserts tag rows by exact name only when some prompt gets them, and skips prompts that would pass 20 tags. Prompt tags only. Both bulk/tags forms refuse a non-object body with 400. Ctrl/Cmd+A in the grid and "Select all N loaded" in the bar select every loaded prompt; the offer follows each loaded batch. Closes #442 * feat(vocabulary): tag search, tag sort and an Untagged prompts entry (#444) Tags & categories gains a server-side tag search and sort (keyset pages, cursor scoped to the query) and an "Untagged prompts" row whose Browse opens the Library with tag_mode=untagged. Refine this view gets Tag match, saved Library views keep it, and the routes echo the applied query so a stale server is an error, never an unfiltered list. Recorded as D149. Refs #444 * feat(library): filter prompts by output folder and browse tags with Any (#444) Refine this view on the Prompt library gains the concept's Folder (folders below an output root, read when the dialog opens) and a Tag match select (All / Any). The listing takes folder and tag_mode, matches folders on the normalized file_path under every spelling of each output root, echoes both so a stale server is an error, and saved Library views keep them. Recorded as D150. Refs #444 * docs(library): record Add tags and select-all as D151 (#442) * feat(library): archive paired and delete unpaired selected prompts on the server (#443) A prompt_archive table marks Library parents hidden from the shelf; older builds ignore it. New routes /prompt_manager/library/delete/preview, /library/delete and /library/restore classify a selection (delete, archive, protected, already archived, not found), execute exactly what was reviewed under a records hold, take a verified safety backup (disk checked) before any delete, and restore archived prompts. The shelf, search and the shelf-facing counts honor the archive (?shelf=active|archived); archiving repairs a stale prompt hash so scans find the archived row. * feat(library): Delete… on the selection bar, Show archived prompts and Restore (#443) The Library bar's Delete… previews the selection and opens the shared review: what is deleted (in no saved pair), archived (paired: hidden, restorable) or kept (tagged __protected__), the safety backup and what cannot be undone. Show archived prompts (N) lists archived prompts with Restore on their bar; an archived prompt's page shows an Archived notice with Restore. TESTING.md records D152; the route snapshot gains the three workspace routes. * test(library): cover the Delete… review failure and close-only paths; record coverage floors (#443) * fix(library): let saved-view defaults fill the missing #444 filters libraryViewState passed `?? 'all'` and `?? ''` fallbacks that normalizeViewState already applies to an undefined value. The redundant branches were never taken in the page, which dropped saved-views.mjs below its 95% branch floor. A unit now covers a shelf without the new filters. Refs #444 * fix(library): lay out a loaded page of prompts without waiting for an animation frame (#443) A starved renderer (headless WebKit under CPU load, a background tab) can hold requestAnimationFrame for seconds. The shelf laid out each loaded page only in its next frame, so the grid stayed empty after the first load or a delete's reload while the caption already showed the new count. A loaded batch is now laid out at once; measurements still flush per frame. The regression test holds every animation frame and fails before the fix. * test(saved-views): give the D153 Library view the tag_mode key (#444) The Library saved-view state now holds tag_mode, and the API requires every key, so main's new "Highest rated" spec posts it too. Refs #444 * test(saved-views): give the D153 Library view the folder key (#444) Refs #444 * fix(library): name the saved pair, not negative variants, as the archive reason (#443) A prompt in a saved pair with no negative variants was described in the Delete… review as "1 prompt with negative variants: hidden, restorable". Paired prompts are archived because of the pair, so the Archive row now reads "N prompts in a saved pair: hidden, restorable" and the first sentence explains the pair. Found in the live test of #470. * fix(library): nest Refine folders by path segment; backslash separates only on Windows (#444) Live test B1: the Folder list indented a server list sorted as full-path text, so "-" and "_" sorted before "/" and children landed under the wrong parent (kiko/char under kiko-v6), with case variants interleaved. The options are now a tree built by segment: each folder directly under its own parent, case variants distinct siblings. On POSIX a folder named "Metadata\Klein..." was listed but filtered to no prompts, because normalize_folder read "\" as a separator. It now does so only on Windows (ntpath), as the listing already did. Refs #444 * fix(library): say why an archived prompt is archived, and name variants only when it has any (#443) The archived prompt page's notice always said "This prompt and its negative variants are hidden", also for a saved-pair prompt with none. It now says the prompt is archived because it is in a saved pair, and counts its negative variants only when there are any. From the live test of #470. |
||
|
Vito
|
66ca988d2a |
feat(images): delete selected images, from the library or to the trash (#480) (#488)
* feat(trash): a freedesktop trash backend that renames without replacing or refuses (#480) * feat(images): remove a file's records and hide it with a removal mark (#480) Deletes every record of one file under every recorded spelling (image links, tags, auto-tag marks, pending review items, category, collection memberships, favorite) in one transaction. A Needs review item that is kept keeps its annotation rows (ON DELETE RESTRICT). The image_removals table hides an exact file (locator, size, mtime_ns) until it changes; linking skips a marked file, and an explicit tag, category, favorite or collection write shows it again. * feat(images): review and apply Delete images as a reviewed plan job (#480) POST /prompt_manager/images/delete/preview reviews both modes; the maintenance apply takes {token, mode}. Library mode removes the records and writes the removal mark; trash mode moves the file first and removes its records only after a successful move. The workspace listing, the output scan, the duplicate find and Tag output images skip marked files. * feat(images): Delete… in the Images selection bar, inspector and viewer (#480) The concept's Remove links… place, D159: one review per selection that asks each time (nothing preselected) whether to remove the files from the library only or also move them to the trash. The browser test server uses a fake trash. * test(images): portable delete tests, the native no-trash proof, D159 in TESTING.md (#480) * perf(images): cache the removal marks so a listing page reads no extra table (#480) The marks reload only when this process writes or clears one, or a restore replaces the database. The route snapshot gains the preview route. * test(images): the full inspector specs expect D159's Delete… in the panel and the viewer (#480) * fix(images): the delete review starts nothing without a chosen mode, and dims its primary until then (#480) An aria-disabled primary ignores every activation, a chosen one starts once, and the apply helper refuses a missing mode. Tests cover a forced click, a double click, Enter and Space on the primary, Enter in the dialog and a submit event, and the server refusing an apply without an exact mode. |
||
|
Vito
|
8870496ea3 |
feat(monitor): store certain prompts from workflows without a PromptManager node, with a System switch (#247, PR 2) (#467)
* feat(inspector): use recorded run history when the graph alone can't name the prompt (#386) When a file's own graph leaves the prompt open, both inspect routes ask recorded run history about the file (one run reported it and it was written during that run). The save node that run reported it from anchors the walk like a file-name match, so dataset branches that share one saver prefix show their own prompt, labelled "Recorded run". Still open at one sampler whose PromptManager text was recorded at execution (a language model's output), the recorded text shows as "Execution recorded". Results the file resolves alone never change; a failed lookup judges the file alone. * fix(inspector): recorded texts follow history's saver; conflicts anchor nothing (#386) Review follow-ups: the texts PromptManager recorded are looked up at the sampler history's saver leads to, never at a sampler the unanchored walk named; a file name naming a different saver than history anchors nothing; outputs and texts are read in one transaction; "Recorded run" ranks below every workflow-inferred source; unexpected lookup errors log at error level. * fix(history): look up recorded runs by history's own spelling of the file (#386) The history route inspects a case-normalized target on Windows, while history matches file names exactly, so the occurrence's reference names the file. * refactor(inspector): share the inspector's resolve as inspection_resolution inspect_prompt_evidence now calls inspection_resolution, which returns the resolution to use, the anchored result and whether the recorded saver chose it. Ingestion can call the same helper for files the default profile cannot decide (#247). * feat(ingestion): one per-file prompt decision for the scan and the monitor utils/file_prompt.py holds decide_file_prompt, the ladder the output scan runs today plus the inspector's rule (inspection profile, then the recorded saver) before the legacy parser, with a SCAN and a MONITOR policy. Junk text (blank, numbers, LoRA tags only, model file names) is never stored. graph_prompt_text and parse_comfyui_prompt move here; the parser accepts parsed metadata (#247). * feat(ingestion): store a file's prompt in one transaction, one prompt per file database/scanned_prompts.py links a file to the prompt of one text, creating it when new, inside one BEGIN IMMEDIATE transaction: a file linked to another prompt is a conflict, a failed link leaves no prompt behind and two writers store one prompt. pair_ingestion gains file_link_rows, link_conflict and Marks/mark_created; ingest_resolved_pair takes optional marks for the components it creates. recorded_runs and named_prompt in run_attribution become public (#247). * feat(scan): decide each scanned file with the shared ladder (#247) _ingest_scanned_file calls decide_file_prompt (scan policy) and stores the result: pairs through ingest_resolved_pair and positives through store_scanned_prompt, both with the scan marks. Files the default profile leaves undecided now get the inspector's resolve, with the saver recorded history names, before the legacy parser, so each branch of a shared-prefix dataset run links its own prompt. Junk text is never stored, a file linked to another prompt is a conflict instead of a second link, and a failed link leaves no prompt behind. _BatchHistory reads the stored prompt and the recorded saver from one recorded_runs pass. _parse_comfyui_prompt calls the moved parse_comfyui_prompt. TESTING.md documents the decision. * fix(ingestion): keep the file-name saver anchor out of the ladder (#390) The #390 ruling queues a file that only its saver's file name resolves for Needs review, never linking it. inspection_resolution gains name_anchor; the ladder passes False, so such a file falls to the legacy floor in the scan (and to Needs review) and is never stored by the monitor, while the recorded saver still decides. The inspector keeps the name anchor. * feat(config): gallery.monitoring.save_foreign_runs, default on (#247) * fix(ingestion): no joined PromptManager text from the legacy floor; junk keeps its step's status The legacy parser returns prepend + text + append, so its answer is refused when a PromptManager-family node has a prepend or append part (#343). Junk found at the undecided-graph step keeps that step's unlinked status and review, so an ambiguous file is still queued for Needs review. The monitor policy's ambiguous file no longer asks for the recorded run, and the file size may be read lazily. * fix(ingestion): keep the legacy answer that is a PromptManager node's own text The joined-text refusal also refused the #170 parity files whose PromptManager prepend is a link the browser parser does not follow: its answer is then the node's literal text, which is what the node stores. Refuse only an answer that is not such a node's own text. * feat(monitor): store the certain prompt of a foreign workflow's output (#247) A foreign output (no positive PromptManager node) is decided by the shared ladder with the MONITOR policy. Certain files are stored at settle with RUN_MARKS through survive_switch; files only recorded history can decide wait in their own bounded set and are re-checked once after the run's outputs are sealed, with the sealed reference. The setting GalleryConfig.SAVE_FOREIGN_RUNS is read for every file. * fix(scan): one prompt per file for recorded-run links; writes survive a library move The recorded run's stored prompt now links in one transaction with the file's link rows, so a file linked to another prompt is a conflict instead of a second link. The scan's stores run through survive_library_switch. A single file asks history once (a one-file _BatchHistory), the scan passes in_output=False and reads the file size only for files with prompt metadata. A conflict row reads "Kept: already linked to a different prompt". TESTING.md records the rules. * feat(settings): save_foreign_runs through POST /settings, applied live (#247) GET /settings reports settings.save_foreign_runs; a save takes a strict bool, writes gallery.monitoring.save_foreign_runs and publishes it to GalleryConfig in the commit, with no restart field. The diagnostics report lists it. The legacy contract snapshot gains the added key (an addition, no legacy key moves). * feat(system): Save prompts from other workflows in Sources & scans (#247, D147) A host setting row below Watch for new outputs drafts save_foreign_runs and saves with the one Save row; the server applies it at once, so no restart is named. While monitoring is drafted off the row is disabled and says it applies while Watch for new outputs is on. The concept comparison maps the inserted row and its mutation check covers it. * test(comfyui): a workflow of ComfyUI's own nodes creates and links its prompt (#247) The real-host attribution spec gains the foreign case (CLIPTextEncode, KSampler, SaveImage): its prompt is created and linked at settle and a second run links to the same prompt. The fixture route lists the waiting foreign files. TESTING.md records the monitor's foreign path and D147. * test(vendor): the execution matrix counts the #247 real-host case * fix(monitor): read a foreign file's size only when the ladder needs it (#247) Uses PR 1's lazy file size and pins that a foreign file linked to another prompt keeps its one link on the pair and the positive store paths. * fix(ingestion): match a file's link rows under its resolved spelling too The scan stores realpath spellings and the monitor the watcher's raw path, so a file reached through a symlinked or junction output folder could get a second generated_images row. file_link_rows now queries the path and its realpath (each normalized and raw) and dedupes by row id. TESTING.md notes the reverse order is matched only when the raw spelling is the realpath. * fix(monitor): wait for the run unless history answered; monitoring off stops foreign writes (#247) Review fixes: at settle a certain answer ends the wait only when the lookup found a record or the answer is the inspector's or the recorded saver's; the legacy floor waits for the re-check. A record without a certain prompt ends the wait at once. Foreign writes need both save_foreign_runs and monitoring_enabled, read again just before the store. Dropped foreign files are logged outside the lock (evictions as evictions, due ones too), and a drained re-check never drops a newer reference. * fix(system): the foreign row can be set while the monitor still watches; its note describes it (#247, D147) Review fixes: while monitoring is saved off but a source still reads watching until restart (D128), the row stays enabled with its note; the note is the switch's aria-describedby. The browser spec restores save_foreign_runs on the shared server. TESTING.md moves D147 into order and records the monitor's review rules. * test(comfyui): each branch of a shared-prefix foreign run links to its own prompt (#247 CUJ 1) * fix(ingestion): a switched-off PromptManager node behind a pass-through stores nothing (#90) The #90 graph walk follows only prompt-like inputs, so a PromptManagerText with Save to library off reaching the encoder through another input name (easy showAnything.anything, a custom node's value) was invisible to it, and the inspector's resolve stored its text. The ladder now checks the positive every step names: a resolved or inspected positive whose path passes a node that is off, or a legacy answer that is such a node's own text, makes the file not_saved. * fix(ingestion): any switched-off node upstream of a positive stores nothing (#90) The ladder walks every input upstream of each sampler or guider positive (not negative, model, clip, latent or image inputs) for PromptManager nodes whose Save to library is off or wired from an unreadable node, so the sampler-less step, an off node with a prepend part and a PromptManagerPair behind a pass-through no longer store its text. The legacy-floor guard reads a node's resolved text. Every guard reports not_saved for a switch that is off and not_saved_wired for one it cannot read. * fix(ingestion): the switch walk skips negative inputs by #90's own rule The upstream walk treated only names containing "negative" as negative, so an off node on a neg, neg_text or text_neg input blocked an enabled positive. prompt_graph gains is_negative_input (the _NEGATIVE_KEY rule, also used by follows_input) and the walk uses it. The guards accept a node with null inputs and a side whose provenance is not an object. * fix(ingestion): the switch walk also skips any input named with "negative" #90's rule matches only whole underscore-separated words, so negative2, negativeprompt and NegativePrompt counted as positive inputs and an off node wired only to one of them blocked the file. The walk skips those too; follows_input is unchanged. * fix(monitor): a foreign not_saved decision logs its skip and never waits (#247, #90) The foreign path stored nothing for a ladder not_saved decision, but logged it as "No certain prompt" and, when only the legacy answer was a switched-off node's text, kept the file waiting for a seal that could only store nothing. Such a decision now ends the wait at settle, and both settle and the re-check log the opted-out path's own line (a wired switch says so). Tests cover an only-PromptManager node switched off (opted out, not foreign), a switched-off PromptManagerText behind a pass-through into a core CLIPTextEncode, a wired switch, a negative-only switched-off node, recorded history holding the off text, and foreign saving off. * fix(monitor): an undecided not_saved waits for its run's record (#247, #90) A not_saved from the ladder's undecided step is not final before a record exists: the recorded saver may still resolve another branch. The settle lookup now keeps such a file waiting; with the record it stores nothing and logs the opted-out line. An early not_saved (a switched-off node upstream of any sampler's positive) never reaches the lookup and stays final. Tests: the undecided case (literal and wired switch) waits, then skips; a switched-off node feeding one branch of a shared-prefix run never waits and stores nothing, not even the branch the recorded saver names. * docs(system): renumber the foreign-runs switch deviation to D158 (#247) #430 took D147 on main; the Save prompts from other workflows row is D158. * test(monitor): a switched-off node in the recorded saver's branch is never foreign (#247, #90) PR 3's repair skips a file whose recorded saver's branch reads a switched-off PromptManager node. On the monitor the same shape never reaches the foreign path: a literal switch is the opted-out path's and a wired one the node's own run's. Nothing is stored at settle or after the run, even from a recorded run holding the off text. |
||
|
Vito
|
4040007264 |
fix(workspace): keep open pages valid across a restart and recover a changed source (#430) (#434)
The gallery root scope and the Images listing cursor were keyed with a per-process random key, so every ComfyUI restart made an open tab's scopes answer 409 source-changed (blank Tag image preview, a Generate that did nothing) and its cursors fail Load more. - utils/workspace_keys.py: one random secret in workspace-scope.key beside the database (owner-only, never config.json or the database), two derived keys; per-process fallback with a warning. Move library copies it. - Cursor v2 binds the listing scope in its MAC instead of sending an unkeyed hash of the root paths; an unverifiable cursor answers 400 cursor-stale and Images offers Refresh instead of a dead Retry. - POST /prompt_manager/images/rescope maps a stale target to its current root only when the held scope was issued for it. - Images finds a renumbered file again from the inspector, the viewer and the Tag image dialog (the dialog stays open, draft kept); nothing else is kept by an old root_index:path key. - Tag image dialog: a state that blocks Save shows its reason above the actions with Refresh, disabled actions look disabled, and an empty preview says "Preview unavailable" (D147). |
||
|
Vito
|
4f37af209c |
feat(collections): store collections and add their routes and Images filter (#331, PR 1) (#400)
* feat(collections): store collections in the library database * feat(collections): add, remove and rename collections and keep members through duplicate cleanup * feat(collections): add collection routes and the Images collection filter * feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * fix(collections): address review findings * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(browser): install the slideshow clock before wrapping setTimeout The slideshow fixture wrapped window.setTimeout before page.clock.install(), so the viewer's timer ran on real time and clock.runFor() did not drive it; a late WebKit timer failed 'slideshow exposes exact state labels' on CI. * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. * test(duplicates): a removed copy with a collection and a category moves both (#331, #332) |
||
|
Vito
|
b5a56bee0b |
feat(images): store images' own categories and set them by route (#332, PR 1) (#401)
* feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. |
||
|
Vito
|
ef7e07c765 |
refactor(admin): open System for Diagnostics and logs (#330, PR 1) (#402)
* test(legacy): snapshot the routes and the settings response Pins every registered route and the key paths of GET /prompt_manager/settings while System S13 (#330) moves the legacy pages' tools to native System. Recorded on the unchanged main; cutover (#334) removes it. * test(legacy): prove each removed legacy control has native coverage TESTING.md gains the "Legacy System UI removed by S13" table (no rows yet). legacy-coverage.test.mjs fails when a removal cites a capability that is not covered, copies another native test, lists an id or method twice, or leaves the id or method on its page. legacy-system-links.spec.mjs holds the System link table and guards the kept admin flows. * refactor(admin): open System for Diagnostics and logs The admin page's Diagnostics and View Logs buttons become links that open System › Diagnostics & logs in a new tab. The legacy Diagnostics and System Logs dialogs and their methods go; TESTING.md lists them against their covered capabilities (rows B1–B7). * test(legacy): find a removed modal id passed to showModal and settle the links spec The removal check also finds an id passed as a string to a ...Modal() call and no longer reads data-id= as id=. The no-request test waits for the page's own start-up reads to finish, the sort check compares as the server sorts, and the kept-flow test checks the added prompt shows in the list. |
||
|
Vito
|
a0fc5171e3 |
fix(inspector): pass text through unknown nodes instead of refusing them (#381) (#383)
* fix(inspector): pass text through unknown nodes instead of refusing them (#381) The inspector refused any node class missing from the semantics table, so a PromptManagerPair behind an rgthree Any Switch showed "Prompt uses a node the inspector does not support". The inspection resolver (Python and browser) now reads through unknown nodes: a text input is the text, positive/negative route by output slot, a conditioning input is followed alone, otherwise every linked input except core type inputs (pass_through.skip_inputs). PromptManager nodes are authoritative; several different texts become one candidate each ("Several prompts reach the sampler"). A seeded text node with no conditioning input is generated at run time, and a recorded PromptManager execution is not checked against text that passed through an unknown node. INSPECTION_RESOLVER_VERSION goes to 2; ingestion keeps the default profile. A legacy parity gate records every committed graph whose inspection positive matches the legacy parser's; no graph may leave it. * fix(inspector): keep recorded pair text, string-only seed rule, role inputs and unique candidates (#381) Review of the pass-through: a walk through an unknown node never overrides a PromptManagerPair's execution record (any outcome but work-limit is unverifiable while the record is checked); the run-time seed rule reads only string outputs; a node with positive/negative inputs follows only the role of the requested slot, nothing when it is unlinked; one text offered twice is one candidate. * fix(inspector): anchor on the saver that wrote the file and resolve stages on shared links (#381) An inspected file's name now anchors the inspection on the one saver whose filename_prefix (literal or traced) makes the file's stem <prefix>_<digits>[_]; the samplers in that saver's image ancestry are the candidates. No match, several matches or an unreadable walk anchor nothing, so the file stays "several possible prompts". In a latent walk, an earlier sampler reading the same conditioning links as the later one is the same prompt, not an unattributed stage. Python and the browser resolver behave the same; the legacy parity gate states each graph that differs from legacy on purpose. * fix(inspector): skip only plain first stages on shared links; an unfinished prefix scan anchors nothing (#381) Review of the saver anchor: a same-links earlier stage is skipped only when it is not a detailer, sets no prompt override and continues no other sampler, so an earlier prompt or override behind it is never hidden. A work limit while tracing a saver prefix ends the match scan without an anchor instead of counting that saver as no match. * test(inspector): cover every plain-first-stage outcome on both resolvers (#381) The frontend coverage gate found local-resolver-stages.mjs branches at 92.44% (< 93). Each way an earlier stage stops being a plain first stage (no latent, literal, missing or cyclic latent, a sampler-shaped node, a detailer) is now a resolver test and an anchor-first-stage-* parity vector. |
||
|
Vito
|
48b1292115 |
feat(nodes): let a link drive the prompt text of every PromptManager node (#345)
* feat(nodes): let a link drive the prompt text of every PromptManager node The Focused, Split and Pair editors hide their widgets, and ComfyUI only draws a widget's input socket while the widget is visible. `text` (and the Pair's `negative_text`) therefore had no socket, and a STRING link dropped on the editor fell through to the first free STRING input, `prepend_text`, which is how #342's workflow was wired. - Detach those inputs from their hidden widgets on create and after a workflow load, so they are ordinary sockets in Classic and Nodes 2.0. The schema, widget order and saved values are unchanged. - A STRING link dropped on the editor body fills the main prompt, then the negative, before the frontend's own choice. - Each editor says when its text comes from a link; the typed text is kept for disconnect. The Classic nodes now save the typed text under a link instead of null, and "use prompt" no longer overwrites it while linked. - Real-host tests: mouse drags in both renderers, a workflow saved before this change, and executions proving the linked text is saved, counted, recorded in history and linked to the run's image. Closes #343 * test(coverage): floor for the linked-text module * test(nodes): PromptSearchList text takes a dragged link like a core widget * fix(nodes): linked text review round for #343 - A legacy frontend Primitive linked to an editor's text or the Pair's negative now runs with its value: the graphToPrompt wrapper and the queue hook fill it in (through virtual reroutes), in new and pre-#343 workflows. - Moving a linked text and releasing it on the editor keeps it on `text`: a link being moved counts as free, as the frontend counts it. - A link dropped on a prompt box (or its link note) fills that box's input, so the Pair's Negative prompt box takes `negative_text` (Classic through getInputOnPos, Nodes 2.0 through the box's slot key). - Only exact STRING links are steered; wildcards keep the frontend choice. - A linked Classic text box gives its row back and gets it back on disconnect; the PromptManagerText note uses the theme colour. - The metadata parsers map positional widget values through the widget table when an editor's widget-bound inputs do not line up with it. - Real-host cases for the legacy Primitive (new and saved workflows, a real Run), the moved link, the Negative box, paste/clone/undo and Split. * fix(nodes): resolve a legacy Primitive's text like the host, once per run (#343) - Inputs still bound to their widget (Classic PromptManager/PromptManagerText) are left as the host resolved them; the injection no longer overwrites them. - For the editors' detached text inputs, the Primitive's value goes through its "Run widget replace on values" replacement and the hidden widget's own serializer, so {a|b} wildcards and %date% tokens resolve as on main. - Each queued output is filled once (graphToPrompt wrapper, or the queue hook for an output that did not come through it): a wildcard is drawn once. - Editors inside subgraphs are found under their <subgraph>:<inner> id. - Real-host runPrompt installs its request route before the run starts. - Shared parser case pins the widget-table fallback for a core KSampler. * fix(nodes): pin the Classic wildcard behaviour and match queued ids by type (#343) - Classic PromptManager/PromptManagerText expose _pmTextSerializerReady, settled once their own text serializer is installed; the real-host specs wait on it and assert the Classic text is queued as written, as on main. - The Primitive fill only touches an output entry whose class_type matches the graph node at that id. - TESTING.md notes the host's harmless "Invalid widget" warning per Run. |
||
|
Vito
|
cbc53ee9f1 |
feat(resolver): trace multi-stage chains through pass-through nodes (#308)
* feat(resolver): trace multi-stage chains through pass-through nodes The inspection resolver (server and local Inspect) now compares a sampler with the earlier sampler its latent continues through image-space pass-through nodes: decode, image filter, model upscale, scale and encode, listed in a new validated `lineage` section of the semantics table. Equal prompts resolve (hires-fix, refiner); different prompts are multistage-differs. A later stage whose positive adds conditioning to the shared base prompt through an additive composite (ConditioningCombine or Concat) keeps the base pair, the text legacy linked (three-stage Flux). Image-edit batches stay ambiguous and run-time text stays untraceable. Ingestion keeps the default profile (D1). A card no longer shows a seed when the inspector names a chain stage whose seed differs from the parser's pick. * feat(resolver): leave out run-time PromptManager parts, labelled Maintainer option A: when a PromptManager prepend_text or append_text is fed directly by a node whose output is decided at run time (LoraManager's TriggerWord Toggle, now a runtime class, or PromptSearchList), the prompt resolves to the node's own text, as legacy showed. The side's source is runtime-parts-omitted and the resolved reason says that text added at run time, such as trigger words, is not included; the prompt block shows that line (D-NEW-R1). The toggle's output is never computed. Literal and traced parts are combined as before; an indirect run-time part, a run-time main text or an unknown feeder keep their outcome. The browser port changes identically. Ingestion is unchanged. * fix(resolver): keep run-time and added-conditioning labels honest A text that left out a run-time PromptManager part is now labelled when any node on its traced path omitted it, so the label survives encoders, concatenations and chained PromptManagers. Comparisons include what a text leaves out, so a partial prompt never equals a complete one: a branch with trigger words and one without stay ambiguous, and such stages are multistage-differs. Only classes listed as run time are left out. The added-conditioning floor labels its positive (D126, "Conditioning added at a later stage is not included."); the run-time line is D125. A card hides its seed only when the stage the inspector names reads a different seed than the parser; an agreeing seed is kept, as on main. The browser port moves its lineage, latent and part helpers into local-resolver-stages.mjs, keeping parity. |
||
|
Vito
|
b2cc489fd4 |
feat(inspect): copy settings summary and local workflow roles (#286)
* feat(inspect): copy settings summary in the full inspector The legacy metadata page's COPY ALL (LI-4) becomes "Copy settings summary" after the six facts of the shared full inspector, so the Images viewer, the recorded history's current file and Inspect all offer it; the Images side panel keeps the concept's footer. The text is the legacy copyAllData template with its placeholders, filled from what the inspector shows. A unit test runs the template read from web/metadata.html on the legacy parse of every parity PNG: 22 of 25 are byte-equal, and the other three differ only by the ZeroOut guard (#249) or facts the legacy parse missed. Copying follows the concept's prompt copy: the toast "Settings summary copied.", or the select-to-copy dialog (now titled per copy) without a clipboard. The concept comparisons map the button as D-NEW-1. * feat(inspect): local execution evidence and prompt tracing A browser port of the server's inspection resolver for one sampler (PR L3 Task 8, LI-1). local-resolver.mjs answers inspect_prompt_evidence's {prompts, resolution} for a local read's records: execution evidence (read_pair_execution and its per-node match), every traced strategy (encoders, strings, concatenations, PromptManager and its historical classes, passthrough, zeroed, selector branches, composites, guiders), the latent stage walk, the R1 overwritten-text guard and the inspection profile's visit, depth, text, path, built and scanned bounds, with the same reason codes and outcome lines. py-json.mjs reads the records as CPython's json.loads does (dict order and last-wins keys, int versus float, BigInt past 2^53, NaN) and measures and strips text as str does. Several sampler candidates, detailer stages and A1111 parameters raise LocalResolverGap until Task 9. Nothing is wired into Inspect yet. Python is the oracle: test_local_inspect_parity compares the port with the server on the parity corpus, 107 named record sets, every work bound at its edge and 600 seeded fuzzed graphs, and requires gaps to be exactly the Task 9 paths. Sixteen porting mutations each fail the suite. The Node unit test replays committed server answers, which the Python suite keeps current. * fix(inspect): a list or object workflow node type answers malformed The R1 overwritten-text check looked every UI workflow node's `type` up among the subgraph IDs. A `type` that is a JSON list or object raised TypeError there, so the inspect route answered 500 "Prompt inspection failed". Such a workflow is now malformed evidence: the resolution is unresolved with reason_code `malformed` and the inspector's own "Prompt evidence is malformed" line, never exception text. The browser port answers the same, and the parity corpus gains the list, object, nested-subgraph and skipped-ID shapes. * fix(inspect): a list or object graph class_type is never a preview output With several sampler candidates the inspector first drops stages whose outputs reach only previews. That pass looked each API graph node's `class_type` up in the preview class set, so a list or object `class_type` raised TypeError and the inspect route answered 500. Only a string names a class now; such a node is an ordinary terminal, as the rest of the resolver already treats it. * feat(inspect): local candidate selection matching the inspector The browser port of the inspection resolver now answers every record set the server does, so no path is left to the server any more: - several sampler candidates, with the identical-branch rule (#270): a compare grid whose branches all resolve to one pair (text and recipe) is resolved, any other mix is ambiguous; - stages whose outputs reach only previews are dropped first; - detailer stages are judged with the samplers in their image ancestry (multistage-differs, cycles, depth and the per-stage visit budgets), and a sampler that only feeds detailers is not a candidate of its own; - A1111 `parameters`, with Python's multiline anchors, `\s` and Unicode `\d`, the bare footer and Template trailers, counted in code points. The parity corpus gains the selection cases and a seeded fuzzer, and every inspection vector now gives the server's prompts, resolution and facts locally. 16 of 17 mutations are caught; the uncaught one (not flattening an ambiguous upstream stage) cannot change any answer. * feat(inspect): local prompts resolved like the server's A file chosen on the Inspect page now shows the prompts and resolution the server's inspector gives the same records: buildLocalPayload takes its roles from the local resolver instead of the interim open roles, so the legacy fill applies exactly where it does for Images. The resolver reads the server's own node table. The page fetches web/js/inspection-semantics.json once through the API client as it mounts, before any selection; the static route serves that one JSON file by exact path (every other JSON stays forbidden). A table that cannot be fetched answers "The inspector's node table is unavailable." as the server does, and a later selection asks again. The Copy settings summary deviation is numbered D117. * fix(inspect): skip a workflow node whose type is a list or object The overwritten-PromptManager walk now skips such a node, on the server and in the browser port, instead of marking the whole file malformed. A prompt that resolves still resolves; the other nodes still feed the check. * fix(inspect): one JSON depth cap for both browser parsers py-json.mjs and local-video.mjs now refuse past the same 1,000 nested arrays or objects, counting every container as CPython's scanner does. Boundary vectors at 999, 1,000 and 1,001 for a deep prompt and a deep workflow run through the Python oracle, and TESTING.md gives the measured per-version server limits and both divergence directions. * fix(inspect): realpath containment for the exact-path node table route inspection-semantics.json now goes through the same realpath check as every other static file, so a symlink out of web/ is refused. The forbidden-path test asserts exactly 403. * fix(inspect): abortable, stepped local reads with a busy state The wait for the node table now ends with the selection's signal. A local read resolves and projects in bounded steps with a turn of the event loop between them, and the inspector carries aria-busy while it loads. A budget test covers maximal 2 MiB graphs and checks the stepped read is byte-identical to one synchronous build. The D117 wrap rule now targets only the summary button, and TESTING.md scopes the restart line to PRs L1 and L2. * fix(inspect): record the two CPython-JSON emulations as a follow-up |
||
|
Vito
|
0095db8cbc |
feat(inspect): local evidence readers for images and videos (#277)
* feat(inspect): bounded browser PNG reader matching the server's
The Inspect page will read a chosen file entirely in the browser, and the
record it shows must equal what Images shows for the same bytes. This adds
to web/js/png-metadata.js (loaded by the legacy pages as before, with
parsePngTextChunks unchanged):
- readPngMetadataFile(file, {signal, prefix, yieldControl, decompress}):
a port of utils/prompt_inspector.py's _png_metadata, _PngRecords,
_png_text, _inflate and _batch_normalized. It slices only chunk headers,
80-byte keyword prefixes and the retained prompt, workflow, parameters
and comfyassets_prompt_pairs records, CRC-checks what it keeps, applies
the same budgets (2 MiB shared role budget encoded and decoded, 4 MiB
workflow that becomes limited, 8 MiB and 4 MiB duplicate budgets, 4 MiB
duplicate decoded parameters, 10,000 chunks, the batch-index rule for
parameters copies) and compares duplicate chunks byte for byte. Every
range is checked against file.size before slicing and charged against a
19,794,389-byte ceiling, so a multi-gigabyte PNG costs only its metadata.
Compressed text inflates through DecompressionStream with one 256-byte
write in flight and the output capped at the record's allowance; an
AbortSignal rejects the read with an AbortError and leaves no unhandled
rejection; yieldControl runs about every 8 ms of work.
- classifyPrefix(bytes): png, jpeg, webp, iso-bmff, matroska, avi, asf or
null from the first 32 bytes.
- readImageHeader(file, kind): JPEG (SOFn walk as Pillow) and WebP (VP8,
VP8L, VP8X) dimensions from at most 64 KiB.
tests/test_local_inspect_parity.py drives tests/js/support/local-inspect-
oracle.mjs over the committed and test-time inspection parity corpus plus
compressed, oversized, Latin-1, batch-duplicate and all-four-record shapes,
and asserts records, record formats, dimensions and readability equal
read_inspection_file on each file; the two JPEG/WebP fixtures are checked
against Pillow. tests/js/png-local-reader.test.js pins each server rule,
the sparse 3 GiB read, the prefix reuse, inflation back-pressure and abort.
tests/fixtures/local-inspect/tiny.{jpg,webp} are CC0 FFmpeg test patterns.
* feat(inspect): bounded browser video tag readers
Add web/js/workspace/local-video.mjs: ISO-BMFF (MP4/MOV/M4V) and Matroska
(MKV/WebM) walkers that read only the container tags the server asks ffprobe
for, slicing box and element headers, carrier names and retained values within
the server's budgets and never a byte inside mdat or a Cluster; a port of
utils/video_metadata.select_video_carrier with the record budgets of
read_inspection_file; and a byte-exact json.loads/json.dumps round trip so
envelope objects reserialize as the server spells them.
Every naming rule mirrors ffprobe 9.0.2 on hand-built files: legacy ilst atoms,
mdta keys, joined repeats, data types, Matroska language and default handling,
and Tags behind a Cluster found only through a SeekHead.
Tests: node units over byte-exact container builders and the committed
fixtures; a Python parity test comparing the local carrier with the server's
selection from the same tag maps (mocked probe) and from the same files (real
ffprobe, including 1 MiB + 2.5 MiB per-key records); an ffprobe
characterization of the recorded fixture maps.
* feat(inspect): server-identical local projection and payload
Port utils/inspection_response.py to the browser: a strict parse with Python's
Decimal spelling, duplicate-key rejection, NaN/Infinity as null and integers
past 2^53-1 as text; the prompt, workflow and carrier projection with the
depth-64 and 20,000-node ceilings; _metadata; and the 8 MiB omission steps
(workflow graph, then prompt graph, then all raw) measured as UTF-8 bytes.
local-inspection.mjs builds the inspection_version 1 wire for one chosen File
from the Task 1 and Task 2 readers (observation selected-local-file, interim
unresolved roles, unavailable library), fits it to the response budget, and
exposes localFileAdapter for the shared controller, inspector and viewer:
handles are session-owned by a private token, so a tampered or hand-made
descriptor is rejected and blob: URLs are never resolved through the API
client. projectInspectionPayload admits selected-local-file only for a
local-file identity.
A Python oracle test compares projectMetadata with _metadata byte for byte over
the parity PNG corpus, the committed video fixtures and tag maps, 75 committed
adversarial record sets (projection-cases.json) and node-ceiling sets built at
test time.
* test(inspect): import oracle modules by file URL for Windows
The local-inspect oracle imported product modules with path.join(...), which
Node parses as a URL scheme on Windows (D:\... -> ERR_UNSUPPORTED_ESM_URL_SCHEME),
failing every tests/test_local_inspect_parity.py oracle test on the Windows CI
job. Every dynamic import in tests/js/support now goes through moduleUrl, a
pathToFileURL wrapper, and a node unit proves the Windows form on any host and
scans the oracles so a path import cannot return.
* fix(inspect): read video tags as ffprobe reports them
Same bytes now give the server's record in five shapes where the local
reader was stricter or kept different tags (PR #277 final review I1, I2,
M1, M2, M4):
- Invalid UTF-8 in a tag value is replaced as ffprobe's default
string_validation=replace does (av_utf8_decode, one U+FFFD per failing
decode step), instead of failing the read. Key names and TagNames that
do not decode are no carrier and are skipped.
- An ilst item whose keys index is 0 or out of range is skipped, as
ffmpeg does, instead of failing the read.
- Only the first keys box is read, as ffmpeg does.
- A first per-key workflow over 4 MiB keeps its place in the carrier
choice as an unread marker, so the server's carrier (per-key, workflow
limited) is chosen; when the server's first probe would overflow, the
re-probe leaves it out and its own bound applies.
- An envelope integer of more than 4300 digits fails the Python json
port, so that envelope is skipped as CPython's json.loads skips it.
Each shape was checked against ffprobe 9.0.2 and read_inspection_file;
the real-ffprobe parity test now builds them, and a 600-case random-byte
comparison of the replacement against ffprobe matched.
* perf(inspect): measure the local response size natively and yield between steps
fitInspectionPayload counted the UTF-8 size of an 8-12 MB wire with a
per-character loop and ran every omission step on the main thread in one
turn (PR #277 final review M7). The size is now counted by TextEncoder
encodeInto through one bounded scratch buffer, and each re-measure
follows a yieldControl turn, with an abort signal checked between steps.
The fit is async; the fitted payload is the same as before.
* fix(inspect): hold video tags as ffmpeg's dictionary does, decode Mac Roman text
Three more same-bytes divergences, found while verifying the review
fixes against ffprobe 9.0.2 and read_inspection_file:
- ffmpeg keeps one tag per ASCII-folded name under the spelling set
last (ISO-BMFF joins the values with ";", Matroska keeps the last), so
PROMPT and Prompt are one record on the server; the local reader kept
both and failed the read as a duplicate. A legacy comment atom and a
later COMMENT key likewise leave no exact-case envelope.
- ffprobe's format_tags filter folds ASCII case only, so a name that is
"workflow" only under Unicode case rules is never reported; the local
reader took it.
- ISO-BMFF data types 0 and 3 are Mac Roman text to ffmpeg, written into
twice the input size (a long value is cut); the local reader decoded
type 0 as UTF-8 and skipped type 3.
The real-ffprobe parity test builds each shape.
* docs(testing): list the local Inspect divergences; raise touched coverage floors
TESTING.md now tracks the local Inspect (#249 PR L1) divergences that
remain after the review fixes, each checked on a host with ffprobe 9.0.2:
media kind by bytes versus extension (parity is for the same bytes with a
matching extension), JPEG dimensions past the 64 KiB header read, classic
QuickTime udta text atoms, raw versus escaped tag bounds, the 10,000
element ceiling, unknown EBML DocTypes, the JSON depth approximation and
the C _decimal assumption.
coverage:update raised the branch floors of the three local Inspect
modules this PR touches; the unrelated activity-scan raise it proposed is
left at main's values.
|
||
|
Vito
|
a44cd27c2e |
fix(parse): build array String() without recursion (#255)
utils.generation_params converted nested arrays to strings recursively, so a crafted file with a value nested about 900 arrays deep raised RecursionError, where the browser parser returns a result. The join now runs on an explicit stack. A JS-parity case with 900-deep values pins it. |
||
|
Vito
|
36564b4767 |
feat(parse): one generation-parameter parser in Python; scan links what the resolver cannot decide (#253)
* feat(parse): port the browser generation-parameter parser to Python Adds utils/generation_params.py, a line-for-line port of extractGenerationParams in web/js/comfy-metadata.js, the parser behind the gallery, admin and metadata pages. The scan and live monitor can use it so every surface reads prompts the same way. The JavaScript parser is unchanged. The port reproduces the JavaScript value semantics the result depends on (property order, truthiness, String(), trim(), strict equality, Map keys, JSON.parse number rounding, and the inputs on which it throws), and runs its recursive tracing on an explicit stack so crafted graphs cannot exhaust the Python stack. Tests: - behaviour tests for each tracing rule, with JavaScript results as expected values, including the visit-limit boundary; - golden vectors of the JavaScript parser's results for every named case and every PNG fixture, so the comparison runs without Node; - a live comparison against the JavaScript parser when Node is present; - a randomized differential test over 9,000 generated graphs and workflows, coherent and corrupted, including inputs where the JavaScript throws. Five deliberate porting mistakes (trim, truthiness, prepend handling, visit limit, pair slot) each fail the tests. On a 38,602-file output folder, all 18,491 files with a workflow give identical results. * fix(scan): link the traced prompt when the pair resolver cannot decide The output scan skipped every file the pair resolver called ambiguous or unresolved, so workflows where several samplers share one prompt (a base pass, an upscale pass and a detailer) never reached the Library from a scan. It now links the positive prompt the shared parser traces for those files, the prompt the inspector and the legacy pages show. Files the resolver decides are stored as before, and graphs without a recognized sampler keep the legacy extractor (#252). Read-only scans of a 38,602-file output folder into temporary databases: - links: 1,500 before, 7,954 after; none of the 1,500 dropped or changed; - of the added links that differ from the pre-redesign scan, 1,275 replace a number, a LoRA tag, a fragment or a truncated prompt that scan stored. * refactor(parse): return a uniform pair from the SameValueZero key |
||
|
Vito
|
6d2213ead5 |
feat(inspect): declarative node support matrix (#250)
* feat(inspect): add a declarative inspection node semantics table
* fix(inspect): bound and isolate the semantics table
Bound the semantics loader: at most 1 MiB is read, containers may nest
at most eight levels, and a RecursionError from the JSON parser becomes
a SemanticsError instead of escaping.
The table is no longer loaded at import. get_semantics() reads it on
first use and caches only a valid result, and the resolver imports it
lazily, only for a profile with semantics on. The default ingestion
profile never imports the module, so a missing or corrupt table cannot
break scan, monitor or admin import. Inspection of a prompt graph with
an unusable table is unresolved with the reason "Inspection node
semantics are unavailable" and an error log, rather than a crash.
INSPECTION_PROFILE now has semantics on; no strategy consumes the table
yet, so resolution is unchanged.
* feat(inspect): resolve guider-based samplers
Under INSPECTION_PROFILE the resolver discovers samplers from the
semantics table and follows SamplerCustomAdvanced through its guider
link (Reroutes allowed) to a listed guider:
- CFGGuider: positive and negative, so the pair resolves.
- BasicGuider: positive only. The state is "absent" with the reason
"Guider has no negative input", the existing missing-negative path.
- Any other guider, including DualCFGGuider (D17), is unsupported-node
and names that guider.
Guider hops draw visits, scan and depth from the existing budgets. A
SamplerCustomAdvanced upstream in latent ancestry is an earlier stage,
so the result stays multistage-ambiguous. Positive and negative
provenance record guider_id. FaceDetailer (detailer-stage) is not a
candidate yet: deciding it needs output ancestry. DEFAULT_PROFILE never
loads the table and is unchanged.
Vector changes (inspection profile only, default vectors unchanged):
- f07b: no-sampler -> resolved (CFGGuider).
- f17: no-sampler -> unsupported-node (FluxGuidance, Task 2.3).
- f01b: no-sampler -> unsupported-node (EmptyFlux2LatentImage latent,
Task 2.5).
- f01: no-sampler -> ambiguous (24 SamplerCustomAdvanced branches).
The no-sampler tests (resolver and inspect route) now use a sampler
class the inspector does not know, because f17 has a supported one.
* feat(inspect): trace conditioning through supported pass-through and selector nodes
* feat(inspect): treat detailers as sampler stages
* fix(inspect): choose selectors that share one condition together
* fix(inspect): bound detailer stage chains by the depth limit
* feat(inspect): support common text encoders and string sources
* fix(inspect): keep pair execution evidence behind unreadable string sources
* feat(inspect): accept any non-sampler latent origin
* fix(inspect): queue each node once in breadth-first ancestry walks
The latent and detailer image walks queued every link before checking
whether its node was already seen, and popped from the list head. A dense
shared fan-in grew the queue far past the visit budget before any visit
was charged. Both walks now use a deque and enqueue each node at most
once, charging every examined edge as scanned work; outcomes are
unchanged.
* fix(inspect): treat unlisted pipe and embed samplers as possible stages
The latent and detailer ancestry walks only flagged unlisted nodes with a
linked latent_image, guider, positive or negative input, so easy-use pipe
samplers, Impact pipe detailers and embed-driven video samplers were walked
through and the final sampler's prompt resolved silently. A separate
stage-shaped input set (adding pipes, embeds and custom-sampler parts)
now makes such nodes unsupported-node, named.
* fix(inspect): walk each sampler's latent ancestry once per resolution
Every selector branch run and candidate re-walked the same latent
ancestry, so conditioning-carrying origins such as WanImageToVideo turned
a three-selector graph from ambiguous into work-limit. The walk outcome is
now cached per sampler; branch runs of one candidate pay for it once and
other candidates are charged its visit cost, as the text memo does. Walks
that ran out of work are not cached. Every link input is still walked.
* fix(inspect): treat unlisted model-driven samplers as possible stages
Unlisted samplers that link a model with conditioning or a latent under
other input names (XlabsSampler, generic video samplers) were still walked
through, so the final sampler's prompt resolved silently. An unlisted node
with a linked model plus conditioning, neg_conditioning, samples or latent
is now unsupported-node, named; model patches such as LoraLoader and
ModelSampling nodes are not.
* feat(inspect): compute facts from the resolved sampler
* fix(inspect): merge facts across sampler branches when ambiguous
* fix(inspect): merge facts over unresolved stages and share stage inputs via the table
* feat(inspect): explain why a prompt could not be resolved
* feat(inspect): port legacy parser rules the new resolver missed (PromptManager)
R1: before 3.2.3 a save patch overwrote every PromptManager node's saved
text with the last-run node's prompt. The legacy viewer recognizes the
signature (comfy-metadata.js createContext: two or more PromptManager
workflow nodes with one API text and different widget values). Inspection
now refuses those nodes as malformed instead of resolving to another
node's prompt; the inspector hands the parsed workflow to the resolver for
this check only. Lands before the new pass-throughs (R6) that would
otherwise make these graphs resolve to the wrong text.
R7: this repository's node classes up to 07a5af3 (PromptManagerV2,
Positive, Negative, Detailer, V2Text, PositiveText, NegativeText) resolve
with their own outputs and their exact strip and ", " join.
The default (ingestion) profile ignores both.
* feat(inspect): port legacy parser rules the new resolver missed (A1111)
R3: an accumulating batch saver repeats the A1111 parameters record once
per batch item, identical except "Batch index: N". The bounded PNG reader
now keeps the last copy (Pillow's rule, which #165 relied on) instead of
calling the file unreadable. Any other difference, including a missing or
second batch index, is still a conflict, and decoding the copies draws from
the duplicate role budget.
R4: parameters text that starts with "Negative prompt:" has an empty
positive (#165), instead of showing the delimiter line as the positive.
R5: the settings footer may be followed by sd-dynamic-prompts' one-line
"Template:" and "Negative Template:" records, and may be "Steps: N" alone
when there is no full footer.
R10: commas inside A1111's quoted settings values no longer split them, so
an "ADetailer prompt" with a comma keeps the other facts known.
The facts' A1111 reader mirrors R4, R5 and R10; the default (ingestion)
profile keeps the historical split. Also from the Task 2.7 review: a guard
test fails on any uncoded unresolved reason without a user line (M1), and
the encoder bounds role lines to the client's 512 code units (M4).
* feat(inspect): port legacy parser rules the new resolver missed (installed nodes)
R6 and R9 for nodes whose sources are installed beside ComfyUI, each
entry's input names and output slots read from the node's code (the task
report lists file and line):
- detailer stages UltimateSDUpscale (image), UltimateSDUpscaleNoUpscale
(upscaled_image) and DetailerForEach (image), judged with their upstream
samplers like FaceDetailer;
- pass-throughs ControlNetInpaintingAliMamaApply and LTXVConditioning
(positive 0, negative 1) and FluxKontextMultiReferenceLatentMethod;
- encoder Text to Conditioning, strings TextInput and DisplayText;
- facts model pass-throughs Lora Loader (LoraManager), Power Lora Loader
(rgthree), ModelPatchTorchSettings, CFGNorm, PerturbedAttentionGuidance.
Impact Pack encodes a detailer's non-empty wildcard instead of its positive
input, so FaceDetailer and DetailerForEach now list it as a prompt
override: a literal wildcard is decided at run time and a linked one names
its source, instead of resolving to the positive input. The overwritten
PromptManager refusal (R1) still holds behind the new pass-throughs. The
default profile does not read the table.
* feat(inspect): port legacy parser rules the new resolver missed (upstream nodes)
R6 and R9 for nodes not installed beside ComfyUI here. Their input names
and output slots come from each pack's source at the commit the saved
workflows record in the node's cnr_id and ver properties (the task report
lists repository, commit, file and line):
- samplers ClownsharKSampler_Beta (RES4LYF), KSampler (Efficient) and
ttN KSampler_v2 (latent input "latent");
- pass-throughs SeedVarianceEnhancer and ApplyInstantIDAdvanced (positive
1, negative 2; output 0 is the model);
- selector Switch conditioning [Crystools];
- encoder smZ CLIPTextEncode: its text when with_SDXL is false; when true
the connected CLIP decides at run time which inputs are read, so the
prompt is known only when text, text_g and text_l agree;
- strings ttN text, CR Text, CR Prompt Text and TextBoxMira;
- facts model pass-through CR Apply LoRA Stack.
Inputs that replace what a sampler reads are not followed: RES4LYF options
(may carry a guider, and with sigmas replace the settings), an Efficiency
script and a ttN XY plot leave the prompt unsupported and the facts
unresolved. The overwritten PromptManager refusal (R1) still holds behind
SeedVarianceEnhancer. The default profile does not read the table.
* test(inspect): cover batch copies past the role budget and orphan widgets
* fix(inspect): match the batch index only in the settings footer
Review I-1: duplicate A1111 parameters copies were compared after blanking
"Batch index: N" anywhere in the record, so copies whose prompts differ
only by that phrase were accepted. The field is now blanked only inside
the record's one "Steps: N, ..." footer line, which must hold it exactly
once; any other difference stays a conflict.
Review M-4: the inspector parses the workflow record only when the API
graph can show the overwritten-PromptManager signature (a PromptManager
class, or a node with PromptManager's search_text input).
* fix(inspect): refuse overwritten PromptManager text inside subgraphs
Controller ruling on review M-1: PromptManager nodes inside subgraphs run
as "<instance>:<inner>" API IDs, nested instances chaining their IDs. The
overwritten-text check now walks subgraph instances (bounded in depth and
nodes) and reads a text widget stored by name, so these nodes are refused
like top-level ones, anchored or not. A definition without an instance
runs nothing and is ignored.
Review M-5: the historical adapters skip an explicit null prepend or append,
as the 07a5af3 code's "if prepend_text:" did.
* test(inspect): import unittest one way in the inspection suites
Code scanning flagged the mixed 'import unittest' and 'from unittest import
mock' forms. Use 'import unittest' plus 'import unittest.mock' everywhere.
* fix(history): name the recorded run's current-file prompts apart from its frozen fields
Now that the inspector resolves more saved files, the Recorded run dialog can
hold two regions named "Positive prompt" (and two of each Copy/Send action):
the frozen run field and the inline current file's prompt. Assistive
technology could not tell them apart, and the real-host chronology test read
two <pre> elements.
Prompt blocks take an optional context label. The inline current file inside
a recorded run names its regions "Current file positive prompt" and its
actions "Copy/Send current file positive prompt"; the visible headings stay
as in the concept. The real-host test now also scopes its frozen-field reads
to the "Frozen run prompt texts" section, and a disposable-server workflow
pins the distinct names with a resolved current file.
* fix(inspect): drop preview-only stages from unanchored candidates
A stage whose outputs reach only non-file outputs (PreviewImage and the other
non_file_outputs) cannot have written a saved file, yet it stayed an
unanchored candidate, so a preview-only FaceDetailer made the saved branch
ambiguous (f01b).
The inspection profile now drops such stages before detailer absorption,
using two linear backward passes over the graph (charged like the consumer
map; a failed pass keeps every stage). A stage without consumers, or one that
also reaches any other terminal, is kept, and nothing is dropped when every
stage is preview-only. Raw plus detailed double saves both write files and
stay ambiguous. A file in the host temp folder may itself be a preview, so it
is inspected with PREVIEW_INSPECTION_PROFILE, which keeps those stages.
Workspace facts follow the same choice: an absent or unresolved result that
names a sampler merges facts across that sampler's stages only, so the
dropped detailer's settings no longer blur the saved file's facts.
f01b now resolves (absent negative, guider has none); its vector is updated
and the expected failure removed. The default profile is unchanged:
differential against
|
||
|
Vito
|
048d599386 |
fix(inspect): bound inspection work, not graph size (#244)
* test(inspect): add synthetic inspection parity fixtures and legacy characterization * test(inspect): pin test-time parity vectors and keep fixtures fully synthetic * fix(inspect): bound resolver work instead of graph size * fix(inspect): bound shared-node and stripped-text work in the inspection profile Keep one memo per resolution whose entries carry their unfolded visit cost; a memo hit charges that cost to the current candidate's allowance only. Cache node validation and charge input keys, stripped characters and copied provenance entries to a whole-resolution work budget before doing the work. The default profile is unchanged. * fix(inspect): keep memoized depth honest across inspection candidates A memo entry now records its height below the reusing depth; a hit raises depth-limit exactly where a fresh walk would, so each candidate's outcome matches its anchored result and does not depend on trace or node order. The default profile is unchanged. * test(inspect): pin memo height propagation Three samplers share one chain, each later one hanging below a memo hit. The third candidate must report depth-limit in both trace orders and when anchored; removing the memo-hit height propagation fails this test. The provenance test now asserts a plain subset. * fix(inspect): tolerate ComfyUI node metadata quirks in projections Projections now take a kind. Each prompt node keeps exactly class_type and inputs; _meta, is_changed and other node keys are validated and dropped, so "is_changed": [NaN] no longer marks the whole prompt graph malformed. A nonfinite value inside inputs, anywhere in a workflow, or in a retained carrier member projects to null and is never reported as a number. The carrier applies the prompt/workflow kinds to its members. Exact numbers are unchanged (Decimal spelling, unsafe integers as strings) and duplicate keys are still malformed. The 500-node gate is replaced by a 20,000-node backstop (workflow count includes subgraph definitions). Raw records are untouched. The f14 inspection vector's facts become known; default vectors are unchanged. * fix(inspect): budget metadata per record so large workflows never hide prompts PNG inspection no longer counts every text chunk against one shared 2 MiB budget. Role records (prompt, parameters, the pair execution record) keep a 2 MiB budget together, encoded and decoded, and exceeding it still fails the whole read. A workflow has its own 4 MiB budget: a chunk over it is skipped by seek, and a compressed one stops inflating at budget + 1. Only the workflow is then reported as limited (record format "limited", no raw text, workflow_graph limited); prompts, facts and the prompt graph are unaffected. Each text chunk's keyword is read first (at most 80 bytes). Chunks that are not retained are skipped by length without being read or checksummed, so memory stays bounded. Byte-identical duplicates of a retained record are accepted and do not consume budget; differing duplicates still raise "Duplicate prompt metadata", and a duplicate of a limited workflow stays limited. The chunk-count limit is unchanged. InspectionRead gains limited_records (empty by default). The reader is only used by the inspection routes, so ingestion, scan and import are unchanged. Vectors: f04-inspection (2.5 MB workflow, within the new workflow budget) and f15-inspection (identical duplicate chunks) now read and resolve, with the same outcome and facts as f05-inspection. Default vectors are unchanged. * fix(inspect): keep prompt evidence longest under the response budget * fix(inspect): bound identical duplicate chunk reads * fix(inspect): fail closed when role duplicates exceed their budget * fix(inspect): use the bounded inspection profile end to end Inspection routes resolve with INSPECTION_PROFILE and expose reason_code and node on the resolution; scan, live monitor and admin import keep the default profile (pinned by spies over the real resolver on f03). JS facts drop the entries and workflow node-count gates; per-lookup visit sets and the server's node ceiling bound the work. The validator accepts the textless `limited` workflow format, follows the workflow-first response omission order, and checks the closed reason-code vocabulary. The file inspector labels limited records and disables their workflow download. vectors.json: 12 inspection vectors gain their reason codes; f03 facts become known and f01 facts ambiguous. Default vectors are unchanged. A new parity test asserts vectors.json facts through the workspace JS. * chore(tools): add read-only inspection corpus tool tools/inspection_corpus.py runs the inspection route pipeline and the workspace JS facts of a chosen code tree over a media folder, read-only, and reports the #240 invariants H1 to H8 with aggregate counts. Rows hold statuses and SHA-256 digests of resolved role texts, never file names or prompt text unless --with-paths is given. Outputs inside the root or a code tree are refused. The facts-over-vectors test now fails instead of skipping when CI is set and Node.js is missing. * docs(testing): document bounded inspection work * fix(tools): fail the corpus run when facts or inspection did not run Rows whose facts were never computed (no or exiting Node.js) now fail H3, and a new harness line fails on any tool exception or missing facts, so a run where nothing was measured can no longer exit 0. Unreadable files stay inspector outcomes. A missing --node binary is refused, an exited harness is reaped, H8 has a 50 ms floor for tiny baselines, and JSONL rows are sorted by id. * docs(inspect): correct ceiling wording and bandit annotations Describe the two 20,000-node ceilings (prompt entries, workflow UI plus subgraph nodes) in TESTING.md, and use bare nosec ids so Bandit stops warning about unparsed test names. * test(inspect): make parity observation tuples explicit |
||
|
Vito
|
590aa2cd0e |
feat(workspace): add native Images browsing and infinite scroll (#201)
* feat(workspace): add native image browsing and continuous loading * test(workspace): assert empty collections through their status * fix(workspace): rescan refreshed images and expose ancestor folders |