b264dac2e8d1bea9ff7ef7cd6c61d3c19145dc98
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vito
|
d88cf9e7c6 |
fix(quality): sweep the Code Quality standard findings (#558)
* fix(quality): preserve cursor iteration and option validation Keep prefetched rows on ManagedCursor so partial loops and explicit fetches share one position, and stop existing iterators after close or retirement. Preserve chunked reads and native SQLite result lifetime on reexecution and errors. Catch OptionError before ValueError so legacy auto-tag requests receive the actual validation error. Remove the overwritten ViewerJS title option. Regression tests failed before the fixes for lost rows, reads after close/retirement, and misleading option errors. Focused Python checks pass; cursor checks also pass on Python 3.13 and 3.14. Refs #321. * fix(quality): make the test doubles legal raises and hashes The fake queue no longer raises an attribute that starts as None. The oversized kind records hashes instead of raising from __hash__, which also catches a hash that production code swallows. Hostile.__bool__ raises the TypeError a truth test may raise. * fix(quality): drop the extra saveCoverage argument * refactor(quality): remove assignments that are overwritten before use * refactor(quality): remove the remaining overwritten test assignments * refactor(quality): drop imports repeated inside functions * refactor(quality): give duplicate identity keys one shape * refactor(quality): name the unused job event loop variable * refactor(quality): release model gates in finally, not except BaseException * refactor(quality): end the job runners with an explicit return * refactor(quality): use the precise unittest asserts * refactor(quality): import unittest.mock one way in the tests * fix(quality): say why each empty except is safe, log the broad ones * refactor(quality): stop binding values the tests never read * refactor(quality): remove the unused preview aliases and tag page query * refactor(quality): remove re-exports nothing imports * refactor(quality): parenthesize deliberate string joins in lists * refactor(quality): remove unused JavaScript variables and imports * refactor(quality): use the precise assert in the model status test |
||
|
Vito
|
d02a911748 |
feat(library): delete or archive selected prompts, Show archived prompts and Restore (#443) (#470)
* feat(library): set the category on many selected prompts (#437) The Prompt library shelf gets the concept's Images selection checkbox on each card and its .bulkbar (N selected, Set category, Clear), recorded as D148. Set category opens the shared one-value dialog and posts {prompt_ids, category, expected_epoch} to POST /prompt_manager/bulk/category: the server checks the storage epoch the shelf was read under, then sets the category on the Library parents in one BEGIN IMMEDIATE transaction, writing only rows that change. A restored storage answers 409 storage-changed and writes nothing. The legacy admin and images_filter bodies are unchanged. This replaces the legacy admin page's bulk Set Category, a cutover prerequisite (#334). Closes #437 * feat(library): add tags to many selected prompts and select every loaded prompt (#442) The Library selection bar (D148) gets Add tags and select-all of the loaded shelf, the legacy admin page's bulk Add Tags and select-all checkbox. Add tags posts {prompt_ids, tags, expected_epoch} to POST /prompt_manager/bulk/tags: in one BEGIN IMMEDIATE transaction the server checks the storage epoch, normalizes the names as image tags are, gives each Library parent only the names it lacks (casefold), finds or inserts tag rows by exact name only when some prompt gets them, and skips prompts that would pass 20 tags. Prompt tags only. Both bulk/tags forms refuse a non-object body with 400. Ctrl/Cmd+A in the grid and "Select all N loaded" in the bar select every loaded prompt; the offer follows each loaded batch. Closes #442 * feat(vocabulary): tag search, tag sort and an Untagged prompts entry (#444) Tags & categories gains a server-side tag search and sort (keyset pages, cursor scoped to the query) and an "Untagged prompts" row whose Browse opens the Library with tag_mode=untagged. Refine this view gets Tag match, saved Library views keep it, and the routes echo the applied query so a stale server is an error, never an unfiltered list. Recorded as D149. Refs #444 * feat(library): filter prompts by output folder and browse tags with Any (#444) Refine this view on the Prompt library gains the concept's Folder (folders below an output root, read when the dialog opens) and a Tag match select (All / Any). The listing takes folder and tag_mode, matches folders on the normalized file_path under every spelling of each output root, echoes both so a stale server is an error, and saved Library views keep them. Recorded as D150. Refs #444 * docs(library): record Add tags and select-all as D151 (#442) * feat(library): archive paired and delete unpaired selected prompts on the server (#443) A prompt_archive table marks Library parents hidden from the shelf; older builds ignore it. New routes /prompt_manager/library/delete/preview, /library/delete and /library/restore classify a selection (delete, archive, protected, already archived, not found), execute exactly what was reviewed under a records hold, take a verified safety backup (disk checked) before any delete, and restore archived prompts. The shelf, search and the shelf-facing counts honor the archive (?shelf=active|archived); archiving repairs a stale prompt hash so scans find the archived row. * feat(library): Delete… on the selection bar, Show archived prompts and Restore (#443) The Library bar's Delete… previews the selection and opens the shared review: what is deleted (in no saved pair), archived (paired: hidden, restorable) or kept (tagged __protected__), the safety backup and what cannot be undone. Show archived prompts (N) lists archived prompts with Restore on their bar; an archived prompt's page shows an Archived notice with Restore. TESTING.md records D152; the route snapshot gains the three workspace routes. * test(library): cover the Delete… review failure and close-only paths; record coverage floors (#443) * fix(library): let saved-view defaults fill the missing #444 filters libraryViewState passed `?? 'all'` and `?? ''` fallbacks that normalizeViewState already applies to an undefined value. The redundant branches were never taken in the page, which dropped saved-views.mjs below its 95% branch floor. A unit now covers a shelf without the new filters. Refs #444 * fix(library): lay out a loaded page of prompts without waiting for an animation frame (#443) A starved renderer (headless WebKit under CPU load, a background tab) can hold requestAnimationFrame for seconds. The shelf laid out each loaded page only in its next frame, so the grid stayed empty after the first load or a delete's reload while the caption already showed the new count. A loaded batch is now laid out at once; measurements still flush per frame. The regression test holds every animation frame and fails before the fix. * test(saved-views): give the D153 Library view the tag_mode key (#444) The Library saved-view state now holds tag_mode, and the API requires every key, so main's new "Highest rated" spec posts it too. Refs #444 * test(saved-views): give the D153 Library view the folder key (#444) Refs #444 * fix(library): name the saved pair, not negative variants, as the archive reason (#443) A prompt in a saved pair with no negative variants was described in the Delete… review as "1 prompt with negative variants: hidden, restorable". Paired prompts are archived because of the pair, so the Archive row now reads "N prompts in a saved pair: hidden, restorable" and the first sentence explains the pair. Found in the live test of #470. * fix(library): nest Refine folders by path segment; backslash separates only on Windows (#444) Live test B1: the Folder list indented a server list sorted as full-path text, so "-" and "_" sorted before "/" and children landed under the wrong parent (kiko/char under kiko-v6), with case variants interleaved. The options are now a tree built by segment: each folder directly under its own parent, case variants distinct siblings. On POSIX a folder named "Metadata\Klein..." was listed but filtered to no prompts, because normalize_folder read "\" as a separator. It now does so only on Windows (ntpath), as the listing already did. Refs #444 * fix(library): say why an archived prompt is archived, and name variants only when it has any (#443) The archived prompt page's notice always said "This prompt and its negative variants are hidden", also for a saved-pair prompt with none. It now says the prompt is archived because it is in a saved pair, and counts its negative variants only when there are any. From the live test of #470. |
||
|
Vito
|
5e00ee479f |
feat(vocabulary): tag search, tag sort and an Untagged prompts entry on Tags & categories (#444, PR 1) (#452)
* feat(vocabulary): tag search, tag sort and an Untagged prompts entry (#444) Tags & categories gains a server-side tag search and sort (keyset pages, cursor scoped to the query) and an "Untagged prompts" row whose Browse opens the Library with tag_mode=untagged. Refine this view gets Tag match, saved Library views keep it, and the routes echo the applied query so a stale server is an error, never an unfiltered list. Recorded as D149. Refs #444 * test(saved-views): give the D153 Library view the tag_mode key (#444) The Library saved-view state now holds tag_mode, and the API requires every key, so main's new "Highest rated" spec posts it too. Refs #444 |
||
|
Vito
|
cdb771d722 |
feat(images): filter, inspect and count by each image's effective category (#332, PR 2) (#406)
* feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * feat(vocabulary): count and rename image categories (#332) Tags & categories lists every category value with image_count, the stored image_categories rows holding it. Values only images hold follow every prompt-backed value (first_pid at SQLite's INTEGER maximum), so keyset cursors cross the boundary. /library/options keeps its own query and never lists image-only values. Manage category counts image rows (impact.images), treats an image-held target as existing, renames image rows in the same transaction, allows renaming an image-only value, and hashes the sorted image rowids into the impact token so a change between preview and apply is refused with vocabulary-conflict. Every image-category statement reads idx_image_categories_category. * feat(images): the category filter matches each image's effective category (#332) An image's own category wins; without one it inherits its link's: the recipe's category when set, else the positive prompt's (MC4, one rule with the inspector chip). Ambiguous and unlinked images inherit nothing, so an unlinked image with its own category is now listed. The link read no longer filters by category in SQL; the page filter, select all N matching and saved views compare the effective value. A failed own-category read answers the existing listing-failed 500. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * feat(inspector): show an image's own category in the inspector and History (#332) The inspector's library facts gain category_scope "image": an image's own category overrides the inherited one for any link status (linked, ambiguous, none). Images reads it for output sources only; History reads it once per inspection inside its snapshot, keyed by the same file key from the symlink-resolved output folder. A failed read hides the chip rather than show a possibly wrong value. The chip's hidden prefix names the origin ("Image category: "); the visible origin text comes in PR 3. * perf(images): keep the category filter's link read narrow; record D141, D142, D145 (#332) Under a category the link read keeps only links whose raw recipe or positive category is the value (Python still decides the inherited one), the links of images whose own category is the value are read by path, and plain reads skip the inherited-category work. The own-category map is keyed by normalized path. TESTING.md records D141, D142 and D145 and amends D13, D36 and D39; the filter budget seeds 2,000 own categories. * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * fix(images): Windows path keys for own categories; plan check for the category read (#332) Review fixes: on Windows the own-category map keys each file with normalize_image_path (abspath drops trailing dots and spaces), and the plan test also covers the category's narrowing clause. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. |
||
|
Vito
|
b7cf210f0f |
feat(annotations): image tag storage, vocabulary participation and capture store (#276)
* feat(annotations): add image annotation schema with restore validation
Install the image annotation group beside image_favorites and saved_views:
media_annotation_metadata (one row, schema version 1), media_annotation_files
(one private locator per physical file), media_annotation_bytes (one record
per observed SHA-256 with the verified size and mtime_ns) and media_image_tags
(the ordered tag set of a byte record, joined into the global tags registry).
The group migrates in a SAVEPOINT that validates before and after creating
it, with no backfill from prompt or pair tags and no filesystem probe.
annotation_schema_error reads only: it compares the catalog objects after
whitespace compaction, the columns, foreign keys and indexes, and the
persisted values (UUIDs, hex digests, bounds, the 4096-character locator cap
measured past embedded NULs, exactly one metadata row). verify_database_file
checks it with allow_missing=not require_library_state, so old backups
restore and a migrated restore stage must carry it.
Tests pin the exact DDL, a fresh install and idempotent reopen, a current
database without the group gaining it with legacy rows untouched, every
partial or malformed shape rejected without repair, install failure rolling
back and retrying, a WAL backup/restore round trip, staged-failure
immutability of source, live and safety files with every handle closed, and
the require_library_state gate.
* feat(annotations): persist per-image tag sets in the shared vocabulary
Add database/annotations.py (AnnotationOperations, mixed into
PromptDatabase): read_image_tags, replace_image_tags and add_image_tags
run in one _pair_transaction fenced by the storage epoch, own the byte
record by exact locator and digest, always supply UUID ids, rewrite the
ordered junction and bump tag_revision plus image_index_revision.
Revision mismatches raise AnnotationCommandError("tag-conflict") with
the current set; the identical set at the current or previous revision
is replayed without writes; a union over 100 names is "tag-limit".
normalize_image_tags validates each name with validate_vocabulary_tag_name
and folds case-insensitive repeats keeping the first spelling.
refresh_image_fingerprint, current_image_tags (chunked, fingerprint
matched, newest verification wins) and iter_image_tag_records serve the
S1 cheap reads.
Vocabulary participation: _move_tag_links also carries media_image_tags,
legacy merge_tags moves image-only sources, tag rows gain image_count
(files with any byte record carrying the tag), impact gains images and
rename/delete receipts gain images_changed. vocabulary-state.mjs accepts
the optional counts and names images in the delete text.
Browser fixtures clear the three annotation data tables on reset (never
the metadata row); TESTING.md documents the schema beside favorites.
* feat(annotations): add bounded private media capture store
Add utils/media_capture.py: CaptureStore copies one rooted media file
into an owner-only prompt-manager-captures-* temp directory, streaming
it in 1 MiB chunks while computing the whole-file SHA-256, and compares
the opened descriptor with the path (dev, ino where meaningful, size,
mtime_ns, ctime_ns) before and after the copy; any disagreement raises
CaptureChanged and removes the copy. Symlinked files or directories,
root escapes, non-media names and directories raise CaptureDenied; the
locator is normcase(join(realpath(root), relative)), the listing's
identity, so a symlinked root matches it.
Limits are the plan's: 256 MiB per file on stat and stream
(CaptureTooLarge), 8 completed captures, 512 MiB reserved and 2
capture/verify workers with nonblocking admission (CaptureCapacity),
900 s idle and 1800 s hard lifetimes on an injected clock. acquire and
release_use keep an in-use capture past expiry, release and
invalidate_all until its last use ends; close removes only owned files
and the owned directory. A header-only probe of the copy marks Generate
unsupported above 4096 px per side or 16,777,216 px, for unreadable
headers and for video; verify re-hashes the current file without a
copy. Every handle closes before unlink.
tests/test_media_capture.py covers exact bytes and owner-only modes,
alias and escape denial, the symlinked-root locator, change detection
during the copy (size, same-size rewrite with restored mtime through
the POSIX change time, inode replacement), both size caps, count, byte
and worker admission, idle and hard expiry with in-use retention,
cleanup scope, handle release at unlink, dimension and video
readiness, and the allowlist equality with the inspection service.
TESTING.md documents the store and the Windows change-time limit.
* test(annotations): wait on the POSIX change time only where it exists
The same-size rewrite case waits for the filesystem clock to pass the
source's ctime so the detection is deterministic on Linux's coarse
timestamps; Windows reports the creation time as st_ctime, where the
probe could never advance, so the wait runs only on POSIX.
* fix(annotations): compare the change time only between fstat reads
Python 3.12+ on Windows reports the change time from stat but the
creation time from fstat, so a freshly written file whose creation and
write straddle a clock tick failed the pre-read identity check with
CaptureChanged. The path-versus-descriptor checks now compare size,
mtime_ns and dev/ino; ctime_ns is compared between the fstat before
and after the copy, where it still detects a same-size rewrite with a
restored mtime on POSIX.
The symlinked-root test asserted the locator against the unresolved
temp path, which macOS (/var -> /private/var) and Windows (short names)
resolve differently; it now compares against the real path.
* test(browser): give the inspector evidence matrix the concept comparisons' budget
The touch run of 'full inspector attaches matched theme and mobile
evidence with reachable content' captures 32 screenshots across four
viewports and took 23.5 s, then 31.6 s, in the sharded coverage job,
hitting the 30 s default. Like the concept comparison specs it now sets
an explicit 120 s timeout; the CI-like pinned run (4 CPUs, 3 workers,
repeat-each 3) passes 198/198 with the matrix at 18-20 s on touch.
* fix(annotations): compare capture identity like with like and keep paths out of errors
Native Windows CI (Python 3.12/3.13) rejected untouched files with
CaptureChanged because the path stat and the descriptor fstat disagree
there (change time versus creation time, possibly dev/ino encodings).
The descriptor is now checked against the path only on the API-stable
facts (size, mtime_ns); the full identity (size, mtime_ns, ctime_ns,
dev/ino where meaningful) is compared between the fstat before and
after the copy and between the path stats before and after. A test
injects shifted fstat results to simulate the Windows discrepancy, and
a fresh file is captured twice with no platform guard.
Error messages carry strerror only, never the path (the path goes to
the debug log); a test asserts no CaptureError names the root. Files
are unlinked outside the store lock; close() during an in-flight
capture lets that worker remove its temp file and then the directory.
get() documents that readers of capture.path must acquire/release_use.
The test fixture resolves its temp root (macOS /private/var, Windows
8.3 names). TESTING.md states the real identity contract.
* fix(annotations): move image revisions when a fingerprint or the vocabulary changes
A replayed save that observes a new verified (size, mtime_ns) now bumps
image_index_revision, like refresh_image_fingerprint, so caches keyed
on it see the record become current again; an identical replay still
moves nothing.
Vocabulary rename, merge and delete (native and legacy paths) bump the
tag_revision of every image set carrying the source tag, in the same
transaction and before the links move or cascade, plus one
image_index_revision bump when any set was affected. A Tag dialog
opened before the change therefore gets tag-conflict with the renamed
set as current instead of recreating the old name.
* test(annotations): accept change detection wherever fstat exposes a change time
Native Windows CI on Python 3.13 detected the same-size, mtime-restored
rewrite through the descriptor's change time (3.10-3.12 report the
creation time there and complete the torn copy). The test asserted the
platform, not the signal: it now accepts CaptureChanged plus temp
removal wherever it happens, requires detection on POSIX and on
Windows 3.13+, and otherwise requires the digest to describe the
copied bytes. A new test injects a drifting fstat change time so the
detection path runs on every platform; TESTING.md records the limit as
"Windows before Python 3.13".
* test(annotations): assert the rewrite outcome, not the platform
Native Windows CI detected the same-size, mtime-restored rewrite on
3.12 and missed it on 3.13 in the same run: the path stat's change time
(Python 3.12+ fast path) only moves when the clock ticks between the
pre-stat and the rewrite, and the tick wait was skipped on Windows. The
wait now runs everywhere with fresh probe names (Windows tunnels the
creation time of a reused name), and the test asserts the outcome: a
detected change leaves no temp file; an undetected one is accepted on
Windows only and must produce a digest of the copied bytes; POSIX must
detect it. The injected-fstat test keeps the detection path covered on
every platform.
* fix(images): keep the Filters label node across a listing reload
Every listing reload reports the active refinement count, and
setActiveFilters rewrote the button's textContent even when the label
was unchanged, replacing its text node. WebKit drops a click whose
mousedown hit a node that was detached before mouseup, so a Safari
user who types a search and clicks Filters inside the 250 ms debounce
lost the click, and the webkit page workflows timed out in the
source-change recovery tests (1 in ~30 runs of the pinned image; an
instrumented probe showed pointerdown/mousedown, a DOM mutation,
pointerup/mouseup and no click event).
The label is rewritten only when it changes. The toolbar unit test
counts textContent writes (1 for an unchanged count before, 0 after),
and a workspace-images test pins the text node identity across a
search reload: 0/6 before the fix on chromium and webkit, 6/6 after,
with the recovery and Refine tests 36/36 in the pinned CI-like run.
|
||
|
Vito
|
ab333d3838 |
feat(vocabulary): backend for native tags and categories (#241)
* feat(tags): keep unused tags in the shared vocabulary * feat(vocabulary): add bounded tag and category reads * feat(vocabulary): guard tag create, rename and merge * feat(vocabulary): preview and apply category consolidation * refactor(vocabulary): share canonical token hashing * test(vocabulary): prove retained tags survive restore * test(vocabulary): cover category paging tie-break and clarify contract docs * test(vocabulary): assert rejected tag creations write nothing |