53cfd8b0358087f748c301d61ba67d9e5610c19c
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vito
|
4032f4b6ab |
feat(system): one tagging configuration and Tag output images as a reviewed job (#328, PR 2) (#366)
* feat(autotag): model status without paths, and a busy answer that names the operation * feat(autotag): model downloads run as Activity jobs * fix(autotag): keep each download route's error shape and harden the fake download * feat(autotag): one server-side tagging configuration * fix(autotag): the legacy batch and review work again * feat(autotag): tag output images as a reviewed Activity job * test(autotag): the legacy batch, review and unload page workflows * fix(autotag): server-free tagging defaults, cancel before load, and named legacy refusals * fix(autotag): name only the workspace batch in a legacy refusal frame |
||
|
Vito
|
fb9134f081 |
feat(system): Backup & export (System S9) (#339)
* feat(system): back up, restore and export natively System > Backup & export replaces its hand-off with the concept's section on the S8 backend: Back up downloads the verified copy and the status line shows only what GET /backup/status reports (D103); Review restore stages a chosen file and shows the server's review in the shared review dialog, Cancel discards the stage and Restore runs the Database restore job, after which the open views reload (D105); Export JSON plus the components and saved-pairs exports (D103); Import / inspect opens the shared Import & inspect dialog; the safety-backup policy saves through the Save row and scheduled backups offer Manual only (D104). Overview's Backup status row reads the same status. Refs #326 * test(system): compare Backup & export with the concept system-backups-concept.spec compares the section, the Restore preview and the Import & inspect dialog with the concept in both themes at 1600 and 390 wide, mapping only D103, D104, D105 (and D15/D55 for the shared Import dialog), and proves a renamed label, a removed row, a lost danger mark and swapped tokens are noticed. A review's grid values now wrap anywhere (D105), so a long backup file name keeps the concept's two equal columns at phone width. Refs #326 * docs(testing): Backup & export contract (D103-D105) Refs #326 * test(coverage): raise the frontend floors S9 measured system-backups.mjs gets its measured floor in place of the provisional one; admin.js, activity-scan.mjs and maintenance-review.mjs branches go up. Refs #326 * fix(system): keep a refused or handed-back restore review usable A Restore refused as busy now opens the same review again with Discard / Restore, and a restore job whose leases did not drain hands its review back (scope.review_kept) to the page that started it, which reopens it; every other path that abandons a token after Restore sends its DELETE, so held reviews can no longer lock restore out. The kept-review line says where the review is, and a full review store says what to do (I1). applyPlan resolves with its outcome and its job tracking now has failing-before proof (N5). Also: the safety-backup toggle says what the policy does (D104, M2); the restore picker has no type filter so safety backups can be chosen (M3); "Export all prompts JSON" names what /export holds (N1); small snapshots show KB (N3); the duplicated holding and save helpers are shared (N4); a dismissal test covers Cancel, x, Escape, a replacing dialog and leaving (M1). Refs #326 * test(coverage): raise the system-backups branch floor Refs #326 |
||
|
Vito
|
8252d9d226 |
feat(system): Integrations and a staged LoRA import (System S10) (#336)
* fix(lora): detect LoraManager in ComfyUI's extra custom_nodes folders Auto-detection looked only in ComfyUI's main custom_nodes folder (#144). It now also searches every custom_nodes folder ComfyUI registers through folder_paths, main folder first, skipping folders that cannot be read. * fix(lora): stage imports before replacing the collection (#221) The LoRA import deleted the whole lora-manager category first and then saved each record in its own transaction, so a discovery, download or database failure left the previous collection gone or partial. The import now runs in three stages: discover and fingerprint the metadata files, stage every record (CivitAI downloads go into the image cache first, counted), then replace the category in one BEGIN IMMEDIATE transaction (replace_category_prompts). Any failure leaves the previous prompts, tags and image links exactly as they were. Prompts a saved pair uses block the replacement; an empty replacement of a non-empty collection needs a reviewed import. POST /prompt_manager/lora/import/preview reviews an import without writing or requesting anything; POST /lora/import/apply {token} runs it as the Activity job "LoRA reference import" (records admission, cancellable while it downloads, the safety backup policy before the replace, refused with plan-changed when the files changed since the review). The legacy POST /lora/scan runs the same stages with its SSE event shape. * feat(lora): the LoRA save takes the settings revision POST /prompt_manager/lora/enable accepts the optional expected_revision that GET /settings returns, so the System Save row refuses to write over a config.json changed elsewhere (settings-changed, nothing written). The legacy page sends none and still merges into what is on disk. * feat(system): configure ComfyUI and LoRA integrations natively System › Integrations replaces its hand-off with the concept's two surfaces. ComfyUI: only the send destination and renderer the bridge supports can be chosen; the other options and Return to canvas are disabled with the reason, and Host compatibility shows the renderer the open ComfyUI windows report (each bridge destination now carries renderer: classic or nodes2) (D106). LoRA & resources: detection, the enabled setting and the loaded trigger words read as separate facts, the read-only LoraManager path row follows the toggle (D106), and the CivitAI key is never filled in: typing drafts a new key, blank keeps the saved one and Clear saved key drafts its removal (D114). They save through the one Save row (/lora/enable with the settings revision). LoRA reference data → Review import shows the server's read-only review (replacement, what saved pairs block, downloads, backup box) and Start runs the LoRA reference import Activity job (D107), with its own Activity descriptor and Review again into Integrations when the files or pairs overtook it. * test(system): compare Integrations with the concept system-integrations-concept.spec compares System › Integrations (both themes, 1600 and 390 wide) with the concept, mapping only D106 and D114, and the LoRA import review with the concept's record-changing review, mapping D107; each comparison catches a removed row, a renamed label or option, a re-enabled unsupported option and a swapped token. On the real ComfyUI host, each bridge destination reports the renderer its window draws nodes with, before and after switching it, and System › Integrations shows it. * test(coverage): record the Integrations frontend floors coverage:update after the full frontend run: system-integrations.mjs at its measured floor, and the raised maintenance review, job view and maintenance floors. * test(system): wait for the renderer line before clicking Integrations rows CI Firefox (390 wide) clicked the LoraManager toggle while the renderer check's line above it changed text and moved the rows, so the click did not reach the switch. The specs now wait for that line to settle (a visible state) before interacting. * fix(lora): recheck the files before the replace and stop an abandoned scan Review fixes for the staged LoRA import (#221): - The metadata files are listed and fingerprinted again right before the replace transaction, so a file changed, removed or added while the downloads ran aborts the import with nothing deleted (System job: plan-changed, Review again; legacy scan: an error frame in its own words). The tests now change real files behind a held download. - A legacy scan whose page closes stops staging before its next LoRA and keeps its records hold until staging has stopped; CivitAI images are written to a temp file beside the cache entry and moved in place, so a half-written image is never seen as cached. - Start refuses with plan-changed when the integration was turned off or moved to another install since the review. - A failed job keeps the skipped count and the names of the LoRAs that could not be prepared. - The replace binds the category instead of one parameter per prompt (SQLite's variable limit), /lora/status says whether a detected install can be used, and the legacy scan logs expected refusals as warnings. * fix(system): partial saves, key focus and readable selects in Integrations A save that writes the LoRA part but not the settings part now keeps the saved part, adopts the new revision and highlights only what failed, so the next save needs no conflict. The CivitAI key status is a persistent live region, Clear saved key returns focus to the key field, and the key draft is dropped when the section is left. The send-destination and renderer selects keep their width on narrow screens. A detected LoraManager outside custom_nodes says it cannot be enabled. * fix(lora): keep the save outcome, honour the umask and sweep stale downloads Leaving Integrations drops the CivitAI key draft only when a key was typed, so the last Save's outcome survives and Sources still marks a refused source. The key status line is rewritten only when it changes. Cached CivitAI images are opened like a plain save, so they keep the umask's mode, and each import removes .pm-*.part files older than an hour from the image cache. A legacy scan stopped because its page left ends cancelled in Activity, and staging is awaited to the end even if the handler is cancelled again. * fix(lora): the import review takes a lease beside a restore The LoRA import review reads the library, so like the maintenance and restore previews it now runs under a request lease: it is refused with 409 busy while a restore runs, and a restore waits for it before it switches the file. Tests pin that a restore is refused while an import runs and that the review, the reviewed import and the admin page's scan are refused while a restore runs. |
||
|
Vito
|
9ab25f9aaa |
feat(system): Maintenance UI (System S7) (#322)
* fix(gallery): clear only Prompt Manager cache entries (#222) The legacy gallery's Clear cache deleted every localStorage key starting pm_ or promptManager_ (the workspace theme and the dismissed update notice among them) and every CacheStorage cache whose name held gallery or thumbnail, which on ComfyUI's shared origin can belong to another tool. web/js/cache-ownership.js names what a clear may remove from an inventory of the real writers: every key Prompt Manager writes is a preference and no page opens CacheStorage or IndexedDB, so nothing is removed and the toast says so. A test fails when a new storage writer is not classified. The server thumbnail clear stays its own request; a refused or unreachable server changes nothing and is reported, and refused browser storage is reported as such. * feat(system): run reviewed maintenance natively System › Maintenance is native: the concept's notice and its Database and Images & cache surfaces replace the hand-off. - Integrity & relationship checks runs the health check; the report's Review missing links reviews a fresh Missing-file links plan. - Review plan shows the server's read-only plan (D100): scope, protected, backup policy and media lines, the listed records, how long the review holds and the backup box; Apply reviewed operation sends only its token and opens the job in Activity. busy names the running job, plan-used links the job that applied it, an expired or replaced review is reviewed again, and a lost answer says Outcome not confirmed and points to Activity. - Find duplicates starts the search; its view offers Review groups (D101): ten groups at a time, Keep radios per copy, each keeper sent as {group_id, root_index, path}; the dialog says History keeps the removed paths in its run records. - Thumbnail cache opens the thumbnail dialog; Metadata cache is disabled with its reason while nothing is owned (D102, Q5). - Activity descriptors for maintenance, duplicates and duplicate_cleanup. - The legacy duplicates dialog's View Full names its source (?root=). * test(system): compare Maintenance with the concept system-maintenance-concept.spec compares System › Maintenance, the plan dialog of each operation and the duplicates dialog with the concept in both themes and both viewports, mapped only by D100–D102, and proves a mutated row, label, danger mark or token is noticed. The duplicates dialog's Review duplicate cleanup is the concept's plain button. * test(system): Maintenance no longer hands off; raise coverage floors The System shell test expected Maintenance's D94 hand-off; the section is native now. coverage:update sets the new modules' floors from the measured run (cache-ownership 94, maintenance-review 94, system-maintenance 92 lines) and raises the floors other files passed. * test(admin): close the preview worker after the last job in teardown A scan publishes its last event, which ends the stream the test reads, before it hands its missing previews to the automatic worker. That hand-off can run after tearDownAsync closed the worker, so it starts a new one whose .part files were still open when the final drain checked for open handles (py3.10 on ubuntu, test_scan_reads_metadata_on_several_threads). The drain now closes the worker for good once every job has ended. * fix(system): address the S7 final review - I1: an apply answer is a refusal only when our server sent it (a known code, or a 4xx whose body says success:false); a proxy's 5xx, an unreadable 2xx or a dropped connection says Outcome not confirmed and reads Activity again. ApiError gains 'refused'; the server's own pre-start failure is coded 'apply-failed'. - M1: the lost-response test lets the apply run, then drops its answer. - M2: a failed apply records its code in the job's scope; Activity offers Review again for an overtaken plan (Maintenance for a duplicate cleanup whose sources changed). - M3: a failed duplicate cleanup toasts the server's reason. - M4: a review button holds while its preview is read (one preview per double press). - M5: the storage guard also catches bracket and property assignment and CacheStorage/IndexedDB opens; why the Toolkit build is skipped is written down. - M6: a refused storage listing says so on the Metadata cache row. - M7, M8: D100 text corrected; workflow_storage has its title. - M9: a busy start names the job the server named. * test: follow #319 in the Maintenance tests - The thumbnail dialog has only Scope since #319. - The admin teardown refuses new preview workers on its own API instead of closing with final=True, which since #319 also shuts the process-wide encoders that later test modules use. * fix(system): keep the server's search-again line on a duplicate review A duplicate cleanup review the server refuses with 409 sources-changed showed the generic "could not be reviewed" message; it now shows the server's "The sources changed since the search. Search for duplicates again." * fix(system): a plan with nothing to do offers only Close A reviewed maintenance plan whose scope is 0 records ("No duplicate prompt text") offered Apply reviewed operation, which started an empty job. It now shows its rows, "There is nothing to apply; nothing was changed." and a Close button. * fix(system): wrap long copy paths in the duplicates review A Keep label is the copy's source and path, and dataset folders often have long names with no space to break at, so the review scrolled sideways. The label now wraps anywhere inside its compare item (D101). |
||
|
Vito
|
6505d339aa |
feat(system): thumbnails on Activity (#311)
* feat(system): run thumbnail generation as an Activity job
GET /prompt_manager/images/thumbnails/plan counts what a run would do
(targets, fresh, remaining past the cap, source labels) and writes
nothing. POST /prompt_manager/jobs/thumbnails {mode, size, quality}
starts the "thumbnails" job: admission guarded, cancellable at a file
boundary, outcome groups generated / skipped / errors. size is the
longest edge (128-2048), quality the encoder, and rebuild writes fresh
previews again through the same atomic rename, so a failed file keeps
its old preview and originals are only read.
The legacy gallery stream now starts or attaches to the same job and
streams its unchanged frames. A client that goes away no longer stops
the work; the gallery's Cancel asks the server to cancel the job.
* feat(activity): per-kind job descriptors
One descriptor table (activity-kinds.mjs) decides each job kind's pill
verb, toasts, line, tiles, outcomes, openable rows and labels; the
store re-exports it. The scan and image tagging views are unchanged.
A thumbnail job counts Generated, Skipped and Errors, reads
"Generating thumbnails" in the pill and "Thumbnails finished · N
generated · M errors" when it ends.
Any other kind (the health check, a held legacy-page operation) gets
the generic descriptor with no tiles and no Results, instead of the
scan's zero tiles. A running job without a known total shows an
indeterminate progress bar and no percentage, and only a job the
server can cancel shows Cancel.
* feat(system): review and start thumbnail jobs
Sources' Thumbnail generation -> Review options and Overview's Review
thumbnails open the concept's Thumbnail maintenance dialog: scope, quality
(preselected from Image loading quality) and preview size limit, a plan
line from the read-only thumbnail plan (D99) and Start, which starts the
thumbnails Activity job and opens it. A size outside 128-2048 requests
nothing; a refused start shows the busy link toast.
* fix(system): thumbnail job review fixes (#249 S5)
- The legacy stream's first frame is job {id, started}, sent after the
subscription so no runner frame is missed; it reads the registry after
prepare, so a job started in that gap is attached to, not refused.
- Every exit sets a truthful final status line; nothing to do never
reads "Processing file 0 of 0".
- Cancel is checked per file during the planning walk; a job cancelled
there publishes no start frame.
- The ffmpeg fallback keeps the chosen quality (-q:v) and the aspect
ratio; the placeholder uses the chosen JPEG settings.
- A locked preview on Windows reads "Preview in use by another program;
the old one was kept", not the raw WinError.
- Thumbnail work no longer defers the monitor's Needs review recording:
it writes previews only, never records.
- The disconnect test now proves the job outlives its stream (one worker,
9 files, three batches); the attach test waits on an event.
* fix(gallery): cancel the stream's own thumbnail job by id
The settings panel and the startup dialog cancel the job named by the
stream's job frame, and only one their stream started. A Cancel before
the frame waits for it, a double Cancel posts once, and a job another
tab or System started keeps running. The startup dialog now settles on
Cancel instead of staying on "Cancelling...".
* fix(activity): indeterminate pill bar and the nothing-to-do line
The pill's mini bar sweeps while a job's total is unknown instead of
showing an empty fill. A finished thumbnail job with nothing to do shows
the server's final line. Coverage floors raised for gallery.js and
activity.mjs.
|