feat(system): document where the library lives and how it moves
README: where the library lives, the upgrade path (Move library / Keep it here), PROMPT_MANAGER_DATA_DIR, explicit paths unchanged, and the way back to an older version. TESTING.md: D129 and the library location contract (resolution order, move steps, crash windows, coded refusals, Windows handling).
This commit is contained in:
@@ -751,9 +751,34 @@ ComfyUI_PromptManager/
|
||||
|
||||
## Configuration
|
||||
|
||||
### Where your library lives
|
||||
|
||||
New installs keep the library (`prompts.db`, `config.json` with the CivitAI key, the
|
||||
database backups and the logs) in ComfyUI's system user folder
|
||||
`ComfyUI/user/__promptmanager/`. An update, a reinstall or ComfyUI-Manager replacing
|
||||
the node folder no longer touches it, and ComfyUI's `/userdata` web API does not
|
||||
serve `__` system folders. Thumbnails stay in `output/thumbnails`.
|
||||
|
||||
- **Upgrading from an earlier version:** your library stays where it is, in the node
|
||||
folder, until you move it; nothing is moved on its own. The workspace shows a notice
|
||||
above every page and System › Overview a "Library location" row. **Move library**
|
||||
copies the database with SQLite's backup API, verifies the copy (integrity check, every
|
||||
workflow document, the same rows in every table), copies `config.json` and the backups,
|
||||
switches to the new folder without a restart and only then renames the old files
|
||||
`prompts.db.moved-to-__promptmanager` and `config.json.moved-to-__promptmanager` (never
|
||||
deleted; the old backups stay too). If anything fails, the library stays in the node
|
||||
folder, unchanged. **Keep it here** stops the notice; the Overview row stays. Take a
|
||||
backup first (System › Backup & export or the dashboard's Backup) as with any change.
|
||||
- **Your own folder:** set `PROMPT_MANAGER_DATA_DIR` to an absolute folder outside
|
||||
ComfyUI's output, input and temp folders and your image sources. If it is empty and a
|
||||
node-folder library exists, that library stays in use until you move it there.
|
||||
- **An explicit `database.default_path`** in `config.json` and `PROMPT_MANAGER_CONFIG_PATH`
|
||||
keep working as before (described below); such a library is not offered a move.
|
||||
|
||||
### Database Settings
|
||||
|
||||
You can customize the database path by creating a `config.json` file in the extension root:
|
||||
You can customize the database path by creating a `config.json` file in the library's
|
||||
folder (the node folder for a library that was not moved):
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -970,6 +995,10 @@ MIT License - see LICENSE file for details.
|
||||
|
||||
## Changelog
|
||||
|
||||
### Unreleased (Library location)
|
||||
|
||||
- **📁 Library outside the node folder**: new installs keep `prompts.db`, `config.json`, the database backups and the logs in `ComfyUI/user/__promptmanager/`, where an update or reinstall of the node cannot delete them and ComfyUI's `/userdata` API does not reach. An existing library stays in the node folder until you press **Move library** (a verified copy, then a live switch; the old files are renamed `*.moved-to-__promptmanager`, never deleted) or **Keep it here**. `PROMPT_MANAGER_DATA_DIR` names your own folder. After moving, restart ComfyUI once so the logs follow. Going back to an older version after a move: an older version does not look in `user/__promptmanager`, so stop ComfyUI and copy `user/__promptmanager/prompts.db` (with its `-wal` file, if one is left) and `config.json` into the node folder; the `*.moved-to-__promptmanager` files are the library as it was at the move, without later changes.
|
||||
|
||||
### Unreleased (Workflows stored once)
|
||||
|
||||
- **🗜️ Smaller database**: each distinct workflow and prompt graph is stored once, compressed, instead of again for every image. Images recorded by this version use it from the first start; the maintenance job "Store each workflow once" rewrites the existing records and then reclaims the space. A 20,000-image test library went from 1.4 GB to 60 MB. The job always takes a verified backup first: the first time, `prompts.db.pre-workflow-storage_<date time>` beside the database, kept until you delete it; on later runs, a normal pooled safety backup. No image or video file is opened or changed.
|
||||
|
||||
+74
-1
@@ -2257,7 +2257,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged
|
||||
`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
|
||||
reader can keep it byte-identical (`test_health_check_never_writes`).
|
||||
|
||||
Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D113; D121–D128 assigned):
|
||||
Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D113; D121–D129 assigned):
|
||||
|
||||
- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
|
||||
unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server
|
||||
@@ -2404,6 +2404,20 @@ Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D
|
||||
`Watch for new outputs saves monitoring_enabled; a restart applies it` and `a save
|
||||
refused for one unusable source names it and marks its row`
|
||||
(workspace-system-sources.spec) and the system-state and system-sources node tests.
|
||||
- **D129** (library location, S6b): the concept shows neither where the library lives
|
||||
nor a way to move it. System › Overview's "Library health" gains a "Library location"
|
||||
`.operation` row after Backup status, in that row's own markup: "<path> · <why>",
|
||||
plus "You chose to keep it here." and any `PROMPT_MANAGER_DATA_DIR` problem, with
|
||||
"Move library" for a node-folder library and "Details" (a `dl.meta-grid` dialog)
|
||||
otherwise. While a node-folder library is offered (`location.offer`), the shell shows
|
||||
the concept's `.notice` (the Sources notice's markup: `div > h2 + p`, then
|
||||
`.row-actions` with "Keep it here" and the `ca-primary` "Move library") in
|
||||
`#library-banner` above `main`, in main's own width and padding, with no bottom margin
|
||||
(main's padding spaces it); `workspace.css` carries the `/* D129 */` rules. Move
|
||||
library opens the shared review (From, To, Size, Moves with it, Old files). Pinned by
|
||||
`system shell and overview match the concept` (the mapping inserts the row) and
|
||||
`library notice matches the concept notice` with its mutation check
|
||||
(system-concept.spec), and the D129 tests of workspace-system.spec.
|
||||
- **D124** (inspector): with "Load metadata on selection" off, the Images panel and
|
||||
the viewer's details show "Metadata loads when you ask for it." and a "Load metadata"
|
||||
button instead of reading the file. The concept has no such state, so no concept
|
||||
@@ -2775,6 +2789,65 @@ Restoring a deduped backup on a version before this one is not supported: that
|
||||
version's restore accepts the file and then reads rewritten images without their
|
||||
workflow (README changelog).
|
||||
|
||||
### Library location contract (#249 System S6b)
|
||||
|
||||
`utils/library_location.py` is the one resolver of every data path: the database
|
||||
(`database/operations.py` `_resolve_db_path`, diagnostics), config.json
|
||||
(`PromptManagerConfig.get_config_path`, so the `ConfigStore`), the backups (beside the
|
||||
database), the logs (`utils/logging_config.py`) and the LoRA image cache
|
||||
(`py/lora_utils.py`). Order: a valid `PROMPT_MANAGER_DATA_DIR` (absolute, never inside
|
||||
ComfyUI's output, input or temp folder, a public user folder, an extra gallery root or
|
||||
a gallery source; an invalid one is ignored and reported as a problem code), used when
|
||||
it holds a library or no node-folder library exists; an explicit
|
||||
`database.default_path` other than `prompts.db` (relative to the node folder, as
|
||||
before) and `PROMPT_MANAGER_CONFIG_PATH`, honoured as before; ComfyUI's
|
||||
`user/__promptmanager` (`folder_paths.get_system_user_directory("promptmanager")`, or
|
||||
`get_user_directory()/__promptmanager`) once its marker `library-location.json` says
|
||||
`moved`; a node-folder `prompts.db` that was not moved; else `user/__promptmanager`
|
||||
(new installs; outside ComfyUI the node folder). An empty environment folder never
|
||||
replaces a node-folder library (`data-dir-empty`). Nothing is moved or copied at
|
||||
startup (`tests/test_library_location.py`, `tests/test_library_location_wiring.py`).
|
||||
|
||||
`POST /prompt_manager/system/library/move` (`py/api/library_location.py`,
|
||||
`database/library_move.py`) starts "Move library": kind `library_move`, admission
|
||||
`exclusive`, not cancellable. Refusals answer before any job: 400 `not-movable`,
|
||||
`same-folder`, `target-invalid`; 409 `target-has-library`, `target-conflict`, `busy`;
|
||||
507 `insufficient-space` (the free space of the target's volume, for the database, its
|
||||
backups and config.json plus 16 MiB). The job copies with SQLite's backup API from one
|
||||
read transaction (counts, document digest and `data_version` of that snapshot) into a
|
||||
`.pm-move-*` temp file in the target folder (`journal_mode=DELETE`, fsync, 0600),
|
||||
verifies it (`verify_database_file`, every workflow document, the same rows in every
|
||||
table, the same documents), copies every `prompts.db.backup_*` and
|
||||
`prompts.db.pre-workflow-storage_*` (times kept, size and SHA-256 checked), then, under
|
||||
the old file's `BEGIN IMMEDIATE` and the config lock, rechecks `data_version` (a write
|
||||
meanwhile copies again, up to three times, then `library-busy`), copies config.json
|
||||
(0600, checked), writes the marker `pending`, gives every file its final name with
|
||||
`os.replace` inside the target folder (a different drive is a copy, never a rename),
|
||||
writes the marker `moved` and switches the process (`switch_process`: the resolver's
|
||||
override, `PromptModel.repoint_instances` closes every connection of every model of the
|
||||
old file and points it at the new one, the review and history observers are re-keyed,
|
||||
the API's plans and health report are dropped). Only then are the old `prompts.db` and
|
||||
`config.json` renamed `*.moved-to-<folder>` after a `wal_checkpoint(TRUNCATE)`; a file
|
||||
still open elsewhere is left in place and named (`old-in-use`, `old-locked`); the old
|
||||
backups stay. A failure before the marker says `moved` removes everything it wrote and
|
||||
leaves the old library in use, untouched (`TestFailures`); a crash leaves the old
|
||||
library in use (no or a `pending` marker; the next move replaces its own leftovers) or
|
||||
the new one (`moved`) (`TestCrashWindows`). `GET /prompt_manager/system/library` and the
|
||||
Overview's `location` give `{path, source, movable, keep, offer, target, problem,
|
||||
size_bytes}` with paths relative to ComfyUI; `POST .../keep {keep}` persists
|
||||
`library.keep_in_node_folder` through the `ConfigStore`. Errors are coded and name no
|
||||
path (`tests/test_api_library_location.py`). The real-host `library-location.spec.mjs`
|
||||
(execution suite, its own fresh host on an OS-chosen port, no configuration) proves a new
|
||||
install creates `user/__promptmanager/prompts.db` and its logs, adds nothing but byte
|
||||
code to the node folder, and that `/userdata` serves none of it.
|
||||
|
||||
Windows: every connection to the old file is closed before its rename (the source
|
||||
snapshot connection, every model's connections, the checkpoint connection); temp files
|
||||
are closed before `os.replace`; renames stay inside one folder; folder identity compares
|
||||
`(st_dev, st_ino)` and `normcase(realpath)`, so case variants and links of a served
|
||||
folder are refused. The browser test server closes its own log handlers before its temp
|
||||
root (which now holds the logs) is removed.
|
||||
|
||||
## Workspace data fields for the concept pages (#249)
|
||||
|
||||
These fields are additive. The legacy `/prompt_manager/images/output` listing (no
|
||||
|
||||
Reference in New Issue
Block a user