Compare commits

...
Author SHA1 Message Date
Vito Sansevero 1e1ed5be02 fix(lora): the import review takes a lease beside a restore
The LoRA import review reads the library, so like the maintenance and
restore previews it now runs under a request lease: it is refused with
409 busy while a restore runs, and a restore waits for it before it
switches the file. Tests pin that a restore is refused while an import
runs and that the review, the reviewed import and the admin page's scan
are refused while a restore runs.
2026-10-25 10:25:58 -07:00
Vito Sansevero 4d21f48a8a Merge origin/main (System S8) into feat/327-system-s10-integrations 2026-10-25 10:25:57 -07:00
Vito Sansevero a073dc613e fix(lora): keep the save outcome, honour the umask and sweep stale downloads
Leaving Integrations drops the CivitAI key draft only when a key was
typed, so the last Save's outcome survives and Sources still marks a
refused source. The key status line is rewritten only when it changes.

Cached CivitAI images are opened like a plain save, so they keep the
umask's mode, and each import removes .pm-*.part files older than an
hour from the image cache. A legacy scan stopped because its page left
ends cancelled in Activity, and staging is awaited to the end even if
the handler is cancelled again.
2026-10-25 09:48:55 -07:00
Vito Sansevero 2ff74fda25 fix(system): partial saves, key focus and readable selects in Integrations
A save that writes the LoRA part but not the settings part now keeps the
saved part, adopts the new revision and highlights only what failed, so
the next save needs no conflict. The CivitAI key status is a persistent
live region, Clear saved key returns focus to the key field, and the key
draft is dropped when the section is left. The send-destination and
renderer selects keep their width on narrow screens. A detected
LoraManager outside custom_nodes says it cannot be enabled.
2026-10-25 09:02:15 -07:00
Vito Sansevero d247e4c2e7 fix(lora): recheck the files before the replace and stop an abandoned scan
Review fixes for the staged LoRA import (#221):

- The metadata files are listed and fingerprinted again right before the
  replace transaction, so a file changed, removed or added while the
  downloads ran aborts the import with nothing deleted (System job:
  plan-changed, Review again; legacy scan: an error frame in its own
  words). The tests now change real files behind a held download.
- A legacy scan whose page closes stops staging before its next LoRA and
  keeps its records hold until staging has stopped; CivitAI images are
  written to a temp file beside the cache entry and moved in place, so a
  half-written image is never seen as cached.
- Start refuses with plan-changed when the integration was turned off or
  moved to another install since the review.
- A failed job keeps the skipped count and the names of the LoRAs that
  could not be prepared.
- The replace binds the category instead of one parameter per prompt
  (SQLite's variable limit), /lora/status says whether a detected install
  can be used, and the legacy scan logs expected refusals as warnings.
2026-10-25 08:36:46 -07:00
Vito Sansevero e3ee34bfd0 test(system): wait for the renderer line before clicking Integrations rows
CI Firefox (390 wide) clicked the LoraManager toggle while the renderer
check's line above it changed text and moved the rows, so the click did
not reach the switch. The specs now wait for that line to settle (a
visible state) before interacting.
2026-10-25 08:04:54 -07:00
Vito Sansevero 1b4b9e3af1 test(coverage): record the Integrations frontend floors
coverage:update after the full frontend run: system-integrations.mjs at
its measured floor, and the raised maintenance review, job view and
maintenance floors.
2026-10-25 07:42:51 -07:00
Vito Sansevero 9ec57aaa86 test(system): compare Integrations with the concept
system-integrations-concept.spec compares System › Integrations (both
themes, 1600 and 390 wide) with the concept, mapping only D106 and D114,
and the LoRA import review with the concept's record-changing review,
mapping D107; each comparison catches a removed row, a renamed label or
option, a re-enabled unsupported option and a swapped token.

On the real ComfyUI host, each bridge destination reports the renderer
its window draws nodes with, before and after switching it, and System ›
Integrations shows it.
2026-10-25 07:16:42 -07:00
Vito Sansevero 1172366f83 feat(system): configure ComfyUI and LoRA integrations natively
System › Integrations replaces its hand-off with the concept's two
surfaces. ComfyUI: only the send destination and renderer the bridge
supports can be chosen; the other options and Return to canvas are
disabled with the reason, and Host compatibility shows the renderer the
open ComfyUI windows report (each bridge destination now carries
renderer: classic or nodes2) (D106). LoRA & resources: detection, the
enabled setting and the loaded trigger words read as separate facts,
the read-only LoraManager path row follows the toggle (D106), and the
CivitAI key is never filled in: typing drafts a new key, blank keeps the
saved one and Clear saved key drafts its removal (D114). They save
through the one Save row (/lora/enable with the settings revision).

LoRA reference data → Review import shows the server's read-only review
(replacement, what saved pairs block, downloads, backup box) and Start
runs the LoRA reference import Activity job (D107), with its own
Activity descriptor and Review again into Integrations when the files or
pairs overtook it.
2026-10-25 07:16:35 -07:00
Vito Sansevero 818f2e22e1 feat(lora): the LoRA save takes the settings revision
POST /prompt_manager/lora/enable accepts the optional expected_revision
that GET /settings returns, so the System Save row refuses to write over
a config.json changed elsewhere (settings-changed, nothing written). The
legacy page sends none and still merges into what is on disk.
2026-10-25 06:57:01 -07:00
Vito Sansevero 6a2dae0f78 fix(lora): stage imports before replacing the collection (#221)
The LoRA import deleted the whole lora-manager category first and then
saved each record in its own transaction, so a discovery, download or
database failure left the previous collection gone or partial.

The import now runs in three stages: discover and fingerprint the
metadata files, stage every record (CivitAI downloads go into the image
cache first, counted), then replace the category in one BEGIN IMMEDIATE
transaction (replace_category_prompts). Any failure leaves the previous
prompts, tags and image links exactly as they were. Prompts a saved pair
uses block the replacement; an empty replacement of a non-empty
collection needs a reviewed import.

POST /prompt_manager/lora/import/preview reviews an import without
writing or requesting anything; POST /lora/import/apply {token} runs it
as the Activity job "LoRA reference import" (records admission,
cancellable while it downloads, the safety backup policy before the
replace, refused with plan-changed when the files changed since the
review). The legacy POST /lora/scan runs the same stages with its SSE
event shape.
2026-10-25 06:52:34 -07:00
Vito Sansevero 82f7fc8a6f fix(lora): detect LoraManager in ComfyUI's extra custom_nodes folders
Auto-detection looked only in ComfyUI's main custom_nodes folder (#144).
It now also searches every custom_nodes folder ComfyUI registers through
folder_paths, main folder first, skipping folders that cannot be read.
2026-10-25 06:42:08 -07:00
31 changed files with 4368 additions and 281 deletions
+125 -1
View File
@@ -2399,7 +2399,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged
`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
reader can keep it byte-identical (`test_health_check_never_writes`).
Recorded deviations (plan D60–D70, D79–D81 renumbered D92–D102 and D111–D113; D121–D131 assigned):
Recorded deviations (plan D60–D70, D74, D75, D79–D82 renumbered D92–D102, D106, D107 and D111–D114; D121–D131 assigned):
- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server
@@ -2615,6 +2615,69 @@ Recorded deviations (plan D60–D70, D79–D81 renumbered D92–D102 and D111–
is D93's style. `maintenance matches the concept` (both themes and sizes) maps only this
row; `maintenance comparison detects mutations` proves it is the only difference and
catches a removed row, a renamed operation, a lost danger mark and a swapped token.
- **D106** (Integrations, S10, Q8): the send bridge always asks for the ComfyUI window
and node, and the nodes always follow ComfyUI's own renderer, so "Default send
destination" offers only "Ask each time" and "Host compatibility" only "Follow host
renderer"; the concept's other options stay listed but disabled, each row says why in a
`p.small` under its description, and nothing is stored for them. "Return to canvas
after send" is off and disabled with "Not available: sending applies the text in
ComfyUI and keeps this page open." Host compatibility adds a `p.small[role=status]`
with the renderer the open ComfyUI windows report over the bridge (`renderer`:
`classic` or `nodes2` on each Prompt Manager destination,
`web/js/prompt-bridge-adapter.js`; `#renderer-status`): "Detected: Nodes 2.0 (1 ComfyUI window).", "…
Classic and Nodes 2.0 (2 ComfyUI windows).", "ComfyUI is open; add a Prompt Manager
node to see which renderer it uses." or "No ComfyUI window is open in this browser, so
its renderer is not known."; it looks again when the page becomes visible, and only the
newest check paints. The two selects never shrink for the reason lines and, at phone
width, may grow to 160px (the concept caps them at 120px and cuts "Follow host
renderer"), so each shows its chosen option whole (`selectsFit` in the comparison). "LoraManager
integration" replaces the concept's sample line with the detection ("Detected:
custom_nodes/ComfyUI-Lora-Manager." / "Not detected in ComfyUI’s custom_nodes
folders." / "Found at …, but outside ComfyUI’s custom_nodes folders, so it can’t be
enabled." for a linked install the server would refuse, from `/lora/status`'s
`detected_usable`); the legacy read-only "LoraManager path" row follows it; "Automatically
include trigger words" adds "Trigger words are (not) loaded." (the trigger cache, a
separate fact). The notes read as the row description (`.setting-row p`). Pinned by
`integrations matches the concept` (both themes and sizes) and its mutation check
(system-integrations-concept.spec), the D106 tests of
workspace-system-integrations.spec, and `each destination reports its window
renderer, and System › Integrations shows it (S10)` on the real host
(tests/comfyui/prompt-bridge.spec).
- **D107** (Integrations, S10): "LoRA reference data" reads "Review import" (the
concept's button only simulated a job) and opens "Review LoRA import" in the concept's
record-changing review (its `maintenancePlan` shell): `h3` "LoRA reference data", the
`dl.meta-grid` Scope ("Replace N imported prompts with M LoRAs"), Found ("N metadata
files · M readable"), Protected (the saved pairs that block it), Network ("Downloads
up to N example images from CivitAI[ with your saved key]" / "No downloads"), Backup
and Media on disk, a `details` "LoRAs found (N)", the hold line plus "Example images
downloaded before a failure stay in the image cache." in the `p.small`, the D55 backup
box and Cancel / "Start import" (`danger` when it would remove every import). A review
that saved pairs block, or with nothing to import, shows its rows and only Close.
Start runs the "LoRA reference import" Activity job (D111 descriptor: Imported, Linked
to existing, Skipped, Images downloaded, Download errors, Errors; Cancel only while it
downloads; a failure ends with the server's reason). Pinned by `LoRA import review
matches the concept` and its mutation check (system-integrations-concept.spec) and
the import tests of workspace-system-integrations.spec.
- **D114** (Integrations, S10): the CivitAI key field is never filled in (the page never
receives the key). Typing drafts a new key, a blank field keeps the saved one (no
draft), and "Clear saved key" (shown only while a key is saved) drafts its removal.
A `p.small#civitai-key-status[role=status]` (it also describes the field) says "A key
is saved." / "No key saved." / "The new key replaces the saved one when you save." /
"The saved key is removed when you save."; pressing "Clear saved key" keeps the
keyboard on the key field, and the line is rewritten only when its text changes. A typed
key is dropped when the section is left (the field comes back empty); leaving without one
keeps the last Save's outcome, so Sources still marks a refused source. The field is `autocomplete=new-password`, so browsers never offer it
as a login; the concept's
prototype warning is dropped. The LoRA rows save through the one Save row:
`POST /lora/enable` with `expected_revision` (a config.json changed elsewhere is
`settings-changed`), then `/settings` for any other host change with the revision the
first save returned (`writeHostInParts`). When only one part is written, the store keeps
what saved, adopts its revision and highlights only the rest: the status is the
server's reason (an unusable source) or "Some changes were saved to the server
configuration; the highlighted ones were not.", and the next Save needs no conflict
(`a save where only the LoRA part is written says so; the next save needs no conflict`). Pinned by `key never prefilled; blank keeps; Clear saved key
clears; saved through Save preferences (D114)` and `a LoRA save over settings changed
elsewhere is a conflict, never a blind write` (workspace-system-integrations.spec).
- **D113** (Sources): the Worker threads range ends at the server's `cpu_count` (the
concept's fixed `max=16`) and shows the server's real `worker_threads` (the concept's
sample 4).
@@ -3052,6 +3115,67 @@ Restoring a deduped backup on a version before this one is not supported: that
version's restore accepts the file and then reads rewritten images without their
workflow (README changelog).
### LoRA reference import contract (#221, #249 System S10)
The LoRA import is staged (`py/lora_import.py`): discover the LoraManager
`*.metadata.json` files and fingerprint them (path, size, modification time); stage each
record (CivitAI example images go into the image cache first, best effort, counted in
`downloaded` / `download_errors`; each image is written to a temp file beside its cache
entry and moved in place, so a half-written image is never seen as cached); the files are
listed and fingerprinted again (a file changed, removed or added while staging aborts with
nothing written); then `PromptDatabase.replace_category_prompts`
(`database/category_import.py`) replaces the `lora-manager` category in one
`BEGIN IMMEDIATE` transaction. Any failure leaves the previous prompts, tags and image
links exactly as they were; images already downloaded stay in the cache and the job says
so. A record whose preparation fails is skipped (legacy); a prompt a saved pair uses
blocks the whole replacement (`PairDependencyError`, nothing deleted); a replacement with
no record over a non-empty collection needs `allow_empty` (a reviewed import), so the
legacy scan refuses it.
`POST /prompt_manager/lora/import/preview` (body `{}`) writes nothing and requests
nothing (`test_preview_writes_nothing_and_makes_no_requests`: DB and `-wal` bytes, `-shm`
size, rows, no cache folder, the downloader never called) and answers the maintenance
`Plan` shape plus `lora_files`, `with_metadata`, `current_imports`, `blocked_by_pairs`,
`planned_downloads` and `network`; a blocked plan has no token. `POST
/prompt_manager/lora/import/apply {token}` starts the `lora_import` job (202): title "LoRA
reference import", admission `records`, cancellable only while it stages, the safety
backup the policy promises before the replace. Refusals: `plan-changed` (409, the files
changed since the review, or the integration was turned off or moved to another install),
`plan-expired`, `plan-used`, `storage-changed`, `busy`,
`insufficient-space`, `apply-failed`. The job fails with `plan-changed` (the files or the
imports changed meanwhile), `blocked-by-pairs`, `nothing-to-import`, `backup-failed` or
`import-failed`, each with nothing written; the job keeps what staging skipped, by name,
whatever happened after. The tests change real files behind a held download
(`test_files_changed_while_staging_fail_the_job`, removed and added files too). The
replace binds the category, not one parameter per prompt, so SQLite's variable limit
never caps a collection. Cached CivitAI images are written to a `.pm-*.part` file in the
same folder and renamed into place, with the umask's mode as a plain save gives; each import
first removes `.pm-*.part` files older than an hour from the image cache and its LoRA
folders (`TestStaleDownloadSweep`). The legacy `POST /lora/scan` runs the same stages under its SSE
event shape; a page that closes its connection stops staging before its next LoRA, and the
scan keeps its records hold until staging has stopped, then ends `cancelled` in Activity
(`test_a_disconnected_legacy_scan_stops_and_holds_until_it_stopped`) (`test_legacy_scan_events_unchanged_on_success`), holds
the registry as "LoRA import (admin page)" and takes no safety backup. Both imports are
`records` jobs, so a restore (`exclusive`, #249 S8) is refused while one runs and they are
refused while a restore runs; the review is a request lease ("LoRA import review"), refused
(409 `busy`) during a restore, and a restore waits for it (`LoraImportRestoreTests`). `POST /lora/enable`
accepts the optional `expected_revision`. LoraManager detection also searches every
`custom_nodes` folder ComfyUI registers through `folder_paths` (#144), main folder first.
`tests/browser/workspace_integrations_fixture.py` (`/__test__/integrations/*`) makes a fake
LoraManager install and LoRA folder under the server's temp root, holds or fails the
import, saves a blocking pair and answers CivitAI downloads with a stub opener, so no
test reaches the network.
```sh
python3 -m unittest tests.test_lora_import tests.test_api_lora tests.test_lora_database \
tests.test_lora_utils -v
node --test tests/js/workspace-system-integrations.test.js tests/js/prompt-bridge-adapter.test.js
npm run --prefix tools/vendor test:workflows -- workspace-system-integrations.spec.mjs \
system-integrations-concept.spec.mjs \
--project chromium --project touch --project firefox --project webkit
```
### Browser cache ownership (#222)
The legacy gallery's Clear cache used to delete every localStorage key starting `pm_` or
+198
View File
@@ -0,0 +1,198 @@
"""Atomic replacement of one imported category (#221, #249 System S10).
The LoRA reference import used to delete its whole category and then save
each record in its own transaction, so a failure part way left the previous
collection deleted and the new one partial. ``replace_category_prompts``
does both in one ``BEGIN IMMEDIATE`` transaction: the category's current
prompts (with their tags and image links) are replaced by the staged
records, or, on any error, nothing changes.
A staged record is ``{text, hash, tags, images}``. A record whose hash a
prompt outside the category already has links its images to that prompt
(the legacy import did the same); a second record with the hash of an
earlier one links its images to that one. Neither is a new prompt.
"""
from typing import Any, Dict, List, Optional, Sequence
try:
from .maintenance import PlanChanged
from .pairs import PairDependencyError
from .workflow_documents import documents_of_rows, release_documents
except ImportError: # pragma: no cover - flat test imports
from database.maintenance import PlanChanged
from database.pairs import PairDependencyError
from database.workflow_documents import documents_of_rows, release_documents
class EmptyReplacement(ValueError):
"""The replacement has no record and would remove every current import."""
# The category's prompt ids, as a subquery (one bound parameter).
_IN_CATEGORY = "SELECT id FROM prompts WHERE category = ?"
# Each record's outcome and the result counter it adds to.
_COUNTERS = {"imported": "imported", "linked": "linked", "duplicate": "duplicates"}
def _placeholders(values: Sequence[Any]) -> str:
return ",".join("?" * len(values))
class CategoryImportOperations:
"""Read the state of an imported category and replace it atomically."""
def category_import_state(self, category: str) -> Dict[str, Any]:
"""``{ids, paired}``: the category's prompt ids (ascending) and how
many of them a saved pair uses (those block a replacement)."""
with self.model.get_connection() as conn:
ids = [
row[0]
for row in conn.execute(
"SELECT id FROM prompts WHERE category = ? ORDER BY id",
(category,),
).fetchall()
]
paired = conn.execute(
"SELECT COUNT(*) FROM prompts WHERE category = ? AND id IN "
"(SELECT positive_prompt_id FROM prompt_pairs UNION "
"SELECT negative_prompt_id FROM prompt_pairs)",
(category,),
).fetchone()[0]
return {"ids": ids, "paired": paired}
def existing_prompt_hashes(
self, hashes: Sequence[str], exclude_category: str
) -> set:
"""The given hashes a prompt outside ``exclude_category`` already has."""
found = set()
wanted = list(dict.fromkeys(hashes))
with self.model.get_connection() as conn:
for start in range(0, len(wanted), 500):
chunk = wanted[start : start + 500]
rows = conn.execute(
f"SELECT hash FROM prompts WHERE hash IN ({_placeholders(chunk)}) "
"AND (category IS NULL OR category != ?)",
[*chunk, exclude_category],
).fetchall()
found.update(row[0] for row in rows)
return found
def replace_category_prompts(
self,
category: str,
records: List[Dict[str, Any]],
*,
expected_ids: Optional[List[int]] = None,
allow_empty: bool = False,
) -> Dict[str, Any]:
"""Replace every prompt of ``category`` with ``records`` in one transaction.
Raises (and changes nothing): ``PlanChanged`` when ``expected_ids`` is
given and the category's prompt ids differ from it;
``PairDependencyError`` when a saved pair uses one of its prompts;
``EmptyReplacement`` when ``records`` is empty while the category has
prompts, unless ``allow_empty`` (an empty replacement that was
reviewed). Any other error rolls everything back too.
Returns ``{removed, imported, linked, duplicates, images, rows}``:
prompts removed and created, records linked to an existing prompt or
to an earlier record with the same text, image links added, and per
record ``{name, outcome}`` (``imported``/``linked``/``duplicate``).
"""
with self._pair_transaction() as conn:
ids = [
row[0]
for row in conn.execute(
"SELECT id FROM prompts WHERE category = ? ORDER BY id",
(category,),
).fetchall()
]
if expected_ids is not None and ids != list(expected_ids):
raise PlanChanged("The imported prompts changed since the review")
if not records and ids and not allow_empty:
raise EmptyReplacement(
"Nothing to import; the previous imports were kept"
)
self._ensure_category_unpaired(conn, category)
removed = self._delete_category_rows(conn, category, ids)
result = {
"removed": removed,
"imported": 0,
"linked": 0,
"duplicates": 0,
"images": 0,
"rows": [],
}
created = {}
for record in records:
self._import_record(conn, category, record, created, result)
return result
def _delete_category_rows(self, conn, category: str, ids: List[int]) -> int:
"""Delete the category's prompts with their image links and tags.
No commit. The rows are selected by category (just verified against
``ids`` under the write lock), so no statement binds one parameter per
prompt and SQLite's variable limit never caps a collection.
"""
if not ids:
return 0
documents = documents_of_rows(conn, "prompt_id", ids) # chunked
conn.execute(
f"DELETE FROM generated_images WHERE prompt_id IN ({_IN_CATEGORY})",
(category,),
)
release_documents(conn, documents)
conn.execute(
f"DELETE FROM prompt_tags WHERE prompt_id IN ({_IN_CATEGORY})",
(category,),
)
return conn.execute(
"DELETE FROM prompts WHERE category = ?", (category,)
).rowcount
@staticmethod
def _ensure_category_unpaired(conn, category: str) -> None:
"""Raise PairDependencyError when a saved pair uses a category prompt."""
row = conn.execute(
f"SELECT id FROM prompt_pairs WHERE positive_prompt_id IN ({_IN_CATEGORY}) "
f"OR negative_prompt_id IN ({_IN_CATEGORY}) LIMIT 1",
(category, category),
).fetchone()
if row:
raise PairDependencyError(
f"Prompt belongs to saved pair {row[0]}; delete the pair first "
"or save changed text as a new prompt"
)
def _import_record(self, conn, category, record, created, result) -> None:
"""Save one staged record on ``conn``, or link it to the prompt it repeats."""
prompt_hash = record["hash"]
name = record.get("name") or record["text"]
if prompt_hash in created:
prompt_id, outcome = created[prompt_hash], "duplicate"
else:
row = conn.execute(
"SELECT id FROM prompts WHERE hash = ?", (prompt_hash,)
).fetchone()
if row is not None:
prompt_id, outcome = row[0], "linked"
else:
prompt_id = self._insert_prompt(
conn,
record["text"],
category,
list(record.get("tags") or []),
None,
None,
prompt_hash,
)
outcome = "imported"
created[prompt_hash] = prompt_id
result[_COUNTERS[outcome]] += 1
for image in record.get("images") or []:
if self._insert_image_link(conn, prompt_id, image):
result["images"] += 1
result["rows"].append({"name": name, "outcome": outcome})
+100 -70
View File
@@ -26,6 +26,7 @@ from .history_reads import HistoryReadOperations
from .health import HealthOperations, iter_consistency_issues
from .duplicate_copies import DuplicateCopyOperations
from .workflow_storage import WorkflowStorageOperations
from .category_import import CategoryImportOperations, EmptyReplacement # noqa: F401
from .workflow_documents import (
DocumentReader,
documents_of_rows,
@@ -366,6 +367,7 @@ class PromptDatabase(
MaintenanceOperations,
DuplicateCopyOperations,
WorkflowStorageOperations,
CategoryImportOperations,
):
"""Database operations class for managing prompts."""
@@ -455,26 +457,9 @@ class PromptDatabase(
try:
with self.model.get_connection() as conn:
cursor = conn.execute(
"""
INSERT INTO prompts (
text, category, tags, rating, notes, hash, created_at,
updated_at
) VALUES (?, ?, NULL, ?, ?, ?, ?, ?)
""",
(
text.strip(),
category,
rating,
notes,
prompt_hash,
datetime.datetime.now(datetime.timezone.utc).isoformat(),
datetime.datetime.now(datetime.timezone.utc).isoformat(),
),
prompt_id = self._insert_prompt(
conn, text, category, tags, rating, notes, prompt_hash
)
prompt_id = cursor.lastrowid
if tags:
self._sync_prompt_tags(conn, prompt_id, tags)
conn.commit()
self.logger.debug(f"Successfully saved prompt with ID: {prompt_id}")
return prompt_id
@@ -490,6 +475,32 @@ class PromptDatabase(
)
return existing["id"]
def _insert_prompt(
self,
conn: sqlite3.Connection,
text: str,
category: Optional[str],
tags: Optional[List[str]],
rating: Optional[int],
notes: Optional[str],
prompt_hash: Optional[str],
) -> int:
"""Insert one prompt row and its tags on ``conn`` (no commit)."""
now = datetime.datetime.now(datetime.timezone.utc).isoformat()
cursor = conn.execute(
"""
INSERT INTO prompts (
text, category, tags, rating, notes, hash, created_at,
updated_at
) VALUES (?, ?, NULL, ?, ?, ?, ?, ?)
""",
(text.strip(), category, rating, notes, prompt_hash, now, now),
)
prompt_id = cursor.lastrowid
if tags:
self._sync_prompt_tags(conn, prompt_id, tags)
return prompt_id
def record_prompt_use(self, prompt_id: int, times: int = 1) -> bool:
"""
Count runs of a prompt and mark it as most recently used.
@@ -1670,64 +1681,83 @@ class PromptDatabase(
return 0
# Proceed with linking
if not isinstance(metadata, dict):
metadata = {}
filename = os.path.basename(image_path)
file_path = normalize_image_path(image_path)
file_size, width, height, image_format = _coerce_file_info(metadata)
# Each workflow and prompt graph is stored once (new form).
documents = stored_values(
conn,
{
"workflow_data": dumps_strict(metadata.get("workflow", {})),
"prompt_metadata": dumps_strict(metadata.get("prompt", {})),
},
image_id = self._insert_image_link(
conn, prompt_id_int, image_path, metadata
)
conn.commit()
# INSERT OR IGNORE skips a file already linked to this prompt
# (same prompt_id + normalised full path)
cursor = conn.execute(
"""
INSERT OR IGNORE INTO generated_images
(prompt_id, image_path, filename, file_path, file_size, width,
height, format, workflow_data, prompt_metadata, parameters,
workflow_document_id, prompt_document_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
if not image_id:
self.logger.debug(
f"Image {normalize_image_path(image_path)} already linked to "
f"prompt {prompt_id_int}"
)
return 0
return image_id
def _insert_image_link(
self,
conn: sqlite3.Connection,
prompt_id_int: int,
image_path: str,
metadata: Optional[Dict[str, Any]] = None,
) -> int:
"""Insert one image link on ``conn`` (no commit).
Returns the new image row's id, or 0 when the file is already linked
to this prompt (``INSERT OR IGNORE``).
"""
if not isinstance(metadata, dict):
metadata = {}
filename = os.path.basename(image_path)
file_path = normalize_image_path(image_path)
file_size, width, height, image_format = _coerce_file_info(metadata)
# Each workflow and prompt graph is stored once (new form).
documents = stored_values(
conn,
{
"workflow_data": dumps_strict(metadata.get("workflow", {})),
"prompt_metadata": dumps_strict(metadata.get("prompt", {})),
},
)
# INSERT OR IGNORE skips a file already linked to this prompt
# (same prompt_id + normalised full path)
cursor = conn.execute(
"""
INSERT OR IGNORE INTO generated_images
(prompt_id, image_path, filename, file_path, file_size, width,
height, format, workflow_data, prompt_metadata, parameters,
workflow_document_id, prompt_document_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
prompt_id_int,
image_path,
filename,
file_path,
file_size,
width,
height,
image_format,
documents["workflow_data"],
documents["prompt_metadata"],
dumps_strict(metadata.get("parameters", {})),
documents["workflow_document_id"],
documents["prompt_document_id"],
),
)
if cursor.rowcount == 0:
# Already linked: a document made only for this row goes.
release_documents(
conn,
(
prompt_id_int,
image_path,
filename,
file_path,
file_size,
width,
height,
image_format,
documents["workflow_data"],
documents["prompt_metadata"],
dumps_strict(metadata.get("parameters", {})),
documents["workflow_document_id"],
documents["prompt_document_id"],
),
)
if cursor.rowcount == 0:
# Already linked: a document made only for this row goes.
release_documents(
conn,
(
documents["workflow_document_id"],
documents["prompt_document_id"],
),
)
conn.commit()
if cursor.rowcount == 0:
self.logger.debug(
f"Image {file_path} already linked to prompt {prompt_id_int}"
)
return 0
return cursor.lastrowid
return 0
return cursor.lastrowid
def get_prompt_images(self, prompt_id: str) -> List[Dict[str, Any]]:
"""
+4
View File
@@ -19,6 +19,7 @@ Split into domain-specific mixins for maintainability:
- MaintenanceRoutesMixin: reviewed maintenance plans (preview, apply)
- WorkflowStorageRoutesMixin: store each workflow once, then Reclaim space
- DuplicateFilesMixin: duplicate image files as find and removal jobs
- LoraImportRoutesMixin: the reviewed, staged LoRA reference import job
"""
import asyncio
@@ -64,6 +65,7 @@ from .jobs import JobRegistry
from .logging_routes import LoggingRoutesMixin
from .autotag_routes import AutotagRoutesMixin
from .lora_integration import LoraIntegrationMixin
from .lora_import_routes import LoraImportRoutesMixin
try:
from ...database.operations import PromptDatabase
@@ -309,6 +311,7 @@ class PromptManagerAPI(
LoggingRoutesMixin,
AutotagRoutesMixin,
LoraIntegrationMixin,
LoraImportRoutesMixin,
):
"""REST API handler for PromptManager operations and web interface.
@@ -557,6 +560,7 @@ class PromptManagerAPI(
self._register_logging_routes(routes)
self._register_autotag_routes(routes)
self._register_lora_routes(routes)
self._register_lora_import_routes(routes)
# Register gzip compression middleware (once)
global _gzip_registered
+454
View File
@@ -0,0 +1,454 @@
"""The staged LoRA reference import as a reviewed Activity job (#221, S10).
``POST /prompt_manager/lora/import/preview`` (body ``{}``) reviews an import:
it lists the LoraManager metadata files and parses them, and counts the
current imports, those a saved pair uses and the example images an import
would download. It writes nothing and requests nothing. It answers a
``Plan`` (the maintenance plan's shape, ``py/api/maintenance.py``) plus
``{lora_files, with_metadata, current_imports, blocked_by_pairs,
planned_downloads, network:{planned_downloads, api_key}}``. A plan that saved
pairs block carries no token (nothing can be applied).
``POST /prompt_manager/lora/import/apply {token}`` starts the job (202
``{job}``): kind ``lora_import``, titled "LoRA reference import", admission
``records``, cancellable while it downloads only. Refused before it starts:
``plan-changed`` (409: the metadata files changed since the review),
``plan-expired``, ``plan-used``, ``storage-changed``, ``busy``,
``insufficient-space`` (as maintenance). The job stages the records
(downloads into the cache first, each counted in ``downloaded`` or
``download_errors``), takes the safety backup the policy promises, then
replaces the collection in one transaction bound to the reviewed imports. Any
failure leaves the previous imports exactly as they were, and the job says so.
The legacy ``POST /lora/scan`` runs the same stages (``stage_lora_import``
and ``replace_lora_imports``) under its SSE event shape.
"""
import asyncio
import os
import sqlite3
from aiohttp import web
from .. import lora_import, lora_utils
from .job_routes import busy_response
from .jobs import LORA_IMPORT_JOB_KIND, JobBusy
from .maintenance import (
MAX_TOKEN_LENGTH,
_failure,
_fail_job,
_invalid,
_iso,
disk_plan,
short_space_message,
)
from .maintenance_plans import ScopeTooLarge
try:
from ...database.category_import import EmptyReplacement
from ...database.maintenance import PlanChanged
from ...database.pairs import PairDependencyError
except ImportError: # pragma: no cover - flat test imports
from database.category_import import EmptyReplacement
from database.maintenance import PlanChanged
from database.pairs import PairDependencyError
LORA_OPERATION = "lora_import"
# The review reads the library: a request lease, so it is refused while a
# restore runs and a restore waits for it before switching the file (S8).
LORA_PREVIEW_LEASE = "LoRA import review"
CATEGORY = lora_import.LORA_CATEGORY
SOURCE_CHANGED = (
"The LoRA metadata files changed since the review; nothing was changed. "
"Review the import again."
)
IMPORTS_CHANGED = (
"The imported LoRA prompts changed since the review; the previous imports "
"were kept. Review the import again."
)
BLOCKED = (
"Saved pairs use some imported LoRA prompts, so the collection cannot be "
"replaced; the previous imports were kept."
)
INTEGRATION_CHANGED = (
"The LoraManager integration changed since the review; nothing was changed. "
"Review the import again."
)
EMPTY = "No LoRA could be prepared for import; the previous imports were kept."
FAILED = "The import failed; the previous LoRA imports were kept."
CACHE_NOTE = "Images already downloaded stay in the image cache."
CANCELLED = "Cancelled before anything was written; the previous imports were kept."
DISABLED = "LoraManager integration is not enabled"
NO_PATH = "LoraManager path not configured"
def _noun(count, word, plural=None):
return word if count == 1 else (plural or word + "s")
def _count(count, word, plural=None):
return f"{count:,} {_noun(count, word, plural)}"
def _import_line(result, skipped):
parts = [f"Imported {_count(result['imported'], 'LoRA')}"]
if result["linked"] or result["duplicates"]:
linked = result["linked"] + result["duplicates"]
parts.append(f"{linked:,} linked to an existing prompt")
if skipped:
parts.append(f"{skipped:,} skipped")
return " · ".join(parts)
class LoraImportRoutesMixin:
"""Mixin providing the reviewed LoRA import (preview, apply, job)."""
def _register_lora_import_routes(self, routes):
@routes.post("/prompt_manager/lora/import/preview")
async def lora_import_preview_route(request):
return await self.preview_lora_import(request)
@routes.post("/prompt_manager/lora/import/apply")
async def lora_import_apply_route(request):
return await self.apply_lora_import(request)
# ── shared stages ─────────────────────────────────────────────────
async def _lora_source(self):
"""``(path, None)``: the configured, allowed LoraManager path, or
``(None, (code, message))`` when the import cannot run."""
from ..config import IntegrationConfig
if not IntegrationConfig.LORA_MANAGER_ENABLED:
return None, ("lora-disabled", DISABLED)
lm_path = IntegrationConfig.LORA_MANAGER_PATH
if not lm_path:
return None, ("lora-path", NO_PATH)
allowed = await self._run_in_executor(
lora_utils.resolve_lora_manager_path, lm_path
)
if allowed is None:
from .lora_integration import _LORA_PATH_ERROR
return None, ("lora-path", _LORA_PATH_ERROR)
return allowed, None
async def stage_lora_import(self, files, *, on_record=None, cancelled=None):
"""Stage ``files`` off the loop: records, skips and download counts."""
from ..config import IntegrationConfig
cache_dir = await self._run_in_executor(lora_utils.get_lora_image_cache_dir)
await self._run_in_executor(lora_utils.sweep_stale_downloads, cache_dir)
loop = asyncio.get_running_loop()
def report(index, record, staged):
if on_record is not None:
loop.call_soon_threadsafe(on_record, index, record, staged)
return await self._run_in_executor(
lambda: lora_import.stage(
files,
cache_dir,
IntegrationConfig.CIVITAI_API_KEY,
on_record=report,
cancelled=cancelled or (lambda: False),
logger=self.logger,
)
)
async def lora_files_unchanged(self, lm_path, expected):
"""Whether the metadata files under ``lm_path`` still fingerprint as
``expected`` (listed again, so an added file counts too)."""
def current():
return lora_import.fingerprint(lora_import.discover(lm_path))
try:
return await self._run_in_executor(current) == expected
except OSError:
return False
async def replace_lora_imports(self, db, staged, *, expected_ids, allow_empty):
"""Swap the collection for ``staged.records`` in one transaction."""
return await self._run_in_executor(
lambda: db.replace_category_prompts(
CATEGORY,
staged.records,
expected_ids=expected_ids,
allow_empty=allow_empty,
)
)
# ── preview ───────────────────────────────────────────────────────
async def preview_lora_import(self, request):
try:
async with self._jobs.lease(LORA_PREVIEW_LEASE):
return await self._preview_lora_import(request)
except JobBusy as busy:
return busy_response(busy)
async def _preview_lora_import(self, request):
data, error = await self._read_maintenance_body(request, ())
if error is not None:
return error
lm_path, refused = await self._lora_source()
if refused is not None:
return _failure(refused[1], 400, refused[0])
db = self.db
identity = self._database_identity(db)
try:
cache_dir = await self._run_in_executor(
lambda: lora_utils.get_lora_image_cache_dir(create=False)
)
found = await self._run_in_executor(lora_import.preview, lm_path, cache_dir)
state = await self._run_in_executor(db.category_import_state, CATEGORY)
except Exception:
self.logger.exception("LoRA import preview failed")
return _failure("The LoRA import could not be reviewed.", 500)
if identity != self._database_identity(self.db):
return _failure(
"The database was replaced while it was reviewed. Review again.",
409,
"storage-changed",
)
return await self._lora_plan_response(db, identity, lm_path, found, state)
async def _lora_plan_response(self, db, identity, lm_path, found, state):
from ..config import IntegrationConfig
current = len(state["ids"])
blocked = state["paired"]
policy = self._safety_backup_policy()
try:
disk, _short, size = await self._run_in_executor(
disk_plan, db.model.db_path, backup=policy != "off", vacuum=False
)
except (OSError, sqlite3.Error):
self.logger.exception("Could not measure the disk space for a plan")
return _failure("The LoRA import could not be reviewed.", 500)
token = None
expires_at = None
if not blocked:
payload = {
"path": lm_path,
"fingerprint": found["fingerprint"],
"expected_ids": state["ids"],
"empty": found["with_metadata"] == 0,
"backup": policy,
}
try:
token = self._maintenance_plans.issue(
LORA_OPERATION, found["fingerprint"], payload, identity
)
except ScopeTooLarge:
return _failure(
"This import is too large to review at once.", 413, "too-large"
)
expires_at = _iso(self._maintenance_plans.peek(token).expires_at)
name = os.path.basename(db.model.db_path)
downloads = found["planned_downloads"]
reasons = (
[f"{_count(blocked, 'imported prompt')} used by saved pairs"]
if blocked
else []
)
return web.json_response(
{
"success": True,
"operation": LORA_OPERATION,
"token": token,
"expires_at": expires_at,
"lora_files": found["lora_files"],
"with_metadata": found["with_metadata"],
"current_imports": current,
"blocked_by_pairs": blocked,
"planned_downloads": downloads,
"scope": {
"count": found["with_metadata"],
"label": (
f"Replace {_count(current, 'imported prompt')} with "
f"{_count(found['with_metadata'], 'LoRA')}"
),
},
"protected": {"count": blocked, "reasons": reasons},
"backup": {
"policy": policy,
"destination_label": f"{name}.backup_<date and time>",
"expected_bytes": size if policy != "off" else None,
},
"media": {
"deletes_files": False,
"label": "No files are removed; image links are replaced",
},
"network": {
"planned_downloads": downloads,
"api_key": bool(IntegrationConfig.CIVITAI_API_KEY),
},
"details": [{"label": n, "detail": ""} for n in found["names"]],
"disk": disk,
},
headers={"Cache-Control": "no-store"},
)
# ── apply ─────────────────────────────────────────────────────────
async def apply_lora_import(self, request):
data, error = await self._read_maintenance_body(request, ("token",))
if error is not None:
return error
token = data.get("token")
if not isinstance(token, str) or not 0 < len(token) <= MAX_TOKEN_LENGTH:
return _invalid("token must be the plan's token")
plan, refused = self._peek_plan(token)
if refused is not None:
return refused
if plan.operation != LORA_OPERATION:
return _failure(
"This review expired. Review the plan again.", 410, "plan-expired"
)
store = self._maintenance_plans
db = self.db
if plan.db_identity != self._database_identity(db):
store.discard(token)
return _failure(
"The database was replaced since the review. Review again.",
409,
"storage-changed",
)
lm_path = plan.payload["path"]
# M5: the integration must still be on and at the reviewed install.
current, _refused = await self._lora_source()
if current != lm_path:
store.discard(token)
return _failure(INTEGRATION_CHANGED, 409, "plan-changed")
try:
files = await self._run_in_executor(lora_import.discover, lm_path)
now = await self._run_in_executor(lora_import.fingerprint, files)
backup = self._backup_promised(plan)
_disk, short, _size = await self._run_in_executor(
disk_plan, db.model.db_path, backup=backup, vacuum=False
)
except Exception:
self.logger.exception("LoRA import could not start")
return _failure(
"The import could not start. Nothing was changed.", 500, "apply-failed"
)
if now != plan.fingerprint:
store.discard(token)
return _failure(SOURCE_CHANGED, 409, "plan-changed")
if short is not None:
return _failure(short_space_message(short), 507, "insufficient-space")
plan, refused = self._peek_plan(token)
if refused is not None:
return refused
try:
job = self._jobs.start(
LORA_IMPORT_JOB_KIND,
lora_import.JOB_TITLE,
admission="records",
cancellable=True,
scope={"operation": LORA_OPERATION, "lora_files": len(files)},
groups=lora_import.GROUPS,
runner=lambda job: self._run_lora_import(job, db, plan, files, backup),
)
except JobBusy as busy:
return busy_response(busy)
store.consume(token, LORA_OPERATION, job_id=job.id)
return web.json_response({"success": True, "job": job.summary()}, status=202)
# ── the job ───────────────────────────────────────────────────────
async def _run_lora_import(self, job, db, plan, files, backup_on):
total = len(files)
job.update(total=total, status_line="Preparing LoRA records")
job.log_line(f"Found {_count(total, 'LoRA metadata file')}")
def on_record(index, record, staged):
job.update(
processed=index + 1,
progress=int(90 * (index + 1) / max(total, 1)),
status_line=f"Prepared {index + 1:,} of {total:,}",
)
job.publish({"type": "progress", "progress": job.progress})
try:
staged = await self.stage_lora_import(
files, on_record=on_record, cancelled=lambda: job.cancelled
)
except lora_import.ImportCancelled:
job.log_line(f"{CANCELLED} {CACHE_NOTE}")
job.update(status_line=CANCELLED)
job.finish("cancelled")
return
except Exception:
self.logger.exception("LoRA import staging failed")
return _fail_job(job, "import-failed", f"{FAILED} {CACHE_NOTE}")
self._count_downloads(job, staged)
# M4: what staging skipped stays on the job whatever happens next.
job.record_counted(
"skipped",
len(staged.skipped),
[{"path": "", "detail": name} for name in staged.skipped],
)
# Past this point nothing stops halfway: the replace is one transaction.
# Published, so a view that offered Cancel during the downloads drops it.
job.cancellable = False
job.publish({"type": "progress", "progress": job.progress})
if job.cancelled:
job.log_line(f"{CANCELLED} {CACHE_NOTE}")
job.update(status_line=CANCELLED)
job.finish("cancelled")
return
ok, _backup = await self._safety_backup_step(job, db, backup_on)
if not ok:
return
# I1: the files are listed and fingerprinted again right before the
# replace, so a file changed, removed or added while the downloads ran
# (LoraManager refreshing its CivitAI info) aborts with nothing deleted.
if not await self.lora_files_unchanged(plan.payload["path"], plan.fingerprint):
return _fail_job(job, "plan-changed", f"{SOURCE_CHANGED} {CACHE_NOTE}")
job.update(status_line="Replacing the LoRA collection", progress=95)
await self._replace_in_job(job, db, plan, staged)
def _count_downloads(self, job, staged):
downloaded = staged.stats.get("downloaded", 0)
errors = staged.stats.get("download_errors", 0)
job.record_counted("downloaded", downloaded)
job.record_counted("download_errors", errors)
if downloaded or errors:
job.log_line(
f"Downloaded {_count(downloaded, 'example image')}"
+ (f"; {errors:,} could not be downloaded" if errors else "")
)
async def _replace_in_job(self, job, db, plan, staged):
try:
result = await self.replace_lora_imports(
db,
staged,
expected_ids=plan.payload["expected_ids"],
allow_empty=plan.payload["empty"],
)
except PlanChanged:
return _fail_job(job, "plan-changed", f"{IMPORTS_CHANGED} {CACHE_NOTE}")
except PairDependencyError:
return _fail_job(job, "blocked-by-pairs", f"{BLOCKED} {CACHE_NOTE}")
except EmptyReplacement:
return _fail_job(job, "nothing-to-import", f"{EMPTY} {CACHE_NOTE}")
except Exception:
self.logger.exception("LoRA import failed")
return _fail_job(job, "import-failed", f"{FAILED} {CACHE_NOTE}")
rows = {"imported": [], "linked": []}
for row in result["rows"]:
group = "imported" if row["outcome"] == "imported" else "linked"
rows[group].append({"path": "", "detail": row["name"]})
job.record_counted("imported", result["imported"], rows["imported"])
job.record_counted(
"linked", result["linked"] + result["duplicates"], rows["linked"]
)
line = _import_line(result, len(staged.skipped))
if result["removed"]:
job.log_line(f"Replaced {_count(result['removed'], 'previous import')}")
job.log_line(line)
job.update(progress=100, processed=job.total, status_line=line)
+193 -136
View File
@@ -1,7 +1,9 @@
"""LoraManager integration API routes for PromptManager."""
import asyncio
import contextlib
import json
from pathlib import Path
import threading
from aiohttp import web
@@ -37,6 +39,21 @@ def _lora_settings_mutation(values):
return mutate
# The admin page's own words for the refusals it can meet (it has no review).
_FILES_CHANGED = (
"The LoRA metadata files changed during the scan; the previous imports "
"were kept. Run the scan again."
)
_IMPORTS_CHANGED = (
"The imported LoRA prompts changed during the scan; the previous imports "
"were kept. Run the scan again."
)
class LoraFilesChanged(Exception):
"""A metadata file changed, appeared or went while the legacy scan staged."""
def _lora_error_frame(message):
"""An import stream's error event, as the admin page's modal reads it.
@@ -46,6 +63,23 @@ def _lora_error_frame(message):
return {"type": "error", "status": message, "progress": 0, "message": message}
def _closed(request):
"""A check (safe off the loop) for whether ``request``'s client left."""
def closed():
transport = request.transport
return transport is None or transport.is_closing()
return closed
def _stopped(error) -> bool:
"""Whether a legacy scan ended because its page closed the connection."""
from .. import lora_import
return isinstance(error, (ConnectionError, lora_import.ImportCancelled))
class LoraIntegrationMixin:
"""Mixin providing LoraManager detection, scanning, and trigger word endpoints."""
@@ -108,7 +142,11 @@ class LoraIntegrationMixin:
"""Get current LoraManager integration status."""
try:
from ..config import IntegrationConfig
from ..lora_utils import detect_lora_manager, get_trigger_cache
from ..lora_utils import (
detect_lora_manager,
get_trigger_cache,
resolve_lora_manager_path,
)
config = IntegrationConfig.get_config()["lora_manager"]
cache = get_trigger_cache()
@@ -117,6 +155,12 @@ class LoraIntegrationMixin:
detected_path = await self._run_in_executor(
detect_lora_manager, config.get("path", "")
)
# A detected install reached through a link outside custom_nodes is
# found but cannot be enabled (S10 review N5): say which.
usable = bool(detected_path) and (
await self._run_in_executor(resolve_lora_manager_path, detected_path)
is not None
)
return web.json_response(
{
@@ -126,6 +170,7 @@ class LoraIntegrationMixin:
"trigger_words_enabled": config["trigger_words_enabled"],
"has_civitai_api_key": bool(config.get("civitai_api_key")),
"detected": detected_path is not None,
"detected_usable": usable,
"detected_path": (
self._public_path(detected_path) if detected_path else ""
),
@@ -168,6 +213,9 @@ class LoraIntegrationMixin:
async def lora_enable(self, request):
"""Enable or disable LoraManager integration and save it to config.json.
``expected_revision`` (optional, GET /settings' ``revision``) refuses
a save over a config.json changed meanwhile (``settings-changed``).
The values are committed through the config store first (#220) and
published at runtime only once config.json holds them; a save that
cannot be written reports ``persisted: false`` and changes nothing.
@@ -180,6 +228,11 @@ class LoraIntegrationMixin:
for flag in _LORA_FLAGS:
if flag in data and not isinstance(data[flag], bool):
return self._bad_lora_request(f"{flag} must be true or false")
# The System Save row sends the revision it read (#220); the
# legacy page sends none and merges into what is on disk.
expected_revision = data.get("expected_revision")
if expected_revision is not None and not isinstance(expected_revision, str):
return self._bad_lora_request("expected_revision must be a string")
enabled = data.get("enabled", False)
path = data.get("path", "")
trigger_words = data.get("trigger_words_enabled", False)
@@ -226,7 +279,7 @@ class LoraIntegrationMixin:
_saved, revision = await self._run_in_executor(
get_config_store().commit,
_lora_settings_mutation(values),
expected_revision=None,
expected_revision=expected_revision,
publish=publish,
)
except (ConfigUnreadable, RevisionConflict, OSError) as exc:
@@ -308,43 +361,69 @@ class LoraIntegrationMixin:
) as job:
await response.prepare(request)
try:
await self._lora_scan_stream(allowed, send_progress)
await self._lora_scan_stream(
allowed, send_progress, disconnected=_closed(request)
)
except asyncio.CancelledError:
# The server cancelled the handler when the page left.
job.finish("cancelled")
raise
except Exception as e:
self.logger.exception("LoRA scan failed")
job.finish("failed")
await send_progress(_lora_error_frame(self._public_error(e)))
await response.write_eof()
# A page that left stopped the scan; nothing failed (R2).
job.finish("cancelled" if _stopped(e) else "failed")
await self._send_scan_failure(e, send_progress)
with contextlib.suppress(ConnectionError, RuntimeError):
await response.write_eof()
return response
except Exception as e:
self.logger.error(f"LoRA scan failed: {e}")
return self._error_response(e)
async def _lora_scan_stream(self, lm_path, send_progress):
"""Import every LoRA under ``lm_path``, reporting through ``send_progress``."""
from ..config import IntegrationConfig
from ..lora_utils import (
download_civitai_images,
find_lora_directories,
get_example_prompt_from_metadata,
get_lora_image_cache_dir,
get_preview_images_from_metadata,
get_trigger_words_from_metadata,
get_model_name_from_metadata,
read_lora_metadata,
async def _send_scan_failure(self, error, send_progress):
"""The error frame for a failed legacy scan; quiet when the page left.
Expected refusals (files or imports changed, saved pairs, nothing to
import) are warnings in the admin page's own words; anything else is
logged with its traceback.
"""
from .lora_import_routes import (
EmptyReplacement,
PairDependencyError,
PlanChanged,
)
await send_progress(
{
"type": "progress",
"status": "Clearing previous lora-manager imports...",
"progress": 0,
}
)
if _stopped(error):
self.logger.info("LoRA scan stopped: the page closed its connection")
return
if isinstance(error, PlanChanged):
message = _IMPORTS_CHANGED
elif isinstance(error, LoraFilesChanged):
message = str(error)
else:
message = self._public_error(error)
if isinstance(
error,
(PlanChanged, LoraFilesChanged, EmptyReplacement, PairDependencyError),
):
self.logger.warning(f"LoRA scan refused: {message}")
else:
self.logger.exception("LoRA scan failed")
with contextlib.suppress(ConnectionError, RuntimeError):
await send_progress(_lora_error_frame(message))
# Clear previous imports so reimport is always clean
await self._run_in_executor(self.db.delete_prompts_by_category, "lora-manager")
async def _lora_scan_stream(
self, lm_path, send_progress, disconnected=lambda: False
):
"""Import every LoRA under ``lm_path``, reporting through ``send_progress``.
The staged path of the System import (#221, ``lora_import_routes``):
discover, stage every record (downloads into the cache first), then
replace the collection in one transaction. A failure at any stage
raises with the previous imports untouched; an empty replacement of a
non-empty collection is refused (it needs the reviewed import).
"""
from .. import lora_import
await send_progress(
{
@@ -353,19 +432,8 @@ class LoraIntegrationMixin:
"progress": 2,
}
)
lora_dirs = await self._run_in_executor(find_lora_directories, lm_path)
# Collect all metadata files
meta_files = []
for d in lora_dirs:
dir_path = Path(d)
meta_files.extend(dir_path.rglob("*.metadata.json"))
total = len(meta_files)
imported = 0
skipped = 0
files = await self._run_in_executor(lora_import.discover, lm_path)
total = len(files)
await send_progress(
{
"type": "progress",
@@ -375,116 +443,105 @@ class LoraIntegrationMixin:
}
)
cache_dir = get_lora_image_cache_dir()
reviewed = await self._run_in_executor(lora_import.fingerprint, files)
events = asyncio.Queue()
state = await self._run_in_executor(
self.db.category_import_state, lora_import.LORA_CATEGORY
)
# Records whose prompt already exists outside the collection are
# linked, not imported (the running counts; the end ones are exact).
known = set()
for i, meta_file in enumerate(meta_files):
metadata = await self._run_in_executor(read_lora_metadata, meta_file)
if not metadata:
skipped += 1
continue
def on_record(index, record, _staged):
events.put_nowait((index, record))
model_name = get_model_name_from_metadata(metadata)
trigger_words = get_trigger_words_from_metadata(metadata)
# Collect all images: local previews + downloaded civitai examples
preview_paths = await self._run_in_executor(
get_preview_images_from_metadata, metadata, meta_file
)
civitai_paths = await self._run_in_executor(
download_civitai_images,
metadata,
meta_file,
cache_dir,
IntegrationConfig.CIVITAI_API_KEY,
)
# Merge, local first, dedup
seen = set(preview_paths)
all_images = list(preview_paths)
for cp in civitai_paths:
if cp not in seen:
all_images.append(cp)
seen.add(cp)
# Build prompt text: prefer example prompt, then model name
example_prompt = get_example_prompt_from_metadata(metadata)
prompt_text = example_prompt or model_name
# Build tags
tags = ["lora-manager", f"lora:{model_name}"]
tags.extend(trigger_words)
# Save to database via existing mechanism
try:
import hashlib
prompt_hash = hashlib.sha256(
prompt_text.strip().lower().encode("utf-8")
).hexdigest()
counts = {"imported": 0}
async def report(index, record):
if record is not None and record["hash"] not in known:
known.add(record["hash"])
existing = await self._run_in_executor(
self.db.get_prompt_by_hash, prompt_hash
self.db.existing_prompt_hashes,
[record["hash"]],
lora_import.LORA_CATEGORY,
)
if existing:
# Link all images
for pp in all_images:
await self._run_in_executor(
self.db.link_image_to_prompt,
existing["id"],
pp,
)
skipped += 1
else:
prompt_id = await self._run_in_executor(
self.db.save_prompt,
prompt_text,
"lora-manager", # category
tags,
None, # rating
None, # notes
prompt_hash,
)
if prompt_id:
for pp in all_images:
await self._run_in_executor(
self.db.link_image_to_prompt,
prompt_id,
pp,
)
imported += 1
else:
skipped += 1
except Exception as e:
self.logger.warning(f"Failed to import LoRA {model_name}: {e}")
skipped += 1
# Progress update for every LoRA
progress = int(5 + (90 * (i + 1) / max(total, 1)))
img_count = len(all_images)
status = f"{model_name}"
if img_count:
status += f" ({img_count} images)"
if not existing:
counts["imported"] += 1
status = record["name"] if record else files[index].name
if record and record["images"]:
status += f" ({len(record['images'])} images)"
await send_progress(
{
"type": "progress",
"status": status,
"progress": progress,
"processed": i + 1,
"imported": imported,
"skipped": skipped,
"progress": int(5 + (90 * (index + 1) / max(total, 1))),
"processed": index + 1,
"imported": counts["imported"],
"skipped": index + 1 - counts["imported"],
}
)
# Each record's event is queued (through the loop) before staging
# ends, so reporting until staging is done and then draining the
# queue reports every record in order. I2: when the page goes away
# (its connection closes, or a report cannot be sent) staging stops
# before its next LoRA, and this returns only once it has stopped, so
# the caller's records hold covers every download.
stop = threading.Event()
staging = asyncio.ensure_future(
self.stage_lora_import(
files,
on_record=on_record,
cancelled=lambda: stop.is_set() or disconnected(),
)
)
try:
while not staging.done():
getter = asyncio.ensure_future(events.get())
await asyncio.wait(
{getter, staging}, return_when=asyncio.FIRST_COMPLETED
)
if getter.done():
await report(*getter.result())
else:
getter.cancel()
while not events.empty():
await report(*events.get_nowait())
finally:
stop.set()
# Staging must have stopped before the records hold is released,
# even when the handler is cancelled again while it waits (R3).
interrupted = False
while not staging.done():
try:
await asyncio.shield(staging)
except asyncio.CancelledError:
interrupted = True
except Exception:
pass
if interrupted:
raise asyncio.CancelledError
staged = await staging
if not await self.lora_files_unchanged(lm_path, reviewed):
raise LoraFilesChanged(_FILES_CHANGED)
await send_progress(
{
"type": "progress",
"status": "Replacing previous lora-manager imports...",
"progress": 96,
}
)
result = await self.replace_lora_imports(
self.db, staged, expected_ids=state["ids"], allow_empty=False
)
await send_progress(
{
"type": "complete",
"progress": 100,
"total": total,
"imported": imported,
"skipped": skipped,
"imported": result["imported"],
"skipped": total - result["imported"],
}
)
+173
View File
@@ -0,0 +1,173 @@
"""The staged LoRA reference import (#221, #249 System S10).
The import reads LoraManager's ``*.metadata.json`` files and turns each LoRA
into one prompt of the ``lora-manager`` category, tagged with its name and
trigger words and linked to its preview images. It runs in three stages, and
only the last writes to the database:
1. **discover**: list the metadata files (``discover``) and fingerprint them
(path, size and modification time; ``fingerprint``). ``preview`` adds the
counts a review shows. Nothing is written and nothing is requested.
2. **stage**: read each file and build its record; CivitAI example images are
downloaded into the image cache first (best effort, counted). A cancelled
or failed stage leaves the database untouched; downloaded images stay in
the cache.
3. **replace**: the files are listed and fingerprinted once more (a file
changed, removed or added while staging aborts the import), then
``PromptDatabase.replace_category_prompts`` swaps the old collection for
the staged records in one transaction (or changes nothing).
Both the System job (``POST /lora/import/apply``) and the legacy ``POST
/lora/scan`` run these stages (``py/api/lora_integration.py``).
"""
import hashlib
from dataclasses import dataclass, field
from pathlib import Path
from typing import Callable, Dict, List, Optional
from . import lora_utils
try:
from ..utils.hashing import generate_prompt_hash
except ImportError: # pragma: no cover - flat test imports
from utils.hashing import generate_prompt_hash
LORA_CATEGORY = "lora-manager"
JOB_TITLE = "LoRA reference import"
# The job's outcome groups, in its counter-tile order.
GROUPS = ("imported", "linked", "skipped", "downloaded", "download_errors", "errors")
METADATA_SUFFIX = "*.metadata.json"
class ImportCancelled(Exception):
"""The import was cancelled while staging; nothing was written."""
def discover(lora_manager_path: str) -> List[Path]:
"""Every LoRA metadata file, sorted and without repeats.
Raises whatever listing a folder raised: a discovery that cannot see
every folder must not replace the collection with part of it.
"""
seen = set()
files = []
for folder in lora_utils.find_lora_directories(lora_manager_path):
for path in Path(folder).rglob(METADATA_SUFFIX):
key = str(path)
if key not in seen:
seen.add(key)
files.append(path)
files.sort(key=str)
return files
def fingerprint(files: List[Path]) -> str:
"""What a review saw of ``files``: each path, size and modification time."""
digest = hashlib.sha256()
for path in files:
try:
stat = path.stat()
entry = f"{path}\0{stat.st_size}\0{stat.st_mtime_ns}\n"
except OSError:
entry = f"{path}\0missing\n"
digest.update(entry.encode("utf-8", "surrogatepass"))
return "sha256:" + digest.hexdigest()
def preview(lora_manager_path: str, cache_dir: Path) -> Dict:
"""The review's counts (blocking; reads only, requests nothing).
``{lora_files, with_metadata, planned_downloads, fingerprint, names}``:
the metadata files found, those that parse, the example images an import
would download (not yet in the cache) and the first LoRA names.
"""
files = discover(lora_manager_path)
with_metadata = 0
planned = 0
names = []
for path in files:
metadata = lora_utils.read_lora_metadata(path)
if not metadata:
continue
with_metadata += 1
planned += lora_utils.planned_civitai_downloads(metadata, path, cache_dir)
if len(names) < 50:
names.append(lora_utils.get_model_name_from_metadata(metadata))
return {
"lora_files": len(files),
"with_metadata": with_metadata,
"planned_downloads": planned,
"fingerprint": fingerprint(files),
"names": names,
}
def build_record(metadata: Dict, path: Path, images: List[str]) -> Dict:
"""The prompt a LoRA becomes: its example prompt (else its name), tags, images."""
model_name = lora_utils.get_model_name_from_metadata(metadata)
text = lora_utils.get_example_prompt_from_metadata(metadata) or model_name
tags = ["lora-manager", f"lora:{model_name}"]
tags.extend(lora_utils.get_trigger_words_from_metadata(metadata))
return {
"name": model_name,
"text": text,
"hash": generate_prompt_hash(text),
"tags": list(dict.fromkeys(tags)),
"images": images,
}
@dataclass
class StagedImport:
"""The records an import will write, and what staging counted."""
total: int
records: List[Dict] = field(default_factory=list)
skipped: List[str] = field(default_factory=list)
stats: Dict[str, int] = field(
default_factory=lambda: {"downloaded": 0, "download_errors": 0}
)
def stage(
files: List[Path],
cache_dir: Path,
api_key: str = "",
*,
on_record: Optional[Callable[[int, Dict, "StagedImport"], None]] = None,
cancelled: Callable[[], bool] = lambda: False,
logger=None,
) -> StagedImport:
"""Read each file and build its record, downloading its example images.
Blocking. A file that cannot be read, or whose record fails to build, is
skipped by its name. ``cancelled()`` is asked before each file (raises
ImportCancelled); ``on_record(index, record_or_None, staged)`` follows
each one.
"""
staged = StagedImport(total=len(files))
for index, path in enumerate(files):
if cancelled():
raise ImportCancelled()
record = None
metadata = lora_utils.read_lora_metadata(path)
if not metadata:
staged.skipped.append(path.name)
else:
try:
local = lora_utils.get_preview_images_from_metadata(metadata, path)
remote = lora_utils.download_civitai_images(
metadata, path, cache_dir, api_key, stats=staged.stats
)
images = list(dict.fromkeys([*local, *remote]))
record = build_record(metadata, path, images)
staged.records.append(record)
except Exception as error: # one LoRA never stops the import
name = lora_utils.get_model_name_from_metadata(metadata)
if logger is not None:
logger.warning(f"Failed to prepare LoRA {name}: {error}")
staged.skipped.append(name)
if on_record is not None:
on_record(index, record, staged)
return staged
+151 -37
View File
@@ -82,23 +82,46 @@ def detect_lora_manager(custom_path: str = "") -> Optional[str]:
if p.is_dir() and _looks_like_lora_manager(p):
return str(p.resolve())
# 2. Auto-detect via custom_nodes (case-insensitive scan)
# 2. Auto-detect in every custom_nodes folder (case-insensitive scan):
# ComfyUI's own first, then the extra ones it registered (#144).
folders = []
root = find_comfyui_root()
if root:
custom_nodes = root / "custom_nodes"
if custom_nodes.is_dir():
for entry in custom_nodes.iterdir():
if (
entry.is_dir()
and "lora" in entry.name.lower()
and "manager" in entry.name.lower()
and _looks_like_lora_manager(entry)
):
return str(entry.resolve())
folders.append(root / "custom_nodes")
folders.extend(custom_nodes_directories())
seen = set()
for custom_nodes in folders:
key = os.path.normcase(os.path.abspath(str(custom_nodes)))
if key in seen:
continue
seen.add(key)
found = _lora_manager_in(Path(custom_nodes))
if found:
return found
return None
def _lora_manager_in(custom_nodes: Path) -> Optional[str]:
"""The LoraManager install directly under ``custom_nodes``, or None."""
try:
entries = sorted(custom_nodes.iterdir()) if custom_nodes.is_dir() else []
except OSError:
return None
for entry in entries:
try:
if (
entry.is_dir()
and "lora" in entry.name.lower()
and "manager" in entry.name.lower()
and _looks_like_lora_manager(entry)
):
return str(entry.resolve())
except OSError:
continue
return None
def _looks_like_lora_manager(path: Path) -> bool:
"""Heuristic: does this directory look like a LoraManager install?"""
# Must have __init__.py (ComfyUI extension) or README.md
@@ -491,47 +514,98 @@ def _download_one(url: str, local_path: Path, api_key: str) -> Optional[str]:
from PIL import Image
img = Image.open(BytesIO(raw))
img.thumbnail((_THUMB_MAX_SIZE, _THUMB_MAX_SIZE), Image.LANCZOS)
img = img.convert("RGB")
img.save(str(local_path), "JPEG", quality=85)
with Image.open(BytesIO(raw)) as img:
img.thumbnail((_THUMB_MAX_SIZE, _THUMB_MAX_SIZE), Image.LANCZOS)
rgb = img.convert("RGB")
_save_atomically(rgb, local_path)
return str(local_path.resolve())
except Exception as e:
logger.debug(f"Failed to download {url}: {e}")
return None
def download_civitai_images(
metadata: Dict, metadata_path: Path, cache_dir: Path, api_key: str = ""
) -> List[str]:
"""Download civitai example images to a local cache directory.
_PART_PREFIX = ".pm-"
_PART_SUFFIX = ".part"
# A temp download older than this was left by a killed process, not a live one
# (each download has a 10 s timeout).
STALE_PART_SECONDS = 3600
Uses thumbnail URLs (512px) instead of full-size originals, and
downloads in parallel (up to 8 concurrent) for speed.
Args:
api_key: CivitAI API key for authenticated downloads (NSFW content).
def _save_atomically(image, local_path: Path) -> None:
"""Write ``image`` as JPEG so ``local_path`` only ever names a whole file.
Returns:
List of absolute paths to downloaded image files.
The image goes to a ``.pm-*.part`` file in the same folder, which is
closed before ``os.replace`` puts it in place (Windows cannot replace an
open file); a failure removes the temp file. A reader (another import's
cache check) never sees a half-written image. The temp file is opened
like ``img.save`` opens a file, so the image keeps the umask's mode.
"""
from concurrent.futures import ThreadPoolExecutor
import secrets
while True:
temp = local_path.parent / (
f"{_PART_PREFIX}{secrets.token_hex(8)}{_PART_SUFFIX}"
)
try:
handle = open(temp, "xb")
except FileExistsError:
continue
break
try:
with handle:
image.save(handle, "JPEG", quality=85)
os.replace(temp, local_path)
except BaseException:
try:
os.remove(temp)
except OSError:
pass
raise
def sweep_stale_downloads(cache_dir: Path) -> None:
"""Remove ``.pm-*.part`` files a killed download left in the image cache.
Only that exact pattern, only in ``cache_dir`` and its LoRA folders, and
only once older than ``STALE_PART_SECONDS``; a file that cannot be read
or removed is left for a later import.
"""
import time
cutoff = time.time() - STALE_PART_SECONDS
try:
folders = [cache_dir] + [p for p in cache_dir.iterdir() if p.is_dir()]
except OSError:
return
for folder in folders:
try:
candidates = list(folder.glob(f"{_PART_PREFIX}*{_PART_SUFFIX}"))
except OSError:
continue
for path in candidates:
try:
if path.is_file() and path.stat().st_mtime < cutoff:
os.remove(path)
except OSError:
logger.debug(f"Could not remove stale download {path}")
def _civitai_cache_plan(
metadata: Dict, metadata_path: Path, cache_dir: Path
) -> Tuple[Path, List[str], List[Tuple[str, Path]]]:
"""``(lora_cache, cached, tasks)`` for a LoRA's CivitAI example images.
``cached`` are the images already in the cache, ``tasks`` the
``(url, local_path)`` downloads still needed. Reads only; creates nothing.
"""
civitai = _get_civitai(metadata)
images = civitai.get("images", []) or []
if not images:
return []
file_name = metadata.get("file_name", "")
if not file_name:
file_name = metadata_path.name.replace(".metadata.json", "")
lora_stem = Path(file_name).stem
lora_cache = cache_dir / Path(file_name).stem
lora_cache = cache_dir / lora_stem
lora_cache.mkdir(parents=True, exist_ok=True)
# Build download tasks
cached = []
tasks = [] # (url, local_path)
for img in images:
@@ -548,9 +622,42 @@ def download_civitai_images(
cached.append(str(local_path.resolve()))
else:
tasks.append((url, local_path))
return lora_cache, cached, tasks
def planned_civitai_downloads(
metadata: Dict, metadata_path: Path, cache_dir: Path
) -> int:
"""How many example images an import would download (no request, no write)."""
return len(_civitai_cache_plan(metadata, metadata_path, cache_dir)[2])
def download_civitai_images(
metadata: Dict,
metadata_path: Path,
cache_dir: Path,
api_key: str = "",
stats: Optional[Dict[str, int]] = None,
) -> List[str]:
"""Download civitai example images to a local cache directory.
Uses thumbnail URLs (512px) instead of full-size originals, and
downloads in parallel (up to 8 concurrent) for speed.
Args:
api_key: CivitAI API key for authenticated downloads (NSFW content).
stats: When given, ``downloaded`` and ``download_errors`` are
increased by the downloads that succeeded and failed.
Returns:
List of absolute paths to downloaded image files.
"""
from concurrent.futures import ThreadPoolExecutor
lora_cache, cached, tasks = _civitai_cache_plan(metadata, metadata_path, cache_dir)
if not tasks:
return cached
lora_cache.mkdir(parents=True, exist_ok=True)
# Download in parallel
downloaded = []
@@ -563,15 +670,21 @@ def download_civitai_images(
if result:
downloaded.append(result)
if stats is not None:
stats["downloaded"] = stats.get("downloaded", 0) + len(downloaded)
stats["download_errors"] = (
stats.get("download_errors", 0) + len(tasks) - len(downloaded)
)
return cached + downloaded
def get_lora_image_cache_dir() -> Path:
def get_lora_image_cache_dir(create: bool = True) -> Path:
"""Get the directory used to cache downloaded LoRA example images.
``<data folder>/data/lora_images`` (``utils.library_location``): the node
folder for a library that was not moved, else ComfyUI's
``user/__promptmanager``.
``user/__promptmanager``. ``create=False`` only names it (a preview
writes nothing).
"""
try:
from ..utils import library_location
@@ -582,7 +695,8 @@ def get_lora_image_cache_dir() -> Path:
cache = Path(
library_location.cache_dir("lora_images", extension_root=str(ext_root))
)
cache.mkdir(parents=True, exist_ok=True)
if create:
cache.mkdir(parents=True, exist_ok=True)
return cache
+4
View File
@@ -23,6 +23,9 @@ from tests.browser.request_hold import register_request_holds
from tests.browser import slow_reset_watch
from tests.browser.workspace_activity_fixture import register_activity_commands
from tests.browser.workspace_system_fixture import register_system_commands
from tests.browser.workspace_integrations_fixture import (
register_integrations_commands,
)
from tests.browser.workspace_concept_fixture import (
MAX_EXTRA_IMAGES as CONCEPT_MAX_EXTRA_IMAGES,
MAX_SEEDLESS_IMAGES as CONCEPT_MAX_SEEDLESS_IMAGES,
@@ -203,6 +206,7 @@ def create_app(root: Path) -> web.Application:
reveal_stub = register_reveal_stub(routes, api, root)
register_activity_commands(app, routes, api)
register_system_commands(app, routes, api, root)
register_integrations_commands(app, routes, api, root)
request_holds = register_request_holds(app, routes)
slow_reset_watch.install(app, api)
@@ -0,0 +1,343 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from './fixtures.mjs';
import { SIZES, THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate } from './concept-inventory.mjs';
// #249 System S10 Task 28: System › Integrations (app.js renderSystem 682–684) and the LoRA import review against the
// immutable concept, on the concept-shaped fixture with a fake LoraManager install. Each concept measurement is mapped
// only by the recorded deviations (D106, D107, D114), each commented where it applies; after that the difference must
// be empty, and a mutated label, option, row or token must be noticed. The concept starts its sample import without a
// review, so the import review is compared with the concept's own record-changing review, `maintenancePlan`
// (953–958), whose shell, grid, note and actions it shares.
test.use({ locale: 'en-US', timezoneId: 'UTC' });
test.beforeEach(async ({ page, request, browserName }) => {
await startCoverage(page, browserName);
await seedConcept(request);
await expect(await request.post('/__test__/system/reset')).toBeOK();
await expect(await request.post('/__test__/integrations/reset')).toBeOK();
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
page.conceptErrors = [];
page.on('pageerror', error => page.conceptErrors.push(error.message));
});
test.afterEach(async ({ page, request, browserName }) => {
await saveCoverage(page, browserName);
await request.post('/__test__/jobs/restart');
await request.post('/__test__/integrations/reset');
await request.post('/__test__/system/reset');
expect(page.conceptErrors).toEqual([]);
});
const PREVIEW_URL = '/prompt_manager/lora/import/preview';
/** Runs in the page: Integrations' surfaces under `mainSelector`, every row's text and control, geometry and tokens. */
function measureIntegrations(mainSelector) {
const layout = document.querySelector(`${mainSelector} .settings-layout`);
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
const text = node => node?.textContent.trim() ?? null;
const control = row => {
const select = row.querySelector(':scope > select');
if (select) return { kind: 'select', id: select.id, options: [...select.options].map(option => [option.textContent, option.disabled]),
selected: select.selectedOptions[0]?.textContent ?? null, disabled: select.disabled };
const input = row.querySelector(':scope > input');
if (input) return { kind: `input.${input.type}`, id: input.id, value: input.value, readOnly: input.readOnly };
const toggle = row.querySelector(':scope > * input[type="checkbox"]');
return toggle ? { kind: 'switch', id: toggle.id, wrapper: name(toggle.parentElement), checked: toggle.checked,
disabled: toggle.disabled, ...css(toggle.parentElement, 'width', 'height') } : null;
};
const surfaces = [...layout.querySelectorAll(':scope > section.surface')];
const firstRow = surfaces[0].querySelector('.setting-row');
const note = layout.querySelector('.setting-row p.small');
return {
layout: [...layout.children].map(name),
surfaces: surfaces.map(surface => ({ tree: [...surface.children].map(name), heading: text(surface.querySelector(':scope > h2')),
intro: text(surface.querySelector(':scope > p')),
rows: [...surface.querySelectorAll(':scope > .setting-row')].map(row => ({ tree: [...row.children].map(name),
text: [...row.firstElementChild.children].map(node => [name(node), text(node), node.localName === 'label' ? node.htmlFor : null]),
control: control(row) })),
operations: [...surface.querySelectorAll(':scope > .operation')].map(row => ({ tree: [...row.children].map(name),
title: text(row.querySelector('h3')), description: text(row.querySelector('p')),
actions: [...row.querySelectorAll('.row-actions > button')].map(item => [text(item), item.className]) })) })),
surface: css(surfaces[1], 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius',
'marginTop'),
row: css(firstRow, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', 'borderTopWidth',
'borderTopColor', 'marginTop'),
label: css(firstRow.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
description: css(firstRow.querySelector('p'), 'fontSize', 'color', 'marginTop', 'maxWidth'),
select: css(firstRow.querySelector('select'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor', 'color', 'borderTopColor'),
password: css(layout.querySelector('.setting-row input[type="password"]'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor',
'color', 'borderTopColor'),
// The concept's `.small` text (its Save row status); production's row notes use the same class.
small: css(layout.querySelector('#settings-status'), 'fontSize', 'color'),
note: note && css(note, 'fontSize', 'color'),
// Whether each select is as wide as the browser lays out a copy holding only its chosen option, i.e. it shows
// that option whole (the browser's own select sizing, arrow included).
selectsFit: [...layout.querySelectorAll('.setting-row > select')].map(select => {
const copy = select.cloneNode(false);
copy.removeAttribute('id');
copy.append(new Option(select.selectedOptions[0]?.textContent ?? ''));
Object.assign(copy.style, { position: 'absolute', visibility: 'hidden', width: 'max-content', minWidth: '0',
maxWidth: 'none', flex: 'none' });
select.parentElement.append(copy);
const needed = copy.getBoundingClientRect().width;
copy.remove();
return select.getBoundingClientRect().width + 0.5 >= needed;
}),
pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth,
};
}
const SEND_REASON = 'Sending always asks for the ComfyUI window and node, so “Ask each time” is the only option.';
const RENDERER_REASON = 'Prompt Manager nodes always follow ComfyUI’s own renderer setting.';
const RENDERER_NONE = 'No ComfyUI window is open in this browser, so its renderer is not known.';
const RETURN_REASON = 'Not available: sending applies the text in ComfyUI and keeps this page open.';
const PATH_DESCRIPTION = 'Found automatically in ComfyUI’s custom_nodes folders; read-only, as on the existing settings page.';
/**
* The concept's Integrations with the recorded deviations applied, for the fixture's LoraManager (detected, disabled,
* no key, trigger words off and not loaded).
*/
function mapIntegrationsConcept(reference) {
const [comfy, lora] = reference.surfaces;
const small = text => ['p.small', text, null];
const firstOnly = row => ({ ...row, control: { ...row.control, options: row.control.options.map(([label], index) => [label, index > 0]) } });
const [send, renderer, back] = comfy.rows;
const [enable, triggers, key] = lora.rows;
return {
...reference,
surfaces: [
{ ...comfy, rows: [
// D106: only "Ask each time" can be chosen, and the row says why.
{ ...firstOnly(send), text: [...send.text, small(SEND_REASON)] },
// D106: only "Follow host renderer" can be chosen; the reason, then the detected renderer.
{ ...firstOnly(renderer), text: [...renderer.text, small(RENDERER_REASON), small(RENDERER_NONE)] },
// D106: Return to canvas is not available: off, disabled and saying why.
{ ...back, text: [...back.text, small(RETURN_REASON)], control: { ...back.control, checked: false, disabled: true } },
] },
{ ...lora,
// D106: the LoraManager path row (legacy, read-only) follows the integration toggle.
tree: [lora.tree[0], lora.tree[1], 'div.setting-row', 'div.setting-row', 'div.setting-row', 'div.setting-row', 'div.operation'],
rows: [
// The real detection replaces the concept's sample line; the integration is off in the fixture.
{ ...enable, text: [enable.text[0], ['p', 'Detected: custom_nodes/ComfyUI-Lora-Manager.', null]],
control: { ...enable.control, checked: false } },
{ tree: ['div', 'input'],
text: [['label', 'LoraManager path', 'lora-path'], ['p', PATH_DESCRIPTION, null]],
control: { kind: 'input.text', id: 'lora-path', value: 'custom_nodes/ComfyUI-Lora-Manager', readOnly: true } },
// The trigger cache is a separate fact from the setting: its own line.
{ ...triggers, text: [...triggers.text, small('Trigger words are not loaded.')], control: { ...triggers.control, checked: false } },
// D114: no prototype warning; the saved-key line under it.
// (No key is saved in the fixture, so "Clear saved key" is not offered.)
{ ...key, text: [key.text[0], ['p', 'Credentials stay in the server configuration.', null], small('No key saved.')],
control: { ...key.control, value: '' } },
],
// D107: the import is reviewed first ("Review import"; the concept's button only simulated a job).
operations: lora.operations.map(operation => ({ ...operation, actions: [['Review import', '']] })) },
],
// D106: both selects show their whole chosen option at every width (at 390 the concept cuts "Follow host
// renderer"; production lets the two selects grow to 160px and never shrink for the reason lines).
selectsFit: reference.selectsFit.map(() => true),
select: reference.select.maxWidth === '120px' ? { ...reference.select, maxWidth: '160px' } : reference.select,
// The D106/D114 row notes are `p.small` inside the row text, so they read as its description (`.setting-row p`).
note: { fontSize: reference.description.fontSize, color: reference.description.color },
};
}
async function conceptIntegrations(page, theme) {
await openConcept(page, 'system', theme);
await page.locator('#main .section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
await expect(page.locator('#main .settings-layout > section.surface')).toHaveCount(2);
await page.mouse.move(0, 0);
return page.evaluate(measureIntegrations, '#main');
}
async function productionIntegrations(page, theme) {
await openProduction(page, 'system/integrations', theme);
// Compare once the LoRA state and the renderer check answered.
await expect(page.locator('#workspace-main #lora-path')).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
await expect(page.locator('#workspace-main #renderer-status')).toHaveText(RENDERER_NONE);
await page.mouse.move(0, 0);
return page.evaluate(measureIntegrations, '#workspace-main');
}
const fakeLoraManager = async (request, data = {}) =>
expect(await request.post('/__test__/integrations/lora-manager', { data })).toBeOK();
for (const theme of THEMES) for (const [width, height] of SIZES) {
test(`integrations matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
await page.setViewportSize({ width, height });
await fakeLoraManager(request);
const concept = await conceptIntegrations(page, theme);
await page.screenshot({ path: testInfo.outputPath(`concept-integrations-${theme}-${width}.png`), fullPage: true });
const production = await productionIntegrations(page, theme);
await page.screenshot({ path: testInfo.outputPath(`production-integrations-${theme}-${width}.png`), fullPage: true });
expect(diffInventory(mapIntegrationsConcept(concept), production)).toEqual([]);
expect(production.pageScroll).toBe(false);
});
}
// D106 below the concept's smallest size: on a 320px phone both selects still show their whole option, without page
// scroll (S10 re-review N9).
for (const theme of THEMES) test(`integrations selects fit at 320 wide ${theme}`, async ({ page, request }) => {
await page.setViewportSize({ width: 320, height: 700 });
await fakeLoraManager(request);
const production = await productionIntegrations(page, theme);
expect(production.selectsFit).toEqual([true, true]);
expect(production.pageScroll).toBe(false);
});
test('integrations comparison detects mutations', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await fakeLoraManager(request);
const concept = await conceptIntegrations(page, 'dark');
const expected = mapIntegrationsConcept(concept);
const production = await productionIntegrations(page, 'dark');
expect(diffInventory(expected, production)).toEqual([]);
// Unmapped, the section differs only where D106, D107 and D114 are recorded.
expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 4).join('.'))
.filter((path, index, all) => all.indexOf(path) === index).sort()).toEqual([
'note', 'note.color', 'note.fontSize', 'surfaces.0.rows.0', 'surfaces.0.rows.1', 'surfaces.0.rows.2', 'surfaces.1.operations.0',
'surfaces.1.rows.0', 'surfaces.1.rows.1', 'surfaces.1.rows.2', 'surfaces.1.rows.3', 'surfaces.1.rows.length',
'surfaces.1.tree.5', 'surfaces.1.tree.6', 'surfaces.1.tree.length'].sort());
const measure = () => page.evaluate(measureIntegrations, '#workspace-main');
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
// A removed row.
const undoRow = await mutate(page, { remove: ['#workspace-main section.surface:nth-of-type(2) > .setting-row:nth-of-type(3)'] });
expect(await paths()).toEqual(expect.arrayContaining(['surfaces.1.rows.length', 'surfaces.1.tree.length']));
await undoRow();
// A changed label and a changed option.
await page.evaluate(() => { document.querySelector('#workspace-main label[for="renderer"]').textContent = 'Renderer'; });
expect(await paths()).toEqual(['surfaces.0.rows.1.text.0.1']);
await page.evaluate(() => { document.querySelector('#workspace-main label[for="renderer"]').textContent = 'Host compatibility'; });
await page.evaluate(() => { document.querySelector('#workspace-main #send-destination option').textContent = 'Always ask'; });
expect(await paths()).toEqual(expect.arrayContaining(['surfaces.0.rows.0.control.options.0.0']));
await page.evaluate(() => { document.querySelector('#workspace-main #send-destination option').textContent = 'Ask each time'; });
// An unsupported option that became selectable.
await page.evaluate(() => { document.querySelectorAll('#workspace-main #renderer option')[1].disabled = false; });
expect(await paths()).toEqual(['surfaces.0.rows.1.control.options.1.1']);
await page.evaluate(() => { document.querySelectorAll('#workspace-main #renderer option')[1].disabled = true; });
// A select squeezed until its chosen option is cut (S10 review M3).
const undoSqueeze = await mutate(page, { css: '#workspace-main #renderer { max-width: 70px !important; }' });
expect(await paths()).toEqual(['selectsFit.1']);
await undoSqueeze();
// A swapped token.
const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' });
expect(await paths()).toEqual(expect.arrayContaining(['row.borderTopColor']));
await undoToken();
expect(diffInventory(expected, await measure())).toEqual([]);
});
/** Runs in the page: the open review dialog: its title, body and the concept's tokens. */
function measureReview() {
const dialog = document.querySelector('dialog#modal[open]');
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
const text = node => node?.textContent.trim() ?? null;
const body = dialog.querySelector('#modal-body');
const grid = body.querySelector(':scope > dl.meta-grid');
const note = [...body.querySelectorAll(':scope > p.small')].at(-1);
const actions = body.querySelector(':scope > .dialog-actions');
return {
title: text(dialog.querySelector('#modal-title')),
tree: [...body.children].map(name),
heading: { text: text(body.querySelector(':scope > h3')),
...css(body.querySelector(':scope > h3'), 'fontSize', 'fontWeight', 'color', 'marginBottom') },
grid: { rows: [...grid.children].map(row => [text(row.querySelector('dt')), text(row.querySelector('dd'))]),
...css(grid, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap'),
dt: css(grid.querySelector('dt'), 'fontSize', 'color', 'textTransform'), dd: css(grid.querySelector('dd'), 'fontSize', 'color') },
details: body.querySelector(':scope > details') && { summary: text(body.querySelector(':scope > details > summary')),
tree: [...body.querySelector(':scope > details').children].map(name) },
backup: body.querySelector(':scope > .notice') && { tree: [...body.querySelector(':scope > .notice').children].map(name),
text: [...body.querySelectorAll(':scope > .notice > div:first-child > *')].map(node => [node.localName, text(node)]),
buttons: [...body.querySelectorAll(':scope > .notice button')].map(text) },
note: { text: text(note), ...css(note, 'fontSize', 'color', 'marginTop') },
actions: { tree: [...actions.children].map(name), ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
buttons: [...actions.children].map(item => ({ text: text(item), className: item.className,
...css(item, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') })) },
};
}
/**
* D107: the server's import plan in the concept's review: "Review LoRA import" with "LoRA reference data" as its `h3`,
* the concept's four rows with the plan's values plus Found and Network, the LoRAs it found (`details`), the hold
* line with the cache sentence in the concept's `p.small`, the D55 backup box, and Cancel / "Start import".
*/
function mapImportConcept(reference, plan) {
const [cancel, start] = reference.actions.buttons;
return {
...reference,
title: 'Review LoRA import',
tree: ['h3', 'dl.meta-grid', 'details', 'p.small', 'div.notice', 'div.dialog-actions'],
heading: { ...reference.heading, text: 'LoRA reference data' },
grid: { ...reference.grid, rows: [['Scope', plan.scope.label], ['Found', '2 metadata files · 2 readable'],
['Protected', 'Nothing is protected'], ['Network', 'Downloads up to 2 example images from CivitAI'],
['Backup', `Create and verify before changes (${plan.backup.destination_label})`],
['Media on disk', 'No files are removed; image links are replaced']] },
details: { summary: 'LoRAs found (2)', tree: ['summary', 'pre'] },
backup: { tree: ['div', 'div.row-actions'], text: [['h3', 'Back up the database first (recommended)'],
['p', 'Downloads a copy of the prompts database before this operation changes any records.']], buttons: ['Back up now'] },
note: { ...reference.note, text: 'Nothing changes until you apply. This review holds for 15 minutes. '
+ 'Example images downloaded before a failure stay in the image cache.' },
actions: { ...reference.actions, buttons: [cancel, { ...start, text: 'Start import' }] },
};
}
async function conceptReview(page, theme) {
await openConcept(page, 'system', theme);
await page.locator('#main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click();
await page.locator('#main [data-plan="duplicate prompts"]').click();
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review maintenance plan');
await page.mouse.move(0, 0);
return page.evaluate(measureReview);
}
async function productionReview(page, request, theme) {
await openProduction(page, 'system/integrations', theme);
await expect(page.locator('#workspace-main #lora-path')).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
// The renderer line above settles first: its new text moves the rows below (a click must not land mid-shift).
await expect(page.locator('#workspace-main #renderer-status')).toHaveText(RENDERER_NONE);
// The import reviews what is enabled: save the integration on first.
await page.locator('#workspace-main #lora-enable').check();
await page.getByRole('button', { name: 'Save preferences', exact: true }).click();
await expect(page.locator('#settings-status')).toHaveText('Saved to the server configuration.');
const answer = page.waitForResponse(response => response.url().endsWith(PREVIEW_URL));
await page.locator('#workspace-main').getByRole('button', { name: 'Review import', exact: true }).click();
const plan = await (await answer).json();
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review LoRA import');
await page.mouse.move(0, 0);
return { plan, production: await page.evaluate(measureReview) };
}
for (const theme of THEMES) for (const [width, height] of SIZES) {
test(`LoRA import review matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
await page.setViewportSize({ width, height });
await fakeLoraManager(request, { loras: 2, civitai: true });
const concept = await conceptReview(page, theme);
expect(concept.tree).toEqual(['p.demo-note', 'h3', 'dl.meta-grid', 'p.small', 'div.dialog-actions']);
await page.screenshot({ path: testInfo.outputPath(`concept-plan-review-${theme}-${width}.png`) });
const { plan, production } = await productionReview(page, request, theme);
await page.screenshot({ path: testInfo.outputPath(`production-lora-review-${theme}-${width}.png`) });
expect(diffInventory(mapImportConcept(concept, plan), production)).toEqual([]);
});
}
test('LoRA import review comparison detects mutations', async ({ page, request }) => {
await page.setViewportSize({ width: 1600, height: 1100 });
await fakeLoraManager(request, { loras: 2, civitai: true });
const concept = await conceptReview(page, 'dark');
const { plan } = await productionReview(page, request, 'dark');
const expected = mapImportConcept(concept, plan);
const paths = async () => diffInventory(expected, await page.evaluate(measureReview)).map(item => item.path);
expect(await paths()).toEqual([]);
await page.evaluate(() => { document.querySelectorAll('dialog#modal[open] dl.meta-grid dt')[3].textContent = 'Downloads'; });
expect(await paths()).toEqual(['grid.rows.3.0']);
await page.evaluate(() => { document.querySelectorAll('dialog#modal[open] dl.meta-grid dt')[3].textContent = 'Network'; });
const undoGrid = await mutate(page, { css: 'dialog#modal dl.meta-grid { column-gap: 3px; }' });
expect(await paths()).toEqual(expect.arrayContaining(['grid.columnGap']));
await undoGrid();
const undoBox = await mutate(page, { remove: ['dialog#modal[open] #modal-body > .notice'] });
expect(await paths()).toEqual(expect.arrayContaining(['backup', 'tree.length']));
await undoBox();
expect(await paths()).toEqual([]);
});
@@ -0,0 +1,432 @@
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
import { test, expect } from './fixtures.mjs';
import { seedConcept } from './concept-inventory.mjs';
// System › Integrations (#249 System S10 Task 27) on the concept-shaped fixture with a fake LoraManager install
// (workspace_integrations_fixture.py): the D106 ComfyUI rows and renderer status, the LoRA rows and the D114 key, saved
// through the one Save row, and the D107 import review run as an Activity job. No test reaches the network: CivitAI
// downloads are answered by the fixture's stub opener.
const workspace = '/prompt_manager/workspace';
const PREVIEW_URL = '/prompt_manager/lora/import/preview';
const APPLY_URL = '/prompt_manager/lora/import/apply';
const ENABLE_URL = '/prompt_manager/lora/enable';
const KEPT = 'The import failed; the previous LoRA imports were kept. Images already downloaded stay in the image cache.';
test.beforeEach(async ({ page, request, browserName }) => {
await startCoverage(page, browserName);
await seedConcept(request);
await expect(await request.post('/__test__/system/reset')).toBeOK();
await expect(await request.post('/__test__/integrations/reset')).toBeOK();
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
page.systemErrors = [];
page.on('pageerror', error => page.systemErrors.push(error.message));
});
test.afterEach(async ({ page, request, browserName }) => {
await saveCoverage(page, browserName);
await request.post('/__test__/integrations/gate', { data: { hold: false } });
await request.post('/__test__/release');
await request.post('/__test__/jobs/restart');
await request.post('/__test__/integrations/reset');
await request.post('/__test__/system/reset');
expect(page.systemErrors).toEqual([]);
});
const toast = page => page.locator('#workspace-toast');
const layoutOf = page => page.locator('#workspace-main .settings-layout');
const dialog = page => page.locator('dialog#modal[open]');
const status = page => page.locator('#settings-status');
const save = page => page.getByRole('button', { name: 'Save preferences', exact: true });
const settingOf = (page, label) => layoutOf(page).locator('.setting-row').filter({ has: page.locator(`label:text-is("${label}")`) });
async function serverJson(request, url) {
const response = await request.get(url);
await expect(response).toBeOK();
return response.json();
}
async function lora(request, data = {}) {
await expect(await request.post('/__test__/integrations/lora-manager', { data })).toBeOK();
}
async function openIntegrations(page) {
// A fresh page each time: the drafts live for the page session (Q10), and a hash-only goto would not reload.
if (page.url().endsWith('#/system/integrations')) await page.reload();
else await page.goto(`${workspace}#/system/integrations`);
await expect(layoutOf(page).locator(':scope > section.surface')).toHaveCount(2);
// The LoRA rows are painted once the server's LoRA state was read.
await expect(settingOf(page, 'LoraManager integration').locator('input')).toBeEnabled();
// The renderer line settles first: its new text moves the rows below, so no click lands mid-shift.
await expect(settingOf(page, 'Host compatibility').getByRole('status')).not.toHaveText('Looking for open ComfyUI windows…');
return layoutOf(page);
}
/** Enable the integration (and optionally its trigger words) through the Save row. */
async function enable(page, { triggers = false } = {}) {
await settingOf(page, 'LoraManager integration').locator('input').check();
if (triggers) await settingOf(page, 'Automatically include trigger words').locator('input').check();
await save(page).click();
await expect(status(page)).toHaveText('Saved to the server configuration.');
}
/** The open review's `[dt, dd]` rows. */
const gridOf = page => dialog(page).locator('dl.meta-grid > div').evaluateAll(items => items.map(item =>
[item.querySelector('dt').textContent, item.querySelector('dd').textContent]));
async function ended(request, id) {
await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).not.toBe('running');
return (await serverJson(request, `/prompt_manager/jobs/${id}`)).job;
}
/** A second page of this origin that serves a fake ComfyUI window on the send bridge. */
async function fakeComfyWindow(context, renderer) {
const host = await context.newPage();
await host.goto(`${workspace}#/overview`);
await host.evaluate(async drawn => {
const { createBridgeHost } = await import('/prompt_manager/js/shared/toolkit/prompt-bridge.js');
window.fakeHost = createBridgeHost({ namespace: '/', origin: location.origin, label: 'PromptManager — ComfyUI',
adapter: { listTargets: () => (drawn ? [{ id: '7', label: 'Prompt Manager', type: 'PromptManager', roles: ['positive'],
renderer: drawn }] : []), preview() { throw new Error('unused'); }, apply() { throw new Error('unused'); } } });
}, renderer);
return host;
}
test('send destination offers Ask each time; unsupported options disabled with the reason (D106)', async ({ page }) => {
const layout = await openIntegrations(page);
expect(await layout.locator(':scope > section.surface > h2').allTextContents()).toEqual(['ComfyUI', 'LoRA & resources']);
const send = settingOf(page, 'Default send destination');
expect(await send.locator('option').evaluateAll(options => options.map(option => [option.textContent, option.disabled])))
.toEqual([['Ask each time', false], ['Selected Prompt Manager node', true], ['Create a Prompt Manager (+/-) node', true]]);
await expect(send.locator('select')).toHaveValue('Ask each time');
await expect(send.locator('p.small')).toHaveText(
'Sending always asks for the ComfyUI window and node, so “Ask each time” is the only option.');
const renderer = settingOf(page, 'Host compatibility');
expect(await renderer.locator('option').evaluateAll(options => options.map(option => [option.textContent, option.disabled])))
.toEqual([['Follow host renderer', false], ['Classic', true], ['Nodes 2.0', true]]);
await expect(renderer.locator('p.small').first()).toHaveText('Prompt Manager nodes always follow ComfyUI’s own renderer setting.');
const back = settingOf(page, 'Return to canvas after send');
await expect(back.locator('input')).toBeDisabled();
await expect(back.locator('input')).not.toBeChecked();
await expect(back.locator('p.small')).toHaveText('Not available: sending applies the text in ComfyUI and keeps this page open.');
// Nothing on this surface is ever drafted: the Save row stays clean.
await expect(status(page)).toHaveText('No unsaved changes.');
await expect(save(page)).toHaveAttribute('aria-disabled', 'true');
});
test('host compatibility shows the detected renderer as status (D106)', async ({ page, context }) => {
await openIntegrations(page);
const line = settingOf(page, 'Host compatibility').getByRole('status');
await expect(line).toHaveText('No ComfyUI window is open in this browser, so its renderer is not known.');
const host = await fakeComfyWindow(context, 'nodes2');
try {
await page.reload();
await expect(line).toHaveText('Detected: Nodes 2.0 (1 ComfyUI window).');
await host.evaluate(() => window.fakeHost.dispose());
await host.evaluate(async () => {
const { createBridgeHost } = await import('/prompt_manager/js/shared/toolkit/prompt-bridge.js');
window.fakeHost = createBridgeHost({ namespace: '/', origin: location.origin, label: 'PromptManager — ComfyUI',
adapter: { listTargets: () => [], preview() {}, apply() {} } });
});
await page.reload();
await expect(line).toHaveText('ComfyUI is open; add a Prompt Manager node to see which renderer it uses.');
} finally {
await host.close();
}
});
test('LoRA rows show detected / enabled / cache-loaded separately', async ({ page, request }) => {
await openIntegrations(page);
const integration = settingOf(page, 'LoraManager integration');
const triggers = settingOf(page, 'Automatically include trigger words');
await expect(integration.locator('p').first()).toHaveText('Not detected in ComfyUI’s custom_nodes folders.');
await expect(integration.locator('input')).not.toBeChecked();
await expect(triggers.locator('p.small')).toHaveText('Trigger words are not loaded.');
// Enabling what is not installed is refused by the server: the toggle stays highlighted, nothing was saved.
await integration.locator('input').check();
await save(page).click();
await expect(status(page)).toHaveText('Some changes were not saved. Review the highlighted settings.');
await expect(integration.locator('input')).toHaveAttribute('aria-invalid', 'true');
expect((await serverJson(request, '/__test__/integrations/state')).enabled).toBe(false);
await lora(request);
await openIntegrations(page);
await expect(integration.locator('p').first()).toHaveText('Detected: custom_nodes/ComfyUI-Lora-Manager.');
await expect(integration.locator('input')).not.toBeChecked();
await enable(page, { triggers: true });
await expect(integration.locator('input')).toBeChecked();
await expect(triggers.locator('p.small')).toHaveText('Trigger words are loaded.');
expect(await serverJson(request, '/__test__/integrations/state')).toMatchObject({ enabled: true, trigger_words: true });
// A reload reads the saved state back.
await openIntegrations(page);
await expect(integration.locator('input')).toBeChecked();
await expect(triggers.locator('input')).toBeChecked();
await expect(status(page)).toHaveText('No unsaved changes.');
});
test('LoraManager path row shows the detected path read-only, as legacy (D106)', async ({ page, request }) => {
await openIntegrations(page);
const row = settingOf(page, 'LoraManager path');
const input = row.locator('input');
await expect(input).toHaveAttribute('readonly', '');
await expect(input).toHaveValue('');
await expect(input).toHaveAttribute('placeholder', 'Not detected');
await lora(request);
await openIntegrations(page);
await expect(input).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
await input.focus();
await page.keyboard.type('elsewhere');
await expect(input).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
await expect(status(page)).toHaveText('No unsaved changes.');
});
test('key never prefilled; blank keeps; Clear saved key clears; saved through Save preferences (D114)', async ({ page, request }) => {
const enables = [];
page.on('request', item => { if (item.url().endsWith(ENABLE_URL)) enables.push(item.postDataJSON()); });
await lora(request);
await openIntegrations(page);
const key = settingOf(page, 'CivitAI API key');
const field = key.locator('input[type="password"]');
const clear = key.getByRole('button', { name: 'Clear saved key', exact: true });
await expect(field).toHaveValue('');
await expect(key.locator('p.small')).toHaveText('No key saved.');
await expect(clear).toHaveCount(0);
// A typed key is dropped when the section is left: the field comes back empty with nothing hidden to save (N4).
await field.fill('sk-dropped');
await expect(status(page)).toHaveText('Unsaved changes.');
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Appearance', exact: true }).click();
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
await expect(status(page)).toHaveText('No unsaved changes.');
await expect(key.locator('p.small')).toHaveText('No key saved.');
// Typing a key drafts it; Save sends it once through /lora/enable with the settings revision.
await field.fill('sk-test-key');
await expect(key.locator('p.small')).toHaveText('The new key replaces the saved one when you save.');
await expect(status(page)).toHaveText('Unsaved changes.');
await save(page).click();
await expect(status(page)).toHaveText('Saved to the server configuration.');
expect(enables).toHaveLength(1);
expect(enables[0]).toMatchObject({ civitai_api_key: 'sk-test-key', enabled: false });
expect(enables[0].expected_revision).toMatch(/^(sha256:|absent)/);
await expect(field).toHaveValue('');
await expect(key.locator('p.small')).toHaveText('A key is saved.');
expect((await serverJson(request, '/__test__/integrations/state')).key_saved).toBe(true);
// Never prefilled, even after a reload; the page never receives the key.
const statusBodies = [];
page.on('response', async response => {
if (response.url().endsWith('/prompt_manager/lora/status')) statusBodies.push(await response.text());
});
await openIntegrations(page);
await expect(field).toHaveValue('');
await expect(key.locator('p.small')).toHaveText('A key is saved.');
expect(statusBodies.join('')).not.toContain('sk-test-key');
// The announced line is rewritten only when its text changes, not on every edit elsewhere (S10 re-review N8).
await page.evaluate(() => {
window.__keyNoteWrites = 0;
new MutationObserver(records => { window.__keyNoteWrites += records.length; })
.observe(document.querySelector('#civitai-key-status'), { childList: true, characterData: true, subtree: true });
});
const triggers = settingOf(page, 'Automatically include trigger words').locator('input');
await triggers.check();
await expect(status(page)).toHaveText('Unsaved changes.');
await triggers.uncheck();
await expect(status(page)).toHaveText('No unsaved changes.');
expect(await page.evaluate(() => window.__keyNoteWrites)).toBe(0);
// A blank field keeps the saved key: nothing to save.
await field.fill(' ');
await expect(status(page)).toHaveText('No unsaved changes.');
await field.fill('');
// Clear saved key drafts the removal; Save removes it. The keyboard stays on the key field, and the line that
// says so is announced (S10 review M2).
await expect(key.locator('p.small')).toHaveAttribute('role', 'status');
await expect(field).toHaveAttribute('aria-describedby', 'civitai-key-status');
await clear.focus();
await page.keyboard.press('Enter');
await expect(key.locator('p.small')).toHaveText('The saved key is removed when you save.');
await expect(clear).toHaveCount(0);
await expect(field).toBeFocused();
await save(page).click();
await expect(status(page)).toHaveText('Saved to the server configuration.');
expect(enables.at(-1)).toMatchObject({ clear_civitai_api_key: true });
expect(enables.at(-1)).not.toHaveProperty('civitai_api_key');
await expect(key.locator('p.small')).toHaveText('No key saved.');
expect((await serverJson(request, '/__test__/integrations/state')).key_saved).toBe(false);
});
test('a LoRA save over settings changed elsewhere is a conflict, never a blind write', async ({ page, request }) => {
await lora(request);
await openIntegrations(page);
await settingOf(page, 'Automatically include trigger words').locator('input').check();
// Another page saves the settings meanwhile: the revision this page read is stale.
await expect(await request.post('/prompt_manager/settings', { data: { result_timeout: 42 } })).toBeOK();
await save(page).click();
await expect(status(page)).toHaveText('Settings changed elsewhere. Reload this section to see them.');
expect((await serverJson(request, '/__test__/integrations/state')).trigger_words).toBe(false);
// The draft stays; the next Save carries the reread revision and succeeds.
await save(page).click();
await expect(status(page)).toHaveText('Saved to the server configuration.');
expect((await serverJson(request, '/__test__/integrations/state')).trigger_words).toBe(true);
});
test('a save where only the LoRA part is written says so; the next save needs no conflict (S10 review I3)',
async ({ page, request }) => {
await lora(request);
await expect(await request.post('/__test__/system/missing-source', { data: { inside: true } })).toBeOK();
await openIntegrations(page);
const triggers = settingOf(page, 'Automatically include trigger words').locator('input');
await triggers.check();
// In Sources, the drive is unplugged after the page read the list, and another source is removed.
const nav = page.locator('#workspace-main .section-nav');
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
const sourceRow = title => layoutOf(page).locator('#source-list .operation').filter({ has: page.locator('h3', { hasText: title }) });
await expect(sourceRow('unplugged')).toBeVisible();
await expect(await request.post('/__test__/system/unplug')).toBeOK();
await sourceRow('ComfyUI input').getByRole('button', { name: 'Remove', exact: true }).click();
await save(page).click();
// The LoRA part was saved, the sources were refused: the status gives the server's reason for the source, and
// only the source row is highlighted.
await expect(status(page)).toHaveText("output/unplugged can't be used: Gallery root does not exist. Remove it or reconnect it, then save.");
await expect(sourceRow('unplugged')).toHaveAttribute('aria-invalid', 'true');
expect((await serverJson(request, '/__test__/integrations/state')).trigger_words).toBe(true);
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
await expect(triggers).toBeChecked();
await expect(triggers).not.toHaveAttribute('aria-invalid', 'true');
// The next save carries the revision the saved part left: it saves, with no "changed elsewhere".
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
await sourceRow('unplugged').getByRole('button', { name: 'Remove', exact: true }).click();
await save(page).click();
await expect(status(page)).toHaveText('Saved to the server configuration.');
expect((await serverJson(request, '/prompt_manager/settings')).sources.configured.map(source => source.path)).toEqual(['output']);
});
test('a partial save made from Integrations keeps its outcome when Sources opens (S10 re-review R1)',
async ({ page, request }) => {
await lora(request);
await expect(await request.post('/__test__/system/missing-source', { data: { inside: true } })).toBeOK();
await openIntegrations(page);
const nav = page.locator('#workspace-main .section-nav');
const sourceRow = title => layoutOf(page).locator('#source-list .operation').filter({ has: page.locator('h3', { hasText: title }) });
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
await expect(sourceRow('unplugged')).toBeVisible();
await expect(await request.post('/__test__/system/unplug')).toBeOK();
await sourceRow('ComfyUI input').getByRole('button', { name: 'Remove', exact: true }).click();
// Back in Integrations, a LoRA change, then Save from there.
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
await settingOf(page, 'Automatically include trigger words').locator('input').check();
await save(page).click();
const reason = "output/unplugged can't be used: Gallery root does not exist. Remove it or reconnect it, then save.";
await expect(status(page)).toHaveText(reason);
// Leaving Integrations (no key was typed) keeps the outcome: Sources shows the reason on the refused row.
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
await expect(sourceRow('unplugged')).toHaveAttribute('aria-invalid', 'true');
await expect(status(page)).toHaveText(reason);
});
test('Review import shows replacement, blocked items and network effects with the backup box; Start opens the job (D107)',
async ({ page, request }) => {
await lora(request, { loras: 3, civitai: true, previous: 2 });
const previews = [];
const applies = [];
page.on('request', item => {
if (item.url().endsWith(PREVIEW_URL)) previews.push(item.postDataJSON());
if (item.url().endsWith(APPLY_URL)) applies.push(item.postDataJSON());
});
await openIntegrations(page);
await enable(page);
const before = await serverJson(request, '/__test__/integrations/state');
const review = layoutOf(page).locator('.operation').filter({ hasText: 'LoRA reference data' })
.getByRole('button', { name: 'Review import', exact: true });
await review.click();
await expect(dialog(page).locator('#modal-title')).toHaveText('Review LoRA import');
await expect(dialog(page).locator('#modal-body > h3')).toHaveText('LoRA reference data');
expect(previews).toEqual([{}]);
expect(await gridOf(page)).toEqual([
['Scope', 'Replace 2 imported prompts with 3 LoRAs'],
['Found', '3 metadata files · 3 readable'],
['Protected', 'Nothing is protected'],
['Network', 'Downloads up to 3 example images from CivitAI'],
['Backup', 'Create and verify before changes (browser.db.backup_<date and time>)'],
['Media on disk', 'No files are removed; image links are replaced'],
]);
await expect(dialog(page).locator('details > summary')).toHaveText('LoRAs found (3)');
await expect(dialog(page).locator('.notice h3')).toHaveText('Back up the database first (recommended)');
await expect(dialog(page).locator('p.small').last()).toContainText('stay in the image cache');
// The review wrote and downloaded nothing; Cancel neither.
await dialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
await expect(dialog(page)).toHaveCount(0);
await expect(review).toBeFocused();
expect(await serverJson(request, '/__test__/integrations/state')).toEqual(before);
await review.click();
await dialog(page).getByRole('button', { name: 'Start import', exact: true }).click();
await expect(dialog(page)).toHaveCount(0);
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
await expect(toast(page)).toHaveText(/LoRA reference import started\.|LoRA import finished · 3 imported · 0 errors/);
expect(applies).toHaveLength(1);
expect(Object.keys(applies[0])).toEqual(['token']);
const job = await ended(request, page.url().split('/').pop());
expect([job.kind, job.title, job.state]).toEqual(['lora_import', 'LoRA reference import', 'finished']);
expect(job.counters).toMatchObject({ imported: 3, downloaded: 3, download_errors: 0, errors: 0 });
const main = page.locator('#workspace-main');
await expect(main.locator('h1')).toHaveText('LoRA reference import');
await expect(main.locator('.stat-row .stat dt')).toHaveText(['Imported', 'Linked to existing', 'Skipped', 'Images downloaded',
'Download errors', 'Errors']);
await expect(main.getByRole('button', { name: 'Cancel import', exact: true })).toHaveCount(0);
const after = await serverJson(request, '/__test__/integrations/state');
expect(after.imports).toEqual(['style_01 example prompt', 'style_02 example prompt', 'style_03 example prompt']);
expect(after.downloads).toBe(3);
});
test('saved pairs block the import: the review explains it and offers only Close', async ({ page, request }) => {
await lora(request, { previous: 1 });
await expect(await request.post('/__test__/integrations/pair')).toBeOK();
await openIntegrations(page);
await enable(page);
await layoutOf(page).getByRole('button', { name: 'Review import', exact: true }).click();
expect((await gridOf(page))[2]).toEqual(['Protected', '1 imported prompt used by saved pairs']);
await expect(dialog(page).locator('p.small')).toHaveText(
'Delete or change the saved pairs that use these prompts, then review the import again. Nothing was changed.');
expect(await dialog(page).locator('.dialog-actions button').allTextContents()).toEqual(['Close']);
await dialog(page).getByRole('button', { name: 'Close', exact: true }).click();
await expect(dialog(page)).toHaveCount(0);
});
test('a failed import keeps old imports and its job says so', async ({ page, request }) => {
await lora(request, { loras: 2, previous: 2 });
await openIntegrations(page);
await enable(page);
const before = await serverJson(request, '/__test__/integrations/state');
await expect(await request.post('/__test__/integrations/fail-import')).toBeOK();
await layoutOf(page).getByRole('button', { name: 'Review import', exact: true }).click();
await dialog(page).getByRole('button', { name: 'Start import', exact: true }).click();
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
await expect(toast(page)).toHaveText(`LoRA import failed · ${KEPT}`);
const job = await ended(request, page.url().split('/').pop());
expect([job.state, job.scope.failure, job.counters.errors]).toEqual(['failed', 'import-failed', 1]);
await expect(page.locator('#workspace-main')).toContainText(KEPT);
expect((await serverJson(request, '/__test__/integrations/state')).imports).toEqual(before.imports);
expect(before.imports).toEqual(['previous lora import 1', 'previous lora import 2']);
});
test('the import can be cancelled while it downloads; then nothing is written', async ({ page, request }) => {
await lora(request, { loras: 2, civitai: true, previous: 1 });
await openIntegrations(page);
await enable(page);
const before = await serverJson(request, '/__test__/integrations/state');
await expect(await request.post('/__test__/integrations/gate', { data: { hold: true } })).toBeOK();
await layoutOf(page).getByRole('button', { name: 'Review import', exact: true }).click();
await dialog(page).getByRole('button', { name: 'Start import', exact: true }).click();
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
const id = page.url().split('/').pop();
await expect.poll(async () => (await serverJson(request, '/__test__/integrations/state')).downloads).toBeGreaterThan(0);
const cancel = page.locator('#workspace-main').getByRole('button', { name: 'Cancel import', exact: true });
await expect(cancel).toBeVisible();
// The cancel must have reached the server before the held download goes on (S10 review M1).
await Promise.all([
page.waitForResponse(response => /\/prompt_manager\/jobs\/[0-9a-f]+\/cancel$/.test(response.url()) && response.ok()),
cancel.click(),
]);
await expect(await request.post('/__test__/integrations/gate', { data: { hold: false } })).toBeOK();
const job = await ended(request, id);
expect(job.state).toBe('cancelled');
expect((await serverJson(request, '/__test__/integrations/state')).imports).toEqual(before.imports);
await expect(cancel).toHaveCount(0);
});
+3 -3
View File
@@ -409,11 +409,11 @@ test('out-of-order overview responses never repaint a newer section', async ({ p
await nav.getByRole('button', { name: 'Appearance', exact: true }).click();
await nav.getByRole('button', { name: 'Overview', exact: true }).click();
await expect.poll(held).toBe(2);
// Integrations still hands off (S10), so its body is one known panel.
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
// Auto-tagging still hands off (S11), so its body is one known panel.
await nav.getByRole('button', { name: 'Auto-tagging', exact: true }).click();
await expect(await request.post('/__test__/release')).toBeOK();
await expect.poll(held).toBe(0);
// Both late answers reached a section that had left: Integrations keeps its own body.
// Both late answers reached a section that had left: Auto-tagging keeps its own body.
await expect(layout.locator(':scope > section.handoff')).toBeVisible();
await page.evaluate(() => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve))));
await expect(layout.locator('.summary-metrics')).toHaveCount(0);
+6 -4
View File
@@ -413,7 +413,7 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept
await sidebarLink(page, 'System').click();
// System's Overview is native (S2 Task 6); a section not delivered yet still hands off (D94).
await main.locator('.section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
await main.locator('.section-nav').getByRole('button', { name: 'Auto-tagging', exact: true }).click();
const handoff = main.locator('section.handoff');
await expect(handoff).toHaveClass(/\bsurface\b/);
expect(await handoff.evaluate(surfaceStyle)).toMatchObject(SURFACE);
@@ -463,16 +463,18 @@ test('System is native: the concept shell, section navigation, sections, hand-of
// The section changes in place: focus stays on the pressed section, not the page title.
await expect(button).toBeFocused();
// S2 Task 6: Overview is native (workspace-system.spec.mjs), and so are Appearance (S3 Task 8) and Sources & scans
// (S4 Task 11, workspace-system-sources.spec.mjs, its update notice shown in a fresh browser) and Maintenance (S7
// Task 19, workspace-system-maintenance.spec.mjs); every other section hands off.
// (S4 Task 11, workspace-system-sources.spec.mjs, its update notice shown in a fresh browser), Integrations (S10
// Task 27, workspace-system-integrations.spec.mjs) and Maintenance (S7 Task 19,
// workspace-system-maintenance.spec.mjs); every other section hands off.
if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible();
if (hash === 'sources') await expect(layout.locator('#source-list .operation')).toHaveCount(2);
expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`)))
.toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'section.surface', 'div.save-row'],
sources: ['div.notice', 'section.surface', 'div.save-row'],
integrations: ['section.surface', 'section.surface', 'div.save-row'],
maintenance: ['div.notice', 'section.surface', 'section.surface', 'div.save-row'] }[hash]
?? ['section.surface handoff', 'div.save-row']);
if (['overview', 'appearance', 'sources', 'maintenance'].includes(hash)) continue;
if (['overview', 'appearance', 'sources', 'integrations', 'maintenance'].includes(hash)) continue;
const handoff = layout.locator('section.handoff');
await expect(handoff.locator('h2')).toHaveText('Your current tools are ready');
expect(await handoff.locator('a.workspace-link').evaluateAll(links => links.map(link => link.getAttribute('href'))))
@@ -0,0 +1,198 @@
"""System › Integrations test hooks for the browser tests (#249 System S10).
A fake LoraManager install lives under this server's temp root; nothing ever
reaches a real LoraManager, ComfyUI install or the network:
- ``POST /__test__/integrations/reset`` turns the integration off (no path,
no key, no trigger words), removes the fake install, its LoRA folder and
the image cache, forgets every LoRA import and injected failure, and opens
the download gate;
- ``POST /__test__/integrations/lora-manager`` ``{loras, civitai, previous}``
writes ``custom_nodes/ComfyUI-Lora-Manager`` (detected through the
``folder_paths`` stub's base path) and ``loras`` metadata files under
``models/loras``, each with a local preview PNG; ``civitai`` adds one
CivitAI example image per LoRA (served by the stub opener below);
``previous`` saves that many earlier ``lora-manager`` imports with tags and
image links;
- ``POST /__test__/integrations/gate`` ``{hold}`` holds every CivitAI
download until it is released (``{"hold": false}``), so a test sees the job
while it downloads;
- ``POST /__test__/integrations/fail-import`` makes the next collection
replacement fail (a database error after staging);
- ``POST /__test__/integrations/pair`` saves a pair whose positive is the first
imported prompt (a saved pair that blocks the replacement);
- ``GET /__test__/integrations/state`` answers the imported prompt texts,
whether a CivitAI key is saved and the downloads served.
CivitAI downloads are always answered by a stub opener (a small PNG), so no
test ever opens a network connection.
"""
import io
import json
import shutil
import threading
from pathlib import Path
from aiohttp import web
from PIL import Image
MAX_LORAS = 20
CATEGORY = "lora-manager"
def _png():
buffer = io.BytesIO()
Image.new("RGB", (8, 8), "#3a6ea5").save(buffer, "PNG")
return buffer.getvalue()
class _Response:
def __init__(self, body):
self._body = io.BytesIO(body)
self.headers = {"Content-Length": str(len(body))}
def read(self, amount=-1):
return self._body.read(amount)
def __enter__(self):
return self
def __exit__(self, *exc):
return False
def register_integrations_commands(app, routes, api, root: Path):
from py import lora_utils
from py.config import IntegrationConfig
from utils.hashing import generate_prompt_hash
lora_manager = root / "custom_nodes" / "ComfyUI-Lora-Manager"
loras = root / "models" / "loras"
gate = threading.Event()
gate.set()
served = []
failures = {"replace": 0}
pairs = [] # the blocking pairs this fixture saved
real_replace = type(api.db).replace_category_prompts
def stub_open(request, timeout=10):
served.append(request.full_url)
gate.wait(30)
return _Response(_png())
lora_utils._open_url = stub_open
def replace(db, *args, **kwargs):
if failures["replace"] > 0:
failures["replace"] -= 1
raise RuntimeError("Injected replacement failure")
return real_replace(db, *args, **kwargs)
type(api.db).replace_category_prompts = replace
def reset():
IntegrationConfig.LORA_MANAGER_ENABLED = False
IntegrationConfig.LORA_MANAGER_PATH = ""
IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED = False
IntegrationConfig.CIVITAI_API_KEY = ""
lora_utils.get_trigger_cache().clear()
gate.set()
served.clear()
failures["replace"] = 0
cache = Path(lora_utils.get_lora_image_cache_dir(create=False))
for folder in (lora_manager, loras, cache):
shutil.rmtree(folder, ignore_errors=True)
for pair_id in pairs:
api.db.delete_prompt_pair(pair_id)
pairs.clear()
real_replace(api.db, CATEGORY, [], allow_empty=True)
@routes.post("/__test__/integrations/reset")
async def integrations_reset(request):
reset()
return web.json_response({"success": True})
@routes.post("/__test__/integrations/lora-manager")
async def make_lora_manager(request):
data = await request.json() if request.can_read_body else {}
count = data.get("loras", 2)
previous = data.get("previous", 0)
for value in (count, previous):
if isinstance(value, bool) or not isinstance(value, int):
raise web.HTTPBadRequest()
if not 0 <= count <= MAX_LORAS or not 0 <= previous <= MAX_LORAS:
raise web.HTTPBadRequest()
(lora_manager / "py").mkdir(parents=True, exist_ok=True)
(lora_manager / "__init__.py").write_text("")
loras.mkdir(parents=True, exist_ok=True)
for number in range(count):
stem = f"style_{number + 1:02}"
civitai = {"trainedWords": [f"trigger{number + 1}"]}
if data.get("civitai") is True:
civitai["images"] = [
{
"url": f"https://image.civitai.com/x/{stem}.jpeg",
"meta": {"prompt": f"{stem} example prompt"},
}
]
meta = {
"file_name": f"{stem}.safetensors",
"model_name": f"Style {number + 1:02}",
"civitai": civitai,
}
(loras / f"{stem}.metadata.json").write_text(json.dumps(meta))
(loras / f"{stem}.png").write_bytes(_png())
for number in range(previous):
text = f"previous lora import {number + 1}"
prompt_id = api.db.save_prompt(
text=text,
category=CATEGORY,
tags=[CATEGORY, f"lora:previous-{number + 1}"],
prompt_hash=generate_prompt_hash(text),
)
api.db.link_image_to_prompt(str(prompt_id), str(loras / "old.png"))
return web.json_response({"success": True})
@routes.post("/__test__/integrations/gate")
async def download_gate(request):
data = await request.json()
if not isinstance(data.get("hold"), bool):
raise web.HTTPBadRequest()
if data["hold"]:
gate.clear()
else:
gate.set()
return web.json_response({"success": True})
@routes.post("/__test__/integrations/fail-import")
async def fail_import(request):
failures["replace"] = 1
return web.json_response({"success": True})
@routes.post("/__test__/integrations/pair")
async def pair(request):
with api.db.model.get_connection() as conn:
row = conn.execute(
"SELECT text FROM prompts WHERE category = ? ORDER BY id LIMIT 1",
(CATEGORY,),
).fetchone()
if row is None:
raise web.HTTPConflict()
pairs.append(api.db.save_prompt_pair(row[0], "blurry"))
return web.json_response({"success": True})
@routes.get("/__test__/integrations/state")
async def state(request):
imports = api.db.search_prompts(category=CATEGORY, limit=MAX_LORAS * 2)
return web.json_response(
{
"imports": sorted(prompt["text"] for prompt in imports),
"key_saved": bool(IntegrationConfig.CIVITAI_API_KEY),
"enabled": IntegrationConfig.LORA_MANAGER_ENABLED,
"trigger_words": IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED,
"downloads": len(served),
}
)
return reset
+18
View File
@@ -190,3 +190,21 @@ test('bridge remains live through both renderer switches and workflow reload wit
await expect(page.locator('.pm-focused').getByRole('textbox',{name:'Main prompt',exact:true})).toHaveValue('After reload');
});
});
test('each destination reports its window renderer, and System › Integrations shows it (S10)',async({page,context},testInfo)=>{
const initial=testInfo.project.metadata.initialRenderer;
const id=await addNode(page,'PromptManager',{text:'Renderer check'});
await withClient(page,context,async clientPage=>{
const name=renderer=>renderer?'Nodes 2.0':'Classic';
const status=clientPage.locator('#workspace-main #renderer-status');
expect((await destination(clientPage,id)).target.renderer).toBe(initial?'nodes2':'classic');
await clientPage.evaluate(()=>{location.hash='#/system/integrations';});
await expect(status).toHaveText(`Detected: ${name(initial)} (1 ComfyUI window).`);
// Switching the renderer in ComfyUI is what the next check reports; nothing is sent or queued.
await page.evaluate(async renderer=>{const {app}=await import('/scripts/app.js');await app.ui.settings.setSettingValue('Comfy.VueNodes.Enabled',renderer);await app.loadGraphData(app.graph.serialize());},!initial);
await expect.poll(()=>page.evaluate(()=>LiteGraph.vueNodesMode)).toBe(!initial);
expect((await destination(clientPage,id)).target.renderer).toBe(initial?'classic':'nodes2');
await clientPage.reload();
await expect(status).toHaveText(`Detected: ${name(!initial)} (1 ComfyUI window).`);
});
});
+11
View File
@@ -17,6 +17,17 @@ test('lists only explicit PromptManager destinations and roles for all five sche
assert.equal(targets[0].id,'0'); assert.equal(targets[0].label,'My prompts');
assert.deepEqual(createPromptAdapter({}).listTargets(),[]);
});
test('each destination says which renderer its window draws nodes with (System › Integrations, S10)',async()=>{
const {createPromptAdapter}=await source; const {app}=fixture();
assert.equal(createPromptAdapter(app,{runtime:{LiteGraph:{vueNodesMode:true}}}).listTargets()[0].renderer,'nodes2');
assert.equal(createPromptAdapter(app,{runtime:{LiteGraph:{vueNodesMode:false}}}).listTargets()[0].renderer,'classic');
assert.equal(createPromptAdapter(app,{runtime:{LiteGraph:{}}}).listTargets()[0].renderer,'classic');
assert.equal(Object.hasOwn(createPromptAdapter(app,{runtime:{}}).listTargets()[0],'renderer'),false);
// Read at every listing: a window that switches renderer reports the new one.
const runtime={LiteGraph:{vueNodesMode:false}}; const adapter=createPromptAdapter(app,{runtime});
runtime.LiteGraph.vueNodesMode=true;
assert.equal(adapter.listTargets()[0].renderer,'nodes2');
});
test('previews exact composition, updates requested visible/serialized values, preserves other fields',async()=>{
const {createPromptAdapter}=await source; const {app,node,widget}=fixture(); const adapter=createPromptAdapter(app);
const {preview,snapshot}=adapter.preview('7',{positive:'new'});
@@ -0,0 +1,197 @@
// System › Integrations (#249 System S10 Task 27): the host values and saves behind the LoRA rows (D114), the D106
// renderer and detection lines, and the D107 import review built from the server's plan.
const test = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const integrations = import('../../web/js/workspace/system-integrations.mjs');
const kinds = import('../../web/js/workspace/activity-kinds.mjs');
const CONCEPT = fs.readFileSync(path.join(__dirname, '../../docs/concepts/workspace/app.js'), 'utf8');
const SOURCE = fs.readFileSync(path.join(__dirname, '../../web/js/workspace/system-integrations.mjs'), 'utf8');
test('the section keeps the concept rows, labels and options in order', () => {
const start = CONCEPT.indexOf('case "integrations":');
const body = CONCEPT.slice(start, CONCEPT.indexOf('break;', start));
const rows = [...body.matchAll(/setting\("([^"]+)", "([^"]+)", "([^"]+)"/g)].map(([, id, label]) => [id, label]);
assert.deepEqual(rows.map(([id]) => id), ['send-destination', 'renderer', 'return-canvas', 'lora-enable', 'lora-triggers',
'civitai-key']);
for (const [id, label] of rows) {
assert.ok(SOURCE.includes(`id: '${id}', label: '${label}'`), `${id} keeps the concept label "${label}"`);
}
for (const option of ['Ask each time', 'Selected Prompt Manager node', 'Create a Prompt Manager (+/-) node',
'Follow host renderer', 'Classic', 'Nodes 2.0']) assert.ok(body.includes(option) && SOURCE.includes(`'${option}'`), option);
for (const surface of ['ComfyUI', 'LoRA & resources']) assert.ok(SOURCE.includes(`'h2', '${surface}'`), surface);
assert.ok(body.includes('"LoRA reference data", "Review and import resource metadata."'));
// Prototype copy is never shipped (Global constraints): the sample detection line and the mock warning.
assert.ok(!SOURCE.includes('Sample detection'));
assert.ok(!SOURCE.includes('Do not enter a real key'));
assert.ok(!SOURCE.includes('Simulate import'));
});
test('LoRA status becomes host values: drafted flags and read-only facts kept apart', async () => {
const { loraValues } = await integrations;
assert.deepEqual(loraValues({ enabled: true, trigger_words_enabled: false, path: 'custom_nodes/ComfyUI-Lora-Manager',
has_civitai_api_key: true, detected: true, detected_path: 'custom_nodes/ComfyUI-Lora-Manager', trigger_cache_loaded: true }), {
lora_enabled: true, lora_trigger_words_enabled: false, clear_civitai_api_key: false,
lora_path: 'custom_nodes/ComfyUI-Lora-Manager', lora_detected: true, lora_detected_path: 'custom_nodes/ComfyUI-Lora-Manager',
lora_detected_usable: true, lora_has_key: true, lora_cache_loaded: true,
});
const empty = loraValues({});
assert.equal(empty.lora_enabled, false);
assert.equal(empty.lora_path, '');
assert.equal(empty.lora_has_key, false);
// The key itself is never part of what the page holds from the server.
assert.ok(!Object.keys(loraValues({ civitai_api_key: 'secret' })).includes('civitai_api_key'));
});
test('host changes split between /lora/enable and /settings', async () => {
const { splitHostChanges } = await integrations;
assert.deepEqual(splitHostChanges({ lora_enabled: true, result_timeout: 5 }),
{ lora: { lora_enabled: true }, settings: { result_timeout: 5 } });
assert.deepEqual(splitHostChanges({ result_timeout: 5 }), { lora: null, settings: { result_timeout: 5 } });
assert.deepEqual(splitHostChanges({ civitai_api_key: 'k', clear_civitai_api_key: false }),
{ lora: { civitai_api_key: 'k', clear_civitai_api_key: false }, settings: null });
assert.deepEqual(splitHostChanges(undefined), { lora: null, settings: null });
});
test('key never prefilled; blank keeps; Clear saved key clears; saved through Save preferences (D114)', async () => {
const { loraPayload, keyText, KEY_COPY } = await integrations;
const saved = { lora_enabled: true, lora_trigger_words_enabled: false, lora_path: 'custom_nodes/LM', lora_has_key: true };
// The whole LoRA state goes with every save; the saved path is kept; no key unless one was typed.
assert.deepEqual(loraPayload({ lora_trigger_words_enabled: true }, saved),
{ enabled: true, trigger_words_enabled: true, path: 'custom_nodes/LM' });
assert.deepEqual(loraPayload({ civitai_api_key: ' ' }, saved), { enabled: true, trigger_words_enabled: false, path: 'custom_nodes/LM' });
assert.deepEqual(loraPayload({ civitai_api_key: ' new-key ' }, saved),
{ enabled: true, trigger_words_enabled: false, path: 'custom_nodes/LM', civitai_api_key: 'new-key' });
assert.deepEqual(loraPayload({ clear_civitai_api_key: true, civitai_api_key: 'ignored' }, saved),
{ enabled: true, trigger_words_enabled: false, path: 'custom_nodes/LM', clear_civitai_api_key: true });
assert.deepEqual(loraPayload({ lora_enabled: true }, {}), { enabled: true, trigger_words_enabled: false, path: '' });
assert.equal(keyText({ lora_has_key: true }, {}), KEY_COPY.saved);
assert.equal(keyText({ lora_has_key: false }, {}), KEY_COPY.none);
assert.equal(keyText({ lora_has_key: true }, { clear_civitai_api_key: true }), KEY_COPY.clearing);
assert.equal(keyText({ lora_has_key: false }, { civitai_api_key: 'k' }), KEY_COPY.replacing);
assert.equal(keyText({}, {}), null);
assert.deepEqual(KEY_COPY, { saved: 'A key is saved.', none: 'No key saved.', clearing: 'The saved key is removed when you save.',
replacing: 'The new key replaces the saved one when you save.' });
});
test('LoRA rows show detected / enabled / cache-loaded separately', async () => {
const { detectionText, cacheText } = await integrations;
assert.equal(detectionText({ lora_detected: true, lora_detected_path: 'custom_nodes/ComfyUI-Lora-Manager' }),
'Detected: custom_nodes/ComfyUI-Lora-Manager.');
assert.equal(detectionText({ lora_detected: true, lora_detected_path: '' }), 'Detected.');
assert.equal(detectionText({ lora_detected: false }), 'Not detected in ComfyUI’s custom_nodes folders.');
assert.equal(detectionText({}), null);
// S10 review N5: a linked install outside custom_nodes is found but can't be enabled.
assert.equal(detectionText({ lora_detected: true, lora_detected_path: 'LoRA-Manager', lora_detected_usable: false }),
'Found at LoRA-Manager, but outside ComfyUI’s custom_nodes folders, so it can’t be enabled.');
assert.equal((await integrations).loraValues({ detected: true, detected_usable: false }).lora_detected_usable, false);
assert.equal(cacheText({ lora_cache_loaded: true }), 'Trigger words are loaded.');
assert.equal(cacheText({ lora_cache_loaded: false }), 'Trigger words are not loaded.');
assert.equal(cacheText({}), null);
});
test('host compatibility shows the detected renderer as status (D106)', async () => {
const { rendererText, RENDERER_COPY } = await integrations;
const host = (...renderers) => ({ hostId: 'h', label: 'PromptManager — ComfyUI', targets: renderers.map((renderer, id) =>
({ id: String(id), roles: ['positive'], renderer })) });
assert.equal(rendererText([]), RENDERER_COPY.none);
assert.equal(rendererText(null), RENDERER_COPY.none);
assert.equal(rendererText([host()]), RENDERER_COPY.noNodes);
assert.equal(rendererText([host('vue')]), RENDERER_COPY.noNodes);
assert.equal(rendererText([host('classic')]), 'Detected: Classic (1 ComfyUI window).');
assert.equal(rendererText([host('nodes2', 'nodes2')]), 'Detected: Nodes 2.0 (1 ComfyUI window).');
assert.equal(rendererText([host('nodes2'), host('classic')]), 'Detected: Classic and Nodes 2.0 (2 ComfyUI windows).');
});
test('the import review shows replacement, blocked items and network effects (D107)', async () => {
const { importGrid } = await integrations;
const plan = { lora_files: 3, with_metadata: 2, current_imports: 5, blocked_by_pairs: 0, planned_downloads: 4,
scope: { count: 2, label: 'Replace 5 imported prompts with 2 LoRAs' }, protected: { count: 0, reasons: [] },
backup: { policy: 'on', destination_label: 'prompts.db.backup_<date and time>' }, network: { planned_downloads: 4, api_key: true },
media: { deletes_files: false, label: 'No files are removed; image links are replaced' } };
assert.deepEqual(importGrid(plan), [
['Scope', 'Replace 5 imported prompts with 2 LoRAs'],
['Found', '3 metadata files · 2 readable'],
['Protected', 'Nothing is protected'],
['Network', 'Downloads up to 4 example images from CivitAI with your saved key'],
['Backup', 'Create and verify before changes (prompts.db.backup_<date and time>)'],
['Media on disk', 'No files are removed; image links are replaced'],
]);
const blocked = importGrid({ ...plan, planned_downloads: 1, network: { api_key: false }, protected: { count: 2,
reasons: ['2 imported prompts used by saved pairs'] } });
assert.deepEqual(blocked[2], ['Protected', '2 imported prompts used by saved pairs']);
assert.deepEqual(blocked[3], ['Network', 'Downloads up to 1 example image from CivitAI']);
assert.deepEqual(importGrid({})[0], ['Scope', '0 LoRAs']);
assert.deepEqual(importGrid({})[3], ['Network', 'No downloads']);
assert.deepEqual(importGrid({})[5], ['Media on disk', 'No files are removed']);
});
test('a LoRA import job reads in Activity: tiles, outcomes and the kept collection on failure', async () => {
const { kindOf, finishToast, outcomeLine, viewOf } = await kinds;
const view = kindOf('lora_import');
assert.equal(view.pillVerb, 'Importing LoRAs');
assert.deepEqual(view.tiles.map(([group]) => group), ['imported', 'linked', 'skipped', 'downloaded', 'download_errors', 'errors']);
const counters = { imported: 3, linked: 1, skipped: 0, downloaded: 2, download_errors: 0, errors: 0 };
const job = { kind: 'lora_import', title: 'LoRA reference import', state: 'finished', cancellable: false, counters };
assert.equal(outcomeLine(job), '3 imported · 0 errors');
assert.equal(finishToast(job), 'LoRA import finished · 3 imported · 0 errors');
assert.equal(finishToast({ ...job, state: 'cancelled' }), 'LoRA import cancelled · 3 imported');
const failed = { ...job, state: 'failed', status_line: 'The import failed; the previous LoRA imports were kept.' };
assert.equal(finishToast(failed), 'LoRA import failed · The import failed; the previous LoRA imports were kept.');
assert.equal(finishToast({ ...failed, status_line: '' }), 'LoRA import failed · 3 imported · 0 errors');
assert.equal(viewOf(job).tiles.length, 6);
// The legacy page's held import (no counters) reads as the generic job.
assert.equal(viewOf({ kind: 'lora_import', title: 'LoRA import (admin page)', cancellable: false, counters: {} }).tiles.length, 0);
});
test('a failed import the files or saved pairs overtook is reviewed again in Integrations', async () => {
const { reviewAgainOf } = await import('../../web/js/workspace/activity-job-view.mjs');
const failed = failure => ({ kind: 'lora_import', state: 'failed', scope: { operation: 'lora_import', failure } });
for (const failure of ['plan-changed', 'storage-changed', 'plan-expired', 'blocked-by-pairs']) {
assert.deepEqual(reviewAgainOf(failed(failure)), { section: '#/system/integrations' }, failure);
}
assert.equal(reviewAgainOf(failed('import-failed')), null);
assert.equal(reviewAgainOf({ ...failed('plan-changed'), state: 'cancelled' }), null);
});
test('a host save in two parts reports exactly which part saved (S10 review I3)', async () => {
const { writeHostInParts } = await integrations;
const saved = { lora_enabled: true, lora_trigger_words_enabled: false, lora_path: 'custom_nodes/LM' };
const calls = [];
const ok = revision => async body => { calls.push(body); return { success: true, persisted: true, revision }; };
const refuse = (code, message) => async body => { calls.push(body); throw Object.assign(new Error(message), { code }); };
const run = (changes, postLora, postSettings) => writeHostInParts({ changes, saved, expected_revision: 'r0', postLora, postSettings });
// Both parts: the second carries the revision the first returned.
assert.equal((await run({ lora_trigger_words_enabled: true, result_timeout: 5 }, ok('r1'), ok('r2'))).revision, 'r2');
assert.deepEqual(calls.map(body => body.expected_revision), ['r0', 'r1']);
// LoRA saved, settings refused: the LoRA keys and the new revision ride on the settings error.
calls.length = 0;
const later = await run({ lora_trigger_words_enabled: true, gallery_root_paths: ['x'] }, ok('r1'),
refuse('invalid-source', 'x cannot be used')).catch(error => error);
assert.equal(later.code, 'invalid-source');
assert.deepEqual(later.partial, { saved: ['lora_trigger_words_enabled'], revision: 'r1' });
// LoRA refused (not detected), settings still sent with the unchanged revision and saved.
calls.length = 0;
const first = await run({ lora_enabled: true, result_timeout: 5 }, refuse(null, 'LoraManager not found'), ok('r2'))
.catch(error => error);
assert.equal(first.message, 'LoraManager not found');
assert.deepEqual(first.partial, { saved: ['result_timeout'], revision: 'r2' });
assert.deepEqual(calls.map(body => body.expected_revision), ['r0', 'r0']);
// A conflict on the first write stops there; a lone part fails as itself.
calls.length = 0;
const conflict = await run({ lora_enabled: false, result_timeout: 5 }, refuse('settings-changed', 'changed'), ok('r2'))
.catch(error => error);
assert.equal(conflict.code, 'settings-changed');
assert.equal(conflict.partial, undefined);
assert.equal(calls.length, 1);
const alone = await run({ result_timeout: 5 }, ok('r1'), refuse('busy', 'scan running')).catch(error => error);
assert.equal(alone.code, 'busy');
assert.equal(alone.partial, undefined);
const both = await run({ lora_enabled: true, result_timeout: 5 }, refuse(null, 'a'), refuse(null, 'b')).catch(error => error);
assert.equal(both.message, 'b');
assert.equal(both.partial, undefined);
});
+40
View File
@@ -190,6 +190,7 @@ test('status text per outcome', async () => {
host: 'Saved to the server configuration.',
runtime: 'Applied until the server restarts.',
partial: 'Some changes were not saved. Review the highlighted settings.',
partialSaved: 'Some changes were saved to the server configuration; the highlighted ones were not.',
tabOnly: 'Saved for this tab only; browser storage is unavailable.',
conflict: 'Settings changed elsewhere. Reload this section to see them.',
restart: 'Restart ComfyUI to apply:',
@@ -364,3 +365,42 @@ test('a host save refused for one source says the server reason and names it unt
assert.equal(drafts.invalidSource(), null);
assert.equal(drafts.statusText(), 'Unsaved changes.');
});
test('a partial host save keeps what saved, adopts its revision and fails only the rest (S10 review I3)', async () => {
const { SETTINGS_STATUS } = await source;
const domains = adapters({ host: { lora_trigger_words_enabled: false, gallery_root_paths: ['a'] } });
const drafts = await loaded(domains);
// The LoRA part is written (config.json moves to sha256:lora), then /settings refuses one source.
domains.outcome.host = () => {
throw Object.assign(new Error('missing can\'t be used: not found. Remove it or reconnect it, then save.'), {
code: 'invalid-source', source: 'missing', partial: { saved: ['lora_trigger_words_enabled'], revision: 'sha256:lora' } });
};
drafts.set('host', 'lora_trigger_words_enabled', true);
drafts.set('host', 'gallery_root_paths', ['a', 'missing']);
const first = await drafts.save();
assert.equal(first.host, 'partial');
assert.deepEqual([...drafts.failed('host')], ['gallery_root_paths']);
assert.equal(drafts.saved('host').lora_trigger_words_enabled, true);
assert.deepEqual(drafts.changes('host'), { gallery_root_paths: ['a', 'missing'] });
assert.equal(drafts.invalidSource(), 'missing');
assert.equal(drafts.statusText(), 'missing can\'t be used: not found. Remove it or reconnect it, then save.');
// The next save carries the revision the saved part produced: no false "changed elsewhere".
domains.outcome.host = null;
drafts.set('host', 'gallery_root_paths', ['a']);
const second = await drafts.save();
assert.equal(second.host, 'saved');
assert.equal(domains.calls.at(-1)[2].expected_revision, 'sha256:lora');
assert.deepEqual(domains.calls.at(-1)[1], { gallery_root_paths: ['a'] });
// Without a server reason, the status says some were saved and the highlighted ones were not.
domains.outcome.host = () => { throw Object.assign(new Error('boom'), { partial: { saved: ['x'], revision: 'sha256:three' } }); };
drafts.set('host', 'x', 1);
drafts.set('host', 'y', 2);
assert.equal((await drafts.save()).host, 'partial');
assert.equal(drafts.statusText(), SETTINGS_STATUS.partialSaved);
assert.deepEqual([...drafts.failed('host')], ['y']);
// A partial result that saved nothing is a plain failure.
domains.outcome.host = () => { throw Object.assign(new Error('boom'), { partial: { saved: [] } }); };
drafts.set('host', 'z', 3);
assert.equal((await drafts.save()).host, 'failed');
assert.equal(drafts.statusText(), SETTINGS_STATUS.partial);
});
+64 -4
View File
@@ -36,7 +36,7 @@ from aiohttp.test_utils import AioHTTPTestCase
from database.operations import PromptDatabase
from py.api import PromptManagerAPI
from py import config_store, lora_utils
from py import config_store, lora_import, lora_utils
from py.config import IntegrationConfig, PromptManagerConfig
from py.api import settings_routes as settings_module
from py.api import lora_integration as lora_module
@@ -252,6 +252,21 @@ class TestLoraStatusRoute(LoraAPITestCase):
self.assertFalse(os.path.isabs(public))
self.assertNotIn(str(self.tmp_root), body)
self.assertFalse(data["trigger_cache_loaded"])
self.assertTrue(data["detected_usable"])
async def test_status_says_a_detected_install_outside_custom_nodes_is_unusable(
self,
):
# S10 review N5: found (a linked install), but enabling it is refused.
outside = _make_lora_manager(self.tmp_root)
self.detect_mock.return_value = str(outside)
resp = await self.client.request("GET", "/prompt_manager/lora/status")
data = await resp.json()
self.assertEqual((data["detected"], data["detected_usable"]), (True, False))
self.detect_mock.return_value = None
resp = await self.client.request("GET", "/prompt_manager/lora/status")
self.assertFalse((await resp.json())["detected_usable"])
async def test_status_error_is_500(self):
with patch("py.lora_utils.get_trigger_cache", side_effect=RuntimeError("x")):
@@ -528,6 +543,48 @@ class TestLoraEnableRoute(LoraAPITestCase):
self.assertEqual(self.config_file.read_bytes(), b"{not json")
self.assertFalse(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
async def test_enable_with_a_stale_revision_changes_nothing(self):
# System S10: the Save row sends the revision it read; a file changed
# since is a settings-changed conflict and nothing is written.
self.config_file.write_text(json.dumps({"web_ui": {"result_timeout": 9}}))
resp = await self.client.request("GET", "/prompt_manager/settings")
revision = (await resp.json())["revision"]
self.config_file.write_text(json.dumps({"web_ui": {"result_timeout": 10}}))
before = self.config_file.read_bytes()
resp = await self._post_json(
"/prompt_manager/lora/enable",
{
"enabled": False,
"trigger_words_enabled": True,
"expected_revision": revision,
},
)
body = await resp.json()
self.assertEqual((resp.status, body["code"]), (409, "settings-changed"))
self.assertEqual(self.config_file.read_bytes(), before)
self.assertFalse(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
resp = await self._post_json(
"/prompt_manager/lora/enable",
{
"enabled": False,
"trigger_words_enabled": True,
"expected_revision": body["revision"],
},
)
saved = await resp.json()
self.assertEqual(resp.status, 200)
self.assertTrue(saved["revision"].startswith("sha256:"))
self.assertTrue(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
resp = await self._post_json(
"/prompt_manager/lora/enable",
{"enabled": False, "expected_revision": 7},
)
self.assertEqual(resp.status, 400)
async def test_enable_requires_boolean_flags_and_body_cap(self):
for body in (
{"enabled": "yes"},
@@ -762,9 +819,9 @@ class TestLoraScanRoute(LoraAPITestCase):
async def test_scan_error_hides_the_server_path(self):
self._enable()
err = PermissionError(13, "Permission denied", str(self.tmp_root / "x.db"))
with patch.object(self.api.db, "delete_prompts_by_category", side_effect=err):
with patch.object(self.api.db, "replace_category_prompts", side_effect=err):
resp = await self.client.request("POST", "/prompt_manager/lora/scan")
body = await resp.text()
body = await resp.text() # the stream ends inside the patch
self.assertNotIn(str(self.tmp_root), body)
self.assertIn("x.db", body)
# The admin page's modal shows the failure as its status and closes.
@@ -862,7 +919,10 @@ class TestLoraScanRoute(LoraAPITestCase):
loras.mkdir()
_write_metadata(loras, "neon", words=["glow"])
with patch.object(self.api.db, "save_prompt", side_effect=RuntimeError("db")):
# Preparing one LoRA's record fails: it is skipped, the import goes on.
with patch.object(
lora_import, "build_record", side_effect=RuntimeError("bad record")
):
resp, events, _ = await self._scan(loras)
self.assertEqual(events[-1]["imported"], 0)
+912
View File
@@ -0,0 +1,912 @@
"""The staged LoRA reference import (#221, #249 System S10).
A failed import must leave the previous ``lora-manager`` collection exactly as
it was (prompts, tags and image links); a preview writes nothing and requests
nothing; an apply does what was reviewed or nothing. Discovery, metadata and
CivitAI downloads are disposable fixtures: no network, no LoraManager install.
"""
import asyncio
import json
import os
import sqlite3
import sys
import tempfile
import threading
import time
import unittest
from pathlib import Path
from unittest.mock import patch
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
try:
from tests.test_api_lora import (
SECRET_KEY,
LoraAPITestCase,
_FakeResponse,
_make_lora_manager,
_png_bytes,
_write_metadata,
)
except ImportError: # discovered with tests/ as the top-level directory
from test_api_lora import (
SECRET_KEY,
LoraAPITestCase,
_FakeResponse,
_make_lora_manager,
_png_bytes,
_write_metadata,
)
from database.category_import import EmptyReplacement
from database.maintenance import PlanChanged
from database.operations import PairDependencyError, PromptDatabase
from py import lora_import, lora_utils
from py.api.jobs import RESTORE_JOB_KIND
from py.api.lora_import_routes import LORA_PREVIEW_LEASE
from py.config import IntegrationConfig, PromptManagerConfig
from utils.hashing import generate_prompt_hash
CATEGORY = "lora-manager"
PREVIEW = "/prompt_manager/lora/import/preview"
APPLY = "/prompt_manager/lora/import/apply"
SCAN = "/prompt_manager/lora/scan"
def _snapshot(db):
"""Every row of the tables an import touches, in a fixed order."""
tables = {
"prompts": "SELECT id, text, category, hash FROM prompts ORDER BY id",
"prompt_tags": "SELECT prompt_id, tag_id FROM prompt_tags ORDER BY 1, 2",
"tags": "SELECT id, name FROM tags ORDER BY id",
"generated_images": (
"SELECT id, prompt_id, image_path, file_path FROM generated_images "
"ORDER BY id"
),
}
with db.model.get_connection() as conn:
return {
name: [tuple(row) for row in conn.execute(sql).fetchall()]
for name, sql in tables.items()
}
def _seed_previous_imports(db):
"""Two earlier imports with tags and image links, and one other prompt."""
ids = []
for name in ("old-one", "old-two"):
prompt_id = db.save_prompt(
text=f"{name} prompt",
category=CATEGORY,
tags=[CATEGORY, f"lora:{name}"],
prompt_hash=generate_prompt_hash(f"{name} prompt"),
)
db.link_image_to_prompt(str(prompt_id), f"/cache/{name}/a.jpg")
db.link_image_to_prompt(str(prompt_id), f"/cache/{name}/b.jpg")
ids.append(prompt_id)
db.save_prompt(
text="mine",
category="portraits",
tags=["keep"],
prompt_hash=generate_prompt_hash("mine"),
)
return ids
def _record(text, images=(), name=None):
return {
"name": name or text,
"text": text,
"hash": generate_prompt_hash(text),
"tags": [CATEGORY, f"lora:{name or text}"],
"images": list(images),
}
class CategoryReplacementTests(unittest.TestCase):
"""``replace_category_prompts``: one transaction, all or nothing."""
def setUp(self):
handle = tempfile.NamedTemporaryFile(delete=False, suffix=".db")
handle.close()
self.path = handle.name
self.db = PromptDatabase(self.path)
self.ids = _seed_previous_imports(self.db)
def tearDown(self):
self.db.close_all()
for suffix in ("", "-wal", "-shm"):
if os.path.exists(self.path + suffix):
os.unlink(self.path + suffix)
def test_transaction_failure_rolls_back_everything(self):
before = _snapshot(self.db)
records = [_record("new one", ["/c/n1.jpg"]), _record("new two", ["/c/n2.jpg"])]
real = self.db._insert_image_link
calls = []
def failing(conn, prompt_id, image, metadata=None):
calls.append(image)
if len(calls) == 2: # after the old rows went and one record landed
raise sqlite3.OperationalError("disk I/O error")
return real(conn, prompt_id, image, metadata)
with patch.object(self.db, "_insert_image_link", side_effect=failing):
with self.assertRaises(sqlite3.OperationalError):
self.db.replace_category_prompts(CATEGORY, records)
self.assertEqual(len(calls), 2)
self.assertEqual(_snapshot(self.db), before)
def test_pair_dependency_blocks_without_partial_delete(self):
# A saved pair reuses an imported prompt as its positive component.
self.db.save_prompt_pair("old-two prompt", "blurry")
before = _snapshot(self.db)
self.assertEqual(
self.db.category_import_state(CATEGORY), {"ids": self.ids, "paired": 1}
)
with self.assertRaises(PairDependencyError):
self.db.replace_category_prompts(CATEGORY, [_record("new one")])
self.assertEqual(_snapshot(self.db), before)
def test_explicit_empty_replacement_requires_review_flag(self):
before = _snapshot(self.db)
with self.assertRaises(EmptyReplacement):
self.db.replace_category_prompts(CATEGORY, [])
self.assertEqual(_snapshot(self.db), before)
result = self.db.replace_category_prompts(CATEGORY, [], allow_empty=True)
self.assertEqual(result["removed"], 2)
self.assertEqual(self.db.category_import_state(CATEGORY)["ids"], [])
self.assertEqual(len(self.db.search_prompts(category="portraits")), 1)
def test_changed_imports_since_review_are_plan_changed(self):
before = _snapshot(self.db)
with self.assertRaises(PlanChanged):
self.db.replace_category_prompts(
CATEGORY, [_record("new one")], expected_ids=self.ids[:1]
)
self.assertEqual(_snapshot(self.db), before)
def test_replacement_links_repeats_and_existing_prompts(self):
records = [
_record("fresh", ["/c/f.jpg"]),
_record("Fresh ", ["/c/f2.jpg"], name="fresh-again"), # same hash
_record("mine", ["/c/m.jpg"], name="mine-lora"), # exists elsewhere
]
result = self.db.replace_category_prompts(
CATEGORY, records, expected_ids=self.ids
)
self.assertEqual(
(result["removed"], result["imported"], result["linked"]), (2, 1, 1)
)
self.assertEqual(result["duplicates"], 1)
self.assertEqual(result["images"], 3)
imported = self.db.search_prompts(category=CATEGORY)
self.assertEqual([p["text"] for p in imported], ["fresh"])
self.assertEqual(len(self.db.get_prompt_images(imported[0]["id"])), 2)
mine = self.db.search_prompts(category="portraits")[0]
self.assertEqual(len(self.db.get_prompt_images(mine["id"])), 1)
self.assertEqual(
[row["outcome"] for row in result["rows"]],
["imported", "duplicate", "linked"],
)
# N1: a collection larger than SQLite's bound-variable limit is replaced.
def test_more_imports_than_sqlite_variables(self):
conn = self.db.model.get_connection()
if not hasattr(conn, "setlimit"):
self.skipTest("sqlite3.Connection.setlimit needs Python 3.11")
rows = [
(f"bulk lora {n}", CATEGORY, generate_prompt_hash(f"bulk lora {n}"))
for n in range(1200)
]
with conn:
conn.executemany(
"INSERT INTO prompts(text, category, hash, created_at, updated_at) "
"VALUES(?, ?, ?, '2026-01-01', '2026-01-01')",
rows,
)
conn.setlimit(sqlite3.SQLITE_LIMIT_VARIABLE_NUMBER, 999)
state = self.db.category_import_state(CATEGORY)
self.assertEqual((len(state["ids"]), state["paired"]), (1202, 0))
result = self.db.replace_category_prompts(
CATEGORY, [_record("only one")], expected_ids=state["ids"]
)
self.assertEqual((result["removed"], result["imported"]), (1202, 1))
class LoraImportAPITestCase(LoraAPITestCase):
"""The routes with LoraManager enabled over a disposable LoRA folder."""
async def get_application(self):
app = await super().get_application()
self.lm = _make_lora_manager(self.custom_nodes)
self.loras = self.tmp_root / "loras"
self.loras.mkdir()
self.cache_dir = self.tmp_root / "img_cache"
IntegrationConfig.LORA_MANAGER_ENABLED = True
IntegrationConfig.LORA_MANAGER_PATH = str(self.lm)
IntegrationConfig.CIVITAI_API_KEY = SECRET_KEY
self.requests = []
self.gate = None # a threading.Event a download waits on
self.downloading = threading.Event()
def fake_open(req, timeout=None):
self.requests.append(req)
self.downloading.set()
if self.gate is not None:
self.gate.wait(10)
return _FakeResponse(_png_bytes())
self._patches = [
patch.object(
lora_utils, "find_lora_directories", return_value=[str(self.loras)]
),
patch.object(
lora_utils, "get_lora_image_cache_dir", return_value=self.cache_dir
),
patch.object(lora_utils, "_open_url", fake_open),
patch.object(PromptManagerConfig, "SAFETY_BACKUP", False),
]
for item in self._patches:
item.start()
self.old_ids = _seed_previous_imports(self.api.db)
return app
async def tearDownAsync(self):
if self.gate is not None:
self.gate.set()
for job in self.api._jobs.list():
if job.task is not None and not job.task.done():
await job.task
for item in reversed(self._patches):
item.stop()
await super().tearDownAsync()
def _two_loras(self):
_write_metadata(
self.loras,
"neon",
words=["glow"],
images=["https://civitai.com/a.png"],
prompt="neon street",
)
(self.loras / "neon.png").write_bytes(_png_bytes())
_write_metadata(self.loras, "plain")
async def _post(self, url, body=None):
resp = await self._post_json(url, {} if body is None else body)
return resp, json.loads(await resp.text())
async def _apply_reviewed(self):
resp, plan = await self._post(PREVIEW)
self.assertEqual(resp.status, 200, plan)
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual(resp.status, 202, body)
job = self.api._jobs.get(body["job"]["id"])
return plan, job
@staticmethod
def _sse(text):
return [
json.loads(line[len("data: ") :])
for line in text.splitlines()
if line.startswith("data: ")
]
class LoraImportRouteTests(LoraImportAPITestCase):
async def test_preview_writes_nothing_and_makes_no_requests(self):
self._two_loras()
(self.loras / "broken.metadata.json").write_text("{{{")
db_path = self.api.db.model.db_path
before = {s: _read(db_path + s) for s in ("", "-wal")}
shm = _size(db_path + "-shm")
rows = _snapshot(self.api.db)
resp, plan = await self._post(PREVIEW)
self.assertEqual(resp.status, 200, plan)
self.assertEqual(
{k: plan[k] for k in ("lora_files", "with_metadata", "current_imports")},
{"lora_files": 3, "with_metadata": 2, "current_imports": 2},
)
self.assertEqual(plan["blocked_by_pairs"], 0)
self.assertEqual(plan["planned_downloads"], 1)
self.assertEqual(plan["network"], {"planned_downloads": 1, "api_key": True})
self.assertTrue(plan["token"])
self.assertEqual(plan["scope"]["count"], 2)
self.assertFalse(plan["media"]["deletes_files"])
self.assertEqual(self.requests, [])
self.assertFalse(self.cache_dir.exists())
self.assertEqual({s: _read(db_path + s) for s in ("", "-wal")}, before)
self.assertEqual(_size(db_path + "-shm"), shm)
self.assertEqual(_snapshot(self.api.db), rows)
self.assertNotIn(SECRET_KEY, json.dumps(plan))
self.assertNotIn(str(self.tmp_root), json.dumps(plan))
async def test_preview_requires_the_integration(self):
IntegrationConfig.LORA_MANAGER_ENABLED = False
resp, body = await self._post(PREVIEW)
self.assertEqual((resp.status, body["code"]), (400, "lora-disabled"))
async def test_blocked_preview_offers_no_token(self):
self._two_loras()
self.api.db.save_prompt_pair("old-one prompt", "blurry")
resp, plan = await self._post(PREVIEW)
self.assertEqual(resp.status, 200)
self.assertEqual(plan["blocked_by_pairs"], 1)
self.assertIsNone(plan["token"])
self.assertEqual(plan["protected"]["count"], 1)
async def test_changed_source_after_preview_is_409(self):
self._two_loras()
resp, plan = await self._post(PREVIEW)
rows = _snapshot(self.api.db)
meta = self.loras / "plain.safetensors.metadata.json"
meta.write_text(meta.read_text() + " ")
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (409, "plan-changed"))
self.assertEqual(self.api._jobs.list(), [])
self.assertEqual(self.requests, [])
self.assertEqual(_snapshot(self.api.db), rows)
async def test_apply_imports_through_the_job(self):
self._two_loras()
plan, job = await self._apply_reviewed()
self.assertEqual(
(job.kind, job.title, job.admission),
("lora_import", plan_title(), "records"),
)
await job.task
self.assertEqual(job.state, "finished", job.log)
self.assertEqual(
{k: job.counters[k] for k in lora_import.GROUPS},
{
"imported": 2,
"linked": 0,
"skipped": 0,
"downloaded": 1,
"download_errors": 0,
"errors": 0,
},
)
self.assertFalse(job.cancellable)
self.assertEqual(
self.requests[0].get_header("Authorization"), f"Bearer {SECRET_KEY}"
)
imported = self.api.db.search_prompts(category=CATEGORY)
self.assertEqual(sorted(p["text"] for p in imported), ["neon street", "plain"])
neon = next(p for p in imported if p["text"] == "neon street")
self.assertEqual(len(self.api.db.get_prompt_images(neon["id"])), 2)
self.assertEqual(len(self.api.db.search_prompts(category="portraits")), 1)
# The token is single-use.
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (409, "plan-used"))
async def test_discovery_failure_keeps_previous_imports(self):
self._two_loras()
before = _snapshot(self.api.db)
# Legacy scan: discovery raises once the stream is open.
with patch.object(
lora_utils, "find_lora_directories", side_effect=OSError("share gone")
):
resp = await self.client.request("POST", SCAN)
events = self._sse(await resp.text())
self.assertEqual(events[-1]["type"], "error")
self.assertEqual(_snapshot(self.api.db), before)
# Job: staging fails after the review.
with patch.object(lora_import, "stage", side_effect=OSError("share gone")):
_plan, job = await self._apply_reviewed()
await job.task
self.assertEqual(job.state, "failed")
self.assertIn("previous LoRA imports were kept", job.status_line)
self.assertEqual(job.scope["failure"], "import-failed")
self.assertEqual(_snapshot(self.api.db), before)
async def test_a_failed_replacement_keeps_previous_imports_and_says_so(self):
self._two_loras()
before = _snapshot(self.api.db)
with patch.object(
self.api.db,
"replace_category_prompts",
side_effect=sqlite3.OperationalError("database is locked"),
):
_plan, job = await self._apply_reviewed()
await job.task
self.assertEqual(job.state, "failed")
self.assertEqual(job.counters["errors"], 1)
self.assertEqual(job.counters["downloaded"], 1)
self.assertIn("previous LoRA imports were kept", job.status_line)
self.assertIn("stay in the image cache", job.status_line)
self.assertEqual(_snapshot(self.api.db), before)
async def test_cancel_during_downloads_writes_nothing(self):
self._two_loras()
before = _snapshot(self.api.db)
self.gate = threading.Event()
_plan, job = await self._apply_reviewed()
loop = asyncio.get_running_loop()
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
self.assertTrue(job.summary()["cancellable"])
self.assertTrue(self.api._jobs.cancel(job.id))
self.gate.set()
await job.task
self.assertEqual(job.state, "cancelled")
self.assertEqual(_snapshot(self.api.db), before)
self.assertTrue(any("stay in the image cache" in line for line in job.log))
async def test_empty_reviewed_import_removes_the_collection(self):
# No metadata at all: the review says so, and its apply is the flag.
resp, plan = await self._post(PREVIEW)
self.assertEqual((plan["with_metadata"], plan["current_imports"]), (0, 2))
resp, body = await self._post(APPLY, {"token": plan["token"]})
job = self.api._jobs.get(body["job"]["id"])
await job.task
self.assertEqual(job.state, "finished", job.log)
self.assertEqual(self.api.db.search_prompts(category=CATEGORY), [])
# The legacy scan has no review: an empty replacement is refused.
_seed_previous_imports(self.api.db)
before = _snapshot(self.api.db)
resp = await self.client.request("POST", SCAN)
events = self._sse(await resp.text())
self.assertEqual(events[-1]["type"], "error")
self.assertEqual(_snapshot(self.api.db), before)
async def test_apply_refused_while_a_records_job_holds(self):
self._two_loras()
resp, plan = await self._post(PREVIEW)
async with self.api._jobs.hold("scan", "Metadata scan", admission="records"):
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (409, "busy"))
# The token was not used: it still applies.
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual(resp.status, 202)
await self.api._jobs.get(body["job"]["id"]).task
async def test_safety_backup_policy_runs_before_the_replace(self):
self._two_loras()
order = []
real = self.api.db.replace_category_prompts
async def step(job, db, backup_on):
order.append(("backup", backup_on))
return True, "prompts.db.backup_x"
def replace(*args, **kwargs):
order.append(("replace", None))
return real(*args, **kwargs)
with (
patch.object(PromptManagerConfig, "SAFETY_BACKUP", True),
patch.object(self.api, "_safety_backup_step", side_effect=step),
patch.object(self.api.db, "replace_category_prompts", side_effect=replace),
):
plan, job = await self._apply_reviewed()
await job.task
self.assertEqual(plan["backup"]["policy"], "on")
self.assertEqual(order, [("backup", True), ("replace", None)])
async def test_a_disconnected_legacy_scan_stops_and_holds_until_it_stopped(self):
# I2: the admin page closes its modal while the downloads run.
for number in range(4):
_write_metadata(
self.loras,
f"lora{number}",
images=[f"https://civitai.com/{number}.png"],
)
before = _snapshot(self.api.db)
self.gate = threading.Event()
resp = await self.client.request("POST", SCAN)
await resp.content.readline() # the stream is open
loop = asyncio.get_running_loop()
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
resp.close()
await asyncio.sleep(0.05) # the server sees the closed connection
# Its records hold stays while the in-flight download runs.
self.assertIsNotNone(self.api._jobs.active("lora_import"))
self.gate.set()
for _ in range(500):
if self.api._jobs.active("lora_import") is None:
break
await asyncio.sleep(0.01)
self.assertIsNone(self.api._jobs.active("lora_import"))
# R2 (re-review): the user left and nothing failed, so Activity says so.
job = next(j for j in self.api._jobs.list() if j.kind == "lora_import")
self.assertEqual(job.state, "cancelled")
# Staging stopped at the next LoRA: no download runs after the hold.
served = len(self.requests)
self.assertLess(served, 4)
await asyncio.sleep(0.1)
self.assertEqual(len(self.requests), served)
self.assertEqual(_snapshot(self.api.db), before)
async def test_an_import_sweeps_stale_temp_downloads_from_the_cache(self):
# N7 (re-review): a download killed mid-write leaves a .pm-*.part file;
# the next import removes it once it is an hour old.
self._two_loras()
stale = self.cache_dir / "old_lora" / ".pm-dead.part"
stale.parent.mkdir(parents=True)
stale.write_bytes(b"x")
stamp = time.time() - 2 * 3600
os.utime(stale, (stamp, stamp))
resp = await self.client.request("POST", SCAN)
await resp.text()
self.assertFalse(stale.exists())
async def test_legacy_scan_events_unchanged_on_success(self):
self._two_loras()
(self.loras / "broken.metadata.json").write_text("{{{")
resp = await self.client.request("POST", SCAN)
events = self._sse(await resp.text())
self.assertEqual(resp.status, 200)
progress_keys = {
"type",
"status",
"progress",
"total",
"processed",
"imported",
"skipped",
}
for event in events[:-1]:
self.assertEqual(event["type"], "progress")
self.assertLessEqual(set(event), progress_keys)
self.assertIsInstance(event["status"], str)
self.assertIsInstance(event["progress"], int)
per_record = [e for e in events if "processed" in e]
self.assertEqual([e["processed"] for e in per_record], [1, 2, 3])
self.assertEqual(
events[-1],
{
"type": "complete",
"progress": 100,
"total": 3,
"imported": 2,
"skipped": 1,
},
)
imported = self.api.db.search_prompts(category=CATEGORY)
self.assertEqual(sorted(p["text"] for p in imported), ["neon street", "plain"])
class LoraImportRefusalTests(LoraImportAPITestCase):
"""Every refusal leaves the previous imports as they were."""
async def test_preview_refusals(self):
IntegrationConfig.LORA_MANAGER_PATH = ""
resp, body = await self._post(PREVIEW)
self.assertEqual((resp.status, body["code"]), (400, "lora-path"))
IntegrationConfig.LORA_MANAGER_PATH = str(_make_lora_manager(self.tmp_root))
resp, body = await self._post(PREVIEW)
self.assertEqual((resp.status, body["code"]), (400, "lora-path"))
self.assertNotIn(str(self.tmp_root), json.dumps(body))
IntegrationConfig.LORA_MANAGER_PATH = str(self.lm)
resp, body = await self._post(PREVIEW, {"extra": 1})
self.assertEqual((resp.status, body["code"]), (400, "invalid-request"))
with patch.object(lora_import, "preview", side_effect=OSError("gone")):
resp, body = await self._post(PREVIEW)
self.assertEqual(resp.status, 500)
self.assertFalse(body["success"])
generations = iter([(0, "a"), (1, "a")])
with patch.object(
self.api, "_database_identity", side_effect=lambda db: next(generations)
):
resp, body = await self._post(PREVIEW)
self.assertEqual((resp.status, body["code"]), (409, "storage-changed"))
async def test_apply_refusals(self):
self._two_loras()
before = _snapshot(self.api.db)
resp, body = await self._post(APPLY, {"token": ""})
self.assertEqual((resp.status, body["code"]), (400, "invalid-request"))
resp, body = await self._post(APPLY, {"token": "unknown"})
self.assertEqual((resp.status, body["code"]), (410, "plan-expired"))
other = self.api._maintenance_plans.issue(
"optimize", "f", {"backup": "off"}, self.api._database_identity(self.api.db)
)
resp, body = await self._post(APPLY, {"token": other})
self.assertEqual((resp.status, body["code"]), (410, "plan-expired"))
resp, plan = await self._post(PREVIEW)
with patch.object(lora_import, "discover", side_effect=OSError("gone")):
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (500, "apply-failed"))
short = ("database folder", "the safety backup", 10, 1)
with patch("py.api.lora_import_routes.disk_plan", return_value=({}, short, 0)):
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (507, "insufficient-space"))
self.api._db_generation += 1
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (409, "storage-changed"))
self.assertEqual(self.api._jobs.list(), [])
self.assertEqual(_snapshot(self.api.db), before)
async def _failed_job(self, **patches):
before = _snapshot(self.api.db)
_plan, job = await self._apply_reviewed()
await job.task
self.assertEqual(job.state, "failed", job.log)
self.assertEqual(_snapshot(self.api.db), before)
return job
async def test_start_refused_when_the_integration_changed_since_the_review(self):
# M5: disabled, or moved to another install, after the review.
self._two_loras()
before = _snapshot(self.api.db)
resp, plan = await self._post(PREVIEW)
IntegrationConfig.LORA_MANAGER_ENABLED = False
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (409, "plan-changed"))
self.assertIn("integration changed", body["error"])
IntegrationConfig.LORA_MANAGER_ENABLED = True
resp, plan = await self._post(PREVIEW)
other = _make_lora_manager(self.custom_nodes / "other")
target = self.custom_nodes / "Other-Lora-Manager"
other.rename(target)
IntegrationConfig.LORA_MANAGER_PATH = str(target)
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual((resp.status, body["code"]), (409, "plan-changed"))
self.assertEqual(self.api._jobs.list(), [])
self.assertEqual(_snapshot(self.api.db), before)
async def test_imports_changed_after_the_review_fail_the_job(self):
self._two_loras()
resp, plan = await self._post(PREVIEW)
self.api.db.save_prompt(
text="late", category=CATEGORY, prompt_hash=generate_prompt_hash("late")
)
before = _snapshot(self.api.db)
resp, body = await self._post(APPLY, {"token": plan["token"]})
job = self.api._jobs.get(body["job"]["id"])
await job.task
self.assertEqual((job.state, job.scope["failure"]), ("failed", "plan-changed"))
self.assertEqual(_snapshot(self.api.db), before)
async def test_a_pair_saved_after_the_review_blocks_the_job(self):
self._two_loras()
resp, plan = await self._post(PREVIEW)
self.api.db.save_prompt_pair("old-one prompt", "blurry")
before = _snapshot(self.api.db)
resp, body = await self._post(APPLY, {"token": plan["token"]})
job = self.api._jobs.get(body["job"]["id"])
await job.task
self.assertEqual(
(job.state, job.scope["failure"]), ("failed", "blocked-by-pairs")
)
self.assertIn("Saved pairs", job.status_line)
self.assertEqual(_snapshot(self.api.db), before)
async def test_nothing_prepared_keeps_the_collection(self):
self._two_loras()
with patch.object(lora_import, "build_record", side_effect=ValueError("x")):
job = await self._failed_job()
self.assertEqual(job.scope["failure"], "nothing-to-import")
# M4: the job keeps what staging skipped, by name, though it failed.
self.assertEqual(job.counters["skipped"], 2)
self.assertEqual(
sorted(row["detail"] for row in job.outcomes["skipped"]), ["neon", "plain"]
)
async def _change_during_download(self, change):
"""Start the reviewed import, run ``change()`` while a download is held."""
self._two_loras()
before = _snapshot(self.api.db)
self.gate = threading.Event()
_plan, job = await self._apply_reviewed()
loop = asyncio.get_running_loop()
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
change()
self.gate.set()
await job.task
self.assertEqual(job.state, "failed", job.log)
self.assertEqual(job.scope["failure"], "plan-changed")
self.assertIn("Review the import again", job.status_line)
self.assertEqual(_snapshot(self.api.db), before)
return job
async def test_files_changed_while_staging_fail_the_job(self):
# I1: LoraManager rewrites a metadata file while the downloads run.
meta = self.loras / "plain.safetensors.metadata.json"
await self._change_during_download(
lambda: meta.write_text(meta.read_text() + " ")
)
async def test_a_file_removed_while_staging_fails_the_job(self):
meta = self.loras / "plain.safetensors.metadata.json"
await self._change_during_download(meta.unlink)
async def test_a_file_added_while_staging_fails_the_job(self):
await self._change_during_download(lambda: _write_metadata(self.loras, "late"))
async def test_legacy_scan_refuses_files_changed_while_it_stages(self):
self._two_loras()
before = _snapshot(self.api.db)
self.gate = threading.Event()
meta = self.loras / "plain.safetensors.metadata.json"
resp = await self.client.request("POST", SCAN)
loop = asyncio.get_running_loop()
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
meta.write_text(meta.read_text() + " ")
self.gate.set()
events = self._sse(await resp.text())
self.assertEqual(events[-1]["type"], "error")
self.assertIn("changed during the scan", events[-1]["message"])
self.assertEqual(_snapshot(self.api.db), before)
async def test_a_failed_safety_backup_changes_nothing(self):
self._two_loras()
from py.api import maintenance
with (
patch.object(PromptManagerConfig, "SAFETY_BACKUP", True),
patch.object(
maintenance,
"take_safety_backup",
side_effect=maintenance.SafetyBackupFailed("no room"),
),
):
job = await self._failed_job()
self.assertEqual(job.scope["failure"], "backup-failed")
async def test_cancel_right_after_the_start_writes_nothing(self):
self._two_loras()
before = _snapshot(self.api.db)
resp, plan = await self._post(PREVIEW)
resp, body = await self._post(APPLY, {"token": plan["token"]})
job = self.api._jobs.get(body["job"]["id"])
self.api._jobs.cancel(job.id) # at once, while it stages
await job.task
self.assertEqual(job.state, "cancelled")
self.assertEqual(_snapshot(self.api.db), before)
def plan_title():
return lora_import.JOB_TITLE
def _read(path):
try:
with open(path, "rb") as handle:
return handle.read()
except FileNotFoundError:
return None
def _size(path):
return os.path.getsize(path) if os.path.exists(path) else None
class LoraImportRestoreTests(LoraImportAPITestCase):
"""The import and a restore (#249 S8) never run beside each other."""
async def test_a_restore_is_refused_while_the_import_runs(self):
self._two_loras()
self.gate = threading.Event()
_plan, job = await self._apply_reviewed()
loop = asyncio.get_running_loop()
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
resp = await self.client.request("POST", "/prompt_manager/restore")
body = json.loads(await resp.text())
self.assertEqual(resp.status, 409)
self.assertIn(lora_import.JOB_TITLE, body["error"])
# The workspace restore job is refused for the same running import.
blocker = self.api._jobs.blocker(RESTORE_JOB_KIND, admission="exclusive")
self.assertIs(blocker, job)
self.gate.set()
await job.task
self.assertEqual(job.state, "finished")
async def test_the_import_is_refused_while_a_restore_runs(self):
self._two_loras()
resp, plan = await self._post(PREVIEW)
self.assertEqual(resp.status, 200, plan)
before = _snapshot(self.api.db)
async with self.api._jobs.hold(
RESTORE_JOB_KIND, "Restore library", admission="exclusive"
):
# A review reads the library: refused, not run beside the switch.
resp, body = await self._post(PREVIEW)
self.assertEqual(resp.status, 409, body)
self.assertEqual(body["code"], "busy")
self.assertEqual(body["running"]["title"], "Restore library")
resp, body = await self._post(APPLY, {"token": plan["token"]})
self.assertEqual(resp.status, 409, body)
self.assertEqual(body["code"], "busy")
# The admin page's scan says so in its own error event.
resp = await self.client.request("POST", SCAN)
events = self._sse(await resp.text())
self.assertEqual(events[-1]["type"], "error")
self.assertIn("Restore library", events[-1]["message"])
self.assertEqual(_snapshot(self.api.db), before)
self.assertIsNone(self.api._jobs.active("lora_import"))
async def test_a_running_review_holds_a_lease_a_restore_waits_for(self):
self._two_loras()
seen = []
real_preview = lora_import.preview
def preview(*args):
seen.append(self.api._jobs.leases)
return real_preview(*args)
with patch.object(lora_import, "preview", preview):
resp, plan = await self._post(PREVIEW)
self.assertEqual(resp.status, 200, plan)
self.assertEqual(seen, [{LORA_PREVIEW_LEASE: 1}])
self.assertEqual(self.api._jobs.leases, {})
class LoraImportStageTests(unittest.TestCase):
def setUp(self):
self.tmp = Path(tempfile.mkdtemp())
self.addCleanup(lambda: __import__("shutil").rmtree(self.tmp, True))
def test_discover_sorts_and_drops_repeats(self):
a = self.tmp / "a"
b = self.tmp / "a" / "b"
b.mkdir(parents=True)
_write_metadata(b, "zeta")
_write_metadata(a, "alpha")
with patch.object(
lora_utils, "find_lora_directories", return_value=[str(a), str(b)]
):
files = lora_import.discover(str(self.tmp))
self.assertEqual([f.name for f in files][0], "alpha.safetensors.metadata.json")
self.assertEqual(len(files), 2)
def test_fingerprint_follows_size_and_mtime(self):
meta = _write_metadata(self.tmp, "one")
first = lora_import.fingerprint([meta])
self.assertEqual(lora_import.fingerprint([meta]), first)
meta.write_text(meta.read_text() + " ")
self.assertNotEqual(lora_import.fingerprint([meta]), first)
meta.unlink()
self.assertNotEqual(lora_import.fingerprint([meta]), first)
def test_stage_cancel_before_a_file_raises(self):
meta = _write_metadata(self.tmp, "one")
with self.assertRaises(lora_import.ImportCancelled):
lora_import.stage([meta], self.tmp, cancelled=lambda: True)
if __name__ == "__main__":
unittest.main()
+196 -1
View File
@@ -12,6 +12,7 @@ import shutil
import sys
import tempfile
import threading
import time
import types
import unittest
import urllib.parse
@@ -524,6 +525,114 @@ class TestDownloadOne(unittest.TestCase):
self.assertFalse(self.target.exists())
class TestDownloadIsAtomic(unittest.TestCase):
"""I2 (S10 review): a cached image is never seen half written."""
def setUp(self):
self.tmpdir = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.tmpdir, True)
self.target = self.tmpdir / "out.jpg"
def test_the_image_appears_only_once_complete(self):
seen = []
real_replace = os.replace
def replace(source, destination):
# At the moment the name appears, the temp copy is whole and closed.
seen.append((Path(source).parent, self.target.exists()))
with open(source, "rb") as handle:
self.assertEqual(handle.read(2), b"\xff\xd8")
return real_replace(source, destination)
opener = _FakeOpener(_FakeResponse(_png_bytes()))
with (
patch.object(lora_utils, "_open_url", opener),
patch.object(lora_utils.os, "replace", side_effect=replace),
):
result = lora_utils._download_one(
"https://civitai.com/api/download/1", self.target, ""
)
self.assertEqual(seen, [(self.tmpdir, False)])
self.assertEqual(os.path.realpath(result), os.path.realpath(self.target))
self.assertEqual(sorted(p.name for p in self.tmpdir.iterdir()), ["out.jpg"])
def test_a_failed_write_leaves_neither_the_image_nor_a_temp_file(self):
opener = _FakeOpener(_FakeResponse(_png_bytes()))
with (
patch.object(lora_utils, "_open_url", opener),
patch.object(lora_utils.os, "replace", side_effect=OSError("disk full")),
):
result = lora_utils._download_one(
"https://civitai.com/api/download/1", self.target, ""
)
self.assertIsNone(result)
self.assertEqual(list(self.tmpdir.iterdir()), [])
def test_an_unreadable_image_leaves_no_temp_file(self):
opener = _FakeOpener(_FakeResponse(b"not an image"))
with patch.object(lora_utils, "_open_url", opener):
result = lora_utils._download_one(
"https://civitai.com/api/download/1", self.target, ""
)
self.assertIsNone(result)
self.assertEqual(list(self.tmpdir.iterdir()), [])
@unittest.skipIf(os.name == "nt", "POSIX file modes")
def test_the_image_gets_the_permissions_a_plain_save_gives(self):
# S10 re-review N7: the cached JPEG honours the umask like img.save did.
previous = os.umask(0o022)
self.addCleanup(os.umask, previous)
opener = _FakeOpener(_FakeResponse(_png_bytes()))
with patch.object(lora_utils, "_open_url", opener):
lora_utils._download_one(
"https://civitai.com/api/download/1", self.target, ""
)
self.assertEqual(self.target.stat().st_mode & 0o777, 0o644)
class TestStaleDownloadSweep(unittest.TestCase):
"""S10 re-review N7: a killed download's ``.pm-*.part`` is swept later."""
def setUp(self):
self.cache = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.cache, True)
self.lora = self.cache / "my_lora"
self.lora.mkdir()
def _file(self, path, age_seconds):
path.write_bytes(b"x")
stamp = time.time() - age_seconds
os.utime(path, (stamp, stamp))
return path
def test_only_old_temp_downloads_are_removed(self):
old = self._file(self.lora / ".pm-abc123.part", 2 * 3600)
old_top = self._file(self.cache / ".pm-top.part", 2 * 3600)
fresh = self._file(self.lora / ".pm-new.part", 60)
image = self._file(self.lora / "abc.jpg", 2 * 3600)
other = self._file(self.lora / "keep.part", 2 * 3600)
prefixed = self._file(self.lora / ".pm-keep.jpg", 2 * 3600)
deep = self.lora / "nested"
deep.mkdir()
nested = self._file(deep / ".pm-deep.part", 2 * 3600)
lora_utils.sweep_stale_downloads(self.cache)
self.assertFalse(old.exists())
self.assertFalse(old_top.exists())
for kept in (fresh, image, other, prefixed, nested):
self.assertTrue(kept.exists(), kept.name)
def test_a_missing_cache_is_fine(self):
lora_utils.sweep_stale_downloads(self.cache / "absent")
def test_a_file_that_cannot_be_removed_is_left(self):
old = self._file(self.lora / ".pm-abc.part", 2 * 3600)
with patch.object(lora_utils.os, "remove", side_effect=PermissionError):
lora_utils.sweep_stale_downloads(self.cache)
self.assertTrue(old.exists())
class _FakeHTTPSHandler(urllib.request.HTTPSHandler):
"""Serves canned responses through a real urllib opener, recording requests.
@@ -817,9 +926,95 @@ class TestDetectLoraManager(unittest.TestCase):
self.assertTrue(lora_utils._looks_like_lora_manager(lm))
def test_nothing_found(self):
with patch.object(lora_utils, "find_comfyui_root", return_value=None):
with (
patch.object(lora_utils, "find_comfyui_root", return_value=None),
patch.object(lora_utils, "custom_nodes_directories", return_value=[]),
):
self.assertIsNone(lora_utils.detect_lora_manager())
def test_auto_detects_in_an_extra_custom_nodes_folder(self):
# #144: ComfyUI's main custom_nodes has no LoraManager; an extra
# custom_nodes path registered through folder_paths (extra model
# paths, --base-directory) has it.
root = _make_comfy_root(self.tmp)
extra = self.tmp / "shared_nodes"
extra.mkdir()
lm = _make_lora_manager(extra, "ComfyUI-Lora-Manager")
fake = types.SimpleNamespace(
base_path=str(root),
get_folder_paths=lambda name: (
[str(root / "custom_nodes"), str(extra)]
if name == "custom_nodes"
else []
),
)
with (
patch.dict(sys.modules, {"folder_paths": fake}),
patch.object(lora_utils, "find_comfyui_root", return_value=root),
):
self.assertEqual(lora_utils.detect_lora_manager(), str(lm.resolve()))
def test_main_custom_nodes_wins_over_an_extra_folder(self):
root = _make_comfy_root(self.tmp)
main = _make_lora_manager(root / "custom_nodes")
extra = self.tmp / "shared_nodes"
extra.mkdir()
_make_lora_manager(extra)
with (
patch.object(lora_utils, "find_comfyui_root", return_value=root),
patch.object(
lora_utils,
"custom_nodes_directories",
return_value=[(root / "custom_nodes").resolve(), extra.resolve()],
),
):
self.assertEqual(lora_utils.detect_lora_manager(), str(main.resolve()))
def test_an_entry_that_cannot_be_checked_is_skipped(self):
# S10 review N5: a permission error on one entry, not a missing folder.
root = _make_comfy_root(self.tmp)
blocked = root / "custom_nodes" / "a-lora-manager"
blocked.mkdir()
lm = _make_lora_manager(root / "custom_nodes", "b-Lora-Manager")
real_is_dir = Path.is_dir
def is_dir(path):
if path == blocked:
raise PermissionError(13, "Permission denied", str(path))
return real_is_dir(path)
with (
patch.object(lora_utils, "find_comfyui_root", return_value=root),
patch.object(lora_utils, "custom_nodes_directories", return_value=[]),
patch.object(Path, "is_dir", is_dir),
):
self.assertEqual(lora_utils.detect_lora_manager(), str(lm.resolve()))
def test_a_folder_that_cannot_be_listed_is_skipped(self):
root = _make_comfy_root(self.tmp)
with (
patch.object(lora_utils, "find_comfyui_root", return_value=root),
patch.object(lora_utils, "custom_nodes_directories", return_value=[]),
patch.object(Path, "iterdir", side_effect=PermissionError(13, "denied")),
):
self.assertIsNone(lora_utils.detect_lora_manager())
def test_an_unreadable_extra_folder_is_skipped(self):
root = _make_comfy_root(self.tmp)
missing = self.tmp / "gone"
extra = self.tmp / "shared_nodes"
extra.mkdir()
lm = _make_lora_manager(extra)
with (
patch.object(lora_utils, "find_comfyui_root", return_value=root),
patch.object(
lora_utils,
"custom_nodes_directories",
return_value=[missing, extra.resolve()],
),
):
self.assertEqual(lora_utils.detect_lora_manager(), str(lm.resolve()))
class TestFindLoraDirectories(unittest.TestCase):
+3 -2
View File
@@ -15,14 +15,15 @@ test('real-ComfyUI suite registers Classic and Nodes 2.0 startup projects', () =
assert.match(output, /\[nodes2-start\]/);
});
test('execution script discovers only the new matrix and preserves all 78 original stock cases plus the native editor host case', () => {
test('execution script discovers only the new matrix and preserves all 78 original stock cases plus the native editor and renderer host cases', () => {
const manifest = JSON.parse(readFileSync(new URL('../../tools/vendor/package.json', import.meta.url), 'utf8'));
assert.equal(manifest.scripts['test:executions'], 'playwright test --config playwright.execution.config.mjs');
const list = config => execFileSync(process.execPath, ['tools/vendor/node_modules/@playwright/test/cli.js',
'test', '--config', config, '--list'], { cwd: root, encoding: 'utf8',
env: { ...process.env, PM_COMFY_ROOT: '/unused-pinned-checkout' } });
const stock = list('tools/vendor/comfyui.config.mjs');
assert.match(stock, /Total: 82 tests in 8 files/); // + local Inspect host case (inspect-sender.spec.mjs, 2 projects)
// + local Inspect host case (inspect-sender.spec.mjs, 2 projects), + the S10 renderer case (prompt-bridge.spec.mjs, 2 projects)
assert.match(stock, /Total: 84 tests in 8 files/);
assert.doesNotMatch(stock, /execution-history/);
assert.doesNotMatch(stock, /execution-outputs/);
assert.doesNotMatch(stock, /library-location/); // it starts its own fresh host (#249 S6b)
+10 -4
View File
@@ -163,7 +163,7 @@
},
"workspace/activity-job-view.mjs": {
"lines": 97,
"branches": 86,
"branches": 87,
"functions": 98,
"statements": 97
},
@@ -415,7 +415,7 @@
},
"workspace/maintenance-review.mjs": {
"lines": 94,
"branches": 81,
"branches": 85,
"functions": 90,
"statements": 94
},
@@ -503,11 +503,17 @@
"functions": 89,
"statements": 98
},
"workspace/system-integrations.mjs": {
"lines": 96,
"branches": 88,
"functions": 91,
"statements": 96
},
"workspace/system-maintenance.mjs": {
"lines": 92,
"lines": 93,
"branches": 80,
"functions": 92,
"statements": 92
"statements": 93
},
"workspace/system-overview.mjs": {
"lines": 96,
+11 -2
View File
@@ -47,10 +47,19 @@ function redraw(app,node) {
app.graph?.setDirtyCanvas?.(true,true);
}
export function createPromptAdapter(app) {
// The renderer this window draws nodes with (System › Integrations, S10): Nodes 2.0 while LiteGraph's Vue node mode is
// on, else Classic; unknown (omitted) without LiteGraph.
function renderer(runtime) {
const graph = runtime?.LiteGraph;
if (!graph) return {};
return {renderer:graph.vueNodesMode === true ? 'nodes2' : 'classic'};
}
export function createPromptAdapter(app, {runtime = globalThis} = {}) {
const snapshots = new WeakMap();
function listTargets() {
return nodes(app.graph).filter(node => schemas.has(node.type)).map(node => ({...target(node),roles:Object.keys(schemas.get(node.type))}));
const drawn = renderer(runtime);
return nodes(app.graph).filter(node => schemas.has(node.type)).map(node => ({...target(node),roles:Object.keys(schemas.get(node.type)),...drawn}));
}
function preview(targetId,rawRoles) {
const roles = validateRoles(rawRoles);
+6 -2
View File
@@ -82,8 +82,8 @@ const OVERTAKEN = new Set(['plan-changed', 'storage-changed', 'plan-expired']);
/**
* Where a failed apply's Review again goes (S7 review M2): a maintenance plan the library overtook is reviewed again
* (`{operation}`); a duplicate cleanup whose sources or plan changed needs a new search, in Maintenance (`{section}`);
* otherwise null.
* (`{operation}`); a duplicate cleanup whose sources or plan changed needs a new search, in Maintenance (`{section}`); a
* LoRA import the files or saved pairs overtook is reviewed again in Integrations (`{section}`, S10); otherwise null.
*/
export function reviewAgainOf(job) {
if (job.state !== 'failed') return null;
@@ -94,6 +94,10 @@ export function reviewAgainOf(job) {
if (job.kind === 'duplicate_cleanup' && (OVERTAKEN.has(failure) || failure === 'sources-changed')) {
return { section: '#/system/maintenance' };
}
// System S10: a LoRA import whose files or imports changed, or that saved pairs block, is reviewed in Integrations.
if (job.kind === 'lora_import' && (OVERTAKEN.has(failure) || failure === 'blocked-by-pairs')) {
return { section: '#/system/integrations' };
}
return null;
}
+23 -1
View File
@@ -13,7 +13,8 @@
* `scan` is exactly the D56 view and `image-tags` the D89 one. `thumbnails` counts Generated (green), Skipped and
* Errors (red). System S7: `maintenance` (a reviewed plan's apply) counts Changed, Protected and Errors and, when it
* failed, ends with the server's reason; `duplicates` (the search) counts Groups found; `duplicate_cleanup` counts
* Removed, Kept and Errors. Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
* Removed, Kept and Errors; S10: `lora_import` counts Imported, Linked to existing, Skipped, Images downloaded, Download
* errors and Errors and, when it failed, ends with the server's reason (the previous imports were kept). Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
* the generic descriptor: title-based copy and no tiles or outcomes, so it never shows another kind's zeros (S1 review
* M6, S2 review M3). `viewOf(job)` keeps only the groups the job itself counts, so a hold (no groups) has none either.
*
@@ -46,6 +47,8 @@ export const groupsOf = job => {
};
/** "3 removed" (System S7: a duplicate cleanup). */
export const removedOf = job => counted(job, 'removed', 'removed');
/** "12 imported" (System S10: the LoRA reference import). */
export const importedOf = job => counted(job, 'imported', 'imported');
/** "2 output folders": what a scan or a thumbnail job walks. */
function foldersOf(job) {
@@ -85,6 +88,11 @@ const tagLine = job => `${taggedOf(job)} · ${errorsOf(job)}`;
const thumbnailLine = job => `${generatedOf(job)} · ${errorsOf(job)}`;
const changedLine = job => `${changedOf(job)} · ${errorsOf(job)}`;
const cleanupLine = job => `${removedOf(job)} · ${errorsOf(job)}`;
const importLine = job => `${importedOf(job)} · ${errorsOf(job)}`;
/** A job that ends with the server's reason when it failed (the LoRA import keeps the old collection and says so). */
const reasonFinish = (name, kept, line) => job => (job.state === 'failed' && job.status_line
? `${name} failed · ${job.status_line}` : namedFinish(() => name, kept, line)(job));
/** A maintenance apply ends with its counters, or, when it failed, the server's reason (plan-changed: review again). */
function maintenanceFinish(job) {
@@ -182,6 +190,20 @@ export const KINDS = Object.freeze({
openable: groups('kept'),
log: 'Cleanup log',
}),
// System S10: the reviewed LoRA reference import; cancellable only while it downloads (the server says when).
lora_import: Object.freeze({
...GENERIC,
pillVerb: 'Importing LoRAs',
finishToast: reasonFinish('LoRA import', importedOf, importLine),
line: importLine,
kept: importedOf,
tiles: tiles([['imported', 'Imported', 'added'], ['linked', 'Linked to existing', ''], ['skipped', 'Skipped', 'warn'],
['downloaded', 'Images downloaded', ''], ['download_errors', 'Download errors', 'warn'], ['errors', 'Errors', 'bad']]),
outcomes: tiles([['imported', 'Imported'], ['errors', 'Errors'], ['skipped', 'Skipped'], ['linked', 'Linked to existing']]),
log: 'Import log',
cancelLabel: 'Cancel import',
cancel: 'Cancel import',
}),
});
/** The descriptor of a job `kind`; any kind without one gets the generic descriptor. */
+8 -6
View File
@@ -55,10 +55,10 @@ export const OPERATION_TITLES = Object.freeze({
workflow_storage: 'Store each workflow once',
});
// The review is gone or no longer matches the database: review it again.
const REVIEW_AGAIN = new Set(['plan-expired', 'storage-changed']);
const REVIEW_AGAIN = new Set(['plan-expired', 'storage-changed', 'plan-changed']);
// Refusals whose server message is the whole truth.
const SERVER_SAYS = new Set(['insufficient-space', 'nothing-to-do', 'too-large', 'invalid-request', 'not-found',
'job-running', 'no-groups', 'apply-failed', 'sources-changed']);
'job-running', 'no-groups', 'apply-failed', 'sources-changed', 'lora-disabled', 'lora-path']);
const number = value => value.toLocaleString('en-US');
@@ -169,13 +169,15 @@ async function holding(trigger, busyText, work) {
}
/**
* Apply a reviewed plan's `token`; the job opens in Activity. `reviewAgain()` reviews the operation once more when the
* server says the review expired or the database was replaced.
* Apply a reviewed plan's `token` (at `url`: maintenance's apply, or the LoRA import's, S10); the job opens in Activity.
* `reviewAgain()` reviews the operation once more when the server says the review expired, the database was replaced
* or (the LoRA import) its files changed.
*/
export async function applyPlan({ api, store, scope, token, reviewAgain, notify = showWorkspaceToast, navigate = navigateTo }) {
export async function applyPlan({ api, store, scope, token, reviewAgain, notify = showWorkspaceToast, navigate = navigateTo,
url = APPLY_URL }) {
let result;
try {
result = await api.post(APPLY_URL, { token });
result = await api.post(url, { token });
} catch (error) {
if (scope.signal.aborted) return;
const refusal = applyRefusal(error);
+446
View File
@@ -0,0 +1,446 @@
/**
* System › Integrations (#249 System S10 Task 27): the concept's `renderSystem` "integrations" body (app.js 682–684).
*
* - "ComfyUI": Default send destination, Host compatibility and Return to canvas after send. D106 (Q8): the send
* bridge always asks for the window and node, and the nodes always follow ComfyUI's own renderer, so only "Ask each
* time" and "Follow host renderer" can be chosen; the other options and the return toggle are disabled, with the
* reason under the row, and nothing is stored for them. Host compatibility also says which renderer the open
* ComfyUI windows use, as the send bridge reports it (`renderer` on each Prompt Manager node it lists).
* - "LoRA & resources": "LoraManager integration" (enabled, a host draft) says where LoraManager was detected;
* D106 adds the read-only "LoraManager path" row, as on the legacy page. "Automatically include trigger words"
* (a host draft) says whether the trigger words are loaded. The three are separate facts (`/lora/status`).
* D114: the CivitAI key field is never filled in; typing drafts a new key, a blank field keeps the saved one, and
* "Clear saved key" (shown while a key is saved) drafts its removal; the line under it says "A key is saved." or "No key
* saved." All of them are
* saved by the one Save row (`/lora/enable` with the settings revision, system.mjs).
* - "LoRA reference data" → "Review import" (D107; the concept's button only simulated a job): the server
* reviews the import (`POST /lora/import/preview`, read-only) and the shared review dialog shows what it replaces,
* what saved pairs block, what it downloads and the backup box; Start runs the "LoRA reference import" Activity job.
*/
import { createButton } from '../shared/toolkit/core.js';
import { createBridgeClient } from '../shared/toolkit/prompt-bridge.js';
import { settingRow, operationRow } from './system-controls.mjs';
import { HOST_UNAVAILABLE } from './system-appearance.mjs';
import { closeWorkspaceDialog, openContentDialog, openReviewDialog } from './workspace-dialogs.mjs';
import { BACKUP_COPY, applyPlan, backupText, planNote, protectedText, reviewRefusal } from './maintenance-review.mjs';
import { downloadDatabaseBackup } from './activity-scan.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { navigateTo } from './state.mjs';
import { noun } from './plural.mjs';
export const LORA_STATUS_URL = '/prompt_manager/lora/status';
export const LORA_ENABLE_URL = '/prompt_manager/lora/enable';
export const IMPORT_PREVIEW_URL = '/prompt_manager/lora/import/preview';
export const IMPORT_APPLY_URL = '/prompt_manager/lora/import/apply';
export const IMPORT_TITLE = 'Review LoRA import';
export const IMPORT_START = 'Start import';
/** The host draft keys this section saves through `/lora/enable`; every other host key goes to `/settings`. */
export const LORA_KEYS = Object.freeze(['lora_enabled', 'lora_trigger_words_enabled', 'civitai_api_key', 'clear_civitai_api_key']);
/** D106: why the options the bridge cannot honour are disabled. */
export const REASONS = Object.freeze({
send: 'Sending always asks for the ComfyUI window and node, so “Ask each time” is the only option.',
renderer: 'Prompt Manager nodes always follow ComfyUI’s own renderer setting.',
returnCanvas: 'Not available: sending applies the text in ComfyUI and keeps this page open.',
});
export const RENDERER_COPY = Object.freeze({
looking: 'Looking for open ComfyUI windows…',
none: 'No ComfyUI window is open in this browser, so its renderer is not known.',
noNodes: 'ComfyUI is open; add a Prompt Manager node to see which renderer it uses.',
unavailable: 'The renderer could not be checked in this browser.',
});
const RENDERER_NAMES = Object.freeze({ classic: 'Classic', nodes2: 'Nodes 2.0' });
export const KEY_COPY = Object.freeze({ saved: 'A key is saved.', none: 'No key saved.',
clearing: 'The saved key is removed when you save.', replacing: 'The new key replaces the saved one when you save.' });
export const NOTHING_TO_IMPORT = 'There is nothing to import; nothing was changed.';
export const BLOCKED_NOTE = 'Delete or change the saved pairs that use these prompts, then review the import again. Nothing was changed.';
export const CACHE_NOTE = 'Example images downloaded before a failure stay in the image cache.';
const number = value => value.toLocaleString('en-US');
function el(tag, text, className) {
const node = document.createElement(tag);
if (text !== undefined) node.textContent = text;
if (className) node.className = className;
return node;
}
/** The host values a `/lora/status` answer gives: drafted ones (enabled, trigger words, clear) and read-only facts. */
export function loraValues(status) {
return {
lora_enabled: status?.enabled === true,
lora_trigger_words_enabled: status?.trigger_words_enabled === true,
clear_civitai_api_key: false,
lora_path: typeof status?.path === 'string' ? status.path : '',
lora_detected: status?.detected === true,
lora_detected_path: typeof status?.detected_path === 'string' ? status.detected_path : '',
lora_detected_usable: status?.detected_usable !== false,
lora_has_key: status?.has_civitai_api_key === true,
lora_cache_loaded: status?.trigger_cache_loaded === true,
};
}
/** Host draft changes split by route: `{lora, settings}`, each null when it has none. */
export function splitHostChanges(changes) {
const lora = {};
const settings = {};
for (const [key, value] of Object.entries(changes ?? {})) (LORA_KEYS.includes(key) ? lora : settings)[key] = value;
return { lora: Object.keys(lora).length ? lora : null, settings: Object.keys(settings).length ? settings : null };
}
/**
* The `/lora/enable` body for the drafted LoRA `changes` over the `saved` host values: the whole LoRA state (the route
* saves all of it), the saved path, and the key only when one was typed (a blank or absent key keeps the saved one).
*/
export function loraPayload(changes, saved) {
const value = key => (Object.hasOwn(changes, key) ? changes[key] : saved?.[key]);
const body = { enabled: value('lora_enabled') === true, trigger_words_enabled: value('lora_trigger_words_enabled') === true,
path: saved?.lora_path || '' };
if (changes.clear_civitai_api_key === true) body.clear_civitai_api_key = true;
else if (typeof changes.civitai_api_key === 'string' && changes.civitai_api_key.trim()) body.civitai_api_key = changes.civitai_api_key.trim();
return body;
}
/**
* Write host `changes` in their two parts (S10 review I3): the LoRA keys through `postLora` (`/lora/enable`), then the
* other keys through `postSettings` (`/settings`), each with the revision the previous write left. Resolves with the
* last answer when everything saved. When only one part saved, rejects with the other part's error carrying
* `partial: {saved: [keys], revision}`, so the store keeps what saved and adopts the new revision; a
* conflict on the first write (another page changed config.json) stops before the second.
*/
export async function writeHostInParts({ changes, saved, expected_revision, postLora, postSettings }) {
const { lora, settings } = splitHostChanges(changes);
let revision = expected_revision;
let result = null;
const written = [];
let refused = null;
if (lora) {
try {
result = await postLora({ ...loraPayload(lora, saved), expected_revision: revision });
if (typeof result?.revision === 'string') revision = result.revision;
written.push(...Object.keys(lora));
} catch (error) {
if (!settings || error?.code === 'settings-changed') throw error;
refused = error;
}
}
if (settings) {
try {
result = await postSettings({ ...settings, expected_revision: revision });
if (typeof result?.revision === 'string') revision = result.revision;
written.push(...Object.keys(settings));
} catch (error) {
if (!written.length) throw error;
throw Object.assign(error, { partial: { saved: written, revision } });
}
}
if (refused) {
throw Object.assign(refused, { partial: { saved: written, revision } });
}
return result;
}
/** The renderer line for discovered bridge hosts (D106). */
export function rendererText(hosts) {
if (!Array.isArray(hosts) || hosts.length === 0) return RENDERER_COPY.none;
const seen = new Set();
for (const host of hosts) for (const target of host?.targets ?? []) if (RENDERER_NAMES[target?.renderer]) seen.add(target.renderer);
if (!seen.size) return RENDERER_COPY.noNodes;
const names = ['classic', 'nodes2'].filter(name => seen.has(name)).map(name => RENDERER_NAMES[name]);
return `Detected: ${names.join(' and ')} (${number(hosts.length)} ComfyUI ${noun(hosts.length, 'window')}).`;
}
/** The LoraManager row's line: where it was detected, or that it was not. */
export function detectionText(values) {
if (values.lora_detected === undefined) return null;
if (!values.lora_detected) return 'Not detected in ComfyUI’s custom_nodes folders.';
if (values.lora_detected_usable === false) {
// A linked install outside custom_nodes is found but cannot be enabled (S10 review N5).
return `Found at ${values.lora_detected_path || 'a linked folder'}, but outside ComfyUI’s custom_nodes folders, so it can’t be enabled.`;
}
return values.lora_detected_path ? `Detected: ${values.lora_detected_path}.` : 'Detected.';
}
/** The trigger-words row's line (the cache, separate from the setting). */
export function cacheText(values) {
if (values.lora_cache_loaded === undefined) return null;
return values.lora_cache_loaded ? 'Trigger words are loaded.' : 'Trigger words are not loaded.';
}
/** D114: the key line for the saved state and the drafts. */
export function keyText(values, changes) {
if (changes?.clear_civitai_api_key === true) return KEY_COPY.clearing;
if (typeof changes?.civitai_api_key === 'string' && changes.civitai_api_key.trim()) return KEY_COPY.replacing;
if (values.lora_has_key === undefined) return null;
return values.lora_has_key ? KEY_COPY.saved : KEY_COPY.none;
}
/** D107: the import review's rows from the server's plan. */
export function importGrid(plan) {
const downloads = Number.isSafeInteger(plan.planned_downloads) ? plan.planned_downloads : 0;
const network = downloads
? `Downloads up to ${number(downloads)} example ${noun(downloads, 'image')} from CivitAI${plan.network?.api_key ? ' with your saved key' : ''}`
: 'No downloads';
const found = Number.isSafeInteger(plan.lora_files) ? plan.lora_files : 0;
const readable = Number.isSafeInteger(plan.with_metadata) ? plan.with_metadata : 0;
return [
['Scope', plan.scope?.label || `${number(readable)} ${noun(readable, 'LoRA')}`],
['Found', `${number(found)} metadata ${noun(found, 'file')} · ${number(readable)} readable`],
['Protected', protectedText(plan.protected)],
['Network', network],
['Backup', backupText(plan.backup)],
['Media on disk', plan.media?.label || 'No files are removed'],
];
}
function namesList(plan) {
const names = (Array.isArray(plan.details) ? plan.details : []).map(row => row?.label).filter(label => typeof label === 'string');
if (!names.length) return [];
const list = el('details');
const count = Number.isSafeInteger(plan.with_metadata) ? plan.with_metadata : names.length;
list.append(el('summary', names.length < count ? `LoRAs found (${number(names.length)} of ${number(count)})` : `LoRAs found (${number(names.length)})`));
const body = el('pre', names.join('\n'));
body.tabIndex = 0;
list.append(body);
return [list];
}
function closeOnly({ scope, trigger, plan, note }) {
const grid = el('dl', undefined, 'meta-grid');
for (const [term, value] of importGrid(plan)) {
const entry = el('div');
entry.append(el('dt', term), el('dd', value));
grid.append(entry);
}
const close = createButton('Close');
close.addEventListener('click', () => closeWorkspaceDialog());
const actions = el('div', undefined, 'dialog-actions');
actions.append(close);
openContentDialog({ scope, trigger, title: IMPORT_TITLE, nodes: [el('h3', 'LoRA reference data'), grid, ...namesList(plan),
el('p', note, 'small'), actions] });
}
/** Show the reviewed import; Start applies exactly this token (D107). */
export function showImportPlan(plan, { api, store, scope, trigger = null, reviewAgain, notify = showWorkspaceToast,
navigate = navigateTo, backup = () => downloadDatabaseBackup() }) {
if (!plan.token) {
closeOnly({ scope, trigger, plan, note: plan.blocked_by_pairs ? BLOCKED_NOTE : NOTHING_TO_IMPORT });
return;
}
if (!plan.with_metadata && !plan.current_imports) {
closeOnly({ scope, trigger, plan, note: NOTHING_TO_IMPORT });
return;
}
const removesAll = !plan.with_metadata;
const sentence = [planNote(plan), removesAll ? 'No LoRA metadata was found: this removes every imported LoRA prompt.' : '', CACHE_NOTE]
.filter(Boolean).join(' ');
openReviewDialog({ scope, trigger, title: IMPORT_TITLE, heading: 'LoRA reference data', grid: importGrid(plan),
options: namesList(plan), sentence, backup: { copy: BACKUP_COPY, onBackup: backup }, cancelLabel: 'Cancel',
primary: { label: IMPORT_START, danger: removesAll },
onStart: () => applyPlan({ api, store, scope, token: plan.token, reviewAgain, notify, navigate, url: IMPORT_APPLY_URL }) });
}
/** Review the import (read-only) and show its plan; the pressed button reads "Reviewing…" until it answered. */
export async function reviewImport({ api, store, scope, trigger = null, notify = showWorkspaceToast, navigate = navigateTo }) {
if (trigger?.getAttribute('aria-disabled') === 'true') return;
const again = () => reviewImport({ api, store, scope, trigger, notify, navigate });
const label = trigger?.textContent;
trigger?.setAttribute('aria-disabled', 'true');
if (trigger) trigger.textContent = 'Reviewing…';
try {
let plan;
try {
plan = await api.post(IMPORT_PREVIEW_URL, {}, { signal: scope.signal });
} catch (error) {
if (!scope.signal.aborted) notify(reviewRefusal(error));
return;
}
if (!scope.signal.aborted) showImportPlan(plan, { api, store, scope, trigger, notify, navigate, reviewAgain: again });
} finally {
trigger?.removeAttribute('aria-disabled');
if (trigger) trigger.textContent = label;
}
}
/** The row's control (`data-setting`), wherever the Toolkit switch nests it. */
const controlOf = (row, id) => row.querySelector(`[data-setting="${id}"]`);
/** A `p.small` status line under a row's description, created on first use. */
function noteOf(row) {
let note = row.children[0].querySelector(':scope > p.small');
if (!note) {
note = el('p', undefined, 'small');
row.children[0].append(note);
}
return note;
}
function setNote(row, text) {
if (text) noteOf(row).textContent = text;
else row.children[0].querySelector(':scope > p.small')?.remove();
}
/** A select row whose options after the first are disabled (D106). */
function onlyFirst(spec) {
const row = settingRow({ ...spec, control: { type: 'select', options: spec.options } });
for (const option of controlOf(row, spec.id).options) option.disabled = option.index > 0;
return row;
}
/** The "ComfyUI" surface (D106) and its renderer detection over the send bridge. */
function comfySurface({ scope, clientFactory }) {
const surface = el('section', undefined, 'surface');
surface.append(el('h2', 'ComfyUI'), el('p', 'One send contract across cards, prompt history and every metadata view.'));
const send = onlyFirst({ id: 'send-destination', label: 'Default send destination',
description: 'A destination picker previews both roles before applying.',
options: ['Ask each time', 'Selected Prompt Manager node', 'Create a Prompt Manager (+/-) node'] });
const renderer = onlyFirst({ id: 'renderer', label: 'Host compatibility',
description: 'Use the shared compatibility layer for Classic and Nodes 2.0.',
options: ['Follow host renderer', 'Classic', 'Nodes 2.0'] });
const returnRow = settingRow({ id: 'return-canvas', label: 'Return to canvas after send',
description: 'Apply text without queuing generation.', control: { type: 'toggle', checked: false } });
controlOf(returnRow, 'return-canvas').disabled = true;
setNote(send, REASONS.send);
setNote(returnRow, REASONS.returnCanvas);
const status = el('p', RENDERER_COPY.looking, 'small');
status.setAttribute('role', 'status');
status.id = 'renderer-status';
renderer.children[0].append(el('p', REASONS.renderer, 'small'), status);
surface.append(send, renderer, returnRow);
let client = null;
// Overlapping checks (two quick visibility changes) paint only the newest answer (S10 review N2).
let latest = 0;
const detect = async () => {
const check = ++latest;
let text;
try {
client ??= clientFactory();
text = rendererText(await client.discover({ waitMs: 300 }));
} catch {
text = RENDERER_COPY.unavailable;
}
if (check === latest && !scope.signal.aborted) status.textContent = text;
};
scope.cleanup(() => client?.dispose());
// Another tab may open or switch ComfyUI meanwhile: look again when this page is shown again.
scope.listen(document, 'visibilitychange', () => { if (document.visibilityState === 'visible') detect(); });
detect();
return surface;
}
function bridgeFactory() {
const url = new URL(window.ApiClient.resolveUrl('/prompt_manager/'), window.location.href);
return () => createBridgeClient({ namespace: url.pathname.slice(0, -'prompt_manager/'.length), origin: window.location.origin });
}
/**
* Mount Integrations in `container` (System's `.settings-layout`); returns the section's two surfaces. `drafts` is the
* System draft store (its host domain carries the LoRA values, system.mjs), `activity` the page's Activity store.
*/
export function mountIntegrations(container, { api, activity, scope, drafts, clientFactory = bridgeFactory(),
notify = showWorkspaceToast, navigate = navigateTo }) {
const comfy = comfySurface({ scope, clientFactory });
const surface = el('section', undefined, 'surface');
surface.append(el('h2', 'LoRA & resources'), el('p', 'Preserves LoraManager detection, data import and positive-only trigger injection.'));
const enable = settingRow({ id: 'lora-enable', label: 'LoraManager integration', description: 'Checking for LoraManager…',
control: { type: 'toggle', checked: false }, scope, onChange: value => drafts.set('host', 'lora_enabled', value) });
const pathRow = settingRow({ id: 'lora-path', label: 'LoraManager path',
description: 'Found automatically in ComfyUI’s custom_nodes folders; read-only, as on the existing settings page.',
control: { type: 'text' } });
const pathInput = controlOf(pathRow, 'lora-path');
pathInput.readOnly = true;
pathInput.placeholder = 'Not detected';
// Wide enough for the usual install path within the concept's 230px control limit.
pathInput.size = 34;
const triggers = settingRow({ id: 'lora-triggers', label: 'Automatically include trigger words',
description: 'Trigger words apply only to positive text; connected node inputs remain authoritative.',
control: { type: 'toggle', checked: false }, scope, onChange: value => drafts.set('host', 'lora_trigger_words_enabled', value) });
const key = settingRow({ id: 'civitai-key', label: 'CivitAI API key', description: 'Credentials stay in the server configuration.',
control: { type: 'password' } });
const keyInput = controlOf(key, 'civitai-key');
// Never a login the browser offers to save or fill (S10 review N3).
keyInput.autocomplete = 'new-password';
keyInput.spellcheck = false;
keyInput.placeholder = 'Optional';
// D114: the saved-key line is announced (a polite live region) and describes the field (S10 review M2).
const keyNote = el('p', '', 'small');
keyNote.id = 'civitai-key-status';
keyNote.setAttribute('role', 'status');
key.children[0].append(keyNote);
keyInput.setAttribute('aria-describedby', keyNote.id);
const clear = createButton('Clear saved key');
clear.dataset.setting = 'clear_civitai_api_key';
clear.dataset.key = 'clear_civitai_api_key';
const keyActions = el('div', undefined, 'row-actions');
keyActions.append(clear);
key.children[0].append(keyActions);
scope.listen(keyInput, 'input', () => {
// D114: a blank field keeps the saved key: it is no draft at all.
drafts.set('host', 'civitai_api_key', keyInput.value.trim() ? keyInput.value : undefined);
if (keyInput.value.trim()) drafts.set('host', 'clear_civitai_api_key', false);
});
scope.listen(clear, 'click', () => {
keyInput.value = '';
// The button leaves with the saved key: keep the keyboard on the field it belongs to (S10 review M2).
keyInput.focus();
drafts.set('host', 'civitai_api_key', undefined);
drafts.set('host', 'clear_civitai_api_key', true);
});
const importRow = operationRow({ scope, title: 'LoRA reference data', description: 'Review and import resource metadata.',
label: 'Review import', onClick: trigger => reviewImport({ api, store: activity, scope, trigger, notify, navigate }) });
surface.append(enable, pathRow, triggers, key, importRow);
const controls = [[enable, 'lora-enable', 'lora_enabled'], [triggers, 'lora-triggers', 'lora_trigger_words_enabled']]
.map(([row, id, name]) => ({ row, name, input: controlOf(row, id) }));
for (const control of controls) control.input.dataset.key = control.name;
keyInput.dataset.key = 'civitai_api_key';
// The saved LoRA state as last painted: a save (or another tab's change read back) that changes it reads
// `/lora/status` again, so detection, the key line and the trigger cache follow what the server now holds.
let shown;
const repaint = () => {
const values = drafts.saved('host');
const changes = drafts.changes('host');
const unread = values.lora_enabled === undefined;
const failed = drafts.hostState?.() === 'failed' || (drafts.hostState?.() === 'loaded' && unread);
for (const { input, name } of controls) {
const value = drafts.get('host', name);
input.disabled = value === undefined;
if (value !== undefined) input.checked = value === true;
}
enable.children[0].children[1].textContent = failed && unread ? HOST_UNAVAILABLE
: detectionText(values) ?? 'Checking for LoraManager…';
pathInput.value = values.lora_path || values.lora_detected_path || '';
setNote(triggers, cacheText(values));
keyInput.disabled = unread;
// A live region: rewritten only when it says something new, so it is not re-announced on every edit.
const note = keyText(values, changes) ?? '';
if (keyNote.textContent !== note) keyNote.textContent = note;
// D114: "Clear saved key" is offered only while a saved key would stay.
if (values.lora_has_key === true && changes.clear_civitai_api_key !== true) key.children[0].append(keyActions);
else {
if (keyActions.contains(document.activeElement)) keyInput.focus();
keyActions.remove();
}
const state = JSON.stringify([values.lora_enabled, values.lora_trigger_words_enabled, values.civitai_api_key,
values.clear_civitai_api_key]);
const saved = values.civitai_api_key !== undefined || values.clear_civitai_api_key === true;
if (!unread && shown !== undefined && (state !== shown || saved)) {
if (saved) keyInput.value = '';
shown = undefined;
drafts.reloadHost();
return;
}
if (!unread) shown = state;
};
scope.cleanup(drafts.subscribe(repaint));
// A typed key is held only while its field shows it: leaving the section drops it, since the field comes back
// empty and a hidden key draft could not be seen or discarded (S10 review N4). Only a typed key: any set clears
// the last Save's outcome, which Sources still needs (S10 re-review R1).
scope.cleanup(() => {
if (drafts.changes('host').civitai_api_key !== undefined) drafts.set('host', 'civitai_api_key', undefined);
});
repaint();
return [comfy, surface];
}
+17 -1
View File
@@ -19,6 +19,8 @@
* 'settings-changed'` when another page changed a drafted value (a conflict: the browser values are read again),
* `writeHost(changes, {expected_revision})` the settings response (`{success, persisted, revision}`, and the
* `restart_required_fields` the status names), `writeRuntime(changes)` anything; a write that throws failed. A host
* write that saved part of its changes throws with `partial: {saved: [keys], revision}`: those keys are saved, the
* revision is adopted and only the rest fail (`partial`, S10). A host
* write refused as `invalid-source` (its error's `source` and server message) makes that message the status and
* `invalidSource()` that source, so Sources marks its row (S4).
*/
@@ -32,6 +34,8 @@ export const SETTINGS_STATUS = Object.freeze({
host: 'Saved to the server configuration.',
runtime: 'Applied until the server restarts.',
partial: 'Some changes were not saved. Review the highlighted settings.',
// S10 review I3: a host save written in two parts where only one was saved.
partialSaved: 'Some changes were saved to the server configuration; the highlighted ones were not.',
tabOnly: 'Saved for this tab only; browser storage is unavailable.',
conflict: 'Settings changed elsewhere. Reload this section to see them.',
// D128: a saved setting the running server applies only after a restart.
@@ -55,6 +59,7 @@ export function statusText(outcome) {
const results = DRAFT_DOMAINS.map(domain => outcome?.[domain]);
if (results.includes('conflict')) return SETTINGS_STATUS.conflict;
if (results.includes('failed')) return outcome?.invalid?.message || SETTINGS_STATUS.partial;
if (results.includes('partial')) return outcome?.invalid?.message || SETTINGS_STATUS.partialSaved;
const saved = [];
if (outcome?.browser === 'saved') saved.push(SETTINGS_STATUS.browser);
if (outcome?.browser === 'tab-only') saved.push(SETTINGS_STATUS.tabOnly);
@@ -177,10 +182,21 @@ export function createSystemDrafts(adapters) {
commit(domain, written);
return 'applied';
} catch (error) {
fail(domain, written);
if (domain === 'host' && error?.code === INVALID_SOURCE_CODE && typeof error.message === 'string') {
hostDetail.invalid = { source: typeof error.source === 'string' ? error.source : null, message: error.message };
}
const partial = domain === 'host' ? error?.partial : null;
if (partial && Array.isArray(partial.saved)) {
// S10 review I3: part of the host save was written (config.json moved on): keep what saved, adopt the
// revision it produced, and fail only the rest, so the next save is not a false conflict.
if (typeof partial.revision === 'string') revision = partial.revision;
const saved = new Map([...written].filter(([key]) => partial.saved.includes(key)));
commit(domain, saved);
fail(domain, new Map([...written].filter(([key]) => !saved.has(key))));
if (error?.code === CONFLICT_CODE) return 'conflict';
return saved.size ? 'partial' : 'failed';
}
fail(domain, written);
return error?.code === CONFLICT_CODE ? 'conflict' : 'failed';
}
}
+16 -7
View File
@@ -5,7 +5,8 @@
* (`span.small#settings-status` + "Save preferences"). D92: no "Sample configuration" chip, the status copy is
* truthful and "Save preferences" is aria-disabled while nothing is unsaved. A section not delivered yet shows the
* per-section hand-off to its legacy tool (D94), never mock placeholders. Overview (system-overview.mjs),
* Appearance (system-appearance.mjs), Sources & scans (system-sources.mjs) and Maintenance (system-maintenance.mjs) are native.
* Appearance (system-appearance.mjs), Sources & scans (system-sources.mjs), Integrations (system-integrations.mjs) and
* Maintenance (system-maintenance.mjs) are native.
*
* Switching sections keeps the shell (and focus on the pressed button) and disposes the old section's scope, so its
* listeners and timers never outlive it. The drafts are module-level (Q10): they live for the page session.
@@ -18,6 +19,7 @@ import { mountOverview } from './system-overview.mjs';
import { mountAppearance } from './system-appearance.mjs';
import { mountSources } from './system-sources.mjs';
import { mountMaintenance } from './system-maintenance.mjs';
import { LORA_ENABLE_URL, LORA_STATUS_URL, loraValues, mountIntegrations, writeHostInParts } from './system-integrations.mjs';
import { busyToast } from './activity-scan.mjs';
import { showWorkspaceToast } from './toast.mjs';
import { displayPreferences, readPreferences, reconcilePreferences, subscribePreferences, writePreferences } from './preferences.mjs';
@@ -30,7 +32,6 @@ const ADMIN = Object.freeze({ href: '/prompt_manager/admin', label: 'Open existi
* settings, thumbnails, duplicates and cache tools.
*/
export const SECTION_HANDOFFS = Object.freeze({
integrations: [ADMIN],
tagging: [ADMIN],
backups: [ADMIN],
diagnostics: [ADMIN],
@@ -77,14 +78,21 @@ export function systemDrafts(api) {
},
writeRuntime: undelivered,
async readHost() {
const result = await api.get(SETTINGS_URL, { cache: 'no-store' });
// S10: the LoRA state rides along; when it cannot be read its values stay missing (Integrations says so)
// and every other host setting still loads.
const [result, lora] = await Promise.all([api.get(SETTINGS_URL, { cache: 'no-store' }),
api.get(LORA_STATUS_URL, { cache: 'no-store' }).catch(() => null)]);
// The store reads a missing value as none and a missing revision as unreadable (no host save). #317: which
// thumbnail formats the server can write rides along as a read-only value (never drafted or sent).
return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders }, revision: result.revision };
return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders, ...(lora ? loraValues(lora) : {}) },
revision: result.revision };
},
async writeHost(changes, { expected_revision }) {
// S10: the LoRA keys save through /lora/enable, the rest through /settings, each with the current revision;
// a save where only one part was written reports which (writeHostInParts, review I3).
try {
return await api.post(SETTINGS_URL, { ...changes, expected_revision });
return await writeHostInParts({ changes, saved: drafts.saved('host'), expected_revision,
postLora: body => api.post(LORA_ENABLE_URL, body), postSettings: body => api.post(SETTINGS_URL, body) });
} catch (error) {
// A sources change waits for a running record-changing job (S4): name it, as a refused start does (D57).
if (error?.code === 'busy') showWorkspaceToast(busyToast(await runningJob(error)));
@@ -119,13 +127,14 @@ function markFailed(layout, store) {
}
/**
* The sections delivered so far: Overview (S2 Task 6), Appearance (S3 Task 8), Sources & scans (S4 Task 11) and
* Maintenance (S7 Task 19).
* The sections delivered so far: Overview (S2 Task 6), Appearance (S3 Task 8), Sources & scans (S4 Task 11),
* Integrations (S10 Task 27) and Maintenance (S7 Task 19).
*/
export const DELIVERED_SECTIONS = Object.freeze({
overview: mountOverview,
appearance: mountAppearance,
sources: mountSources,
integrations: mountIntegrations,
maintenance: mountMaintenance,
});
+6
View File
@@ -423,3 +423,9 @@ pre.prompt-text { font: inherit; font-size: 12px; line-height: 1.75; margin-bloc
/* D101: a duplicate copy's Keep label is its source and path, often long names with no space to break at; it wraps
inside its `.compare-item` so the review never scrolls sideways. */
.duplicate-groups .compare-item label { overflow-wrap: anywhere; /* D101 */ }
/* D106: Integrations' two selects keep their chosen option readable: they never shrink for the row's reason lines, and
at phone width they may grow past the concept's 120px cap (which cuts "Follow host renderer") up to 160px. */
.setting-row :is(#send-destination, #renderer) { flex-shrink: 0; /* D106 */ }
@media (max-width: 700px) {
#workspace-main .setting-row :is(#send-destination, #renderer) { max-width: 160px !important; /* D106 */ }
}